Utility and simulated share methodology

This page is the technical companion to the discrete-choice reporting outputs in MX8 Labs. It documents how utility scores are estimated, how simulated shares are computed, and how uncertainty is summarized in reports.

For user-facing setup of the question types this estimator powers, see Choice-Based Conjoint and Running MaxDiff. The weighted aggregation described below shares its effective-sample-size machinery with the Weighting methodology used elsewhere in reporting.

Scope

MX8 Labs computes respondent-level utilities for MaxDiff and choice-based conjoint (CBC) questions using a Hierarchical Bayes (HB) multinomial logit model. Report outputs are then derived from posterior draws:

• Utility scores (part-worth/item utility)
• Scaled scores
• Ranks
• Simulated share of preference

Point estimates and uncertainty are produced by combining:

• posterior variation across utility draws, and
• weighted sampling variation using effective sample size.

Inputs and notation

Let:

• respondents be indexed by $i = 1, \dots, N$,
• tasks by $t = 1, \dots, T_i$ per respondent,
• alternatives in a task by $a$,
• utility components (MaxDiff items or conjoint attribute-levels) by $k$.

For each respondent $i$, the model estimates a vector of part utilities:

$$\boldsymbol{\beta}_i = (\beta_{i1}, \dots, \beta_{iK})$$

Each alternative utility is a linear sum of included components.

Design parsing

The estimator infers question design from reporting rows. A configured role column is used when its values have valid best/worst semantics. That valid role column takes precedence over other topic columns and identifies the question as MaxDiff. If the configured column is absent or invalid, the estimator searches the topic columns for one whose values or bounds identify both best and worst roles. Questions without a valid role column are treated as conjoint/CBC when component topic columns exist; otherwise they are treated as MaxDiff with responses as item components.

For MaxDiff, if both best and worst selections are present, the best task is modeled with positive sign and the worst task is modeled on the remaining alternatives with negative sign.

Hierarchical Bayes estimator

Utilities are estimated with a respondent-level random-coefficients logit model:

$$\boldsymbol{\beta}_i = \boldsymbol{\mu} + \mathbf{z}_i \odot \boldsymbol{\sigma}$$

with priors:

• $\mu_k \sim \mathcal{N}(0,1)$
• $\sigma_k \sim \text{HalfNormal}(1)$
• $z_{ik} \sim \mathcal{N}(0,1)$

For each task, utility for alternative $a$ is:

$$U_{ita} = s_t \sum_{k=1}^{K} x_{itak}\beta_{ik}$$

where:

• $x_{itak} \in \{0,1\}$ indicates whether component $k$ appears in alternative $a$,
• $s_t \in \{+1,-1\}$ is the task sign (used for MaxDiff worst handling).

Choice probability is softmax:

$$P(y_{it}=a) = \frac{\exp(U_{ita})}{\sum_{a'} \exp(U_{ita'})}$$

Posterior sampling uses JAX/NumPyro-backed No-U-Turn Sampling (NUTS) with four parallel chains, 1,000 warm-up iterations, and 1,000 retained draws per chain. A deterministic subset of 200 posterior draws is cached for downstream reporting.

Cached utility outputs

For each respondent-component pair, MX8 Labs persists:

• Posterior mean utility
• Posterior draw utilities (draw_id indexed)

These cached draws drive all downstream utility/share reporting.

Caches are isolated by estimator version. Estimator v3 writes regenerated utility outputs and terminal-failure markers under its own cache version, so results produced by retired estimator versions are not reused.

Utility score reporting

For utility mode, each row value is the cached posterior draw utility directly:

$$v_{idk}^{\text{utility}} = \beta_{idk}$$

where $d$ indexes retained posterior draws.

Conjoint rows are labeled as attribute: level; MaxDiff rows use item names.

Simulated share reporting

Simulated share is computed within each respondent and draw by exponentiating utilities and normalizing:

$$s_{idr} = \frac{\exp(v_{idr})}{\sum_{r'} \exp(v_{idr'})}$$

where $r$ indexes reported options (items or attribute-level rows in derived mode, or configured scenarios in explicit simulation mode).

Two simulation paths are supported:

1. Derived share: share over the natural reported rows for the question.
2. Scenario simulation: share over user-defined scenarios.

For scenario simulation:

• MaxDiff scenario utility is the utility of the specified item.
• Conjoint scenario utility is the sum of the selected attribute-level utilities in the profile.

Shares are then softmax-normalized across scenarios for that respondent and draw.

Weighted aggregation and uncertainty

For any reported row and tab cell, MX8 Labs aggregates draw-level values using respondent reporting weights.

Within each draw $d$:

$$\hat{m}_d = \frac{\sum_i w_i v_{id}}{\sum_i w_i}$$

The final estimate is the mean across draws:

$$\hat{m} = \frac{1}{D}\sum_{d=1}^{D}\hat{m}_d$$

Total variance is decomposed as:

$$\widehat{\mathrm{Var}}(\hat{m}) = \underbrace{\mathrm{Var}_d(\hat{m}_d)}_{\text{posterior variance}} + \underbrace{\frac{1}{D}\sum_{d=1}^{D}\widehat{\mathrm{Var}}_{\text{sampling},d}}_{\text{sampling variance}}$$

Sampling variance per draw uses weighted variance with Kish effective sample size:

$$n_{\mathrm{eff}} = \frac{(\sum_i w_i)^2}{\sum_i w_i^2}, \quad \widehat{\mathrm{Var}}_{\text{sampling},d} = \frac{\sum_i w_i(v_{id}-\hat{m}_d)^2 / \sum_i w_i}{n_{\mathrm{eff}}}$$

Reported standard error is:

$$\widehat{\mathrm{SE}} = \sqrt{\widehat{\mathrm{Var}}(\hat{m})}$$

Related output modes

The same posterior draws also support:

• Scaled scores: per respondent/draw min-max scaling to 0-100.
• Ranks: dense descending rank per respondent/draw.

These are transformations of cached utility draws before the same weighted draw-aggregation pipeline.

Guardrails and failure modes

1. Utility estimation requires valid choice-task observations with timestamps and at least one selected alternative pattern.
2. Sampling diagnostics include divergences, maximum R-hat, and minimum bulk effective sample size. Divergences, missing diagnostics, R-hat above 1.01, or bulk effective sample size below 400 are logged as convergence warnings for operational review.
3. Convergence warnings do not reject otherwise finite utility estimates: finite outputs are cached and remain available for reporting and export.
4. If utility sidecars are missing, report requests return not-ready status until cache generation completes.
5. If no valid observations exist, outputs are non-finite, or estimation otherwise fails, utility generation fails terminally for that question. Reporting returns a user-facing validation or estimation error, and an explicit dataset-cache refresh is required to retry a terminal execution failure.

Assumptions and limitations

1. Current estimator is JAX/NumPyro-backed HB multinomial logit (hierarchical_bayes_discrete_choice_v3).
2. The implementation uses additive utility within alternatives and standard softmax choice probabilities.
3. Report uncertainty is an approximation that combines posterior and effective-sample-size weighted sampling variance; it is not a full complex-survey design variance estimator.
4. Simulation outputs are preference shares under the model, not market shares.

Reproducibility

For fixed data, estimator settings, and seed, utility estimation and retained draw selection are deterministic, and downstream share calculations are deterministic transformations of those cached draws.

Exporting respondent-level outputs

The cached posterior outputs described above can be exported for offline analysis:

• Utility Scores Export Format — respondent-level utilities in raw, zero-centered, and share-scaled forms.
• Raw Draws Export Format — unaggregated posterior draws per respondent and parameter, for custom uncertainty work and convergence diagnostics.