Weighted results FAQ

Using the MX8 Labs Research Platform, all your results in cross-tabs or insights are automatically weighted to make up for sample skews.

If you're looking for the mathematical detail behind any of the answers below, see Weighting methodology.

How do I set up weighting?

Weighting is set up when you configure quotas and weighting targets for your respondent source.

For example, if you field with the US Genpop standard screener, you can define target demographics in the quota tab.

You can see the weights setup in the summary tab:

And edit them by clicking the edit button:

How should I structure long or complex weighting schemes?

For long or complex weighting schemes, separate individual sections using H1 headings (# in markdown) in your weighting definition. The platform interprets the markdown into JSON weight targets, and using H1 sections to break the document into discrete blocks helps it parse each section reliably — for example, one H1 per demographic dimension (age, gender, region) or one H1 per weighting stage. This is especially useful when a single scheme combines several joint targets or covers multiple population frames.

How do you run the weighting?

Weighting runs automatically when the survey closes. It can also run while fieldwork is still open once the minimum base is met, so you can read directional weighted results before close.

MX8 uses an IPFN (Iterative Proportional Fitting) algorithm that iteratively adjusts respondent weights until configured targets are matched within tolerance.

Do you support two-stage weighting?

Yes. A dataset can run up to two stages:

1. Pre-weighting stage
2. Main weighting stage

When both are configured, pre-weighting runs first and writes intermediate respondent weights. Main weighting then runs on top of those weights. Final respondent weight is multiplicative:

$$\text{final weight} = \text{pre-weight multiplier} \times \text{main-weight multiplier}$$

This is useful when you need to correct more than one thing in sequence, for example:

• Stage 1: align a first-party list or panel composition.
• Stage 2: align reporting outputs to market-level targets.

Can I choose which respondents are used for weighting?

Yes. You can choose the respondent universe used for weighting, and this can differ by stage when two-stage weighting is configured.

Typical choices include:

• Completes only.
• Completes plus selected terminated respondents.
• A constrained subgroup used for frame correction in pre-weighting.

Choose the universe based on the inference you want to support:

• Use a broader universe when you need representativeness with respect to all eligible entrants.
• Use completes-only when you need representativeness for the final analyzable sample.
• Use different universes across stages when one stage is about correcting source composition and another is about final reporting inference.

Why would I use different respondent sets across stages?

Because different stages often answer different methodological questions:

• Pre-weighting can normalize source or recruitment bias before analysis.
• Main weighting can calibrate the analysis population to reporting targets.

Using separate respondent sets lets you avoid forcing one compromise universe to do both jobs badly.

What happens if the platform can't weight my data?

Before weighting runs, the platform checks that calibration is safe to attempt. It confirms that every question referenced by targets is present, that there are enough respondents complete across weighting questions, and that every category with a positive target has at least one eligible respondent in the selected respondent set.

If any stage fails, weighting is skipped entirely (all-or-nothing) and the dataset is reported with unit weights. Diagnostics show which check failed so you can fix the root issue (usually by collapsing sparse categories, revising targets, or adding sample).

What's the distribution of my weights?

You can inspect weight distribution by downloading raw data in Excel or SPSS format and charting a histogram.

How do I weight boosts?

Boost respondent sources are typically used to increase sample for a subgroup without creating a separate weighted inference frame. Boost samples inherit the survey's weighting configuration.

How do I weight nested quotas?

Weighting runs automatically for nested quotas in your configured respondent source.

Can I turn off weighting?

If you set the weighting strategy to None, results are unweighted. Caveat emptor: without proper weighting, your results will not reflect the actual population, and conclusions drawn from them can be misleading. Only do this when unweighted inference is what you actually want — for example, when you're describing the realized sample itself rather than generalising to a population.

What is effective sample size?

When respondents carry unequal weights, the "information content" of your sample is smaller than the raw respondent count. We report Kish effective sample size for each reporting cell:

$$n_{\text{eff}} = \frac{\left(\sum w_i\right)^2}{\sum w_i^2}$$

If all weights are equal, $n_{\text{eff}}$ equals the raw count. As weights become more unequal, $n_{\text{eff}}$ falls below it. We use $n_{\text{eff}}$ — not raw counts — when computing standard errors and running significance tests, so that the precision we report reflects the true information content of the weighted data rather than the inflated weighted totals.

What is weighting efficiency?

Weighting efficiency is the dataset-level summary of precision cost:

$$\text{Efficiency} = \frac{n_{\text{eff}}}{n}$$

An efficiency near 1 means weighting has barely moved the needle on precision. Lower efficiency, or a long right tail on the weight histogram, means a small number of respondents are carrying disproportionate weight. That's not wrong — it's the correct response to an imbalanced sample — but it's a signal to check whether the targets are achievable from the realized sample, or whether some categories should be collapsed.

Why don't percentages in my results exactly match my target scheme?

Weighting targets define calibration constraints, but reported bases reflect your selected reporting universe and the realized sample. If incidence and termination patterns differ across groups, reported percentages can differ from intake shares while still being correctly calibrated for the inference frame you configured.

A worked example makes this concrete. Suppose you're running a survey of ESPN subscribers against a US genpop respondent source, and 1000 people enter the experience — 500 men and 500 women. ESPN skews male, so suppose 80% of the women are screened out for not being subscribers. You end up with 700 respondents who completed: 500 men and 200 women.

When the platform weights, it includes the 300 terminated women too — because the calibration is to the entrant population, not just the completes. With that universe, men and women end up evenly weighted, and the report tells you 500 / 700 ≈ 71.4% of ESPN subscribers are women.

That isn't consistent with the 50/50 source you specified upfront, but it is nationally representative — which is the thing that matters for the inference. The mismatch between source proportion and report proportion is the calibration working as intended.