A lift report compares respondents who were subject to something against a matched group who were not, then reports the difference in outcomes. This page explains how the treated group and matched control are constructed, how the difference is calculated, and which claims each design supports.
The measurement is general. Measuring brand lift from advertising is its most common application, but nothing in the method is specific to advertising: the platform needs a per-respondent indicator saying who was treated, one or more outcome questions, and a set of variables to match on. Anything expressible that way can be measured this way.
What can be measured
| Application | How the treated group is identified | Typical outcomes |
|---|---|---|
| Advertising campaign | Ad-server exposure matched to respondents | Awareness, recall, consideration, favorability, intent |
| Creative or concept test | Random assignment inside the survey | Appeal, message take-out, purchase intent |
| Message and claim testing | Random assignment inside the survey | Agreement, persuasion, believability |
| Sponsorship or event | Attendance or exposure recorded in the survey | Brand association, sentiment |
| Packaging or in-store stimulus | Random assignment, or observed exposure | Findability, appeal, switching intent |
| Direct mail, email, or CRM campaign | Matched against a first-party contact list | Recall, action taken, sentiment |
| Product trial or usage | Declared usage or a customer-file match | Satisfaction, advocacy, repeat intent |
Each outcome can be reported overall or cut by one question or dataset tag, such as brand, creative or message. Different outcomes can use different sources, which remain separate in the output. One study covering fifty brands can produce fifty separate brand estimates; estimates based on overlapping respondents are not statistically independent. See Reaching low-incidence audiences.
How strong a claim the result supports
This is the first question a reviewer should ask, and the answer depends entirely on how the treated group came to be treated. Matching on demographics equalizes the variables you matched on. It does not turn an observational comparison into an experiment.
| How the treated group was formed | Design | What the result supports |
|---|---|---|
| Random assignment inside the survey | Experimental | A causal effect of the stimulus on the outcome, within the sample |
| Server-verified media exposure, matched | Quasi-experimental | An adjusted association between exposure and the outcome. A causal interpretation requires additional assumptions, including no unmeasured confounding; verified exposure alone does not establish causality |
| Declared or recalled exposure | Observational | Association. Recall is itself correlated with brand familiarity, which is frequently the outcome being measured |
| Behavioral or status split, such as customers against non-customers | Observational | Association only. Matching on demographics does not remove selection into the group |
State the design alongside any reported result. A quasi-experimental comparison supports a different claim from a randomized experiment and should not be described as randomized.
Establishing the treated group
Whatever the application, the mechanism is the same: an indicator is recorded against the respondent and carried onto their answers, and the lift report reads it back.
The group indicator can be a survey question or dataset tag. An s.tag() block carries a tag onto every question inside the block. Select the indicator and its target values in the Lift model step; other answered values form the comparison group. Configuration is documented in Reporting lift against control groups.
Random assignment in-survey
Respondents are randomly allocated to see or not see the stimulus, and the allocation is tagged. Because assignment is random, the comparison estimates a causal effect rather than an association. This design applies to creative, concept, message, and packaging tests whether or not media has run.
Matched media exposure
Where the treatment is real-world media, exposure records are sent to MX8 Labs from the ad server and matched against respondents.
Two ingest routes are supported. A pixel hosted by MX8 Labs is embedded in the ad server or tag manager and fires on render, capturing exposures in real time; this suits display and rich media. Server-to-server delivery uploads gzip-compressed CSV files of exposure records to an MX8-managed S3 bucket, which suits connected TV, audio, server-rendered placements, and any case where IP addresses should be hashed before leaving the client's environment.
Each record carries an IP address, plaintext or MD5-hashed, plus optional identifiers: a user identifier of up to 128 characters, and dimensions such as brand or creative. A retention window is set per source to match the campaign timeline.
IP address is the join key. A match is recorded only when four conditions all hold:
| Condition | What is checked |
|---|---|
| IP address parity | The same IP appears in both the exposure log and the response pool |
| Same study window | Both records fall inside the fielding window defined for the campaign |
| Respondent passes validation | The respondent cleared the full quality pipeline |
| Exposure is attributable | The ad server recorded a real impression event, not a bot or crawler fetch |
The third condition carries most of the weight. Matching against an uncleaned respondent pool produces a match rate rather than a measurement, because duplicates, bots, and spoofed sessions all carry IP addresses. MX8 Labs validates the respondent pool before the join runs. In the production data summarized in our January 2026 survey fraud analysis, that validation excluded 10 to 20 percent of incoming respondents; the rate varies by study and source. See IP Address Hygiene and Exposure Matching and Data quality methodology.
The unit of attribution is the household by default. IP addresses are shared behind household NAT, so the person who saw the ad and the person who answered the survey are not necessarily the same individual. This is a property of any IP-joined design, and for most consumer campaigns in-home exposure is the relevant signal anyway. Studies requiring person-level attribution are configured separately, with sample design and in-survey verification to support the stricter standard.
Exposure records with no matching respondent still count toward reach but contribute nothing to respondent-level analysis. Respondents with no matching exposure form the untreated pool.
Because survey-based lift asks people directly, it is independent of the pixel for the outcome measurement even when the pixel supplies the exposure signal. It stays usable on media where no click exists, and resistant to the identifier degradation eroding attribution-based measurement.
Frequency and recency
Matched exposures carry two further attributes: frequency, the number of exposures for that value, and days_since_exposure, the recency of the most recent one where available.
This allows treatment to be defined as something more precise than a binary, such as respondents seen three or more times in the last seven days, or the questionnaire to branch on exposure intensity, or light and heavy exposure to be compared directly. For wear-out and frequency-response questions this is the difference between a usable study and a null result. See What are exposure sources.
Declared exposure
Where no feed and no randomization exist, the indicator can be set from a recognition or recall question. This is the weakest route, for the reasons in the table above, and reporting should describe it as declared rather than verified.
Constructing the control group
The control group is drawn from untreated respondents and matched to the treated group on chosen variables. Start with characteristics that influence the outcome before exposure. For conversion, ask what would make someone more likely to convert even without the campaign: prior category purchasing, existing customer status, purchase frequency or pre-existing need. Include demographics where they help explain the outcome or confounding, rather than selecting age, gender and region automatically. Control variables are used only for matching; lift is not measured on them.
Choose the variables before inspecting lift, using knowledge of both the outcome and how people became exposed. Prioritize plausible confounders that affect both exposure and the outcome, alongside relevant baseline predictors of the outcome. This is guidance for choosing matching characteristics, not a claim that the report fits a conversion-propensity model.
Do not match on characteristics the campaign could have changed. Recall, consideration and purchase intent measured after exposure may be part of the campaign's effect. Matching on them can remove the effect being measured or introduce bias. Existing customer status and purchase history must also refer to the period before exposure. Asking a question earlier in the survey does not make it a pre-exposure measure; use baseline records or measures with a clear pre-exposure reference period, and acknowledge recall limitations.
Matching is repeated rather than performed once. A result based on one matched control can be sensitive to which eligible untreated respondents are selected.
For the configured number of iterations, the platform constructs a balanced control group from the available untreated pool and averages the results. Selection is deterministic: the same respondent data, configuration, and reporting implementation produce the same matched controls. Repeating the match reduces dependence on any one eligible control selection; it does not remove sampling variance or unmeasured confounding. New lift reports created in the interface use five iterations by default, while the reporting API accepts an explicit iteration count.
Check achieved and retained bases, overlap between the groups and balance on the selected characteristics after matching. Repeat these checks for the brands and segments carrying conclusions. If the available data or diagnostics cannot establish balance, report that limitation. More matching iterations do not repair a lack of comparable controls. Size the study for the usable matched base, allowing for matching losses and the smallest planned reporting cell; see Sample size and precision.
Design outcomes for X+ reporting
Use rating scales for attitudinal outcomes such as consideration, favorability and purchase intent. Consistent numeric coding lets the report aggregate the proportion scoring X or higher (X+), such as 4+ on a five-point scale or 7+ on a 0 to 10 scale. Keep factual purchases, unaided awareness and brand selection in formats appropriate to those measures rather than forcing every question into a scale.
Use identical wording, anchors and coding in treated and control groups, with higher ratings consistently more positive for X+ reporting. Define the denominator and handling of missing answers and Don't know responses before analysis. Do not encode Don't know as a high rating that would enter an X+ total.
Specify the primary outcome, headline X+ threshold, planned brands and segments, matching variables and final analysis date or sample size before inspecting results. Additional thresholds can describe the response pattern, but selecting whichever threshold gives the largest or most favorable lift makes the finding exploratory. Keep in-flight reads directional unless the analysis accounts for repeated testing.
Computing lift
Lift is computed for each outcome using its selected cut-by question or tag. With None, the outcome produces an overall result. With a cut-by source, it produces a result for each source value; the source identity is retained even when another outcome uses the same displayed value from a different source.
Outcomes are handled by type:
- Categorical outcomes - awareness, recall, brand selected, option chosen. Lift is calculated separately for each response value.
- Numeric and scale outcomes - consideration on a 0 to 10 scale, a 1 to 5 rating. With Each value or higher (≥), lift is calculated at cumulative thresholds, for example the proportion scoring 7 or above, 8 or above, 9 or above. Thresholds show changes in the share reaching a defined response level that a mean alone does not describe. Lead with the threshold specified in the analysis plan.
For each stimulus value the sequence is: split respondents into treated and untreated candidates, build a matched control on the control variables, calculate outcome rates for both groups, take the difference, test it, and average across matching iterations.
The difference in rates is absolute lift, expressed in percentage points. A target rate of 36% against a control rate of 30% is +6 percentage points. Relative lift divides that difference by the control rate: 6 ÷ 30 = +20%. Label relative lift separately; it is undefined when the control rate is zero.
Significance
For each matching iteration, the report applies Fisher's exact test to the target and control outcome counts. It then averages the p-values across iterations and keeps results whose Mean P-Value is less than or equal to the configured P-value threshold. This is an average of the iteration tests, not a pooled significance test or an adjustment for multiple comparisons.
A threshold of 0.95 provides a broad exploratory view; it does not mean 95% confidence. Results visible under that setting may fail a conventional 0.05 cutoff. Read the mean p-value with the lift and group sizes, and state the threshold alongside reported findings. The Lift report guide shows a worked example using the result details.
The p-value threshold is configurable. A 0.05 setting is a stricter filter on the mean p-value than 0.10 or 0.95; it does not turn the average into a pooled test with a 5% false-positive rate. Selecting 0.05 alone does not make a result confirmatory or establish its suitability for an external claim. Confirmatory inference requires a method appropriate to the study design, matching procedure and planned comparisons. The matched Lift calculation uses respondent counts and Fisher tests; it does not apply the weighted cross-tab standard-error calculation to those tests. Details show mean group sizes, outcome rates, lift and mean p-value. See Understanding stat testing for the distinction from cross-tab comparisons.
Testing many thresholds, outcomes, brands or segments creates multiple comparisons; overlapping X+ thresholds are not independent confirmations. Disclose which comparisons were planned, which were exploratory and any multiplicity adjustment used. The report's mean p-value does not make that adjustment, and choosing a favorable row or a loosely defined pattern is not a substitute for it.
Negative lift is off by default; enable it as a reporting best practice. The p-value filter still applies, so visible rows alone do not cover every planned outcome. Include negative and inconclusive findings in the study readout as well as positive findings. A result that fails the filter is not evidence of zero effect: distinguish no clear evidence of lift from evidence of no effect, especially with small matched bases.
Reading a result
Illustrative rates for media exposure matched from an ad server, with brand as the cut-by source and pre-exposure category purchasing and customer status as matching variables:
| Brand | Metric | Treated | Control | Lift |
|---|---|---|---|---|
| Brand A | Awareness | 62% | 54% | +8 pts |
| Brand A | Consideration 7+ | 38% | 33% | +5 pts |
These rates describe higher awareness and consideration among exposed respondents in the matched population. They do not establish statistical significance on their own. A reported headline must also state the matched group sizes, outcome threshold, reporting window, study design and mean p-value filter, with the denominator rules documented. Because exposure was not randomly assigned, this is a quasi-experimental association after matching on the selected characteristics. It does not support a claim about the full sample, respondents outside the matched population or the mechanism.
Limitations
Matching consumes sample. Every treated respondent needs a comparable counterpart. A thin untreated pool, or a treated group with an unusual profile, reduces the usable base and widens intervals. Campaign reach governs the split you get, and because reach against a sampled audience is usually low, the exposed group is nearly always the scarce side while control is abundant. Sample size and precision gives the arithmetic, and Campaign size and measurability converts a media plan's impressions and frequency into the sample required.
More control variables is not automatically better. Use a small, justified set of pre-exposure characteristics relevant to the outcome and confounding. Unnecessary variables can consume sample without improving the comparison; variables affected by exposure can bias it. Do not drop a necessary confounder simply to increase the retained base.
Matching is not randomization. Demographic balance removes the imbalance you matched on and nothing else. Where treatment was not randomly assigned, unmeasured confounding remains possible and should be acknowledged.
IP is not a person. For matched media exposure, household-level attribution is the default.
Inference applies to the matched population, not the full achieved sample and not the market.
Treatment measurement bounds everything downstream. No amount of statistical care compensates for an unreliable exposure feed or a vague recall question.
This is not media mix modeling. Lift measures the effect of a specific treatment on measured outcomes for a specific audience. It does not allocate budget across channels, model diminishing returns, or substitute for MMM or geo-based incrementality testing. It answers a narrower question, directly.
Component references
| Component | Reference |
|---|---|
| Lift report configuration | Reporting lift against control groups |
| Sending exposure data | What are exposure sources |
| Pixel setup | Creating a pixel exposure source |
| Server-to-server setup | Creating an S3 snapshot exposure source |
| Respondent validation before matching | IP Address Hygiene and Exposure Matching |
| Sizing the study | Sample size and precision |
| Whether a campaign is large enough to measure | Campaign size and measurability |
| Weighting and effective sample size | Weighting methodology |
| Significance testing | Understanding stat testing |
For the product view of this methodology — live tracking, incrementality testing, and what a study involves — see brand lift and incrementality.