A lift report compares respondents who were subject to something against a matched group who were not, then reports the difference in outcomes. This page explains how the treated group and matched control are constructed, how the difference is calculated, and which claims each design supports.
The measurement is general. Advertising brand lift is its most common application, but nothing in the method is specific to advertising: the platform needs a per-respondent indicator saying who was treated, one or more outcome questions, and a set of variables to match on. Anything expressible that way can be measured this way.
What can be measured
| Application | How the treated group is identified | Typical outcomes |
|---|---|---|
| Advertising campaign | Ad-server exposure matched to respondents | Awareness, recall, consideration, favorability, intent |
| Creative or concept test | Random assignment inside the survey | Appeal, message take-out, purchase intent |
| Message and claim testing | Random assignment inside the survey | Agreement, persuasion, believability |
| Sponsorship or event | Attendance or exposure recorded in the survey | Brand association, sentiment |
| Packaging or in-store stimulus | Random assignment, or observed exposure | Findability, appeal, switching intent |
| Direct mail, email, or CRM campaign | Matched against a first-party contact list | Recall, action taken, sentiment |
| Product trial or usage | Declared usage or a customer-file match | Satisfaction, advocacy, repeat intent |
A single stimulus topic (brand, creative, message, or any other label) makes the analysis run separately for each value of it. One study covering fifty brands produces fifty independent lift reads from one dataset. See Reaching low-incidence audiences.
How strong a claim the result supports
This is the first question a reviewer should ask, and the answer depends entirely on how the treated group came to be treated. Matching on demographics equalizes the variables you matched on. It does not turn an observational comparison into an experiment.
| How the treated group was formed | Design | What the result supports |
|---|---|---|
| Random assignment inside the survey | Experimental | A causal effect of the stimulus on the outcome, within the sample |
| Server-verified media exposure, matched | Quasi-experimental | The effect of having been exposed, net of the matched characteristics. Media targeting means exposure is not random, so residual confounding is possible |
| Declared or recalled exposure | Observational | Association. Recall is itself correlated with brand familiarity, which is frequently the outcome being measured |
| Behavioral or status split, such as customers against non-customers | Observational | Association only. Matching on demographics does not remove selection into the group |
State the design alongside any reported result. A quasi-experimental comparison supports a different claim from a randomized experiment and should not be described as randomized.
Establishing the treated group
Whatever the application, the mechanism is the same: an indicator is recorded against the respondent and carried onto their answers, and the lift report reads it back.
The indicator is set in the survey with an s.tag() block, so every question inside the block carries the respondent's status automatically. Configuration is documented in Reporting lift against control groups.
Random assignment in-survey
Respondents are randomly allocated to see or not see the stimulus, and the allocation is tagged. Because assignment is random, the comparison estimates a causal effect rather than an association. This design applies to creative, concept, message, and packaging tests whether or not media has run.
Matched media exposure
Where the treatment is real-world media, exposure records are sent to MX8 Labs from the ad server and matched against respondents.
Two ingest routes are supported. A pixel hosted by MX8 Labs is embedded in the ad server or tag manager and fires on render, capturing exposures in real time; this suits display and rich media. Server-to-server delivery uploads gzip-compressed CSV files of exposure records to an MX8-managed S3 bucket, which suits connected TV, audio, server-rendered placements, and any case where IP addresses should be hashed before leaving the client's environment.
Each record carries an IP address, plaintext or MD5-hashed, plus optional identifiers: a user identifier of up to 128 characters, and dimensions such as brand or creative. A retention window is set per source to match the campaign timeline.
IP address is the join key. A match is recorded only when four conditions all hold:
| Condition | What is checked |
|---|---|
| IP address parity | The same IP appears in both the exposure log and the response pool |
| Same study window | Both records fall inside the fielding window defined for the campaign |
| Respondent passes validation | The respondent cleared the full quality pipeline |
| Exposure is attributable | The ad server recorded a real impression event, not a bot or crawler fetch |
The third condition carries most of the weight. Matching against an uncleaned respondent pool produces a match rate rather than a measurement, because duplicates, bots, and spoofed sessions all carry IP addresses. MX8 Labs validates the respondent pool before the join runs. In the production data summarized in our January 2026 survey fraud analysis, that validation excluded 10 to 20 percent of incoming respondents; the rate varies by study and source. See IP Address Hygiene and Exposure Matching and Data quality methodology.
The unit of attribution is the household by default. IP addresses are shared behind household NAT, so the person who saw the ad and the person who answered the survey are not necessarily the same individual. This is a property of any IP-joined design, and for most consumer campaigns in-home exposure is the relevant signal anyway. Studies requiring person-level attribution are configured separately, with sample design and in-survey verification to support the stricter standard.
Exposure records with no matching respondent still count toward reach but contribute nothing to respondent-level analysis. Respondents with no matching exposure form the untreated pool.
Because survey-based lift asks people directly, it is independent of the pixel for the outcome measurement even when the pixel supplies the exposure signal. It stays usable on media where no click exists, and resistant to the identifier degradation eroding attribution-based measurement.
Frequency and recency
Matched exposures carry two further attributes: frequency, the number of exposures for that value, and days_since_exposure, the recency of the most recent one where available.
This allows treatment to be defined as something more precise than a binary, such as respondents seen three or more times in the last seven days, or the questionnaire to branch on exposure intensity, or light and heavy exposure to be compared directly. For wear-out and frequency-response questions this is the difference between a usable study and a null result. See What are exposure sources.
Declared exposure
Where no feed and no randomization exist, the indicator can be set from a recognition or recall question. This is the weakest route, for the reasons in the table above, and reporting should describe it as declared rather than verified.
Constructing the control group
The control group is drawn from untreated respondents and matched to the treated group on chosen variables: gender, age band, region, prior category usage, or any other question designated as a control. Control variables are used only for matching; lift is not measured on them.
Matching is repeated rather than performed once. A result based on one matched control can be sensitive to which eligible untreated respondents are selected.
For the configured number of iterations, the platform constructs a balanced control group from the available untreated pool and averages the results. Selection is deterministic: the same respondent data, configuration, and reporting implementation produce the same matched controls. Repeating the match reduces dependence on any one eligible control selection; it does not remove sampling variance or unmeasured confounding. New lift reports created in the interface use five iterations by default, while the reporting API accepts an explicit iteration count.
Computing lift
Lift is computed separately for each value of the stimulus topic, so one study measures each brand, creative, or message independently rather than in aggregate.
Outcomes are handled by type:
- Categorical outcomes - awareness, recall, brand selected, option chosen. Lift is calculated separately for each response value.
- Numeric and scale outcomes - consideration on a 0 to 10 scale, a 1 to 5 rating. Lift is calculated at thresholds, for example the proportion scoring 7 or above, 8 or above, 9 or above. This matters because a stimulus often moves the top of a scale without moving the mean, and a mean-only comparison hides that.
For each stimulus value the sequence is: split respondents into treated and untreated candidates, build a matched control on the control variables, calculate outcome rates for both groups, take the difference, test it, and average across matching iterations.
Significance
Every reported difference is tested, and results that do not clear the configured threshold are suppressed rather than shown with a caveat.
The p-value threshold is configurable. Use 0.05 for externally reported results, 0.10 for directional reads, and looser settings only for explicitly exploratory analysis. Weighted margin of error is reported on all results, and where weighting is applied, standard errors use a Kish effective-sample-size adjustment so precision claims degrade as weights become more heterogeneous; see Weighting methodology and Understanding stat testing. Negative lift is off by default and can be enabled to include outcomes where the treated group performed worse than control.
Reading a result
Media exposure matched from an ad server, stimulus topic set to brand, gender as the control variable:
| Brand | Metric | Treated | Control | Lift |
|---|---|---|---|---|
| Brand A | Awareness | 62% | 54% | +8 pts |
| Brand A | Consideration 7+ | 38% | 33% | +5 pts |
This supports the claim that exposure to the Brand A campaign was associated with higher awareness and consideration among the matched population, net of the demographic differences matched on, at the configured significance threshold. Because exposure was not randomly assigned, it is a quasi-experimental read. It does not support a claim about the full sample, about respondents outside the matched population, or about mechanism.
Limitations
Matching consumes sample. Every treated respondent needs a comparable counterpart. A thin untreated pool, or a treated group with an unusual profile, reduces the usable base and widens intervals. Campaign reach governs the split you get, and because reach against a sampled audience is usually low, the exposed group is nearly always the scarce side while control is abundant. Sample size and precision gives the arithmetic and the total sample required at each level of reach.
More control variables is not better. Each one makes the comparison cleaner in principle and reduces power in practice. The right number is the smallest set that plausibly confounds the outcome.
Matching is not randomization. Demographic balance removes the imbalance you matched on and nothing else. Where treatment was not randomly assigned, unmeasured confounding remains possible and should be acknowledged.
IP is not a person. For matched media exposure, household-level attribution is the default.
Inference applies to the matched population, not the full achieved sample and not the market.
Treatment measurement bounds everything downstream. No amount of statistical care compensates for an unreliable exposure feed or a vague recall question.
This is not media mix modeling. Lift measures the effect of a specific treatment on measured outcomes for a specific audience. It does not allocate budget across channels, model diminishing returns, or substitute for MMM or geo-based incrementality testing. It answers a narrower question, directly.
Component references
| Component | Reference |
|---|---|
| Lift report configuration | Reporting lift against control groups |
| Sending exposure data | What are exposure sources |
| Pixel setup | Creating a pixel exposure source |
| Server-to-server setup | Creating an S3 snapshot exposure source |
| Respondent validation before matching | IP Address Hygiene and Exposure Matching |
| Sizing the study | Sample size and precision |
| Weighting and effective sample size | Weighting methodology |
| Significance testing | Understanding stat testing |