This page documents the respondent validation methodology applied to every MX8 Labs study. It is the methodology companion to two existing references: Fraud prevention measures, which lists the public signal categories, and IP Address Hygiene and Exposure Matching, which documents the pipeline in the context of ad-exposure work.
Scope
Quality control uses five layers:
- Deduplication - is this the same person or device we have already recorded on this study?
- Fraud and bot screening - is this a real human, answering in good faith, in the environment they appear to be in?
- Identity verification - where the study demands it, can this respondent prove they control a real phone number?
- In-survey attention and consistency checks - is this respondent engaging with the questions?
- In-field monitoring - is any individual respondent source degrading while the study runs?
Available evidence for layers one and two is collected on eligible browser sources, including trusted sources. Their duplicate and suspect exclusion controls are configured independently per source. Layer three is added where the sensitivity of the study justifies the friction. Layer four is designed into the questionnaire. Layer five is visible while the study is live.
In the production data summarized in our January 2026 survey fraud analysis, this pipeline excluded 10 to 20 percent of incoming respondents before they reached analysis. That figure describes the observed period, not a guaranteed rejection rate for every study.
Evidence is collected independently of whether a source is configured to exclude respondents. This lets researchers inspect findings on trusted or record-only browser sources as well as on sources with active screening. Source-specific restrictions still apply to non-browser, synthetic and test data.
Layer 1: Deduplication
Duplicate checks draw on session cookies, IP addresses and device fingerprints. The recorded duplicate finding and the decision to exclude are separate: Allow duplicate respondents controls whether a finding causes duplicate termination.
| Signal | Role | Failure mode it covers |
|---|---|---|
| First-party session cookie | Checks whether the browser session is consistent with the recorded respondent | The straightforward repeat entrant |
| IP address | Secondary signal where cookies are cleared or unavailable; also flags high-density traffic from a shared network | Cookie clearing; coordinated entry from one network |
| Device fingerprint | Device identifier designed to remain stable across a respondent's sessions, including where cookies are unavailable | Sophisticated duplicates that defeat the first two |
Each signal has limits. Cookies can be cleared, IP addresses rotate legitimately on mobile networks and are shared behind household NAT, and fingerprints can change with browser and operating system updates. A match is evidence to review, not proof that two records represent the same person. Device matching spans eligible live respondent sources within the same survey; test and simulated respondent sources are excluded from that matching.
Device intelligence analyzes multiple browser attributes to help identify duplicate and suspicious devices. It is used alongside cookie and IP checks rather than as a standalone decision. The public categories of behavioral, network, bot, device, and mobile checks are listed in Fraud prevention measures.
Layer 2: Fraud and bot screening
Deduplication establishes uniqueness. It does not establish legitimacy: a unique respondent can still be a bot, an automated agent, or a professional fraud operation.
Screening evaluates five families of signal in real time, described at category level in Fraud prevention measures:
- Behavioral - how the browser session is configured and whether it has been altered to avoid identification.
- Network - the consistency and history of the network the respondent connects from.
- Bot and automation - whether the session shows the characteristics of automated participation.
- Device - the coherence of the reported device profile.
- Mobile integrity - whether a mobile device and its app environment are in an unmodified state.
Available evidence is combined into the MX8 suspect score, from 0 to 5. The source's Quality filter selects which MX8 scores cause exclusion: 0 disables suspect termination, while 5 excludes any score from 1 to 5. See the complete filter table.
Unavailable, pending and unsupported results are not scores of zero.
Layer 3: Identity verification
Deduplication and fraud screening operate on signals the respondent emits. Some studies need something stronger: proof that the respondent controls a real, unique phone number.
SMS verification adds that step inside the questionnaire. The respondent enters a phone number, receives a one-time code by SMS, and must enter it correctly before continuing. Because a phone number is costly to acquire at scale and difficult to reuse undetected, this is a materially harder control to defeat than any browser-side signal, and it also serves as a deduplication key in its own right.
The added friction is justified on:
- High-sensitivity or confidential studies, including anything behind gated stimulus.
- Studies where duplicate participation would be materially damaging to the result.
- Studies where a meaningful incentive creates an incentive to game entry.
It is not applied by default, because it raises break-off and narrows the reachable population: respondents without a mobile number, or unwilling to share one, are excluded. That is a coverage cost, and it should be a deliberate trade rather than a standing setting. See Verifying phone numbers.
Where a study is fielded over SMS or voice rather than the web, the phone number is inherently the identifier, and these respondent sources are treated as trusted for fingerprint-based checks accordingly.
Layer 4: In-survey attention and consistency
Automated screening addresses who the respondent is. Questionnaire-level checks address whether they are engaging.
Applied by default, with no configuration:
- Straight-lining detection on grid questions, which prompts the respondent to review when every row carries an identical answer.
- Duplicate and suspect evidence checks on eligible browser sources; exclusion follows the source's independent controls.
Available to design into the questionnaire:
- Overt instructional checks - an explicit instruction the respondent must follow, applied either as a warn-and-retry validator or as an immediate termination.
- Covert consistency checks - logically contradictory answer patterns across questions, which catch respondents who pass overt checks by reading carefully but answer without engaging.
- Speed traps - elapsed time between any two answered questions is available in survey logic, and a per-question timer can delay advancement, so dwell-time thresholds can be set per question rather than only across the whole survey.
- Escalation - a running count of failed validations, allowing termination after repeated failures rather than on a single slip.
- Open-end validation - AI-assisted checking of free-text responses for coherence and relevance to the question asked. See AI-enhanced validation for text questions.
Guidance we give on design: one or two overt checks in a ten- to fifteen-minute survey is normally sufficient, options in check questions should be randomized so that position cannot be learned, and overt and covert checks should be mixed. Overloading a questionnaire with checks degrades the experience for genuine respondents and raises break-off, which is itself a quality problem.
Worked examples are in Using "Gotcha" Questions in MX8 Labs to Improve Survey Quality.
Layer 5: In-field monitoring
Quality can be reviewed overall and per source while the study runs. The field report separates duplicate and suspect-score distributions from recorded quality-policy terminations, including unattributed exclusions. It uses the report's selected source/date snapshot and distinguishes unavailable evidence from a clean result. Raw exports provide respondent-level quality fields where available. See Field reports and the sourcing detail in Respondent sourcing methodology.
Defaults and configurability
The active controls form part of the study configuration and should be checked when reviewing a dataset.
- Bot detection is on by default. The existing survey-level setting can be disabled where every entrant is known to be human.
- Duplicate and suspect exclusion are independent. Standard browser-source defaults disallow duplicates and use Quality filter 5. Allow duplicates and use filter 0 for record-only monitoring; available evidence is still retained.
- Existing trusted browser sources retain evidence. Trust affects termination policy, rather than establishing that no security data is collected.
- Some source types have separate behavior. Synthetic, profile synthetic, third-party import, Twilio voice/text and internal test sources do not expose the same browser controls.
See Fraud prevention measures for the source settings and the complete score/filter table.
Account security is a separate control
Respondent validation and platform access control answer different questions, and a review should keep them apart.
The layers above govern who enters a dataset. Access to the data once it exists is governed separately: platform users authenticate with multi-factor authentication, permissions are configured per user per account through granular roles covering survey editing, reporting, fielding, media, going live, and administration, and only users holding the relevant permission can put a survey live and incur cost. Studies can be hosted in a specific region where data residency requires it.
See Understanding roles and permissions and Local data residency.
What this methodology does not do
It does not correct for coverage or non-response bias. Screening removes invalid respondents. It does not make the surviving sample representative. Composition is addressed by calibration, and the limits of that framework are set out in Weighting methodology.
It does not apply to synthetic respondents. Synthetic Twins and Synthetic Profiles are generated rather than recruited, so human-quality screening is not meaningful for them. They are labeled distinctly throughout the platform, and their inferential treatment is documented in Synthetic data and statistical inference.
It does not claim a zero false-negative rate. The 10 to 20 percent figure above describes exclusions during the published observation period, not a fixed platform rate or a guarantee of what remains. Fraud techniques change, and no screening system detects every invalid respondent.
Privacy
Browser security evidence, device identifiers, respondent-linked scores and duplicate findings are treated as personal data. Hashing a lookup key does not make the underlying records anonymous: device markers remain associated with respondents and respondent sources. Allowing duplicates or disabling suspect termination does not stop available evidence being retained on eligible browser sources.
Fingerprinting is used for survey security and fraud prevention. Field reports show aggregate quality distributions; supported raw downloads can contain respondent-level scores, duplicate findings and recorded termination reasons. Detailed security signals are not included in field reports or raw downloads. Share report links and downloaded files only with intended recipients.
Retention and privacy-erasure controls cover the associated quality metadata and device markers. Ordinary clearing of test responses preserves testing history; it is different from a permanent privacy-erasure request. See Privacy Compliance and Local data residency. MX8 Labs' current certification status is published at trust.mx8labs.com.