Documentation

Traditional or Chat based survey experience?

Survey design is evolving rapidly, with chat-based experiences gaining traction alongside traditional web forms. Both have strengths and limitations, making choosing between them key to ensuring high-quality data collection. This article examines the pros and cons of each approach and offers guidance on when to use them effectively. We'll also explore how the MX8 Labs Research Platform supports traditional and chat styles, allowing flexibility to meet your survey needs.

The style is set once per survey in the Theme tab — it is not mixed within a single study. Both styles render on desktop and mobile; the difference is how each question is laid out on the screen, not which device the respondent can use.

Preview the respondent experience

Use a test link to review the actual questionnaire on desktop and mobile. Check long prompts, images, answer labels, keyboard navigation and any numeric starting values. For comparison tasks, check that every attribute and concept heading is readable without relying on a desktop-only layout. See Testing and Reviewing Your Survey and the compatibility matrix.

Traditional Web Form Surveys:

Traditional web forms, especially on a PC or laptop, are ideal for complex grid-style questions, such as Likert scales or matrix comparisons. In our head-to-head testing they complete just as quickly as chat overall, but they do not hold on to everyone equally: lower-income respondents abandon forms at roughly twice the rate of higher-income respondents.

Traditional desktop survey with a two-column numeric grid, three rows and a Send button

Pros:

  • Best for Complex Grids: Traditional surveys are ideal for presenting grid-style questions. These questions, commonly used for Likert scales or matrix ratings, benefit from the horizontal space, reducing scrolling and improving readability.
  • Familiarity: Many respondents are accustomed to this format, making it a reliable choice for straightforward data collection. In our parallel test, higher-income respondents in particular stayed with forms better than with chat.

Cons:

  • Uneven fatigue: In our parallel test, abandonment on forms ran from 9.3% among 50k+householdsto19.150k+ households to 19.1% among under-50k households. The people surveys most often under-represent are the ones forms lose.
  • Stalled sessions: Hour-plus, walk-away-and-come-back sessions are five times as common on forms (1.5% vs 0.3% of completes), which shows up as distracted answers and a less predictable field.

Chat-Based Surveys

In chat mode, the platform presents one question at a time in a thread that resembles a messaging app. This reduces cognitive load and keeps the exchange moving — at no cost in time, since median completion matches traditional forms. The current Chat experience also supports complete grids and comparison questions. Dense layouts still need careful preview on smaller screens.

Desktop chat survey preview with the question thread, answer choices, input field and stored test values

This test preview also shows the tester toolbar and stored values; those controls help you review the survey while testing.

Pros:

  • Keeps harder-to-reach groups in: In our parallel test, fatigue among under-$50k respondents fell from 19.1% on forms to 13.3% on chat, and every group we measured stayed within a narrow 13–16% band.
  • Cleaner data: Chat produced fewer poor-quality removals (17.1% vs 20.2% of entrants), and each complete carried about 14% more effective sample after weighting.
  • Optimized for Mobile Devices: Chat-based surveys are perfect for mobile-first experiences, where vertical scrolling is more intuitive.

Cons:

  • Dense grids need preview: Complete grids are supported in Chat, but long labels and many columns can still need more room than a small screen provides.
  • Higher-income respondents prefer forms: Fatigue among $50k+ respondents rose from 9.3% on forms to 14.0% on chat in our parallel test, so forms retain affluent and professional samples better.
  • Potential for Misinterpretation: Respondents might find it harder to revisit or reference previous questions for clarity.

Theming in MX8 Labs

The MX8 Labs Research Platform supports both traditional and chat styles, offering flexibility to suit different survey styles:

  • Traditional Designed for traditional web forms with grid-style questions and detailed input fields. It is ideal for desktop users or surveys requiring comparison tasks.
  • Chat: Optimized for chat-based surveys, focusing on single-question delivery and mobile-friendly design.

You can configure the look and feel of the survey in the Theme tab of the survey page. As well as the survey style, you can choose the colors you want to use in the study and change the images.

Traditional Or Chat Based Survey Experience

This adaptability allows researchers to choose the format that best matches their audience and survey content, ensuring a seamless user experience across devices.

Head-to-Head Performance

The figures below describe the parallel study we ran; they are not a fresh performance measurement of the current interface.

We fielded the same 40-question study in both styles at once: parallel cells, matched samples, 2,803 respondents in total. On the toplines the two styles matched. Abandonment was 14.4% of starters in both cells, and median completion time was about ten minutes in both. Earlier versions of this page reported chat completing around a minute faster with a 2–3 point completion-rate advantage; the parallel test does not support either claim, and we have corrected it here.

The differences sit underneath the averages. On forms, abandonment ran from 9.3% among 50k+householdsto19.150k+ households to 19.1% among under-50k households; on chat, every group stayed between roughly 13% and 16%. Chat retained lower-income respondents markedly better (19.1% → 13.3%), while higher-income respondents abandoned chat somewhat more (9.3% → 14.0%). Chat also produced fewer poor-quality removals (17.1% vs 20.2%), carried about 14% more effective sample per complete after weighting, and had far fewer stalled, hour-plus sessions (0.3% vs 1.5% of completes).

For the answers themselves, the styles are interchangeable on standalone questions — 16 of 19 showed no detectable difference — and open-ended answers were equally detailed in both, at about 9–10 words per answer. The researchers tested the earlier row-by-row Chat grid flow. They did not test the current complete-grid interface. Keep tracker presentation consistent, or bridge a layout change with a parallel wave.

The retention advantages above were measured on a study with one six-statement grid using that earlier Chat flow. They do not establish how a grid-heavy study performs with the current interface. Preview both current layouts for your questionnaire.

Question-Type Compatibility

Every MX8 Labs instrument renders in both browser styles. Grid rating, grid select, multi-select grid and numeric grid now appear as complete grid questions in Chat as well as Traditional. Voice and text flows retain compatible split questions. Preview dense grids before choosing a layout. See the Chat vs. Traditional compatibility matrix for a per-instrument breakdown and recommendations.

Use an ordinary question group to collect related questions under one heading. Use a repeated question group when the same text, numeric, single-select or multi-select questions apply to several rows, such as children, visits or brands. Each child keeps its own validation and reporting identity; the group heading is not a reporting question.

Landscape forms show each row's repeated controls together. Chat keeps the grouped controls in the conversation, retaining row context. Supported groups can include images and messages, and whole-group validation can check related answers together. Preview the actual group on desktop and mobile to check long labels, correction messages and stored answers before fielding.

For supported controls, authoring examples and answer handling, see Question groups.

When to Use Each Style

While both styles have their place, we default to chat for general-population work. It keeps the people surveys most often under-represent in the study, produces cleaner completes, and costs nothing in time. Use traditional web forms where grid-style questions dominate, or where your sample skews affluent or professional.

The MX8 Labs Research Platform empowers you to combine these approaches effectively, ensuring your survey experience is tailored to your audience and research goals.

Chat-Based:

  • Best For: General-population studies where representation matters, samples skewing lower-income, mobile-first surveys, and cost-sensitive work where cleaner data stretches the budget.
  • Examples: Quick feedback surveys, conversational market research, or quizzes with a storytelling element.

Traditional Web Forms:

  • Best For: Surveys with detailed or grid-based questions, affluent or professional samples, or when the survey will likely be completed on a desktop.
  • Examples: Employee satisfaction surveys, detailed academic research, or complex market segmentation tasks.