Documentation

This or That Question

When and Why to Use

Use this to ask binary forced-choice questions across multiple comparisons. It's ideal for:

  • Preference testing between paired items
  • Quick-fire brand or concept comparisons
  • Engaging users with fast-paced choice tasks

Supports randomization and media-based comparisons.

Other Names and Formats

A This or That Question with two options per row is a pairwise comparison, paired comparison, paired preference question, head-to-head comparison, or binary forced-choice question. Respondents choose one option from each pair.

Pair Groups

Pass a dictionary to row_options to place whole comparison pairs under non-selectable group headings. Defaults and returned answers keep the same row1, row2 and subsequent IDs assigned from the authored pair order. See Choice and Grid Groups for syntax and randomization rules.

Chat Experience

  • Each pair appears as a row with two side-by-side buttons or tiles
  • Users select one item at a time
With imagesWithout images
Markdown ImagesMarkdown Many Options

Traditional Experience

  • All pairs displayed in grid format
  • Responsive to keyboard or remote selection
  • Layout adapts to show multiple pairs simultaneously
With images, mobile optimizedWithout images
This Or That Question Figure 01This Or That Question Figure 02

Explicit Don't know response

Set dont_know_option="Don't know" for a separate choice in each pair. Its label must be distinct from the row's ordinary options. The response keeps the generated row ID and stores the configured label for an opted-out row. Respondents can switch back to either ordinary option without changing another pair.

Configuration Options

OptionTypeRequiredDefaultDescription
dont_know_optionstrno''Optional explicit opt-out label; per row for grids, otherwise for the whole question
questionstringyes-The main prompt for the task
row_optionsList[List[str | MediaItem]] | Dict[str, List[List[str | MediaItem]]]yes-Pairs as a flat list or groups under display headings
randomizeboolnoFalseShuffle the order of rows
randomize_columnsboolnoFalseShuffle left/right order within each pair
defaultDict[str, str | List[str]] | List[Dict]norandom choicesDefault test responses keyed by generated row ID (row1, row2, ...); a per-row list supplies candidates; or a list of candidate dictionaries — each simulated respondent picks one at random
recodesDict[str, str]no-Map each exact selected option string to a reported value
imageMediaItemno-Top-level image shown above the task
image_label_fieldstrno-Media field used as the label for image options
show_image_labelboolnoTrueWhether to show labels on image tiles
image_sizeTuple[int, int]no(600, 600)Size of displayed images
custom_validatorCallable[[dict[str, str]], str | None]no-Called with the complete parsed grid in both chat and traditional experiences. Return an error message to reject the response, or None to accept.
number_secondsintno0Seconds to wait before allowing the respondent to continue
tagss.tag()no-Used for token substitution and reporting
idstrno-Optional stable identifier for this question

Example Code

Code
s.this_or_that_question( "Which of the following do you prefer?", row_options=[ ["Apples", "Oranges"], ["Coke", "Pepsi"], ["Dogs", "Cats"], ["Winter", "Summer"] ] )

With recodes:

Code
s.this_or_that_question( "Which of the following do you prefer?", row_options=[["Netflix", "Disney+"], ["Prime", "Apple TV"]], recodes={ "Netflix": "streaming", "Disney+": "streaming", "Prime": "ecommerce", "Apple TV": "streaming" } )

With custom validator:

Code
s.this_or_that_question( "Choose your favorite in each pair", row_options=[["Day", "Night"], ["Hot", "Cold"]], custom_validator=lambda d: "Please vary your responses" if len(dict.fromkeys(d.values())) == 1 else None )

Notes

  • Returns a DictResponse of StringResponse values keyed by generated row ID — one-based (row1, row2, ...) in both chat and traditional experiences
  • Defaults always use the one-based keys (row1, row2, ...), in both experiences
  • Each row_options entry typically contains the two items being compared; rows with more options are allowed, and the respondent still selects one option per row
  • Randomizing column order helps reduce left/right bias
  • recodes simplify grouped reporting or segmentation, and map exact option strings only — numeric ranges and percentage keys are not supported
  • Use custom_validator to enforce answer patterns or detect straight-lining