MX8 Labs reports can apply statistical tests to table cells and use color to show the result of each comparison:
- Green: Significantly higher result
- Red: Significantly lower result
- Blue: Normal result, no significant difference
- Transparent: Result is below the reporting threshold (default minimum: 20 respondents)
By default, MX8 Labs uses a residual t-test at a 95% confidence level and does not color cells with fewer than 20 respondents. The residual test identifies cells that are significantly higher or lower than expected from the row and column distribution.
Reports also support row-based and column-based comparisons. The appropriate mode depends on which cells the analysis intends to compare.
Choosing a different configuration
- To conduct more granular analyses within smaller subsets of data.
- If a different confidence level (e.g., 90% or 99%) is required to align with specific research standards or regulatory guidelines.
- To perform comparative analyses focused explicitly on rows or columns rather than overall data distribution.
Adjusting the confidence level
Confidence testing identifies results that are statistically significant compared to other responses. The default confidence level in MX8 Labs is 95%, indicating there's only a 5% chance that the marked differences happened by random variation.
When to change
- When a higher confidence level (e.g., 99%) is necessary for stricter confidence in findings.
- When lower confidence levels (e.g., 90%) can highlight potential trends or insights for exploratory analysis.
Using a residual t-test
This approach identifies cells within a report table that are significantly higher or lower than expected from the distribution between rows and columns.
When to use
- This is for standard exploratory analysis or to identify unexpected anomalies in the data.
- If a more focused comparative analysis across rows or columns is required, consider using a row - or column-based t-test.
Using a row-based t-test
This test compares each cell against other cells in the same row. Cells marked with labels indicate those cells in the same row that are statistically lower.
When to use
- When analyzing category performance within individual rows.
- Applicable when assessing item-level differences clearly across horizontal data.
Using a column-based t-test
This test compares each cell against other cells in the same column. Labels indicate that cells in the same column are statistically lower than those highlighted.
When to use
- For vertical comparisons across segments or groups.
- When the intended comparison is between cells in the same column.
Changing the reporting threshold
MX8 Labs applies a reporting threshold to each response. If the number of respondents for a specific cell falls below this threshold, the results are not colored in the reports.
This is applied at the level of the individual response to the question, not at the question itself, so that we can filter out significantly low responses to an asymmetric question. For example, let's ask people about their ethnicity. We're more likely to get statistically significant results for common ethnicities. Still, we don't want to filter out ethnicity options entirely because of the small number of respondents in a specific minority.
The default threshold for reporting is 20 respondents.
When to change
- Increase the threshold when cells below a larger minimum base should be left unmarked.
- Lower the threshold when smaller cells must remain visible, while accounting for their lower precision.