Validation coverage

Question types, sub-scenarios, and validation checks supported by Metaforms Data Validation when scripting in Python.

Metaforms' Python Data Validation module generates checks across a wide range of question types and survey designs. This page summarises what is supported today and what is on the roadmap, so QA and DP teams know up-front which scenarios they can rely on and which ones still need manual scripting.

---

Question types and sub-scenarios supported today

The matrix below lists each question type and the sub-scenarios for which Metaforms automatically generates Python validation scripts.

Question type	Sub-scenarios covered
Single Select	Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating, loop
Multi Select	Basic, with terminations, with filter, with piping logic, with open-end text, with Exclusive Option, loop
Single Grid	Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating (including Straight-Liner), loop
Multi Grid	Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating, loop
Open-End Text	Basic, Grid, Grid with piping logic, with filter, loop
Open-End Numeric	Basic, Grid, Grid with piping logic, with filter, loop
Ranking	Basic, with filter, with piping logic, with open-end text, with open-end numeric, loop
Hidden	Single Punch, Multi Punch, with terminations, with filter, with piping logic, with open-end text, with open-end numeric
Leastfill	Without priority, with filter, Single Punch, Multi Punch, with piping logic
Segmentation	Multi Punch, with filter
Bipolar	Basic, with filter

---

Per-respondent checks generated

Within each supported question type, the AI generates one or more of the following respondent-level checks depending on the questionnaire logic.

Check	What it verifies
Answer Option Range Check	Response values fall within the expected set of options for the question (e.g., flags a 6 for a 1–5 single-select).
Null Check	A response exists where the questionnaire requires one (incl. specifically for termination answer options on multi-selects and hidden qns).
Empty String Check	Open-end text fields are not blank when the question should have been answered.
String Length Check	Open-end text responses meet the expected length bounds.
Numeric Range Check	Open-end numeric responses fall within the expected min/max range.
Exclusive Answer Check	Exclusive options (e.g., "None of the above") are not selected alongside other options.
Filter Condition – Entry Check	Only respondents who meet the filter condition were shown the question.
Filter Condition – Option Check	Piped/masked answer options shown to a respondent match what the prior question's logic should have produced.
Back Check (reverse routing)	Respondents who should have been skipped past a question were actually skipped — the reverse of the entry-condition check.
Straight-Liner Check	Flags grid responses where the same answer is selected across every row.
Min and Max Check (Ranking, Leastfill)	The number of selected/ranked items falls within the configured min/max bounds.
Duplicate Rank Check	Flags rankings where the same rank value is assigned to more than one item.
Sequence Check	Verifies ranks form a contiguous sequence (1, 2, 3, …) with no gaps.
"At least 1" Check	Verifies at least one response is provided in a grid open-end where required.

---

Project-level QC checks

In addition to per-question validation, Metaforms runs the following QC checks across the full dataset:

• Status check — Verifies respondent status (complete, terminate, screen-out) is consistent with the data captured.

• Duplicate ID check — Flags duplicate respondent IDs across the dataset.

• IP check — Flags suspicious IP patterns (e.g., multiple respondents from the same IP).

• Straight-liners (per question) — Flags respondents straight-lining a single grid question.

• Diagonal straight-liners (per question) — Flags diagonal selection patterns in a grid.

• Straight-liners (multi-question) — Flags straight-lining across multiple grid questions, as defined in the questionnaire setup.

• Trap question check — Flags respondents who failed an attention/trap question.

• Speeders – LOI — Flags respondents whose length-of-interview falls below the expected threshold.

---

On the roadmap

Support for the following is on our near-term roadmap:

Question types

• Conjoint

• MaxDiff

• Hidden quota variables

• Hidden filter-text variables

• Leastfill with priority

QC checks

• Sense checks (cross-field plausibility, e.g., totals that cannot exceed a known bound)

• Price-range plausibility checks