> For the complete documentation index, see [llms.txt](https://help.metaforms.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.metaforms.ai/data-processing/python/validation-coverage.md). # Validation coverage Metaforms' Python Data Validation module generates checks across a wide range of question types and survey designs. This page summarises what is supported today and what is on the roadmap, so QA and DP teams know up-front which scenarios they can rely on and which ones still need manual scripting. *** ### Question types and sub-scenarios supported today The matrix below lists each question type and the sub-scenarios for which Metaforms automatically generates Python validation scripts. | Question type | Sub-scenarios covered | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | **Single Select** | Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating, loop | | **Multi Select** | Basic, with terminations, with filter, with piping logic, with open-end text, with Exclusive Option, loop | | **Single Grid** | Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating (including Straight-Liner), loop | | **Multi Grid** | Basic, with terminations, with filter, with piping logic, with open-end text, with open-end numeric, Scaling/Rating, loop | | **Open-End Text** | Basic, Grid, Grid with piping logic, with filter, loop | | **Open-End Numeric** | Basic, Grid, Grid with piping logic, with filter, loop | | **Ranking** | Basic, with filter, with piping logic, with open-end text, with open-end numeric, loop | | **Hidden** | Single Punch, Multi Punch, with terminations, with filter, with piping logic, with open-end text, with open-end numeric | | **Leastfill** | Without priority, with filter, Single Punch, Multi Punch, with piping logic | | **Segmentation** | Multi Punch, with filter | | **Bipolar** | Basic, with filter | *** ### Per-respondent checks generated Within each supported question type, the AI generates one or more of the following respondent-level checks depending on the questionnaire logic. | Check | What it verifies | | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | | **Answer Option Range Check** | Response values fall within the expected set of options for the question (e.g., flags a `6` for a 1–5 single-select). | | **Null Check** | A response exists where the questionnaire requires one (incl. specifically for termination answer options on multi-selects and hidden qns). | | **Empty String Check** | Open-end text fields are not blank when the question should have been answered. | | **String Length Check** | Open-end text responses meet the expected length bounds. | | **Numeric Range Check** | Open-end numeric responses fall within the expected min/max range. | | **Exclusive Answer Check** | Exclusive options (e.g., "None of the above") are not selected alongside other options. | | **Filter Condition – Entry Check** | Only respondents who meet the filter condition were shown the question. | | **Filter Condition – Option Check** | Piped/masked answer options shown to a respondent match what the prior question's logic should have produced. | | **Back Check (reverse routing)** | Respondents who should have been skipped past a question were actually skipped — the reverse of the entry-condition check. | | **Straight-Liner Check** | Flags grid responses where the same answer is selected across every row. | | **Min and Max Check** (Ranking, Leastfill) | The number of selected/ranked items falls within the configured min/max bounds. | | **Duplicate Rank Check** | Flags rankings where the same rank value is assigned to more than one item. | | **Sequence Check** | Verifies ranks form a contiguous sequence (1, 2, 3, …) with no gaps. | | **"At least 1" Check** | Verifies at least one response is provided in a grid open-end where required. | *** ### Project-level QC checks In addition to per-question validation, Metaforms runs the following QC checks across the full dataset: * **Status check** — Verifies respondent status (complete, terminate, screen-out) is consistent with the data captured. * **Duplicate ID check** — Flags duplicate respondent IDs across the dataset. * **IP check** — Flags suspicious IP patterns (e.g., multiple respondents from the same IP). * **Straight-liners (per question)** — Flags respondents straight-lining a single grid question. * **Diagonal straight-liners (per question)** — Flags diagonal selection patterns in a grid. * **Straight-liners (multi-question)** — Flags straight-lining across multiple grid questions, as defined in the questionnaire setup. * **Trap question check** — Flags respondents who failed an attention/trap question. * **Speeders – LOI** — Flags respondents whose length-of-interview falls below the expected threshold. *** ### On the roadmap Support for the following is on our near-term roadmap: #### Question types * Conjoint * MaxDiff * Hidden quota variables * Hidden filter-text variables * Leastfill with priority #### QC checks * Sense checks (cross-field plausibility, e.g., totals that cannot exceed a known bound) * Price-range plausibility checks --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://help.metaforms.ai/data-processing/python/validation-coverage.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.