A model card is the one-document answer to the question a reviewer will ask about any computational step in a regulated procedure: what is this model for, what was it built on, how well does it work, where does it stop working, and which version produced the result in front of me. ICH Q2(R2) brought multivariate models explicitly into the validation framework and distinguishes model impact; ICH Q14 puts the analytical procedure, including its computational parts, into a lifecycle. A model card is how you keep both answerable without rewriting a validation report every time something changes.
Why the impact level comes first
ICH Q2(R2) treats a model differently depending on how much the reported result depends on it. The common distinction is:
- Low impact: the model supports a decision that is confirmed by another measurement, or the result is verified by a reference method.
- High impact: the reported result comes from the model and nothing else confirms it.
The impact level decides the depth of everything else: how much validation data, how tight the acceptance criteria, how formal the change control. Write it in the first section of the card, with the reasoning, because every reviewer's next question depends on it.
What belongs on the card
1. Identity and version.
Name, version, date it entered use, who owns it, and the identifier that appears in every result record.
2. Intended use.
The question the model answers, the input it expects, the output it produces with units, and the procedures that use it. Equally important: the uses explicitly out of scope.
3. Data.
What the model was built on: number of records, sources and their licences, the time period, how the data were curated and what was excluded. For a model trained on customer data, who owns it and what it may be used for.
4. Train, validation and test split.
How the data were divided, and by what rule. This is the single most informative line on the card. A random split on chemical data inflates performance, because near-duplicates land on both sides. A split by molecular scaffold, by batch, by site or by time period asks the question that matters: how does the model behave on something genuinely new.
5. Performance, with the split that produced it.
Every figure carries the metric name, the dataset it was measured on, the split rule and the sample size. A single number with no split description is not evidence. Where a comparison with a published figure is given, state that the published value comes from the source literature and is not a same-split comparison.
6. Uncertainty.
How the model expresses uncertainty, and whether that expression is calibrated, measured how. A prediction interval that is stated to cover 90 % and covers 90 % on held-out data is a result; a confidence score with no coverage check is decoration.
7. Applicability domain.
How the model decides that an input lies outside the chemistry or process space it knows, what happens then, and how accuracy differs inside and outside. Include the behaviour the procedure requires: refuse, warn, or flag for review.
8. Known limitations and failure modes.
The cases where the model is weakest, in plain terms. This section protects you. A limitation documented in advance is a limitation; the same limitation found by an inspector is a finding.
9. Qualification for intended use.
What the user, not the supplier, measured on their own data and against which acceptance criteria. For high impact models this is the core of the validation package.
10. Change history.
Every version with what changed, why, the comparability evidence against the previous version, and the change control reference.
How the card connects to the procedure
The analytical procedure, or the SOP, refers to the model card by name and version. The validation report references the qualification section. The result record carries the version identifier. The change control process requires an updated card before a new version goes into use. That chain is what makes a model maintainable in a regulated environment: when something changes, exactly one document is the source, and everything else points at it.
A practical consequence: the card is a controlled document. It is reviewed and approved like an SOP, not maintained as a wiki page.
Lifecycle: when does the card need revisiting?
- A new model version, always.
- A change in the input data pipeline, because the model's effective input changed even if the model did not.
- Performance drift detected in routine monitoring, which should itself be defined in the card.
- An extension of intended use, for example to a new product family, which needs its own qualification data rather than an assumption of transferability.
- A change in the software environment that moves numerical results beyond the stated tolerance.
Limits of the statement
- A model card is documentation. It makes a model assessable, it does not make it fit for purpose.
- The structure above is a working template drawn from the current ICH texts and common inspection expectations, not a prescribed format. Check the current version of Q2(R2), Q14 and the applicable GMP annexes, and align the card with your own quality system.
- Performance figures are only transferable to your use if your data resemble the test data. When they do not, the qualification section, not the supplier's numbers, is what counts.
Frequently asked questions
What is a model card in a regulated context?
A controlled document that states a computational model's identity and version, intended use, data basis, split rule, performance with metrics and sample sizes, uncertainty behaviour, applicability domain, limitations, user qualification and change history.
Does ICH Q2(R2) cover machine learning models?
Q2(R2) brings multivariate analytical procedures into the validation framework and differentiates the expected effort according to the model's impact on the reported result. It does not set out a separate machine learning chapter, so the general validation principles apply with that impact distinction.
Why does the data split matter so much?
Because it decides what the performance figure means. A random split on chemical data lets near-duplicates appear in both training and test sets, which inflates the apparent accuracy. A split by scaffold, batch, site or time measures behaviour on genuinely new cases.
What is the difference between a low impact and a high impact model?
In a low impact case another measurement or reference method confirms the result; in a high impact case the reported result rests on the model alone. High impact means more validation data, tighter acceptance criteria and stricter change control.
Who is responsible for qualifying a supplied model?
The user. Supplier documentation can reduce the work, but qualification for the intended use, on the user's data and against the user's acceptance criteria, stays with the user.