Every QSAR model returns a number for every molecule you feed it. That is the problem. A model trained on 1,000 marketed drugs will still hand you a confident-looking probability for a peptide macrocycle, a boron-containing warhead or a per-fluorinated surfactant it has never seen anything like. The applicability domain is the machinery that tells you which of those numbers you are allowed to act on.
What the applicability domain actually is
The applicability domain (AD) is the region of chemical space where a model's error estimate is still supported by evidence. It is OECD validation principle 3 for QSAR models, alongside a defined endpoint, an unambiguous algorithm, goodness-of-fit measures and a mechanistic interpretation. Without an AD statement, a QSAR result is not a validated prediction. It is an extrapolation with a decimal point.
Practically, the AD answers one question: how much of the training set is near this molecule? If the answer is "nothing within reach", the model is interpolating between points that do not exist, and the held-out accuracy you read off a benchmark table does not transfer.
The three AD methods you will actually meet
Most production systems combine two or three of these. They fail in different ways, which is why combining them is worth the effort.
| Method | What it measures | Fails when |
|---|---|---|
| Nearest-neighbour similarity (Tanimoto on fingerprints) | Structural distance to the closest training compound | Similar structure, different mechanism (activity cliffs) |
| Feature range / bounding box | Whether each descriptor falls inside the training min-max | Every descriptor is in range individually but the combination is novel |
| Leverage (Williams plot, hat matrix) | Distance from the descriptor-space centroid, relative to a h* cutoff | Assumes a linear model; misleading for tree ensembles |
Our analysis checks for every prediction whether the compound chemically fits what the prediction is based on, and reports the result separately. A compound counts as in-domain only when both checks in the table below pass.
Worked example: the same molecule, three verdicts
Take a representative EGFR inhibitor from the marketed-drug class (gefitinib, SMILES COc1cc2ncnc(Nc3ccc(F)c(Cl)c3)c2cc1OCCCN1CCOCC1). This is not a novel compound; it is a well-characterised reference. Below are the results of our applicability domain check, run 2026-07-25.
| Endpoint | In domain | Nearest-neighbour similarity | Descriptor coverage |
|---|---|---|---|
| Tox21 NR-AR | yes | 1.00 | 0.9635 |
| hERG | yes | 0.75 | 0.9650 |
| DILI | no | 0.00 | 0.9808 |
Same molecule, three different answers. The Tox21 NR-AR result is 1.00 because gefitinib is literally in the Tox21 screening library, so the nearest neighbour is itself. hERG sits at 0.75, comfortably inside a large and chemically diverse training set. DILI returns 0.00 similarity, so the Tanimoto gate fails even though 98 percent of its descriptors are in range. That is the bounding-box failure mode in the table above, caught by the similarity gate.
What has to happen with that verdict
A prediction outside the applicability domain must not carry the same weight as one inside it. The rule we apply is blunt on purpose: if the applicability domain check fails, the prediction is not decision-ready, regardless of the endpoint's headline accuracy. A model does not get to keep its credibility on a molecule it has never seen.
Here is what that looks like on the same compound. For hERG, in domain, the probability is 0.15, the risk level medium, and the prediction set at 90 percent coverage is unambiguous. For DILI, out of domain, the probability is 0.90, also with an unambiguous prediction set.
Note what did *not* happen. The DILI probability is still 0.90. We do not hide it or round it away. What changes is the confidence, which collapses, and the risk call, which becomes "uncertain" rather than "high". A 0.90 out-of-domain DILI flag is a reason to look at the compound with a toxicologist. It is not a reason to kill the series.
Conformal prediction: the other half of the answer
Similarity tells you whether the model has seen this neighbourhood. Conformal prediction tells you how wide the answer has to be to keep a coverage guarantee. Our analysis returns a prediction set at a stated coverage level (0.90 above). The useful signal is the set size:
- one class in the set: one class survives at 90 percent coverage. The model is committing.
- both classes in the set: both survive. The model cannot separate them at that coverage level. This is the honest "I do not know", and it is a different failure from being out of domain.
A molecule can be in-domain with a two-class set (the model has seen this chemistry and it is genuinely ambiguous) or out-of-domain with a one-class set (a confident-looking answer with no evidential support). Read both. They are not redundant.
Why headline accuracy does not settle this
On a held-out, scaffold-split test set, hERG reaches an AUC of 0.856 on n = 7,500 test compounds and DILI an AUC of 0.909 on n = 72. The DILI number is higher. It is also the far weaker claim: 72 test compounds, and a training set narrow enough that a marketed EGFR inhibitor falls outside it. Any AUC quoted without a test-set size and a split type is decoration.
We report scaffold-split numbers as the headline because random splits leak. The gap is visible in our own data: Tox21 NR-AR is 0.830 scaffold-split versus 0.934 random-split, both on n = 1,453 test compounds. Same model, same endpoint. The 0.934 is what you get when analogues of your test compounds sit in the training set. The AD check is what tells you which of those two regimes your specific query is in.
Honest limits
What the applicability domain does not tell you:
- It does not validate the prediction. In-domain means the model has support here. It does not mean the model is right. hERG at 0.856 scaffold AUC still misclassifies compounds inside its domain.
- Similarity is not mechanism. Two compounds at Tanimoto 0.9 can sit on opposite sides of an activity cliff. Fingerprint similarity is a proxy for mechanistic relevance, not a substitute.
- Thresholds are conventions. A Tanimoto cutoff around 0.3 is a widely used working default, not a law of nature. Tighten it for regulatory-facing work.
- Coverage of the reference set matters. For some endpoints a full similarity reference set is available, for others it is not, and where it is missing the gate fails closed to out-of-domain. That is deliberate. An absent reference set must never read as "in domain".
- None of this is a regulatory filing. This analysis is triage and decision support. An ICH M7 or ICH S-series submission needs validated assays and expert review.
Frequently asked questions
What is the applicability domain in QSAR?
The applicability domain is the region of chemical space where a QSAR model's predictions are supported by its training data. It is OECD validation principle 3 for QSAR models. Outside that region, the model extrapolates and its reported accuracy no longer applies. A prediction without an applicability domain statement is not a validated result, and most regulatory reviewers will treat it that way.
How is the applicability domain calculated?
Three methods dominate. Nearest-neighbour similarity measures Tanimoto distance from the query to the closest training compound. Feature range checks whether each descriptor falls inside the training minimum and maximum. Leverage measures distance from the descriptor-space centroid against an h* cutoff. Our analysis combines nearest-neighbour similarity with descriptor coverage, and requires both to pass before treating a compound as in-domain.
What happens when a prediction is out of domain?
In our analysis it is treated as not decision-ready: the confidence is cut sharply and the risk level is set to "uncertain". The probability is still reported. For the EGFR inhibitor gefitinib, the DILI prediction stayed at 0.90 but counted as uncertain, because nearest-neighbour similarity to the DILI training set was 0.00.
How does conformal prediction differ from an applicability domain check?
They answer different questions. The applicability domain asks whether the model has seen this chemistry. Conformal prediction asks how wide the answer must be to hold a stated coverage guarantee, typically 90 percent. A prediction set containing both classes means the model cannot separate them, even in familiar chemistry. Read both: a compound can be in-domain and ambiguous, or out-of-domain and falsely decisive.
Can I use a high AUC as proof a model will work on my compounds?
No. AUC is measured on a specific test set with a specific split. In our analysis DILI reaches 0.909 AUC on only 72 test compounds, and hERG 0.856 on 7,500. The larger, lower number is the stronger claim. Always ask for the test-set size and whether the split was scaffold-based or random. Random splits inflate scores by leaking analogues into training.
What is a reasonable Tanimoto threshold for the applicability domain?
0.3 on Morgan-type fingerprints is a common working default in the literature. It is a convention, not a physical constant. For regulatory-facing or safety-critical decisions, raise it and require a tighter feature-coverage bound as well. Report the threshold you used alongside every prediction, because the same molecule flips domain status when the cutoff moves.
Related reading
Evaluation for this topic
We evaluate this for your lab.
- ICH M7, Tox21, CYP450, structural alerts, ADMET triage.