A model trained on your own data beats a public model when your chemistry is not in the public data, when your assay differs systematically from the public one, when your target property is not measured publicly at all, or when relative ranking inside a narrow series matters more than absolute accuracy. It does not beat a public model when your dataset is small and chemically diverse, when your measurements are noisier than the effect you want to predict, or when the public model already covers your space well. Which case you are in is a question with a cheap answer, and it is worth answering before anyone trains anything.
The four conditions where in-house data win
1. Your chemical space is under-represented publicly.
Public chemistry datasets are dominated by what gets published and submitted: drug-like small molecules, common scaffolds, standard conditions. If your work is in organometallics, polymers, agrochemical scaffolds, process impurities, specialty surfactants or an unusual heterocycle series, a public model is extrapolating on every one of your compounds while your own 400 measurements are all inside the space you care about.
2. Your assay differs systematically.
The same nominal property measured with a different protocol, buffer, temperature or readout gives a shifted value. A public model learns the published protocol. If your results differ from it by a consistent offset, a model on your data learns your number, which is the one your decisions use.
3. The property is not publicly measured.
In-house endpoints, formulation stability under your own stress protocol, yield of your own reaction type, a sensory or application property specific to your products. There is no public model to beat; the only options are your data or nothing.
4. You need ranking, not absolute values.
Inside a congeneric series, a model with a systematic bias can still order candidates correctly. Trained on your series, it can be very good at exactly the decision you make, which is "which of these ten do we make next", without being accurate in an absolute sense.
The three conditions where they do not
1. Small and diverse.
A hundred compounds spread across unrelated scaffolds teach a model almost nothing generalisable. The same hundred inside one series can be enough. Diversity helps a model only when the data volume supports it.
2. Measurement noise above the effect.
If your assay's repeatability is 0.5 log units and the differences you want to resolve are 0.3 log units, no model can resolve them. The ceiling for any model is set by the noise in the labels. Measure your own repeatability before you measure a model.
3. The public model already covers you.
If your compounds sit inside the applicability domain of a good public model and your assay matches the published one, a custom model is work without a benefit. Fine-tuning may still buy a small gain; training from scratch almost certainly will not.
The comparison that answers the question
Before any training project, run this:
1. Take 50 to 100 of your own measurements spanning the range you care about, including the extremes. 2. Predict them with the public model and record the error distribution, not just the mean error. A model with low mean error and two catastrophic misses may be unusable for your decision. 3. Check domain coverage. How many of your compounds does the public model flag as inside its applicability domain? If most are outside, that is your answer. 4. Measure your own label noise. Replicate measurements of the same sample, by different analysts or on different days. This is the floor no model can go below. 5. Then decide. If the public error is well above your label noise and your compounds are outside its domain, in-house training has room to work. If the public error is already close to your label noise, it does not.
This costs a few days and prevents the common failure mode, which is a six-month modelling project that ends at the accuracy the public model already delivered.
Fine-tuning before training from scratch
Between "use the public model" and "train your own" sits the option that is usually best: start from a model trained on public data and adapt it with your measurements. The public data carry the general chemistry, your data carry your series and your assay. This typically needs an order of magnitude fewer in-house measurements than training from scratch, and it degrades gracefully: on compounds unlike your training set it falls back towards the general behaviour instead of producing nonsense.
The practical requirements are the same either way: your data in a consistent format, one documented protocol per endpoint, the failures and the inactive results included, and a split that does not let near-duplicates into both sides.
What to keep in mind about your own data as an asset
Measurements you already own are the one input a competitor cannot buy. Three habits protect that value: record the negative and failed results, because a model learns from them and a lab notebook usually does not keep them; keep the protocol version with every measurement, because a protocol change is a different label; and keep raw values rather than only pass or fail conclusions, because a threshold discards most of the information.
Limits of the statement
- An in-house model inherits your data's blind spots. It will be confident and wrong on chemistry you have never measured, which is exactly where new projects go.
- Performance figures from a custom model are only meaningful with the split rule stated. A random split on a congeneric series will report an accuracy you will never see in use.
- A model trained on customer or partner data raises ownership and confidentiality questions that belong in the contract before the first file moves.
- Maintenance is part of the cost. A model that is not retrained as the chemistry moves becomes a historical artefact with a user interface.
Frequently asked questions
When is it worth training a model on your own data?
When your chemistry is under-represented in public data, your assay differs systematically from the published protocol, the property is not publicly measured, or you need reliable ranking inside a narrow series. Each of these is testable with a short comparison before the project starts.
How much data does it take to beat a public model?
It depends on diversity more than on count. A few hundred measurements inside one chemical series can be enough with fine-tuning; a few hundred spread across unrelated scaffolds usually are not. The comparison described above tells you more than any rule of thumb.
Does assay noise limit model accuracy?
Yes, directly. A model cannot predict a label more precisely than the label is measured. Replicate measurements of the same samples give you that floor, and it should be known before you set an accuracy target.
Is fine-tuning better than training from scratch?
Usually, for chemistry. A model pretrained on public data brings general behaviour that your dataset is too small to learn, and your data specialise it to your series and your protocol. It also fails more gracefully outside your data.
What makes in-house data useful for modelling?
Consistent protocols with recorded versions, raw values rather than pass or fail, inclusion of negative and failed results, machine-readable structures, and a record of which measurements belong to which series so a sensible split can be made.