Every optimisation campaign starts with the same decision, usually made out of habit: vary one factor at a time, run a designed experiment, or let each round of experiments be planned from all the results so far. We wanted numbers instead of habit, so we compared the three under rules that are fair to each method.
How we compared them
We simulated 5,040 complete campaigns on reaction and process models with 2 to 15 factors, 40 instances per setting, the same measurement noise and the same budget of 60 experiments for every strategy. For classical DoE we used the rule a careful team would use: run the design, run the model optimum once as a confirmation, and keep whichever condition measured best. On top, we ran every strategy on three published reaction screens in which the lab had measured every condition (a C-N coupling, a Suzuki coupling and a C-H arylation).
All of this is simulation on models and public data. No new experiments, no customer data.
Result 1: up to 4 factors, classical DoE is hard to beat
With 2 to 4 factors, a classical response-surface design was as fast as the adaptive approach or faster. That matches what experienced DoE users see: a small, well-designed experiment covers a small space efficiently, and it gives you an interpretable model on top.
If your problem has four factors or fewer, there is little reason to change what works.
Result 2: from about 8 factors, the picture turns
With 8 to 10 factors, the adaptive approach reached the result of the DoE plan with 1.3 to 3.1 times fewer experiments than both DoE plans we compared. On the public screens it was 1.7 times faster on the C-N coupling, 1.5 times faster on the Suzuki coupling, and tied on the C-H arylation.
At 12 to 15 factors it was on par with classical DoE and 1.4 to 2.0 times faster than definitive screening. The advantage does not keep growing with every added factor, and we would not claim that it does.
Result 3: the worst case matters more than the best case
Across 42 comparisons at equal budget, the adaptive approach was never more than 8.6 percentage points behind the best strategy. For OFAT the worst gap was 11.1, for random picking 13.8 and for the DoE plans 19.3. If you do not know in advance how difficult your problem is, the strategy with the smallest worst case is the safer default.
Where DoE stays the reference
An optimum is not a filing. When you need a documented design space, robustness evidence or a model a reviewer can read, a designed experiment remains the tool. A practical sequence is to locate the optimum adaptively and then confirm and map the region around it with a compact local design.
What the study does not show
- Simulated campaigns on models and public screens, not a laboratory study.
- The public screens have categorical factors only.
- Results hold for the strategies and settings we tested. Other DoE packages, settings or adaptive methods can behave differently.
The full study
The 24-page whitepaper contains the methods, every number with its confidence interval, a section on how many experiments to run per round, five case studies and a decision guide.
FAQ
Is OFAT ever the right choice?
On the public screens OFAT matched a main-effects DoE, and its worst case in the model study (11.1 percentage points behind the best strategy) was smaller than that of the DoE plans (19.3). It stays useful for quick checks of a single factor around a known good condition.
How many factors make a problem too large for classical DoE?
In our study the turning point was around 8 factors. Below 4, classical designs won or tied. Between 4 and 8 the answer depends on the model and the budget.
Does adaptive design replace DoE for regulatory work?
No. For a design space in a filing, a designed experiment remains the reference. Adaptive design helps you find where to put that design.