For a method transfer or a comparability study, the question is not whether two labs or two methods give different results, but whether the difference is small enough not to matter. A t-test answers the first question; the two one-sided tests procedure (TOST) answers the second. With TOST you define an acceptance margin in advance, for example ±2.0 % for an assay, compute the 90 % confidence interval of the mean difference, and conclude equivalence if the whole interval lies inside the margin. USP <1224> on method transfer and USP <1210> on statistical tools describe exactly this approach.
Why is the t-test the wrong question?
A t-test starts from the hypothesis "no difference" and looks for evidence against it. Two things go wrong when it is used for transfer:
- Precise labs are punished. With tight data, a difference of 0.5 % becomes statistically significant, although it is analytically irrelevant.
- Sloppy studies pass. With few determinations and wide scatter, nothing is significant, and "no significant difference" is read as "comparable". Absence of evidence becomes evidence of absence.
Both errors appear in the two examples below.
How does TOST work?
1. Fix the margin before the study. The equivalence margin ±θ is a scientific decision: the largest difference that would not change any decision taken on the result. For an assay with a specification of 95.0 to 105.0 %, ±2.0 % is common; for impurities a margin relative to the reporting threshold is used. 2. Run the study. Typically the same homogeneous batches analysed in the sending and receiving lab, with replicates on several days or by several analysts. 3. Compute the 90 % confidence interval of the mean difference (receiving minus sending). The 90 % comes from two one-sided tests, each at 5 %. 4. Decide. If the entire interval lies inside −θ to +θ, equivalence is shown. If any part lies outside, it is not shown, regardless of where the mean is.
Example 1: significant, but equivalent
Assay of one batch, six determinations in each lab, margin ±2.0 %.
| Sending lab | Receiving lab | |
|---|---|---|
| Results (%) | 99.6, 100.1, 99.3, 100.4, 99.8, 100.0 | 99.0, 99.5, 98.7, 99.8, 99.1, 99.3 |
| Mean (%) | 99.87 | 99.23 |
| Standard deviation (%) | 0.39 | 0.39 |
Mean difference −0.63 %, pooled standard deviation 0.39 %, standard error 0.22 %. With t(0.95; 10) = 1.812, the 90 % confidence interval is −1.04 % to −0.23 %.
A t-test gives p = 0.018: "significantly different". TOST says: the whole interval lies inside ±2.0 %, so the labs are equivalent for this purpose. The receiving lab reads about 0.6 % lower, which is worth understanding (reference standard handling, integration), but it is not a reason to fail the transfer.
Example 2: not significant, but not proven
Same margin, but only three determinations per lab and more scatter.
| Sending lab | Receiving lab | |
|---|---|---|
| Results (%) | 99.2, 100.6, 98.9 | 100.4, 98.6, 101.2 |
| Mean (%) | 99.57 | 100.07 |
Mean difference +0.50 %, pooled standard deviation 1.14 %, 90 % confidence interval −1.48 % to +2.48 %.
A t-test gives p = 0.62: "no significant difference". TOST says: the interval reaches beyond +2.0 %, so equivalence is not shown. The study is too small and too noisy to support the claim. More determinations, or finding the source of the scatter first, is the way forward.
How do you choose the margin?
The margin is where most transfer protocols are weakest. Three anchors help:
- Specification width. The margin should leave room for process variability inside the specification. A margin of ±2.0 % on a ±5.0 % specification is defensible; ±4.0 % is not.
- Method capability. The margin must be achievable with the method's intermediate precision and a reasonable number of determinations. A sample size calculation before the study shows whether it is.
- Decision impact. If a shift of the size of the margin would change a release decision or a stability conclusion, the margin is too wide.
Write the margin, its justification and the sample size into the protocol before data exist. A margin chosen after seeing the data is not a test.
Beyond two labs: comparability after a change
The same logic applies when the question is a method change, a new instrument platform or a process change: define what "the same" means quantitatively, then show that the confidence interval of the difference lies inside that definition. For several batches or levels, equivalence is tested per level or with a mixed model; for slopes, such as stability trends, the comparison is on the slope difference.
Limits of the statement
- TOST shows that the mean difference is small. It does not show that precision is equivalent; that needs a separate comparison of variances if it matters.
- Equivalence holds for the samples and range tested. Extrapolation to other strengths or impurity levels needs its own justification.
- The result is only as good as the margin. A statistically correct test against an unjustified margin is still not evidence.
Frequently asked questions
What is the TOST equivalence test?
Two one-sided t-tests that check whether the mean difference is greater than −θ and smaller than +θ. In practice this is the same as checking whether the 90 % confidence interval of the difference lies entirely inside the margin ±θ.
Why 90 % and not 95 % confidence?
Each of the two one-sided tests is performed at a 5 % significance level. Together they correspond to a two-sided 90 % confidence interval, while keeping the overall error of falsely claiming equivalence at 5 %.
What acceptance criterion is typical for an assay method transfer?
A mean difference within ±2.0 % absolute is common for drug substance and product assay, combined with criteria on precision. Impurity methods often use margins relative to the reporting threshold or the specification limit.
Can a transfer pass if the t-test shows a significant difference?
Yes. Statistical significance only says the difference is unlikely to be zero. If the confidence interval lies inside the equivalence margin, the difference is too small to matter and the transfer passes.
Which guidance describes equivalence testing for method transfer?
USP <1224> covers transfer of analytical procedures and USP <1210> statistical tools including equivalence testing. The ISPE good practice guide on technology transfer and ICH Q14 for the lifecycle context are also relevant.