What Is Method Comparison in Quality Control and Why Does It Matter?

When a laboratory or production line introduces a new measurement system, the first question is not "which one is better" but "do these two methods agree closely enough to be used interchangeably?" Method comparison answers that question with statistics rather than opinion. It is a core practice in clinical chemistry, manufacturing, and any regulated industry where a new test method must be validated against a reference method.

What It Is

Method comparison is a set of statistical procedures used to evaluate the agreement between two measurement methods applied to the same samples. Unlike a simple correlation (which only shows that the methods move together), method comparison quantifies bias (systematic difference) and scatter (random difference) across the measuring range.

The most widely used approaches are:

• Bland–Altman analysis – plots the difference between paired measurements against their mean, and calculates the limits of agreement (LoA) as mean difference ± 1.96 × SD of differences.
• Deming regression – a linear regression that accounts for random error in both methods (unlike ordinary least squares, which assumes only one method has error).
• Passing–Bablok regression – a non-parametric alternative that is robust to outliers and does not assume normally distributed errors.

These methods are referenced in clinical guidelines and in ISO 5725 (Accuracy and precision of measurement methods and results), which provides the framework for assessing trueness and precision in method validation studies.

How It Works: Key Formulas and Steps

### 1. Bland–Altman Limits of Agreement

For \(n\) paired measurements \((x_i, y_i)\):

• Compute the difference \(d_i = y_i - x_i\) and the mean \(m_i = (x_i + y_i)/2\).
• Mean difference (bias): \(\bar{d} = \frac{1}{n}\sum d_i\).
• Standard deviation of differences: \(s_d\).
• Limits of agreement: \(\bar{d} \pm 1.96 \cdot s_d\) (for 95% coverage, assuming differences are approximately normally distributed).

Interpretation: If the LoA interval is within a pre-defined clinically or industrially acceptable range, the two methods can be used interchangeably.

### 2. Deming Regression

The Deming model assumes errors in both \(x\) and \(y\) with a known variance ratio \(\lambda = \sigma^2_\epsilon / \sigma^2_\delta\). The slope estimate is:

\[
\hat{b} = \frac{(s_{yy} - \lambda s_{xx}) + \sqrt{(s_{yy} - \lambda s_{xx})^2 + 4\lambda s_{xy}^2}}{2 s_{xy}}
\]

where \(s_{xx}\), \(s_{yy}\) are variances and \(s_{xy}\) is the covariance. The intercept is \(\hat{a} = \bar{y} - \hat{b}\bar{x}\). When \(\lambda = 1\), it is the "orthogonal" regression case.

### 3. ISO 5725 Reproducibility Context

ISO 5725-1 and 5725-2 define repeatability (same operator, same equipment, short time) and reproducibility (different laboratories or conditions). Method comparison studies often report the standard deviation of the differences between methods as an estimate of combined reproducibility, and check whether the observed bias is statistically significant using a paired \(t\)-test.

A Worked Illustrative Example

Example data (illustrative only): Ten patient samples were measured for glucose (mg/dL) by the new test strip (Method B) and the laboratory reference analyzer (Method A).

Sample | Method A | Method B | Difference (B−A)

• 1 | 85 | 88 | +3
• 2 | 92 | 90 | −2
• 3 | 105 | 108 | +3
• 4 | 118 | 120 | +2
• 5 | 130 | 133 | +3
• 6 | 142 | 140 | −2
• 7 | 155 | 158 | +3
• 8 | 168 | 170 | +2
• 9 | 180 | 183 | +3
• 10 | 195 | 198 | +3

• Mean difference \(\bar{d} = 1.8\) mg/dL.
• SD of differences \(s_d \approx 2.25\) mg/dL.
• Limits of agreement: \(1.8 \pm 1.96 \times 2.25 = 1.8 \pm 4.41\), i.e., from −2.61 to +6.21 mg/dL.

If the clinical acceptance criterion is ±10 mg/dL, the new method agrees sufficiently with the reference. The small positive bias (+1.8) suggests the test strip reads slightly high, but it is consistent and within tolerance.

Common Pitfalls

• Using correlation (r) instead of agreement – a high \(r\) does not mean the methods agree; it only shows a linear relationship.
• Ignoring heteroscedasticity – if the scatter increases with concentration, the simple LoA is misleading; consider log-transformation or proportional Bland–Altman plots.
• Applying ordinary least squares when both methods have error – this biases the slope toward zero; use Deming or Passing–Bablok.
• Not pre-defining acceptable limits – the LoA are meaningless without a clinical or industrial tolerance criterion.
• Small sample sizes – with fewer than ~40 pairs, the 1.96 multiplier should be replaced with the \(t\)-distribution critical value for the actual degrees of freedom.

---

Method comparison is a disciplined way to prove that a new measurement process is fit for purpose. Whether you are validating a glucometer, a torque gauge, or a chemical assay, the same statistical logic applies. To run Bland–Altman or Deming analyses quickly without manual spreadsheet errors, use the free interactive tool at https://www.6sq.com/tools/med_compare/ – it handles the calculations and plots so you can focus on the decision.