Estimating precision > Verifying the precision of a measurement proc…the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.

15 documentation 1/04/25 11:18

Tutorials > Verifying the precision of a measurement procedure against a performance

claim and estimating the bias (CLSI EP15-A3)

Estimating precision

Estimate the precision of the measurement procedure.

1. Open the file tutorials\EP15-A3.xlsx.

The worksheet opens showing 3 columns. The Sample column identifies the levels of
the analyte in the sample (3 levels). Run identifies the day number of each run (5 runs).
Ferritin (ug/L) identifies the measured value for 3 replicates of sample in each run.

2. Click a cell in the dataset.

3. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Precision drop-
down, and then click 1 Factor.

The analysis task pane opens.

4. In the Y drop-down list, select Ferritin .

5. In the Factor A drop-down list, select Run .

6. In the By drop-down list, select Sample .

7. In the Estimator drop-down list, select SD.

8. Clear the Detailed components check box.

9. On the Analyse-it ribbon tab, in the MSA group, click Variability plot.

10. On the Analyse-it ribbon tab, in the MSA group, click Identify Outliers.

11. Click Calculate.

The analysis report opens.

The variability plots show a simple visual assessment of the closeness of agreement
between the measured quantity values. The purple lines show the mean of each run and

https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/estimating-precision Page 1 sur 3

Estimating precision > Verifying the precision of a measurement proc…the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.15 documentation 1/04/25 11:18

the blue line the overall grand mean, allowing you to see the variation between and within
each run.

You should observe the scatter of the points to ensure there are no obvious problems. If a
result appears spurious then you should investigate and correct any mistakes. If the data
appear to be unusable, stop the evaluation and begin an expanded evaluation of the

https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/estimating-precision Page 2 sur 3

Estimating precision > Verifying the precision of a measurement proc…the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.15 documentation 1/04/25 11:18

sources of measurement error or contact the manufacturer. We note that one observation
in Sample 1, Run 1 is highlighted as a potential outlier but do not know the reason for the
aberrant result. We will continue with this observation included in the analysis and return
to it later.

The abbreviated variance components table shows precision expressed numerically as the
standard deviation (SD) and coefficient of variation (CV).

Next topic: Testing precision against a performance claim

https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/estimating-precision Page 3 sur 3

Testing precision against a performance claim > Verifying the precision o…ting the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.15 documentation 1/04/25 11:12

Tutorials > Verifying the precision of a measurement procedure against a performance

claim and estimating the bias (CLSI EP15-A3)

Testing precision against a performance

claim

Test if the precision meets the manufacturer's performance claim.

You should use this procedure when you already have a performance claim from a
manufacturer's package insert and you want to test whether your precision is significantly
greater than the claim. It is possible for the imprecision from a study to be greater than
the manufacturer's claim due to the chance alone. This procedure ensures that the
manufacturer's claims are only falsely rejected 5% of the time when they are in fact true.

1. On the Analyse-it ribbon tab, in the Precision group, click Test Claim.

The hypothesis test section on the Precision panel opens.

2. In the Hypotheses grid, under the Repeatability SD/CV column, type the performance
claims as an absolute value (for example, type 5 for a SD of 5 mg/dL) or relative value
(for example, type 5% for a CV of 5%).

Level Repeatability CV

1 3.3%

2 2.0%

3 1.6%

3. In the Hypotheses grid, under the Laboratory SD/CV column type the performance
claims as an absolute(for example, type 5 for a SD of 5 mg/dL) or relative value (for
example, type 5% for a CV of 5%).

Level Laboratory CV

1 5.3%

2 3.4%

3 2.8%

https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/testing-precision Page 1 sur 3

Testing precision against a performance claim > Verifying the precision o…ting the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.15 documentation 1/04/25 11:12

4. In the Significance level edit box, type 5%, and then select the Familywise error rate
check box.

CLSI EP15 uses a familywise significance level so the overall significance level is a
maximum of 5% regardless of the number of levels (for example, for 1 level the
significance level is 5%, for 2 levels the significance level is 5%/2 = 2.5% for each level,
for 3 levels the significance level is 5%/3 = 1.6% for each level).

5. Click Recalculate.

The analysis report updates.

The detailed variance components table shows the observed and expected SD/CV along
with the hypothesis test p-value for each level.

Sample1

Expected
Component SD CV SD/CV p-value
Repeatability 1.15 4.5% 3.3% 0.01191
BetweenRun 0.76 3.0%
5.4% 0.41992
https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/testing-precision Page 2 sur 3
Testing precision against a performance claim > Verifying the precision o…ting the bias (CLSI EP15-A3) > Tutorials | Analyse-it® 6.15 documentation 1/04/25 11:12

WithinLaboratory 1.38 5.4% 5.3% 0.41992

HO:o≤0o
Theimprecisionislessthanorequaltotheexpectedimprecision.
H1:0>00
Theimprecisionisgreaterthantheexpectedimprecision.
*Rejectthenullhypothesisinfavorofthealternativeatthe5%familywise(1.67%individual)significancelevel.
*Donotrejectthenullhypothesisatthe5%familywise(1.67%individual)significancelevel.

Sample2

Expected
Component SD CV SD/CV p-value
Repeatability 1.78 1.3% 2.0% 0.99151
BetweenRun 1.59 1.1%
WithinLaboratory 2.39 1.7% 3.4% 0.99351

HO:0≤0o
Theimprecisionislessthanorequaltotheexpectedimprecision.
H1:0>0o
Theimprecisionisgreaterthantheexpectedimprecision.
*Donotrejectthenullhypothesisatthe5%familywise(1.67%individual)significancelevel.

Sample3

Expected
Component SD CV| SD/CV p-value
Repeatability 10.65 1.7% 1.6% 0.29581
BetweenRun 10.13 1.6%
WithinLaboratory 14.70 2.4% 2.8% 0.72971

HO:0≤0o
Theimprecisionislessthanorequaltotheexpectedimprecision.
H1:0>0o
Theimprecisionisgreaterthantheexpectedimprecision.
*Donotrejectthenullhypothesisatthe5%familywise(1.67%individual)significancelevel.

All the hypothesis tests are not significant and highlighted green in this example. If any
are significant they are highlighted in red and you should contact the manufacturer for
further assistance in diagnosing the problem. If the hypothesis test is not statistically
significant but the imprecision estimate is much larger than the claim, you may want to
repeat the study with more data to be able to detect smaller departures from the claim.

Next topic: Identifying and excluding outliers

https://analyse-it.com/docs/tutorials/measurement-systems-analysis/ep15/testing-precision Page 3 sur 3