RRML Vol 18 Martie 2010 Calculating Uncertainty in Medical Laboratories
RRML Vol 18 Martie 2010 Calculating Uncertainty in Medical Laboratories
RRML Vol 18 Martie 2010 Calculating Uncertainty in Medical Laboratories
65
COURSE NOTES
A model for calculating measurement uncertainty in
medical laboratories
Model pentru calcularea incertitudinii de msurare n
laboratoarele medicale
Irina L. Dumitriu1, Bogdan Gurzu1, Simona M. Sltineanu1, Liliana Foia2,
Traian Mutiu3, Corina chiriac4, Mioara Achirecesei4, Maria Enea4
1. Department of Functional Sciences, Faculty of Medicine, Gr. T. Popa University of Medicine and
Pharmacy, Iasi
2. Department of Semiology and Prevention, Faculty of Dental Medicine, Gr. T. Popa University of
Medicine and Pharmacy, Iasi
3. Politehnica University of Timisoara
4. Medical Investigations PRAXIS Iai.
Abstract
Introduction: All medical laboratories that require recognition for competency assessment have to estimate the uncertainty of measurement of assay test results where relevant and possible (ISO 15189:2007
Medical laboratories - Particular requirements for quality and competence). The repeated quantitative examination of an analyte with the same method will offer more or less different results. This is happening because the
outcome of an assay depends not only upon the analyte itself, but also upon few error factors that could yield
doubts about the obtained result. The mathematical, quantitative expression of this doubt is known as uncertainty
of measurement (UM). Methods: It is the responsibility of each medical laboratory to identify all error sources
that can be quantified and converted in standard deviations that could be used to estimate the type A or B of uncertainty. In the case of Romanian medical laboratories, the European Accreditation (EA) accepted as reference
documents for UM estimation the Guide to the Expression of Uncertainty in Measurement (GUM) and Romanian
Standard SR ENV 13005. Discussion and conclusion: In this paper, authors present and discuss the modalities
of UM estimation in two different situations: when the used reference materials (calibrators) are or are not
traceable to certified reference materials (CRM). Complete and informative UM reporting can only lead to better
decisions in healthcare.
Keywords: precision, accuracy, uncertainty of measurement, calibrator.
Rezumat
Introducere. Toate laboratoarele medicale care doresc recunoatere de competen trebuie s-i estimeze incertitudinea de msurare acolo unde este relevant i posibil (SR EN ISO 15189, Standard Romn, Laboratoare medicale, Cerine particulare pentru calitate i competen, 2007). n cazul unor msurari repetate
ale unui analit obinem rezultate diferite, mai mult sau mai puin apropiate ntre ele, dei valoarea analitului
66
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
este aceeai. Valoarea unui rezultat msurat nu depinde numai de valoarea nsi, ci i de o serie de factori de
eroare, care aduc o nencredere, un dubiu asupra rezultatului obinut. Exprimarea matematic, cantitativ a
acestei nencrederi se numete incertitudine de msurare (UM). Material i metod. Este responsabilitatea fiecrui laborator s-i indentifice toate sursele de eroare care pot fi cuantificate i convertite n deviaii standard
pe baza crora s-i estimeze UM de tip A i de tip B. Pentru laboratoarele medicale din Romnia, European
Acreditation (EA) recunoate ca documente de baz pentru estimarea UM, Guide to the Expression of Uncertainty in Measurement (GUM) i Standardul Romn SR ENV 13005 (Ghid pentru exprimarea incertitudinii de
msurare). Discuii i concluzii. Autorii prezint modalitile de estimare a UM n laboratoarele medicale din
ara noastr cnd calibratorii utilizai sunt trasabili sau nu la materiale de referin certificate (MRC). Raportarea corect i complet a UM influeneaz decizia terapeutic.
Cuvinte-cheie: precizie, acuratee, incertitudine de msurare, calibrator.
All medical laboratories that require recognition for competency assessment have to estimate the uncertainty of measurement of assay
test results where relevant and possible (1).
The term uncertainty means a doubt
and uncertainty of measurement (UM) is a
doubt on the validity of outcome measurements.
UM is only applicable to results of numerical
quantitative measurement.
According to GUM the result of a
measurement is only an approximation or estim-
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
67
X =
xi
x =1
n
n
(Xi X )2
SD = i = 1
n 1
SD
CV =
100 , where
X
X = average of measured values
Xi = individual measured values
SD = standard deviation
CV = coefficient of variation
n = number of measurements
68
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
(x1 x ) + (x 2 x )
2
Equation [2]:
SD =
+ ... + x x
n
n 1
n
2
Xi X
, where
= i =1
n 1
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
of uncertainty, known as the combined standard uncertainty (UMc). UMc is given by the
square root of the sum of the squares:
UM c = U2 + U2B [3].
A
Expanded uncertainty (U) is calculated by multiplying the standard uncertainty
UMc with an appropriate coverage factor k:
U = k UMc [4], for a desired level
of confidence of 95%, k is approximately 2.
Uncertainty reported with the result is
expanded uncertainty and is then conveniently
expressed as:
Y U [5], where:
Y = measured result,
U = expanded uncertainty.
The Guide to the expression of uncertainty in measurement (GUM) (2) was developed for estimating UM in all laboratories
types. In non-medical laboratories the potential
sources of uncertainty are usually readily identifiable, quantifiable and converted in standard
deviations in fields such as physical and chemical measurements (e.g. electrical, materials, optics, etc). These standard deviations are used for
calculation of UMc.
The Working Group for the Medical
Testing Laboratory recognizes that the implementation of the uncertainty of measurement requirement offers opportunities for pathology
laboratories to add value to their diagnostic services, particularly in educating users to better
understand the limitations of tests, and in recognizing when clinically significant changes in patient results have or have not occurred. ISO
15189 also recognizes that the rigor of estimating uncertainty of measurement may be based
on the needs of the client.
In medical testing there are many potential uncertainties that can significantly affect
test results (for example: poor specimen collection or transport, patient related factors such as
69
70
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
ibrator from the specifications of the manufacturer. This process it is called calibration and
represents a set of operations that establish, under specified conditions, the relationship
between values of quantities indicated by a
measuring instrument or measuring system, or
values represented by a material measure or a
reference material, and the corresponding values
realized by standards (3). Therefore, the result
y of creatinine from patient sample could be calculated by rule of three:
2500 OD................. 3.6 (calibrator value)
1800 OD.................. y
y=
ODsample
ODcalibrator
valuecalibrator [6],
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
The results obtained by measuring control materials are analyzed considering Westgard rules and the acceptance / rejection criteria
of each laboratory. Internal control tests with
control materials are similar to diagnostic tests
of patient samples. Similarly with diagnostic
tests in patients, which identify the health problems of the patient, the tests of internal control
identify the health problems (errors) of the
measurement system (10). Rapid detection and
adequate treatment is depending upon precise
and quick identification of the health
problem.
The reference materials (calibrators, etalons, standards) are considered Xi quantities
upon which final test result depends, while control material is used for verifying and validating
the correctness of calibration but does not have
a direct influence on final result.
4. Estimation of a value for all input quantities
The result of a quantitative measurement Y is a functional relationship f (Xi), where
Xi represents all the factors upon which the
measurement depends (X1, X2, X3 ... Xn) (4, 7).
All relevant sources of uncertainties that could
71
72
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
Equation [8]:
2
2
2
2
UM c = U measurementprocedure + Ucalibrator + UBias(external control) ... + Uother factors .
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
73
Figure 6. Combined standard uncertainty comprises type A standard uncertainty (UA, precision of
measurements) and type B standard uncertainty (UB, accuracy of measurements).
UA is statistically calculated from values of independent measurements with Gaussian distribution. Regarding
type B uncertainty, there are many kinds of UB according to literature specifications: UB1, UB2, .. ,UBn. When a
medical laboratory chooses to estimate its accuracy of measurements from Bias intercomparisons, the standard
deviation of Bias (Bias values from many participations to external control have rectangular distribution) has to
be calculated following the formula SD Bias =
a
, where a = Bias (adapted after SR ENV 13005:2005).
3
be converted in standard deviations used for estimating type A and type B standard uncertainty.
Type A standard uncertainty is assessed
from internal quality control data (statistical calculation on at least 30 values of control serum
from at least 30 different consecutive days).
Type A standard uncertainty illustrates the precision of laboratory measurements.
When the laboratory uses traceable calibrators to certified reference materials (CRM)
with attributed UM, this can be treated as type B
standard uncertainty. Type B standard uncertainty shows the accuracy of measurements.
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
74
Table 1. Estimation of UM when the medical laboratory has data about UMcalibrator
Quantity
Measurement
Units
Reference intervals
Test principle
Calibrator
traceability
Precision /
Imprecision
(Type A uncertainty)
CREATININE
Creatinine concentration in human serum
mg/dL,
mol/L
Men 0,6 1,1 mg/dl
Women 0,5 0,9 mg/dl
53 97
mol/L
44 80
mol/L
Creatinine in alkaline solution reacts with picrate to form a colored complex. The rate of formation of
the complex is measured.
Is traceable to the NIST reference material 909b level 2 and by the reference method IC-GC/MS
Internal quality control data on multiparameter control serum for: 1.01.2009 31.03.2009
Number of measurements
72
SD
0.17 mg/dl
CV (%)
4.84 %
Calibrator traceability
Uncertainty of
calibrator
(Type B uncertainty)
UMcombined (UMc)
UA
UB
Calibrator
Target value
Creatinine
3,6 mg/dl
Total Uncertainty
(Uexpanded)
0,26 mg/dl
Uncertainty is calculated as the half of the 95% confidence interval of the calibrator assigned value. The
true value should fall in the range (assigned valueuncertainty) with 95% probability.
The medical laboratory is responsable for taking account other types
2
2
UB from apriori data beside UMcalibrator.
UM c = U + U
Umeasurement procedure = standard deviation of quality control results for period 1.01.2009 31.03.2009
UA = 0,17 mg/dL
UMcombined (calibrator) obtained from manufacturer as standard deviation. The manufacturer offers
UMexpanded (for a level of confidence of 95%, k=2), so that the laboratory should divide by 2 the
UMexpanded offered by manufacturer.
UM
expanded 0, 26
UB = UMcombined (calibrator) =
=
= 0,13 mg/dL
k
2
UMc = UMcombined
2
2
UMc = SD (precision
from internal control) + UM combined (calibrator) =
= 0,17 2 + 0,132 = 0,30 2 = 0,30
UMc
Reporting UM
Note:
the traceability of calibrators to CRM provides the accuracy of measurements;
in this case, the usage of other types of UB is supplementary, but not mandatory because the traceability of calibrators to CRM covers the accuracy of measurements (Figure 6);
the proportion brought by other types of UB to the UMc is lower. The medical laboratory could
choose whether to take into account the Ubias (from external control) beside U calibrator for calculation of UMc.
U = UMexpanded = UMc k = 0,30 2 = 0,60
k =2, for level of confidence of 95%
Y U, where
Y = measured result
U = expanded uncertainty
Mean value of creatinine 3.55 0,60 mg/dl
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
Unfortunately, in most medical laboratories from our country calibrators without traceable values are used. If an appropriate reference
material or reference procedure is unavailable,
then alternative approaches may be used, e.g.
external quality assessment data or inter-laboratory comparisons (13).
In this case, accuracy of measurements
is estimated by type B uncertainty calculated as
standard deviation of Bias values from external
control. Bias is the difference between measured result and average results of participant
medical laboratories based on an intercomparisons scheme (external control). Bias values from
many external control participations have a rectangular distribution (variance) given by:
(max min )2 = 2a 2 = 4a 2 [11] (6).
U 2 (x ) =
i
12
12
12
For this reason, standard deviation of
Bias is calculated by formula:
( )
U = SD =
4a 2
=
12
4a 2
2a
a
=
=
43 2 3
3
[12],
(Figure 6).
In this paper the authors summarize the
estimation of UM in two situations: when the
assigned values of traceable calibrators to CRM
are or not accompanied by UM as shown in
Tables 1 and 2.
The assigned values of traceable calibrators to CRM are accompanied by UM as
shown in Table 1.
Manufacturers do not always provide
the uncertainty of the values assigned to calibrators. In this case UM is estimated by following the model shown in Table 2.
Conclusions
1. Uncertainty Measurement of a measured
result increases its reliability.
2. A reasonable estimation of UM includes precision (UA) and accuracy of measurements (UB):
75
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
76
Table 2. Estimation of UM when the medical laboratory does not have data about UMcalibrator
Quantity
Measurement
Units
Reference intervals
Test principle
Calibrator
traceability
Precision /
Imprecision
(Type A uncertainty)
SD Bias =
Type B uncertainty
(Figure 6)
SD Bias =
BIAS max .
0,17
= 0,09 UI/ml
UMc = UMcombined
2
2
UM c = U A + U B
UMc
2
2
UMc = SD (precision from internal control) + SD Bias
=
(external control)
=
Reporting UM
2
2
UI/ml
0.32 + 0,09 = 0,33
Note: The medical laboratory is responsible for taking into account other UB types from apriori
data beside UMBias (from external control)
U = UMexpanded = UMc k = 0,33 2 = 0,66 UI/ml
k =2, for level of confidence of 95%
Y U, where
Y = measured result
U = expanded uncertainty
Mean value of TSH 4.03 0,66 UI/ml
7. Assessing and reporting measurement uncertainty will help reduce the differences
between laboratories results, which translates
into prompt clinical decision, low costs, higher
efficiency and highest confidence of clinicians
and patients in testing results.
Acknowledgements
This study was supported by Medical
Investigations Praxis S.R.L., Iasi, Romania. and
the National University Research Council
CNCSIS, grant PN_2_ID_PCE_2670/2008.
Revista Romn de Medicin de Laborator Vol. 18, Nr. 1/4, Martie 2010
Abbreviations list
CRM
CV
EA
GUM
OD
SD
UM
UMc
UMexpanded, U
References
1. International Organization of Standardization, ISO/TC
212. ISO 15189:2007, Medical laboratories - Particular requirements for quality and competence, 2nd Edition.
Geneva, 2007: p37
2. ISO/IEC Guide 98-3:2008. Uncertainty of measurement
- Part 3: Guide to the expression of uncertainty in measurement (GUM:1995). International Organization for Standardization (ISO)/ International Electrotechnical Commission
(IEC): Geneva, 2008: p 3-58
3. BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, OIML,
The International Vocabulary of Metrology, Basic and General Concepts and Associated Terms - VIM, Joint Committee
for Guides in Metrology- JCGM 200:2008 [ISO/IEC Guide
77
99]: p 22-28
4. Eurachem / Citac Guide CG4, Quantifying Uncertainty
in Analytical Measurement, Second Edition, 2001: p16-30
5. Technical Report EA- 4/16 (rev00). Guidelines on the
expression of uncertainty in quantitative testing. European
Co-operation for Accreditation (EA), 2003: p 6-16
6. SR ENV 13005:2005. Romanian Standard for estimation of measurement uncertainty, Third Edition; 2005: p 8-18
7. White GH, Farrance I. Uncertainty of measurement in
quantitative medical testing: a laboratory implementation
guide. Clin Biochem Rev. 2004;25(4):S1-S24
8. Hibbert B. Quality assurance for the analytical chemistry laboratory. Oxford University Press US, 2007: p 255
9. Taylor BN, Kuyatt KE. Guidelines for Evaluating and
Expressing the Uncertainty of NIST Measurement Result National Institute of Standards and Technology Technical
Note 1297. U.S. Government Printing Office, Washington
DC, 1994: p 2-6
10. Are there similar performance characteristics for QC
and diagnostic tests? "Westgard Rules" and Multirules.
Westgard QC (n.d.) Retrieved November 01, 2009 from
http://www.westgard.com/mltirule.htm#performance
11. European Diagnostic Manufactures Association
(EDMA). Estimation of uncertainty of measurement in medical laboratories, 2006: p 2-5
12. The Romanian Accreditation Association (RENAR).
Instruction for validation of methods used in medical laboratories, RENAR Code: I-12, First Edition, 2007: p 1-9
13. White GH. Basics of estimating measurement uncertainty. Clin Biochem Rev. 2008 Aug;29 Suppl 1:S53-60