European Federation of National Associations of

Measurement, Testing and Analytical Laboratories

7HFKQLFDO5HSRUW1R


7HFKQLFDO 5HSRUW
-XQH













0HDVXUHPHQW8QFHUWDLQW\


LQ7HVWLQJ


A short introduction on how to characterise



accuracy and reliability of results

including a list of useful references
 This short introduction to measurement uncertainty and its implementation into the
laboratory as requirement for accreditation according to ISO/IEC 17025 is in-
tended to provide help for the inexperienced rather than the expert and therefore
necessarily simplifies some topics.

The document is in the progress state and it is intended, to add further examples
from non-chemical fields of testing. Proposals for such examples are very wel-
come.

We would like express our thanks to Adriaan M H. van der Veen, Christian Ran-
son, Matthias Rößlein, Michele Desenfant, Tomas Quintana and Vitor Ramos for
their very helpful comments. We are grateful to Holger Frenz for Example 2,
Joachim Abshagen and Janusz Morkowski for permission to present Example 3
and Matthias Rößlein for permission to use Example 4.

June 2002

EUROLAB Technical Secretariat

c/o BAM, Unter den Eichen 87
12205 Berlin, Germany
Tel.: +49-30-8104-3762
Fax: +49-30-8104-4628
e-mail: [email protected]

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 2 - 27

Measurement Uncertainty in Testing
A short introduction on how to characterise
accuracy and reliability of results
including a list of useful references

Contents:

1. Some aspects of measurement uncertainty 4

2. Determination of measurement uncertainty -

ways for estimating uncertainties in practice 5
Estimating uncertainties in practice 6
How to use existing quality assurance data 6
Aim: fit for purpose 8

3. The mathematical analytical way

8 steps to obtain the measurement uncertainty 8
The single steps are carried out as follows 8

4. Examples for estimating measurement uncertainty 12

Example 1: Chemical Testing: Ion-Chromatorgaphy 12
Example 2: Mechanical Testing: Hardness according to Brinell 13
Example 3: Determination of emission measurement and sampling
uncertainties estimated by well experienced experts 16
Example 4: Determining the measurement uncertainty in preparing a
calibration standard by the mathematical analytical approach 18

5. Conclusions 21

6. Compilation of the main and some specific literature on

measurement uncertainty 23

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 3 - 27

1. Some aspects of measurement uncertainty

What is measurement uncertainty ?

• Measurement results are never exact, nor absolutely free of doubts. Therefore
the “measurement uncertainty” is part of the result of a measurement. It is a
measure for the accuracy of the result.
• Measurement uncertainty is derived from standard deviations.
• Definition: Measurement uncertainty is ”A parameter associated with the result of
a measurement, that characterises the dispersion of the values that could rea-
1
sonably be attributed to the measurand” (VIM and GUM [1])
Who needs measurement uncertainties ?
• The customer needs to get an idea of the “accuracy“ of the result,
• measurement uncertainty has to be taken into account particularly when regard-
2
ing specification limits (⇒ legal and liability aspects),
• testing laboratories need uncertainties with their calibration certificates, so that
they can state the uncertainty of their own measurements.
Where do measurement uncertainties in testing come from ?
• There are many possible sources of uncertainty, e.g. sampling, instrument drifts
and calibration, homogenisation and dilution effects, human factors, environ-
mental effects, ...
Who needs to give measurement uncertainties ?
• An estimation of a measurement’s uncertainty is required for testing and calibra-
tion laboratories complying with ISO/IEC 17025.
5.4.6.1: ”A calibration laboratory ... shall have and shall apply a procedure to es-
timate the uncertainty of measurement for all calibrations ...”,
5.4.6.2: ”Testing laboratories shall have and shall apply procedures for estimat-
ing uncertainty of measurement”
• Whether those uncertainties have to be stated in the test report depends on re-
quirements by the test method, requirements by the customer, or whether con-
formance to specification has to be assessed (ISO/IEC 17025, 5.10.3).
• In calibration, uncertainties have to be stated in the certificate (as they are re-
quired by the user of the calibrated equipment).
How to obtain and state measurement uncertainties
• There are clear guidelines for
- calculating / estimating uncertainties for each source separately, then
- combining of the contributions from each uncertainty source and finally
- stating the uncertainty of a result.
• These guidelines are given in the GUM, ”Guide to the expression of uncertainty
in measurement” [1], the ”main book” of measurement uncertainty, edited by
ISO, harmonising internationally the estimation and reporting of measurement
uncertainties.
• The basis of any evaluation of measurement uncertainty is a statistical approach.
• However, it may be implemented in facilitated ways, e.g. by estimating the “over-
3
all uncertainty” involving precision and/or validation data available in the labora-
tory.

1
Vocabulary of Basic and General Terms in Metrology, ISO, Geneva, 1993, ISBN 92-67-10175-1
2
e.g. ILAC G8 [69]
3
e.g. ISO/IEC 17025 (5.4.6) citing ISO 5725 [29]

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 4 - 27

How are measurement uncertainties expressed ?
• Report whether a single standard deviation is used or whether an expanded un-
certainty with the respective coverage factor and level of confidence is stated
with the result.
• Example: ”Height 20.051 ± 0.022 cm. The reported expanded uncertainty is
based on a standard uncertainty multiplied by a coverage factor k = 2, providing
a level of confidence of approximately 95%.”
• It may be useful to (briefly) state how the uncertainty was obtained and what it
includes.
• Two significant digits [1] (unless there are other requirements).
• The statement must never give a false positive impression of the uncertainty as-
sociated with the measurement (ISO/IEC 17025, clauses 5.4.6.2 and 5.4.6.3).

2. Determination of measurement uncertainty -

ways for estimating uncertainties in practice
First of all: a laboratory that has a good quality management system should have little effort
to state the uncertainty of a result.
The principles for correct application of measurement uncertainties are given in the GUM [1].
For further reading:
- The guide for ”Quantifying Uncertainty in Analytical Measurement”
by EURACHEM /CITAC [3] can be highly recommended for (analytical) chemists.
- Good explanations and examples from the calibration field are also found in ”Guidelines
to the Expression of the Uncertainty of Measurements in Calibrations” [2].
Besides these technical papers ILAC (International Laboratory Accreditation Co-operation)
published the strategy paper ILAC-G17:2002 "Introducing the Concept of Uncertainty of
Measurement in Testing in Association with the Application of the Standard ISO/IEC 17025"
[19] which is also applied by EA (European Co-operation for Accreditation). Asia Pacific
Laboratory Accreditation Cooperation (APLAC), too, has published a draft policy [20],
taking into account sector-specific requirements.
Chapter 3 gives a short summary of the procedure for estimation of the test result and the
accompanying measurement uncertainty in 8 steps as described in the GUM (chapter 8) [1].
GUM groups uncertainty components into type A and type B according to the way these data
were obtained. Type A components are calculated by statistical means from repeated meas-
urements while type B components are taken from other sources e.g. manufacturer's manu-
als, validation information or average control charts. For further details see chapter 3.
Besides this mathematical analytical approach also more pragmatic approaches are in con-
formity with GUM (and also in conformity with the requirements of ISO/IEC 17025). There-
fore before starting the procedure of uncertainty determination it is worth looking for all in-
formation available, which might reduce the effort for the uncertainty evaluation. The aim is
to find a way fit for purpose.
Such information can be grouped data, combining the contribution from several uncertainty
components, like the standard deviation within interlaboratory comparisons. From this data it
may be possible to already estimate the “overall uncertainty”. Also using uncertainty data
that have been assessed by type B estimation may simplify the approach. As stated in GUM,
type A and type B uncertainty components are of the same nature and value. For example
type B might be even better than type A when only a few repetitions have been performed.
However, before using these data it has to be checked whether the conditions apply.
With this information in hand the operator will be able to decide, which of the steps of uncer-
tainty estimation are still necessary for the specific experiment.

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 5 - 27

Estimating uncertainties in practice
There are different possibilities to estimate measurement
uncertainty budgets. The strict mathematical way is described
most extensively in the GUM, but the other methods are, too,
well compliant with the GUM.
Vo lu m e P u ri ty

In the strict mathematical analytical method (see chapter 3),

- all components are estimated separately and C on c e n tra tio n

- the single uncertainties are combined applying the

law of propagation of uncertainty. M a ss

Another possibility is to already use grouped data, in order to

directly estimate the “overall-uncertainty”,
- e.g from quality assurance data that are available in the
laboratory (e.g. standard deviations from repetition of all
steps (this would be type A) or precision data4, e.g. the
standard deviation of an interlaboratory comparison (type B))
- where all uncertainty components are included,
but not evaluated separately. In this respect this is a
„black-box“-approach.
However, in practise mostly a combined approach will be the most suitable, in which some
components or steps are grouped. This combined approach will apply very often, as it is of-
ten impossible to estimate each uncertainty individually.
The data available in a laboratory will often be data from quality control which are commonly
4
in the form of precision data , i.e. reproducibility or repeatability standard deviations.
The use of precision data for uncertainty estimation is described in the French standard FD
X 07-021 [23], the Eurachem Guide [3] and a draft paper prepared by ISO TC69/SC6/WG7
[35]. Precision data are described in ISO 5725 [30].
It is important to stress, that using practically obtained data for an “overall uncertainty”, like stan-
dard deviations within control charts, from interlaboratory comparisons or validation data, may be
easier. However, this approach is not a priori less valid. Experience shows that uncertainty estima-
tions obtained by the mathematical analytical approach are often too small. This is due to the fact
that it is difficult to draw a comprehensive model equation and therefore there is a tendency to omit
some significant components. Therefore any measurement uncertainty estimated by the mathe-
matical approach should always be compared with practical data. Ideally of course, measurement
uncertainties obtained either way should be of the same size.

How to use existing quality assurance data

ƒ Using data stated in standards
Some written standards specify the form of presentation or define how to state the meas-
urement uncertainty. When the laboratory demonstrates its ability to use the standard
method properly, it may give the result as stated in the standard.
In other cases the method validation data are given in the standard, e.g. through interla-
boratory comparison. These data may be also used for estimation of the measurement
uncertainty.
EUROLAB demands to include such uncertainty statements in all new standards.

4
Precision data may be obtained under repeatability or reproducibility conditions (compare ISO 5725)

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 6 - 27

ƒ In-house methods
In-house methods have to be validated by the laboratory (ISO/IEC 17025, 5.4.5). Many
validation methods imply measurements from which the standard deviation may be ob-
tained and used for uncertainty estimation of the method. Reference materials and qual-
ity control materials may be of great help in such a validation process.
ƒ Interlaboratory comparisons
Interlaboratory comparisons are a useful tool in uncertainty evaluation. The series of
measurements of the different laboratories delivers the results from a great number of
independent measurements under different conditions and provides the standard devia-
tion s of them. These data may well be used by a laboratory (having performed satisfac-
torily) as the standard uncertainty of the analysed parameter, provided that the compari-
son covers all relevant uncertainty components and steps (ISO/IEC 17025, 5.4.6.3). It
should be stressed that for the uncertainty estimation the standard deviation from all par-
ticipating laboratories should be used and not the difference of the single laboratory’s re-
sult form this standard deviation.
ƒ Average control charts
Average control charts are used by laboratories as a quality tool for instruments and
methods. The standard deviation s is determined in the charts (e.g. to set the 2s warn-
ing– and 3s control limits) and can be used directly as input for uncertainty evaluation
(see Example 1b). However, one should consider whether relevant reproducibility ele-
ments of uncertainty may be missing.
ƒ Estimation by experts
Estimation by experts on a certain experiment often reflects the uncertainty of an ex-
periment very well, resulting from experience and knowledge. It is very difficult though to
give advice on this source of uncertainty determination. The bottom line is that the labo-
ratory should be able to demonstrate that it performs within that uncertainty estimate.

No actions for uncertainty determination are required:

- at this time for qualitative and semi-quantitative methods, or
- where a well-recognised test method specifies the limits of the values of the major
sources of uncertainty of measurement and specifies the form of presentation of cal-
culated results. In this case the laboratory is considered to have satisfied uncertainty
of measurement requirements by following the test method and the reporting instruc-
tions [20].

The laboratory should ...

ƒ check - whether all relevant uncertainty sources are considered: e.g. sampling,
preparation, dissolving, dilution
- whether the conditions of the quality assurance data are comparable with the treated
problem, e.g. matrix and composition, range of values, repeatability / reproducibility
contributions
- whether the presumed simplifications are tolerable
- whether it is a field of testing with special requirements (reference materials,
health, safety, ...)
ƒ give a clear indication which steps are not included in the uncertainty budget
ƒ start with the data available in the laboratory and strive for continuously improving the
uncertainty statements with increasing number of data available and state of knowledge.

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 7 - 27

Aim: fit for purpose
The most important rule is: effort and expenditure for determination of uncertainties should
be clearly guided by the principle ”fit for purpose”, that is, it should be good enough to meet
the requirements of the user of the measurement data, but do not overdo it!
For testing laboratories the calculation of uncertainties following the strict mathematical ap-
proach involving all 8 steps (as described in chapter 3), carrying out several measurements
for each uncertainty source and involving the complicated mathematical equations for calcu-
lation of the uncertainty, may not be efficient or even applicable.
Instead, the experimenter should try to identify the relevant sources of measurement uncer-
tainty. This information will help him to improve the quality system. Therefore the experi-
menter should try to find sources of information, which are of key importance for controlling
the measurement procedure (system), thus allowing to implement the evaluation procedure
most efficiently.
Using grouped data or “overall uncertainties” are possible ways to estimate measurement
uncertainties on a practical basis. Furthermore the GUM [1] encourages Type B evaluation
of uncertainties from existing data, which usually reduces the expenses considerably, recog-
nising that Type B evaluation can be as reliable as Type A evaluation (4.3.2).

3. The mathematical analytical way:

8 steps to obtain test result and uncertainty
This procedure for evaluation and expressing uncertainty is described in detail in the
GUM [1]. Before start: Check for all possible information simplifying the uncertainty
determination

Step 1) Specify measurand, express mathematically the equation relating meas-

urand and input quantities. Identify all uncertainty sources.
Step 2) Determine the input quantities.

Step 3) Quantify the standard uncertainties of all single components.

Step 4) Identify the covariances (of correlated input quantities).

Step 5) Calculate the result of the measurement from the input quantities.

Step 6) Calculate the combined uncertainty.

Step 7) Calculate the expanded uncertainty.

Step 8) Give the result together with the uncertainty as estimated.

The single steps are carried out as follows:

Step 1) Specify measurand

Express mathematically the equation relating measurand, i.e. the quantity subject to
measurement, and input quantities. Identify all uncertainty sources.
A good way for listing the components is in ”cause and effect diagrams” [3].

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 8 - 27

 The equation will be of the form of a function f: y = f(x1,x2,x3,...,xN), expressing that
the measurand (result) is defined by a function depending on the N single quantities
x1,x2,x3,...,xN.

Step 2) Determine the input quantities x1,x2,x3,...,xN. Determine the values of the input
quantities, e.g. through measurement, from tables and from specifications.

Step 3) Quantify the standard uncertainties of all single components

There are two ways to determine the single uncertainties:
a. Calculating it by statistical means from repeated measurements (Type A)
b. Estimating values from other sources (called Type B)
Type A: Standard uncertainties calculated from a series of n measurements.
The following mathematical formulas have to be applied.
1 n q + q 2 + q3 + ... + q n
xj = q =
n
∑ q = 1
i =1 i
n
arithmetic mean (or average) of a series of n independent
measurements with
n: number of independent measurements
qi: value of a measurement for determination of
the input quantity xj
1 experimental standard deviation of a single
∑
n
s ( qi ) = ( qi − q ) 2 measurement
(n − 1) i =1

s ( qi ) 1 experimental standard deviation of the mean

∑
n
s(q) = = (qi − q) 2
n n(n − 1) i =1

u ( x j ) = s(q)
the standard deviation will be used directly as standard uncertainty
of the respective single component xj, measured n times

Type B evaluation of standard uncertainties:

This method uses prior information like: reported uncertainties of a reference mate-
rial, calibration certificates, previous measurement data, accepted values of con-
stants, experience on behaviour of similar materials and equipment, resolution, in-
stability, environmental conditions, results from interlaboratory comparisons, aver-
age control charts. Note: prior information may have been originally derived by sta-
tistical methods.
In principle we are always looking for standard deviations (or their squares, called
variances).
If values for uncertainties for single components are given as standard deviations of
a measurement they can be directly used as u(xj). For example a standard deviation
in an interlaboratory comparison is reported as the single standard deviation s,
which can be directly used as standard uncertainty u(xj) = s.
In other cases the statistical distributions have to be taken into account. Often it will
then be a bit more difficult when there are statements like ”at a level of confidence
of 95%5” or ”20.051 ± 0.022 cm” with ± 0.022 being the maximum deviation ex-
pected. In the first case normal distribution is presumed, in the second case the op-
erator has to decide whether rectangular or triangular distribution or others apply:

5
the required level of confidence will vary for different fields: while 95% may be sufficient in a technical field of
testing, 99.7% may be required e.g. in the health sector and for legal applications.

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 9 - 27

 Normal distribution / confidence intervals given: expanded uncertaint y
e.g. if a calibration certificate states the confidence u(x j ) =
k
interval to be 95%, the uncertainty of that component
is derived through: (with k=2 ca. 95% confidence,
5
k=3 ca. 99.7% confidence).
Rectangular distribution: (a+ − a− ) / 2 and
Upper and lower limits (a- and a+) are given: u( x j ) =
3
(example: last digit of a digital display)
if a+ − a− = 2a j then u ( x ) = a j
j
3
Triangular distribution: (a+ − a− ) / 2 and
applies, if values close to the measured value (centre) u( x j ) =
6
are more likely than the values close to the limits.
(Example: volume of a flask: 100 ml ± 0.1 ml; volumes
if a+ − a− = 2 a j then u ( x ) = a j
close to the nominal value are more likely than the j
extremes) 6

u( x j )
the standard uncertainty is derived from prior information
either directly or after consideration of the statistical distribution

Step 4) Identify the covariances (of correlated input quantities)

If two input quantities have a common source of uncertainty (e.g. they both depend
on an uncertain temperature), their uncertainties are not independent and therefore
they are said to be correlated.
For further details see paragraph F.1.2.3 of the GUM [1].
In practice, correlation effects often are not known and the approximation (or as-
sumption) is made that there is no correlation.
Note: Not taking into account existing correlations may result in a wrong estimate of
measurement uncertainty.

Step 5) Calculate the result of the measurement from the input quantities
The result is: y = f(x1,x2,x3,...,xN)

Step 6) Calculate the combined uncertainty from the uncertainties of the single compo-
nents (as determined in 3), taking into account possible covariances.

The combined uncertainty is the square root of:

2
(first order Taylor approximation) for non-correlated n
 ∂f  2
( y ) = ∑   u ( xi )
2
uncertainty components u combined
i =1  ∂xi 
[For reasons of completeness the equation for n n
 ∂f  ∂f 
ucombined ( y ) = ∑∑   u ( xi , x j )
2
correlated uncertainty components is also given:]
 
i =1 j =1  ∂xi  ∂x j 

 ∂f 
In those equations the partial derivative   is called sensitivity coefficient.
 ∂xi 
The equations described above look very complicated to most experimenters.
In practice they become much simpler for some simple mathematical relations (in
case that no correlations exist):

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 10 - 27

 for equations of the measurand involving
ucombined ( y ) = u1 + u2 + ... + un
2 2 2
only sums or differences:
y = x1 + x 2 + x3 − x 4 ...
for equations involving only products or 2 2 2
ucombined ( y ) u1 u2 u
quotients: = 2
+ 2 + ... + n 2
y = x1 ⋅ x 2 ⋅ x3 / x 4 ... y x1 x2 xn

for equations of the measurand involving 2 2 2

u combined ( y ) a 2 u1 ( x1 ) b 2u 2 ( x2 ) z 2u n ( xn )
exponents: y = x1 x 2 ...x n
a b z = 2
+ 2
+ ... + 2
y x1 x2 xn

A useful help for calculation of difficult uncertainty problems is the spreadsheet

method as described in [3] using numerical differentiation.

Step 7) Calculate the expanded uncertainty

The expanded uncertainty, denoted by U, is a measure of uncertainty, given for the
appropriate distribution function of the result (e.g. normal distribution), for a given
level of confidence (e.g. 95%). The result of a measurement is then conveniently
expressed as Y = y ± U. This is interpreted that the best estimate of the value at-
tributable to the measurand Y is y and the interval y-U to y+U may be expected to
encompass a large fraction p of the distribution of values that could reasonably be
attributed to Y. p is the probability or level of confidence of the interval.

Y = y ± U = y ± k ⋅ u combined ( y )

U = k ⋅ u combined ( y ) U: expanded uncertainty

k: coverage factor
k = 2 ⇒ level of confidence of 95%6,5
k = 3 ⇒ level of confidence of 99.7%
Alternatively, Student distributions7, involving degrees of freedom are sometimes
applied, especially when the number of replicates of the measurement carried out is
small.

Step 8) Give the result together with the uncertainty as estimated

Results should be given in the form of the following statement:
”Gas pressure: 200.3 ± 2.2 Pa. The reported expanded uncertainty is based on a
standard uncertainty multiplied by a coverage factor k = 2, providing a level of confi-
dence of approximately 95%.”
Two significant digits should be given [1] (unless more digits are required to avoid a
loss of information). It may be useful to state briefly how the uncertainty was ob-
tained. However, if any steps or relevant components (sampling, preparation steps,
actual measurement,...) are not included, this should be indicated.

6
if normal distribution may be assumed
7
Student- or t-distribution, see also GUM [1] G.3

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 11 - 27

4. Examples for estimating measurement uncertainty
The following examples show possibilities for estimation of uncertainties in different testing
fields.
Example 1 shows sources of data a laboratory could look for trying to estimate the uncer-
tainty of the measurement of 100 mg sulfate in waste water determined with ion-
chromatography. The data available in the laboratory may be the method standard, profi-
ciency test results or control charts.
It is the task of the laboratory to decide, which data are best suitable and which data may not
be applicable for reasons of matrix differences, different range of values or because not all
uncertainty components may be covered.

Example 1
Chemical Testing
1.a) Estimation of the uncertainty of the measurement of 100 mg sulfate in waste wa-
ter determined with ion-chromatography from proficiency test results.
th
(Step 3) The laboratory has participated satisfactorily in the 4 all-German waste-water
proficiency testing. The standard deviation of all of the laboratories‘ results was
s = 4%.
(Step 6) s = u = 4 % ⇒ u = 4.0 mg/l for 100 mg/l sulfate
(Step 7) Uexpanded= k· u = 2· 4.0 mg/l = 8.0 mg/l
(Step 8) Sulfate: 100.0 mg/l or: Sulfate: 100.0 ± 8.0 mg/l
measurement uncertainty: 8.0 mg/l (k=2)
This measurement uncertainty was derived from interlaboratory comparison re-
sults. It is expressed as an expanded uncertainty and was obtained by multiplica-
tion of the combined uncertainty with the factor k = 2, relating to a level of confi-
dence of 95%.
Note: Some publications e.g. [35] promote the approach of combining repeatability and repro-
ducibility uncertainties: ucombhined = u repeat 2 + u reproduce 2 , which is too pessimistic. The above method
is to be preferred, and sufficient if the proficiency test covers all relevant uncertainty compo-
nents.

1b) Estimation of the uncertainty of the measurement of 100 mg sulfate in waste water
determined with ion-chromatography from control-charts.
(Step 3) The laboratory maintains an individual control chart for determination of sulfate.
The standard deviation from it is sCC = 3.8% .
While the control sample of 5 mg/l is measured directly, the sample of 100 mg/l
had to be diluted. This dilution step has to be taken into account for the uncer-
tainty budget. The dilution step with a dilutor was carried out and weighed 10
times and delivered a standard deviation sDil = 1%.
(Step 6) ucombined = u CC 2 + u Dil 2 = 3.8 2 + 12 = 3.9 %
ucombined = 3.9 mg/l for 100 mg/l sulfate
(Step 7) Uexpanded= k· ucombined = 2· 3.9 mg/l = 7.8 mg/l
(Step 8) Sulfate: 100.0 mg/l
measurement uncertainty: 7.8 mg/l (k=2)

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 12 - 27

 This measurement uncertainty was obtained from the laboratory’s control charts
and is expressed as an expanded uncertainty ...
Note:
This is an example where an additional step has to be considered in addition to quality as-
surance data that were available in the laboratory. No allowance has been made for any
laboratory bias (which is normally required).

1c) Estimation of the uncertainty of the measurement of 100 mg sulfate in waste water
determined with ion-chromatography from validation data in the method standard.
(Step 3) The validation data for determination of sulfate in wastewater with ionchroma-
tography are given in ISO 10304-2. In an interlaboratory comparison of an indus-
trial waste water for method validation the reproducibility standard deviation was
6.1%.
(Step 6) s = u = ucombined = 6.1 % ⇒ ucombined = 6.1 mg/l for 100 mg/l sulfate
(Step 7) Uexpanded= k· ucombined = 2· 6.1 mg/l = 12.2 mg/l
(Step 8) Sulfate: 100 mg/l
measurement uncertainty:12 mg/l (k=2)
This measurement uncertainty was derived from method validation data in accordance
to ISO 10304-2. ...

Note: The laboratory must have proved to be able to perform in accordance with this stan-
dard method.

Example 2
Mechanical Testing
Hardness testing according to Brinell
At the moment it is difficult to calculate the measurement uncertainty in the field of mechani-
cal testing. This is because of the lack of knowledge how to use the approaches developed
in various other fields of chemistry or calibration. Nevertheless there are possible way of
estimating the measurement uncertainty to comply with the demands of clients and test stan-
dards. The first area in mechanical testing for which a complete system was evaluated is the
following example which soon should be a part of the test standards.
This examples can be used to calculate the measurement uncertainty in hardness testing.
This is the first approach how to do this calculation. It is expected that this model will be
used in the ISO standards 6506, 6507 and 6508 in the near future. This model was verified
using the data of about 95 laboratories, participating in an European proficiency test.
The measurement uncertainty calculated is based on, hardness testing according to Brinell.
For this test it is measured how a ball can deform a sample plate.
This example was calculated on the basis of a hardness reference block (CRM) 246.8 ± 1.5
HBW 2.5/187.5 according to the test standard EN ISO 6506. The block was certified by the
MPA NRW Dortmund.

The single values of the calibration were:

246.9 245.8 246.3 247.9 247.0 HBW 2.5/187.5, measured by MPA NRW,
resulting in a standard deviation of s X CRM = 0.77 HBW 2.5/187.5.

The uncertainty was estimated according to:

U exp anded = 2 * u E2 + u X2 CRM + u CRM

2
+ u H2 + u x2

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 13 - 27

The single uncertainty components are listed below.
The evaluated uncertainty of 2.1% seems to be reasonable.

EUROLAB Technical Report 1/2002 - Measurement Uncertainty in testing 14 - 27

 Sources of uncertainty Abbre- Formula Literature/Certificate Sample calculation
viation
STEP

[..] = HBW 2.5/187.5

1 Uncertainty according to the u E , 2 r acc.

maximum permissible error u E ,2 r 0.02 ⋅ 246.8
uE uE = (SR = 2.8 · Sr) EN ISO 6506-2, uE = =1.763
2.8 Table 2: 2% for this diameter 2.8
of the ball
2 Uncertainty of the standardizing u X CRM , 2 σ u X CRM , 2 σ acc. calibration certifi- 1.5
machine for calibration of CRM u X CRM u X CRM = u X CRM = = 0.75
2 cate of CRM 2
3 Mean value and standard devia- 1 n
tion of the calibration of CRM X CRM = ⋅ ∑ X i CRM
X CRM , n I =1 X i acc. calibration certificate X CRM = 246.8
of CRM
s X CRM 1 n s X CRM = 0.77
s X CRM = ∑
n − 1 i =1
(X i ( CRM ) − X CRM ) 2

4 Standard uncertainty of CRM t * s X CRM t=1.15 (Student-factor) for n=5 1.15 ⋅ 0.77
uCRM uCRM = uCRM = = 0.39
n and 68.3% confidence level 5
5 Mean value and standard devia- 1 n
tion of the measurement on CRM H = ⋅ ∑ Hi Single measurements:
H , n I =1 H i acc. EN ISO 6506, part 2, 246.0–245.0–246.0–246.0-246.0
sH 1 n No. 5.6 H = 245.8 / s H = 0.45
sH = ∑
n − 1 i =1
(H i − H ) 2

15
 Sources of uncertainty Abbre- Formula Literature/Certificate Sample calculation
viation
STEP

[..] = HBW 2.5/187.5

6 Standard uncertainty of hardness t ⋅ sH t=1.15 (Student-factor) for n=5 1.15 ⋅ 0,45

uH uH = uH = = 0.23
testing machine when measuring
CRM n and  5
7 Mean value and standard devia- 1 n
tion of the testing of a sample x = ⋅ ∑ xi Single measurements:
n I =1 288.0–290.0–285.0–285.0-282.0
x , x i : 5 measurements on sam-
sx n x = 286.0
1 ple
sx = ∑
n − 1 i =1
( x i − x) 2 s x = 3.08
8 Standard uncertainty when
measuring a sample t ⋅ sx t=1.15 (Student-factor) for n=5 1.15 ⋅ 3.08
ux ux = ux = =1.59
(including sample preparation, n and  5
e.g. polishing)
9 Calculation of the expanded
1.762 + 0.752 + 0.392
measurement uncertainty U = 2⋅
steps 1 to 8, + 0.232 + 1.59 2
U U = 2⋅ u + u
2
E
2
X CRM
+u 2
CRM +u +u2
H
2
x 95% confidence level

U = 5.1
10 Overall result steps 7 and 9
X = x ±U 95% X = 286.0 ± 5.1
ALTERNATIVE (in %)
11 Calculation of the relative ex- ~ U ~ 5.1
A panded measurement uncertainty ~ U= ⋅ 100% steps 1 to 9,
U= ⋅ 100% = 2.1%
U 95%
X CRM 246.8
12 Overall result steps 7 and 11A
A
X = x ±U 95% X = 286.0 HBW 2.5 / 187.5 ± 2.1%

16
Example 3
Determination of emission measurement and sampling
uncertainties estimated by well experienced experts
In 1998 an interlaboratory estimation was carried out among 26 institutes from Germany and
Switzerland measuring air polluting emissions according to VDI technical method standards.
The results are given in [86] “Verlauf und Ergebnisse einer geordneten Ringschätzung der
Unsicherheiten von Emissionsmessungen” (Process and results of an well-ordered interlabo-
ratory estimation of the uncertainties from emission measurements) by Janusz S. Morkowski,
Umweltbundesamt Texte 54/99, Berlin, 1999.
Five popular and well known VDI technical methods of emission measurement had been
chosen. Each of them consist of several procedural steps that may influence the final result
of the measurement. The methods chosen were:
1) VDI 2066, Blatt 2: Filterhead with stuffing tube for dust determination (gravimetrically)
2) VDI 2066 Blatt 7: Filterhead with a plane filter for dust determination
3) VDI 2456 Blatt 10: Total nitrogenoxides
4) VDI 3480 Blatt 1: HCl with different analytical techniques
5) VDI 3481 Blatt 2: Total organic carbon by sorption method.
The laboratories received a questionnaire specifying the procedure of the estimation and
were instructed by the organiser.
Uncertainty components for the different steps of the procedures (methods) were to be esti-
mated according to an estimation form. Those steps included planning, preparation, sam-
pling, determination of the gas volume, sample transportation, preparation of the sample,
measurement and analysis in the laboratory, evaluation of the result.
These single uncertainty components of the procedure were combined according to the un-
certainty propagation law of Gauss to the total measurement uncertainty UT.
Estimations were made both for the “standard case”, S-case, which can be expected for
about two thirds of the tests and for the “problematical case” X-case, which should also
cover most problematic cases (e.g. evil conditions, sampling not representative, ...) with a
probability of 95% and more.
75 sampling personnel and 20 employees of the laboratories participated in the estimation test
being completely independent from each other. The following table gives the results of the
estimation test in % (for the S-case). For methods 3 and 4 only the results for the photometri-
cal analysis are given here. The entire tables of the results can be found in [86].
As it can be seen from the table the estimated total uncertainties for the 5 different methods
vary from 13% to 21% with the sampling step being one of the major components.
The total uncertainties are calculated first for all steps of the procedure UT(I) and secondly
also for the case where the planning step is not included in the calculation UT(II). In the latter
case the uncertainties are reduced especially for the filter tests. The relations of X-case to S-
case are approximately 2.7.
For one of the methods, the dust determination with plane filters, also practical interlabora-
tory comparison data were available. Laboratory intercomparisons at the Emission-Simu-
lation-Plant at the HLfU at Kassel from 1994 to 1996 give a mean standard deviation of
15.4% for small concentrations of dust in emissions.
Because planning, which is usual in regular emission measurements is not needed in this
case, the estimated uncertainty data UT(II) of the dust determination with plane filters without
the planning step can be used for comparison. The value estimated in the interlaboratory
estimation is 15.5%, which is nearly equal to the value obtained in the HLfU-Plant.

17
Method standard VDI 2066 VDI 2066 VDI 2456 VDI3480 VDI 3481
Bl.2 Bl.7 Bl. 10 Bl. 1 Bl.2
Measurand Dust Dust NO+NO2 HCl TOC
Analytical technique: gravimetric gravimetric photometric photometric sorption
Planning 11.06 8.27 3.59 3.32 3.26
Preparation of sampling 6.44 6.85 3.56 5.43
Sampling 12.55 11.46 10.11 6.21 15.97
Transport and sample preparation 2.34 2.61 8.15 3.31 5.38
Preparation of sample / treatment of 3.61 8.09 10.06
the filter /sorption material
Determination of the volume 4.33 4.21 4.33 3.87
Additional parameters for volume 3.68 4.32 2.83 3.73
determination
Other influences 0.46 0.67 0.41 0.75
Analytical measurement 8.37 8.11
Other influences 1.12
Total uncertainty UT(I) 19.3 17.6 17.3 12.9 21.3
From these mean values of the
single uncertainty components
Total uncertainty UT(II) 15.8 15.5 16.9 12.5 21.1
as above without the planning step
Relation X-case/S-case 3.04 2.93 2.39 2.64 2.62

Though the single estimation results (which are not given here) are distributed widely, the
results of the estimation (average and the limits of a rather narrow range of confidentiality)
may be a useful approach for the determination of measurement uncertainties in this field.
The results also show that it is important that the estimator knows the testing procedure well,
that he is very experienced in the field concerned and that the details on the conditions of
the measurement as a basis for the estimation are provided.
The estimation of the uncertainties for the sampling step of the different methods provide
examples which may serve as a basis also for other methods.
As it can be seen from the table the estimated total uncertainties for the 5 different methods
vary from 13% to 21% with the sampling step being one of the major components.
The total uncertainties are calculated first for all steps UT(I) and secondly also for the case
where the planning step is not included UT(II). In the latter case the uncertainties are re-
duced especially for the filter tests. The relations of X-case to S-case are approximately 2.7.
For one of the methods, the dust determination with plane filters, also practical interlabora-
tory comparison data were available. Laboratory intercomparisons at the Emission-
Simulation-Plant at the HlfU at Kassel from 1994 to 1996 give a mean standard deviation of
15.4% for small concentrations of dust in emissions.
Because planning is not needed in this case, the estimated uncertainty data UT(II) of the dust
determination with plane filters without the planning step can be used for comparison. The
value estimated in the interlaboratory estimation is 15.5%, which is well comparable to the
practically obtained value.
Though the single estimation results (which are not given here) vary widely, the results show
that estimation may be a useful approach for determination of measurement uncertainties.
The results also show that it is important that the estimator knows the testing procedure well
and is provided details on the conditions as a basis for the estimation.
The estimation of the uncertainties for the sampling step of the different methods provide
examples which may serve as a basis for other methods.

18
Example 4
Determining the measurement uncertainty in preparing a calibra-
tion standard by the mathematical analytical approach
Useful examples for measurement uncertainty from the single components, closer following
the strict mathematical analytical way, can be found in references [2] and [3].

The following example leans on Example A1 of the EURACHEM /CITAC Guide ”Quantifying
Uncertainty in Analytical Measurement” [3]. It has been chosen to clarify the 8 steps of the
procedure.
The aim is to prepare a calibration standard of Cd in HNO3 for AAS. To obtain the Cd cali-
bration standard of ca. 1000mg/l the following procedure was applied:
1. Weighing a piece of metal (surface cleaned).
2. Dissolving the metal in a 100 ml flask by adding 1 ml HNO3 and filling with deionised
water.

Step 1) Specify measurand, express mathematically the equation relating measurand

and input quantities. Identify all uncertainty sources.
1000 ⋅ m ⋅ P cCd: concentration of the calibration standard obtained
cCd = [mg / l ]
V m: mass of the clean high purity Cd piece [mg]
P: Purity of the metal
V: volume of the flask [ml]
1000: conversion factor from ml to l

Listing the components of uncertainty

1. Purity of the Cd: supplier’s certificate: 99.99 ± 0.01 %.
2. Mass of the metal from weighing in the flask. The piece weighed 0.10028g. The manu-
facturer’s literature identifies 3 uncertainty sources for tare weighing:
2.1 repeatability,
2.2 readability of the balance scale,
2.3 calibration (involving sensitivity of the balance and linearity).
Sensitivity can be neglected because weighing was done on the same balance over a
narrow range. Buoyancy correction can be neglected [3] being very small.
3. Volume of the solution: 100ml. Uncertainty sources are:
3.1 uncertainty in the certified internal volume of the flask
3.2 Filling the flask to the mark
3.3 Temperature influences

”Cause and effect” diagram:

19
 Volume Purity

Temperature

Calibration

Repeatability
Concentration
cCd
Readability Readability
m (gross)

Linearity Linearity
Repeatability Repeatability

Sensitivity Sensitivity

Calibration Calibration

Step 2) Determine the input quantities

1. Purity of the Cd: 99.99 ± 0.01 % i.e. 0.9999 ± 0.0001.
2. Mass of the Cd: 0.10028 g.
3. Volume of the solution: 100ml

Step 3) Quantifying the single uncertainty components

1. Purity of the Cd: Type B evaluation: 99.99 ± 0.01 % i.e. 0.9999 ± 0.0001.
A rectangular distribution is assumed, because there is no further information. Therefore
the standard uncertainty of the purity is:
0.0001
u ( P) = = 0.000058
3
2. Mass of the Cd: 0.10028 g. The manufacturer of the balance recommends
0.05 mg as uncertainty estimation, this value can be taken directly as
u (m) = 0.05mg (Type B)
3. Volume of the solution: 100ml
3.1 uncertainty in the certified internal volume of the flask: the manufacturer quotes a
volume for the flask of 100 ± 0.1 ml at 20°C.
No confidence level is given, so a triangular distribution was chosen, because in
an effective production process the nominal value is more likely than extremes.
Therefore
0.1ml
u (V1 ) = = 0.04ml (Type B)
6
3.2 Filling the flask to the mark: An experiment of 10 fill and weigh experiments gave a
standard deviation of 0.02 ml. This can be used directly as
u (V2 ) = 0.02ml (Type A)
3.3 Temperature influences: The laboratory temperature varies between the limits of
20°C ± 4. The volume expansion of water is large compared to flask material,
which is therefore neglected. The volume expansion of water is 2.1 ⋅ 10 −4 / °C ,
leading to volume variation of
± (100 ⋅ 4 ⋅ 2.1 ⋅ 10 −4 )ml = ±0.084ml
Assuming rectangular distribution gives
0.084
u (V3 ) = = 0.05ml
3
20
 The three volume effects add to each other and are treated like a sum. The combined
uncertainty from volume effects is then

u (Vtotal ) = 0.04 2 + 0.02 2 + 0.05 2 = 0.07 ml

Step 4) Identify the covariances (of correlated input quantities)

Correlation effects are not known and the approximation is made that there is no correlation.

Step 5) Calculate the result of the measurement from the input quantities
Determination of the concentration
1000 ⋅ m ⋅ P 1000 ⋅ 100.28 ⋅ 0.9999
cCd = [mg / l ] = mg / l = 1002.7 mg / l
V 100
The concentration of the calibration standard is 1002.7 mg/l.

Step 6) Calculate the combined uncertainty

Because the above equation is a multiplicative expression, the uncertainties are combined
by:

u combined (cCd ) u ( P) 2 u (m) 2 u (Vtotal ) 2 0.000058 2 0.05 2 0.07 2

= + + = + + = 0.0009
cCd P2 m2 Vtotal
2
0.9999 2 100.28 2 100 2
u combined (cCd ) = 0.9mg / l
Comparing the uncertainties from the components shows that volume and mass uncertain-
ties contribute in a similar way to the overall uncertainty, while the purity has almost no influ-
ence on it.

Step 7) Calculate the expanded uncertainty

The expanded uncertainty is
U = k ⋅ u combined (cCd ) = 2 ⋅ 0.9mg / l = 1.8mg / l
The coverage factor k is chosen to be 2 as recommended by the GUM [1].

Step 8) Give the result together with the uncertainty as estimated

The concentration of the Cd standard is 1002.7 ± 1.8 mg/l. The reported expanded
uncertainty is based on a standard uncertainty multiplied by a coverage factor k = 2,
providing a level of confidence of approximately 95%.

21
5. Conclusions
There are many possibilities and ways for uncertainty estimation of methods or components
thereof, based on experience and general knowledge gathered from practice.
Estimates can also be based on a list of the main influencing factors. Such a list is helpful in
any case, as it facilitates final checking of the estimation method for covering all relevant
components.
The laboratory’s expertise is needed to decide whether the respective data can be used for
the uncertainty estimation with view to: observed range, whether all major components are
included, e.g. preparation steps, dilution, .... It may be difficult to put all these components
and the available data together into a model. The assessment of measurement uncertainties
can only be based on the present state of the art, however. Laboratories should use the data
available for the evaluation of measurement uncertainty.
The degree of rigor and detail of mathematical models differs widely among fields of meas-
urement. In particular in the testing area, it is often not possible to draw up a very detailed
model, as the method does not allow it, or the method does not require it. In those cases, a
much simpler approach may be applied (ISO/IEC 17025, 5.4.6.2). Furthermore, the evalua-
tion of measurement uncertainty is still evolving, and as a result there are great differences
in the progress made in the various fields of measurement and testing. Therefore in different
fields uncertainty estimations will be at a different stage of development, and the realisation
may need different amounts of time and may need to be implemented stepwise.
It is often argued, that customers are confused by uncertainty statements. Therefore
ISO/IEC 17025 (5.10.3) states, that measurement uncertainty does not always have to be
stated but only in those cases where it is relevant for the client.
Finally, it should be realised that measurement uncertainty is of added value, both for the
laboratory in terms of its quality management and for the customer as well.

Still it is not the task of laboratories alone to fulfil uncertainty requirements:

ƒ Accreditation bodies are asked to explain what they expect from laboratories. Education
of the assessors is very important. They will have to look into the uncertainty statements
and check whether laboratories can comply with them. This will be especially important
with view to a fair competition between laboratories.
ƒ Manufacturers of instruments and reference material are required to deliver useful data,
in particular concerning realistic and “GUM-compliant” uncertainty statements.
ƒ Normalisation bodies (e.g. CEN, ISO) should give detailed information on uncertainties
from method validation with new standards.
ƒ Clients have to get information on measurement uncertainty. The leaflet “Important in-
formation to our customers concerning the quality of measurements“ [17] published by
SP (Swedish National Testing and Research Institute) and other organisations may be a
helpful tool for clients to understand the meaning of measurement uncertainty.

22
6. Compilation of the main and some specific literature on the field of measurement uncertainty
Title Author Year Remarks Source

Basic literature including methods for practical determina-

tion of measurement uncertainties
[1] Guide to the expression of the uncertainty in measurement BIPM/IECIIFCC/ISO/OIML/IUPAC/IUP 1993 / 1995 "GUM", main document on uncertainty, estab- ISO 110 p., approximately CHF 92,00
AC ISBN 92 67 10188 9 lishes general guidelines for evaluating and
expressing measurement uncertainty

[2] Expression of the Uncertainty of Measurement in Calibration EA-4/02, EA Taskforce for Revision of 1999 Very good, comprehensible document. Many Can be downloaded from EA home-
WECC doc 19-1990 well explained examples from calibration field. page (www.european-accreditation.org)
[3] Quantifying Uncertainty in Analytical Measurement, EURACHEM Eurachem / CITAC 1995 / 2000 Most comprehensible document for meas- 120 p.; download from
/CITAC Guide urement uncertainties. Many very good exam- www.measurementuncertainty.org
ples. Good explanation of "Spreadsheet or www.eurachem.bam.de
method".
[4] The Expression of Uncertainty and Confidence in Measurement NAMAS NIS 3003 1997 Good description of the concepts of uncer- NAMAS, United Kingdom Accreditation
tainty determination, examples mainly from Service, 21-47 High Street, Feltham,
calibration area. Middlesex TW13 4UN

[5] Guidelines for evaluating and expressing uncertainty of NIST Barry N. Taylor and Chris E. Kuyatt, 1993 Good description of the concepts of uncer- Download from http://physics.nist.gov
measurement results NIST tainty determination

More information on uncertainty determination

[6] Estimating Uncertainties in Testing Keith Birch, BMTA 2001 Very good, with examples BMTA, Teddington, Middlesex, UK,
Measurement Good Practice Guide Nr. 36 TW11 0NQ, ISSN 1368-6550
[7] The NIST Reference on Constants, Units, and Uncertainty NIST homepage 2000 Very good, short introduction on to how to http://physics.nist.gov/cuu/Uncertainty/i
evaluate and calculate uncertainties. ndex.html
[8] Estimating Uncertainties in Testing, A Guide to Estimating and BMTA 1995 BMTA, PO Box 101, Teddington, Mid-
reporting Uncertainties of Measurement in Testing dlesex, TW11 0NQ, United Kingdom
Tel: +44 (0)20 8943 5524
[9] NPL – Measurement Good Practice Guide, National Physics NPL , Stephanie Bell 1999 Good, simple introduction for beginners. NPL 25 £, free chapters:
Laboratory, A beginners guide to Uncertainty in Measurement Some chapters free on internet. http://www.npl.co.uk/npl/publications/go
od_practice/uncert/
[10] Determining and Reporting Measurement Uncertainties National Conference of Standards 1995 30$, USA, Boulder CO;
Laboratories RP 12 http://www.ncslinternational.org/publicat
ions/pubs-list.cfm
[11] U.S. Guide to the Expression of Uncertainty in Measurement ANSI/NCSL Z540-2-1997 1997 50$, USA, Boulder CO;
http://www.ncslinternational.org/publicat
ions/pubs-list.cfm
[12] ISO/BIPM-Leitfaden:Meßunsicherheit Dr. W. Kessel 1998 Example of a weighing experiment, in Ger- www.metrodata.de/papers/waage.html
man.
[13] Estimating measurement uncertainty: reconciliation using a cause S.L.R. Ellison, V.J. Barwick; Accred. 1998
and effect approach Qual. Assur. 3, P.101-105
[14] Swiss Accreditation Service: Dok. 706.d "Validjerung von Prüfver- Swiss Accreditation Service: Dok. 1995
fahren" 706.d

23
 Implementation of uncertainty
[15] Uncertainty of test results ("Result Uncertainty") DAR-EM22, DAR ATF Ad-hoc group 1996 General notes on application of uncertainty. DAR-EM22, http://www.dar.bam.de/
"Uncertainty in testing" Good table of publications on uncertainty. under documents, ATF
[16] The Expression of Uncertainty in Quantitative Testing EA-3/02, (previously EAL-G23) 1996 Guidance for accreditors for implementation http://www.european-accreditation.org/
of uncertainty requirements
[17] 1. Important information to our customers concerning the quality SP, P.O. Box 857, SE-501, 15 Boras, 2000 Leaflet on measurement uncertainty in prac- SP, P.O. Box 857, SE-501, 15 Boras,
of measurements Sweden, Tel: +46-33-13-55-02 tice Sweden, Tel: +46-33-13-55-02
2. Measurement Uncertainty – Surveys about Customers’ Knowl- U. Örnemark, Magnus Holmgren 2001 Surveys about Customers’ Knowledge, Reac- [email protected]
edge, Reactions and Needs tions and Needs
[18] Measurement Uncertainty UKAS 2000 Short introduction to measurement uncer- www.ukas.com Accreditation topics:
tainty Measurement Uncertainty
[19] ILAC-G17:2002: Introducing the Concept of Uncertainty of Meas- ILAC and EA 2001 Important policy www.ilac.org
urement in Testing in Association with the Application of the
Standard ISO/IEC 17025
[20] APLAC Policy and Guidance on the estimation of Uncertainty of APLAC 2001 Good and useful document, gives sector
Measurement in Testing – Draft November 2001 oriented advice
[21] A2LA policy A2LA 2000 An implementation strategy into accreditation www.a2la.org

[22] Assessment of Uncertainties of Measurement for Electrical Test- Nata Australia 1992
ing
Standards associated with measurement uncertainty
[23] French Standardisation: Aid in the procedure for estimating and AFNOR Description how to use precision data for
using uncertainty in measurement and test results – FD X 07-021 uncertainty estimation
[24] ISO/DIS 10576-1 Statistical methods “Guidelines for the evaluati- ISO, TC 69 2001, draft
on of conformity with specified requirements”
[25] ISO/TS 14253-2:1999 Geometrical Product Specifications (GPS) - ISO 1999 ISO, www.iso.ch; CHF 164.00
- Inspection by measurement of workpieces and measuring
equipment -- Part 2: Guide to the estimation of uncertainty in GPS
measurement, in calibration of measuring equipment and in prod-
uct verification
[26] ISO 3951:1989 Sampling procedures and charts for inspection by ISO 1989 ISO, www.iso.ch; CHF 188.00
variables for percent nonconforming
[27] ISO 6974 Natural gas -- Determination of composition with de- ISO 2000
fined uncertainty by gas chromatography
[28] ISO 13752 Air quality -- Assessment of uncertainty of a meas- ISO 1998
urement method under field conditions using a second method as
reference
[29] ISO 7066 Assessment of uncertainty in calibration and use of flow ISO 1997
measurement devices
[30] ISO 5725-1-6: 1994/Cor. 2001 Accuracy (trueness and precision) ISO 1994 ISO, www.iso.ch
of measurement methods and results
[31] DIN 40080 Stichprobenprüfung anhand qualitativer Merkmale DIN 1993 Beuth Verlag
DIN ISO 2859-1, Ausgabe:1993-04
Annahmestichprobenprüfung anhand der Anzahl fehlerhafter
Einheiten oder Fehler (Attributprüfung)

24
[32] DIN 53804 T3/T4 Statistische Auswertungen, Ordinal- DIN 1982 / 1985 Beuth Verlag
/Attributmerkmale
[33] DIN 25424 Fehlerbaumanalye DIN 1990 Beuth Verlag
[34] DIN 1319 Teil 3 "Auswertung v. Messungen einer einzelnen Meß- DIN 1996 / 1999 Beuth Verlag
größe"; Meßunsicherheit; Teil 4 "Behandlung von Unsicherheiten
bei der Auswertung von Messungen"
[35] Statistical assessment of the uncertainty of measurement results: ISO TC69/SC6/WG7 2001 Draft
Guide to the use of repeatability, reproducibility and trueness
estimates in measurement uncertainty estimation
Software for Uncertainty calculation
[36] Tools for the test laboratory to implement measurement uncer- Sven Nytoft Rasmussen; Nordtest 1999 Very good comparison of the concepts of the Nordtest, P.O. Box 116, FIN-02151
tainty budgets Techn. Report 430 different computer programs Espoo, Finland, Phone +358-0-455-
4600
[37] MUSAC (Measurement Uncertainty in Analytical Chemistry) EMPA, Creasoft AG and others 2001 software system for application in the chemi- information at: www.musac.ch.; price
cal laboratory: e.g. titration, HPLC, GC, not jet known
GC/MS, ICP/OES, ICP/MS and AAS, first part
available September 2001
[38] AESoft Uncertainty Atkinson Engineering, Inc 1000 $, free demo version;
www.aesoft.com/Unc1.html
[39] GUM Workbench Metrodata GmbH, Grenzach-Wyhlen, 1100 $, free demo version; Metrodata
Germany GmbH; http://www.gum.dk;
http://www.metrodata.de/
[40] DFM-GUM Danish Institute of fundamental me- 300 $;
trology http://www.dfm.dtu.dk/en/consult/dfm-
gum.htm
[41] Uncertainty Analyzer Integrated Sciences Group. Uncer- 1994-96 700 $, USA, Bakersfield,
tainty Analyser 1.0 manual Ca.www.isgmax.com;
http://www.quametec.com/downloads.ht
m
[42] Expression Buddy James E. Presley and Daniel B. free software; download at
Presley http://www.jpresley.com/
[43] “Uncert” Project

Books
[44] Experimentation and Uncertainty Analysis for Engineers Hugh W.Coleman, W. Glenn Steel 1999 New York, Wiley & Sons Inc. ISBN 0-
471-12146-0
[45] Meßunsicherheit und Meßdatenauswertung Klaus Weise, Wolfgang Wöger 1999 Weinheim, Wiley-VCH; ISBN: 3-527-
29610-7
[46] Messunsicherheiten: Theorie und Praxis Franz Adunka 1998 Essen, Vulkan-Verlag, ISBN 3-8027-
2186-1
[47] Estimer l'incertitude, Mesures - Essais Christophe Perruchet, Marc Priel 2000
[48] Uncertainty, calibration, and probability; the statistics of scientific C. F. Dietrich 1991 Adam Hilger, Bristol; ISBN: 0-7503-
and industrial measurement 0060-4
[49] Uncertainty models for knowledge-based systems : a unified Irwin R. Goodman and Hung T. 1985 North-Holland, Amsterdam, ISBN: 0-
approach to the measurement of uncertainty Nguyen 444-87796-7
[50] Measurement uncertainty: methods and applications Ronald H. Dieck 1992, 1997 Instr. Soc. Of America, ISBN: 1-55617-
628-7

25
[51] Handbuch Validierung in der Analytik Stavros Kromidas 1999 Wiley-VCM ISBN: 3-527-28748-5
[52] Calculation & Reporting Uncertainties of Measurements in Testing N. Kukadia, Tenby Industries Limited, 1996
Birmingham, UK
Some examples of special topics
[53] Uncertainty of quantitative determinations derived by cultivation of
microorganisms, Centre for metrology and accreditation, Helsinki,
2002

[54] Guidelines for expressing the Uncertainty of Measuerement re- Philips, Eberhardt and Parry, NIST; 1997 Taking into account uncorrected bias download from
sults containing uncorrected Bias Journal of Research of the Nat. Inst. http://nvl.nist.gov/pub/nistpubs/jres/102/
Of Standards and Technology, 102, S. 5/j25phi.pdf
577
[55] Calculation of Measurement Uncertainty Using Prior Information S.D. Phillips, W.T. Estler, M.S. Leven- 1998 Treating bias in uncertainty evaluation download from
son, K.R. Eberhardt http://nvl.nist.gov/pub/nistpubs
[56] Uncertainty and Dimensional Calibrations Doiron and Stoup, Journal of Research 1997 Examples for variables contributing in calibra- download from
of the Nat. Inst. Of Standards and tion http://nvl.nist.gov/pub/nistpubs
Technology, 102, S. 647
[57] An Interpretation of the Guide to the expression of Uncertainty in R.N. Kacker, NIST Techni-Pubs SP 2000 20$, ordering at
Measurement 500-244 http://nvl.nist.gov/pub/nistpubs
[58] Observing validation, uncertainty determination and tracebility in Magnus Holmgren, NT Techn. Report 1998 www.vtt.fi/nordtest/tec403.htm
developing Nordtest test methods 403
[59] Uncertainty of pH Measurements H. Jensen, L. Nielsen, Nordtest Techn 1995 Nordtest, P.O. Box 116, FIN-02151
Report 284 Espoo, Finland, Phone +358-0-455-
4600
[60] Traceable calibration and uncertainty of measurements in scan- Halldur Gudmundsson, Thomas 1994 Nordtest, P.O. Box 116, FIN-02151
ning electron microscopy Runarsson, Nordtest Techn Report Espoo, Finland, Phone +358-0-455-
252 4600
[61] Traceable calibration and uncertainty of measurements and tests B. Steffen, H.Kallio, Freygardur Thor- 1994 Nordtest, P.O. Box 116, FIN-02151
steinsson, Nordtest Techn. Report 251 Espoo, Finland, Phone +358-0-455-
4600
[62] Calibration, traceability and uncertainty Nordtest-Seminar in Espoo, NT Techn. 1995
Report 305
[63] A view on the assessment of the technical competence of testing Jarl Forstén, Nordtest Techn Report 1991
laboratories 149
[64] The use of uncertainty estimates in testing Nordtest SP Report 1993:47 1993
[65] Nordtest’s Views on Measurement in Metrology and Testing Nordtest, Position Paper 1994 Nordtest, P.O. Box 116, FIN-02151
Espoo, Finland, Phone +358-0-455-
4600
[66] Uncertainty - to a certain level Nordtest Position paper 005 1998 Ways for assessing uncertainties for different Nordtest, P.O. Box 116, FIN-02151
situations of tests and measurement Espoo, Finland, Phone +358-0-455-
4600
[67] Estimating Measurement Uncertainty Erwin Achermann, Oscar Chinellato; 2000 Theoretical Scientific Computing: The "Maxi- ETH Zürich, Institute of Scientific Com-
Technical Report 346. ETH Zürich mum Likelihood (fitting of a) Functional Rela- puting;
tionship" (MLFR) http://www.inf.ethz.ch/research/wr/publi
cations/tr.html
[68] Instrument and Apparatus - Measurement Uncertainty, part 1 ANSI/ASME PTC 19.1-1985 (1990) 1985 / 1990 ASME, New York
[69] Measurement uncertainty of radiated emissions David A. Hill, Motohisa Kanda 1997
[70] The fitness for purpose of analytical methods Eurachem 1998 Method validation studies ISBN 0-948926-12-0

26
[71] ILAC-G13. Guidelines for the requirements for the Competence of ILAC 2000 http://www.ilac.org/
Providers of Proficiency Testing Schemes (uncertainty assess-
ment in the Annex)
[72] ILAC-G8. Guidelines on Assessment and Reporting of Compli- ILAC Rules for supplier and client concerning the http://www.ilac.org/
ance with Specifications compliance of a product
[73] NAMAS NIS 80 NAMAS NIS 80
[74] Uncertainties of measurement for NAMSA Electrical product NAMAS NIS 20 1989
testing laboratories NIS 20, NAMAS
[75] Estimation and Expression of Measurement Uncertainty in Chemi- Nordic Committee on Food Analysis, 1997
cal Analysis NMKL Procedure 5
[76] Measurement uncertainty considerations for co-ordinate measur- S.D. Phillips, B. Borchardt, G. Caskey 1993
ing machines
[77] Step by step estimate of uncertainty of test results J.S. Morowski, EMPA, Switzerland, at 1994 Examples of estimation of uncertainties from EUROLAB
2nd EUROLAB Symposium "Testing practice
for the year 2000", Florence, April
1994
[78] Assessment and Practical Use of Uncertainties in test results Hans Andersso, SP, Sweden at 2nd 1994 Calculating measurement uncertainties EUROLAB
EUROLAB Symposium "Testing for
the year 2000", Florence, April 1994
[79] Uncertainty in testing Norbert Müller, arsenal research, 1999 Important aspects of harmonisation of the use EUROLAB - in discussion
Austria of uncertainties
[80] Practical Experiences with Uncertainty Evaluation at VTT Chemi- Veikko Komppa, VTT, Finland at 2nd 1997 Examples from practice, graphics: measure- Eurachem/D / EUROLAB-D
cal Technology Workshop "Measurement Uncertainty ment uncertainty against concentration
in Chemical Analysis", Berlin Septem-
ber 1997
[81] Interlaboratory Study and Validation Data Stephen L.R. Ellison, Laboratory of the 1997 Use of interlaboratory comparisons for uncer- Eurachem/D / EUROLAB-D
government Chemist, UK at 2nd Work- tainty determination
shop "Measurement Uncertainty in
Chemical Analysis", September 1997

[82] Autosamplers - a major uncertainty factor in HPLC Analysis Pre- S. Küppers, B. Renger, V.R. Meyer, 2000
cision LC,GC Europe, 2/2000
[83] The evaluation of the uncertainty in knowing a directly measured I.H. Lira, W. Wöger, PTB, Braun- 1998
quantity schweig, Germany; Pontificia Universi-
dad Catolica de Chile, Santiago
Meas. Sci. Technol. 9, 1167-1173
[84] Is the estimation of measurement uncertainty a viable alternative S. Küppers, Accred. Qual. Assur. 3 1998
to validation? 1998 p. 412-415
[85] NIST list of recent and new documents NIST homepage, search option Good variety of publications, e.g. special http://nvl.nist.gov/pub/nistpubs/nistpubs
literature on measurement uncertainty, some .htm
can be downloaded
[86] Verlauf und Ergebnisse einer geordneten Ringschätzung der J.S. Morkowski, Umweltbundesamt, See Example 3 of this document 7,67 Euro;
Unsicherheiten von Emissionsmessungen (Process and results of Texte 54/99, Berlin, 1999 www.umweltbundesamt.de
an ordered interlaboratory estimation of the uncertainties from
emission measurements)

27