120  Measurement Automation Monitoring, Apr. 2016, no. 04, vol.

62, ISSN 2450-2855

Kamila GROMCZAK, Ksenia OSTROWSKA, Adam GĄSKA, Jerzy SŁADEK

CRACOW UNIVERSITY OF TECHNOLOGY, LABORATORY OF COORDINATE METROLOGY
37 Jana Pawla II Ave., 31-864 Cracow

The universal validation algorithm of coordinate

measuring methods
Abstract whose selected dimensions are measured using different measuring
systems, software, and the three above-mentioned coordinate
The paper presents the universal validation algorithm of coordinate measuring methods.
measuring methods determined by the analysis and adaptation of
appropriate validation parameters, techniques and models. The paper also
The issue of validation of measurement methods is the difficult
provides the proposals of acceptance/rejection criteria of the analysed subject, often overlooked and neglected but also desirable in any
measurement method. These criteria are a set of mathematical formulas field of measurement. Measurements of length using coordinate
selected from numerous worldwide research publications and metrological technique are a research field where validation of both standardized
studies, which have been tested, revised and adapted for the purposes of and non-standardized methods, is highly suggested. Therefore, an
the validation. adequate validation procedure has been developed. It has been
verified by the countless number of measurements. The hardest part
Keywords: validation algorithm, validation model, coordinate measuring
methods.
of the validation of the methods is the development or adaptation of
a validation model, which is a mathematical model and enables the
decision whether to adopt or reject a validated method.
1. Validation of the laboratory methods Knowledge about the validation of measurement methods is
reduced to the [N1] point 5.4. Most auditors of management
PN-EN ISO / IEC 17025: 2005 indicates that laboratories should systems in laboratories do not explore the validation area. It is
prefer the current measurement methods described in the well- only assessed by the question whether the methods used in the
known international standards. However, when there is a need to laboratory are developed on the basis of standards. However, most
develop and implement specific and individual methods, it should laboratories use methods only partially developed on the basis of
be done as a result of the planned activities – the validation standards and consider them to be validated.
process. There are lots of factors that affect the proper Internationally, the issue of validation of measurement methods,
organization of the validation process such as professional in the field of general metrology, is developed by the National
laboratory staff with extensive knowledge and measuring skills, Measurement Institutes such as PTB, NPL or NIST.
adequate resources provided by the top management of the In coordinate metrology, the effective methods of evaluation of
laboratory which include, first of all, appropriate measuring the measurement accuracy have been developed as exemplified by
machines with software and effective communication. All non- the series of standards ISO 15530 but so far the universal
standardized methods should be agreed with a potential client, and validation model of these methods has not been presented.
well-validated. Validation of measuring methods, as the process of
confirming that the chosen method used to perform a particular
specified measurement is suitable for the intended purpose,
1.2. The universal validation algorithm
provides the reliability and consistency of the measurements. That of coordinate measuring methods
is why it is so important both in the scientific community as well
as in production engineering. 1. ANALYSIS OF COORDINATE METHODS [1] [8]
Validation of the measuring methods is a process that is very
difficult to plan and implement on the one hand, and on the other 1.1. Substitution Method - validated, developed on the basis of
hand - an interesting issue that is rarely discussed in scientific ISO/TS 15530-3:2011 [N2]
community. Well-validated method may often become Method is based on repeated measurements of a calibrated
a bargaining chip for companies when choosing a partner - standard, whose shape and dimensions are similar to the tested
a laboratory for cooperation [1]. object.
This method makes possible experimental determination of
1.1. Validation of coordinate measuring realized measurement accuracy using the calibration of
a reference object or a standard of size and shape similar to the
methods measured object. Not the sources of errors are important for the
measurement results performed with it, but the assessment of
Intensive development of coordinate measuring technique requires
their effects in form of deviations from the nominal dimension.
better and better, validated methods for assessing the accuracy of
The substitution method can be used in mass production, where
measurements. Mandatory condition for allowing their daily use is
from the economic point of view, it becomes profitable.
their validation. For this purpose the universal validation algorithm
However it is required then to ensure stable conditions for the
containing all the elements required by point 5.4 of ISO 17025 [N1]
measurement realization.
and internal accreditation requirements of Polish Centre for
Accreditation has been developed. These elements include, inter
1.2. Multi-position method - non-validated, developed on the
alia, the choice of the most important parameters of the validation
basis of ISO/TS 15530-2 [N3]
and appropriate validation techniques. Validation technique must
Method is based on repeated measurements of an uncalibrated
provide the best laboratory and environmental conditions for
object, taking into account the various distribution of the points.
carrying out the measurements so that the results of the validation
The object should be measured five times at each designated
parameters were reliable and unambiguous. In the case of coordinate
position.
measuring methods two techniques of validation are considered:
The analysis of the results is made on the basis of the analysis
comparison technique with another method and interlaboratory
of standard deviations of the measured characteristic, separating
comparisons. The algorithm was developed for calibration
two components of uncertainty: the impact of CMM repeatability
laboratory and takes into account the validation of three measuring
for particular and the effect of CMM geometric errors in
methods: multi-position method, substitution method and Virtual
connection with the influence of the of measuring tip
CMM method. Specifically for the purpose of the validation, the
qualification process.
appropriate multi-feature check was developed and manufactured,
Measurement Automation Monitoring, Apr. 2016, no. 04, vol. 62, ISSN 2450-2855  121

1.3. Virtual CMM Method - non-validated, developed by the where:

laboratory scientists – measurements for i=1,2,3,..,n
This method involves the measurement simulation using – average of all measurements
a virtual machine. A simulation method is used when there is no – number of all measurements.
solution to the problem in an analytical way, or if the
measurement model is too complicated, and the estimation of the
measurement uncertainty is difficult, or even impossible.
This method is based on constructing the simulative model of a
measurement made in the considered system. The model should
include all of the error sources that are known and possible to
predict, and use the information about the possible error values
in each place of the measuring volume of the system.

2. ANALYSIS OF LABORATORY COMPETENCE

 calibrations in accordance with the requirements of the

standard ISO 17025
 coordinate measuring machines with appropriate calibration
and measurement capability CMC
 proficiency and qualification of laboratory staff Fig. 1. Virtual CMM Method
 housing and environmental conditions
 literature analysis 4. SELECTION OF VALIDATION TECHNIQUE

3. SELECTION OF VALIDATION PARAMETERS [1] 4.1. Comparison with the validated method
 multi-position method with the substitution method
In the case of coordinate measurements, validation parameters are  Virtual CMM method with the substitution method
expanded uncertainties U calculated for the following methods:
4.2. Interlaboratory comparison
Multi-position method Interlaboratory comparison with a reference laboratory in
expanded measurement uncertainty U accordance with the requirements specified by the standard ISO
17043 [N4] and the documents of the accreditation body. In the
case of calibration laboratories, the reference laboratory should
= ∙ + + + + (1) be selected in accordance with the criterion of independence and
accuracy, which means the choice of a laboratory with
appropriate scope of accreditation and calibration and
where:
measurement capability CMC.
- expanded uncertainty of the measurement,
- coverage factor,
5. SELECTION OF THE MEASURING CHECK
- standard uncertainty contribution originating from
repeatability of the CMM,
- standard uncertainty contribution related to the
geometrical errors of the CMM,
- standard uncertainty of the correction that is applied to
the average measurement result on the basis of the average
distance proportional error of length measurement,
- uncertainty related to the measurement of diameter,
- uncertainty connected to thermal influences.
Fig. 2. EUMETRON Multi feature check
Substitution Method
expanded measurement uncertainty U
Multi feature check - complex measuring standard for the
purposes of measurement accuracy assessment and the
= ∙ + + + (2) determination of measurement uncertainty for nearly all features
and dimensions applicable to the coordinate measuring
technique.
where: Due to the nature of the validation process and the accuracy of
- expanded uncertainty of the measurement coordinate systems, this standard is suitable for performing lots
- coverage factor of measuring tasks on CMMs thanks to the diversity of its
- standard uncertainty of the parameter of the calibrated geometric structure, which includes e.g. cylindrical holes of
workpiece or measurement standard various diameters and depths, circle hole, flat, stepped and end
up - standard uncertainty of the measurement procedure surfaces, short and long tapers with various opening angles.
ub - standard uncertainty of the systematic error
uw - standard uncertainty associated with the variations in the 6. SELECTION OF VALIDATION CRITERIA –
uncalibrated workpieces MATEMATICAL MODELS [2- 10]

Virtual CMM Method Model of statistical consistency control

uncertainty standard deviation
The validation model is based on the concept of consistency
∑( − )
= (3) control, which refers to classical statistics such as the weighted
−1 mean of the methods and the chi-squared test. To apply this
method, the laboratory must meet three important requirements:
122  Measurement Automation Monitoring, Apr. 2016, no. 04, vol. 62, ISSN 2450-2855

- measurements must be taken in adequate, environmental Model of metrological compatibility

conditions of stability
- the measurement program must comply with the principle of | − |
independence, the measurements made by the reference ≤ (9)
( )+ ( )−2 ( , ) ( ) ( )
method cannot affect the completed measurements made
previously using the method being validated
where:
- Gaussian distributions need to be assigned to each measurand.
, ( ) - values being validated with the standard uncertainties,
, ( ) - reference values with the standard uncertainties,
This model starts with the calculation of the weighted mean (4),
( , ) - correlation coefficient ( , ) between the
the so called Reference Value (RV):
variables and
/( ( ) + /( ( ) + / ( )
= (4) ∑ [( − ̅ )( − ̅ )]
1/ ( ) + 1/ ( ) + 1/ ( ) ( , )= (10)
{[∑ ( − ̅ ) ] ∙ [∑ ( − ̅ ) ]}
where: and
, , - mean values of the results obtained by multi-position - the threshold determined on the basis of the measurements
method , substitution method and virtual CMM method , and calculations for coordinate metrology.
( ), ( ), ( ) - uncertainties calculated according to the
particular method. The number in the denominator (9) is related to the correlation of
pdfs represented by [ , ( )] and [ , ( )].
The next step of the model is to perform the chi-squared test The application of this model is possible if the measurements
which involves determining the probability of and is (measured value with its associated standard uncertainty) are
calculated as (5): traceable to the same reference what gives the VIM’s definition
of metrological comparability [N5, definition 2.46]. According
to another definition of metrological traceability found in VIM
Pr { ( )> } < 0.05 (5) [N5], the results must be related to a practical realization of the
reference through a documented chain of unbroken calibrations,
where: each contributing to the total measurement uncertainty [N5,
( − ) +( − ) +( − ) definition 2.41].
= (6) The result of the metrological compatibility test makes it
( )+ ( )+ ( )
possible to determine whether measurement results with their
corresponding uncertainties, obtained using the validated
where:
coordinate method [ , ( )] are sufficiently comparable with
- parameter describing the probability distribution of the
the set of results obtained from the reference method
results , , referenced to the value in view of their [ , ( )], and can thus be considered acceptable.
individual variances of the results, The answer is positive, because the fulfilled test of metrological
and compatibility demonstrates the insignificance of differences
= − 1 - degrees of freedom between the measured values and , in view of their
uncertainties ( ) and ( ).
This test, if fails, assumes the consistency of coordinate Taking into account the adopted economic consequences, the
measuring methods used. In this case, next steps have to be
conventional value of is 2, but the threshold may be adjusted to
performed. If the test rejects the hypothesis about the consistency
a different set value for various purposes. In this case, the
of the results obtained using the considered methods, the
threshold is chosen from the set of values of calculated
validation ends with a negative result.
metrological compatibility models.
The lower the threshold , the better to prove the consistency
The standard uncertainty associated with the reference value
and reliability of the results of measurements.
( ) needs to be calculated using (7):

1 1 1 1 7. DEVELOPMENT AND APPROVAL OF

= + + (7) “VALIDATION PROGRAM OF COORDIINATE
( ) ( ) ( ) ( ) MEASURING METHODS”
where: The validation program of coordinate measuring methods
( ) is the standard uncertainty of determination of the specifies the exact people and dates for the various stages of
reference value. validation including measurements and statistical studies.
The validation program should be approved by the top
The most important is the “validation acceptance interval” for management of the laboratory to provide adequate financial
the results obtained with considered methods: resources for the implementation of this program.
< − ( ); + ( )> (8)
8. MEASUREMENTS
This interval is the proposition of a mathematical range, which
Measuring chosen values of the multi feature check using three
should overlap with the intervals that contain the true value of
multi-position, substitution and Virtual CMM methods on the
the measured quantity, obtained using the methods being
coordinate measuring machine PMM 12106 intended for
validated.
calibration spatial reference objects, with proven accuracy CMC
If all the three intervals < − ( ); + ( ) >, < − = 0.6 µm + 0.7L µm/m and calibrated length standards that
( ); + ( ) >, < − ( ); + ( ) > have the common
ensures the accuracy of the performed measurements.
part with the validation acceptance interval then the validation Due to the specific nature of coordinate measurements, the
ends with the positive result and two methods that were under standard described in Section 5 possesses many distinct
validation may now be considered as the validated methods. geometrical features and dimensions (Fig. 2). Apart from
different types of dimensions (from a metrological standpoint),
Measurement Automation Monitoring, Apr. 2016, no. 04, vol. 62, ISSN 2450-2855  123

e.g. external, internal, mixed and intermediate, all geometrical mathematical analysis of the estimation of the results and their
features, such as: shape, direction, position, and even radial and uncertainty according to numerous global publications and
axial runout were taken into account in the measurements. metrological studies. These models are confirmed by the validation
Measurements should be made by laboratory staff with proven of two – developed and improved in the Laboratory of Coordinate
measurement proficiency, selected from accredited personnel, Metrology (LCM) methods: Multi-position method based on
who meet, specified by the management, eligibility criteria for repeated measurements of an uncalibrated object and the matrix
performing calibration, results authorization and operation of the method used to build the system of virtual measuring machine to
specified coordinate system. assess the accuracy of the measurement on Line. The results of
Measurements should be taken under normal laboratory work validation of the mentioned above method using the model of
with appropriate supervision of environmental conditions. All statistical consistency control are described in details in [10].
the intermediate data associated with the preparation, The presented algorithm and the developed models of validation
measurements and analysis of the results must be protected in of measurement methods in calibration laboratories are the
order to ensure full traceability of the validation process. response to current market requirements and the designation of
a new, broad research area. The recipients of the model can be
accredited calibration laboratories, research institutions and
national conformity assessment bodies. The model may also be, in
the future, used in the validation of measurement methods in
production conditions, because the calculation of measurement
uncertainty, as the basis for judgments on the conformity with the
specifications of the product geometry, will become the standard.

3. Conclusions
The presented algorithm is purposely developed for the
validation of coordinate measuring methods. It can be used to
meet the requirements of the standard ISO 17025 (point 5.4) with
the rational choice of a validation model. These presented models
of validation are the proposition of methodology that may be used
for validating the new methods that have recently been developed,
internal laboratory procedures and methods that are rarely used in
practice and cannot be considered as validated. Fulfilling the
criterion presented in this paper is the necessary condition for
Fig. 3. Multi-feature check mounted in the measuring space of PMM 12106
assuming the considered method to be a validated one. The works
on improvement of the models are now carried out in LCM.
9. STATISTICAL DEVELOPMENT OF VALIDATION With small amendments, the presented validation algorithm and
PARAMETERS models may also be used in other fields of science and may
contribute to significant facilitation in demonstrating that the used
Calculations of measuring uncertainties in accordance with methods are validated and produce correct results.
Section 3.
10. APPLICATION OF VALIDATION MODELS This work was partially funded by the Polish National Center for Research and
Development No: LIDER/024/559/L-4/12/NCBR/2013.
Calculation of the mathematical models in accordance with
Section 6. 4. References
Presentation of the results in tabular form and graphically.
If the calculated validation parameters, uncertainties, do not [1] Sładek J.: Accuracy of coordinate measurements. Publishing House of
meet the conditions specified in the established models, a Cracow University of Technology, ISBN 978-83-7242-643-7, Cracow
detailed analysis of the validation and measuring process and 2011.
possible change of parameters from Sections 3 to 6 are required. [2] Kacker RN, Kessel R, Sommer KD, Bian X: Comparison of statistical
11. ACCEPTANCE OR REJECTION OF COORDINATE consistency and metrological compatibility. XIX IMEKO World
MEASURING METHOD Congress Fundamental and Applied Metrology, 6-11, 2009, Lisbon,
Portugal.
The decision on acceptance or rejection of the results obtained [3] Kacker RN, Kessel R, Sommer KD: Assessing differences between
from the measurements using methods to be validated. results determined according to the Guide of Expression of
The results of the validation (positive or negative) should be Uncertainty in Measurement. Journal of research of the national
included in a detailed report. This report, in addition to the main institute of Standards and Technology, 115, 453-459, 2010.
decisions, should include the data from the various stages of [4] Kacker RN, Kessel R, Sommer KD: Metrological compatibility and
validation according to the present algorithm. statistical consistency. 10th International Symposium on Measurement
This report, since its approval, becomes a part of the supervised and Quality Control 2010, September 5-9.
document in the management system in the laboratory and, [5] Cox M.G.: The evaluation of key comparison data. Metrologia 39 (6)
therefore, is a subject to the verification by the Polish Centre for 2002 587-8.
Accreditation. [6] Kacker RN, Forbes A, Kessel R, Sommer KD: Bayesian posterior
predictive p-value of statistical consistency in interlaboratory
evaluations. Metrologia 45 (5) 2008 512-23.
[7] Kacker RN, Forbes A, Kessel R, Sommer KD: Classical and Bayesian
2. Model of validation of coordinate interpretation of the Birge test of consistency and its generalized
measuring methods version for correlated results from interlaboratory evaluations.
Metrologia 45 (3) 2008 257-64.
As a part of the presented algorithm, the validation models were [8] Sładek J., Gawlik K.: Looking for uncertainty of measurement -
also developed. They are the basis for accepting or rejecting Virtual Machines based on the Matrix Method using artificial neural
new/changed measuring methods and are the innovation in the field networks. IV International Congress on Precision Machining 2007,
of coordinate metrology. They are universal models that contain the Sandomierz-Kielce, 2007.
124  Measurement Automation Monitoring, Apr. 2016, no. 04, vol. 62, ISSN 2450-2855

[9] Delčev, S., Gučević, J., Ogrizović, V.: Necessity of involvement of Ksenia OSTROWSKA, PhD, eng.
calibration laboratories in proficiency testing schemes. 11th IMEKO
Adjunct in Laboratory of Coordinate Metrology in
TC14 International Symposium on Measurement and Quality Control, Cracow University of Technology. Participant of many
ISMQC 2013 173-6. research projects. Head of the project financed by
[10] Gromczak K., Gąska A., Ostrowska K., Sładek J., Harmatys W., Gąska NCBiR within the LIDER program. Area of research:
P., Gruza M., Kowalski M.: Validation model for coordinate measuring accuracy assessment of measurements using redundant
coordinate measuring systems, calibration of the optical
methods based on the concept of statistical consistency control. systems used for measurement of 3D objects.
Precision Engineering, doi:10.1016/j.precisioneng.2016.03.021.
[N1] EN ISO/IEC 17025:2005 -General requirements for the competence
of testing and calibration laboratories.
[N2] ISO 15530-3:2011 -- Geometrical product specifications (GPS) -- e-mail: [email protected]
Coordinate measuring machines (CMM): Technique for
determining the uncertainty of measurement -- Part 3: Use of
Adam GĄSKA, PhD, eng.
calibrated workpieces or measurement standards.
[N3] ISO/CD TS 15530-2 GPS – Use of multiple measurement strategies. Adjunct in Laboratory of Coordinate Metrology.
[N4] ISO/IEC 17043:2010 - Conformity assessment -- General Participant of many research projects. Now he is a head
requirements for proficiency testing. of the project financed by the NCN regarding modeling
of 5-axis kinematic systems (SONATA program). He
[N5] International Vocabulary of Metrology – Basic and General was the head of the project financed by NCBiR within
Concepts and Associated Terms (VIM 3rd edition) JCGM the LIDER program, regarding usage of Virtual CMM
200:2012. models in industrial conditions. He is also interested in
mobile measuring systems like AACMMs or Laser
_____________________________________________________ Tracking systems.
Received: 07.01.2016 Paper reviewed Accepted: 02.03.2016

e-mail: [email protected]

Kamila GROMCZAK, MSc, eng. Prof. Jerzy SŁADEK, DSc, eng.

Quality manager in accredited calibration laboratory - Professor in Cracow University of Technology. He

Laboratory of Coordinate Metrology in Cracow organized Coordinate Metrology Laboratory - one of
University of Technology. Participant of many research leading calibration laboratories in Europe. His
projects. Founder of the consulting firm MAXYMA. professional, scientific and educational activity is related
Trainer and advisor in management systems according to coordinate metrology. He is the author of 343
to ISO 17025 in testing and calibration laboratories. publications and 3 scientific books including one
Area of research: coordinate measuring technique fundamental: Coordinate Metrology - Accuracy of
mainly validation of measurement methods. Systems and Measurements.

e-mail: [email protected] e-mail: [email protected]