1
CDQC_MK
What’s new/different in ISO 17025:2017
“Decision rule associated to the
measurement uncertainty”
Katerina Brezovska
Center for Drug Quality Control (OMCL_MK)
ISO 17025:2017 training for OMCLs ,
19 - 21 September 2018, Strasbourg, France
Outline 2
Introduction
Comparison to the version 17025:2005
Description of the clause
Which OMCL activities are impacted by
the clause
How to satisfy the requirements in OMCLs
Introduction
3
Definitions
Uncertainty of measurement (UoM):
a parameter associated with the result of a
measurement that characterises the dispersion of
the values that could reasonably be attributed to
the measurand. ISO/IEC GUIDE 98-3:2008
Decision rule
Rule that describes how measurement uncertainty
is accounted for when stating conformity with a
specified requirement. ISO IEC 17025:2017
Introduction
4
Conformity assessment
probabilities using decision rule
Decision Rule
Compliance Non - compliance
√ √
Compliance Non - compliance
(false positive) False negative
Each decision rule orientation implies different levels of risk!
Introduction
5
Link between the decision rule and
the level of risk for wrong decision
Lower
Upper
Specification
Specification
Limit
Limit
Result
GL=Guard Limit
(includes Uncertainty)
GL GL
Simple pass and fail High Risk
Sample fail if the measured value exceed Risk that part of
the limit by the expanded uncertainty the result is OOS
Acceptance zone is whit in the Guard limit Low risk
ISO IEC 17025:2017 vs ISO IEC 1725:2005
Decision rule 6
Clear definition of 3. Terms and definitions
the decision rule
ISO IEC 17025:2017 7.1 Review of requests,
tenders and contracts (7.1.3)
Specified in Clause
7.8.6 Reporting statements of
conformity (7.8.6.1 and 7.8.6.2)
Informative
A.2 Establishing metrological traceability
ISO IEC 17025:2005
The concept of decision 5.10 Reporting of results
rule is recognised (5.10.3.1 and 5.10.4.2)
Description of the clause
7
ISO IEC 17025:2017
7 Process requirements
7.1 Review of requests, tenders and contracts
Example:
7.1.3 When the customer requests a statement of
conformity to a specification
CONTRACT or standard for the test
or
…In calibration
cases where(e.g. pass/fail,rule
the decision in-tolerance/out-of-
is not inherent to
tolerance), the specification
the specification limits, the or standard
decision ruleand
will the
be
applied…
decision rule shall be clearly defined….
or NEW
...The statement
….shall of conformity
be communicated willagreed
to, and be based on the
following
with, decision rule…
the customer.
Unless inherent in the requested specification or standard!
Example: Contract or Review of requests
Decision rule for content determination to be 8
communicated with the customer
Perform Estimate measurement
testing uncertainty
Evaluate the result
(95 % confidence)
The value is between The value exceed
LSL + 2u and USL -2ut LSL + 2u and/or USL -2ut
Apply procedure
Sample for OOS
passes investigation
u=combined standard uncertainty
Description of the clause 9
ISO IEC 17025:2017
NEW
7.8 Reporting of results
7.8.6 Reporting statements of conformity
7.8.6.1
Example:…the laboratory shall document the decision rule
employed, taking into TEST
account the level of risk
REPORT
(such as false accept and false reject and statistical
assumptions) associated with the decision rule employed, and
…The
apply theresult of rule.
decision determination of content of active
ingredient complies with the specification limit.
NOTE Where the decision rule is prescribed by the customer,
Theregulations
decisionorrule states
normative that “the
documents, result consideration
a further is compliant of
if the
the value
level of is
riskbetween lower limit of specification
is not necessary.
+ 2u ….statement
7.8.6.2 and upperclearly limit identifies:
of specification -2u, at
probability
…c) of 95%”…
the decision rule applied (unless it is inherent in the
requested specification or standard).
ISO IEC 17025:2017 10
Informative in Annex A
A.2 Establishing metrological traceability
A.2.3 ….This approach, in which the specification
limits are imported as the source of uncertainty, is
dependent upon:
- the use of an appropriate decision rule to establish
conformity;……….
ISO IEC 17025:2017 vs ISO IEC 1725:2005
11
Measurement uncertainty
ISO IEC 17025:2017
7.6 Evaluation of measurement uncertainty
…shall evaluate…
…identify all contributions that are of significance,
including those arising from sampling…
…based on an understanding of the theoretical principles
or practical experience of the performance of the method..
ISO IEC 17025:2005
5.4.6 Estimation of uncertainty of measurement
…shall have and shall apply procedures…
…at least attempt to identify all components and
make a reasonable estimation..
…Based on knowledge and performance data…
Description of the clause 12
ISO IEC 17025:2017
7.6 Evaluation of measurement uncertainty
7.6.1 Laboratories shall identify the contributions to
measurement uncertainty.
When evaluating measurement uncertainty, all
contributions that are of significance, including those
arising from sampling, shall be taken into account using
appropriate methods of analysis.
7.6.2 A laboratory performing calibrations, including
of its own equipment, shall evaluate the
measurement uncertainty for all calibrations.
Example:
Identification of contributions to
measurement uncertainty 13
Description of the clause 14
ISO IEC 17025:2017
7.6.3 A laboratory performing testing shall evaluate
measurement uncertainty.
Where
Examplesthe test method precludes rigorous evaluation of
measurement uncertainty, …theoretical principles or practical
experience of the performance
Theoretical principles of the
(well-established method.
physical theory):
• Temperature effect on volumes and densities
NOTE 1 ...well-recognized
• IUPAC test method
quoted uncertainties specifies
for Atomic limits to the values of
weights
the major sources of measurement uncertainty and specifies the form
of presentation
Practical of theofcalculated
experience results of
the performance the laboratory
the method: is considered
to have satisfied
• Certified 7.6.3 bymaterials,
reference following the test method and reporting
instructions.
• Validation study data,
• Collaborative study data (e.g. establishment of chemical
NOTE 2 For a particular method where the measurement uncertainty
reference standards or validation of new test method)
of the results has been established and verified there is no need to
• PTS study data,
evaluate measurement uncertainty for each result if the laboratory
• Control charts.
can demonstrate that the identified critical influencing factors are
under control.
ISO IEC 17025:2017 vs ISO IEC 1725:2005
15
Measurement uncertainty
6.4 Equipment 6.5 Metrological traceability
ISO IEC 17025:2017 7.2 Selection, verification
and validation of methods
7.5 Technical records 7.8 Reporting of results
ISO IEC 17025:2005
5.6 Metrological traceability
5.4 Test and calibration methods and
method validation
5.10 Reporting of results
Measurement uncertainty
16
in ISO IEC 17025:2017
6.4 Equipment
6.4.1 …..
NOTE 2 … certified reference materials, specified
properties with certified values, their associated
measurement uncertainty and metrological
traceability…
6.4.5…shall be capable of achieving the
measurement accuracy and/or measurement
uncertainty required to provide a valid result.
6.4.6 ..shall be calibrated when… measurement
uncertainty affects the validity of the reported
results...
Example:
Calibration of the balance 17
Uncertainty of calibration of the balance
Uncertainty of the daily usage of the balance
(uncertainty of the weighing result)
Weight of standard
substance/sample
Under revision:
PA/PH/OMCL (12) 77 7R QUALIFICATION OF EQUIPMENT
ANNEX 8: QUALIFICATION OF BALANCES
Appendix: Estimation of the Uncertainty of calibration
Measurement uncertainty
18
in ISO IEC 17025:2017
6.5 Metrological traceability
6.5.1 …documented unbroken chain of calibrations,
each contributing to the measurement uncertainty,
linking them to an appropriate reference.
A.2 Establishing metrological traceability
A.2.3 ….This approach, in which the specification
limits are imported as the source of uncertainty, is
dependent upon:
- the use of an appropriate decision rule to establish
conformity;……….
Measurement uncertainty
19
in ISO IEC 17025:2017
7.2 Selection, verification and validation of methods
7.2.1 Selection and verification of methods
7.2.1.1 …shall use appropriate methods and procedures
for all laboratory activities and, where appropriate, or
evaluation of the measurement uncertainty ...
7.2.2 Validation of methods
7.2.2.1…NOTE 2…f) evaluation of measurement
uncertainty of the results…
7.2.2.3… NOTE Performance characteristics can
include, but are not limited to, measurement range,
accuracy, measurement uncertainty of the results…
Measurement uncertainty
20
in ISO IEC 17025:2017
7.5 Technical records
7.5.1…identification of factors affecting the
measurement result and its associated
measurement uncertainty.
Measurement uncertainty
21
in ISO IEC 17025:2017
7.8 Reporting of results
7.8.3 Specific requirements for test reports
c) where applicable, the measurement uncertainty… when:
- it is relevant to the validity or application of the test results;
- a customer's instruction so requires, or
- the measurement uncertainty affects conformity to a
specification limit;
7.8.5 Reporting sampling - specific requirements
…information required to evaluate measurement
uncertainty for subsequent testing or calibration…
Which OMCL activities are
22
impacted by the clause?
Decision rule Compliance testing activities
7.1 Review of requests, tenders
Change in Documentation
and contracts
4. Management 7. Process
requirements requirements
4.4 SOP for Review of 7.1 SOP for Review of
requests, tenders and requests, tenders and
contracts contracts
Include clearly
defined decision
rule in the contract
Which OMCL activities are 23
impacted by the clause?
Decision rule Compliance testing activities
Change in Documentation
7.8.6 Reporting statements of Statement of conformity
conformity Reporting of results
5. Technical 7. Process
requirements requirements
5.10 SOP for 7.8 SOP for Reporting of
Reporting of results results
Documentation of the
employed decision rule
Decision rule in test
report
Which OMCL activities are
24
impacted by the clause
Measurement uncertainty
ALL quantitative testing activities
6.4 Equipment
Only Editorial
changes in the
documentation
6.5 Metrological traceability
7.2 Selection, verification and validation of methods
7.6 Evaluation of measurement uncertainty
Which OMCL activities are
25
impacted by the clause
Measurement uncertainty
Sampling activities
7.6 Evaluation of measurement Change in Documentation
uncertainty Reporting of results
7. Process requirement
7.6 SOP Evaluation of measurement uncertainty
Identify and Take into account contributors
arising from the sampling
7.8. SOP for Reporting of results
Report sampling/information regarding
measurement uncertainty
How to satisfy the
26
requirements in OMCLs
OMCL Guidelines:
PA/PH/OMCL (13) 82 2R “Validation of analytical method”
PA/PH/OMCL (13) 113 2R “Evaluation and reporting of results”
Under revision:
PA/PH/OMCL (xx) xx UNCERTAINTY OF MEASUREMENT
PA/PH/OMCL (xx) xx
UNCERTAINTY OF MEASUREMENT 27
CORE DOCUMENT
1. INTRODUCTION … applicable to all activities
(compliance and other testing activities)…
2. SCOPE
3. GLOSSARY AND DEFINITIONS
4. GENERAL OMCL POLICY FOR EVALUATION OF UoM
5. GENERAL APPROACH FOR ESTIMATION OF THE UoM
5. DECISION ON COMPLIANCE OF THE TEST RESULT
TAKING INTO ACCOUNT THE UoM
6. REPORTING THE RESULT WITH UoM
PA/PH/OMCL (xx) xx
UNCERTAINTY OF MEASUREMENT General
28
Approach for Compliance testing activities:
tests performed, using official or validated analytical procedures to verify
that a pharmaceutical substance or medicinal product examined conforms
with the specification limits given in the monograph or in the marketing
authorisation.
Employing well-recognised methods,
The results are obtained by following the described
analytical procedure and reporting instructions,
Provided that all uncertainty contributors are under control:
testing is performed by qualified personnel
using suitable reference standards
calibrated/qualified equipment,
system suitability criteria are satisfied
the repeatability is evaluated against pre-defined
acceptance criteria.
The ISO 17025 requirements (7.6.3) are fulfilled !
PA/PH/OMCL (xx) xx
UNCERTAINTY OF MEASUREMENT General
29
Approach for Other testing activities:
• screening,
• analysis of unknown products,
• trace analysis,
• validation of methods,
• setting limits for performance
test…
• or whenever OMCL considers
it necessary.
Rigorous evaluation of
measurement uncertainty!
PA/PH/OMCL (xx) xx
UNCERTAINTY OF MEASUREMENT 30
ANNEXES
Annex 1: Estimation of measurement uncertainty
using Bottom-up approach
Annex 2: Estimation of measurement uncertainty
using Top-down approach:
2.1 Use of repeatability data for the estimation of UoM;
2.2 Use of data from validation studies for the estimation of UoM ;
2.3 Use of data from control charts for the estimation of UoM;
2.4 Use of certified reference materials for the estimation of UoM;
2.5 Use of data from collaborative studies for the estimation of UoM;
2.6 Use of data from PTS for the estimation of UoM
How to satisfy the requirements in
OMCLs ? 31
The level of risk for Conformity
assessment if decision rule is
not applied
Ad Hoc
methods
Published
methods
In house
developed
methods
MAH methods
Compendial
methods
How to satisfy the requirements?
Approach of CDQC_MK 32
1. Compliance testing activities
STATEMENT OF CONFORMITY IS REQUIRED!
7.6.3 is satisfied by following the
Defined specification limits test method and reporting
instructions.
Well-recognized test methods
The decision rule inherent in
Compendial methods the requested specification
Methods in marketing
7.1 and 7.8.6 satisfied
authorization documentation
In house developed methods The decision rule not inherent
in the requested specification
Published methods
Apply the decision rule
How to satisfy the requirements?
Approach of CDQC_MK 33
2. Other quantitative testing activities
Necessary to estimate the measurement uncertainty and
apply a decision rule where conformity assessment is required
Confirming out-of-specification results, especially in case
when the test could not be repeated
Setting limits for performance tests of measurement
apparatus and critical parameters of methods
Adhoc methods
How to satisfy the requirements?
Approach of CDQC_MK 34
Main activity:
Harmonize the Documents/Records of the Quality System of
the laboratory to the requirements stated in the ISO
17025:2017
Additional activities related to Decision rule:
1. Clearly define in the contract with the customer:
in which cases estimated uncertainty of measurement
will be taken into account in the statement of
conformity to a specification;
in which cases the uncertainty of measurement will be
reported;
2. Communicate the decision rule applied
3. Include the decision rule applied, in the Test Report when
not inherent in the specification.
How to satisfy the requirements in
OMCLs ? 35
7.8.2.2. “ the laboratory shall be responsible for all the information
provided in the report, except when the information is provided by the
customer…a disclaimer shall be put on the report when the information
is supplied by the customer and can affect the validity of result..”
Example: Use of uncertainty of measurement of purity of reference
standard, provided by manufacturers, with a certificate in estimation of
the uncertainty budget.
In case this is not provided by the manufacturer:
Uncertainty of the measurement of purity of reference standard is
considered negligible in overall uncertainty ?
Question open for discussion !
How to satisfy the requirements in
OMCLs ? 36
Depending on a case by case!
Based on a scientific judgment!
• The same standard is used for method validation..
• Standard is certified by the same method…
In certain cases it may be necessary to estimate the
uncertainty of the purity of reference standard.
Summary 37
New ISO IEC 17025:2017 applies the same
principles regarding measurement uncertainty and
decision rules associated to the measurement
uncertainty compared to the previous version, but
described in a more prescriptive way.
The clause for Decision rule is clearer, giving
choices to agree with the customer, unless
required by specification, regulations or standards.
38
“The evaluation of uncertainty is neither a routine task nor a
purely mathematical one; it depends on detailed knowledge of
the nature of the measurand and of the measurement method
and procedure used.
The quality and utility of the uncertainty quoted for the result of a
measurement therefore ultimately depends on the
understanding, critical analysis, and integrity of those who
contribute to the assignment of its value.”
Eurachem / CITAC Guide CG 4, Quantifying Uncertainty in Analytical
Measurements, (3rd Edition) (2012)
THANK YOU FOR YOUR
ATTENTION
