The new ISO

16140 standards.

Paul in ’t Veld,
Food and Consumer Product
Safety Authority (NWVA),
the Netherlands
Background (1)

› CEN-ISO standards are developed › Popular standard, currently more

by ISO TC34/SC9 and CEN than 100 alternative methods
TC275/WG6. validated
› In 2003 the ‘old’ ISO 16140 › In 2005 it became clear that
(validation of alternative there was a need to revise the
methods) was published. 16140 and there was a need for
more standards on validation.
› This standard was published after
10 years of development. › SC9 and WG6 decided in 2006 to
set up a working group for this.
› Starting point was the EU
(EURECA) project called Microval. › This working group (WG3) started
in 2006, under ISO SC9 lead,
› Developed to accommodate the with the following mandate:
need for fast and reliable
alternative methods

AOAC Europe/NMKL Nordval symposium Oslo 2

4 June 2019
Background (2)

WG 3 (method validation) 5. Development standard on

mandate: intermediate validation
1. Revision ISO 16140 (2003) 6. Development standard on
validation confirmation
2. Development standard on
methods
verification
3. Development standard on
validation standardised
reference methods
4. Development standard on
single lab validation

AOAC Europe/NMKL Nordval symposium Oslo 3

4 June 2019
Current status “WG3” standards (May 2019)

ISO 16140-1: Vocabulary (publised July 2016)

ISO 16140-2: Protocol for the validation of alternative (proprietary)
methods against a reference method (publised July 2016)
ISO 16140-3: Protocol for the verification of reference and validated
alternative methods implemented in a single laboratory
(in preparation for FDIS vote)
ISO 16140-4: Protocol for single-laboratory (in-house) method
validation (ready for FDIS vote)
ISO 16140-5: Protocol for factorial interlaboratory validation for
nonproprietary methods (ready for FDIS vote)
ISO 16140-6: Protocol for the validation of alternative (proprietary)
methods for microbiological confirmation and typing procedures (ready for
FDIS vote)
ISO 17468: Technical requirements and guidance on establish-ment or
revision ofNordval
AOAC Europe/NMKL a standardized
symposium Oslo reference method (published July 2016) 4
4 June 2019
 To facilitate the selection
of the appropriate
standard a flow scheme
is included in the
introduction of the parts
3 to 6.

Starting point is that

methods should be
validated.

AOAC Europe/NMKL Nordval symposium Oslo 5

4 June 2019
 Are specific (e.g. legal) To validate alternative
START: Is the method validated Choose
NO requirements given to use ISO NO
(performance characteristics are given)? (proprietary) methods ISO 16140-2
16140 (2003) or ISO 16140-2?

To validate non- Choose

proprietary methods ISO 16140-5

To do a single- Choose
YES YES laboratory validation ISO 16140-4

To validate reference Choose

methods ISO 17468
Choose
Is the method validated according to ISO 16140-4? ISO 16140-2

YES

NO
Apply method only in that particular
laboratory (incl. scope extension)

Is the (food) category to be

Choose
tested in the scope of the NO For extension of the scope of a reference method
ISO 17468
method?
For extension of the scope of an alternative (proprietary) method validated Choose
according to ISO 16140-2 ISO 16140-2
YES
For extension of the scope of a non-proprietary method validated according to Choose
ISO 16140-5 ISO 16140-5

For use of the (food) type in a single laboratory, in the case of:
a] an alternative (proprietary) method validated according to ISO 16140-2 or
Choose
Choose b] a non-proprietary method validated according to ISO 16140-5 or
ISO 16140-4
ISO 16140-3 c] a reference method with performance characteristics or
AOAC Europe/NMKL Nordval symposium Oslo d] a reference method without performance characteristics
(verification)
6
4 June 2019
ISO 16140-2 (validation of alternative proprietary methods)

› Successor of ISO 16140 of sample/matrices (food

(2003). items)
› Basis is the comparison › Interlaboratory study
between a reference method establishes the ‘reproducibility’
and an alternative method. of the method using a single
› Protocol for qualitative and food item.
quantitative methods › Evaluation of the data using
› Both protocols have 2 phases; preset criteria, alternative
method can be better when
a method comparison study proven.
and an interlaboratory
study.
› Method comparison study
focusses on testing a diversity
AOAC Europe/NMKL Nordval symposium Oslo 7
4 June 2019
ISO 16140-2 (validation of alternative proprietary methods)

Qualitative study (MCS): Quantitative study (MCS):

› Sensitivity study: › Relative trueness study:
 use of (naturally) contaminated  comparison between (naturally)
samples, contaminated samples,
 minimum of 5 food categories each  minimum of 5 food categories each
having a minimum of 60 samples. having 15 samples per category.
 interpretation of data depending on a  grafical interpretation of the data
paired or unpaired study design. (Bland-Altman en scatter plots)
› RLOD study: › Accuracy profile study:
 determination of the relative level of  combination of evaluation of precision
detection using artificially and trueness (= accuracy) of the
contaminated samples, method.
 1 matrix per category, 20 samples per  6 samples each with 5 replicates for
matrix. each category tested.
› Inclusivity/exclusivity study: › Inclusivity/exclusivity study:
 use of 50/30 strains.  use of 50/30 strains.
(Salmonella 100 strains)

AOAC Europe/NMKL Nordval symposium Oslo 8

4 June 2019
ISO 16140-2 (validation of alternative proprietary methods)

Interlab. study is essential in

the validation of the method.

Other new aspects:

› Paired versus unpaired studies
› Acceptability Limits (AL).
› Division of matrices into
categories, types and items
› Broad range of foods validation
= > 5 food different categories
AOAC Europe/NMKL Nordval symposium Oslo 9
4 June 2019
ISO 16140-2 (validation of alternative proprietary methods)

› Part 2 focusses on the › Similar to this is part 6 with

independent validation of the exception that this is for
proprietary methods. confirmation methods
› Validates the entire method › So starting with a suspected
from thee food sample to colony until confirmed end
(confirmed) end result result (e.g. Maldi-Toff systems)
› Procedure is important for the › Focus on inclusivity/exclusivity
use of alternative methods with number of strains
according to European dependent on level of
legislation confirmation (genus or
(Directive 2073/2005) subspecies, ..)

AOAC Europe/NMKL Nordval symposium Oslo 10

4 June 2019
ISO 16140-3 (method verification) (1)

Definitions
› Validation: establishment of › Verification: demonstration
the performance that a validated method
characteristics of a method functions in the user’s hands
and provision of objective according to the method’s
evidence that the performance specifications determined in
requirements for a specified the validation study and is fit
intended use are fulfilled. for its purpose.

Part 3 is only applicable to validated alternative or reference methods!

AOAC Europe/NMKL Nordval symposium Oslo 11

4 June 2019
ISO 16140-3

AOAC Europe/NMKL Nordval symposium Oslo 12

4 June 2019
ISO 16140-3 (method verification) (2)

Parameters to be determined in verification.

Method Performance Implementation (Food) item
characteristic Verification verification

Qualitative Estimated LOD50 (eLOD50) ✓ ✓

Quantitative Intralaboratory
reproducibility standard ✓ Not applicable
deviation (SIR)

Estimated bias (eBias) Not applicable ✓

AOAC Europe/NMKL Nordval symposium Oslo 13

4 June 2019
ISO 16140-3 (method verification) (3)

Estimated LOD50
› Three protocols available.
Inoculation level of the test portion
Protocol 9 × LOD50 3 × LOD50 1 × LOD50 3 to 5 cfu Blank Total number of
/test /test /test /test replicates
portion portion portion portion

1 1 4 4 - 1 10

2 - 3 5 – 1 9

3 - - - 7 1 8

AOAC Europe/NMKL Nordval symposium Oslo 14

4 June 2019
ISO 16140-3 (method verification) (4)

Estimated LOD50
› Criterium eLOD< 4 x LOD50 › Most alternative methods do
not have LOD50 data but
› Example for 3 dilutions
RLOD!
High inoculation Intermediate Low inoculation level Blank level eLOD50
level inoculation level
= 18 cfu/test = 6 cfu/test = 2 cfu/test portion cfu/test portion
portion portion
1/1 4/4 4/4 0/1 < 2,0
1/1 4/4 3/4 0/1 = 1,0
1/1 4/4 2/4 0/1 = 1,4
1/1 4/4 1/4 0/1 = 2,0
1/1 4/4 0/4 0/1 = 3,0
AOAC Europe/NMKL Nordval symposium Oslo 15
4 June 2019
ISO 16140-3 (method verification) (5)
Estimated bias (eBias)
› Criterium: eBias < 0,5 log
Mean result for batch Absolute difference in
log10 cfu/g results between
artificially
Artificially contaminated Inoculum suspension contaminated
(food) item [without (food) item] (food) item and the
inoculum suspension
Sample 1, 2,06
test portion 1 2,17 −0,11
(average of 1,87 and
Sample 1, 2,25)
test portion 2
Sample 2, 3,99
test portion 1 4,29 −0,30
(average of 3,93 and
Sample 2, 4,04)
Test portion 2
AOAC Europe/NMKL Nordval symposium Oslo 16

Sample
4 June 2019
3,
test portion 1 3,68
ISO 16140-3 (method verification) (6)
Intralab. Reproducibility (SIR)
› Link with ISO 19036 (2019)
› Is the technical uncertainty as
in ISO 19036

AOAC Europe/NMKL Nordval symposium Oslo 17

4 June 2019
ISO 16140-3 (method verification) (7)
Intralab. Reproducibility (SIR)
› Criterium: SIR < 2 x SR
(lowest observed mean)
SR values from the validation study
(Food) item
Low level Intermediate High level Mean value

Liquid 0,32 0,50 0,48 0,43

pasteurized egg
Minced meat 0,28 0,36 0,57 0,40
(Animal) feed 0,18 0,17 0,20 0,18
Pasteurized milk 0,24 0,18 0,19 0,20
Tiramisu 0,22 0,28 0,13 0,21
AOAC Europe/NMKL Nordval symposium Oslo 18
4 June 2019
ISO 16140-3 (method verification) (8)
› Also part on verification of
validated confirmation
methods
- 5 strains inclusivity
- 5 strains exclusivity
› General: When results do not
comply Root cause
analysis!!!
› What to do with non-
validated reference
methods?
› What to do with existing
methods?
AOAC Europe/NMKL Nordval symposium Oslo 19
4 June 2019
SC9 guidance document (1)
› A guidance document is › Crucial for the acceptance of
drafted in order to guide: part 3 is the lack of
– user laboratories, requirement to repeat
verification/validation of
– (technical) assessors, existing methods!
– accreditation bodies but also › In addition a transition period
– regulatory agencies (until 1-1-2027) is included to
be able to verify non-validated
how to implement 16140-3. reference (ISO/CEN) standard
methods.
› Main points:

AOAC Europe/NMKL Nordval symposium Oslo 20

4 June 2019
SC9 guidance document (1)
› A guidance document is › Crucial for the acceptance of
drafted in order to guide user part 3 is the lack of
laboratories, (techncial) requirement to redo
assessors, accreditation bodies verification/validation of
but also regulatory agencies existing methods!
how to look upon the › In addition a transition period
implementation of 16140-3.
(until 1-1-2027) is included to
be able to verify non-validated
reference (ISO/CEN) standard
methods.

AOAC Europe/NMKL Nordval symposium Oslo 21

4 June 2019
Other parts
Part 4: single lab validation Part 5: Factorial design ILS
› Validation of a method against › Alternative design for
or without a reference method interlaboratory study using
less laboratories.
› Based on traditional way of
validation and factorial design › No to replace ILS from 16140-
validation 2.
› Part on validation of
confirmation methods Part 7: …….
› Results only valid for the
lab that did the validation!

AOAC Europe/NMKL Nordval symposium Oslo 22

4 June 2019
 Thank you for your attention

Questions????

AOAC Europe/NMKL Nordval symposium Oslo 23

4 June 2019