Validation and Equivalence of

Rapid Methods Validations

Robert Koeritzer
Sr. Technical Manager
3M Food Safety
3M 2009. All Rights Reserved.

Past-President, AOAC International

Agenda
Key Global Standards
Comparison of Global Validations
Examples & Discussion
Trends in Method Validations

Analytical Science in Trade/Commerce

Codex Technical Committees

Import Testing

Emerging Chemical
Contaminates
3

Quality Systems

Emerging Pathogens

Key Global Standards

CODEX
IDF / FIL
ISO

What is Codex Alimentarius ?

A collection of:

standards, codes of practice, guidelines and other recommendations

Some texts are general, some specific

requirements related to a food or group of foods

operation and management of production processes

operation of government regulatory systems for food safety and consumer protection

International Dairy Federation (FIL-IDF)

Represented in 49 countries around the globe

Full members have voting rights (associate members do not)
Membership fee is linked to the annual production of milk, from all species
Established one of the first truly International set of standards & methods

IDF and ISO

IDF & ISO have been cooperating in the area of methods of analysis for
milk
ilk andd milk
ilk products
d t ffor > 40 years
December 2000 - A joint agreement was signed with ISO to publish
international standard methods

International Standards Organization

The world's largest developer of standards

Make development, manufacturing and supply of products and services more

efficient, safer and cleaner

Make trade between countries easier and fairer

Provide governments with a technical base for health, safety and environmental
legislation

Aid in transferring technology to developing countries

Safeguard consumers and users of products and services

Who makes up ISO?

National standards institutes of 156 countries

1 member per country

ISO is a non-governmental organization

Members are NOT delegates from national governments

ISO occupies
i a position
iti between
b t
public
bli & private
i t sectors:
t
Some members have jobs in the government
Other members work in the private sector (associations, national partnerships)

10

Hallmarks of the ISO brand

Equal footing
No matter what the size or strength of a countrys economy, each participating
member in ISO has one vote.

Voluntary
ISO standards are voluntary
ISO has no legal authority to enforce their implementation
Some ISO stds are adopted in countries as part of their regulatory framework.
Some may become market requirements, such as ISO 9000 quality management
systems, and size of bank cards

11

Hallmarks of the ISO brand

Market-driven
ISO develops standards only when there is a market requirement

Consensus
Are based on consensus among the interested parties
Standards are reviewed at least every five years to decide whether they should be
maintained, updated or withdrawn.

Worldwide
3,000 ISO technical groups (technical committees, subcommittees, working groups
etc.) in which some 50,000 experts participate annually to develop ISO standards.

12

European standards
EN ISO 17025
Accreditation

Horizontal
i
l Methods
h d
Alternative methods
Reference methods
International
European national
Routine
ISO NF ISO
National
or EN NF EN,
NF, DIN, or EN ISONF EN ISO

13

Manufaacturer
QC//QA

diluents

Preparation of
test samples

EN ISO 6887-1

EN ISO 16140
Validation

ISO 11133 : Guidelines on QA and p

performance testing
g of culture
media
ISO 7218 : General rules for microbial examination

European Recognition Committees

(Alternative Method)
Country

France

Germany

UK

Nordic

Europe

Standards
Bodies

AFNOR

DIN

BSI

NMKL

CEN

Recognition
Systems

AFNOR

N
None,
Technical
report

EMMAS NordVal

Since 2004, Standardisation of the alternative method

performance assessment with EN ISO 16140 is possible

MicroVal

European requirements for alternative methods

Requirements:
The use of alternative analytical methods is acceptable, when the
methods are validated against the reference method set in Annex and
certified by a third party in accordance with the protocol set in EN/ISO
standard 16140 or other internationally accepted similar protocols.
If the
th food
f d business
b i
operator
t wishes
i h to
t use analytical
l ti l methods
th d other
th
than those validated and certified as described above, the methods
shall at least be well documented and scientifically validated.

Consequences:

15

AFNOR validated methods will be accepted everywhere in Europe

AOAC validated methods will be accepted if they have been validated
against an EN ISO method

Three major Global Validation Organizations

AFNOR (ISO 16140 protocol)
MicroVal (ISO 16140 protocol)
AOAC International ((AOAC protocol
p
Harmonized with ISO 16140))

16

Validation Methods Comparison

Overall
AOAC R-I, PTM

AOAC Intl OMA

ISO 16140:2003(E)

Proprietary

Proprietary & Noncommercial

Proprietary &
Non-commercial

Reference
Methods

AOACI, FDA, USDA, ISO

AOACI, FDA, USDA

ISO, CEN

Time to
Ti
t
Validation

6 12 months
6-12
th

12-18
12
18 months
th
(minimum)

12 months
th
(minimum)

Methods
Reviewed

Required yearly

N/A

Required every 4 yrs

Type of
Method

17

Part 1: Methods Comparison

[P
[Pre-collaborative]
ll b ti ]

AOAC R-I, PTM

AOAC Intl OMA

ISO 16140:2003(E)

Food Categories

Claim Dependent

6 (for all or most foods)

5 (for all or most foods)

Food Types

Claim Dependent

20 representative food types

3 per category

3 (H, M, L) + uninoculated

3 (H, M, L) + uninoculated

5 levels

if necessary

if necessary

5 per level

5 per level

2 per level (preferably 5-10)

Required
q
in 1 food type
yp

Required
q
in 1 food type
yp

NA

Required

Suggested

NA

Level of Analyte
Quantitative

# of Samples

Competitive
p
micro
flora
Ruggedness

18

Part 2: Methods Comparison

I t l b t
Inter-laboratory
study
t d (Collaborative)
AOAC R-I, PTM

AOAC Intl OMA

ISO 16140:2003(E)

Participating Labs

NA

Minimum of 8

Minimum of 8

Food Categories

NA

6 (for all or most foods)

1 Food Proposed 4/2010

Food Types

NA

1 per category

Level of Analyte

NA

3 contamination levels,
levels plus
uninoculated

3 contamination levels,
levels plus
uninoculated

if necessary

if necessary

2 per level

2 per level

# of Samples

19

NA

Association Francaise de Normalization (AFNOR)

The French standards association (AFNOR) heads the Group

Founded in 1926 and is a state-approved organization under the administrative
supervision of the Ministry for Industry
It has a membership of approximately 3000 companies (3M is one)

20

AFNOR Studies

Conduct comparison at a certified lab

Compare methods to ISO methods using ISO 16140
Pay & re-validate methods every 4 yrs
Audit of Manufacturing site every 4-6 yrs

21

AFNOR
Th principles
The
i i l off th
the certification
tifi ti procedure
d

The method validation study is carried out in 2 phases:

[technical protocol based on EN ISO 16140]

Phase 1: (preliminary study)

Characterize the method and assess its effectiveness by
comparing it to the effectiveness of the standardized reference
method.
th d
Comparison carried out at the expert laboratory.
Diversity
Di
it off samples
l andd experimental
i
t l conditions
diti
(sufficient number of
naturally contaminated samples) reflect real conditions in the field.

22

AFNOR
Th principles
The
i i l off th
the certification
tifi ti procedure
d

Phase 2: (inter-laboratory study)

Compare the efficacy of the method to be validated and the
reference method, in several laboratories (8-10 minimum).
Under defined conditions of reproducibility and repeatability.
AFNOR Difference
E
Emphasis
h i on sample
l di
diversity
it ((natural
t l contamination)
t i ti )
Confirmation of positive results
Evaluation of the quality assurance procedure at the production site

23

AOAC Whats in a Name?

1884 - Association of Official Agricultural Chemists

Later - Association of Official Analytical Chemists

Membership changed to include microbiologists, food science personnel as well as

chemists

g
of international influence and membership,
p, so the current legal
g
Became an organization
name...

AOAC INTERNATIONAL
Association of Analytical Communities

24

AOACs Greatest Assets

Independent Evaluation

international
non-government
not-for-profit
tf
fit scientific
i tifi association
i ti providing
idi iindependent
d
d t volunteer-based
l t
b d assessmentt off
methods

Transparency

validation data is p
published in a scientific p
peer-review jjournal,, the Journal of AOAC
INTERNATIONAL - available worldwide

Open discussion and decision-making of the representatives who attend and work on
committees

methods are defensible, for demonstrating regulatory compliance

L l
Legal

25

AOAC Method Validation Programs

OFFICIAL METHODS OF ANALYSIS PROGRAM

Multi-laboratory validation for nonproprietary and commercial proprietary methods where
the highest degree of confidence in performance is required to generate credible,
defensible, and reproducible results.

PEER-VERIFIED METHODS
Independent laboratory validation for nonproprietary methods where rapid validation and
some degree of confidence in performance is needed.

PERFORMANCE TESTED METHODS

Independent laboratory validation of performance claims for commercial proprietary
methods where rapid validation and some degree of confidence is needed. Program is
administered by the AOAC Research Institute, a subsidiary of AOAC INTERNATIONAL.

26

Initial Method Validation

Pre
Pre-collaborative
collaborative

AOAC RI (Performance Tested)

Performance parameters

Precision
Accuracy
Ruggedness
Linearity
Limit of detection

Comparison to other methods

Acceptable method performance (Expert Lab study)
Meets Claims in package insert

27

Method Collaborative Study

8-10 laboratories minimum

Minimum of 8 labs with acceptable data (chemistry and quantitative microbiology)
Minimum of 10 labs (qualitative microbiology)

5 samples or test materials (chemistry)

Materials must cover the range of assay applicability

Blind duplicates
Standard RM or Control

28

Method Submitted to AOAC

Method of analysis
Detailed procedure

In-house validation data

Demonstrated performance
p

Study Director appointed

Collaborative studyy pprotocol
Detailed study design

29

Protocol Review by AOAC

Safety advisor
Statistician
St ti ti i
Adequate sample analysis

General Referee
Scientific review

Appointed Method Committee

Official Methods Board

30

Collaborative Study Report

Detailed final method
Study
St d reportt
Comments from laboratories

Statistical analysis
Critical performance evaluation

Recommendations

31

MicroVal Offers:
An open platform for all European Certification Bodies.

32

Aiming at a harmonized interpretation of the EN ISO 16140.

Working towards a harmonized international market for all test kits.

Is headed by one European platform with all European stakeholders involved.

A neutral MicroVal secretariat not part of a certification organization.

33

34

Petrifilm Plate Method Validations

Example Country Approvals
Canada HPB

Aerobic Count plates

Enterobacteriaceae Count plates

Coliform Count plates

Rapid Coliform Count plates

High Sensivity Coliform Count plates

E coli and Coliform Count

E.
Co nt plates

35

Select E. coli Count plates

Staph Express Count plates

+
+
+

+
+

Yeast and Mold Count plates

Environmental Listeria Plate

+ [PTM]

Tecra Method Validations

Example Country Approvals

Salmonella Unique

NZFSA

Listeria VIA
Staphylococcus Enterotoxin VIA
E.coli O157 VIA
Staph aureus VIA

36

FDA/
BAM

Salmonella VIA
Salmonella Ultima

Canada
HPB

+ [PTM]

+
+

Why is Hygiene (ATP) Testing not Validated?

37

Topic proposed by AOAC Research Institute and discussed at CEN

Need consensus on performance criteria
Need reference material and consensus on protocol
Little end
end-user
user demand for validation
Little test kit manufacturer demand for validation

Method Validations - Possibilities

Third-Party Validation
AFNOR (ISO 16140)

[proprietary methods]

AOAC SLV, AOAC PTM and AOAC OMA [Chem & non-proprietary methods]
AOAC PTM and AOAC OMA

(harmonized with ISO 16140) [proprietary methods]

MicroVal (ISO 16140) [proprietary methods]

IDF / ISO
SO [C
[Chem
e & non-proprietary
o p op eta y methods]
et ods]

Country/Government Validation (Examples)

N
NordVal
dV l (Nordic
(N di Countries)
C
i )
Canada HPB (HPB Methods & Lab Procedures)
China ((SN and GB Methods))
38

Trends in Method Validation: Business Perspective

Cost

Reference Lab Government/

Regulatory
Diagnostic
Manufacturer

Food Processor

Time to Results
Routine
Diagnostic
Use

Accuracy
Inclusivity
Exclusivity

Validation Costs: People/Resources,

People/Resources Time
Time, Euros
39

Diagnostic Manufacturer Validation Costs and Time

Validation Costs per Method

$70,000
$60 000
$60,000
$50,000
$40,000
$30,000
$20,000
$10 000
$10,000
$-

Study/Lab Expenses
Application Fees

AOAC-PTM AOACOMA

AFNOR

Time to Complete Validation (per Method)

Acronyms:
AOAC-PTM: Performance Tested Method
AOAC-OMA: Official Method of Analysis

40
35
30
25
Weeks 20
15
10
5
0

Protocol, Analysis,
Submit Results
Complete Study
AOAC PTM Initial
Approval
AOAC-PTM AOAC-OMA

40

AFNOR

Food Processors: Wait and Re-Validate?

Post Method Approval Validation
Food Matrix and Diluent Validation
Local Flora Evaluation
Local Reference Method Validation
For EACH country!

Protection from Litigation

OR: Internal Validation (AOAC or AFNOR protocol)

41

Third-Party approvals are taking too long

I
Internal
lS
Study:
d cost can bbe US$ 200,000
200 000
External (reference lab) Study usually very limited in scope
Participation in Proficiency Testing Programs

Governments & Regulatory Agencies Validations

Regulatory Balancing Act
International approvals acceptable
for international commerce
Are validated proprietary methods
acceptable for testing local foods?
If yes,
what
h t validations
lid ti
are
acceptable?

Government Challenges
Scarce Resources
Who ppays
y for the validation study?
y
Proprietary Method Validation
Processes
Many proprietary methods are
seeking governmental validation
42

Better test
Accuracy
Consumer
Safety

Time and
Resources

Governments & Regulatory Agencies Validations (cont)

Collaborative Model: AOAC Official Method of Analysis
Food
Processor

Government
Agency

AOAC
Collaborative

Diagnostic
Manufacturer

Regulatory
egu ato y Agency
ge cy
Method Adoption

Are Third Party validations diminishing in importance?

Fewer Food Processors are volunteering for collaborative studies

AOAC Performance Tested Method

3rd Party Validation
Lower Cost; Shorter Time
AOAC PTM approvals are more compatible with rapidly changing technology
43

Validation Trends
Cost and Time Requirements Increasing
Increasing
c eas g number
u be oof pproducts
oducts requiring
equ g validation
a dat o
Increasing numbers of regulatory agencies
Products marketed to more and more countries (import
regulations)

Convergence
Harmonization of methods
AOAC and ISO 16140 closely aligned
Fit for Purpose Validations
AOAC Performance Tested Method (AOAC RI)
Screening versus Actionable
44

45