12/10/2017

Calculation of the Standard

Deviation for Proficiency
Assessment in Microbiological
Proficiency Testing Schemes
9th Eurachem workshop October-2017

ISO 17025. Section 5.9

“The laboratory shall have quality control procedures in order to assess the
validity of tests and calibrations undertaken”.

Internal Quality Control

 Quality control of reagents and culture media Inoculum
Prepared sample
 Control of working conditions METHOD 1 METHOD 2
 Control charts
 Results quality assessment

External Quality Control: Proficiency Testing Schemes (PTs)

Criteria of provider’s Selection:
• Technical competence (Accreditation)
• Use of natural matrices
• Number of participants
• Analysis and statistical treatment of results

1
 12/10/2017

σpt: Standard deviation for proficiency assessment

ISO 17043: σpt defines the acceptable level of variation between

laboratories for a particular test

It is intended to represent the maximum uncertainty associated with

results that would be considered fit for a particular purpose.

Quality of PT items: Homogeneity and Stability

Performance assessment (z score)

σpt: Standard deviation for proficiency assessment

How can σpt be calculated?

ISO 13528:

• Robust Standard deviation among labs in the Test

• Experience with previous rounds of PT for the same parameter
• Fixed value
• By perception of experts
• Using the repeatability and reproducibility of the method…

Calculation from a pool of historic data (2005-2013) Microbiological

collected in the frame of 21 different PT rounds limitations

2
 12/10/2017

Microbiological limitations
1. Low uniformity and stability of microbiological samples
Over dispersion and agglutination phenomena
Distributions follow a Poisson or Binomial negative more than a Gaussian model

2. Microbiological taxonomies are not precise

3. Differences in microbiological samples containing the same

microorganism (Variability)
”Natural” organisms vs Culture Collection organisms

4. Microbiological analysis depends on the behavior more than on

the constitution (Living cells)
The number of cells is an approximation of the number of living cells.
Changes in cellular morphology and in the metabolic activity.

5. Microbiological methods are not robust

Matrix effect (abiotic and biotic factors); Cellular stress; Culture conditions: culture
media quality, dilutions, counting, incubation Tª/time; Analysts training …

Calculation of σpt in microbiology

1. The coefficient of variation (CV) values were calculated for each
*
microbiological parameters from historical data in each matrix: S
*
×100
X

2. The CV values were plotted against the microorganism concentration

(assigned value:X*)

3. A mathematical function was obtained for each microorganism.

In each new round:

1. Applying the corresponding equation, and considering the obtained X*,

the CV value is calculated for each parameter.

CV × X *
2. Finally, σpt is calculated following: σ pt = 1.5 ×
100

3
 12/10/2017

Calculation of σpt in microbiology

Total coliforms

Calculation of σpt in microbiology

Faecal coliforms

4
 12/10/2017

Calculation of σpt in microbiology

Escherichia coli

Calculation of σpt in microbiology

Enterococci

5
 12/10/2017

Calculation of σpt in microbiology

POTABLE WATER (bottled water and swimming pool water)

Pseudomonas aeruginosa Staphylococcus aureus

y = 22.865e-0.526x 30.0 y = 62.434e-0.822x
14.0
R² = 0.3253 25.0 R² = 0.6473
12.0
10.0 20.0
CV

CV
8.0 15.0
6.0 10.0
4.0
5.0
2.0
0.0
0.0
1.00 1.50 2.00 2.50
1.00 1.50 2.00 2.50
x* x*

Total aerobic count at 37ºC Total aerobic count at 22ºC

12.0 12.0 y = 16.82e-0.714x
y = 14.491e-0.647x
10.0 R² = 0.33 10.0 R² = 0.3888

8.0 8.0
CV
CV

6.0 6.0
4.0 4.0
2.0 2.0
0.0 0.0
1.00 1.50 2.00 2.50 1.00 1.50 2.00 2.50
x* x*

Calculation of σpt in microbiology

Clostridium perfringens POTABLE WATER X* (cfu) σpt (log)

Total coliforms 78 0.19
Faecales coliforms 74 0.26
E. coli 73 0.21
Enterococci 111 0.13
C. perfringens 45 0.50
P. aeruginosa 157 0.24
S. aureus 38 0.41
Total aerobic count at 37ºC 44 0.12
Total aerobic count at 22ºC 43 0.13
WASTE WATER X* (cfu) σpt (log)
Total coliforms 1.34x104 0.48
Faecal coliforms 2.78x103 0.51
E. coli 9.20x102 0.46
C. perfringens 2.28x104 0.45
Enterococci 1.80x104 0.22

6
 12/10/2017

Values of σpt in Food microbiology

ISO 19036:2006 Microbiology of food and animal
feeding stuffs. Guidelines for the estimation of
measurement uncertainty for quantitative determinations

Reproducibility standard deviation (SR):

Range Average
Total coliforms (0.12-0.78) 0.27

E. coli (0.11-0.47) 0.25

Total aerobic count (0.09-0.45) 0.24

Other PT provider: σpt=0.35 log fixed for

the majority of microbiological tests.

AEAS: Guide for water laboratory

management; Part 1: criteria for quality
analysis performance (2016) SR≤0.2

Determination of σpt in microbiological PTs: Examples

Escherichia coli - POTABLE WATER Escherichia coli – SURFACE WATER
Previous Current Previous Current
2017-ROUND I Formula formula 2014- ROUND I Formula formula
(1999-2004) (2005-2013) (1999-2004) (2005-2013)
Nº LABS 126 126 Nº LABS 34 34
Assigned value (X*) 1.83 1.83 Assigned value (X*) 1.77 1.77
Robust SD (S*) 0.08 0.08 Robust SD (S*) 0.24 0.24
SDPA (σpt) 0.25 0.21 SDPA (σpt) 0.25 0.36
Uncertainty (μ) 0.01 0.01 Uncertainty (μ) 0.05 0.05
Outliers 3 3 Outliers 4 4
|Z|| ≤ 2 128 128 |Z| ≤ 2 35 35
2 < |Z|| < 3 0 0 2 < |Z| < 3 0 2
Z≥3 1 1 Z≥3 3 1
Total results 129 129 Total results 38 38

Escherichia coli - WASTE WATER

Previous Current
2017-ROUND II Formula formula
(1999-2004) (2005-2013)
Nº LABS 24 26
Assigned value (X*) 2.49 2.49
Robust SD (S*) 0.15 0.17
SDPA (σpt) 0.28 0.53
Uncertainty (μ) 0.04 0.04
Outliers 6 4
|Z| ≤ 2 25 27
2 < |Z| < 3 1 2
Z≥3 4 1
Total results 30 30

7
 12/10/2017

Conclusions

The use of this approach for the determination of the σpt:

 Very easy to be performed.

 Takes into account the microorganism, its concentration and the matrix used.

 Fits for the established purposes.

 Ensures that the value used for performance assessment and for
homogeneity and stability studies remains realistic and relevant to each
microorganism.

 Could be a significant contribution to the improvement of microbiological PT

assessment.

THANK YOU FOR YOUR ATTENTION

www.ielab.es
[email protected]