2nd Edition

EP26

e
User Evaluation of Acceptability of a Reagent
Lot Change

pl
m
Sa

This guideline includes recommendations for laboratories on

evaluating a new reagent lot, based on a protocol that uses
patient samples to detect clinically important changes from the
current lot.

A guideline for global application developed through the Clinical and Laboratory Standards Institute consensus process.
 EP26-Ed2
April 2022
Replaces EP26-A

User Evaluation of Acceptability of a Reagent Lot Change

Jesper V. Johansen, PhD Isolde Seiden Long, PhD, DCC, FCACB, DABCC
Lorin M. Bachmann, PhD, DABCC, MT(ASCP) W. Gregory Miller, PhD
Nikolina Babic, PhD, DABCC Nils B. Person, PhD, FAACC
Paul Michael D’Agostino, PhD Johnathan Eric Stanford, MHA, MLS(ASCP)CM
Uliana Danilenko, PhD Nico Vandepoele
A. Paul Durham, MA Anca Roxana Varlan, PhD
Marc D. Goldford, BS

e
Abstract
Clinical and Laboratory Standards Institute guideline EP26—User Evaluation of Acceptability of a Reagent Lot Change

pl
provides recommendations for laboratories on evaluating a new reagent lot, based on a protocol that uses patient
samples to detect clinically important changes from the current lot. It provides guidance on determining whether lot-
to-lot differences are significant and whether an observed difference is acceptable based on the established criteria.
The protocol attempts to balance the need to detect changes in reagent performance that may adversely affect patient
results with the fact that reagent lot verification is a relatively frequent task that places demands on the laboratory’s
limited resources. The more extensive initial setup of the protocol at the individual site is a one-time task performed in
advance, making the subsequent testing of new reagent lots a straightforward procedure.
Clinical and Laboratory Standards Institute (CLSI). User Evaluation of Acceptability of a Reagent Lot Change. 2nd ed. CLSI
m
guideline EP26 (ISBN 978-1-68440-137-6 [Print]; ISBN 978-1-68440-138-3 [Electronic]). Clinical and Laboratory Standards
Institute, USA, 2022.
Sa

The Clinical and Laboratory Standards Institute consensus process, which is the mechanism for moving a document through two
or more levels of review by the health care community, is an ongoing process. Users should expect revised editions of any given
document. Because rapid changes in technology may affect the procedures, methods, and protocols in a standard or guideline,
users should replace outdated editions with the current editions of CLSI documents. Current editions are listed in the CLSI catalog
and posted on our website at www.clsi.org.

If you or your organization is not a member and would like to become one, or to request a copy of the catalog, contact us at:

P: +1.610.688.0100 F: +1.610.688.0700 E: [email protected] W: www.clsi.org

EP26-Ed2

Copyright ©2022 Clinical and Laboratory Standards Institute. Except as stated below, any reproduction of content from a
CLSI copyrighted standard, guideline, derivative product, or other material requires express written consent from CLSI. All
rights reserved. Interested parties may send permission requests to [email protected].
CLSI hereby grants permission to each individual member or purchaser to make a single reproduction of this publication
for use in its laboratory procedures manual at a single site. To request permission to use this publication in any other
manner, e-mail [email protected].

Suggested Citation
CLSI. User Evaluation of Acceptability of a Reagent Lot Change. 2nd ed. CLSI guideline EP26. Clinical and Laboratory

e
Standards Institute; 2022.
Previous Edition:
September 2013

pl
m
Sa

EP26-Ed2
ISBN 978-1-68440-137-6 (Print)
ISBN 978-1-68440-138-3 (Electronic)
ISSN 1558-6502 (Print)
ISSN 2162-2914 (Electronic) Volume 42, Number 11

ii
 EP26-Ed2

Contents
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .i
Committee Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Standard Precautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

e
1.4 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Chapter 2: Path of Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1 Stage 1: Determining Protocol Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Stage 2: Evaluating Candidate Reagent Lots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

pl
2.3 Defining When Lot-to-Lot Verification Should Be Performed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Chapter 3: Establishing Key Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.1 Determining Critical Difference for a Lot-to-Lot Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Defining the Rejection Limit for a Lot-to-Lot Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Determining the Concentration(s) at Which to Evaluate Lot-to-Lot Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
m
Chapter 4: Selecting Samples for Reagent Lot Comparability Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1 Patient Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.2 Pooled Patient Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.3 Reference and Quality Control Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Chapter 5: Evaluating Lot-to-Lot Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.1 Stage 1: Planning Reagent Lot-to-Lot Difference Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2 Stage 2: Evaluating Lot-to-Lot Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Sa

5.3 Investigating Between-Lot Differences in Patient Sample or Quality Control Results . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 6: Limitations to the Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.1 Potential Sources of Systematic Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.2 Equivalence Tests Across Multiple Instruments and Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.3 Problems With Using Reagent Lot Equivalence Testing to Monitor Long-Term Trends . . . . . . . . . . . . . . . . . . . . . . . . 43

Chapter 7: Examples of Evaluating Between-Lot Shifts Using Patient Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

7.1 Stage 1: Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
7.2 Stage 2: Lot Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

v
EP26-Ed2

Contents (Continued)
Chapter 8: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Chapter 9: Supplemental Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Appendix A. Tables to Determine the Number of Samples Needed and the Rejection Limit . . . . . . . . . . . . . . . . . . . . . 65
Appendix B. Statistical Considerations for Determining the Number of Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Appendix C. Determining Within–Reagent Lot Imprecision and Repeatability–to–Within–Reagent Lot

e
Imprecision Ratios Using Precision Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Appendix D. Determining Within–Reagent Lot Imprecision and Repeatability–to–Within–Reagent Lot
Imprecision Ratios From Multiple Precision Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Appendix E. Estimating the False Rejection Rate on Retest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
The Quality Management System Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

pl
m
Sa

vi
 EP26-Ed2

Foreword
A change in reagent lot may lead to changes in measurement procedure performance. Possible causes of this
phenomenon include changes in reagent component materials, instability of a component in a reagent, damage during
transportation or storage, or incorrect calibration of the new reagent lot. Consequently, it is good laboratory practice to
verify the consistency of patient sample results when a new reagent lot is introduced.
Historically, testing of QC samples has often been used as a primary tool to verify new reagent lot performance.
However, although testing QC samples is key to monitoring measurement procedure performance over time, it may not
be a reliable indicator of lot-to-lot consistency for all measurement procedures. A new reagent lot may lead to a shift in

e
the results obtained with QC samples. These changes in QC results are often caused by a difference in the interaction
of the QC material with the current vs new reagent lots, commonly referred to as a matrix effect, although there is no
change in the measurement procedure performance as measured with patient sample results. It is also possible for a
reagent lot–related change in measurement procedure performance to affect patient sample results with little or no
apparent effect on QC sample results. In such instances, an insignificant change in QC results from one reagent lot to the
next could mask a significant change in patient sample results.

two reagent lots.

Overview of Changes

pl
This guideline describes a systematic approach for detecting significant changes in measurement procedure performance
for patient samples due to reagent lot changes and for confirming that patient sample results are consistent between

This guideline replaces the previous edition of the approved guideline, EP26-A, published in 2013. Several changes were
made in this edition, including:
m
• More clearly delineating the two stages of the protocol to clarify that the setup stage is performed only once, before
any new reagent lot evaluations
• Providing additional detail about the statistical techniques used, so that the included tables can be extended as
needed
• Revising discussion of allowable total analytical error (TEa) as a basis for determining critical difference (CD) to align
with current recommendations and to improve clarity regarding the relationship between the CD and TEa
• Expanding the examples of reagent lot change evaluation to provide more detail on determining the CD and other
Sa

critical parameters
This guideline describes a practical approach for screening new reagent lots for clinically significant performance changes
with patient samples. This protocol is designed to use a small number of samples. Thus, lots can be screened quickly with
limited resources. The protocol consists of two stages:
• Stage 1 sets up the protocol for each analyte. This stage involves making decisions about the medically acceptable
differences caused by reagent lot change and the acceptable risks associated with incorrect inferences. However, this
stage can be performed before any reagent lots are evaluated.
• Stage 2 is the evaluation of a new reagent lot, using the protocol developed in stage 1. This stage is simple and rapid
and is performed for every new reagent lot.

vii
EP26-Ed2

Additionally, the process described enables the laboratory to determine the effectiveness of the protocol used, including
the expected probability of detecting a significant lot-to-lot difference and the probability of falsely rejecting an
acceptable lot. The process also shows how factors such as measurement procedure imprecision and choice of CD affect
the effectiveness and practicality of the chosen protocol. No single fixed protocol is appropriate for all measurement
procedures. Therefore, this guideline provides recommendations on developing specific protocols.
NOTE: The content of this guideline is supported by the CLSI consensus process and does not necessarily reflect the views
of any single individual or organization.

key words

e
Commutability Matrix effect Quality control

Critical difference Matrix-related bias Reagent lot

pl
m
Sa

viii
EP26-Ed2

User Evaluation of Acceptability of a Reagent Lot Change

1 Introduction
1.1 Scope
This guideline describes a statistically sound protocol for evaluating the consistency of patient sample results
when a new analytical reagent lot replaces a reagent lot currently in use. It is designed for use with quantitative
measurement procedures, and more generally for measurement procedures that report on a continuous scale.

e
The same principles can be applied to measurement procedures that convert results from a continuous scale to
a qualitative report based on a cutoff value. This guideline is intended for use in the medical laboratory and is
designed to work within the practical limitations of that environment.
This guideline is not intended for use with measurement procedures that provide only qualitative or
semiquantitative results. It is also not intended for measurement procedures for which a shift in patient results is
expected with new reagent lots. For some measurement procedures, a shift in patient results with a new reagent

pl
lot is usual and expected, because the reagents are biological materials that may have lot-to-lot differences.
Such measurement procedures include prothrombin time and activated partial thromboplastin time. The usual
processes for clinical use of these measurement procedures account for this expected difference, and new lot
evaluation as described in this guideline is not necessary or useful. Guidance for these measurement procedures
provides detail on handling reagent lot changes. See CLSI documents H471 and H54.2
Additionally, this guideline is not intended to describe procedures for reagent manufacturers. The requirements
of reagent lot-to-lot testing by manufacturers, as well as the resources available, are different from those of
m
the medical laboratory. However, reagent manufacturers may use this guideline to understand the types of
verification studies that may be performed in their customers’ laboratories.

1.2 Background
The potential for a change in performance with a new reagent lot has been shown for both QC and patient
samples. This possibility is recognized by regulatory and accreditation organizations, which have incorporated
verification of the performance of a new reagent lot into their recommendations for good laboratory practice.3-11
The goal of both reagent manufacturers and medical laboratories is to provide accurate patient results.
Sa

Reagent manufacturers use several procedures to validate the performance of a new reagent lot during the
manufacturing process. Reagents are available to medical laboratories only when the performance criteria are
met. As part of the overall quality process, manufacturers may compile information on the expected lot-to-
lot consistency of patient sample results, as established internally or at other laboratories. However, because
of differences in the study designs used, the manufacturer’s protocols and acceptance criteria for lot-to-lot
variability may not be applicable for medical laboratories. Specific acceptability limits apply only to the associated
protocol for which the limits were developed. Therefore, this guideline focuses on establishing a critical
difference (CD), which is based on an acceptability limit defined by the laboratory according to the measurement
procedure’s clinical use.

2 © Clinical and Laboratory Standards Institute. All rights reserved.

 EP26-Ed2

RL for Mean Difference

0.90 • CD 0.80 • CD 0.70 • CD 0.60 • CD 0.55 • CD
CD/SWRL Sr /SWRL Power N Power N Power N Power N Power N
1.5 1.00 0.594 5 0.698 6 0.800 7 0.910 10 0.951 12
1.5 0.95 0.588 7 0.692 11 0.800 20 0.904 114 — —

Statistical power (true Number of samples

e
rejection rate, 1 − β or to be tested
1 − false acceptance rate)

Abbreviations: CD, critical difference; N, number of samples; RL, rejection limit; Sr, repeatability; SWRL, within–reagent lot imprecision.
Symbol: β, probability of making a false lot acceptance for a single concentration level.

Figure 5. Interpretation of Tables A1 to A3 in Appendix A Entries

pl
To use Tables A1 to A3 in Appendix A, the laboratory needs to know the ratio of the CD to the measurement
procedure’s SWRL (CD/SWRL in the first column) and the ratio of the measurement procedure’s Sr to its SWRL
(Sr/SWRL in the second column). The Sr, SWRL, and CD must be applicable to the set of samples tested at a given
target concentration interval. If two (or more) sets of samples will be tested at two (or more) target concentration
intervals, the appropriate Sr, SWRL, and CD for each target concentration interval need to be available. The
laboratory director should:
1. Locate the measurement procedure’s CD/SWRL in the first column.
m
2. Locate the measurement procedure’s Sr/SWRL from the rows in the second column that correspond to the
ratio in step 1.
3. Move across the row from the cell located in step 2 until the number in the “Power” column is greater
than or equal to the desired “Power” (typically 0.80 or 0.90). The number in the adjacent “N” column is
the number of samples that needs to be tested with each reagent lot at a specified target concentration
interval to detect a difference greater than or equal to the CD.

5.1.1.2 Example of Using Table A1 in Appendix A

Sa

For the measurand in question, the laboratory wants to achieve a statistical power of at least 90% (meaning
that the probability of not detecting a clinically unacceptable difference between lots is no more than 10%).
The CD/SWRL is 3.0. The Sr/SWRL is 1.00. The laboratory plans to evaluate the reagent lot at a single measurand
concentration. Figure 6 illustrates this example. The laboratory director should start at the left-hand column
(CD/SWRL) and go down to the row that contains a CD/SWRL ratio of 3.0 in the first column and an Sr/SWRL ratio of
1.00 in the second column. The columns labeled “Power” indicate the statistical power achievable using the
number of samples in the adjacent “N” column (to the right of the “Power” column). In this example, the “Power”
column under the “0.60 • CD” heading indicates that the Power is 0.929. The corresponding cell indicates that the
number of patient samples that needs to be tested to achieve this statistical power is three. Finally, the laboratory
director should go up the column containing this cell to find that it needs to use an RL 0.60 times the CD to
achieve the desired statistical power.

© Clinical and Laboratory Standards Institute. All rights reserved. 35

EP26-Ed2

7
1 Examples of Evaluating Between-Lot Shifts Using Patient Data
This chapter includes examples of the protocol applied to several representative analytes. The example analytes
are glucose, aspartate transaminase (AST), sodium (Na), thyroid-stimulating hormone (TSH), and high-density
lipoprotein (HDL) cholesterol. Subchapters 7.1.1 and 7.1.2 summarize the process used in stage 1. Subchapter 7.1.3
discusses the details for each individual analyte. Calculations for glucose and HDL cholesterol are presented in
both mg/dL and mmol/L. The values are rounded to the commonly reported number of significant digits. Because
of different rounding, the results obtained using one set of units may not exactly match those that use the other
set of units at every step. The examples are designed to be consistent within one set of units and to result in the
same protocol for that analyte. Hence, intermediate results may not convert exactly between mg/dL and mmol/L.

e
Although intermediate results are rounded when displayed in the examples, unrounded values are used for
subsequent calculations.

7.1 Stage 1: Setup

7.1.1 Determining Key Parameters

pl
The first step is to determine the key parameters: target concentration(s) (based on medical decision
concentrations), CD at each concentration, SWRL, and Sr. The SWRL and Sr values are obtained from the
manufacturer’s documentation of measurement procedure performance or in-house laboratory performance
studies. Table 4 summarizes the relevant values. Using one or more of the approaches outlined in Subchapter 3.1,
the laboratory director must determine CD values at each of the relevant target concentrations for each analyte.
The examples provided in Subchapters 7.1.3.1 to 7.1.3.5 illustrate the process of determining appropriate and
practical CD values.
m
Table 4. Example Data for Glucose, AST, Na, TSH, and HDL Cholesterol
Analyte Target Concentration SWRL Sr Sr /SWRL
Glucose, mmol/L 2.8 0.05 0.03 0.60
8.3 0.11 0.08 0.73
16.6 0.25 0.19 0.76
Glucose, mg/dL 50 1.0 0.6 0.60
150 2.1 1.5 0.71
300 4.5 3.5 0.78
Sa

AST, IU/L 40 1.3 0.8 0.62

200 4.1 1.3 0.32
Na, mmol/L 140 1.0 0.7 0.70
TSH, mIU/L 0.35 0.018 0.009 0.50
5.4 0.20 0.16 0.80
HDL cholesterol, mmol/L 1.24 0.024 0.014 0.58
HDL cholesterol, mg/dL 48 0.9 0.5 0.56
Abbreviations: AST, aspartate transaminase; HDL, high-density lipoprotein; Na, sodium; Sr, repeatability; SWRL, within–reagent lot imprecision;
TSH, thyroid-stimulating hormone.

46 © Clinical and Laboratory Standards Institute. All rights reserved.

 e
pl
m
Sa

PRINT ISBN 978-1-68440-137-6

ELECTRONIC ISBN 978-1-68440-138-3

EP26-Ed2