CLINICAL PRACTICE

Teaching Method Validation in the

Clinical Laboratory Science Curriculum
TARA C MOON, VICKY A LEGRYS

With the Clinical Laboratory Improvement Amendment’s Address for Correspondence: Tara C Moon MS CLS(NCA),
(CLIA) final rule, the ability of the Clinical Laboratory Suite 4100 Bondurant Hall, Campus Box 7145, Division of
Scientist (CLS) to perform method validation has become Clinical Laboratory Science, The University of North Carolina
increasingly important. Knowledge of the statistical methods at Chapel Hill, Chapel Hill NC 27599-7145. (919) 843-4353,
and procedures used in method validation is imperative for (919) 966-5200 (fax). [email protected].
clinical laboratory scientists. However, incorporating these
concepts in a CLS curriculum can be challenging, especially The following paper was presented at the 2006 ASCLS Annual

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

at a time of limited resources. This paper provides an outline Meeting, July 25-29, 2006, Chicago IL.
of one approach to addressing these topics in lecture courses
and integrating them in the student laboratory and the clini- Central to the role of any clinical laboratory scientist is the
cal practicum for direct application. desire to report accurate patient results. Method validation is
an imperative part of that process. There are many occasions
ABBREVIATIONS: ASCLS = American Society for Clinical when practitioners use the concepts of method validation
Laboratory Science; CLIA = Clinical Laboratory Improve- such as establishing new methods, implementing commer-
ment Amendment; CLS = Clinical Laboratory Science; cial tests, or performing periodic assessments of established
JCAHO = Joint Commission on Accreditation of Healthcare methods. The American Society for Clinical Laboratory
Organizations; NAACLS = National Accrediting Agency for Science (ASCLS) includes these skills in its definition of the
Clinical Laboratory Science. CLS profession and scope of practice.1 Additionally, method
validation is a requirement of laboratory regulations which
INDEX TERMS: clinical laboratory science; education state that performance of new methods be verified prior to
methods; methods; statistics; teaching techniques. reporting patient test results and periodic assessment of ac-
curacy and precision must occur.2
Clin Lab Sci 2008;21(1):19
Despite the attention that quality in the laboratory has
Tara C Moon MS CLS(NCA) is assistant professor, Division received as of late and the new regulations and policies that
of Clinical Laboratory Science, The University of North Carolina have resulted, it remains an area that can be improved. Many
at Chapel Hill, Chapel Hill NC. method validation procedures are still carried out inappropri-
ately or are interpreted incorrectly.3 In the past, when the Joint
Vicky A LeGrys DA CLS(NCA) is professor, Division of Commission on Accreditation of Healthcare Organizations
Clinical Laboratory Science, The University of North Carolina (JCAHO) has inspected laboratories, the most frequently
at Chapel Hill, Chapel Hill NC. cited deficiency involved quality control problems.4 Reasons
for these deficiencies in spite of improved regulations and
awareness are unknown. It is clear, however, that correcting
problems with laboratory methods is difficult and requires
The peer-reviewed Clinical Practice Section seeks to publish case stud-
higher cognitive processes of analysis coupled with a strong
ies, reports, and articles that are immediately useful, are of a practical command of the method evaluation process. Giving CLS
nature, or contain information that could lead to improvement in the students a more solid foundation in this area could prove to
quality of the clinical laboratory’s contribution to patient care, includ- diminish these inadequacies in future laboratories.
ing brief reviews of books, computer programs, audiovisual materials,
or other materials of interest to readers. Direct all inquiries to Berna- In addition to ASCLS, the National Accrediting Agency for
dette Rodak MS CLS(NCA), Clin Lab Sci Clinical Practice Editor, Clinical Laboratory Science (NAACLS) also includes method
Clinical Laboratory Science Program, Indiana University, Clarian validation in their description of the profession.5 To many,
Pathology Laboratory, 350 West 11th Street, 6002F, Indianapolis IN these skills seem abstract and are often more difficult to
46202. [email protected]

VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE 19

 C LINICAL PRACTICE

integrate in a CLS curriculum. Educators are very comfort- work culminates with the clinical practicum in which the
able teaching technical practices, but the situations and students investigate data and problems from actual laboratory
analytical processes that are required for method evaluation scenarios. Other statistical and research design methods, such
can be more difficult to simulate in an educational setting. as odds ratios, type I and type II errors, and analysis of vari-
Barriers to teaching method validation in the CLS curriculum ance, are not included in this discussion as they are covered
can include limited time, reagents, and automation. in a separate research course that is offered during the fall
semester of the second year of the CLS curriculum.
Due to the dynamic nature of the field, faculty struggle when
choosing the amount of content to include in CLS educa- R E S O U RC E S F O R M E T H O D VA L I D AT I O N
tional programs. It can be easy to dismiss an emphasis on INFORMATION
method validation in a CLS curriculum for entry-level posi- There are a variety of Internet resources available for both
tions because it is often viewed as a skill that is only needed the student and instructor on the topic of method valida-
by the experienced CLS. However, in Beck and Doig’s 2005 tion. The Centers for Medicare and Medicaid Services has an

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

survey of managers, practitioners, and educators, respondents overview of the Clinical Laboratory Improvement Amend-
indicated that they expected a CLS to be competent in the ment (CLIA) at http://www.cms.hhs.gov/CLIA and the
procedures of method validation at entry-level.6 About half specific requirements for method validation for nonwaived
of those surveyed stated that they expected the CLS to be and modified tests from Subpart K can be found at http://
able to assess and evaluate methods, adopt new methods, www.cms.hhs.gov/CLIA/downloads/apcsubk1.pdf (Table 1).
and perform method evaluation studies with no additional Other governmental websites with information about CLIA
education.6 Practitioners expect that these skills are acquired include the Centers for Disease Control (www.phppo.cdc.
in educational programs and are not learned on the job or gov/clia/default.aspx) and the Food and Drug Administra-
with additional training. tion (www.fda.gov/cdrh/CLIA/index.html). The College of
American Pathologists Laboratory Accreditation Program
Clearly, technology in the laboratory is changing every day. Inspection Checklist for Chemistry, available from http://
With some assays, it is becoming easier to operate the instru- www.cap.org/apps/docs/laboratory_accreditation/check-
ment that performs the assay on a routine basis. However, lists/chemistry_and_toxicology_april2006.pdf, contains
the cognitive skills that are required to perform the quality
control, validate the assay, and ensure that the test results are
valid are much greater. Additionally, expansions in the area of
molecular diagnostics have required more complex method Table 1. CLIA requirements for method validation2
validation for those assays that are developed in-house. It is
easy to imagine that the role of the CLS in method valida- Test type Required verifications
tion will only increase in the future. No matter what the of manufacturer’s
technology is, how it evolves, or what level of practitioner is performance
performing the testing, method validation will always be an
integral part of the clinical laboratory and an essential skill New, unmodified, Accuracy
for students to obtain in their educational programs. nonwaived tests Precision
Reportable range
The following is a review of how one educational program Reference ranges
has incorporated the concepts of method validation into
the CLS curriculum. This overview focuses on the course Test system that Accuracy
materials and methods that are used to teach the concepts has been modified Precision
of method validation and prepare CLS students for entry- or developed in-house Analytic sensitivity
level responsibilities. Instruction for these methods comes Analytic specificity to
intermittently throughout the two year curriculum using a include interfering
variety of instructional techniques. Didactic lectures are used substances
to present information in the students’ first semester, while Reportable range
laboratory exercises are used for active learning and simula- Reference intervals
tion in the second semester. The method validation course

20 VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE

 C LINICAL PRACTICE

items and commentary defining and describing the practice associated with method validation, often due to differences
and appropriate documentation for method validation. The in their backgrounds and abilities in mathematics and basic
Clinical Laboratory Standards Institute (CLSI) Evaluation statistics. To help the students master the material, tutoring
Protocol documents (EP series) contain useful procedures sessions are available from a second year student in the late
for implementing method validation studies and analyzing afternoons, twice a week. The student tutor prepares and
the results (www.clsi.org). evaluates additional problems in method validation, work-
ing closely with the instructor to monitor performance. The
Dr. James Westgard’s website (www.westgard.com) contains tutoring sessions have been very beneficial as they allow the
an extensive set of lessons on method validation that are very student additional opportunities to integrate the information
useful for students. Each lesson contains the purpose of the and master the material.
given experiment, factors to consider, examples of data cal-
culations, criteria for acceptance, and selected references. In Laboratory course

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

addition, the website contains data calculation tools that can The application of method validation procedures is imple-
be used by the student in assignments to generate standard mented in the second semester of the program’s first year in
deviation, linear plots, and reportable range. Other commer- the clinical chemistry laboratory. It is included in the chem-
cially available sources of method validation software include istry laboratory for several reasons: 1) there is little automa-
EP Evaluator (http://www.dgrhoads.com) and Analyze It tion in the student laboratory, 2) manual methods are used
(http://www.analyse-it.com/products/clinical/overview.htm). primarily in the first year laboratory courses, and 3) many of
Some of these products can be purchased for limited use for the reagent kits on which we have long depended have been
student assignments. discontinued. The inability to obtain certain reagent kits left
some vacant time in the chemistry laboratory schedule which
IMPLEMENTATION IN THE CLS CURRICULUM was not available in any other course. The reagent kits that
Didactic lectures are available are simple to use and require very little applica-
Because the lecture method is effective in disseminating a large
amount of information, we initially present the information
concerning method validation in the students’ first semester
Laboratory Mathematics and Quality Assurance course. The Table 2. Topics for method validation (MV) lectures
topics covered in the lectures are listed in Table 2 and begin
with a description of the importance of method validation. Unit Topics
They move from the simplest assays (precision studies) to the 1 Purpose of MV
more complex (interference and lower limit of detection), 2 CLIA requirements for MV
ending with a conclusion reviewing all the parts of method 3 Types of analytical errors
validation and summarizing the key points. The lectures are • Determining total allowable error
supplemented with problem sets, requiring the student to cal- 4 Precision studies
culate parameters such as imprecision, linearity, and bias, and 5 Linearity/analytic measurement range
to apply appropriate criteria to evaluate the results for accept- 6 Method comparison studies
ability. Students are evaluated on the material based on their • Bias plots
answers to the problem sets and scores on several quizzes. • Comparison plots
7 Verifying/establishing reference intervals
A limitation of the lecture format is that the student is not 8 Comparing analytical sensitivity and speci-
involved in the generation of data and the ideas and applica- ficity to clinical sensitivity and specificity
tion can seem abstract, especially if the material is presented 9 Recovery studies
to the student early in their curriculum before they have had 10 Interference studies
the opportunity to become familiar with clinical assays. The 11 Detection limits
instructor may need to provide additional clinical details 12 Drawing conclusions: Is the method
and ensure that the student has later opportunities in the acceptable?
curriculum to apply the concepts presented in the lecture • Appropriate criteria
to actual practice of method evaluation. • Factors influencing decision making
Some students struggle with the calculations and theory

VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE 21

 C LINICAL PRACTICE

tion of chemistry theory. However, by done by hand in the student labora- ware for the data analysis assignment
adding the method validation concepts tory. However, the potential for hu- and method evaluation paper. In
to the procedures, students can become man error in the manual calculations addition to their printed results and
proficient at some basic skills as well as and hand-drawn graphs is contrasted graphs, students also submit a written
incorporate a more complex cognitive with the benefits of using a computer interpretation of the statistical results.
level to the laboratory exercise. when the students use statistical soft- To make interpretations, students are

Method validation is performed by

comparing the two manual methods, Table 3. Laboratory schedule and activities
glucose oxidase and glucose hexokinase.
Glucose was chosen because it is inex- Laboratory exercises Goals
pensive, it is a very common analyte, Session Topic
and the manual methods for glucose

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

1 Perform glucose assays with Develop competency with
determination are not time consuming. controls and unknowns. procedure.
These factors allow more ground to be
covered in the student laboratory. 2 Analytic measurement Assess the useful analytical
range and precision range of a method and
The method validation studies span verify the reportable
four sessions of the clinical chemistry range from the
laboratory course. Students begin by manufacturer.
performing the procedure to develop Define acceptable limits
technical competency and then move for glucose.
on to the specific method validation Estimate imprecision.
procedures listed in Table 3. In the in-
terest of time, students are divided into 3 Method comparison Estimate inaccuracy,
two groups for the laboratory exercises. systematic error.
Half of the class performs the glucose
oxidase procedure and the other half 4 Recovery study Determine proportional
performs the glucose hexokinase pro- error.
cedure. At the end of each laboratory
session, the groups share their data Additional assignments Goals
and have time to compare the results Session Topic
of the two methods. While allowing 2 Data analysis exercise Evaluate raw data using
students to share data compromises computer software.
error reduction, it also allows for expo-
sure to more experiments and overall, 5 Method evaluation paper Using all the concepts of
a more robust laboratory experience. method validation,
With explanation, the students are able prepare a written evalu-
to understand this tactic and it does ation of two methods
not detract from their understanding and a recommendation
of the principles. In addition to the for the use of one of
laboratory exercises, two assignments the methods in the
are given: a data analysis exercise and clinical laboratory.
a method evaluation paper.
Describe the effects of the
Because learning the concepts and implementation of a
mathematical operations is imperative new method on clinical
to the students’ understanding of the practice.
procedures, calculations and graphs are

22 VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE

 C LINICAL PRACTICE

required to research CLIA guidelines and use online resources rotations and projects in method validation. Other programs
to determine acceptable limits. They are asked to make deci- may not have the time to allow students to actually perform
sions regarding their data and its acceptability in the clinical the assays, however, in these situations, students can use the
laboratory, as well as relate this information to the clinical data generated by others and apply their own data analysis
setting and explain what this means for patient results. and interpretation using commercially available evaluation
software. Often it is not until students are involved in the
The overall goal for the method evaluation paper is for the performance of “real lab” method validation do they begin
student to take all of the concepts and methods that have been to understand the principles and appreciate the importance
performed over the previous laboratory sessions, provide an of error detection as it relates to patient care.
interpretation of the entire picture, and give a recommenda-
tion for the use of one of the methods. Prior to this, the stu- SUMMARY
dents’ analyses have included one procedure at a time. For the By including method validation as an integral part of the

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

method evaluation paper, they are given additional data and CLS curriculum, we emphasize quality and give students
some summary statistics to analyze. This data complements the background they need to feel competent in applying this
the data that they collected in their laboratory exercises. They knowledge on the job. The approach described in this review
are asked to address specific questions regarding t-test statistics, has lessened issues of student difficulties with calculations
the linear regression line, and correlation coefficient. In this and their struggles to integrate the many concepts of method
final assignment, a complete analysis of all of the studies and validation. This was accomplished primarily by the addition
a summary is expected. The students are required to integrate of activities such as organized student tutoring, student
the concepts from their lecture course and the application of preparation of a written synopsis with an interpretation of
those concepts in the laboratory, then apply them to make a results, and the use of real cases in the clinical practicum.
decision regarding the clinical utility of a new method.
CLS students need to have a solid background in method
Writing the method evaluation paper is where the students validation, because performing method validation in a real
struggle the most. They are able to analyze individual pieces world setting can be tedious and time consuming, especially if
of information during each laboratory exercise, but they it is done incorrectly. We try to give our students the necessary
have trouble putting those pieces together to give a complete skills for this process when they face it in the real world, rec-
analysis. They are hesitant to recommend one method over ognizing that most College of American Pathologists (CAP)
the other as they are still trying to develop some confidence checklists ask for a plan in place for validating methods,
in their skills and become nervous about making an incorrect without indicating how to validate methods. Teaching CLS
decision. Another area they struggle with is relating these students to perform these procedures properly is important
studies to the clinical situation. They understand what error is to save time, money, and resources when they are functioning
present and they can explain it, but they have some difficulty in a clinical setting in their future roles.
applying that knowledge to patient values, understanding
how it may influence an interpretation of a value, and what Instilling in students an appreciation for quality and qual-
implications adopting a new method may have for clinicians ity standards and regulations throughout the CLS program
and their patients. These issues are often resolved during the is imperative. The studies of method validation provide a
clinical practicum when students perform additional method unique opportunity to discuss quality from a slightly differ-
evaluations and are able to consult with practitioners about ent perspective. In addition, expertise in method validation
real laboratory cases. is a skill that is very specific to CLS practice. While another
healthcare professional may be able to perform a point-of-
Clinical practicum care test, he or she will not be able to perform the method
Method validation occurs in all areas of the clinical labora- validation for that instrument. Some of the procedures that
tory and is well suited to some degree of student involve- are included in our scope of practice can overlap with other
ment. Depending on the design and scheduling of clinical healthcare professionals, but method validation does not.
rotations in the CLS curriculum, students may be able to This is something that only the CLS is educated to do and
perform the validation assays with supervision. Some cur- we should continue to strive for excellence in this arena.
ricula are designed to allow students to complete specialized

VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE 23

 C LINICAL PRACTICE

Clin Lab Sci encourages readers to respond with thoughts, ques- qualification. Final Rule. Federal Registry 2003. CLIA subpart K,
tions, or comments regarding this article. Email responses to analytical systems, section 493.1253. Available from http://www.
phppo.cdc.gov/clia/regs/subpart_k.aspx#493.1253. Accessed
[email protected]. In the subject line, please type “CLIN LAB
2007 Jun 7.
SCI 21(1) CP MOON”. Selected responses will appear in the 3. Westgard JO. Points of care in using statistics in method comparison
Dialogue and Discussion section in a future issue. Responses studies. Clin Chem 1998 Nov;44(11):2240-2.
may be edited for length and clarity. We look forward to hear- 4. Belanger AC. The joint commission and CLIA: A 5-year retrospec-
ing from you. tive. MLO: Medical Laboratory Observer 1998 Feb;30(2):46-8.
5. National Accrediting Agency for Clinical Laboratory Science. Stan-
dards of accredited educational programs for the clinical laboratory
REFERENCES scientist/medical technologist. Available from http://www.naacls.
1. American Society for Clinical Laboratory Science. 2001 Scope of org/PDFviewer.asp?mainUrl=/docs/standards_cls-mt.pdf. Accessed
practice, position paper. Available from http://www.ascls.org/posi- 2007 Jun 7.
tion/scope_of_practice.asp. Accessed 2007 Jun 7. 6. Beck SJ, Doig K. CLS competencies expected at entry-level and
2. US Department of Health and Human Services. 2003 Laboratory beyond. Clin Lab Sci 2002 Fall;15(4):220-8.

Downloaded from http://hwmaint.clsjournal.ascls.org/ on March 30 2020

requirements relating to quality systems and certain personnel

A THANK YOU TO

SUSAN J. LECLAIR
FOR HER DEDICATION AND COMMITMENT

TO CLINICAL LABORATORY SCIENCE

EDITOR-IN-CHIEF, 2000 – 2007

CLINICAL PRACTICE EDITOR, 1993 – 1999
CONSULTING EDITOR (HEMATOLOGY/HEMOSTASIS), 1990 – 1993

From the Clinical Laboratory Science Editorial Board

24 VOL 21, NO 1 WINTER 2008 CLINICAL LABORATORY SCIENCE