Clinical issues

Risk assessment and quality control

Be ready for the new guidance

By Max Williams, MPA

R isk Based QC. Patient centered QC. Risk Assessment.

Patient-focused QC. Individualized Quality Control.
These terms have been popping up in articles and confer-
ences and on the Internet over the past year. Now, as we approach
the time when the Centers for Medicare and Medicaid Services
allowed to follow their organization’s requirements, but as with the
EQC, we can expect that accrediting organizations will also adopt
the concepts of IQCP as an alternative to default QC.

Choosing a starting point

(CMS) is expected to unveil the initial guidance for Individual- The first, most obvious question is which test should be reviewed
ized Quality Control Plans (IQCP), it’s time to brush up on risk by the risk assessment. You might want to start with a point-of-care
management and understand what risk management aspects of the test in which the current QC plan is based on EQC. Or you might
IQCP may mean for laboratories. It is likely IQCP will be a key want to start with a test that has a wealth of internal documentation
focus for point-of-care testing (POCT), and therefore it will be the and lab experience in order to better build your risk assessment
rare hospital laboratory that will not have to make a decision on skills. Other viable options are to select a test that you know is
how to implement it. a poor performer or problematic, or a test for which you believe
Most labs that use point-of-care (POC) devices have established the QC plan is not consistent with its performance—either too
their frequency based on the equivalent quality control (EQC) rules much QC for a strong performer or too little for a weak performer.
published by CMS in 2003. With implementation of IQCP, it is Regardless of where you start, start simply. The process has the
anticipated that labs will be required to revert their POC devices potential to be overwhelming the first time around.
back to the default rule of two levels of QC per day or conduct a
risk assessment to determine whether the appropriate QC for their Standards and guidelines
current number and frequency is adequate or needs to be revised. There are three risk management documents that have current
relevance and will prove helpful to laboratories:
The change 1) ISO 14971, Application of risk management to medical devices,
According to CMS, Clinical Laboratory Improvement Amendment which was a reference document for the authors of the second
(CLIA) guidelines will provide an educational period during which document;
EQC will be phased out, a period that is expected to be at least one 2) CLSI’s EP23, Laboratory Quality Control Based on Risk Man-
survey cycle long (two years).1 Labs accredited by other organiza- agement; and, of course,
tions such as The Joint Commission (TJC), CAP, or COLA, will be 3) the forthcoming CMS guidance on IQCP.

Severity of Harm

Probability of Harm Negligible Minor Serious Critical Catastrophic

Frequent unacceptable unacceptable unacceptable unacceptable unacceptable

Probable acceptable unacceptable unacceptable unacceptable unacceptable

Occasional acceptable acceptable acceptable unacceptable unacceptable

Remote acceptable acceptable acceptable acceptable unacceptable

Improbable acceptable acceptable acceptable acceptable acceptable

Table 1. EP23-A risk acceptability matrix

QA/QC

ISO 14971 is intended for use by manufacturers developing Step 2. Process mapping. Don’t try to do this alone or with just one
medical devices and while “not intended for clinical decision mak- or two others. Most business theorists suggest the most productive
ing,” there is valuable information in section H, “Guidance on risk working groups/committees have between four and 10 members.
management for in vitro diagnostic medical devices.”2 The guid- Gather the staff who run the test method, use the test results, and
ance document provides valuable example lists of possible errors collects the patient specimens, and start mapping out the process.
by users, hazards, and risk mitigation controls. As a manufacturer’s If you can, include in the group the phlebotomists and nursing staff
guidance document, the standard is helpful, but it is not adequate who collect the sample. If that’s not possible, work with them at
as a single source for developing a risk management plan for a a later time but avoid the temptation to assume you know their
clinical laboratory. The document is sold on ISO (International steps of the process; after all, you’re in the discovery mode, and
Organization for Standardization) standards websites. it’s important to understand what really occurs, not just what is
Developed for clinical laboratories, EP23 is the first document supposed to occur. The same holds true for the clinicians who order
to provide step-wise instructions. The document covers all the steps the test and act upon the result. Include them in the process and
of preparing a risk management plan and provides sample checklists understand what they do when the test result comes back different
and specific examples for a hypothetical glucose instrument.3 The from what they expected. Process mapping is where you will likely
document was released in late 2011 and has been well-received. discover “tribal knowledge” for performing the test that may vary
The criticism it has received has been primarily focused on the need from the written procedure.
for more precise instructions. James Westgard, PhD, has cited a Step 3. Identification of hazards. Once the procedure is mapped
few gaps in the current version, such as the lack of hazard detec- out, you begin to ask “what can go wrong” during routine opera-
tion as a part of the calculation of overall risk and the recognition tions for each step of the process: what hazards are present. A
of the role statistical quality control and instrument performance hazard is a failure that could cause potential harm to the patient.
plays as a starting point in any risk management plan. At present, Once again, this is best performed in a group setting. One idea
this is the only document available to labs that clearly lays out will remind someone else of another hazard, and the collaboration
the path, and we can expect that the next version will address any will uncover areas for further exploration and research. Once the
shortcomings. This document is the best tool labs have to date and hazards are identified, they can then be categorized into one of five
is sold on the CLSI website. areas: Operator, Environment, System (Instrument), Reagents, or
CMS officials have stated that they will base their upcom- Specimen. These categories will make it easier to later identify
ing IQCP guidance on principles of EP23. CLIA guidelines are types of controls necessary to reduce unwanted risk.
necessarily written from a government regulatory perspective, Step 4. Evaluate the risk for each hazard. This is the step where
intended to establish a baseline for minimally acceptable quality you decide how likely it is that a failure will occur and, once it
and intended to be applicable to all labs, from the most to the least occurs, how likely it is that it will lead to harming a patient and
sophisticated. Therefore, it is anticipated that the IQCP guidance how severe that patient harm could be. This step is more subjective
may not be a highly technical guidance document, and labs may than the others and is likely to create a great deal of discussion
need to utilize EP23 or avail themselves of additional training to among the team. It will be important to work with clinicians to
make an effective risk management plan. The IQCP guidelines understand clinical implications for an incorrect result. Given the
will be released as part of the surveyor’s Interpretive Guidelines. subjectiveness of these determinations, it is recommended that the
An additional valuable resource is Westgard QC’s Six Sigma team learn as much as possible about performing this step before
Risk Analysis.4 This book contains a thorough review of the prin- finalizing its assessment of the risk levels.
ciples of risk assessment and management and the pros and cons Step 5. Determine acceptability of risks. Once the risks are as-
of this approach. It contains examples for each step of the process signed, the next step is to look at severity and probability of harm
and compares how the various documents address the same point. to determine whether the risks are acceptable. This is an area where
guidance from standards and guidelines differs, and ultimately it
The process will be up to each lab to determine where to draw the line between
There’s no way around it: a risk assessment is going to take work. acceptable and unacceptable. CLSI’s Risk Acceptability Matrix
The good news is that unless you are bringing in a new test, you (Table1, page 18) is but one example and one way for creating the
already have a great deal of the necessary information at hand, chart. Ultimately, it will be up to the team to determine whether
and you have the valuable experience of your staff that has run a given risk is acceptable or not. Sometimes the judgment can be
the test and investigated failures. The number of phases or steps a difficult one—for example, if a given failure is occasional and
of the process may vary depending on the source material, but for the severity of the harm is serious.
the purposes of an introduction to the process, let’s break it down
into 10 steps that cover three phases. Risk control phase
Step 6. Risk reduction/control/mitgation. For any risk that is
Risk assessment phase deemed unacceptable, the lab should identify ways to reduce the
Step 1. Information review. It’s important to take the time to probability of harm, using prevention and detection methods, in
pull together all the information you have on the test. This could order to bring the risk down to an acceptable level. This may be
include instrument manuals, package inserts, technical bulletins, through a variety of means such as revised operator training, posted
regulatory/accrediting organization requirements, QC and PT re- warnings, more robust QC rules, greater surveillance of the process,
cords, training procedures, prior failure investigation records, and or even repeat testing for values exceeding a specified threshold. In
any institutional clinical guidance on the test or analyte. Without cases where the test manufacturer indicates that certain aspects of
all the information handy, the team may be tempted to rely on the test are monitored through built-in controls, the lab may wish
memory, and having it in one location will save time later when to conduct a study to verify the effectiveness of the control before
you are researching various details. including it as a control measure.
QA/QC

Step 7. Evaluate residual risk. Once controls are assigned, the the lab need to perform a failure investigation, evaluate a new test
team then reviews the risks again and re-determines the risk level. method, or look to improve the process of the current method.
Any risk not prevented or detected 100% of the time is considered ————————
residual risk. In contemplating the risk management process, a few of the
Step 8. Further risk control procedures. If any risks still remain steps may seem daunting to document, and certainly time consum-
unacceptable, the team must decide how to bring that test into an ing, but the process, if performed diligently, will provide insights
acceptable level for continued operation. In cases where it is not that in the end will improve the quality of the test and improve
possible to mitigate the risk to an acceptable level, the team should patient safety.
work with clinicians to determine if the benefits of performing the
test outweigh the residual unacceptable risk.
Max Williams, MPA, is Division Global Marketing Manager,
Risk monitoring phase Quality Systems Division, for Bio-Rad Laboratories.
Step 9. Determine a monitoring plan. Since the goal of the risk
management plan is to maintain or improve the quality of the test,
it’s important to define a means to measure the quality to ensure
that it meets the needs of the laboratory and the intended use of
the test. One could argue that monitoring clinician complaints is
an adequate means of monitoring quality, but this may only be true
in cases where there is a close relationship between the lab and the References
clinical staff. In large institutions, it may be that the lab is made
aware of only the most significant errors and some incorrect results 1. CMS letter to State Survey Agency Directors, dated March 9, 2012. Implementing the Individu-
are never reported back to the lab. It is better to have measures alized Quality Control Plan (IQCP) for Clinical Laboratory Improvement Amendments (CLIA).
in place that are more likely to be logged or otherwise recorded. http://www1b.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertification
Step 10. Write the quality control plan. A good plan does not GenInfo/Downloads/SCLetter12_20-.pdf . Accessed September 24, 2012.
need to be voluminous to be effective, but it should be complete 2. International Organization for Standardization. ISO 14971:2007 Medical devices—application
enough that staff (and auditors) understand the factors that were of risk management to medical devices. www.iso.org.
considered in its creation. The lab may wish to include the process 3. Clinical Laboratory Standards Institute. Laboratory Quality Control Based on Risk Manage-
map and risk tables, but at a minimum they should be referenced ment; Approved Guideline. EP23-A. 2011. www.clsi.org/source/orders/free/ep23-a.pdf.
and maintained separately. They will be invaluable later should 4. Westgard JO. Six Sigma Risk Analysis. 2011. www.westgard.com.

Reprinted with permission from Medical Laboratory Observer, November 2012 • www.mlo-online.com