Using Statistical Process Control Chart Techniques to Ensure Quality of Care in

Pharmacy Department of a Hospital

T. W. Chien1, Agnes L.F. Chan 2*, Henry W.C. Leung 2

1
Department of Administration, Chi Mei medical Center, Tainan, Taiwan
2
Department of Pharmacy, Chi Mei Medical Center, Tainan, Taiwan
([email protected])

1991, when a series on the special features of CQI were

Abstract - To ensure high quality pharmaceutical care in published [9, 10].
a 1200-bed teaching hospital in southern Taiwan, patient- In 1998, our hospital identified key performance
centered performance indicators, developed based on indicators and used a CQI system to measure the quality
medication-use indicators, and pharmacy operational of pharmaceutical services and to track performance
indicators were routinely monitored using sequential
indicators over time. We then established the CQI-PPAS
detection analysis of control charts. A sequential control
system, which was a pharmacist performance appraisal
chart approach was proposed to automatically inspect the
outliers at the last time point and email each responsible system (PPAS) linked to the CQI system, to identify
pharmacist their outlier control chart for discussion. Forty inefficiency and to monitor the process and outcome. The
indicators were used to measure the quality of CQI-PPAS system is not only intended to provide
pharmaceutical care. Using sequential detection, the outliers concrete performance measures through indicator
of the indicators beyond threshold and trend were examined monitoring, it is also intended to provide monetary
using deviation and threshold criteria. A summary report incentives to pharmacists when they perform well,
showed that there was no difference between both types of thereby facilitating improved care [11, 12].
indicators, and that they both improved each year. The purpose of monitoring pharmaceutical care is to
ensure that all the patients’ drug therapy is indicated,
Keywords - control chart, appraisal system, continuous
effective, safe, and convenient [13]. To achieve these
quality improvement
outcomes, performance indicators related to structure,
process, and quality of pharmaceutical care were
I. INTRODUCTION developed.

As healthcare costs continue to rise worldwide,

healthcare providers, and healthcare payers, e.g., II. METHODOLOGY
insurance companies, in particular, feel pressured to cut
costs. Healthcare reform is a major political problem in 1) The CQI committee members in the pharmacy
many countries. In the United States, for example, department developed a draft of departmental CQI criteria
President Obama recently achieved a healthcare reform and reviewed potential indicators based on a literature
“victory” after a prolonged and tortuous debate [1-3]. The review through consecutive rounds of ratings and
quality of care is another important issue that has been the discussion. The committee members then prioritized the
subject of innumerable public discussions and concern CQI indicators based on whether they adequately
worldwide over the past decade. Healthcare payers, measured all aspects of pharmacy operations to assure
especially single-payer national healthcare systems, are high-quality pharmaceutical care according to the
concerned about how the providers measure and monitor department goals of improving patient safety and reducing
quality of care and clinical outcomes. costs. The list of selected CQI indicators was presented at
Quality improvement must be based on evidence-based a CQI committee meeting for review and approval.
research and an ongoing assessment of data and
information [4, 5]. Using information technology in health 2) The PPAS was linked with CQI indicators, and an
care is essential if there is to be substantial improvement ongoing monitoring system was developed by the
in patient safety [6]. Continuously monitoring and pharmacy department in 1998 and implemented on
periodically evaluating the performance of individual September 1, 1999. The overall system was called the
providers, e.g., physicians and pharmacists, is important CQI-PPAS system. Developing this performance
in hospital settings [7]. The concept of continuous quality appraisal system involved two steps. First, the CQI
improvement (CQI) has been familiar to hospital committee (the pharmacy director, the indicator
pharmacists since the 1980s, when drug-use evaluation coordinator, and five team supervisors) in the pharmacy
standards were set by the Joint Commission on department was organized. The CQI committee was and
Accreditation of Healthcare Organizations (now simply remains responsible for approving the supervisor-
The Joint Commission, a private sector United States- indicated performance of all pharmacists each month.
based not-for-profit organization) [8]. It has been Then, the pharmacy director met with the director of
implemented extensively in pharmacy departments since administration to stipulate the PPAS criteria and to

978-1-4577-0628-8/11/$26.00 ©2011 IEEE 273

 Proceedings of the 2011 IEEE ICQR

determine the budget for providing monetary incentives been transformed using a logarithm function (e.g., 0.5 ×
within the pharmacy department. Procedures involved in ln (p/(1  p)) to normalized distribution [17].
developing a CQI indicators evaluation system are shown 4) We programmed, using VBA (Visual Basic for
as following: Applications), a routine Excel spreadsheet to
a) Identify key performance indicators (medication automatically identify data property for appropriate
use and operational performance) control charts to use, to detect the outcome of each
b) Determine the acceptable threshold of each indicator’s most recent point, to compare it with the
indicator previous data, and then to plot the control chart if the
c) Determine standard procedures for data collection datum is out of control in either way: (1) beyond the
& tracking expected threshold (t > 1.0) according formula (1), where
d) Data collection by frontline pharmacists OE denotes the detected value of the last time point, n =
e) Data analysis by a pharmacy supervisor 12 if observed values are 12, and sd is the standard
f) Data submitted to CQI committee deviation yielded by the previous 12 months, or (2)
g) Develop improvement strategies beyond the limited deviation of the control chart by one or
h) Reassess and modify the performance indicators more standard deviations. The results of the plotted
and thresholds continually to reflect the goals and control chart are emailed to the CQI facilitator responsible
objectives of the department and hospital for inspecting the indicators.
OE − threshold
3) The selected CQI indicators were divided into two t= ,(1)
categories: medication use indicators and operational sd
process indicators. The thresholds were determined based n -1
on previous tracking data and were then periodically
reassessed. In 1998, we started selecting indicators for III. RESULTS
operational processes (Table 1), used to monitor the
quality of the operational systems used in the pharmacy A. An annual sequential control chart (SCC) report
department, and indicators for pharmaceutical care (i.e.,
medication use indicators) (Table 2), used to identify An SCC report from 2007 showed no significant
issues with system performance. difference (χ2 = 0.046; p = .83) in the count distribution
4) A control chart is used to detect the most recent associated with those two types of indicators at the last
(i.e., the last time point) results of each indicator and time point of 2007 (Table 1). Most of the indicators
compare them with the previous data [14-16]. Three types (91.66%) are within a range of one standard deviation
of data property (viz., count, defect, and ratio) for (SD). Three indicators show identical variance across all
indicators were used in our periodical evaluation. time points, indicating no variance existed across
Different variations were designed to set the control limits consecutive months.
examined by the c-chart for count events, np-chart for
ratio, XmR chart without significant trend, and XmR
trend with significant trend for proportional data that have

TABLE 1 Testing the distribution of counts between two types of indicators using sequential control charts
>-1 SD to
Types of indicator Threshold
others
Scoring judgment: >-1 SD Expected <+1 SD <+2 SD <+3 SD >=+3 SD Total χ2 prob.
Operational process 7 1 2 0 0 0 10 0.046 0.83
% 70 10 20 0 0 0 100
Expected 6.25 1.25 1.25 0.42 0.42 0.42 10
Medication-used indicator 8 2 1 1 1 1 14
% 57.14 14.29 7.14 7.14 7.14 7.14 100
Expected 8.75 1.75 1.75 0.58 0.58 0.58 14
Total 15 3 3 1 1 1 24
% 62.5 12.5 12.5 4.17 4.17 4.17 100

274
 Proceedings of the 2011 IEEE ICQR

Fig. 1 Outlier at latest time point plotted by statistical process control chart
Note. 1.08 = 0.5 × 0.6831 ÷ (1- 0.6831) transformed by Excel programmed routine

B. The control chart for plotting outliers at the last either by reference to the patient’s past or because the
detected point current indicators have values outside the normal range,
and the future can be predicted only in light of the past.
Based on Figure 1 (XmR-chart), the outlier (indicator
24), which is 3 SD beyond the criterion, is plotted and C. Implications of the results and suggested actions
automatically emailed to the CQI facilitator, who makes
an SCC report and discusses it with the pharmacist We created a monitoring system that focuses on a
supervisor who will focus on the outlier and make a relatively small number of key areas to be easily, quickly,
reason-cause analysis (RCA). The results of the SCC and clearly graphically compared, a system that will help
report and RCA will then be submitted to the CQI
hospital pharmacies make large improvements in the daily
committee for discussion. The values on the Y-axis have
operations and in the services they and their hospitals
been transformed using a logarithm function because they
provide to patients.
are proportional data (e.g., 1.52 = 0.5 × 0.73 ÷ (1  0.73)).
D. Strength of this study
IV. DISCUSSION
The CQI-PPAS is the first system for evaluating
A. Key findings pharmacy department performance in Taiwan. The
combination of CQI indicators and PPAS indicators
allows managers to measure the contribution that
The sequential detection system using control charts to
pharmacists make to reduce the risk-to-benefit ratio for
monitor hospital operational indicators and quality-of-care
patients [18]. In addition, PPAS indicators enable the
indicators helps our hospital’s pharmacy department by pharmacy department to track operational trends and
making our pharmacists almost instantly aware of identify strengths and weaknesses with the goal of
potential problems with pharmacy operations and certain defining and quantifying the performance of key
aspects of patient care. departmental functions. The CQI indicators integrate the
clinical and technical operational activity and provide
B. What this study contributes to current knowledge qualitative information about the range of pharmacy
activities, including medicine supply and pharmaceutical
Both the emailed outlier control chart and the care, contributed to individual patient care. A thoughtful
summarized report can be integrated to quality-of-care in performance appraisal can make employees more aware
clinical settings, especially using a graphical of what they need to do to receive a good performance
representation with time points that provide a context for review; it can also motivate them to achieve
understanding the data. The present can be understood organizational goals [19].

275
 Proceedings of the 2011 IEEE ICQR

employees within the department. We therefore

The CQI-PPAS is outcome-oriented, so rewards are established our system from the top down (i.e., from the
allocated according to how well individual pharmacists hospital management down to the pharmacists in the
meet the CQI indicator goals; thus, pharmacists in pharmacy department). We selected appropriate CQI
different groups have their own CQI goals and indicators using evidence-based criteria, and we
performance expectations that differ substantially. This established performance appraisal criteria based on
fosters pharmacist interdependence, teamwork, and historical performance data in our pharmacy and on the
cooperation. Ultimately, the CQI-PPAS inspires group consensus for desired future performance.
pharmacists to be self-motivated in order to provide the Maintaining this complex performance tracking system,
best possible performance. including entering data into each tracking database and
transferring it into an easy-to-read cumulative report, did
Our CQI indicator data showed continuous not pose any major problems, however.
improvement, and pharmacists now treat patients more
equitably and in a friendlier manner. These results are in The pharmacy director and pharmacy supervisors in the
line with the hypothesis of Bowen et al. [18] and Angaran different pharmacy groups were primarily responsible for
[20] that pharmacists with an equitable performance making this process more efficient and manageable. The
appraisal and feedback system treat customers more fairly. CQI-PPAS development process was quite similar to that
reported by Roberts et al. [11], who proposed to define,
E. Further studies and suggestions quantify, and track trends in performance levels and
identified the strong and weak areas of the key functions
An integral part of the CQI concept is to have monthly of the pharmacy department. Furthermore, using CQI
reviews of the indicator values in combination with indicators that measure the standard of clinical care
annual reviews and revisions. Monthly CQI indicator data provided to the public is in line with hospital accreditation
are posted on the pharmacy’s information board and criteria in Taiwan.
website for personnel to review. For example, the
“frequency of phlebotomy timed inappropriately vs. time There are limited data for pharmacy performance
of drug administration” was not always within the desired evaluation systems, and no data suitable for a direct
limit; therefore, we developed a strategy to improve the comparison with the CQI-PPAS. Managing a hospital
outcome rather than changing the target value for the next pharmacy practice and ensuring satisfactory patient care is
year. The pharmacist responsible for improvement inherently difficult: the cognitive processes and actions
received a positive credit performance appraisal. involved are not always easy to measure or quantify.

The advantage of establishing a CQI process is to V. CONCLUSION

minimize both medication errors [26, 27] (e.g., dispensing
errors, medication errors due to the healthcare delivery This report documents our development of
system) and adverse drug events (e.g., an inappropriate performance appraisal criteria linked to CQI indicators in
vancomycin infusion). One report [21] suggested that order to monitor the performance of a hospital pharmacy
between 2% and 3% of all hospital admissions in as a whole, and to establish an integrated system that (1)
Australia may be medication-related. The annual cost of inspires self-motivation within a pharmacy department
preventable medication errors was $350 million in and (2) graphically inspects a control chart that signals an
Australia and $17 to $29 billion in the USA [21]. Severe alert or an alarm if current indicator values deviate
adverse drug events (ADEs) may also increase the direct significantly from those at the last time point. Future
medical costs incurred by hospitals when treating ADE- research is recommended to evaluate the CQI-PPAS
injured patients. By reducing such errors, the costs
system using objective and statistical theory based on
associated with these types of errors will also decrease [22,
algorithms and techniques such as the sequential detection
23]. Thus, the continuous monitoring of CQI indicators
will indirectly reduce costs in the healthcare system. system used in this study to increase the quality of
pharmacy department service and to allow healthcare
F. Limitations of the study providers to contain costs.

In establishing a performance appraisal system, we

may have encountered problems previously faced by other ACKNOWLEDGMENT
researchers. These include the initial selection of
indicators, setting acceptable performance standards, and This study was supported by grant CMFHR9820
designating an appropriate person within the department from the Chi Mei Medical Centre, Taiwan.
to be responsible for tracking and reporting the data as
well as improving performance [13]. Because the CQI-
PPAS is complex and dynamic, it requires a significant REFERENCES
resource commitment from management and all

276
 Proceedings of the 2011 IEEE ICQR

[1] ER Brown. “Health USA. A national health program [18] DE Bowen, SW Gilliland, R Folger,”HRM and
for the United States”, JAMA, 267(4), pp.552-8.1992. service fairness: how being fair with employees
[2] S Woolhandler, DU Himmelstein, M Angell, QD spills over to customers", Organizational Dynamics,
Young, “Proposal of the physicians' working group 27, pp.7-2,1999.
for single-payer national health insurance”, JAMA, [19] GL Moss, “Improving staff performance using the
290(6), pp.798-805,2003. appraisal process”, Optometry, 72(2),pp.126-7,2001.
[3] L Manchikanti, JA Hirsch,”Obama health care for all [20] DM Angaran,”Quality assurance to quality
Americans: practical implications”, Pain Physician, improvement: measuring and monitoring
12(2), pp.289-304, 2009. pharmaceutical care”, Am J Hospital
[4] J Moreland, J Richardson, DH Chan, J O’Neill, A Pharmacy,48(9),pp.1901-7,1991.
Bellissimo, RM Grum, L Shanks, “Evidence-based [21] U.S. Department of Health & Human Services,”
guidelines for the secondary prevention of falls in Reducing and Preventing Adverse Drug Events To
older adults”, Gerontology, 49(2), pp.93-116,2003. Decrease Hospital Costs”, AHRQ Publication
[5] RS Hughes, CM Clancy, “Nurses' role in patient Number 01-0020; 2001.
safety”, Journal of Nursing Care Quality, 24(1), pp.1- http://www.ahrq.gov/qual/aderia/aderia.htm
4, 2009. (Accessed 5 March 2011).
[6] DW Bates, M Cohen, LL Leape, JM Overhage, MM [22] AL Chan, HY Lee, CH Ho, TM Cham, SJ Lin, “ Cost
Shabot, T Sheridan, “ Reducing the frequency of Evaluation of Adverse Drug Reactions in
errors in medicine using information technology”, J Hospitalized patients in Taiwan: A Prospective,
Am Med Inform Assoc., 8(4), pp.299-308, 2001. Descriptive, Observational Study. Curr Ther Res
[7] TW Chien, WC Wang, SB Lin, CY Lin, HR Guo, SB Clin Exp., 69(2), pp.118-129,2008.
Su, “KIDMAP, a web-based system for gathering [23] R Segal, “A framework for evaluating the work of
patients' feedback on their doctors”, BMC Med Res pharmacists”,Top Hosp Pharm Manage, 12(1), pp.9-
Methodol, 9:38, 2009. 19,1992.
[8] DM Nadzam, “Development of medication-use
indicators by the Joint Commission on Accreditation
of Healthcare Organizations”, Am J Hosp Pharm.,
48(9), pp. 1925-1930, 1991.
[9] DM Angaran, “ Selecting, developing, and evaluating
indicators”, Am J Hosp Pharm., 48(9), pp.1931-7,
1991.
[10]MJ Melby, “A mid-sized hospital's experience in
indicator data collection”, Am J Hosp Pharm., 48(9),
pp.1937-40, 1991.
[11]MB Roberts, MR Keith, “Implementing a
performance evaluation system in a correctional
managed care pharmacy”, Am J Health Syst Pharm.,
59(11), pp.1097-104, 2002.
[12]GL Moss GL, “Improving staff performance using the
appraisal process”, Optometry, 72(2), pp.126-7,
2001.
[13]MB Roberts, MR Keith, “Implementing a
performance evaluation system in a correctional
managed care pharmacy”, Am J Health Syst
Pharm.,59(11), pp. 1097-104,2002.
[14]S Sharma,”Applied Multivariate Techniques”,
Hoboken, NJ: John Wiley & Sons, p. 383, 1996.
[15] PJ Chiam, A Feyi-Waboso,”The use of control charts
in monitoring post cataract surgery endophthalmitis”,
Eye,23, pp.1028-1031, 2009.
[16] LA Hanna, “The use of statistical control charts to
monitor and improve the management of education
department resources”, Journal for Nurses in Staff
Development, 25(3), pp.118-124,2009.
[17] A Morton, A Clements, M Whitby,” Hospital adverse
events and control charts: the need for a new
paradigm”,Journal of Hospital Infection,73,pp.225-
231, 2009.

277