11884

This PDF is available at http://nap.nationalacademies.
org/11884
Advancing Quality Improvement Research:

Challenges and Opportunities: Workshop
Summary (2007)
DETAILS
60 pages | 6 x 9 | PAPERBACK
ISBN 978-0-309-10623-8 | DOI 10.17226/11884
CONTRIBUTORS
Samantha Chao, Rapporteur; Forum on the Science of Health Care Quality
Improvement and Implementation; Board on Health Care Services; Institute of
BUY THIS BOOK Medicine
FIND RELATED TITLES SUGGESTED CITATION

Institute of Medicine. 2007. Advancing Quality Improvement Research: Challenges
and Opportunities: Workshop Summary. Washington, DC: The National Academies
Press. https://doi.org/10.17226/11884.
Visit the National Academies Press at nap.edu and login or register to get:
– Access to free PDF downloads of thousands of publications
– 10% off the price of print publications
– Email or social media notiﬁcations of new titles related to your interests
– Special offers and discounts
All downloadable National Academies titles are free to be used for personal and/or non-commercial
academic use. Users may also freely post links to our titles on this website; non-commercial academic
users are encouraged to link to the version on this website rather than distribute a downloaded PDF
to ensure that all users are accessing the latest authoritative version of the work. All other uses require
written permission. (Request Permission)
This PDF is protected by copyright and owned by the National Academy of Sciences; unless otherwise
indicated, the National Academy of Sciences retains copyright to all materials in this PDF with all rights
reserved.
FORUMResearch:
Advancing Quality Improvement ON THEChallenges
SCIENCEandOFOpportunities:
HEALTH CARE QUALITY
Workshop Summary
IMPROVEMENT AND IMPLEMENTATION
ADVANCING
QUALITY
IMPROVEMENT
RESEARCH
CHALLENGES AND OPPORTUNITIES
WORKSHOP SUMMARY
Samantha Chao, Rapporteur
Forum on the Science of Health Care Quality

Improvement and Implementation
Board on Health Care Services
THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu
Copyright National Academy of Sciences. All rights reserved.

Advancing Quality Improvement Research: Challenges and Opportunities: Workshop Summary
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Gov-
erning Board of the National Research Council, whose members are drawn from
the councils of the National Academy of Sciences, the National Academy of Engi-
neering, and the Institute of Medicine.
This study was supported by a grant between the National Academy of Sciences
and the Robert Wood Johnson Foundation. Any opinions, findings, conclusions,
or recommendations expressed in this publication are those of the author(s) and
do not necessarily reflect the view of the organizations or agencies that provided
support for this project.
International Standard Book Number-13 978-0-309-10623-8

International Standard Book Number-10 0-309-10623-0
Additional copies of this report are available from the National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202)
334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page
at: www.iom.edu.
Copyright 2007 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America.
The serpent has been a symbol of long life, healing, and knowledge among almost
all cultures and religions since the beginning of recorded history. The serpent
adopted as a logotype by the Institute of Medicine is a relief carving from ancient
Greece, now held by the Staatliche Museen in Berlin.
Institute of Medicine (IOM) 2007. Advancing quality improvement research: Challenges

and opportunities, workshop summary. Washington, DC: The National Academies
Press.


The National Academy of Sciences is a private, nonprofit, self-perpetuating

society of distinguished scholars engaged in scientific and engineering research,
dedicated to the furtherance of science and technology and to their use for the
general welfare. Upon the authority of the charter granted to it by the Congress
in 1863, the Academy has a mandate that requires it to advise the federal govern-
ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the
National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the char-
ter of the National Academy of Sciences, as a parallel organization of outstand-
ing engineers. It is autonomous in its administration and in the selection of its
members, sharing with the National Academy of Sciences the responsibility for
advising the federal government. The National Academy of Engineering also
sponsors engineering programs aimed at meeting national needs, encourages
education and research, and recognizes the superior achievements of engineers.
Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of

Sciences to secure the services of eminent members of appropriate professions
in the examination of policy matters pertaining to the health of the public. The
Institute acts under the responsibility given to the National Academy of Sciences
by its congressional charter to be an adviser to the federal government and, upon
its own initiative, to identify issues of medical care, research, and education.
Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of

Sciences in 1916 to associate the broad community of science and technology
with the Academy’s purposes of furthering knowledge and advising the federal
government. Functioning in accordance with general policies determined by the
Academy, the Council has become the principal operating agency of both the
National Academy of Sciences and the National Academy of Engineering in pro-
viding services to the government, the public, and the scientific and engineering
communities. The Council is administered jointly by both Academies and the
Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and
vice chair, respectively, of the National Research Council.
www.national-academies.org

FORUM ON THE SCIENCE OF HEALTH CARE QUALITY

IMPROVEMENT AND IMPLEMENTATION
THOMAS F. BOAT (Co-Chair), Director, Children’s Hospital Research

Foundation and Chair, Department of Pediatrics, University of
Cincinnati College of Medicine, OH
PAUL H. O’NEILL (Co-Chair), Former U.S. Secretary of the Treasury,
Pittsburgh, PA
PAUL B. BATALDEN, Director, Health Care Improvement Leadership
Development, Dartmouth Medical School, Hanover, NH
IGNATIUS BAU, Program Director, The California Endowment, San
Francisco, CA
JAY E. BERKELHAMER, Senior Vice President of Medical Affairs,
Children’s Healthcare of Atlanta, GA
MARSHALL H. CHIN, Associate Professor of Medicine and Co-Director,
General Internal Medicine Research, University of Chicago, IL
CAROLYN M. CLANCY,* Director, Agency for Healthcare Research and
Quality, Rockville, MD
CATHERINE D. DE ANGELIS, Editor in Chief, Journal of the American
Medical Association Scientific Publications and Multimedia
Applications, Chicago, IL
JULIE L. GERBERDING,* Director, Centers for Disease Control and
Prevention, Atlanta, GA
JEREMY GRIMSHAW, Director, Clinical Epidemiology Program, Ottawa
Health Research Institute, Ontario, Canada
JEROME H. GROSSMAN, Senior Fellow, John F. Kennedy School of
Government, Harvard University, Cambridge, MA
JUDITH GUERON, Scholar in Residence, Manpower Demonstration
Research Corporation, New York
ANDREA KABCENELL, Executive Director for Pursuing Perfection,
Institute for Healthcare Improvement, Cambridge, MA
RICHARD KAHN, Chief Scientific and Medical Officer, American
Diabetes Association, Alexandria, VA
RAYNARD S. KINGTON,* Deputy Director, Office of the Director, The
National Institutes of Health, Bethesda, MD
JOEL KUPERSMITH,* Chief Research and Development Officer,
Veterans Health Administration, Washington, DC
LAURA C. LEVITON, Senior Program Officer, The Robert Wood Johnson
Foundation, Princeton, NJ
*Denotes Ex-Officio Members

BRIAN S. MITTMAN, Co-Editor in Chief, Implementation Science and

VA Greater Los Angeles Healthcare System, Sepulveda, CA
LESLIE NORWALK,* Acting Administrator, Centers for Medicare and
Medicaid Services, Washington, DC
STEPHEN M. SHORTELL, Blue Cross of California Distinguished
Professor of Health Policy and Management, University of California,
Berkeley
MARITA G. TITLER, Director, Institute for Translational Practice,
University of Iowa City Health Care System and University of Iowa,
Department of Nursing Services and Patient Care, Iowa City
IOM Forum Staff

SAMANTHA CHAO, Forum Director
MICHELLE BAZEMORE, Senior Program Assistant
MICHELE ORZA, Acting Board Director, Board on Health Care Services
vi

Reviewers
This report has been reviewed in draft form by individuals

chosen for their diverse perspectives and technical expertise, in
accordance with procedures approved by the NRC’s Report Review
Committee. The purpose of this independent review is to provide
candid and critical comments that will assist the institution in mak-
ing its published report as sound as possible and to ensure that
the report meets institutional standards for objectivity, evidence,
and responsiveness to the study charge. The review comments and
draft manuscript remain confidential to protect the integrity of the
deliberative process. We wish to thank the following individuals for
their review of this report:
ANDREA KABCENELL, Institute for Healthcare Improvement,

Cambridge, MA
PETER J. PRONOVOST, Departments of Anesthesiology and
Critical Care, Surgery, and Health Policy and Management, Center
for Innovations in Quality Patient Care, Quality and Safety Research
Group, The Johns Hopkins University, Baltimore, MD
STEPHEN M. SHORTELL, School of Public Health, University
of California, Berkeley
MARITA G. TITLER, Research, Quality and Outcomes Man-
agement, Department of Nursing Services and Patient Care, The
University of Iowa, Iowa City
vii

viii REVIEWERS
Although the reviewers listed above have provided many con-

structive comments and suggestions, they were not asked to endorse
the final draft of the report before its release. The review of this
report was overseen by coordinator ARTHUR A. LEVIN, of the
Center for Medical Consumers, New York. Appointed by the Insti-
tute of Medicine, he was responsible for making certain that an
independent examination of this report was carried out in accor-
dance with institutional procedures and that all review comments
were carefully considered. Responsibility for the final content of this
report rests entirely with the author and the institution.

Contents
SUMMARY 1
INTRODUCTION 9
1 LESSONS IN QUALITY IMPROVEMENT 11

Non-Health Care Service Sector, 11
Integrated Health Care Delivery System, 14
Hospital Perspective, 17
Nursing Perspective, 19
2 APPROACHES TO QUALITY IMPROVEMENT RESEARCH 21

Methods, 24
Areas Where More Knowledge Is Needed, 25
3 BARRIERS TO QUALITY IMPROVEMENT AND

QUALITY IMPROVEMENT RESEARCH 27
Barrier of Focus, 28
The Role of Context, 28
Where Does It Belong?, 29
Resource Barriers, 29
Barriers to Recruitment and Training, 30
Levers for Strengthening Quality Improvement Research, 32
Barriers to Performing Quality Improvement and Quality
Improvement Research, 33
Barrier of Sustainability, 35
ix

CONTENTS
4 OPPORTUNITIES 37
Short Term, 37
Long Term, 38
The Importance of Strategies for Change, 38
5 GENERAL REACTIONS 40
Leveraging Other Industries, 40
Context, 41
Areas for Further Discussion, 41
Clarifying Communication, 42
The Need for Further Knowledge, 42
REFERENCES 44
APPENDIXES
A Workshop Agenda 45
B Workshop Participants 47

Summary*
T
he Institute of Medicine’s Forum on the Science of Health Care
Quality Improvement and Implementation held a workshop
on January 16, 2007, in Washington, DC. The workshop had its
roots in an earlier forum meeting when forum members discussed
what is meant by the terms “quality improvement” and “imple-
mentation science” and became convinced that they mean different
things to different people. At the time, the members also discussed
the need to identify barriers to quality improvement research and
to implementation science. Thus the purpose of this workshop
was to bring people together from various arenas to discuss what
quality improvement is, and what barriers exist in the health care
industry to quality improvement and also to research about quality
improvement.
The summary that ensues is thus limited to the presentations
and discussions during the workshop itself. We realize that there is a
broader scope of issues pertaining to this subject area but are unable
to address them in this summary document.
*The Forum’s role was limited to planning the workshop, and the workshop sum-
mary has been prepared by the workshop rapporteur as a factual summary of what
occurred at the workshop.

ADVANCING QUALITY IMPROVEMENT RESEARCH
Lessons in Quality Improvement

The workshop’s first session was devoted to experiences that
various institutions have had with quality improvement. Recogniz-
ing the wealth of experiences available outside of health care ser-
vices, the workshop included presenters from outside the health care
service industry as well as from inside. This includes discussions
from a variety of perspectives: non-health care services, health plans,
hospitals, and nursing. It was not possible, however, to include
examples from all settings, including smaller physician practice set-
tings and long-term care settings.
Non-Health Care Service Sector

Although improving quality requires the use of specific tools,
developing those tools and putting them to use is only part of the
challenge. As Scot Webster of Medtronic, a manufacturer of medical
devices, explained, the larger part of improvement is actually chang-
ing culture and driving change.
Webster focused on three barriers to operating with high quality
and efficiency: lead time, external variability, and internal variabil-
ity. Lead time is the period of time from the beginning to the end of
a process. Variability refers to differences in conditions or in how a
process is performed; external variability refers to differences that
cannot be controlled by the process’s operator, while internal vari-
ability refers to processes that can be. Many tools exist to improve
quality and to deal with these barriers. Medtronic chose to combine
the tools of Six Sigma and Lean in an innovative technique called
Lean Sigma, which has positively affected Medtronic’s business.
Although it can produce impressive results, Lean Sigma should not
be seen as the answer for all quality problems, Webster cautioned.
Quality improvement is 30 percent application of various tools and
70 percent working together to create a culture of continual change,
he said, and to sustain quality improvement, institutions need to
incorporate it into their culture.
Lean Sigma is also not a replacement for creativity or the experi-
ence of health care providers, Webster said. While health care has a
high ratio of external to internal variability, and external variability
is by definition outside of one’s ability to control, Lean Sigma could
still be used as a tool to support the performance of health care
professionals, he said.

SUMMARY
Integrated Health Care Delivery System

As an integrated health care delivery system, Kaiser Perman-
ente has a unique approach to quality improvement, Scott Young of
Kaiser’s Care Management Institute said. Quality is integrated into
every level at Kaiser, from medical centers to the national program
office. Kaiser views quality, safety, service, and cost as the four
dimensions that lead to improvements in care.
One of the factors underlying differences in health care out-
comes is the uneven application of evidence-based care, which
results in unwanted variations. These variations can often be linked
to missed opportunities to improve quality and—at the extreme—to
safety issues, close calls, and near misses that occur every day. These
in turn, Young explained, are the basis for poor outcomes, adverse
events, increased morbidity and mortality, and potential increased
medical liability. Collectively, he said, these issues are viewed as
the “iceberg of safety,” with increased morbidity and mortality and
potential increased medical liability at the tip of the iceberg. Quality
improvement programs can help prevent patient safety issues in
health care.
The task of improving quality is made possible by support
systems available throughout Kaiser, such as its electronic medical-
record system called KP HealthConnect, its care-management
programs, the KP Elder Care Network, and the use of evidence-based
medicine. There are six components to the company’s approach to
quality: measurement and evaluation, care management, evidence-
based medicine, health information technology, innovative practice
models, and team-based care.
Hospital Perspective
Craig Miller of Baptist Health Care System described how this
hospital system changed its culture. In 1997, Miller said, Baptist was
a place that provided poor quality care. Once the hospital leadership
recognized that change was necessary to improve employee satisfac-
tion and to solve financial problems, Baptist began to focus its efforts
on the people associated with the system—the patients and the
employees. With this focus, Baptist transformed itself into a hospital
system that now provides excellent quality care, as evidenced by the
system winning the 2003 Baldrige Quality Award.
Baptist built its vision of change around five pillars of excellence:
people, service, quality, growth, and finance. In addition to these
pillars, Baptist used the Baldrige criteria for excellence to transform

itself. To operationalize the changes required to achieve their vision,

Baptist adopted five keys: create and maintain a great culture; select
and retain great employees; commit to service excellence; continu-
ously develop great leaders; and hardwire success with systems of
accountability.
Nursing Perspective
Nurses are central to improving the quality of health care deliv-
ery, said Marita Titler of the University of Iowa Hospitals and Clinics.
Titler presented four major points to illustrate the role of nurses in
quality improvement, including an overview of the quality improve-
ment program at the university, strategies used to implement perfor-
mance improvement, challenges in improving quality, and markers
of success. The University of Iowa Hospitals and Clinics bases its
implementation of new processes and procedures on seven prin-
ciples, Titler said. The first of these principles is that education is
necessary but not sufficient in order to change practice behaviors.
The second is that implementation is not necessarily sustainable;
constant tracking and improvement are required to improve the
likelihood that a change will be sustained. The third principle is to
facilitate doing the right things. The fourth is that data need to be
effectively transformed into useable and actionable information.
The fifth principle is to have a clear focus for implementation. The
sixth is coordination among all players, which is especially useful
in complex interventions. The seventh principle is to pilot or try
the intervention prior to implementing the change system-wide.
Improving care requires a number of strategies that integrate these
seven principles and at the center of them is engaging the workforce,
Titler said.
APPROACHES TO QUALITY IMPROVEMENT research

There is a lack of understanding of how to connect the differ-
ent strategies available for improving quality, said Paul Batalden of
Dartmouth. He offered the following formula as a way of thinking
about how the various factors of quality improvement fit together:
Baldrige
criteria are: leadership; strategic planning; focus on patients, other cus-
tomers, and markets; measurement, analysis, and knowledge management; workforce
focus; process management; and results (Baldrige National Quality Program, 2007).

SUMMARY
Generalizable Measured
Particular
scientific + → performance
context
evidence improvement
Generalizable scientific evidence and particular contexts link

together in a cycle that is a form of experiential learning. This cycle
not only describes how a large majority of evidence-based medicine
is developed, but it also captures how evidence-based medicine
is integrated into practice. Thus, Batalden suggested, experiential
learning can be seen as one of the underpinnings of quality improve-
ment and quality improvement research.
Jeremy Grimshaw of the Ottawa Health Research Institute
offered a different approach to quality improvement research. Imple-
mentation research can be described as studies of how the uptake of
research findings is promoted. Implementation research focuses on the
challenge of delivering evidence-based care to patients, specifically
on the technical aspects of care. The aim is to develop a generaliz-
able evidence base that can be used to improve the implementation
of research findings and enhance decision making at the local level.
This research is inherently interdisciplinary, involving health care
professionals, organization scientists, engineers, and others.
Despite the suggestion by some attendees that these approaches
to quality improvement research oppose one another, others thought
that the discussion revealed more similarities than differences. In
particular, Batalden and Grimshaw agreed on the purpose of quality
improvement research and also agreed that the evidence base needs
to be developed to the point that it can build on itself. Batalden and
Grimshaw stated that both approaches were necessary to develop
the needed body of knowledge.
Methods
Quality improvement is analyzed using a variety of study
designs, including systematic reviews, controlled trials, case reports,
and hybrid quantitative/qualitative reports, Batalden said. These
different methods have different strengths, each with its own set of
advantages and disadvantages.
There is disagreement in the field about the use of what some
believe to be the “gold standard,” randomized controlled trials
Implementation research is the term used in Europe to refer to quality improve-

ment research, noted Grimshaw. They are not identical, but cover many of the same
areas.

(RCTs). Some people do not believe that RCTs are useful in complex
social contexts, such as quality improvement processes, while others
believe RCTs to be an extremely valuable method for evaluating
these interventions. Given that different interventions lend them-
selves to specific evaluation methods, Grimshaw and Batalden con-
cluded that one should always attempt to choose the best possible
study design, given the individual circumstances.
Barriers to Quality Improvement and Quality

Improvement Research
There is very little data available to guide the development of
quality improvement research, of health sciences research, and of
medicine in general, said Harold Pincus of Columbia University and
New York-Presbyterian Hospital. This lack of data is closely related
to the eight major barriers to quality improvement and to quality
improvement research that workshop participants identified.
The first barrier is that quality improvement efforts can have
many divergent purposes. Some see the purpose as improving per-
formance, a process that occurs mainly through experiential learning.
This process differs significantly from scientific research, whose pur-
pose is to discover generalizable truths through hypothesis testing,
noted Frank Davidoff of the Institute for Healthcare Improvement.
A second barrier is the role of specific contexts. Understanding
the effects of specific local contexts and characteristics of what is
generalizable across settings is extremely valuable in the implemen-
tation of interventions, Grimshaw said.
The third barrier is the lack of agreement about which academic
area should be home to quality improvement research, Pincus said.
Quality improvement research could potentially be considered an
interdisciplinary research field, serving as a bridge between multiple
disciplines. While many agree with the concept of interdisciplinary
research in theory, it is extremely difficult to put into practice.
The fourth barrier is the “mismatch” between training and prac-
tice: Most people doing medical quality improvement projects have
little or no research training, while most people with research train-
ing are not doing quality improvement projects. Different strategies
will need to be developed for recruitment in different audiences.
Fifth, ethical oversight in quality improvement remains largely
ambiguous and can be a large obstacle for many researchers. Quality
improvement can be seen either as an intrinsic element of clinical
care and medicine or as a form of clinical research. This in turn leads
to questions as to whether quality improvement research should be

SUMMARY
considered human subjects research, which would require ethics

review and institutional review board (IRB) approval.
The sixth barrier identified during the workshop is the existence
of methodological differences between the biological sciences and the
social sciences. Quality improvement research faces the same chal-
lenges—such as biases, confounders, and difficulties with measure
ment—that clinical research does. However, quality improvement
studies are not subject to the tightly controlled conditions of clinical
interventions. It is therefore difficult to know if “proven interven-
tions” are generalizable.
The seventh difficulty facing the development of quality improve-
ment research is that much of what is published is poorly conducted.
Because of a variety of factors, only a relatively small amount of
quality improvement research is actually published, Davidoff said,
and much of what is published is not generalizable and so fails to
provide a basis for future efforts upon which to build.
The last barrier identified during the workshop was the barrier
of communication. The lack of a common vocabulary for quality
improvement and implementation research terms has hindered fur-
ther progress, Grimshaw said.
Opportunities
Both short-term and long-term opportunities exist for strength-
ening the science of quality improvement. In the short term, the
opportunities identified by workshop participants centered on
strengthening the evidence base for quality improvement. This
can be achieved by using the most rigorous methods possible to
assess interventions and by clarifying the focus of quality improve-
ment projects. Long-term opportunities include creating strategies
to improve professional development and effect cultural change
among all stakeholders.
General Reactions
General reactions to the workshop discussions were given at
the end of the day by both forum members and audience members.
Many of their comments focused on the need to leverage experi-
ences from other disciplines. The role of context should also be more
carefully studied, as well as communication between researchers
and between researchers and implementers. Other areas for the
forum to pursue were also proposed.


Introduction
W
ith its Forum on the Science of Health Care Quality
Improvement and Implementation, the Institute of Medi-
cine (IOM) convenes representatives from academia,
clinical practice, government, and industry in a neutral setting where
they can discuss various ideas about improving the science behind
health care quality improvement and implementation. Through
these discussions, forum members attain a better understanding of
what the needs of the science are, and they begin breaking down the
communication barriers that prevent advances in the field.
The workshop “The Path to Quality Improvement: Approaches
and Barriers” was held on January 16, 2007, in Washington, D.C.
It was the result of a forum conversation that had taken place in
December 2006. The forum had identified a need to understand
what was meant by the terms “quality improvement” and “imple-
mentation science,” and during the ensuing discussion it became
clear to forum participants that these terms mean different things to
different people. Forum participants also discussed the need to iden-
tify the barriers to quality improvement research and to implemen-
tation science. The purpose of this workshop was therefore to bring
people together from various arenas to discuss the scope of quality
improvement in a broad sense. The forum members felt it would be
valuable to hear about lessons learned not only from within health
care settings but from outside of health care as well. Because of the
limited time available at this workshop, not all relevant perspectives

10 ADVANCING QUALITY IMPROVEMENT RESEARCH
and issues could be discussed. The forum members plan to integrate

other views and topics in its future activities.
The following chapters describe and summarize the presenta-
tions and discussions that took place during that workshop, and the
content is therefore limited to what was presented and discussed
during the workshop itself. We realize that there is a broader scope
of issues pertaining to this subject area, but we are unable to address
them in this summary.

Lessons in Quality Improvement∗
T
his section includes discussions from a variety of perspectives:
non-health care services, health plans, hospitals, and nursing.
It was not possible, however, to include examples from all
settings, such as smaller physician practice settings and long-term
care settings.
Non-health care Service sector

Although improving quality requires the use of specific tools,
developing these tools and putting them to use is only part of the
challenge. As Scot Webster of Medtronic explained, the larger part
of improvement is changing culture and driving change.
Webster offered Medtronic’s experience with quality improve-
ment as an example. Medtronic manufactures a wide variety of med-
ical devices, including pacemakers and insulin pumps. Every five
seconds, Webster said, somewhere around the world a Medtronic
device is implanted. In 2006 Medtronic’s net sales were $11.3 billion;
the company spends approximately 10 percent to 15 percent of its
revenue each year on research and development. The quality of its
products is imperative, but as Medtronic is a high-volume company,
*The Forum’s role was limited to planning the workshop, and the workshop sum-
mary has been prepared by the workshop rapporteur as a factual summary of what
occurred at the workshop.
11

the efficiency of its operations and the flow of its processes are also
critical factors in its success.
For these reasons Medtronic set itself the goals of assuring that
it produced high-quality products while at the same time increas-
ing efficiency and improving flow. Webster highlighted three issues
that Medtronic found to be important in reaching these goals: lead
time, external variability, and internal variability. Lead time is the
period of time from the beginning to the end of a process. A patient
who must sit in the waiting room of an emergency room for three
hours is an example of a need to reduce lead time. Variability refers
to differences in conditions or in how a process is performed; exter-
nal variability refers to differences that cannot be controlled by the
process’s operator, while internal variability refers to processes that
can be. An epidemic would be an example of external variability,
Webster said, while incorrect prescriptions would be an example
of internal variability. If an organization can reduce lead time and
internal variability, he said, it can gain the flexibility it needs to
manage external variability, which in turn will lead to improved
customer experiences and reduced costs. These three issues—lead
time, external variability, and internal variability—are important not
just in manufacturing, Webster said, but in health care as well.
There are a number of tools that can be used to improve quality
and focus on the problems of lead time and variability, Webster said.
In its efforts to maximize profits, Medtronic chose two: Six Sigma
and Lean. In particular, Medtronic combined the two tools to create
an innovative technique it called Lean Sigma. The company created
Lean Sigma for three reasons, Webster said.
The first reason was that the goals of both of these tools are to
decrease error and reduce waste from processes. Six Sigma focuses
on the efficiency of a single process, using standard deviations as a
measure to track performance. The methodology Six Sigma follows
is called DMAIC, for Define, Measure, Analyze, Improve, and
Control. The first step is to characterize problems with products or
outcomes by defining what the problems are and then finding ways
to measure performance. After measuring performance, these result-
ing data undergo statistical analyses to identify the problem with the
process. Only when the process problem is identified can the process
be improved, whether through automation or perhaps by something
as simple as turning off a knob. The last step of DMAIC is control,
which refers to the need to sustain change so that the problem does
not recur. Statistical testing and evidence are two essential compo-
nents of Six Sigma, Webster noted.
Lean also follows the DMAIC methodology, Webster explained,

LESSONS IN QUALITY IMPROVEMENT 13
but its focus is on the flow of multiple processes as opposed to the

efficiency of a single process. Lean is a tool for improving qual-
ity, as guided by customer demands and the desire to minimize
waste, while also allowing for flexibility. A central part of Lean is
the balancing of resources. In a hospital setting, for example, that
could mean that when a nurse in one department has spare time, he
can be retasked to help out in another area. In a factory, machines
can be moved around so that the flow of parts from one machine
to another may require less time. If each process in a system takes
the same amount of time, Webster argued, patients would not wait
unnecessarily in a hospital and inventory would not sit idle between
processes in a factory. In Lean, Webster said, all inputs are measured
so that customer requirements can be met with minimal wasted
resources.
Webster estimated that Medtronic currently is running over
1,000 Lean Sigma projects, with some dramatic results. For example,
a factory in Galway, Ireland that manufactures stents reduced its
lead time from 17 hours to 1.7 hours and at the same time increased
output from 500 units per shift to 800 units while using approxi-
mately the same number of employees. The factory used Lean to
balance flow and production and Six Sigma to reduce variability in
many of the processes.
The second reason Medtronic chose to use Lean Sigma, Webster
said, is because Lean and Six Sigma complement each other well.
Depending on the problem, either Lean or Six Sigma can be used,
he said, and some situations will require the use of both.
The third reason Medtronic combined Lean and Six Sigma is
because of their ability to form a science out of process. Analyzing
processes in order to identify reasons for variability and to deter-
mine which processes statistically yield better outcomes requires
that data be accumulated about those process, and through Lean
Sigma, Medtronic has developed an evidence base that it uses for
improvement. The DMAIC methodology and the science of Lean
Sigma apply to all systems with processes, Webster said. In par-
ticular, they are not limited to the manufacturing world and would
make sense to apply in health care.
But quality improvement is more than just tools, Webster said.
Indeed, he estimated it to be only about 30 percent application of
various tools and 70 percent working to create a culture of continual
change. Once an improvement event has occurred, the operators
have been equipped with the tools needed to improve, but sus-
taining the improvement demands that the operators incorporate
quality improvement as part of their own culture. Medtronic has

experienced this transformation, as demonstrated by the thousands

of employees who now focus on quality, Webster said. “You have
to get it in the DNA. You have got to get it in the culture. It is
not about being good at the projects. It is about having this as the
way we lead.” Once quality improvement is embedded in the cul-
ture, Webster said, quality experts are no longer necessary because
improvement is self-sustaining. By embedding quality improvement
in its culture, Medtronic is developing its future leaders, he said.
At Medtronic, each business runs itself without a high level of
corporate command, which has allowed each business to develop its
own culture and its own priorities. Each business must then inde-
pendently find the need to use tools such as Lean Sigma.
In offering advice to the health-care industry, Webster first cau-
tioned that Lean Sigma should not be seen as the answer for all
quality problems. There are many problems that require a “just do
it” approach, for instance, smaller projects that do not require Lean
Sigma. Webster suggested keeping the focus of quality improvement
efforts small at first. Involvement in other operations could distract
from the focus of process improvement, as the goal of improvement
should not be to improve margins or technology, but to produce
better outcomes. Lean Sigma should be used to support the creativity
and experience of health care providers, not as a replacement.
Integrated health care delivery system

As an integrated health care delivery system, Kaiser Permanente
has a unique approach to quality improvement, said Scott Young of
Kaiser’s Care Management Institute. The communities Kaiser serves
and its 8.6 million members are at the center of Kaiser’s mission.
Having an integrated delivery system means that Kaiser’s multi-
specialty group practices, its hospitals, and its insurance are all
affiliated. This allows the company to coordinate care across its
members’ lives; to be accountable for the quality, cost, safety, and
service of its members; to have a unified medical record that enables
capabilities to measure and improve care; and to directly link cover-
age design and services rendered, Young said.
Quality is integrated into every level at Kaiser, from medical
centers to the national program office, Young said. Kaiser views
quality, safety, cost, and service as the four dimensions that lead
to improvements in care. The quality of care received around the
country is uneven, Young said, using a chart of 30-day mortality
after acute heart attack (Figure 1) to demonstrate his point. The
30-day mortality of Kaiser patients averaged 8 percent, with values

25
= statistically sig. p<0.01

20
KP Non KP Providers
15
13%
10
8%
Source: 2002 study by the California Office of Statewide Health Planning and Development (OSHPD) which found that
FIGURE 1 Comparison
Kaiser Permanente of 30-day
members have a significantly greatermortality rates after acute heart attack.
Kaiser
chance ofPermanente members
survival from heart attacks than-nonhave a significantly
members. greater
The study, released in chance
February 2002, showedof
thatsurvival
the
survival of heart attack patients at all Kaiser Permanente hospitals
from heart attacks than non-members. The study, released in February
was better than the statewide average. Overall mortality was 8 percent versus the statewide average of 12 percent.2002,
showed that the survival of heart attack patients at all Kaiser Permanente
hospitals was better than the statewide average. Overall mortality was 8
percent versus the statewide average of 12 percent.
SOURCE: 2002 study by the California Office of Statewide Health Planning
and Development (OSHPD).
1-1
ranging from 7 percent to 10 percent. This is in marked contrast to

other providers, which had an average mortality rate of 13 percent
and a much higher range.
Underlying this difference in outcomes, Young said, are uneven
applications of evidence-based care, which result in unwanted varia-
tions. In particular, these variations can often be linked to missed
opportunities to improve quality and—at the extreme—to safety
issues, close calls, and near misses that occur every day. These
safety issues and near misses can often result in poor outcomes
and adverse events. These are the basis for what Young views to be
the “iceberg of safety,” with increased morbidity and mortality and
potential increased medical liability at the tip of the iceberg. Too
often the underlying reasons for increased morbidity and mortal-
ity go unnoticed. Quality improvement programs can help prevent
patient safety issues in health care.

At Kaiser, 1 percent of members are associated with approxi-

mately 30 percent of total costs, Young said. The majority of these
high-cost individuals are members living with multiple chronic con-
ditions. Service is the dimension supporting the other three dimen-
sions of quality, safety, and cost. It is Kaiser’s belief that there is a
need to make care patient-centric, to address critical needs for its
communities, and to meet both member and purchaser expectations.
Meeting goals in these four dimensions, Young said, is Kaiser’s key
to improving the quality of care for patients.
The task of improving quality is made possible by support
systems available throughout Kaiser, such as its electronic medi-
cal record system called KP HealthConnect, care management
programs, KP Elder Care Network, and the use of evidence-based
medicine. There are six components to the company’s approach to
quality: measurement and evaluation, care management, evidence-
based medicine, health information technology, innovative practice
models, and team-based care.
Young noted that improvement can only be made in areas that
can be measured. To evaluate its progress, Kaiser measures its per-
formance with common metrics, such as the Health Plan Employer
Data and Information Set (HEDIS) and the Joint Commission on
Accreditation of Healthcare Organizations (JCAHO), as well as
tracks progress in safety, service, and efficiency. Data can be viewed
nationally, by region, or by individual medical center.
Kaiser’s care management programs are focused on making the
right thing to do the easy thing to do. Young described the need to
use tools such as guidelines, effective and innovative care models,
support teams of professionals, and technology. An important part
of Kaiser’s care management programs is to provide care that is
personal, effective, and efficient to members with chronic illness. An
example Young used is an improvement program aimed at diabetic
members and other members at risk for cardiovascular disease.
Improvements were seen in those patients prescribed a specific
drug regimen, including the use of aspirin, angiotensin-converting-
enzyme inhibitors (ACE-I) or angiotensin-receptor blockers (ARBs),
plus lipid-lowering drugs. The care these members received was
noted in select provider panels, and the effect of the regimen was
studied. Kaiser modeled the effects of the drug combination, which
showed significant reductions in cardiovascular disease and lower
costs. The drug regimen is currently being rolled out in all eight of
Kaiser’s regions, noted Young.
Kaiser has invested heavily in its health information technology
programs, which consist of an integrated electronic health record (KP

HealthConnect), kp.org (personal health record available to all mem-

bers), and support tools to coordinate care for patients with ongoing
and chronic illnesses. For the electronic health record alone, Kaiser
will invest $4 billion. These programs have made possible more effec-
tive use of evidence, improved provider and member communica-
tion, and better support for providers in the delivery of care.
Evidence-based medicine is being embedded into KP Health-
Connect to help integrate proven and effective care into the delivery
systems in a real-time manner. Evidence-based medicine in this
capacity exists in a variety of forms, such as clinical guidelines,
preventive services, clinical libraries, outcomes reports, best practice
alerts, and health maintenance reminders. These highly actionable
tools are currently being integrated into clinical work streams and
serve as a significant component in Kaiser’s quality improvement
efforts, Young said. These implementation efforts involve providers
and staff at all levels, from national guideline directors to frontline
clinicians. However, gaps in the evidence base must be filled in
order to produce better care, noted Young.
Team-based care has long been a fixture at Kaiser. The multi-
specialty groups there strive to provide a host of physical and vir-
tual services to best serve members’ needs. These capabilities now
include email and virtual provider visits and electronic pharmacy
and lab services. As health care shifts more toward the home, vir-
tual, and self-care environments, more innovative care models will
need to be developed, Young concluded.
Hospital Perspective
Craig Miller of Baptist Health Care System described how this
particular hospital system changed its culture. Baptist is a four-
hospital system in northern Florida and southern Alabama. In 1997,
Miller said, Baptist was a place that provided poor quality care and
had low employee morale, below-average physician satisfaction,
and poor patient satisfaction. When the hospital leadership realized
that change was necessary to improve employee satisfaction and
to solve financial problems, Baptist began to focus its efforts on the
people associated with the system—the patients and the employees.
With this focus, Baptist transformed into a hospital system that now
provides excellent quality care, as evidenced by the system winning
the 2003 Baldrige Quality Award.
Baptist built its vision of change around five pillars of excellence,
Miller said: people, service, quality, growth, and finance. In addition
to these pillars, Baptist focused on the Baldrige criteria for excel-

lence to transform itself. To operationalize the changes required to

achieve their vision, Baptist adopted five keys: create and maintain
a great culture; select and retain great employees; commit to service
excellence; continuously develop great leaders; and hardwire suc-
cess with systems of accountability.
Miller said that the first key—create and maintain a great cul-
ture—is based on strong communication. This communication is
enabled by involving all of Baptist’s employees in feedback, educa-
tion, surveys, and employee forums. To make this meaningful and
engaging, Baptist changed its culture and began in a structured
manner to share stories about patients, innovations by individual
providers, and patient-provider interactions.
Employees are the foundation of the organization, Miller said,
and many improvements begin as innovations by employees. There-
fore, Baptist’s second key to excellence is selecting and retaining
great employees. Employees are given a sense of ownership over
the selection process. For example, interviewees are asked ques-
tions by their potential peers during the interview process. Baptist
also focused on retaining employees. Management personally rec-
ognizes those individuals who exemplify desired behaviors, and
every quarter outstanding employees are acknowledged for being
exceptional leaders by the company’s legends and champions pro-
gram. One particular focus of this second key to excellence is engag-
ing physicians. By making physicians a central part of the culture
change, Baptist was able to improve both clinical aspects of care and
employee morale. It was noted that although employee satisfaction
does not necessarily result in delivery of high quality care, employee
satisfaction is used as a quality indicator. Employee morale has
improved tremendously and turnover has decreased, leading to cost
savings at Baptist.
A commitment to service is the third key to excellence. An exam-
ple Miller used here is a strategy called scripting, where employees
are given scripts to follow during particular situations. By using
scripting, a consistent level of service performance can be ensured,
Miller said. Furthermore, hospital leaders are constantly going
on rounds, making themselves available to their employees and
patients so they can address problems firsthand. Apologizing for
mistakes and then learning from them has become a large part of
Baptist’s culture, Miller added.
Baldrigecriteria are: leadership; strategic planning; focus on patients, other cus-

tomers, and markets; measurement, analysis, and knowledge management; workforce
focus; process management; and results (Baldrige National Quality Program, 2007).

The fourth key to excellence is continuous development of great

leaders. Baptist puts a lot of effort into developing its leaders. “The
big difference between winners and losers, whether they are orga-
nizations or individuals, is that winners understand that learning,
teaching, and leading are inextricably intertwined,” Miller said,
quoting Noel Tichy, director of Global Business Partnership at the
University of Michigan. Baptist teaches its leaders how to learn and
listen, skills that are disseminated to other employees through what
Miller calls “cascade learnings.”
Hardwiring success through implementing systems of account-
ability is the fifth key. An important element of this is having 90 day-
work plans that require action to help reach a system goal. These
plans require each pillar of excellence to be addressed, establishing
goals and objectives, identifying responsible individuals, and pro-
viding measurable outcomes for each. Transparency in communica-
tion and goals is critical, explained Miller.
Baptist’s focus on quality has led it to become one of Fortune’s
100 Best Companies to Work For for the sixth straight year. The
quality of care provided has improved dramatically, Miller said. Its
journey led it to the Baldrige National Quality Award. Its journey
began by making changes within.
Nursing perspective
Nurses are central to improving the quality of health care deliv-
ery, said Marita Titler of the University of Iowa Hospitals and Clinics.
In her facility, the department of nursing has a quality management
committee. The group has broad representation, bringing together
nurses from each clinical division and from other areas of focus,
such as infection control. Work groups have also been put in place
that report to the quality-management committee to target specific,
interdisciplinary issues, such as pain, skin care, fall prevention, and
medication management.
Improvements are driven by data. Issues that data are collected
on include medication errors, falls, pain indicators, and the Centers
for Medicare and Medicaid Services indicators, such as discharge
instructions for heart failure patients. Interdisciplinary approaches
are often required to make improvements, Titler said.
The University of Iowa Hospitals and Clinics bases its imple-
mentation of new processes and procedures on seven principles,
Titler said. The first of these principles is that education is necessary
but not sufficient in order to change practice behaviors. The second
is that implementation is not necessarily sustainable; constant track-

ing and improvement is required to improve the likelihood that a

change will be sustained. The third principle is to facilitate doing
the right things. The fourth is that data need to be effectively trans-
formed into useable and actionable information. The fifth principle
is to have a clear focus for implementation. The sixth is coordination
among all players, which is especially useful in complex interven-
tions. And the seventh principle is to pilot or try the intervention
prior to implementing the change systemwide.
Improving care requires a number of strategies that integrate
these seven principles, and at the center of them is engaging the
workforce, Titler said. Data are necessary for making care evidence
based, and data should be collected not only for outcomes of care
but also for the care processes that contribute to those outcomes.
Data should be analyzed before, during, and following implementa-
tion of evidence-based practice changes to understand the impact of
the improved care delivery.
Presenting data at the patient care unit or clinic level using
statistical tools is helpful for nurse managers, and facilitates staff
involvement in process improvements. Such tools include statistical
process control charts, run charts, and Pareto charts. Other important
strategies for improving care include listening to staff, presenting
and discussing the evidence base for clinical practices such as fall
prevention, and engaging unit-based change champions in process
improvement and point-of-care coaching. The work of evidence-
based practice improvement must be made visible through mecha-
nisms such as internal newsletters, publications, and senior leadership
reports. Interdisciplinary and interdepartmental collaboration are
essential and the role of leaders is critical in engaging employees in
change, Titler said. Leaders need to develop action plans to increase
transparency, such as defining accountable persons, identifying an
intervention’s effect on patient care, and making sure the plan for
implementation is well understood. Without a leader’s vision and
guidance, effective and well-planned practice improvements are
unlikely to be sustained. Key questions for evaluating the success of
quality improvement programs include: Have goals of the prior year
been achieved? Are core metrics improving? Are people working col-
laboratively across departments and disciplines to improve patient
care? Are staff seeking out quality improvement personnel for guid-
ance? Challenges in improving quality of care, Titler said, include
system issues such as using clinical documentation systems, compet-
ing demands by various external agencies, and using a mechanistic
rather than a complex adaptive system approach. Improving systems
and care processes is the role of all involved in health care.

Approaches to
Quality Improvement Research
A
lthough a number of different quality improvement strate-
gies exist, Paul Batalden of Dartmouth noted that overall
there is a lack of understanding of how to connect these dif-
ferent strategies in efforts to improve quality. Thus Batalden set forth
a framework for connecting various strategies in quality improve-
ment and quality improvement research. That framework, Batalden
said, rests on three assumptions. First, the overall goal is to achieve
better health care. Second, better health care should be based on as
much knowledge as possible. And third, improving the quality of
health and of health care is not as easy as it first seems (Batalden
and Davidoff, 2007).
Batalden defined quality improvement as “the combined and
unceasing efforts of everyone—health care professionals, patients
and their families, researchers, payers, planners, educators—to
make changes that will lead to better patient outcome, better sys-
tem performance, and better professional development” (Batalden
and Davidoff, 2007, p. 2). If quality improvement efforts are to be
sustainable, Batalden said, all three of these goals must be focused
on, not just one or two. Unfortunately, he said, in most cases the
three goals are pursued independently, not collectively. For example,
efforts are often made to improve patient outcomes and system
performance, but the formative process of how health care profes-
sionals are trained and how to achieve better health care through
21

improvements in the quality of this training has generally received

little attention.
Batalden offered the following formula as a way of thinking
about how the various factors of quality improvement fit together:
Generalizable Measured
Particular
scientific + → performance
context
evidence improvement
In short, by taking general scientific understandings and apply-

ing them to a particular context, one should be able to achieve a
measurable improvement in performance. There are five distinct
knowledge systems underlying this formula, Batalden explained.
The first relates to generalizable scientific evidence, which Batalden
described as understanding how to minimize the effect of context.
The second system of knowledge focuses on understanding the
specific variables that define a particular context. The third involves
measuring the stability of change over time; this system of knowl-
edge underlies measured performance improvement, which is the
outcome—that is, the right-hand side—of the above formula. While
Batalden stresses the importance of measurements over time, such
measures often do not exist; instead, it is often the case that only dis-
crete pre-and post-intervention measurements are taken. The fourth
system of knowledge is implied in the plus sign in the formula: It is
the knowledge involved in choosing the correct evidence to a par-
ticular context. The fifth system of knowledge, which Batalden calls
the knowledge of execution, is the formula’s arrow.
Generalizable scientific evidence and particular contexts link
together, creating a cycle that is a form of experiential learning. The
cycle begins with testing the implications of concepts in new situa-
tions, Batalden explained. These tests lead to concrete experiences,
and observations and reflections made from these experiences are
then analyzed to form new abstract concepts and generalizations,
which can then be tested in new situations, beginning another cycle.
Without further testing and analysis, however, this is just experience.
This cycle not only describes how a large majority of evidence-based
medicine is developed, but it also captures how evidence-based
medicine is largely integrated into practice. In fact, Batalden said,
the quality improvement field has been substantially handicapped
by the idea that only one method to control quality can be used at
a time to affect change. While a lot can be learned from studying
the effects of individual quality improvements, much can also be
learned experientially from a multitude of efforts to improve health

APPROACHES TO QUALITY IMPROVEMENT RESEARCH 23
care. Thus, Batalden suggested, experiential learning can be seen as

one of the underpinnings of quality improvement.
Jeremy Grimshaw of the Ottawa Health Research Institute
offered a different approach to quality improvement research. There
is a consistent failure to translate research findings into clinical prac-
tice, as evidenced by studies showing 30 percent to 40 percent of
patients not receiving the care they should (McGlynn et al., 2003)
and 20 percent to 25 percent of patients receiving unnecessary or
potentially harmful care (Grol, 2001), and, Grimshaw said, over
coming this failure is a major focus of health care quality improve-
ment. One way to fix this failure, he suggested, would be to instill
evidence into clinical practice at a variety of levels: the structural, the
organizational, the group or team, and the individual health profes-
sional levels (Ferlie and Shortell, 2001). To target the stakeholders at
the various levels, different interventions would be needed, depend-
ing on the identified barriers. The challenge, Grimshaw said, will
be to equip health care providers with the correct tools to properly
deliver evidence-based treatments.
Implementation research can be described as studies of how
the uptake of research findings is promoted, explained Grimshaw.
Implementation research focuses on the challenge of delivering
evidence-based care to patients. The aim is to develop a generaliz-
able evidence base that can be used to improve the implementa-
tion of research findings and enhance decision making at the local
level. This research is inherently interdisciplinary, involving health
care professionals, organization scientists, engineers, and others,
Grimshaw noted.
Some workshop attendees questioned just how much these two
views of quality improvement research differ. Despite the sugges-
tion by some attendees that these approaches to quality improve-
ment research oppose each other, others thought that the discussion
revealed more similarities than differences. In particular, Batalden
and Grimshaw agreed on the purpose of quality improvement
research and also agreed that the evidence base needs to be devel-
oped to the point that it can build on itself.
Implementation research is the term used in Europe to refer to quality improve-

ment research, noted Grimshaw. These terms are not equal, but they cover many of
the same areas.

Methods
Quality improvement is analyzed using a variety of methods,
Batalden said. These include systematic reviews, controlled trials,
case reports, and hybrid quantitative/qualitative reports. These dif-
ferent methods have different strengths. The workshop discussions
focused only on systematic reviews and randomized controlled
trials.
Rigorous evaluations add valuable information to the overall
knowledge base and provide a solid base of research that can be built
upon, Grimshaw said. The majority of such evaluation approaches
today emphasize a diagnostic process that first identifies barriers,
then addresses the most important barriers with specific interven-
tions, and, finally, evaluates the effects of the different interventions
through rigorous evaluation designs.
Randomized controlled trials (RCTs) and other such rigorous
research methods can provide better evidence of effectiveness than
other types of methods when assessing specific questions, Grimshaw
argued. RCTs should be used to evaluate such questions as what the
likelihood is that an intervention will yield the desired effect, what
the direct effects will be of that intervention and of its alternatives,
under what circumstances the intervention will succeed or fail, and
what resources are required to do the intervention, he said.
Grimshaw described the disagreement in the field about the use
of what some believe to be the “gold standard,” RCTs, when mak-
ing evaluations of the effectiveness and efficiency of interventions.
There is some antipathy to the use of RCTs in complex social con-
texts, such as quality improvement processes, while others believe
RCTs to be an extremely valuable method of evaluating these inter-
ventions. Responding to those who do not believe in using RCTs
for quality improvement, Grimshaw said that there are many mis-
conceptions about RCTs. It is often assumed by critics, for instance,
that all randomized trials use the methods of explanatory (focused
on efficacy) drug trials that require tight inclusion criteria, that they
largely ignore context, and that they are expensive. But this is not
necessarily true, Grimshaw argued. Randomized trials of quality
improvement interventions tend to be pragmatic (focused on effec-
tiveness) and attempt to elucidate whether an intervention will be
effective in a real-world setting, not in an optimal one. Such RCTs
frequently have broad inclusion criteria and can be designed to gain
better understanding of the influence of context on the effectiveness
of quality improvement interventions and why changes occurred.
One method of achieving this, for instance, is to use observational

APPROACHES TO QUALITY IMPROVEMENT RESEARCH 25
approaches in conjunction with data from RCTs to test multilevel

hypotheses about which interventions work and which do not.
Quasi-experimental trials are another method of evaluating
interventions, Grimshaw said. Additionally, uncontrolled before-
and-after studies, controlled before-and-after studies, and inter-
rupted time-series analyses are all frequently used alternatives to
RCTs. However, many of the criticisms of RCTs also apply to these
other methods. RCTs build on the knowledge generated by obser-
vational studies and case studies, Grimshaw said.
Grimshaw and Batalden agreed that the best method to evaluate
a given intervention will depend on the specifics of that interven-
tion, and one should always attempt to choose a mixture of the best
possible method, given the individual circumstances.
Areas where more knowledge is needed

Based on Batalden’s proposed model of quality improvement,
many areas exist where more knowledge is needed to achieve better
patient outcomes, system performance, and professional develop-
ment. To improve patient and population outcomes, examples of
areas where knowledge is needed include better measures of out-
comes, improved confidence in these measures, and understand-
ing of the causes of variation in both outcomes and measurement.
Improving system performance will require enhanced measures, a
better understanding of the various evaluation methods used, and
an understanding of the role of standards in introducing change.
Better professional development requires a more comprehensive
understanding of such issues as competence, accreditation and licen-
sure, and fostering of cooperation among professionals. Another area
of knowledge that needs building, Batalden said, is how employees
are trained, developed, and held accountable. Two other areas that
are not well understood are the roles that leadership and governing
boards play in quality improvement. Building this body of research
knowledge will demand developments in many areas, Grimshaw
said. Theoretical developments are needed to provide frameworks
and predictive theories for creating generalizable research, such as
understanding how interventions are chosen and interpreted and
how to change individual and organizational behaviors. Method-
ological developments are also required, as well as exploratory stud-
ies aimed at understanding the experiential learning that takes place
in individual settings and organizations. Rigorous evaluations need
to be undertaken of the effectiveness and efficiency of interventions,

Grimshaw said; these bodies of knowledge can be synthesized to

determine the generalizability of findings.
Finally, Grimshaw noted that partnerships are needed to encour-
age communication among various stakeholders, such as theorists,
researchers, and implementers. Such partnerships are necessary in
order to understand what types of knowledge are needed and how
that knowledge can best be developed; but, he said, few such part-
nerships exist today.

Barriers to Quality Improvement and

Quality Improvement Research
I
n the early 1990s, a number of hospitals created rapid-response
teams or medical-emergency teams to identify and intervene
early in the care of clinically critical patients. The promise of
these teams was that they could help provide better care to many
of a hospital’s most at-risk patients, and in some instances the rates
of cardiac arrest, post-surgical complications, and overall mortal-
ity were shown to have improved, at least informally. Over time,
rapid-response teams became increasingly popular. In 2003 there
were about 100 such teams in U.S. hospitals, and by 2005 there were
a couple of thousand teams, said Frank Davidoff of the Institute for
Healthcare Improvement.
This widespread adoption, however, was not accompanied by a
strong evidence base. Between 1992 and 2004, Davidoff said, only 17
reasonably creditable accounts of rapid-response teams were pub-
lished. In fact, the strength of evidence regarding the effectiveness
of rapid-response teams remains only moderate. The question that
must be raised, Davidoff commented, is this: Why has the evidence
taken so long to develop?
There is very little data available to guide the development of
quality improvement research, of health sciences research, and of
medicine in general, stated Harold Pincus of Columbia University
and New York-Presbyterian Hospital. The lack of data, coupled with
the insufficient development of the basic science of quality improve-
ment research, Jeremy Grimshaw described, has led to a situation
27

where quality improvement has not been guided by evidence-based

learning. Quality improvement has been based largely on experien-
tial learning, but this knowledge has yet to be adequately captured
in the literature. The spread of ideas from experiential learning has
thus been poor.
This session of the workshop was devoted to discussing the
barriers to quality improvement and, in particular, to developing
an evidence base for use in quality improvement and in quality
improvement research.
Barrier of focus
Many workshop speakers emphasized the need to concentrate
on the particular purpose of quality improvement projects and
research. Davidoff noted that quality improvement efforts can have
many divergent purposes. Some believe the purpose is improv-
ing performance, a process that occurs mainly through experien-
tial learning and which differs significantly from scientific research,
whose purpose, Davidoff noted, is to discover generalizable truths
through hypothesis testing.
The emphasis on experiential learning that has evolved may lead
to the conclusion that many of those doing quality improvements
are uninterested in studying and writing about their experiences,
Davidoff said. For them, discovering the generalizable truths about
the efficacy and effectiveness of quality improvement interventions
may be largely a secondary consideration.
The role of Context

Understanding specific contexts and what is generalizable across
settings is extremely valuable in the implementation of interven-
tions, Grimshaw said. He also stated that work in the field attempts
to generate evidence while considering context and its effect on
processes.
If local contexts are not considered, then the lessons learned
from interventions will not be generalizable and will fail to improve
the health care system, Batalden cautioned. For example, the prac-
tice of health care policy is local, while the policy of health care is
not, Batalden added. The local uptake of health care policies, thus,
must be considered when working to improve care. Active research
and knowledge development are needed, both locally and across
local settings.

BARRIERS TO QUALITY IMPROVEMENT RESEARCH 29
Where does it belong?

Contributing to the infrastructure problem is lack of an agreed
academic area for where quality improvement research should be
taught, researched, and developed, noted Pincus. While the field
could be part of schools or hospitals, it is unclear what school—i.e.,
public health, medicine, or nursing—or department would be most
appropriate to hold it. One answer might be that health care quality
improvement research should be considered an interdisciplinary
research field, serving as a bridge between multiple disciplines.
As an interdisciplinary field, Pincus suggested, it could potentially
become a discipline of its own, following the precedent set by other
fields, such as neuroscience.
But while many agree in theory with the concept of inter
disciplinary research, it is extremely difficult in practice, Pincus said.
Disincentives to interdisciplinary research outnumber incentives. In
particular, Pincus identified three disincentives to interdisciplinary
research: conceptual, procedural, and structural and financial. The
conceptual barriers include the lack of common understanding and
language across different disciplines. “When people with completely
different scientific backgrounds get together to solve a common
problem, they have to learn a different way of speaking, a different
language,” Pincus quoted Nobel Prize Winner Alan MacDiarmid.
Procedural barriers include disincentives in career development,
for example the time it takes to learn about all these various areas
of study. The inherent departmental nature of academic medicine
raises questions about whether a department will sponsor faculty
who receive grants technically falling into other departments. This,
along with issues concerning how indirect costs will be shared, are
examples of structural and financial barriers. All of these, Pincus
said, must be considered in identifying strategies for enhancing and
expanding quality improvement research.
Resource barriers
Limited data exist about the resources allocated to health care
quality improvement. According to the Coalition for Health Ser-
vices Research, an estimated $1.5 billion of federal funding was
spent on health services research in fiscal year 2006 (Coalition for
Health Services Research, 2006). In another study, about 1.5 percent
of 2002 biomedical research funding was in health services and
policy research, equating to less than 0.1 percent of total U.S. health
care expenditures (Moses et al., 2005), cited Pincus. Because these
statistics refer to funding for all of health services research, not just

quality improvement research, even less is actually spent on under-

standing what works to improve performance. In countries outside
the United States, Grimshaw said, the situation is no better, with
funding policies that are often insufficient and inconsistent.
A related issue, Pincus said, is stakeholders’ lack of motivation
to build the infrastructure that is needed to attract health services
researchers. The main sources of funding for health services research
include several federal agencies such as the Agency for Healthcare
Research and Quality (AHRQ), the U.S Department of Veterans
Affairs (VA), and the Centers for Medicare and Medicaid Services
(CMS), foundations, and to a lesser degree, the National Institutes
of Health (NIH). Very little funding comes from industry (i.e., hos-
pitals, insurance, and pharmaceuticals) and voluntary health orga-
nizations, Pincus said. Thus it is important to develop strategies for
diversifying funding sources and enhancing contributions from both
industry and the NIH.
During the discussion, Richard Kahn of the American Diabetes
Association (ADA) noted that as a voluntary health organization,
the ADA does not receive many applications for quality improve-
ment research grants. The applications that it does receive tend not
to be well put together and are not of highly sophisticated research
designs. Despite the general feeling that the organization awards
few grants for quality improvement research, Kahn said that more
funding would be provided if better applications were received.
Barriers to recruitment and training

Davidoff characterized the following “mismatch” between
training and practice: Most people doing medical quality improve-
ment projects have little or no research training, while most people
with research training are not doing quality improvement projects.
Furthermore, he observed, few people know how to study quality
effectively. Quality improvement research is unfamiliar to most prac-
titioners, mainly because quality improvement is, at its core, more a
social process of behavior and organization change than a biological
or physical process. This mismatch is a large barrier to improving
the state of quality improvement research, Davidoff said.
Theroles of behavior and organization change are extremely relevant to the under
standing of quality improvement and implementation science, but because of the
scope of the workshop, discussion of these issues was limited.

Recruitment
The science behind recruiting and training researchers to study
quality improvement is not well understood, Pincus said. There is
some evidence that such strategies as involving people in research
during medical school or before, undertaking full-time research fel-
lowships, protecting faculty time, and training people in research-
intensive departments can help produce successful researchers,
Pincus said. In particular, exposure to research experiences is critical
to the recruitment of future researchers.
Another issue is how potential quality improvement and patient
safety researchers should be recruited into an interdisciplinary field.
Pincus likened it to marketing and the strategy of market segmen-
tation. Different strategies are needed for involving and recruiting
different audiences—undergraduates as opposed to residents and
post-doctoral students or health professionals versus those in fields
outside of the health professions.
The problem of ownership presents yet another barrier to recruit-
ment. It is often difficult to attribute ideas and quality improvement
interventions to one specific person. Additionally, rewarding a single
person for an idea may not be appropriate because quality improve-
ment has to become part of the culture, and therefore belongs to
everyone, noted Jay Berkelhamer of Children’s Healthcare of Atlanta.
This is a problem, and reward systems are yet to be built and may
indeed reward multiple people, Davidoff agreed. Pincus also noted
the trend of moving toward a “team science” approach.
One further difficulty is building a critical mass of interested
people, Davidoff said. Although it is not clear whether a critical
mass has yet been reached in quality improvement research, there
are at least some examples of movement toward that goal. For
example, Davidoff said, the Institute for Healthcare Improvement’s
annual meetings gathers around 6,000 people, and both Batalden
and Davidoff commented on the number of residents they have seen
who are interested in this work. Quality improvement is now also on
the agendas of many medical specialty certifying boards, said David
Stevens of the Association of American Medical Colleges. Andrea
Kabcenell of the Institute for Healthcare Improvement commented
that getting involved in quality improvement needs to be made more
democratic and accessible. However, recruitment is confounded by
the problems of publication and lack of career opportunities.

Training
The types of training needed to be successful in quality improve-
ment have not been specified, but before evidence can be accu-
mulated on the issue, Pincus said, it will be necessary to develop
an infrastructure to train and develop people to go into the field
of quality improvement research. There are a few models for this
type of training, such as the VA Quality Scholars Program and the
Robert Wood Johnson Foundation’s Clinical Scholars Program.
Curricula should be developed, Pincus suggested, by focusing on
those skills currently believed to be important for quality improve-
ment researchers. Values, mentorship, research opportunities, and
flexibility must be part of the environment provided. People should
also be taught practical skills, such as tips on conducting successful
research and receiving grants. And, Pincus said, if quality improve-
ment is to be treated as an interdisciplinary field, special attention
should be given to the criteria for how promotion and tenure should
be executed.
Quality improvement research involves not only those research-
ers who will become principal investigators but also many other
professionals, such as clinician educators and administrators, whose
roles and development must also be considered. When developing
training strategies, professionals from other disciplines should be
included. Early recruitment and proper training are well-supported
strategies, but difficult to implement, Pincus said.
Levers for strengthening

quality improvement research
Pincus identified three levers to improve quality improvement
research. The first is humanistic—that is, the research will ultimately
result in better care for patients. The second lever is in the policy
arena. Policy levers, such as accreditation or payors providing
matching funds for quality improvement research, must be utilized.
Quality improvement research can also be leveraged by strategies
focused on individual researchers, for example, providing salary
support for protected research time or altering tenure and promo-
tion policies to respond to the special barriers of interdisciplinary
research.

Barriers to performing quality improvement

and quality improvement research
Ethics
Ethical oversight in quality improvement remains largely
ambiguous. For example, Davidoff, citing the work of continuing
education expert Philip Nowlen, said that what distinguishes pro-
fessionals from other people is “the obligation of professionals to
‘move unceasingly toward new levels of performance.’” From this
perspective, quality improvement can be seen as an intrinsic element
of clinical care. Others, however, believe quality improvement to
be a form of clinical research, which raises the question of whether
quality improvement research is human-subjects research. This is an
important question because human-subjects research requires ethics
review and institutional review board (IRB) approval.
The purpose of IRB approval is not to decide whether clini-
cal care is ethical, Davidoff said, but the prospect of undergoing
IRB approval, which can be extremely slow and inconsistent, has
deterred some people from studying quality improvement. At the
heart of this issue is determining whether a project falls under the
rubric of quality improvement, which would not be subject to an
ethics review, or whether it is research that would require ethics
review. Currently, the distinction between these types of projects is
not well delineated, Davidoff said. Constructs need to be developed
that can help sharpen the distinction.
One member of the audience brought up the issue of confiden-
tiality, asking how the Health Insurance Portability and Account-
ability Act (HIPAA) impedes researchers’ abilities to collect data.
Davidoff responded that HIPAA does not prevent quality improve-
ment research from being conducted, although there are many rules
that need to be followed, referencing the more complete discussion
of this issue in a report from the Hastings Center (The Hastings
Center, 2006).
These concerns argue, Davidoff said, that it would be best if the
health care system itself developed ethical guidelines for quality
improvement instead of allowing the task to be subsumed by the
administrative structure responsible for clinical research ethics.
Methodology
Methodological differences between the biological sciences
and the social sciences offer another barrier, Davidoff said. Quality
improvement research faces the same challenges—such as biases,

confounders, and difficulties with measurement—that clinical

research does, but there are also methodological problems specific
to quality improvement research. For example, clinical interventions
are often studies of efficacy that are conducted in highly controlled
environments with rigorous population sampling and close monitor-
ing of response rates. By contrast, quality improvement studies are
not performed under such tightly controlled conditions. This is by
design, Davidoff noted, but it makes it difficult to know if “proven
interventions” in quality improvement research are generalizable
and actually yield improved outcomes.
Another common challenge in methodology, which is com-
pounded in quality improvement studies, is containing the inter-
vention. As Davidoff explained, if individual participants or small
sections of single large institutions are the units of study, they need
to be prevented from sharing ideas with others in the organization
to the greatest extent possible, so as not to “contaminate” the trial.
This is extremely difficult to accomplish; an alternative, Davidoff
said, is to let the unit of study be entire organizations (cluster trials),
but that gives rise to another problem: that of heterogeneity. In this
approach, all types of care settings—large, medium, or small; teach-
ing or nonteaching; rural or urban—would be included in the trial.
This may increase generalizability, but it would also decrease the
internal validity of such a trial.
Such methodological problems have caused many studies of
quality improvement to be methodologically flawed, Davidoff
concluded.
Publication
Publication is seldom seen as an essential element of quality
improvement, Davidoff said, because quality improvement studies
are often dependent on local context and do not identify and share
generalizable truths. Furthermore, Grimshaw said, much of what
is published is poorly reported. This stems from a lack of writing
experience by those doing quality improvement work, Davidoff sug-
gested. When writing about complex systems and social processes,
the need for writing experience becomes even more pronounced.
Unfortunately, there is limited guidance as to how published articles
documenting quality improvement efforts should be structured. One
exception is an article written by Davidoff and Batalden that pro-
poses guidelines for how write-ups of quality improvement studies
should be structured in an effort to improve them in the eyes of
reviewers, editors, and readers (Davidoff and Batalden, 2005).

Journals should begin to be more active in and receptive to

quality improvement research, which would in turn help stimulate
interest in the field by future researchers. Journals need to begin to
rethink some of what is published, Davidoff said, offering the sec-
tion in Annals of Internal Medicine on improving patient care as an
example.
Collectively, these issues of role and structure result in a rela-
tively small amount of learning being published, Davidoff said,
and much of what is published is not generalizable and so fails to
provide a basis upon which to base future efforts.
Communication
The lack of a common vocabulary for quality improvement and
implementation research terms is a barrier to further progress. This
is compounded by the fact that frameworks for how this research
should be approached are not widely known. The result, Grimshaw
concluded, is that those doing research in these areas have difficulty
communicating with each other, which contributes to the problem of
studies not building on previous findings, as discussed previously.
The difficulties are augmented when the research is performed in an
interdisciplinary setting, Pincus added.
Barrier of sustainability
Scot Webster spoke of the important role that culture change
plays in improving quality. In particular, Webster noted Medtronic’s
culture of grass roots, bottom-up sustainability. Due to this corpo-
rate culture, individual employees and units are able to initiate
improvement projects on their own. If the employees did not believe
in quality improvement as part of their culture, sustained change
would not occur, Webster said.
Marita Titler observed that overcoming problems with employee
engagement requires addressing the false notion that interventions
and improving the quality of care do not affect employees. This means
that employees must understand why interventions are important.
Otherwise, interventions are at risk of being seen merely as addi-
tional short-term projects adding to workloads, and not as priorities.
People have to believe in the improvements, not just see them as
short-term solutions, Webster agreed, adding that culture change and
change management must be included as areas of focus.
Titler also emphasized the barriers caused by problems at the
system level. System-level issues that can potentially detract from

advances in quality improvement include accounting for change,

managing competing demands, and understanding that change is
complex. Purely mechanistic approaches for change in complex
systems are often inadequate; instead, more complex, adaptive
approaches may be necessary. These barriers must be addressed in
order to induce change, Titler concluded.

Opportunities
T
he many barriers described in the previous section can be
translated into areas of opportunity for quality improvement
research. As Scott Young of Kaiser’s Care Management Insti-
tute noted, future quality improvement efforts will need to take into
account improvements in care for those with multiple chronic condi-
tions, transitions in care, new technologies, robust evidence-based
medicine, and innovative care environments.
Short term
There are a number of concrete actions that can be taken to
make quality improvement and quality improvement studies better
in the short term. One area of focus, for example, is the develop-
ment of an evidence base. And as Frank Davidoff of the Institute
for Healthcare Improvement said, managing the heterogeneity of
research will require that the goals of quality improvement projects
be more focused. Heterogeneity cannot be ignored, but there are
ways to control it within the constraints of the real world. Adapting
a lesson from randomized trial advocate Tom Chalmers, Davidoff
proposed quality improvement interventions be assessed using the
most rigorous methods possible immediately after the intervention
has been introduced. In this way, the strength of study results may
increase due to the ability to randomize.
37

Additionally, randomized controlled trials should build on pre-

vious work that characterized context and intervention develop-
ment, Batalden said. The gap between technical knowledge limits
and the needs of the particular applications of knowledge must be
bridged.
The focus of quality improvement projects should also be clari-
fied, suggested Davidoff. Projects often do not distinguish between
the goals of determining the efficacy of the clinical intervention itself
and of assessing the effectiveness of the care system in delivering
the intervention. Different types of evidence may be required for the
different goals, Davidoff noted. With a greater emphasis on clearly
defining the focus of studies, the field could produce useful, gener-
alizable knowledge that is actually needed, Grimshaw said.
Understanding the ethical issues is also an important, tangible
change that can help improve the state of quality improvement
research, Davidoff said.
Long term
One long-term solution to improving quality improvement
and quality improvement research would be to provide training
in research methods to people doing work in quality improve-
ment, Davidoff suggested. This could help fix the aforementioned
mismatch between training and practice. Davidoff also noted that
changes should be made to professional education. Examples of
such changes would include teaching collaborative skills, training
physicians in the manner that health care should be delivered, and
encouraging provider partnerships.
Academic and editorial cultures also need to change in order for
quality improvement and quality improvement research to develop.
These stakeholders need to recognize the social and intellectual
values of quality improvement work, Davidoff said. Other opportu-
nities include learning more about experiential learning and lever-
aging other research disciplines. Davidoff noted that other research
disciplines, such as social sciences and economics, could be useful in
moving forward. One important step in achieving this could be rec-
ognizing that quality improvement research, as an interdisciplinary
field, will require special attention, Pincus noted.
The importance of Strategies for change

Strategies should be developed to take action on these oppor-
tunities, Pincus said. He proposed six strategies to help advance

OPPORTUNITIES 39
uality improvement research: an infusion of new dollars, diversifi-

q
cation of funding sources, creation of institutional homes for quality
improvement research, recognition of the need for diverse strategies
for different audiences, exposure to future researchers at multiple
and early points in academic development, and enhancement of
data about what strategies are most effective in engaging investiga-
tors and developing substantial programs. While these strategies
are all important, a prioritization was not offered. Indeed, as Titler
noted, because of the complexity of the health care system and the
often conflicting needs of various stakeholders, it is extremely dif-
ficult to prioritize among strategies to develop quality improvement
and quality improvement research.

General Reactions
T
he following general reactions were offered by members of
the forum as well as members of the audience during the last
session of the workshop.
Leveraging other industries

Thomas Boat of the University of Cincinnati commented on
the need to learn from industry about how to market one’s efforts.
Reducing variation in health care practices will be a challenge, but
may potentially yield great results.
In response to Boat’s comments, Jay Berkelhamer of Children’s
Healthcare of Atlanta offered the example of mass-customization
automobile manufacturer Maserati, which produces between 5,000
and 10,000 cars a year (Cropley, 2007). No two Maseratis are alike,
as they are all made to specification, but the production of the cars is
standardized. Lessons may potentially be learned about how to stan-
dardize while allowing for individual differences, Berkelhamer said.
Judith Gueron of Manpower Demonstration Research Corpo-
ration also discussed lessons learned from other industries. For
example, she pointed out that 40 years ago educational reforms
were suggested, with the recommendation that changes take place
in schools of education, but significant advances have yet to be seen.
Applying that lesson to health care, Gueron suggested that medical
schools might not be the best place to focus reform efforts.
40

GENERAL REACTIONS 41
Laura Leviton of the Robert Wood Johnson Foundation dis-

cussed positive deviance and similar methods from other areas. In
education literature, for example, the term “school improvement
methods” is used. Some of these methods have been successfully
implemented in other fields and could be useful in health care,
Leviton said.
Context
The difficult portion of reducing variation, Boat said, will be
allowing medical practices to account for context while treating
individual patients and understanding the various confounders of
genomic background, environmental exposures, and psychosocial
contexts.
Ignatius Bau of the California Endowment agreed that the dis-
cussion of context is critical. There is the assumption that context
does not matter in quality improvement, he said, but this assump-
tion conflicts with the research agenda that attempts to produce
generalizable processes. What is known about changing culture,
such as changing provider behavior and changing processes of care
in team-based environments, should be a necessary component of
this conversation. Bau also discussed the need to understand both
resistance to change and why, according to performance measures,
best practices are not followed every time a patient is treated.
Areas for further discussion

There is a large co-occurrence of mental, addictive, and general
health conditions, Pincus said, citing the findings of the IOM report
Improving the Quality of Health Care for Mental and Substance-Use
Conditions (Institute of Medicine, 2006). But resource investment in
a quality improvement infrastructure and improvements in quality
that have been made in health care have not been paralleled in
mental health care. Partnerships, among other strategies, should
also be considered, he said. AHRQ’s Integrated Delivery System
Research Network could be applied in mental health settings, for
example. Quality improvement methods have not yet permeated the
areas of mental health and substance use, where they could poten-
tially have a great impact.
The Integrated Delivery System Research Network is a research model, connect-

ing top researchers and health care systems (Agency for Healthcare Research and
Quality, 2002).

David Introcaso of the Department of Health and Human Ser-

vices noted that the forum could benefit from listening to the per-
spectives of patient advocates and hearing about the disease burdens
of vulnerable populations. There is a great need to think about inno-
vation and how to guide innovation, Introcaso said, and a model
should be developed of how research is created, how knowledge is
created, and how knowledge is transferred.
The questions of how to define and develop a discipline are
complicated, but they are ones that should be considered by the
forum, said Alex Ommaya of the Department of Veterans Affairs.
Anthony Rosner of the Foundation for Chiropractic Education
and Research remarked on the future challenges chronic conditions
will bring, asking the forum to address health promotion and pre-
vention in future discussions. Boat also urged the forum to consider
the large variety of settings in which health care is delivered.
Clarifying communication
Marshall Chin and Jeremy Grimshaw recognized the many
complementarities discussed during the workshop. The workshop
also reflected the plethora of activities around quality improvement
and quality improvement research throughout the country and the
world. In order to build on all these efforts, there is a need to become
more specific in writing about these efforts. Researchers also must be
cognizant of describing the level at which an intervention is acting,
Grimshaw said.
Berkelhamer commented that those institutions in the academic
community doing the research are not necessarily those doing well
in practice centers.
Titler described the current state of communication in the field.
For the most part, implementers and researchers work in silos and
do not communicate well with each other. This lack of communica-
tion perhaps signals the need for these groups to move away from
these distinctions and work together, Titler said.
The need for further knowledge

Kahn emphasized the role of leadership. The presentations made
in the morning sessions largely reflected much of what is already
known to work, noted Kahn. The questions that naturally arise
are: Why did it take so long to adopt? Why isn’t everybody doing
it? What are the environmental characteristics that allow people to
improve?

GENERAL REACTIONS 43
Davidoff discussed the need to understand dynamic knowledge,

or knowledge in action. This is the knowledge that people harness
in doing their jobs, but it is generally not articulated well. Gueron
responded that dynamic knowledge comes from a different concep-
tual base for innovation and leadership than static knowledge.

References
Agency for Healthcare Research and Quality. 2007. Integrated delivery system research
network: Fact sheet. Agency for Healthcare Research and Quality. http://www.ahrq.
gov/research/idsrn.htm (accessed March 15, 2007).
Baldrige National Quality Program. 2007. 2007 Health care criteria for performance excel-
lence. Gaithersburg, MD: National Institute of Standards and Technology.
Batalden, P. B., and F. Davidoff. 2007. What is “quality improvement” and how can it
transform healthcare? Quality and Safety in Health Care 16(1):2-3.
Coalition for Health Services Research. 2006. Federal funding for health services research.
Washington, DC: Coalition for Health Services Research.
Cropley, S. 2007. New Maserati GranTurismo unveiled. Autocar Magazine. http://www.
speedtv.com/articles/automotive/newmodels/35516/ (accessed March 13, 2007).
Davidoff, F., and P. Batalden. 2005. Toward stronger evidence on quality improve-
ment. Draft publication guidelines: The beginning of a consensus project. Quality
and Safety in Health Care 14(5):319-325.
Ferlie, E. B., and S. M. Shortell. 2001. Improving the quality of health care in the
United Kingdom and the United States: A framework for change. Milbank Quar-
terly 79(2):281-315.
Grol, R. 2001. Successes and failures in the implementation of evidence-based guide-
lines for clinical practice. Medical Care 39(8 Suppl 2):II46-II54.
Hastings Center. 2006. The ethics of using QI methods to improve health care quality and
safety. Washington, DC: Hastings Center.
IOM (Institute of Medicine). 2006. Improving the quality of health care for mental and
substance-use conditions: Quality chasm series, Board on Health Care Services. Wash-
ington, DC: The National Academies Press.
McGlynn, E. A., S. M. Asch, J. Adams, J. Keesey, J. Hicks, A. DeCristofaro, and E. A.
Kerr. 2003. The quality of health care delivered to adults in the United States.
New England Journal of Medicine 348(26):2635-2645.
Moses, H., III, E. R. Dorsey, D. H. M. Matheson, and S. O. Thier. 2005. Financial
anatomy of biomedical research. Journal of the American Medical Association
294(11):1333-1342.
44

Appendix A
Workshop Agenda
THE PATH TO QUALITY IMPROVEMENT:

APPROACHES AND BARRIERS
Sponsored by
The Institute of Medicine’s
Forum on the Science of Health Care Quality
Improvement and Implementation
The National Academies Keck Building
Washington, D.C.
8:15 a.m. Welcome
8:20 a.m.
Lessons in Quality Improvement from Outside
Health-Care Services
— Scot Webster, Medtronic
— Q&A
Moderator: Tom Boat
9:25 a.m. Break
45

9:40 a.m. Quality Improvement in Medical Systems

— Scott Young, Kaiser Care Management Institute
— Craig Miller, Baptist Hospital Inc.
— Marita Titler, University of Iowa City Health
Care System
— Q&A
Moderator: Jay Berkelhamer

12:15 p.m. Lunch
1:00 p.m.
Perspectives on Quality Improvement and Quality-
— Paul Batalden, Health Care Improvement
Leadership Development,
Dartmouth Medical School
— Jeremy Grimshaw, Ottawa Health Research
Institute
— Q&A
Moderator: Brian Mittman
3:00 p.m. Break
3:15 p.m.
Barriers to Quality Improvement and Quality-
— Harold Pincus, Columbia University and
New York Presbyterian Hospital
— Frank Davidoff, Institute for Healthcare
Improvement
— Q&A
Moderator: Jerome Grossman
5:15 p.m. General Reactions from the Forum

Moderator: Tom Boat
6:00 p.m. Adjourn

Appendix B
Workshop Participants
Pamela Ballou-Nelson
Adventist Midwest Management Services
Mara Benner
Gentiva Health Services
Emily Devoto
Louis Diamond
Thomson Healthcare
Antoniya Dimova
Varna University of Medicine, Bulgaria
Molla Donaldson
George Washington University School of Medicine & Health
Services
Denise Dougherty
Agency for Healthcare Research and Quality
Suzanne Felt-Lisk
Mathematica Policy Research
47

Nagla Fetouh
Christine Mirzayan Fellowships
The National Academies
Veronica M. Friel
Jenissa Haidari
American Academy of Otolaryngology
Michael Halpern
American Cancer Society
Michael Harrison
Alex Hathaway
GlaxoSmithKline
David Introcaso
Office of the Assistant Secretary for Planning and Evaluation,
Department of Health and Human Services
Nkemdiri Iruka
National Committee for Quality Assurance
Sandra Isaacson
Rima Jolivet
American College of Nurse-Midwives
Stacie Jones
American Academy of Otolaryngology
Janet Karnoski
VA Center of Excellence
Joe Kimura
Harvard Vanguard Medical Associates
Barry Kramer
National Institutes of Health

APPENDIX B 49
Linda McKibben
Robert McNellis
American Academy of Physician Assistants
Robert Manduca
Swarthmore College
Karen Pennar
Hudson Health
Chesley Richards
Centers for Disease Control and Prevention
Anthony Rosner
Foundation for Chiropractic Education and Research
David Stevens
Association of American Medical Colleges
Robin Stombler
Auburn Health Strategies, LLC
Patricia Trifunov
GlaxoSmithKline
Greg Volkar
America’s Health Insurance Plans
Thomas Williams
TRICARE Management Activity
Laura Winner
Johns Hopkins Medicine
Steven Woolf
Virginia Commonwealth University
Junya Zhu
Brandeis University


11884

Uploaded by

Copyright:

Available Formats

11884

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

11884

Uploaded by

Copyright:

Available Formats

This PDF is available at http://nap.nationalacademies.

Advancing Quality Improvement Research:

FIND RELATED TITLES SUGGESTED CITATION

Samantha Chao, Rapporteur

Forum on the Science of Health Care Quality

Board on Health Care Services

THE NATIONAL ACADEMIES PRESS

Copyright National Academy of Sciences. All rights reserved.

International Standard Book Number-13 978-0-309-10623-8

Copyright 2007 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America.

Institute of Medicine (IOM) 2007. Advancing quality improvement research: ­Challenges

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

The National Academy of Sciences is a private, nonprofit, self-perpetuating

The Institute of Medicine was established in 1970 by the National Academy of

The National Research Council was organized by the National Academy of

Copyright National Academy of Sciences. All rights reserved.

FORUM ON THE SCIENCE OF HEALTH CARE QUALITY

THOMAS F. BOAT (Co-Chair), Director, Children’s Hospital Research

*Denotes Ex-Officio Members

Copyright National Academy of Sciences. All rights reserved.

BRIAN S. MITTMAN, Co-Editor in Chief, Implementation Science and

IOM Forum Staff

Copyright National Academy of Sciences. All rights reserved.

This report has been reviewed in draft form by individuals

ANDREA KABCENELL, Institute for Healthcare Improvement,

Copyright National Academy of Sciences. All rights reserved.

Although the reviewers listed above have provided many con-

Copyright National Academy of Sciences. All rights reserved.

1 LESSONS IN QUALITY IMPROVEMENT 11

2 APPROACHES TO QUALITY IMPROVEMENT RESEARCH 21

3 BARRIERS TO QUALITY IMPROVEMENT AND

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

 ADVANCING QUALITY IMPROVEMENT RESEARCH

Lessons in Quality Improvement

Non-Health Care Service Sector

Copyright National Academy of Sciences. All rights reserved.

Integrated Health Care Delivery System

Copyright National Academy of Sciences. All rights reserved.

 ADVANCING QUALITY IMPROVEMENT RESEARCH

itself. To operationalize the changes required to achieve their vision,

APPROACHES TO QUALITY IMPROVEMENT research

Copyright National Academy of Sciences. All rights reserved.

Generalizable scientific evidence and particular contexts link

Implementation research is the term used in Europe to refer to quality improve-

Copyright National Academy of Sciences. All rights reserved.

 ADVANCING QUALITY IMPROVEMENT RESEARCH

Barriers to Quality Improvement and Quality

Copyright National Academy of Sciences. All rights reserved.

considered human subjects research, which would require ethics

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

10 ADVANCING QUALITY IMPROVEMENT RESEARCH

and issues could be discussed. The forum members plan to integrate

Copyright National Academy of Sciences. All rights reserved.

Lessons in Quality Improvement∗

Institute of Medicine (IOM) 2007. Advancing quality improvement research: Challenges

ADVANCING QUALITY IMPROVEMENT RESEARCH

ADVANCING QUALITY IMPROVEMENT RESEARCH

ADVANCING QUALITY IMPROVEMENT RESEARCH