Garg 2005

REVIEW
Effects of Computerized Clinical Decision

Support Systems on Practitioner Performance
and Patient Outcomes
A Systematic Review
Amit X. Garg, MD Context Developers of health care software have attributed improvements in pa-
Neill K. J. Adhikari, MD tient care to these applications. As with any health care intervention, such claims re-
Heather McDonald, MSc quire confirmation in clinical trials.
M. Patricia Rosas-Arellano, MD, PhD Objectives To review controlled trials assessing the effects of computerized clinical
decision support systems (CDSSs) and to identify study characteristics predicting
P. J. Devereaux, MD benefit.
Joseph Beyene, PhD Data Sources We updated our earlier reviews by searching the MEDLINE, EMBASE,
Justina Sam, BHSc Cochrane Library, Inspec, and ISI databases and consulting reference lists through Sep-
tember 2004. Authors of 64 primary studies confirmed data or provided additional
R. Brian Haynes, MD, PhD information.
C
OMPUTERIZED CLINICAL DECI- Study Selection We included randomized and nonrandomized controlled trials that
sion support systems evaluated the effect of a CDSS compared with care provided without a CDSS on prac-
(CDSSs) are information sys- titioner performance or patient outcomes.
tems designed to improve Data Extraction Teams of 2 reviewers independently abstracted data on methods,
clinical decision making. Characteris- setting, CDSS and patient characteristics, and outcomes.
tics of individual patients are matched Data Synthesis One hundred studies met our inclusion criteria. The number
to a computerized knowledge base, and and methodologic quality of studies improved over time. The CDSS improved
software algorithms generate patient- practitioner performance in 62 (64%) of the 97 studies assessing this outcome,
specific recommendations. Practition- including 4 (40%) of 10 diagnostic systems, 16 (76%) of 21 reminder systems, 23
ers, health care staff, or patients can (62%) of 37 disease management systems, and 19 (66%) of 29 drug-dosing or
manually enter patient characteristics prescribing systems. Fifty-two trials assessed 1 or more patient outcomes, of which
into the computer system; alterna- 7 trials (13%) reported improvements. Improved practitioner performance was
associated with CDSSs that automatically prompted users compared with requiring
tively, electronic medical records can users to activate the system (success in 73% of trials vs 47%; P=.02) and studies in
be queried for retrieval of patient char- which the authors also developed the CDSS software compared with studies in
acteristics. Computer-generated rec- which the authors were not the developers (74% success vs 28%; respectively,
ommendations are delivered to the cli- P=.001).
nician through the electronic medical Conclusions Many CDSSs improve practitioner performance. To date, the effects
record, by pager, or through printouts on patient outcomes remain understudied and, when studied, inconsistent.
placed in a patient’s paper chart. Such JAMA. 2005;293:1223-1238 www.jama.com
systems have been developed for a
myriad of clinical issues, including di-
agnosis of chest pain, treatment of in-
fertility, and timely administration of Author Affiliations: Division of Nephrology (Drs Garg Sciences Centre and Interdepartmental Division of Criti-
immunizations. These systems pro- and Rosas-Arellano) and Department of Epidemiol- cal Care (Dr Adhikari), Population Health Sciences, Hos-
ogy and Biostatistics (Dr Garg), University of West- pital for Sick Children (Dr Beyene), and Faculty of Medi-
vide several modes of decision sup- ern Ontario, London; Departments of Clinical Epide- cine (Ms Sam), University of Toronto, Toronto, Ontario.
port, including alerts of critical val- miology and Biostatistics (Drs Garg, Adhikari, Corresponding Author: R. Brian Haynes, MD, PhD,
Devereaux, and Haynes and Ms McDonald) and Medi- Clinical Epidemiology and Biostatistics, Faculty of Health
cine (Drs Devereaux and Haynes), McMaster Univer- Sciences, McMaster University, 1200 Main St W, Room
See also pp 1197 and 1261. sity, Hamilton, Ontario; Department of Critical Care 2C10B, McMaster University, Hamilton, Ontario,
Medicine, Sunnybrook and Women’s College Health Canada L8N 3Z5 ([email protected]).
©2005 American Medical Association. All rights reserved. (Reprinted) JAMA, March 9, 2005—Vol 293, No. 10 1223
Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

DECISION SUPPORT, PRACTITIONER PERFORMANCE, AND PATIENT OUTCOMES
ues, reminders of overdue preventive Studies Eligible for Review ized controlled trial, and cohort studies
health tasks, advice for drug prescrib- We included English-language ran- (complete strategies available from the
ing, critiques of existing health care or- domized and nonrandomized trials with authors). Pairs of reviewers indepen-
ders, and suggestions for various ac- a contemporaneous control group that dently evaluated the eligibility of all stud-
tive care issues. compared patient care with a CDSS to ies identified in our search. Disagree-
routine care without a CDSS and evalu- ments were resolved by a third reviewer
As with any health care innovation, ated clinical performance (ie, a mea- or by consensus. Full-text articles were
CDSSs should be rigorously evaluated sure of process of care) or a patient out- retrieved if any reviewer considered a ci-
before widespread dissemination into come. We stipulated that the CDSS had tation potentially relevant. Supplemen-
clinical practice. Various stages in this to provide patient-specific advice that tary methods of finding studies in-
assessment process have been previ- was reviewed by a health care practi- cluded a review of article reference lists,
ously described. Iterative qualitative and tioner before any clinical action. Stud- articles citing included studies as listed
quantitative assessment begin early in ies were excluded if the system (1) was in the Science Citation Index, PubMed
the software development cycle.1,2 When used solely by medical students, (2) related articles feature, informatics con-
preliminary testing suggests that a CDSS only provided summaries of patient in- ference proceedings, information pro-
improves clinical care or patient out- formation, (3) provided feedback on vided by primary study authors, and
comes, confirmatory controlled trials are groups of patients without individual other recent reviews.6-11 Where data from
warranted. We previously reviewed con- assessment, (4) only provided com- a trial were distributed in more than 1
trolled trials of computer-aided quality puter-aided instruction, or (5) was used publication, we cited the principal pub-
assurance3 and CDSSs published up to for image analysis. Studies assessing lication.
19924 and 1998.5 This field is rapidly CDSS diagnostic performance against
evolving because of technological ad- a defined gold standard were not in- Data Abstraction
vances, increasing access to computer cluded in this review unless clinical use Pairs of reviewers independently ab-
systems in clinical practice, and grow- of the diagnostic CDSS was also com- stracted the following data from all
ing concern about the process and qual- pared with routine care. Based on these studies meeting eligibility criteria: study
ity of medical care. We therefore up- criteria, we reevaluated all studies from setting, study methods, CDSS charac-
dated previous reviews to provide a our previous reviews for inclusion. teristics, patient characteristics, and out-
cumulative summary of controlled trials comes. Disagreements were resolved by
evaluating the effectiveness of CDSSs on Finding Relevant Studies a third reviewer or by consensus. We
practitioner performance and patient We have previously described our meth- attempted to contact primary authors
outcomes. ods for finding relevant studies until of all included studies to confirm data
March 1998.5 For this update, we ex- and provide missing data.
METHODS amined citations from MEDLINE, All studies were scored for method-
Research Questions EMBASE, Evidence-Based Reviews da- ological quality on a 10-point scale con-
The primary questions of this review tabases (Cochrane Database of System- sisting of 5 potential sources of bias,
were (1) Do CDSSs improve practi- atic Reviews, ACP Journal Club, Data- which we have described elsewhere.5 In
tioner performance or patient out- base of Abstracts of Reviews of Effects, brief, we considered the method of al-
comes? and (2) Which CDSS and study- and Cochrane Central Register of Con- location to study groups (random, 2, vs
level factors are associated with effective trolled Trials), and Inspec biblio- quasi-random, 1, vs selected concur-
CDSSs? A priori, we hypothesized that graphic databases from 1998 through rent controls, 0), the unit of the alloca-
studies reporting better outcomes September 2004. All citations were tion (a cluster such as a practice, 2, vs
would assess CDSSs that automati- downloaded into Reference Manager, physician, 1, vs patient, 0), the pres-
cally prompted users (vs requiring the version 10.0 (Thomson ISI Research- ence of baseline differences between the
user to actively initiate the system), Soft, Philadelphia, Pa). An experienced groups that were potentially linked to
were built into an electronic medical librarian developed the search strate- study outcomes (of particular impor-
record or computer order entry sys- gies using sensitive terms for identify- tance for observational studies; no base-
tem (vs a stand-alone system), pro- ing clinical studies of CDSSs. We pilot- line differences present or appropriate
vided reminders (vs information on dis- tested search strategies and modified statistical adjustments made for differ-
ease management, drug dosing, or them to ensure that they identified ences, 2, vs baseline differences present
diagnosis), were tested using less rig- known eligible articles. The final strat- and no statistical adjustments made, 1,
orous study methods, were studied by egies used the terms computer-assisted vs baseline characteristics not reported,
their software developers (vs by evalu- decision making, computer-assisted diag- 0), the objectivity of the outcome (ob-
ators not involved in the CDSS de- nosis, computer-assisted therapy, deci- jective outcomes or subjective out-
sign), described pilot testing, and de- sion support systems, reminder systems, comes with blinded assessment, 2, vs
scribed user training. hospital information systems, random- subjective outcomes with no blinding but
1224 JAMA, March 9, 2005—Vol 293, No. 10 (Reprinted) ©2005 American Medical Association. All rights reserved.

clearly defined assessment criteria, 1, vs we cited the measures of association and Of all trials, 88% were randomized. Of
subjective outcomes with no blinding P values reported in the studies. the randomized trials, 49% were clus-
and poorly defined, 0), and the com- ter randomized and 40% used a cluster
pleteness of follow-up for the appropri- RESULTS as the unit of analysis or adjusted for
ate unit of analysis (⬎90%, 2, vs 80 to Finding and Selecting Studies clustering in the analysis. Twenty-four
90%, 1, vs ⬍80% or not described, 0). randomized trials and 1 cohort study re-
From 3997 screened citations, we re-
The unit of allocation was included be- ported a power calculation for a pre-
trieved 226 full-text articles, and 100
cause of the possibility of group con- specifed difference between groups on
trials met our criteria for review. The
tamination in trials in which interven- a specific outcome. Fifteen of these trials
chance-corrected agreement between 2
tions were applied to clinicians even (60%) calculated sample size based on
independent reviewers for article in-
though individual patients were allo- a practitioner performance outcome, 9
clusion was good (␬=0.81; 95% con-
cated to the intervention and control (36%) based on a patient outcome, and
fidence interval [CI], 0.73-0.88).
groups.12 Contamination bias would lead 1 (4%) based on the cost of prescribed
to underestimating the effect of a CDSS. Description of Studies medications. Only 2 studies examined
The studies substantially differed in patient outcomes without measuring
The 100 trials examined more than
the type and number of outcomes as- practitioner performance. Of the 88 ran-
3826 practitioners or practices (me-
sessed. In addition, the majority of stud- domized trials, 52% described an ap-
dian, 42; range, 2-300 [when re-
ies did not define a single outcome for propriate method of generating ran-
ported]) caring for more than 92 895
statistical testing. We aimed to effi- dom numbers and 28% reported
patients (median, 488; range, 19-
ciently summarize the benefits of CDSSs allocation concealment. On the 10-
12 989 [when reported]) from 1973 to
and to identify CDSS and study charac- point methods scale, the mean score was
2004.14-113 The number of eligible trials
teristics that predicted success. For a 7 (SD, 1.7) and the range was 2 to 10.
increased with time: 1 in 1970-1974, 4
given study we abstracted all reported
in 1975-1979, 10 in 1980-1984, 13 in Description of Users and CDSSs
practitioner performance and patient
1985-1989, 20 in 1990-1994, 26 in
health outcomes. Situations where the The 100 trials had the following char-
1995-1999, and 26 in 2000–Septem-
CDSS worsened outcomes were rare. acteristics: 92% of trials enrolled physi-
ber 2004. Of these 100 trials, most were
Thus, for each study we defined the ef- cians as primary users, 48% enrolled
conducted in the United States (69%),
fects of CDSSs in terms of success, de- training health care practitioners (in-
followed by the United Kingdom (14%),
fined as an improvement in at least 50% terns and residents) as users, 34% de-
Canada (5%), Australia (4%), Italy
of outcomes measured, each at a 2-sided scribed pilot testing with users prior to
(2%), and Austria, France, Germany, Is-
significance level less than .05. implementation, 42% described user in-
rael, Norway, and Switzerland (1%
structional training at the time of imple-
Statistical Analysis each). Sixty-nine percent of trials de-
mentation, 76% took place in academic
scribed funding from the public sec-
Reviewer agreement on study eligibil- centers, and 33% were inpatient-based.
tor and 16% from the private sector. De-
ity was quantified using the Cohen ␬.13 In 47% of studies, the CDSS was part of
velopers of CDSS software were also
Study and CDSS characteristics predict- an electronic medical record or com-
study authors in 72% of trials. Ninety-
ing success were analyzed and inter- puter order entry system. Most of these
seven trials described the effect of CDSS
preted with the study as the unit of were early generation systems lacking the
on at least 1 measure of health care
analysis. Data were summarized using full functionality of current systems. In
practitioner performance. Fifty-two
descriptive summary measures, includ- 15% of studies, the CDSS had a graphi-
trials assessed at least 1 patient out-
ing proportions for categorical vari- cal user interface. Feedback from the
come. We successfully contacted au-
ables and mean (standard deviation) for CDSS occurred at the time of patient care
thors of 91 trials, and authors of 64 trials
continuous variables. Univariable and in 88% of studies; in 60% the user was
provided additional information or con-
multivariable logistic regression mod- automatically prompted to use the sys-
firmed the accuracy of abstracted data.*
els, adjusted for study methodological tem (vs the user actively initiating the sys-
quality, were used to investigate asso- Methodological Quality tem), and in 91% the CDSS suggested
ciations between the outcomes of inter- Assessment new orders (vs critiquing existing or-
est and study-specific covariates de- ders). Expert physician opinion or clini-
Trial methodological rigor increased
fined in our a priori hypotheses. All cal practice guidelines usually formed the
with time—36% of trials before the year
analyses were carried out using the SAS knowledge base for the CDSS.
2000 were cluster randomized, com-
statistical package, version 8.2 (SAS In- The process of data entry into the
pared with 67% after this time (P=.01).
stitute Inc, Cary, NC). We interpreted CDSS was clear in 80% of trials, some of
Pⱕ.05 as indicating statistical signifi- which used more than 1 method. Exist-
*References 15-18, 20, 21, 24-33, 35-40, 42, 43, 46,
cance; all P values are 2-sided. When re- 47, 49, 51, 52, 56, 60-64, 67, 68, 71, 73-75, 80, 81, ing personnel most often entered data
porting results from individual studies, 83-98, 101, 106, 113-115. (attending or training physician, 38%;

other health care staff [eg, nurses, clerks], tient outcomes failed to demonstrate an quency of redundant testing and un-
29%), although many trials used staff improvement in the primary analy- necessary hospital admissions and hos-
paid by research funds (21%) or auto- sis.34 Post hoc subgroup analyses, how- pital length of stay, with 3 of 4 trials
mated data capture from an electronic ever, demonstrated a significant reduc- reporting improvements (Table 6).
medical record (30%). The method of de- tion in winter hospitalization and Five CDSSs (18%) examining patient
livering computer recommendations to emergency department visits in pa- outcomes described improvements. One
the clinician was clear in 81% of trials. tients eligible for pneumococcal or in- CDSS improved blood pressure control
Most CDSSs directly provided the rec- fluenza vaccination. One trial exam- (70% of patients had controlled blood
ommendation on a computer screen ined the effect of adding a cervical pressure with CDSS use vs 52% with rou-
viewed by the practitioner (41% of all cancer screening reminder to an exist- tine care; P⬍.05).54 A second CDSS
trials) or generated printed reports that ing mammography reminder sys- reduced urinary incontinence in nurs-
were placed in medical charts by health tem.30 This trial suggested no interac- ing home residents over a 10-week period
care staff (29%) or by staff paid by re- tion between the 2 reminders on (23% incontinent with CDSS vs 69% with
search funds (16%). Only 13% of trials screening efficacy. routine care; P⬍.01).66 A third CDSS
evaluated the impact of the CDSS on cli- improved scores of barotrauma (P⬍.001)
nician workflow, with more than half of Systems for Disease Management and organ dysfunction (P = .04) in
these CDSSs requiring more time and ef- There were 40 studies of CDSSs for ac- mechanically ventilated patients with
fort from the user compared with paper- tive health conditions. These CDSSs im- acute respiratory distress syndrome.70
based methods. proved practitioner performance in 23 One participating center in this trial pro-
(62%) of 37 studies evaluating this vided data demonstrating lower tidal vol-
Systems for Diagnosis outcome. Of the 27 trials measuring umes (Pⱕ.03) and a reduction in expo-
There were 10 trials evaluating diagnos- patient outcomes, 5 (18%) demon- sure to high plateau pressures in the
tic systems (TABLE 1). All studies mea- strated improvements. group receiving CDSS-guided mechani-
sured practitioner performance, and the For diabetes care, practitioner per- cal ventilation (P⬍.001).114 A fourth
CDSS was beneficial in 4 studies (40%). formance was usually judged by rates CDSS reduced patient-reported asthma
Two of the 4 successful CDSSs were di- of retinal, foot, urine protein, blood exacerbations (8% vs 17%; odds ratio,
agnostic systems for cardiac ischemia in pressure, and cholesterol examina- [OR], 0.43; 95% CI, 0.21-0.85), emer-
the emergency department, and these de- tions, with 5 (71%) of 7 trials report- gency nebulizer use (1% vs 5%; OR, 0.13;
creased the rate of unnecessary hospital ing improvements (TABLE 3). Simi- 95% CI, 0.01-0.91), and the need for
or coronary care admissions by 15% larly, in studies of cardiovascular additional consultations for asthma man-
(P⬍.05).18,20 The third increased mood prevention, performance was judged by agement (22% vs 34%; OR, 0.59; 95% CI,
disorder screening in a posttraumatic blood pressure and cholesterol assess- 0.37-0.95) over 6 months.73 A fifth CDSS
stress disorder clinic by 25% (P=.008).15 ment, identification of smoking, and use reduced hospital length of stay (P=.02)
The fourth improved the time to diag- of cardioprotective medications, with for patients with a variety of general medi-
nosis of acute bowel obstruction (1 hour 5 (38%) of 13 trials reporting improve- cal diagnoses.83
when computer was used vs 16 hours ments (TABLE 4). One CDSS provided In post hoc secondary or subgroup
when diagnosis was made with con- electrocardiogram recommendations to analyses, some trials described statis-
trast radiography; P⬍.001).23 Of the 5 improve thrombolytic prescribing in tically significant improvements in
trials assessing patient outcomes, none emergency departments.61 Other CDSSs thrombolytic prescribing with the
reported an improvement. varied in purpose, providing recom- CDSS,61 as well as patient outcomes of
mendations for urinary incontinence, disease-specific emergency depart-
Reminder Systems for Prevention human immunodeficiency virus infec- ment visits, 6 5 hospital length of
There were 21 trials evaluating re- tion management, functional assess- stay,45,54,116,117 body weight,54,116,117 dia-
minder systems for prevention ment, and acute respiratory distress stolic blood pressure,59,115,118 serum lip-
(TABLE 2). All trials measured practi- syndrome, with 6 of 9 reporting im- ids,51,58 and a reduced estimated risk of
tioner performance, and the CDSS was provements (TABLE 5). Clinical deci- future cardiovascular events.58
beneficial in 16 studies (76%). Perfor- sion support system corollary orders
mance outcomes were usually rates of were used to monitor the effects of other Systems for Drug Dosing
screening, counseling, vaccination, test- prescribed treatments, such as the need and Drug Prescribing
ing, medication use, or the identifica- for renal biochemistry measurements There were 29 trials of drug dosing and
tion of at-risk behaviors. Successful use in patients receiving amphotericin B,79 prescribing (TABLE 7 and TABLE 8).
of CDSSs was typically demonstrated with all 4 trials reporting improve- Single-drug dosing improved practi-
in ambulatory care, although 1 system ments (TABLE 6). Trials testing CDSS tioner performance in 15 (62%) of 24
was successful in hospitalized pa- performance to reduce unnecessary studies, and 2 of the 18 systems assess-
tients.44 The single trial measuring pa- health care utilization measured the fre- ing patient outcomes reported an im-

provement (Table 7 and Table 8). An- The 24 single-drug dosing systems ated the serum drug level in medica-
other 5 systems used computer order ranged from a simple calculator for par- tions with a high risk of toxicity. In a
entry for multidrug prescribing enteral nutrition to more complex al- study of heparin dosing for patients re-
(Table 8). Four of these systems im- gorithms that considered the pharma- ceiving thrombolysis for myocardial in-
proved practitioner performance, but cokinetics of warfarin, aminoglycosides, farction, the proportion of individuals
none improved patient outcomes. or theophylline. Most studies evalu- with an adverse thrombotic or cardiac
Table 1. Trials of Computer-Assisted Diagnosis*

Improvement Improvement
Methods No. of Patient in Practitioner in Patient
Source Score Sites Indication Performance Outcomes Outcomes Performance† Outcomes†
Diagnostic systems for
mental health
Lewis et al,14 5 1 Common mental Rate of patient referral to Symptom score No No
1996 disorders for mental health, after 6 wk
outpatients psychotropic
medications,
psychological
consultations
Cannon et al,15 7 1 Mental health Screening for mood ... Yes ...
2000 diagnosis for disorder, complete
outpatients documentation for
major depressive
disorder
Schriger et al,16 6 1 Psychiatric interview Psychiatric diagnosis and ... No ...
2001 and diagnosis in referrals,
emergency documentation of
department complete psychiatric
history
Rollman et al, 9 17 Major depression Use of antidepressants, Depression score No No
200217 diagnosis for discussion about after 6 mo
outpatients depression with
patients
acute cardiac ischemia
Pozen et al,18 3 1 Acute cardiac Inappropriate coronary ... Yes ...
1984 ischemia in care unit admission for
emergency patients without
department ischemic heart disease
Wyatt,19 1989 6 1 Chest pain in Time to transfer to ... No ...
emergency coronary care unit,
department time to see physician,
total time in
emergency
department
Selker et al,20 4 10 Electrocardiogram Inappropriate hospital or Mortality in first 30 d, Yes No
1998 interpretation for coronary care unit in-hospital
cardiac ischemia in admission for patients complications,
emergency without acute ischemic need for
department heart disease rehospitalization
other conditions
Wexler et al,21 2 1 Admitted pediatric No. of consultations ... No ...
1975 inpatients without requested, time to
clear diagnosis diagnosis, orders for
unnecessary
laboratory tests
Wellwood 6 1 Acute abdominal pain Appropriate diagnosis for Unnecessary No No
et al,22 1992 in emergency appendicitis, surgery with
department unnecessary hospital negative findings
admissions
Bogusevicius 7 1 Acute small bowel Time to diagnosis, correct Bowel necrosis, Yes No
et al,23 2002 obstruction in diagnosis morbidity,
surgical inpatients mortality,
hospital length
of stay
*Ellipses indicate outcome was not assessed. Methods score based on 10-point scale (see the “Methods” section).
†Improvement was defined as a statistically significant positive effect on at least 50% of outcomes measured. Most studies had inadequate statistical power to detect a clinically important
improvement in patient outcomes.

Table 2. Trials of Computer-Assisted Reminders for Cancer Screening, Vaccination, and Other Types of Preventive Care
Improvement
Methods No. of in Practitioner
Source Score Sites Indication Performance*
Reminders primarily for
cancer screening
Turner et al,24 8 1 Outpatient screening (stool occult blood, digital rectal examination, Papanicolaou test, breast No
1989 examination, mammography)
McPhee et al,25 9 1 Outpatient screening (stool occult blood, digital rectal examination, sigmoidoscopy, pelvic Yes
1989 examination, Papanicolaou test, breast examination, mammography)
McPhee et al,26 10 Multiple Outpatient screening (digital rectal examination, stool occult blood, sigmoidoscopy, pelvic Yes
1991 examination, Papanicolaou test, breast examination, mammography) and preventive
counseling (smoking assessment and counseling, dietary assessment and counseling)
Burack et al,27 7.5 5 Mammography for outpatients Yes
1994
Burack et al,28 7.5 2 Mammography for outpatients Yes
1996
Burack and 7.5 4 Mammography for outpatients Yes
Gimotty,29
1997
Burack et al,30 7.5 3 Papanicolaou test for outpatients; in addition to physician prompt, a patient reminder (personal Yes
2003 letter) was generated by the system and was part of the intervention
Reminders primarily for
vaccination
Chambers 9 1 Influenza vaccination for outpatients Yes
et al,31 1991
Flanagan et al,32 7.5 1 Tetanus, hepatitis, pneumococcal, measles, and influenza vaccination for outpatients No
1999
Tang et al,33 5 1 Influenza vaccination for outpatients Yes
1999
Reminders for
preventive care†
McDonald 8 1 Cancer screening (stool occult blood, mammogram), counseling (weight reduction), Yes
et al, 34 immunization (influenza, pneumococcal) in addition to ⬎1000 physician behavior rules for
1984 outpatients
Tierney et al,35 5 1 Cancer screening (stool occult blood, Papanicolaou test, mammogram), pneumococcal Yes
1986 vaccination, tuberculosis skin test, use of antidepressants, metronidazole for trichomonas,
cardiovascular medications (␤-blockers, long-acting nitrates, aspirin), prophylactic
antacids, and calcium supplements for outpatients
Ornstein et al,36 9 1 Cancer screening (stool occult blood, mammography, Papanicolaou test), cholesterol No
1991 measurement, and tetanus vaccination for outpatients
Rosser et al,37 7.5 1 Cancer screening (Papanicolaou test), blood pressure measurement, assessment of smoking Yes
1991 status, and vaccination (influenza, tetanus toxoid) in outpatients
Tape and 7.5 1 Cancer screening (stool occult blood, Papanicolaou test, mammogram, Yes
Campbell,38 proctosigmoidoscopy), thyroid function screening, vaccination (tetanus, pneumococcal,
1993 influenza) for outpatients
Turner et al,39 6 44 Cancer screening (stool occult blood, Papanicolaou test, breast examination, mammogram) No
1994 and influenza vaccination for outpatients
Frame et al,40 6 5 Cancer screening (stool occult blood, Papanicolaou test, breast examination, mammogram), Yes
1994 cardiovascular disease preventive screening (blood pressure, cholesterol, body weight),
identification of at-risk behavior (smoking), patient education (self-examination, recognition
of postmenopausal bleeding), and vaccination (tetanus) in outpatients
Overhage 10 1 Cancer screening (Papanicolaou test, mammogram), cardiovascular disease preventive No
et al,41 screening and medications (cholesterol, ␤-blockers, aspirin), diabetes care reminders
1996 (retinal examination, urinalysis), vaccination (pneumococcal, rubella, hepatitis B), and an
additional 11 reminders for hospital inpatients
Bonevski et al,42 7 Multiple Cancer screening (Papanicolaou test), cardiovascular disease preventive screening (blood Yes
1999 pressure, cholesterol), and identification of 3 risk behaviors (smoking, excessive alcohol
use, benzodiazepine use) in outpatients
Demakis et al,43 10 12 Screening (urinalysis, retinal examination, foot examination), monitoring (glycated hemoglobin), Yes
2000 and counseling (dietary advice) to prevent diabetic complications in outpatients; reminders
for other conditions including vaccination, smoking cessation, appropriate ␤-blocker and
nonsteroidal anti-inflammatory use; cholesterol screening
Dexter et al,44 10 1 Vaccination (pneumococcal, influenza), prophylactic heparin and aspirin use for hospital Yes
2001 inpatients
*Improvement was defined as a statistically significant positive effect on at least 50% of outcomes measured. Practitioner performance outcomes were the rate of screening, medication
use, and/or identification of at-risk behaviors. Improvement in patient outcomes was not assessed except in McDonald et al,34 in which there was no improvement in body weight, blood
pressure, hospitalizations, or emergency department visits.
†These systems were designed for more than 1 type of condition, including cancer screening, vaccination, and cardiovascular disease prevention.

event was significantly lowered with the Studies in which users were automati- No other predefined study-level covar-
CDSS (0/25 with the CDSS vs 6/26 in cally prompted to use the system de- iate was associated with CDSS success.
usual care; P = .02).97 One warfarin- scribed better performance compared In a post hoc analysis of the 85 studies
dosing CDSS reduced hospital length with studies in which users had to ac- that measured practitioner perfor-
of stay from 20 to 13 days (P = .01).87 tively initiate the system (success in 44/60 mance and enrolled physicians, we did
Two systems reduced hospital length studies [73%] vs 17/36 studies [47%]; not find an association (P = .40) be-
of stay in patients receiving theophyl- P=.02; unadjusted OR, 2.8; 95% CI, 1.2- tween performance and physician expe-
line (from 8.7 to 6.3 days; P=.03)98 and 6.6; OR adjusted for methodological rience (trainee vs attending physician).
aminoglycosides (20.3 to 16.0 days; quality, 3.0; 95% CI, 1.2-7.1). Simi-
P=.03),104 although the majority of pa- larly, studies in which the authors also COMMENT
tient outcomes measured were not im- created the CDSS reported better per- We identified 100 randomized and non-
proved in these trials. formance compared with those in which randomized trials testing a wide vari-
the trialists were independent of the ety of CDSSs, with the number of trials
Study Factors Associated CDSS development process (success in and their methodological quality in-
With CDSS Success 51/69 studies [74%] vs 5/18 studies creasing over time. Of the 97 con-
Given sparse data for patient outcomes, [28%]; P=.001; unadjusted OR, 6.7; 95% trolled trials assessing practitioner per-
we only assessed study-level predictors CI, 1.7-25.3; OR adjusted for method- formance, the majority (64%) improved
of improved practitioner performance. ological quality, 6.6; 95% CI, 1.7-26.7). diagnosis, preventive care, disease
Table 3. Trials of Computer-Assisted Diabetes Management*

Methods No. of in Practitioner in Patient
Source Score Sites Indication Patient Outcomes Performance†‡ Outcomes†
Thomas et al,45 1983 4 1 Computer-generated reminders Change in blood pressure, Yes No
for outpatients obesity, glucose,
hospitalization, emergency
department visits
Mazzuca et al,46 1990 8 1 Counseling (exercise and dietary ... No ...
advice), glucose control
monitoring, medication use,
education for outpatients
Nilasena and 8 2 Screening (foot examination, ... No ...
Lincoln,47 1995 retinal examination, renal
tests), cardiovascular disease
prevention, neurological
assessment, and glycemic
control in outpatients
Lobach and 9 1 Screening (foot examination, ... Yes ...
Hammond,48 complete physical, retinal
1997 examination, cholesterol,
urine protein), vaccination
(influenza and
pneumococcal), as well as
glycated hemoglobin
monitoring for outpatients
Montori et al,49 2002 5 2 Screening (microalbuminuria, Glycated hemoglobin, total Yes No
retinal examination, cholesterol, blood pressure,
cholesterol, foot examination) calculated 10-y
and counseling (exercise and Framingham risk score
dietary advice, smoking
cessation) to prevent
complications in outpatients;
system also identified drug
contraindications
Filippi et al,50 2003 9 Multiple Aspirin use in outpatients ... Yes ...
Meigs et al,51 2003 7 1 Screening (retinal examination, Change in glycated Yes No
foot examination, glycated hemoglobin, low-density
hemoglobin, blood pressure, lipoprotein cholesterol,
cholesterol), use of blood pressure
cholesterol-reducing and
blood pressure medications
in outpatients
*Ellipses indicate outcome was not assessed.
†Improvement was defined as a statistically significant positive effect on at least 50% of outcomes measured. Most studies had inadequate statistical power to detect a clinically
important improvement in patient outcomes.
‡Practitioner performance outcomes were the rate of screening (such as retinal examination or urine protein measurement), medication use, and/or identification of at-risk behaviors.

Table 4. Trials of Computer-Assisted Cardiovascular Disease Management and Prevention*

Improvement in Improvement
Methods No. of Practitioner in Patient
Source Score Sites Indication Patient Outcomes Performance†‡ Outcomes†
Coe et al,52 1977 6 2 Blood pressure management in Diastolic blood pressure, drug ... No
outpatients adverse effects
Barnett et al,54 4 1 Follow-up for patients with elevated Diastolic blood pressure ⬍100 Yes Yes
1983 blood pressure mm Hg or receiving treatment
Rogers et al,53 6 1 Management of hypertension, obesity, Systolic and diastolic blood Yes No
1984 and renal disease in outpatients pressure, hospitalization,
weight (improved), hospital
length of stay (improved)
Brownbridge 4 3 Hypertension management in ... Yes ...
et al,55 1986 outpatients: prompts for
hypertension care (such as urine
protein measurement, pulse
assessment and retinal
examination)
McAlister et al,56 7 50 Recommendations for Diastolic blood pressure No No
1986 antihypertensive use ⬍90 mm Hg
Rossi and Every,57 8 1 Alerts to substitute calcium channel ... Yes ...
1997 blocker antihypertensives to those
recommended in practice
guidelines in outpatient
hypertensives
Lowensteyn et al,58 7 Multiple Calculating coronary risk factor profile Blood pressure, body mass index, Yes No
1998 for outpatients smoking cessation,
cholesterol (total, LDL,
total/ HDL ratio) (improved)
Predicted 8-y coronary risk factor
score (improved)
Hetlevik et al,59 9 29 Diagnosis, treatment, and follow-up Glycated hemoglobin, smoking No No
1999 recommendations for status, body mass index,
hypertension, diabetes mellitus, cholesterol, risk score for
and hypercholesterolemia in future myocardial infarction,
outpatients; identification of diastolic blood pressure
smokers (improved)
Montgomery 8 27 Calculation of risk of new Predicted 5-y cardiovascular risk No No
et al,60 2000 cardiovascular event in outpatients score
Selker et al,61 2002 6 28 Thrombolytic prescribing in Mortality, stroke, bleeding No No
emergency department, with
recommendations printed on
electrocardiograms
Ansari et al,62 2003 9 1 ␤-Blocker use in outpatients with Emergency department visit or No No
congestive heart failure hospitalization, mortality
Tierney et al,63 8 1 Appropriate medications for patients Quality of life (SF-36), emergency No No
2003 with ischemic heart disease and department visits for heart
congestive heart failure; exercise disease, hospitalizations,
promotion, weight loss, and chronic heart disease
smoking cessation; treatment of questionnaire
hypertension and
hypercholesterolemia
Weir et al,64 2003 9 16 Antiplatelets and anticoagulant Predicted relative risk reduction of No No
prescribing in patients with an future ischemic vascular
acute ischemic stroke or transient events, hemorrhagic vascular
ischemic attack; included both events
inpatients and outpatients
Murray et al,65 9 1 Hypertension management and drug Health-related quality of life, No No
2004 prescriptions for outpatients emergency department visits
(2 ⫻ 2 factorial trial; randomization and hospitalizations, systolic
for physician to receive CDSS, and diastolic blood pressure
and randomization for pharmacist
to receive CDSS)
Abbreviations: CDSS, clinical decision support system; HDL, high-density lipoprotein; LDL, low-density lipoprotein; SF-36, Short Form 36.
‡Practitioner performance outcomes were adherence to recommended guidelines that usually included assessment of cardiac risk factors (blood pressure, cholesterol, smoking) and the
use of cardioprotective medications. The exception was Selker et al,61 in which practitioner performance outcomes were proportion receiving thromobolytics, use of thrombolytics
within 1 hour of initial electrocardiogram, and achievement of cardiac reperfusion.

management, drug dosing, or drug when studied. Fifty-two trials as- portant differences. Only 7 trials re-
prescribing. However, the effects of sessed patient outcomes, often in a ported improved patient outcomes with
these systems on patient health re- limited capacity without adequate sta- the CDSS, and no study reported ben-
main understudied—and inconsistent tistical power to detect clinically im- efits for major outcomes such as mor-
Table 5. Trials of Computer-Assisted Management for Other Active Health Conditions*

Source Score Sites Indication Performance Outcomes Patient Outcomes Performance† Outcomes†
Petrucci et al,66 7 2 Recommendations for Nurse knowledge of Rate of urinary Yes Yes
1991 nurse management of incontinence incontinence
urinary incontinence in
nursing homes
Rubenstein 8 1 Detection and management Physician recognition of Functional status (physical, Yes No
et al,67 1995 of functional status functional status psychological, and
impairments in problems, social) at 6 mo as
outpatients; patient recommended measured by
self-reported interventions questionnaire
information was undertaken to improve
collected for patient functioning
computer-assisted
system
Safran et al,68 6 1 Screening, treatment, and Vaccination, Need for physician visits; Yes No
1995 management ophthalmologic emergency and
recommendations for referral, CD4 cell count hospital admission;
outpatients with human and blood cell count, mortality
immunodeficiency virus Pneumocystis jiroveci
infection prophylaxis
Dexter et al,69 10 1 Reminders to discuss and Rates of discussions and ... Yes ...
1998 complete advanced documentation
directives in outpatients
East et al,70 1999 8 10 Mechanical ventilation ... Survival to hospital ... Yes
management discharge, intensive
(respiratory evaluation, care unit length of stay,
oxygenation, ventilation, barotrauma score
weaning, and (improved), multiorgan
extubation) in critically ill dysfunction score
patients with acute (improved)
respiratory distress
syndrome
Kuperman 6 1 Automated physician alerts Time to ordering of Adverse events (death, Yes No
et al,71 1999 via pager for critical treatment for critical cardiac arrest, transfer
laboratory results for laboratory value, time to intensive care unit,
hospital inpatients to resolution of alerting stroke, renal
condition impairment) within 48 h
of alerting event
Christakis et al,72 8 1 Recommendations for Unnecessary antibiotic ... Yes ...
2001 antibiotic use in prescriptions,
outpatient children prescriptions of
with otitis media excessive duration
McCowan 6 17 Recommended guidelines Review of Symptoms, need for oral No Yes
et al,73 2001 for treatment of self-management plan, steroid, need for
asthma in outpatients inhaler technique, and hospital services,
treatment adherence patient-initiated
with patient; issuance consultation to manage
of peak flow meter asthma (improved),
exacerbation of asthma
(self-report; improved),
emergency nebulizer
use (improved)
Eccles et al,74 9 62 Recommendations for Adherence to guidelines Self-reported quality of life No No
2002 angina and asthma including medication (generic and
management in prescribing, screening, disease-specific
outpatients and assessment of measures), symptoms
at-risk behaviors
Lesourd et al,75 7 3 Hormonal ovarian No. of missed menstrual Pregnancy No No
2002 stimulation for cycles
infertile women

tality. Surrogate patient outcomes such Determinants of CDSS Success tation include failure of practitioners to
as blood pressure and glycated hemo- Recent literature has called for a better use the CDSS, poor usability or integra-
globin were not meaningfully im- understanding of factors that predict tion into practitioner workflow, or prac-
proved in moststudies. CDSS success.119 Barriers to implemen- titioner nonacceptance of computer
Table 6. Trials of Computer Use to Monitor the Effects of Other Prescribed Treatments (Corollary Orders) or to Reduce Unnecessary Health
Care Utilization*
Source Score Sites Indication Practitioner Outcomes Patient Outcomes Performance† Outcomes†
Systems to monitor the
effects of corollary
orders
McDonald,76 5 1 Laboratory tests to monitor Adherence to ... Yes ...
1976 potential medication recommended care
adverse effects (such
as measurement of
serum creatinine,
potassium or
hemoglobin) in a
diabetes clinic
Young,77 1981 4 1 Recommended Adherence to ... Yes ...
investigations for 79 recommended
medical problems in ordering
hospital inpatients
Fihn et al,78 8 6 Frequency of Ability to increase Hemorrhagic and Yes No
1994 anticoagulant intervals between thromboembolic
monitoring in visits, proximity to complications
outpatients target international
normalized ratio
value
Overhage et al,79 10 1 Recommended tests or Compliance with orders Hospital length of stay, Yes No
1997 treatments to monitor maximum creatinine
or ameliorate the effects level during
of other tests or hospitalization
treatments for
hospital inpatients
Systems to reduce
unnecessary health
care utilization
Tierney et al,80 5 1 Prompts to dissuade Frequency of ... Yes ...
1988 ordering of routine unnecessary testing
unnecessary diagnostic
tests, such as
electrolyte levels, blood
counts, chest
radiographs, and
electrocardiograms in
outpatients
Tierney et al,81 9 1 Alerts for drug allergies Cost per patient Hospital length of stay, Yes No
1993 and drug interactions, admission need for hospital
choices for cost- readmission
effective testing as
part of inpatient
computerized order
entry for medications,
tests, and nursing
orders
Hales et al,82 6 1 Computer system for Rate of unnecessary ... No ...
1995 hospital admission admissions
screening
Shea et al,83 7 1 Messages for hospital ... Hospital length of stay ... Yes
1995 inpatients on diagnosis,
expected length of stay
Bates et al,84 8 1 Reminders for redundant Rate of redundant test ... Yes ...
1999 clinical laboratory tests ordering
in hospital inpatients

Table 7. Trials of Computer-Assisted Anticoagulant Dosing*

Methods No. of Practitioner Performance in Practitioner in Patient
Source Score Sites Indication Outcomes Patient Outcomes Performance† Outcomes†
Warfarin
Abbrecht 6 1 Warfarin initiation in Proportion of days in ... Yes ...
et al,85 postoperative therapeutic range, average
1982 cardiac surgery number of days to achieve
inpatients therapeutic INR
Carter et al,86 6 1 Warfarin initiation for Time to achieve therapeutic Time to hospital discharge No No
1987 hospital inpatients stable INR after first dose
White et al,87 8 2 Warfarin initiation for Time to achieve therapeutic Bleeding complications, Yes Yes
1987 hospital inpatients INR, time to reach stable hospital length of stay
warfarin dose, time (improved)
above therapeutic INR
White and 8 1 Warfarin maintenance Proportion of time with ... No ...
Mungall,88 for outpatients therapeutic INR, need for
1991 follow-up appointments for
anticoagulation adjustment
Poller et al,89 8 1 Warfarin maintenance Proportion achieving target Bleeding complications, No No
1993 for outpatients INR, average follow-up mortality
time for appointments
needed for anticoagulation
Fitzmaurice 6 2 Warfarin maintenance Proportion of time with Mortality, bleeding, Yes No
et al,90 for outpatients therapeutic INR, number and thrombotic
1996 of follow-up appointments complications
needed to adjust
anticoagulation
Vadher et al,91 6 1 Warfarin initiation and Time to achieve therapeutic Bleeding and thrombotic No No
1997 maintenance for INR, time with therapeutic complications
inpatients INR, number of
supratherapeutic and
subtherapeutic INR levels
Vadher et al,92 6 1 Warfarin maintenance Proportion of time with Thrombotic episodes, Yes No
1997 for outpatients; therapeutic INR, number bleeding complications
system used by of days between INR
nurse practitioner testing, number of test
compared with measurements
training physicians
in routine care
Ageno and 7 1 Warfarin maintenance Proportion of time with ... No ...
Turpie,93 for outpatients therapeutic INR,
1998 with mechanical proportion of INRs within
heart valves therapeutic range, number
of required dose
adjustments; number of
INR measurements
Poller et al,94 8 5 Warfarin initiation and Time to achieve therapeutic ... Yes ...
1998 maintenance for INR, proportion of time
outpatients with therapeutic INR
Fitzmaurice 8 12 Warfarin maintenance Proportion of time with Mortality, adverse events Yes No
et al,95 for outpatients; therapeutic INR, (bleeding or thrombosis)
2000 nurse-led clinic proportion of patients with
with point-of-care therapeutic INR
testing
Manotti et al,96 5 5 Warfarin maintenance Proportion of time with ... Yes ...
2001 for outpatients therapeutic INR over 1 y,
proportion of patients
achieving therapeutic
stable INR at 1 mo,
number of physician
follow-up appointments for
anticoagulation control
Heparin
Mungall 8 2 Heparin dosing used Therapeutic anticoagulation Composite of cardiovascular Yes Yes
et al,97 with acute after 24 h events (ie, recurrent chest
1994 myocardial pain, need for additional
infarction treated thrombolytics, stroke,
with thrombolytic cardiac arrest) (improved)
Abbreviation: INR, international normalized ratio.

Table 8. Trials of Computer-Assisted Drug Dosing and Prescribing*

Practitioner Improvement in Improvement
Methods No. of Performance Practitioner in Patient
Source Score Sites Indication Outcomes Patient Outcomes Performance† Outcomes†
Drug-dosing systems
Theophylline/aminophylline
Hurley et al,98 6 1 Theophylline dosing for Proportion of patients Theophylline toxicity, Yes No
1986 inpatients within therapeutic peak expiratory flow
range rate, asthma
symptom
questionnaire,
mortality, duration
of hospitalization
(shorter with CDSS)
Gonzalez et al,99 7 1 Aminophylline dosing Proportion of patients Aminophylline toxicity, No No
1989 in emergency within therapeutic emergency
department range department
discharge, peak
expiratory flow rate
Verner et al,100 6 1 Theophylline dosing in Proportion of patients Clinical score of Yes No
1992 emergency within therapeutic respiratory status,
department range peak expiratory flow
rate
Casner et al,101 6 1 Theophylline dosing Proportion of time Theophylline toxicity, No No
1993 for inpatients within therapeutic number of hospital
range days
Aminoglycosides
Begg et al,102 6 1 Aminoglycoside dosing Proportion of patients Mortality, decrease in Yes No
1989 for inpatients within therapeutic creatinine clearance
range
Hickling et al,103 5 1 Aminoglycoside dosing Proportion of patients Estimated creatinine Yes No
1989 in intensive care within therapeutic clearance
unit range
Burton et al,104 7 1 Aminoglycoside dosing Peak concentration of Mortality due to infection, Yes No
1991 for inpatients aminoglycoside response to therapy,
within therapeutic increase in serum
range creatinine level,
hospital length of stay
(shorter with CDSS)
Other medications
Peck et al,105 7 1 Digoxin dosing for Achievement of Digoxin toxicity, change No No
1973 outpatients with actual digoxin in heart failure
congestive heart concentration medications
failure relative to target
concentration
Rodman et al,106 8 1 Lidocaine dosing for Proportion needing Lidocaine toxicity Yes No
1984 hospital inpatients additional lidocaine
dose, achievement
of therapeutic dose
within 30 min
Ryff-de Leche 4 1 Insulin dosing for Blood glucose within Hypoglycemic events, Yes No
et al,107 outpatients therapeutic range, glycated hemoglobin
1992 glucose level ⬍4.0 level
mmol/L (72 mg/dL)
Horn et al,108 5 1 Parenteral nutrition Time required to ... No ...
2002 dosing for hospital calculate nutrition
inpatients composition and
amount,
inappropriate
ordering
(continued)
recommendations.120 In our review, stud- regression of 11 studies of computer or- identified better performance in studies
ies in which users were automatically der entry.121 Compared with manual ini- in which the trial authors also devel-
prompted to use the system described tiation, automatic prompting may oped the CDSS software. Potential ex-
better performance compared with improve integration into practitioner planations of this finding include the
studies in which users were required workflow as well as provide better op- motivational effect of a developer’s en-
to actively initiate the system. A similar portunities to correct inadvertent defi- thusiasm, creation of more usable and in-
finding was also reported in a meta- ciencies in care. In this review, we also tegrated software, better access to tech-

Table 8. Trials of Computer-Assisted Drug Dosing and Prescribing (cont)

Practitioner Improvement in Improvement
Methods No. of Performance Patient Practitioner in Patient
Source Score Sites Indication Outcomes Outcomes* Performance† Outcomes*†
Drug-prescribing systems
McDonald,109 1976 7 1 390 Recommended Adherence to ... Yes ...
management recommendations
protocols guiding
drug use, recognition
of adverse drug
reactions, and
laboratory tests in
outpatients
McDonald et al,110 8 1 410 Computerized Adherence to ... Yes ...
1980 management rules recommendations
dealing primarily with
use and follow-up of
medications in
outpatients
White et al,111 1984 7.5 1 Alerts of potential drug Adherence to ... Yes ...
interactions and recommendations
toxicity with digoxin
in inpatients
Rotman et al,112 7 1 Recommendation for Adherence to Adverse drug No No
1996 less expensive drug recommendations interactions
substitute when
available, alerts for
drug interactions in
outpatients
Tamblyn et al,113 8 Multiple Alerts for prescribing Inappropriate ... Yes ...
2003 errors, including drug prescriptions per
contraindications in 1000 visits,
outpatients discontinuation
of potentially
inappropriate
prescriptions
Abbreviation: CDSS, computerized clinical decision support system.
nical support and training, improved thors, and conducted a multivariable terminants of CDSS success imply sub-
on-site promotion and tailoring, biases analysis of study-level covariates. stantial imprecision in the strength of
in assessing outcomes, and selective pub- However, limitations of this review these associations, which may be non-
lication of successful trials. Most of the should be appreciated. We included only causal. Furthermore, it is possible that
CDSSs in this review were “home English-language studies. The CDSSs CDSSs for disease management pro-
grown,” and the importance of local were grouped into categories based on moted the implementation of ineffec-
champions to facilitate implementation clinical applications rather than on other tive therapies, or that CDSSs of drug dos-
cannot be underestimated. aspects of CDSS design.122 Although trial ing used incorrect pharmacokinetic
methods are improving with time, this models. Although this appears to be an
Strengths and Weaknesses summary is limited by the methods used unlikely explanation for the lack of effect
of This Review in the primary studies. We were unable on patient outcomes, we did not evalu-
We identified relevant controlled trials to use meta-analysis to pool effect sizes, ate the appropriateness of CDSS algo-
through a comprehensive search of the given substantial differences among pri- rithms or recommendations. Finally, we
literature. We extended our previous re- mary studies in the types of CDSSs and summarized controlled trials of CDSSs
view from 1998 in a number of impor- outcomes evaluated. In addition, we de- and did not consider less rigorous but
tant ways.5 Using better-defined inclu- fined improvement as a positive effect on more common designs, such as before-
sion criteria, we reconsidered all prior at least 50% of outcomes measured. This after studies.
articles and identified 37 new articles. approach, along with the strict inclu-
To identify CDSS and study character- sion criteria of this review, may have un- When to Adopt a CDSS
istics that predicted positive effects, we derestimated the influence of some sys- for Practice
abstracted relevant data from all ar- tem and study methodological factors on The decision to adopt a CDSS for local
ticles in duplicate, confirmed our ab- CDSS success. The wide confidence inpatient care is complex and is influ-
stractions with a majority of primary au- tervals for the statistically significant de- enced by many considerations. Those

responsible for CDSS implementation need to measure effects on local out- Analysis and interpretation of data: Garg, Adhikari,
McDonald, Rosas-Arellano, Devereaux, Beyene, Sam,
are typically administrators, informa- comes and be prepared to iteratively Haynes.
tion technology managers, and clini- modify their system in response to prac- Drafting of the manuscript: Garg, Adhikari, Haynes.
Critical revision of the manuscript for important in-
cians, all of whom are increasingly tice-based knowledge.2 tellectual content: Garg, Adhikari, McDonald,
pushed by technology and guided by While some perceive that CDSSs im- Rosas-Arellano, Devereaux, Beyene, Sam, Haynes.
Statistical analysis: Garg, Adhikari, Beyene.
government regulations.123 Important prove efficiency and reduce costs, the Obtained funding: Garg, Haynes.
issues include CDSS user acceptance, current supporting evidence is limited. Administrative, technical, or material support: Garg,
workflow integration, compatibility Although some studies have assessed Rosas-Arellano, Haynes.
Study supervision: Garg, Haynes.
with legacy applications, system ma- the costs when outcomes were
turity, and upgrade availability. Some improved, 4 0 , 4 5 , 7 9 - 8 1 , 8 4 , 1 2 8 the cost- Financial Disclosures: None reported.
Funding/Support: Dr Garg was supported by a Ca-
are concerned about increased practi- effectiveness of these systems remains nadian Institutes of Health Research (CIHR) Clinician
tioner dependence on CDSSs, with unknown. Many studies suggested Scientist Training Award. Dr Devereaux was sup-
eroded capacity for independent deci- the CDSS was inefficient, requiring ported by a CIHR Senior Research Fellowship.
Role of the Sponsors: No funding source or sponsor
sion making.31 Finally, cheaper, non- more time and effort from the user had any role in the design and conduct of the study;
computerized alternatives may be compared with paper-based meth- collection, management, analysis, or interpretation of
the data; or preparation, review, or approval of the
equally or more effective in improving ods.14,15,38,64,81,95,112 Finally, most CDSSs manuscript.
care and reducing medical errors.124-127 used research funding to facilitate imple- Acknowledgment: We acknowledge the work of Linda
Sheridan, who provided administrative help, and Tom
One of the primary considerations in mentation. As highlighted in this re- Flemming, the librarian who helped with the litera-
adopting a CDSS is its clinical effective- view, up to 21% of trials used staff paid ture searches. We thank Dereck Hunt, MD, MSc, and
ness: To what extent should it be proven by research funds for data entry or CDSS William Clark, MD, for their help and support.
beneficial before mass deployment? recommendation delivery. When invest-

Clearly, some testing is required, as a ing in a commercially available system, REFERENCES
CDSS can have unanticipated effects funding for support personnel is an 1. Wyatt J, Spiegelhalter D. Evaluating medical ex-
pert systems: what to test and how? Med Inform
when used in patient care.85 Some high- additional cost to be considered. (Lond). 1990;15:205-217.
light the need for multicenter cluster- There is currently widespread enthu- 2. Sim I, Gorman P, Greenes RA, et al. Clinical deci-
randomized controlled trials demon- siasm for introducing electronic medi- sion support systems for the practice of evidence-
based medicine. J Am Med Inform Assoc. 2001;8:527-
strating improvements in important cal records, computerized physician 534.
patient outcomes.12 Using such a stan- order entry systems, and CDSSs into hos- 3. Haynes RB, Walker CJ. Computer-aided quality
assurance. Arch Intern Med. 1987;147:1297-1301.
dard, this review suggests that the ma- pitals and outpatient settings. In other 4. Johnston ME, Langton KB, Haynes RB, Mathieu A.
jority of available systems are not yet commercial, industrial, and scientific Effects of computer-based clinical decision support sys-
tems on clinician performance and patient outcome.
ready for mainstream use. Most trials spheres of activity, computers have be- Ann Intern Med. 1994;120:135-142.
were unable to enroll enough clusters or come ubiquitous and have improved 5. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects
patients for adequate statistical power to safety, productivity, and timeliness. of computer-based clinical decision support systems
on physician performance and patient outcomes: a sys-
detect improvements in patient out- Given this progress, computerization of tematic review. JAMA. 1998;280:1339-1346.
comes. Unfortunately, this situation is the health care environment should of- 6. Kaushal R, Shojania KG, Bates DW. Effects of com-
puterized physician order entry and clinical decision
unlikely to change soon, given the sub- fer tremendous benefits. However, up- support systems on medication safety: a systematic
stantial time and resources needed to take has been slow, and multiple chal- review. Arch Intern Med. 2003;163:1409-1416.
7. Payne TH. Computer decision support systems.
conduct such trials, particularly in the lenges have arisen at every phase of Chest. 2000;118:47S-52S.
area of preventive health. Furthermore, software development, testing, and 8. Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Com-
puter-based guideline implementation systems: a sys-
CDSSs are limited by the cumulative implementation. The progress of CDSSs tematic review of functionality and effectiveness. J Am
knowledge used to program their rec- has mirrored these trends. Systems are Med Inform Assoc. 1999;6:104-114.
ommendations. It would be unrealistic proliferating, their technical perfor- 9. Kaplan B. Evaluating informatics applications—
clinical decision support systems literature review. Int
to require repeat CDSS testing every time mance and usability are improving, and J Med Inform. 2001;64:15-37.
advances in the knowledge base be- the number and quality of evaluations is 10. Gordon RB, Grimshaw JM, Eccles M, Rowe RE,
Wyatt JC. On-screen computer reminders: effects on
come available. Thus, for initial consid- increasing. These evaluations have professional practice and health care outcomes. Coch-
eration, it may be reasonable to require shown that many CDSSs improve prac- rane Database Syst Rev. 1998;2:CD001096.
11. Mitchell E, Sullivan F. A descriptive feast but an
proof of CDSS effectiveness only on practitioner performance. However, further evaluative famine: systematic review of published ar-
titioner performance, particularly if such research is needed to elucidate the ef- ticles on primary care computing during 1980-97. BMJ.
outcomes represent current accepted fects of such systems on patient health. 2001;322:279-282.
12. Chuang JH, Hripcsak G, Heitjan DF. Design and
standards in care. In our review, many analysis of controlled trials in naturally clustered en-
systems met this requirement. How- Author Contributions: Drs Garg, Adhikari, and Rosas- vironments: implications for medical informatics. J Am
Arellano had full access to all of the data in the study Med Inform Assoc. 2002;9:230-238.
ever, this does not preclude the need for and take responsibility for the integrity of the data and 13. Fleiss JL, Cohen J. The equivalence of weighted
subsequent trials or in-practice assess- the accuracy of the data analysis. kappa and the intraclass correlation coefficient as mea-
Study concept and design: Garg, Adhikari, Haynes. sures of reliability. Educ Psychol Meas. 1973;33:613-619.
ment to confirm system performance in Acquisition of data: Garg, Adhikari, McDonald, 14. Lewis G, Sharp D, Bartholomew J, Pelosi AJ. Com-
improving patient health. Institutions Rosas-Arellano, Sam, Haynes. puterized assessment of common mental disorders in

primary care: effect on clinical outcome. Fam Pract. mann S. Randomized study of online vaccine remind- hypertension by computer and physician: a prospec-
1996;13:120-126. ers in adult primary care. Proc AMIA Symp. 1999:755- tive controlled study. J Chronic Dis. 1977;30:81-92.
15. Cannon DS, Allen SN. A comparison of the ef- 759. 53. Rogers JL, Haring OM, Goetz JP. Changes in pa-
fects of computer and manual reminders on compli- 33. Tang PC, LaRosa MP, Newcomb C, Gorden SM. tient attitudes following the implementation of a medi-
ance with a mental health clinical practice guideline. Measuring the effects of reminders for outpatient in- cal information system. QRB Qual Rev Bull. 1984;10:
J Am Med Inform Assoc. 2000;7:196-203. fluenza immunizations at the point of clinical 65-74.
16. Schriger DL, Gibbons PS, Langone CA, Lee S, opportunity. J Am Med Inform Assoc. 1999;6:115-121. 54. Barnett GO, Winickoff RN, Morgan MM, Ziel-
Altshuler LL. Enabling the diagnosis of occult psychi- 34. McDonald CJ, Hui SL, Smith DM, et al. Remind- storff RD. A computer-based monitoring system for
atric illness in the emergency department: a random- ers to physicians from an introspective computer medi- follow-up of elevated blood pressure. Med Care. 1983;
ized, controlled trial of the computerized, self- cal record: a two-year randomized trial. Ann Intern 21:400-409.
administered PRIME-MD diagnostic system. Ann Emerg Med. 1984;100:130-138. 55. Brownbridge G, Evans A, Fitter M, Platts M. An
Med. 2001;37:132-140. 35. Tierney WM, Hui SL, McDonald CJ. Delayed feed- interactive computerized protocol for the manage-
17. Rollman BL, Hanusa BH, Lowe HJ, et al. A ran- back of physician performance versus immediate re- ment of hypertension: effects on the general practi-
domized trial using computerized decision support to minders to perform preventive care: effects on phy- tioner’s clinical behaviour. J R Coll Gen Pract. 1986;36:
improve treatment of major depression in primary care. sician compliance. Med Care. 1986;24:659-666. 198-202.
J Gen Intern Med. 2002;17:493-503. 36. Ornstein SM, Garr DR, Jenkins RG, Rust PF, Ar- 56. McAlister NH, Covvey HD, Tong C, Lee A, Wigle
18. Pozen MW, D’Agostino RB, Selker HP, Syt- non A. Computer-generated physician and patient re- ED. Randomised controlled trial of computer assisted
kowski PA, Hood WB. A predictive instrument to im- minders: tools to improve population adherence to se- management of hypertension in primary care. Br Med
prove coronary-care-unit admission practices in acute lected preventive services. J Fam Pract. 1991;32:82-90. J (Clin Res Ed). 1986;293:670-674.
ischemic heart disease: a prospective multicenter clini- 37. Rosser WW, McDowell I, Newell C. Use of re- 57. Rossi RA, Every NR. A computerized interven-
cal trial. N Engl J Med. 1984;310:1273-1278. minders for preventive procedures in family medicine. tion to decrease the use of calcium channel blockers
19. Wyatt JR. Lessons learnt from the field trial of CMAJ. 1991;145:807-814. in hypertension. J Gen Intern Med. 1997;12:672-678.
ACORN, an expert system to advise on chest pain. In: 38. Tape TG, Campbell JR. Computerized medical re- 58. Lowensteyn I, Joseph L, Levinton C, Abraham-
Barber B, Cao D, Quin D, Wagner G, eds. Proceed- cords and preventive health care: success depends on owicz M, Steinert Y, Grover S. Can computerized risk
ings of the 6th World Conference on Medical Infor- many factors. Am J Med. 1993;94:619-625. profiles help patients improve their coronary risk? the
matics; Singapore; 1989:111-115. 39. Turner RC, Peden JGJ, O’Brien K. Patient- results of the Coronary Health Assessment Study
20. Selker HP, Beshansky JR, Griffith JL, et al. Use of carried card prompts vs computer-generated prompts (CHAS). Prev Med. 1998;27:730-737.
the acute cardiac ischemia time-insensitive predictive to remind private practice physicians to perform health 59. Hetlevik I, Holmen J, Kruger O. Implementing clini-
instrument (ACI-TIPI) to assist with triage of patients maintenance measures. Arch Intern Med. 1994;154: cal guidelines in the treatment of hypertension in gen-
with chest pain or other symptoms suggestive of acute 1957-1960. eral practice: evaluation of patient outcome related
cardiac ischemia: a multicenter, controlled clinical trial. 40. Frame PS, Zimmer JG, Werth PL, et al. Computer- to implementation of a computer-based clinical de-
Ann Intern Med. 1998;129:845-855. based vs manual health maintenance tracking: a con- cision support system. Scand J Prim Health Care. 1999;
21. Wexler JR, Swender PT, Tunnessen WWJ, Oski trolled trial. Arch Fam Med. 1994;3:581-588. 17:35-40.
FA. Impact of a system of computer-assisted diagnosis. 41. Overhage JM, Tierney WM, McDonald CJ. Com- 60. Montgomery AA, Fahey T, Peters TJ, MacIntosh
AJDC. 1975;129:203-205. puter reminders to implement preventive care guide- C, Sharp DJ. Evaluation of computer based clinical de-
22. Wellwood J, Johannessen S, Spiegelhalter DJ. How lines for hospitalized patients. Arch Intern Med. 1996; cision support system and risk chart for management
does computer-aided diagnosis improve the manage- 156:1551-1556. of hypertension in primary care: randomised con-
ment of acute abdominal pain? Ann R Coll Surg Engl. 42. Bonevski B, Sanson-Fisher RW, Campbell E, Car- trolled trial. BMJ. 2000;320:686-690.
1992;74:40-46. ruthers A, Reid AL, Ireland M. Randomized controlled 61. Selker HP, Beshansky JR, Griffith JL. Use of the
23. Bogusevicius A, Maleckas A, Pundzius J, Skau- trial of a computer strategy to increase general practi- electrocardiograph-based thrombolytic predictive in-
dickas D. Prospective randomised trial of computer- tioner preventive care. Prev Med. 1999;29:478-486. strument to assist thrombolytic and reperfusion therapy
aided diagnosis and contrast radiography in acute small 43. Demakis JG, Beauchamp C, Cull WL, et al. Improv- for acute myocardial infarction: a multicenter, ran-
bowel obstruction. Eur J Surg. 2002;168:78-83. ing residents’ compliance with standards of ambula- domized, controlled, clinical effectiveness trial. Ann In-
24. Turner BJ, Day SC, Borenstein B. A controlled trial tory care: results from the VA Cooperative Study on tern Med. 2002;137:87-95.
to improve delivery of preventive care: physician or Computerized Reminders. JAMA. 2000;284:1411-1416. 62. Ansari M, Shlipak MG, Heidenreich PA, et al. Im-
patient reminders? J Gen Intern Med. 1989;4:403-409. 44. Dexter PR, Perkins S, Overhage JM, Maharry K, proving guideline adherence: a randomized trial evalu-
25. McPhee SJ, Bird JA, Jenkins CN, Fordham D. Pro- Kohler RB, McDonald CJ. A computerized reminder ating strategies to increase beta-blocker use in heart
moting cancer screening: a randomized, controlled trial system to increase the use of preventive care for hos- failure. Circulation. 2003;107:2799-2804.
of three interventions. Arch Intern Med. 1989;149:1866- pitalized patients. N Engl J Med. 2001;345:965-970. 63. Tierney WM, Overhage JM, Murray MD, et al. Ef-
1872. 45. Thomas JC, Moore A, Qualls PE. The effect on cost fects of computerized guidelines for managing heart dis-
26. McPhee SJ, Bird JA, Fordham D, Rodnick JE, Os- of medical care for patients treated with an automated ease in primary care. J Gen Intern Med. 2003;18:967-
born EH. Promoting cancer prevention activities by pri- clinical audit system. J Med Syst. 1983;7:307-313. 976.
mary care physicians: results of a randomized, con- 46. Mazzuca SA, Vinicor F, Einterz RM, Norton JA, 64. Weir CJ, Lees KR, MacWalter RS, et al. Cluster-
trolled trial. JAMA. 1991;266:538-544. Kalasinski LA. Effects of the clinical environment on randomized, controlled trial of computer-based decision
27. Burack RC, Gimotty PA, George J, et al. Promot- physicians’ response to postgraduate medical support for selecting long-term anti-thrombotic therapy
ing screening mammography in inner-city settings: a education. Am Educ Res J. 1990;27:473-488. after acute ischaemic stroke. QJM. 2003;96:143-153.
randomized controlled trial of computerized remind- 47. Nilasena DS, Lincoln MJ. A computer-generated 65. Murray MD, Harris LE, Overhage JM, et al. Fail-
ers as a component of a program to facilitate reminder system improves physician compliance with ure of computerized treatment suggestions to im-
mammography. Med Care. 1994;32:609-624. diabetes preventive care guidelines. Proc Annu Symp prove health outcomes of outpatients with uncom-
28. Burack RC, Gimotty PA, George J, Simon MS, Dews Comput Appl Med Care. 1995:640-645. plicated hypertension: results of a randomized
P, Moncrease A. The effect of patient and physician 48. Lobach DF, Hammond WE. Computerized deci- controlled trial. Pharmacotherapy. 2004;24:324-337.
reminders on use of screening mammography in a sion support based on a clinical practice guideline im- 66. Petrucci K, Petrucci P, Canfield K, et al. Evaluation
health maintenance organization: results of a ran- proves compliance with care standards. Am J Med. of UNIS: urological nursing information systems. Proc
domized controlled trial. Cancer. 1996;78:1708-1721. 1997;102:89-98. Annu Symp Comput Appl Med Care. 1991:43-47.
29. Burack RC, Gimotty PA. Promoting screening 49. Montori VM, Dinneen SF, Gorman CA, et al. The 67. Rubenstein LV, McCoy JM, Cope DW, et al. Im-
mammography in inner-city settings: the sustained ef- impact of planned care and a diabetes electronic man- proving patient quality of life with feedback to phy-
fectiveness of computerized reminders in a random- agement system on community-based diabetes care. sicians about functional status. J Gen Intern Med. 1995;
ized controlled trial. Med Care. 1997;35:921-931. Diabetes Care. 2002;25:1952-1957. 10:607-614.
30. Burack RC, Gimotty PA, Simon M, et al. The effect 50. Filippi A, Sabatini A, Badioli L, et al. Effects of an 68. Safran C, Rind DM, Davis RB, et al. Guidelines for
of adding Pap smear information to a mammogra- automated electronic reminder in changing the anti- management of HIV infection with computer-based
phy reminder system in an HMO: results of random- platelet drug-prescribing behavior among Italian gen- patient’s record. Lancet. 1995;346:341-346.
ized controlled trial. Prev Med. 2003;36:547-554. eral practitioners in diabetic patients: an intervention 69. Dexter PR, Wolinsky FD, Gramelspacher GP, et al.
31. Chambers CV, Balaban DJ, Carlson BL, Gras- trial. Diabetes Care. 2003;26:1497-1500. Effectiveness of computer-generated reminders for in-
berger DM. The effect of microcomputer-generated 51. Meigs JB, Cagliero E, Dubey A, et al. A con- creasing discussions about advance directives and
reminders on influenza vaccination rates in a university- trolled trial of Web-based diabetes disease manage- completion of advance directive forms: a randomized,
based family practice center. J Am Board Fam Pract. ment: the MGH Diabetes Primary Care Improve- controlled trial. Ann Intern Med. 1998;128:102-110.
1991;4:19-26. ment Project. Diabetes Care. 2003;26:750-757. 70. East TD, Heermann LK, Bradshaw RL, et al. Effi-
32. Flanagan JR, Doebbeling BN, Dawson J, Beek- 52. Coe FL, Norton E, Oparil S, et al. Treatment of cacy of computerized decision support for mechani-

cal ventilation: results of a prospective multi-center ran- 91. Vadher B, Patterson DL, Leaning M. Evaluation 110. McDonald CJ, Wilson GA, McCabe GP Jr. Phy-
domized trial. Proc AMIA Symp. 1999:251-255. of a decision support system for initiation and control sician response to computer reminders. JAMA. 1980;
71. Kuperman GJ, Teich JM, Tanasijevic MJ, et al. Im- of oral anticoagulation in a randomised trial. BMJ. 1997; 244:1579-1581.
proving response to critical laboratory results with au- 314:1252-1256. 111. White KS, Lindsay A, Pryor TA, Brown WF, Walsh
tomation: results of a randomized controlled trial. 92. Vadher BD, Patterson DL, Leaning M. Compari- K. Application of a computerized medical decision-
J Am Med Inform Assoc. 1999;6:512-522. son of oral anticoagulant control by a nurse- making process to the problem of digoxin intoxication.
72. Christakis DA, Zimmerman FJ, Wright JA, et al. practitioner using a computer decision-support sys- J Am Coll Cardiol. 1984;4:571-576.
A randomized controlled trial of point-of-care evi- tem with that by clinicians. Clin Lab Haematol. 1997; 112. Rotman BL, Sullivan AN, McDonald TW, et al.
dence to improve the antibiotic prescribing practices 19:203-207. A randomized controlled trial of a computer-based phy-
for otitis media in children. Pediatrics. 2001;107:E15. 93. Ageno W, Turpie AG. A randomized compari- sician workstation in an outpatient setting: implemen-
73. McCowan C, Neville RG, Ricketts IW, Warner FC, son of a computer-based dosing program with a tation barriers to outcome evaluation. J Am Med In-
Hoskins G, Thomas GE. Lessons from a randomized manual system to monitor oral anticoagulant therapy. form Assoc. 1996;3:340-348.
controlled trial designed to evaluate computer deci- Thromb Res. 1998;91:237-240. 113. Tamblyn R, Huang A, Perreault R, et al. The medi-
sion support software to improve the management of 94. Poller L, Shiach CR, MacCallum PK, et al. Multi- cal office of the 21st century (MOXXI): effectiveness
asthma. Med Inform Internet Med. 2001;26:191-201. centre randomised study of computerised anticoagu- of computerized decision-making support in reduc-
74. Eccles M, McColl E, Steen N, et al. Effect of com- lant dosage. Lancet. 1998;352:1505-1509. ing inappropriate prescribing in primary care. CMAJ.
puterised evidence based guidelines on manage- 95. Fitzmaurice DA, Hobbs FD, Murray ET, Holder RL, 2003;169:549-556.
ment of asthma and angina in adults in primary care: Allan TF, Rose PE. Oral anticoagulation management 114. McKinley BA, Moore FA, Sailors RM, et al. Com-
cluster randomised controlled trial. BMJ. 2002;325:941. in primary care with the use of computerized decision puterized decision support for mechanical ventilation
75. Lesourd F, Avril C, Boujennah A, Parinaud J. A com- support and near-patient testing: a randomized, con- of trauma induced ARDS: results of a randomized clini-
puterized decision support system for ovarian stimu- trolled trial. Arch Intern Med. 2000;160:2343-2348. cal trial. J Trauma. 2001;50:415-424.
lation by gonadotropins. Fertil Steril. 2002;77:456-460. 96. Manotti C, Moia M, Palareti G, Pengo V, Ria L, 115. Hetlevik I, Holmen J, Kruger O, Kristensen P,
76. McDonald CJ. Use of a computer to detect and Dettori AG. Effect of computer-aided management on Iversen H, Furuseth K. Implementing clinical guide-
respond to clinical events: its effect on clinician the quality of treatment in anticoagulated patients: a lines in the treatment of diabetes mellitus in general
behavior. Ann Intern Med. 1976;84:162-167. prospective, randomized, multicenter trial of AP- practice: evaluation of effort, process, and patient out-
77. Young DW. Improving the consistency with which ROAT (Automated PRogram for Oral Anticoagulant come related to implementation of a computer-
investigations are requested. Med Inform (Lond). 1981; Treatment). Haematologica. 2001;86:1060-1070. based decision support system. Int J Technol Assess
6:13-17. 97. Mungall DR, Anbe D, Forrester PL, et al. A pro- Health Care. 2000;16:210-227.
78. Fihn SD, McDonell MB, Vermes D, et al. A comput- spective randomized comparison of the accuracy of com- 116. Rogers JL, Haring OM, Wortman PM, Watson
erizedinterventiontoimprovetimingofoutpatientfollow- puter-assisted versus GUSTO nomogram–directed hep- RA, Goetz JP. Medical information systems: assess-
up: a multicenter randomized trial in patients treated with arin therapy. Clin Pharmacol Ther. 1994;55:591-596. ing impact in the areas of hypertension, obesity and
warfarin. J Gen Intern Med. 1994;9:131-139. 98. Hurley SF, Dziukas LJ, McNeil JJ, Brignell MJ. A renal disease. Med Care. 1982;20:63-74.
79. Overhage JM, Tierney WM, Zhou XH, McDonald randomized controlled clinical trial of pharmacoki- 117. Rogers JL, Haring OM. The impact of a com-
CJ. A randomized trial of “corollary orders” to pre- netic theophylline dosing. Am Rev Respir Dis. 1986; puterized medical record summary system on inci-
vent errors of omission. J Am Med Inform Assoc. 1997; 134:1219-1224. dence and length of hospitalization. Med Care. 1979;
4:364-375. 99. Gonzalez ER, Vanderheyden BA, Ornato JP, Com- 17:618-630.
80. Tierney WM, McDonald CJ, Hui SL, Martin DK. stock TG. Computer-assisted optimization of amino- 118. Hetlevik I, Holmen J, Kruger O, et al. Imple-
Computer predictions of abnormal test results: effects phylline therapy in the emergency department. Am J menting clinical guidelines in the treatment of hyper-
on outpatient testing. JAMA. 1988;259:1194-1198. Emerg Med. 1989;7:395-401. tension in general practice. Blood Press. 1998;7:270-
81. Tierney WM, Miller ME, Overhage JM, McDonald 100. Verner D, Seligmann H, Platt S, et al. Computer 276.
CJ. Physician inpatient order writing on microcom- assisted design of a theophylline dosing regimen in acute 119. Holbrook A, Xu S, Banting J. What factors de-
puter workstations: effects on resource utilization. bronchospasm: serum concentrations and clinical termine the success of clinical decision support systems?
JAMA. 1993;269:379-383. outcome. Eur J Clin Pharmacol. 1992;43:29-33. AMIA Annu Symp Proc. 2003:862.
82. Hales JW, Gardner RM, Jacobson JT. Factors im- 101. Casner PR, Reilly R, Ho H. A randomized con- 120. Reisman Y. Computer-based clinical decision aids:
pacting the success of computerized preadmission trolled trial of computerized pharmacokinetic theoph- a review of methods and assessment of systems. Med
screening. Proc Annu Symp Comput Appl Med Care. ylline dosing versus empiric physician dosing. Clin Phar- Inform (Lond). 1996;21:179-197.
1995:728-732. macol Ther. 1993;53:684-690. 121. Kawamoto K, Lobach DF. Clinical decision
83. Shea S, Sideli RV, DuMouchel W, Pulver G, Arons 102. Begg EJ, Atkinson HC, Jeffery GM, Taylor NW. support provided within physician order entry systems:
RR, Clayton PD. Computer-generated informational Individualised aminoglycoside dosage based on phar- a systematic review of features effective for changing
messages directed to physicians: effect on length of macokinetic analysis is superior to dosage based on clinician behavior. AMIA Annu Symp Proc. 2003:361-
hospital stay. J Am Med Inform Assoc. 1995;2:58-64. physician intuition at achieving target plasma drug 365.
84. Bates DW, Kuperman GJ, Rittenberg E, et al. A concentrations. Br J Clin Pharmacol. 1989;28:137-141. 122. Berlin A, Sorani M, Sim I. Characteristics of out-
randomized trial of a computer-based intervention to 103. Hickling K, Begg E, Moore ML. A prospective patient clinical decision support systems: a taxo-
reduce utilization of redundant laboratory tests. Am randomised trial comparing individualised pharmaco- nomic description. Medinfo. 2004;2004:578-581.
J Med. 1999;106:144-150. kinetic dosage prediction for aminoglycosides with pre- 123. Fox J, Thomson R. Clinical decision support sys-
85. Abbrecht PH, O’Leary TJ, Behrendt DM. Evalu- diction based on estimated creatinine clearance in criti- tems: a discussion of quality, safety and legal liability
ation of a computer-assisted method for individual- cally ill patients. Intensive Care Med. 1989;15:233-237. issues. Proc AMIA Symp. 2002:265-269.
ized anticoagulation: retrospective and prospective 104. Burton ME, Ash CL, Hill DP Jr, Handy T, Shep- 124. Ofman JJ, Badamgarav E, Henning JM, et al. Does
studies with a pharmacodynamic model. Clin Phar- herd MD, Vasko MR. A controlled trial of the cost ben- disease management improve clinical and economic
macol Ther. 1982;32:129-136. efit of computerized Bayesian aminoglycoside outcomes in patients with chronic diseases? a system-
86. Carter BL, Taylor JW, Becker A. Evaluation of three administration. Clin Pharmacol Ther. 1991;49:685-694. atic review. Am J Med. 2004;117:182-192.
dosage-prediction methods for initial in-hospital stabi- 105. Peck CC, Sheiner LB, Martin CM, et al. Computer- 125. Warsi A, Wang PS, LaValley MP, Avorn J, So-
lization of warfarin therapy. Clin Pharm. 1987;6:37-45. assisted digoxin therapy. N Engl J Med. 1973;289:441- lomon DH. Self-management education programs in
87. White RH, Hong R, Venook AP, et al. Initiation 446. chronic disease: a systematic review and method-
of warfarin therapy: comparison of physician dosing 106. Rodman JH, Jelliffe RW, Kolb E, et al. Clinical ological critique of the literature. Arch Intern Med.
with computer-assisted dosing. J Gen Intern Med. studies with computer-assisted initial lidocaine therapy. 2004;164:1641-1649.
1987;2:141-148. Arch Intern Med. 1984;144:703-709. 126. Grimshaw JM, Thomas RE, MacLennan G, et al.
88. White RH, Mungall D. Outpatient management 107. Ryff-de Leche A, Engler H, Nutzi E, Berger M, Effectiveness and efficiency of guideline dissemina-
of warfarin therapy: comparison of computer- Berger W. Clinical application of two computerized dia- tion and implementation strategies. Health Technol
predicted dosage adjustment to skilled professional betes management systems: comparison with the log- Assess. 2004;8:iii-iv, 1-72.
care. Ther Drug Monit. 1991;13:46-50. book method. Diabetes Res. 1992;19:97-105. 127. Stone EG, Morton SC, Hulscher ME, et al. In-
89. Poller L, Wright D, Rowlands M. Prospective com- 108. Horn W, Popow C, Miksch S, et al. Develop- terventions that increase use of adult immunization
parative study of computer programs used for man- ment and evaluation of VIE-PNN, a knowledge- and cancer screening services: a meta-analysis. Ann
agement of warfarin. J Clin Pathol. 1993;46:299-303. based system for calculating the parenteral nutrition Intern Med. 2002;136:641-651.
90. Fitzmaurice DA, Hobbs FD, Murray ET, Bradley of newborn infants. Artif Intell Med. 2002;24:217-228. 128. McDonald CJ, Hui SL, Tierney WM. Effects of
CP, Holder R. Evaluation of computerized decision sup- 109. McDonald CJ. Protocol-based computer remind- computer reminders for influenza vaccination on mor-
port for oral anticoagulation management based in pri- ers, the quality of care and the non-perfectability of bidity during influenza epidemics. MD Comput. 1992;
mary care. Br J Gen Pract. 1996;46:533-535. man. N Engl J Med. 1976;295:1351-1355. 9:304-312.

Garg 2005

Uploaded by

Copyright:

Available Formats

Garg 2005

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Garg 2005

Uploaded by

Copyright:

Available Formats

REVIEW

Effects of Computerized Clinical Decision

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 1. Trials of Computer-Assisted Diagnosis*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 3. Trials of Computer-Assisted Diabetes Management*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 4. Trials of Computer-Assisted Cardiovascular Disease Management and Prevention*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 5. Trials of Computer-Assisted Management for Other Active Health Conditions*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 7. Trials of Computer-Assisted Anticoagulant Dosing*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 8. Trials of Computer-Assisted Drug Dosing and Prescribing*

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Table 8. Trials of Computer-Assisted Drug Dosing and Prescribing (cont)

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

beneficial before mass deployment? recommendation delivery. When invest-

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

Downloaded From: http://jama.jamanetwork.com/ by a Georgian Court University User on 05/12/2015

You might also like