Open access Review

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
Dashboards for visual display of patient
safety data: a systematic review
Daniel R Murphy,1,2 April Savoy,3,4,5 Tyler Satterly,1,2 Dean F Sittig ‍ ‍,6,7
Hardeep Singh1,2

To cite: Murphy DR, Savoy A, ABSTRACT of healthcare delivery. Despite increased

Satterly T, et al. Dashboards Background Methods to visualise patient safety data can research and quality improvement efforts,
for visual display of patient support effective monitoring of safety events and discovery
safety data: a systematic
how data on patient safety events is commu-
of trends. While quality dashboards are common, use and nicated to people who will act on these data
review. BMJ Health Care Inform
impact of dashboards to visualise patient safety event data
2021;28:e100437. doi:10.1136/ is not well understood. For instance, due
bmjhci-2021-100437 remains poorly understood.
to national quality reporting programmes,
Objectives To understand development, use and direct or
Received 06 July 2021 indirect impacts of patient safety dashboards. such as the Centers for Medicare & Medicaid
Accepted 22 September 2021 Methods We conducted a systematic review in Services’ Quality Payment Program,2 which
accordance with the Preferred Reporting Items for adjusts healthcare organisation’s reimburse-
Systematic Reviews and Meta-­Analyses guidelines. We ment rates based on meeting certain quality
searched PubMed, EMBASE and CINAHL for publications measures, dashboards have been used exten-
between 1 January 1950 and 30 August 2018 involving sively to visualise and disseminate process-­
use of dashboards to display data related to safety targets based quality measures such as understanding
defined by the Agency for Healthcare Research and
how well haemoglobin A1c is controlled
Quality’s Patient Safety Net. Two reviewers independently
reviewed search results for inclusion in analysis and
across all of a clinic’s patients. However, an
resolved disagreements by consensus. We collected data understanding of how commonly dashboards
are used for patient safety-­specific measures

copyright.
© Author(s) (or their on development, use and impact via standardised data
employer(s)) 2021. Re-­use collection forms and analysed data using descriptive and how effective they are at advancing
permitted under CC BY-­NC. No statistics. patient safety efforts and safety culture
commercial re-­use. See rights Results Literature search identified 4624 results which remains unknown.
and permissions. Published by were narrowed to 33 publications after applying inclusion
BMJ.
Dashboards have been used extensively
and exclusion criteria and consensus across reviewers. within and outside healthcare and serve as a
1
Center for Innovations in Publications included only time series and case study
Quality, Effectiveness and Safety, form of visual information display that allows
designs and were inpatient focused and emergency
Michael E DeBakey VA Medical
department focused. Information on direct impact of
for efficient data dissemination.3 4 Dash-
Center, Houston, Texas, USA boards aggregate data to provide overviews of
2 dashboards was limited, and only four studies included
Department of Medicine, Baylor
informatics or human factors principles in development or key performance indicators to facilitate deci-
College of Medicine, Houston,
Texas, USA postimplementation evaluation. sion making, and when used correctly, enable
3
Purdue School of Engineering Discussion Use of patient-­safety dashboards has efforts to improve an organisation’s structure,
and Technology, Indiana grown over the past 15 years, but impact remains poorly process and outcomes.4 5 For dashboards to
University Purdue University understood. Dashboard design processes rarely use play a strategic role in communicating patient
at Indianapolis, Indianapolis, informatics or human factors principles to ensure that the
Indiana, USA
safety data, it is essential they are designed
available content and navigation assists task completion,
4
Center for Health Information to relay key information about performance
communication or decision making.
and Communication, Richard L
Conclusion Design and usability evaluation of patient
effectively.6 Thus, the dashboard design must
Roudebush VA Medical Center, consider informatics and human factors prin-
safety dashboards should incorporate informatics and
Indianapolis, Indiana, USA
5 human factors principles. Future assessments should also ciples to ensure information is efficiently
Center for Health Services
Research, Regenstrief Institute, rigorously explore their potential to support patient safety communicated. Informatics and human
Inc, Indianapolis, Indiana, USA monitoring including direct or indirect impact on patient factors approaches have been successful in
6
School of Biomedical safety. the design and evaluation of user interfaces
Informatics, The University of in healthcare, and have variably been applied
Texas Health Science Center at
Houston, Houston, Texas, USA
to dashboard development.7 One common
7
The UT-­Memorial Hermann INTRODUCTION approach is user-­centred design, which is an
Center for Healthcare Quality & Since the 2000 release of the Institute of iterative design process that aims to optimise
Safety, Houston, Texas, USA Medicine’s landmark report, To Err is Human: usability of a display by focusing on users
Correspondence to
Building a Safer Healthcare System,1 healthcare and their needs through requirement anal-
Dr Dean F Sittig; organisations have increasingly gathered, ysis, translation of requirements into design
​dean.​f.​sittig@​uth.​tmc.​edu analysed and used data to improve the safety elements, application of design principles

Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437 1

Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
Table 1 Agency for Healthcare Research and Quality Safety Targets
No Safety topic Examples
1 Alert Fatigue Failure to recognise ventilator alarm
2 Device-­related complication Device malfunction
3 Diagnostic errors Delayed stroke diagnosis, test misinterpretation
4 Discontinuities, gaps and hand-­off problems Missed critical lab result
5 Drug shortages Antibiotics shortage
6 Failure to rescue Death from postpartum haemorrhage
7 Fatigue and sleep deprivation Resident errors due to sleep deprivation
8 Identification errors Wrong-­patient procedures
9 Inpatient suicide Death of hospitalised patient
10 Interruptions and distractions Incorrect surgical counts due to distractions
11 Medical complications Falls, pressure ulcers, nosocomial infections, thromboembolism
12 Medication safety Dispensing errors, medication-­related hypoglycaemic or renal failure
13 MRI safety Harm related to unsafe MRI practice
14 Nonsurgical procedural complications Bedside procedure complications
15 Overtreatment Complications after inappropriate antibiotic use
16 Psychological and social complications Privacy violations
17 Second victims Clinician emotional harm after adverse event
18 Surgical complications Unexpected return to surgery, surgical site infection
19 Transfusion complications Transfusion of incompatible blood types

From: https://psnet.ahrq.gov/Topics.
MRI, Magnetic Resonance Imaging.

and evaluation.8 Considering dashboards, usability would (MEDLINE via PubMed, EMBASE and CINAHL). Publi-

copyright.
be defined as the extent to which a dashboard can be cations were eligible for inclusion if they included discus-
used by clinicians to understand and achieve specified sion about a dashboard for displaying patient safety event
goals with effectiveness, efficiency and satisfaction in clin- data in the healthcare setting. Patient safety event data
ical settings.9 were based on the list of ‘Safety Targets’ (table 1) on the
Three main goals that guided this study were: (1) To Agency for Healthcare Research and Quality’s (AHRQ)
understand the frequency and settings of use of patient Patient Safety Network (PSNet),10 and excluded process
safety dashboards in healthcare, (2) To determine the measures. Because of the variety of topics within patient
effectiveness of dashboards on directly or indirectly safety, we ultimately used only the word ‘dashboard’ in
impacting patient safety at healthcare organisations our keyword and title search of all three databases, since
and (3) To determine whether informatics and human this maximised the number of known publications iden-
factors principles are commonly used during dashboard tified without excluding relevant publications. Thus, our
development and evaluation. Our study focused on dash- inclusion parameters, in PICOS format, were:
boards that displayed the frequency or rate of events, Population: Organisations providing medical care.
that is, those that facilitated retrospective review of past Interventions: Dashboards used to disseminate patient
safety events to reduce these types of events in the future safety data (defined as measures related to any topic
or dashboards that identified safety events of individual defined as a ‘Safety Target’ (table 1) by the AHRQ).10
patients in real-­time in order to mitigate further harm. Comparators: Settings with and without the use of
We excluded dashboards that only displayed risk of an patient safety dashboards.
event. Outcomes: (1) Settings where patient safety dashboards
were used and (2) Impact of use of patient safety dash-
boards on reducing patient safety events.
METHODS
Time frame: Studies published in English from 1
Design
January 1950 to 30 August 2018.
We conducted a systematic literature review in accor-
Setting: Ambulatory care, inpatient and emergency
dance with the Preferred Reporting Items for Systematic
department settings.
Reviews and Meta-­Analyses guidelines.

Search strategy and inclusion criteria Screening process

We searched all available published and unpublished After manually removing duplicates and non-­ journal
works in English using three literature databases publications (eg, magazine articles and book chapters),

2 Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437

 Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
Figure 1 Flow chart of literature search results and the selection process of accepted/excluded publications.

two authors (DRM and TS) with expertise in clinical RESULTS

care, informatics and human factors reviewed titles and Our literature search identified a total of 4624 results
abstracts of each remaining article or abstract. Works (PubMed: 693, CINAHL: 2590, Embase: 1341). After 933
were only included if they described display of patient duplicates were removed, 3691 result entries remained.
safety event data (based on AHRQ’s PSNet list of Patient One reviewer (TS) subsequently removed 2134 maga-

copyright.
Safety Targets) on a dashboard. Publications that zine articles, newspaper articles, thesis papers, confer-
discussed only non-­safety event-­related aspects of quality ence papers, reports that were unrelated to the topic
(eg, haemoglobin A1c control or rates of mammography of patient safety, as well as publications not in English.
screening) were excluded. Similarly, literature on dash- Titles and abstracts of the remaining 1557 articles and
boards displaying risk factors to prevent patient safety conference abstracts were independently reviewed by two
events rather than events themselves (eg, intensive care reviewers (TS and DRM). Reviewers manually reviewed
screens that display a particular patient’s heart rate and titles and abstracts and excluded (A) publications that
oxygenation saturation or calculate a real-­time risk level) did not include discussion of a dashboard as a primary
were beyond the scope of this study and were excluded. or secondary focus, and (B) publications where dash-
We reviewed all publications potentially meeting study boards were mentioned, but the dashboard did not
criteria in full. Reviewers discussed each inclusion, and include measures related to any of the AHRQ ‘Safety
disagreements regarding whether an article or abstract Targets’ (table 1). After exclusions, reviewers identified a
met criteria were resolved by consensus. combined total of 81 publications that warranted further
review of the entire publication. Reviewers discussed
Publication evaluation each publication, and after consensus, identified 33 final
Three authors (DRM, TS and AS) independently publications that warranted inclusion in the analysis.
extracted data from each identified publication using Reference sections of each publication were reviewed for
a structured review form. Reviewers specifically identi- additional sources but did not identify additional publica-
fied (1) the setting the dashboard was used in, (2) the tions. Figure 1 displays a flow chart of the search strategy.
patient safety topic displayed on the dashboard, (3) the
type of informatics or human factors principles used in Search results
dashboard design or usability evaluation performed on The final set included 33 publications, including 5 confer-
the final dashboard and (4) the impact of the dashboard, ence abstracts and 28 full articles (table 2). The earliest
both related to reducing patient safety events in the publications describe use of patient safety measures on
setting where it was used and other impacts identified by a dashboard in 2004, 2005 and 2006,11–13 followed by a
each publication’s authors. To assess the level of evidence paucity of additional publications until 2010.
in improving patient safety, reviewers also assessed the
study type and whether a control or other comparison Clinical settings
group was used. Findings are aggregated and reported All patient safety dashboards were used in the hospital
using descriptive statistics. setting, often at the level of the entire hospital or hospital

Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437 3

Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
Table 2 Final studies using patient safety dashboards identified during literature search
Citation Type Setting Safety topic Study type
14
Anand (2015) Article Paediatric cardiac ICU Pressure ulcers, unplanned extubation, hospital Case report
infections (CAUTI, CLABSI, VAP)
Bakos (2012)24 Article Trauma centre Hospital infections (CLABSI) Case report
17
Chandraharan (2010) Article Maternity ward Postpartum haemorrhage Case report
Coleman (2013)18 Article Hospital wards Medication-­related events Time series
Collier (2015)22 Article Inpatient maternity and Pressure ulcers Case report
paediatrics wards
Conway (2012)25 Article Trauma centre Surgical site infections Case report
Dharamshi (2011)27 Article Surgery Return to surgery Case report
Donaldson (2005)12 Article Surgery, critical care floors Pressure ulcers, falls Case report
23
Fong (2017) Article Pharmacy Medication-­related events Case report
Frazier (2012)29 Article Whole hospital Falls, hospital infections (MRSA, C. Diff, VAP, Case report
CLABSI, CAUTI), Pressure ulcers
Gardner (2015)36 Article Whole hospital Falls Case report
Hebert (2018)15 Article Cardiac surgery unit and Hospital infections (VAP) Time series
ICU
Hendrickson (2013)30 Abstract Whole hospital Hospital infections (VAP, CLABSI, CAUTI, MRSA, Case report
VRE, C. Diff)
Hyman (2017)37 Article Whole hospital Hospital infections (CLABSI, CAUTI, CAP), falls, Case report
VTE
Johnson (2006)13 Article Whole hospital Medication-­related events Case report
Lau (2012)19 Abstract Hospital oncology and GI Delays in biopsy follow-­up Case report
departments
Lo (2014)33 Article Whole hospital Hospital infections (CAUTI) Case report
Mackie (2014)35 Article Whole hospital Pressure ulcers Time series

copyright.
Madison (2013)31 Abstract Whole hospital Hospital infections (CLABSI) Case report
26
Mane (2018) Article Emergency department Delays in CVA diagnosis Case report
Mayfield (2013)16 Abstract ICU, oncology ward Hospital infections (CLABSI, VAP) Case report
Mazzella-­Ebstein (2004)11 Article Hospital wards Pressure ulcers, falls, DVTs Case report
41
Milligan (2015) Article Whole hospital Hypoglycaemic Time series
Mlaver (2017)20 Article Hospital floor Pressure ulcers, hypoglycaemic Case report
Nagelkerk (2014)50 Article Paediatrics ward Hospital deaths Case report
51
Pemberton (2014) Article Dental hospital Wrong-­site surgery, falls, medication errors Case report
Rao (2011)32 Abstract Whole hospital Hospital infections (VAP) Case report
Ratwani (2015)38 Article Whole hospital Falls Case report
34
Riley (2010) Article Whole hospital Hospital infections (MRSA, C. Diff), falls, pressure Case report
ulcers, medication errors
Rioux (2007)28 Article Surgery Surgical site infections Time series
39
Skledar (2013) Article Whole hospital Medication-­related events Case report
Stone (2018)40 Article Whole hospital Medication-­related events Case report
Waitman (2011)21 Article Hospital wards, pharmacy Renal failure Case report

CAUTI, catheter-­associated urinary tract infection; C. Diff, Clostridium difficile; CLABSI, central line-­associated blood stream infection; CVA, cerebrovascular
ccident; DVT, deep vein thrombosis; GI, gastrointestinal; ICU, intensive care unit; MRSA, methicillin-­resistant Staphylococcus aureus; VAP, ventilator-­associated
pneumonia; VRE, vancomycin-­resistant enterococcus infection; VTE, Venous Thromboembolism;

system. Several patient safety dashboards were used in of infection tracked included central line-­related blood
ICUs,12 14–16 hospital wards,11 12 17–22 pharmacies,21 23 emer- stream infections,14 16 29–31 ventilator-­ associated pneu-
gency departments and trauma centres,24–26 and surgical monia, 14 16 29 30 32
catheter-­associated urinary tract infec-
settings.12 27 28 No use of patient safety dashboards was tions,14 29 30 33 methicillin-­resistant Staphylococcus aureus
identified in the ambulatory care setting.
infections,29 30 34 vancomycin-­resistant Enterococcus infec-
Patient safety topics tions30 and Clostridium difficile infections.29 30 Dashboards
The most common use of patient safety dashboards (11 additionally displayed rates of pressure ulcers,11 12 14 20 22 34 35
of 33) was tracking hospital infections (figure 2). Types patient falls11 29 36–38 and medication-­related errors,13 18 23 39 40

4 Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437

 Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
Figure 2 Number of publications identified by dashboard patient safety topic.

followed less commonly by other patient safety topics (See patient safety data and automatically update dashboards.
table 2 for all safety topics and figure 2 for chart of topic Direct impact on culture and staffing levels of patient
frequencies). safety personnel were not described in any of the studies.
However, as described above, several studies implemented
Impact of Dashboard use and level of evidence dashboards as a package with other patient safety-­focused

copyright.
Of all studies identified, 5 used a time series efforts, suggesting changes in culture, infrastructure, and
design15 18 28 35 41 while the remaining 28 used case report staffing likely occurred, but concomitantly with the dash-
designs describing specific implementations of patient board implementation rather in response to it.
safety dashboards without statistical analyses performed.
Of the five time series studies, Coleman et al18 identified a Usability
0.41% decrease in missed doses of medications other than Only two studies used a human factors approach for design
antibiotics (p=0.007); however, it was part of four concur- and evaluation of dashboards. Ratwani and Fong38 described
rent interventions to reduce missed and delayed medica- a development process employing commonly accepted
tion doses, and thus, the specific impact of the dashboard human factors design principles,42 followed by focus groups
was unclear. Similarly, Milligan et al41 reported a reduction with users and a 2-­ week pilot phase to collect usability
in hypoglycaemic rates, Rioux et al28 reported a decrease data and make improvements to the dashboard. Mlaver et
in surgical site infections over a 6-­year period after dash- al20 used a participatory design approach that employed
board implementation, and Mackie et al35 reported a collaboration with users during iterative refinements. Two
reduction in hospital-­acquired pressure ulcers; however, additional studies discussed more limited efforts to obtain
in each case, the dashboard was one aspect of a broader feedback. Dharamshi et al27 performed a limited usability
campaign to reduce the respective patient safety events. analysis with an anonymous survey of dashboard users at
Other studies, including Bakos, Chandraharan, Collier, 6-­months after implementation to understand factors that
Conway, Hebert, Hendrickson and Hyman,15 17 22 24 25 30 37 limited the usability of the dashboard. Stone et al40 itera-
reported a subjective reduction in patient safety events, tively obtained feedback from physician users between
but did not describe a statistical analysis. The remaining dashboard revisions. However, the majority of studies did
publications did not include discussion of the direct or not describe the use of an informatics or human factors
indirect impact of the dashboard on patient safety events. approach that considered usability design principles, user-­
Most publications that evaluated the dashboard focused centred design processes or usability evaluation methods.
instead on sensitivity and specificity of dashboard measures, Thus, there was little evidence of design elements that were
employee satisfaction with the dashboards and reduction in most useful or usable across scenarios or settings.
time required to gather data for the dashboard compared
with previous manual data collection. Another impact of
dashboards described included dissemination of patient DISCUSSION
event data in real time or closer to real time than previ- Our systematic review identified 33 publications discussing
ously possible due to algorithms that monitor electronic the use of dashboards to communicate and visualise

Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437 5

Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
patient safety data. All publications were published since to identify the impact or effect of the dashboard. As with
2004, suggesting increased measurement of patient safety many clinical informatics interventions, there could be
after the 1999 publication of To Err is Human. All publica- numerous social and/or technical factors that may have
tions involved display of patient safety events in the inpa- influenced the reported outcomes beyond the dash-
tient setting, the most common of which were hospital board. Rigorous informatics and human factors design
acquired infections. There may, thus, exist opportunities approaches44–47 are needed to improve the use and impact
for similar efforts in the ambulatory setting (eg, falls, lost of patient safety dashboards. Because intervention devel-
referrals, abnormal test results lost to follow-­up or medi- opment is often time constrained, rapid qualitative assess-
cation prescribing errors). ment approaches or human factors methods involving
Overall, the level of evidence that dashboards directly rapid prototyping,48 49 for example, can be adapted to
or indirectly impact patient safety was limited. Only five meet the shorter timelines needed for rapid cycle quality
of the publications used time series designs with the improvement. This will ensure dashboards are useful
remaining designs comprised of case reports of dashboard and usable and generate much needed evidence about
implementations either alone or as part of broader patient efficiency, effectiveness and satisfaction in various care
safety interventions. No interventional studies were iden- settings.
tified. Most studies reported on accuracy of the measures Our study has several limitations. First, it is subject to a
displayed or survey-­based user satisfaction with the dash- potential reporting bias. While we analysed publications
board, rather than the dashboards’ impact on patient based on the content reported, it is possible that addi-
safety events. Studies that provided data on reductions in tional statistical analyses and usability assessments were
patient safety events either did not report statistical anal- performed that were not reported. Furthermore, there
yses to support the reduction, or more commonly, were is likely to be greater use of patient safety dashboards
part of a broad process improvement effort containing developed as part of routine quality improvement efforts
multiple interventions, making it difficult to tease out within healthcare organisations, but these may not be
which intervention truly impacted safety. While it can be published. Nevertheless, this is an area that is ripe for
argued that the intent of a patient safety dashboard is additional research. Second, there was a significant vari-
to communicate data about the extent of safety issues at ability in how dashboards were described, ranging from
an organisation and support other improvement efforts, basic text descriptions to full-­ colour screenshots. This
the act of showing data via a dashboard may alone have variability made performing standardised usability assess-

copyright.
an impact of motivating quality and safety efforts. Dash- ments impossible. Finally, our search was limited to the
boards likely have impacts on safety culture and indirectly publications present in the databases we searched. While
lead to allocation of resources to reducing patient safety we used three different databases to mitigate this impact,
events. The studies identified did not describe these if publications did not appear in any of our search data-
impacts in response to dashboard implementation, and bases, they would have been missed.
thus, this topic warrants future exploration. In conclusion, we identified a growing use of patient
Most publications described dashboard development safety dashboards, largely focused on displaying inpatient
as a quality improvement approach to addressing a safety events. Due to limited use of informatics and human
specific organisational problem or to meet institutional factors-­based approaches during development or postim-
or national standards. Several studies reported high user plementation evaluation, the usability of such dashboards
satisfaction with the dashboard, though these were often was difficult to assess. Furthermore, because of limited
limited assessments and did not capture whether users evaluation of the impact of dashboards and because dash-
fully understood the content of the dashboard. With four boards were often implemented as part of a variety of
exceptions, studies lacked informatics or human factors process improvement efforts, the literature is not clear on
design approaches during development, application of direct impact of dashboard implementation on patient
standardised design principles and use of usability evalua- safety events. Because well-­ designed dashboards have
tions. Without informatics, human factors or user-­centred potential to support patient safety monitoring, our study
design approaches, information requirements from should encourage integration of informatics and human
users may not be well understood. Thus, there is limited factors principles into design and usability evaluation of
evidence about the dashboard acceptance, frequency of dashboards as well as assessment of their direct or indi-
use or whether dashboards satisfactorily met the needs rect impact on patient safety.
of intended users. For example, a common mention
was use of colour coding following a traffic light scheme Twitter Dean F Sittig @DeanSittig and Hardeep Singh @HardeepSinghMD
(red=poor status, yellow=warning, green=good status), Contributors DRM, DS and HS developed idea for this systematic review. DRM
without a formal evaluation of the usability for the 8% and TS performed the literature search. DRM, TS and AS critically reviewed and
extracted data from the publications identified. All authors contributed to the writing
of men and 0.5% of women in the population with red-­
of the initial manuscript and of revising subsequent versions. All authors had control
green colour blindness.43 over the decision to publish. DRM had access to the full data set and accepts full
Some dashboards were implemented within a bundle responsibility for the finished article.
of other interventions. The lack of dashboard usability Funding This project was funded by an Agency for Healthcare Research and
testing before and after implementation made it difficult Quality Mentored Career Development Award (K08-­HS022901) and partially

6 Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437

 Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
funded by the Houston VA HSR&D Center for Innovations in Quality, Effectiveness 16 Mayfield J, Wood H, Russo AJ, et al. Facility level dashboard utilized
and Safety (CIN 13-­413). HS is additionally supported by the VA Health Services to decrease infection preventionists time disseminating data. Am J
Research and Development Service (IIR17-­127; Presidential Early Career Award for Infect Control 2013;41:S54.
Scientists and Engineers USA 14-­274), the VA National Center for Patient Safety, the 17 Chandraharan E. Clinical dashboards: do they actually work in
practice? Three-­year experience with the maternity Dashboard. Clin
Agency for Health Care Research and Quality (R01HS27363), and the Gordon and Risk 2010;16:176–82.
Betty Moore Foundation (GBMF 5498 and GBMF 8838). AS is additionally supported 18 Coleman JJ, Hodson J, Brooks HL, et al. Missed medication
by the VA HSR&D Center for Health Information and Communication (CIN 13-­416), doses in hospitalised patients: a descriptive account of quality
National Institutes of Health, National Center for Advancing Translational Sciences, improvement measures and time series analysis. Int J Qual Health
and Clinical and Translational Sciences Award (KL2TR002530 and UL1TR002529). Care 2013;25:564–72.
There are no conflicts of interest for any authors. 19 Lau S, Wehbi M, Varma V. Electronic tool for tracking patient care
after positive colon biopsy. Am J Clin Pathol 2012;138:A122.
Disclaimer These funding sources had no role in the design and conduct of 20 Mlaver E, Schnipper JL, Boxer RB, et al. User-­Centered collaborative
the study; collection, management, analysis, and interpretation of the data; and design and development of an inpatient safety dashboard. Jt Comm
preparation, review, or approval of the manuscript. J Qual Patient Saf 2017;43:676–85.
21 Waitman LR, Phillips IE, McCoy AB, et al. Adopting real-­time
Competing interests None declared. surveillance dashboards as a component of an enterprisewide
Patient consent for publication Not applicable. medication safety strategy. Jt Comm J Qual Patient Saf
2011;37:326–AP4.
Provenance and peer review Commissioned; externally peer reviewed. 22 Collier M. Pressure Ulcer Incidence: The Development and Benefits
Data availability statement Data sharing not applicable as no datasets generated of 10 Year’s-­experience with an Electronic Monitoring Tool (PUNT) in
a UK Hospital Trust. EWMA J 2015;15:15–20.
and/or analysed for this study. Not applicable.
23 Fong A, Harriott N, Walters DM, et al. Integrating natural language
Open access This is an open access article distributed in accordance with the processing expertise with patient safety event review committees
Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license, which to improve the analysis of medication events. Int J Med Inform
permits others to distribute, remix, adapt, build upon this work non-­commercially, 2017;104:120–5.
24 Bakos KK, Zimmermann D, Moriconi D. Implementing the clinical
and license their derivative works on different terms, provided the original work is
Dashboard at VCUHS. NI 2012 2012;2012:11.
properly cited, appropriate credit is given, any changes made indicated, and the use 25 Conway WA, Hawkins S, Jordan J, et al. The Henry Ford health
is non-­commercial. See: http://​creativecommons.​org/​licenses/​by-​nc/​4.​0/. system no harm campaign: a comprehensive model to reduce harm
and save lives. Jt Comm J Qual Patient Saf 2012;38:318–AP1.
ORCID iD 26 Mane KK, Rubenstein KB, Nassery N, et al. Diagnostic performance
Dean F Sittig http://​orcid.​org/​0000-​0001-​5811-​8915 dashboards: tracking diagnostic errors using big data. BMJ Qual Saf
2018;27:567–70.
27 Dharamshi R, Hillman T, Shaw R. Increasing engagement with clinical
outcome data. Br J Healthc Manag 2011;17:585–9.
28 Rioux C, Grandbastien B, Astagneau P. Impact of a six-­year control
REFERENCES programme on surgical site infections in France: results of the
1 Kohn LT, Corrigan JM, Donaldson MS, Committee on Quality of INCISO surveillance. J Hosp Infect 2007;66:217–23.

copyright.
Health Care in America, Institute of Medicine. To err is human: 29 Frazier JA, Williams B. Successful implementation and evolution of
building a safer health system. The National Academies Press, 2000. Unit-­Based nursing Dashboards. Nurse Leader 2012;10:44–6.
2 Quality payment program.United States Centers for Medicare & 30 Hendrickson C, Guelcher A, Guspiel AM, et al. Results of increasing
Medicaid Services. Available: https://​qpp.​cms.​gov/ [Accessed the frequency of healthcare associated infections (HAI) data reported
6/6/2021]. to managers with an embedded quality improvement process. Am J
3 Hugo JV, S SG. Human factors principles in information Dashboard Infect Control 2013;41:S114.
design. Idaho Falls, ID (United States: Idaho National Lab. (INL), 31 Madison AS, Johnson D, Shepard J. Leveraging internal knowledge
2016. to create a central line associated bloodstream infection surveillance
4 Dowding D, Randell R, Gardner P, et al. Dashboards for and reporting tool. Am J Infect Control 2013;41:S56.
improving patient care: review of the literature. Int J Med Inform 32 Rao R. Application of six sigma process to implement the infection
2015;84:87–100. control process and its impact on infection rates in a tertiary health
5 Sarikaya A, Correll M, Bartram L, et al. What do we talk about care centre. BMC Proc 2011;5.
when we talk about Dashboards? IEEE Trans Vis Comput Graph 33 Lo Y-­S, Lee W-­S, Chen G-­B, et al. Improving the work efficiency of
2019;25:682–92. healthcare-­associated infection surveillance using electronic medical
6 Dowding D, Merrill JA. The development of Heuristics for evaluation records. Comput Methods Programs Biomed 2014;117:351–9.
of Dashboard Visualizations. Appl Clin Inform 2018;9:511–8. 34 Riley S, Cheema K. Quality Observatories: using information
7 Carayon P, Hoonakker P. Human factors and usability for health to create a culture of measurement for improvement. Clin Risk
information technology: old and new challenges. Yearb Med Inform 2010;16:93–7.
2019;28:071–7. 35 Mackie S, Baldie D, McKenna E, et al. Using quality improvement
8 Norman DA, Draper SW. User centered system design: new science to reduce the risk of pressure ulcer occurrence – a case
perspectives on Human-­Computer interaction. 1st ed. CRC Press, study in NHS Tayside. Clin Risk 2014;20:134–43.
1986. 36 Gardner LA, Bray PJ, Finley E, et al. Standardizing falls reporting:
9 20282-­2 usability of consumer products and products for public use, using data from adverse event reporting to drive quality
2013International Organization of Standards. Available: https://www.​ improvement. J Patient Saf 2019;15:135–42.
iso.​org/​obp/​ui/#​iso:​std:​iso:​ts:​20282:-​2:​ed-​2:​v1:​en 37 Hyman D, Neiman J, Rannie M, et al. Innovative use of the
10 PSNet topics, 2021. Available: https://​psnet.​ahrq.​gov/​topics-0 electronic health record to support harm reduction efforts. Pediatrics
[Accessed 6/6/2021]. 2017;139:e20153410–e10.
11 Mazzella-­Ebstein AM, Saddul R. Web-­Based nurse executive 38 Ratwani RM, Fong A. 'Connecting the dots': Leveraging visual
dashboard. J Nurs Care Qual 2004;19:307–15. analytics to make sense of patient safety event reports. J Am Med
12 Donaldson N, Brown DS, Aydin CE, et al. Leveraging nurse-­related Inform Assoc 2015;22:312–7.
dashboard benchmarks to expedite performance improvement and 39 Skledar SJ, Niccolai CS, Schilling D, et al. Quality-­Improvement
document excellence. J Nurs Adm 2005;35:163–72. analytics for intravenous infusion pumps. Am J Health Syst Pharm
13 Johnson K, Hallsey D, Meredith RL, et al. A nurse-­driven system 2013;70:680–6.
for improving patient quality outcomes. J Nurs Care Qual 40 Stone AB, Jones MR, Rao N, et al. A Dashboard for monitoring
2006;21:168–75. Opioid-­Related adverse drug events following surgery using a
14 Anand V, Cave D, McCrady H, et al. The development of a congenital national administrative database. Am J Med Qual 2019;34:45-­52.
heart programme quality dashboard to promote transparent reporting 41 Milligan PE, Bocox MC, Pratt E, et al. Multifaceted approach to
of outcomes. Cardiol Young 2015;25:1579–83. reducing occurrence of severe hypoglycemia in a large healthcare
15 Hebert C, Flaherty J, Smyer J, et al. Development and validation of system. Am J Health Syst Pharm 2015;72:1631–41.
an automated ventilator-­associated event electronic surveillance 42 Shneiderman B. The eyes have it: a task by data type taxonomy for
system: a report of a successful implementation. Am J Infect Control information Visualizations. Proceedings 1996 IEEE Symposium on
2018;46:316–21. Visual Languages 1996:336–43.

Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437 7

Open access

BMJ Health Care Inform: first published as 10.1136/bmjhci-2021-100437 on 6 October 2021. Downloaded from http://informatics.bmj.com/ on January 2, 2024 by guest. Protected by
43 Roskoski R. Guidelines for preparing color figures for everyone and user experience of connected health systems: a three-­phase
including the colorblind. Pharmacol Res 2017;119:240–1. methodology. JMIR Hum Factors 2017;4:e8.
44 Holden RJ, Carayon P. SEIPS 101 and seven simple SEIPS tools. 48 Wilson J, Rosenberg D. Chapter 39 - Rapid Prototyping for User
BMJ Qual Saf 2021. doi:10.1136/bmjqs-2020-012538. [Epub ahead Interface Design. In: Helander M, ed. Handbook of Human-­Computer
of print: 26 May 2021]. Interaction. North. Holland, 1988: 859–75.
45 Carayon P, Hoonakker P, Hundt AS, et al. Application of human 49 McMullen CK, Ash JS, Sittig DF, et al. Rapid assessment of clinical
factors to improve usability of clinical decision support for diagnostic information systems in the healthcare setting: an efficient method for
decision-­making: a scenario-­based simulation study. BMJ Qual Saf time-­pressed evaluation. Methods Inf Med 2011;50:299–307.
2020;29:329–40. 50 Nagelkerk J, Peterson T, Pawl BL, et al. Patient safety culture
46 Savoy A, Militello LG, Patel H, et al. A cognitive systems engineering transformation in a children's Hospital: an interprofessional
design approach to improve the usability of electronic order forms for approach. J Interprof Care 2014;28:358–64.
medical consultation. J Biomed Inform 2018;85:138–48. 51 Pemberton MN, Ashley MP, Shaw A, et al. Measuring patient
47 Harte R, Glynn L, Rodríguez-­Molinero A, et al. A Human-­Centered safety in a UK dental Hospital: development of a dental clinical
design methodology to enhance the usability, human factors, effectiveness dashboard. Br Dent J 2014;217:375–8.

copyright.

8 Murphy DR, et al. BMJ Health Care Inform 2021;28:e100437. doi:10.1136/bmjhci-2021-100437