Abstract
Objective
We sought to develop and evaluate an electronic health record (EHR) genetic testing tracking system to address the barriers and limitations of existing spreadsheet-based workarounds.
Materials and Methods
We evaluated the spreadsheet-based system using mixed effects logistic regression to identify factors associated with delayed follow up. These factors informed the design of an EHR-integrated genetic testing tracking system. After deployment, we assessed the system in 2 ways. We analyzed EHR access logs and note data to assess patient outcomes and performed semistructured interviews with users to identify impact of the system on work.
Results
We found that patient-reported race was a significant predictor of documented genetic testing follow up, indicating a possible inequity in care. We implemented a CDS system including a patient data capture form and management dashboard to facilitate important care tasks. The system significantly sped review of results and significantly increased documentation of follow-up recommendations. Interviews with key system users identified a range of sociotechnical factors (ie, tools, tasks, collaboration) that contribute to safer and more efficient care.
Discussion
Our new tracking system ended decades of workarounds for identifying and communicating test results and improved clinical workflows. Interview participants related that the system decreased cognitive and time burden which allowed them to focus on direct patient interaction.
Conclusion
By assembling a multidisciplinary team, we designed a novel patient tracking system that improves genetic testing follow up. Similar approaches may be effective in other clinical settings.
Keywords: clinical decision support, genetic testing, transition of care
INTRODUCTION
Genetic testing can unlock the promise of precision medicine, potentially informing prognosis, management, surveillance, and recurrence risk.1 However, to be useful, the testing must be reviewed and interpreted by professionals who can implement care based on the knowledge gained and inform the patient or their family members of its meaning. To facilitate such care, researchers have identified a range of considerations for incorporating genetics into the electronic health record (EHR) including data standards, data sharing, and the return of results process.2–4 While management of genetic testing presents many unique challenges, such as insurance prior authorization, multiple specialized reference labs, and long turn-around times, parallels can also be made to more general test result workflows. Various studies have demonstrated clinician dissatisfaction, inefficiencies, and safety problems in the EHR and in managing a broad range of clinical tests in internal medicine and primary care.5,6
For example, one study found that three-fourths of physicians did not routinely notify patients of normal test results and that up to one-third of physicians did not always notify patients about abnormal test results.7 The same study found that fewer than one-fourth of physicians had a reliable method for identifying patients overdue for follow up of abnormal test results. A recent analysis of malpractice cases by a large insurer showed that about one-quarter of diagnosis-related malpractice cases can be attributed to failures in the follow up system.5 Taken together, these studies suggest that current test result follow up systems do not meet the needs of patients and physicians. When clinicians are not provided tools to support their work, they create their own tools and processes referred to as workarounds. These workarounds can be either paper-based or electronic (eg, spreadsheets or external databases).
While studies do exist on the productivity of genetic counselors,8–11 there appears to be a gap in the literature regarding the specific task of results tracking. From our personal observations and comments of other genetics professionals, the most widely used EHRs lack comprehensive functionality to track genetic test results. Some genetic counselors in our study report that they were instructed in graduate school to use spreadsheets to manage result tracking. These spreadsheets qualify as an electronic workaround, but they have many limitations including being separate from the patient record; being susceptible to security issues, loss or being out of date; and frequently preventing multiple simultaneous users. There have been attempts to design more effective test result systems,12 including systems specifically addressing genetic testing. These systems suffer from many of the same issues listed above for spreadsheet-based workarounds, and often require workflows outside the EHR. In a recent study, Scott and Martin2 implemented a genetic testing data collection system within a popular commercial EHR. This facilitated centralized reporting in their practice; however, the authors did not describe a comprehensive system for management of patients over time.
As an initial area to improve care, we chose to focus on inpatient genetic testing. At the Children’s Hospital of Philadelphia, like many academic pediatric hospitals, the inpatient clinical genetics evaluation workflow involves the primary care team’s attending physician requesting consultation of a geneticist (Figure 1). The geneticist evaluates the patient and makes recommendations for genetic testing. It is the responsibility of the primary care team to determine if the recommendations fit well with the overall treatment plan and then send samples to the hospital or outside reference laboratory for analysis. The laboratory issues a test report which is in turn entered into the patient’s EHR record. Because the genetic testing takes weeks or even months to complete and often includes more than a single laboratory reporting results at different times, the patient may be discharged before any results become available.
Figure 1.
Swimlane diagram of the clinical genetics inpatient consultation workflow. Patients admitted to the hospital may be suspected by their primary inpatient care team of having an underlying genetic etiology for their medical problems. A consultation may be requested. The genetics team (including an attending physician) evaluates the patients and leaves recommendations for the primary care team. These recommendations often include genetic testing, which is then sent to the hospital genomics lab or an outside reference lab. Because genetic testing may take weeks or months, the patient may be discharged prior to testing resulting. Once testing is complete, the test report is entered into the patient’s EHR record. Ideally, the medical geneticist becomes aware of the result and interprets them in the context of the patient. The geneticist informs the patient or their family members and documents their recommendations in the EHR for other care team members.
The clinical genetics team should become aware of the results and interpret them based on the patient’s presentation and family history and document their recommendations in the EHR. Additional challenges may hamper this workflow, including high demand for limited clinical resources, turnover of care teams, mismanaged paper documents, and manual result entry into the EHR. Historically, our clinical genetics team developed paper records or Microsoft Office® documents as workarounds. The development of such artifacts as “workarounds” to the EHR is a practice not uncommon in other medical contexts.13–15 Despite implementing these types of workarounds in our own health system, we found anecdotally that results sometimes go unnoticed (Figure 2).
Figure 2.
Project overview. (A) We became aware of anecdotal reports of results going unnoticed because of multiple overlapping failures (ie, the Swiss cheese model of adverse events) in our practice. (B) To better understand the problem, we performed an analysis of patient characteristics that were associated with the presence or absence of documented follow up recommendations. (C) Guided by this analysis, we gathered initial requirements for a successful system to address these issues. We adopted an agile project development lifecycle and (D) designed the system, (E) built and deployed the system, and (F) initiated a limited clinical beta test. (G) Based on our testing, we identified issues and iterated on the system. (H) We then provided user education and deployed the system to the entire inpatient service. Following implementation, we (I) analyzed patient outcomes and (J) conducted semistructured interviews of users.
Researchers in other health systems have reported attempts at improving these workflows by implementing data collection directly in the EHR, but ultimately users reportedly export the data outside the EHR into spreadsheet programs for further management.2 Our objective was to improve genetic testing result interpretation and documentation of follow up recommendations. Specifically, we (1) sought to understand the factors potentially associated with follow up, (2) developed a clinical decision support (CDS) mechanism using a comprehensive in-EHR tracking system (3), assessed the effectiveness of our intervention using access logs and EHR documentation, and (4) interviewed end users of the system to understand factors key to successful implementation.
MATERIALS AND METHODS
The Institutional Review Board at Children’s Hospital of Philadelphia determined that this study met exemption criteria under 45 CFR 46.104(d)4(iii) as protocol 20-018111. A waiver of HIPAA authorization under 45 CFR 164.512(i)(2)(ii) was granted to access identifiable information from the medical records. Statistical analyses and data visualization were performed with the R statistical programming language. Multivariate mixed-effects logistic regression analyses were performed with the GLMMadaptive package to facilitate efficient calculation of marginal effects conditional on the average patient. Survival analysis was performed using the survminer package.
Setting and participants
This study was performed at the Children’s Hospital of Philadelphia, a pediatric quaternary referral hospital. The Section of Clinical Genetics cares for approximately 5500 children per year with suspected or confirmed genetic disease in the ambulatory setting. Additionally, the section maintains an inpatient consultation service which provides evaluation and management recommendations for approximately 800 children per year. This manuscript focuses on the inpatient consultation workflow. At the time of the study, the section included 10 attending physicians, 11 residents or fellows, and 3 genetic counselors. Clinician participants for system development and user interviews were recruited from the section. Trainees from other divisions such as Child Neurology and Developmental Pediatrics were involved in a small minority of consultations.
Analysis of factors associated with results follow up
To understand factors associated with results follow up, we accessed EHR data for children undergoing inpatient genetic consultation during the 1 year prior to implementation of the tracking system. We collected demographic information about the participants including sex, age at time of consultation, gestational age, self-reported race and ethnicity, and home address census tract. From the hospital admission, we collected information about the care team, genetic testing performed, and documentation of follow up recommendations.
System development and deployment
We examined the various systems employed by our division for patient tracking over the previous 10-year period. We undertook informal interviews of users of these systems both over time and fulfilling various roles. From these interviews, we identified strengths and limitations of these systems to inform the design requirements of a user-centered EHR-based system. We adopted an Agile project development approach (Figure 2). Early designs were reviewed with primary users of the system and modified based on their feedback. During the design process, we identified a number of critical design requirements that could not be addressed by base EHR functionalities. To deal with these, we co-opted functionalities originally developed by our EHR vendor for disparate tasks and combined them in novel ways to create functionality not provided by the standard EHR. To overcome other obstacles, we were forced to program custom low-level functions to enable key functionality.
The initial system was beta tested in a limited clinical environment before deployment to the entire inpatient service. During this process, we identified and addressed issues which included technical, workflow, and usability problems. We next developed and delivered training to end users. After deployment, based on feedback from users, we made rapid iterative changes to the system.
System outcomes
To assess performance before and after implementation of our tracking system, we identified 2 outcome measures: (1) time from result to access by genetic team member, and (2) time to follow up recommendation documented in a clinical note. We collected information about clinical encounters, note documentation, and genetic testing from the EHR. Because the time of result review is not consistently recorded by the EHR, we used detailed access logs—the record of all user actions throughout the system. We first created a list of all members of the genetics team (attending, resident, fellow, and genetic counselors). We next identified log entry types as a proxy which could reasonably indicate that the user reviewed genetic testing results (eg, “results review accessed” or “results table viewed”). Although we cannot be certain that the user actually viewed the genetic test result, we believe this is a reasonable proxy as the genetics team would have little reason to be accessing the patient’s results other than to view the genetic test. Likewise, we would expect that errors should occur in both the control and intervention groups equally.
Post-deployment user interviews
We developed a semistructured interview guide to elicit open-ended responses from users of the tracking tool. To capture information on the broader work domain, interview questions were guided by the sociotechnical model, Safety Engineering in Patient Safety (SEIPS) 2.0.16 We recruited a sample of genetic counselors, residents, fellows, and attending physicians in the inpatient genetics section. Participants provided informed consent and were then interviewed individually using web conferencing software where the session was audio recorded and then machine transcribed. Two study team members deductively analyzed the transcripts using themes based on components of SEIPS 2.0. An initial review of transcripts was performed where results were compared, and consensus reached on thematic analysis.
RESULTS
Factors associated with results follow up
To inform our development of a more successful system, we analyzed the outcomes of 768 inpatient genetics consultations during the 1-year period prior to implementation of the tracking system (Supplementary Figure S1). Fifty-eight percent of patients had no subsequent documentation from a provider or staff member that included clinical genetics. Of those patients, 78% (or 45% of the total), had genetic testing performed after their consultation but still had no follow up documented by the genetics team. It is possible that some of these patients had results reviewed and documented by their primary team. We worried that limitations in our follow up infrastructure perpetuated inequities in the delivery of care to patients with historically marginalized backgrounds. To this end, we performed an analysis of factors associated with documentation of follow up recommendations using a mixed-effects logistic regression (Table 1). Unsurprisingly, we found that genetic testing was strongly significantly associated with documentation of follow up recommendations. We also found that patient reported race was significantly associated with follow up documentation (P = .011, Wald test), with patients reporting non-Hispanic Black background less likely to have recommendations documented (raw 40.4% vs 42.1% for all other groups, adjusted marginal odds ratio 0.82). These analyses validated our concerns about inequities.
Table 1.
Mixed effects logistic regression predictors of follow up documentation
| Predictor | Category | Odds ratio | P value |
|---|---|---|---|
| Any genetic testing | No | Reference | .0008 |
| Yes | 2.01 | ||
| Sex | Female | Reference | .1335 |
| Male | 0.81 | ||
| Hospital length of stay (weeks) | 1.03 | <.0001 | |
| Admission type | Born in hospital | Reference | .0221 |
| Other | 1.43 | ||
| Preferred language | English | Reference | .2736 |
| Other | 0.73 | ||
| Race | Non-Hispanic White | Reference | .0114 |
| Hispanic or Latino | 1.31 | ||
| Non-Hispanic Black | 0.82 | ||
| Other | 1.00 | ||
| Payor | Commercial | Reference | .2154 |
| Government | 0.78 | ||
| Charity | 0.32 | ||
| Observations: 768 | |||
System design
A review of our workflows revealed that, for over a decade, our division relied on tables contained within Microsoft Word® (the word processing software) documents housed on network share drives to keep track of inpatient consults. To review the patient test results, the system involved manually copying information out of the EHR. This was a process that was extremely time consuming, caused conflicts with multiple users attempting to access the document simultaneously, and was not ideal from a security perspective. We anecdotally heard patient stories of results going unnoticed and not being returned to families, that were only discovered when the family or other care provider followed up, often months later. As an initial attempt at addressing these issues, we created a database in REDCap to track patients we had previously evaluated.17 However, we found that it was not consistently used by providers and suffered many of the same inefficiencies of the word processing documents (particularly the need to duplicate information).
Building upon our previous attempts and statistical analysis, we developed a tracking system with 2 main components. Part 1 is a data capture instrument which is available to clinicians in note templates and from patient lists (Figure 3). The capture form is optimized to reduce double documentation and provides customized suggestions but gives the clinician an opportunity to alter the parameters based on clinical judgment. Part 2 is a management dashboard which gives clinicians an overview of patients with outstanding tasks, provides easy access to information required to make decisions, and facilitates communication with the primary team and the patient’s family (Figure 4). One key CDS tool is the inclusion of icons to clearly delineate the patient’s status in the diagnostic process. These include “testing recommended but not yet in process,” “testing in process,” and “results to return.” The system automatically displays “overdue” icons based on standard estimates of test turnaround times. The user can easily sort or filter the list to view only their patients or those with overdue tasks to follow up.
Figure 3.
Data capture instrument for the in-EHR tracking system. We created a data capture form based on our EHR vendor’s standard functionality (a SmartForm™). The system automatically enters data by pulling it from elsewhere in the system. For example, “Inpatient Consult” is automatically selected if the patient is currently admitted. However, clinicians may override results. Clinicians are then asked to fill in information which cannot be readily ascertained from other sources. The data is stored in discrete fields for later use in reporting and documentation.
Figure 4.
Management dashboard for the in-EHR tracking system. The dashboard is implemented using the vendor’s standard functionality (a Reporting Workbench™ Report). The top half of the management dashboard is a patient list which provides a high-level overview of the patient’s status within their genetic diagnostic odyssey. The standard functionality provides powerful filtering options to allow the clinician to, for example, only view their own patients by default. Icons provide key information about patient status. The bottom half of the dashboard provides additional details about the selected patient, including their genetic test results and recent communications with the patient’s family. There is also access to the data entry form (Figure 3) to document changes in patient status.
As part of deployment of the system, we delivered user education through a virtual meeting demonstrating the expected workflow and allowed users to ask questions. The development team also met individually with the users to answer questions and to receive feedback on barriers to work and system deficiencies. We created a “Job Aid” document which details use of the system as well as provides technical details about how the system actually functions.
System outcomes
Uptake of the tracking system by the user base was swift. In the first 6 months, 333 of 373 potentially eligible inpatient consults (89.3%) were entered into the system (Supplementary Figure S2). The system facilitated multiple diagnoses that would have otherwise been missed, including a case of 22q11.2 deletion syndrome because genetic testing was inadvertently not sent by the primary management team. Considering patients evaluated in the first 6 months of deployment of the system, 171 of 275 (62.2%) inpatient consults who had genetic testing and were tracked by the system had follow up documentation within 60 days (Figure 5). This is a significantly greater proportion than those not tracked by the system over the previous year, 276 of 530 (52.1%, P < .005, 2-sided Fisher’s exact test). Although documentation rates differed among individual attending physicians, use of the tracking system was associated with increased follow up documentation by all but 1 provider. However, due to the small number of consults seen by the single provider with decreased rates (n = 8), the decrease is potentially due to randomness.
Figure 5.
Changes in documentation of follow-up recommendations following system implementation. The left panel shows the percentage of patients who underwent genetic testing who had subsequent documentation of recommendations by genetics in the 60 days immediately following the results becoming available. The bar graph compares patients where the tracking system was not used (orange) to those where it was used (blue) in the 12 months preceding and 6 months following the implementation of the CDS tracking system. The numbers of patients were compared with a Fisher’s exact test. The right panel shows the same data but for each of the 8 consulting attending physicians who evaluated patients during both periods. Consulting attending number 4 (green dots and line) who decreased documented follow up percentage consulted on only 8 patients total, indicating possible stochasticity rather than an effect of the system itself.
Using EHR access log data, we found that review of genetic results was completed by a clinical genetics provider within 1 month for a great majority of patients regardless of tracking system utilization (Figure 6). However, the system helped facilitate review of small subset of patients that may otherwise have gone unnoticed (P < .001, Kolmogorov-Smirnov test). Indeed, all results were reviewed within 3 months (max 73 days) with the tracking system, whereas 8.2% of results remain unaccessed by that time without the system.
Figure 6.
Survival analysis of time until genetic testing result review. The top panel shows the percentage of patients whose EHR lab results sections were reviewed by a genetics team member (genetic counselor, genetics resident, or genetics attending physician) over time for patients who were not tracked by the system (orange) compared to those who were tracked by the system (blue). Crosses indicate patients that were censored because their genetic testing had only been resulted for that long at the time of analysis. The bottom panel indicates the number of patients at risk for not having their results reviewed.
Post-deployment user interviews
We interviewed a sample of 6 clinicians from the genetics department: (3) genetic counselors, (1) genetics resident physician, and (2) attending physicians. Interview transcripts were deductively analyzed using components of the SEIPS model (Table 2).16 The SEIPS model is a framework for evaluating and facilitating interactions between technical, social, and organizational factors that influence healthcare system performance and outcomes. Genetic counselors reported being the primary users of the system, managing the results tracking of all patients and responding to any changes, issues, or results with individual patients and coordinating a response with members of the clinical team. Physicians reported being secondary users whose work is typically impacted indirectly by the system, though they also interact with the system directly to review their patients or assess practice patterns.
Table 2.
SEIPS 2.0 components, themes identified in our interviews and illustrative quotes
| SEIPS component | Concept | Example user quote |
|---|---|---|
| Persons |
|
|
| Tools and technology |
|
|
| Tasks |
|
|
| Organization |
|
“One of the columns in [the previous workaround was] a little checkbox for every task that we recommended for the patient, and if we were to look at those, we’d see that like 50% at least of the things don’t have an X through them because there was no bandwidth to go back and check literally hundreds of patients.” |
| External environment |
|
“I would say it takes 5–7 messages per patient to get an insurance authorization.” |
| Adaptation |
|
“[I use it to] look back at what other people have done for similar problems … if they had a similar presenting problem and I was kind of wavering on … what to do, it can help guide management for current patients based on what previous attendings have done.” |
| Professional work |
|
|
| Collaborative work |
|
“I cannot tell you how many saliva kits are out there sitting on people’s kitchen tables under a pile of mail that have never been returned.” |
| Outcomes |
|
“Since I started using [the system], I have felt a sense of safety … and sleeping without waking up in the middle of the night thinking about little Johnny and if his test result is back.” |
In discussing the previous system, participants had myriad comments on the barriers and limitations of spreadsheet-based workarounds. Genetic counselor participants described how their graduate school training (at different universities) included the use of spreadsheets to track results and how spreadsheet use was widespread in their clinical experience in other organizations. The participants described the spreadsheet-based systems as highly manual and prone to missing and incorrect data. They noted that spreadsheets were difficult to share, version track, and update. They also described that they imposed cumbersome workloads including double documentation tasks, stress, and other inefficiencies. Perhaps most importantly, they reported it created a single point of failure and patient safety hazard. Overall, genetic counselor participants described the previous system as a source of burnout and job dissatisfaction, taking time and attention away from important patient work. One participant noted, “I think it’s why genetic counselors are so frequently going to industry, because [in clinical practice there are many] patients that need things and that you’re worried they’re going to slip through the cracks because there is no way to track them.”
Genetic counselor participants described the new system as having a significant impact on their work by improving efficiency, confidence, and trust in the information. They described it improving communication of information and collaboration leading to distributed responsibility among the clinical team to improve safety. They also mentioned that it improves job and professional satisfaction by freeing up time and effort for more meaningful and impactful patient work. Participants provided many examples of how the tracking tool facilitated communication and collaboration including between genetic counselors and across the clinical genetics team. Moreover, it also facilitated communication with the rest of the patient’s care team outside genetics, such as the inpatient treatment team or their primary care pediatrician.
Both genetic counselor and physician participants described several unanticipated positive adaptations of the tracking system. Genetic counselors described how the centralized sortable and filterable display of all patients with testing allowed them to go beyond managing results and to anticipate and plan work for the upcoming week. A resident physician participant described using the system as an informal teaching tool such that searching for patients with a similar diagnosis could inform possible evaluation courses for other patients. An attending physician participant described how they used the system to understand their own practice patterns.
DISCUSSION
Overall, we have implemented an in-EHR tracking system to serve as a CDS tool that facilitates genetic testing and follow up, improving both patient care and patient safety. The tool was quickly adopted by the clinical team. Our data suggests that the ability to easily see an overview of outstanding care tasks may expedite review of results and documentation of interpretation and recommendations. Interviews with users of the system also revealed some surprising results. First, we learned from multiple participants that the creation of workarounds external to the EHR such as Microsoft Excel® was recommended at genetic counseling training programs. Thus, it seems likely that such workarounds are in use across health systems, although additional study is needed. Second, participants realized the limitations of the previous workflow and how that affected their ability to provide proper patient care prior to the implementation of the system. The potential existence of patients with unreviewed genetic testing that might significantly influence their care was a major concern. Participants expressed appreciation that the system acts as a safety net to help prevent patients from slipping through the cracks. Some felt that this functionality of the system may help with burnout. This may become a more critical issue to address if genetic testing continues to become a larger component of healthcare.
In addition to improving the care of individual patients, the structured data captured by our tracking system can also facilitate quality improvement and research going forward. While the system includes fields for capturing a diagnosis and gene, this simple functionality fails to capture the complexity of genetic variants or multiple diagnoses. We felt this nuance was outside the scope of our system, particularly when the vendor continues development of a genomics module. Our health system began implementation of the vendor’s standard genomics workflows shortly after the tracking system was launched. We hope to integrate the genomics functionality into our tracking system in the near future.
While our analysis and interviews suggest that our tracking system can improve care, our approach required significant investment. First, implementation of the system required many hundreds of person-hours across a highly invested multidisciplinary team. We assembled key stakeholders from genetic counseling, clinical genetics, clinical informatics, and information services. Second, while most of the key features of our system were built using the vendor’s standard functionality, some of the time-saving features required highly complicated build or creation of custom code. This required inclusion of team members with substantial technical expertise and a robust system for documentation and testing. These technical dependencies may also complicate long-term maintenance of the tracking system as the vendor makes changes to the EHR and individual team members with technical knowledge leave the institution. Despite these issues which may hamper development in a smaller hospital setting, we are making this tool available to the broader community for those who have the expertise to deploy it. Other customers of the EHR vendor can access some components through the vendor’s user-created content repository (Community Library) or contact the authors for assistance in packing the build for transfer (using Turbo Charger™).
Moreover, our study was conducted in a single health system which may limit generalizability of results. Only a small number of genetic counselors and physicians were available for interview so their perspectives may not be reflective of individuals in these roles at other health systems. The improvement in follow up we observed in our pre-post evaluation could be susceptible to secular trends, but there were no other concurrent efforts to improve genetic test result follow up to suggest such trends may have affected our results.
Our initial review demonstrated health inequity. Specifically, patients reporting non-Hispanic black background were less likely to have recommendations documented in the EHR. Due to the small number of patients, we lacked power to determine if our system was able to address inequities in the delivery of genetic care that we discovered. Understanding if our intervention improved inequity and elucidating the underlying structural causes of both higher and lower rates of follow-up documentation will be the focus of continued research.
Although we studied and addressed a specific clinical genetics and genetic testing workflow in 1 academic hospital, we suspect that the approach is highly relevant to other clinical settings. We have heard from clinicians at more than 10 other children’s hospitals that they also use Excel spreadsheets to track their patients (personal communication). We have also received feedback of workarounds in other pediatric specialties as well as in our previous study of workarounds used to facilitate palivizumab dosing in former premature infants.18 While our tracking system is highly customized to particularities of the clinical genetic and genetic testing workflows, we suspect that a slightly modified approach could be used to track patients undergoing other evaluations requiring multiple tests to track often with longer follow up times, for example, neurology or rheumatology evaluation. Furthermore, our approach can be applied to multidisciplinary hospital teams, for example, vascular anomaly programs, to facilitate the nurse coordinator workflow of tracking multiple tests among specialties.
In conclusion, we identified results review and follow up as a gap in genetics care that persisted despite extra-EHR workarounds including spreadsheets and custom databases. We assembled a multidisciplinary team and created a comprehensive in-EHR tracking system optimized to limit double documentation and to improve efficiency by providing rapid access to needed patient information. We found uptake of the system was swift and facilitated significantly improved review and return of patient results with the perception of marked improvement in patient care. Additional work will be required to ease implementation at other clinical sites, to apply this system to other programs, and to understand if this work can address health inequity.
Supplementary Material
ACKNOWLEDGMENTS
We thank the interview study participants for their willingness to speak openly about their experiences and use of the system.
Contributor Information
Ian M Campbell, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Division of Clinical Genetics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Division of General Pediatrics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA; Center for Applied Genomics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Dean J Karavite, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Morgan L Mcmanus, Division of Clinical Genetics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Fred C Cusick, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
David C Junod, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Sarah E Sheppard, Division of Clinical Genetics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Eli M Lourie, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Division of General Pediatrics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.
Eric D Shelov, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Hakon Hakonarson, Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA; Center for Applied Genomics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Anthony A Luberti, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Naveen Muthu, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
Robert W Grundmeier, Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Division of General Pediatrics, Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania, USA; Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.
FUNDING
This study received no funding.
AUTHOR CONTRIBUTIONS
IMC conceptualized the study, performed software development, performed formal analysis, performed investigation, curated data, performed project management, performed visualization, and wrote the original draft of the manuscript. DJK conceptualized the study, performed formal analysis, performed investigation, curated data, and wrote the original draft of the manuscript. MLM and SES conceptualized the study, performed validation, and critically reviewed the manuscript. FCC and DCJ performed software development and critically reviewed the manuscript. EML, EDS, HH, and AAL provided oversight and leadership, performed project administration, and critically reviewed the manuscript. NM and RWG conceptualized the study, performed investigation, provided oversight and leadership, performed project management, and critically reviewed the manuscript.
SUPPLEMENTARY MATERIAL
Supplementary material is available at Journal of the American Medical Informatics Association online.
CONFLICT OF INTEREST STATEMENT
The authors have no competing interests.
DATA AVAILABILITY
The data underlying this article, other than interview transcripts, will be shared on reasonable request to the corresponding author. Interview transcripts cannot be shared due to participant privacy.
REFERENCES
- 1. Ross LF, Saal HM, David KL, et al. ; American College of Medical Genetics and Genomics and the American Academy of Pediatrics. Technical report: ethical and policy issues in genetic testing and screening of children. Genet Med 2013; 15 (3): 234–45. [DOI] [PubMed] [Google Scholar]
- 2. Scott A, Martin DM.. Development and implementation of an electronic medical record module to track genetic testing results. Genet Med 2021; 23 (5): 972–5. [DOI] [PubMed] [Google Scholar]
- 3. Nestor JG, Fedotov A, Fasel D, et al. An electronic health record (EHR) log analysis shows limited clinician engagement with unsolicited genetic test results. JAMIA Open 2021; 4 (1): ooab014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Crump JK, Del Fiol G, Williams MS, et al. Prototype of a standards-based EHR and genetic test reporting tool coupled with HL7-compliant infobuttons. AMIA Jt Summits Transl Sci Proc 2018; 2018: 330–9. [PMC free article] [PubMed] [Google Scholar]
- 5. Poon EG, Gandhi TK, Sequist TD, et al. “I wish i had seen this test result earlier!”: dissatisfaction with test result management systems in primary care. Arch Intern Med 2004; 164 (20): 2223–8. [DOI] [PubMed] [Google Scholar]
- 6. Hickner JM, Fernald DH, Harris DM, et al. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf 2005; 31 (2): 81–9. [DOI] [PubMed] [Google Scholar]
- 7. Boohaker EA, Ward RE, Uman JE, et al. Patient notification and follow-up of abnormal test results: a physician survey. Arch Intern Med 1996; 156 (3): 327–31. [PubMed] [Google Scholar]
- 8. Patel D, Blouch EL, Rodgers-Fouché LH, et al. Finding a balance: reconciling the needs of the institution, patient, and genetic counselor for optimal resource utilization. J Genet Counsel 2018; 27 (6): 1318–27. [DOI] [PubMed] [Google Scholar]
- 9. Kieke MC, Conta JH, Riley JD, et al. The current landscape of genetic test stewardship: a multi‐center prospective study. J Genet Couns 2021; 30 (4): 1203–10. [DOI] [PubMed] [Google Scholar]
- 10. Cutting EM, Overby CL, Banchero M, et al. Using workflow modeling to identify areas to improve genetic test processes in the University of Maryland Translational Pharmacogenomics Project. AMIA Annu Symp Proc 2015; 2015: 9. [PMC free article] [PubMed] [Google Scholar]
- 11. Attard CA, Carmany EP, Trepanier AM.. Genetic counselor workflow study: the times are they a-changin’? J Genet Couns 2019; 28 (1): 130–40. [DOI] [PubMed] [Google Scholar]
- 12. Poon EG, Wang SJ, Gandhi TK, et al. Design and implementation of a comprehensive outpatient results manager. J Biomed Inform 2003; 36 (1–2): 80–91. [DOI] [PubMed] [Google Scholar]
- 13. Perry SJ, Wears RL.. Underground adaptations: case studies from health care. Cogn Tech Work 2012; 14 (3): 253–60. [Google Scholar]
- 14. Mount-Campbell AF, Evans KD, Woods DD, et al. Value and usage of a workaround artifact: a cognitive work analysis of “brains” use by hospital nurses. J Cogn Eng Decis Mak 2019; 13 (2): 67–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Patterson ES. Workarounds to intended use of health information technology: a narrative review of the human factors engineering literature. Hum Factors 2018; 60 (3): 281–92. [DOI] [PubMed] [Google Scholar]
- 16. Holden RJ, Carayon P, Gurses AP, et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11): 1669–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (2): 377–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Utidjian LH, Hogan A, Michel J, et al. Clinical decision support and palivizumab. Appl Clin Inform 2015; 6 (4): 769–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data underlying this article, other than interview transcripts, will be shared on reasonable request to the corresponding author. Interview transcripts cannot be shared due to participant privacy.