Errors in the delivery of medical care in developed nations have been the focus of intense research and innovation over the last 2 decades,[1–3] largely due to the attention brought to this issue following the publication of a report from the Institutes of Medicine. The report entitled “To Err is Human: building a better health system” highlighted the complexity of our health care system and the potential for harm to occur. This report brought to light the fact that a staggering number of patients were harmed, sometimes even resulting in death, while being cared for in hospitals.
Unfortunately, despite the publication of this report and subsequent attempts to improve patient care, quality, and safety, the problem of morbidity and mortality attributable to health care delivery remains significant. A more recent published review concludes that approximately 250,000 deaths per year in the United States alone could be attributed at least in part to the direct act of being cared for and not from injury or illness. This death rate is a staggering number, and both the lay and medical communities should be very concerned in the face of continued harm from resulting from in hospital care.
There has been extensive research into determining which particular aspects of the health care delivery process are most prone to error and are therefore inherently more dangerous than others. These specific aspects of the health care delivery process have been investigated as targets for quality improvement projects to render the process safer for patients.[2,5,6] One such area particularly prone to error is that of communication. Within care delivery communication, one of the most dangerous moments is that of the hand-off. A hand-off occurs in medicine when a person, either a nurse caring for the patient at the bedside or the physician responsible for treating the patient, passes that responsibility to another health care provider by way of a formal or informal report, such as at the beginning or end of their shift. Historically the format in which these hand-offs occur has varied. Some take place via telephone, others via transmitted written documents, others in-person with or without a whole team present. It has been noted that there is great variability in the content, attention to detail, and the overall setting of hand-offs.[8–10] Within our hospital, a hand-off would be between resident physician teams, when 1 team comes on duty, and 1 goes off duty. With an increased focus in the last 20 years on fatigue and educational requirements for resident physician trainees, there have been regulatory interventions to limit continuous hours that trainee physicians are on duty,[11–13] also resulting in a decrease in the number of hours an individual physician (or team of physicians) can spend on duty. This decrease in hours has led in turn to an increased number of hand-offs in some settings. Given that hand-offs are common, error-prone, and an identified area of danger in patient care, there have been studies aimed at improving the quality of such hand-offs in an attempt to decrease medical errors.
Starmer and colleagues found in 2013 that implementation of a standardized hand-off process and a training bundle, resulted in decreased medical errors at a major children's hospital, as well as unexpected benefits, such as more time spent at the bedside with patients. Implementation of standardized hand-offs and training session as an intervention was subsequently replicated in further studies from the same group on a multi-site/multi-institutional basis. This standardized process is known as I-PASS (Illness Severity, Patient Summary, Action List, Situational Awareness, Synthesis by receiver). This process has now been widely implemented within various pediatric residency teaching systems. However, there has been significantly less experience with I-PASS type hand-off bundles within the non-pediatric residency training environment. The lack of research outside the pediatric residency training environment impacts generalizability to other non-pediatrics general medicine units as the degree of co-morbidity and case complexity increases dramatically on general medicine units. The Family Medicine environment is of particular interest because these units care for not only a broad variety of medical patients, but also care for the greatest spread of patients in terms of age. This patient population makes Family Medicine an ideal setting for studying the generalizability of I-PASS interventions.
The current study aimed to determine if a similar reduction in medical errors would be observed in the Family Medicine Residency program of an academic teaching hospital with a more varied sample of patients.
Following approval by the Institutional Review Board of The Guthrie Clinic, we conducted a retrospective review of all admissions to the Family Medicine teaching service (FMTS) which occurred between February 2016 and February 2017. All changes to the resident hand-off process and associated training bundle was completed in August of 2016. The Robert Packer Hospital is a 254-bed community tertiary care teaching hospital which has several post-graduate physician training programs accredited by the Accreditation Council on Graduate Medical Education. The FMTS is a general medicine service operated by the Family Medicine Residency at the Robert Packer Hospital. The service is comprised of 3 post-graduate year (PGY) trainees with 1 trainee from each year; PGY-1, PGY-2, and PGY-3. The service is PGY-led and supervised by the academic teaching physician faculty from the Family Medicine Residency. The FMTS admits patients of all ages and disease states unless they require a level of care which is not available at this institution. The FMTS does not care for patients within the intensive care unit and is staffed by both a day and a night team. The day team works 11 hours followed by a hand-off to the night team, which then works 13 hours overnight. The work schedule did not change between the pre- and post-intervention periods.
2.1 Pre-intervention period
In the pre-intervention period, hand-offs were completed using an ad-hoc tool to pass information between teams. There was no standardized format, nor was there any specific training on how to complete a hand-off. Some elements of a standardized hand-off process were already in place. All team members had to be present for hand-off unless an emergency had called them away, and all hand-offs had to be in person. The pre-existing hand-off tool was a computer written document. In the pre-intervention period, the hand-off location was variable, although mostly occurred in the Family Medicine Residency office area. However, it also occurred at times on the Labor and Delivery unit, or a medical floor in a conference room. The process for hand-offs before this study had remained the same for approximately the preceding 10 years.
The intervention was implemented during August of 2016. The intervention used was modeled on the intervention in the original I-PASS study. Firstly, the hand-off tool was changed to incorporate the structure from the I-PASS study group mentioned previously. The tool was designed and implemented in the electronic medical record (EMR), which was viewable by all members of the team on their individual computers. This tool was updated by the team, which was about to sign out, to allow incoming team members to view the chart and discuss the case while performing the sign-out process. The tool allowed a team member to view the current sign out, edit the sign-out or view previously edited sign-outs. A strict no-interruption rule was followed during hand-off, and a poster campaign was used to remind facility nursing staff to not interrupt the resident team during this critical time in which sign-out was occurring.
As laid out in the original and subsequent I-PASS literature, the benefit was greatest when a bundled approach was used. We implemented 2 mandatory learning sessions on critical components of hand-offs and how to avoid medical errors involved in such processes. All residents in the program were required to attend these learning sessions and to use the new hand-off method. The learning sessions were led by patient safety staff and institutional leadership and consisted of a total of 2 hours of instruction (1 hour per session). This training was a group lecture which all residents attended at the same time. Any resident who was not able to attend the session due to clinical or personal responsibilities were mandated to view a recording of the lecture they had missed.
Additional components of the I-PASS study were also implemented, including allowing the residents to review the new hand-off tool before utilization. Faculty observation of, and participation in, the hand-off process was also encouraged. Before the intervention period, this was not a common occurrence (See Appendix A for a comparison of the hand-off process before and after the intervention occurred, http://links.lww.com/MD/D269).
At the time of the intervention, there was no intention to research the intervention, and no resident physician trainees were aware that their hand-off behavior would be studied.
2.3 Main outcome measures
To be consistent with the original work by Starmer, we applied the same definitions that were used in the original paper on medical errors. They defined a medical error as a failure of a preventable nature in the process of care delivery itself. Adverse events were defined as those events which were either preventable or not preventable, which occurred in an unintended manner, and as a consequence of medical care itself and that has resulted in patient harm. While the I-PASS study, and previous work by that study group, prospectively studied charts in real-time using multiple dedicated research nurse staff, the current project was not resourced to that level. This type of active surveillance method has been well documented in the literature.[17–19] We did, however, make use of a novel surveillance strategy for detection of medical errors. As our institution has an integrated EMR across all care areas (in-patient, out-patient, and surgical all use the same EMR), recording of all medical errors takes place within an online error reporting system. This system is accessible to all clinical and non-clinical members of the care team and serves as a repository for all patient safety reporting within the institution. Data was collected for all admissions 6 months before and after the implementation of the intervention. These admissions were then used to query the incident reporting database and all incidents for the admission in question were put into a master data file. These incidents were reviewed and classified according to their original classification: Adverse event, non-intercepted potential adverse event, error with little potential for harm, or exclusion (incident reported within the system which did not fit the definitions previously mentioned or unrelated to the study population). The primary outcome measure was the difference in medical error rates per 100 admissions between the pre- and post-intervention periods.
2.4 Statistical analysis
Demographic characteristics were compared between the pre and post-intervention periods using Pearson χ2 for any dichotomous variables and Wilcoxon rank sum (2 sample) test for any continuous variables. Error rates per 100 admissions were compared using Poisson regression analysis. This regression method was used as it was in the original and subsequent Starmer methods.[14,15] A dichotomous variable was set in the model for whether the event occurred pre or post-intervention. The regression model did not include any other demographic or patient information in the model as predictor variables as they were not significantly statistically different when compared pre to post-intervention periods. All statistical tests were completed with statistical significance considered at P Values <.05. All data analysis was completed using R version 3.5.1 (The R Foundation for Statistical Computing).
The original study used a known surveillance method and described approximately 55 errors per 100 admissions. Using this as a baseline estimate of the rate of medical error, a sample size of 648 admissions (324 from pre- and 324 from post-intervention periods) would be required to detect at 20% difference at 80% power. Given the difference in surveillance method, lack of literature and lack of experience in the proposed data collection method at our institution, we proceeded by doubling the sample size collection to approximately 1200 (600 pre- and 600 post-intervention). We estimated, given the number of admissions at our institution, that this sample size would require 6 months of pre and post data.
For the study period in question (from February 2016 to February 2017), a total of 1290 admissions were analyzed (n = 614 pre-intervention; n = 676 post-intervention). There were no significant differences between patient characteristics in the pre- and post-intervention periods (Table 1).
There was a statistically significant decrease in total medical errors across pre- and post-intervention periods studied. The overall medical error rate dropped from 6.0 (95% CI, 4.2–8.3) to 2.2 (95% CI, 1.2–3.6) per 100 admissions (P < .001). Medical errors which had been deemed preventable dropped from 0.65 (95% CI, 0.18–1.67) to 0.15 (95% CI, 0.03–0.82) per 100 admissions (P = .194). Non-intercepted potential adverse events dropped from 1.3 (95% CI, 0.56–2.57) to 0.44 (95% CI, 0.09–1.30) per 100 admissions (Table 2).
Similar to the published literature from the pediatric population, when compared to pre-intervention, the implementation of a new hand-off tool and training on hand-offs resulted in decreased medical errors within our institution. This decrease is not surprising given the high-quality evidence in the literature describing how training in these critical patient safety tasks can result in decreased medical errors.[14,15] It is important to see, however, that the effect holds in our significantly different care environment. It is interesting to note that when breaking down medical errors in the categories used by Starmer et al, we can see that while all categories of events decreased in absolute terms pre- to post-intervention period, the most significant (and indeed the only individually statistically significant) decrease was noted in those with little to no risk of harm. We hypothesize that the standard inclusion of additional detail from the EMR, and attention to the process itself may result in fewer minor errors. One possible avenue of future research which may help continue to reduce errors, especially those with more potential for harm, is related to medication errors. While still a small absolute number of errors causing harm, a majority of them involved medications administered to patients. There is an emerging specialty in the field of Pharmacy in which clinically trained pharmacists are part of the multi-disciplinary medical team. The addition of these clinical pharmacists has been shown to improve several patient related outcomes. It may be possible to continue to leverage this process improvement by incorporating the addition of the clinical pharmacist to the actual hand-off process.
Within our study, we chose to proceed with a bundled approach to a hand-off intervention as was done in the original Starmer study. From a logistical and practical perspective, with a program of our size, we were unable to create subgroups within our residency training environment to provide, for example, training only, with no hand-off tool. While this present study is then subsequently unable to tease apart individual effects of the specific components of the handoff bundle, the decision to use a bundled approach has previously been shown in other areas of patient safety (iatrogenic infections, surgical complications, ventilator-associated pneumonia), to reduce error rates when used together.[21,22] The original I-PASS study did attempt to separate the effect of a written versus a computerize hand-off tool within the bundle but failed to show any difference in medical error rates between their 2 study units. This study does not specifically answer the question of whether or not all of the bundle elements are required to be present to have an effect. An area of further study could be to assess if there are varying effects from the training sessions, the computerized tool, and the standardized process.
There are several limitations to consider when putting this finding into context. First, it is well known that there is difficulty in self-reporting of incidents by medical staff. This underreporting can be seen when we compare our absolute event rate baseline of 6.02 to the reference rate in the literature of 55 per 100 admissions. The research is confounded by the fact that our study took place in a completely electronic health record environment with a 100% computerized provider order entry system with real-time pharmacy support and computerized decision-making support. This environment is likely less prone to error than some of the study contexts described previously. Nonetheless, this significant change in event rate affects the required power calculation and estimated prevalence of medical errors. We attempted to adjust for this issue by expanding our sample size and can see while that was effective in reaching significance for the overall main outcome measure, the individual break-down of medical errors does not have significant differences aside from errors with little potential for harm.
Our study is also limited by the retrospective nature of review involved, and while participating residents were not blinded to the intervention or stratified into control groups (only 1 FMTS exists at our institution), there was no indication at the time that their behavior was being studied. We cannot say with certainty that this change represents a causative agent of change, which accounts for the change in medical error rates. Anecdotally, when long-time faculty, institutional leadership, and other physicians were surveyed informally by the authors, there was a sense that the overall medical error rate had remained stable over the years since the implementation of the EMR, except for specific targeted interventions to reduce or remove a particular care delivery problem. Certainly, a future direction for research would include prospective study design, testing the intervention in another residency program. Ideally, this could be conducted in an institution larger than our own, which has at least 2 teaching services. This design would allow for not only prospective hypothesis testing but would allow us to add a control group to this observational study design and remove potential confounding variables.
In conclusion, we were successful in demonstrating a real-world reduction in medical error rates on our in-patient teaching unit, using a standardized hand-off tool and training, through the use of retrospective data collected completely from our electronic patient safety reporting tool. Further research is needed to determine if the overall rate of medical errors is dramatically lower in care environments such as this, or if under-reporting of events is so dramatic as to make this type of research impractical or invalid using this type of data source. In addition, research should focus on the role of standardization in hand-offs in Family Medicine Residency programs, as this is a generally underrepresented area of focus in the hand-off literature.
Thank you to Jane Eldridge, Susan Boardman, and Laurie Twigg for assistance in collecting data; T. Erik Schackow, MD, PhD, for his expertise in EPIC data reporting and research data collection, Laura Fitzgerald, MPH, for her assistance in ethics review, and Vicky Hickey, BSc, for her assistance in study design and manuscript review.
Conceptualization: Zachary Dewar, Theresa Yurkonis, Maximos Attia.
Data curation: Zachary Dewar.
Formal analysis: Zachary Dewar, Maximos Attia.
Investigation: Zachary Dewar, Maximos Attia.
Methodology: Zachary Dewar, Theresa Yurkonis, Maximos Attia.
Project administration: Zachary Dewar, Maximos Attia.
Resources: Zachary Dewar.
Supervision: Zachary Dewar, Maximos Attia.
Writing – original draft: Zachary Dewar.
Writing – review & editing: Zachary Dewar, Theresa Yurkonis, Maximos Attia.
Zachary Dewar orcid: 0000-0002-4697-7023.
. Makary MA, Daniel M. Medical error
—the third leading cause of death in the US. BMJ 2016;353:i2139.
. Leape LL, Woods DD, Hatlie MJ, et al. Promoting patient safety by preventing medical error
. JAMA 1998;280:1444–7.
. Leape LL. Institute of medicine medical error
figures are not exaggerated. JAMA 2000;284:95–7.
. Donaldson MS, Corrigan JM, Kohn LT. To Err is Human: Building a Safer Health System. Vol. 6. Washington, DC, USA: National Academies Press; 2000.
. Weingart SN, Wilson RM, Gibberd RW, et al. Epidemiology of medical error
. BMJ 2000;320:774.
. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.
. Cohen MD, Hilligoss PB. The published literature on handoffs in hospitals: deficiencies identified in an extensive review. BMJ Qual Saf 2010;19:493–7.
. Graham KL, Marcantonio ER, Huang GC, et al. Effect of a systems intervention on the quality and safety of patient handoffs in an internal medicine residency program. J Gen Intern Med 2013;28:986–93.
. Arora V, Johnson J, Lovinger D, et al. Communication failures in patient sign-out
and suggestions for improvement: a critical incident analysis. BMJ Qual Saf 2005;14:401–7.
. Vidyarthi AR, Arora V, Schnipper JL, et al. Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign-out
. J Hosp Med 2006;1:257–66.
. Nasca TJ, Day SH, Amis ES. The new recommendations on duty hours from the ACGME
task force. N Engl J Med 2010;363:e3(1)–6).
. Barger LK, Ayas NT, Cade BE, et al. Impact of extended-duration shifts on medical errors, adverse events, and attentional failures. PLoS Med 2006;3:e487.
. Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns’ work hours on serious medical errors in intensive care units. N Engl J Med 2004;351:1838–48.
. Starmer AJ, Sectish TC, Simon DW, et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff
bundle. JAMA 2013;310:2262–70.
. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff
program. N Engl J Med 2014;371:1803–12.
. Group I-PASS
. About the I-PASS
Study Group, 2014. Available at: http://www.ipassstudygroup.com/about#MembersPartners
[access date November 2, 2018].
. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA 1995;274:29–34.
. Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors and adverse drug events. J Gen Intern Me 1995;10:199–205.
. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA 2001;285:2114–20.
. Kaboli PJ, Hoth AB, McClimon BJ, et al. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med 2006;166:955–64.
. Haynes AB, Weiser TG, Berry WR, et al. Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.
. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725–32.
. Sandars J, Esmail A. The frequency and nature of medical error
in primary care: understanding the diversity across studies. Fam Pract 2003;20:231–6.