Journal Logo

Empirical Investigations

A Standardized Handoff Simulation Promotes Recovery From Auditory Distractions in Resident Physicians

Matern, Lukas H. BA; Farnan, Jeanne M. MD, MHPE; Hirsch, Kristen W.; Cappaert, Melissa MA; Byrne, Ellen S.; Arora, Vineet M. MD, MAPP

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: August 2018 - Volume 13 - Issue 4 - p 233-238
doi: 10.1097/SIH.0000000000000322

Abstract

Handoffs, or transfers of patient information and responsibility between health care providers, have dramatically increased in frequency in graduate medical education (GME) since the implementation of resident work hour regulations in 2003. However, the greater frequency of handoffs occurring between inexperienced physician trainees raises concerns regarding patient safety and quality of care. Poorly executed handoffs, particularly those omitting important clinical information or conveying ambiguous instructions, demonstrably increase the risk of adverse events among hospitalized patients.1,2

It is therefore critical to provide high-quality handoff instruction to trainees upon entry into residency, and many leading institutions have responded to this challenge accordingly. For example, the Joint Commission designated the handoff as a National Patient Safety Goal in 2006, citing the limitation of interruptions and distractions as a key measure for implementation. Other US organizations, including the Accreditation Council of Graduate Medical Education and the Association of American Medical Colleges, have prioritized handoff training with initiatives such as the Association of American Medical Colleges' adoption of the handoff as a Core Entrustable Professional Activity for Entering Residency.

A number of GME programs have also developed simulation exercises to meet the need for effective handoffs from the beginning of residency.3–5 Despite the burgeoning use of handoff exercises, however, most current simulations do not closely mimic live resident handoffs, which are frequently subject to numerous distractions. Numerous studies suggest that handoffs among clinicians are riddled with interruptions, with virtually all handoffs disrupted by side conversations, pagers, and other distractions.6,7 Noise levels in busy resident handoff environments, which commonly exceed recommended volume thresholds, also impede conversational understanding.8 This state of affairs poses a significant threat to patient safety, with recent studies demonstrating that tasks requiring careful communication, such as drug administration and medication reconciliation, may be compromised by common interruptions.9–12 This is consistent with findings from the communication psychology literature, which indicates that hospital distractions negatively impact clinical task completion and efficiency.13,14

Given the detrimental impacts of low-quality handoffs on patient care, it is critical to prepare novice clinicians for safe handoff execution and effective communication in realistic hospital environments rife with distraction. This study aims to analyze trainees' responses to common auditory interruptions, particularly hospital noise and pagers, when these are introduced into a reproducible handoff simulation. In addition, this investigation examines handoff performance metrics within such a simulation, thereby assessing the impacts of a didactic handoff module on residents' communication skills and strategies within the exercise. We anticipate that our findings may be helpful in communication skills training at GME programs beyond our own.

METHODS

Setting and Participants

This study was conducted at the University of Chicago Medicine, a large, urban academic medical center. Eligible participants included 127 interns (76 female, 51 male) representing 60 medical schools. Of the interns, a total of 122 were entering categorical residency programs in internal medicine (36), pediatrics (23), internal medicine/pediatrics (4), general surgery (8), emergency medicine (16), obstetrics/gynecology (7), psychiatry (6), anesthesiology (8), otolaryngology (2), plastic surgery (2), orthopedic surgery (5), neurosurgery (2), and urology (3), whereas the remaining five interns were beginning a preliminary medicine internship year. The University of Chicago Institutional Review Board granted this research an exemption as a component of a required educational experience.

Handoff Training Module

A didactic online module was distributed to interns via Red Riding Hood, a content delivery application by Click to Play Media (Berkeley, CA), at 14 days before the handoff simulation to allow sufficient time for review. The contents of the module included (1) a 4-minute video highlighting handoff pitfalls, (2) a 15-minute screencast recorded by an investigator (VA) on handoff best practices, and (3) a 7-question multiple-choice assessment, which interns were required to pass to participate in the simulation. The screencast included a primer on situational challenges to the handoff and taught the elements of the effective verbal handoff using the SBAR (Situation, Background, Assessment, Recommendation) mnemonic device, which was selected based on its simplicity, broad applicability, and tested efficacy in a variety of healthcare settings since its introduction into clinical use.15–17 Tested in a 2010 pilot and a previous study, this module was updated with evidence-based recommendations on avoiding handoff distractions and common pitfalls, including a primer on the egocentric heuristic and the effects of sleep deprivation on communication. The module also included new content with regard to minimizing distractions, assessing the receiver's comprehension with teach-back techniques, and prioritizing patients by acuity.7,18,19

Trainees also completed a five-question presurvey and a three-question postsurvey to assess previous handoff experience, satisfaction with previous handoff training, self-reported preparedness before the module, module effectiveness and perceived impacts on future medical practice, and preparedness after the module (see Document, Supplementary Digital Content 1, Questionnaires; https://links.lww.com/SIH/A375).

The Observed Simulated Handoff Experience

To assess handoff performance, we used a modified Multi-Patient Observed Simulated Handoff Experience.18,19 Forty-eight hours before the simulation, interns were provided with prepopulated written sign-out forms developed by faculty (J.M.F., V.A.) and including three patient cases (see Document, Supplementary Digital Content 2, Written sign-out forms; https://links.lww.com/SIH/A376). Written sign-outs were distributed to improve standardization in the evaluation of interns' verbal handoff skills. Participants completed all simulations at the University of Chicago Simulation Center, a clinical space retrofitted for simulation-based training. Interns were each allotted 15 minutes to hand off their patients to trained receivers using the sign-out forms (see Document, Supplementary Digital Content 3, Door chart; https://links.lww.com/SIH/A377).

All trainees were assigned to one of the following three handoff conditions during the exercise: (1) no interruptions, (2) a looped recording of hospital noise, or (3) recorded hospital noise with standardized pages. The 40-second noise recording was extracted from video footage of a live resident workroom. Volumes were measured at equivalent continuous sound levels of 58 to 70 dB (peak sound pressure levels = 82–91 dB) at a distance of 2 meters from high-fidelity stereo speakers; these values were based on previous investigations of hospital noise levels.6 Although participants were not explicitly instructed to request noise reduction, receivers were authorized to reduce noise volume when prompted by the trainee. To simulate pager interruptions, an investigator (L.M.) delivered pages to trained receivers at 90 and 240 seconds. Each page included a script for the receiver: the first contained a neutral message, and the second featured a conversational hook to distract the sender (see Document, Supplementary Digital Content 4, Electronic pager scripts; https://links.lww.com/SIH/A378).

Evaluation and Feedback

After each simulation, receivers provided feedback using a four-item Handoff Mini-Clinical Examination Exercise (Mini-CEX) instrument and an eight-item Handoff Behaviors Checklist that were used and concurrently validated in a previous study.20 Drawing on the literature, the Handoff Behaviors Checklist assessed eight behaviors of a successful interrupted handoff, including rapid recovery from interruptions and avoidance of side conversations; receivers marked whether each task was completed (“yes”) or not completed (“no”) (see Document, Supplementary Digital Content 5, Handoff behaviors checklist; https://links.lww.com/SIH/A379).21 The Handoff Mini-CEX, an instrument that allowed intern performance to be evaluated concurrently on a nine-point Likert-type scale, was used to rate handoff settings, communication skills, professional conduct, and overall sign-out quality (see Document, Supplementary Digital Content 6, Handoff Mini-CEX; https://links.lww.com/SIH/A38). Professional videographers filmed all handoffs with the consent of interns and receivers.

Twelve chief residents or fellows from various specialties (general medicine, hospital medicine, and pediatrics) served as third-party receivers. Before the exercise, receivers were instructed in simulation objectives and evaluation methods. The online training module, Handoff Behaviors Checklist, and Handoff Mini-CEX were also provided to receivers in advance.

Immediately after the GME orientation day, trainees completed an additional six-item Likert-type postsimulation survey assessing realism of the simulation, satisfaction with performance, perceived usefulness of the simulation, and self-reported preparation for future handoffs. Finally, a three-item survey was administered to interns at 1-month follow-up via Medhub, an evaluation portal by Medhub, Inc (Ann Arbor, MI), to reassess handoff skills and perceptions of the simulation's usefulness after 1 month of practical handoff experience during internship.

Statistical Analysis

Descriptive statistical methods were used to summarize all data. Likert-type survey data were dichotomized such that interns who rated their preparedness at “4” or “5” were labeled as “prepared.” χ2 tests and Kruskal-Wallis equality-of-populations tests were used to compare interruption conditions, survey items, and checklist criteria; Mini-CEX scores were compared using analysis of variance. Robust tests of standard deviation were used to assess score distributions. Finally, an investigator (L.M.) reviewed simulation footage to determine the time points at which requests for noise reduction were made. Fisher exact tests were used to compare the percentages of interns satisfying criteria on the Handoff Behaviors Checklist, whereas t tests were used to compare the Mini-CEX scores of interns who requested noise reduction to those who did not. After Bonferroni correction, statistical significance was defined at a P value of less than 0.004. All data analysis was conducted using STATA 13 software (College Station, TX).

RESULTS

Intern Handoff Performance

A total of 125 (98.4%) of 127 interns completed the simulation. Roughly one third of handoffs simulations involved noise only, another third involved noise and pager interruptions, and the remaining third remained uninterrupted (Table 1). According to receiver ratings on the Handoff Behaviors Checklist, significant variation in handoff audibility was observed between interns receiving no interruption and either interruption condition (48.8% noise and 71.8% noise + pager vs. 100.0% uninterrupted, P < 0.001, χ2). However, participants also varied significantly in their abilities to create a “shared space” using the written sign-out form, defined as effectively referencing the form during the handoff (71.1% noise vs. 30.2% uninterrupted and 43.6% noise + pager, P < 0.001, χ2). No significant differences were observed between handoff conditions in other areas (Fig. 1). Most interns faced with pager interruptions were rated as recovering effectively [31/39 (84.2%)] and avoiding side conversations [35/38 (89.7%)].

T1
TABLE 1:
Intake Characteristics of Entering Postgraduate Year 1 Interns Before Participation in the Handoff Simulation
F1
FIGURE 1:
Prioritization: Did the resident appropriately prioritize the patients (in order of acuity) when delivering the handoff to the receiver? If/Thens, To-dos: Did the resident communicate specific action steps and inform to the receiver of what to do if certain situations arise? Opportunities for Questions: Did the resident encourage and provide the receiver with appropriate opportunities to ask questions? Amount of Information: Did the resident provide an appropriate amount of information about the patients? Shared Space: Did the resident orient and use the sign-out form in such a manner as to create a shared space and understanding between the resident and the receiver? Audible Delivery: Did the resident effectively compensate for the level of noise in the room?

On Handoff Mini-CEX, participants receiving noise with or without pages scored lower on establishing appropriate settings (5.7 ± 0.4 noise and 6.2 ± 0.3 noise + pager vs. 8.0 ± 0.1 uninterrupted, P < 0.001, analysis of variance). There were no significant differences by interruption condition in ratings of communication, professionalism, or overall handoff quality (Fig. 2).

F2
FIGURE 2:
Setting: Did the resident make an attempt to create a setting free from interruptions with minimal noise? Communication Skills: Did the resident communicate in a manner that was organized, made use of concrete language, and included all essential information? Professionalism: Did the resident focus on the task at hand and make appropriate and respectful comments regarding the patient, family, and staff? Overall Sign-Out Quality: Did the resident conduct a handoff of overall unsatisfactory, satisfactory, or superior quality? Mini-CEX Average: A mean of the numerical scores assigned to the above categories.

Intern Responses to Noise

Video footage demonstrated that 32 (39.5%) of 81 interns exposed to ambient noise requested noise reduction an average of 32 seconds into the encounter. Interns who requested noise reduction were significantly more likely to be rated as audible than interns who did not (90.6% vs. 38.8%, P < 0.001, Fisher exact test). Participants requesting noise reduction also scored higher on establishing appropriate settings (7.87 ± 0.62 vs. 4.73 ± 1.79, P < 0.001, t test). Overall handoff quality, however, was rated similarly regardless of noise reduction.

Postboot Camp Surveys

Surveys from intern participants who indicated that the module had prepared them to perform handoffs averaged higher overall scores on the Handoff Behaviors Checklist, although this result did not achieve statistical significance (0.686 ± 0.022 vs. 0.614 ± 0.029, P = 0.026, t test). There were no significant differences in ratings of the simulation's realism between interns who received interruptions and those who did not (97.6% with interruption vs. 93.2% control, P = 0.220, Kruskal-Wallis).

At 1-month follow-up, 100 (80.0%) of 125 interns reported performing more effective handoffs as a result of the exercise. Another 106 (84.8%) of 125 interns expressed greater confidence in their handoff skills. Responses to the simulation were overwhelmingly positive; interns commented that “all of the scenarios were helpful” and that they “particularly liked the handoff simulation” and “hope[d] [to] have more of these simulations in the future.”

DISCUSSION

Our findings indicate that after completing a preparatory training module, incoming residents encountering common interruptions performed equally effective handoffs in simulation compared with those receiving no interruptions. Although interns' abilities to communicate audibly and to share a written sign-out form varied between conditions, overall performance was not affected by interruptions; likewise, interns receiving noise and pages recovered proficiently from interruptions and avoided side conversations. Thus, with the preparatory guidance offered by the didactic curriculum, interns were able to adhere to handoff best practices within the simulation regardless of whether auditory interruptions were present.

It is worth examining additional mechanisms for these findings. For instance, it is possible that a plurality of interns had previous experience in overcoming hospital distractions through the use of coping strategies or through the adoption of an “inherently focused workplace mentality.”22 Of note, interns exposed to noise only were observed to share the sign-out document with receivers more frequently. This finding highlights the importance of instructing trainees in the use of adjunctive written communication tools, which have been repeatedly shown to safeguard the handoff in hospital environments rife with distractions.7,23–27 However, it also suggests that the inclusion of noise within the simulation may produce a more realistic environment, one that encourages interns to better use such communication tools and compensate for distractions present in a live hospital setting. Interns receiving the combined noise + pager condition, unlike those receiving noise only, may not have shared the sign-out as readily because of the disruption of communication between intern and receiver by the standardized pages, which may have given the intern the impression that the receiver was not engaged appropriately in the handoff. This may further underscore the importance of minimizing unnecessary interruptions (eg, nonurgent text messages and pages) to preserve the integrity of handoffs between residents. In any event, we recommend further study of compensatory strategies used by physician trainees to recover from and mitigate the effects of interruptions during the handoff.

Although a minority of interns requested volume reduction, doing so improved audibility and the quality of the handoff setting. This supports the notion that active efforts to improve the handoff setting by reducing noise may be valuable in preserving high-quality communication, although true healthcare environments are not always amenable to significant noise reduction. It may therefore be recommended that interns seek out the most distraction-free handoff environment possible while silencing cell phones and minimizing side conversations. However, overall handoff effectiveness was rated similarly between interns who reduced noise and who did not; this may be attributable to variation in interns' use of the preparatory module or to the structure of the simulation itself, because interns were not explicitly informed that the option to lower noise volume was available. Future studies, in addition to accounting for dedicated preparation for the simulation, may explicitly instruct trainees to request noise reduction during the exercise.

Participants' responses to the simulation itself were extremely positive, and their responses on follow-up assessment after 1 month indicate that most interns felt that the exercise had improved their handoff skills in clinical practice. It may be concluded that the simulation is effective in bolstering residents' confidence in and ability to perform handoffs despite significant distractions. At our institution, we plan to continue including simulated interruptions in future iterations of the training exercise.

This study had several limitations. Although interns were recent graduates of 60 medical schools, all simulations were conducted at a single large urban institution. Pager interruptions were delivered only to trained handoff receivers; this may have helped enable interns, acting as senders of the handoff, to better avoid side conversations and remain on task. Similarly, although the delivery of interruptions was generally successful, the researchers were not able to fully account for infrequent delays or random error in the timely transmission of electronic pages. Similarly, it was not possible to account fully for unexpected sources of distraction beyond investigator-generated pages and ambient noise, which may have included outside sirens, noise from adjacent rooms, or random interruptions produced by individual interns themselves. The fidelity of the results was also limited by the study's partial reliance on self-reported metrics. Interrater reliability was not assessed specifically for this study; however, this was done in a recent previous study that used a majority of the same raters in an identical setting at the same institution.18 Finally, it was not possible to account fully for interns' varying levels of preparation for the simulation itself; therefore, completion of the online training module was assumed to provide all interns with an equal knowledge baseline with regard to handoff skills.

To our knowledge, this investigation is the first to test a handoff simulation featuring reproduced hospital distractions. We show that a preparatory training module addressing handoff technique and pitfalls is effective in equalizing most handoff performance metrics despite realistic interruptions. Interestingly, although participants avoided interruption-related pitfalls, interns exposed to hospital noise used the written sign-out form more effectively. It is therefore likely that the incorporation of auditory distractions into training simulations may also encourage interns to practice using compensatory strategies to overcome obstacles to effective handoff performance, thus augmenting the educational value of the simulation. Future investigations may focus on the utility of written sign-out documents and the effects of threats to handoff performance in live hospital environments. Based on our findings and the interns' positive feedback after the exercise, we are able to advocate for the inclusion of interruptions as a means of enhancing the educational impacts of handoff training simulations.

ACKNOWLEDGMENT

The authors thank Dr. Michael Simon, Barry Kamin, the staff of the University of Chicago Office of Graduate Medical Education, and all physicians who acted as trained receivers in this simulation. The authors thank the staff of the University of Chicago Simulation Center for use of their facilities. Finally, the authors thank Samantha Ngooi, Samantha Anderson, and Lisa Spampinato for their logistical support; Michael McGinty for his compilations of survey data; and Mark Saathoff for his audiovisual recordings of live hospital environments.

REFERENCES

1. Horwitz LI, Moin T, Krumholz HM, Wang L, Bradley EH. Consequences of inadequate sign-out for patient care. Arch Intern Med 2008;168(16):1755–1760.
2. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care 2005;14(6):401–407.
3. Didwania A, Kriss M, Cohen ER, McGaghie WC, Wayne DB. Internal medicine postgraduate training and assessment of patient handoff skills. J Grad Med Educ 2013;5(3):394–398.
4. Allen S, Caton C, Cluver J, Mainous AG, Clyburn B. Targeting improvements in patient safety at a large academic center: an institutional handoff curriculum for graduate medical education. Acad Med 2014;89(10):1366–9.
5. Markova T, Dillon L, Coticchia J. Implementing an Institutional Objective Simulated Handoff Evaluation (OSHE) for Assessing Resident Handoff Skill. J Gen Practice 2015;3(4):203.
6. Greenstein EA, Arora VM, Staisiunas PG, Banerjee SS, Farnan JM. Characterising physician listening behaviour during hospitalist handoffs using the HEAR checklist. BMJ Qual Saf 2013;22(3):203–209.
7. Dante A, Andrigo I, Barone F, et al. Occurrence and duration of interruptions during nurses' work in surgical wards: findings from a multicenter observational study. J Nurs Care Qual 2016;31(2):174–182.
8. Yoder JC, Staisiunas PG, Meltzer DO, Knutson KL, Arora VM. Noise and sleep among adult medical inpatients: far from a quiet night. Arch Intern Med 2007;172(1):68–70.
9. Peterson SM, Gurses AP, Regan L. Resident to resident handoffs in the emergency department: an observational study. J Emerg Med 2014;47(5):573–579.
10. Feleke SA, Mulatu MA, Yesmaw YS. Medication administration error: magnitude and associated factors among nurses in Ethiopia. BMC Nurs 2015;14(53):1–8.
11. Duruk N, Zencir G, Eser I. Interruption of the medication preparation process and an examination of factors causing interruptions. J Nurs Manag 2016;24(3):376–83.
12. Berg LM, Kallberg A, Ehrenberg A, et al. Factors influencing clinicians' perceptions of interruptions as disturbing or non-disturbing: a qualitative study. Int Emerg Nurs 2016;27:11–6.
13. Westbrook JI, Coiera E, Dunsmuir WT, et al. The impact of interruptions on clinical task completion. Qual Saf Health Care 2010;19(4):284–289.
14. Parker J, Coiera E. Improving clinical communication: a view from psychology. J Am Med Inform Assoc JAMIA 2000;7(5):453–461.
15. Haig KM, Sutton S, Whittington J. SBAR: a shared mental model for improving communication between clinicians. Jt Comm J Qual Patient Saf 2006;32(3):167–175.
16. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care 2004;13(Suppl 1):85–90.
17. Velji K, Baker GR, Fancott C, et al. Effectiveness of an adapted SBAR communication tool for a rehabilitation setting. Healthc Q 2008;11(3 Spec No.):72–79.
18. Gaffney S, Farnan JM, Hirsch K, McGinty M, Arora VM. The Modified, Multi-Patient Observed Simulated Handoff Experience (M-OSHE): assessment and feedback for entering residents on handoff performance. J Gen Intern Med 2016;31(4):438–41.
19. Farnan JM, Paro JA, Rodriguez RM, et al. Hand-off education and evaluation: piloting the observed simulated hand-off experience (OSHE). J Gen Intern Med 2010;25(2):129–134.
20. Arora VM, Berhie S, Horwitz LI, Saathoff M, Staisiunas P, Farnan JM. Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise. J Hosp Med 2014;9(7):441–446.
21. Newble D. Techniques for measuring clinical competence: objective structured clinical examinations. Med Educ 2004;38(2):199–203.
22. Anderson CE, Nicksa GA, Stewart L. Distractions during resident handoffs: Incidence, sources, and influence on handoff quality and effectiveness. JAMA Surg 2015;150(5):396–401.
23. Tapia NM, Fallon SC, Brandt ML, Scott BG, Suliburk JW. Assessment and standardization of resident handoff practices: PACT project. J Surg Res 2013;184(1):71–77.
24. Donnelly MJ, Clauser JM, Weissman NJ. An intervention to improve ambulatory care handoffs at the end of residency. J Grad Med Educ 2012;4(3):381–384.
25. Starmer AJ, Spector ND, Srivastava R, Allen AD, Landrigan CP, Sectish TC. I-pass, a mnemonic to standardize verbal handoffs. Pediatrics 2012;129(2):201–204.
26. Starmer AJ, O'Toole JK, Rosenbluth G, et al. Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs. Acad Med 2014;89(6):876–884.
27. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med 2014;371(19):1803–1812.
Keywords:

Handoffs; patient safety; graduate medical education; simulation

Supplemental Digital Content

Copyright © 2018 Society for Simulation in Healthcare