Communication failures among busy hospital services, particularly those occurring during patient handoffs, are a major source of preventable adverse events in health care.1–5 The Joint Commission recently reported that nearly two-thirds of all sentinel events, or unexpected patient safety events causing death or serious harm, can be attributed to failures in communication.4 Among such communication breakdowns in surgical departments, nearly two-thirds involve an inadequate handoff.3 Since 2011, the Accreditation Council for Graduate Medical Education (ACGME) has required residency training programs to provide formal housestaff education in handoffs, and to use a system to monitor handoff quality.6 Even though evidence-based reports on how to fulfill these requirements are numerous,5,7–10 recommendations remain limited, and universally accepted methods are lacking.
The day-to-day functioning of surgical services is noteworthy in that it often involves care in disparate hospital locations including the emergency department, the operating room (OR), the postanesthesia care unit (PACU), and the surgical intensive care unit (ICU). Surgical team members often have multiple demands for their time and attention, and therefore OR-to-ICU handoffs often compete with issues elsewhere. This observation is important as OR-to-ICU handoffs are complex, involving communication among multiple teams (surgery, anesthesia, critical care, respiratory care, and nursing). In addition, these transitions often involve the simultaneous physical transfer of both patient and equipment.11 Indeed, studies have shown OR-to-ICU handoffs to be error prone and associated with inappropriate multitasking, poor teamwork and communication, and distracting interruptions.12 Some studies have drawn analogies between handoffs and tasks in other complex, multiprofessional settings such as Formula 1 motor racing or square-rigger sailing, suggesting that the prioritization of safety and teamwork that has worked to reduce errors in these settings could serve as an example of how a similar approach might benefit patient care.13,14
Independently, a number of studies have investigated the concept of the “weekend effect.” This theory suggests that patients presenting for care outside of regular working hours are at risk for greater morbidity and mortality,15–21 particularly for conditions with high baseline case fatality rates.21 Specifically, off-hour presentation of patients with myocardial infarction or stroke has been associated with higher risk-adjusted mortality than weekday presentation.16,18 Although the causal relationship remains unclear, the authors of these studies propose numerous contributing factors to explain these findings, including staffing, provider coverage practices, resources, and intensity of care.15,16,18,19,21 Other fields such as trauma surgery may not suffer from this diurnal variation partly because of their standardized 24/7 care.20
The increasing prevalence of telemedicine with in-room video cameras offers an appealing opportunity to capture handoffs and evaluate multiple quality measures of OR-to-ICU handoffs. In this study, we analyzed in-room video recordings of real OR-to-ICU handoffs to determine the tempo and quality of team interactions. We sought to demonstrate how existing telemedicine technology can be used to evaluate handoffs, and we hypothesized that handoff quality would be worse on nights and weekends than during regular weekday hours.
The Hospital of the University of Pennsylvania is a 700-bed, quaternary, urban teaching hospital. Its 24-bed surgical ICU cares for patients originating from a variety of primary surgical services, excluding cardiac and neurological surgery patients. The ICU care team is composed of registered nurses; critical care fellows; surgical, emergency medicine, and/or anesthesia residents; attending intensivists (anesthesiologists or trauma surgeons); and advanced practice providers (APPs). The ICU averages 210 admissions per month. During weekday hours (Monday–Friday, 0700–1700), the ICU care team is divided into two teams managing 10 to 12 patients each; each team comprises an attending intensivist, 1 to 2 critical care fellows, 1 to 2 APPs, and 2 to 3 residents. During night and weekend hours, there is one ICU care team that comprises 1 critical care fellow and 2 residents. During both weekday and night and weekend hours, the nurse-to-patient ratio in the ICU is either 1:1 or 1:2, with the precise ratio determined by patient acuity.
Per ACGME requirements, all new housestaff undergo a didactic handoff curriculum organized at the departmental level that differs among specialties and is not ICU specific in content, format, or location. APPs and nurses do not receive formal handoff training apart from what they received in physician assistant or nursing school or from observing their unit peers.
Telemedicine services and performance improvement video recordings
Each surgical ICU patient room is equipped with a camera and microphone, which transmit a high-fidelity video and audio signal to the University of Pennsylvania Penn E-lert remote monitoring service housed in a separate location. The cameras are located above the foot of the patient’s bed, mounted on the wall close to the ceiling. In December 2008, the hospital established a performance-improvement-driven institutional policy that allows video recording of any ICU bedside event requiring the immediate presence of a physician or APP through the existing bedside telemedicine technology. The majority of recordings are initiated at the discretion of the bedside nurse but can also be triggered by any ICU or telemedicine provider. Recorded events most often involve cardiac arrests, procedures, and rapid deteriorations but also frequently include admissions and readmissions of patients to an ICU room. Unit nurses are encouraged to record all such events, and the practice of recording events is prevalent, with an average of two to five recordings per day in the ICU. All ICU care providers are aware that video recording could occur in any room and at any time to monitor events related to patient care but are not alerted that they are actively being recorded prior to or during the recordings. The hospital obtains consent for video recordings from the patient or a legally authorized representative as part of a global ICU admission consent form, which encompasses the admission, bedside procedures, telemedicine monitoring, and video recording for quality improvement (QI) purposes.
Per institutional QI policy, these digital recordings are stored on a secure, encrypted computer server only accessible to QI officers. Unrelated to this study, a selection of these recordings is regularly presented at monthly multidisciplinary video review conferences under the auspices of QI activities. Per the institution’s policy, all video recordings must be destroyed within 28 days of the recording date.
When videos of admissions or readmissions from the OR are obtained, they are initiated on the patient’s entry into the room and last for 45 minutes, regardless of in-room activity. In the first recorded minute, the camera is positioned facing the door to accurately capture who was entering and exiting the room during this high-traffic interval immediately following patient entry. After the first minute and thereafter, the camera is positioned to face the patient’s bed. Every 30 seconds the camera pans once around the room to better capture all events in the room. During the last 30 minutes of recordings, camera pans occur once every minute.
Only recordings of handoffs involving patients who came directly from the OR were included in the study.
Selection of handoff recordings
We obtained institutional review board approval from the University of Pennsylvania to review only recorded admissions and readmissions from the OR. This approval allowed the project to be conducted exempt from additional consent as the recordings were obtained under the auspices of QI activities and under the argument that acquiring consent from every provider, visitor, and allied personnel that could momentarily appear in the recorded segments would not be practicable before or after the event was recorded.
From July 2014 to January 2015, study personnel (M.E.B., D.Z.) rotated per their schedule availability and accessed all admission and readmission videos that were on the server prior to deletion by a QI officer per institutional policy. In total, this amounted to a convenience sample of 32 recordings of OR-to-ICU bedside handoffs available for study that M.E.B. and D.Z. scored in a prospective observational manner (see below).
Video-recorded handoff assessments
Video review of handoffs differs from in-person observation and scoring in terms of the information that can be easily captured, so we focused our assessment on characteristics and quality measures such as timing, teamwork skills, and communication skills. As there are no validated handoff assessment tools for video-recorded, naturally occurring (i.e., not simulated) handoffs, we adapted validated in-person assessment measures from a combination of published reports as detailed below.
Two of the authors evaluated video footage offline: a senior medical student with previous training and experience in OR-to-ICU handoff assessment (M.E.B.), and a senior critical care APP whom M.E.B. trained (D.Z.). The evaluators were not blinded to the time of handoffs because the presence of windows on camera provided a general sense of day versus night.
M.E.B. analyzed all videos for basic handoff characteristics, including number of people in the room in the first recorded minute; total time spent in the room by the OR anesthesia provider; total time spent in the room by the surgery provider; time interval after patient entry at which all providers (ICU physician [intensivist, fellow, or resident] or APP, surgery, and OR anesthesia providers) had left the room (defined as no provider returning within five minutes of leaving); time interval after patient arrival to commencement of first verbal handoff report; total duration of handoff reports, as well as the duration of the surgery provider’s and OR anesthesia provider’s reports; the presence or absence of a report from the surgery provider to an ICU provider (ICU physician or APP) or nurse; the presence or absence of a report from the OR anesthesia provider to an ICU provider or nurse; and the presence or absence of a physical examination by an ICU provider or nurse (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A430).
Both evaluators independently analyzed the recorded videos for measures of handoff quality, including teamwork skills; report delivery skills (scored separately for the surgeon and anesthesiologist); passive and active listening skills of the ICU provider receiving each report; provider and nursing engagement in the handoff; number of interruptions; and number of unprofessional comments or actions during the report. See Table 1 for the operational definitions for teamwork skills, report delivery skills, and engagement; we adapted these definitions from performance criteria used in a prior study by Catchpole et al.13 For a given handoff, the evaluators gave each teamwork skill a score of 1 if it was met or 0 if it was not met, and calculated a sum of the teamwork skills (maximum = 5). Evaluators rated report delivery skills and engagement as superior, satisfactory, or unsatisfactory, per the descriptions in Table 1. The evaluators also assessed the presence or absence of listening skills using a validated scoring instrument for physician listening behaviors, including eye contact, affirmatory statements, head nodding, note taking, and question asking and/or interactive comments (e.g., read-backs).22 Each handoff could have up to two reports—one each from surgeon and the OR anesthesiologist—and the evaluators scored these reports separately for listening skills. Finally, the evaluators tallied the number of interruptions (pages, messages, phone calls, or side comments that interfered with handoff continuity) and unprofessional comments or actions (inappropriate comments about patients or staff, or distractions such as laughter or unrelated personal conversation) (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A430).
M.E.B. evaluated all 32 videos for these quality measures. D.Z. only evaluated 10 of these videos because of decreased availability to evaluate videos prior to their deletion per institutional policy. Both evaluators followed the aforementioned handoff evaluation guide with a list of the operational definitions to improve consistency in evaluating handoffs (see Table 1).
We compared mean scores of basic characteristics and quality measures of handoffs occurring during weekday hours (Monday–Friday, 0700–1700) with those occurring at night and weekend hours. Of note, late-night handoffs (those occurring after 2400) were uncommon.
For basic handoff characteristics, we used the Mann–Whitney U test to compare means for continuous variables (number of people in the patient’s room and durations). For measures of time and handoff report duration, we excluded outliers greater than two standard deviations from the mean. This resulted in the exclusion of no more than two data points per comparison. We used Pearson chi-square test to compare categorical variables (types of handoff reports, number of handoffs in which a physical examination was performed by the ICU provider or nurse).
For handoff quality measures, 10 of the handoffs were scored by both evaluators (see above); for these handoffs, we used averages for measures in which the two evaluators assigned different scores. We used Pearson chi-square test to compare categorical variables (number of handoffs meeting a given teamwork or listening skill, number of handoffs with satisfactory or superior ratings for report delivery skills and engagement). We used Fisher exact test in place of Pearson chi-square test in cases where an expected frequency in the chi-square was 5 or lower. We used the Mann–Whitney U test to compare the sum of teamwork skills met and the number of interruptions and unprofessional comments and actions. We calculated P values using the exact two-tailed significance and considered a P value < .05 significant. We used SPSS Statistics Version 23 (IBM, Armonk, New York) for all data analyses.
Basic handoff characteristics
We compared 16 weekday handoffs to 16 night and weekend handoffs. There were no significant differences in basic characteristics between the two groups (Table 2). The mean duration of the surgeon’s handoff report tended to be 36% longer on nights and weekends (mean 2 minutes 8 seconds) than on weekdays (mean 1 minute 34 seconds), but this failed to reach significance (P = .10).
Handoff quality measures
We compared 16 weekday handoffs (with 28 reports) to 16 night and weekend handoffs (with 29 reports). The teamwork, engagement, and report delivery skills quality measures were not found to be inferior on nights and weekends compared with weekdays (Table 3). Notably, on nights and weekends, handoff report delivery skills were more likely to be rated as satisfactory or superior (P = .03); this was primarily due to reports delivered by the surgeon (P = .002), as OR anesthesiologists’ delivery skills were scored similarly for both time frames. There was also a trend toward improved engagement among providers on nights and weekends (P = .07), again largely due to higher scores among the surgeons during this time (P = .04), as OR anesthesiologists’ engagement was scored similarly for both time frames.
Because 6 of the 16 handoffs in both the weekday and night and weekend time frames involved a handoff received by a nurse without a receiving ICU physician or APP present, we performed a subgroup analysis excluding those handoffs (Table 4). The results remained similar with better surgeon report delivery skills on nights and weekends (P = .03), while other measures remained noninferior.
For the 10 videos assessed by both evaluators, there was strong agreement overall with Cohen’s kappa values of 0.82 for report delivery skills, 0.72 for engagement, 0.52 for all listening skills combined, 0.62 for number of interruptions, 0.64 for number of unprofessional comments or actions, and ranging from 0.62 to 1.0 for each of the teamwork skills.
This study demonstrates a technique to unobtrusively monitor the quality of OR-to-ICU handoffs at all hours of the day and night using existing in-room telemedicine technology. Contrary to our hypothesis, the current study demonstrates that bedside OR-to-ICU handoffs on nights and weekends in an academic hospital are not inferior to those occurring during workday hours. Basic handoff characteristics and most quality measures were similar between weekdays and nights and weekends, and two subjective aspects of surgeon participation—report delivery skill and engagement in the handoff process—were rated more favorably on nights and weekends than on weekdays.
Although there are a number of reasons why we had hypothesized that night and weekend handoffs would be of lesser quality, including provider fatigue and decreased schedule predictability, there are also several possible explanations for why handoff performance on nights and weekends remained similar, if not slightly better. First, it may be that the lighter operative caseload on nights and weekends resulted in surgeons and other providers being less rushed or distracted and thus more capable of engaging in the handoff process. Second, although we did not differentiate the level of training among surgeons giving handoffs, it may be that surgical providers who played a greater role in the operation or who had more experience with handoffs were more commonly handing over patients on nights and weekends.
Previous reports on the quality of handoffs on different days and at different times of the day are limited and reveal conflicting results.23,24 We recently evaluated OR-to-ICU handoffs during the weekdays versus nights and weekends using study personnel present within the patient’s room, who evaluated the handoffs in full view of the clinical providers. In that setting, we found significantly better transmitter communication scores on nights and weekends compared with weekdays; however, a Hawthorne effect may have been present.23 The current study may have gauged handoff practices more accurately, as the evaluation was carried out essentially without the transmitters or receivers being aware that they were being evaluated. Another study of change-of-shift handoffs in obstetrical inpatients found that resident handoffs were better in the mornings than evenings, while the opposite was true of the nursing handoffs.24
Only a handful of studies have reported on the use of video recording to assess the quality of naturally occurring handoffs.25–28 One study used a conference room camera to assess end-of-the-week handoffs on Fridays between attending physicians and found that these handoffs rarely followed commonly recommended communication schemes.25 Two studies recorded handoffs in the PACU and concluded that the use of a checklist improved OR-to-PACU handoffs and that patient information and equipment were transferred simultaneously in 65% of cases.26,27 Only one other study has assessed OR-to-ICU handoffs; in this study, after implementing a standardized handoff protocol in a pediatric cardiac surgery ICU, the authors used video to demonstrate long-term sustainability.28 It is notable that in all but one of these studies28 the subjects were fully cognizant that their handoff performance was being recorded, potentially affecting their behavior and the quality of the information transmitted.
This study highlights the unobtrusive nature of leveraging existing telemedicine infrastructure to video record handoffs for analysis of quality measures. Video recording of naturally occurring handoffs may serve as an effective means for operationalizing the ACGME’s requirement to provide formal education on and monitor patient handoffs by housestaff. Indeed, other studies have demonstrated that standardized or simulated video recordings are very useful in teaching and monitoring handoff skills as part of a structured curriculum.29,30 These types of recordings, however, lack the individualized assessment, feedback, and coaching that videos of real handoffs could provide. The power of video recording in improving quality of care has already been demonstrated in other areas, for example, with hand washing compliance, colonoscopy, and surgical technique in the OR.31–34 In these studies, improvements were partly attributed to an awareness of video recording,33 group feedback via the posting of compliance rates,34 or individualized coaching sessions with a peer-judged surgical expert.32 These methods are viable options that could be applied in tandem with video recordings to improve handoff quality.
As an observational evaluation, our study has important limitations. First, given the preliminary nature of this work, our sample size is limited and reflects a convenience sample of transitions of care in a single surgical ICU. Although we were unable to randomly select handoffs for recordings, we included all videos capturing admissions and readmissions from the OR for QI purposes that we were able to evaluate before their deletion per institutional policy in the study. Second, the scoring of quality measures studied such as teamwork skills, engagement, report delivery skills, and listening skills is inherently subjective, and there was no available validated scoring instrument for video review. The evaluators used a scoring tool with operational definitions to improve standardization, and although double review could not be conducted for all handoffs due to availability, the few handoffs that did receive two evaluations demonstrated good interrater agreement. Third, because the wall-mounted cameras capture only events occurring within a patient’s room, only handoffs occurring at the bedside were observed; other forms of information transfer, such as those completed by phone or electronic medical record, were not included in our assessment. Finally, the audio quality of some video recordings was limited, and sometimes simultaneously occurring conversations impaired the ability of the evaluator to understand what verbal information was being exchanged. Despite these limitations, this study is the first to use existing telemedicine video technology to record handoffs and allow quality measures to be assessed unobtrusively.
This study presents a novel approach of assessing handoff quality in OR-to-ICU handoffs using in-room video recording via an existing telemedicine infrastructure. Using this approach, the quality of night and weekend handoffs in a single urban teaching hospital was found to be no worse and sometimes better than those during weekday hours. Video analysis may emerge as an ideal unobtrusive QI methodology to monitor handoffs and improve education and compliance with institutional handoff policies.
Acknowledgments: The authors wish to thank Molly Elson and Isabelle Oka for their assistance with video recording and data acquisition, as well as Amanda Martin, MSN, ACNP-BC, for her assistance with handoff evaluation.
1. Horwitz LI, Moin T, Krumholz HM, Wang L, Bradley EH. Consequences of inadequate sign-out for patient care. Arch Intern Med. 2008;168:17551760.
2. Kitch BT, Cooper JB, Zapol WM, et al. Handoffs causing patient harm: A survey of medical and surgical house staff. Jt Comm J Qual Patient Saf. 2008;34:563570.
3. Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133:614621.
4. Joint Commission. Sentinel event data: Root causes by event type 2004–2015. https://hcupdate.files.wordpress.com/2016/02/2016-02-se-root-causes-by-event-type-2004-2015.pdf
. Accessed December 21, 2016.
5. Starmer AJ, Spector ND, Srivastava R, et al.; I-PASS Study Group. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371:18031812.
6. Nasca TJ, Day SH, Amis ES Jr; ACGME Duty Hour Task Force. The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3.
7. Riesenberg LA, Leitzsch J, Massucci JL, et al. Residents’ and attending physicians’ handoffs: A systematic review of the literature. Acad Med. 2009;84:17751787.
8. Wohlauer MV, Arora VM, Horwitz LI, Bass EJ, Mahar SE, Philibert I; Handoff Education and Assessment for Residents (HEAR) Computer Supported Cooperative Workgroup. The patient handoff: A comprehensive curricular blueprint for resident education to improve continuity of care. Acad Med. 2012;87:411418.
9. DeRienzo CM, Frush K, Barfield ME, et al.; Duke University Health System Graduate Medical Education Patient Safety and Quality Council. Handoffs in the era of duty hours reform: A focused review and strategy to address changes in the Accreditation Council for Graduate Medical Education common program requirements. Acad Med. 2012;87:403410.
10. Gordon M, Findley R. Educational interventions to improve handover in health care: A systematic review. Med Educ. 2011;45:10811089.
11. Bonifacio AS, Segall N, Barbeito A, Taekman J, Schroeder R, Mark JB. Handovers from the OR to the ICU. Int Anesthesiol Clin. 2013;51:4361.
12. Segall N, Bonifacio AS, Schroeder RA, et al.; Durham VA Patient Safety Center of Inquiry. Can we make postoperative patient handovers safer? A systematic review of the literature. Anesth Analg. 2012;115:102115.
13. Catchpole KR, de Leval MR, McEwan A, et al. Patient handover from surgery to intensive care: Using Formula 1 pit-stop and aviation models to improve safety and quality. Paediatr Anaesth. 2007;17:470478.
14. Henkind SJ, Sinnett JC. Patient care, square-rigger sailing, and safety. JAMA. 2008;300:16911693.
15. Cavallazzi R, Marik PE, Hirani A, Pachinburavan M, Vasu TS, Leiby BE. Association between time of admission to the ICU and mortality: A systematic review and metaanalysis. Chest. 2010;138:6875.
16. Saposnik G, Baibergenova A, Bayer N, Hachinski V. Weekends: A dangerous time for having a stroke? Stroke. 2007;38:12111215.
17. Laupland KB, Shahpori R, Kirkpatrick AW, Stelfox HT. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23:317324.
18. Kostis WJ, Demissie K, Marcella SW, Shao YH, Wilson AC, Moreyra AE; Myocardial Infarction Data Acquisition System (MIDAS 10) Study Group. Weekend versus weekday admission and mortality from myocardial infarction. N Engl J Med. 2007;356:10991109.
19. Ensminger SA, Morales IJ, Peters SG, et al. The hospital mortality of patients admitted to the ICU on weekends. Chest. 2004;126:12921298.
20. Carr BG, Reilly PM, Schwab CW, Branas CC, Geiger J, Wiebe DJ. Weekend and night outcomes in a statewide trauma system. Arch Surg. 2011;146:810817.
21. Bell CM, Redelmeier DA. Mortality among patients admitted to hospitals on weekends as compared with weekdays. N Engl J Med. 2001;345:663668.
22. Greenstein EA, Arora VM, Staisiunas PG, Banerjee SS, Farnan JM. Characterising physician listening behaviour during hospitalist handoffs using the HEAR checklist. BMJ Qual Saf. 2013;22:203209.
23. Hochman BR, Barry ME, Lane-Fall MB, et al. Handoffs in the intensive care unit: Are off hours a vulnerable time? [Published online ahead of print December 8, 2015.] Am J Med Qual. doi: 10.1177/1062860615617238.
24. Goff SL, Knee A, Morello M, Grow D, Bsat F. Handoff quality for obstetrical inpatients varies depending on time of day and provider type. J Reprod Med. 2014;59:95102.
25. Ilan R, LeBaron CD, Christianson MK, Heyland DK, Day A, Cohen MD. Handover patterns: An observational study of critical care physicians. BMC Health Serv Res. 2012;12:11.
26. Salzwedel C, Bartz HJ, Kühnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: A randomized controlled trial. Int J Qual Health Care. 2013;25:176181.
27. van Rensen EL, Groen ES, Numan SC, et al. Multitasking during patient handover in the recovery room. Anesth Analg. 2012;115:11831187.
28. Northway T, Krahn G, Thibault K, et al. Surgical suite to pediatric intensive care unit handover protocol: Implementation process and long-term sustainability. J Nurs Care Qual. 2015;30:113120.
29. Arora VM, Berhie S, Horwitz LI, Saathoff M, Staisiunas P, Farnan JM. Using standardized videos to validate a measure of handoff quality: The handoff mini-clinical examination exercise. J Hosp Med. 2014;9:441446.
30. Telem DA, Buch KE, Ellis S, Coakley B, Divino CM. Integration of a formalized handoff system into the surgical curriculum: Resident perspectives and early results. Arch Surg. 2011;146:8993.
31. Makary MA. The power of video recording: Taking quality to the next level. JAMA. 2013;309:15911592.
32. Hu YY, Peyre SE, Arriaga AF, et al. Postgame analysis: Using video-based coaching for continuous professional development. J Am Coll Surg. 2012;214:115124.
33. Rex DK, Hewett DG, Raghavendra M, Chalasani N. The impact of videorecording on the quality of colonoscopy performance: A pilot study. Am J Gastroenterol. 2010;105:23122317.
34. Armellino D, Trivedi M, Law I, et al. Replicating changes in hand hygiene in a surgical intensive care unit with remote video auditing and feedback. Am J Infect Control. 2013;41:925927.