Secondary Logo

Share this article on:

Handovers Among Staff Intensivists: A Study of Information Loss and Clinical Accuracy to Anticipate Events*

Dutra, Mariana, MD, MSc1; Monteiro, Mariana V., MD1; Ribeiro, Karina B., PhD, Stat2; Schettino, Guilherme P., MD1; Kajdacsy-Balla Amaral, Andre Carlos, MD3,4

doi: 10.1097/CCM.0000000000003320
Feature Articles
Editor's Choice

Objectives: Handovers are associated with medical errors, and our primary objective is to identify missed diagnosis and goals immediately after a shift handover. Our secondary objective is to assess clinicians’ diagnostic accuracy in anticipating clinical events during the night shift.

Design: Single-center prospective observational cohort study.

Setting: Thirty-bed tertiary ICU in Sao Paulo, Brazil.

Patients: Three-hundred fifty-two patient encounters over 44 day-to-night handovers.

Interventions: None.

Measurements and Main Results: We used a multimethods approach to measure transmission of information among staff physicians on diagnoses and goals for the night shift. We surveyed clinicians immediately after a handover and identified clinical events through chart abstractions and interviews with clinicians the next morning. Nighttime clinicians correctly identified 454 of 857 diagnoses (53%; 95% CI 50–56) and 123 of 304 goals (40%; 95% CI, 35–46). Daytime clinicians were more sensitive (65% vs 46%; p < 0.01) but less specific (82% vs 91%; p < 0.01) than nighttime clinicians in anticipating clinical events at night, resulting in similar accuracy (area under the receiver operating characteristic curve, 0.74 [95% CI, 0.68–0.79] vs 0.68 [95% CI 0.63–0.74]; p = 0.09). The positive predictive value of both daytime and nighttime clinicians was low (13% vs 17%; p = 0.2). Gaps in diagnosis and anticipation of events were more pronounced in neurologic diagnoses.

Conclusions: Among staff intensivists, diagnoses and goals of treatment are either not conveyed or retained 50–60% of the cases immediately after a handover. Clinicians have limited ability to anticipate events, and the expectation that anticipatory guidance can inform handovers needs to be balanced against information overload. Handovers among staff intensivists showed more gaps in the identification of diagnostic uncertainty and for neurologic diagnoses, which could benefit from communication strategies such as cognitive checklists, prioritizing discussion of neurologic patients, and brief combined clinical examination at handover.

1Unidade de Terapia Intensiva, Hospital Sírio-Libanês, São Paulo, Brazil.

2Instituto de Ensino e Pesquisa do Hospital Sírio-Libanês, São Paulo, Brazil.

3Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.

4Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, ON, Canada.

*See also p. 1861 and 1863.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (http://journals.lww.com/ccmjournal).

The authors have disclosed that they do not have any potential conflicts of interest.

For information regarding this article, E-mail: andrecarlos.amaral@sunnybrook.ca

Miscommunication during handovers is associated with medical errors and inefficient care. Trainees often report “surprises” or unexpected changes in care due to incomplete handovers (1) and feel inadequately prepared for as high as 80% of events that occur at night. Failure to properly convey diagnoses and goals is implicated in more than 20% of malpractice claims (2 , 3), is associated with a two-fold increase in preventable adverse events (4), and leads to unnecessary tests and increased costs (5 , 6). Lack of standardization and formal guidance or training leads to variation and inconsistency in handovers (7).

The perception that clinicians fail to properly communicate is based on studies that are potentially biased by hindsight. Miscommunication is identified as the culprit in the light of an unwanted outcome, but it is not clear whether it would have been given the same relevance had an event not occurred (8). Experts believe that the goals of nighttime handovers include providing data on tasks to be accomplished and anticipatory guidance to raise awareness about potential problems at night. Anticipatory guidance has been incorporated into structured handover tools, such as the one used in a before-after study that showed a 23% decrease in the rates of medical errors (9). However, the accuracy of these predictions is not reported in the literature.

Furthermore, most of the literature on handovers has focused on trainees and structured handovers. Recent data suggest that staff intensivists do not use predefined structured schemes for handovers (10 , 11), and we have previously observed that more experienced clinicians providing cross-coverage frequently change diagnoses and plans of care overnight. Exposure to these cross-covering clinicians was associated with lower mortality (12), suggesting that their role at night goes beyond completing tasks and “putting out fires.” Similarly, in a U.K. study looking at intensivist covering patterns, the presence of the daytime staff intensivist at handover was associated with increased mortality, suggesting the potential for perpetuating anchoring biases during handover (13).

In this study, we address these knowledge gaps by using a standardized instrument to measure transmission of information among staff intensivists and their anticipatory accuracy for nighttime events.

Back to Top | Article Outline

MATERIALS AND METHODS

Study Design and Setting

Our study comprised an observational cohort of handovers in critically ill patients in a 30-bed ICU in a private hospital in Sao Paulo, Brazil. This ICU has a high-intensity staffing model, with mandatory intensivist staffing consultation for all patients. Board-certified intensivists are present in the ICU 24 hours a day, with a staffing ratio of one staff for every 11 patients. There are two shifts, a daytime shift, from 7:30 AM to 07:30 PM, and a nighttime shift, from 7:30 PM to 7:30 AM. Intensivists in the day shift are usually on for all consecutive weekdays, whereas intensivists in the night shift will typically work one to two times a week. We used a convenience sample of weekday handovers to collect the data over a period of 25 weeks.

There is no formalized handover protocol, education, or form. Handovers typically occur in person, away from the bedside with note taking on blank paper. The Institutional Review Board at Hospital Sírio-Libanês reviewed and approved the study (study number: 274.124).

Back to Top | Article Outline

Data Collection and Variables

The primary outcome variable was the number of correctly identified diagnoses and goals by the nighttime clinician. Because clinicians may use different terms to identify similar diagnoses and goals, this analysis usually requires qualitative interpretation that may be subject to ascertainment bias. To avoid this post hoc interpretation, we used a multimethods approach to identify common ICU-related diagnoses and goals, which was then used for data collection after a handover (electronic supplementary material, Supplemental Digital Content 1, http://links.lww.com/CCM/D829). Briefly, this system uses a two-tiered approach for diagnoses, where clinicians initially select one or more problems (e.g., respiratory failure) and then one or more diagnoses within each category of problem (e.g., pulmonary edema).

We collected this information from the daytime and nighttime clinicians separately and independently. Data collection happened immediately after the 7:30 PM handover and before nighttime clinicians started chart and patient reviews to avoid providing more information about patients’ diagnoses and goals. We specifically elected to not inform intensivists that they would be surveyed on any particular day to avoid introducing changes to their usual practice, for the same reason, handovers were also not directly observed.

Our secondary outcome variable was clinicians’ accuracy in anticipating events during the night shift, defined as new treatments, interventions, or diagnostic procedures ordered after 7:30 PM and before 7:30 AM of the following day (nighttime events). We asked clinicians to identify for each patient whether they anticipated any nighttime events and what type of events they would expect. These were clustered into five predefined domains (hemodynamic, respiratory, metabolic, neurologic, and hematologic).

We collected outcomes by interviewing the nighttime clinician in the morning after their shift and by abstracting data from the patient’s chart for new orders and new notes describing an unexpected event.

Back to Top | Article Outline

Analysis

Correct Identification of Diagnoses and Goals.

Since we are interested in the ability of daytime clinicians to convey their perceived diagnoses and goals for the nighttime, the diagnoses and goals reported by the daytime clinician are considered the “gold standard.”

The primary analysis reports the percentage of diagnoses and goals reported by the daytime clinician, which were correctly identified by the nighttime clinician. Each individual diagnosis or goal was coded as correctly identified by the nighttime clinician if it matched the diagnosis or goal reported by the daytime clinician. For patients with more than one diagnosis or goal, each one was coded separately. To calculate the percentage of correctly identified diagnoses or goals, we divided the number of correct diagnoses or goals identified by the nighttime clinician by the total number of diagnosis or goals reported by daytime clinicians. For example, if the daytime clinicians reported pulmonary edema in 100 patients, and the nighttime clinicians reported 70 diagnoses of pulmonary edema in these same 100 patients, the correct identification would be 70%. Due to the clustered nature of the data (patients may have more than one diagnosis or goal), we used generalized estimating equations to report the percentage of diagnoses and goals correctly identified by the nighttime clinician (14).

Back to Top | Article Outline

Accuracy of Anticipatory Guidance for Nighttime Events.

We analyzed anticipatory accuracy with sensitivity and specificity, using standard formulas. We compared daytime and nighttime clinician’s anticipatory accuracy using the area under the receiver operating characteristic curves (AUC) for two logistic regression models. Each model included nighttime event as the dependent variable and either daytime or nighttime clinician’s anticipations as the independent variable.

Other statistical comparisons are made with chi-square tests as appropriate, and alpha level is set at 0.05. All analyses were conducted on Stata 14 (Statacorp LLC, College Station, TX).

Back to Top | Article Outline

RESULTS

We surveyed 30 intensivists participating in 44 handovers, including a total of 352 individual patient handovers in 212 patients. Median age was 69 (interquartile range [IQR], 50–80.3), median Simplified Acute Physiology Score II was 47 (IQR, 37–56), and the median ICU length of stay at the time of handover was 1 day (IQR, 0–5 d). There was a median of 2 diagnoses (IQR, 1–4) and 1 goal (IQR, 0–1) per patient.

Daytime clinicians reported a total of 857 diagnoses, of which nighttime clinicians correctly identified 454 (53%; 95% CI, 50–56). There were 31 patients with diagnoses considered unclear by the daytime clinician, and the nighttime clinician identified only four (13%; 95% CI, 1–25%). Nighttime clinicians identified 306 diagnoses not originally reported by the daytime clinician.

We observed differences in the correct identification of diagnosis within each of the five problem categories (Fig. 1) (p < 0.01 for trend). In particular, patients with changes in level of consciousness were less likely to have the diagnosis correctly identified, and patients with shock were more likely to have the diagnosis correctly identified.

Figure 1

Figure 1

Nighttime clinicians correctly identified 123 of the 304 goals (40%; 95% CI, 35–46) reported by the daytime clinicians. The most common goals were to wean vasoactive agents (59 goals; 49% identified; 95% CI, 36–62), monitor C-reactive protein (50 goals; 48% identified; 95% CI, 34–63), manage electrolyte disturbances (29 goals; 41% identified; 95% CI, 23–61), promote a negative fluid balance (28 goals; 36% identified; 95% CI, 18–56), focus on end-of-life care (28 goals; 54% identified; 95% CI, 34–72), and others (110 goals; 28% identified; 95% CI, 20–38).

There were a total of 85 events in 72 individual patients at night. Daytime clinicians anticipated more potential complications than nighttime clinicians (411 vs 230; p < 0.01), and their predictions were more sensitive (65% vs 46%; p < 0.01) but had lower specificity (82% vs 91%; p < 0.01). The discriminatory ability of daytime and nighttime clinicians for any event was not different (AUC, 0.74 [95% CI, 0.68–0.79] vs 0.68 [95% CI, 0.63–0.74]; p = 0.09). The positive predictive value of both daytime and nighttime clinicians was low and not statistically different (13% vs 17%; p = 0.2) (Table 1).

TABLE 1

TABLE 1

Back to Top | Article Outline

DISCUSSION

We observed that among staff intensivists, diagnoses and goals of treatment are either not conveyed or retained in 50–60% of the cases immediately after a handover. Our data are the first to demonstrate this loss of information free of hindsight bias and in a cohort of staff intensivists. Furthermore, our study is the first to investigate clinicians’ abilities to anticipate nighttime events, which yielded a low positive predictive value of 13%.

Our study has several strengths. The most important is the use of immediate posthandover survey to identify missed information. Although missed information and errors of communication are commonly reported in the handover literature, the identification of these errors after assessing patients is biased by hindsight (8) and a fresh set of eyes (12). Although communication may be one of the sources of these errors, it cannot be excluded that they may also occur due to cognitive errors (15) (the person providing the handover did not identify a problem) or to the evolving course of a new diagnosis, which may become clear once more information is available over time. Our design effectively removes these two problems and suggests that missed information is an immediate phenomenon after a handover in a unit that does not use a structured format.

Previous handover studies suggest that up to 75% of nighttime events could have been anticipated and discussed during handovers (16). The ability to anticipate problems is thought to be a key element in ensuring patient safety and continuity of care (6 , 16). In fact, it is arguable that handover for a night shift would benefit from mostly focusing on anticipatory guidance to increase awareness of potential problems. Our data demonstrate that anticipation of events by daytime clinicians has a low positive predictive value due to a high number of anticipated events. This suggests that anticipation of events occurs at the expense of an increased amount of information during handover, which has two potential consequences. First, it may lead to information overload and cognitive fatigue (17), and second, it may lead to desensitization of providers, similar to “alarm fatigue” (18).

Our structured approach to measuring diagnosis and goals has two distinct advantages that can support further research in this field. First, it avoids coding open-text diagnosis post hoc, which may lead to biases of interpretation. Second, it provides a cue-sheet with information that facilitates the identification of common diagnosis and goals, decreasing the chance of missing elements due to inattention.

There are important limitations to our study. First, this is a single-center study, which impacts its external validity. In centers with structured methods for handover, the retention of diagnoses and goals may be higher; however, event anticipation is still a cognitive task for which no formal tools or training have been shown to improve. It is also important to highlight that staff intensivists do not use a structured format for handovers (10). Second, it is unclear whether the missed diagnoses or goals led to any undesirable outcomes. It is possible that these gaps in communication may not necessarily lead to lower quality of care; however, if by principle the provision of accurate and complete information is the gold standard, these gaps are unwanted, and their association with outcomes should be further explored. Third, given that handovers were not directly observed, it is not possible to identify whether the missed diagnoses and goals are due to omission of information by the daytime clinician or inadequate retention of information by the nighttime clinician. However, although mechanistically these are different problems, the same handover tools, such as preprinted lists of diagnoses and goals, can improve them. Fourth, we also observed that nighttime clinicians reported an additional 306 diagnosis not identified by the daytime clinician. Within the scope of this study, we cannot explain these findings and whether they represent misunderstanding, preconceived diagnosis about the patients from previous encounters, or diagnosis suggested by cues at the bedside that were not readily observed by the daytime clinician.

Our data also suggest two new potential areas for future research and for quality improvement. Almost 10% of patients had a problem considered unclear by the daytime clinician. However, the nighttime clinician did not identify almost 90% of these patients. Since patients with diagnostic uncertainty are more likely to have overtesting, increased healthcare expenditure and errors (19), missing this information potentially exposes a more vulnerable population. Unfortunately our study is limited by the lack of patient-centered outcomes, and we cannot make inferences about these findings. Communication gaps were more pronounced for patients with neurologic diagnosis. Correct identification of diagnosis and the accuracy of event anticipation were poorer for this group of patients. We suggest that these findings may be related to two distinct problems. First, these neurologic events could be sudden (such as new onset seizures) and impossible to anticipate; however, the most common neurologic events were delirium/agitation (data not shown), which are potentially predictable, and the most common missed neurologic diagnosis were changes in level of consciousness due to metabolic abnormalities and delirium. Second, it is possible that other problems, such as shock and respiratory failure, are usually self-evident to staff intensivists due to multiple and objective sources of information (monitors, ventilators, vital signs, laboratory), whereas neurologic status has very little objective information (other than intracranial monitors) and requires a more nuanced assessment, which may benefit from strategies such as brief combined clinical examination during handover, cognitive checklists (20), or handing over patients with changes in level of consciousness at the beginning of the handover, to allow for longer discussions (21). These strategies could be studied in the setting of a multifaceted intervention to improve handovers. Staff intensivists are an important group to target such interventions, as these gaps have not been previously described in this population, they do not use structured handovers (10), and, in a survey of North American intensivists, staff intensivists prefer succinct handovers with limited detail (11).

Back to Top | Article Outline

CONCLUSIONS

Among staff intensivists, diagnoses and goals of treatment are either not retained or conveyed in 50–60% of the time immediately after a handover. The ability of clinicians to anticipate events has a low positive predictive value of 13%, and the expectation that anticipatory guidance can help in patient safety needs to be balanced against information overload. Handovers among staff intensivists showed more relevant gaps in the identification of diagnostic uncertainty and for neurologic diagnoses, which could benefit from communication strategies such as cognitive checklists, prioritizing discussion of neurologic patients, and brief combined clinical examination at handover. Due to variation in practice and limited use of structured handovers, staff intensivists are an important population for clinical trials aiming to improve quality of care via handovers.

Back to Top | Article Outline

REFERENCES

1. Philibert IUse of strategies from high-reliability organisations to the patient hand-off by resident physicians: Practical implications. Qual Saf Health Care 2009; 18:261–266
2. Gandhi TK, Kachalia A, Thomas EJ, et alMissed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims. Ann Intern Med 2006; 145:488–496
3. Kachalia A, Gandhi TK, Puopolo AL, et alMissed and delayed diagnoses in the emergency department: A study of closed malpractice claims from 4 liability insurers. Ann Emerg Med 2007; 49:196–205
4. Petersen LA, Brennan TA, O’Neil AC, et alDoes housestaff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med 1994; 121:866–872
5. Fins JJProfessional responsibility: A perspective on the Bell Commission reforms. Bull N Y Acad Med 1991; 67:359–364
6. Arora V, Johnson J, Lovinger D, et alCommunication failures in patient sign-out and suggestions for improvement: A critical incident analysis. Qual Saf Health Care 2005; 14:401–407
7. Catchpole K, Sellers R, Goldman A, et alPatient handovers within the hospital: Translating knowledge from motor racing to healthcare. Qual Saf Health Care 2010; 19:318–322
8. Caplan RA, Posner KL, Cheney FWEffect of outcome on physician judgments of appropriateness of care. JAMA 1991; 265:1957–1960
9. Starmer AJ, Spector ND, Srivastava R, et alI-PASS Study Group: Changes in medical errors after implementation of a handoff program. N Engl J Med 2014; 371:1803–1812
10. Ilan R, LeBaron CD, Christianson MK, et alHandover patterns: An observational study of critical care physicians. BMC Health Serv Res 2012; 12:11
11. Lane-Fall MB, Speck RM, Ibrahim SA, et alAre attendings different? Intensivists explain their handoff ideals, perceptions, and practices. Ann Am Thorac Soc 2014; 11:360–366
12. Kajdacsy-Balla Amaral AC, Barros BS, Barros CC, et alNighttime cross-coverage is associated with decreased intensive care unit mortality. A single-center study. Am J Respir Crit Care Med 2014; 189:1395–1401
13. Wilcox ME, Chong CA, Niven DJ, et alDo intensivist staffing patterns influence hospital mortality following ICU admission? A systematic review and meta-analyses. Crit Care Med 2013; 41:2253–2274
14. Zeger SL, Liang KYLongitudinal data analysis for discrete and continuous outcomes. Biometrics 1986; 42:121–130
15. Croskerry PAchieving quality in clinical decision making: Cognitive strategies and detection of bias. Acad Emerg Med 2002; 9:1184–1204
16. Borowitz SM, Waggoner-Fountain LA, Bass EJ, et alAdequacy of information transferred at resident sign-out (in-hospital handover of care): A prospective survey. Qual Saf Health Care 2008; 17:6–10
17. Guastello SJ, Boeh H, Gorin H, et alCusp catastrophe models for cognitive workload and fatigue: A comparison of seven task types. Nonlinear Dynamics Psychol Life Sci 2013; 17:23–47
18. Winters BD, Cvach MM, Bonafide CP, et alSociety for Critical Care Medicine Alarm and Alert Fatigue Task Force: Technological distractions (part 2): A summary of approaches to manage clinical alarms with intent to reduce alarm fatigue. Crit Care Med 2018; 46:130–137
19. Bhise V, Rajan SS, Sittig DF, et alDefining and measuring diagnostic uncertainty in medicine: A systematic review. J Gen Intern Med 2018; 33:103–115
20. Ely JW, Graber ML, Croskerry PChecklists to reduce diagnostic errors. Acad Med 2011; 86:307–313
21. Cohen MD, Ilan R, Garrett L, et alThe earlier the longer: Disproportionate time allocated to patients discussed early in attending physician handoff sessions. Arch Intern Med 2012; 172:1762–1764
Keywords:

communication; cognitive error; critical care; handover; patient safety; quality

Supplemental Digital Content

Back to Top | Article Outline
Copyright © by 2018 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.