Effective communication is essential to patient safety,1–4 and communication failure is a common contributor to adverse events.2,5,6 Interprofessional communication failures, particularly during care transitions, can exacerbate gaps in care, leading to errors.7,8 Transitions of care are typically associated with a handover, often between clinicians with different skill sets, knowledge, training, perspectives, and expectations.9 The postoperative handover between anesthesia providers (APs) and postanesthesia care unit (PACU) registered nurses (RNs) is one such interprofessional care transition.
Successful handovers prevent unwarranted changes in goals, decisions, priorities, or plans and help to prevent missing tasks or redoing those performed by the previous person in the role.10,11 Handovers (also called handoffs or sign outs) have been studied, particularly in physicians-in-training, with the introduction of work-hour restrictions and the consequent increase in shift changes.12,13 Multiple studies have identified numerous issues with handovers, including inadequate training in how to perform effective handovers, a lack of a standardized process, clinicians’ inadequate interpersonal/communication skills, insufficient time for effective transfer of information, information loss, numerous distractions and interruptions, and limited opportunity (or cultural impediments) to ask questions or voice concerns.14–16 One observational study found that physicians and nurses with higher levels of clinical training who were receiving a handover verbal update caused fewer interruptions and did more assertive questioning to detect erroneous statements than clinicians with lower levels of training.17 Nevertheless, there is still insufficient research to be able to identify the most critical attributes that determine handover success or failure under different circumstances.9
We were particularly interested in studying PACU handovers because flawed handovers could lead to subsequent perioperative problems, and previous research suggests that these handovers are less than optimal.14,18–21 For example, small observational studies found PACU handovers varied appreciably in content and duration.14,18,19,22 In another study, PACU RNs were dissatisfied with 52% of handovers, and more than half of the observed handovers were judged to be of poor quality.20
We sought to test the hypothesis that a multimodal intervention that included an electronic handover form and simulation-based interprofessional training of AP and PACU RN personnel would significantly improve the effectiveness of actual PACU handovers (i.e., positive transfer of training).23 The goal was to substantially change the practice behavior of a large cohort of clinical professionals.
Study Venue and Participants
The study was approved by the Vanderbilt Human Subjects Protection Program and was conducted in an adult PACU at Vanderbilt University Hospital (VUH) and a pediatric PACU at Monroe Carell Jr. Children’s Hospital at Vanderbilt. In both units, baseline field observations of actual patient handovers preceded the simulation training (Fig. 1). The study cohort consisted of clinicians who performed handovers from anesthesia care to postanesthesia care—the PACU RNs and the APs, including anesthesiology residents and Certified Registered Nurse Anesthetists, in the 2 study units. As an institutional quality improvement (QI) project, simulation-based handover training sessions were mandatory for all PACU clinicians, excluding faculty because faculty do not typically conduct handovers. For data collection during actual handovers, the study received IRB approval and a waiver of written, informed consent. All PACU clinicians were assigned a random confidential code number to link training status to observed handovers.
We designed our handover improvement curricula using a systems approach,24 based on modern learning theory,25,26 practical training methods (e.g., skill acquisition,27 deliberate practice,28 and reflective practice29), and simulation-based training techniques.30–32 A Curriculum Committee was formed consisting of AP and RN educators, a provider-provider communications expert, and 2 faculty experienced in developing simulations using standardized patients and standardized clinicians (SCs)—trained actors portraying, in this study, either a patient, an AP, or a PACU RN.
Initially, 2 Committee members observed >50 actual PACU handovers using an ethnographic approach. Results were coded and then discussed in the Committee. Consistent with the extant literature,10,12,14,15 pre-study findings included significant variability in handover content, detail, organization, and communication style and in the use of different paper-based handover report forms.
Electronic Handover Report Form
Because each PACU used a (different) written handover report form, for consistency across units, we created a standardized electronic form. This was an opportunity to incorporate the Situation Background Assessment Recommendation (SBAR) handover structure33 to be taught in a webinar and during simulation training. The researchers worked closely with informaticians and front-line PACU providers to create the eHandover report (Fig. 2) that automatically printed in the PACU when the surgeon was noted to be closing the incision. The new tool was deployed in all PACUs before initiation of simulation training.
The Committee developed a Web-based instructional seminar (webinar) to introduce all clinicians to the project. The webinar described the handover roles and responsibilities of the clinicians, barriers to effective handovers, strategies to overcome these barriers, and the nature of the planned simulation training. Webinars were delivered via the same mechanism as other institutional Web-based training.
As a key component of the introductory didactic webinar, 10 video vignettes were crafted to represent both effective and ineffective handovers. These vignettes were derived from direct observations, reports from clinicians, and the hospital event-reporting system. The scenario scripts were vetted by a panel of expert clinicians for accuracy and authenticity and were videotaped using simulated clinicians (see Table, Supplemental Digital Content 1, http://links.lww.com/AA/B90, which provides representative scripted scenarios for initial and refresher course webinars). The video vignettes were also used to train study observers and to test and validate the instruments subsequently used to observe and score live handovers.
Four simulation scenarios were created to provide trainees with reproducible opportunities to perform handovers related to the course objectives.34 The scenarios were based on specific communication behaviors and obstacles to successful handovers (see Table, Supplemental Digital Content 2, http://links.lww.com/AA/B91, which provides summaries of the 4 scenarios used in the initial simulation training sessions). Educational objectives were refined by the team and validated by other experienced clinicians. The training was designed to address both the specifics of the SBAR PACU handover—what information needs to be transmitted and the structure within which it should be transmitted—and the specific guidance on effective interpersonal communication strategies. A goal was to emphasize that how one communicates is as important as what one communicates.
Thus, the general curricular objectives from which all the scenarios derived were:
- Critical information content and use of a structured/standardized approach (e.g., SBAR);
- Identifying and emphasizing the most important information;
- Sensitivity to the work requirements of the other person, including tasks to be accomplished before, during, and after the handover;
- Listening carefully;
- Reading nonverbal cues;
- Attention management—particularly dealing with interruptions and distractions;
- Task prioritization—balancing patient care demands and communication requirements;
- Dealing with time pressure and with competing priorities; and
- Being appropriately assertive when one’s (or the patient’s) needs are not being met.
Specific objectives and training points were developed for each scenario. The scenarios, based on actual cases, were refined through iterative cycles of pilot testing and review of findings.35,36 The scenarios required an intensive level of scripting and SC training because of the wide range of possible interactions with trainees and to ensure that a realistic PACU environment was replicated. The final scenarios, modeled by instructors and SCs, and practiced by the trainees, emphasized the desired communication behaviors. Each scenario was designed to elicit a range of realistic responses from trainees, thus facilitating post-scenario debriefing.
The 4 scenarios were modified for use in the children’s hospital simulation training. Without changing the educational objectives or learning points, we created 4 new clinical vignettes based on actual pediatric anesthesia cases. The only substantive changes were the substitution of pediatric anesthesia and nursing relevant clinical content. Two new adult scenarios were developed for the refresher course (see Refresher Course).
A mannequin simulator (SimMan™ or SimBaby™ Laerdal Medical, Wappingers Falls, NY) or an actor was used, as appropriate, as the emerging patient in each of the scenarios. To support the learning goals of each scenario, the responses of the mannequin were programmed or the actor was trained to respond to trainees’ actions and behaviors. In all training scenarios, the PACU clinician trainee either gave or received a handover to or from an SC, respectively. The SCs were trained to consistently portray their clinical role accurately and to respond appropriately to the trainee.
Handover Evaluation Tool Development
Evaluation instruments were refined in parallel with curriculum development. The handover evaluation tool was created with substantial input from the Curriculum Development Team. This tool was iteratively refined and pilot tested by conducting handover observations first from the simulated handover videos and then in PACUs that were not study sites. To ensure that the content validity of the tool was acceptable, and that it included all of the relevant handover components, behavioral anchors, and checklist items, the pilot-tested version of the evaluation tool was reviewed by experienced PACU RNs and APs who were not going to participate in the intervention. Finally, an expert panel of attending anesthesiologists and experienced PACU RNs individually used the evaluation tool to review and rate handovers in each video vignette. Subsequently, the panel convened to discuss their individual ratings as a group to ensure sufficient plausibility and realism. The ratings of the panel were also used as the scoring standard by which observer trainees were evaluated.
The final instrument (Fig. 3) contained 8 ordinal rating scales reflecting attributes of handover quality (introduction/social greeting, readiness for report, elements of handover information/content transmitted [SBAR elements], content organization/clarity, content completeness, comprehension confirmation, level of engagement, and coordination/conflict resolution) and a global score. Most items were rated on 5-point scales, with each rating anchored by specific behavioral markers. The global performance score assessed the overall effectiveness of a handover, from 1 (not at all effective) to 5 (extremely effective).37,38
Four nurses were trained to observe and rate PACU handovers. Multiple videotaped handovers as well as 16 hours of actual handovers were observed and scored with an experienced mentor (JMS) who provided feedback.39 The observer ratings of simulated handover videos were iteratively compared with the ratings agreed upon by the expert panel to achieve reliability, defined as matching both the global rating and at least one-half of the component ratings, as well as being within one rating point on all component ratings. Rater stability was reconfirmed via video assessment of every 50 handover observations and by random monitoring via field observations by the trainer/mentor.
Simulation Training Courses
Initial Training and Course Logistics
The 2 scheduled AP trainees per session received an e-mail at least 2 days before their scheduled training that instructed them to study brief but detailed clinical synopses of the patients they would encounter during the session. These synopses replicated preoperative evaluation documentation as well as actual electronic anesthesia care records edited to match the patient characteristics they would encounter during training. RN participants received only the applicable eHandover report form immediately before the handover. Participants (all of whom had been informed they would attend handover training) were briefed that they would perform a practice handover with a colleague before the training and then again afterwards to see what they had learned. Faculty anesthesiologists were excluded from the simulation training sessions because at our institution: (1) faculty are rarely, if ever, actively involved in PACU handovers, and (2) there were not enough PACU RNs to allow the additional interprofessional sessions.
The 4 trainees in each simulation session, 2 APs and 2 RNs, were relieved from clinical duties during regular work hours to attend the 2-hour course, taught in our Center for Experiential Learning and Assessment by both a faculty anesthesiologist and a perioperative nurse educator. Each session began with a brief introduction to the study and the training. Then, the 4 trainees were paired (AP1 with RN1 and AP2 with RN2) and tasked to perform a routine handover in the simulated PACU that served as a pre-training baseline. Thereafter, the clinicians individually participated in 1 of the 4 training scenarios during which nonparticipants unobtrusively observed their colleagues, mostly via a video feed. The trainee conducted a handover with an SC and either a human or mannequin-simulated patient in a realistic PACU setting. The SC was a nonclinician actor who was specially trained (by an experienced standardized patient educator) over several weeks to accurately and reproducibly portray a role (AP or RN) that elicited specific learning objectives of that scenario. Immediately after each scenario, instructor-facilitated peer debriefings occurred that generally lasted longer than the preceding scenario.
During the last 10 minutes of the session, alternately paired AP and RN learners (AP1 with RN2 and AP2 with RN1) performed a post-training simulated handover.40 The instructors concluded the session with summary feedback and by soliciting questions and concerns. All clinician participants were asked to complete an anonymous course evaluation form.
Beginning 6 months after their initial course, the VUH PACU clinicians returned for a 1-hour simulation-based refresher course. This course included 2 new training scenarios that addressed attributes that remained a challenge in actual PACU handovers. The refresher curriculum emphasized interprofessional communication techniques, including the need to speak up when safety issues arise during a handover.
A new 15-minute webinar preceded the refresher training and incorporated a new video case focused on speaking up, a proactive role for an AP or RN to ensure a safe and effective handover (see Table, Supplemental Digital Content 1, http://links.lww.com/AA/B90, which provides a representative scripted scenario for the refresher course webinar).
Because of the resource intensity of the simulation-based training course, in the latter part of the intervention period, we modified the initial course for a cohort of new (untrained) trainees. We video recorded SCs playing both roles from the 4 original adult handover scenarios. The scripts were based on the experience of the team with the previous 61 initial course sessions and were designed to elucidate the core teaching points and optimize the value of the intended post-video debriefing period. For each of the scenarios, we created 2 video versions: one that exemplified a best practice of how a trainee should respond to the situation presented and one that exemplified the opposite response to the situation. With the exception of trainees viewing the videos rather than performing handovers with SCs, the conduct and logistics of this course were identical to the earlier initial course sessions.
Observation of Actual PACU Handovers
Starting before any intervention, trained nurses observed and scored in real-time actual handovers in the 2 PACUs during daytime shifts, generally between 12 PM and 5 PM. Six to 12 cases were observed each week in each PACU. Sampling was stratified according to time of day and day of week to assure that handovers occurring on different days of the week and different times of day were studied. All observational data in the PACUs were collected in the same way throughout the study (i.e., both before and after the training intervention). Research nurses, who were blinded to study hypotheses and who did not practice in the study site, were assigned to observe and score handovers, generally unaware of training and experience of participants. For each handover, the observer noted the date/time and recorded the names of the providers, which were converted to random ID numbers. The patient age, gender, and American Society of Anesthesiologists (ASA) risk classification41 were obtained from medical records.
For purposes of data analysis, we categorized the observed and rated AP-RN dyads as either trained or untrained in the following way: A trained dyad was an AP-RN pair doing a PACU handover in which either one or both of the individuals had previously participated in the simulation-based training. An untrained dyad was an AP-RN pair doing a PACU handover in which neither one of the individuals had previously participated in the simulation-based training.
Feedback to Providers
Periodically (i.e., monthly or bimonthly), we distributed to the PACU clinicians statistical process control charts that depicted overall average scores of a unit for handover effectiveness by date. Electronic and posted materials (i.e., flyers posted at the study sites/units) summarized the results of the period and encouraged improvement in specific areas.
QI Follow-Up Project
To assess the retention of the handover effectiveness improvements observed in the original study, we conducted a QI follow-up project using the same methods and observers to measure ongoing handover acceptability between January and May of 2012 at the adult hospital and between February and June of 2012 at the children’s hospital. The same nurse researcher observed and rated, at each PACU location, an additional 39 handovers. Although the relatively small sample size does not allow for modeling to compare those who were trained with those who were not trained at each location, it permits an evaluation of the retention of the improvements in handover effectiveness 3 years after the original study. Moreover, because this was a post hoc analysis, we did not combine these subsequent data with the original study data.
The primary analyses sought to examine (1) the impact of training on handover effectiveness over time and (2) the measured handover attributes (subelement rating items) that were predictive of the global ratings.
Based on pilot observations, we expected that most pretraining handovers would receive global ratings <3 (low effectiveness). Thus, the prestated objective of our intervention was to improve actual handovers post-training, so that a preponderance received global effectiveness ratings of ≥3. For ease of interpretation (with a recognized loss in estimation precision), we dichotomized the global handover performance score as either acceptable (≥3) or unacceptable (<3). The statistical model permitted hypothesis testing for the impact of training over time and directly produced the log odds of acceptable handovers as a function of the independent variables. Logistic regression was used with acceptable handover (yes/no) as the dependent variable and the main and interaction effects of training status (yes if either the AP or the RN was trained and no if not), time since study initiation, and location (adult or pediatric PACU) to examine the impact of training over time. To control for potential confounding, we included patient age, gender, ASA physical status, rater ID, the amount of time the AP and RN had each been on duty that day, and the interactions between location and patient age, ASA physical status, and clinician’s time on duty as adjustment covariates. Periods of intervention were parameterized so that their effect could differ between baseline, post-baseline to refresher, and post-refresher time periods using piecewise linear splines. For ease of exposition, we transformed the log odds and report the models’ results on the probability scale. To account for the correlation introduced by observing APs and RNs repeatedly over time, model parameters were estimated using generalized estimating equations with a conservative independence working covariance structure along with robust standard error estimates.42 By assuming an independence/diagonal working covariance structure, we lose efficiency relative to other covariance structures (e.g., compound symmetric), but the results are more likely to be valid because of the potential violations of the full covariate conditional mean assumptions.43,44 Model fit was assessed graphically and using the Hosmer-Lemeshow goodness-of-fit test.45
Attributes and Global Ratings
A multiple logistic regression model with “acceptable handover” as a binary-dependent variable examined the relations between the attributes (subelement rating items) and the global handover performance score. The rated items were included in the model simultaneously, and model parameters were estimated using generalized estimating equations with an independence working covariance structure along with 95% confidence intervals (CIs) using robust standard errors estimates. The importance of each variable was also calculated as the difference in the likelihood ratio χ2 value and the degrees of freedom.
All analyses were performed in R version 2.15 or later.46
We conducted 112 simulation training sessions for 151 APs and 160 RNs, as seen in Table 1. This included 61 initial courses (35 at the adult hospital and 26 at the children’s hospital), 40 refresher courses for 75 APs and 74 RNs (adult hospital), and 11 video courses for 20 APs and 19 RNs. In the PACU, 981 handovers were observed between 344 unique providers (225 APs and 119 RNs).
Observer Ratings of Actual PACU Handovers
Between July 2007 and November 2008, 981 handovers were observed: 389 in the pediatric PACU and 592 in the main adult PACU (Table 2).a Four handovers (<0.5%) were missing model–required covariates and thus were excluded from regression analyses. The observed handovers involved 225 APs and 119 RNs with >90% involving unique AP-RN pairings. Handovers during the baseline (pre-training) period in the adult PACU were all performed by untrained clinicians. However, 19.6% of handovers during the baseline phase of pediatric PACU included APs who had received the adult PACU simulation training (i.e., anesthesia residents and Certified Registered Nurse Anesthetists who worked at both hospitals). Previous time on duty of the AP-RN and observed patient characteristics were comparable within locations across study phases.
Table 3 summarizes observers’ ratings from 977 handovers during 17 months. The results supported our hypothesis that the intervention would improve the quality of actual PACU handovers. The proportion of acceptable handovers increased significantly from 7% (95% CI, 3%–17%) to 70% (63%–76%) in the adult PACU from the baseline to the post-refresher time period and from 22% (17%–29%) to 72% (65%–79%) in the pediatric PACU from baseline to the post-training period.
The outcome used in all regression analyses was the overall global performance score provided by those reviewing each handover after they rated the handover on a set of subelements. Because the global score is not an explicit multidimensional instrument, it is important to describe the subelement items that implicitly contributed to the handover acceptability outcome. Table 4 provides a summary of each rated subelement and the results from a multiple logistic regression analysis of handover acceptability (i.e., acceptable/not acceptable) on all of the subelement items. The goal of this analysis is to describe the factors that implicitly contributed to handover acceptability as assessed by handover reviewers. The overall R2 for this model was 0.88, which implies that the weighted sum of these items (weights based on log odds ratios) can be used to predict the vast majority of the variability in the binary acceptability outcome. In addition to odds ratios, CIs, and P values for each item, we also provide a measure of variable importance based on the χ2 likelihood ratio statistic minus the degrees of freedom. This quantity permits a ranking of variable importance and is 0 if a variable provides no information for predicting the outcome (see Ref. 47). Six variables predicted most of the variation in handover acceptability, all of which have P values <0.001 and variable importance values >10. The variables include (in rank order): Organization and clarity, Comprehension, SBAR: situation, SBAR: assessment, SBAR: recommendation, and Introduction/greeting. Even though the acceptability outcome variable is not an explicit multidimensional instrument, it can be predicted with near certainty using the subelement items.
Figure 4 shows the effects of training on handover acceptability over time (after study initiation). Handovers involving at least one trained participant (i.e., trained dyads) resulted in immediate improvements. In the adult PACU, the percent of acceptable handovers was 18% (95% CI, 11%–28%) at day 40 (end of baseline), 56% (46%–66%) at day 225 (roughly 6 months after start of simulation training), and 68% (57%–76%) at day 405 (1 year after start of simulation training), and so the odds ratio of observing an acceptable handover was 5.68 (95% CI, 3.30–9.76) comparing day 225 to day 40, and 9.28 (5.08–17.0) comparing day 405 to day 40, and 1.63 (1.12–2.38) comparing day 405 to day 225. Similar improvements in handover acceptability for trained dyads were observed in the pediatric PACU (Fig. 4). The scores of handovers by untrained dyads (i.e., neither AP nor RN previously trained) also improved over time. The percent of acceptable handovers was 10% (95% CI, 5%–19%) at day 40 (end of baseline), 24% (11–45%) at day 225, and 57% (33–78%) at day 405 and thus the odds ratio of observing an acceptable handover was 2.68 (95% CI, 0.82–8.71) comparing day 225 to day 40, 11.32 (3.44–37.32) comparing day 405 to day 40, and 4.22 (1.29–13.79) comparing day 405 to day 225.
In the QI follow-up period, approximately 3 years after completion of the original intervention observation period (Fig. 1), about one-half of the participants in the 78 observed handovers (49% at adult hospital and 59% at children’s hospital [see Table, Supplemental Digital Content 3, http://links.lww.com/AA/B92, which provides attributes of observed handovers by PACU and phase for 3-year follow-up period]) had received the original handover improvement training. In the adult PACU, 87% (95% CI, 72–95) of the handovers received global rating scores of ≥3 (see Table, Supplemental Digital Content 4, http://links.lww.com/AA/B93, which provides handover performance scores and other outcome measures for 3-year follow-up period). Using the Pearson χ2 test, this is a significant improvement (P < 0.001) compared to the 7% (3–17) with acceptable scores observed during the study baseline period (Table 3) and is even a marginally significant improvement compared to the 70% (63–76) of acceptable handovers in the post-refresher period (P = 0.045). In the pediatric PACU, the proportion of handovers receiving acceptable global handover rating scores of ≥3 was 56% (95% CI, 40–72), a significant improvement compared to the 22% (17–29) observed during the baseline period (P < 0.001) and not statistically different from the 72% (65–79) of acceptable handovers in the post-training period in the pediatric PACU (P = 0.072).
After a multimodal interprofessional training intervention, APs and RNs in large adult and pediatric PACUs in an academic medical center were rated as performing significantly more effective handovers. Thus, mostly experienced clinicians’ real-world handover practices improved significantly as a result of the intervention. After full implementation, the handovers of new AP and RN clinicians, those who had not received the simulation training portion of the intervention, were rated the same as those of clinicians who had received formal training. As a result, a new level of handover quality was attained and maintained and, in the adult PACU, potentially improved further >3 years post-intervention. In the pediatric PACU, although the high level of post-intervention improvement diminished somewhat in later years, performance of adequate handovers was still much greater than in the preintervention period.
What May Have Contributed to the Initial Improvements in Handover Ratings?
The marked increase in handover ratings from baseline to the end of the study could have been attributed to many factors. The design of our study cannot distinguish the effects of the different intervention elements. The eHandover form (i.e., automatically printing out with every case) was initiated in the adult PACU before the webinar or simulation training. The eHandover form provided the SBAR structure, but the clinicians did not receive training in its use until taking the webinar, just before simulation training. Observations and printing out of the eHandover form started in the pediatric PACU before the simulation training of those providers, but the initial effects seen could be explained by cross-contamination of already trained anesthesia residents. Initial improvements in handover ratings appear to be associated with the proportion of clinicians in that PACU who had undergone simulation-based training. The simulation experience added practical mentored experience in the use of eHandover tool and focused training in critical interpersonal communication skills. It is possible that there was an additional effect of the structured observations and direct performance feedback. Thus, future studies may more clearly define the relative contributions of different components of this intervention.
What Contributed to the Improvements in Handover Ratings of Untrained Clinicians?
In the main study, the improved handover ratings were maintained for >7 months after pediatric PACU participant training was completed and for >5 months after the second (refresher) simulation course for the adult PACU participants. Possible explanations for the maintenance of the effect during the study include ongoing observations (e.g., the Hawthorne effect), the continual reminder provided by the eHandover printouts at the patient bedside in each PACU, monthly handover performance feedback, or unofficial actions of champions and early adopters. Why, by the end of the study period, were the handover ratings of untrained dyads statistically indistinguishable from those of clinicians who received the simulation-based training? We see no other explanation than that there was some kind of in situ learning by AP and RN clinicians who had never received the simulation training. The ongoing printout and bedside availability of the eHandover form invariably played a role in this learning. One can suppose that previously trained providers taught new APs and RNs either explicitly (this is how we do handovers here) or implicitly through role modeling of the proper use of the eHandover form. Many experts in the business, human factors, and the social science fields would call this a unit-level or organizational culture change.48–50 Future studies should include prospective hypotheses and measures to assess the mediators of organizational changes similar to those observed in the present study.
Follow-Up Assessment of Handover Effectiveness
Three years after completion of the original study, we conducted a QI project using the same methods and observers to assess handover effectiveness in the 2 study PACUs. Handover ratings remained significantly better than during the pre-study baseline period and were at least as good as post-refresher levels. The handover effectiveness in the adult PACU appeared to be better than that in the pediatric PACU. Although this may be attributed to sampling error, it is notable that handovers 3 years later in the adult PACU took longer (7.0 ± 4.4 minutes) compared to those at the end of the formal study (6.4 ± 3.8 minutes) but had become briefer in the pediatric PACU (4.8 ± 2.7 vs 6.1 ± 3.2 minutes). However, because analyses could not be adjusted for potential confounders, including those associated with temporal trends in the 3 years since the end of the formal study, one should not overinterpret these results. Nevertheless, the results are sufficiently robust to suggest that there was not appreciable regression toward pre-study baseline handover performance. Whether improvements persist during a longer follow-up period will need to be examined.
Why Is This Study Important?
The goal of this study was to advance our knowledge of how to change real-world clinician behavior in an area, care transitions, known to be associated with degraded care quality and potential patient harm.2,5–8 Deficient handovers are almost always because of communication failures between the handover giver and receiver. We believed that a handover improvement intervention that solely addressed handover mechanics (e.g., use the SBAR format) would not be compelling, effective, or sustainable, especially for experienced clinicians. Thus, we designed our training intervention to include practical training in interpersonal skills (e.g., being sensitive to the other person’s needs, speaking up when your own needs are not being met). Moreover, we believed that a single training experience, no matter how effective and powerful, would not be sufficient to sustain behavior change.51 Thus, we used a multimodal approach that included a continual in-the-moment bedside reminder (the eHandover form) as well as post-training performance feedback.
This study supplements the existing literature on handovers10–17 while addressing several gaps. There are studies that have shown that high-fidelity clinical simulation training can improve teamwork6 although none has focused specifically on handovers. To our knowledge, no one had previously shown unambiguous changes in multidisciplinary clinicians’ handover behavior in the real world after an improvement intervention. Perhaps most importantly, few if any QI interventions have demonstrated sustained levels of performance for 3 years with no attention to the initiative other than the continued use of the eHandover form. Finally, we note that a video-based course is as well received, and may be as effective, as live interactions with SCs, at least in the context of a broader multimodal intervention.
Our study did not measure any patient outcomes. However, given the very low modern risk of preventable anesthesia-related major morbidity and mortality, it is extremely difficult to power an intervention study to accomplish this.52 A general approach in any domain where serious adverse outcomes are uncommon but highly undesirable (e.g., nuclear power, aviation) is to identify and study meaningful process measures that are reasonable surrogates of adverse outcomes (i.e., if this process deviation happened enough times, a bad outcome will likely result). Similarly, in teamwork and communication improvement interventions, changes in observable behavior (and resulting actions or inactions) are used as a primary dependent measure, and these surrogates have been tied to better patient outcomes.6,53,54
It is possible that the effects observed were attributed to experimental bias. Ongoing observation of the clinician participants may have led to improved performance (i.e., a Hawthorne effect). However, performance did not improve during the pre-training period when clinicians were also observed multiple times. Moreover, in far more invasive observational studies (e.g., during live video recording of actual cases), clinicians rapidly and consistently become inured to the camera and the research observers.55,56
Because the intervention was multimodal, any (or all) of the components could have contributed to the changes in ratings observed. The observers/raters, who underwent a rigorous training protocol and were regularly tested for reliability, were unaware of the goals of the study or the content of the multimodal intervention. Thus, we believe that the ratings were reliable and unbiased throughout the study. The use of a multimodal QI intervention seemed reasonable since, a priori, we did not know whether we could change the behavior of practicing clinicians, most of whom had been doing PACU handovers the same way for years. Thus, we wanted an intervention with the highest likelihood to produce substantive and potentially sustained behavior change, which has been shown to be difficult to accomplish in health care. Now, future studies might examine how each of the modal elements actually contributed to the effect demonstrated in this initial multimodal study.
This study was conducted with federal grant funding by an experienced research team to design, implement, and evaluate innovations for handovers in care. It demonstrates key features of a tool for structuring handovers and strategies for staff development in coordination of handovers. We do not yet know how well this intervention will generalize to other health care facilities or other types of handovers. The goal of this study was to advance our knowledge of how to change real-world clinician behavior in a meaningful manner and provide evidence on the potential impact of system engineering approaches to handovers. Even in highly successful QI interventions such as those intended to eliminate central line–associated blood stream infections57 or the conduct of preoperative briefings,58,59 there is frequently customization on a local level with different tools showing variable effectiveness based on the local context (i.e., culture and environmental factors). A logical next step would be to replicate this intervention at one or more other clinical institutions. We have provided, and are willing to continue to provide, consultation, conceptual framework, and other materials (training scripts, videos, and rating forms) used in our project to other hospitals at no cost.
This research study was estimated to cost almost $600,000 with approximately 60% coming from direct grant support and approximately 40% from institutional in-kind support for replacement workers, nurse observers, and simulation instructors. However, at a cost of approximately $1300 per trained clinician, equivalent to the cost of sending a clinician to a multi-day out-of-town continuing education conference, this intervention changed experienced clinicians’ actual practice behavior and altered the culture of communication in our PACUs for up to 3 years after training.
In conclusion, we introduced a multimodal intervention that included a new electronic handover report form, a didactic webinar, mandatory simulation training focused on improving interprofessional communication, and post-training performance feedback. After the intervention, APs and RNs in large adult and pediatric PACUs in an academic medical center were rated as performing significantly better handovers. Thus, mostly experienced clinicians’ real-world handover practices improved significantly as a result of the intervention. The results of the study support the hypothesis that a multimodal improvement intervention can significantly enhance the effectiveness of interprofessional PACU handovers, and the effects appear to be maintained for several years.
A multimodal intervention that generates improvements such as changing clinicians’ real-world practice behavior and then maintaining that change in the face of myriad competing priorities and initiatives is significant, given that creating an intervention that produces sustained behavior change has proven difficult to accomplish in health care. Future research can delineate the relative contributions of different elements of such multimodal interventions to both adoption and sustainability of behavior change.
Name: Matthew B. Weinger, MD.
Contribution: This author helped in study conception, experimental design, scenario design, measurement instrument design, survey design, data analysis and interpretation, and manuscript preparation.
Attestation: Matthew B. Weinger approved the final manuscript. He attests to the integrity of the original data and the analysis reported in this manuscript and is the archival author.
Name: Jason M. Slagle, PhD.
Contribution: This author helped in study conception, experimental design, measurement instrument design and reliability assessment, data analysis and interpretation, and manuscript preparation.
Attestation: Jason M. Slagle approved the final manuscript. He attests to the integrity of the original data and the analysis reported in this manuscript.
Name: Audrey H. Kuntz, EdD, RN.
Contribution: This author helped in study conception, scenario design, measurement instrument design, data interpretation, and manuscript preparation.
Attestation: Audrey H. Kuntz approved the final manuscript.
Name: Jonathan S. Schildcrout, PhD.
Contribution: This author helped in data analysis plan, data analysis and interpretation, and manuscript preparation.
Attestation: Jonathan S. Schildcrout approved the final manuscript.
Name: Arna Banerjee, MD.
Contribution: This author helped in scenario design, measurement instrument design, data interpretation, and manuscript preparation.
Attestation: Arna Banerjee approved the final manuscript.
Name: Nathaniel D. Mercaldo, MS.
Contribution: This author helped in data analysis plan, data analysis and interpretation, and manuscript preparation.
Attestation: Nathaniel D. Mercaldo approved the final manuscript.
Name: James L. Bills, EdD.
Contribution: This author helped in scenario design, measurement instrument design, and manuscript review.
Attestation: James L. Bills approved the final manuscript.
Name: Kenneth A. Wallston, PhD.
Contribution: This author helped in measurement instrument design and reliability assessment.
Attestation: Kenneth A. Wallston approved the final manuscript.
Name: Theodore Speroff, PhD.
Contribution: This author helped in experimental design, data interpretation, and manuscript preparation.
Attestation: Theodore Speroff approved the final manuscript.
Name: Emily S. Patterson, PhD.
Contribution: This author helped in experimental design, data interpretation, and manuscript preparation.
Attestation: Emily S. Patterson approved the final manuscript.
Name: Daniel J. France, PhD, MPH.
Contribution: This author helped in experimental design, survey design, data analysis and interpretation, and manuscript preparation.
Attestation: Daniel J. France approved the final manuscript.
This manuscript was handled by: Sorin J. Brull, MD, FCARCSI.
Besides the authors, the other members of the postanesthesia care unit (PACU) Handover Improvement Project Team, all of whom contributed to the conduct of the study, were (alphabetically) as follows: Rebecca Arndt, Ray Booker, Edward Byrd, Judith Hall, Susie Leming-Lee, Abeer Madbouly, William T. O’Byrne, Debianne Peterman, James W. Pichert, Eric Porterfield, Lisa Rawn, Dila Vuksanaj, Chuan Zhou, and Matthew Zimmerman. The authors acknowledge the leadership vision and unwavering support of Warren Sandberg and Michael Higgins (Anesthesiology) and Nancy Feistritzer (Perioperative Nursing) as well as the PACU nurse managers and our Anesthesiology Department’s divisional and clinical management team without whom this project would not have been possible. The authors recognize the extensive editorial assistance of Martha Tanner. The authors also acknowledge the administrative and/or technical contributions of the following individuals: Autumn Y. Clark, John Heidekooper, Andrew Cross, and Lori M. Kelly.
a The observation sampling strategy was based on the availability of the observers with a goal of 10–15 observations/week per study site. We deliberately sampled across time of day and day of week.
1. Institute of Medicine Committee on Quality Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. 2001 Washington, DC National Academy of Sciences
3. Kohn LT, Corrigan JM, Donaldson MS To Err is Human: Building a Safer Health System. 1999 Washington, DC National Academy Press
4. Singer JI, Dean J. Emergency physician intershift handovers: an analysis of our transitional care. Pediatr Emerg Care. 2006;22:751–4
5. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14:401–7
6. Risser DT, Rice MM, Salisbury ML, Simon R, Jay GD, Berns SD. The potential for improved teamwork to reduce medical errors in the emergency department. The MedTeams Research Consortium. Ann Emerg Med. 1999;34:373–83
7. Cook RI, Render M, Woods DD. Gaps in the continuity of care and progress on patient safety. BMJ. 2000;320:791–4
8. Beckman U, Gillies D, Berenholtz S, Wu A, Pronovost P. Incidents relating to the intrahospital transfer of critically ill patients. Intens Care Med. 2004;30:1579–85
9. Cohen MD, Hilligoss PB. The published literature on handoffs in hospitals: deficiencies identified in an extensive review. Qual Saf Health Care. 2010;19:493–7
10. Patterson ES, Roth EM, Woods DD, Chow R, Gomes JO. Handoff strategies in settings with high consequences for failure: lessons for health care operations. Int J Qual Health Care. 2004;16:125–32
11. Patterson ES, Wears RL. Patient handoffs: standardized and reliable measurement tools remain elusive. Jt Comm J Qual Patient Saf. 2010;36:52–61
12. Arora VM, Manjarrez E, Dressler DD, Basaviah P, Halasyamani L, Kripalani S. Hospitalist handoffs: a systematic review and task force recommendations. J Hosp Med. 2009;4:433–40
13. Nagpal K, Vats A, Lamb B, Ashrafian H, Sevdalis N, Vincent C, Moorthy K. Information transfer and communication in surgery: a systematic review. Ann Surg. 2010;252:225–39
14. Segall N, Bonifacio AS, Schroeder RA, Barbeito A, Rogers D, Thornlow DK, Emery J, Kellum S, Wright MC, Mark JBDurham VA Patient Safety Center of Inquiry. Durham VA Patient Safety Center of Inquiry. . Can we make postoperative patient handovers safer? A systematic review of the literature. Anesth Analg. 2012;115:102–15
15. Møller TP, Madsen MD, Fuhrmann L, Østergaard D. Postoperative handover: characteristics and considerations on improvement: a systematic review. Eur J Anaesthesiol. 2013;30:229–42
16. Bhabra G, Mackeith S, Monteiro P, Pothier DD. An experimental comparison of handover methods. Ann R Coll Surg Engl. 2007;89:298–300
17. Rayo MF, Mount-Campbell AF, O’Brien JM, White SE, Butz A, Evans K, Patterson ES. Interactive questioning in critical care during handovers: a transcript analysis of communication behaviours by physicians, nurses and nurse practitioners. BMJ Qual Saf. 2014;23:483–9
18. Smith AF, Pope C, Goodwin D, Mort M. Interprofessional handover and patient safety in anaesthesia: observational study of handovers in the recovery room. Br J Anaesth. 2008;101:332–7
19. Schwilk B, Gravenstein N, Blessing S, Friesdorf W. Postoperative information transfer: a study comparing two university hospitals. Int J Clin Monit Comput. 1994;11:145–9
20. Anwari JS. Quality of handover to the postanaesthesia care unit nurse. Anaesthesia. 2002;57:488–93
21. Bonifacio AS, Segall N, Barbeito A, Taekman J, Schroeder R, Mark JB. Handovers from the OR to the ICU. Int Anesthesiol Clin. 2013;51:43–61
22. van Rensen EL, Groen ES, Numan SC, Smit MJ, Cremer OL, Tates K, Kalkman CJ. Multitasking during patient handover in the recovery room. Anesth Analg. 2012;115:1183–7
23. Ford JK, Weissbein DA. Transfer of training: an update review and analysis. Perform Improve Quart. 1997;10:22–41
24. Henriksen K, Dayton E. Issues in the design of training for quality and safety. Qual Saf Health Care. 2006;15 Suppl 1:i17–24
25. Donovan MS, Bransford JD, Pellegrino JWCommittee on Developments in the Science of Learning. Committee on Developments in the Science of Learning. How People Learn: Bridging Research and Practice (National Research Council Report). 2000 Washington DC National Academies Press
26. Shulman LS The Wisdom of Practice: Essays on Teaching, Learning, and Learning to Teach. 2004 Hoboken, NJ Jossey-Bass/John Wiley & Sons
27. Dreyfus HL, Dreyfus S A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition. 1980 Berkeley, CA University of California Press
28. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79:S70–81
29. Schon DA Educating the Reflective Practitioner: Towards a New Design for Teaching and Learning in the Professions. 1987 San Francisco, CA Jossey-Bass
30. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28
31. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44:50–63
32. Scalese RJ, Obeso VT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med. 2008;23 Suppl 1:46–9
33. Haig KM, Sutton S, Whittington J. SBAR: a shared mental model for improving communication between clinicians. Jt Comm J Qual Patient Saf. 2006;32:167–75
34. Oser R, Salas E, Merket D, Bowers CASalas E, Bowers CA, Edens E. Applying resource management training in naval aviation: a methodology and lessons learned. In: Improving Teamwork in Organizations: Applications of Resource Management Training. 2001 Mahwah, NJ Lawrence Erlbaum:283–301
35. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127:757–63
36. Weinger M, Wiklund M, Gardner-Bonneau D Handbook of Human Factors in Medical Device Design. 2011 Boca Raton, FL CRC Press/Taylor Francis
37. Boulet JR, Murray DJ. Simulation-based assessment in anesthesiology: requirements for practical implementation. Anesthesiology. 2010;112:1041–52
38. Boulet JR, Murray D. Review article: assessment in anesthesiology education. Can J Anesth. 2012;59:182–92
39. Slagle J, Weinger MB, Dinh MT, Brumer VV, Williams K. Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology. Anesthesiology. 2002;96:1129–39
40. Hemmer PA, Hawkins R, Jackson JL, Pangaro LN. Assessing how well three evaluation methods detect deficiencies in medical students’ professionalism in two settings of an internal medicine clerkship. Acad Med. 2000;75:167–73
41. Hall JC, Hall JL. ASA status and age predict adverse events after abdominal surgery. J Qual Clin Pract. 1996;16:103–8
42. Liang K, Zeger S. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13–22
43. Pepe M, Anderson G. A cautionary note on inference for marginal regression models with longitudinal data and general correlated response data. Commun Stat Simulat Comput. 1994;23:939–51
44. Schildcrout JS, Heagerty PJ. Regression analysis of longitudinal binary data with time-dependent environmental covariates: bias and efficiency. Biostatistics. 2005;6:633–52
45. Hosmer DW, Lemeshow S Applied Logistic Regression. 2000 New York, NY Wiley
46. R Development Core Team. A Language and Environment for Statistical Computing. 2010 Vienna, Austria R Foundation for Statistical Computing Available at: http://www.R-project.org
47. Teno JM, Harrell FE Jr, Knaus W, Phillips RS, Wu AW, Connors A Jr, Wenger NS, Wagner D, Galanos A, Desbiens NA, Lynn J. Prediction of survival for older hospitalized patients: the HELP survival model. Hospitalized Elderly Longitudinal Project. J Am Geriatr Soc. 2000;48:S16–24
48. Kotter JP What Leaders Really Do. 1999 Boston, MA Harvard Business School Press
49. Kotter JP. The key to changing organizational culture. Forbes. 2012
50. Ravasi D, Schultz M. Responding to organizational identity threats: exploring the role of organizational culture. Acad Manage J. 2006;49:433–58
51. Weinger MB. The pharmacology of simulation: a conceptual framework to inform progress in simulation research. Simul Healthc. 2010;5:8–15
52. Forrest JB, Cahalan MK, Rehder K, Goldsmith CH, Levy WJ, Strunin L, Bota W, Boucek CD, Cucchiara RF, Dhamee S. Multicenter study of general anesthesia. II. Results. Anesthesiology. 1990;72:262–8
53. Hickson GB, Federspiel CF, Pichert JW, Miller CS, Gauld-Jaeger J, Bost P. Patient complaints and malpractice risk. JAMA. 2002;287:2951–7
54. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf. 2014;23:299–18
55. Weinger MB, Gonzales DC, Slagle J, Syeed M. Video capture of clinical care to enhance patient safety. Qual Saf Health Care. 2004;13:136–44
56. Slagle JM, Anders S, Porterfield E, Arnold A, Calderwood C, Weinger MB. Significant physiological disturbances associated with non-routine event containing and routine anesthesia cases. J Patient Safe. 2014 [ePub ahead of print]
57. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, Sexton B, Hyzy R, Welsh R, Roth G, Bander J, Kepros J, Goeschel C. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355:2725–32
58. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AASafe Surgery Saves Lives Study Group. Safe Surgery Saves Lives Study Group. . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–9
59. Weiser TG, Haynes AB, Dziekan G, Berry WR, Lipsitz SR, Gawande AASafe Surgery Saves Lives Investigators and Study Group. Safe Surgery Saves Lives Investigators and Study Group. . Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010;251:976–80
Supplemental Digital Content
© 2015 International Anesthesia Research Society