An Anesthesia Preinduction Checklist to Improve Information Exchange, Knowledge of Critical Information, Perception of Safety, and Possibly Perception of Teamwork in Anesthesia Teams

Abstract

BACKGROUND: 

An anesthesia preinduction checklist (APIC) to be performed before anesthesia induction was introduced and evaluated with respect to 5 team-level outcomes, each being a surrogate end point for patient safety: information exchange (the percentage of checklist items exchanged by a team, out of 12 total items); knowledge of critical information (the percentage of critical information items out of 5 total items such as allergies, reported as known by the members of a team); team members’ perceptions of safety (the median scores given by the members of a team on a continuous rating scale); their perception of teamwork (the median scores given by the members of a team on a continuous rating scale); and clinical performance (the percentage of completed items out of 14 required tasks, e.g., suction device checked).

METHODS: 

A prospective interventional study comparing anesthesia teams using the APIC with a control group not using the APIC was performed using a multimethod design. Trained observers rated information exchange and clinical performance during on-site observations of anesthesia inductions. After the observations, each team member indicated the critical information items they knew and their perceptions of safety and teamwork.

RESULTS: 

One hundred five teams using the APIC were compared with 100 teams not doing so. The medians of the team-level outcome scores in the APIC group versus the control group were as follows: information exchange: 100% vs 33% (P < 0.001), knowledge of critical information: 100% vs 90% (P < 0.001), perception of safety: 91% vs 84% (P < 0.001), perception of teamwork: 90% vs 86% (P = 0.028), and clinical performance: 93% vs 93% (P = 0.60).

CONCLUSIONS: 

This study provides empirical evidence that the use of a preinduction checklist significantly improves information exchange, knowledge of critical information, and perception of safety in anesthesia teams—all parameters contributing to patient safety. There was a trend indicating improved perception of teamwork.

In Brief

Published ahead of print March 24, 2015

The World Health Organization (WHO) recommended the use of a surgical safety checklist in their 2009 Guidelines for Safe Surgery,¹ and checklists have since become a part of standard surgical care.² They have been associated with reduced complications and mortality rates,^3–7 better adherence to safety standards,⁸ improved communication and teamwork, and economic benefits.^2,9,10 The WHO Guidelines for Safe Surgery also encourage a formal inspection of the anesthetic equipment, breathing circuit, medications, and a patient’s anesthetic risk before each case.

The reason for introducing the separate anesthesia preinduction checklist (APIC), in addition to the already introduced WHO surgical safety checklist, was that the WHO surgical safety checklist contains only a few supercritical anesthesia items (e.g., checks of saturation sensor, but not electrocardiogram or blood pressure monitoring).

In this study, we sought to evaluate whether the APIC complementing the WHO surgical safety checklist is suited to improve 5 team-level outcomes, each shown to be critical surrogate end points for patient safety: information exchange, knowledge of critical information, team members’ perceptions of both safety and teamwork, and clinical performance.

Figure 1 outlines the series of subitems assessed for the outcome scores: information exchange, knowledge of critical information, and clinical performance.

Figure 1: Odds ratios (ORs) with 95% confidence intervals on logarithmic scales of the subitems assessed for the outcomes. A, Information exchange. B, Knowledge of critical information. C, Clinical performance. If a contingency table contained a value of 0, the ORs were calculated by adding 0.5 to each value. The cross-table analyses (Supplemental Table 4, Supplemental Digital Content, https://links.lww.com/AA/B89) showed that information exchange about monitoring and suction device function was associated with improved performance of the respective clinical performance subitems blood pressure interval set to ≤2 minutes and suction device checked. Consequently, the OR for suction device checked was significantly improved in the anesthesia preinduction checklist (APIC) group. ECG = electrocardiogram; BP = blood pressure; FiO₂ = fraction of inspired oxygen; SpO₂ = peripheral capillary oxygen saturation.

METHODS

This study was approved by the ethics committee of the Canton of Zurich (KEK StV-No. 07/12), Zurich, Switzerland. The requirement for written patient consent was waived. During the study, no information that would identify a team member or a patient was collected.

The APIC

Supplemental Figure 1 (Supplemental Digital Content, https://links.lww.com/AA/B89) shows the APIC as evaluated in this study. Table 1 provides a description of each checklist item, according to a single-page checklist manual that we distributed with the APIC.

Table 1: Description of Items of the APIC

The APIC contains the before-induction-of-anesthesia items of the WHO surgical safety checklist (items 1–4, 11) and additional items (5–10, 12, 13) selected in a Delphi process by 7 consultant anesthesiologists of the study hospital. After several rounds of discussion and evaluation, these experts, each with >10 years of clinical anesthesia experience, identified the minimal items they considered critical to be exchanged and checked before every induction and that were not already included on the WHO surgical safety checklist.

The APIC is available as a laminated card (21 × 15 cm), normally attached to the ventilator, at all anesthesia preparation sites. The check is performed when the entire anesthesia team is present with the patient in the preparation room, preferably during preoxygenation and definitely before any invasive procedures or drug administration. Any team member (i.e., physician or nurse) can start by reading the first checklist item, while any other team member can confirm that they checked a respective item or provide an appropriate answer (e.g., Anesthetic and surgical procedure? and We will perform oral endotracheal intubation for laparoscopic appendectomy.) The team members complete all items in sequence before proceeding with the induction of anesthesia.

The APIC is not intended to be a replacement for a thorough preoperative evaluation, which is performed hours to days before a surgical procedure, and which team members, according to institutional policy, are required to review before each case. The APIC serves as a check and briefing of only safety-critical items at the last possible moment. Its aims are avoidance of omissions and promotion of a common understanding (i.e., shared mental model) of the situation among the team members.

Outcome Parameters

The outcomes evaluated in this study and the rationales behind their selection were as follows:

1. Information exchange (the percentage of checklist items exchanged by a team, out of 12 total items). A recent study found improvements in the exchange of critical information to be associated with a reduced number of nonroutine events (near misses).¹¹ Furthermore, several studies have identified the failure to exchange critical information as a common cause for patient harm.^12–14
2. Knowledge of critical information (the median percentage of critical information items out of 5 total items such as allergies, reported as known by the team members after the anesthesia induction had been completed). Several studies have described the perception of critical information, such as the availability and state of equipment, as the first step in the development of situation awareness.^15–18 One study even found the failure to observe available information as the most common cause of inadequate decision making.¹⁹
3. Team members’ perception of safety (the median scores given by members of a team on a continuous rating scale).
4. Team members’ perception of teamwork (the median scores given by members of a team on a continuous rating scale). Improved perceptions of safety climate and teamwork have been associated with improved postoperative outcomes.²⁰
5. Clinical performance (the percentage of completed items out of 14 required tasks). Evaluating items for this outcome, such as checking the suction device, is a standard of care in the study hospital and adherence to this standard is a surrogate marker for patient safety.²¹

Procedure

This prospective interventional study comparing anesthesia teams performing inductions using (APIC group) versus not using (control group) the APIC was performed using a multimethod approach. It consisted of on-site observations followed by surveys of the participating team members. Only inductions of general anesthesia for elective adult cases were included.

With the use of iSurvey software (Harvest Your Data, Wellington, New Zealand), a tablet computer–based (iPad^®; Apple Inc., Cupertino, CA) data collection tool was created. Using this tool, 5 trained observers (all consultant anesthesiologists) rated information exchange and clinical performance during anesthesia inductions and performed the team member surveys after the observations during a stable phase of anesthesia. Team members were consecutively handed the tablet computer and individually, privately, and anonymously indicated the critical information items they knew by selecting them on a multiple-choice form and rated their perception of safety and of teamwork on 2 continuous scales anchored from 0% (very poor) to 100% (very good). Because of the limited time available to conduct the surveys in the operating areas, 2 simple questions with high face validity were chosen to measure perceptions of safety and teamwork. Single-item measurements for these outcomes have been used in previous studies.²²

The items used to measure clinical performance were based on a protocol developed by Burtscher et al.^23,24 and validated by using a Delphi approach to measure the performance of teams during induction of general anesthesia. This measure has been tested for interrater reliability and used to assess team performance in a number of studies.^23,25

Supplemental Table 1 (Supplemental Digital Content, https://links.lww.com/AA/B89) outlines the complete data collection protocol used in this study.

Information exchange was defined as explicit communication about an item between ≥2 team members (e.g., Suction device checked? and Yes, I checked the suction device.). The start of an observation was defined as the arrival of the last member of an anesthesia team at the site of anesthesia induction. The observation ended with the administration of the last drug to induce anesthesia. Data collection took place in 7 operating areas with a total of 34 operating rooms (ORs) in a university hospital setting. In 4 areas with a total of 16 ORs, the APIC was introduced 4 months before the beginning of the study (APIC group). In 3 areas with a total of 18 ORs, the APIC remained absent during the study period (control group). All teams in both study groups used the WHO surgical safety checklist, which was introduced >2 years before the beginning of this study. This checklist did not include the anesthesia-specific items added by the APIC.

Participants

Anesthesia staff consisted of 45 consultants, 90 residents, and 100 certified and student nurses. Only one induction on any one operating list was captured. The observers were instructed to intervene only if predefined safety-critical situations, such as signs of myocardial ischemia, were unnoticed by the anesthesia team.

Interrater Reliability of the Observation Tool

Before starting the data collection, interrater reliability for the instruments of the observation tool used by the observers for on-site observations was tested (i.e., information exchange and clinical performance). To validate the tool and to ensure that observations were standardized and comparable among raters, a simulation involving a full-scale patient simulator and realistic OR environment was used. Three different preinduction scenarios were created and recorded as multiangle videos. All scenarios involved a consultant anesthesiologist, a resident, and an anesthesia nurse performing inductions that involved minor clinical errors (e.g., not setting the automatic blood pressure interval according to standard procedure) and/or omissions of information exchange (e.g., patient allergies were not discussed). Each observer was trained by a psychologist specialized in human factors and received a comprehensive introduction to observational methods, APIC use, and use of the observation tool. Subsequently, all observers watched the recorded inductions and independently rated all 3 scenarios. To assess interrater reliability, Fleiss kappa was calculated with 95% confidence interval (CI) for each scenario. Interrater reliability was high with Fleiss kappa of 1.00 for scenario 1 (absolute agreement between observers), 0.88 (95% CI, 0.78–1.00) for scenario 2, and 0.85 (95% CI, 0.72–0.97) for scenario 3.

Statistical Analysis

Statistical analyses were performed using Stata 11.2 (StataCorp, College Station, TX) and GraphPad Prism 6.0 software (GraphPad Software, La Jolla, CA). Variation of the outcomes was not known before the study. Thus, the sample size was determined by using 2-group Fisher exact test of equal proportions (for binary outcomes) expecting values of 99% for the median outcome scores in the APIC group and values of 89% in the control group. This analysis was repeated for each of the 5 team-level outcomes (information exchange, knowledge of critical information, perception of safety, perception of teamwork, and clinical performance). It showed that with 100 teams in each study group, differences of 10% could be detected for each of the 5 outcomes with ≥83% power and a 0.05 2-tailed significance level. This power analysis was conservative, and the actual power of the study was higher because the Mann-Whitney U test uses continuous values to examine the differences between the median outcome scores and has a higher power in comparison with tests that examine binary variables.

To assess the relations between the outcomes, univariate and multivariate regression analyses were performed. To test the statistical significance of the reported odds ratios for the subitems of the outcome scores, Fisher exact test was used. To address clustering of team members within teams, logistic regression analysis was performed with team member knowledge of a subitem as a dependent variable, the use of the APIC as an independent variable, and robust standard error (with team identification number as a cluster). To test the robustness of the results, 4 prespecified subgroup analyses were performed: (1) consultant-led teams using versus not using the APIC, (2) resident-led teams using versus not using the APIC, (3) consultant-led versus resident-led teams using the APIC, and (4) consultant-led versus resident-led teams not using the APIC. Additionally, cross-tables were created to assess the percentages of teams in both study groups that had 0, 1, 2, 3, 4, or all 5 outcome scores, and specific combinations of outcome scores, above a cutoff value of 90%. The cutoff was defined at 90% based on the assumption of the power analysis that the values of the median outcome scores would be 89% before introduction of the APIC.

To assess the relation between information exchange about an item and performance of the respective clinical performance subitems, cross-table analyses were performed.

To test whether there was a difference between information exchange using the APIC, compared with information exchange performed without the APIC, cross-table analyses were performed to compare the percentages of teams in both study groups that exchanged information about an item with or without the APIC, and in which all team members had knowledge about the related subitem or, respectively, performed the related subitem. For the cross-table analyses, P values were calculated by applying the 2-group Fisher exact test.

Because, in this study, there were multiple correlated P values, P values between 0.05 and 0.01 were treated as trends and P values <0.01 were considered statistically significant. All P values and 95% CIs were computed 2 tailed.

RESULTS

Data Collection

The data collection period spanned 131 days. The average time to complete 1 collection (i.e., on-site observation and team member surveys) was 25 minutes; the average time interval between the last question of the on-site observation and the beginning of the team member survey was 3 minutes.

Study Groups

Table 2: Team Characteristics and Team Compositions

The APIC group included 105 teams (285 team members), all of which had used the APIC. The control group included 100 teams (272 team members), none of which had used the APIC. Table 2 shows the characteristics and differences of the teams in both study groups.

Outcomes

Adherence to APIC use in the OR areas in which it was introduced was 88%, with a median completion rate of 100%. The values of all outcome scores with the exception of clinical performance and perception of teamwork were significantly higher in the APIC group. There was a trend indicating that the APIC also positively affected perceptions of teamwork. Figure 2 shows the box plots for these results.

Figure 2: Box plots with whiskers of the median team scores in the checklist group and the control group. The bottoms and tops of the boxes are the 25th and 75th percentiles, and the bands in the boxes are the medians. The ends of the whiskers represent the 5th and 95th percentile. A, Information exchange 100% (range, 83%–100%) vs 33% (range, 0%–100%). B, Knowledge of critical information 100% (range, 60%–100%) vs 90% (range, 60%–100%). C, Perception of safety 91% (range, 33%–100%) vs 84% (range, 30%–100%). D, Perception of teamwork 90% (range, 50%–100%) vs 86% (range, 39%–100%). E, Clinical performance 93% (range, 79%–100%) vs 93% (range, 71%–100%). F, Multivariate assessment of the outcomes: odds ratios (per percent of the corresponding score) for the 5 outcomes with 95% confidence intervals on a logarithmic scale. Black signifies multivariate logistic regression adjusted for the other outcomes; blue shows univariate logistic regression.

As shown by the regression analyses, information exchange was the most important, independently significant outcome. High information exchange was a strong predictor of APIC use, and the multivariate data suggest a causal relation of information exchange with the other outcomes. Ninety-five percent of the teams in the APIC group achieved 100% information exchange versus only 2% of the teams in the control group. Surprisingly, despite a significant improvement in information exchange, overall clinical performance did not improve with APIC use. Consequently, in the multiple regression, a nonsignificant decline with APIC was observed after adjustment for other outcomes (Fig. 2F). Supplemental Figure 2 (Supplemental Digital Content, https://links.lww.com/AA/B89) shows a scatter plot detailing the relation between information exchange and clinical performance.

The odds ratios for the subitems were significantly >1 when the APIC was used for all of the individual information exchange subitems (all P < 0.001), for knowledge of the subitems anesthesia consent (P < 0.001) and perioperative surgical medication (P < 0.001), and for the clinical performance subitem suction device checked (P = 0.008). There was a trend indicating improved knowledge of the subitems allergies (P = 0.043) and airway (P = 0.013). The odds ratios for the subitems are outlined in Figure 1.

In the subgroup analyses, use of the APIC improved the values of all outcome scores in consultant-led and resident-led teams with statistical significance for information exchange, knowledge of critical information, and perception of safety. Perception of teamwork remained significantly improved in consultant-led teams using the APIC. Table 3 shows the results of the subgroup analyses.

Table 3: Subgroup Analyses

When the APIC was used, the anesthesia teams were significantly more likely to have 4 (P < 0.001) and 5 (P < 0.001) outcome scores above the cutoff value of 90%. Also, there was a trend showing that anesthesia teams were more likely to have 3 (P = 0.029) outcome scores above the cutoff value. Table 4 outlines the results of the cross-table analyses. These results were robust in the subgroups of consultant-led and resident-led teams (outlined in Supplemental Tables 2 and 3, Supplemental Digital Content, https://links.lww.com/AA/B89).

Table 4: Cross-Table Analyses

Information exchange about an item was associated with a significantly improved rate of performance for the subitem, suction device checked (P < 0.001). This result is outlined in Supplemental Table 4 (Supplemental Digital Content, https://links.lww.com/AA/B89).

When information was exchanged using the APIC, in comparison with information exchange without APIC use, the likelihood that all members of a team had knowledge of the respective information was significantly higher for the subitem perioperative surgical medication (odds ratio, 7; P = 0.001) and showed a higher trend for the subitem anesthesia consent (odds ratio, 5; P = 0.014). See Supplemental Table 5 for these results (Supplemental Digital Content, https://links.lww.com/AA/B89). For the clinical performance subitems, there was no significant difference between information exchange with versus without APIC (see Supplemental Table 6, Supplemental Digital Content, https://links.lww.com/AA/B89).

In the control group, 2 observers reported the occurrence of critical events. During an induction, the consultant on a team mistook 1 patient for another and realized this error only after induction. During another case, the team accidentally induced anesthesia at the beginning of preoxygenation because the vaporizer was still open from the previous case.

DISCUSSION

We found that the implementation of the APIC was associated with significant improvements in information exchange, knowledge of critical information, and perceptions of safety in anesthesia teams, as well as a trend indicating that the APIC improved perceptions of teamwork. Because the high degree of information exchange almost entirely correlated with APIC use, we cannot determine whether the suggested causal relation with the other outcomes was caused by this or other factors related to APIC use. For example, both the improved information exchange and the structured review and briefing of critical information conducted by the entire team during APIC performance may have caused the associated improvements in knowledge of critical information. This team process of checklist performance may also be an explanation for why information exchange performed with the APIC—when compared with information exchange without APIC use—was associated with improved likelihoods for all team members to have knowledge of several critical information subitems (e.g., anesthesia consent).

Because medical mishaps and errors are mainly due to communication breakdowns in teams,^12–14 and information exchange and knowledge of critical information have been identified as crucial predictors of team performance,^26,27 we propose that the improvements in these 2 outcomes are of great importance.

The overall clinical performance scores did not differ between groups, probably because of a ceiling effect, given that elective anesthesia inductions are a routine task with already low probability for clinical errors and omissions. However, the observers reported critical events only in the control group. The performance of the APIC may have uncovered the factors leading to these events. For example, it seems plausible that the performance of APIC items 1 and 2 (patient identification), with the entire anesthesia team present immediately before anesthesia induction, would have alerted the consultant anesthesiologist about the actual identity of the patient. According to the standard procedure in the study hospital, a team member checked patient identity with the WHO surgical safety checklist at the patient’s arrival in the OR area. However, this information was not exchanged with the consultant who joined the team only shortly before the induction and assumed they were treating a different patient. Performing APIC item 12 (respirator function), which includes checking the integrity of the breathing circuit, may have alerted the team that the vaporizer was still open from the previous case. Because information sharing and processing within teams usually tend to be poor and to require rigorous coordination,²⁸ the performance of the APIC immediately before induction may be helpful for information coordination and provide an opportunity for questioning the status quo and for speaking up with questions, ideas, and corrections. Speaking up is associated with improved clinical performance in anesthesia and surgical teams.^25,29,30

Because team members do not have to remember items when using the APIC, it serves as a tool for overcoming the limitations of prospective memory; that is, remembering to perform actions at the appropriate time.³¹ Prospective memory is susceptible to failure (e.g., forgetting to check the suction device because of imposed time pressure) when disturbances occur.

Previous studies have reported benefits of checklist use during anesthesia for identifying missing items before induction,³² improving the quality of postanesthesia handovers, and managing simulated cases of cesarean delivery,³³ local anesthetic systemic toxicity, and OR crisis scenarios.^34,35

One of the major challenges, which will ultimately determine whether a checklist will lead to an improvement in patient safety, is the checklist's acceptance by clinicians. Although use of the APIC was highly recommended by the management of the department during the study, it was not used by 12% of the teams in OR areas where the APIC had been introduced. We suggest several reasons for this omission, including not knowing how to use the APIC, lack of acceptance of checklists in general, or reluctance to adapt to changing organizational routines. To enable the maximal potential of checklists, and to promote their use, we believe that the development and implementation of checklist training initiatives for anesthesia teams could be helpful in the future. Also, we would like to emphasize that the APIC evaluated in this study is one possible example of an anesthesia preinduction checklist, the content and implementation of which should undergo continuous reevaluation and refinement.

This study has limitations. It was nonrandomized, and the selection of cases depended on the availability of the observers. Although they were able to choose only from inductions taking place at the time they were available, the occurrence of selection bias cannot be excluded.

There was cross-contamination between the study groups because some team members rotated between OR areas with and without the APIC. However, because team members who previously used the APIC may still implicitly work according to the APIC, even while not explicitly using it, cross-contamination would improve the performance of the control group and bias the results toward an underestimation of the effects and not an overestimation.³⁶

The study data may be biased because the data collection took place in 7 separate OR areas. However, inductions are performed according to institutional directives valid in all operation areas; hence, the possibility that certain differences in the areas influenced the results cannot be fully excluded.

Finally, there was an imbalance of consultant involvement (94% in the APIC group and 75% in the control group). The subgroup analyses, however, show that this did not cause the results, because the improvements in information exchange and knowledge of critical information remained consistent and significant in consultant-led teams and resident-led teams. Perceptions of safety and teamwork remained significantly improved in consultant-led teams, and there was a trend indicating improved perception of safety in resident-led teams using the APIC.

CONCLUSIONS

This study provides empirical evidence that the use of a preinduction checklist significantly improves information exchange, knowledge of critical information, and perceptions of safety as well as a trend indicating that the APIC improves perceptions of teamwork in anesthesia teams—all factors contributing to patient safety.

DISCLOSURES

Name: David W. Tscholl, MD.

Contribution: This author helped design the study, analyze the data, and write the manuscript.

Attestation: David W. Tscholl has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Name: Mona Weiss.

Contribution: This author helped design the study, analyze the data, and write the manuscript.

Attestation: Mona Weiss has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Name: Michaela Kolbe, PhD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Michaela Kolbe reviewed the analysis of the data and approved the final manuscript.

Name: Sven Staender, MD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Sven Staender reviewed the analysis of the data and approved the final manuscript.

Name: Burkhardt Seifert, PhD.

Contribution: This author helped design the study, analyze the data, and write the manuscript.

Attestation: Burkhardt Seifert has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Name: Daniel Landert.

Contribution: This author helped design the study and write the manuscript.

Attestation: Daniel Landert approved the final manuscript.

Name: Bastian Grande, MD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Bastian Grande approved the final manuscript.

Name: Donat R. Spahn, MD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Donat R. Spahn reviewed the analysis of the data and approved the final manuscript.

Name: Christoph B. Noethiger, MD.

Contribution: This author helped design the study, analyze the data, and write the manuscript.

Attestation: Christoph B. Noethiger has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

This manuscript was handled by: Sorin J. Brull, MD, FCARCSI.

ACKNOWLEDGMENTS

The authors are grateful to Thomas Horisberger, MD, Caveh Madjdpour, MD, Adrian Marty, MD, Martin Schläpfer, MD, PhD, and Michael Tucci, MD, for conducting the on-site observations. The authors are also grateful to Carl Schick, MD, for proofreading the manuscript and to all physicians and nurses at the University Hospital Zurich for their participation.