Journal Logo

Empirical Investigations

Training Induces Cognitive Bias

The Case of a Simulation-Based Emergency Airway Curriculum

Park, Christine S. MD; Stojiljkovic, Ljuba MD, PhD; Milicic, Biljana MD, PhD; Lin, Brian F. BS; Dror, Itiel E. PhD

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: April 2014 - Volume 9 - Issue 2 - p 85-93
doi: 10.1097/SIH.0b013e3182a90304
  • Free

Abstract

Performance may be compromised when cognitive bias influences decision making. Cognitive bias is an inclination in judgment based on incomplete perspective and preformed patterns of thought. This inclination is held at the expense of other valid, potentially better, alternatives. Especially when decisions have to be made and executed quickly, individuals may be even more prone to bias, which leads to cognitive error.1 Decision-making biases are systematic errors rather than random ones2 and represent failures to make optimal decisions.3 Because cognitive bias may result in exclusion of appropriate actions or deviation from an algorithm, patient care and safety may be threatened.

All training may be vulnerable to induction of bias. As training methods grow in effectiveness, a potential trade-off exists whereby the development of skills entails an increase in vulnerability to problems such as bias.1 Technology-enhanced learning has increased impact on the cognitive representations that are acquired, but training-based bias may also be more likely to emerge.4 For example, in cognitive science, it has been demonstrated that the order of presentation biases what and how information is recalled.5 The relationship between training and bias has not been well established in the medical domain. Effective learning outcomes in simulation-based training have been well described,6–11 and it is important to also explore the potential development of associated biases.

To test this concept, we used a simulation-based curriculum for training novice residents in the management of the unanticipated emergency airway. An unanticipated, cannot-ventilate, cannot-intubate airway is an emergent and potentially fatal event requiring swift intervention,12 and training for this event is essential. Airway complications are a leading cause of morbidity and mortality in the American Society of Anesthesiologists Closed Claims Database, with 67% of difficult airways occurring during the induction of anesthesia.13 The American Society of Anesthesiologists has developed an algorithm for difficult airway management, including the emergent cannot-ventilate, cannot-intubate event.14 The sequence of action begins with less invasive techniques (mask ventilation, direct laryngoscopy, and other devices). This is followed by attempted rescue with a supraglottic airway and culminates in a final invasive step of emergency cricothyroidotomy. However, especially when performed in an emergency, surgical cricothyroidotomy is associated with severe, life-threatening complications, and it should be used only after noninvasive alternatives fail.15 Simulation is well suited for training skills in such low-frequency high-consequence events.

We investigated whether curriculum design, specifically training sequence, could influence bias in the setting of simulation-based medical education. We hypothesized that initial training in only 1 technique would result in a bias for that technique in a simulated cannot-ventilate, cannot-intubate scenario and that completion of training in both techniques would correct this bias regardless of the training sequence.

MATERIALS AND METHODS

The study, using a randomized, prospective crossover design (Fig. 1), was approved by the institutional review board of Northwestern University (Chicago, IL). All incoming postgraduate year 2 (PGY-2) residents to the Northwestern University anesthesiology residency program were considered for inclusion. The exclusion criterion was previous postgraduate training in anesthesiology. Written informed consent was obtained from the participants before enrollment. No resident was excluded, and all agreed to participate. Training was conducted in the Simulation Technology and Immersive Learning laboratory in the Center for Education in Medicine. Evaluations were conducted in the Northwestern Center for Clinical Simulation in the Department of Anesthesiology, using a full-body mannequin simulator (HPS; Medical Education Technologies, Inc, Sarasota, FL).

F1-3
FIGURE 1:
Study design of parallel groups with crossover to training in alternate method after 3 weeks.

The primary outcome measure was the use of supraglottic airway and surgical cricothyroidotomy techniques (initiated or not initiated) in a simulated, emergency cannot-ventilate, cannot-intubate scenario for residents during 3 evaluation sessions: baseline, mid (3 weeks), and final evaluation (6 weeks). In the primary analysis, we documented initiation of supraglottic airway and cricothyroidotomy for all residents enrolled in the study. In a subanalysis, we considered the subset of residents who did not attempt either cricothyroidotomy or supraglottic airway at the baseline evaluation.

The secondary outcome measure was response time: we measured airway takeover time, time to initiation of supraglottic airway, time to initiation of surgical cricothyroidotomy, and interval time. Times were measured in seconds from a running time counter on the video. Airway takeover time was defined as length of time from the entry of the resident into the scenario to take over the airway from the medical student. Time to supraglottic airway was defined as the interval from airway takeover to initiation of supraglottic airway and time to cricothyroidotomy as interval from airway takeover to initiation of cricothyroidotomy. Interval time was defined as time elapsed between initiation of supraglottic airway and initiation of cricothyroidotomy.

Airway management maneuvers used during each scenario were documented by post hoc video review by a single rater who was blinded to subject identity, group assignments, and test date. The expected sequence of airway maneuvers was mask ventilation, direct laryngoscopy, supraglottic airway placement, and cricothyroidotomy. All 4 airway maneuvers were rated as initiated or not initiated. A maneuver was documented and timed when the resident physically initiated the technique. For supraglottic airway, this was defined as introduction of the device into the mouth. For surgical cricothyroidotomy, this was defined as initiation of a skin incision, which also ended the scenario. All noninvasive airway maneuvers were designed to be ineffective.

Procedure

Twenty-three clinical anesthesiology residents during their first 6 weeks of anesthesiology training following an internship (PGY-1) year were eligible and elected to participate in the study. During the initial 6-week period, residents received operating room training with supervision at a 1:1 faculty-to-resident ratio, daily 1-hour didactic lectures covering basic concepts of anesthesia, and simulation-based training in a variety of critical events, including intraoperative hypotension, intraoperative hypoxemia, intraoperative cardiac arrest, postanesthesia care unit events, handoffs, teamwork, and emergency airway management (supraglottic airway and surgical cricothyroidotomy), during eight 3-hour sessions in the simulation laboratory during routine work hours.

Before the baseline test, residents were randomly divided into 2 groups using a computer-generated random numbers table. All residents were familiarized with the simulator environment and mannequin during a general orientation. At the baseline evaluation session before the educational intervention, all residents underwent individual testing in the unanticipated emergency airway scenario as part of a set of other intraoperative critical event scenarios as described in a previous article, including hypoxemia caused by bronchospasm, hypoxemia caused by endobronchial intubation and circuit leak, hypotension caused by hypovolemia, and hypotension caused by myocardial ischemia.16 No debriefing took place following the test scenarios. Evaluation sessions were videotaped for post hoc evaluation of performance only by the rater.

Approximately 1.5 weeks after the baseline evaluation, residents participated in a 3-hour emergency airway module. Each group received a standardized lecture covering the American Society of Anesthesiologists Difficult Airway Algorithm in detail. The PowerPoint based lecture was a step-by-step description of the algorithm. It included photographs and descriptions of equipment pertinent to difficult airway management, including pharyngeal airways, supraglottic airways, equipment for direct and video laryngoscopy, intubating stylets, fiber-optic intubation, and emergency surgical cricothyroidotomy. All residents received a copy of the algorithm for reference. There were 3 total instructors blinded to group assignment, a single instructor for the supraglottic airway training sessions, and 2 instructors for the cricothyroidotomy training sessions. Instructors agreed upon the lecture content and agreed to adhere to the PowerPoint material. Because the goal was to investigate the effect of technical skills training and training sequence on decision making, we standardized the instruction for both supraglottic airway and cricothyroidotomy to restrict variables in training and sources of bias across experimental conditions. Instruction was limited to technical direction only during the training sessions. Other than reading the American Society of Anesthesiologists Difficult Airway Algorithm, instructors did not provide case studies, anecdotes, or further comment on decision making. Instructors did not discuss risks and benefits, including acute and chronic morbidity and mortality, because of its potential to introduce bias beyond that of the technical training. After the lecture, 1 group (SGA group) received initial standardized practical training in supraglottic devices and practiced placement of a variety of supraglottic airway devices on partial task trainers. The other group (CRIC group) received initial standardized practical training in emergency surgical cricothyroidotomy. They practiced 3 different emergency surgical cricothyroidotomy techniques, 3-step, 4-step, and Seldinger techniques,17–19 on partial-task trainers using sheep tracheas covered with artificial skin. Equipment consisted of a Melker cricothyroidotomy kit20 (Cook Medical, Inc, Bloomington, IN) and an Eschmann stylet (Bell Medical, Inc, St. Louis, MO).

A midtest was performed at 3 weeks of training. The evaluation was conducted in the same manner as the baseline evaluation. The groups were then crossed over, receiving the identical lecture given at the previous training session accompanied by practical training in the other technique (ie, the SGA group now practiced cricothyroidotomy, and the CRIC group now practiced supraglottic airway). This session occurred approximately 1.5 weeks after the midtest. A final evaluation, again in the same manner as the baseline evaluation, was conducted at the conclusion of week 6. Residents also completed a survey at the final evaluation, indicating their difficult airway experience during their PGY-1 year and first 6 weeks of training.

Emergency Airway Scenario

An unanticipated cannot-ventilate, cannot-intubate scenario was developed for use at all 3 evaluation sessions. The simulated patient histories varied slightly at baseline, mid, and final evaluations, but each simulated patient history was a healthy adult, without known airway pathology, predictors of difficult airway, or previous anesthetics. Each patient was presenting for elective ambulatory surgery (orthopedic, aesthetic plastic, and gynecologic), requiring general endotracheal anesthesia. Before each testing session, the resident received general prebriefing instructions as follows: (1) he/she is assuming care of a patient in the operating room under anesthesia, (2) think out loud, and (3) ask at any time for anything, or anyone, needed. The scenario began with the resident being asked for assistance with airway management by a confederate acting as a medical student. The medical student informed the resident that he was a medical student and that after he and the attending anesthesiologist had induced anesthesia with propofol 200 mg and rocuronium 50 mg, the attending anesthesiologist had to emergently leave and was unavailable.

We used this opening of the scenario to obligate the resident to assume management of the airway. When a resident called for help from an anesthesiologist, he/she was informed that help had been urgently called but was not immediately available. If a resident called for help from a surgeon, he/she was informed that the surgeon did not feel comfortable with invasive airway management and that an otolaryngology surgeon was not immediately available. Simulating the typical setup in the operating room, an anesthesia cart contained supplies such as induction and resuscitation medications and intravenous catheters of various sizes, a styletted 7.5-mm endotracheal tube and laryngoscope fitted with a #3 Macintosh blade were on the anesthesia machine, and supraglottic airways (sizes 3 and 4 LMA) were available in the top drawer of the anesthesia machine. If the resident verbalized a request for a supraglottic airway, the medical student immediately provided the supraglottic airways. Information regarding fiber-optic bronchoscope, video laryngoscope, and other airway equipment availability was not provided before the scenario and would not have been readily available if requested. The cricothyroidotomy kit was not in the room. If the resident verbalized a request for a cricothyroidotomy kit, the medical student immediately provided the kit.

The vital sign sequence was identical at all testing sessions. Time zero for the scenario was the entry of the resident into the scenario. The initial vital signs remained unchanged for 15 seconds, followed by 2 phases of progressive oxygen desaturation and sympathetic stimulation and two phases of profound oxygen desaturation and development of bradycardia and relative hypotension (Fig. 2). The scenario ended at 5 minutes or when the resident initiated the cricothyroidotomy procedure, whichever came first. The termination time of the scenario was selected because cerebral hypoxia lasting longer than 3 to 5 minutes is associated with permanent brain injury.21 The medical student confederate remained in the room throughout the scenario and was knowledgeable about the patient history upon inquiry. The medical student was able to assist with patient monitoring as requested and obtaining and preparing, but not using, any requested equipment. The medical student did not prompt the resident, ask questions, make unsolicited observations, or suggest any course of action.

F2-3
FIGURE 2:
Scenario progression.

Statistical Analysis

The number of participants was a convenience sample determined by the number of incoming residents (n = 23) to the anesthesiology residency program. There were 11 residents assigned to the SGA group and 12 residents assigned to the CRIC group (Fig. 1). In a post hoc power analysis, the sample size of 11 in the SGA group achieves 98% power to detect the difference in supraglottic airway placements between the baseline and midtests using McNemar test for categorical variables with binomial attributes with α = 0.05 (DSS Statistical Power Calculator, Fort Worth, TX). The sample size of 12 in the CRIC group achieves 87% power to detect the difference in cricothyroidotomy placement between the baseline and midtest using McNemar statistics (α = 0.05). A P < 0.05 was required to reject the null hypothesis. Descriptive data for both groups were expressed as mean (SD) for continuous measures or percentage of a group for discrete measures. Numeric data were tested for normal distribution using the Kolmogorov-Smirnov test. Independent samples t test and repeated-measures analysis of variance were used to assess differences in response times. Categorical data were analyzed using the McNemar and Fisher exact tests. Statistical analyses were performed using SPSS software version 20 (Chicago, IL).

RESULTS

There was no difference between groups with regard to sex or type of internship. There was no difference with respect to previous exposure to difficult airway management techniques. Previous exposure was defined as either observation of or participation in the use of a technique. None of the residents who reported previous exposure to either supraglottic airway or cricothyroidotomy initiated the respective technique at the baseline test. There was no difference with respect to difficult airway experience during the first 6 weeks. Finally, there was no difference with respect to elective clinical airway experience throughout the study period (Table 1).

T1-3
TABLE 1:
Demographics and Airway Experience

We compared the initiation of supraglottic airway and cricothyroidotomy techniques within each group (within group analysis) and also assessed differences based on group assignment (between-group analysis).

Within-Group Analysis

Within-group analysis of supraglottic airway and cricothyroidotomy initiation is presented in Table 2. The SGA group significantly increased initiation of supraglottic airway from baseline to midtest, and performance was maintained from midtest to final evaluation. With regard to cricothyroidotomy, the SGA group did not change initiation of cricothyroidotomy from baseline to midtest. After the cricothyroidotomy training in the second 3-week period, the SGA group significantly increased initiation of cricothyroidotomy from midtest to final evaluation.

T2-3
TABLE 2:
Within-Group Analysis

The CRIC group significantly increased initiation of cricothyroidotomy from baseline to midtest, and performance was maintained from midtest to final evaluation. With regard to supraglottic airway, the CRIC group did not change initiation of supraglottic airway from baseline to midtest. Even after receiving training the supraglottic airway in the second 3-week period, the CRIC group demonstrated no significant change in initiation of supraglottic airway technique at the final evaluation.

Between-Group Analysis

Between-group analysis of initiation of supraglottic airway and cricothyroidotomy techniques is presented in the upper panel of Table 3. Significantly more residents in the CRIC group than in the SGA group attempted supraglottic airway at the baseline test. However, there was no significant association between previous exposure to SGA for difficult airway management and SGA initiation during the baseline test (McNemar, P = 1.000). No baseline difference was observed between groups for initiation of cricothyroidotomy.

T3-3
TABLE 3:
Between-Group Analysis

Only 9% of the residents in the SGA group and 17% of residents in the CRIC group attempted all 4 airway maneuvers (mask ventilation, direct laryngoscopy, supraglottic airway, and cricothyroidotomy) at the midtest (P = 1.000). Sixty-four percent of residents in the SGA group and 42% of residents in the CRIC group attempted all 4 airway maneuvers at the final evaluation (P = 0.29).

Subgroup Analysis

Further analysis focused on the performance of residents who did not attempt either technique at the baseline test. Therefore, 5 residents who attempted supraglottic airway and 3 residents who initiated cricothyroidotomy at baseline test were excluded. The subgroup analysis is presented in the bottom panel of Table 3.

After completion of training in both techniques, there were no differences in initiation of cricothyroidotomy: in both subgroups, 80% of the residents initiated cricothyroidotomy. However, with regard to supraglottic airway, although 80% of residents from the SGA subgroup attempted supraglottic airway, none of the residents from the CRIC subgroup attempted supraglottic airway.

With regard to overall performance, 70% of the residents in the SGA subgroup attempted all 4 airway maneuvers. By contrast, none of the residents from the CRIC subgroup attempted all 4 airway maneuvers (P = 0.03).

Use of Other Airway Techniques and Calling for Help

Although mask ventilation and direct laryngoscopy were not part of the practical training curriculum, all residents performed mask ventilation as a first maneuver at mid and final evaluations. Ninety-two percent of the residents performed direct laryngoscopy at the midtest, and 100% performed direct laryngoscopy at the final evaluation.

No resident requested fiber-optic bronchoscope, video laryngoscope, or any other airway equipment at baseline, mid, or final evaluations. No resident attempted needle cricothyroidotomy at baseline, mid, or final evaluations.

Despite having been informed that the attending anesthesiologist was unavailable, all residents at baseline, mid, and final evaluations still correctly called for help during the first desaturation phase of the scenario.

Response Times

In addition to maneuvers attempted, response times were measured. These data were normally distributed.

Airway Takeover Time

Airway takeover times are presented in Table 4. Both groups significantly decreased takeover time among evaluation sessions. However, there was no significant difference among groups at any evaluation session.

T4-3
TABLE 4:
Airway Takeover Time

Final Evaluation: Time to Supraglottic Airway and Cricothyroidotomy

The times to initiation of supraglottic airway and cricothyroidotomy maneuvers in those residents who performed both supraglottic airway and cricothyroidotomy after training in both techniques are presented in Figure 3. The mean time to supraglottic airway initiation was longer in the CRIC group compared with the SGA group. The time to cricothyroidotomy incision was significantly faster in the CRIC group than in the SGA group. In addition, the interval between initiation of supraglottic airway and cricothyroidotomy in those residents who performed both supraglottic airway and cricothyroidotomy was significantly shorter in the CRIC group than in the SGA group.

F3-3
FIGURE 3:
Response times. Times to supraglottic airway placement and cricothyroidotomy were measured in those residents who performed both supraglottic airway and cricothyroidotomy at the final test (SGA group, n = 7; CRIC group, n = 5). SGA Time: Time in seconds from takeover to initiation of supraglottic airway placement; CRIC time, time in seconds from takeover to initiation of cricothyroidotomy incision; interval time, interval in seconds between SGA time and CRIC time. The mean SGA time was not significantly different between the SGA group (58 [13] seconds) and CRIC group (77 [21] seconds) P = 0.076. On the contrary, CRIC time was shorter in CRIC group (121 [21] seconds), as compared with SGA group (150 [22] seconds) P = 0.048. The interval between SGA time and CRIC time in those residents who performed both supraglottic airway and cricothyroidotomy was significantly shorter in the CRIC group (44 [17] seconds) than in the SGA group (91 [19] seconds) P = 0.001. Statistical test, independent samples t test.

DISCUSSION

After initial training in only 1 technique, both groups demonstrated bias for their respective technique. This was as expected. Despite a detailed presentation of the difficult airway algorithm, a didactic lecture did not prevent bias in either group after training in only 1 device.

The chief finding of this study was the appearance of an unexpected asymmetrical effect of training sequence after the conclusion of emergency airway training in both supraglottic airway and cricothyroidotomy. Beginning with training in cricothyroidotomy, the CRIC group demonstrated persistent bias for cricothyroidotomy and did not increase initiation of supraglottic airway despite subsequent supraglottic airway training and routine experience with supraglottic airway during clinical duties. By contrast, beginning with practical training in supraglottic airway resulted in increased initiation of both devices during simulated emergency airway management. To our knowledge, this is the first report of such a finding in medical education.

The analysis of final test response times offers further insight regarding the difference in behavior. Although all residents began with efforts at mask ventilation and direct laryngoscopy, the CRIC group was slower to initiate supraglottic airway and initiated cricothyroidotomy sooner than the SGA group. By contrast, the SGA group was faster to initiate supraglottic airway and initiated cricothyroidotomy later. There are no definitive standards for timing of supraglottic airway and cricothyroidotomy initiation because a number of additional clinical variables must be considered. Nevertheless, for the identical conditions presented to all participants in this study, the asymmetry of the groups based on training sequence is notable. For the CRIC group, the short interval time between supraglottic airway and cricothyroidotomy may represent a reduction of window of opportunity to rescue and pose a greater risk for bypassing supraglottic airway.

Furthermore, we analyzed the use of techniques in the subgroups including only residents who performed neither supraglottic airway nor cricothyroidotomy at baseline. Although group sizes are small, the convergent data from the subgroups complement the overall finding of a bias effect.

Cognitive error, such as those resulting from cognitive bias, is a thought process error distinct from knowledge or technical deficits. The types of cognitive error and predisposing biases are many, but they are all low-visibility, latent failures.22–25 Latent failure is a precondition that remains hidden until a triggering event exposes the potential harm to patients. Simulation has been used as a method to investigate latent errors.26 In this study, simulation served a dual purpose as a training methodology and strategy to uncover latent conditions.

Cognitive error seems to be a prevalent latent problem in anesthesiology.24 Indeed, the phenomenon of bypassing supraglottic airway for cricothyroidotomy has been observed in other studies. In a study of practicing anesthesiologists, subjects participated in a session reviewing difficult airway guidelines and hands-on cricothyroidotomy teaching. In cannot-ventilate, cannot-intubate scenarios after the session, the number of participants bypassing supraglottic airway was nearly doubled, and the time to cricothyroidotomy was significantly decreased.27 In another study, 54% of senior anesthesiology residents bypassed supraglottic airway in an obstetric cannot-ventilate, cannot-intubate scenario, whereas 94% performed cricothyroidotomy.28 The observations of these studies support our data and highlight the importance of cognitively informed curriculum design and delivery.

Supraglottic airway devices are used every day for routine elective airway management during surgical anesthesia as well as for airway rescue. In contrast, cricothyroidotomy is a rarely necessary, and always emergent, procedure. Knowing that trainees use supraglottic airways in routine practice, educators may prefer to focus on the high-stakes skill of cricothyroidotomy. In 2 studies assessing practicing anesthesiologists in a cannot-ventilate, cannot-intubate scenario, cricothyroidotomy was a training goal and outcome measure, but supraglottic airway placement was not.29,30 This may be educationally and clinically justified, but such training may induce cognitive bias. In this study, all residents had clinical experience with elective supraglottic airways for routine airway management during surgical anesthesia. However, for the CRIC group, additional experience with this device in one cognitive frame (routine/elective) did not result in application of that skill in another cognitive frame (simulated emergency airway).

The persistent bias for cricothyroidotomy may have partly been an effect of primacy, in which learning or memory is biased toward the first stimulus presented.5 Primacy has been observed in a variety of settings.31,32 However, a corresponding primacy effect in the SGA group was not observed, arguing against primacy as a dominant effect. Cricothyroidotomy may also have a more salient, or memorable, mental representation because its nature as a rare and invasive technique draws more attention than frequently used, noninvasive techniques. Furthermore, as a stimulus becomes routine, such as elective experience with the supraglottic airway, the amount of attention to and additional learning about the routine stimulus may be impaired.33 The ongoing elective experience in supraglottic airway might therefore have contributed to an even weaker learning outcome when supraglottic airway training followed cricothyroidotomy training.

The SGA group received training in the expected order of actions (ie, supraglottic airway as rescue attempt, followed by cricothyroidotomy as a final intervention), but the CRIC group received training in a scrambled order. The chunking34 and scrambling of clinical actions during training may also have contributed to the group differences.

Metacognition, processing by the learner about when and how to use particular strategies for problem solving, is an important component of decision making. Dual process theory is a metacognitive model that provides insight into cognition. According to this theory, 2 cognitive systems are used to reason and make decisions. System I, the “intuitive” system, is a fast and automated response system best used for routine decisions. System II, the “analytical” system, is slower and cognitively demanding, using conscious application of learned rules.35,36 Individuals default to a state requiring low cognitive effort without checking,37 even during urgent situations.38 System I thinking is therefore prone to bias and judgment error. For the CRIC group, a default to system I thinking may have contributed to the persistent bias. It seems that both reasoning strategies are important in avoiding error,39 and because system I is always active, its bias can only be mitigated by the enhanced monitoring and vigilance of system II.40 In addition to teaching skills and knowledge, educators may consider strategies to teach learners to engage both systems of thinking, especially in high-stakes situations, to minimize bias.

Limitations

Because of scheduling limitations, the didactic lecture could not be given to all residents simultaneously, and a single instructor could not teach all sessions. Therefore, the lecture was standardized for content and delivery, and efforts were made to limit the number and to maximize consistency of instructors. We cannot tell whether residents would have identified a correct sequence of maneuvers in a written test. We considered assessing knowledge via a written test, but whether a test either preceded or followed the scenario, one could have significantly influenced the other. In addition, our primary outcome measure was not knowledge, but deployment of decisions; therefore, we did not administer a written test.

We did not obtain qualitative measures such as structured interviewing partly because doing so at the baseline or midtest could have had significant influence on the subsequent evaluations, and partly because we did not expect to observe a bias effect after completion of training in both techniques. We therefore do not know the participants’ assumptions and thoughts about the decisions they made or what was expected of them. For example, we cannot rule out the possibility of experimenter bias, in which subjects, consciously or unconsciously, attempt to predict what might be the desired answer. This may have influenced the timed measures in the simulation test.

We assessed a single variable of training sequence. Although we have discussed several potential contributing sources of bias, this data set does not allow the specific delineation of the types of bias, a topic for further investigation.

Not all residents achieved 100% initiation of all 4 airway maneuvers. We would expect 100% achievement only with a mastery-training design, where the amount of training may vary for each subject until the set goal of 100% is achieved. Standardization of the amount and type of training for each participant was an essential study design element to study the effect of training sequence. Bias relating to mastery-training experimental design is another variable worthy of investigation.

Finally, this study assessed initiation of airway management techniques in one type of simulated scenario, with limited time and opportunities for choice of action, in a relatively small sample of novice residents from 1 institution during 6 weeks. We do not know how performance in a simulated event would correspond with real clinical behaviors, and interpretation of the data should be undertaken with caution. Data from longer-term follow-up are necessary, and additional data from subjects with more clinical experience, over a range of scenarios, and additional variables in training strategy, will further enhance understanding of the complex relationships between training and bias.

CONCLUSIONS

More than 75% of US and Canadian anesthesiology residency programs use simulation for airway management training.41 Simulation is used for training supraglottic airway placement42 and cricothyroidotomy skills.43,44 As the goal of transferring skills to the clinical environment is progressively achieved,45,46 understanding the cognitive underpinnings of medical education in general and simulation-based training in particular is an opportunity to guide educators how best to train medical professionals.47

Selection and order of presentation of examples and stimuli is likely to play a role in determining the outcome and effectiveness of training. The examples, order, and manipulation of saliency have been shown to play a critical role in training US Air Force pilots.48,49 In the case of emergency airway training, it seems that similar elements affected decision making and vulnerability to cognitive bias.

Our study showed that the sequence of training in a simulation-based emergency airway curriculum contributed to the formation of cognitive bias. Therefore, curricular design should consider potential effects of cognitive bias and error. Additional studies can further characterize the causes of and corrections for training-based biases and errors. By deciphering the underpinnings of cognitive bias in education, educators will be able to tailor strategies to minimize bias to improve patient safety.

REFERENCES

1. Dror IE. The paradox of human expertise: why experts get it wrong. In: Kapur N, ed. The Paradoxical Brain. Cambridge, UK: Cambridge University Press; 2011: 177–188.
2. Gilovich T, Griffin D, Kahneman D. Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge, UK: Cambridge University Press; 2002.
3. Rehak LA, Adams B, Belanger M. Mapping biases to the components of rationalistic and naturalistic decision making. Proc Hum Factors Ergonomics Soc 2010; 54: 324–328.
4. Dror I. Technology enhanced learning: the good, the bad, and the ugly. Pragmatics Cogn 2008; 16: 215–223.
5. DiGirolamo G, Hintzman D. First impressions are lasting impressions: a primacy effect in memory for repetitions. Psychon Bull Rev 1997; 4: 121–124.
6. Bruppacher HR, Alam SK, LeBlanc VR, et al. Simulation-based training improves physicians’ performance in patient care in high-stakes clinical setting of cardiac surgery. Anesthesiology 2010; 112: 985–992.
7. Daniels K, Arafeh J, Clark A, Waller S, Druzin M, Chueh J. Prospective randomized trial of simulation versus didactic teaching for obstetrical emergencies. Simul Healthc 2010; 5: 40–45.
8. McCoy CE, Menchine M, Anderson C, Kollen R, Langdorf MI, Lotfipour S. Prospective randomized crossover study of simulation vs. didactics for teaching medical students the assessment and management of critically ill patients. J Emerg Med 2011; 40: 448–455.
9. Niazi AU, Haldipur N, Prasad AG, Chan VW. Ultrasound-guided regional anesthesia performance in the early learning period: effect of simulation training. Reg Anesth Pain Med 2012; 37: 51–54.
10. Wang CL, Schopp JG, Petscavage JM, Paladin AM, Richardson ML, Bush WH. Prospective randomized comparison of standard didactic lecture versus high-fidelity simulation for radiology resident contrast reaction management training. AJR Am J Roentgenol 2011; 196: 1288–1295.
11. Park CS. Simulation and quality improvement in anesthesiology. Anesthesiol Clin 2011; 29: 13–28.
12. Benumof JL. Management of the difficult adult airway. With special emphasis on awake tracheal intubation. Anesthesiology 1991; 75: 1087–1110.
13. Peterson GN, Domino KB, Caplan RA, Posner KL, Lee LA, Cheney FW. Management of the difficult airway: a closed claims analysis. Anesthesiology 2005; 103: 33–39.
14. American Society of Anesthesiologists Task Force on Management of the Difficult Airway. Practice guidelines for management of the difficult airway: an updated report by the American Society of Anesthesiologists Task Force on Management of the Difficult Airway. Anesthesiology 2003; 98: 1269–1277.
15. Isaacs JH Jr, Pedersen AD. Emergency cricothyroidotomy. Am Surg 1997; 63: 346–349.
16. Park CS, Rochlen LR, Yaghmour E, et al. Acquisition of critical intraoperative event management skills in novice anesthesiology residents by using high-fidelity simulation-based training. Anesthesiology 2010; 112: 202–211.
17. Holmes JF, Panacek EA, Sakles JC, Brofeldt BT. Comparison of 2 cricothyrotomy techniques: standard method versus rapid 4-step technique. Ann Emerg Med 1998; 32: 442–446.
18. Schaumann N, Lorenz V, Schellongowski P, et al. Evaluation of Seldinger technique emergency cricothyroidotomy versus standard surgical cricothyroidotomy in 200 cadavers. Anesthesiology 2005; 102: 7–11.
19. MacIntyre A, Markarian MK, Carrison D, Coates J, Kuhls D, Fildes JJ. Three-step emergency cricothyroidotomy. Mil Med 2007; 172: 1228–1230.
20. Melker JS, Gabrielli A. Melker cricothyrotomy kit: an alternative to the surgical technique. Ann Otol Rhinol Laryngol 2005; 114: 525–528.
21. Hemphill JC III, Smith WS, Gress DR. Neurologic Critical Care, Including Hypoxic-Ischemic Encephalopathy and Subarachnoid Hemorrhage, Harrison’s Principles of Internal Medicine, 18th edition. Edited by Longo DL, Fauci AS, Kasper DL, Hauser SL, Jameson JL, Loscalzo J. New York, McGraw-Hill, 2012. Available at: http://www.accessmedicine.com.ezproxy.galter.northwestern.edu/content.aspx?aID=9111325. Accessed June 5, 2013.
22. Arnstein F. Catalogue of human error. Br J Anaesth 1997; 79: 645–656.
23. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 2003; 78: 775–780.
24. Stiegler MP, Neelankavil JP, Canales C, Dhillon A. Cognitive errors detected in anaesthesiology: a literature review and pilot study. Br J Anaesth 2012; 108: 229–235.
25. Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what’s the goal? Acad Med 2002; 77: 981–992.
26. Blike GT, Christoffersen K, Cravero JP, Andeweg SK, Jensen J. A method for measuring system safety and latent errors associated with pediatric procedural sedation. Anesth Analg 2005; 101: 48–58.
27. Borges BC, Boet S, Siu LW, et al. Incomplete adherence to the ASA difficult airway algorithm is unchanged after a high-fidelity simulation session. Can J Anaesth 2010; 57: 644–649.
28. Balki M, Cooke ME, Dunington S, Salman A, Goldszmidt E. Unanticipated difficult airway in obstetric patients: development of a new algorithm for formative assessment in high-fidelity simulation. Anesthesiology 2012; 117: 883–897.
29. Kuduvalli PM, Jervis A, Tighe SQ, Robin NM. Unanticipated difficult airway management in anaesthetised patients: a prospective study of the effect of mannequin training on management strategies and skill retention. Anaesthesia 2008; 63: 364–369.
30. Siu LW, Boet S, Borges BC, et al. High-fidelity simulation demonstrates the influence of anesthesiologists’ age and years from residency on emergency cricothyroidotomy skills. Anesth Analg 2010; 111: 955–960.
31. Mantonakis A, Rodero P, Lesschaeve I, Hastie R. Order in choice: effects of serial position on preferences. Psychol Sci 2009; 20: 1309–1312.
32. Carney D, Banaji M. First is best. J Behav Decis Mak 2012; 22: 378–389.
33. Hintzman D, Curran T, Oppy B. Effects of similarity and repetition on memory: registration without learning? J Exp Psychol Learn Mem Cogn 1992; 14: 240–247.
34. Ruitenberg MF, De Kleine E, Van der Lubbe RH, Verwey WB, Abrahamse EL. Context-dependent motor skill and the role of practice. Psychol Res 2012; 76: 812–820.
35. Pelaccia T, Tardif J, Triby E, Charlin B. An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. Med Educ Online 2011; 16. Available at: http://med-ed-online.net/index.php/mep/article/view/5890. Accessed November 6, 2013.
36. Thompson VA, Prowse Turner JA, Pennycook G. Intuition, reason, and metacognition. Cogn Psychol 2011; 63: 107–140.
37. Kahneman D, Klein G. Conditions for intuitive expertise: a failure to disagree. Am Psychol 2009; 64: 515–526.
38. Croskerry P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ Theory Pract 2009; 14 (Suppl 1): 27–35.
39. Eva KW, Hatala RM, Leblanc VR, Brooks LR. Teaching from the clinical reasoning literature: combined reasoning strategies help novice diagnosticians overcome misleading information. Med Educ 2007; 41: 1152–1158.
40. Kahneman D. Thinking Fast and Slow. New York, NY: Farrar, Straus and Giroux; 2011: 28.
41. Pott LM, Randel GI, Straker T, Becker KD, Cooper RM. A survey of airway training among U.S. and Canadian anesthesiology residency programs. J Clin Anesth 2011; 23: 15–26.
42. Laiou E, Clutton-Brock TH, Lilford RJ, Taylor CA. The effects of laryngeal mask airway passage simulation training on the acquisition of undergraduate clinical skills: a randomised controlled trial. BMC Med Educ 2011; 11: 57.
43. Wong DT, Prabhu AJ, Coloma M, Imasogie N, Chung FF. What is the minimum training required for successful cricothyroidotomy?: a study in mannequins. Anesthesiology 2003; 98: 349–353.
44. Friedman Z, You-Ten KE, Bould MD, Naik V. Teaching lifesaving procedures: the impact of model fidelity on acquisition and transfer of cricothyrotomy skills to performance on cadavers. Anesth Analg 2008; 107: 1663–1669.
45. Ross AJ, Kodate N, Anderson JE, Thomas L, Jaye P. Review of simulation studies in anaesthesia journals, 2001–2010: mapping and content analysis. Br J Anaesth 2012; 109: 99–109.
46. Nestel D, Groom J, Eikeland-Husebo S, O’Donnell JM. Simulation for learning and teaching procedural skills: the state of the science. Simul Healthc 2011; 6 (Suppl): S10–S13.
47. Dror I, Schmidt P, O’Connor L. A cognitive perspective on technology enhanced learning in medical training: great opportunities, pitfalls and challenges. Med Teach 2011; 33: 291–296.
48. Dror IE, Stevenage SV, Ashworth ARS. Helping the cognitive system learn: exaggerating distinctiveness and uniqueness. Appl Cogn Psychol 2008; 22: 573–584.
49. Ashworth ARS, Dror IE. Objective identification as a function of disriminability and learning presentations: the effect of stimulus similarity and canonical frame alignment on aircraft identification. J Exp Psychol 2001; 6: 148–157.
Keywords:

Cognitive bias; Cognitive error; Simulation; Difficult airway; Emergency airway; Cricothyroidotomy

© 2014 by Lippincott Williams & Wilkins, Inc.