STEPS COMMON TO ALL SCENARIOS
1. Assemble a Team of Scenario Developers
Members of the scenario development team should include those with clinical expertise, knowledge of the CRM principles being assessed, and familiarity with the measurement system being used. For example, the researchers in our planning team included a multidisciplinary group of nurses and physicians well-versed in the principles of CRM. Several members were proficient with the ANTS scale.
2. Consider Study Participants When Selecting Patient Characteristics
Next, consider the levels of clinical expertise of the participants as well as their medical disciplines. The selection of patient characteristics can be tailored to the experience of study participants, such as pairing pediatric nurses with pediatric patients. In this example, the physiologic parameters and basic traits of the simulator patient (such as age and weight) were identical for each scenario. This standardization contributed to symmetry among the scenarios in the series, acting as a control to eliminate the differing clinical issues associated with different patients and different clinical signs. Also, participants can focus on CRM principles rather than on technical details given that a standard patient weight simplifies the calculation of drug dosages.
3. Choose a “Crisis Event”
To further deemphasize clinical and technical issues, we avoided highly complex medical problems and presented circumstances where mastery of behavioral skills would promote smooth handling of the crisis. Additionally, based on the clinical expertise of the design team, a straightforward, typical, single pathology was chosen as the triggering event for the crisis. For example, each of the four scenarios involves a medication error leading to a hypoxic event. Medication errors are a daily risk encountered by all health care providers and provide a logical clinical basis for hypoxia, making the resulting scenarios realistic. This crisis event can be recognized by multiple levels of clinicians from medical students to nurses to attending physicians, and therefore was appropriate for participants with different skill sets.
4. Develop Scenario Flow
The goal of this step is to determine how and when the study participants will appear in the scenario and how varying the makeup of the crisis team can elicit CRM behaviors. For example, the scenario begins when pediatric nurses assume care of the simulated patient from an actor. These first responders may then call for help from a more specialized pool of health care providers, such as pediatric residents, who may eventually involve a participating anesthesia resident. The basic flow of this scenario algorithm tests the ability of the team to recognize the crisis early, identify a leader, declare an emergency, exchange information, and support the other team members. Because exchange of information among the group is a key element of team working, it is essential to provide opportunities for several handovers so that this action can be assessed. If appropriate, a new leader may be chosen, who is challenged to display new authority. This person must assess the capabilities of the evolving group and assign tasks appropriately.
In this example, the hypoxic event begins to unfold relatively quickly. If it is not resolved, the simulated patient becomes bradycardic, requiring the participants to initiate Pediatric Advanced Life Support (PALS) protocols. The scenario ends when the patient is stabilized, either by securing the airway or by reversing the medication error (Fig. 1).
Because the makeup of the team members present at the crisis varies, much like real-life emergencies, these scenarios require the group to support each other in achieving each goal. As the flow of the scenarios demand that the roles of the players change by taking on new roles and needing to continuously exchange old and new information, it becomes apparent which teams are supportive and work cohesively during the crisis. Additionally, the varying makeup of the teams demands good task allocation and distribution of work by the leader.23
Steps Unique to Each Scenario
Until this point, every scenario in this series has the same framework to create parallel scenarios. But by changing enough variables, such as location of the crisis event, the particular cause of the problem (slow incipient onset of hypoxia, followed by apnea), and the roles of the actors, the scenarios appear different, despite using identical pathophysiology.
5. Develop Events Leading up to Crisis
The first step in developing individual scenarios is to create varying background stories leading up to the crisis event. For example, each of the patients has a different specific reason for becoming hypoxic, such as accidentally receiving a narcotic overdose for a procedure. The sequence of events before the first responders arrive must be brief but adequately convoluted so that the participants will need to work together to ascertain the correct history. Here they have opportunities to use situational awareness skills, identify resources, gather details, and exchange information.
6. Select a Setting for the Crisis
The location and setting of each scenario is varied to create the sense of a new scenario and preserve suspension of disbelief. The participants are put in generalized, recognizable settings instead of locations unfamiliar to them. For example, in two of the scenarios, the nurses are asked to transport a patient to his room from the MRI scanner or postanesthesia recovery room. These are typical, reasonable tasks for this group of trainees.
7. Define the Distracters and Distractions
The next step is to define the distractions and interrupters that are present during the crisis. Task management is challenged by altering the position of the patient or the bed and adding a noise disturbance (e.g., a loud radio or loud TV). Live actors who divert the attention of the participants challenge team working and situation awareness. Two types of actors were chosen: a “clueless” personality who offers no help and constantly draws attention away from the patient, and a “potentially helpful” individual who possesses some relevant information regarding the pathogenesis of the problem and may be used to perform certain tasks during the crisis if requested.
Proper planning and preparing can help the team to prioritize current and possible future events. As equipment malfunctions are a common issue faced by multiple levels of health care providers, two types of equipment issues are represented: equipment failures and absent equipment. These failures can be managed by successful teamwork and create opportunities for decision making. For example, the nature of the crisis event requires the team members to assess the airway and provide oxygen for the hypoxic patient. Equipment failures and distractions rapidly create new problems and trigger new crisis events as the initial problem increases in severity.23 A faulty bag-valve-mask and empty or missing oxygen tank connector are smaller problems that can distract from the main issue (proper patient care) and add up to create a sequence of events more problematic than the initial crisis.23 If the participants never realize that there is an equipment failure, the crisis continues to worsen and challenges the participants to set new priorities and find the origin of new problems. In this way, the scenario regenerates opportunities to demonstrate effective task management.
This method creates symmetrical scenarios that require groups to deal with other types of distractions: predisposing factors and latent errors. These are the underlying aspects of the crisis that are present before the event.23 They are usually hidden factors and pose no distraction until the adverse consequences declare themselves in the midst of the crisis. The team is challenged by a clueless nursing student to assess his limited capabilities and use assertiveness to remove him from the crisis if he becomes a liability. His annoying presence is not important and he can be ignored initially until the nurses become preoccupied with the ensuing emergency. Then, his persistence challenges the leader's vigilance and may cause mismanagement of the crisis. He can thwart the team's ease in exchanging information and coordinating activities. Alternatively, an efficient team can make this person a nonmedical resource, and turn the most annoying distracter into a valuable helper or messenger.
Equipment failures also provide opportunities for team members to exchange information about missing or faulty equipment. The team leader is confronted with issues in coordinating activities, relegating distractions, and reducing fixation errors. The identification of some equipment issues may be limited to those with more in-depth knowledge of the tools; an anesthesiologist may recognize a faulty bag-valve-mask more readily than a pediatric nurse and will need to report this information to the leader. Since there are several failures in each scenario, there is a higher chance that the team will recognize and address this type of issue.
8. Make Resources Available
Despite introducing multiple distractions to challenge the subjects' CRM skills, multiple resources should be available during the crisis to assist them when they seek help and information. Replacement equipment and additional patient documentation, such as a history and physical, anesthesia record, or other pertinent medical records should be available. Actors can positively contribute to the efforts of the group; they should be instructed to give information when specifically questioned by the participants but refrain from volunteering any supplemental information. Incorporating teams of providers with different areas of expertise force them to turn towards each other as resources.
9. Write a Script for the Actors
Scripts of key phrases and comments for each scenario were written and distributed to the actors to serve as a guide for their dynamic portrayal as the human distracters. Specific cues such as “Wow, the patient looks really blue,” were mixed with unhelpful comments similar to, “This is my first day working in MRI,” and more distracting statements like, “You need to move this patient because we have a full schedule of scans to do.” Upon completion and finalization of the scenarios in our series, we provided wireless headsets to the actors and practiced scenario run-throughs to facilitate smooth and standardized performances.
The product of our practical example is the series of scenarios shown in Table 5. Organizing the data in this way not only helps to identify the symmetrical glitches among these four scenarios, but also aids in setting up the details of the actual simulation. Because we assessed the CRM skills of participants using the ANTS system, we were required to ensure that this scenario development plan includes adequate testing of the nontechnical skills described in this particular assessment tool. Taking each separate component of the scenario and matching it to a particular element in a scoring system produces a graphical representation like Table 6. Many elements are assessed more than once by each of the circumstances. This duplication is intentional and increases the probability that the participants will have opportunities to demonstrate these behaviors.
This is the first methodological description of the development of a series of interchangeable simulation scenarios for the purpose of teamwork skills evaluation. It allows researchers to create a series of symmetrical scenarios to test crisis behavioral skills such as communication, avoiding fixations, team working, leadership, and situation awareness. This method deemphasizes the importance of clinical knowledge and generates conditions within the scenario to reward effective task management and use of resources. These scenarios are likely amenable to evaluation by various behavioral rating scales, including ANTS (Table 6).
The scenarios created by using this method make it possible to assess a group's anticipation and avoidance of more serious problems. For example, the simple problem of hypoxia is critical enough to require the team to anticipate further problems. They must prepare for the possibility of cardiac arrest while simultaneously addressing the immediate concerns. As they evolve, these scenarios require team members to either avoid or anticipate intubation. They must make the decision to reverse the medication error with naloxone before the patient becomes too unstable and needs a definitive airway. Gaba and coauthors refer to this concept as the “prediction of future states” and point out that proper anticipation also affects the time frame available for future actions.23
If a leader is acknowledged early in each scenario, then the ability of the team to enact sound risk evaluation and decision making is improved.23 The leader should assume the role of primary decision maker and not be involved in the task management.23 Gaba labels this person the “supervisory controller,” who must reflect on past choices, identify options, balance risks, and make decisions.23 The leader should be vigilant and constantly improve his situation awareness, understand the data, verify the data, continually reevaluate circumstances, and determine which actions are effective.23 This scenario development scheme provides opportunities to gather information from the patient, monitors, and actors, who can be resources as well as distracters. As the crisis unfolds and becomes more serious, new data is introduced and it becomes the responsibility of the leader and the team members to recognize and appreciate the new information.
This method was also developed to assess effective communication between leaders and followers, as it is one of the most important qualities of a good team. The exchange of ideas involves patient assessment, resource identification, and decision making. The noise distractions obscure clear and specific instructions but they are manageable by the participants who can reduce the commotion by eliminating the distraction. Finding solutions to equipment errors requires followers to close the communication loop and report back to the leader.23 Effective communication also involves giving precise commands, allocating specific tasks to specific people, and supporting open exchange of information while avoiding conflict.
Most of the research performed on CRM exists in the aviation and anesthesiology literature. This methods paper expands those boundaries by incorporating multiple types of practitioners. We chose to assign pediatric nurses to the role of first responders, as nurses typically discover a crisis. However, the order in which groups arrive to the scene is interchangeable and creates a different dynamic with each new arrangement. Any mix of providers is possible within this framework, from nonmedical personnel and technicians to nurses and physicians. This can accurately simulate the sequence of genuine crisis events.
One of our first challenges was to make scenarios that are similar in design while preserving the suspension of disbelief. This concept is the ability of the participants to imagine themselves in a real-life crisis and put on hold the knowledge that they are in a simulated environment with an electronic patient. With each new 5-year-old patient, it may be more difficult for participants to imagine themselves in a brand new crisis. Furthermore, it is possible that they may expect a patient with hypoxia as they encounter each new scenario in the series. Their ability to predict the circumstances common to all scenarios may affect certain elements of crisis resource management, such as situation awareness and decision making. The nurses may first place oxygen on the patient because they recall that particular intervention from a previous crisis, not because they have properly assessed the situation and have recognized the problem. We did not specifically ask our participants for feedback regarding our series of scenarios and any perceived similarities. This may be one of the first steps in future validation of this method. Even so, a certain amount of predictability among the scenarios may be acceptable to preserve their design similarity.
The greatest advantage of this system is the ability to create many different scenarios based on the same framework and the same sequence of programmed simulation events. The number of comparable scenarios that can be generated is endless and their order can be randomized for use in controlled trials. Given the same basic equipment and number of people required to act in each scenario, multiple CRM sessions can be given to a group of participants within a short period of time with minimal preparation.
As other scoring systems for assessing CRM behaviors are developed, this method can be adapted for objective evaluation with those new tools. Informal subjective evaluation during CRM debriefings is also achievable if no validated scoring system is used to evaluate performance. Further research may involve replicating the method for the development of scenarios for a greater variety of health care providers interested in receiving CRM training. The method can be expanded to test clinical competency for simple clinical problems for any level of provider. It may also prove useful for developing crisis management scenarios that assess technical skills and procedural competency for courses such as PALS and ACLS.
Careful design of CRM scenarios is essential for assessment and future research of teamwork skills to move forward. This method can guide the development of simulation scenarios that consistently assess crisis behavioral skills. As one of the first methodologies described for scenario development, more rigorous testing and use by other researchers is necessary to further define its limitations. However, this novel concept of symmetrical design for a series of simulation scenarios represents an important new technique for the scenario developer's toolbox.
1. Wayne DB, Butter J, Siddall VJ, et al.: Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation
technology and deliberate practice. J Gen Intern Med
2. Stefanidis D, Sierra R, Korndorffer JR, Jr, et al.: Intensive continuing medical education
course training on simulators results in proficiency for laparoscopic suturing. Am J Surg
3. Hassan I, Maschuw K, Rothmund M, et al.: Novices in surgery are the target group of a virtual reality training laboratory. Eur Surg Res
4. Andreatta PB, Woodrum DT, Birkmeyer JD, et al.: Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg
5. Sutherland LM, Middleton PF, Anthony A, et al.: Surgical simulation
: a systematic review. Ann Surg
6. Chou B, Handa VL: Simulators and virtual reality in surgical education
. Obstet Gynecol Clin North Am
7. Korndorffer JR, Jr, Dunne JB, Sierra R, et al.: Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg
8. Seymour NE, Gallagher AG, Roman SA, et al.: Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg
9. Blum MG, Powers TW, Sundaresan S: Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy. Ann Thorac Surg
10. Colt HG, Crawford SW, Galbraith O, 3rd: Virtual reality bronchoscopy simulation
: a revolution in procedural training. Chest
11. Ost D, DeRosiers A, Britt EJ, et al.: Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med
12. Holzman RS, Cooper JB, Gaba DM, et al.: Anesthesia crisis resource management
: real-life simulation
training in operating room crises. J Clin Anesth
13. Yule S, Flin R, Paterson-Brown S, Maran N: Non-technical skills for surgeons in the operating room: a review of the literature. Surgery
14. Howard SK, Gaba DM, Fish KJ, et al.: Anesthesia crisis resource management
training: teaching anesthesiologists to handle critical incidents. Aviat Space Environ Med
15. Reznek M, Smith-Coggins R, Howard S, et al.: Emergency medicine crisis resource management
(EMCRM): pilot study of a simulation
-based crisis management course for emergency medicine. Acad Emerg Med
16. Yee B, Naik VN, Joo HS, et al.: Nontechnical skills
in anesthesia crisis management with repeated exposure to simulation
17. Gaba DM, Howard SK, Flanagan B, et al.: Assessment of clinical performance during simulated crises using both technical and behavioral ratings. Anesthesiology
18. Fletcher G FR, McGeorge P, Glavin R, et al.: Rating non-technical skills: Developing a behavioural marker system for use in anaesthesia. Cognition Technol Work
19. Fletcher G, Flin R, McGeorge P, et al.: Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth
20. Reader T, Flin R, Lauche K, Cuthbertson BH: Non-technical skills in the intensive care unit. Br J Anaesth
21. Flin R ML, Goeters K: Development of the NOTECHS (non-technical skills) system for assessing pilots' CRM skills. Human Factors Aerospace Safety
22. Kim J, Neilipovitz D, Cardinal P, et al.: A pilot study using high-fidelity simulation
to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation
, and Crisis Resource Management
I Study. Crit Care Med
23. Gaba DM, Fish KJ, Howard SK. Crisis management in anesthesiology.
New York: Churchill Livingstone; 1994.
Keywords:© 2007 Society for Simulation in Healthcare
Simulation; Nontechnical skills; Crisis resource management; Crisis management behaviors; Scenario development; Behavioral skills; Anesthesia nontechnical skills; Education; Teamwork; Research; Communication