Severe sepsis/septic shock (SS/SS) is a complicated and often life-threatening critical illness that requires a complex and multifaceted treatment strategy. Two recent developments have targeted implementation of specific evidence-based measures to improve treatment of SS/SS: Volunteer Hospital Association’s (VHA) Sepsis Bundle and the International consensus guidelines contained within the Surviving Sepsis Campaign.1–3 This collection of sepsis guidelines has been used by intensive care units (ICUs) and applied to patients with SS/SS with the expectation that full application of a group of related evidence-based measures will improve patient outcomes. However, the education offered as part of the implementation strategies for this and other complex ICU treatment plans is often presented in didactic sessions, which are individual and passive, and may neglect the importance of other more dynamic factors occurring during the critical moments of a crisis scenario in the ICU. It is unknown whether these factors may lead to errors or omissions of the sepsis guidelines or other evidence-based ICU bundles.
Simulation is rapidly developing into a paradigm shift of a process unchanged for centuries in medical education.4–8 Simulation training has been used to improve skills of the individual healthcare professional and to train teams on working together cohesively.9–12 Although often used to train technical skill that require advanced hand-eye coordination, simulation training allows clinicians to practice cognitive skills and provides a means for teams to train for catastrophic events and crisis situations without placing the patient’s life at risk and provides a platform for meaningful discussion of the problems identified.11 In fact, studies have shown that nontechnical crisis management skills and team performance can improve with simulation-based education.13,14 These skills can be measured and tested, as demonstrated historically in the aviation and nuclear power industries and more recently in hospital operating rooms and ICUs.15,16 Furthermore, teamwork and communication problems affect the performance of ICU teams, and miscommunication is a key contributor to medical errors.17–19 Attempts to address these problems in other industries, such as the military and the aviation industry, have led to the use of crew resource management (CRM).17,20–24 In the healthcare industry, simulation serves not only as an adjunct to reading and didactic lecturing but also allows the students to work together as a team and experience rare and life-threatening clinical scenarios on realistic human mannequins in an actual clinical setting.9,10,15,18,19,21–23,25–27
The following is a description of a quality improvement project incorporating conventional didactic introduction of the sepsis treatment bundle to a typical multidisciplinary ICU team, followed by a multidisciplinary teams of clinicians through mannequin-based simulation (MDMS) of a patient in septic shock. The purpose was to reinforce education of the sepsis guidelines and explore whether knowledge of the treatment for SS/SS after the conventional introduction of the sepsis bundle would ensure acceptable compliance with these evidence-based guidelines during a simulated crisis scenario. It was also intended to assess whether the MDMS could help identify and correct errors.
The Connecticut Simulation Center at Hartford Hospital is a 2500 square foot complex that includes three Laerdal high-fidelity human simulators, a state of the art-simulated emergency room, ICU, operating room, and a classroom/debriefing room with live and recordable video feed from each simulation room. The simulated ICU room contains a fully stocked difficult-airway cart, cardiac arrest cart, monitors, IV pumps, and a computer (with computer order entry and intranet sites) to allow the staff to review protocols and research medications. There are three control stations where the operator and instructors run real-time scenarios behind a one-way mirror. The control rooms are fully equipped with 360° viewing and videotaping capability and computers running the Laerdal SimMan program (SimMan version 2.2, Laerdal Medical Corporation, Wappingers Falls, NY).
Hartford Hospital introduced VHAs Transformation of the Intensive Care Unit Sepsis Bundle and the Society of Critical Care Medicine’s Surviving Sepsis guidelines in all Surgical ICUs in January 2004, 14 months before this quality improvement exercise. These guidelines were promoted as best practices for the treatment of SS/SS through a multifaceted approach consisting of hospital-wide grand rounds, lectures, bedside teaching, and distribution of literature and pamphlets and were expected to be followed according to the published guidelines. Seventy-four participants (12 separate teams) partook in this quality improvement exercise from March 2005 to June 2006 and, thus, participated in this program with a waiver of consent granted by the institutional review board. Multidisciplinary teams were chosen from existing healthcare professionals from a single ICU and were considered to be representative of a typical team from that unit. Specific team composition did vary but consisted of four to eight individuals from the following professional groups: surgery critical care fellows, midlevel practitioners (either a physician assistant or a nurse practitioner), resident physicians (postgraduate year 1 or postgraduate year 2 surgical, anesthesia, or emergency medicine monthly ICU rotators), ICU nurses, and respiratory therapists (Table 1). All subjects were selected from one ICU, and there was the potential for all participants to have worked together before because the exercise was designed to facilitate better cohesion within the existing teams. Previous experience treating any form of sepsis was unknown. No other educational simulations for septic shock were performed in our center, and specifically, no other multidisciplinary simulation sessions of any kind had been performed to our knowledge or the knowledge of the Simulation Center or its directors. All team members were given a self-guided sepsis learning packet 1 week before the simulation session and told of its content and purpose. In addition, background teamwork materials were also distributed as part of the learning packet.28 Two of the critical care fellows (the team leaders) were allowed to undergo the MDMS a total of two times each and were tested both times. No further analysis was performed regarding multiple exposures, as these repeated exposures were not expected to worsen task completion or outcome but if anything result in improvement in task completion. All other subjects were exposed to the MDMS only once.
Before beginning the simulation session, each team member was asked to complete a 10-item multiple choice pretest (Appendix A); all items were related to the sepsis bundle, and 9 of the 10 items were specifically related to one or more defined tasks that would be expected during the simulation (see later for more detail on the tasks). Although, pre- and posttests were taken by all senior surgical critical care attending physicians (totally four) with 100% score achievement, tests were not validated by samples of novices.
Each team then underwent one 30- to 35-minute mannequin-based simulation of a patient in septic shock broken into three parts. Part 1 (first 15–20 minutes): the patient is admitted to the ICU hypoxic, in early acute respiratory distress syndrome (ARDS), intubated, and being manually ventilated. The patient is physiologically unstable with very low blood pressure and rapid heart rate, inadequate intravascular volume, fever, in florid shock from a severe infection (sepsis), and in need of resuscitation. Part 2 (8–10 minutes): now 1 hour postadmission with the patient’s vital signs stabilizing, but the patient’s blood pressure is still low, and the patient still demonstrates signs of inadequate circulation. Part 3 (8–10 minutes): now 6 hours postadmission with vital signs stabilizing but requiring high doses of vascular constricting drugs (vasopressors) to maintain an adequate blood pressure, but other markers of adequate circulation are still unacceptable.
The ICU teams were instructed to perform exactly as they would in real circumstances, including physical examinations, procedures, communication methods, and charting. All materials that would normally be needed for treating the patient were available in the room such as a mechanical ventilator, ICU medications, intravenous fluids, supplies, gowns, gloves, bedding, forms, and ICU flow sheets. X-rays and laboratory work were available in a time frame consistent with real life. A Nursing Facilitator was present in the room to troubleshoot any equipment issues or logistic questions but did not participate in any clinical information sharing.
Two observers, a critical care physician and a critical care nurse, observed the team remotely from the Simulation Center’s conference room, and in real time, they scored each group on a checklist of 12 evidence-based guidelines (Table 3) related to the Sepsis Bundle and the Surviving Sepsis Campaign Guidelines. The scenario was designed such that the mannequin would require the 12 necessary items on the checklist in Table 3 for optimal treatment for sepsis. For example, the patient is found with low blood pressure and septic requiring fluid resuscitation, intravenous vasopressors to increase and maintain an adequate blood pressure. In addition, the teams are aware of the general diagnosis of severe sepsis and must identify the source of the infection and begin treatment. In the final phase, despite adequate fluid resuscitation, the markers for adequate circulation have still not been met. Hemodynamic calculations are revealed indicating cardiogenic (heart) failure and anemia (low red blood cell count) prompting a blood transfusion and an agent to increase cardiac contractility. Each observer indicated whether the task was completed and what errors occurred. Ratings were done in real time, openly with no attempt at keeping them independent. Immediately after the scenario, the completed tasks were tallied and recorded. If the two raters did not agree on any individual task or did not come to the same conclusion, the data were recorded as indicating task completion in favor of the team without further discussion. In addition, each session was videotaped and reviewed by the two raters for educational value. No change in task completion was allowed; however, all videos were scrutinized looking specifically for errors due to communication, teamwork, and lack of situational awareness. Potential/probable causes of each error were also noted.
After the simulation, the same senior critical care physician led each group in a debriefing session. This physician was well versed in the current treatments for severe sepsis and septic shock and attended multiple conferences and lecture on the principles of CRM and self-educated through review of the literature but did not undergo any official training. The debriefing was begun by asking the team how well they thought they performed in managing this patient focusing on task completion and sepsis management. The debriefing continued with asking a set of questions that included but were not limited to those in Appendix B. The questions placed emphasis on specific CRM principles including team management, situational awareness, assertiveness, and closed loop communication. The expectation was to stimulate a discussion led by the debriefer and help the teams verbalize and confirm their perceptions of how well they thought they worked and communicated with each other and how well they managed the patient before review of the video. The video was then reviewed with the debriefer pausing at key moments and repeating the appropriate questions in Appendix B as well as other CRM principles and tasks related to the management of this patient with septic shock. The teams were allowed to identify their own mistakes and self-reflect on their previous responses to the questions in Appendix B and determine the possible causes of incomplete tasks. The team was expected to come up with some appropriate conclusions on their own to fill their own needs and knowledge gaps. The debriefer encouraged this self-discovery and investigation of causes of these errors and directed the discussion by pointing out mistakes that were not caught by the team, also identifying their successes and guiding the team through the concepts of CRM that could have been used to enhance their performance. Through this exercise, the debriefer also ensured that all intended CRM principles were discussed including leadership and team management skills (verbal and nonverbal communication styles, encouraging input from all team members, and acknowledging information received from individuals), recognition of adverse situations mainly through discussion of situational awareness, crosschecking, and communication processes such as practicing assertive statements and closed loop communication.21,27,29
The sepsis guidelines were reviewed and clarifications of all evidence-based measures were addressed including evidence for and against more controversial treatments contained within the bundle. A posttest of the same 10 items was administered. All quantitative analyses (specific tests described in Results section) were conducted using the statistical package, SPSS v 14.0.
Performance on the “pretest,” assessed 1 week after the didactic materials were received but before the simulation demonstrated an average score of 64.6% ± 16.6% with a range in scores from 30% to 100%. There was, however, a wide range of scores by question; there was evidence of a high level of individuals’ knowledge for 4 of the 10 questions (Table 2). Similarly, there was evidence of a higher level of groups’ knowledge, with at least one individual scoring 70% or higher in all groups, 80% or higher in all but one of the groups, and 90% or higher in eight of the 12 groups. For the specific questions, at least one person in each of the 12 groups had the correct knowledge for each of the 10 questions in 118 of the 120 instances.
During the simulation exercise, the success in completion of the sepsis guideline tasks was 60.4% (range = 41.7%–75%). Eight of the 12 groups (66.7%) failed to set the correct endpoints of resuscitation or obtain source control. Seven of the 12 (58.3%) failed to administer broad spectrum antibiotic coverage. Fifty percent of the groups failed to send cultures or sent them after administration of the antibiotics (Table 3).
Relationship of Knowledge and Task Completion
Several analyses were done to explore the relationship between knowledge as expressed on the pretest and task competence. The most basic measure was the correlation of mean pretest scores for each of the 12 groups with the proportion of the 12 tasks that each group completed. That analysis demonstrated a positive relationship between overall knowledge of the sepsis guidelines and proportion of task completed with, a statistically significant Pearson Product Moment correlation of 0.73 (P < 0.007).
A more detailed analysis, however, suggests a different conclusion. Ten of the 12 tasks related specifically to nine of the 10 questions (two of the questions relate to more than one task and two of the tasks are related to two or three of the questions). To obtain an overall measure of task/question relationship, the performance on each task was paired with the proportion of the group responding correctly to the related question or questions. Over the 10 tasks and 12 groups, this created 120 task/question combinations. A point biserial correlation was calculated relating the dichotomous outcome of task completion to the continuous proportion of correct responses. This analysis resulted in an extremely low correlation of 0.058 (n.s.) between task completion and specific related knowledge of group members.
The project was successful in getting the participants to talk about their experiences during the simulation, realizing mistakes that were made only after review of the video, especially the issues they felt led to incomplete tasks. Among those, frequently encountered were: lack of leadership and team management skills, assertiveness or limited empowerment to question perceived wrong orders, disregard or possibly failure to hear suggestions of correct treatment made by others, failure to verify medications and orders, and lack of situational awareness. Situational awareness refers to the subjects’ recognition of their surroundings and what is occurring in real time. It involves constantly reassessing the situation and anticipating any likely events and emerged as an important topic in all discussions.
The posttest after both the simulation exercise and the debriefing showed a significant increase in mean individual participant scores up to a mean of 83% from the pretest of 64.6% [paired t test t (72) = 10.258; P < 0.001].
Interestingly, despite implementation of the surviving sepsis campaign guidelines, using VHAs sepsis screening and treatment tool, providing hospital-wide lectures, bedside teaching, and distribution of the evidence-based sepsis guidelines learning packets to each individual in preparation for the simulation, overall pretest score means were still modest (64.6%) as were completion of the sepsis guidelines during simulation (60.4%). This suggests that our current implementation method may not be sufficient in solidifying adequate knowledge of the sepsis guidelines to our healthcare team members and facilitate the appropriate application of this knowledge. Pretest knowledge scores did indicate that all teams had at least one member with an adequate knowledge base of the sepsis treatment guidelines. Thus, we suggest that adequate “team” knowledge was available in each group to complete all tasks, when examining correct responses to each question from each team member before the simulation session. Unfortunately, knowledgeable leadership or collective knowledge of specific tasks did not manifest in effective completion of the sepsis guidelines in our MDMS with even the top scoring groups failing to complete 25% of the guidelines. In fact, knowledge of each specific question correlated poorly with completion of the corresponding task. Hierarchy was reviewed with no significant evidence that having the most experienced and knowledgeable practitioner in the group (fellow or midlevel practitioner) related to improved task completion.
The debriefing sessions revealed inadequate teamwork and communication as one possible explanation for the ineffective completion of the sepsis bundle. Subjective evidence only was visible on review of the videos but did result in valuable discussion with the team. For example, before video review, all groups seemed to have some false perceptions that specific tasks were completed when in actuality they were not. The teams became fully aware of this point on review of the videos resulting in crucial discussion on why this occurred. Communication deficiencies were evident as the teams were allowed to come to their own conclusions, and realization of these errors was central to the discussion of the specific CRM principles. Exercises for improvement in these areas were reviewed with the entire team and was dependent on the specific types of errors that occurred within each group. For example, Larry, the team leader says “we should give some epinephrine,” which is ignored, not heard, or perhaps not taken seriously by the nurse due to the passive nature of the statement. We review the event and suggest that Larry say, “Betty, please give one milligram of epinephrine now.” Betty would verbally acknowledge the order then ideally respond with “one milligram of epinephrine given.” This one example also allowed us to discuss the importance of multiple CRM principles such as closed loop communication, assertiveness, and team management skills, such as eye contact, nonverbal communication, and the ability to engage all team members. The video was paused multiple times in all groups to define and discuss and repeat these specific CRM skills as they occurred. Situational awareness or lack thereof was persistent throughout all simulations and was elicited by pausing the video and asking the group to answer specific questions such as “What is going on right now? How much intravenous fluid do you think has been given? What dose of vasopressors is the patient on?” Identifying what had been ordered and what had been given or accomplished often revealed multiple orders given yet few tasks actually completed or acknowledged. Realization of these discrepancies was essential in helping our groups understand the behaviors of a high-functioning team. Review of the video helped emphasize that teams often overestimated their ability to work and communicate together during a crisis and ultimately successfully complete the sepsis bundle. Although some knowledge-based errors occurred, discussions on teamwork and communication topics were at the forefront and became the major component of the debriefing sessions seeming to have at least the potential to play a significant role in the treatment of critically ill patients or any stressful environment or crisis. In fact, CRM education has now become part of the discussion during mandatory debriefing sessions after real-life cardiac arrests in our surgical ICU.
There were multiple limitations to this project. This article only reports data extracted from a quality improvement project to improve sepsis awareness and treatment while promoting good teamwork and communication skills. Videos were intended for educational value only, and test questions were designed to reflect knowledge of the evidence-based sepsis guidelines and were reviewed and approved by attending physicians of the department of surgery/critical care. These were intended as teaching tools and not as a research instruments. For this reason, there was no control group, control period, or sham intervention to examine the effects on test scores without simulation, and test scores were not validated looking only at MDMS generally; there was no test of simulation parameters such as number of simulations or specific scenarios. The simulation scenario itself does recreate the realistic stress of an ICU crisis but still remains artificial and may not accurately produce similar team or clinical responses. Unfortunately, the true stress of a realistic ICU crisis can only be approached and not reproduced with current simulation technology. Team members may feel the effects of being observed and videotaped, which could have affected performance. However, this “Hawthorne” effect usually increases levels of performance.30–32 Additionally, some of the tasks might have been difficult to assess with simulation, and there was no attempt to test for interrater reliability among the reviewers. There was, however, the debriefing session that provided participants a judgment-free arena for clarification. Finally, the correlational analysis of the 120 task-question combinations successfully focused on specific task/knowledge relationships but may stretch the bounds of the assumption of independence for the cross tabulation’s χ2 statistic.
We suggest that knowledge alone is insufficient to orchestrate and resolve a successful crisis situation. This may be due to many unknown factors. We suggest in this simulation model that chaos ensues creating disorganization, ineffective leadership, unclear communication style, and results in poor compliance with current management guidelines. In addition, healthcare providers may overestimate their teamwork and communication abilities and are seldom instructed on these CRM principles relying on chance clinical encounters alone to gain experience.33 These types of communication and teamwork errors can be difficult to identify and correct during the events of a real crisis. MDMS provides an avenue for teams and individuals to identify and self-reflect on clinical and communication errors and when combined with reenforcement of sound team behavior may facilitate correction of these deficits. It is a promising structure in which to teach communication and teamwork skills through CRM or other methods and ultimately improve compliance with the introduction of new clinical treatment bundles. Further studies are needed to examine the nature and impact of improving these skills in a simulated or clinical environment. Future research will address performance without the effect of simulation, directly comparing the effects of teamwork and communication with and without CRM education with repeated simulation testing. In conclusion, MDMS seems to be useful in identifying clinical errors unrelated to knowledge base and suggests a need for additional investigation of potential causes, including teamwork and communication.
1. Pronovost PJ, Berenholtz SM. Improving Sepsis Care in the Intensive Care Unit: An Evidence-Based Approach
. Irving, TX: VHA, Inc.; 2004; VHA Research Series, 2004.
2. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644–1655.
3. Dellinger RP, Carlet JM, Masur H, et al. Surviving Sepsis Campaign Management Guidelines Committee. Surviving Sepsis Campaign guidelines for management of severe sepsis and septic shock. Crit Care Med
4. Halamek LP, Kaegi DM, Gaba DM, et al. Time for a new paradigm in pediatric medical education: teaching neonatal resuscitation in a simulated delivery room environment. Pediatrics
5. Ziv A, Erez D, Munz Y, et al. The Israel Center for Medical Simulation: a paradigm for cultural change in medical education. Acad Med
6. McFetrich J. A structured literature review on the use of high fidelity patient simulators for teaching in emergency medicine. Emerg Med J
7. Kneebone R, Nestel D, Wetzel C, et al. The human face of simulation: patient-focused simulation training. Acad Med
8. Bradley P. The history of simulation in medical education and possible future directions. Med Educ
9. DeVita MA, Schaefer J, Lutz J, Dongilli T, Wang H. Improving medical crisis team performance. Crit Care Med
10. Rooke GA, Carline J, et al. Evaluation of anesthesia residents using mannequin-based simulation: a multiinstitutional study. Anesthesiology
11. Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA
12. Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg
13. Yee B, Naik VN, Joo HS, et al. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesth
14. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. J Trauma
15. Sexton HJB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: cross sectional surveys. BMJ
16. Helmreich RL, Schaefer HG. Team performance in the operating room. In: Bogner M, ed. Human Error in Medicine
. Hillsdale, NJ: Laurence Erlbaum; 1994.
17. Kohn L, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System
. 1st ed. Washington, DC: National Academies Press; 2000:79.
18. Sherwood G, Thomas E, Bennett DS, Lewis P. A teamwork model to promote patient safety in critical care. Crit Care Nurs Clin North Am
19. Leipzig RM, Hyer K, Ek K, et al. Attitudes toward working on interdisciplinary healthcare teams: a comparison by discipline. J AM Geriatr Soc
20. Helmreich RL, Merritt AC. Culture at Work in Aviation and medicine: National, Organization, and Professional Influences
. Aldershot, UK: Avebury Aviation, Ashgate Publishing Limited; 1998.
21. Salas E, Rhodenizer L, Bower CA. The design and delivery of crew resource management training: exploiting available resources. Hum Factors
22. Risser DT, Rice MM, Salisbury ML, Simon R, Jay GD, Berns SD. The potential for improved teamwork to reduce medical errors in the emergency department. The MedTeams Research Consortium. Ann Emerg Med
23. Musson DM, Helmreich RL. Team training and resource management in healthcare: current issues and future directions. Harv Health Policy Rev
24. Grogan EL, Stiles RA, France DJ, et al. The impact of aviation-based teamwork training on the attitudes of health-care professionals. J Am Coll Surg
25. Kaissi A, Johnson T, Kirschbaum MS. Measuring teamwork and patient safety attitudes of high-risk areas. Nurs Econ
26. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTEams project. Health Serv Res
27. Moorthy K, Munz Y, Forrest D, et al. Surgical crisis management skills training and assessment: a simulated-based approach to enhancing operating room performance. Ann Surg
28. Kaissi A, Johnson T, Kirschbaum MS. Measuring teamwork and patient safety attitudes of high-risk areas. Nurs Econ
29. France DJ, Stiles R, Gaffney FA, et al. Home study program: crew resource management training—clinicians’ reactions and attitudes. AORN J
30. Buchanan D, Huczynski A. Organizational Behavior
. 3rd ed. Vol. 7. London: Prentice Hall; 1997.
31. Lied TR, Kazandjian VA. A Hawthorne strategy: implications for performance measurement and improvement. Clin Perform Qual Health Care
32. De Amici D, Klersy C, Ramajoli F, Brustia L, Politi P. Impact of the Hawthorne effect in a longitudinal clinical study: the case of anesthesia. Control Clin Trials
33. Moorthy K, Munz Y, Adams S, Pandey V, Darzi A. A human factors analysis of technical and team skills among surgical trainees during procedural simulation in a simulated operating theatre. Ann Surg