A fundamental goal unites healthcare organizations: to consistently provide safe, effective, and efficient care. Healthcare organizations fall short of this goal when medical errors and adverse events occur.1 Historically, individual deficits in knowledge or skill were thought to be a primary source of these errors.1 Mitigation strategies, defined as interventions to prevent errors from recurring,1 traditionally targeted individuals. As one possible mitigation strategy, simulation and debriefing plays a role in uncovering individual deficits, understanding individual knowledge or skill gaps, and providing an opportunity for individual improvement.2–14 Despite simulation-based training, similar errors and adverse events have often been observed involving different individuals in different clinical environments,12,14–17 bringing into question the focus on the individual as a primary solution to the elimination of errors and quality improvement.
Systems engineering, the science of designing and managing complex systems throughout their life cycle,18 has a different premise—that errors and adverse events are often due not to individual deficits but to gaps in the elements of the complex healthcare system that surrounds an individual. These elements include people (ie, all members of the healthcare team), hardware, software, facilities, policies, documents, processes, and their interactions.18,19 The Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model provides a conceptual framework for the components and interrelationships of the healthcare system.20 The SEIPS 2.0 model describes a system as having the following three parts: the work system, processes, and outcomes. The work system includes technology, tools, tasks, team members, physical (internal) environment, external environment (ie, macro-level factors outside an organization), organizational structure, and culture. Processes include care delivery and communication. Outcomes include patient, employee, and organizational safety and quality metrics.20 The SEIPS 2.0 model is an expanded version of the original SEIPS model,21 incorporating human factors concepts of configuration (ie, dynamic, hierarchical, interactive properties of systems), engagement (ie, differentiating work activities by who is actively engaged in performing them), and adaptation (ie, how dynamic systems evolve in planned and unplanned ways).20 The model is not linear, with processes and outcomes feeding back to affect the work system.20
With the SEIPS 2.0 model in mind, systems-focused simulation and systems-focused debriefing (SFD), recreates complex systems of care and seeks to understand system issues, gaps in different components of the healthcare system.19 Systems-focused simulation includes elements designed to test the system, such as conducting simulation in situ (the actual place of real patient care), using real equipment (not training equipment), and fully implementing processes (eg, pushing the Code Blue button and waiting for the Code Blue team to receive notification via their pagers, then respond, as opposed to having them waiting outside the simulation area and available as soon as verbally requested).14,15,19,22 Systems-focused simulation combines simulation methodology with elements of systems engineering and human factors/ergonomics, the science of understanding the interactions of humans and their environment.23 Although deeper discussions of systems engineering, human factors, ergonomics, and conducting systems-focused simulations are beyond the scope of this article, reviews by Stone19 and Auerbach22 provide additional resources.
The goal of a systems-focused simulation and SFD is to identify systems gaps, including latent safety threats (LSTs). Latent safety threats have been defined as systems-based threats that could predispose to medical errors.17 Examples include faulty equipment or a lack of sufficient resources.12,14–17 Once gaps and LSTs are identified, mitigation strategies can be explored, evaluated and implemented. Ultimately, the goal is to improve patient safety and the overall quality of care by reducing accidental or preventable injuries.24 Current debriefing frameworks fall short in considering elements of the SEIPS 2.0 model,20 and as a result, LSTs that occur may be left unaddressed. Systems-focused debriefing requires a new conceptual framework for debriefing to address this gap.
The Promoting Excellence and Reflective Learning in Simulation (PEARLS) Debriefing Framework and Script11,25 represents a blended debriefing structure that includes learner self-assessment (plus/delta), directive feedback, and/or focused facilitation. The PEARLS assists simulation educators in tailoring educational strategies for different learners, learning objective(s), timeframes, and simulation settings.11 To maximize the identification of systems issues in SFD, we modified the original PEARLS framework and script. A traditional simulation debriefing identifies individual learners' knowledge, skill, and communication gaps,5–10,26–29 whereas an SFD explores the work system (tools, tasks, technology, organization, people, environment) and processes that can lead to unsafe and unintended outcomes.25
The PEARLS for Systems Integration (PSI) framework provides a debriefing framework and a script for SFD. The term systems integration refers to the process of bringing together elements of a system to ensure they function as a coordinated whole.30 Systems-focused debriefing for systems integration uses debriefing strategies to identify errors or safety threats that reflect the risks and hazards providers and patients face in context of poorly designed systems.22,31 In PSI, the leader of SFD is a facilitator, not a simulation educator. Her or his primary goal is to uncover and explore issues, not to educate participants, serve as a content expert, identify solutions, or become an operational owner of potential solutions. A facilitator should ideally have a foundational understanding of healthcare systems and improvement science through additional coursework (eg, Institute of Healthcare Improvement's online basic certificate in quality and safety), hands-on experiences and/or collaborations with systems engineers, human factors experts, and other content experts. A fundamental understanding of system components and how to distinguish systems issues from individual or team issues is important. In PSI, simulation and debriefing involve participants, not learners. As a participant, each person is a representative of “their professional role” within the hospital. His or her goal is to consider the system components, which support or hinder anyone who performs their role. In an SFD, the facilitator focuses on the identification of system issues and guiding participants through the predetermined objectives. In a learner-focused debrief, the focus of the facilitator is on closing performance gaps and guiding reflection of positive performance.11 Although the two roles are not mutually exclusive, the ability to conduct an effective SFD requires additional understanding, which we highlight in the PSI framework.
The PSI framework includes (a) participant assessment of predetermined objectives, (b) facilitated discussion focused on systems issues identification, and (c) providing information, background, or limitations in the form of directive feedback (Fig. 1). In more than 25 combined years of experience in systems-focused simulations and SFD extended across multiple patient care departments and settings, we have developed strategies to focus debriefing on uncovering system issues and LSTs. This blended approach, including participant system assessment (eg, use of plus/delta to identify what went well and what could be improved), directive feedback (eg, explain rationale for a process design), and focused facilitation (eg, debriefing method used to uncover underlying system issues), contributes to efficient and effective SFDs.
;)
FIGURE 1: Debriefing flowchart for PSI. This flow chart shows all key phases of the PSI framework.
The purpose of this article is to (a) introduce the modified PSI framework and an accompanying debriefing script and (b) describe a blended approach to SFD. This is the first article in an intended series, describing systems-focused simulation and SFD. Future topics include further details on the creating systems-focused simulations (the prework phase), the follow-up work occurring after an SFD including the strategies required to address issues identified during an SFD, and faculty development.
PEARLS FOR SYSTEMS INTEGRATION FRAMEWORK
The PSI framework adapts the PEARLS framework11 and outlines the following five distinct phases: prework, description, reactions (optional), analysis, and summary. These phases enable teams to identify systems issues and, as time permits, generate possible solutions (Fig. 1). Table 1 provides a succinct summary of the comparisons between the original PEARLS framework and the modified PSI.
TABLE 1: Table Comparing Original PEARLS Framework and the PSI Framework
Prework: Identification of Stakeholder Objectives
The prework phase is foundational for preparing and facilitating a system-focused simulation and SFD. Meeting with stakeholders to develop predetermined objectives occurs early on, before developing the scenarios or arranging logistics (eg, recruiting participants). Stakeholder objectives provide a structure for SFD. Stakeholders may include clinical (eg, doctors, nurses, allied health, etc.), administrative (eg, unit or departmental leadership), nonhealthcare professionals (eg, patients, families), and support roles (eg, supply chain, environmental services, unit clerks) that make existing and new environments, processes, or systems functional, by providing critical services or supplies. Stakeholders are the content experts of the new or revised environment or process and can help identify areas of concern and/or significant change from which the objectives of the systems-focused simulation are developed.
In the prework phase, stakeholders identify and prioritize potential high-impact and high-risk changes. High-impact changes are those staff will face frequently (eg, routine patient clinic visit for an electroencephalogram). High-risk changes have the potential for patient harm (eg, a patient having hypoxic seizure, needing bag-mask ventilation) or staff harm. These predetermined, prioritized stakeholder objectives will serve to guide the discussion of an SFD. This differs from traditional simulation debriefing, with objectives generated by the frontline teams involved in the simulation, and a debriefing focused on closing gaps in individual knowledge or skills.26,32,33Figure 2 demonstrates examples of stakeholder predetermined high-risk and high-impact objectives for the evaluation of a new space: a neurology clinic.
FIGURE 2: PEARLS for system integration debriefing framework example: evaluation of a new neurology clinic. This figure demonstrates an example of facilitator debriefing through each phase of the PSI framework.
The number of the stakeholder groups (and subsequent participants who will participate in the debrief) varies; that is, fewer stakeholders for a simple process change on one unit, greater numbers for a process involving multiple professions and units. Similarly, the number of participants in the simulation should reflect the variety of groups affected by the proposed system. To facilitate robust systems evaluation, inclusion of all groups that may be impacted by potential changes in the system could result in larger group debriefings. The PSI framework takes this issue into consideration. A detailed description of all the components of the prework phase for a systems-focused simulation is beyond the scope of this article but will be addressed in a future publication.
Description Phase
Systems-focused debriefing begins with a description phase. Systems-focused debriefing assumes that a prebrief occurred before the simulation, outlining the differences between a systems-focused and a learner-focused simulation and its goals. The purpose of the description phase is to reinforce a shared mental model between facilitators and participants for the SFD. The difference from the original PEARLS model11 is that it contains two components: (a) a statement of purpose, to identify systems issues and LSTs that threaten the quality of care and patient safety and (b) a brief summary of key simulation events. For teams who have only been exposed to traditional simulation and debriefing, reiterating the differences in debriefing approach helps focus the discussion. The facilitator may ask whether there are questions related to the purpose of an SFD (eg, a group that is new to SFD has experience with learner-focused simulations). Additional comments may include, “This debriefing will look different than learner-focused debriefing. We have predetermined objectives that we need your feedback on, and want to hear what worked well and what could be improved.” The facilitator may provide additional methods for participants to provide system-focused feedback, such as follow-up e-mails or connection time after the debriefing.
Typically, in PSI, the facilitator summarizes the purpose, a difference from the original PEARLS framework.11 In PSI, the facilitator ensures that this summary is complete, succinct, and in alignment with stakeholder predetermined objectives. The risks of a participant giving the summary are that it may not be brief, it may include issues beyond predetermined objectives, or given the variability in participants experience with identification of system issues, it may derail the team. In Figure 2, there is a detailed description of the facilitator script for our neurology clinic example.
Reactions Phase (Optional)
In PSI, the reactions phase is optional and follows the description phase. This differs from the original PEARLS framework.11 Traditionally, a reactions phase explores the emotional reactions of individual learners and is used to identify the learner agenda (ie, what the learners hope to discuss and explore) in the debriefing. In PSI, the SFD agenda is established primarily by the predetermined stakeholder objectives. The participant agenda, if not in alignment with predetermined objectives, is secondary to ensure allotted time focuses on the highest priority topics. If a reaction does not align with predetermined objectives, the facilitator's goal is to acknowledge, determine whether there is a systems issue, and then redirect the SFD. A facilitator can follow up after all predetermined objectives have been debriefed if time allows, or alternatively, after the debrief has ended. An example: “Having the transport route door locked was frustrating (acknowledge). Can you tell me more about that (focused facilitation)? Let's capture that and follow up after our SFD (redirect). Given our limited time, we want to ensure we discuss some other critical issues.”
Facilitators may consider including a reactions phase. Emotionally charged simulations may benefit from a brief discussion of reactions, enabling participants to move on to identifying LSTs and systems issues.
Considerations for including a reactions phase involve available time, group size, and number of stakeholder predetermined objectives. In our experience, reaction phases are challenging for groups more than 30 or when debriefing time is limited. A lengthy reactions phase can consume valuable time, may delay discussion of predetermined objectives, and require managing topics of lower priority.
Figure 2 details a reactions phase for the neurology clinic example. This example highlights a reaction in alignment with a predetermined objective, showing how the facilitator transitions into the analysis phase.
Analysis Phase
The PSI analysis phase methodically identifies and explores systems issues, focusing discussion on predetermined objectives using the framework provided by SEIPS 2.0.20 This differs from the original PEARLS framework, which emphasizes both predetermined objectives and learner-defined objectives.11,34 The PSI retains blended debriefing, using plus/delta to guide assessment of predetermined objectives, focused facilitation to elucidate underlying system causes, and directive feedback to provide rationale as needed. Each method of debriefing concentrates the participant's efforts on identifying systems issues.
Plus/delta, a method of debriefing focusing on what went well (the “plus”) and what did not go as well (the “delta”), promotes expeditious review of predetermined objectives, specifically for patient safety and systems issue.5,35 The “plus” identifies those systems components that are working well, whereas the “delta” identifies systems issues that need modification. The facilitator may address one predetermined objective at a time or ask participants to generate a comprehensive list. Figure 2 provides two possibilities to open the analysis phase of an SFD.
Secondary debriefing strategies include focused facilitation and directive feedback. At any point, focused facilitation (eg, advocacy inquiry8,9,28) can uncover more details. Inquiry questions should be phrased to uncover underlying systems issues, including participant perspectives. Follow-up questions help redirect discussion toward relevant topics/issues. See Figure 2 for an example of focused facilitation and the use of advocacy inquiry for an equipment-related issue in the neurology clinic. Use directive feedback to explain the rationale for how a task, environment, or process was designed or limitations that may impact solution generation.36,37 In Figure 2, directive feedback explains the rationale for placement of the new clinic alarms.
Discussion time for each objective varies. At a minimum, explore each predetermined objective, identifying systems issues. If time allows, facilitators may identify and explore participant objectives related to systems issues. With even more time, participants may generate potential solutions for quick implementation and identify operational owners or follow-up groups.
Time management during SFD is critical. A general recommendation is to limit an SFD debrief to 60 to 90 minutes for a 30- to 45-minute systems-focused simulation. It is difficult for participants to remain engaged for extensive periods; however, you need to hear from the varied participants because different roles may have different observations and experiences. Consider distributing SFD into smaller time intervals, with more focused scenarios, to achieve the overall goals.
During the analysis phase, participants often prematurely jump from issue identification to problem-solving. The primary goal of the debrief is for system issue identification. Risks of jumping to problem-solving are that the issues are not completely understood that could result in inadequate or weak interventions. Capturing some potential solutions can serve as a good starting point for follow-up work and help with overall time management. Figure 3 outlines an example of evaluating a new communication/handover tool. In the analysis phase, participants identify the lack of a shared mental model. Because the participants had more time, they suggested potential technological solutions (eg, new phones, new alarms) or role changes (eg, let's have the unit clerk make an overhead announcement).
FIGURE 3: PEARLS for system integration debriefing framework example: evaluation of a new communication/handover tool and patient transport process between the emergency department and operating room. This figure demonstrates an example of facilitator debriefing through each phase of the PSI framework.
Sometimes, participants identify an unanticipated systems issue. In Figure 2, the summary includes a participant-generated issue: lack of standard work for those in the team who respond for an emergency. If time is limited, capture the issue for future follow-up. With more time, implement focused facilitation (eg, “Tell me more about who responds to an emergency”) to get at the underlying cause (eg, no standard work for emergency roles), then redirect or close the analysis phase (eg, “Let's follow-up on that after the SFD” or “Any other issues we should capture for later follow-up?”). Closing the analysis phase signals the SFD is moving into the summary phase.
Summary Phase
In the PSI summary phase, facilitators highlight key systems issues identified and suggested improvements, if any. This is in contrast to learner-focused debriefings in the original PEARLS framework.11 In learner-focused debriefs, the summary phase usually provides opportunities for learners to state take home points.11 In PSI, flip charts or whiteboards can be used to capture identified systems issues and LSTs, as well as possible improvement ideas, throughout the debriefing. These immediately visible and available summaries, reviewed by the facilitator in the summary phase, cross-checked by the participants, validate participants' contributions and provide an accurate and transparent summary. This ensures a shared mental model of the debriefing outcomes and that no key findings were missed.
Organizations may vary in how they capture, disseminate, and implement outcomes from SFD. We recommend sharing a summary (ie, report, e-mail, follow-up meeting) with stakeholders that includes debriefing findings and next steps. Identifying operational owners, developing timelines, and establishing working groups are often required to facilitate change. In addition, we recommend the stakeholder group communicates ongoing improvement efforts with participants and their representative groups to promote engagement and organizational change. The extent of the post follow-up work is dependent on the scope, scale, and findings of the SFD. A detailed discussion of the follow-up work after a systems-focused simulation is beyond the scope of this article but will be addressed in a future publication.
DISCUSSION
Systems-focused simulation and SFD, using PSI, takes learning beyond the individual and team to the next level: the healthcare system. By blending system-focused debriefing strategies with the PEARLS,11 and building on the SEIPs 2.0 concepts of system component interrelationships,20 our PSI framework can help facilitators lead effective and efficient SFD to identify safety issues and opportunities for process and system improvements.
A deliberate systems-focused simulation, followed by an SFD using PSI, can identify institutional deficiencies, inefficiencies, and unplanned adaptations affecting safety and quality.13,38,39 The PSI provides an opportunity to implement an interprofessional and interdepartmental approach to creating solutions. For participants in an SFD, identification of systems issues and exploration of solutions across professional groups can be enlightening.40 For leadership, identification of recurring issues, across different environments and processes, can be informative at the organizational level. Without a robust infrastructure, those learnings cannot be transferred across an organization.41 As simulation programs, we can use systems-focused simulation and SFD to identify systems issues; however, the key stakeholders are the ones that ultimately have the opportunity to develop lasting solutions.42 Without stakeholder engagement, sponsorship and commitment, quality improvement will be limited to weaker solutions, which are lower on the hierarchy of intervention effectiveness.42,43 After an SFD, it is imperative that stakeholder groups disseminate findings, assign operational owners, develop recommendations, and communicate their efforts and manage change; this leads to organizational learning from an SFD. The goal of this article was to introduce a structured debriefing model for an SFD. More detailed information on the follow-up work after a systems-focused simulation and SFD is planned for a future publication.
We see limitations and potential hurdles moving forward. The mastery of debriefing using the PSI framework takes time and practice. Most facilitators' experience is with learner-focused debriefing and the original PEARLS framework.11 Distinguishing the similarities and differences between traditional PEARLS11 and PSI is the first step in helping most simulation facilitators develop the additional skills necessary for SFD. Faculty development opportunities can help facilitators develop these new skills. In addition, collaboration with experts outside of simulation programs, such as human factors specialists, system engineers, quality and patient safety experts, and process improvement specialists, can provide opportunity to understand SEIPS 2.020 and the fundamentals of quality and system improvement. This is particularly important because formal faculty development programs for SFD are lacking. Figures 4A and 4B are scripts that can guide faculty development. Co-facilitation of SFD allows simulation faculty to provide colleagues with immediate feedback, while they personalize their use of PSI scripts. Figures 2 and 3 provide scripted examples for two types of systems issues: a new process and a new environment.
FIGURE 4: A, PEARLS for system integration debriefing tool – front side. This figure provides the objective, task, and sample phrases for each phase of the PSI framework. B, PEARLS for system integration debriefing tool – back side. This figure provides a more detailed view of sample facilitator phrases in the analysis phase, as well as system issue categories for capturing debrief feedback.
Systems-focused debriefing is most successful when facilitators have foundational knowledge in learner-focused debriefing, principles of systems-based approach, human factors, patient safety, and process improvement methodologies. The organization of this extensive body of knowledge, with a clear trajectory for training and evaluating system-focused facilitators, must be implemented into future faculty development efforts.
Just as facilitators are more likely to have experience with learner-focused simulations, so too are participants. Participant redirection requires several strategies: reminding them frequently of the purpose of SFD, refocusing when it gets off track, and being explicit about predetermined objectives. Using advanced proactive and reactive debriefing strategies when needed help ensure participants feel heard and supported to share their point of view.44
Another limitation of the PSI involves the reactions phase. Inexperienced or large groups may introduce emotional reactions or raise issues not in alignment with predetermined objectives, nonsignificant safety issues, or systems issues that are out of scope. Issues not amenable to change (ie, fixed room design) or historic interpersonal challenges can threaten to derail an SFD. To manage this dynamic, facilitators should aim to acknowledge the issue, identify its value, and then redirect the focus to predetermined stakeholder objectives.
Because healthcare systems strive to improve patient safety and quality of care, we anticipate system-focused simulations and SFD will play a greater role in a structured approach to issue identification, interprofessional problem-solving, and transferring knowledge. Our simulation teams have been successful in SFD and PSI to proactively design, develop, and test new processes (ie, protocols, patient care pathways) and new environments.19,22 Systems-focused debriefing and PSI may also be combined with traditional quality and process improvement tools to assist with identification of system issues.45 Because improvement hinges on change, establishing infrastructure that captures issues, assigns stakeholders responsibility, and provides the resources for problem-solving and implementation is crucial to realizing systems-wide improvement.
CONCLUSIONS
The PSI builds on established, effective debriefing practices, adding a new conceptual framework to help facilitators stretch beyond individual healthcare providers and teams, to better understand and explore gaps and improvement opportunities in healthcare systems. It provides a new framework and script to help those developing SFD skills. Future directions include more empiric study of the PSI framework and script and more extensive study of a faculty development program.
ACKNOWLEDGMENTS
The authors thank Simon Huang for his work in the graphic design of the PSI framework.
REFERENCES
1. Bates DW, Singh H. Two decades since To Err Is Human: an assessment of progress and emerging priorities in patient safety.
Health Aff (Millwood) 2018;37(11):1736–1743.
2. Kolbe M, Grande B, Spahn DR. Briefing and debriefing during simulation-based training and beyond: content, structure, attitude and setting.
Best Pract Res Clin Anaesthesiol 2015;29(1):87–96.
3. Kolbe M, Marty A, Seelandt J, Grande B. How to debrief teamwork interactions: using circular questions to explore and change team interaction patterns.
Adv Simul 2016;1:29.
4. Kolbe M, Weiss M, Grote G, Knauth A, Dambach M, Spahn DR, Grande B. TeamGAINS: a tool for structured debriefings for simulation-based team trainings.
BMJ Qual Saf 2013;7:541–553.
5. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More than one way to debrief: a critical review of healthcare simulation debriefing methods.
Simul Healthc 2016;11(3):209–217.
6. Cheng A, Eppich W, Grant V, Sherbino J, Zendejas B, Cook DA. Debriefing for technology-enhanced simulation: a systematic review and meta-analysis.
Med Educ 2014;48(7):657–666.
7. Brett-Fleegler M, Rudolph J, Eppich W, et al. Debriefing assessment for simulation in healthcare: development and psychometric properties.
Simul Healthc 2012;7(5):288–294.
8. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: combining rigorous feedback with genuine inquiry.
Anesthesiol Clin 2007;25(2):361–376.
9. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education.
Acad Emerg Med 2008;15(11):1010–1016.
10. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning.
Simul Healthc 2007;2(2):115.
11. Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing.
Simul Healthc 2015;10(2):106–115.
12. Auerbach M, Kessler DO, Patterson M. The use of in situ simulation to detect latent safety threats in paediatrics: a cross-sectional survey.
BMJ Simul Technol Enhanced Learn 2015;1:77–82.
13. Gaba DM. The future vision of simulation in health care.
Qual Saf Health Care 2004;13(Suppl 1):i2–i10.
14. Adler MD, Mobley BL, Eppich WJ, Lappe M, Green M, Mangold K. Use of simulation to test systems and prepare staff for a new hospital transition.
J Patient Saf 2018;14:143–147.
15. Ventre KM, Barry JS, Davis D, et al. Using in situ simulation to evaluate operational readiness of a children's hospital-based obstetrics unit.
Simul Healthc 2014;9(2):102–111.
16. Geis GL, Pio B, Pendergrass TL, Moyer MR, Patterson MD. Simulation to assess the safety of new healthcare teams and new facilities.
Simul Healthc 2011;6(3):125.
17. Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situ simulation detection of safety threats and teamwork training in a high risk emergency department.
BMJ Qual Saf 2013;22:468–477.
18. International Council on Systems Engineering. INCOSE Web site. Available at:
https://www.incose.org/about-systems-engineering/about-systems-engineering. Accessed February 14, 2019.
19. Stone KP, Huang L, Reid JR, Deutsch ES. Systems integration, human factors, and simulation. In: Comprehensive Healthcare Simulation: Pediatrics. Comprehensive Healthcare Simulation. Springer, Cham; 2016:67–75.
20. Holden RJ, Carayon P, Gurses AP, et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients.
Ergonomics 2013;56(11):1669–1686.
21. Carayon P, Schoofs Hundt A, Karsh BT, et al. Work system design for patient safety: the SEIPS model.
Qual Saf Health Care 2006;15(Suppl 1):i50–i58.
22. Auerbach M, Stone KP, Patterson MD. The role of simulation in improving patient safety. In: Comprehensive Healthcare Simulation: Pediatrics. Comprehensive Healthcare Simulation. Springer, Cham; 2016:55–65.
23. International Ergonomics Association. IEA Web site. Available at:
https://www.iea.cc/whats/index.html. Accessed February 14, 2019.
24. AHRQ. Patient Safety Network. Available at:
https://www.psnet.ahrq.gov/glossary/patientsafety. Accessed February 14, 2019.
25. Bajaj K, Meguerdichian M, Thoma B, Huang S, Eppich W, Cheng A. The PEARLS Healthcare Debriefing Tool.
Acad Med 2018;93(2):336.
26. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment.
Simul Healthc 2006;1(1):49–55.
27. Cheng A, Grant V, Robinson T, et al. The Promoting Excellence and Reflective Learning in Simulation (PEARLS) approach to health care debriefing: a faculty development guide.
Clin Simul Nurs 2016;12(10):419–428.
28. Cheng A, Morse KJ, Rudolph J, Arab AA, Runnacles J, Eppich W. Learner-centered debriefing for health care simulation education: lessons for faculty development.
Simul Healthc 2016;11(1):32–40.
29. Eppich WJ, Hunt EA, Duval-Arnould JM, Siddall VJ, Cheng A. Structuring feedback and debriefing to achieve mastery learning goals.
Acad Med 2015;90(11):1501.
30. Blanchard BS, Fabrycky WJ.
Systems Engineering and Analysis. 5th ed. Prentice Hall International Series in Industrial and Systems Engineering.New Jersey: Pearson; 2011.
31. Russ AL, Fairbanks FJ, Karsh BT, Militello LG, Saleem JJ, Wears RL. The science of human factors: separating fact from fiction.
BMJ Qual Saf 2013;22(10):802–808.
32. Sawyer T, White M, Zaveri P, et al. Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine.
Acad Med 2015;90(8):1025–1033.
33. Cheng A, Grant V, Dieckmann P, Arora S, Robinson T, Eppich W. Faculty development for simulation programs: five issues for the future of debriefing training.
Simul Healthc 2015;10(4):217–222.
34. Festa M, Sigalet E, Eppich WJ, Cheng A, Grant VJ. Simulation Education Program Development. In: Comprehensive Healthcare Simulation: Pediatrics. Comprehensive Healthcare Simulation. Springer, Cham; 2016:355–371.
35. Mullan PC, Wuestner E, Kerr TD, Christopher DP, Patel B. Implementation of an in situ qualitative debriefing tool for resuscitations.
Resuscitation 2013;84(7):946–951.
36. Archer JC. State of the science in health professional education: effective feedback.
Med Educ 2010;44(1):101–108.
37. Lefroy J, Watling C, Teunissen PW, Brand P. Guidelines: the do's, don'ts and don't knows of feedback for clinical education.
Perspect Med Educ 2015;4(6):284–299.
38. Hurst J, Greene L, Gough S. W5 How to improve the quality of the design and evaluation of interprofessional in-situ simulation-based education.
BMJ Simul Technol Enhanced Learn 2017;3(Suppl 2):A3–A4.
39. Varkey P, Reller MK, Resar RK. Basics of quality improvement in health care.
Mayo Clin Proc 2007;82(6):735–739.
40. Akhtar M, Casha J, Ronder J, Sakel M, Wright C, Manley K. Leading the health service into the future: transforming the NHS through transforming ourselves - ProQuest.
Int Pract Develop J 2016;6(2) Available at:
https://www.fons.org/Resources/Documents/Journal/Vol6No2/IPDJ_0602_05.pdf. Accessed May 3, 2019.
41. Noble DJ, Lemer C, Stanton E. What has change management in industry got to do with improving patient safety?
Postgrad Med J 2011;87(1027):345–348.
42. Veterans Affairs National Center for Patient Safety (2015).
Root cause analysis tools. Available at:
https://www.patientsafety.va.gov/docs/joe/rca_tools_2_15.pdf. Accessed July 24, 2018.
43. Silver SA, Harel Z, McQuillan R, et al. How to begin a quality improvement project.
Clin J Am Soc Nephrol 2016;11(5):893–900.
44. Grant VJ, Robinson T, Catena H, Eppich W, Cheng A. Difficult debriefing situations: a toolbox for simulation educators.
Med Teach 2018;40(7):703–712.
45. Slakey DP, Simms ER, Rennie KV, Garstka ME, Korndorffer JR Jr. Using simulation to improve root cause analysis of adverse surgical outcomes.
Int J Qual Health Care 2014;26(2):144–150.
