Journal Logo

Empirical Investigations

Pilot-Phase Findings From High-fidelity In Situ Medical Simulation Investigation of Emergency Department Procedural Sedation

Kobayashi, Leo MD; Dunbar-Viveiros, Jennifer A. RN, MS; Devine, Jeffrey RN, NREMT-P; Jones, Mark S.; Overly, Frank L. MD; Gosbee, John W. MD, MS; Jay, Gregory D. MD, PhD

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: April 2012 - Volume 7 - Issue 2 - p 81-94
doi: 10.1097/SIH.0b013e31823b9923

Abstract

Procedural sedation (PS) is becoming widespread.1 Situated between anxiolysis and general anesthesia, the inducible therapeutic states of moderate and deep sedation exist wherein patients “respond purposefully to verbal commands” or “cannot be easily aroused but respond purposefully following painful or repeated stimulation,” respectively.2 The attendant reduction in anxiety, pain, and discomfort improves clinical conditions for procedural interventions, yet there is a consequent increase in complications because of the need to use potent medications.1,3,4

Emergency department PS (EDPS) is now commonly used for a broad spectrum of applications and represents a substantial proportion of PS performed by nonanesthesiologists.5,6 Retrospective reviews of EDPS reveal adverse event rates as high as 17%7–9; prevention or mitigation of complications may be possible through effective implementation of patient safety systems and training programs in pertinent knowledge, specific skills, processes, and teamwork behaviors for EDPS operators. A preemptive approach to EDPS—one that emphasizes protocol-driven preparatory steps before sedation, an enhanced awareness of and readiness for adverse events, and maintenance of vigilance during and after sedation—may help attain this objective.

Investigators hypothesized that the use of sophisticated, fully interactive manikin-based medical simulation (SIM) to study EDPS and its operators may help elucidate and improve sedation training and practice, while simultaneously enhancing staff preparedness and vigilance for significant complications. In situ medical simulations within actual clinical settings have successfully replicated target medical issues at the point of care, where specific variables of interest can be modified or substituted for the purpose of training,10–14 research,15–17 or patient safety.18–21 Examples of successful on-site applications for systems probing and context-dependent performance assessment include pediatric medical and trauma patient care scenarios in ED resuscitation bays,22 teamwork and communications sessions held inside actual obstetric suites,12,23,24 and cardiopulmonary resuscitation preparation in a new hospital complex.25

Building on prior efforts26–30 to apply the capabilities of SIM toward the promotion of an EDPS patient safety agenda, investigators initiated the Simulation Learning Initiative in Procedural Sedation Training for Routine Engagement of Anticipatory Maneuvers (SLIPSTREAM) program to (1) assess the feasibility and limitations of high-fidelity in situ EDPS simulations, (2) evaluate the ability of a simulation-based approach to generate objective, reproducible EDPS data for meaningful patient safety analysis, and (3) engage in development of an applied simulation methodology to differentiate safe from unsafe EDPS practice. This article presents pilot-phase aspects of the SLIPSTREAM program and its preliminary acquisition, characterization, and analysis of EDPS operator performance data sets.

METHODS

Setting and Sample

The program’s pilot phase was conducted at an academic 719-bed regional referral hospital and level 1 trauma center. All live EDPS at the site is conducted by attending physicians [with or without supervised upper-level emergency medicine (EM) residents] in ED critical care room (CCR) spaces. Personnel from the institution’s simulation center and departments of emergency medicine and nursing education developed and implemented the program after an EDPS literature review and internal quality management chart review of study site EDPS documentation. The program’s study design (Fig. 1) and research protocol were reviewed and approved by the Institutional Review Board of the study site.

FIGURE 1
FIGURE 1:
The SLIPSTREAM program research design (pilot phase).

After exploratory prepilot sessions without subjects, 10 pilot subjects were recruited from postgraduate first-year EM residents and attending EM physicians; these groups were selected for pilot-phase sessions to determine the ability of the study protocol to evaluate and measure a broad range of EDPS performances and to allow for differentiation of inexperienced operators from experienced physicians.

Simulation Scenario Development

Investigators created 2 on-site SIM scenarios “A” and “B” for EDPS operator immersion with a SimMan manikin (Laerdal, Wappingers Falls, NY), configured as part of a portable in situ simulation system suite. Scenario construction and content were guided by institutional EDPS-related documentation deficiencies that were identified during preliminary chart review and that were posited to serve as representative markers of EDPS clinical practice hazards, for example, procedural site and side were formally documented for 86.1% of EDPS analyzed, and physician time-out completion for 55.7%. Data from the published literature6,7 that highlight the significance and relative frequency of respiratory/ventilatory complications factored prominently in scenario development, along with patient safety concerns arising from prior simulation-based investigations that confirmed flaws in the study site’s resuscitation equipment supply chain.

The overall structure of both scenarios and their programming was identical by design except for specific patient characteristics (ie, patient identifiers, age, gender, presence of underlying medical conditions, and allergies), such that the scenarios were interchangeable with respect to overall structure, scenario flow/event timing, and adverse events (Fig. 2). [The scenarios were designed in this manner to allow baseline, then experimental (with novel EDPS patient safety intervention), observations of subject performance during the postpilot (study) phase without the risks associated with repeating a single scenario twice for each subject.]

FIGURE 2
FIGURE 2:
Conceptual models of EDPS in clinical practice and during simulation.

Both scenarios featured presedation, sedation, adverse event, and sedation recovery phases with manikin states, specific physiologic parameters, and scripted branch points that were based on research objectives, anticipated EDPS operator actions, and institutional EDPS medication availabilities. As scenario timeline was critical to implementing comparable and reproducible EDPS simulations, the sedative medication administration time [ie, time of injection of a solitary agent (eg, etomidate) or time of injection of the last agent in a multimedication regimen (eg, midazolam and fentanyl)] was defined as the scenario-synchronizing sedation start time. This enabled consistency in sedation scenario progression, with both procedural completion (ie, successful shoulder reduction) and onset of hypoventilation occurring at 5 minutes after induction, apnea by 6 minutes after induction, and hypoxia [oxygen saturation (pulse oximetry) (SpO2), 73%] by 7 minutes after induction. Preprogrammed recovery from EDPS adverse event was triggered through appropriate ventilatory management by subjects; predefined time limits specified when hypoxic cardiac arrest would occur in the case of inadequate resuscitative and supportive measures.

Each scenario had 4 distinct and unique patient safety–related study probes embedded for detection by subjects:

  •  Probe 1: the potential for difficult airway/sedation management based on simulation patient history and/or physical examination (beard for scenario A; asthma history and opioid allergy in scenario B),
  •  Probe 2: the need for anticipatory requisitioning of airway “rescue” devices and/or reversal medications because of probe 1,
  •  Probe 3: defective EDPS-related equipment [bag-valve mask (BVM) without oxygen tubing in scenario A; nonfunctioning pulse oximetry probe in scenario B], and
  •  Probe 4: cueing for the optimal method of airway and ventilatory management (expeditious endotracheal intubation for “can’t BVM” state in scenario A; avoidance of invasive ventilatory management for BVM-responsive hypoxia in scenario B).

Because of the challenges in precisely ascertaining the determinants of EDPS safety, these binary investigative probes served as surrogate markers that populated a health care Haddon matrix31 for multifactorial assessment of EDPS elements expected to contribute significantly to patient safety relative to the origin, progression, and propagation of medical errors and adverse events. [A Haddon matrix with preevent (EDPS preparation), event (EDPS, monitoring, and adverse event), and postevent (EDPS adverse event management and recovery) phases was conceptually applied with the following probe-assigned attributes: host factors = patient characteristics impacting EDPS; agent factors = provider EDPS abilities and readiness for complications; environmental factors = EDPS/resuscitation equipment functionality and availability, and institutional EDPS policies/procedures.]

Study Research Tool Development

Three complementary tool sets were used to address distinct aspects of EDPS practice. Existing institutional forms for EDPS patient care documentation [T-sheet (T-System, Dallas, TX), procedural consent, procedure time-out, down-time medication/laboratory/imaging order sheets, and 3-page sedation record] were used as 1 set of study observation tools for indirect charting-based examination of EDPS clinical performance. Investigators developed a second set of tools in the form of research checklist forms that were derived from published EDPS forms7,31–34 and expert consultant input (F.L.O. and J.W.G.) to allow observer-/video-based assessment. Both of these study tool sets covered presedation (assessment and protocol compliance, equipment check), intrasedation (medication selection and ordering, monitoring, complications), and postsedation/recovery (assessment and protocol compliance) clinical performance and timeliness. All checklists used during the pilot sessions were unweighted because of the challenge of predicting high-yield elements without baseline performance data.

The third research tool set comprised scenario-specific forms to quantify subjects’ situational awareness in a retrospective format akin to the postevent Situation Awareness Global Assessment Technique.35 This tool set was developed with a modified Delphi method and contained queries on critical EDPS patient care elements that would be expected to be essential for appropriate and sufficient operator situational awareness (eg, patient name and allergies, EDPS team member names, procedural sidedness, severity of vital sign aberrances).

Pilot-Phase Simulation Session Conduct

Three prepilot sessions were completed without subjects by study investigators (L.K. as session director, J.A.D.-V. and J.D. as study scenario facilitators, and M.S.J. as simulation specialist/audiovisual technician) for preliminary research protocol testing and troubleshooting. Pilot subjects were then recruited and participated in 10 pilot sessions to test the EDPS research scenarios for feasibility, realism, flow/progression, and programming bugs. The research protocol, in situ simulation environment, audiovisual recording setup, and survey system were monitored for process incompatibilities, equipment deficiencies, and unexpected logistic problems.

Before subject arrival, all simulation elements and equipment, in situ environment, and research materials were configured and reviewed against a session checklist for consistent and reproducible scenario-specific functionality. Subjects underwent informed consent by study coinvestigator (J.A.D.-V.) on arrival and then completed a presimulation Web survey on demographic characteristics and prior EDPS experience. A scripted orientation to the study setting addressed the patient simulator, in situ simulation CCR environment, study scenario facilitator roles [orthopedic physician proceduralist and EDPS registered nurse (RN)] and session parameters (eg, no subject interaction with live ED environment or personnel beyond study CCR). Pilot subjects subsequently engaged in 1 EDPS simulation each (randomized to either scenario A or B) with on-site performance observation and audiovisual recording. On scenario completion, the situational awareness tool was administered along with an exit Web survey that queried subjects’ perceptions of the simulations. Research material collection and in situ environment cleanup were completed with a session checklist to ensure postsimulation safety of the patient care space.

Data Analysis

Analysis of EDPS Simulation Methodology

Investigators qualitatively analyzed the EDPS scenarios and pilot-phase sessions on an ongoing basis to monitor the feasibility, logistics, and practical requirements of the research protocol. Findings were used to recursively revise the research protocol for optimal simulation session preparation, subject orientation, session conduct, and environment cleanup.

All pilot-phase simulations were quantitatively assessed for structural equivalence between scenarios A and B and for the consistency of scenarios as experienced by the 2 subject groups. Specifically, program-driven intervals—from administration of sedation induction agents until completion of shoulder reduction procedure and until onset of apnea and of hypoxia, as well as for recovery after appropriate ventilatory support—were monitored and compared using Mann-Whitney U tests, along with manikin logs of simulated vital signs and physiologic parameters.

Subject responses to the exit survey that gauged scenario realism, relevance, and impact on clinical practice were used to indirectly assess for face validity of the scenarios.

Analysis of EDPS Operator Performance

Subject performances were examined for overall EDPS patient care and for phase-specific care during presedation (planning and evaluation, as well as preparation), sedation (induction, monitoring, maintenance, and adverse event management), and sedation recovery phases. Data on 141 variables of simulated EDPS operator performance were extracted from subject-completed EDPS documentation and situational awareness assessment tool sets as well as investigator application of research checklists during simulation and off-line video review. Fisher exact and Mann-Whitney U tests were completed on these data to assess for between-group differences. Tests for correlation were attempted to detect any relationship between subject demographic or prior EDPS experience characteristics and the completion, timeliness, and duration of checklist actions (either self-documented or observed), detection of study probes, and situational awareness. These analyses were used to assess for scenario content validity and discriminant power.

Derivation of Simulation EDPS Safety Composite Score

After detection and characterization of significant performance differences in critical EDPS patient safety behaviors between inexperienced and experienced operators during the pilot-phase sessions, the investigators derived a Simulation EDPS Safety Composite Score from measured variables. Modeling with pilot subject data sets led to the adoption of a score cutoff value with optimal test characteristics for differentiation of safe from unsafe EDPS practice.

RESULTS

Ten pilot simulations with 5 EM interns [40% women; age, 26.6 ± 0.5 years (range, 26–27 years)] and 5 attending emergency physicians {20% women [no significant difference (NS)]; age, 39.8 ± 7.9 years (range, 30–48 years); P = 0.006} were completed during the course of 3 months in 2010. No subjects in the intern group had experience as a primary live adult EDPS operator; attending subjects reported an average experience of 51 ± 16 live adult EDPS (range, 30–75; P = 0.006). Confidence levels in adult EDPS skills on a 3-point ordinal score (0 = “not comfortable,” 2 = “very comfortable”) were 0 and 1.4 ± 0.5 (range, 1–2; P = 0.006) for intern and attending subjects, respectively; exposure to medical simulation did not differ between groups.

Research Protocol Findings

Protocol Revisions

Despite extensive preparation and troubleshooting with the 3 prepilot sessions, several small changes to the research protocol were necessary in response to pilot session findings. One notable modification was the strict specification of RN facilitation role with respect to the defective BVM and pulse oximetry equipment (ie, later subjects were required to explicitly request and use-test replacement equipment in case of defect detection). Relevant data points for 2 prerevision subjects were excluded as “protocol errors” due to imprecise facilitation during the correction of equipment issues identified by the subjects. Separate from minor changes to assessment tool sets and session setup/cleanup checklists, the scenario content, programming, props, and probes otherwise remained intact. No changes met Institutional Review Board resubmission criteria.

Structural Equivalence of Simulation Scenarios A and B

Times from sedation induction until completion of shoulder reduction by the proceduralist facilitator [scenario A at 303 ± 25 seconds vs. scenario B at 307 ± 17 seconds (NS)], onset of apnea [302 ± 1 vs. 301 ± 2 seconds (NS)], onset of hypoxia [362 ± 1 vs. 363 ± 3 seconds (NS)], and recovery of spontaneous respirations [540 ± 52 vs. 498 ± 20 seconds (NS)] were recorded. Maximal heart rates [132 ± 0 beats per minute (bpm) for scenario A vs. 136 ± 15 for scenario B (NS)], minimal respiratory rates [0 vs. 0 breaths per minute (bpm) (NS)], lowest percentage of SpO2’s [73% ± 0% vs. 85% ± 5% (P = 0.012)], and end-tidal carbon dioxide (ETCO2) ranges [18–48 vs. 19–45 mm Hg (P = 0.012 for maximum ETCO2 only)] were logged during pilot sessions (see time vs. simulated vital sign parameter plots, Supplemental Digital Content 1, http://links.lww.com/SIH/A25 which represents the changes in simulated patients’ vital signs over time between scenarios A and B).

Consistency of Simulation Scenarios as Experienced by Pilot Groups

The order of participation in pilot sessions and the mix of scenario types experienced were not significantly different between the intern and attending subject groups. Subject initiation of sedation was recorded at 747 ± 199 seconds after simulation start for intern subjects’ scenarios and 849 ± 147 seconds (NS) for attending subjects’ scenarios (NS). Postinduction times until proceduralist facilitator completion of shoulder reduction [interns’ scenarios at 309 ± 16 seconds vs. attending physicians’ scenarios at 301 ± 25 seconds (NS)], onset of apnea [301 ± 1 vs. 302 ± 2 seconds (NS)], onset of hypoxia [362 ± 1 vs. 363 ± 3 seconds (NS)], and recovery of spontaneous respirations (518 ± 41 vs. 520 ± 50 seconds (NS)] were not different. Maximal heart rates [133 ± 15 bpm for interns’ scenarios vs. 135 ± 6 bpm for attending physicians’ scenarios (NS)], minimal respiratory rates [0 vs. 0 bpm (NS)], lowest percentage of SpO2’s [81% ± 9% vs. 77% ± 6% (NS)], and ETCO2 ranges [19–48 vs. 15–48 mm Hg (P = 0.06 for minimum ETCO2 only)] were logged during pilot sessions (see time vs. simulated vital sign parameter plots in Supplemental Digital Content 2, http://links.lww.com/SIH/A26, which represents the changes in simulated patients’ vital signs over time between intern subjects’ scenarios and attending physician subjects’ scenarios).

Simulation Experience Survey Data

On 11-point ordinal scales (0–10), intern subjects scored their simulation experience as realistic (8.2 ± 1.5), relevant (9.0 ± 0.7), and having impact on their clinical practice (9.4 ± 0.9); attending surveys recorded scores of 7.4 ± 0.9 (NS), 9.0 ± 0.7 (NS), and 7.4 ± 0.9 (P = 0.014) for the same queries.

Clinical Performance Metrics

EDPS Simulation Timeline

Intern pilot subjects spent 751 ± 174 seconds (range, 526–933 seconds) in the presedation phase, of which 465 ± 113 seconds (62% ± 8%; range, 280–575 seconds) involved direct patient interaction, that is, speaking with, examining, or monitoring the patient; their sedation induction, maintenance, and recovery phases were completed in 916 ± 167 seconds (range, 726–1175 seconds), of which 727 ± 97 seconds (80% ± 11%; range, 643–862 seconds) involved direct patient monitoring; intern subject total scenario time was 1666 ± 245 seconds (range, 1381–1945 seconds).

Attending pilot subjects spent 848 ± 146 seconds (range, 745–1106 seconds) in the presedation phase, of which 493 ± 62 seconds [60% ± 13% (NS); range, 427–569 seconds) involved direct patient interaction; they completed sedation and recovery phases in 980 ± 166 seconds (range, 798–1181 seconds), of which 723 ± 124 seconds [74% ± 5% (NS); range, 559–804 seconds) involved direct patient monitoring; attending subject total scenario time was 1828 ± 124 seconds (NS; range 1641–1968 seconds).

EDPS Simulation Critical Action Performance

Several independent EDPS simulation performance characteristics that served to differentiate the subject groups were detected in preparatory patient assessment, equipment review, and institutional EDPS protocol compliance. Select findings are as follows (see Appendices A and B for all results):

EDPS Preparation

No interns applied nasal cannula oxygen or nasal prong ETCO2 detectors; all attending subjects applied both (P = 0.008 and P = 0.004, respectively). Excepting presedation review of imaging and laboratory results (20% for the intern group vs. 100% for the attending group, P = 0.024), compliance with all remaining elements of institutional preparatory protocol for performance of medical procedures and EDPS was met without significant differences between subject groups. Time-out completion [40% vs. 80% (P = 0.262)], physical examination of airway characteristics [40% vs. 100% (P = 0.083)], and confirmation of procedural side and site [20% vs. 60% (P = 0.262)] did not attain statistical significance at α = 0.05.

EDPS Induction

Two intern subjects correctly selected patient- and indication-appropriate sedation medications only after requesting RN facilitator assistance (as permitted by research protocol), and incorrect induction medications were ordered by 1 intern subject who did not seek assistance; the induction procedures of these 3 subjects were addressed with postsession debriefing. All simulated patients progressed to a sedated state.

EDPS Monitoring and Recovery Assessment

Pilot groups did not differ in recognizing patient hypoventilation, hypoxia, and apnea (see the next paragraphs for adverse event management performance). Of all subjects, 80% disclosed the EDPS-related adverse event to the patient.

Detection of Simulated EDPS Safety Probes

Probe 1: Recognition of potential for difficult airway or sedation management did not differ between intern subjects (60%) and attending subjects (80%, NS).

Probe 2: A laryngeal mask airway requested by a single intern was the only airway rescue device ordered across all pilot sessions. One intern subject and 1 attending subject each requested availability of reversal medications (ie, naloxone and flumazenil).

Probe 3: The difference in recognition of defective EDPS-related equipment did not attain statistical significance between the 2 pilot groups [33% (n = 3 due to protocol errors) vs. 80% (n = 5), NS].

Probe 4: All interns provided adequate ventilatory support; only 1 intern optimally managed the scripted EDPS-related scenario complications. All attending physicians performed the optimal clinical action (P = 0.024) regardless of scenario type (nA = 3, nB = 2). Timeliness of airway/ventilatory response was similar for the intern group (BVM at 31 ± 21 seconds in scenario A; endotracheal intubation at 112 ± 39 seconds in scenario B) and the attending group [40 ± 8 seconds in scenario A; 103 ± 35 seconds in scenario B (statistical analysis not possible because of inadequate sample sizes)].

Situational Awareness

Intern subjects’ scores (53% ± 17%) and attending subjects’ scores (51% ± 14%) on scenario situational awareness assessment tool set were similar (NS). None of the 13 elements of the tool set detected performance differences between groups (Appendix C).

EDPS Clinical Documentation Metrics

Institutional EDPS Documentation Completion

Completion of current institutional procedural and EDPS-specific (before and after sedation) paperwork by pilot subject groups was scored at 58% ± 26%, 74% ± 16%, and 0% ± 0%, respectively, for interns, and 95% ± 7% (P = 0.038), 81% ± 15% (NS), and 27% ± 31% (P = 0.07), respectively, for attending physicians.

EDPS Research Checklist Completion

Intern subjects and attending subjects differed in their composite unweighted scores on investigator-completed research protocol EDPS checklists for presedation assessment/protocol (19 elements) and postsedation reassessment (13 elements), with scores of 56% ± 9% and 32% ± 18%, respectively, vs. 80% ± 8% (P = 0.006) and 57% ± 20% (P = 0.048), respectively. Scores on the 15-element composite presedation equipment checklist did not differ between groups [interns: 25% ± 18% vs. attending physicians 35% ± 22% (NS)].

Derivation of Simulation EDPS Safety Composite Score

Overall performances of the pilot subject groups were compared for completion and documentation of presedation, intrasedation, and postsedation clinical actions. Nine independent elements and 1 multi-item element were selected for their discriminant power and/or implicit importance in EDPS clinical practice and patient safety to derive the Simulation EDPS Safety Composite Score (Figs. 3A, B). The following EDPS safety functions were included: (pre-sedation)-patient assessment for difficult EDPS; EDPS equipment check; pulse oximetry monitor application; end-tidal capnometry monitor application; appropriate EDPS medication selection; time-out (patient identity); time-out (procedure type, side, and site); (intra-sedation)-EDPS adverse event detection, EDPS adverse event optimal management; and (post-sedation)-13-item assessment for patient recovery from EDPS.

FIGURE 3
FIGURE 3:
A and B, Simulation EDPS Safety Composite Score forms were derived from pilot-phase performance data for scenario A (Fig. 3A, left) and scenario B (Fig. 3B, right). See text for derivation and performance characteristics.

Composite scores as independently scored by 2 investigators (L.K. and J.A.D.-V.) were 3.98 ± 0.43 and 5.23 ± 0.54, respectively, for intern subjects (n = 3, due to 2 invalid session data points from simulation protocol violations) and 8.77 ± 1.25 and 9.14 ± 0.85, respectively, for attending subjects (n = 5) (Table 1). The intraclass correlation coefficient was 0.925 between the 2 scorers (full data set scoring by L.K., focused Simulation EDPS Safety Composite Score scoring by J.A.D.-V.). One-sample t tests using the average composite score of the intern subjects as the hypothetical mean to analyze the attending subjects’ composite scores exhibited significance (P = 0.0010 and 0.0005, respectively, for scores from each of the 2 scoring investigators, P = 0.0007 for average scores across investigators). Direct analysis for correlation between subject sedation experience and composite score was limited by concerns regarding data anchoring because the intern subject group uniformly lacked live EDPS experience. However, a cutoff value of 7.7 on the EDPS safety function–derived composite score was supported by pilot data set modeling, which differentiated experienced from inexperienced operators with promising test characteristics.

TABLE 1
TABLE 1:
Comparison of Pilot Groups’ Simulation EDPS Composite Scores Across Subject Characteristics

DISCUSSION

Various factors are increasing the use of PS in acute care environments such as the ED. The resulting changes in the characteristics of care providers, environments, and patients involved with sedation inevitably alter the nature and degree of risk involved. An unpublished 2006 review of records of EDPS performed during a 2-year period at the study institution’s adult and pediatric EDs found that physician presedation time-outs were documented for less than two thirds of EDPS charts, approximately 80% recorded age or pertinent history, and intrasedation care details and postsedation evaluations were often missing. In this context, the system-probing capabilities of in situ medical simulation provide an opportunity to better understand and address sedation-related safety concerns with an on-site assessment, education, and research methodology. The SLIPSTREAM pilot-phase simulation sessions initiated an examination into the feasibility, validity, and utility of an investigational simulation-based framework for the systematic study of EDPS performance.

Working within the research protocol’s investigative scope and parameters, program personnel created 2 interchangeable simulation scenarios for in situ application. Each of these scenarios replicated the clinical presentation, typical care progression and interactions, and associated health care “footprint” (eg, medical record/paperwork, need for ED equipment, environment, and multidisciplinary care provider team) of an ED patient with a shoulder dislocation necessitating EDPS. The field observations and pilot subjects’ survey responses regarding the simulation session experiences were limited by small sample sizes but seem strongly supportive of the face validity, reproducibility, and consistency of the scenarios, as well as research protocol feasibility and structural integrity.

During pilot-phase sessions, the scenarios successfully elicited realistic EDPS operator performance and acquired objective, high-resolution data. Inexperienced and experienced physicians were found to differ during simulation in their observed presedation preparation, management of adverse events, assessment of patient recovery, and completion of relevant documentation; these data begin to suggest content validity and the discriminant power of the EDPS simulation scenarios. Conversely, EDPS operator situational awareness as measured during simulation did not seem to vary—whether the uniformly low scores are attributable to a faulty research tool set or represent actual clinical practice is unclear at this time.

Pilot group differences in overall completeness of institutional documentation forms and of research checklisting likely reflected underlying dissimilarities in compliance with departmental protocol, EDPS preparedness, and postsedation vigilance—all operator characteristics that depend on provider observation of patient safety behaviors and clinical experience. To maximize the utility and usability of EDPS simulation as a research and assessment probing methodology, several of the objective, verifiable, and distinguishing EDPS performance metrics were extracted to derive a simple 10-point Simulation EDPS Safety Composite Score. The modeled test characteristics of this proposed composite score seem to support its potential to assess the safety of EDPS operator performance, although it may ultimately be limited in its ability to differentiate between operators observing minimum safety precautions and actions and those clinicians practicing optimally safe EDPS.

Based on the development work completed during the pilot phase, the SLIPSTREAM program is proceeding with its controlled experimental research phase, which features before-and-after EDPS simulations with a novel just-in-time EDPS clinical process guidance system intervention. These investigations with larger subject samples will continue to evaluate the simulation-based EDPS research methodology for its utility.

Limitations

Because the research design was constrained with a small pilot subject pool to generate a “good enough” (ie, noncomprehensive) method to detect safe EDPS performance, statistical power was limited; true differences in some dimensions of EDPS performance between interns and attending EM physicians were likely underdetected (type II error). This limitation was prominently noted for several EDPS clinical performance metrics such as detection of defective EDPS-related equipment [33% in the intern group (n = 3) vs. 80% in the attending group (n = 5)]. Type I error also remains a distinct and concurrent possibility for other aspects of simulated EDPS, even with the substantial differences in clinical experience and expertise between subject groups.

Assurance of the experiential consistency of the 2 structurally equivalent scenarios was attempted through time-synchronized and programmed changes in simulator states, use of thematically linked probes (ie, EDPS-complicating patient factors and equipment failure), and monitoring for stability of scenario complexity and flow; however, the possibility exists that the scenarios were not equivalent and thereby elicited unequal EDPS performance between subjects and across subject groups.

CONCLUSIONS

The SLIPSTREAM program pilot phase engaged in the development of a simulation-based methodology to objectively examine EDPS practice and patient safety. Despite the small number of subjects, pilot sessions resulted in study protocol revisions, pertinent EDPS operator performance data sets, and a promising composite scoring system with complementary simulation scenarios for assessment of patient safety during EDPS.

ACKNOWLEDGMENTS

The authors thank Anna C. Cousins for her insight and assistance in manuscript preparation and Jason T. Machan, PhD, for his assistance with statistical analysis.

REFERENCES

1. Bahn EL. Procedural sedation and analgesia: a review and new concepts. Emerg Med Clin North Am 2005; 23: 503–517.
2. American Society of Anesthesiologists. Practice guidelines for sedation and analgesia by non-anesthesiologists. Anesthesiology 2002; 96: 1004–1017.
3. Krauss B, Green SM. Sedation and analgesia for procedures in children. N Engl J Med 2000; 342: 938–945.
4. Doyle L, Colletti JE. Pediatric procedural sedation and analgesia. Pediatr Clin North Am 2006; 53: 279–292.
5. Miner JR, Krauss B. Procedural sedation and analgesia research: state of the art. Acad Emerg Med 2007; 14: 170–178.
6. Sacchetti A, Stander E, Ferguson N, Maniar G, Valko P. Pediatric procedural sedation in the community emergency department: results from the ProSCED registry. Pediatr Emerg Care 2007; 23: 218–222.
7. Pena BM, Krauss B. Adverse events of procedural sedation and analgesia in a pediatric emergency department. Ann Emerg Med 1999; 34: 483–491.
8. Roback MA, Wathen JE, Bajaj L, Bothner JP. Adverse events associated with procedural sedation and analgesia in a pediatric emergency department: a comparison of common parenteral drugs. Acad Emerg Med 2005; 12: 508–513.
9. Frymann SJ, Cumberbatch GL, Stearman AS. Reduction of dislocated hip prosthesis in the emergency department using conscious sedation: a prospective study. Emerg Med J 2005; 22: 807–809.
10. Draycott TJ, Crofts JF, Ash JP, et al.. Improving neonatal outcome through practical shoulder dystocia training. Obstet Gynecol 2008; 11: 14–20.
11. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WC. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest 2008; 133: 56–61.
12. Miller KK, Riley W, Davis S, Hansen HE. In situ simulation: a method of experiential learning to promote safety and team behavior. J Perinat Neonatal Nurs 2008; 22: 105–113.
13. Nunnink L, Welsh AM, Abbey M, Buschel C. In situ simulation-based team training for post-cardiac surgical emergency chest reopen in the intensive care unit. Anaesth Intensive Care 2009; 37: 74–78.
14. Allan CK, Thiagarajan RR, Beke D, et al.. Simulation-based training delivered directly to the pediatric cardiac intensive care unit engenders preparedness, comfort, and decreased anxiety among multidisciplinary resuscitation teams. J Thorac Cardiovasc Surg 2010; 140: 646–652.
15. Blum R, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg 2005; 100: 1375–1390.
16. Kobayashi L, Overly F, Williams KA, Shapiro MJ, Jay G. Educational and research implications of portable human patient simulation in acute care medicine. Acad Emerg Med 2008; 15: 1166–1174.
17. Kobayashi L, Lindquist DG, Jenouri IM, et al.. Comparison of sudden cardiac arrest resuscitation performance data obtained from in-hospital incident chart review and in situ high-fidelity medical simulation. Resuscitation 2010; 81: 463–471.
18. Kobayashi L, Shapiro MJ, Sucov A, et al.. Portable advanced medical simulation for new emergency department testing and orientation. Acad Emerg Med 2006; 13: 691–695.
19. Herzer KR, Rodriguez-Paz JM, Doyle PA, et al.. A practical framework for patient care teams to prospectively identify and mitigate clinical hazards. Jt Comm J Qual Patient Saf 2009; 35: 72–81.
20. Rodriguez-Paz JM, Mark LJ, Herzer KR, et al.. A novel process for introducing a new intraoperative program: a multidisciplinary paradigm for mitigating hazards and improving patient safety. Anesth Analg 2009; 108: 202–210.
21. Kobayashi L, Dunbar-Viveiros JA, Sheahan BA, et al.. In situ simulation comparing in-hospital first responder sudden cardiac arrest resuscitation using semi-automated defibrillators and automated external defibrillators. Simul Healthc 2010; 5: 82–90.
22. Hunt EA, Hohenhaus SM, Luo X, Frush KS. Simulation of pediatric trauma stabilization in 35 North Carolina emergency departments: identification of targets for performance improvement. Pediatrics 2006; 117: 641–648.
23. Birch L, Jones N, Doyle PM, et al.. Obstetric skills drills: evaluation of teaching methods. Nurse Educ Today 2007; 27: 915–922.
24. Hamman WR, Beaudin-Seiler BM, Beaubien JM, et al.. Using in situ simulation to identify and resolve latent environmental threats to patient safety: case study involving a labor and delivery ward. J Patient Saf 2009; 5: 184–187.
25. Villamaria FJ, Pliego JF, Wehbe-Janek H, et al.. Using simulation to orient code blue teams to a new hospital facility. Simul Healthc 2008; 3: 209–216.
26. Farnsworth ST, Egan TD, Johnson SE, Westenskow D. Teaching sedation and analgesia with simulation. J Clin Monit Comput 2000; 16: 273–285.
27. Medina LS, Racadio JM, Schwid HA. The sedation, analgesia, and contrast media computerized simulator: a new approach to train and evaluate radiologists’ responses to critical incidents. Pediatr Radiol 2000; 30: 299–305.
28. Blike GT, Christoffersen K, Cravero J, Andeweg SK, Jensen J. A method for measuring system safety and latent errors associated with pediatric procedural sedation. Anesth Analg 2005; 101: 48–58.
29. Shavit I, Keidan I, Hoffman Y, et al.. Enhancing patient safety during pediatric sedation. Arch Pediatr Adolesc Med 2007; 161: 740–743.
30. Keidan I, Gravenstein D, Berkenstadt H, Ziv A, Shavit I, Sidi A. Supplemental oxygen compromises the use of pulse oximetry for detection of apnea and hypoventilation during sedation in simulated pediatric patients. Pediatrics 2008; 122: 293–298.
31. Mohr JJ, Barach P, Cravera JP, et al.. Microsystems in health care: part 6. Designing patient safety into the microsystem. Jt Comm J Qual Saf 2003; 29: 401–408.
32. Hoffman GM, Nowakowski R, Troshynski TJ, Berens RJ, Weisman SJ. Risk reduction in pediatric procedural sedation by application of an American Academy of Pediatrics/American Society of Anesthesiologists Process Model. Pediatrics 2002; 109: 236–243.
33. Babl F, Priestley S, Krieser D, et al.. Development and implementation of an education and credentialing programme to provide safe paediatric procedural sedation in emergency departments. Emerg Med Australas 2006; 18: 489–497.
34. Pershad J, Gilmore B. Successful implementation of radiology sedation service staffed exclusively by pediatric emergency physicians. Pediatrics 2006; 117: e413–e422.
35. Endsley MR. Measurement of situation awareness in dynamic systems. Hum Factors 1995; 37: 65–84.

APPENDIX A: SLIPSTREAM PILOT-PHASE DATA FROM RESEARCH TOOL SET 1

Table A1
Table A1:
Pilot Subject Institutional EDPS Forms Completion: Description of Subject Groups’ Simulation Documentation as Recorded on Institutional Forms for ED Procedures*
Table A2
Table A2:
Pilot Subject Institutional EDPS Forms Completion: Description of Subject Groups’ Simulation Documentation as Recorded on Institutional Forms for EDPS*
Table A3
Table A3:
Pilot Subject Institutional EDPS Forms Completion: Description of Subject Groups’ Simulation Documentation as Recorded on Institutional Forms for Postsedation Reassessment*

APPENDIX B: SLIPSTREAM PILOT-PHASE DATA FROM RESEARCH TOOL SET 2

Table B1
Table B1:
Pilot Subject EDPS Research Checklist Forms Completion (Presedation Phase): Description of Subject Groups’ Simulation Presedation Performance as Recorded by EDPS Research Checklist Forms (Assessment and Protocol)*
Table B2
Table B2:
Pilot Subject EDPS Research Checklist Forms Completion (Presedation Phase): Description of Subject Groups’ Simulation Presedation Performance as Recorded by EDPS Research Checklist Forms (Equipment)*
Table B3
Table B3:
Pilot Subject EDPS Research Checklist Forms Completion (Postsedation Phase): Description of Subject Groups’ Simulation Postsedation Performance as Recorded by EDPS Research Checklist Forms (Assessment and Protocol)*

APPENDIX C: SLIPSTREAM PILOT-PHASE DATA FROM RESEARCH TOOL SET 3

Table C1
Table C1:
Pilot Subject Simulation EDPS Situational Awareness: Description of Subject Groups’ Performance on Retrospective Intrasimulation Situational Awareness Study Tool
Keywords:

Adverse effects; Deep sedation; Moderate sedation; Emergency treatment; Health care quality improvement; Patient simulation; Safety management

Supplemental Digital Content

© 2012 Lippincott Williams & Wilkins, Inc.