Introduction: We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations.
Methods: Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development.
Results: The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy.
Conclusions: This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.
From the Department of Anesthesiology, University of Pittsburgh Medical Center and the Winter Institute for Simulation Education and Research (WISER), Pittsburgh, PA (W.R.M.); The Division of Anesthesiology Critical Care Medicine, Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN (A.B.); Research and Data Resources, Foundation for Advancement of International Medical Education and Research, Philadelphia, PA (J.R.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine; and Department of Dental Public Health, University of Pittsburgh School of Dental Medicine, Pittsburgh, PA (T.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA (E.T.); Department of Anesthesiology, The Mayo Clinic, Rochester, MN (L.T.); Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (S.DM.); Department of Anesthesiology and Critical Care Medicine, and UNM BATCAVE Simulation Center, University of New Mexico School of Medicine, Albuquerque, NM (J.P.R.); Department of Biostatistics, Vanderbilt University, Nashville, TN (M.S.S.); Clinical Skills and Simulation, Cooper Medical School of Rowan University, Cooper University Hospital, Camden, NJ (A.B.); Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, and Center for Medical Simulation, Boston, MA (J.C) Immersive Learning, Stanford University, Stanford, CA; and VA Palo Alto, Palo Alto, CA (D.M.G.); Department of Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (A.L.); Department of Anesthesiology, Northwestern University Feinberg School of Medicine, Chicago, IL (C.P.); Department of Anesthesiology and Neurosurgery, Pennsylvania State University College of Medicine, Hershey, PA (E.S.); Anesthesiology Department, University of California, Los Angeles, CA (R.S.); and Center for Research and Innovation in Systems Safety, Institute for Medicine and Public Health, Vanderbilt University; Geriatric Research, Education, and Clinical Center (GRECC); VA Tennessee Valley Healthcare System; and Anesthesiology, Biomedical Informatics, and Medical Education, Vanderbilt University School of Medicine, Nashville, TN (M.B.W.).
Reprints: William R. McIvor, MD, Peter M. Winter Institute for Simulation Education Research, 230 McKee Place, Suite 300, Pittsburgh, PA 15213 (e-mail: firstname.lastname@example.org).
The authors declare no conflict of interest.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.simulationinhealthcare.com).