Institutional members access full text with Ovid®

Share this article on:

A Taxonomy of Delivery and Documentation Deviations During Delivery of High-Fidelity Simulations

McIvor, William R. MD; Banerjee, Arna MD; Boulet, John R. PhD; Bekhuis, Tanja PhD, MS; Tseytlin, Eugene MS; Torsher, Laurence MD; DeMaria, Samuel Jr MD; Rask, John P. MD; Shotwell, Matthew S. PhD; Burden, Amanda MD; Cooper, Jeffrey B. PhD; Gaba, David M. MD; Levine, Adam MD; Park, Christine MD; Sinz, Elizabeth MD, FCCM; Steadman, Randolph H. MD, MS; Weinger, Matthew B. MD, MS

doi: 10.1097/SIH.0000000000000184
Empirical Investigations

Introduction We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations.

Methods Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development.

Results The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy.

Conclusions This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

Supplemental digital content is available in the text.

From the Department of Anesthesiology, University of Pittsburgh Medical Center and the Winter Institute for Simulation Education and Research (WISER), Pittsburgh, PA (W.R.M.); The Division of Anesthesiology Critical Care Medicine, Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN (A.B.); Research and Data Resources, Foundation for Advancement of International Medical Education and Research, Philadelphia, PA (J.R.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine; and Department of Dental Public Health, University of Pittsburgh School of Dental Medicine, Pittsburgh, PA (T.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA (E.T.); Department of Anesthesiology, The Mayo Clinic, Rochester, MN (L.T.); Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (S.DM.); Department of Anesthesiology and Critical Care Medicine, and UNM BATCAVE Simulation Center, University of New Mexico School of Medicine, Albuquerque, NM (J.P.R.); Department of Biostatistics, Vanderbilt University, Nashville, TN (M.S.S.); Clinical Skills and Simulation, Cooper Medical School of Rowan University, Cooper University Hospital, Camden, NJ (A.B.); Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, and Center for Medical Simulation, Boston, MA (J.C) Immersive Learning, Stanford University, Stanford, CA; and VA Palo Alto, Palo Alto, CA (D.M.G.); Department of Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (A.L.); Department of Anesthesiology, Northwestern University Feinberg School of Medicine, Chicago, IL (C.P.); Department of Anesthesiology and Neurosurgery, Pennsylvania State University College of Medicine, Hershey, PA (E.S.); Anesthesiology Department, University of California, Los Angeles, CA (R.S.); and Center for Research and Innovation in Systems Safety, Institute for Medicine and Public Health, Vanderbilt University; Geriatric Research, Education, and Clinical Center (GRECC); VA Tennessee Valley Healthcare System; and Anesthesiology, Biomedical Informatics, and Medical Education, Vanderbilt University School of Medicine, Nashville, TN (M.B.W.).

Reprints: William R. McIvor, MD, Peter M. Winter Institute for Simulation Education Research, 230 McKee Place, Suite 300, Pittsburgh, PA 15213 (e-mail:

The authors declare no conflict of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (

© 2017 Society for Simulation in Healthcare