Skip Navigation LinksHome > January 2014 - Volume 120 - Issue 1 > Simulation-based Assessment to Identify Critical Gaps in Saf...
Anesthesiology:
doi: 10.1097/ALN.0000000000000055
Education: Original Investigations in Education

Simulation-based Assessment to Identify Critical Gaps in Safe Anesthesia Resident Performance

Blum, Richard H. M.D.; Boulet, John R. Ph.D.; Cooper, Jeffrey B. Ph.D.; Muret-Wagstaff, Sharon L. Ph.D.; for the Harvard Assessment of Anesthesia Resident Performance Research Group

Medical Education
Collapse Box

Abstract

Background: Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty.
Methods: An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys.
Results: Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ2 = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice.
Conclusion: The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

© 2014 American Society of Anesthesiologists, Inc.

Publication of an advertisement in Anesthesiology Online does not constitute endorsement by the American Society of Anesthesiologists, Inc. or Lippincott Williams & Wilkins, Inc. of the product or service being advertised.
Login

Article Tools

Share