Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate?

Ahle, Samantha L. MD, MHS; Schuller, Mary MSEd; Clark, Michael J. PhD; Williams, Reed G. PhD; Wnuk, Gregory MHSA; Fryer, Jonathan P. MD, MHPE; George, Brian C. MD, MAEd

doi: 10.1097/ACM.0000000000002936
Research Reports
Buy
SDC

Purpose Medical educators have developed no standard way to assess the operative performance of surgical residents. Most residency programs use end-of-rotation (EOR) evaluations for this purpose. Recently, some programs have implemented workplace-based “microassessment” tools that faculty use to immediately rate observed operative performance. The authors sought to determine (1) the degree to which EOR evaluations correspond to workplace-based microassessments and (2) which factors most influence EOR evaluations and directly observed workplace-based performance ratings and how the influence of those factors differs for each assessment method.

Method In 2017, the authors retrospectively analyzed EOR evaluations and immediate postoperative assessment ratings of surgical trainees from a university-based training program from the 2015–2016 academic year. A Bayesian multivariate mixed model was constructed to predict operative performance ratings for each type of assessment.

Results Ratings of operative performance from EOR evaluations vs workplace-based microassessment ratings had a Pearson correlation of 0.55. Postgraduate year (PGY) of training was the most important predictor of operative performance ratings on EOR evaluations: Model estimates ranged from 0.62 to 1.75 and increased with PGY. For workplace-based assessment, operative autonomy rating was the most important predictor of operative performance (coefficient = 0.74).

Conclusions EOR evaluations are perhaps most useful in assessing the ability of a resident to become a surgeon compared with other trainees in the same PGY of training. Workplace-based microassessments may be better for assessing a trainee’s ability to perform specific procedures autonomously, thus perhaps providing more insight into a trainee’s true readiness for operative independence.

S.L. Ahle is a general surgery resident, Yale School of Medicine, New Haven, Connecticut.

M. Schuller is manager, Surgical Education, Department of Surgery, Northwestern University, Chicago, Illinois.

M.J. Clark is lead statistician, Consulting for Statistics, Computing and Analytics Research, University of Michigan, Ann Arbor, Michigan.

R.G. Williams was, at the time of the research reported here, adjunct research professor, Indiana University School of Medicine, Indianapolis, Indiana.

G. Wnuk is program manager, Center for Surgical Training and Research, University of Michigan, Ann Arbor, Michigan.

J.P. Fryer is vice chair of education, Department of Surgery, Northwestern University, Chicago, Illinois.

B.C. George is assistant professor and director, Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan; ORCID: https://orcid.org/0000-0002-9404-5255.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A728.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: The institutional review board of Yale School of Medicine has granted exemption for this study.

Correspondence should be addressed to Samantha L. Ahle, Department of Surgery, Yale University School of Medicine, 330 Cedar St., FMB 107, New Haven, CT 06519; telephone: (203) 785-7890; email: samantha.ahle@yale.edu; Twitter: @YaleSurgery.

Copyright © 2019 by the Association of American Medical Colleges