PRS AAPS Oral Proofs 2016
Ricardo J. Bello, MD, MPH, Melanie R. Major, BS, Damon S. Cooney, MD, PhD, Scott D. Lifchez, MD, Gedge D. Rosson, MD, Carisa M. Cooney, MPH
From the Johns Hopkins University, Baltimore, Md.
PURPOSE: Evaluating resident skill acquisition is challenging, particularly for surgical specialties. The required ACGME Next Accreditation System documentation increases evaluation burden to individual training programs. In 2013, we developed the Operative Entrustability Assessment (OEA) to facilitate compliance with ACGME-mandated changes and to document resident operative performance at point of care. This web-based tool allows real-time and transparent operative skill evaluation. The aim of this study was to evaluate the construct validity of OEA, as reflected by associations with clinical process indicators.
METHODS: We reviewed autologous-based breast reconstructions since implementation of OEA. We assessed associations between self-assessment and evaluator OEA scores and operative time using analysis of variance and multivariable linear regression.
RESULTS: From September 2013 to July 2015, 94 OEAs were completed for autologous breast reconstructions. Average OEA scores were 4.28 (±0.85) for self-assessment and 4.06 (±0.95) for attending evaluations. Self-assessed OEA was significantly associated with shorter operative time (32.64 minutes per OEA level; P = 0.047), even after adjusting for confounding (P = 0.008). We found weak evidence for an association between evaluator OEA scores and operative time (28.33 minutes per OEA level; P = 0.055), which was significant after adjusting for confounding (P = 0.018).
CONCLUSIONS: OEA demonstrates construct validity as increasing competency level is negatively correlated with operative time, adding evidence to the usefulness of this tool through which programs and residents can evaluate real-time progress while complying with ACGME requirements.