Share this article on:

Sharpening the Eye of the OSCE with Critical Action Analysis

Payne, Nancy J. MD; Bradley, Elizabeth B. PhD; Heald, Evan B. MD; Maughan, Karen L. MD; Michaelsen, Veronica E. MD; Wang, Xin-Qun PhD; Corbett, Eugene C. Jr MD

doi: 10.1097/ACM.0b013e3181850990
Clinical Skills Education

Purpose When interpreting performance scores on an objective structured clinical examination (OSCE), are all checklist items created equal? Although assigning priority through checklist item weighting is often used to emphasize the clinical importance of selected checklist items, the authors propose the use of critical action analysis as an additional method for analyzing and discriminating clinical performance in clinical skill assessment exercises. A critical action is defined as an OSCE checklist item whose performance is critical to ensure an optimal patient outcome and avoid medical error. In this study, the authors analyzed a set of clerkship OSCE performance outcome data and compared the results of critical action analysis versus traditional checklist item performance scores.

Method OSCE performance scores of 398 third-year clerkship students from 2003 to 2006 at the University of Virginia School of Medicine were analyzed using descriptive statistics and a logistic regression model. Through a consensus process, 10 of 25 OSCE cases were identified as containing critical actions.

Results Students who scored above the median correctly performed the critical actions more often than those scoring lower. However, for 9 of 10 cases, 6% to 46% of higher-scoring students failed to perform the critical action correctly.

Conclusions Failure to address this skill assessment outcome is a missed opportunity to more fully understand and apply the results of such examinations to the clinical performance development of medical students. Including critical action analysis in OSCE data interpretation sharpens the eye of the OSCE and enhances its value in clinical skill assessment.

Dr. Payne is associate professor of pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia.

Dr. Bradley is instructor of medicine, University of Virginia School of Medicine, Charlottesville, Virginia.

Dr. Heald is associate professor of medicine, University of Virginia School of Medicine, Charlottesville, Virginia.

Dr. Maughan is associate professor of family medicine, University of Virginia School of Medicine, Charlottesville, Virginia.

Dr. Michaelsen is assistant professor of medical education, University of Virginia School of Medicine, Charlottesville, Virginia.

Mr. Wang is senior biostatistician, Department of Public Health Sciences, University of Virginia School of Medicine, Charlottesville, Virginia.

Dr. Corbett is Brodie Professor of Medicine and professor of nursing, University of Virginia School of Medicine, Charlottesville, Virginia.

Correspondence should be addressed to Dr. Corbett, Box 800901 UVA HSC, Charlottesville, VA 22908; e-mail: (ecc9h@virginia.edu).

© 2008 Association of American Medical Colleges