Share this article on:

Is There a Gap Between Program Directors' Expectations and Residents' Performance? Comparing Predictions With Outcomes for Competence in Patient Care

Carraccio, Carol MD, MA; Englander, Robert MD, MPH; Adams, David MD, PhD; Giudice, Erin MD; Olsen, A. Gail RN, MA, MBA, PhD

doi: 10.1097/ACM.0b013e3181e16119
Graduate Medical Education

Purpose To determine whether a gap exists between program directors' expectations for performance and residents' actual performance on learning objectives designed to address the Accreditation Council for Graduate Medical Education's (ACGME's) competencies.

Method In 2007–2008, the authors analyzed data on the performance of 40 pediatric residents across all levels of training who were assessed a total of 8,974 times during 2002–2005 on 35 learning objectives developed to address the ACGME patient care competency. Residents “passed” the objectives if assessments met the expected performance standards for each level of training that were set by pediatric program directors in a previous survey. To address concurrent validity, the authors also reviewed faculty evaluations of resident performance using a modified version of the American Board of Internal Medicine competency card.

Results The mean pass rate (percentage of residents meeting or exceeding the predetermined standard) for the 35 learning objectives was 92% for first-year, 84% for second-year, and 72% for third-year residents. Actual performance met expected performance standards for all patient care learning objectives except those for procedural competence and some related to patient management. However, faculty competency card evaluations demonstrated that residents met or exceeded competence in patient care, suggesting that program directors set unrealistic standards for these few items.

Conclusions Program directors' expectations predicted residents' actual performance for the majority of patient care learning objectives. The authors offer lessons learned to inform the next iteration of performance standards for clinical competence.

Dr. Carraccio is professor, Department of Pediatrics, University of Maryland School of Medicine, Baltimore, Maryland.

Dr. Englander is associate professor, Department of Pediatrics, University of Connecticut School of Medicine, Farmington, Connecticut.

Dr. Adams is a pediatrician and biochemical geneticist practicing in Bethesda, Maryland.

Dr. Giudice is assistant professor, Department of Pediatrics, University of Maryland School of Medicine, Baltimore, Maryland.

Dr. Olsen is program developer, Department of Pediatrics, University of Maryland School of Medicine, Baltimore, Maryland.

Correspondence should be addressed to Dr. Carraccio, University of Maryland, Department of Pediatrics, Rm N5W56, 22 South Greene Street, Baltimore, MD 21201; telephone: (410) 328-5213; fax: (410) 328-0646; e-mail: ccarraccio@peds.umaryland.edu.

© 2010 Association of American Medical Colleges