How and Why Internal Medicine Clerkship Directors Use Locally Developed, Faculty-Written Examinations: Results of a National Survey

Kelly, William F. MD; Papp, Klara K. PhD; Torre, Dario MD, PhD; Hemmer, Paul A. MD, MPH

Academic Medicine:
doi: 10.1097/ACM.0b013e318258351b
Assessment
Abstract

Purpose: To describe how and why internal medicine clerkship directors (CDs) use locally developed, faculty-written (LFW) examinations.

Method: In 2009, the Clerkship Directors in Internal Medicine conducted an annual, online, confidential survey of its 107 U.S. and Canadian institutional members, including questions about LFW examinations. Data were analyzed using descriptive statistics and coding of free text.

Results: Sixty-nine of 107 members (64.5%) responded. The National Board of Medical Examiners (NBME) examination was administered by 93% (63/68), LFW examinations were used by 33% (22/67), and both types were used by 22% (17/67)—compared with 85%, 36%, and 12% in 2005. LFW examinations were frequently created by the CD alone (9/22; 41%) and consisted of one test (12/22; 52.2%), but some schools gave two (6/22; 26.1%), three (2/22; 8.6%), or four or more (3/22; 13%). Multiple-choice examinations were most common (26/38; 68.4%), followed by short-answer (8/38; 21.1%) and essay (4/38; 10.5%). Most were graded using preestablished criteria; half required a minimum passing score (60% most common). LFW exams were most commonly 5% to 10% of the total grade. Only a minority of CDs reported having reliability estimates or a control group for their exams. Most (70%) reported using LFW exams to cover content felt to be underrepresented by the NBME.

Conclusions: Findings strongly suggest that a minority of internal medicine CDs use LFW examinations, mostly to measure achievement not assessed by the NBME. However, validity evidence is not consistently being gathered, which may limit judgments based on exam results.

Author Information

Dr. Kelly is associate professor of medicine, associate clerkship director, and associate residency program director, Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Papp was associate professor and director, Center for the Advancement of Medical Learning, Case Western Reserve University School of Medicine, Cleveland, Ohio, at the time of the study. She is now associate dean, SUNY Downstate Medical Center, Brooklyn, New York.

Dr. Torre was associate professor and associate program director, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, at the time of the study. He is now subinternship director and associate medicine residency director, Drexel University College of Medicine, Philadelphia, Pennsylvania.

Dr. Hemmer is professor and vice chairman for educational programs, Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Correspondence should be addressed to Dr. Kelly, USUHS-EDP, 4301 Jones Bridge Rd., Bethesda, MD 20814; telephone: (301) 295-2010; e-mail: wkelly@usuhs.mil.

© 2012 Association of American Medical Colleges