Letters to the Editor
We appreciate the thoughtful comments expressed by Drs. Dillon, Clauser, and Melnick and by Dr. Wong. We are pleased that our study questioning the validity of using United States Medical Licensing Examination (USMLE) Step 1 and 2 scores for making postgraduate medical resident selection decisions has struck a responsive chord. Evidence-based medical education will advance as this and other validity issues are addressed by research programs that are thematic, sustained, and cumulative.
A solid foundation of medical knowledge is certainly needed for postgraduate education and practice. The USMLEs are the best available measures of acquired medical knowledge and are excellent for medical licensure decisions, as their name affirms. But physicians do not report for work in hospitals and clinics and spend the day answering multiple-choice questions. Instead, they obtain patient histories and examine patients; exercise judgment; express compassion and caring; solve problems; communicate with patients, families, and colleagues; educate themselves and others; perform complex clinical procedures; and display professionalism in many ways. Continued reliance on USMLE scores to predict the ability of an individual trainee to perform these diverse tasks is unsupported by data.
It is also important for us to clarify that the studies we performed address more than “isolated clinical and procedural skills.” For example, competence measured by our advanced cardiac life support checklists not only involves rapid decision making about cardiac physiology and medication dosing but also requires demonstration of team leadership skills and communication with patients about suggested treatments.1 Dillon and colleagues are right that important medical criteria are broad and deep, and few measures exist that yield reliable data to probe such skills and dispositions. We encourage the National Board of Medical Examiners to look beyond examinations that test only basic science and clinical knowledge and to develop rigorous assessments for the broad array of clinical skills used in everyday medical practice that can be used for residency selection decisions. The study we reported is a small step on a long research journey whose departure is behind schedule.
We also agree with Dr. Wong that leaders in postgraduate medical education need to “identify those traits, skill sets, and attitudes that best predict excellence in their particular specialties rather than simply focusing on a number.” The medical education research agenda is clear. So is the need to exercise caution when using test scores for unintended purposes.
William C. McGaghie, PhD
Jacob R. Suker, MD, Professor of Medical Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois; firstname.lastname@example.org.
Elaine R. Cohen
Research associate, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Diane B. Wayne, MD
Associate professor, residency program director, and vice chair of education, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois.