Letters to the Editor
In Reply to Prakash:
Dr. Prakash reminds us that one of the important competencies for safe and effective patient care is the ability to differentiate between the most and least likely diagnoses, and to recognize treatments that are most and least likely to produce positive clinical outcomes.
In the realm of assessment, there are many ways to measure an individual’s ability to identify or pick the best pathway, but, unfortunately, multiple-choice question (MCQ) tests do not readily measure the skill of recognizing “what not to do.” The experience of the United States Medical Licensing Examination (USMLE) program and other assessment organizations indicates that having test takers respond to questions where the task is to identify what is least likely results frequently in an outcome that is hard to interpret, reflecting a mix of genuine knowledge (or ignorance) about the topic and confusion about the question or task. For this reason, the general practice has been to avoid the use of negatively phrased MCQs in USMLE and many other high-stakes examination programs. The good news is that other existing formats more successfully assess the examinee’s ability to differentiate good and bad clinical pathways. For example, the computer-based case simulations of the USMLE Step 3 examination, which require the examinee to manage a simulated patient, are sensitive to the selection of inappropriate treatment pathways. Another example is the Step 2 Clinical Skills examination, which requires the examinee to interact with a real person portraying a patient. In this format, an approach to the patient interaction that is not related to or appropriate for the clinical presentation will adversely affect the examinee’s score on that case.
The important competency identified by Dr. Prakash is one of many that challenge traditional testing formats like the MCQ. Using practice materials and test construction texts that focus on MCQs as a measure of what is perceived to be important or unimportant to the practice of medicine is ill advised. The USMLE program continues to work with faculty and practitioners from across the country to identify and assess the competencies that support safe and effective practice. Some competencies are well measured with traditional assessment methods, some require innovative approaches, and some may never be well measured by standardized examinations. The interested stakeholder should not interpret this as a reflection of a competency’s relative importance in practice.
Gerard F. Dillon, PhD
Vice president, USMLE, National Board of Medical Examiners, Philadelphia, Pennsylvania; firstname.lastname@example.org.