Secondary Logo

Journal Logo

From the Editor

The USMLE Step 1 Examination: Can Pass/Fail Make the Grade?

West, Colin P. MD, PhD; Durning, Steven J. MD, PhD; O’Brien, Bridget C. PhD; Coverdale, John H. MD, MEd; Roberts, Laura Weiss MD, MA

Author Information
doi: 10.1097/ACM.0000000000003537
  • Free

On February 12, 2020, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) announced that the United States Medical Licensing Examination (USMLE) Step 1 will transition to pass/fail results reporting in 2022. This examination is one of several tests from the USMLE that must be passed to practice medicine in the United States. Performance on Step 1 is one of many factors considered in the selection of candidates for residency programs and is heavily weighted in many medical specialties.1 The FSMB and the NBME, co-sponsors of the examination, view this change in reporting test performance as “an important first step toward facilitating broader, system-wide changes to improve the transition from undergraduate to graduate medical education.”2 Understandably, such a significant decision has sparked a robust debate about its merits among many stakeholders in medical education, including students, medical school leaders, residency program directors, licensing organizations, and members of the international community. As the editors of Academic Medicine, we see an important opportunity to support and encourage ongoing dialogue among stakeholders. To begin this process, we asked thought leaders in our community to provide their perspectives on the change in Step 1 results reporting.

In this editorial we highlight key themes from those leaders’ Invited Commentaries,3–9 published in this issue, regarding the advantages, disadvantages, and other potential consequences of transitioning Step 1 to pass/fail results reporting. We also suggest efforts necessary to further optimize learning environments after pass/fail results reporting is implemented for Step 1. Our goal in this summary of the Invited Commentaries, along with other articles in Academic Medicine’s collection of reports on the Step 1 transition to pass/fail results reporting, is to help readers grapple with this policy change and its implications.

Key Themes

Below we outline three key themes raised in the Invited Commentaries. The themes are intended to frame ongoing discussion in the academic medicine community; they are not listed in order of priority and are by no means comprehensive.

A catalyst for improved assessment and curricular engagement

The shift to pass/fail results reporting of Step 1 may serve as a catalyst to drive educators toward an improved assessment system in medical school. The shift in results reporting presents an opportunity to diversify metrics and tools used for resident selection, but achieving any desired improvements will require leaning into rather than shying away from the change. The potential diversification of metrics used for resident selection is in line with more “holistic” perspectives on learner performance. Factors in a holistic review of candidates might include interpersonal and communication skills, experiences in learning about research methods and processes, commitment to wellness, demonstration of professionalism, leadership activities, and engagement in local community needs, in addition to tests of medical knowledge. All of these experiences and skills are widely seen to have value when judging the merits of a residency candidate.

Adopting new assessment approaches will come with inevitable challenges. Program directors already struggle with how best to compare applicants from different programs, particularly in the absence of medical school performance data providing robust predictive validity. Indeed, many program directors have argued for the need for a national standard for selecting candidates. In 2020, for instance, nearly 45,000 applicants applied for roughly 37,000 spots through the National Resident Matching Program, and only about half of the applicants were senior students graduating from U.S. medical schools.10 Without a national scored examination, students from more highly ranked institutions or elite universities with name recognition may have a competitive advantage in residency selection.11 In addition, with Step 1 as a pass/fail examination, the results of other national examinations may be more heavily weighted in future resident selection processes.5,12 Finally, novel focused and holistic assessments may be seen as necessary, but new evaluative approaches can be complicated and expensive to administer, and might be less psychometrically sound.13 Moreover, integration of novel evaluative approaches will undoubtedly add to the stress and workload experienced by residency program directors, faculty, and administrative staff, so creating efficient processes to inform novel evaluative approaches should be a priority.

A reduced focus on achieving the highest scores on Step 1 may promote curricular engagement by faculty and students. It has been argued14 that the Step 1 examination was not designed to be used for residency selection and has had the inadvertent impact of limiting curriculum change. Step 1’s position as a residency gateway examination—a “high-stakes” test that medical students know they need to not only pass but also excel at to be competitive for many residencies—can stifle faculty efforts to revise medical school coursework. An orientation toward preparing students for Step 1 performance can diminish curricular flexibility in designing unique educational experiences aligned with the mission of the medical school.6 The weight of achieving top test scores on Step 1 can detract learners from fully engaging with curricula at their home institutions.15,16 Being overly focused on numeric scores on a national exam in the middle of medical school may also distract attention from other important components of clinical training, such as physical examination skills, critical thinking, and community engagement.17 Medical students may prioritize activities that strengthen test-taking techniques over activities that contribute to their overall potential as future physicians and leaders, such as exploration of opportunities to combine their medical education with scientific, public health, policy, or community-based interests.

An opportunity to reduce disparities in medical education and assessment

Pass/fail reporting may lessen education and assessment disparities, but overall, the impact of the change to pass/fail reporting on workforce diversity is unclear. Schools that focus primarily on selection of the narrow group of highest achievers on standardized examinations may advance an unfair system and miss opportunities to broaden and build a more inclusive and representative future physician workforce.18

Proponents of the Step 1 transition to pass/fail results reporting maintain that the change will enable a more holistic approach that may help to “level the playing field” for potentially disadvantaged applicants to the most competitive residency programs. The reporting of results as pass/fail rather than as a numeric score may help to reduce the adverse impact of test bias, which has been shown to be an unfair structural barrier to achievement experienced by some individuals who identify as belonging to underrepresented minority groups.18 Students who lack access to commercial examination aids (e.g., because of the financial burden) may be less disadvantaged than in the past with the change to pass/fail results reporting.15

Transitioning to a pass/fail scoring system removes any possibility that students will be ranked according to their Step 1 examination scores. This transition may decrease students’ sense of competitiveness on the Step 1 exam, but could exacerbate certain disparities. The Step 1 score provides a familiar standardized metric of residency candidates’ qualifications. Without a USMLE Step 1 score, graduates from less competitive medical schools and international medical schools may be disadvantaged. International medical graduates contribute greatly to high-quality medical care in the United States through their diversity, talents, and international perspectives,19 but their medical school training environments may be less well understood. With the transition to pass/fail results reporting, more graduates might be selected for competitive residencies from competitive or prestigious medical schools on the basis of the reputation or familiarity of these schools.11

The need for collaboration across the medical education continuum

Collaboration across every stage of medical education will be required to effectively balance the benefits and risks of evolving assessment measures and criteria. The change in results reporting for Step 1 should be viewed as one thread in a larger tapestry of change, with attendant extensions to assessment throughout the medical education continuum. Ongoing input from all stakeholder groups across this continuum will be necessary in all efforts to foster and sustain a capable, resilient physician workforce.

For learners in particular, understanding the ripple effect of pass/fail results reporting for the Step 1 exam across the educational experience will be critical. Striving to attain the highest Step 1 examination score can induce tremendous anxiety. Previous work20–22 has demonstrated that moving tests from graded to pass/fail results reporting can reduce anxiety. We do not know, however, whether moving a national examination that represents a residency selection standard to a pass/fail format will actually increase or decrease test anxiety. Learners will be allowed fewer attempts to pass the Step 1 exam, which might increase anxiety for some. Without a Step 1 examination score, it becomes less clear how residencies will review and rank large volumes of applications. As an unintended consequence, the USMLE Step 2 clinical knowledge and clinical skills examinations may become even more important, and thus more anxiety provoking, for students. Some6 suggest that, at the present time, becoming and being a physician involves test taking (e.g., for board certification) and that we do not serve our students well by sidestepping this skill.

Addressing all possibilities for evaluation within the context of the benefits and risks of revised assessment approaches, including the change to Step 1, will require thoughtful consideration and response from every level of medical education and every participant group. Across the full spectrum of medical education, we must remain focused on the critical need to optimize alignment of assessment with the overarching goals of medical education. We cannot simply replace imperfect tools with poorly validated approaches, trading one set of problems for another.

Moving Forward

With these themes in mind, we outline key principles that we believe must be followed if our learning environments are to move forward.

Continuing the conversation

All who are affected by the change in Step 1 results reporting need to be able to voice their views about the changes and to share findings with others. The collection of articles in this issue is only a beginning. The dialogue needs to extend across the continuum of medical education (medical school, residency, fellowship, and beyond), and mechanisms to facilitate this ongoing dialogue should continue to be developed.

Broadening stakeholder participation

All voices must be “in the room”—including those of medical students, faculty, residency leadership, education-focused and assessment-focused organizations, representatives from the public, and other stakeholders—to clarify and deepen understanding of the Step 1 results reporting change and its implications. International voices are also needed to share both learner and leadership perspectives across differing education systems. A substantive discussion of the role of testing in enabling medical education to develop a capable, resilient health care workforce is necessary. Further discussion could lead to ideas for improvements to assessment and licensing systems worldwide.

The need for innovation

The change to pass/fail results reporting offers an opportunity to modernize the medical school curriculum. Medical schools could share observed benefits as well as monitor for additional unexpected consequences. Broadening funding streams for education innovations (e.g., from the American Medical Association) could serve as a further catalyst for change.

Curricular engagement

Ongoing efforts should support learners’ and medical schools’ engagement in diverse initiatives. Innovative curricula, efforts to promote professionalism and leadership, research, identifying and addressing community needs, and other activities might be afforded greater prominence given a reduced emphasis on Step 1 scores.

Opportunities and Challenges Ahead

The best possible evidence will be necessary to maximize the potential for benefit and minimize the risk of negative consequences from the transition to pass/fail results reporting.23 The FSMB and NBME sought to strengthen the path for learners moving from undergraduate to graduate medical education by making the change in results reporting for Step 1. Will their intent be fulfilled? Will the change in results reporting broaden and foster representativeness of our workforce? Lessen the anxiety of learners? Shift the weight to other “high-stakes” national examinations with numeric scoring? Lead to creative curricular innovations and foster faculty and learner engagement?

The relationship between assessment approaches and educational goals warrants careful inquiry. We encourage efforts to promote multi-institutional research to guide evidence-informed decisions on assessment. Such research might include a national evaluation of stress levels in medical education related to Step 1’s transition to pass/fail results reporting. Evidence to date suggests that objective academic performance is not adversely affected by a pass/fail system,15,16 but assessments of how knowledge in the clinical sciences is affected, if at all, and whether any systematic differences in student performance develop after the transition, will be needed. There must also be research on how residency selection decisions occur and the implications of those decisions for the capability and resilience of the physician workforce. Research on medical education and workforce development necessitates the creation and expansion of related national grant opportunities, evermore a topic of importance.

The transition to pass/fail results reporting for USMLE Step 1 presents both opportunities and challenges. Exciting prospects include enhanced assessment processes, reduced disparities in medical education, and increased collaboration across the medical education continuum. Broad stakeholder participation and rigorous implementation and outcome studies are necessary to fully realize these benefits.

Colin P. West, MD, PhD
Associate editor, Academic Medicine.
Steven J. Durning, MD, PhD
Deputy editor, Academic Medicine.
Bridget C. O’Brien, PhD
Associate editor, Academic Medicine.
John H. Coverdale, MD, MEd
Associate editor, Academic Medicine.
Laura Weiss Roberts, MD, MA
Editor-in-chief, Academic Medicine.

References

1. Gauer JL, Jackson JB. The association of USMLE Step 1 and Step 2 CK scores with residency match specialty and location. Med Educ Online. 2017;22:1358579.
2. United States Medical Licensing Examination. Change to pass/fail results score reporting for Step 1. https://www.usmle.org/incus. Accessed May 22, 2020.
3. Boulet JR, Pinsky WW. Reporting a pass/fail outcome for USMLE Step 1: Consequences and challenges for international medical graduates. Acad Med. 2020;95:1322–1324.
4. Kogan JR, Hauer KE. Sparking change: How a shift to Step 1 pass/fail scoring could promote the educational and catalytic effects of assessment in medical education. Acad Med. 2020;95:1315–1317.
5. Lin GL, Nwora C, Warton L. Pass/fail score reporting for USMLE Step 1: An opportunity to redefine the transition to residency together. Acad Med. 2020;95:1308–1311.
6. McDade W, Vela MB, Sánchez JP. Anticipating the impact of the USMLE Step 1 pass/fail scoring decision on underrepresented-in-medicine students. Acad Med. 2020;95:1318–1321.
7. Pershing S, Co JPT, Katznelson L. The new USMLE Step 1 paradigm: An opportunity to cultivate diversity of excellence. Acad Med. 2020;95:1325–1328.
8. Salari S, Deng F. A stepping stone toward necessary change: How the new USMLE Step 1 scoring system could affect the residency application process. Acad Med. 2020;95:1312–1314.
9. Whelan AJ. The change to pass/fail scoring for Step 1 in the context of COVID-19: Implications for the transition to residency process. Acad Med. 2020;95:1305–1307.
10. National Resident Matching Program. 2020 Main Residency Match by the numbers. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/03/2020-Match-by-the-Numbers-FINAL.pdf. Accessed May 22, 2020.
11. Crane MA, Chang HA, Azamfirei R. Medical education takes a step in the right direction. Where does that leave students? [published online ahead of print March 6, 2020]. JAMA. doi:10.1001/JAMA.2020.2950.
12. Andolsek KM. One small step for Step 1. Acad Med. 2019;94:309–313.
13. Radabaugh CL, Hawkins RE, Welcher CM, et al. Beyond the United States Medical Licensing Examination score: Assessing competence for entering residency. Acad Med. 2019;94:983–989.
14. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical Licensing Examination Step 1 scores in residency selection. Acad Med. 2016;91:12–15.
15. Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives on the “Step 1 Climate” in preclinical medical education. Acad Med. 2019;94:302–304.
16. Ronner L, Linkowski L. Online forums and the “Step 1 climate:” Perspectives from a medical student Reddit user. Acad Med. 2020;95:1329–1331.
17. Gesundheit N. A crisis of trust between U.S. medical education and the National Board of Medical Examiners. Acad Med. 2020;95:1300–1304.
18. Roberts LW. The new MCAT exam and the continuing imperative of holistic review in the selection of medical students. Acad Med. 2020;95:323–326.
19. Desai A, Hegde A, Das D. Change in reporting of USMLE Step 1 scores and potential implications for international medical graduates [published online ahead of print March 6, 2020]. JAMA. doi:10.1001/jama.2020.2956.
20. Shiralkar MT, Harris TB, Eddins-Folensbee FF, Coverdale JH. A systematic review of stress-management programs for medical students. Acad Psychiatry. 2013;37:158–164.
21. Spring L, Robillard D, Gehlbach L, Simas TA. Impact of pass/fail grading on medical students’ well-being and academic outcomes. Med Educ. 2011;45:867–877.
22. Wasson LT, Cusmano A, Meli L, et al. Association between learning environment interventions and medical student well-being: A systematic review. JAMA. 2016;316:2237–2252.
23. Katsufrakis PJ, Chaudhry HJ. Improving residency selection requires close study and better understanding of stakeholder needs. Acad Med. 2019;94:305–308.
Copyright © 2020 by the Association of American Medical Colleges