Secondary Logo

Journal Logo

Invited Commentaries

Reporting a Pass/Fail Outcome for USMLE Step 1: Consequences and Challenges for International Medical Graduates

Boulet, John R. PhD, MA; Pinsky, William W. MD

Author Information
doi: 10.1097/ACM.0000000000003534
  • Free

Abstract

The transition of United States Medical Licensing Examination (USMLE) Step 1 (basic science) to a pass/fail exam was announced by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) on February 12, 2020, following extensive discussion by stakeholders.1 This policy will not be implemented before January 2022, but the announcement has caused a stir within the medical education community. In this Invited Commentary, we will briefly review some of the potential positive and negative consequences of this decision and discuss the challenges international medical graduates (IMGs)—a group that represents approximately one-quarter of residents and practicing physicians in the United States—could face considering this and other future policy changes. We will also share our thoughts on how changes in the residency selection process could help all medical school graduates, including IMGs, better navigate the path to graduate medical education (GME).

Consequences of Step 1 Pass/Fail and Other Policy Changes

Historically, USMLE Step 1 numerical scores have been used by program directors in the United States to decide which applicants to interview as part of the residency selection process. While limited evidence exists to suggest that Step 1 scores predict residency performance, other than on similar types of “knowledge-based” assessments,2 the large numbers of applications submitted for individual positions and by individual applicants—especially IMGs—have forced many programs to use Step 1 scores as an initial screening mechanism. For internal medicine residency programs beginning in 2019–2020, for example, non-U.S.-citizen IMGs (non-US-IMGs) submitted an average of 98 applications; at the same time, graduates of U.S. MD-granting medical schools submitted an average of 31 applications, up from 26 in 2014–2015.3 Although various organizations, including the Association of American Medical Colleges (AAMC) and the National Resident Matching Program (NRMP), are providing much-needed information to encourage students and graduates to make more informed residency application decisions, program directors will still need to sift through numerous applications after Step 1 reporting changes to a pass/fail outcome. Screening candidates based on Step 1 scores, whether appropriate or not, has been one way to relieve the burden.

The move to pass/fail reporting on Step 1 will have positive and negative consequences. Importantly, these may be quite different for IMGs than for graduates of U.S. medical schools. IMGs have a much more difficult time securing residency slots, as reflected in their lower NRMP Match rates.4 Those who wish to obtain GME positions in the United States must find ways to distinguish themselves. Step 1 scores currently serve this purpose.

The Educational Commission for Foreign Medical Graduates (ECFMG), both through our operational programs and our foundation (Foundation for Advancement of International Medical Education and Research), has direct contact with IMGs around the world. In our travels, students and graduates of international medical schools commonly ask us, “How can I improve my chances of getting a residency position in the United States?” Our answer has been that while many strategies may be helpful—including the completion of U.S.-based observerships, publishing research studies, networking, and applying to programs and specialties that have a track record of accepting IMGs—USMLE scores, primarily Step 1 scores, often determine whether applicants are offered an interview. With the change of Step 1 to pass/fail, it will no longer be advisable to tell these students and graduates to do everything they can to achieve a high Step 1 score. So, what should we tell them, if anything? The answer will depend on a variety of factors, including future changes to both ECFMG certification requirements and USMLE policies.

To be eligible for ECFMG certification, a prerequisite for IMG entry into residency programs accredited by the Accreditation Council for Graduate Medical Education, IMGs must (a) have their medical school credentials verified and (b) pass Step 1, USMLE Step 2 Clinical Knowledge (CK), and USMLE Step 2 Clinical Skills (CS).5 Taking and passing these examinations can be quite expensive. For IMGs, the fee for the Step 2 CS exam alone is $1,600; this does not include costs associated with travel and lodging.6 As of 2023, to be eligible to take any of these exams, IMGs will be required to be attending or to be a graduate of a medical school program accredited by an agency recognized by the World Federation for Medical Education (WFME).7 These program and exam requirements, combined with the fierce competition for residency positions, present challenges for IMGs.

Adding to these challenges, 2 other future USMLE policy changes were announced along with the move to a Step 1 pass/fail outcome.8,9 First, no earlier than January 1, 2021, the total number of attempts an examinee may take per Step or Step Component will be reduced from 6 to 4. While this is unlikely to affect many IMGs, it sends a subtle message that obtaining an ECFMG certificate will be more difficult in the future. Second, no earlier than March 1, 2021, a prerequisite for taking Step 2 CS will be instituted: Examinees will be required to pass Step 1 to be eligible to take Step 2 CS.

All these policy changes could impact IMGs’ motivation to take the exams required for ECFMG certification and make them think twice about attempting to secure residency positions in the United States. First, they will need to pass Step 1 to be eligible to take Step 2 CS. Even if they pass Step 1, they will not get a score; should program directors require a standardized score, they will be pushed to take Step 2 CK. IMGs may also choose to take Step 2 CK first and abandon their hopes of securing a residency position if they do not achieve a high score. The addition of the ECFMG’s 2023 accreditation requirement will effectively preclude some students from taking any Step exams.

Changing Step 1 to a pass/fail exam is likely to have different impacts on specific groups of IMGs. For all IMGs, the move to pass/fail may relieve the profound stress of having to secure a high Step 1 score to get a choice residency position, but that anxiety may be transferred to achieving a high Step 2 CK score. For U.S.-citizen IMGs (US-IMGs), most of whom attend 4-year medical schools with curricula like those of U.S. schools, the transition to pass/fail on Step 1 will likely mirror that for U.S. medical students. If there is a push by residency programs for Step 2 CK scores, and more of an emphasis on clinical science, the institutions that host large numbers of US-IMGs could modify their curricula (e.g., integrate basic and clinical sciences early in the curriculum) and rework their internal rules regarding exam eligibility and timing.

For non-US-IMGs, especially those in 6-year (or longer) programs, the transition may not be as easy. Often in these programs the basic science part of the curriculum is entrenched in the first few years of the program. While these medical schools could change their curricula to emphasize more clinical components earlier, they may not be motivated to do so for the sake of a few students who might wish to take Step 2 CK earlier in their studies. Non-US-IMGs could still take Step 1 early in their studies, hopefully pass, and then be eligible to take Step 2 CS. However, without some other metric by which to differentiate themselves from other residency applicants, they may need a Step 2 CK score. To prepare for Step 2 CK and be successful, they will need appropriate clinical experience, something that may only be possible later in their school’s curriculum.

Navigating the Path to GME

We would be remiss if we did not discuss ways to help all students and graduates navigate the licensure and transition-to-GME processes. While moving Step 1 to pass/fail should have some beneficial effects, such as discouraging all students from neglecting coursework early in medical school to cram for this exam, there need to be opportunities for individuals, especially IMGs, to differentiate themselves as residency applicants. As discussed above, moving to use of Step 2 CK scores would not be easy for some students and could simply shift anxiety to a different exam.

Some IMGs may be identified for residency interviews based on the international reputation of their medical school and faculty. Unfortunately, this reputation says little about individual applicants’ abilities. As part of the ECFMG’s 2023 initiative, the ECFMG will provide a list of international medical school programs accredited by a WFME-recognized agency by the end of 2020. This list should be helpful to program directors over the next 2 years as they make decisions about whom to interview.

However, without more structured and reliable holistic review of applications, the residency selection process will continue to be challenging for IMGs. The Invitational Conference on USMLE Scoring—convened by the AAMC, American Medical Association, ECFMG, FSMB, and NBME—brought together many stakeholders to discuss student well-being and issues with USMLE scoring, which led to moving Step 1 to pass/fail. These discussions also recognized that the transition from undergraduate medical education (UME) to GME needs to be studied and, ultimately, revamped. This broader system-wide review of the UME–GME transition is being conducted by groups such as the Coalition for Physician Accountability10 and is likely to spur innovations in assessment, portfolio management, credentialing, and selection methodologies. For IMGs, implementing alternate standardized assessments that provide scores or rankings, taken outside the licensing examination sequence, may provide means for individuals to differentiate themselves. An electronic portfolio of achievements, including relevant experience, additional courses taken, and other credentials earned, could also be helpful. Finally, improvements in selection methodologies, including the use of artificial intelligence to screen and assess applicants based on information that is not easily quantified and aggregated (e.g., recorded interviews, letters of reference, personal statements, social media presence), may also hold some promise for the future.

In Sum

The change to Step 1 pass/fail reporting, combined with other future policy changes, will affect when and how IMGs navigate the examinations required for licensure in the United States. These changes may also influence IMGs’ decisions about whether to come to the United States for residency training. While the positive and negative consequences of changing Step 1 reporting can be debated, the deliberate process employed to inform this decision helped reveal other issues, such as with the UME–GME transition. Improving the residency selection process will help IMGs continue to be an important and productive part of the U.S. physician workforce.

References

1. United States Medical Licensing Examination. Change to pass/fail score reporting for Step 1. https://www.usmle.org/incus. Accessed April 30, 2020.
2. Sutton E, Richardson JD, Ziegler C, Bond J, Burke-Poole M, McMasters KM. Is USMLE Step 1 score a valid predictor of success in surgical residency? Am J Surg. 2014;208:1029–1034.
3. Association of American Medical Colleges. Table C-3: Residency applicants to ACGME-accredited programs by specialty and medical school type, 2019-2020. https://www.aamc.org/system/files/2019-12/2019_FACTS_Table_C-3.pdf. Accessed April 30, 2020.
4. National Resident Matching Program. Results and Data: 2020 Main Residency Match. 2020.Washington, DC: National Resident Matching Program;
5. Educational Commission for Foreign Medical Graduates. Requirements for certification. https://www.ecfmg.org/certification/requirements-for-certification.html. Updated August 27,2019. Accessed April 30, 2020.
6. Educational Commission for Foreign Medical Graduates. Fees and payment. https://www.ecfmg.org/fees/index.html. Accessed May 25, 2020.
7. Shiffer CD, Boulet JR, Cover LL, Pinsky WW. Advancing the quality of medical education worldwide: ECFMG’s 2023 medical school accreditation requirement. J Med Regul. 2019;105:8–16.
8. United States Medical Licensing Examination. Attempt limit: Change to the USMLE attempt limit policy. https://www.usmle.org/attemptlimit.html. Accessed April 30, 2020.
9. United States Medical Licensing Examination. Step 2 CS prerequisite: Exam prerequisite for Step 2 Clinical Skills. https://www.usmle.org/prerequisiteforCSexam.html. Accessed April 30, 2020.
10. Chaudhry HJ, Kirch DG, Nasca TJ, et al. Navigating tumultuous change in the medical profession: The coalition for physician accountability. Acad Med. 2019;94:1103–1107.
Copyright © 2020 by the Association of American Medical Colleges