Secondary Logo

Journal Logo

Perspectives

High-Value Interviewing: A Call for Quality Improvement in the Match Process

Frush, Benjamin W. MD, MA; Byerley, Julie MD, MPH

Author Information
doi: 10.1097/ACM.0000000000002545
  • Free

Abstract

The Match process is a seminal experience for fourth-year medical students and an integral component of sustaining successful residency training programs. However, this process has shortcomings that have received much attention recently, with a focus on ways to modernize practices to improve its effectiveness.1–4 With regard to residency interviews, there are four areas in which there is significant room for improvement: interview offers and scheduling,5 the number of interviews completed versus needed to match,6,7 postinterview communication (PIC),8,9 and the significant financial expense entailed.10,11

Although there have been some proposals for addressing these aspects of the Match process,1,4 none thus far have analyzed the problems and proposed solutions using the framework and language of quality improvement (QI). QI, the “systemic, formal approach to the analysis of practice performance and efforts to improve performance,”12 is becoming increasingly entrenched as the paradigm for problem solving in contemporary medical education.13 Closely aligned with QI is the concept of high-value care, which aims to “improve health, avoid harms, and eliminate wasteful practices”14 in order to provide “the best care for the patient, with the optimal result for the circumstances, delivered at the right price.”15 The high-value care approach champions parsimony and evidence-based decision making aimed at achieving better outcomes at lower costs to patients and the system.

Given that QI efforts oriented toward high-quality outcomes direct much problem solving and decision making in medicine, we find it curious that a QI approach is not as readily applied to solve problems outside the clinical setting. To this end, we contend that applying a QI approach to solve residency interview problems in the Match process is warranted to create an experience that both applicants and program directors would consider high value. Such a QI initiative would work to mitigate the significant financial expense, anxiety, and overall inefficiency of the current interview practices and thereby improve the quality of the Match process, primarily for applicants but also for residency programs. We propose that this high-value interview process would address issues in the four previously mentioned areas in the following ways: by creating a discrete interview offer week to streamline interview offers and scheduling, by implementing an interview cap to limit the number of interviews applicants may schedule, by enforcing a PIC moratorium for both students and programs, and by affording financial benefits to applicants and programs through adoption of these proposals.

Offer Week

Residency programs begin receiving Electronic Residency Application Service applications on September 15, and applicants may hear from programs at any point after October 1, when programs receive the medical student performance evaluations provided by applicants’ medical schools.16 Because there is no standardized schedule that indicates when applicants will hear from programs, students must vigilantly check their e-mail for interview offers—which are frequently time sensitive—and attempt to secure and schedule interviews in an ad hoc fashion. Programs likewise must navigate the complex task of extending offers to applicants and waiting to hear from them before deciding to extend offers to a second round of applicants.

To address this disorganized process for interview offers, we propose establishing a firm date on which programs should be expected to extend interview invitations to students and a date by which students should be expected to respond and confirm interview dates. Given that applicants sometimes decline interview offers and programs subsequently extend offers to different applicants, a standard window for all interview offers should be established. We propose a five-day period: Initial offers would be extended on Monday and confirmed by Wednesday, with subsequent offers extended on Wednesday and confirmed by Friday. This period could take the form of an “offer week,” during which applicants know they will be receiving and deciding on interview offers, and programs know they will be completing all of their interview scheduling.

Setting a specific five-day period for all interview offers would benefit applicants by eliminating their anxiety over missing a crucial e-mail while sleeping, rounding, or otherwise being away from their cell phones or computers. In our experience, anxiety due to the unpredictable timing of interview offers is widespread and acts as a distraction from valuable clinical experiences. In the current process, there is a sense of urgency for students to answer e-mails immediately to secure preferable interview spots—and sometimes to secure interviews at all, because some programs extend more offers than they have interview spots available.17 Concentration of all interview offers in this five-day period would add an important element of predictability to this process for students, and course directors and other school administrators ideally would afford students more flexibility during this time period. Allowing a 48-hour window (as opposed to a 24-hour window) for responding to first- and second-round interview offers would provide students on challenging clinical rotations and long call shifts with ample time to respond to offers and would lessen distractions. This definitive interview offer window would additionally streamline interview scheduling for programs by addressing the challenges related to juggling offers in an inefficient manner, which sometimes results in interview spots ultimately being unfilled or overextended by wait lists.

Although dedicating time and energy to concentrate all programs’ efforts to offer and schedule interviews would require a good deal of coordination, it would ultimately benefit both applicants and programs. By conferring predictability to this ad hoc system, the offer week would improve the quality of the interview scheduling process for both programs and applicants by reducing stress and inefficiency.

Interview Cap

There is a prevailing sentiment among students that it behooves them to secure and complete the maximum number of interviews possible to ensure a match because of concerns about the scarcity of available residency positions.4 The significant investment of time, resources, and energy into the interview process is therefore viewed, at least implicitly, as a necessary cost. However, this sentiment contrasts starkly with the judicious, evidence-based approach students learn to practice when approaching clinical problems. Just as students are not taught to immediately prescribe antibiotics for every potentially bacterial upper respiratory infection to err on the safe side, students should not be encouraged to apply to and interview at more residency programs simply to play it safe. Therefore, to address the pressure to secure as many interviews as possible and to compile long rank-order lists—as well as the consequent expense entailed—we propose placing a cap on the number of interviews an applicant is allowed to schedule. This number of interviews should be informed by evidence rather than anxiety.

Data compiled by the National Resident Matching Program (NRMP) attest to the high numbers of interviews completed by applicants. The average rank-order list for matched U.S. senior applicants in the 2018 Match was 12.49 programs, a number that increases yearly, despite the fact that most applicants match to one of their top three choices.6 However, NRMP specialty-specific information indicates that for most specialties, the number of ranked programs at which an applicant achieves a high level of probability of matching (e.g., > 90%) tends to be between 6 and 10 programs (with the exception of competitive subspecialties), with little relative benefit added thereafter.7 The discrepancy between these numbers indicates that interview practice may not be following the evidence.* Implementing an interview cap set at the specialty-specific, evidence-based number that provides a high probability of matching would reduce excess interviews and expenses for both applicants and programs. In addition to making the process more efficient, it would help allocate more interviews to less competitive applicants, for whom securing these interviews could prove vital to matching, while not detracting from more highly qualified applicants’ probability of matching successfully.

PIC Moratorium

Expectations regarding PIC vary among specialties.19 The NRMP’s Match Communication Code of Conduct states that the Match process should be free from “coercion or undue or unwarranted pressure” and that program directors should not “engage in post-interview communication that is disingenuous for the purpose of influencing applicants’ ranking preferences.”20 PIC is initiated by both residency programs and applicants and has been shown to affect how applicants rank programs,21 although program directors have divergent opinions regarding its efficacy.8 Concerns about the ethics of PIC are not new,22 and evidence suggests that the information conveyed by students and sometimes by programs can be ambiguous and inaccurate.9,21,23 There have been efforts to implement guidelines regarding PIC, but such guidelines do not seem to have deterred the practice thus far.8

We propose that the most effective and ethically appropriate standard would be enforcing a policy forbidding any communication after the interview. All applicants should be afforded the same interview experience, and rank-order lists compiled by programs and applicants should be based strictly on which applicants or programs, respectively, they feel would be the best matches. Indeed, the successful operation of the Match algorithm depends on these assumptions,4 not on concerns about reciprocal ranking.

Importantly, this moratorium should apply to applicants as well as programs. PIC is frequently initiated by students, particularly to inform programs that they will be ranking them highly,22 which may place program directors in an uncomfortable position. Further, two of the primary reasons program directors report for engaging in PIC are that they perceive that applicants expect to hear from them and for purposes of cordiality.8 Thus, a PIC moratorium that applies to applicants and programs would likely gain traction once applicants realized that programs would no longer reach out or respond following interviews.

While the PIC moratorium would not necessarily reduce financial expenses, it would enhance the quality of the Match process by emphasizing the noncoercive nature of the Match and ensuring the integrity of the Match algorithm. The moratorium would, moreover, eliminate a practice that some program directors view as unhelpful and counterproductive8 and that is a source of stress for both program directors and applicants.21

Financial Benefits

Finally, a theme running through the issues outlined above is the high cost for applicants and programs, which is one of the chief downsides of the current interview process.10,11,24 Implementing the changes we suggest would decrease costs on both sides of the process.

Establishing a specific timeline for interview offers and scheduling in the form of an offer week would allow students to organize their interviews geographically. In the current ad hoc model, students accept the offers extended regardless of geography and without knowing what other offers might come. Some students will travel across the country many times because of the rush to take whatever is offered. Although a certain level of travel inefficiency may be unavoidable, adding predictability to the timing of interview offers should enhance the ability of applicants to schedule interviews in a more organized (and thus less expensive) fashion.

Implementing an interview cap would also decrease costs to applicants, initially by (potentially) reducing the number of programs to which they apply, and subsequently by reducing both travel and lodging expenses. It would also financially benefit residency programs and their sponsoring institutions. The costs of lodging subsidies, food, and gifts for applicants add up quickly and would be best spent on applicants seriously considering the program. The physician time that is spent interviewing applicants instead of working in clinical settings or advancing the program’s research and education priorities also comes at a significant cost. Although these investments are important, most programs interview many times the number of applicants needed to fill their positions. We argue that filling positions can be accomplished within a smaller interview pool through implementation of an interview cap.

The potential financial benefits of enforcing a PIC moratorium are more difficult to quantify because this proposal aims to improve the quality of the Match process and experience rather than reduce monetary costs per se. However, given the anxiety associated with PIC and the well-established link between mental health and financial burden in medicine, there is potentially an indirect financial benefit to this proposal.25,26

Conclusions

Although our proposals for creating a high-value interview process may spark disagreement, we feel that they contribute to the robust discussions surrounding interview practices in the Match.4 Unlike many of the deeply complex issues in medical education, the issues associated with residency interviews potentially lend themselves to (relatively) straightforward remedies. Our proposals represent starting points for this discussion.

We acknowledge that there are downsides to our proposals. Each decreases the freedom and flexibility that students and programs have in the Match process, similar to the way that some systemization is put in place in clinical settings to improve quality and value in health care delivery at the expense of individual patient or practitioner autonomy. However, consonant with the ethos of QI, we believe that these decreased opportunities for individualization of the Match process are justified in the pursuit of value and quality in medical education.

Finally, we believe that our proposals can serve a pedagogical purpose by providing examples of how QI methodology may be applied creatively to problems outside the clinical setting. QI efforts focused on high-value interviewing should approach the problems faced in the Match process by applicants and residency programs using the same methods students are taught for approaching problems faced in the hospital by the patients and institutions they are learning to serve. Our proposals also demonstrate that QI efforts should be directed not only at improving patient care but also at improving student and physician experiences. We believe that students and the programs that will employ them in the future deserve such efforts.

Acknowledgments:

The authors would like to acknowledge students and colleagues at the University of North Carolina (UNC) School of Medicine for helpful conversations in developing the thoughts presented in this article. The UNC School of Medicine is a participant in the American Medical Association (AMA) Accelerating Change in Medical Education Consortium. Some of these ideas were developed in discussion with that group.

*The number of programs ranked is not always the same as the number of interviews completed because applicants may choose not to rank programs at which they have interviewed. There is a paucity of information as to how discrepant these numbers are, although the median number of interviews attended and programs ranked was identical for matched U.S. seniors in the 2017 Match.18

References

1. Hariton E, Bortoletto P, Ayogu N. Residency interviews in the 21st century. J Grad Med Educ. 2016;8:322–324.
2. Wills C, Hern HG Jr, Alter H. Residency applicants prefer online system for scheduling interviews. West J Emerg Med. 2015;16:352–354.
3. Deloney LA, Perrot LJ, Lensing SY, Jambhekar K. Radiology resident recruitment: A study of the impact of web-based information and interview day activities. Acad Radiol. 2014;21:931–937.
4. Ray C, Bishop SE, Dow AW. Rethinking the Match: A proposal for modern matchmaking. Acad Med. 2018;93:45–47.
5. Hern HG Jr, Wills CP, Alter HJ, et al. Residency applicant preferences of online systems for scheduling interviews. J Grad Med Educ. 2016;8:759–762.
6. National Resident Matching Program. Impact of length of rank order list on Match results: 2002–2018 Main Residency Match. http://www.nrmp.org/wp-content/uploads/2018/03/Impact-of-Length-of-Rank-Order-List-on-Match-Results-2018-Main-Match.pdf. Published March 2018. Accessed November 21, 2018.
7. National Resident Matching Program. Charting outcomes in the Match: US allopathic seniors: Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2018 Main Residency Match. 2nd ed. http://www.nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-in-the-Match-2018-Seniors.pdf. Published July 2018. Accessed November 21, 2018.
8. Chacko KM, Reddy S, Kisielewski M, Call S, Willett LL, Chaudhry S. Postinterview communications: Two surveys of internal medicine residency program directors before and after guideline implementation. Acad Med. 2018;93:1367–1373.
9. Grimm LJ, Avery CS, Maxfield CM. Residency postinterview communications: More harm than good? J Grad Med Educ. 2016;8:7–9.
10. Blackshaw AM, Watson SC, Bush JS. The cost and burden of the residency match in emergency medicine. West J Emerg Med. 2017;18:169–173.
11. Agarwal N, Choi PA, Okonkwo DO, Barrow DL, Friedlander RM. Financial burden associated with the residency match in neurological surgery. J Neurosurg. 2017;126:184–190.
12. American Academy of Family Physicians. Basics of quality improvement. https://www.aafp.org/practice-management/improvement/basics.html. Accessed November 8, 2018.
13. Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: Current state and future directions. Med Educ. 2012;46:107–119.
14. American College of Physicians. Clinical information: High value care. https://www.acponline.org/clinical-information/high-value-care. Published 2017. Accessed January 2, 2018.
15. Smith M, Saunders R, Stuckhardt L, et al.; Committee on the Learning Health Care System in America; Institute of Medicine; Chapter 8: Achieving and rewarding high value care. In: Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2013. Washington, DC: National Academies Press; https://www.ncbi.nlm.nih.gov/books/NBK207237/?utm_medium=referral&utm_source=r360. Accessed November 12, 2018.
16. Association of American Medical Colleges. Match timeline. https://www.aamc.org/cim/residency/application/applying/337902/matchtimeline.html. Accessed November 23, 2018.
17. Ramkumar PN, Navarro SM, Chughtai M, Haeberle HS, Taylor SA, Mont MA. The orthopaedic surgery residency application process: An analysis of the applicant experience. J Am Acad Orthop Surg. 2018;26:537–544.
18. National Resident Matching Program. Results of the 2017 NRMP applicant survey by preferred specialty and applicant type. http://www.nrmp.org/wp-content/uploads/2017/09/Applicant-Survey-Report-2017.pdf. Published September 2017. Accessed November 21, 2018.
19. Brooks JT, Reidler JS, Jain A, LaPorte DM, Sterling RS. Post-interview communication during application to orthopaedic surgery residency programs. J Bone Joint Surg Am. 2016;98:e84.
20. National Resident Matching Program. Match communication code of conduct. http://www.nrmp.org/communication-code-of-conduct. Accessed November 12, 2018.
21. Jena AB, Arora VM, Hauer KE, et al. The prevalence and nature of postinterview communications between residency programs and applicants during the Match. Acad Med. 2012;87:1434–1442.
22. Carek PJ, Anderson KD. Residency selection process and the Match: Does anyone believe anybody? JAMA. 2001;285:2784–2785.
23. Grimm LJ, Desser TS, Bailey JE, Maxfield CM. Applicant to residency program translation guide. J Am Coll Radiol. 2015;12:622–623.
24. Polacco MA, Lally J, Walls A, Harrold LR, Malekzadeh S, Chen EY. Digging into debt: The financial burden associated with the otolaryngology Match. Otolaryngol Head Neck Surg. 2017;156:1091–1096.
25. Berg S. At Stanford, physician burnout costs at least $7.75 million a year. AMA Wire. https://wire.ama-assn.org/life-career/stanford-physician-burnout-costs-least-775-million-year. Published November 17, 2017. Accessed November 12, 2018.
26. Shanafelt T, Goh J, Sinsky C. The business case for investing in physician well-being. JAMA Intern Med. 2017;177:1826–1832.
Copyright © 2018 by the Association of American Medical Colleges