Among respondents, 65.8% (7,219/10,967) reported being asked at least one potentially illegal question during the interview process (see Table 1). Of the potentially illegal questions asked, most frequently questions were about applicants’ families; 53.3% (5,710/10,710) of respondents were asked about their marital status and 24.0% (2,592/10,798) were asked if they had children. Another related and frequently asked question involved their plans for child rearing or childbearing, with 13.8% (1,450/10,512) reporting that they had been asked such a question. Other potentially illegal questions addressed religion (982/10,368; 9.5%), ethnicity (969/10,747; 9.0%), age (758/10,740; 7.1%), gender (692/10,917; 6.3%), or sexual orientation (121/10,360; 1.2%). About one in six applicants (1,608/10,378; 15.5%) were asked about their level of commitment to the residency program. For a breakdown by specialty of the potentially illegal questions received, see Table 2.
Of those who were asked potentially illegal questions, respondents reported that an average of 5.2 (standard deviation [SD] 3.6) programs asked about marital status, 4.1 (SD 3.3) asked about parental status, 4.0 (SD 3.6) asked about ethnicity, 3.7 (SD 3.2) asked questions related to gender, 3.1 (SD 2.7) asked about age, 3.3 (SD 2.8) asked about plans to have children, 2.6 (SD 2.7) asked about religion, and 3.8 (SD 4.0) asked about sexual orientation.
Early versus late survey respondents
In comparing early versus late respondents, we found small yet statistically significant differences across some demographic charac teristics (see Table 1). We found no significant differences between the two groups with regard to receiving potentially illegal questions about age, gender, family planning, ethnicity, religion, or sexual orientation. However, we did find differences with regard to questions about marital status, parental status, and commitment to the program.
Effect of gender on likelihood of being asked potentially illegal questions
In a model that adjusted for age, race, marital status, parental status, and specialty, female respondents were significantly more likely to have received a potentially illegal question (68.7% vs. 63.4%) compared with male respondents (adjusted odds ratio of female gender [aORf] = 1.35, 95% confidence interval [CI]: 1.23–1.47, P < .001). Although female respondents were significantly more likely to be asked about their marital status or commitment to the program (see Table 3), they also experienced the most marked difference in prevalence of questions regarding gender (9.6% vs. 2.9%, aORf = 3.94, 95% CI: 3.22–4.83, P < .001) or plans to have children (17.0% vs. 10.6%, aORf = 1.89, 95% CI: 1.67–2.15, P < .001).
Overall, male respondents were less likely to be asked potentially illegal questions. Yet, female respondents were less likely to receive questions about religion (8.0% vs. 11.1%, aORf = 0.77, 95% CI: 0.67–0.89, P = .001) or sexual orientation (0.8% vs. 1.5%, aORf = 0.48, 95% CI: 0.32–0.73, P < .001).
Effect of specialty on likelihood of being asked potentially illegal questions
Among the 96.8% (10,611/10,967) of respondents who indicated their specialty, those in surgical specialties were more likely to have received a potentially illegal question compared with those in nonsurgical specialties (81.9% vs. 61.5%; see Table 2). The greatest prevalence of these questions were reported by respondents from neurosurgery (86.0%), orthopedic surgery (84.7%), obstetrics–gynecology (82.0%), and general surgery (81.4%).
The adjusted odds ratio of a surgical applicant being asked a potentially illegal question versus a nonsurgical applicant [aORs] was 2.97 (95% CI: 2.63–3.35, P < .001; see Table 3). This difference was most notable with regard to questions about gender—15.5% of surgical applicants were asked this type of question versus 3.9% of nonsurgical applicants (aORs = 5.29, 95% CI: 4.44–6.30, P < .001). The likelihood of a surgical applicant being asked a potentially illegal question increased for the following domains: age (aORs = 2.98, 95% CI: 2.49–3.57, P < .001), marital status (aORs = 2.50, 95% CI: 2.25–2.78, P < .001), parental status (aORs = 1.84, 95% CI: 1.64–2.06, P < .001), plans to have children (aORs = 2.09, 95% CI: 1.83–2.39, P < .001), ethnicity (aORs = 1.26, 95% CI: 1.05–1.51, P = .014), religion (aORs = 1.45, 95% CI: 1.24–1.71, P < .001), and commitment to the program (aORs = 1.86, 95% CI: 1.64–2.11, P < .001).
Discomfort as a result of being asked potentially illegal questions
A significant percentage of respondents reported discomfort after being asked potentially illegal questions (see Table 4). Questions about commitment to the program were most frequently associated with discomfort; 77.2% of respondents reported feeling “somewhat uncomfortable” or “very uncomfortable” when asked this question. The prevalence of discomfort for each category of question is depicted in Table 4.
When asked about gender, more respondents reporting discomfort were women (86.0%) than those reporting no discomfort (74.0%, P < .001). The same was true for those reporting discomfort when asked about marital status (60.0% vs. 49.8%, P < .001), parental status (59.7% vs. 45.5%, P < .001), or plans for child rearing (77.0% vs. 51.5%, P < .001). The results of an adjusted ordinal regression model are presented in Table 4; female respondents were significantly more likely to report discomfort in response to being asked potentially illegal questions in almost all categories.
Effect of being asked potentially illegal questions on respondents’ rank lists
A sizable percentage of respondents reported that potentially illegal questions in each category caused them to place the program lower on their rank list; however, the majority reported that their rank lists were not affected by these questions (see Table 5). Certain questions were associated with more respondents ranking the program lower (negative impact); 39.8% of respondents who were asked about their level of commitment to the program, 30.7% who were asked about gender, and 25.4% who were asked about plans for child rearing all reported ranking the program lower because of the question. Asking about sexual orientation was more polarizing; 31.9% of respondents reported that being asked about sexual orientation caused them to rank the program lower, while 25.0% reported that the same question caused them to rank the program higher.
Female gender, in an ordinal logistic regression model, was significantly associated with a greater likelihood of indicating a negative impact on rank list when asked about gender (aORf = 3.78, 95% CI: 2.16–6.64, P < .001), marital status (aORf = 2.37, 95% CI: 1.94–2.89, P < .001), parental status (aORf = 3.25, 95% CI: 2.47–4.28, P < .001), plans for child rearing (aORf = 3.57, 95% CI: 2.57–4.95, P < .001), or religion (aORf = 1.90, 95% CI: 1.34–2.69, P < .001) (see Table 5). Surgical specialty was associated with a greater likelihood of indicating a negative impact when asked about plans for child rearing (aORs = 1.58, 95% CI: 1.19–2.09, P < .001).
In this study, we surveyed 10,967 applicants to residency programs in 19 specialties. We found that two-thirds were asked at least one potentially illegal question. In the surgical specialties, that rose to 82%, and the odds of receiving a potentially illegal question as a surgical applicant compared with as a nonsurgical applicant were 2.97. In addition, women were much more likely than men to be asked questions regarding their gender, marital status, and plans to have children in the future. Finally, 16% of applicants were asked for a commitment (either verbal or otherwise) to the program during their interview; of those who received such a question, 77% reported discomfort and 40% reported ranking the program lower as a result.
These findings, along with those from previously published studies, underscore a continuing problem in residency interviewing—applicants are being asked potentially illegal questions.14–20 Whereas most studies have focused on a single specialty, our prior work explored these issues among applicants to five specialties (internal medicine, general surgery, orthopedic surgery, obstetrics–gynecology, and emergency medicine) in the 2006–2007 NRMP Match. Our current study expands on that work by including all specialties in the 2012–2013 NRMP Match to offer a broader perspective on the topic. In comparing the results of these two studies, we found that this issue continues to be a problem across medicine.
Our findings have important implications for medical educators, especially those involved in residency training programs. First, given the prevalence of potentially illegal questions, our findings support the need for standardizing interviewers’ training on the federal and state laws governing equal employment hiring practices. Programs may not be aware that some interviewers are asking these questions, and interviewers may not be aware of the risk they are taking in asking them or of the harmful consequences that may occur. Certain questions, if asked, are at best inappropriate and may be illegal if asked disproportionally of one group compared with another or if applicants’ responses influence ranking decisions. Furthermore, asking these questions, regardless of legality, may violate a moral or ethical standard among medical educators.
Second, respondents to our survey from surgical specialties, particularly women, were more likely to be asked potentially illegal questions. This finding is very concerning, especially with regard to specialties in which women are still underrepresented, both in the workforce and in training programs. Surgical programs may consider educating interviewers to avoid inappropriate questioning, to reduce discomfort, and to avoid dissuading applicants, particularly women, from wanting to join their program. However, as we have demonstrated, gender differences in receiving potentially illegal questions were not limited to surgical specialties. Across all specialties, program directors could improve their recruiting and retention of applicants by clearly delineating boundaries for interviewers, given that the residency interview process is protected by both federal and state law as a formal job interview.
Finally, despite NRMP rules, interviewers continue to ask applicants about their level of commitment to the program. In some specialties, more than 30% of respondents to our survey reported receiving this type of question. Although not potentially illegal in the same way as other questions, it is a direct and clear breach of the Match Participation Agreement set forth by the NRMP. Given applicants’ discomfort with such questions, residency programs should establish and enforce policies designed to uphold this agreement.
In the last few years, more interest has developed in standardized interviews, such as the multiple mini-interview (MMI), which can be helpful in medical school and residency selection.22 Set questions could limit applicants’ exposure to potentially illegal questions; however, these models have their own drawbacks. Hopson et al23 described a study using 71 interns as surrogates for residency applicants and noted a correlation between MMI scores and clerkship grades. Yet, the authors also noted less preference for MMI-style interviews. Soares et al24 used a mixed traditional and MMI-style interview and found that applicants vastly preferred the traditional interview over the MMI-style interviews.
Our study has some important limitations. First, our response rate was 51%. The survey may be inherently biased, and our findings might overestimate the prevalence of potentially illegal interview questions because those who identified such questions may have been more inclined to respond. However, even using the most conservative estimate of nonrespondent bias, assuming that all nonrespondents did not receive any potentially illegal questions, we estimate that 30% of the entire pool of U.S. residency applicants (7,219/21,457) reported that they had received a potentially illegal question. Prior analyses of nonrespondent bias have suggested that late respondents share characteristics with nonrespondents.21 We compared early and late survey respondents and found no significant differences between the two groups with regard to receiving potentially illegal questions about age, gender, family planning, ethnicity, religion, or sexual orientation. This suggests that nonrespondents are exposed to potentially illegal questions at a significant rate, similar to respondents.
Second, our results are based on respondents’ interpretation of the questions they were asked during interviews rather than on the actual questions they were asked. The survey made no mention of the words “illegal,” “inappropriate,” or “discrimination” and was designed to sound as neutral as possible. This may have led to a biased estimate of the prevalence of potentially illegal questions, either too low or too high, depending on respondents’ experiences. For example, in some cases respondents may have brought up the issues themselves and the interviewer only continued the line of questioning. If sensitive information is offered freely by an applicant, the interviewer may continue to discuss that information.
In summary, the prevalence of potentially illegal questions during residency interviews is high. Among respondents to our survey (including late respondents), women were more likely than men to receive questions about gender, marital status, and family planning. Applicants to surgical specialties were more likely to report being asked potentially illegal questions in all question categories. Applicants often reported discomfort at being asked such questions and reported ranking a program lower as a result of a question. Educators should recognize that such questions have a negative impact on applicants and consider outreach to interviewers to provide information regarding acceptable interview procedures. A formal interview code of conduct could address these issues within the difficult process of selecting future medical practitioners.
Acknowledgments: The authors wish to thank Ms. Moira Edwards and Ms. Renee Overton of the Association of American Medical Colleges and the Electronic Residency Application Service for their invaluable help and Michael Ward, JD, for employment law opinions.
1. Kasuri v. St. Elizabeth Hospital Medical Center, 897 F.2d, 845 (6th Cir 1990).
2. Civil Rights Act of 1964 § 7, 42 U.S.C. § 2000e et seq (1964).
3. Americans with Disabilities Act of 1990, Pub. L. No. 101336104 Stat. 328 (1990).
4. Age Discrimination in Employment Act, 29 U.S.C. § 621 et seq.
5. California Fair Employment and Housing Act, 1290012996.
6. Odima v. Westin Tucson Hotel, 53 F.3d 1481, 1494 (9th Cir. 1995).
7. Stout v. Potter, 276 F.3d 1118, 1121 (9th Cir. 2002).
8. U.S. Department of Labor. Employment applications. 29 CFR. §1625.5.
9. U.S. Department of Labor. Guidelines on discrimination because of religion. 29 CFR. §1605.1–1605.3.
10. California Government Code Section 12940(d).
11. California Government Code Section 7287.3(b)(1).
12. U.S. Department of Labor. Discrimination against married women. 29 CFR. §1604.4.
13. Phillips v. Martin Marietta Corp, 400 U.S. 542, 544 (1971).
14. Association of American Medical Colleges. 1990 Medical School Graduation Questionnaire, Summary Report for All Schools. 1990.Washington, DC: Association of American Medical Colleges.
15. Association of American Medical Colleges. 1996 Medical School Graduation Questionnaire, Summary Report for All Schools. 1996.Washington, DC: Association of American Medical Colleges.
16. Ciesielski-Carlucci C, Hern HG Jr, Kushner TK. Avoiding discriminatory medical school admission and residency interviewing. Acad Med. 1994;69:975.
17. Teichman JM, Anderson KD, Dorough MM, Stein CR, Optenberg SA, Thompson IM. The urology residency matching program in practice. J Urol. 2000;163:18781887.
18. Hern HG Jr, Simon B. Are you single? The prevalence of inappropriate and illegal questions during residency interviews. 2002 SAEM annual meeting abstracts. Acad Emerg Med. 2002;9:383.
19. Thurman RJ, Katz E, Carter W, et al. Emergency medicine residency applicant perceptions of unethical recruiting practices and illegal questioning in the match. Acad Emerg Med. 2009;16:550557.
20. Hern HG Jr, Alter HJ, Wills CP, Snoey ER, Simon BC. How prevalent are potentially illegal questions during residency interviews? Acad Med. 2013;88:11161121.
21. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307:18051806.
22. Dore KL, Kreuger S, Ladhani M, et al. The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med. 2010;85(10 suppl):S60S63.
23. Hopson LR, Burkhardt JC, Stansfield RB, Vohra T, Turner-Lawrence D, Losman ED. The multiple mini-interview for emergency medicine resident selection. J Emerg Med. 2014;46:537543.
24. Soares WE 3rd, Sohoni A, Hern HG, Wills CP, Alter HJ, Simon BC. Comparison of the multiple mini-interview with the traditional interview for U.S. emergency medicine residency applicants: A single-institution experience. Acad Med. 2015;90:7681.
Supplemental Digital Content
Copyright © 2016 by the Association of American Medical Colleges