Secondary Logo

Journal Logo

How Prevalent Are Potentially Illegal Questions During Residency Interviews? A Follow-up Study of Applicants to All Specialties in the National Resident Matching Program

Hern, H. Gene Jr MD, MS; Trivedi, Tarak MD; Alter, Harrison J. MD, MS; Wills, Charlotte P. MD

doi: 10.1097/ACM.0000000000001181
Research Reports
Free
SDC

Purpose To describe the prevalence and effects on applicants of being asked potentially illegal questions during the residency interview process by surveying all residency applicants to all specialties.

Method The authors surveyed all applicants from U.S. medical schools to residency programs in all specialties in 2012–2013. The survey included questions about the prevalence of potentially illegal questions, applicants’ level of comfort with such questions, and whether such questions affected how applicants ranked programs. Descriptive statistics, tests of proportions, t tests, and logistic regression modeling were used to analyze the data.

Results Of 21,457 eligible applicants, 10,976 (51.1%) responded to the survey. Overall, 65.9% (7,219/10,967) reported receiving at least one potentially illegal question. More female respondents reported being asked questions about gender (513/5,357 [9.6%] vs. 148/5,098 [2.9%]), marital status (2,895/5,283 [54.8%] vs. 2,592/4,990 [51.9%]), or plans for having children (889/5,241 [17.0%] vs. 521/4,931 [10.6%]) than male respondents (P < .001). Those in surgical specialties were more likely to have received a potentially illegal question than those in nonsurgical specialties (1,908/2,330 [81.9%] vs. 5,311/8,281 [64.1%]). Questions regarding their commitment to the program were reported by 15.5% (1,608/10,378) of respondents. Such potentially illegal questions negatively affected how respondents ranked programs.

Conclusions Two-thirds of applicants reported being asked potentially illegal questions. More women than men reported receiving questions about marital status or family planning. Potentially illegal questions negatively influence how applicants perceive and rank programs. A formal interview code of conduct or interviewer training could help to address these issues.

H.G. Hern Jr is vice chair for education, Department of Emergency Medicine, Highland Hospital, Alameda Health System, Oakland, California, and associate clinical professor, Department of Emergency Medicine, University of California, San Francisco, School of Medicine, San Francisco, California.

T. Trivedi is an emergency medicine resident, Department of Emergency Medicine, Highland Hospital, Alameda Health System, Oakland, California.

H.J. Alter is vice chair for research, Department of Emergency Medicine, Highland Hospital, Alameda Health System, Oakland, California, and associate clinical professor, Department of Emergency Medicine, University of California, San Francisco, School of Medicine, San Francisco, California.

C.P. Wills is residency director, Department of Emergency Medicine, Highland Hospital, Alameda Health System, Oakland, California, and associate clinical professor, Department of Emergency Medicine, University of California, San Francisco, School of Medicine, San Francisco, California.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: The Highland Hospital, Alameda Health System (formerly Alameda County Medical Center) institutional review board approved the survey instrument and study protocol prior to distribution.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A342.

Correspondence should be addressed to H. Gene Hern Jr, Highland Hospital, Alameda Health System, 1411 E. 31st St., Oakland, CA 94602; telephone: (510) 437-4800; e-mail: hhern@alamedahealthsystem.org.

Most states have laws prohibiting employers from asking job applicants questions related to marital status, family planning, age, ethnicity, religion, or sexual preferences.1–5 The term “potentially illegal questions” refers to those that are meant to reveal, or have the effect of revealing, an applicant’s membership in a class protected by federal and state civil rights laws. Interviewers may pose informal questions to try to get to know an applicant.1 However, if these questions are asked, they can be used as evidence of discrimination if the applicant is not hired.6–13

Residents are recognized by the courts as “employees” covered by these civil rights laws.1–5 However, violations of these laws have occurred during residency interviews.14–20 In 1990, the Association of American Medical Colleges found that 35% of residency applicants were asked about family planning/marital status.14 Moreover, 38% of women were asked about their pregnancy intentions. A 1993 study found that more than half of the residency applicants surveyed received questions that they felt were “inappropriate, uncomfortable, or possibly discriminatory.”16

In recent years, most research on this subject has been conducted within individual specialties. For example, a 2000 survey of urology residency applicants found that every female respondent was asked about marital status, and 60% were asked about family planning.17 In 2001, 86% of the 184 emergency medicine residency applicants surveyed reported perceived violations of fair employment laws.18 Four years later, 30% of emergency medicine residency applicants still indicated that they were asked potentially illegal questions.19 In 2007, our group found that 65% of the 7,028 applicants to five different specialties reported being asked such questions.20

To expand on our previous research, we aimed to describe the prevalence and effects of being asked potentially illegal questions during the residency interview process by surveying all residency applicants to all specialties.

Back to Top | Article Outline

Method

All applicants to all specialties in the National Resident Matching Program (NRMP) in 2012–2013 were eligible to participate. After both residency programs and applicants submitted their NRMP rank lists in February 2013, up to three e-mail messages (approximately one week apart) were sent directing applicants to the survey instrument. The instructions clearly stated that the information collected would remain confidential and that only the researchers would have access to individual responses. To ensure that respondents’ identities were not revealed to the researchers, the Electronic Residency Application Service staff sent all e-mail messages, the survey instrument was located on a public Web site (www.surveymonkey.com), and all data remained anonymous. The Survey Monkey Web site prohibits the same IP address from submitting multiple responses to each question, so individuals cannot complete the survey more than once.

Back to Top | Article Outline

Survey

We developed our survey instrument based on a review of the literature and a study group consensus. In addition, a survey methodologist edited the instrument.

The purpose of the study was intentionally stated in neutral terms: “Your answers to this survey will help improve the overall interview process for future classes.” Respondents were asked if, during the course of all their residency interviews, they received questions related to the following categories: (1) gender, (2) age, (3) marital status, (4) parental status, (5) plans for child rearing, (6) ethnicity, (7) religion, and (8) sexual orientation. We inquired, for example, “Were you asked any questions about your gender in relation to your specialty choice or training?” Those who indicated that they had received such a question then were asked, “How comfortable did you feel sharing this information?” Response options ranged from “very comfortable” to “very uncomfortable” on a five-point modified Likert scale.

Next, respondents were asked, “What effect did this type of question have on your decision to rank this program?” Response options ranged from “much less likely to rank it highly” to “much more likely to rank it highly” on a five-point modified Likert scale.

Finally, respondents were asked if they had encountered a situation in which they were asked for a commitment, verbal or otherwise, to rank a program highly. The survey clearly stated that responses should be based on questions initiated by the interviewers regarding information that the applicant had not previously disclosed either in her or his file or in the interview prior to the question being asked.

The survey also solicited demographic information, such as specialty, age, gender, the number of programs to which the respondent had applied, the number that offered an interview, and the number at which the respondent interviewed. See Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A342 for a copy of the complete survey instrument.

Back to Top | Article Outline

Data analysis

We used STATA 13.0 (STATACorp, College Station, Texas) to analyze our data. Statistical analyses included calculating basic prevalence rates, tests of proportions, and a t test to compare means. To identify any demographic characteristics associated with being asked a potentially illegal question, we applied logistic regression modeling. We analyzed being asked a potentially illegal question as the dependent variable, and the model accounted for the following independent variables—age, gender, race, marital status, parental status, and if the respondent applied to a surgical specialty. We then recoded the dependent variables as binary responses using respondents’ reported level of discomfort and impact on rank list after a potentially illegal question was asked. Responses indicating that a question made a respondent “very uncomfortable” or “somewhat uncomfortable” were coded as positive outcomes for discomfort. Responses indicating that a respondent was “much less likely to rank highly” or “less likely to rank highly” were coded as positive outcomes for negative impact on rank lists. The independent variables stayed the same.

Surgical specialty was incorporated into the logistic regression models because of our prior findings demonstrating a higher prevalence of potentially illegal questions among surgical programs.20 Surgical specialties were defined as general surgery, neurosurgery, orthopedic surgery, obstetrics–gynecology, and otolaryngology. Non-response bias was analyzed using early (after the first two e-mails) versus late (after the third e-mail) respondent comparisons.21 For all statistical analyses, we used P < .001 as the cutoff for reporting significant associations.

The Alameda County Medical Center (now Alameda Health System) institutional review board approved the survey instrument and study protocol prior to distribution.

Back to Top | Article Outline

Results

Overall prevalence of potentially illegal questions

Of the 21,457 U.S. residency applicants in 2012–2013, 10,967 replied to our survey (51.1% response rate). Mean respondent age was 26.6; 51.3% were female, and 57.6% had a marital status of single. Other basic demographic information is presented in Table 1. Respondents represented all 19 specialties in the NRMP Match (see Table 2).

Table 1

Table 1

Table 2

Table 2

Among respondents, 65.8% (7,219/10,967) reported being asked at least one potentially illegal question during the interview process (see Table 1). Of the potentially illegal questions asked, most frequently questions were about applicants’ families; 53.3% (5,710/10,710) of respondents were asked about their marital status and 24.0% (2,592/10,798) were asked if they had children. Another related and frequently asked question involved their plans for child rearing or childbearing, with 13.8% (1,450/10,512) reporting that they had been asked such a question. Other potentially illegal questions addressed religion (982/10,368; 9.5%), ethnicity (969/10,747; 9.0%), age (758/10,740; 7.1%), gender (692/10,917; 6.3%), or sexual orientation (121/10,360; 1.2%). About one in six applicants (1,608/10,378; 15.5%) were asked about their level of commitment to the residency program. For a breakdown by specialty of the potentially illegal questions received, see Table 2.

Of those who were asked potentially illegal questions, respondents reported that an average of 5.2 (standard deviation [SD] 3.6) programs asked about marital status, 4.1 (SD 3.3) asked about parental status, 4.0 (SD 3.6) asked about ethnicity, 3.7 (SD 3.2) asked questions related to gender, 3.1 (SD 2.7) asked about age, 3.3 (SD 2.8) asked about plans to have children, 2.6 (SD 2.7) asked about religion, and 3.8 (SD 4.0) asked about sexual orientation.

Back to Top | Article Outline

Early versus late survey respondents

In comparing early versus late respondents, we found small yet statistically significant differences across some demographic charac teristics (see Table 1). We found no significant differences between the two groups with regard to receiving potentially illegal questions about age, gender, family planning, ethnicity, religion, or sexual orientation. However, we did find differences with regard to questions about marital status, parental status, and commitment to the program.

Back to Top | Article Outline

Effect of gender on likelihood of being asked potentially illegal questions

In a model that adjusted for age, race, marital status, parental status, and specialty, female respondents were significantly more likely to have received a potentially illegal question (68.7% vs. 63.4%) compared with male respondents (adjusted odds ratio of female gender [aORf] = 1.35, 95% confidence interval [CI]: 1.23–1.47, P < .001). Although female respondents were significantly more likely to be asked about their marital status or commitment to the program (see Table 3), they also experienced the most marked difference in prevalence of questions regarding gender (9.6% vs. 2.9%, aORf = 3.94, 95% CI: 3.22–4.83, P < .001) or plans to have children (17.0% vs. 10.6%, aORf = 1.89, 95% CI: 1.67–2.15, P < .001).

Table 3

Table 3

Overall, male respondents were less likely to be asked potentially illegal questions. Yet, female respondents were less likely to receive questions about religion (8.0% vs. 11.1%, aORf = 0.77, 95% CI: 0.67–0.89, P = .001) or sexual orientation (0.8% vs. 1.5%, aORf = 0.48, 95% CI: 0.32–0.73, P < .001).

Back to Top | Article Outline

Effect of specialty on likelihood of being asked potentially illegal questions

Among the 96.8% (10,611/10,967) of respondents who indicated their specialty, those in surgical specialties were more likely to have received a potentially illegal question compared with those in nonsurgical specialties (81.9% vs. 61.5%; see Table 2). The greatest prevalence of these questions were reported by respondents from neurosurgery (86.0%), orthopedic surgery (84.7%), obstetrics–gynecology (82.0%), and general surgery (81.4%).

The adjusted odds ratio of a surgical applicant being asked a potentially illegal question versus a nonsurgical applicant [aORs] was 2.97 (95% CI: 2.63–3.35, P < .001; see Table 3). This difference was most notable with regard to questions about gender—15.5% of surgical applicants were asked this type of question versus 3.9% of nonsurgical applicants (aORs = 5.29, 95% CI: 4.44–6.30, P < .001). The likelihood of a surgical applicant being asked a potentially illegal question increased for the following domains: age (aORs = 2.98, 95% CI: 2.49–3.57, P < .001), marital status (aORs = 2.50, 95% CI: 2.25–2.78, P < .001), parental status (aORs = 1.84, 95% CI: 1.64–2.06, P < .001), plans to have children (aORs = 2.09, 95% CI: 1.83–2.39, P < .001), ethnicity (aORs = 1.26, 95% CI: 1.05–1.51, P = .014), religion (aORs = 1.45, 95% CI: 1.24–1.71, P < .001), and commitment to the program (aORs = 1.86, 95% CI: 1.64–2.11, P < .001).

Back to Top | Article Outline

Discomfort as a result of being asked potentially illegal questions

A significant percentage of respondents reported discomfort after being asked potentially illegal questions (see Table 4). Questions about commitment to the program were most frequently associated with discomfort; 77.2% of respondents reported feeling “somewhat uncomfortable” or “very uncomfortable” when asked this question. The prevalence of discomfort for each category of question is depicted in Table 4.

Table 4

Table 4

When asked about gender, more respondents reporting discomfort were women (86.0%) than those reporting no discomfort (74.0%, P < .001). The same was true for those reporting discomfort when asked about marital status (60.0% vs. 49.8%, P < .001), parental status (59.7% vs. 45.5%, P < .001), or plans for child rearing (77.0% vs. 51.5%, P < .001). The results of an adjusted ordinal regression model are presented in Table 4; female respondents were significantly more likely to report discomfort in response to being asked potentially illegal questions in almost all categories.

Back to Top | Article Outline

Effect of being asked potentially illegal questions on respondents’ rank lists

A sizable percentage of respondents reported that potentially illegal questions in each category caused them to place the program lower on their rank list; however, the majority reported that their rank lists were not affected by these questions (see Table 5). Certain questions were associated with more respondents ranking the program lower (negative impact); 39.8% of respondents who were asked about their level of commitment to the program, 30.7% who were asked about gender, and 25.4% who were asked about plans for child rearing all reported ranking the program lower because of the question. Asking about sexual orientation was more polarizing; 31.9% of respondents reported that being asked about sexual orientation caused them to rank the program lower, while 25.0% reported that the same question caused them to rank the program higher.

Table 5

Table 5

Female gender, in an ordinal logistic regression model, was significantly associated with a greater likelihood of indicating a negative impact on rank list when asked about gender (aORf = 3.78, 95% CI: 2.16–6.64, P < .001), marital status (aORf = 2.37, 95% CI: 1.94–2.89, P < .001), parental status (aORf = 3.25, 95% CI: 2.47–4.28, P < .001), plans for child rearing (aORf = 3.57, 95% CI: 2.57–4.95, P < .001), or religion (aORf = 1.90, 95% CI: 1.34–2.69, P < .001) (see Table 5). Surgical specialty was associated with a greater likelihood of indicating a negative impact when asked about plans for child rearing (aORs = 1.58, 95% CI: 1.19–2.09, P < .001).

Back to Top | Article Outline

Discussion

In this study, we surveyed 10,967 applicants to residency programs in 19 specialties. We found that two-thirds were asked at least one potentially illegal question. In the surgical specialties, that rose to 82%, and the odds of receiving a potentially illegal question as a surgical applicant compared with as a nonsurgical applicant were 2.97. In addition, women were much more likely than men to be asked questions regarding their gender, marital status, and plans to have children in the future. Finally, 16% of applicants were asked for a commitment (either verbal or otherwise) to the program during their interview; of those who received such a question, 77% reported discomfort and 40% reported ranking the program lower as a result.

These findings, along with those from previously published studies, underscore a continuing problem in residency interviewing—applicants are being asked potentially illegal questions.14–20 Whereas most studies have focused on a single specialty, our prior work explored these issues among applicants to five specialties (internal medicine, general surgery, orthopedic surgery, obstetrics–gynecology, and emergency medicine) in the 2006–2007 NRMP Match. Our current study expands on that work by including all specialties in the 2012–2013 NRMP Match to offer a broader perspective on the topic. In comparing the results of these two studies, we found that this issue continues to be a problem across medicine.

Our findings have important implications for medical educators, especially those involved in residency training programs. First, given the prevalence of potentially illegal questions, our findings support the need for standardizing interviewers’ training on the federal and state laws governing equal employment hiring practices. Programs may not be aware that some interviewers are asking these questions, and interviewers may not be aware of the risk they are taking in asking them or of the harmful consequences that may occur. Certain questions, if asked, are at best inappropriate and may be illegal if asked disproportionally of one group compared with another or if applicants’ responses influence ranking decisions. Furthermore, asking these questions, regardless of legality, may violate a moral or ethical standard among medical educators.

Second, respondents to our survey from surgical specialties, particularly women, were more likely to be asked potentially illegal questions. This finding is very concerning, especially with regard to specialties in which women are still underrepresented, both in the workforce and in training programs. Surgical programs may consider educating interviewers to avoid inappropriate questioning, to reduce discomfort, and to avoid dissuading applicants, particularly women, from wanting to join their program. However, as we have demonstrated, gender differences in receiving potentially illegal questions were not limited to surgical specialties. Across all specialties, program directors could improve their recruiting and retention of applicants by clearly delineating boundaries for interviewers, given that the residency interview process is protected by both federal and state law as a formal job interview.

Finally, despite NRMP rules, interviewers continue to ask applicants about their level of commitment to the program. In some specialties, more than 30% of respondents to our survey reported receiving this type of question. Although not potentially illegal in the same way as other questions, it is a direct and clear breach of the Match Participation Agreement set forth by the NRMP. Given applicants’ discomfort with such questions, residency programs should establish and enforce policies designed to uphold this agreement.

In the last few years, more interest has developed in standardized interviews, such as the multiple mini-interview (MMI), which can be helpful in medical school and residency selection.22 Set questions could limit applicants’ exposure to potentially illegal questions; however, these models have their own drawbacks. Hopson et al23 described a study using 71 interns as surrogates for residency applicants and noted a correlation between MMI scores and clerkship grades. Yet, the authors also noted less preference for MMI-style interviews. Soares et al24 used a mixed traditional and MMI-style interview and found that applicants vastly preferred the traditional interview over the MMI-style interviews.

Our study has some important limitations. First, our response rate was 51%. The survey may be inherently biased, and our findings might overestimate the prevalence of potentially illegal interview questions because those who identified such questions may have been more inclined to respond. However, even using the most conservative estimate of nonrespondent bias, assuming that all nonrespondents did not receive any potentially illegal questions, we estimate that 30% of the entire pool of U.S. residency applicants (7,219/21,457) reported that they had received a potentially illegal question. Prior analyses of nonrespondent bias have suggested that late respondents share characteristics with nonrespondents.21 We compared early and late survey respondents and found no significant differences between the two groups with regard to receiving potentially illegal questions about age, gender, family planning, ethnicity, religion, or sexual orientation. This suggests that nonrespondents are exposed to potentially illegal questions at a significant rate, similar to respondents.

Second, our results are based on respondents’ interpretation of the questions they were asked during interviews rather than on the actual questions they were asked. The survey made no mention of the words “illegal,” “inappropriate,” or “discrimination” and was designed to sound as neutral as possible. This may have led to a biased estimate of the prevalence of potentially illegal questions, either too low or too high, depending on respondents’ experiences. For example, in some cases respondents may have brought up the issues themselves and the interviewer only continued the line of questioning. If sensitive information is offered freely by an applicant, the interviewer may continue to discuss that information.

In summary, the prevalence of potentially illegal questions during residency interviews is high. Among respondents to our survey (including late respondents), women were more likely than men to receive questions about gender, marital status, and family planning. Applicants to surgical specialties were more likely to report being asked potentially illegal questions in all question categories. Applicants often reported discomfort at being asked such questions and reported ranking a program lower as a result of a question. Educators should recognize that such questions have a negative impact on applicants and consider outreach to interviewers to provide information regarding acceptable interview procedures. A formal interview code of conduct could address these issues within the difficult process of selecting future medical practitioners.

Acknowledgments: The authors wish to thank Ms. Moira Edwards and Ms. Renee Overton of the Association of American Medical Colleges and the Electronic Residency Application Service for their invaluable help and Michael Ward, JD, for employment law opinions.

Back to Top | Article Outline

References

1. Kasuri v. St. Elizabeth Hospital Medical Center, 897 F.2d, 845 (6th Cir 1990).
2. Civil Rights Act of 1964 § 7, 42 U.S.C. § 2000e et seq (1964).
3. Americans with Disabilities Act of 1990, Pub. L. No. 101336104 Stat. 328 (1990).
4. Age Discrimination in Employment Act, 29 U.S.C. § 621 et seq.
5. California Fair Employment and Housing Act, 1290012996.
6. Odima v. Westin Tucson Hotel, 53 F.3d 1481, 1494 (9th Cir. 1995).
7. Stout v. Potter, 276 F.3d 1118, 1121 (9th Cir. 2002).
8. U.S. Department of Labor. Employment applications. 29 CFR. §1625.5.
9. U.S. Department of Labor. Guidelines on discrimination because of religion. 29 CFR. §1605.1–1605.3.
10. California Government Code Section 12940(d).
11. California Government Code Section 7287.3(b)(1).
12. U.S. Department of Labor. Discrimination against married women. 29 CFR. §1604.4.
13. Phillips v. Martin Marietta Corp, 400 U.S. 542, 544 (1971).
14. Association of American Medical Colleges. 1990 Medical School Graduation Questionnaire, Summary Report for All Schools. 1990.Washington, DC: Association of American Medical Colleges.
15. Association of American Medical Colleges. 1996 Medical School Graduation Questionnaire, Summary Report for All Schools. 1996.Washington, DC: Association of American Medical Colleges.
16. Ciesielski-Carlucci C, Hern HG Jr, Kushner TK. Avoiding discriminatory medical school admission and residency interviewing. Acad Med. 1994;69:975.
17. Teichman JM, Anderson KD, Dorough MM, Stein CR, Optenberg SA, Thompson IM. The urology residency matching program in practice. J Urol. 2000;163:18781887.
18. Hern HG Jr, Simon B. Are you single? The prevalence of inappropriate and illegal questions during residency interviews. 2002 SAEM annual meeting abstracts. Acad Emerg Med. 2002;9:383.
19. Thurman RJ, Katz E, Carter W, et al. Emergency medicine residency applicant perceptions of unethical recruiting practices and illegal questioning in the match. Acad Emerg Med. 2009;16:550557.
20. Hern HG Jr, Alter HJ, Wills CP, Snoey ER, Simon BC. How prevalent are potentially illegal questions during residency interviews? Acad Med. 2013;88:11161121.
21. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307:18051806.
22. Dore KL, Kreuger S, Ladhani M, et al. The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med. 2010;85(10 suppl):S60S63.
23. Hopson LR, Burkhardt JC, Stansfield RB, Vohra T, Turner-Lawrence D, Losman ED. The multiple mini-interview for emergency medicine resident selection. J Emerg Med. 2014;46:537543.
24. Soares WE 3rd, Sohoni A, Hern HG, Wills CP, Alter HJ, Simon BC. Comparison of the multiple mini-interview with the traditional interview for U.S. emergency medicine residency applicants: A single-institution experience. Acad Med. 2015;90:7681.

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2016 by the Association of American Medical Colleges