Spafford, Marlee M. OD, PhD, FAAO
Among North American optometric, medical, and dental programs, the admission interview may be conducted for the purpose of gathering information, verifying information, making decisions, and/or predicting performance. 1–7 Gearing the interview toward recruiting applicants or improving public relations tends to be reserved more to undersubscribed programs. 2–4, 7, 8 Although health care admission committees should ascertain the purpose(s) of each tool in their decision-making process, the majority of them do not. 9 Articulating a purpose includes establishing whether the tool will be used to select or eliminate candidates. For example, tools that consider candidate traits such as physical appearance or racial and religious identification are more apt to be used to eliminate rather than select candidates. 10, 11
The selection interview may involve consideration of candidate traits such as knowledge (e.g., job demands), 2, 3 people skills (e.g., communication), 2, 3, 12 attitude orientation (e.g., work ethic), 12 managerial aptitude (e.g., creativity), 12 team orientation (e.g., cooperation), 12 physical attributes (e.g., attractiveness), 13, 14 and/or social position (e.g., racial identification). 10, 15 According to Powis, 9 the selection tool must be designed to reliably measure the intended candidate trait(s). Unfortunately, selection interviews often fail in this regard. 2, 3, 9, 10 For example, the majority of health care admission interviewers want to evaluate humanistic skills but the interview design is geared toward evaluating cognitive skills. 2, 3, 6, 7, 16, 17
Proponents of the health care admission interview make two main arguments for its inclusion in the selection process. First, the interview has the potential to measure unique humanistic skills that are difficult or impossible to measure with other selection tools. 2–4, 18 Second, academic selection tools (e.g., GPA) are unable to predict clinical competence. 18–22 Critics of the interview point to its poor validity coupled with its high implementation costs. 2, 16, 17, 23–26
The continued use of the admission interview despite its inability to predict successful practice provides an interesting conundrum. The predictive ability (i.e., validity) of a measurement tool depends, in part, on its reliability. 27, 28 Generally, however, the reliability (i.e., repeatability) of interview scores has been found to be favorable both between and within interviewers. 29–31 This leaves the other determinants of validity: the nature of the constructs measured and the type of criterion measure used. 27, 28 Thus, a very reliable measure that captures the wrong information will compromise the instrument’s validity.
In this study, I pursued the idea that the validity of the interview is threatened when there are incongruities either between the expectations and the experiences of an interview or between the interviewers’ and the candidates’ perceptions of the interview. In addition, I considered that limitations to interview validity cause fewer problems if participants do not identify validity as the main purpose of the interview. To investigate these postulates, I studied the expectations and experiences among interview participants at a Canadian optometry program. This study was a part of my doctoral dissertation. 17
MATERIALS AND METHODS
Case study setting
The University of Waterloo School of Optometry (UWSO) provides one of only two Doctor of Optometry programs in Canada. Admission decisions are made by an 11-member Admission Committee that has faculty, administrator, practitioner, and student representation. Sixty candidates are admitted annually to the UWSO. The applicant-to-place ratio is approximately 6:1. The Committee considers university background and transcript performance, the OAT scores, three references, an autobiographic sketch, an essay, and an interview when possible. UWSO selection decisions depend most on the university transcript performance and OAT scores, 32, 33 although university prerequisite completion, the interview, and the autobiographic sketch become important factors when academically homogenous candidates are being considered. 33
An interview has been part of the UWSO admission process since 1972. Approximately 60% of the applicants are interviewed. The decision to interview is based on the candidate’s academic performance and geographical location. 17, 23, 32 Most candidates who receive an offer of admission have been interviewed (e.g., 94.7% between 1992 and 1996). Interviewers are UWSO full-time faculty members who are scheduled into the interviews by their availability. All faculty members are asked to interview and almost all comply. No formal training in interviewing techniques is provided to the interviewers. The interview is a 30-min, semistructured meeting with a two-member interview team. Each interviewer provides one of five overall interview scores, 1.0, 1.5, 2.0, 2.5, or 3.0, where 1.0 is strongest, 2.0 is average, and 3.0 is weakest. At the time of this study, the UWSO admission literature indicated that the mission statement of the interview was to “provide an opportunity to clarify information submitted in the application and to appraise personal qualifications which may bear on the applicant’s success in the practice of optometry.”34 Further descriptions of the UWSO selection process have been published. 17, 23, 32, 33
A research questionnaire was employed to collect data. The questionnaire was developed through reviewing relevant literature, considering my 12 years experience as an UWSO Admission Officer, interviewing six UWSO faculty members and six UWSO students who had been interviewed during the admission process, consulting an educational survey expert on faculty at the Ontario Institute for Studies in Education/University of Toronto (OISE/UT), and conducting a test pilot with two UWSO faculty members and one UWSO student.
The questionnaire items addressed the following issues: (1) respondent information, (2) expectations of an ideal optometry admission interview in terms of its purpose and content, (3) experiences of the UWSO interview in terms of its purpose and content, and (4) possible future changes to the UWSO interview (the full questionnaire is available from the author). At the end of the questionnaire, respondent comments were solicited. Opinions were sought using a five-point Likert scale design of: (A) strongly agree, (B) agree, (C) neither agree nor disagree, (D) disagree, and (E) strongly disagree.
Between early May and early June of 1996, the questionnaires were sent to all 1996 interviewed UWSO applicants (n = 157; 86 women, 71 men) and to all UWSO interviewing faculty except me (n = 23; 7 women, 16 men). Interviewed applicants represented 59% of the 1996 applicant pool. The mailings were timed so that the potential respondents would receive the questionnaire within about 5 weeks of their 1996 UWSO interview experience. Each mailing contained: (1) the letter of information, (2) the questionnaire, and (3) a self-addressed, prestamped OISE/UT envelope. Potential participants were asked to return their questionnaires, either completed or not completed, by June 30, 1996. Reminder letters were sent to nonrespondents in late June and early August. Clerical staff members produced the participant address labels, posted the questionnaires, monitored returned questionnaires, and posted reminder letters to maintain participant anonymity. The study received administrative approval from the UWSO Director and ethical approval from the OISE/UT Ethical Review Committee and the Human Research Ethics Committee at the UW Office of Human Research and Animal Care.
Responses were tabulated such that the five-choice Likert items were converted to numerical values: strongly agree, 5; agree, 4; neither agree nor disagree, 3; disagree, 2; and strongly disagree, 1. Both frequency and statistical analyses of the data were performed. The number of dependent variables (i.e., 11 purposes, 31 candidate traits, and 10 changes) prevented meaningful statistical comparisons per variable because, given enough variables, some statistically significant differences were likely to occur. Principle component (PC) analysis was performed to reduce the data into a smaller number of orthogonal component themes (see Tables 1 to 3). The PC analysis of the ordinal raw data yielded component scores that were continuous and normally distributed. A Bonferroni correction was performed to calculate the appropriate α level for t-test comparisons of the component scores.
Independent t-tests compared applicant perceptions with interviewer perceptions. The independent t-test was also employed to test for significant differences between the perceptions of two applicant groups: internal applicants (i.e., those who had already studied at UW) vs. external applicants (i.e., those who had not studied at UW). This comparison was undertaken because although UWSO receives applications from candidates attending universities and colleges across Canada and occasionally beyond its borders, a large proportion of the applicants already attend UW. In 1996, 33.1% of those interviewed were internal applicants. At many Canadian universities and colleges, there may be only a few UWSO applicants who rarely know their local competition. The large proportion of internal applicants provides an opportunity for hearsay to affect the perception of the UWSO interview and therefore a justification for comparing internal vs. external applicant perceptions. Paired t-tests compared a group’s expectations of an ideal interview with that of the UWSO interview experience.
The statistical analysis strategy employed in this study has been reported previously. 35 Although the statistician I consulted supported the analysis, it is important to acknowledge that consensus among statisticians does not exist regarding the appropriateness of ultimately subjecting continuous data to t-tests when the raw data are ordinal.
The perceptions gleaned from the questionnaire represented one point of time in the school’s history. Admission statistics were examined for the years 1992 to 1996 to address whether the 1996 test year was representative of other applicant years. Demographic variables were compared across the years using the χ2 test and academic variables were compared using one-way analysis of variance. Demographic comparisons included the number of women vs. number of men, the number of interviewed vs. number of uninterviewed applicants, and the number of on-site interviews vs. the number of off-site interviews. Academic comparisons included the university/college transcript overall average, the OAT Total Science score, the mean interview score, the interviewer score difference, and the autobiographic sketch score. In each case, the null hypothesis (HO) was that no yearly difference existed at a 0.01 α level of significance.
The overall questionnaire response rate was 71.7% (n = 129) with a higher response rate among interviewers (87.0%; n = 20; six women, 14 men) than applicants (69.4%; n = 109; 62 women, 47 men). The internal-to-external-applicant breakdown was 38 and 71, respectively. The professor-to-lecturer rank split among interviewers was 13 and 7, respectively, with 10 of the interviewers having sat on the UWSO Admission Committee in the previous 5 years. The questionnaire response rate was considered robust, especially for a mail-in survey, 36 and was sufficient to pursue analysis.
The yearly comparisons of the 1992 to 1996 admission data revealed no significant differences for either the demographic variables (χ24 ≤ 6.53, p ≥ 0.16) or the academic variables (F4 ≤ 2.31, p ≥ 0.06). 17 This group of tests suggests that the 1996 applicant pool was generally typical of other recent UWSO admission years and therefore a reasonable group to study.
Applicants largely credited the strength of their written application for being granted a UWSO interview: postsecondary transcripts (85.3%), OAT scores (77.1%), autobiographic sketch (82.6%), province/territory of residence (23.9%), and unique application (9.3%). Interviewers participated in the UWSO interview particularly out of a sense of duty to the administration: asked to help (95.0%), wanted to help administration (90%), considered interviewing a duty to profession and public (70%), thought interviews were important (80%), believed participation entitled interviewer to a voice (55%), considered self to be a good interviewer (60%).
Response frequencies of the questionnaire items revealed that applicants and interviewers shared a common vision of the ideal optometry admission interview. Its purpose was to gather information from candidates, clarify information in the application, provide information to candidates (an aspect of public relations), and select candidates by appraising their ‘people skills,’ ‘professional skills,’ and ‘attitude orientation.’ Both groups agreed that ‘biased traits’ should not factor into the ideal admission interview. In contrast, the perceived purpose of the UWSO interview was more limited. Its purpose appeared to be geared toward gathering information from candidates and selecting them by appraising mostly their ‘professional skills,’ and to a limited extent, some ‘people skills’ and ‘attitude orientation.’
The ideal optometry interview and the UWSO interview’s mission statement (see Case Study Setting, above) shared a similar set of purposes: to gather information, to clarify information, and to select candidates (gathering information and selecting candidates are implied purposes of the UWSO interview in that they are prerequisites of predicting performance). The only difference was that only the UWSO interview mission statement identified prediction as a goal. The UWSO admission literature did not specify what specific candidate traits were to be assessed so a comparison with the ideal interview was not possible.
Tables 1 to 3 list the results of the principle component analysis. For each principle component (PC), the individual questionnaire items and the percent of total variance are shown. The component themes were named according to the questionnaire items composing them. For example, the first principle component (PC-1) for interview purpose was named ‘public relations’ in Table 1 because the items that composed the theme had been identified as ‘public relations’-type purposes in the literature. 2–4 As a result of PC analysis, the 11 purpose items (Table 1) were reduced to five themes. Using a Bonferroni correction, the α level for comparisons of t-tests was 0.01. The 31 candidate trait items (Table 2) were reduced to nine themes (Bonferroni correction: α = 0.006). The 10 interview change items (Table 3) were reduced to three themes (Bonferroni correction: α = 0.02).
The independent t-test analysis revealed shared interview expectations and experiences with only one significant group difference. Applicants expected the ideal interview would assess ‘attitude orientation’ (PC 3) significantly more than did interviewers (t121 = 3.05, p = 0.003), although there were no significant group differences in terms of the experiences of the UWSO interview (t21 ≤ 2.94, p ≥ 0.008). There were also no significant group differences with respect to the expectations of the ideal interview’s purpose (t124 ≤ 1.22, p ≥ 0.23) or the experiences with the UWSO interview’s purpose (t124 ≤ 2.46, p ≥ 0.02).
An independent t-test group comparison of the internal applicants (n = 38) with the external applicants (n = 71) revealed no significant differences in their expectations of the ideal interview’s purpose (t105 ≤ 2.07, p ≥ 0.04) and content (t103 ≤ 2.06, p ≥ 0.04). They also shared common experiences of the UWSO interview’s purpose (t104 ≤ 0.76, p ≥ 0.45) and content (t99 ≤ 2.36, p ≥ 0.02).
According to the paired t-tests, there were significant deviations between what participants experienced in the UWSO interview and what they expected from an ideal admission interview. These differences were more widespread among applicants than interviewers.
Applicants perceived the purpose of the UWSO interview as significantly less geared than the ideal interview toward promoting public relations, recruiting candidates, gathering information, or verifying information (Table 4). Relative to their ideal expectations, the applicants also experienced the UWSO interview as significantly overemphasizing the assessment of the candidate’s ‘professional skills’ and ‘biased traits’ and significantly underemphasizing the evaluation of ‘attitude orientation,’ ‘team orientation,’ ‘managerial aptitude,’ and ‘physical skill’ (Table 5).
Interviewers perceived the UWSO interview purpose as significantly less oriented than the ideal interview toward verifying candidate information (Table 4). Their experiences of the UWSO interview led them to believe it was significantly less oriented than the ideal interview toward emphasizing the candidate’s ‘team orientation,’ ‘managerial skills,’ ‘physical skill,’ and aspects of ‘people skills’ (Table 5).
Frequency analysis of the questionnaire item addressing whether the UWSO interview should be left unchanged, subtly revised, significantly revised, or eliminated revealed a notable difference between applicants and interviewers (Table 6). The majority of applicants (n = 98) favored subtle revisions (63.3%); in fact, almost 25% of the applicants indicated no change, whereas the majority of the interviewers (n = 17) favored significant revisions (58.8%). Interviewers agreed significantly more than applicants that the UWSO interview should change in terms of ‘interviewer effect’ (Table 7). A frequency analysis of the questionnaire items pertaining to possible changes to the UWSO interview revealed that it was interviewer training, the topics covered, and the weight of the interview in admission decisions that differentiated the groups. These areas were felt by interviewers to need particular attention.
According to 1996 UWSO applicants and interviewers, the ideal optometry admission interview should gather information from candidates, verify information in the application, provide information to candidates, and select candidates by appraising their ‘people skills,’ ‘professional skills,’ and ‘attitude orientation.’ Although the expectations of the ideal interview resembled the mission statement of the UWSO interview, these expectations deviated significantly from the participant experiences of the UWSO interview, particularly among applicants. The similarity among the applicant pools between 1992 and 1996 suggests that the 1996 respondents were typical of recent UWSO applicants.
The description of the ideal interview included four of the six possible purposes for providing an admission interview. 2–4 The only purposes not included were those more associated with an undersubscribed program, 2–4, 7, 8 which UWSO is not. The emphasis on assessing humanistic traits was apparent in the vision of the ideal optometry admission interview. According to UWSO participants, the ideal interview should not consider ‘biased traits’ (i.e., fashion, attractiveness, religious or racial affiliation, visible disability). Eliminating the influence of ‘biased traits’ from the selection interview is desirable because these traits tend to be used to exclude rather than include candidates. 9–11, 13–15, 37–40
The ideal interview provides a template with which the UWSO faculty members can begin to revise their admission interview if they deem it is deficient. The UWSO interview has been shown to be a poor predictor of academic or clinical performance in the optometry program. 17, 23 The low validity of the UWSO interview is explained, at least in part, by the inappropriate nature of the constructs being examined during the interview. The similarity between the UWSO interview mission statement and the participant expectations of the ideal interview suggests that UWSO participants largely supported the intentions of the program’s interview. In contrast, participant experiences of the UWSO interview notably deviated from that intent. The UWSO interview appeared to be limited to gathering information and selecting candidates.
The greatest criticism of admission interviews is usually their low validity. 2, 16, 17, 23–26 Although the UWSO interview mission statement indicated that prediction was a goal, neither the UWSO applicants nor the interviewers identified prediction as a goal of either the ideal interview or the UWSO interview. This finding suggests that the failure of the UWSO interview to predict performance is not as great an issue to its participants as other unmet objectives (e.g., verifying candidate information and providing information to candidates).
UWSO applicants and interviewers shared in common their expectations of the ideal interview and their experiences of the UWSO interview. In the one significant group difference obtained, applicants more than interviewers expected that the ideal interview would assess ‘attitude orientation’ (PC 3). This difference was not surprising because applicants would be expected to place great importance on demonstrating elements of ‘attitude orientation’ such as motivation, work ethic, and perseverance to convince the interviewers of their sincerity to become an optometrist. Intuitively, one might have expected that the perceptions of these two groups would significantly differ in more ways because of their different life experiences and agendas for participating in the process. This viewpoint was supported by one applicant’s comment, “[The] interview team should include an optometry student. A student might be able to pick up some things that a faculty member won’t due to the different points of view of their positions.” The closest studies with which to compare the present findings have involved the admission process of medical residencies. 41, 42 In these programs, significant incongruities were demonstrated between what traits applicants and their raters believed were being assessed. Unlike this study, these investigations did not examine the purpose of the selection process and their analyses did not focus on one selection tool.
The shared vision of the ideal interview suggests that it may be possible to design an interview that can satisfy the agendas of both UWSO applicants and their interviewers. This congruency is desirable because the behavior of the participants can reinforce shared goals and lead to the participants feeling satisfied with their interview experience. Satisfaction with the interview will translate into a perception that it is a worthwhile endeavor. In fact, face validity has been found to be an important determinant of an interview’s perceived fairness. 43, 44
The similarity in perceptions of the ideal interview suggests that participants, through a variety of means, come to attach a common meaning to the event known as the optometry admission interview. By applying the concept of symbolic interactionism from the field of sociology, 45, 46 the understanding of what an interview means is derived from experience with interviews and interactions with others regarding interviews. This study supports a view that the interview has come to symbolize a valued step in the selection process during which the intended focus is on the assessment of the candidate’s humanistic skills. In this regard, the UWSO appears typical of other health care programs in that most: (1) want to evaluate humanistic skills in their selection process, 2, 3, 7, 9, 47 (2) include an interview in their selection process, 2, 3, 5–7 (3) state that their interview is intended to assess humanistic skills, 2, 3 and (4) place more weight on the interview than other selection tools that have the potential to assess humanistic skills. 2, 3, 6
There were significant disparities between the participants’ experiences of the UWSO interview and their expectations of an ideal optometry admission interview. This was particularly true of applicants whose experiences deviated significantly from their expectations in four of the five purpose themes and seven of the nine candidate trait themes. In contrast, the experiences of interviewers deviated significantly from their expectations in one of the five purpose themes and five of the nine candidate trait themes.
Selection and prediction occur after the interview. Unlike the applicants, interviewers had the opportunity to observe or participate in these functions of the UWSO interview. Not surprisingly, only the interviewers perceived that the UWSO interview significantly underemphasized selection and prediction. Half of the interviewer respondents had sat on the UWSO Admission Committee at some point in the previous 5 years. Studies of the UWSO interview have shown that its weight is less than that of the university transcript or the OAT score, 32 although it can significantly influence the final offers in a class. 17, 33 All interviewers had the opportunity of observing how admitted applicants performed in the optometry program. These experiences of faculty led to their perceptions of the UWSO interview.
The applicant’s experience of the UWSO interview depended on one or more of three factors: (1) information gleaned from the UWSO admission booklet, (2) applicant hearsay about the UWSO interview, and (3) interviewer behavior during the UWSO interview. The admission booklet contained very little information about the interview (i.e., three sentences) and therefore its impact was minimal. As the largest applicant group, internal applicants were most susceptible to hearsay because they knew that many of their classmates were also applying to the UWSO. There was no significant difference in expectations or experiences between internal and external UWSO applicants, thereby suggesting the hearsay was not a major factor in determining the applicant perceptions gleaned in this study.
With the potential influences of the admission booklet and hearsay being minimal, it is the interviewers’ behavior that largely dictated the applicant’s experience of her/his UWSO interview. Relative to the applicants’ expectations of an ideal interview, the UWSO interview was significantly less geared toward all purposes determined in the interview. This pattern of differences suggests that the UWSO interviewers needed guidance on how to conduct the interview.
The UWSO interview’s significant underemphasis on recruitment relative to the applicants’ expectations of an ideal interview was highlighted by several applicants’ comments on the questionnaire. One candidate wrote, “Nobody makes you feel welcomed at UW or offered to show [you] around.” The comment was echoed by two others, including one who wrote, “The impression I’ve gotten is, as the only English-speaking optometry program for Canadians, that Waterloo does not provide encouragement and information to reduce discomfort during [the] application process. I sensed a colder more rigid interview environment than others I’ve experienced.” The UWSO has not been in the place of some optometry programs that must recruit to fill their program with quality applicants. Some UWSO applicants simultaneously apply to American optometry programs. These applicants are more likely to encounter various recruitment and promotional strategies, including promotional videos about the program, visits from American admission representatives to UW, and tours of American institutions during candidate visits. In the presence of these promotional and recruitment efforts on the part of other optometry programs, the UWSO seems less interested in its applicants. The perceived underemphasis on recruitment in the UWSO interview has been most noted among women applying to the UWSO. 35 This finding has been interpreted as evidence that the UWSO interviewers may have unintentionally sent a less welcoming message to women applying to the program.
Since the early 1990s, UWSO interviewers have not been allowed to access the candidate’s application before or during the interview. This policy change resulted from a study showing that when UWSO interviewers were allowed to access the candidate’s application, the interview scores reflected the academic grades found in the written application rather than the interview performance. 23 This type of scoring bias has been found in other health care admission interviews. 19, 48–50 Prior to the study, I knew that participants would agree that the UWSO interview did not verify information found in the application. What was unknown was how this experience would compare with their expectations of an ideal optometry interview. Interestingly, both applicants and interviewers agreed the ideal interview should aim to verify candidate information and that the UWSO interview fell significantly short of this expectation. Although the denial of access to applications was rooted in a desire to improve interview validity, the policy was not well received. In the case of the interviewers, they had been apprised of the study’s results and the rationale for the policy change. Despite having this information, the denial of access to the applications seemed to create a crisis of confidence in the interview. Interviewers would rather risk having information that influenced them than go into the interview ‘blind.’ Interviewers were not required to notify the interviewee that they had had no access to the candidate’s application or to explain the rationale for this policy. Therefore, it was up to either the discretion of the interviewers to tell the candidate or the assertiveness of the applicant to inquire about the interviewers’ lack of knowledge of the candidate.
The belief that UWSO interviewers would be familiar with an application before the interview was reasonable. The UWSO admission booklet, which most candidates received, had not been updated yet to reflect that verification of information was not an intended purpose of the UWSO interview. In addition, interviewer access to applicant files is commonplace among employment and admission interviews. Finally, UWSO applicants believed that it was the strength of their written application that led to their interview. Candidates, who expected the interviewers to be familiar with their application yet received no explanation for the interviewers not reading it, would naturally have come to negative conclusions about the interviewers. The perception of the UWSO interview as cold and uncaring is rooted, at least in part, in the failure of the UWSO to convey the interview’s purpose to its applicants. Based on misinformation, applicants believed that UWSO interviewers could not be bothered to review their application.
Almost the entire 30-min UWSO interview involved the interviewers asking the candidate questions. In the last 2 to 3 minutes, interviewees had the opportunity to ask questions of the interviewers. This design accounts, at least in part, for why candidates did not believe the UWSO interview was sufficiently geared toward aspects of promoting public relations. There was little time to provide functions like providing information to the candidate. In fact, one respondent wrote, “Let the students know the role/importance of the interview. More information should be given regarding the interview so that students know how to prepare for it.” Presumably this candidate wanted more information before the interview either in the admission booklet or during on-site admission presentations. Attention to promoting public relations includes reducing applicant stress and providing information to candidates. One candidate indicated that the lack of information provided during the UWSO admission process unnecessarily increased the stress of the competition. Although the UWSO faculty value their gate-keeping role, the perceived inattention to promoting public relations on the part of the UWSO personnel led applicants to believe the UWSO interview and selection process was uncaring and unfair.
What was initially surprising was the finding that an interview, which was almost completely oriented toward asking the candidate questions, was one that was perceived to be significantly deficient in its attention to gathering information. This finding can be explained if candidates believed that the ‘wrong’ information was being gathered. The large number of significant differences between the candidate traits that applicants believed the ideal interview should assess and those traits that they believed the UWSO interview did assess supports this interpretation. It should be noted that this perception was held largely by the women applying to the UWSO, 35 suggesting an unintentional differential treatment of women and men.
Relative to the applicants’ expectations of the ideal interview, candidate trait assessment in the UWSO interview significantly overemphasized ‘professional skills’ and ‘biases’ and significantly underemphasized ‘attitude orientation,’ ‘team orientation,’ ‘managerial aptitude,’ and ‘physical skill.’ The UWSO interview questions were designed to evaluate knowledge and problem solving. ‘Professional skills’ included knowledge about the profession, ethical principles, and moral decision-making. Although these were valued traits, they were deemed to be weighted too highly in the UWSO interview. According to applicants, the UWSO faculty should refocus the interview’s content. The need for this change is highlighted by four of the candidates’ written comments: (1) “interviewers should not ask specific optometry questions that will be taught in the program”; (2) “the [UW] interview should concentrate on items that are not reflected by their [the candidates’] GPA, OAT, and essay/references”; (3) “I was surprised that not one question was asked about myself”; and (4) “I may not get accepted to the [UW optometry] school because I didn’t know specific optometry questions like the difference between the Optometrists Association and the College of Optometrists or if I knew the muscle reflex arcs of the eye muscles or exercises a goalie could do with his eyes to improve his/her performance. I don’t think these questions reflect my potential or reveal anything about myself, my motivation or why I would be an excellent optometrist/student of optometry.”
The perceived overemphasis on ‘biased’ traits is somewhat worrisome. Applicants seemed concerned that UWSO interviewers were too affected by the candidate’s appearance (i.e., fashion, beauty, or visible disability) or cultural background (i.e., religious or racial identification). The UWSO administration may want to provide training for its interviewers to recognize and eliminate errors in information gathering and interpretation. This type of training has been shown to significantly improve reliability by helping interviewers gather and process relevant information. 51–53 Structuring the interview (e.g., asking the same questions of all candidates and developing behavioral anchors for scoring) may afford the greatest improvement in reliability. 10, 54 Numerous types of rater errors on selection interviews, particularly among untrained interviewers, have been described in the literature. 1, 17 Unfortunately, the use of interviewers untrained in the techniques of interviewing is commonplace among health care professional admission interviews. 1–3
Interviewers, although to a lesser degree than applicants, also believed the focus of the UWSO interview was inaccurate. Relative to the interviewers’ expectations of an ideal interview, the UWSO interview significantly underemphasized the assessment of the candidates’ ‘team orientation,’ ‘managerial aptitude,’ aspects of ‘people skills’ (i.e., communication and interpersonal skills), and ‘physical skill.’ The concerns that interviewers and applicants shared regarding the UWSO interview pertained to the emphasis on ‘team orientation,’ ‘managerial aptitude,’ and ‘physical skill.’ The results suggest that the content of the UWSO interview should expand to include the assessment of a broader array of candidate traits that are humanistic skill-oriented.
Despite demonstrating a more widespread disparity between interview expectations and experiences, the applicants supported subtle or no revision to the UWSO interview. The applicants hesitancy to recommend major change can be accounted for, at least in part, by three respondent factors: (1) their trust in my assurance of participant anonymity, (2) their motivation for participating in the UWSO interview, and (3) their experience with the UWSO interview.
Notwithstanding reassurances about respondent anonymity, applicants may have been hesitant to suggest that significant revision of the UWSO interview was needed. If respondent anonymity had been violated, applicants had more to lose than interviewers. Some applicants might have feared that negative statements about the future of the UWSO interview would be shared with the Admission Committee. A reasonable criticism of this interpretation is that a hesitancy to recommend substantial change should have been accompanied by a similar shyness to report a difference between experiences and expectations. Unsolicited comments from a few applicant-respondents after gaining entry into the UWSO program indicated that they had “toned down” their experiences and their recommendations for fear that either the UWSO interview would be eliminated or their identities revealed. If these concerns were prevalent, then the disparity between expectations and experiences was greater among applicants than reported.
The motivation for attending an interview differs substantially for applicants and interviewers. Applicants generally support interviews because they assume that an interview is a reward earned from a strong written application and that a personal meeting will strengthen their application. By contrast, interviewers face many competing demands for their time including research, teaching, and administrative duties. Unless faculty members appreciate some compelling benefit to interviewing applicants, they will participate largely out of a sense of duty.
Applicants typically have very limited experience with the UWSO interview. The average applicant-respondent attended 1.3 UWSO interviews, whereas the average interviewer-respondent conducted 8.6 interviews each year and had been an UWSO interviewer at least 10 years. The greater experience with the UWSO interview and the admission process led to greater concerns on the part of the interviewers. One interviewer wrote, “The interview can only hurt an applicant. Anyone who is not interviewed is assumed to be better than anyone who is interviewed poorly.” Another wrote, “ the [UWSO] interview works to predict who will fail as an optometrist and optometry student, rather than who will succeed.” The greater experience with this selection tool accounts for the interviewers’ more negative view of the UWSO interview, yet it probably helped them to identify what changes should be made to the interview. Respondents, particularly interviewers, identified interviewer training, the topics covered, and the weight of the interview in admission decisions as issues that needed addressing. The desire to seek interviewer training was an encouraging sign. Changing the topics covered was consistent with the disparity between expectations and experiences of candidate traits.
Five previously discussed contextual factors 35 should be considered when evaluating the applicability of this study’s results to other health care professional programs. First, the 1996 UWSO applicants were typical of UWSO applicants in recent years. Second, the purpose of the ideal interview described in this study is typical of that described in the literature. Third, the UWSO is an oversubscribed program and therefore its interview is less focused on recruitment and public relations. Fourth, the decision to participate and the opinions expressed by respondents may have been influenced by my role as an UWSO Admission Officer at the time of the study. Finally, the timing of the questionnaire was such that at least 90% of the applicants did not know the UWSO Admission Committee’s decision when they completed their questionnaire.
Interest in learning more about the selection process through the examination of participant perceptions has become apparent in the literature only recently. 41–44, 55–57 The bulk of this literature has focused on perceived fairness of selection tools including the interview. This study provides a detailed description of what participants are seeking in an optometry admission interview. Through this process, another determinant of perceived fairness is suggested. When there are significant and widespread differences between what participants expect and what they experience in an interview, they are more likely to perceive their interview experience as unfair. Future studies of the UWSO interview should specifically examine the participants’ perception of the interview’s fairness.
The UWSO has supported critical self-study of its admission process to create opportunities for improvements in its valued professional gate-keeping role. According to this study, the ideal optometry admission interview should gather information from candidates, verify information in the application, provide information to candidates, and select candidates by appraising their ‘people skills,’ ‘professional skills,’ and ‘attitude orientation.’ The expectations of interview purpose and content deviated significantly from the participant experiences of the UWSO interview, particularly among applicants. The need to change the UWSO interview was most apparent to the more experienced interviewers who identified interviewer training, topics covered, and interview weight as the foci for improvements. Establishing and disseminating clear goals for the UWSO interview will help both applicants and interviewers to understand their role in the interview and to support its function.
The late Dr. George Geis (Ontario Institute for Studies in Education/University of Toronto) provided advice on the questionnaire development. Dr. Amy Bakelaar (UWSO graduate) entered the data. Erin Harvey (UW) provided statistical support.
This study was supported by a Canadian Optometric Education Trust Fund grant.
1. Spafford MM. The professional school admission interview: a review of the literature. Optom Educ 1993; 10:12–7.
2. Spafford MM. Optometry admission interviewing practices in Canada and United States. Optom Vis Sci 1995; 72:589–97.
3. Johnson EK, Edwards JC. Current practices in admission interviews at U.S. medical schools. Acad Med 1991; 66:408–12.
4. Edwards JC, Johnson EK, Molidor JB. The interview in the admission process. Acad Med 1990; 65:167–77.
5. Willer B, Keill S, Isada C. Survey of U.S. and Canadian medical schools on admissions and psychiatrically at-risk students. J Med Educ 1984; 59:928–36.
6. Puryear JB, Lewis LA. Description of the interview process in selecting students for admission to U.S. medical schools. J Med Educ 1981; 56:881–5.
7. Myslinski NR, Jeffrey RI. The dental admissions interview. College and University 1985:160–79.
8. Galazka SS, Kikano GE, Zyzanski S. Methods of recruiting and selecting residents for U.S. family practice residencies. Acad Med 1994; 69:304–6.
9. Powis DA. Selecting medical students. Med Educ 1994; 28:443–69.
10. Campion MA, Pursell ED, Brown BK. Structured interviewing: raising the psychometric properties of the employment interview. Personnel Psychol 1988; 41:25–42.
11. McDonald T, Hakel MD. Effects of applicant race, sex, suitability, and answers on interviewers’ questioning strategy and ratings. Personnel Psychol 1985; 38:321–34.
12. Lombardi DN. Handbook of Personnel Selection and Performance Evaluation in Healthcare: Guidelines for Hourly, Professional and Managerial Employees. San Francisco: Jossey-Bass Publishers, 1988.
13. Dipboye RL, Arvey RD, Terpstra DE. Sex and physical attractiveness of raters and applicants as determinants of resumé evaluations. J Appl Psychol 1977; 62:288–94.
14. Scheuerle J, Guilford AM, Garcia S. Employee bias associated with cleft lip/palate. J Appl Rehab Counseling 1982; 13:6–8,45.
15. Prewett-Livingston AJ, Feild HS, Veres JG III, Lewis PM. Effects of race on interview ratings in a situational panel interview. J Appl Psychol 1996; 81:178–86.
16. Smith SR, Vivier PM, Blain AL. A comparison of the first-year medical school performances of students admitted with and without interviews. J Med Educ 1986; 61:404–6.
17. Spafford MM. Optometry admission interviews: a case study of participant expectations and experiences. PhD thesis, Ontario Institute for Studies in Education of the University of Toronto, 1998.
18. Spooner CE Jr. Help for the gatekeepers: comment and summation on the admission process. Acad Med 1990; 65:183–7.
19. Tarico VS, Altmaier EM, Smith WL, Franken EA Jr, Berbaum KS. Development and validation of an accomplishment interview for radiology residents. J Med Educ 1986; 61:845–7.
20. Walker JD, Killip DE, Fuller JL. The significance of the admission interview in predicting students’ performance in dental school. J Med Educ 1985; 60:569–71.
21. Murden R, Galloway GM, Reid JC, Colwill JM. Academic and personal predictors of clinical success in medical school. J Med Educ 1978; 53:711–9.
22. Gough HG, Hall WB. The prediction of academic and clinical performance in medical school. Res Higher Educ 1975; 3:301–14.
23. Spafford MM. Admission and optometry grade comparisons among students receiving different types of admission interviews. Optom Vis Sci 1994; 71:47–52.
24. Smith SR. Medical school and residency performances of students admitted with and without an admission interview. Acad Med 1991; 66:474–6.
25. Vargo JW, Madill HM, Davidson PR. The preadmission interview as a predictor of academic grades and fieldwork performance. Can J Occup Ther 1986; 53:211–5.
26. Bridle MJ. Student selection: a comparison of three methods. Can J Occup Ther 1987; 54:113–7.
27. McDaniel MA, Whetzel DL, Schmidt FL, Maurer SD The validity of employment interviews: a comprehensive review and meta-analysis. J Appl Psychol 1994; 79:599–616.
28. Messick S. Validity. In: Linn RL, ed. Educational Measurement, 3rd ed. New York: American Council on Education, 1989: 13–103.
29. Mitchell G, Mitchell D, McGregor M. Selection of medical students—are interview evaluations consistent? S Afr Med J 1987; 71:774–6.
30. McManus IC, Richards P. Reliability of short-listing in medical student selection. Med Educ 1989; 23:147–51.
31. Richards P, McManus IC, Maitlis SA. Reliability of interviewing in medical student selection. Br Med J (Clin Res Ed) 1988; 296:1520–1.
32. Spafford MM. Quantifying an optometry programme’s use of academic variables and interview data in its admission decisions. Can J Optom 1994; 56:85–93.
33. Spafford MM. Primary and secondary selection tools of an optometry admission committee. Optom Educ. Accepted for publication.
34. University of Waterloo School of Optometry. Doctor of Optometry Program. Waterloo, Ontario, Canada: University of Waterloo, 1996.
35. Spafford MM, Beal PI. Interview expectations and experiences of women and men applying to an optometry program. Optom Vis Sci 1999; 76:500–10.
36. Moser CA, Kalton G. Survey Methods in Social Investigation. 2nd ed. London: Heinemann Educational, 1971.
37. Johnson R, Heal LW. Private employment agency responses to the physically handicapped applicant in a wheelchair. J Appl Rehab Couns 1976; 7:12–21.
38. Barr SH, Hitt MA. A comparison of selection decision models in manager vs. student samples. Personnel Psychol 1986; 39:599–617.
39. Lin TR, Dobbins GH, Farh JL. A field study of race and age similarity effects on interview ratings in conventional and situational interviews. J Appl Psychol 1992; 77:363–71.
40. Davis LE, Strube MJ, Cheng LC. Too many Black, too many Whites: is there a racial balance? Basic Appl Social Psychol 1995; 17:119–35.
41. Villanueva AM, Kaye D, Abdelhak SS, Morahan PS. Comparing selection criteria of residency directors and physicians’ employers. Acad Med 1995; 70:261–71.
42. Zagumny MJ, Rudolph J. Comparing medical students’ and residency directors’ ratings of criteria used to select residents. Acad Med 1992; 67:613.
43. Smither JW, Reilly RR, Millsap RE, Pearlman K, Stoffey RW. Applicant reactions to selection procedures. Personnel Psychol 1993; 46:49–76.
44. Steiner DD, Gilliland SW. Fairness reactions to personnel selection techniques in France and the United States. J Appl Psychol 1996; 81:134–41.
45. Spencer M. Foundations of Modern Sociology. 7th ed. Scarborough: Prentice Hall Canada Inc., 1996.
46. Blumer H. Symbolic Interactionism: Perspective and Method. Berkeley: University of California Press, 1969.
47. McGaghie WC. Perspectives on medical school admission. Acad Med 1990; 65:136–9.
48. Elam CL, Andrykowski MA. Admission interview ratings: relationship to applicant academic and demographic variables and interviewer characteristics. Acad Med 1991; 66:S:13–5.
49. Litton-Hawes E, MacLean IC, Hines MH. An analysis of the communication process in the medical admissions interview. J Med Educ 1976; 51:332–4.
50. Shaw DL, Martz DM, Lancaster CJ, Sade RM. Influence of medical school applicants’ demographic and cognitive characteristics on interviewers’ ratings of noncognitive traits. Acad Med 1995; 70:532–6.
51. Howard JL, Ferris GR. The employment interview context: social and situational influences on interviewer decisions. J Appl Social Psychol 1996; 26:112–36.
52. Schuh AJ. Effects of training on leniency, variability, and the halo effect in interviewer ratings. Catalog of Selected Documents in Psychology 1973; 3:20–1.
53. Wexley KN, Sanders RE, Yukel GA. Training interviewers to eliminate contrast effects in employment interviews. J Appl Psychol 1973; 57:233–6.
54. Maurer SD, Fay C. Effect of situational interviews, conventional structured interviews, and training on interview rating agreement: an experimental analysis. Personnel Psychol 1988; 41:329–44.
55. Gilliland SW. Effects of procedural and distributive justice on reactions to a selection system. J Appl Psychol 1994; 79:691–701.
56. Kravitz DA, Stinson V, Chavez T. Evaluation of tests used in making selection and promotion decisions. International Journal of Selection and Assessment 1996; 4:24–34.
57. Rynes SL, Connerley ML. Applicant reactions to alternative selection procedures. J Bus Psychol 1993; 7:261–77.