Purpose: Traditional admissions personal interviews provide flexible faculty–student interactions but are plagued by low inter-interview reliability. Axelson and Kreiter (2009) retrospectively showed that multiple independent sampling (MIS) may improve reliability of personal interviews; thus, the authors incorporated MIS into the admissions process for medical students applying to the University of Toronto’s Leadership Education and Development Program (LEAD). They examined the reliability and resource demands of this modified personal interview (MPI) format.
Method: In 2010–2011, LEAD candidates submitted written applications, which were used to screen for participation in the MPI process. Selected candidates completed four brief (10–12 minutes) independent MPIs each with a different interviewer. The authors blueprinted MPI questions to (i.e., aligned them with) leadership attributes, and interviewers assessed candidates’ eligibility on a five-point Likert-type scale. The authors analyzed inter-interview reliability using the generalizability theory.
Results: Sixteen candidates submitted applications; 10 proceeded to the MPI stage. Reliability of the written application components was 0.75. The MPI process had overall inter-interview reliability of 0.79. Correlation between the written application and MPI scores was 0.49. A decision study showed acceptable reliability of 0.74 with only three MPIs scored using one global rating. Furthermore, a traditional admissions interview format would take 66% more time than the MPI format.
Conclusions: The MPI format, used during the LEAD admissions process, achieved high reliability with minimal faculty resources. The MPI format’s reliability and effective resource use were possible through MIS and employment of expert interviewers. MPIs may be useful for other admissions tasks.
Dr. Hanson is associate dean, Undergraduate Medicine, Admissions and Student Finances, and associate professor of psychiatry, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Mr. Kulasegaram is PhD candidate, McMaster University, Hamilton, Ontario, Canada, and research fellow, Wilson Centre for Health Professions Education Research, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Dr. Woods is a scientist, Wilson Centre for Health Professions Education Research, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Ms. Fechtig is curriculum coordinator, Manager and Collaborator Competencies, Center for Interprofessional Education, and Leadership Education and Development program, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Dr. Anderson is professor, Institute of Health Policy, Management and Evaluation, and program director, Leadership Education and Development program, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
Correspondence should be addressed to Dr. Hanson, Faculty of Medicine, University of Toronto, 1 King’s College Circle, Room 2135, Toronto, Ontario, Canada M5S 1A1; telephone: (416) 946-7972; e-mail: firstname.lastname@example.org.