Academic Medicine

Skip Navigation LinksHome > October 2012 - Volume 87 - Issue 10 > Admissions File Review: Applying the Multiple Independent S...
Academic Medicine:
doi: 10.1097/ACM.0b013e3182674629
RIME: Medical School Admissions

Admissions File Review: Applying the Multiple Independent Sampling (MIS) Methodology

Hanson, Mark D. MD, MEd; Kulasegaram, Kulamakan Mahan; Coombs, Deborah L. MMS; Herold, Jodi PhD

Collapse Box

Abstract

Purpose: Although multiple independent sampling (MIS) has been adapted for admissions interviews, its application for assessing written materials in the admissions file has been limited. Currently, admissions file review at the University of Toronto medical school involves one rater per file to enable holistic assessment, which may introduce a halo effect—that is, impressions of one component influencing the evaluation of other components. The authors examined whether MIS file review, through which multiple raters evaluate specific file components independently, may reduce this effect.

Method: The authors selected a stratified random sample of 300 applicant files from the 2010–2011 admissions cycle for rescoring by MIS. They divided each of the 300 applicant files into their four components (academic transcript, autobiographical sketch, personal statement, reference letters) and rebundled them into packages of 38 same-component items (purposely creating some overlap among packages to assess inter-rater reliability). The authors distributed each package to 1 of 36 raters; thus, each rater evaluated only one of four components across many applicants. The authors compared the inter-component reliability and factor analysis of MIS with that of holistic scoring.

Results: Ratings were returned for all applicants. Inter-component reliability (Cronbach alpha) was 0.69 for holistic scoring and 0.29 for MIS. Factor analysis showed all components loading heavily onto one factor in the holistic approach and onto three factors in the MIS method.

Conclusions: Using MIS to assess the admissions file may reduce the halo effect and should be considered when evaluating applicants’ written submissions.

© 2012 Association of American Medical Colleges

Login

Article Tools

Share

Article Level Metrics