Leddy, John J.; Moineau, Geneviève; Puddester, Derek; Wood, Timothy J.; Humphrey-Murto, Susan
Every year, medical schools are faced with the daunting task of selecting a class of medical students from a vast pool of highly qualified applicants. The goal of admission committees is to select applicants that not only will excel in their undergraduate studies but also will sustain this success throughout their medical practice. Difficulties in practice related to physician behavior have been traced back to difficulties in undergraduate medical education.1 If these difficulties could be identified at the time of admissions, it might prevent problematic students from entering medical school.
Selection of medical students has historically been based on a combination of cognitive and noncognitive measures. Cognitive measures, such as academic ability, processing of information, reasoning comprehension, and decision making, are measured by grade point average (GPA) and the Medical College Admission Test (MCAT) score. These cognitive measures offer very good predictability of success in the preclinical years and more moderate correlations with performance in clinical years as well as licensing examinations.2 They do not, however, adequately measure noncognitive qualities.2
Recent North American recommendations have focused on the importance of noncognitive qualities in determining future medical practice success and highlight the need for meaningful ways of identifying and measuring these desired attributes in applicants.3 Desirable noncognitive qualities include empathy, resiliency, altruism, reliability, integrity, compassion, and communication skills.4 Currently, tools used to measure noncognitive qualities include the autobiographical sketch, reference letters, multiple mini-interview, and the semistructured interview. As traditional methods, such as the autobiographical sketch and reference letters, are poorly predictive of future performance5 and interviews in any format have a moderate correlation of future performance,6 it is important to consider potential new methods of assessing applicants' noncognitive attributes.
A noncognitive measure that could be used during the admissions process is emotional intelligence (EI). EI is the ability to carry out accurate reasoning about emotions and the ability to use emotions and emotional knowledge to enhance thought. Often, EI is measured by self-report scales where scores can be highly influenced by personality traits7 and readily inflated in a high-stakes environment (such as the medical school admissions process).8 The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT V2.0) offers a direct assessment of a person's capacity to identify emotions in others, to use emotions to facilitate thought, to understand emotional vocabulary and meanings, and to know how to manage emotions.9 The MSCEIT V2.0 is not strongly influenced by personality traits and has robust psychometric properties.10,11
For physicians, EI is particularly important because understanding patients' emotions and controlling one's own emotions during stressful clinical encounters are essential to working effectively in health care teams and providing the best care for patients. Previous studies have found that self-reported EI scores positively correlated with communication skills and peer assessment of problem-based learning.12,13 Few studies have measured EI in medical students using an ability-based EI test like the MSCEIT V2.0.14,15 One such study reported no correlation between the EI measured during years 5 and 6 of a six-year curriculum and data collected at admission (i.e., admission test, entrance rank, interview score).16
Because an understanding of the relationship between EI measured before matriculation and traditional admissions measures is needed to determine if measurement of EI should be considered for use in the admission process, we conducted a study to determine whether medical school applicant scores on the MSCEIT V2.0 correlated with traditional medical school admissions measures at the University of Ottawa.
The University of Ottawa offers its four-year MD program to anglophone and francophone students. The Faculty of Medicine considers applications from candidates with a weighted GPA (wGPA) above the cutoff determined by the admissions committee. The wGPA is calculated using the three most recent years of full-time undergraduate studies. The years are weighted in a 3:2:1 ratio, in reverse chronological order. The MCAT is not used because it cannot be administered for candidates applying to the francophone stream of our bilingual program. Eligible applicants' autobiographical sketches are assessed using a six-item rating scale that quantifies the level of education, employment, volunteering, extracurricular activities, awards, and research contributions. Each item is assessed according to a specific scoring key. One trained faculty rates each category on all applicants' autobiographical sketches. In 2006 and 2007, the reliability (internal consistency) of the autobiographical sketch was 0.68 and 0.69, respectively. Autobiographical sketch scores and the wGPA determine eligibility for a semistructured interview. Applicants are interviewed individually and are scored by three independent raters who subsequently determine a final score through an iterative process. In 2006 and 2007, the interrater reliability among interviewers was 0.89 and 0.93, respectively. Interview scores are then used to generate the priority list for offers of admission. When interview scores are tied, the wGPA is used as a deciding factor.
Mayer-Salovey-Caruso Emotional Intelligence Test
The MSCEIT V2.0 is an online, 141-item, multiple-choice, ability-based measure of EI. Items are scored against the proportion in the normative reference sample of the general adult population (n = 5,000) that endorsed the same answer. Item scores are compiled to generate a total EI score. The MSCEIT scores are reported in a similar fashion to intelligence scales where 100 is the mean reference score and the standard deviation is 15. A person with a score above 115 is thought to demonstrate a high level of EI. Someone scoring below 85 is considered to have perceptions of emotion so unique as to potentially cause him or her interpersonal problems. Using data from our cohorts, the split-half reliability coefficients for the MSCEIT V2.0 were 0.87 and 0.89, and Cronbach alpha was 0.86 and 0.87 for 2006 and 2007, respectively.
Data collection and analysis
The study included two cohorts of candidates applying for medical school in September 2006 or 2007. Potential participants were notified prior to their medical school admissions' interview that they would be asked to participate in this study. All participants provided written informed consent and were advised that their participation in this study would not influence the admissions process. Participants were offered an honorarium of $20 to complete the test. Participants completed the MSCEIT V2.0 in English or French immediately after their interview in March, approximately six months prior to matriculation. Completed tests were scored by Multi-Health Systems Incorporated, the distributor of the instrument. Descriptive statistics and Pearson correlations were calculated for each cohort.
In the spring of 2006, 333/475 (70%) applicants completed the MSCEIT V2.0; in the 2007 cohort, 326/490 (67%) did so. There was no difference in gender between study participants and the entire pool of applicants (female: 58% versus 60% for 2006; 57% versus 57% for 2007). Applicants who participated were not more likely to be admitted than nonparticipants (31.5% versus 31.6% for 2006; 30.7% versus 32.3% for 2007).
The mean total EI score was 97 (range = 56–120, SD = 11) for the 2006 cohort and 97 (range = 41–128, SD = 11) for the 2007 cohort. Female applicants had higher mean total EI scores than male applicants for both cohorts (98.5 versus 94.1 for 2006; 99.1 versus 94.5 for 2007; P < .01 for both cohorts). No statistically significant difference was found between the mean total EI score of applicants who were offered admission or ranked highly enough on the waiting list to reasonably expect an offer of admission versus applicants who were not.
Table 1 shows the correlations, for both cohorts, between the applicants' total EI scores and the wGPA, autobiographical sketch score, and interview score. Among applicants in the 2007 cohort, there was a weak negative correlation between total EI scores and wGPA (r = −0.13, P < .05), but this relationship was not found in the 2006 cohort. In both cohorts, total EI scores did not correlate with the autobiographical sketch score or the interview score.
Few candidates scored below 85 on the EI (50/333 [15.0%] for the 2006 cohort and 44/326 [13.5%] for the 2007 cohort). When analysis was limited to include only students with EI scores <85, EI scores did not correlate with wGPA or the autobiographical sketch score, but a weak correlation was found between total EI scores and interview scores in the 2006 cohort only (r = 0.32, P < .05).
In our cohort, medical students had mean EI scores that were similar to the U.S. adult population and previous samples of medical students.14,16 The range of applicants' EI scores (41–128) extended beyond the range of mean total EI scores (85–115) in the general population, suggesting that our correlations were not adversely affected by any restriction in the range of measured EI scores. Female applicants had higher total EI scores than male applicants. Similar gender differences have also been found in population studies17 and previous studies of medical students.11,15 The difference was, however, small and of questionable relevance.
In this study, EI scores did not correlate with other admissions measures used at our medical school, with the exception of a weak negative correlation between EI scores and wGPA among the 2007 cohort of applicants (r = −0.13). Because EI is conceptualized as measuring a noncognitive attribute, we were not surprised that EI did not correlate with GPA, a purely cognitive measure. In contrast, both the autobiographical sketch and the semistructured interview are thought to measure noncognitive attributes and hence might overlap with the construct being measured by the MSCEIT V2.0 instrument. The lack of any significant correlation between the EI scores and our autobiographical or interview scores led us to conclude that the EI construct being measured by the MSCEIT V2.0 is fundamentally different from the noncognitive traits captured in our admission process. This is further supported by the lack of a relationship between EI scores and current admission decisions.
A limitation of our study relates to the challenges of measuring EI. We chose the MSCEIT because it offers a direct assessment of a person's EI, has acceptable content validity and internal consistency,11 and overcomes bias related to self-report tests. Some groups have questioned the consensus scoring method used for the MSCEIT.18 We agree that consensus scoring might measure conformity tendencies, but we argue that conformity in interpreting and handling emotions might play an important role in a candidate's overall ability to perceive and manage emotions. Although there was no gender or acceptance rate difference between those who agreed to participate and those who did not, self-selection may have biased our samples. Also, our results may not be generalizable because this was a single-institution study and admissions procedures vary in each medical school. However, our findings corroborate data from a recent study that showed no correlation between the MSCEIT scores measured in senior (years 5 and 6) undergraduate medical students and the admissions selection criteria.16 Finally, the predictive validity of MSCEIT scores of medical school applicants is currently unknown. Further research is needed to determine the stability of EI scores and investigate whether EI scores correlate with academic performance, clinical skills, professionalism, and longer-term indicators of success in medical training and practice.
Strengths of our study include using an abilities measure of EI, measuring EI before matriculation, and including both unsuccessful and successful applicants. Additionally, our study, using two consecutive cohorts of applicants, provides the first evidence that MSCEIT scores measured during the admissions process do no correlate with wGPA, autobiographical sketch scores, or interview scores of applicants.
The authors wish to thank T. Rainville, D. Clary, and M.-H. Urro for their assistance, as well as Dr. L.N. Dyrbye for her valuable comments and suggestions.
The study was supported by a grant from the Ontario Ministry of Health and Long Term Care.
This study received approval from the Ottawa Hospital research ethics board.
1Papadakis MA, Teherani A, Banach MA, et al. Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. 2005;353:2673–2682.
2Siu E, Reiter HI. Overview: What's worked and what hasn't as a guide towards predictive admissions tool development. Adv Health Sci Educ Theory Pract. 2009;14:759–775.
3The Future of Medical Education in Canada (FMEC): A Collective Vision for MD Education. Ottawa, Ontario, Canada: Association of Faculties of Medicine of Canada; 2010.
4Albanese MA, Snow MH, Skochelak SE, Huggett KN, Farrell PM. Assessing personal qualities in medical school admissions. Acad Med. 2003;78:313–321.
5Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: The contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34–42.
6Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009;43:767–775.
7Mayer JD, Roberts RD, Barsade SG. Emotional intelligence. Annu Rev Psychol. 2008;59:507–536.
8Brackett MA, Mayer JD. Convergent, discriminant and incremental validity of competing measures of emotional intelligence. Pers Soc Psychol Bull. 2003;29:1147–1158.
9Grubb WL, McDaniel MA. The fakability of Bar-On's emotional quotient inventory short form: Catch me if you can. Hum Perf. 2007;20:43–59.
10Mayer JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2.0. Emotion. 2003;3:97–105.
11Arora S, Ashrafian H, Davis R, Athanasiou T, Darzi A, Sevdalis N. Emotional intelligence in medicine: A systematic review through the context of the ACGME competencies. Med Educ. 2010;44:749–764.
12Stratton TD, Elam CL, Murphy-Spencer AE, Quinlivan SL. Emotional intelligence and clinical skills: Preliminary results from a comprehensive clinical performance examination. Acad Med. 2005;80(10 suppl):S34–S37.
13Austin EJ, Evans P, Magnus B, O'Hanlon K. A preliminary study of empathy, emotional intelligence and examination performance in MBChB students. Med Educ. 2007;41:684–689.
14Borges NJ, Stratton TD, Wagner PJ, Elam CL. Emotional intelligence and medical specialty choice: Findings from three empirical studies. Med Educ. 2009;43:565–572.
15Todres M, Tsimtsiou Z, Stephenson A, Jones R. The emotional intelligence of medical students: An exploratory cross-sectional study. Med Teach. 2010;32:e42–e48.
16Carr SE. Emotional intelligence in medical students: Does it correlate with selection measures? Med Educ. 2009;43:1069–1077.
17Joseph DL, Newman DA. Emotional intelligence: An integrative meta-analysis and cascading model. J Appl Psychol. 2010;95:54–78.
18Roberts RD, Zeidner M, Matthews G. Does emotional intelligence meet traditional standards for an intelligence? Some new data and conclusions. Emotion. 2001;1:196–231.