Performance on the Medical College Admission Test (MCAT) is a significant factor in the decision to admit an applicant to medical school; thus, the validity and use of the new MCAT exam is intertwined with questions about who should and will be the doctors of tomorrow, particularly as arguments grow about the importance of a diverse physician workforce for the health of our population. To answer these questions, the medical education community has focused largely on the medical school admissions process to identify the “right” students.
In this issue, Busche and colleagues offer evidence that the new MCAT exam remains a valid predictor of performance in the preclerkship curriculum and that it is not systematically biased against any racial or socioeconomic groups.1 Students who do well on the new MCAT exam are as well or better prepared for a career as a physician as those who took the previous version of the exam, and they are likely to reason better and can apply knowledge of important concepts in the social and behavioral sciences. However, whether due to less access to preparation materials2 or inequalities in their premedical education,3 students from lower socioeconomic groups do not perform as well on the new MCAT exam, which has led some to advocate that performance on the MCAT exam is given too much weight in the admissions process.3,4
As educators struggle to define the characteristics of the “right” candidates for medical school and design processes to identify and admit those applicants, we should consider the message we may be sending inadvertently by calling for the MCAT exam to play a reduced role in admissions decisions.
The Role of the MCAT Exam in Admissions
Several years ago, I had the good fortune to serve on the 5th Comprehensive Review of the MCAT Review Committee (MR5 Committee), convened by the Association of American Medical Colleges (AAMC) to help guide the development of the new MCAT exam. The committee was a large group that included faculty and premedical advisors from colleges and medical schools across the United States. Both public and private institutions were represented. Over the course of the 3 years that the committee met, our discussions were wide ranging, earnest, and reflective of the competing priorities of the members.5 We wanted future doctors with intellectual rigor, strong critical thinking skills, good communication skills, and who were humanists.
The committee recommended that the MCAT exam be changed to emphasize critical thinking skills (e.g., the ability to analyze and interpret data) rather than factual recall. In addition, a section on the behavioral and social sciences was added to signal that being an excellent doctor requires more than knowledge of the biological and chemical sciences. In retrospect, I think we spent less time than we should have on other important traits like curiosity and resilience (or “grit”), as curiosity about the patient as a person and about the pathophysiology of her problem is one way to bridge differences between patients and doctors and to minimize cognitive bias and error.6 Becoming and being an excellent doctor is challenging intellectually, emotionally, and physically; if passion and perseverance are the essence of grit,7 we surely need those traits in our doctors. Of course, there are limits to what one can learn from a standardized exam score, yet holistic admissions processes are designed in large part to help us discern the presence of these characteristics in applicants.
Nevertheless, the MCAT exam is the starting point. Scores are predictive of students’ performance in the first phase of medical school and of their ability to learn, digest, and apply the scientific principles needed for the practice of medicine. However, the premedical curricula available at colleges likely affects how applicants perform on the MCAT exam, so individuals from lower socioeconomic groups and those attending under-resourced colleges may well be disadvantaged when they take the MCAT exam. Because some medical schools place a high value on MCAT scores, the exam has been perceived to be a barrier to achieving greater diversity in the physician workforce.3,4 How then do we determine what role the MCAT exam should play in the admissions process?
Some members of the MR5 Committee advocated that the new MCAT exam be pass/fail. Others pointed out that further distinctions among students are needed since there are more students applying than there are slots available at medical schools. If admissions committees cannot assess applicants’ academic performance using their MCAT scores, they will default to using other measures, such as grade point average and the rigor of the applicant’s undergraduate institution, potentially disadvantaging students from lesser known, under-resourced colleges. With the assistance of researchers at the AAMC, the MR5 Committee modeled the impact of changing the MCAT exam to pass/fail. This analysis demonstrated that many students from lower socioeconomic groups who were admitted to medical school in the present system of reporting actual MCAT scores would have been excluded in a pass/fail system. Consistent with this observation, Terregino and colleagues found that schools that accepted more applicants with midrange MCAT scores had more diverse matriculating classes.4 These findings suggest that the holistic admissions process works.
Graduation rates for these students remain high. But when only 2% of students fail to complete medical school for academic reasons,8 we must also be sure that we have not lowered our expectations and standards during medical school while arguing for less emphasis on MCAT performance when making admissions decisions.
Is Competence Enough?
Over the last 2 decades, the idea of admitting applicants who are merely “smart enough” for medicine has emerged. Competency-based medical education (CBME) is designed to ensure that students do not advance through medical school unless they are competent; however, “competent” generally has been defined as the bare minimum required for acceptability. While the original notion of CBME was that students would remain in place until they achieved competence (or were counseled to pursue another profession if they failed to improve after remediation), today much greater attention is devoted to shortening training by moving students along as soon as they achieve competence.9 Less attention is given to the fact that competence is context specific and that experience (i.e., seeing many patients under varying conditions) is an important element in the development of meaningful competence.
The implications of having fewer medical school experiences, if students move through the curriculum more quickly, are compounded by the changing environment at the residency level. As a medical intern, I evaluated more than 400 patients whom I admitted to the hospital. In contrast, a typical intern today personally admits fewer than 100 patients. The failure of educators to consider the importance of experience for their learners has led to concerns about the true competence and autonomy of residency graduates.10,11
Entrustable professional activities (EPAs) theoretically would set the bar higher than “competence,” but a recent review of the literature revealed that the data do not yet support the use of this approach as an assessment tool.12 The inter-rater reliability for determining entrustability is fair at best.13 Frame of reference training,14 which requires a major investment in time and resources for a medical school with many faculty, may be necessary to truly attain validity in EPA decisions. Furthermore, few faculty members are willing to commit to stating that a student cannot proceed to the next phase of the curriculum because of EPA ratings.
From my experience and what I have heard from colleagues around the country, it is difficult to slow down, if not stop, a student’s progression through medical school because of academic performance issues. A former chair of the Harvard Medical School promotions and review board observed that faculty view their students like their patients, that is, they never abandon them, and multiple students over the years have described their medical school experience not as “pass/fail” but as “pass/pass.” Failure is not truly a grading option for most faculty, and students seem to know this. There are invariably mitigating reasons for the poor performance, claims that educators will address the deficits in the next phase of the curriculum, or concerns about how the student will perceive the setback. Too often I have heard that deficiencies will be addressed by the student’s residency program.
While much has been written about “productive failure,” we as educators have become intensely protective of our students.15 We do not allow them to fail. Additionally, if we move a student along for several years before concluding that a career in medicine is not right for that person, we are reluctant to remove her from school because she may be saddled with significant debt and no degree.
As educators, we are in a unique position. We have a responsibility to our students to support and prepare them for practice, but we also have a responsibility to the public to produce excellent doctors. We know that MCAT scores predict performance in the early years of medical school.1 We should not discount their role in identifying who those excellent doctors may be.
An Alternative Solution
About 15 years ago, as discussions about the role of MCAT scores in the admissions process became more acute, I remember hearing what I thought was a perplexing argument at national meetings. Prominent educational leaders seemed to create a false dichotomy: We can have “smart” doctors or “compassionate, humanistic” doctors. I think there is a third option: We can and should have doctors who are both.
With that goal in mind, there is probably no magic formula for picking the “right” medical students. In fact, there are many types of “right” students, given the range of clinicians, researchers, and educators we need. Yet even in meeting these needs, we cannot make major compromises. For example, the doctor with a great academic record and research potential may care for patients one day, so she must also have the requisite interpersonal and communication skills to graduate. Similarly, the doctor who is a great communicator also needs the cognitive acumen to think critically and avoid the pitfalls of pattern recognition and diagnostic error. During our multifaceted admissions process, I believe that it is acceptable to take a risk on a promising applicant who may not rank as highly as someone else in one or another of the attributes needed to be an excellent doctor, but we must also commit to truly maintaining high standards for our students and to working hard to improve students’ areas of weakness during medical school. If remediation is not successful, however, we must redirect students to other career paths.
Ideally, we should not need to take risks in the admissions process. As a society, we are more likely to fix problems after they arise than to invest in their prevention. For example, we know that the social determinants of health lead to untold misery, yet we invest in treatments for the resulting diseases rather than address the root causes of the problem. The MCAT exam is an accurate assessment of an applicant’s preparation in the foundational principles of the biological, chemical, and social sciences needed for the study of medicine. If as a community our efforts to foster a more diverse physician workforce are being thwarted by socioeconomic factors that impair the ability of talented students to compete effectively, we should address the true cause of the problem (e.g., inadequate preparation in high school and college to handle the intellectual challenges of medicine), rather than create a workaround once those students reach medical school.
Potential solutions include reaching out to under-resourced colleges and universities with predominantly underserved student populations to work with those students who entered college unprepared for the rigorous premedical curricula or with those who do not have access to such curricula. We also could use new educational technology, such as the HMX online courses,16 to supplement the premedical curricula at these under-resourced colleges. We could develop more postbaccalaureate programs designed specifically for disadvantaged students to help them enter medical school with the necessary academic foundation to succeed. Finally, we can encourage our own universities to invest faculty time and resources in the development and implementation of these kinds of programs so that all students have the opportunity to master the premedical curricula and demonstrate their talents on the MCAT exam and beyond in medical school. Together, we can create a physician workforce that is made up of smart, humanistic, and diverse doctors, without making compromises during the medical school admissions process.
The author thanks David Roberts, MD, and Steven Weinberger, MD, for their review of an earlier draft of this article.
1. Busche K, Elks ML, Hanson JT, et al. The validity of scores from the new MCAT exam in predicting student performance: Results from a multisite study. Acad Med. 2020:95:387–395.
2. Girotti JA, Chanatry JA, Clinchot DM, et al. Investigating group differences in examinees’ preparation for and performance on the new MCAT exam. Acad Med. 2020:95:365–374.
3. Lucey CR, Saguil A. The consequences of structural racism on MCAT scores and medical school admissions: The past is prologue. Acad Med. 2020:95:351–356.
4. Terregino CA, Saguil A, Price-Johnson T, Anachebe NF, Goodell K. The diversity and success of medical school applicants with scores in the middle third of the MCAT score scale. Acad Med. 2020:95:344–350.
5. Schwartzstein RM, Rosenfeld GC, Hilborn R, Oyewole SH, Mitchell K. Redesigning the MCAT exam: Balancing multiple perspectives. Acad Med. 2013;88:560–567.
6. Simpkin AL, Schwartzstein RM. Tolerating uncertainty—The next medical revolution? N Engl J Med. 2016;375:1713–1715.
7. Duckworth A. Grit: The Power of Passion and Perseverance. 2016.New York, NY: Simon and Schuster, Inc..
8. Garrison G, Mikesell C, Matthew D. Medical school graduation and attrition rates. AAMC Analysis in Brief. April 2007;7:1–2.
9. Raymond JR Sr, Kerschner JE, Hueston WJ, Maurana CA. The merits and challenges of three-year medical school curricula: Time for an evidence-based discussion. Acad Med. 2015;90:1318–1323.
10. Pellegrini VD Jr.. Perspective: Ten thousand hours to patient safety, sooner or later. Acad Med. 2012;87:164–167.
11. Hashimoto DA, Bynum WE 4th, Lillemoe KD, Sachdeva AK. See more, do more, teach more: Surgical resident autonomy and the transition to independent practice. Acad Med. 2016;91:757–760.
12. Meyer EG, Chen HC, Uijtdehaage S, Durning SJ, Maggio LA. Scoping review of entrustable professional activities in undergraduate medical education. Acad Med. 2019;94:1040–1049.
13. Kelleher M, Kinnear B, Sall D, et al. A reliability analysis of entrustment-derived workplace-based assessments [published online ahead of print September 17, 2019]. Acad Med. doi: 10.1097/ACM.0000000000002997.
14. Newman LR, Brodsky D, Jones RN, Schwartzstein RM, Atkins KM, Roberts DH. Frame-of-reference training: Establishing reliable assessment of teaching effectiveness. J Contin Educ Health Prof. 2016;36:206–210.
15. Lukianoff G, Haidt J. The Coddling of the American Mind. 2018.New York, NY: Penguin Press.
16. Harvard Medical School. HMX courses. https://onlinelearning.hms.harvard.edu/hmx/courses
. Accessed November 5, 2019.