Article In Brief
Every medical school that withdrew from participating in US News & World Report's annual rankings has pledged to provide prospective students and the public with data about the schools' performance, curriculum, and standards.
Nine of the nation's top-ranked medical schools announced in January that they will no longer participate in US News and World Report's annual rankings, citing philosophical differences with the system's emphasis on factors such as standardized test scores. Harvard Medical School led the exodus on Jan. 17 with a letter from faculty of medicine Dean George Q. Daley, MD, to the campus community.
On Jan. 20, Columbia University Vagelos College of Physicians and Surgeons followed suit, with a letter from Dean Katrina Armstrong, MD, that said the rankings “perpetuate a narrow and elitist perspective on medical education.”
And within the next week, Stanford University School of Medicine, University of Pennsylvania Perelman School of Medicine, Icahn School of Medicine at Mount Sinai, University of Chicago Pritzker School of Medicine, Washington University School of Medicine in St. Louis, Weill-Cornell, and Duke University School of Medicine also announced their withdrawals.
Acknowledging that prospective students and the public should still have access to data about these schools' performance, curriculum, and standards, each school that withdrew from the US News rankings has pledged to provide comprehensive information, focused on the measures they see as most important, on their own websites.
“Our metrics will reflect and assess the efforts and accomplishments of our faculty in education, research, and patient care as well as the innovation and impact of faculty and trainees on biomedicine and their roles in developing tomorrow's leaders,” Stanford University School of Medicine Dean Lloyd Minor, MD, wrote in his Jan. 23 letter to faculty.
“Our reporting will also represent our tripartite mission and key priorities that our students have identified as important to their educational experience, including access to extensive patient care and research opportunities. Moreover, our process will reflect our core values, emphasizing diversity, equity, and inclusion, and will ensure that our metrics are measurable, verifiable, and transparent.”
Support from Neurology Educators
Neurology educators at several leading medical schools expressed strong support for the schools' decision to withdraw from the rankings system. Lisa Merlin, MD, FAAN, distinguished professor and vice chair for education in the departments of neurology, physiology, and pharmacology at SUNY Downstate Health Sciences University, likened the trend to the decision a few years ago by several medical schools to suspend student nominations to the Alpha Omega Alpha (AOA) honor society, protesting a racially inequitable selection policy, which was largely determined by grades and scores on standardized tests.
“This served to inspire AOA to update and revise its selection criteria to make it more equitable and increase inclusivity,” she noted. “And many of the same schools are also trying to get US News to reassess how it determines its hospital rankings as well.”
“Rankings are an expression of values,” said Douglas Larsen, MD, MEd, professor of neurology and pediatrics and vice chair for education at Washington University School of Medicine. “The US News measures focus primarily on things such as test scores, GPA, selectivity, and research dollars. None of those is a direct measure of a student's educational experience, and many of those metrics have a direct effect on maintaining and perpetuating structural racism. We know that the schools that heavily depend on test scores and GPAs have the least diverse student bodies; if you just depend on those numbers, you are not able to account for background differences and life experiences, and you end up with a highly skewed student body that doesn't represent the United States and perpetuates the inequities that have existed in our society for centuries.”
Washington University's curricular reform process was a major driver of its decision to pull out of the rankings, Dr. Larsen said. “We made health equity a centerpiece of our curriculum, and in order to walk the walk, we have had to look at our own admissions policies, our own relationship with our community, and really ensure that we have adopted antiracist principles. You can't do that and then be complicit in another system that is built on structural racism.”
The rankings also self-perpetuate, said Tracey Cho, MD, FAAN, clinical professor of neurology and vice chair for education at the University of Iowa School of Medicine. “They elevate the institutions that have traditionally had brand-name prestige and the most money, with metrics that skew toward students who have resources and are good test-takers,” he said.
“Then, those students who have high GPAs and high MCATs will want to go to those highly ranked schools, and the cycle continues. But these ‘entrance scores’ are well established to be numbers only, biased toward White males who have traditionally had more of those resources and opportunities and not reflective of who's going to be an effective physician,” Dr. Cho said.
“Medical schools have moved in recent years to a much more inclusive approach to admissions, taking into account factors such as obstacles students have overcome, how much mentoring they have had, and other characteristics they would bring that would be good for the medical school, good for the profession, and good for patient care.”
Other Flaws in the System
Dr. Cho also objects to the US News' separation of medical schools into two categories of “best in research” and “best in primary care.”
“That's an artificial, binary division,” he said. “And for the research schools, one key criterion is total federal research dollars, and that particular metric is based on size alone. More faculty means more federal grants, which also becomes a self-fulfilling prophecy in terms of getting more grants. It disadvantages smaller schools that might also have outstanding research programs and might be more appropriate for certain students.”
Another driving factor in the rankings is “reputation,” assessed by a survey of medical school deans, department chairs, and residency program directors. “That survey had a 28 percent response rate last time,” Dr. Cho said. “As you can imagine, many medical school deans and department chairs may not be intimately familiar with other schools' curricula and quality of education, so it comes down to brand names and marketing budgets.”
US News recently began to rate schools in certain individual specialties, too, including anesthesiology, family medicine, internal medicine, obstetrics and gynecology, pediatrics, psychiatry, radiology, and surgery. These fields were chosen to best reflect the curriculum taught at medical schools, according to the publication's methodology statement, Dr. Cho said. “That's an implicit message that neurology is not an important specialty for a medical school's quality, which I think does a disservice to our field and hurts the reputation of the specialty and neurology as a pipeline when there is already a shortage of neurologists. It may seem like a minor thing, but I think it has a negative impact on the field of neurology and recruiting.”
Ending the Numbers Game
Moving away from these rankings is a particularly important opportunity for neurology, said Dr. Larsen. “Our field really needs to grow in its diversity, and having medical schools emphasize that and remove the structural barriers to that diversity is going to be important for neurology in the long run,” he said. “It will require years for this to play out, for us to create these diverse cohorts and have those students move through the system and become residents and faculty, but it is essential if we are to have a neurology workforce that can meet the needs of a diverse population.”
He likens the ranking system to diagnostic shortcuts common in neurology. “Think of the muscle strength scale or the reflex scale,” he said. “We love our shortcuts, but they only work if everyone knows what we're talking about and the numbers mean the same thing to everyone. The US News methodology is an ineffective shortcut rather than an assessment of the actual educational experience. The entire premise is fundamentally flawed.”
“I think most people recognize that these rankings are, and should be, only one factor in selecting a program. I don't think most students use US News as their primary measure in evaluating medical schools. I agree that it's an outdated and flawed ranking system, but the more important conversation centers around how we can improve and update measures of quality that are more consistent with contemporary times.”—DR. KOTO ISHIDA
“Our field really needs to grow in its diversity, and having medical schools emphasize that and remove the structural barriers to that diversity is going to be important for neurology in the long run. It will require years for this to play out, for us to create these diverse cohorts and have those students move through the system and become residents and faculty, but it is essential if we are to have a neurology workforce that can meet the needs of a diverse population.”—DR. DOUGLAS LARSEN
Koto Ishida, MD, associate professor of neurology and associate chair of education for the department of neurology at NYU Langone Grossman School of Medicine, which has made a public announcement that it will continue submitting data to US News, predicted that the withdrawals wouldn't change much about how prospective medical students evaluate their educational choices.
“I think most people recognize that these rankings are, and should be, only one factor in selecting a program,” she said. “I don't think most students use US News as their primary measure in evaluating medical schools. I agree that it's an outdated and flawed ranking system, but the more important conversation centers around how we can improve and update measures of quality that are more consistent with contemporary times. With many interviews still largely virtual and the difficulty of visiting every program in person, it's up to the program to tell you what they think is great about their school.”
Dr. Cho agrees that developing a more appropriate ranking system would be difficult. “I'm not sure there's a mechanism for US News and World Report to have a more useful or reflective ranking system that actually measures educational quality,” he said.
“These metrics are very hard to develop, and it's not something that you can just plug in simple numbers that apply to all schools equally. More accurate measures, which are more qualitative, are a lot more work, and I don't know how practical it would be to change the ranking system in a way that meaningfully captures educational quality. It's a popular tool for medical students because it gives you a number that at least has some surface evidence-based factors going into it. When you pull back the curtain, it's clearly not achieving what's in the students' best interest, but right now, there is no single source to replace it.”
The curricular reform that has swept US medical schools over the past two decades also made rankings and comparisons among these schools more difficult than before. “Medical school curricula used to be pretty homogeneous, with all the same courses and similar structures,” Dr. Larsen said. “But there are now so many diverse means by which medical schools are differentiating themselves in their curricula and educational approaches, it will be difficult for people to come up with a standardized ranking system in the future.”
None of the institutions whose medical schools have withdrawn from the US News ranking system have given any indication that their hospitals would do the same; several have specifically said that they will not. “Medical school and hospital rankings are separate and independent and use different methodologies,” wrote Stanford dean Dr. Minor.
“Medical schools have moved in recent years to a much more inclusive approach to admissions, taking into account factors such as obstacles students have overcome, how much mentoring they have had, and other characteristics they would bring that would be good for the medical school, good for the profession, and good for patient care.”—DR. TRACEY CHO
Dr. Cho hopes students and the public do not turn to the hospital rankings as a proxy for the associated medical school. “Anyone who's worked in any role in hospital administration will know that it's a game where you try to move your pieces to match what they are calculating, and one factor can take you from the top 10 to somewhere in the 30s, without any change in what you're doing to care for your patients,” he said.
He called for neurology educators to work together to help identify meaningful measures of teaching effectiveness. “These are complex and not easily generated, but it's our job to work out how you assess whether the teaching of a medical student is effective, in terms of goals like patient outcomes and how prepared your students are when they leave your school to take on the care of patients. That's the ultimate goal—we want to make better doctors. If neurology educators can help identify a group of metrics that can more accurately assess the effectiveness of our teaching, and provide meaningful data that students and the public can use to inform their decision-making, that would be extremely valuable.”
