Evidence-Based Medicine: What Neurologists Need to Remember During the COVID-19 Pandemic
By Orly Avitzur
May 21, 2020
Article In Brief
At a time when clinicians are anxious to find effective treatments for COVID-19, the AAN's guidelines expert underscores the importance of evidence-based medicine to avoid rushing treatments without understanding potential toxicity and adverse events.
On March 24, ProPublica reported that some doctors were hoarding unproven COVID-19 medicine by writing prescriptions for themselves and their families. Notably, pharmacists were seeing a surge in unusual and fraudulent prescriptions for chloroquine and hydroxychloroquine (Plaquenil) after President Donald Trump first promoted them on March 19 as possible treatments for COVID-19, the coronavirus, based on a small study suggesting that the combination of hydroxychloroquine and azithromycin (Z-Pak) could be effective in treating COVID-19.
The following day, Anthony S. Fauci, MD, director of the National Institute of Allergy and Infectious Diseases cautioned in a press conference: “...the information that you're referring to specifically is anecdotal, it was not done in a controlled clinical trial. So, you really can't make any definitive statement about it.”
Gary S. Gronseth, MD, FAAN, professor and chair of neurology at the University of Kansas, evidence-based medicine (EBM) consultant to the AAN, agrees. “From a broad perspective, EBM helps us to distinguish what we want to believe from what we really know,” he said. “The wish for an effective pharmacologic treatment is illustrative; everyone wants to believe that there is an effective drug available right now,” he added.
“Even the suggestion of effectiveness is enough to get physicians—who should know better—to write prescriptions of hydroxychloroquine for themselves and loved ones—just in case,” he said. “In fact, this drug can be very toxic, and has shown efficacy only in very high [toxic] doses in basic science studies,” he continued. “More likely than not, it will turn out not to be helpful and even somewhat harmful,” said Dr. Gronseth, explaining that most untested therapies turn out not to be effective—people typically get the side effects and no benefit.
“Our hopes and fears are powerful things that drive many of our decisions. Now is a time that we need good evidence, not fear, to make decisions.” The EBM perspective helps us to make this distinction, he added. “Fortunately, the WHO and FDA are sponsoring studies that should provide answers relatively soon.”
A few weeks prior to the outbreak of COVID-19, Neurology Today had asked Dr. Gronseth to reflect on the guidelines process at the AAN, one that has spanned the past 27 years of his career. In 1993, Dr. Gronseth was asked to write the first AAN evidence-based guideline on EEG in headache, and soon thereafter, invited to join the AAN Guideline Committee. His ensuing contribution to EBM in neurology has been immeasurable. His comments are edited and excerpted below.
What were the early days of guidelines development like at the AAN?
We were finding our way in the evidence-based world and learning how to conduct systematic reviews, comprehensive literature searches, perform data extraction, and rate the quality studies. It was like drinking from a fire hose. Over time, the guidelines became extremely rigorous. Because they were so evidence-based, it was often difficult to make useful recommendations. Subsequently, we developed a consensus-based process that allowed us to go a bit beyond the evidence and develop more useful recommendations. More recently, we have become more quantitative, routinely performing meta-analyses and formally incorporating feasibility, patient preferences and cost considerations in the recommendation development process.
What would surprise readers to hear about the guidelines process?
I am unsure if members understand how much effort is involved in creating guidelines. If we were to hire an evidence-based center to do a systematic review, it would cost around $1 million. The five members of the AAN guidelines staff—two of whom are librarians—and the volunteer neurologists on the Guideline Subcommittee perform multiple systematic reviews every year, along with the help of a few methodologists. It's a herculean effort.
Which guideline do you think has been most impactful?
They are all meaningful to me, and there is a story behind each guideline. However, the EEG in headache guideline is the one which changed the course of all future AAN guidelines, I think. Prior to its publication in 1995, a lot of EEGs were routinely ordered in headache patients. Probably to no one's surprise now, a systematic review of the literature demonstrated that this was unnecessary and the AAN published a guideline against this practice. Publishing that guideline told the medical community that the AAN was serious about doing the right thing, even though it meant that some of our members would not be paid for those services any longer. Another impactful guideline was the 2000 guideline on thymectomy for myasthenia gravis, which concluded that we did not know if this invasive procedure was effective. After its release, the neuromuscular research community decided to perform a rigorous randomized controlled trial to finally answer the question. (A new practice parameter update on thymectomy for myasthenia gravis was subsequently reported on March 25.)
Which guideline has been most controversial?
In 2012, the Agency for Healthcare Research and Quality (AHRQ) contracted with the University of Minnesota's evidence-based group to do a systematic review of disease-modifying therapies (DMT) for the treatment of multiple sclerosis (MS). The evidence suggested that in some patients it may be appropriate to stop DMT if they had progressed to secondary progressive MS. The MS medical community protested, and the AAN guidelines team was tasked with doing our own systematic review regarding disease-modifying therapies in MS. This process, led by Alexander D. Rae-Grant, MD, FAAN, and Tamara M. Pringsheim MD, FAAN, was difficult. The rhetoric that ensued was often intense and accompanied by accusations of conflicts of interest. In the end, the findings were similar, and the AAN was able to produce a useful set of nuanced practice recommendations.
Another guideline which met with controversy was the tPA guideline the AAN attempted to do in conjunction with the American College of Emergency Physicians (ACEP). The collaboration began in the 2000s and was stalled after resistance from some members of the emergency medicine community. We finally convinced the ACEP to publish a guideline supporting the use of tPA for stroke patients. Subsequently, after an uproar from some ACEP members, it was retracted, and replaced with a new ACEP guideline that in many ways supported skepticism of the effectiveness of tPA in acute stroke.
Most guidelines are controversial in one way or another. Intense discussion—even between subcommittee members during guideline development—is the norm. The evidence is usually not clear cut and we all have different biases, different perspectives, and differences in treatment habits. Despite that, and even after strong disagreements that highlight these differences, some of my closest professional relationships and friendships are with people I have had extremely intense arguments with on the subcommittee; it's a fascinating dynamic.
How do you deal with conflicts of interest in guideline development?
Our criteria have become stricter over the years, but there is a delicate balance because we need content experts, and they often have conflicts of interest. Currently, anyone who has a significant conflict of interest is not allowed to participate in the systemic review process. However, people with conflicts are allowed to have input into the recommendation development process. We ensure that more than half the people voting on recommendations are unconflicted and use a modified Delphi process to ensure that all voices are heard. Of course, the Guideline Subcommittee, along with its supervising committee and the AAN board of directors, have ultimate authority over the recommendations. There are a lot of checks and balances.
Can you respond to practicing neurologists who are concerned that guidelines are too proscriptive?
It's often a catch-22 for the Guideline Subcommittee: if we put out a guideline that doesn't make strong recommendations, we hear complaints that we didn't advise clinicians what to do. On the other hand, if we put out a guideline that makes a strong recommendation, we hear complaints like, “Don't tell us what to do.” Of course, guideline recommendations are much more nuanced than that. Guidelines summarize the evidence and highlight its deficiencies. They also emphasize that it always comes down to the judgment of the neurologist and patient, as described in a 2018 Neurology Clinical Practice article, “Approach to assessing and using clinical practice guidelines.”
How would you like to see guidelines evolve over time?
I want them to become even more quantitative eventually using bias adjustment and, when appropriate, probabilistic decision analyses. Quantification of effect measures, patient values, and costs are enormously useful in helping us make decisions. Since joining the committee, developing tools to facilitate more quantified analyses has always been a goal. I figure that by 2030, the quantification framework for the subcommittee will be in place. Once that happens, I will retire—or maybe not—there are always ways to help to make the process better.
Dr. Gronseth had no disclosures.