Subscribe to eTOC

Evidence-Based Guidelines and Performance Measures
How Does Neurology Measure Up?


Neurologist experts and AAN staff discuss the strides neurology is making in guideline development, dissemination, and implementation; as well as the efforts on behalf of the AAN to create meaningful quality measures that improve care for patients.

Compliance with clinical evidence-based guidelines remains low at a time when government agencies are focusing more on improving patient outcomes and streamlining health care costs. But, according to an editorial in the Dec. 5 Journal of the American Medical Association (JAMA), guideline developers could do more to change that trend. By rethinking their goals and developing strategies that address barriers to compliance, “they could eliminate preventable harm, suboptimal patient outcomes or experiences, or waste of resources,” writes Peter J. Pronovost, MD, PhD, director of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine.

Among strategies, Dr. Pronovost suggests that guideline committees include specific checklists of prioritized interventions supported by empirical evidence; that they address possible barriers to change and share successful implementation strategies; and that they reflect multidisciplinary approaches and teams of experts.

To assess how well neurology has done in these areas, Neurology Today interviewed leaders of the AAN committees on guideline development, practice improvement, and quality measures about what goes on behind the scenes of these initiatives.

The AAN's Senior Evidence-based Medicine Methodologist Gary Gronseth, MD, professor and vice-chairman of the University of Kansas Medical Center's neurology department, said, for one, that he had sent out the JAMA piece to members of the AAN guideline committees for consideration and discussion at their next meeting.

He said he supported suggestions mentioned in the JAMA paper, such as working on guideline development with implementation scientists and including an “unambiguous checklist with interventions liked in time and space (e.g. on admission or at clinic discharge).” However, he added, given limited resources and information, some of the suggestions are “aspirational.”

What we definitely can do at the AAN, said Dr. Gronseth, is “make much better and more useful tools for neurologists, which potentially can be incorporated into the electronic medical record [EMR] by vendors,” as suggested by Dr. Pronovost. Furthermore, one improvement already in the works, Dr. Gronseth said, is the merger of two AAN subcommittees — dissemination and implementation with systematic review and guideline development — into one. The goal is to ensure that dissemination and implementation are on the agenda from the very beginning of the guideline discussion.

But how can implementation and dissemination be quantified? And what does success mean in this context? It is difficult to measure, admitted Dr. Gronseth, but “we have an AHRQ [Agency for Health Research and Quality] grant specifically to research that: How well are we impacting both patients and neurologists? Are they aware of what the guidelines found? Based on this knowledge of the guidelines, is there intent to actually change behavior to educate patients better? Or are patients going to change their own behavior based on the guidelines?”

Richard Dubinsky, MD, MPH, professor and residency program director at the University of Kansas School of Medicine, who heads the AAN's Practice Improvement Subcommittee, said some of the biggest challenges to the process are “not knowing if we are making a difference, difficulty in measuring patient level outcomes, the inertia of current practice, and the misunderstanding of guideline conclusions and recommendations.” For example, Dr. Dubinsky noted that the “authority of the conclusions and recommendations” in the guidelines is sometimes confusing to clinicians — “these are not standards that must be followed, but rather guidelines that a physician takes into account along with their own experience and judgment and patient preference, when reaching a decision.”

Thomas Getchius, director of clinical practice at the AAN, said that once a guideline is developed and approved for publication, “there is a separate team of physicians who work with AAN staff to identify the key message, but also the barriers to implementation — whether those are barriers to implementing the guideline itself or barriers to knowledge or awareness around the guideline. The team then identifies primary audiences for the guidelines, as well as who are the other partners and stakeholders involved in the implementation and dissemination process.”

To help committee members frame the guidelines so that they can be optimally disseminated and implemented, the AAN guidelines staff asks panel members to articulate key messages for different audiences, including the public, clinicians, and patients. Among the questions, for example: What changes in practice or intended health outcomes are hoped to be achieved from dissemination of the guideline recommendations? What are the barriers to implementation (i.e. reasons practices that inform the selected guideline recommendations are not currently followed)? What are the opportunities (i.e., resources, programs, and services) that would assist in or prevent implementation of the guideline? Which clinician-oriented organizations ought we to reach out to for dissemination of news of the guideline?

Responding to the JAMA paper's suggestion to include checklists in all guidelines, Getchius said the AAN works hard to develop summary tools for each of its guidelines — “they cannot always be checklists, but we try to develop a clinician's summary which provides a more brief version of the clinical questions and recommendations. We also develop plain language summaries that physicians often give to their patients.”

Indeed, one of the biggest challenges is ensuring that adherence to the guidelines translates into cost-effective, high-quality care. In the Nov. 28 issue of the New England Journal of Medicine (NEJM), Robert A Berenson, MD, Institute Fellow at the Urban Institute, and Deborah R. Kaye, MD, also of the Urban Institute, wrote a perspective piece addressing some of the biggest flaws in the push for value-based payments and physician quality reporting.

“[D]espite being offered various ways to report quality data (by means of a Web interface, registries, and administrative claims), less than 30% of eligible professionals actually report their data to CMS.” These numbers are an indication that physicians “simply do not respect the measures, and for good reason,” the editorialists wrote.

David Z. Wang, DO, director of the OSF/INI Stroke Network and Comprehensive Stroke Center at OSF Saint Francis Medical Center in Peoria, IL, and Co-Chair of the newly formed Quality and Safety Subcommittee (QSS) at the AAN, said that the fundamental challenge in quality measure development for doctors is still “value vs. volume and how they are rewarded.” It is unlikely, he added, that a single system can be used to uniformly measure all practitioners on their performances.

Thus, when the AAN develops its quality measures, “we take into careful consideration appropriate and relevant care settings, exclusions, exceptions, and patient characteristics,” he said. Over the past seven years, the QSS has developed eight quality measure sets and a total of 72 measures for stroke and stroke rehabilitation, epilepsy, Parkinson's, dementia, headache, ALS, distal symmetrical polyneuropathy and muscular dystrophy, Dr. Wang told Neurology Today. “Some of them have been endorsed by the National Quality Forum and used by the Centers for Medicare & Medicaid Services.” PQRS has adopted a number of AAN measure sets, including those for epilepsy, Parkinson's disease, and dementia. The QSS development process is a truly collaborative venture between neurologists and AAN staff members Rebecca Swain-Eng, Gina Gjorvad, and Becky Schierman, noted Dr. Wang.

But it remains a challenge for AAN to evaluate how successfully the measures have been implemented and whether they have improved patient care, said Dr. Wang. The AAN is currently contemplating the feasibility of developing its own data registry like the one used by the AHA/ASA for their Get-With-The-Guidelines data to show the improved practice and outcome of stroke care, in order to be able to provide such information, he added.

The measure sets, along with the guidelines, are also incorporated into the neurology Performance in Practice component of maintenance of certification (NeuroPI), said Donald Iverson, MD, a neurologist with Humboldt Neurological Medical Group in Eureka, CA and the editor of NeuroPI. They also use other societies' guidelines (e.g. AHA/ASA stroke guidelines), or craft their own, in conjunction with QSS and GDS chairs when necessary.

We define “meaningfulness,” said Dr. Iverson, by using measures that have been demonstrated to lead to improved outcomes — such as assessment for fall risks in the home or referral of stroke patients to rehabilitation. “The feedback from participants in NeuroPI on the utility of the measures in improving patient care has been 90 percent positive,” he said; “however, their reimbursement is not contingent upon participation or performance. If it was, we might receive some of the same criticisms in the NEJM article.”

Improved patient care really has to be the ultimate goal of these initiatives, said Dr. Gronseth. “If cost goes down as a result of these efforts, that's great, but that should not be the end game.” There seems to be “a false hope,” he continued, that these sorts of initiatives are going to solve the health care crisis, “but I think that's much bigger than this. In developing many of the guidelines, we end up discovering that we don't know what the best recommendation for treatment is. The NEJM piece certainly emphasized that we don't know how to measure value. It's important that we improve quality of care for the sake of patients first.”

The Institute of Medicine has clearly defined what quality care is for our country, said Dr. Wang — “STEEEP, which stands for Safety, Timeliness, Efficient, Effective, Equal and Patient-centered.” But switching to “value-based care instead of volume-based care” is very difficult because the current reimbursement structure for physicians is still by volume, especially in private practice, he said. For instance, “if a neurologist sees five patients with multiple sclerosis a day and spends quality time with each one of them by addressing all of their concerns, he will be paid for these five visits only but patient satisfaction will be high.

“If the next door neurologist could squeeze in 10 MS patients in a day, he will be paid for the 10 visits — even if some patients felt the doctor did not spend enough time with them.” Eventually, the neurologist who sees five patients will either have to close his clinic, or start seeing more patients in order to stay afloat financially.


The Guideline Development Subcommittee (GDS) follows a well-defined process for the development of an evidence-based guideline, starting from an evidence-based systematic review. The AAN process is designed to rigorously evaluate the strength of the evidence and formulate explicit practice recommendations to improve patient outcomes.

The Quality and Safety Subcommittee (QSS) promotes integrating quality measures into clinical practice through development of measures and reporting of performance excellence.

The Practice Improvement Subcommittee disseminates and implements clinical practice guidelines to improve neurologic practice.

NeuroPI: Performance Improvement in Neurology are clinical modules designed to help neurologists meet the American Board of Psychiatry and Neurology performance in practice requirement for Maintenance of Certification.


•. Pronovost PJ. Viewpoint: Enhancing physicians' use of clinical guidelines. JAMA 2013; 310(23):2501–2502.
•. Berenson RA, Kaye DR. Perspective: Grading a physician's Value — The misapplication of performance measurement. N Engl J Med 2013; 369:2079–2081.
•. Neurology Today's interview with Donald Iverson, MD: “What You Can Do to Improve Your Practice”:
    •. More Neurology Today stories on practice improvement:
      •. AAN Performance Improvement in Neurology:
        •. AAN Resources for Quality Measures: