ARTICLE IN BRIEF
Chairs of various AAN committees that develop guidelines respond to the Institute of Medicine recommendations for “gold standards” in guidelines development.
The AAN already comes close to meeting the new standards issued by the Institute of Medicine (IOM) in March for developing practice guidelines based on rigorous review of the medical literature, according to members active in the process. However, conflict-of-interest policies must be tightened, public and patient input must be added to the process, and guidelines must strive to provide more specific recommendations for clinicians, they told Neurology Today.
The two IOM reports — “Finding What Works in Health Care,” and “Clinical Practice Guidelines We Can Trust” — attempt to establish a “gold standard” for assessing medical evidence and applying the evidence to the development of practice guidelines. In an effort to increase trust in practice guidelines, the IOM issued eight recommendations. (See “IOM Recommendations: Clinical Practice Guidelines We Can Trust.”)
The AAN committees dedicated to evidence assessment and guideline development already follow these recommendations in spirit, and are moving closer to coinciding with them, according to Christopher T. Bever Jr., MD, professor of neurology at the University of Maryland Medical School and chair of the AAN Quality Measurement and Reporting (QMR) Subcommittee.
“I think the AAN is well positioned to comply with the IOM recommendations,” he said. “We're way ahead of many other organizations, but there are some areas where we'll have to restructure, invest additional resources, and develop additional procedures.”
“We don't have to develop a wholesale new system to come into compliance,” said Gary Gronseth, MD, professor and vice chair of neurology at the University of Kansas Medical Center, “but we do have to do some tweaking, some of it very resource-intensive.”
HOW TO MANAGE CONFLICTS
One of those tweaks will involve the AAN's monitoring of conflicts of interest among members who serve on committees that review medical literature and develop practice guidelines. Currently, the AAN allows members with potential conflicts of interest to serve on such committees.
According to the new IOM guidelines, people with conflicts should constitute a minority of a committee's members, and the chairman must be free of conflicts.
“We need content experts to participate in guideline development because they have the knowledge needed to understand the nuances of the literature, and to develop the key clinical questions,” said Dr. Gronseth, the AAN's consultant for evidence-based medicine. “Often, however, such experts have potential conflicts of interest. For example, they may obtain income from performing a procedure that is the topic of the guidelines. Rather than completely exclude potentially conflicted experts from guidelines development, the AAN has included some, but to minimize bias at least half the panel members must be free of conflicts.”
Dr. Bever himself has been cutting financial ties with companies that manufacture products that may be involved in practice recommendations. “I've tried to free myself of conflicts as much as I could,” he said. “I think it is possible for people to restructure their lives so that they are not on speaker panels for companies, or taking honoraria or other gifts.”
In addition, the IOM recommends that potential committee members should be required to disclose all planned activities that could present a conflict in the future, as well as any activities such as research that could indicate intellectual bias.
“Currently we don't ask people to disclose planned activities,” said John D. England, MD, Grace Benson Professor and chair of neurology at Louisiana State University, and chair of the AAN Guidelines Development Subcommittee. “And intellectual bias is hard to measure. We do a good job of managing conflict of interest, but we're probably going to have to ask a few more questions when we assess individuals who will contribute to the development of clinical practice guidelines.”
Another change will involve the inclusion in the review process of “gray” literature, such as abstracts from meetings, and studies not necessarily published in peer-reviewed journals.
“To minimize reporting bias the IOM recommends that non-peer-reviewed evidence be included,” said Dr. Gronseth. “This means doing a hand search of the literature because such articles often are not indexed in online databases, such as Medline. Such searches are very resource intensive.”
TAKING GUIDELINES PUBLIC
The IOM also recommends that patients and/or patient advocates be included in the clinical guideline development process — at least when clinical questions are formulated and the draft guidelines are reviewed, and that strategies should be in place to increase their effective participation by offering training in the appraisal of evidence.
“We've already started to do this,” said Dr. England. “It's a hard thing to do. It requires extra funding, and a commitment to finding the right patients and organizations. That can be problematic. In a new collaborative project we're putting together – a muscular dystrophy project under the auspices of the AAN, the American Association of Neuromuscular Electrodiagnostic Medicine, and the CDC – we do have patient advocacy groups helping us. However, we may not be able to do that for every single guideline.”
The IOM also recommends a reporting period for public comment, which the AAN currently does not have. Richard Dubinsky, MD, MPH, chair of the AAN Practice Improvement and vice-chair of the Quality Measures Reporting Subcommittees, believes that establishing a public comment period for proposed practice guidelines is a good idea even though they already undergo exhaustive review.
“Each guideline, for example, undergoes about 80 reviews before publication,” said Dr. Dubinsky, professor of neurology and program director at the University of Kansas Medical Center in Kansas City, KS. “We also solicit reviews from organizations that would have a stake in the guidelines. For example, with the recently published guidelines on painful diabetic neuropathy the committee invited comment from the American Diabetic Association and the Neuropathy Association. All those reviews and comments have to be answered by the author panel. Then it goes up to the practice committee, and to the board, and from there it goes to neurology for another blinded peer review. You get the most carefully reviewed papers anyone could ever have.”
But, according to Dr. England, having a reporting period for public comment could pose a problem for some organizations, said Dr. England. “At a recent meeting with the IOM, some society representatives said they would never meet that guideline because of the journal embargo requirement,” he said. “Guidelines are ultimately published in journals, and journals, including Neurology, have embargoes.”
However, this should not be a problem for the AAN, according to Dr. Gronseth. “The IOM's additional requirement means putting guidelines up on the web and soliciting comments from all stakeholders,” he said. “For many professional societies, this may be a major issue, but we've discussed this with [Robert Gross, MD] the editor-in-chief of Neurology, and he doesn't think this will be a problem because the posted document will be a draft, not a finished document. We'll have to work through that.”
WHEN EVIDENCE IS AMBIGUOUS
The IOM also recommends that practice guidelines provide more specific guidance than the AAN typically provides when evidence is ambiguous. For example, after reviewing the literature on the use of plasmapheresis for treating myasthenia gravis patients in crisis, the AAN guidelines note the absence of randomized clinical trials that might provide evidence of the effectiveness of the treatment.
“Because the available evidence in the peer-reviewed literature was not of the highest quality, we had to give it a level U recommendation, which means the level of evidence was insufficient to support or refute the use of plasmapheresis for the treatment of myasthenia gravis patients in crisis,” said Dr. England. “Our recommendation is kind of squishy — we leave the decision up to the doctor's judgment. The problem is that almost everybody who's a neuromuscular expert uses plasmapheresis to treat myasthenia gravis patients in crisis, and the general consensus, based on clinical judgment, expert opinion, case studies and so on, is that it's useful.”
The IOM advises including such information, which would provide “actionable” recommendations that physicians could use to inform their judgments.
Dr. Gronseth had hoped for more guidance from the IOM on developing such actionable recommendations. “We've always struggled with this when developing guidelines, and we had hoped the IOM would provide more guidance about what to do when the evidence is insufficient,” he said. “There's often tension between offering the judgment of experts on the panel and recommending informed decision making — telling the neurologist that the evidence doesn't indicate the right thing to do. Our philosophy has always leaned toward informed decision-making — if we don't find strong evidence we don't make strong recommendations. We had been working on a method for providing more guidance when there's not enough evidence, and we hoped the IOM report would give us some ideas.”
Dr. England said that actionable recommendations will require AAN committees to develop some sort of formalized, transparent consensus process that will allow members to develop practice guidelines even in the absence of strong evidence. This would help committee members “come up with a process by which the group opinion is acceptable to all experts involved,” he said. “It's a way of achieving expert consensus. Physicians most certainly want and need to know what the scientific evidence says about a particular clinical question; however, they also look to guidelines to provide them with actionable recommendations. They want to know what to do with patients, so we're going to have to modify our recommendation process.”
He believes developing a formal consensus process that will enable committee members to agree on practice guidelines should be a priority.
IOM REPORT HIGHLIGHTS: CLINICAL PRACTICE GUIDELINES WE CAN TRUST
The Institute of Medicine identified eight areas for improvement:
- Transparency: The process by which a clinical practice guideline is developed and funded should be detailed explicitly and publicly accessible.
- Conflicts of Interest: Individuals being considered for guideline development groups should declare all interests and activities potentially resulting in conflicts of interest, by written disclosure. Disclosures should reflect all current and planned commercial, non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the guideline development group. All members of the guidelines group should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory board of, entities whose interests could be affected by the clinical practice guidelines.
- Guideline Development Group Composition: The group should be multidisciplinary and balanced; patients and the public should be involved by including a current or former patient and a patient advocate or patient consumer organization representative in the guideline development group.
- Systematic Reviews: Clinical practice guideline development should use systematic reviews that meet standards set by the Institute of Medicines Committee on Standards for Systematic Reviews of Comparative Effectiveness Research. See http://bit.ly/hQnDhp
- Establish evidence foundations for and rating strength of recommendations: For each recommendation, there should be an explanation of the reasoning underlying the recommendation; a clear description of potential benefits and harms; a description and explanation of any differences of opinion regarding the recommendation.
- Articulation of Recommendations: Recommendations should be articulated in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed.
- External Reviews: External reviewers should include all relevant stakeholders: scientific and clinical experts, organization, agencies (federal government, for example), patients, and representatives of the public.
- Update Guidelines: The publication date of clinical practice guidelines, date of pertinent systematic evidence review, and proposed date for future reviews should be documented in the clinical practice guideline. Literature should be monitored regularly following the publication of guidelines to identify new potentially relevant evidence, and the guidelines should be updated when new evidence suggests the need for modifying clinically important recommendations.
For the complete list of recommendations, see http://bit.ly/kJarbd.