Does Statistical Significance Translate Into Clinical Efficacy?
Not Always, Especially When It Comes to PD Trials
Therapies that produce less than a 2.5-point improvement on the Unified Parkinson's Disease Rating Scale (UPDRS) motor score or 4.3 points on the total scale are either of no or little clinical importance to patients, according to the largest study yet of clinically important differences (CID).
The results should encourage clinicians, study designers, and federal regulators to focus on therapies whose effect sizes are large enough to merit the risk of side effects and the costs of treatment — not to mention the hopes of patients and their families, according to the authors of the new study.
“As neurologists, we may be impressed by interventions that result in statistically significant differences in large studies,” said the first author of the study, Lisa M. Shulman, MD, the Eugenia Brin Professor of Parkinson's Disease and Movement Disorders at the University of Maryland and co-director of the Maryland Parkinson's Disease and Movement Disorders Center. “But for the patient, these changes may be too small to be observed. Clinical management should be guided by those interventions that result in less disability and better quality of life.”
The study in the January Archives of Neurology enrolled 653 subjects with all levels of PD severity. A cross-sectional distribution-based and anchor-based analyses of UPDRS scores was performed using change scores from three external standards: disability (a 10 percent difference on the Schwab and England Activities of Daily Living Scale), disease stage (one stage on the Hoehn and Yahr Scale), and quality of life (one standard deviation on the 12-Item Short Form Health Survey).
Concordance among the three scales showed that a “minimal” CID was 2.3 to 2.7 points on the UPDRS motor score, and 4.1 to 4.5 on the UPDRS total scale. A “moderate” CID was 4.5 to 6.7 points on the UPDRS motor score and 8.5 to 10.3 on the UPDRS total score. A “large” CID was 10.7 to 10.8 points on the UPDRS motor score and 16.4 to 17.8 on the UPDRS total score.
While many antiparkinsonian agents show treatment effects within these ranges, published clinical trials of monoamine oxidase B (MAO-B) inhibitors, including selegiline and rasagiline, showed treatment effects less than those judged clinically significant in the new paper.
“More important than the expense of these agents is that we're giving patients false hope,” said the senior author of the paper, William J. Weiner, MD, chairman of neurology at the University of Maryland and co-director, with Dr. Shulman, of the Maryland Parkinson's Disease and Movement Disorders Center. For agents whose results in clinical trials fall short of the minimal CID seen in the new study, he said: “We just don't have the evidence of clinically significant benefit.”
Rather than encouraging patients with early PD to take one of the MAO inhibitors, he said: “They should be encouraged to enroll in drug trials looking at new ways to see if we can slow disease progression.”
Neurologists specializing in movement disorders who were not involved in the study expressed different views of its implications.
“I think it would be a good thing if the FDA only approved drugs that produced a clinically meaningful outcome,” said Kathleen M. Shannon, MD, professor of neurological sciences and director of the Huntington's Disease Society of America Center of Excellence Clinic at Rush University Medical Center in Chicago. “We have so many drugs, and how they're used is determined so much by how they're marketed, rather than what's known about how well they actually work. If you could be sure the FDA was approving only drugs that produce clinically meaningful effects, it would be much easier for clinicians and much better for patients.”
In the case of rasagiline, she said: “When patients ask about it, I just lay it out for them how tiny the effect size is. Then we talk about how much the drug costs. Not a single one of my patients has gone on the drug based on an informed discussion.”
But drug effects that are initially insignificant may prove beneficial over time, said Jay Nutt, MD, director of the Oregon Health & Science University Parkinson Center of Oregon and Movement Disorders Program. “The most important concept is it's not just what the drug does today, but what it's going to do five years down the road,” Dr. Nutt said.
Referring to the new study, he said: “It's an important paper. This is the best attempt yet at assessing what kinds of changes on the UPDRS have some clinical significance.”
But, he added: “What is selling a drug like rasagiline is not the effect you see now, but the idea that down the road, the patients taking rasagiline are going to have a lower UPDRS score in five years than those not taking it. The evidence for that is hard to come by. A lot of drug is sold on the hope, not proof, that it has disease-modifying effects.”
Dr. Weiner said that prescribing a drug based on unproved benefits is difficult to justify. “I would say that's unbridled optimism,” Dr. Weiner said. “Maybe there's going to be an effect in five or 10 years down the road. But to therefore administer these terribly expensive drugs right now is unjustified.”
Yet his colleague and co-author, Dr. Shulman, conceded that she does sometimes prescribe MAO inhibitors to patients after informing them of the evidence. “My own clinical approach is to be very open with my patients and involve them in a shared decision making process,” Dr. Shulman said. “Our patients are very needy. I'm happy to suggest all options that are available. But I try to be clear on what patients can expect from these medications.”
A 2006 study in Movement Disorders by Anette Schrag, MD, and colleagues from the Institute of Neurology at University College London — involving 603 newly diagnosed PD patients — found that larger improvements, five points on the UPDRS motor score and eight on the UPDRS total score, were necessary for patients to see a clinically important benefit. The new study suggested that patients' perceptions of clinically important differences may change as their disease progresses, with patients becoming more perceptive to differences in earlier stages.