ARTICLE IN BRIEF
A new analysis found that nearly one-third of trials with negative findings, including those involving neurology, are never published in peer-reviewed journals. Neurology trialists offer their perspectives.
Nearly one-third of large clinical trials remain unpublished some five years after completion, according to an October 2013 analysis published in the British Medical Journal (BMJ). Using the clinicaltrials.gov database, the authors identified 585 trials involving 500 or more participants that were completed prior to January 2009. Published results could not be found in the literature for 171 (29 percent) of those trials, involving nearly 300,000 study participants. Industry-sponsored trials were significantly more likely to go unreported than those that had no industry funding (32 percent vs. 18 percent, p=0.003).
Although authors did not break down their findings by specialty, lead author Christopher W. Jones, MD, now an attending physician in the department of emergency medicine at Cooper Medical School of Rowan University in New Jersey, reviewed the study data at Neurology Today's request. He found 29 neurology-specific trials, eight of which remained unpublished — a non-publication rate of 28 percent. “The non-publication rate for Alzheimer trials was particularly bad, with five of the eight trials being unpublished,” Dr. Jones said. [See “Unpublished Neurology Trials with Negative Findings.”]
These findings aren't particularly surprising: “publication bias” is a longstanding problem in biomedical science. It's well known that studies with negative findings, especially those sponsored by industry, often never see the light of PubMed, the National Library of Medicine database of biomedical abstracts.
The problem isn't limited to any one specialty. A 2008 New England Journal of Medicine analysis of antidepressant studies filed with the Food and Drug Administration (FDA) found that 67 percent of trials deemed negative by the agency went unpublished. And it's been estimated that only one in five oncology drug trials ever has its results published.
But the new analysis adds to this picture by focusing only on large trials involving 500-plus participants. “This hasn't really been done before, and we weren't sure if it would be an issue in really large studies,” said Dr. Jones. “We were hoping that it wouldn't be as big a problem, because in general, these are studies that have been intended to be published all along. They are going to be attractive to journal editors and shouldn't be prohibitively difficult to get published, and they are very important in terms of informing clinical practice. The considerable time and expense involved in running trials of this size suggests that non-publication is a conscious decision by the sponsors.”
Pressure from drug and device manufacturers to withhold negative data about their products is a longstanding problem. Walter Koroshetz, MD, deputy director of the National Institute of Neurological Disorders and Stroke, recalled a stroke trial from more than a decade ago that was particularly controversial. “The investigator stood up at a national meeting to say that he'd been warned by legal professionals that he could not present the data that he had planned to present. And then he sat down. Everyone present was very upset.” (Representatives from pharmaceutical companies and other industry organizations either did not respond to communications from Neurology Today for this story, or declined to comment.)
PROBLEMS WITH NON-INDUSTRY TRIALS
While industry trials had a much higher non-publication rate than those without industry funding, the non-industry trials had their own problems. “First, I think even 18 percent is too many trials of this size to be conducted and then not published,” said Dr. Jones. And when it comes to reporting the results of unpublished trials to Clinicaltrials.gov — a legal requirement for most drug and device trials no matter what their sponsorship — industry actually does better than non-industry counterparts.
Of the 150 unpublished industry-funded studies, 25 percent (38) had entered at least some results in Clinicaltrials.gov, whereas none of the 11 NIH-sponsored trials or the 20 trials with other funding had done so.
“I was actually surprised by these numbers for such large trials,” said Merit Cudkowicz, MD, the Julienne Dorn professor of neurology at Harvard Partners at Harvard Medical School, director of the Massachusetts General Hospital Neurological Clinical Research Institute, and a co-director of the Northeast ALS Consortium. “They allowed a five-year window for these results to be published, and it's really not acceptable for trials of this scope not to be published in that period of time. You're asking people to be part of these studies to try to advance science, and if you're not sharing that data, you're not making an effort to do that.”
Dr. Cudkowicz said that more than 80 percent of the trials she had been involved with had been published. “There was one that we did in amyotrophic lateral sclerosis [ALS] in the mid-1990s that was never published. We had a publication committee and there was intent to publish, but somehow time passed and it never came out. The results were announced, and it wasn't like someone told us we could never publish, but a full paper never followed.”
For every company that tries to bury a negative trial, Dr. Cudkowicz said, there are many others that publish in a timely manner. “Most do what is right. I don't think we should blame them all.”
She cited, by example, the EMPOWER study of dexpramipexole versus placebo for ALS, sponsored by Biogen Idec. “We just finished that trial and the paper's already been published, even though it was negative.” The double-blind, randomized phase 3 study, involving 943 participants, appeared in the November 2013 issue of the Lancet Neurology, a journal that Dr. Cudkowicz said has been particularly responsive in publishing negative results.
The creation of journals that are specifically focused on publishing negative results — such as the Journal of Negative Results in Biomedicine — may help to assuage the lack of interest among the usual array of journals in publishing such findings. And Dr. Jones noted that a number of other journals (like Lancet Neurology) have begun to publicly state their willingness to publish negative studies. For example, the BMJ website states: “Negative studies contribute to the evidence base and are considered important if they are well designed and well executed, and if the results will help clinicians make treatment decisions.”
But if not just pilot studies and smaller trials, but even large, randomized phase 3 trials are regularly going unpublished, the issue is clearly not just one of a lack of interest in the market. What can be done to address this problem?
“It's my opinion that there's a potential role for institutional review boards [IRBs] to provide more oversight of the dissemination of trial results,” Dr. Jones said. “Right now IRBs generally do a very good job of looking out for trial participant rights before and during the conduct of clinical trials, but they generally aren't involved once trials have been completed. I think you can make the case that making results available really is an ethical issue for trial participants. “
He also pointed to legislation currently bogged down in a Congressional committee, the Trial and Experimental Studies Transparency (TEST) Act. It would expand the 2007 legislation that originally mandated Clinicaltrials.gov, and close some of its loopholes.
“It was introduced as a result of situations like the one with the drug [oseltamivir] Tamiflu,” said Dr. Jones. “With that drug, the data that had been published and made available to the scientific community was a really limited subset of the data submitted to the FDA. As a result, a meta-analysis of the published oseltamivir data looked like it was really good for preventing complications related to influenza, but the regulatory statements from the FDA indicated that, based on the data they saw, it really wasn't that efficacious. The TEST Act would help to ensure that any data submitted in support of a new drug application would have to be made publicly available.”
All of this feeds into the broader issues of transparency, said Dr. Koroshetz. “We're trying to build a better culture of data sharing, but it's very difficult. For investigators, their data are their careers, their lifeblood, something they can write papers on for a long time. They're suspicious that if they lose control of the data, then someone else will take advantage.”
He described a current NINDS traumatic brain injury project that includes a large database. “We are not really funding research in this project unless people agree to share their data,” he said. “We initially wanted to make it publicly available, as occurs in the Alzheimer's Disease Neuroimaging Initiative (ADNI) project, but the investigators would not agree to go to that level. So all the data will become publicly available once the study is over in six years. We got the investigators to agree to compromise and allow certain data elements, which are not essentially tied to the hypotheses that they're trying to answer, to be made available in a shorter timeframe.”
Perhaps patients themselves have the most control over this situation, if they are made aware of it and choose to exercise their power to refuse to participate in trials. “Most people think that, when they participate in a trial, they are serving the common good,” said Dr. Koroshetz. “I think it's worthwhile getting information about trials going unpublished out to the general public. Patients have the power to make a change, by not participating unless they have something in writing guaranteeing that the data are going to be published.”
UNPUBLISHED NEUROLOGY TRIALS WITH NEGATIVE FINDINGS
The following eight major neurology trials (with more than 500 subjects) were found by Christopher W. Jones, MD, an attending physician in the department of emergency medicine at Cooper Medical School of Rowan University in New Jersey, to remain unpublished nearly five years after completion. Information on the trials was accessed from Clinicaltrials.gov on December 12, 2013.
- NCT00322036: Global Efficacy Study of MPC-7869 to Treat Patients With Alzheimer's. Sponsor: Myrexis, Inc. Study terminated; Myriad has discontinued the development of Flurizan.
- NCT00217763: European Study of 3APS in Mild to Moderate Alzheimer's Disease Patients. Sponsor: Bellus Health, Inc. Estimated study completion date: December 2007.
- NCT00083421: Effects of ONO-2506PO in Patients With Alzheimer's Disease. Sponsor: Ono Pharma USA Inc. Study completed: July 2007.
- NCT00087724: A Randomized Study to Evaluate FK962 in Subjects With Mild to Moderate Alzheimer's Disease. Sponsor: Astellas Pharma Inc. Study completed: September 2006.
- NCT00231946: ALADDIN Study - Phase III: Antigonadotropin-Leuprolide in Alzheimer's Disease Drug INvestigation (VP-AD-301). Sponsor: Voyager Pharmaceutical Corporation. No study completion date given, but record last updated September 2007.
- NCT00046761: A Study to Evaluate the Effects of ONO-2506 Intravenous Infusion in Patients With Acute Ischemic Stroke. Sponsor: Ono Pharma USA Inc. Study completed: May 2005.
- NCT00229177: Study of ONO-2506 in Patients With Acute Ischemic Stroke. Sponsor: Ono Pharmaceutical Co. Ltd. Study completed: September 2006.
- NCT00283738: MIST II PFO-Migraine Trial With BioSTAR® Bioabsorbable Septal Repair Implant. Sponsor: NMT Medical. Study completed: March 2008.
•. Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical trials: Cross sectional analysis. BMJ. 2013; 347:f6104.
•. Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008; 358:(3):252–260.
•. Cudkowicz ME, van den Berg LH, Shefner JM, et al. for the EMPOWER investigators. Dexpramipexole versus placebo for patients with amyotrophic lateral sclerosis (EMPOWER): A randomised, double-blind, phase 3 trial. Lancet Neurol. 2013; 12:(11):1059–1067. E-pub 2013 Sep 23.