Guo, Sun-Wei MMed, PhD; Evers, Johannes L. H. MD, PhD
In response to increasing public demand for more transparency of clinical trials, ClinicalTrials.gov, an Internet-based public depository for information on studies of drugs, including biologicals, medical devices, and procedures, was established in 1999.1 The U.S. Congress mandated the expansion of ClinicalTrials.gov to have all nonphase I trials of U.S. Food and Drug Administration (FDA)–regulated drugs, biologicals, and medical devices registered at inception. It also mandated public reporting of summary results among trials of FDA–approved drugs and devices within a timeframe.
Despite this mounting pressure, extensive studies have found that between 22% and 47% of trials do not publish their findings at all.2,3 Such a selective reporting poses a serious threat to professional access to all trial results and to the validity of evidence-based medicine.
In a survey conducted 4 years ago, we found that 57 endometriosis-related clinical trials were registered at ClinicalTrials.gov.4 Among 15 completed phase II or III trials that evaluated the efficacy of various promising compounds, only three (20%) had their results published; the remaining 12 (80%) did not. Concerned, we called for more transparency in registration and disclosure of trials on endometriosis.4
Four years have since elapsed. We revisited the issue recently to see whether there is any improvement in the situation. This revisit is justifiable on the grounds that endometriosis is a common and debilitating gynecologic condition5 and costly to manage.6 Despite exponential growth in endometriosis literature and many promising preclinical studies, there is a palpable disappointment over the frustratingly slow pace in drug development for endometriosis7 and, as such, one wonders how endometriosis trials fare as compared with nonendometriosis trials. In addition, we made an attempt to identify factors associated with the potential to publish the results of completed trials. Finally, we reviewed and summarized the current state of transparency of clinical trials on endometriosis.
MATERIALS AND METHODS
All trials registered as interventional were retrieved in an Excel file using a C++ computer program that queried the ClinicalTrials.gov site using the key word “endometriosis” on October 25, 2012. For each retrieved trial, the program also queried PubMed to extract publication information, which also was entered in the Excel file. The resultant Excel file was double-checked manually.
The included trial had to be interventional in nature. Interventional trials on procedures, diagnostics, behaviorals, devices (except drug-eluting intrauterine systems), or others were excluded. In addition, trials that focused explicitly on diseases or conditions other than endometriosis or adenomyosis, such as fibroids or hot flushes, also were excluded. Trials that focused on healthy women as participants but tested drugs or biologicals that were used in endometriosis trials, such as the selective progesterone receptor modulator Proellex (telapristone, CDB-4124), CDB-2914, and ADHEXIL (a fibrin preparation supplemented with tranexamic acid intended to prevent adhesion), were included. For some trials that listed more than a single phase (eg, phases I and II), phase II was determined by examining the context and the intention of the trial (eg, efficacy). The included trials contained 15 completed trials that we examined previously.4
By definitions provided at ClinicalTrials.gov, a trial that has a recruitment status of “suspended” means that “[t]he clinical study has stopped recruiting or enrolling participants early, but it may start again.” A “terminated” trial means that “[t]he clinical study has stopped recruiting or enrolling participants early and will not start again. Participants are no longer being examined or treated.” If “[t]he clinical study stopped before enrolling its first participant,” its recruitment status is “withdrawn.” A trial is given the recruitment status of “unknown” if in ClinicalTrials.gov it had a status of “recruiting,” “not yet recruiting,” or “active, not recruiting,” and whose status has not been confirmed within the past 2 years.
Because this study used only data extracted from a publicly accessible registry and had no access to any patient data, the study was exempted from the ethics approval from the Institutional Ethics Review Board of Shanghai Obstetrics and Gynecology Hospital.
On October 25, 2012, the search yielded 144 trials of endometriosis that were registered at the site. Among them, 106 trials were of interventional nature and the remaining 38 were observational trials. After removal of trials that met the exclusion criteria mentioned, 71 trials were included for this study.
The C++ program also determined the publication status of each retrieved trial with the “completed” status by querying PubMed using the trial identifier. Unfortunately, it is not universally accepted to put the trial identifier as a footnote to the article. If the PubMed search did not yield any publication, the publication status was searched on PubMed using the detailed description of the trial and name of the principal investigator. This yielded some matches. E-mails then were sent to the corresponding authors of published studies to confirm that the published study was indeed the trial in question. The publication status of three trials, NCT00225186 (of dienogest), NCT00225199 (of dienogest), and NCT00212342 (of a low-dose oral contraceptive), were determined in this manner. However, not all principal investigators listed in the registry were contacted or, if contacted, not all responded to our e-mail inquiry.
For trials with at least one publication, the date of publication was determined from PubMed. The date of publication was typically the date when the article was made available electronically (ie, Epub date). We used publication date instead of date of submission because the former can be easily verified and has no prejudice against any particular trial when used across the board. After all, it is the publication of the trial result that matters to the evidence-based medicine. For those published articles without a specific date but only with month and year, the first day of that month was assigned. For those trials without any publication, time from trial completion to the last day of follow-up, ie, October 25, 2012, was used as censored data. Because the study completion date at ClinicalTrials.gov only contains information on year and month, but not day, the 15th of that month was assigned. In this way, the length of follow-up (ie, right-censored) or time to publication could be determined in R 2.15.2, a language and environment for statistical computing.9
For each trial, its status, phase, drug name (and thus category), sample size, sponsorship (industry, National Institutes of Health [NIH], and others), the duration of the trial (time elapsed between start date and completion date), and the time to publication (or length of follow-up if not published) were used in this study. One trial, NCT01620528, had a status of “enrolling by invitation” and, for ease of analysis, was recoded as “recruiting.”
The difference in frequency between two or more groups was evaluated using Fisher exact test. The comparison of medians between two or more groups was made using the Wilcoxon rank-sum test and Kruskal-Wallis test, respectively. The exact method was used when calculating 95% confidence intervals (CIs) for proportions. To estimate the time to publication data, Kaplan-Meier analysis, log-rank test, and Cox regression model were used. Because the sample size was moderate, the bootstrap method for censored survival data8 also was used to compute the 95% CI for the estimated hazard ratio.
P<.05 was considered statistically significant. All computations were made with R 126.96.36.199 The censboot and boot.ci functions in the R software package, boot, were used for bootstrap computation.
The characteristics of the 71 trials are listed in Table 1. The majority of them (83.1%) were phase II or III safety and efficacy trials (Table 1). Among the 71 trials, 41 (57.7%) were funded by industry, and this number did not deviate significantly from the reported 63%10 (P=.36). The specific names or codenames of drugs and biologicals used in these trials, along with their drug class, are listed in Table 2. Among the 71 trials, 35 (49.3%; 95% CI 37.2–61.5%) were completed; 10 (14.1%) were either suspended, terminated, or withdrawn; and five (7.0%) had unknown recruitment status, leaving 21 (29.6%) trials that were active (five [7.0%]), not yet recruiting (two [2.8%]), or recruiting (14 [19.7%]).
Four (5.6%) trials, two completed and funded by industry, one withdrawn, and one terminated, did not provide information on the number of target enrollment. Three (4.2%) other trials did not provide study start and completion dates. The trial phase appeared to be associated with sponsorship (P=.015), with pharmaceutical companies supporting more phase I, II, or III trials, whereas the academic institutions were associated with more phase IV trials. In five trials with “unknown” status, one was sponsored by the NIH, and the remaining four were sponsored by academic institutions, making the nonindustry sponsorship associated more with trials with “unknown” status than industry sponsorship (P=.011).
Table 2-b Classes of...Image Tools
There was an apparent surge of trial registration at approximately or after September 13, 2005, the deadline date set by the International Committee of Medical Journal Editors when its journals would not publish the results of any clinical trial that had not been appropriately registered at ClinicalTrials.gov or another qualified public registry,11 most notably for industry-sponsored trials. Besides the surge, the number of registered trials appeared to increase gradually after 2007, as compared with that in the period before the deadline (Fig. 1).
A. Histogram of the ...Image Tools
The characteristics of the 35 completed trials are listed in Table 3. From the table, it is clear that the majority (77.1%) targeted endometriosis-associated dysmenorrhea. There was no trial that was designed specifically for recurrence of endometriosis after surgery.
Because of the small number of NIH-sponsored trials (n=3), we added NIH to the list of other nonindustry sponsors. Although industry seemed to have more phase II and III trials, nonindustry sponsors supported more phase IV trials; but the difference did not reach statistical significance (P=.16; Fig. 2A and 2B). However, sponsorship was associated with the number of patients recruited into the trial (P=.048), with industry-sponsored trials having significantly larger sample sizes than nonindustry-sponsored trials (Fig. 2C). Industry-sponsored trials also had significantly shorter trial duration (ie, faster to finish) than nonindustry ones (P=.001; Fig. 2D).
A. Pie chart of noni...Image Tools
Trial sponsorship was associated with the drug category (P=.039 or, when one trial with the cryptic compound code DR-2001 was removed, P=.029), with industry-sponsored trials being associated more with trials of gonadotropin-releasing hormone antagonists, contraceptives, selective estrogen receptor modulators, and selective progesterone receptor modulators. In contrast, nonindustry-sponsored trials were associated more with trials of aromatase inhibitors. For the completed trials, the characteristics of those published and unpublished trials are shown in Table 4.
Of the 35 completed trials, results were published for 11 (31.4%; 95% CI 16.9–49.3%); results were not published for the remaining 24 (68.6%). The median length of follow-up was 40.4 months, with 25% and 75% quantiles being 24.9 and 69.4 months, respectively (range 8.3–135.5 months). As expected, the publication rate in Medline-indexed biomedical journals increased as the length of follow-up increased (Fig. 3). The publication rate of 31.4% was significantly lower than the reported 66.3% (P<.001) publication rate surveyed recently among 546 completed nonendometriosis trials registered at ClinicalTrials.gov.10
Cumulative percentag...Image Tools
Among the 11 published trials, one (9.1%) was published within 1 year after completion, six (54.5%) were published within 2 years, and nine (81.8%) were published within 3 years. When two trials with a length of follow-up less than 1 year were excluded, the publication rate became 30.3%. When eight trials with a length of follow-up less than 2 years were removed, the publication rate decreased to 18.5%. The overall time-to-publication “survival” curve or cumulative publication rate as a function of time for all completed trials is shown in Figure 3.
Among 24 completed trials for which results have not yet been published, 20 (83.3%) were funded by pharmaceutical companies, seemingly on a par with the overall Infiximab percentage reported (63%).10 The other four (16.7%) were investigator-initiated trials by research institutions or universities. The 24 trials were completed as early as July 2001, and as late as February 2012.
In univariable analysis, the Cox regression model identified that the sponsorship (industry compared with nonindustry; β −1.365, standard error 0.255; P=.026) (Fig. 4), trial phase (P=.01), and the duration of the trial (P=.006), but not the sample size of the trial or the year of registration, were associated with the tendency to publish. In particular, industry sponsorship had a hazard ratio for publication of 0.255 (95% CI 0.077–0.848 [or 0.045–0.890 by the bootstrap method]). In multivariable analysis with sponsorship, trial phase, trial duration, trial size, and year of registration as covariates, trial phase and trial duration were found to be associated with the publication potential (but the bootstrap method indicated that only the trial duration was significant). After removing three phase I and two phase IV trials, the Cox model identified that in the remaining 19 trials, sponsorship was the only covariate significantly associated with the publication potential (P=.014; the bootstrap method yielded the 95% CIs for the hazard ratios of 0.026–0.769). In addition, after removing trials that had length of follow-up of less than 1 or 2 years, the multivariable Cox regression model consistently indicated that the sponsorship was significantly associated with the publication potential (P=.058 and P=.043, respectively; by the bootstrap method, the 95% CIs for the hazard ratio were 0.044–0.912 and 0.025–1.010, respectively). In fact, the sponsorship also was significantly associated with the trial size and duration (Fig. 2C and 2D), with industry-sponsored trials having significantly larger sizes and shorter durations than nonindustry-sponsored ones. In other words, industry sponsorship was the main reason for more trials with shorter duration and more phase II and III trials. Taken together, it was concluded that sponsorship is the major factor for differential publication rate (Fig. 4).
Cumulative percentag...Image Tools
The median time from study completion to publication was 26.5 months for nonindustry-sponsored trials. For industry-sponsored trials, the median time to publication could not be calculated because by the end of follow-up, the publication rate failed to reach 0.5.
For 24 trials for which results had not been published, the length of follow-up ranged from 8.3 to 135.5 months, with a median length of 62.4 months or 5.2 years. Among them, 23 (95.8%) and 22 (91.7%) had a follow-up longer than 1 and 2 years, respectively. The trial (NCT00160446) with the longest follow-up but without published results was a phase II trial of asoprisnil, a selective progesterone receptor modulator, which was completed in July 2001.
Collectively, the 24 completed trials for which results were not published had a total number of target enrollment of more than 2,191. Two industry-sponsored trials, NCT00244452 (phase II) and NCT00212277 (phase III), did not provide the information on the target enrollment, even though it is a data element required by International Committee of Medical Journal Editors and the World Health Organization International Clinical Trials Registry Platform.
Among the 71 interventional trials, there was one suspended trial, seven terminated trials, and two withdrawn trials, which, collectively, constituted 14.1% (10/71) of all interventional trials. More details about these trials are listed in Table 5.
Two withdrawn trials were both nonindustry-sponsored. For three terminated trials that were nonindustry-sponsored, the reason for termination was given on all occasions. It was either attributable to poor enrollment or to reported serious adverse effects of the drug of interest (Table 5). For four terminated trials sponsored by industry, the reason either was attributable to adverse events in the trial or was not provided (Table 5).
Two of the terminated trials of a selective progesterone receptor modulator also mentioned the terminated trials at the company's web site (www.reprosrx.com/proellex.html, accessed October 31, 2012) in the context that the FDA notified Repros that the full clinical hold status, because of adverse events in the trial, for the company's Proellex Investigational New Drug Applications was lifted to partial clinical hold status. Although the two terminated trials had a length of follow-up of 39.4 and 38.4 months, respectively, neither has been published. One terminated trial, NCT00115661, sponsored by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, later published part of its results as a case series of three patients.12
Thirty-five of 71 interventional trials of endometriosis registered at ClinicalTrials.gov were completed as of October 25, 2012. Among those 35, results were published for 11 (31.4%), including two that led to the regulatory approval of a drug; results of more than two thirds of the trials were not published. Trials sponsored by industry were nearly four times less likely to have their results published than were nonindustry-sponsored trials (Fig. 4), even though industry-sponsored trials typically had larger sample sizes and were completed sooner than those funded by nonindustry (Fig. 2). More conspicuously, there have been no published phase II or III trials sponsored by industry that actually report a “negative” result. This seems to be consistent with previous studies reporting an association between industry sponsorship and the reporting of results favoring the experimental drugs marketed by the sponsoring companies.13–19
Compared with the publication rate of 20% found 4 years ago, the current rate has increased only marginally (31.4%, 95% CI 16.0–46.8%; P=.51) and still lies significantly below the reported 66.3% surveyed recently among 546 completed nonendometriosis trials registered at ClinicalTrials.gov.10
Some trials, mostly funded by industry, do not provide adequate information such as the number of target enrollment and study start and completion dates. This seems to be curious because the number of target enrollment is a data element required by International Committee of Medical Journal Editors and the World Health Organization International Clinical Trials Registry Platform. Ross et al19 have shown that completed trials that do not provide study completion dates within ClinicalTrials.gov were less likely to be published than those that did. In other words, failure to provide a study completion date within ClinicalTrials.gov may be considered a risk factor for nonpublication after study completion.
The completed yet unpublished 24 trials involved more than 2,190 trial participants. Admittedly, some of these patients may have benefited from receiving otherwise unavailable treatment. Yet, others simply may have volunteered largely because of altruistic motivation. In all cases, they participated knowing all too well that there is a certain degree of risk of adverse events or inferior efficacy, or both, intrinsic to all experimental studies. Nonetheless, many of them chose to take part in the hope that their participation in a trial would generate generalizable medical knowledge that might benefit not only themselves but also other and future patients, scientists, and physicians, so that collectively the trial and other scientific research would ultimately improve patient care. However, this can happen only when sufficient details of the clinical trial are made available to the public in a timely fashion.1
It is clear that the demands for more objective information and for reducing publication bias are important driving forces behind the establishment of clinical trial registries.20 Yet, “trial registration can prevent publication bias only if it is coupled with a commitment to make results from all studies available and a mechanism for implementing and policing this.”21 Reporting results of all trials involving patients can be considered a moral obligation,22 transcending patent rights and commercial confidentiality.23
The lack of transparency in clinical trials of endometriosis negatively effects basic research scientists, who often attempt to evaluate the validity or relevance, or both, of their work in terms of findings regarding clinical efficacy and safety, or they try to come up with better drug targets by figuring out why some compounds seem to work well in animal studies but fail in humans. After all, failure may be infinitely more instructive and educational than success.
The market for efficacious drugs for endometriosis with acceptable safety and cost profiles is enormous. With a reported prevalence in the range of up to 22% in women of reproductive age and approximately 50% in women with infertility,24,25 the worldwide collective health care cost of endometriosis is substantial.26 For those in drug research, and for patients with endometriosis, the stakes are very high.
We posit that more transparency of registered clinical trials on the treatment of endometriosis, especially the disclosure of results of clinical trials within a reasonable timeframe, will actually benefit, rather than hurt, all sponsors of clinical trials. Disclosure is also a moral imperative to researchers, sponsors, reviewers, and journal editors alike.
Therefore, once again, we call on all academic and industrial sponsors of endometriosis clinical trials to facilitate open and voluntary disclosure of their trial results within a reasonable timeframe (within which data collection, analyses, and dissemination can be performed adequately). We call on journal editors and reviewers to encourage the publication of negative outcome trials, and we call on researchers who designed and conducted trials of endometriosis to publish trial results even though they may be negative. We also call for the use of trial identifiers when publishing trial results, and for the proper disclosure of major trial outcome in the trial registry. In this electronic age, online disclosure of trial results on the Internet, accessible to the public, entails minimal costs. And there are many options for disclosing trial results, even for full disclosure of raw, but anonymous, data.21 This, if implemented, should greatly reduce the possibility of suppressing unfavorable trial results, provide “the ultimate test and promoter of statistical quality,” and allow for public scrutiny. It also will lead to more accurate systematic reviews, better clinical decision-making, improved patient care, and improved research efficiency and safety.27 Ultimately, it will benefit the drug developers and the patients with endometriosis.
1. Drazen JM, Wood AJ. Trial registration report card. N Engl J Med 2005;353:2809–11.
2. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med 2008;5:e217.
3. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLoS Med 2008;5:e191.
4. Guo SW, Hummelshoj L, Olive DL, Bulun SE, D'Hooghe TM, Evers JL. A call for more transparency of registered clinical trials on endometriosis. Hum Reprod 2009;24:1247–54.
5. Giudice LC, Kao LC. Endometriosis. Lancet 2004;364:1789–99.
6. Simoens S, Dunselman G, Dirksen C, Hummelshoj L, Bokor A, Brandes I, et al.. The burden of endometriosis: costs and quality of life of women with endometriosis and treated in referral centres. Hum Reprod 2012;27:1292–9.
7. Vercellini P, Crosignani P, Somigliana E, Vigano P, Frattaruolo MP, Fedele L. ‘Waiting for Godot': a commonsense approach to the medical treatment of endometriosis. Hum Reprod 2011;26:3–13.
8. Burr D. A comparison of certain bootstrap confidence intervals in the Cox model. J Am Stat Assoc 1994;89:1290–302.
9. Inhaka R, Gentleman RR. R: a language for data analysis and graphics. J Comput Graph Statist 1996;5:1923–7.
10. Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med 2010;153:158–66.
11. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al.. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 2004;351:1250–1.
12. Moravek MB, Ward EA, Lebovic DI. Thiazolidinediones as therapy for endometriosis: a case series. Gynecol Obstet Invest 2009;68:167–70.
13. Bero L, Oostvogel F, Bacchetti P, Lee K. Factors associated with findings of published trials of drug-drug comparisons: why some statins appear more efficacious than others. PLoS Med 2007;4:e184.
14. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003;290:921–8.
15. Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, et al.. The uncertainty principle and industry-sponsored research. Lancet 2000;356:635–8.
16. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ 2002;325:249.
17. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med 2009;361:1963–71.
18. Nkansah N, Nguyen T, Iraninezhad H, Bero L. Randomized trials assessing calcium supplementation in healthy children: relationship between industry sponsorship and study outcomes. Public Health Nutr 2009;12:1931–7.
19. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 2009;6:e1000144.
20. Sim I, Chan AW, Gulmezoglu AM, Evans T, Pang T. Clinical trial registration: transparency is the watchword. Lancet 2006;367:1631–3.
21. Wager E. Publishing clinical trial results: the future beckons. PLoS Clin Trials 2006;1:e31.
22. Chalmers I. Underreporting research is scientific misconduct. JAMA 1990;263:1405–8.
23. Jefferson T, Doshi P, Thompson M, Heneghan C. Ensuring safe and effective drugs: who can do what it takes? BMJ 2011;342:c7258.
24. Mahmood TA, Templeton A. Prevalence and genesis of endometriosis. Hum Reprod 1991;6:544–9.
25. Olive DL, Schwartz LB. Endometriosis. N Engl J Med 1993;328:1759–69.
26. Simoens S, Hummelshoj L, D'Hooghe T. Endometriosis: cost estimates and methodological perspective. Hum Reprod Update 2007;13:395–404.
27. Wood AJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med 2009;360:824–30.