Publication of studies registered in Clinical Trials Registry of India: An audit of mood disorder research protocols from 2009-2019 : Indian Journal of Psychiatry

Secondary Logo

Journal Logo

ORIGINAL ARTICLE

Publication of studies registered in Clinical Trials Registry of India: An audit of mood disorder research protocols from 2009-2019

Menon, Vikas; Varadharajan, Natarajan1; Joseph, Rini2; Praharaj, Samir Kumar3; Andrade, Chittaranjan4

Author Information
Indian Journal of Psychiatry 65(1):p 68-74, January 2023. | DOI: 10.4103/indianjpsychiatry.indianjpsychiatry_366_22
  • Open

Abstract

INTRODUCTION

From 2005, the International Committee of Medical Journal Editors (ICMJE) requires that all clinical trials involving human subjects be prospectively registered, i.e., at or before patient enrolment into the study.[1] This was done to promote transparency in conducting and reporting clinical trials and to reduce problematic outcomes such as publication bias and selective outcome reporting. However, whether and to what extent these recommendations have achieved their intended outcome(s) remains uncertain.

Evidence-based medicine depends greatly on the publication of high-quality clinical data that guides practice and research. Publication bias arises when studies with “positive” or statistically significant results are more likely to be published than studies with ‘negative’ or statistically nonsignificant results.[2,3] Such selective publication not only distorts science but may also harm patients; e.g. seeing only studies that show positive results for a new, expensive intervention, clinicians begin prescribing that intervention to their patients.[4,5] Publication bias also poses a threat to the validity of systematic reviews and meta-analyses by generating spuriously inflated effect size estimates for interventions or risk factors;[6] this can misguide practitioners and patients alike.

Not completing a trial, not publishing a completed trial, and selective publication of completed trials are all issues that have ethical implications in addition to obscuring the advancement of science. Patients who have participated in trials suffer inconveniences and even risks related to the trials; they do this so that science may progress, and future patients may benefit; and so, if the study data are not published, their hardship was in vain. There are also resource implications because the resources expended in conducting a study are of little value when the findings remain unpublished and inaccessible; such concerns are even more of a concern in low- and middle-income countries where resources are scarce and health care spending is limited. These issues also raise key questions about best practices in ensuring that all trial results are made accessible to the public and stakeholders in case of a legitimate delay in publication.

Evidence suggests that less than half of the trials registered on national registries are eventually published.[7–9] Most of these trials are from high-income countries. There is a paucity of information on this subject from low- and middle-income countries. In India, all clinical trials are required to be registered in the Clinical Trials Registry India (CTRI) database, which is a free and online public record system established by the National Institute of Medical Statistics, one of the permanent institutes of the Indian Council of Medical Research.

The CTRI was established in October 2005 and formally launched in July 2007.[10] Subsequently, the Drugs Controller General of India (DCGI) made prospective registration (registration before enrolment of the first participant) a mandatory requirement for all regulatory trials from June 15, 2009.[11] Later, in a major move, the prospective registration facility was extended to all studies (observational and non-regulatory interventional trials), and not just regulatory trials, from April 1, 2018.[11,12] However, many non-regulatory studies, particularly observational ones, are often not registered on CTRI as it is not mandatory;[11] this scenario mirrors that in larger international registries such as Clinical trials.gov maintained by the National Institute of Health.[13]

With this background, we conducted the present study. Our objectives were, first, to determine the proportion of CTRI-registered mood disorder study protocols that are published in the literature, and second, to evaluate the frequency and nature of protocol deviations in published articles and whether these deviations were updated in the CTRI database. We chose to study only mood disorders because they represent the single largest group of psychiatric disorders in terms of protocols registered on the CTRI and because it would be resource-prohibitive to study all psychiatric research protocols; similar sampling approaches have been used by prior authors who sought to examine similar research questions.[8] We chose to include both clinical trials and observational studies because the concerns (expressed in previous paragraphs) related to clinical trials apply to observational studies, as well. However, observational studies are less likely to get registered on CTRI due to reasons we explain later.

METHODS

Study protocols: Selection criteria and search strategy

We retrieved all observational studies and clinical trials related to patients with mood disorders or to their caregivers. In the case of clinical trials, we included all interventions, such as drugs, lifestyle modification, psycho-educational treatments, and behavioral treatments. We extracted data for studies registered in the CTRI database between June 15, 2009 and December 31, 2019. We used the three advanced search options at the CTRI website: ‘scientific title of the study,’ ‘health condition/problem studied,’ and ‘intervention and comparator agent’ to identify relevant observational studies and interventional clinical trials.

For the first two options, we searched the database using disorder keywords such as “depression”, “depressive disorder”, “major depressive disorder”, “treatment-resistant depression”, “mania”, “bipolar disorder”, and “bipolar depression”, entered individually. For the third option, we searched using names of psychotropic drugs and other treatments, such as “olanzapine”, “risperidone”, “cognitive behaviour therapy”, “electroconvulsive therapy”, repetitive transcranial magnetic stimulation, and deep brain stimulation, again entered individually. Two authors (NV, RJ) independently searched the CTRI database using these strategies, after which a final list of studies satisfying the eligibility criteria was made by removing duplicates. We excluded studies involving interventions related to Ayurveda, Unani, and other alternative systems of medicine because the results of such studies tend to be published in journals and other destinations that might not have been accessible to us in our literature search. We also excluded studies that were not conducted primarily in patients with the disorders listed above.

Study publication: Variables of interest and search strategy

The main outcome variables of interest were publication of the study corresponding to the registered protocol (yes/no) and presence of protocol deviations (in terms of sample size, objectives, outcomes, or any combination of these). To locate the published counterparts of registered protocols, we searched literature with the study name using a sequential MEDLINE (through PubMed) and Google search. If the study was not found using this method, we next performed PubMed and Google searches using study keywords and the last names of the authors. The full text of the published papers was retrieved and assessed for deviation from the corresponding CTRI protocol by two authors (NV, RJ) and evaluated by the first author (VM) independently. Following these sequential steps, if a study was still not located, it was classified as unpublished in the literature.

Data extraction from protocols and papers

Data extraction was performed by two authors (NV, RJ) using a pre-designed data extraction form. For every included study protocol, we abstracted the following information from the trial registry database: CTRI registration number, year of CTRI registration, whether the study was registered retrospectively or not, study population (adults, women only, elderly only, children and adolescents, combination of these), type of study (observational versus interventional), type of interventional study design (randomized controlled trial [RCT] versus Non-RCT), type of study based on objective (efficacy, safety, efficacy and safety, biomarker, other), type of intervention (drugs, behavioural, both, others, not applicable), phase of study (for interventional trials), study duration (in weeks), sample size, date of last modification, part of post graduate thesis or not, condition studied (major depression, bipolar disorder, combination of these with other disorders), funding status (funded or non-funded) and source of funding (industry funded versus non-industry funded), whether funding support was obtained from within or outside India, multicentric study or not, DGCI approval applicable or not, study completion status (completed or not), and publication status in the database (published or not).

In addition, we recorded the following details from the published paper, if identified: sample size, primary, and secondary outcomes. This was to assess our secondary objective of evaluating the frequency and type of protocol deviations in the conduct and reporting of the study. We also recorded the MEDLINE indexation status of the journal in which the article was published. For evaluating protocol deviations, two investigators (NV, RJ) examined the introduction and methods sections of the published papers to check the stated objectives, sample size, primary, and secondary outcomes; these were matched with information available in the registered protocols. We resolved differences amongst ourselves through mutual discussion between authors till consensus; all such instances were also verified by a senior investigator. For this study, a journal was considered indexed if it was included in MEDLINE and located through a PubMed search.

Statistical analysis

Descriptive statistics (frequencies with percentages) were used to present data. Multivariable logistic regression was used to identify factors associated with publication of registered protocols (yes/no). The covariates for regression analysis were chosen a priori, based on consensus among the authors and expectation of influence on the outcome of interest. These included study design (RCT versus non-RCT), multicentric study (yes/no), retrospectively registered (yes/no), and funded study (yes/no). To avoid overfitting the regression model, we restricted to four co-variates. Results of the multivariable analysis was expressed as odds ratios (OR) with 95% confidence intervals. As this was an exploratory analysis, P values <0.05 were taken as statistically significant.

Ethics approval

The study involved examination of analysis of information available in the public domain. Hence, no formal ethics approval was necessary for carrying out the present work.

RESULTS

Characteristics of included research protocols

A total of 129 registered protocols were included in the study after the search and review process [Figure 1]. There were 106 (82.2%) interventional studies and 23 (17.8%) observational studies. Only 78 (60.5%) of the registered studies had been prospectively registered. More than a third of the study protocols (n = 48; 37.2%) had been registered during 2018 and 2019. The most common subject of study was major depression (n = 75, 58.1%); this was followed by bipolar disorder (n = 28, 21.7%), both bipolar disorder and schizophrenia (n = 14, 10.9%), recurrent depression (n = 10, 7.8%), and other mood disorders (n = 2, 1.6%). The characteristics of included study protocols are shown in Table 1. Assessment tools were mentioned in 88 (68.2%) protocols; of these, 64 (72.7%) used clinician-rated measures while 24 (27.3%) used self-report instruments to measure outcomes.

F1
Figure 1:
Flow diagram of study selection process
T1
Table 1:
Characteristics of registered study protocols (n=129)

Most of the registered research protocols (n = 106, 82.2%) mentioned a source of funding; of these 36 (27.9%) were industry funded [Table 1]. Most studies recruited adults excluding the elderly (n = 114, 88.4%). Some studies were conducted exclusively on women (n = 7, 5.4%), adults including the elderly (n = 3, 2.3%), children and adolescents (n = 3, 2.3%), and exclusively on the elderly (n = 1, 0.8%). Phase IV (n = 21, 16.3%) and phase III (n = 21, 16.3%) trials were the most common, followed by phase II (n = 11, 8.5%), phase I (n = 4, 3.1%), phase II/III (n = 1, 0/8%), and phase III/IV (n = 1, 0.8%) trials. The duration of studies ranged from 2 to 200 (median, 55) weeks. The sample sizes ranged from n = 22 to n = 2236 (median, n = 92).

Publication status, nature, and type of protocol deviation

Only 65 studies (50.4%) were listed as having been completed. The distribution of entries regarding publication status in the CTRI database was as follows: not published (n = 121, 93.4%), published (n = 8, 6.6%).

Publications were located for only 43 registered studies (33.3%); of these, only 28 (21.7%) were published in MEDLINE indexed journals. Twelve publications were located from 30 protocols registered during 2009 to 2012, 9 publications from 27 protocols registered during 2013 to 2016, and 18 publications from 72 protocols registered between 2017 and 2019.

The frequency and nature of protocol deviations in published articles are presented in Table 2. More than half of the published papers exhibited protocol deviations (n = 25/43, 58.1%); many of these were related to deviations in sample size (n = 18/43, 41.9%, sample size higher (n = 10) or lower (n = 8) than stated in protocol), but discrepancies between registered and published primary and secondary outcomes were also noted in some studies. None of these studies had modified the registered study protocol in the trial registry.

T2
Table 2:
Frequency and nature of protocol deviations among published articles (n=43)

Protocol characteristics associated with publication status

Results of the logistic regression analysis [Table 3] showed that retrospectively registered trials were more likely to be published than prospectively registered trials (OR = 2.98, 95% CI = 1.32–6.71); none of the other associations examined were significant. There was a near significant association between funding status and publication; non-funded studies were more likely to be published than funded ones.

T3
Table 3:
Logistic regression analysis showing factors associated with publication status

DISCUSSION

We found that less than a third of registered studies eventually found their way into the academic literature; more than half of these were published with protocol deviations which were usually not updated in the registered protocol. The most common type of protocol deviation noted among published trials was variations in sample size; however, we also noted variations between registered and published outcomes in a few studies. Further, study completion status was not updated in many of the registered protocols and very few had an updated publication status.

These observations coming from a developing country, with chronic underspending on healthcare, and even less spending on medical research and development,[14,15] is significant for two reasons. First, any research, big or small, involves significant expenditure of human and material resources. Hence, non-publication of trial or study findings implies wastage of resources. Second, selective publication of trial results systematically distorts academic literature, compromises the findings of systematic reviews and meta-analyses, impacts evidence-based practice, undermines clinical guidelines and recommendations, and represents a violation of ethical responsibilities of investigators towards trial participants.[9]

Our findings broadly align with prior observations about the extent of selective publication of trials registered in registries such as Clinicaltrials.Gov[8,9,16,17]; all of them found that a considerable proportion of registered trials remained unpublished. Non-publication or substantial delay in publication has also been noted for trials on rare diseases,[7] funded trials,[18] trials for drugs approved by the Food and Drug Administration,[19,20] and even vaccine related trials.[21] On a contrasting note, an older study that used a broader definition of publication (that included research abstracts as well as non-indexed sources) found that nearly 93% of National Institute of Health (NIH) funded trials were published.[22] Among industry sponsored studies, a larger body of evidence suggests that between 25% to 50% of trials remain unpublished several years following completion.[23,24] Together, these findings suggest that a substantial proportion of research, whether funded or not, remain unpublished and unavailable for informing practice and research.

Some of our other findings are concerning too. Nearly half of the protocols registered on the CTRI database originated from a postgraduate academic thesis. This probably underscores the fact that a large share of research and publications that happens in psychiatric teaching hospitals in India is linked to postgraduate dissertations, the completion of which is mandatory for trainees to appear for the exit examinations. Postgraduate dissertations are often limited in their scope as the trainee has a little more than a year to execute the research and must submit the completed work a few months before the exit examination; this automatically rules out conduct of studies with complex logistics and large sample sizes due to the time bound nature of the project. Of specific concern was the fact that more than three-quarters of the study protocols received funding but were not eventually published; this further supports our assertion about wastage of resources. It is also possible that trainees may not be sufficiently motivated to pursue publication of their postgraduate dissertation work following successful completion of their degree requirements. Interestingly, a prior analysis of eventual publication status of MD postgraduate psychiatry dissertations in India showed that less than a third attained full publication status at 5–15 years following study completion.[25]

Rather reassuringly, most of the published papers with protocol deviations showed only deviation in their sample sizes, usually considered a minor deviation. Discrepancies between registered and published outcomes were rare occurrences; however, it must be noted that because only limited information is provided in the CTRI database, it was difficult to conclusively ascertain deviations. Prior studies have shown modifications, omissions, and selective reporting of outcomes in published trials vis-à-vis their protocol counterparts.[26,27] The registered study protocol was not modified for any of the studies with protocol deviations. This has both scientific and ethical implications.

In our analysis, four studies showed deviation in primary outcomes, a serious change that could imply selective reporting and threaten the validity of the study by providing misleading conclusions. Three studies had made changes to secondary outcomes, mostly by adding new outcomes in the published paper or by deleting some outcomes from the registered protocol; this may imply HARKing (hypothesizing after results are known) or cherry picking, both of which are questionable research practices.[28] While they may have some hypothesis generating value, presenting them as a priori formulation amounts to research misconduct and must be avoided.

Retrospectively registered studies were more likely to be published than prospectively registered ones; other factors examined such as being a multicentric, funded, or interventional study were not significantly associated with eventual publication. Time is an important consideration here; retrospectively registered studies would have all been necessarily registered before April 1, 2018; the date on which prospective study registration in CTRI became mandatory. Hence, authors of such studies would have had more time to find a publisher for their work compared to prospectively registered studies. Our findings about a lack of association between funding and publication also find support in prior research.[16,29]

There is a clear need to prioritize publication of completed studies. Funding agencies should insist on providing a time plan for project publications in the Gantt chart submitted in the funding application and make the release of final installment of funds contingent upon publication and dissemination of findings. National registries should mandate researchers to report results from completed studies on trial repositories, regardless of their eventual publication; this will allow other researchers to access these results and build on it thereby advancing science. It also allows researchers an avenue to fulfill their ethical obligations of disseminating their findings to their study participants who have sacrificed their time and, possibly, placed themselves at some risk to contribute to science. The Institutional Review Boards and funding agencies should pursue investigators to publish their work in a time bound manner and, additionally, update the study records on the CTRI database.

Admittedly, not all steps from completion to publication of a work are under a researcher’s control. Sometimes, there may be legitimate reasons for non-publication of a completed study; for instance, authors may have had a backlog of papers and the completed study is in the queue, authors may have lost interest in pursuing the study till publication because it returned negative findings, or simply, authors do not have the resources to analyze the data competently and write a good paper. In the last couple of years, it is also possible that the ongoing COVID-19 pandemic may have impacted study completion and publication.

Our work had some limitations. First, our publication figures may represent a conservative estimate as we have included protocols registered up to December 2019. Second, not all studies on mood disorders conducted in the country during the period of interest may have been captured; this may be particularly true for studies carried out before April 1, 2018 when prospective study registration was not mandatory. Third, interventional trials overrepresented our study sample; this is probably owing to the erroneous perception amongst the Indian academic community that the CTRI database is only meant for registration of interventional trials. This perception may be partly attributed to its name “clinical trials registry”; observational studies are generally not referred to as trials. Fourth, a possibility of publication bias coloring the study findings cannot be discounted; it is a well-documented finding in psychiatry that negative trials are less likely to be published than their positive counterparts.[30] More pertinently, there is a documented evidence of such bias influencing publication rates among industry sponsored drug trials.[31]

To sum up, despite a growing emphasis on publications for promotion and career advancement, we found that only a third of study protocols registered on the Indian trial registry database are eventually published. Further, most of the published papers exhibited minor protocol deviations and researchers did not update these deviations subsequently on the trial registry. Our findings are in sync with global literature and suggest a need for action by stakeholders to ensure dissemination and public access to study results. Future research must also examine the complementary research question–how many published intervention trials are registered with trial registry databases; this too has key implications for scientific validity and integrity of biomedical research.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

REFERENCES

1. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration:A statement from the International Committee of Medical Journal Editors. N Engl J Med 2004;351:1250–1.
2. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:281–6.
3. Misakian AL, Bero LA. Publication bias and research on passive smoking:Comparison of published and unpublished studies. JAMA 1998;280:250–3.
4. Joober R, Schmitz N, Annable L, Boksa P. Publication bias:What are the challenges and can they be overcome? J Psychiatry Neurosci 2012;37:149–52.
5. Murad MH, Chu H, Lin L, Wang Z. The effect of publication bias magnitude and direction on the certainty in evidence. BMJ Evid Based Med 2018;23:84–6.
6. van Aert RCM, Wicherts JM, van Assen MALM. Publication bias examined in meta-analyses from psychology and medicine:A meta-meta-analysis. PLoS One 2019;14:e0215052.
7. Rees CA, Pica N, Monuteaux MC, Bourgeois FT. Noncompletion and nonpublication of trials studying rare diseases:A cross-sectional analysis. PLoS Med 2019;16:e1002966.
8. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials. Gov:A cross-sectional analysis. PLoS Med 2009;6:e1000144.
9. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov:Cross sectional analysis. BMJ 2012;344:d7292. doi:10.1136/bmj.d7292.
10. Tharyan P, Ghersi D. Registering clinical trials in India:A scientific and ethical imperative. Natl Med J India 2008;21:31–4.
11. National Institute of Medical Statistics. Clinical Trials Registry - India (CTRI). 2020. Available from: http://ctri.nic.in/Clinicaltrials/login.php. [Last accessed on 2022 Apr 18].
12. Vardhana Rao MV, Maulik M, Gupta J, Panchal Y, Juneja A, Adhikari T, et al. Clinical Trials Registry –India:An overview and new developments. Indian J Pharmacol 2018;50:208–11.
13. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies:Is it time? CMAJ 2010;182:1638–42.
14. National Health Systems Resource Centre. National Health Accounts: Estimates for India 2014–15. 2017. Available from: https://nhsrcindia.org/sites/default/files/2021-06/NHA-Estimates-Report-FinalWeb-Optimized-PDF-Version-22.11.17.pdf. [Last accessed on 2021 May 27].
15. Rao MG, Choudhury M Health Care Financing Reforms in India. National Institute of Public Finance and Policy; 2012. Available from: https://www.nipfp.org.in/media/medialibrary/2013/04/wp_2012_100.pdf. [Last accessed on 2021 May 27].
16. Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A. Discontinuation and non-publication of surgical randomised controlled trials:Observational study. BMJ 2014;349:g6870. doi:10.1136/bmj.g6870.
17. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials:Cross sectional analysis. BMJ 2013;347:f6104. doi:10.1136/bmj.f6104.
18. Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926–34.
19. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications:A literature analysis. PLoS Med 2008;5:e191. doi:10.1371/journal.pmed. 0050191.
20. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the food and drug administration:Review of publication and presentation. PLoS Med 2008;5:e217.
21. Manzoli L, Flacco ME, D'Addario M, Capasso L, De Vito C, Marzuillo C, et al. Non-publication and delayed publication of randomized trials on vaccines:Survey. BMJ 2014;348:g3058.
22. Dickersin K, Min YI. NIH clinical trials and publication bias. Online J Curr Clin Trials 1993. Doc No 50:[4967 words;53 paragraphs].
23. Decullier E, Lhéritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France:Retrospective cohort study. BMJ 2005;331:19.
24. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 2008;3:e3081.
25. Srivastava S, Agarwal V, Subramanyam A, Srivastava M, Sathyanarayana Rao TS, Rao GP, et al. A pilot study of MD (psychiatry) theses-based research. Indian J Psychiatry 2018;60:127–30.
26. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials:Comparison of protocols to published articles. JAMA 2004;291:2457–65.
27. Chan AW, Krleza-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171:735–40.
28. Andrade C. HARKing, Cherry-Picking, P-hacking, fishing expeditions, and data dredging and mining as questionable research practices. J Clin Psychiatry 2021;82:20f13804.
29. Pica N, Bourgeois F. Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics 2016;138:e20160223.
30. Porter RJ, Boden JM, Miskowiak K, Malhi GS. Failure to publish negative results:A systematic bias in psychiatric literature. Aust N Z J Psychiatry 2017;51:212–4.
31. Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med 2010;153:158–66.
Keywords:

Clinical trial registry; clinical trials; psychiatry; publication bias; randomized controlled trials; selective publication

Copyright: © 2023 Indian Journal of Psychiatry