Share this article on:

Comparison of Registered and Reported Outcomes in Randomized Clinical Trials Published in Anesthesiology Journals

Jones, Philip M. MD, MSc*†; Chow, Jeffrey T. Y. BMSc*; Arango, Miguel F. MD; Fridfinnson, Jason A. MD; Gai, Nan MD; Lam, Kevin*; Turkstra, Timothy P. MD, M Eng

doi: 10.1213/ANE.0000000000002272
Healthcare Economics, Policy, and Organization: Original Clinical Research Report

BACKGROUND: Randomized clinical trials (RCTs) provide high-quality evidence for clinical decision-making. Trial registration is one of the many tools used to improve the reporting of RCTs by reducing publication bias and selective outcome reporting bias. The purpose of our study is to examine whether RCTs published in the top 6 general anesthesiology journals were adequately registered and whether the reported primary and secondary outcomes corresponded to the originally registered outcomes.

METHODS: Following a prespecified protocol, an electronic database was used to systematically screen and extract data from RCTs published in the top 6 general anesthesiology journals by impact factor (Anaesthesia, Anesthesia & Analgesia, Anesthesiology, British Journal of Anaesthesia, Canadian Journal of Anesthesia, and European Journal of Anaesthesiology) during the years 2007, 2010, 2013, and 2015. A manual search of each journal’s Table of Contents was performed (in duplicate) to identify eligible RCTs. An adequately registered trial was defined as being registered in a publicly available trials registry before the first patient being enrolled with an unambiguously defined primary outcome. For adequately registered trials, the outcomes registered in the trial registry were compared with the outcomes reported in the article, with outcome discrepancies documented and analyzed by the type of discrepancy.

RESULTS: During the 4 years studied, there were 860 RCTs identified, with 102 RCTs determined to be adequately registered (12%). The proportion of adequately registered trials increased over time, with 38% of RCTs being adequately registered in 2015. The most common reason in 2015 for inadequate registration was registering the RCT after the first patient had already been enrolled. Among adequately registered trials, 92% had at least 1 primary or secondary outcome discrepancy. In 2015, 42% of RCTs had at least 1 primary outcome discrepancy, while 90% of RCTs had at least 1 secondary outcome discrepancy.

CONCLUSIONS: Despite trial registration being an accepted best practice, RCTs published in anesthesiology journals have a high rate of inadequate registration. While mandating trial registration has increased the proportion of adequately registered trials over time, there is still an unacceptably high proportion of inadequately registered RCTs. Among adequately registered trials, there are high rates of discrepancies between registered and reported outcomes, suggesting a need to compare a published RCT with its trial registry entry to be able to fully assess the quality of the study. If clinicians base their decisions on evidence distorted by primary outcome switching, patient care could be negatively affected.

Supplemental Digital Content is available in the text.Published ahead of print July 10, 2017.

From the Departments of *Epidemiology and Biostatistics, and Anesthesia and Perioperative Medicine, Schulich School of Medicine and Dentistry, the University of Western Ontario, Ontario, Canada.

Accepted for publication May 11, 2017.

Published ahead of print July 10, 2017.

Funding: J.T.Y.C. is supported by an Ontario Graduate Scholarship (Queen Elizabeth II Graduate Scholarship in Science and Technology), the funder of which had no role in any part of the study. P.M.J. is supported by internal departmental funding.

Conflicts of Interest: See Disclosures at the end of the article.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

Reprints will not be available from the authors.

Address correspondence to Philip M. Jones, MD, MSc, Room C3-110, University Hospital, London Health Sciences Centre, 339 Windermere Rd, London, Ontario, Canada N6A 5A5. Address e-mail to pjones8@uwo.ca.

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Generally considered to be at the top of the evidence hierarchy for evaluating therapeutic interventions, randomized clinical trials (RCTs) and systematic reviews/meta-analyses of RCTs provide important evidence for clinical decision-making.1 While RCTs provide high-quality evidence by minimizing selection bias through random allocation to groups, there are many other factors that may distort or bias the results. One tool used to improve RCT quality is trial registration, the purpose of which is to reduce publication bias (ie, not publishing studies deemed to be negative, uninteresting, or potentially damaging to the study sponsor) and selective outcome reporting (ie, publishing only a subset of all outcomes measured, often favoring outcomes that are statistically significant).2 Recognizing the importance of trial registration, the International Committee of Medical Journal Editors mandates that all clinical trials commencing after July 1, 2005, need to be registered in a public trials registry to be published in any of the International Committee of Medical Journal Editors member journals.3

Prespecifying outcomes help protect against data dredging and/or selective reporting, in which only favorable outcomes are reported and unfavorable outcomes are obscured.4 If authors elect to report different outcomes than the outcomes they registered, trial registries can also serve as an audit trail for the authors’ original intentions, allowing researchers to check published RCTs for potential outcome switching.5 Other researchers have reviewed the overall prevalence of outcome switching in the medical literature, finding a median discrepancy rate of 41% (interquartile range [IQR], 33%–48%) when comparing prospectively registered outcomes to reported outcomes.6 Although selective reporting has been studied in general and in specialty journals, only 1 study has focused on the anesthesiology specialty, finding that 48% of registered trials had a major discrepancy when comparing the registry entry and the published article.7 While De Oliveira et al7 included RCTs published during 1 year (2013) from the top 5 general anesthesiology journals by impact factor, 3 of the included journals, Anesthesiology, Anaesthesia, and Anesthesia & Analgesia (A&A), only mandate trial registration for clinical trials that begin patient enrollment after 2013,8 2014 (S. Jarvis-Atton, Managing Editor at Anaesthesia, written communication, August 2016), and 2014 (C. Williams-Klamborowski, Managing Editor at Anesthesia & Analgesia, written communication, August 2016), respectively. No study has yet performed a longitudinal analysis of outcome discrepancies in the anesthesia literature nor has any study performed a detailed analysis of secondary outcome discrepancies.

Our objective was to expand upon previous work by longitudinally examining the rate of adequate trial registration in the anesthesiology literature, focusing on RCTs published in 2007, 2010, 2013, and 2015 for the top 6 general anesthesiology journals. We also planned to thoroughly investigate outcome discrepancies (ie, differences between the outcomes registered and the outcomes reported) in both primary and secondary outcomes.

Back to Top | Article Outline

METHODS

Publicly available, our study protocol was finalized in September 2016, and all analyses (except for the raw incidences of adequate trial registration) were conducted after this time.9

Using the official websites for these journals, Tables of Contents were independently hand searched by 2 reviewers to identify eligible RCTs, with disagreements resolved by consensus. RCTs were only included if they were published in 2007, 2010, 2013, or 2015 in one of the top 6 general anesthesiology journals as determined by impact factor: Anaesthesia, A&A, Anesthesiology, British Journal of Anaesthesia (BJA), Canadian Journal of Anesthesia(CJA), and European Journal of Anaesthesiology(EJA).10 An electronic database was used to systematically screen and extract data from RCTs regarding general study characteristics and trial registration status. An RCT was defined as a prospective study assessing randomly allocated health care interventions in human participants and identified by authors clearly reporting random allocation of participants to the study arms through words like “random,” “randomized,” and “randomised” in the publication. Observational studies (eg, case–control or cohort studies), learning curve studies, cadaver studies, cost-effectiveness studies, dose-finding or dose-response studies, which were not designed to test a clinical intervention, and diagnostic test accuracy studies were excluded. We also excluded meta-analyses, editorials, letters to the editor, narrative reviews, animal studies, manikin studies, simulation studies, duplicate reports, reanalyses of previously published RCTs, and studies published in the correspondence section or in supplemental issues. If studies included 2 trials within 1 article, only data from the larger trial or phase were used.

RCTs were considered adequately registered if the trials were registered in a publicly available registry before the first patient was enrolled and if the registry entry had a clearly defined primary outcome. The trial registration status was systematically ascertained through the use of full-text searching, trial registry searching, and emails to the corresponding author. Trial registry searching used the following registries: Clinicaltrials.gov, the International Standard Randomized Controlled Trial Number Register, and the World Health Organization Clinical Trials Search Portal. If we could not positively determine the registration status of a trial by examining the full text of the article or via registry searches, we sent up to 2 standardized emails to the corresponding author with a 1-week gap between emails to determine registration status. If the registration status of the trial still could not be determined after the 2 emails, the trial was considered to be unregistered. Trials registered using only the EudraCT database (https://www.clinicaltrialsregister.eu) were considered not registered because trials contained within the European Union Clinical Trials Registry (EUCTR) were not made publicly available until recently.11

For each registered trial, the date of trial registration and first patient enrollment were collected to determine whether the trial was adequately registered. Trial registration date was defined as the date submitted for trial registration in the trials registry. If the date of first patient enrollment was not specified, the study start date was used instead. If the study start date specified only the month and year, trials were considered inadequately registered if the month occurred after the trial registration date. Trials registered after the first patient was enrolled or labeled as retrospectively registered in the trials registry were considered inadequately registered. An outcome was considered clearly defined if the outcome was clearly and unambiguously identified with a specific time frame in the trials registry. Only RCTs that were assessed to be adequately registered were further analyzed to determine the extent of outcome discrepancies.

The journal, year, funding source, number of authors, sample size, and adequate registration status were also extracted from each included RCT to describe the general characteristics of the included RCTs. To address the primary objective, the number of outcomes reported in the trial registry, the number of outcomes reported in the published article, and the presence of outcome discrepancies for included RCTs were also assessed. If the primary outcome was not explicitly described in the publication, the outcome used in the sample size calculation was considered to be the primary outcome. The number of participants per secondary outcome reported was calculated by dividing the sample size (number of participants randomized) by the number of secondary outcomes reported. Due to the number of RCTs included, RCTs were assigned to a single reviewer for data extraction, and a second reviewer was consulted if a variable was unclear, with resolution by consensus.

To provide more granularity, the type of outcome discrepancy was extracted with as much detail as possible. RCTs were considered to have a primary outcome discrepancy if:

  • (1) A registered primary outcome was not reported as a primary outcome (eg, 24-hour morphine consumption registered but not reported as a primary outcome);
  • (2) A reported primary outcome was not registered as a primary outcome (eg, 6-hour morphine consumption reported but not registered); or
  • (3) The timing of a reported primary outcome was different from the timing of a registered primary outcome (eg, the registered outcome was 30-day mortality but the reported outcome was 7-day mortality).

A registered primary outcome that was not reported as a primary outcome might have been not reported at all or reported as a secondary outcome, while a reported primary outcome that was not registered as a primary outcome might have been not registered at all or registered as a secondary outcome.

Secondary outcomes were defined using a similar structure. A registered secondary outcome that was not reported as such might have been not reported or reported as a primary outcome, while a reported secondary outcome that was not registered might have been not registered or registered as a primary outcome. Because of this hierarchical detail, some outcomes are identical but phrased differently (ie, a registered primary outcome reported as a secondary outcome is the same as a reported secondary outcome registered as a primary outcome; a reported primary outcome registered as a secondary outcome is the same as a registered secondary outcome reported as a primary outcome).

When an outcome discrepancy was identified, the reviewer determined the discrepancy to favor statistical significance if the reported outcome in question was statistically significant, as described by the authors. However, when a registered primary outcome was reported as a secondary outcome, the discrepancy was said to favor statistical significance if the registered primary outcome in question was not statistically significant. In discrepancies for which a registered primary or secondary outcome was not reported at all, statistical significance favoring could not be determined. Outcomes were considered statistically significant if they were reported as such by the authors. Stata 14 statistical software (StataCorp, College Park, TX) was used for all calculations and figures. The number and percentage of RCTs with outcome discrepancies were summarized along with other descriptive statistics such as trial characteristics and number of outcomes. No inferential statistical tests were performed.

Back to Top | Article Outline

RESULTS

Of the 860 RCTs that fulfilled the inclusion/exclusion criteria, 556 were not registered, and 202 were either registered after the first participant was enrolled or they did not have a clear unambiguously defined primary outcome in the registry entry, resulting in 102 adequately registered trials assessed for the outcome discrepancy part of the study (Figure 1).

Figure 1

Figure 1

Table 1 describes the general characteristics for all of the included RCTs. Figure 2 demonstrates a decreasing number of RCTs published in each journal over time, with only 155 RCTs identified in 2015 (compared to 316 in 2007). However, over time, an increasing proportion of these RCTs was adequately registered (Figure 3). This trend was consistent across all included journals, but overall, only 38% of RCTs were adequately registered in 2015. A detailed breakdown of why RCTs were considered inadequately registered is available in Table 1, with the reason “no trial registration located” decreasing over time, and the reason “registration occurred after first patient being enrolled” correspondingly increasing over time.

Table 1

Table 1

Figure 2

Figure 2

Figure 3

Figure 3

The majority of adequately registered trials had only 1 primary outcome, but nearly one-third of RCTs had more than 1 (Table 2). There was a large discrepancy between the median (IQR) number of secondary outcomes registered (4 [1–8]) and the number of secondary outcomes reported (18 [10–29]; Table 2). The median (IQR) number of study participants per secondary outcome reported declined from 2007 (23 [13–33]) to 2015 (6 [2–11]; Table 2).

Table 2

Table 2

Table 3

Table 3

Table 3 presents the data on outcome discrepancies, showing that 92% of adequately registered trials in 2015 had at least 1 primary or secondary outcome discrepancy, with 59% of them favoring statistical significance. Results for RCTs published in 2007 are difficult to interpret due to the small number of adequately registered trials, but RCTs published in 2013 and 2015 had a slightly smaller percentage of RCTs with at least 1 primary or secondary outcome discrepancy compared to RCTs published in 2007 and 2010. When stratifying by type of outcome discrepancy, adequately registered trials published in 2007 and 2010 had a higher percentage of having at least 1 primary outcome discrepancy, but the trend is reversed for secondary discrepancies, with adequately registered trials published in 2013 and 2015 having a higher percentage of having at least 1 secondary outcome discrepancy. Correspondingly, in 2015, many of the outcome discrepancies were related to issues with the secondary outcome, with 90% of adequately registered trials having at least 1 secondary outcome discrepancy. In 2015, 86% of adequately registered trials had at least 1 reported secondary outcome not registered, accounting for most of the secondary outcome discrepancies observed. Overall, the outcome discrepancies with the lowest percentages were having at least 1 registered secondary outcome reported as a primary outcome (6%) and having at least 1 registered primary outcome not reported at all (11%). The number and percentage of outcome discrepancies stratified by journal can be found in Supplemental Digital Content, Appendix 1, http://links.lww.com/AA/B850.

Back to Top | Article Outline

DISCUSSION

We found at least 2 major problems in the anesthesia literature: low rates of adequate RCT registration and high rates of discrepancies between the outcomes registered and the outcomes actually reported in a journal. While the proportion of adequately registered trials trended upward for all 6 journals over time, the overall percentage of 38% in 2015 is still inadequate. However, since each journal had different proportions of adequately registered trials, the overall value may not be easily interpretable.

Although transparency of research conduct is the primary reason for an investigator to register his/her RCT, for some researchers, journal-specific policies may have impacted their registration decision. Anesthesiology, BJA, and European Journal of Anaesthesiology described their mandatory trial registration policy in their Instructions to Authors sections starting in 2013,12 2009,13 and 2015,14 respectively. Both Anaesthesia and A&A confirmed a mandatory trial registration policy starting in 2014, after sending emails to the editorial office, while CJA still only strongly recommends trial registration in its Instructions to Authors section.15 Interestingly, despite the lack of a policy-mandating RCT registration, CJA did not have the lowest proportions of adequately registered trials, indicating that written journal policy is not necessarily the primary determinant of acceptance of inadequately registered RCTs.

Journals requiring RCTs to be registered in a public trials registry should, by definition, have a near-100% proportion of adequately registered trials, but the results of this study demonstrate otherwise. We suspect that this is because not all of the details of trial registration are systematically being checked by journals, particularly whether trials are registered before the first study participant is enrolled. Also known as retrospective registration, registering trials after patient enrollment may not help to reduce publication and selective reporting bias.16 The largest problem with retrospective registration is that it makes it appear as though there is complete alignment between registration and reporting, giving the impression of complying with the registration policy of a journal without actually doing so. We therefore recommend heightened vigilance of all journals to recognize this problem.

We compared the results of our study with De Oliveira et al’s7 study, which also examined the proportion of adequately registered trials in Anaesthesia, A&A, Anesthesiology, and BJA in 2013. Our study expanded on this work in 2 major ways: first, by examining only adequately registered RCTs (defined as trials that were registered in a publicly available trial registry before the first patient was enrolled and a registry entry containing a clearly defined primary outcome), and second, by performing a longitudinal examination over 4 different years rather than just a single year. Overall, the De Oliveira et al7 study found that 35% of published clinical trials in the anesthesia literature were prospectively registered, about 2 times larger than the 19% we found (for those journals in 2013) in our study. For each individual journal, we found fewer RCTs published and fewer adequately registered RCTs than did De Oliveira et al.7 This difference is likely due to different inclusion/exclusion criteria since De Oliveira et al7 identified trials through a PubMed search, while we used hand searching of each journal’s Table of Contents and inclusion criteria that were more stringent (eg, exclusion of dose-finding or dose-response studies, which were not designed to test a clinical intervention). As noted previously, our stricter definition of what constituted an adequately registered RCT may have also contributed to the discrepancy since the De Oliveira et al7 study only required adequately registered trials to be prospectively registered.

Even within adequately registered RCTs, significant discrepancies between registered and published outcomes can affect the quality of published trials.2 In some cases, this outcome switching is warranted, and authors should explain the reasons for the changes, but outcome switching may also be used to spin and distort the data to have more favorable conclusions.6,17 While the number of adequately registered trials with at least 1 primary or secondary outcome discrepancy in 2015 showed a decrease from the years 2007 and 2010, 92% of adequately registered trials published in 2015 still had at least 1 primary or secondary outcome discrepancy. For adequately registered trials published in 2015, 90% of trials had at least 1 secondary outcome discrepancy, with most of these discrepancies due to the reported secondary outcome not being registered. Historically, serendipitous findings (which are often secondary outcomes) have been responsible for many important scientific discoveries, so researchers should feel free to report these unexpected findings. However, to maintain the important scientific tradition of being open and transparent about the methods of a study, these unregistered outcomes should be labeled as “post hoc” or “exploratory.”

The Centre for Evidence-Based Medicine Outcome Monitoring (COMPare) project also found that in the top 5 medical journals, each trial silently added, on average, 5.3 new outcomes.5 In the current study, 42% of RCTs published in 2015 had at least 1 primary outcome discrepancy, which is concerning since the primary outcome is the objective of the trial, and distortion of the primary outcome will affect every aspect of the trial. No specific type of primary outcome discrepancy accounted for most of the discrepancies, but when the reported primary outcome was not registered as the primary outcome, there was a higher chance that the discrepancy favored statistical significance. While a systematic review of all studies assessing outcome discrepancies found a median percentage of primary outcome discrepancies of 32% (IQR, 25%–45%) and a median percentage of secondary outcome discrepancies of 54% (IQR, 33%–68%),6 the current study found 45% and 89%, respectively, suggesting that RCTs in anesthesiology journals may have more outcome discrepancies than general and other specialty journals. Comparing again to the De Oliveira et al7 study, which focused on anesthesiology journals, the authors found that 43% of trials had a primary outcome discrepancy, and 79% of trials had a secondary outcome discrepancy, similar to the results of our study.

With each successive year, more adequately registered RCTs have registered and reported more than 1 primary outcome. The median number of registered and reported secondary outcomes fluctuates between years, but the number of reported secondary outcomes is always higher than the number of registered secondary outcomes. This corresponds to the high percentage of secondary outcome discrepancies observed, especially with reported secondary outcomes that were not registered. To maintain the effectiveness of trial registration, investigators should ensure that all outcomes are included in the registry entry, even secondary outcomes. The number of participants per secondary outcome reported is consistent (approximately 6, except in 2007), which may be biased by the low number of adequately registered trials. When more variables are measured without increasing the sample size, there is a higher probability of getting a statistically significant result by chance alone.18 There is considerable variation in the number of secondary outcomes reported, but investigators should avoid measuring too many secondary outcomes if the sample size is small. It is concerning that in 2015, 25% of adequately registered RCTs had 2 or fewer study participants per secondary outcome analyzed.

An unexpected finding of our study (an unplanned, post hoc secondary outcome) that is deeply concerning is the large decline in the absolute number of RCTs being reported in the anesthesia literature. Potential reasons for this include a lack of grant funding to support research, the ever-increasing and frustrating bureaucratic “red tape” involved in conducting research, a shift in RCT publication from the highest-impact factor journals to lower-impact factor journals, or researchers eschewing randomized study designs in favor of observational designs (which may also involve less regulation and be less onerous to conduct).

Limitations of our study include the arbitrary selection of years examined, the number of reviewers, the exclusion of EUCTR-registered RCTs, and the presence of nonresponding corresponding authors. In this study, only the years 2007, 2010, 2013, and 2015 were included, with no data pertaining to RCTs published in unmeasured years. While 2 reviewers were used to screen for eligible RCTs, only 1 reviewer extracted data from each RCT, potentially increasing the amount of error. To reduce this error, reviewers were trained before data extraction with the same guidelines, and a second reviewer was consulted if any variable was unclear.

Another limitation was that all RCTs registered in the EUCTR were considered unregistered and subsequently excluded from analyses concerning adequately registered RCTs. Since the EUCTR was only available for public access starting in 2011,11 RCTs in 2007 and 2010 using that registry would not have been publicly available. To maintain consistency, RCTs in subsequent years were also considered to be unregistered because many studies published in 2013 and 2015 would likely have commenced their trial planning and/or recruitment stages before 2011. For this reason, approximately 6 and 4 RCTs were excluded in 2013 and 2015, respectively. Since articles with authors who were unable to be contacted after 2 attempts were considered unregistered, the proportion of adequately registered trials may have been underestimated if nonresponding authors had actually registered their trials. There is potential for bias because authors with recently published studies may be more easily contacted. However, there is only a small chance of this misclassification because publications and clinical trial registries were also searched for relevant registration information.

In conclusion, an alarming proportion of published RCTs in the anesthesia literature are still inadequately registered despite long-standing international guidelines recommending trial registration. We suggest that journal editors be more vigilant when enforcing their registration policies. Peer reviewers should also consider it a responsibility to reconcile the registered and published outcomes when reviewing manuscripts. Unfortunately, even among adequately registered trials, important discrepancies still sometimes exist between the registered outcomes and the reported outcomes. Primary outcome switching can systematically distort the anesthesia literature and potentially negatively affect patient care if clinicians base their practice on the distorted evidence. Continued longitudinal research of the incidence and severity of these problems is warranted.

Back to Top | Article Outline

ACKNOWLEDGMENTS

We appreciate the support and advice offered by Neil Klar, PhD (Department of Epidemiology and Biostatistics, the University of Western Ontario) and Janet Martin, PharmD (Department of Anesthesia and Perioperative Medicine and Department of Epidemiology and Biostatistics, the University of Western Ontario).

Back to Top | Article Outline

DISCLOSURES

Name: Philip M. Jones, MD, MSc.

Contribution: This author helped design the study, analyze the data, write the manuscript, conceive the study idea, supervise the study, conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: This author is an associate editor at the Canadian Journal of Anesthesia.

Name: Jeffrey T. Y. Chow, BMSc.

Contribution: This author helped design the study, analyze the data, write the manuscript, conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

Name: Miguel F. Arango, MD.

Contribution: This author helped conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

Name: Jason A. Fridfinnson, MD.

Contribution: This author helped conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

Name: Nan Gai, MD.

Contribution: This author helped conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

Name: Kevin Lam.

Contribution: This author helped conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

Name: Timothy P. Turkstra, MD, M Eng.

Contribution: This author helped conduct the study, collect the data, and revise the manuscript.

Conflicts of Interest: None.

This manuscript was handled by: Nancy Borkowski, DBA, CPA, FACHE, FHFMA.

Back to Top | Article Outline

REFERENCES

1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ Guide to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2015.3rd ed. New York, NY: McGraw-Hill Education.
2. Jones PM. Publication bias in the anesthesiology literature: shifting the focus from the “positive” to the “truth”. Can J Anaesth. 2016;63:658–663.
3. De Angelis C, Drazen JM, Frizelle FA, et al; International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351:1250–1251.
4. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–2465.
5. Goldacre B, Drysdale H, Powell-Smith A, et al. The COMPare Trials project. 2016. Available at: www.COMPare-trials.org. Accessed December 1, 2016.
6. Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13:282.
7. De Oliveira GS Jr, Jung MJ, McCarthy RJ. Discrepancies between randomized controlled trial registry entries and content of corresponding manuscripts reported in anesthesiology journals. Anesth Analg. 2015;121:1030–1033.
8. Editorial Office. Instructions for authors. Anesthesiology. 2015. Available at: http://anesthesiology.pubs.asahq.org/public/instructionsforauthors.aspx. Accessed August 1, 2016.
9. Chow J, Jones P. Identifying areas to improve the quality of reporting in randomized clinical trials published in anesthesiology journals: a study protocol for a series of literature surveys assessing quality of trial registration, adherence to abstract reporting guideline. 2016. Available at: https://doi.org/10.6084/m9.figshare.4490582.v1.
10. Thomson Reuters. Journal citation reports—anesthesiology. ISI Web Knowl. 2016. Available at: http://admin-apps.webofknowledge.com/JCR/JCR. Accessed August 15, 2016.
11. Benstetter M, Haubenreisser S. EU Clinical Trials Register goes live. Eur Med Agency. 2011. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Press_release/2011/03/WC500104032.pdf. Accessed August 1, 2016.
12. Anesthesiology Editorial Office. Instructions for authors. Anesthesiology. Available at: http://anesthesiology.pubs.asahq.org/public/instructionsforauthors.aspx. Accessed August 15, 2016.
13. Mahajan RP; British Journal of Anaesthesia Editorial TeamInstructions to authors. Br J Anaesth. 2016. Available at: http://www.oxfordjournals.org/our_journals/bjaint/for_authors/general.html. Accessed August 15, 2016.
14. European Journal of Anaesthesiology Editorial Office. Guidance for authors on the preparation and submission of manuscripts to the European Journal of Anaesthesiology. Eur J Anaesthesiol. 2016. Available at: http://edmgr.ovid.com/eja/accounts/ifauth.htm. Accessed August 15, 2016.
15. Backman S, Baker A, Beattie S, et al. 2011 Canadian Journal of Anesthesia Guide for Authors. Can J Anesth Can d’Anesthésie. 2011;58:668–696.
16. Harriman SL, Patel J. When are clinical trials registered? An analysis of prospective versus retrospective registration. Trials. 2016;17:187.
17. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058–2064.
18. Friedman LM, Furberg CD, DeMets DL, Reboussin DM, Granger CB. Fundamentals of Clinical Trials. 2015.5th ed. New York, NY: Springer International Publishing.

Supplemental Digital Content

Back to Top | Article Outline
© 2017 International Anesthesia Research Society