Institutional members access full text with Ovid®

Share this article on:

The Value of Lesser-Impact-Factor Surgical Journals As a Source of Negative and Inconclusive Outcomes Reporting

Kanaan, Ziad MD, PhD*; Galandiuk, Susan MD*,†; Abby, Margaret BA; Shannon, Katherine V. BS*; Dajani, Daoud MBBS, MSc*; Hicks, Nathan BS*; Rai, Shesh N. PhD§

doi: 10.1097/SLA.0b013e31820d9b04
Original Article

Objectives: (1) Examine the tendency of peer-reviewed surgical journals to publish positive reports or negative and inconclusive outcome articles as a function of the journals’ impact factor (IF). (2) Examine the frequency with which surgical journal editors/publishers adhere to the International Committee of Medical Journal Editors statement on sources of funding and/or conflicts of interest (COI).

Background: Evidence-based medicine is often used as a template for measuring quality of medical care. Clinicians put their faith in peer-reviewed articles as quality-assured and reliable information. However, peer-reviewed literature does not provide balanced access to positive, negative, and inconclusive reports. Funding may also influence the decision to publish certain articles and can thus add to the reported bias in the literature.

Methods: Articles from 15 surgical journals comprising 3 separate journal groups based upon 2006 impact factor (IF) rankings were reviewed. All were published in 2007. Manuscripts were classified by 5 independent reviewers as having positive, negative, or inconclusive primary and secondary outcomes and for statements on funding/COI. Positive reports were defined as P < 0.05, null hypothesis rejected; negative reports defined as P < 0.05, null hypothesis accepted; and inconclusive reports defined as P > 0.05. Inter-observer consistency was affirmed. Separate analysis of randomized controlled trials (RCT) was performed to assess for the quality of published positive and negative trials.

Results: We evaluated 2457 published articles. Positive primary outcomes were reported in 67% to 100% of studied articles in selected journals. Negative and inconclusive primary outcomes were less likely to be reported, except for one journal that reported a high of 33% negative articles. Higher-ranked journals published fewer negative and inconclusive studies (5%–7%) than both medium- and lowly-ranked journals (P < 0.0001). The proportion of RCTs published varied, constituting 18% to 21% of articles in the 5 high-ranked journals compared to 6% to 14% in the 5 more lowly ranked journals (P < 0.0001). Reporting of COIs and funding were more frequent in high-IF compared to low-IF journals (P < 0.0001).

Conclusions: Quality rather than outcome should be the measure on which a publication decision is made; commercial bias may further complicate this balance. Lower IF-rated journals may serve a decidedly useful purpose by publishing more negative and inconclusive outcome studies. The practice of focusing disproportionately on the positive outcomes of most studies may result in unbalanced evidence.

Evidence-based medicine is often used as a template for measuring quality of medical care. Articles from 15 surgery journals comprising 3 separate journal groups based upon 2006 impact factor (IF) rankings were reviewed. All were published in 2007. We found that negative and inconclusive articles were more often found in lower-IF surgical journals (R2 = 0.986). Reports of funding/COI were highly variable among the studied journals; interestingly, RCTs seemed to report their funding more frequently.

From the *Price Institute of Surgical Research †the Section of Colorectal Surgery ‡Department of Surgery §the Department of Bioinformatics and Biostatistics, University of Louisville School of Medicine, Kentucky.

Presented at the Sixth International Congress on Peer Review in Biomedical Publications, September 10 to 12, 2009, Vancouver, British Columbia, Canada.

Supported by the John W. and Caroline Price Family Trust and Mrs. Sara Shallenberger.

The John W. and Caroline Price Family Trust had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

The funding source had no role in the production of this manuscript.

Additional data are available upon request.

Reprints: Susan Galandiuk, MD, Department of Surgery, Louisville, KY 40292. E-mail:

Copyright © 2011 Wolters Kluwer Health, Inc. All rights reserved.