Secondary Logo

Journal Logo

Conflicts of Interest and Publication Bias

Ryan, Margaret MD, MPH; Faix, Dennis MD, MPH; Smith, Tyler MS, PhD; Gray, Gregory C. MD, MPH

Journal of Occupational and Environmental Medicine: September 2016 - Volume 58 - Issue 9 - p e338
doi: 10.1097/JOM.0000000000000846
LETTERS TO THE EDITOR
Free

Defense Health Agency Pacific Region Immunization Branch, University of California San Diego School of Medicine, (Dr Ryan), Military Population Health, Naval Health Research Center (Dr Faix), Department of Community Health, National University, San Diego, CA (Dr Smith), Duke University School of Medicine, Durham, NC (Dr Gray).

Address correspondence to: Margaret Ryan, MD, MPH, University of California San Diego, San Diego, CA 92130 (m1ryan@ucsd.edu).

The authors have no conflicts of interest.

Readers are invited to submit letters for publication in this department. Submit letters online at http://joem.edmgr.com. Choose “Submit New Manuscript.” A signed copyright assignment and financial disclosure form must be submitted with the letter. Form available at http://www.joem.org under Author and Reviewer information.

To the Editor:

We read with great interest the article published in JOEM on, “Financial Conflicts of Interest and Study Results in Environmental and Occupational Health Research.”1 The authors reviewed 373 publications from 2012 and concluded that, when potential financial conflicts of interest were reported, nonsignificant study results, or “negative findings,” were more likely to be reported. In addition, 7 of 12 publications with US military affiliated authors shared “negative findings.” The authors concluded that investigators with financial conflicts of interest or military affiliations were more likely to report negative results from their work. We would like to respectfully share our concerns about these conclusions in the hope that readers will use care in interpreting the original article.

We are, or we have been, military-affiliated investigators, and we fully acknowledge that this prompts our interest in the topic. We also have academic affiliations. When several students interpreted the recent JOEM publication as meaning that industry-sponsored and military-affiliated authors should not be viewed as credible, our concern was heightened.

One methodologic concern about the original JOEM article relates to the determination of “positive,” “negative,” or “mixed” results. We appreciate that the two authors acknowledged challenges in determining how each reviewed study's results fit into one of these categories. Despite the close ties of the two authors, they disagreed on categorization of 14% of the publications they reviewed.1 Each of us, at one time, has been principal investigator of the US military's large Millennium Cohort Study.2 When we tried to categorize findings of Millennium Cohort publications,3 we found approximately that 25% of abstracts noted some nonsignificant (“negative”) associations, but nearly all full reports had “mixed” results. Even with rigorous methodologic definitions, we would find it difficult to classify the results of most epidemiologic studies into a simple dichotomy of “negative” versus “other” findings.

Beyond the study classification issue, our greatest concern with the referenced article is the implied assumption that publishing negative results is somehow shameful. We encourage students to always view negative results as nearly as important as positive results. Publishing negative results can be evidence of the absence of bias in treatment trials, as one of the authors himself has suggested.4 Military members have no financial conflicts of interests and, in our experience, military sponsors have encouraged investigators to submit all study results for publication, whether negative or positive. This practice seems commendably consistent with the views of leaders in the field of epidemiology.5 Negative findings are vital to aligning research and policy priorities, and such results can be appropriately reassuring to policymakers and the public.6

In the referenced article, the set of publications to which all others are compared originates primarily from academic institutions. The authors acknowledge that positive publication bias may be a factor in this comparison group.1 We believe that this issue cannot be dismissed, and it is extremely important in interpreting results. Although journal editors have worked hard to welcome the submission of negative results,5,6 it is clear that academic investigators remain reluctant to write-up and submit nonsignificant findings. In academia, reputations are forged, grants are awarded, and promotions are achieved by investigators who publish impactful positive results. Although this is understandable, the strong bias toward positive results in academic publications must be fully acknowledged.7–9

In conclusion, while we commend the authors’ efforts in addressing the important topic of conflict of interest, we encourage caution in interpreting results. The future of research should include greater transparency in reporting all conflicts and potential biases, as well as transparent sharing of data sources among academic, industry, and government investigators. Both negative results and positive results should be viewed as valuable. Only in this way do we move the field forward, determining true causal associations in epidemiology and the true value of our prevention efforts in public health.

Back to Top | Article Outline

REFERENCES

1. Friedman L, Friedman M. Financial conflicts of interest and study results in environmental and occupational health research. J Occup Environ Med 2016; 58:238–247.
2. Gray GC, Chesbrough KB, Ryan MA, et al. for the Millennium Cohort Study Group. The Millennium Cohort Study: a 21-year prospective cohort study of 140,000 military personnel. Mil Med 2002; 167:483–488.
3. US Department of Defense. Millennium Cohort Study Research Publications, Website. Available at: http://www.millenniumcohort.org/research. Accessed June 21, 2016.
4. Friedman LS, Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med 2004; 19:51–56.
5. Wilcox AJ. A positive approach to negative results. Epidemiology 2014; 25:165.
6. Lash TL, Kaufman JS. Seeking persuasively null results. Epidemiology 2015; 26:449–450.
7. Franco A, Malhotra N, Simonovits G. Social science. Publication bias in the social sciences: unlocking the file drawer. Science 2014; 345:1502–1505.
8. Coburn KM, Vevea JL. Publication bias as a function of study characteristics. Psychol Methods 2015; 20:310–330.
9. McCrary J, Christensen G, Fanelli D. Conservative tests under satisficing models of publication bias. PLoS One 2016; 11:e0149590.
Copyright © 2016 by the American College of Occupational and Environmental Medicine