LETTERS TO THE EDITOR
Readers are invited to submit letters for publication in this department. Submit letters online at http://joem.edmgr.com. Choose “Submit New Manuscript.” A signed copyright assignment and financial disclosure form must be submitted with the letter. Form available at http://www.joem.org under Author and Reviewer information.
To the Editor:
We read with great interest the article published in JOEM on, “Financial Conflicts of Interest and Study Results in Environmental and Occupational Health Research.”1 The authors reviewed 373 publications from 2012 and concluded that, when potential financial conflicts of interest were reported, nonsignificant study results, or “negative findings,” were more likely to be reported. In addition, 7 of 12 publications with US military affiliated authors shared “negative findings.” The authors concluded that investigators with financial conflicts of interest or military affiliations were more likely to report negative results from their work. We would like to respectfully share our concerns about these conclusions in the hope that readers will use care in interpreting the original article.
We are, or we have been, military-affiliated investigators, and we fully acknowledge that this prompts our interest in the topic. We also have academic affiliations. When several students interpreted the recent JOEM publication as meaning that industry-sponsored and military-affiliated authors should not be viewed as credible, our concern was heightened.
One methodologic concern about the original JOEM article relates to the determination of “positive,” “negative,” or “mixed” results. We appreciate that the two authors acknowledged challenges in determining how each reviewed study's results fit into one of these categories. Despite the close ties of the two authors, they disagreed on categorization of 14% of the publications they reviewed.1 Each of us, at one time, has been principal investigator of the US military's large Millennium Cohort Study.2 When we tried to categorize findings of Millennium Cohort publications,3 we found approximately that 25% of abstracts noted some nonsignificant (“negative”) associations, but nearly all full reports had “mixed” results. Even with rigorous methodologic definitions, we would find it difficult to classify the results of most epidemiologic studies into a simple dichotomy of “negative” versus “other” findings.
Beyond the study classification issue, our greatest concern with the referenced article is the implied assumption that publishing negative results is somehow shameful. We encourage students to always view negative results as nearly as important as positive results. Publishing negative results can be evidence of the absence of bias in treatment trials, as one of the authors himself has suggested.4 Military members have no financial conflicts of interests and, in our experience, military sponsors have encouraged investigators to submit all study results for publication, whether negative or positive. This practice seems commendably consistent with the views of leaders in the field of epidemiology.5 Negative findings are vital to aligning research and policy priorities, and such results can be appropriately reassuring to policymakers and the public.6
In the referenced article, the set of publications to which all others are compared originates primarily from academic institutions. The authors acknowledge that positive publication bias may be a factor in this comparison group.1 We believe that this issue cannot be dismissed, and it is extremely important in interpreting results. Although journal editors have worked hard to welcome the submission of negative results,5,6 it is clear that academic investigators remain reluctant to write-up and submit nonsignificant findings. In academia, reputations are forged, grants are awarded, and promotions are achieved by investigators who publish impactful positive results. Although this is understandable, the strong bias toward positive results in academic publications must be fully acknowledged.7–9
In conclusion, while we commend the authors’ efforts in addressing the important topic of conflict of interest, we encourage caution in interpreting results. The future of research should include greater transparency in reporting all conflicts and potential biases, as well as transparent sharing of data sources among academic, industry, and government investigators. Both negative results and positive results should be viewed as valuable. Only in this way do we move the field forward, determining true causal associations in epidemiology and the true value of our prevention efforts in public health.
1. Friedman L, Friedman M. Financial conflicts of interest and study results in environmental and occupational health research. J Occup Environ Med
2. Gray GC, Chesbrough KB, Ryan MA, et al. for the Millennium Cohort Study Group. The Millennium Cohort Study: a 21-year prospective cohort study of 140,000 military personnel. Mil Med
4. Friedman LS, Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med
5. Wilcox AJ. A positive approach to negative results. Epidemiology
6. Lash TL, Kaufman JS. Seeking persuasively null results. Epidemiology
7. Franco A, Malhotra N, Simonovits G. Social science. Publication bias in the social sciences: unlocking the file drawer. Science
8. Coburn KM, Vevea JL. Publication bias as a function of study characteristics. Psychol Methods
9. McCrary J, Christensen G, Fanelli D. Conservative tests under satisficing models of publication bias. PLoS One