Share this article on:

Preventing Publication of Falsified and Fabricated Data: Roles of Scientists, Editors, Reviewers, and Readers

Korte, Sanne M. MSc; van der Heyden, Marcel A. G. PhD

Journal of Cardiovascular Pharmacology: February 2017 - Volume 69 - Issue 2 - p 65–70
doi: 10.1097/FJC.0000000000000443
Commentary

Department of Medical Physiology, Division of Heart and Lungs, University Medical Center Utrecht, Utrecht, the Netherlands.

Reprints: Marcel A. G. van der Heyden, PhD, Yalelaan 50, 3584 CM Utrecht, the Netherlands (e-mail: m.a.g.vanderheyden@umcutrecht.nl).

The authors report no conflicts of interest.

The accompanying manuscript was unsolicited, reflecting the opinions of the authors, but not necessarily those of the editors or the publisher. It was subjected to peer review and editorial review/revision. We believe it important to share the authors' opinion with our readership.

Received July 12, 2016

Accepted October 11, 2016

Back to Top | Article Outline

INTRODUCTION

In science, a research paper is often the final result of a process of hard work, perseverance, skills, and good ideas. Accurate communication of new research findings in the context of previous work stands at the core of the scientific process and progress, and scientific misconduct can have devastating effects on the scientific community and society. The October 2014 issue of the Journal of Cardiovascular Pharmacology published a paper by Liu et al on inward rectifier channel activation by zacopride and its impact on cardiac remodeling and function. The original source paper is as follows: Liu CF, Liu QH, Liu EL, Zhai XW, Zhang L, Luo TE, Zhang WF, Feng QL, Cui XL, Zhao ZQ, Cao JM, Wu BW. Activation of IK1 channel by zacopride attenuates left ventricular remodeling in rats with myocardial infarction. J Cardiovasc Pharmacol. 2014;64:345–356. The reason why we refer this source in the text and not in the reference list is because we believe retracted papers should not appear in citation statistics. The paper was subsequently retracted in the March 2015 issue of the same journal for reasons of “flawed data representations” and the authors apologized for “making such serious and obvious mistakes in this article, which has the potential to mislead individuals working in the field.”1 We take this case as a starting point for evaluating publication fraud and then suggest a number of actions that may improve its prevention.

Analysis of the Liu et al paper demonstrated that parts of the panels of figures 1, 2, and 5 in the original publication were overlapping. Figure shows 2 identical echocardiographic recordings said to be obtained from 2 animals under different experimental conditions. One image is stretched and has altered contrast with respect to the other image (here Fig. 1A). Figure 2 displays 3 partly overlapping regions of the same immunohistochemical section as being derived from 3 separate animals under different experimental conditions (Fig. 1B). Finally, Figure 5 depicts one electron microscopic image twice, to represent 2 different experimental animals. The second image is mirrored, turned, and enlarged with respect to the first (Fig. 1C). Although data presentation was flawed in this case, fraud has not been proven formally. Some of the authors submitted a new version of a manuscript including new artwork but apparently reporting the same study as judged from the study design and averaged data, to another journal on December 20, 2015, and published it on August 2, 2016.2 Seven authors featuring the original Journal of Cardiovascular Pharmacology paper do not reappear on this version of the paper. Sadly, the authors not only potentially misled the Journal of Cardiovascular Pharmacology readers by their flawed data representations but also those of other journals. We found that 2 other published papers from authors on the Liu et al paper were retracted in 2015.3,4 The retraction note for the first one indicates that they “have used different fields of the same samples … to represent two distinct groups on two occasions in Figs. 4 and 5.” Furthermore, they stated that “some raw data for … the use of echocardiography for Figs. 8 and 9a were not available for further analysis to exclude that also the same samples were used for two distinct groups.”3 The retraction note for the second paper states that the paper was retracted by the authors “because of publication of questionable figures in the article” some of which showed overlap with a previously published paper of the same authors.4

More recently, a similar case appeared in the Journal of Cardiovascular Pharmacology. Again, an identical echocardiography recording was presented as being derived from 2 different experimental animals in which one recording was somewhat stretched compared with the other. These authors retracted their paper upon being informed of the editorial concerns about the duplication.5

These types of errors are not unique to publications in any one journal, but seem widespread in the research field. In the well-known proven case of scientific fraud by the stem cell researcher Hwang, duplications of 4 microscopic photographs said to being derived from different cell lines were revealed. The authors' initial defense that these were the result of accidental mistakes did not hold after an independent investigation by the Seoul National University was conducted.6

Different organizations, local boards, and journals use slightly different definitions of scientific misconduct. But they all have in common that the fraud is a deliberate act to mislead, as is clearly indicated by the definition from the Office of Research Integrity.7 Obviously, research is often a complicated chain of demanding experimentation, technical challenges, and data interpretation and it is clear that honest mistakes can be made during this process. These obviously fall out of the definition of scientific misconduct. On the other hand, data falsification (ie, the act of altering existing data or manipulating research processes to obtain favorable data), data fabrication (ie, making up data), and plagiarism (eg, using text or ideas from others without proper citation) all fall within the definition of research misconduct.7 We would like to use these acts in the context of “publication fraud,” which narrows the scope and distinguishes them from other forms of scientific misconduct such as unethical behavior with respect to human and animal studies, or sabotage. We will not discuss “salami” or “slice-and-dice” publication in which large data sets are split up to prepare multiple publications, whereas one comprehensive publication would suffice. Although this practice is, in our view, unwarranted, it is within the gray area of scientific conduct.

There are no good estimates of the scale of publication fraud in science. In the Netherlands, 2 large series of fraud have been detected in recent years, one in social psychology and one in cardiology.8–10 Other research areas and countries have also faced similar misfortune.11,12 The website “Retraction Watch” reports on an almost daily basis on new retractions that often result from publication fraud being detected.13 With the exception of high-impact publications, the majority of these flawed papers will go unnoticed by the scientific community as a whole, let alone the general public.

Back to Top | Article Outline

THE DETRIMENTAL EFFECTS OF PUBLICATION FRAUD

Fraudulent papers cause damage at several levels. First of all, when undetected, false data and their general implications may push a research field into the wrong direction and thus may lead to fruitless follow-up research efforts by colleagues causing losses of valuable research money and time and frustration on the part of the people involved. Second, because research funding and publication opportunities are limited, honest scientists may be “outcompeted” by fraudulent ones to a varying extent. In this way, sparse resources may be wasted. Whereas fraud seems to involve only a minor segment of the total research budgets available,14 it is likely that most fraud goes undetected. Third, fraudulent data in medical research fields may cause harm to patients, especially when medical guidelines are based on these, for instance, as seen in the Poldermans case (perioperational use of beta-blockers),9,10 the Study 329 case (paroxetine use in adolescents and suicide),15 or when public opinion is based on these as seen in the Wakefield case (Measles, Mumps and Rubella MMR vaccine and the perceived, but nonexisting, association with autism and bowel disease).16,17 Fourth, the general public may lose faith in science, which may affect the willingness to support, contribute financially to, or participate in research.

Back to Top | Article Outline

HOW TO PREVENT PUBLICATION FRAUD

The question arises whether anyone involved in the publication chain of the several examples mentioned above could have discovered the figure irregularities or manipulations. We believe that the vast majority of researchers are honest and make every effort to present their data in the best possible manner to represent reality. Most editors, reviewers, colleagues, and readers will share this opinion, and data falsification and fabrication is not the first thing that comes to mind when reviewing a manuscript or paper. Unfortunately, falsification and fabrication of data is a reality in the scientific community. Furthermore, some specialized commercial bureaus exist in some parts of the world whose business is to prepare scientific manuscripts and steer them through the peer review process, and the possibility of malice exists here as well.18–20

To counter these threats to our scientific community, we will discuss a number of implicit and explicit control mechanisms for scientists, editors, reviewers, and readers, which may decrease the occurrence of publication fraud.

Back to Top | Article Outline

Scientists

Students are our future scientists. Many universities have implemented practical courses on research integrity, including publication fraud. Nevertheless, improvements in such courses can be made, and comprehensive courses should be implemented in each curriculum worldwide. Not only will such types of education increase awareness, they will also encourage a mindset in which issues can be discussed earlier and easier. In our experience, lectures and e-modules that present and discuss clear examples of publication fraud in the most recent existing scientific literature have much more impact than “only” presenting “ethical guidelines” to students. In assignments, we challenge our students by stating that “figure irregularities” can be found within 1 hour in papers that were published the week before.

Students are very well able to distinguish between “good” and “bad” science and often wonder why so many cases of publication fraud go undetected. One of our students put it as follows: “I did know that I should be critical of a text, but the figures I always took for granted. After the course, I now start by viewing the tables and figures before reading and being influenced by the authors description of what should be observed.” If consequences (Table 1)21–25 and incentives (Table 2)14,26–29 of fraudulent behavior are discussed with students, this hopefully will produce a next generation of scientists who value “good science” more than the sheer number of publications or impact factors they achieve.

At the work site where primary data are produced, senior scientists must take responsibility for promoting a culture of “good science.” They should be aware of the subjects their juniors are working on at an almost daily basis and also should have insight in the experimental methodology, and the accompanying challenges and pitfalls. In the digital era, data falsification and fabrication are easier to achieve as compared with times when microscopically images, northern and western blot results, to name a few, were analog and artwork for publications was often prepared with the help of an institutional photographer or colleague. Current software for figure preparation is easy to use: the author, alone, can handle the entire process from data generation through final figure preparation,30 eliminating the implicit control mechanism derived from the involvement of a second person. Therefore, raw data sharing should be performed on a weekly basis within a team to prevent subsequent bias and manipulation. Furthermore, seniors should keep open minds. Well-planned and performed experiments will provide good data, even when the outcome is not in line with their hypothesis. Junior scientists should be made aware of the serious consequences publication fraud can have (Table 1).

Although apparently less frequent, data manipulation can also result from the action of the principal or senior investigator as seen in the Diederik Stapel and Dipak Das cases.24,31 In such cases, the consequences can be more far reaching.32 Juniors should be aware that these practices may have a direct influence on their own careers. Their doctoral work can become compromised without their being aware of it, their laboratory can be closed, and funding can be restricted or terminated. Therefore, juniors should take responsibility for checking their individual contributions to the final version of a manuscript and contemplate whether the statements and overall conclusions made correspond with their own findings. This latter point obviously requires a safe environment in which perceived irregularities and gray areas can be discussed with peers openly. In cases in which juniors feel no other choice than to bring the fraud of their supervisor into the open, they must do so very carefully and preferably should contact their institution's ethics officer.

Institutions have the responsibility to create an optimal environment to perform good science, which includes active research ethics policies. Visibility of the ethical guidelines and offices can be promoted in many places. Institutions may install whistleblower tools, which will help them to identify individual investigators who pose a hostile/abusive environment for their juniors. In response to accusations, institutions must act in accordance with established institutional, local or national guidelines, or guidelines provided by the scientific community. But above all, a safe environment that promotes discussion of research ethics and gray areas seems crucial. Finally, governmental legislature and funding organizations, at local, national, and international levels, can further enforce good scientific practice by active prosecution of fraud and reclaiming ill-used research funding. Because many universities and research institutes are closely linked to and financed by governments, common sense would state that such actions could be implemented without too many difficulties.

Finally, publication pressure and impact factor stress with respect to competition for tenure positions and grants put pressure upon scientists and may impact publication fraud. We will not enter into this discussion but refer the readers to a few interesting findings and opinions in this area.27,28,33

Back to Top | Article Outline

Editors and Reviewers

Reviewers are not policemen but are primarily recruited for their scientific expertise to evaluate the soundness of manuscripts on a voluntary basis. We think that publishers and journals should do their best to avoid sending fraudulent manuscripts to reviewers. First, we think that journals would benefit by adapting a policy to obtain written statements from each author on the validity of the data they provided and the conclusions linked to their contribution to the manuscript. Second, many journals screen manuscripts for plagiarism upon submission using special software, although the need for manual inspection still seems to exist.34 Several journals communicate that they screen artwork for irregularities after acceptance.35,36 These actions most likely reduce the number of fraudulent manuscripts that enter their submission portals, even when the analysis is being performed on randomly picked papers only. Preferably, however, screening of artwork should be done before sending a manuscript to reviewers so they only have to concentrate on the soundness of the scientific content. Unfortunately, such triage is time consuming, even when performed by trained experts in the field, and prolongs total manuscript handling time. If this check could be performed using software at a reasonable price, the process could become common practice for all journals. Until then, we feel that editors and reviewers should be aware of the possibility they are evaluating fraudulent work, as stated earlier by others: “be watchful, but not distrustful.”37

From our experience, reviewers feel unequipped for detecting publication fraud and think of it as an additional time-consuming task in the evaluation process of a manuscript. In our opinion, reviewers should only evaluate those experimental sections on which they have sufficient knowledge and, preferably hands-on, experience. A simple figure/table checklist to inform the editor which parts of the manuscripts fall within the reviewers' expertise will suffice. By doing so, editors should obtain insight for each manuscript whether there are blank spots left in the evaluation process and may decide whether additional reviewers should be invited. Handling of manuscripts consisting of multidisciplinary content likely would benefit here.

From a reviewer's point of view, it may be very efficient to start manuscript evaluation by screening the primary data for irregularities, and when found, immediately report these to the editor who should then decide on the issue. When data manipulation is clear beyond any doubt, a manuscript should be taken out of the review process as soon as possible, relieving all reviewers from the need to evaluate a fraudulent manuscript. In addition, the editor should report the issue to the institution at which the authors are employed in an attempt to prevent resubmission of fraudulent material to another journal. Obviously, it is much harder to detect data manipulation when experimental results are presented in bar graphs, diagrams, or tables only. Reviewers should be aware that asking for insight in the raw data is a normal procedure, and journals may facilitate these requests. In fact some journals, for example, PLOS One, implemented the policy to ask that “original uncropped and unadjusted blots and gels, including molecular size markers, should be provided in either the figures or the supplementary files.” Journals could help to educate their reviewers by providing clear examples, analogous to Figure 1, of fraudulent behavior in the research area in which they operate. We think that reviewers would find it encouraging also when journals explicitly state that sharing concerns on potential misconduct is appreciated. Furthermore, journals must be transparent in the way they protect the reviewer who exposes fraudulent behavior from subsequent legal or career consequences.

When publication fraud is detected, the paper should be retracted and the retraction note should clearly state the reason for retraction. Unfortunately, misconduct is the main reason for retractions to occur,38 but too often, wording in retraction notes is ambiguous.23,39 In our opinion, retraction notes should more clearly state whether data falsification, fabrication, or plagiarism is suspected or proven by using these terms or, when editors feel reluctant, at least provide a detailed description of the irregularities observed, analogous to our descriptions given in Figure 1. Two recently published retraction notes provide clear point-by-point descriptions of irregularities found in figures and are good examples of such factual descriptions.4,40 Moreover, this practice will more clearly distinguish acts of scientific misconduct from honest mistakes and errors resulting in a retraction.

Back to Top | Article Outline

Readers

Readers are a major resource in efforts to decrease publication fraud. We and others often detect publication fraud in clusters of papers from the same research group, institute, and/or authors.1,3,4,24 Therefore, when one encounters publication fraud in a certain paper or manuscript, it is often, useful to screen additional published papers from the same authors for “irregularities.” With some authors, the practice of publication fraud is used on a regular scale, can go on for years, and can travel with an author from one institution to another. In our experience, many journals and editors, but unfortunately not all, react positively to such information brought forward by readers and act swiftly. It would help if journals emphasize that these contributions of readers are appreciated, and if they install tools on their portals or provide special email addresses for reporting such cases. Publishers and journals should also make clear statements on how they handle whistleblower privacy and on potential legal implications for readers who decide to expose publication fraud. It should be noted, however, that current rules put the onus on investigating intent of fraud on the accused individual's institution. And so, even when the efforts of an observant reader, reviewer, or editor lead to retraction of a manuscript, the proof of intent to deceive may be lacking because of institutional or even national policy. In that event, the same individuals remain free to continue their deceitful practices.

Back to Top | Article Outline

CONCLUSIONS

Publication of false data continues to occur despite the existing control procedures in the publication process. Prevention of these practices and/or their postpublication correction may be enhanced by following a number of practical recommendations (Table 3). Scientists and students should be educated to recognize and deal with such practices; they must learn about the effects of fraudulent behavior on science in general and on their own career perspectives. Furthermore, faculty should provide an open atmosphere in the laboratory that promotes safe discussion of these issues. Journals can install prereview screening tools and encourage and facilitate reporting of suspected publication fraud during review and postpublication. Such initial screening for irregularities can be time saving for reviewers. Finally, readers are an important resource in detecting publication fraud, and they may elect to take responsibility to inform the editor to remove offending manuscripts from the literature. It is unlikely that science will ever be free of misconduct, but by understanding the problem and acting accordingly, we will likely improve our performance as generators of new, valid knowledge.

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors thank Bastiaan du Pré (Department of Medical Physiology, UMC Utrecht) and Bert Theunissen (Descartes Centre for the History and Philosophy of Sciences) for their advice.

Back to Top | Article Outline

REFERENCES

1. Wu BW. Activation of IK1 channel by zacopride attenuates left ventricular remodeling in rats with myocardial infarction: Retraction. J Cardiovasc Pharmacol. 2015;65:296.
2. Liu C, Liu E, Luo T, et al. Opening of the inward rectifier potassium channel alleviates maladaptive tissue repair following myocardial infarction. Acta Biochim Biophys Sin (Shanghai). 2016. In press.
3. Wang ZF, Wang NP, Harmouche S, et al. Retraction note to: postconditioning promotes the cardiac repair through balancing collagen degradation and synthesis after myocardial infarction in rats. Basic Res Cardiol. 2015;110:12.
4. Pang XF, Zhang LH, Bai F, et al. Retraction notice to “Dual ACE-inhibition and angiotensin II AT1 receptor antagonism with curcumin attenuate maladaptive cardiac repair and improve ventricular systolic function after myocardial infarction in rat heart”. Eur J Pharmacol. 2015;761:423.
5. Morphological identification of TRPC7 in cardiomyocytes from normal and renovascular hypertensive rats: retraction. J Cardiovasc Pharmacol. 2016;68:96.
6. Van der Heyden MAG, Derks van de Ven T, Opthof T. Fraud and misconduct in science: the stem cell seduction: implications for the peer-review process. Neth Heart J. 2009;17:25–29.
7. Office of Research Integrity. US Department of Health & Human Services. Definition of research misconduct [Office of Research Integrity web site]. 2011. Available at: http://ori.hhs.gov/definition-misconduct. Accessed June 9, 2016.
8. Sheldon T. Dutch research community is shocked by cases of scientific misconduct. BMJ. 2011;343:d7690.
9. Poldermans D, Boersma E, Bax JJ, et al; Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group. Bisoprolol reduces cardiac death and myocardial infarction in high-risk patients as long as 2 years after successful major vascular surgery. Eur Heart J. 2015;36:633.
10. Erasmus MC Follow-up Investigation Committee. Report on the 2012 follow-up investigation of possible breaches of academic integrity. 2012. Available at: http://http://www.erasmusmc.nl/5663/135857/3675250/3706798/Integrity_report_2012-10.pdf?lang=en%26lang=en. Accessed May 27, 2016.
11. Ana J, Koehlmoos T, Smith R, et al. Research misconduct in low- and middle-income countries. PLoS Med. 2013;10:e1001315.
12. Resnik DB, Master Z. Policies and initiatives aimed at addressing research misconduct in high-income countries. PLoS Med. 2013;10:e1001406.
13. Retraction Watch. Available at: http://retractionwatch.com/. Accessed June 6, 2016.
14. Stern AM, Casadevall A, Steen RG, et al. Financial costs and personal consequences of research misconduct resulting in retracted publications. Elife. 2014;3:e02956.
15. Le Noury J, Nardo JM, Healy D, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320.
16. Murch SH, Anthony A, Casson DH, et al. Retraction of an interpretation. Lancet. 2004;363:750.
17. The Editors of The Lancet. Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 2010;375:445.
18. Hvistendahl M. China's publication bazaar. Science. 2013;342:1035–1039.
19. Haug CJ. Peer-review fraud—hacking the scientific publication process. N Engl J Med. 2015;373:2393–2395.
20. Cohen A, Pattanaik S, Kumar P, et al. Organised crime against the academic peer review system. Br J Clin Pharmacol. 2016;81:1012–1017.
21. Hames I. Peer Review and Manuscript Management in Scientific Journals: Guidelines for Good Practice. Oxford, MS: Wiley-Blackwell; 2007.
22. National Institutes of Health (NIH). Scientific Integrity: NIH progress for handling research misconduct allegations. 2010. Available at: http://grants.nih.gov/grants/research_integrity/research_misconduct.htm. Accessed June 12, 2016.
23. Williams P, Wager E. Exploring why and how journal editors retract articles: findings from a qualitative study. Sci Eng Ethics. 2013;19:1–11.
24. Levelt CommitteeNoort Committee & Dreth Committee. Flawed science: the fraudulent research practices of social psychologist Diederik Stapel. 2012. Available at: https://http://www.tilburguniversity.edu/upload/3ff904d7-547b-40ae-85fe-bea38e05a34a_Final%20report%20Flawed%20Science.pdf. Accessed June 13, 2016.
25. Reardon S. US vaccine researcher sentenced to prison for fraud. Nature. 2015;523:138–139.
26. Casadevall A, Fang FC. Reforming science: methodological and cultural reforms. Infect Immun. 2012;80:891–896.
27. Holleman WL, Gritz ER. Biomedical burnout. Nature. 2013;500:613–614.
28. Holleman WL, Cofta-Woerpel LM, Gritz ER. Stress and morale of academic biomedical scientists. Acad Med. 2015;90:562–564.
29. Vasconcelos S, Leta J, Costa L, et al. Discussing plagiarism in Latin American science. Brazilian researchers begin to address an ethical issue. EMBO Rep. 2009;10:677–682.
30. Suvarna SK, Ansary MA. Histopathology and the “third great lie.” When is an image not a scientifically authentic image? Histopathology. 2001;39:441–446.
31. Blog nature (Dipak Das) Callaway E. In vino non veritas? Red wine researcher implicated in misconduct case. Available at: http://blogs.nature.com/news/2012/01/in-vino-non-veritas-red-wine-researcher-implicated-in-misconduct-case.html. Accessed August 25, 2016.
32. Bonetta L. The aftermath of scientific fraud. Cell. 2006;124:873–875.
33. Hubel DH. The way biomedical research is organized has dramatically changed over the past half-century: are the changes for the better? Neuron. 2009;64:161–163.
34. Lykkesfeldt J. Strategies for using plagiarism software in the screening of incoming journal manuscripts: recommendations based on a recent literature survey. Basic Clin Pharmacol Toxicol. 2016;119:161–164.
35. Rossner M. Hwang case review committee misses the mark. J Cell Biol. 2007;176:131–132.
36. Imagine. Nat Cell Biol. 2007;9:355.
37. Beautification and fraud. Nat Cell Biol. 2006;8:101–102.
38. Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci USA. 2012;109:17028–17033.
39. Marcus A, Oransky I. What studies of retractions tell us. J Microbiol Biol Educ. 2014;15:151–154.
40. Zhang W, Bai M, Xi Y, et al. Retraction notice to “Multiple inflammatory pathways are involved in the development and progression of cognitive deficits in APPswe/PS1dE9 mice” [Neurobiol Aging 33 (2012) 2661–2677]. Neurobiol Aging. 2016;38:219.
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.