Secondary Logo

Peer review in scholarly publishing part A: why do it?

Koshy, Kirona,b,; Fowler, Alexander, J.b,c; Gundogan, Buketb,d; Agha, Riaz, A.b,e

doi: 10.1097/IJ9.0000000000000056
Education Articles
Open
SDC

Peer review is a process whereby scientific experts evaluate a manuscript and provide feedback, offering a recommendation of whether the work is suitable for publication. In this article, we discuss the principles behind peer review and the different forms that it can take. We discuss the importance of peer reviewing, its drawbacks and the reasons an individual may want to become a peer reviewer.

aBrighton and Sussex University Hospitals, Brighton

bAcademic Surgical Collaborative

cDepartment of Emergency Medicine, Guys and St Thomas NHS Foundation Trust, London

dEast and North Hertfordshire NHS Trust, Hertfordshire

eBalliol College, University of Oxford, Oxford, UK

Published online 12 January 2018

Corresponding author. Address: Royal Sussex County Hospital, Brighton, United Kingdom. Tel.: +01273 696955; fax: +01273 696955. E-mail address: kironk7@gmail.com (K. Koshy).

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Received November 9, 2017

Accepted November 9, 2017

Back to Top | Article Outline

What is scientific peer review?

Publication forms a vital part of the scientific process. It allows the dissemination of novel ideas or validation/refutal of existing concepts. Before publication, peer review is a process whereby scientific experts evaluate a manuscript and provide feedback on the work, offering a recommendation of whether the work is suitable for publication1,2.

There are a number of different forms of peer review:

  • Single-blinded peer review—reviewers are aware of the authors’ identities, but the authors are not aware of the reviewers’ identities. This is the most common form of peer review.
  • Double-blinded peer review—reviewers are not aware of the authors’ identities and the authors are not aware of the reviewers’ identities.
  • Open peer review—reviewers are aware of the authors’ identities and authors are aware of the reviewers’ identities. A disadvantage of this method (and single-blinded peer review) is that reviewers may be biased by knowing the identities of the authors.
  • Postpublication peer review—this is a newer form of peer review, whereby peer review takes place after publication. Manuscripts or “preprints,” that have not yet been peer reviewed can be published on “preprint servers.” Important scientific organizations such as the Medical Research Council, now actively encourage preprint servers, as well as citation of preprints and their use in grant applications3. This means that traditional forms of publication bias are reduced. This includes the “file drawer problem” whereby studies with positive findings are more likely to be published, while those with negative/nonconfirmatory results are ignored. As content can be evaluated by the scientific community as a whole, the biases of individual reviewers are minimized.
  • Preprint publication also bypasses the bottleneck of peer reviewing, so manuscripts can be published more quickly. However, in the interim between publication and peer review, this can mean mistakes and poor quality research is open to the scientific community2,4.
  • Collaborative peer review—during the peer review process, peer reviewers can view each other’s comments, interact, and produce a final peer review report. This has been investigated by Elsevier, in the journals Molecular Cell, Neuron, and Cell5. Compared with the traditional process, results were largely positive from reviewers, editors and authors in terms of the final product. However, the process usually took longer than traditional peer review.
  • Interactive peer review—in this process, work is first screened by an editor for ethical issues and unacceptable writing. Then peer reviewers work directly with authors—often on an online forum. Reviewers provide real-time feedback to authors and can collaborate with other reviewers in this process and hold discussion with authors about recommendations. Once all authors and reviewers unanimously agree on the final product, the manuscript can be published. Peer reviewers are also listed on the paper and are invited to provide a supplementary commentary6. This process has been used by the frontiers publishing group. Although seen to be a more collaborative process and potentially faster due to real-time discussion, this method has not been adopted by many mainstream journals.
Back to Top | Article Outline

Why peer review?

Drawbacks of peer reviewing

Peer reviewing can be a time consuming process. The vast majority of peer reviewers are unpaid and therefore, some will find it to be a poor use of their time.

The peer review process has also heavily criticized. Issues that have been raised include:

  • Bias—reviewers also have inherent bias due to their own research interests and a myriad of other reasons. As they can act as gatekeepers to publication, this can skew the published literature and disillusion authors. This is an area that requires more research7. Other potential conflicts of interests include reviewing articles where the peer reviewer has previously collaborated or published a paper with one of the co-authors, or results of a manuscript may cause financial harm to a reviewer (eg, if a new scientific device competes with the reviewer’s device8.
  • Inconsistency—rarely do 2 peer reviewers form the same impression and provide similar recommendations. This can provide conflicting messages to authors.
  • Delay in time to publication—as peer review forms the main bottleneck to publication, this can be frustrating for authors and lead to publications that are no longer relevant or contemporaneous at the time of publication.

These issues have been discussed for decades, however, despite other means being proposed, there does not appear to be a more effective method9. Furthermore, Research Councils UK stated that “the strengths of peer review far outweigh the weaknesses”2.

Back to Top | Article Outline

Advantages of peer reviewing

Peer reviewing has many advantages, which are expanded upon below.

Back to Top | Article Outline

Importance to the scientific community and readers

Peer review is important because it serves to uphold the quality of the literature as well as advance the scientific knowledgebase10. In theory, peer reviewers serve to filter out poor research. As a result, readers may put more faith in what they read in scientific journals since the published work has already been vetted by a “peer” who has expertise in the field. However, as peer reviewers cannot pick up all cases of fraudulent work and poor research, the honus still lies on authors to submit good quality work.

Back to Top | Article Outline

Importance to governments, responsible authorities, and industry

Scientific research has far reaching applications beyond academia. It is instructive in governmental policies, regional schemes, and in industry2. All of these areas rely on high quality research, of which peer review in instrumental. Furthermore, the peer review process itself is utilized in the production and evaluation of governmental policies. An example of this is “health technology assessment” used by the World Health Organisation11.

Back to Top | Article Outline

Importance to authors

The purpose of peer reviewing is not just to filter poor research, but also to improve it. Providing comments and suggesting revisions to authors is with the intent of improving the ultimately finished product. This is often an invaluable tool for authors since it allows them to produce a more polished and rigorous piece of work.

Back to Top | Article Outline

Importance to editors

A well peer-reviewed manuscript is essential for journal editors in deciding whether a manuscript is suitable for publication. Editors value peer reviews from experts in the field who can often advise if the work will make a contribution to the field or if the manuscript is robust8.

Back to Top | Article Outline

Prestige

Being invited to peer review a paper suggests the reviewer’s knowledge of the field and/or critical appraisal skills are respected enough to be entrusted with gauging the quality of scientific research. Working as a peer reviewer, therefore, carries a level of prestige that can be used in one’s credentials.

Back to Top | Article Outline

Awareness of the field

Peer reviewing allows access to a larger breadth of the scientific literature. It also enables reviewers to read the most up-to-date research that others do not yet have access to and before it is put into the public domain8. Being privy to the latest developments in your research field can stimulate new ideas and encourage innovative solutions to existing research.

Back to Top | Article Outline

Training

Peer reviewing requires critical analysis of research. This is a skill that is developed through practice and experience in peer reviewing. This skill is often turned inwards to critically appraise and improve one’s own research and writing. Furthermore, it provides some inside knowledge on the level of quality required for publication. There is a now a push for increased standardization of training in peer reviewing, to help peer reviewers develop these skills2.

Back to Top | Article Outline

Peer reviewer recognition

Several platforms have emerged to recognize the work peer reviewers do. Publons emerged in 2012 as a service for academics to showcase their peer review and editorial contributions12. This platform has now been adopted by many publishing groups13,14. Publons merits are a method of measuring peer reviewer contribution, whereby peer reviewers gain points on their profile for each peer review and further points if this review is verified and made publicly available. These points provide quantitative proof of a peer reviewers contributions. There is the possibility that this could lead to peer reviewing being the latest focus of career advancement. However, given the important attributes required to peer review and the skills developed, this trend can easily be justified.

Back to Top | Article Outline

Bonuses from publishing groups

A UK House of Commons report 2011 highlighted.

“All publishers need to have in place systems for recording and acknowledging the contribution of those involved in peer review2. While the majority of peer reviewers are unpaid, journals often provide other perks to their reviewers, as added incentives and recognition of their work. Some journals publish the names of the reviewers in the final paper, or list reviewers at the end of an issue for the entire year15–18. Elsevier also gives 30 days access to its products SCOPUS and ScienceDirect19. One example is Elsevier’s Reviewer Recognition Programme20. This scheme aims to acknowledge and reward the work of peer reviewers. Peer reviewers are provided special access to the publishing group’s journal resources and discounts on affiliated services such as Elsevier Web shop and Elsevier Book Store. Awards are also provided to the top reviewers of the year.

Back to Top | Article Outline

Publication

Some journals now offer to publish peer reviewer alongside the main paper or in batch with each review given its own digital object identifier. This provides further recognition of the peer reviewers contribution to the field. Publication of peer reviews also increases the transparency of the peer review process; it provides accountability and responsibility to peer reviewers and clarity to authors21.

Publication of peer review has many legal and ethical intricacies for reviewers, authors, and publishers. This is an ongoing debate22, however, it is likely that the trend for peer review publication will continue.

Back to Top | Article Outline

Conclusions

Peer reviewing forms a fundamental part of the scientific process and has a long tradition. Although it does have drawbacks, such as being time-consuming and having potential conflicts of interest, overall there are many advantages to the peer review system. It is important, both for authors and editors, who greatly benefit from the expertise and recommendations delivered by a good peer reviewer. Moreover, it ensures, as much as possible, to filter and uphold the quality of the scientific literature and help advance the scientific knowledgebase. Reviewers are now increasingly recognized for their work, providing peer reviewing with a bright future.

Back to Top | Article Outline

Ethical approval

Ethical approval was not required for this study.

Back to Top | Article Outline

Sources of funding

There were no external sources of funding for this study.

Back to Top | Article Outline

Author contribution

All authors contributed toward the final paper. A.J.F., K.K., B.G., and R.A.A.: production of content/editing.

Back to Top | Article Outline

Conflict of interest disclosure

R.A.A. is the founding and executive editor of the IJS Publishing Group. The remaining authors declare that they have no financial conflict of interest with regard to the content of this report.

Back to Top | Article Outline

Research registration unique identifying number (UIN)

No research registration was required for this study.

Back to Top | Article Outline

Guarantor

The guarantor for this study is author R.A.A.

Back to Top | Article Outline

Acknowledgment

The authors are indebted to all members who helped them complete this study.

Back to Top | Article Outline

References

1. Wilson J. Peer review: the nuts and bolts. BioMed Central: Sense About Science; 2014.
2. House of Commons. Peer review in scientific publications. House of Commons London The Stationary Office Limited: House of Commons London; 2011.
3. Preprints: Medical Research Council. 2017. Available at: www.mrc.ac.uk/research/policies-and-guidance-for-researchers/preprints/. Accessed June 8, 2017.
4. Novella S. The importance and limitations of peer-review. Science Based Medicine. 2008. Available at: https://sciencebasedmedicine.org/the-importance-and-limitations-of-peer-review/. Accessed August 11, 2017.
5. Carniol K, Pham J, Lemasurier M. Experimenting with collaborative peer review. 2014. Available at: www.elsevier.com/reviewers-update/story/innovation-in-publishing/experimenting-with-collaborative-peer-review. Accessed June 8, 2017.
6. Pain E. How interactive peer review works. 2013. Available at: www.sciencemag.org/careers/2013/04/how-interactive-peer-review-works. Accessed June 8, 2017.
7. Jennings CG. What you can’t measure, you can’t manage: the need for quantitative indicators in peer review. Nature. 2006.
9. Nature editorial. Revolutionizing peer review? Nat Neurosci 2005;8:397.
10. Frank G. The essential role of peer review. EMBO Rep 2001;2:743.
11. World Health Organisation. 2013. Health technology assessment. Available at: www.who.int/medical_devices/assessment/en/. Accessed June 8, 2017.
12. Publons: Publons; 2017. Available at: https://publons.com/home/. Accessed June 8, 2017.
13. Schneditz D, Slaughter MS. Announcing Publons to enhance reviewer experience. ASAIO J 2017;63:235.
14. Malchesky PS. Track and verify your peer review with Publons. Artif Organs 2017;41:217.
15. Rovira J. Annual acknowledgement of manuscript reviewers. Cost Eff Resour Alloc 2016;14:5.
16. Catena F, Moore F. World Journal of Emergency Surgery reviewer acknowledgement 2015. World J Emerg Surg 2016;11:12.
17. Grosfeld JL. 2016 Reviewer acknowledgement. J Pediatr Surg 2016;51:2130–5.
18. Plum X metrics. Acknowledgement to reviewers 2016. Int J Surg 2016;36 (part A):397–401.
19. Tools and resources for reviewers. Elsevier. 2015. Available at: www.elsevier.com/reviewers/tools-and-resources-for-reviewers. Accessed June 8, 2017.
20. Elsevier reviewer recognition programme. Elsevier. 2017. Available at: www.reviewerrecognition.elsevier.com/. Accessed June 8, 2017.
21. Agha RRD. Peer-review developments at the IJS—publishing reviewer reports. Int J Surg 2014;12:1003–4.
22. Hoke T, Moylan E. How to peer review. Springer. 2016. Available at: www.springer.com/us/authors-editors/authorandreviewertutorials/howtopeerreview/why-serve-as-a-peer-reviewer/10286390. Accessed August 8, 2017.
Keywords:

Peer review; Critical appraisal; Publication; Guide; How to

Copyright © 2018 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of IJS Publishing Group, Ltd.