Peer review in scholarly publishing part A: why do it?

Abstract

Peer review is a process whereby scientific experts evaluate a manuscript and provide feedback, offering a recommendation of whether the work is suitable for publication. In this article, we discuss the principles behind peer review and the different forms that it can take. We discuss the importance of peer reviewing, its drawbacks and the reasons an individual may want to become a peer reviewer.

What is scientific peer review?

Publication forms a vital part of the scientific process. It allows the dissemination of novel ideas or validation/refutal of existing concepts. Before publication, peer review is a process whereby scientific experts evaluate a manuscript and provide feedback on the work, offering a recommendation of whether the work is suitable for publication^1,2.

There are a number of different forms of peer review:

• Single-blinded peer review—reviewers are aware of the authors’ identities, but the authors are not aware of the reviewers’ identities. This is the most common form of peer review.
• Double-blinded peer review—reviewers are not aware of the authors’ identities and the authors are not aware of the reviewers’ identities.
• Open peer review—reviewers are aware of the authors’ identities and authors are aware of the reviewers’ identities. A disadvantage of this method (and single-blinded peer review) is that reviewers may be biased by knowing the identities of the authors.
• Postpublication peer review—this is a newer form of peer review, whereby peer review takes place after publication. Manuscripts or “preprints,” that have not yet been peer reviewed can be published on “preprint servers.” Important scientific organizations such as the Medical Research Council, now actively encourage preprint servers, as well as citation of preprints and their use in grant applications³. This means that traditional forms of publication bias are reduced. This includes the “file drawer problem” whereby studies with positive findings are more likely to be published, while those with negative/nonconfirmatory results are ignored. As content can be evaluated by the scientific community as a whole, the biases of individual reviewers are minimized.
• Preprint publication also bypasses the bottleneck of peer reviewing, so manuscripts can be published more quickly. However, in the interim between publication and peer review, this can mean mistakes and poor quality research is open to the scientific community^2,4.
• Collaborative peer review—during the peer review process, peer reviewers can view each other’s comments, interact, and produce a final peer review report. This has been investigated by Elsevier, in the journals Molecular Cell, Neuron, and Cell⁵. Compared with the traditional process, results were largely positive from reviewers, editors and authors in terms of the final product. However, the process usually took longer than traditional peer review.
• Interactive peer review—in this process, work is first screened by an editor for ethical issues and unacceptable writing. Then peer reviewers work directly with authors—often on an online forum. Reviewers provide real-time feedback to authors and can collaborate with other reviewers in this process and hold discussion with authors about recommendations. Once all authors and reviewers unanimously agree on the final product, the manuscript can be published. Peer reviewers are also listed on the paper and are invited to provide a supplementary commentary⁶. This process has been used by the frontiers publishing group. Although seen to be a more collaborative process and potentially faster due to real-time discussion, this method has not been adopted by many mainstream journals.

Why peer review?

Drawbacks of peer reviewing

Peer reviewing can be a time consuming process. The vast majority of peer reviewers are unpaid and therefore, some will find it to be a poor use of their time.

The peer review process has also heavily criticized. Issues that have been raised include:

• Bias—reviewers also have inherent bias due to their own research interests and a myriad of other reasons. As they can act as gatekeepers to publication, this can skew the published literature and disillusion authors. This is an area that requires more research⁷. Other potential conflicts of interests include reviewing articles where the peer reviewer has previously collaborated or published a paper with one of the co-authors, or results of a manuscript may cause financial harm to a reviewer (eg, if a new scientific device competes with the reviewer’s device⁸.
• Inconsistency—rarely do 2 peer reviewers form the same impression and provide similar recommendations. This can provide conflicting messages to authors.
• Delay in time to publication—as peer review forms the main bottleneck to publication, this can be frustrating for authors and lead to publications that are no longer relevant or contemporaneous at the time of publication.

These issues have been discussed for decades, however, despite other means being proposed, there does not appear to be a more effective method⁹. Furthermore, Research Councils UK stated that “the strengths of peer review far outweigh the weaknesses”².

Advantages of peer reviewing

Peer reviewing has many advantages, which are expanded upon below.

Importance to the scientific community and readers

Peer review is important because it serves to uphold the quality of the literature as well as advance the scientific knowledgebase¹⁰. In theory, peer reviewers serve to filter out poor research. As a result, readers may put more faith in what they read in scientific journals since the published work has already been vetted by a “peer” who has expertise in the field. However, as peer reviewers cannot pick up all cases of fraudulent work and poor research, the honus still lies on authors to submit good quality work.

Importance to governments, responsible authorities, and industry

Scientific research has far reaching applications beyond academia. It is instructive in governmental policies, regional schemes, and in industry². All of these areas rely on high quality research, of which peer review in instrumental. Furthermore, the peer review process itself is utilized in the production and evaluation of governmental policies. An example of this is “health technology assessment” used by the World Health Organisation¹¹.

Importance to authors

The purpose of peer reviewing is not just to filter poor research, but also to improve it. Providing comments and suggesting revisions to authors is with the intent of improving the ultimately finished product. This is often an invaluable tool for authors since it allows them to produce a more polished and rigorous piece of work.

Importance to editors

A well peer-reviewed manuscript is essential for journal editors in deciding whether a manuscript is suitable for publication. Editors value peer reviews from experts in the field who can often advise if the work will make a contribution to the field or if the manuscript is robust⁸.

Prestige

Being invited to peer review a paper suggests the reviewer’s knowledge of the field and/or critical appraisal skills are respected enough to be entrusted with gauging the quality of scientific research. Working as a peer reviewer, therefore, carries a level of prestige that can be used in one’s credentials.

Awareness of the field

Peer reviewing allows access to a larger breadth of the scientific literature. It also enables reviewers to read the most up-to-date research that others do not yet have access to and before it is put into the public domain⁸. Being privy to the latest developments in your research field can stimulate new ideas and encourage innovative solutions to existing research.

Training

Peer reviewing requires critical analysis of research. This is a skill that is developed through practice and experience in peer reviewing. This skill is often turned inwards to critically appraise and improve one’s own research and writing. Furthermore, it provides some inside knowledge on the level of quality required for publication. There is a now a push for increased standardization of training in peer reviewing, to help peer reviewers develop these skills².

Peer reviewer recognition

Several platforms have emerged to recognize the work peer reviewers do. Publons emerged in 2012 as a service for academics to showcase their peer review and editorial contributions¹². This platform has now been adopted by many publishing groups^13,14. Publons merits are a method of measuring peer reviewer contribution, whereby peer reviewers gain points on their profile for each peer review and further points if this review is verified and made publicly available. These points provide quantitative proof of a peer reviewers contributions. There is the possibility that this could lead to peer reviewing being the latest focus of career advancement. However, given the important attributes required to peer review and the skills developed, this trend can easily be justified.

Bonuses from publishing groups

A UK House of Commons report 2011 highlighted.

“All publishers need to have in place systems for recording and acknowledging the contribution of those involved in peer review”². While the majority of peer reviewers are unpaid, journals often provide other perks to their reviewers, as added incentives and recognition of their work. Some journals publish the names of the reviewers in the final paper, or list reviewers at the end of an issue for the entire year^15–18. Elsevier also gives 30 days access to its products SCOPUS and ScienceDirect¹⁹. One example is Elsevier’s Reviewer Recognition Programme²⁰. This scheme aims to acknowledge and reward the work of peer reviewers. Peer reviewers are provided special access to the publishing group’s journal resources and discounts on affiliated services such as Elsevier Web shop and Elsevier Book Store. Awards are also provided to the top reviewers of the year.

Publication

Some journals now offer to publish peer reviewer alongside the main paper or in batch with each review given its own digital object identifier. This provides further recognition of the peer reviewers contribution to the field. Publication of peer reviews also increases the transparency of the peer review process; it provides accountability and responsibility to peer reviewers and clarity to authors²¹.

Publication of peer review has many legal and ethical intricacies for reviewers, authors, and publishers. This is an ongoing debate²², however, it is likely that the trend for peer review publication will continue.

Conclusions

Peer reviewing forms a fundamental part of the scientific process and has a long tradition. Although it does have drawbacks, such as being time-consuming and having potential conflicts of interest, overall there are many advantages to the peer review system. It is important, both for authors and editors, who greatly benefit from the expertise and recommendations delivered by a good peer reviewer. Moreover, it ensures, as much as possible, to filter and uphold the quality of the scientific literature and help advance the scientific knowledgebase. Reviewers are now increasingly recognized for their work, providing peer reviewing with a bright future.

Ethical approval

Ethical approval was not required for this study.

Sources of funding

There were no external sources of funding for this study.

Author contribution

All authors contributed toward the final paper. A.J.F., K.K., B.G., and R.A.A.: production of content/editing.

Conflict of interest disclosure

R.A.A. is the founding and executive editor of the IJS Publishing Group. The remaining authors declare that they have no financial conflict of interest with regard to the content of this report.

Research registration unique identifying number (UIN)

No research registration was required for this study.

Guarantor

The guarantor for this study is author R.A.A.