Physicians in academic medicine have most likely submitted a scientific manuscript for publication. After all, the famous saying is “publish or perish.” If all of your submitted manuscripts have been accepted straightaway without revision, you are one of the lucky few who no longer need to continue reading. The majority of authors, however, have had manuscript submissions that resulted in some form of rejection. Rejection should not be unexpected, given that only 9 percent of the 6000 annual manuscript submissions to the Journal of the American Medical Association are accepted for publication.1 Nearly 85 percent of submissions to Plastic and Reconstructive Surgery are rejected. Even manuscripts that later resulted in a Nobel Prize have been rejected for publication.2 For example, Nature rejected manuscripts on Cerenkov radiation (used in nuclear reactors) and work on photosynthesis by Deisenhofer, Huber, and Michel (who determined the first crystal structure of a protein that is essential to photosynthesis), and initially rejected, but eventually accepted, Stephen Hawking’s manuscript on black-hole radiation. Rejected. Unaccepted. Refused. Declined. Whatever the word, they all have the same meaning. However, authors must remember that it is normal to go through a “grieving process” after receiving a letter of rejection. The purpose of this article is to discuss the process of peer review in scientific manuscripts, to provide pointers on how to tackle rejection and revision, and to offer guidance on how to be a good peer reviewer.
WHAT IS PEER REVIEW?
A peer-reviewed journal is one that submits most of its published articles for outside review. Peer review became the standard in biomedical journals in English-speaking countries in approximately the mid-twentieth century.3 The International Congress on Peer Review in Biomedical Publication was formed and held its first congress in 1989 to improve the manner in which science is reviewed, selected, and disseminated.4 Peer reviewers are considered to be experts in their field who critically assess manuscripts submitted for publication.5 Peer reviewers help editors decide whether to publish a manuscript, and they ensure that the best manuscripts are published by providing the authors with critical feedback. Peer review is usually assumed to raise the quality of the end product,6 and clinicians give greater weight to findings published in peer-reviewed journals.7 However, there is no standard method for conducting peer review, and a Cochrane study found that there was a lack of evidence to support the use of peer review to improve biomedical studies.8 Studies assessing the impact of training sessions for peer reviewers have found mainly that they did not improve the quality of reviews.9–12 One study found that peer reviewers are biased in favor of orthodox forms of treatment versus alternative therapies,13 but another study found no such bias.14 Because of these contradictory findings, the Cochrane study suggested that further research was needed on the effects of peer review.
Each journal has its own process regarding how peer reviews are conducted. Several use a blinded review in which the reviewers’ identity is not revealed to the authors, and others use masking (also known as double-blinding when used in combination with blinding) to withhold the authors’ identity (e.g., name, institution) from the reviewers.15 Some studies have found that masking improves the quality of the review,16 whereas other have found no difference in the quality17 or in the likelihood of manuscript acceptance using masking.18 Furthermore, some researchers have argued that masking is time-consuming and difficult because of authors’ inadvertent or deliberate self-identification throughout the manuscript.18–22 One study found that 46 percent of reviewers were able to correctly identify the authors of masked studies.23 Thus, blinding may reduce the chance of receiving a biased review, but it should not greatly affect the quality of the review or the chance of a manuscript being accepted.
Submitted manuscripts are first scanned by the editor to determine whether the manuscript is suitable for the journal. If it is deemed not to be suitable, the editor will probably reject the manuscript outright before sending it for peer review. The editor may also reject the manuscript outright if it has a “fatal flaw.” Fatal flaws could include a lack of importance of the research topic or an inappropriate study design.24 Other fatal flaws may include invalid data, conclusions that are not supported by the data, or material that is not original or timely.25 Fatal flaws cannot be corrected with additional data or clarity because they are errors in logic or approach.26
The editor then routes the manuscript for peer review, perhaps to two or three reviewers, depending on the journal. After the reviews are complete, dispositions may include “rejection without further submission,” “rejection with an opportunity to resubmit,” “major revision without a promise of acceptance,” “minor revision,” “acceptance subject to minor revision,” or “outright acceptance.”27
DEALING WITH REJECTION
After receiving notification that a manuscript requires major revision or is rejected without the opportunity for revision, authors may feel disappointment and even resentment. However, the criticisms should not be taken personally.28 As a note of encouragement, one study found that 76 percent of manuscripts rejected from the Journal of Bone and Joint Surgery were published by another journal within an average of less than 2 years, and 27 percent of these manuscripts were published in high-impact general orthopaedic journals.29 So, if the first journal does not accept your manuscript, you must try, try again.
When a major revision is required, the authors should read the reviewers’ comments, then put them aside for a few days to think about how to reply so that an emotional response is avoided.28 Pausing to think about a response is also a good tip when receiving any type of critical feedback.30 Rather than spending time and energy reacting to criticism, authors should respond to the reviewers’ comments.28 The authors should not spend too much time wallowing in their sorrows because a long delay before resubmission may suggest that there is a problem or that the paper is not a high priority.31 Thus, try to revise and/or resubmit as quickly as possible.
SUBMITTING A REVISION
After accepting what has happened, the next step is to determine how to proceed with a rejected manuscript. How to proceed depends on what type of rejection was received. Figure 1 illustrates the choices that an author could make after receiving the feedback from his or her manuscript submission. Comments from the editor and/or reviewers are usually valuable pieces of information that can only improve a manuscript. After reading these comments carefully, the author then needs to decide whether the revised manuscript should be submitted to a new journal that is more applicable to his or her research or if the manuscript can be resubmitted to the same journal.
If the manuscript requires major or minor revisions, the author should follow the journal’s guidelines for revision as noted in the instructions to authors. A reviewer response usually involves addressing each of the reviewer’s comments and specifying where in the manuscript the changes have been made. The journal instructions may require that edits are indicated in a different color font. In the reviewer response, the author should specify where changes were made to the manuscript, such as, “As the reviewer requested, on page 5, paragraph 2, line 6, we have included the mean age of the sample as 56.5 years.” Reviewers or editors should not be expected to be “manuscript detectives” to find all of the revisions made by the authors.32 Authors will make everyone’s life easier by spelling out the changes the first time around. Responses should most likely include changes to the text of the manuscript rather than just address the reviewer in the response letter. After all, if a peer reviewer had a reasonable question about a manuscript, others will likely have the same question when they read it. If no specific format is suggested, a good way to proceed is to copy and paste an itemized list of the reviewer’s comments into the response document. The author should provide a response to each comment underneath. If the reviewer provided a compliment, the author can thank him or her for that compliment.33 Examples of responses are provided in Table 1. Authors do not have to agree with all of the critiques, but they should compromise to make changes that they are able to make and explain why they choose not to make certain changes. “A well-prepared response document should be complete, polite, and based on evidence, not emotion!”33 Authors should avoid inflammatory remarks such as “we do not agree with the reviewer” or “we respectfully disagree.” Instead, remarks could be rephrased as “we agree with the reviewer that our response rate is low; however….” It is most likely that a resubmitted manuscript will be returned to the original reviewers, so they may have the upper-hand in its fate. On the other hand, authors do not need to be servile and agree to all of the reviewers’ requests. Authors should respond thoughtfully and without emotional overtone. In the end, authors need to use the reviewers’ comments to improve the manuscript. The Golden Rules for responding to reviewers are (1) answer completely; (2) answer politely; and (3) answer with evidence such as citations from the literature or additional analyses.33
If the reviewer requests extensive changes (requiring more than one page), the authors can reasonably suggest that these additions are better suited in a subsequent article.33 In the response, authors should not question whether the manuscript would have received a more favorable outcome had it been reviewed by other reviewers.34 If the authors choose to submit to another journal, they may be able to suggest reviewers upon submission. Some journals suggest including the decision letters and responses to the reviewer’s critiques in a submission to their journal. Before submitting to a different journal, authors should clarify any points that were misunderstood by previous reviewers and editors.35
HOW TO BE A GOOD PEER REVIEWER
Being asked to review a manuscript is a responsibility that all reviewers should honor. Of course, the reviewer’s comments are beneficial to authors, but reviewing can also be beneficial to the reviewer by improving critical thinking skills.32 Although peer review takes time and does not involve monetary compensation, the request should not be taken lightly. After receiving a request to provide a review, one should first decide whether one has the time to do it. A thorough review for an experienced reviewer is expected to take approximately 3 hours36,37 and will range from one half to two pages.26 A reviewer’s tardiness can potentially affect the career of a younger colleague who is relying on a publication for promotion or tenure.38 Furthermore, if a peer reviewer only has enough time to provide a cursory, one-sentence review, this is not helpful to the editor or to the authors. Second, one should decline the review if the manuscript does not fit one’s expertise.36 A study found that reviewers with training in epidemiology or statistics provided higher-quality reviews.37 Likewise, the review should be declined if there is a conflict of interest. A conflict of interest may include knowing the authors personally, investigating the same topic, or working for a company with an interest in the manuscript results.36 Working in the same institution as the authors is a conflict unless the institution is large enough that the reviewer and the authors are not working as colleagues.39
Reviewers may wish to first give a quick read of the manuscript to determine its quality after deciding to proceed with the review. If the quality is poor, including one of the fatal flaws discussed previously, the reviewer may wish to include only major comments in the revision. Similarly, if the manuscript is poorly written such that it interferes with understanding or interpretation, the reviewer may request the authors to rewrite it before the reviewer provides a full review.32 Other journals have received an increasing number of manuscript submissions from authors who reside outside of the United States.40,41 The American Journal of Roentgenology found that the acceptance rate of manuscripts from countries where English is not the primary language was significantly lower than the acceptance rate in countries where English is the primary language.42 However, language barrier was not a major reason for manuscript rejection from most countries. In fact, “major language problems” was the cause of rejection of only two articles, both from Japan. If a reviewer encounters language barriers from international authors, he or she can guide them to Web sites for companies that provide editing for scientific writing. Our observation is that authors whose native language is not English do have a distinct advantage. If international authors spend the effort to write their paper to conform to standard scientific English and the flow of the paper is clear and understandable, reviewers will tend to view the submission more favorably because they recognize the intense effort by the authors to deliver a competitive product. Therefore, all authors need to spend the time and resources necessary to ensure that a paper is not rejected because of poor prose and unconventional expressions.
The structure of a review could begin with a short summary of the key findings and value of the manuscript. A summary benefits the reviewer, editor, and authors by giving the reviewer’s perspective of the manuscript. Comments to the authors can then be structured under major and minor points. It is best to itemize the comments so that the authors can easily address them. Major comments are critical to the validity of the study and could include whether the appropriate study sample was chosen and if proper statistical tests were conducted. Plastic and Reconstructive Surgery will reject a paper outright if inappropriate or incorrect statistical analysis is presented. The editor will use these major comments to decide whether to accept or reject the manuscript.32,36 Minor comments should also be itemized and may include clarification of specific sentences, requests for additional data, or grammatical errors. However, reviewers should not focus on correcting grammar and spelling. If possible, the reviewer can refer to specific line numbers where clarification is requested. The reviewer should not include his or her recommendation as to their decision on the manuscript in their comments to the authors. These recommendations as well as confidential comments to the editor should be separate. “A critical aspect of all reviews is consistency between the comments to the authors and the recommended disposition, and comments to the editor are often the best solution.”32 For example, if the reviewer suggests rejecting the manuscript but provides comments to the authors for revisions, he or she may wish to clarify to the editor that the topic of the paper is worthwhile but the way the manuscript is currently written is not of sufficient quality for publication.32 A checklist of study criteria for reviewers is provided in Table 2. Although the editor may have scanned the manuscript for fatal flaws, the reviewer should still be on alert for them. Reviewers need to provide constructive advice on how authors can improve their research and communicate their findings clearly.35 During journal club review, students and residents are often trained to identify faults in papers. Turning off the instinct to find faults can be difficult, but the reviewer must remember that there are limitations in every study. Likewise, the manuscript should be judged on the study that was conducted rather than the study that the reviewer thinks the authors should have conducted.31 Reviewing should be approached as a collaborative effort to encourage colleagues. Authors need to be given specific recommendations on how they can improve their manuscript. There is no need for rudeness or criticism. Instead of saying that a study needs to expand the background section, the reviewer can refer to a few specific studies for the authors to consider. If the authors failed to include a reviewer’s publications that are relevant to the study, he or she can simply guide them toward those papers.38
Giving and receiving critical feedback are both difficult jobs. Authors who have received a rejection letter should remember to respond politely and with evidence rather than reacting in anger to any criticism. On the other hand, peer reviewers should remember this variant of the Golden Rule: “Review unto others as you would have them review unto you.”35
This work was supported in part by grants from the National Institute of Arthritis and Musculoskeletal and Skin Diseases and the National Institute on Aging (R01 AR062066) and from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01 AR047328) and a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) to Dr. Kevin C. Chung.
2. . Coping with peer rejection. Nature. 2003;425:645
3. Burnham JC. The evolution of editorial peer review. JAMA. 1990;263:1323–1329
4. Rennie D, Flanagin A, Godlee F, Groves T. Seventh international congress on peer review and biomedical publication, September 2013—Call for research. JAMA. 2012;307:726–727
5. . Uniform requirements for manuscripts submitted to biomedical journals: Ethical considerations in the conduct and reporting of research: Peer review. http://icmje.org/ethical_3peer.html
. Accessed July 25, 2013
6. Rennie DGodlee F, Jefferson T. Editorial peer review: Its development and rationale. Peer Review in Health Science. 20032nd edition London BMJ Books:1–13
7. Sievert ME, McKinin EJ, Johnson ED, Reid JC, Mitchell JA. Beyond relevance—Characteristics of key papers for clinicians: An exploratory study in an academic setting. Bull Med Libr Assoc. 1996;84:351–358
8. Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2007:MR000016
9. Callaham ML, Wears RL, Waeckerle JF. Effect of attendance at a training session on peer reviewer quality and performance. Ann Emerg Med. 1998;32(3 Pt 1):318–322
10. Callaham ML, Schriger DL. Effect of structured workshop training on subsequent performance of journal peer reviewers. Ann Emerg Med. 2002;40:323–328
11. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: Randomised controlled trial. BMJ. 2004;328:673–677
12. Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008;101:507–514
13. Resch KI, Ernst E, Garrow J. A randomized controlled study of reviewer bias against an unconventional therapy. J R Soc Med. 2000;93:164–167
14. Ernst E, Resch KL. Reviewer bias against the unconventional? A randomized double-blind study of peer review. Complement Ther Med. 1999;7:19–23
15. Davidoff F. Masking, blinding, and peer review: The blind leading the blinded. Ann Intern Med. 1998;128:66–68
16. McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The effects of blinding on the quality of peer review. A randomized trial. JAMA. 1990;263:1371–1376
17. Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D. Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators. JAMA. 1998;280:240–242
18. Alam M, Kim NA, Havey J, et al. Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: A randomized multi-rater study. Br J Dermatol. 2011;165:563–567
19. Block AJ. Blinded reviews. Chest. 1998;114:1501–1502
20. Liebeskind DS. The fallacy of double-blinded peer review. AJR Am J Roentgenol. 2003;181:1422; reply 1422–reply 1423
21. Katz DS, Proto AV, Olmsted WW. Incidence and nature of unblinding by authors: our experience at two radiology journals with double-blinded peer review policies. AJR Am J Roentgenol. 2002;179:1415–1417
22. Yankauer A. How blind is blind review? Am J Public Health. 1991;81:843–845
23. Fisher M, Friedman SB, Strauss B. The effects of blinding on acceptance of research papers by peer review. JAMA. 1994;272:143–146
24. Bordage G. Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Acad Med. 2001;76:889–896
26. Brand RA. Reviewing for clinical orthopaedics and related research. Clin Orthop Relat Res. 2012;470:2622–2625
27. Peh WC, Ng KH. Dealing with returned manuscripts. Singapore Med J. 2009;50:1050–1052; quiz 1053
28. Provenzale JM. Revising a manuscript: Ten principles to guide success for publication. AJR Am J Roentgenol. 2010;195:W382–W387
29. Okike K, Kocher MS, Nwachukwu BU, Mehlman CT, Heckman JD, Bhandari M. The fate of manuscripts rejected by The Journal of Bone and Joint Surgery
(American Volume). J Bone Joint Surg Am. 2012;94:e130–139
30. Chung KC. First hand: Giving and receiving feedback. J Hand Surg Am. 2013;38:151–152
31. Demaria A. Manuscript revision. J Am Coll Cardiol. 2011;57:2540–2541
32. Rosenfeld RM. How to review journal manuscripts. Otolaryngol Head Neck Surg. 2010;142:472–486
33. Williams HC. How to reply to referees’ comments when submitting manuscripts for publication. J Am Acad Dermatol. 2004;51:79–83
34. Woolley KL, Barron JP. Handling manuscript rejection: Insights from evidence and experience. Chest. 2009;135:573–577
35. Drubin DG. Any jackass can trash a manuscript, but it takes good scholarship to create one (how MBoC promotes civil and constructive peer review). Mol Biol Cell. 2011;22:525–527
36. Spigt M, Arts IC. How to review a manuscript. J Clin Epidemiol. 2010;63:1385–1390
37. Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA. 1998;280:231–233
38. Dutta MJ. The ten commandments of reviewing: The promise of a kinder, gentler discipline! Health Commun. 2006;20:197–200
40. Chen MY, Jenkins CB, Elster AD. Internationalization of the American Journal of Roentgenology
: 1980–2002. AJR Am J Roentgenol. 2003;181:907–912
41. Ozsunar Y, Unsal A, Akdilli A, Karaman C, Huisman TA, Sorensen AG. Technology and archives in radiology research: A sampling analysis of articles published in the AJR and Radiology
. AJR Am J Roentgenol. 2001;177:1281–1284
42. Ehara S, Takahashi K. Reasons for rejection of manuscripts submitted to AJR by international authors. AJR Am J Roentgenol. 2007;188:W113–W116
43. Peregrin T. How to cope with manuscript rejection. J Am Diet Assoc.. 2007;107:190, 2–3
44. Gardner MJ, Bond J. An exploratory study of statistical assessment of papers published in the British Medical Journal
. JAMA. 1990;263:1355–1357
48. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JPSTROBE Initiative. . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–349