Peer Review, Confidential Comments to the Editor, and the Golden Rule : Annals of Plastic Surgery

Secondary Logo

Journal Logo

Editorial

Peer Review, Confidential Comments to the Editor, and the Golden Rule

Swanson, Eric MD

Author Information
Annals of Plastic Surgery 90(1):p 1-3, January 2023. | DOI: 10.1097/SAP.0000000000003320
  • Open

Like all medical specialties, plastic surgery journals solicit reviews as part of the peer review process. Two boxes are available in which to make online comments, “Reviewer Comments to Author,” and “Reviewer Confidential Comments to Editor.” Comments to authors are shared with the authors; comments to the editor are not. This common practice, which compromises transparency, has largely gone unchallenged. Authors are disadvantaged in 2 ways. First, authors are not informed of the identities of the reviewers. Second, they are unable to access some of the reviewers’ comments. By contrast, the reviewers know the identity of the authors (in a single blind format). Even if the author’s identity is not provided (double blind), the author is often recognized by the reviewers’ familiarity with their work and self-citations.1

EX PARTE COMMUNICATIONS

Similar to a trial, the editor acts as the judge, entrusted with an unbiased review of an author's manuscript (the defendant). The “jury” of reviewers is tasked with giving a recommendation to accept, reject, or (frequently) recommend a revision. The editor makes the final determination, the “verdict.”

In a court of law, it is a violation of due process for a judge to visit and exchange information with one party without the other party being present.2 Such one-sided conversations are called “ex parte.” They are prohibited because one party has the opportunity to influence the judge without the other party being present to rebut any accusations or even to be aware of them. Violations of this principle are taken seriously. Criminal verdicts may be overturned if the appellate court discovers that an ex parte communication influenced the verdict. Judges and lawyers may be disciplined.2

Unfortunately, a reviewer may, improperly, use this confidential space to inform the editor of prejudicial material. For example, the reviewer, who disagrees with the author, might inform the editor that the author has had multiple lawsuits and has been disciplined by a state medical board. This information prejudices the editor against the author, contributing to rejection of a manuscript. The editor is not informed that this state board action, which fueled the lawsuits, was instigated by competitors of the author, one of whom was a member and past president of the board, and the other his partner, who acted as the expert for the board. Subsequently, both competitors settle a federal anticorruption lawsuit for abuse of a public office, filed by the author. This sort of damaging ex parte communication is not a hypothetical scenario, but one that happened to me. The rejected manuscript was, fortunately, published in another journal (Annals of Plastic Surgery).3 This editorial helped debunk the discredited 14-point plan to reduce the risk of Breast Implant–Associated Anaplastic Large-Cell Lymphoma.

THE GOLDEN RULE

Reviewer advice may include the Peer Review Golden Rule. The principle is to “review unto others as you would have them review unto you.”4–6 On the surface, this would seem to be a reasonable request. However, it is not clear that the golden rule is consistent with the objective of maintaining a high level of quality in published articles. In a trial setting, the judge does not instruct jurors to observe the golden rule and decide as if they were the defendant. The jury's responsibility is to deliberate according to the facts and the law. Similarly, reviewers and editors are expected to rule according to the merits of the manuscript, regardless of other considerations such as how they would feel if they were the author.

Those reviewers who find they recommend rejections more often than revisions may be reassured that this ratio is justified. Among manuscripts received by prominent plastic surgery journals, approximately 80% to 85% are rejected.4,7 However, many rejected manuscripts eventually find publication in other journals.4 Today, an increased number of outlets for publication are available, including new online venues. A classic 2005 study showed that more than half of all published medical research findings are false.8 There is clearly a need for gatekeepers.

In addition to the golden rule of reviewing, advice from the editorial office may include an instruction not to leave the confidential comments box empty. In fact, this recommendation was emailed to me recently by a journal, probably because I never make comments in the confidential box. Reviewers are instructed, “if you have serious concerns about the manuscript, but fear they may verge on insulting or accusatory, express those types of comments in the ‘confidential comments to editor’ section.”5 It is interesting to consider whether making confidential comments satisfies the golden rule. How would the reviewer feel about hidden comments if he or she were the author? If reviewers cannot say what they need to say in the space viewed by the author, perhaps they should not be saying it.

To better assess the value of confidential comments, it is useful to have data. Recently, O'Brien et al9 published their study of 358 reviews of 168 manuscripts submitted to a health professions education journal with a single blind review process. Approximately half of the reviews (49%) included comments to the editor. Among 86 reviews recommending “reject,” the majority of the comments (80%) to the editor were also contained in the comments to the authors. The remaining 20% were judged to be irrelevant to the manuscript, begging the question as to why this box is needed.

TRANSPARENCY BENEFITS AUTHORS

In terms of spotting plagiarism (which is unusual), ethical breeches (in plastic surgery, this is often in the imaging), and undisclosed conflicts of interest (this occurs frequently), the party most concerned about their discovery is the author. By pointing out such problems during the peer review stage, the reviewer is doing the author a favor. The reviewer should not hesitate to mention such issues in the Comments to Author section. This way, the problem is quickly brought to the author's attention and may be corrected before publication rather than afterward in a letter to the editor.10 For example, an article promoting radiofrequency featured a photograph of a patient who had an undisclosed abdominoplasty. Unfortunately, for the authors, this error was missed by the reviewers, and the publication still contains this embarrassing error, which is likely to mislead the reader regarding the true capabilities of radiofrequency.11

Today, experienced reviewers often read the disclosure paragraph first. They may check the Open Payments Web site12 to look for undisclosed financial conflicts. This site, created by the Sunshine Act, may reveal undisclosed compensation to authors. The amount may be extraordinary (eg, US $700,000).13 Clearly, the author would want to know about such an oversight before publication in the chosen journal or in a different one if the manuscript is rejected. Although the Confidential Comments space is recommended for ethical breeches or conflicts of interest, these subjects are rarely raised (3%).9 To withhold such important issues from the author, and let him or her suffer the consequences, would seem to violate the golden rule.

Of course, the reviewer's identity is still protected if the reviewer chooses to mention such problems in the Comments to the Author section. Alternatively, the reviewer has the option to correspond directly with the editorial office. Reporting ethical breeches is not just expected of reviewers, but of any plastic surgeon, and is mandated by our code of ethics,14 which states that members “should expose, without hesitation, illegal or unethical conduct of fellow Members of the profession.”

TRANSPARENCY BENEFITS REVIEWERS

Transparency can benefit the reviewer as well as the author. Reviewers are often asked to provide a published discussion of a manuscript that they reviewed. If the reviewers' criticisms are fully revealed to the author during peer review, the author has an opportunity to respond in the “Responses to Reviewers” box. A reviewer error may be corrected before the discussion is published, rather than after. An example is 2 discussants erroneously alleging academic indiscretions on the part of an author for reporting that a reference was published in a plastic surgery journal when in fact it was “not even a peer-reviewed publication anywhere” (discussants' italics).15 A correction is later published.16 Obviously, the fewer corrections that are needed in our journals, the better.

Another benefit of transparency for reviewers is that they are recognized for their efforts. Reviewing is a time-consuming job. Experienced reviewers are expected to spend 3 hours doing a thorough review.4 Some reviewers who are heavily sought by journals may do dozens of unpaid reviews annually, often in the evenings and on weekends. By removing anonymity, reviewers may be recognized by their peers, and by authors, for their efforts to improve the quality of publications.

THE DOWNSIDE OF CONFIDENTIAL COMMENTS

This discussion leads to the question, why have a space for confidential comments to the editor in the first place? According to the Association of American Medical Colleges, confidential comments to the editor are meant to “recommend additional review by someone with specific expertise, make specific comments on the quality of the manuscript, provide opinions about the relevance or significance of the work, or raise potential ethical concerns.”17 Some journals recognize the potential for misuse. The online journal PLoS One instructs reviewers to use the comments to the editor section only to declare potential or perceived competing interests.18 No other confidential comments are permitted. The Committee on Publication Ethics advises reviewers to ensure that confidential comments are consistent with their report for authors; most feedback should be put in the report that authors will see.19

Plastic surgeons have told me of harrowing, personalized, and overheated reviews that have blocked or seriously delayed their publications and led them to seek out other journals simply to avoid a specific reviewer who is hostile to their manuscript. Confidential comments are likely to be even more inflammatory. When scientifically sound, and in some cases landmark, manuscripts are published in a nonplastic surgery journal or an international journal (albeit respected ones), or a much less visible publication with a low impact factor,20–22 such influence is suspected, especially when the authors come to conclusions that challenge the conventional wisdom. The famous dictum, “primum non nocere” does not just apply to patient care. Derogatory secret remarks are likely to harm researchers and their careers and may suppress important medical advances.

Concerns related to publication or research ethics can always be handled outside the peer review system, by email or telephone. Gratuitous comments thanking the editor for the opportunity to serve as a reviewer, curiosity as to what the editor thinks about the study, second-guessing their review and requesting feedback, explaining how they were being charitable to the author despite reservations, or reiterating their comments to the authors (all examples provided in the study by O’Brien et al)9 are unnecessary.

Receiving a negative editorial decision that does not align with the reviewers' generally positive comments is frustrating for authors. Compliments in one section followed by hidden criticism in the other are hard to reconcile with any degree of sincerity or good faith, let alone the golden rule. Nonalignment of the tone of the comments (9% of reviews)9 does not conform to ethical guidelines, which call for nonconflicting comments.17,19 Confidential comments are always more critical.9 Why should the confidential comments be more definitive in determining the outcome than the comments to the authors?

Receiving a rejection with no reasons or comments, possibly based on unshared confidential comments, is unfair to authors.9 Peer review serves the dual purpose of assisting editors with publication decisions and providing authors with constructive feedback.9 Authors have gone to some trouble to conduct the study and submit their manuscripts but are not given the benefit of a review of its strengths and weaknesses in return.

Gossip is defined as casual or unconstrained conversation about other people, typically involving unconfirmed details,23 often told in their absence, and damaging to their reputation. Of course, humans, being humans, tend to listen to gossip. As Olympia Dukakis’s character quipped in Steel Magnolias, “Well, you know what they say: if you don’t have anything nice to say about somebody, come and sit beside me!”24 An editor once told me, “I know who hates who.” We have all seen online clickbait with teasers such as “actor costars who hated each other in real life,” and it can be difficult to resist clicking them. One can only imagine how many libel lawsuits would be filed if all confidential comments to the editor were made public!

REVIEWER ANONYMITY

Whether reviewers' identities should be confidential in the first place is at least open to question. The concern is that reviewers may not be forthcoming with honest reviews if the author is a person of influence in a specialty and can impact the reviewer's career.9 However, the downside is that anonymous reviewers may make loose or derogatory comments that they would not make if their identity were known. A goal of open peer review is to improve the quality of reviews and eliminate problems caused by nontransparency.25 Some journals either require reviewers to share their identity or give them that option.9 A recent study found that reviews for articles in journals with an open peer review policy were significantly less harsh than those with an anonymous review process.26 Cursory reviews and tardiness, even typographical errors, are less likely to be a problem. There is a safeguard to protect the reviewer who is squeamish about revealing his or her identity—that individual may decline to serve as a reviewer.

Ethical guidelines include the following provision: “If they [the reviewer] are the editor handling a manuscript and decide themselves to provide a review of that manuscript, do this transparently and not under the guise of an anonymous review if the journal operates blind review.19 This instruction raises the question, why is there a double standard for editors and reviewers regarding anonymity? These guidelines also call for reviewers to reveal any close personal or professional connection they may have with an author.19 Reviewers are expected to decline an invitation if they believe that revealing their identity (in an open peer review format) would compromise their review.19

THE SOLUTION

The first step is an easy one. Simply make all reviewer comments visible to authors. Be transparent. Delete the unnecessary gossip column. Let reviewers keep their anonymity, at least for the time being. Let the balance fall closer to neutral rather than weighted against the author. Like honesty, transparency is the best policy.

“I've always believed that a lot of the trouble in the world would disappear if we were talking to each other instead of about each other.”—Ronald Reagan27

REFERENCES

1. Chung KC, Shauver MJ, Malay S, et al. Is double-blinded peer review necessary? The effect of blinding on review quality. Plast Reconstr Surg. 2015;136:1369–1377.
2. CriminalDefenseLawyer. What is an ex parte communication? Available at: https://www.criminaldefenselawyer.com/resources/criminal-defense/criminal-defense-case/what-ex-parte-communication. Accessed August 20, 2022.
3. Swanson E. A 1-point plan to eliminate breast implant–associated anaplastic large-cell lymphoma. Ann Plast Surg. 2018;80:565–466.
4. Kotsis SV, Chung KC. Manuscript rejection: how to submit a revision and tips on being a good peer reviewer. Plast Reconstr Surg. 2014;133:958–964.
5. Plastic and Reconstructive Surgery & PRS Global Open Reviewer Handbook. Available at: https://cdn-links.lww.com/permalink/prs/d/prs_2022_02_09_editorialoffice_reviewerhandbook2022_sdc1.pdf. Accessed August 20, 2022.
6. Lineaweaver WC. Some thoughts on the ethics of last-minute editing. Ann Plast Surg. 2022;89:1–2.
7. Lineaweaver WC. Appeals. Ann Plast Surg. 2015;74:273.
8. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2:e124.
9. O'Brien BC, Artino AR Jr., Costello JA, et al. Transparency in peer review: exploring the content and tone of reviewers' confidential comments to editors. PLoS One. 2021;16:e0260558.
10. Swanson E. Concerns regarding photographic representation of the results of radiofrequency treatment. Aesthet Surg J. 2019;39:NP354–NP356.
11. Dayan E, Chia C, Burns AJ, et al. Adjustable depth fractional radiofrequency combined with bipolar radiofrequency: a minimally invasive combination treatment for skin laxity. Aesthet Surg J. 2019;39(suppl 3):S112–S119.
12. Search Open Payments. Available at: https://openpaymentsdata.cms.gov/. Accessed August 20, 2022.
13. Swanson E. Concerns regarding dishonesty in reporting a large study of patients treated with Allergan Biocell breast implants. Ann Plast Surg. 2022;88:585–588.
14. Code of Ethics of the American Society of Plastic Surgeons. Available at: https://www.plasticsurgery.org/documents/Governance/asps-code-of-ethics.pdf. Accessed August 20, 2022.
15. Adams WP Jr., Calobrace MB. Discussion: the questionable role of antibiotic irrigation in breast augmentation. Plast Reconstr Surg. 2019;144:253–257.
16. Discussion: the questionable role of antibiotic irrigation in breast augmentation: correction. Plast Reconstr Surg. 2020;146:1212.
17. Durning SJ, Carline JD. Review criteria for research manuscripts, 2nd ed. Available at: https://depts.washington.edu/dbpeds/Review%20Criteria%20For%20Research%20Manuscripts.pdf. Accessed August 21, 2022.
18. PLOS ONE Guidelines for Reviewers. Available at: https://journals.plos.org/plosone/s/reviewer-guidelines. Accessed August 20, 2022.
19. Committee on Publication Ethics. COPE Ethical Guidelines. Available at: https://journals.lww.com/cns-journal/Documents/Ethical_guidelines_for_peer_reviewers_0.pdf#:~:text=The%20COPE%20Ethical%20Guidelines%20for%20Peer%20Reviewers%20set,peer%20reviewers%20should%20adhere%20during%20the%20peer-review%20process. Accessed August 18, 2022.
20. Samargandi OA, Joukhadar N, Al Youha S, et al. Antibiotic irrigation of pocket for implant-based breast augmentation to prevent capsular contracture: a systematic review. Plast Surg (Oakv). 2018;26:110–119.
21. Walker JN, Hanson BM, Pinknes CL, et al. Insights into the microbiome of breast implants and periprosthetic tissue in breast implant–associated anaplastic large cell lymphoma. Sci Rep. 2019;9:10393.
22. Cordeiro PG, Ghione P, Ni A, et al. Risk of breast implant associated anaplastic large cell lymphoma (BIA-ALCL) in a cohort of 3546 women prospectively followed long term after reconstruction with textured breast implants. J Plast Reconstr Aesthet Surg. 2020;73:841–846.
23. Definition of gossip. Available at: https://www.bing.com/search?q=gossip+definition&cvid=14aa7f42f9444cea8e93188f77de739e&aqs=edge.0.0l9.3263j0j1&pglt=675&FORM=ANSPA1&PC=HCTS. Accessed August 20, 2022.
24. Quotes. Steel magnolias. Available at: https://www.quotes.net/mquote/92087. Accessed August 21, 2022.
25. Ross-Hellauer T. What is open peer review? A systematic review. Available at: https://doi.org/10.12688/f1000research.11369.2. Accessed August 18, 2022.
26. Le Sueur H, Dagliati A, Buchan I, et al. Pride and prejudice—what can we learn from peer review?Med Teach. 2020;42:1012–1018.
27. Ronald Reagan quotes. Available at: http://www.planetofsuccess.com/blog/2019/ronald-reagan-quotes/.
Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc.