Expanding Group Peer Review: A Proposal for Medical Education Scholarship : Academic Medicine

Secondary Logo

Journal Logo

Invited Commentaries

Expanding Group Peer Review: A Proposal for Medical Education Scholarship

Dumenco, Luba MD; Engle, Deborah L. EdD, MS; Goodell, Kristen MD; Nagler, Alisa JD, MA, EdD; Ovitsh, Robin K. MD; Whicker, Shari A. EdD, MEd

Author Information
doi: 10.1097/ACM.0000000000001384
  • Free
  • AM Rounds Blog Post

Abstract

Several outcomes can mitigate the sting of the cost of conference travel and attendance. Money and time “well spent” may be determined by forming and fostering relationships with like-minded colleagues, discovering more effective and innovative approaches to our work, feeling energized by new knowledge and plans for its implementation, or discovering fresh ideas for sharing “what works” through scholarship and research. While any one of these conference outcomes would be encouraging, a combination represents the holy grail. So, imagine our delight when we experienced four of these four outcomes in one 2-hour session at the 2015 Association of American Medical Colleges Medical Education Meeting.

The Story of Our Collaboration

The title of the workshop we attended, “Accept, Reject, or Revise? Improving Scholarship in Medical Education Through Improving Peer Review,” conveyed a sense of hope for improving not only the way in which we serve as reviewers but also the way in which we write as authors. We each entered with optimism and happened to sit at the same table; most of us did not know each other prior to the session. The presenters, Academic Medicine and MedEdPORTAL editors, provided a thorough review of the peer-review process and proposed some benefits of being a peer reviewer (which we will elaborate on below). They also provided an overview of the updated, second edition of “Review Criteria for Research Manuscripts” (RCRM),1 sharing its checklist of criteria that reviewers can use to evaluate submitted manuscripts. After the overview, the real work began. To illustrate the peer-review process, each table work group assumed the role of reviewer and used the RCRM checklist to review an actual manuscript that was submitted to the journal, focusing on a specific section of the paper. The process was illuminating to say the least.

Through our discussions, our table work group began to realize that the ways in which we would have reviewed the manuscript as individuals were very different and certainly not as robust as the way in which we reviewed the manuscript as a group. Because each contributor in our group had a different area of expertise, the group review process elicited many more factors than we would have considered on our own. It encouraged us to give more thought and close consideration to areas that we individually might have easily accepted or dismissed outright. This process allowed for the development of more balanced and nuanced feedback on the manuscript, advancing our development as reviewers and equipping us with new skills so that we may further develop as authors.

After all table work groups had reported on their sections of the manuscript, the presenters took an informal poll to assess the groups’ publication recommendations. Although it was clear that prior to the group review experience, individual reviewers would have scored the manuscript very differently, 100% of the table work groups agreed to recommend inviting the authors to resubmit the manuscript with major revisions. After the poll, the workshop participants reviewed the actual reviewer feedback provided to the authors, and we were struck by how much richer and more meaningful the groups’ feedback would have been.

Our table work group walked away from the workshop with the holy grail. We had formed new relationships with an interprofessional group of colleagues from different institutions, discovered an innovative approach to reviewing and writing manuscripts, and felt energized by our new appreciation for other reviewers’ perspectives that would also enhance our own reviewing and writing. We collaborated on a plan to reconvene and write up the group peer-review process together to share our experience with other medical educators. In this Commentary, we review and discuss reasons individuals might serve as peer reviewers, consider the potential benefits of group peer review (as we found), and propose ideas for ways to study and implement group peer review in medical education scholarship.

Why Review?

Peer review lends validity and scientific credibility to scholarly journals.2–4 With the prevalent acceptance of the peer-review process in the scientific community, articles that do not pass through the process may not be as respected as those that do.4 The peer-review process requires the dedication and work of volunteers, who contribute considerable time and effort.3 Individuals qualified to review manuscripts undoubtedly have their own commitments and deadlines,2 and they often receive little acknowledgment or formal reward for their service as peer reviewers.3 Some journals will acknowledge this contribution by publishing a list of reviewers in an issue and/or sending reviewers a letter of acknowledgment annually. Journals may also offer awards or spots on the editorial board to their best reviewers.5 Mostly, though, dedicated service as a reviewer is not known to the public and represents a donation of time and energy. There is no doubt that the work can be arduous and underappreciated.3 So why do people do it?

For many reviewers, there are numerous benefits to reviewing manuscripts, far outweighing the perceived burden of the work required. Reviewers’ motivation to participate in the peer-review process may be uniquely individual and may evolve with time. Historically, reviewers were motivated to contribute in return for prestige and fame.4 The position of reviewer has long been viewed as one of distinction. As Polak5 writes, “being invited to become a reviewer carries the implication that the person is being recognized as an expert in the field—a definite ego boost.”

Further, the process provides an opportunity for reviewers to contribute to the body of knowledge in their discipline.3 Reviewers help determine the quality and direction of research in a specific field, which can be rewarding in itself.4 Reviewers’ sense of professional responsibility may also stem from their need to have their own work reviewed. Thus, reviewers who also write and submit manuscripts experience a dependence on and gratitude for the meaningful feedback they receive from others, which enhances their writing and helps ensure quality publications.

In addition, reviewers may find reward in the process as a form of teaching. According to Garmel,3 “the satisfaction that reviewers gain from educating and mentoring peers (even those they do not know) is rewarding.” Similarly, reviewers may see the process as an opportunity for learning2 through which “they will find that reviewing others’ manuscripts improves their own research, study designs, and writing.”3

Although there are multiple studies describing peer-reviewer benefits, there is a dearth of literature on the reasons individuals serve as peer reviewers. Ware6 reports that peer reviewers responding to a survey “preferred to offer more altruistic explanations for why they reviewed,” with most agreeing that they reviewed “to play [their] part as a member of the academic community.” Respondents endorsed more “self-interested reasons” related to advancing their career or reputation less often.

There is also limited research on the group peer-review process. That our experience at the workshop demonstrates the value of a group model for reviewing manuscripts is not surprising, as there is ample evidence supporting the superiority of teamwork over individual work for enhancing learning and collaboration.7–10

Teamwork and Team-Based Learning Theory

The importance of teamwork in medicine and in medical education is evident. As medical educators, we intentionally incorporate the social constructivist approaches of team-based, small-group, and interprofessional learning into our curricula with the aims of increasing engagement, discourse, cooperation, collaboration, and critical thinking.7,8 Our dynamic group peer-review experience reminded us that team learning directly influences team performance and that the exchange of ideas among team members allows individuals’ knowledge and experience to be combined to improve team effectiveness.10

Our experience of collaborating within our group to arrive at a single set of recommendations for the reviewed manuscript demonstrated for us the importance of the team learning processes described by Decuyper et al.11 Specifically the processes of sharing (communicating knowledge and experience, opinions, and creative ideas), co-construction of meaning (building new meaning from the combined input of individual members), and constructive conflict (questioning of assumptions, identifying and negotiating conflicting information or perceptions) were integral components of our teamwork.11

Although we might have been able to predict the superiority of teamwork over individual work in this workshop, we were all impressed by the excellent quality of the review feedback that resulted from the highly engaged and concerted effort of the group process, compared with our own individual feedback and even the original reviewers’ comments.

A Proposal: Expand Group Peer Review

Given the described benefits of the group peer-review model for both the authors whose work is under review and the reviewers themselves, we propose that institutions, journal editors, and professional organizations consider ways to formally study and expand the team review model. It would be helpful to gather evidence to support the model and to determine whether there are specific types of papers that would benefit most from group peer review (e.g., manuscripts that are particularly complex, those that receive widely variable reviews from the initial individual reviewers).

There are multiple possibilities for expansion of the group peer-review process. Scholarly peer-review workshops could be held alongside professional meetings in the way that exam item–writing workshops are held, such that a large number of submissions could receive high-quality team reviews in a short period of time. Individual institutions could run group peer-review workshops for junior faculty to help them become better reviewers and develop their scholarly writing skills, and to encourage them to write and review themselves. Institutions or departments could also use a group peer-review process to improve the quality of manuscripts before they are submitted, potentially improving their acceptance rates. Finally, groups of faculty and/or learners could review already-published papers, as a modified journal club that focuses on writing and scholarship rather than the content of the research.

The benefits of teamwork in medicine, teaching, and learning are well established. We encourage the editors and readers of Academic Medicine and other peer-reviewed journals to explore using a team-based approach in the peer review of medical education scholarship, which potentially could create a new methodology for developing skills in research and scholarly writing and could enhance the quality of our profession’s scholarship.

References

1. Durning SJ, Carline JD. Review Criteria for Research Manuscripts. 2015. 2nd ed. Washington, DC: Association of American Medical Colleges; https://members.aamc.org/eweb/upload/Review%20Criteria%20For%20Research%20Manuscripts.pdf. Accessed July 11, 2016.
2. Snell L, Spencer J. Reviewers’ perceptions of the peer review process for a medical education journal. Med Educ. 2005;39:9097.
3. Garmel GM. Reviewing manuscripts for biomedical journals. Perm J. 2010;14:3240.
4. Benos DJ, Bashari E, Chaves JM, et al. The ups and downs of peer review. Adv Physiol Educ. 2007;31:145152.
5. Polak JF. The role of the manuscript reviewer in the peer review process. AJR Am J Roentgenol. 1995;165:685688.
6. Ware M. PRC Summary Papers 4. Peer Review: Benefits, Perceptions and Alternatives. 2008. London, UK: Publishing Research Consortium; http://publishingresearchconsortium.com/index.php/prc-documents/prc-research-projects/35-prc-summary-4-ware-final-1/file. Accessed July 11, 2016.
7. Davis BG. Tools for Teaching. 2009.2nd ed. San Francisco, CA: Jossey-Bass.
8. Page SE. The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. 2007.Princeton, NJ: Princeton University Press.
9. Barkley EF, Major CH, Cross KP. Collaborative Learning Techniques: A Handbook for College Faculty. 2014.2nd ed. San Francisco, CA: Jossey-Bass.
10. Kozlowski SW, Ilgen DR. Enhancing the effectiveness of work groups and teams. Psychol Sci Public Interest. 2006;7:77124.
11. Decuyper S, Dochy F, Van den Bossche P. Grasping the dynamic complexity of team learning: An integrative model for effective team learning in organisations. Educ Res Rev. 2010;5(2):111133.
Copyright © 2016 by the Association of American Medical Colleges