We thank Dr. Phillips and colleagues for their letter detailing their experience with peer review on their blog. The process they describe sounds similar to ours, and we are excited to see other blogs taking on scholarly approaches to fostering future authors.
We read with interest the findings of increased readership attributable to the new peer review process. While our viewership has also increased (CanadiEM.org now receives > 40,000 page views per month), our article described the results of a survey which investigated the acceptability and feasibility of the coached peer review innovation. We did not focus on increased readership/followership metrics because we did not feel that it would be possible to remove the effect of other confounders.
However, we agree that growing numbers of followers are a likely result of increased quality. In fact, we have previously worked on an initiative that uses followership to infer the impact and quality of blogs.1 The Social Media Index has been met with skepticism within the online community associated with the Free Open Access Medical education (FOAM) movement,2 which led us to embark on a longer research program to develop quality evaluation tools for blog posts.3–6
Moving forward, we believe that it is imperative for the scholars within our field to consider outcome measures more explicitly tied to quality when investigating initiatives like Dr. Phillips and colleagues’ and ours. Unfortunately, the quality tools that we have developed have not yet been formally validated, but in the future it may be possible to audit our initiatives using these tools. By rating the quality of our posts before and after the implementation of coached peer review, we could provide further validity evidence that the coached peer review process improves the writing of junior blog contributors.
We look forward to hearing more about Dr. Phillips and colleagues’ work and possibly collaborating in the future.
Teresa M. Chan, MD, FRCPC, MHPE
Assistant professor of emergency medicine, McMaster University, Hamilton, Ontario, Canada, and senior editor, CanadiEM.org; firstname.lastname@example.org.
Brent Thoma, MA, MD, MSc, FRCPC
Assistant professor of emergency medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada, and editor-in-chief, CanadiEM.org.
1. Thoma B, Sanders JL, Lin M, Paterson QS, Steeg J, Chan TM. The social media index: Measuring the impact of emergency medicine and critical care websites. West J Emerg Med. 2015;16:242–249.
3. Lin M, Joshi N, Grock A, et al. Approved instructional resources series: A national initiative to identify quality emergency medicine blog and podcast content for resident education. J Grad Med Educ. 2016;8:219–225.
5. Chan TM, Grock A, Paddock M, Kulasegaram K, Yarris LM, Lin M. Examining reliability and validity of an online score (ALiEM AIR) for rating free open access medical education resources [published online ahead of print March 29, 2016]. Ann Emerg Med. doi: 10.1016/j.annemergmed.2016.02.018.
6. Chan TM, Thoma B, Krishnan K, et al. Derivation of two critical appraisal scores for trainees to evaluate online educational resources: A METRIQ study. West J Emerg Med. 2016;17:574–584.