All 12 surgical divisions were represented in our study. Presenters (PGY3–8) included 24 surgical residents during the preintervention stage and 19 during the postintervention stage. Learners included 42 surgical faculty, 107 residents, and 48 medical students. We estimate that these learners represent about 80% of all M&MC attendees, although we did not collect data to compare participants and nonparticipants.
During the preintervention stage, presenters discussed 36 cases, which matched the 36 corresponding multiple-choice questions. During the postintervention phase, presenters discussed 30 cases, which matched the 30 corresponding multiple-choice questions.
Presenters’ quantitative evaluations of the intervention were very positive, with all ratings above the scale midpoint (3). In their free-text comments, presenters universally thought the standardized M&MC presentation framework was simple to implement, did not add to their preparation time, and provided a helpful guide for structuring presentations. Learners preferred the standardized format to “no” format.
Presentation quality, that is, presentation quality as rated by the faculty assessors, improved significantly after the intervention (see Table 2). We noted improvements in overall, Background, Assessment, and Recommendations scores. We noted no improvement in Situation or literature review scores.
We collected a total of 1,247 multiple-choice question responses (829 pre- and 418 postintervention) from learners. Of these, faculty completed 203 responses (16%), residents 743 (60%), and medical students 301 (24%). All learners’ performances improved significantly after the intervention (see Table 3). However, senior learners (senior residents and attendings) performed betterthan junior learners (medical students and junior residents) both before and after the intervention (see Table 3). The magnitude of the difference, however, was reduced after the intervention, although it remained significant.
The degree of difficulty for the multiple-choice questions matched before and after the intervention. The mean degree of difficulty measured 3.11 before the intervention (standard deviation = 0.84) and 3.14 after the intervention (standard deviation = 0.77) (P = .75).
To provide high-quality patient care, physicians must continuously engage in the objective review of adverse events. M&MCs can play a pivotal role in this process as well as in ensuring patient safety and quality care if they engage audiences in clinical learning that leads to systematic process change. The surgical M&MC is a forum that provides surgeons an opportunity to review and confidentially discuss medical errors and adverse events. Although many consider the M&MC to be the cornerstone of surgical education, no one has objectively evaluated the explicit learning function of the M&MC or standardized the conference process. Consequently, an ad hoc approach to the M&MC format has evolved without formal evaluation of its educational value.
We conducted our study to fill this gap in the literature. We designed our study to determine whether implementing a standardized format for case presentations during M&MCs improved presenters’ presentation skills and attendees’ learning. We first developed a comprehensive, evidence-based, structured format for M&MC presentations on the basis of the existing surgical literature and the SBAR framework.10,11,15–17 Into this structured format, we incorporated other elements that we identified during our literature review to be critical to an effective M&MC presentation. This new standardized format offered presenters a clear and concise organization for including clinical information, such as the presentation of facts, a summary of the literature, and a discussion of how medical evidence supports best clinical practices. Our postintervention data indicate that the quality of the M&MC presentations improved after we implemented our adapted SBAR framework. Importantly, this format resulted in the clearer delivery of the critical learning point(s) of the presentations.
We noted significant improvements in presentation quality for three sections—Background information, Assessment and root cause analysis, and Recommendations for preventing future complications. The improvement in Recommendations is of particular importance because providing clear recommendations for preventing future errors is critical both in the maturation process of all trainees and to providing safer care to future patients. We did not find improvement in the literature review section, which may reflect our assessors’ bias as faculty surgeons, who generally have a better grasp of the pertinent literature than trainees. This finding suggests that educators should focus on improving discussions of the pertinent evidence base in relation to the adverse event in future efforts to enhance M&MCs.
Educational outcomes also improved significantly after the intervention. We found that knowledge acquisition improved for all learners, including for experienced attending-level surgeons. This improvement was not confounded by the degree of difficulty of the multiple-choice questions, as we ensured that both pre- and postintervention questions had a similar degree of difficulty. Interestingly, we found that the performance of the junior learners after the intervention matched that of the senior learners before the intervention. This finding suggests that our intervention boosted the knowledge of the junior learners to that of their senior peers before the intervention. In other words, the standardized presentation format improved junior learners’ understanding of the presenter’s recommendations. In addition, the most pronounced improvements in presentation quality were in the Assessment and Recommendations sections. We believe that these two sections improved concomitantly because as the presenters’ ability to communicate critical material improved, so, too, did learners’ understanding of the case presentations’ learning points.
Our study has several limitations. First, presenters were aware that others were evaluating their presentations both before and after the intervention. Thus, improvements in their performance may be attributable to a Hawthorne effect. However, presenters’ preintervention performance did not improve over the course of the study, and their improvement was only significant after the intervention, leading us to believe that their knowledge of the evaluations had little effect on their performance. Next, our study was a relatively small, single-institution study, which could have resulted in faculty assessor observer bias. We tried to minimize this bias by training the assessors12 and systematically checking interassessor reliability during the study. Our conclusions will be more generalizable when others test our results on a larger scale, at additional institutions, and with external assessors. Further, although the improvements we found were statistically significant, they were rather small in absolute terms. As statistical significance does not equate to educational significance, further evaluation is needed of the impact of our intervention on the translation of learning to practice. Finally, we measured educational outcomes only at the time of the M&MC presentation. Measuring the long-term effects of our intervention is needed to establish whether presenters retain those skills they learned and learners retain the knowledge they gained.
Our findings from this study also carry implications for the larger academic medicine community. The M&MC represents an authentic, relevant, and contemporary workplace-based learning opportunity that offers the potential for peer learning and reflection on surgical care provision, such that medical errors and omissions can be identified and dealt with in a manner that prevents their reoccurrence and improves patient care. We developed and evaluated an evidence-based, practical, and inexpensive tool that enhances the educational outcomes of such conferences. Our standardized SBAR framework provides a simple, easily implementable set of guidelines for inexperienced trainees on how to give an effective educational presentation to both junior and senior surgeons. This simple intervention can have a positive impact on practice-based education without overstretching hospital budgets, which is especially important today in a time of financial austerity for many institutions. For those programs without access to NBME expertise, the elaborate evaluation process we completed may not be feasible or necessary for implementing our standardized MC&C format. Finally, others can use the tool we developed to assess presentation quality to provide residents with formative feedback, and it can serve as a focus for presentation skills debriefing. Educators then can use the recently validated Objective Structured Assessment of Debriefing tool18 to evaluate these debriefings, linking presentation quality to educational outcomes.
The M&MC provides an opportunity for educators to assess the knowledge, skills, and attitudes of trainees from novices to experts. As George Miller19 pointed out, there is “no single assessment method (that) can provide all the data required for the judgment of anything so complex as the delivery of professional services by a successful physician.” Our findings demonstrate that the M&MC can provide a forum for evaluating what trainees know and how they apply this knowledge to patient care. Although our tool does not directly evaluate residents’ clinical competence or performance, the M&MC itself provides the opportunity for educators to evaluate residents’ application and demonstration of their knowledge and ability to integrate that knowledge into their daily clinical practice. We hope that improvements in M&MC attendees’ discussions and analysis of surgical adverse events will lead to improvements in clinical performance, reductions in surgical complications, and enhancements to patient safety. Further longitudinal research should explore whether a standardized M&MC presentation format can enhance clinical outcomes by preventing the recurrence of adverse events.
The M&MC can play a pivotal role in educating residents and improving patient safety. Our standardization of the M&MC presentation format using the SBAR framework improved the quality of residents’ presentations and attendees’ educational outcomes. We recommend using such a standardized presentation format to enhance the educational value of the M&MC, with the ultimate goal of improving surgeons’ knowledge, skills, and patient care practices.
Acknowledgments: The authors thank John G. Hunter, MD, OHSU chair of surgery; Karen E. Deveney, MD, OHSU vice chair of education and program director of surgery; Karen Kwong, MD, associate program director of surgery; and all of the OHSU surgical residents who participated in this study.
Funding/Support: Drs. Sevdalis and Arora are affiliated with the Imperial Centre for Patient Safety and Service Quality, which is funded by the United Kingdom’s National Institute for Health Research.
Other disclosures: None.
Ethical approval: The OHSU institutional review board approved this study, and the OHSU Department of Surgery sanctioned it.
Previous presentations: The authors presented the findings from this study at the Association for Surgical Education and Association for Program Directors in Surgery, Surgical Education Week, Boston, Massachusetts, March 24, 2011.
1. Vincent C, Moorthy K, Sarker SK, Chang A, Darzi AW. Systems approaches to surgical quality and safety: From concept to measurement. Ann Surg. 2004;239:475–482
2. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377–384
3. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–271
4. Gawande AA, Thomas EJ, Zinner MJ, Brennan TA. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery. 1999;126:66–75
5. Arora S, Sevdalis N, Suliman I, Athanasiou T, Kneebone R, Darzi A. What makes a competent surgeon? Experts’ and trainees’ perceptions of the roles of a surgeon. Am J Surg. 2009;198:726–732
6. Gordon LA Gordon’s Guide to the Surgical Morbidity and Mortality Conference. 1994 Philadelphia, Pa Hanley & Belfus
7. Gordon LA. Can Cedars-Sinai’s “M+M Matrix” save surgical education? Bull Am Coll Surg. 2004;89:16–20
10. Gore DC. National survey of surgical morbidity and mortality conferences. Am J Surg. 2006;191:708–714
11. Harbison SP, Regehr G. Faculty and resident opinions regarding the role of morbidity and mortality conference. Am J Surg. 1999;177:136–139
13. Mitchell EL, Lee DY, Arora S, et al. SBAR M&M: A feasible, reliable, and valid tool to assess the quality of, surgical morbidity and mortality conference presentations. Am J Surg. 2012;203:26–31
14. Kirkpatrick DL, Kirkpatrick JD Evaluating Training Programs: The Four Levels. 2009 San Francisco, Calif Berrett-Koehler Publishers
15. Murayama KM, Derossis AM, DaRosa DA, Sherman HB, Fryer JP. A critical evaluation of the morbidity and mortality conference. Am J Surg. 2002;183:246–250
16. Risucci DA, Sullivan T, DiRusso S, Savino JA. Assessing educational validity of the morbidity and mortality conference: Apilot study. Curr Surg. 2003;60:204–209
17. Prince JM, Vallabhaneni R, Zenati MS, et al. Increased interactive format for morbidity & mortality conference improves educational value and enhances confidence. J Surg Educ. 2007;64:266–272
18. Arora S, Ahmed M, Paige J, et al. Objective structured assessment of debriefing: Bringing science to the art of debriefing in surgery. Ann Surg. 2012;256:982–988
19. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 suppl):S63–S67
Appendix 1 Example of a Multiple-Choice Question and Likert Scale Evaluating the Educational Outcomes of a Surgical Morbidity and Mortality Conference Presentation, Oregon Health & Sciences University, 2009–2011
A 65-year-old male underwent an uncomplicated left carotid endarterectomy for asymptomatic >80% internal carotid stenosis. His postoperative course was unremarkable, and he was discharged to home on postoperative day one. Seven days later, the patient is brought to the emergency department for new-onset mild confusion and abnormal speech. His wife states that the patient has been complaining of severe headaches for several days. In the emergency department, his vital signs are as follows: HR 83, BP 160/92, RR 14, oxygen saturation 93% room air. On physical examination, he has no neurologic deficits, and his surgical wound is clean. A head CT scan reveals no intracranial pathology.
What is the most appropriate next step in the management of this patient?
- A. Duplex ultrasonography of the carotid arteries
- B. Cerebral angiography
- C. Admit patient to ICU for blood pressure control
- D. Admit patient to ICU for tPA administration
- E. Reassure patient and his wife and schedule a clinic appointment within the week
Did you know the answer prior to the morbidity and mortality conference?
© 2013 Association of American Medical Colleges