Journal Logo

ORIGINAL ARTICLES

Multisource Evaluation of Surgeon Behavior Is Associated With Malpractice Claims

Lagoo, Janaka MD; Berry, William R. MD, MPH, MPA; Miller, Kate PhD, MPH; Neal, Brandon J. MPH; Sato, Luke MD; Lillemoe, Keith D. MD; Doherty, Gerard M. MD§; Kasser, James R. MD; Chaikof, Elliot L. MD, PhD||; Gawande, Atul A. MD, MPH; Haynes, Alex B. MD, MPH

Author Information
doi: 10.1097/SLA.0000000000002742

Abstract

As physicians serve as both members and leaders of multispecialty patient-centered health care teams, there is an increasing focus on teamwork and communication in patient care. Multisource feedback (MSF), or 360-degree review, refers to evaluations of a person derived from 2 or more distinct categories of individuals, such as supervisors, peers, and trainees. It has been a mainstay in performance evaluations in many industries for decades and is now being increasingly utilized in medicine.1,2 Three hundred sixty-degree reviews focus on generating a comprehensive perspective of the physician's performance through aggregation and analysis of this diverse feedback and offer a unique view into nontechnical skills (eg, situation awareness, decision making, teamwork and communication, and leadership) of surgeons.1

Previous work suggests that medical malpractice claims are associated with poor teamwork and communication,3–7 which is likely related to behaviors that can be captured by 360-degree review. Generally, a small number of physicians account for a disproportionate share of malpractice claims.8–12 Surgeons who have large numbers of unsolicited patient observations (eg, voluntary patient complaints) in the 2 years before the patient's operation are at an increased risk of surgical and medical complications.13,14 A growing body of research points to a direct link between malpractice rates and poor physician-patient communication.10,15–17 These surgeons may also interact with their peers in the operating room and other perioperative care settings in ways that could negatively affect team performance and contribute to the risk for complications.18–20 Disrespect and rudeness toward other professionals is shown to affect willingness to share information and seek help, which may in turn affect both procedural and diagnostic performance.21–23

The significance of the potential association between surgeon behavior and malpractice is multi-fold. First, malpractice claims may be a marker of behavior in and out of the operating room that can adversely affect patients both through clinical outcomes and through damaging relationships.24 Second, malpractice claims have a financial cost to both hospitals and individual surgeons.25 Third, malpractice claims related to surgeon behavior have the potential to erode an institution's culture of safety – especially in regards to the principles of teamwork and communication.26 Lastly, the impact of the occurrence of malpractice claims on surgeon well-being is significant. Claims are strongly related to burnout (P < 0.0001), depression (P < 0.0001), and recent thoughts of suicide (P < 0.0001) among surgeons.27

This study builds on previous research from our group assessing the perceived value of a 360-degree review sponsored by a malpractice insurance company for a group of 8 diverse hospitals, affiliated with a common university system.28 We seek to use a unique data set that merges direct, multisource, and systematic assessments of surgeon behavior with medical malpractice claims, to analyze the relationship between 360-degree reviews of surgeons and their malpractice history.

METHODS

Setting

In 2005, The Risk Management Foundation of the Harvard Medical Institutions, Inc., the malpractice insurance and patient safety company insuring the Harvard-affiliated hospitals, convened a surgical safety and quality collaborative led by the surgical department heads across multiple institutions. This group has produced and published a number of system-wide improvement initiatives focused on communication, professionalism, and teamwork.29,30 In 2011, the collaborative developed a Code of Excellence (COE; in Appendix, http://links.lww.com/SLA/B396) defining a minimum standard of conduct expected of all affiliated surgeons in 11 domains: service, respect, teamwork, excellence, ethical discipline, personal responsibility to patients, openness, education, humility, health, and conflict of interest.29,30 This COE defines the expected behaviors that are assessed in MSF, applied to all surgeons in the system.

360-Degree Review

From 2012 to 2013, participating hospitals implemented a 360-degree review process using a proprietary web-based system (PULSE 360 Program; Miami, FL). The 360-degree review tool consists of 47 questions (categorized as motivating behaviors, demotivating behaviors, well-being concerns, impact insight, rater familiarity, clinical practice style, and comment questions). Using a Likert scale, raters selected how much each statement applied to the physician: 1 - Not at all, 2 - To a little extent, 3 - To some extent, 4 - To a great extent, and 5 - To a very great extent.

A working group of surgeons revised and expanded the tool to fully capture the themes of performance covered by the COE. Each question was scored on a 5-item Likert scale based on level of agreement and was mapped to a COE theme. The revised tool includes 39 questions that are categorized to reflect the Harvard Surgery Code of Excellence and includes the following 10 domains: educates, excellence, ethical discipline, humility, openness, respect, service teamwork, personal responsibility to patients, and conflict of interest. For the analysis, 16 of the 39 questions on the 360-degree review were completed by less than half of the reviewers and thus, were excluded from analysis. Ultimately, we analyzed responses to 23 questions measuring 7 of the original 11 Harvard Surgery Code of Excellence domains (conflict of interest, ethical discipline, personal responsibility, and health were excluded). This modification of the Pulse 360 tool was validated in a prior study from our group.28

360-Degree Review Data

Three-hundred eighty-five surgeons, from 4 university-affiliated community hospitals and 4 academic medical centers, underwent 360-degree review. Surgeons came from the departments and/or divisions of Cardiac, Thoracic, Vascular, Orthopedic, Plastic, and General Surgery (including oncology, trauma, transplant, colorectal, acute care, critical care, and minimally invasive). Participation by surgeons was mandatory, with the stipulation that identifiable results would be used for no other purpose than physician development. Each surgeon was given the opportunity to suggest 20 to 30 individuals as evaluators, including peers, referring physicians, trainees, nurses, ancillary operating room staff, administrative assistants, or supervisors. The final list of reviewers was determined by the department or division head and the review process occurred over a 2 to 3-month period. All 360-degree review results were anonymous and without distinguishing characteristics to prevent identification of the reviewers. Each department or division head determined how the reviews were distributed to surgeons and whether formal debriefing and/or follow-up coaching was provided.

Malpractice Claims Data

The claims data, which was supplied by The Risk Management Foundation of the Harvard Medical Institutions, Inc., consisted of the number of malpractice claims filed for each physician between January 1, 2000, and December 31, 2015. No further clinical or medicolegal information was included, such as date of claim, result of claim, or specifics of care delivered. “Malpractice claim” is defined here as a written claim or demand for payment filed for the failure, on the part of a health care provider, to furnish health care services or against the services furnished by health care providers.

Merger of 360-Degree Review Results and Malpractice Claims Data

After significant discussions between researchers, surgeon leaders, and the Risk Management Foundation, clearance was granted to merge the malpractice and 360-degree review data. Given the highly sensitive nature of the data, the claims and review data sets were merged on the basis of randomly assigned identifiers that preserved anonymity of subjects. The Institutional Review Board of the Harvard Human Research Protection Program deemed the project exempt from review.

Statistical Analysis

All statistical analyses were performed using SAS software, version 9.4 (SAS Institute, Cary, NC). The Likert scale for negative behaviors was reverse coded for analysis to maintain consistency in interpretation with positive behaviors. Proportions were calculated for categorical variables and the mean and median were calculated for continuous variables. We examined the association between 360-degree review scores and malpractice claims by comparing the surgeons in the bottom decile by mean score versus the remainder of the study population. We examined these relationships when using all raters, as well as when limiting the analysis to peer raters. Odds ratios (ORs) were then calculated using exact logistic regressions to give the odds of having at least 1 lawsuit given being in the bottom 10% for each question, adjusting for years of malpractice coverage during the study period. For primary analysis, we looked at the association based on all reviewers, and for secondary analysis, we looked at the association based on peer reviews alone.

RESULTS

Malpractice data were obtained on the 264 surgeons, for whom it were available (from the original sample of 385 surgeons who underwent the 360-degree review). Out of the 264 reviewed surgeons, 237 (89.8%) came from general surgery (including cardiac, thoracic, vascular, and plastic surgery) and 27 (10.2%) came from orthopedic surgery. There were 8472 ratings by rater type: 4222 ratings by peer physicians (49.8%) and 4250 ratings by clinical and administrative staff/supervisors or managers (50.2%). The range of claims among these 264 surgeons was 0 to 8, with 48.1% of surgeons having at least 1 claim (Fig. 1). Figure 2 shows the mapping of the Code of Excellence onto the 360-degree review and establishes the format for how Figs. 3 to 5 present the 360-degree review and malpractice data. For the 23 questions (measuring 7 domains of performance) analyzed, the range of ratings for each of the review questions is shown in Fig. 3. Over 85% of ratings are 4's and 5's on the 5-item Likert scale. However, there were scores of 1 given on all the questions as well. The overall directionality of the data shows an association between behaviors (as captured by the 360-degree review) and malpractice claims.

FIGURE 1
FIGURE 1:
Number of malpractice claims per surgeon. Majority of surgeons had 0 to 1 claim.
FIGURE 2
FIGURE 2:
Modified Harvard Surgical Code of Excellence. The domains of the Harvard Surgical Code of Excellence are mapped to the Pulse 360-Degree Review items. The questions in bold were answered by over half of all respondents and thus included in our analysis. Figures 3–5 present data on these questions.
FIGURE 3
FIGURE 3:
Distribution of 360-degree review scores by raters. The questions are organized per Code of Excellence (COE) domains, as detailed in Fig. 2. The vast majority of ratings were 4 s and 5 s. However, ratings of 1 s and 2 s are noted for every question.
FIGURE 4
FIGURE 4:
Lawsuits versus mean score in bottom 10% on each question in the 360-degree review (organized by COE domains), peer physician ratings, adjusted for number of years covered. Odds ratios of having had at least 1 lawsuit (0/1) given being in the bottom 10% in mean score on each question. Peer physician raters, minimum 10 raters, 95% CIs adjusted for clustering.
FIGURE 5
FIGURE 5:
Lawsuits versus mean score in bottom 10% on each question in the 360-degree review (organized by COE domains), all raters, adjusted for number of years covered. Odds ratios of having had at least 1 lawsuit (0/1) given being in the bottom 10% in mean score on each question. All rater types, minimum 10 raters, 95% CIs adjusted for clustering.

For our analysis on the association of malpractice claims with peer 360-degree review results, 8 of 14 positive behaviors were significantly associated with not having malpractice claims and 4 of 9 negative behaviors were associated with having malpractice claims (P < 0.05). Table 1 summarizes the highest ORs of having at least 1 malpractice claim given being in the bottom 10% in mean score on each question among peer physician ratings.

TABLE 1
TABLE 1:
Highest Odds Ratios of Having at Least 1 Malpractice Claim Given Being in the Bottom 10% in Mean Score on Each Question Among Peer Physician Ratings

Those in the bottom decile for the negative item “snaps at others“ are associated with an increased likelihood of incurring claims [OR 5.92, confidence interval (CI) 1.77–26.15, P = 0.0017]. Similarly, those in the bottom decile for the positive item “considers suggestions” are associated with an increased likelihood of incurring claims (OR 5.99, CI 1.68–28.12, P = 0.0028). Overall, the association of surgeon behavior and malpractice claims was stronger when assessed with peer ratings (Fig. 4) than those from all raters (Fig. 5), although the direction of association was consistent.

DISCUSSION

Surgeon behavior (as assessed by the 360-degree review, which measures nontechnical skills) is associated with the risk of malpractice claims. This highlights the importance of nontechnical skills such as teamwork and communication in exposure to malpractice risk. Previous research from our group showed the significant value of 360-degree reviews for surgeons.28 Our study now builds on this work by showing that surgeon behaviors (identified by the 360-degree reviews) are associated with an increased risk of malpractice claims. Our work also adds to the growing body of research pointing to a link between malpractice rates and poor physician communication and empathy.17,31,32 To our knowledge, an analysis that merges direct, multisource, and systematic assessments of surgeon behavior with medical malpractice claims has not previously been performed. In addition, in all respects, we show consistent directionality that surgeon behavior is associated with malpractice claims. This reinforces the significant impact of physician behavior on patients, other physicians, and the institution at large.

In our study, there was a difference between peer physician ratings versus the ratings of others (clinical administrative staff, self, and supervisors/managers) in the number of behaviors significantly associated with malpractice claims. We believe that peers are uniquely positioned to observe and analyze the behavior and performance of their colleagues, as they have undergone similar training, understand the unique daily challenges of their profession, interact with similar healthcare staff, and likely have patients with similar health needs. On the contrary, we believe that other clinical or administrative staff likely observe surgeons in more confined settings (ie, operating room nurses only interacting with surgeons in the operating room) and thus are likely limited in the range of behaviors they can observe and provide feedback on. Surgeons interactions with patients as measured indirectly through numbers of voluntary patient complaints seem to be carried into the operating room and other perioperative care settings in ways that may negatively affect team performance and contribute to the risk of complications.13,18,20 We agree with these findings and believe that elements of the 360-degree review can serve as proxies for the patient-physician relationship. For example, the domains of humility and respect from the 360-degree review capture behaviors such as whether one admits when they are wrong and whether one listens effectively, which can be easily translatable to direct physician-patient interactions. More broadly, how one treats his/her colleagues, superiors, and other staff (as captured through the 360-degree review) can serve as a proxy for how one treats his/her patients. Several studies have looked at how best to assess patient safety culture and have found culture surveys to be a particularly powerful tool, which also emphasizes the important role of nontechnical surgical skills in achieving safety culture and high-quality patient care.20 Therefore, our study adds to the body of work that came before it and lends weight to patient safety and risk mitigation efforts that focus on surgeons’ ability to communicate respectfully and effectively in diverse interactions (including but not limited to those with patients and other medical professionals).

Limitations

Although the associations identified by this analysis are compelling, the study has limitations. First, we are assuming that the behaviors measured from the 360-degree reviews reflect behaviors that are relatively consistent over time and therefore contemporaneous with claims. Given that our coverage period is from 2000 to 2015 and the 360-degree reviews were conducted from 2012 to 2013, this seems likely. Second, this is an observational study and thus neither causation nor modifiability of this relationship can be ascertained, nor can the direction of causation; it is possible that the experience of incurring a suit contributes to burnout and poor interpersonal relations. Third, given the highly sensitive nature of both our data sets (360-degree review results and malpractice claims data), de-identification and upholding anonymity was of utmost importance. This limited our ability to include information regarding the nature of the malpractice claims, case complexity, and surgeon demographics (eg, sex, age, years in practice, relative value units) in our analysis. Specifically, the lack of data on surgeon volume and complexity prohibits us from accounting for the potential relationship between these factors and malpractice claims. However, the nature of these relationships is far from clear; while higher volume surgeons see more patients and thus may be more likely to incur claims, lower volume surgeons may have less experience and worse outcomes.13,33 Lastly, our study is limited to general (including subspecialist) and orthopedic surgeons in 4 academic hospitals, which may limit overall generalizability.

Given these limitations, we are clear that we cannot evaluate causation and focus on association alone. Importantly, malpractice claims are imperfectly linked with clinical care quality. Nonetheless, claims may reflect an important breakdown in the expected care process and a poor outcome in some fashion, despite the multiple factors that contribute to the filing of a claim. To our knowledge, this is the first analysis that merges malpractice claims with direct, multisource, and systematic assessments of surgeon behavior.

CONCLUSION

Despite certain limitations, our study shows that surgeon behavior (as assessed by the 360-degree review) is associated with malpractice claims. We do not know how modifiable these behaviors are; however, we can postulate some interventions to potentially improve these behaviors. More frequent 360-degree reviews would ideally support ongoing reflection and behavior improvement among surgical staff and ultimately might lead to the prioritization of certain positive behaviors (as captured by 360-degree reviews) in hiring decisions. Second, a combination of surgical coaching and focused-remediation for those identified as having concerning negative behaviors through 360-degree reviews could serve as a model to improve surgeon behavior and mitigate the risk to patients. Third, modified/shorter 360-degree reviews could potentially increase feasibility and applicability. Fourth, more predictive modeling for those most likely to incur malpractice claims could enable institutions to better target their risk mitigation strategies. Lastly, further understanding of attitudes underlying the overt negative behaviors (captured by 360-degree reviews) is vital to the development of future interventions for surgeons to promote teamwork, leadership, and effective communication.

Although malpractice is a complex issue, our study adds to the growing literature that problematic surgeon behavior is associated with many negative consequences including malpractice claims. Thus, in the future, targeted promotion of positive surgeon behaviors and modification of negative ones could potentially aid in the mitigation of malpractice risk and ultimately in the improvement of the quality of patient care.

Acknowledgments

We would like to thank the Harvard Surgical Chiefs Collaborative and the Physician Development Program/PULSE 360 Program. We would also like to thank additional members of the Ariadne Labs team who contributed to data analysis [Stuart Lipsitz and Zhonghe (Elena) Li] and manuscript development (Ami Karlage).

REFERENCES

1. Donnon T, Al Ansari A, Al Alawi S, et al. The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review. Acad Med 2014; 89:511–516.
2. Meng L, Metro DG, Patel RM. Evaluating professionalism and interpersonal and communication skills: implementing a 360-degree evaluation instrument in an anesthesiology residency program. J Grad Med Educ 2009; 1:216–220.
3. Gogos AJ, Clark RB, Bismark MM, et al. When informed consent goes poorly. Med J Aust 2011; 195:340–344.
4. Gordham CG, Anandalwar SP, Son J, et al. Malpractice in colorectal surgery: a review of 122 medicolegal cases. J Surg Res 2015; 199:351–356.
5. Griffen FD, Turnage RH. Reviews of liability claims against surgeons: what have they revealed? Adv Surgery 2009; 43:199–209.
6. Hultman CS, Halvorson EG, Kaye D, et al. Sometimes you can’t make it on your own: the impact of a professionalism curriculum on the attitudes, knowledge, and behaviors of an academic plastic surgery practice. J Surg Res 2013; 180:8–14.
7. Hultman CS, Gwyther R. Stuck in a moment: an ex-ante analysis of patient complaints in plastic surgery. Ann Plastic Surg 2015; 74 (Suppl 4):S241–S246.
8. Hickson GB, Federspiel CF, Pichert JW, et al. Patient complaints and malpractice risk. JAMA 2002; 287:2951–2957.
9. Moore IN, Pichert JW, Hickson GB, et al. Rethinking peer review: detecting and addressing medical malpractice claims risk. Vanderbilt Law Rev 2006; 59:1175–1206.
10. Moore PJ, Adler N, Robertson PA. Medical malpractice: the effect of doctor-patient relations on medical patient perceptions and malpractice intentions. Western J Med 2000; 173:244–250.
11. Mukherjee K, Pichert JW, Cornett MB, et al. All trauma surgeons are not created equal: asymmetric distribution of malpractice claims risk. J Trauma 2010; 69:549–556.
12. Studdert DM, Bismark MM, Mello MM, et al. Prevalence and characteristics of physicians prone to malpractice claims. N Engl J Med 2016; 374:354–362.
13. Cooper WO, Guillamondegui, Hines J, et al. Use of unsolicited patient observations to identify surgeons with increased risk for postoperative complications. JAMA Surg 2017; 152:522–529.
14. Murff HJ, France DJ, Blackford J, et al. Relationship between patient complaints and surgical complications. Qual Saf Health Care 2006; 15:13–16.
15. Hickson GB, Jenkins AD. Identifying and addressing communication failures as a means of reducing unnecessary malpractice claims. N C Med J 2007; 68:362–364.
16. Hickson GB, Entman SS. Physician practice behavior and litigation risk: evidence and opportunity. Clin Obstet Gynecol 2008; 51:688–699.
17. Nazione S, Pace K. An experimental study of medical error: do apology, empathy, corrective action, and compensation alter intentions and attitudes. J Health Commun 2015; 20:1422–1432.
18. Cochran A, Elder WB. Effects of disruptive surgeon behavior in the operating room. Am J Surg 2015; 209:65–70.
19. Makary MA, Sexton JB, Freischlag JA, et al. Patient safety in surgery. Ann Surg 2006; 243:628–632.
20. Makary MA, Sexton JB, Freischlag JA, et al. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg 2006; 202:746–752.
21. Carayon P, Schoofs Hundt A, Karsh BT, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl 1):i50–i58.
22. Riskin A, Erez A, Foulk TA, et al. The impact of rudeness on medical team performance: a randomized trial. Pediatrics 2015; 136:487–495.
23. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care 2004; 13:330–334.
24. Mazzocco K, Petitti DB, Fong KT, et al. Surgical team behaviors and patient outcomes. Am J Surg 2009; 197:678–685.
25. Orosco RK, Talamini J, Chang DC, et al. Surgical malpractice in the US, 1990-2006. J Am Coll Surg 2012; 215:480–488.
26. Hickson GB, Pichert JW, Webb LE, et al. A complementary approach to promoting professionalism: identifying, measuring, and addressing unprofessional behaviors. Acad Med 2007; 82:1040–1048.
27. Balch CM, Oreskovich MR, Dyrbye LN, et al. Personal consequences of malpractice lawsuits on American surgeons. J Am Coll Surg 2011; 213:657–667.
28. Nurudeen SM, Kwakye G, Berry WR, et al. Can 360 reviews help surgeons? Evaluation of multisource feedback for surgeons in a multi-institutional quality improvement project. J Am Coll Surg 2015; 221:837–844.
29. Arriaga AF, Elbardissi AW, Regenbogen SE, et al. A policy-based intervention for the reduction of communication breakdowns in inpatient surgical care: results from a Harvard surgical collaborative. Ann Surg 2011; 253:849–854.
30. ElBardissi AW, Regenbogen SE, Greenberg CC, et al. Communication practices on 4 Harvard Surgical Services: a surgical safety collaborative. Ann Surg 2009; 250:861–865.
31. Ghalandarpoorattar SM, Kaviani A, Asghari F. Medical error disclosure: the gap between attitude and practice. Postgrad Med J 2012; 88:130–133.
32. Hickson G, Pichert JW. Youngberg B. Identifying and addressing physicians at high risk for medical malpractice claims. Principles of Risk Management and Patient Safety. Sudbury, MA: Jones & Bartlett Publishers Inc; 2012. 347–368.
33. Fan CJ, Pawlik TM, Daniels T, et al. Association of safety culture with surgical site infection outcomes. J Am Coll Surg 2016; 222:122–128.
Keywords:

malpractice claims; multisource evaluation; surgeon behavior

Supplemental Digital Content

Copyright © 2018 The Author(s). Published by Wolters Kluwer Health, Inc.