Secondary Logo

Share this article on:

The Effect of Level I Evidence on Surgical Decision Making in the United States Versus Canada

Jawa, Andrew, MD; Pittman, Jason L., MD, PhD; Carducci, Michael P., BA; Koenig, Scott, MD; Bhandari, Mohit, MD, FRCSC, PhD; Tornetta, Paul III, MD

JAAOS Global Research & Reviews: December 2018 - Volume 2 - Issue 12 - p e056
doi: 10.5435/JAAOSGlobal-D-18-00056
Research Article

Background: In this study, we examined the difference that randomized trials favoring either surgery or nonsurgical treatment had on the surgical indications of American versus Canadian surgeons.

Methods: One randomized trial favoring surgical management of clavicle fractures and another one favoring nonsurgical management of Achilles tendon ruptures were used. American and Canadian orthopaedic surgeons were surveyed regarding their surgical indications for these injuries.

Results: More than 2000 US and 200 Canadian responses were received. For clavicles, 57% of US respondents indicated that the trial changed their practice, with 64% operating on more fractures, compared with Canadians at 78% and 68%, respectively. For Achilles, 37% of US respondents indicated that the trial changed their practice, with 29% operating on fewer ruptures, compared with Canadians at 72% and 67%, respectively.

Conclusion: American surgeons seem more willing to alter their practice to “evidence-based” indications for a trial that favors surgery rather than one that does not.

From Department of Orthopedic Surgery, New England Baptist Hospital, Boston, MA (Dr. Jawa, Carducci, Dr. Bhandari); Department of Orthopedic Surgery, the University of Alabama at Birmingham Hospital, Birmingham, AL (Dr. Pittman); Boston Sports and Shoulder Center, Waltham, MA (Dr. Jawa, Mr. Carducci); North Shore University Hospital, Manhasset, NY (Dr. Koenig); the Division of Orthopedic Surgery, McMaster University, Hamilton, ON, Canada (Dr. Bhandari); and the Department of Orthopedic Surgery, Boston University Medical Center, Boston, MA (Dr. Tornetta).

Correspondence to Dr. Jawa: AndrewJawa@gmail.com

Dr. Jawa or an immediate family member is a member of a speakers' bureau or has made paid presentations on behalf of and serves as a paid consultant to DJO Global. Dr. Pittman or an immediate family member serves as a paid consultant to DePuy. Dr. Bhandari or an immediate family member serves as a paid consultant to Amgen Co, Bioventus, CONMED Linvatec, DePuy, DJ Orthopedics, Eli Lily, Ferring Pharmaceuticals, Merck, Moximed, Sanofi-Aventis, Smith & Nephews, Stryker, and Zimmer Biomet and has received research or institutional support from Bioventus and Eli Lily. Dr. Tornetta or an immediate family member has received royalties from Smith & Nephew and has stock or stock options held in Exploramed. Neither of the following authors nor any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Mr. Carducci and Dr. Koenig.

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Interpretation and application of clinical evidence varies between surgeons. Evidence-based medicine calls for physicians to reconcile their practices with reported data in each clinical situation and requires that physicians evaluate the quality and relevance of different studies.1 To assist physicians, orthopaedic journals have adopted a five-level scale, evaluating the quality of evidence presented in each study. Level I evidence, the highest grade, is defined as a high-quality randomized trial with sufficient power and follow-up to reach statistically valid conclusions. Less than 14% of published literature is made up by randomized trials, making available level I evidence an important guideline in clinical decision making. However, despite the classifications of evidence, surgeons must also consider the study population, setting, and applicability to the physician's own practice.

Decisions regarding surgery are highly important, but there is often a lack of clear clinical data to support surgical indications. The influence of clear, high-quality evidence available to physicians is therefore important but currently understudied. This study aimed to examine physicians' changes in practice in response to clinical evidence supporting either surgical intervention or nonsurgical treatment by presenting them with high-quality randomized clinical trials. We hypothesized that physicians would more readily incorporate data favoring surgery into their clinical practice. Furthermore, we hypothesized that both American and Canadian physicians would have the same reactions to clinical evidence when making treatment decisions involving surgery.

Back to Top | Article Outline

Methods

We selected two multicenter randomized controlled trials (RCT), one favoring surgery and another one favoring nonsurgical treatment, published in the same journal at least 2 years before this study.2 , 3 We evaluated them using the Jadad score and CONSORT reporting criteria.4 - 6 Both trials scored a three out of five possible points on the Jadad scale, where ≥3 points qualifies as a high-quality study.6 Each study also met 28 of the 37 CONSORT criteria.5 The trials were thus both of a similarly high quality and appropriate for consideration by orthopaedic surgeons.

The first study, published in 2007 by the Canadian Orthopaedic Trauma Society, is a level I randomized trial comparing outcomes of nonsurgical treatment with outcomes of open reduction with internal fixation of 100% displaced midshaft clavicle fractures.2 The study favors surgical intervention. The second study, published by Willits et al3 in 2010, is a multicenter randomized trial comparing outcomes of surgical repairs of Achilles tendon ruptures with outcomes of nonsurgical management of the same injury. The study qualifies as level I evidence favoring nonsurgical treatment.

A questionnaire was designed to evaluate surgeons' decisions after reading level I clinical evidence. It included a five-scenario survey constructed for each RCT that described patients who met the inclusion criteria but did not meet the exclusion criteria for each treatment. The survey regarding clavicle fracture patients also included a radiograph corresponding to the described injury (Figure 1). Responding physicians then recommend surgical or nonsurgical treatment for each patient scenario (Table 1). The questionnaire also asked five demographic questions related to the surgeon's practice, whether the participant was familiar with the RCT being presented, and whether the trial affected their practice.

Figure 1

Figure 1

Table 1

Table 1

We surveyed 19,574 orthopaedic surgeons in the United States (18,843) and Canada (731). Each surgeon was sent a survey corresponding to only one of the clinical trials, ensuring that physicians were responding solely to the scenarios presented. Receiving only one study prevented surgeons from deducing the intent of the study, improving the integrity of responses. Surveys were sent to members of the American Academy of Orthopaedic Surgeons and the Canadian Orthopaedic Association. Each surgeon was randomly assigned a study, and the results from the surveys were collected via SurveyMonkey (www.surveymonkey.com). After the survey was sent out, two reminder e-mails were sent at 2-week intervals for 6 weeks or until the survey was completed.

Back to Top | Article Outline

Results

A total of 2,674 surveys were returned, with 1,546 responding to the clavicle fracture and 1,128 to the Achilles rupture. The breakdown by country is shown in Table 2.

Table 2

Table 2

Back to Top | Article Outline

Clavicle Fracture Management

The responses to the five scenarios are shown in Table 3. Surgery was favored by most of both American and Canadian surgeons in the same three scenarios. However, surgery was not favored in two scenarios, one involving the oldest patient (65 years of age) and the other involving an obese patient (body mass index of 35).

Table 3

Table 3

Seventy-one percent of American surgeons and 90% of Canadian surgeons were familiar with the trial. Among American surgeons, 54% indicated that the evidence changed their practice and 64% indicated that they manage more clavicle fractures surgically than they did 5 years ago. Conversely, among Canadian surgeons, 78% indicated that the evidence changed their practice and 68% indicated that they recommend surgical intervention for clavicle fractures more often than they did 5 years ago.

The practice settings of American respondents were 16% academic, 67% nonacademic, and 17% hybrid, whereas those of Canadian respondents were 47% academic, 29% nonacademic, and 24% hybrid (Table 4).

Table 4

Table 4

Back to Top | Article Outline

Achilles Tendon Rupture Management

The responses to the five scenarios for the Achilles trial are shown in Table 5. Surgery was favored by most American surgeons in four of five scenarios but favored by Canadian surgeons in only one of five scenarios. American surgeons favored nonsurgical treatment in the scenario involving the oldest patient (65 years of age), and Canadian respondents favored surgery only in the scenario involving the youngest and most active patient (20-year-old collegiate athlete).

Table 5

Table 5

Seventy-seven percent of American surgeons and 87% of Canadian surgeons were familiar with the RCT. Thirty-seven percent of American respondents indicated that the evidence changed their practice, with 29% managing more Achilles ruptures nonsurgically than they did 5 years ago. Conversely, 72% of Canadian physicians altered their practice in response to the study, with 67% managing fewer Achilles ruptures surgically.

The practice settings of American respondents were 17% academic, 67% nonacademic, and 16% hybrid, whereas those of Canadian respondents were 37% academic, 38% nonacademic, and 25% hybrid (Table 4).

Back to Top | Article Outline

Discussion

Both studies presented to American and Canadian physicians were level I clinical evidence and should have similar respective effect on clinical decision making. However, American physicians seemed more willing to adjust their practice for evidence favoring surgical intervention, whereas Canadian physicians were willing to change their practice for evidence favoring both surgical and nonsurgical treatment. Several potential explanations are available for this discrepancy. Historically, wait times for procedures in Canada are longer or assumed to be longer than those in the United States.7 Given the acute nature of clavicle fractures and Achilles ruptures, a shorter delay in surgery may make surgical intervention a more attractive option for American physicians. Although there is notable, often successful, effort in Canada to reduce wait times for surgery and consultation,7 physicians may still tend toward nonsurgical treatment when possible based on established practice.

Furthermore, fee reimbursement in the United States favors surgery over conservative treatment. American physicians receive reimbursement based on relative value units of procedures performed.8 For each unit, American physicians receive additional reimbursement from one of numerous possible private or public insurance providers.8 In Canada, the single-payer provincial system allows for greater bargaining power with physicians to negotiate lower fees and less profitable surgical reimbursement.9 As a result, Medicare and other American insurers have historically reimbursed physicians for surgical procedures at markedly higher rates than the Canadian health system.10 Welch et al10 showed that Medicare reimburses nearly three times more than the Canadian average for orthopaedic surgery, whereas fees for office visits are nearly identical for both state programs. Thus, American physicians may favor surgical intervention in more cases than their Canadian counterparts partially because of financial incentives.

Familiarity with new research may also play a role in physician decision making. Physicians are more likely to review emerging literature when practicing in an academic setting. In this study, Canadian respondents were more frequently from an academic setting than American respondents, and unsurprisingly, they were more likely to have heard of each RCT (Table 4). Here, increased exposure to new clinical evidence appears to correlate with greater willingness to translate that evidence into clinical practice. Additionally, both RCTs in this study were performed by Canadian research groups.2 , 3 Although both were published in international journals based in the United States, the study site location may play a role in physicians adopting clinical evidence. Physicians may be more familiar with the work of their direct colleagues or more trusting of studies performed within their own healthcare system.

This study is partially limited by the difference in acute condition described by each of the example studies. Achilles ruptures and clavicle fractures, though similar in their acute nature, do not necessarily demand the same management plans. However, because of the limited number of high-quality RCTs available, comparisons between the two still have clinical value. Furthermore, we would expect each individual study to weigh equally on decision making regardless of surgeon location. Additionally, we did not inquire about surgeons' subspecialties. Although we expect all respondents are appropriately trained, physicians specializing in foot and ankle or upper extremity care are likely more familiar with emerging research.

Integrating new evidence into surgical decision making helps improve medical practice and advance the standard of care. Although physicians should account for the entire body of available evidence when evaluating surgical indications, we hoped to model responsiveness to emerging literature through two high-quality RCTs. Ideally, both American and Canadian physicians would alter their practice based on high-quality clinical evidence, regardless of the nature of treatment supported by the study.

Back to Top | Article Outline

References

1. Okike K, Kocher MS: Evidence-based orthopaedics: Levels of evidence and guidelines in orthopaedic surgery, In: Flynn JM, ed: Orthopaedic Knowledge Update. American Academy of Orthopaedic Surgeons, 2011, pp 157–165.
2. Canadian Orthopaedic Trauma Society: Nonoperative treatment compared with plate fixation of displaced midshaft clavicular fractures: A multicenter, randomized clinical trial. J Bone Joint Surg Am 2007;89:1–10.
3. Willits K, Amendola A, Bryant D, et al: Operative versus nonoperative treatment of acute Achilles tendon ruptures: A multicenter randomized trial using accelerated functional rehabilitation. J Bone Joint Surg Am 2010;92:2767–2775.
4. Jadad AR, Moore RA, Carroll D, et al: Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 1996;17:1–12.
5. Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. Ann Intern Med 2010;152:726–732.
6. McCormick F, Cvetanovich GL, Kim JM, et al: An assessment of the quality of rotator cuff randomized controlled trials: Utilizing the Jadad score and CONSORT criteria. J Shoulder Elbow Surg 2013;22:1180–1185.
7. Ross JS, Detsky AS: Choice? Making health care decisions in the United States and Canada. JAMA 2009;302:1803–1804.
8. Fader AN, Xu T, Dunkin BJ, Makary MA: A perverse quality incentive in surgery: Implications of reimbursing surgeons less for doing laparoscopic surgery. Surg Endosc 2016;30:4665–4667.
9. Katz SJ, Zuckerman S, Welch WP: Comparing physician fee schedules in Canada and the United States. Health Care Financ Rev 1992;14:141–149.
10. Welch WP, Katz SJ, Zuckerman S: Physician fee levels: Medicare versus Canada. Health Care Financ Rev 1993;14:41–54.
Copyright © 2018 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the American Academy of Orthopaedic Surgeons