Conflict of Interest in Orthopaedic Research: An Association Between Findings and Funding in Scientific Presentations

Okike, Kanu BA; Kocher, Mininder S. MD, MPH; Mehlman, Charles T. DO, MPH; Bhandari, Mohit MD, MSc

Journal of Bone & Joint Surgery - American Volume:
doi: 10.2106/JBJS.F.00994
Scientific Articles
Abstract

Background: Financial conflict of interest has been associated with an increased likelihood that authors will report positive study outcomes. The purpose of this study was to investigate the association between types of declared conflict of interest and reported study outcomes in orthopaedic research.

Methods: The abstracts of all podium presentations given at the 2001 and 2002 Annual Meetings of the American Academy of Orthopaedic Surgeons were analyzed by three orthopaedic surgeons with advanced training in clinical epidemiology. The findings reported in each abstract were graded as positive, negative, neutral, or not applicable. Self-reported conflict of interest was recorded and classified.

Results: Conflicts of interest were reported in 40.8% (212) of 519 abstracts. The interobserver reliability of the grading of the study findings was acceptable (intraclass correlation coefficient, 0.725). Rates of conflict of interest related to royalties, stock options, or consultant or employee status varied significantly by subspecialty field (p < 0.001). The overall rate of positive study findings was 84.0% (436 of the 519 abstracts). Positive findings were more common in studies authored by individuals with a conflict of interest related to royalties (98.4% [sixty of sixty-one] compared with 88.0% [381 of 433] for studies authored by individuals without a conflict of interest related to royalties; relative risk = 1.1 [95% confidence interval = 1.0 to 1.1]; p = 0.02), in studies authored by individuals with a conflict of interest related to stock options (100.0% [twenty-nine of twenty-nine] compared with 84.7% [394 of 465]; relative risk = 1.2 [95% confidence interval = 1.0 to 1.3]; p = 0.04), and in studies authored by individuals with a conflict of interest related to consultant or employee status (97.8% [ninety-one of ninety-three] compared with 89.0% [357 of 401]; relative risk = 1.1 [95% confidence interval = 1.0 to 1.2]; p = 0.01). Positive findings were not more common in studies authored by individuals with a conflict of interest related to research or institutional funding (93.5% [143 of 153] compared with 91.8% [313 of 341]; relative risk = 1.0 [95% confidence interval = 0.95 to 1.5]; p = 0.65). In the multivariate analysis, the factors that remained significant predictors of positive outcomes were royalties (p = 0.002) and consultant or employee status (p = 0.038).

Conclusions: Self-reported conflicts of interest are common in orthopaedic research, particularly in the subspecialty fields of adult reconstruction of the knee, adult reconstruction of the hip, and spine. Presentations authored by individuals with a conflict of interest related to royalties, stock options, or consulting or employee status were significantly more likely to describe positive findings. While there may be distinct benefits associated with industry support of orthopaedic research, safeguards must be established to maintain public trust in the medical research establishment.

Author Information

1 Department of Orthopaedic Surgery, Children's Hospital, Harvard Medical School, 300 Longwood Avenue, Boston, MA 02115. E-mail address for K. Okike: kanu_okike@hms.harvard.edu

2 Department of Pediatric Orthopaedic Surgery, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, 3333 Burnet Avenue, Cincinnati, OH 45229

3 Department of Orthopaedic Surgery, Hamilton General Hospital, 7 North, Suite 727, Barton Street East, Hamilton, ON L8L 2X2, Canada

Article Outline

Medical conflict of interest is defined as “a set of conditions in which professional judgment concerning a primary interest (such as a patient's welfare or the validity of research) tends to be unduly influenced by a secondary interest (such as financial gain).”1 Concerns about medical conflict of interest relate to its potential influence on patient care, clinical practice, and medical research. The concern about financial conflict of interest in medical research is well founded: one-quarter of all biomedical researchers receive funding from industry, and industry provides 70% of the money for drug trials in the United States2,3.

The implications of the intimate involvement of industry in medical research are many. Most worrisome are recent studies demonstrating a significant association between industry funding and the publication of positive findings (p = 0.001, p < 0.05, p < 0.001, and p = 0.014)4-7. A recent meta-analysis by Bekelman et al. indicated that industry sponsorship rendered a researcher more likely to arrive at pro-industry conclusions and to experience restrictions on data sharing or the publication of results2.

Industry funding is prevalent in orthopaedic research. At the 2002 American Academy of Orthopaedic Surgeons (AAOS) Annual Meeting, one-third of all presenters reported a financial conflict of interest8. In a recent study of conflict of interest in orthopaedic research, Leopold et al. found that commercial funding was significantly associated with publication of a positive outcome (p = 0.025)9. However, that study did not distinguish between the different types of financial conflict of interest (for example, research and institutional support compared with royalties, stock options, and employee or consultant status). The purpose of the current study was to investigate the association between different types of financial conflict of interest and study outcomes in presentations of orthopaedic research.

Back to Top | Article Outline

Materials and Methods

Eligibility Criteria

All (608) podium presentations given at the 2001 and 2002 Annual Meetings of AAOS, including clinical as well as basic-science presentations, were reviewed. Since the system used to classify study findings applies only to studies in which an item is evaluated (as described below), all (eighty-nine) studies not involving an evaluation were excluded. Studies to be excluded were identified by means of hand-searching of presentation abstracts. The 519 studies that did involve an evaluation of some kind were independently reviewed by three orthopaedic surgeons with advanced training in clinical epidemiology from three different institutions across North America.

Back to Top | Article Outline
Conflict of Interest and Study Findings

The orthopaedic subspecialty field of each paper was recorded as adult hip reconstruction, adult knee reconstruction, basic science, foot and ankle, hand and wrist, pediatrics, practice management/nonclinical, rehabilitation medicine, shoulder and elbow, sports medicine and arthroscopy, spine, musculoskeletal tumor and metabolic disease, or trauma.

Conflict of interest was self-reported and was coded, according to AAOS criteria, as research or institutional support (code A), miscellaneous funding (code B), royalties (code C), stock options (code D), or consultant/employee (code E).

Reviewers classified the findings of each study as positive, negative, neutral, or not applicable. Studies that favored the experimental item over the existing standard of care, or otherwise arrived at a favorable conclusion regarding the item being evaluated, were graded as having positive findings. Conversely, studies that led to an unfavorable conclusion regarding the item being evaluated or favored the existing standard of care over the experimental item were graded as having negative findings. Studies that showed the experimental item to be as good as the existing standard of care were graded as neutral. All other studies, including those in which no value judgment was made, were categorized as not applicable. Discrepancies in the grading of a study outcome were resolved by consensus.

Back to Top | Article Outline
Data Analysis

Reviewer agreement in the grading of study outcomes was assessed with reliability analysis with use of the intraclass correlation coefficient (2,1)10. Comparisons of rates of conflict of interest according to subspecialty and comparisons of rates of positive findings according to the type of conflict of interest were done with use of the Fisher exact test. Relative risks and associated 95% confidence intervals were calculated for these comparisons. Multivariate analysis was performed with use of a binary logistic regression model with backward stepwise variable selection. Statistical analysis was performed with SPSS (version 14.0; SPSS, Chicago, Illinois) and SAS (version 9; SAS, Cary, North Carolina) software.

Back to Top | Article Outline

Results

Study Characteristics

Of 608 podium presentations given at the 2001 and 2002 AAOS Annual Meetings, 519 met the inclusion criteria. The most common subspecialty fields were adult hip reconstruction (16.4%; eighty-five of the 519 presentations), adult knee reconstruction (12.5%; sixty-five), shoulder and elbow (12.1%; sixty-three), and trauma (10.2%; fifty-three) (Table I). The levels of evidence were generally low, with case series (Level IV) accounting for 58.6% (304) of the 519 studies, and randomized controlled trials (Level I) accounting for only 8.1% (forty-two) (Table I).

Back to Top | Article Outline
Conflicts of Interest

The authors self-reported financial conflict of interest as research/institutional support (code A) in 31.0% (161) of the 519 abstracts, as royalties (code C) in 12.3% (sixty-four), as stock options (code D) in 4.8% (twenty-five), and as consultant/employee status (code E) in 18.9% (ninety-eight). Some authors (9.8%; fifty-one) reported miscellaneous conflicts (code B). In all, financial conflicts of interest were reported in 40.8% (212) of the 519 papers presented (Table II).

The rates of conflicts of interest related to royalties, stock options, or consultant/employee status varied significantly by subspecialty field (p < 0.001) (Table III). Such conflicts of interest were most common in the subspecialty fields of adult knee reconstruction (41.5%; twenty-seven of sixty-five), adult hip reconstruction (41.2%; thirty-five of eighty-five), and spine (40.0%; twenty of fifty). Such conflicts of interest were least common in the subspecialty fields of trauma (9.4%; five of fifty-three), foot and ankle (11.1%; two of eighteen), and pediatric orthopaedics (11.1%; four of thirty-six). Overall rates of conflict of interest (all types) also varied significantly by subspecialty field (p < 0.01) (Table III).

Back to Top | Article Outline
Study Findings

The interobserver reliability for the classification of study findings was acceptable (intraclass correlation coefficient, 0.725). The overall rate of positive findings was 84.0% (436 of 519), whereas the rates of negative and neutral findings were 6.4% (thirty-three of 519) and 4.4% (twenty-three of 519), respectively.

Back to Top | Article Outline
Relationship Between Study Findings and Conflicts of Interest

Positive findings were more common in studies authored by individuals with a conflict of interest related to royalties (98.4% [sixty of sixty-one] compared with 88.0% [381 of 433]; relative risk = 1.1 [95% confidence interval = 1.0 to 1.1]; p = 0.02), in studies authored by individuals with a conflict of interest related to stock options (100.0% [twenty-nine of twenty-nine] compared with 84.7% [394 of 465]; relative risk = 1.2 [95% confidence interval = 1.0 to 1.3]; p = 0.04), and in studies authored by individuals with a conflict of interest related to consultant or employee status (97.8% [ninety-one of ninety-three] compared with 89.0% [357 of 401]; relative risk = 1.1 [95% confidence interval = 1.0 to 1.2]; p = 0.01). Positive findings were not more common in studies authored by individuals with a conflict of interest related to research or institutional funding (93.5% [143 of 153] compared with 91.8% [313 of 341]; relative risk = 1.0 [95% confidence interval = 0.95 to 1.5]; p = 0.65) (Table IV).

Back to Top | Article Outline
Multivariate Analysis

The final multivariate logistic regression model included a variable for subspecialty as well as five variables related to the type of conflict of interest (research/institutional support, miscellaneous funding, royalties, stock options, and consultant/employee status). In the multivariate analysis, two factors remained significant predictors of positive study outcomes: royalties (relative risk = 1.15 [95% confidence interval = 1.11 to 1.17]; p = 0.002) and consultant/employee status (relative risk = 1.15 [95% confidence interval = 1.02 to 1.17]; p = 0.038). Subspecialty, research/institutional support, miscellaneous funding, and stock options were not found to be significant predictors of study outcome (Table V).

Back to Top | Article Outline

Discussion

Our observational study of podium presentations given at the 2001 and 2002 Annual Meetings of the AAOS demonstrated that self-reported conflicts of interest are common in orthopaedic research, particularly in the subspecialty fields of adult reconstruction of the knee, adult reconstruction of the hip, and spine. In the multivariate analysis, royalties and consultant/employee status were found to be significantly associated with positive study findings.

In contrast to previous investigators who examined the association between study outcomes and generic conflict of interest, we differentiated between different types of financial conflict of interest using the AAOS classification scheme. This resulted in a more detailed examination of financial conflict of interest of authors of presentations of orthopaedic research. Another strength of our study was the large number of abstracts that were analyzed. The major limitation of this study is that conflict of interest was self-reported. In particular, it is likely that conflicts may have been (intentionally or unintentionally) underreported and that the type of conflict may have been identified inaccurately in some cases. The high prevalence of positive results in the studies (84%) is also a limitation and may have been due in part to a tendency for studies with positive results to be preferentially selected for presentation. This trend appears analogous to the phenomenon of publication bias seen in the editorial review process, whereby the strength or direction of a study's findings influences whether or not it will be published11. Finally, the modest nature of the observed associations (relative risks on the order of 1.1 to 1.2) may be considered a limitation of our study. These modest values may be due, at least in part, to the fact that we compared rates of positive findings, which were quite common in the study sample. Had we chosen to compare rates of non-positive findings, the relative risks would have been on the order of 0.12 to 0.17 (corresponding to a decrease of 83% to 88%).

Other studies have identified a relationship between conflict of interest and the reporting of positive research findings. Kjaergard and Als-Nielsen7 reviewed all randomized clinical trials published in the British Medical Journal between 1997 and 2001 and found that authors' conclusions were significantly more positive in trials funded by for-profit companies than they were in trials by investigators without competing interests or in trials funded by both for-profit and non-profit organizations (p = 0.014). Friedman and Richter6 reviewed all original manuscripts published in the Journal of the American Medical Association and the New England Journal of Medicine during 2001 and found a “strong association” between investigator conflict of interest and positive findings. When the relationship between conflict of interest and negative findings was examined, the strength of the association increased, leading the authors to conclude that “the odds are extremely small that negative results would be published by authors with conflict of interest.” In addition, a systematic review carried out by Bekelman et al.2 demonstrated that industry funding of research was associated with positive outcomes, limitations on data-sharing, and restrictions on the publication of results. One of us (M.B.) and colleagues5 examined 332 randomized clinical trials (including 37% for which industry support had been declared) and found a significant association between industry funding and positive outcomes (p < 0.05). On the other hand, Clifford et al.12 found no association between the funding source and the study outcome after reviewing 100 randomized clinical trials of pharmaceutical products in five leading medical journals.

In a study of the orthopaedic literature, Leopold et al.9 reviewed all articles published in The Journal of Bone and Joint Surgery (American volume), the Journal of Arthroplasty, and the American Journal of Sports Medicine between 1999 and 2000 and found that 78.9% of commercially funded studies, as opposed to 63.3% of non-industry-funded studies, demonstrated a positive outcome (p = 0.025). They also found that neither the country of origin nor the involvement of a statistician or epidemiologist was an important predictor of study outcome. The overall methodological quality of the studies included in their review was low, with 21% being prospective, 10.5% stating a hypothesis, and only 3.5% involving a randomized design.

Leopold et al.9 made the appropriate epidemiological point that the observed association between industry funding and positive outcomes does not necessarily imply causation. A commonly offered explanation for the association between industry funding and positive findings is that for-profit companies preferentially fund studies that are likely to succeed13 and are more likely to suppress publication of negative studies14. In the current study, however, royalties, stock options, and consultant/employee status were associated with an increased likelihood of positive outcomes, whereas the receipt of research or institutional support from industry was not. If well-targeted industry funding and industrial suppression of negative results were the only factors driving the observed association between industry funding and positive outcomes, one would expect this association to hold for all types of industry funding and not just for incentives directed at individual investigators. Although it is disturbing, one must consider the possibility that some investigators may be swayed by the financial benefits offered by industry. In the clinical arena, it has been shown that physicians' prescribing patterns are indeed affected by industry incentives, in spite of the fact that many physicians do not believe this will happen15.

There may be distinct benefits associated with industry support of orthopaedic research. Industry funding promotes technological advancement and in many cases allows research that would otherwise not be possible16. However, industry funding of clinical research is also associated with a number of risks. In addition to jeopardizing the safety of human research subjects, academic-industrial relationships may pose threats to the integrity of the research process by restricting communication within the scientific community and causing some researchers to (knowingly or unknowingly) bias their findings in favor of the funding sponsor17. Even if industrial support does not actually influence the way that an investigator evaluates an implant or device, public perception that such research is biased threatens to erode trust in the scientific community.

To date, most attempts to address conflict of interest in clinical research have centered on disclosure. The AAOS, for example, began requiring all presenting authors to self-disclose their conflicts of interest at the 1985 Annual Meeting8. Now, more than two decades later, additional steps may be required to ensure the integrity of orthopaedic research and maintain public trust in the medical research establishment. Although disclosure is an important first step, it has not been shown to eliminate the negative effects of conflict of interest when used in isolation18. Additional safeguards regarding the presentation (and publication) of research could include penalties for failure to accurately disclose all conflicts of interest, acceptance of research support only when it does not impose restrictions, mandatory standardized research contracts with industry that prevent the suppression of negative results, and independent statistical review. Other potential options include the pooling of funding derived from industry sources and the restriction of presentation and publication by individuals with a direct financial interest such as royalties, stock options, or consultant or employee status. The elimination of financial conflicts of interest in medical research also remains a laudable goal18. ▪

Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. One or more of the authors or a member of his or her immediate family received, in any one year, payments or other benefits in excess of $10,000 or a commitment or agreement to provide such benefits from commercial entities (DePuy, Smith and Nephew, Stryker, and Zimmer). Also, commercial entities (DePuy, Medtronic, Stryker, and Synthes) paid or directed in any one year, or agreed to pay or direct, benefits in excess of $10,000 to a research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

Investigation performed at the Department of Orthopaedic Surgery, Children's Hospital, Harvard Medical School, Boston, Massachusetts

1. . Understanding financial conflicts of interest. N Engl J Med. 1993;329: 573-6.
2. , Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289: 454-65.
3. . Uneasy alliance—clinical investigators and the pharmaceutical industry. N Engl J Med. 2000;342: 1539-44.
4. , Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA. 2003;290: 921-8.
5. M, Busse JW, Jackowski D, Montori VM, Schunemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004;170: 477-80.
6. , Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med. 2004;19: 51-6.
7. , Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325: 249.
8. , Prasarn M, Kubiak EN, Koval KJ. Conflict of interest in orthopaedic research. J Bone Joint Surg Am. 2004;86: 423-8.
9. , Warme WJ, Fritz Braunlich E, Shott S. Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res. 2003;415: 293-301.
10. , Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86: 420-8.
11. . The existence of publication bias and risk factors for its occurrence. JAMA. 1990;263: 1385-9.
12. , Barrowman NJ, Moher D. Funding source, trial outcome and reporting quality: are they related? Results of a pilot study. BMC Health Serv Res. 2002;2: 18.
13. . Randomized clinical trials: what gets published, and when? CMAJ. 2004;170: 481-3.
14. The “file drawer” phenomenon: suppressing clinical evidence. CMAJ. 2004;170: 437, 439.
15. , Wateska L. The effects of pharmaceutical firm enticements on physician prescribing patterns. There's no such thing as a free lunch. Chest. 1992;102: 270-3.
16. , Buckwalter JA, Talman CL, Happe DG. Industrial support of orthopaedic research in the academic setting. Clin Orthop Relat Res. 2003;412: 45-53.
17. . Academic-industrial relationships in the life sciences. N Engl J Med. 2003;349: 2452-9.
18. , Rothman DJ, Blank L, Blumenthal D, Chimonas SC, Cohen JJ, Goldman J, Kassirer JP, Kimball H, Naughton J, Smelser N. Health industry practices that create conflicts of interest: a policy proposal for academic medical centers. JAMA. 2006;295: 429-33.
19. , Yu KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA. 1998;280: 1690-1.
Copyright 2007 by The Journal of Bone and Joint Surgery, Incorporated