Innovation is required to develop novel interventions in order to improve quality of care. While surgeons are highly innovative, the uncontrolled introduction of unproven innovations may have adverse consequences for patients. The purpose of this article is to propose an innovation cycle whereby surgical innovations can be safely introduced into clinical practice, with a focus on the specific roles of opinion leaders, journals, specialty societies, and certification bodies.
Standardization in most aspects of production leads to more consistent and better outcomes1. While clinical medicine is more complex than industrial production, evidence-based medicine is intended to standardize and reduce variation in clinical care with the express purpose of improving quality and possibly reducing overall costs. The rationale for evidence-based orthopaedics is that best evidence should guide physicians and patients in choosing among treatment options. The consistent and widespread adoption of best practice should lead to improved clinical outcomes by decreasing inappropriate care, addressing unmet needs, and ensuring that patients receive the best treatment. Thus, evidence-based care seeks to standardize clinical practice.
Innovation, in contrast, requires experimentation and variation in practice. Innovation has the potential to create dramatic new interventions that lead to large improvements in patient outcomes. Innovation may include a new surgical technique, a new implant, a new drug, or a different service delivery model. Thus, these two strategies are potentially at odds since evidence-based practice seeks standardization and reduced variation in practice, while innovation requires variation in practice. However, both aim to improve the quality of care.
The orthopaedic literature has many examples of innovations in surgical techniques, such as total meniscectomy of the knee, a procedure that was widely adopted but proved not only to be ineffective but also harmful to patients in a few cases2-5. We propose a concept that we have termed the “innovation cycle,” which would control the introduction of innovation into clinical practice, thereby allowing and encouraging innovation; this cycle would use a graduated approach in order to minimize the potential harm to patients. However, we recognize that the proposed approach would involve substantial logistical and political challenges. This proposed innovation cycle differs from the phase-1 to phase-3 cycles used by the United States Food and Drug Administration (FDA). Phase-1 studies are safety studies that are conducted on healthy volunteers who take subtherapeutic drug doses. Phase-2 studies are small, short-term clinical trials, while phase-3 trials are large, long-term clinical trials.
The proposed innovation cycle is based on the premise that certain study designs are more appropriate at specific stages6. Levels of evidence are useful to describe the types of studies at each stage. Levels of evidence are simple tools based on study design for four types of studies: therapeutic, prognostic, diagnostic, and economic analysis. For each type of study, there are five levels with specific criteria. Higher levels of evidence are in general less prone to bias. While a full appreciation requires a complete critical appraisal, levels of evidence are a simple and rapid assessment of study quality6.
The Innovation Cycle
Clinical practice is based on an accepted or standard treatment (Fig. 1). Ideally, the accepted treatment is based on best evidence. The first stage of the innovation cycle begins when someone has a “cool idea.” In the surgical domain, this idea might involve a new implant or technique, or a better way of treating and/or diagnosing a clinical condition. Although each type of treatment strategy has a different approval process, studies are usually performed to support their adoption. With surgical techniques, no approval is usually needed. While regulations vary by country, in the U.S. (regulated by the FDA), new drugs generally require randomized trials, whereas the introduction of substantially equivalent devices usually can be approved with lesser-quality studies. The FDA has a regulatory mandate, but this process does not encompass all innovations relevant to surgery. Many ideas never proceed beyond the “amazing new idea” stage to determine if the idea works.
One example where the innovation cycle could have been implemented is with the procedure of surgical dislocation of the hip for an unstable slipped capital femoral epiphysis. This surgical technique was introduced as an innovative strategy to address the catastrophic complication of osteonecrosis, particularly related to a slipped capital femoral epiphysis. Based on a strong base of anatomical research, this innovative new idea was developed to safely dislocate the hip with a surgical procedure7.
The next step in the proposed innovation cycle is to establish proof of concept. Proof of concept can be achieved with a Level-IV uncontrolled case series. If the outcome is poor, the innovation cycle should stop at this point. Subsequent case series may reevaluate a modification of the innovation and/or use in an alternative patient population. For many innovative interventions, however, a case series (at least in an uncontrolled study) will demonstrate that patients appear to have reasonable outcomes with the innovation.
Regarding surgical dislocation, several case series have reported encouraging results that suggest low rates of osteonecrosis8,9, which demonstrate the proof of concept that the hip can be safely surgically dislocated. Because of the limitations of case series, however, proof of concept unfortunately is not the same as proof of effectiveness. Too often in orthopaedics, surgeons have proceeded directly from a case series for proof of concept to standard treatment, leading to many treatment options of uncertain effectiveness for every clinical condition. Based on the results of case series, use of this major unproven procedure of surgical dislocation has become widespread before comparative research has demonstrated its superiority over established techniques. Another example concerns a case series that implicated the measles, mumps, rubella, and varicella vaccine as a cause of autism, which led to widespread resistance to immunization. While this conclusion was later retracted10, the impact of the false results continues to influence parents who are considering immunizing their child.
After proof of concept, an alternative possible next step is to enter the first patient into a randomized clinical trial. However, this paradigm most likely would not be successful for two reasons. First, innovations frequently need refinement, and therefore, it is almost always premature to enter the first patient into a randomized clinical trial. Second, it would not be feasible to evaluate every innovation with use of a randomized clinical trial because the specialty of orthopaedics broadly lacks the expertise and infrastructure to accomplish this.
It might seem peculiar to be proposing a role for case series in an innovation cycle when evidence-based medicine places so little value on this design. The explanation is that the orthopaedic literature does not distinguish between two possible roles for case series: proof of concept for an innovation and establishment of best practice. Unfortunately, as noted above, case series can almost never establish best practice by proving treatment superiorly. This has not only led to many treatment options of uncertain effectiveness for many conditions, but it has also blurred the distinction between proof of concept and demonstration of treatment effectiveness for orthopaedic surgeons. Orthopaedic surgeons are overwhelmed by new developments and techniques, and they would likely welcome the opportunity to focus on changes to practice that are based only on major innovations that have actually been proven to benefit patients.
The orthopaedic literature is the primary source of evidence for orthopaedic surgeons. Given that journals have the vital role of determining what is published in the orthopaedic literature, journals and editors could be key in implementing the innovation cycle by no longer publishing case series as a means to demonstrate treatment effectiveness or establish best practice. Editors would have to change editorial policy and educate readers and reviewers about the narrow role of case series (Table I). This change would have several positive effects.
TABLE I -
Recommendations for Instituting the Innovation Cycle
||1. Provide authors and reviewers with guidelines that case series should follow (i.e., published only for innovative treatments and not to demonstrate treatment effectiveness).
||2. Develop editorial policies that distinguish different roles for case series and clarify that journals will publish case series only as innovative therapy.
||1. Educate membership as to the role of case series as a proof of concept.
||2. Develop guidelines for program committees to accept case series only as proof of concept for innovative therapy.
||3. Develop committees to assess innovative interventions and advise members when to adopt interventions based on quality of evidence.
||4. Encourage, solicit, and possibly support trials to evaluate innovative interventions.
|Opinion leaders/program directors
||1. In all educational material, indicate when interventions are innovative but not ready for widespread adoption.
||1. Develop criteria for innovative interventions and certify surgeons based on safe introduction into surgical practice.
||1. Develop guidelines and processes for the introduction of innovative interventions.
First, because higher levels of evidence are less prone to bias, the quality of orthopaedic literature would increase. Second, for the reader, it would clearly signal that a case series only has the purpose of establishing proof of concept for innovative interventions rather than demonstrating that the innovation is effective or proposing that the innovation should be widely adopted. Third (discussed below), it would also signal a specific role for specialty societies and opinion leaders, including department chairs and program directors, to slow or control the introduction of a major innovative intervention into clinical practice.
Program committees at annual orthopaedic society meetings would also have an important role. Because peer review in these settings is somewhat limited, putting case series in appropriate perspective should be part of an organization’s educational process: attendees should be informed that these types of presentations do not establish best practices. If journals and specialty societies were to publish or present case series only as proof of concept, then surgeons would understand that an innovation, such as surgical dislocation, is not yet ready for widespread adoption.
The next step in the innovation cycle would be to determine if the innovation might be better than standard treatments. This step could be accomplished with retrospective studies within single or multiple centers, comparing the innovative treatment with standard or accepted treatments. These retrospective controlled Level-III studies could be performed relatively rapidly in high-volume centers. These studies could be encouraged and coordinated by specialty societies at relatively minimal cost. Although Level-III retrospective controlled studies are prone to bias, that bias almost always favors the new or innovative treatment; thus, if the new innovation is found to be inferior to standard treatment, then the introduction of the innovation should be stopped.
One example of this type of study is the recent publication that examined the clinical outcome for corrective osteotomy for distal radial malunion11. This study retrospectively compared an innovative opening-wedge technique with a standard closing-wedge technique for radial malunion. This retrospective comparison showed that the new innovative opening-wedge osteotomy provided inferior results compared with the closing-wedge osteotomy. This would suggest that the newer technique should not be used unless it is further refined and shown to be beneficial in other retrospective comparisons. However, although an innovation may appear to be effective with a Level-III study, there is no proof that the innovation is effective; Level-I and II studies have the primary role of proving effectiveness. As another example, we could evaluate the rate of osteonecrosis that develops with surgical hip dislocation compared with closed reduction, particularly related to a slipped capital femoral epiphysis12. If the innovation appeared to be effective with use of Level-III studies, it would be the role of opinion leaders and specialty societies to slow the introduction of the innovation until it is actually proven to be a better procedure.
We suggest that opinion leaders and program directors take a key role in promoting the innovation cycle by advising against widespread adoption of unproven innovations. While some opinion leaders may have relationships with industry, declaration of conflicts of interest, along with a specialty-wide understanding of the innovation cycle, could hopefully minimize this potential bias. Specialty societies will need to take on the critical role of advising their membership that innovative new treatments should not be widely adopted until definitive evidence is available (as discussed below). While not all specialty societies have the funds to fully support clinical trials, research funds could be used for pilot projects and feasibility studies. At a minimum, specialty societies should create health technology assessment committees that could evaluate major innovations, determine when definitive proof of effectiveness is available, and advise their membership accordingly. A precedent for this role occurred when the American College of Surgeons advised their membership to not pursue laparoscopic colectomy until prospective trials demonstrated its safety and effectiveness13. Other specialty societies that take this role include the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical, which has the specific purpose of performing health technology assessment to control the introduction of new surgical innovations, and the National Institute for Health and Care Excellence in the United Kingdom (http://www.nice.org.uk/), which is responsible for determining the effectiveness of health care interventions. Finally, certification bodies, such as the American Board of Orthopaedic Surgery, could take on the important role of monitoring the use of unproven innovative therapy in the certification and the maintenance of certification process.
The next stage in the innovation cycle would be to prove that the innovation is actually a better approach than previous approaches. This stage can be accomplished with either Level-I randomized or Level-II prospective comparison studies. The innovation would be evaluated with controls with clinically relevant outcomes in several or many centers. Referring again to the example of surgical hip dislocation, a prospective comparison, preferably randomized, would evaluate the rate of osteonecrosis when surgical dislocation is compared with closed reduction of a slipped capital femoral epiphysis2. This step should occur before widespread adoption and have the support of opinion leaders and specialty societies. Unfortunately, clinical trials take many years to finish, but specialty societies should support and possibly fund multicenter prospective and, hopefully, randomized studies. Clinical trials are expensive; however, alternative models that include fewer patients from many centers can be less costly2.
In deciding which type of trial to conduct, it is important to understand the difference between efficacy and effectiveness. A trial that demonstrates efficacy would show that the treatment works under ideal conditions, whereas a trial that demonstrates effectiveness would show that it works under usual conditions. The type of trial that would be appropriate is partially dependent on the expectations of where the innovation would ultimately be used. If the intervention is primarily designed for high-volume specialty centers, then an efficacy trial would be most appropriate. If the intent is to have widespread utilization of the innovation, then an effectiveness trial would be most appropriate. If an intervention is not shown to be useful in an efficacy trial, it almost certainly would not be useful in an effectiveness trial. If the innovation is shown to be useful in an efficacy trial, then it is uncertain as to whether the innovation would be superior in widespread implementation, thus demonstrating the importance of deciding which type of trial is most appropriate.
One example of this stage of the innovation cycle is randomized clinical trials that examine vertebroplasty for patients with a spine fracture. In two placebo-controlled trials, vertebroplasty was not helpful in improving the clinical outcome of patients with spine fractures14,15. While there may be issues related to the patients who were selected, the results of these trials suggest that physicians should not use this innovative intervention; indeed, this was the conclusion reached by an American Academy of Orthopaedic Surgeons (AAOS) practice guideline committee. AAOS practice guidelines and the recommendations of specialty societies could guide professional certification, including the American Board of Orthopaedic Surgery’s Maintenance of Certification program, to determine best practice based on higher levels of evidence (Levels I or II).
The next stage in the innovation cycle would be to determine whether the intervention works on a large scale (i.e., determining if things go wrong that were not suspected). This stage requires either a registry or an administrative claims database, which specifically would look for rare events, small differences in outcomes, or late events. This stage occurs after an innovation has been widely adopted. Because of the expense involved, registries will likely be reserved for high-volume and expensive procedures; administrative claims databases provide less-detailed clinical information but are much less expensive than registries. If the innovation cycle has been followed up to this stage, it is likely that few interventions would turn out to be harmful in the long term. However, too often, registries have shown that innovations introduced without the rigorous studies described above can lead to poor outcomes. For example, the five-year results from the Australian Orthopaedic Association National Joint Replacement Registry regarding the Articular Surface Replacement hip prosthesis have shown high revision rates compared with other hip arthroplasties3. As another example, an administrative claims database could be used to determine the rate of osteonecrosis with the surgical hip dislocation innovation described earlier.
The final step in the innovation cycle would be determining which patients would benefit most from the intervention. This stage requires prospective evaluations on different types of patients with Level-I, II, and III studies. One example is a recent study that examined the effect of physiologic posterolateral rotatory laxity on the clinical outcome of anterior cruciate ligament reconstruction16. This study demonstrated that patients with posterolateral ligament laxity had inferior results to those without ligament laxity. Although this study does not provide information as to whether there is more benefit than harm in anterior cruciate ligament reconstruction in patients with posterolateral laxity, it does aid physicians and patients in deciding upon appropriate interventions.
An additional consideration for individual centers is to determine whether or not they have the expertise to introduce an innovation after it has been proven effective in other centers. This is a quality improvement process that controls the introduction of an innovation to an individual center. For example, at The Hospital for Sick Children, there is a fourteen-step application whereby the surgeon-in-chief controls the introduction of an innovative therapy. As part of this process, there is an evaluative step that includes peer review. At the end of the process, the innovation either becomes standard practice or is shifted to an ethics approval process for a research study, or there is continued evaluation through a quality improvement process.
While the innovation cycle has much potential benefit, it must be recognized that important barriers to changing the status quo originate from industry, patients, and resistance within the orthopaedic community. Table I details the specific roles and recommendations that journals, societies, leaders/directors, certification bodies, and hospitals should have in the process. We propose that the lead should come from journal editors, followed closely by specialty societies. With the action and support of editors and specialty societies, certification bodies would then be in a position to judge acceptable practice. If journals, specialty societies, and certification bodies tackle this difficult issue together, we suspect that opinion leaders and industry will follow.
In summary, an innovation cycle would require different types of study at different stages. If adopted, an innovation cycle would place important responsibilities on opinion leaders and specialty societies to advise their members concerning the introduction of new technologies. In addition, it would place a special responsibility on journals to publish case series to introduce highly innovative treatments rather than publishing them as a means of demonstrating treatment effectiveness. There is a time and a place for distinctive studies at different stages. The innovation cycle, if followed correctly, could ultimately save money and provide better outcomes for patients.
1. Bozic KJMJ, Maselli J, Pekow PS, Lindenauer PK, Vail TP, Auerbach AD . The influence of procedure volumes and standardization of care on quality and efficiency in total joint replacement surgery. J Bone Joint Surg Am. 2010 Nov 17;92(16):2643-52.
2. Wright JG, Yandow S, Donaldson S, Marley L ; Simple Bone Cyst Trial Group. A randomized clinical trial comparing intralesional bone marrow and steroid injections for simple bone cysts. J Bone Joint Surg Am. 2008 Apr;90(4):722-30.
3. de Steiger RN, Hang JR, Miller LN, Graves SE, Davidson DC . Five-year results of the ASR XL Acetabular System and the ASR Hip Resurfacing System: an analysis from the Australian Orthopaedic Association National Joint Replacement Registry. J Bone Joint Surg Am. 2011 Dec 21;93(24):2287-93.
4. Tejwani NC, Immerman I . Myths and legends in orthopaedic practice: are we all guilty? Clin Orthop Relat Res. 2008 Nov;466(11):2861-72. Epub 2008 Aug 26.
5. Roos EMAOHRCELSL, Ostenberg A, Roos H, Ekdahl C, Lohmander LS . Long-term outcome of meniscectomy: symptoms, function, and performance tests in patients with or without radiographic osteoarthritis compared to matched controls. Osteoarthritis Cartilage. 2001 May;9(4):316-24.
6. Wright JG. A practical guide to assigning levels of evidence. J Bone Joint Surg Am. 2007 May;89(5):1128-30.
7. Gautier EKG, Ganz K, Krügel N, Gill T, Ganz R . Anatomy of the medial femoral circumflex artery and its surgical implications. J Bone Joint Surg Br. 2000 Jul;82(5):679-83.
8. Slongo T, Kakaty D, Krause F, Ziebarth K . Treatment of slipped capital femoral epiphysis with a modified Dunn procedure. J Bone Joint Surg Am. 2010 Dec 15;92(18):2898-908.
9. Sankar WN, McPartland TG, Millis MB, Kim YJ . The unstable slipped capital femoral epiphysis: risk factors for osteonecrosis. J Pediatr Orthop. 2010 Sep;30(6):544-8.
10. Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 2010 Feb 6;375(9713):445.
11. Wada T, Tatebe M, Ozasa Y, Sato O, Sonoda T, Hirata H, Yamashita T . Clinical outcomes of corrective osteotomy for distal radial malunion: a review of opening and closing-wedge techniques. J Bone Joint Surg Am. 2011 Sep 7;93(17):1619-26.
12. Alves C, Steele M, Narayanan U, Howard A, Alman B, Wright JG . Open reduction and internal fixation of unstable slipped capital femoral epiphysis by means of surgical dislocation does not decrease the rate of avascular necrosis: a preliminary study. J Child Orthop. 2012;6:277-83.
13. Clinical Outcomes of Surgical Therapy Study Group. A comparison of laparoscopically assisted and open colectomy for colon cancer. N Engl J Med. 2004 May 13;350(20):2050-9.
14. Buchbinder R, Osborne RH, Ebeling PR, Wark JD, Mitchell P, Wriedt C, Graves S, Staples MP, Murphy B . A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. N Engl J Med. 2009 Aug 6;361(6):557-68.
15. Kallmes DF, Comstock BA, Heagerty PJ, Turner JA, Wilson DJ, Diamond TH, Edwards R, Gray LA, Stout L, Owen S, Hollingworth W, Ghdoke B, Annesley-Williams DJ, Ralston SH, Jarvik JG . A randomized trial of vertebroplasty for osteoporotic spinal fractures. N Engl J Med. 2009 Aug 6;361(6):569-79.
16. Kim SJ, Choi DH, Mei Y, Hwang BY . Does physiologic posterolateral laxity influence clinical outcomes of anterior cruciate ligament reconstruction? J Bone Joint Surg Am. 2011 Nov 2;93(21):2010-4.
Disclosure: None of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of any aspect of this work. One or more of the authors, or his or her institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. No author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.