Share this article on:

Where Do We Find the Best Evidence?

McCarthy, Colleen M. M.D., M.S.; Collins, E Dale M.D.; Pusic, Andrea L. M.D., M.H.S.

Plastic & Reconstructive Surgery: December 2008 - Volume 122 - Issue 6 - pp 1942-1947
doi: 10.1097/PRS.0b013e31818d2098
Special Topics

Summary: As pressure on resources increases, decision makers in health care are increasingly seeking high-quality, scientific evidence to support clinical and health policy choices. Ultimately, legislators will look to develop performance measures based on evidence, rather than on consensus or commonality of practice. As plastic surgeons, we can take an increased role in producing impartial evidence on the efficacy of our surgical interventions. We can move away from the retrospective reporting of cases and nonrandomized studies and instead rely on prospective, randomized trials addressing important clinical issues. In doing so, we will challenge the common practice of basing clinical decisions solely on what we are taught, procedures we are most comfortable doing, or procedures that are most financially rewarding. These efforts will not only afford us the best opportunity to provide optimal care to patients but also allow us to demonstrate the inherent quality of our surgical specialty.

New York, N.Y.; and Lebanon, N.H.

From the Plastic and Reconstructive Surgery Service, Department of Surgery, Memorial Sloan-Kettering Cancer Center, and the Norris Cotton Cancer Center, Dartmouth-Hitchcock Medical Center.

Received for publication October 1, 2007; accepted February 20, 2008.

Disclosure: None of the authors has a financial conflict of interest to disclose with regard to the content of this article.

Colleen M. McCarthy, M.D., M.S., Memorial Sloan-Kettering Cancer Center, 1275 York Avenue, Room MRI-1007, New York, N.Y. 10021, mccarthc@mskcc.org

“Half of what we teach you here is wrong. Unfortunately, we do not know which half.” - —Dean’s address, first day of medical school1

We have all become accustomed, perhaps even immune, to the term “evidence-based medicine.” Of course, each of us makes decisions based on evidence. Our evidence is derived from years of medical and surgical practice, from careful interpretation of the scientific literature, and from our personal and professional experiences. All of this “evidence” is important and relevant to the best possible patient care, but where do we find the best evidence to guide our decisions?

As practicing physicians and surgeons, we learn much of our craft in an apprenticeship model. We learn by careful observation, adopting and translating the techniques and traditions of our mentors, the experts. Until recently, no one questioned the validity of this practice model. It was the accepted norm, and many techniques passed from one generation to the next were based on personal and anecdotal experience, never having been subjected to a more rigorous evaluation.

In the current environment of quality surveillance and healthcare regulation, there is rising pressure on physicians and surgeons to provide scientifically based evidence to support our treatment recommendations and practices. We are pressed by payers to provide peer-reviewed literature that demonstrates the efficacy of our prescribed treatments. We are pushed by patients to give evidence of personal experience and success with procedures compared with published benchmarks. We are prodded by governmental agencies to provide the highest levels of evidence to guide development of “quality measures.” In this environment, the expert opinions we have relied on in the past will be insufficient to carry us successfully into the future of healthcare.

Back to Top | Article Outline

HOW DO WE DEFINE “BEST EVIDENCE”?

Before we get to where to find the best evidence, perhaps it would be useful to define more clearly just what it is. A number of groups and organizations have attempted to classify and stratify high-quality versus lower-quality evidence. Table 1 presents a synthesis and description of the basic categorization of “levels of evidence.” Randomized controlled trials represent the highest level of evidence (level I), followed by large cohort studies (level II), case-control studies (level III), case series (level IV), and expert opinion (level V).2

Even a superficial perusal of our plastic surgery journals suggests they are replete with descriptions of procedures by individual surgeons and the outcomes of those procedures in “expert” hands (Fig. 1). At best, these studies can be categorized as level IV evidence. At worst, they represent the lowest level of evidence, expert opinion. David Sackett, the founding father of evidence-based medicine, writes, “The first sin committed by experts consists in adding their prestige and their position to their opinions, which give the latter far greater persuasive power than they deserve on scientific grounds alone.”3

It must be said that while case reports and/or case series may be considered lower-level evidence, they still have a role to play. Specifically, they can be used to generate hypotheses, communicate rare events, and provide descriptive reports that may be useful when defining or modifying existing techniques. The call for higher-level evidence in plastic surgery is not intended to imply that we should ignore what can be learned from true experts, whose insights may never be captured by formal scientific investigation. Instead, when a new technique is conceived, it should be tested under controlled conditions to generate evidence for its efficacy before being widely disseminated. There is no reason to limit innovation or experimentation; rather, we seek to ensure that potentially inferior treatments do not replace superior ones simply for reasons of novelty or peer pressure.

Back to Top | Article Outline

“WHY RANDOMIZED CONTROLLED TRIALS?”

Why are randomized controlled trials the accepted standard of clinical research? Because they provide the most rigorous way of determining whether a cause-effect relationship exists between treatment and outcome.4 Other study designs can detect associations between an intervention and an outcome, but they cannot rule out the possibility that the association was caused by a third factor or confounding variable. Randomization minimizes the risk that systematic differences exist among patient groups.5,6

Back to Top | Article Outline

CHALLENGES TO PERFORMING RANDOMIZED CONTROLLED TRIALS IN SURGERY

Unfortunately, the difficulties of conducting randomized controlled trials in surgery are many and include not only blinding and concealment but also patient recruitment, financial cost, and ethical concerns.7–9 It has been argued that enrolling surgical patients in a randomized controlled trial creates ethical “discontent” among some surgeons. This is particularly true when the surgeons involved believe that one treatment is superior to the other. On the other hand, experience shows that failure to perform randomized trials may result in harmful treatments being perpetuated. For example, patients with severe emphysema were once routinely treated with lung volume reduction surgery until a randomized controlled trial demonstrated that, for many, this practice significantly increased the risk of death.10 Similarly, extracranial-intracranial bypass was routinely performed to prevent future strokes in patients who had suffered minor strokes, based on evidence gleaned from multiple case studies. The practice continued until a large, randomized controlled trial demonstrated that patients who had the bypass had a relative worsening of functional status compared with those who did not undergo the procedure.11

Ethical principles require that healthy subjects not be disadvantaged by participation in clinical research. According to this concept, to carry out a randomized controlled trial, “clinical equipoise” must exist. The requirement for clinical equipoise is satisfied if there is genuine uncertainty within the expert medical community–-not necessarily on the part of an individual investigator–-about the preferred treatment. Under such circumstances, it is ethical to offer patients entry into a trial.

Such surgical equipoise is abundant in plastic surgery. Longaker and Rohrich wrote in a recent editorial, “Innovation … has distinguished plastic surgeons historically and will continue to do so in the future.”12 Plastic surgeons are constantly seeking to improve on current practice and provide durable solutions to clinical problems. With innovation, however, comes the need to evaluate outcomes: Is there a clinically significant difference in donor-site morbidity following pedicled transverse rectus abdominis musculocutaneous (TRAM) flaps, free TRAM flaps, or deep inferior epigastric perforator flaps? Does mesotherapy improve body contour? Does the maintenance of breast shape following reduction mammaplasty differ based on the technique performed? Does using AlloDerm in implant-based breast reconstruction improve patient outcomes, and is it cost-effective? Do venous couplers minimize the risk of venous thrombosis? The answers to these clinical questions, where genuine uncertainty exists in regard to their answers, can best be answered by randomized controlled trials while meeting the requirements for clinical equipoise.

In some circumstances, a randomized controlled trial may be ethical but unfeasible because of recruitment difficulties. Indeed, once an intervention becomes widespread, it can prove impossible to recruit clinicians who are willing to “experiment” with alternatives. Strong patient preferences may also limit recruitment and bias outcomes, if not accommodated within the study design. Given these constraints, it remains an ideal that evaluation of innovative techniques, procedures, or devices should occur before their uptake into clinical practice.

Conducting a rigorous randomized controlled trial to evaluate a surgical intervention also demands special consideration of issues such as variability in surgical proficiency and standardization of techniques. One way to overcome the problem of differential expertise bias is an “expertise-based randomized controlled trial.” Patients are randomized to a “surgeon” rather than to a treatment arm.13 One of the advantages here is that surgeons will perform only the procedure in which they have expertise, avoiding the problem of differential expertise. In addition, procedural crossovers are less likely to occur because surgeons are performing the procedures with which they are most comfortable. Finally, expertise-based randomized trials may have greater applicability and feasibility than conventional trials.14

Perhaps, one of the biggest challenges to performing randomized controlled trials is the cost of funding such a study. These trials often involve a small number of centers with high volumes of patients and dedicated study personnel. Thus, performing such a trial can be quite expensive. Unlike with pharmacologic trials, surgical trials rely mainly on funding from academia. To this end, most of our plastic surgical societies will provide grant support on a competitive basis for high-quality research, including randomized controlled trials, in plastic surgery. Not only can we rely on our specialist societies for financial support but we can also look to them to provide us with both knowledge and knowledge infrastructure. For example, many researchers in other surgical disciplines have access to a members-only, clinical trials Web site. Such a forum for plastic surgeons could both encourage collaboration and participation in multicenter studies and allow the membership at large to monitor the progress of ongoing clinical trials. In addition, information about other academic, philanthropic, and/or industry funding opportunities could be communicated.

Back to Top | Article Outline

NOT ALL RANDOMIZED CONTROLLED TRIALS ARE CREATED EQUAL

Although many of these issues make surgical trials more challenging to perform, they rarely preclude their performance. In fact, in view of their high cost and treatment-associated risk, it is all the more important that surgical interventions be supported by the best possible evidence. Yet, while high-quality randomized controlled trials can provide the strongest evidence of therapeutic effectiveness, the mere description of a study as “randomized” does not allow clinicians to infer validity. In a recent review, Offer and Perks8 reported that few studies in plastic surgery are designed as randomized controlled trials and those that are should be of a higher quality.15,16 To improve the quality of trials and to ensure that trials are carried out properly, the Consolidated Standards of Reporting Trials (CONSORT) statement was published.12,17 This report, detailing items to be included in the methods section of a randomized controlled trial, has been endorsed by other high-quality, peer-reviewed publications, including The Lancet, the BMJ, and the Journal of the American Medical Association.18 Thus, given that poor design may lead to biased outcomes, surgical researchers should strive to produce evidence with methodologic rigor (Table 2).

Back to Top | Article Outline

NOT ALL QUESTIONS MAY BE ANSWERED USING A RANDOMIZED CONTROLLED TRIAL

Although the randomized controlled trial is considered the accepted standard in research design, used to establish the effectiveness of a treatment, it is not a panacea for all clinical questions. The strength of the randomized controlled trial is that conditions are tightly controlled to minimize bias and the risk of arriving at an incorrect conclusion. However, because of this, their results may lack generalizability. Whereas variations in surgical practice are minimized in randomized controlled trials, outcomes studies try to determine what role these factors play in the “real world.” Moreover, they can be used to increase our understanding of the effect of various external factors on outcome.

Similarly, in situations where there is a question of harm causation, a randomized controlled trial may be unnecessary or inappropriate. In 2003, Smith and Pell published an entertaining article, “Parachute Use to Prevent Death and Major Trauma Due to Gravitational Challenge.”19 They used the lack of randomized controlled trials in testing parachutes to show that situations still exist where such trials are unnecessary and, more importantly, unethical. When surgical equipoise does not exist or randomized controlled trials are simply not feasible, the efficacy of surgical treatments must instead rely on well-designed observational or cohort studies. These studies present an opportunity to study outcomes temporally related to a specific exposure or intervention and allow the calculation of incidence rates and relative risk. Two or more groups may be compared; yet, without a regulated control group, extraneous factors may distort the findings of these observational studies.

Whatever method of evaluation is deemed appropriate, good study design remains critical to minimizing bias. Lessons learned from randomized controlled trials can be applied to other study designs and can help improve the quality of prospective studies and retrospective research. As with randomized controlled trials, analyses can adopt inclusion and exclusion criteria, perform sample size calculations before outcome analyses, match participants for prognostic variables, and/or use statistical modeling to adjust for the effect of confounding.

Back to Top | Article Outline

OUTCOME MEASURES

While a randomized controlled trial can provide the strongest evidence of the efficacy of an intervention, clinically meaningful outcome measures are required if the true success of a surgical intervention is to be evaluated. Traditional methods of evaluation based on morbidity and mortality are often necessary but seldom sufficient. In addition to the traditional outcomes, the efficacies of many surgical procedures are now being evaluated with respect to their impact on health-related quality of life. This is of particular importance in plastic surgery, where interventions may significantly affect day-to-day functioning or quality of life, yet may have little impact on mortality or length of life.

Research efforts that evaluate outcomes from a surgeon’s perspective fail to appreciate that the patient’s viewpoint may be significantly different. For this reason, careful consideration of patient-reported outcomes is vitally important to research efforts. Thus, there is an ongoing need for disease-specific, patient-based outcome measures that satisfy accepted health measurement criteria. Using such instruments in randomized controlled trials, we can measure the success of different surgical therapies from patients’ perspectives and identify patient groups who are most likely to benefit from specific surgical techniques.

Back to Top | Article Outline

CONCLUSIONS

As plastic surgeons, we can assume an increased leadership role in producing impartial evidence on the efficacy of our surgical interventions. We can move away from the retrospective reporting of cases and nonrandomized studies and instead rely on prospective, randomized trials addressing important clinical issues. In doing so, we will challenge the common practice of basing clinical decisions solely on what we are taught, procedures we are most comfortable performing, or interventions that are most financially rewarding.

These efforts will not only afford us the best opportunity to provide optimal care to patients but also allow us to demonstrate the inherent quality of our surgical specialty. In addition, impartial analysis of evidence from well-designed studies will provide a credible source of information about what forms of care are effective. As pressure on resources increases, decision makers in health care are increasingly seeking high-quality scientific evidence to support clinical and health policy choices. Ultimately, legislators are looking to develop performance measures based on evidence rather than on consensus or commonality of practice. Consequently, a strong evidence base will be an attraction for performance incentives and advocacy for our specialty.

Back to Top | Article Outline

REFERENCES

1. Sanders, L. Medicine’s progress, one setback at time. The New York Times, March 16, 2003.
2. American Society of Plastic Surgeons. Scales for Rating Levels of Evidence, September 2007. Available at: http://www.plasticsurgery.org/medical_professionals/health_policy/Scales-for-Rating-Levels-of-Evidence.cfm. Accessed September 20, 2007.
3. Michel, L. A. The epistemology of evidence-based medicine. Surg. Endosc. 21: 145, 2007.
4. Straus, S., Haynes, B., Glasziou, P., et al. Misunderstandings, misperceptions, and mistakes. ACP Journal Club 146: A8, 2007.
5. Center for Evidence-Based Medicine. Levels of Evidence, March 2002. Available at: www.cebm.net. Accessed August 27, 2007.
6. Fung, E. K., and Lore, J. M., Jr. Randomized controlled trials for evaluating surgical questions. Arch. Otolaryngol. Head Neck Surg. 128: 631, 2002.
7. Young, J. M., and Solomon, M. J. Improving the evidence base in surgery: Sources of bias in surgical studies. Aust. N.Z. J. Surg. 73: 504, 2003.
8. Offer, G. J., and Perks, A. G. In search of evidence-based plastic surgery: The problems faced by the specialty. Br. J. Plast. Surg. 53: 427, 2000.
9. McLeod, R. S. Issues in surgical randomized controlled trials. World J. Surg. 23: 1210, 1999.
10. National Emphysema Treatment Trial Research Group. Patients at high risk of death after lung-volume-reduction surgery. N. Engl. J. Med. 345: 1075, 2001.
11. Haynes, R. B., Mukherjee, J., Sackett, D. L., et al. Functional status changes following medical or surgical treatment for cerebral ischemia: Results of the extracranial-intracranial bypass study. J.A.M.A. 257: 2043, 1987.
12. Longaker, M. T., and Rohrich, R. J. Innovation: A sustainable competitive advantage for plastic and reconstructive surgery. Plast. Reconstr. Surg. 115: 2135, 2005.
13. Khan, A. A., Murthy, A. S., and Ali, N. Randomized controlled trials in plastic surgery (Letter). Plast. Reconstr. Surg. 117: 2080; reply 2081, 2006.
14. Devereaux, P. J., Bhandari, M., Clarke, M., et al. Need for expertise based randomised controlled trials. B.M.J. (Clin. Res. Ed.) 330: 88, 2005.
15. Karri, V. Randomised clinical trials in plastic surgery: Survey of output and quality of reporting. J. Plast. Reconstr. Aesthet. Surg. 59: 787, 2006.
16. Veiga Filho, J., Castro, A. A., Veiga, D. F., et al. Quality of reports of randomized clinical trials in plastic surgery. Plast. Reconstr. Surg. 115: 320, 2005.
17. Moher, D., Schulz, K. F., and Altman, D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. J.A.M.A. 285: 1987, 2001.
18. Sade, R. M. “Surgical research or comic opera” redux. Ann. Thorac. Surg. 82: 1173, 2006.
19. Smith, G. C., and Pell, J. P. Parachute use to prevent death and major trauma related to gravitational challenge: Systematic review of randomised controlled trials. B.M.J. (Clin. Res. Ed.) 327: 1459, 2003.
Back to Top | Article Outline

Recent Supplements in Plastic and Reconstructive Surgery®

* Body Contouring Surgery after Massive Weight Loss (January 2006)

* Consensus Recommendations for Soft-Tissue Augmentation with Nonanimal Stabilized Hyaluronic Acid (Restylane®) (March 2006)

* Current Concepts in Wound Healing (June 2006)

* Semipermanent and Permanent Dermal/Subdermal Fillers (September 1, 2006)

* Advances in Breast Augmentation (December 2006)

* Temporary Dermal and Soft-Tissue Fillers (November 2007)

* Craniofacial Trauma (December 2007)

* Silicone Breast Implants: Outcomes and Safety (December 2007)

* Advances in Facial Rejuvenation: Botulinum Toxin Type A, Hyaluronic Acid Dermal Fillers, and Combination Therapies—Consensus Recommendations (May 2008)

Cited By:

This article has been cited 1 time(s).

Plastic and Reconstructive Surgery
Evidence-Based Medicine and Hospital Reform: Tracing Origins Back to Florence Nightingale
Aravind, M; Chung, KC
Plastic and Reconstructive Surgery, 125(1): 403-409.
10.1097/PRS.0b013e3181c2bb89
PDF (711) | CrossRef
Back to Top | Article Outline
©2008American Society of Plastic Surgeons