The lack of evidence in medicine was recently demonstrated in an article by Gina Kolata, health columnist for The New York Times, discussing the evidence surrounding beta-carotene and its use as a dietary supplement to protect against cancer. It seems rather intuitive that eating more fruits and vegetables containing beta-carotene, an orange-red organic compound that is a precursor to vitamin A, should make one healthier. After all, our mothers have known all along the wonderful antioxidant effect of beta-carotene, and they have been encouraging us to eat more carrots and broccoli because these vegetables are healthful foods that can make us stronger and run faster, and potentially protect us against ailments. To demonstrate this point, several promising observational studies have shown that people who eat more fruits and vegetables have lower cancer risk,8–10 and laboratory studies have demonstrated the positive effects of beta-carotene on the immune system of the rat.11 Therefore, not only does eating more beta-carotene–containing foods or consuming beta-carotene supplements appear logical, this belief is also supported by level II and level III studies that corroborated the beneficial effects of beta-carotene.
“Well, not so fast,” as stated by one of the football gurus in the college pregame show. Two randomized controlled trials in which study subjects took beta-carotene pills or placebos failed to demonstrate a protective effect against cancer and heart disease. In fact, these studies showed a statistically significant increase in lung cancer mortality of subjects taking high levels of beta-carotene over the subjects receiving placebo pills.12–14 Results of these trials dashed the hopes of millions of people who swore by this supplement. So how can this be, when all the cumulative evidence supports the effectiveness of this magic pill in protecting one from disastrous health problems? In addition, how valid is the evidence from the multiple observational and animal studies that showed the efficacy of beta-carotene?
The answer lies in the basic epidemiology concept of confounding variables and biases. In these observational studies, the study subjects who took beta-carotene were very different from the subjects who did not take this dietary supplement. The subjects who took beta-carotene were potentially more motivated to be healthy; perhaps they exercised more, refrained from smoking, or ate less fatty food. Although an epidemiologist can usually account for potential confounding variables, unknown confounders can contribute to differences between two study groups. Because it is impossible for a statistician to control for unknown confounders and selection biases that can be introduced into the study unbeknownst to the investigators, observational studies are categorized as level II (prospective) or level III (retrospective) evidence studies. By contrast, in clinical trials, the study subjects are allocated randomly to either intervention or no-intervention groups.15 For example, the intervention group that underwent a surgical treatment or took a medication is compared with the control group that received an alternative surgical treatment or took the placebo pill. Therefore, the random allocation of intervention, provided that the sample size is sufficiently large, should balance out known and unknown confounders. Thus, in this strict experimental situation, level I evidence is produced that can potentially define whether the intervention is truly effective.
Despite the allure of randomized controlled trials, there are certain perils with clinical trials (Table 4). For example, the results may not be generalizable to the entire population because they often are conducted in large academic settings under strict inclusion and exclusion criteria. Patients in these academic centers may be quite different from those in the general population, and the level of expertise in these centers may be higher than in some community hospitals. Therefore, randomized controlled trials are often considered as efficacy studies that are conducted under ideal situations, when compared with effectiveness studies that are conducted without randomization designs. In an effort to achieve generalizable results, the National Institutes of Health put forth a conscious effort to request inclusion of minority groups in National Institutes of Health–funded clinical trials to ensure that the results can be applicable to all racial groups and for various socioeconomic strata.
EVIDENCE-BASED MEDICINE, THE AMERICAN SOCIETY OF PLASTIC SURGEONS, AND PLASTIC AND RECONSTRUCTIVE SURGERY: A NEW INITIATIVE
It is distressing when a long-held belief regarding a treatment’s effectiveness is dispelled by a more rigorous study design. However, striving for the highest level of evidence in clinical research is essential and will guide the decision-making process, to allow us to provide the best treatment for our patients. It is unnerving to challenge a long-established practice. For example, dextran is routinely used after free flap surgery, and its use may be justified if evidence shows that it can prevent free flap failure. However, because the risk of flap failure in high-volume centers is quite low, it will require the recruitment of thousands of subjects to reach adequate statistical power. The cost and effort associated with a dextran study may be prohibitive. Therefore, randomized controlled trials should be designed when the study question is so important that the results derived from level I evidence can have an important impact in changing practice patterns. Plastic and Reconstructive Surgery is embarking on an exciting initiative to promote evidence-based medicine. With more effort by researchers to conduct randomized controlled trials, we may witness the walls coming down on many of the sacrosanct practices that we have cherished. The demise of the beta-carotene mirage may serve as a guide, compelling us to seek high-level evidence in our own practices.
The American Society of Plastic Surgeons (ASPS) has been spearheading evidence-based medicine through the work of three important committees: the Quality and Performance Measurement Committee, the Health Policy Committee, and the Patient Safety Committee. The premier journal of this specialty—Plastic and Reconstructive Surgery—plans to partner with the ASPS in this movement. The designation of Kevin C. Chung, M.D., M.S., as Section Editor for Outcomes and Evidence-Based Medicine will strive to introduce the concept of evidence-based medicine to our contributing authors and readers. Of course, presenting this concept cannot occur overnight and will require a major shift in how the specialty approaches scientific literature and how it applies evidence in decision-making and clinical care. Several of the proposed evidence-based medicine initiatives aim to:
1. Critically appraise clinical studies with a preference to accept articles with higher levels of evidence.
2. Highlight a key article in each issue of Plastic and Reconstructive Surgery that can serve as a model for an evidence-based medicine–type article.
3. Provide tutorial guides on various aspects of evidence-based medicine such as how to craft practice guidelines, how to distill evidence based on statistical principles, and how to perform economic analysis projects.
4. Invite discussions of selected evidence-based medicine articles by experts who are able to evaluate whether recommendations for new treatments have sufficient evidence to support adoption.
5. Publish systematic reviews and, if possible, meta-analyses that summarize data from the literature and provide readers with an unbiased scientific critique of evidence for plastic surgery technologies and treatments.
6. Collaborate with the ASPS in promoting high-level evidence studies (e.g., deriving evidence for deep venous thrombosis prophylaxis and analyzing outcomes of reconstruction for lower leg injuries).
These are ambitious goals that may take several years to complete and involve many simultaneous and intense efforts. Plastic surgery must be more methodical in designing and conducting research, and even more critical in evaluating research publications; therefore, Plastic and Reconstructive Surgery will look for opportunities to capitalize on initiatives already underway that address these issues. For example, the Plastic Surgery Educational Foundation is partnering with the Research Council to hold an inaugural symposium, on May 30 and 31, 2009, following the Research Council meeting in Pittsburgh, which will include topics on how to conduct clinical studies. We will encourage organized plastic surgery to support high-impact research questions and grant applications that will attract a critical mass of clinical researchers who are capable of conducting evidence-based medicine research studies. These efforts will result in publication of higher level scientific evidence, thereby promoting the mission of plastic surgery to ensure the safety and best possible outcomes for our patients.
The evidence-based medicine movement in plastic surgery is synergistic with advocacy efforts by the ASPS, which depend on sound scientific data to justify fair insurance-coverage criteria and adequate third-party payer reimbursement for plastic surgery procedures. Unless plastic surgery can provide rich data to support the global scope of plastic surgery practices, payers will continue to use imperfect and often skewed data to their advantage and thus continue to decrease reimbursement. Plastic surgery should lead in the evidence movement or others will be more than happy to lead us. Plastic surgery is a specialty of innovation and the challenge has been presented to us—we must engage and energize our specialty to produce publications of highest levels of evidence, thereby promoting plastic surgeons as the leading scientists in the surgical disciplines.
Supported in part by a grant from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01 AR047328) and a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) (K.C.C.).
1. Davis Sears E, Burns PB, Chung KC. The outcomes of outcome studies in plastic surgery: A systematic review of 17 years of plastic surgery research. Plast Reconstr Surg.
2. Chung KC, Ram AN. Evidence-based medicine: The fourth revolution in American medicine? Plast Reconstr Surg.
3. Sackett DL, Straus SE, Richardson WS, et al., eds. Evidence-Based Medicine: How to Practice and Teach EBM.
2nd ed. Philadelphia: Elsevier/Churchill Livingstone; 2000.
4. Chung KC, Rohrich RJ. Measuring quality of surgical care: Is it attainable? Plast Reconstr Surg.
5. Chung KC, Shauver M. Measuring quality in healthcare and its implications for pay-for-performance initiatives. Hand Clin.
6. Shah HM, Chung KC. Archie Cochrane and his vision for evidence-based medicine. Plast Reconstr Surg.
7. McCarthy CM, Collins ED, Pusic AL. Where do we find the best evidence? (Discussion). Plast Reconstr Surg.
8. Riboli E, Norat T. Epidemiologic evidence of the protective effect of fruit and vegetables on cancer risk. Am J Clin Nutr.
9. Steinmetz KA, Potter JD. Vegetables, fruit, and cancer: I. Epidemiology. Cancer Causes Control
10. Block G, Patterson B, Subar A. Fruit, vegetables, and cancer prevention: A review of the epidemiological evidence. Nutr Cancer
11. Bendich A, Shapiro SS. Effect of beta-carotene and canthaxanthin on the immune responses of the rat. J Nutr.
12. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. N Engl J Med.
13. Hennekens CH, Buring JE, Manson JE, et al. Lack of effect of long-term supplementation with beta carotene on the incidence of malignant neoplasms and cardiovascular disease. N Engl J Med.
14. Omenn GS, Goodman GE, Thornquist MD, et al. Effects of a combination of beta carotene and vitamin A on lung cancer and cardiovascular disease. N Engl J Med.
15. Chung KC, Burns PB. A guide to planning and executing a surgical randomized controlled trial. J Hand Surg (Am.)
Submit your manuscript today through PRS’ Enkwell. The Enkwell submission and review website helps make the submission process easier, more efficient, and less expensive for authors, and makes the review process quicker, more accessible, and less expensive for reviewers. If you are a first-time user, be sure to register on the system.
©2009American Society of Plastic Surgeons