Data from 2013 were then compared to data collected from 1983, 1993, and 2003 for which the mean LOE was 4.42, 4.25, and 4.16, respectively. An analysis of variance showed a significant year-to-year difference (P < 0.001), indicating an overall shift toward higher quality LOE research (Fig. 3). Although lower quality LOE studies (ie, level IV or V) initially comprised the vast majority of publications, a substantial decrease in these levels has occurred over time: from 92.8% in 1983; 87.6%, 1993; 86.9%, 2003; to 51.4% in 2013. At the same time, level I and II studies, deemed as higher quality, increased more than 5-fold between 1983 and 2013: 3.4% to 17.4%.
Of the articles reviewed in 2014, we found that 216 of 489 published articles met our inclusion criteria. There were 12 randomized controlled trials, 3 meta-analyses, 7 systematic reviews, 37 prospective studies, 130 retrospective studies, 14 case series, 3 cross-sectional studies, and 10 case reports. Of the studies, 4 (1.9%) were level I evidence studies; 35 (16.2%), level II; 75 (34.7%), level III; 89 (41.2%), level IV; and 13 (6.0%), level V. The mean LOE of PRS articles in 2014 was 3.33, demonstrating continued higher LOE. There was also a decrease in the percentage of level IV and V studies to 47.2% (from 51.4% in 2013), whereas higher quality level I and II studies had increased to 18.1% (from 17.4%).
The results of this study show continued, higher quality research, as measured by LOE, although surgical research still lags behind that of other medical disciplines. For example, even in 1995, 53% of internal medicine decisions were supported by randomized controlled trial evidence,10,11 whereas only 17% of studies in PRS in 2013 were level I and II evidence. The lag in surgical research in plastic surgery has occurred because of the difficulty in applying more sophisticated study designs to clinical questions in surgery. This lag is also evident in other surgical disciplines. For example, in Diseases of the Colon and Rectum, there were 5 randomized controlled surgical trials in 1990, 13 in 1995, and 17 in 2000.12
Medical disciplines continue to make a concerted, progressive effort toward evidence-based practice as a way not only to validate and standardize care but also to ensure optimal outcomes. However, inherent difficulties remain in conducting randomized controlled trials in surgery. Hassanein et al13 described several common obstacles: frequent lack of equipoise both for investigators and for patients, higher costs of surgical interventions, difficulties in enrolling patients in a placebo control, and the subjective nature of outcomes. It is certainly difficult for surgeons to justify randomizing patients to the investigative arm of a study with unproven outcomes when they have had relatively positive experiences with the technique being used for the control group. Similarly, it may also be challenging to convince patients to subject themselves to an experimental procedure with unknown outcomes when they are aware that a traditionally reliable option exists. Perhaps more than in any other specialty, success in plastic surgery relies more on patient-reported outcomes and surgeon assessment of outcomes than on any numeric metric, which makes quantifying outcomes in this specialty much more difficult. Furthermore, unlike trials designed to determine efficacy of medications, trials of surgical techniques can be difficult to control because the techniques may be hard to reproduce and are dependent upon surgeon experience and skill.
However, those involved in surgical specialties are continuing to improve the quality of published research studies and have made great strides over recent years, especially in neurosurgery but also in orthopedic, otolaryngologic, and ophthalmologic surgery.14,15 Although plastic surgery lags slightly behind these other specialties, the gap has narrowed substantially. In the past, lower ranked LOE publications, that is, levels IV and V, comprised the vast majority of publications (more than 80% of publications each year). Recently, however, this percentage has decreased substantially (47.2% in 2014), replaced by higher quality research. Before 1980, <5 studies of level I evidence were published per year in all of the plastic surgery literature. That number has since risen to >30 studies per year.16 Within the aesthetic subcategory of plastic surgery, level I evidence has increased from 0% to 2.5% over the past decade.15
As necessary as it is to improve the quality of research studies, one cannot dismiss weaker studies simply because of LOE. Many innovations and discoveries in plastic surgery have come from intrepid pioneers willing to try something new on a small scale. Conversely, a study can be very well conducted yet provide little impact if the question under investigation is of little consequence or interest. Furthermore, one must remember that while a randomized controlled trial may serve as the highest form of evidence, not every question posed can or should be answered with a controlled trial. A question regarding epidemiology may be better answered by a cross-sectional study, and concerns about long-term effects of any intervention should be examined with a longitudinal study, neither of which are often considered level I evidence. Therefore, one must be careful to remember that LOE is not necessarily synonymous with clinical guidelines because practitioner experience, patient characteristics, and research data all contribute to clinical decision making.
A disadvantage of the current study is that it examines only 1 journal in plastic surgery, PRS. We did choose the plastic surgery journal with the highest impact factor, which serves as a proxy measure for the best publication. Generalizations about LOE in other general plastic surgery journals and those with a more specific focus, such as in the disciplines of hand, microsurgery, or craniofacial surgery, can not necessarily be made based on this study. This study also compares data gathered only at 10-year intervals and therefore does not account for year-to-year discrepancies. We were able to include data from 2014 because that data were available at the time we submitted the manuscript, and we thought that it would be interesting to determine whether the tendency toward increased quality continued.
An analysis of the reasons for the increase in the quality of publications is beyond the scope of this article. However, there are several potential factors. For example, substantial changes have been made to the editorial policies of the journal, the instructions to reviewers, education of the plastic surgery community about LOE, education within residency programs, and added research requirements for graduation from residency programs. Also, the addition of new avenues for publication, such as open access journals, raises the question of whether these other types of publications siphon a portion of the lower quality submissions.
As information has become more readily available and consumers have grown more aware of treatment options, the need for guidelines and protocols that correspond to the best outcomes has become increasingly necessary. Applied correctly, EBM can also challenge the current, unsustainable increase in health care costs by eliminating unsubstantiated practices and appropriately allocating resources.17,18
Our study showed that the quality of research in plastic surgery has seen a continued upsurge as evidenced by an overall improvement in published LOE over the past 3 decades.
1. Oxman AD, Sackett DL, Guyatt GH. Users’ guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA 1993;270:20932095.
2. Feinstein AR. Clinical Judgment. 1967:Baltimore (MD): Williams and Wilkins; 414.
3. Cochrane AL. Effectiveness and Efficiency: Random Reflections on Health Services. 1972:London: Nuffield Provincial Hospitals Trust; 92.
4. The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121:1193254.
5. Sullivan D, Chung KC, Eaves FF 3rd, et al. The level of evidence pyramid: indicating levels of evidence in Plastic and Reconstructive Surgery articles. Plast Reconstr Surg. 2011;128:311314.
8. Chung KC, Ram AN. Evidence-based medicine: the fourth revolution in American medicine? Plast Reconstr Surg. 2009;123:389398.
9. Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121:207e211e.
10. Hui AC, Mak J, Wong SM, et al. The practice of evidence-based medicine in an acute medical ward: retrospective study. Hong Kong Med J. 2000;6:343348.
11. Ellis J, Mulligan I, Rowe J, et al. Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. Lancet 1995;346:407410.
12. Ko CY, Sack J, Chang JT, et al. Reporting randomized, controlled trials: where quality of reporting may be improved. Dis Colon Rectum 2002;45:443447.
13. Hassanein AH, Herrera FA, Hassanein O. Challenges of randomized controlled trial design in plastic surgery. Can J Plast Surg. 2011;19:e28e29.
14. Sinno H, Neel OF, Lutfy J, et al. Level of evidence in plastic surgery research. Plast Reconstr Surg. 2011;127:974980.
15. Chuback JE, Yarascavitch BA, Eaves F 3rd, et al. Evidence in the aesthetic surgical literature over the past decade: how far have we come? Plast Reconstr Surg. 2012;129:126e134e.
16. McCarthy JE, Chatterjee A, McKelvey TG, et al. A detailed analysis of level I evidence (randomized controlled trials and meta-analyses) in five plastic surgery journals to date: 1978 to 2009. Plast Reconstr Surg. 2010;126:17741778.
17. Swanson JA, Schmitz D, Chung KC. How to practice evidence-based medicine. Plast Reconstr Surg. 2010;126:286294.
Copyright © 2016 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of The American Society of Plastic Surgeons. All rights reserved.
18. Chung KC, Swanson JA, Schmitz D, et al. Introducing evidence-based medicine to plastic and reconstructive surgery. Plast Reconstr Surg. 2009;123:13851389.