Secondary Logo

Journal Logo

When Is “Never Enough” (Data) … Enough?

Johnson, Rebecca L. MD; Wedel, Denise J. MD; Kopp, Sandra L. MD

doi: 10.1213/ANE.0000000000002202
Editorials: Editorial

From the Department of Anesthesiology, Mayo Clinic College of Medicine and Science, Rochester, Minnesota.

Accepted for publication April 3, 2017.

Funding: None.

The authors declare no conflicts of interest.

Reprints will not be available from the authors.

Address correspondence to Sandra L. Kopp, MD, Department of Anesthesiology, Mayo Clinic College of Medicine and Science, 200 First St SW, Rochester, MN 55905. Address e-mail to

In this issue of Anesthesia &Analgesia, Smith et al1 report the results of a systematic review and meta-analysis comparing postoperative data from adult patients who received neuraxial anesthesia (NA), with or without concomitant general anesthesia (GA), to patients receiving GA alone. The primary outcome was 30-day mortality, with secondary outcomes being cardiopulmonary morbidity, surgical site infection, thromboembolic events, blood transfusions, and resource use. The authors included observational studies (27) and randomized clinical trials (11) published during the period from 2010 through the end of May 2016, which included major truncal and lower extremity surgery reporting the chosen outcomes. There were 1,076,394 records from observational studies and 1134 from randomized control trials. The authors reported no difference in 30-day mortality with either NA alone or in combination with GA compared to GA alone. Secondary outcomes had variable results: (1) Combined NA-GA compared to GA alone showed reduced odds of pulmonary complication, surgical site infection, blood transfusion, thromboembolic events, length of stay, and intensive care unit admission. However, the mixed group also had increased odds of myocardial infarction, although no difference in the odds of pneumonia or cardiac complication. (2) When NA alone was compared to GA, there were decreased odds of any pulmonary complication, surgical site infection, blood transfusion, thromboembolic event, length of stay, and intensive care unit admission. There was no difference in the odds of cardiac complication, myocardial infarction, or pneumonia. The authors conclude that although there is no difference in mortality, NA may improve pulmonary outcomes and reduce resource use compared to GA.

NA has held an important place for decades in the management of surgical patients with purported advantages in many outcomes, often supported by small studies. However, the “Holy Grail” of a prospectively gathered database large enough to definitively answer important outcome questions has not been achieved. Many recent clinical trends have eroded into the use of NA, including the use of newer, more effective anticoagulants for thromboembolism prophylaxis, surgical wound, and joint infiltration with long-acting local anesthetics; popularity of truncal blocks (eg, transverse abdominis plane or paravertebral block) rather than neuraxial injection for pain management; and inability to definitively show significant positive effects on mortality and other rare outcomes. Despite this, most practitioners and surgeons continue to feel that NA plays an important role in the management of selected surgical patients; and supportive studies, especially those with large numbers, are still viewed positively in the literature. In spite of this, perhaps because of this, the authors of this editorial question: (1) whether the initial question posed was too broad; (2) whether the evidence presented was properly synthesized; and (3) whether the included meta-analysis was appropriately assembled.

Controversial topics with a large volume of literature lend themselves well to synthesis through systematic review and, if the amount of data allows, meta-analysis.2 A systematic review, unlike a traditional narrative review, possesses razor-focused eligibility criteria defined by populations, interventions, comparisons, and outcomes. In addition, they should be designed to address a relevant and original clinical question. Reviews of comparative effectiveness regarding the choice for NA, like Smith et al1 may no longer be novel, and critics may argue that continued retrospective investigations are adding limited new information to our understanding of this complex issue.3

The methodology for systematic review is a complex exercise that requires significant expertise. Critical reviewers need only to know the “secret” of this study design boils down to one thing—to what extent a reader has confidence in the study design and assurance in the findings.2 Two fundamental problems can undermine this trust. The first is the extent to which the authors of the systematic review have applied methods that establish credibility, and the second is readers’ comfort with the estimates of effect. Credibility is the extent to which the design and conduct of the review (protocol) is likely to have protected against misleading results. Given this, clinicians should consider certain limitations in this report by Smith et al.1

Primarily, we are concerned that the population included in this systematic review includes multiple surgery types (eg, major truncal and lower limb surgery) with disparate perioperative treatment goals. Not unlike the highly cited and highly critiqued article by Rodgers et al,4 a previous landmark paper on reduction of postoperative mortality and morbidity with NA, Smith et al1 included an extremely wide range of surgical patients. By including multiple surgeries with competing perioperative treatment goals, this review also resonates as too broad. For instance, we question whether it is valid to compare an elective total hip arthroplasty with an urgent hip fracture repair or lump major intraperitoneal surgery that combines both GA with NA with surgical procedures using NA alone. We believe this could confound the selection of any individualized anesthesia care plan, introducing far too much practice variation by provider into the data available for analysis.

Next, it is quite possible that the large administrative claim database studies included within this review have overlapping patients (among each other), which violates statistical independence. Although we appreciate the attempts by Smith et al1 to limit such possible patient overlap from these large patient repositories, we still fail to see the added value of including these monstrous studies within their meta-analysis. Even with “extra” methodologic measures described in the article, there remains the potential for patient overlap and “double counting” with other single-center studies that make up these databases. These doubts of validity may have been prevented if this article had restricted its meta-analysis to focus on randomized clinical trials or single-center prospective observational studies. The choice of Smith et al1 to include these multiple large-scale administrative database studies was purposeful to increase overall sample size, yet, by limiting to studies of >500, the authors failed to include other single-center observational studies provided in other reviews on this topic.5

Moreover, we disagree with the authors’ desire to include large volumes of patients within their meta-analysis from these multiple large-scale administrative database studies in order to increase precision. Analyzing these large database studies along with more discrete trials makes the meta-analysis redundant. Individually, these very large studies possess sufficiently high power to stand alone, and in fact, pooling or aggregating these with more precise single-center prospective studies does little to inform on the single best effect estimate. One must only look at the inconsistency in results (heterogeneity) appearing within the many forest plots of the paper to find support for our concerns. These large-scale repository studies represent more of the weight (from the weighted averages) of the meta-analysis and more markedly influence the width of the effect estimate within the confidence intervals for many outcomes. What would the results look like without the administrative database studies included within the meta-analysis? Unfortunately, we are unable to comment as the article did not exclude these stand-alone repositories within a specific sensitivity analysis. For all the reasons above, we conclude the credibility of this reviews' methods are low, further reducing our confidence in their findings.6

It is clear that the authors are also concerned with the quality of their results because they concluded that, “due to the confounders inherent in observational studies … results should be interpreted with caution.” It is not unreasonable to seek an answer to the question, “is the use of neuraxial anesthesia alone or when combined with general anesthesia associated with superior perioperative outcomes when compared to general anesthesia for major surgery of the trunk and lower extremity.” Unfortunately, the overall scope is simply too broad to provide meaningful results. Despite, many researchers’ attempts to answer our Holy Grail question, it does not seem that we are any closer to an answer today than Rodgers et al4 were in 2000. We recognize that data obtained from large databases have made a significant impact on our current knowledge, although due to the inherent limitations and the inability to perform the massive randomized clinical trial required, we continue searching for a definitive answer regarding the safest or even ideal anesthetic. Perhaps we are asking the wrong question and should get away from mortality as an outcome, and instead move toward more discrete and smaller-scale, patient-centric outcomes.

Although we encourage researchers to continue seeking information to help identify the best anesthetic for our patients, we believe that it is time to say “enough” with study designs that do not provide the data needed to make a conclusion but instead may ultimately confuse the issue rather than clarify it.

Back to Top | Article Outline


Name: Rebecca L. Johnson, MD.

Contribution: This author helped conceive the editorial, composed the body, and approved the final version of the manuscript.

Name: Denise J. Wedel, MD.

Contribution: This author helped conceive the editorial, composed the introduction, and approved the final version of the manuscript.

Name: Sandra L. Kopp, MD.

Contribution: This author helped conceive the editorial, composed the discussion and approved the final version of the manuscript.

This manuscript was handled by: Richard C. Prielipp, MD.

Back to Top | Article Outline


1. Smith LM, Cozowicz C, Uda Y, Memtsoudis S, Barrington MJNeuraxial and combined neuraxial/general anesthesia compared to general anesthesia for major truncal and lower limb surgery: a systematic review and meta-analysis. Anesth Analg. 2017;125:1931–1947.
2. Murad MH, Montori VM, Ioannidis JP, et al.How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA. 2014;312:171–179.
3. Johnson RL, Habermann EB, Horlocker TTWaiting to exhale: neuraxial anesthesia in patients with chronic obstructive pulmonary disease. Anesth Analg. 2015;120:1189–1191.
4. Rodgers A, Walker N, Schug S, et al.Reduction of postoperative mortality and morbidity with epidural or spinal anaesthesia: results from overview of randomised trials. BMJ. 2000;321:1493.
5. Kopp SL, Berbari EF, Osmon DR, et al.The impact of anesthetic management on surgical site infections in patients undergoing total knee or total hip arthroplasty. Anesth Analg. 2015;121:1215–1221.
6. Guyatt GH, Oxman AD, Kunz R, et alGRADE Working Group. Incorporating considerations of resources use into grading recommendations. BMJ. 2008;336:1170–1173.
Copyright © 2017 International Anesthesia Research Society