We thank Dr. Raghunathan et al.
for their letter regarding their disappointment that we did not publish a cautionary editorial regarding the reports by Wilder et al.1
and Kalkman et al.2
in the April issue of Anesthesiology. These clinical articles, which were published with laboratory work presented at the Anesthesiology/Foundation for Anesthesia Education and Research session at the 2008 Annual Meeting of the American Society of Anesthesiologists, were accompanied by an editorial by Drs. Patel and Sun,3
thought leaders in research regarding the mechanisms and clinical relevance of neurodevelopment after exposure to anesthetics. Regarding the clinical article, they concluded in their editorial, “Although two retrospective studies herein suggest that a correlation between anesthetic exposure early in life is associated with learning and behavioral abnormalities later in life, the data cannot be considered to be evidence of the existence of anesthetic neurotoxicity in humans. The absence of rigorously conducted prospective randomized trials precludes recommendations on clinical practice.”3
In our opinion, this statement expresses appropriate and adequate express caution regarding the application of these data to clinical practice.
We would also like to comment regarding the statistical analysis and presentation in the article by Kalkman et al.
as questioned by Dr. Raghunathan et al.
Their study focused on effect sizes and not on statistical significance judgments. This was a prudent choice because of the pilot nature and goals of the effort. This clear focus on effect sizes is made abundantly clear by the fact that in the article by Kalkman et al.2
there is not a single P
value reported. Instead, Kalkman et al.2
referenced the size of the observed effects throughout. For a properly powered study, making a claim about an effect that is not statistically significant is, indeed, anathema. However, in this clearly defined pilot study, reminding a reader than an observed effect size did not reach statistical significance is actually a responsible practice. The uncovered effect sizes in a pilot study are estimates of their population values, but as Kalkman et al.2
overtly stated, these estimates are in the context of very wide confidence intervals.
We strongly believe that there is a place for small n research in Anesthesiology. Small n research is tricky to report. We have a sophisticated community of researchers (mostly bench scientists) who successfully add to our knowledge base while using studies that are not optimally powered. Again, this reinforces the importance of clear effect size reporting (as in the two mentioned studies), a priori power analyses to overtly report assumptions, and exact P value reporting to arm a reader with enough information to properly interpret experimental effects.
Regarding their statement on post hoc
power analyses, Raghunathan et al.
are wise to be concerned about power calculations that are based on observed P
values. We agree with this sentiment, articulately voiced by Hoenig and Heisey,4
and for that reason actively discourage such power calculations. The provided power calculation, though, was clearly presented as the primary aim of the study, and posits that the observed risks are the population values, and to reject a null hypothesis of no added risk (under a traditional set of inference assumptions), a future prospective study would need to study 2,268 children (thus making it similar to power analyses conducted throughout the research world; this one is simply in print). There is a difference between stating “These differences would be statistically significant with n patients” versus
“If these differences are population values, we need n patients to reject a null hypothesis in our next study.” In that regard, Kalkman et al.2
have succeeded in providing a context for interpreting their study.
Timothy T. Houle, Ph.D.,*
James C. Eisenach, M.D.†
*Statistical Editor, Anesthesiology, Wake Forest University School of Medicine, Winston-Salem, North Carolina.email@example.com. †Editor-In-Chief, Anesthesiology, Wake Forest University School of Medicine.
1. Wilder RT, Flick RP, Sprung J, Slavica K, Barbaresi WJ, Mickelson C, Gleich SJ, Schroeder DR, Weaver AL, Warner DO: Early exposure to anesthesia and learning disabilities in a population-based birth cohort. Anesthesiology 2009; 110:796–804
2. Kalkman CJ, Peelen L, Moons KG, Veenhuizen M, Bruens M, Sinnema G, de Jong TP: Behavior and development in children and age at the time of first anesthetic exposure. Anesthesiology 2009; 110:805–12
3. Patel P, Sun L: Update on neonatal anesthetic toxicity: Insight into molecular mechanisms and relevance to humans. Anesthesiology 2009; 110:703–8
4. Hoenig JM, Heisey DM: The abuse of power: The pervasive fallacy of power calculations for data analysis. Am Stat 2001; 55:19–24
© 2009 American Society of Anesthesiologists, Inc.