Secondary Logo

Journal Logo

Is a Formal Statistical Analysis Always Necessary?

Woehlck, Harvey, MD

doi: 10.1213/ANE.0b013e318214f90e
Editorials: Editorials
Free

From the Department of Anesthesiology, Medical College of Wisconsin, Milwaukee, Wisconsin.

Funding: Departmental funds.

The author declares no conflict of interest.

Address correspondence to Harvey Woehlck, MD, Medical College of Wisconsin, 9200 W. Wisconsin Ave., Milwaukee, WI 53226. Address e-mail to hwoehlck@mcw.edu.

Accepted February 7, 2011

Malignant hyperthermia (MH) susceptibility in a patient may not be known in time to properly prepare a machine by established routines. Modern anesthesia machines may not allow vaporizer removal, and it is theoretically possible that the vaporizers may leak trace amounts of volatile anesthetics or accidentally be turned on. Modern machines also contain varying amounts of plastic internal components that may “off-gas” (slowly release) volatile anesthetics for a clinically long time. To make matters worse, no practical method exists for an anesthesiologist to verify that trace amounts of residual anesthetic are not being delivered, because most clinical gas monitors cannot display <0.1% (1000 parts per million [ppm]) of a volatile anesthetic. Because established routines may no longer apply to the modern practice of anesthesia in this regard, if one had a device that could guarantee that these trace quantities of anesthetic could not reach the MH-susceptible patient, we would be relieved. Activated charcoal has been used since the 18th century for the adsorption of gases and other substances. In this issue of Anesthesia & Analgesia, Birgenheier et al.1 present a study that shows the remarkable efficacy of activated charcoal filters in the breathing circuit on reducing residual volatile anesthetic concentrations to miniscule values.

Or do they? The careful reader will immediately notice that each control and experimental condition was performed with an n = 1. Although the origin is obscure, it has been said that a study with an n of 1 has all the scientific validity of a miracle. Just what can be proven with this statistical analysis, or more correctly, lack thereof? Is this bad science? Does this set back scientific methodology by hundreds of years?

Modern statistical analyses now utilize complex algorithms that may have required hours or days of handwritten calculations prior to the advent of personal computers. It is not only common practice, but nearly an expectation, that scientific studies will have a valid statistical analysis. Today, we can ask many things of statistical analyses. In this case, we might pose a null hypothesis that the difference between residual anesthetic concentrations with and without charcoal filters could be due to chance sampling and random variability. In other words, inferential statistical analyses can be null hypothesis significance tests.2,3 We can ask for frequentist statistical analyses and validate a claim with P values and confidence intervals for an effect. Conversely, similar problems could be approached with Bayesian inferences and ask for the probability that a test result is a false positive given test parameters and an incidence rate for a disease.

But even though statistical analyses constitute a basic tenet of scientific exploration in fields beyond this study of charcoal filters, we should also carefully examine when statistics are truly useful or required. Are all prospective studies with <80% power scientifically unsound?4 If we decree that a rigorous statistical analysis is always necessary, might there be a price to pay for an adequate sample size? Will the statistical burden harm the research process by inhibiting innovation, as well as wasting effort and money?4 If living subjects are involved in a study, what ethical dilemmas may result when large numbers of control group patients are subjected to ineffective treatments merely to bolster a P value? Does statistical significance mean practical or “plain-English significance?”2 Examples of questionable statistical relevance can be even more extreme and dramatic: if the basis for parachute use is purely observational, is it unscientific to use a parachute because no randomized controlled trial has ever shown that parachutes are more effective than a free-fall landing?5 How many discrete and independent observations are needed to prove that the “Earth is round (P < 0.05)”?2

Although the above may appear to progress into a sarcastic rant, it should encourage us to critically examine our procedures so that common sense prevails. In this day and age, how can we accept a study with a sample size of 1? Even though the results make intuitive sense, should we discount them because of an inadequate sample size for statistical analyses? In this case, would we be performing statistics merely for statistics' sake? Just as everyone who understands global positioning satellites has a priori knowledge that the earth is “round,” we also have extensive a priori knowledge that charcoal filters will remove most volatile organic compounds (VOCs) until the filters become saturated. Let's be clear: The adsorption of VOCs, including volatile anesthetics, by activated charcoal is not an interesting and current scientific problem. Hundreds of industrial processes use adsorption by activated charcoal, and we need look no further than the evaporative emissions control system of our own automobile to see the process in use. One does not need to delve into epistemological trivia to know this. With modern manufacturing techniques, it is probable that there is little variability between charcoal filters, and manufacturers may provide the end user with the appropriate data with a descriptive statistical analysis on request. Similarly, we also know that the authors' measurement technique is also subject to clinically insignificant variability. If the lack of statistics is personally troubling, consider this study1 to be analogous to an in vitro case report, merely providing one example of how effective the process may be under presumably ideal situations. Better yet, consider the study to be a sensitivity analysis of conditions that produce variability in the function of charcoal filters. Activated charcoal filters can, and probably will, reduce residual concentrations of volatile anesthetics in modern plastic-laden machines below an easily measurable concentration, and almost certainly below a concentration that can trigger MH. If we have concerns about residual volatile anesthetics, charcoal filters serve our needs at the present time. Regardless of the statistical significance or lack of it reported by the authors in this study, common sense dictates that charcoal filters will be a useful adjunct to established prophylactic or treatment measures for MH-susceptible patients.

Instead, we should focus our efforts on meaningful problems. What represents something potentially interesting is that even though the authors chose a default value of 5 ppm of anesthetic agent as a triggering threshold concentration, we do not yet actually know the triggering concentration for each volatile anesthetic agent. These values are probably different for each genetic mutation that can result in MH susceptibility. Unfortunately, that study should never be performed in humans, and probably cannot be performed cost effectively with today's technology in an animal model. And, of course, one must remember that the treatment for an established episode of MH is not a charcoal filter, but the rapid administration of dantrolene combined with appropriate physiologic support.

Back to Top | Article Outline

DISCLOSURES

Name: Harvey Woehlck, MD.

Contribution: This is the sole author of the manuscript.

Attestation: Harvey Woehlck approved the final manuscript.

Back to Top | Article Outline

REFERENCES

1. Birgenheier N, Stoker R, Westenskow D, Orr J. Activated charcoal effectively removes inhaled anesthetics from modern anesthesia machines. Anesth Analg 2011;112:1363–70
2. Cohen J. The Earth is round (p < .05). Am Psychol 1994;49: 997–1003
3. Rodgers JL. The epistemology of mathematical and statistical modeling. Am Psychol 2010;65:1–12
4. Bacchetti P. Current sample size conventions: Flaws, harms, and alternatives. BMC Med 2010;8:17
5. Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ 2003;327:1459–61
© 2011 International Anesthesia Research Society