We thank Drs. Levy and Berry for their interest in our article and for their insightful comments. They both express concern that the association between low Bispectral Index values and 1-yr postoperative mortality may be overinterpreted as evidence for a causal relationship between inappropriate anesthesia care and mortality. Let us state at the outset that we agree that such a conclusion would not be appropriate. We hope that our investigation serves as an impetus for further hypothesis generation and additional prospective research.
Both Dr. Levy and Dr. Berry offer hypotheses to provide potential explanations for our findings. Dr. Levy cites the imprecise nature of the Charlson Comorbidity Index used in our covariate statistical model; whereas Dr. Berry postulates that low Bispectral Index values were potentially a marker for patients who were more likely to die.
We agree that the Charlson Comorbidity Index, by virtue of being an ordinal measure, is an imperfect metric of patient comorbidity. There are undoubtedly elements of comorbidity that are not captured in the Charlson score, and these residual, unmeasured elements may spuriously confer significance on measures such as cumulative deep hypnotic time or blood pressure. This is unfortunately one major reason for the limited inferences one can make from any observational study. Our dichotomization of the Charlson Index score itself, however, is probably not an explanation for our findings. We dichotomized the Charlson score to help ensure the stability of the model and to reduce the degrees of freedom and thus the risk of overfitting. There are many Charlson categories and relatively few deaths, and the modeling algorithm requires that each category/outcome combination have an adequate number of occurrences to ensure model convergence and an accurate estimation of the relative risks. We described this in the Methods section. However, when we run the multivariate Cox regression using the full Charlson score, we see essentially identical results (Table 1). Although another significant variable was added (history of previous myocardial infarction), the relative risk attributable to cumulative deep hypnotic time increases from 24.4%/h (in the published model) to 29.9%/h and becomes more statistically significant, increasing from P = 0.0121 to P = 0.0021, respectively. The relative risk and significance of systolic hypotension remains approximately the same. It should be noted that the significance of cumulative deep hypnotic time increased despite the increase of the degrees of freedom of the Charlson score from 1 to 10, providing ample opportunity for overfitting to the Charlson score. Thus the dichotomization of the Charlson score may actually have obscured some of the effect of cumulative deep hypnotic time.
We do not dispute that Dr. Berry’s interpretation is possible. He postulates that “the patients who were more susceptible to general anesthesia, as measured by a lower Bispectral Index value, were more likely to die within the year after surgery.” We are unable to evaluate this, as we do not have intraoperative anesthetic agent data to determine if the correlation between anesthetic dose and Bispectral Index values in the patients who died was different from that in the rest of the of the population. Dr. Berry nevertheless raises the interesting notion that “susceptibility to general anesthesia” might actually be a “marker” for increased risk of death. This would be important to study, but we agree with him that additional evidence is required before a causal relationship between deep hypnotic time and mortality can be established.
Dr. Drummond raises far more serious objections to our paper. Since the initial abstract presentation, a much more complex and comprehensive data analysis was performed with the intent to ensure the most accurate results. We believe that our findings are quite robust. Typical concerns in this setting are that the findings are a snapshot of noise or that the number of data points is too few for the number of variables examined. Precisely because of such concerns, we performed an analysis of the variable selection frequencies using a bootstrap analysis. The variables in the final multivariate model were also those most frequently selected in an analysis of 500 random samples. Those results indicate a fairly stable association between hypnotic depth and mortality. Based on these considerations, we believe our findings are valid, i.e., that they represent something real in the data and are not likely attributable to the play of chance in a particular sample. As Dr. Cohen points out in his editorial, “the findings in this study are indirectly supported by other recent studies” (1) including 5057 patients and up to 2 yr of followup.
Many observational studies use analysis methods that were arrived at after the data was collected; e.g., the Framingham Heart Study. This objection would have greater merit if we were engaging in post hoc analysis of a randomized trial in which the critical endpoint for which randomization was performed was being abandoned or if subgroup analysis was being performed inappropriately. As we stated in our publication, an association between depth and mortality was an unexpected finding and clearly not one we were testing prospectively. It would seem that not accepting for publication studies using post-data collection-determined statistical methods would rule out publication of many hypothesis-generating studies to the detriment of the advancement of scientific knowledge.
Dr. Drummond raises the important issue of conflict of interest. This concern undoubtedly is driven by the bias, whether conscious or unconscious, that a significant conflict of interest can generate. We agree that this must always be a great concern in science, but we strongly disagree with his premise that the participation of one author in this study “inevitably and irrevocably taints the credibility of the results.” Such an approach would reject the important scientific contributions made by numerous medical researchers in industry and fails to acknowledge the impact of other potential sources of conflict of interest involved in the publication and debate of scientific investigations (e.g., academic competition, political agendas). We believe that the quality of the science is paramount as long as all potential conflicts of interest are disclosed, as they were in our article (and also discussed in the accompanying editorial). In addition, we had the data analyzed by an independent statistician at Duke University Medical Center before the submission of the manuscript, and the results of this second analysis were in agreement with the findings reported in the study.
Our study was not designed to address the “paradox” that concerns Dr. Drummond. We believe that the take-home message from our work is that perioperative anesthetic management may influence long-term patient outcomes. This highlights the critical role that anesthesiologists play in ensuring optimum patient safety. There are clearly many possibilities for further practice improvement that will need to be studied in further trials.
Terri G. Monk, MD, MS
B. Craig Weldon, MD
Department of Anesthesiology
Duke University Medical Center
Vikas Saini, MD
The Cardiovascular Specialists LLC
Jeffrey C. Sigl, PhD
Aspect Medical Systems