Letters to the Editor: Letters & Announcements
I agree with many of the comments included in the 3 Letters to the Editor describing concerns about the article by Monk et al. and my editorial response. The authors of the letters raise some very important issues regarding the quality of the manuscript, the appropriateness of publishing the article, and, perhaps most importantly, about the editorial review process.
First, the study design was flawed, as noted in each letter. The conclusions drawn in the manuscript regarding the predictive value of the cumulative deep hypnotic time are based on a statistical manipulation of data that were collected for a different purpose. Although the manuscript does acknowledge the fact that the analysis was a post hoc analysis from a study that was designed for a different purpose, the actual purpose of the study and the nature of this analysis might have been more openly acknowledged.
With respect to the validity of the conclusions of the study, it is true that this kind of analysis will identify associations that warrant further investigation. My editorial emphasized the need to validate the findings from this study before accepting them as a guide to clinical management. In addition, because the findings of this study are in some ways contradictory to the findings from other studies regarding anesthetic depth and outcome, the need for more rigorous investigations that take into account comorbidities and have better controls over both the study design and analysis are essential.
The second important issue raised in these letters is the appropriateness of publishing the manuscript at all. The letters raise a much more important issue, not only in regard to this study but also to the process by which professional journals have an obligation to disseminate peer-reviewed information that might impact clinical management. Although the study was not as well-designed as it should have been, some interesting concerns about patient management were raised that warrant dissemination, debate, and further study. The flaws that are noted in my editorial and in the Letters to the Editor are accurate. On the other hand, the current interest in the role of the Bispectral Index monitor on patient outcomes and concerns about patient safety mandate that we do our best to disseminate information that might affect outcomes. This does not mean that the information should not be critically evaluated. For example, would the current concerns about the cardiac risk associated with the administration of COX-2 inhibitors have been more addressed more rapidly if preliminary data were more widely disseminated? Does a manuscript like the one submitted by Monk et al. deserve publication despite its flaws in order to foster debate about a very important and timely issue? I think the critical review of this manuscript and the decision by the editor of the Journal to publish it have created exactly the kind of discussion and debate that will allow the anesthesia community the opportunity to determine whether there is any clinical value to the cumulative deep hypnotic time and define how it might—or more likely might not—alter patient management and outcome. Mission accomplished?
Finally, Drummond and Patel propose a very laborious method to confirm that the design of a study is appropriate and that the statistical analysis is valid. Their comments reinforce the fact that manuscripts defining a study and its outcome are complicated and analysis sophisticated. Because of these complexities, the editorial process is also complex and susceptible to errors. The editorial process used by Anesthesia & Analgesia (and most other professional journals) is designed to provide a thorough review of a manuscript, to assess its validity, and to provide assurances, as best it can, that subjects have been treated in an ethically sound manner and the research and statistical analyses are appropriate. Although the process is not perfect, it does attempt to address these issues as thoroughly as possible. At the same time, the authors are asked to provide assurances that they have fulfilled their obligations with respect to proper research design, that the manuscript accurately reflects the study design and data analysis, and that the authors have each participated in the design and completion of the project and preparation of the manuscript. It is unrealistic to think that the editorial board should question the ethics of the authors, should second guess their motives, or should expect more extensive validation that the manuscript reflects the work that was actually performed. The system should be reassessed, but it should also be recognized that the peer review process is just that.
Neal Cohen, MD
University of California, San Francisco
San Francisco, CA