Multimodality neuromonitoring has become commonplace in tertiary care neurocritical care units. Indeed, the development of the discipline of neurocritical care medicine has, in part, been associated with the increasingly complex nature of this monitoring . Although advances in neuromonitoring have provided insight into the pathophysiology and physiological response to therapy, beneficial effects on important patient outcomes have not been definitively established. The randomized controlled clinical trial with subsequent systematic review showing consistency of results is the methodological gold standard for the evaluation of efficacy . However, with the realization that a monitor is not therapeutic itself, such an evaluation must necessarily take into account the performance characteristics of the monitor, the clinician's ability to interpret monitoring data and the therapeutic intervention chosen based on the monitoring data. This article will discuss the challenges related to the demonstration of the efficacy of neuromonitoring. In addition, this article will address effectiveness, as it is arguably more relevant, in the context of the developing field of neurocritical care.
Assuming the monitor's performance characteristics, including accuracy and stability, are known prior to national regulatory approval for clinical use, the clinician's knowledge of these performance characteristics and the ability to interpret the data derived from a monitor is paramount to the demonstration of efficacy. If the pulmonary artery catheter is used as an example, the evidence suggests clinicians have difficulty in interpreting this complex invasive monitor. Gnaegi and colleagues  demonstrated a substantial proportion of fully trained intensivists were unable to correctly determine the pulmonary artery occlusion pressure from a pressure trace with clearly identified end-expiration. Importantly, many clinicians were unable to identify important safety considerations, such as positioning in a systemic artery at the time of catheter insertion. Others have found similar results [4,5]. Full evaluation of the interpretive ability of neurointensivists with respect to multimodality monitoring has not been undertaken.
Some monitors in neurocritical care such as tissue oxygenation produce a number without waveform analysis, while others such as continuous electroencephalogram (EEG) rely purely on waveform analysis. Although the interpretation of data from both types of monitor requires an understanding of the underlying physiology and performance characteristics of the monitor, waveform analysis requires specialized knowledge and is associated with additional complexity, subjectivity and potential for inter- and intra-observer variability. For example, the evaluation of experts has revealed considerable variability in EEG interpretation . Furthermore, Leira and colleagues  found recognition of epileptifom discharges by bedside caregivers was ‘disturbingly low'. A simple educational intervention only modestly improved such ability. They suggested, with the exception of EEG technicians, we should not rely on non-trained personnel to interpret emergent or continuous EEGs . Similarly, it would be plausible to suggest that expert interpretation of intracranial pressure (ICP) waveform analysis  would provide additional information that could benefit patient management above the numerical value of ICP but would potentially be associated with interpretative error when used by non-experts. Obviously, monitoring devices have no chance of favourably influencing outcome if used incompetently. Therefore, the first step to demonstrate efficacy of monitoring is to ensure adequate training in interpretation and include ongoing surveillance for proficiency.
A monitor cannot affect patient outcome unless it identifies a preclinical abnormality and prompts timely and appropriate therapeutic intervention. This abnormality must be an integral determinant of patient outcome. For example, intracranial hypertension cannot be reliably detected by clinical examination, is amenable to both medical and surgical therapy and has been associated with unfavourable neurological outcome. There is suggestion in the literature that patients with traumatic brain injury cared for in centres providing ICP monitoring in compliance with Brain Trauma Foundation have better outcomes . Furthermore, there are numerous single-centre studies suggesting protocolized management, including ICP treatment, in traumatic brain injury improves outcome [10-14]. However, the protocols employed differ greatly, making it unclear whether ICP management truly improves outcome. Some suggest monitoring merely increases the interaction of patient and healthcare provider, which results in better outcomes.
A key limitation in the demonstration of efficacy of monitoring in neurocritical care is the complexity of care generated by multimodality monitoring. If one considers continuously monitoring 10-20 interrelated physiological parameters in a modern neurocritical care unit and each parameter has 10 possible interventions, the enormous potential number of co-interventions represents a formidable challenge in clinical trial design. Given the little gold standard outcome evidence for any one intervention in neurocritical care, one would expect to see a considerable variability in the management of these patients from one clinician to another and from one centre to another. This is indeed the case even when strict protocols are in place . This may reduce trial sensitivity. Ultimately, trials of monitoring efficacy are tests of the therapeutic regimen employed as a result of the monitoring data. Negative results of such studies imply an ineffective regimen and one cannot conclude a monitor is not efficacious until all reasonable therapeutic regimens have been tested.
Realistic expectations in clinical trial design
New therapies are tested against standards of care that have developed over hundreds of years. Therefore it is implausible to expect to see dramatic improvements in the outcome from monitors, especially when used to ‘fine-tune' existing therapies and no new therapies are employed. For example, is it realistic to expect a monitor developed to guide the adequacy of resuscitation in septic shock in a protocol that includes adequate antibiotic therapy to have the same effect on outcome as the implementation of antibiotic therapy did decades ago? Considering a condition in which the mortality is 30%, a trial designed to detect a 10% absolute reduction in death will require 824 patients (α = 0.05, β = 0.10). However, this represents a 33% relative reduction in mortality, which is likely unrealistic for a monitor used to guide existing therapies. Perhaps a more realistic expectation may be a 1% absolute risk reduction. However, this would require almost 90 000 patients. These numbers are daunting to funding agencies and challenge the most dedicated investigator. However, this feat is not impossible. Critical care research networks such as the Canadian Critical Care Trials Group and the Australian and New Zealand Intensive Care Society Clinical Trials Group have developed to collaborate on large-scale clinical studies.
Efficacy or effectiveness?
Efficacy refers to whether controlled trials show a treatment effect. Effectiveness refers to whether the treatment transfers well to real-world populations. The issue can be thought of as involving internal and external validity: Efficacy shows that internal validity is present, but effectiveness is needed to show external validity or generalizability. Effectiveness studies are pragmatic studies and generally do not rely on treatment directed by protocol. An example of an efficacy study of pulmonary artery catheter use was the trial performed in high-risk surgical patients , in which patients were treated with a strict protocol to optimize haemodynamics and oxygen delivery guided by the pulmonary artery catheter and compared with management without the pulmonary catheter. An example of an efficacy study is the PAC-Man (Pulmonary Artery Catheters in patient Management) study where patients were randomized to receive a pulmonary artery or not and subsequent management in both groups was left to the discretion of the clinicians. Neither study showed a clear benefit related to the use of pulmonary artery catheter use. Can we conclude that pulmonary artery catheter use is not beneficial? No. In the former study, a plausible explanation for the negative result is the mandated therapeutic regimen was not beneficial or a portion was beneficial but balanced by detrimental aspects. The latter study may have been subject to a greater degree of interpretive and therapeutic variability such that a small beneficial effect may have been negated. Therefore, the possibility exists that pulmonary artery catheter use does result in benefit, but limitations in trial methodology have prevented the demonstration of this effect. It is also possible that the detailed data on haemodynamics provided by the pulmonary artery catheter cannot modify the disease process sufficiently to influence disease outcome.
A key consideration in proposing an efficacy design is the aforementioned sample size. Ideally, investigators participating in a monitor study would be highly trained and proficient in interpretation of the monitor of interest and management of the condition of interest. Furthermore, it would be performed in a tightly controlled environment such as a single neurocritical care unit. However, most major neurocritical care centres will see 50-100 suitable patients per year of the main neurocritical care conditions (e.g. severe TBI, SAH, ICH). If one considers a 70% consent rate, for a study requiring even 1000 patients to be completed in less than 3 yr, at least five centres are required. This introduces intra-centre variability into the analysis. Furthermore, as many non-tertiary centres do not have specialized neurocritical care, if these centres are included in an effort to speed recruitment, one must consider the introduction of interpretive variability of the monitor at sites with infrequent use. If these centres are not included and the study duration is prolonged, the changes in standards of care for co-interventions must be factored into the analysis.
With respect to the introduction of new monitoring technology, it must be realized that it is difficult to conduct high-quality studies early in the life of a new technology and that ignoring a large body of literature suggesting a technology is beneficial may unethically deprive the public of access to that technology. Furthermore, without the implementation of technology, large-scale clinical trials cannot be completed. Instead of the gold standard systematic review of randomized controlled trials for the assessment of efficacy, a health technology assessment approach may be a more reasonable evaluation of monitoring devices. This can be accomplished even when supporting evidence is suboptimal . The key differences relate to methodological standards, repetition of previous studies, breadth vs. depth, inclusion of content experts and policy-makers, performance of economic evaluations, making policy recommendations and active dissemination. With such a health technology assessment, suggesting some patients may benefit from monitoring, this will allow introduction and develop subsequent clinical experience that will ultimately allow efficacy studies to be appropriately conducted.
1. Suarez JI. Outcome in neurocritical care: advances in monitoring and treatment and effect of a specialized neurocritical care team. Crit Care Med
2. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA
3. Gnaegi A, Feihl F, Perret C. Intensive care physicians' insufficient knowledge of right-heart catheterization at the bedside: time to act? Crit Care Med
4. Iberti TJ, Daily EK, Leibowitz AB, Schecter CB, Fischer EP, Silverstein JH. Assessment of critical care
nurses' knowledge of the pulmonary artery catheter. The Pulmonary Artery Catheter Study Group. Crit Care Med
5. Iberti TJ, Fischer EP, Leibowitz AB, Panacek EA, Silverstein JH, Albertson TE. A multicenter study of physicians' knowledge of the pulmonary artery catheter. Pulmonary Artery Catheter Study Group. JAMA
6. Williams GW, Luders HO, Brickner A, Goormastic M, Klass DW. Interobserver variability in EEG interpretation. Neurology
7. Leira EC, Bertrand ME, Hogan RE et al.
. Continuous or emergent EEG: can bedside caregivers recognize epileptiform discharges? Intensive Care Med
8. Guendling K, Smielewski P, Czosnyka M et al.
. Use of ICM+ software for on-line analysis of intracranial and arterial pressures in head-injured patients. Acta Neurochir Suppl (Wien)
9. Bulger EM, Nathens AB, Rivara FP, Moore M, MacKenzie EJ, Jurkovich GJ. Management of severe head injury: institutional variations in care and effect on outcome. Crit Care Med
10. Eker C, Asgeirsson B, Grande PO, Schalen W, Nordstrom CH. Improved outcome after severe head injury with a new therapy based on principles for brain volume regulation and preserved microcirculation. Crit Care Med
11. Fakhry SM, Trask AL, Waller MA, Watts DD. Management of brain-injured patients by an evidence-based medicine protocol improves outcomes and decreases hospital charges. J Trauma
: 492-499; discussion 499-500.
12. Patel HC, Menon DK, Tebbs S, Hawker R, Hutchinson PJ, Kirkpatrick PJ. Specialist neurocritical care and outcome from head injury. Intensive Care Med
13. Rosner MJ, Rosner SD, Johnson AH. Cerebral perfusion pressure: management protocol and clinical results. J Neurosurg
14. Ng I, Lew TW, Yeo TT et al.
. Outcome of patients with traumatic brain injury managed on a standardised head injury protocol. Ann Acad Med Singapore
15. Clifton GL, Choi SC, Miller ER et al.
. Intercenter variance in clinical trials of head trauma - experience of the National Acute Brain Injury Study: Hypothermia. J Neurosurg
16. Sandham JD, Hull RD, Brant RF et al.
. A randomized, controlled trial of the use of pulmonary-artery catheters in high-risk surgical patients. N Engl J Med
17. Rotstein D, Laupacis A. Differences between systematic reviews and health technology assessments: a trade-off between the ideals of scientific rigor and the realities of policy making. Int J Technol Assess Health Care