Monitoring depth of anaesthesia is an essential feature of daily anaesthetic practice. It serves several purposes, among them the prevention of awareness and the limitation of the amount of anaesthetic administered to the patient. Unfortunately, there is neither an undisputed definition of ‘depth of anaesthesia’ nor a gold standard of measuring it. For ether anaesthesia, the Guedel classification had been such a standard, but this is not valid for balanced or intravenous (i.v.) anaesthesia. Anaesthesiologists generally use a subtle combination of clinical signs to survey end-points of anaesthesia . Immobility, i.e. the absence of a motor response to a painful stimulus, is one of these clinical end-points, and the unresponsiveness to painful stimuli is used in the definition of minimal alveolar concentration . Nevertheless movement to a noxious stimulus has been shown to be a reaction which is generated on the spinal level [3,4]. Another desired feature of adequate anaesthesia is autonomic stability, i.e. the absence of a cardiovascular response to painful stimuli. This can be evaluated using the PRST score (pressure, heart rate (HR), sweating, tears) developed by Evans , but the PRST score is not appropriate for patients with high-dose opioid or β blocker therapy, or patients with haemodynamic instability .
In 1977 the isolated forearm technique was described for the first time by Tunstall as an indicator for patient awareness during Caesarean section . The disadvantage of this technique is the immobilization of the hand due to ischaemia of the arm more than 30 min after the inflation of the Tourniquet.
Hypnosis and prevention of explicit and implicit awareness are major aims of general anaesthesia and controversy exists about whether classical signs of depth of sedation and anaesthesia can be used to prevent awareness .
For this reason attempts have been made to measure depth of anaesthesia by monitoring the electroencephalogram (EEG) or evoked potentials. EEG monitoring has proved to be a useful measure of anaesthetic drug effect in pharmacodynamic research  and also in maintaining a constant level of sedation during operative procedures . Recently it has been shown that monitoring of the EEG using specialized monitors can reduce the occurrence of awareness .
Although equipment for monitoring central nervous system function in respect of ‘anaesthetic depth’ is available, (e.g. BIS, Entropy, evoked potentials) this technology has been adopted by only few anaesthesiologists . Most anaesthesiologists still rely on clinical signs of anaesthetic depth.
It is generally assumed that more experienced anaesthesiologists can assess anaesthetic depth better than inexperienced ones. To test the notion that assessment of anaesthetic depth can be learned by experience we compared the assessment of anaesthetic depth between experienced and inexperienced anaesthesiologists.
Since, as noted above, there is no gold standard for measuring the ‘true’ anaesthetic depth, we compared the subjective assessment with EEG indices as surrogate parameters of the underlying true anaesthetic depth. Since there is considerable heterogeneity in anaesthetic end-points, we left the definition of ‘anaesthetic depth’ the discretion of each anaesthesiologist.
We hypothesized that despite this heterogeneity in definition, assessment by more experienced anaesthesiologists would show a closer association with EEG indices of anaesthetic depth, even if the anaesthesiologists were blinded to the EEG monitor.
Following local Ethics Committee approval and written informed consent, 100 ASA I or II patients scheduled for elective surgery under general anaesthesia were enrolled in the study. Excluded were patients scheduled for surgery which might interfere with cerebral function (e.g. carotid endarterectomy, cardiopulmonary bypass, neurosurgery), neurological disease, and alcohol or drug abuse.
Participating anaesthesiologists (and cases assigned to them) were divided in two groups: ‘experienced’ anaesthesiologists with an experience in anaesthetic practice of more than 4 yr and ‘inexperienced’ anaesthesiologists with <2 yr experience. A prospective randomization of case assignments between experienced and inexperienced anaesthesiologists was not possible on a large scale in daily routine in the department. A pilot study revealed that a randomization of patients would have caused a bias in patient age and duration of the anaesthetic with more experienced anaesthesiologists being more often assigned to older patients and longer surgery. Therefore, cases in the two groups were prospectively matched to obtain equal group median for patient age and expected duration of anaesthesia.
Anaesthesiologists were allowed to perform anaesthesia at their discretion, no restrictions were made to the drugs used. However, due to the ‘standard operating procedures’ for anaesthesia at our department, total i.v. anaesthesia is the prevailing anaesthesia technique. Anaesthesiologists in the ‘inexperienced’ group were supervised by an attending physician. However, the attending physician refrained from interference with the study unless patient safety was considered threatened. In contrast, anaesthesiologists in the ‘experienced’ group managed the anaesthetic case of the study independently.
Prior to anaesthesia, self-adhesive electrodes for monitoring BIS (BIS XP; Aspect Medical Systems, Newton, MA) and Entropy (Datex-Ohmeda; Helsinki, Finland) were attached to the forehead of the patients.
Following the connection of standard monitoring (HR, non-invasive blood pressure, pulse oximetry) anaesthesia was induced with propofol and fentanyl.
Anaesthesiologists, who were blinded towards the EEG parameters, were asked to assess anaesthetic depth subjectively using an 11-point numeric scale (0: very deep anaesthesia; 10: patient awake) and a 5-point verbal scale (very deep, deep, light, almost awake, fully awake). Anaesthesiologists were given no clues how to define the scales or which anaesthetic end-points (e.g. hypnosis, analgesia, immobility) the term ‘anaesthetic depth’ should comprise.
Assessment was performed and recorded in intervals of 2 min, starting from 2 min after injection of propofol for induction until start of surgery. During surgery, assessment was performed every 10 min and when the anaesthesiologist considered a change in concentration necessary until the anaesthesiologist decided to reduce the concentration of anaesthetics at the end of surgery. From this point on, assessment was performed at 2-min intervals until the patient was considered ready for transfer to the recovery room by the anaesthesiologist. This way, the sampling of a roughly equal number of data points during steady-state anaesthesia and during induction and recovery was ensured. In addition to the subjective assessment of anaesthetic depth, anaesthesiologists were asked at each assessment whether they considered the anaesthetic depth at this time point as ‘appropriate’, ‘too deep’, or ‘too shallow’.
Recovery times were measured as the time from end of surgery (end of skin closure) until the patient was ready for transfer to the recovery room (Fig. 1). To be ready for transfer, the patient needed to be extubated (or the laryngeal mask removed) and to be able to follow simple commands.
Postoperatively (24–48 h after surgery), patients were interviewed for intraoperative awareness using two simple questions: ‘Do you remember anything in between anaesthesia induction and recovery from anaesthesia?’ and ‘Did you dream during the anaesthetic?’
BIS, and entropy of the EEG (SE, ‘state entropy’) as well as the spectral entropy of the combined signal of the EEG and the electromyogram of the frontalis muscle (RE, ‘response entropy’) were recorded using a S/5 Anaesthesia Monitor (Datex-Ohmeda) equipped with a BIS-module and a M-entropy module. Data were stored on a PC using S/5 software (Datex-Ohmeda).
The association between the subjective assessment and the EEG parameters was calculated using the prediction probability PK introduced by Smith , using a macro for EXCEL (Microsoft, Redmond, Washington, USA) kindly supplied by the author. Prediction probability is a non-parametric correlation measure, indicating the probability that a parameter (here: the subjective assessment) correctly predicts anaesthetic depth, i.e. in this case an electrophysiological indicator of anaesthetic depth. A PK value of 1.0 indicates perfect prediction, whereas a value of 0.5 indicates that the predictive value of the parameter is not better than chance alone. PK values were compared using the paired-data jackknife analysis, differences with P < 0.05 were considered significant.
All other calculations were made using standard statistical software (Prism 3.0, Graphpad Software, San Diego, CA). Values of P < 0.05 were considered significant.
There were 14 anaesthesiologists in the experienced group and 11 in the inexperienced group. Anaesthesiologists in the ‘inexperienced’ group had a mean experience in the operating theatre of 12 months (range 6–20 months). A total of 100 patients participated in the study, 50 patients in each group. The characteristics of the patients in both groups are shown in Table 1. No significant difference existed for age, ASA-status and duration of surgery or duration of anaesthesia (U-Test). The frequency of usage of anaesthetic drugs did also not differ between the two groups (Table 2, Fisher's exact test).
The association between the EEG parameters (BIS, SE and RE) and the subjective assessment of anaesthetic depth on the arbitrary 11-point numeric scale is shown in Figure 2. For comparison, the relation between BIS and entropy parameters is demonstrated in Figure 3. Linear regression of SE vs. BIS yielded a slope of 0.72 ± 0.01, a y-intercept of 11 ± 1.2 and a correlation coefficient, r2, of 0.60. For RE vs. BIS, linear regression yielded a slope of 0.85 ± 0.02, a y-intercept of 9.4 ± 1.4 and a correlation coefficient, r2, of 0.62.
The data plotted in Figure 2 were used to calculate the prediction probability PK for the subjective assessment of anaesthesiologists of the experienced and inexperienced group to predict BIS, SE and RE as surrogates of the underlying anaesthetic depth (Table 3). Judging from Figure 2 and the PK values, the association between the subjective assessment of anaesthetic depth and the EEG parameters tended to be better for experienced anaesthesiologists. However, a significant difference between the experienced and inexperienced group was found only for BIS (P < 0.05), where low values in the subjective assessment (meaning ‘deep’ anaesthesia) were associated with low BIS values in the experienced, but not in the inexperienced group (Fig. 2a).
To be better able to compare the assessment during anaesthesia, we then excluded all data during induction (until 10 min after start of induction) and recovery (from the point where the concentration of the anaesthetic was reduced). In addition, we excluded all data points where anaesthetic depth was considered either ‘too deep’ or ‘too shallow’ by the anaesthesiologist. The remaining data are shown in Figure 4 for the verbal scale of anaesthetic depth assessment. Intraoperatively, the assessment values ‘awake’ and ‘almost awake’ did not occur.
For each of the assessment values, ‘very deep’, ‘deep’ and ‘light’, there was a wide scatter in the EEG parameters. However, in the experienced group, a clear trend of the median values to increase with decreasing ‘depth of anaesthesia’ was visible. This was not the case in the inexperienced group.
Surprisingly, a rather large percentage of the data points at the assessment ‘deep’ or ‘very deep’ showed BIS values above the recommended value of 60  (13.2% in the experienced and 34.3% in the inexperienced group) (Fig. 5).
The recovery time, i.e. the time from end of surgery until the patient was ready for transfer to the recovery room was significantly shorter for the ‘experienced’ group than for the ‘inexperienced’ group of anaesthesiologists (median and 25th–75th percentiles were 3.5 (2.0–7.5) vs. 7.0 (4.5–11.0) min, respectively, P = 0.003, two-tailed U-test).
To exclude the possibility that anaesthesiologists in the experienced group simply have patients at a very light level of anaesthesia during the last minutes of surgery to achieve shorter recovery times, we analysed the time course of the EEG parameters during the last 10 min of surgery (shown in Fig. 5 for the BIS). No significant difference between the ‘experienced’ and ‘inexperienced’ group was found (Friedman's test).
With this study we have demonstrated that for a group of more experienced anaesthesiologists, the association between subjectively assigned values of anaesthetic depth and EEG parameters of anaesthetic depth is better than for anaesthesiologists with little experience.
This may not seem surprising since it has been shown for many parts of anaesthetic practice that experience leads to better performance [14–16]. What is more astonishing is the poor correlation between EEG parameters and the subjective assessment of anaesthetic depth in both groups. However, the concept of ‘depth of anaesthesia’ is rather elusive  and no clues were given to the participating anaesthesiologists how to define ‘depth of anaesthesia’, in order to portray the clinical situation. The poor correlation between subjective assessment of anaesthetic depth and BIS is especially astonishing for BIS, since BIS has been derived from a database correlating EEG parameters with clinical descriptors of anaesthetic depth . In this light, it is noteworthy that the correlation is as good (or as bad) for the entropy parameters, which are pure mathematical derivatives of the EEG  and have not been generated from a correlation with clinical assessment of anaesthetic depth.
Important aspects of anaesthetic depth are immobility and the absence of conscious perception and memory to that perception . When focussing on the latter aspect, ‘depth of anaesthesia’ should be rather called ‘depth of hypnosis’  and can be defined as the probability of becoming conscious at a time point during the anaesthetic.
It is not possible to measure the true depth of hypnosis in a patient. Therefore, we used parameters of the processed EEG as surrogate markers. BIS has been shown to correlate with the probability of memory formation during anaesthesia with different anaesthetics [15,21,22]. However, the results differ between studies [23–25] and the variability in the relation between BIS and memory formation is not known. For SE/RE until now no studies have been published relating these entropy parameters to memory formation, but they appear to correlate well with BIS values [26–31]. This has been confirmed in our study (Fig. 3).
Next to the variability of the EEG parameters themselves we assume the main reason for the poor correlation between EEG parameters and the subjective assessment of anaesthetic depth is the confusion of different anaesthetic end-points: immobility, sympathetic response, hypnosis.
Anaesthetics in this study were almost exclusively propofol-based, so anaesthesiologists lacked any direct information of anaesthetic concentration and had to rely on the clinically observable end-points immobility and sympathetic response.
It has been shown that the immobilizing action of anaesthetics follows a different time course than the EEG parameters (which correlate with hypnosis) . Most anaesthetics in this study were performed without neuromuscular blockers, or neuromuscular blockers were given only for intubation. Thus anaesthesiologists may have used patient movements as a major criterion for assessing ‘depth of anaesthesia’. If this would be an important part of the assessment, those parameters which include electromyogram (EMG) activity (BIS and RE) should have shown a better correlation than the SE, which does not rely on EMG activity. However, this was not the case. Although patient movements are clearly a sign of impeding arousal and the use of neuromuscular blocker is a risk factor for awareness , immobility and hypnosis follow different time courses . The same is probably true for the sympathetic response, the other main guideline for subjective assessment of ‘depth of anaesthesia’. EEG parameters such as the BIS or the entropy parameters, on the other hand, are parameters of the end-point ‘hypnosis’ and thus correlate poorly with the end-points used for subjective assessment .
We speculate that it would be helpful to educate anaesthesiologists in the differences of the various anaesthetic end-points to facilitate subjective assessment in order to improve the clinical performance. We have seen shorter recovery times for more experienced anaesthesiologists, so this outcome parameter could be improved by additional educational or monitoring measures. We demonstrated that the shorter recovery time was not due to a premature lightening of anaesthetic depth. Probably experienced anaesthesiologists are better in discerning deep stages of anaesthesia and thus reducing the amount of anaesthetic taken up by the patients. One approach to this problem may be a better education of anaesthesiologists in pharmacokinetics using concepts such as the ‘context sensitive half-time’ or the ‘relevant effect-site decrement time’ . In addition, target-controlled infusion systems could help the anaesthesiologists to better control anaesthetic uptake. However, studies comparing target and manually-controlled propofol infusions have led to conflicting results concerning emergence times [36,37].
The use of target-controlled infusion systems may also decrease the incidence of awareness during anaesthesia , but large studies are lacking on that subject. In our data, no memory formation was detected by the simple interview despite a large percentage of intraoperative BIS values in a range where memory formation is possible. However, the interview is sensitive only to explicit memory formation and was performed at a time where recall may not have developed in all patients in our study . Although we have not observed awareness in our study, there are sufficient data in the literature to support the conclusion that the high percentage of high BIS values indicates a risk of awareness for the patients of our study [11,39]. In other terms, both inexperienced and experienced anaesthesiologists sometimes did not provide ‘deep enough’ depth of hypnosis (defined as the probability of becoming conscious/aware) during surgery in our study. Conversely some patients who appeared to be awake had BIS values as low as 60. This may be due to the variability of the BIS or due to inter-individual differences in the interpretation of the subjective assessment scale.
In summary, our data show that anaesthetic experience leads to a better association of subjective assessment of ‘depth of anaesthesia’ with EEG parameters of hypnotic depth and shorter recovery times. However, relying solely on clinical signs of ‘anaesthetic depth’ is associated with a high incidence of undesirable shallow depth of hypnosis.
Conflict of interest: Monitor and electrodes were supplied by Datex-Ohmeda (Helsinki, Finland). Results of the study were presented in part at the ASA Annual Meeting 2004.
1. Cullen DJ, Eger EI, Stevens WC et al
. Clinical signs of anesthesia. Anesthesiology
2. Eger EI, Saidman LJ, Brandstater B. Minimum alveolar anesthetic concentration: a standard of anesthetic potency. Anesthesiology
3. Antognini JF, Schwartz K. Exaggerated anesthetic requirements in the preferentially anesthetized brain. Anesthesiology
4. Rampil IJ, Mason P, Singh H. Anesthetic potency (MAC) is independent of forebrain structures in the rat. Anesthesiology
5. Evans JM. Pain and awareness during general anaesthesia. Lancet
6. Schwender D, Daunderer M, Klasing S, Mulzer S, Finsterer U, Peter K. Monitoring intraoperative awareness. Vegetative signs, isolated forearm technique, electroencephalogram, and acute evoked potentials. Anaesthesist
7. Tunstall ME. Detecting wakefulness during general anaesthesia for caesarean section. BMJ
8. Heier T, Steen PA. Assessment of anaesthesia depth. Acta Anaesthesiol Scand
9. Leslie K, Sessler DI, Smith WD et al
. Prediction of movement during propofol/nitrous oxide anesthesia. Performance of concentration, electroencephalographic, pupillary, and hemodynamic indicators. Anesthesiology
10. Struys M, Versichelen L, Byttebier G, Mortier E, Moerman A, Rolly G. Clinical usefulness of the bispectral index for titrating propofol target effect-site concentration. Anaesthesia
11. Myles PS, Leslie K, McNeil J, Forbes A, Chan MT. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. Lancet
12. Myles PS, Symons JA, Leslie K. Anaesthetists' attitudes towards awareness and depth-of-anaesthesia monitoring. Anaesthesia
13. Smith WD, Dutton RC, Smith NT. Measuring the performance of anesthetic depth indicators. Anesthesiology
14. Kelley SD. Monitoring level of consciousness during anesthesia & sedation. A Clinician's Guide to the Bispectral Index
. Newton, MA, USA: Aspect Medical Systems, 2003.
15. Eger EI. What is general anesthetic action? Anesth Analg
16. Konrad C, Schupfer G, Wietlisbach M, Gerber H. Learning
manual skills in anesthesiology: is there a recommended number of cases for anesthetic procedures? Anesth Analg
17. Prys-Roberts C. Anaesthesia: a practical or impractical construct? Br J Anaesth
18. Todd MM. EEGs, EEG processing and the bispectral index1. Anesthesiology
19. Viertio-Oja H, Maja V, Sarkela M et al
. Description of the Entropy algorithm as applied in the Datex-Ohmeda S/5 Entropy Module. Acta Anaesthesiol Scand
20. Jensen EW, Litvan H, Struys M, Martinez VP. Pitfalls and challenges when assessing the depth of hypnosis during general anaesthesia by clinical signs and electronic indices. Acta Anaesthesiol Scand
21. Kerssens C, Klein J, van der WA, Bonke B. Auditory information processing during adequate propofol anesthesia monitored by electroencephalogram bispectral index. Anesth Analg
22. Liu J, Singh H, White PF. Electroencephalographic bispectral index correlates with intraoperative recall and depth of propofol-induced sedation. Anesth Analg
23. Deeprose C, Andrade J, Varma S, Edwards N. Unconscious learning
during surgery with propofol anaesthesia. Br J Anaesth
24. Deeprose C, Andrade J, Harrison D, Edwards N. Unconscious auditory priming during surgery with propofol and nitrous oxide anaesthesia: a replication. Br J Anaesth
25. Lubke GH, Kerssens C, Phaf H, Sebel PS. Dependence of explicit and implicit memory on hypnotic state in trauma patients. Anesthesiology
26. Ellerkmann RK, Liermann VM, Alves TM et al
. Spectral entropy and bispectral index as measures of the electroencephalographic effects of sevoflurane. Anesthesiology
27. Schmidt GN, Bischoff P, Standl T, Hellstern A, Teuber O, Schulte EJ. Comparative evaluation of the Datex-Ohmeda S/5 Entropy Module and the bispectral index monitor during propofol–remifentanil anesthesia. Anesthesiology
28. Tiren C, Anderson RE, Barr G, Owall A, Jakobsson JG. Clinical comparison of three different anaesthetic depth monitors during cardiopulmonary bypass. Anaesthesia
29. Vakkuri A, Yli-Hankala A, Talja P et al
. Time-frequency balanced spectral entropy as a measure of anesthetic drug effect in central nervous system during sevoflurane, propofol, and thiopental anesthesia. Acta Anaesthesiol Scand
30. Vanluchene AL, Vereecke H, Thas O, Mortier EP, Shafer SL, Struys MM. Spectral entropy as an electroencephalographic measure of anesthetic drug effect: a comparison with bispectral index and processed midlatency auditory evoked response. Anesthesiology
31. Vanluchene AL, Struys MM, Heyse BE, Mortier EP. Spectral entropy measurement of patient responsiveness during propofol and remifentanil. A comparison with the bispectral index. Br J Anaesth
32. Rehberg B, Bouillon T, Gruenewald M et al
. Comparison of the concentration-dependent effect of sevoflurane on the spinal H-reflex and the EEG in humans. Acta Anaesthesiol Scand
33. Sandin RH, Enlund G, Samuelsson P, Lennmarken C. Awareness during anaesthesia: a prospective case study. Lancet
34. Kissin I. Depth of anesthesia and bispectral index monitoring. Anesth Analg
35. Schnider TW, Shafer SL. Evolving clinically useful predictors of recovery from intravenous anesthetics. Anesthesiology
36. Breslin DS, Mirakhur RK, Reid JE, Kyle A. Manual versus target-controlled infusions of propofol. Anaesthesia
37. Passot S, Servin F, Allary R et al
. Target-controlled versus manually-controlled infusion of propofol for direct laryngoscopy and bronchoscopy. Anesth Analg
38. Lequeux PY, Cantraine F, Levarlet M, Barvais L. Absence of explicit and implicit memory in unconscious patients using a TCI of propofol. Acta Anaesthesiol Scand
39. Glass PS, Bloom M, Kearse L, Rosow C, Sebel P, Manberg P. Bispectral analysis measures sedation and memory effects of propofol, midazolam, isoflurane, and alfentanil in healthy volunteers. Anesthesiology