Residual neuromuscular block persisting into the recovery period occurs in approximately 40% of patients who receive neuromuscular blocking drugs during general anesthesia, placing them at risk of postoperative respiratory complications and delayed discharge from the postanesthetic care unit.1 As clinical signs of muscle weakness cannot reliably detect residual neuromuscular block, experts recommend using objective, quantitative neuromuscular monitoring to ensure the train-of-four (TOF) ratio exceeds 0.90 at the end of anesthesia.1–6
Objective neuromuscular monitoring devices include mechanomyography (MMG), electromyography (EMG), and acceleromyography (AMG). MMG measured at the adductor pollicis is the gold standard, but is impractical for perioperative use.2 Both EMG and AMG are commercially available and easy to use in the clinical setting.7 Studies have demonstrated that EMG TOF ratio measured at the first dorsal interosseus muscle is equivalent to MMG during the late phase of recovery (TOF ratio > 0.70).8–11 Hence, EMG at the first dorsal interosseus is an alternative gold standard for detecting residual neuromuscular block in clinical settings. Additionally, EMG yields more consistent responses because it is not affected by restriction of movement of the muscle.12 Use of AMG reduces the incidence of residual neuromuscular block when compared with subjective neuromuscular monitoring. However, approximately 5% of patients monitored using AMG still had residual neuromuscular block.13–15 To establish an AMG-based criterion for residual neuromuscular block, the bias between AMG and EMG TOF ratio must be known.
A recent systematic review drew attention to methodological shortcomings of previous studies comparing AMG and EMG TOF ratio during recovery from neuromuscular block.16 A contralateral comparison might be influenced by arm-to-arm variations such as electrode positioning, arm dominance, arm temperature, location of the IV catheter, and location of the blood pressure cuff. An ipsilateral comparison, where the AMG and EMG TOF ratios cannot be simultaneously measured, needs to account for the baseline difference between successive measurements during spontaneous recovery of neuromuscular function. Additionally, all previous studies overlooked the assessment of precision, which is a critical step in the Bland–Altman analysis of agreement between 2 methods. The review called for well-designed and sufficiently powered studies to further investigate the agreement between AMG and EMG.
This study evaluated the precision and agreement between TOF ratio obtained by AMG at the adductor pollicis and EMG at the first dorsal interosseus of the same hand during recovery from nondepolarizing neuromuscular block. We hypothesized that the bias between AMG and EMG TOF ratio is at least 0.10 based on the suggestion that an AMG TOF ratio of at least 1.00 should be achieved to exclude residual neuromuscular block.1,2
This study received approval from the Sydney Adventist Hospital Human Research Ethics Committee (EC00141:2011–022). Written informed consent was obtained from all patients. Adult patients requiring neuromuscular block for surgery at the Sydney Adventist Hospital were recruited. Patients with neuromuscular disorders were excluded. The following data were collected: age, sex, body mass index, dominant arm, study arm, and neuromuscular blocking drugs used.
The anesthesiologist in charge of each patient was responsible for making all clinical decisions. Neuromuscular function was monitored on the same arm using AMG (Infinity® Trident® NMT SmartPod® with software VF8, Dräger, Lübeck, Germany) and EMG (NMT-EMG with software 891647-2.0, Datex Ohmeda, Helsinki, Finland). TOF stimulation was set to deliver 4 supramaximal stimuli every 20 seconds, with frequency 2 Hz and pulse width 200 microseconds. AMG was used to record TOF ratios at the adductor pollicis muscle. EMG was used to record TOF ratios at the first dorsal interosseus muscle.
Before the initiation of neuromuscular block, the study arm was immobilized on an arm board with adhesive tape. Preparation of the skin and placement of the Red Dot Ag/AgCl Micropore™ backed Electrodes (3M, St. Paul, MN) followed current recommendations.17 To measure TOF ratio by EMG, 5 electrodes were needed. This included 2 stimulating electrodes over the ulnar nerve, 1 ground electrode, the recording electrode over the first dorsal interosseus muscle and the neutral electrode on the index finger (Fig. 1). To measure TOF ratio by AMG, we used the same stimulating electrodes and the piezoelectric sensor in the AMG hand adapter. The AMG hand adapter applies a small preload of 75 to 150 g, depending on the size of the patient’s hand. The arm was covered with a forced-air warming blanket to maintain the temperature of the study site above 32°C and monitored by the thermistor embedded in the hand adapter of the AMG. The supramaximal stimulating current for each device was established automatically by built-in functions, and we confirmed that both devices achieved stable signals by visual inspection that the EMG and AMG waveform had a smooth biphasic contour before neuromuscular blocking drugs were administered. Adjustment of gain to 100% twitch height using single twitch stimulation (calibration) was not necessary because this procedure equally amplifies all 4 responses, so the TOF ratio is not affected by this procedure.7
Data recording commenced when TOF ratio recovered to 0.20 by EMG and terminated on completion of surgery or administration of neuromuscular block antagonist (because accelerated recovery would inflate the difference between measurements). Multiple comparisons were made. Each time, 2 consecutive AMG measurements were compared with 2 consecutive EMG measurements (Fig. 2). When switching between AMG and EMG, one device would be detached from the stimulating electrodes and the other device would be attached in its place, taking no more than 10 seconds. The recording order of AMG and EMG was reversed after each comparison because always having one device preceding the other may introduce systematic bias. Under this protocol, 4 consecutive EMG measurements would be recorded between each pair of comparisons. These data were used for a control comparison between EMG and itself to establish the baseline difference between successive measurements during spontaneous recovery of neuromuscular function.
AMG and EMG TOF ratios were compared by the Bland–Altman analysis for repeated measurements performed using the statistical software R (version 2.14.1, R Foundation for Statistical Computing, Vienna, Austria).18 This analysis involves assessing the agreement between the devices by the bias and the 95% limits of agreement. Small bias and narrow limits of agreement indicate strong agreement. Additionally, this analysis accounts for the effect of instrumental imprecision by evaluating the repeatability coefficient of each device.18,19 A small repeatability coefficient indicates high precision of the device.
A hierarchy of analyses were planned. Initially, a standard Bland–Altman analysis would be performed, with multiple comparisons of AMG and EMG TOF ratio from the same patient treated as if they were from different patients. Compared with a random effects approach, the standard Bland–Altman analysis would yield an identical bias but the limits of agreement would be slightly narrower and therefore more conservative. If agreement between AMG and EMG was shown to be unsatisfactory by the standard Bland–Altman analysis, then we would not conduct further analysis. Otherwise, a random effects approach would be adopted. In the end, only the standard Bland–Altman analysis was performed.
One-way analysis of variance was performed on sets of consecutive TOF ratios measured by the same device. The 95% repeatability coefficient is defined as 1.96 times the standard deviation of the differences between 2 consecutive measurements. That is,
was estimated by the square root of the residual mean square. The 95% confidence interval (CI) for the repeatability coefficient was constructed using the sample variance of r,
where n is the number of comparisons. The interval (−r, r), known as the 95% repeatability limits, specifies the range within which 2 measurements by the same device will differ in 95% of cases.
The bias (
) is defined as the mean difference of AMG minus EMG TOF ratio. The corresponding limits of agreement are defined as
was estimated by
The 95% CI for the limits of agreement was constructed using the sample variance of L,
, where n is the number of comparisons. The limits of agreement capture 95% of differences between AMG and EMG TOF ratio. We defined clinically acceptable agreement between AMG and EMG as a bias <0.025 and limits of agreement within −0.050 to 0.050, provided that the control comparison between EMG and itself can fulfill these criteria.
To ensure that reliable estimates of repeatability coefficient (r), bias (
), and limits of agreement (L) are obtained, we performed an a priori sample size calculation using parameters obtained from a pilot study. We decided that the 95% CIs must be within ±0.0250 of the estimates. For the repeatability coefficient (r) to achieve this, at least 24 comparisons were required. This was done by solving
with a pilot estimate of
Similarly, we solved
0.0250 with a pilot estimate of
and found that at least 43 and 128 comparisons were required to obtain reliable estimates for bias (
) and limits of agreement (L), respectively. To reduce the underestimation of the limits of agreement by the standard Bland–Altman analysis for repeated measurements, we ensured the number of comparisons per patient was less than the number of patients.19 Therefore, we required at least 128 comparisons between AMG and EMG from at least 16 patients.
We collected 261 comparisons between AMG and EMG from 26 patients (Table 1). Two comparisons were excluded due to external movement of the study arm. This exceeded the sample size required for sufficiently reliable estimates of repeatability coefficient (r), bias (
), and limits of agreement (L).
AMG was significantly less precise than EMG for measuring TOF ratio over the range 0.20 to 1.00 (Table 2). When the comparisons were stratified into different levels of neuromuscular recovery, we found AMG to be less precise over medium range TOF ratios (0.40–0.59 and 0.60–0.79) compared with other ranges.
We found a bias of 0.176 (95% CI, 0.162–0.190) between AMG and EMG for measuring TOF ratio over the range 0.20 to 1.00, with wide and positively skewed limits of agreement (Table 3, Fig. 3). The agreement between AMG and EMG was not clinically acceptable by our a priori definition. The control comparison between EMG and itself showed that the baseline difference between successive measurements during spontaneous recovery of neuromuscular function was a negligible amount of 0.010 (95% CI, 0.005–0.014). The bias between AMG and EMG is significantly larger than the baseline. When the comparisons were stratified into different levels of neuromuscular recovery, we found a smaller bias between AMG and EMG over low range TOF ratios (0.20–0.39) compared with higher ranges.
TOF ratios measured by AMG and EMG are not interchangeable for detecting residual neuromuscular block. Our study demonstrated that AMG overestimates EMG TOF ratio by 0.176 (95% CI, 0.162–0.190), with wide and positively skewed limits of agreement. The baseline difference of 0.010 (95% CI, 0.005–0.014) between successive measurements during spontaneous recovery of neuromuscular function accounted for a negligible amount of the overall bias. Furthermore, the lack of agreement cannot be explained by the imprecision of either device because the limits of agreement were significantly wider than the repeatability limits. An uncorrected AMG TOF ratio overestimates EMG by at least 0.15. Correction techniques are discussed later.
According to 1 systematic review, the agreement between AMG and EMG TOF ratio has been evaluated by 2 previous studies with acceptable methodology, despite some shortcomings.16 The studies reported different estimates of the bias between AMG and EMG TOF ratio. The bias estimate of 0.125 reported by Kopman et al.21 is consistent with our result of 0.176 (95% CI, 0.162–0.190). Using the standard error of the bias estimate reported by Kopman, we calculated the 95% CI to be 0.061 to 0.181.21 As this interval overlaps with our result, there is no significant difference between the 2 bias estimates. Both the study by Kopman and our study conducted an ipsilateral comparison, applied a small preload with AMG, and used EMG at the first dorsal interosseus muscle. However, some key distinctions must be made. An ipsilateral comparison needs to account for the baseline difference between successive measurements during spontaneous recovery of neuromuscular function. Kopman recognized this as a potential source of error, but did not assess the magnitude of the potential error as we have done by a control comparison between EMG and itself. We have further demonstrated that the lack of agreement between AMG and EMG TOF ratio also cannot be explained by the imprecision of either device. Kopman performed an a priori sample size analysis based on hypothesis testing rather than parameter estimation, resulting in an underestimation of the required sample size to obtain a sufficiently reliable bias estimate. Our bias estimate with a narrow 95% CI is more reliable.
In contrast, Dahaba et al.’s20 bias estimate of 0.050 is significantly different to both Kopman’s21 and our results. Dahaba did not apply a preload with AMG and used EMG at a different muscle. We studied the first dorsal interosseus because EMG at this muscle is an alternative gold standard for detecting residual neuromuscular block in clinical settings. The first dorsal interosseus is also preferable when using EMG because its response exhibits good morphology with large peak to peak amplitude and relatively little baseline drift.12 Furthermore, Dahaba’s contralateral comparison might have been influenced by arm-to-arm variations such as electrode positioning, arm dominance, and arm temperature.22 Like Kopman,21 Dahaba20 omitted the assessment of precision, which is a critical step in the Bland–Altman analysis of agreement between 2 methods.18,19 We have evaluated the agreement between AMG and EMG TOF ratio by an ipsilateral comparison to address these methodological issues.
Is it possible to correct an AMG TOF ratio so that it can be used interchangeably with EMG to detect residual neuromuscular block? One technique is normalization. This involves dividing an AMG TOF ratio by the preinduction control, which has an average value of 1.15.23 Although normalization can effectively reduce the bias,24 it is impractical to perform in clinical settings. Additionally, some AMG devices available for clinical use, such as the Infinity Trident NMT SmartPod, have been programmed to limit the displayed TOF ratio to 1.00 so the preinduction control cannot be obtained and normalization cannot be performed. Another technique used by AMG devices such as the TOF Watch® and the TOF WatchS® (Organon, Dublin, Ireland) is to display the T4/T2 ratio rather than the T4/T1 ratio under circumstances where T2 > T1. However, there are discrepancies between the T4/T2 ratio and the normalized TOF ratio.25 The simplest correction involves adjusting an AMG TOF ratio by the bias. We believe subtracting at least 0.15 from an AMG TOF ratio gives a reliable estimate of the EMG TOF ratio. However, this technique should be used with caution given the poor reliability of AMG and the wide limits of agreements between AMG and EMG. Our findings provide evidence to support the suggestion by some experts that an AMG TOF ratio of at least 1.00, perhaps with an additional waiting period, may be necessary to exclude residual neuromuscular block before tracheal extubation.26,27 This waiting period would vary according to factors including the choice of relaxant and reversal drug, patient gender and age, core temperature, and renal and liver function. Future studies should clinically validate such a proposal.
The limitations of our evaluation of the agreement between AMG and EMG TOF ratio should be addressed. First, we treated multiple observations from the same patient as if they were from different patients. We have taken the recommended measures to minimize the correlation between the patients and the comparisons.19 Compared with a random effects model, the standard Bland–Altman analysis would yield an identical bias but the limits of agreements would be slightly narrower and therefore more conservative.19 Even so, the agreement between AMG and EMG was still unsatisfactory. As such, we decided that there was no need to adopt a random effects model. Second, to pool comparisons of TOF ratios over the range 0.20 to 1.00, we assumed that the bias is relatively constant at different levels of neuromuscular recovery. Stratified analysis found a smaller bias over low range TOF ratios (0.20–0.39), but no significant difference among the remaining strata. Third, we assumed that the bias between AMG and EMG TOF ratios was not influenced by differences in factors such as patient characteristics or the anesthetic technique. This assumption holds because we are comparing AMG and EMG TOF ratios under identical conditions in 1 patient at 1 moment, allowing each patient to act as his or her own control. Additionally, the sensitivity of adductor pollicis and first dorsal interosseus share close resemblance and is not affected by the choice of relaxant.28,29
In conclusion, AMG and EMG TOF ratio cannot be used interchangeably to assess residual neuromuscular block. After accounting for instrumental imprecision and the baseline difference between successive measurements during spontaneous recovery of neuromuscular function, AMG overestimates EMG TOF ratio by at least 0.15. Anesthesiologists should ensure by objective monitoring that patients do not have residual neuromuscular block before sending them to the recovery room. Therefore, anesthesiologists must be aware of how much AMG overestimates recovery. Residual neuromuscular block, defined as an EMG or MMG TOF ratio of <0.90, cannot be excluded immediately on reaching an AMG TOF ratio of 0.90 or indeed 1.00. This limits the clinical usefulness of AMG devices that can only display TOF ratio up to 1.00, such as the Infinity Trident NMT SmartPod.
Name: Sophie S. Liang, BSc (Adv.).
Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.
Attestation: Sophie S. Liang has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.
Name: Paul A. Stewart, MBBS, FANZCA.
Contribution: This author helped design the study, conduct the study, and write the manuscript.
Attestation: Paul A. Stewart has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Name: Stephanie Phillips, BMed, FANZCA, FRCA.
Contribution: This author helped design the study, conduct the study, and write the manuscript.
Attestation: Stephanie Phillips has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
This manuscript was handled by: Dwayne R. Westenskow, PhD.
The authors thank Professor J. Martin Bland, Department of Health Sciences, University of York, UK, and Associate Professor Gillian Heller, Department of Statistics, University of Macquarie, Australia, for their comments on the statistical analysis, and Dr. Adam Osomanski, Sydney Adventist Hospital, Australia, and Peter Xie, Sydney Medical School, Australia, for their contribution to data collection and manuscript preparation.
1. Murphy GS, Brull SJ. Residual neuromuscular block: lessons unlearned. Part I: definitions, incidence, and adverse physiologic effects of residual neuromuscular block. Anesth Analg. 2010;111:120–8
2. Brull SJ, Murphy GS. Residual neuromuscular block: lessons unlearned. Part II: methods to reduce the risk of residual weakness. Anesth Analg. 2010;111:129–40
3. Donati F. Neuromuscular monitoring: what evidence do we need to be convinced? Anesth Analg. 2010;111:6–8
4. Kopman AF. Managing neuromuscular block: where are the guidelines? Anesth Analg. 2010;111:9–10
5. Miller RD, Ward TA. Monitoring and pharmacologic reversal of a nondepolarizing neuromuscular blockade should be routine. Anesth Analg. 2010;111:3–5
6. Viby-Mogensen J, Claudius C. Evidence-based management of neuromuscular block. Anesth Analg. 2010;111:1–2
7. Fuchs-Buder T, Claudius C, Skovgaard LT, Eriksson LI, Mirakhur RK, Viby-Mogensen J8th International Neuromuscular Meeting. . Good clinical research practice in pharmacodynamic studies of neuromuscular blocking agents II: the Stockholm revision. Acta Anaesthesiol Scand. 2007;51:789–808
8. Kopman AF. The relationship of evoked electromyographic and mechanical responses following atracurium in humans. Anesthesiology. 1985;63:208–11
9. Engbaek J, Skovgaard LT, Friis B, Kann T, Viby-Mogensen J. Monitoring of the neuromuscular transmission by electromyography (I). Stability and temperature dependence of evoked EMG response compared to mechanical twitch recordings in the cat. Acta Anaesthesiol Scand. 1992;36:495–504
10. Shanks CA, Jarvis JE. Electromyographic and mechanical twitch responses following suxamethonium administration. Anaesth Intensive Care. 1980;8:341–4
11. Engbaek J, Roed J, Hangaard N, Viby-Mogensen J. The agreement between adductor pollicis mechanomyogram and first dorsal interosseous electromyogram. Acta Anaesthesiol Scand. 1994;38:869–78
12. Kalli I. Effect of surface electrode position on the compound action potential evoked by ulnar nerve stimulation during isoflurane anaesthesia. Br J Anaesth. 1990;65:494–9
13. Mortensen CR, Berg H, el-Mahdy A, Viby-Mogensen J. Perioperative monitoring of neuromuscular transmission using acceleromyography prevents residual neuromuscular block following pancuronium. Acta Anaesthesiol Scand. 1995;39:797–801
14. Gätke MR, Viby-Mogensen J, Rosenstock C, Jensen FS, Skovgaard LT. Postoperative muscle paralysis after rocuronium: less residual block when acceleromyography is used. Acta Anaesthesiol Scand. 2002;46:207–13
15. Murphy GS, Szokol JW, Marymont JH, Greenberg SB, Avram MJ, Vender JS, Nisman M. Intraoperative acceleromyographic monitoring reduces the risk of residual neuromuscular blockade and adverse respiratory events in the postanesthesia care unit. Anesthesiology. 2008;109:389–98
16. Claudius C, Viby-Mogensen J. Acceleromyography for use in scientific and clinical practice: a systematic review of the evidence. Anesthesiology. 2008;108:1117–40
17. Smans J, Korsten HH, Blom JA. Optimal surface electrode positioning for reliable train of four muscle relaxation monitoring. Int J Clin Monit Comput. 1996;13:9–20
18. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60
19. Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. JBiopharm Stat. 2007;17:571–82
20. Dahaba AA, Rehak PH, List WF. Assessment of accelerography with the TOF-GUARD: a comparison with electromyography. Eur J Anaesthesiol. 1997;14:623–9
21. Kopman AF, Chin W, Cyriac J. Acceleromyography vs. electromyography: an ipsilateral comparison of the indirectly evoked neuromuscular response to train-of-four stimulation. Acta Anaesthesiol Scand. 2005;49:316–22
22. Claudius C, Skovgaard LT, Viby-Mogensen J. Arm-to-arm variation when evaluating neuromuscular block: an analysis of the precision and the bias and agreement between arms when using mechanomyography or acceleromyography. Br J Anaesth. 2010;105:310–7
23. Suzuki T, Fukano N, Kitajima O, Saeki S, Ogawa S. Normalization of acceleromyographic train-of-four ratio by baseline value for detecting residual neuromuscular block. Br J Anaesth. 2006;96:44–7
24. Claudius C, Skovgaard LT, Viby-Mogensen J. Is the performance of acceleromyography improved with preload and normalization? A comparison with mechanomyography. Anesthesiology. 2009;110:1261–70
25. Kopman AF, Kopman DJ. An analysis of the TOF-watch algorithm for modifying the displayed train-of-four ratio. Acta Anaesthesiol Scand. 2006;50:1313–4
26. Capron F, Alla F, Hottier C, Meistelman C, Fuchs-Buder T. Can acceleromyography detect low levels of residual paralysis? A probability approach to detect a mechanomyographic train-of-four ratio of 0.9. Anesthesiology. 2004;100:1119–24
27. Eikermann M, Vogt FM, Herbstreit F, Vahid-Dastgerdi M, Zenge MO, Ochterbeck C, de Greiff A, Peters J. The predisposition to inspiratory upper airway collapse during partial neuromuscular blockade. Am J Respir Crit Care Med. 2007;175:9–15
28. Harper NJ. Comparison of the adductor pollicis and the first dorsal interosseous muscles during atracurium and vecuronium blockade: an electromyographic study. Br J Anaesth. 1988;61:477–8
29. Phillips S, Stewart PA, Freelander N, Heller G. Comparison of evoked electromyography in three muscles of the hand during recovery from non-depolarising neuromuscular blockade. Anaesth Intensive Care. 2012;40:690–6