Secondary Logo

Journal Logo


Evaluating learning curves for intubation in a simulator setting: a prospective observational cumulative sum analysis

Vennila, Rajagopal; Sethuraman, Dhandapani; Charters, Peter

Author Information
European Journal of Anaesthesiology: November 2012 - Volume 29 - Issue 11 - p 544-545
doi: 10.1097/EJA.0b013e328356ba54


Tracheal intubation is one of the core skills to be learnt by anaesthetic trainees. Simulator-based training helps to develop knowledge, practice skills and rehearse drills and such experience does translate to better patient care.1 Therefore, we undertook this project to evaluate the feasibility of using CUMulative SUM (CUSUM) analysis as a training and assessment tool for teaching intubation skills in novices in a simulated environment.

This prospective observational study was approved by the Hospital Research and Development Department (Royal Liverpool University Hospital, trust reference number 3767, approved on 6 February 2009, Director: Professor J. Rhodes). The Sefton and Cheshire Research Ethics Committee (Victoria Building, Liverpool, UK; Coordinator Ms K. Roberts, communication dated 12 December 2008) had earlier reviewed the protocol and opined that a formal ethics review was not necessary which conforms to the nationwide practice for simulator studies.2

Fifteen novice learners with no previous training or experience in intubation were recruited for the study. Informed written consent was obtained. Intubation conditions were standardised with an AirSim Multi intubation trainer (Trucorp Ltd, Belfast, N. Ireland) in a fixed position at appropriate height, grade 1 Cormack–Lehane view using a Macintosh-3 blade (Penlon Ltd, Abingdon, UK) and a 7.0 mm endotracheal tube. After a standardised 30-min training session, followed by a demonstration of the technique and three guided practice intubations, the volunteers intubated the AirSim manikin repeatedly until one of the endpoints (CUSUM curve crossing the lower boundary limit, maximum of 85 intubation attempts or inability to complete the study protocol) was reached. All intubations were supervised by one of the investigators who also acted as an assistant and continual feedback was provided. The first 10 of these intubations were deemed ‘practice attempts’ and excluded from analysis. A maximum of 45 s was allowed for intubation.

Primary observations were as follows: successful intubation, time-to-intubate (TTI) as previously described3 and total number of intubation attempts for CUSUM competence. On the basis of data from previously published studies,3,4 acceptable failure rate (p0) was set at 3%; unacceptable failure rate (p1) was set at 7.5%, being two and half times beyond the acceptable failure rate; both α-error and β-error were set at 0.1 and CUSUM curve was plotted as described by de Oliveira Filho et al.5 The TTIs were divided into four consecutive quarters which were analysed by Kaplan–Meier survival analysis using SPSS software (IBM Corporation, New York, USA). Statistical significance was set at 0.05 by log-rank analysis and plateau was deemed to have been achieved when there was no statistical difference between two consecutive quarters.

Eleven of the 15 candidates achieved ‘competence’ as demonstrated by their CUSUM curve crossing the lower boundary level (see, Supplementary Digital Content, Only one failed intubation was observed in the entire pooled data for the fourth quarter (one of approximately 240 intubations). TTI improved dramatically with this training, with an overall mean of just 12.4 s for the last quarter (Fig. 1). By the end of the session, the TTI and success rate of all the candidates were comparable with previously published data for experienced laryngoscopists.3,4

Fig. 1
Fig. 1:
No captions available.

Kaplan–Meier curve for the pooled TTI data over the four quarters showed consistent improvements in the intubation over the four quarters (Supplementary Digital Content, Kaplan–Meier analysis of TTI failed to show any intergroup variations in three candidates (1, 11 and 15), indicating that their TTI was statistically identical over the four quarters. These candidates achieved plateau by the end of their first quarter (i.e. approximately 27 intubations). ‘Pairwise strata analysis’ of the TTIs in the rest showed the absence of any statistically significant difference between the third and fourth quarters in four candidates (candidates 2, 6, 7 and 8), indicating that they reached ‘plateau’ by the end of the second quarter (approximately 43 attempts). Candidate 3 reached ‘plateau’ by the end of the third quarter, after 59 attempts. Seven of the 15 candidates were still improving their performance during the fourth quarter. The performance of these seven candidates in their fourth quarter was further analysed by comparing the two halves within this quarter. This analysis showed that a further five candidates (9, 10, 12, 13 and 14) achieved ‘plateau’ during this fourth quarter (approximately 78 intubations). Candidates 4 and 5 still showed a significant variation in the TTI during this subgroup analysis indicating that they were still in the learning process when the study terminated.

Simulators are advantageous for trainees to perfect their technical skills. It is our duty to ensure that novices are confident and competent in their skills before attempting to perform complex procedures on patients. Our study shows that institution of an intense CUSUM-based training resulted in all the novices achieving success rate and TTI comparable to experienced laryngoscopists. Hence, we have now instituted a similar training for all our novices in our institute.

Our study also unequivocally demonstrates that even after 60 attempts, the majority of the novices do not reach a plateau in their learning curves, confirming a prolonged learning process for intubation skills and adds evidence to Maharaj et al.6 Although many institutions have applied the CUSUM method to ensure quality control in other specialties,7 our data support the use of CUSUM for training and evaluation of novices’ intubation skills.

Limitations of our study include repetitive intubation leading to ‘learning the manikin’ and fatigue during the study.


Assistance with the letter: the authors would like to thank Dr Jane Harper for her valuable contribution in preparation of this manuscript.

Financial support and sponsorship: the study was sponsored and funded by the Royal Liverpool University Hospital, Liverpool, UK.

Conflicts of interest: none declared.


1. Tuggy ML. Virtual reality flexible sigmoidoscopy simulator training: impact on resident performance. J Am Board Fam Pract 1998; 11:426–433.
2. Cann C, Hall JE, Sudheer PS, Turley A. Is ethical approval necessary for manikin studies? Anaesthesia 2005; 60:94– reply 95-96.
3. Sethuraman D, Darshane S, Guha A, Charters P. A randomised, crossover study of the Dorges, McCoy and Macintosh laryngoscope blades in a simulated difficult intubation scenario. Anaesthesia 2006; 61:482–487.
4. Lim TJ, Lim Y, Liu EH. Evaluation of ease of intubation with the GlideScope or Macintosh laryngoscope by anaesthetists in simulated easy and difficult laryngoscopy. Anaesthesia 2005; 60:180–183.
5. de Oliveira Filho GR. The construction of learning curves for basic skills in anesthetic procedures: an application for the cumulative sum method. Anesth Analg 2002; 95:411–416.
6. Maharaj CH, Costello JF, Higgins BD, et al. Learning and performance of tracheal intubation by novice personnel: a comparison of the Airtraq and Macintosh laryngoscope. Anaesthesia 2006; 61:671–677.
7. Noyez L. Control charts, Cusum techniques and funnel plots. A review of methods for monitoring performance in healthcare. Interact Cardiovasc Thorac Surg 2009; 9:494–499.
© 2012 European Society of Anaesthesiology