Full-scale and life-like simulation scenarios have become an integral part of modern education with such teaching having been used in both anaesthesia and emergency medicine.1,2 Apart from the simple learning and refining of manual skills, many additional aspects of training such as team interactions and other social and psychological factors have been introduced into simulator-based education.3–5 Factors such as distraction, perceived workload and associated stress have recently been identified as impacting on performance in anaesthesia and emergency situations.6,7 Flin et al.8 identified human factors such as situation awareness and task difficulty as contributing to serious events in airway management. In another study by Fernandez et al.,9 the presence of a family member had a negative effect on physicians’ performance during cardiopulmonary resuscitation (CPR). As a result, there have been attempts to incorporate stress and distraction into simulation scenarios through the use of actors. These disturbances did increase perceived levels of stress in general, but in contrast to the aforementioned clinical studies, had no consistent effect on clinical performance levels.10 However, it could be argued that an actor will only be able to generate an unpredictable, nonstandardised and nonmeasurable degree of stress and distraction, which is unique to each actor and each participant within a given simulation. An alternative approach could be to investigate and train the different neurocognitive dimensions from which common stress may be originating. Although stress is multidimensional and hard to quantify, perceived workload as one major component of stress is more easily accessible to scientific investigations and is, therefore, the main focus of our study.11
Apart from coping with the social and emotional aspects of emergency situations, a high workload deriving from the need to fulfil competing tasks simultaneously may become the real challenge. For example, the requirement to simultaneously perform CPR and tracheal intubation, supervise assisting personnel and care for an observing family member can lead to a fractioning of neuropsychological capabilities known as divided attention, which will in all probably have a negative influence on performance. This dilemma of executing two or more tasks simultaneously is recognised in psychology as the dual-task paradigm12–14 and may be assessed by the paced auditory serial addition test (PASAT)15 (used courtesy of Professor Dr T. Schenk, Neuropsychology Division, Department of Neurology, Erlangen University). Since its introduction into the clinical assessment of cognitive functioning in 1974, the PASAT has been proven to be a highly sensitive test to investigate attention, distraction and dual-tasking.15–19 The PASAT occupies a large portion of a test individual's attention and short-term memory and has been used to quantify even minor neurocognitive deficits in several neurological disorders.
The objective of our study was to introduce and validate the PASAT for use in simulation scenarios as a standardised and measurable source of divided attention that would exert a predictable increase in workload on participants.
Materials and methods
Ethical approval for this study was provided by the Ethical Committee of Friedrich-Alexander-University, Erlangen, Germany (Chairperson Prof P. Betz) on 7 November 2013. Detailed informed consent was obtained from all test individuals. This randomised, crossover, interventional simulation study was undertaken at the Centre for Training and Simulation affiliated to the Department of Anaesthesiology at Erlangen University Hospital in the summer of 2014. Volunteers were invited to secure a manikin's airway using different techniques with (dual-task) and without the PASAT (single-task). We assessed performance for each task and measured the perceived workload using the National Aeronautics and Space Administration's task load index (NASA-TLX).20,21
The study population was recruited from anaesthesia consultants, registrars and fellows, medical students in years 5 and 6 and paramedics (n = 31). Paramedics and medical students each had a 60-min introductory lecture on airway management and 30 min of manikin training before partaking in our trial. Anaesthetists were all accustomed to the simulation equipment and had 15 min before the actual experiment to familiarise themselves with airways devices that were to be used.
Airway management interventions were performed on a Laerdal Intubation Torso (Laerdal Medical, Stavanger, Norway). A laryngeal mask airway (LMA; Solus LMA; Intersurgical, Sankt Augustin, Germany) was the supraglottic airway used. Endotracheal intubation was achieved with an internal diameter 7.0 mm endotracheal tube (Safety Clear; Rüsch, Kernen, Germany). Laryngoscopy was with a standard laryngoscope with a size 3 Macintosh blade (Heine, Herrsching, Germany) or a videolaryngoscope with a size 3 blade (Glidescope GVL; Verathon Inc., Bothell, Washington, USA).
Each participant had to perform three different airway interventions, each in the fastest time possible: insertion of a supraglottic airway; endotracheal intubation with conventional laryngoscopy; and endotracheal intubation with videolaryngoscopy. Each intervention had to be performed twice as either a single task or in combination with the PASAT as a dual task. The sequence of these tasks was randomised. The participants performed all tasks in isolation, with no external assistance provided. Preparation of each scenario was standardised with uniformity of written and verbal instructions, materials and arrangements equipment.
The PASAT consisted of a voice recording of a virtually endless succession of single digits (one to nine) spoken at three-second intervals and played back on a simple loudspeaker. Participants had to form subtotals of the last two digits of the succession for the duration of each airway task (Table 1). Answers had to be enunciated and were registered as correct, incorrect or missing. To ensure that the PASAT had exerted the desired effect, at least 50% of calculations had to be correct. A video showing the corresponding author (J.P.) in an example of the dual-tasking scenario involving conventional tracheal intubation is available online (supplemental video file 1, http://links.lww.com/EJA/A74) and may help to demonstrate the general set-up and workings of our study.
Workload was measured using an unweighted version of the NASA-TLX after the completion of each airway intervention. The NASA-TLX is a six-dimensional Likert scale based questionnaire, which assesses the individual perception of mental, physical and temporal demands as well as performance, effort and frustration with respect to the task (or in our case combined dual-task) being undertaken.
Before the test runs, each participant was asked to rate their extent of previous simulation experience and subjective assessment of multitasking capabilities. After completion of the study, participants evaluated the dual-tasking scenario on a Likert scale for general aspects of difficulty, acceptability and likeness to real emergency situations using grades 1 (totally agree) to 6 (totally disagree).
Statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina, USA). General characteristics are presented by mean, median and 95% confidence intervals (CIs) where appropriate. The primary endpoint of our investigation was the difference in the time taken to complete each airway intervention, defined as the time elapsed from the first grasp towards equipment till the successful detection of bilateral ventilation by auscultation, when completed as a single or dual-task. Differences in duration were adjusted for the effects of learning during the study, by accounting for the order of performance. A failed attempt was defined as any attempt exceeding 180 s, and for those cases, time measurement was uniformly set to 181 s. As this only happened to three participants (two of these failed the dual-task conventional tracheal intubation, one additionally failed the dual-task videolaryngoscopy tracheal intubation), no further statistical actions were taken. Secondary endpoints included the number of procedural errors (dental trauma indicated by click sounds by the manikin, omission of auscultation, endobronchial intubation, oesophageal intubation); and perceived workload as measured by the NASA-TLX. Multivariate regression was used to check for the influence of test individuals’ characteristics such as sex and level of expertise.
A total of 150 volunteers completed our experiment. Four balanced subgroups in relation to the individual level of professional expertise were formed: anaesthetic fellows and consultants (n = 41); anaesthetic registrars (n = 41); medical students (n = 37); and paramedics (n = 31). The mean (range) age of participants was 32.55 (19 to 57) years with 91 (61%) men.
Evaluation of the simulation scenario showed that 95% of the participants graded it as ‘entertaining’, 98% to be ‘fair’, 99% generally ‘challenging’ and 80% classified the type of distraction and stress as true to a real-life emergency.
The comparison of airway management interventions with and without a second task as a source of divided attention found a significant increase in the completion time for all three airway techniques. The calculation of specific period-effects demonstrated no influence of the sequence of tasks on the effects of dual-tasking with the mean (95% CI) absolute period effects for supraglottic airway 2.2 (1.2 to 3.3) s, conventional tracheal intubation 5.8 (2.0 to 9.5) s and video-assisted tracheal intubation 2.8 (−0.5 to 6.2) s.
Despite the obvious differences in the baseline performance of airway management between the four different professional groups, the negative effects of dual tasking were present for all levels of expertise (Figs 1 and 2). When given as a mean (95% CI) for the entire study population, insertion of the LMA took 6.4 (5.4 to 7.5) s longer, whereas participants needed additional 17.8 (14.1 to 21.6) s for a conventional tracheal intubation and 23.3 (19.9 to 26.6) s for a video-assisted tracheal intubation when compared with the single task (Table 2). The frequency of procedural mistakes differed between techniques. Although insertion of the supraglottic airway was rarely associated with errors in both the single and dual-task exercises, both conventional and video-assisted tracheal intubation were subject to more mistakes with a distinct increase of the number of mistakes under dual-task conditions (Table 3).
Dual-tasking led to increased workload ratings (NASA-TLX score) in all three techniques (Table 4; supplemental digital content figures X1-X3, http://links.lww.com/EJA/A73). For the single-task interventions, the supraglottic airway showed the lowest mean (95% CI) workload rating 14.3 (12.3 to 16.3) and video-assisted tracheal intubation was perceived as the most demanding technique with a workload rating of 24.8 (22.3 to 27.2). All three techniques showed a similar increase in workload rating under dual-task conditions of about 30 points. The increase in NASA-TLX under the influence of the dual-task settings can be seen for each airway technique and each level of expertise in Figs 3–5.
Adjustment for age did not show any significant effect. The correlated characteristics ‘level of professional expertise’ and ‘experience of previous simulator training’ were analysed in separate linear regression models that were adjusted for sex and period effect. The estimated mean increase in time consumption remained positive for all possible parameter values. Table 5 summarises the results of the two linear regression models that were adjusted by sex and period effect and that differ in the adjusted characteristic. That is, each regression model contains the dependent variables sex and period effect along with the adjusted characteristic, which is either ‘level of professional expertise’ or ‘experience of previous simulator training’. Increases in frustration according to level of experience and airway technique are shown in Figure 6.
Distractions and concurrent tasks may have a negative impact on the performance of healthcare providers in emergency situations.9 Designated training under dual-tasking seems to show promise as a tool to prepare both medical students and professionals for the conflict of performance versus divided attention arising from complex emergency situations. The published medical literature, however, does not provide a validated training scenario for dual-tasking.22,23 We, therefore, investigated a novel combination of educational tools for the simulation of dual-tasking by pairing airway management with the PASAT. As reported in the Results section, time consumption, the number of procedural mistakes and perceived workload all increased with dual-tasking for all the airway techniques investigated across all subgroups of professional expertise.
A perceived workload score of more than 50 on the NASA-LTX has been suggested as the threshold at which performance generally decreases.20,24 Under dual-task conditions, such a score was reached on a regular basis particularly for conventional and video-assisted tracheal intubation. Further analyses of the NASA-TLX data revealed the specific influences of our distraction model on certain subdimensions of workload. Although the physical demands of the tasks were largely unaltered, mental and temporal demands increased, as did the levels of effort required and the degree of frustration experienced. These profound influences on performance and workload across all disciplines demonstrate the effectiveness and validity of our training model. Every single participant was affected to some degree by our dual-task scenario.
Interestingly, placing a supraglottic airway took the least time, produced the fewest mistakes and was rated with the lowest workload scores of all techniques, under both single and dual-task conditions. Amongst all 150 participants, there was only one mistake registered with the single-task and only five with the dual-task. Even under dual-tasking, the supraglottic airway was successfully placed in less than 30 s in 69% of cases. These findings support the use of supraglottic airway devices in stressful or emergent working conditions as a fast, simple and robust airway technique.
The potential ongoing usefulness of our test may be evaluated by the consideration of its possible pros and cons. In addition to the demonstrated effectiveness, we consider the markedly positive evaluation of our test by the participants as its greatest advantage. Participants thought our test to be entertaining, challenging, fair and true to life; a combination of characteristics that seem derisible for the implementation of any test into already cramped medical teaching curricula. Our dual-tasking scenario is able to generate its effects for all participants regardless of individual factors such as age, sex, level of professional expertise or amount of previous simulator training. In contrast to the sources of distraction used in previous simulations such as music, television or actors, our use of the PASAT allows a reliable and standardised degree of divided attention, which is directly discernible through the rate of correct calculations. The implementation of the PASAT is not associated with significant financial outlay or logistical support, as just a loudspeaker is required for the exercise. A possible limitation of our study, which would have to be investigated separately, would be the use of the PASAT in simulations involving multiple candidates, wherein there maybe acoustic interference with the verbal aspects of the test.
In spite of these limitations, our simulation of dual-tasking seems to be a promising tool in medical education for students and professionals alike. Future work could explore whether dual-task training was able to enhance skill acquisition or mitigate the negative emotional experiences of stressful emergency scenarios. Admittedly, when compared with the aviation industry and the elaborate simulation scenarios available for the training of pilots and astronauts, our approach seems a little minimalistic.25,26 Nevertheless, it may serve as one of the first steps towards understanding and overcoming the limitations of medical practice that are based on human factors.
Our study introduces the PASAT as a reliable and standardisable source of divided attention to enhance the simulator training of medical emergencies. In our setting of airway management the PASAT was able to substantially increase the perceived workload and severely impact performance markers in terms of increased completion time and error rates. Our scenario was able to reproduce its effects across all evaluated individual factors such as sex, age, previous experience of simulation training and, most importantly, all levels of medical expertise. This makes it universally applicable across the training spectrum, from medical students to senior professionals. The PASAT is cheap and is simple in its technical and logistical aspects. Most importantly for an educational tool, it is highly acceptable among participants and rated as true-to-life for emergency situations. In spite of possible limitations in simulation scenarios involving several participants, our approach offers a novel possibility for education through simulation.
Acknowledgements relating to this article
Assistance with the study: none.
Financial support and sponsorship: none.
Conflicts of interest: none.
Presentation: this study was presented at Euroanaesthesia 2015 in Berlin, Germany.
1. Devitt JH, Kurrek MM, Cohen MM, Cleave-Hogg D. The validity of performance assessments using simulation. Anesthesiology
2. Ilgen JS, Sherbino J, Cook DA. Technology-enhanced simulation in emergency medicine: a systematic review and meta-analysis. Acad Emerg Med
3. St Pierre M, Scholler A, Strembski D, Breuer G. Do residents and nurses communicate safety relevant concerns? Simulation study on the influence of the authority gradient. Der Anaesthesist
4. Hunziker S, Pagani S, Fasler K, et al. Impact of a stress coping strategy on perceived stress levels and performance during a simulated cardiopulmonary resuscitation: a randomized controlled trial. BMC Emerg Med
5. Kennedy CC, Cannon EK, Warner DO, Cook DA. Advanced airway management simulation training in medical education: a systematic review and meta-analysis. Crit Care Med
6. Slagle JM, Weinger MB. Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center. Anesthesiology
7. Schulz CM, Skrzypczak M, Schneider E, et al. Assessment of subjective workload in an anaesthesia simulator environment: reliability and validity. Eur J Anaesthesiol
8. Flin R, Fioratou E, Frerk C, et al. Human factors in the development of complications of airway management: preliminary evaluation of an interview tool. Anaesthesia
9. Fernandez R, Compton S, Jones KA, Velilla MA. The presence of a family witness impacts physician performance during simulated medical codes. Crit Care Med
10. Bjorshol CA, Myklebust H, Nilsen KL, et al. Effect of socioemotional stress on the quality of cardiopulmonary resuscitation during advanced life support in a randomized manikin study. Crit Care Med
11. Bakker ADE. The job demands-resources model: state of the art. J Manag Psychol
12. Darling S, Della Sala S, Logie RH, Cantagallo A. Neuropsychological evidence for separating components of visuo-spatial working memory. J Neurol
13. Kahneman D. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall; 1973.
14. Tsang PS. Can pilots time-share better than nonpilots? Appl Ergon
15. Tombaugh TN. A comprehensive review of the Paced Auditory Serial Addition Test (PASAT). Arch Clin Neuropsychol
16. Hart SG, Staveland LE. Hancock PAM. Development of NASA-TLX (task load index): results of empirical and theoretical research. Human mental workload
. Amsterdam: Elsevier; 1988. 139–143.
17. Miyake S. Factors influencing mental workload indexes. J UOEH
18. Gronwall D, Wrightson P. Delayed recovery of intellectual function after minor head injury. Lancet
19. Gronwall DM. Paced auditory serial-addition task: a measure of recovery from concussion. Percept Motor Skills
20. Strauss ES, Spreen O. A compendium of neuropsychological tests: administration, norms and commentary. 3rd ed.New York: Oxford University Press; 2006.
21. Fos LA, Greve KW, South MB, et al. Paced visual serial addition test: an alternative measure of information processing speed. Appl Neuropsychol
22. Smyth M, Perkins GD. Stress and cardiopulmonary resuscitation performance. Crit Care Med
23. Seagull FJ. Human factors tools for improving simulation activities in continuing medical education. J Contin Educ Health Prof
24. Mazur LM, Mosaly PR, Hoyle LM, et al. Subjective and objective quantification of physician's workload and performance during radiation therapy planning tasks. Pract Radiat Oncol
25. (NASA) NAaSA. A review of human performance models for the prediction of human error. Washington, DC: NASA; 2001.
26. Petrosoniak A, Hicks CM. Beyond crisis resource management: new frontiers in human factors training for acute care medicine. Curr Opin Anaesthesiol
Supplemental Digital Content
© 2016 European Society of Anaesthesiology