Workload is a construct used to describe the extent to which an operator has engaged the cognitive and physical resources required for task performance (1). Workload is multidimensional and complex, being affected by external task demands, environmental and organizational factors, psychological factors, and current perceptual and cognitive capabilities. Workload assessment is important for evaluating cognitive requirements of jobs and for predicting workers’ capacity for additional tasks. Clinical workload may be important to the occurrence of and recovery from human error. An objective description of clinical workload may also aid in our understanding of anesthesia care’s task requirements, providing a more comprehensive basis for improvements in the design of anesthesia equipment, working conditions, and in anesthesia provider training.
Workload can be assessed concurrently in real-time using several different methods (2), including psychological (e.g., rating scales), procedural (response to task related demands), or physiological measures (changes in heart rate [HR], respiratory rate, galvanic skin response, etc.). Workload has been reported to be greatest during the anesthetic induction and emergence (2).1,2,3 During the use of transesophageal echocardiography, workload was found to be greater than during other monitoring or recording tasks (3). Kain et al. (4) replicated the preliminary findings of Toung et al.,1,4 showing that the increased HR response to the stress of providing anesthesia care was inversely related to clinical experience. Perhaps most telling, providers’ HR increased an average of 31% during critical incidents, and this increase was inversely correlated with provider experience.
As the anesthesia residents (trainees) progress through their first, second, and third clinical anesthesia years (CA1, CA2, and CA3, respectively), they gradually acquire the required skills and knowledge to function independently as a consultant anesthesiologist. Close interaction of faculty anesthesiologists or senior residents with inexperienced clinicians during actual patient care in an apprenticeship model is the standard approach to anesthesia training and is widely viewed as an essential component of the anesthesiology residency. However, we hypothesized that this hands-on patient care training adds to the clinical workload of the more experienced care provider (teacher) who is simultaneously responsible for safe and efficient anesthesia care during actual surgical procedures. To test this hypothesis, we measured the workload of anesthesia providers during clinical cases in which they were providing one-on-one supervision of a fourth-year medical student or first-month anesthesia resident (teaching cases) compared with similarly experienced clinicians providing anesthesia care on their own without the presence of a trainee (non-teaching cases). This study had two primary purposes: (a) to concurrently assess anesthesia providers’ clinical workload using multiple measures and (b) to assess the effects of teaching on clinical workload.
Methods
After approval by the University of California, San Diego Human Research Protection Program and acquisition of subjects’ written informed consent, 24 elective general anesthetic cases were studied. Research subjects included faculty anesthesiologists (n = 2), nurse anesthetists (n = 8), and resident anesthesiologists (n = 14;Table 1). With the exception of one subject who had a chronic disease (insulin-dependent diabetes mellitus) but was not on any cardiac or blood pressure medications, the subjects were cardiologically normal, did not have any systemic illnesses (e.g., coronary artery disease, hypertension, etc.), and were not taking any cardiac medications at the time of the study. All of the study cases were performed in the main operating rooms (OR) of the University of California, San Diego Medical Center or the Veterans Affairs San Diego Medical Center. Twelve of the cases were teaching cases in which the research subject who was the primary care provider was accompanied nearly continuously by either a first-month CA1 anesthesia resident (i.e., in July) or a fourth-year medical student doing an anesthesiology clerkship. In these cases, the subject assumed a primary teaching role during the case. In the other 12 nonteaching cases, the subject was the primary anesthesia provider with the presence of other anesthesia personnel restricted to attending anesthesiologist supervision (when appropriate) and transitions of care during relief breaks.
;)
Table 1: Case Demographics*
Immediately before each study case, the subject was attached to a 5-lead, 2-channel Holter monitor (Del Mar Avionics Model 456A, Irvine, CA). To ensure that the procedural and physiological data would correlate with the real-time events in the OR, the time on the display of the Holter monitor, the OR clock, and the time displayed on the laptop computer used for recording data were synchronized. During each case, a single trained observer (SR) sat in the OR and categorized the clinician’s activities into 37 possible tasks using established behavioral task analysis software (AnesLog version 2.6α) running on a Macintosh (Apple Computer, Cupertino, CA) laptop computer, as has been described previously (2,3,5). The observer sat in the OR facing the anesthesia workspace so that she had a clear view of the anesthesia provider without interfering with patient-care activities. Data collection commenced as the patient entered the room and was terminated when the patient left the room. Data collection was suspended when the subject left the OR on a break.
Workload density (3,6) was calculated at 1- and 5-min intervals by multiplying the duration of each task performed in that interval by a task-specific workload factor score (determined from a previous survey (6)). Manual tasks, such as laryngoscopy (workload factor score of 2.84), typically entail larger workload than do, for example, observing (1.05 ± 0.4) or conversing (0.95 ± 0.36) tasks. Intraoperative didactic teaching was previously rated by 115 anesthesia providers to yield a larger workload (2.51) than many other tasks (6). The resulting data are a continuous, real-time, quantitative description of procedural workload for each case.
At 9- to 14-min random intervals, vigilance latency was assessed by noting the response time to a small, bright-red (1-cm diameter) light that was positioned in close proximity to the physiological monitors within the anesthetic workstation (2,3). Subjects were instructed to respond either verbally or with a manual indication as soon as they detected the illumination of the light. Vigilance latency (in seconds) was measured from the time that the observer switched on the vigilance light (after being prompted by the computer) to the time that the subject indicated detection (either verbally or with a hand signal) of the illuminated light. The response latency (reaction time) to this alarm was used as an indirect measure of workload or spare capacity, based on the well-documented observation that lower priority tasks are neglected during periods of large workload (2,3,7).
At 7- to 15-min random intervals, psychological workload was measured using a Borg Workload scale (8) that ranged from 6 (e.g., completely sedentary subject) to 20 (e.g., during a full-blown OR resuscitation) (2,3). Subjects were explicitly advised during the prestudy briefing that the typical workload rating at the time of a routine oral asleep intubation was approximately 12. The computer first prompted the observer to record her rating of the subjects’ psychological workload rating and then to query the subject for his or her self-rating.
Physiological workload was assessed by measuring the subject’s HR (4,9,10). HR was measured continuously and then averaged every minute and every 5 min. The Holter monitor tapes were analyzed using a Del Mar Avionics Stratascan Model 563 system with commercial analysis software (version F, Del Mar Avionics). An experienced Holter monitor analyst processed all of the Holter data using standard clinical protocols. Maximum, minimum, and mean HR, and its standard deviation, for each epoch were recorded. Uninterpretable data (e.g., artifact) were excluded, and at least 65% of the beats in each epoch needed to be interpretable for that data point to be included in the analysis.
To permit aggregation and comparison of cases of differing lengths, data were segregated into identified benchmark time points within each case (relative to the time of initiation of endotracheal intubation and of extubation). HR, workload density, and vigilance latency were analyzed with two-way mixed analysis of variance (teaching versus case time). A separate two-way analysis of variance of 1-min HR values explored the effects of provider experience against case time. Psychological workload was analyzed with the Mann-Whitney U-test. Data are presented as mean ± standard error of the mean (SEM) and P values <0.05 were considered statistically significant.
Results
The 12 teaching and 12 nonteaching cases are summarized in Table 1. Subjects averaged 6 ± 1 yr of experience (4 ± 2 yr for teaching and 9 ± 2 yr for non-teaching; P = 0.14). The mean case start time for all cases was 9:13 am ± 22 min (i.e., no significant difference between teaching and nonteaching cases). The average duration (time patient was in the OR) was 175 ± 20 min for teaching and 125 ± 12 min for nonteaching cases (P = 0.046). For the 22 cases in which the subject was a nonfaculty anesthesia care provider, the attending was present in the OR during 31% ± 10% of teaching and 30% ± 11% of nonteaching cases (P = 0.63). The patient ASA status was 2.3 ± 0.2 for teaching and 1.9 ± 0.2 for nonteaching cases (P = 0.14). With the exception of longer case duration in teaching cases, there were no statistically significant differences in attending presence, patient, case, or provider attributes between the two study groups.
There was appreciable intra- and intersubject variability in alarm-light response. Alarm-light detection was consistently slower in teaching than in nonteaching cases throughout the entire anesthetic (Fig. 1). The response latency was significantly different between teaching and nonteaching cases during the induction (97 ± 19 versus 44 ± 12 s; P < 0.01) and emergence (75 ± 20 versus 24 ± 7 s; P < 0.01) but not maintenance (50 ± 6 versus 20 ± 2; P > 0.05). In teaching cases, vigilance latency during the induction was significantly more than during maintenance (P < 0.01), but this was not the case during nonteaching cases.
Figure 1.:
The response latency (in seconds) of anesthesia providers to detect the illumination of an alarm light (a measure of vigilance, see text for details) is shown for various time points during anesthesia cases studied (I-0 = time of intubation, I+10 = 10 min after intubation, M-0 = midpoint between intubation and extubation, M+10 = 10 min after M-0, and E-0 = time of extubation). Over the entire case, vigilance was significantly impaired in teaching (▪) compared with nonteaching (□) cases (*P < 0.01). Due to significant intra- and intersubject response variability, there were no significant differences at individual time points.
Both self-reported and observer-scored workload values were significantly less for teaching cases than nonteaching cases during the induction but not maintenance or emergence. In teaching cases, clinician self-reported (Self) and observer-scored (Obs) workload were larger during the induction (10.1 ± 0.4 Self; 10.5 ± 0.3 Obs) and emergence (10.8 ± 0.3 Self; 11.1 ± 0.3 Obs) than during maintenance (8.5 ± 0.3 Self; 8.4 ± 0.1 Obs). Similarly, in nonteaching cases, workload was larger during the induction (11.3 ± 0.5 Self; 11.4 ± 0.3 Obs) and emergence (11.1 ± 0.5 Self; 10.5 ± 0.4 Obs) compared with maintenance (8.8 ± 0.2 Self; 8.8 ± 0.1 Obs).
The clinicians’ HR values were the most rapid during the induction, decreased in maintenance, and increased again during emergence (Fig. 2). HR did not differ significantly between teaching and nonteaching cases. However, across all cases, clinicians of different levels of experience differed in their HR responses. Junior residents (those with <2 yr of training) had significantly faster minute-by-minute mean HR than experienced (≥5 yr) clinicians during both the induction and emergence (Fig. 3). During the anesthetic induction, the junior residents’ HR increased acutely and was at that time also more than that of senior residents.
Figure 2.:
Five-minute epochs of minimum (○), mean (▴), and maximum (□) clinician heart rate (HR) are depicted for time points throughout the anesthetic for teaching (open symbols) and non-teaching (filled in symbols) cases. Mean and maximum (but not minimum) HR values at the time of intubation (I-0) or 10 min thereafter (I+10) were generally increased compared with those during the maintenance phase (M-0 and M+5; † or §P < 0.05). Although clinicians’ HR tended to increase again during emergence, this did not attain statistical significance. Note: E-10 and E-5 are 10 and 5 min, respectively, before extubation. † = significantly different from HR at I-5 and I-0 (all cases), P at least <0.05; § = significantly different from HR at I-0 only all cases, P < 0.05. There was significant effect of teaching.
Figure 3.:
Mean heart rate (HR) is presented in 1-min intervals for all of the study cases (both teaching and nonteaching) for three different cohorts of anesthesia providers based on years of clinical experience. First and second (CA1 and CA2) residents (○) have the most rapid HR throughout the case and a noticeable peak in provider HR at the time of patient intubation. CA3 residents (□) had intermediate HR values, with a significantly lower HR peak at the time of patient intubation compared with junior residents (†P < 0.05). Experienced clinicians (>5 yr of experience; ▴) had the lowest HR values throughout the anesthetic that were significantly less (*P < 0.05; §P < 0.01) than the HR of inexperienced providers at many time points during both the induction and emergence. Because cases are all of different lengths, only the 30 min surrounding intubation (Intub; 10 min before and 20 min after; open symbols) and the 30 min surrounding extubation (Extub; 20 min before and 10 min after; filled in symbols) were included in the aggregate data displayed.
In nonteaching cases, the calculated workload density was higher during the induction than during maintenance and emergence (Fig. 4). In teaching cases, workload density was more variable and did not differ over the course of the anesthetic. After intubation, workload density was consistently more in teaching than in nonteaching cases (P < 0.05).
Figure 4.:
Workload density was calculated in 5-min intervals and is displayed for teaching (▪) and nonteaching (□) cases. In nonteaching, but not in teaching cases, there was a significant decrease in workload density after the induction and intubation (§; P < 0.05 compared with I = 0) that persisted for the remainder of the cases. Workload density was significantly increased in teaching when compared with nonteaching (*P at least <0.05 at all time points) cases beginning 5 min after intubation (I+5) and ending 5 min before extubation (E-5).
Discussion
This study describes the results of concurrently measuring clinical workload during anesthesia care using several different workload assessment tools derived from human factors research in other domains (1,7,8,10–15). The results replicate previous work from our laboratory (2,3,6,16)2 and by our colleagues (17,18)3 showing that during the administration of general anesthesia, in routine elective cases: (a) the induction is a period of large (but variable) workload; (b) the maintenance period is generally a period of small workload; and (c) there is a second peak in workload during emergence. More importantly, this study uniquely documents the effects of intraoperative teaching on the clinical workload and vigilance of the anesthesia instructors. This study’s results also corroborate and expand upon the recent report of Kain et al. (4), demonstrating a physiological stress response of anesthesia providers that parallels clinical workload. Together, our work and Kain et al.’s research validates the early non–peer-reviewed reports of Toung et al.,1,4 showing that this physiological stress response during the administration of anesthesia is partially mitigated by clinical experience.
There were significant effects of teaching and of phase of anesthetic care (at least in teaching cases), but not of level of experience, on the latency of response to the alarm light. The nonteaching response latencies observed were consistent with that seen in a previous study of experienced residents who were not actively teaching (3). The response to the illumination of an alarm light placed in the anesthesia-monitoring array has been described as both a measure of vigilance and of workload. However, it meets all of the criteria of a classic vigilance probe. The alarm-light probe occurred randomly, had a small response burden, and required clinician alertness, selection of information, and conscious effort to detect and respond (19). Anesthesiologists’ vigilance to auditory (20) and visual (2,3,17,18) alarm cues has been studied previously.
However, the alarm light can also be considered a procedural workload technique because it represents a secondary task probe embedded in normal workflow (21). As such, it is a measure of “spare capacity”—a clinician who is totally absorbed in other tasks (i.e., high workload) will be much less likely to detect or note the light’s illumination. A limitation of this probe is that if the response is delayed, it is difficult to distinguish between failed perception, lack of recognition, or failed response (e.g., too busy). Thus, these data must be interpreted cautiously with consideration of the clinical context and the results of other concurrent task probes.
Consistent with previous studies (2,3), clinician self-reported and observer-scored workload were greater at the time of the induction and emergence compared with the maintenance period. In addition, the workload was larger during induction for nonteaching cases than in teaching cases. The lower psychological workload values in teaching cases during induction may be because of a bias of both the primary care provider and the observer in which cognitive (e.g., supervisory and teaching) tasks were undervalued compared with the manual tasks of the induction (e.g., laryngoscopy and intubation). In contrast to our previous studies (2,3), the clinicians’ self-reported workload ratings in this study were not more than those of the trained observer. This may be because of the criterion used by this particular observer. A future study is required to formally examine the inter- and intra-rater reliability of this and other subjective workload measures for use in anesthesia. We have worked with Westenskow et al. to use a modification of the NASA-TLX workload metric (22) to assess anesthesiologist workload after realistic patient simulations (23).
The anesthesia providers’ HR responses paralleled increases in psychological workload and are consistent with previous results in emergency medicine physicians (24) and in pilots (14). These findings also confirm the inverse relationship between the physiological stress response and clinician experience first noted by Toung et al.,1,4 and more recently elucidated by Kain et al. (4). During the induction, the HR response of junior residents was not only increased compared with that of experienced clinicians, but was also even more than that of residents with only a year or two greater clinical experience. Because none of the subjects were on HR-altering medications (and there were no differences in caffeine use between groups), this likely represents a real effect of clinical experience on the stress response to the provision of anesthesia care. However, due to logistical constraints, we were unable to place the Holter monitors on our subjects sufficiently before the start of their clinical day to capture true baseline HR values.
This study has many limitations. The between-subjects design and small number of cases studied, which stemmed from logistical and resource constraints, may have introduced confounds caused by variability in provider experience, case type or duration, or other factors that were not controlled. In three of the teaching cases, the subject was a first-year resident who was delivering anesthesia care while teaching a fourth-year medical student. The presence of inexperienced residents in these teaching cases could have biased the study’s results. However, even with the data from these three CA-1 teaching cases excluded, all of the principle findings, including the significant effects of teaching on workload and vigilance, were sustained. A future study using a within-subjects experimental design whereby the same experienced clinician is studied twice doing similar cases, once while teaching and once when not teaching, would be more rigorous and illuminating, especially if the case order could be randomized. It will also be important to examine further the relationship between workload and the occurrence of adverse events (4).
Although the data were collected from three different hospitals, the results reflect the clinicians in a single academic anesthesia department and thus may not generalize to all anesthesia providers. Differences in the nature of the cases studied in the two groups might have affected the results. The teaching cases were significantly longer in duration than the non-teaching cases, and in addition, the two groups may have differed in other attributes such as the type of surgical procedure performed. Because true resting HR was not obtained, analyses cannot reflect actual increases or decreases from a set baseline HR. Artifact in electrocardiogram readings caused by movement or noise in OR environment may have affected HR analysis, although the analysis software was specifically designed to exclude such artifacts.
Accurate assessment of clinician workload is an important step in identifying vulnerabilities in complex health care systems. By understanding how various stressors impact clinical performance, experts can better design systems and training to prevent adverse effects, thus improving safety. In this study, all of the workload measures reflected apparent changes in anesthesia providers’ workload during different phases of routine anesthesia cases. Psychological, physiological, and procedural workload measures documented the increased work demands of induction and emergence. Maintenance generally was a period of low workload. Intraoperative teaching increased workload density and may have impaired vigilance. The fact that clinicians who were teaching responded more slowly to the alarm light suggests that intraoperative teaching can distract from other clinical tasks. Contrary to our a priori hypothesis, clinicians’ HRs did not differ significantly between teaching and nonteaching cases. Different workload measures may measure different types of workload or be more sensitive under different task conditions.
The authors would like to extend their gratitude to all the subjects who participated in this study. The assistance of Dr. David Gonzales and Yvette Rascon were instrumental to the completion of this study. Dr. Amy Bronstone provided invaluable editorial assistance. Dr. Sonia Jain reviewed and improved our statistical analysis. Previous pilot work conducted by Shakha Vora, Meredith Culp, Neide Fehrenbacher, and Dr. Oliver W. Herndon was essential to the current project. We gratefully acknowledge the advice and efforts of Robert Worth, CCVT in the performance of the Holter monitor analysis.
References
1. Backs RW, Ryan AM, Wilson GF. Psychophysiological measures of workload during continuous manual performance. Hum Factors 1994;36:514–31.
2. Weinger MB, Herndon OW, Paulus MP, et al. Objective task analysis and workload assessment of anesthesia providers. Anesthesiology 1994;80:77–92.
3. Weinger MB, Herndon OW, Gaba DM. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology 1997;87:144–55.
4. Kain ZN, Chan K-M, Katz JD, et al. Anesthesiologists and acute perioperative stress: a cohort study. Anesth Analg 2002;95:177–83.
5. Slagle J, Weinger MB, Dinh MT, et al. Assessment of the intra-and inter-rater reliability of clinical task analysis methodology. Anesthesiology 2002;96:1129–39.
6. Vredenburgh AG, Weinger MB, Williams KJ, et al. Developing a technique to measure anesthesiologists’ real-time workload. Proceedings of the IEA 2000/HFES 2000 Congress 2000;44:4241–4.
7. Harris RL, Tole JR, Stephens AT, Ephrath AR. Visual scanning behavior and pilot workload. Aviat Space Environ Med 1982;53:1067–72.
8. Borg G. Simple rating methods of perceived exertion. In: Borg G, ed. Physical work and effort. Oxford, England: Permagon Press, 1977:39–47.
9. Kakimoto Y, Nakamura A, Tarui H, et al. Crew workload in JASDF C-1 transport flights. I. Change in heart rate and salivary cortisol. Aviat Space Environ Med 1988;59:511–6.
10. Wierwille WW. Physiological measures of aircrew mental workload. Hum Factors 1979;21:575–93.
11. Casali JG, Wierwille WW. On the measurement of pilot perceptual workload: a comparison of assessment techniques addressing sensitivity and intrusion issues. Ergonomics 1984;27:1033–50.
12. Hicks TG, Wierwille WW. Comparison of five mental workload assessment procedures in a moving-base driving simulator. Hum Factors 1979;21:129–43.
13. Kankowitz BH, Casper PA. Human workload in aviation. In: Wiener EL, Nagel DC, eds. Human factors in aviation. San Diego: Academic Press, 1988:157–87.
14. Tattersall A, Hockey G. Level of operator control and changes in heart rate variability during simulated flight maintenance. Hum Factors 1995;37:682–98.
15. Wierwille WW, Rahimi M, Casali JG. Evaluation of 16 measures of mental workload using a simulated flight task emphasizing mediational activity. Hum Factors 1985;27:489–502.
16. Weinger MB, Vredenburgh AG, Schumann CM, et al. Quantitative description of the workload associated with airway management precedures. J Clin Anesth 2000;12:273–82.
17. Loeb RG. Monitor surveillance and vigilance of anesthesia residents. Anesthesiology 1994;80:527–33.
18. Loeb RG. Manual record keeping is not necessary for anesthesia vigilance. J Clin Monit 1995;11:9–13.
19. Weinger MB, Englund CE. Ergonomic and human factors affecting anesthetic vigilance and monitoring performance in the operating room environment. Anesthesiology 1990;73:995–1021.
20. Cooper JO, Cullen BF. Observer reliability in detecting surreptitious random occlusions of the monaural esophageal stethoscope. J Clin Monit 1990;6:271–5.
21. Bortolussi M, Hart S, Shively R. Measuring moment-to-moment pilot workload using synchronous presentations of secondary tasks in a motion-based trainer. Aviat Space Environ Med 1989;60:124–9.
22. Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds. Human mental workload north. Holland: Elsevier Science Publishers, 1988:139–83.
23. Syroid ND, Agutter J, Drews FA, et al. Development and evaluation of a graphical drug display. Anesthesiology 2002;96:565–75.
24. Adams SL, Roxe DM, Weiss J, et al. Ambulatory blood pressure and holter monitoring of emergency physicians before, during, and after a night shift. Acad Emerg Med 1998;5:871–7.
1 Toung TJK, Donham RT, Rogers MC. The stress of giving anesthesia on the electrocardiogram of anesthesiologists (abstract). Anesthesiology 1984;61:A465.
Cited Here
2 Weinger MB, Shen H, Culp M, et al. Real-time workload assessment during anesthesia for outpatient surgery (abstract). Anesth Analg 1995;80:S548.
Cited Here
3 Dutton RP, Xiao Y, Bernhard W, et al. Measured versus predicted stress during elective and emergency airway management (abstract). Anesthesiology 1997;87:A444.
Cited Here
4 Toung TJK, Donham RT, Rogers MC. The effect of previous medical training on the stress of giving anesthesia (abstract). Anesthesiology 1986;65:A473.
Cited Here
