Facilitated Nurse Medication-Related Event Reporting to Improve Medication Management Quality and Safety in Intensive Care Units : Nursing Research

Secondary Logo

Journal Logo


Facilitated Nurse Medication-Related Event Reporting to Improve Medication Management Quality and Safety in Intensive Care Units

Xu, Jie; Reale, Carrie; Slagle, Jason M.; Anders, Shilo; Shotwell, Matthew S.; Dresselhaus, Timothy; Weinger, Matthew B.

Author Information
Nursing Research 66(5):p 337-349, September/October 2017. | DOI: 10.1097/NNR.0000000000000240


Medication safety presents an ongoing challenge for clinicians working in complex, fast-paced, critical care environments. Prior research suggests that medications are involved in most intensive care unit (ICU) patient safety incidents. In one study, Rothschild et al. (2005) found that medications were responsible for 78% of serious errors in the ICU. Reported rates of ICU medication-related errors that either caused patient harm (i.e., preventable adverse drug events [ADEs]) or had the potential to cause patient harm (i.e., near misses) range from 9.2 to 12.8 per 1,000 patient-days and from 116.8 to 276 per 1,000 patient-days, respectively (Carayon et al., 2014; Rothschild et al., 2005). Carayon et al. (2014) found an average of 2.9 preventable or potential ADEs per ICU patient admission.

The complexity of the ICU work environment and associated high nursing workloads may contribute to medication errors and vice versa. Critical care nurses routinely juggle multiple cognitive and physical tasks, reflected in different dimensions of workload (e.g., physical, cognitive, emotional) under time pressure. Increased workload can adversely affect providers’ quality of work life as well as the quality and safety of care in the ICU (Carayon & Gürses, 2005). Seynaeve et al. (2011) reported a significant association between ICU nursing workload and the occurrence of ADEs. Increased nursing workload is an important factor associated with medication administration errors (Frith, 2013).

An important patient safety intervention is the creation of robust reporting systems to enable the healthcare system to learn from its mistakes (World Health Organization, 2005). However, traditional quality assurance programs and voluntary medication event reporting systems have important limitations. For example, most of these reporting systems only capture events that led to the occurrence or near occurrence of adverse outcomes, ignoring myriad other events that may also be harbingers of unsafe processes or conditions (Slagle et al., 2015). An alternative is the construct of “nonroutine events” (NREs) that affords an efficient method for capturing a broad range of potentially dangerous conditions and process improvement opportunities (Oken et al., 2007; Weinger & Slagle, 2002). An NRE is defined as, “any event that is perceived by care providers or skilled observers to be unusual, out-of-the-ordinary, or atypical” (Weinger & Slagle, 2002). NRE reporting has been established as a valuable methodological approach for identifying patterns of patient quality and safety risks, as well as guidance on what might have gone awry (Oken et al., 2007; Weinger & Slagle, 2002; Weinger, Slagle, Jain, & Ordonez, 2003). It allows for the capture and analysis of additional information about the underlying clinical system and work processes without the negative connotations and biases associated with “medical errors,” thus increasing the likelihood of such events being reported and providing opportunities for problem identification and proactive solutions or interventions to prevent future-related suboptimal deviations or events (Weinger & Slagle, 2002). In fact, contemporaneous reporting of NREs yields far more events and a higher incidence of injury events than do traditional hospital reporting systems (Oken et al., 2007).

We sought to investigate the incidence and nature of nurse-reported, medication-related events (MREs) in the ICU. MREs are a subset of NREs defined as, “any event involving the medication process that deviated from optimal care for a specific patient.” Conceptually, MREs include medication errors (i.e., any preventable event that may lead to inappropriate medication use, which may result in patient injury (Gandhi, Seder, & Bates, 2000) and ADEs (i.e., any patient injury resulting from drug-related medical intervention (Gandhi et al., 2000; see Figure 1). MREs may also include events that do not meet the definition of either medication errors or ADEs. Examples of such events include the following: a medication may be not available in the automated dispensing machine within the scheduled time because of a delay in medication delivery from the pharmacy; and the computer system may temporarily lose Internet connection so that the nurses are not able to obtain up-to-date patient information. These events are not “near misses” as they do not have a clearly identifiable path for patient injury. Nonetheless, capture and analysis of such MREs can yield information about system latent errors that in the future could cause patient injury under other circumstances (Kohn, Corrigan, & Donaldson, 2000; Weinger et al., 2003).

Conceptual relationships among nonroutine events, medication-related events, medication errors, and adverse drug events.


The overall goal of this study was to explore the utility of facilitated MRE reporting in identifying system deficiencies and the relationship between MREs and nurses’ work in the ICU. Facilitated MRE reporting means that the MRE data are collected via interviews by trained interviewers using a survey instrument (Oken et al., 2007). We investigated nurses’ work in terms of the activities that the nurses performed and the nurses’ perception about that work. This study investigated the relationship between the occurrence of MREs and workload, because workload is considered to be a major contributor to patient safety in the ICU among various work factors (Carayon, Alvarado, & Systems Engineering Initiative for Patient Safety, 2007; Carayon & Gürses, 2005; Ream et al., 2007). The potential association between MRE occurrence and nurses’ negative moods was also examined because negative moods have been reported to affect nurses’ teamwork (Chang, Teng, Chu, Chang, & Hsu, 2012), job performances (Stewart & Barling, 1996), and burnout (He, Chen, Zhan, & Wu, 2014). The study was designed to answer four specific research questions (RQs):

  1. What is the frequency of nurse-reported MREs in the ICU?
  2. What are the characteristics (e.g., medication management phases in which they occurred, contributory factors, and reported immediate patient impacts) of the reported MREs?
  3. What is the relationship between MREs and nurses’ activities, including task volume and percentage time spent on different tasks?
  4. Is there an association between MREs and nurses’ perceived workload and negative moods?



This was an observational study where a single trained researcher observed and collected data from nurse participants in the ICU. A nurse researcher who was experienced in the research method used in this study trained the observer in 10 guided 2-hour practice observations.


Power analysis was conducted based on the sample size needed to detect a “medium” (d = 0.5; Cohen, 1992) difference in a continuous variable for observations with and without MREs. The percentage of observations that would contain at least one MRE was estimated to be 35% based on previous research (Oken et al., 2007; Slagle et al., 2015). The results from G*Power software (Faul, Erdfelder, Lang, & Buchner, 2007) recommended a sample size of N = 152.

This study was conducted in three ICUs in three different teaching hospitals located in California: two medical-surgical adult ICUs and one medical-surgical pediatric ICU. The two adult ICUs had 18 and 13 beds; the pediatric ICU had 24 beds. All of the ICUs had nurse-to-patient staffing ratios of at least 1:2, with 1:1 ratios for complex patients.

Nurses were recruited by communications disseminated by unit leadership prior to data collection and by the observer on the day of observation. Data were collected over a 7-month period. Purposeful sampling was used to ensure that the observations were conducted across different times of day (morning, afternoon, and after-hours) and observation sites. The observer strived to observe as many different nurses as possible. Each hospital’s institutional review board approved the protocol of this study.

Data Collection

In each observation, the observer shadowed one nurse participant for data collection. At the beginning of the observation, written informed consent and self-reported workload, negative mood, and demographic information were obtained from the nurse.

The observation was conducted based on behavioral task analysis (BTA), a formal structured observation technique that provides quantitative measures of work processes and other attributes of clinical performance (Fraind, Slagle, Tubbesing, Hughes, & Weinger, 2002). The BTA was conducted according to a categorization scheme, which defined nurses’ activities as 59 different tasks in 12 task categories (including medication, direct patient care, documentation/reading, administration, observation, conversation, assistance, teaching/learning, housekeeping, transportation, personal, and miscellaneous). The scheme was generated based on prior schemes used in the ICU (Wong et al., 2003) and the operating room (Fraind et al., 2002; Slagle, Weinger, Dinh, Brumer, & Williams, 2002; Weinger, Herndon, & Gaba, 1997). Through an iterative process—which refined and validated each task, task definition, and category—the final task categories were specific to ICU nursing tasks. The observer continuously recorded the times and durations of all of the nurse’s activities on a touch-screen tablet computer via custom BTA software. The software automatically logged the task and time initiated. The software also supported the recording of multitasking by allowing the observer to specify a period of time when concurrent tasks were performed.

The intended duration of the observations was 4 hours. At the end of the observation period, the nurse reported their workload, negative mood, and any MREs that had occurred as described in more detail below.


MRE (MRE Occurrence and MRE Reports)

At the end of each observation, MRE information was collected via a structured interview with the nurse participants (see Document, Supplemental Digital Content 1, https://links.lww.com/NRES/A263). The brief interview guide—modified from a more general previously used instrument (Oken et al., 2007)—consisted of open-ended probes designed to elicit information about any MRE that might have occurred. MRE occurrence was a binary variable that was positive regardless of the number of MREs reported in an observation. MRE reports were the participants’ narrative responses that were used for further qualitative analysis of the nature of the MREs.

Task Volume

Task volume (TV) of a task (or a task category) is defined as the number of tasks performed per hour and was calculated as the observed number of instances of a task (or all the tasks within a task category) divided by observation duration.

Total Task Volume

Total task volume (TTV) was the sum of the TVs of the 12 task categories. TTV was used as an indicator of observed workload on the situational level (i.e., workload within a certain time period), which is different from workload on the unit level (i.e., nurse/patient ratio), the job level (i.e., workload required by the job characteristics of ICU nursing), or the patient level (i.e., general care requirements based the condition of a patient; Carayon & Gürses, 2005).

Percentage of Time on Task

Percentage of time (PT) spent on a task (or a task category) was calculated as the total time that a nurse spent on a task (or all the tasks in a task category) divided by the observation session duration.

Perceived Workload (Preobservation Workload, Postobservation Workload, and Workload Change)

Nurse’s perceived workload at the beginning and the end of each observation was measured by the NASA Task Load Index, a well-validated, six-item workload scale based on a robust conceptual model (Hart & Staveland, 1988) and has been used extensively to measure clinicians’ workloads (e.g., Horner et al., 2011; Mazur, Mosaly, Hoyle, Jones, & Marks, 2013; Mohammadi, Mazloumi, Kazemi, & Zeraati, 2015). The Task Load Index includes six dimensions of workload: mental demand, physical demand, temporal demand, performance, effort, and frustration. For each dimension, the nurses rated their current status from 0 = lowest to 9 = highest. An overall score is calculated by summing the ratings of the six dimensions. The score indicates perceived workload on the situational level (Carayon & Gürses, 2005). Workload change was calculated by subtracting the preobservation from the postobservation workload score.

Mood (Preobservation Negative Mood, Postobservation Negative Mood, and Negative Mood Change)

Pre- and postobservation negative mood was measured using a 13-item modified version of the Profile of Mood States Scale (Fraind et al., 2002; Slagle et al., 2002). Eleven of the items were negative dimensions (e.g., stressed), and two items were positive dimensions (e.g., relaxed). The two positive mood items are reverse scored. The final score of the measure indicates negative mood. For each item, the subject rated their current status on that dimension on a 10-point scale from 1 = not at all to 10 = very much. Mood change was calculated by subtracting preobservation from the postobservation negative mood score.

Data Analysis

Missing Data

Missing data occurred in the workload and negative mood data: 35 observations (28%) had missing data on at least one of the four pre- or postobservation variables. A multiple imputation procedure was applied to the data set using the Amelia II package (Honaker, King, & Blackwell, 2011) in R (R Core Team, 2017). Thirty-five imputations were performed based on the rule of thumb that the number of imputations should be at least equal to the percentage of observations with incomplete data (White, Royston, & Wood, 2011).

Statistical Models

All the statistical analyses were conducted using the R platform (R Core Team, 2017). Linear mixed-effects (LME) and generalized linear mixed-effects (GLME) models incorporating random intercepts of study sites and nurses were used to account for the random effects of those two variables (Bates et al., 2017; West, Welch, & Galecki, 2014). For the fixed effects, all the variables were centered and all the possible two-way interaction terms were included. The Satterthwaite approximation for degrees of freedom was used to assess statistical significance in the LME models (Kuznetsova, Brockhoff, & Christensen, 2014). When missing data were involved, the LME/GLME model was fitted to each imputed data set, and the results were combined using Rubin’s rules (Rubin, 1987).

Analysis for RQ1

Observations that contained at least one reported MRE were identified as MRE-containing observations. Number of MREs reported in each MRE-containing observation was also counted.

Analysis for RQ2

The MRE reports were qualitatively analyzed and event category, event type, medication management process phase, contributory factors, patient outcome, and patient impact severity level were coded for each event based on the reports. Three subject matter experts (including a board-certified anesthesiologist and patient safety expert, an experienced registered nurse, and a human factors engineer) independently coded the data, and then consensus was achieved. Event category and patient outcome was coded inductively with the aim of organizing data into meaningful groups while maintaining essential details. Other codings were done in a deductive fashion with predefined coding schemes. Medication management phases were based on a previously published scheme (Carayon et al., 2014; Pingenot, Shanteau, & Sengstacke, 2009): ordering (i.e., physician orders the medication), dispensing (i.e., pharmacy prepares and delivers the medication to the ICU), stocking (i.e., medication become available in the medicine cabinet or automated dispensing machine), administering (i.e., nurse prepares and administers the medication to the patient), and monitoring (i.e., nurse or other providers monitor the patient for effects of the medication). The types of medication events included medication errors, ADEs, near misses, and MREs that were neither medication errors nor ADEs. Contributory factors were coded using elements of the healthcare work system (Carayon et al., 2006): patient (e.g., unexpected reaction to therapy), provider (e.g., pharmacists and physician actions or inactions), team (e.g., communication failures), task/process (e.g., deficiencies in medication dispensing and stocking), technology (e.g., usability and technical issues with electronic health record systems or infusion pumps), environment (e.g., distracting noises), and organization (e.g., lack of training provided by the organization). Patient impact severity level was coded as five categories: none, mild, moderate, severe, and death.

Analysis for RQ3

Quantitative indicators of nurses’ activities, including TTV and each task category’s TV and PT, were calculated. The correlations between these indicators and MRE occurrence were tested with LME models, controlling for shift type, and preobservation workload and negative mood.

Analysis for RQ4

First, the effects of preobservation workload and negative mood on MRE occurrence were tested using a GLME model, controlling for shift type. Second, the effects of MRE occurrence on workload change and negative mood change were tested in LME models, controlling for shift type and preobservation workload and negative mood.

More details on the statistical analysis and outputs are available in Document, Supplemental Digital Content 2, https://links.lww.com/NRES/A264.


Sample Characteristics

A total of 153 observations were collected from 109 nurses. Twenty-nine observations were excluded because nurse shift-to-shift handoffs occurred during these observations so that they had unique characteristics and complexities. Thus, we analyzed 124 observations conducted during either the day shift (n = 98; 8 AM to 5 PM) or night shift (n = 26; 10 PM to 4 AM). The mean observation duration was 194 minutes (range = 125–248, SD = 30). Eighty-six nurse participants were included in the analysis. The numbers of participants in each of the three ICUs were 17, 36, and 34 (with one nurse working at two sites). All the participants were registered nurses. The mean years of experience of the nurses were 12.7 (range = 0.3–35, SD = 9.3). Sixty-six nurses were female (77%). The mean age of the nurses was 38.5 years (range = 23–59, SD = 8.9). Sixty (70%) nurses were observed only once, 23 (27%) were observed on two or three separate occasions, two nurses were observed on four separate occasions, and one nurse was observed five times.

RQ1: Frequency of MRE-Containing Observations

MREs were reported in 44 of 124 observations (35%). MREs were reported at similar rates on day shift (37%) compared to night shift (31%; adjusted odds ratio = 0.45, 95% CI [0.10, 2.08], p = .30; see Table 1, Model 1 for model estimates for this analysis). A total of 60 MREs were reported in the 44 MRE-containing observations (1.4 MREs reported per MRE-containing observation). In the 36-day-shift observations with MREs, a total of 50 MREs were reported (1.4/observation). On the night shift, 10 MREs were reported in the eight MRE-containing observations (1.3/observation).

Combined Model Estimates From Models Fitted to Multiple Imputation Datasets

RQ2: MRE Descriptions, Contributory Factors, and Patient Impact

Table 2 provides a summary of the results from the qualitative analysis. Table 3 shows five examples of MRE report summaries and their corresponding coding. Iterative coding of MRE event descriptions yielded 19 event categories. The most frequent MRE categories were medication not available (30%), unexpected response to therapy (18%), and medication delivery route disrupted (12%). All other MRE categories occurred in less than 10% of the MREs.

Medication-Related Event Descriptions, Contributory Factors, and Patient Impacts
Examples of MRE Report Summaries and Codes

MREs occurred in all five phases of medication management, although they were most common (33% of all MREs) during the medication administration phase. Other the medication management phases with a substantial amount of the MREs were stocking (25%) and monitoring (20%). All but one of the MREs categorized as occurring in the monitoring phase were unexpected responses to therapy (11 events); the other was a medication delivery route disruption due to infusion pump problem.

Eleven MREs (18%) were either ADEs (n = 7) or medication errors (n = 4). None of the ADEs appeared to be preventable. All four medication errors were reported during day shifts and categorized as wrong orders (i.e., occurring during the medication ordering phase). None of these errors had any patient impact.

The top healthcare work system contributory factors for the MREs were task/process (43%), technology (23%), provider (22%), and patient (18%). For MREs in which task/process factors were rated as contributors, 22 (85%) of them involved the pharmacy with 18 (69%) occurring in the dispensing or stocking phases. When technology was a contributory factor, MREs were related to issues with health information technology (8 events, or 62%) or infusion pumps (5 events, or 38%).

The most frequent patient outcome of the MREs was delay of therapy (42% of all the MREs; see Table 2). Seventeen (34%) MREs during the day shift had no patient impact. In contrast, only 10% of the MREs on the night shift did not negatively affect the patient. In 23 of 25 delayed therapy MREs, the level of patient impact was not assessable, and these were coded as unknown impact.

Of the 49 MREs that were neither medication errors nor ADEs, 32 (65%) had negative patient impact. Most of these MREs led to delay of care (25 of 49, or 51%). Other negative patient outcomes included inadequate pain control (4%), hemodynamic instability (4%), and other (6%) outcomes (e.g., thrombus). In terms of identifiable level of patient impact, eight (16%) events had levels that ranged from mild (n = 2), moderate (n = 5), to severe (n = 1).

RQ3: Nurse Activities and Correlations With MREs

Nurses averaged 133 tasks per hour (i.e., TTV). More tasks were performed during the night (150.3) than the day (128.4) shift (b = 18.94, 95% CI [2.72, 35.16], p = .02). After controlling for shift type, preobservation workload was positively associated with TTV (b = 1.34, 95% CI [0.37, 2.30], p = .007), but preobservation negative mood was not (b = −0.08, 95% CI [−0.70, 0.54], p = .80; see Table 1, Model 2 for model estimates). The nurses spent the most time performing conversational tasks (30.3%), direct patient care tasks (24.7%), documentation/reading tasks (18.5%), and medication tasks (10.9%). These tasks were also the most commonly performed tasks at 39.7, 42.1, 13.6, and 15.6 instances per hour, respectively. The PT spent on all other task categories was relatively low (<6%). In addition, the nurses spent an average of 4.7% of their time multitasking (range = 0.3–14.5, SD = 3.2).

There was significantly more TTV in observations with MREs (141.5) than those without MREs (128.4) after controlling for shift type, preobservation workload, and negative mood (b = 24.27, 95% CI [6.51, 42.03], p = .008; see Table 1, Model 3).

Table 4 compares the TV and PT in observations with and without MREs for each of the task category. MRE-containing observations were associated with more direct patient care tasks and more documentation/reading tasks. Although time spent on all types of medication tasks was unrelated to the occurrence of MREs, both the obtain/confirm medication and medication documentation/review tasks were more common in MRE-containing observations.

Task Volume and Percentage of Time for Each Task Category in the Observations With and Without Medication-Related Events

RQ4: MREs and Change in Workload and Negative Mood

Neither preobservation workload (AOR = 1.07, 95% CI [0.99, 1.15], p = .11; see Table 1, Model 1) nor preobservation negative mood (AOR = 0.95, 95% CI [0.88, 1.02], p = .16) was significantly associated with MRE occurrence.

There was a significant MRE occurrence × Shift type, two-way interaction effect (b = 8.65, 95% CI [0.70, 16.60], p = .03), after controlling for preobservation workload and negative mood (see Table 1, Model 4). Specifically, although workload increased significantly compared to preobservation workload in MRE-containing observations overall (when all the other variables were held at their mean; b = 4.57, 95% CI [0.39, 8.75], p = .03), the increase in workload was more prominent in night shift observations (when night shift was used as the reference level, b = 8.89, 95% CI [1.67, 16.11], p = .02; see Figure, Supplemental Digital Content 3, https://links.lww.com/NRES/A265, for a visualization). Additional exploratory analyses were conducted to understand these findings. First, no nurse participant characteristics (age, gender, self-reported years of training, hours of sleep before the shift, or difficulty falling asleep the rest cycle before the shift) explained the shift observation differences. Second, neither preobservation workload nor postobservation workload was significantly associated with shift type.

No significant relationship was found in the LME model, which was fitted to test the association between occurrence of MREs and negative mood change.


In this study, there was a high incidence of facilitated MRE reports—more than one third of the 4-hour observations periods contained at least one MRE. This is a far higher incidence than that seen with traditional reporting of medication errors or medication-related adverse events. MRE reporting captured rich information about the medication management system that occurred in all phases of the medication management process and identified potential latent failure modes that can cause patient harm. Various system deficiencies involving tasks/processes, technology, and care providers were identified through analysis of the MRE reports. Observations containing MREs were associated with nurses doing more tasks, reflected in higher TTV, and specifically performing more direct patient care tasks, documentation/reading tasks, and some types of medication tasks (i.e., obtain/confirm and document/review medications). MRE-containing observations were also associated with higher nurse self-reported workload, which increased significantly during the observation period when the MRE occurred—notably during the night shift. MREs did not correlate with nurses’ mood states.

MRE Reporting

There is ample literature on the potential for worse patient outcomes when medication errors or ADEs occur (Frith, 2013; Martins, Giordani, & Rozenfeld, 2014). However, as seen in this study, most of the medication-related NREs (or MREs) are neither medication errors nor ADEs. This is the first study we are aware of that describes ICU nurse reported MREs and their potential for patient impact. The repercussions of delayed therapy—the most frequent consequence of our MREs—may not be immediately observable but can still pose a serious safety threat in critically ill patients. For example, delayed antibiotic administration is associated with increased mortality in septic patients and those with pneumonia or meningitis (Cartmill et al., 2012). Delays in dispensing also predispose patients to omitted doses and resulting undertreatment (Carayon et al., 2014). Inadequate pain control was another frequent MRE patient outcome. Pain is rated by patients as one of their top worries in the ICU (Turner, Briggs, Springhorn, & Potgieter, 1990).

MREs provide a “window” on potential system failure modes of a healthcare facility’s medication management system (Reason, 1997). Even when MREs do not cause harm, they represent probabilistic opportunities to cause harm, and their underlying contributors should be addressed systematically. In analyzing the MREs, we identified process, technology, and system problems across the different medication management phases in the three different ICUs in three hospitals. System redesigns targeting these factors could help prevent future MREs and/or reduce their risk for patient harm.

Traditional event reporting systems do not include NREs that are not adverse events or near misses (Weinger et al., 2003). Research also showed that underreporting is a ubiquitous problem for error/event reporting systems, whether voluntary or “mandatory” (Flynn, Barker, Pepper, Bates, & Mikeal, 2002). The use of the NRE (and MRE) framework broadens the scope of what clinicians consider reportable “events” (Weinger & Slagle, 2002), reduces the stigma of reporting “errors” or adverse consequences, focuses more on processes than people, and provides ample data to inform system improvement (Weinger et al., 2003). Furthermore, the use of a low-cost nonclinician “facilitator” to collect the reports substantially increases reporting (Oken et al., 2007). Documentation and analysis of NREs has helped improve the safety of care in pediatric cardiac surgery (Schraagen et al., 2011). This study demonstrates the potential value of NRE reporting to improve medication management in ICUs.

MREs and Nurses’ Work

The ICU nurses’ mean TTV was similar to the results of Douglas et al. (2013), who found a mean TTV in four ICUs of 125 tasks per hour. However, these numbers may not be directly comparable because the task lists were not identical. Nevertheless, the 10% increase in TTV (to 142 tasks per hour) in the MRE containing observations was significant. Because of our methodology of capturing multitasking, this increase could reflect a combination of more tasks or more task switching; both could contribute to increased workload.

The occurrence of MREs correlated with increased nursing workload. In routine care, ICU nurses use adaptive work strategies such as activity stacking to reprioritize tasks (Ebright, Patterson, Chalko, & Render, 2003). For example, moving to other activities when they cannot complete a current task, or when new information or necessary tasks arise (Ebright et al., 2003). When an MRE occurs, the nurse may need to allocate extra cognitive and manual resources to deal with the event and prevent or minimize patient impact. For example, in one MRE, the nurse was unable to scan the new IV medication bag delivered from the pharmacy. This medication task and all associated patient care were delayed while the nurse paged the pharmacy and waited for them to deliver a new label. In addition to delays in therapy, depending on the MRE, additional tasks might be required such as troubleshooting alarming infusion technology or calling the computer help desk. Because many of the MREs required the nurse to seek additional information or assistance, we were surprised that communication tasks did not increase in MRE-containing observations. However, because the task data covered the entire observation period and the MREs occurred at variable times during the observations, we cannot distinguish between task patterns that preceded the MRE and those that followed.

In addition to workload, there appeared to be a variety of other contributory factors to MRE occurrence. Contributors to MREs included all of the healthcare work system components described by Carayon et al. (2006): task/process, providers, patient, and technology. With only 60 MREs and circumscribed postobservation interviews, we could not identify all types of potential contributory factors. Previous studies have identified additional factors that could contribute to clinician workload and MRE occurrence, for example, staffing levels (Dang, Johantgen, Pronovost, Jenckes, & Bass, 2002) and patient acuity (Kiekkas et al., 2007).

Although we did not find a correlation between preobservation workload and the occurrence of MREs, MREs could still be caused by workload increases prior to the actual MRE during the observation. Increased workload has been described as an important contributor to the occurrence of medication and other types of medical errors (Carayon & Gürses, 2005; Douglas et al., 2013). Under high workload, there may not be sufficient time to appropriately conduct clinical procedures or provide sufficient staff supervision or patient monitoring (Tarnow-Mordi, Hau, Warden, & Shearer, 2000). If increased workload predisposes to an MRE, the effort and resources needed to cope with the MRE could make the situation worse. Future research should measure workload continuously in real time (Weinger et al., 1997) using sensitive physiological measures, such as clinician heart rate (Weinger, Reddy, & Slagle, 2004), or intermittently with sufficient frequency using validated tools, such as the Borg workload scale (Weinger et al., 1997).

Increased workload of nurses associated with MREs was more prominent during night shifts. In contrast to previous studies (Armstrong et al., 2015), overall workload was not lower in night shifts as compared to day shifts. Differences in staffing might be a possible explanation of the greater workload increase when MREs occurred at night; lower nurse-to-patient or physician-to-patient ratios can increase workload (Neuraz et al., 2015). Yet, we have no direct evidence for staffing differences, nor other possible differences between nurses on these two shift schedules.

Increased workload of nurses associated with MREs was more prominent during night shifts.

In a prior study using similar methods (Cao et al., 2008), anesthesia residents working at night spent significantly less time on manual tasks and more time on monitoring tasks and had more negative moods but similar workload than during the day. Although this older study was in a different clinical domain (and no NREs were reported), together the two studies suggest that the methods are sufficiently sensitive to capture potentially important differences in clinician behavior under different working conditions. Other BTA studies have further validated these methods in ICU settings (Carayon et al., 2015; Douglas et al., 2013).


First, MREs as well as workload and mood were collected at the end of each observation through nurse self-report. This might lead to response bias. For example, after observation periods that contained MREs, nurses might be more likely to report higher workloads. The use of BTA enabled us to address this limitation by showing that both TTV and the PT on tasks were different throughout MRE-containing observations compared with observations that were not followed by an MRE report. In addition, nurses’ moods were not different between MRE-containing observations and those that did not contain an MRE. Second, the time of the MREs was not reliably captured. We cannot ascertain if task activity (or workload) occurred before, during, or after the MRE occurrences. Given the incidence of MREs, in future research, one could videotape care and capture these events in real time. This would facilitate event verification, as well as provide rich data to understand timing, contributing factors, and consequences of the event and its management. In studies in the operating room, videotaping NREs has been shown to be feasible and have merit (Oken et al., 2007; Slagle et al., 2015). Third, the identification and coding of MREs were based solely on the nurse’s report without corroborating with other sources of data (e.g., the medical record). Although a review of clinical documentation might have informed some of the coding, particularly patient impact severity level, for most of the MREs, if they appeared in the documentation at all, it would not typically include information about why the MRE occurred (e.g., intravenous medication delivery interrupted by the patient’s inadvertent occlusion of the intravenous tube). Fourth, we did not collect information about patients in this study. More information such as patient acuity would have facilitated our ability to understand the MREs and their impact. Some of the effect of these variables may have been partially controlled by measuring nurses’ preobservation workload and negative mood. Finally, although we strived to obtain an even distribution of observations across different times, sites, and nurses, logistical constrains (e.g., observer availability in late night hours) led to an imbalanced sample. We controlled for potential effects of site and repeated measurement of the same nurse statistically whenever possible, but these and other unknown factors may still bias the results.


This observational study collected and analyzed ICU nurses’ self-reported MREs. MREs occurred in all phases of the medication management process and had contributory factors that reflected system-level latent errors. Most of the MREs were associated with degraded care processes that either contributed to or could have caused adverse patient outcomes. This study also explored the relationships among reported MREs and nurses’ activities, workload, and moods. MRE-containing observations correlated with nurses’ higher TV, changed task distribution, and increased workload. The capture and analysis of MREs can provide valuable information for optimizing ICU clinical work, quality of care, and patient safety.


Armstrong E., de Waard M. C., de Grooth H. J., Heymans M. W., Reis Miranda D., Girbes A. R., & Spijkstra J. J. (2015). Using nursing activities score to assess nursing workload on a medium care unit. Anesthesia & Analgesia, 121, 1274–1280. doi:10.1213/ANE.0000000000000968
Bates D. M., Maechler M., Bolker B., Walker S., Christensen R. H. B., Singman H., … Green P. (2017). lme4: Linear mixed-effects models using ‘Eigen’ and S4. Retrieved from https://cran.r-project.org/web/packages/lme4/index.html
Cao C. G., Weinger M. B., Slagle J., Zhou C., Ou J., Gillin S., … Mazzei W. (2008). Differences in day and night shift clinical performance in anesthesiology. Human Factors, 50, 276–290. doi:10.1518/001872008X288303
Carayon P., Alvarado C. J., & Systems Engineering Initiative for Patient Safety. (2007). Workload and patient safety among critical care nurses. Critical Care Nursing Clinics of North America, 19, 121–129. doi:10.1016/j.ccell.2007.02.001
Carayon P., & Gürses A. P. (2005). A human factors engineering conceptual framework of nursing workload and patient safety in intensive care units. Intensive & Critical Care Nursing, 21, 284–301. doi:10.1016/j.iccn.2004.12.003
Carayon P., Schoofs Hundt A., Karsh B. T., Gurses A. P., Alvarado C. J., Smith M., & Flatley Brennan P. (2006). Work system design for patient safety: The SEIPS model. Quality & Safety in Health Care, 15, i50–i58. doi:10.1136/qshc.2005.015842
Carayon P., Wetterneck T. B., Alyousef B., Brown R. L., Cartmill R. S., McGuire K., & Wood K. E. (2015). Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. International Journal of Medical Informatics, 84, 578–594. doi:10.1016/j.ijmedinf.2015.04.002
Carayon P., Wetterneck T. B., Cartmill R., Blosky M. A., Brown R., Kim R., … Walker J. (2014). Characterising the complexity of medication safety using a human factors approach: An observational study in two intensive care units. BMJ Quality and Safety, 23, 56–65. doi:10.1136/bmjqs-2013-001828
Cartmill R. S., Walker J. M., Blosky M. A., Brown R. L., Djurkovic S., Dunham D. B., … Wood K. E. (2012). Impact of electronic order management on the timeliness of antibiotic administration in critical care patients. International Journal of Medical Informatics, 81, 782–791. doi:10.1016/j.ijmedinf.2012.07.011
Chang H. Y., Teng C. I., Chu T. L., Chang H. T., & Hsu W. H. (2012). Impact of nurse agreeableness and negative mood of nursing supervisors on intention to help colleagues. Journal of Advanced Nursing, 68, 636–646. doi:10.1111/j.1365-2648.2011.05776.x
Cohen J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1, 98–101. doi:10.1111/1467-8721.ep10768783
Dang D., Johantgen M. E., Pronovost P. J., Jenckes M. W., & Bass E. B. (2002). Postoperative complications: Does intensive care unit staff nursing make a difference? Heart & Lung, 31, 219–228. doi:10.1067/mhl.2002.122838
Douglas S., Cartmill R., Brown R., Hoonakker P., Slagle J., Schultz Van Roy K., … Carayon P. (2013). The work of adult and pediatric intensive care unit nurses. Nursing Research, 62, 50–58. doi:10.1097/NNR.0b013e318270714b
Ebright P. R., Patterson E. S., Chalko B. A., & Render M. L. (2003). Understanding the complexity of registered nurse work in acute care settings. Journal of Nursing Administration, 33, 630–638. doi:10.1097/00005110-200312000-00004
Faul F., Erdfelder E., Lang A. G., & Buchner A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191. doi:10.3758/BF03193146
Flynn E. A., Barker K. N., Pepper G. A., Bates D. W., & Mikeal R. L. (2002). Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. American Journal of Health-System Pharmacy, 59, 436–446.
Fraind D. B., Slagle J. M., Tubbesing V. A., Hughes S. A., & Weinger M. B. (2002). Reengineering intravenous drug and fluid administration processes in the operating room: Step one: Task analysis of existing processes. Anesthesiology, 97, 139–147. doi:10.1097/00000542-200207000-00020
Frith K. H. (2013). Medication errors in the intensive care unit: Literature review using the SEIPS model. AACN Advanced Critical Care, 24, 389–404. doi:10.1097/NCI.0b013e3182a8b516
Gandhi T. K., Seder D. L., & Bates D. W. (2000). Identifying drug safety issues: From research to practice. International Journal for Quality in Health Care, 12, 69–76. doi:10.1093/intqhc/12.1.69
Hart S. G., & Staveland L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock P. A., Meshkati N. (Eds.), Human mental workload (pp. 139–183). Oxford, UK: North-Holland. doi:10.1016/S0166-4115(08)62386-9
He S., Chen Y., Zhan J., & Wu J. (2014). Job burnout, mood state, and cardiovascular variable changes of doctors and nurses in a children’s hospital in China. ISRN Nursing, 2014, 386719. doi:10.1155/2014/386719
Honaker J., King G., & Blackwell M. (2011). Amelia II: A program for missing data. Journal of Statistical Software, 45, 1–47. doi:10.18637/jss.v045.i07
Horner R. D., Szaflarski J. P., Ying J., Meganathan K., Matthews G., Schroer B., … Raphaelson M. (2011). Physician work intensity among medical specialties: Emerging evidence on its magnitude and composition. Medical Care, 49, 1007–1011. doi:10.1097/MLR.0b013e31822dcdc7
Kiekkas P., Brokalaki H., Manolis E., Samios A., Skartsani C., & Baltopoulos G. (2007). Patient severity as an indicator of nursing workload in the intensive care unit. Nursing in Critical Care, 12, 34–41. doi:10.1111/j.1478-5153.2006.00193.x
Kohn L. T., Corrigan J. M., & Donaldson M. S. (Eds.). (2000). To err is human: Building a safer health system. Washington, DC: National Academies Press.
Kuznetsova A., Brockhoff P. B., & Christensen R. H. B. (2014). lmerTest: Tests for random and fixed effects for linear mixed effect models. Retrieved from https://cran.r-project.org/web/packages/lmerTest/index.html
Martins A. C., Giordani F., & Rozenfeld S. (2014). Adverse drug events among adult inpatients: A meta-analysis of observational studies. Journal of Clinical Pharmacy and Therapeutics, 39, 609–620. doi:10.1111/jcpt.12204
Mazur L. M., Mosaly P. R., Hoyle L. M., Jones E. L., & Marks L. B. (2013). Subjective and objective quantification of physician’s workload and performance during radiation therapy planning tasks. Practical Radiation Oncology, 3, e171–e177. doi:10.1016/j.prro.2013.01.001
Mohammadi M., Mazloumi A., Kazemi Z., & Zeraati H. (2015). Evaluation of mental workload among ICU ward’s nurses. Health Promotion Perspectives, 5, 280–287. doi:10.15171/hpp.2015.033
Neuraz A., Guérin C., Payet C., Polazzi S., Aubrun F., Dailler F., … Rimmelé T. (2015). Patient mortality is associated with staff resources and workload in the ICU: A multicenter observational study. Critical Care Medicine, 43, 1587–1594. doi:10.1097/CCM.0000000000001015
Oken A., Rasmussen M. D., Slagle J. M., Jain S., Kuykendall T., Ordonez N., & Weinger M. B. (2007). A facilitated survey instrument captures significantly more anesthesia events than does traditional voluntary event reporting. Anesthesiology, 107, 909–922. doi:10.1097/01.anes.0000291440.08068.21
Pingenot A. A., Shanteau J., & Sengstacke L. T. (2009). Description of inpatient medication management using cognitive work analysis. Computers, Informatics, Nursing, 27, 379–392. doi:10.1097/NCN.0b013e3181bcad2f
R Core Team. (2017). The R Project for statistical computing. Retrieved from http://www.R-project.org
Ream R. S., Mackey K., Leet T., Green M. C., Andreone T. L., Loftis L. L., & Lynch R. E. (2007). Association of nursing workload and unplanned extubations in a pediatric intensive care unit. Pediatric Critical Care Medicine, 8, 366–371. doi:10.1097/01.pcc.0000269379.40748.af
Reason J. (1997). Managing the risks of organizational accidents. Aldershot, UK: Ashgate.
Rothschild J. M., Landrigan C. P., Cronin J. W., Kaushal R., Lockley S. W., Burdick E., … Bates D. W. (2005). The Critical Care Safety Study: The incidence and nature of adverse events and serious medical errors in intensive care. Critical Care Medicine, 33, 1694–1700. doi:10.1097/01.CCM.0000171609.91035.BD
Rubin D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: Wiley.
Schraagen J. M., Schouten T., Smit M., Haas F., van der Beek D., van de Ven J., & Barach P. (2011). A prospective study of paediatric cardiac surgical microsystems: Assessing the relationships between non-routine events, teamwork and patient outcomes. BMJ Quality & Safety, 20, 599–603. doi:10.1136/bmjqs.2010.048983
Seynaeve S., Verbrugghe W., Claes B., Vandenplas D., Reyntiens D., & Jorens P. G. (2011). Adverse drug events in intensive care units: A cross-sectional study of prevalence and risk factors. American Journal of Critical Care, 20, e131–e140. doi:10.4037/ajcc2011818
Slagle J., Weinger M. B., Dinh M. T., Brumer V. V., & Williams K. (2002). Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology. Anesthesiology, 96, 1129–1139. doi:10.1097/00000542-200205000-00016
Slagle J. M., Anders S., Porterfield E., Arnold A., Calderwood C., & Weinger M. B. (2015). Significant physiological disturbances associated with non-routine event containing and routine anesthesia cases. Journal of Patient Safety, 11, 198–203. doi:10.1097/pts.0000000000000081
Stewart W., & Barling J. (1996). Daily work stress, mood and interpersonal job performance: A mediational model. Work & Stress, 10, 336–351. doi:10.1080/02678379608256812
Tarnow-Mordi W. O., Hau C., Warden A., & Shearer A. J. (2000). Hospital mortality in relation to staff workload: A 4-year study in an adult intensive-care unit. Lancet, 356, 185–189. doi:10.1016/S0140-6736(00)02478-8
Turner J. S., Briggs S. J., Springhorn H. E., & Potgieter P. D. (1990). Patients’ recollection of intensive care unit experience. Critical Care Medicine, 18, 966–968. doi:10.1097/00003246-199009000-00012
Weinger M. B., Herndon O. W., & Gaba D. M. (1997). The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology, 87, 144–155. Discussion 29A-30A. doi:10.1097/00000542-199707000-00019
Weinger M. B., Reddy S. B., & Slagle J. M. (2004). Multiple measures of anesthesia workload during teaching and nonteaching cases. Anesthesia & Analgesia, 98, 1419–1425. doi:10.1213/01.ANE.0000106838.66901.D2
Weinger M. B., & Slagle J. (2002). Human factors research in anesthesia patient safety. Journal of the American Medical Informatics Association, 9, S58–S63. doi:10.1197/jamia.M1229
Weinger M. B., Slagle J., Jain S., & Ordonez N. (2003). Retrospective data collection and analytical techniques for patient safety studies. Journal of Biomedical Informatics, 36, 106–119. doi:10.1016/j.jbi.2003.08.002
West B. T., Welch K. B., & Galecki A. T. (2014). Linear mixed models: A practical guide using statistical software (2nd ed.). Boca Raton, FL: CRC Press.
White I. R., Royston P., & Wood A. M. (2011). Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine, 30, 377–399. doi:10.1002/sim.4067
Wong D. H., Gallegos Y., Weinger M. B., Clack S., Slagle J., & Anderson C. T. (2003). Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Critical Care Medicine, 31, 2488–2494. doi:10.1097/01.ccm.0000089637.53301.ef
World Health Organization. (2005). WHO draft guidelines for adverse event reporting and learning systems: From information to action. Geneva, Switzerland: Author. Retrieved from http://apps.who.int/iris/bitstream/10665/69797/1/WHO-EIP-SPO-QPS-05.3-eng.pdf

intensive care; medication errors; nursing; voluntary patient safety event reporting; workload

Supplemental Digital Content

Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved