Arterial blood gas (ABG) results are routinely used by critical-care nurses to assess the respiratory and acid-base status of critically ill patients in ICUs.1 In the ICU setting, accurate and timely interpretation of ABG data can be a key factor in successful clinical decision making. Nursing interventions dependent on rapid interpretation of ABG data include communicating with the patient's healthcare team, optimization of ventilation and oxygenation, titration of vasoactive infusions, medication administration, and initiation of advanced cardiac life support.
Given the limited number of variables used to interpret acid-base data, computerized decision support for ABG analysis has high potential for success as a healthcare application, and several decision-support tools have been developed. One study of 34 physicians (including residents and staff physicians) found that accuracy rates for unaided ABG data interpretation given traditional laboratory data in numerical format were, on average, 39%, with a range of 0% to 80%.2 Vergara3 found that ABG data interpretation was more accurate when physicians used an algorithm-based software application, compared with using a paper-based algorithm tool or subjectively interpreting data without decision support (88% vs 64% vs 43%, P < .0001). A major limitation of this study was the sample size of three physicians, although a total of 30 trials were conducted with each participant. Other authors have published computer-generated algorithms for ABG data interpretation,4-6 presenting verification of the algorithms' accuracy, however, lacking an evaluation of clinician performance with the use of these tools.
General limitations of the computerized decision-support applications in the aforementioned research include (1) the clinician is sometimes removed from the raw data or may not pay attention to the raw data, which can lead to errors when the decision-support system is incorrect7; (2) reliance on computer software for the entire decision-making process could lead to a reduction in critical-thinking skills through lack of practice; and (3) the applications were exclusively designed for and evaluated by physicians. Another limitation was that the applications were not derived from principles of human factors psychology and human-systems interaction and therefore may not fit the clinician's actual decision-support needs. The goal in this research and development project was to create a computer-aided tool for critical-care nurses that leverages graphical representation-based concepts such as pattern recognition to promote rapid and accurate data interpretation, without removing the nurse from the raw data or the critical-thinking process.
Organizing and presenting information in a format that facilitates data interpretation and supports decision making are a core focus of human factors research.8 Graphical representation of data can attribute meaning to individual parameter values and integrate discrete variables into useful concepts for the user.9,10 Data graphics and visualizations can incorporate redundancy and highlight alarm settings through shape, color, and movement.11,12 Another advantage of graphical data displays over numerical data lists is that once the user develops the ability to recognize patterns, they perform fewer mental transformations to interpret the data, thus decreasing the time required to analyze the information and possibly reducing the number of data interpretation errors.12,13
To address gaps and limitations observed in previous ABG analysis software, an interdisciplinary team of professionals in nursing, bioengineering, information design, and human factors psychology developed a novel graphical visualization of ABG data that incorporates graphical user-interface design and human-systems interaction principles. The overall objective was to develop an intuitive, meaningful, and ergonomically efficient visualization for interpreting acid-base data from an ABG laboratory analysis. This visualization supports clinicians interpreting ABG laboratory results so that they can rely less on memory and recall, which tend to be error prone. Fundamental design concepts, as described by Lewis Miller et al11 and Tufte,12,14 were used as the basis for developing the visualization. These included (1) graphical organization of visual elements that highlights symmetry when variables are within a reference range and asymmetry when variables are out of range, (2) perceptual grouping of graphical elements to promote visual cueing and pattern recognition of normal versus abnormal states, and (3) use of color coding to decrease the need for memorizing relationships between variables and meaning. To demonstrate the effectiveness of the visualization, nurses' performance with the new display design was compared with an industry standard numeric digital data presentation. The objective of the evaluation was to determine whether the novel graphical visualization improved the accuracy of ABG data interpretation, decreased the time required for correctly identifying altered states, and decreased the mental effort required for data interpretation.
Development of the Graphical ABG Visualization
The graphical ABG visualization shown in Figure 1 was developed following a user-centered iterative design process that enabled the developers to solicit user feedback on the proposed designs prior to the performance evaluation. During the design process, focus groups were established at each major design iteration with one or two critical-care clinicians and one or two nursing students. In the initial stages, several design iterations were presented to the clinicians and students for the purposes of obtaining comparative feedback on several design concepts. The primary goal of the focus group discussions was to identify the components of the display(s) that were most intuitive and informative versus those that were confusing and unhelpful. After these feedback sessions, the design team would generate another round of design iterations, and the process was repeated. After four feedback sessions and design iterations, the design team was confident that the final graphic encoded the most relevant information for the nurse user and was consistent with the potential user's mental model of acid-base data.
The graphical ABG visualization centered on the methods that nurses are traditionally taught to interpret ABG reports.15-18 The relationships between the physiology underlying the ABG data, the cognitive steps that clinicians are trained to interpret ABG data, and the principles encoded into the graphical ABG visualization are described herein and illustrated in Figure 1.
1. When interpreting ABG results, first the pH is examined to determine the patient's acid-base status. pH is the variable displayed in the center and is represented by a sliding scale that clearly shows when the value is outside the reference range. In addition, abnormal pH values are labeled as "acidosis" and "alkalosis," to decrease the number of cognitive steps that the user must make to interpret the pH alteration and to decrease the potential for error. Hash marks on the slider's vertical axis make the degree of deviation from normal more salient. The value for pH is displayed on the slider.
2. When interpreting ABG results, the nurse must next identify the cause of the acidosis or alkalosis. For example, if low pH is accompanied by high PaCO2, then the condition being reflected by these values is a respiratory acidosis (Figure 1B). If the low pH is accompanied by low bicarbonate ion concentration (HCO3−), then the condition represented is a metabolic acidosis (Figure 1C). To make the identification salient, the "cause" of the imbalance is indicated by a color in the border of the pH icon slider that corresponds to either PaCO2 (blue on the left side of the pH slider) or HCO3− (pink on the right side). A mixed gas (eg, respiratory and metabolic acidosis) is represented by a border on the pH icon that is half pink and half blue (not shown). Values for PaCO2 and HCO3− are represented in a similar manner to pH in that they are colored icons on a slider that moves up and down to represent high and low values, with hash marks on the slider's vertical axis to represent the degree of deviation beyond the reference range. pH, PaCO2, and HCO3− are all represented on the same scale where the reference ranges and deviations have been normalized.
3. Compensation for an acid-base disorder is identified in the following manner. First, if the border on the pH icon is blue, thus identifying the acid-base balance as "metabolic," and the other variable is out of range on the same side of the horizontal axis (in Figure 1C, the pink icon for PaCO2), then the user can assume that compensation is occurring.
The trending of ABG information is critical in determining the progress of a patient over the course of treatment in many critical-care environments. Therefore, an additional consideration in the design development was to determine an effective means for communicating the physiological information as it changes over time. This feature of the visualization was designed to enable the clinician to determine at a glance whether the patient's ABG profile was moving toward a more critical or more normal condition (Figure 2).
In the evaluation of the graphical ABG visualization, participants served as their own controls by completing the testing sequences for both the graphical ABG and digital ABG results. The order of testing (graphical vs numeric) was randomly assigned. This evaluation of the new visualization display tool was conducted with senior baccalaureate nursing students and critical-care nurses for the purpose of assessing its utility among novices and experts.
Fifteen critical-care nurses and 15 students in their final residency semester of a baccalaureate nursing program volunteered to participate in this study. Nurses were recruited from an ICU at the university's Academic Health Sciences Center, and students were recruited from a single cohort at a nationally accredited college of nursing. Institutional review board approval was granted, and informed consent was obtained from the participants.
Data Collection Instruments
To assess the utility of presenting ABG results graphically, a computer-based testing protocol was designed and executed. The test was based on a series of ABG results presented either in the graphical ABG visualization format or in the traditional numeric format. The numeric ABG results included high (H) and low (L) indicators if the values were abnormal. A single question with multiple response choices was associated with each ABG result. The individual questions were grouped into three orthogonal blocks representing three types of information available from the ABG data. The blocks of question types and the number of questions in each block were as follows:
* Five questions assessing the face value recognition for acid-base information (ie, high, low, normal)
* Five questions assessing identification of acid-base state (ie, metabolic acidosis, respiratory alkalosis, mixed gas) and presence of respiratory or renal compensation
* Five questions assessing change in acid-base state over time (ie, trending of acid-base state)
All response sets included the response choice, "Not enough information is provided"; however, in all cases there was sufficient information to answer the questions. Although the questions for the two display conditions were different, the difficulty of each question pair was matched. The computer-recorded user responses and time to complete each question.
The NASA Task Load Index (NASA-TLX) was administered following completion of each testing session for the purpose of ensuring that an improvement in nurse performance did not come at the expense of increased workload. The NASA-TLX assesses mental, physical, and temporal demand, as well as effort, frustration, and perceived performance following a given task19; in this case, the task was the interpretation of ABG data.
Prior to the testing, participants were asked to rate their past experience interpreting data from an ABG report using the following items that were rated on a 1- to 10-point scale: (1) How often do you interpret ABG information? (1 = "never," 10 = "always") and (2) How confident are you in interpreting ABG information? (1 = "unsure," 10 = "certain"). At the end of the study session, users completed a subjective usability questionnaire pertaining to the graphical ABG visualization.
Each participant user received a 10-minute training session consisting of a computerized presentation that identified the components of the graphical ABG visualization and the digital presentation of ABG results. The training session was scripted and read verbatim. Examples of ABG data interpretation in presentation formats were given. At incremental points throughout the training, the user's comprehension was assessed through oral quizzing. In the event of an incorrect quiz response, the material pertaining to the quiz item was reviewed until the user was able to demonstrate competency to the trainer by correctly identifying altered acid-base states using the graphical visualization tool.
COMPUTER-BASED DISPLAY TESTING
Participants (nurses and nursing students) were randomly assigned to one of two groups. Test computers presented the first group with 15 questions about ABG data presented in the traditional numerical format, followed by 15 questions about ABG data presented in the graphical format. The second group received the questions in the reverse order. Participants filled out the NASA-TLX after each group of questions so that perceived workload could be compared for the two types of information displays. The usability questionnaire was completed at the end of the entire study session.
Since each participant user completed the testing procedures using both display types, accuracy and time data were analyzed using paired t tests with display condition (ie, numeric vs graphical) as the independent variable. The analyses for nurses and nursing students were conducted separately since the aim of the study was to evaluate the effects of the graphical visualization, rather than compare the performance of novice and experts. Average response accuracy and time for each participant were calculated for each block of questions. The average response time values for incorrect answers were excluded from the analysis. Data from the NASA-TLX were also analyzed using paired t tests in a similar manner.
The critical-care nurses in this study had on average 12.1 years' experience in critical care (±9.1 years), while the students, as described earlier, were in the final semester of their baccalaureate nursing program. Critical-care nurses reported an average rating of 8.5 (±1.5) of 10 in the frequency that they interpret ABG data, while nursing students reported an average of 3.2 (±1.8). In terms of confidence when interpreting ABG data, critical-care nurses reported an average of 7.1 (±1.1) of 10, and nursing students reported an average of 5.5 (±2.0).
Accuracy of Data Interpretation
Results describing the accuracy of data interpretation from the computer-based ABG testing protocol are presented in Table 1. Individual group comparisons revealed that accuracy improved significantly on the "identification of acid-base state" (block II) and the "acid-base trending" (block III) questions when participants were using the graphical ABG visualization. Nursing students showed the greatest improvement in their test scores, with an average increase of 33% on the "identification" questions. Nursing students demonstrated reduced accuracy on the "individual variable recognition" (block I) when using the graphical visualization; however, this difference was not statistically significant (P = .10), and the average score was greater than 90%. The critical-care nurses improved their accuracy by an average of 17% on "identification" questions; however, there was a large amount of variability in accuracy rates in both display conditions. Of note was the finding that nursing students achieved 100% accuracy in correctly identifying trends in acid-base states with the graphical ABG visualization, an improvement of 36% over their use of the numeric results.
There was no correlation between improvement in response accuracy (measured as a difference score) and self-reported frequency of interpreting ABG results or with self-reported confidence in interpreting ABG results. Years of experience among the critical-care nurses was not correlated to response accuracy, using either the numeric or graphical display, or improvement in response accuracy.
Response Time for Data Interpretation
Results describing the average response times for correct answers on the computer-based ABG testing protocol are presented in Table 2. For the critical-care nurses in this study, the average time to correctly answer the "identification of acid-base state" (block II) questions was significantly reduced when they used the graphical ABG visualization. Average response times for the two other blocks were statistically equivalent. For the nursing students, the average time to correctly answer all questions types was significantly reduced when they used the graphical ABG visualization. Nursing students were, on average, 50% faster when using the graphical ABG visualization while interpreting data in block II.
NASA-TLX (Workload) Results
Nursing students reported a significant reduction in NASA-TLX scores using the graphical ABG visualization (68.6 vs 49.3, paired t14 = 5.3, P < .001). Critical-care nurses reported no reduction in NASA-TLX scores associated with the use of the graphical ABG visualization (60.3 vs 59.1).
Usability and Subjective Evaluation of the Graphical ABG Visualization Tool
The graphical ABG visualization was highly rated on subjective measures by critical-care nurses and nursing students (Table 3). Eighty-seven percent of users in this study reported that if they had a choice, they would "in some cases try" or "always try" to be in a patient room where the graphical ABG visualization was being used. Seventy-three percent of users reported that they would either insist or always try to recommend the use of the graphical ABG visualization in addition to their current equipment. Subjective comments from the critical-care nurses included the following:
I think this would be a great tool. It would take some time to change the nurses' mode of thinking from numbers to pictures.
Add the [graphical visualization], don't take away what we already have.
[I need] more training and experience with it.
No significant difference in response accuracy or response time was found for the identification level questions (block I), which verifies that the graphical format does not impede a nurse's ability to identify deviations from normal. Response accuracy in the identification of acid-base states and trends for critical-care nurses and nursing students was significantly increased when they were using the graphical ABG visualization. This result indicates that the graphical presentation of ABG data was more effective in communicating altered pH, PaCO2, and HCO3− values to the users. This increase in performance was also associated with a significant reduction in response time and perceived workload for nursing students. One reason for this discrepant finding may be that nursing students have had more exposure to new technologies and may be better at assimilating and applying technology training. Experienced nurses may require more training and practice time than novice nurses or nursing students before they can report reduced mental effort.
Although response times for data interpretation were reduced for all question types among the nursing students and for the identification of acid-base states among the nurses, the differences reported are not clinically significant. However, shorter response times indicate that the mental processing required during the interpretation of complex clinical data may be more efficient when the clinician can see a meaningful graphical representation of the data. When developing a new decision-support technology, the goal is to improve accuracy without increasing the time that it takes a clinician to interpret the data. In this study, the improvements in accuracy with which nurses and nursing students interpreted the data with the graphical ABG visualization corresponded to an equivalent time or a reduction in the amount of time needed to extract the meaningful information. Therefore, this visualization may help nurses overcome the "speed/accuracy trade-off" during high-stress situations when decisions must be made quickly.14,20
Unrelated to the graphical visualization, but of considerable clinical importance, were the low accuracy rates for the unaided interpretation of acid-base data by the critical-care nurses. These results demonstrate that (1) self-confidence in data interpretation of ABG data may not reflect skill level, a finding similar to the research of Hingston et al2; (2) ongoing training in the interpretation of laboratory results is necessary; and (3) some form of clinical decision support for the interpretation of ABG data is essential.
Limitations and Future Research
The effect of graphical data representation on intervention focused decision making was not assessed, and therefore, no data are available to demonstrate the effect of the graphical ABG visualization on clinical outcomes. To demonstrate the impact of this form of information presentation on treatment outcome, further investigations are planned using a high-fidelity patient simulator as the primary dependent measure apparatus. Future efforts in the design of the graphical ABG visualization will also focus on presenting other variables from the ABG in a similarly meaningful and intuitive way.
The increased accuracy observed when nurses and nursing students used the graphical ABG visualization corresponded to an associated reduction in the amount of time needed to extract the meaningful information from the information array. Among students, increased accuracy also corresponded to reduced workload. These objective findings combined with the positive subjective evaluations support the integration of the graphical ABG visualization into the current digital presentations of numerical ABG data.