Epstein, Richard H. MD, CPHIMS*; Dexter, Franklin MD, PhD†
Hospital comparisons of quality of care using anesthesia information management systems (AIMS) data can include assessment of the intraoperative hemodynamics of patients undergoing noncardiac surgery.1,2 There was no relationship between outcome and occurrence of either a median mean arterial pressure (MAP) <50 mm Hg or a median systolic blood pressure <70 mm Hg over 10-minute periods.3 However, for cases with an operative duration of >3.5 hours, the presence of at least one median systolic blood pressure during any 5-minute interval >160 mm Hg increased the odds of morbidity or mortality 2.1-fold.1 Each 5-minute increase in the duration of MAP <75 mm Hg while the bispectral index was <45 and the minimum alveolar concentration of volatile anesthetic was <0.7 was associated with increased hospital length of stay and 30-day mortality. a Incidences of hypotension are heterogeneous among hospitals, with patients' odds of sustaining ≥10 minutes ≥30% reduction in MAP from baseline ranging from 0.12 to 2.50.4
Because comparing hospitals' blood pressure data involves handling edited values, assessments of quality can be inaccurate. For example, one hospital had “at least one …[manually edited systolic blood pressure] value … in 10% of” anesthetics.5 Alterations decreased the variance of systolic blood pressure in 92% of anesthetics.5 “The relative risks (95% confidence intervals [CI]) of data [deletion or substitution] for systolic blood pressure <80 mm Hg or systolic blood pressure ≥160 mm Hg (vs 80–160 mm Hg) were 25.8 (24.1–27.6) and 7.1 (6.6–7.7), respectively.”5 Handwritten anesthesia records had “peak [systolic blood pressure], standard deviation, and fractional rate of change … less than, and the trough and median … larger than, those in” simultaneously recorded data.6
Comparing hospitals' blood pressure data also involves consideration of missing values (“gaps”), caused by7 anesthesia providers manually deleting values (as “artifacts”) or interruptions in monitoring resulting in absence of such signals. One hospital had at least one gap ≥10 minutes with no measured blood pressure for 39% of anesthetics.8,b Three hospitals had incidences of 7.1%, 2.7%, and 1.5%, respectively.8 The presence of these unexplained gaps cannot be ignored when comparing hospitals, because their incidences match or exceed the 1% to 2% rates “of perioperative cardiac averse events” in noncardiac surgery.3 Gaps are rarely seen in paper records because providers interpolate missing values when charting. The hospital with a 1.5% incidence of cases with at least one gap in its AIMS had a 0.0% incidence in paper records completed by the same providers during a contemporaneous time frame.8
Whether the missing blood pressure values themselves can be ignored, or special statistical consideration is required, when hospitals are compared is unknown. For example, turning patients from supine to prone causes hemodynamic changes. Suppose that, when turning patients, anesthesia providers at a hospital routinely pause blood pressure monitoring for >5 minutes,8 the American Society of Anesthesiologists standard for the maximum interval between measurements.c Then, ignoring blood pressure gaps >5 minutes would cause spurious epidemiological (quality monitoring) results for the hospital. The MAP at the beginning and end of gaps would differ from the MAP during similar intervals during which the blood pressure was recorded. We tested this hypothesis using data from a hospital with an AIMS not permitting manual editing or deleting of blood pressure data, thus isolating our study to the issue of the MAP during gaps.
The study was approved by the Thomas Jefferson University IRB without a requirement for informed consent. The MAPs recorded in the AIMS were from noninvasive blood pressure cuffs and arterial lines. The data were extracted from the 14,860 noncardiac cases previously performed at the hospital's tertiary surgical suite and ambulatory surgery center over 2 successive 1-year periods (Table 1). The first year studied was before institution of a near real-time alert system notifying the provider of a blood pressure gap (“prealerts”) (Table 1). The second year studied was after starting alerts (“postalerts”).8 We picked 1-year periods arbitrarily because we had no a priori estimate for the variability in bias among gaps or knowledge as to whether the alerts would influence the results.
Causes of the gaps have been reported previously.8 The dates of the current study overlap sufficiently with the dates of the previous study that the most common causes of gaps are those shown in Reference 8, Figure 5: indeterminate > position change > blood pressure monitor failure.
For each MAP value recorded in the AIMS record, we obtained the corresponding timestamp and calculated the time between successive MAP measurements (gap). Nearly all MAP gaps (97.6%) were ≤15 minutes. Thus, we studied gaps that were prolonged (i.e., >5 minutes, exceeding the American Society of Anesthesiologists' standardc), but ≤20 minutes (Fig. 1).
All gaps were identified for each case with at least one gap. Starting with the longest measured gap for a case, a match from the same case was determined at random, without replacement, from among intervals (1) equal to the gap interval, and (2) with all successive pressure measurements within the interval ≤5 minutes. If an exact match was not found, a match was made from intervals 1 minute shorter than the gap interval. If not possible, a match was made from intervals 2 minutes shorter, and so forth until a match was found or no candidate intervals were present. For example, if there was a 12-minute gap between successive blood pressures, matches would be considered among all pairs where the blood pressure timestamps were 12 minutes apart and no gaps were present in that 12-minute interval. If no pairs were found that were 12 minutes apart, then pairs 11 minutes apart were considered, then pairs 10 minutes apart, etc. The process was then repeated for all other gaps from the case, if any. If a corresponding control interval was not found, then that gap was excluded from analysis. A match was obtained for 15,338 of the 22,046 gaps (70%, Table 1). The intervals for the matched controls were slightly briefer than gap intervals (−0.6 ± 0.01 minutes, 10th percentile −1.0 minute, 90th percentile 0.0 minutes). Data are reported as mean ± SE of the mean.
The 95% confidence intervals (CIs) for mean differences in MAP were calculated using 2-sided Student paired t tests.9,d Asymptotic Spearman nonlinear (i.e., rank) correlation coefficients were calculated between MAP data and gap intervals. We focused on the mean difference because of its relevance to linear interpolation across the gap when compensating for the missing value(s) during statistical analysis. Mean absolute differences were compared between gaps and matched periods to assess hemodynamic variability. The odds ratio was also calculated for the incidence of gaps having large (20 mm Hg) changes in MAP relative to matching periods. Because each end point was calculated using paired (matched) data, our results are likely insensitive to incidences of gaps, number of gaps per hour, etc., which is important because these characteristics differ among hospitals.8 The corresponding exact 95% CI and P values were calculated using the McNemar test (StatXact-9; Cytel Software Corp., Cambridge, MA).
The MAP at the start of gaps was larger than the MAP at the start of matching intervals (controls) (P < 0.00001), but by only 2.9 mm Hg (95% CI 2.6–3.3 mm Hg). The MAP at the end of gaps was also larger than the MAP at the end of matching intervals (P < 0.00001), but on average by just 3.8 mm Hg (95% CI 3.4–4.1 mm Hg). Thus, as hypothesized, MAP values at the start and end of gaps are different from paired (control) values. However, our results show that the differences are too small to be relevant for quality monitoring that averages over many cases.
The difference was taken between the first MAP at the end of a prolonged gap and the MAP at the start of the gap. The difference was also taken between the MAP at the end minus the beginning of the matched control period. There was no association between the pairwise difference of differences and durations of gaps (P = 0.47, Spearman r = 0.00, 95% CI −0.01 to 0.02) (Fig. 1). The mean difference of differences was 1.5 ± 0.4 mm Hg for the 5164 pairs in the “prealert” period and 0.5 ± 0.3 mm Hg for the 10,174 pairs in the “postalert” period. The mean difference was 0.7 ± 0.3 mm Hg among gaps from cases without an arterial line. The differences were significantly different (P = 0.0003), but hemodynamically negligible (0.9 mm Hg, 95% CI 0.4–1.4 mm Hg).
To assess the impact of gaps on monitoring hemodynamic variability, the absolute difference was taken between the first MAP at the end of a prolonged gap and the MAP at the start of the gap. The same was done for the matched control interval. There was no association between the absolute MAP difference for the gap minus the absolute MAP difference for the matched pair and the duration of the gap (P = 0.11, Spearman r = −0.01, 95% CI −0.03 to 0.01). The mean difference of absolute differences was 4.4 ± 0.3 mm Hg before alerts were implemented and 4.3 ± 0.2 mm Hg thereafter. The mean difference of absolute difference was 4.4 ± 0.2 mm Hg overall and among gaps from cases without an arterial line. The mean difference of absolute MAP differences was significantly larger than zero (95% CI 4.0–4.7 mm Hg, P < 0.00001). For 20.9% of gaps, the absolute MAP difference exceeded 20 mm Hg for the gap but not the matching control interval. For 11.3% of gaps, the absolute MAP difference exceeded 20 mm Hg for the matching interval but not for the gap. Thus, the odds ratio equaled 1.85 (i.e., 20.9/11.3) for gaps being associated with hemodynamic changes exceeding 20 mm Hg (95% CI 1.75–1.96, P < 0.00001).e
We found that MAPs bracketing prolonged gaps are not equivalent to those during matched control intervals, but the hemodynamic differences (not variability) are small when averaged over many (hundreds) of cases. Statisticians and clinicians analyzing AIMS data for quality monitoring can thus use whatever interpolation (i.e., missing value) method is suitable for their planned statistical analysis. For example, if the MAP is 60 mm Hg at the start and end of a gap, then for purposes of summing minutes of MAP <75 mm Hg,a considering the MAP to have been 60 mm Hg throughout the gap will result in an overall unbiased estimate of minutes less than this threshold. However, these results should not be interpreted to suggest that gaps in the monitoring of blood pressure are not important for individual patients, especially because there is more hemodynamic variability during periods of gaps than during other periods of the same cases. Certainly, it is best not to have monitoring gaps during anesthesia.
Our results show the importance of fixing the underlying problems in monitor and information system design and performance and of addressing provider behaviors that lead to the gaps in the first place (e.g., discontinuing monitoring during position changes).8 Because no monitor or software is perfect, providers should enter explanations in the AIMS when such gaps occur, as recommended by the American Society of Anesthesiologists' standards.c When there is ancillary evidence of hemodynamic stability during a prolonged gap (e.g., no change in other vital signs), noting this may also be appropriate. A reasonable quality metric is the incidence of unexplained gaps in the AIMS record.8
In contrast to our results that epidemiological findings based on means and minutes are unlikely to be influenced by occasional 10-minute gaps in recorded blood pressure, there is likely to be large impact (bias on results) from anesthesia providers “correcting” automatically determined pressures (see second paragraph of the introductory text). Such editing and smoothing of hemodynamic data when manually charting or editing AIMS records5,6 should not be ignored when performing quality monitoring, because the incidence is equivalent8 to or larger thanb the incidence of adverse events. At the AIMS installation used for our study, this was not an issue, because the AIMS was configured to prevent users from editing any vital sign data acquired automatically from monitors. If a provider thinks a blood pressure value is erroneous (i.e., an artifact) and wishes to note this, a comment is added, rather than altering the recorded value. Adopting this approach is important because most AIMS can be configured to permit or prevent such editing.10 Editing blood pressure values does not reduce legal risk because the AIMS database records an audit trail of such changes, the edits are discoverable, and their presence may suggest obfuscation of the medical record.10 Conveniently, statistical compensation for manual invalidation is straightforward, because audit logs contain the original values. Thus, where editing is permitted, analyses should be performed separately using the original and the modified values.
Our study's primary limitation is that the results apply only to the statistical analysis of hundreds or thousands of cases (e.g., quality monitoring), not to individual gaps or patients. Our results do not imply that absence of blood pressure monitoring for 10 minutes is safe (Fig. 1). Rather, the missing blood pressures do not differ substantively, on average, from control (paired) values. Because blood pressure sometimes changes quickly during anesthesia, interruptions of monitoring will occasionally delay recognition of critical events. This is important because gaps occur frequently during periods associated with hemodynamic changes such as patient positioning,8 highlighted by the odds ratios for large hemodynamic changes being larger for gaps than for matched periods. Second, our findings are from one hospital (see Table 1). We do not know the extent to which results would differ among hospitals. Third, we only studied cases with >5-minute gaps in blood pressure monitoring. To study the impact of patient characteristics on the presence of gaps, we would have needed to control for provider, yet many cases have more than one provider. Thus, analysis would essentially involve identifying the provider present at the start and end of every 5- to 20-minute interval and taking into account the nonrandom assignment of intervals (cases) to provider (e.g., cases in which prone positioning is involved). We started with a simpler approach in this study and stopped after determining the lack of importance of blood pressure gaps epidemiologically.
In summary, for comparing mean arterial pressure among hospitals' cases using data from hundreds or thousands of cases, statistical issues related to interruptions in the recording of blood pressure in the AIMS records are of minor importance, and can be handled by statistical methods appropriate to the analysis being performed. In contrast, such benchmarking cannot ignore the large impact when AIMS users are allowed to modify (i.e., “smooth”) the values that have been recorded automatically.5,6 Monitoring of hospitals should also compare the incidence of the vital sign gaps themselves.8 The incidence of hemodynamic variability (e.g., rapid changes in MAP >20 mm Hg) can be underestimated for hospitals that have vital sign gaps.
Franklin Dexter is the Statistical Editor and Section Editor for Economics, Education, and Policy for the Journal. This manuscript was handled by Dwayne R. Westenskow, Section Editor for Technology, Computing, and Simulation and Dr. Dexter was not involved in any way with the editorial process or decision.
Name: Richard H. Epstein, MD, CPHIMS.
Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.
Attestation: Richard H. Epstein has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.
Name: Franklin Dexter, MD, PhD.
Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.
Attestation: Franklin Dexter has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
a Saager L, Greenwald SD, Kelley SD, Schubert A, Sessler DI. Duration of a “triple low” of blood pressure, BIS & anesthetic concentration predicts poor outcomes. New Orleans, LA: American Society of Anesthesiologists, 2009:A880. BIS is product of Aspect Medical Systems, Norwood, MA. Cited Here...
b Epstein RH, Ehrenfeld JM, Sandberg WS, Vigoda MM. Frequency of prolonged gaps in blood pressure documentation in anesthesia information management systems. San Diego, CA: IARS, 2009:S-99. Cited Here...
c American Society of Anesthesiologists. Standards for basic anesthetic monitoring, effective July 1, 2011. Cited Here...
d We repeated the calculations using confidence intervals for skewed distributions.9 Because of the large sample size of 15,338 gaps, the 4 confidence intervals were all the same to within the reported 0.1 mm Hg. Figure 1 shows that the difference of differences were only slightly skewed. For the mean absolute differences, there was modest skewness of g1 = 0.6. Cited Here...
e The odds ratios similarly equaled 1.84 for threshold of 10 mm Hg (95% CI 1.75–1.93) and 2.08 for threshold of 30 mm Hg (95% CI 1.92–2.25). Cited Here...