For hospitals that have elected to follow the Patient Blood Management standards from the American Association of Blood Banks (AABB), review of all transfusions not adhering to the hospital’s transfusion guidelines is required.1 For audit of intraoperative red blood cell (RBC) transfusion, the intraoperative records from all transfused cases would need to be manually evaluated because automated screening methods have not been published. We developed a methodology to select the subset of cases with intraoperative RBC transfusion that would meet review criteria. This methodology has utility because many cases would thereby be excluded from review, reducing labor requirements related to the audit process.
In hospitals following Patient Blood Management standards, the transfusion committee chooses suitable hemoglobin (Hb) thresholds, incorporating patient-specific comorbidities, and based on recent meta-analyses and systematic reviews.2–4 The American Society of Anesthesiologists Practice Guidelines for Perioperative Blood Management states: “transfusion of red blood cells is rarely necessary when the hemoglobin concentration is more than 10 g/dL.”5 The specified Hb threshold can be the strongest predictor of RBC transfusion for individual surgical procedures, as demonstrated for the occurrence of transfusion in patients undergoing carotid endarterectomy (odds ratio = 57.4 for hematocrit <30%).6 Haspel and Uhl7 published a narrative of “how [they] audit hospital blood product utilization.” Their suggested “indication for transfusion” in the “hemodynamically stable, non-bleeding patient” was Hb ≤7 g/dL. They used a threshold of 10 g/dL in the “hemodynamically stable patient with end-organ ischemia.” The RBC transfusion would be audited if initiated at a greater Hb than the threshold.7
In this study, we extend a methodology published for 5 common procedures based on Hb targets and triggers8,9 to the audit decision for the thousands of different surgical procedures among adult patients (see Supplemental Digital Content, Appendix, http://links.lww.com/AA/C22, for details). Our goal was not to develop a method to evaluate each transfusion decision (eg, with estimation of the rate of blood loss and natural language processing of free text comments), but rather to identify the subset of cases with RBC transfusion for manual review using the anesthesia record.
The patient’s Hb would be the critical criterion for choosing whether to audit the RBC transfusion if the intraoperative blood loss were known, without error, to be <500 mL.10,a However, the estimated blood loss (EBL) in the anesthesia information management system (AIMS) is missing for many cases.11 For example, at Massachusetts General Hospital, even after excluding all outpatient surgery, the EBL was not entered for 40.1% of cases.12 Second, reported EBL values are often biased and/or rounded.11,13,14 Also, for surgical procedures uncommonly performed, the sample sizes are too small to compensate for missing or inaccurate EBL, precluding accurate estimation of the a priori risk of RBC transfusion.15–20
Previously, we addressed these 3 limitations in using EBL for the decision of whether to perform a preoperative blood type and antibody screen (“Type and Screen”).11,21 We showed that a valid and reliable solution is to rely on the fundamental predictive characteristic of the procedure: the median EBL from the historical data in the hospital’s AIMS. This solution works because the median is robust to methods of imputation for the missing values.11
In the current study, we tested 2 hypotheses:
Hypothesis #1: Most cases (>50%) selected for auditing would be among procedures with median EBL <500 mL.
Hypothesis #2: More cases with RBC transfusion would be selected for auditing based on (a) absent Hb or missing EBLb than (b) the nadir Hb among procedures with median EBL ≥500 mL.c
The importance of hypothesis #1, if accepted, is that the decision to audit should not focus on procedures with large median blood loss because most cases with RBC transfusion would be for procedures that usually have low blood loss. Without automated decision-making to choose selected cases for auditing, the perioperative records would need to be reviewed manually for each case.
The importance of hypothesis #2, if accepted, is that audit decisions cannot validly ignore cases with a missing value for Hb or EBL.20 In other words, only considering high median blood loss cases with nadir Hb above the transfusion threshold would miss more than half the cases that should be audited for appropriateness of the transfusion.
The Thomas Jefferson University Institutional Review Board approved this study with waiver of informed consent. The data analyzed were deidentified AIMS data from all 400,000 cases performed at Thomas Jefferson University Hospital among patients >16 years of age over a period of 11 years, 2006–2016, including 12,616 cases with RBC transfusion (Table 1). The data contained no patient-identifiable fields; individual anesthesiologists and surgeons were also deidentified. The University of Iowa Institutional Review Board declared that the statistical analysis of the deidentified data did not meet the regulatory definition of human subjects research.
The median EBL for each case was calculated for its primary scheduled surgical procedure (Figure 1).11 For example, if a case was scheduled for a primary procedure of “laryngectomy” and a secondary procedure of “insertion of gastrostomy tube,” the EBL attributed to the case was the median EBL among all laryngectomies. For purposes of calculating the median EBL for the case, if the EBL was missing, and no RBC units were administered, 0 mL was substituted. This substitution does not mean that we are assuming the EBL was 0 mL, just that it was less than the median. (Any value less than the median could have been used without affecting the results, because the median would have been unchanged.) If there were RBC administration and the EBL were missing, the imputed EBL was the allogeneic RBC units × 500 mL + cell saver mL transfused, if applicable.
The preoperative Hb used was the most recent value within 30 days before the patient entered the operating room, including such measurements on the day of surgery. The “intraoperative” Hb included values measured within 2 hours after the patient exited the operating room (ie, within 2 hours after arrival to the postanesthesia care unit or the intensive care unit). At Thomas Jefferson University Hospital, Hb values obtained by anesthesia providers using point-of-care devices (HemoCue, Brea, CA) and manually entered into the AIMS were included, as were values obtained via arterial blood gases (sent to the chemistry laboratory) and complete blood counts (sent to the hematology laboratory). An electronic interface between the laboratory computer system and the AIMS was present. For the timing of preoperative and intraoperative Hb, the timestamp used was when the data were transmitted to the AIMS. Some Hb would have been known sooner by a phone call from the laboratory to the operating room, because “stat” values are often verbally reported before they are entered into the laboratory database. However, we used the nadir Hb (ie, minimum value) among values measured either preoperatively and intraoperatively, and thus ensured that results would be insensitive to the timing. For details, see Supplemental Digital Content, Appendix, http://links.lww.com/AA/C22.
Analyses were performed based on whether there was any RBC transfusion during the case, rather than based on transfusion of each RBC unit, for 2 reasons. First, the cost of auditing (ie, labor) is proportional to the number of anesthetic records audited, not the number of RBC units transfused. Second, transfusion decisions, modeled based on the number of RBC units versus based on the incidence of transfusion, are not comparable for many procedures.11 We previously showed that, when many procedures are considered, there is no single appropriate probability distribution for modeling the number of RBC units transfused.11 The simple conceptual model of transfusion using a Poisson distribution is invalid for many procedures.11 Thus, the modeling by procedure was done based on a Bernoulli distribution—transfused or not transfused—for each case.11
Hypotheses #1 and #2 were evaluated using 6 tests of ratios of independent binomial proportions (ie, the Bernoulli distribution) (StatXact 11.0; Cytel, Inc, Cambridge, MA). For hypothesis #1, the 2 groups compared were (a) cases with the primary scheduled procedure having a median EBL <500 mL versus (b) cases with the primary scheduled procedure having median EBL ≥500 mL. For hypothesis #1, 4 different conditions were considered, resulting in a comparison of 4 different ratios (Table 2). For hypothesis #2, groups compared with (a) cases with RBC transfusion and primary scheduled procedure with median EBL <500 mL and absent Hb or missing EBL20 versus (b) cases with RBC transfusion and primary scheduled procedure with median EBL ≥500 mL and nadir Hb below the transfusion threshold. For hypothesis #2, 2 different Hb thresholds were considered, 9 and 10 g/dL, resulting in a comparison of 2 ratios (Table 3). The lower value was selected to provide a reasonable cap on the percentage of cases with intraoperative transfusion that would need to be audited, not as a recommendation for when intraoperative transfusion should take place. The higher value was analyzed as a sensitivity analysis. For both hypothesis #1 and #2, ratios >1.0 favored the hypothesis.
The unconditional confidence intervals (CIs) for the ratios were calculated by inverting the one-sided tests; these intervals were asymptotic because of the large denominators.22 Each of the 8 analyses in this paper treated the 2 proportions being compared as statistically independent; however, cases overlapped between the 2 groups, putting the assumptions of statistical independence in question. Therefore, we repeated each of the 8 analyses while treating the data as quality control chart (time series) data. We created sequential batches of 1-year intervals. The ratios studied were calculated for each batch (ie, year). Then, the 11 ratios (ie, 1 per year) were compared among years to 1.0 using the one-group Student's t test. All P values and CIs were 1-sided, matching the hypotheses. To be conservative, we limit our results to findings P < .0001; the CIs are all 99%.
Finally, to consider the implications of hypotheses #1 and #2, in the last section of the Results (“Application of Results”), we compared the incidences of audited RBC transfusion among (deidentified) anesthesiologists. If there was only 1 anesthesiologist on the case, that anesthesiologist was considered responsible for the transfusion(s), regardless of when the anesthesiologist signed in on the case. If there were multiple anesthesiologists (ie, the previous anesthesiologist was relieved), the anesthesiologist who signed into the case closest to but before the time of the first transfusion was considered responsible. If there were multiple anesthesiologists but none signed in before the start of the first transfusion, the transfusion was attributed to the first anesthesiologist who signed in. Statistical analysis was performed first for the 127 anesthesiologists each with at least 1 transfusion (Table 1). We used the exact χ2 test (StatXact) because multiple cells’ expected values were small (ie, <5). Subsequently, to study individual outlier anesthesiologists, the exact χ2 test was partitioned among the anesthesiologists each with at least 100 cases, with or without transfusion.23 (This is the same as performing multiple Fisher exact tests.) Bonferroni correction was used for these P values to control the familywise error rate for the 41 comparisons. By definition, control for the familywise error rate at the 0.01 level resulted in control of the false discovery rate to <1.0%.
No statistical power analysis was performed because we used all available data and our sample size was enormous (400,000 cases). Post hoc, the sample size was effectively large based on all 12 of the P values testing hypotheses #1 and #2 achieving P < .0001. All 6 of the relative risks of transfusion audit had lower 99% confidence limits exceeding 2.52 (Tables 2–3).
Characteristics of the 400,000 cases studied and the 12,616 cases with RBC transfusion are described in Table 1.
Excluding cases with no Hb documented in the perioperative period, the estimated relative risk of audit among cases with intraoperative RBC transfusion was 3.0 greater for cases with EBL <500 mL compared to cases with EBL ≥500 mL using a Hb audit threshold of 9.0 g/dL (99% CI, 2.6–3.4; P < .0001; Table 2). For the same comparison using a Hb audit threshold of 10.0 g/dL, the estimated relative risk of audit was 3.0 (99% CI, 2.5–3.7; P < .0001; Table 2). If cases with absent Hb or missing EBL20 also are audited, the estimated relative risks are 5.6 (99% CI, 5.1–6.2) for a Hb audit threshold of 9.0 g/dL and 8.9 (99% CI, 7.6–10.3) for a Hb audit threshold of 10.0 g/dL (both P < .0001; Table 2). Hypothesis #1 was accepted; most cases (>50%) selected for auditing would be among procedures with median EBL <500 mL.
Table 4 explains the result above. Although a greater percentage of cases received RBC transfusion among procedures with median EBL ≥500 mL (26.8%) vs <500 mL (2.4%), most cases with transfusion were among procedures with median EBL <500 mL (P < .0001). Sorting all 12,616 cases with transfusion by the median EBL of the procedure, the 50th percentile procedure had median EBL = 200 mL. Thus, half of the cases with transfusion were for procedures with median EBL less than the median EBL for hip replacement (Table 1).
Among the cases with intraoperative RBC transfusion, 17.7% were for procedures with median EBL <500 mL where there was no documented Hb (from ≤30 days preoperatively through 2 hours after operating room exit) or no documented EBL20 (Table 1). This finding contrasted with 4.8% and 1.9% of transfused cases of a procedure with median EBL ≥500 mL that would be audited based on receiving an RBC transfusion with a nadir Hb >9.0 and >10.0, respectively. Thus, the relative risks of audit for cases with missing Hb or EBL were 3.66 and 9.31 with a nadir Hb >9.0 and >10.0, respectively (both P < .0001; Table 3). Hypothesis #2 was accepted; more cases with RBC transfusion would be selected for auditing based on (a) absent Hb or missing EBL than (b) the nadir Hb among procedures with median EBL ≥500 mL.
Application of Results
For hospitals following AABB Patient Blood Management standards,1 we evaluated whether the automatic audit of allogeneic transfusions, displayed in Figure 1, detects significant heterogeneity among clinicians. Figure 2 shows that it would be misleading to report only anesthesiologists’ average nadir Hb versus analyzing their individual cases’ nadir Hb. From Supplemental Digital Content, Appendix, http://links.lww.com/AA/C22, the same applies to other summary measures of intraoperative Hb. Instead, comparisons are made among anesthesiologists’ percentages of cases with transfusion and either absent Hb, missing EBL,20 or (followed by a space before "nadir") nadir Hb >9 g/dL. There was significant heterogeneity among the 127 anesthesiologists (P < .0001; Table 1). There also was significant heterogeneity using a threshold nadir Hb of 10 g/dL (P < .0001). There were 41 anesthesiologists with at least 100 cases with transfusion. For both 9 and 10 g/dL thresholds, there were 5 outlier anesthesiologists, each with large percentages of cases transfused above the threshold. For each outlier, the Bonferroni corrected P value was <.0001.
Our findings show that if a threshold nadir Hb were selected as the basis for the audit decision, most cases audited would be of procedures with median EBL <500 mL (hypothesis #1). This finding has substantial implications for the auditing of RBC transfusion, including the comparisons of transfusion decisions among clinicians (Figure 2; Table 1). Specifically, most cases (ie, >50%) with RBC transfusion were among patients undergoing procedures during which the incidence of transfusion is <20%. Furthermore, approximately one-third of cases were among patients undergoing procedures with a transfusion incidence of <10% (Table 1). Consequently, graphical analyses of incidences of cases with RBC transfusion audited among anesthesiologists (or among surgeons) by procedure have a large chance of spuriously detecting outliers (ie, Type I error) or, with adjustment for multiple comparisons, failing to detect variation among anesthesiologists (Type II error).24,25 In contrast, our approach (Figure 1) of pooling among procedures, and using the nadir or absent Hb for each case with RBC transfusion, both detects substantial numbers of cases with RBC transfusions to audit and distinguishes among clinicians, while controlling for the familywise and false discovery rate. If a case is audited based on the nadir Hb, then, by definition, it would have been audited based on other intraoperative Hb criteria (see Supplemental Digital Content, Appendix, http://links.lww.com/AA/C22), because those other Hb values are higher than the nadir.
Our findings show that the auditing decision for RBC transfusions administered in operating rooms is considerably more complicated than recommendations applicable to transfusions outside the operating room based on a threshold Hb.7 More cases with RBC transfusion would be selected for auditing based on (a) absent Hb or missing EBL than (b) the nadir Hb among procedures with median EBL ≥500 mL. The latter are the procedures with sufficiently large incidences of transfusion that their comparisons among anesthesiologists have been published.8,9 A missing EBL value is important to the decision of whether to audit a case with a procedure having median EBL <500 mL. The absence of the EBL is a key factor because without review of the medical record, there are insufficient data to assume that the intraoperative blood loss exceeded the equivalent of donation of 1 unit of blood (approximately 500 mL), a procedure that takes approximately 10 minutes,26 at a blood donation center.20
Cases with procedures having median EBL <500 mL accounted for the majority of intraoperative RBC units and the vast majority of cases with RBC transfusion (Table 1). The number of cases needing review likely would decrease if institutions were to send personalized e-mails on a regular basis to providers listing their cases with RBC transfusion and missing Hb or EBL,20 as we previously demonstrated for other interventions designed to change intraoperative behavior.27 Under circumstances with transfusion and missing Hb or EBL, it is likely that auditing will reveal for many, but not all cases, that the transfusion was appropriate. Supplemental Digital Content, Table B, http://links.lww.com/AA/C22, provides incidences of missing values from another hospital (University of Miami Hospital) with the same AIMS, but different use of hard stops.d At that hospital, the same issues of missing values applied, but, as expected, with slightly different incidences: 28% among all cases and 19% among cases with RBC transfusion. The implication of our findings of hypothesis #2 for other hospitals is that when choosing RBC transfusions for audit, cases with missing values for Hb or EBL20 should not be ignored.e These implications apply regardless of whether using nadir Hb or other summary measures (see Supplemental Digital Content, Appendix, http://links.lww.com/AA/C22).
We considered Hb thresholds of 9 and 10 g/dL. Thresholds based on more restrictive transfusion practices (eg, threshold Hb of 8 g/dL) would have resulted in more auditing than we calculated. Thresholds adjusted based on the patient’s risk (eg, patient with cardiovascular disease undergoing vascular surgery as opposed to a healthy woman undergoing cesarean delivery) would require software curated over time when changes occur in the electronic medical record system or diagnosis codes are revised.2,3
Missing values in the AIMS does not mean that the anesthesia care providers were oblivious to the EBL. It is highly likely that they were aware of the extent of bleeding. Rather, we are addressing a systems-based issue related to missing clinical documentation at the studied hospital. For example, if there were hard stops preventing closure of the AIMS record at the end of the case unless the EBL was entered, there would be fewer missing EBL. Similarly, the absence of a Hb means only that the Hb was not documented in the electronic record; the Hb may have been known (eg, measured using a point-of-care device, but not entered into the AIMS). Finally, some missing preoperative Hb values may have been scanned into the record as images, and thus not accessible through querying the AIMS database.
In conclusion, we showed validity, reliability, and utility of a new statistical method to automatically select cases with RBC transfusion to audit. The automation (Figure 1) is suitable for hospitals electing to follow the AABB guidelines and for hospitals that wish to monitor the appropriateness of intraoperative transfusion as part of their local patient blood management program. Our methodology is not limited to high-volume procedures, but applies to all surgical procedures.
Name: Franklin Dexter, MD, PhD.
Contribution: This author helped design the study, obtain the data, perform the analysis, and write the manuscript.
Name: Richard H. Epstein, MD.
Contribution: This author helped design the study, obtain the data, perform the analysis, and write the manuscript.
Name: Johannes Ledolter, PhD.
Contribution: This author helped perform the analysis and write the manuscript.
Name: Susan M. Dasovich, MD.
Contribution: This author helped design the study and write the manuscript.
Name: Jay H. Herman, MD.
Contribution: This author helped write the manuscript.
Name: Joni M. Maga, MD.
Contribution: This author helped obtain the data and write the manuscript.
Name: Eric S. Schwenk, MD.
Contribution: This author helped obtain the data and write the manuscript.
This manuscript was handled by: Marisa B. Marques, MD.
1. Standards Program Committee of the AABB. Standards for a Patient Blood Management Program. Process Control. 2014:1st ed. Bethesda, MD: American Association of Blood Banks (AABB); 10–11.
2. Hovaguimian F, Myles PS. Restrictive versus liberal transfusion strategy in the perioperative and acute care settings: a context-specific systematic review and meta-analysis of randomized controlled trials. Anesthesiology. 2016;125:46–61.
3. Docherty AB, O’Donnell R, Brunskill S, et al. Effect of restrictive versus liberal transfusion strategies on outcomes in patients with cardiovascular disease in a non-cardiac surgery setting: systematic review and meta-analysis. BMJ. 2016;352:i1351.
4. Carson JL, Guyatt G, Heddle NM, et al. Clinical practice guidelines from the AABB: red blood cell transfusion thresholds and storage. JAMA. 2016;316:2025–2035.
5. American Society of Anesthesiologists Task Force on Perioperative Blood Management. Practice guidelines for perioperative blood management: an updated report by the American Society of Anesthesiologists Task Force on Perioperative Blood Management. Anesthesiology. 2015;122:241–275.
6. Stangenberg L, Curran T, Shuja F, Rosenberg R, Mahmood F, Schermerhorn ML. Development of a risk prediction model for transfusion in carotid endarterectomy and demonstration of cost-saving potential by avoidance of “type and screen.” J Vasc Surg. 2016;64:1711–1718.
7. Haspel RL, Uhl L. How do I audit hospital blood product utilization? Transfusion. 2012;52:227–230.
8. Frank SM, Savage WJ, Rothschild JA, et al. Variability in blood and blood component utilization as assessed by an anesthesia information management system. Anesthesiology. 2012;117:99–106.
9. Ejaz A, Spolverato G, Kim Y, Frank SM, Pawlik TM. Identifying variations in blood use based on hemoglobin transfusion trigger and target among hepatopancreaticobiliary surgeons. J Am Coll Surg. 2014;219:217–228.
10. Palmer T, Wahr JA, O’Reilly M, Greenfield ML. Reducing unnecessary cross-matching: a patient-specific blood ordering system is more accurate in predicting who will receive a blood transfusion than the maximum blood ordering system. Anesth Analg. 2003;96:369–375.
11. Dexter F, Ledolter J, Davis E, Witkowski TA, Herman JH, Epstein RH. Systematic criteria for type and screen based on procedure’s probability of erythrocyte transfusion. Anesthesiology. 2012;116:768–778.
12. Wanderer JP, Anderson-Dam J, Levine W, Bittner EA. Development and validation of an intraoperative predictive model for unplanned postoperative intensive care. Anesthesiology. 2013;119:516–524.
13. Bose P, Regan F, Paterson-Brown S. Improving the accuracy of estimated blood loss at obstetric haemorrhage using clinical reconstructions. BJOG. 2006;113:919–924.
14. McConnell JS, Fox TJ, Josson JP, Subramanian A. “About a cupful”—A prospective study into accuracy of volume estimation by medical and nursing staff. Accid Emerg Nurs. 2007;15:101–105.
15. Dexter F, Macario A. What is the relative frequency of uncommon ambulatory surgery procedures performed in the United States with an anesthesia provider? Anesth Analg. 2000;90:1343–1347.
16. Dexter F, Traub RD, Fleisher LA, Rock P. What sample sizes are required for pooling surgical case durations among facilities to decrease the incidence of procedures with little historical data? Anesthesiology. 2002;96:1230–1236.
17. Dexter F, Epstein RH, Bayman EO, Ledolter J. Estimating surgical case durations and making comparisons among facilities: identifying facilities with lower anesthesia professional fees. Anesth Analg. 2013;116:1103–1115.
18. Dexter F, Ledolter J, Hindman BJ. Quantifying the diversity and similarity of surgical procedures among hospitals and anesthesia providers. Anesth Analg. 2016;122:251–263.
19. Dexter F, Epstein RH. For assessment of changes in intraoperative red blood cell transfusion practices over time, the pooled incidence of transfusion correlates highly with total units transfused. J Clin Anesth. 2017;39:53–56.
20. O’Neill L, Dexter F, Park SH, Epstein RH. Uncommon combinations of ICD10-PCS or ICD-9-CM operative procedure codes account for most inpatient surgery at half of Texas hospitals. J Clin Anesth. 2017;41:65–70.
21. Dexter F, Ledolter J. Bayesian prediction bounds and comparisons of operating room times even for procedures with few or no historic data. Anesthesiology. 2005;103:1259–1267.
22. Miettinen O, Nurminen M. Comparative analysis of two rates. Stat Med. 1985;4:213–226.
23. Agresti A. Categorical Data Analysis2002:2nd ed. Holden, NJ: John Wiley & Sons; 82, 85.
24. Glance LG, Li Y, Dick AW. Quality of quality measurement: impact of risk adjustment, hospital volume, and hospital performance. Anesthesiology. 2016;125:1092–1102.
25. Dexter F, Hindman BJ. Do not use hierarchical logistic regression models with low-incidence outcome data to compare anesthesiologists in your department. Anesthesiology. 2016;125:1083–1084.
26. American Red Cross. Donation FAQ’s. Available at: http://www.redcrossblood.org/donating-blood/donation-faqs
. Accessed June 8, 2017.
27. Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth Analg. 2015;121:678–692.