Cost Savings From Reduced Catheter-Related Bloodstream Infection After Simulation-Based Education for Residents in a Medical Intensive Care Unit

Abstract

Introduction: 

Interventions to reduce preventable complications such as catheter-related bloodstream infections (CRBSI) can also decrease hospital costs. However, little is known about the cost-effectiveness of simulation-based education. The aim of this study was to estimate hospital cost savings related to a reduction in CRBSI after simulation training for residents.

Methods: 

This was an intervention evaluation study estimating cost savings related to a simulation-based intervention in central venous catheter (CVC) insertion in the Medical Intensive Care Unit (MICU) at an urban teaching hospital. After residents completed a simulation-based mastery learning program in CVC insertion, CRBSI rates declined sharply. Case-control and regression analysis methods were used to estimate savings by comparing CRBSI rates in the year before and after the intervention. Annual savings from reduced CRBSIs were compared with the annual cost of simulation training.

Results: 

Approximately 9.95 CRBSIs were prevented among MICU patients with CVCs in the year after the intervention. Incremental costs attributed to each CRBSI were approximately $82,000 in 2008 dollars and 14 additional hospital days (including 12 MICU days). The annual cost of the simulation-based education was approximately $112,000. Net annual savings were thus greater than $700,000, a 7 to 1 rate of return on the simulation training intervention.

Conclusions: 

A simulation-based educational intervention in CVC insertion was highly cost-effective. These results suggest that investment in simulation training can produce significant medical care cost savings.

Recent reimbursement decisions by the Centers for Medicare and Medicaid Services¹ have increased national focus on reducing preventable complications in hospitalized patients. Prior work has demonstrated that interventions which reduce preventable complications such as catheter-related bloodstream infections (CRBSI) also decrease hospital costs.^2–5 Although there is increased interest in cost savings associated with interventions to reduce hospital-acquired infections, little is known about the cost-effectiveness of interventions such as simulation-based education.

Simulation has become an increasingly popular technique to teach and assess the skills of trainees in invasive procedures.^6,7 Several studies show that simulation-based education improves actual patient care in areas such as emergency airway management,⁸ laparoscopic surgery,^9,10 colonoscopy,^11,12 bronchoscopy,¹³ and Advanced Cardiac Life Support.¹⁴

Beginning in 2006, Northwestern University Internal Medicine and Emergency Medicine residents completed simulation-based mastery learning in central venous catheter (CVC) insertion before clinical rotations in the Medical Intensive Care Unit (MICU).^15,16 This program resulted in improved trainee skill and reduced CVC insertion complications, including a significant decrease in the rate of CRBSI.^15–17 The aim of this study was to estimate hospital cost savings related to the reduction in CRBSI and examine the extent to which these savings offset simulation training costs.

METHODS

Study Design and Data Collection

This was an intervention evaluation study of cost savings related to a simulation-based intervention in CVC insertion in the MICU of Northwestern Memorial Hospital (NMH), an 897-bed tertiary-care urban teaching hospital. The NMH MICU is a 20-bed facility that treats approximately 1500 patients/yr. Five or six second and third year internal medicine and emergency medicine residents rotate in the MICU every 4 weeks.

Previous research demonstrated that the simulation-based mastery learning program in CVC insertion, as opposed to conventional hospital infection control efforts such as patient care bundles, was primarily responsible for a highly significant reduction in the MICU CRBSI rate.¹⁷ In the prior study, 98% of CVCs were inserted by residents, and there was an 84.5% decrease in CRBSIs. Importantly, this reduction was sustained for 16 months after the intervention.¹⁷ Based on concurrent comparison with an NMH surgical intensive care unit staffed by residents who had not received the simulation intervention and lengthy discussions with clinicians, infection control nurses and hospital staff, who could not identify any other mechanism for the observed highly significant reduction in CRBSI rates, we concluded that the observed decline in the rate of CRSBI was a direct result of simulation-based education (Figure 1).¹⁷

Figure 1.: Quarterly catheter-related bloodstream infection (CRBSI) rates before and after a simulation-based educational intervention in the Medical Intensive Care Unit (MICU).

This study calculated the financial implications of the simulation-based intervention by comparing CRBSI rates for the year before the intervention (December 2005 to November 2006) to the year after simulator-trained residents began staffing the MICU (December 2006 to November 2007). Using hospital cost accounting data, we derived the estimated incremental cost and length of stay (LOS) associated with MICU patients with a CVC and a CRBSI during the study period. We then compared savings from reduced CRBSI after the simulation training intervention to the cost of the simulation training. The Northwestern University Institutional Review Board approved the study.

Simulation Intervention

From December 2006 to November 2007, 69 second and third year internal medicine and emergency medicine residents completed a mandatory simulation-based education program in CVC insertion before rotating in the MICU.^15–17 The training included two 2-hour education sessions featuring a lecture, ultrasound training, deliberate practice with the CVC simulator (Simulab's CentralLineMan), and instructor feedback. The CVC simulator features realistic tissue with ultrasound compatibility, an arterial pulse and self-sealing veins and skins. Education sessions used uniform didactic material on CVC indications and complications, as well as stepwise demonstrations of internal jugular and subclavian CVC insertions. Evidence-based guidelines for CRBSI reduction were emphasized (hand washing, full sterile barrier technique, chlorhexidine skin preparation, avoidance of the femoral site, and prompt CVC removal).^2–4 These sessions were supervised by a senior hospitalist faculty member with expertise in CVC insertion (J.H.B.). Residents were evaluated before (pretest) and after (posttest) simulation training using a validated checklist.¹⁵ At posttest, residents were required to meet or exceed a minimum passing score set by an expert panel for both internal jugular and subclavian procedures.¹⁸ After completing simulation-based mastery learning in CVC insertion, the simulator-trained residents rotated through the MICU beginning in December 2006.

Cost Accounting Framework and CRBSI Rates

CRBSI incidence was measured by NMHs Infection Control and Prevention staff using standard criteria in accordance with protocols described by the National Healthcare Safety Network, formerly the National Nosocomial Infections Surveillance System.^3,19 All positive blood cultures from the MICU were identified, and medical records were reviewed by trained infection control personnel to determine whether criteria for CRBSI were met. Infection control personnel determined CRBSI incidence in accordance with routine hospital policies and were blind to the nature of this study and the timing of the simulation-based intervention.

Using NMH administrative data with International Classification of Diseases (ICD-9)-coded procedures, we determined that 477 patients had a CVC inserted in the MICU during the study period. Patients were included in the analysis only if the CVC insertion was performed during the patients' documented MICU stay. These records were then matched to infection control CRBSI incidence data. We found that before our simulation-based intervention, the average infection rate was 4.2/100 MICU CVC admissions (11 infections in 239 CVC patients). After the intervention, when all second and third year residents (n = 69) in the MICU received simulation-based training, the rate was reduced to 0.42/100 admissions (one infection in 238 CVC patients), thus preventing an estimated 9.95 CRBSI cases in the year after the intervention. We used the estimated additional cost, and LOS of these 9.95 prevented CRBSIs to compute cost savings for the year after simulation training.

Hospital costs were obtained from the hospital's finance department. Their cost estimates, including both direct and indirect costs, are allocated to activity charge codes within each department based on either relative value units, such as labor minutes or supply costs, or ratio of cost to charges. All costs were adjusted to 2008 dollars using the Bureau of Labor Statistics Annual Consumer Price Index for inpatient hospital services.²⁰ Simulation training costs were based on a facility rental rate of $45/h, and staff and faculty support was estimated as a salary percentage. Supply costs were compiled and also adjusted to 2008 dollars using the Bureau of Labor Statistics Annual Consumer Price Index for medical supplies.²⁰

Statistical Analysis

Two methods are customarily used to analyze the cost-effectiveness of nonrandomized quality improvement interventions.^21–23 One approach uses a regression-derived propensity score to equate similar groups of patients based on their predicted probability of experiencing a specific adverse event. These results are then used to estimate cost differences between matched cases and controls, based solely on matched controls with a propensity score similar to the cases to avoid obvious selection biases that might favor the intervention.^21,22 A second, more direct approach relies on estimating the attributable cost of adverse events directly from a risk-adjusted regression analysis of costs, using the coefficient on adverse event incidence in the regression model to estimate the additional costs associated with each adverse event prevented.²³ We performed both of these analyses to estimate the costs and additional hospital days associated with CRBSI (SPSS Version 17, Chicago, IL).

First, we estimated the predicted probability of CRBSI for all 477 MICU patients who had a CVC inserted during the study period from a multiple logistic regression model controlling for patient age, sex, and each patient's Charlson Score, a proxy for severity of illness.²⁴ The Charlson score is based on 17 clinically significant comorbidities, derived from secondary ICD-9 diagnosis codes based on the Deyo administrative data method,²⁵ and predicts 1-year mortality for hospitalized medical patients. Each comorbidity is assigned a specific weight based on the degree of its relative risk, and these weights are combined to generate an overall score for each patient.²⁴ The resulting logistic regression-predicted probability of CRBSI produces a CRBSI propensity score (ranging from 0.0 to 1.0) for each patient.

Then, we eliminated all patients whose probability of a CRBSI was lower than any observed patient with a CRBSI from our analysis; these patients were deemed to have noncomparable risk and were thus unsuitable controls. We compared mean hospital costs between “case” patients with a CRBSI and the remaining “control” patients who did not have a CRBSI across four strata (quartiles) of roughly equal CRBSI propensity scores. Summing costs across each case-control strata allowed us to make an initial estimate of true additional costs (and hospital days) related to the CRBSI incidence.

Next, we produced a duplicate analysis in which we again eliminated all patients with CRBSI propensity scores that were lower than any observed patient with a CRBSI. We estimated the association of CRBSI incidence with costs and LOS using linear regression models that controlled for MICU CVC patients' age, sex, and Charlson score. The coefficient on CRBSI incidence in those models reflects the expected costs and hospital days associated with CRBSI after controlling for the effects of case mix differences. We present linear rather than log-transformed results because of ease of interpretation and the fact that antilogged results were very similar.

RESULTS

Simulation Costs

During the first year of the intervention, operating costs to train 69 residents, including supplies, faculty and staff time, and space rental, totaled $111,916. These initial costs included onetime purchases of Simulab's CentralLineMan, an ultrasound machine and a supply cart. Other supply costs included CVC kits, replaceable CentralLineMan tissues, ultrasound probe covers, and sterile gowns and drapes (Table 1). The predicted annual cost to maintain the project was $89,455 in 2008 dollars.

Table 1: Year 1 Simulation-Based Intervention Costs Adjusted to 2008 Dollars

Case Control Estimated Cost and LOS of a CRBSI

During the preintervention period, the 239 CVC patients had a mean age of 60.5 years (SD = 16.4), were 56% female, and had a mean Charlson score of 3.54. The 238 CVC patients in the postintervention period had a mean age of 61.1 years (SD = 17.1), were 55% female, and had a mean Charlson score of 3.97; these differences were nonsignificant. The regression-derived CRBSI propensity scores for predicted probability of CRBSI ranged from 0.2% to 6.9% across all 477 patients with MICU CVC insertions. The 12 actual CRBSI patients had propensity scores ranging from 1.2% to 6.8%. Sixty-eight non-CRBSI cases were thus excluded as controls because their predicted probability of infection was less than 1.2%. The remaining 409 “control” patients were divided into four strata based on nearly equal propensity scores (Table 2) and matched to “case” patients with CRBSI in the same predicted probability strata.

Table 2: Total Costs and Length of Stay by Propensity Score Quartile

Using the propensity score matched case-control comparison method, we averaged the quartile results and estimated the mean costs and LOS (total hospital days and MICU days) of a patient with a CRBSI, when compared with the mean costs and LOS of a patient without a CRBSI. Mean differences between case and control patients across quartiles ranged from $45,000 to $100,000, from 5 to 22 hospital days, and 13 to 27 MICU days. Summing across these quartile differences, we attributed overall mean additional costs of $82,730, 14.2 additional patient hospital days and 12.1 patient MICU days to the incidence of a CRBSI (Table 2).

Linear Regression Estimated Cost and LOS of a CRBSI

Linear regression models of cost and LOS controlling for age, sex, and Charlson score produced very similar results for the cost associated with an incident CRBSI. The coefficients on CRBSI in these models provided an estimate of $82,005 (P < 0.001, 95% CI = $51,299–$112,791), 13.8 additional patient hospital days (P = 0.001, 95% CI = 5.9–21.6), and 12.2 additional patient MICU days (P < 0.001, 95% CI = 8.8–20.8) associated with a CRBSI.

Cost Savings

Using these two methods and 9.95 CRBSI prevented by the simulation-based intervention, the total annual estimated savings were $823,164 and $815,950,141 and 137 patient hospital days and 120 and 121 MICU days. When compared with the cost of our intervention ($111,916), the net savings ranged from $704,034 to $711,248 (a 7–1 rate of return).

DISCUSSION

Several authors have called for studies linking medical education and simulation to reduced downstream healthcare costs.^26–28 Our results add to what is known about simulation- based education by documenting cost savings from improved quality of care that far exceed the cost of the educational intervention. Our findings demonstrate convincingly that simulation-based education in CVC insertion was a cost-effective intervention that directly benefited patients. Specific patient benefits included fewer CRBSIs and decreased length of MICU and total hospital stays.

This study confirms prior work linking the presence of a CRBSI to prolonged LOS and increased hospital costs.^4,23 Both methods produced a CRBSI-associated cost of approximately $82,000. This is higher than past estimates of CRBSI-associated total hospital costs ranging from $34,508²² to $63,572.²³ However, these estimates were based on data from 1998 to 2000, and ours reflect 2006–2007 data adjusted to 2008 dollars.

When interpreting any study of cost savings, it is critical to be sure that the intervention was truly responsible for lower costs. This is due to the fact that system, cultural, and process changes can also result in improved quality of care.²⁹ However, we believe that the simulation-based intervention was the primary reason for the substantial cost savings we report. This is because no other process or quality improvement changes occurred in the MICU during the study period, and CRBSI rates did not decline in other ICU settings at NMH. Additionally, MICU staff members (fellows, faculty, and nurses) were not involved in our study and were likely unaware that simulation training was ongoing. For these reasons, we do not believe that cuing, reinforcement of CVC insertion algorithms, or other changes in unit procedures and protocols were responsible for the reduction in hospital costs shown in this study.

This study has several additional limitations. First, it reflects the experience at one hospital over a relatively short period of time. However, it is similar in scope to other reports regarding CRBSI rate reduction in critical care settings.^2,5,21 Second, our regression models were simple. Inclusion of more detailed clinical data related to illness severity and susceptibility to infection might improve predictions of cost and LOS and change model coefficients. However, our cost estimates may be conservative because they do not include significant costs often incurred after hospital discharge such as home intravenous antibiotics and home health nursing services. Third, our facility rental fee of $45/h may be modest compared with fees at other simulation centers. This is primarily due to the fact that we only used space and did not incur charges for equipment, simulator staff, or supplies. Despite this, our analysis shows our intervention would remain cost-effective with higher facility rental fees. Specifically, a rate of $1000/h still provides nearly 2 to 1 return.

In summary, our results show that a simulation-based education program in the MICU of an academic teaching was very cost-effective. There are ample reasons for clinicians, hospitals, and patients to seek to decrease the incidence of CRBSI. With the trend toward reduced Medicare reimbursement to hospitals for patient care involving hospital-acquired conditions including CRBSI,¹ it is imperative for providers to consider interventions, such as the use of simulation, to improve quality, and reduce reimbursement risk. These findings highlight the promise of simulation-based education to reduce costs and mitigate a significant source of harm in the critical care environment.