Nurses are critical to ensuring the safe care of patients. However, studies investigating patient outcomes associated with nurse staffing have shown mixed results (1–8). Whereas multiple observational studies have identified associations between lower patient-to-nurse ratios (higher nurse staffing) and lower rates of complications within ICUs (1–3), interventions to increase nurse staffing on general wards have not shown improvements in patient outcomes (4–8). Observational findings from the ICU—where complex nursing needs often differ from general wards—have not been explored using experimental or quasiexperimental study designs.
In response to concerns that unregulated patient-to-nurse ratios in the ICU may lead to patient harm (9); Massachusetts became the first state to mandate 1:1 or 2:1 patient-to-nurse staffing in the ICU guided by an acuity tool in March of 2016. The passage of the Massachusetts mandate created a natural experiment to evaluate the downstream effects of statewide policies applied to nurse staffing in the ICU. We hypothesized that Massachusetts ICU nurse staffing regulations would result in decreased complications and mortality for critically ill patients, when compared with patients admitted to ICUs across the country unaffected by Massachusetts regulations.
On June 30, 2014, Massachusetts General Law c. 111, § 231, “An Act relative to patient limits in all hospital intensive care units” was signed into law (10). The law established that 1) “in all intensive care units the patient assignment for the registered nurse shall be 1:1 or 1:2” and 2) that this determination should be based “on the stability of the patient as assessed by [an] acuity tool and by the staff nurses in the unit.” Regulations governing the implementation of the law were unanimously approved as 958 Code of Massachusetts Regulations 8.00 (11) on June 10, 2015. Academic medical centers, as defined by the Center for Health Information and Analysis (12), were given a compliance date of March 31, 2016. All other hospitals were required to comply with mandate requirements by January 31, 2017. Hospitals were required to report on four ICU quality measures, including central line-associated bloodstream infections, catheter-associated urinary tract infections, hospital-acquired pressure ulcers, and patient falls with injury.
Data Source and Cohort Definitions
We conducted a retrospective cohort study of patients 18–99 years old admitted to ICUs at 246 medical centers and affiliated hospitals in the United States. Data were abstracted from Vizient (Clinical Data Base/Resource Manager and Action-OI databases), a collection of administrative claims and billing information from the largest network of healthcare systems in the United States. We used difference-in-difference methods to compare outcomes in academic Massachusetts and non-Massachusetts comparator hospitals during three time periods relative to the nurse staffing compliance date for academic hospitals: a “Baseline” period: April 1, 2014, to March 31, 2015; a “Preparation” period: April 1, 2015, to March 31, 2016; and a “Post-mandate” period: April 1, 2016, to March 31, 2017 (Fig. 1; Supplemental Methods, Supplemental Digital Content 1, http://links.lww.com/CCM/D753).
The primary outcome was the change in risk-standardized mortality ratio during the Baseline and Post-mandate periods for patients hospitalized within Massachusetts academic ICUs, when compared with patients hospitalized in non-Massachusetts academic ICUs. Risk-standardized hospital mortality was calculated by Vizient 2016 Mortality Risk Adjustment Models (13). To contextualize changes in standardized mortality ratios, we also reported changes in the component observed and expected mortality rates.
We evaluated two clinical secondary outcomes: 1) a composite in-hospital complication rate composed of four complications for which reporting was mandated (central line-associated bloodstream infections, catheter-associated urinary tract infections, hospital-acquired pressure ulcers, and patient falls with injury) (1–3); and 2) rates of do-not-resuscitate (DNR) orders. Previous studies showed associations between ICU staffing and DNR rates that may influence mortality (14). We also assessed for changes in nurse staffing before and after the mandate compliance date.
Difference-in-difference approaches can be used to evaluate effects of policy interventions introduced at a specific time and can achieve unbiased effect estimates when temporal trends are the same within the intervention and comparison groups (despite differences in group characteristics) (15). After establishing parallel baseline trends in mortality between Massachusetts and non-Massachusetts hospitals in the 2 years preceding the compliance date (p for difference in trends = 0.58), we investigated the effects of the nurse staffing mandate for risk-standardized mortality using generalized linear models with a Poisson distribution and the natural log of expected mortality as an offset variable (16). Difference-in-difference estimates were evaluated using similar methods for unadjusted hospital mortality, expected mortality, complications, and DNR orders. Three Massachusetts hospitals (11 ICUs) and 33 hospitals (88 ICUs) outside of Massachusetts reported nurse staffing data during both the Baseline and Post-mandate period; we calculated the ratio of total patient-days to the total nurse-days worked by licensed nurses to yield patient-to-nurse ratios for the Baseline and Post-mandate period and compared the change in nurse staffing using t tests.
Sensitivity and Subgroup Analysis
We performed four sensitivity and subgroup analyses. First, we used a controlled interrupted time series approach (17 , 18) to evaluate the change in unadjusted mortality after implementation of the nursing mandate among academic hospitals in Massachusetts when compared with outside of Massachusetts. The controlled interrupted time series design allowed an alternative approach to assess changes to mortality at the date of mandate implementation that did not depend on risk adjustment using a standardized mortality ratio. We assessed the difference in monthly mortality rates for MA and non-MA hospitals in the 2 years before and 1 year after the legislation compliance date. Due to the presence of first-order autocorrelation (Durbin-Watson test = 1.5; p = 0.026), we used an exact maximum likelihood to account for autocorrelated errors and seasonality. Models included independent variables for time (i.e., number of months from start of follow-up), the onset of the policy intervention (i.e. pre- vs postimplementation), and time after the policy intervention (19). Polynomial models were also explored for best fit.
Second, we conducted a difference-in-difference analysis for risk-standardized and unadjusted mortality in nonacademic community hospitals. Previous analyses have shown that nurse staffing can vary between academic and community hospitals (20), and that state nursing mandates may affect hospitals differentially depending on prelegislation staffing levels (4 , 5). Nonacademic community hospitals were required to comply with mandate requirements by January 31, 2017 date; therefore, two time periods were compared: February 1, 2015, to June 30, 2015, and February 1, 2017, to June 30, 2017, for Massachusetts and non-Massachusetts comparator nonacademic hospitals.
Third, we examined the effect of the nurse staffing mandate in the subgroup of mechanically ventilated patients (using International Classification of Diseases [ICD], 9th Edition and ICD, 10th Edition procedure codes) (21). The presence of mechanical ventilation accounts for a significant increase in acuity score on a tool such as the Therapeutic Intervention Scoring System-76 (22), and therefore, patients requiring mechanical ventilation may be more likely to have nurse staffing increased as a result of the mandate.
Fourth, we performed a post hoc exploratory analysis to evaluate changes in patient outcomes within Massachusetts alone using interrupted time series analysis (17 , 18). This analysis was conducted to evaluate whether the changes in nurse staffing after implementation of state regulations were associated with changes in patient outcome, without relying on assumptions that Massachusetts was comparable to other states. Methods were similar to the controlled time series approach, with exact maximum likelihood used to account for autocorrelated errors (Durbin-Watson test = 1.4; p = 0.01) and seasonality, independent variables for time, the onset of the policy intervention, and time after the policy intervention (19) and polynomial models explored for best fit.
All statistical testing was through SAS 9.4 (SAS Institute, Cary, NC), with two-tailed and performed with α equals to 0.05. A sample size of 7,000 was required to detect a 10% relative change in mortality with α equals to 0.05 and power of 0.90. A sample size of 10,000 was required to detect a 25% relative change in complications with α equals to 0.05 and power of 0.90. This study of deidentified composite hospital data was deemed exempt from review by the Boston University Medical Institutional Review Board.
During the Baseline period, we identified 29,754 ICU admissions in the six academic hospitals in Massachusetts and 572,951 ICU admissions at 114 academic hospitals outside of Massachusetts. In the Preparation period between passage of the staffing law and mandated implementation date, we identified 29,553 ICU admissions in Massachusetts and 580,837 ICU admissions outside of Massachusetts. In the Post-mandate period, we identified 30,058 ICU admissions in Massachusetts teaching hospitals and 567,608 ICU admissions at Association of American Medical Colleges teaching hospitals outside of Massachusetts. Characteristics of the six cohorts are shown in Table 1.
Nurse Staffing Changes
Patient-to-nurse ratios decreased in Massachusetts after nurse staffing legislation (Baseline: 1.38 ± 0.16 to Post-mandate: 1.28 ± 0.15, p = 0.006) and outside of Massachusetts in the same time periods (Baseline: 1.35 ± 0.19 to Post-mandate: 1.31 ± 0.17, p = 0.002), without significant difference-in-differences for change in nurse staffing (–0.05 [95% CI, –0.03 to 0.13]; p = 0.20).
Mortality, Complications, and Advance Directives
After implementation of nurse staffing regulations, risk-standardized mortality remained stable in Massachusetts (1.20 [95% CI, 1.16–1.24] to 1.15 [1.11–1.19]; p = 0.11) but decreased outside of Massachusetts (1.15 [1.14–1.16] to 1.09 [1.08–1.10]; p < 0.001), without a significant difference-in-difference (odds ratio [OR], 1.01; 95% CI, 0.96–1.06; p = 0.69) (Fig. 2). Observed mortality (OR, 0.96; 95% CI, 0.91–1.00; p = 0.09) and expected mortality (OR, 0.95; 95% CI, 0.89–1.00; p = 0.06) also did not change significantly in Massachusetts after nursing legislation, when compared with outside of Massachusetts (Fig. 2).
The rate of complications was stable between the Baseline period and the Post-mandate period in Massachusetts (0.68–0.67%; p = 0.88) and outside of Massachusetts (0.72–0.72%; p = 0.83), with no significant difference in trends (p = 0.92). The proportion of ICU patients with a DNR status increased in Massachusetts (13.5–15.4%; p < 0.001) and outside of Massachusetts (12.3–14.5%; p < 0.001), without a significant difference-in-difference (p = 0.07).
Sensitivity analyses showed similar results to the primary analysis. Controlled interrupted time series analysis did not demonstrate a significant change in observed mortality in Massachusetts when compared with ICUs outside of Massachusetts following the Massachusetts policy implementation (β = –0.81; p = 0.23) (Supplemental Fig. 1, Supplemental Digital Content 1, http://links.lww.com/CCM/D753). Similarly, analysis of nonacademic community hospitals did not show a significant difference-in-difference of risk-standardized mortality (OR, 0.99; 95% CI, 0.91–1.13; p = 0.89). Risk-standardized mortality in Massachusetts decreased from 1.29 (95% CI, 1.21–1.37) to 1.14 (95% CI, 1.07–1.22) (p = 0.01), whereas outside of Massachusetts, the mortality index decreased similarly from 1.10 (95% CI, 1.11–1.19) to 0.96 (95% CI, 0.93–0.99) (p < 0.001) among nonacademic hospitals. Analysis of complication rates and DNR status revealed results similar to academic hospitals (Supplemental Results, Supplemental Digital Content 1, http://links.lww.com/CCM/D753). Mechanically ventilated patients also did not show a significant difference-in-difference in risk-standardized mortality (OR, 1.02; 95% CI, 0.96–1.08; p = 0.57), rate of complications (p = 0.42), or in rates of DNR orders (p = 0.31). Analysis of interrupted time series within Massachusetts alone was also not associated with a significant change in observed mortality in Massachusetts (adjusting for first-order autocorrelation, β for immediate change in mortality = –1.13; p = 0.18), although some quadratic models suggested increasing mortality trends after implementation (Supplemental Fig. 2, Supplemental Digital Content 1, http://links.lww.com/CCM/D753).
We performed an analysis of the effect of a Massachusetts law mandating the use of a risk score within ICUs to assign patient-to-nurse ratios of either 2:1 or 1:1, based upon patient acuity and anticipated care intensity. Increases in nurse staffing in Massachusetts after institution of statewide regulations were not significantly higher than staffing trends in states that did not pass ICU nurse staffing legislation. We did not identify improvements in patient outcomes associated with the Massachusetts nursing regulations. Results were robust to multiple sensitivity and subgroup analyses.
Our findings may be considered from two perspectives. First, the difference-in-difference analyses comparing changes in Massachusetts after the implementation of ICU nurse staffing regulations to changes in other states were analogous to an intention-to-treat analysis of the Massachusetts nurse staffing policy legislation. Our results suggest that the Massachusetts nursing regulations were not associated with changes in staffing or patient outcomes when compared with secular trends. Second, when examined apart from secular changes in other states, modest nurse staffing increases in Massachusetts (the change from 1.38 to 1.28 patients per nurse is approximately one extra nurse per 20-bed ICU per 12-hr shift) remained unassociated with changes in hospital mortality.
Few states have passed laws regulating nurse staffing. California previously mandated maximum patient-to-nurse ratios within acute care hospitals but did not use scoring systems to determine 1:1 staffing levels. Although California regulations were associated with an increase in nurse staffing, outcomes of hospitalized did not improve after the California law, findings similar to Massachusetts ICUs (4 , 6 , 8 , 23).
Previous observational studies of nurse staffing have identified associations between lower patient-to-nurse ratios and improved outcomes (1 , 2 , 24 , 25). However, these observational studies were unable to distinguish the causal effect of nurse staffing on patient outcomes—patients in hospitals with a higher commitment to nursing staffing may also benefit from higher overall hospital commitment to a quality of care and nurse retention, improved nurse-physician collaboration, or other predictors of patient safety (1 , 24 , 26–29). In the absence of large randomized trials to determine the causal effect of nurse staffing on patient outcomes, studies of policy interventions provide natural experiments to evaluate the effectiveness of interventions. Whereas baseline demographic differences between Massachusetts and non-Massachusetts hospitals reflect existing underlying demographic differences in the population, the difference-in-difference analysis compares within state changes and therefore is likely robust to baseline demographic differences. Multiple factors may have contributed to the minimal changes in nurse staffing and observed lack of improvement in patient outcomes associated with Massachusetts’ ICU nurse staffing regulations, including the following: 1) adequate nurse staffing and competent nurse assignment determination before the mandate; 2) significant leeway given to hospitals in both the determination of the acuity/severity tool to be used to guide nursing assignments and unclear implementation of the tool in guiding nursing assignments; 3) multidisciplinary staffing factors important for patient outcomes in addition to nursing care (pharmacists, respiratory therapists, among others); 4) secular trends of increased ICU nurse staffing outside of Massachusetts; and 5) a diluted effect on mortality due to unchanged or increased hospital-ward mortality as a result of earlier transfer from the ICU to the floor in the setting of ICU nurse staffing limitations. Hospitals used different tactics in designing or choosing acuity tools to determine nurse staffing, which may reflect uncertainty as to how to optimally distribute nursing resources.
Our findings should be interpreted in the context of limitations. First, although our analysis is based on deidentified individual patient data, risk-standardized mortality ratios were averaged across hospitals. However, interrupted time series analysis of observed mortality rates, which did not incorporate risk adjustment, corroborated the risk-standardized mortality findings. Second, nursing regulations may have had heterogeneous effects within certain patient populations or larger effects at hospitals with lower premandate staffing. However, results were similar in a subgroup of mechanically ventilated patients who were more likely to be affected by acuity score-based nurse staffing, as well as in nonacademic medical centers with generally higher patient-to-nurse ratios (5). Further studies may seek to evaluate for heterogeneity of policy effects across other patient populations, such as across type of ICU. Third, measured complications were rare (< 1% of patients), but sample size calculations suggested sufficient power to detect differences in complication rates. The complications chosen for the composite analysis were the four complications for which reporting to the state was mandated, but given rarity, were likely not ideal measures of improvements in nurse-staffing-sensitive ICU morbidity. The low rates of complications may reflect that other external forces aside from state legislation (e.g., Center for Medicaid Services and the Joint Commission) have already maximally reduced occurrence rates. Further studies should evaluate other candidate measures of nurse-staffing-sensitive morbidity; for example, Kane et al (24) identified unplanned extubations as a measure potentially sensitive to nurse staffing. Unfortunately, few academic hospitals contributed data regarding nurse staffing and therefore power to detect changes in nurse staffing was limited—although effect estimates suggest small differences in staffing changes after the legislation. Fourth, Massachusetts instituted both maximum patient-to-nurse ratios and the use of patient severity scores in a bundled approach; we were unable to distinguish effects of the patient-to-nurse ratio mandate from the regulation of nurse staffing based upon patient severity scores. It is possible that our findings are the result of suboptimal nursing resource allocation as a result of the acuity tool alone, and that using other methods of nurse staffing distribution may yield different outcomes. Fifth, we did not have data on costs attributable to this legislation, although costs associated with development and implementation of severity score-based nurse staffing and reporting compliance in response to new mandates may be substantial (30). Sixth, other outcomes important to patients (such as patient and family satisfaction, emotional harms, or postintensive care syndrome) and nurses (job satisfaction or creation of a healthier work environment leading to higher nurse retention rates) may be sensitive to nurse staffing regulations and deserve further investigation. A longer follow-up comparison period may also reveal that state legislation similar to MA may prophylactically prevent decrease in nurse staffing or may more readily facilitate appropriate increases in nurse staffing in response to rising patient complexity. Conversely, unintended adverse consequences (longer emergency department wait times or floor-to-ICU transfer times due to limits on ICU staffing, reduction in staffing with more experienced nurses) should be investigated as well. Finally, results may differ among hospitals that chose not to submit information to the Vizient database. Participation in Vizient is voluntary in nature and may reflect a sample of improvement-focused hospitals; however, all MA academic centers submitted data to Vizient, decreasing the likelihood that our findings do not represent MA hospital academic medical center outcomes.
In conclusion, Massachusetts General Law c. 111, § 231, which regulates the staffing assignments of nurses to patients in the ICU based upon patient acuity tools, failed to demonstrate improvements in patient mortality or complication rates among critically ill patients, potentially due to small effects on nurse staffing. Given the excess costs of enacting and enforcing statewide legislation to implement acuity-guided ICU nurse staffing, future efforts to regulate patient-to-nurse ratios should carefully first consider the extent of the effects of the mandate on nurse staffing levels. In line with previous studies outside of the ICU, our study of ICU nurse staffing adds further evidence suggesting that statewide legislation to mandate nurse staffing strategies may not effectively improve patient outcomes.
We thank Jane Foley, RN, BSN, MHA, Associate Chief Nurse at Beth Israel Deaconess Medical Center, for her insights.
1. Needleman J, Buerhaus P, Mattke S, et al. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002; 346:1715–1722
2. Stone PW, Mooney-Kane C, Larson EL, et al. Nurse working conditions and patient safety outcomes. Med Care 2007; 45:571–578
3. Lake ET, Shang J, Klaus S, et al. Patient falls: Association with hospital Magnet status and nursing unit staffing. Res Nurs Health 2010; 33:413–425
4. Mark BA, Harless DW, Spetz J, et al. California’s minimum nurse staffing legislation: Results from a natural experiment. Health Serv Res 2013; 48:435–454
5. McHugh MD, Brooks Carthon M, Sloane DM, et al. Impact of nurse staffing mandates on safety-net hospitals: Lessons from California. Milbank Q 2012; 90:160–186
6. Cook A, Gaynor M, Stephens M Jr, et al. The effect of a hospital nurse staffing mandate on patient health outcomes: Evidence from California’s minimum staffing regulation. J Health Econ 2012; 31:340–348
7. Donaldson N, Shapiro S. Impact of California mandated acute care hospital nurse staffing ratios: A literature synthesis. Policy
Polit Nurs Pract 2010; 11:184–201
9. McCluskey PD. Debate Renewed on Nurse Staffing. 2015Boston, MA, The Boston Globe,
11. Massachusetts Health Policy
Commission: Patient Assignment Limits for Registered Nurses in Intensive Care Units in Acute Hospitals. In: Regulation 958 CMR 800. 2015Boston, MA, Massachusetts Registrar,
14. Kerlin MP, Harhay MO, Kahn JM, et al. Nighttime intensivist staffing, mortality, and limits on life support: A retrospective cohort study. Chest 2015; 147:951–958
16. Warton EM, Parker MM, Karter AJ. How D-I-D you do that? Basic Difference-in-Differences Models in SAS®. In: Proceedings of the Western Users of SAS Software 2016 Conference, September 7, 2016. San Francisco, CA, 2016
17. Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: A tutorial. Int J Epidemiol 2017; 46:348–355
18. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 2013; 13:S38–S44
19. Wagner AK, Soumerai SB, Zhang F, et al. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002; 27:299–309
20. Welton JM, Unruh L, Halloran EJ. Nurse staffing, nursing intensity, staff mix, and direct nursing care costs across Massachusetts hospitals. J Nurs Adm 2006; 36:416–425
21. Kerlin MP, Weissman GE, Wonneberger KA, et al. Validation of administrative definitions of invasive mechanical ventilation across 30 intensive care units. Am J Respir Crit Care Med 2016; 194:1548–1552
22. Keene AR, Cullen DJ. Therapeutic Intervention Scoring System: Update 1983. Crit Care Med 1983; 11:1–3
23. Donaldson N, Bolton LB, Aydin C, et al. Impact of California’s licensed nurse-patient ratios on unit-level nurse staffing and patient outcomes. Policy
Polit Nurs Pract 2005; 6:198–210
24. Kane RL, Shamliyan TA, Mueller C, et al. The association of registered nurse staffing levels and patient outcomes: Systematic review and meta-analysis. Med Care 2007; 45:1195–1204
25. Penoyer DA. Nurse staffing and patient outcomes in critical care: A concise review. Crit Care Med 2010; 38:1521–1528; quiz 1529
26. Pronovost PJ, Dang D, Dorman T, et al. Intensive care unit nurse staffing and the risk for complications after abdominal aortic surgery. Eff Clin Pract 2001; 4:199–206
27. Needleman J, Buerhaus PI, Stewart M, et al. Nurse staffing in hospitals: Is there a business case for quality? Health Aff (Millwood) 2006; 25:204–211
28. Sakr Y, Moreira CL, Rhodes A, et al; Extended Prevalence of Infection in Intensive Care Study Investigators: The impact of hospital and ICU organizational factors on outcome in critically ill patients: Results from the Extended Prevalence of Infection in Intensive Care study. Crit Care Med 2015; 43:519–526
29. Pronovost PJ, Jenckes MW, Dorman T, et al. Organizational characteristics of intensive care units related to outcomes of abdominal aortic surgery. JAMA 1999; 281:1310–1317
30. Wall MJ, Howell MD. Variation and cost-effectiveness of quality measurement programs. The case of sepsis bundles. Ann Am Thorac Soc 2015; 12:1597–1599