“Failure to rescue” (FTR), which is death after a major complication, has recently gained popularity as an important quality measure for health care institutions, including those caring for adult trauma patients.1 While not every complication is preventable, the ability to rapidly recognize and appropriately treat complications has become regarded as a key concept in patient safety. Approaches to measure FTR as a quality measure were introduced by Silber and colleagues in 1992.2 The institute of Medicine report “To Err is Human” published in 1999 suggested that as many as 98,000 people die each year from preventable medical errors. This resulted in an increased emphasis on quality improvement and safety in health care, which largely focused on reducing medical errors.3 This approach has been supported by studies which showed that people who suffer from complications are more likely to die, suggesting that complications drive mortality, and hospitals with more complications would have higher mortality rates.4–6 The Centers for Medicare & Medicaid Services subsequently focused on reducing complication rates by denying hospitals reimbursement for complications that could reasonably have been prevented through the application of evidence-based guidelines.7 However, there is a growing body of literature showing that it is the failure to recognize and rescue patients from complications that drives mortality, rather than the rate of complications.8 The National Quality Forum now recognizes FTR as a major quality indicator.9
Trauma is the leading cause of death and disability for Americans between 1 years and 44 years of age and one of the leading causes of mortality worldwide.10,11 It is responsible for nearly one third of all lost years of productive life.12 Previous studies examining the impact of FTR on hospital mortality rate have focused on adults. Children suffer from distinct mechanisms of injury and have physiologic differences which affect their incidence of complications and recovery. The objective of this study is to examine whether FTR is an important mechanism driving outcome differences across low- and high-mortality hospitals caring for pediatric trauma patients, as it has shown to be in the adult population.
This study was conducted using data from the National Trauma Databank (NTDB) on children hospitalized with traumatic injuries between 2007 and 2014. The NTDB research data sets is the largest aggregation of data available on injured children and is composed of over 5 million records from over 900 trauma centers in the United States. The NTDB includes information on patient demographics, hospital characteristics, Abbreviated Injury Score codes, mechanism of injury (based on ICD-9-CM codes), deidentified hospital keys, physiology values, and in-hospital mortality and complications. For the current study, a merged data set was created with variables of specific interest using the Research Data Set files obtained from the Committee on Trauma of the American College of Surgeons for the years 2007 to 2014. Further description of the NTDB data sets can be accessed at https://www.ntdb.org.
To improve the quality of data, we only included hospitals in which fewer than 5% of children were missing demographic information, fewer than 25% were missing physiologic information (Glasgow Coma Scale [GCS] motor or systolic blood pressure), fewer than 25% of the children were transferred to a higher level of care, and fewer than 10% of the children were missing complication data (i.e., those lacking complications and an indication for a complication-free hospital course). This was in line with the methodology by Glance et al.1 when studying the adult trauma population in the NTDB. From this initial cohort of 173,959 children in 175 hospitals, we excluded those with burns and nontraumatic mechanisms of injury (24,644), children transferred to a higher level of care to limit patient duplication (4,959), children who were discharged from the emergency room (20,192), children with missing complications (1,107), and children who were dead on admission (337). The final study cohort consisted of 125,057 children in 175 hospitals.
Hospital quality was estimated using hierarchical logistic regression based on the Trauma Mortality Probability Model (TMPM-ICD9).13 This model was augmented with the addition of age, sex, mechanism of injury (blunt vs. penetrating), transfer status, age adjusted systolic blood pressure and the GCS motor component based on the model developed and validated by Glance et al.1 Multiple imputation was used to impute missing values of the motor component of the GCS and the systolic blood pressure (as normal vs. abnormal for age), using the SAS implementation of the fully conditional specification method of multiple imputation described by Van Buuren.14 Prior studies have validated the use of multiple imputation for accurately imputing missing physiologic data in the NTDB and have shown that multiple imputation techniques used to impute missing data results in hospital quality measures that are nearly identical to those based on data sets without missing values.15,16 Hospitals were specified as a random effect. Model parameters estimated using multiple imputation were combined using Rubin's rule.17 The estimates were exponentiated and used to calculate the expected mortality. The predicted probability of death for each patient was summed at the center level to obtain a predicted mortality rate for each center. The predicted mortality at each center was compared with the observed mortality, producing an observed to expected mortality ratio. Hospitals whose adjusted observed to expected mortality ratio was less than 1 and whose 80% confidence interval (CI) did not include 1 were classified as low-mortality outliers. High-mortality outliers had adjusted odds ratios greater than 1, and an 80% CI that did not include 1. The rest of the hospitals were considered to be average mortality. To evaluate the effect of American College of Surgeons (ACS) verification level on hospital outcomes, an additional analysis was conducted by stratifying children based on the ACS verification level of the treating trauma center.
The goal of our analysis was to examine the effect of hospital quality on the incidence of major complications and FTR. Major complications were defined as one of the following postinjury events: myocardial infarction (MI), cardiac arrest, pulmonary embolism (PE), pneumonia, acute respiratory distress syndrome, acute renal failure, sepsis, stroke, or abdominal compartment syndrome. These complications were chosen due to prior studies which have shown these to have the highest attributable mortality among trauma patients or they had been used previously to study FTR in trauma patients.1,18,19 Additionally, although some, such as MI and PE, are exceedingly rare in children, we felt that the others are still the most pertinent potentially treatable major complications in the pediatric trauma population.
In the initial analysis, a logistic regression model was used to assess the association between hospital quality with complications and FTR. For this initial analysis, where the dependent variable was complications, the analysis was based on the entire patient sample (125,057). For the subsequent analysis, where the dependent variable was FTR, the analysis was based only on the subset of children who experienced a major complication (939). The explanatory (independent) variable was hospital quality, expressed as: low-mortality outlier and high-mortality outlier hospitals. Exploratory analyses were conducted using logistic regression on an imputated data set. In the multivariate analyses, we adjusted for patient demographics (age and sex), whether a patient was transferred in from another hospital, GCS motor score, hypotensive for age (as a categorical variable), mechanism of trauma (penetrating vs. blunt) and injuries based on the modified TMPM model.13 Sensitivity analyses were also conducted including all patients and hospitals regardless of complications, and excluding patients who died within 48 hours of admission. p < 0.05 was considered significant for all analyses. Data management and statistical analysis were performed using R v. 3.4.2 and SAS v. 9.4 (Statistical Analytics Software Institute, Cary, NC). An IRB waiver was obtained for this study.
Of the 125,057 children, 31,600 were treated at low-mortality hospital outliers, 7,014 at high-mortality hospital outlier, and the rest at the average-mortality hospitals. Children in the low-mortality hospitals were slightly older (7 years vs. 6 years; p < 0.01) and more likely to present after penetrating trauma (0.05% vs. 0.03%; p < 0.01) compared to children in high-mortality hospitals. There was no difference in sex, GCS at presentation or age adjusted blood pressure on arrival among the low- and high-mortality hospitals (Table 1).
The overall mortality rate was 1% (1,298). Children at low-mortality hospitals were significantly less likely to die (adjusted odds ratio [OR], 0.33; 95% CI, 0.28–0.38; p < 0.01) compared with average-mortality hospitals, whereas children in high-mortality hospitals were four times more likely to die than at average-mortality hospitals (adjusted OR, 4.27; 95% CI, 3.63–5.03; p < 0.01). The rate of major complications was 0.8% (939). The most frequent major complications were pneumonia (0.42%) and acute respiratory distress syndrome (0.29%). The least common major complication was pulmonary embolus (0.01%). The unadjusted rate of major complications was lower in low-mortality hospitals (OR, 0.61; 95% CI, 0.44–0.83; p < 0.01). There was no consistent pattern for individual complications across hospital quality terciles (Table 2). Of note, average-mortality hospitals had more major complications and FTR than high-mortality hospitals; however, overall mortality was lower than the high-mortality outliers.
After adjusting for patient demographics, transfer status, mechanism of trauma, and injury severity, children in low-mortality hospitals still had a significantly lower rate of complications (adjusted OR, 0.71; 95% CI, 0.61–0.83; p < 0.01) and lower FTR rate compared to children in high-mortality hospitals (adjusted OR, 0.53; 95% CI, 0.34–0.83; p < 0.01) (Table 3). Major complications predicted mortality better than FTR (OR, 15.33 vs. 4.05), although both were statistically significant. Presence of a penetrating injury doubled the risk of FTR, motor GCS score of 1 on arrival increased the risk by four, and hypotension on arrival by three (Table 3).
To limit potential bias, a sensitivity analysis was performed including all centers and patients regardless of complication reporting, which showed less major complications in low-mortality centers (OR, 0.82; 95% CI, 0.70–0.95; p < 0.01) without any differences in FTR (OR, 0.67; 95% CI, 0.44–1.01; p = 0.05). Additionally, a second sensitivity analysis was performed excluding all children who died within 48 hours of arrival, because these deaths may have been related to the injury as opposed to a complication. This also showed lower complications in low-mortality hospitals compared to high-mortality hospitals (OR, 0.81; 95% CI, 0.68–0.96; p = 0.02), as well as a similar FTR rate (OR, 0.80; 95% CI, 0.39–1.64; p = 0.55).
When stratified by trauma certification level, ACS Trauma verification level did not predict either major complications (Level I, reference; Level II, 1.00 [0.74–1.34]; Level III, 1.51 [0.69, 3.31]), mortality (Level I, reference; Level II, 1.11 [0.78–1.58]; Level III, 1.34 [0.51–3.51]) or FTR rates (Level I, reference; Level II, 0.63 [0.28–1.40]; Level III, 4.70 [0.58–38.43]). Essentially, there was no correlation between performance status and ACS verification level.
Trauma center mortality varies significantly across institutions, leading to clinically relevant discrepancies in patient mortality between centers. While programs, such as the Trauma Quality Improvement Program, aim to identify best practices at high-performing centers and disseminate these practices across all institutions, the types of practices which lead to significantly improved hospital performance are poorly understood. The “failure-to-rescue” principle is gaining popularity as it seems to be a more sensitive marker of hospital quality compared to major complication rate, which has a weaker correlation with mortality. According to this principle, low- and high-mortality hospitals have similar major complication rates, but the better-quality hospitals are better at rescuing patients from complications and thus have lower mortality rates. Glance and colleagues1 demonstrated that adult trauma patients in high-mortality hospitals were more likely to die after a major complication compared to patients in low-mortality hospitals; however, major complication rates were similar among all hospitals. Similarly, Ghaferi and colleagues6 using the American College of Surgeons National Surgical Quality Improvement Program, showed that, for adult patients undergoing major inpatient general and vascular surgical procedures, patients in high-mortality hospitals were more likely to die after major complications compared to patients in low-mortality hospitals, despite major complication rates being similar among hospitals. This inverse correlation between hospital mortality and major complications has been shown in multiple studies.20–22 The likelihood that a patient is identified and rescued in a timely fashion after developing a major complication serves as a metric of the ability of the hospital and its staff in treating a potentially life-threatening complication.23 The finding that large gaps exist in FTR rates between high- and low-mortality hospitals highlights the importance of FTR as an important source of variation in hospital quality.
Interestingly, this concept was not demonstrated in pediatric trauma patients. In this study based on the NTDB, we found that low-mortality hospitals (defined by the modified TMPM) have both lower rates of major complications and are better able to rescue children who have major complications (lower FTR) compared with high-mortality hospitals. In light of work by Kardooni et al.24 showing the potential bias introduced by eliminating facilities without complications, we performed a sensitivity analysis without filtering for reporting of complications. A second sensitivity analysis was performed eliminating all death within 48 hours of admission assuming that many of these deaths may be due to the injury as opposed to complications. Both of these sensitivity analyses showed a clear decrease in complications among high-performing centers (low-mortality outliers) without any difference in FTR. These findings indicate that unlike for centers treating adult trauma patients, the key driver of hospital quality in centers doing pediatric trauma is the major complication rate, rather than FTR.
Interestingly, when hospitals were stratified by trauma center level, there were no correlations. We expected to see level I trauma centers exhibit fewer complications, lower FTR and lower mortality than others, but this was not the case. Comparison of trauma and nontrauma centers was not performed. It was previously reported that trauma centers reduce mortality compared to nontrauma centers,21 but a recent study performed by the National Study on the Costs and Outcomes of Trauma showed that patients treated in trauma centers were more likely to have complications compared to patients in nontrauma centers for similar injuries.25
In general, pediatric trauma patients have fewer major complications and lower mortality than adult trauma patients. Only 0.8% of patients in our analysis had major complications and the overall mortality rate was 1%; although notably, 21% of children experiencing a major complication subsequently died. Our findings imply that processes of care which aid in the prevention and early recognition of major complications, as well as those that enable the rescue of patients with complications, likely contribute to overall trauma center performance.
Our findings suggest that for pediatric trauma patients, efforts to improve the timely recognition and treatment of complications should complement current patient safety measures to reduce overall complication rates. This requires a multidisciplinary collaborative approach by physicians, nurses and other support staff which emphasizes early recognition, good communication, teamwork, and effective interventions. Rapid response teams, use of evidence-based clinical decision support algorithms which utilize data from electronic medical records, and monitors to provide early alerts are examples of system changes which may play key roles in achieving hospital-wide changes leading to lower mortality after major complications.26–28
Our study has several limitations. First, data quality is always a concern when dealing with large data sets. To mitigate this, the ACS has implemented a rigorous training program and created the national trauma data standard in 2007 to improve the quality and standardization of data. Second, a portion of hospitals contributing data to the NTDB do not code any complications.24 We attempted to minimize this problem by limiting the study sample to patients admitted to hospitals where fewer than 10% of the patients were missing complication data. However, because the NTDB does not audit hospital data, it is not possible for us to determine1 whether hospital coding of complications within our study sample is complete, and2 whether missing data on complications is distributed evenly between low- and high-mortality hospitals. Third, this study only examines one dimension of quality, which is mortality. It is possible that reductions in mortality after major complications are associated with worse functional outcomes (notable since there is a high proportion of neurologic injuries in pediatric trauma patients), in which case a lower FTR rate may not be considered equivalent to higher quality care. As the NTDB lacks data on functional outcomes it was not possible to address this. Fourth, a limitation of our analysis, and others like it that rely on mortality to classify hospitals is that FTR rate and mortality rate are by construction closely related. However, a priori, it is not obvious that FTR would be strongly associated with overall mortality rate if the primary driver of differences in hospital mortality performance were differences in hospital complication rate, as opposed to differences in FTR. In theory, patients with major complications could have experienced similar mortality outcomes at low- and high-mortality hospitals.
At hospitals caring for pediatric trauma patients, our findings suggest that the major complication rate is the key driver for differences in hospital quality. The role of FTR seems less impactful in pediatric trauma than that in adult trauma. Thus, achieving lower mortality rates in pediatric trauma and improving quality of care should be more focused on the prevention of major complications, rather than early recognition and management of complications.
E.R. and W.Z. performed the data management and statistical analysis. E.R. wrote the article. S.S. and A.V. were involved in the planning of the project and editing of the article. B.N-M. was the PI involved in all aspects of the project. B.J. contributed by giving the oral presentation at the PTS meeting.
Funding source: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflicts of Interest: none.
1. Glance LG, Dick AW, Meredith JW, Mukamel DB. Variation in hospital complication rates and failure-to-rescue for trauma
patients. Ann Surg
2. Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital and patient characteristics associated with death after surgery. A study of adverse occurrence and failure to rescue
. Med Care
3. Institute of Medicine Committee on Quality of Health Care in A. In: Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System
. Washington (DC), National Academies Press (US) Copyright 2000 by the National Academy of Sciences. All rights reserved; 2000.
4. Merkow RP, Ju MH, Chung JW, Hall BL, Cohen ME, Williams MV, Tsai TC, Ko CY, Bilimoria KY. Underlying reasons associated with hospital readmission following surgery in the United States. JAMA
5. Pradarelli JC, Healy MA, Osborne NH, Ghaferi AA, Dimick JB, Nathan H. Variation in Medicare expenditures for treating perioperative complications
: the cost of rescue. JAMA Surg
6. Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med
7. Services CfMM. Hospital-Acquired Conditions
8. Sheetz KH, Waits SA, Krell RW, Campbell DA Jr, Englesbe MJ, Ghaferi AA. Improving mortality following emergent surgery in older patients requires focus on complication rescue. Ann Surg
. 2013;258(4):614–617; discussion 617–8.
9. Quality AfHRa Failure to Rescue
10. Xu Jiaquan, Kochanek Kenneth D, Bastian Brigham A. National Center for Health Statistics
11. Krug EG, Sharma GK, Lozano R. The global burden of injuries. Am J Public Health
12. Institute of Medicine Committee on Injury P, Control. In: Bonnie RJ, Fulco CE, Liverman CT, eds. Reducing the Burden of Injury: Advancing Prevention and Treatment
. Washington (DC): National Academies Press (US). Copyright 1999 by the National Academy of Sciences. All rights reserved; 1999.
13. Glance LG, Osler TM, Mukamel DB, Meredith W, Wagner J, Dick AW. TMPM-ICD9: a trauma
mortality prediction model based on ICD-9-CM codes. Ann Surg
14. van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med
15. Moore L, Hanley JA, Lavoie A, Turgeon A. Evaluating the validity of multiple imputation for missing physiological data in the national trauma
data bank. J Emerg Trauma Shock
16. Glance LG, Osler TM, Mukamel DB, Meredith W, Dick AW. Impact of statistical approaches for handling missing data on trauma
center quality. Ann Surg
17. D. R. Multiple Imputation for Nonresponse in Surveys
. Hoboken, NJ: John Wiley & sons.
18. Ingraham AM, Xiong W, Hemmila MR, Shafi S, Goble S, Neal ML, Nathens AB. The attributable mortality and length of stay of trauma
: a matched cohort study. Ann Surg
19. Haas B, Gomez D, Hemmila MR, Nathens AB. Prevention of complications
and successful rescue of patients with serious complications
: characteristics of high-performing trauma
centers. J Trauma
20. Arozullah AM, Henderson WG, Khuri SF, Daley J. Postoperative mortality and pulmonary complication rankings: how well do they correlate at the hospital level? Med Care
21. Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A, Ferguson TB Jr, Grover FL, Edwards FH. The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg
. 2003;75(6):1856–1864; discussion 1164–5.
22. Mehta A, Efron DT, Stevens K, Manukyan MC, Joseph B, Sakran JV. Hospital variation in mortality after emergent bowel resections: the role of failure-to-rescue. J Trauma Acute Care Surg
23. Silber JH, Romano PS, Rosen AK, Wang Y, Even-Shoshan O, Volpp KG. Failure-to-rescue: comparing definitions to measure quality of care. Med Care
24. Kardooni S, Haut ER, Chang DC, Pierce CA, Efron DT, Haider AH, Pronovost PJ, Cornwell EE 3rd. Hazards of benchmarking complications
with the National Trauma
Data Bank: numerators in search of denominators. J Trauma
. 2008;64(2):273–277; discussion 277–9.
25. Ang DN, Rivara FP, Nathens A, Jurkovich GJ, Maier RV, Wang J, MacKenzie EJ. Complication rates among trauma
centers. J Am Coll Surg
26. Jones D, Egi M, Bellomo R, Goldsmith D. Effect of the medical emergency team on long-term mortality following major surgery. Crit Care
27. Hoff BM, Ford DC, Ince D, Ernst EJ, Livorsi DJ, Heintz BH, Masse V, Brownlee MJ, Ford BA. Implementation of a mobile clinical decision support application to augment local antimicrobial stewardship. J Pathol Inform
28. Yang Z, Huang Y, Jiang Y, Sun Y, Zhang YJ, Luo P. Clinical assistant diagnosis for electronic medical record based on convolutional neural network. Sci Rep