Background : Survival risk ratios (SRRs) are database-specific point estimates of survival associated with each injury International Classification of Diseases, Ninth Revision (ICD-9) code. They are derived for each injury code by dividing the number of patients who survive the code by the total number of patients that display the code. SRRs are used to measure injury severity and distribution and are most prominently featured in the ICD-9 Injury Severity Score (ICISS), which is the product of a patient’s SRRs. Because SRRs are important for trauma scoring, it is important that they be derived from a representative trauma population. The purpose of this study was to compare a new set of SRRs derived from the National Trauma Data Bank (NTDB) with SRRs derived years ago from the North Carolina Hospital Discharge Database (NCHDD).
Methods : Tests for differences of proportions were applied to determine which ICD-9 codes have significantly different SRRs in an attempt to characterize the database differences. Two different ICISSs were calculated for 170,853 eligible patients using the two different sets of SRRs, NCHDD and NTDB. The NTDB SRRs were calculated and applied using a 10-fold cross-validation to avoid bias in estimation. Estimates of discrimination for both ICISSs were calculated using the area under the receiver-operating characteristic curve. R2 and Akaike information criterion statistics were compared.
Results : A modest statistical case is made for using the NTDB SRRs rather than the NCHDD SRRs.
Conclusion : Researchers should begin using the NTDB SRRs for their outcome modeling and for ICISS calculation.
Trauma research depends on an accurate quantification of the severity of a patient’s injuries. The Abbreviated Injury Scale (AIS) has served as the foundation for injury severity classification for more than 30 years and spawned several trauma measures that summarize the effects of multiple injuries into one score. 1 The most popular of these approaches has been the Injury Severity Score (ISS), developed by Baker et al. in 1974. 2 Other AIS-derived scores based solely on anatomic descriptors such as the New Injury Severity Score (which allows for multiple injuries to a body region, unlike the ISS) and the Anatomic Profile Score proved to be more powerful predictors of mortality than the venerable ISS but have failed to supplant it. 3,4
An important advance in trauma severity assessment in recent years has been the development of the International Classification of Diseases, Ninth Revision (ICD-9) ISS (ICISS), which uses empirically derived survival risk ratios (SRRs). 5 An SRR is a database-specific point estimate of survival associated with a single injury and is defined as the number of patients who survive the injury divided by the total number of patients who display the injury. Once SRRs are available for each ICD-9 code corresponding to each known injury, the ICISS is the product of the SRRs that correspond to a patient’s set of injuries. Osler et al. calculated SRRs from the North Carolina Hospital Discharge Database (NCHDD), a nontrauma database, and applied the ICISS methodology to a set of data from the New Mexico Trauma Database. These authors and others have shown that ICISS is superior to ISS in terms of its ability to discriminate between survivors and nonsurvivors. 6–8
Other studies of larger populations confirmed these results. Sacco et al., in the first attempt to compare all of the existing AIS- and ICD-9–based scoring approaches, used data from the Pennsylvania Trauma Outcome Study and also reported that the ICISS had more discriminatory power than the ISS using SRRs from the NCHDD. 6 Meredith et al. came to the same conclusion as Sacco et al. while conducting the largest comparative scoring study to date using data from the National Trauma Data Bank (NTDB) and SRRs from the NCHDD. 7
Champion et al. in 1980, expanding on work by Sacco et al. in 1976, were among the first to recognize the importance of empirically measured estimates of injury severity. They argued for the adoption of the Anatomic Index (AI), which was based on HICDA-8 codes (the forefathers of ICD-9 codes) that were readily available in most medical practices. 9 The AI methodology considered only a patient’s worst injury and involved a complex calculation. A comparison of AI and ISS showed little difference in the measures (both had misclassification rates of 4.2%). The utility of an easily accessible severity measure such as the AI, they opined, was reason enough to reject the subjective and expensive ISS; however, AI failed to displace the ISS.
Levy et al. in 1982 proposed the Revised Estimated Survival Probability (RESP) and introduced a weighted version of an SRR that was calculated from a set of patients that sustained only one injury. 10 This weighted SRR was called a “single condition survival rate.” Because of the study design, the RESP score was applied only to patients with multiple injuries and was calculated as the product of the single condition survival rates for a patient’s set of injuries. Levy et al. wrote, “The basis of this model lies in the multiplicative law of probability; that is, if each trauma condition were acting independently of the other trauma conditions present with respect to threat of life, then the probability of a patient with multiple traumas surviving hospitalization would be equal to the product of the single condition survival rates.” The RESP was found to be significantly linearly associated with mortality in several subsets of patients but was not compared with any other score. When Goldberg et al. made this comparison in 1984, the ISS was shown to be superior to the RESP in determining mortality and other measures of outcome. 11 The advent of the ICISS methodology 12 years later raised the first expectation that injury severity scores using empiric methodologies could be equally as powerful as AIS-based scores.
The implications of a powerful ICD-9 predictor of mortality are many. ICD-9 codes are more readily available and familiar to hospital personnel than AIS codes. Also, AIS coding is significantly more expensive than ICD-9 coding. However, legitimate doubts persist in the eyes of some researchers about the ICISS methodology. Among the objections to the ICISS is the NCHDD database from which the original SRRs were calculated. The NCHDD had very low overall mortality (2.5%), which is not thought to be representative of trauma center mortality across the country. Also, the NCHDD is only a regional sample and might reflect regional coding conventions, injury patterns, and injury prevalence not ordinarily seen in other systems. The purpose of this study is to introduce a new set of SRRs calculated from NTDB data and to numerically compare them with the original NCHDD SRRs.
Accurate SRRs are critical to trauma research. The fact that SRRs are empirically derived gives them an advantage over other scoring systems and renders them among the easiest and most powerful scores now available. Also, independent SRRs, similar to those used in the RESP score, could be used to compare actual and expected survival rates to account for the impact of injury interactions. It is very important that the SRRs be reflective of the general trauma population’s injury load for their maximum potential to be realized.
From the Departments of Surgery (J.W.M.) and Public Health (P.D.K.), Wake Forest University School of Medicine, Winston-Salem, North Carolina, and Department of Surgery, University of Vermont (T.O.), Burlington, Vermont.
Submitted for publication August 24, 2002.
Accepted for publication May 16, 2003.
Address for reprints: J. Wayne Meredith, MD, Department of General Surgery, Wake Forest University School of Medicine, Medical Center Blvd., Winston-Salem, NC 27157-1063; email: firstname.lastname@example.org.