Background : The International Classification of Diseases, Ninth Revision Injury Severity Score (ICISS) is criticized because it relies on survival risk ratios (SRRs) that are contaminated by incidents with multiple injuries. An SRR for an International Classification of Diseases, Ninth Revision code is the number of patients who survive the injury divided by the number who display it. The ICISS is the product of SRRs that correspond to a patient’s injuries. Traditional SRRs are derived from databases that include patients with multiple injuries and are biased toward mortality, making them nonindependent. Independent SRRs are derived from incidents where patients sustained only an isolated injury. The objective of this study is to compare the mortality prediction abilities of independent and traditional SRRs via the ICISS.
Methods : A 10-fold cross-validation design was used to estimate independent and traditional SRRs and their resulting ICISSs from 192,347 National Trauma Data Bank patients. Logistic regression modeled the scores as a function of mortality. The area under the receiver operating characteristic curve measured discrimination. Model fit was measured with the Akaike information criterion, a deviance statistic (lower is better). R2 values were compared to determine which score explained the most variance.
Results : The independent ICISS statistically outperforms the traditional ICISS.
Conclusion : Traditional SRRs used by the ICISS produce less accurate estimates of mortality than independent SRRs. The ICISS can be calculated in 97.9% of incidents using independent SRRs.
Trauma outcome prediction models incorporate data that are collated into trauma scores to make risk assessments. An ideal trauma score would use anatomic, demographic, physiologic, and comorbidity data and produce accurate assessments of risk for outcomes such as mortality, morbidity, length of stay, and so forth. Most of the work in trauma outcome prediction has focused on accounting for anatomic injury severity and its statistical relationship with survival.
These anatomic injury severity scores may be generally divided into two classes—Abbreviated Injury Scale (AIS) 1 severity-based scores and empirically derived scores. The AIS is the chief diagnostic coding tool of the trauma surgeon. An AIS code describes each individual injury from a traumatic incident. Of most interest to the outcome modeler is the “AIS severity” portion of the AIS code. The severity designation is determined by a board of experts and ranges from 1 (for a very minor injury) to 6 (injuries thought to be unsurvivable). AIS-based scores include the popular Injury Severity Score (ISS) 2 and other, more powerful scores such as the New Injury Severity Score 3 and the Anatomic Profile Score. 4
Empirically based scores use estimates of injury severity derived from real, observed data, not assessments determined by surgeons. The International Classification of Diseases, Ninth Revision (ICD-9) Injury Severity Score (ICISS) 5 was formulated by Osler et al. and uses survival risk ratios (SRRs) to garner estimates of risk. A survival risk ratio is a database-specific survival estimate associated with each ICD-9 code 6 and is derived by dividing the number of people in the database who survive an injury by the number of people that display the injury. In recent studies, the ICISS has been shown to be more powerful than the ISS and, because of its universal availability and easy computation, has increasingly been applied to injured populations of patients. 7,8
However, the ICISS is not without limitations. The original ICISS introduced by Osler et al. was hampered by two chief criticisms. First, the original set of SRRs introduced by Osler et al. were determined from incidents stored in the North Carolina Hospital Discharge Database (NCHDD), which is not a trauma database. Estimates of SRRs from this database may have been affected by a low, unrepresentative overall death rate (2.5%) and could have been biased by the fact that the database contained incidents from just one geographic area with hospitals that have varying levels of commitment to trauma care. These concerns were addressed by Meredith et al., who calculated a set of SRRs from the National Trauma Data Bank (NTDB) 9 and whose overall death rate (5.4%) is more in line with what is expected from a trauma database. 10 Also, the NTDB represents a more geographically diverse patient population because data are collected from more than 130 trauma care institutions from all over the country. The SRRs from the NTDB were shown to be more robust in terms of their prediction of mortality (via the ICISS) than the older NCHDD SRRs.
The second criticism of the ICISS focuses on the SRR calculation itself. Data from more serious injuries contribute the illusion of severity to the SRRs of minor injuries when they are calculated from patients that have multiple injuries. For instance, a patient may sustain a gunshot wound that causes mortal internal injuries. However, additional or artificial mortality is assigned to the minor laceration caused by the gunshot because of the calculation method. Instead of the minor laceration having an SRR of 1 (no mortality risk), the SRR might be 0.96 because of the number of superficial wounds that are concurrent with more serious injuries.
Mathematically, it is true then that the SRRs described above are not independent estimates of mortality risk because multiple injuries contribute to each SRR. Instead, they are estimates of survival that approach the true survival associated with injuries but always underestimate it. Although the performance of these mathematically flawed SRRs is reassuring, independent estimates of survival risk would have at least one key advantage. The ICISS, if it used independent SRRs, would cease to be just a score and would represent an estimate of the overall probability of survival. Independent SRRs could easily be garnered from incidents of isolated injury because there are not multiple injuries to contaminate or bias the SRR estimates, as is the case with traditional SRR estimates.
In fact, the forerunners of the SRR methodology realized the advantage of the independence approach. Levy et al. in 1982 proposed the Revised Estimated Survival Probability (RESP) and introduced a weighted version of an SRR that was calculated from a set of patients that sustained only one injury. 11 This weighted pseudo-SRR was called a “single condition survival rate.” Because of the study design, the RESP score was only applied to patients with multiple injuries and was calculated as the product of the single condition survival rates for a patient’s set of injuries (just like the ICISS). Levy et al. wrote, “The basis of this model lies in the multiplicative law of probability; that is, if each trauma condition were acting independently of the other trauma conditions present with respect to threat of life, then the probability of a patient with multiple traumas surviving hospitalization would be equal to the product of the single condition survival rates.” The RESP was found to be significantly linearly associated with mortality in several subsets of patients but was not compared with any other score. When Goldberg et al. made this comparison in 1984, the ISS was shown to be superior to the RESP in determining mortality and other measures of outcome. 12 The advent of the ICISS methodology 12 years later raised the first expectation that injury severity scores using empiric methodologies could be as equally powerful as AIS-based scores. However, the notion of independence in SRRs was not carried over from this earlier work by Levy et al. The purpose of this study is to formulate a set of independent SRRs and compare their predictive ability (in terms of mortality) to that of traditional SRRs in the multiplicative model via the ICISS.
From the Departments of Surgery (J.W.M.) and Public Health (P.D.K.), Wake Forest University School of Medicine, Winston-Salem, North Carolina, and Department of Surgery, University of Vermont, College of Medicine (T.M.O.), Burlington, Vermont.
Submitted for publication October 7, 2002.
Accepted for publication June 17, 2003.
Poster presentation at the 61st Annual Meeting of the American Association for the Surgery of Trauma, September 26–28, 2002, Orlando, Florida.
Address for reprints: J. Wayne Meredith, MD, Department of General Surgery, Wake Forest University School of Medicine, Medical Center Blvd., Winston-Salem, NC 27157-1063; email: firstname.lastname@example.org.