Secondary Logo

Journal Logo

Attempting to validate the overtriage/undertriage matrix at a Level I trauma center

Davis, James W. MD; Dirks, Rachel C. PhD; Sue, Lawrence P. MD; Kaups, Krista L. MD

Journal of Trauma and Acute Care Surgery: December 2017 - Volume 83 - Issue 6 - p 1173–1178
doi: 10.1097/TA.0000000000001623
AAST 2016 Plenary Papers

BACKGROUND The Optimal Resources Document mandates trauma activation based on injury mechanism, physiologic and anatomic criteria and recommends using the overtriage/undertriage matrix (Matrix) to evaluate the appropriateness of trauma team activation. The purpose of this study was to assess the effectiveness of the Matrix method by comparing patients appropriately triaged with those undertriaged. We hypothesized that these two groups are different, and Matrix does not discriminate the needs or outcomes of these different groups of patients.

METHODS Trauma registry data, from January 2013 to December 2015, at a Level I trauma center, were reviewed. Overtriage and undertriage rates were calculated by Matrix. Patients with Injury Severity Score (ISS) of 16 or greater were classified by activation level (full, limited, consultation), and triage category by Matrix. Patients in the limited activation and consultation groups were compared with patients with full activation by demographics, injuries, initial vital signs, procedures, delays to procedure, intensive care unit admission, length of stay, and mortality.

RESULTS Seven thousand thirty-one patients met activation criteria. Compliance with American College of Surgeons tiered activation criteria was 99%. The Matrix overtriage rate was 45% and undertriage was 24%. Of 2,282 patients with an ISS of 16 or greater, 1,026 were appropriately triaged (full activation), and 1,256 were undertriaged. Undertriaged patients had better Glasgow Coma Scale score, blood pressure, and base deficit than patients with full activation. Intensive care unit admission, hospital stays, and mortality were lower in the undertriaged group. The undertriaged group required fewer operative interventions with fewer delays to procedure.

CONCLUSION Despite having an ISS of 16 or greater, patients with limited activations were dissimilar to patients with full activation. Level of activation and triage are not equivalent. The American College of Surgeons Committee on Trauma full and tiered activation criteria are a robust means to have the appropriate personnel present based on the available prehospital information. Evaluation of the process of care, regardless of level of activation, should be used to evaluate trauma center performance.

LEVEL OF EVIDENCE Therapeutic and care management, level III.

From the Department of Surgery (J.W.D., R.C.D., L.P.S., K.L.K.), UCSF Fresno, Fresno, California.

Submitted: August 25, 2016, Revised: May 10, 2017, Accepted: June 2, 2017, Published online: July 17, 2017.

This study was presented at the 75th annual meeting of the American Association for the Surgery of Trauma and Clinical Congress of Acute Care Surgery, September 14, 2016 in Waikoloa, Hawaii.

Address for reprints: James W. Davis, MD, Department of Surgery, UCSF Fresno, 1st Floor Admin, 2823 Fresno Street, Fresno, CA; email:

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

The American College of Surgeons (ACS) Committee on Trauma (COT) Optimal Resources Document (ORD) defines mandatory minimum criteria for trauma team activation that are based on mechanism of injury, physiologic, and anatomic criteria from the prehospital environment. The ORD describes a methodology to evaluate potential overtriage and undertriage using the retrospectively calculated Injury Severity Score (ISS) to evaluate the appropriateness of the trauma activation with tiered trauma response. This evaluation, which classifies patients with an ISS of 16 or greater as major trauma patients, seeks to ensure that delays in care are minimized without overburdening the system.

The 2006 ORD focused on prehospital triage, noting that “it was generally agreed” that a 25% to 50% rate of nonmajor trauma patients taken to a trauma center (overtriage) was acceptable to maintain a rate of major trauma patients taken to a nontrauma center (undertriage) at less than 5%.1 However, in the 2014 ORD, it was stated that “Most agree that an acceptable percentage of overtriage is in the range of 25% to 35%,” whereas the undertriage rate remained at 5%.2 Additionally, the 2014 ORD introduced the Matrix methodology (Matrix), which uses ISS to evaluate overtriage and undertriage of trauma team activation.2 The performance improvement section then requires that the potential overtriage and undertriage cases, based on the level of trauma team activation, and rates of overtriage and undertriage be reviewed quarterly; failure to do so can result in a Criterion Deficiency (16–7).2

Assessing overtriage and undertriage based on the retrospectively calculated ISS is potentially flawed. Prehospital providers and emergency department (ED) physicians and nurses do not have the ability to determine the ISS before activating the trauma team; therefore, trauma centers use the COT-mandated criteria (mechanism, physiologic, and anatomic) to determine the level of trauma team activation. Performance is then evaluated by a number of process and outcome measures (delays to laparotomy, craniotomy or catheter based intervention for hemorrhage control, and so on). The purpose of this study was to assess the effectiveness of the Matrix by comparing patients appropriately triaged with those undertriaged by Matrix methodology. We hypothesized that patients in these two groups are different and that the ISS-driven Matrix does not accurately discriminate the needs or outcomes of these different groups of patients.

Back to Top | Article Outline


Trauma registry data from January 1, 2013, through December 31, 2015, at Community Regional Medical Center, a 650 bed, ACS verified level I trauma center in Fresno, California, were reviewed. Patients that arrived at CRMC, directly from the scene or transferred from another facility, were activated according to modified ACS tiered trauma team activation (Fig. 1). Patients can be activated by field emergency medical services, by the mobile intensive care nurse (MICN) in the radio room or by emergency medicine physicians in the ED. Full trauma team activation includes an attending trauma surgeon, a senior surgical resident (postgraduate year 4 or 5), and a junior resident (postgraduate year 2 or 3) from the trauma service as well as an emergency medicine attending physician, and a senior and junior emergency medicine resident. The trauma team is required to be present on arrival of the patient or within 15 minutes of notification without advanced warning. A limited activation (trauma response) includes the same emergency medicine personnel and a junior and senior surgical resident within 30 minutes. A trauma consultation includes the emergency medicine team and a senior surgical resident or designee within 30 minutes. All limited activations must be discussed or seen with the trauma attending within 30 minutes of being seen.

Figure 1

Figure 1

Patients in the trauma registry were classified by activation level (full, limited, consultation) and by Matrix triage category according to the following definitions: undertriage, ISS of 16 or greater without full activation; overtriage, ISS less than 16 with full activation; appropriate triage, ISS of 16 or greater with full activation or ISS less than 16 without full activation. Patients whose activation level was upgraded were classified according to the highest level of activation. Errors in activation level by ED staff or MICNs received real time and ongoing feedback from the trauma division to the ED, and these data are tracked through the trauma program performance improvement process. Overtriage and undertriage rates were calculated according to Matrix methodology:

Patients at risk for undertriage (ISS ≥ 16) underwent further analysis based on activation level and form the study cohort. Patients with full activation (appropriate triage) were compared with those with limited activation or consultation (undertriage) by demographics, initial vital signs, packed red blood cell (pRBC) transfusion within the first 24 hours of arrival, number of laparotomies, craniotomies and craniectomies, angioembolizations, and outcomes. Delay to laparotomy was defined as greater than 1 hour from hypotension (systolic blood pressure less than 90 mm Hg) or greater than 4 hours from arrival to laparotomy. Delay to craniotomy for subdural or epidural hematoma was defined as greater than 4 hours from admission to craniotomy.

Continuous data are presented as mean ± SD or as median (interquartile range), and categorical data as percentages. Statistical analyses were performed using the Statistical Package for Social Sciences (IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.) Groups were compared using χ2 and Mann-Whitney U tests with significance attributed to a p value less than 0.05. The study was approved by the Institutional Review Board at Community Medical Centers and the University of California, San Francisco/Fresno.

Back to Top | Article Outline


During the 3-year study period, 7,031 patients had trauma team activation. Of these patients, 1,874 (27%) had a full activation, 1,061 (15%) had a limited activation, and 4,096 (58%) had a consultation. The compliance rate with tiered activation criteria was 99%, with the 1% of mistriaged patients being undertriaged. Of the patients with a full activation, 848 had an ISS less than 16, resulting in a Matrix overtriage rate of 45%. The undertriage rate was 24%; 1,256 patients had an ISS of 16 or greater but did not have a full activation.

The overall mortality rate for all trauma patients was 8%; with 7% in the appropriate triage group, significantly lower in the undertriage group at 4% (p < 0.001), and significantly higher in the overtriage group at 13% (p < 0.001). Of the overtriaged patient deaths, 100 (89%) of 112 occurred in patients that arrived moribund and were either dead on arrival, in cardiac arrest or died during resuscitative efforts without complete diagnostic evaluation.

There were 2,282 patients with an ISS of 16 or greater, making up the study cohort (Fig. 2). Of these patients, 1,026 were appropriately triaged by Matrix and had full trauma team activation. The remaining 1,256 had some level of tiered activation (378 limited activation and 878 consultation) and were defined as undertriaged by Matrix.

Figure 2

Figure 2

Patients who were appropriately triaged by the Matrix criteria were more significantly injured than the undertriaged groups (Table 1). The initial vital signs differed between groups; patients appropriately triaged had lower Glasgow Coma Scale (GCS) score, blood pressure, and worse base deficits and had greater ISS than those in the undertriaged groups. Additionally, the undertriaged patients were older and more likely to have fall as the mechanism of injury. Although undertriaged patients were more likely to have head injuries (Head_Abbreviated Injury Scale [AIS] score, ≥3) than the appropriately triaged patients, the median GCS was 15 in these groups (Table 1).



Outcomes are compared in Table 2. The undertriage groups required fewer operative interventions (laparotomy and craniotomy) and had fewer delays in these interventions compared with the appropriately triaged patients. Additionally, undertriaged patients were less likely to require intensive care unit (ICU) admission and had shorter ICU and hospital lengths of stay and lower mortality than the appropriately triaged patients.



A greater percentage of patients in the appropriately triaged group required pRBCs within the first 24 hours of arrival than those who were undertriaged (p < 0.001). Of those requiring blood, patients in the appropriately triaged group required more units (p < 0.001). Laparotomy was required in 58 patients in the undertriage group, 34 in the limited activation patients and 24 in the consult patients. Forty-four laparotomies were performed in less than 4 hours from admission (76%), and none of these patients were hypotensive. Of the 14 delayed laparotomies, 11 were felt to be justified with six failing nonoperative management. Additionally, there were two patients with substance abuse and denial of trauma before completion of computed tomography scan, two patients who refused consent, and one patient that went to the operating room within 40 minutes of dropping his blood pressure and trauma team involvement. There were three delays to laparotomy that were felt not to be justified, two patients were hemodynamically stable, one had a diaphragmatic hernia, the other had a traumatic abdominal wall hernia. The remaining patient had a limited activation, was hypotensive, and underwent laparotomy at 61 minutes. This was classified as a mistriage and a delay to OR (1 minute).

There were 26 angioembolizations performed in the undertriage group (16 in the limited activations, 10 in the consults) (Table 2). The time to procedure did not differ between groups (full, 4 ± 4 hours; limited, 6 ± 5 hours; consultation, 6 ± 5 hours, all p > 0.05). All of the patients in the undertriage group were hemodynamically stable on arrival. No patient in this group was identified as having an emergency angioembolization or a delay to angiographic treatment.

Back to Top | Article Outline


The ACS-COT activation guidelines were able to identify the patients in need of the highest level of trauma team activation in this study. Further, the patients with an ISS of 16 or greater that had tiered activation to a lower level by ACS criteria were hemodynamically stable with higher GCS score and better outcomes than those with ISS of 16 or greater that met higher-level criteria. Matrix methodology using the retrospectively derived ISS was not accurate in discriminating the needs of these distinctly different patient groups.

The ISS was described in 1974 as a method of expressing the cumulative effect of injury on multiple body regions.3 This score is calculated by individually squaring the AIS score4 of the three most severely injured body regions and adding the three squares. The ISS was found to correlate better with mortality than the AIS score from which it was derived. An ISS of 16 correlated to a mortality of 10%.5 The ISS has subsequently been used to compare outcomes at trauma centers,3,6 used in the calculation of probability of survival Trauma and Injury Severity Score (TRISS)5 and as a reference for validating base deficit as an indicator of trauma severity.7

The 2006 ORD,1 stated “a trauma system should establish and monitor acceptable rates of undertriage and overtriage.” In this ORD, undertriage and overtriage were described solely as a trauma system issue. Undertriage was described as major trauma patients taken to a nontrauma center, and overtriage was described as a decision that incorrectly classified a patient as needing trauma center care when retrospective analysis indicated that such care was unnecessary. Acceptable rates were defined as less than 5% undertriage and 25% to 50% overtriage. The methods described for determining these rates were the determination of the preventable trauma deaths at nontrauma centers or using an ISS greater than 15 to differentiate major from nonmajor trauma patients.1 A retrospective study, with 1,112 patients, using the Glue Grant database found an increased risk of mortality (odds ratio, 3.8) when patients were not initially triaged to a trauma center.8 Another study similarly found that patients with major trauma (defined as ISS > 15 or death within 24 hours of presentation) transported to nontrauma centers had increased mortality versus those taken directly to a trauma center.9

The 2014 ORD (Orange Book), in the chapter on prehospital trauma care, again used the ISS threshold of 16 or greater to define a major trauma patient but changed the overtriage goal to 25% to 35%.2 Despite this, there are still no studies validating the recommended overtriage and undertriage ranges used in the 2006 or 2014 ORD. Indeed, one study, in Pennsylvania, suggested that to comply with the 5% undertriage rate, trauma centers would have to increase their capacity by fivefold and that “physicians at the nontrauma centers would need to be able to better discriminate between moderate to severe and other injuries.”10 A study using the Nationwide Emergency Department Sample database found that more than one third of major trauma patients (ISS ≥ 16) in US EDs were “undertriaged.” Using a simulation analysis of the data set, and considering undertriage as a trauma system issue, the authors estimated that a capacity increase of 51% would be required to accommodate all undertriaged major trauma patients at Level I or Level II trauma centers.11

For the first time, the 2014 ORD included the level of trauma team activation at the trauma center as an indicator of potential overtriage and undertriage. The “Matrix method” was introduced and was described as a simple method for calculating overtriage and undertriage rates based on the ISS and level of trauma team activation. The performance improvement and patient safety section states that “potential overtriage and undertriage cases should be identified and reviewed quarterly” and that failure to do so can result in a Criterion Deficiency (16–7).2 Because the Matrix method is a simple way to calculate overtriage and undertriage, it is used routinely at many trauma centers rather than other, more labor intensive methods. However, to date, there is no published study that we have been able to identify that validates this methodology.

The correlation between the ISS and the resource needs of trauma patients has been previously questioned. In a 1990 review of 814 consecutive trauma patients, admitted to a Level I trauma center, Baxt and Upenieks12 noted that there was a significant need for intervention (operative intervention, fluid replacement, and invasive central nervous system monitoring) with ISS ranges between 0 and 20. Furthermore, 28% of the patients requiring intervention had an ISS less than 15 and 17% of patients with an ISS of 14 or greater required no intervention at all. The study notes that using ISS as “the single means to define major trauma may need to be reevaluated.”

The first study, to our knowledge, that described overtriage and undertriage based on trauma team activation in the trauma center was a 2009 retrospective study.13 This study evaluated the accuracy of field triage of patients, as determined by paramedic manned prehospital services versus anesthesia manned prehospital services, taken to a trauma center after the introduction of trauma team activation guidelines. Trauma team activation was based on ISS, physiologic, and mechanism of injury criteria. Overtriage and undertriage were calculated based on institutional criteria that are different than those listed in the ORD. The authors reported an overtriage rate of 55% and an undertriage rate of 10%. The authors noted no difference in unadjusted 30-day mortality and a twofold increased odds ratio of mortality (adjusted for ISS) in undertriaged patients at the trauma center.13

In a prospective performance improvement study, a trauma center modified their activation policy in an effort to reduce their undertriage rate from 15% and overtriage from 75%. They reported using the ACS-COT activation criteria and reducing the undertriage rate to 5%. However, in the study, undertriage was defined as patients with an ISS greater than 15 for which either a major or modified response was not activated.14

The current study has very different results from previous reports. Tiered trauma team activation based on the ACS COT criteria for full trauma team activation at an ACS verified Level I trauma center showed that “undertriaged” patients had better vital signs and GCS and had significantly better outcomes than patients appropriately triaged by these criteria. The ACS COT activation criteria were able to discriminate the patients that needed the resources of the full trauma team for the patients with the greatest mortality risk. Less seriously injured patients were adequately cared for with a tiered response, with good outcomes and few delays in care. A study of 4,910 trauma patients, similarly found that patients meeting trauma team activation criteria had more severe injuries, higher mortality and longer ICU and hospital lengths of stay than patients not meeting activation criteria.15

The current study has the inherent limitations of all retrospective studies. The activation level recorded in the trauma registry was the highest level that the patient received, that is, patients whose activation level was upgraded from change in condition were only recorded as that higher level. The data are from a single institution; however, with a study cohort of 2,282 patients, it represents a larger data set than a number of the previously reported prehospital and nontrauma center studies. Additionally, it is the only investigation, to our knowledge, evaluating the use of the Matrix methodology for determining undertriage and overtriage rates and demonstrates that the Matrix is unable to differentiate between high and low acuity patients.

If not for the efforts of the former chair of the verification review committee, the Matrix would not exist, and it is unlikely that the undertriage and overtriage goals described in the ORDs would ever have been evaluated. However, triage to the trauma center and level of activation at the trauma center are not equivalent. A well-functioning trauma center is expected to and should be able to adapt to changing patient condition regardless of activation level. Indeed, trauma centers routinely manage patients that are dropped off at the door by private vehicle and are expected to maintain the same standards of care without delay as they would for patients with EMS notification. Performance improvement criteria that already exist and are evaluated should detect issues with process like delays to consultation and treatment (delays to surgery or catheter based intervention) and outcomes. This study demonstrated that there were in fact, fewer delays in the groups with lower levels of activation, mostly because there was less need in these less injured patients. The evaluation of the processes of care, regardless of level of activation, seems more appropriate to evaluate trauma center performance than the retrospectively derived ISS. Additionally, nationally benchmarked outcomes data Trauma Quality Improvement Program (TQIP) should be used to assess performance and identify areas for focused improvement efforts.

In addition to a lack of research validating the Matrix, use of the ISS score to determine the appropriateness of trauma team activation has inherent difficulties. Because the ISS cannot be calculated until all diagnoses have been made, it is typically not calculated until after the patient is discharged from the trauma center. Therefore, this score is not available to prehospital providers, the MICN or the ED physician. Instead, the ACS-COT criteria are used for trauma team activation. Therefore, triage decisions should be evaluated based on the ACS-COT activation criteria.

This study demonstrates that, despite having an ISS of 16 or greater, patients with limited activation or consultations were dissimilar to patients with full activation. These data suggest that the ACS-COT full and tiered activation criteria are a robust means to have the appropriate personnel present based on available prehospital information. Triage and level of activation are not equivalent. Evaluation of the process of care, regardless of level of activation, should be used to evaluate trauma center performance.

Back to Top | Article Outline


There are no conflicts of interest or financial disclosures.

J.W.D. participated in the study concept and design, data analysis, manuscript writing, review and revision. R.D. participated in the study design, data collection and analysis, and article review and revision. L.S. participated in the critical review of the article and revision. K.K. participated in the critical review of the article and revision.

Back to Top | Article Outline


1. Committee on Trauma, American College of Surgeon. Resources for optimal care of the injured patient. Chicago, Il: American College of Surgeons; 2006.
2. Committee on Trauma, American College of Surgeon. Resources for optimal care of the injured patient. Chicago, Il: American College of Surgeons; 2014.
3. Baker SP, O’Neill B, Haddon W Jr, Long WB. The Injury Severity Score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14:187–196.
4. Rating the severity of tissue damage. I. The abbreviated scale. JAMA. 1971;215:277–280.
5. Boyd CR, Tolson MA, Copes WS. Evaluating trauma care: the TRISS method. Trauma Score and the Injury Severity Score. J Trauma. 1987;27:370–378.
6. Baker SP, O’Neill B. The Injury Severity Score: an update. J Trauma. 1976;16:882–885.
7. Davis JW, Parks SN, Kaups KL, Gladen HE, O'Donnell-Nicol S. Admission base deficit predicts transfusion requirements and risk of complications. J Trauma. 1996;41(5):769–774.
8. Nirula R, Maier R, Moore E, Sperry J, Gentilello L. Scoop and run to the trauma center or stay and play at the local hospital: hospital transfer's effect on mortality. J Trauma. 2010;69:595–599.
9. Haas B, Gomez D, Zagorski B, Stukel TA, Rubenfeld GD, Nathens AB. Survival of the fittest: the hidden cost of undertriage of major trauma. J Am Coll Surg. 2010;211:804–811.
10. Mohan D, Rosengart MR, Farris C, Cohen E, Angus DC, Barnato AE. Assessing the feasibility of the American College of Surgeons benchmarks for the triage of trauma patients. Arch Surg. 2011;146:786–792.
11. Xiang H, Wheeler KK, Groner JI, Shi J, Haley KJ. Undertriage of major trauma patients in the US emergency departments. Am J Emerg Med. 2014;32:997–1004.
12. Baxt WG, Upenieks V. The lack of full correlation between the Injury Severity Score and the resource needs of injured patients. Ann Emerg Med. 1990;19:1396–1400.
13. Rehn M, Eken T, Krüger AJ, Steen PA, Skaga NO, Lossius HM. Precision of field triage in patients brought to a trauma centre after introducing trauma team activation guidelines. Scand J Trauma Resusc Emerg Med. 2009;17:1.
14. Escobar MA, Morris CJ. Using a multidisciplinary and evidence-based approach to decrease undertriage and overtriage of pediatric trauma patients. J Pediatr Surg. 2016;51(9):1518–1525.
15. Tinkoff GH, O'Connor RE. Validation of new trauma triage rules for trauma attending response to the emergency department. J Trauma. 2002;52:1153–1158.

Overtriage; undertriage; matrix

© 2017 Lippincott Williams & Wilkins, Inc.