Secondary Logo

The Assignment of American Society of Anesthesiologists Physical Status Classification for Adult Polytrauma Patients

Results From a Survey and Future Considerations

Kuza, Catherine M., MD*; Hatzakis, George, MD, PhD, BS, MS*; Nahmias, Jeffry T., MD, MHPE†,‡

doi: 10.1213/ANE.0000000000002450
Trauma: Original Clinical Research Report
Free
SDC

BACKGROUND: The American Society of Anesthesiologists (ASA) physical status (PS) classification system assesses the preoperative health of patients. Previous studies demonstrated poor interrater reliability and variable ASA PS scores, especially in trauma scenarios. There are few studies that evaluated the assignment of ASA PS scores in trauma patients and no studies that evaluated ASA PS assignment in severely injured adult polytrauma patients. Our objective was to assess interrater reliability and identify sources of discrepancy among anesthesiologists and trauma surgeons in designating ASA PS scores to adult polytrauma patients.

METHODS: A link to an online survey containing questions assessing attitudes regarding ASA PS classification, demographic information, and 8 fictional trauma cases was e-mailed to anesthesiologists and trauma surgeons. The participants were asked to assign an ASA PS score to each scenario and explain their choice. Rater-versus-reference and interrater reliability, beyond that expected by chance, among respondents was analyzed using the Fleiss kappa analysis.

RESULTS: A total of 349 participants completed the survey. All 8 cases had inconsistent ASA PS scores; several cases had scores ranging from I to VI and variable emergency (E) designations. Using weighted kappa (Kw) analysis for a subset of 201 respondents (101 trauma surgeons [S] and 100 anesthesiologists [A]), we found moderate (Kw = 0.63; SE = 0.024; 95% confidence interval, 0.594–0.666; P < .001) interrater-versus-reference reliability. The interrater reliability was fair (Kw = 0.43; SE = 0.037; 95% confidence interval, 0.360–0.491; P < .001).

CONCLUSIONS: This study demonstrates fair interrater reliability beyond that expected by chance of the ASA PS scores among anesthesiologists and trauma surgeons when assessing adult polytrauma patients. Although the ASA PS is used in some trauma risk stratification models, discrepancies of ASA PS scores assigned to trauma cases exist. Future modifications of the ASA PS guidelines should aim to improve the interrater reliability of ASA PS scores in trauma patients. Further studies are warranted to determine the value of the ASA PS score as a trauma prognostic metric.

Published ahead of print September 7, 2017.

From the *Department of Anesthesiology, Keck School of Medicine, University of Southern California, Los Angeles, California

Department of Surgery, Baystate Medical Center, Springfield, Massachusetts

University of California, Irvine, Orange, California.

Published ahead of print September 7, 2017.

Accepted for publication July 27, 2017.

Funding: None.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

Reprints will not be available from the authors.

Address correspondence to Catherine M. Kuza, MD, 1520 San Pablo St, Suite 3451 Los Angeles, CA 90033. Address e-mail to Catherine.kuza@med.usc.edu.

The American Society of Anesthesiologists (ASA) physical status (PS) classification was developed in 1941 with the intent to compare data and reflect patients’ preoperative state, but not prognosticate operative risk.1,2 The ASA adopted it to quantify a patient’s overall health status and provided examples of illnesses for each of the 6 ASA PS categories in recent guidelines.3,4 It is currently used to categorize a patient’s physiologic reserve at the time of surgery, aid in reimbursement of services, evaluate hospital performance, allocate resources, and develop institutional and government policies.3,5,6

Several studies have demonstrated the ASA PS to be a significant predictor of morbidity and mortality in surgical patients.5,7 It has a moderate ability to predict postoperative mortality and complications, especially for emergent surgeries.8,9 The predictive validity of the ASA PS score led to its use in risk adjustment models for nontrauma surgical outcomes such as the National Surgical Quality Improvement Program (NSQIP) and in the comparison of the quality of surgical care between hospitals.9 The ASA PS has also been used to predict mortality and outcomes in trauma patients. Higher ASA PS scores were predictive of mortality rates, discharge disposition, and complications in adult trauma patients.10 The preinjury ASA PS score is an independent predictor of mortality after trauma and has been used in 2 European trauma survival prediction models as a core data variable.11–13

Since the ASA PS score is used by the NSQIP, international trauma registries, and survival prediction models assessing risk and comparing the quality of care among hospitals, it should be reliable and have high interrater agreement.11–13 Previous studies have shown low to moderate interrater reliability and discrepancies in ASA PS scoring, especially in trauma patients.6,9,14

Although most studies have demonstrated inconsistent ASA PS scoring among anesthesiologists, the reliability of ASA PS assignment in adult polytrauma patients is unknown. Our main objectives were to assess the interrater reliability of ASA PS scores assigned to adult polytrauma cases by anesthesiologists and trauma surgeons and evaluate the factors associated with score discrepancies. We hypothesized that there is significant variability in ASA PS scores assigned to trauma patients.

Back to Top | Article Outline

METHODS

A survey comprised of questions assessing attitudes regarding ASA PS classification, demographic information, and 8 fictional yet highly realistic trauma scenarios (Supplemental Digital Content 1, Appendix 1, http://links.lww.com/AA/B985) that was created by an anesthesiologist and trauma surgeon. Demographic information included physician specialty, years of practice, type of current practice and institution, time since they last provided care for a trauma patient, billing questions, and their knowledge of the ASA PS classification and its application to trauma patients. The participants were asked to assign an ASA PS score to each patient scenario and determine if the case warranted an emergency (E) designation. An optional blank space was provided to allow for the justification of their choice. They were not provided with the recent ASA definitions.

The study was reviewed and approved by the institutional review board. An online link to the survey was e-mailed to Eastern Association for the Surgery of Trauma (EAST) members, composed of physicians, nurses, and other trauma providers. Of this group, 1562 were trauma surgeons and 167 were nonphysicians or physicians in nonsurgical specialties. Additionally, it was e-mailed to anesthesiology attending physicians through the Trauma Anesthesiology Society, to the chairmen and/or administrative assistants of 110 anesthesiology programs in the United States based on a list of anesthesiology training programs, and to the private practices of colleagues. We received a response from 43 of the 110 (39%) programs/colleagues contacted stating they would forward the survey to their department. We did not survey anesthesiology residents, fellows, or nurse anesthetists.

The cases were designed to address the following issues in ASA PS and E assignment: (1) a previously healthy patient with life-threatening hemorrhagic shock; (2) morbid obesity with a nonlife-threatening injury; (3) brain-dead organ procurement; (4) massive trauma; (5) a healthy elderly patient with a minor injury; (6) hemorrhagic shock in a Jehovah’s witness refusing blood products; (7) a previously stable patient who develops septic shock; and (8) intracranial hemorrhage with mass effect.

Survey responses were collected in REDCap.15 The ASA PS scores assigned to each case were analyzed. The median ASA score and interquartile range were determined for each scenario. To achieve a comparative experience of arm A (A = anesthesiologists) versus arm S (S = trauma surgeons; n = 101), we used only those anesthesiologists in practice for >10 years at level I trauma centers (n = 100). We believed that selecting the most experienced anesthesiologists would “correct” for the experiential element against the trauma surgeons.

The Fleiss kappa was used to analyze interrater reliability beyond that expected by chance.16–19 Rater reliability was analyzed using weighted kappa (Kw) analysis, as Kw is preferred over unweighted kappa (Kuw) for ordinal variables (eg, ASA PS). The interrater variability along with the Z-test/scores as well as agreement percentages is provided in Table 1. The Kw was estimated using quadratic weights, defined as Wi = 1 – i2/(k − 1)2, where i is the difference between categories and k is the total number of categories. Data were analyzed using R Programming Language.20

Table 1

Table 1

A post hoc sample size calculation and power analysis,21 based on Kw analysis (as per Table 1, A:Kw = 0.69 denoted as K0, S:Kw = 0.57 denoted as K1), suggested that a total sample of n = 197 would have 0.8016 power at levels of α = 0.1 and β = 0.198. These calculations were made on the grounds that K0 is the value of kappa under the null hypothesis, H0 and K1 is the value of kappa under the alternative hypothesis, H1 in which case power levels correspond to the detection of a true kappa value of 0.57 in a test of H0: kappa = 0.69 versus H1: kappa <> 0.69. Thus, the subset of 201 participants selected should have adequate power for our purposes. Recent advances in the determination of sample size calculations by Gwet22–24 based on Cantor’s25 pivotal work regarding interrater reliability would have enabled us to conduct a priori calculations as per Supplemental Digital Content 2, Appendix 2, http://links.lww.com/AA/B986.

Back to Top | Article Outline

RESULTS

Data were collected over a period of 2 months, starting from the time the online link to the survey was e-mailed to the target population. A total of 349 participants from 41 states in the United States and Washington, DC, and 6 foreign countries completed the survey. Among the respondents, there were 235 anesthesiologists (approximate response rate of 1.8%), 101 trauma surgeons (6.5% response rate), 11 participants identifying as “other,” and 2 not specifying their specialty. The demographic characteristics of all the participants who completed the survey are provided in Table 2. About half of the participants (179 [51.2%]) reported being in practice for >10 years, 151 (43.3%) cared for trauma patients within a week of taking the survey, 276 (79 %) worked at level I trauma centers, and 330 (94.6%) worked at teaching hospitals.

Table 2

Table 2

The attitudes of all participants toward ASA PS are demonstrated in Table 3. There were 293 (84%) providers who considered ASA PS to be an indicator of surgical risk, whereas 46 (13.2%) did not, and 9 (2.5%) were unsure. When assigning ASA PS, 250 (71.6%) providers used the postinjury status, while 95 (27.2%) providers used the preinjury status. Most anesthesiologists (180 [77%]) and trauma surgeons (63 [62%]) used the postinjury status.

Table 3

Table 3

The various ASA PS scores, as designated by anesthesiologists and trauma surgeons, for the 8 cases are provided in Table 4. None of the cases had 100% agreement on one ASA PS score. ASA PS scores for case 2 varied between 3 scores. ASA scores for cases 3, 4, and 8 ranged over 6 scores; cases 1, 6, and 7 ranged over 5 scores; and case 5 ranged over 4 scores.

Table 4

Table 4

Table 5

Table 5

Table 6

Table 6

We compared the responses of 101 trauma surgeons (S) to a subset of 100 anesthesiologists (A). Table 5 depicts the interquartile range, median and reference ASA PS scores, and ASA scoring frequencies for the 201 responders for each of the 8 cases. The intra- and interrater-versus-reference reliability, as determined by Kw analysis, is provided in Table 1, along with a standardized interpretation of kappa.26 There was a higher degree of agreement among anesthesiologists alone compared to trauma surgeons alone and anesthesiologists and trauma surgeons combined. The intrarater-versus-reference Kw was fair for trauma surgeons (Kw = 0.57; SE = 0.034; 95% confidence interval [CI], 0.517–0.628; P < .001) and moderate for anesthesiologists (Kw = 0.69; SE = 0.034; 95% CI, 0.641–0.730; P < .001). The interrater-versus-reference reliability for surgeons and anesthesiologists combined was moderate (Kw = 0.63; SE = 0.024; 95% CI, 0.594–0.666; P < .001). The interrater agreement was 91% and the reliability as evaluated by the Fleiss kappa was fair (Kw = 0.43; SE = 0.037; 95% CI, 0.360–0.491; P < .001; Table 6).

Back to Top | Article Outline

DISCUSSION

To our knowledge, our study is the first to evaluate the ASA PS classification in severely injured adult polytrauma patients. Our results demonstrate significant variability of ASA PS scores in both anesthesiologists and trauma surgeons. There were disagreements in the ASA PS for all 8 scenarios; scores ranged from I to IV in 2 scenarios, I–VI in 3 scenarios, and I–V in the rest. Even cases with clearly defined examples in the ASA guidelines (eg, brain-dead organ procurement) had variable ASA scores. Furthermore, the E designation varied in 5 cases.

Although our results demonstrate great variability of ASA PS assignment in polytrauma patients, our data interpretation is subject to several limitations. A major limitation of our study is the low response rate achieved. Since the survey was e-mailed through an intermediary, we could not confirm the total number of participants who received the survey and, thus, we could only calculate an approximate response rate for anesthesiologists. We estimated 30 anesthesiologists per site and 43 programs distributing the survey to their anesthesiologists, for a total of 1290 (ie, 235/1290 = 1.8% response rate). As for trauma surgeons, there were 1562 EAST members who were surgeons, but this number includes residents and fellows. It is possible that our response rate would be higher if we knew the exact number of trauma attending physicians who actually received our e-mail.

Our study was subject to nonresponse bias, and the method of survey distribution precluded comparison of responders to nonresponders. Characteristics of nonresponders that may have differed from responders include working at a nonteaching hospital or private practice setting, not working at a trauma center, having minimal exposure to trauma patients, and not recently reviewing the ASA guidelines; this may have resulted in differences in the ASA PS assignments. There is potential bias from self-selected rather than randomly selected responders, because participants who had trauma experience were more likely to take the survey, making the generalizability of the results questionable.

Additionally, we selected a subgroup of anesthesiologists, who worked at level I trauma centers and were in practice >10 years, to have comparable sample sizes for comparison in our statistical analysis. Thus, our findings may be subject to selection bias. Although we cannot clearly assuage concerns about selection bias, nearly 90% of our respondents worked at level I or II trauma centers; therefore, a greater ASA PS agreement may have been expected. Since the providers who frequently care for the most severely injured trauma patients had only fair interrater reliability, we believe our findings are important and valid.

Despite our study limitations, our results are consistent with those of similarly designed studies, which report great variability in ASA scoring (Supplemental Digital Content 3, Table 1, http://links.lww.com/AA/B987).1,6,12,14,27–30 Two studies that surveyed providers’ assignment of ASA PS scores, which included 1 to 2 adult and pediatric trauma cases, observed poor interrater reliability and greater ASA PS discrepancies in trauma compared to general surgery cases.6,29 Conversely, a study assessing anesthesiologists’ assignment of ASA PS score to patients with isolated orthopedic trauma injuries found considerable agreement of the ASA scores. However, the sample size was small (33 participants); the anesthesiologists surveyed were from the same institution; the results have not been reproduced; and the fictional cases did not reflect the complex physiology seen in polytraumatic injuries.30 Ringdal et al12 also found high degree of uniformity of ASA PS scores assigned to 50 trauma cases. However, 10 trauma registry coders at a single institution retrospectively scored the cases, making it difficult to generalize these results. Additionally, the participants were explicitly asked to assign ASA PS score based on preinjury status, which increased the likelihood of ASA PS score agreement.

Although the ASA PS classification system is well known and internationally utilized, there are several reasons that could account for the discrepancy in ASA PS scores for trauma patients.28 Changes to the ASA guidelines and definitions may contribute to interrater ASA inconsistencies, especially if they were not recently reviewed.2,31 Additionally, the paucity of specific examples for trauma patients in the 2014 ASA guidelines (ie, only ASA V had specific examples for trauma patients) may contribute to the disagreement in ASA PS scores.32 Our respondents reported factors echoed by other studies,1,6,14,28 contributing to ASA PS inconsistencies, which included old age, hemorrhagic shock, septic shock, potential airway difficulty and aspiration risk, potential surgical complications, obesity, refusal of blood transfusion, orthopedic injuries (open versus closed fracture), and massive trauma. Furthermore, whether pre- or postinjury status should be utilized and when E designation is appropriate were common factors among all the cases resulting in a wide range of ASA PS scores.

Minor injuries and less emergent cases (eg, case 2) resulted in greater ASA PS agreement. Emergent and complex polytrauma cases, such as the remaining 7 cases, resulted in a wide scatter of ASA PS scores and exemplified problems with the ASA classification system in adult polytrauma patients.

Different interpretations of “massive trauma” and “constant threat to life” can lead to major differences in ASA PS scores. The ASA guidelines32 provide massive trauma as an example of ASA V; however, this is a subjective example. Cases 1, 4, 6, and 8 are examples of massive trauma, yet had variability in ASA PS assignments. There appeared to be confusion as to what injuries constitute “massive trauma.” Cases 1 and 4 are examples of hemorrhagic shock, a common and often lethal condition associated with trauma. Hemorrhagic shock is not cited as an example in the ASA guidelines, and it may be beneficial to assign it to a specific ASA PS category.

Case 6, a Jehovah’s witness refusing all blood products in hemorrhagic shock, illustrated the subjective nature of the ASA classification system. Many respondents stated that a religious belief system should not change the ASA PS assignment, while others reasoned that refusal of blood products in the setting of hemorrhagic shock warranted a higher ASA PS. Case 8 provided an example of a patient with massive trauma, traumatic brain injury with mass effect, and unlikely survival without surgery. Although there should have been unanimous agreement on an ASA PS of VE, it still had discrepancies. This case highlighted the subjective nature of ASA guidelines and suggested providers were unfamiliar with the recent ASA guidelines. These cases convey the importance of encouraging provider education on the most recent ASA PS guidelines, providing a better definition of massive trauma, and assigning specific examples of traumatic injuries and common physiologic consequences, such as hemorrhagic shock, to corresponding ASA PS scores.

Case 3, a brain-dead organ donor undergoing procurement, should have had the highest ASA agreement, yet responses varied even among anesthesiologists. Furthermore, there was inconsistent E assignment; clarifying this in the ASA guidelines should improve interrater agreement.

Case 5, an 89-year-old man with hypertension and a closed fracture, raised several interesting points. Many responders assigning a higher ASA PS argued that elderly patients have physiologic changes that hamper their recovery and healing from trauma, lack physiological reserve, and are more prone to perioperative complications. Old age is not stratified by the ASA guidelines, but there is a precedent for age, as there is an ASA classification for preterm babies.

This case also raised issues with respect to E assignment. A closed fracture should be emergently brought to the operating room if there is vascular or nerve compromise. Several respondents stated that if this were an open fracture, this would be an emergent case. However, recent trauma orthopedic guidelines no longer consider open fractures to be a surgical emergency.33

The patient in case 7 developed septic shock. While the ASA guidelines state that sepsis warrants ASA IV, the ASA category for septic shock is not specified. Patients in septic shock have an overall higher mortality rate of 40% to 80% compared to 10% to 20% in patients with sepsis.34 Given the relative frequency of emergency operations on patients in septic shock, the ASA guidelines should provide an agreed-upon ASA PS assignment for septic shock.

To be an effective bedside tool, the ASA PS guidelines cannot be cumbersome, and it is impractical to have a detailed list of every traumatic injury. However, an example of traumatic injuries for each ASA category would be helpful. A recent study demonstrated that when providers were given a reference with examples of ASA PS and asked to assign ASA PS to 10 hypothetical cases, there was increased agreement of ASA scores compared to using only the ASA definitions without examples.35 A summary of the suggested changes to the ASA PS classification system to improve ASA agreement is provided in Supplemental Digital Content 4, Table 2, http://links.lww.com/AA/B988.

It is important that the ASA PS classification system have high interrater reliability and agreement, because the ASA PS has been used in several risk prediction models, such as the American College of Surgeons risk prediction calculator.9 Although ASA PS was not initially intended to be a predictor of surgical risk, several studies demonstrated higher ASA PS scores to be associated with increased postoperative complications, hospital length of stay, blood loss, delirium, and morbidity and mortality.7–11,27,31,36,37 The current National Trauma Database (NTDB) prediction model does not utilize ASA PS in its risk calculation. It uses patient variables upon initial presentation; it does not factor the patient’s condition if he or she subsequently declines or requires surgical intervention at a later time. Additionally, the NTDB potentially underreports important comorbidity and outcome data.38 Several studies demonstrated that incorporating preexisting medical conditions into trauma survival prediction models improved the prediction of mortality and outcomes.11,39–43 The ASA PS may be a potentially useful variable, because it is a familiar, easy to use tool that assigns a numeric value to patient comorbidities. Two studies using preinjury ASA PS in trauma survival prediction models reported improved predictive power of survival.11,12 Kilgo et al44 stated that a patient’s worst injury discriminates survival better than currently used injury scores. A postinjury ASA PS score may be useful in NTDB risk prediction models and might have better outcome predictive ability in trauma patients than preinjury ASA PS. Further research on this is warranted.

In conclusion, there is fair interrater reliability of ASA PS scores assigned to polytrauma patients. At present, there are too many inconsistencies in ASA PS classification for it to be optimally used to stratify surgical risk in trauma patients. Future ASA guideline updates should specify whether to use pre- or postinjury status and provide specific examples of traumatic injuries (ie, hemorrhagic shock) in an attempt to increase interrater agreement.

Back to Top | Article Outline

ACKNOWLEDGMENTS

We thank Maureen McCunn, MD, MIPP, FCCM, Associate Professor, Department of Anesthesiology and Critical Care at the University of Maryland School of Medicine, Division of Trauma Anesthesiology, R Adams Cowley Shock Trauma Center, University of Maryland Medical System, Baltimore, MD, USA, and Ali Salim, MD, Division Chief, Trauma, Burns, and Surgical Critical Care, Professor of Harvard Medical School, Department of Trauma, Burn and Surgical Critical Care, Brigham and Women’s Hospital, Boston, MA, USA, for their help in the preparation of this manuscript. Additionally, we also thank the members of the ASA Committee on Trauma and Emergency Preparedness (COTEP) and Trauma Anesthesiology Society (TAS) for their help in distributing the survey and their support.

Back to Top | Article Outline

DISCLOSURES

Name: Catherine M. Kuza, MD.

Contribution: This author helped contribute to study design, data collection, and preparation, authorship, and revisions to the manuscript.

Name: George Hatzakis, MD, PhD, BS, MS.

Contribution: This author helped contribute to the statistical methodology, analysis and interpretation of the data, and manuscript writing and editing process.

Name: Jeffry T. Nahmias, MD, MHPE.

Contribution: This author helped contribute to study design, data collection, and preparation, authorship, and revisions to the manuscript.

This manuscript was handled by: Richard P. Dutton, MD.

Back to Top | Article Outline

REFERENCES

1. Saklad MGrading of patients for surgical procedures. Anesthesiology. 1941;2:281–284.
2. Owens WD, Felts JA, Spitznagel EL JrASA physical status classifications: a study of consistency of ratings. Anesthesiology. 1978;49:239–243.
3. Fitz-Henry JThe ASA classification and peri-operative risk. Ann R Coll Surg Engl. 2011;93:185–187.
4. Dripps RD, Lamont A, Eckenhoff JEThe role of anesthesia in surgical mortality. JAMA. 1961;178:261–266.
5. Davenport DL, Bowe EA, Henderson WG, Khuri SF, Mentzer RM JrNational Surgical Quality Improvement Program (NSQIP) risk factors can be used to validate American Society of Anesthesiologists Physical Status Classification (ASA PS) levels. Ann Surg. 2006;243:636–641.
6. Aronson WL, McAuliffe MS, Miller KVariability in the American Society of Anesthesiologists Physical Status Classification Scale. AANA J. 2003;71:265–274.
7. Wolters U, Wolf T, Stützer H, Schröder TASA classification and perioperative variables as predictors of postoperative outcome. Br J Anaesth. 1996;77:217–222.
8. Tiret L, Hatton F, Desmonts JM, Vourc’h GPrediction of outcome of anaesthesia in patients over 40 years: a multifactorial risk index. Stat Med. 1988;7:947–954.
9. Sankar A, Beattie WS, Wijeysundera DNHow can we identify the high-risk patient? Curr Opin Crit Care. 2015;21:328–335.
10. Stewart D, Janowak C, Liepert A, et alASA-PS is associated with mortality rate among adult trauma patients. Academic Surgical Congress Abstracts, No. 61.18. Available at: http://www.asc-abstracts.org/abstracts/68-18-asa-ps-is-associated-with-mortality-rate-among-adult-trauma-patients/. Accessed August 21, 2017.
11. Skaga NO, Eken T, Søvik S, Jones JM, Steen PAPre-injury ASA physical status classification is an independent predictor of mortality after trauma. J Trauma. 2007;63:972–978.
12. Ringdal KG, Skaga NO, Steen PA, et alClassification of comorbidity in trauma: the reliability of pre-injury ASA physical status classification. Injury. 2013;44:29–35.
13. Jones JM, Skaga NO, Søvik S, Lossius HM, Eken TNorwegian survival prediction model in trauma: modelling effects of anatomic injury, acute physiology, age, and co-morbidity. Acta Anaesthesiol Scand. 2014;58:303–315.
14. Haynes SR, Lawler PGAn assessment of the consistency of ASA physical status classification allocation. Anaesthesia. 1995;50:195–199.
15. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JGResearch electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381.
16. Cohen JA coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.
17. Fleiss JL, Cohen J, Everitt BSLarge sample standard errors of kappa and weighted kappa. Psychol Bull. 1969;72:323–327.
18. Cohen JWeighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70:213–220.
19. Fleiss JLMeasuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–382.
20. Hornik KR FAQ.” The Comprehensive R Archive Network. 2.1 What is R? Available at: https://cran.r-project.org/. Accessed November 26, 2015.
21. Flack VF, Afifi AA, Lachenbruch PA, Schouten HJASample size determinations for the two rater kappa statistic. Psychometrika.1988;53:321–325.
22. Gwet KLComputing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61:29–48.
23. Gwet KLVariance estimation of nominal-scale inter-rater reliability with random selection of raters. Psychometrika. 2008;73:407–430.
24. Gwet KLHandbook of Inter-Rater Reliability. 20102nd ed. Gaithersburg, MD: Advanced Analytics, LLC; 11–184.
25. Cantor ABSample-size calculations for Cohen’s Kappa. Psychol Methods. 1996;1:150–153.
26. Shrout PEMeasurement reliability and agreement in psychiatry. Stat Methods Med Res. 1998;7:301–317.
27. Ranta S, Hynynen M, Tammisto TA survey of the ASA physical status classification: significant variation in allocation among Finnish anaesthesiologists. Acta Anaesthesiol Scand. 1997;41:629–632.
28. Mak PHK, Campbell RCH, Irwin MGThe ASA physical status classification: inter-observer consistency. Anaesth Intens Care. 2002;30:633–640.
29. Aplin S, Baines D, DE Lima JUse of the ASA Physical Status Grading System in pediatric practice. Paediatr Anaesth. 2007;17:216–222.
30. Ihejirika RC, Thakore RV, Sathiyakumar V, Ehrenfeld JM, Obremskey WT, Sethi MKAn assessment of the inter-rater reliability of the ASA physical status score in the orthopaedic trauma population. Injury. 2015;46:542–546.
31. Daabiss MAmerican Society of Anaesthesiologists physical status classification. Indian J Anaesth. 2011;55:111–115.
32. ASA Physical Status Classification System. 2014. Available at: https://www.asahq.org/resources/clinical-information/asa-physical-status-classification-system. Accessed November 2, 2016.
33. Halawi MJ, Morwood MPAcute management of open fractures: an evidence-based review. Orthopedics. 2015;38:e1025–e1033.
34. Martin GSSepsis, severe sepsis and septic shock: changes in incidence, pathogens and outcomes. Expert Rev Anti Infect Ther. 2012;10:701–706.
35. Hurwitz EUsing examples best when classifying ASA physical status. Anesthesiology News. January 11, 2016. Available at: http://www.anesthesiologynews.com/Clinical-Anesthesiology/Article/01-16/Using-Examples-Best-When-Classifying-ASA-Physical-Status/34622/ses=ogst. Accessed August 21, 2017.
36. Grosflam JM, Wright EA, Cleary PD, Katz JNPredictors of blood loss during total hip replacement surgery. Arthritis Care Res. 1995;8:167–173.
37. Zakriya KJ, Christmas C, Wenz JF Sr, Franckowiak S, Anderson R, Sieber FEPreoperative factors associated with postoperative change in confusion assessment method score in hip fracture patients. Anesth Analg. 2002;94:1628–1632.
38. Hemmila MR, Jakubus JL, Wahl WL, et alDetecting the blind spot: complications in the trauma registry and trauma quality improvement. Surgery. 2007;142:439–448.
39. Morris JA Jr, MacKenzie EJ, Edelstein SLThe effect of preexisting conditions on mortality in trauma patients. JAMA. 1990;263:1942–1946.
40. McGwin G Jr, MacLennan PA, Fife JB, Davis GG, Rue LW IIIPreexisting conditions and mortality in older trauma patients. J Trauma. 2004;56:1291–1296.
41. Brennan PW, Everest ER, Griggs WM, et alRisk of death among cases attending South Australian major trauma services after severe trauma: the first 4 years of operation of a state trauma system. J Trauma. 2002;53:333–339.
42. Moore L, Lavoie A, Le Sage N, et alUsing information on preexisting conditions to predict mortality from traumatic injury. Ann Emerg Med. 2008;52:356–364.e2.
43. Moore L, Lavoie A, Turgeon AF, et alImproving trauma mortality prediction modeling for blunt trauma. J Trauma. 2010;68:698–705.
44. Kilgo PD, Osler TM, Meredith WThe worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. J Trauma. 2003;55:599–606.

Supplemental Digital Content

Back to Top | Article Outline
© 2017 International Anesthesia Research Society