In 2007, Gawande et al1 proposed an Apgar score for surgical patients. This Surgical Apgar Score (SAS) was to be calculated at the end of surgery providing both a numerical grade of the patient’s condition and prognostic information regarding the patient’s chances of major complications or death. This score was derived from a retrospective analysis of data from the National Surgical Quality Improvement Program database and then validated in 2 prospective cohorts that included 102 colectomy patients and 767 patients undergoing vascular or general surgery procedures. A simple 10-point score based on estimated blood loss (EBL), lowest heart rate, and lowest mean arterial pressure (MAP) was developed. This score was found to be significantly associated with death or major morbidity at 30 days. Those with scores of ≤4 had a 30-day mortality or major morbidity rate of 58.6%, while those with a score of 9 or 10 had a mortality or major morbidity rate of only 3.6%. Following the score’s derivation and validation, the same group, this time led by Regenbogen et al2 confirmed that the prognostic value of the score was derived from intraoperative parameters after correcting for preoperative and intraoperative risk. Regenbogen et al3,4 further validated the score in another group of general and vascular surgery patients and evaluated the ability of the score to predict morbidity following colectomy.
The SAS is calculated from the EBL, lowest MAP, and lowest heart rate. Much like Virginia Apgar’s scoring system to evaluate newborns, the SAS is also a 10-point scale with higher scores indicative of better outcomes.1 The parameters of the score are provided in Table 1.
TABLE 1. -
The 10-Point Surgical Apgar Score
|Estimated blood loss (mL)
|Lowest mean arterial pressure (mm Hg)
|Lowest heart rate (BPM)
The Surgical Apgar Score is calculated at the end of any general or vascular surgery operation from the estimated blood loss, lowest mean arterial pressure, and lowest heart rate entered in the anesthesia record during the operation. The score is the sum of the points from each category.
*Occurrence of pathologic bradyarrhythmia, including sinus arrest, atrioventricular block or dissociation, junctional or ventricular escape rhythms, and asystole also receive 0 points for lowest heart rate.
BPM indicates beats per minute; mL, milliliters; mm Hg, millimeter of mercury.
Prediction of surgical risk is important for providing patients accurate information and helping to guide clinician decision-making in the perioperative period. Many risk stratification tools have been developed from analysis of surgical databases. Most, however, have relied on a combination of preoperative patient factors and specific surgical considerations. While intraoperative performance is known to impact patient outcomes, this information has been largely left out of current risk stratification tools. The SAS is unique in that it takes into account intraoperative performance to provide an estimation of postoperative morbidity and mortality that clinicians may be able to use to better risk-stratify patients following surgery. Since the inception of the SAS in 2007, it has been evaluated across a number of specific surgical disciplines, large heterogeneous surgical populations, and various countries. This is a review of the current literature that has evaluated the performance of the SAS with the aim of better understanding its discriminatory ability in the perioperative period.
MATERIALS AND METHODS
The Meta-Analysis of Observational Studies in Epidemiology Guidelines for reporting systematic reviews of observational studies were followed when performing this systematic review. Appropriate exclusions were made as data was not meta-analyzed.5,6 A Preferred Items Checklist was used in the writing of the article and is provided in Appendix 1 (https://links.lww.com/AOSO/A190).5 The search strategy and flow chart were adopted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.7,8
Search Strategy and Study Eligibility
The search strategy was drafted and agreed upon by all authors. A database search was performed on March 3, 2020, and included Ovid MEDLINE, Ovid Healthstar, and Excerpta Medica dataBASE databases. A gray literature search was also performed in accordance with previously published methods.9 Results were limited to English language and Journal Articles with no restriction on date of publication. Review articles were excluded as they did not provide new information, and abstract-only publications were excluded as study quality could not be adequately assessed. As this study aimed to evaluate the SAS for performance in discrimination, those that did not report an area under the receiver operating characteristic curve (AUROC) or concordance (c-) statistic were excluded. The search strategy as well as study inclusion and exclusion criteria are described in Appendix 2 (https://links.lww.com/AOSO/A191).
Two authors performed data extraction blindly and independently. Differences in extracted data were resolved by consensus of all three authors with reference to the article in question. Study details were extracted to a standardized table and included reference details, location, dates of data collection, study design, surgical discipline, patient population, sample size, outcomes measured, objective performance results, the author’s conclusions, sources of funding, and conflicts of interest.
Quality Assessment of Studies
Quality assessment of observational studies was performed in accordance with the Newcastle-Ottawa Scale for Assessing Non-Randomized Studies in Meta-analyses where patient selection, comparability, and outcome are assessed.10 Patient selection was evaluated in terms of representativeness of the exposed cohort, selection of the nonexposed cohort, ascertainment of exposure, and demonstration that the outcome of interest was not present at study onset. Comparability within the study was evaluated by determining measurement and control of variables between groups. Study outcome was evaluated by considering method of ascertainment of outcome, as well as adequate duration and completeness of follow-up. Two independent investigators assessed study quality blindly during data extraction. Differences were resolved by consensus of all three authors. We determined that 6 or more out of 9 of the aforementioned criteria would constitute sufficient quality for inclusion in this review. The Newcastle-Ottawa Scale is outlined in Appendix 3 (https://links.lww.com/AOSO/A192).
Data Analysis and Statistical Considerations
The performance of the SAS in various populations was evaluated with measures of discrimination. Discrimination, or how well a score correctly identifies an outcome, is reported using the c-statistic or AUROC. Here we will report and refer to the AUROC or c-statistic interchangeably as metrics of discrimination. This metric ranges from 0.5 indicating random concordance to 1.0 indicating perfect concordance.11 Values of less than 0.7 indicate poor performance, while those from 0.7 to 0.9 indicate moderate performance and values greater than 0.9 indicate high performance.12 If either the AUROC or c-statistic was reported, it was extracted as a measure of discrimination. Descriptions of the relationship between the SAS and patient outcomes or notable observations were extracted for illustration of applicability and future investigations, although they were not included in the analysis. Measures of calibration, or how well the predictive model reflects actual outcome prevalence, are also important in the assessment of a new clinical metric. We chose to limit our review to measures of discrimination as this best illustrates the utility of the score in clinical practice.
The formal database search identified 302 articles from primary literature databases. Another 2 articles were identified when searching gray literature databases. After removal of duplicates, 130 articles remained. Titles and abstracts were reviewed resulting in 2 additional duplicates removed, 1 article discarded for irrelevant content, 7 articles discarded for being descriptive/letters or editorials, 3 articles discarded for being review articles, and 44 articles discarded as they were abstract-only publications. The 73 remaining articles were obtained in full text and reviewed in detail. Two articles were excluded for not achieving the minimum quality assessment requirements. Nine articles were excluded as they did not evaluate the original SAS, while 26 articles were excluded as they did not measure SAS performance as indicated by a published AUROC or c-statistic. Reference lists from all 73 articles that were reviewed in full were hand searched with no new articles identified. The 36 remaining articles are included in this review.1,3,4,13–45 The authors who first derived and validated the SAS were successfully contacted and confirmed that no additional unpublished studies or data was available. Figure 1 summarizes the review process.
All of the 36 articles included in this review were observational in design with 19 being retrospective cohort analyses and 13 being prospective cohort analyses. Two studies were mixed retrospective and prospective cohort analysis and another 2 articles were retrospective case series.
The quality assessment results are included in Supplemental Table 1 (https://links.lww.com/AOSO/A194). Study quality was assessed using the Newcastle-Ottawa scale as all included studies were observational.10 Two studies were excluded due to inadequate methodological quality, and the remaining 36 articles included in this review were determined to have low risk of bias.
Outcomes are summarized in Supplemental Table 1 (https://links.lww.com/AOSO/A194). Study outcomes were primarily 30-day morbidity and mortality through 2 studies reported intensive care unit (ICU) admission rates, 1 reported perioperative myocardial infarction, and 2 studies evaluated in-hospital morbidity and mortality. The majority of studies used the National Surgical Quality Improvement Program morbidity definition as used by Gawande et al1 or the Clavien-Dindo classification system for postsurgical complications.46,47 Morbidity definitions are summarized in Appendix 4 (https://links.lww.com/AOSO/A193). Twenty-five studies reported mortality rates, which ranged from 0% to 29.5%.
All 36 studies reported metrics of discrimination. When using the SAS to correctly identify any postoperative morbidity, the AUROC or c-statistic ranged from 0.561 in a thoracic surgery (esophagectomy) population to 0.872 in an orthopedic spine (lumbar fusion) surgery population.31,33 When using the SAS to identify mortality the AUROC or c-statistic ranged from 0.629 in an orthopedic surgery (hip fracture) population to 0.92 in a general/vascular surgery population.1,24 When considering the ability of the SAS to identify postoperative ICU admission, 2 studies were identified with AUROC or c-statistics of 0.69 and 0.76.20,22 Supplemental Table 2 (https://links.lww.com/AOSO/A195) lists all extracted measures of discrimination. Results are summarized in Figure 2.
Results by Surgical Discipline
General and Vascular
In patients undergoing general and vascular surgeries, 14 articles reported measures of discrimination for postoperative morbidity or morbidity and mortality with metrics of discrimination ranging from 0.63 to 0.79.30,32 Two studies reported discrimination for postoperative ICU admission with AUROC/c-statistics ranging from 0.69 for delayed ICU admission to 0.76 for immediate ICU admission.20,22 In these same general and vascular surgery populations, 5 studies reported measures of discrimination for postoperative mortality with metrics ranging from of 0.74 to 0.92.1,41 Notably, the SAS had higher discriminatory ability in younger patient populations undergoing emergent surgeries.19,32
Orthopedic and Spine Surgery
In patients undergoing orthopedic surgeries, 6 studies measured discrimination for morbidity, with statistics ranging from 0.59 to 0.87.18,33 Two studies reported discrimination for mortality, with AUROC/c-statistics, reported as 0.63 and 0.76.24,38 While measures of discrimination for morbidity and mortality in the general orthopedic surgery population were some of the lowest in this review, the subset of orthopedic spine surgery found good performance with morbidity discrimination ranging from 0.75 to 0.87.25,33
Two studies measured discrimination for morbidity in the thoracic surgery population. In both reports of those undergoing esophagectomy, they found the SAS to have modest discriminatory metrics of 0.56 and 0.68.26,31
Two studies measured discriminatory ability for postoperative morbidity in the hepatobiliary population. In 1 study of patients undergoing hepatectomy for hepatocellular carcinoma, the AUROC/c-statistic was found to be 0.72, while in a group of patients with pancreatic cancer undergoing either a distal pancreatectomy or pancreaticoduodenectomy, the metric was found to be 0.60.42,44
In the urological surgery population, 1 study reported discrimination for mortality with an AUROC/c-statistic of 0.75.16
In a group of patients undergoing craniotomy for resection of meningioma, Hsu et al34 found the SAS to predict 30-day morbidity with an AUROC/c-statistic of 0.768.
In patients undergoing nonlaparoscopic gynecologic surgery, 1 study reported an AUROC/c-statistic of 0.80 for the SAS in predicting postoperative morbidity and mortality.35
Head and Neck Surgery
Two studies measured discrimination for 30-day morbidity in patients undergoing surgery for head and neck cancer. AUROC/c-statistics were reported as 0.70 and 0.73.37,43
Mixed Surgical Disciplines
Four studies evaluated the discriminatory ability of the SAS across mixed surgical disciplines. Three studies reported metrics for postoperative morbidity ranging from 0.62 to 0.73.14,17,39 One study reported on postoperative mortality and found the SAS to be moderately predictive of mortality with a statistic of 0.77.14 One study evaluated the SAS as a predictor of postoperative troponin leak or myocardial infarction in a mixed surgical population and found it to be moderately predictive with an AUROC/c-statistics of 0.80 and 0.81.29
The SAS is a 10-point score derived from the lowest intraoperative heart rate, lowest MAP, and EBL.1 This metric is unique in that it relies on intraoperative rather than preoperative data to predict postoperative risk of morbidity and mortality. Since first derived and validated by Gawande et al,1 the SAS has been thoroughly evaluated in various settings. Early on, Regenbogen et al2 investigated whether the SAS measured intraoperative performance or if it just reflected preoperative risk. They found that the SAS remained strongly correlated with postoperative outcomes even after accounting for a patient’s acute condition, comorbidities, and operative complexity. They concluded that the SAS appeared to detect differences in intraoperative management. More recently, the SAS has demonstrated consistent discriminatory ability across multiple disciplines. Higher levels of discrimination for both morbidity and mortality have been found in the general, vascular, orthopedic spine, orthopedic trauma, and gynecologic surgery populations as well as in younger patient groups.1,3,19,20,22,24,25,32,33,38,40 Lower levels of discrimination for both morbidity and mortality have been noted in general orthopedic surgery populations.13,18,27 Studies looking at other endpoints such as postoperative ICU admission or the presence of cardiac injury also found that the SAS had a high level of discriminatory ability.20,22,29
While the SAS has shown to provide moderate discriminatory ability for postoperative morbidity and moderate to high discrimination for postoperative mortality, the score has not performed as well in some general orthopedic surgery populations.13,18,27 Contrary to this, the SAS performed well in the subset of orthopedic patients undergoing spine surgeries and traumatic hip fracture repair.24,25,33,38 A possible explanation for this may be that general orthopedic surgeries are often elective, with medically optimized patients and ideal operating conditions where blood loss, hemodynamic compromise, and mortality are infrequent. Alternatively, elective orthopedic procedures are often facilitated with regional anesthesia, which again may influence hemodynamics and blood loss. Of course, an ideal score would provide perfect discrimination across a multitude of scenarios. Given that a predictive model is derived from a specific population, it is to be expected to perform better in circumstances that most closely resemble that initial derivation. The utility of the SAS is illustrated in its ease of use and the fact that it maintains a moderate level of discrimination for postoperative morbidity and a moderate to high level of discrimination for postoperative mortality across a wide range of surgical conditions. Looking at other commonly used predictive models, the performance of the SAS is strikingly similar. For example, the CHADS2 and CHA2DS2-VASc scores used for the prediction of cardioembolic stroke have c-statistics ranging from 0.60–0.80 to 0.64–0.79, respectively.48 While it was beyond the scope of this review, a number of investigators have made adjustments to the SAS or combined it with specific preoperative risk factors to better predict postoperative morbidity and mortality.38,40,43,44 This adaptability again illustrates the strength of this metric. The true and unique value of the SAS is that it clearly shows that intraoperative performance has an impact on postoperative risk and that combining this data with risk stratification tools currently in place may lead to better prediction of postoperative outcome. Of course, the hope is that a better predictive model will influence postoperative care where these complications can be mitigated.
Our study has a number of limitations. First of all, we kept our inclusion criteria broad so as to prevent any limitation on the scope of included data. Exclusion criteria were derived to ensure that studies met prespecified quality requirements and that only new data was included, which directly assessed the SAS. Excluding studies that did not report measures of discrimination improved data analysis at the expense of excluding data on score calibration. Notably, all studies that we found were observational in design. While study quality was assessed and a minimal risk of bias was required for inclusion, observational studies have intrinsically higher rates of bias that is difficult control. Secondarily, the included studies had a high degree of heterogeneity with regards to reported outcomes and statistical analysis. This made it difficult to compare results between studies and impossible to consolidate results into a meta-analysis. In addition, although we performed a gray literature search and contacted the original authors for unpublished data, the majority of studies included were found in the primary database search, and as such, our study remains at risk of publication bias. Finally, despite an exhaustive literature search, a small number of appropriate articles may have been overlooked.
Risk prediction is a key component of providing safe and informed patient care, and implementation of risk prediction models may improve outcomes by enabling early preventative measures. The SAS is a unique risk prediction metric in that it is simple to calculate and relies only on intraoperative parameters. It has shown to provide moderate discriminatory ability for postoperative morbidity and a moderate to high level of discrimination for postoperative mortality across multiple surgical disciplines. This discriminatory ability may provide significant value to clinicians when determining appropriate postoperative management. While this retrospective data is reassuring, the SAS may not be applicable to all surgical procedures. Further studies are required to determine whether the use of the SAS can influence postoperative management, and if this influence can lead to a positive impact on patient outcomes.
All authors were involved in study design, data extraction, article review, and final article approval. E.S.P. performed the literature search.
1. Gawande AA, Kwaan MR, Regenbogen SE, et al. An Apgar score for surgery. J Am Coll Surg. 2007;204:201–208.
2. Regenbogen SE, Lancaster RT, Lipsitz SR, et al. Does the Surgical Apgar Score
measure intraoperative performance? Ann Surg. 2008;248:320–328.
3. Regenbogen SE, Ehrenfeld JM, Lipsitz SR, et al. Utility of the Surgical Apgar Score
: validation in 4119 patients. Arch Surg. 2009;144:30–36; discussion 37.
4. Regenbogen SE, Bordeianou L, Hutter MM, et al. The intraoperative Surgical Apgar Score
predicts postdischarge complications after colon and rectal resection. Surgery. 2010;148:559–566.
5. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008–2012.
6. Mallen C, Peat G, Croft P. Quality assessment of observational studies is not commonplace in systematic reviews. J Clin Epidemiol. 2006;59:765–769.
7. Moher D, Liberati A, Tetzlaff J, et al.; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.
8. Moher D, Shamseer L, Clarke M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
9. Canadian Agency for Drugs and Technologies in Health (CA). Grey Matters: A Practical Tool for Searching Health-Related Literature. 2015. Available at: https://www.cadth.ca/resources/finding-evidence/grey-matters
. Accessed July 18, 2016.
10. Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomized studies in meta-analyses. Ottawa ON, Canada, The Ottawa Hospital Research Institute. Available at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
. Accessed July 6, 2016.
11. Pencina MJ, D’Agostino RB Sr. Evaluating discrimination of risk prediction models: the C statistic. JAMA. 2015;314:1063–1064.
12. Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240:1285–1293.
13. Wuerz TH, Regenbogen SE, Ehrenfeld JM, et al. The Surgical Apgar Score
in hip and knee arthroplasty. Clin Orthop Relat Res. 2011;469:1119–1126.
14. Haynes AB, Regenbogen SE, Weiser TG, et al. Surgical outcome measurement for a global patient population: validation of the Surgical Apgar Score
in 8 countries. Surgery. 2011;149:519–524.
15. Ohlsson H, Winso O. Assessment of the Surgical Apgar Score
in a Swedish setting. Acta Anaesthesiol Scand. 2011;55:524–529.
16. Roghmann F, von Bodman C, Loppenberg B, et al. Is there a need for the Fournier’s gangrene severity index? Comparison of scoring systems for outcome prediction in patients with Fournier’s gangrene. BJU Int. 2012;110:1359–1365.
17. Thorn CC, Chan M, Sinha N, et al. Utility of the Surgical Apgar Score
in a district general hospital. World J Surg. 2012;36:1066–1073.
18. Urrutia J, Valdes M, Zamora T, et al. Can the Surgical Apgar Score
predict morbidity and mortality in general orthopaedic surgery? Int Orthop. 2012;36:2571–2576.
19. Dullo M, Ogendo SWO, Nyaim EO. Surgical Apgar Score
predicts post-laparotomy complications. Ann Afr Surg. 2013;10:24–29.
20. Sobol JB, Gershengorn HB, Wunsch H, et al. The Surgical Apgar Score
is strongly associated with intensive care unit admission after high-risk intraabdominal surgery. Anesth Analg. 2013;117:438–446.
21. Miki Y, Tokunaga M, Tanizawa Y, et al. Perioperative risk assessment for gastrectomy by Surgical Apgar Score
. Ann Surg Oncol. 2014;21:2601–2607.
22. Glass NE, Pinna A, Masi A, et al. The Surgical Apgar Score
predicts postoperative ICU admission. J Gastrointest Surg. 2015;19:445–450.
23. Jering MZ, Marolen KN, Shotwell MS, et al. Combining the ASA physical classification system and continuous intraoperative Surgical Apgar Score
measurement in predicting postoperative risk. J Med Syst. 2015;39:147.
24. Sakan S, Pavlovic DB, Milosevic M, et al. Implementing the Surgical Apgar Score
in patients with trauma hip fracture. Injury. 2015;46(suppl 6):S61–S66.
25. Urrutia J, Valdes M, Zamora T, et al. An assessment of the Surgical Apgar Score
in spine surgery. Spine J. 2015;15:105–109.
26. Eto K, Yoshida N, Iwatsuki M, et al. Surgical Apgar Score
predicted postoperative morbidity after esophagectomy for esophageal cancer. World J Surg. 2016;40:1145–1151.
27. Wied C, Foss NB, Kristensen MT, et al. Surgical Apgar Score
predicts early complication in transfemoral amputees: retrospective study of 170 major amputations. World J Orthop. 2016;7:832–838.
28. Ejaz A, Gani F, Frank SM, et al. Improvement of the Surgical Apgar Score
by addition of intraoperative blood transfusion among patients undergoing major gastrointestinal surgery. J Gastrointest Surg. 2016;20:1752–1759.
29. House LM, Marolen KN, St Jacques PJ, et al. Surgical Apgar Score
is associated with myocardial injury after noncardiac surgery. J Clin Anesth. 2016;34:395–402.
30. Cihoric M, Toft Tengberg L, Bay-Nielsen M, et al. Prediction of outcome after emergency high-risk intra-abdominal surgery using the Surgical Apgar Score
. Anesth Analg. 2016;123:1516–1521.
31. Stroyer S, Mantoni T, Svendsen LB. Evaluation of the Surgical Apgar Score
in patients undergoing Ivor-Lewis esophagectomy. J Surg Oncol. 2017;115:186–191.
32. Ngarambe C, Smart BJ, Nagarajan N, et al. Validation of the Surgical Apgar Score
after laparotomy at a tertiary referral hospital in Rwanda. World J Surg. 2017;41:1734–1742.
33. Ou CY, Hsu SY, Huang JH, et al. Surgical Apgar Score
in patients undergoing lumbar fusion for degenerative spine diseases. Clin Neurol Neurosurg. 2017;152:63–67.
34. Hsu SY, Ou CY, Ho YN, et al. Application of Surgical Apgar Score
in intracranial meningioma surgery. PLoS One. 2017;12:e0174328.
35. Kurata K, Chino Y, Shinagawa A, et al. Surgical Apgar Score
predicts 30-day morbidity in elderly patients who undergo non-laparoscopic gynecologic surgery: a retrospective analysis. Int J Surg. 2017;48:215–219.
36. Mastalerz K, Kenig J, Olszewska U, et al. The Surgical Apgar Score
and frailty as outcome predictors in short- and long-term evaluation of fit and frail older patients undergoing elective laparoscopic cholecystectomy - a prospective cohort study. Wideochir Inne Tech Maloinwazyjne. 2018;13:350–357.
37. Day KE, Prince AC, Lin CP, et al. Utility of the modified Surgical Apgar Score
in a head and neck cancer population. Otolaryngol Head Neck Surg. 2018;159:68–75.
38. Kotera A. The Surgical Apgar Score
can help predict postoperative complications in femoral neck fracture patients: a 6-year retrospective cohort study. JA Clin Rep. 2018;4:67.
39. Goel N, Manstein SM, Ward WH, et al. Does the Surgical Apgar Score
predict serious complications after elective major cancer surgery? J Surg Res. 2018;231:242–247.
40. Kenig J, Mastalerz K, Mitus J, et al. The Surgical Apgar Score
combined with comprehensive geriatric assessment improves short- but not long-term outcome prediction in older patients undergoing abdominal cancer surgery. J Geriatr Oncol. 2018;9:642–648.
41. Kenig J, Mastalerz K, Lukasiewicz K, et al. The Surgical Apgar Score
predicts outcomes of emergency abdominal surgeries both in fit and frail older patients. Arch Gerontol Geriatr. 2018;76:54–59.
42. Tomimaru Y, Takada K, Shirakawa T, et al. Surgical Apgar Score
for predicting complications after hepatectomy for hepatocellular carcinoma. J Surg Res. 2018;222:108–114.
43. Prince AC, Day KE, Lin CP, et al. Utility of the Surgical Apgar Score
in head and neck squamous cell carcinoma. Otolaryngol Head Neck Surg. 2018;159:466–472.
44. Ding G, Zhou L, Chen W, et al. Utility of the Surgical Apgar Score
in pancreatic cancer and modification. Laparosc Endosc Rob Surg. 2019;2:89–93.
45. Singh K, Hariharan S. Detecting major complications and death after emergency abdominal surgery using the Surgical Apgar Score
: a retrospective analysis in a caribbean setting. Turk J Anaesthesiol Reanim. 2019;47:128–133.
46. Khuri SF, Daley J, Henderson WG, et al. The National VA Surgical Risk Study. Risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg. 1995;180:519–531.
47. Clavien PA, Barkun J, de Oliveira ML, et al. The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg. 2009;250:187–196.
48. Chen JY, Zhang AD, Lu HY, et al. CHADS2 versus CHA2DS2-VASc score in assessing the stroke and thromboembolism risk stratification in patients with atrial fibrillation: a systematic review and meta-analysis. J Geriatr Cardiol. 2013;10:258–266.