The risk score test characteristics for 6-week MACE, 1-year MACE, and 90-day cardiac readmission are enumerated in Table 8. For outcome of 6-week MACE, HEART score vs. TIMI score had a positive predictive value (PPV) of 1.9% vs. 3.25%, negative predictive value (NPV) of 100% vs. 99%, Sn of 100% vs. 83%, and Sp of 62% vs. 82%, respectively. McNemar exact test for Sn comparison between the 2 diagnostic tests P = .45 and McNemar test for Sp comparison was P < .001.
For 1-year MACE, HEART score vs. TIMI score had a PPV 4.42% vs. 7.79%, NPV of 99.4% vs. 99.3%, Sn of 82 vs. 71, and Sp of 62% vs. 82%, respectively. McNemar exact test for Sn comparison between the 2 diagnostic tests P = .25 and McNemar test for Sp comparison was P < .001.
For 90-day cardiac readmission, HEART score vs. TIMI score had a PPV 20.19% vs. 31.17%, NPV of 92.8% vs. 92%, Sn of 64 vs. 48, and Sp 65% vs. 85%, respectively. McNemar test for Sn and Sp comparison between the 2 diagnostic tests yielded P < .001.
DCA as shown in Figure 2 helps to distinguish between the 3 strategies based on the net benefit of the model over a range of acceptable miss rate thresholds.
Figure 2A shows net benefit of using either the HEART score or the alternative strategies of admit all or admit none, irrespective of the score, to predict outcome of 6-week MACE. The X-axis shows range of threshold probabilities with an upper limit of 4% for predicting 6-week MACE.
At threshold probability of 1%, the net benefit of using the HEART score is higher than net benefit of either strategies of admit all or admit none. Decision curve demonstrates that at 2% threshold probability, where the curves cross, the net benefit of using the HEART score slowly starts to get lower than net benefit of admit none. The net benefit of admit all remains lower than either strategy at all threshold probabilities. Similarly, at threshold probability of 3% and 4%, the decision curve suggests, net benefit of admit none is a better strategy than using HEART score.
Figure 2B shows net benefit of using either the TIMI score or the alternative strategies of admit all or admit none, irrespective of the score, to predict outcome of 6-week MACE. The X-axis shows range of threshold probabilities with an upper limit of 4% for predicting 6-week MACE.
At threshold probabilities of 1% and 2%, the net benefit of using the TIMI score is higher than net benefit of either strategies of admit all or admit none. The net benefit of admit none slightly increases with increase in threshold probability at 2.5% (where the curves cross), 3%, and 4%. The decision curve also suggests, net benefit of admit all remains lower than net benefit of either strategies at all possible threshold probabilities.
Figure 2C shows net benefit of using either HEART score or the alternative strategies to admit all or admit none, irrespective of the score, to predict outcome of 1-year MACE. The X-axis shows range of threshold probabilities with an upper limit of 4% for predicting 1-year MACE.
At threshold probabilities of 1% and 2%, the net benefit of using the HEART score is higher than net benefit of admit all or admit none. Similarly, is the case for threshold probabilities of 3% and 4%. The alternative strategies here are either admit all or admit none. The alternative strategy decision curves cross at a threshold probability between 2% and 4%. Beyond threshold probability of 2%, the net benefit of admit none is higher than admit all. Overall the decision curve graph suggests that at all threshold probabilities, using HEART score has a higher net benefit than the other strategies.
Figure 2D shows net benefit of using either the TIMI score or the alternative strategies to admit all or admit none, irrespective of the score, to predict outcome of 1-year MACE. The X-axis shows range of threshold probabilities with an upper limit of 4% for predicting 1-year MACE.
At all threshold probabilities between 1% and 4%, the net benefit of using the TIMI score is higher than net benefits of admit all or admit none.
Figure 2E shows a decision curve with net benefit of using either the HEART score or the alternative strategies to admit all or admit none, irrespective of the score, to predict outcome of 90-day cardiac readmission. The X-axis shows range of threshold probabilities with an upper limit of 15% for predicting readmissions at 90 days.
Between threshold probabilities of 1% and 7%, the net benefit of admit all, is higher than net benefit of using the HEART score. Beyond threshold probability of 7% (where the curves cross), the net benefit of using the HEART score becomes higher than net benefit of admit all. The net benefit of admit none remains lower than net benefit of both the alternative strategies at all threshold probabilities between 1% and 12%.
Figure 2F shows net benefit of using either the TIMI score or the alternative strategies to admit all or admit none, irrespective of the risk score, to predict outcome of 90-day cardiac readmission. The X-axis shows range of threshold probabilities with an upper limit of 15% for predicting 90-day cardiac readmission.
At all threshold probabilities between 1% and 12%, the net benefit of admit all is higher than the net benefit of both the alternative strategies. The decision curves for admit all and using the TIMI risk score cross at about 12%. Between threshold probability of 12% and 15%, the net benefit of using TIMI score becomes higher than net benefit of admit all. The net benefit of admit none remains lower than the net benefit of both the alternative strategies at all threshold probabilities between 1% and 12%.
AA are twice as likely than whites to select the ED as their usual place of healthcare (OR 2.24, 95% CI 1.22–4.08) and with higher prevalence of hypertension among AA, hypertensive heart disease may present with ischemic symptoms despite not having significant CAD. Thomas et al reported that in patients with CAD, AA have lower long-term survival compared to whites (hazard ratio 2.54, 95% CI 1.60–4.04, P < .001). Cardiac risks scoring performance for CV event outcomes are understudied in AA population.
Our study aimed at studying AA population with non-high HEART scores and compares that to TIMI score. Our study suggests, HEART score has better overall discrimination than the TIMI score to predict 6-week MACE in non-high CV risk AA population, consistent with prior reports.[9,22,23] Our c-statistic of 0.86 for HEART indicates an excellent ability to discriminate patients presenting with chest pain for their risk of 6-week MACE.
We also demonstrate that both TIMI and HEART score could moderately discriminate patients for 1-year MACE and 90-day cardiac readmission outcomes. There was no difference in the discrimination ability of both scores since c-statistic for both the scores were in moderate range (c-statistic of 0.69 vs. 0.71) and cannot accurately distinguish patients at risk for 90-day cardiac readmission and 1-year MACE. A significant amount of patients with low HEART score had recurrent admission within 90-days (7.2%) and only 3 patients (0.6%) with low HEART score had MACE during 1 year follow-up.
Kline et al calculated that a 2% miss rate for 30-days follow-up period after initial evaluation should be acceptable based on the testing threshold at which the risk of harm from further testing equals or exceeds the chance of benefit from confirming ACS. However, the most frequently sited acceptable miss rate is less than 1%, which suggests that clinicians may expect diagnostic strategies for the assessment of suspected ACS to achieve a Sn of 99% or higher for AMI or other MACE and a NPV >99%. Based on this, we decided to take miss rate range with an upper limit of 4% as acceptable clinical thresholds for decision-making for 6-week and 1-year MACE outcomes. According to the Centers for Medicare & Medicaid Services, the national readmission rate (i.e., instances when patients return to the same or different hospital within 30 days of discharge) was 17.5% in 2013; hence we decided to take 15% as the upper limit as acceptable clinical threshold for 90-day cardiac readmission outcome.
Our population had low event counts for 6-week MACE (n = 6), so we compared the risk scoring characteristics for HEART and TIMI, specifically looking at NPV and Sn. The predictive capability of HEART score was higher as compared to TIMI score for primary outcome of 6-week MACE (NPV 100, Sn 100 vs. NPV 99, Sn 82). Similarly, for 1-year MACE (n = 17), predictive capability of HEART score was comparable to TIMI score (NPV 99, Sn 82 vs. NPV 99, Sn 71) and 90-day cardiac readmission (NPV 93, Sn 64 vs. NPV 92, Sn 48). Thus, HEART score has higher NPV and Sn when compared to TIMI score for 6-week MACE, 1-year MACE, and 90-day cardiac readmission but TIMI score has higher PPV and Sp values than HEART score for 6-week and 1-year MACE and 90-day cardiac readmission in low risk AA patients in our cohort and this is similar to previous reports[9,22](Table 4).
DCA using HEART score for chest pain patients to predict 6-week MACE (Fig. 2A) shows that using the HEART score tool for threshold probabilities of 1% to 2% miss rate is optimal strategy over other alternatives, while if threshold probability is between 2% and 4%, the optimal strategy should be discharging these patients. Similarly, DCA using TIMI score for 6-week MACE (Fig. 2B) shows that using TIMI score for threshold probability of 1% to 2.5% miss rate is optimal strategy over other alternative strategies, while is threshold probability of 2.5% to 4% miss rate, the optimal strategy should be to discharge these patients.
DCA using HEART and TIMI score for 1-year MACE (Fig. 2C and D) shows that using the risk stratification tool is best strategy with maximal net benefit when compared to alternative strategies at all clinical acceptable threshold probabilities.
DCA using HEART score for 90-day cardiac readmission (Fig. 2E) shows that using the risk stratification tool is the optimal strategy between threshold probabilities of 7% and 15%, but if the threshold probability is between 1% and 7%, admitting these patients is optimal strategy. DCA using TIMI score for 90-day cardiac readmission (Fig. 2F) shows that using risk stratification tool is optimal between threshold probabilities of 12% and 15%. Between threshold probabilities of 1% and 12%, the optimal strategy is admitting these patients.
There are multiple benefits of HEART over the TIMI risk score. Firstly, the TIMI risk score was established to risk stratify patients presenting with ACS but since its validation, clinicians have extended its use to risk stratify all cause chest pain patients presenting to the ED. In contrast, HEART was specifically established to stratify all patients presenting to emergency room with chest pain and have been validated prospectively. Secondly, the TIMI score does not include patient history component, which defines the characteristics of chest pain, even though clinicians rely heavily on this and guidelines advise to involve history as part of decision-making process.[26,27]
Our study compares 2 of the best-known risk scores in ED settings in non-high CV risk AA population, and our results suggests HEART score is superior to TIMI for 6-week MACE.
There is evolving demand for easy and rapid evaluation protocols, such as the use of coronary computerized tomography (CT) angiography and identification of patients who might safely defer stress testing. Cardiac risk scores are generally used to identify such patients who may be eligible for these protocols.[26,27] With further evidence from DCA, we can triage eligible patients who will benefit from requiring the scoring tool and further evaluation strategies (e.g., stress test vs. no stress test in low-risk patients).
The findings of our study have important clinical implications even though the number of events is low. The HEART score allows clinicians to immediately decide about the treatment plan in the ED. Almost two thirds of the patients in our cohort were “low” risk with HEART score 0 to 3, and none of them had MACE during 6-week follow-up. These findings will allow clinicians to avoid redundant diagnostic testing. It will allow clinicians to triage patients who will benefit from early discharge, as evidenced by DCA in our study, and others who will require additional testing with either stress testing or CT coronary angiography.
HEART weights were retrieved from retrospective chart review of medical records and the “history” component of the HEART criteria was scored as “moderately suspicious” for all patients that did not have any clear documentation (n = 9 ∼1%). A potential for differential misclassification bias that is affecting the HEART score may have been introduced, with bias toward the null, since none of the patients had a documented history component scored as “highly suspicious.”
Lastly, since this is a single center medical chart review of data on majority low risk AA subjects, generalizability of the results may be limited in other racial groups across other practices, but the superiority of c-statistic of HEART risk scores in addition to high Sn and NPV of HEART score for 6-week MACE outcome to TIMI score from our study is consistent with prior published reports.[10,28]
In non-high CV risk AA patients, HEART score is better predictive tool for 6-week MACE with Sn and NPV of 100%, c-statistic of 0.86, and OR of 3.11 when compared to TIMI score in patients presenting to ED with chest pain. DCA shows that net benefit of using HEART score is equally predictive of 6-week MACE when compared to TIMI and that the optimal strategy for a 2% to 4% miss rate threshold probability should be to discharge these patients from the ED.
Data curation: Reshma R. Golamari, Sandhya Vunnam, Smitha Moparthi, Neethi Venkatappa.
Formal analysis: Priyanka T. Bhattacharya, Wei Yang, Stephen E. Kimmel.
Investigation: Priyanka T. Bhattacharya.
Methodology: Priyanka T. Bhattacharya, Stephen E. Kimmel.
Project administration: Denis J. Dollard, Jose Missri.
Supervision: Stephen E. Kimmel.
Writing – original draft: Priyanka T. Bhattacharya, Reshma R. Golamari.
Writing – review & editing: Priyanka T. Bhattacharya, Reshma R. Golamari, Stephen E. Kimmel.
Priyanka T Bhattacharya orcid: 0000-0001-6922-7084.
. Owens PL, Barrett ML, Gibson TB, et al. Emergency department care in the United States: a profile of national data sources. Ann Emerg Med 2010;56:150–65.
. Pines JM, Isserman JA, Szyld D, et al. The effect of physician risk tolerance and the presence of an observation unit on decision making for ED patients with chest pain
. Am J Emerg Med 2010;28:771–9.
. Hollander JE, Robey JL, Chase MR, et al. Relationship between a clear-cut alternative noncardiac diagnosis and 30-day outcome in emergency department patients with chest pain
. Acad Emerg Med 2007;14:210–5.
. Kline JA, Charles LJ, Charles VP Jr, et al. Pretest probability assessment derived from attribute matching. BMC Med Inform Decis Making 2005;5:26.
. Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA 2000;284:835–42.
. Alley W, Mahler SA. Clinical decision aids for chest pain
in the emergency department: identifying low-risk patients. Open Access Emerg Med 2015;7:85–92.
. Six AJ, Backus BE, Kelder JC. Chest pain
in the emergency room: value of the HEART score. Neth Heart J 2008;16:191–6.
. Poldervaart JM, Reitsma JB, Backus BE, et al. Effect of using the HEART score in patients with chest pain
in the emergency department: a stepped-wedge, cluster randomized trial. Ann Intern Med 2017;166:689–97.
. Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain
in the emergency department: a multinational validation study. Crit Pathw Cardiol 2013;12:121–6.
. Sun BC, Laurie A, Fu R, et al. Comparison of the HEART and TIMI risk scores for suspected acute coronary syndrome in the emergency department. Crit Pathw Cardiol 2016;15:1–5.
. Napoli AM, Choo EK, Dai J, Desroches B. Racial disparities in stress test utilization in an emergency department chest pain
unit. Crit Pathw Cardiol 2013;12:9–13.
. Lopez L, Wilper AP, Cervantes MC, et al. Racial and sex differences in emergency department triage assessment and test ordering for chest pain
, 1997-2006. Acad Emerg Med 2010;17:801–8.
. Robert H. Aseltine, J., PhD, Jun Yan, PhD, Claudia b. Gruss, MD, Catherine Wagner, edd, And Matthew Katz, Ms, Connecticut Hospital Readmissions Related to Chest Pain
and Heart Failure: Differences by Race, Ethnicity, and Payer. connecticut medicine, Feb 2015. 79 (2): 69-76
. Kaul P, Lytle BL, Spertus JA, et al. Influence of racial disparities in procedure use on functional status outcomes among patients with coronary artery disease. Circulation 2005;111:1284–90.
. Mathews R, Chen AY, Thomas L, et al. Differences in short-term versus long-term outcomes of older black versus white patients with myocardial infarction: findings from the Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early Implementation of American College of Cardiology/American Heart Association Guidelines (CRUSADE). Circulation 2014;130:659–67.
. Steyerberg EW, Vickers AJ. Decision curve analysis: a discussion. Med Decis Making 2008;28:146–9.
. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565–74.
. Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ 2016;352:i6.
. Brown LE, Burton R, Hixon B, et al. Factors influencing emergency department preference for access to healthcare. West J Emerg Med 2012;13:410–5.
. Sorrentino MJ. Racial disparities in coronary artery disease prevalence. Ann Intern Med 1993;119:861.
. Thomas KL, Honeycutt E, Shaw LK, Peterson ED. Racial differences in long-term survival among patients with coronary artery disease. Am Heart J 2010;160:744–51.
. Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ACC guideline for the management of patients with non-ST-elevation acute coronary syndromes: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;130:e344–426.
. Swap CJ, Nagurney JT. Value and limitations of chest pain
history in the evaluation of patients with suspected acute coronary syndromes. JAMA 2005;294:2623–9.
. Than M, Herbert M, Flaws D, et al. What is an acceptable risk of major adverse cardiac event in chest pain
patients soon after discharge from the emergency department?: a clinical survey. Int J Cardiol 2013;166:752–4.
. Swenson C, Schraufnagel D, Sadikot R. Treating bronchiectasis. Am J Respir Crit Care Med 2017;196:P19–20.
. Litt HI, Gatsonis C, Snyder B, et al. CT angiography for safe discharge of patients with possible acute coronary syndromes. N Engl J Med 2012;366:1393–403.
. Mahler SA, Riley RF, Hiestand BC, et al. The HEART Pathway randomized trial: identifying emergency department patients with acute chest pain
for early discharge. Circ Cardiovasc Qual Outcomes 2015;8:195–203.
. Stopyra J.P., et al. Chest Pain
Risk Stratification: A Comparison of the 2-Hour Accelerated Diagnostic Protocol (ADAPT) and the HEART Pathway.
cardiac risk; chest pain; decision analysis; history, electrocardiogram, age, risk factors, and initial troponin score; thrombolysis in myocardial infarction score
Supplemental Digital Content
Copyright © 2019 The Authors. Published by Wolters Kluwer Health, Inc. All rights reserved.