Recent advances in biomarker technology have enabled earlier reassurance for patients who present to the Emergency Department (ED) with symptoms that are compatible with an acute coronary syndrome (ACS). There is now convincing evidence that serial cardiac troponin testing can help clinicians to rule out ACS over as little as 2 h with a contemporary assay [1,2] and 1 h with a high sensitivity assay . It may even now be possible to ‘rule out’ acute myocardial infarction (AMI) with a single blood test, obviating the need for serial sampling [4,5].
The Manchester Acute Coronary Syndromes (MACS) and Troponin-only MACS (T-MACS) decision aids are validated tools that can both ‘rule in’ ACS, ‘rule out’ ACS and risk stratify remaining patients after a single blood test in the ED [6,7]. Both MACS and T-MACS calculate the probability that a patient has ACS and use that probability to stratify patients into one of four risk groups. The key difference between these models is that MACS incorporates two biomarkers (cardiac troponin and heart-type fatty acid binding protein), whereas T-MACS incorporates only cardiac troponin. However, both rely on biomarker measurement.
In the prehospital environment and when patients arrive in the ED for triage, the ability to test biomarker concentrations is not usually available. A decision aid that can ‘rule in’, ‘rule out’ and accurately risk stratify patients with suspected ACS without requiring biomarker measurement would have substantial advantages in these settings. Our primary objective was to derive and externally validate a decision aid, called ‘history and electrocardiogram MACS’ (HE-MACS), to achieve this.
Design and setting
We undertook this work in three stages: a derivation study followed by two external validation studies. We elected not to pool data from the two validation studies because of the potential for important heterogeneity between the studies. The derivation study took place at Manchester Royal Infirmary (an inner city university-affiliated hospital with a regional cardiology service); validation study took place at Stepping Hill Hospital, Stockport (a suburban community hospital) and validation study was the Bedside Evaluation of Sensitive Troponin study. This is a multicentre study incorporating an extensive programme of research. For this analysis, we included patients from Manchester Royal Infirmary and St George’s NHS Trust, London, UK. Each of the three studies is a prospective diagnostic accuracy study. All analyses presented here are secondary analyses.
In each study, we included adult patients presenting to the ED with suspected cardiac chest pain. We excluded patients with another medical condition necessitating hospital admission and those whose symptoms had occurred over 24 h (derivation study and validation study 1) or 12 h (validation study 2) prior to arrival. In validation study 2 we excluded patients who were diagnosed with ST-elevation myocardial infarction in the ED. Full details of the methodology for the derivation study and validation study 1 have previously been reported [6,7]. All patients provided written informed consent and the studies were approved by the National Research Ethics Service.
Data collection and processing
Clinical data were recorded by the treating clinician in the ED using a bespoke case report form (Supplementary Appendix, Supplemental digital content 1, http://links.lww.com/EJEM/A229). ECGs were also interpreted by the treating clinician. All patients underwent serial cardiac troponin testing. In the derivation study and validation study 1, samples were drawn on arrival in the ED and 12 h after symptom onset. In validation study 2, samples were drawn on arrival and 3 h later. The troponin assays in use were cardiac troponin T (99th percentile 0.01 ng/ml, 10% coefficient of variation at 0.03 ng/ml, Roche Elecsys 4th generation; Roche Diagnostics, GmBH, Penzberg, Germany) in the derivation study and high sensitivity cardiac troponin T (99th percentile 14 ng/l, 10% coefficient of variation at 12 ng/l, Roche Elecsys 5th generation, Roche Diagnostics) in both validation studies. Patients were followed up after 30 days by telephone and chart review. If patients were persistently uncontactable we contacted their general practitioner.
The primary outcome for this analysis was a diagnosis of ACS. Patients were considered to have ACS if they had AMI or a major adverse cardiac event (MACE) within 30 days. The diagnosis of AMI was adjudicated by two independent investigators with disagreements being resolved by discussion. AMI was defined in accordance with the universal definition of myocardial infarction, requiring a rise and/or fall of cardiac troponin with at least one concentration above the 99th percentile of the assay in conjunction with symptoms or signs compatible with myocardial ischaemia, ECG changes, imaging evidence of new loss of viable myocardium or angiographic identification of intracoronary thrombus . MACE was defined as death (all cause), AMI or coronary revascularization (coronary artery bypass grafting or percutaneous coronary intervention) occurring within 30 days of presentation.
We derived HE-MACS by forward stepwise logistic regression. On the basis of previous work in the derivation study [6,9], variables that had been found to predict (P < 0.05) the diagnosis of ACS on univariate analysis and that had good interobserver reliability (κ > 0.6) were considered for inclusion in the model . We explored the relationship between age and the incidence of MACE by stratifying age into deciles and cross-tabulating this with the incidence of MACE. Having noted an approximately linear relationship, we considered age as a continuous variable in the model. Similarly, we explored the relationship between systolic blood pressure and the incidence of MACE. In this instance, we noted a markedly higher incidence of MACE in the bottom decile, which related to a cutoff set at ~100 mmHg. We, therefore, considered systolic blood pressure as a dichotomous variable with a cutoff set at 100 mmHg. Calibration plots were created by stratifying the calculated probability of ACS (using HE-MACS) into deciles. The incidence of MACE was then plotted against the mean probability in each decile. The intercept and slope were calculated by linear regression using GraphPad Prism, version 5.04 (GraphPad Inc., La Jolla, California, USA).
Having derived the HE-MACS model, we then stratified patients into four risk groups, based on the decisions that are likely to be available to clinicians who would apply HE-MACS in practice: ‘very low risk’ (possible immediate ‘rule out’), ‘low risk’ (potentially suitable for care in an ambulatory environment), ‘moderate risk’ (potentially suitable for care in the ED environment) and ‘high risk’ (potentially ‘rule in’ ACS). The thresholds for assigning patients to the different risk groups were set by receiver operating characteristic (ROC) analysis. First, we calibrated the model to achieve 100% sensitivity for ‘very low risk’ versus all other categories. While 100% is not the minimum acceptable sensitivity, we aimed for optimum sensitivity in the derivation phase in the knowledge that sensitivity is likely to be lower on validation. For ‘low risk’ versus all other groups, we calibrated the model to achieve 90% sensitivity (roughly equivalent to a normal initial high sensitivity cardiac troponin T concentration ). Finally, we aimed to achieve a specificity of 95% for the high-risk group.
To evaluate diagnostic performance in each individual study, we calculated test characteristics with 95% confidence intervals (CIs) and calculated the area under the ROC curve using the nonparametric method. We then calculated pooled sensitivity, specificity and likelihood ratios using the DerSimonian Laird (random effects) model. We used a χ2-test to test for statistical heterogeneity between studies. The proportions of patients identified as being at ‘very low risk’ in each validation study were aggregated to yield the pooled proportion.
Statistical analyses were undertaken using SPSS, version 23.0 (SPSS Inc., Chicago, Illinois, USA) except for (a) calculation of test characteristics, for which we used MedCalc, version 220.127.116.11 (MedCalc Software, Mariakerke, Belgium); and (b) and diagnostic meta-analysis, for which we used Meta-Disc . To calculate the lower bound of the 95% CI when the negative predictive value (NPV) was 100% we used the epi.stats package in R, version 3.5.0 (University of Auckland, Auckland, New Zealand). As this work is a secondary analysis, we did not undertake a sample size calculation specific to the derivation and validation of HE-MACS. However, the derivation cohort had been powered to derive a decision rule with 15 predictors, assuming a 20% prevalence of ACS with 5% loss to follow-up, requiring a total of 790 participants.
We included 796 patients in the derivation study. A total of 153 (19.2%) patients had prevalent AMI. After 30 days, a total of 118 (14.8%) patients had undergone coronary revascularization, seven (0.9%) had died (all of which had an adjudicated initial diagnosis of AMI) and two (0.3%) had incident AMI. Thus, a total of 179 (22.5%) patients had ACS (either prevalent AMI or at least one incident MACE within 30 days). In validation study 1, we included 474 patients of which 80 (16.9%) had AMI and 93 (19.6%) had ACS. In validation study 2, there were 659 participants including 74 (11.2%) with AMI and 91 (13.8%) with ACS. Follow-up was complete to 30 days in each study. A participant flow diagram is shown in Fig. 1. Baseline characteristics of participants in each study are shown in Table 1.
The HE-MACS model we derived incorporated eight variables, as shown in Table 2. This model had an area under the ROC curve of 0.82 (95% CI: 0.78–0.86). We used the ROC analysis to stratify patients into four risk groups, as detailed in the Supplementary Appendix (Supplemental digital content 1, http://links.lww.com/EJEM/A229). The model was well calibrated with a slope of 0.99 and intercept 0.0027. A calibration plot is available in the Supplementary Appendix (Supplemental digital content 1, http://links.lww.com/EJEM/A229). The proportion of patients assigned to each risk group and the number of AMI and ACS in each group are shown in Table 3.
The test characteristics of the model in the validation studies are shown in Table 4, together with pooled estimates of sensitivity and specificity. The areas under the ROC curves in the validation studies were 0.78 (95% CI: 0.73–0.83) and 0.73 (95% CI: 0.67–0.78) respectively. Calibration plots are shown in the Supplementary Appendix (Supplemental digital content 1, http://links.lww.com/EJEM/A229). There was no evidence of between-study heterogeneity for the sensitivity of HE-MACS (P = 0.24, I2: 27.6%), but there was evidence of heterogeneity for specificity (P < 0.001, I2: 92.2%). In the validation studies, the total proportion ‘ruled out’ would have been 5.5% in validation study 1 and 12.1% in validation study 2) with one missed ACS. The one patient missed had an adjudicated diagnosis of AMI. This 30-year-old male presented with a typical chest pain and had high-sensitive cardiac troponin T concentrations of 198 and 316 ng/l. Although that patient was assigned an adjudicated diagnosis of AMI, subsequent cardiac magnetic resonance imaging was consistent with a diagnosis of myocarditis, which was the final diagnosis assigned by the patient’s clinical team.
We have successfully derived and externally validated a decision aid that could ‘rule out’ ACS in 9.4% of patients using only the history and ECG with high NPV. The HE-MACS can also be used to effectively risk stratify the remaining patients. To our knowledge, there are currently no similar validated tools that can achieve this without biomarker testing, which requires the availability of laboratory equipment. This special and unique feature, therefore, opens up a number of important opportunities to apply HE-MACS more widely than the original MACS and T-MACS models.
First, HE-MACS could be used at triage when patients first arrive in the ED. Achieving rapid ‘rule-out’ for patients soon after arrival, without biomarker evaluation, could help to alleviate the growing pressures faced by crowded EDs, and accurately assign patients triage category (i.e. time to be seen by a clinician).
Second, the algorithm could be used in the prehospital environment. To date, the requirement to measure biomarker concentrations has been an important barrier to the implementation of rapid ‘rule-out’ strategies in this field. One potential solution would be to implement a point of care biomarker testing. However, the ability to ‘rule out’ ACS without any biomarker measurement would be cheaper, easier to implement and would facilitate more rapid decisions. Finally, the algorithm could be used in settings where blood testing is not available, such as in primary care. This is often the first medical contact for patients with chest pain and a high percentage of those patients are referred to the ED . Thus, HE-MACS could help to avoid unnecessary ED referrals and immediately rule out ACS in those settings.
To our knowledge, there are no comparable triage tools for use in patients with suspected ACS that do not involve the use of biomarkers and that have used adjudicated outcomes with contemporary reference standards. A systematic review from 2016 found no similar triage tools from the era of cardiac troponin testing that have been validated in the emergency setting . Another systematic review, from 2013, which focused on clinical prediction models for use in the prehospital setting, identified five relevant studies but none have been validated to ‘rule out’ ACS without biomarker testing and based on contemporary reference standards .
The volume of previous work in this field highlights the clinical demand for a validated tool that could be used to rapidly ‘rule out’ ACS without biomarker testing. It also demonstrates the importance of our current findings. We have derived and externally validated a decision aid that could be rapidly applied either in the prehospital environment or upon patient arrival in the ED. Its use could obviate the need for further investigation in 9.4% of patients. While this may appear to be a modest proportion of the overall total, it is worth noting that chest pain is one of the most common reasons for emergency ambulance calls and the most common reason for emergency hospital admission. The overall impact of reducing unnecessary transport to the hospital or facilitating rapid discharge from ED triage for these patients is therefore likely to be substantial.
It is now possible to measure cardiac biomarkers using portable devices with a turnaround time of 10–15 min. If such a technology is successfully validated in the prehospital environment, for example, alongside a validated risk score , and if the effectiveness of such technology is greater than the HE-MACS algorithm derived here, then the algorithm presented here may have limited applicability. However, the advantage of the algorithm presented here is that it does not require any biomarker testing with its associated costs, and could therefore be used even when such technology is not available.
It is important to acknowledge that the HE-MACS model has been derived and validated in secondary analyses of prospectively collected data. All of the clinical data, for example, ECG interpretation, were recorded based on the interpretation of emergency physicians. Also, having HE-MACS derived and validated from different populations result in different rule out proportions. In addition, having a wide confidence interval in the validation study 2 (91.8–99.8) for the NPV could indicate that one of 10 patients could be missed. Before clinical implementation, therefore, we must evaluate HE-MACS in the settings in which it is likely to have the most impact, including the prehospital field and ED triage or other. To do this, we must, therefore, determine the feasibility of applying the model when used by paramedics and ED triage nurses, and determine whether diagnostic accuracy is maintained.
We did not include pain severity as a potential predictor in this work, as these data were not collected during the derivation study. However, previous work from validation study 1 has shown that pain severity is poorly predictive of AMI .
We have derived and validated the HE-MACS clinical decision aid that could be used to rapidly risk stratify patients with suspected cardiac chest pain without requiring biomarker evaluation. Importantly, this model could allow ACS to be rapidly ‘ruled out’ in 9.4% of patients with high sensitivity and NPV. We must now validate the model when applied by paramedics in the prehospital setting and by nurses at triage in the ED.
The Validation of the Early Vascular Markers of Acute Coronary Syndromes study was funded by a research grant from the Royal College of Emergency Medicine (2010), and was sponsored by Stockport NHS Foundation Trust.
The Bedside Evaluation of Sensitive Troponin (BEST) study received funding from Abbott Point of Care, European Union Horizons-2020 (by FABPulous BV) and the Royal College of Emergency Medicine, and was sponsored by Manchester University NHS Foundation Trust.
Conflicts of interest
There are no conflicts of interest.
1. Than MP, Pickering JW, Aldous SJ, Cullen L, Frampton C.M., Peacock WF, et al. Effectiveness of EDACS versus ADAPT accelerated diagnostic pathways for chest pain: a pragmatic randomized controlled trial embedded within practice. Ann Emerg Med. 2016; 68:93–102
2. Than M, Cullen L, Aldous S, Parsonage WA, Reid CM, Greenslade J, et al. 2-Hour accelerated diagnostic protocol to assess patients with chest pain symptoms using contemporary troponins as the only biomarker: the ADAPT trial. J Am Coll Cardiol. 2012; 59:2091–2098
3. Mueller C, Giannitsis E, Christ M, Ordóñez-Llanos J, deFilippi C, McCord J, et al. Multicenter evaluation of a 0-hour/1-hour algorithm in the diagnosis of myocardial infarction with high-sensitivity cardiac troponin
T. Ann Emerg Med. 2016; 68:76–87
4. Carlton E, Greenslade J, Cullen L, Body R, Than M, Pickering J.W, et al. Evaluation of high-sensitivity cardiac troponin
I levels in patients with suspected acute coronary syndrome. JAMA Cardiol. 2016; 1:405–412
5. Pickering JW, Than MP, Cullen L, Aldous S., Ter Avest E., Body R., et al. Rapid rule-out of acute myocardial infarction
with a single high-sensitivity cardiac troponin
T measurement below the limit of detection: a collaborative meta-analysis. Ann Intern Med. 2017; 166:715–724
6. Body R, Carley S, McDowell G, Pemberton P, Burrows G, Cook G, et al. The Manchester Acute Coronary Syndromes
(MACS) decision rule for suspected cardiac chest pain: derivation and external validation. Heart. 2014; 100:1462–1468
7. Body R, Carlton E, Sperrin M, Lewis PS, Burrows G, Carley S, et al. Troponin-only Manchester Acute Coronary Syndromes
(T-MACS) decision aid: single biomarker re-derivation and external validation in three cohorts. Emerg Med J. 2017; 34:349–356
8. Thygesen K, Alpert JS, Jaffe AS, Simoons ML, Chaitman BR, White HD, et al. Third universal definition of myocardial infarction. Eur Heart J. 2012; 33:2551–2567
9. Body R, Carley S, Wibberley C, McDowell G, Ferguson J, Mackway-Jones K. The value of symptoms and signs in the emergent diagnosis of acute coronary syndromes
. Resuscitation. 2010; 81:281–286
10. Body R, Carley S, McDowell G, Jaffe AS, France M, Cruickshank K, et al. Rapid exclusion of acute myocardial infarction
in patients with undetectable troponin using a high-sensitivity
assay. J Am Coll Cardiol. 2011; 58:1332–1339
11. Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol. 2006; 6:31
12. Van Severen E, Willemsen R, Vandervoort P, Sabbe M, Dinant G-J, Buntinx F. How do patients with chest pain access Emergency Department care? Eur J Emerg Med. 2017; 24:423–427
13. Ayerbe L, González E, Gallo V, Coleman CL, Wragg A, Robson J. Clinical assessment of patients with chest pain; a systematic review of predictive tools. BMC Cardiovasc Disord. 2016; 16:18
14. Nehme Z, Boyle MJ, Brown T. Diagnostic accuracy of prehospital clinical prediction models to identify short-term outcomes in patients with acute coronary syndromes
: a systematic review. J Emerg Med. 2013; 44:946–954
15. Ishak M, Ali D, Fokkert MJ, Slingerland RJ, Dikkeschei B, Tolsma RT, et al. Fast assessment and management of chest pain without ST-elevation in the pre-hospital gateway: rationale and design. Eur Heart J Acute Cardiovasc Care. 2015; 4:129–136
16. Body R, Lewis PS, Carley S, Burrows G, Haves B, Cook G. Chest pain: if it hurts a lot, is heart attack more likely? Eur J Emerg Med. 2016; 23:89–94