Despite a decline in coronary heart disease deaths by more than 50% between 1961 and 2016, coronary heart disease is still a leading cause of mortality in the United Kingdom.1 In Scotland, acute coronary syndrome (ACS) is a major cause of mortality with 6697 deaths in 2016.2 ST elevation myocardial infarction (STEMI) is the most acutely critical subtype of ACS with highest 30-day mortality.3–5 Time is critical for STEMI management as mortality increases with treatment delays.4,6 Prehospital STEMI identification has been shown to reduce treatment delays and improve mortality.7,8
The importance of timely prehospital recognition of STEMI is well established,7 yet there are still recognized difficulties. Prehospital difficulties include the absence of complete medical records and lack of diagnostic support tools, such as imaging, which increases the risk of ACS misdiagnosis and creates a low positive predictive value for prehospital ACS diagnosis.9 A low positive predictive value increases inappropriate treatment of ACS including cardiac catheterization laboratory (“cath-lab”) activation.10 Over activation of the cath-lab is a potentially avoidable strain on a valuable clinical resource. False mobilization increases workload of the cath-lab team and often requires unnecessary redirection of emergency medical services to deliver patients to cath-lab centers outside their normal operating zones.
Conversely, under diagnosis of STEMI has obvious negative consequences. Delayed presentation of STEMI has significantly decreased long-term survival rates (73% survival with late presenters vs. 93% survival with early presenters)11 and even late treatment of STEMI via reperfusion of the culprit occluded artery has no benefit in mortality compared to conservative medical therapy.12 In addition, subtypes of ACS such as non-ST–elevation myocardial infarction (NSTEMI) and unstable angina can be just as critical as a STEMI as ST elevation on the electrocardiogram (ECG) is not exclusive for acute coronary artery occlusion,13 that is, a proportion of NSTEMI are actually caused by an occluded coronary artery.
Clinical decision support systems (CDSS) are platforms that combine multiple clinical data inputs (termed “components” in this review) to produce a single output, which can be a diagnosis, clinical advice or risk stratification, that can help clinicians with difficult decision-making.9 For instance, CDSS have already been developed for use in the emergency department for ACS,14,15 where these tend to focus on a high negative predictive value to prioritize safe discharge. In the community, there is increased difficulty for out-of-hospital practitioners, like general practitioners (especially those in remote and rural communities) and ambulance crews, to make triage decisions in patients with ACS without the clinical diagnostic tools that are available in the hospital. These difficulties are compounded with suspected ACS that presents without obvious ST elevation as a nondiagnostic ECG creates further ambiguity. This challenge has been the target of CDSS-related research to assist prehospital clinicians to manage patients who have suspected ACS.16
With great interrogation of technology in healthcare, there is a large potential for computer-aided diagnosis of ACS in the prehospital setting. Computer-aided decision support has already been shown to be beneficial determining allocation for level of life support in the emergency department.17 In addition, computer-aided ECG interpretation algorithms have been developed to improve prehospital and emergency department ACS identification to reduce the delay or misdiagnosis of ACS associated with prolonged door-to-balloon time.18
However, computer-aided ECG interpretation is still limited by ECG artifact and other nonischemic causes of ST elevation such as early-repolarization and thus interpretation of the ECG should be done in combination of other components such as symptoms and medical history.19,20
In the prehospital environment, there are concerns that CDSS can cause delays compared to standard care9 and that these systems might reduce the autonomy of clinicians.21 However, previous studies have shown the benefit of prehospital CDSS for patients with stroke22 and spinal injury.23 One review looked at prehospital CDSS for ACS but excluded tests using computer-aided decision systems and biomarker tests.24 With advances in computer technology and point-of-care testing, the use of these components is now increasingly realistic in a prehospital setting.
The aim of this systematic review was to describe the accuracy of CDSS and their individual components in the prehospital management of ACS.
The search strategy followed the guidelines set by the preferred reporting items for systematic review and meta-analysis.25 The review protocol was designed with guidance from the preferred reporting items for systematic review and meta-analysis protocol statement and was registered with Prospero (registration number: 116600).26
The search strategy was designed and executed by the first author. Five databases were searched: EMBASE, Medline, Cochrane Library, Web of Science, and CINAHL. The searches were performed between December 2018 and January 2019. Grey literature was also reviewed for any additional sources. The search terms used are in Appendix 1 (supplemental material, available at https://links.lww.com/HPC/A216).
Study Selection and Eligibility
Abstracts and titles were screened and selected if they were adjudged to be relevant to the review aim. Duplicates were excluded. The review focused on the use of CDSS in a prehospital setting where patients presented with symptoms suggestive of ACS. Definitions for ACS included STEMI, NSTEMI, or unstable angina as per European Society of Cardiology guidelines.27 Prehospital was defined as contact with first emergency responders (including paramedics, medical dispatch callers, general practitioners). Studies carried out in the hospital environment or emergency department were excluded. Patient history was defined as subjective symptoms reported by the patient (eg, chest pain, shortness of breath, and clamminess), while vital signs/examination were defined as objective noninvasive clinical measurements obtained by clinical staff (eg, heart rate, blood pressure, and oxygen saturations).
Inclusion criteria were as follows:
- Published source
- Data on patient diagnosis or outcome such as major adverse cardiovascular events
- Set in a prehospital setting
- Use of CDSS as an intervention
- Patients with suspected ACS
- English language
Exclusion criteria were as follows:
- No data on outcomes
- Inclusion of emergency department/inhospital decision aids
- Inclusion of nonsuspected ACS patients
- No definition of myocardial infarction (MI)
- Not in English language
Full-text versions of the papers selected were obtained and analyzed. Papers were then included or excluded based on the criteria. A second reviewer judged the selection process and analyzed the eligible papers separately by the criteria for consensus. Any disagreements were resolved by discussion between the 2 reviewers to reach a consensus. Cohen’s kappa coefficient was performed between the reviewers to analyze the rate of agreement.
Assessment of Quality and Risk of Bias
Quality assessment was conducted using a quality assessment tool for diagnostic accuracy studies (QUADAS).28 Articles were analyzed to ensure there was no obvious missing data and that patients progressed through the study as described. Studies were excluded from analysis where there was a high or unclear risk of bias. They were then ranked according to level of evidence as determined by published hierarchy of evidence, which takes into account any validation and impact analysis of CDSS.29
Data were extracted using a data extraction tool that was piloted with 2 initial studies and subsequently refined. The data types that were extracted are outlined in Appendix 2 (supplemental material, available at https://links.lww.com/HPC/A216). The primary outcome recorded from studies was a final diagnosis of ACS accuracy.
Data analysis was performed using statistical analysis software SPSS 24.0 (SPSS Inc., Chicago, IL). The sensitivity, specificity, and positive and negative predictive value of CDSS were examined. The results were reported as percentages and analyzed as continuous data. Whether a history, examination/vital signs, ECG, and biomarker components were included in the study, then this was described as binary (yes or no) and treated as categorical data. Independent-samples t test was used to analyze the difference of mean accuracy (percentage) between CDSS with and without components. A P value ≤0.05 was considered to be statistically significant. Because of the considerable heterogeneity between the papers selected, formal meta-analysis was deemed not possible.
Study Selection and Quality Assessment
Figure 1 outlines the search and selection process for this review. The titles and abstracts for 11,439 articles were screened. A total of 199 articles were initially identified through this process and reviewed. Of these, 182 articles did not fulfill eligibility criteria, leaving 17 articles.
The studies were assessed for their quality using the QUADAS 2 tool.28 Four studies were rejected from the study due to high risk of bias. A further 5 studies were assessed to have some minimal or moderate bias, all with patient selection, as would be expected with nonrandomized prospective and observational studies.20,30–33 Only 2 studies had validation phases for their CDSS and there was no impact analysis with any of the 17 articles thus undermining of the potential quality of evidence as judged by the predefined hierarchy.31,33 Ideally, validation and impact analysis would be required before any CDSS could be judged suitable for implementation in other health localities. The second reviewer screened the 17 selected studies and Cohen’s kappa coefficient for interobserver agreement between the 2 reviewers was calculated at k = 0.46, which equates to moderate agreement. Following collaboration with the second reviewer and QUANDAS 2 tool quality control, 9 studies from the 17 were excluded, leaving 8 studies that were included in the analysis.16,20,30–35
Seven of the 8 studies were prospective in nature, with use of CDSS performed “on-site” by a general practitioner, emergency medical services staff, or medical dispatcher.16,30–35 Two studies were retrospective analyses of patients and included either computer-aided ECG interpretation or decision-making.20,34Table 1 displays the demographic characteristics of the patients in the 8 studies. A total of 354,259 patients were in the studies combined; however, 1 study contributed 347,989 patients making up 98% of the total population. The average number of patients excluded was 69, the majority being from 1 study which only had data on 15% of patients.30 Mean age was 65 years and 54% of participants were male. Half of the studies were conducted in the Netherlands, with a further 2 in the United States and the remainder in Sweden and Japan. Five studies involved emergency medical services, 2 involved general practitioners, and 1 involved a medical call dispatch team.
There was a large degree of heterogeneity in the 8 studies. The first study was published in 1996, and the last 22 years later in 2018.16,33 As noted above, 7 of the studies were prospective and 1 was retrospective in design. The composition of the CDSS components differed, with 7 studies involving patient history16,30–35; 6 involving prehospital ECG interpretation16,20,31,33–35; 5 involving examination and vital signs16,31–33,35; and 2 involving a prehospital biomarker test.16,35 The last 2 studies were the only ones to develop CDSS that incorporated all 4 components.16,35 With regards to the outcomes measured, 3 reported ACS (MI including unstable angina)30,32,33; 3 others reported STEMI20,31,34; 1 reported NSTEMI16; and 1 reported a major adverse cardiovascular event35(defined as any MI, primary PCI, coronary artery bypass graft or any cause of mortality). The definition of MI also differed between studies, with 330,32,34 using the universal guidance on the diagnosis of MI,36 2 other studies16,35 used the third universal definition of MI,37 while the final 3 used a combination of ECG findings, biomarkers, and history to diagnose ACS.20,31,33 The incidence of the ACS also was widely different between the studies, ranging from 0.02% to 50%.
The results of the analysis of the outcomes, sensitivities, specificity, positive predictive value, and negative predictive value of the studies are described in Table 2. The sensitivity between the studies varied from 100% to 58%, with specificity varying from 100% to 10%, positive predictive value between 100% and 7%, and negative predictive value between 100% and 30%. Table 3 shows that only the inclusion of patient history was found to have a significant impact on improving accuracy of sensitivity and negative predictive value of CDSS.
The utility of CDSS for ACS in prehospital settings is yet to be established. This systematic review of the literature, the first to be conducted on the topic, found considerable variations in the components of CDSS that were examined in existing studies. The extent of the heterogeneity precluded a formal meta-analysis; however, a comparison of which components were key in successful CDSS was performed.
This review found that the use of the patient history component in CDSS remains highly important in diagnosis with significant improvement on the sensitivity (P = 0.002) and negative predictive value (P < 0.001). These findings highlight the potential of CDSS that incorporate patient history in a “rule-out” capacity for an ACS diagnosis. The significant impact of patient history in this review may have been due to patient history being the most prevalent tool in CDSS with it being included in 7 of the 8 studies reviewed.16,30–35 In comparison, prehospital ECG was used in 6 of the 8 studies,16,20,31,33–35 vital signs and examinations were used in 5 studies,16,31–33,35 and biomarkers were used in just 2 studies.16,35
Interestingly, in the 2 studies16,35 that used all 4 components (the ECG, a point-of-care biomarker, patient history, and vital signs/examinations), both achieved high sensitivity (100% and 96%) and negative predictive value (100% and 97%) but poor specificity (43% and 29%) and positive predictive value (29% and 21%). However, both studies excluded patients with clear ST elevation, thus focusing on the risk stratification of patients between ACS and non-ACS rather than triage of patients with NSTEMI or STEMI. Three of the studies20,31,34 looked exclusively at the identification of STEMI and had a larger range of specificity (88%–100%) and positive predictive value (7%–100%). The heterogeneity of the findings appears to be dependent on the aim of a prehospital CDSS to differentiate between a “rule-in/out” for ACS or between NSTEMI and STEMI.
Recent advances have allowed the use of high-sensitivity troponins to achieve a high degree of sensitivity in the diagnosis of ACS, leading to the reduction of unstable angina diagnosis.38 One of the studies reviewed demonstrated that there is the capability of the traditional point-of-care troponin to be used in CDSS.16 However, there were issues reported with the test used in this study, including device errors, inability to obtain blood, and the risk of false negatives when samples were taken shortly after symptom onset. The study was also limited by the use of a single troponin value in isolation, where clinicians are unable to observe any trends and a raised troponin does not always indicate myocardial ischemia but may be a result of myocardial injury.27 A computer-based machine learning algorithm for the diagnosis of MI has been developed with a paired troponin, analyzing the rate of change of troponin along with age and sex showing strong sensitivity at 97.8% and specificity of 92.2%.39 However, the study required a second troponin at 1–3 hours following the initial troponin measurement and therefore would not be feasible in the prehospital environment. The value of an isolated troponin in the prehospital situation maybe more apparent in combination with other components of CDSS such as patient history and suggestive ECG features. In addition, the use of prehospital high-sensitivity troponin tests in comparison to the inhospital test may aid in the sensitivity when identifying ACS where shorter time from symptom onset to test can reduce sensitivity.40
The use of contemporary risk stratification algorithms for MI has been shown to be effective following hospital admission, with examples like the History, ECG, Age, Risk factors, Troponin (HEART) score, Thrombolysis in Myocardial Infarction (TIMI) score and Global Registry of Acute Coronary Events (GRACE) score.41–43 Two studies16,35 used the HEART score as the clinical decision algorithm to aid in ACS risk stratification, with 1 study35 modifying the score with the use of a high-sensitivity troponin rather than the conventional fourth-generation troponin measurement. Although there was excellent sensitivity (100%) and negative predictive value (100%) for the modified HEART score algorithm, specificity (43%) and positive predictive value (29%) were less accurate. This could be due to the designation of intermediate and high values in the modified HEART scores as a “positive” score in this review. When adjusting for only the high scores on the modified HEART algorithm, then specificity increases to 87% and positive predictive value to 51%. As the authors acknowledge, the main objective of the HEART score is to rule-out rather than rule-in ACS; however, the risk stratification element could aid the rapid transfer of high-risk patients to specialist cardiac facilities.35
The greatest area for future development in CDSS is with computer-aided interpretation. Three of the CDSS in this review incorporated computer integration for either ECG interpretation or for the final clinical decision.20,30,34 The accuracy of MI diagnosis is seen in the 2 studies that utilized computer-aided interpretation of ECG, with high specificity (100% and 99%) and positive predictive value (100% and 83%).20,34,44 However, 1 study which looked only at the digital ECG for the decision support had lower sensitivity (58%) and negative predictive value (30%).20 The use of computer-aided decision-making is a rapidly developing field with advances in radiology and pathology especially.45 However, the role of computer-aided decision-making in ECG interpretation has been previously reported with varying sensitivity and specificity.46,47 Deep learning techniques for ECG interpretation have enormous potential to improve ECG ACS detection with the ability to detect subtle signs of ischemia and continually learn from their findings.48
Computer-aided decision-making was not only limited to ECG interpretation. One study looked at the use of a computer-aided decision system for medical dispatch to patients presenting with chest pain, with the only component being patient history, and it found good sensitivity (92%) and negative predictive value (97%) but poor specificity (41%) and positive predictive value (17%).30 The use of a computer-aided decision system can help assimilate a large amount of data when assessing a patient and help prioritize patients dependent on certain features in the history and risk factors. Innovations in computerized ACS diagnosis highlight the potential of machine learning where constant refinement of the algorithm accuracy can produce increasingly accurate decisions.39
This use of computer-aided decision systems in the prehospital setting can be advantageous, where often there is no experienced cardiologist present and paramedic crews, with limited training, may have to interpret the clinical situation and ECG alone.47 The one study which had the computer ECG interpretation with combination of a clinical screening tool led to high sensitivity (86.9%) and specificity (98.5%) suggesting that an integrated approach with other components could be beneficial.34
There are several important limitations with this study. Due to the high volume of ACS research, and in combination with the broad-search strategy, there is a possibility that some literature has been missed. This search strategy was employed to aid the identification of studies that examined principle components, such as patient history within CDSS, before the adoption of new technologies, such as prehospital ECG and biomarkers.
The considerable heterogeneity in CDSS which limited the statistical analyses that could be done, particularly with 1 study contributing 98% of the population for statistical analysis, hence the results must be taken with caution. In addition, MI was variously defined using the published universal definitions of MI,16,30,32,34,35 a combination of an ECG and biomarker criteria31,33 or by ECG alone.20 There was a notable variation in the incidence of ACS, ranging from 0.02%31 to 50%.20 This was due to patient selection for analysis, with the first study having included all patients presenting to emergency services (n = 347,989), whereas the second study focused exclusively on prehospital transmitted ECGs suspected of STEMI (n = 200) and therefore targeted a select patient group with a higher incidence of ACS.
Despite this, the review was able to document the nature and extent of the heterogeneity of the studies, including the components of CDSS and the methods used to examine them. It also provided the opportunity to examine what components were important in the prehospital diagnosis of ACS, and to compare the value of individual components and combinations thereof.
Further research would be useful to assess the accuracy of the high sensitivity of CDSS involving multiple components combined with the high specificity of computer-aided decision systems. CDSS research requires further validation in different clinical environments before CDSS are deployed for widespread use. In addition, impact analysis also helps judge whether the beneficial effects of CDSS would remain once fully incorporated into clinical use. Other effects that CDSS have on users need to be explored, including automation bias where the clinician can overtrust the decision aid.49 The user interface design of CDSS is another area that needs further research. Human factors and interaction design guidelines are often ignored in designing CDSS.50 However, 1 study used human–computer interaction design principles to design CDSS to aid ECG interpretation.51 They used eye-tracking analysis of ECG interpretation52 and their understanding of human cognition and working memory to breakdown the ECG interpretation process into manageable tasks on CDSS to eventually present multiple automated diagnoses to prevent automation bias and to encourage differential decision-making. While CDSS are mostly concerned with the provision of algorithmic text-based suggestions, future study may also involve better use of intelligent dynamic graphics as part of the algorithmic output for depicting more spatiotemporal data to augment the decision support.53 Finally, new studies that evaluate diagnostic CDSS would ideally focus on sensitivity, specificity, positive predictive value and negative predictive value of CDSS algorithms and use consistent definitions of MI. Studies that also use consistent definitions and outcomes between them would help with the development of a successful CDSS algorithm that integrate multiple components to provide an effective clinical aid.
CDSS are increasingly prevalent in healthcare and in combination with computer-aided decision and point-of-care biomarkers, they could provide a way of improving the accuracy of prehospital diagnosis and outcomes of treatment. With risks associated with delayed treatment of ACS and, alternatively, pressures on hospital resources such as cardiac cath-lab activation, there is an opportunity to create an efficient and safe diagnostic pathway before hospital admission. This review has highlighted the importance of patient history in diagnosis but also the potential for combining components such as biomarkers and computer-aided decision ECG interpretation in the integration of CDSS for suspected ACS.
The authors have no conflicts of interest to declare.
1. British Heart Foundation. Cardiovascular Disease UK Statistics Factsheet. 2018. https://www.bhf.org.uk/what-we-do/our-research/heart-statistics
. Accessed November 5, 2018.
2. Information Services Division. Scottish Heart Disease Statistics. Year Ending 31 March 2017. http://www.isdscotland.org/Health-Topics/Heart-Disease/Publications/2018-01-30/2018-01-30-Heart-Disease-Report.pdf
. Accessed October 26, 2018.
3. Scottish Intercollegiate Guidelines Network (SIGN). Acute Coronary Syndrome. 2016. Edinburgh: SIGN; https://www.sign.ac.uk/sign-148-acute-coronary-syndrome.html
. Accessed October 26, 2018.
4. National Institute for Health and Care Excellence. Myocardial Infarction with ST-Segment Elevation: Acute Management: Guidance and Guidelines
. 2013. https://www.nice.org.uk/guidance/cg167/chapter/1-Recommendations
. Accessed October 10, 2018.
5. García-García C, Subirana I, Sala J, et al. Long-term prognosis of first myocardial infarction according to the electrocardiographic pattern (ST elevation myocardial infarction, non-ST elevation myocardial infarction and non-classified myocardial infarction) and revascularization procedures. Am J Cardiol. 2011;108:1061–1067.
6. De Luca G, Suryapranata H, Ottervanger JP, et al. Time delay to treatment and mortality in primary angioplasty for acute myocardial infarction: every minute of delay counts. Circulation. 2004;109:1223–1225.
7. Ting HH, Krumholz HM, Bradley EH, et al.; American Heart Association Interdisciplinary Council on Quality of Care and Outcomes Research, Emergency Cardiovascular Care Committee; American Heart Association Council on Cardiovascular Nursing; American Heart Association Council on Clinical Cardiology. Implementation and integration of prehospital ECGs into systems of care for acute coronary syndrome: a scientific statement from the American Heart Association Interdisciplinary Council on Quality of Care and Outcomes Research, Emergency Cardiovascular Care Committee, Council on Cardiovascular Nursing, and Council on Clinical Cardiology. Circulation. 2008;118:1066–1079.
8. Ioannidis JP, Salem D, Chew PW, et al. Accuracy and clinical effect of out-of-hospital electrocardiography in the diagnosis of acute cardiac ischemia: a meta-analysis. Ann Emerg Med. 2001;37:461–470.
9. Hagiwara MA, Sjöqvist BA, Lundberg L, et al. Decision support system in prehospital care: a randomized controlled simulation study. Am J Emerg Med. 2013;31:145–153.
10. Park JH, Moon SW, Kim TY, et al. Sensitivity, specificity, and predictive value of cardiac symptoms assessed by emergency medical services providers in the diagnosis of acute myocardial infarction: a multi-center observational study. Clin Exp Emerg Med. 2018;5:264–271.
11. McNair PW, Bilchick KC, Keeley EC. Very late presentation in ST elevation myocardial infarction: predictors and long-term mortality. Int J Cardiol Heart Vasc. 2019;22:156–159.
12. Hochman JS, Lamas GA, Buller CE, et al.; Occluded Artery Trial Investigators. Coronary intervention for persistent occlusion after myocardial infarction. N Engl J Med. 2006;355:2395–2407.
13. Man S, Rahmattulla C, Maan AC, et al. Acute coronary syndrome with a totally occluded culprit artery: relation of the ST injury vector with ST-elevation and non-ST elevation ECGs. J Electrocardiol. 2014;47:183–190.
14. Body R, Almashali M, Morris N, et al. Diagnostic accuracy of the T-MACS decision aid with a contemporary point-of-care troponin assay. Heart. 2019;105:768–774.
15. Björk J, Forberg JL, Ohlsson M, et al. A simple statistical model for prediction of acute coronary syndrome in chest pain patients in the emergency department. BMC Med Inform Decis Mak. 2006;6:28.
16. van Dongen DN, Tolsma RT, Fokkert MJ, et al. Pre-hospital risk assessment in suspected non-ST-elevation acute coronary syndrome: a prospective observational study. Eur Heart J Acute Cardiovasc Care. 2020;9(1_suppl):5–12.
17. Gellerstedt M, Bång A, Herlitz J. Could a computer-based system including a prevalence function support emergency medical systems and improve the allocation of life support level? Eur J Emerg Med. 2006;13:290–294.
18. Schläpfer J, Wellens HJ. Computer-interpreted electrocardiograms: benefits and limitations. J Am Coll Cardiol. 2017;70:1183–1192.
19. Bosson N, Sanko S, Stickney RE, et al. Causes of prehospital misinterpretations of ST elevation myocardial infarction. Prehosp Emerg Care. 2017;21:283–290.
20. Bhalla MC, Mencl F, Gist MA, et al. Prehospital electrocardiographic computer identification of ST-segment elevation myocardial infarction. Prehosp Emerg Care. 2013;17:211–216.
21. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144:201–209.
22. Bray JE, Martin J, Cooper G, et al. An interventional study to improve paramedic diagnosis of stroke. Prehosp Emerg Care. 2005;9:297–302.
23. Burton JH, Harmon NR, Dunn MG, et al. EMS provider findings and interventions with a statewide EMS spine-assessment protocol. Prehosp Emerg Care. 2005;9:303–309.
24. Nehme Z, Boyle MJ, Brown T. Diagnostic accuracy of prehospital clinical prediction models to identify short-term outcomes in patients with acute coronary syndromes: a systematic review. J Emerg Med. 2013;44:946–954.e6.
25. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700.
26. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
27. Thygesen K, Alpert JS, Jaffe AS, et al. Fourth universal definition of myocardial infarction (2018). Eur Heart J. 2019;40:237–269.
28. Whiting PF, Rutjes AW, Westwood ME, et al.; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529–536.
29. McGinn T, Wyer P, McCullagh L, et alGuyatt G, Rennie D, O'Meade, et al. Diagnosis; clinical prediction rules. In: Users’ Guides to the Medical Literature. A Manual for Evidence-Based Clinical Practice. 2015:3rd ed. New York: McGraw-Hill; 407–418.
30. Gellerstedt M, Rawshani N, Herlitz J, et al. Could prioritisation by emergency medicine dispatchers be improved by using computer-based decision support? A cohort of patients with chest pain. Int J Cardiol. 2016;220:734–738.
31. Sakai T, Nishiyama O, Onodera M, et al.; CASSIOPEIA study group. Predictive ability and efficacy for shortening door-to-balloon time of a new prehospital electrocardiogram-transmission flow chart in patients with ST-elevation myocardial infarction - results of the CASSIOPEIA study. J Cardiol. 2018;72:335–342.
32. Bruins Slot MHE, Rutten FH, van der Heijden GJMG, et al. Diagnosing acute coronary syndrome in primary care: comparison of the physicians’ risk estimation and a clinical decision rule. Fam Pract. 2011;28:323–328.
33. Grijseels EW, Deckers JW, Hoes AW, et al. Implementation of a pre-hospital decision rule in general practice. Triage of patients with suspected myocardial infarction. Eur Heart J. 1996;17:89–95.
34. Wilson RE, Kado HS, Percy RF, et al. An algorithm for identification of ST-elevation myocardial infarction patients by emergency medicine services. Am J Emerg Med. 2013;31:1098–1102.
35. Ishak M, Ali D, Fokkert MJ, et al. Fast assessment and management of chest pain patients without ST-elevation in the pre-hospital gateway (FamouS Triage): ruling out a myocardial infarction at home with the modified HEART score. Eur Heart J Acute Cardiovasc Care. 2018;7:102–110.
36. Thygesen K, Alpert JS, White HD, et al.; Joint ESC/ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction. Universal definition of myocardial infarction. Circulation. 2007;116:2634–2653.
37. Thygesen K, Alpert JS, Jaffe AS, et al.; Joint ESC/ACCF/AHA/WHF Task Force for the Universal Definition of Myocardial Infarction. Third universal definition of myocardial infarction. Circulation. 2012;126:2020–2035.
38. Shah AS, Anand A, Sandoval Y, et al.; High-STEACS Investigators. High-sensitivity cardiac troponin I at presentation in patients with suspected acute coronary syndrome: a cohort study. Lancet. 2015;386:2481–2488.
39. Than MP, Pickering JW, Sandoval Y, et al. Machine learning to predict the likelihood of acute myocardial infarction. Circulation. 2019;140:899–909.
40. Chapman AR, Stewart S, Mills NL. Contemporary point of care cardiac troponin testing in suspected acute coronary syndrome. Heart. 2019;105:740–741.
41. Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol. 2013;12:121–126.
42. Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA. 2000;284:835–842.
43. Fox KA, Dabbous OH, Goldberg RJ, et al. Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE). BMJ. 2006;333:1091.
44. Garvey JL, Zegre-Hemsey J, Gregg R, et al. Electrocardiographic diagnosis of ST segment elevation myocardial infarction: an evaluation of three automated interpretation algorithms. J Electrocardiol. 2016;49:728–732.
45. Petrick N, Sahiner B, Armato SG 3rd, et al. Evaluation of computer-aided detection and diagnosis systems. Med Phys. 2013;40:087001.
46. Clark EN, Sejersten M, Clemmensen P, et al. Automated electrocardiogram interpretation programs versus cardiologists’ triage decision making based on teletransmitted data in patients with suspected acute coronary syndrome. Am J Cardiol. 2010;106:1696–1702.
47. Le May MR, Dionne R, Maloney J, et al. Diagnostic performance and potential clinical impact of advanced care paramedic interpretation of ST-segment elevation myocardial infarction in the field. CJEM. 2006;8:401–407.
48. Xiao R, Xu Y, Pelter MM, et al. A deep learning approach to examine ischemic ST changes in ambulatory ECG recordings. AMIA Jt Summits Transl Sci Proc. 2018;2017:256–262.
49. Bond RR, Novotny T, Andrsova I, et al. Automation bias in medicine: the influence of automated diagnoses on interpreter accuracy and uncertainty when reading electrocardiograms. J Electrocardiol. 2018;51(6s):S6–S11.
50. Liberati EG, Ruggiero F, Galuppo L, et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci. 2017;12:113.
51. Cairns AW, Bond RR, Finlay DD, et al. A computer-human interaction model to improve the diagnostic accuracy and clinical decision-making during 12-lead electrocardiogram interpretation. J Biomed Inform. 2016;64:93–107.
52. Bond RR, Zhu T, Finlay DD, et al. Assessing computerized eye tracking technology for gaining insight into expert interpretation of the 12-lead electrocardiogram: an objective quantitative approach. J Electrocardiol. 2014;47:895–906.
53. Bond RR, Finlay DD, Nugent CD, et al. Methods for presenting and visualising electrocardiographic data: from temporal signals to spatial imaging. J Electrocardiol. 2013;46:182–196.