Secondary Logo

Share this article on:

Impact of sex-specific differences in calculating the pretest probability of obstructive coronary artery disease in symptomatic patients

a coronary computed tomographic angiography study

Zhang, Yinga; Liu, Yujiea; Zhang, Hongb; Zhou, Jiaa

doi: 10.1097/MCA.0000000000000696
CT Angio/Risk Assessment

Objectives Little is known about the impact of sex-specific differences in calculating the pretest probability (PTP) of obstructive coronary artery disease. We sought to determine whether the calculation of PTP differ by sex in symptomatic patients referred to coronary computed tomographic angiography (CCTA).

Patients and methods The characteristics of 5777 men and women who underwent CCTA were compared. For each patient, PTP was calculated according to the updated Diamond–Forrester method (UDFM) and the Duke clinical score (DCS), respectively. Follow-up clinical data were also recorded. Area under the receiver operating characteristic curve, integrated discrimination improvement, net reclassification improvement, and the Hosmer–Lemeshow goodness-of-fit statistic were used to assess the models’ performance.

Results The area under the receiver operating characteristic curve of UDFM and DCS showed little difference in men (0.782 vs. 0.785, P=0.4708) and women (0.668 vs. 0.654, P=0.1255), and calibration of neither model was satisfactory. Compared with UDFM, DCS showed positive integrated discrimination improvement (10% in men, P<0.0001, and 8% in women, P<0.0001, respectively), net reclassification improvement (12.17% in men, P<0.0001, and 27.19% in women, P<0.0001, respectively), and obviously reduced unnecessary noninvasive testing for women with negative CCTA.

Conclusion Although the performance of neither model was favorable, DCS offered a more accurate calculation of PTP than UDFM and application of DCS instead of UDFM would result in a significant decrease in inappropriate testing, especially in women.

Departments of aCardiology

bRadiology, Tianjin Chest Hospital, Tianjin, China

Correspondence to Jia Zhou, MD, Department of Cardiology, Tianjin Chest Hospital, 261 Taierzhuangnan Road, Tianjin 300000, China Tel: +86 155 2248 5560; fax: +86 881 85164; e-mail:

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Received August 1, 2018

Received in revised form November 17, 2018

Accepted December 8, 2018

Back to Top | Article Outline


Although women tend to have a lower prevalence of obstructive coronary artery disease (CAD), previous evidence indicated that women experience relatively worse outcomes compared with men among patients presenting with stable chest pain 1–3. Of multiple potential explanations for this difference, variation by sex in the decision-making of diagnostic strategy may emerge as a particularly strong candidate 4,5. Recently, data from the PROMISE trial have suggested that influences of sex on the entire diagnostic pathway highlighted the need for sex-specific approaches to the evaluation and diagnosis of obstructive CAD 6.

Current guidelines for the evaluation of patients presenting with stable chest pain dictate further diagnostic strategy on the basis of the pretest probability (PTP) of obstructive CAD 7–10. Although the Updated Diamond–Forrester method (UDFM) 11 and the Duke clinical score (DCS) 12 are the most validated and recommended models, quite a few observations have suggested that both models overestimated the actual prevalence of obstructive CAD, especially in populations with low prevalence 13–17. However, few studies have systematically examined the impact of sex-related differences on the assessment of PTP by UDFM or DCS.

Thus, we recruited a coronary computed tomography angiography (CCTA)-based cohort to validate and compare the relative accuracy for estimating PTP of obstructive CAD by UDFM and DCS in symptomatic men and women, respectively.

Back to Top | Article Outline

Patients and methods

Study population

This study included consecutive patients who presented with stable chest pain and were referred for CCTA in our institution from October 2015 to September 2017. Patients were excluded if they fulfilled one of the following criteria: acute coronary syndrome, previous CAD or coronary revascularization (percutaneous coronary intervention or coronary artery bypass grafting), impaired renal function (serum creatinine>120 μmol/l), New York Heart Association class III or IV heart failure, atrial fibrillation, aortic disease, age more than 90 years, or patients with unassessable segments because of artifact. This retrospective and observational study was approved by the ethics committee of our institution.

Back to Top | Article Outline

Data collection and definitions

The following variables were registered: sex, age, BMI, hypertension, hyperlipidemia, diabetes, smoking, changes in resting ECG, and family history of premature CAD. Chest pain symptoms were categorized as nonanginal chest pain, atypical angina, or typical angina as described previously by Diamond 18. Hypertension was defined as blood pressure of at least 140/90 mmHg or the use of anti-hypertension medication. Hyperlipidemia was defined as total cholesterol of at least 220 mg/dl, low-density lipoprotein cholesterol of at least 140 mg/dl, fasting triglycerides of at least 150 mm/dl, or receiving treatment with oral lipid-lowering agents. Diabetes was defined as fasting glucose levels over 7 mmol/l or treatment currently with either diet, oral glucose-lowering agents, or insulin. Smoking was defined as current smoking or smoking in the past 6 months. A family history of premature coronary heart disease was defined as myocardial infarction (MI) occurring before the age of 50 years in a first-degree male relative or before the age of 55 years in a first-degree female relative. Changes in the resting ECG were defined as at least 1 mm depression in at least two adjacent leads. The PTP of obstructive CAD was calculated using UDFM 11 and DCS 12,19, respectively.

Back to Top | Article Outline

Coronary computed tomographic angiography

All scans were performed using a second-generation dual-source CCTA scanner (Somatom Definition Flash; Siemens Medical Solutions, Forchheim, Germany). Sublingual nitroglycerine and heart-rate control for a target heart rate of at least 70 beats/min were administered as appropriate. Afterwards, a contrast-enhanced CCTA was performed with detector collimation of 2×128×0.6 mm, a slice thickness of 0.6 mm, gantry rotation time of 280 ms, heart rate adaptive pitch of 0.2–0.5, tube current of 290–560 mAs/rotation, and tube voltage of 80–120 kV. Contrast volume was 60–90 ml (Ultravist, 370 mgI/ml; Schering AG, Guangzhou, China), followed by a normal saline of 50 ml, and was injected intravenously into an antecubital vein at 5 ml/s. Bolus tracking is used to synchronize the arrival of contrast in the coronary arteries and the initiation of the scan, and the region of interest was set at the root of the ascending aorta. Data acquisition was initiated with a delay of 5 s after the signal attenuation threshold (100 HU) was reached in region of interest. Image scan was triggered from 30 to 80% of the R–R interval.

Three experienced observers, who were blinded to the clinical data, evaluated the CCTA data on a Syngo Multimodality workstation (Siemens, Munich, Germany). Interobserver disagreements were resolved by consensus. In image analyses, all segments at least 2 mm in diameter were identified and analyzed using the CAD-RADS Coronary Artery Disease – Reporting and Data System 20. Obstructive CAD was defined as present if a patient had at least one lesion with at least 50% diameter stenosis.

Back to Top | Article Outline


Follow-up information was obtained by a phone call and/or a physician visit after CCTA. The major adverse cardiovascular event (MACE) was composed of cardiac death, nonfatal MI, unstable angina hospitalization, and late revascularization. Cardiac death was defined as any death caused by cardiac disease or for which no other cause could be found. MI was defined when at least two of the following three criteria were fulfilled: chest pain or equivalent symptom complex, positive cardiac biomarkers, or typical ECG changes 21. Late revascularizations (>60 days after CCTA) are more likely to be associated with CAD progression.

Back to Top | Article Outline

Statistical analysis

Continuous variables were compared using Student’s t-tests or Mann–Whitney U-tests as appropriate and were expressed as mean±SD. Count variables were expressed as frequencies with percentages and differences in the percentages were assessed using the χ 2-test or Fisher’s exact test as appropriate. To compare the predictive value of UDFM and DCS in men and women, the present study used distinct approaches. First, the improvement in discrimination was quantified using the area under receiver-operator characteristic curve (AUC) 22 and the integrated discrimination improvement (IDI) 23. Second, we evaluated whether both models correctly classified patients into different categories of PTP. Using the PTP categories less than 15, 15–85, and more than 85% 8,9, the net reclassification improvement (NRI) 23 was assessed. Third, Hosmer–Lemeshow (H-L) tests divided patients into ten groups according to deciles of PTP, each of which was represented by a dot on the calibration plot and an overall χ 2 statistic was calculated separately for men and women 24. All statistical analyses were carried out by MedCalc (version 15.2.2; MedCalc Software, Mariakerke, Belgium) and SAS (version 9.2; SAS Institute Inc., Cary, North Carolina, USA). Two-tailed P value less than 0.05 was considered statistically significant.

Back to Top | Article Outline


Sex-specific means or corresponding percentages of baseline variables are described in Table 1. The study cohort included 5777 patients, of whom 1882 (33%) were found to have obstructive CAD on CCTA. The mean ages were 55 years for men and 59 years for women. Women were more likely than men to have low BMI, hypertension, hyperlipidemia, and a family history of premature CAD. The prevalence of diabetes and changes in resting ECG were similar in men and women. Compared with women, men were more likely to smoke and have obstructive CAD on CCTA. The most common symptom in both sexes was atypical angina, which was reported by 46% of women versus 41% of men.

Table 1

Table 1

Comparison of discrimination using AUC and IDI is shown in Table 2. An evaluation of AUC showed little difference between the performance of UDFM and DCS in men (0.782 vs. 0.785, P=0.4708) and women (0.668 vs. 0.654, P=0.1255), respectively. By contrast, the IDI for UDFM was negative compared with DCS in men (−12%, P<0.0001) and women (−8%, P<0.0001), respectively.

Table 2

Table 2

To assess calibration, predicted probabilities of obstructive CAD were compared with observed probabilities detected on CCTA in deciles of predicted probabilities (Fig. 1). The differences between observed and predicted probabilities were evident in H-L calibration plots: UDFM manifested a predominance of overestimation in men, but underestimation in women, and DCS overestimated the prevalence of obstructive CAD in both men and women. Statistically, no calibration was acceptable (UDFM in men: H-L χ 2=153.97, P<0.01; DCS in men: H-L χ 2=110.44, P<0.01; UDFM in women: H-L χ 2=174.98, P<0.01; DCS in women: H-L χ 2=140.37, P<0.01).

Fig. 1

Fig. 1

Table 3 shows the classification of women on the basis of UDFM and DCS into categories of PTP (<15, 15–85, and >85%). Of the 2078 negative women, by DCS, 471 were reclassified correctly to a lower PTP category, but 40 were classified to a higher PTP category. Of the 698 positive men, 126 were reclassified correctly to a higher PTP category, but 81 were classified to a lower PTP category. Thus, compared with UDFM, the NRI for DCS was 20.75% for negative, 6.45% for positive, and 27.20% overall (P<0.0001). The results were different when the analysis was carried out on men (Table 4). The NRI was as follows: 1.76% for negative, 15.04% for positive, and 16.80% overall (P<0.0001).

Table 3

Table 3

Table 4

Table 4

During a median follow-up of 13 months (interquartile range: 6–20 months), 203 (3.5%) patients were lost to follow-up. MACEs occurred in 249 (4.3%) patients, including 11 (0.2%) cardiovascular deaths, 28 (0.5%) nonfatal MIs, 93 (1.6%) unstable angina hospitalization, and 117 (2.0%) late revascularizations.

UDFM classified more than half (1188/2078) the negative women into the medium PTP group, for which noninvasive testing was recommend according to the current guidelines. However, among the 1188 negative women, only 31 MACEs occurred (2.6%, one cardiovascular death, three nonfatal MIs, eight unstable anginas, and 19 late revascularizations). Use of DCS instead of UDFM would imply a change of diagnostic strategy in these negative women: 40% (471/1188) into the low PTP group, for which no further test was recommend. Moreover, among the 471 negative women, only seven MACEs occurred (1.5%, no cardiovascular death, one nonfatal MI, one unstable angina, and five late revascularizations).

Back to Top | Article Outline


In this CCTA-based analysis of patients with symptoms suggestive of stable CAD, we observed substantial sex-based differences in the clinical presentation and diagnostic evaluation. Although DCS seemed to perform better than UDFM with a positive NRI and IDI, neither of them showed a favorable AUC and calibration in women. Moreover, use of DCS instead of UDFM would alter the diagnostic strategy in women, resulting in a decrease in unnecessary testing. To our knowledge, this is the first comparative description of the sex-based differences in the calculation of PTP by the most proposed models, UDFM and DCS.

Consistent with our study, data from the PROMISE trial found that although women were more likely to be characterized as having a low PTP, they had larger traditional risk factor burden than men, except for BMI, diabetes, and smoking 6. Furthermore, previous investigations have suggested that the most common symptom type was atypical angina and women are more likely to present with atypical angina than men 6,25, which were similar to that noted in the present analysis. Thus, selection of optimal PTP models to ensure that they sufficiently account for the discrepancy between the higher prevalence of traditional risk factors and the lower reported prevalence of obstructive CAD is crucial in the development of a diagnostic strategy for women presenting with stable chest pain 5,6.

A number of previous studies have indicated that either UDFM or DCS overestimated the actual prevalence of obstructive CAD, especially in populations with low prevalence 13–17, which was confirmed by the unsatisfactory calibration in the current analysis, especially for women. By UDFM, more than half women were classified into the low PTP group, resulting in a marked underestimation manifested in the H-L calibration plots. Moreover, in women, the AUCs for both models were moderate (0.668 for UDFM and 0.654 for DCS), which was in line with a recently published study 26, yielding an AUC of 0.61 for UDFM and 0.59 for DCS in women referred to CCTA.

We noted two potential reasons for the unsatisfactory performance when applying the traditional age, sex, and chest pain typicality-based approach for women. First, the PTP models might show divergent predictions across patients differing only by sex 4. For example, according to UDFM, a 45-year-old female patient with atypical angina had a 14% PTP of obstructive CAD, whereas a male patient with same characteristics had a 38% PTP. It is worth noting that an 85-year-old female patient, who is the oldest patient in the present study with typical angina had a 76% PTP by UDFM. As a result, no woman had a PTP of more than 85% and was classified into the high PTP group. The reclassification table for women (Table 3) showed that more than half the negative and almost all the positive women were classified into the medium PTP group by UDFM.

Second, the prevalence of CCTA-based obstructive CAD did not correlate well with the presence or type of symptoms 25. The calculation of PTP by traditional models such as UDFM depended on a patient’s age, sex, and angina typicality 11. However, especially in women, a previous study had shown that patient symptoms categorized according to the classical definition 18 had a limited ability to predict obstructive CAD 25,27,28. This may be account for the unfavorable performance by UDFM in women, whereas DCS that include other risk factors and weakened prediction effect of angina typicality 19 improved the calculation accuracy of PTP with a positive NRI and IDI in this study. Similarly, Almeida et al. 14 and Jensen et al. 16 suggested that DCS seemed to perform better than UDFM in the prediction of obstructive CAD. It is worth noting that in the present research, DCS improved risk stratification through different mechanism pathways in men and women. DCS showed an NRI of 16.80% in men, which was ascribed to the reclassification of 30.91% (286/1184) of positive men to the higher PTP group, whereas 22.67% (471/2078) of negative women were reclassified into the lower PTP group, resulting in an NRI of 27.20%.

Recently, several large and real-world trials that were completed in symptomatic individuals showed low rates of cardiovascular event and positive noninvasive testing 22,27–29, especially in women 6,29. In conformity with this, according to the reclassification table in our study, UDFM classified more than half of the negative women into the medium PTP group, which was recommended to undergo noninvasive testing 7–9, and the MACE rate of these women was only 2.6%. In particular, noninvasive testing, especially stress testing, showed well-documented significant false-positive rates in women 5. Therefore, calculation of PTP by UDFM would lead to unnecessary testing and confounded approaches to the evaluation and diagnosis of obstructive CAD in women. Conversely, DCS classified most negative women into the low PTP group, resulting in a positive NRI. Moreover, the rate of MACEs in negative women reclassified into the low PTP group was extremely low. Thus, application of DCS instead of UDFM could alter the diagnostic strategy safely and effectively, leading to an evident decrease in unnecessary testing.

To increase the precision of PTP models in women, further studies may benefit from the following three factors. First, development of sex-specific equations is likely to contribute more toward balancing the variation by sex in the decision-making of the diagnostic strategy for CAD 30,31. Second, with the inclusion of some female-specific risk factors, such as estrogen status and gestational diabetes mellitus, the predictive ability of PTP models improved significantly 26,32. Third, novel markers, such as coronary artery calcium scores 13, which are manifestations of subclinical atherosclerosis, have shown the potential to improve the precision of PTP models. From a pathophysiological point of view, compared with risk factors, atherosclerosis per se is more reliable 33. Recently, using CCS, an extended model developed by Genders et al. 17, improved the prediction compared with the clinical model (cross-validated c statistic improvement from 0.79 to 0.88, NRI 102%). Furthermore, an analysis involving data from the PROMISE study showed that addition of CCS improved differentiation and calibration of traditional PTP models in women 34.

The present study has limitations that warrant acknowledgement. First, this was a retrospective and single-center analysis. We focused on the initial presentation and evaluation for patients with symptoms suggestive of stable CAD, but those patients who were referred to other tests were excluded, resulting in a marked selection bias. Thus, further multicenter and prospective studies are needed. Second, although the scan was performed by an experienced technician and the image was evaluated by three physicians by consensus, CCTA may overestimate the possibility of obstructive CAD because of the excellent negative predictive value and the moderate positive predictive value compared with invasive coronary angiogram 35. Third, the conclusions of this study should be validated and confirmed in comparative cost-effectiveness analyses with long-term outcome data. Fourth, a new model, the PROMISE minimal-risk tool, was developed recently to identify patients with stable chest pain at very low risk of CAD and clinical events 36. This risk score provided a novel strategy to identify patients in whom noninvasive testing might be deferred safely and outperformed existing PTP models 37. Thus, more external validation investigations are needed in the future to determine whether the application of the PROMISE minimal-risk tool can reduce unnecessary testing safely and effectively, especially for women.

Back to Top | Article Outline


The clinical presentation and calculation of PTP differed significantly by sex in patients presenting with stable chest pain and referred to CCTA. Although the performance of neither model was satisfactory, DCS yielded a more accurate calculation of PTP than UDFM and application of DCS instead of UDFM would result in a significant decrease in inappropriate testing in women. These data suggest that continued investigations in this area are warranted to balance the sex-specific differences in the calculation of PTP.

Back to Top | Article Outline


This wok was funded by the Research Program of Tianjin Chest Hospital (no. 2018XKC10), the Key Program of Medical Industry of Tianjin (no. 16KG132), and grants from the Committee on Science and Technology, Jinnan District, Tianjin.

Back to Top | Article Outline

Conflicts of interest

There are no conflicts of interest.

Back to Top | Article Outline


1. Hemingway H, McCallum A, Shipley M, Manderbacka K, Martikainen P, Keskimaki I. Incidence and prognostic implications of stable angina pectoris among women and men. JAMA 2006; 295:1404–1411.
2. Davis MB, Maddox TM, Langner P, Plomondon ME, Rumsfeld JS, Duvernoy CS. Characteristics and outcomes of women veterans undergoing cardiac catheterization in the Veterans Affairs Healthcare System: insights from the VA CART Program. Circ Cardiovasc Qual Outcomes 2015; 8:S39–S47.
3. Wenger NK. Gender disparity in cardiovascular disease: bias or biology? Expert Rev Cardiovasc Ther 2012; 10:1401–1411.
4. Paulus JK, Shah ND, Kent DM. All else being equal, men and women are still not the same: using risk models to understand gender disparities in care. Circ Cardiovasc Qual Outcomes 2015; 8:317–320.
5. Baldassarre LA, Raman SV, Min JK, Mieres JH, Gulati M, Wenger NK, et al. Noninvasive imaging to evaluate women with stable ischemic heart disease. JACC Cardiovasc Imaging 2016; 9:421–435.
6. Hemal K, Pagidipati NJ, Coles A, Dolor RJ, Mark DB, Pellikka PA, et al. Sex differences in demographics, risk factors, presentation, and noninvasive testing in stable outpatients with suspected coronary artery disease: insights from the PROMISE Trial. JACC Cardiovasc Imaging 2016; 9:337–346.
7. Wolk MJ, Bailey SR, Doherty JU, Douglas PS, Hendel RC, Kramer CM, et al. ACCF/AHA/ASE/ASNC/HFSA/HRS/SCAI/SCCT/SCMR/STS 2013 multimodality appropriate use criteria for the detection and risk assessment of stable ischemic heart disease: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Thoracic Surgeons. J Am Coll Cardiol 2014; 63:380–406.
8. Fihn SD, Blankenship JC, Alexander KP, Bittl JA, Byrne JG, Fletcher BJ, et al. ACC/AHA/AATS/PCNA/SCAI/STS focused update of the guideline for the diagnosis and management of patients with stable ischemic heart disease. J Am Coll Cardiol 2014; 64:1929–1949.
9. Task Force M, Montalescot G, Sechtem U, Achenbach S, Andreotti F, Arden C, et al. ESC guidelines on the management of stable coronary artery disease: the Task Force on the management of stable coronary artery disease of the European Society of Cardiology. Eur Heart J 2013; 34:2949–3003.
10. Skinner JS, Smeeth L, Kendall JM, Adams PC, Timmis A. NICE guidance. Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin. Heart 2010; 96:974–978.
11. Genders TS, Steyerberg EW, Alkadhi H, Leschka S, Desbiolles L, Nieman K, et al. A clinical prediction rule for the diagnosis of coronary artery disease: validation, updating, and extension. Eur Heart J 2011; 32:1316–1330.
12. Pryor DB, Harrell FE Jr, Lee KL, Califf RM, Rosati RA. Estimating the likelihood of significant coronary artery disease. Am J Med 1983; 75:771–780.
13. Takamura K, Kondo T, Fujimoto S, Hiki M, Matsumori R, Kawaguchi Y, et al. Incremental predictive value for obstructive coronary artery disease by combination of Duke Clinical Score and Agatston score. Eur Heart J Cardiovasc Imaging 2016; 17:550–556.
14. Almeida J, Fonseca P, Dias T, Ladeiras-Lopes R, Bettencourt N, Ribeiro J, et al. Comparison of Coronary Artery Disease Consortium 1 and 2 Scores and Duke Clinical Score to Predict Obstructive Coronary Disease by Invasive Coronary Angiography. Clin Cardiol 2016; 39:223–228.
15. Fujimoto S, Kondo T, Yamamoto H, Yokoyama N, Tarutani Y, Takamura K, et al. Development of new risk score for pre-test probability of obstructive coronary artery disease based on coronary CT angiography. Heart Vessels 2015; 30:563–571.
16. Jensen JM, Voss M, Hansen VB, Andersen LK, Johansen PB, Munkholm H, et al. Risk stratification of patients suspected of coronary artery disease: comparison of five different models. Atherosclerosis 2012; 220:557–562.
17. Genders TS, Steyerberg EW, Hunink MG, Nieman K, Galema TW, Mollet NR, et al. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ 2012; 344:e3485.
18. Diamond GA. A clinically relevant classification of chest discomfort. J Am Coll Cardiol 1983; 1:574–575.
19. Pryor DB, Shaw L, McCants CB, Lee KL, Mark DB, Harrell FE, et al. Value of the history and physical in identifying patients at increased risk for coronary artery disease. Ann Intern Med 1993; 118:81–90.
20. Cury RC, Abbara S, Achenbach S, Agatston A, Berman DS, Budoff MJ, et al. CAD-RADS(TM) Coronary Artery Disease – Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Cardiovasc Comput Tomogr 2016; 10:269–281.
21. Bittencourt MS, Hulten E, Polonsky TS, Hoffman U, Nasir K, Abbara S, et al. European Society of Cardiology-Recommended Coronary Artery Disease Consortium Pretest Probability Scores More Accurately Predict Obstructive Coronary Disease and Cardiovascular Events Than the Diamond and Forrester Score: The Partners Registry. Circulation 2016; 134:201–211.
22. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29–36.
23. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27:157–172; [discussion 207–212].
24. Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer–Lemeshow test revisited. Crit Care Med 2007; 35:2052–2056.
25. Cheng VY, Berman DS, Rozanski A, Dunning AM, Achenbach S, Al-Mallah M, et al. Performance of the traditional age, sex, and angina typicality-based approach for estimating pretest probability of angiographically significant coronary artery disease in patients undergoing coronary computed tomographic angiography: results from the multinational coronary CT angiography evaluation for clinical outcomes: an international multicenter registry (CONFIRM). Circulation 2011; 124:2423–2432.
26. Rademaker AA, Danad I, Groothuis JG, Heymans MW, Marcu CB, Knaapen P, et al. Comparison of different cardiac risk scores for coronary artery disease in symptomatic women: do female-specific risk factors matter? Eur J Prev Cardiol 2014; 21:1443–1450.
27. Rovai D, Neglia D, Lorenzoni V, Caselli C, Knuuti J, Underwood SR. Limitations of chest pain categorization models to predict coronary artery disease. Am J Cardiol 2015; 116:504–507.
28. O’Keefe-McCarthy S. Women’s experiences of cardiac pain: a review of the literature. Can J Cardiovasc Nurs 2008; 18:18–25.
29. Pagidipati NJ, Hemal K, Coles A, Mark DB, Dolor RJ, Pellikka PA, et al. Sex differences in functional and CT angiography testing in patients with suspected coronary artery disease. J Am Coll Cardiol 2016; 67:2607–2616.
30. Yang X, Li J, Hu D, Chen J, Li Y, Huang J, et al. Predicting the 10-year risks of atherosclerotic cardiovascular disease in Chinese population: the China-PAR project (Prediction for ASCVD Risk in China). Circulation 2016; 134:1430–1440.
31. Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007; 297:611–619.
32. Morise AP, Haddad WJ, Beckner D. Development and validation of a clinical score to estimate the probability of coronary artery disease in men and women presenting with suspected coronary disease. Am J Med 1997; 102:350–356.
33. Mureddu GF, Brandimarte F, Faggiano P, Rigo F, Nixdorff U. Between risk charts and imaging: how should we stratify cardiovascular risk in clinical practice? Eur Heart J Cardiovasc Imaging 2013; 14:401–416.
34. Genders TSS, Coles A, Hoffmann U, Patel MR, Mark DB, Lee KL, et al. The external validity of prediction models for the diagnosis of obstructive coronary artery disease in patients with stable chest pain: insights from the PROMISE trial. JACC Cardiovasc Imaging 2018; 11:437–446.
35. Arbab-Zadeh A, Hoe J. Quantification of coronary arterial stenoses by multidetector CT angiography in comparison with conventional angiography methods, caveats, and implications. JACC Cardiovasc Imaging 2011; 4:191–202.
36. Fordyce CB, Douglas PS, Roberts RS, Hoffmann U, Al-Khalidi HR, Patel MR, et al. Identification of patients with stable chest pain deriving minimal value from noninvasive testing: the PROMISE minimal-risk tool, a secondary analysis of a randomized clinical trial. JAMA Cardiol 2017; 2:400–408.
37. Adamson PD, Fordyce CB, McAllister DA, Udelson JE, Douglas PS, Newby DE. Identification of patients with stable chest pain deriving minimal value from coronary computed tomography angiography: an external validation of the PROMISE minimal-risk tool. Int J Cardiol 2018; 252:31–34.

coronary computed tomographic angiography; obstructive coronary artery disease; pretest probability; sex-specific differences

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.