Increasing rates of cardiovascular diseases (CVD) in low-income and middle-income countries (LMIC), and occurring at younger ages than seen in high-income countries, urge the need for action.1–3 Many public health interventions have arisen to prevent CVD in LMIC, including policies to reducing tobacco usage and salt intake. CVD prevention at the individual level remains necessary and cost-effective.4,5 The World Health Organization (WHO) recommends using CVD risk assessment tools to direct evidence-based medication, including aspirin and statin therapy, for primary prevention of asymptomatic individuals in relation to any given level of CVD risk.6 In addition, recent evidence augments these recommendations by showing that blood pressure lowering protects patients according to their CVD risk status. Because larger benefits were observed among groups with greater baseline CVD risk, risk equations appear well positioned to also inform blood pressure-lowering treatment decisions.7
However, to date, no CVD risk assessment tool has been developed using longitudinal data from LMIC. The likelihood of this occurring in the near term is very small. The most recent comprehensive systematic pooling of worldwide longitudinal cardiometabolic data, which analyzed 96 cohorts totaling 1.8 million subjects,8 looked prospectively to the relationship and burden of 2 important outcomes, ie, coronary heart disease (CHD) and stroke. Yet, out of the 97 studies, only one was from the South American region (Brazil).8 In this context, existing CVD risk assessment tools do fulfill a role on informing day-to-day clinical practice in most LMIC. However, frontline clinical workers and policy makers in LMIC settings do not have sufficient guidance on which tool to use better.
Although the Framingham Heart Study resulted in the first cardiovascular risk prediction model, the Framingham risk score (FRS), additional risk scores have been developed accounting for novel CVD risk factors and different populations.9 Based on the FRS, the third Adult Treatment Panel incorporated risk stratification in determining eligibility for statin therapy.10 However, the FRS risk assessment tool used in the third Adult Treatment Panel focuses in predicting 10-year risk of coronary events, defined as nonfatal myocardial infarction (MI) and CHD-related mortality. In 2014, the American College of Cardiology/American Heart Association (ACC/AHA) introduced the Pooled Cohort risk equations (ACC/AHA model),11 which used data from several community-based US-specific cohort studies to develop new sex-specific and race-specific equations to predict 10-year risk for atherosclerotic CVD events, including nonfatal MI, CHD-related mortality, and stroke.12 The ACC/AHA model has been validated in a separate USA cohort and demonstrated good calibration for the population it was designed.13 However, when compared with a European cohort, these models showed poor calibration and moderate to good discrimination.14 It is unclear how well the currently available risk scores assess or predict CVD risk in LMIC and in Latin American settings. With the exception of the FRS, most of CVD risk assessment tools lack external validation in LMIC.9
We aimed to compare the ACC/AHA model with 6 other CVD risk tools to assess the concordance of predicted CVD risk in a random sample from 5 geographically diverse Peruvian populations. Moreover, to estimate the clinical impact of adopting the use of each risk assessment tool in Peru, we compared the proportion of individuals classified as high CVD risk according to the established cut-offs for each risk assessment tool.
We used secondary cross-sectional data from 2 Peruvian population-based epidemiological studies: the PERU MIGRANT Study15 and the CRONICAS cohort study.16 The former, n = 989, described the CVD risk profiles in 3 Peruvian populations: Ayacucho (rural, highlands), Lima (urban, sea level), and rural-to-urban migrants in Lima. The CRONICAS cohort study has been designed as a prospective study of cardiopulmonary risk factors in over 3000 individuals in 4 Peruvian populations: Lima (highly urban, sea level), Tumbes (semiurban, sea level), rural, and urban Puno (highlands). In this study, we used the baseline data from CRONICAS. Both studies used a structured questionnaire and collected anthropometric and laboratory data15,16 required for CVD risk estimation (Table 1).
CVD Risk Prediction Tools
We compared the ACC/AHA model with 6 other CVD risk prediction tools: laboratory Framingham risk score (FRS-lipids),17 non-laboratory Framingham risk score using body mass index (FRS-non-lab),17 Reynolds risk score (RRS),19,20 systematic coronary risk evaluation (SCORE),21 WHO’s Risk Chart for the Americas Region (WHO/International Society of Hypertension),18 and the risk chart developed by the Lancet chronic diseases group.22 The last 2 risk models provide a chart-based categorization of high risk and were included in our analyses because they were created specifically for use in developing countries.
Participants with known CVD, MI, or stroke were excluded from our analysis. To enable comparisons across scores, we restricted our analysis to participants within 45–65 years age range.
Continuous and categorical 10-year CVD risks were calculated, using the risk algorithms or charts, where available. Further details about each risk prediction tools included in this study are shown in Table 1. We assessed agreement in pair-to-pair analyses against the ACC/AHA model. For those scores where risk equations were available, we calculated predicted risk as a continuous variable and used Lin’s concordance correlation coefficient (CCC).23 CCC quantifies the agreement ranging from −1 to 1, with perfect agreement at 1. CCC has the following classification according to strength of agreement (theoretical): >0.99 almost perfect, 0.95–0.99 substantial, 0.90–0.95 moderate, and <0.90 poor.24 Empirical cut-offs for delimiting optimal concordance, where CCC values obtained after contrasting different versions of the same CVD risk scores, ie, FRS-Lipids versus FRS-non-lab; SCORE-1 versus SCORE-2, and SCORE-3 versus SCORE-4, are shown separately in Supplemental Tables 1 and 2 (Supplemental Digital Content, http://links.lww.com/HPC/A198).
We calculated percentages of people at high CVD risk for every score, using the established cut-off points in their original reports: ACC/AHA model ≥7.5%, FRS ≥20%, RRS ≥20%, Lancet Chronic Diseases ≥15%, SCORE ≥5%, and WHO/International Society of Hypertension ≥20%. Furthermore, a subsequent analysis was conducted using a similar definition of high CVD risk at 20% across all tools for visual assessment on their percentages of people classified as high risk. Additional agreement evaluation between dichotomous variables of CVD risk was performed using Kappa index (Supplemental Table 3, Supplemental Digital Content, http://links.lww.com/HPC/A198).
STATA 12 software (STATA Corp, College Station, TX, USA) was used for analysis. In all estimations 95% confidence intervals were calculated.
The combined datasets included a total of 4604 participants. The following were excluded: 184 participants with prior episodes of MI or stroke and 2165 participants outside of the scores’ age range. To avoid individual-level data duplication, 72 records from the PERU MIGRANT study were also removed, as these subjects also took part in the CRONICAS Cohort Study. Therefore, data from a total of 2183 participants, mean age 54.3 (SD ± 5.6) years, were included in the analysis.
Table 2 shows study populations’ characteristics. Across sites, most of the participants reported a monthly family income of $196 USD or less, except in Tumbes and urban Puno where the majority of participants reported a monthly income between $197 and $535 USD. Both rural sites, Ayacucho and rural Puno, had the lowest body mass index and highest mean high-density lipoprotein cholesterol. Relative to the other study sites, mean glucose was much higher in Tumbes and lowest in Ayacucho, and these 2 sites had higher mean systolic blood pressures.
Agreement Between Risk Scoring Tools
Agreement between CVD risks in continuous format (Table 3) was evaluated using CCC. Overall, when scores were compared with ACC/AHA model, in the total sample as well as specific sites, we found poor CCC agreement values. The highest agreement was observed in comparisons between ACC/AHA model and FRS-non-lab (40%) and FRS-Lipid scores (44%). The lowest agreement was observed between ACC/AHA model and SCORE-3 and SCORE-4, with 14% and 17%, respectively.
Agreement between risk scores also differed in magnitude by study sites. The highest agreement values were observed in rural Ayacucho, 74% between ACC/AHA model and FRS-non-lab, and in rural Puno, 77% between ACC/AHA model and RRS.
Prevalence of High CVD Risk as Per Recommended Guidelines
Figure 1 shows the prevalence of high CVD risk status in the pooled dataset as per recommendations by each risk assessment tool’s original publication. ACC/AHA model depicted the highest proportion of people at high CVD risk predicted at 10 years, with a prevalence of 29.0% (95% confidence interval, 26.9–31.0%). When analyzed by specific study sites, ACC/AHA model also had the highest prevalence of individuals at high CVD risk relative to other scores (Supplemental Fig. 1, Supplemental Digital Content, http://links.lww.com/HPC/A198). Supplemental Table 3 (Supplemental Digital Content, http://links.lww.com/HPC/A198) shows Kappa for agreement between categories of predicted high risk by scores, with findings indicating poor agreement.
Using a similar definition of high risk at 20% across all tools, FRS yielded the highest overall prevalence of high-risk status (Fig. 2). This pattern was the same across subgroup populations, except in Ayacucho where high-risk status was much higher with ACC/AHA model (Supplemental Fig. 2, Supplemental Digital Content, http://links.lww.com/HPC/A198).
This study compares 6 CVD risk assessment tools in a pooled dataset with different Peruvian populations, representing low-income communities in diverse geographical settings, ie, urban and rural, coastal and mountainous areas. Poor agreement was found for interscore agreement evaluating CVD risk in 10 years. If these risk assessment tools were to be applied following their respective guidelines there would be wide variation in the proportion of groups classified as high risk for CVD. A remarkable high number of people would be eligible to initiate pharmacological therapy according to our results. Our study provides insights about the applicability of CVD high-risk recommendations for LMIC contexts by using real data across a diversity of settings, including poor overlap between risk assessment tools, which signals to major discrepancies that merit careful attention.
Clinical and preventive usefulness can be drawn from our results. We found poor agreement between the newest ACC/AHA model and other available tools. Moreover, there is huge variation between the proportions of the total population classified as high-risk by each risk assessment tool, potentially impairing recommendations for CVD prevention at the individual level. These findings limit the utility of risk prediction scores for public health and health systems planning, particularly in resource-limited settings. In these countries, further economic studies may be needed to assess cost-effectiveness of risk prediction as primary prevention.25
Variations observed in risk-score agreement between different Peruvian sub-populations may be explained by differences in baseline predictors’ profiles. The 2 rural sites in our study, Puno and Ayacucho, had different proportions of high risk as determined by ACC/AHA model. These discrepancies were also appreciated in a discordance between which individuals were classified as high risk by each risk assessment tool. Given the rise of noncommunicable diseases in LMIC, further calibration and validation of these tools are needed in such settings, and longitudinal studies could help to clear this cloudy panorama.26
In line with Krumholz et al’s27 reflections, our study also raises substantial concerns about the generalizability of the risk equations and also raises the question of whether a threshold for treatment, which is admittedly arbitrary and is imbued with values about what level of risk is worth treating, is relevant to all countries. Given WHO’s recommendations of using risk assessment scores to determine eligibility for evidence-based therapy, including statin therapy, it is necessary to have accurate and precise CVD risk assessment tools. The relevance of CVD risk assessment is compounded by increasing interest in polypills—low-dose combinations of blood-pressure lowering, cholesterol lowering, and antiplatelet medications—as a form of risk reduction for individuals with globally increased high risk rather than interventions tailored to specific risk factor control.28
Using hypothetical data applied to 25 different risk calculators, Allan et al29 also found poor agreement, in the order of 67%, similar to our study, highlighting the need to calibrate CVD risk assessment models to every population when applying clinical guidelines for pharmacological therapy. Current risk-scoring strategies are limited to USA and European populations,30 and the INTERHEART modifiable risk score, originated from multi-country cross-sectional data, is restricted to the prediction of MI.31 Although FRS has been validated for diverse US ethnic groups,32 it has also been demonstrated that FRS is not stable in predicting risk, overestimates and underestimates, in non-USA groups.26,33–36 The ACC/AHA model has been tested against US cohorts showing discrepancies.13,37,38 Similarly, the ACC/AHA model was shown to yield poor calibration against FRS-lipids and SCORE in the Rotterdam cohort study.14 Although our pooled dataset does not have 10-year follow-up to externally validate any of the risk assessment models, the variability in risk prediction demonstrated in our study highlights the need to externally validate CVD risk assessment models in LMIC.
Populations from different geographical and epidemiological profiles enrich the value of our study, including rural, urban, lowland and high altitude sites, much common across Latin America. Some limitations deserve consideration. Not all tool’s equations were available, thus chart-based scores did not provide risk assessments as a continuous variable, yet in our analysis we included the most commonly used risk scores. Another limitation that affects our comparability arises from differences in the definitions used in risk scores’ predictors and outcomes. For example, WHO and SCORE address CVD-related mortality, whereas the ACC/AHA model focuses on fatal and nonfatal atherosclerotic CVD; however, we would have expected a higher proportion of overlapping between tools because mortality is included in ACC/AHA model. Also, we are cognizant that the comparison of yields using different cut-offs, as per current recommended guidelines, is limited. The reason for doing this assessment is that most healthcare providers will not necessarily be aware of the technical and modeling details behind a “10-year high-risk” label that guidelines tend to “have a paternalistic tone [and] tell physicians what should be done”27 and, in the absence of locally available guidelines, they will likely drive practitioners’ prescription practices, misassembling level of risk with the consequent overtreatment of people with low level of risk. Therefore, our work signals salient differences and poor agreements that in turns calls for more education around the usage of risk prediction tools in LMIC together with the need to advance the development of LMIC-specific validation efforts.
In summary, we have shown poor agreement when available scores were compared with the newly released ACC/AHA model. We have also shown that in Peruvian population, a high proportion of individuals would be classified into high CVD risk category. These findings emphasize the uncertainty of choosing any of these tools, into both clinical and public health fields, in LMIC realities. Our work signals to a major and urgent need to improve the evidence base for the development and appropriate use of appropriate risk scores for CVD in LMIC.
The authors are indebted to all participants who kindly agreed to participate in the study. Special thanks to all field teams for their commitment and hard work, especially to Lilia Cabrera, Rosa Salirrosas, Viterbo Aybar, Sergio Mimbela, and David Danz for their leadership in each of the study sites, as well as Marco Varela for data coordination. Special thanks to Tracy Wolbach and Patricia Davis (Westat) for their great support provided as part of the NHLBI Trainee Seed Grant Award experience. CRONICAS Cohort Study Group—Cardiovascular disease: Antonio Bernabé-Ortiz, Juan P. Casas, George Davey Smith, Shah Ebrahim, Héctor H. García, Robert H. Gilman, Luis Huicho, Germán Málaga, J. Jaime Miranda, Víctor M. Montori, Liam Smeeth; Chronic obstructive pulmonary disease: William Checkley, Gregory B. Diette, Robert H. Gilman, Luis Huicho, Fabiola León-Velarde, María Rivera, Robert A. Wise; Training and Capacity Building: William Checkley, Héctor H. García, Robert H. Gilman, J. Jaime Miranda, Katherine Sacksteder. PERU MIGRANT Study—Antonio Bernabé-Ortiz, Lilia Cabrera, Héctor H. García, Robert H. Gilman, J. Jaime Miranda, Julio A. Poterico, Renato Quispe, Candice Romero, Juan F. Sánchez, Liam Smeeth.
J.J.M. conceived the idea. J.C.B.A., F.P., J.A.P., T.P., and J.J.M. further developed the idea and obtained funding for its secondary analysis. J.C.B.A. led the statistical analysis. J.C.B.A., R.Q., F.P., J.A.P., G.A.V., M.B., and T.P., as part of a trainee-led team, wrote the initial drafts of this manuscript. J.J.M., R.H.G., W.C., and L.S. conceived, designed, and supervised the overall fieldwork studies. J.J.M., A.B.O., and W.C. coordinated and supervised fieldwork activities in Ayacucho, Lima, Tumbes, and Puno. All authors participated in writing of manuscript, provided important intellectual content, and gave their final approval of the version submitted for publication.
The authors declare that they have no competing interests.
1. Ezzati M, Riboli E.. Behavioral and dietary risk factors for noncommunicable diseases. N Engl J Med. 2013;369:954–964
2. Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095–2128
3. World Health Organization. Cardiovascular diseases
(CVDs). Fact sheet No. 317. 2013 Updated March 2013. Available from: http://www.who.int/mediacentre/factsheets/fs317/en/
. Accessed June 3, 2014
4. Cooney MT, Dudina AL, Graham IM.. Value and limitations of existing scores for the assessment of cardiovascular risk: a review for clinicians. J Am Coll Cardiol. 2009;54:1209–1227
5. Lloyd-Jones DM.. Cardiovascular risk prediction: basic concepts, current status, and future directions. Circulation. 2010;121:1768–1777
6. World Health Organization. Prevention of Cardiovascular Disease. Guidelines for Assessment and Management of Cardiovascular Risk. 2007 Geneva WHO
7. Sundstrom J, Arima H, et al.Blood Pressure Lowering Treatment Trialists Collaboration. Blood pressure-lowering treatment based on cardiovascular risk: a meta-analysis of individual patient data. Lancet. 2014;384:591–598
8. Lu Y, Hajifathalian K, et al.Global Burden of Metabolic Risk Factors for Chronic Diseases Collaboration. Metabolic mediators of the effects of body-mass index, overweight, and obesity on coronary heart disease and stroke: a pooled analysis of 97 prospective cohorts with 1.8 million participants. Lancet. 2014;383:970–83
9. Bitton A, Gaziano TA.. The Framingham heart study’s impact on global risk assessment. Prog Cardiovasc Dis. 2010;53:68–78
10. National Cholesterol Education Program Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adults Treatment Panel III). . Third Report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report. Circulation. 2002;106:3143–3421
11. Goff DC Jr, Lloyd-Jones DM, Bennett G, et al.American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 Pt B):2935–2959
12. Goff DC Jr, Lloyd-Jones DM, Bennett G, et al.American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25 Suppl 2):S49–S73
13. Muntner P, Colantonio LD, Cushman M, et al. Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations. JAMA. 2014;311:1406–1415
14. Kavousi M, Leening MJ, Nanchen D, et al. Comparison of application of the ACC/AHA guidelines, Adult Treatment Panel III guidelines, and European Society of Cardiology guidelines for cardiovascular disease prevention in a European cohort. JAMA. 2014;311:1416–1423
15. Miranda JJ, Gilman RH, García HH, et al. The effect on cardiovascular risk factors of migration from rural to urban areas in Peru
MIGRANT study. BMC Cardiovasc Disord. 2009;9:23
16. Miranda JJ, Bernabe-Ortiz A, Smeeth L, et al.CRONICAS Cohort Study Group. Addressing geographical variation in the progression of non-communicable diseases in Peru
: the CRONICAS cohort study protocol. BMJ Open. 2012;2:e000610
17. D’Agostino RB Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117:743–753
18. World Health Organization. WHO/ISH Risk Prediction Charts. 2007 Geneva WHO
19. Ridker PM, Paynter NP, Rifai N, et al. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds risk score for men. Circulation. 2008;118:2243–2251, 4p following 2251
20. Ridker PM, Buring JE, Rifai N, et al. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds risk score. JAMA. 2007;297:611–619
21. Conroy RM, Pyörälä K, Fitzgerald AP, et al.SCORE project group. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24:987–1003
22. Lim SS, Gaziano TA, Gakidou E, et al. Prevention of cardiovascular disease in high-risk individuals in low-income and middle-income countries: health effects and costs. Lancet. 2007;370:2054–2062
23. Steichen TJ, Cox NJ.. A note on the concordance correlation coefficient. Stata J. 2002;2:183–189
24. McBride G.. A proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. NIWA Client Report: HAM2005-062. 2005
25. . Statins for millions more? Lancet. 2014;383:669
26. Cortes-Bergoderi M, Thomas RJ, Albuquerque FN, et al. Validity of cardiovascular risk prediction models in Latin America and among Hispanics in the United States of America: a systematic review. Rev Panam Salud Publica. 2012;32:131–139
27. Krumholz HM.. The new cholesterol and blood pressure guidelines: perspective on the path forward. JAMA. 2014;311:1403–1405
28. Lonn E, Bosch J, Teo KK, et al. The polypill in the prevention of cardiovascular diseases
: key concepts, current status, challenges, and future directions. Circulation. 2010;122:2078–2088
29. Allan GM, Nouri F, Korownyk C, et al. Agreement among cardiovascular disease risk calculators. Circulation. 2013;127:1948–1956
30. Siontis GC, Tzoulaki I, Siontis KC, et al. Comparisons of established risk prediction models for cardiovascular disease: systematic review. BMJ. 2012;344:e3318
31. McGorrian C, Yusuf S, Islam S, et al.INTERHEART Investigators. Estimating modifiable coronary heart disease risk in multiple regions of the world: the INTERHEART Modifiable Risk Score. Eur Heart J. 2011;32:581–589
32. D’Agostino RB Sr, Grundy S, Sullivan LM, et al.CHD Risk Prediction Group. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA. 2001;286:180–187
33. Brindle P, Emberson J, Lampe F, et al. Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ. 2003;327:1267
34. Liu J, Hong Y, D’Agostino RB Sr, et al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-Provincial Cohort Study. JAMA. 2004;291:2591–2599
35. Hense HW, Schulte H, Löwel H, et al. Framingham risk function overestimates risk of coronary heart disease in men and women from Germany–results from the MONICA Augsburg and the PROCAM cohorts. Eur Heart J. 2003;24:937–945
36. Thomsen TF, McGee D, Davidsen M, et al. A cross-validation of risk-scores for coronary heart disease mortality based on data from the Glostrup Population Studies and Framingham Heart Study. Int J Epidemiol. 2002;31:817–822
37. Ridker PM, Cook NR.. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762–1765
38. Lloyd-Jones DM, Goff D, Stone NJ.. Statins, risk assessment, and the new American prevention guidelines. Lancet. 2014;383:600–602