Share this article on:

Improving risk prediction performance for a better guidelines application

Marchant, Ivannya; Bejan-Angoulvant, Theodorab; Le, Ha Hac; Gueyffier, Françoisc

doi: 10.1097/HJH.0000000000000194
Editorial Commentaries

aDepartamento de Preclínicas, Escuela de Medicina, Universidad de Valparaíso, Valparaíso, Chile

bService de Pharmacologie Clinique, Centre Hospitalier Régional et Universitaire de Tours, Tours

cCNRS, UMR 5558, Laboratoire de Biométrie et Biologie Evolutive, Service de, Pharmacologie Clinique, Lyon, France

Correspondence to Francois Gueyffier, CNRS, UMR 5558, Laboratoire de Biométrie et Biologie Evolutive, Service de, Pharmacologie Clinique, Lyon F-69376, France. E-mail:

Predicting a global risk of cardiovascular events is an efficient way to orient health resources towards individuals who will the most benefit from cardiovascular primary prevention, at least at a fixed horizon time, for example, 5 or 10 years. This is why taking into account cardiovascular risk in the decision to treat people with hypertension or with hypercholesterolemia by blood pressure (BP)-lowering drugs or by statins is advocated for more than 20 years by international and national guidelines [1]. Cardiovascular risk level and the level of evidence for the benefit have become the pillars of the decision, far above the pretreatment level of BP or cholesterol which will be reduced by treatment [2,3].

This strategy raises several issues that are not yet all resolved:

  1. The interaction between the level of risk factor and treatment benefit in terms of relative risk; if there is no interaction, then there is no spontaneous threshold to define a risk factor level that would justify per se the preventive prescription;
  2. The interaction between the level of cardiovascular risk and relative treatment benefit; the above discussion translates here for the definition of the appropriate risk threshold;
  3. The appropriateness of the risk prediction tool, with many ancillary issues: which outcomes to consider, fatal or not, coronary or not, and so on? At which time horizon? Which covariates to be included?

The study by Ravera et al. [4] deals with an important issue, to consider or not renal function as a specific risk factor to be included in predicting cardiovascular risk. That renal function deterioration is associated with a higher cardiovascular risk is beyond any doubt. The strong association between this deterioration and other powerful covariates such as age, sex or high blood pressure could explain why in some studies it did not appear as significantly associated to cardiovascular risk per se. As highlighted by Ravera et al., considering it as a dichotomous condition, with the consequent reduction of power, may have been another explanation. Conversely, renal impairment in atrial fibrillation has been recently found to be a strong predictor of stroke or systemic embolism [5].

Their comparison between the most cited and considered-as-validated risk score, that is, the Framingham one [6], and another one designed for UK hypertensive individuals [7], clearly answers the question: individual classification was significantly improved when using the INDividual Data ANalysis of Antihypertensive trials (INDANA) risk score incorporating renal function parameter, compared to the Framingham risk score or to the INDANA risk score without taking renal function into account. That this result was observed on two independent cohorts, one in UK and another in Italy, seems to prevent a better geographical fit as an explanation for the better performance for INDANA score.

The INDANA approach presents several theoretical advantages: it was parameterized on untreated hypertensive individuals, and it was built on several cohorts, from various origins. The latter could explain a better generalizability, advantage shared with the Systematic Coronary Risk Evaluation-building approach [8], whereas the Framingham study does represent a geographically limited population.

The comparison of the ability of two risk scores to discriminate those who will develop the event of interest and those who will not is traditionally based on receiver-operating characteristic (ROC) curves, which tell us how a score jointly improves sensitivity and specificity. New markers have been evaluated based on their ability to improve the area under the ROC curve in comparison to the risk score without the marker. However, very large associations of the new marker with the outcome of interest are required to enlarge the area under the ROC curve meaningfully [9]. More recently introduced, the net reclassification improvement (NRI) statistics helps us better to understand what does change using a given risk score in terms of our ability to predict the occurrence of events, and the rate of errors we avoid compared to another score. Complementary to the ROC measurements, the NRI may reveal significant gain in the performance of a risk score by adding a marker that has shown little or no significant improvement in area under ROC curves. In addition to discrimination, the score calibration is another important performance to consider when we want using scores to inform medical decision. Indeed, to fully apply the recommendations, we have merely to multiply the absolute risk level obtained from such a risk score by the synthetic risk ratio calculated in a systematic review, which directly yields the two risks, with and without treatment. This allows immediately knowing the absolute benefit to be expected from treatment, which we can express as the number needed to treat to prevent one event, at the time horizon used by the score. It is frequent to consider that scores have a similar discrimination power, but differ usually in their calibration, which could be corrected by a mere change of the intercept value [10].

Together with the results from systematic reviews, using risk scores to inform medical decisions is an obvious way to evolve towards personalized and more evidence-based medicine. Adjusting therapeutic burden to the level of benefit to be expected, instead of the deceitful apparent response on the risk factors, could represent a significant improvement in our use of limited health resources: it would help us to avoid increasing absurdly the drug burden of the individuals who do not appear to respond below the targets that were arbitrarily defined. Following a rigorous methodology, Ravera et al. have assessed all main aspects of the performance of INDANA score providing reliable support to their conclusions. The validation of a risk score that takes into account renal function impairment and improves the performance of mortality risk assessment in hypertensive individuals represents an important step toward the abovementioned goals.

The potentially huge importance of risk scores in clinical daily practice will require more specific epidemiological studies: the first step of medicine personalization will be to relate the most relevant exposition factors to the occurrence of morbid events to be prevented, the most precisely and specifically in every geographical area of the world. This geographical specificity seems required to get appropriate calibration, and adjust synthetic scores obtained on meta-analyses performed at individual level.

Back to Top | Article Outline


Conflicts of interest

There are no conflict of interest in relation to the content of this article.

Back to Top | Article Outline


1. Zannad F, Dallongeville J, Macfadyen RJ, Ruilope LM, Wilhelmsen L, De Backer G, et al. Prevention of cardiovascular disease guided by total risk estimations: challenges and opportunities for practical implementation: highlights of a CardioVascular Clinical Trialists (CVCT) Workshop of the ESC Working Group on CardioVascular Pharmacology and Drug Therapy. Eur J Prev Cardiol 2012; 19:1454–1464.
2. ESH-ESC The Task Force for the Management of Arterial Hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC). J Hypertens 2013; 31:1281–1357.
3. Stone NJ, Robinson J, Lichtenstein AH, Bairey Merz CN, Lloyd-Jones DM, Blum CB, et al. 2013 ACC/AHA Guideline on the Treatment of Blood Cholesterol to Reduce Atherosclerotic Cardiovascular Risk in Adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2013; [Epub ahead of print] .
4. Ravera M, Cannavò R, Noberasco G, Guasconi A, Cabib U, Pieracci L, et al. High performance of a risk calculator that includes renal function in predicting mortality of hypertensive patients in clinical application. J Hypertension 2014; 32:1245–1254.
5. Piccini JP, Stevens SR, Chang Y, Singer DE, Lokhnygina Y, Go AS, et al. Renal dysfunction as a predictor of stroke and systemic embolism in patients with nonvalvular atrial fibrillation: validation of the R2CHADS2 index in the ROCKET AF (Rivaroxaban Once-daily, oral, direct factor Xa inhibition Compared with vitamin K antagonism for prevention of stroke and Embolism Trial in Atrial Fibrillation) and ATRIA (AnTicoagulation and Risk factors In Atrial fibrillation) Study Cohorts. Circulation 2013; 127:224–232.
6. Anderson KM, Odell PM, Wilson WF, Kannel WB. Cardiovascular disease risk profiles. Am Heart J 1990; 121:293–298.
7. Pocock SJ, McCormack V, Gueyffier F, Boutitie F, Fagard RH, Boissel JP. A score for predicting risk of death from cardiovascular disease in adults with raised blood pressure, based on individual patient data from randomised controlled trials. BMJ 2001; 323:75–81.
8. Marchant I, Boissel JP, Kassaï B, Bejan T, Massol J, Vidal C, et al. SCORE should be preferred to Framingham to predict cardiovascular death in French population. Eur J Cardiovasc Prev Rehab 2009; 16:609–615.
9. Pencina M, D’Agostino R, Vasan R. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 157–172.
10. Ducimetiere P, Richard JL, Cambien F, Rakotovao R, Claude JR. Coronary heart disease in middle-aged Frenchmen. Comparisons between Paris Prospective Study, Seven Countries Study, and Pooling Project. Lancet 1980; 1:1346–1350.
© 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins