In the general population, cardiovascular disease (CVD) risk can be predicted from the measurement of a variety of established risk factors with direct and synergistic effects. Therefore, the assessment of the precise impact of individual risk factors is complex, and several risk scoring systems have been developed to assess the absolute CVD risk of an individual.
Patients with renal failure have increased CVD risk, the reasons for which are still incompletely understood. Renal transplant recipients (RTRs) have lower risk for CVD than patients who remain on the transplant waiting list, but when compared with the general population, CVD risk is increased in RTRs (1). In addition to the uremic milieu, the cardiovascular system of RTRs is subjected to the metabolic adverse effects of immunosuppressive medications, and perhaps, as a consequence of this, some traditional CVD risk factors seem to have different impact in RTRs compared with the general population (2).
Because of the absence of a CVD risk calculator designed specifically for RTRs, the risk scoring method based on Framingham equation has been used widely also in RTRs (3–9). However, care needs to be taken in the application of CVD risk calculators to populations that differ from those in which they were derived. Any CVD risk scoring system needs to be thoroughly evaluated against epidemiological data before it is introduced and updated in line with changing trends in risk factors.
Previous attempts to predict CVD and mortality risk in RTRs have been limited to administrative and clinical data sets (10–12). Using the Assessment of Lescol in Renal Transplantation (ALERT) extension trial data, we have constructed and internally validated a seven-variable CVD risk equation and a six-variable mortality risk equation for prevalent RTRs.
The factors associated with CVD and mortality in the ALERT study have previously been described (13). The current analysis included the prolonged follow-up. The mean follow-up from the time of entry to the ALERT study to the last study visit in the extended study was 6.7 years. The baseline characteristics of the assessment and test population are given in Table 1.
Major Adverse Cardiac Events
Of the 2102 patients in the ALERT trial, 1401 were randomly selected to assessment population. Because of missing baseline values, 392 patients were excluded, leaving 1009 patients for candidate variable selection. There were 165 events in that group (11.8%).
The candidate variables that were not forced into the regression analysis were subjected to backward Cox regression analysis of time to major adverse cardiac event (MACE)—defined as cardiac death, nonfatal myocardial infarction, or coronary revascularization procedure. This resulted in selection of seven variables. Statistical information on these variables is given in Table 2. Once the seven variables were selected, the number of missing values was reduced, and the number of patients included in the analyses could be increased to 1329. Study treatment allocation was excluded as a variable because there was no treatment–risk factor interaction.
The remaining 701 patients served as a test sample. Because of missing baseline values, 33 patients were excluded, leaving 668 patients for the current analyses. In this group, there were 82 events (11.7%).
To estimate the probability of MACE, the coefficients from the Cox regression model with the seven factors were used to calculate the prognostic index for each patient. The underlying probability of 7-year MACE-free survival was 0.856. The formulas are given in the Appendix with a calculation example. A Web page to calculate the risk of MACE in an individual patient is available (see Cardiovascular Risk Calculator for Renal Transplant Recipients, http://www.anst.uu.se/insov254/calculator/).
Calibration was assessed graphically by plotting the observed events in the test sample against predicted events in risk deciles (Fig. 1A). Because there was a 22% treatment effect in the ALERT trial, the calculated MACE was reduced by 11% when compared with the observed number of events in the test sample. Linear regression in the calibrated calculated MACE risk deciles in the test sample produced an R2 of 0.76, and the regression equation was y=0.40+0.75×calculated MACE. The deviations from the line of identity occurred in those at the highest and the lowest risks. The Hosmer-Lemeshow (HL) test yielded a chi-square value of 11.47 with a df of 8 (P=0.245) (i.e., calibration was good).
The discrimination of the model was good with an area under the receiver operating characteristic (ROC) curve of 0.738 in the assessment sample (Fig. 2A); in the test sample, the area under the ROC curve was identical (0.740).
Because of missing baseline data for 313 patients, the development sample for mortality was limited to 1088 patients, in which there was 204 (18.7%) deaths.
The candidate variables that were not forced into the regression were subjected to backward Cox regression analysis of time to death. This resulted in selection of six variables; statistical information of these variables is given in Table 3. Once the six variables were selected, the number of missing values was reduced, and the number of patients included in the analyses could be increased to 1347.
In the test sample, there were 25 patients with missing baseline values, leaving 676 patients for the regression analyses. Of these patients, 110 (15.7%) died during the follow-up.
To estimate the probability of death, the coefficients from the Cox regression model with the six factors were used so that the prognostic index per patient could be calculated. The underlying probability of 7-year survival was 0.863. The formulas are given in the Appendix with a calculation example, and the Web page for mortality risk calculation in an individual patient is available (see Cardiovascular Risk Calculator for Renal Transplant Recipients, http://www.anst.uu.se/insov254/calculator/).
In the ALERT extension trial, fluvastatin treatment showed no effect on total mortality, and therefore, it was not included in the calibration of the model using the test sample. Calibration was assessed graphically by plotting the observed events in the test sample against predicted events in risk deciles (Fig. 1B). Linear regression in the calculated mortality risk deciles in the test sample was associated with an R2 of 0.876, and the regression equation was y=1.315+0.958×calculated death. The HL test yielded a chi-square value of 13.08 with a df of 8 (P=0.109) (i.e., calibration was acceptable).
The discrimination of the model was good with an area under the ROC curve of 0.734 in the assessment sample (Fig. 2B). In the test sample, the area under the ROC curve was 0.720.
Using data from the ALERT trial, we have constructed and internally validated CVD and mortality risk equations for prevalent RTRs. The need for a risk calculator specifically tailored for RTRs is clear because of the high incidence of CVD in these patients. To our knowledge, the calculator is the first CVD and mortality risk assessment tool in RTRs, which has been developed using a closely monitored clinical trial population.
Current recommendations on the prevention of CVD in clinical practice stress the need to base intervention on an assessment of the individual’s total burden of risk rather than on the level of any particular risk factor (14). Several authors have previously used the Framingham score for absolute CVD risk prediction in RTRs, mostly when comparing the effects of different immunosuppressants (3–9). The consensus is that the Framingham risk equation is not adequate to predict CVD risk in this population (15). The Framingham risk score is particularly imprecise in those RTRs with high risk (2, 16–17). Moreover, the Pocock calculator that incorporates baseline prevalent CVD has been shown to underestimate CVD rates in RTRs (18). In addition to difficulty in managing CVD risk in individual patients, imprecise CVD risk prediction in RTRs may lead to inadequate power calculations in clinical trials, produce false-negative results, and subsequently lead to erroneous conclusions.
Using data from a multicenter clinical trial has advantages and disadvantages when developing a risk prediction equation. The number of events is sufficient, and most importantly, events are reported accurately and independently validated, and the registration of risk factors is thorough. However, high-risk patients may have been excluded because of study inclusion criteria, limiting the generalizability of the analysis, and RTRs are also not a homogenous population when it comes to CVD risk. Because the calculator was derived from the ALERT trial population, a transplant population with moderate CVD risk, care needs to be taken when applying the prediction equation on risk extremes. Figure 1(A) illustrates this issue, showing deviation from the line of identity in low- and high-risk patients.
It is likely that residual factors not included in the model play a role in CVD development in RTRs. CVD risk prediction is an ongoing work because our understanding of the pathogenesis of CVD in patients with renal disease, including RTRs, increases. New risk markers are constantly being proposed and may be incorporated into risk prediction algorithms. However, it has become apparent from general population studies that, for models possessing reasonably good discrimination, very large independent association of the new marker with the outcome is required to result in a meaningfully larger area under the ROC curve. Therefore, when studying the added predictive ability of new markers, future studies in RTRs should preferably investigate even model performance measures beyond c statistics, such as net reclassification improvement or integrated discrimination improvement approach (19–21).
When calculating the risk of mortality in the ALERT population, it was noted that, whereas current smoking was associated with clearly increased risk, the mortality risk associated with previous smoking was almost identical to that of never having smoked. Thus, the calculator could prove to be a valuable educational tool when motivating a patient to stop smoking.
To appreciate the findings, some limitations need to be addressed. The prediction model has been validated internally for calibration and discrimination in the ALERT data set. Although internal validation is helpful, it cannot provide information about the model’s performance elsewhere (22). It is therefore crucial to quantify the performance of the model on a new series of patients, ideally in different locations and on different immunosuppressive therapy protocols. We have searched the Multinational Observational Study in Transplantation (MOST) database for suitability as external validation population but found that several risk factors were missing for a large proportion of patients. To our knowledge, there is an ongoing observational prospective cohort study in Spain, which may prove suitable for external validation in the future (23).
It also needs to be considered that the current prediction model has not been tested on data from pretransplantation evaluation and in the early posttransplantation period. The patients were included to the ALERT trial more than 6 months after transplantation.
In conclusion, a formula for CVD and mortality risk calculation for prevalent RTRs has been developed and internally validated using closely monitored data from a clinical trial. Accurate risk prediction is important for physician decision support, quality of care assessment, and patient education. However, the assessment of usefulness of the current model requires clinical judgment and depends on context. To continue CVD risk calculator development and validation in RTRs, international collaboration is mandatory.
MATERIALS AND METHODS
The ALERT trial was an investigator-initiated and investigator-led, randomized, double-blind, parallel group study designed to investigate the effects of fluvastatin on cardiac and renal endpoints in RTRs. The ALERT core study and the ALERT extension study design, baseline data, and outcomes have been published previously (24–26).
Briefly, in the ALERT core trial, 2102 adult RTRs were recruited from nephrology and transplant clinics in Belgium, Denmark, Finland, Norway, Sweden, Switzerland, United Kingdom, and Canada between June 1996 and October 1997. The patients had received renal or combined renal and pancreas transplants more than 6 months before randomization. All patients were on cyclosporine-based immunosuppression, but no one received statins before inclusion. The total fasting cholesterol levels ranged from 4 to 9 mM (4–7 mM for those with previous cardiac event). Patients who had experienced an acute rejection episode in the last 3 months or who presented with a predicted life expectancy of less than 1 year were excluded. In addition, patients with unstable angina or hospital-verified myocardial infarction less than 6 months before randomization were excluded. Of the patients, 10% showed family history of premature CVD, 7% experienced angina pectoris, 3% had previously experienced myocardial infarction, and 3% had experienced stroke. The causes of renal failure were glomerulonephritis (35%), polycystic kidney disease (15%), diabetic nephropathy (13%), pyelonephritis/interstitial nephritis (12%), hypertension/nephrosclerosis (5%), vasculitis/systemic lupus erythematosus (2%), unknown (5%), and other (13%).
Before the conclusion of the ALERT core study, a 2-year extension was planned as a formal amendment to the study protocol. All patients who completed the ALERT study were eligible to enter the extended follow-up period, during which all participants were offered fluvastatin treatment. Patients who did not wish to receive study medication during the extension but permitted collection of data were included as follow-up patients. In addition, follow-up data were obtained retrospectively for patients who chose not to participate in active follow-up in the extension phase but who gave consent for data collection at the end of the extension study. Data that were collected retrospectively included the occurrence of cardiac events and graft loss or death during the 2 years since the patient’s last visit in the core study. The Critical Events Committee consisted of two nephrologists and two cardiologists who were unaware of treatment assignment. All endpoints were adjudicated by the Critical Events Committee and classified after agreement by consensus (25). The study adhered to the International Conference on Harmonisation guidelines and for good clinical practice and was performed in accordance with the Declaration of Helsinki. All participants provided written informed consent, and the ethics committee at each participating center approved the trial.
The risk calculator was constructed for the study primary endpoint, MACE. A calculator was also constructed and validated for total mortality.
The total study population was randomly divided into assessment (67% of the study population) and test (33% of the study population) samples. All analyses were performed using SPSS version 18.0 (SPSS Inc., Chicago, IL).
In the assessment sample, potential prognostic factors were selected on the basis of clinical judgment and include age; previous coronary heart disease; smoking; baseline serum creatinine level; diabetes mellitus (including posttransplantation diabetes); baseline plasma low- and high-density lipoprotein and triglyceride; gender; total time on renal replacement therapy; treatment of acute rejection episodes; systolic blood pressure; body mass index; baseline serum calcium, phosphate, parathyroid hormone, glucose, interleukin 6, and C-reactive protein; and number of transplants. These factors, registered at the time of inclusion in the study, were then subject to a backward stepwise Cox regression likelihood ratio elimination procedure. Age, smoking, diabetes mellitus, history of coronary heart disease, and serum creatinine, which are known to be significant risk factors, were forced into the regression analysis (13, 27). To avoid inclusion of too many false significant risk factors in the final model, P values for inclusion and exclusion during the procedure were both required to be less than 0.01. C-reactive protein and triglycerides were best modeled by using the natural logarithm.
We then calculated both the prognostic index (sum of products of the regression coefficient and centralized risk factor level) and probability of survival per patient. In that way, the underlying probability of surviving can be calculated for a given period in an individual patient and will not vary by patient. More details about actual calculations can be found in the Appendix.
Calibration refers to the ability of a model to align the number of estimated and observed events across the entire risk range. The calibration of the final model was performed internally in the test sample that comprised 33% of the overall data set. For each patient in the test sample, the regression coefficients from the assessment sample were multiplied with the factor level centralized around its mean and added together across the factors. The expected number of events and “nonevents” (event-free survival) for each decile of the distribution was estimated by calculating the sum of the probabilities of events and nonevents across all individuals within each decile. This number was then compared with the observed number of events per group. The HL chi-square test with a df of 8 was used to compare the calculated with the observed number of events in these groups. This should not exceed 16.9 if the model fit is acceptable. A correlation plot was made to illustrate the degree of association between calculated and observed number of events.
The discriminatory ability of the model was assessed by calculating the standard ROC curve. The c statistics ranges from 0.5 (no predictive ability) to 1 (perfect discrimination) and is based on a comparison of the ranks of the predicted probabilities in individuals with and without the chosen outcome, being the ratio of the probability of predicting an event in those with an event to those without an event.
The authors thank Novartis for access to the ALERT trial data. Novartis also made it possible to get access to the MOST database and made a financial contribution covering the costs of the MOST database programming. The authors thank Mr. Villu Jürimaa for his assistance in creating the Web page.
1. Sarnak MJ, Levey AS, Schoolwerth AC, et al.. Kidney disease as a risk factor for development of cardiovascular disease: a statement from the American Heart Association Councils on Kidney in Cardiovascular Disease, High Blood Pressure Research, Clinical Cardiology, and Epidemiology and Prevention. Circulation 2003; 108: 2154.
2. Kasiske BL, Chakkera HA, Roel J. Explained and unexplained ischemic heart disease risk after renal transplantation. J Am Soc Nephrol 2000; 11: 1735.
3. Artz MA, Boots JM, Ligtenberg G, et al.. Improved cardiovascular risk profile and renal function in renal transplant patients after randomized conversion from cyclosporine to tacrolimus. J Am Soc Nephrol 2003; 14: 1880.
4. Rike AH, Mogilishetty G, Alloway RR, et al.. Cardiovascular risk, cardiovascular events
, and metabolic syndrome in renal transplantation: comparison of early steroid withdrawal and chronic steroids. Clin Transplant 2008; 22: 229.
5. Rogers CC, Alloway RR, Boardman R, et al.. Global cardiovascular risk under early corticosteroid cessation decreases progressively in the first year following renal transplantation. Transplant Proc 2005; 37: 812.
6. Woodle ES. A prospective, randomized, multicenter, double-blind study of early corticosteroid cessation versus long-term maintenance of corticosteroid therapy with tacrolimus and mycophenolate mofetil in primary renal transplant recipients
: one year report. Transplant Proc 2005; 37: 804.
7. Kramer BK, Zulke C, Kammerl MC, et al.. Cardiovascular risk factors and estimated risk for CAD in a randomized trial comparing calcineurin inhibitors in renal transplantation. Am J Transplant 2003; 3: 982.
8. Marcen R, Chahin J, Alarcon A, et al.. Conversion from cyclosporine microemulsion to tacrolimus in stable kidney transplant patients with hypercholesterolemia is related to an improvement in cardiovascular risk profile: a prospective study. Transplant Proc 2006; 38: 2427.
9. Blum CB. Effects of sirolimus on lipids in renal allograft recipients: an analysis using the Framingham risk model. Am J Transplant 2002; 2: 551.
10. Hernandez D, Sanchez-Fructuoso A, Gonzalez-Posada JM, et al.. A novel risk score for mortality
in renal transplant recipients
beyond the first posttransplant year. Transplantation 2009; 88: 803.
11. Israni AK, Snyder JJ, Skeans MA, et al.. Predicting coronary heart disease after kidney transplantation: Patient Outcomes in Renal Transplantation (PORT) Study. Am J Transplant 2010; 10: 338.
12. Hernandez D, Rufino M, Bartolomei S, et al.. A novel prognostic index for mortality
in renal transplant recipients
after hospitalization. Transplantation 2005; 79: 337.
13. Jardine AG, Fellström B, Logan JO, et al.. Cardiovascular risk and renal transplantation: post hoc analyses of the Assessment of Lescol in Renal Transplantation (ALERT) study. Am J Kidney Dis 2005; 46: 529.
14. Conroy RM, Pyorala K, Fitzgerald AP, et al.. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003; 24: 987.
15. Silver SA, Huang M, Nash MM, et al.. Framingham risk score and novel cardiovascular risk factors underpredict major adverse cardiac events in kidney transplant recipients. Transplantation 2011; 92: 183.
16. Ducloux D, Kazory A, Chalopin JM. Predicting coronary heart disease in renal transplant recipients
: a prospective study. Kidney Int 2004; 66: 441.
17. Kiberd B, Panek R. Cardiovascular outcomes in the outpatient kidney transplant clinic: the Framingham risk score revisited. Clin J Am Soc Nephrol 2008; 3: 822.
18. Kiberd B, Keough-Ryan T, Panek R. Cardiovascular disease reduction in the outpatient kidney transplant clinic. Am J Transplant 2003; 3: 1393.
19. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, et al.. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 2008; 27: 157.
20. Zethelius B, Berglund L, Sundstrom J, et al.. Use of multiple biomarkers to improve the prediction of death from cardiovascular causes. N Engl J Med 2008; 358: 2107.
21. Soveri I, Arnlov J, Berglund L, et al.. Kidney function and discrimination of cardiovascular risk in middle-aged men. J Intern Med 2009; 266: 406.
22. Altman DG, Vergouwe Y, Royston P, et al.. Prognosis and prognostic research: validating a prognostic model. BMJ 2009; 338: b605.
23. Pita-Fernandez S, Pertega-Diaz S, Valdes-Canedo F, et al.. Incidence of cardiovascular events
after kidney transplantation and cardiovascular risk scores: study protocol. BMC Cardiovasc Disord 2011; 11: 2.
24. Holdaas H, Fellstrom B, Holme I, et al.. Effects of fluvastatin on cardiac events in renal transplant patients: ALERT (Assessment of Lescol in Renal Transplantation) study design and baseline data. J Cardiovasc Risk 2001; 8: 63.
25. Holdaas H, Fellstrom B, Jardine AG, et al.. Effect of fluvastatin on cardiac outcomes in renal transplant recipients
: a multicentre, randomised, placebo-controlled trial. Lancet 2003; 361: 2024.
26. Holdaas H, Fellstrom B, Cole E, et al.. Long-term cardiac outcomes in renal transplant recipients
receiving fluvastatin: the ALERT extension study. Am J Transplant 2005; 5: 2929.
27. Soveri I, Holdaas H, Jardine A, et al.. Renal transplant dysfunction—importance quantified in comparison with traditional risk factors for cardiovascular disease and mortality
. Nephrol Dial Transplant 2006; 21: 2282.
APPENDIX Risk Estimation From the Cox Model and Calculation Example
We have selected a 50-year-old patient for practicality. In Table 4, the levels of the risk factors are given. First, we have to calculate the centralized prognostic index (CPI): CPI=Σβi×Xi−Σβi×ũi, where βi is the regression coefficient, Xi is the level of risk factor i of a patient, and ũi is the corresponding average value in the ALERT assessment population. For the prognostic factors, estimates of βi are given in Tables 2 and 3 for MACE and mortality, respectively. The underlying probability of surviving the given period t, So(t), can be calculated as a constant independent of the risk factor profile. In this database, So(t)=0.856 for MACE and So(t)=0.863 for death at t=7 years. The survival probability for any patient with a given risk factor set Xi0 is then given by the formula:
where exp is the exponential function.
MACE-Free Survival Calculation
The CPI for this patient becomes
Thus, the 7-year MACE-free survival probability for this patient is calculated to be
Survival probability for the same patient can be calculated:
Thus, the 7-year survival probability for this patient is calculated to be
Risk in an individual patient can be calculated using the calculator available at: http://www.anst.uu.se/insov254/calculator/.