CKD affects more than 10% of the population globally and is strongly associated with cardiovascular disease (CVD) morbidity and mortality.12–3 Therefore, individuals with CKD are typically assumed to be at high risk for atherosclerotic CVD (ASCVD).2 However, the accurate classification of ASCVD risk among individuals with CKD is still controversial. The Kidney Disease: Improving Global Outcomes (KDIGO) 2013 lipid guideline considers CKD patients aged 50 years or older as high risk regardless of CKD severity or the status of traditional CVD risk factors.4 The European Society of Cardiology guideline categorizes eGFR 30–59 ml/min per 1.73 m2 as high risk, and eGFR <30 ml/min per 1.73 m2 or patients with diabetes and albuminuria as very high risk.5 The 2013 American Heart Association (AHA) and American College of Cardiology (ACC) guideline did not include CKD measures in ASCVD prediction using the pooled cohort equations (PCEs).6
Recent primary prevention guidelines classify CKD as a risk-enhancing factor.7 Although the KDIGO guideline recommends predicting ASCVD risk in CKD patients younger than 50 years old, there are few established prediction tools developed for this clinical population.8 Overall, inconsistencies in major clinical guidelines are mainly due to inconsistent previous findings.91011–12 The Framingham equations and the ACC/AHA PCEs may inadequately predict ASCVD risk in individuals with CKD.10,13 Furthermore, it is unclear whether incorporating additional novel risk factors will improve risk prediction for ASCVD.14,15
The availability of accurate ASCVD risk prediction equations for individuals with CKD could be informative for patients and clinicians and improve shared decision making for preventive therapy to reduce ASCVD incidence and mortality.16 Therefore, we developed novel ASCVD risk prediction models for use among patients with CKD using data from the Chronic Renal Insufficiency Cohort (CRIC) study.
Study Design and Participants
The CRIC study is a prospective, longitudinal cohort study of participants with mild-to-moderate CKD based on an eGFR entry criterion of 20–70 ml/min per 1.73 m2. A racially and ethnically diverse group of 3939 men and women aged 21–74 years was enrolled from seven clinical centers in the United States (Ann Arbor, MI; Baltimore, MD; Chicago, IL; Cleveland, OH; New Orleans, LA; Philadelphia, PA; and San Francisco, CA) in phase I, between May 2003 and August 2008.17 Patients with New York Heart Association class 3 and 4 heart failure, cirrhosis, HIV infection, polycystic kidney disease, renal cell carcinoma, those receiving chronic dialysis or an organ transplant, and those taking immunosuppressive medications were excluded from the CRIC study. For the current analysis, we excluded those with a self-reported history of CVD at baseline (n=1335), yielding 2604 participants. The study was approved by the institutional review board at each institution and all participants provided written informed consent.
Candidate Predictor Variables
A more detailed description of the methods is available in the Supplemental Methods. Candidate predictor variables considered for inclusion in the prediction models are displayed in Supplemental Figure 1. Candidate variables included those in the ACC/AHA PCEs6: age, sex, race and ethnicity, total cholesterol, HDL cholesterol, systolic BP, use of BP-lowering medications, history of diabetes, and current smoking. Additional candidate variables were chosen based on associations with ASCVD among patients with CKD identified in prior reports, and included metabolic factors, kidney disease, lipid metabolism, mineral metabolism, and inflammation factors, and cardiac biomarkers. Because self-reported race is a controversial and ill-defined social construct,18 we used the CKD Epidemiology Collaboration cystatin C equation as a measure of eGFR, which does not include a race adjustment factor.19 We considered two sets of variables for development of the prediction models: one set including only readily clinically available variables (Supplemental Figure 1; black text) and another set containing all candidate predictor variables, including biomarkers that are less likely to be measured in routine clinical practice (Supplemental Figure 1; blue text). In a secondary analysis, we evaluated the performance of the “CKD patch,” which incorporates eGFR and urinary albumin-creatinine ratio (ACR) measurements into the ACC/AHA PCEs. In brief, the CKD patch equations were applied to the PCEs recalibrated to the CRIC sample, with expected eGFR and urinary ACR centered at CRIC study-specific averages, as recommended.8
Natural logarithm and nonlinear (e.g., polynomial, square root) transformations of continuous variables were evaluated and retained if they improved model fit. Potential interactions between all candidate predictors were assessed. However, the inclusion of interaction terms was consistently associated with poorer cross-validated model performance and they were not retained in the final models. We could not evaluate some potentially important variables (e.g., coronary artery calcium, pulse-wave velocity, and electrocardiogram variables) because these variables were not available among the majority of CRIC participants. Missing data for other candidate predictors was <5% for all variables except homocysteine (11.3%) and neutrophil gelatinase-associated lipocalin ( 5.7%). Missing values were imputed using a nearest neighbors model.20
The primary end point for the prediction models was incident ASCVD: a composite of incident fatal or nonfatal stroke or myocardial infarction.6 Events are ascertained every 6 months from medical records, and diagnosis was confirmed by two blinded reviewers as possible, probable, or definite events.21 Follow-up time was censored at the earliest occurrence of death, loss to follow-up (n=203; 7.8%), or 10 years of complete follow-up, consistent with prior models developed for ASCVD.6 All deaths were confirmed by death certificate.
A priori, we evaluated the ACC/AHA PCEs in the CRIC sample using two approaches: (1) predicted risk from the published model coefficients6; and (2) variables included in the ACC/AHA PCEs with coefficients recalculated using the CRIC sample. Furthermore, we evaluated several modeling algorithms and selection procedures to develop novel models. Our primary approach used Cox proportional hazards regression models to predict the probability of having an incident ASCVD event within 10 years of baseline.6 Additionally, we used the Fine and Gray subdistribution hazards model to account for the competing risk of death.22 Within both modeling algorithms, we evaluated three predictor selection procedures: (1) backward elimination with a P<0.05 criterion; (2) backward elimination with an Akaike information criterion; and (3) LASSO (“least absolute shrinkage and selection operator”) method with a regularization penalty (λ) one SEM higher than that which minimized cross-validated error.23
All modeling strategies were internally validated using 10 × 10-fold cross-validation, similar to the derivation procedure used to develop the ACC/AHA PCEs.6 The best-performing (see “Performance Metrics” section) modeling procedure was used on the full CRIC sample to develop final clinical and biomarker-enriched models. In a sensitivity analysis, we evaluated the best-performing modeling procedure from the 10 × 10-fold cross-validation using an internal-external validation approach. In this approach, one clinical center is left out at a time to cross-validate models developed in the other centers, which provides additional information on the potential performance of the models in external datasets similar in composition to the CRIC study sample.24
We validated our models externally using data from two independent, community-based prospective cohort studies: the Atherosclerosis Risk in Communities (ARIC) study25 and the Multi-Ethnic Study of Atherosclerosis (MESA).26 We pooled individual-level data among 2222 participants with CKD, based on an eGFR <60 ml/min per 1.73 m2 or urinary ACR ≥30 mg/g. The equations developed in the CRIC study were applied directly to the external validation data, and model performance was assessed.
We evaluated the performance of developed models using measures of discrimination, calibration, and overall goodness of fit.27 Discrimination refers to the ability of a model to correctly identify those who will or will not experience the outcome and was assessed using the time-dependent (i.e., 10-year) area under the receiver operating characteristic curve (AUC).28 Calibration refers to the agreement between observed outcomes and model predictions, and was assessed by plotting the observed versus predicted risk across deciles of predicted risk. Observed risk was estimated using the cumulative incidence function accounting for the competing risk of death. The scaled Brier score, or index of prediction accuracy (IPA), incorporates information on both discrimination and calibration, and was used to evaluate overall goodness of fit.29 A higher IPA indicates a better performing model, an IPA of 1.0 indicates a perfect model, and an IPA ≤0 indicates a useless model. Performance results from the 10 × 10-fold cross-validation were summarized as the mean for point estimates and the 2.5th and 97.5th percentiles for 95% confidence intervals (CIs). We used the categorical net reclassification improvement (NRI) statistic to evaluate the clinical utility of the models,30,31 using predicted risk thresholds used in current clinical practice guidelines (i.e., 7.5% and 20% per the ACC/AHA primary prevention guideline).7 Bootstrapping with 1000 replicates was used to generate 95% CIs for the NRI.
Analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC) and R version 4.1.1 (R Project for Statistical Computing). Two-sided P values <0.05 were considered statistically significant for all analyses.
Among 2604 participants included in the analyses, the mean (SD) age was 55.8 (11.7) years, 52.0% were male, 41.6% had diabetes, and the mean (SD) eGFR was 56.0 (24.7) ml/min per 1.73 m2. During the first 10 years of follow-up from baseline, 252 incident ASCVD events occurred (11.9 events per 1000 person-years). Participants who had an ASCVD event within 10 years were more likely to be older, Black race, currently smoke, have a history of diabetes, and use BP-lowering medications compared with those who did not have an ASCVD event (Table 1). On average, those who had an ASCVD event had higher levels of glycated hemoglobin (HbA1c), systolic and diastolic BP, urinary ACR, troponin-T, and N-terminal prohormone of brain natriuretic peptide (NT-proBNP); and lower levels of HDL cholesterol, eGFR, and hemoglobin compared with those who did not have an event. Within 10 years, 411 participants died from noncardiovascular causes.
Table 1. -
Baseline characteristics of CRIC study participants
||ASCVD Event within 10 Years
|Age, mean (SD), yr
|Male, No. (%)
|Black race, No. (%)
|Body mass index, mean (SD), kg/m2
|Current smoking, No. (%)
|HbA1c, mean (SD), %
|Diabetes mellitus, No. (%)
|Use of glucose-lowering medications, No. (%)
|Systolic BP, mean (SD), mm Hg
|Diastolic BP, mean (SD), mm Hg
|Use of BP-lowering medications, No. (%)
|Total cholesterol, mean (SD), mg/dl
|LDL cholesterol, mean (SD), mg/dl
|HDL cholesterol, mean (SD), mg/dl
|Apolipoprotein B, mean (SD), mg/dl
|Use of statin medications, No. (%)
|eGFR, mean (SD), ml/min per 1.73 m2
|Urine ACR, median [IQR], mg/g
||39.1 [7.3, 391.2]
||34.7 [6.7, 344.7]
||224.0 [18.3, 1062.3]
|Hemoglobin, mean (SD), g/dl
|hsCRP, median [IQR], mg/L
||2.4 [1.0, 6.1]
||2.3 [1.0, 6.0]
||3.0 [1.2, 7.9]
|Troponin-T, median [IQR], pg/ml
||9.7 [4.5, 18.9]
||9.3 [4.1, 17.4]
||16.0 [8.9, 31.6]
|NT-proBNP, median [IQR], pg/ml
||106.5 [49.3, 249.7]
||99.3 [46.5, 229.5]
||207.9 [95.7, 457.5]
IQR, interquartile range.
Model Development and Internal Validation
The results from the 10 × 10-fold internal cross-validation are presented in Supplemental Table 1. Across all candidate variables and selection procedures, the Cox proportional hazards regression models obtained higher estimates of AUC and IPA than the Fine and Gray subdistribution hazards models, whereas the Fine and Gray model improved calibration in the highest predicted probability decile (Supplemental Figure 2). According to the 10 × 10-fold cross-validation, Cox proportional hazards regression with backward elimination using a criterion of P<0.05 had the highest AUC and IPA. In a sensitivity analysis using internal-external validation leaving one CRIC clinical center out at a time, prediction performance was similar to 10 × 10-fold cross-validation, although AUCs were slightly higher and IPAs were slightly lower (Supplemental Table 2).
Model Characteristics and Risk Calculations
Table 2 provides the hazard ratios (HRs) with 95% CIs, β coefficients, and information for calculating predicted 10-year risk of ASCVD for the final models. Model 1 included the ACC/AHA PCEs variables with coefficients recalculated in the CRIC study sample. Model 2 (CRIC clinical model) included age, HDL cholesterol, systolic BP, current smoking, urinary ACR, HbA1c, and hemoglobin. Model 3 (CRIC enriched model) included age, total cholesterol, HDL cholesterol, current smoking, urinary ACR, HbA1c, apolipoprotein B, high-sensitivity C-reactive protein (hsCRP), troponin-T, and NT-proBNP. Example predicted risk calculations using each of the three models are available in Supplemental Tables 3–5.
Table 2. -
Novel CKD ASCVD risk prediction models developed in the CRIC study
||Model 1: ACC/AHA PCEs Variables
||Model 2: CRIC Clinical Model
||Model 3: CRIC Enriched Model
||HR (95% CI)
β, per 1-unit
||HR (95% CI)
β, per 1-unit
||HR (95% CI)
β, per 1-unit
| Age, per 5 yr
||1.15 (1.09 to 1.21)
||1.23 (1.16 to 1.31)
||1.19 (1.12 to 1.26)
| Male sex
||1.09 (0.87 to 1.38)
| Black race
||1.21 (0.96 to 1.51)
| Total cholesterol, per 10 mg/dl
||1.02 (1.00 to 1.04)
||1.08 (1.01 to 1.15)
| HDL cholesterol, per 10 mg/dl
||0.87 (0.80 to 0.95)
||0.88 (0.81 to 0.95)
||0.85 (0.77 to 0.94)
| Systolic BP, per 10 mm Hg
||1.13 (1.08 to 1.19)
||1.08 (1.02 to 1.14)
| Use of BP-lowering medications
||1.54 (0.97 to 2.46)
| History of diabetes
||1.88 (1.49 to 2.37)
| Current smoking
||2.09 (1.57 to 2.78)
||2.03 (1.53 to 2.69)
||1.94 (1.46 to 2.57)
| Log (urine ACR), mg/g
||1.17 (1.11 to 1.23)
||1.12 (1.05 to 1.19)
| HbA1c, per 0.5%
||1.09 (1.05 to 1.12)
||1.08 (1.05 to 1.12)
| Hemoglobin, per 5 g/dl
||0.64 (0.46 to 0.88)
| Apolipoprotein B, per 10 mg/dl
||0.89 (0.79 to 0.99)
| Log (hsCRP), mg/L
||1.11 (1.02 to 1.21)
| Log (troponin-T), pg/ml
||1.23 (1.09 to 1.39)
| Log (NT-proBNP), pg/ml
||1.22 (1.11 to 1.35)
| Baseline survival at 10 years, S(10)
| Predicted probability of ASCVD at 10 years
aThe predicted 10-year probability for each model can be calculated as 1– S(10)exp(ΣbX–betaavg) where b is the regression coefficient (β), X is the individual patient’s level for each risk factor, and betaavg is the sample mean of the β×value sum.
Relative Variable Importance
Figure 1 shows the relative importance of variables included in models 2 and 3. In both models, age accounted for the greatest proportion of the overall model chi-squared. The top five predictors in model 2 were age, urinary ACR, current smoking, HbA1c, and HDL cholesterol. The top five predictors in model 3 were age, HbA1c, current smoking, NT-proBNP, and urinary ACR.
Discrimination and Calibration
All models were well-calibrated, but each overestimated risk within the highest decile of predicted risk (Figure 2 and Supplemental Figure 3). The models developed in the CRIC data were better calibrated across the range of predicted risks compared with the published ACC/AHA PCEs. Applying the published ACC/AHA PCEs coefficients to the CRIC sample yielded an AUC of 0.730 and IPA of 0.058. Model 1 including the ACC/AHA PCEs variables recalculated in the CRIC sample achieved an AUC of 0.736 (95% CI, 0.649 to 0.826) and IPA of 0.072 (95% CI, −0.016 to 0.137), which reflect improved performance compared with the ACC/AHA PCEs using published coefficients (Table 3). Model 2 (AUC, 0.760; 95% CI, 0.678 to 0.851; and IPA, 0.095; 95% CI, 0.013 to 0.173) and model 3 (AUC, 0.771; 95% CI, 0.674 to 0.853; and IPA, 0.105; 95% CI, 0.033 to 0.168) both further improved prediction performance. Model 3 performed best, and was significantly better than model 2 in terms of AUC (P=0.001) and IPA (P=0.007). Applying the CKD patch to the CRIC study sample in a secondary analysis yielded an AUC of 0.752 and IPA of 0.096 (Supplemental Figure 4). Prediction performance was similar in participants with CKD stage 2–3 versus stage 4 (Supplemental Figure 5).
Table 3. -
Performance characteristics of novel CKD ASCVD risk prediction models developed in the CRIC study
||Model 1: ACC/AHA PCEs Variables
||Model 2: CRIC Clinical Model
||Model 3: CRIC Enriched Model
| AUC (95% CI)
||0.736 (0.649 to 0.826)
||0.760 (0.678 to 0.851)
||0.771 (0.674 to 0.853)
P value comparing with ACC/AHA PCEs
P value comparing successive models
|Overall goodness of fit
| IPA (95% CI)
||0.072 (-0.016 to 0.137)
||0.095 (0.013 to 0.173)
||0.105 (0.033 to 0.168)
P value comparing with ACC/AHA PCEs
P value comparing successive models
aHigher values for AUC and IPA indicate better performing models.
bThe AUC and IPA for the published ACC/AHA PCEs in the CRIC study are 0.730 and 0.058, respectively.
cComparison of model 2 with model 1, and model 3 with model 2.
Among the 2222 participants from the external validation cohorts, 337 ASCVD events occurred during the first 10 years of follow-up, and the observed 10-year risk of ASCVD was higher compared with the CRIC study (15.5% versus 10.5%) (Supplemental Table 6). In the external validation data, the ACC/AHA PCEs had an AUC of 0.702 and IPA of 0.053 (Supplemental Figure 6). When the CRIC equations were applied to the external validation data, discrimination and calibration were adequate and improved upon the ACC/AHA PCEs, achieving AUCs of 0.709 and 0.747, and IPAs of 0.087 and 0.117, for the clinical and enriched models, respectively (Supplemental Figure 6).
Model 1 did not improve reclassification of events nor nonevents compared with the published ACC/AHA PCEs (Table 4). Model 2 significantly improved reclassification of nonevents compared with the published ACC/AHA PCEs (6.6%; 95% CI, 3.7% to 9.6%). Model 3 significantly improved reclassification of nonevents compared with both the published ACC/AHA PCEs (10.0%; 95% CI, 6.8% to 13.3%) and model 2 (3.7%; 95% CI, 1.4% to 6.0%), and was the best-performing model in terms of NRI. The numbers of participants reclassified according to each model are presented in Supplemental Tables 7 and 8.
Table 4. -
Reclassification of events and nonevents in the CRIC study
||Proportion of Participants Correctly Reclassified
|| Net Reclassification Index (95% CI)
|Events Reclassified, % (95% CI)
||Nonevents Reclassified, % (95% CI)
| Comparing with ACC/AHA PCEs
| Model 1 versus ACC/AHA PCEs
||−10.0 (−2.5 to −16.5)
||1.5 (−0.9 to 3.9)
||−8.5 (−1.0 to −15.3)
| Model 2 versus ACC/AHA PCEs
||−4.8 (−12.2 to 2.8)
||6.6 (3.7 to 9.6)
||1.8 (−6.2 to 10.0)
| Model 3 versus ACC/AHA PCEs
||−1.4 (−8.9 to 6.4)
||10.0 (6.8 to 13.3)
||8.6 (0.2 to 16.6)
| Comparing with model 1
| Model 2 versus 1
||6.7 (−0.3 to 13.5)
||5.2 (2.8 to 8.0)
||11.9 (4.6 to 18.9)
| Model 3 versus 1
||11.7 (3.6 to 19.4)
||8.7 (5.9 to 11.7)
||20.4 (12.3 to 28.6)
| Comparing with model 2
| Model 3 versus 2
||5.8 (−0.1 to 11.2)
||3.7 (1.4 to 6.0)
||9.5 (3.3 to 15.3)
aModel 1 indicates the ACC/AHA PCEs variables recalculated in the CRIC study; model 2 indicates the CRIC clinical model; and model 3 indicates the CRIC enriched model.
Models are compared for reclassification of events to a higher predicted risk category and nonevents to a lower predicted risk category, using cut points of 7.5% and 20%. More detailed information can be found in Supplemental Tables 7
Using a sample of 2604 participants with CKD stages 2–4 and free of clinical CVD, we developed and validated a set of predictive models for 10-year risk of ASCVD. We found that the published ACC/AHA PCEs had moderate predictive ability for 10-year risk of ASCVD among participants of the CRIC study. However, novel models developed from a pool of >50 candidate predictors afforded significant prediction improvements. Both a model developed using clinically available variables and a biomarker-enriched model significantly improved discrimination and calibration beyond the ACC/AHA PCEs, and significantly improved reclassification of nonevents. These findings support the use of CKD-specific tools to predict the long-term risk of ASCVD among patients with CKD, who may have different risk profiles compared with the general population, for whom prior models were developed.
Patients with CKD are typically considered to be at high risk for ASCVD, irrespective of other individual patient characteristics or laboratory values.4,5 As such, current clinical practice guidelines do not have specific recommendations among patients with CKD other than to consider kidney function markers, including eGFR and albuminuria, as “risk-enhancing factors.”7 Our findings confirm the value of CKD markers for prediction, with urinary ACR in particular being a strong predictor in both our clinical and enriched models. Notably, eGFR was not included in our final models, suggesting that urinary ACR has a relatively stronger prognostic value for development of ASCVD among patients with CKD compared with eGFR.
We found that several other variables were particularly important for prediction of ASCVD among patients with CKD. In both our clinical and enriched models, age was the most important predictor of ASCVD at 10 years, which is an important consideration in an aging US population.32 Beyond age, we note the inclusion in our models of several important predictors including current smoking, HbA1c, hsCRP, and cardiac biomarkers. Diabetes is highly associated with both CKD and CVD.33 Although diabetes status was included in the ACC/AHA PCEs,6 absolute values of long-term glycemia were not incorporated. Our models suggest the information afforded by the values of a glycemic measure, specifically HbA1c, may be superior for ASCVD risk prediction in CKD populations.
The cardiac biomarkers NT-proBNP and troponin-T showed particularly high relative variable importance in our biomarker-enriched model. These biomarkers are increasingly recognized as being strongly associated with risk for ASCVD among patients with CKD, and were more predictive of CVD than other nontraditional kidney biomarkers among 8622 participants from the ARIC study.14 Both NT-proBNP and troponin-T are circulating proteins that become elevated in response to cardiac stretch and injury.34,35 Although typically associated with heart failure among patients with CKD,36 these biomarkers may also provide information on a greater burden of subclinical cardiac damage that contributes directly to the risk of ASCVD. It should be acknowledged that elevated levels of NT-proBNP or troponin-T may reflect, at least in part, impaired renal clearance among CKD patients, but our findings and the findings of others suggest these biomarkers are valuable for ASCVD risk prediction in this population.14,37 Collection of nontraditional, although widely available, laboratory tests could improve clinical evaluation and management of ASCVD among patients with CKD.
There are several clinical and research implications of our findings. CKD is a heterogeneous condition38 and the ischemic heart disease encountered in CKD often involves distal, small arteries that are not amenable to coronary revascularization.39 Frequently CKD patients undergo invasive diagnostic procedures because they are considered to have high CVD risk. Using a prediction tool better equipped to discern ASCVD events could help reduce the number of unnecessary cardiac angiography procedures and associated morbidity. Other recent advances in this area include the development of a CKD patch to incorporate eGFR and ACR measurements into established risk prediction equations.40 An analysis conducted among approximately 9 million participants included in the CKD Prognosis Consortium identified the utility of the CKD patch in improving performance beyond the ACC/AHA PCEs and the Systematic COronary Risk Evaluation (SCORE) model.8 In the current analysis, the CKD patch improved discrimination of ASCVD events compared with the ACC/AHA PCEs. However, the CKD patch did not consider potential improvements afforded by additional biomarkers, nor models specifically developed among patients with CKD. In the current CRIC study sample, our new models provided more accurate predictions compared with previous risk prediction models, particularly in those with 10-year ASCVD risk <20%. Therefore, our models significantly improved reclassification of nonevents compared with the standard ACC/AHA PCEs. Given the heterogeneity of risk among patients with CKD, it may be warranted to employ prediction models specifically developed for this population. For those with predicted risk of 20% or more, it may be prudent to conduct further evaluation, such as coronary artery calcium scoring.41 In addition, our models could be used in future clinical trials to more accurately recruit high risk patients, allowing for smaller sample sizes and more efficient conduct of trials.
There are several strengths in the current analysis. First, we considered >50 candidate predictor variables, measured using standardized methodology in the CRIC study, which improves accuracy and minimizes bias. Second, ASCVD events are rigorously adjudicated. Third, we have a relatively large population of patients with CKD stages 2–4 which was enrolled based on CKD status, which is an improvement over prior analyses that selected CKD patients from general population cohorts based on one assessment of kidney function. However, there are several limitations. First, although we externally validated our models in two population-based cohort studies, the individuals in these datasets were selected based on only one assessment of kidney function and had a higher baseline risk of ASCVD, on average. Our results suggest adequate discrimination and calibration of our models, but future analyses should externally validate and refine our models among large and diverse populations with CKD and varying baseline risks for ASCVD. Second, these models could not incorporate potentially important imaging variables including coronary artery calcium scores and electrocardiogram data. Finally, our models overestimated risk in the highest predicted probability groups in the CRIC study. However, this finding is common to most CVD prediction models, and the overpredictions occur at probabilities well above clinical decision-making cutoff points.7
In conclusion, we have developed new models for 10-year risk prediction of ASCVD among patients with CKD, including a model composed of readily clinically available variables and a biomarker-enriched model. These models significantly improve prediction performance beyond the ACC/AHA PCEs in this population of patients with CKD stages 2–4, particularly for those with 10-year ASCVD risk <20%. Future analyses should externally validate and refine these models in multiple diverse CKD cohorts.
N.B. Allen reports research funding with AHA and National Institutes of Health (NIH)/National Heart, Lung, and Blood Institute (NHLBI). N. Bansal reports associate editor for Kidney360. J.B. Cohen reports honoraria from UpToDate; and research funding with AHA and NIH/NHLBI and NIH/National Center for Advancing Translational Sciences. R. Deo reports consultancy agreements with Biotronik, Boehringer Ingelheim, Janssen Pharmaceuticals, and Pfizer; research funding with iRhythm Technologies; editorial boards for Circulation and Heart Rhythm O2; and steering committee for Kidney Disease: Improving Global Outcomes. M.A. Dobre reports honoraria from Relypsa Inc. and Tricida; and scientific advisor or membership with Tricida (Metabolic Acidosis Working Group) and Relypsa (Resistant Hypertension Working Group). H.I. Feldman reports consultancy with InMed, Inc., Kyowa Hakko Kirin, National Kidney Foundation, and University of California, Los Angeles; steering committee for the CRIC study; member of advisory board of the National Kidney Foundation; editor-in-chief of American Journal of Kidney Disease. J.P. Lash reports scientific advisor or membership with Kidney360. K. Matsushita reports consultancy agreements with Akebia, Kyowa Hakko Kirin, and Mitsubishi Tanabe; research funding from Kyowa Hakko Kirin; honoraria from Fukuda Denshi; and scientific advisor or membership with American Journal of Kidney Disease, Circulation Reports, and Kidney International. M. Rahman reports consultancy with Barologics; research funding from Bayer Pharmaceuticals and Duke Clinical Research Institute; honoraria from Reata, Relypsa, and Bayer; associate editor for CJASN; and editorial board member for American Journal of Nephrology. S. Seliger reports consultancy agreements with Tricida, Inc (Endpoint Adjudication Committee); research funding from Kadmon Pharmaceuticals, Palladio Biosciences, Reata Pharmaceuticals, Roche Diagnostics, and Sanofi US; patents and inventions with University of Maryland, Baltimore and University of Texas, Southwestern: Methods for assessing differential risk for developing heart failure; associate editor for CJASN; member of medical review board for ESRD Network 5; chair of Board of Directors for ESRD Network 5; member of Endpoint Adjudication Committee and of VALOR CKD trial (Tricida, Inc.); and member of the editorial board for Circulation. T. Shafi reports consultancy agreements with Siemens; research funding as clinical trial site investigator with Baxter, CVS, and Natera; honoraria from Cara Therapeutics, NIH, Siemens, State University of New York Downstate, and University of Virginia; and scientific advisor or membership with American Journal of Medicine, American Journal of Kidney Diseases, CJASN, and Kidney360. M. Wolf reports consultancy agreements with Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, and Walden; ownership interest with Akebia, Unicycive, and Walden; honoraria from Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, and Walden; and scientific advisor or membership with Akebia, Unicycive, and Walden. W. Yang reports statistical editor for American Journal of Kidney Disease. J. Flack reports consultancy agreements with Teva Pharmaceuticals, Amgen, Sanofi, Fibrogen, and Janssen; ownership interest with Novocure, Google, Netflix, Albermarle, Alexandria REIT, Amazon, Ambarella, Arch Cap Group, Chargepoint, Enphase, Intercontinental Exchange, Johnson Controls, Estee Lauder, Lucid, Marvell Technology, Mastercard, Microsoft, Nova, Nvidia, Palantir, Paypal, Plains All American Pipeline, Quantamscape, STMicroelectronics, Reconnaissance Energy, Lightbridge, Advanced MicroDevices, Apollo Commercial Real Estate, Ballard Power, Emerson Electric, Essential Utils, IDT Corp, Meta Platforms, and Neogenomics; research funding from Glaxo Smith Kline, Idorsia, Quantam Genomics, ReCor Medical, Vascular Dynamics, and Rox Medical; honoraria from Amgen, Sanofi, and Teva; and advisory or leadership roles with American College of Physicians Board of Regents, American Journal of Hypertension Associate Editor, and Sanofi. All remaining authors have nothing to disclose.
Funding for the CRIC study was obtained under a cooperative agreement from the National Institute of Diabetes and Digestive and Kidney Diseases (U01DK060990, U01DK060984, U01DK061022, U01DK061021, U01DK061028, U01DK060980, U01DK060963, U01DK060902, and U24DK060990). In addition, this work was supported in part by the Perelman School of Medicine at the University of Pennsylvania Clinical and Translational Science Award, National Center for Advancing Translational Sciences (UL1TR000003), Johns Hopkins University (UL1TR000424), University of Maryland General Clinical Research Center (M01 RR-16500), Clinical and Translational Science Collaborative of Cleveland, the National Center for Advancing Translational Sciences component of the NIH roadmap for Medical Research (UL1TR000439), Michigan Institute for Clinical and Health Research (UL1TR000433), University of Illinois at Chicago Clinical and Translational Science Award (UL1RR029879), Tulane COBRE for Clinical and Translational Research in Cardiometabolic Diseases (P20 GM109036), Kaiser Permanente, National Center for Research Resources (UCSF-CTSI UL1 RR-024131), and Department of Internal Medicine, University of New Mexico School of Medicine Albuquerque, New Mexico (R01DK119199). J.D. Bundy was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development career development grant (K12HD043451). The ARIC study is supported by the National Heart, Lung, and Blood Institute (HHSN268201700001I, HHSN268201700002I, HHSN268201700003I, HHSN268201700004I, and HHSN268201700005I). MESA is supported by the National Heart, Lung, and Blood Institute (75N92020D00001, HHSN268201500003I, N01-HC-95159, 75N92020D00005, N01-HC-95160, 75N92020D00002, N01-HC-95161, 75N92020D00003, N01-HC-95162, 75N92020D00006, N01-HC-95163, 75N92020D00004, N01-HC-95164, 75N92020D00007, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, and N01-HC-95169), and by the National Center for Advancing Translational Sciences (UL1TR000040, UL1TR001079, and UL1TR001420).
The authors thank the participants, investigators, and staff of the CRIC, ARIC, and MESA studies for their time and commitment. The authors thank Ms. Yingying Sang for her technical support.
J.D. Bundy, J. He, K. Matsushita, and M. Rahman conceptualized the study; J. Chen, H.I. Feldman, J. He, J.P. Lash, M. Rahman were responsible for funding acquisition; H.I. Feldman, J. He, J.P. Lash, and M. Rahman were responsible for data curation; J.D. Bundy and B.C. Jaeger were responsible for formal analysis and software; J.D. Bundy, J. He, and B.C. Jaeger were responsible for methodology; J.D. Bundy was responsible for visualization; N. Allen, N. Bansal, and K. Matsushita were responsible for validation; J. He provided supervision; J.D. Bundy wrote the original draft; and all authors reviewed and edited the manuscript.
This article contains the following supplemental material online at http://jasn.asnjournals.org/lookup/suppl/doi:10.1681/ASN.2021060747/-/DCSupplemental.
Supplemental Figure 1. Candidate predictor variables considered for inclusion in prediction models.
Supplemental Figure 2. Calibration slope plots of Cox proportional hazards and Fine and Gray subdistribution hazards modeling algorithms.
Supplemental Figure 3. Calibration of the ACC/AHA PCEs in the CRIC study.
Supplemental Figure 4. Prediction performance of the CKD predictor patch in the CRIC study.
Supplemental Figure 5. Prediction performance of novel CRIC study models by CKD stage.
Supplemental Figure 6. Prediction performance of novel CRIC study models in external validation data.
Supplemental Table 1. Discrimination and IPA for 10-year risk prediction of ASCVD.
Supplemental Table 2. Internal-external validation of 10-year risk prediction of ASCVD in the CRIC study.
Supplemental Table 3. Example calculations of predicted 10-year risk of ASCVD (ACC/AHA PCEs variables).
Supplemental Table 4. Example calculations of predicted 10-year risk of ASCVD (CRIC clinical model).
Supplemental Table 5. Example calculations of predicted 10-year risk of ASCVD (CRIC enriched model).
Supplemental Table 6. Baseline characteristics of development (CRIC study) and external validation (ARIC and MESA) cohort participants.
Supplemental Table 7. Reclassification of events for risk prediction models for ASCVD in CKD.
Supplemental Table 8. Reclassification of nonevents for risk prediction models for ASCVD in CKD.
1. Mills KT, Xu Y, Zhang W, Bundy JD, Chen C-S, Kelly TN, et al.: A systematic analysis of worldwide population-based data on the global burden of chronic kidney disease
in 2010. Kidney Int 88: 950–957, 2015
2. Gansevoort RT, Correa-Rotter R, Hemmelgarn BR, Jafar TH, Heerspink HJL, Mann JF, et al.: Chronic kidney disease
and cardiovascular risk: Epidemiology, mechanisms, and prevention. Lancet 382: 339–352, 2013
3. Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY: Chronic kidney disease
and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 351: 1296–1305, 2004
4. Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work Group: KDIGO 2012 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease
. Kidney Int Suppl 3: 1–150, 2013. Available at: https://kdigo.org/wp-content/uploads/2017/02/KDIGO_2012_CKD_GL.pdf
. Accessed December 10, 2021
5. Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al.; ESC Scientific Document Group: 2016 European Guidelines on cardiovascular disease
prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease
Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts)Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 37: 2315–2381, 2016
6. Goff DC, Lloyd-Jones DM, Bennett G, Coady S, D’Agostino RB, Gibbons R, et al.: 2013 ACC/AHA guideline on the assessment of cardiovascular risk: A report of the American College of Cardiology/American Heart Association task force on practice guidelines. Circulation 129[Suppl 2]: S49–S73, 2014
7. Arnett DK, Blumenthal RS, Albert MA, Buroker AB, Goldberger ZD, Hahn EJ, et al.: 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease
: A report of the American College of Cardiology/American Heart Association task force on clinical practice guidelines. Circulation 140: e596–e646, 2019
8. Matsushita K, Jassal SK, Sang Y, Ballew SH, Grams ME, Surapaneni A, et al.: Incorporating kidney disease measures into cardiovascular risk prediction: Development and validation in 9 million adults from 72 datasets. EClinicalMedicine 27: 100552, 2020
9. Matsushita K, Coresh J, Sang Y, Chalmers J, Fox C, Guallar E, et al.: Estimated glomerular filtration rate and albuminuria for prediction of cardiovascular outcomes: a collaborative meta-analysis of individual participant data. Lancet Diabetes Endocrinol 3: 514–525, 2015
10. Ito H, Pacold IV, Durazo-Arvizu R, Liu K, Shilipak MG, Goff DCJ Jr, et al.: The effect of including cystatin C or creatinine in a cardiovascular risk model for asymptomatic individuals: The multi-ethnic study of atherosclerosis
. Am J Epidemiol 174: 949–957, 2011
11. Clase CM, Gao P, Tobe SW, McQueen MJ, Grosshennig A, Teo KK, et al.; ONTARGET (ONgoing Telmisartan Alone and in combination with Ramipril Global Endpoint Trial) and TRANSCEND (Telmisartan Randomized Assessment Study in Angiotensin-Converting-Enzyme-Inhibitor Intolerant Subjects with Cardiovascular Disease
): Estimated glomerular filtration rate and albuminuria as predictors of outcomes in patients with high cardiovascular risk: A cohort study. Ann Intern Med 154: 310–318, 2011
12. Matsushita K, Ballew SH, Coresh J, Arima H, Ärnlöv J, Cirillo M, et al.: Measures of chronic kidney disease
and risk of incident peripheral artery disease: A collaborative meta-analysis of individual participant data. Lancet Diabetes Endocrinol 5: 718–728, 2017
13. Weiner DE, Tighiouart H, Elsayed EF, Griffith JL, Salem DN, Levey AS, et al.: The Framingham predictive instrument in chronic kidney disease
. J Am Coll Cardiol 50: 217–224, 2007
14. Matsushita K, Sang Y, Ballew SH, Astor BC, Hoogeveen RC, Solomon SD, et al.: Cardiac and kidney markers for cardiovascular prediction in individuals with chronic kidney disease
: The Atherosclerosis
Risk in Communities study. Arterioscler Thromb Vasc Biol 34: 1770–1777, 2014
15. Matsushita K, Sang Y, Ballew SH, Shlipak M, Katz R, Rosas SE, et al.: Subclinical atherosclerosis
measures for cardiovascular prediction in CKD. J Am Soc Nephrol 26: 439–447, 2015
16. Fontana M, Asaria P, Moraldo M, Finegold J, Hassanally K, Manisty CH, et al.: Patient-accessible tool for shared decision making in cardiovascular primary prevention: Balancing longevity benefits against medication disutility. Circulation 129: 2539–2546, 2014
17. Lash JP, Go AS, Appel LJ, He J, Ojo A, Rahman M, et al.; Chronic Renal Insufficiency Cohort (CRIC) Study Group: Chronic Renal Insufficiency Cohort (CRIC) study: baseline characteristics and associations with kidney function. Clin J Am Soc Nephrol 4: 1302–1311, 2009
18. Levey AS, Titan SM, Powe NR, Coresh J, Inker LA: Kidney disease, race, and GFR estimation. Clin J Am Soc Nephrol 15: 1203–1212, 2020
19. Inker LA, Schmid CH, Tighiouart H, Eckfeldt JH, Feldman HI, Greene T, et al.; CKD-EPI Investigators: Estimating glomerular filtration rate from serum creatinine and cystatin C. N Engl J Med 367: 20–29, 2012
20. Beretta L, Santaniello A: Nearest neighbor imputation algorithms: A critical evaluation. BMC Med Inform Decis Mak 16[Suppl 3]: 74, 2016
21. Feldman HI, Appel LJ, Chertow GM, Cifelli D, Cizman B, Daugirdas J, et al.; Chronic Renal Insufficiency Cohort (CRIC) Study Investigators: The Chronic Renal Insufficiency Cohort (CRIC) study: Design and methods. J Am Soc Nephrol 14[Suppl 2]: S148–S153, 2003
22. Fine JP, Gray RJ: A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc 94: 496–509, 1999
23. Tibshirani R: The lasso method for variable selection in the Cox model. Stat Med 16: 385–395, 1997
24. Steyerberg EW, Harrell FEJ Jr: Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 69: 245–247, 2016
25. Wright JD, Folsom AR, Coresh J, Sharrett AR, Couper D, Wagenknecht LE, et al.: The ARIC (Atherosclerosis
Risk In Communities) study: JACC Focus Seminar 3/8. J Am Coll Cardiol 77: 2939–2959, 2021
26. Bild DE, Bluemke DA, Burke GL, Detrano R, Diez Roux AV, Folsom AR, et al.: Multi-Ethnic Study of Atherosclerosis
: Objectives and design. Am J Epidemiol 156: 871–881, 2002
27. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al.: Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology 21: 128–138, 2010
28. Blanche P, Dartigues J-F, Jacqmin-Gadda H: Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Stat Med 32: 5381–5397, 2013
29. Kattan MW, Gerds TA: The index of prediction accuracy: An intuitive measure useful for evaluating risk prediction models. Diagn Progn Res 2: 7, 2018
30. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS: Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med 27: 157–172, discussion 207–212, 2008
31. Uno H, Tian L, Cai T, Kohane IS, Wei LJ: A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data. Stat Med 32: 2430–2442, 2013
32. North BJ, Sinclair DA: The intersection between aging and cardiovascular disease
. Circ Res 110: 1097–1108, 2012
33. Perkovic V, Agarwal R, Fioretto P, Hemmelgarn BR, Levin A, Thomas MC, et al.; Conference Participants: Management of patients with diabetes and CKD: Conclusions from a “Kidney Disease: Improving Global Outcomes” (KDIGO) Controversies Conference. Kidney Int 90: 1175–1183, 2016
34. Yasue H, Yoshimura M, Sumida H, Kikuta K, Kugiyama K, Jougasaki M, et al.: Localization and mechanism of secretion of B-type natriuretic peptide in comparison with those of A-type natriuretic peptide in normal subjects and patients with heart failure. Circulation 90: 195–203, 1994
35. de Lemos JA, Drazner MH, Omland T, Ayers CR, Khera A, Rohatgi A, et al.: Association of troponin T detected with a highly sensitive assay and cardiac structure and mortality risk in the general population. JAMA 304: 2503–2512, 2010
36. Bansal N, Hyre Anderson A, Yang W, Christenson RH, deFilippi CR, Deo R, et al.: High-sensitivity troponin T and N-terminal pro-B-type natriuretic peptide (NT-proBNP) and risk of incident heart failure in patients with CKD: The Chronic Renal Insufficiency Cohort (CRIC) study. J Am Soc Nephrol 26: 946–956, 2015
37. Scheven L, de Jong PE, Hillege HL, Lambers Heerspink HJ, van Pelt LJ, Kootstra JE, et al.; PREVEND study group: High-sensitive troponin T and N-terminal pro-B type natriuretic peptide are associated with cardiovascular events despite the cross-sectional association with albuminuria and glomerular filtration rate. Eur Heart J 33: 2272–2281, 2012
38. Levey AS, Coresh J: Chronic kidney disease
. Lancet 379: 165–180, 2012
39. Nogueira J, Weir M: The unique character of cardiovascular disease
in chronic kidney disease
and its implications for treatment with lipid-lowering drugs. Clin J Am Soc Nephrol 2: 766–785, 2007
40. Matsushita K, Sang Y, Chen J, Ballew SH, Shlipak M, Coresh J, et al.: Novel “predictor patch” method for adding predictors using estimates from outside datasets - A proof-of-concept study adding kidney measures to cardiovascular mortality prediction. Circ J 83: 1876–1882, 2019
41. Chen J, Budoff MJ, Reilly MP, Yang W, Rosas SE, Rahman M, et al.; CRIC Investigators: Coronary artery calcification and risk of cardiovascular disease
and death among patients with chronic kidney disease
. JAMA Cardiol 2: 635–643, 2017