After the development of the Kidney Donor Risk Index (KDRI),1 the United Network for Organ Sharing began including the Kidney Donor Profile Index (KDPI), a percentile version of the KDRI, with every organ offer beginning March 26, 2012. Furthermore, modifications to kidney allocation were implemented in late 2014, incorporating KDPI into the system and allocating the top 20% of deceased donor kidneys (by best expected posttransplant survival) to the top 20% of candidates with the best expected posttransplant survival. The transition from the expanded criteria donor (ECD) system to the KDPI system was also intended to incorporate broader sharing of high KDPI kidneys at the time of initial offer, with the intent of increasing utilization of marginal kidneys and decreasing discard rates. In effect, the continuous KDPI scale has replaced the binary ECD designation, transforming the process of evaluating organ offers and informing clinical decision making.
Recent literature has analyzed the outcomes of the transition from ECD to KDPI, examining metrics, such as survival outcomes, discard rates, concerns for subpopulations, regional and national sharing patterns, and so on. Not surprisingly, much of the interest has been on high KDPI organs, variably defined as KDPI >80% or >85%, because these deceased donor kidneys have poorer survival outcomes, are the most costly (due to increased rates of recipient complications), and are the most prone to discard. Several articles have confirmed KDPI’s discriminatory power. The KDPI >35% group has been shown to have worse survival than the KDPI ≤20% group and poorer graft function than the KDPI ≤35% group.2 Similarly, Pelletier et al3 found that diabetics with coronary artery disease have a significantly increased risk of death after receiving high KDPI kidneys compared with receiving low KDPI kidneys. Although, on average, survival benefit was demonstrated among all transplant recipients, even those complicated by delayed graft function, patients who received lower-quality kidneys (>80%) took much longer to derive such a survival benefit.4 Other researchers have used the KDRI framework in assessing the benefits of transplantation. For example, Massie et al5 found that high KDPI kidney transplants are associated with increased short-term (but decreased long-term) mortality risk in some patients compared with those remaining on dialysis. Similarly, patients older than 60 years were found to have significant reductions in mortality after both preemptive kidney transplants and non-preemptive kidney transplants with high-risk KDPI organs relative to waiting for a lower KDPI kidney.6
Despite studies confirming the predictive power of KDPI, various studies have reflected the desire and opportunity for improved predictive ability. Gupta et al2 concluded that high (>85) KDPI grafts behave similarly to moderate (35–85) KDPI grafts in terms of survival and graft function. Similarly, in older patients (>69 y), low- and medium-quality kidneys seemed to yield comparable outcomes.7 Parker et al8 altered some covariates to better discriminate between kidneys from pediatric donors. Because high KDPI organs have a higher rate of discard, studies have also looked into pretransplant donor biopsies to supplement and aid the selection of marginal kidneys.9 Given that the discriminatory power of the KDPI is modest (concordance index of 0.621) and given the role of KDPI in making important clinical and allocation decisions, there continues to exist a demand for a more predictive index of measurement for kidney quality.
The original KDRI article was based on data obtained from the Scientific Registry of Transplant Recipients (SRTR) for transplants during 1995 to 2005. Using a cohort which included updated (2000–2016) SRTR data, we derive an updated KDRI (through a modeling framework which mimics that originally applied) which we compare with the original version. In particular, we test the equality of the updated and original coefficients; we then compare the original and updated KDRI with respect to (a) risk discrimination and (b) predictive accuracy. The objective of our analysis was to either propose an updated KDRI or to endorse the continued use of the original index based on its empirically demonstrated robustness.
MATERIALS AND METHODS
The SRTR data were used for this study. The SRTR data system includes data on all donors, waitlisted candidates, and transplant recipients in the United States, as submitted by the members of the Organ Procurement and Transplantation Network (OPTN), and has been described elsewhere. The Health Resources and Services Administration, US Department of Health and Human Services provides oversight to the activities of the OPTN and SRTR contractors. We included deceased donor transplants to adult recipients (age ≥ 18 y) occurring between January 1, 2000, and December 31, 2016. We excluded repeat and multiorgan transplants, resulting in a final study population size of N = 156069. The study was deemed exempt from institutional review board approval by the University of Michigan Heath Sciences and Behavioral Sciences.
We retained the outcome of interest, graft failure (GF), defined as the earliest of return to dialysis, retransplant, and death, as in the original KDRI article. Patients were followed up from date of transplantation to the earliest of GF, loss to follow-up, or December 31, 2016. In total, from among our study population, there were 59912 GF events: 30196 deaths (50.4%), 29128 (48.6%) returns to dialysis, and 588 (1.0%) retransplants, with a small fraction with the same date entered for 2 different causes. In total, 62% of the study population had a functioning graft at the end of the 16-year study period.
Our primary objective was to, using a methodologic framework analogous to Rao et al, develop an updated KDRI using more recent data. Part of this exercise involved potentially using additional donor and recipient factors in the model (and updated KDRI), or modifying the way that existing factors are used in the KDRI formula. Such an exercise then implies comparing the performance of the original and updated models. The original KDRI score was calculated using data between 1995 and 2005, and contained a number of predictive variables that were not used by the United Network for Organ Sharing to calculate its percentile version, KDPI (ie, HLA mismatches, en bloc transplant, double kidney transplant, and cold ischemia time).
An updated Cox regression model10 with donor, transplant, and recipient factors was fitted to estimate the relative rates of GFs independently associated with each donor factor. All the factors in the original KDRI model were included initially, as well as some other factors that were hypothesized to be important based on an exhaustive scan of covariates available from the SRTR recipient file. Variable selection was based on a variant of stepwise deletion. We started with a very rich model that included many more covariates than employed in the original KDRI analysis. Nonsignificant (P > 0.05) variables were then deleted in sequence, in order of lack of significance. Ultimately, the final model included the following donor factors: age, race, sex, smoking status (≥20 pack-y smoker), diabetes status and duration, hypertension status and duration, cause of death as stroke, donation after circulatory death (DCD), hepatitis C virus (HCV) serology status, serum creatinine, height, and weight. Recipient factors include age, race, sex, US citizenship status, height, weight, panel-reactive antibody (PRA), HCV status, diabetes status, diagnoses (polycystic kidney disease, diabetes, hypertension, etc), year of transplant, pretransplant dialysis and length, ABO blood type, hospitalization status (in intensive care unit [ICU], hospitalized but not in ICU, not hospitalized), functional status, insurance type, education level, transplant center, and indicators for the following comorbidities: cerebral vascular disease, peripheral vascular disease, chronic obstructive pulmonary disease, previous malignancy, drug-treated hypertension, CMV status, Epstein-Barr viral status, and hepatitis B virus status. Donor factors that were statistically significant in the Cox model were extracted to calculate the KDRI. The following transplant factors were included in the model (but not the updated or original KDRI calculation): HLA mismatches, cold ischemia time, en bloc/double kidney transplant, 3 or more inotropic agents at time of incision, and inotropic support. Transplant center was adjusted using indicator variates (ie, fixed effects model).
With respect to missing covariate data, among the 46 covariates in the final model, only 5 had 10% to 15% missing values (hospitalization status, cerebral vascular disease, hypertension, candidate education, and 3 or more inotropic agents at time of incision), whereas 1 (PRA) had 18% missingness. Because the proportion of covariate missing values was so low, we (singly) imputed the median for missing continuous covariates, and the mode for categorical covariates.
Unlike the original KDRI model from Rao et al, we did not stratify by any recipient factors. This was for 2 reasons. First, vast improvements in computation speed meant that computational savings due to such stratification were now trivial. Second, the C statistic for stratified data typically only uses failure time orderings within-stratum. Such an approach will tend to underestimate the risk discrimination ability of the model, to the degree to which the stratification factors predict risk. To evaluate this phenomenon, we computed the C statistic (for both the original and updated KDRI models) both by stratifying by transplant center, as well as representing transplant centers through indicator variates (via a factor statement in R).
The functional form of each continuous covariate (specifically age, weight, height, and donor creatinine) was assessed as follows: categorize the predictor; refit the model with the continuous predictor replaced by categories; plot category-specific parameter estimates against their respective medians (scatter plot); use the shape of the plot to infer the true functional form; evaluate linear splines where indicated, with knots (ie, points at which the slope changes) chosen by inspection. Linear splines that proved to be significant (for donor age, creatinine, and weight) were then used in the final model.
After establishing the final model and coefficients for the updated KDRI, we tested (where applicable) the equality of the updated and original coefficient. Although we no longer had access to the data set used to derive the original KDRI, we recovered the standard errors for each coefficient through Table 1 of Rao et al.1 Each parameter-specific test was based on a standard normal distribution. The tests were conservative in the sense that correlations between the 2 sets of parameter estimates (resulting from overlapping patients in the original and updated study samples) were not taken into account.
We compared the original and updated KDRI with respect to predictive accuracy (also called calibration) and risk discrimination. The models used for both metrics included all recipient and transplant factors from the final model. However, all donor factors were replaced by log KDRI (either original or updated). We quantified predictive accuracy through a censored data analog to the Hosmer-Lemeshow statistic. Specifically, patients were broken into half-deciles based on predicted cumulative hazard at follow-up time. Observed minus expected GFs were then computed and summed across each half-decile. The absolute value of each summed half-decile-specific residual was then taken, after which the sum was taken across all half-deciles. Risk discrimination refers to how well the model correctly orders 2 randomly chosen transplants with respect to GF risk. We quantified risk discrimination by the C statistic, also known as the Index of Concordance.11 C statistics were computed through 10-fold cross-validation. To make the cross-validation calculation feasible (as centers were treated as fixed effects), we adjusted the data slightly by removing all centers with <10 transplants.
All analyses were performed using SAS version 9.4 (SAS Institute; Cary, NC) and R (version 3.2.2).
Based on the updated modeling, structural changes to the KDRI formula included the following. For donor age, the knots (slope change points) were 20 years (age 18 y in the original KDRI) and 40 years (previously age 50 y). An interaction was added between donor height and an indicator (0/1 covariate) for donor age 12 years or younger, that is, the height effect was allowed to be different depending on whether the donor was younger or older than 12 years. We added a similar interaction for donor weight, and added a spline (knot at 80 kg) for donor age older than 12 years. However, we removed the height and weight effect for those with donor age of 12 years or younger due to insignificant P values. For each of donor diabetes and donor hypertension, the “yes” category was subdivided by duration: 0 to 5 years, 6 to 10 years, older than 10 years, and unknown. In addition, donor smoking (>20 pack-y) was added to the updated KDRI formula. Consistent with the OPTN application of the original KDRI formula, HLA mismatches, double/en bloc, and cold ischemia time were not used in the updated index.
Table 1 describes the study population (n = 156069) with respect to donor factors included in the updated KDRI, as well as a subset of recipient factors included in the final model and often reported in describing transplant recipient characteristics. The reference donor had the following characteristics: 40 years of age, male, white, nonsmoker, nondiabetic, nonhypertensive, death not due to cerebrovascular accident, donation not after circulatory death, HCV-negative, 170 cm in height, 80 kg in weight, and with a serum creatinine of 1.0 mg/dL.
A histogram of the updated KDRI is presented in Figure 1. Approximately 50% of the deceased donor kidneys in our sample had an updated KDRI in the 1.0 to 1.4 range.
Figure 2 shows a plot and percentile table of the cumulative distribution of the updated KDRI values. As computed based on the current (ie, 2000–2016) cohort, the median was 1.16. The 90th percentile was 1.64, indicating that 90% of transplant recipients in our study population received a donor kidney with a KDRI of 1.64 or less. The range of KDRI values was 0.74 to 3.91. More than 98% of donor kidneys had an updated KDRI between 0.8 and 2.2.
In Figure 3, we depict the effect of donor age on GF risk based on the updated model. The dots represent parameter (log hazard ratio [HR]) estimates based on a model that categorized donor age; this model was fitted for the purposes of assessing the functional form of donor age. The line represents the fitted linear spline based on the updated KDRI model. For those younger than 20 years, GF risk decreases with age (1.77% per y; obtained by summing the second and third rows of Table 2). Between ages 20 and 40 years, GF risk increases by ≈0.73% per year (per the second row). After age 40 years, GF risk increases by 1.47% per year increase in age (summing the second and fourth rows).
In Figure 4, we contrast the donor age effects from the updated (solid line; repeated from Figure 3) and original (dotted line) KDRI models. Trends are generally quite similar. The updated donor age effect has knots (slope change points) at age 20 and 40 years, whereas the original donor age effect had knots at ages 18 and 50 years. Donor weight effects have also undergone slight changes between the original and the updated models (figure not shown). Mainly, the addition of a knot at 80 kg reflects a leveling off of the donor weight effect for weight >80 kg.
In Table 2, we compare the coefficients used to calculate the original KDRI and updated KDRI. We tested each of the 21 instances where comparable parameters (ie, parameters with the same or very similar interpretation) appeared in each of the original and updated KDRI formula. Differences were nonsignificant in 15 of the 21 tests. The effect of donor age was significantly different between ages 20 and 40 years. A significant difference was observed for donor creatinine, which had a much greater slope (up to Cr = 1.5) in the original KDRI model.
We illustrate the computation of the updated KDRI through an example in Table 3. In this example, the deceased donor is HCV+, white, 29 years of age, 190 cm tall, weighs 75 kg, with a serum creatinine of 1.25, and diabetic (diagnosed 4 y before death). The donor was a non-DCD, nonsmoker whose death was not due to stroke. The updated KDRI for this donor is 1.39 which, based on Figure 2, would be at approximately the 75th percentile.
A scatterplot of updated versus original KDRI (Figure 5) indicated considerable concordance between the 2 scores. The points in this plot represent original and updated KDRI calculations computed for the transplants used to fit the updated model. The close agreement is confirmed by the Spearman (ie, rank) correlation coefficient of ρ = 0.97 (P < 0.0001); note that ρ = 1 would represent perfect correlation, whereas ρ = 0 would reflect an absence of correlation. The pairs of updated and original KDRI were ranked by absolute distance, and the darker points (teal) are within the 95th percentile by distance, whereas the lighter points (blue) are in the top 5%. We can see that the vast majority of points are within a tight ellipse, but a number of outlying and discordant data points surround the ellipse.
Figure 6 shows a histogram of the percentile differences between the original and updated KDRI values. Approximately 47% of the kidneys had a percentile difference between −3 and 3, whereas ≈88% of kidneys had a percentile difference between −10 and 10. This indicates that, in applying the updated KDRI, the vast majority of kidneys did not move beyond a decile from their original ranking.
In Table 4, based on our study population, we cross-classify the top 15% (ie, highest KDRI and, hence highest-risk donor kidneys), lowest 20%, and middle 65%, based on the original and updated KDRI formulas. The original and updated KDRI place the donor kidneys in the same category for approximately 91% of cases, as shown by summing the main diagonal of the 3 × 3 table. Among the 31215 kidneys in the lowest 20% of original KDRI, about 90% are in the lowest 20% based on the updated KDRI (ie, 17.6/20). Of the 23411 kidneys originally in the highest 15% KDRI, 86% (ie, 12.9/15) remain in that category.
To compare the updated and original KDRI with respect to risk discrimination, we calculated various measures of discrimination using the original KDRI and the updated KDRI model. First, to determine the appropriate treatment of transplant centers, we replaced the donor factors with their log KDRI values in the full model and either stratified by transplant center or treated each as a class variable. Treating the transplant centers as class variables increased the concordance index by 0.015 over stratifying by transplant centers, for both full models with log KDRI (original) and log KDRI (updated). Hence, we incorporated transplant centers as class variables in further analyses.
As shown in Table 5, with respect to cross-validated risk discrimination, the updated KDRI (C = 0.652) performs slightly better than the original KDRI (C = 0.651), for a relative increase of 0.15%. By comparison, the original KDRI resulted in a C index of 0.62. Much of the discrepancy between the 2 C indices computed for the original KDRI (current report vs Rao et al1) can be explained by the decision to model centers with stratification (original KDRI article) versus with center-specific indicators (current report). The motivation for such stratification was computational speed, which is no longer an issue at the time of the updated analysis.
To compare the updated versus original KDRI with respect to predictive accuracy, we divided all cases into 20 categories (ie, half-deciles) based on each recipient’s predicted cumulative GF hazard at their end of follow-up. Within each half-decile, we summed the martingale residuals for all subjects, then took the absolute value. We then summed the absolute values across all 20 half-deciles. Martingale residuals are interpreted as observed minus expected, so in effect, this is a category based summation measure of deviation from expected, similar to a Hosmer-Lemeshow statistic for logistic regression. The summed absolute residuals for the updated KDRI model were reduced by 0.09% relative to the original KDRI, indicating an almost negligible increase in predictive accuracy for the updated KDRI (Table 5). Note that the coefficients for the updated KDRI are estimated such as to maximize the partial likelihood12 corresponding to Cox regression. However, this is different from minimizing a criterion based on the residuals. For this reason, it is plausible that the original KDRI performs approximately as well as the updated version in this respect.
For illustration purposes, we created 3 donor scenarios and calculated their KDRI scores under original and updated models (Table 6). All unmentioned covariates are assumed to be their respective reference levels. For all 3 donors, the updated KDRI is quite similar although slightly greater than the original KDRI.
The original KDRI method was proposed as a continuous version which improved upon the binary ECD distinction. Given the increase in the kidney transplant waiting list, the ability to more precisely discriminate deceased donor organ quality was hoped to reduce discard rate and aid more confident clinical decision making. In this report, we attempted to improve on the currently implemented KDRI through a modeling strategy similar to that adopted previously. In addition, we sought to evaluate the performance of the original KDRI in the presence of a much more recent data set.
Our main findings can be summarized as follows. First, few coefficients in the updated KDRI were significantly different from their respective counterparts in the original formula. Among parameters represented in both the original and updated KDRI formulas, only donor age (20–40 y), creatinine (<1.5), and weight (>80 kg) were significantly different. Second, the updated and original KDRI are very highly correlated, and differences between original and updated percentiles were generally very small. Third, the updated KDRI demonstrated negligibly improved risk discrimination and predictive accuracy, compared with its predecessor.
The improvements we made to the KDRI included the following: we adjusted the position of the change points slightly for the slope the donor age effect; we altered the donor height and weight effects, such that each now applies only to donors older than 12 years; we added a change point to the donor weight effect, such that it now levels off after 80 kg; we incorporated duration of diabetes and duration of hypertension, replacing the previous yes/no indicators; we also added a parameter for smoking. Each of these modifications is defensible empirically. Both duration of diabetes and duration of hypertension demonstrated monotone dose-response relationships to GF risk. Although such improvements may be important for various subsets of donor kidneys, the improvements had very little impact overall on the updated KDRI’s performance. Less than 7% of the donors in our study population were diabetic, whereas 26% had a history of hypertension. Only about 5% of donors were younger than 12 years.
The median updated KDRI was 1.16 based on our study population, compared with 1.05 in the analysis of Rao et al. The index of Rao et al contained parameters for HLA mismatches and cold ischemia time. In particular, the reference donor had 2 HLA-B mismatches, 1 HLA-DR mismatch, and 20 hours (ie, greater than average) cold ischemia time. Hence, the reference donor in our updated analysis is inherently lower risk, meaning that the HR (relative GF risk) for all organs in our sample will increase relative to this more idealized reference donor. One could easily divide the computed KDRI by 1.16 to set the sample median to be ≈1.
A number of studies have attempted to improve upon the KDRI, and some have proposed improvements for specific subpopulations, such as pediatric patients.8 However, no recent article has shown significant overall improvement over the KDRI model. Our recent analyses indicate that although the original KDRI might not precisely capture changing trends in more recent data, it is still a robust measure of discrimination of deceased donor kidney quality and can be relied on to make both allocation policy decisions as well as individual clinical decisions.
Our study is subject to the limitations of an observational study. As with the original KDRI article, relationships reported do not represent causality for failure rates, but rather a general description of failure rates and relationships with different types of donor and transplant factors. There might be other unmeasured variables that might increase the discriminatory ability of the index, but might render it less parsimonious and practical for clinical use.
In summary, we have demonstrated that the original KDRI performs very similarly to an updated version derived by modeling more recent data and making some adjustments to the KDRI formula. The original and updated KDRI had approximately equal risk discrimination and predictive accuracy. The most prominent features shared by the original analysis (in Rao et al) and updated analysis (presented in this report) are the data source (SRTR), regression model, and modeling strategy. It is possible that a superior index could be derived outside the space defined by these parameters. In particular, the use of more granular data may be beneficial, even if available for a much smaller sample.
The authors wish to thank the 2 editors and 2 reviewers for their thoughtful comments and suggestions which led to considerable improvement in the article. This work was supported in part by National Institutes of Health grant R01-DK070869, and by a grant from the Michigan Institute for Clinical and Health Research (MICHR). SRTR data were supplied by the Minneapolis Medical Research Foundation through a Data Use Agreement.