Estimating Kidney Failure Risk Using Electronic Medical Records : Kidney360

Journal Logo

Original Investigations: Chronic Kidney Disease

Estimating Kidney Failure Risk Using Electronic Medical Records

Naranjo, Felipe S.; Sang, Yingying; Ballew, Shoshana H.; Stempniewicz, Nikita; Dunning, Stephan C.; Levey, Andrew S.; Coresh, Josef; Grams, Morgan E.

Author Information
Kidney360 2(3):p 415-424, March 2021. | DOI: 10.34067/KID.0005592020
  • Free
  • Infographic
  • SDC

Abstract

Background 

The four-variable kidney failure risk equation (KFRE) is a well-validated tool for patients with GFR <60 ml/min per 1.73 m2 and incorporates age, sex, GFR, and urine albumin-creatinine ratio (ACR) to forecast individual risk of kidney failure. Implementing the KFRE in electronic medical records is challenging, however, due to low ACR testing in clinical practice. The aim of this study was to determine, when ACR is missing, whether to impute ACR from protein-to-creatinine ratio (PCR) or dipstick protein for use in the four-variable KFRE, or to use the three-variable KFRE, which does not require ACR.

Methods 

Using electronic health records from OptumLabs Data Warehouse, patients with eGFR <60 ml/min per 1.73 m2 were categorized on the basis of the availability of ACR testing within the previous 3 years. For patients missing ACR, we extracted urine PCR and dipstick protein results, comparing the discrimination of the three-variable KFRE (age, sex, GFR) with the four-variable KFRE estimated using imputed ACR from PCR and dipstick protein levels.

Results 

There were 976,299 patients in 39 health care organizations; 59% were women, the mean age was 72 years, and mean eGFR was 47 ml/min per 1.73 m2. The proportion with ACR testing was 19% within the previous 3 years. An additional 2% had an available PCR and 36% had a dipstick protein; the remaining 43% had no form of albuminuria testing. The four-variable KFRE had significantly better discrimination than the three-variable KFRE among patients with ACR testing, PCR testing, and urine dipstick protein levels, even with imputed ACR for the latter two groups. Calibration of the four-variable KFRE was acceptable in each group, but the three-variable equation showed systematic bias in the groups that lacked ACR or PCR testing.

Conclusions 

Implementation of the KFRE in electronic medical records should incorporate ACR, even if only imputed from PCR or urine dipstick protein levels.

Visual Abstract

Abstract
Export

Introduction

The kidney failure risk equation (KFRE) is a widely validated tool for estimating the absolute risk of kidney failure over 2- and 5-year time horizons in patients with GFR <60 ml/min per 1.73 m2. The four-variable KFRE incorporates age, sex, GFR, and albuminuria as assessed by the urine albumin-creatinine ratio (ACR) (123–4). These estimates of kidney failure can help guide clinical actions, such as referral for nephrology or vascular access (56–7). Indeed, several health systems are working to implement automatic reporting of the four-variable KFRE in electronic health records (EHR).

Unfortunately, the inputs necessary for the four-variable KFRE are not always present in the EHR. eGFR is commonly assessed and usually available, along with age and sex, but albuminuria is often not. In a study from the Cleveland Clinic, 36% of patients with eGFR 30–59 ml/min per 1.73 m2 did not have any measurement of albuminuria within a year after CKD was confirmed (8). In addition, the type of albuminuria testing can vary, with urine protein-creatinine ratio (PCR), and urine dipstick protein level assessment frequently used to assess albuminuria instead of ACR. We recently developed equations to convert PCR and urine dipstick to ACR, providing a tool to screen for and stage CKD in the absence of ACR measurements (9). However, how best to estimate kidney failure risk in the setting of missing data inputs and different measures of albuminuria is uncertain (4).

Using a database of health information on more than 120 million patients, the major objectives of this study were two-fold. First, we aimed to obtain a “real-world” assessment of frequency and type of albuminuria testing available to health care organizations (HCOs) in the clinical care of patients with eGFR <60 ml/min per 1.73 m2. Second, we aimed to evaluate strategies of kidney failure risk estimation in patients missing ACR, comparing the four-variable KFRE with imputed ACR to the three-variable KFRE that does not require ACR measures, and different “look-back” periods in which to capture albuminuria measures.

Materials and Methods

Study Population

This study used deidentified data from EHRs from OptumLabs Data Warehouse (OLDW). The database contains longitudinal health information on patients, representing a diverse mixture of ages, ethnicities, and geographic regions across the United States. The EHR-derived data include a subset of EHR-derived data that has been normalized and standardized into a single database (10). For the purposes of this study, we selected data from all HCOs meeting our inclusion criteria in the EHR-derived data. The index date was considered the first date in 2013 on which eGFR was <60 ml/min per 1.73 m2, and patients were followed until 5 years after the index date. At the patient level, all patients with eGFR <60 ml/min per 1.73 m2 measured in the outpatient setting in 2013 were included. Each HCO had to have at least 50 ESKD events after 2013 in the group with an available ACR test; with these criteria, 39 HCOs were included (N=976,299) (Supplemental Appendix 1). In a sensitivity analysis, each person had to have a confirmed eGFR <60 ml/min per 1.73 m2 with at least one preceding eGFR <60 ml/min per 1.73 m2 between 90 and 730 days before the index date, to identify a population with potentially higher rates of albuminuria testing. eGFR was calculated using serum creatinine and the CKD Epidemiology creatinine equation (11). Specific information on standardization of assays for serum creatinine was not available, but standardization was routine in laboratories by 2011, before the beginning of the study period (12).

Clinical Variables and Outcomes

Demographic and clinical covariates were determined using the EHR and defined as present on or before the index date. Demographic covariates included age and sex, and clinical variables included diabetes mellitus and hypertension as defined by International Classification of Diseases ninth revision (ICD-9) clinical modification and ICD-10 codes (hypertension codes 40×, I1× and diabetes mellitus codes 250×, E10, E11, and E13). The absence of an encounter was considered the absence of disease; as such, there were no missing values for comorbidities defined by encounters. All laboratory values for ACR, PCR, and urine dipstick protein levels measurement were abstracted within 3 years of the index date.

ESKD was defined as having a current procedural terminology code (90,919–90,999) or health care common procedure coding system code (G0308–G0327, G0257, Q4081) or ICD-9 and ICD-10 code (39.95, 5A1D, 54.98, 3E1M39Z, V45.1, Z91.15, Z99.2, V56, Z49, 585.6, N18.5, N18.6, V42.0, Z94.0, T86.1, 55.69, 585.5, 0TY, 996.81) occurring within 2 or 5 years after the index date in 2013. These ICD codes include kidney transplant, stage 5 CKD, or ESKD. Also included were procedure codes for dialysis, provided there were at least two codes separated by at least 1 month and within a 3-month period. Previous studies have suggested this method is both sensitive (95%) and specific (100%) (13).

Statistical Analyses

Patient characteristics were evaluated within each OLDW HCO. We evaluated the availability of ACR tests on or before the index date in the previous 1, 2, and 3 years. We categorized patients on the basis of the presence or absence of albuminuria testing within each time period, using the guideline-recommended hierarchy of testing (14,15). Group 1 was defined as those patients with an available ACR test; group 2 included patients who had no available ACR but a PCR test; group 3 included patients with no available ACR or PCR but who had a urine dipstick protein level measurement; and group 4 had no ACR, PCR, or urine dipstick protein level measurement. Patient characteristics across these groups were examined, with age described as mean (SD) on the index date in 2013 and sex, diabetes, and hypertension summarized as proportions. Urine ACR and PCR were summarized using median and interquartile range because of skewed distributions. ESKD incidence was calculated within each center as the number of events over person-time at risk, with time at risk accruing from the index date until ESKD event, last active date, defined as the last encounter date in the EHR (defined by OptumLabs), or death.

We considered the groups formed on the availability of albuminuria testing in the 3 years before the index date as our primary analysis. We used the KFRE to estimate 5-year risk of ESKD and determined discrimination using C-statistics (3). For groups 1, 2, and 3, we compared the three-variable KFRE (which incorporates age, sex, and eGFR) with the four-variable KFRE (which incorporates age, sex, eGFR, and ACR), calculating the difference in C-statistic within each HCO and then meta-analyzing using random effects meta-analysis. Because ACR was not available in group 2 and group 3, we used the PCR and urine dipstick protein levels to estimate ACR using the following equations: for PCR (exp [5.2562+0.2445× log (min [PCR/50, 1]) +1.5531× log (max [min (PCR/500, 1) 0.1]) +1.1057× log (max [PCR/500, 1])–0.0793× (if female) +0.0802× (if diabetic) +0.1339× (if hypertensive)]), and for urine dipstick protein (exp [1.9911+0.7370× (if trace) +1.6936× (if +) +3.2783× (if ++) +4.5732× (if >++) +0.0907× (if female) +0.2789× (if diabetic) +0.3307× (if hypertensive)]) (9). For group 4, we reported only the three-variable equation. We repeated these procedures for the 5-year KFRE. Calibration was tested by plotting deciles of expected 2- and 5-year risk against observed risk and calculating the Greenwood-Nam-D’Agostino statistic within each HCO. Deciles with < five events were combined. A smaller Greenwood-Nam-D’Agostino statistic was considered an improvement in calibration and was compared between three-variable KFRE and four-variable KFRE using Wilcoxon matched-pairs signed rank test. Only HCOs with at least 50 ESKD events were included. We also evaluated the net reclassification index using clinically relevant risk categories (<10%, 10%–19%, 20%–39%, 40%+ for 2-year time horizon, corresponding risk categories for the 5-year horizon) comparing three-variable and four-variable KFRE for groups 1, 2, and 3 individually across HCOs and then meta-analyzed. All analyses were performed using Stata 15 (StataCorp 2017; StataCorp LLC, College Station, TX).

Results

Study Population

There were 976,299 patients included from 39 different HCOs (Table 1). Overall, the average age was 72 years (SD, 11 years), and the percentage of women was 59%. The mean eGFR in 2013 was 47 ml/min per 1.73 m2; 32% had diabetes, and 71% had hypertension. There was some variation across HCOs, with the average age ranging from 62 to 75 years and average eGFR ranging from 44 to 49 ml/min per 1.73 m2.

Table 1. - Baseline characteristics of patients with eGFR <60 ml/min per 1.73 m2, by health care organization
Health Care Organization Identification N Age (mean, SD) % Female eGFR (mean, SD) % Diabetes % Hypertension
1 18,273 71 (10) 60 47 (11) 40 85
2 28,690 74 (10) 61 47 (11) 36 87
3 14,560 72 (10) 62 48 (11) 31 72
4 17,833 73 (10) 61 47 (11) 37 85
5 12,342 73 (10) 58 49 (10) 35 80
6 83,266 71 (11) 61 48 (11) 29 67
7 9309 73 (10) 61 47 (10) 35 88
8 10,160 71 (10) 58 47 (11) 43 88
9 16,463 65 (12) 54 49 (11) 32 75
10 65,990 71 (11) 60 47 (11) 40 86
11 6239 62 (12) 56 45 (13) 61 85
12 17,078 73 (10) 62 47 (11) 35 83
13 16,794 73 (9) 59 48 (10) 45 86
14 20,977 71 (12) 57 46 (12) 31 68
15 12,442 75 (9) 55 46 (12) 34 84
16 11,396 74 (10) 56 46 (12) 37 73
17 44,092 72 (10) 61 47 (11) 36 80
18 15,774 71 (11) 58 45 (12) 30 79
19 13,057 72 (10) 51 47 (12) 30 67
20 32,055 73 (10) 61 48 (11) 39 88
21 21,408 71 (12) 55 45 (13) 2 3
22 14,068 71 (10) 60 46 (12) 37 80
23 17,831 73 (9) 58 48 (11) 32 74
24 38,390 71 (11) 59 48 (11) 31 76
25 14,967 72 (10) 61 47 (11) 32 83
26 12,122 72 (11) 55 44 (13) 22 40
27 23,954 72 (10) 59 47 (11) 35 80
28 28,114 72 (11) 59 46 (12) 29 66
29 10,281 73 (11) 62 48 (11) 35 73
30 23,110 72 (10) 59 47 (11) 37 79
31 18,669 71 (10) 60 47 (11) 21 53
32 22,847 72 (12) 54 47 (12) 7 17
33 9443 73 (10) 61 46 (11) 29 60
34 7591 67 (13) 53 46 (13) 30 66
35 103,583 72 (10) 61 48 (10) 35 83
36 104,536 72 (11) 58 46 (12) 24 53
37 11,916 70 (11) 57 44 (14) 36 70
38 12,583 74 (10) 57 48 (11) 42 87
39 14,096 70 (12) 58 45 (13) 6 13
Total 976,299 72 (11) 59 47 (11) 32 71

In the 634,259 participants with confirmed eGFR <60 ml/min per 1.73 m2, the average age was 73 years old, 60% were women, the mean eGFR was 46 ml/min per 1.73 m2, 35% had diabetes, and 76% had hypertension (Supplemental Table 1).

ESKD Incidence

There were 29,653 ESKD events over a mean follow-up of 4.3 years (Supplemental Table 2). The overall incidence of ESKD was 7.05 events per 1000 person-years. The range of ESKD incidence by HCO was 3.67 events per 1000 person-years to 30.3 events per 1000 person-years.

Frequency of ACR Testing within the Previous 3 Years

Overall, only 15% of patients had ACR testing within 1 year before the index date, 18% had testing within 2 years, and 19% had testing within 3 years (Table 2). There was variability in testing by HCOs, with the range of patients who had ACR testing within 1 year between 5% and 68%, within 2 years from 6% to 73%, and within 3 years from 6% to 74%.

Table 2. - Proportion with albumin-creatinine ratio testing according to “look-back” window
Health Care Organization Identification N Albumin-Creatinine Ratio Within 1 yr, N (%) Albumin-Creatinine Ratio Within 2 yr, N (%) Albumin-Creatinine Ratio Within 3 yr, N (%)
1 18,273 3393 (18.6) 4446 (24.3) 4996 (27.3)
2 28,690 4242 (14.8) 5515 (19.2) 6065 (21.1)
3 14,560 2342 (16.1) 2816 (19.3) 3044 (20.9)
4 17,833 2425 (13.6) 3044 (17.1) 3356 (18.8)
5 12,342 1631 (13.2) 2564 (20.8) 3143 (25.5)
6 83,266 16,522 (19.8) 20,734 (24.9) 21,403 (25.7)
7 9309 1055 (11.3) 1304 (14.0) 1335 (14.3)
8 10,160 848 (8.4) 1061 (10.4) 1170 (11.5)
9 16,463 2775 (16.9) 3357 (20.4) 3586 (21.8)
10 65,990 8374 (12.7) 11,185 (17.0) 12,624 (19.1)
11 6239 1712 (27.4) 2174 (34.9) 2420 (38.8)
12 17,078 3169 (18.6) 4105 (24.0) 4534 (26.6)
13 16,794 2281 (13.6) 2945 (17.5) 3209 (19.1)
14 20,977 2884 (13.8) 3722 (17.7) 4315 (20.6)
15 12,442 1777 (14.3) 2144 (17.2) 2331 (18.7)
16 11,396 1588 (13.9) 2101 (18.4) 2308 (20.3)
17 44,092 7708 (17.5) 9213 (20.9) 9843 (22.3)
18 15,774 1150 (7.3) 1550 (9.8) 1671 (10.6)
19 13,057 822 (6.3) 1050 (8.0) 1102 (8.4)
20 32,055 7263 (22.7) 8640 (27.0) 9076 (28.3)
21 21,408 1758 (8.2) 2365 (110.1) 2753 (12.9)
22 14,068 709 (5.0) 891 (6.3) 960 (6.8)
23 17,831 12,169 (68.3) 13,003 (72.9) 13,154 (73.8)
24 38,390 3201 (8.3) 4206 (11.0) 4985 (13.0)
25 14,967 2540 (17.0) 2987 (20.0) 3092 (20.7)
26 12,122 1615 (13.3) 1760 (14.5) 1760 (14.5)
27 23,954 3087 (12.9) 3881 (16.2) 4262 (17.8)
28 28,114 2496 (8.9) 2795 (9.9) 2856 (10.2)
29 10,281 1139 (11.1) 1209 (11.8) 1210 (11.8)
30 23,110 2263 (9.8) 2492 (10.8) 2495 (10.8)
31 18,669 978 (5.2) 1086 (5.8) 1121 (6.0)
32 22,847 3380 (14.8) 4129 (18.1) 4592 (20.1)
33 9443 1060 (11.2) 1299 (17.8) 1415 (15.0)
34 7591 926 (12.2) 1249 (16.4) 1419 (18.7)
35 103,583 13,458 (13.0) 16,968 (16.4) 18,355 (17.7)
36 104,536 10,239 (9.8) 12,189 (11.7) 13,391 (12.8)
37 11,916 1499 (12.6) 1932 (16.2) 2015 (16.9)
38 12,583 3410 (27.1) 4109 (32.7) 4503 (35.8)
39 14,096 1798 (12.8) 2141 (15.2) 2179 (15.5)
Total 976,299 141,686 (14.5) 174,360 (17.9) 188,048 (19.3)

In the population with confirmed eGFR <60 ml/min per 1.73 m2, the proportion with ACR testing was slightly greater than that of the population with nonconfirmed eGFR <60 ml/min per 1.73 m2 (Supplemental Table 3). Of the 634,259 patients, 18% had ACR testing within 1 year, 22% had testing within 2 years, and 23% had testing within 3 years before the index date.

Characteristics of Patients On the Basis of Frequency and Type of Albuminuria Testing

Of the 976,299 participants, 188,048 (19%) had ACR testing (group 1), 16,130 (2%) had PCR testing (group 2), 354,627 (36%) had urine dipstick protein levels (group 3), and 417,494 (43%) did not have any type of albuminuria testing (group 4) in the 3 years before index date (Table 3). Mean age was similar across the groups; however, the proportion of patients with diabetes was much higher in group 1, and eGFR was much lower in group 2. Hypertension was less common in groups 3 and 4.

Table 3. - Characteristics of patients, on the basis frequency and type of albuminuria testing within 3 yr
Characteristic Group 1: Has Albumin-Creatinine Ratio Test Group 2: Has Protein-To-Creatinine Ratio Test Group 3: Has Proteinuria Dipstick Test Group 4: Has No Albuminuria Test
N (%) 188,048 (19) 16,130 (2) 354,627 (36) 417,494 (43)
Age, yr 71 (10) 69 (13) 72 (11) 72 (11)
Female sex (%) 55 48 62 58
eGFR (ml/min per 1.73 m2) 46 (11) 37 (13) 47 (11) 47 (11)
Median (IQR) ACR/PCR/dipstick level, (mg/g, mg/g, category) 21 (8–79) 210 (100–670) Negative NA
Diabetes 75.5% 33.8% 20.1% 21.2%
Hypertension 84.7% 79.0% 71.3% 63.8%
ESKD events (N) 8972 2045 7976 10,660
ESKD incidence, events per 1000 person-yr, 95% CI 10.49 (10.27–10.71) 31.95 (30.60–33.37) 5.31 (5.19–5.42) 5.98 (5.87–6.09)
Death incidence, events per 1000 person-yr, 95% CI 60.55 (60.03–61.08) 80.47 (78.35–82.65) 74.93 (74.50–75.38) 68.32 (67.94–68.71)
Group 1, available ACR test. Group 2, available PCR, but no available ACR. Group 3, available urine dipstick protein level measurement, but no available ACR or PCR. Group 4, no form of albuminuria measured. 95% CI, 95% confidence interval IQR, interquartile range; ACR, albumin-creatinine ratio; PCR, protein-to-creatinine ratio.

Performance of the KFRE in Group 1 (ACR Testing Available)

In group 1, where ACR was available, the incidence of ESKD was 10.49 patients per 1000 person-years and median C-statistic of the four-variable KFRE across 39 HCOs was 0.895. As expected, it outperformed the three-variable KFRE in 36 HCOs, for a meta-analyzed difference in C-statistic of 0.039 (0.036–0.043) (Table 4, columns 2–3). The calibration was improved for both the 2- and 5-year KFRE using the four-variable risk equation compared with the three-variable risk equation (Supplemental Figure 1). The Greenwood-Nam-D’Agostino statistic was smaller in 16 out of 20 HCOs (P=0.001) for the 2-year and 25 out of 36 HCOs (P=0.02) for the 5-year risk equation. The meta-analyzed net reclassification index was 0.139 (95% confidence interval [95% CI], 0.108 to 0.171) and 0.159 (95% CI, 0.137 to 0.180) at 2 and 5 years, respectively (Supplemental Table 4).

Table 4. - C-statistics in groups with albumin-creatinine ratio, protein-to-creatinine ratio/dipstick, and no urine testing within 3 yr
Health Care Organization Identification Group 1: Has Albumin-Creatinine Ratio Test Group 2: Has Protein-To-Creatinine Ratio Test Group 3: Has Proteinuria Dipstick Test Group 4: Has No Albuminuria Test
Three-variable Four-variable Three-variable Four-variable Three-variable Four-variable Three-variable
1 0.840 0.888 0.894 0.922 0.872
2 0.860 0.912 0.902 0.917 0.915
3 0.794 0.836 0.915 0.935 0.865
4 0.886 0.909 0.911 0.929 0.931
5 0.869 0.907
6 0.856 0.894 0.790 0.804 0.876 0.888 0.885
7 0.866 0.920 0.900
8 0.905 0.921 0.920 0.940 0.911
9 0.852 0.898 0.931
10 0.826 0.861 0.840 0.864 0.876 0.895 0.887
11 0.862 0.919 0.840 0.863 0.877
12 0.859 0.892 0.925 0.937 0.869
13 0.815 0.879 0.867 0.888 0.867 0.892
14 0.843 0.884 0.845 0.877 0.851 0.841 0.903
15 0.894 0.912 0.887 0.900 0.923
16 0.867 0.900 0.803 0.830 0.916 0.916 0.889
17 0.864 0.903 0.885 0.902 0.901
18 0.871 0.925 0.840 0.866 0.926
19 0.880 0.897 0.817 0.823 0.855 0.864 0.897
20 0.867 0.909 0.898 0.914 0.921
21 0.854 0.891 0.879 0.887 0.922
22 0.812 0.849 0.898 0.906 0.881
23 0.867 0.897 0.902 0.917 0.960
24 0.903 0.930 0.901 0.918 0.868
25 0.891 0.932 0.892 0.913 0.927
26 0.833 0.877 0.817 0.824 0.882
27 0.851 0.905 0.897 0.908 0.862
28 0.838 0.889 0.763 0.789 0.851 0.868 0.867
29 0.852 0.876 0.884 0.894 0.889
30 0.762 0.854 0.879 0.889 0.898
31 0.844 0.885 0.904 0.915 0.906
32 0.842 0.882 0.877 0.868 0.921
33 0.860 0.895 0.835 0.833 0.930
34 0.870 0.915 0.843 0.850 0.932
35 0.829 0.881 0.852 0.884 0.859 0.881 0.889
36 0.844 0.883 0.861 0.878 0.881
37 0.833 0.868 0.794 0.814 0.887 0.901 0.929
38 0.897 0.930
39 0.845 0.893 0.869 0.888 0.887
Median C-statistic 0.856 0.895 0.828 0.847 0.885 0.897 0.899
Meta-analyzed difference in C-statistic Meta-analyzed difference in C-statistic Meta-analyzed difference in C-statistic
0.039 (0.036–0.043) 0.022 (0.017–0.026) 0.012 (0.010–0.015)
Group 1, available ACR test. Group 2, available PCR, but no available ACR. Group 3, available urine dipstick protein level measurement, but no available ACR or PCR. Group 4, no form of albuminuria measured. Bold indicates that the difference in C-statistic between the 4-variable and 3-variable KFRE in the group is statistically significant. ACR, albumin-creatinine ratio; PCR, protein-to-creatinine ratio.

Performance of the KFRE in Group 2 and 3 (PCR or Dipstick Protein Testing Available)

In group 2, where PCR was available, the incidence of ESKD was 32.0 patients per 1000 person-years and the median C-statistic of the four-variable KFRE using an imputed value for ACR across ten HCOs was 0.847. This outperformed the three-variable KFRE in nine HCOs, for a meta-analyzed difference in C-statistic of 0.022 (0.017–0.026) (Table 4, columns 4–5). Calibration was adequate for both the three-variable and four-variable KFRE (Supplemental Figure 2). The meta-analyzed net reclassification index was 0.099 (95% CI, 0.057 to 0.140) and 0.136 (95% CI, 0.105 to 0.0168) at 2 and 5 years, respectively (Supplemental Table 4).

In group 3, where only dipstick protein was available, the incidence of ESKD was 5.31 patients per 1000 person-years and the four-variable KFRE using an imputed value for ACR again outperformed the three-variable equation (difference in C-statistic, 0.012; 95% CI, 0.010 to 0.015) (Table 4, columns 6–7). Observed risk of kidney failure was similar to predicted risk for most HCOs for the four-variable KFRE but not the three-variable KFRE, which showed systematic bias, with predicted risk greater than observed risk (Supplemental Figure 3). Greenwood-Nam-D’Agostino statistic was smaller in 17 out of 21 HCOs (P=0.001) for 2 years and 28 out of 32 HCOs (P<0.0001) for 5 years. If the three-variable equation were used to classify 2-year risk, 0.4% of those with <10% predicted risk developed kidney failure, 6% of those with 10%–19% predicted risk, 14% of those with 20%–39% predicted risk, and 26% of those with 40% predicted risk. In crossclassification, using imputed ACR from dipstick and the four-variable equation resulted in lower predicted risk than the three-variable equation for most patients. Thus, the net reclassification index for events (N=3682) resulted in a negative value, with a positive value only for the net reclassification for the nonevents (N=350,945).

For group 4, where no measure of albuminuria was available, the incidence of ESKD was relatively rare, at 5.98 patients per 1000 person-years, similar to group 3. The three-variable KFRE performed fairly well, with a median C-statistic of 0.887, although predicted risk was greater than observed risks in most of HCOs, demonstrating systematic bias (Supplemental Figure 4, Tables 4 and 5, column 8).

When differences in C-statistics were meta-analyzed separately within CKD stages G3 and G4–5, the four-variable equation performed better than the three-variable equations for all groups (Supplemental Table 5).

Discussion

Efforts to incorporate the KFRE in EHRs are underway, with the goal of better informing referral to nephrology, multidisciplinary care, and transplant evaluation or vascular access creation (1617181920–21). However, in this study of more than one million patients with eGFR <60 ml/min per 1.73 m2, only 19% of participants had the necessary ACR testing to calculate the four-variable KFRE. An additional 38% had PCR or dipstick protein available, which, with our recently published equation for the conversion of PCR or dipstick protein to ACR, may now be used to impute ACR and calculate the four-variable KFRE. We demonstrate that, for those patients with urine PCR or dipstick protein testing only, using the four-variable KFRE with imputed ACR—despite the known inaccuracy in conversion—was a better option than using the three-variable KFRE without ACR in most patients.

Although the overall ESKD risk in participants with no measures or only dipstick protein measures of albuminuria was low, the three-variable KFRE was not well calibrated in this group. Incorporating measures of dipstick protein tended to lower the KFRE risk prediction, which was useful for the large number of patients without ESKD events. In contrast, patients with events also had lower risk estimates. This may be in part due to the crude conversion: the imputed ACR value from urine dipstick protein has a maximum value of 1000 mg/g of creatinine. Our results suggest individuals with high-risk estimates using the three-variable KFRE may benefit from a more quantitative measure of albuminuria for risk assessment. In general, albuminuria testing remains underutilized, yet vitally important for prognostication.

Previous smaller studies have also reported low rates of albuminuria testing in patients with CKD. The Cleveland Clinic reported 36% of patients with CKD had no proteinuria assessed (8). A British cohort of 12,988 patients showed 36% had ACR testing over a 7-year period, and only 17% had ACR testing within the first year of registration of CKD. Of note, after ACR testing was included in the United Kingdom’s National Institute for Health and Care Excellence Quality and Outcomes Framework CKD targets and there were changes in incentives, ACR testing increased nearly 10% compared with the prior year (22,23). The reason why ACR is not tested more frequently in patients with CKD is uncertain (2425–26). Some suggest an unclear rationale for testing and confusion as to the appropriate method for measuring albuminuria (27,28). Although Kidney Disease: Improving Global Outcomes (KDIGO) has consistently recommended measuring ACR if possible, other guidelines have stressed ACR measurement primarily in patients with diabetes (27,29).

Our study builds on the existing evidence that ascertainment of albuminuria is central to ESKD prognosis (3,4). We found that the four-variable KFRE (with albuminuria) consistently performed better that the three-variable KFRE (without albuminuria), even when ACR was imputed from PCR or urine dipstick protein levels. Others have evaluated the contribution of albuminuria in different ways. In a study from Australia, spot PCR, spot ACR, and 24-hour urine protein assessment were all strongly associated with the composite outcome of death, ESKD, or >30% decline in eGFR (30). In a study estimating the time to ESKD, incorporating age, sex, proteinuria, and eGFR provided a better estimate compared with eGFR alone, supporting the assertion that kidney failure risk should be used in consideration of referral for a nephrology consultation or referral to transplant evaluation rather than eGFR alone (31). In a large meta-analysis of cohorts from around the world, change in albuminuria was strongly associated with risk of ESKD (32,33).

The study has several limitations that should be considered when interpreting the results. There is variability of the laboratory methods across HCOs; serum creatinine and albuminuria tests were not analyzed in a central laboratory. Laboratory data available in the EHR-derived data from OLDW are received from participating provider networks, so there may have been additional laboratory tests performed that were not present in the data. ESKD was captured by diagnostic code and not linkage to the US Renal Data System (USRDS) registry, a method that may miss patients treated outside the provider network. That said, previous studies have demonstrated high accuracy for using ICD codes to identify patients with ESKD (13). A strength of the study is the large number of participants—nearly 5% of the nation’s estimated population with eGFR <60 ml/min per 1.73 m2—from diverse age, sex, ethnic groups, and geographic locations. In this study of over a million patients with eGFR <60 ml/min per 1.73 m2, incorporating albuminuria in ESKD risk prediction resulted in consistent improvement in risk discrimination, even when ACR was imputed from PCR or urine dipstick protein levels. However, very few patients with GFR <60 ml/min per 1.73 m2 received the necessary albuminuria testing. Additional policies or incentives should be instituted to encourage albuminuria testing for all patients with CKD.

Disclosures

A. Levey reports receiving research funding from grants and contracts paid to Tufts Medical Center from the National Institutes of Health (NIH) and the National Kidney Foundation (NKF); contracts to paid from AstraZeneca (DSMB); and reports receiving honoraria from academic medical centers for visiting professorships. J. Coresh reports consultancy agreements with Healthy.io, Kaleido, and Ultragenyx; having an ownership interest in Healthy.io; reports receiving research funding from the NKF and the NIH; and reports being a scientific advisor or member of Healthy.io and NKF. M. Grams reports receiving honoraria from academic institutions for giving grand rounds and American Society of Nephrology for Young Investigator Award; reports being a scientific advisor or member of AJKD, CJASN, and the JASN Editorial Fellowship Committee, NKF Scientific Advisory Board, KDIGO Executive Committee, USRDS Scientific Advisory Board; reports having other interests/relationships with the NKF, which receives funding from Abbvie, Relypsa, and Thrasos; reports receiving travel support from Dialysis Clinics, Inc. to speak at the annual meeting and KDIGO for participation in scientific meetings and the executive committee. S. Dunning reports having an ownership interest in UnitedHealth Group. All remaining authors have nothing to disclose.

Funding

Funding for the project included support from the US National Kidney Foundation and the National Institute of Diabetes and Digestive and Kidney Diseases (R01DK100446-01 and R01DK115534).

Supplemental Material

This article contains supplemental material online at http://kidney360.asnjournals.org/lookup/suppl/doi:10.34067/KID.0005592020/-/DCSupplemental.

Supplemental Appendix 1. Selection of health care organizations for participation in this study.

Supplemental Table 1. Sensitivity analysis of baseline characteristics of patients with eGFR <60 ml/min/1.73 m2, by health system center, in confirmed eGFR <60 ml/min/1.73 m2.

Supplemental Table 2. Incidence of ESKD and death, by center.

Supplemental Table 3. Sensitivity analysisproportion with ACR testing according to “look-back” window, in confirmed eGFR <60 ml/min/1.73 m2.

Supplemental Table 4. Cross classification of risk categories by the 3-variable KFRE and the 4-variable KFRE, by event status, in groups 1, 2, and 3.

Supplemental Table 5. Meta-analyzed difference in c-statistic between 3- and 4-variable equation in groups 1, 2, and 3 within stage G3 and stage G4-5 CKD.

Supplemental Figure 1. Calibration plot. Patients with ACR testing (group 1) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.

Supplemental Figure 2. Calibration plot. Patients with PCR testing (group 2) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.

Supplemental Figure 3. Calibration plot. Patients with dipstick protein testing (group 3) within 3 years, 2-year and 5-year 3-variable and 4-variable KFRE.

Supplemental Figure 4. Calibration plot. Patients without albuminuria testing (group 4) within 3 years, 2-year and 5-year 3-variable KFRE.

Author Contributions

F.S. Naranjo wrote the original draft; J. Coresh and M.E. Grams were responsible for funding acquisition; J. Coresh, M.E. Grams, and Y. Sang were responsible for methodology; M.E. Grams was responsible for conceptualization and supervision; M.E. Grams, N. Stempniewicz, and S.C. Dunning were responsible for data curation; N. Stempniewicz was responsible for resources, and software; S.C. Dunning and S.H. Ballew were responsible for project administration; Y. Sang was responsible for formal analysis; and all authors reviewed and edited the manuscript, and approved the final version of the manuscript.

References

1. Matsushita K, Coresh J, Sang Y, Chalmers J, Fox C, Guallar E, Jafar T, Jassal SK, Landman GWD, Muntner P, Roderick P, Sairenchi T, Schöttker B, Shankar A, Shlipak M, Tonelli M, Townend J, van Zuilen A, Yamagishi K, Yamashita K, Gansevoort R, Sarnak M, Warnock DG, Woodward M, Ärnlöv J; CKD Prognosis Consortium: Estimated glomerular filtration rate and albuminuria for prediction of cardiovascular outcomes: A collaborative meta-analysis of individual participant data. Lancet Diabetes Endocrinol 3: 514–525, 2015 https://doi.org/10.1016/S2213-8587(15)00040-6
2. Hallan S, Astor B, Romundstad S, Aasarød K, Kvenild K, Coresh J: Association of kidney function and albuminuria with cardiovascular mortality in older vs younger individuals: The HUNT II Study. Arch Intern Med 167: 2490–2496, 2007 https://doi.org/10.1001/archinte.167.22.2490
3. Tangri N, Stevens LA, Griffith J, Tighiouart H, Djurdjev O, Naimark D, Levin A, Levey AS: A predictive model for progression of chronic kidney disease to kidney failure. JAMA 305: 1553–1559, 2011 https://doi.org/10.1001/jama.2011.451
4. Tangri N, Grams ME, Levey AS, Coresh J, Appel LJ, Astor BC, Chodick G, Collins AJ, Djurdjev O, Elley CR, Evans M, Garg AX, Hallan SI, Inker LA, Ito S, Jee SH, Kovesdy CP, Kronenberg F, Heerspink HJL, Marks A, Nadkarni GN, Navaneethan SD, Nelson RG, Titze S, Sarnak MJ, Stengel B, Woodward M, Iseki K; CKD Prognosis Consortium: Multinational assessment of accuracy of equations for predicting risk of kidney failure: A meta-analysis [published correction appears in JAMA 315: 822, 2016 10.1001/jama.2016.0342]. JAMA 315: 164–174, 2016 https://doi.org/10.1001/jama.2015.18202
5. Grams ME, Coresh J: Assessing risk in chronic kidney disease: A methodological review. Nat Rev Nephrol 9: 18–25, 2013 https://doi.org/10.1038/nrneph.2012.248
6. Jafar TH, Stark PC, Schmid CH, Landa M, Maschio G, Marcantoni C, de Jong PE, de Zeeuw D, Shahinfar S, Ruggenenti P, Remuzzi G, Levey AS; AIPRD Study Group. Angiotensin-Converting Enzymne Inhibition and Progression of Renal Disease: Proteinuria as a modifiable risk factor for the progression of non-diabetic renal disease. Kidney Int 60: 1131–1140, 2001 https://doi.org/10.1046/j.1523-1755.2001.0600031131.x
7. Giatras I, Lau J, Levey A: Effect of angiotensin-converting enzyme inhibitors on the progression of nondiabetic renal disease: A meta-analysis of randomized trials. Angiotensin-converting-enzyme inhibition and progressive renal disease study group. Ann Intern Med 170: ITC33–ITC48, 2019
8. Jolly SE, Navaneethan SD, Schold JD, Arrigain S, Sharp JW, Jain AK, Schreiber MJ, Simon JF, Nally JV: Chronic kidney disease in an electronic health record problem list: Quality of care, ESRD, and mortality. Am J Nephrol 39: 288–296, 2014 https://doi.org/10.1159/000360306
9. Sumida K, Nadkarni GN, Grams ME, Sang Y, Ballew SH, Coresh J, Matsushita K, Surapaneni A, Brunskill N, Chadban SJ, Chang AR, Cirillo M, Daratha KB, Gansevoort RT, Garg AX, Lacoviello L, Kayama T, Konta T, Kovesdy CP, Lash J, Lee BJ, Major RW, Metzger M, Miura K, Naimark DMJ, Nelson RG, Sawhney S, Stempniewicz N, Tang M, Townsend RR, Traynor JP, Valdivielso JM, Wetzels J, Polkinghorne KR, Heerspink HJL; Chronic Kidney Disease Prognosis Consortium: Conversion of urine protein-creatinine ratio or urine dipstick to urine albumin-creatinine for use in chronic kidney disease screening and prognosis: An individual participant-based meta-analysis. Ann Intern Med 173: 426–435, 2020 https://doi.org/10.7326/M20-0529
10. McCoy RG, Nori VS, Smith SA, Hane CA: Development and validation of healthImpact: An incident diabetes prediction model based on administrative data. Health Serv Res 51: 1896–1918, 2016 https://doi.org/10.1111/1475-6773.12461
11. Levey AS, Stevens LA, Schmid CH, Zhang YL, Castro AF 3rd, Feldman HI, Kusek JW, Eggers P, Van Lente F, Greene T, Coresh J; CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration): A new equation to estimate glomerular filtration rate [published correction appears in Ann Intern Med 155: 408, 2011]. Ann Intern Med 150: 604–612, 2009 https://doi.org/10.7326/0003-4819-150-9-200905050-00006
12. Miller WG, Jones GRD: Estimated glomerular filtration rate; laboratory implementation and current global status. Adv Chronic Kidney Dis 25: 7–13, 2018 https://doi.org/10.1053/j.ackd.2017.09.013
13. Rebholz CM, Coresh J, Ballew SH, McMahon B, Whelton SP, Selvin E, Grams ME: Kidney failure and ESRD in the Atherosclerosis Risk in Communities (ARIC) study: Comparing ascertainment of treated and untreated kidney failure in a cohort study. Am J Kidney Dis 66: 231–239, 2015 https://doi.org/10.1053/j.ajkd.2015.01.016
14. Levin A, Stevens PE: Summary of KDIGO 2012 CKD guideline: Behind the scenes, need for guidance, and a framework for moving forward. Kidney Int 85: 49–61, 2014 https://doi.org/10.1038/ki.2013.444
15. Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work Group, KDIGO 2012 clinical practice guideline for the evaluation and management of chronic kidney disease. Available at: https://kdigo.org/wp-content/uploads/2017/02/KDIGO_2012_CKD_GL.pdf. Accessed January 21, 2021
16. Inston N, Lok CE: Improving precision in prediction: Using kidney failure risk equations as a potential adjunct to vascular access planning. J Vasc Access 20: 95–97, 2019 https://doi.org/10.1177/1129729818786630
17. Fluck R, Kumwenda M: Renal association clinical practice guideline on vascular access for haemodialysis. Nephron Clin Pract 118[Suppl 1]: c225–c240, 2011 https://doi.org/10.1159/000328071
18. Drew DA, Lok CE: Strategies for planning the optimal dialysis access for an individual patient. Curr Opin Nephrol Hypertens 23: 314–320, 2014 https://doi.org/10.1097/01.mnh.0000444815.49755.d9
19. Al-Balas A, Lee T, Young CJ, Barker-Finkel J, Allon M: Predictors of initiation for predialysis arteriovenous fistula. Clin J Am Soc Nephrol 11: 1802–1808, 2016 https://doi.org/10.2215/CJN.00700116
20. Shechter SM, Skandari MR, Zalunardo N: Timing of arteriovenous fistula creation in patients with CKD: A decision analysis. Am J Kidney Dis 63: 95–103, 2014 https://doi.org/10.1053/j.ajkd.2013.06.021
21. Bello AK, Ronksley PE, Tangri N, Kurzawa J, Osman MA, Singer A, Grill AK, Nitsch D, Queenan JA, Wick J, Lindeman C, Soos B, Tuot DS, Shojai S, Brimble KS, Mangin D, Drummond N: Quality of chronic kidney disease management in Canadian primary care. JAMA Netw Open 2: e1910704, 2019 https://doi.org/10.1001/jamanetworkopen.2019.10704
22. Fraser SDS, Parkes J, Culliford D, Santer M, Roderick PJ: Timeliness in chronic kidney disease and albuminuria identification: A retrospective cohort study. BMC Fam Pract 16: 18, 2015 https://doi.org/10.1186/s12875-015-0235-8
23. de Lusignan S, Gallagher H, Jones S, Chan T, van Vlymen J, Tahir A, Thomas N, Jain N, Dmitrieva O, Rafi I, McGovern A, Harris K: Audit-based education lowers systolic blood pressure in chronic kidney disease: The Quality Improvement in CKD (QICKD) trial results [published correction appears in Kidney Int 84: 1289, 2013]. Kidney Int 84: 609–620, 2013 https://doi.org/10.1038/ki.2013.96
24. Carrero JJ, Grams ME, Sang Y, Ärnlöv J, Gasparini A, Matsushita K, Qureshi AR, Evans M, Barany P, Lindholm B, Ballew SH, Levey AS, Gansevoort RT, Elinder CG, Coresh J: Albuminuria changes are associated with subsequent risk of end-stage renal disease and mortality. Kidney Int 91: 244–251, 2017 https://doi.org/10.1016/j.kint.2016.09.037
25. Eckardt KU, Coresh J, Devuyst O, Johnson RJ, Köttgen A, Levey AS, Levin A: Evolving importance of kidney disease: From subspecialty to global health burden. Lancet 382: 158–169, 2013 https://doi.org/10.1016/S0140-6736(13)60439-0
26. Stevens PE, Levin A; Kidney Disease: Improving Global Outcomes Chronic Kidney Disease Guideline Development Work Group Members: Evaluation and management of chronic kidney disease: Synopsis of the kidney disease: Improving global outcomes 2012 clinical practice guideline. Ann Intern Med 158: 825–830, 2013 https://doi.org/10.7326/0003-4819-158-11-201306040-00007
27. Johnson DW: Global proteinuria guidelines: Are we nearly there yet? Clin Biochem Rev 32: 89–95, 2011
28. Fraser SDS, Roderick PJ, Taal MW: Where now for proteinuria testing in chronic kidney disease?: Good evidence can clarify a potentially confusing message. Br J Gen Pract 66: 215–217, 2016 https://doi.org/10.3399/bjgp16X684721
29. Bermúdez RM, García SG, Surribas DP, Castelao AM, Sanjuán JB: Documento de Consenso. Recomendaciones sobre la valoración de la proteinuria en el diagnóstico y seguimiento de la enfermedad renal crónica. Nefrologia 31: 331–345, 2011
30. Ying T, Clayton P, Naresh C, Chadban S: Predictive value of spot versus 24-hour measures of proteinuria for death, end-stage kidney disease or chronic kidney disease progression. BMC Nephrol 19: 55, 2018 https://doi.org/10.1186/s12882-018-0853-1
31. Grams ME, Li L, Greene TH, Tin A, Sang Y, Kao WHL, Lipkowitz MS, Wright JT, Chang AR, Astor BC, Appel LJ: Estimating time to ESRD using kidney failure risk equations: Results from the African American Study of Kidney Disease and Hypertension (AASK). Am J Kidney Dis 65: 394–402, 2015 https://doi.org/10.1053/j.ajkd.2014.07.026
32. Coresh J, Heerspink HJL, Sang Y, Matsushita K, Arnlov J, Astor BC, Black C, Brunskill NJ, Carrero JJ, Feldman HI, Fox CS, Inker LA, Ishani A, Ito S, Jassal S, Konta T, Polkinghorne K, Romundstad S, Solbu MD, Stempniewicz N, Stengel B, Tonelli M, Umesawa M, Waikar SS, Wen CP, Wetzels JFM, Woodward M, Grams ME, Kovesdy CP, Levey AS, Gansevoort RT; Chronic Kidney Disease Prognosis Consortium and Chronic Kidney Disease Epidemiology Collaboration: Change in albuminuria and subsequent risk of end-stage kidney disease: An individual participant-level consortium meta-analysis of observational studies. Lancet Diabetes Endocrinol 7: 115–127, 2019 https://doi.org/10.1016/S2213-8587(18)30313-9
33. de Zeeuw D: Should albuminuria be a therapeutic target in patients with hypertension and diabetes? Am J Hypertens 17: 11S–15S, quiz A2–A4, 2004 https://doi.org/10.1016/j.amjhyper.2004.08.005
Keywords:

chronic kidney disease; albuminuria; electronic health records; kidney failure

Copyright © 2021 by the American Society of Nephrology