Introduction
The Kidney Disease Improving Global Outcomes guidelines (2012) recommend that patients be referred to a nephrologist when eGFR is <30 ml/min per 1.73 m2 or when there is significant proteinuria (urine albumin-to-creatinine ratio ≥300 mg/g). However, the rate of progression to ESKD after that point varies depending on the cause and severity of kidney disease and the existence of comorbidities (1 2 3 4 5
Many patients with CKD report having limited understanding of their illness (6 7 et al. (8 9–12 13 14 et al. (15
Given this gap in our knowledge, we recruited patients and their nephrologists from CKD clinics to assess their estimations of ESKD progression as well as calculated KFRE model–based estimations. We compared patients’ and physicians’ estimations with each other and with the KFRE. We compared patients’ and nephrologists’ risk estimations on the basis of whether the patients were more pessimistic or optimistic than their physicians. We then followed patients for 2 years and compared actual outcomes with the three initial estimations.
Materials and Methods
This study assessed patients at the CKD clinics of the Veterans Affairs (VA) Medical Center San Diego Healthcare System and the University of California, San Diego (UCSD) and the physicians who saw them at the index visit between July 2015 and June 2016. To be included, patients had to be ≥18 years old, be English speaking, have at least one prior visit to the clinic, and have a Modification of Diet in Renal Disease eGFR <60 ml/min per 1.73 m2 . Patients’ clinical information was gathered through chart review, including demographic data and comorbidities. For participating physicians, we collected data on years of practice and site of practice.
To determine patients’ perceived risk of ESKD at 2 years, patients were asked the following question: “On a scale of 0%–100%, please give your best estimate of the chance that you will need dialysis or a kidney transplant in the next 2 years.” Questionnaires were administered in the clinic. The Kidney Knowledge Survey (KiKS) asks about the physiologic functions of the kidneys, kidney failure symptoms, and treatments; it has been shown (11 16 17 5 18
At the end of the same visit, without being provided the KFRE data, the attending nephrologist for the patient was asked the following question: “On a scale of 0%–100%, without using any estimating equations, please give your best estimate of the risk that this patient will need dialysis or a kidney transplant in 2 years.”
The KFRE (15 2 ), serum bicarbonate (milliequivalents per liter), albumin (grams per deciliter), calcium (milligrams per deciliter), phosphorus (milligrams per deciliter), and urine albumin-to-creatinine ratio (milligrams per gram).
After inclusion, patients’ laboratory values and incidence rates of ESKD and mortality were ascertained by chart review. Patients were deemed to have reached ESKD if they received a kidney transplant or started outpatient dialysis. When applicable, death date was determined on the basis of available documentation.
The study population was divided into four groups on the basis of their 2-year KFRE risk: risk <1%, risk =1%–4.9%, risk =5%–14.9%, and risk ≥15%. These KFRE cutoffs were thought to be clinically meaningful and divided the study population into somewhat equivalent population sizes. We compared baseline participants’ characteristics and questionnaire scores between groups using chi-squared tests for categorical variables and ANOVA for continuous variables.
We defined patients as being optimistic if the difference of “physician estimation” − “patient estimation” was high (≥20%) and pessimistic if that difference was low (≤−20%). This range was chosen empirically, because meaningful risk differences are highly situationally dependent. Patients and nephrologists were considered to be in agreement if the difference between their estimations was ≤20% in either direction.
Patients’, nephrologists’, and KFRE estimations of ESKD risk at the index visit were then compared with the actual incidence of ESKD (and death) at 2 years. We assessed bias by comparing estimated risk groups at baseline by patients, physicians, and KFRE with actual outcomes in those groups. Graphs comparing either patients’ or physicians’ predictions with KFRE risk were plotted, showing actual outcomes at 2 years. As a sensitivity analysis, we also examined results when the sample was limited to those participants with eGFR<30 ml/min per 1.73 m2 .
To assess the ability of each risk estimation to predict ESKD outcomes, we plotted receiver operating characteristic and calibration curves of the various predictions. We calculated the integrated discrimination index (IDI) (19
The UCSD and the VA institutional review boards approved this study. Patients and physicians consented to participate in adherence with the Declaration of Helsinki. SAS version 9.4 and SAS Enterprise version 7.1 were used for data analysis, with P values ≤0.05 considered statistically significant.
Results
We approached 335 patients for the study, and 257 enrolled, of whom 86 were veterans; 18 were excluded due to poor vision or language barrier, and 60 declined to participate. About 74% of the sample were men, 59% were white, and the average age was 65 (±13) years old (Table 1 ). Almost 42% completed college, and 57% were past or current smokers. Close to one half of the population had diabetes, and 83% had hypertension. The mean baseline eGFR was 34 (±13) ml/min per 1.73 m2 , and it was measured on average within 9 days of enrollment. Almost 62% (159 of 257) did not know their serum creatinine, and 67% (173 of 257) did not know their eGFR. The 13 participating nephrologists had 14.8 (±9.4) years in practice on average; seven were solely at the UCSD, five were solely at the VA, and one was at both sites.
Table 1. -
Baseline participants’ characteristics by kidney failure risk equation score
Patient Characteristics
KFRE<1%, n =89
KFRE=1%–4.9%, n =68
KFRE=5%–14.9%, n =43
KFRE≥15%, n =55
Age, yr, mean (±SD)
66 (13)
67 (11)
62 (17)
61 (14)
Men, n (%)
71 (80)
48 (71)
28 (65)
40 (73)
Completed college, n (%)
38 (43)
29 (43)
19 (44)
21 (39)
Race, n (%)
Black
9 (10)
6 (9)
7 (16)
5 (9)
White
58 (66)
44 (65)
20 (47)
27 (49)
Other
21 (24)
18 (26)
16 (37)
23 (42)
Smoking status, n (%)
Never smoker
36 (40)
27 (40)
24 (56)
22 (40)
Past smoker
47 (53)
35 (51)
17 (39)
26 (47)
Current smoker
6 (7)
6 (9)
2 (5)
7 (13)
Veterans, n (%)
22 (25)
25 (37)
15 (35)
23 (42)
Diabetes, n (%)
26 (29)
38 (56)
26 (60)
31 (56)
Hypertension, n (%)
69 (78)
60 (88)
33 (79)
49 (89)
eGFR baseline, ml/min per 1.73 m2 , mean (±SD)
47 (6)
35 (6)
27 (7)
17 (6)
KiKS score, mean (±SD)
19 (3)
18 (4)
20 (4)
21 (3)
REALM score, mean (±SD)
7 (1)
7 (1)
7 (1)
7 (1)
MMAS-4 score, mean (±SD)
7 (1)
8 (1)
7 (1)
7 (1)
PHQ-2 score, mean (±SD)
1 (1.5)
1 (1.3)
1 (1.5)
1 (1.6)
Do not know their creatinine, n (%)
52 (58)
43 (63)
29 (67)
33 (60)
Do not know their GFR, n (%)
58 (65)
46 (68)
31 (72)
36 (65)
Do not know cause of their CKD, n (%)
51 (57)
42 (62)
23 (53)
35 (64)
Pessimistic patients, n (%)
5 (6)
13 (19)
8 (19)
7 (13)
Optimistic patients, n (%)
6 (7)
19 (28)
19 (44)
26 (47)
Recall talking about dialysis, %
16 (18)
19 (28)
25 (58)
44 (80)
Patient 2-yr estimation, % (±SD)
7 (15)
16 (22)
25 (28)
45 (38)
Physician 2-yr estimation, % (±SD)
8 (8)
16 (16)
37 (26)
64 (31)
2-yr KFRE average, % (±SD)
1 (0.2)
3 (1)
9 (3)
45 (24)
ESKD at 2 yr, n (%)
1 (1)
1 (1)
4 (9)
27 (49)
Mean eGFR at time of ESKD (±SD)
7
5
9 (3)
9 (3)
% Patients giving same estimation (n )
64 (57)
6 (4)
23 (10)
65 (36)
% Patients whose nephrologist gave same estimation (n )
13 (12)
7 (5)
14 (6)
95 (52)
Death at 2 yr, n (%)
3 (3)
7 (10)
5 (12)
10 (18)
KFRE, kidney failure risk equation; KiKS, Kidney Knowledge Survey; REALM, Rapid Estimate of Adult Literacy in Medicine; MMAS-4, Morisky Medication Adherence Scale-4; PHQ-2, Patient Health Questionnaire-2.
Demographic missing data included the following: level of education (one), ethnicity (one), and hypertension (two). Eighteen patients were lost to follow-up, including one who had already reached ESKD 69 days into the study; the time to follow-up loss was 378 days on average (range, 43–656 days). These participants were considered alive without ESKD.
Median follow-up time was 722 days (interquartile range, 636–791 days; range, 23–888 days), although events (ESKD or death) for comparison with the 2-year estimations were recorded at 2 years of follow-up. At 2 years, almost 13% of patients (33 of 257) reached ESKD, and 9% (25 of 257) died. Among those who reached ESKD, median time to ESKD was 272 days (interquartile range, 122–482 days; range, 14–730 days) and the mean eGFR at the time of ESKD was 9 (±3) ml/min per 1.73 m2 for the 29 of 33 patients with available data. Three patients underwent temporary dialysis for AKI during follow-up (not considered ESKD).
Differences in Risk Estimations at the Index Visit
About 27% of patients gave estimations that were >20% more optimistic than their physicians (in terms of difference “physician estimation” − “patient estimation”), and 13% were >20% more pessimistic (Table 2 ). When comparing these groups, pessimistic patients were younger (58 versus 68 years old; P =0.001), were less likely to be veterans (15% versus 47%; P =0.004), and remembered talking about dialysis (64% versus 49%; P =0.001). Interestingly, pessimistic patients were less educated on average than optimistic patients (fewer obtained a college degree), although this difference was not statistically significant. Their average KFRE was about 10% at 2 years versus 20% for optimistic patients (P <0.001). By contrast, measures of medical literacy were not different across groups (the KiKS and the REALM-SF).
Table 2. -
Baseline participants’ characteristics on the basis of whether patients’ estimations are more pessimistic or optimistic than nephrologists’ estimations
Patient characteristics
Pessimistic Patients, n =33
Agreement Physicians-Patients, n =154
Optimistic Patients, n =70
Age, yr, mean (±SD)
58 (14)
65 (13)
68 (12)
Men, n (%)
21 (64)
114 (74)
54 (77)
Completed college, n (%)
13 (39)
63 (41)
31 (44)
Race, n (%)
Black
4 (12)
13 (9)
10 (14)
White
17 (52)
92 (60)
41 (59)
Other
12 (36)
48 (31)
19 (27)
Smoking status, n (%)
Never smoker
17 (52)
65 (42)
28 (40)
Past smoker
12 (36)
74 (48)
40 (57)
Current smoker
4 (12)
15 (10)
2 (3)
Veterans, n (%)
5 (15)
84 (31)
33 (47)
Diabetes, n (%)
17 (52)
64 (42)
41 (59)
Hypertension, n (%)
26 (81)
126 (82)
61 (87)
eGFR baseline, ml/min per 1.73 m2 , mean (±SD)
33 (12)
38 (13)
26 (11)
KiKS score, mean (±SD)
20 (3)
19 (3)
19 (5)
REALM score, mean (±SD)
7 (1)
7 (1)
7 (1)
MMAS-4 score, mean (±SD)
7 (1)
7 (1)
8 (1)
PHQ-2 score, mean (±SD)
1.5 (1.4)
1.1 (1.5)
1.1 (1.4)
Do not know their creatinine, n (%)
16 (48)
96 (62)
47 (67)
Do not know their GFR, n (%)
21 (64)
99 (64)
53 (76)
Do not know cause of their CKD, n (%)
25 (76)
90 (58)
38 (54)
Recall talking about dialysis, n (%)
21 (64)
50 (32)
34 (49)
Patient 2-yr estimation, % (±SD)
60 (23)
16 (28)
10 (17)
Physician 2-yr estimation, % (±SD)
17 (17)
18 (27)
51 (28)
2-yr KFRE average, % (±SD)
10 (13)
9 (19)
20 (24)
ESKD at 2 yr, n (%)
4 (12)
16 (10)
13 (19)
Death at 2 yr, n (%)
3 (9)
12 (8)
10 (14)
KiKS, Kidney Knowledge Survey; REALM, Rapid Estimate of Adult Literacy in Medicine; MMAS-4, Morisky Medication Adherence Scale-4; PHQ-2, Patient Health Questionnaire-2; KFRE, kidney failure risk equation.
When comparing the three risk estimations, the strongest correlation was between physicians’ and KFRE estimations (r =0.72; P <0.001). Correlation was 0.50 (P <0.001) between physicians’ and patients’ estimations and 0.47 (P <0.001) between patients’ and KFRE estimations.
Differences between Risk Estimations and Actual Event Rates
Actual events (ESKD or death) tended to cluster among those with high ESKD risk estimations but with substantial discrepancies in these estimations (Table 1 ). Table 3 shows the relative bias of each estimation categorized by the percentage chance of ESKD estimated by the patients, physicians, or KFRE versus the actual percentage of outcomes in that group. For physicians, estimations were skewed well above actual risk; no patient whose physician estimated a risk of ESKD <15% actually had ESKD at 2 years. For patients, estimations were also skewed above actual outcomes across most risk groups but less so than physicians’ estimations. The KFRE was generally the least biased compared with actual risk.
Table 3. -
Study population categorized by the percentage risk of ESKD estimated by patients, physicians, or KFRE versus the actual percentage of outcomes in that group
Estimation Source
<1%
1%–4.9%
5%–14.9%
15%–59.9%
60%–79.9%
≥80%
Physicians’ estimation
n
22
29
69
91
22
24
With ESKD at 2 yr, % (n )
0 (0)
0 (0)
0 (0)
9 (8)
32 (7)
75 (18)
Patients’ estimation
n
107
8
52
58
11
21
With ESKD at 2 yr, % (n )
4 (4)
0 (0)
8 (4)
12 (7)
46 (5)
62 (13)
KFRE estimation
n
89
68
43
40
8
7
With ESKD at 2 yr, % (n )
1 (1)
1 (1)
9 (4)
35 (14)
75 (6)
100 (7)
KFRE, kidney failure risk equation.
Figure 1A compares physicians’ and KFRE estimations with ESKD outcomes. Figure 1B compares patients’ and KFRE predictions with ESKD outcomes. Among patients whose KFRE and physicians’ estimations of ESKD risk were both ≥50%, 13 of 17 reached ESKD, and three of the remaining four died before reaching ESKD. A similar result was found among those with both KFRE and patients’ estimations ≥50%, where all patients reached ESKD except for one who died.
Figure 1.: (A) There was substantial discordance between the estimates of 2-year ESKD risk between physicians (panel a) or patients (panel b) and an empiric risk calculator. Blue dots, patients who have not reached ESKD at 2 years; red dots: patients who have reached ESKD at 2 years. (B) Comparison between KFRE risk and patients’ estimations of the 2-year ESKD risk with actual outcomes. Blue dots, patients who have not reached ESKD at 2 years; red dots, patients who have reached ESKD at 2 years.
Quality of Risk Prediction by Patients, Physicians, and KFRE
The KFRE estimation, with a cutoff of 15%, had a specificity of 88% and a sensitivity of 82%. Figure 2 shows the receiver operating characteristic curves for all three predictions; area under the curve (AUC) for KFRE was 0.91, AUC for physicians’ estimations was 0.92, and AUC for patients’ predictions was 0.82. The AUCs for KFRE and physicians’ estimations were not statistically different (P =0.77). Figure 3 shows calibration plots for all three predictors, with KFRE having the best calibration.
Figure 2.: The receiver operating characteristic curves for all three estimations (patients', physicians’, and KFRE) were good, with AUC >0.8. Blue line, ROC curve for KFRE (area under the curve [AUC] =0.9079); green line, ROC curve for patients’ estimations (AUC=0.82); red line, ROC curve for physicians’ estimations (AUC=0.92);
Figure 3.: (A) The kidney failure risk equation (panel A) was better calibrated to actual ESKD risk than were physicians (panel B) or patients (panel C). Physicians and patients both tended to overestimate risk in comparison to actual outcomes.
The IDI for the combined model (KFRE and physicians’ predictions) compared with the KFRE alone model was 0.08 (95% confidence interval, 0.02 to 0.14). The IDI for adding KFRE to the physicians’ alone model yielded an analogous result (0.07; 95% confidence interval, 0.01 to 0.12). We computed Brier scores of 0.071 for KFRE, 0.078 for patients’ predictions, and 0.067 for physicians’ predictions, all indicative of reasonable discrimination.
Analyses on the Subsample with CKD Stages 4 and 5
Other than a higher rate of ESKD in this group and a corresponding shift to higher risk estimates across all three perspectives (physician, patient, and KFRE), analyses on the subsample of 100 participants with eGFR<30 ml/min per 1.73 m2 were similar to those in the whole population.
Discussion
In our cohort of 257 prevalent patients in the CKD clinic, we found significant discordance between the three risk estimations at the index visit. Surprisingly, AUCs were generally high for all three estimations. However, correlation coefficients between patients and physicians, suggestive of communication of risk, were not high, and a substantial number of patients and physicians had important absolute differences (>20% in either direction) in their risk estimations. Overall, almost 13% of patients were pessimistic compared with their physicians. Among them, about 12% reached ESKD, and 9% died, reflecting the respective proportions found within the whole sample. Conversely, physicians gave grimmer estimations than patients for 27% of patients. Among this group of more optimistic patients, up to 19% had kidney failure, and 14% died.
We showed that the KFRE is accurate and precise in this population. As seen in Table 1 , the average KFRE for each group corresponds almost exactly to the percentage of patients actually reaching ESKD in that group (especially in the groups where KFRE is ≥5%). The c statistic showed that both KFRE and physicians’ estimations were precise rank-order predictors of ESKD risk, although nephrologists’ estimations were consistently higher than the KFRE. This means that physicians were able to accurately distinguish patients at higher risk of ESKD from those at lower risk and that they were correct in terms of ranking patients on the basis of this risk. However, their estimations for any given patient tended to be shifted upward compared with KFRE. Patients’ predictions were less precise in terms of rank ordering but only marginally less so than physicians, with a c statistic of 0.82.
The KFRE was a better calibrated tool for 2-year outcomes than physicians’ estimations or patients’ estimations as illustrated by the calibration curves. The IDI showed marginal improvement when combining the physicians’ predictions and the KFRE over either estimate alone.
We find it surprising that a risk equation would be more accurate in terms of risk prediction than experienced nephrologists who know the patients well, know their comorbidities, and have access to more data than those used for the KFRE. Although the physicians were excellent at ranking risk, perhaps nephrologists simply tend to assume that more of their patients will reach ESKD than an “impartial” model would suggest.
To our knowledge, this study is the first to assess the predictions of ESKD risk of patients with CKD and the accuracy of these estimations in relation to their physicians’ estimations, a model estimation, and actual disease outcomes. Estimating ESKD risk is essential, because starting dialysis is a major life change that requires preparation. Using KFRE to educate patients and calibrate nephrologists’ estimations downward may help improve decision making around timing of dialysis initiation and access placement.
In accordance with previous studies (7 8 20 i.e. , had more knowledge) than those with lower KFRE.
This study has several strengths. The large patient population and long follow-up enabled a substantial percentage of ESKD or death outcomes to occur. These events were confirmed via active chart review. Another strength is our ability to classify patients as in concordance or discordance with their physicians’ views of their risk and analyze their outcomes on the basis of this information. We were able to enroll patients at two different clinical sites, capturing variability in clinical approach and patient population.
One limitation of the study was that we did not ask nephrologists to estimate their patients’ perception of their ESKD risk, and therefore, we cannot comment on physicians’ awareness of their patients’ perceptions. We also did not assess whether patients had been referred for dialysis education before our interview and whether their risk perceptions changed over time. This study was conducted in a single center (both at the VA and the UCSD hospital but within the same nephrology practice), with a relatively small number of nephrologists. The median number of patients per physician was 15 (interquartile range, 7–22; range, 5–65), precluding statistically meaningful analyses of physicians’ characteristics (such as gender or years of experience) and its association with ESKD risk estimations.
Our findings that physicians’ estimations are systematically above KFRE should motivate nephrologists to recalibrate their prognostications. One potential action to take given these data would be to automatically provide patients and physicians with an objective risk estimation or provide it to the physicians to communicate with their patients. Additional studies are needed to evaluate how the KFRE could be integrated into routine clinical practice. For situations where the patient’s and the physician’s estimations radically differ from each other or the KFRE, the physician might be prompted to further investigate the reasons for this gap, refer for further education, or reconsider the patient’s risk. Knowing what patients understand of their illness and discussing ESKD risk are essential to plan treatment ahead of time.
Disclosures
None.
Acknowledgments
The authors would like to thank Dr. Navdeep Tangri and Tom Ferguson at the University of Manitoba for sharing the code used to compute the kidney failure risk equation.
O.A.P. and H.A.N. were supported by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) grant 5T32DK104717-02. D.E.R. was supported by the NIDDK grant K23 DK09152 and Veterans’ Affairs Merit award H160180 (Health Services Research and Development).
References
1. Tangri N, Grams ME, Levey AS, Coresh J, Appel LJ, Astor BC, Chodick G, Collins AJ, Djurdjev O, Elley CR, Evans M, Garg AX, Hallan SI, Inker LA, Ito S, Jee SH, Kovesdy CP, Kronenberg F, Heerspink HJ, Marks A, Nadkarni GN, Navaneethan SD, Nelson RG, Titze S, Sarnak MJ, Stengel B, Woodward M, Iseki K; CKD Prognosis Consortium: Multinational assessment of accuracy of equations for predicting risk of kidney failure: A meta-analysis. JAMA 315: 164–174, 201626757465
2. Chiu HHL, Tangri N, Djurdjev O, Barrett BJ, Hemmelgarn BR, Madore F, Rigatto C, Muirhead N, Sood MM, Clase CM, Levin A: Perceptions of prognostic risks in chronic kidney disease: A national survey. Can J Kidney Health Dis 2: 53, 201526688745
3. Tangri N, Kitsios GD, Inker LA, Griffith J, Naimark DM, Walker S, Rigatto C, Uhlig K, Kent DM, Levey AS: Risk prediction models for patients with chronic kidney disease: A systematic review. Ann Intern Med 158: 596–603, 201323588748
4. Institute of Medicine: Crossing the Quality Chasm: A New Health System for the 21st Century, Washington, DC, The National Academies Press, 2001
5. Morisky DE, Ang A, Krousel-Wood M, Ward HJ: Predictive validity of a medication adherence measure in an outpatient setting. J Clin Hypertens (Greenwich) 10: 348–354, 200818453793
6. Tuot DS, Davis E, Velasquez A, Banerjee T, Powe NR: Assessment of printed patient-educational materials for chronic kidney disease. Am J Nephrol 38: 184–194, 201323970127
7. Finkelstein FO, Story K, Firanek C, Barre P, Takano T, Soroka S, Mujais S, Rodd K, Mendelssohn D: Perceived knowledge among patients cared for by nephrologists about chronic kidney disease and end-stage renal disease therapies. Kidney Int 74: 1178–1184, 200818668024
8. Plantinga LC, Boulware LE, Coresh J, Stevens LA, Miller ER 3rd, Saran R, Messer KL, Levey AS, Powe NR: Patient awareness of chronic kidney disease: Trends and predictors. Arch Intern Med 168: 2268–2275, 200819001205
9. Tuot DS, Plantinga LC: What patients don’t know may hurt them: Knowledge and the perception of knowledge among patients with CKD. Kidney Int 80: 1256–1257, 201122126982
10. Wright Nunes JA, Wallston KA, Eden SK, Shintani AK, Ikizler TA, Cavanaugh KL: Associations among perceived and objective disease knowledge and satisfaction with physician communication in patients with chronic kidney disease. Kidney Int 80: 1344–1351, 201121832984
11. Wright JA, Wallston KA, Elasy TA, Ikizler TA, Cavanaugh KL: Development and results of a kidney disease knowledge survey given to patients with CKD. Am J Kidney Dis 57: 387–395, 201121168943
12. Pagels A, Söderquist BK, Heiwe S: Evaluating the Illness Perception Questionnaire on patients with chronic kidney disease in Sweden. Scand J Caring Sci 26: 474–484, 201222117559
13. Perlman RL, Finkelstein FO, Liu L, Roys E, Kiser M, Eisele G, Burrows-Hudson S, Messana JM, Levin N, Rajagopalan S, Port FK, Wolfe RA, Saran R: Quality of life in chronic kidney disease (CKD): A cross-sectional analysis in the Renal Research Institute-CKD study. Am J Kidney Dis 45: 658–666, 200515806468
14. Davison SN: End-of-life care preferences and needs: Perceptions of patients with chronic kidney disease. Clin J Am Soc Nephrol 5: 195–202, 2010
15. Tangri N, Stevens LA, Griffith J, Tighiouart H, Djurdjev O, Naimark D, Levin A, Levey AS: A predictive model for progression of chronic kidney disease to kidney failure. JAMA 305: 1553–1559, 201121482743
16. Arozullah AM, Yarnold PR, Bennett CL, Soltysik RC, Wolf MS, Ferreira RM, Lee SY, Costello S, Shakir A, Denwood C, Bryant FB, Davis T: Development and validation of a short-form, rapid estimate of adult literacy in medicine. Med Care 45: 1026–1033, 200718049342
17. Kroenke K, Spitzer RL, Williams JBW: The patient health questionnaire-2: Validity of a two-item depression screener. Med Care 41: 1284–1292, 200314583691
18. Morisky DE, Green LW, Levine DM: Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care 24: 67–74, 19863945130
19. Pencina MJ, D’Agostino RB Sr., D’Agostino RB Jr., Vasan RS: Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat Med 27: 157–172, 200817569110
20. Plantinga LC, Tuot DS, Powe NR: Awareness of chronic kidney disease among patients and providers. Adv Chronic Kidney Dis 17: 225–236, 201020439091
