Validation of Two Asian Noninvasive Risk Models for Predicting Prevalent Undiagnosed Chronic Kidney Disease in Diabetic Patients Receiving Care at a Reference Hospital in A Sub-Saharan African Country

A Cross-Sectional Study

Abstract

Undiagnosed chronic kidney disease (CKD) is common in people with diabetes mellitus. Validated noninvasive risk models are an attractive CKD screening option in diabetic patients to identify patients who are more likely to be diagnosed with CKD via biological tests. The study aimed to validate the Korean and Thai noninvasive CKD risk prediction models in African Type 2 diabetic patients. This was a hospital-based study. The Modification of Diet in Renal Disease (MDRD) and CKD Epidemiology Collaboration (CKD-EPI) equations were used to estimate the glomerular filtration rate (eGFR). CKD was defined as an eGFR <60 mL/min/1.73 m², and any nephropathy as eGFR <60 mL/min/1.73 m² and/or proteinuria. Discrimination was assessed and compared using c-statistics and non-parametric methods. Calibration performance was assessed before (original models) and after intercept adjustment. A total of 733 patients (421 men) aged 57.0 years (standard deviation = 10.4) were included. The MDRD equation identified 223 (30.4%) participants as having CKD and 377 (51.4%) participants with any nephropathy. The CKD-EPI equation identified fewer cases of CKD and any nephropathy with 194 (26.5%) and 357 (48.7%) cases, respectively. The original Korean model had the highest C-statistics of 0.696 (95% confidence interval: 0.654–0.739) for the outcome of eGFR <60 mL/min/1.73 m² (using the CKD-EPI equation). Discrimination was significantly better in men, older and overweight participants. Intercept adjustment markedly improved calibration. Asian models have modest discrimination and good calibration with modest adjustment in predicting undiagnosed CKD in African diabetic patients; limiting their consideration for use in diabetes care in this setting.

Introduction

The prevalence of chronic kidney disease (CKD) in sub-Saharan Africa is estimated at is 13.9%.[¹²] Undiagnosed CKD is common in diabetics causing premature death.[³⁴] The timely diagnosis and treatment of CKD can slow its progression, hence the necessity for simple screening tools.[³⁴] Risk models for estimating the likelihood of undiagnosed CKD in diabetics exist. Two models have been validated in a mixed-ancestry population in South Africa.[⁵] However, these findings may not be generalizable to black Africans.

Our objective was to assess the performance of two non-invasive risk models for predicting undiagnosed CKD in Type 2 diabetics receiving care at a tertiary hospital in Cameroon.

Subjects and Methods

Study design and setting

This was a hospital-based, cross-sectional study from November 2015 to February 2016. The Douala General hospital is a tertiary care facility located in the Littoral region of Cameroon. It has a well-equipped laboratory, which is accredited by the World Health Organization and the Center for Disease Control. This study was approved by the Institutional Ethics Committee for Research on Human Health, University of Douala (IECUD/501/02/2016/T). Administrative approval was obtained from the authorities of the Douala General Hospital.

File selection

Files of diabetic patients receiving routine outpatient care at the Douala General Hospital from January 1, 2009, to February 28, 2016, were reviewed. Included files had completed sociodemographic, clinical, and laboratory data collected. All measurements in these files were taken following standard operating procedures as described in further details in this section.

Identification of chronic kidney disease risk prediction models

From a systematic review of risk models for predicting CKD,[⁶] the Korean and Thai noninvasive logistic regression CKD risk prediction models were selected for validation.[⁷⁸] This choice of risk model was based on the following criteria: having diabetes as one of its risk factors and having solely noninvasively measured predictors.

The Korean risk model, published in 2011, was developed from a cross-sectional study in a South Korean population. This study included 6,565 participants aged 30 years and above. CKD was defined as estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m². The predictors in this model were: age, female gender, anemia, hypertension (HTN), diabetes, history of cardiovascular disease, and proteinuria.

The Thailand model was also published in 2011. The model was developed from a crosssectional study including 3459 Thais aged 18 years and above. The predictors in the Thailand model are: age, diabetes, HTN, and history of kidney stones (Table 1).

Table 1: Overview of the tested models of prevalent chronic kidney disease prediction and their performance for the original model and the intercept adjusted model based on Modification of Diet in Renal Disease estimated glomerular filtration rate.

Both models used the Modification of Diet in Renal Disease (MDRD) formula to calculate eGFR. Apart from the populations in which the respective models were developed, they have also been tested in the mixed-ancestry population of South Africa.[⁵]

Outcome variables

CKD was defined as eGFR<60 mL/min/1.73 m² and any nephropathy (eGFR<60 mL/min/ 1.73 m² and/or proteinuria) including any of the Kidney Disease Improving Global Outcomes Chronic Kidney Disease stages I to V. These definitions concord with the definitions used in the original models.[⁷⁸]

Diabetes care at the Douala Genera Hospital

During initial visits, patients are received by trained nurses who are in charge of filling in preliminary information on the medical file. Blood pressure is measured using OMRON^®(OMRON intelli sense HEM-907-E7, JAPAN) digital sphygmomanometer. Height is measured using a wall-mounted stadiometer and weight is measured using an analog weighing balance (SECA^® Inc.). Waist and hip circumferences are measured using a non-extensible measuring tape. Urine proteins (albumin) are measured using a urine dipstick (COMBI^® 1).

The patient is then sent to the endocrinologist who takes a detailed medical history and conducts a thorough clinical examination which is well documented. Laboratory investigations such as serum creatinine, serum lipid profile, and full blood count are requested at the initial visit and then annually, unless otherwise indicated. These results are interpreted by the endocrinologist and documented in the patient’s file. Files are arranged in alphabetical order and kept at the secretariat and retrieved during subsequent visits. The patient has a follow-up visit at least every three months. In the case where a patient has had several measurements carried out, only the most recent measurements were considered during this study.

Statistical Analysis

The CKD risk models of interest were validated in the overall sample, and then in sub-groups, using the original formulas with and without any recalibration. The predicted probability of undiagnosed CKD for each participant was estimated using the relevant predictors for each model.[⁷⁸] Models’ performance was assessed through discrimination and calibration. Discrimination (the ability of the models to distinguish those with prevalent undiagnosed CKD from those without the conditions) was assessed with C-statistics and non-parametric methods.[⁹] Calibration (agreement between the probability of the outcome of interest as estimated by the model, and the observed outcome frequencies) was assessed graphically by plotting the predicted risk against the observed outcome rate, supplemented with the Hosmer and Lemeshow goodness-of-fit test.[¹⁰¹¹] The agreement between the expected (E) and observed (O) CKD rates (E/O) was assessed overall and within prespecified groups of participants. We calculated the 95% confidence intervals (CIs) for the expected/observed probabilities (E/O) ratio assuming a Poisson distribution.[¹⁰] We also calculated the Yates slope (difference between mean predicted probability of CKD for participants with and without prevalent CKD, with higher values indicating better performance) and the Brier score (squared difference between predicted probability and actual outcome for each participant with values ranging between zero for a perfect prediction model and one for no match in prediction and outcome).[¹²¹³] To determine the optimal cut-off for maximizing the potential effectiveness of a model, we used the Youden’s J statistic (Youden’s index),[¹⁴] with sensitivity and specificity determined for each threshold.

To minimize differences in CKD prevalence between the development and test populations, and thus improve performance, models were recalibrated to the test-population-specific CKD prevalence using intercept adjustment.[¹⁵] The calculated correction factor is based on the mean predicted risk and the prevalence in the validation set and is the natural logarithm of the odds ratio of the mean observed prevalence and the mean predicted risk.[¹⁵] The main analysis focused on the total sample, and subgroups analyses were by sex, age (<60 years vs. ≥60 years) and body mass index (BMI) (<25 kg/m² vs. >25 kg/m²) for CKD based on the MDRD formula. Additionally, we conducted sensitivity analyses, to assess all the aforementioned aspects of model performance using the CKD Epidemiology Collaboration (CKD-EPI) equation[¹⁶] to estimate kidney function and define CKD. For all analyses, we used the statistical software R Version 3.2.2 (2015-08-14) (The R Foundation for statistical computing, Vienna, Austria). The level of statistical significance was set at P <0.05.

Results

General characteristics of study participants

A total of 1,930 files of diabetic patients were reviewed. Of these, 67 files of type 1 diabetic patients were excluded. Another 1130 files were excluded for being incomplete. Hence, a total of 733 records of type 2 diabetic patients were included (Figure 1). Among these, 421 (57.4%) were males. The mean age was 57.0 ± 10.4 years. Overall, men were significantly taller, had higher weights, and lower BMI but higher waist-to-hip ratios. Men were also more likely to have higher creatinine values, lower total cholesterol with lower high-density lipoprotein cholesterol but higher levels of triglycerides. The mean systolic blood pressure, diastolic blood pressure, fasting blood glucose, and glycated hemoglobin were not significantly different in men and women. These results are summarized in Table 2.

Figure 1: Flow chart of participants.

Table 2: General characteristics of the study participants by gender.

Kidney function and observed prevalent chronic kidney disease

The mean eGFR as estimated using the MDRD equation was 73.3 ± 27.6 mL/min/1.73 m² in the overall sample. Using this same equation, 30.4% (n = 223) had CKD (eGFR <60 mL/min/1.73 m²) with no gender difference. In addition, for any nephropathy defined by eGFR <60 mL/min/1.73 m² and/or albuminuria, 51.4% (377) were diagnosed with a nephropathy with no gender difference (Table 2). The mean eGFR using the CKD-EPI equation was 75.8 ± 25.4 mL/min/1.73 m². Using this equation, the prevalence was 26.5% for CKD and 48.7% for any nephropathy, with no gender difference (Table 2).

Discrimination of the prediction models

The values of C-statistics were 0.663 (95% CI: 0.620∓0.706) and 0.628 (0.671–0.628) for the Korean and Thai models, respectively for the prediction of MDRD-based CKD; and 0. 672 (0.634–0.711) and 0.591 (0.551–0.631) for the prediction of ‘any nephropathy’ (Table 1). The values of C-statistics using the CKDEPI formula to diagnose CKD were 0.696 (0.654–0.739) and 0.651 (0.608–0.695) for the Korean and Thai models, respectively; and 0. 688 (0.650–0.726) and 0.598 (0.558–0.637) for “any nephropathy (Figures 2, 3 and Table 3).

Figure 2: Receiver operating characteristic curves (ROC) showing the discrimination of the original Korean model.Yellow band represents 95% confidence interval; diagonal line is the line of no discrimination. Figure panels are for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the left panel and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the right panel; for eGFR based on MDRD (upper panels) and CKD-EPI (lower panels) equations.

Figure 3: Receiver operating characteristic curves (ROC) showing the discrimination of the original Thai model.Yellow band represents 95% confidence interval; diagonal line is the line of no discrimination. Figure panels are for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the left panel and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the right panel; for eGFR based on MDRD (upper panels) and CKD-EPI (lower panels) equations.

Table 3: Performance for the original model and the intercept adjusted model in the overall population-based Chronic Kidney Disease Epidemiology Collaboration equation defined chronic kidney disease.

Table 4: Hosmer and Lemeshow statistics for the original and adjusted models.

Calibration of the original Korean and Thai models

For the Korean model, regardless of the definition of the outcome, calibration curves were always above the diagonal line of perfect calibration, therefore indicating risk underestimation (Figures 4 and 5). Furthermore, the Expected/Observed (E/O) CKD rates were 0. 16 (95%CI: 0.14–0.18) for eGFR <60 mL/min and 0.095 (0.086–0.106) for any nephropathy based on MDRD formula (Table 1). Meanwhile, using the CKD-EPI equation, the E/O rates were 0.18 (0.16–0.21) for eGFR <60 mL/min and 0.10 (0.09–0.11) for any nephropathy (Table 3). The P-values for the Hosmer-Lemeshow statistics were all below 0.00001.

Figure 4: Calibration curves for the Korean model before intercept adjustment for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the first column and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the second column.For each figure panel the broken diagonal line at 45° represents the ideal calibration. Participants are grouped into percentiles across increasing estimated probability. The vertical lines at the bottom of the graph depict the frequency distribution of the calibrated probabilities. eGFR is from MDRD equation (first row) and CKD-EPI equation (second row).

Figure 5: Calibration curves for the Korean model after intercept adjustment for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the first column and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the second column.For each figure panel, the broken diagonal line at 45° represents the ideal calibration. Participants are grouped into percentiles across increasing estimated probability. The vertical lines at the bottom of the graph depict the frequency distribution of the calibrated probabilities. eGFR is from the MDRD equation (1^st row) and CKD-EPI equation (2^nd row)

Using the Thai model, calibration curves were mostly below the diagonal line of perfect calibration for the outcome of eGFR <60 mL/min/1.73 m², therefore indicating a risk overestimation; meanwhile it was mostly above the perfect calibration line for the outcome of any nephropathy, and therefore indicating risk underestimation, regardless of the estimator of the kidney function used (Figures 6 and 7). The E/O rates were 1.22 (1.07–1.39) and 0.72 (0.65–0.80) for eGFR<60 mL/min and any nephropathy, respectively, based on MDRD (Table 1). Using CKD-EPI equation, the E/O rates were 1.40 (1.21–1.61) and 0.76 (0.68–0.84) (Table 3).

Figure 6: Calibration curves for the Thai model before intercept adjustment for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the first column and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the second column.For each figure panel the broken diagonal line at 45° represents the ideal calibration. Participants are grouped into percentiles across increasing estimated probability. The vertical lines at the bottom of the graph depict the frequency distribution of the calibrated probabilities. eGFR is from MDRD equation (1^st row) and CKD-EPI equation (2^nd row).

Figure 7: Calibration curves for the Thai model after intercept adjustment for the outcome of CKD (eGFR <60 mL/min/1.73 m²) for the first column and “any nephropathy” (eGFR <60 mL/min/1.73 m² or proteinuria) for the second column.For each figure panel the broken diagonal line at 45° represents the ideal calibration. Participants are grouped into percentiles across increasing estimated probability. The vertical lines at the bottom of the graph depict the frequency distribution of the calibrated probabilities. eGFR is from MDRD equation (1^st row) and CKD-EPI equation (2^nd row).

Prediction of prevalent undiagnosed chronic kidney disease by the adjusted prediction models

As expected, simple intercept adjustments led to substantial improvement of the calibration performance of models for all outcomes regardless of the kidney function estimator. For MDRD-defined CKD, there was a residual overall risk underestimation for the two outcomes by the Korean model, and a perfect estimation by the Thai model (Table 1). The pattern was similar for CKD-EPI-defined CKD (Table 3). For MDRD-defined CKD, the calibration curves were in favor of a combination of lower risk strata underestimation and upper risk strata overestimation by the Korean model, and a selective lower risk strata underestimation by the same model for CKDEPI-defined CKD (Figures 4 and 5). The pattern for the Thai model was consistently in favour of a combination of lower risk stratum underestimation and upper risk stratum overestimation of the risk of both outcomes (Figures 6 and 7). Based on the Hosmer-Lemeshow statistic, the intercept-adjusted Thai model had a good agreement in estimating the risk of eGFR <60 mL/min/1.73 m², regardless of the estimator of the kidney function (both P <0.082). In all other cases, there was huge improvement, although the statistics were still in favor of significant disagreements (Table 5).

Table 5: C-statistics comparisons.

Prediction of undiagnosed chronic kidney disease in subgroups

For the Korean model, C-statistics were higher in older participants (0.720 vs. 0.679) for any nephropathy and similar (0.629 and 0.638) for CKD (Table 5); mostly within the same range in men and women, lean and overweight participants. The pattern of the discrimination was similar for outcome predictions in men and women; and across age and BMI subgroups by the Thai model, but with always lower estimates than observed with the Korean model. In general, discrimination at best was acceptable (Table 6). The E/O was in favor of perfect risk estimation of both outcomes in older participants; acceptable risk estimation in women and overweight participants; and risk underestimation in men. There was an 11% risk overestimation in overweight participants for the outcome of ‘any nephropathy (Table 6). These patterns of discrimination and calibration were similar for CKD-EPI-based outcomes prediction (Table 7).

Table 6: Discrimination and calibration statistics for chronic kidney diseases risk model performance in subgroups of participants by gender, age and body mass index, for the outcome of chronic kidney disease based on Modification of Diet in Renal Disease equation, predicted glomerular filtration rate with or without proteinuria.

Table 7: Discrimination and calibration statistics for chronic kidney diseases risk model performance in subgroups of participants by gender, age and body mass index, for the outcome of chronic kidney disease based on Chronic Kidney Disease Epidemiology Collaboration equation predicted glomerular filtration rate with or without proteinuria.

Diagnostic utility of models at their optimal sample-specific risk thresholds

For CKD outcomes based on the MDRD formula, the optimal threshold for the intercept-adjusted Korean model was 0.35 for eGFR <60 mL/min/1.73 m² with a sensitivity of 79% and a specificity of 46%. For “any nephropathy”, the optimal intercept-adjusted threshold was 0.34 with a sensitivity of 58% and a specificity of 68%. For the Thai model and for eGFR <60 mL/min/1.73 m², the sensitivity was 38% and specificity was 81% at an optimal threshold of 0.24. For “any nephropathy”, the sensitivity was 40% and specificity 76% at a threshold of 0.43 (Table 1). Using CKD-EPI equation or MDRD equation as the estimator of glomerular rate, the values of optimal threshold, sensitivity and specificity were similar (Table 3).

Discussion

In diabetic patients receiving routine care at the Douala General Hospital, the Korean and Thailand prediction models had a modest-toacceptable discriminatory ability to predict prevalent undiagnosed CKD, with better performance in men, older and overweight patients. The Korean model underestimated the risk of prevalent undiagnosed CKD, while the Thailand model overestimated risk. However, recalibration through intercept adjustment markedly improved calibration for the two models as expected. At their optimal threshold derived from our sample, the Korean model had better sensitivity whereas the Thai model had better specificity to select participants who are more likely to be diagnosed with CKD through biological tests. The MDRD or CKD-EPI equations used to define the impaired renal function, only slightly influenced the performance of the models.

The discriminatory performance of the models in our study was lower than in the models derivation study. The discrimination of the Korean model was lower than what was obtained when this model was first externally validated in the Asian population (0.78) by Kwon et al in 2012.[⁷] The overall discrimination for both models in our study was lower than what was observed by Mogueo et al in a mixed-ancestry South Africans.[⁵] The drop in performance could be explained by the fact that diabetic patients are at a much higher risk of developing CKD than the general population[¹⁷] and therefore form a more homogeneous population for CKD risk. Hence, in this context, discrimination will be expected to drop. The variation in model performance across subgroups in our study may simply reflect differences in the distribution of the disease and its risk factors. While both the Korean and Thailand models are from Asia, the structure of the populations used to derive the model was not similar, and the two models were based on different sets of predictors; there together would account for some of the differences observed in the performance of the two models in our sample.

The 2014 American VA/DoD clinical practice guideline for the management of chronic kidney disease in primary care promotes early screening and referral to nephrologists.[¹⁸] Several studies have reported high rates of late referrals of patients (even patients receiving ongoing care from other physicians) with CKD to nephrologists.[¹⁹] These risk models offer potentially attractive applications in the prevention, early detection and management of CKD in African diabetic patients. One of their most important applications would be their use as a screening tool to carefully select patients who might benefit from further, especially in resource-limited settings were biological tests may be inaccessible and/or cost-prohibitive. Based on our study results, the Korean model would seem to be the better option of the two because of its higher sensitivity at detecting prevalent undiagnosed CKD. The tested models have not been investigated yet for improvement of outcomes in routine practice, but their performances indicate that these could be used with caution to improve on CKD detection rates in diabetic patients in low-income settings like Africa. Furthermore, healthcare providers can use the risks derived from these models to ameliorate other aspects of patient management and make early referrals of high-risk patients to nephrologists.

Our study strengths include the rigorous and detailed validation approaches; and CKD definition based on both the MDRD and CKDEPI equations. The major limitation of this study was its very high exclusion rate of files with missing data which could have reduced the representativeness of our sample as well as decreased discrimination. Hence, we cannot claim generalizability of our results to a broader African population

Conclusion

This study highlights a modest-to-acceptable performance of the Korean and Thailand models when applied to a group of sub-Saharan African diabetic patients in care. These findings are at variance with the acceptable-to-good performance of the same models in a general African population. While the Korean model can be applied with caution to identify diabetic patients with undiagnosed CKD for the time being; prevalent noninvasive CKD models with improved predictive accuracy are needed for African people with type 2 diabetes.

Conflict of interest: None declared.