Validation of the Kidney Disease Improving Global Outcomes Criteria for AKI and Comparison of Three Criteria in Hospitalized Patients

Abstract

Background and objectives 

AKI is a major clinical problem and predictor of outcome in hospitalized patients. In 2013, the Kidney Disease: Improving Global Outcomes (KDIGO) group published the third consensus AKI definition and classification system after the Risk, Injury, Failure, Loss of Kidney Function, and End-Stage Kidney Disease (RIFLE) and the Acute Kidney Injury Network (AKIN) working group systems. It is unclear which system achieves optimal prognostication in hospital patients.

Design, setting, participants, & measurements 

A retrospective observational study using hospital laboratory, admission, and discharge databases was performed that included adult patients admitted to a teaching hospital in Tokyo, Japan between April 1, 2008, and October 31, 2011. AKI occurring during each hospital stay was identified, and discriminative ability of each AKI classification system based on serum creatinine for the prediction of hospital mortality was assessed. The receiver operating characteristic curve, a graphical measure of test performance, and the area under the curve were used to evaluate how classifications preformed on the study population.

Results 

In total, 49,518 admissions were studied, of which 11.0% were diagnosed with RIFLE criteria and 11.6% were diagnosed with KDIGO criteria, but only 4.8% were diagnosed with AKIN criteria. Overall hospital mortality was 3.0%. AKI staging and hospital mortality were closely correlated in all systems. Discrimination for hospital mortality was similar for RIFLE and KDIGO criteria (area under the curve=0.77 versus 0.78; P=0.02), whereas AKIN discrimination was inferior (area under the curve=0.69 versus RIFLE [P<0.001] versus KDIGO [P<0.001]).

Conclusion 

Among hospital patients, KDIGO and RIFLE criteria achieved similar discrimination, but the discrimination of AKIN was inferior.

Introduction

AKI is a major clinical problem among hospitalized patients (^1–4). In 2004, the Acute Dialysis Quality Initiative published the Risk, Injury, Failure, Loss of Kidney Function, and End-Stage Kidney Disease (RIFLE) AKI definition and classification system (⁵). The RIFLE criteria consider three classes of AKI (risk, injury, and failure) according to the variations in serum creatinine and urine output from baseline values and two outcome classes (loss of kidney function and end-stage kidney disease) (Table 1). The RIFLE criteria have been validated in various settings and more than 500,000 patients (^4,6–13).

Table 1: A comparison of the RIFLE, AKIN, and KDIGO definitions and classifications for AKI using SCr

In 2007, the Acute Kidney Injury Network (AKIN) working group, which was composed of nephrologists and critical care physicians, published the AKIN criteria (¹⁴) (Table1). The AKIN criteria have two components: diagnosis and staging. To diagnose AKI, baseline serum creatinine is not required, but the AKIN system requires at least two values of serum creatinine obtained within a period of 48 hours. AKI is defined by an acute increase in absolute serum creatinine of at least 0.3 mg/dl, a percentage increase in 50%, or a decrease in the urine output (oliguria<0.5 ml/kg per hour for more than 6 hours). Stage 1 corresponds to the risk class in RIFLE, but it also considers an absolute increase in serum creatinine of 0.3 mg/dl. Stages 2 and 3 correspond to the injury and failure classes of RIFLE, respectively. Stage 3 incorporates patients requiring RRT. The AKI classification introduced modifications to capture the impact of small acute increments in serum creatinine in a short period and the effect of RRT (^14–16). Of importance, in the AKIN classification, AKI diagnosis was based on changes over the time course of 48 hours, which was unlike the 1-week timeframe used in RIFLE (¹⁴). The advantages of AKIN have not yet been proven in any comparative studies (^17–21).

Finally, in 2013, the Kidney Disease: Improving Global Outcomes (KDIGO) group published a third consensus definition and classification system for AKI (²²). The new system was based on a combination of the RIFLE and AKIN criteria (^5,14,22). The KDIGO classification keeps both the absolute increase in serum creatinine level of ≥0.3 mg/dl within 48 hours from the AKIN and the 7-day timeframe for the ≥50% increase from the RIFLE (Table 1). This newly published system has been recently assessed in several studies (^23–28). However, its application to in-hospital AKI and its prognostic accuracy compared with the two previous classifications have not been thoroughly evaluated.

Accordingly, we conducted a retrospective study to describe the epidemiology of in-hospital AKI with the new KDIGO classification and compare its predictive ability for hospital mortality with the RIFLE and AKIN classification systems.

Materials and Methods

We performed a retrospective observational study including all patients admitted to a 1074-bed academic hospital in Tokyo, Japan, between April 1, 2008, and October 31, 2011. We screened computerized hospital admission and discharge databases and retrieved variables, such as age, sex, all dates and results of serum creatinine measured during the study period, admission units, and hospital mortality. We excluded patients younger than 15 years of age, patients who received RRT during the admission with CKD stage 5 at admission or at baseline, and patients who stayed in the hospital for less than 2 days. The institutional ethics committee approved the study and waived the need for informed consent; this study did not require any intervention, and data for each patient were anonymous.

AKI was defined only by the serum creatinine criteria of RIFLE, AKIN, and KDIGO classification systems as shown in Table 1. Baseline serum creatinine was defined by the most recent value obtained in our outpatient clinic between 1 and 12 months before admission or if not available, calculated by the simplified Modification of Diet in Renal Disease (MDRD) formula for Japanese patients (assuming a GFR of 75 ml/min per 1.73 m²) as previously reported (²⁹). For analysis, AKI stages were calculated using serum creatinine levels with reference to the preadmission baseline creatinine (or calculated from the MDRD equation) or the lowest creatinine within 7 days for the first 7 days after admission. After day 8, the reference value was the lowest creatinine within the last 7 days. The maximum category derived by each classification system during hospitalization was reported.

Categorical data were reported as proportions, and numerical data were reported as medians with interquartile ranges. The discrimination ability in predicting hospital mortality was assessed using the area under the receiver operating characteristic curve (AUC) by the bootstrap method. Receiver operating characteristic curve was produced to assess the association of each AKI stage by the three criteria with hospital mortality as a univariate analysis. AKI stages were included in the model. Because we used a large dataset, Bonferroni’s adjustment for multiple comparison to a P value <0.02 (0.05/3) was taken to indicate statistical significance. Comparison of the AUC was performed using R version 3.0.0 (The R Foundation for Statistical Computing, Vienna, Austria), and other analyses were performed by JMP Pro 9.0.3 (SAS Institute, Inc., Cary, NC).

Results

During the study period, we screened all 81,738 admissions. Of these admissions, 56,567 patients had laboratory data on renal function during hospital admission, and 49,518 patients fulfilled the criteria for inclusion (Figure 1). Patient characteristics are described in Table 2. Admissions to medical units represented 41.7% of all admissions. Overall, the hospital mortality rate was 3.0%. The median interval of creatinine measurements was 1.5 days (interquartile range=1.0–2.0), and the rate of measurements within a 48-hour interval was 48.9% (145,395/297,071 measurements).

Figure 1: Study flow chart. LOS, length of stay.

Table 2: Patient characteristics and outcomes

The number of patients, hospital mortality, and hospital length of stay according to AKI stages are shown in Figure 2. For all classification systems, hospital mortality increased in accordance with staging. AKIN classification diagnosed fewer patients as having AKI than the other two systems (RIFLE: 5451 [11.0%]; AKIN: 2386 [4.8%]; KDIGO: 5764 [11.6%]), whereas the mortality rate in each stage was highest with the AKIN criteria (Figure 2). By cross-tabulating, stages classified with each criteria shown in Tables 3, 4, and 5 revealed that 3507 (7.0%) admissions were diagnosed as AKI by RIFLE but not AKIN.

Figure 2: The incidence and hospital mortality for each definition of AKI. The column graph shows numbers of patients in each group, and the line chart shows the mortality rate and hospital length of stay (LOS). AKIN, Acute Kidney Injury Network; KDIGO, Kidney Disease: Improving Global Outcomes; Pts, patients; RIFLE, Risk, Injury, Failure, Loss of Kidney Function, and End-Stage Kidney Disease.

Table 3: Contingency analyses of patients classified by RIFLE versus AKIN

Table 4: Contingency analyses of patients classified by AKIN versus KDIGO

Table 5: Contingency analyses of patients classified by KDIGO versus RIFLE

The predictive ability for hospital mortality for each system as assessed with the AUC is shown in Table 6. When each set of criteria was applied to hospital patients, RIFLE and KDIGO showed adequate and similar discrimination and significantly greater discrimination than AKIN criteria. Patients with baseline creatinine data measured before admission were retrieved and analyzed separately, which showed coincident results with the whole cohort.

Table 6: AUC for the RIFLE, AKIN, and KDIGO definitions to predict hospital mortality

Discussion

Key Findings

We studied 49,518 admissions to a university hospital using a large clinical database and compared the predictive performance of the RIFLE (⁵), AKIN (¹⁴), and KDIGO (²²) classification systems for AKI. We found that almost 11% of hospitalized patients were diagnosed as having AKI with the RIFLE and KDIGO criteria, whereas only approximately 5% of patients were classified as having AKI with the AKIN criteria. Staging and hospital mortality, however, were closely correlated for all classification systems. Discriminating ability for hospital mortality was adequate and similar for RIFLE and KDIGO criteria, whereas AKIN discrimination ability was inferior.

Relationship to Previous Studies

Three sets of criteria proposed by nephrology and intensive care expert groups have been validated for their prognostic ability in various populations so far (^{4,6–13,17–21,23–28}). Their comparative performance has recently been evaluated in several studies (^23,26–28). The RIFLE classification has exhibited good prognostic accuracy in terms of mortality in many intensive care unit studies (^7–9) and several studies of hospital patients (^4,6), and this finding was again confirmed in our study.

The AKIN criteria aimed to improve the sensitivity of the RIFLE classification system by incorporating an acute rise in serum creatinine of as little as 0.3 mg/dl, a time constraint of 48 hours for the diagnosis of AKI, and the use of RRT in stage 3 (¹⁴). Compared with the RIFLE criteria, however, the AKIN criteria have failed to improve the predictive ability of the definition and classification of AKI, especially in critically ill patients (^17–19).

KDIGO, a combination of RIFLE and AKIN, is expected to benefit from the two sets of criteria. In the FINNAKI study, KDIGO has been reported to give identical diagnosis to AKIN (²³), and for cardiac surgery patients, KDIGO was equal to AKIN and superior to RIFLE in predicting hospital mortality (²⁸). There are two studies targeting cardiac critical patients to compare RIFLE and KDIGO classifications (^26,27). One study showed AUCs for predicting 30-day adverse outcome (composed of readmission for heart failure, death, or requirement of RRT) of RIFLE=0.76, AKIN=0.74, and KDIGO=0.72 (P=0.02) (²⁶). Another study, using multivariable Cox regression analysis, showed increased hazard ratio for 30-day mortality of 3.51 (95% confidence interval, 2.35 to 5.25) by RIFLE and 3.99 (95% confidence interval, 2.59 to 6.15) by KDIGO (²⁷).

Recently, Zeng et al. (³⁰) reported the epidemiology of AKI using the KDIGO definition and its comparison with the other definitions in a single center, retrospective, observational study. Here, we additionally report a study comparing the three sets of criteria in all hospitalized patients. In our population, AKI occurred in almost 11% of all hospitalized patients by RIFLE and KDIGO criteria, which was close to the value reported in the work by Zeng et al. (³⁰). Also, we found AKIN classification to be inferior in its diagnostic ability, which was not observed in the previous report (³⁰). In critical care settings, monitoring small increments of serum creatinine could bring out a higher sensitivity to diagnose AKI with a timeframe of 48 hours; however, in general wards, serum creatinine is not measured frequently. In our study, more than one half of measurements were performed with intervals over 48 hours (51.0%; 151,676/297,071 measurements). Diagnostic criteria with short timeframes are more likely to fail in such clinical practice in general wards. Another explanation for the underdiagnosis by AKIN is that AKIN could not diagnose those patients who have increased creatinine on admission and are treated in the hospital as AKI patients, because it does not use baseline measurements for diagnosis. Two cases are presented in Figure 3 as examples. Because AKIN classification does not use baseline creatinine for AKI diagnosis, in a case in which creatinine level was elevated on admission but did not increase after admission, AKIN fails to diagnose it as AKI (Figure 3A). Furthermore, AKIN has a 48-hour timeframe for AKI diagnosis, which excludes measurements with longer intervals from evaluation (Figure 3B).

Figure 3: Serum creatinine trend of the patients who were diagnosed as AKI by RIFLE but not AKIN. (A) Creatinine level at admission elevated to 1.5 times the baseline level (RIFLE-Risk) but not increased after admission (no AKI by AKIN). (B) Creatinine level increased to 5.2 by 0.5 mg/dl (RIFLE-Failure) from day 3 to day 6, but the interval was over 48 hours (no AKI by AKIN).

Implications for Clinicians and Researchers

Many clinical studies on AKI are being published using different criteria for the diagnosis and classification of AKI (KDIGO, RIFLE, and AKIN). The use of the three systems is an improvement from the previous use of >25 different classification systems (³¹), but it leaves investigators with uncertainty as to whether one is describing similar populations. Our study has shown that the RIFLE and KDIGO criteria showed good prognostic accuracy and were essentially equivalent in the hospital population and that they were superior to the AKIN system. Furthermore, as shown in Table 1, the RIFLE classification system is the simplest system. In hospital patients, we should be cautious about the use of AKIN criteria, because they might be vulnerable to practice patterns.

Strengths and Limitations

This study is the first report of a comparative validation study for the RIFLE, AKIN, and KDIGO criteria including all hospitalized patients, and it has important implications for AKI epidemiology. Exclusion criteria were children, short hospital stay, and CKD on maintenance hemodialysis. These limited criteria enabled us to minimize selection bias and give a whole picture of AKI in hospitalized patients.

However, this study was conducted in a single center, which may have limitations to its external validity. Because it was a retrospective study, data on causes of admissions, causes of kidney injury, or urine output were not available, which limited additional assessment of the characteristics and diagnosis of AKI. We used recorded creatinine values whenever possible to define baseline creatinine values, and the MDRD equation was used when recorded data were not available. Estimation of baseline serum creatinine by the MDRD equation has been reported to over- or underestimate some patients with AKI in the risk category of RIFLE (³²). However, such errors would have applied to baseline renal functional estimation for all three systems and could not have created a bias in favor of any one system. In fact, results in our study showed the same tendency in the discrimination of the three criteria in the subgroup of those patients with measured baseline creatinine and the whole cohort.

In this study, which involved almost 50,000 hospital patients, we found that close to 11% developed AKI as defined by the RIFLE and KDIGO systems, whereas less than one half of this number were identified by the AKIN system. Furthermore, when we compared these three systems for their ability to predict mortality, the KDIGO and RIFLE systems were equivalent, but the AKIN system was clearly inferior. Our observations do not support the use of AKIN for the identification and/or prognostication of AKI in hospital patients.

Disclosures

None.