The first 2 decades of 21st century medicine in the United States have been marked by substantial changes in the philosophy of healthcare delivery, stimulated in part by landmark Institute of Medicine publications about quality and safety [1,2]. Efforts to define the value of health care by linking quality and costs of care have framed federal and private reimbursement policies, and consequently, an entire sector of the healthcare industry dedicated to measuring quality has emerged. Nephrology was the first major discipline to be tested in this arena with the advent of the Centers for Medicare and Medicaid Services end-stage renal disease (ESRD) quality incentive program (QIP), which tied reimbursement to dialysis unit performance in 2012. Even as the ESRD QIP appears firmly entrenched, and despite criticism of the proliferation of measures and doubt about their utility in assessing quality [3–5], nephrologists and other providers grapple with new pay for performance programs through the Medicare Access & CHIP Reauthorization Act of 2015 .
Delivery of expert acute consultative care remains an essential skill and cases of acute kidney injury (AKI) represent the majority of inpatient consultation requests . The incidence of AKI has been reported as 9.6% in a Department of Veteran's Affairs Healthcare System analysis of over 1.2 million patients  and 22.7% in a cohort in Boston, Massachusetts . A five-fold increase in the incidence of AKI in the United States was reported between 2001 and 2011 in the Healthcare Cost and Utilization Project's (HCUP) National Inpatient Sample (NIS), a nationally representative, stratified sample of approximately 20% of US community hospitals [10▪]. In a meta-analysis on the burden of AKI around the globe, using the Kidney Disease: Improving Global Outcomes (KDIGO) criteria, one in five adults worldwide experiences AKI during a hospital episode of care . Despite high frequency and cost, little has been written about measuring AKI care quality. Indeed, the National Quality Forum (NQF) currently lists 55 kidney disease-related quality measures that are either under evaluation or have been endorsed , but not a single one pertains to AKI care.
The development of healthcare quality measures is challenging. Good quality measures have been characterized as specific, measurable, achievable, relevant, and timely . Schuster et al.  have recently added an additional characteristic – cost-effective – a variable that has been consistently overlooked in the landscape of quality. It is challenging to identify AKI measures that have a strong, evidence-based foundation and also meet the characteristics of good quality measures. We explore candidate quality measures for AKI and assess the evidence for their validity and reliability.
DEFINITION AND CLASSIFICATION OF QUALITY MEASURES
Mainz  defined the key characteristics of an ideal indicator as follows: clearly defined, highly specific and sensitive, valid and reliable, discriminating, relevant, able to permit useful comparisons, and evidence-based. Quality measures may be classified as structural, process, or outcome. Structural measures refer to health system characteristics that impact the system's ability to meet the needs of patients. Process measures reference what providers do and how well they do it. Outcome measures are health states and events that have been impacted by health care. For all measures, and particularly for outcome measures, risk adjustment is critical to allow valid comparisons and may require careful restriction of a defined patient population or stratification of a total sample. Candidate AKI quality measures discussed in this article are classified in Table 1.
STRUCTURAL MEASURES FOR ACUTE KIDNEY INJURY
Timing of nephrology consultation
A delay in nephrology consultation for AKI has been associated with a poorer prognosis in three ICU observational studies [16,19,20]. A 2002 study  defined early and late nephrology consultation as less than and at least 48 h from first ICU day in 215 patients with AKI at four teaching hospitals. Delay in consultation was seen in 28% and these patients had significantly increased mortality, hospital length of stay (LOS), and ICU stay, but differences between early and late referral were attenuated or lost by covariate adjustment and propensity analysis.
More recent studies used Acute Kidney Injury Network (AKIN)  or RIFLE (risk, injury, failure, loss of kidney function, end-stage kidney disease) criteria  to define AKI. In a series of 148 AKI patients, consultation delay of at least 48 h was associated with a significant increase in ICU mortality (65.4 vs. 88.2%) that appeared to persist with covariate and propensity score adjustments . Costa e Silva et al.  studied a larger cohort (366 AKI patients) at a single tertiary care center and found that those seen by nephrology at least 48 h after AKI recognition had a four-fold increased odds for mortality and increased dialysis dependence at discharge, but they could not exclude the influence of residual confounding factors .
These studies are suggestive but not conclusive that early nephrology involvement has an influence on AKI outcomes in the ICU setting. The results are confounded by variables such as the likelihood that a primary team who requests early consultation may be more proactive in other aspects of care that positively impact outcome or that late consultation, by definition, occurs at a later stage of AKI – itself associated with poorer outcomes. Furthermore, timing of consultation is inextricably linked to early recognition, an arguably more important quality measure as discussed below. Timing of nephrology consultation cannot be recommended as a quality measure at this time.
Nephrology follow-up after hospitalization
AKI survivors are at increased risk for recurrent AKI, hospital readmission, cardiovascular events, progression to chronic kidney disease (CKD) and mortality . A nephrology referral permits expert attention to incomplete recovery, safe medication management, blood pressure control, CKD recognition and care, and patient education. Silver and Siew [22▪] recently reviewed the current state of follow-up care for AKI and concluded that less than half of patients with severe AKI receive nephrology care after discharge.
Twenty years ago, Khan et al.  recognized an association between nephrology follow-up for AKI and outcome in a Scottish cohort. Only 22% of patients who experienced AKI at a single center were referred to a nephrologist. Patients were stratified as high, medium, and low risk for short-term mortality. Two-year survival for patients who had nephrology follow-up was 19, 42, and 80% in the high, medium, and low-risk groups compared with respective survival of 14, 30, and 75% without nephrology follow-up. A contemporary study was conducted to assess follow-up of hospitalized patients with severe AKI requiring dialysis over a 12-year period in the province of Ontario, Canada . Patients who had follow-up within 90 days, when compared with propensity score-matched patients without such follow-up, had a reduced incidence of all-cause mortality [Hazard Ratio (HR) 0.76, 95% confidence interval (CI) 0.62–0.93].
Nephrology follow-up after hospitalization with AKI is an appealing quality measure. There is limited observational data supporting its implementation. KDIGO recommends evaluation of patients 3 months after AKI for resolution, new onset, or worsening of preexisting CKD . Although an unlikely candidate for study in a randomized, prospective fashion, nephrology follow-up care is a reasonable measure to consider based upon its potential value in terms of patient safety and long-term risk stratification.
PROCESS MEASURES FOR ACUTE KIDNEY INJURY
Early detection of acute kidney injury
Alert systems to detect AKI have been described, some automated and others relying upon human interactions. Electronic alert systems (e-Alerts) triggered by changes in serum creatinine and/or urinary output have been studied.
An e-Alert system employing AKIN criteria, successfully deployed at a large UK center , identified 3202 AKI episodes that were also assessed by manual laboratory review, and performed with a 0.2% false negative rate and 1.7% false positive rate. The same investigators subsequently evaluated the before and after effects of an AKI care bundle (CB) linked with an interruptive alert that was triggered upon electronic recognition of AKI . The care bundle consisted of simple standardized investigations and interventions for initial evaluation and management of AKI. Care bundle completion within 24 h of the alert was associated with significant reduction in hospital mortality (18 vs. 23%) and reduced progression to higher stages of AKI (3.9 vs. 8.1%) compared with CB completion later than 24 h or not at all.
A Belgian e-Alert system for recognition of RIFLE AKI criteria resulted in an increase in prompt therapeutic interventions with fluids, diuretics, or vasopressors compared with controls, as well as a modest improvement in short-term recovery rates .
Results of e-Alerts have not been universally positive. A single, randomized, controlled clinical trial from the University of Pennsylvania  was unable to show a difference in need for dialysis or mortality outcomes when comparing 1201 AKI episodes that triggered alerts with 1192 usual care AKI episodes. In this study, though, the alerts were only sent once with no mechanism to acknowledge receipt and with no algorithm for treatment.
e-Alerts are a promising mechanism to improve early recognition of AKI and outcomes may be most positively impacted when e-Alerts are coupled with standardized care bundles. However, data to support them as a quality measure are still limited. More work needs to be done to standardize triggers, responses, and mitigation of alert fatigue. Moreover, the cost of such systems needs to be incorporated into a value analysis.
Medication dosing in acute kidney injury
An important safety concern in AKI is failure to dose adjust medications or avoid nephrotoxic agents in the context of reduced kidney function. McCoy et al.  demonstrated that a computerized provider order entry-based (CPOE) alerting system to support medication management after AKI increased the rate and timeliness of modification or discontinuation of targeted medications. The modification or discontinuation (within 24 h of increased creatinine) rate per 100 events for medications included in the interruptive alert significantly improved from 35.2 preintervention to 52.6 postintervention. The greatest effect occurred for medications with the highest potential for toxicity.
Awdishu et al. [31▪▪] performed a prospective, cluster randomized controlled trial of physicians receiving clinical decision support (CDS) for renal dosage adjustments compared with those performing their usual workflow. The study spanned both inpatient and ambulatory care areas of a single large teaching institution. A Best Practice Alert function was used within the electronic health record to identify patients with AKI or CKD based on estimated creatinine clearance. Prescribing orders were appropriately adjusted 17% of the time vs. 5.7% of the time in the intervention and control arms, respectively [odds ratio (OR), 1.89; 95% CI, 1.45–2.47]. Prospective alerts that fired when estimated creatinine clearance met a trigger level were more effective than ‘look-back’ alerts that fired when renal function declined acutely, suggesting that this alert system may have had greatest impact for patients with CKD, but retained good utility for patient safety in AKI.
The development of CDS for a commercial CPOE system does require extensive commitment of personnel, including clinical staff, and substantial financial investment . Expectations are growing for CDS in the healthcare industry. At least one major safety monitoring organization, Leapfrog Group, incorporates this metric into its hospital safety scoring rubric . Controlled studies and CDS experience supporting medication dosing in AKI as a quality metric are sufficient to give this measure serious consideration as a standard.
OUTCOME MEASURES FOR ACUTE KIDNEY INJURY
A considerable effort to demonstrate efficacy of various interventions to prevent contrast-induced AKI and AKI after cardiac surgery has yielded inconclusive results. Trials to study the effects of timing, dose and modality of renal replacement therapy on outcomes after established AKI have likewise been inconclusive. Therefore, quality measures related to such interventions cannot be recommended and are outside the scope of this discussion.
AKI often arises as a complication during inpatient management of a nonrenal, ‘primary’ diagnosis. As such, it is difficult to determine to what extent hospital utilization and mortality should be attributed to AKI as opposed to admitting diagnoses or other complications. Furthermore, it may be difficult to ascribe accountability for performance when a condition is managed by many hands. Nevertheless, for a condition that frequently complicates hospital stays, such as AKI, there are sufficient data to generate expected costs, LOS, readmission rates, and mortality. Further, there is precedent for holding healthcare organizations responsible for performance improvement surrounding specific diagnoses.
It has been long established that even modest changes in serum creatinine during hospitalization are associated with increased mortality, as well as LOS and cost  – findings that persist after multivariate adjustment. A meta-analysis by Coca et al.  encompassing over 47 000 patients with AKI between 1985 and 2007, described a mortality rate of 8.9 deaths/100 person-years in AKI survivors who were discharged from the hospital and followed for at least 6 months compared with 4.3 deaths/100 person-years in those without AKI.
Data abstracted from the HCUP NIS demonstrated that the in-hospital mortality rate for AKI decreased from 21.9% in 2001 to 9.1% in 2011 [10▪]. However, the absolute number of patients experiencing in-hospital mortality increased two-fold, and the characteristics of the population of AKI patients shifted over the decade with significantly increased presence of comorbid hypertension, obesity, and deficiency anemias.
Wang et al.  analyzed over 19 000 hospitalizations in a single center over 1 year and observed an incidence of AKI of 22.7% with an adjusted OR for in-hospital mortality of 4.43. An advantage of this study was that the center, the University of Alabama, participated in the University Health System Consortium and used a series of models for risk adjustment that had a high discrimination for the independent effect of AKI on mortality.
Mortality is utilized for assessing quality in many public reporting domains for a variety of medical conditions. The comorbid characteristics of patients with AKI and associated mortality rates for AKI have shifted over decades, and risk stratification is clearly necessary to make valid comparisons between institutions. However, it appears that suitable data and sufficient capacity to analyze case-mix variation are available such that any system for assessing AKI quality would include this metric of necessity.
Abundant literature exists regarding hospital readmissions in heart failure patients with changes in creatinine. For purposes of the present discussion, attention is focused on the broader population of AKI patients.
Koulouridis et al. , performed a single center retrospective study of 3345 patients with AKI defined by KDIGO criteria and discharged between 2000 and 2007. They found a 15% 30-day readmission rate compared with 11% for matched non-AKI patients (OR, 1.21; 95% CI, 1.08–1.36).
Very recently, Silver et al. published a 10-year cohort study from 2003 to 2013 of more than 150 000 Canadian patients discharged from nearly 200 hospitals after AKI (identified by International Classification of Diseases-10 coding). Patients were matched with non-AKI patients [37▪▪]. The 30-day hospital readmission rate was 16% for AKI patients vs. 11% for non-AKI (HR 1.53, 95% CI 1.50–1.157).
These cohorts were of sufficient size, duration, and currency to consider them as a standard for benchmark 30-day readmission rates for patients with risk-adjusted AKI, and thus, this may be a reasonable quality measure.
Hospital length of stay
One of the older studies referenced for discussion of mortality in this article contains data about impact of AKI on LOS . It is self-evident that AKI complicating a hospital stay will increase the length of hospital stay in many cases. Hypothetically, improved early recognition of AKI and application of standardized, basic care (bundles) might shorten the course of injury and/or provide opportunities for earlier discharge. However, in the absence of more recent anchoring literature, this metric cannot be recommended for use as a quality measure for AKI care.
As discussed with LOS, studies quantifying the cost of care that is ascribed to AKI tend to be older and probably not easily applicable to current care. Nonetheless, newer versions of hospital finance accounting systems provide exhaustive detail that allow for cost attribution. Further, there is certainly precedent for evaluating cost of care with specific diagnoses such as heart failure or myocardial infarction as quality measures. Dedicated analyses would be required to generate cost of care ascribed to AKI, but the data and tools to pursue this as a quality measure exist.
LIMITATIONS OF ESTABLISHING ACUTE KIDNEY INJURY QUALITY MEASURES
The Mainz criteria for indicators suggest that such metrics need to achieve high fidelity for use in the quality arena. Several of the candidate measures for AKI discussed in this article lack satisfactory reliability, discrimination, or sufficient evidence-base. Although the hospital team as a whole maintains responsibility for the patient outcome, attribution to individuals or functional units becomes much more difficult. It is uncertain how much impact nephrology providers can have on AKI outcomes as nephrology teams are dependent upon primary teams for AKI recognition and initial care actions.
It is possible to identify other commonly agreed care processes not addressed in this article that might impact patient outcomes, such as attention to fluid management, serial renal function assessments, acute on CKD (which represents at least 30% of AKI cases), and electrolyte measures (e.g., provider order sets for treatment of various forms of hyponatremia). Although many of these actions are included in consensus guidelines, most are not formerly studied in a manner that allows consideration as high level quality measures. In addition, Weiner and Watnick  have called for parsimony in the proposal of quality measures in ESRD such that just a few core metrics should be selected that are fully vetted rather than a breadth of measures with greater variability in validity and reliability. Conceivably, similar reasoning might apply to AKI.
The field of nephrology has acquired abundant experience with the assessment of healthcare quality through the ESRD QIP. If other areas of nephrology care are to be considered for standardized quality measures, hospital-based and postdischarge care of AKI would appear to be a logical choice. In our estimation, only a small number of candidate quality measures can be argued to meet accepted criteria as valid and reliable quality indicators. Yet, in the spirit of parsimony, a small number of good measures for AKI may be reasonable to place into consideration for evaluation by entities such as NQF or the Renal Physicians Association, which recently launched a Qualified Clinical Data Registry.
Among the candidate measures reviewed herein, we have suggested that nephrology follow-up after an AKI hospital episode, medication management during AKI, in-hospital mortality rates, and 30-day hospital readmission rates come closest to fulfilling Mainz criteria for good indicators of quality. Early detection of AKI, timing of nephrology consultation, hospital LOS, and overall cost of care remain more challenging considerations for quality measurement. Additional study of AKI care quality is warranted to adequately represent and evaluate the spectrum of care delivered by nephrology providers.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
REFERENCES AND RECOMMENDED READING
Papers of particular interest, published within the annual period of review, have been highlighted as:
- ▪ of special interest
- ▪▪ of outstanding interest
1. Institute of MedicineTo err is human: building a safer health system. Washington, DC: National Academy Press; 1999.
2. Institute of MedicineCrossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.
3. Weiner D, Watnick S. The ESRD quality incentive program – can we bridge the chasm? J Am Soc Nephrol 2017; 28:1697–1706.
4. McClellan WM, Plantinga LC, Wilk AS, Patzer RE. ESRD databases, public policy, and quality of care: translational medicine and nephrology. Clin J Am Soc Nephrol 2017; 12:210–216.
5. Diamond LH, Howard AD. The ESRD quality incentive program: the current limitations of evidence and data to develop measures, drive improvement, and incentivize outcomes. Adv Chronic Kidney Dis 2016; 23:377–384.
7. Koyner JL, Cerdá J, Goldstein SL, et al. Acute Kidney Injury Advisory Group of the American Society of NephrologyThe daily burden of acute kidney injury: a survey of U.S. nephrologists on World Kidney Day. Am J Kidney Dis 2017; 64:394–401.
8. Lafrance JP, Miller DR. Acute kidney injury associates with increased long-term mortality. J Am Soc Nephrol 2010; 21:345–352.
9. Wang HE, Muntner P, Chertow GM, Warnock DG. Acute kidney injury and mortality in hospitalized patients. Am J Nephrol 2012; 35:349–355.
10▪. Brown JR, Rezaee ME, Marshall EJ, Matheny ME. Hospital mortality in the United States following acute kidney injury. Biomed Res Int 2016. 4278579.
Contemporary review of postacute kidney injury (AKI) mortality trends in the United States, discussing the factors behind a reduction in in-hospital mortality rate despite an increase in both AKI rate and the absolute number of AKI associated deaths.
11. Susantitaphong P, Cruz DN, Cerda J, et al. Acute Kidney Injury Advisory Group of the American Society of NephrologyWorld incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol 2013; 8:1482–1493.
13. Duckett SJ. Designing incentives for good quality hospital care. Med J Aust 2012; 196:678–679.
14. Schuster MA, Onorato SE, Meltzer DO. Measuring the cost of quality measurement: a missing link in quality strategy. JAMA 2017; 318:1219–1220.
15. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003; 15:523–530.
16. Mehta RL, McDonald B, Gabbai F, et al. Nephrology consultation in acute renal failure: does timing matter? Am J Med 2002; 113:456–461.
17. Mehta RL, Kellum JA, Shah SV, et al. Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury. Crit Care 2007; 11:R31.
18. Bellomo R, Ronco C, Kellum JA, et al. Acute renal failure – definition, outcome measures, animal models, fluid therapy and information technology needs: the second international consensus conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care 2004; 8:R204–R212.
19. Ponce D, Zorzenon C, dos Santos NY, Balbi AL. Early nephrology consultation can have an impact on outcome of acute kidney injury patients. Nephrol Dial Transplant 2011; 26:3202–3206.
20. Costa e Silva VT, Liaño F, Muriel A, et al. Nephrology referral and outcomes in critically ill acute kidney injury patients. PLoS One 2013; 8:e70482.
21. Parr SK, Siew ED. Delayed consequences of acute kidney injury. Adv Chronic Kidney Dis 2016; 23:186–194.
22▪. Silver SA, Siew ED. Follow up care in acute injury: lost in transition. Adv Chronic Kidney Dis 2017; 24:246–252.
Important review of the evidence supporting patient follow-up after AKI, including current state of follow-up care and strategies to improve care.
23. Khan IH, Catto GRD, Edward N, Macleod AM. Acute renal failure: factors influencing nephrology referral and outcome. QJM 1997; 90:781–785.
24. Harel Z, Wald R, Bargman JM, et al. Nephrologist follow-up improves all-cause mortality of severe acute kidney injury survivors. Kidney Int 2013; 83:901–908.
25. Kidney International Supplements (2012) 2, 4; doi:10.1038/kisup.2012.4
26. Selby NM, Crowley L, Fluck RJ, et al. Use of electronic results reporting to diagnose and monitor AKI in hospitalized patients. Clin J Am Soc Nephrol 2012; 7:533–540.
27. Kolhe NV, Staples D, Reilly T, et al. Impact of compliance with a care bundle on acute kidney injury outcomes
: a prospective observational study. PLoS One 2015; 10:e0132279.
28. Colpaert K, Hoste EA, Steurbaut K, et al. Impact of real-time electronic alerting of acute kidney injury on therapeutic intervention and progression of RIFLE class. Crit Care Med 2012; 40:1164–1170.
29. Wilson FP, Shashaty M, Testani J, et al. Automated, electronic alerts for acute kidney injury: a single-blind, parallel-group, randomized controlled trial. Lancet 2015; 385:1966–1974.
30. McCoy AB, Waitman LR, Gadd CS, et al. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis 2010; 56:832–841.
31▪▪. Awdishu L, Coates CR, Lyddane A, et al. The impact of real-time alerting on appropriate prescribing in kidney disease: a cluster randomized controlled trial. J Am Med Inform Assoc 2016; 23:609–616.
A prospective, cluster randomized controlled trial that demonstrated a real-time clinical decision support system improves medication dose adjustment in patients with impaired renal function.
32. Field TS, Rocho P, Lee M, et al. Costs associated with developing and implementing a computerized clinical decision support system for medication dosing for patients with renal insufficiency in the long-term care setting. J Am Med Inform Assoc 2008; 15:466–472.
34. Chertow GM, Burdick E, Honour M, et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol 2005; 16:3365–3370.
35. Coca SG, Yusuf B, Shlipak MG, et al. Long-term risk of mortality and other adverse outcomes after acute kidney injury. Am J Kidney Dis 2009; 53:961–973.
36. Koulouridis I, Price L, Madias NE, Jaber BL. Hospital-acquired acute kidney injury and hospital readmission: a cohort study. Am J Kidney Dis 2015; 65:275–282.
37▪▪. Silver S, Harel Z, McArthur E, et al. 30-day readmissions after an acute kidney injury hospitalization. Am J Med 2017; 130:163–172.
A population-based study over a 10-year period in Ontario, Canada that demonstrated a 20% 30-day hospital readmission rate for patients who survive AKI.