Identification of Patients Expected to Benefit from Electronic Alerts for Acute Kidney Injury

Abstract

Background and objectives 

Electronic alerts for heterogenous conditions such as AKI may not provide benefit for all eligible patients and can lead to alert fatigue, suggesting that personalized alert targeting may be useful. Uplift-based alert targeting may be superior to purely prognostic-targeting of interventions because uplift models assess marginal treatment effect rather than likelihood of outcome.

Design, setting, participants, & measurements 

This is a secondary analysis of a clinical trial of 2278 adult patients with AKI randomized to an automated, electronic alert system versus usual care. We used three uplift algorithms and one purely prognostic algorithm, trained in 70% of the data, and evaluated the effect of targeting alerts to patients with higher scores in the held-out 30% of the data. The performance of the targeting strategy was assessed as the interaction between the model prediction of likelihood to benefit from alerts and randomization status. The outcome of interest was maximum relative change in creatinine from the time of randomization to 3 days after randomization.

Results 

The three uplift score algorithms all gave rise to a significant interaction term, suggesting that a strategy of targeting individuals with higher uplift scores would lead to a beneficial effect of AKI alerting, in contrast to the null effect seen in the overall study. The prognostic model did not successfully stratify patients with regards to benefit of the intervention. Among individuals in the high uplift group, alerting was associated with a median reduction in change in creatinine of −5.3% (P=0.03). In the low uplift group, alerting was associated with a median increase in change in creatinine of +5.3% (P=0.005). Older individuals, women, and those with a lower randomization creatinine were more likely to receive high uplift scores, suggesting that alerts may benefit those with more slowly developing AKI.

Conclusions 

Uplift modeling, which accounts for treatment effect, can successfully target electronic alerts for AKI to those most likely to benefit, whereas purely prognostic targeting cannot.