You could be reading the full-text of this article now if you...

If you have access to this article through your institution,
you can view this article in

Comparison of the Mortality Probability Admission Model III, National Quality Forum, and Acute Physiology and Chronic Health Evaluation IV Hospital Mortality Models: Implications for National Benchmarking*

Kramer, Andrew A. PhD1,2; Higgins, Thomas L. MD, MBA, MCCM3,4; Zimmerman, Jack E. MD, FCCM1,5

Critical Care Medicine:
doi: 10.1097/CCM.0b013e3182a66a49
Clinical Investigations
Abstract

Objective: To examine the accuracy of the original Mortality Probability Admission Model III, ICU Outcomes Model/National Quality Forum modification of Mortality Probability Admission Model III, and Acute Physiology and Chronic Health Evaluation IVa models for comparing observed and risk-adjusted hospital mortality predictions.

Design: Retrospective paired analyses of day 1 hospital mortality predictions using three prognostic models.

Setting: Fifty-five ICUs at 38 U.S. hospitals from January 2008 to December 2012.

Patients: Among 174,001 intensive care admissions, 109,926 met model inclusion criteria and 55,304 had data for mortality prediction using all three models.

Interventions: None.

Measurements and Main Results: We compared patient exclusions and the discrimination, calibration, and accuracy for each model. Acute Physiology and Chronic Health Evaluation IVa excluded 10.7% of all patients, ICU Outcomes Model/National Quality Forum 20.1%, and Mortality Probability Admission Model III 24.1%. Discrimination of Acute Physiology and Chronic Health Evaluation IVa was superior with area under receiver operating curve (0.88) compared with Mortality Probability Admission Model III (0.81) and ICU Outcomes Model/National Quality Forum (0.80). Acute Physiology and Chronic Health Evaluation IVa was better calibrated (lowest Hosmer-Lemeshow statistic). The accuracy of Acute Physiology and Chronic Health Evaluation IVa was superior (adjusted Brier score = 31.0%) to that for Mortality Probability Admission Model III (16.1%) and ICU Outcomes Model/National Quality Forum (17.8%). Compared with observed mortality, Acute Physiology and Chronic Health Evaluation IVa overpredicted mortality by 1.5% and Mortality Probability Admission Model III by 3.1%; ICU Outcomes Model/National Quality Forum underpredicted mortality by 1.2%. Calibration curves showed that Acute Physiology and Chronic Health Evaluation performed well over the entire risk range, unlike the Mortality Probability Admission Model and ICU Outcomes Model/National Quality Forum models. Acute Physiology and Chronic Health Evaluation IVa had better accuracy within patient subgroups and for specific admission diagnoses.

Conclusions: Acute Physiology and Chronic Health Evaluation IVa offered the best discrimination and calibration on a large common dataset and excluded fewer patients than Mortality Probability Admission Model III or ICU Outcomes Model/National Quality Forum. The choice of ICU performance benchmarks should be based on a comparison of model accuracy using data for identical patients.

Author Information

1Cerner Corporation, Vienna, VA.

2Department of Biostatistics, Kansas University Medical Center, Kansas City, MO.

3Critical Care Division, Baystate Medical Center, Springfield, MA.

4Department of Medicine, Tufts University School of Medicine, Boston, MA.

5Department of Anesthesiology and Critical Care Medicine, George Washington University, Washington, DC.

* See also p. 732.

Supported, in part, by Cerner Corporation, Kansas City, MO.

Dr. Kramer is an employee of Cerner Corporation, which holds the marketing rights to APACHE and also markets MPM, and has stock options with Cerner Corporation. Dr. Higgins served as Chair of Project IMPACT research committee (2003–2007) and had access to Project IMPACT data used to develop MPM model during that time. He has stock options with Cerner Corporation and has disclosed that he has previously collaborated with Drs. Zimmerman and Kramer on other MPM and APACHE papers. Dr. Zimmerman consulted for Cerner Corporation (critical care).

For information regarding this article, E-mail: akramer@cerner.com

© 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins