Secondary Logo

Institutional members access full text with Ovid®

Share this article on:

Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom*

Harrison, David A. PhD; Brady, Anthony R. MSc; Parry, Gareth J. PhD; Carpenter, James R. DPhil; Rowan, Kathy DPhil

doi: 10.1097/01.CCM.0000216702.94014.75
Clinical Investigations

Objective: To assess the performance of published risk prediction models in common use in adult critical care in the United Kingdom and to recalibrate these models in a large representative database of critical care admissions.

Design: Prospective cohort study.

Setting: A total of 163 adult general critical care units in England, Wales, and Northern Ireland, during the period of December 1995 to August 2003.

Patients: A total of 231,930 admissions, of which 141,106 met inclusion criteria and had sufficient data recorded for all risk prediction models.

Interventions: None.

Measurements and Main Results: The published versions of the Acute Physiology and Chronic Health Evaluation (APACHE) II, APACHE II UK, APACHE III, Simplified Acute Physiology Score (SAPS) II, and Mortality Probability Models (MPM) II were evaluated for discrimination and calibration by means of a combination of appropriate statistical measures recommended by an expert steering committee. All models showed good discrimination (the c index varied from 0.803 to 0.832) but imperfect calibration. Recalibration of the models, which was performed by both the Cox method and re-estimating coefficients, led to improved discrimination and calibration, although all models still showed significant departures from perfect calibration.

Conclusions: Risk prediction models developed in another country require validation and recalibration before being used to provide risk-adjusted outcomes within a new country setting. Periodic reassessment is beneficial to ensure calibration is maintained.

From the Intensive Care National Audit & Research Centre (DAH, ARB, KR), London; Health Services Research, Acute & Critical Care Research Group and School of Health & Related Research (GJP), University of Sheffield, Sheffield; and Medical Statistics Unit, London School of Hygiene & Tropical Medicine (JRC), London, UK.

Supported in part by the Medical Research Council, London, UK (G9813469).

All work was completed at the Intensive Care National Audit & Research Centre, London, UK. The authors do not have any financial interests to disclose.

© 2006 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins