Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Evaluation of two outcome prediction models on an independent database

Moreno, Rui MD; Miranda, Dinis Reis MD, PhD; Fidler, Vaclav PhD; Van Schilfgaarde, Reinout MD, PhD

Clinical Investigations

Objective To evaluate the performance of the New Simplified Acute Physiology Score (SAPS II) and the admission Mortality Probability Model (MPM0) in a large independent database, using formal statistical assessment.

Design Analysis of the database of a multicenter, multinational, prospective cohort study, EURICUS-I.

Setting Eighty nine intensive care units (ICUs) from 13 European areas.

Patients Data of 16,060 patients consecutively admitted to the participating ICUs were collected during a period of 4 months. Following the original SAPS II and MPM0 criteria, the analysis excluded: patients <18 ys of age; readmissions; patients admitted with acute myocardial infarction; burns; and patients in the post-operative period after coronary artery bypass surgery. All patients with a length of stay <8 hrs were excluded from the study to keep comparability between both systems. A total of 10,027 patients were analyzed.

Interventions Collection of the first 24 hrs' admission data necessary for the calculation of SAPS II and MPM0 and basic demographic statistics. Vital status at discharge from the hospital was registered.

Measurements and Main Results Despite having a good discriminative capability, as measured by the area under the receiver operating characteristic (ROC) curves (SAPS II: ROC = 0.822 +/- 0.005 SEM; MPM0: ROC = 0.785 +/- 0.006 SEM), both models presented poor calibration, with significant differences between observed and predicted mortality (Hosmer-Lemeshow goodness-of-fit tests H and C, p < .0001). Both SAPS II (predicted risk >40%) and MPM0 (predicted risk >30%) overestimated the risk of death. The evaluation of the uniformity of fit of SAPS II and MPM0 demonstrated large variations across the various subgroups of patients.

Conclusions The original SAPS II and MPM0 models did not accurately predict mortality on an independent large international multicenter ICU patient database. Results of studies utilizing general outcome prediction models without previous validation in the target population should be interpreted with prudence. (Crit Care Med 1998; 26:50-61)

From the Intensive Care Unit, Hospital de Santo Antonio dos Capuchos, Lisboa, Portugal (Dr. Moreno); Intensive Care Unit, Department of Surgery, University Hospital of Groningen, Groningen, The Netherlands (Dr. Miranda); Faculty of Medicine, Department of Health Sciences, University Groningen, Groningen, The Netherlands (Dr. Fidler); Department of Surgery, University Hospital of Groningen, Groningen, The Netherlands (Dr. Van Schilfgaarde).

Supported, in part, by the Foundation for Research on Intensive Care in Europe (FRICE) and by a grant from the Commission of the European Communities in the scope of EURICUS-I project (Contract BMH1-CT93-1340).

Address requests for reprints to: Dr. Dinis Reis Miranda, Intensive Care Unit, Department of Surgery, University Hospital of Groningen, P.O. Box 30.001, 9700 RB Groningen, The Netherlands.

© Williams & Wilkins 1998. All Rights Reserved.