Secondary Logo

Institutional members access full text with Ovid®

Share this article on:

Using Automated Clinical Data for Risk Adjustment: Development and Validation of Six Disease-Specific Mortality Predictive Models for Pay-for-Performance

Tabak, Ying P. PhD*; Johannes, Richard S. MD, MS*†; Silber, Jeffrey H. MD, PhD‡§¶

doi: 10.1097/MLR.0b013e31803d3b41
Original Article

Background: Clinically plausible risk-adjustment methods are needed to implement pay-for-performance protocols. Because billing data lacks clinical precision, may be gamed, and chart abstraction is costly, we sought to develop predictive models for mortality that maximally used automated laboratory data and intentionally minimized the use of administrative data (Laboratory Models). We also evaluated the additional value of vital signs and altered mental status (Full Models).

Methods: Six models predicting in-hospital mortality for ischemic and hemorrhagic stroke, pneumonia, myocardial infarction, heart failure, and septicemia were derived from 194,903 admissions in 2000–2003 across 71 hospitals that imported laboratory data. Demographics, admission-based labs, International Classification of Diseases (ICD)-9 variables, vital signs, and altered mental status were sequentially entered as covariates. Models were validated using abstractions (629,490 admissions) from 195 hospitals. Finally, we constructed hierarchical models to compare hospital performance using the Laboratory Models and the Full Models.

Results: Model c-statistics ranged from 0.81 to 0.89. As constructed, laboratory findings contributed more to the prediction of death compared with any other risk factor characteristic groups across most models except for stroke, where altered mental status was more important. Laboratory variables were between 2 and 67 times more important in predicting mortality than ICD-9 variables. The hospital-level risk-standardized mortality rates derived from the Laboratory Models were highly correlated with the results derived from the Full Models (average ρ = 0.92).

Conclusions: Mortality can be well predicted using models that maximize reliance on objective pathophysiologic variables whereas minimizing input from billing data. Such models should be less susceptible to the vagaries of billing information and inexpensive to implement.

From *Department of Clinical Research, Cardinal Health’s MediQual business, Marlborough, Massachusetts; †Division of Gastroenterology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts; ‡Center for Outcomes Research, The Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; §Department of Pediatrics and Anesthesiology and Critical Care Medicine, The University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania; and ¶The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania.

Preliminary models using data from 2000 to 2001 were presented in the following meetings: the 8th Annual Scientific Meeting of Heart Failure Society of America, 2004 (CHF model); the 70th Annual International Scientific Assembly of the American College of Chest Physicians, 2004 (pneumonia model); the 42nd Annual Meeting of the Infectious Disease Society of America, 2004 (septicemia model); the International Stroke Conference, 2005 (ischemic and hemorrhagic stroke models); and the 54th Annual Scientific Session of the American College of Cardiology, 2005 (AMI model). They were published as abstracts in the Journal of Cardiac Failure, Chest, Stroke, and JACC.

Reprints: Ying P. Tabak, PhD, Director, Biostatistics, Department of Clinical Research, Cardinal Health’s MediQual business, 500 Nickerson Road, Marlborough, MA 01752. Email:

© 2007 Lippincott Williams & Wilkins, Inc.