Objective: Using a comprehensive inpatient electronic medical record, we sought to develop a risk-adjustment methodology applicable to all hospitalized patients. Further, we assessed the impact of specific data elements on model discrimination, explanatory power, calibration, integrated discrimination improvement, net reclassification improvement, performance across different hospital units, and hospital rankings.
Design: Retrospective cohort study using logistic regression with split validation.
Participants: A total of 248,383 patients who experienced 391,584 hospitalizations between January 1, 2008 and August 31, 2011.
Setting: Twenty-one hospitals in an integrated health care delivery system in Northern California.
Results: Inpatient and 30-day mortality rates were 3.02% and 5.09%, respectively. In the validation dataset, the greatest improvement in discrimination (increase in c statistic) occurred with the introduction of laboratory data; however, subsequent addition of vital signs and end-of-life care directive data had significant effects on integrated discrimination improvement, net reclassification improvement, and hospital rankings. Use of longitudinally captured comorbidities did not improve model performance when compared with present-on-admission coding. Our final model for inpatient mortality, which included laboratory test results, vital signs, and care directives, had a c statistic of 0.883 and a pseudo-R2 of 0.295. Results for inpatient and 30-day mortality were virtually identical.
Conclusions: Risk-adjustment of hospital mortality using comprehensive electronic medical records is feasible and permits one to develop statistical models that better reflect actual clinician experience. In addition, such models can be used to assess hospital performance across specific subpopulations, including patients admitted to intensive care.
*Division of Research, Kaiser Permanente Northern California, Oakland
†Department of Applied Mathematics and Statistics, Baskin School of Engineering, University of California, Santa Cruz
‡Kaiser Foundation Health Plan, Management, Information and Analysis, Oakland, CA
Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Website, www.lww-medicalcare.com.
Supported by The Permanente Medical Group Inc. and Kaiser Foundation Hospitals Inc. Neither entity played any role in the study design or writing of the manuscript. Funding for some of the algorithms used to extract and format vital signs and care directives also came from a grant from the Sidney Garfield Memorial Fund (Early Detection of Impending Physiologic Deterioration in Hospitalized Patients).
The authors declare no conflict of interest.
Reprints: Gabriel J. Escobar, MD, Division of Research, Kaiser Permanente Northern California, 2000 Broadway Avenue (2101 Webster Annex, 201 R19), Oakland, CA 94612. E-mail: firstname.lastname@example.org.