Objective: To evaluate the performance of risk-adjustment models from the University HealthSystem Consortium and the Agency for Healthcare Research Quality on an administrative dataset for children undergoing congenital cardiac surgery.
Design: Retrospective cross-sectional cohort analysis.
Setting: Multi-institutional database of administrative data provided by the University HealthSystem Consortium.
Patients: Children whose discharge diagnosis had an associated cardiac surgical procedure.
Measurements and Main Results: The performance of two risk-adjustment modeling schemata was measured in terms of discrimination and calibration, and receiver operating characteristic curves were compared. Model calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test. A total of 19,436 patients were included in the analysis with 816 deaths and an unadjusted overall mortality rate of 4.2%. The University HealthSystem Consortium models applied to the entire population resulted in an area under the curve = 0.73, and by comparison, the Agency for Healthcare Research Quality risk-adjustment model revealed area under the curve = 0.86. The risk-adjustment model of the University HealthSystem Consortium subgroup of Circulatory System Major Diagnostic Category 5 showed better performance with area under the curve = 0.81. Calibration using the Hosmer-Lemeshow test failed to show good agreement between the predicted and actual outcomes across the University HealthSystem Consortium mortality risk groups with an overall standardized mortality ratio of 1.2 (95% CI, 1.1–1.3; p < 0.0001) and poor predictive ability for the highest risk group, with a nearly 1.5-fold overprediction of death. The Agency for Healthcare Research Quality model shared similar calibration results with an overall standardized mortality ratio of 1.6 (95% CI, 1.5–1.7; p < 0.0001) and a nearly two-fold underprediction of death in the highest risk group.
Conclusions: Administrative data can be used to create risk-adjustment models in the congenital cardiac surgery population. Risk-adjustment models generated from administrative data may represent an attractive addition to clinically derived models in pediatric congenital cardiac surgery patients and should be considered for use either alone or in combination with clinical data in future analyses where mortality is a measure of performance and quality.
1Department of Pediatrics, Rush University Medical Center, Chicago, IL.
2Rush Center for the Advancement of Patient Safety Research, Rush University Medical Center, Chicago, IL.
3Department of Health Systems Management, Rush University Medical Center, Chicago, IL.
4University HealthSystem Consortium, Chicago, IL.
5Department of Internal Medicine, Rush University Medical Center, Chicago, IL.
*See also p. 548.
Dr. Behal received travel reimbursement from UHC. He was also reimbursed for a presentation at a quality meeting from Dr. Foster Intelligence. The remaining authors have disclosed that they do not have any potential conflicts of interest.
For information regarding this article, E-mail: Jason_Kane@rush.edu