Background. We examine the ability of various publicly available risk models to identify high-cost individuals and enrollee groups using multi-HMO administrative data.
Methods. Five risk-adjustment models (the Global Risk-Adjustment Model [GRAM], Diagnostic Cost Groups [DCGs], Adjusted Clinical Groups [ACGs], RxRisk, and Prior-expense) were estimated on a multi-HMO administrative data set of 1.5 million individual-level observations for 1995–1996. Models produced distributions of individual-level annual expense forecasts for comparison to actual values. Prespecified “high-cost” thresholds were set within each distribution. The area under the receiver operating characteristic curve (AUC) for “high-cost” prevalences of 1% and 0.5% was calculated, as was the proportion of “high-cost” dollars correctly identified. Results are based on a separate 106,000-observation validation dataset.
Main Results. For “high-cost” prevalence targets of 1% and 0.5%, ACGs, DCGs, GRAM, and Prior-expense are very comparable in overall discrimination (AUCs, 0.83–0.86). Given a 0.5% prevalence target and a 0.5% prediction threshold, DCGs, GRAM, and Prior-expense captured $963,000 (approximately 3%) more “high-cost” sample dollars than other models. DCGs captured the most “high-cost” dollars among enrollees with asthma, diabetes, and depression; predictive performance among demographic groups (Medicaid members, members over 64, and children under 13) varied across models.
Conclusions. Risk models can efficiently identify enrollees who are likely to generate future high costs and who could benefit from case management. The dollar value of improved prediction performance of the most accurate risk models should be meaningful to decision-makers and encourage their broader use for identifying high costs.
Risk-adjustment models (risk models) can contribute to efficient care in health maintenance organizations (HMOs) by acting as population-based screens for enrollees at relatively higher risk of generating large future healthcare expenditures, ie, becoming “high-cost.” In particular, they can identify reliable “early warning signs” of future expense that promote secondary prevention through case management programs. Collaborators at 3 research centers associated with U.S. not-for-profit HMOs have collected a large administrative dataset on the entire enrollee populations of 5 HMOs and part of a sixth for the years 1995 to 1996. These data have been used to develop risk models for use in adjusting payments to health plans for enrollee health status; they also represent a unique opportunity to study risk models as high-cost case identifiers.
This article presents the results of an examination based on these data of the ability of 5 independent risk models to identify high-cost enrollee groups. We have conducted similar exercises using the Global Risk-Adjustment Model (GRAM), 1,2 the current version of which is included here and which was developed on the current data. To minimize bias favoring GRAM, we reestimated the other 4 models on the current data and present results only from a separate validation sample. We expect our results to inform health plan administrators, medical directors, and directors of case and disease management programs, all of whom are concerned with efficiently reducing the illness burden in their populations.
From the *Center for Health Research, Northwest and Hawaii, Kaiser Permanente Northwest, Portland, Oregon.
From †HealthPartners Research Foundation, Minneapolis, Minnesota.
From the ‡Center for Health Studies, Group Health Cooperative of Puget Sound, Seattle, Washington.
This study was supported by the Agency for Healthcare Research and Quality, Kaiser Permanente Northwest, Kaiser Permanente Rocky Mountain, Kaiser Permanente Central East, Kaiser Permanente Northeast, HealthPartners, and Group Health Cooperative of Puget Sound.
Address correspondence and reprint requests to: Richard T. Meenan, PhD, MPH, Center for Health Research, Northwest and Hawaii, Kaiser Permanente Northwest, 3800 N. Interstate Avenue, Portland, OR 97227. E-mail: firstname.lastname@example.org