THE Centers for Medicare & Medicaid Services' (CMS) Quality Improvement Organization (QIO) Program is one of the largest federal programs dedicated to improving health quality for Medicare beneficiaries and is an integral part of the US Department of Health and Human Services' National Quality Strategy for providing better care and better health at lower cost. The mission of the QIO Program is to improve the effectiveness, efficiency, economy, and quality of services delivered to Medicare beneficiaries. From August 2011 through July 2014, as part of the QIO 10th Statement of Work (SOW), the CMS contracted with 3 state QIOs to pilot the “Everyone with Diabetes Counts (EDC)” program to reduce disparities and improve diabetes education and health outcomes among Medicare beneficiaries with diabetes. In West Virginia (WV), the pilot focused on beneficiaries with diabetes living in rural counties. The CMS expanded EDC nationally through the Quality Innovation Network-QIO's 11th SOW contracts, which began in August 2014 and will conclude in July 2019. The goals of these analyses were (1) to explore data available from the pilot program to estimate improvements in health and health care quality and costs as a consequence of this program; and (2) to the extent practical, use this information to make improvements for current and future programs.
BACKGROUND: EDC GOALS AND PROGRAM HISTORY
Diabetes is a serious public health concern in the United States, affecting nearly 29.1 million people or more than 9.3% of the population. Of those with the disease, only about three-fourths (21 million people) have been diagnosed, meaning an estimated 8.1 million people in the United States are living with diabetes but are unaware of it. In addition, another 86 million people are estimated to have prediabetes, a condition that puts people at risk for the disease and, if known, can be managed through lifestyle modification and monitoring to reduce risks.1
Among US residents aged 65 years and older in 2010, 11.8 million (26%) were estimated to have diabetes (diagnosed and undiagnosed).2 The Medicare program currently spends approximately 32% of its budget on treating diabetes and its complications,3 and this spending is projected to grow to be $171 billion by the year 2034.4
Disparities in the occurrence of prediabetes, diabetes, and diabetes-related outcomes are prominent in the United States. African American (odds ratio [OR] = 1.8), American Indian/Alaska Native (OR = 2.4), Hispanic (OR = 1.7), and Native Hawaiian/Pacific Islander (OR = 2.4) adults are all more likely to be diagnosed with diabetes than white adults.5 Diabetes is more common among people who live in rural counties than in urban areas (16.7% vs 13.5%).6
In response to these statistics, the CMS launched “Everyone with Diabetes Counts (EDC),”7 designed as a disparities reduction program, through its QIO contracts. One of the components of the EDC program is to promote and offer evidence-based diabetes self-management education (DSME), focusing on Medicare populations most likely to experience worse outcomes from diabetes, including racial and ethnic minority, rural, and Medicare-Medicaid–eligible beneficiaries, low literacy limitations, and those who may have limited English proficiency.
The goals of the program are to (1) improve health equity by improving health literacy and quality of care among Medicare beneficiaries with diabetes and prediabetes. This is accomplished through knowledge empowerment, enabling beneficiaries to become active participants in their care (person/patient engagement); (2) engage beneficiaries and health care providers to decrease the disparity in recommended diabetes monitoring/testing by improving the frequency of performing eye examinations, foot examinations, monitoring of blood pressure control, weight control, and testing for glycated hemoglobin A1c or glycohemoglobin A1c (HbA1c) and lipids; (3) improve actual clinical outcomes of the aforementioned measures; and (4) facilitate sustainable diabetes education resources by engaging public/private/agency/organization partnerships at the community, state, and national levels. The program has 5 components: (1) recruitment and education of beneficiaries using evidence-based DSME curricula; (2) recruitment and education of physician practices and staff; (3) recruitment of community partners/stakeholders; (4) data collection and analysis; and (5) sustainability planning and implementation.8
EDC began in 2007 as a single-state pilot project in Florida to test the feasibility of promoting the intervention through the QIO contract(s). Early success in that pilot led to an expanded test of replicability in other populations through QIOs in Louisiana, Georgia, Maryland, Mississippi, the US Virgin Islands, Washington, District of Columbia, New York, West Virginia, and Texas spanning from 2008 to 2014. The main purpose of the replicability phase was to engage more varied populations in EDC, and QIOs were given the flexibility to structure implementation plans as needed within local contexts, although the DSME curriculum was not modifiable.
We analyzed the impact of the DSME component of the EDC program on hospital utilization in West Virginia during the 10th SOW to infer the cost savings that might be expected during the 11th SOW national expansion of EDC. We used data from the WV project because the program gathered and retained beneficiary information (last name, last 4 digits of social security number [SSN], and [rarely] beneficiary Medicare ID number) that allowed us to identify DSME completers in fee-for-service (FFS) Medicare claims data. In addition, the data were available within the time frame needed to inform best methods for predicting return on investment during the ongoing 11th SOW. The WV pilot targeted Medicare beneficiaries living in rural counties and held DSME classes between February 1, 2013, and June 31, 2014.
We studied the association between completion of DSME and acute inpatient stays, emergency department (ED) visits, and hospital observation (Obs) stays by constructing a study population from the diabetes standard analytic files (SAFs) for West Virginia covering January 1, 2012, through March 31, 2015. The CMS maintains these SAFs to monitor the impact of diabetes on Medicare beneficiaries and the Medicare program. Medicare beneficiaries are included in diabetes SAFs if they have 1 or more inpatient or ED paid claims representing a face-to-face encounter with a diagnosis of diabetes during the measurement period; or 2 or more hospital outpatient, physician office, home health agency, or skilled nursing facility paid claims representing face-to-face encounters with different dates of service that include a diagnosis of diabetes during the measurement period; no diagnoses indicating gestational diabetes or steroid-induced diabetes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] codes of 251.8, 962.0, 648.8X); FFS Medicare benefits for 11 of 12 months; and Medicare Part B coverage for at least 11 of 12 months. Codes used to identify a diagnosis of diabetes mellitus through the ICD-9-CM system are 250.XX, 357.2, 362.01, 362.02, 366.41, and 648.0X. The diabetes SAFs include all beneficiaries who meet the aforementioned requirements over a 1-year period and are alive at the end of the period. The files are updated and distributed quarterly, and West Virginia's file includes about 65 000 beneficiaries. We chose the date range to ensure that our sample would include health care utilization for at least 4 quarters prior to DSME participation for the earliest participants in EDC (first quarter of 2013) and at least 3 quarters after participation for those participating in the last quarter of EDC (second quarter of 2014).
We limited the study population to beneficiaries in the diabetes SAFs to ensure a standard validated definition of diabetes, access to a representative sample of claims within FFS claims data, and assurance that non-DSME comparisons would have no more than 1 month of Medicare Advantage (MA) plan (HMO) insurance during the interval studied; any hospital claims generated during HMO coverage would be unavailable through our FFS claims data source. The requirements for inclusion in the SAFs also ensured that all DSME completers and comparisons used in our analyses would be alive throughout the time intervals for which we compared their hospital events.
Of the 965 DSME completers in the WV EDC pilot, we were able to match Medicare ID (if available) and/or last name and last 4 digits of SSN for 422 DSME completers within the diabetes analytic files covering the study period (Figure). Possible reasons for a beneficiary who completed the DSME not being in the diabetes analytic file include the following: not having a correct identification number, switching from FFS to MA plan coverage, death, or not meeting the definition for inclusion in the SAF. We used the SAS “Proc Survey Select procedure” to identify a random sample of non-DSME beneficiaries with diabetes frequency matched to the completers at a ratio of 4:1 based on age, gender, and county of residence. We required that the beneficiaries with diabetes who were selected as controls be present in the analytic file during the same time periods as the DSME completers to whom they were matched.
We merged the file of Medicare IDs for the study population (completers and controls) with the Medicare Part A claims data to identify claims originating from hospitalizations, ED visits, and Obs stays for each beneficiary. (See the online Supplement available at: http://links.lww.com/FCH/A8, for a more detailed description of the data sets used in this analysis.)
The primary exposure of interest was DSME completion. Our hypothesis was that diabetes has such a profound influence on overall health status that improved management of diabetes would have implications beyond prevention of diabetes-specific diagnoses. Therefore, our main outcome variable was the number of hospitalizations in the quarter of DSME completion and the following 3 quarters, regardless of whether the admissions were explicitly diabetes-related. Secondary outcomes were ED visits and Obs stays during the same time periods. We included the quarter of DSME participation in the postcompletion year but performed sensitivity analyses by retesting the association excluding the quarter of DSME completion for both completers and controls.
We summarized demographic data from the diabetes SAFs as counts and percentages. We examined associations between the numbers of beneficiaries who had hospitalizations, among the DSME completers and other beneficiaries with diabetes, using either χ2 tests or multivariable regression.
To test our main hypothesis, that completing DSME could significantly affect the number of hospital stays for Medicare beneficiaries with diabetes, we tested several models for count data (Poisson, negative binomial, zero-inflated Poisson, or zero-inflated negative binomial). We used the negative binomial, the simplest model that fit our observed distribution (χ2 goodness of fit, P = .66). We tested for inclusion in the final model 7 variables plausibly related to hospital utilization in the post-DSME period (gender, age group, pre-DSME completion hospitalizations, and indicator variables for having an eye examination, HbA1c test, and lipid test during the pre-DSME completion intervention time period), plus the variable of interest (DSME completer vs non-DSME beneficiaries with diabetes). We initially retained any variable that had a P value of less than .20 for any of the 3 outcomes in all 3 models. We dropped any variable with a P value greater than .05 in all 3 of the final models. We calculated the preintervention number of hospitalizations for the 1-year period prior to the intervention for both the DSME completers and the beneficiaries within the diabetes comparison group, based on the time period that the intervention/frequency matching was completed. We also examined for inclusion in the final model all 2-way interactions for these variables. After developing the final models, we used generalized estimating equations (GEEs) to calculate robust standard errors, given the occurrence of multiple events per individual.
The demographics of the study population were well matched between participants and controls (Table 1). Of note, although race was not included as a matching variable, the 2 groups were fairly balanced within race between DSME completers and non-DSME completers with diabetes. Numbers of hospitalizations, ED visits, and Obs stays by DSME completion status are displayed in Table A.1 in the Supplemental Digital Context Appendix (available at: http://links.lww.com/FCH/A8).
Unadjusted analyses of associations between demographic characteristics and clinical care indicators and the odds of having at least 1 hospitalization demonstrated that people completing DSME classes were significantly more likely to have no hospital encounters both before (61.1% vs 55.1%, P = .025) and after (57.8% vs 52.3%, P = .04) class completion. There were no statistically significant differences between the percentage of DSME completers and non-DSME completers, with at least 1 hospital encounter associated with diabetes in either the pre- or postintervention period (preintervention comparison P = .83, postintervention comparison P = .52). DSME completers were also more likely to receive common diabetes monitoring tests (eye examinations, HbA1c test, lipid profile) in both the pre- and postintervention time periods than the non-DSME completers with diabetes during the same intervals (eye examination: pre P < .0001, post P < .0001; HbA1c test: pre P = .014, post P = .0004; lipid test: pre P = .005, post P = .084) (Table 2).
There was no relationship between DSME completion and having at least 1 diabetes-related hospitalization after adjusting for preintervention hospitalizations (P = .156) (data not shown). However, the total number of such hospitalizations was very small (n = 36).
Univariate analyses of associations between demographic and clinical factors, including the number of hospital events prior to DSME completion and the number of hospital events experienced in the post-DSME period, showed all variables of interest meeting our criteria for inclusion in the final models (Table 3).
Table 4 shows the results in our final models for the number of hospitalizations, the number of ED visits, and the number of Obs stays, respectively. None of the 2-way interaction terms were near our criterion for inclusion for any of the models. Significant variables retained after initial modeling for inclusion in the final models were the number of hospitalizations in the pre-DSME period, age, gender, and DSME status.
For every hospitalization in the preintervention period, there was an expected 66% increase in hospitalizations in the postintervention period (P < .0001), and beneficiaries with diabetes in the highest age group were expected to have 27% more hospitalizations than beneficiaries with diabetes in the (66-70 years) referent age group. After adjusting for the number of hospitalizations in the pre-DSME period and all other model variables, beneficiaries with diabetes who completed DSME were expected to have about 29% fewer hospitalizations (P < .0069) than other beneficiaries with diabetes. Males were expected to have 19% fewer ED visits and 33% fewer Obs stays. Both the ED and Obs stay models produced incidence ratios similar to the hospitalization ratio for the DSME variable, but neither was significant.
We used 2 approaches to ensure that our models were sensitive to small deviances in the variability of the independent variables. First, we repeated the analysis using GEE models to obtain robust standard errors. In the case of all 3 models, all variables showed only very small changes in variability and therefore significance. Second, we excluded the quarter in which DSME was completed. Similar to the first approach for this examination of sensitivity, there was very little deviance from the estimates that resulted with and without this quarter of data included.
We estimated cost savings from DSME in West Virginia by calculating an average hospitalization cost to Medicare for the beneficiaries with diabetes who were included in the diabetes analytic file. These average costs were $10 101 per hospitalization, $353 per ED visit, and $1,881 per Obs stay.
Our estimates of total savings are based on percent change in hospitalizations only in both the DSME completers and non-DSME completers group, since the hospitalization outcome model demonstrated a statistically significant difference between DSME completers and the comparison group. For hospitalizations, the percent change in the DSME completers group pre/post-DSME period was −1.7% whereas in the non-DSME group, the change was 10.7%. Applying these percentages to the DSME completers group's pre-DSME total hospitalizations resulted in a difference of 15 hospitalizations. This translates to an estimated cost savings of $151 515 for the study population and equates to $35 900 in savings per 100 DSME completers in West Virginia (see Table A.2 in the Supplemental Digital Content Appendix, available at: http://links.lww.com/FCH/A8).
In this analysis, we examined the DSME component of the 10th SOW “EDC” program in rural counties in West Virginia for Medicare FFS beneficiaries with diabetes enrolled in DSME classes between February 1, 2013, and June 30, 2014. We observed that, on average, beneficiaries with diabetes who were not identified as completing DSME experienced more hospitalizations during the interval corresponding with “postintervention education” than during the interval corresponding with preintervention education. However, this might be expected, as beneficiaries were older and had lived with diabetes. In contrast, DSME completers had similar numbers of hospitalizations pre- and postintervention education, suggesting a possible decrease from the number of hospitalizations that may have been expected in the absence of the DSME intervention.
This analysis included only 4 quarters of data post-DSME, including a quarter in which DSME was delivered. The types of improvements expected from DSME include not just improved glucose control but also reduced weight gain, as well as hypertension control, benefits that might not immediately translate into reduced hospitalizations. A significant impact of the program after only 4 quarters of follow-up is therefore noteworthy and supports program expansion. Although we intend to continue following EDC participants for longer time periods in the future, it was important to complete this analysis as soon as possible after the 10th SOW to better inform program implementation and evaluation efforts for the 11th SOW. As such, we note several additional limitations that are also relevant for future analyses. First, the comparison group identified through a random sample of Medicare beneficiaries with diabetes and frequency matched on age, gender, county of residence, and quarter of service delivery may have been less healthy than the DSME completers included in our data. The orientation and attitudes of DSME completers and/or beneficiaries willing to share their Medicare ID numbers (for the small subset that did share their ID numbers) may differ substantially from other beneficiaries with diabetes. Also, since characteristics such as attitude and self-motivation cannot be measured in claims, we are unable to measure these differences. We addressed this imbalance by adjusting for the number of hospitalizations in the preimplementation period for each individual beneficiary. Similarly, we acknowledge that a beneficiary cannot complete a 6-week DSME class while in the hospital. Therefore, DSME completers would have been expected to have fewer hospitalizations in the first “post-DSME” quarter. However, after removing this quarter, the association between DSME completion and hospitalization was retained, suggesting that the health of DSME completers in the quarter of DSME completion did not drive the lower rate of hospitalization in the remaining 3 quarters.
Our outcomes of interest are all measures of health care utilization, which have been recognized as an appropriate outcome for assessing self-management support programs.9 However, most evaluations of diabetes education programs have used more immediate outcomes. In the 11th SOW, there will be clinical outcomes data available for the following examinations: blood pressure control, weight control, HbA1c test, eye examination and foot examination rates, along with low-density lipoprotein levels, high-density lipoprotein levels, and lipids, for the pre- and post-DSME completion time periods. In addition, QIOs are capturing both pre- and post-DSME patient activation surveys of most beneficiaries completing DSME classes. These surveys contain self-reported responses to questions regarding demographics, previous diabetes education, length of time with diabetes, knowledge about diabetes, and patient self-efficacy. Resulting data will provide supporting evidence that will allow us to understand whether DSME worked as intended by empowering beneficiaries to better care for themselves. However, hard outcomes such as hospitalizations can be more directly linked to cost and therefore to savings for the Medicare program and will likely remain central to future analyses.
Finally, attempting to collect more identifying information and/or requiring Medicare ID from DSME completers may be important for more comprehensive future evaluations. While WV Medical Institute (WV's QIO in the 10th SOW) was able to capture a small sample of Medicare ID numbers from DSME completers, the CMS required only QIOs to gather these data for 10% of those completing DSME. EDC leaders within the QIO program are sensitive to the fact that requiring identifying information for beneficiaries to participate in the program may create an unintended barrier to participation. Therefore, for this study, it is conceivable that some of the beneficiaries sampled as “noncompleters” were simply unidentified DSME completers. In addition, because DSME classes are multiweeks, we cannot know how many beneficiaries included as matched controls might have participated in some classes but did not complete the full numbers of classes as it was designed for the 10th SOW.
Although the total footprint of the rurally focused WV DSME program is small relative to the total number of beneficiaries with diabetes (number of rural beneficiaries in West Virginia completing DSME = 965; total number of beneficiaries in the WV diabetes analytic file = ∼65 000), the inclusion of unidentified or partial completers in the noncompleter sample would have led these analyses to underestimate the impact of the program. Alternatively, if the identified DSME completers were more committed to improving their health than those who are unidentified, this might have resulted in an overestimation of program impact. We addressed this latter possibility by adjusting for hospitalization in the “preimplementation” period but cannot ensure that we entirely addressed this potential bias.
Finally, because of concerns for privacy, data from one pilot state was destroyed and data from a second pilot state could not be transferred in a timely manner, leaving us only with the data from 1 of 3 states for this analysis. Continued advancement of the CMS evaluation agenda requires that in future we become more proactive in data stewardship that allows for data retention and privacy protection simultaneously.
Despite its limitations, the findings of this analysis, suggesting prevention of 15 hospitalizations and more than $35 900 per 100 DSME completers in 1 state, reinforce the value of DSME. There have been many studies of diabetes education as an intervention. Many have shown that DSME is effective for improving clinical outcomes and quality of life in the short term.10–14 Other studies have correlated education with shorter-term outcomes, such as behavioral changes, increased knowledge regarding diabetes, and improved mental health–related quality of life.10 Most published research did not provide insights into potential impact specifically among older persons living in remote rural areas, and few have focused on specific cost drivers such as hospitalization on this population. Finally, this is the first analysis to focus specifically on DSME delivered within the context of a state-based quality improvement program. Historically, QIOs have worked directly with providers and this effort to engage beneficiaries directly represents a new direction for this important national program. As the QIO program continues to learn lessons about how best to recruit and educate beneficiaries to manage diabetes and document their participation, this analysis presents encouraging early news.