The Medicare Shared Savings Program (MSSP) now includes 434 accountable care organizations (ACOs) serving more than 7.7 million beneficiaries nationwide. With ongoing plans to increase the percentage of Medicare payments linked to quality and alternative payment models (including ACOs), these numbers are likely to increase. Medicare ACOs aim to promote higher-quality patient care, while reducing healthcare costs, through improved care coordination, application of information technology, care management programs, and other interventions.[3,4] When ACOs in the MSSP succeed in these goals, they may share in the cost savings with Medicare and earn a “shared savings” payment, which is 1 important driver, among many, for ACO formation.
Early results suggest that ACOs are transforming patient care. Collectively, in the first 2 performance years, ACOs in the MSSP realized a total net savings of $848 million for the Medicare Trust Funds and have shown improvement in 27 out of 33 quality measures, leading to $656 million in shared savings payments.[7,8]
To better understand the ACO landscape, preliminary studies have used early year 1 results to examine whether certain organizational characteristics are associated with financial success.[9–11] The Centers for Medicare & Medicaid Services (CMS) reported that ACOs with more experience in the program were more likely to generate shared savings. Initial analyses have also suggested that ACOs with fewer attributed beneficiaries or greater financial incentives for PCPs were associated with earning shared savings.[9,12] Others have found that ACOs with higher expenditure benchmarks, based on location in high cost regions, were more likely to attain shared savings.[12,13] Although early analyses suggest that hospital inclusion has no effect on ability to receive shared savings, this remains an area of active investigation.[15–17]
Although some patterns may be emerging, many questions remain unanswered. As ACOs continue to evolve, it is important to continually reexamine these patterns to see if they remain or to identify new patterns that may emerge. It is also necessary to ensure that findings, such as differential financial success by geographic region, hold up to analyses controlling for other potential factors.
Answering these questions is critical to inform ongoing policy development: recent CMS rule changes incorporate benchmark adjustments, based on regional spending differences. Further research into the factors determining ACO success in shared savings is relevant to patients and providers, offering insight into how ACOs can evolve nationwide to provide better care at a lower cost.
Accordingly, we used MSSP performance results to evaluate ACO shared savings success from 1 year to the next, and to determine which regions of the country are associated with a higher or lower likelihood of earning shared savings. We also investigated whether certain ACO organizational characteristics, such as the inclusion of a hospital or academic medical center (AMC), or size in terms of beneficiaries covered by an ACO, might explain differences in ACOs’ ability to earn shared savings. Finally, we examined the regional connection between average regional expenditures and ACO financial success.
We collected primary data on ACOs from the CMS ACO performance datasets for year 1 and year 2.[19,20] We examined all 339 MSSP ACOs that initiated in April and July 2012, January 2013, and January 2014.
To analyze whether hospital or AMC inclusion might affect the ability to earn shared savings, we collected information on MSSP ACOs via the CMS-required public reporting information on official ACO websites. Websites were identified either using CMS-provided information or through Internet searches. Data about ACO composition were collected between June 3 and July 23, 2014, for most ACOs; data on 25 ACOs that did not have information online at that time were collected in December 2015. Therefore, we had information on all 339 ACOs.
To investigate other ACO organizational characteristics, we added to our dataset information from publicly available CMS data, such as the number of beneficiaries and the states where beneficiaries reside for each ACO. To analyze regional differences in the proportion of ACOs earning shared savings, we used the states where each ACO's beneficiaries reside to locate ACOs within 10 CMS-defined regions. Some ACOs operate in more than 1 state and more than 1 CMS region; we kept track of this during data collection.
To determine whether the ability of an ACO to earn shared savings was associated with regional fee-for-service (FFS) spending differences, we used publicly available Medicare FFS data at a state level. By accounting for the number of baseline Medicare beneficiaries in each state, we could calculate a weighted average regional total per capita FFS cost for each CMS region. For example, CMS region 7 contains Iowa (IA), Missouri (MS), Kansas (KS), and Nebraska (NE). Therefore, the calculation for per capita cost in CMS region 7 is:
Regional Per Capita Cost = (# of beneficiaries in IA) × (per capita FFS Medicare cost in IA) + (# of beneficiaries in MS) × (per capita FFS Medicare cost in MS) + (# of beneficiaries in KS) × (per capita FFS Medicare cost in KS) + (# of beneficiaries in NE) × (per capita FFS Medicare cost in NE)/Total # of beneficiaries in IA, MI, KS, and NE.
Separately, at the individual ACO level, recognizing that many ACOs operate in multiple states, we approximated the total per capita FFS cost for each ACO's geographic area. If an ACO had beneficiaries residing in more than 1 state, the total number of actual Medicare beneficiaries in each state was used to weight each state's per capita cost, and then a single geographical average per capita cost was calculated as a proxy for the individual ACO. For example, if an ACO had beneficiaries in Maryland (MD) and Virginia (VA), the calculation for their average geographical per capita cost is:
ACO Regional Per Capita Cost = (# of beneficiaries in MD × per capita FFS Medicare cost in MD) + (# of beneficiaries in VA × per capita FFS Medicare cost in VA)/Total # of beneficiaries in MD and VA.
2.1 Data analysis
First, to assess whether ACO performance improves over time, we compared ACOs earning shared savings in performance year 1 versus year 2.
Second, to evaluate regional differences in ACOs’ ability to earn shared savings, we compared CMS-defined regions according to the percentage of ACOs earning shared savings in 2014. We also investigated the organizational characteristics of regions where more ACOs earned shared savings to see if they differed according to previously hypothesized characteristics that might predict success (eg, inclusion of a hospital or AMC, average number of beneficiaries, start date, and total per capita cost). For these comparisons, when an ACO operated in multiple regions, we included them in the calculations for all regions which they operated (though including them in all, none or one region did not affect our results). In order to magnify differences, a bivariate analysis was conducted to compare the 2 most successful regions to all other regions.
Third, we used a logistic regression model to examine whether specific ACO organizational characteristics, including total Medicare per capita FFS cost (at the individual ACO level based on states where beneficiaries reside), hospital inclusion, AMC inclusion, or number of beneficiaries, were associated with the ability of an ACO to earn shared savings. These variables were chosen based on outcomes of prior research, results of our regional bivariate analysis, and the availability of public data.[9,12,14,15]
For comparisons of categorical variables, a χ2 test was used, and for comparisons of means, an unpaired t test was utilized. Study data were collected and managed using REDCap version 5, and statistical analyses, including the multivariate regression, were conducted using STATA version 12 (StataCorp, College Station, TX). Given the use of publicly available data at an organizational level, ethical board approval was not required for the analysis.
3.1 General ACO characteristics
Of the 339 ACOs that initiated operations in 2012 to 2014, 138 (41%) included a hospital, and of these, 47 (34%) contained an AMC within their ACO (Table 1). Forty-eight (14%) ACOs operated in more than 1 region, and the number of ACOs in each CMS region varied (range = 5–93 ACOs). Characteristics of the ACOs and CMS regions are shown in Table 1.
3.2 ACO performance by start date
Overall, a larger proportion of ACOs earned shared savings during their 2nd performance year compared to their 1st (65/214 or 30% in performance year 2 vs 73/339 or 22% in performance year 1; P = .02; data from performance year 2 for ACOs in the 2014 launch cycle were not yet available). Looking only at the most recent performance year, a larger proportion of ACOs that began in the 2012 launch cycle earned shared savings compared to ACOs that initiated operations in 2013 or 2014 (39/111 or 35% vs 47/222 or 21%; P = .002).
3.3 Change in individual ACO performance from year 1 to year 2
Among ACOs that have completed 2 performance years (initiated operations in 2012 or 2013), there was a 25% increase in the number of ACOs that earned a shared savings payment in their 2nd year as opposed to their 1st (65 vs 52) (Fig. 1).
Of the 168 2012/2013-start ACOs that did not earn a shared savings payment in performance year 1, 26 (15%) were able to earn a shared savings payment the following year (Fig. 1).
On the other hand, of the 52 ACOs that earned a shared savings payment in performance year 1, 39 (75%) were able to earn a shared savings payment in performance year 2.
3.4 Shared savings by CMS region
A larger percentage of ACOs in CMS regions 4 (GA, KY, TN, NC, SC, AL, MS, and FL) and 6 (TX, NM, OK, AR, and LA) earned shared savings payments in both performance year 1 and performance year 2 (Fig. 2). In 2014, 18/40 (45%) ACOs in region 6 earned a shared savings payment and 33/93 (35%) ACOs in region 4 earned a shared savings payment compared to 36/203 (18%) ACOs not operating in either region 4 or 6 (P < .001). On the other hand, regions 8 and 10 were less successful than other regions, as none (0%) of the 14 ACOs in region 8 or 10 earned a shared savings payment compared to 86 (27%) ACOs in all other regions combined (P = .05).
3.5 ACO composition by CMS region
In term of ACO composition, ACOs operating in regions 4 and 6 were less likely to include a hospital compared to ACOs in all other regions (31% vs 48%; P = .001) (Table 2). Additionally, ACOs operating in these regions were less likely to include an AMC compared to all other ACOs (7% vs 17%; P = .006). However, there was no statistical difference nationally in the proportion of ACOs earning savings based on the inclusion of a hospital (23% vs 28%; P = .35) or an AMC (23% vs 26%; P = .68).
Regions 4 and 6 also had fewer beneficiaries per ACO compared to all other regions combined (P < .01) (Tables 1 and 2). Conversely, regions 7 and 10 had a larger median number of beneficiaries per ACO compared to all other regions combined (P < .01).
Last, we found that there were no statistically significant differences in the percent of experienced ACOs (2012/2013 start dates) within each region.
3.6 Average cost per beneficiary by CMS region
We found that the 2 regions with the largest proportion of ACOs earning shared savings payments (regions 6 and 4) were also the regions with the highest Medicare total per capita cost for 2014 (Tables 1 and 2). Additionally, the 2 regions where no ACOs earned a shared savings payment (regions 8 and 10) were the 2 regions with the lowest Medicare per capita costs.
3.7 Multivariate regression analysis
When we conducted a multivariate regression analysis, utilizing the variables found to be significantly different in our regional bivariate analysis comparing regions 4 and 6 versus other regions (hospital inclusion, AMC inclusion, number of beneficiaries, and total Medicare per capita cost), only the total Medicare per capita cost remained significant in predicting the ability of an ACO to earn a shared savings payment (P < .001) (see Table, Supplemental Content, http://links.lww.com/MD/B753, which displays the output of the regression analysis). In our model, as the total Medicare per capita cost in the states where the ACO operates increased, the predicted probability the ACO would earn shared savings increased (Fig. 3). For example, the predicted probability that an ACO with a regional per capita cost of $7000 would earn shared savings was 7%. This predicted probability would rise to 53% if the ACO's regional per capita cost were $11,000.
Our study indicates that ACOs operating in areas with higher FFS per capita costs are more likely to be successful at earning shared savings. In addition, experience appears to matter; over time, more ACOs are achieving shared savings. However, the number of beneficiaries in an ACO or the inclusion of a hospital or AMC did not appear to influence an ACO's ability to earn shared savings.
We observed stark differences in the ability of ACOs to earn shared savings according to CMS region. Although ACOs in more successful regions tended to be smaller and were less likely to include a hospital, in our model, only regional total per capita FFS cost predicted ACO success. Additionally, the incremental increase in the probability of an ACO to earn shared savings increased as the per capita cost rose to higher levels.
Our methodology was distinct, but our findings concur with several unpublished preliminary analyses that suggest ACOs operating in more costly hospital referral regions (HRRs) or with higher benchmark expenditures were more likely to be successful in generating savings. However, our results did not support the idea that hospital inclusion (or absence) makes ACOs more likely to earn shared savings or that smaller ACOs (in terms of number of attributed beneficiaries) were more successful. Importantly, we included additional performance year data from additional ACOs and controlled for characteristics not included in these prior studies.
One apparent explanation for this association is that ACOs in high cost regions have more room for improvement. Therefore, there may be excess expenditures that could be reduced more easily through ACOs’ care coordination and quality improvement programs in these regions. Importantly, however, our model accounted for 7% of the observed variation in ACO's ability to earn shared savings. This means there are other factors that influence ACOs’ ability to earn shared savings. Additional factors to consider for future research might include stability of beneficiary assignment within specific ACOs (as keeping care within the ACO has been considered important to ACO success), access to primary care and/or specialty services, socioeconomic status of beneficiaries or associated social determinants of health, allocation of spend between inpatient and outpatient settings, or data on specific ACO interventions, among others. Unfortunately, there is no national dataset of these locally specific ACO efforts.
Experienced ACOs appear more likely to be successful compared to nascent ACOs. Although CMS assessed the percent of ACOs generating any savings, we reached a similar conclusion when analyzing the percent of ACOs earning a shared savings payment. Specifically, 30% of ACOs in their 2nd performance year were able to earn a shared savings payment compared to 22% of ACOs during their 1st performance year. Additionally, ACOs initiating operations in 2012 were more likely to have earned a shared savings payment to date compared to ACOs that initiated operations in 2013 or 2014. There are several possible explanations for these findings. First, ACOs joining the MSSP at an earlier date may have already been part of integrated health systems or were more advanced in their transformation efforts, and thus were better able to implement cost saving measures across their organization. Second, many cost saving initiatives take time to operationalize and then to accrue financial benefits for the organizations, so ACOs that implemented initiatives earlier on may have had more time benefit from these changes. Last, ACOs may be learning from experience, such as which methodologies result in cost savings, and consequently experienced ACOs may be more likely to succeed in this regard.
Further supporting this idea, we found that several ACOs failing to reduce expenditures during their performance year 1 were able to achieve shared savings during year 2. Seeing this improvement is important, as it provides further motivation for ACOs that have yet to achieve savings, and shows that ACOs are able to improve care delivery over time. Closer case study analysis of this cohort of ACOs could help inform on these “lessons learned.”
Additionally, we found that prior ACO success is highly associated with future success, as 75% of ACOs earning shared savings during year 1 were able to achieve savings during year 2. Perhaps this is due to the fact that these ACOs had successful cost saving initiatives that provided continued benefits from year 1 to year 2 or have matured in their deployment or uptake.
Finally, when examining our 2 themes together – regional differences in spending and ACO years of experience – there were no significant differences in the percent of experienced ACOs by region. That is, experience alone did not explain the higher proportion of ACOs that were able to earn shared savings in regions 4 and 6.
Our study had limitations. First, data regarding the composition of ACOs were not verified with the 339 ACOs examined, and participants may have changed since the time of our data abstraction. Second, our findings reflect associations, not necessarily a determination of causality. As for our regression analysis, the Medicare per capita FFS cost was an estimate. Data were not available on the actual distribution of an ACO's population within different states and therefore the weighted average would be an estimate. However, because ACOs that operate in multiple states typically do so in neighboring states, which typically have similar cost profiles, we expect that the impact of this methodology on our results would be minimal. Last, our regression model contained a limited number of variables from publicly available data. Although using publicly available data afforded a comprehensive examination of ACOs, there may be other variables, which once publicly available, could be included in future studies to enhance the explanatory power of such a model.
By examining publicly reported financial data for the 1st 2 performance years of the MSSP, we have examined the increased success of experienced ACOs and have explored reasons for regional differences in the ability of ACOs to earn shared savings. ACOs appear to be getting more successful over time: more ACOs experienced financial success in their 2nd years than their 1st, and more experienced organizations, from the perspective of years in the ACO model, remain the most likely to achieve shared savings. Additionally, the ability to achieve savings varies significantly by region. Variation in regional healthcare costs appears to be playing a large role in these differences, while the number of beneficiaries per ACO and hospital inclusion DO not. Still, differences in regional healthcare costs appear to explain only a small amount of the variation in ACO performance, suggesting a need for additional research at the national, regional, and local ACO levels. As ACOs and the MSSP continue to evolve, it will be important to examine how organizational characteristics and program policies impact all ACOs, in order to provide all ACOs with the opportunity to achieve long-term success while providing high quality care to Medicare beneficiaries.
. Burwell SM. Setting value-based payment goals – HHS efforts to improve U.S. health care. N Engl J Med 2015;372:897–9.
. Fisher ES, Shortell SM, Kreindler SA, et al. A framework for evaluating the formation, implementation, and performance of accountable care organizations. Health Aff (Millwood) 2012;31:2368–78.
. Colla CH, Wennberg DE, Meara E, et al. Spending differences associated with the Medicare Physician Group Practice Demonstration. JAMA 2012;308:1015–23.
. Siddiqui M, Berkowitz SA. Shared savings models for ACOs-incentivizing primary care physicians. J Gen Intern Med 2014;29:832–4.
. Jacob JA. Medicare ACOs improving quality of care: CMS report. JAMA 2015;314:1683.
. Schulz J, DeCamp M, Berkowitz SA. Medicare shared savings program: public reporting and shared savings distributions. Am J Manag Care 2015;21:546–53.
. Epstein AM, Jha AK, Orav EJ, et al. Analysis of early accountable care organizations defines patient, structural, cost, and quality-of-care characteristics. Health Aff (Millwood) 2014;33:95–102.
. Camargo R, Camargo T, Deslich S, et al. Accountable care organizations: financial advantages of larger hospital organizations. Health Care Manag (Frederick) 2014;33:110–6.
. Pham HH, Cohen M, Conway PH. The pioneer accountable care organization model: improving quality and lowering costs. JAMA 2014;312:1635–6.
. Colla CH, Lewis VA, Tierney E, et al. Hospitals participating in ACOs tend to be large and urban, allowing access to capital and data. Health Aff (Millwood) 2016;35:431–9.
. Rittenhouse DR, Shortell SM, Fisher ES. Primary care and accountable care – two essential elements of delivery-system reform. N Engl J Med 2009;361:2301–3.
. Harris P, Taylor R, Thielke R, et al. Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81.
. McWilliams JM, Chernew ME, Dalton JB, et al. Outpatient care patterns and organizational accountability in Medicare. JAMA Intern Med 2014;174:938–45.