Secondary Logo

Journal Logo

Original Articles

Impact of a Provider Tele-mentoring Learning Model on the Care of Medicaid-enrolled Patients With Diabetes

Cantor, Joel C. ScD*; Chakravarty, Sujoy PhD*; Farnham, Jennifer MS*; Nova, Jose MS*; Ahmad, Sana MD, PhD*,†; Flory, James H. MD, MSCE

Author Information
doi: 10.1097/MLR.0000000000001696

Abstract

Affecting nearly 13% of US adults in 2018, diabetes is a common chronic condition with prevalence expected to be as high as 21% by 2050.1–3 Diabetes is responsible for significant public health, social, and economic costs, imposing significant burden on public payers including Medicaid.2,4,5 Even though quality of care for patients with diabetes has improved over the past 2 decades, diabetes remained the seventh leading cause of death in 2017, with many patients failing to meet recommended care guidelines.2 Racial and ethnic minorities and low-income populations are disproportionately affected by diabetes and experience lower quality of care and higher rates of complications.2,6–11 Diabetic patients often rely on primary care providers (PCPs) for care, but PCPs may face challenges in providing optimal care to patients with complex care needs.12–16 The role of PCPs is especially important considering research highlighting potential shortages of endocrinologists.17 With the advent of newer therapeutics and practice innovations, there are significant opportunities to address unmet need for patients with diabetes in primary care settings.14,16,18,19

Project ECHO (Extension for Community Healthcare Outcomes) is a tele-mentoring program for health care providers introduced in 2003. Subsequently, tele-mentoring programs have been applied globally to a broad range of health conditions in nearly 900 individual programs.20,21 Tele-mentoring programs have been shown to improve provider-reported outcomes including satisfaction, confidence, self-efficacy, and behaviors, with more limited evidence supporting efficacy in patient-level outcomes related to hepatitis C, chronic liver disease, chronic pain management and opioid addiction, and geriatric care.22 A limited number of studies have examined outcomes for patients in Project ECHO programs addressing complex endocrinology (hereafter “EndoECHO”).23,24 One study of 2 PCP organizations showed clinically significant improvement in glycemic control for 39 Veterans Health Administration patients with hemoglobin A1c (HbA1c)  > 9.0%.22,25 Researchers evaluating an EndoECHO intervention in 10 New Mexico health centers for Medicaid enrollees with complex diabetes found increased outpatient visits and higher use of beneficial medications, but also documented increased emergency department (ED) utilization and no impact on inpatient admissions.26

Given the scant evidence on patient outcomes among Project ECHO and similar programs generally, and the limited and mixed evidence for patients with diabetes specifically, there is need for additional patient outcome evaluation of Project ECHO.24 This observation was underscored in a report to Congress which concluded that the evidence for improved patient outcomes from ECHO and ECHO-like models was limited.24,27 This article seeks to address this research gap by evaluating an EndoECHO project in New Jersey (NJ) organized by Rutgers Robert Wood Johnson Medical School (RWJMS) to engage PCPs serving Medicaid enrollees to improve care for patients with complex endocrine and metabolic conditions.

This article draws on 5 years of NJ Medicaid data to compare utilization and spending outcomes among Medicaid patients attributed to PCPs participating in the EndoECHO program to those attributed to matched nonparticipating providers. Difference-in-differences (DD) models at the patient level are utilized to examine program effectiveness. We hypothesized that participation in EndoECHO will be associated with reduced total spending; ED, inpatient, and ambulatory endocrinology utilization and spending; increased PCP visits and spending; and higher use of evidence-based care. We also examine patterns of diabetes medication prescribing patterns because they may mediate other utilization and spending outcomes.

METHODS

Intervention

Fifty-three providers, including 48 PCPs (physicians and nurse practitioners) and 5 pharmacists, enrolled in the RWJMS EndoECHO program, which involved weekly 1-hour tele-mentoring sessions between February 2017 and July 2018. Supplemental Digital Content 1 provides further program details (https://links.lww.com/MLR/C400). The curriculum covered topics related to endocrine and metabolic disorders, with nearly half of sessions focusing on diabetes management. The weekly sessions were intended to promote collaborative problem solving, address behavioral and social barriers to better outcomes, and empower PCPs to address complex endocrinology scenarios. The program was led by a faculty comprising endocrinologists, diabetes educators, pharmacists, and social workers, and involved participant case presentations and discussions of evidence-based guidelines.

Data

Study measures are calculated using NJ Medicaid Management Information System (MMIS) data for calendar years 2015–2019, including enrollee information (eg, enrollment dates, demographics, eligibility category) and comprehensive managed care encounter records and associated provider payments as well as fee-for-service claims and payment amounts. EndoECHO participant rosters and attendance information were provided by RWJMS. We used National Provider Identifiers (NPIs) to identify EndoECHO participating PCPs in the MMIS. The study was approved by a university Institutional Review Board.

Intervention and Comparison Groups

Based on discussions with the RWJMS program faculty and leadership before initiation of the evaluation, we excluded PCPs who completed fewer than 4 of the 64 EndoECHO sessions. We attributed adult (age 18 or older) patients with at least 300 days of Medicaid enrollment in 2016 to participating and nonparticipating PCPs based on the plurality of their primary care visits, following the approach used in Medicare provider incentive programs.28,29 We then created a comparison cohort of up to 5 nonparticipating PCPs based on physician and patient panel characteristics using exact and statistical distance matching. Exact matching was based on provider specialty (Family Medicine, Internal Medicine, OB/GYN, Nurse Practitioner) and place of service most associated with the provider (private office, hospital outpatient, or clinic including federally qualified health centers).

We then employed Mahalanobis minimum distance matching using the “mahapick” procedure in STATA 16.0, based on: size of attributed Medicaid patient panel, size of the attributed panel of patients with diabetes, distributions of patient demographics (sex, race/ethnicity, age), share of attributed patients in the Affordable Care Act Medicaid expansion population, share of attributed patients with dual Medicare-Medicaid enrollment, and mean Chronic Illness and Disability Payment System (CDPS) patient risk classification score. To ensure thorough measurement of outcomes, patients with <300 days of Medicaid enrollment in any study year were excluded.

Outcomes and Analysis

We contrasted trends in outcomes between patients attributed to EndoECHO participating and nonparticipating PCPs over periods spanning January 2015 to December 2016 (before initiation of the EndoECHO intervention), January 2017 to June 2018 (during the intervention period), and July 2018 to December 2019 (following its conclusion). Although the intervention took place February 2017 to July 2018, data available do not permit a more precise alignment of dates. We divided study patients into 3 groups for analysis based on diagnostic codes in 2016: (1) patients with diabetes, (2) those without diabetes but with another endocrine or metabolic condition that was addressed in the EndoECHO curriculum, and (3) all other patients.

For each of these groups, we calculated outcomes in 5 domains, measured quarterly: total Medicaid spending and utilization and spending for ED visits, inpatient admissions, primary care visits, and ambulatory visits to endocrinologists. In addition, among patients with diabetes, we calculated trends in outcomes reflecting the adequacy of diabetes management, also measured quarterly: admissions and spending for short-term complications of diabetes, admissions and spending for any complication of diabetes or uncontrolled diabetes without complications, receipt of recommended HbA1c testing, and receipt of recommended eye exams. Finally, in addition to the above noted utilization and spending measures, we compared trends in prescriptions by class of diabetes medication. To ensure stable estimates of prescribing patterns, we compared average prescription rates for each drug class in the preintervention to postintervention periods.

The population with a diagnosis of diabetes was identified in 2016 data using specifications of the Healthcare Effectiveness Data and Information Set (HEDIS).30 Diagnoses of nondiabetes endocrine or metabolic conditions addressed in the EndoECHO curriculum used to classify the second patient group were identified by co-authors S.A. and J.H.F., and are listed in Supplemental Digital Content 2 (https://links.lww.com/MLR/C401).

Primary care visits were defined as evaluation and management visits to a PCP in an ambulatory setting with HCPCS/CPT codes for outpatient or preventive care visits or consultations (99201-99215; 99241-99245; or 99381-99397). Inpatient diabetes short-term complications and the composite diabetes admission measures were calculated using Agency for Healthcare Research and Quality (AHRQ) Prevention Quality Indicators (PQI-01 and PQI-93, respectively).31 Adherence with recommended HbA1c and retinal eye examination testing followed HEDIS technical specifications.32 We followed the diabetes medication classification method used by Flory et al33; and hospitalizations for endocrine and related conditions were classified using AHRQ clinical classification software (this measure is not available for 2015).34

Our matching strategy accounted for potential provider selection into the program that may be correlated with panel characteristics and outcomes.35 We assessed the quality of the match by examining standardized differences between the treatment and matched comparison group in matching attributes. For each outcome measured quarterly we estimated DD models to test for program effects. While the use of a matched comparison group assures the accuracy of estimators even if preintervention trends are not parallel,35,36 we tested for nonparallel trends and, where appropriate, adjusted estimators for differential pretrends.37 We conducted linear probability modeling for binary outcomes such as annual tests or ordinary least square for continuous and count variables. Standard errors are clustered at the provider level to account for non-independence of observations. For the analysis of changes in prescribing patterns, we compared average rates in the preintervention to postintervention periods and applied χ2 tests.

We conducted supplemental analyses to test for variations in program effects and confirm the veracity of our models. Our main DD models measure outcomes among patients with diabetes. Supplementary models test for program effects on spending and utilization variables that are not specifically diabetes related in 2 other groups of patients, that is those with other program-related endocrine conditions and all other patients. While it is possible that some skills acquired during program participation may lead to better care for patients without diabetes, for example, addressing social determinants of health, we did not expect to find substantial program effects in these supplemental analyses. That is, we consider them to be “falsification tests” where findings of significant program effects would raise suspicions of unmeasured selection bias (eg, providers disproportionately motivated to improve are more likely to enroll) or some other methodological artifact.

Finally, to test whether program effects vary by level of PCP participation we estimated an additional model with separate DD parameters for participants attending fewer (between 4 and 19) weekly sessions and those attending more (up to 64 sessions). A finding of a positive “dose response” would strengthen our inferences about program impact.

RESULTS

Twenty-five of the 53 enrolled participants were available for the study. Two were excluded because no NPI was available, 4 because they did not attend at least 4 EndoECHO sessions, 18 had no Medicaid claims in 2016, 2 were not PCPs, and 2 had no primary care claims. On average, participants attended under a third of the sessions (19 of the 64).

Each of the 25 EndoECHO participating PCPs were matched to 5 comparison providers, and among these 125 matches there were 122 unique PCPs (ie, 3 nonparticipants matched to >1 participant). After we applied the criterion of minimum Medicaid enrollment of 300 days for attributed patients in each study year, 119 comparison PCPs remained for available for analysis. A sample disposition chart is provided in Supplemental Digital Content 3 (https://links.lww.com/MLR/C402).

Our final patient sample includes 1776 patients attributed to participating PCPs, including 318 with diabetes, and 9126 attributed to the comparison providers, including 1454 with diabetes. Table 1 shows that intervention and comparison cohorts are well balanced, with just 1 variable, the proportion of patients of Black race, exceeding a standardized difference of 0.10. Although matching was based on the characteristics of all patients attributed to participating and comparison providers, we observed wider standardized differences for some patient characteristics when calculated based on diabetic patients alone (Supplemental Digital Content 3, https://links.lww.com/MLR/C402). Nevertheless, the use of DD models ensures unbiased estimation of policy effects based on relative changes in outcomes between participating and comparison populations.

TABLE 1 - Intervention and Comparison Group Provider Panel Characteristics
Provider Panel Characteristics EndoECHO Group, Mean (SD) Comparison Group, Mean (SD) Standardized Difference
Mean panel size 167.30 (210.42) 158.00 (158.97) −0.05
Mean diabetic panel size 23.44 (27.13) 21.97 (21.64) −0.06
Mean patient age 45.59 (10.69) 45.58 (8.77) 0.001
Mean CDPS score 2.28 (1.62) 2.15 (1.25) −0.09
Proportion of patients
 Male 0.33 (0.17) 0.32 (0.14) −0.05
 Black (non-Hispanic) 0.33 (0.30) 0.27 (0.22) −0.23
 Hispanic 0.20 (0.22) 0.22 (0.18) 0.09
 Dual Medicare-Medicaid eligible 0.18 (0.27) 0.17 (0.19) 0.02
 Eligible under ACA expansion* 0.38 (0.22) 0.37 (0.15) −0.08
*Category also includes a small number of General Assistance patients who were eligible before the expansion.
ACA indicates Affordable Care Act; CDPS, Chronic Illness and Disability Payment System risk adjustment.

Most DD estimates for our primary utilization and spending outcomes among patients with diabetes (Table 2) have signs consistent with hypothesized program effects, with measures of inpatient admissions and spending achieving statistical significance. We found 3.1 fewer inpatient admissions per 100 patient-quarters from the preprogram to postprogram period, a 44.3% reduction, among Medicaid enrollees attributed to EndoECHO participants compared with those of nonparticipating PCPs (P=0.001). From the preprogram to postprogram periods, the number of admissions for “endocrine, nutritional, and metabolic” diseases increased by 19.2% (from 26 to 31) among patients attributed to EndoECHO PCPs but they more than doubled (119.1%; from 89 to 195) among comparison group patients (χ2=4.641, P=0.098). Inpatient spending was $327 lower per patient of intervention PCPs per quarter relative that of nonparticipating PCPs, a reduction of 61.9% (P=0.021). Our estimate of the program effect on total Medicaid spending is close to the magnitude of the reduction in hospital spending but did not achieve statistical significance.

TABLE 2 - Difference-in-Differences Estimates for Patients With Diabetes
Outcome Baseline Mean Estimate (SE) P
General utilization and spending
 ED visits 0.288 −0.036 (0.030) 0.227
 ED spending ($) 91.396 −6.697 (13.218) 0.613
 Inpatient admissions 0.070 −0.031 (0.009)*** 0.001
 Inpatient spending ($) 528.446 −327.366 (140.543)** 0.021
 Primary care visits 1.395 0.073 (0.080) 0.360
 Primary care spending ($) 48.134 7.499 (6.532) 0.253
 Visits to endocrinologists 0.101 −0.015 (0.024) 0.533
 Endocrinologist spending ($) 5.350 0.655 (2.056) 0.751
 Total spending ($) 3788.778 −259.925 (425.595) 0.543
Diabetes-related utilization and spending
 Diabetes short-term complication hospitalizations 0.003 −0.002 (0.003) 0.340
 Diabetes short-term complication hospitalization spending ($) 18.962 −9.911 (13.019) 0.448
 All diabetes hospitalizations 0.007 −0.006 (0.004) 0.138
 All diabetes hospitalization spending ($) 55.347 −60.589 (33.392)* 0.072
  Recommended HbA1c testing 0.627 −0.036 (0.031) 0.239
  Recommended eye exam 0.006 −0.011 (0.006)* 0.094
Prescriptions of diabetes-related medications
 Metformin 0.713 −0.064 (0.048) 0.182
 Insulin 0.322 −0.046 (0.021)** 0.031
 Sulonylurea 0.354 −0.017 (0.027) 0.524
 Dipeptidyl peptidase-4 (DPP4) 0.222 −0.053 (0.039) 0.177
 Glucagon-like peptide 1 receptor agonists (GLP1-RA) 0.055 −0.002 (0.021) 0.927
 Sodium glucose cotransporter 2 inhibitors (SGLT2-I) 0.062 −0.008 (0.014) 0.562
 Combination medications 0.089 −0.008 (0.029) 0.782
N=28,352 all diabetes hospitalizations and 35,440 for all other outcomes.
Three drug classes (Thiazolidinedione, Meglinitinde, and Alphaglucosidase inhibitors) that represented <0.03% of prescriptions in the baseline or postintervention periods are not shown.
ED indicates emergency department; SE, standard error, reported in parenthesis.
*P<0.1.
**P<0.05.
***P<0.01.
Source: New Jersey Medicaid Management Information System.

Among other utilization and spending measures that are not diabetes-specific, only the DD estimate for endocrinologist spending is not in the hypothesized direction. None of the estimates reflecting adequacy of diabetes care management achieved significance at the P<0.05 level. Estimates for the measures of potentially avoidable diabetes-related hospital admissions and spending are signed in the expected direction, but recommended care estimates are not. DD estimates for prescribing of 6 classes of diabetes medications showed no significant changes, except for insulin which decreased more among EndoECHO patients than the comparison patients; this change may reflect improved diabetes management using noninsulin agents.38

Variations in Outcomes by Level of PCP Participation

Table 3 shows outcomes by whether EndoECHO PCPs attended 4 to 19 or >19 of the 64 program sessions. About half (53%) of patients with diabetes in the sample were attributed to 8 PCPs with greater attendance. Nine of the 15 DD estimates for our primary outcomes evince stronger program effects in the group attending more sessions, albeit most without achieving statistical significance. Notably, the “dose response” pattern in the hypothesized direction was evident for utilization and spending in the ED and short-term and total diabetes-related admissions, as well as for inpatient utilization. We estimated 4.5 fewer inpatient stays per 100 patient-quarters of high-attendance PCPs (P<0.001) but only a 1.8 stay reduction per 100 patient-quarters of low-attendance PCPs (N.S.), relative to the comparison group. The composite measure of admissions for uncontrolled diabetes and diabetes complications shows a similar pattern, with a reduction of 1.3 admissions per 100 patient-quarters in the high-attendance group (P=0.017) in contrast to no significant change in the low-attendance group. Estimates of program impacts on primary care utilization and spending and recommended HbA1c testing and eye exams were not consistent with hypothesized program effects among high-attendance PCPs.

TABLE 3 - Difference-in-Differences Estimates for Patients With Diabetes by Number of EndoECHO Sessions Attended
Outcome Baseline Mean Estimate (SE) P
General utilization and spending measures
 ED visits
  4–19 sessions 0.288 −0.015 (0.037) 0.688
  20 or more sessions −0.057 (0.041) 0.167
 ED spending ($)
  4–19 sessions 91.396 0.571 (17.904) 0.975
  20 or more sessions −13.875 (16.268) 0.395
 Inpatient admissions
  4–19 sessions 0.070 −0.018 (0.012) 0.129
  20 or more sessions −0.045 (0.008)*** 0.000
 Inpatient spending ($)
  4–19 sessions 528.446 −568.511 (173.445)*** 0.001
  20 or more sessions −331.090 (171.868)* 0.056
 Primary care visits
  4–19 sessions 1.395 0.155 (0.126) 0.221
  20 or more sessions −0.007 (0.084) 0.934
 Primary care spending ($)
  4–19 sessions 48.134 15.869 (10.541) 0.135
  20 or more sessions −0.766 (6.281) 0.903
 Visits to endocrinologists
  4–19 sessions 0.101 −0.028 (0.030) 0.354
  20 or more sessions −0.002 (0.032) 0.944
 Endocrinologist spending ($)
  4–19 sessions 5.350 3.558 (3.198) 0.268
  20 or more sessions −2.212 (2.135) 0.302
 Total spending ($)
  4–19 sessions 3788.778 451.397 (454.455) 0.323
  20 or more sessions −962.355 (524.307)* 0.069
Diabetes management related measures
 Diabetes short-term complication hospitalizations
  4–19 sessions 0.003 0.002 (0.001)* 0.054
  20 or more sessions −0.007 (0.004)* 0.058
 Diabetes short-term complication hospitalization spending ($)
  4–19 sessions 18.962 7.873 (13.440) 0.559
  20 or more sessions −27.473 (17.333) 0.116
 All diabetes hospitalizations
  4–19 sessions 0.007 −0.0001 (0.005) 0.988
  20 or more sessions −0.013 (0.005)** 0.017
 All diabetes hospitalization spending ($)
  4–19 sessions 55.347 −21.840 (27.826) 0.434
  20 or more sessions −98.854 (49.180)** 0.047
 Recommended HbA1c testing
  4–19 sessions 0.627 −0.021 (0.044) 0.630
  20 or more sessions −0.051 (0.035) 0.142
 Recommended eye examination
  4–19 sessions 0.006 −0.019 (0.009)** 0.036
  20 or more sessions −0.013 (0.007)* 0.085
N=28,352 all diabetes hospitalizations and 35,440 for all other outcomes.
Estimates for Inpatient Spending and Recommended Eye Exam are adjusted for nonparallel preintervention trends.
ED indicates emergency department; SE, standard error reported in parenthesis.
*P<0.1.
**P<0.05.
***P<0.01.
Source: New Jersey Medicaid Management Information System.

Falsification Analysis

We repeated the DD analyses of general utilization and spending outcomes in 2 additional groups of patients: those without diabetes but with another endocrine and metabolic condition that was addressed in the EndoECHO curriculum, and patients with neither diabetes nor any curriculum-related conditions. With the plurality of the EndoECHO curriculum addressing topics in diabetes management, we hypothesized that significant program effects in patients would be found mainly among patients with diabetes. While the program plausibly could lead to better care for patients with other curriculum-related conditions, this group is heterogeneous, thus we expected that we would not find significant evidence of program impacts. Finally, we did not expect to find any program effects among patients with neither diabetes nor other curriculum-related conditions.

Overall, the program effects among diabetic patients were not identified in these 2 groups, with 2 exceptions (Supplemental Digital Content 4, https://links.lww.com/MLR/C403). We found a comparatively large ($623 per patient-quarter) and statistically significant (P=0.006) reduction in total Medicaid spending among patents with curriculum-related conditions, an effect about two-thirds of that found among diabetic patients of PCPs attending 20 or more program sessions. Among patients with neither diabetes nor other curriculum-related condition, we estimated that primary care spending was about $6 lower per patient-quarter (17.9%, P=0.044) among patients attributed to EndoECHO PCPs than among comparison group patients.

DISCUSSION

Project ECHO and similar tele-mentoring programs for health care providers have been adopted widely in the United States and other countries to improve care for patients with a variety of complex conditions, including diabetes and other endocrine diseases.20,21,23 Studies to date have shown that the model has promising impacts on outcomes relating to participating providers, such as indicators of provider knowledge and reports of confidence and self-efficacy, but few studies have linked provider participation to improved patient outcomes.22,23 The analysis in this article applies a novel methodology for evaluating the impact of a Project ECHO program seeking to improve care delivered by PCPs to Medicaid-enrolled patients with diabetes in New Jersey.

The NJ EndoECHO program, hosted by the Robert Wood Johnson Medical School, enrolled 53 providers, of which 25 met study inclusion criteria. The program addressed topics related to management of diabetes and other complex endocrine conditions (Supplemental Digital Content 2, https://links.lww.com/MLR/C401) and spanned 18 months starting January 2017. Program sessions were led by a faculty of endocrinologists, diabetes educators, pharmacists, and social workers in which evidence-based care guidelines and participant case presentations were discussed. PCP participants eligible for the study attended an average of 19 of 64 program sessions.

To evaluate patient outcomes, we first attributed Medicaid patient panels to NJ PCPs based on the plurality of visits in the year before the EndoECHO intervention. We then established a comparison group by matching nonparticipating PCPs to participants based on provider and patient panel characteristics.

Overall, our analysis provides modest evidence that EndoECHO led to better care for Medicaid patients with diabetes. Results showed impressive reductions in inpatient admissions and spending that exhibited evidence of dose-response effects. If replicated, the reduction in inpatient admissions by 3 per 100 patient-quarters would be clinically and operationally important, comparable to the impacts of notably successful policy interventions.39 Our limited sample size means that clinically important benefits went undetected. For example, our nonsignificant point estimate reduction of 3.6 ED visits per 100 patient quarters would certainly be operationally important if one imagines reducing an ED workload by that margin. The absence of findings consistent with program effectiveness among patients without diabetes or other conditions related to the EndoECHO curriculum strengthens confidence that findings among patients with diabetes were the result of their PCPs’ participation in the program.

While potentially subject to unmeasured selection effects, the finding of a dose-response is consistent with the hypothesis that higher PCP participation may have led to greater improvement in patient care. Notably, session attendance in our study was modest, averaging just under 1-in-3 of the sessions offered. Concentrating the most important program content in fewer sessions would reduce demands of participation and potentially improve outcomes. In fact, RWJMS revised its curriculum and reduced the number of sessions in subsequent iterations of their EndoECHO program. Program sponsors might also consider experimenting incentives for higher attendance, for example, by linking continuing education credits to participation in a minimum number of core sessions.

While findings overall support a conclusion that EndoECHO contributed to improved care for diabetic patients of participating PCPs, many results lacked statistical significance. Signs on DD estimates related to utilization and spending for ED, primary care, and endocrinologist services, and total Medicaid spending among patients with diabetes were consistent with program effectiveness but lacked statistical significance. This was also true for measures of hospitalization for short-term complications of diabetes and total diabetes-related admissions but not HbA1c testing and eye examination rates. We observed only one change in prescribing patterns in the DD analysis, reduced use of insulin in the EndoECHO group relative to the comparison group. This change is unlikely to have mediated the observed reduction in hospitalization rates in the EndoECHO group relative to comparison patients.

Findings from this study must be interpreted in light of its limitations. First, this is a single site study of a newly developed program, which may limit its generalizability to other practice contexts or more experienced ECHO hubs. Second, the sample of participating PCPs was small (N=25) and, among these, only 8 attended more than one-third (>20) of the 64 program sessions (although nearly half of study patients were attributed to these 8 PCPs). Third, the sample of patients was limited to those with at least 300 days of Medicaid enrollment during each study years. While near-continuous enrollment is necessary for calculation of reliable utilization and spending outcomes, outcomes for patients with more intermittent enrollment may differ. Finally, in spite of using models that control for differences between treatment and comparison groups, we cannot rule out unmeasured selection bias. However, in DD analysis, unmeasured selection factors that are independent of the timing of the intervention should not influence results.

The analyses presented provides modest evidence of EndoECHO effectiveness, but it also demonstrates the feasibility and value of applying quasi-experimental DD analysis in claims data for the evaluation of provider-level interventions like Project ECHO. Future studies on larger groups of providers with more consistent program participation have potential to provide needed evidence about the effectiveness of such programs.

ACKNOWLEDGMENTS

The authors are also grateful to Kathy Dodsworth-Rugani, PhD, Eric Jahn, MD, and the faculty and staff of the Robert Wood Johnson Medical School Project ECHO program for their support and contribution to our understanding of the Endo ECHO project. The views expressed in this manuscript are solely those of the authors.

REFERENCES

1. Narayan KM, Boyle JP, Geiss LS, et al. Impact of recent increase in incidence on future diabetes burden: US, 2005-2050. Diabetes Care. 2006;29:2114–2116.
2. CDC. National Diabetes Statistics Report, 2020. Atlanta, GA: Centers for Disease Control and Prevention; 2020.
3. Boyle JP, Thompson TJ, Gregg EW, et al. Projection of the year 2050 burden of diabetes in the US adult population: dynamic modeling of incidence, mortality, and prediabetes prevalence. Popul Health Metr. 2010;8:29–29.
4. Kaiser Family Foundation. The role of Medicaid for people with diabetes. Key facts, 2012. Available at: https://www.kff.org/wp-content/uploads/2013/01/8383_d.pdf. Accessed March 29, 2021.
5. American Diabetes Association. Economic costs of diabetes in the US in 2017. Diabetes Care. 2018;41:917–928.
6. AHRQ. National Healthcare Quality and Disparities Report 2018. Rockville, MD: Agency for Healthcare Research and Quality; 2019.
7. Heidemann DL, Joseph NA, Kuchipudi A, et al. Racial and economic disparities in diabetes in a large primary care patient population. Ethn Dis. 2016;26:85–90.
8. Meng Y-Y, Diamant A, Jones J, et al. Racial and ethnic disparities in diabetes care and impact of vendor-based disease management programs. Diabetes Care. 2016;39:743–749.
9. AHRQ. National healthcare quality and disparities report: measures of effective treatment of diabetes. 2016. Available at: https://www.ahrq.gov/research/findings/nhqrdr/chartbooks/effectivetreatment/diabetes.html. Accessed December 1, 2020.
10. McCall DT, Sauaia A, Hamman RF, et al. Are low-income elderly patients at risk for poor diabetes care? Diabetes Care. 2004;27:1060–1065.
11. Andrus MR, Kelley KW, Murphey LM, et al. A comparison of diabetes care in rural and urban medical clinics in Alabama. J Community Health. 2004;29:29–44.
12. Rushforth B, McCrorie C, Glidewell L, et al. Barriers to effective management of type 2 diabetes in primary care: qualitative systematic review. Br J Gen Pract. 2016;66:e114–e127.
13. Shrivastav M, Gibson W, Shrivastav R, et al. Type 2 diabetes management in primary care: the role of retrospective, professional continuous glucose monitoring. Diabetes Spectrum. 2018;31:279–287.
14. Davidson JA. The increasing role of primary care physicians in caring for patients with type 2 diabetes mellitus. Mayo Clin Proc. 2010;85:S3–S4.
15. Cheung A, Stukel TA, Alter DA, et al. Primary care physician volume and quality of diabetes care: a population-based cohort study. Ann Intern Med. 2017;166:240–247.
16. Santos Cavaiola T, Kiriakov Y, Reid T. Primary care management of patients with type 2 diabetes: overcoming inertia and advancing therapy with the use of injectables. Clin Ther. 2019;41:352–367.
17. Vigersky RA, Fish L, Hogan P, et al. The clinical endocrinology workforce: current status and future projections of supply and demand. J Clin Endocrinol Metab. 2014;99:3112–3121.
18. Perreault L, Vincent L, Neumiller JJ, et al. Initiation and titration of basal insulin in primary care: barriers and practical solutions. J Am Board Fam Med. 2019;32:431–447.
19. American Diabetes Association. Promoting health and reducing disparities in populations. Standards of medical care in diabetes. Diabetes Care. 2017;40:S6–S10.
20. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. 2011;364:2199–2207.
21. Bouchonville MF, Paul MM, Billings J, et al. Taking telemedicine to the next level in diabetes population management: a review of the Endo ECHO model. Curr Diabetes Rep. 2016;16:96.
22. McBain RK, Sousa JL, Rose AJ, et al. Impact of Project ECHO models of medical tele-education: a systematic review. J Gen Intern Med. 2019;34:2842–2857.
23. Bouchonville MF, Hager BW, Kirk JB, et al. Endo Echo improves primary care provider and community health worker self-efficacy in complex diabetes management in medically underserved communities. Endocr Pract. 2018;24:40–46.
24. Faherty LJ, Rose AJ, Chappel A, et al. Assessing and expanding the evidence base for project ECHO and ECHO-like models: findings of a technical expert panel. J Gen Intern Med. 2020;35:899–902.
25. Watts SA, Roush L, Julius M, et al. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a Specialty Care Access Network-Extension for Community Healthcare Outcomes model at primary care clinics. J Telemed Telecare. 2016;22:221–224.
26. Blecker S, Lemieux E, Paul MM, et al. Impact of a primary care provider tele-mentoring and community health worker intervention on utilization in medicaid patients with diabetes. Endocr Pract. 2020;26:1070–1076.
27. Fischer SH, Rose AJ, McBain RK, et al. Evaluation of Technology-enabled Collaborative Learning and Capacity Building Models: Materials for a Report to Congress. Santa Monica, CA: RAND Corporation; 2019.
28. CMS. Medicare Shared Savings Program: Shared Savings and Losses and Assignment Methodology: Version 5. Centers for Medicare and Medicaid Services. 2017. Available at: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/Shared-Savings-Losses-Assignment-Spec-V5.pdf. Accessed March 29, 2021.
29. CMS. Two-step attribution for measures included in the value modifier. Fact Sheet: Centers for Medicare and Medicaid Services. 2015. Available at: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/PhysicianFeedbackProgram/Downloads/Attribution-Fact-Sheet.pdf. Accessed March 29, 2021.
30. NCQA. Healthcare Effectiveness Data and Information Set Technical Specifications for Health Plans. Washington, DC: National Committee for Quality Assurance; 2018.
31. AHRQ. Prevention quality indicators technical specifications updates—version 6.0 (ICD-10). 2016. Available at: https://www.qualityindicators.ahrq.gov/Archive/PQI_TechSpec_ICD10_v60.aspx. Accessed March 29, 2021.
32. NCQA. Healthcare effectiveness data and information set measures and technical resources, comprehensive diabetes care. Available at: https://www.ncqa.org/hedis/measures/comprehensive-diabetes-care/. Accessed March 29, 2021.
33. Flory JH, Mushlin AI. Effect of cost and formulation on persistence and adherence to initial metformin therapy for type 2 diabetes. Diabetes Care. 2020;43:e66–e67.
34. AHRQ. Clinical Classifications Software Refined (CCSR) for ICD-10-CM Diagnoses. 2020. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccsr/dxccsr.jsp. Accessed March 29, 2021.
35. Ryan AM, Burgess JF Jr, Dimick JB. Why we should not be indifferent to specification choices for difference-in-differences. Health Serv Res. 2015;50:1211–1235.
36. Ryan AM, Kontopantelis E, Linden A, et al. Now trending: coping with non-parallel trends in difference-in-differences analysis. Stat Methods Med Res. 2019;28:3697–3711.
37. Willage B. Unintended consequences of health insurance: Affordable Care Act’s free contraception mandate and risky sex. Health Econ. 2020;29:30–45.
38. American Diabetes Association. 9. Pharmacologic approaches to glycemic treatment: standards of medical care in diabetes-2021. Diabetes Care. 2021;44:S111–S124.
39. Zuckerman RB, Sheingold SH, Orav EJ, et al. Readmissions, observation, and the hospital readmissions reduction program. N Engl J Med. 2016;374:1543–1551.
Keywords:

diabetes; Medicaid; Project ECHO

Supplemental Digital Content

Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc.