Health Care Utilization and Medical Cost Outcomes from a Digital Diabetes Prevention Program in a Medicare Advantage Population

Abstract

This study examined the effects of a digital diabetes prevention program (DPP) on health care costs and utilization among Medicare Advantage participants. Patients (n = 501) received access to a plan-sponsored, digitally-delivered DPP accessible through computer, tablet, or smartphone. Prior research demonstrated a 7.5% reduction in body weight at 12 months. A comparison group who did not participate in the DPP was constructed by matching on demographic, health plan, health status, and health care costs and utilization. The authors assessed effects on cost and utilization outcomes using difference-in-differences regressions, controlling for propensities to participate and engage in the DPP, in the 12 months prior to DPP enrollment and 24 months after. Though post-enrollment data showed trends in decreased drug spending and emergency department use, increased inpatient utilization, and no change in total nondrug costs or outpatient utilization, the findings did not reach statistical significance, potentially because of sample size. The population had low costs and utilization at baseline, which may be responsible for the lack of observed effects in the short time frame. This study demonstrates the challenges of studying the effectiveness of preventive programs in a population with low baseline costs and the importance of using a large enough sample and follow-up period, but remains an important contribution to exploring the effects of digital DPPs in a real-world sample of individuals who were eligible and willing to participate.

Introduction

Diabetes remains a highly prevalent and pervasive condition in the United States, ranking as the 7^th leading cause of death,¹ with more than 30 million Americans diagnosed with the disease and 84 million at risk for the disease.² The economic burden of diabetes is well known; diabetes is associated with higher lifetime cost for health care expenditures³ and an estimated national burden of $245 billion dollars in combined direct medical costs and indirect costs from productivity loss.⁴ Similarly, people with prediabetes (the clinical precursor to type 2 diabetes [T2D]) have greater medical expenditures when they progress to diabetes.⁵

The empirically validated Diabetes Prevention Program (DPP) has been proven effective for significantly reducing the risk of diabetes among people with prediabetes through improved lifestyle management and modest weight loss of 5% to 7% of initial body weight.⁶ DPP has shown a 34% reduction in diabetes incidence at 10-year follow-up, amounting to an average delay of incidence by 4 years.⁷ Additionally, lower rates of microvascular complications were observed at 15-year follow-up, suggesting benefits to overall cardiovascular health over the long term.⁸ The reach of DPP and access to it has been expanded across the nation through establishment of the National Diabetes Prevention Program, which oversees program quality standards, training, efforts to improve population-level screening and referral, and efforts to expand payment coverage options to deliver DPP.⁹ DPP has been successfully disseminated and translated into clinical practice in a variety of settings, with successful replications in terms of improving health behaviors and reducing population risk for T2D and cardiovascular disease.^10,11

From an economic perspective, the value of DPP has been clearly demonstrated. Comprehensive reviews find DPP offerings and similar programs to be cost-effective and in some cases cost saving, with opportunities for minimizing costs while maintaining outcomes and expanding choice of setting, delivery method, and staffing. Economic simulations of DPP on a population level estimate that the program would produce a gross economic benefit of $17,800 per DPP participant, accumulated over 10 years, and reduce medical care costs per person in the range of $10,400–$11,200.¹² An evaluation of a DPP offering to traditional Medicare participants reported an average quarterly savings of $278 per person and a reduction in inpatient admissions as well as emergency department (ED) visits during the first 3 years.¹³ Similar analyses among a commercially insured population estimated a 3-year net medical cost savings in the range of $3131–$35,037 in a sample of 1000 employees.⁵ Overall, the combined body of work to validate the clinical and health economic benefit of DPP has culminated in the recent approval of coverage for DPP for Medicare beneficiaries.¹⁴

Although the traditional format for delivery of DPP has been through in-person, group-based meetings at fixed geographic locations, advancements in health technology now enable DPP to be delivered in digital, online format without need for physical locations.^15,16 Digital versions of DPP retain the core components of the program while offering expanded reach and access by removing geographic and transportation barriers. The field of digital DPP is young, with published efficacy data spanning fewer than 15 years.^16,17 However, during that time, peer-reviewed research has validated the clinical effectiveness of translating DPP to a digital format.^15,18,19 This has included our research team's previous work, which found a 7.5% reduction in body weight 12 months after enrollment, as well as improved glucose control.²⁰

Economic simulation analyses also estimate significant long-term cost savings that can be realized from digital DPPs,²¹ including those offered to Medicare enrollees.²² However, to date the available data have limited the evidence to small-scale studies and simulation estimations, not actual medical care costs.²³ Therefore, the purpose of this study was to examine short-term cost and utilization outcomes for a small group of Medicare Advantage (MA) beneficiaries who participated in a 12-month digital DPP, compared to matched controls who did not participate in a DPP. This paper makes an important contribution to understanding the early, real-world effects of a digital DPP for eligible individuals who demonstrated willingness to participate in such a program.

Methods

This study used claims data from a national health care organization to examine health care cost and utilization outcomes for digital DPP participants in the 24 months following program enrollment. The research team compared those participants to similar health plan participants who were eligible for the digital DPP but did not participate. Institutional review board approval was obtained from Schulman IRB, with a waiver of consent approved for use of retrospective data that were not originally collected for research purposes.

Digital DPP participants

Potential participants were enrolled in MA plans offered by Humana Inc. To be clinically eligible for the DPP, participants had to have evidence of prediabetes and/or metabolic syndrome according to administrative medical claims, laboratory data, and/or pharmacy data. Metabolic syndrome was identified by the presence of a diagnosis code for metabolic syndrome; the presence of a diagnosis code for prediabetes, hypertension, dyslipidemia, or obesity; evidence of prediabetes using laboratory blood test results; or a high score on the Centers for Disease Control and Prevention (CDC) Diabetes Risk Screening Test.

From this initial pool of potential candidates (n = 15,987), individuals were randomized to either be offered the program (intent-to-treat group) or not (control group). Within the intent-to-treat group, individuals had the choice to accept or decline the offer to enroll in DPP; the accept/decline decision was a nonrandom self-selection decision. A 3-week outreach campaign combining direct mail, automated telephone calls, and email (when addresses were available) was directed to 2 cohorts of plan participants to promote enrollment: 5014 in the first wave and 4484 in the second. Interested people were invited to visit a customized website to confirm their eligibility and enroll in the program. A total of 9498 individuals were contacted, with a response rate of 8.4%. From the 796 applications, 101 were ineligible because of either medical contraindications or lack of insurance coverage, and another 190 started but did not finish the enrollment process. A total of 501 people enrolled and participated in DPP. Sample selection is summarized in Figure 1, and sample characteristics are presented in Table 1.

FIG. 1.

Sample Selection.

Table 1.

Sample Characteristics

	Program participants	Matched controls
N	464	464
Age, mean (SD)	68.9 (2.6)	69.1 (2.6)
Female, n (%)	300 (65%)	269 (58%)
Race/ethnicity, n (%)
White	395 (85%)	407 (88%)
Black	36 (8%)	53 (11%)
Hispanic	3 (<1%)	1 (<1%)
Other	30 (6%)	3 (<1%)
CCI score, mean (SD)	2.84 (1.00)	2.85 (0.99)
Utilization and costs, in 12 months prior to program enrollment
ED visits per 1000	105	96
IP admissions per 1000	55	58
Outpatient visits, PMPM	0.49	0.46
Medical (nondrug) spending, PMPM	$299.37	$278.01
Drug spending, PMPM	$88.92	$89.55

CCI, Charlson comorbidity index; ED, emergency department; IP, inpatient; PMPM, per member per month; SD, standard deviation.

Participants were allowed to enroll at any time during the 2015 calendar year. Each participant's calendar start date in the program was used as the study index date. The preceding 12 months before index date comprised the baseline year. The program spanned 12 months, comprising the treatment year. Following the treatment year, the research team observed participants for an average of 10.6 months, with 395 participants observed for a full 12 months after program completion.

Digital DPP

The program was provided by a digital health care company (Omada Health, Inc., San Francisco, CA). The program is recognized by the CDC Diabetes Prevention Recognition Program⁷ and consists of online peer group support, support and guidance from a trained lifestyle coach, digital tracking tools, and a weekly behavior change curriculum. Participants were placed into geographically-based small groups and assigned to a health coach. Group members were connected to each other through a private online forum where they could post comments and questions, engage in health coach–moderated discussions, and provide social support to one another. Using internet-enabled devices, program participants asynchronously completed new curriculum lessons each week focused on a variety of behavior change topics. The health coaches were employed by the company, had a minimum of a bachelor's degree, and underwent specialized CDC lifestyle coach training. Coaches monitored lesson completion, weight, eating, and activity tracking; group discussion posts; and reached out to support engagement in these areas. In addition to this coach-initiated contact, health coaches responded to participant requests for individualized guidance. Coaches also facilitated group discussion about the weekly curriculum lessons and responded to questions on the group board.

Throughout the program, participants tracked weight loss using a wireless scale, tracked physical activity using a pedometer or personal activity tracker, and logged their daily food intake with either the mobile app or online platform. These tools are displayed immediately on each participant's dashboard and were used to help participants self-monitor their habits and weight loss progress. The program was inclusive of an initial 16-week intensive curriculum focusing on weight loss and a subsequent 36-week curriculum focusing on weight maintenance, for a total of 12 months of novel content. Participants engaged with health coaching, small group discussion, and tracking of body weight/food/physical activity throughout the 12 months. A summary of participant engagement in each of the digital DPP components is shown in Table 2.

Table 2.

Summary of Digital Diabetes Prevention Program Engagement

	Minimum–maximum score per week	Mean score per week (SD)
Logins to website	0–7	4.72 (2.24)
Weigh-ins on wireless scale	0–7	5.58 (1.79)
Meal tracking	0–7	3.24 (2.78)
Exercise tracking	0–7	3.58 (2.61)
Lesson completion	0–1	0.90 (0.25)
Health coach interaction	0–1	0.38 (0.26)
Group discussions	0–1	0.88 (0.24)
Composite score	0–31	19.27 (8.21)

Engagement measures for the sample of 464 participants included in the analytic sample, as described in the Methods section.

SD, standard deviation.

Matched comparison cohort

A matched comparison group of plan participants who did not participate in the digital DPP was constructed for comparison purposes. An initial pool of 6490 people was gathered from health plan participants who were similarly eligible but were randomized to not receive an invitation to participate in the program. Each DPP participant was matched to one of these control individuals according to deciles of propensity to participate in DPP (which will be described), deciles of propensity to engage in DPP (engagement measures described in Table 2), and baseline characteristics of the individuals, including use of email, age group, sex, race, diabetes, prediabetes, obesity, inpatient costs and utilization indexes, ED costs and utilization indexes, and health plan. At a match ratio of 1:1495, the participants were matched to a control. Case/control dyads were removed if health plan membership was missing at index month. In situations with more than 1 match for a DPP participant, the research team randomly selected 1 control for inclusion in the analytic sample for this study. This resulted in a reduction to 464 participants and 464 controls in the analytic sample. Sample selection is summarized in Figure 1.

The 2 propensity scores represent probability of participating in DPP and probability to engage in DPP if enrolled. Both scores were calculated using logistic regressions on the following variables: sex, age, race, geographic region, health plan type, plan characteristics, Charlson comorbidity index (CCI), the individual health conditions that comprise the CCI, having an email address, internet use, baseline inpatient utilization, baseline ED visits, baseline ambulatory physician visits, and Amerilink consumer behavior data.

Measures

Administrative claims data were used to identify 2 cost measures (total medical [nondrug] costs and total prescription drug costs) and 3 measures of health care utilization (ambulatory physician visits including primary care and specialty outpatient care combined, inpatient hospital admissions, and ED visits). All measures were assessed on a per-member per-month (PMPM) basis. Claims from 12 months prior to index date until 23 months after index date were included. Program costs for DPP incurred during the post-index months were not included in expenditures. Actual program costs were billed directly to the health plan.

Analyses

In the analytic sample of DPP participants and matched controls, difference-in-differences regressions were performed on patient–month-level data to identify effects of DPP on 5 dependent variables: medical (nondrug) costs, prescription drug costs, number of outpatient office visits, a binary indicator for any inpatient admission, and a binary indicator for any ED visit. These regressions exploited the variation that occurred across time (pre vs post DPP) and across individuals (participants vs matched controls), to identify the effects of the program. Although this model provides estimates of DPP causal effects, it requires that the parallel trends assumption be made. That is, it must be assumed that in absence of DPP, any difference between the treatment and control group would remain constant. The aforementioned matching procedure gives strength to this assumption.

The regressions were estimated using generalized linear models: cost regressions used gamma distributions with log link function, office visit regressions used Poisson distributions with log link functions, and inpatient admission and ED visit regressions used logistic regressions (ie, binomial distribution with logit link functions). For cost outcomes, the research team included month fixed effects and their interactions to allow calculation of cumulative, month-over-month effects. The regressions with utilization outcomes used indicator variables for program time period (months 0 [index] to 11) and post program (months 12–23). To control for differences across individuals that influence their selection into the digital DPP, and their engagement in DPP, the team included both propensity scores in the regressions (propensity to participate, and propensity to engage, as already discussed). Analyses were performed using SAS (SAS Institute, Inc., Cary, NC). Confidence intervals were generated using bootstrapping with 200 replications.

Results

Table 1 shows demographic characteristics of the digital DPP participants and their matched controls. The groups had similar age, sex, race/ethnicity, comorbidity indexes, and health care costs and utilization, which shows that the matching was successful and that the matched controls were good comparators for the participants. The effects of DPP on spending are depicted in Figures 2 and 3, which show the cumulative difference in spending between participants and controls over the 24 months after DPP enrollment, after adjustment for propensity to participate in DPP and propensity to engage with the program. Table 3 presents the similarly adjusted relative odds of having an ED visit or inpatient admission and the adjusted relative risk of outpatient visits during the 24 months post enrollment.

FIG. 2.

Cumulative medical (nondrug) spending difference: DPP participants minus matched controls.

FIG. 3.

Cumulative prescription drug spending difference: DPP participants minus matched controls.

Table 3.

Effects of Diabetes Prevention Program on Health Care Utilization

IP admission (binary, PMPM)	Odds ratio	95% CI	P
During DPP	0.97	0.45–2.10	0.934
12 months following DPP	1.85	0.88–3.87	0.104

ED visit (binary, PMPM)	Odds ratio	95% CI	P
During DPP	0.71	0.40–1.25	0.236
12 months following DPP	0.68	0.39–1.22	0.195

Outpatient visits (PMPM)	Risk ratio	95% CI	P
During DPP	0.99	0.90–1.08	0.757
12 months following DPP	1.04	0.93–1.15	0.486

Results from difference-in-differences regressions showing effects of digital DPP on the outcome noted. Analyses of IP admissions and ED visits used logistic regression; analysis of outpatient visits used Poisson regressions. All regressions include controls for propensity to enroll in DPP and for propensity to engage in DPP.

N = 928.

CI, confidence interval; DPP, Diabetes Prevention Program; ED, emergency department; IP, inpatient; PMPM, per member per month.

Results show that while cumulative medical (nondrug) costs were slightly lower for DPP participants in some of the observation months, cumulative costs were at similar levels at the end of the 24-month observation period; the differences had a large band of variability and were not statistically significant at any observation. The cumulative difference in prescription drug cost also was insignificant at 24 months, but results show a pattern of increasing difference, favoring the DPP group, starting at about 10 months. No statistically significant differences in utilization were observed, but the direction of findings favored the DPP group and the point estimate for ED visits was consistent with a 30% reduction.

Discussion

This study examined short-term effects of a digital DPP on health care costs and utilization for MA participants. Though the direction of some findings suggested potential cost savings and reduced health care utilization, results do not permit definitive conclusions because of the limitations of the study. Differences in nondrug spending were not statistically significant, but the pattern of observations suggested a trend toward declining drug costs for DPP participants relative to controls. Results also were consistent with a meaningful reduction in ED visits, but the odds ratios were not statistically significant. There was some evidence of increased inpatient admissions in the post-DPP period. Differences in outpatient visits were not detected.

The research team's previous work showed that health outcomes improved for these digital DPP participants during the 12 months of the program (reduced body weight, improved glucose control, decreased total cholesterol, and improved self-reported well-being, depression, and self-care).²⁰ Although short-term clinical benefits were evident, near-term health care utilization changes were not realized as quickly in this small cohort.

The goal of this analysis was to provide an assessment of short-term effects of a digital DPP in a real-world setting involving eligible individuals who demonstrated willingness to participate in such a program. Unfortunately, the short follow-up period, sample size, and individual characteristics may have precluded conclusive results. These participants and the control individuals who were selected as comparable matches had uncharacteristically low health care costs and utilization rates. The team's internal analyses revealed that, compared to the DPP participants analyzed in this study, individuals in the study's source population of MA plans incur greater mean medical spending PMPM, experience more inpatient admissions, and visit the ED more frequently. Compared to matched controls, the overall MA plan population also had higher mean medical spending PMPM, and greater inpatient admissions and ER visits. Additionally, DPP participants (who all had metabolic syndrome or prediabetes) had generally better health characteristics than the MA population with metabolic syndrome or prediabetes (who were not offered DPP), including lower rates of hypertension, hyperlipidemia, and prediabetes.

The relatively low costs and utilization by study participants prior to the study period (ie, at baseline) made it more difficult to achieve an observable effect during the relatively short follow-up period (12 months post program completion). This follow-up period may be insufficient to capture the downstream consequences of early detection and mitigation of health risk in a population with relatively good general health. The type of early intervention investigated in this study nevertheless may have a long-term impact on health care utilization detectable at larger population levels, but this study was not powered to capture near-term smaller changes in health care utilization. Longer term assessments, larger samples, and more broadly targeted programs likely would yield different results.

Previous results showed a positive impact on health outcomes for this program, but the research team was not able to conclude what the impact was on cost in this analysis. In future studies, the cost impact of the DPP program needs to be evaluated in the context of the population being studied and prior health care costs at the start of the program. For a population with relatively tight cost control, the cost savings may be more modest.

The results contribute to the body of studies evaluating the effects of DPPs, most of which have focused on traditional nondigital DPPs. This prior research is summarized in the introduction and found cost savings and utilization reductions. There are at least 2 reasons why the team might not expect this study to match those results. The first is the relatively short follow-up period of the study (12 months post DPP). Indeed, the Centers for Medicare & Medicaid Services Office of the Actuary certification of the DPP noted the difficulty of observing statistically significant effects on short-term costs.¹⁴ Most previous research assessed cost and utilization benefits in the 3 to 10 years after enrollment, therefore allowing more time for effects to manifest. For example, an assessment of an in-person DPP administered via YMCAs found savings of $70 per person per month, and reductions in inpatient admissions and ED visits in the 3 years following enrollment.¹³

The second reason is that there are differences between digital and traditional DPPs in scope, length, and available services. In-person programs operate like traditional classes – participants use classroom sessions to absorb content and engage in discussion in a limited time period (typically 60–90 minutes per week). Digital programs are fundamentally different; they allow brief but frequent interactions throughout the day and week. Rather than an online class with an instructor, the digital DPP is an online service with many tools available to the user at all times. This leads to substantial heterogeneity both across and within digital DPP programs, wherein participants enjoy very different experiences with the program. The result is a patient-centered model with differences in the levels and types of services used by different participants. These fundamental differences between the digital and in-person DPP may explain why one should not expect studies on the 2 types of DPPs to yield the same results.

Because the effects of digital DPP (specifically for Medicare) remain less understood compared to in-person interventions, future program evaluations should be designed to identify which individuals are most likely to engage with/benefit from each delivery format, and economic evaluations should be designed prospectively for sufficient sample sizes to allow for robust analysis.

Limitations

This study has limitations that should be considered when interpreting the findings. First, the study used an observational design. Participants' self-selection into the program may contribute to the positive health gains seen in the population. Although the research team controlled for a number of observed differences between participants and controls, those who chose to enroll may have had unobservable differences in health behaviors that could drive differences in the outcomes of interest. However, this study is an example of how programs would operate under real-world conditions in which qualified eligible members would be invited to participate without restriction. Additionally, the matching (and control variables for propensity to participate and engage in DPP) accounted for a variety of individual, economic, and environmental variables to ensure the groups were relatively comparable at baseline.

A second limitation is the size of the sample analyzed (464 participants and 464 matched controls). The impetus for the economic analyses came after discovery of substantial clinical benefit of DPP on the treated population.²⁰ Economic evaluations such as the present study often follow discovery of clinically meaningful findings, yet tend to be underpowered.²⁴ It is true in this case that the enrolled/treated sample was not recruited with statistical power considerations appropriate for economic analyses. However, it has been argued that the mean net benefit of a new health program should be the decision-making priority, and that the statistical significance of an inferential test can lack relevance.²⁵

Another limitation inherent in a small study sample is that the research team was unable to look at heterogeneous effects of DPP. Future research should use larger samples to identify how effects vary across demographically-defined subsamples of participants, and across different levels and types of engagement in DPP. A final limitation, as already mentioned, was that participants were observed for only 12 months after program completion. This is a short period of time within which to expect significant variation in costs and utilization for a preventive services program.

Conclusion

This study provides some positive short-term observations in a sample that was not powered to detect the economic effects of a DPP. Results reflect a set of real-world participants who chose to enroll in the program and had relatively low health care costs and utilization at baseline. As DPPs continue to evolve, understanding these real-world effects, even in the short term, is critical for patients, medical providers, health plan decision makers, and policy makers. A greater effect would be expected with a more broadly targeted program, and over a longer term follow-up. Despite the lack of near-term statistical significance on health care costs and utilization found in this study, digital DPPs have shown promise for improving health outcomes, and merit continued use in efforts to contain the growing prevalence of diabetes in the United States. Although any possible benefits must be considered alongside the administration costs of a DPP (including marketing, enrollment, software development, equipment costs), there is potential to spread many of these fixed costs over a large number of enrollees if the program is offered at a large scale. The research team's previous work has shown digital DPP to result in health effects that are highly associated with the onset of diabetes, the cost of which has been valued at $245 billion per year.⁴ This paper assessed the costs and health care utilization in the 24 months following DPP enrollment. It is possible that many of the economic effects (via both changes to health care utilization and improved labor force productivity) accrue over a longer period of time. Future research should examine long-term effects of both digital and traditional DPPs and recruit sample sizes large enough to permit conclusions about the value provided to participants, payers, and society as a whole.

Footnotes

Author Disclosure Statement

Dr. Barthold is a consultant to Omada Health and received a consulting fee. Dr. Castro Sweet is employed by Omada Health and receives salary and stock options. Dr. Prewitt, Mr. Gumpina, Mr. Chiguluri, Mr. Matanich, Mr. Pieratt, and Ms. Rogstad are employed by Humana and receive salary and annual incentives. Mr. Cordier was employed by Humana at the time of manuscript preparation and received salary and annual incentives.

Funding Information

The study was supported by resources from Humana Inc. and Omada Health, Inc.

References

National Center for Health Statistics. Health, United States, 2016: With Chartbook on Long-term Trends in Health. Hyattsville, MD: US Government Printing Office, 2017.

Centers for Disease Control and Prevention. National Diabetes Statistics Report, 2017. Atlanta, GA: Centers for Disease Control and Prevention, US Department of Health and Human Services, 2017.

Zhuo

, Zhang

, Barker

, Albright

, Thompson

, Gregg

. The lifetime cost of diabetes and its implications for diabetes prevention. Diabetes Care, 2014; 37:2557–2564.

American Diabetes Association. Economic costs of diabetes in the U.S. in 2012. Diabetes Care, 2013; 36:1033–1046.

Khan

, Tsipas

, Wozniak

. Medical care expenditures for individuals with prediabetes: the potential cost savings in reducing the risk of developing diabetes. Popul Health Manag, 2017; 20:389–396.

Knowler

, Barrett-Connor

, Fowler

, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med, 2002; 346:393–403.

Knowler

, Fowler

, Hamman

, et al. 10-year follow-up of diabetes incidence and weight loss in the diabetes prevention program outcomes study. Lancet, 2009; 374:1677–1686.

Diabetes Prevention Program Research Group. Long-term effects of lifestyle intervention or metformin on diabetes development and microvascular complications: the DPP outcomes study. Lancet Diabetes Endocrinol, 2015; 3:866–875.

Albright

, Gregg

. Preventing type 2 diabetes in communities across the U.S.: the National Diabetes Prevention Program. Am J Prev Med, 2013; 44(4 suppl 4):S346–S351.

10.

Dunkley

, Bodicoat

, Greaves

, et al. Diabetes prevention in the real world: effectiveness of pragmatic lifestyle interventions for the prevention of type 2 diabetes and of the impact of adherence to guideline recommendations: a systematic review and meta-analysis. Diabetes Care, 2014; 37:922–933.

11.

Ali

, Echouffo-Tcheugui

, Williamson

. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program?. Health Aff, 2012; 31:67–75.

12.

Dall

, Storm

, Semilla

, Wintfeld

, O'Grady

, Narayan

. Value of lifestyle intervention to prevent diabetes and sequelae. Am J Prev Med, 2015; 48:271–280.

13.

Alva

, Hoerger

, Jeyaraman

, Amico

, Rojas-Smith

. Impact of the YMCA of the USA diabetes prevention program on Medicare spending and utilization. Health Aff, 2017; 36:417–424.

14.

Office of the Actuary. Certification of Medicare Diabetes Prevention Program. Baltimore, MD: Center for Medicare and Medicaid Services, U.S. Department of Health and Human Services, 2016.

15.

Sepah

, Jiang

, Peters

. Translating the Diabetes Prevention Program into an online social network: validation against CDC standards. Diabetes Educ, 2014; 40:435–443.

16.

McTigue

, Conroy

, Hess

, et al. Using the internet to translate an evidence-based lifestyle intervention into practice. Telemed J E Health, 2009; 15:851–858.

17.

Tate

, Jackvony

, Wing

. Effects of internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial. JAMA, 2003; 289:1833–1836.

18.

Block

, Azar

, Romanelli

, et al. Diabetes prevention and weight loss with a fully automated behavioral intervention by email, web, and mobile phone: a randomized controlled trial among persons with prediabetes. J Med Internet Res, 2015; 17:e240.

19.

Kirley

, Sachdev

. Digital health-supported lifestyle change programs to prevent type 2 diabetes. Diabetes Spectr, 2018; 31:303–309.

20.

Castro Sweet

, Chiguluri

, Gumpina

, et al. Outcomes of a digital health program with human coaching for diabetes risk reduction in a Medicare population. J Aging Health, 2018; 30:692–710.

21.

, Chen

, Dall

, Iacobucci

, Perreault

. Return on investment for digital behavioral counseling in patients with prediabetes and cardiovascular disease. Prev Chronic Dis, 2016; 13:E13.

22.

Chen

, Su

, Becker

, et al. Clinical and economic impact of a digital, remotely-delivered intensive behavioral counseling program on Medicare beneficiaries at risk for diabetes and cardiovascular disease. PLoS One, 2016; 11:e0163627.

23.

Tice

, Chapman

, Shore

, et al. Diabetes Prevention Programs: Effectiveness and Value. Final Evidence Report and Meeting Summary. Boston, MA: Institute for Clinical and Economic Review, 2016.

24.

Briggs

. Economic evaluation and clinical trials: size matters. BMJ, 2000; 321:1362–1363.

25.

Claxton

. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ, 1999; 18:341–364.