Rethinking the Endpoints of mHealth Intervention Research in Diabetes Care

Current State of mHealth in Diabetes

Given the enormous worldwide burden of chronic disease, mHealth implementation studies have focused on diabetes and other chronic conditions, and more specifically on patient self-management strategies for these conditions. Diabetes naturally lends itself toward technology usage, with patients routinely using glucometers, continuous glucose monitoring (CGM) devices, and insulin pumps. These devices are becoming increasingly sophisticated and interconnected. The mHealth opportunities in diabetes are extensive, and multiple players are involved in the development and advancement of connectivity in diabetes care. With more than 1000 diabetes-related mobile apps currently available for download, the high level of both supply and demand is obvious. However, much is still unknown about the best ways to incorporate and rigorously evaluate this technology to achieve maximal benefit in diabetes care. ⁹

One of the major advantages of app-based diabetes management, and by extension app-based research, is that data can be accessed remotely. Remote access to individual data introduces the opportunity for patients living in rural areas and those with limited resources to connect with their clinicians to receive excellent individualized care, akin to those able to directly see their provider. ¹⁰ Could we foresee a world in which face-to-face clinic visits become near obsolete for certain chronic conditions in favor of remote access to care?

Current Strategies in mHealth Research in Diabetes

The current gold standard for intervention research studies is the randomized controlled trial (RCT). ¹¹ RCTs are considered to be the ideal research method to evaluate new pharmaceutical agents, since they reduce unfounded causality and bias. Taking the lead from this idealized evidence-based research milieu, most diabetes technology intervention studies have similarly employed the RCT as the “highest-quality evidence available.” ¹⁰ The many advantages of RCTs are well-established. However, as a result of some of these well-recognized strengths, we propose the suggestion that RCTs may not be the optimal method to assess mHealth interventions.

The end points of an RCT are essentially fixed at its onset, and revisions to a particular protocol are avoided as much as possible during the intervention period of a trial, even as new evidence and advances in technology develop. This begs the question: how does an RCT conclusively evaluate mHealth technology when the technology may well be obsolete by the conclusion of the trial? By way of example, early insulin pump intervention studies were performed in an era preceding the availability of rapid acting analog insulins. Modest benefits in favor of insulin pump therapy were reported, but how is this relevant today with all of the changes to the standard of diabetes care since then?

A large sample size is desirable to ensure study results are appropriately powered and generalizable. Most RCTs of sufficient size are multiyear, multisite endeavors of great cost. Given the incredible pace of technological advancement, the pump and sensor devices currently under investigation in RCTs will most likely be publicly reported and available for widespread use right at the time when the technology is no longer on the “cutting edge” and the next generation of technology is in development.

The careful evaluation of mHealth applications is even more critical given the proliferation and multitude of available options. If mHealth is to truly be integrated and accepted into the health care system, it must be relevant, safe and unequivocally medically beneficial to patients and clinicians. Answering these questions may not be best achieved by an RCT, as RCTs are not conducive to the ever-changing technological landscape. Most RCT participants in recent artificial pancreas diabetes technology studies are also not typical “real-world” patients in terms of sociodemographic or economic variables. ¹²

In their 2013 report on the current and future role of health care apps, the IMS Institute for Healthcare Informatics reported that although observational studies are more easily performed, they are insufficient to show an effect on outcomes, and RCTs are necessary to provide evidence-based recommendations. ¹³ Without RCTs to show health outcome benefits, the IMS Institute suggested it would be unlikely that there will be widespread adoption of apps by either providers or patients. In line with this report, a search of clinicaltrials.gov for “diabetes” and “mHealth” reveals 70 studies, 27 of which are actively recruiting, soon to be recruiting, and available for expanded access. ¹⁴ Of these 27 open studies, 22 are RCTs. The study duration for these studies ranges from three months to five years, with the majority being between six and 12 months.

Prior RCTs of mHealth diabetes care interventions have recruited patients over as little as three months, which is likely fast enough to ensure the relevance of the intervention, though it may limit the sample size and statistical power. In addition, any effect that is observed within a short follow up time could be attributable to the novelty of using a new app, and not the effect of the app itself. RCTs performed over longer time periods eliminate the risk of the novelty period, but at the possible expense of the intervention no longer being relevant or being replaced by its next iteration. In addition, RCTs are often limited in physical reach; patients who are economically or geographically isolated may be excluded, and thus may not have the opportunity to benefit from many of the mHealth interventions. In unpublished data from our urban safety-net institution, where more than 70% of patients receive insurance coverage through Medicare or Medicaid, over 90% of patients with diabetes and obesity reported having smartphones. Although historically this group is underrepresented in RCTs, they may benefit from mHealth interventions in different ways, as mHealth tools may result in increased access to care, when often this access is curtailed by competing sociodemographic complexities.

Proposed Alternative Study Designs

The demand for robust mHealth in diabetes care for both patients and clinicians is readily apparent, but to succeed, researchers must consider shifting from a focus on RCTs to other trial designs that may be more appropriate for mHealth interventions and allow for simultaneous analysis of multiple outcomes. New methodologies, such as Continuous Evaluation of Evolving Behavioral Intervention Technologies (CEEBIT), monitor the efficacy of an intervention over time and through progressive versions, which will allow for longer term study of an app while accounting for the rapid changes that are expected in technology. ¹⁵ Frameworks such as this are particularly desirable for tracking outcomes from glucometer, CGM, and insulin pump technologies, as they are increasingly integrated with mHealth platforms.

For shorter-term outcomes, cohort studies may best suggest associations between interventions and relevant clinical outcomes. Although this study design does not allow for causality attribution, strong associations between an app and improved endpoints may provide temporal evidence that validate routine use of specific mHealth interventions. This is particularly attractive as observational data from patient experience can be more readily analyzed, thus allowing for more rapid incorporation of an app into routine care.

If the focus in research remains on clinical trials, then perhaps pragmatic clinical trials are more appropriate than traditional RCTs. This trial design innately allows researchers to evaluate the effectiveness of interventions in the real-life setting. From our experience working at a safety-net health care institution, pragmatic trials are invaluable, particularly due to the inconsistent access to internet and mobile resources among our resource-limited patient population and allowance for more generalizability to a broad population. From a similar perspective, response-adaptive randomization in clinical trials may be a way to keep traditional RCTs more relevant, as patients would preferentially be randomized to beneficial interventions over the course of a study based on interim analyses.

Rather than waiting years for the results of an RCT proving that an intervention definitively leads to improvement in an outcome of interest, mHealth interventions can be promoted almost immediately by considering alternative study designs. Even if we are not able to clearly attribute direct responsibility for improvement in the outcome of interest to the mHealth intervention, we may conclude that it was, at the minimum, a meaningful contributor to an overall outcome. While it is unlikely that RCTs will soon be widely abandoned as the preferred and most common clinical trial design for mHealth in diabetes, a willingness to consider other methodologies for rapid adoption of mHealth strategies is necessary to assure clinicians and patients, alike, that mHealth interventions can be directly incorporated into routine medical care with positive effect on outcomes.