User Experience of an Innovative Mobile Health Program to Assist in Insulin Dose Adjustment: Outcomes of a Proof-Of-Concept Trial

Abstract

Background:

Many patients with diabetes require insulin therapy to achieve optimal glycemic control. Initiation and titration of insulin often require an insulin dose adjustment (IDA) program, involving frequent exchange of blood glucose levels (BGLs) and insulin prescription advice between the patient and healthcare team. This process is time consuming with logistical barriers.

Objective:

To develop an innovative mobile health (m-Health) mobile-based IDA program (mIDA) and evaluate the user adherence and experience through a proof-of-concept trial.

Methods:

In the program, an m-Health system was designed to be integrated within a clinical IDA service, comprising a Bluetooth-enabled glucose meter, smartphone application, and clinician portal. Insulin-requiring patients with type-2 diabetes mellitus and stable BGL were recruited to use the m-Health system to record and exchange BGL entries, insulin dosages, and clinical messages for 2 weeks. The user experience was evaluated by a Likert scale questionnaire.

Results:

Nine participants, aged 58 ± 14 years (mean ± SD), completed the trial with average daily records of 3.1 BGL entries and 1.2 insulin dosage entries. The participants recognized the potential value of the clinical messages. They felt confident about managing their diabetes and were positive regarding ease of use and family support of the system, but disagreed that there were no technical issues. Finally, they were satisfied with the program and would continue to use it if possible.

Conclusions:

The m-Health system for IDA showed promising levels of adherence, usability, perception of usefulness, and satisfaction. Further research is required to assess the feasibility and cost-effectiveness of using this system in outpatient settings.

Introduction

Diabetes is a severe progressive chronic disease, affecting over 380 million adults worldwide.¹ Suboptimal glycemic control substantially increases the risk of numerous and serious complications, which led to disability, comorbidity, and death.^2,3

To achieve optimal glycemic control, patients with diabetes are required to maintain or bring blood glucose levels (BGLs) within recommended ranges on a daily basis. Insulin therapy is the core clinical strategy to maintain BGL when lifestyle interventions and oral medications gradually become ineffective with the progression of diabetes.⁴ To help the patients use insulin, which they usually self-inject daily, and prevent side effects, including life-threatening hypoglycemia, many diabetes centers offer insulin dose adjustment (IDA) programs. In a traditional IDA program, diabetes nurses in collaboration with endocrinologists ask patients to use a paper-based diary to record BGL, symptoms, and insulin dosages on a daily basis at home. The nurses then communicate with the patients to review the recorded data and accordingly adjust insulin dosages, as well as insulin types, if needed. The communication is usually through telephone and/or face-to-face consultations twice a week for about 6 weeks. The traditional IDA program is typically time consuming, resource intensive, and open to transcription and recording errors. Moreover, many patients on insulin remain unable to achieve glycemic targets.⁵ Large studies reported that less than 40% of insulin-requiring patients with type-2 diabetes mellitus (T2DM) achieved the target glycosylated hemoglobin (HbA1_C) <7.0%,⁶ and over 45% had inadequate glycemic control (HbA1_C >7.6%).⁷ Barriers to effective use of insulin therapy are complex, likely including time constraints, fear of side effects and complications, and lack of clinical support.^8,9 These issues and barriers demonstrate a strong need for new care models to improve the efficiency and effectiveness of the IDA interventions.

In the past decade, mobile health (m-Health)—defined as medical and public health practice supported by mobile devices such as mobile phones and other wireless devices¹⁰—has presented new opportunities to improve diabetes care.^11
–13 Different from telemonitoring stations and personal computer-based systems, m-Health applications are inexpensive, ubiquitous, intuitive, and flexible for various lifestyles.¹⁰ Currently a large number of m-Health applications have been developed for diabetes, but the evidence on the efficacy remains limited,^14,15 especially for m-Health applications with communication and intervention from care providers.¹⁶ With respect to IDA interventions, only pilot trials are available in the literature, focusing on self-monitoring of diabetes,¹⁷ automated messages,^18,19 remote clinical monitoring,²⁰ and/or coaching.^21
–23 In addition, these studies were mainly designed to assist the patients in self-adjusting insulin dose within a certain safe range under a prescribed insulin therapy. Hence, their clinical interventions were limited, especially for patients with complex and/or progressive diabetes conditions requiring new insulin regimen(s) and collaborative care.

To improve a clinical IDA service, we developed an m-Health system to enable an innovative mobile-based IDA program (mIDA). Different from many other m-Health studies, the mIDA program was specifically designed to be integrated within an IDA service to function as an alternative option of the service, with a full range of IDA interventions. The objective of this article is to evaluate the adherence and user experience of the mIDA program through a proof-of-concept trial.

Materials and Methods

According to a recent recommendation for telehealth research,²⁴ we initiated the mIDA study from the first stage of proof of concept, focusing on the design of a new care model and evaluation of user experience and technical issues. In this study, the mIDA program was mainly designed by the clinicians from an IDA service center at the Princess Alexandra Hospital (PAH), Queensland, Australia, according to the IDA service requirements and clinical guidelines. In the software development, the framework of an m-Health system previously validated for cardiac rehabilitation²⁵ was utilized. Therefore, many essential aspects of the mIDA implementation such as ease of use, data integrity, and data security could be assured. We conducted a proof-of-concept trial to evaluate the user experience and technical issues of the program. The trial received ethics approval from Metro South Human Research Ethics Committee, Brisbane, Queensland, Australia (Ref: HREC/14/QPAH/686).

The care model using the mIDA program is shown in Figure 1. In the model, patients are provided with a smartphone application (app) and a wireless (Bluetooth) glucose meter (Accu-Chek^® Aviva Connect meter, Roche Diagnostics GmbH, Mannheim, Germany). The patients use the glucose meter to measure BGL at home. The BGL entries are wirelessly transferred from the meter to the app (Fig. 2A). Using the app, patients review the insulin prescription (Fig. 2B) and BGL status (Fig. 2C) and manually record entries of insulin dosages and comments on self-observed health conditions such as hypoglycemia. All the entries recorded in the app are automatically uploaded to a web-based clinician portal. Diabetes nurses, under the supervision of endocrinologists at the PAH, use a computer with Internet access to the portal and review the uploaded data. The nurses then adjust the insulin prescription (according to the usual department's protocols) and send the new prescription through the portal to the patients' mobile app. They also use the portal to schedule appointments with other members of the multidisciplinary team, such as podiatrists and dietitians, and share patients' data with the specialists as required. On discharge of a patient from the service, a summary of the patient's management is generated from the portal for communication back to the treating clinician and general practitioner (GP).

Fig. 1.

The care model of the m-Health enabled home care program for clinical insulin dose adjustment. Color images available online at: www.liebertpub.com/tmj

Fig. 2.

(A) The user interface of the mobile phone application for the users to wirelessly transfer BGL entries from a glucose meter to the mobile application. (B) The user interface to review insulin prescribed by the diabetes nurses from a tertiary diabetes care center. (C) Smartphone plots to show the status of BGL control. BGL, blood glucose level. Color images available online at: www.liebertpub.com/tmj

In this proof-of-concept trial, insulin-requiring patients with stable BGL (i.e., not requiring insulin dose adjustment) were recruited; and the diabetes nurses did not alter the patients' management plan, including insulin prescriptions, in any way. The participants were asked to manage their diabetes and use their insulin prescriptions as usual. The purpose of doing so was to prevent any unforeseeable risks of inappropriate use of insulin, considering that unknown technologies and usability issues could potentially confuse the patients and even affect decision-making on insulin prescriptions.

Patients were recruited from the PAH Diabetes and Endocrinology Outpatients Department. The inclusion criteria were as follows: 18 years of age or older, T2DM requiring insulin with stable BGL, education level of year 6 (or equivalent), willingness to measure BGL 3–4 times a day, willingness to return to the diabetes clinic within 3 weeks of starting the study, and in possession of a smartphone. Patients were excluded if they had multiple comorbidities, unable to use a mobile phone, living in areas with no coverage of 3G/4G cellular Internet, or not able to read and understand English.

Each participant recruited in the trial was provided with a wireless glucose meter (Accu-Chek Connect). If a participant's smartphone was incompatible with the app and/or the glucose meter, the participant was provided with a compatible smartphone. A diabetes nurse trained each participant face to face. During the training, the nurse used the clinician portal to register the participant and enter insulin types and dosages according to his/her insulin prescription. After the registration, the m-Health system automatically sent to the participant's mobile phone a short message service message with login information and a website address for downloading the app. The nurse then guided the participant to read the message, download the app, and log in to the app. The nurse also helped pair the glucose meter with the participant's mobile phone. Finally, the nurse guided the participants to practice essential management tasks, such as taking a BGL measurement with the new meter and reviewing on the app the prescription entered through the clinician portal at the beginning. The training took on average 45 min.

The participants were asked to use the app for 2 weeks. During that time period, the participants were asked to use the trial glucose meter to measure BGL three to four times per day (before breakfast, lunch, dinner, or bed) and use the app to record insulin dosages taken and comments on a daily basis. The nurse reviewed the participants' data twice a week through the clinician portal and sent up to two text messages to test the messaging feature of the system.

The user experience was assessed by a questionnaire with Likert scales. The Likert scales were scored as 1: Strongly Disagree, 2: Disagree, 3: Neutral, 4: Agree, and 5: Strongly Agree. To evaluate the user adherence, we extracted and analyzed the participants' entries recorded through the m-Health system (the glucose meter, smartphone app, and clinician portal). A participant was considered to adhere to the program, if he/she provided three BGL entries or more per day as recommended in the trial.

Results

Ten patients were recruited to participate in the trial. Of the 10 participants recruited, 1 participant withdrew from the trial due to change of health status. Nine participants completed the trial, aged 58 ± 14 years (mean ± SD), with HbA1c of 8.6% ± 2.9% (mean ± SD) at baseline. During the trial period, the participants on average recorded 3.1 BGL entries, 1.2 insulin dosage entries, and 1.0 comment entry every day. The results of BGL entries during the trial period are shown in Table 1. Seven participants (78% of the participants) were considered to adhere to the program. The system achieved 100% precision in automatic transmission of the BGL entries (n = 390) from the glucose meters to the clinical portal.

Table 1.

The Values of Blood Glucose Level (mmol/L, Mean ± Standard Deviation) at Different Times in Relation to the Meal, Recorded by the Nine Participants Over the Two-Week Period

INDEX	BEFORE BREAKFAST	AFTER BREAKFAST	BEFORE LUNCH	AFTER LUNCH	BEFORE DINNER	AFTER DINNER	BEFORE BED
1	7.2 ± 1.4	—	—	—	8.5 ± 1.6	—	9.1 ± 1.7
2^a	10.0 ± 2.2	8.9 ± 3.9	8.0 ± 2.8	8.5 ± 2.3	7.3 ± 1.2	9.6 ± 2.6	8.6 ± 3.1
3^a	6.7 ± 0.6	—	6.9 ± 1.0	—	6.0 ± 0.6	—	—
4	7.1 ± 1.5	—	—	—	6.6 ± 2.5	—	—
5^a	9.6 ± 2.8	—	9.3 ± 4.3	13.3 ± 3.2	10.5 ± 2.9	14.6 ± 6.1	9.5 ± 0.6
6^a	6.8 ± 1.2	6.5 ± 0.6	6.2 ± 1.1	7.0 ± 1.2	5.7 ± 0.6	—	6.6 ± 0.6
7^a	6.3 ± 0.4	—	7.0 ± 1.0	—	7.5 ± 1.8	—	8.1 ± 2.5
8^a	10.5 ± 1.9	11.4 ± 1.8	9.8 ± 2.3	11.9 ± 3.7	10.6 ± 6.4	—	—
9^a	8.3 ± 1.1	—	7.7 ± 1.6	—	9.1 ± 1.8	—	—

Patients who adhered to the trial, achieving an average rate of more than 3 BGL entries per day, required in the trial.

BGL, blood glucose level.

The user experience results are shown in Figure 3. The participants were generally satisfied with the instructions provided in the program, agreed that the glucose meter and smartphone application were easy to use and it was easy to upload the data entries (scores >4). They reported that the app gave greater self-confidence to self-manage their diabetes, and the participants' families were generally considered to be supportive of the program with both these questions scoring highly at 4.3.

Fig. 3.

Scores of the questionnaire to evaluate the user experience.

The participants were positive and disagreed that participation in this study took too much time (score = 1.7). They would continue to use the system if possible (score = 4.2). Importantly, they considered that using the system gave them confidence to manage their diabetes (score = 4.3). Finally, they were generally satisfied in using the system (score = 4.3), although some participants experienced technical issues during the trial (score = 2.6). We reviewed the responses to the free-text question asking participants what could be improved and found that the technical issues were mainly commented. For example, the testimonies stated “Sometimes a time delay in transferring data from phone,” and “unclear about ‘Bluetooth’ needs to be ‘on’ on the phone and glucose meter” for transferring data. Some participants also expected improvement in ensuring automatic uploads to the web portal in real time rather than at midnight each day and having the ability to generate a PDF report that could be viewed and communicated to other members of the healthcare team such as GP.

Discussion

This study is the first step toward the integration of an innovative m-Health system within a tertiary diabetes service to enable an alternative program for IDA interventions. Different from the traditional IDA programs, the mIDA program mainly uses smartphone and Internet applications to remotely monitor the patients' BGL and deliver the interventions. Although many m-Health applications for diabetes are available, the clinical evidence, especially for supporting care services, remains limited.^14,26,27 The study results on the adherence and user experience are essential for further validation of the mIDA program in clinical trials and help understand potential issues and benefits of using m-Health to improve diabetes care.

This study demonstrated that the majority of the participants (78% of the participants) adhered to the program and provided sufficient BGL data for the IDA assessments and interventions. Patients on insulin therapy are generally required to monitor BGL before meals and snacks and at bedtime in their daily lives (normally three or more times daily),²⁸ but patient adherence to monitoring recommendation is practically suboptimal, with reported nonadherence rates of 50–60% in patients with type-1 diabetes^29,30 and >45% with T2DM.³¹ Our adherence result demonstrated potential to use the mIDA program to achieve a good patient adherence rate at the crucial time of initiating or adjusting insulin treatment. This result also adds evidence to support the use of m-Health for diabetes care.

The evaluation of user acceptance demonstrated three important findings. First, the study found that the participants generally agreed that the application was easy to use. This finding is important, because the use of technology, including m-Health devices, is often considered complex and even frustrating³² especially for an aging population.^33,34 Second, the usefulness of the program was well recognized. This study demonstrated that the participants were willing to take the advice sent through the mobile application seriously and continue using the system if possible. The participants also reported the recognition of their family members to support use of the IDA program. The recognized usefulness indicates a practical need for the participants and their families to receive clinical interventions and support to help manage diabetes. Finally, the participants overall were satisfied with the system and disagreed that the study consumed much of their time. Such satisfaction with new technology has been reported in other studies on telemedicine interventions for both type-1 and type-2 diabetes.^27,35 All these findings support the positive adherence outcome in the analysis of recorded data and demonstrate a great potential to use m-Health for IDA interventions to improve diabetes care.

Despite the positive findings, some technical issues were identified by the participants, including delayed Bluetooth transmissions and difficulties in operating the Bluetooth glucose meter. To prevent the issues in our future trials, we have modified the m-Health system by adding progress notifications and structured operation instructions in the app to overcome these issues. In fact, issues of unreliable data transmission and interoperability have been recognized as a major barrier to practical use of m-Health applications.³⁶ With increasing improvement in m-Health communication standards and regulations,^37,38 m-Health systems will, however, become more and more reliable and practical to use. Nevertheless, these issues should not be overlooked at any stage. We will continue to evaluate the user experience and technical issues in our future studies toward final adoption of the mIDA program.

The study is limited to a small sample size, although other reports with a sample size of 10 participants³⁹ and even smaller sizes of 4–5 participants were considered sufficient to reflect 80% of usability issues.^40,41 The evaluation is also limited to the trial duration of a 2-week period, which is shorter than a traditional 6-week IDA program. Budget constraints did not allow a longer trial period. However, we estimated that a 2-week trial period would be sufficient for the participants to perceive the major concept of the program and experience the majority of technical issues, because short trial durations, including 2 weeks, have been successfully used for such evaluation.^42,43 In addition, patients were only recruited with stable BGL; and the trial did not include patients with type-1 diabetes. Because of these limitations, further studies with larger sample sizes, full mIDA interventions, and patients with different diabetes types are needed to evaluate the use of the mIDA program in the general diabetes population.

Conclusion

This proof-of-concept trial evaluating the user experience of the mIDA system found very promising degrees of usability, perception of usefulness, and satisfaction. The study also found a high level of patient adherence, potentially attributed by the positive user experience findings. Although the study has some limitations, the findings will assist in modifying the system to improve patient usability and support further validation of the mIDA program within the context of a potentially more efficient model of diabetes care that can improve diabetes outcomes.

Footnotes

Acknowledgments

The authors gratefully acknowledge the clinical nurses, including Kerry Porter and Denise Bennetts, at the PAH for their contributions to the recruitment and management of participants in the trial. The authors also acknowledge Roche Australia (www.roche-australia.com/) for providing the wireless glucose meters (Accu-Chek^® Connect) in this proof-of-concept trial. This research was conducted with the support of funding from the Queensland Government through the Advance Queensland Fellowship granted to F.F. and Diabetes Queensland Scholarship granted to A.M. The authors also recognize the support of the Centre of Research Excellence in Telehealth funded by the National Health and Medical Research Council (NHMRC; grant ID: APP1061183).

Disclosure Statement

No competing financial interests exist.

References

Guariguata

, Whiting

, Hambleton

, Beagley

, Linnenkamp

, Shaw

. Global estimates of diabetes prevalence for 2013 and projections for 2035. Diabetes Res Clin Pract, 2014; 103:137–149.

Kalyani

, Saudek

, Brancati

, Selvin

. Association of diabetes, comorbidities, and A1C with functional disability in older adults: Results from the National Health and Nutrition Examination Survey (NHANES), 1999–2006. Diabetes Care, 2010; 33:1055–1060.

World Health Organisation. Global report on diabetes. 2016. Available at http://apps.who.int/iris/bitstream/10665/204874/1/WHO_NMH_NVI_16.3_eng.pdf (last accessed July 1, 2017 ).

Marathe

, Gao

, Close

. American Diabetes Association Standards of Medical Care in Diabetes 2017. J Diabetes, 2017; 9:320–324.

McBrien

, Naugler

, Ivers

, et al. Barriers to care in patients with diabetes and poor glycemic control-a cross-sectional survey. PLoS One, 2017; 12:14.

Saydah

, Fradkin

, Cowie

. Poor control of risk factors for vascular disease among adults with previously diagnosed diabetes. JAMA, 2004; 291:335–342.

Alvarez Guisasola

, Mavros

, Nocea

, Alemao

, Alexander

, Yin

. Glycaemic control among patients with type 2 diabetes mellitus in seven European countries: Findings from the Real-Life Effectiveness and Care Patterns of Diabetes Management (RECAP-DM) study. Diabetes Obes Metab, 2008; 10 Suppl 1:8–15.

Brod

, Alolga

, Meneghini

. Barriers to initiating insulin in type 2 diabetes patients: Development of a new patient education tool to address myths, misconceptions and clinical realities. Patient, 2014; 7:437–450.

Kunt

, Snoek

. Barriers to insulin initiation and intensification and how to overcome them. Int J Clin Pract Suppl, 2009; 6–10.

10.

World Health Organisation. mHealth New horizons for health through mobile technologies. 2011. Available at www.who.int/goe/publications/goe_mhealth_web.pdf (last accessed July 1, 2017 ).

11.

Holtz

, Lauckner

. Diabetes management via mobile phones: A systematic review. Telemed J E Health, 2012; 18:175–184.

12.

Eng

, Lee

. The promise and peril of mobile health applications for diabetes and endocrinology. Pediatr Diabetes, 2013; 14:231–238.

13.

Fatehi

, Gray

, Russell

. Mobile Health (mHealth) for diabetes care: Opportunities and challenges. Diabetes Technol Ther, 2017; 19:1–3.

14.

, McMahon

, Gross

, Adam

, Wyman

. Usability and clinical efficacy of diabetes mobile applications for adults with type 2 diabetes: A systematic review. Diabetes Res Clin Pract, 2017; 131:70–81.

15.

Joiner

, Nam

, Whittemore

. Lifestyle interventions based on the diabetes prevention program delivered via eHealth: A systematic review and meta-analysis. Prev Med, 2017; 100:194–207.

16.

Holmen

, Wahl

, Cvancarova Smastuen

, Ribu

. Tailored communication within mobile apps for diabetes self-management: A systematic review. J Med Internet Res, 2017; 19:e227.

17.

Kim

, Jeong

. A nurse short message service by cellular phone in type-2 diabetic patients for six months. J Clin Nurs, 2007; 16:1082–1087.

18.

Bee

, Batcagan-Abueg

, Chei

, et al. A Smartphone application to deliver a treat-to-target insulin titration algorithm in insulin-naive patients with type 2 diabetes: A pilot randomized controlled trial. Diabetes Care, 2016; 39:e174–e176.

19.

Turner

, Larsen

, Tarassenko

, Neil

, Farmer

. Implementation of telehealth support for patients with type 2 diabetes using insulin treatment: An exploratory study. Inform Prim Care, 2009; 17:47–53.

20.

Vähätalo

, Virtamo

, Viikari JS

T R

. Cellular phone transferred self blood glucose monitoring: Prerequisites for positive outcome. Pract Diabetes, 2004; 21:3.

21.

Levy

, Moynihan

, Nilo

, et al. The mobile insulin titration intervention (MITI) for insulin adjustment in an urban, low-income population: Randomized controlled trial. J Med Internet Res, 2015; 17:e180.

22.

Hsu

, Lau

, Huang

, et al. Utilization of a cloud-based diabetes management program for insulin initiation and titration enables collaborative decision making between healthcare providers and patients. Diabetes Technol Ther, 2016; 18:59–67.

23.

Baron

, Hirani

, Newman

. A randomised, controlled trial of the effects of a mobile telehealth intervention on clinical and patient-reported outcomes in people with poorly controlled diabetes. J Telemed Telecare, 2017; 23:207–216.

24.

Fatehi

, Smith

, Maeder

, Wade

, Gray

. How to formulate research questions and design studies for telehealth assessment and evaluation. J Telemed Telecare, 2017; 23:759–763.

25.

Varnfield

, Karunanithi

, Lee

, et al. Smartphone-based home care model improved use of cardiac rehabilitation in postmyocardial infarction patients: Results from a randomised controlled trial. Heart, 2014; 100:1770–1779.

26.

Arnhold

, Quade

, Kirch

. Mobile applications for diabetics: A systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older. J Med Internet Res, 2014; 16:e104.

27.

Wood

, Clements

, McFann

, Slover

, Thomas

, Wadwa

. Use of telemedicine to improve adherence to American Diabetes Association standards in pediatric type 1 diabetes. Diabetes Technol Ther, 2016; 18:7–14.

28.

American Diabetes Association. 1. Promoting health and reducing disparities in populations. Diabetes Care, 2017; 40(Suppl 1):S6–s10.

29.

Hansen

, Pedersen-Bjergaard

, Heller

, et al. Frequency and motives of blood glucose self-monitoring in type 1 diabetes. Diabetes Res Clin Pract, 2009; 85:183–188.

30.

Mostrom

, Ahlen

, Imberg

, Hansson

, Lind

. Adherence of self-monitoring of blood glucose in persons with type 1 diabetes in Sweden. BMJ Open Diabetes Res Care, 2017; 5:e000342.

31.

Virdi

, Daskiran

, Nigam

, Kozma

, Raja

. The association of self-monitoring of blood glucose use with medication adherence and glycemic control in patients with type 2 diabetes initiating non-insulin treatment. Diabetes Technol Ther, 2012; 14:790–798.

32.

Demidowich

, Lu

, Tamler

, Bloomgarden

. An evaluation of diabetes self-management applications for Android smartphones. J Telemed Telecare, 2012; 18:235–238.

33.

El-Gayar

, Timsina

, Nawar

, Eid

. Mobile applications for diabetes self-management: Status and potential. J Diabetes Sci Technol, 2013; 7:247–262.

34.

Sarkar

, Gourley

, Lyles

, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med, 2016; 31:1417–1426.

35.

Fatehi

, Martin-Khan

, Smith

, Russell

, Gray

. Patient satisfaction with video teleconsultation in a virtual diabetes outreach clinic. Diabetes Technol Ther, 2015; 17:43–48.

36.

PLOS Medicine Editors. A reality checkpoint for mobile health: Three challenges to overcome. PLoS Med, 2013; 10:e1001395.

37.

Martin

, Ding

, D'Souza

, Karunanithi

. Evaluation of bluetooth low power for physiological monitoring in a home based cardiac rehabilitation program. Stud Health Technol Inform, 2012; 178:124–129.

38.

Cortez

, Cohen

, Kesselheim

. FDA regulation of mobile health technologies. N Engl J Med, 2014; 371:372–379.

39.

Georgsson

, Staggers

. Quantifying usability: An evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc, 2016; 23:5–11.

40.

Nielsen

, Landauer

. A mathematical-model of the finding of usability problems. Hum Factors Comput Syst, 1993; 206–213.

41.

Virzi

. Refining the test phase of usability evaluation—How many subjects is enough. Hum Factors, 1992; 34:457–468.

42.

Arsand

, Muzny

, Bradway

, Muzik

, Hartvigsen

. Performance of the first combined smartwatch and smartphone diabetes diary application study. J Diabetes Sci Technol, 2015; 9:556–563.

43.

Nelson

, Bethune

, Lagotte

, Osborn

. The usability of diabetes MAP: A web-delivered intervention for improving medication adherence. JMIR Hum Factors, 2016; 3:e13.