Leveraging Technology: Creating and Sustaining Changes for Health

Abstract

Objective: The rapid growth and evolution of health-related technology capabilities are driving an established presence in the marketplace and are opening up tremendous potential to minimize and/or mitigate barriers associated with achieving optimal health, performance, and readiness. This article summarizes technology-based strategies that promote healthy habits related to physical activity, nutrition, and sleep. Materials and Methods: The Telemedicine and Advanced Technology Research Center convened a workshop titled “Leveraging Technology: Creating & Sustaining Changes for Health” (May 29–30, 2013, Fort Detrick, MD). Participants included experts from academia (n=3), government (n=33), and industry (n=16). A modified Delphi method was used to establish expert consensus in six topic areas: (1) physical activity, (2) nutrition, (3) sleep, (4) incentives for behavior change, (5) usability/interoperability, and (6) mobile health/open platform. Results: Overall, 162 technology features, constructs, and best practices were reviewed and prioritized for physical activity monitors (n=29), nutrition monitors (n=35), sleep monitors (n=24), incentives for change (n=36), usability and interoperability (n=25), and open data (n=13). Conclusions: Leading practices, gaps, and research needs for technology-based strategies were identified and prioritized. This information can be used to provide a research and development road map for (1) leveraging technology to minimize barriers to enhancing health and (2) facilitating evidence-based techniques to create and sustain healthy behaviors.

Introduction

The U.S. Army Surgeon General's Performance Triad is represented by physical activity, nutrition, and sleep—three key areas that affect the cognitive and physical performance, personal well-being, readiness and resilience, and the health of the military members, retirees, and their families.¹ The Performance Triad serves as a foundational component in the transformation of Army medicine from a healthcare system into a system for health.¹ The availability of affordable, personalized, health-related technology is rapidly growing; such technology can potentially be used to minimize and/or mitigate barriers (e.g., lack of time, insufficient knowledge, lack of goal setting and tracking, inadequate social support, few incentives to change behavior) to achieving optimal health, performance, and readiness. Emerging evidence indicates that utilization of technology (e.g., Web-based programs, text messaging) can enhance behavior change intervention effectiveness.^2
–4

Physical Activity Monitoring Technologies

Physical activity monitoring technologies have proliferated in the last three decades. Technological advances have allowed accelerometer-based, motion-sensing technologies to be miniaturized to the micrometer scale and integrated with wireless capabilities.⁵ These advances have enabled direct-to-consumer commercialization of motion-sensing technology for gaming, mobile devices, and physical activity monitoring. The linkage between Web- and mobile-based dashboards and physical activity monitoring may enable valuable behavior change interventions.

Evidence supporting the effectiveness of activity monitoring technology for promotion of physical activity behavior is emerging.⁶ Self-monitoring through the use of pedometer-based interventions has increased awareness and physical activity during 1–6-month trials.⁷ A recent systematic review in the Journal of the American Medical Association ⁸ demonstrated that devices used to track activity resulted in a significant increase in physical activity (2,491 steps per day more versus control participants). Individuals who used tracking devices increased their physical activity by 26.9% over the baseline. A significant predictor of increased physical activity was having in place a step goal, such as 10,000 steps per day. The authors determined that use of devices that automatically track physical activity resulted in a significant decrease in body mass index (of 0.38 kg/m²) and a significant decrease in systolic blood pressure (of 3.8 mm Hg). However, the long-term effects of physical activity monitoring on the maintenance of behavior change remains unknown.

Nutrition and Food Intake Monitoring Technologies

Researchers have demonstrated four strategies for successful long-term weight maintenance: (1) eating a diet low in fat, (2) frequent self-monitoring of body weight (daily or weekly), (3) tracking food (calorie) intake, and (4) high levels of regular physical activity.^9,10 Although these strategies are well documented, they require user knowledge, tracking, and time. Use of nutrition and food intake monitoring technologies that address these barriers has been correlated with dietary intake improvements^11,12 and successful weight loss.^13
–15 One of the proposed benefits of nutritional monitoring devices is that they provide an extensive and automated food library, which decreases the user's burden to manually track nutrient and calorie intake. A systematic literature review found a significant association between the use of self-monitoring and self-weighing (to include use of Internet and mobile devices and applications) and weight loss.¹⁶ However, techniques to improve compliance with these tools are required.

Nutrition and food intake monitoring devices can provide automated and tailored feedback and education. Investigators have determined that tailored feedback and prompting are important components of successful behavioral interventions for improving nutrition and food intake habits.^17,18 A recent randomized controlled trial found that education supplemented by self-monitoring and feedback resulted in twice as much weight loss (3.5±3.8 kg) compared with education alone (1.4±2.7 kg).¹⁹ The percentage of individuals who achieved at least a 5% weight loss was 40.5% in the education/self-monitoring/feedback group compared with 13.2% in the education-only group. Further investigation is warranted to determine optimal prompting parameters (e.g., extent, frequency) to effectively maintain weight loss over time.²⁰

Sleep Monitoring Technologies

Most sleep medicine clinicians and researchers agree that patients should prospectively record their sleep nightly with a sleep diary to assess and track treatment effects.^21,22 In fact, the sleep diary is considered the “gold standard” for subjective sleep assessment.²³ Prestwich et al.²⁴ found that healthy participants who received sleep education and logged sleep hours increased their nightly sleep by 50 min. A recent study of medical students that focused on changing health behaviors found the requirement to monitor, chart, graph, and journal about their behavior change for 6 weeks was important for goal achievement—and the process made them feel healthier—but the time required to manually log progress was a barrier.²⁵

Although self-reported sleep parameters can be helpful to track sleep over time, they differ substantially from objective measurement of sleep (notably, latency to sleep and wake after sleep onset).²⁶ Therefore, sleep monitoring technologies that provide objective, accurate measures of sleep are important. Polysomnography (PSG) has been the traditional gold standard for objective sleep measurement²⁷ and diagnosis of sleep disorders.²⁸ However, PSG is costly and impractical for long-term and home utilization. Wrist-worn, accelerometer-based actigraphy is a mature technological alternative to PSG.^29
–31 In fact, actigraphy is widely used within the sleep research community to track daily total sleep time over days or weeks. Initial evidence suggests that technologies for tracking sleep behaviors can increase awareness of sleep patterns and lead to healthier sleep habits.³² However, the reliability and validity of new devices (as compared with PSG) have yet to be determined.^33,34

Creating and Sustaining Behavior Change

Technology approaches can be tailored to address specific components of behavior change, such as reward types, user autonomy, and habit. Researchers have examined the extent to which different reward types (i.e., extrinsic versus intrinsic) drive behavior change in multiple settings, including educational gaming.³⁵ Extrinsic motivations (e.g., derived from rewards, prizes, or payments) have been effective in the short term,^36,37 whereas intrinsic motivations (e.g., motivation from the purpose or outcome itself) have been found to be more effective for long-term maintenance of behavior changes.³⁸ Interventions with substantial self-guided components and autonomy building,³⁹ which can be enabled by technology approaches, have been found to be effective while also being more scalable (e.g., can be distributed to large groups of people with reasonable methods and cost) because they do not rely heavily on healthcare practitioner participation.

Habits guide about 45% of human behavior.⁴⁰ Thus, behavior change technologies will inevitably need to address habit, either by disrupting existing unhealthful habits and/or by forming new healthful ones. Because of the unique way that habits are triggered, interventions that are effective at changing people's intentions (thoughtful, planned, and novel actions) are ineffective at changing habits.^41
–43 Rather than addressing intentions alone, interventions targeting habits must first remove or change the contextual triggers (e.g., environmental triggers or preceding actions in a sequence) that prompt people to mindlessly repeat familiar routines (e.g., mindless eating).^44,45 Experiments on mindless eating have shown, for example, that eating “out of habit” can be disrupted simply by moving to a new context not associated with eating a specific food or by simply using one's nondominant hand to eat.⁴⁶

Usability and Interoperability

Usability (ease of use) and interoperability are critical characteristics of technology-based behavior change interventions to maximize user compliance and effectiveness. Instructional design—the process of making acquisition of knowledge more efficient, effective, and appealing—has been found to be a critical factor in ensuring that technological approaches are user-friendly.⁴⁷ For example, wearability (e.g., size, location, appropriateness for dress), frequency, and time required to change batteries and upload data can negatively impact users' experience with technology, potentially reducing the use of the technology and exposure to the intervention. These factors impact marketability and the extent to which users will continue using such technologies over time.

Mobile Health and Open Data

The proliferation of technology for health monitoring and behavior change is generating vast amounts of data and is creating a greater demand for analytical capabilities. It has been estimated that by 2015, 2 billion smartphones will be in use,⁴⁸ and 500 million people will be using mobile healthcare applications.⁴⁹ The ubiquitous nature of global positioning system capabilities in smartphones is driving an emphasis on geospatial data collection and analysis for healthcare. “Big Data” can be used to identify trends, improve access, increase efficiency, and reduce costs. Realization of these improvements would be beneficial given that the United States pays 1.5 times more per capita for healthcare than any other developed county yet ranks below average on healthcare outcomes such as infant mortality and life expectancy.⁵⁰ Health monitoring devices and analytical tools from “Big Data” could also be applied to efforts aimed at creating and sustaining behavior change interventions.

Objectives

Although existing evidence is encouraging, research has not kept pace with the recent growth in personal health devices. As a result, there is a lack of consensus on the leading practices in this industry. To address this shortcoming, the Telemedicine and Advanced Technology Research Center of the U.S. Army Medical Research and Materiel Command convened a workshop titled “Leveraging Technology: Creating & Sustaining Changes for Health” (May 29–30, 2013, Fort Detrick, MD). Participants explored technology-based strategies for promoting healthy habits related to physical activity, nutrition, and sleep. This article prioritizes and summarizes identified best practices and research gaps in leveraging technology to create and sustain behavior changes to positively impact health.

Materials and Methods

A modified Delphi method^51

–54 was used to identify and prioritize leading practices (including features and constructs) for strategic development of technology-based behavior change interventions related to physical activity, nutrition, and sleep. The Delphi method is an anonymous, iterative process for gathering information and building consensus among experts about a specified topic.⁵¹ The Delphi method typically involves administering multiple rounds of a structured survey to a panel of experts who evaluate and/or provide commentary on specific information elements of interest.^52,53 The results of each round are shared with the panel experts before the survey is subsequently reevaluated in the next round. Through multiple iterations of this survey process, the Delphi method enables feedback and discussion among experts to drive and document resulting consensus.⁵⁴

Our application of a modified Delphi method consisted of three rounds. The first round identified leading clinical practices that could be enabled by technology during an April 2013 Telemedicine and Advanced Technology Research Center workshop titled “Incentives to Create and Sustain Changes for Health” (April 24–25, 2013, Fort Detrick).^55,56 Moderated workshop discussion was informed by a review of over 500 publications in the areas of (1) public health and messaging, (2) influence of environment and habit, (3) goal setting and tracking, (4) incentives for change, and (5) influence of peers and social networks on change. Although the focus of the review was on technology solutions to promote optimal activity, nutrition, and sleep, other public health literature was reviewed to help ensure that relevant technology requirements and capabilities were identified. Workshop participants identified 122 technology requirements and capabilities across six topic areas: (1) physical activity, (2) nutrition, (3) sleep, (4) incentives for behavior change, (5) usability/interoperability, and (6) mobile health/open platform.

The second and third Delphi rounds were conducted during the May 2013 workshop, “Leveraging Technology: Creating & Sustaining Changes for Health.” During the second round, government (n=33) and academic (n=3) participants rated leading practices for each of the six topic areas on a scale of 0 to 10 (0=not important, 5=nice to have, 10=very important or must have). The second round was followed by industry presentations and a moderated panel discussion among all workshop participants. Industry participants (n=16) were asked to address the identified best practices as they applied to commercial products and implementations. Industry experts participated in the group discussions but were excused from the room during voting to allow academic and government participants to discuss the technology features and constructs privately prior to the third round. To guide discussion, mean scores and standard deviations from the second round were provided to workshop participants. Additional leading practices identified during the discussion period were added to the survey for the third round of evaluation. Participants were asked to provide qualitative comments during the final round of voting. This process was conducted for each of the six topic areas.

Results

Academic and government workshop attendees participated in the overall Delphi voting process (see Appendix). Voter participation per round for each topic area is summarized in Table 1.

Table 1.

Summary of Survey Responses Received

VOTING ROUND	DELPHI ROUND	PHYSICAL ACTIVITY	NUTRITION	SLEEP	INCENTIVES	USABILITY AND INTEROPERABILITY	MOBILE HEALTH AND OPEN DATA
1	2	26	29	25	25	23	21
2	3	29	27	25	26	22	28

Results from the Delphi review are summarized in Tables 2 –7. For all of these tables, the list of leading practices is provided based on the final rankings (third Delphi round, postdiscussion) for all six topic areas. Initial rankings (n=122) (second Delphi round, prediscussion) are listed in the third column. Based on the group discussion, additional leading practices (n=40) were added for the third round of voting after the moderated discussion; hence, they do not include a prediscussion (second Delphi round) score.

Table 2.

Physical Activity Monitoring Technologies

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Counts steps	9.8 (0.6)	9.3 (1.5)
2 (2)	Tracks distance traveled	8.9 (1.9)	9.1 (1.0)
3 (3)	Is waterproof	8.7 (1.4)	9.0 (1.3)
4 (6)	Provides accurate and reliable data	8.6 (1.7)	8.8 (1.3)
5 (3)	Tracks time and duration of activity intensities	8.8 (1.3)	8.8 (1.4)
6 (5)	Tracks progress over time (e.g., day, week, month, year)	8.7 (1.8)	8.7 (1.7)
7 (9)	Tracks multiple types of aerobic activities (e.g., walk, run, aerobics, bike)	8.0 (1.9)	8.7 (1.8)
8 (N/A)	API output allows researchers to access data and allows third-party application development	N/A	8.2 (2.4)
9 (N/A)	Associated Web- or mobile-based application provides evidence-based guidance	N/A	8.2 (2.1)
10 (N/A)	Automatically can monitor different activities without user input (e.g., walk, run, aerobics, bike)	N/A	7.9 (2.3)
11 (14)	Tracks multiple types of strength training activities	7.1 (2.5)	7.6 (2.5)
12 (N/A)	Supports self-learning, adjustable algorithms, or machine learning to improve accuracy, validity, and reliability	N/A	7.6 (2.6)
13 (13)	Specifically tracks biking	7.1 (2.7)	7.5 (2.2)
14 (N/A)	Demonstrates a stability over time (e.g., drift, hysteresis)	N/A	7.5 (2.3)
15 (12)	Provides encouragement and motivation to facilitate increased physical activity	7.5 (2.7)	7.3 (2.2)
16 (11)	Counts stairs/incline/altitude (e.g., includes an altimeter)	7.5 (1.8)	7.3 (1.7)
17 (8)	Tracks calories burned	8.2 (2.5)	7.2 (2.8)
18 (7)	Provides visual feedback on device (e.g., doesn't require user to log into an application to view current data)	8.3 (2.6)	7.2 (2.2)
19 (N/A)	Tracks inactivity (e.g., prolonged sitting)	N/A	7.1 (2.9)
20 (N/A)	Is GPS-enabled to allow the monitor to provide suggestions for physical activity based on current location (e.g., nearby gym, walking trails, park while traveling)	N/A	7.0 (1.7)
21 (10)	Incorporates features that enable users to track lifestyle and other activities, including balance and stretching	7.8 (2.1)	6.7 (2.7)
22 (15)	Monitors heart rate	6.2 (2.8)	6.7 (2.5)
23 (N/A)	Has a sensitivity in the device that allows user to accurately track both improvements and decrements in physical activity level over time	N/A	6.6 (2.3)
24 (N/A)	Has features that allow comparison of results with those of friends, groups, or group goals	N/A	6.1 (3.2)
25 (16)	Provides a sedentary alarm to warn of prolonged inactivity	5.5 (2.3)	5.4 (3.0)
26 (17)	Provides suggested workouts and physical activity routines based on goals and preferences	5.3 (2.9)	5.3 (2.3)
27 (N/A)	Monitors recovery time to warn of overexertion and possible overuse	N/A	5.0 (2.7)
28 (N/A)	Incorporates secondary peripheral devices (e.g., bike pedal device, foot pod, weight scale, GPS device) to improve accuracy for specific events	N/A	3.8 (2.5)
29 (18)	Incorporates and monitors pulse oximetry, heart rate, and other physiological measures	3.7 (2.5)	2.9 (2.4)

Data are mean (standard deviation) values.

API, application programming interface; GPS, global positioning system; N/A indicates a variable added during discussion period by the subject matter expert panel, and initial ranking is not applicable for those features.

Table 3.

Nutrition Monitoring Technologies

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Tracks weight over time based on user data and/or use of a smart scale	9.7 (0.6)	9.6 (1.1)
2 (4)	Includes credible and searchable food library/database (e.g., USDA)	9.0 (1.5)	9.4 (0.9)
3 (2)	Estimates calorie intake based on user data	9.1 (1.3)	9.4 (1.0)
4 (5)	Makes it easy to add entries into food library	8.8 (1.6)	9.1 (0.9)
5 (3)	Has dynamic content databases (e.g., routinely updated with nutrition information, additions)	9.0 (1.3)	9.1 (1.1)
6 (7)	Has a customizable food library (preferred meals/common foods)	8.6 (1.8)	8.7 (1.3)
7 (6)	Maintains food diary	8.7 (1.8)	8.7 (1.1)
8 (N/A)	Has device features and/or a Web site designed to make foot input easier	N/A	8.4 (2.3)
9 (13)	Provides tailored information to user based on data user inputs with the use of known algorithms	8.0 (2.0)	8.4 (1.4)
10 (20)	Provides caloric prescription based on personal information that shifts according to the information changing over time	7.4 (1.8)	8.2 (1.5)
11 (11)	Allows addition of entries/recipes into the food library	8.2 (1.9)	8.2 (2.1)
12 (N/A)	Tracks particular food intake goals	N/A	8.0 (1.9)
13 (10)	Tracks nutrition/food intake information over time (e.g., proteins, fat, carbohydrates)	8.2 (2.2)	7.9 (1.7)
14 (12)	Incorporates military weight standards for tracking, comparisons, and goal setting	8.0 (2.2)	7.8 (2.3)
15 (8)	Compares calories consumed to calories burned (e.g., integrates with physical activity monitoring technology)	8.2 (1.8)	7.8 (2.4)
16 (17)	Has a meal planner that provides suggested food for meals and snacks based on personalized goals	7.5 (1.6)	7.6 (1.7)
17 (23)	Has a prompting program to help ensure compliance and use of food diary	7.0 (2.9)	7.6 (2.0)
18 (N/A)	Tracks time of day eating to help educate individuals on trends	N/A	7.5 (2.5)
19 (N/A)	Tracks particular food vice/habit to help educate individuals on trends or concerns	N/A	7.4 (2.1)
20 (N/A)	Tracks nutrient timing with physical activity to help educate individuals on how to optimize performance through sports nutrition	N/A	7.4 (2.2)
21 (9)	Tracks calories consumed to calories burned	8.2 (1.7)	7.4 (2.4)
22 (N/A)	Provides the user with alternatives to commonly consumed food that may be similar but have less calories (e.g., suggestion to eat this instead of that type of food tool)	N/A	7.4 (2.8)
23 (N/A)	Tracks the number of times a day an individual eats and uses that information for coaching and mentoring	N/A	7.3 (2.6)
24 (18)	Has a meal planner linked with recipes to facilitate healthy cooking	7.5 (1.6)	7.2 (1.8)
25 (15)	Can use a barcode scanner to enter food to a diary and food library	7.8 (2.2)	7.2 (2.1)
26 (14)	Tracks body mass index over time	7.9 (2.1)	7.1 (2.6)
27 (19)	Interfaces with digital scale to make entering weight over time easier	7.5 (2.0)	6.9 (2.4)
28 (22)	Can use digital pictures of food to measure portion size	7.1 (2.8)	6.8 (2.1)
29 (21)	Can use digital pictures of food to determine estimated calories	7.1 (2.7)	6.7 (2.2)
30 (16)	Calculates and tracks percentage of body fat over time	7.7 (2.0)	6.5 (2.5)
31 (N/A)	Uses a meal planner that allows individualization compared to a meal prescription	N/A	6.4 (2.5)
32 (24)	Generates a grocery list once user approves weekly meal plan	6.7 (1.8)	6.3 (2.3)
33 (N/A)	Uses GPS to determine common locations where a user eats to spot trends (e.g., overeating typically occurs when user is at a particular address)	N/A	6.2 (3.6)
34 (N/A)	Uses GPS to provide suggested healthy restaurants in the current location and “warnings” based on past trends	N/A	6.0 (3.3)
35 (25)	Provides body image assessment	5.7 (3.3)	5.3 (3.4)

Data are mean (standard deviation) values.

GPS, global positioning system; N/A indicates a variable added during the discussion period by the subject matter expert panel, and initial ranking is not applicable for those features; USDA, U.S. Department of Agriculture.

Table 4.

Sleep Monitoring Technologies

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Tracks time and duration slept	9.8 (0.4)	9.9 (0.3)
2 (3)	Automatically tracks sleeps (e.g., the system doesn't require user input to tell the device the user is going to sleep)	9.5 (1.1)	9.7 (0.5)
3 (2)	Uses a validated algorithm to predict start/stop times for sleep and sleep quality	9.8 (0.8)	9.7 (1.0)
4 (4)	Provides evidence-based recommendations to help a user improve sleep	8.8 (1.3)	8.7 (1.5)
5 (5)	Displays the results of sleep quality and quantity in an intuitive manner	8.7 (1.3)	8.6 (1.4)
6 (6)	Is water-resistant/does not need to be removed to shower or swim (e.g., user can wear the device 24 h/day)	8.6 (2.0)	8.6 (1.7)
7 (N/A)	Easily measures naps in addition to nighttime sleep	N/A	8.0 (2.0)
8 (N/A)	Provides feedback to the user about estimated fatigue levels based on quality and quantity of recent sleep	N/A	8.0 (2.0)
9 (7)	Uses evidence to program sleep recommendations that are provided to the consumer to review	8.4 (1.5)	7.9 (1.9)
10 (8)	Has validated studies of system accuracy, validity, and reliability available for consumers to review	8.2 (2.4)	7.8 (2.1)
11 (N/A)	Provides measures of readiness, performance, and effectiveness based on sleep quality and quantity	N/A	7.7 (1.9)
12 (N/A)	Automatically adjusts to time zones	N/A	7.7 (1.8)
13 (10)	Allows off-wrist detection (e.g., able to differentiate sedentary behavior, off-wrist detection, sleep)	7.0 (3.3)	7.2 (2.7)
14 (N/A)	Provides user with a measure of future accident risk based on prior sleep quantity and quality	N/A	6.9 (2.7)
15 (N/A)	Has sensors that incorporate other measures besides actigraphy to improve algorithms (e.g., heart rate)	N/A	6.6 (2.8)
16 (9)	Tracks the location the device is worn (accuracy of the actigraph device is based on being worn on the nondominant wrist)	7.8 (2.4)	6.5 (2.6)
17 (N/A)	Ability to monitor and stop work when fatigue level is predicted to be dangerous based on prior history of sleep quantity and quality	N/A	6.5 (2.7)
18 (11)	Incorporates alarm clock function (e.g., vibration)	7.0 (2.7)	6.2 (2.4)
19 (N/A)	Links nutrition information (e.g., caffeine, alcohol) with sleep information to enhance coaching and education	N/A	6.0 (2.4)
20 (N/A)	Links mood information to sleep to enhance coaching and education	N/A	5.9 (3.1)
21 (N/A)	Translates poor sleep quantity and quality to blood alcohol levels to enhance coaching and education	N/A	5.8 (2.2)
22 (12)	Provides time on device (e.g., watch functions)	5.9 (3.1)	5.7 (2.7)
23 (N/A)	Has an ambient light sensor to improve validity and reliability of sleep algorithms	N/A	4.0 (2.7)
24 (13)	Plays sounds or music (e.g., MP3 player incorporated with device)	3.3 (2.3)	1.6 (2.0)

Data are mean (standard deviation) values.

N/A indicates a variable added during discussion period by the subject matter expert panel. Initial ranking is not applicable for those features.

Table 5.

Incentives to Create and Sustain Behavior Changes for Health

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Creates/provides individual reports and summaries to user	9.8 (0.4)	9.5 (0.8)
2 (2)	Facilitates goal setting	9.5 (0.7)	9.4 (1.1)
3 (3)	Facilitates goal tracking	9.5 (0.7)	9.4 (1.1)
4 (4)	Provides meaningful information tailored for the individual users	9.4 (0.8)	9.3 (1.1)
5 (6)	Provides personalized and evidence-based health messaging to create and sustain change	8.8 (1.4)	9.0 (1.3)
6 (14)	Provides incentives to help obtain goals	8.4 (1.4)	8.9 (1.2)
7 (5)	Provides strategies to meet goals	9.0 (0.9)	8.8 (1.2)
8 (17)	Can engage those with a low readiness to change	8.0 (2.3)	8.7 (1.7)
9 (7)	Has self-learning algorithms to personalize goals and messages	8.7 (1.1)	8.6 (1.5)
10 (8)	Shares the latest evidence based on personalized goals	8.7 (1.3)	8.6 (1.5)
11 (11)	Personalizes messages based on goals	8.7 (1.3)	8.5 (1.5)
12 (9)	Learns “habits” and uses that data to sustain good habits	8.7 (1.2)	8.5 (1.3)
13 (N/A)	Increases the fun factor to facilitate compliance and behavior change	N/A	8.5 (1.8)
14 (15)	Supports competition and challenges with family, friends, and groups (e.g., leader boards)	8.2 (1.4)	8.4 (1.4)
15 (13)	Can adjust frequency, timing, and tone of messages to match individual preferences	8.4 (1.9)	8.4 (1.8)
16 (18)	Tracks and displays group/collective progress and accountability	7.8 (1.5)	8.2 (1.4)
17 (16)	Provides support/encouragement (auto or live)	8.2 (1.4)	8.1 (1.3)
18 (10)	Supports multimodal interaction (e.g., text, e-mail, video, other)	8.7 (1.3)	8.0 (1.7)
19 (N/A)	Is able to evolve incentives over time to help facilitate personal goals	N/A	7.9 (1.6)
20 (23)	Provides badges and awards	7.4 (1.6)	7.9 (1.7)
21 (21)	Provides online coaching with subject matter expert support	7.5 (2.1)	7.8 (1.6)
22 (19)	Tracks and compares results with groups, friends, and family	7.8 (1.7)	7.8 (1.9)
23 (29)	Provides an avenue for family engagement with geographic separation (e.g., allows a deployed parent to engage with family members)	6.8 (2.5)	7.7 (1.7)
24 (12)	Supports mindful cues and habit cues to help facilitate behavior change and healthy habit formation	8.4 (1.5)	7.6 (1.9)
25 (20)	Sends automated reminders	7.7 (2.0)	7.6 (2.3)
26 (N/A)	Has the right balance between extrinsic and intrinsic motivation	N/A	7.5 (1.6)
27 (25)	Facilitates shared experiences	7.3 (1.5)	7.2 (1.9)
28 (32)	Has messages that integrate family/friends	6.5 (2.0)	7.2 (1.8)
29 (30)	Makes use of a geotag to suggest healthy options (e.g., routes, restaurants)	6.8 (2.3)	7.1 (2.0)
30 (24)	Can scale feedback	7.4 (2.3)	7.0 (1.6)
31 (27)	Makes use of social media	7.2 (2.3)	6.8 (2.1)
32 (31)	Has body image assessment	6.6 (3.2)	6.6 (3.2)
33 (26)	Uses gamification principles	7.3 (2.0)	6.4 (2.2)
34 (22)	Provides network architecture to create “small worlds”	7.4 (1.8)	6.4 (2.0)
35 (28)	Integrates with a calendar	7.2 (2.4)	6.0 (2.3)
36 (33)	Provides an avatar-based coach	5.9 (3.0)	4.8 (2.9)

Data are mean (standard deviation) values.

N/A indicates a variable added during discussion period by the subject matter expert panel. Initial ranking is not applicable for those features.

Table 6.

Usability and Interoperability of Technologies

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Is intuitive for entering data	9.8 (0.4)	9.8 (0.4)
2 (3)	Has a adequate power supply/duration (e.g., battery life)	9.7 (0.6)	9.7 (0.6)
3 (2)	Is easy to set up	9.8 (0.5)	9.7 (0.6)
4 (6)	Easily connects to iOS, Android, and Windows operating systems	9.4 (1.5)	9.6 (0.7)
5 (4)	Connects seamlessly and wirelessly	9.7 (0.8)	9.5 (1.1)
6 (5)	Provides an intuitive display of tracking	9.6 (0.7)	9.3 (0.7)
7 (9)	Is easy to recharge	9.0 (1.5)	9.3 (1.0)
8 (8)	Is available at a reasonable cost	9.3 (1.4)	9.3 (1.1)
9 (7)	Connects to a Mac or PC	9.3 (1.3)	9.2 (1.4)
10 (13)	Has an appropriate size, location, and form factor	8.6 (2.3)	9.1 (1.3)
11 (15)	Provides rapid feedback	8.1 (2.5)	8.6 (1.2)
12 (11)	Is able to integrate multiple biosensors into its software	8.7 (1.4)	8.3 (2.0)
13 (10)	Allows for adjustable privacy settings	8.8 (1.8)	8.2 (2.2)
14 (12)	Provides easy access to all data collected (e.g., activity, nutrition, sleep)	8.6 (2.2)	8.2 (2.5)
15 (14)	Uses adaptable software (self-learning)	8.2 (1.8)	8.1 (1.4)
16 (N/A)	Maintains attention on goals while minimizing the “friction” associated with using the device	N/A	8.0 (2.3)
17 (N/A)	Provides an attractive or fashionable look to the device	N/A	8.0 (1.8)
18 (18)	Synthesizes/combines information from activity, nutrition, and sleep to provide holistic coaching and education	7.4 (2.6)	7.7 (2.1)
19 (23)	Has a travel mode (e.g., ability to use the system without Internet access)	6.2 (2.9)	7.7 (2.6)
20 (19)	Does not require a paid subscription to use online tools	7.1 (3.2)	7.5 (2.9)
21 (16)	Provides customizable displays (e.g., adjust the quantity and level of data provided)	7.7 (1.9)	6.7 (2.3)
22 (17)	Provides corporate wellness software tools to track groups	7.5 (1.9)	6.6 (1.9)
23 (20)	Provides online educational materials (e.g., videos, pamphlets)	7.1 (2.4)	6.2 (2.5)
24 (22)	Serves as a wristwatch or has other useful functions to facilitate compliance with wearing the device	6.3 (2.7)	6.0 (2.3)
25 (21)	Incorporates natural language processing	6.6 (2.9)	5.8 (2.4)

Data are mean (standard deviation) values.

iOS, Apple operating system; N/A indicates a variable added during discussion period by the subject matter expert panel, and initial ranking is not applicable for those features; PC, personal computer (non-Mac).

Table 7.

Mobile Health and Open Data

FINAL RANK (INITIAL RANK)	FEATURES, CONSTRUCTS, AND LEADING PRACTICES	ROUND 2	ROUND 3
1 (1)	Exchanges data easily between hardware devices and software (e.g., application, cloud storage)	9.5 (1.0)	9.9 (0.3)
2 (2)	Makes information and resources open, easy to find, accessible, and usable	9.5 (1.4)	9.7 (0.5)
3 (3)	Safeguards individual privacy and confidentiality	9.5 (1.4)	9.5 (1.2)
4 (6)	Is able to upload data from many similar devices, allowing multiple tracking devices to be used for the same software program	9.0 (1.1)	9.3 (0.9)
5 (4)	Allows devices to connect seamlessly with third-party application integration	9.3 (1.0)	9.2 (0.9)
6 (5)	Is able to integrate data from multiple biosensors to provide an overall assessment of health	9.2 (1.5)	9.0 (1.2)
7 (N/A)	Has multitiered data management data access and permissions to facilitate appropriate sharing of specific data based on family, friends, healthcare provider, wellness coach, etc.	N/A	8.6 (2.3)
8 (8)	Can share data with wellness coaches and healthcare providers	8.5 (1.8)	8.5 (1.6)
9 (7)	Can integrate data with an electronic health record	8.5 (1.8)	8.4 (2.0)
10 (N/A)	Uses standardized terminology and industry standards to facilitate multiple commercial off-the-shelf devices to communicate with the electronic health record	N/A	8.4 (1.8)
11 (10)	Provides tools for healthcare providers to help automate interactions with patients to facilitate behavior change and attainment of health goals	8.3 (1.9)	8.2 (1.6)
12 (9)	Provides automated alerts (red, yellow, green) for providers to facilitate just-in-time communication for those who are struggling to meet personalized health goals	8.4 (1.5)	8.2 (1.6)
13 (N/A)	Provides searchable data (e.g., “big data”) to facilitate trend analysis	N/A	7.8 (2.7)

N/A indicates a variable added during discussion period by the subject matter expert panel, and initial ranking is not applicable for those features.

Discussion

The objective of this workshop was to gain consensus and to prioritize technology-based interventions to promote physical activity, nutrition, and sleep. During this workshop, experts from academia, industry, and government reviewed 162 technology features, constructs, and leading practices. The review panel assessed features associated with physical activity monitors (n=29), nutrition monitors (n=35), sleep monitors (n=24), incentives for change (n=36), usability and interoperability (n=25), and open data (n=13). Leading practices, gaps, and research needs for technology-based interventions were identified by the review panel. Overall, an integrated solution was recommended (Fig. 1) in which biosensors and digital tools are integrated to provide automated and quantifiable information to assist technology users in meeting their health goals. Personalized messages, training plans, incentives, and tools to foster healthy habits should be incorporated into this integrated solution, and users should be able to share this information with family, friends, wellness coaches, and healthcare providers to facilitate behavior change.

Fig. 1.

Technology has the potential to provide an integrated solution to promote health, optimize physical and cognitive performance, and enhance personal readiness in Service Members and military families. The following components are fundamental to an integrated technology solution: (A) Biosensors, such as accelerometers, altimeters, digital weight scales, heart rate monitors, actigraphs, pulse oximeters, and blood pressure cuffs, and digital tools (e.g., digital food diaries) can be integrated to provide automated and quantifiable information to help users track appropriate outcome metrics. (B) Smartphone and mobile applications integrated with biosensors can enable use of computerized algorithms to support evidence-based recommendations and strategies for individual education, self-monitoring, and behavior change. (C) Technologies can be used to share personal health data with family, friends, and peers via leader boards, competitions, and social media venues at the user's discretion. Such social exchange mechanisms provide avenues for sharing goals, creating competition, and sustaining behavior change. (D) Applications that incorporate appropriate levels of extrinsic and intrinsic rewards can be implemented to facilitate goal achievement. (E) Technologies can be further developed to allow individuals to share their health data with wellness coaches (e.g., community health nurses, health educators, exercise physiologists, strength and conditioning specialists) to assist with online coaching and mentorship. (F) Technologies can be shared with healthcare providers (e.g., primary care providers, sleep experts, physical therapists, psychologists, dietitians), to augment clinical evaluations, prevention, and treatment plans. Technological solutions should provide the ability for active duty members to share appropriate data with military physical fitness, resilience, and organizational leadership (G,H) to better achieve organizational goals aimed at enhancing unit readiness.

Physical Activity Monitoring Technologies

Three of the top five features for physical activity monitoring techniques focused on fundamental activity-measuring capabilities: counting steps, measuring the distance traveled, and determining the time/intensity of activity. There was agreement among the review panel experts that steps walked and distance traveled are meaningful metrics to the typical user; they also serve as appropriate metrics for goal setting and tracking. General agreement was also evident that device algorithms should be robust with a capability to automatically track multiple types of aerobic (e.g., walking, running, aerobics, biking) and lifestyle activities (e.g., housework, yardwork) and to provide online tools to track activities not easily monitored with the existing technology (e.g., strength training, weight lifting).

Identification of major classes of activities (e.g., sitting, walking, sport participation, household chores) has been the focus of recent research efforts using pattern recognition and signal detection.⁵⁷ However, differentiation of less structured physical and lifestyle activities (e.g., occupational activity versus yard work versus household chores) presents a significant technical challenge. Tracking stairs (altimeter), periods of inactivity, and other activities (e.g., balance and stretching) scored lower in relevance. It is interesting that the ability to calculate calories burned was in the bottom 50% of features that experts deemed relevant. Although it is critical to balance caloric intake with energy expenditure, it was noted that direct and accurate measurement of energy expenditure from accelerometer-based monitoring technology remains an unsolved technical challenge.

For accuracy, devices should be worn continuously. Because the military operational environment can be harsh, ruggedness and waterproofing were rated high. A waterproof device helps to ensure that the device does not need to be removed for daily activities involving being submerged in water (e.g., showering, bathing, swimming), which obviates the concern that users might forget to put the device on after such activities.

Accuracy (e.g., reliability and validity) and precision were rated as fundamental requirements of physical activity monitoring devices. A major concern was that construct validity remains unknown as the majority of direct-to-consumer technologies use proprietary methods for quantifying device outputs (e.g., steps, energy expenditure, activity type). In general, the proposal-to-publication cycle times endemic to the research community are not keeping pace with the rapid commercial development of personal monitoring technology. One solution may be to develop and use standard research paradigms for rapidly evaluating multiple technologies simultaneously. This solution also would circumvent any apparent conflict of interest issues and would provide an independent, objective determination of device suitability.

In the future, machine-learning or self-learning capabilities could be developed to improve the accuracy and precision of device algorithms at the user level. For example, after the user runs a set distance (e.g., 2 miles), the system could calibrate the existing algorithms for that individual to facilitate more accurate predictions of distance, intensity, and energy expenditure based on accelerometer input. Different user experience levels (e.g., athletes versus novices) may require different levels of accuracy to track improvements over time. Little is known about the stability of these devices for accurately measuring device outputs over time. Although the majority of the review panel considered that accurate and reliable data were a fundamental requirement, others felt that if the primary goal of the technology is behavior change, a “good enough” accuracy threshold may exist.

Nutrition Monitoring Technologies

Based on evidence indicating the importance of tracking weight for successful weight loss and maintenance,⁸ this feature was ranked as the highest-priority requirement for nutrition and food intake monitoring technologies. Tracking can be accomplished through manual entry or the integration of digital weight scales that upload data to the Web or to mobile applications.

Digital food libraries ranked as the next highest priority, with 7 out of the top 10 features related to digital food libraries. One important feature is credible and easy-to-use food databases that decrease the burden of data entry include the ability to add entries into the food library and allow customization of the library based on frequency of entry of personal meals and/or food choices. It was recognized that all currently available nutrition and food intake monitoring technologies require some form of manual input on the part of the user to track food intake. Perhaps because of this required effort, ease of use was viewed as critical. Potential solutions included barcode scanning and digital pictures to estimate portion size and caloric intake. Developers are currently investigating algorithms for automated analysis of food photographs to calculate portion size and calorie intake. However, this capability remains a significant technical challenge as data processing in these applications is currently largely reliant on human processing. It may be several years before such a solution can be implemented.

A third priority was the capability to integrate an individual's weight, caloric intake, physical activity levels, and personal goals to provide tailored information and caloric prescription. Easy-to-use meal planning tools are critical to facilitating behavior change. Additionally, technology solutions that integrate access to nutrition expertise, guidance, and information were viewed as important. Furthermore, most of the existing nutrition and food intake monitoring technologies do not flag unhealthy behaviors (e.g., eating only 500 calories per day). Thus, incorporation of safeguards against such misuses of nutrition applications should be ensured.

Sleep Monitoring Technologies

Sleep/wake algorithms used in existing actigraphs overestimate sleep and inconsistently identify daytime sleep episodes (e.g., naps).⁵⁸ Five of the top 10 features for sleep technology focused on accuracy (e.g., sleep/wake algorithm accuracy) and validation (e.g., against the gold standard, PSG). Review panel results indicate that sleep monitoring technologies should use PSG-validated algorithms to automatically track sleep without requiring user input (e.g., to indicate “lights out” or “lights on”). These algorithms should also accurately identify both nighttime sleep and naps throughout the day to obtain an accurate measurement of total sleep per 24-h interval. The latter requires that the device (1) automatically detects nonuse or “off-wrist” periods (e.g., time periods when the device is not being worn) and (2) differentiates sleep from waking sedentary behavior. Incorporation of other physiological monitors (e.g., heart rate) or ambient light sensors may help differentiate sleep from wake. Studies documenting the validity of the sleep detection and analysis algorithms should be openly available for review.

Outputs provided by sleep monitoring technologies should be intuitive and provide evidence-based recommendations for improving sleep. In addition to meeting these technological challenges, it was pointed out that, ideally, a defined sleep/wake history should be translated into a meaningful readiness or performance metric. This metric could include translating sleep information into a measure of mental effectiveness, the potential risk for accidents, or an equivalent blood alcohol level. These translations would assist users' understanding of the potential negative impacts of poor sleep.

Incentives to Create and Sustain Changes in Health

Three of the top five features in this area focused on the need for automated reports and information for the end-user. The panels deemed the information should be meaningful, be tailored to the individual, and incorporate health recommendations derived from evidence-based interventions to create and sustain behavior change. Users should be able to select their preferred mode of communication (Web-based, mobile application, text message, e-mail, video, etc.). The information provided should be periodically updated based on data collected from the biosensors and personalized health goals. The panel specified that information shared with the end-user should adjust frequency, timing, and message tone to match individual preferences. For example, one study found that a “prescription” from a physician to use an Internet-based intervention resulted in 65% of patients visiting the site, but adding an e-mail reminder to use the program increased the probability that patients used the site by 45%.⁵⁹ Generally speaking, however, there is limited research guidance about the best timing and types of messaging. Specifically, some individuals may respond positively to more frequent messaging, whereas others may “tune-out” messages if they are provided too frequently. A comprehensive understanding of optimal frequency, timing, and tone of messages was identified as a current research gap.

Online coaching with an expert should be evaluated as an adjunct to the automated and algorithm-based coaching tools. Not all individuals will require the same type and intensity of interventions: some may benefit from less intensive interventions, whereas others may require more intensive interventions.^60,61 For technology-delivered interventions, some users may benefit from automated technologies, but others may need a coach's support to be successful. Scalable solutions such as stepped care may be beneficial when resources are limited by ensuring only the necessary amount of time, expertise, and attention are provided to address an individual's needs.⁶⁰

Although information sharing was ranked higher, 7 of the top 10 features focused on goal setting and tracking. The review panel results indicated that technology should provide both strategies and incentives to help individuals reach personal health goals. Messages should be personalized and adaptable based on an individual's preferences, readiness to change, and pace of goal achievement. Messaging should also integrate leading evidence from behavior change interventions to facilitate goal achievement. Smart logic and algorithms can be an effective means to achieve this integration.

A third theme identified was the importance of incorporating fun (enjoyment) as a fundamental component for behavior change technologies (e.g., gamification, competition, incentives, social support). Leader boards can be used to create competition on shared goals among family, friends, and groups. Tools to create social groups can enhance shared experiences, provide group accountability, and facilitate progress toward a common goal. In the military setting, these social features could facilitate family engagement in positive, healthy behaviors when a member of that family is geographically separated (e.g., deployments, training). Incentive systems that incorporate appropriate extrinsic (e.g., badges, points, awards) and intrinsic (e.g., personal goal achievement, self-efficacy) incentives were considered to be important for both short- and long-term adoption of new, healthier habits.

Usability and Interoperability

The fundamental construct discussed for this area was the importance of decreasing the user burden to implement and maintain interaction with the technology. Conceptually, any technology that is designed to minimize barriers to creating healthy habits should not itself be or create a barrier. For example, entering data into the device must be simple, intuitive, and as automated as possible to decrease time requirements. Also, devices must possess adequate battery life so that users do not have to frequently remove the device to recharge it. Technologies should also be (1) easy to set up, (2) able to be personalized, and (3) intuitive. Data should be automatically uploaded from biosensors to Web- and mobile-based applications or require only a minimum of user input (e.g., Bluetooth^® [Bluetooth SIG, Kirkland, WA], wireless). Compliance issues result when users must remove devices to upload the data, recharge the device battery, or frequently change sensors. For these devices to be incorporated into wellness programs for large organizations, there must be seamless communication across both Web-based and multiple mobile-based platforms to allow the users to access the information using one's own device. Additionally, a device should be produced in a cosmetic form that is comfortable, fashionable, and delivered at a reasonable cost for the target audience.

Mobile Health and Open Data

With regard to mobile health and open data, there was strong agreement on the review panel that most of these features were essential to any future technology solution. Six of the 13 features for mobile health and open data scored 9.0 or greater, and all were above 7.7 points. The highest scoring features related to the ease of moving data between the biosensor/device and software applications and the ease with which data could be used by a variety of third-party applications. Biosensor data should be easily uploadable to different third-party software applications, and software applications should have the capability to upload data from multiple biosensors from different manufacturers. The latter falls under the category of compatibility issues—perhaps best solved by developing industry standards to ensure that software programs allow users to “bring your own device.” Industry standards should also be developed to ensure that data can be loaded into electronic health records.

Ensuring security, privacy, and confidentiality of personal data was identified as important. A multitiered data management system may be required that enables users to grant access and permissions for sharing specific data with designated others, such as family, friends, healthcare providers, and wellness coaches, and with electronic health records. Additionally, users should be able to determine what type of data (e.g., activity, nutrition, weight, sleep) to be shared with each group.

In addition to sharing health data with wellness coaches, healthcare providers, and the electronic health records, tools should enable interactions (automated or otherwise) with users to help them change targeted behaviors (e.g., achieve daily step goal, avoid using the phone or computer in bed) and reach health goals. These interactions could include notifying providers via automated alerts (e.g., red, yellow, green) to facilitate just-in-time communication for those users who are struggling to meet their personalized health goals, as well as alerting users when it may be time to access additional help through a coach or other healthcare provider.

Limitations and Future Directions

The features, constructs, and leading practices reviewed in this workshop focused on elements considered essential for providing a technology solution to the military to enhance health, optimize cognitive and physical performance, and increase individual and military unit readiness. The guiding objective (using the Delphi method) was to identify key features for a technology solution. As outlined in the discussion, identified features tended to be relatively mainstream. Although the review also included features that were more innovative and forward thinking, these features typically scored in the bottom 50% of those reviewed. A spiral development process was assumed, meaning that features that scored lower may be integrated into future device refinement iterations after more research has validated their utility.

As technology solutions mature and are introduced to the market, it becomes increasingly important to address how patient health data is integrated into the clinical workflow and the broader healthcare system. Health technology developers need to determine how data sharing will enable a comprehensive, 360° view of a patient's health. Healthcare providers, navigators, and health coaches need to determine how to use this unprecedented, data-driven perspective to encourage patients to better manage their own health and health goals. A potential framework for healthcare data integration is described in Figure 1. Development of complementary frameworks, as well as in-depth technical, regulatory, policy-related aspects of health data integration, is a priority focus of future work.

Conclusions

Analysis and prioritization of leading practices, features, and constructs can be used to provide a research and development road map for (1) leveraging technology to minimize barriers to enhancing health and (2) facilitating evidence-based techniques to create and sustain healthy behaviors. Future research and development should focus on an integrated solution to promote health, optimize physical and cognitive performance, and enhance personal readiness in service members and military families. Capability gaps identified and insights gained can also be used to support future Broad Agency Announcements, Program Announcements, and Small Business Innovation Research topics.

Footnotes

Acknowledgments

The authors are grateful for the contribution of workshop participants from academic, government, and commercial industries (see ). The authors would also like to acknowledge the following for reviewing this manuscript: MAJ Mark Mellott, Performance Triad, U.S. Army Medical Command; Dr. Janet Harris, JPC-1 Medical Training and Health Information Systems, U.S. Army Medical Research and Materiel Command; and COL Daniel Kral and LTC Felicia Langel, Telemedicine and Advanced Technology Research Center, U.S. Army Medical Research and Materiel Command.

Disclosure Statement

F.P.T. has equity ownership in BeHealth Solutions, LLC, a company developing and disseminating online interventions. The terms of this arrangement have been reviewed and approved by the University of Virginia in accordance with its conflict of interest policy. M.A. is an employee of Booz Allen Hamilton. D.N. is an employee of Empirica Research. D.S.T., E.E., J.D.G., A.H., J.K., K.J.K., B.L., J.M., T.S., V.T., N.W., and D.J.P. declare no competing financial interests exist.

Appendix: Workshop Participants

References

U.S. Army Public Health Command Public Affairs Office. “Performance Triad” to change focus of Army medicine. OneHealth, 2012; 3:3–4.

Bennett

, Herring

, Puleo

, Stein

, Emmons

, Gillman

. Web-based weight loss in primary care: A randomized controlled trial. Obesity, 2010; 18:308–313.

Webb

, Joseph

, Yardley

, Michie

. Using the internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res, 2010; 12:e4.

Kennedy

, Powell

, Payne

, Ainsworth

, Boyd

, Buchan

. Active assistance technology for health-related behavior change: An interdisciplinary review. J Med Internet Res, 2012; 14:e80.

Chen

, Janz

, Zhu

, Brychta

. Redefining the roles of sensors in objective physical activity monitoring. Med Sci Sports Exerc, 2012; 44(1 Suppl 1):S13–S23.

Freak-Poli

RLA

, Cumpston

, Peeters

, Clemes

. Workplace pedometer interventions for increasing physical activity. Cochrane Database Syst Rev, 2013; 4:CD009209.

Korkiakangas

, Alahuhta

, Husman

, Keinänen-Kiukaanniemi

, Taanila

, Laitinen

. Pedometer use among adults at high risk of type 2 diabetes, Finland, 2007–2008. Prev Chronic Dis, 2010; 7:A37.

Bravata

, Smith-Spangler

, Sundaram

, Gienger

, Lin

, Lewis

, et al. Using pedometers to increase physical activity and improve health: A systematic review. JAMA, 2007; 298:2296–2304.

Guare

, Wing

, Marcus

, Epstein

, Burton

, Gooding

. Analysis of changes in eating behavior and weight loss in type II diabetic patients. Which behaviors to change. Diabetes Care, 1989; 12:500–503.

10.

Wing

, Hill

. Successful weight loss maintenance. Annu Rev Nutr, 2001; 21:323–341.

11.

Krukowski

, Harvey-Berino

, Bursac

, Ashikaga

, West

. Patterns of success: Online self-monitoring in a web-based behavioral weight control program. Health Psychol, 2013; 32:164–170.

12.

Arem

, Irwin

. A review of web-based weight loss interventions in adults. Obes Rev, 2011; 12:e236–e243.

13.

Acharya

, Elci

, Sereika

, Styn

, Burke

. Using a personal digital assistant for self-monitoring influences diet quality in comparison to a standard paper record among overweight/obese adults. J Am Diet Assoc, 2011; 111:583–588.

14.

Tate

, Wing

, Winett

. Using Internet technology to deliver a behavioral weight loss program. JAMA, 2001; 285:1172–1177.

15.

Turner-McGrievy

, Tate

. Tweets, apps, and pods: Results of the 6-month Mobile Pounds Off Digitally (Mobile POD) randomized weight-loss intervention among adults. J Med Internet Res, 2011; 13:e120.

16.

Burke

, Wang

, Sevick

. Self-monitoring in weight loss: A systematic review of the literature. J Am Diet Assoc, 2011; 111:92–102.

17.

Fry

, Neff

. Periodic prompts and reminders in health promotion and health behavior interventions: Systematic review. J Med Internet Res, 2009; 11:e16.

18.

Klasnja

, Pratt

. Healthcare in the pocket: Mapping the space of mobile-phone health interventions. J Biomed Inform, 2012; 45:184–198.

19.

Wing

, Crane

, Thomas

, Kumar

, Weinberg

. Improving weight loss outcomes of community interventions by incorporating behavioral strategies. Am J Public Health, 2010; 100:2513–2519.

20.

Stewart

, Han

, Allen

, Bathalon

, Ryan

, Newton

, Williamson

. H.E.A.L.T.H.: Efficacy of an internet/population-based behavioral weight management program for the U.S. Army. J Diabetes Sci Technol, 2011; 5:178–187.

21.

Bootzin

, Friedman

. The assessment of insomnia. Behav Assess, 1981; 3:107–126.

22.

Buysse

, Ancoli-Israel

, Edinger

, Lichstein

, Morin

. Recommendations for a standard research assessment of insomnia. Sleep, 2006; 29:1155–1173.

23.

Carney

, Buysse

, Ancoli-Israel

, Edinger

, Krystal

, Lichstein

, et al. The consensus sleep diary: Standardizing prospective sleep self-monitoring. Sleep, 2012; 35:287–302.

24.

Prestwich

, Rankin

, Housman

. Tracking sleep times to reduce tiredness and improve sleep in college students. Californian J Health Promot, 2007; 5:148–156.

25.

Kushner

, Kessler

, McGaghie

. Using behavior change plans to improve medical student self-care. Acad Med, 2011; 86:901–906.

26.

Short

, Gradisar

, Lack

, Wright

, Carskadon

. The discrepancy between actigraphic and sleep diary measures of sleep in adolescents. Sleep Med, 2012; 13:378–384.

27.

Rechtschaffen

, Kales

. A manual of standardized terminology, techniques and scoring systems for sleep stages of human subjects. Washington, DC: U.S. Public Health Service, 1968.

28.

American Academy of Sleep Medicine. International classification of sleep disorders: Diagnostic and coding manual. Rochester, MN: American Academy of Sleep Medicine, 2005.

29.

Van de Water

ATM

, Holmes

, Hurley

. Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography—A systematic review. J Sleep Res, 2011; 20:183–200.

30.

Morgenthaler

, Alessi

, Friedman

, Owens

, Kapur

, Boehlecke

, et al. Practice parameters for the use of actigraphy in the assessment of sleep and sleep disorders: An update for 2007. Sleep, 2007; 30:519–529.

31.

Martin

, Hakim

. Wrist actigraphy. Chest, 2011; 139:1514–27.

32.

Shirazi

, Clawson

, Hassanpour

, Tourian

, Schmidt

, Chi

, et al. Already up? Using mobile phones to track & share sleep behavior. Int J Hum Comput Stud, 2013; 71:878–888.

33.

Ancoli-Israel

, Cole

, Alessi

, Chambers

, Moorcroft

, Pollak

. The role of actigraphy in the study of sleep and circadian rhythms. Sleep, 2003; 26:342–392.

34.

De Souza

, Benedito-Silva

, Pires

MLN

, Poyares

, Tufik

, Calil

. Further validation of actigraphy for sleep studies. Sleep, 2003; 26:81–85.

35.

Konetes

. The function of intrinsic and extrinsic motivation in educational virtual games and simulations. J Emerg Tech Web Intell, 2010; 2:1–26.

36.

Deci

, Koestner

, Ryan

. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull, 1999; 125:627–668; discussion 692–700.

37.

Gubler

, Larkin

, Pierce

. The dirty laundry of employee award programs: Evidence from the field. NOM Unit Working Paper Number 13-069. Cambridge, MA: Harvard Business School, 2013.

38.

Curry

, Wagner

, Grothaus

. Evaluation of intrinsic and extrinsic motivation interventions with a self-help smoking cessation program. J Consult Clin Psychol, 1991; 59:318–324.

39.

Williams

, Gagné

, Ryan

, Deci

. Facilitating autonomous motivation for smoking cessation. Health Psychol, 2002; 21:40–50.

40.

Wood

, Quinn

, Kashy

. Habits in everyday life: Thought, emotion, and action. J Pers Soc Psychol, 2002; 83:1281–1297.

41.

Wood

, Neal

. A new look at habits and the habit-goal interface. Psychol Rev, 2007; 114:843–863.

42.

Van't Riet

, Sijtsema

, Dagevos

, De Bruijn

G-J

. The importance of habits in eating behaviour. An overview and recommendations for future research. Appetite, 2011; 57:585–596.

43.

Webb

, Sheeran

. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol Bull, 2006; 132:249–268.

44.

Verplanken

, Wood

. Interventions to break and create consumer habits. J Public Pol Mark, 2006; 25:90–103.

45.

Verplanken

, Walker

, Davis

, Jurasek

. Context change and travel mode choice: Combining the habit discontinuity and self-activation hypotheses. J Environ Psychol, 2008; 28:121–127.

46.

Neal

, Wood

, Wu

, Kurlander

. The pull of the past: When do habits persist despite conflict with motives?. Pers Soc Psychol Bull, 2011; 37:1428–1437.

47.

Hilgart

, Ritterband

, Thorndike

, Kinzie

. Using instructional design process to improve design and development of internet interventions. J Med Internet Res, 2012; 14:e89.

48.

Hyers

. Global smartphone installed based forecast by operating system for 88 countries: 2007 to 2017. Available at: www.strategyanalytics.com/default.aspx?mod=reportabstractviewer&a0=8558 (last accessed June 15, 2014 ).

49.

research2guidence. Mobile health market report 2013–2017: The commercialization of mhealth organizations, Vol. 3. Berlin: research2guidance, 2013:116.

50.

The Organisation for Economic Co-operation and Development. Health at a glance 2011: OECD indicators. Paris: Organisation for Economic Co-operation and Development, 2011:200.

51.

Brown

. Delphi process. Santa Monica, CA: RAND Corporation, 1968:1–15.

52.

Hasson

, Keeney

, McKenna

. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000; 32:1008–1015.

53.

Skulmoski

, Hartman

, Krahn

. The Delphi method for graduate research. J Inform Technol Educ, 2007; 6:1–21.

54.

Hsu

C-C

, Sandford

. The Delphi technique: Making sense of concensus. Pract Asses Res Eval, 2007; 12(10):1–8.

55.

Teyhen

, Aldag

, Centola

, Edinborough

, Ghannadian

, Haught

, et al. Key enablers to facilitate healthy behavior change: Workshop summary. J Orthop Sports Phys Ther, 2014; 44:378–387.

56.

Teyhen

, Aldag

, Centola

, Edinborough

, Ghannadian

, Haught

, et al. Incentives to create and sustain healthy behaviors: Technology solutions and research needs. Mil Med, 2013 (in press).

57.

Staudenmayer

, Pober

, Crouter

, Bassett

, Freedson

. An artificial neural network to estimate physical activity energy expenditure and identify physical activity type from an accelerometer. J Appl Physiol, 2009; 107:1300–1307.

58.

Kanady

, Drummond

SPA

, Mednick

. Actigraphic assessment of a polysomnographic-recorded nap: A validation study. J Sleep Res, 2011; 20:214–222.

59.

Ritterband

, Borowitz

, Cox

, Kovatchev

, Walker

, Lucas

, et al. Using the internet to provide information prescriptions. Pediatrics, 2005; 116:e643–e647.

60.

Haaga

. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol, 2000; 68:547–548.

61.

Espie

. “Stepped care”: A health technology solution for delivering cognitive behavioral therapy as a first line insomnia treatment. Sleep, 2009; 32:1549–1558.