The application of mHealth to monitor implementation of best practices to support healthy eating and physical activity in afterschool programs

Abstract

Background:

Childhood obesity continues to be a global epidemic and many child-based settings (e.g. school, afterschool programs) have great potential to make a positive impact on children’s health behaviors. Innovative and time-sensitive methods of gathering health behavior information for the purpose of evaluation and strategically deploying support are needed in these settings.

Purpose:

The aim is to (1) demonstrate the feasibility of mobile health (mHealth) for monitoring implementation of healthy eating and physical activity (HEPA) standards and, (2) illustrate the utility of mHealth for identifying areas where support is needed, within the afterschool setting.

Methods:

Site leaders (N = 175) of afterschool programs (ASPs) were invited to complete an online observation checklist via a mobile web app (Healthy Eating and Physical Activity Mobile, HEPAm) once per week during ASP operating hours. Auto-generated weekly text reminders were sent to site leaders’ mobile devices during spring and fall 2015 and 2016 and spring 2017 school semesters. Data from HEPAm was separated into HEPA variables, and expressed as a percent of checklists where an item was present. A higher percentage for a given item would indicate an afterschool has higher compliance with current HEPA standards.

Results:

A total of 141 site leaders of ASPs completed 13,960 HEPAm checklists. The average number of checklists completed per ASP was 43 (range 1–220) for healthy eating and 50 (range 1–230) for physical activity. For healthy eating, the most common challenge for ASPs was ‘Staff educating children about healthy eating’, and for physical activity checklists, ‘Girls only physical activity is provided at ASP’.

Conclusion:

HEPAm was widely used and provided valuable information that can be used to strategically deploy HEPA support to ASPs. This study gives confidence to the adoption of mHealth strategies as a means for public health practitioners to monitor compliance of an initiative or intervention.

Keywords

children public health school setting physical activity health promotion obesity/overweight

Introduction

Mobile health (mHealth) is the use of mobile phones, software applications, and portable devices to facilitate the accomplishment of health objectives (1). The increasing use of mHealth within the wider healthcare and public health communities is attributed to its ability to identify patient needs among hard to reach communities, allow patients more access to healthcare due to reduced time and expense of travel, and allow community health workers to assess compliance with interventions and public health guidelines (2 –4).

One area where mHealth can hold considerable utility is its use in large-scale dissemination and implementation research where often the individuals and/or settings involved are geographically dispersed. Given the costs associated with sending people to conduct on-site visits to evaluate implementation, public health interventions and initiatives across geographically dispersed individuals/settings require the use of tools (e.g. mobile phones, text messaging) that are low cost, widely accessible, useful for lower literacy audiences, and can provide real-time feedback to the user (5,6). Theoretically, using an mHealth approach within dissemination and implementation research could serve as a low cost option to track and monitor implementation of a program, intervention, or set of policy standards. The application of mHealth, therefore, should help to reduce the geographic and economic constraints associated with the scaling up of public health interventions or initiatives. In addition, capturing information in this way would allow public health practitioners at a centralized location to use this information to identify patterns in responses to strategically deploy the necessary support to end-users who may be struggling with implementation, while minimizing unnecessary visits to locations with a high level of implementation.

Global rates of child overweight and obesity are at an all-time high, placing childhood obesity at the forefront of public health concern (7). Understandably, the majority of research investigating and intervening upon obesogenic behaviors, such as healthy eating and physical activity (HEPA), have focused on settings where children spend a large portion of their time, such as in schools and outside-of-school programs (e.g. afterschool programs, clubs, and camps) (8). Afterschool programs (ASPs) have the potential to influence the HEPA of more than 10 million USA children (ages 5–13 years) attending yearly (9). Recognizing this potential, various associations and leading youth development organizations (e.g. National Afterschool Association, YMCA of the USA) have adopted standards pertaining to daily HEPA in ASPs. For example, an ASP must provide 30 min for physical activity and serve a fruit or vegetable, daily (10). Currently, ASPs struggle to meet HEPA benchmarks (11 –13). Thus, support for achieving the HEPA standards is needed, yet the number of ASPs and their geographical separation necessitates this to be low-cost.

Theoretically, mHealth strategies would allow for real-time identification of HEPA information, and provide public health practitioners and support agencies with specific information related to HEPA promotion straight from those individuals who are responsible for the day-to-day operations of the ASP; commonly referred to as ‘site leaders’ (14). Targeted approaches to assist ASPs in achieving HEPA standards that are tailored, cost-effective, and timely can be developed and delivered based on the information received from ASPs. Researchers recognize the potential of mHealth approaches as a tool to prevent or treat childhood obesity (15 –17), yet, this remains a relatively unexplored approach, particularly in monitoring and evaluating policies that may impact weight-related aspects of child-care environments (18). The purpose of this study is to (1) demonstrate the feasibility of employing an mHealth approach to monitor implementation of policy standards targeting HEPA, and to (2) illustrate the utility of how the information can be used to identify HEPA areas of support, within an ASP setting.

Methods

About HEPAm

An interdisciplinary team consisting of computer scientists, public health experts, graphic designers, and ASP site leaders (i.e. end-users) were involved in the design of a mobile web app conceived as Healthy Eating and Physical Activity Mobile (HEPAm). The HEPAm mobile web app was designed to gather information pertaining to an ASPs’ daily HEPA practices. There are two main content areas – ‘healthy eating’ and ‘physical activity’ – with separate observation checklists for each composed of ‘fill in the blank’, ‘select all’, and ‘yes/no’ response scale items for healthy eating (total of 29 items) and physical activity (total of 16 items). HEPAm is intended to be completed during program time on a respondent’s mobile device (i.e. smartphone or tablet) connected to the internet. All items on the observation checklists either collect contextual information regarding the ASP (e.g. grade level of children being observed) or align with existing HEPA standards and best practices for achieving HEPA standards in ASPs (19,20). Figure 1 illustrates the HEPAm user interface. Prior to launch, several rounds of beta testing were conducted with internal (public health experts) and external users (site leaders of ASPs) to assess initial acceptance, usability, and feasibility, during the month of January 2015.

Figure 1.

HEPAm user interface showing a (a) login page, (b) checklist option, (c) snack observation checklist, and (d) physical activity checklist.

Study overview

ASPs located in a southeastern state of the United States of America (USA) were recruited through several large organizations committed to improving HEPA across all of their programs. A total of 175 site leaders representing 175 ASPs were invited to register for HEPAm between spring 2015 and 2017. Site leaders were asked to complete HEPAm at least once per week (i.e. fill out one healthy eating and one physical activity checklist) during their programs operating hours. After initial registration with HEPAm, auto-generated weekly text reminders were sent to program leaders’ mobile devices immediately before the start of the program every Monday during the 2015 and 2016 spring and fall and 2017 spring school semesters. All methods were approved by the University of South Carolina’s institutional review board.

Data analysis

Data from the HEPAm checklists were downloaded and expressed as a percent of checklists where an item was present for each individual user (Table 1). The HEPA standards call for ASPs to achieve daily compliance that would be indicated by scoring 100% on HEPAm checklist items (13). A higher percentage for a HEPAm checklist item would indicate an ASP had higher compliance with the HEPA standards. For example, if a site leader observed a fruit being served for snack on only one of the days they completed HEPAm – but completed three HEPAm checklists overall – then the HEPA compliance score given for that specific item would be 33%. Each completed HEPAm checklist had a timestamp that allowed for the investigation of whether end-users were completing the checklists during typical ASP operating hours (~2–6 p.m.) (21). Data were examined by identifying the ASPs reporting 0% HEPA compliance with any of the HEPAm checklist items. Data were analyzed summer 2017 using Stata (v.14.1, College Station, TX).

Table 1.

Identification of healthy eating and physical activity (HEPA) areas where support is needed in ASPs from HEPAm data.

Level of demand on resources	Support	HEPA checklist items	% of HEPAm checklists^b		SD (±)	Range	ASPs reporting 0% HEPA compliance^c
Level of demand on resources	Support	HEPA checklist items	Mean	Median	SD (±)	Range	#	%
Healthy eating
High	Email/ telephone/site visit/training	Serve water for snack	61	75	41	0–100	23	21.1
		Serve a fruit or vegetable for snack	57	63	39	0–100	15	13.8
		Did not serve sugar sweetened beverages for snack	82	100	28	0–100	3	2.8
Medium	Email/ telephone	Staff did not drink inappropriate drinks	97	100	8	67–100	0	0.0
		Staff did not eat inappropriate foods	99	100	4	67–100	0	0.0
		Staff sitting/eating snack with children	30	85	30	0–100	6	5.5
Low	Email	Staff educating children about healthy eating	14	0	25	0–100	89	63.3
		Staff encouraging children to eat healthy foods	48	50	35	0–100	22	20.2
		Snack menu present at ASP	76	100	36	0–100	16	14.7
		Snack menu posted for parents at ASP	61	81	42	0–100	26	23.9
Physical activity (PA)
High	Email/ Telephone/Site Visit/Training	Children do not wait in lines for PA	87	100	21	0–100	2	1.9
		Children are not eliminated from PA	89	100	17	20–100	0	0.0
		PA with small team sizes	86	100	22	0–100	2	1.9
		Girls only PA is provided at ASP	3	0	12	0–100	122	86.7
Medium	Email/ Telephone	PA schedule followed	79	91	27	0–100	4	3.8
		Staff engaged with children during PA	72	80	30	0–100	7	6.7
		Staff encouraging children during PA	74	78	26	0–100	3	2.9
Low	Email	PA schedule present at ASP	83	95	23	0–100	2	1.9
		PA schedule posted for parents at ASP	60	75	39	0–100	22	21.0
		Staff not withholding PA from children	96	100	15	0–100	2	1.9
		Staff wearing appropriate clothing for PA	89	100	22	0–100	1	1.0

Based on ASPs operating hours of 2:00–6:00 p.m.

Higher percentage indicates greater HEPA compliance.

Based on 141 ASPs.

ASPs: afterschool programs; HEPA: healthy eating and physical activity; PA: physical activity.

Lastly, the level of demand placed on the public health practitioners to help ASPs meet HEPA standards was assigned to HEPAm checklist items by categorizing each checklist item as low, medium, or high demand on public health practitioner resources (Table 1). Support was classified into the following three categories: email communication (low demand on resources), email and telephone communication (medium demand on resources), and email, telephone, and site visit/training (high demand on resources). The creation of the three categories and the assignment of HEPA checklist items were established on cost analysis information from previous published research projects in large diverse samples of ASPs across the southeastern state, and is representative of the monetary investment required to provide the relevant level of support (14,22 –26). For example, these multi-year large-scale randomized controlled trials intervening on HEPA outcomes required trained research assistants to visit, evaluate, train, and provide feedback to staff on HEPA outcomes. The demand on the resources required to do so was quantified and categorized into different levels (low, medium, high) and, thus, provides a real-world accurate representation of the level of demand on resources to categorize HEPA checklist items in this study.

Results

Feasibility

A total of 141 unique ASPs registered for HEPAm out of the 175 ASPs that were contacted representing ~80% of potential users in the southeastern state of the USA. The total number of checklists completed between February 2015 and May 2017 was 13,960. Of these total checklists, 6842 were healthy eating (ASP mean = 43, range = 1–220) and 7118 were physical activity (ASP mean = 50, range 1–230). Users registered for HEPAm at different times between spring 2015 and 2017, however, approximately 86% of HEPAm users completed a HE and a PA checklist at least once per school week. Over two thirds of the checklists (~9350) were completed during typical ASP operating hours (~2–6 p.m.) by HEPAm users.

Utility

HEPA checklist items (N = 21) directly aligning with the HEPA standards are presented in Table 1. Across all ASPs, no site leader reported 100% HEPA compliance for all of the HEPAm checklist items. Overall, the number of ASPs reporting 0% HEPA compliance for any given HEPAm checklist item ranged from 0 to 91 ASPs. For healthy eating checklist items, the most common challenge and area requiring support represented 89 out of 141 ASPs (63.3%) for ‘Staff educating children about healthy eating’. For physical activity checklist items, the most common challenge and area requiring support represented 122 out of the 141 ASPs (86.7%) for ‘Girls only physical activity is provided at ASP’.

Discussion

This study demonstrated the feasibility and utility of collecting information on HEPA practices in a large sample of ASPs via a mobile web app. From the information provided by HEPAm users (i.e. site leaders), specific areas where support was needed was identified along with the level of demand on resources required to address HEPA promotion within this area. Adopting this unique approach to this public health setting has several advantages including (1) site leaders being well-positioned to self-monitor the HEPA practices of their program, (2) the use of mobile technology helps to alleviate geographical barriers related to collecting real-time information, (3) public health practitioners can continuously monitor compliance of multiple ASPs’ with the HEPA standards, and (4) public health practitioners can use the information to strategically allocate support to ASPs as needed, avoiding unnecessary costs often associated with employing a ‘one size fits all’ approach (27).

In broader terms, this study illustrates the potential of mHealth to give public health practitioners the ability to receive real-time, constant feedback and evaluate implementation of an initiative or intervention (i.e. meeting HEPA standards) of a large number of users (i.e. >140 ASPs) from a centralized location. Similar to other studies adopting an mHealth approach (28), this study did not employ any existing standardized or validated measurement tools or methods to assess feasibility, however, the percent of potential users who registered and completed HEPAm checklists (>80%) and the percent of registered users (>85%) who completed at least one HEPAm checklist (>85%) establishes confidence in the degree of feasibility of adopting this approach in this setting, and is in accordance with other studies incorporating mHealth approaches reporting weekly user compliance (29).

Furthermore, utilizing mHealth to surveil and identify areas where support is needed is a method that can allow effective implementations and the opportunity to scale up. In the case of larger dissemination/implementation studies this is a noteworthy advantage, as broadening the amount of users by gaining access to hard-to-reach or new populations makes research more generalizable (30). As stated previously, mHealth can allow for the continued monitoring of compliance in several instances such as intervention implementation or adherence (2). In this illustration of utility, using a mHealth approach not only gives public health practitioners the ability to examine and monitor patterns in data but empowers the practitioners (e.g. site leaders) with a self-monitoring feedback tool that can be used to monitor compliance with organization/industry standards (e.g. HEPA Standards). Further, public health practitioners have the ability to examine the information on a program-by-program basis or at an organizational level (e.g. multiple ASPs operate within a given geographical area) with provision for strategic and data-driven deployment of resources.

The data collected in this study can be used to illustrate the utility of HEPAm as a means to identify and evaluate HEPA progress, and strategically deploy support to an ASP as needed. For example, the majority of ASPs struggled to implement healthy eating education at their programs (89 of the 141 ASPs reported 0% HEPA compliance). Here, any monetary investment needed from the public health practitioner is considered ‘low’. Simple email communication between the site leader and public health practitioners to provide new or different healthy eating education resources and strategies (e.g. nutrition trivia sheets, fruit and vegetable coloring templates, fun facts during snack times) would be an appropriate action. Conversely, ASPs in this study struggled to provide a girls-only physical activity at their program (87% of ASPs reported 0% HEPA compliance). In this instance, the support and monetary investment is considered ‘high’, most likely in the form of a site visit from qualified personnel to observe current physical activity promotion practices and subsequently train ASP staff on best practices/strategies for incorporating a girls only physical activity opportunity into their ASP. Additionally, site visits to the ASP would be supplemented with continuous follow-up phone calls and email communication between the site leaders and public health practitioner(s). Thus, collecting data using mHealth methods allows for continuous, up-to-date monitoring of HEPA compliance across several ASPs and gives public health practitioners the ability to evaluate progress and tailor deployment of resources to address any area of concern.

Strengths of this study include the novel approach of using mHealth to monitor compliance with a set of standards in a setting that cares for children and the capacity of mHealth to capture real-time continuous data across a large sample of users. In the specifics of this illustration, no site leaders reported 100% compliance with HEPA standards across all items, giving a degree of credibility to the information reported by the users. The large sample of ASPs (n > 140) that registered and completed over 13,000 checklists also demonstrates the feasibility and utility of employing mHealth in this particular setting. This study is limited in its generalizability beyond the ASP setting, although the adoption and utility of such an approach in other settings with similar intentions (e.g. continuous monitoring of compliance to initiatives or interventions; identifying where support is required) is not expected to produce widely disparate conclusions in terms of utility. This study is lacking any formal measure regarding the reliability of the reported data and any formal qualitative information (e.g. high users versus low users) from the end-users’ (i.e. site leaders) perspective regarding the burden of using mobile technology during ASP time which would help assess feasibility in more detail.

In summary, this study demonstrated the utility and feasibility of employing an mHealth approach to monitor compliance and identify areas where support is required in a public health setting. Specifically, users demonstrated that HEPAm is a feasible approach to collecting large volumes of information on how ASPs are performing in relation to an initiative or intervention – in this case, the HEPA standards. When evaluating compliance with policies or standards, HEPAm should not be viewed as a replacement to on-the-ground evaluation by appropriately trained personnel, but serve as a bridge between policies or standards and practice by enhancing the ability to identify where programs struggle and deliver appropriate support to a large number of geographically-dispersed users. Neighboring public health disciplines have demonstrated success through the use of mHealth strategies to facilitate the creation of professional networks, by which real-time advice, information, and support can be provided (2). This study gives confidence to the adoption of mHealth strategies as a means for public health practitioners to monitor compliance of an initiative or intervention and provide tailored, intentional support to a large number of geographically dispersed users.

Footnotes

Declaration of conflicting interests

None declared.

Funding

Research reported in this publication was supported by the National Heart, Lung, And Blood Institute of the National Institutes of Health under Award Number R01HL112787. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

ORCID iD

Keith Brazendale

Justin B. Moore

References

Olla

Shimskey

. mHealth taxonomy: a literature survey of mobile health applications. Health Technol. 2015; 4: 299–308.

Braun

Catalani

Wimbush

Israelski

. Community health workers and mobile technology: a systematic review of the literature. PLoS One. 2013; 8: e65772.

Stinson

Burley

BSpThy

Briggs

. End user and implementer experiences of mhealth technologies for noncommunicable chronic disease management in young adults: systematic review. J Med Internet Res. 2017; 19: e406.

Chen

Chai

Dong

Niu

Zhang

. Effectiveness and appropriateness of mhealth interventions for maternal and child health: systematic review. JMIR mHealth and uHealth. 2018; 6: e7.

Fox

Duggan

. Mobile Health 2012. Washington, DC: Pew Internet & American Life Project; 2012.

Turner-McGrievy

Hales

Schoffman

Valafar

Brazendale

Weaver

, et al. Choosing between responsive-design websites versus mobile apps for your mobile behavioral intervention: presenting four case studies. Transl Beh Med. 2016; 7(2): 224–232.

World Health Organization. Global Strategy on Diet, Physical Activity and Health [Internet]. WHO; 2016 [cited 2017 Aug 3] Available from: http://www.who.int/dietphysicalactivity/childhood_why/en/.

Ross

Flynn

Pate

. What is really causing the obesity epidemic? A review of reviews in children and adults. J Sports Sci. 2016; 34: 1148–1153.

Alliance

. America After 3PM: Afterschool Programs in Demand. Washington, DC: Afterschool Alliance; 2014.

10.

Beets

Weaver

Turner-McGrievy

Moore

Webster

Brazendale

, et al. Are we there yet? Compliance with physical activity standards in YMCA afterschool programs. Child Obes. 2016; 12: 237–246.

11.

Beets

Rooney

Tilley

Beighle

Webster

. Evaluation of policies to promote physical activity in afterschool programs: are we meeting current benchmarks? Prev Med. 2010; 51: 299–301.

12.

Trost

Rosenkranz

Dzewaltowski

. Physical activity levels among children attending after-school programs. Med Sci Sports Exerc. 2008; 40: 622.

13.

Beets

Weaver

Turner-McGrievy

Huberty

Ward

Freedman

, et al. Making healthy eating and physical activity policy practice: the design and overview of a group randomized controlled trial in afterschool programs. Contemp Clin Trials. 2014; 38: 291–303.

14.

Beets

Weaver

Turner-McGrievy

Huberty

Ward

Pate

, et al. Making policy practice in afterschool programs: a randomized controlled trial on physical activity changes. Am J Prev Med. 2015; 48: 694–706.

15.

Turner

Spruijt-Metz

Wen

Hingle

. Prevention and treatment of pediatric obesity using mobile and wireless technologies: a systematic review. Pediatr Obes. 2015; 10: 403–409.

16.

Nezami

Ward

Lytle

Ennett

Tate

A mHealth randomized controlled trial to reduce sugar-sweetened beverage intake in preschool-aged children. Pediatr Obes. Epub ahead of print 8 November 2017. DOI: 10.1111/ijpo.12258.

17.

Chen

Lieffers

Bauman

Hanning

Allman-Farinelli

. The use of smartphone health apps and other mobile health (mHealth) technologies in dietetic practice: a three country study. J Hum Nutr Diet. 2017; 30: 439–452.

18.

Larson

Looby

Frost

Nanney

Story

. What can be learned from existing investigations of weight-related practices and policies with the potential to impact disparities in US child-care settings? A narrative review and call for surveillance and evaluation efforts. J Acad Nutr Diet. 2017; 117: 1554–1577.

19.

Beets

Tilley

Kim

Webster

. Nutritional policies and standards for snacks served in after-school programmes: a review. Public Health Nutr. 2011; 14: 1882–1890.

20.

Beets

Wallner

Beighle

. Defining standards and policies for promoting physical activity in afterschool programs. J School Health. 2010; 80: 411–417.

21.

Brazendale

Beets

Weaver

Huberty

Beighle

Pate

. Wasting our time? Allocated versus accumulated physical activity in afterschool programs. J Phys Act Health. 2015; 12: 1061–1065.

22.

Beets

Weaver

Turner-McGrievy

Huberty

Ward

Freedman

, et al. Making healthy eating policy practice: a group randomized controlled trial on changes in snack quality, costs, and consumption in after-school programs. Am J Health Promot. 2015; 30: 521–531.

23.

Beets

Weaver

Tilley

Turner-McGrievy

Huberty

Ward

, et al. Salty or sweet? Nutritional quality, consumption, and cost of snacks served in afterschool programs. J School Health. 2015; 85: 118–124.

24.

Beets

Weaver

Moore

Turner-McGrievy

Pate

Webster

, et al. From policy to practice: strategies to meet physical activity standards in YMCA afterschool programs. Am J Prev Med. 2014; 46: 281–288.

25.

Weaver

Beets

Beighle

Webster

Huberty

Moore

. Strategies to increase after-school program staff skills to promote healthy eating and physical Activity. Health Promot Pract. 2016; 17: 88–97.

26.

Beets

Weaver

Turner-McGrievy

Saunders

Webster

Moore

, et al. Evaluation of a statewide dissemination and implementation of physical activity intervention in afterschool programs: a nonrandomized trial. Transl Behav Med. 2017; 7: 690–701.

27.

Kreuter

Strecher

Glassman

. One size does not fit all: the case for tailoring print materials. Ann Behav Med 1999; 21: 276–283.

28.

Al Ayubi

Parmanto

Branch

Ding

. A persuasive and social mHealth application for physical activity: a usability and feasibility study. JMIR mHealth uHealth. 2014; 2: e25.

29.

Gabrielli

Dianti

Maimone

Betta

Filippi

Ghezzi

, et al. Design of a mobile app for nutrition education (TreC-LifeStyle) and formative evaluation with families of overweight children. JMIR mHealth uHealth. 2017; 5: e48.

30.

Kumar

Nilsen

Abernethy

Atienza

Patrick

Pavel

, et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med. 2013; 45: 228–236.