Retention and engagement of rural caregivers of adolescents in a short message service intervention to reduce sugar-sweetened beverage intake

Abstract

Objective

This study investigates a 6-month short message service (SMS) intervention to reduce adolescent sugar-sweetened beverage (SSB) intake. The objectives are to describe caregiver retention and SMS engagement as well as explore differences by caregiver characteristics.

Methods

Caregivers completed a baseline survey then messages were sent two times per week. Message types included the following: SSB intake assessments, educational information, infographic URLs, and strategies. Engagement was measured through interaction with these messages and included: assessment completion, reminders needed, number of strategies chosen, and URLs clicked.

Results

Caregivers (n = 357) had an average baseline SSB intake of 23.9 (SD = 26.8) oz/day. Of those, 89% were retained. Caregivers with a greater income and education were retained at a higher rate. Average engagement included: 4.1 (SD = 1.3) of 5 assessments completed with few reminders needed [4.1 (SD = 3.7) of 14 possible], 3.2 (SD = 1.1) of 4 strategies selected, and 1.2 (SD = 1.6) of 5 URLs clicked. Overall, average engagement was relatively high, even where disparities were found. Demographic characteristics that were statistically related to lower engagement included younger age, lower income, lower educational attainment, single caregivers, lower health literacy. Furthermore, caregivers with a reduced intention to change SSB behaviors completed fewer assessments and needed more reminders. Higher baseline SSB intake was associated with lower engagement across all indicators except URL clicks.

Conclusions

Results can be used to develop targeted retention and engagement strategies (e.g., just-in-time and/or adaptive interventions) in rural SMS interventions for identified demographic subsets.

Trial registration

Clincialtrials.gov: NCT03740113.

Keywords

Text-message sugar-sweetened beverages adolescents mHealth engagement retention caregivers parents

Introduction

Addressing excessive sugar-sweetened beverage (SSB) consumption in the rural Appalachian region is a public health priority, particularly for adolescents and adults. Appalachian adolescents and adults consume around 470 kcal and 457 kcals from SSB per day, respectively, which is two to three times higher than national levels.^1–3 High SSB consumption is consistently linked to adverse health outcomes such as obesity, type 2 diabetes, cardiovascular disease, and dental caries.^4–7 Involving caregivers of adolescents is critical when intervening on SSB behaviors, given their influence through role modeling, parenting practices, and control of the home environment.^8–10 Targeting caregivers provides a two-fold benefit, as it may help reduce their own intake and help them decrease their child's SSB intake.¹¹ However, there are many challenges to reaching adolescents and their caregivers in rural areas, such as lack of access to or availability of primary prevention programs, lack of transportation to in-person programs, and the natural geographical dispersion.¹² In two systematic reviews including 32 interventions, only one study explicitly targeted a rural region, two studies effectively used scalable technology, and no study focused on adolescent caregiver behaviors and role modeling.^13,14

Mobile health (mHealth) approaches, particularly short message service (SMS) or text messaging, may reduce health disparities.^15,16 Interventions using SMS are engaging and effective in improving a wide variety of health behaviors for adolescents and their caregivers, however; few mHealth interventions are conducted in rural areas of the United States.^17–20 Moreover, the few published rural SMS interventions targeting SSB are small scale, pilot and feasibility studies.^21,22 Consequently, less is known about rural engagement in mHealth, a critical predictor for intervention success.²³ Within mHealth, SMS interventions are particularly well suited for rural populations compared to other modes (e.g., native applications), because they only require a mobile phone with cellular connectivity and do not necessitate a large data plan. Historically, the “digital divide” has been a primary concern for implementation of mHealth interventions in rural areas.²⁴ However, due to technological advances, this divide is shrinking, especially related to mobile phone use.²⁵ According to Pew, 97% of Americans own a cell phone and 85% own a smartphone. Pew researchers looked at this by population type as well and found that ownership in rural areas was similarly high, 94% for mobile phone ownership and 80% for smartphone ownership.²⁵ This provides a key opportunity to evaluate how rural populations engage in mHealth interventions, specifically SMS interventions, and determine if certain characteristics may be associated with retention and engagement.

This study addresses gaps in the literature by conducting an exploratory process evaluation on the retention and engagement data from the on-going Kids SIPsmartER trial that aims to improve SSB behaviors in rural Appalachian middle schoolers.²⁶ As part of the multi-level approach, caregivers receive a 6-month-long SMS intervention. Using the first two years of caregiver intervention process data, the objectives of this study are to describe caregiver retention and SMS engagement as well as explore differences by caregiver characteristics.

Methods

Study description

Details of the complete Kids SIPsmartER trial methodology have been described elsewhere.²⁶ In summary, Kids SIPsmartER is a cluster randomized control trial that enrolls 7th grade students in a 6-month school-based, multi-level, behavioral intervention that is grounded in the Theory of Planned Behavior. It incorporates health, media, numeracy, and public health literacy concepts. Paralleling the classroom intervention for students, caregivers receive an SMS-based intervention to their mobile phones for the same period. This exploratory process evaluation study uses caregiver SMS process data from the middle schools that received Kids SIPsmartER during the first two years of the trial; the middle schools randomized to control are not included in these analyses because caregivers did not receive an SMS intervention in this condition.

Intervention

Caregivers received SMS messages to their mobile phones for 6 months, that were either two-way or one-way. These included combinations of assessments (two-way), personalized strategies (one-way), educational messages (one-way), and infographics (one-way) (Figure 1). Message content, framing, and procedures for the caregiver SMS intervention underwent pilot testing that informed the current trial.¹¹

Figure 1.

Short message service (SMS) intervention structure and timeline.

Assessment messages were designed to be two-way, in which caregivers reported daily SSB frequency over the past week for themselves and their child (e.g., How often did you/your child drink sugary drinks last week? Reply with number. 1-[Less than 1time] 2-[1 time] 3-[2–3 times] 4-[4–6 times] 5-[Everyday] 6-[2 per day] 7-[3 or more per day]). Based on responses, caregivers received personalized feedback on their SSB changes (e.g., Congrats! Both you and [Child Name] drank little to no sugary drinks last week. A healthy family is a happy family, keep up the awesome work!). Following this, caregivers were given the opportunity to choose a strategy topic relevant to their barriers. These choices included parenting tips, tasty alternatives, breaking habits, home and shopping tips, and dealing with friends and family. The caregiver's choice was then stored for future message use.

Based on the caregiver's choice during the assessment messages, personalized strategy messages were curated to provide guidance to reduce caregiver and child SSB intake (e.g., Wondering where to start breaking your habit? Try keeping a log of when and where you drink sugary drinks). If a caregiver reported very low or no SSB consumption for themselves and their child, they did not receive these messages. Conversely, if the caregiver and/or child were consuming SSBs and a strategy topic was not selected, they were assigned a random strategy topic.

Caregivers also received educational messages that paralleled their child's classroom lessons. Seven of these messages were one-way messages that displayed as a typical message on the caregiver's phone (e.g., Drink less, live more, throw sugar out the door! Limit sugary drinks to 8oz for adults and 0 for kids to improve health for the whole fam). Five of these messages, were considered infographic messages, which included a brief message followed by a clickable URL link. This URL link redirected the caregiver to an informational image (graphs or pictures to convey educational content, see Figure 2 for example) displayed on their mobile phone's browser.

Figure 2.

Example of an infographic used in the SMS intervention.

Participants

Caregivers of all targeted 7th grade students received informational letters from school principals, flyers, and phone calls requesting their participation. Caregivers provided written or verbal consent for both themselves and their child. Before starting the SMS intervention, caregivers received a one-page educational hand-out. Caregivers were excluded from the intervention if they did not have a functioning mobile phone/phone number. No direct incentive was provided for taking part in the SMS portion of the program. However, caregivers who agreed to be part of the intervention received a $10 gift card for completing a baseline survey.

Procedure

A baseline survey was sent home with the students for their caregivers to complete and send back with their child. Caregivers must have completed a baseline survey and have a functioning cellphone number to be enrolled into the SMS portion of the program. Once baseline survey was returned, caregivers were enrolled to receive messages.

Caregivers received a maximum of two messages per week, for 6 months (See Figure 1). Caregivers first received an introductory message followed by the first assessment messages. A total of five assessment messages were delivered every five to six weeks during the intervention. The first and fifth assessment included up to four reminders to prompt assessment completion, whereas the other three assessments included up to two reminders. In between each assessment message, caregivers received educational and personalized strategy messages. Toward the second half of the program, caregivers received messages less frequently, and some weeks would only receive one personalized strategy message.

Qualtrics Research Suite, hosted by the University of Virginia, was used to deliver the SMS messages, host the infographic images, and temporarily store data. This tool provides research-grade security and protection of data. Additionally, Qualtrics allows for tailoring and personalization of messages (e.g., using participant data, such as names and SSB intake, to carry forward to future messages), and for automation of message delivery to reduce researcher burden and error.

Measures

Baseline caregiver characteristics: Demographics included age, sex, race/ethnicity, income, education, marital status, and health literacy. Caregivers selected their race from American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or other Pacific Islander, White or Caucasian, or other, and selected ethnicity from Hispanic, non-Hispanic, or not sure. Given the regional demographics, race/ethnicity was combined and recoded into two categories, non-Hispanic White and other. Income was reported across 14 categories and recoded into two categories (<$50,000 and >$50,000) for comparison. Following similar procedures as previously published studies,^27–29 health literacy was assessed using three single-item subjective measures on a 5-point Likert scale.^30–32 The score from each of the three measures was averaged to create a total score ranging from 1 (lowest health literacy) to 5 (highest health literacy). We then recoded the total score into dichotomized categories of “low” (scores that ranged from 1–4) and “high” (scores that ranged from 4.1–5). Caregiver SSB related variables included behavioral intentions (2-item; 7-point Likert scale), attitudes toward reducing SSB (2-items; 7-point Likert scale), and baseline SSB intake. An abbreviated version of the validated BEV-Q15 assessed baseline SSB intake.³³ For each SSB (regular soda, sweetened juice drinks, energy/sports drinks, sweetened tea, and sweetened coffee), caregivers reported frequency and amount. Using standardized scoring procedures, daily totals for each SSB type were summed to obtain total ounces of SSB per day.

Retention and Engagement: Caregiver retention was measured using a dropout rate. Any participant who did not complete the first and second SMS assessments, or who notified the study team that they did not wish to continue were considered dropouts. Engagement indicators included the 1) number of assessments completed (0–5), 2) number of reminders sent to encourage assessment completion (0–14), 3) number of times a strategy topic was chosen (0–4) and 4) number of infographics opened (0–5).

Data analysis

SPSS Version 27.0 was used to conduct data analyses. Variables were described using means and standard deviations. Due to non-normal distribution of the data, non-parametric tests were used. Chi-square and Mann-Whitney U were used to examine caregiver baseline characteristics related to retention. Kruskal-Wallis, Mann-Whitney U, and Spearman's rank correlation tests were used to examine the relationship between the baseline caregiver characteristics and the engagement indicators. All reported p-values tests are two-tailed tests and p ≤ 0.05 was interpreted as significant.

Results

Participants

From the five schools included in this study, 620 caregivers were invited to participate. Of these, 357 (58%) agreed to be a part of the SMS intervention and returned a baseline survey. As shown in Table 1, most of the sample was between the ages of 35–44 years (52%), female (91%), White (95%), and reported annual household income greater than or equal to $50,000 (56%). Additionally, most had at least some college education (68%), were married (72%), and had high health literacy (86%). Related to SSB intake, most of the sample had a strong intention to change their behaviors (53%) and a positive attitude toward reducing intake (69%). As shown, most caregivers consumed SSB, with distributions somewhat similar across categories from low to very high. Average baseline SSB intake was 23.9 (SD = 26.8) oz/day.

Table 1.

Baseline caregiver characteristics and their relationships to retention rates.

Characteristics	Totaln = 357, n(%)	Number and proportion retained within each characteristic n = 316, n(%)	Chi square, p-value
Age
≤34 years	57 (16%)	48 (84%)	x²= 5.13, P = .07
35–44 years	185 (52%)	170 (92%)
≥45years	104 (29%)	87 (84%)
Sex
Male	31 (9%)	25 (81%)	x²= 1.96, P = .16
Female	321 (91%)	286 (89%)
Race
White	312 (95%)	275 (88%)	x²= 0.38, P = .54
Other	15 (5%)	14 (93%)
Household Income
<$50,000	127 (44%)	106 (84%)	x²= 4.71, P = .03
≥$50,000	159 (56%)	146 (92%)
Education
HS, GED or less	109 (32%)	87 (80%)	x²= 11.78, P = .001
Some college or more	229 (68%)	212 (93%)	x²= 11.78, P = .001
Marital status
Married	249 (71%)	221 (89%)	x²= 0.21, P = .65
Single	100 (29%)	87 (87%)
Health literacy
Low	46 (14%)	37 (80%)	x²= 3.40, P = .07
High	293 (86%)	263 (90%)

Retention

Of the 357 enrolled caregivers, 316 (89%) were retained. Of the 41 (11%) dropouts, three actively notified the study that they wished to discontinue. All others were passive dropouts and stopped receiving intervention content after no response was recorded for the second assessment message. Attempts were made to contact these individuals. Using call attempt data and Qualtrics notification regarding non-functioning numbers, it was determined that 11 (3%) were non-functioning numbers once the intervention started. When analyzing differences by caregiver baseline characteristics, caregivers with a higher household income (>$50,000/year)(P = 0.03) and some college education or more were retained at a higher rate (P = 0.001). No other baseline caregiver characteristics were significantly related to retention. Yet, there was a trend (P < .10) for caregivers who were between the ages of 35–44 and who had high health literacy to be retained at a somewhat higher rate (Table 1).

There were no significance differences between retained and dropout participants for baseline intention to change scores [retained M = 4.5 (SD = 2.5), dropout M = 4.0 (SD = 2.4); P = .22], baseline attitude towards reducing SSB scores [retained M = 4.9 (SD = 1.4), dropout M = 4.9 (SD = 1.5); P = .56], or baseline SSB intake [retained M = 23.0 (SD = 26.0) ounces, dropout M = 31.2 (SD = 31.5) ounces; P = .14] (Table 2).

Table 2.

Differences in caregiver sugar-sweetened beverage (SSB) related characteristics for those retained or not retained in the study.

Characteristics	n	Mean (SD)	Mann-Whitney U test, p-value
Intention to Change score
Retained	314	4.5 (2.2)	𝑈 = 7193.5, P = .22
Not Retained	41	4.0 (2.4)
Attitude Towards Reducing SSB score
Retained	313	4.9 (1.4)	𝑈 = 6063.0, P = .56
Not Retained	41	4.9 (1.5)
Baseline SSB Intake (fluid ounces)
Retained	316	23.0 (26.0)	𝑈 = 5569.5, P = .14
Not Retained	41	31.2 (31.5)

Engagement

Among those retained, caregivers on average completed 4.1 (SD = 1.3) of 5 assessments (82%) and required 4.1 (SD = 3.7) out of 14 possible reminders across all assessment points. They also chose a strategy on average 3.2 (SD = 1.1) of 4 possible opportunities (80%) and opened 1.2 (SD = 1.6) of 5 infographics (24%) (Table 3).

Table 3.

Comparisons among baseline caregiver characteristics and engagement indicators among caregivers who were retained (n = 316).

Characteristics	Number of Assessments Completed (1–5)	Number of Reminders Needed to Complete Assessments (0–14)	Number of Times a Strategy was Selected (0–4)	Number of Infographic URLs Clicked (0–5)
Overall Averages (M,SD)	4.1 (1.3)	4.1 (3.7)	3.2 (1.1)	1.2 (1.6)
Age (M,SD)
≤34 years (n = 48)	3.6 (1.6)^a	6.0 (4.0)^a	2.8 (1.4)^a	1.1 (1.6)^a
35–44 years (n = 170)	4.2 (1.2)^b	3.8 (3.4)^b	3.3 (1.1)^b	1.0 (1.5)^a
≥45 years (n = 87)	4.2 (1.3)^b	3.9 (3.9)^a,b	3.2 (1.0)^b	1.7 (1.7)^b
Comparison test	H = 11.61, P = .003	H = 7.67, P = .02	H = 11.80, P = .003	H = 22.05, P < .001
Sex (M,SD)
Male (n = 25)	4.3 (1.3)	4.1 (3.9)	3.2 (1.1)	1.4 (1.9)
Female (n = 286)	4.1 (1.3)	4.1 (3.7)	3.2 (1.1)	1.2 (1.5)
Comparison test	𝑈 = 4912.0, P = .90	𝑈 = 4595.0, P = .48	𝑈 = 3561.0, P = .92	𝑈 = 3476.5, P = .81
Race (M,SD)
White (n = 275)	4.1 (1.3)	4.1 (3.7)	3.2 (1.1)	1.2 (1.6)
Other (n = 14)	4.5 (1.2)	3.1 (3.9)	3.4 (0.9)	1.4 (1.8)
Comparison test	𝑈 = 2869.0, P = .11	𝑈 = 1885.0, P = .20	𝑈 = 2097.0, P = .54	𝑈 = 1939.5, P = .99
Household Income (M,SD)
<$50,000 (n = 106)	4.0 (1.4)	4.3 (3.7)	3.2 (1.1)	1.1 (1.6)
≥$50,000 (n = 146)	4.3 (1.2)	3.5 (3.2)	3.3 (1.1)	1.3 (1.6)
Comparison test	𝑈 = 11,538.0, P = .02	𝑈 = 8576.5, P = .03	𝑈 = 8231.0, P = .39	𝑈 = 8226.0, P = .36
Education (M,SD)
HS, GED or less (n = 87)	3.9 (1.4)	4.6 (3.6)	3.0 (1.2)	1.3 (1.7)
Some college + (n = 212)	4.2 (1.2)	3.8 (3.7)	3.3 (1.1)	1.2 (1.6)
Comparison test	𝑈 = 15,557.5, P < .001	𝑈 = 9466.0, P < .001	𝑈 = 10,401.5, P = .07	𝑈 = 9141.0, P = .90
Marital Status (M,SD)
Married (n = 221)	4.2 (1.3)	3.9 (3.6)	3.2 (1.1)	1.2 (1.6)
Single (n = 87)	3.9 (1.4)	4.6 (3.8)	3.0 (1.2)	1.2 (1.6)
Comparison test	𝑈 = 10,855.0, P = .04	𝑈 = 13,862.0, P = .10	𝑈 = 8682.0, P = .11	𝑈 = 9198.0, P = .53
Health Literacy (M,SD)
Low (n = 37)	3.6 (1.5)	4.8 (3.9)	2.8 (1.2)	1.5 (1.8)
High (n = 263)	4.2 (1.2)	3.9 (3.6)	3.3 (1.1)	1.2 (1.5)
Comparison test	𝑈 = 8574.0, P = .001	𝑈 = 5440.0, P = .04	𝑈 = 6065.0, P = .008	𝑈 = 4367.5, P = .28

^a,b,c

Kruskal-Wallis tests were used to assess if there were any significant differences between the means of the categories Post-hoc analyses were done using the Dunn method. Values without the same superscript letter are significantly different (p < 0.05).

When analyzing differences by caregiver baseline characteristics, statistically significant differences were found among several engagement indicators (Table 3). Caregiver age was associated with all four engagement indicators. Caregivers who were 34 years old or younger completed fewer assessments and selected fewer strategies (P = .003) compared to all other age groups. Caregivers who were 34 years old or younger also needed significantly more reminders (P = .02) compared to those who were 33–35 years of age. Caregivers who were 45 years or older clicked on more infographic URLs, relative to the other age groups (P < .001). Compared to caregivers with higher income, those with low income (<$50,000) completed less assessments (P = .02) and needed more reminders (P = .03). Relative to caregivers with some college education, caregivers with a high school education or lower completed fewer assessments (P < .001) and needed more reminders to complete assessments (P < .001). Caregivers who were single were less likely to complete assessments compared to married caregivers (P = .04). Compared to caregivers with high health literacy, those with lower health literacy also completed fewer assessments (P = .001), needed more reminders (P = .04), and chose fewer strategies (P = .008)

When analyzing differences key by caregiver baseline SSB related characteristics, those who had a stronger intention to change at baseline completed more assessments (P = .019) and needed fewer reminders (P = .021). Additionally, those who had a higher baseline intake completed fewer assessments (P = .003), needed more reminders to complete assessments (P = .005), and chose fewer strategies (P = .026) (Table 4).

Table 4.

Correlations among caregiver sugar-sweetened beverage (SSB) related characteristics and engagement indicators among caregivers who were retained (n = 316).

Characteristics	Number of Assessments Completed (1–5)	Number of Reminders Needed to Complete Assessments (0–14)	Number of Times a Strategy was Selected (0–4)	Number of Infographic URLs Clicked (0–5)
Intention to Change Score	ρ=.124 P = .019	ρ=−.123 P = .021	ρ=.102 P = .071	ρ=.062 P = .270
Attitudes Toward Reducing SSB Score	ρ=−.088 P = .099	ρ=.070 P = .189	ρ = −.095 P = .093	ρ = −.080 P = .157
Baseline SSB Intake (fluid ounces/day)	ρ = −.157 P = .003	ρ = .149 P = .005	ρ = −.125 P = .026	ρ = −.036 P = .520

Discussion

Principal findings and comparison with prior work

Results from this exploratory process study showed relatively high retention (89%) rates among rural caregivers in an SMS intervention targeting SSB reduction. Assessment and strategy messages were completed at reasonably high rates (>80%). Our overall retention and engagement results expand on previous research with SMS-based interventions to caregivers of children targeting healthy eating, SSB intake, and other health behaviors. Brown and colleagues, whose intervention focused on rural Native American parents of 3–5-year-old children (n = 17), had a 100% retention rate and a 95% response rate to their SMS messages.²¹ Similarly, Aldoory and colleagues focused on rural low-income mothers (n = 37) and had a 93% retention rate and greater than 75% of their sample reported viewing the text messages.²² While the retention rates are slightly higher in these two studies, our study had a substantially larger sample (n = 357) and still found a comparable engagement level in assessment messages that required response (82%).

Importantly, our results are also comparable to other large sample studies in urban and other geographically diverse settings, further demonstrating the effects of the shrinking “digital divide.”^34,35 For example, Price and colleagues used SMS to target multiple behaviors including SSB in a sample of 160 caregivers of 6–12-year-old children. Their results at one year indicated a similar level of engagement with the text messaging intervention (75%).³⁴ Overall, SMS interventions are very accessible, low-burden, and easy to use, and these characteristics may be contributing to the high engagement seen across geographically diverse settings. Although more testing is needed through randomized controlled trials in rural settings, together these data suggest using mobile phone approaches, and in particular their SMS capability, is engaging and may be an effective way to target SSB intake in rural caregivers and adolescents.

Although in this study we identified several differences in engagement by caregiver characteristics that were significant, average engagement was still relatively high across all participants. For example, when looking at the income data, those in the lower income category (<$50,000) significantly completed fewer assessment (4.0 out of 5 total). Comparatively those who reported a greater income completed 4.3 out of 5 total, only 0.3 assessments more. Other characteristics that differed included income and education for retention and age, education, marital status, health literacy, intention to change SSB behaviors, and baseline SSB intake for engagement indicators. The results suggest that dropouts and dips in engagement may be related to expected attrition, yet the significant findings underline emerging patterns that can help inform retention and engagement strategies in similar interventions for rural caregivers. To preemptively increase engagement, caregivers can be screened for phenotypes that match these characteristics,³⁶ and targeted strategies can be employed. These targeted strategies may include soliciting specific user-feedback in the design on the interventions²³ and using adaptive, just-in-time support, including live human and peer encouragement,³⁷ during the intervention to promote continued engagement.

Especially important to highlight are our findings around baseline SSB intake. We found that higher baseline SSB consumption trended toward lower retention in the intervention as well as significantly lower engagement for three of the four indicators. This finding is similar to a study by Kim and colleagues who explored specific digital phenotypes associated with engagement in a digital health intervention for weight loss.³⁶ In their machine learning based study, a higher intake of high calorie foods was predictive of lower engagement.³⁶ Given that reducing SSB intake was our primary outcome, these findings may have implications on the effectiveness of the intervention. Thus, the underlying mechanism that triggers these high consumers to drop out and engage less needs to be clarified and addressed. Literature on digital health recruitment and engagement suggests that common barriers include personal agency, motivation, and values related to the targeted health behaviors.³⁸ In our study, we examined baseline intentions and attitudes and found that a stronger intention to change behaviors was associated with needing fewer reminders to complete an assessment. Yet, no other patterns were found. However, other potential predictors such as motivation, awareness and personal agency were not explored and could be considered for future work.³⁸

Interestingly, although engagement was high across three of the indicators related to the assessment messages, engagement was lowest for the infographic URL link clicks within the SMS (24%). Other studies targeting different behaviors have utilized this methodology in diverse, low-income urban samples and have found similar results. For example, in an SMS-based intervention targeting breastfeeding rates in women enrolled in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), engagement with multimedia messages (i.e., link clicks) was around 24% for prenatal women and 20% for postpartum women.³⁹ Similarly, a study sending multimedia messages to caregivers to encourage positive parent–child engagement around literacy and language development, qualitatively reported that a majority of parents reported never opening or seeing the links.⁴⁰ With further prompts to elaborate how engagement could be increased, participants in this study suggested more encouragement to click links, highlighting the topic covered in the links, and possibly embedding links within the message to reduce participant burden of added clicks.⁴⁰ While our study did highlight the topic that would be covered within the SMS text before the participant clicked the link, future research could explore adding reminders or embedding the infographic within the message to increase engagement. Ongoing summative data is being collected in the KidsSIPsmarter trial to understand why this infographic URL click rate is so low. Several hypotheses being qualitatively explored are data usage concerns, time, disinterest, forgetfulness, and the caregiver's perceptions of the security of the link.

Limitations

Our study has several limitations that should be consider when interpreting and applying findings. First, related to generalizability, this research includes mostly female, rural Appalachian middle school caregivers and may have limited implications beyond this population and region. Second, results related to health literacy should be cautiously interpreted because no validation studies have been conducted on the procedures used to combine the score from the three individual measures. Third, the engagement analysis was restricted to those who were retained and those who engaged with the specific messages. Therefore, it is important to consider this when extrapolating the results on associated characteristics to the general population. Fourth, we were unable to capture engagement with text messages that did not require a response or click, including for the educational and personalized strategy messages and where the key behavioral content is delivered in this study. Notably, this limitation is consistent across the majority of SMS-based interventions. Fifth, we recognize that multiple tests were conducted in our exploratory process evaluation. Rather than adjusting our p-values for multiple tests, which is known to be overly conservative and introduce error (e.g., increased likelihood of type II error), we report exact p-values.⁴¹ Readers should interpret the p-values accordingly. Finally, this study was not specifically powered to examine differences in retention or engagement, so the non-significant findings should be interpreted cautiously, especially for those with trending p-values. Despite these limitations, our findings fill notable gaps in the literature pertaining to rural engagement and retention in an SMS-based intervention focused on pervasive and risky SSB behaviors.

Future research

While this study serves as a great primer on engagement in SMS-based health interventions for rural populations, more research is needed. Specifically, a more comprehensive understanding pertaining to the timing and reasons for caregivers’ disengagement is needed to better inform engagement strategies (e.g., Do caregivers disengage because they feel they have benefitted or because the content is not helpful?). To accomplish this, future SMS engagement research should examine a combination of subjective qualitative interviews with parents and objective criteria such as those reported in this study. Future research should also explore how SMS engagement influences changes in SSB intake, in both caregiver and child. Finally, while our trial uses SMS to reach caregivers of adolescents, future efforts should also explore the feasibility and efficacy of SMS strategies to reduce SSB directed toward adolescents.

Conclusions

In conclusion, our exploratory process study determined a relatively high rate of enrollment, engagement, and retention similar to other rural and urban studies. We also found several caregiver characteristics, including age, income, education, health literacy, baseline intention to change SSB behaviors and baseline SSB intake, that can be used to help define retention and engagement strategies or develop targeted approaches. These findings are an important contribution to the literature because they demonstrate the practicality of using SMS-based approaches in rural settings, provide insight into how rural caregivers engage with SMS messages, and provide data to preemptively increase engagement.

Footnotes

Abbreviations

Acknowledgments

The authors would like to acknowledge the peer reviewers for their review of the manuscript.

Contributorship

MY and JZ conceptualized the study. MY and DB managed and analyzed the data. All authors were involved in the development of the SMS intervention and consulted on data analysis. MY wrote the first draft of the study. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

All study procedures have been approved by the University of Virginia Institutional Review Board.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Institutes of Health (NIH), National Institute on Minority Health and Health Disparities [R01MD012603]. The funder had no role in the study design, analysis or interpretation of the data, writing of the report, or the decision to submit the paper for publication.

Guarantor

MY.

ORCID iD

Maryam Yuhas

References

Lane

Porter

Hecht

, et al. Kids SIP smart ER: a feasibility study to reduce sugar-sweetened beverage consumption among middle school youth in central appalachia. Am J Health Promot 2018; 32: 1386–1401.

Rosinger

Herrick

Gahche

, et al. Sugar-sweetened beverage consumption among US youth, 2011–2014. NCHS Data Brief 2017; 271: 1–8. PMID: 28135184.

Zoellner

Estabrooks

Davy

, et al. Exploring the theory of planned behavior to explain sugar-sweetened beverage consumption. J Nutr Educ Behav 2012; 44: 172–177.

Malik

Pan

Willett

, et al. Sugar-sweetened beverages and weight gain in children and adults: a systematic review and meta-analysis. Am J Clin Nutr 2013; 98: 1084–1102.

Imamura

O’Connor

, et al. Consumption of sugar sweetened beverages, artificially sweetened beverages, and fruit juice and incidence of type 2 diabetes: systematic review, meta-analysis, and estimation of population attributable fraction. Br Med J 2015; 351: h3576.

Cheungpasitporn

Thongprayoon

Edmonds

, et al. Sugar and artificially sweetened soda consumption linked to hypertension: a systematic review and meta-analysis. Clin Exp Hypertens 2015; 37: 587–593.

Bernabé

Vehkalahti

Sheiham

, et al. Sugar-sweetened beverages and dental caries in adults: a 4-year prospective study. J Dent 2014; 42: 952–958.

Yuhas

Porter

Hedrick

, et al. Using a socioecological approach to identify factors associated with adolescent sugar-sweetened beverage intake. J Acad Nutr Diet 2020; 120: 1557–1567.

Brown

Ogden

. Children’s eating attitudes and behaviour: a study of the modelling and control theories of parental influence. Health Educ Res 2004; 19: 261–271.

10.

Watts

Miller

Larson

, et al. Multicontextual correlates of adolescent sugar-sweetened beverage intake. Eat Behav 2018; 30: 42–48.

11.

Yuhas

Porter

Brock

D-JP

, et al. Development and pilot testing of text messages to help reduce sugar-sweetened beverage intake among rural caregivers and adolescents: mixed methods study. JMIR Mhealth Uhealth 2019; 7: e14785.

12.

Thomas

DiClemente

Snell

. Overcoming the triad of rural health disparities: how local culture, lack of economic opportunity, and geographic location instigate health disparities. Health Educ J 2014; 73: 285–294.

13.

Vargas-Garcia

Evans

Prestwich

, et al. Interventions to reduce consumption of sugar-sweetened beverages or increase water intake: evidence from a systematic review and meta-analysis. Obes Rev 2017; 18: 1350–1363.

14.

Lane

Porter

Estabrooks

, et al. A systematic review to assess sugar-sweetened beverage interventions for children and adolescents across the socioecological model. J Acad Nutr Diet 2016; 116: 1295–1307. e1296.

15.

Anderson-Lewis

Darville

Mercado

, et al. Mhealth technology use and implications in historically underserved and minority populations in the United States: systematic literature review. JMIR Mhealth Uhealth 2018; 6: e8383.

16.

Brewer

Fortuna

Jones

, et al. Back to the future: achieving health equity through health informatics and digital health. JMIR Mhealth Uhealth 2020; 8: e14512.

17.

Militello

Kelly

Melnyk

. Systematic review of text-messaging interventions to promote healthy behaviors in pediatric and adolescent populations: implications for clinical practice and research. Worldviews Evid Based Nurs 2012; 9: 66–77.

18.

Militello

Melnyk

Hekler

, et al. Automated behavioral text messaging and face-to-face intervention for parents of overweight or obese preschool children: results from a pilot study. JMIR Mhealth Uhealth 2016; 4: e21.

19.

Chu

JTW

Wadham

Jiang

, et al. Effect of MyTeen SMS-based Mobile intervention for parents of adolescents: a randomized clinical trial. JAMA Netw Open 2019; 2: e1911120–e1911120.

20.

Callender

Thompson

. Family TXT: feasibility and acceptability of a mHealth obesity prevention program for parents of pre-adolescent African American girls. Children (Basel) 2018; 5: 81.

21.

Brown

Harris

Dybdal

, et al. Feasibility of text messaging to promote child health in a rural community on an American Indian reservation. Health Educ J 2019; 78: 557–569.

22.

Aldoory

Yaros

Prado

, et al. Piloting health text messages for rural low-income mothers: effects of source similarity and simple action steps. Health Promot Pract 2016; 17: 702–710.

23.

Birnbaum

Lewis

Rosen

, et al. Patient engagement and the design of digital health. Acad Emerg Med 2015; 22: 754.

24.

Hong

Cho

. Has the digital health divide widened? Trends of health-related internet use among older adults from 2003 to 2011. J Gerontol B Psychol Sci Sc 2017; 72: 856–863.

25.

“Mobile Fact Sheet”, Pew Research Center. Washington, D.C, https://www.pewresearch.org/internet/fact-sheet/mobile/ (2021, accessed 25th October 2021).

26.

Zoellner

Porter

You

, et al. Kids SIPsmartER, a cluster randomized controlled trial and multi-level intervention to improve sugar-sweetened beverages behaviors among Appalachian middle-school students: rationale, design & methods. Contem Clin Trials 2019; 83: 64–80.

27.

Estabrooks

You

Hedrick

, et al. A pragmatic examination of active and passive recruitment methods to improve the reach of community lifestyle programs: the talking health trial. Intl J Behav Nutr Phys Act 2017; 14: 1–10.

28.

Zoellner

Porter

You

, et al. The reach and effectiveness of SIP smart ER when implemented by rural public health departments: a pilot dissemination and implementation trial to reduce sugar-sweetened beverages. Transl Behav Med 2020; 10: 676–684.

29.

Zoellner

Hill

You

, et al. The influence of parental health literacy status on reach, attendance, retention, and outcomes in a family-based childhood obesity treatment program, Virginia, 2013–2015. Prev Chronic Dis 2017; 14: 160421.

30.

Wallace

Rogers

Roskos

, et al. Brief report: screening items to identify patients with limited health literacy skills. J Gen Intern Med 2006; 21: 874–877.

31.

Morris

MacLean

Chew

, et al. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006; 7: 1–7.

32.

Jeppesen

Coyle

Miser

. Screening questions to predict limited health literacy: a cross-sectional study of patients with diabetes mellitus. Ann Fam Med 2009; 7: 24–31.

33.

Hedrick

Comber

Flack

, et al. Development of a brief questionnaire to assess habitual beverage intake: sugar-sweetened beverages and total beverage energy intake. J Am Diet Assoc 2010; 110: A26.

34.

Price

Ferisin

Sharifi

, et al. Development and implementation of an interactive text messaging campaign to support behavior change in a childhood obesity randomized controlled trial. J Health Commun 2015; 20: 843–850.

35.

Nezami

Ward

Lytle

, et al. A mHealth randomized controlled trial to reduce sugar-sweetened beverage intake in preschool-aged children. Pediart Obes 2018; 13: 668–676.

36.

Kim

Yang

Ahn

W-Y

, et al. Machine learning analysis to identify digital behavioral phenotypes for engagement and health outcome efficacy of an mHealth intervention for obesity: randomized controlled trial. J Med Internet Res 2021; 23: e27218.

37.

Fortuna

Brooks

Umucu

, et al. Peer support: a human factor to enhance engagement in digital health behavior change interventions. J Technol BehavSci 2019; 4: 152–161.

38.

O’connor

Hanlon

O’donnell

, et al. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak 2016; 16: 1–15.

39.

Martinez-Brockman

Harari

Perez-Escamilla

. Lactation advice through texting can help (LATCH): an analysis of intensity of engagement via two-way text messaging. J Health Commun 2017; 31: 650.618–650.618.

40.

Pila

Lauricella

Wartella

. Using short message (SMS) and multimedia messaging (MMS) to encourage positive parent–child engagement around literacy and language development. Mobile Media & Communication 2019; 7: 265–285.

41.

Perneger

. What's wrong with Bonferroni adjustments. Br Med J 1998; 316: 1236–1238.