Abstract
This study examined the feasibility of using tailored text messages to promote adherence to longitudinal protocols and determined what facets of text message tone influence motivation. Forty-three older adults (Mage = 73.21, SD = 5.37) were recruited to engage in video-game-based cognitive training for 10 consecutive days. Participants received encouraging text messages each morning that matched their highest or lowest ranking reasons for participating in the study, after which they rated how effective each message was in motivating them to play the games that day. After 10 days, participants rated all possible messages and participated in semi-structured interviews to elicit their preferences for these messages. Results showed that messages matching participants’ reasons for participating were more motivating than mismatched messages. Further, participants preferred messages that were personalized (i.e., use second person voice) and in formal tones. Messages consistent with these preferences were also rated as more motivating. These findings establish the feasibility of using message tailoring to promote adherence to longitudinal protocols and the relevance of tailoring messages to be personal and formal.
Introduction
Population aging, coupled with age-related cognitive declines, represents an unprecedented challenge for the United States and the world (Vespa et al., 2020). Aside from normative declines, it is estimated that about one in every five older adults will experience cognitive impairments and more than one in nine older adults will develop dementia (Rajan et al., 2021). Home-based, technology-delivered cognitive training has the potential to prevent or reverse age-related cognitive declines (e.g., Mewborn et al., 2017; Nguyen et al., 2022), although the efficacy of such programs is subject to debate (Simons et al., 2016).
Previous literature suggests that adherence tends to be low for cognitive training (Hardy et al., 2015; Owen et al., 2010). For instance, in a 3-month home-based cognitive training study, the older adult sample on average only engaged in 22 hours of training out of the total 60 hours required (Boot, Champion et al., 2013). Low adherence could limit the potential of cognitive training in two important ways. First, low adherence in clinical trials can interfere with the ability to determine the efficacy of cognitive training (Boot et al., 2013). Second, should cognitive training prove to be effective, low adherence could limit how much benefit individuals derive from these programs (Bagwell & West, 2008; Willis & Caskie, 2013). The current study investigated the feasibility of using tailored motivational messages delivered via short messaging service (SMS) to boost adherence to home-based, technology-delivered cognitive training.
Message tailoring is a communication strategy that involves developing personalized messages based on pre-assessed individual difference variables and characteristics (Kreuter et al., 2000). The Elaboration Likelihood Model (Petty & Cacioppo, 1986) proposed that attitudes may be influenced through central route processing (i.e., through one’s thoughtful consideration of the merits of a persuasive message) or peripheral route processing (i.e., through cues in the persuasion context). Message tailoring is expected to increase perceived relevance of the message and subsequently increase persuasion by increasing elaboration and thoughtful consideration of message contents (Lustria & Cortese, 2020). Numerous studies have demonstrated that tailored short messages and reminders can improve intervention adherence and promote health outcomes in the general population (see, e.g., Laranjo et al., 2021; Mauch et al., 2022; Sahin et al., 2019; Tong et al., 2021 for reviews). Several studies also showed that interventions utilizing a SMS component can effectively promote various health outcomes in middle-aged and older adults, but results are inconsistent for different health outcomes (see, for example, D’Amore et al., 2022; Elavsky et al., 2019; Jeem et al., 2022; Tam et al., 2022 for reviews). Since interventions usually combine text messaging with other components (Elavsky et al., 2019), it is unclear how effective tailored short messages alone might be in promoting behavior change and intervention adherence in older adults.
While SMS could be useful for providing motivational support at the time when participants were most in need (i.e., just-in-time support; Nahum-Shani et al., 2015), this is a very lean medium for conveying important information. A few quantitative studies showed that users prefer messages that are polite, positively framed, free of textese (i.e., spelling abbreviations in SMS), and grammatically correct (Heron et al., 2019; Muench et al., 2014; Vilar-Lluch et al., 2023). Older adults share those preferences (Muench et al., 2014). However, it is unclear whether preferences on those subtle linguistic variations in messages (i.e., differences in tones, structures, length, and other facets) would have noticeable impact on the persuasiveness of the messages or on health outcomes. For instance, a recent study on the effectiveness of COVID vaccination campaign showed that skeptic, unvaccinated, and fully vaccinated respondents had the same preference on linguistic variations in messages (Vilar-Lluch et al., 2023). This finding cast doubts on the relevance of message preferences identified in previous studies and showed the need to focus on the impact of linguistic variations on proximal outcomes.
The current study seeks to demonstrate the feasibility of using tailored messaging to boost motivation (Aim 1), with the hopes that this will translate to increased adherence to longitudinal protocols. Given this aim, the corollary and exploratory goal was to understand which ones of the preferred linguistic variations in messages might have an impact on how motivating older adults perceive a message to be (Aim 2). The study serves as a pilot for the feasibility of a component of a larger intervention (i.e., stage 1b of the NIH stage model for behavioral intervention development; Onken et al., 2014). To inform the tailoring effort, a prior study was conducted to understand older adults’ reason to participate in research studies. We found that almost all older adults listed helping with research as a reason. Older adults can further be categorized as: Research helpers, who endorse few other reasons than help with research; brain health advocates, who want to improve cognitive health besides helping with research; fun seekers, who are driven by fun seeking besides helping with research; and multiple motivation enthusiasts, who want to improve cognitive health, have fun, learn about new technologies, and help with research (Carr et al., 2022).
Building on that previous study, we expect older adults will be motivated by at least one of the four reasons (i.e., improve brain health, help with research, seek fun, and learn about new technologies) and be more motivated to engage when they receive an SMS that matches with their top-ranking reason. Regarding the second aim, we generated preferences on linguistic variations of the sample through semi-structured interviews and quantitatively tested whether messages with features preferred by the participants were more motivating than messages without those features. We expect that older adults might be more motivated by messages with some features they preferred if preference on minor linguistic variations has any impact on proximal outcomes.
Overall, this study contributes to limited research exploring feasibility of using tailored SMS to motivate engagement in older adults and the relevance of manipulating linguistic variations when tailoring messages. These findings will be used to create tailored motivational messages that will be used in combination with Artificial Intelligence to build a just-in-time adherence support system.
Methods
Design
The study used a within-subjects experimental design to assess the relative effectiveness of tailored text messages on older adults’ motivation to adhere to a 10-day computerized cognitive intervention. It also used a mixed-methods design to examine the relevance of participants’ preferences for linguistic variations in text messages on the effectiveness of the messages. Participants received daily text messages that matched with their self-stated reasons for participating in cognitive training as well as messages that did not match these reasons. All procedures and materials were approved by the Institutional Review Board of the Florida State University (STUDY00002051).
Participants
Forty-three older adults aged 66 to 85 years (M = 73.21, SD = 5.37) were recruited from a research participant registry (Institute for Successful Longevity registry) and through ads placed in a local newspaper in Tallahassee, Florida, and surrounding areas. Participants were pre-screened for cognitive impairment with Modified Telephone Interview for Cognitive Status (TICS-M) and those who scored below 35 were excluded from participation (Cook et al., 2009). We also exclude older adults who self-report having problems with vision that cannot be corrected or self-report other health conditions they think might make it difficult for them to use touch screen devices. Participants were predominantly highly educated (n = 34 had bachelor’s degree and above), white (n = 41), and female (n = 30). Participants received a $50 gift card for participating.
Procedures
Development of the Experimental Treatment
Tailored messages representing 4 motivation categories identified in a previous study (Carr et al., 2022) were developed prior to the experiment. Ten messages were developed iteratively by the study team for each of the following categories: (1) preserve cognition, (2) advance science/research, (3) enjoyment, and (4) improve technology proficiency. Messages related to “preserving cognition” conveyed the idea that playing the training games can potentially improve cognitive functioning. “Advance science/research” messages conveyed the idea that their participation provides valuable data to advance scientific discovery. “Enjoy playing games” messages emphasize that games and activities can be fun and enjoyable. Finally, “improve technology proficiency” messages conveyed the idea that playing games provides opportunities to get better with tablet computers. A total of 40 messages were designed (see complete list in Supplemental Materials).
Baseline
Participants were instructed to rank order their reasons for participation (i.e., “preserve cognition,” “advance science/research,” “enjoy playing games,” and “improve technology proficiency”). They were then provided with a tablet with preinstalled cognitive training games along with materials on how to play the games. See the Supplemental Materials for descriptions of the games. These games were gamified cognitive tasks that are similar to brain training games on the market and have been used in previous studies on adherence (e.g., Harrell et al., 2021).
Cognitive Training Intervention
Once participants became familiar with the cognitive training games, they were instructed to play the games for 30 minutes per day at a time and place of their choice for 10 consecutive days. Each morning during the 10-day cognitive intervention, participants received a text message and were instructed to reply with a rating on how motivating the message was in encouraging them to play the game. This message was either tailored according to their highest ranked reason to participate (i.e., a matched message) or according to their lowest ranked reason to participate (i.e., a mismatched message). After receiving the daily message, participants were sent a separate text message asking how motivating the message was. Ratings were based on participants’ responses to the question “How motivating is this message?” and measured using a 5-point scale (1 = not at all; 5 = very). Daily message ratings collected during the 10-day cognitive intervention were used to examine the effects of tailored messaging on motivation and thus serve as the outcome variable for Aim 1. Participants received matched messages on 1 day and mismatched messages on the next, so that each participant received five matched and five mismatched messages over 10 days. The order in which matched or mismatched messages were received first was counterbalanced among participants.
Follow-Up
Similarly, immediately after the cognitive intervention, participants rated all 40 messages developed for the study in a follow-up survey. Messages were presented in random order for each participant. Participants rated how motivating each message was on a 5-point scale (1 = not at all; 5 = very) in a survey. They were also invited back for a semi-structured interview to talk about their experiences. Participants were asked about what facets of the text messages made them more motivating. They were also shown specific examples of highly and poorly rated messages to facilitate elaboration of their preferences. The semi-structured interview data were used to operationalize message design facets (i.e., the independent variable for Aim 2) and message rating at follow-up served as the dependent variable for Aim 2.
Analysis
Aim 1: Efficacy of Message Tailoring to Encourage Adherence to Cognitive Intervention Games
Multilevel modeling was used to analyze daily message ratings during the 10 consecutive days. Age, gender, education, race, message length, and punctuation (i.e., whether message contains “!”) were controlled in the model. Age and message length was between-person centered to facilitate meaningful intercept interpretations. Slopes for all within-person predictors were allowed to vary but were subsequently constrained if the estimated variations were not significant. A priori power analysis suggested that a level 1 sample size of 10 combined with level 2 sample size of 40 have 99% power to detect a medium effect (d = 0.30) at level 1, 48% power to detect a medium effect (d = 0.30) at level 2, and 26% power to detect a medium effect (d = 0.30) cross-level interaction (Arend & Schäfer, 2019). Although message tailoring typically showed medium to large effects on health outcomes in previous meta-analysis (e.g., blood pressure medication adherence: d = 0.65, Tam et al., 2022; Lifestyle behavior outcomes: d = 0.66, Tong et al., 2021), effect size was chosen conservatively to ensure the study has enough power. Given that the effects of primary interest to the current research questions are all on level 1, our sample size was sufficient to detect effects.
Aim 2: Effects of Linguistic Variations Preferences on Motivation
We started with familiarizing interview transcripts and extracted mentioning of linguistic variations (e.g., tones and structures). Those comments informed us of linguistic variation preferences of the sample. Quantitative content analysis was applied on all 40 messages based on preferences mentioned in interviews. A code book (see Supplemental Materials) was constructed iteratively following guidelines and recommendations from Neuendorf (2017). Two research assistants blind to the purpose of the study were trained to code all the messages independently. Coding reliability was assessed for each linguistic aspect using Cohen’s kappa. A kappa value of .60 and above would suggest acceptable reliability (Neuendorf, 2017). Multilevel modeling, using a process similar to the one described earlier, was then conducted to analyze how different aspects of message tone influenced message ratings.
Results
Adherence to the cognitive intervention games over the 10-day period was high, with a mean of 9 days and a mode of 10 days.
Aim 1: Efficacy of Message Tailoring to Encourage Adherence to Cognitive Intervention Games
For daily message ratings, time stamps of responses were examined before analysis. To ensure we captured participants' initial response to the message, we only included ratings that occurred on the same day the message was sent. Out of the 430 total possible ratings, 303 were included in the analysis.
Multilevel modeling results with daily message ratings during the cognitive training intervention as the dependent variable showed that, on average, messages sent during the cognitive intervention received a rating of 3.05 on a 5-point-scale (Table 1, Intercept). Participants rated messages sent later in the intervention as more motivating compared to those sent earlier in the intervention. Moreover, ratings were also higher for messages that addressed participants’ primary reason (i.e., matched messages) for participation. Specifically, messages sent on day 10 were estimated to be about 0.56 points more motivating than messages sent on the first day of the study on a 5-point-scale, and matched messages were estimated to be 0.81 points more motivating than mismatched messages (Table 1, Day, Motivation Match). Results also showed that female participants rated the messages as more motivating than males (Table 1, Gender). The model explained 9% of variance at the within person level.
Multilevel Modeling Estimates (and Standard Errors) of Effects of Message Tailoring on Daily Message Rating.
Note. Motivation match: 0 = match, 1 = mismatch. Race: 0 = White, 1 = Non-white. Education: 0 = Some college and below, 1 = Bachelor’s degree and above. Gender: 0 = Male, 1 = Female.
p < .05. **p < .01. ***p < .001.
Aim 2: Effects of Linguistic Variations Preferences on Motivation
Some participants mentioned in the interview that they preferred messages that are personalized, informational and use precise terms. They also mentioned a preference for formal tones and had mixed preferences on messages that explicitly encourage them to challenge themselves or to be competitive. Based on those findings we created the following codes for the quantitative content analysis: (1) personalization (i.e., personalized vs. generic tone), and (2) language (i.e., playful vs. formal tone). Definition of the codes were presented in Supplemental Materials. Coding of differences in personalization (κ = .947) and language (κ = .796) had acceptable reliability. Multilevel modeling results with message ratings at follow up as the dependent variable suggested that controlling for demographics and motivation match, messages that used a more personalized tone were more motivating whereas messages using playful language were less motivating. Simply referring to the participant in the message increased motivation rating by 0.29 points and using formal language increased motivation rating by 0.67 points on a 5-point-scale (Table 2, Personal Orientation, Playful Language). Results also showed that female participants rated the messages as more motivating than males (Table 2, Gender), and messages addressing participants’ primary reason to participate were rated as more motivating (Table 2, Motivation Match). The model explained 29% of variance in rating differences at the within person level.
Multilevel Modeling Estimates (and Standard Errors) of Effects of Message Characteristics on Message Rating.
Note. Motivation match: 0 = match, 1 = mismatch. Race: 0 = White, 1 = Non-white. Education: 0 = Some college and below, 1 = Bachelor’s degree and above. Gender: 0 = Male, 1 = Female.
p < .05. **p < .01. ***p < .001.
Discussion
The current pilot study examined the efficacy of tailored text messages on motivating adherence to a 10-day cognitive training program among older adults and explored the relevance of preferred linguistic variations in making a message more motivating. To address the primary aim of the study, we examined participants’ ratings of the text messages they received during the intervention period and found that motivational match influenced these ratings, with participants reporting messages more relevant to their reason for participation as being more motivating. This finding is consistent with the idea that message tailoring may be a useful tool to help older adults maintain motivation in the domain of cognitive training. Clinical trials (He et al., 2023) are underway to explore whether these motivational differences translate to improved long-term adherence. These messages will be tailored not only to participants’ motivations but will also be timed to occur (1) when an adherence lapse has been remotely detected, and (2) at a time predicted when participants would be most receptive to a reminder. This smart-reminder approach will be compared to an approach that considers neither motivation nor timing in the delivery of adherence reminders.
To address the second aim of the study, we reviewed interview transcripts and conducted a content analysis of all text messages developed for the study. We identified two main aspects of message tone (personalization and language tones) that influenced participants’ perceptions of how motivating a message is. Messages that were more personalized (i.e., used second person voice) and messages that did not use a playful tone tended to be more motivating regardless of whether the message was tailored to match their motivation to participate. Personalization, such as making heavy use of the second person “you,” has been proposed as a major strategy to create customized messages in numerous studies (e.g., Hawkins et al., 2008; Lustria et al., 2009, 2013, 2016). Our findings supported these propositions by showing that simply planting second person “you” in messages could slightly increase how motivating the message is perceived regardless of content. The finding that formal messages tend to be more motivating to older adults was in line with findings from previous research. For example, Kuerbis et al. (2017) found a strong preference in older adults for messages with proper spelling and grammar over messages with “textese” or grammar mistakes as well as a preference for serious messages over messages that attempted to be humorous. The current findings build on this previous work and provided evidence showing that formal and properly written messages are more motivating among older adults. It is possible that a more casual tone threatens the source credibility of the messages and could be seen as inappropriate in research study contexts that are usually perceived as formal. Older cohorts might also value formality more than younger cohorts. These findings were used to further refine and select messages to be deployed in upcoming clinical trials.
In addition to these primary findings of interest, results also show that participants rated text messages sent later in the intervention period as more motivating regardless of motivation match, which may seem counterintuitive. A potential explanation is that older adults were experiencing the novelty effect (i.e., a temporary increase in motivation and engagement when exposed to something new; Rogers et al., 2009) when they first encounter the training games. As novelty effect wears off, older adults may have relied on external forms of adherence support more. As such, the results could suggest that the text messages sent later in the intervention were successful in providing older adults with support to adhere to their cognitive training. The elevated responses to messages over time might also reflect the cognitive dissonance between participants’ dwindling interest in the protocol on the one hand and being near ceiling adherence on the other. To ease this dissonance between their attitudes and behaviors, older adults might attribute their high adherence to external cues and motivators (e.g., the motivational messages we sent), leading to inflated or distorted message ratings over time.
These findings are interesting because previous studies have shown that the nature, timing, and frequency of push notifications can influence end-users’ responsiveness to notifications (Chang et al., 2023; Muench & Baumel, 2017; NeCamp et al., 2020). Studies have shown that increased frequency of messages sent may lead to negative effects such as notification fatigue, habituation, distraction, and in some cases, decreased effects on targeted behaviors (Morrison et al., 2017; Suggs et al., 2013). The absence of notification fatigue in our study might be due to the short duration and the low message frequency. Participants in our study received one message per day, which is acceptable to most older adults according to our prior focus group study (Dieciuc et al., 2023). That said, there is a need to explore the effects of motivational messaging on actual adherence over time in longitudinal studies. To this point, we plan to investigate the long-term effects of tailored, just-in-time motivational messages on adherence in a larger randomized controlled trial, where we will specifically manipulate both the nature of text messages and timing by which they are sent.
Finally, it is also interesting to note that female participants systematically rated the messages as more motivating both during the cognitive intervention and at follow-up regardless of motivation match. This might be due to social desirability bias, which has been found to be stronger in females than males in previous research in other fields, such as ethical decision making (Yang et al., 2017). We do not yet know whether higher motivation ratings in females transfers into higher actual adherence behaviors in longer-term studies.
While the current study showed the potential of tailored motivational messages on promoting adherence in longitudinal protocols, the current findings need to be considered alongside some limitations. First, the effectiveness of message tailoring on intervention adherence was not evaluated directly. The decision to use a proxy is mainly due to the ceiling effect in actual adherence during the 10-day short study period. Given the well documented intention-behavior gap in health behavior change research (e.g., Rhodes & de Bruijn, 2013; Webb & Sheeran, 2006), it is possible that matched motivational messages may be perceived to be more motivating but may not lead to actual engagement when adherence starts to falter. Well-designed efficacy studies (e.g., He et al., 2023) are needed to determine the actual effectiveness of this approach. Although the sample size is enough to detect the effects of primary interest, it does not have enough power to detect interactions. It is possible that the effect of message tailoring and the influence of message tone may be moderated by demographics and other individual difference characteristics. Future studies with larger samples can further our understanding of these potential individual difference factors and the generalizability of the results to those with lower income or diverse background. Moreover, it needs to be noted that participants were aware of our focus on the text messages during the study, so it is possible that some of them could overanalyze the messages rather than evaluating them based on initial instincts. This might lead to biases and inflate the findings on the effects of linguistic variations. Finally, this study did not examine other facets of text messages (e.g., structure and length) mentioned in previous literature but not in the interview. Future research could explore if preferences for those facets would influence proximal outcomes more systematically.
Limitations notwithstanding, our study showed the potential of motivation-based message tailoring in promoting adherence in longitudinal protocols. When developing tailored messages, we recommend that researchers include personal identifiers to increase personal relevance and use formal tones to maximize credibility and fit with the research context.
Supplemental Material
sj-docx-1-ggm-10.1177_23337214231224571 – Supplemental material for Adherence Promotion With Tailored Motivational Messages: Proof of Concept and Message Preferences in Older Adults
Supplemental material, sj-docx-1-ggm-10.1177_23337214231224571 for Adherence Promotion With Tailored Motivational Messages: Proof of Concept and Message Preferences in Older Adults by Shenghao Zhang, Michael Dieciuc, Andrew Dilanchian, Mia Liza A. Lustria, Dawn Carr, Neil Charness, Zhe He and Walter R. Boot in Gerontology and Geriatric Medicine
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grant R01AG064529 from the National Institute on Aging.
Ethical Approval
This study is approved by the Institutional Review Board at Florida State University (STUDY00002051).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
