Recruiting Military Veterans into Alcohol Misuse Research: The Role of Social Media and Facebook Advertising

Abstract

Background:

The use of digital technology within health care service delivery, monitoring, and research is becoming progressively popular, particularly given the ongoing COVID-19 pandemic. Mobile health (m-health) apps, one form of digital technology, are increasingly being used to promote positive health related behavior change. Therefore, it is important to conduct research to understand the efficacy of m-health apps. The process of participant recruitment is an essential component in producing strong research evidence, along with ensuring an adequately powered sample to conduct meaningful analyses and draw robust conclusions.

Methods:

In this work we outline and reflect on the strategies used to recruit help-seeking military veterans into an intervention study, which aimed to evaluate the efficacy of an app (Drinks:Ration) to modify behavior in alcohol misusers. Recruitment strategies included through (1) partner organizations and (2) social media and Facebook advertising (ads).

Results:

Facebook ads were live for a period of 88 days and were viewed by a total audience of 29,416 people. In total 168 military veterans were recruited across all recruitment strategies, meaning that Drinks:Ration exceeded its recruitment targets. Half of the sample (n = 84) were recruited through social media, including Facebook ads.

Conclusions:

The current article highlighted that targeted Facebook ads were an efficient strategy to recruit military veterans into a digital intervention trial aiming to reduce alcohol consumption because they reduced the amount of time and resources required to contact a large number of potentially eligible individuals for our study. This article acts as a starting point for other researchers to evaluate their recruitment pathways for recruiting military veterans into alcohol misuse research.

Background

With a growing strain on health care services, particularly in the wake of the COVID-19 pandemic, digital technologies are becoming a necessary adjunct in health care delivery and monitoring and have the potential to revolutionize clinical practice and research.¹ The COVID-19 pandemic, associated lockdowns, and social distancing measures have stimulated international changes in health care service delivery and research on a massive scale.² Over the last decade, there has been increased awareness around the potential to use digital technology to improve access to health services, but the pandemic has accelerated this move.^2,3 Mobile health (m-health) apps are one form of digital intervention used to promote health related behavior change,^4,5 for example, to promote a reduction in alcohol consumption.^6,7

Alcohol misuse involves drinking at harmful levels or being dependent on alcohol.⁸ To reduce the risks associated with alcohol, the drinking guidelines in the United Kingdom suggest not to regularly consume >14 U.K. units (140 mL pure alcohol; 10 g of pure alcohol per unit) of alcohol per week.⁹ There are a range of interventions to support the management of alcohol misuse and these often depend on the severity of the problem.¹⁰

Smartphone-based brief alcohol interventions offer the potential to manage alcohol consumption and have several advantages over traditional interventions, including that they can be used anywhere and at any time, helping to target “hard to reach” populations and those underserved by current treatment options.¹¹ In addition, digital technologies can help bridge the treatment gap caused by geographical inequalities.

Smartphone apps are accessed at the user's discretion, providing control to the user and potentially reducing any perceived power imbalance with professionals. Increasing anonymity can reduce the perceived stigma associated with seeking face-to-face help for alcohol misuse.¹² Examples include Drink Less¹³ and Drinkaware⁶ for the U.K. general population, A-CHESS^14,15 designed to support patients recently discharged from residential alcohol misuse treatment in the United States, and Drinks:Ration^16

–19 designed to support alcohol misuse among U.K. veterans. Although smartphone-based alcohol interventions have been shown to be efficacious,²⁰ they are limited by low levels of engagement and high rates of attrition.^21
–23

Recruitment is one of the most challenging parts of psychological research, with only 20% of clinical trials completed on time, mainly because of participant recruitment challenges.²⁴ For smartphone-based alcohol interventions to be useful, it is important to understand both how and why people engage with research studies of this nature. Social media offers a unique opportunity for recruitment and intervention in health research, and it is becoming a popular recruitment tool due to its wide scope. It is usually easily accessible, making it a viable strategy to reach large populations.²⁵

The most popular social media platforms (e.g., Facebook, Twitter, Instagram) host several 100 million users. For instance, as of 2021, Facebook had >3 billion users worldwide and over 100 billion messages were shared daily.²⁶ This gives it great potential for recruiting participants into research quickly. The ability of social media to target specific populations is a unique advantage over traditional approaches such as flyers, newspaper adverts, and radio adverts.

Previous research has proposed social media as an effective recruitment strategy for research studies.^25,27,28 However, results are often inconsistent with one review reporting social media as the most effective recruitment strategy in only 12 out of 30 included studies, compared to traditional methods.²⁹ Authors of the review described characteristics such as participant age or being a “hard-to-reach” population as influential on the effectiveness of social media as recruitment strategy.²⁹ A review on the use of Facebook for recruitment into health research reported that most studies included in the review recruited young age groups or focused on specific demographics.³⁰

Given the vast number of unique users, social media sites, including Facebook, are turning to monetized advertising. One systematic review exploring the use of Facebook in recruiting participants for health research reported that Facebook advertising (ads) was a successful tool for participant recruitment as it was able to access “hard to reach” populations.³⁰

Facebook ads can target specific groups of people by selecting a set of criteria, for example, demographics such as age, gender, and geographical location.^31,32 The benefits of Facebook ads include shorter recruitment periods and improved participant selection of young and “harder to reach” demographics compared to traditional recruitment methods such as email invitations.³⁰ This may lead to reduced costs because a shorter recruitment period means less staff time. Yet, the use of social media advertising, such as that provided by Facebook, has associated costs so the cost versus yield of this strategy needs to be considered.

It is particularly challenging to recruit into randomized controlled trials (RCTs) because participants must be willing to be assigned randomly to the intervention or control arm of the study, and once recruited, RCTs require commitment to follow-up.²⁸ A recent review suggested that poor recruitment was a key predictor of RCTs being discontinued, noted in over 75% of such trials.³³ Response rates are widely considered to be a key indicator of data quality.³⁴

DRINKS:RATION

The Drinks:Ration app (previously called “InDEx”), a 28-day brief alcohol intervention delivered through a smartphone app, was designed to reduce alcohol misuse among ex-serving military personnel,^16
–18 referred herein as veterans. The Drinks:Ration study is an RCT aiming to assess the efficacy of Drinks:Ration in reducing weekly self-reported alcohol consumption. The planned strategy outlined in the study protocol¹⁹ was to recruit participants through one partner organization, Combat Stress, a U.K. veterans mental health charity. Eligible participants were U.K. Armed Forces veterans, who owned a smartphone, had received formal treatment for mental health symptoms, and drank >14 U.K. units of alcohol per week.

All procedures for the Drinks:Ration study were conducted online, including consent, baseline, and follow-up. Eligible participants were invited to download the Drinks:Ration app onto their smartphone, use the app for a minimum 28-day period, and complete follow-up questionnaires after 28-, 84-, and 168 days of use.

The study aimed to recruit a minimum of 37 participants into both the intervention and control arm, a total of 74 participants. However, to account for attrition at follow-up, we increased the baseline target to 74 participants in each arm, a total of 148 participants. In keeping with online-only trial procedures, participants were recruited through online mechanisms, including email, websites, social media platforms, and social media ads.

The current article aims to explore the use of social media and Facebook ads as an efficient strategy to recruit help-seeking military veterans into alcohol misuse research and to provide information for other researchers considering similar recruitment strategies.

RECRUITMENT STRATEGIES

Recruiting participants into health research should involve strict participant inclusion criteria. Traditional recruitment methods in research include letters and flyers. However, these methods often miss underrepresented populations and can be slow.^35,36 Previous research has considered members of the military to be a “hard to reach” population,^32,37 although research focusing on military populations is growing rapidly. Help-seeking veterans, the target population for the Drinks:Ration study, are considered to be a “hard to reach” population, especially due to the poor rates of help seeking among this population, with only around 50% of veterans seeking help for their mental health.³⁸

Recruitment strategies for Drinks:Ration included recruiting directly from partner organizations; (1) patients engaged in mental health treatment at Combat Stress, and (2) veterans who took part in the King's Centre for Military Health Research Health and Wellbeing Cohort Study. Due to the challenges of recruiting during the COVID-19 pandemic, and with a poor response to recruitment efforts through these partner organizations, we had to consider new approaches to recruitment to meet targets and deadlines.

We contacted several U.K. military and veteran organizations and charities to help us to promote the study. More than 20 promoted the study by sharing details on their social media platforms (including Facebook, Twitter, and Instagram), on the organization/charity website, and, or, in digital newsletters.

Finally, we used Facebook ads to further promote the study ( Appendix Fig. A1). Facebook ads were targeted toward 21–60 year olds living in the U.K. who met specific criteria. To develop and define the criteria for recruitment, we consulted experts in the field. Agreed criteria included the following: (1) interests, such as Royal Air Force, Veterans Day, British Armed Forces, Army, Veterans, Royal Marines, Marines, Navy, Special Forces, Armed Forces, Remembrance Day, Air Force and (2) school/university/employers, such as Royal Navy, The Royal British Legion, British Armed Forces, Royal Marines, HM Armed Forces, Royal Air Force, or British Army.

Overall, Drinks:Ration exceeded its recruitment target (n = 168), with half of the sample (n = 84, 50.0%) being recruited through social media, including Facebook ads. Facebook ads were active for a period of 88 days between January and April 2021. The ads were viewed by a total audience of 29,416 people, of whom 88% were male and approximately half were aged between 55 and 60 years old. Potential participants could register their interest and complete a short eligibility questionnaire through the study website. A total of 3,059 potential participants clicked the link on the advert, and 87 potential participants registered their interest in the study (Fig. 1).

Fig. 1.

Recruitment breakdown for Facebook advertisements.

Of these potential participants, those who met inclusion criteria for the study were then asked to provide a contact email address to receive additional information about the study and instructions on how to download the app. Of those who registered their interest, 48 participants downloaded and signed up to the app (0.2% of total audience) and 27 were eligible for inclusion in the RCT (0.1% of total audience). Reasons for ineligibility included the following: (1) not reporting drinking at least 14 U.K. units of alcohol per week, (2) not having sought formal help for mental health, and (3) not being a veteran of the U.K. Armed Forces. It is not possible to identify the reasons for declining or ineligibility for each individual recruitment strategy, only for the study as a whole. The total cost for Facebook ads was £1139.95 (USD $1567.43), costing £13.10 (USD $18.10) per expression of interest, £23.75 (USD $32.66) per consent, and £42.22 (USD $58.05) per eligible consent for the RCT. Figure 1 displays the statistics for recruitment using Facebook ads.

RETENTION

In total 168 participants signed up to the app and completed the baseline questionnaires (day 0). Of these, 84 were recruited through partner organizations and 84 were recruited through social media. The 1-month (day 28) outcome assessment was completed by 101 participants (60.1%). Of these 61 were from partner organizations and 40 from social media. The 3-month (day 84) outcome assessment was completed by 98 participants (58.3%). Of these, 51 were from partner organizations and 47 from social media. The 6-month (day 168) outcome assessment was completed by 57 participants (33.9%). Of these, 27 were from partner organizations and 30 from social media. In total, 22 participants withdrew participation at various points in the study (11 from partner organizations and 11 from social media).

CHALLENGES FOR RECRUITMENT

When considering all recruitment strategies used in Drinks:Ration, four key groups of challenges to recruitment were encountered: (1) alcohol consumption (not meeting eligibility for trial), for example, nondrinkers, (2) communication and technology related, for example, emails bouncing, technical issues, and unwillingness to use a smartphone, (3) data-related and other barriers, for example, not providing baseline data, and (4) COVID-19. Figure 2 shows the breakdown of reasons given for declining to participate in the study (n = 245) and ineligibility after sign-up (n = 53) across all recruitment strategies.

Fig. 2.

Reasons for declining participation and ineligibility for all recruitment strategies used for the Drinks:Ration study.

Alcohol consumption

For the participants recruited through partner organizations, one common reason for declining to take part in the Drinks:Ration study was either being a nondrinker or consuming less than the amount of alcohol outlined in inclusion criteria (14 U.K. units of alcohol per week). After reading the participant information sheet, these individuals self-disclosed through email that they did not meet the study eligibility criteria and declined the invitation to take part (n = 77, 31.4%). In addition, of those who proceeded to sign up to the app, 31 were excluded from the RCT because they did not meet the required levels of alcohol consumption of 14 U.K. units per week.

This recruitment challenge was unexpected given the literature on drinking culture in the military. Previously, the U.K. Armed Forces have used alcohol to encourage bonding between personnel and to cope with difficult experiences. Alcohol misuse is more common in the U.K. Armed Forces compared to the general population (10% vs. 3%, respectively).³⁹ Given the high prevalence of alcohol misuse in the U.K. Armed Forces, potentially the sample targeted for recruitment may not be representative of the wider U.K. Armed Forces veteran population.

Despite this, in the veteran population more broadly, research suggests that alcohol use among veteran populations is reducing, particularly since the start of the pandemic.^40

–43 This could be one reason why recruiting U.K. help-seeking military veterans with hazardous, and, or harmful alcohol use has been particularly challenging for the Drinks:Ration study.

Communication and technology

When sending participation invitations through partner organizations, one key issue was being noncontactable (n = 97, 40.0%), for instance, having an invalid email address which caused the email to bounce. To account for these communication difficulties, a random sample of 100 potential participants who had not responded to the email invitations were contacted through a postal mail out. However, this only led to 2/100 (2.0%) responses. The total cost of materials required for the postal mail out was £99.67 (USD $137.03); therefore, the cost per consent was £49.84 (USD $68.54). It also required a full day's work by a research assistant (not included in the total cost). Therefore, due to the low yield, a postal mail out was regarded as not time efficient to continue for all other nonresponders.

A few potential participants experienced technology-related challenges (n = 8, 3.3%) such as technical problems when downloading the app and unwillingness to use a smartphone. Despite the fact that the app was available to iOS and Android users, and there was provision of technical support offered to all participants, some experienced technical issues during the downloading and sign-up stage of the study. For those who reported unwillingness to use a smartphone, this was because they did not own a smartphone. Therefore, it is important to consider bias toward those already using smartphones.

Data-related and other barriers

Of all potential participants, 21 (8.5%) did not complete the baseline questionnaires when signing up to the app and therefore were ineligible to proceed with trialing the app in the RCT. Without this information, we would be unable to compare outcome data to baseline data to determine the efficacy of the app. All participants who signed up but did not complete baseline measures were contacted through email to offer support. The reasons for not completing baseline measures remain unclear. However, one potential explanation could be that these individuals were not prepared to spend the time required to complete the baseline measures.

Another obstacle faced during recruitment was participants having previous negative experiences with partner organizations (n = 22, 9.0%). This led them to withdraw their consent to be contacted by the partner organization and therefore decline the invitation to take part in the study. For this reason, social media for recruitment may be advantageous over partner organizations.

COVID-19

When considering challenges to recruitment for the Drinks:Ration study the COVID-19 pandemic may have had the largest impact. Due to the nature of the pandemic, recruitment was challenging, and we had to seek alternative recruitment strategies than were set out in the study protocol.¹⁹ The original recruitment strategy was to recruit patients engaged in mental health treatment at a partner organization, Combat Stress. However, this produced a 3.3% (72/2207) response rate, <50% of the target number of participants required. We then invited a sample of potential participants from another partner organization, the King's Centre for Military Health Research, based at King's College London University. This provided a 3.9% (12/305) response rate, still leaving us short of the target number of participants.

One potential reason for the poor response rates could be digital fatigue. The COVID-19 pandemic with the associated lockdowns and social distancing measures have forced a rapid change in our use of digital technologies, including for education and employment, health care, and even social interaction. For many, the thought of taking on yet another digital based commitment may have been too much. For those invited through partner organizations, some had already been contacted to take part in other research so did not want to take on another commitment.

In addition, recent research on a sample of treatment-seeking military veterans found that symptoms of post-traumatic stress disorder and common mental health difficulties were the most commonly reported to be exacerbated by the pandemic.⁴⁰ A deterioration in mental health caused by the pandemic may be another potential explanation for the low response rates.

LIMITATIONS OF RECRUITMENT STRATEGIES

There are some potential limitations of the recruitment strategies outlined previously. It is important to consider and acknowledge that these limitations may have affected the sample recruited. Recent research suggests that m-health studies show inconsistencies in sample age and are unable to recruit samples representative of geographical and ethnic diversities.⁴⁴

In Drinks:Ration, an effort was made to reach female veterans, as well as male veterans, by contacting female veteran organizations and charities to help advertise the study. In the sample recruited for Drinks:Ration, females represented 7.7% and males represented 92.3% of study participants. In the U.K. Armed Forces, personnel are made up of ∼11% females and 89% males,⁴⁵ so these numbers are already relatively comparable to the wider military community. In addition, as males are at an increased risk of drinking excessively,⁴⁶ it is possible that they may naturally be more highly represented in study samples.

We initially delayed recruitment by 6 months, from April 2020 to October 2020, aiming to minimize the impact of the COVID-19 lockdown. However, social distancing and restrictions to daily living remained in place, and shortly after beginning recruitment, the U.K. entered its second national lockdown. It is therefore important to consider the impact of the lockdown on recruitment for the Drinks:Ration study.

With national lockdown measures in place, many individuals were spending an increasing amount of time online, for instance, an Ofcom Report from 2021 reported that the U.K.'s internet use surged to record levels in 2020, with adults spending on average 3.5 h/day online and ∼1.5 h watching online services, with ∼80% of time spent online using mobile devices.⁴⁷ This may have biased the sample toward individuals who were confident with technology and more willing to participate in a study which required them to download and sign-up to the app themselves.

Internet use increases were more pronounced during the three U.K. national lockdowns.⁴⁷ This surge in internet use over the course of the pandemic may have led to digital fatigue and may begin to explain why we initially struggled to recruit for the study. Unfortunately, we were unable to analyze the demographics or the reasons for nonresponse to invitations from partner organizations to take part in the study. Nonresponse was assumed to be refusal to participate in the study. The nature of the COVID-19 pandemic has created an opportunity to expand and utilize digital technology and shift to digital technology in health care delivery and research.

Another potential limitation of using social media for recruitment is digital exclusion, as it excludes those who do not have access to the internet. Factors which contribute to the risk of digital exclusion include a lack of skills, access, and motivation.⁴⁸ However, the proportion of people with access to the internet is steadily increasing. An ONS Report from 2020 reported that 96% of households in Great Britain had internet access, increasing from 93% in 2019 and 57% in 2006.⁴⁹ Individuals with severe mental illness are also at an increased risk of digital exclusion.⁵⁰ This vulnerable population may be disproportionately reliant on digital technology due to the provision of self-help strategies and the increased risk of self-isolation during the COVID-19 pandemic.⁵⁰ However, some symptoms of severe mental illness may hinder their use of digital technology for this group.

In addition, participant recruitment solely through social media would exclude individuals who do not use social media. A cross-sectional study exploring the profile of U.K. veterans seeking support for mental health difficulties reported that the population were likely to be older (45+ years),⁵¹ and therefore may have been less likely to use social media. Research is beginning to raise the question as to whether samples recruited through social media are representative of the general population.⁵² However, we used multiple recruitment strategies which would help to alleviate this limitation.

Despite the limitations, strengths included exceeding the minimum targets for participant recruitment despite the challenges faced.

INTERPRETATIONS AND RECOMMENDATIONS

Our study contributes to the growing evidence base on the use of social media and Facebook ads for recruiting into health research. This article describes options for evaluating the pathways for recruiting military veterans into alcohol misuse research.

To overcome some of the communication and technology-related challenges, we recommend that future studies should consider problems arising from technical issues and ensure that their technology is compatible with a wide variety of smartphone devices. Some individuals do not own a smartphone, and therefore, it would be appropriate for future research to consider the type of potential participants that might be excluded from these studies, and the impact that their exclusion may have. Future research should consider providing a Web-based version of the app for participants with technical issues and those who do not own a smartphone. It is also important for future research recruiting through social media to consider the impact of previous experiences with partner organizations, which may influence the uptake from the ads.

Future studies should continue to explore the use, opportunities, and barriers of using social media as a recruitment strategy in health research, including the accessibility of technology for participants. In our experience, the use of social media helped us to recruit a large number of eligible participants into our study, minimizing the resources used and the time taken, in comparison to other possible recruitment strategies. Research should also continue to report on the demographics of participants recruited through social media, as well as the cost-effectiveness of different recruitment strategies. This will allow for the assessment of the viability of social media as a recruitment tool.

Discussion and Conclusions

This article aimed to reflect on some key lessons learned from recruitment into the Drinks:Ration study, to provide advice for future studies researching military veterans or other occupational groups with similar characteristics. Overall, targeted Facebook ads were a good strategy for recruiting veterans for an alcohol misuse RCT given the large targeted audience compared to other recruitment strategies.

Our findings add to previous studies assessing the efficacy of social media and Facebook ads. There is growing evidence for the use of Facebook as a useful recruitment tool, and one systematic review has suggested that Facebook should be considered in future health research.³⁰ In Drinks:Ration, Facebook ads produced a vast amount of interest in a trial for a novel alcohol intervention app for military veterans. The fact that the ads were targeted to individuals meeting some of the participant eligibility criteria (i.e., being a veteran of the U.K. Armed Forces), and that potential participants had to complete a short eligibility questionnaire, may have contributed to the success of the recruitment efforts for this study using these methods compared to our other recruitment methods.

To conclude, targeted Facebook ads were an efficient strategy to recruit military veterans into a digital intervention trial aiming to reduce alcohol consumption because they reduced the amount of time and resources required to contact a large number of potentially eligible individuals for our study.

Footnotes

Authors' Contributions

The full team designed the study. D.L. conducted participant recruitment and data collection. C.W. wrote the first draft, and all other authors contributed to each version and approved the final article.

Disclosure Statement

D.M. is a trustee for the Forces in Mind Trust (the funder of the Drinks:Ration project). N.T.F. is a trustee of a charity supporting the well-being of veterans and their families.

Funding Information

The Drinks:Ration study was funded by the Forces in Mind Trust (Grant number: FIMT/0323KCL), a funding scheme run by the Forces in Mind Trust using an endowment awarded by the National Lottery Community Fund. The funding organization had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the article; and decision to submit the article for publication.

Appendix

References

Hutchings

Key points The impact of Covid-19 on the use of digital technology in the NHS. 2020. Available at https://www.nuffieldtrust.org.uk/files/2020-08/the-impact-of-covid-19-on-the-use-of-digital-technology-in-the-nhs-web-2.pdf (last accessed June 22, 2021 ).

Webster

. Virtual health care in the era of COVID-19. Lancet, 2020; 395:1180–1181.

UK Government. Health Secretary: driving digital in the NHS. 2021. Available at https://www.gov.uk/government/news/health-secretary-driving-digital-in-the-nhs (last accessed February 7, 2022 ).

Wickersham

, Petrides

, Williamson

, Leightley

. Efficacy of mobile application interventions for the treatment of post-traumatic stress disorder: A systematic review. Digit Health, 2019; 5:2055207619842986.

McKay

, Wright

, Shill

, Stephens

. Using health and well-being apps for behavior change: A systematic search and rating of apps. JMIR mHealth uHealth, 2019; 7:e11926.

Attwood

, Parke

, Larsen

, Morton

. Using a mobile health application to reduce alcohol consumption: A mixed-methods evaluation of the drinkaware track & calculate units application. BMC Public Health, 2017; 17:394.

Garnett

, Crane

, Michie

, West

, Brown

. Evaluating the effectiveness of a smartphone app to reduce excessive alcohol consumption: Protocol for a factorial randomised control trial. BMC Public Health, 2016; 16:536.

NHS. Alcohol Misuse. NHS Online. 2018. Available at https://www.nhs.uk/conditions/alcohol-misuse (last accessed April 19, 2021 ).

UK Government. UK Chief Medical Officers' Low Risk Drinking Guidelines. 2016. 2016 . Available at https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/545937/UK_CMOs__report.pdf (last accessed April 19, 2021 ).

10.

NHS. Alcohol misuse—Treatment. NHS Digital. 2018. Available at https://www.nhs.uk/conditions/alcohol-misuse/treatment (last accessed July 30, 2021 ).

11.

Cunningham

, Kypri

, McCambridge

. The use of emerging technologies in alcohol treatment. Alcohol Res Health, 2010; 33:320–326.

12.

Boumparis

, Schulte

MHJ

, Riper

. Digital mental health for alcohol and substance use disorders. Curr Treat Options Psychiatry, 2019; 6:352–366.

13.

Garnett

, Crane

, West

, Michie

. The development of drink less: An alcohol reduction smartphone app for excessive drinkers. Transl Behav Med, 2019; 9:296–307.

14.

McTavish

, Chih

M-Y

, Shah

. How patients recovering from alcoholism use a smartphone intervention. J Dual Diagn, 2012; 8:294–304.

15.

Gustafson

, McTavish

, Chih

M-Y

, et al. A smartphone application to support recovery from alcoholism. JAMA Psychiatry, 2014; 71:566–572.

16.

Leightley

, Puddephatt

, Jones

, et al. A smartphone app and personalized text messaging framework (InDEx) to monitor and reduce alcohol use in ex-serving personnel: development and feasibility study. JMIR mHealth uHealth, 2018; 6:e10074.

17.

Puddephatt

, Leightley

, Palmer

, et al. A qualitative evaluation of the acceptability of a tailored smartphone alcohol intervention for a military population: Information about drinking for ex-serving personnel (InDEx) app. JMIR mHealth uHealth, 2019; 7:e12267.

18.

Leightley

, Puddephatt

, Goodwin

, Roberto

, Fear

. InDEx: Open source iOS and android software for self-reporting and monitoring of alcohol consumption. J Open Res Softw, 2018; 6:13.

19.

Leightley

, Rona

, Shearer

, et al. Evaluating the efficacy of a mobile app (Drinks:Ration) and personalized text and push messaging to reduce alcohol consumption in a Veteran population: Protocol for a randomized controlled trial. JMIR Res Protoc, 2020; 9:e19720.

20.

Marcolino

, Oliveira

JAQ

, D'Agostino

, Ribeiro

, Alkmim

MBM

, Novillo-Ortiz

. The impact of mHealth interventions: Systematic review of systematic reviews. JMIR mHealth uHealth, 2018; 6:e23.

21.

Linardon

, Fuller-Tyszkiewicz

. Attrition and adherence in smartphone-delivered interventions for mental health problems: A systematic and meta-analytic review. J Consult Clin Psychol, 2020; 88:1–13.

22.

Garnett

, Crane

, West

, Brown

, Michie

. Identification of behavior change techniques and engagement strategies to design a smartphone app to reduce alcohol consumption using a formal consensus method. JMIR mHealth uHealth, 2015; 3:e73.

23.

Milward

, Deluca

, Drummond

, Kimergård

. Developing typologies of user engagement with the BRANCH alcohol-harm reduction smartphone app: Qualitative study. JMIR mHealth uHealth, 2018; 6:e11692.

24.

Allison

. Can web 2.0 reboot clinical trials?. Nat Biotechnol, 2009; 27:895–902.

25.

Fenner

, Garland

, Moore

, et al. Web-based recruiting for health research using a social networking site: An exploratory study. J Med Internet Res, 2012; 14:e20.

26.

Facebook. Company Info. 2021. Available at https://about.facebook.com/company-info (last accessed May 26, 2021 ).

27.

Ramo

, Prochaska

. Broad reach and targeted recruitment using facebook for an online survey of young adult substance use. J Med Internet Res, 2012; 14:e1878.

28.

Patel

, Doku

, Tennakoon

. Challenges in recruitment of research participants. Adv Psychiatr Treat, 2003; 9:229–238.

29.

Topolovec-Vranic

, Natarajan

. The use of social media in recruitment for medical research studies: A scoping review. J Med Internet Res, 2016; 18:e5698.

30.

Whitaker

, Stevelink

, Fear

. The use of facebook in recruiting participants for health research purposes: A systematic review. J Med Internet Res, 2017; 19:e290.

31.

Cavallo

, Lim

, Ishler

, et al. Effectiveness of social media approaches to recruiting young adult cigarillo smokers: Cross-sectional study. J Med Internet Res, 2020; 22:e12619

32.

Pedersen

, Helmuth

, Marshall

, Schell

, PunKay

, and Kurz

. Using facebook to recruit young adult veterans: Online mental health research. JMIR Res Protoc, 2015; 4:e63.

33.

Briel

, Olu

, von Elm

, et al. A systematic review of discontinued trials suggested that most reasons for recruitment failure were preventable. Clin Epidemiol, 2016; 80:8–15.

34.

Biemer

, Lyberg

. Introduction to survey quality. John Wiley & Sons, 2003. Available at https://books.google.co.uk/books?hl=en&lr=&id=Etcyd75g9SYC&oi=fnd&pg=PR7&ots=hoY-GxKBPv&sig=su-MIKD_9IjRLPtlGPHrZMb_gdk&redir_esc=y#v=onepage&q&f=false (last accessed May 24, 2021 ).

35.

Tate

, LaRose

, Griffin

, et al. Recruitment of young adults into a randomized controlled trial of weight gain prevention: Message development, methods, and cost. Trials, 2014; 15:326.

36.

Frandsen

, Walters

, Ferguson

. Exploring the viability of using online social media advertising as a recruitment method for smoking cessation clinical trials. Nicotine Tob Res, 2014; 16:247–251.

37.

Coll

, Weiss

, Yarvis

. No one leaves unchanged: Insights for civilian mental health care professionals into the military experience and culture. Soc Work Health Care, 2011; 50:487–500.

38.

Iversen

, Dyson

, Smith

, et al. ‘Goodbye and good luck’: The mental health needs and treatment experiences of British ex-service personnel. Br J Psychiatry, 2005; 186:480–486.

39.

Stevelink

SAM

, Jones

, Hull

, et al. Mental health outcomes at the end of the British involvement in the Iraq and Afghanistan conflicts: A cohort study. Br J Psychiatry, 2018; 213:690–697.

40.

Murphy

, Williamson

, Baumann

, Busuttil

, Fear

. Exploring the impact of COVID-19 and restrictions to daily living as a result of social distancing within veterans with pre-existing mental health difficulties. BMJ Mil Heal, 2020; 0:1–6.

41.

Hendrikx

, Williamson

, Baumann

, Murphy

. The impact of the COVID-19 pandemic on treatment-seeking veterans in the United Kingdom with preexisting mental health difficulties: A longitudinal study. J Trauma Stress, 2022; 35:330–337.

42.

Palmer

, Norton

, Jones

, Rona

, Goodwin

, Fear

. Trajectories of alcohol misuse among the UK Armed Forces over a 12-year period. Addiction, 2021; 117:57–67.

43.

Sharp

M-L

, Serfioti

, Jones

, et al. UK veterans' mental health and well-being before and during the COVID-19 pandemic: A longitudinal cohort study. BMJ Open, 2021; 11:e049815.

44.

Pratap

, Neto

, Snyder

, et al. Indicators of retention in remote digital health studies: A cross-study evaluation of 100,000 participants. npj Digit Med, 2020; 3:1–10.

45.

UK Government. UK Armed Forces Biannual Diversity Statistics. 2021. Available at https://www.gov.uk/government/publications/ministry-of-defence-outcome-delivery- (last accessed January 13, 2022 ).

46.

NHS

Digital

. Statistics on

Alcohol

, England 2020. 2020 . Available at https://digital.nhs.uk/data-and-information/publications/statistical/statistics-on-alcohol (last accessed January 13, 2022 ).

47.

Ofcom. Online Nation. 2021. Report. 2021 . Available at https://www.gov.uk/find-digital-market-research/online-nation-report-2021-ofcom (last accessed August 11, 2021 ).

48.

Lloyds Bank. Lloyds Bank UK Consumer Digital Index. 2020. 2020 . Available at https://www.lloydsbank.com/assets/media/pdfs/banking_with_us/whats-happening/211109-lloyds-consumer-digital-index-2020-eds.pdf (last accessed August 11, 2021 ).

49.

ONS. Internet access—households and individuals, Great Britain. 2020. Available at https://www.ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/bulletins/internetaccesshouseholdsandindividuals/2020 (last accessed June 8, 2021 ).

50.

Spanakis

, Peckham

, Mathers

, Shiers

, Gilbody

. The digital divide: Amplifying health inequalities for people with severe mental illness in the time of COVID-19. Br J Psychiatry, 2021; 219:529–531.

51.

Murphy

, Ashwick

, Palmer

, Busuttil

. Describing the profile of a population of UK veterans seeking support for mental health difficulties. J Ment Health, 2017; 28:654–661.

52.

Mellon

, Prosser

. Twitter and Facebook are not representative of the general population: Political attitudes and demographics of British social media users. Res Politics, 2017; 4:1–9.