Designing a brief behaviour change intervention to reduce sexually transmitted infections: a discrete choice experiment

Abstract

To understand whether people attending sexual health (SH) clinics are willing to participate in a brief behavioural change intervention (BBCI) to reduce the likelihood of future sexually transmitted infections (STIs) and to understand their preferences for different service designs, we conducted a discrete choice experiment (DCE) with young heterosexual adults (aged 16–25 years), and men who have sex with men (MSM) aged 16 or above, attending SH clinics in England. Data from 368 participants showed that people particularly valued BBCIs that involved talking (OR 1.45; 95%CI 1.35, 1.57 compared with an ‘email or text’-based BBCIs), preferably with a health care professional rather than a peer. Findings also showed that 26% of respondents preferred ‘email/texts’ to all other options; the remaining 14% preferred not to participate in any of the offered BBCIs. These results suggest that most people attending SH clinics in England are likely to participate in a BBCI if offered, but the type/format of the BBCI is likely to be the single important determinant of uptake rather than characteristics such as the length and the number of sessions. Moreover, participants generally favoured ‘talking’-based options rather than digital alternatives, which are likely to require the most resources to implement.

Keywords

Sexually transmitted infections health services prevention risk reduction behaviour change preferences young people men who have sex with men

Introduction

Sexual behaviour is associated with a continuing and substantial burden of sexually transmitted infections (STIs), with 440,000 cases diagnosed in England, UK in, 2014.¹ The most commonly-diagnosed STIs were Chlamydia, genital warts, gonorrhoea and genital herpes (78% of all infections). Moreover, the number of Chlamydia cases has continued to increase over the past decade and gonorrhoea has become a particular concern as highly-resistant infections are starting to emerge.¹

Young heterosexual adults (aged 16–25 years old) and men who have sex with men (MSM) remain the two groups at greatest risk of STIs in the UK.¹ The impact of this continued high number of new infections is significant in terms of individual health losses, an increased likelihood of further transmissions and costs to the National Health Service (NHS); around £620 million in 2011 excluding new HIV diagnoses.²

The UK Department of Health’s Sexual Health Framework has prioritised prevention and support for behavioural change interventions, alongside access to sexual and reproductive health services, particularly for those most vulnerable to poor sexual health (SH).³ Clinics that provide SH services provide an opportunity to engage those at greatest likelihood of infection in sexual risk reduction interventions^4,5 at a potentially teachable moment. Although challenging, introducing brief sexual risk reduction interventions into busy clinical settings on a large scale is essential if there is to be a population-level impact. Indeed, the UK Government’s White Paper ‘Healthy Lives, Healthy People’ emphasises a commitment to behaviour change interventions as a strategy to reduce preventable illness.⁶ Research has shown they can help people adopt health-promoting behaviour patterns, including safer sex practices^7,8 and that many could be suited to SH clinic settings. However, of the interventions that are known to be acceptable and effective in changing risk and STI testing behaviours such as web-based applications and face-to-face counselling,^9–11 it is unclear which are the most preferred and therefore most likely to be used.¹²

This study is part of the SANTE project which aims to determine the feasibility of conducting a trial of brief behavioural change interventions (BBCIs) in SH services to reduce the incidence of STIs in young people and MSM.¹³ A systematic review¹¹ of BBCIs suitable for use in SH settings identified a variety of effective interventions, such as brief 1:1 counselling and web-based materials. The aims of the work are threefold. To determine the intervention characteristics, of those found effective, that are most likely to encourage people attending SH clinics to participate in a BBCI and the extent to which sociodemographic factors and previous STI testing/treatment are associated with potential use. A further aim was to identify changes to service characteristics that would cause people’s BBCI preference to change.

Methods

We assessed the potential uptake and preferences for different BBCI characteristics using a discrete choice experiment (DCE). DCEs are a method of estimating the relative strength of preference for different service options.¹⁴ It is a hypothetical questionnaire approach that is often used when it is difficult to observe actual choices yet there is desire to understand drivers of consumer demand.¹⁵ DCEs have a theoretical foundation in random utility theory, which is based on the assumptions of economic rationality and utility maximization.¹⁴ In stating a preference, an individual is assumed to choose the option that produces the highest individual benefit (or ‘utility’). Moreover, the utility generated by choosing an option is assumed to depend on the sum of the utilities associated with its composing attribute levels.¹⁶ The results indicate the independent relative strength of preference for each defined service characteristic. Moreover, the preference scores can be combined to demonstrate if a theoretical service configuration is likely to be preferred, and therefore chosen, over others.

This DCE used a cross-sectional design in which participants were asked to choose between four competing BBCIs or to indicate a preference not to participate with any of them (an ‘opt out’ option) (Table 1 and Figure 1).¹⁷ Each BBCI was described according to a number of key characteristics, such as who mediates the intervention, or the length of each session, referred to as ‘attributes’. Each attribute had a number of ‘levels’, e.g. a ‘nurse’ or ‘counsellor’, which were rotated by scenario. Each participant was asked to complete 12 DCE questions.

Table 1.

The list of choices, attributes and levels and their applicability to each option.

	Options
Attribute	Email or text containing health advicea	Online session by yourself	Online group session	Talking with at least one person	Opt out
Type of contact	Emails or texts from a NHS service containing health information	Interactive online information including videos and quizzes	A Facebook Group Chat or Twitter (or similar online social media)	A 1:1 phone conversation, 1:1 face-to-face meeting at a clinic or group face-to-face meeting at a clinic	NA
Type of session	Reading emails/texts	Typing questions and responses	Read/watching online and ticking boxes via a web page or app	Talking	NA
Length of each sessions	NA	Up to 15b, 30 or 60 minutes	Up to 15b, 30 or 60 minutes	Up to 15b, 30 or 60 minutes	NA
Number of sessions	NA	1b, 2–3 or 4–6	1b, 2–3 or 4–6	1b, 2–3 or 4–6	NA
The person who mediates the sessions	NA	NA	A health counsellorb, nurse or peer	A health counsellorb, nurse or peer	NA

NA: not applicable.

^aThe base option for the reported odds ratios.

^bThe base level for each attribute.

Figure 1.

Example of a DCE question. DCE: discrete choice experiment.

Choice of BBCI options, attributes and levels

The BBCI options (labels), attributes and levels were informed by triangulation of data from the formative analysis, which included a qualitative study that explored patient and health care professionals’ (HCP) perspectives of BBCIs and a systematic literature review of BBCIs feasible to deliver in SH settings (Figure 2).¹¹ The qualitative study consisted of semi-structured interviews with 15 men and women aged 16–25 years and 20 MSM who were aged 16 or above, who were recruited from London- and Brighton-based SH clinics and 26 SH providers from across England. The systematic review identified 33 randomised controlled trials (RCTs) that included waiting-room-, self- and brief healthcare provider-delivered interventions.¹¹ Interventions considered for inclusion had evidence supporting their effectiveness and were considered potentially feasible to implement given existing clinical service setups and resources in the UK.

Figure 2.

DCE design and analysis flow chart. DCE: discrete choice experiment.

The four BBCIs included in the final design were ‘talking’ to someone (which was taken to denote talking therapies such as counselling and motivational interviewing), an ‘email or text containing health advice’, an ‘online session by yourself’ or an ‘online group session’ (see Table 1). The attributes included the type of contact, the type of activities involved in each session, the length and number of sessions and, where appropriate, the person who mediates the sessions. The number of sessions (1 to 6) and their length (up to 15 minutes to an hour) were deliberately kept short, as the remit was to investigate preferences for ‘brief’ behaviour change interventions.

Recruitment

Participants were recruited from three SH clinics in Brighton and London, UK during November and December, 2015 and were either heterosexual and aged between 16 and 25 years or a MSM and at least 16 years old, as they are the target groups specified for the project. No other inclusion or exclusion criteria were applied. Participants were recruited by researchers in the clinics, and asked to complete a paper-based version of the instrument. They were also asked to provide background information including educational status, previous STI testing history and previous STI diagnosis. All data were collected anonymously and written informed consent was obtained. In order to assess if our sample contained similar proportions of people in terms of age/sexual orientation, comparisons were made with GUMCADv2 data derived from the recruiting clinics.¹⁸ London Westminster Research Ethics Committee granted ethical approval (REC reference 15/LO/0690).

Sample size and DCE questionnaire design

Specifying sample sizes for DCE studies is difficult because the number of attributes, levels and choice sets is unknown in advance of designing the instrument. Orme, however, has suggested that a reasonable study size is around 300; we aimed to recruit 350.¹⁹

Twenty-four clinic attendees completed a pilot version of the questionnaire, which was generated using an orthogonal approach. The final design was generated using a D-efficient approach with priors from the pilot.¹⁷ Eight versions of the questionnaire were produced in which the ordering of the DCE questions and options differed.

Statistical analysis

All attribute levels were dummy-coded (1 for group membership, 0 otherwise) except when estimating the alternative specific constants (ASCs). These terms represent the extent to which people preferred one of the four BBCI options or opting out when all other factors are disregarded. Effects coding was used for the ASCs to avoid confounding with the attribute base levels on the main attributes.¹⁷ ‘Email or texts’ was used as the reference option in all of the analyses.

Assessing service and participant level characteristics associated with preferences

The DCE responses were analysed using conditional logit (CLOGIT) and latent class models (LCMs). The former is the basic form of analysis and identifies the relative preference of each attribute level but the results are presented for the average participant meaning it may mask important heterogeneities. LCMs also identify attribute level preferences but allow for heterogeneity by grouping respondents into classes that have similar preferences and identifying characteristics associated with probable class membership.²⁰ The class membership characteristics included in this analysis were selected on the basis of potential interest from a policy perspective and were: born in the UK (yes/no), previously diagnosed with an STI (yes/no), age/MSM status (categorised as heterosexual 16–20 years, heterosexual 21–25, MSM 16–25, MSM 26–50 and MSM 51+) and having tested for a STI within the past year (yes/no).

Assessing how participants’ preferences for BBCI format might change if offered alternative service configurations (simulations)

From a policy perspective, it is useful to understand if people’s preferences, and therefore the likelihood they will engage with an intervention, can be changed by offering them different service options. To do this, a number of scenarios were generated using the LCM results to identify the service configurations that would be required for participants in each class to change away from their most preferred BBCI format to the next most valued option.

Results

A total of 368 eligible people completed the questionnaire. Ninety percent (331/368) of respondents completed all 12 DCE questions, resulting in 21,495 DCE observations overall. Seventeen people only completed the first DCE question. Approximately 52% of responses were from Brighton, 59% were male and 62% were born in the UK (Table 2). Comparisons with the GUMCADv2 data suggested that our sample contained higher and lower proportions of MSM aged 16–25 and MSM aged 26–50, respectively, but was similar in most other respects (Table 2).

Table 2.

Demographics for the 368 eligible respondents and GUMCADv2 data.

	Respondents	GUMCADv2 data^a
	N (%)	N (%)
Recruitment location
Brighton	191 (52.0)	–
London	177 (48.0)	–
Born
UK	229 (62.2)	–
Outside UK	114 (40.0)	–
Missing	25 (6.8)	–
Gender
Male	217 (59.0)	–
Female	145 (39.4)	–
Transgender	6 (1.6)	–
Missing	0 (0)	–
Age/gender group (years)
Heterosexual 16–20	67 (18.2)	4025 (17.4)
Heterosexual 21–25	116 (31.5)	8474 (36.4)
MSM 16–25	42 (11.4)	1355 (5.8)
MSM 26–50	92 (25.0)	7222 (31.0)
MSM 50+	24 (6.5)	2190 (9.4)
Missing	27 (7.3)
Previously diagnosed STI
Yes	169 (45.9)	–
No	169 (45.9)	–
Missing	30 (8.2)	–
STI tests in previous year
0	80 (21.7)	–
1	106 (28.8)	–
2	83 (22.6)	–
3+	81 (22.0)	–
Missing	18 (4.9)	–

^aData taken from GUMCADv2 2015 data¹⁸; Chi² test performed on non-missing data, p < 0.0001.

Almost 20% of respondents (71/368) consistently chose one particular BBCI type, either the ‘talking’ (28/71) or email/text based options (34/71). There was minimal evidence to suggest that a particular attribute level dominated participants' choices in so much that only a very small proportion of respondents always chose the option with the shortest duration (up to 15 min, <1%), the fewest number of sessions (one, <1%), organised by nurses (<1%) or by other HCPs (<1%), whenever these options were presented. These findings provide some evidence that participants were willing to ‘trade’ between different BBCI design options, which is an important requirement of DCE studies if they are to be meaningful.

CLOGIT analysis to assess the preferences for service characteristics

The basic CLOGIT model was statistically significant, as it explained more of the variation in the data than a model with no independent variables (likelihood ratio Chi square test, p < 0.0001). McFadden’s pseudo R² was 0.13, indicating that the model fitted the data moderately well²¹ and predicted 41.7% (1793/4299) of choices correctly. The signs on the model coefficients were in a logical order demonstrating credibility in the underlying model. For example, people generally preferred shorter to longer sessions and a smaller to a larger number of sessions.

The results showed that participants generally preferred all BBCI formats to nothing (‘opting out’) but, ‘talking’ was clearly the most favoured option compared with ‘email or texts’ (OR 1.45; 95%CI 1.35, 1.57) (Figure 3). However, ‘email and texts’ were preferred to ‘online group meetings’ (OR 0.42; 95%CI 0.38, 0.47), ‘online 1:1 meetings’ (OR 0.44; 95%CI 0.39, 0.49) and to ‘nothing’ (OR 0.30; 95%CI 0.27, 0.34).

Figure 3.

Results from the conditional logistic (CLOGIT) regression model.F2F: face-to-face.

‘Face-to-face (F2F) group sessions’ were generally less preferred to ‘F2F 1:1 sessions’ (OR 0.66; 95%CI: 0.57, 0.78) or to ‘1:1 phone calls’, although the latter comparison did not achieve statistical significance (OR 0.87; 95%CI 0.73, 1.02) (Figure 3). The analysis also showed that participants generally preferred fewer to more sessions, and shorter sessions were more highly valued than longer sessions. Additionally, the results indicated that participants had a strong preference for sessions to be facilitated by health counsellors rather than peers (OR 0.53; 95%CI 0.46, 0.60), but they were indifferent about the type of HCP (OR 1.02; 95%CI 0.90, 1.15 when health counsellors were compared to nurses).

LCM to allow for heterogeneity in preferences and participant level predictors

The LCM predicted a higher proportion of correct choices than the CLOGIT analysis, 73% (2419/3312) (Figure 4). Three respondent classes were identified. Participants in each class were generally similar in how they valued the number/length of sessions, the choice of facilitator and whether or not meetings were one-to-one or group based. However, they differed markedly in terms of which general BBCI option they preferred. People who were more likely to be in class 1 (60%) clearly favoured ‘talking’ interventions, although all other options were preferred to ‘opting out’. Respondents who were more likely to be in class 2 (14%) had a strong overall preference for ‘opting out’, although their next strongest preference was also for ‘talking’ interventions. Last, those more likely to be in class 3 (26%) expressed a clear preference for ‘email or text’-based BBCIs compared to all other options and were generally indifferent between the three remaining BBCIs and ‘opting out’. None of the sociodemographic variables included in the latent class analysis were predictive of likely class membership.

Figure 4.

Latent class model results.F2F: face-to-face.

Simulations to identify how preferences for BBCI format could be changed by latent class

Latent class group 1

Participants in this class were only likely to prefer the online group or online 1:1 sessions (i.e. the next most favoured options) than ‘talking’ if each online session was much shorter (up to 15 minutes instead of between 30 and 60 minutes if ‘talking’), the number of sessions was as small as possible (a one-off rather than repeated ‘talking’ sessions) and the ‘talking’ option was mediated by a peer rather than any other HCP. In all of the other configurations, the probability of people preferring, and therefore choosing, the ‘talking’ option remained the highest.

Latent class group 2

The likelihood of class 2 members choosing the ‘talking’ instead of their generally preferred ‘opt out’ option only occurred when it consisted of no more than three sessions, each lasting no longer than 30 min and when facilitated by a nurse or health counsellor, not a peer. The possibility of members of this class choosing any of the remaining BBCI options remained negligible in all of the remaining scenarios.

Latent class group 3

No scenarios were identified in which members of this class preferred anything other than the ‘email/text’-based option.

Discussion

The study found that the uptake of BBCIs to prevent STIs could be high if offered to young heterosexual people and MSM attending UK SH clinics. However, it also suggests that the BBCI format (e.g. talking, emails/texts) is individually more likely to influence demand than factors such as the length and number of each session. Importantly, ‘talking’ was the most preferred BBCI format suggesting that personal interaction is important to people.²² None of the assessed sociodemographic variables, such as a previous STI diagnosis, were found to predict choice suggesting, on the basis of these results alone, that there is no reason to tailor intervention formats for specific socioeconomic groups.^22–24

This research was commissioned as part of a project to determine the feasibility of conducting a RCT of BBCIs. This was on the premise that a number of interventions have been shown to be effective but have not been implemented in a standardised or systematic way in the UK's NHS, or on a scale that could have a population-level impact on STI diagnosis rates.¹¹ Further, the research brief was clear that any implementation would have to be provided with little or no additional resources being provided to SH services. In this context, identifying the characteristics of potential BBCIs that result in high levels of acceptability is key. The strong preference for some form of intervention involving talking to a HCP is therefore challenging. However, a number of scenarios were identified in which people who indicated a general preference for ‘talking’-based options instead preferred an online alternative. Moreover, there are also precedents for effective digital interventions being implemented to address public health needs, such as those to support smoking cessation²⁵ and problem alcohol consumption.^26,27 After initial investment, such interventions can be scaled up rapidly with little additional resources. An important determinant of success is ensuring the engagement of potential users, for which this DCE provides useful evidence. Indeed, the results suggest that for the majority of participants, the proposed BBCIs were generally preferred over nothing at all. Thus, they are consistent with the strategy proposed in the SANTE project in which attendees at SH services would be offered a tailored package of BBCIs, identified through the formative work, based on their risk of STIs.²⁸

It is difficult to compare the results of this DCE with those of existing studies. This is largely because other studies have either tended to assess levels of acceptability rather than preferences per se, evaluated a different set of prevention techniques (e.g. condom use and microbicides) or did not focus on SH clinic attendees.^29–31 DCEs move beyond purely assessing acceptability, by generating estimates of the order and relative importance of intervention characteristics. This said, however, interest is increasing in developing BBCIs as digital interventions^32–34 particularly since the advent of social media, to potentially reach people who are not currently engaged with SH services and because they are considered to be comparatively cost efficient. While these studies have involved different types of online BBCIs they have broadly demonstrated they are effective and acceptable to users.^29,35 Our results echo these findings in that they suggest online formats, while not the most preferred options, were preferred by most participants to being offered nothing at all. Together these results suggest that while those already in contact with SH services would prefer to use more resource-intensive BBCIs, this does not discount the possibility that a high uptake of appropriate digital interventions could be achieved.

The strengths of this preference study were that it used a DCE design, which is more realistic than simply asking people what they prefer, and that it was underpinned by a systematic literature review¹¹ and a qualitative study to identify the BBCI formats and service characteristics. However, there were also a number of limitations with it. First, a number of options, such as having videos in clinics and distributing leaflets containing health advice, were excluded from the final DCE design. Videos were omitted on the basis that evidence from the other formative studies suggested that they were unlikely to be acceptable in a mixed clinic setting due to their content, and leaflets were seen as an adjunct to other interventions. Second, our focus was on establishing preferred BBCI characteristics, but they are clearly not the only type of intervention available for preventing STIs. For example, human papillomavirus vaccination³⁶ or HIV pre-exposure prophylaxis with antiretrovirals.³⁷ Finally, while there is evidence to show that results generated by DCEs correlate with actual choices³⁸ they still involve presenting participants with hypothetical choices so that their results should be taken as indicative of preferences rather than firm predictions of demand.

Fourteen percent of respondents (latent class 2) indicated that they would prefer not to participate in any of the BBCIs. The results indicate that shortening the length or number of the sessions and ensuring that, where applicable, they are one-to-one and mediated by HCPs, is unlikely to change their view, unless the ‘talking’ option is extremely brief and also facilitated by a HCP rather than a peer.

In summary, this DCE clearly indicates that most people attending SH clinics in England would be willing to engage with a BBCI if offered but that the choice of format is critical in ensuring maximum uptake. Most people appear to prefer talking-based interventions, particularly when they are 1:1 with HCPs. Although not generally the most preferred options, there does appear to be a role for email/text and digital BBCIs, which are likely to be less costly to implement and easier to scale up. However, the study does not suggest which BBCI is likely to be the most clinically- and cost-effective nor does it consider the content or the theoretical underpinning of the intervention.³⁹

Informed consent was obtained from all individual participants in the study. The London Westminster Research Ethics Committee (REC reference 15/LO/0690) gave ethical review and approval.

Footnotes

Acknowledgements

The authors would particularly like to thank everyone who participated in the study and the staff at the Claude Nicol Centre, Brighton and Sussex University Hospitals, the Mortimer Market Centre and Archway Centre, Central and North West London NHS Foundation Trust.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper presents independent research commissioned by the UKs National Institute for Health Research (NIHR) under its Health Technology Assessment programme (Grant Reference Number 12/191/05). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

References

Public Health England: Infection report volume 9 number 22. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/437433/hpr2215_STI_NCSP_v6.pdf (2015, accessed 22 June 2017).

Development Economics: Unprotected nation: the financial and economic impacts of restricted contraceptive and sexual health services. Available at: http://www.fpa.org.uk/sites/default/files/unprotected-nation-sexual-health-full-report.pdf (2013, accessed 26 April 2017).

Department of Health: a framework for sexual health improvement in England. Available at: https://www.gov.uk/government/publications/a-framework-for-sexual-health-improvement-in-england (2013, accessed 26 April 2017).

Cook

Comer

Wiesenfeld

et al . Alcohol and drug use and related disorders: an underrecognized health issue among adolescents and young adults attending sexually transmitted disease clinics. Sex Transm Dis 2006; 33: 565–570.

Howards

Thomas

Earp

JA.

Do clinic-based STD data reflect community patterns?.

Int J STD AIDS 2002; 13: 775–780.

Department of Health: Healthy lives, healthy people: our strategy for public health in England. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/216096/dh_127424.pdf (2010, accessed 26 April 2017).

Johnson

Carey

Marsh

et al . Interventions to reduce sexual risk for the human immunodeficiency virus in adolescents, 1985–2000. Arch Pediatr Adolesc Med 2003; 157: 381–388.

Kalichman

Carey

Johnson

BT.

Prevention of sexually transmitted HIV infection: a meta-analytic review of the behavioral outcome literature. Ann Behav Med 1996; 18: 6–15.

Albarracin

Gillette

Earl

et al . A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull 2005; 131: 856–897.

10.

Johnson

Scott-Sheldon

Carey

MP.

Meta-synthesis of health behavior change meta-analyses. Am J Public Health 2010; 100: 2193–2198.

11.

Long

Abraham

Paquette

et al . Brief in-service interventions to prevent sexually transmitted infections: a systematic review. Prev Med 2016; 91: 364–382.

12.

Glasgow

Bull

Gillette

et al . Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med 2002; 23: 62–69.

13.

Gilson

Llewellyn

Shahmanesh

et al. Sexual risk reduction interventions for patients attending sexual health clinics; feasibility to conduct an effectiveness trial. Study protocol version 1.0. Available at: https://www.journalslibrary.nihr.ac.uk/programmes/hta/1219105#/ (2014, accessed 26 April 2017).

14.

Ryan

Discrete choice experiments in health care. BMJ 2004; 328: 360–361.

15.

Mangham

Hanson

McPake

How to do (or not to do) … Designing a discrete choice experiment for application in a low-income country. Health Policy Plann 2009; 24: 151–158.

16.

Lancaster

A new approach to consumer theory. J Pol Econ 1966; 74: 132–157.

17.

ChoiceMetrics: Ngene 1.1.2 User manual and reference guide. Version 15/07/2014, 2014.

18.

Public Health England: Genitourinary medicine clinic activity dataset (GUMCADv2). Available at: https://www.gov.uk/guidance/genitourinary-medicine-clinic-activity-dataset-gumcadv2 (accessed 21 July 2016).

19.

Orme

Sample size issues for conjoint analysis. Available at: http://www.sawtoothsoftware.com/download/techpap/samplesz.pdf (2010, accessed 26 April 2017).

20.

Lagarde

Erens

Mays

Determinants of the choice of GP practice registration in England: evidence from a discrete choice experiment. Health Policy 2015; 119: 427–436.

21.

Hauber

Gonzalez

Groothuis-Oudshoorn

et al . Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health 2016; 19: 300–315.

22.

Rietmeijer

CA.

Risk reduction counselling for prevention of sexually transmitted infections: how it works and how to make it work. Sex Transm Infect 2006; 83: 2–9.

23.

National Institute for Health and Care Excellence: Sexually transmitted infections and under-18 conceptions: prevention. Available at: https://www.nice.org.uk/guidance/ph3/resources/sexually-transmitted-infections-and-under18-conceptions-prevention-55452476869 (2007, accessed 26 April 2017).

24.

National Institute for Health and Care Excellence. : HIV testing: increasing uptake among people who may have undiagnosed HIV, 2016.

25.

Free

Knight

Robertson

et al . Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single-blind, randomised trial. The Lancet 2011; 378: 49–55.

26.

Carra

Crocamo

Bartoli

et al . Impact of a mobile E-health intervention on binge drinking in young people: the digital-alcohol risk alertness notifying network for adolescents and young adults project. J Adolesc Health 2016; 58: 520–526.

27.

Champion

Newton

Teesson

Prevention of alcohol and other drug use and related harm in the digital age: what does the evidence tell us?.

Curr Opin Psychiatry 2016; 29: 242–249.

28.

Gilson

Shahmanesh

Abraham

et al . Sexual health risk reduction interventions – the Sante Project. Available at: http://www.nets.nihr.ac.uk/__data/assets/pdf_file/0005/130667/PRO-12-191-05.pdf (2014, accessed 30 June 2016).

29.

French

McCarthy

Baraitser

et al . Young people's views and experiences of a mobile phone texting intervention to promote safer sex behavior. JMIR Res Protoc 2016; 4: e26.

30.

Pachankis

Lelutiu-Weinberger

Golub

et al . Developing an online health intervention for young gay and bisexual men. AIDS Behav 2013; 17: 2986–2998.

31.

Terris-Prestholt

Hanson

MacPhail

et al . How much demand for new HIV prevention technologies can we really expect? Results from a discrete choice experiment in South Africa. PLoS ONE 2013; 8: e83193.

32.

Lelutiu-Weinberger

Pachankis

Gamarel

et al . Feasibility, acceptability, and preliminary efficacy of a live-chat social media intervention to reduce HIV risk among young men who have sex with men. AIDS Behav 2015; 19: 1214–1227.

33.

Mustanski

Garofalo

Monahan

et al . Feasibility, acceptability, and preliminary efficacy of an online HIV prevention program for diverse young men who have sex with men: the keep it up! intervention. AIDS Behav 2013; 17: 2999–3012.

34.

Bailey

Mann

Wayal

et al. Public health research. In: Sexual health promotion for young people delivered via digital media: a scoping review. NIHR Journals Library, Southampton, UK, 2015.

35.

Swendeman

Rotheram-Borus

MJ.

Innovation in sexually transmitted disease and HIV prevention: internet and mobile phone delivery vehicles for global diffusion. Curr Opin Psychiatry 2010; 23: 139–144.

36.

Palefsky

Giuliano

Goldstone

et al . HPV vaccine against anal HPV infection and anal intraepithelial neoplasia. N Engl J Med 2011; 365: 1576–1585.

37.

McCormack

Dunn

Desai

et al . Pre-exposure prophylaxis to prevent the acquisition of HIV-1 infection (PROUD): effectiveness results from the pilot phase of a pragmatic open-label randomised trial. The Lancet 2016; 387: 53–60.

38.

Lancsar

Swait

Reconceptualising the external validity of discrete choice experiments. Pharmacoeconomics 2014; 32: 951–965.

39.

Abraham

Kelly

West

et al . The UK National Institute for Health and Clinical Excellence public health guidance on behaviour change: a brief introduction. Psychol Health Med 2009; 14: 1–8.