Navigating Nightlife Research: Understanding the Impact of Recruitment Methods on Participants’ Reported Characteristics

Abstract

Street-intercept interviews are the gold standard recruitment method in night-time economy research; however, online recruitment methods are an important tool for researchers, particularly when conducting pilot projects or when resources are limited. The two studies presented aim to further our understanding of how key outcome variables relevant to nightlife research vary between samples recruited through different methods. Study 1 compared survey data from a Qualtrics online panel sample (n = 475) and a social media sample (n = 465) on key outcome variables including: frequency of alcohol consumption, pre-drinking and experiences of nightlife-related aggression and harassment in the past 12 months. Study 2 compared experiences of nightlife harassment within the last 3 months across the Qualtrics panel online survey sample and a street-intercept interview sample (n = 223). Analyses were split by participant gender. For Study 1, chi-square analyses indicated that female social media participants were significantly more likely to report experiences of sexual harassment within night-time entertainment precincts within the past 12 months compared to female panel participants (p < .001). Several demographic differences also emerged across samples. In Study 2, female street-interview participants were significantly more likely to report both sexual (p < .01) and physical (p < .001) harassment in night-time entertainment precincts within the last 3 months compared to female panel participants. There were no such differences across methodology type for male participants. The current study demonstrated that the proportion of participants reporting harms within the nighttime economy differed based on the data collection method used. The findings suggest that researchers should be cautious about underreporting females’ experiences of nightlife-related sexual and physical harassment when utilising samples from online panel providers. The findings may assist researchers in selecting the most appropriate participant recruitment method, given their specific research question.

Keywords

research methodology nightlife alcohol and drugs aggression

Introduction

Recruitment methods in survey research have undergone a profound shift in recent decades, with a decline in response rates from traditional methods such as telephone and mail-based surveys (Stedman et al., 2019; Vaske et al., 2024). Meanwhile, the advancement of the internet and social media has allowed for the development of a new wave of participant recruitment practices. Social media websites are online spaces that facilitate information dissemination and peer connection (Gelinas et al., 2017). Recruitment on these platforms can involve paid advertisements, whereby the platform promotes the survey to relevant members. Snowballing is another common recruitment technique, in which participants are asked to share the survey link on their social media accounts in order to identify additional participants (Leighton et al., 2021). Snowballing is a popular recruitment strategy as it requires minimal effort and little to no cost from researchers (Sadler et al., 2010). However, with many researchers utilising social media recruitment, potential respondents may become increasingly less likely to engage with online surveys (Lehdonvirta et al., 2021). In addition, the success of advertisements varies greatly across studies (e.g., Kapp et al., 2013; Shaver et al., 2019), and snowballing may be less successful when the topic is ‘extremely’ personal or sensitive (Waters, 2015). More recently, market research panels (e.g., Qualtrics & MTurk) have become a common participant recruitment method in social psychology research (Belliveau et al., 2022). These organisations facilitate participant recruitment on behalf of the researchers by distributing the survey to large groups of individuals who have previously consented to be solicited for survey recruitment. While panel samples tend to cost more than traditional social media recruitment methods, their increase in usage likely stems from benefits such as fast-data turn-around and the ability to recruit targeted samples (Belliveau et al., 2022)

In night-time economy research, street-intercept interviews are the gold standard participant recruitment method, as they allow for the collection of samples that are locally representative of patrons present in the nightlife environment. Street-intercept interviews directly target the population of interest, have demonstrated a consistently high response rate across studies (77% to 91%; De Andrade et al., 2023; Miller et al., 2016), and allow for the assessment of response rates and bias (Graham et al., 2014). However, such projects are resource-intensive and not necessarily suitable for pilot projects or small research teams. Patrons may also show less willingness to disclose sensitive information (e.g., experiences of harm) in face-to-face interviews, whereas the anonymous nature of online surveys may encourage greater openness and disclosure (Murdoch et al., 2014). For these reasons, online survey recruitment methods are an important and promising tool for examining alcohol use and experiences of harm in the nightlife context.

Research suggests that survey methodology can influence participant responses, with one study finding that participants’ level of reported risky behaviour changed depending on the length of the interview (Coomber et al., 2018). However, little is known about the impact of online recruitment methods on participant characteristics (e.g., sociodemographics, drinking behaviours) compared to street-interviews. Waldron et al. (2020) compared a social media sample to a sample of nightlife patrons and festival attendees and found that estimates of drug use and nightlife engagement were significantly more conservative in the online sample, with the exception of nightclub attendance (Waldron et al., 2020). However, this was a large study (N = 9,682), not representative of smaller pilot studies that might benefit most from online research methods. Additionally, the study did not examine participants’ experiences of harm and did not compare different methods of online data collection. Developing a clearer understanding of the differences in participant characteristics, behaviours and experiences across sample types could help inform the selection of the most appropriate recruitment technique for nightlife researchers, particularly when researchers are unable to recruit a representative nightlife sample.

The current paper aims to address the gaps in existing literature through two studies. Study 1 will compare participant characteristics across an online panel sample and a social media sample, and study 2 will compare the online panel sample with a representative nightlife patron sample. Specifically, the two studies will seek to answer the following research questions:

RQ1 (Study 1). How do online panels and social media samples differ in terms of sociodemographic characteristics, alcohol use, frequency of nightlife visits and experiences of harassment?

RQ2 (Study 2). Do online panel samples and street-intercept interview samples significantly differ in experiences of harassment/aggression?

Study 1

Method

The data utilised in this study came from a larger project around sexual harm in Australian nightlife venues. Ethical approvals were granted from Deakin University Human Research Ethics Committee (2022-284; 2022-112).

Participants and Procedure

Data were obtained from a social media sample and an online panel sample. The data were comparable as an identical base survey was utilised across both samples. For the purpose of the wider study, a small number of additional items were administered to the panel sample only; however, these were positioned after all sections relevant to the present analysis. All key measures analysed were identical across both samples. Individuals were able to participate in either survey if they were over the age of 18 and had visited a nightlife venue at least once in the past 12 months. Both surveys were hosted on the online survey platform, Qualtrics.

Social Media Recruitment

A two-component social media recruitment method was utilised. Social media snowballing occurred whereby researchers disseminated a flyer and link to the survey across Twitter, Facebook and Instagram and encouraged other users to share this information. To increase the sample size, paid advertisements ($500 AUD) across Facebook and Instagram were also employed. Participants who clicked the link on the flyers or advertisements were redirected to the survey. Participants were required to read a plain language statement and provide informed consent before completing the 20-min survey. After survey completion, participants had the option to enter the draw to win 1 of 10 $50 gift vouchers by providing an email address on a separate webpage. Data collection occurred from January 2023 until September 2023, with a total of 587 participants clicking the survey link. Some participants did not consent to the survey (n = 2), consented but did not complete any further questions (n = 14) or had not entered a nightlife venue in the past 12 months (n = 8). Seventy participants were excluded as they had completed less than 38% of the survey. This cut-off was determined based on whether participants had completed at least one of the non-sociodemographic variables of interest. Following the removal of individuals who did not specify their gender and gender-diverse individuals (n = 28), 465 participants (84.5% female) were included in the social media sample. Participants’ ages ranged from 18 to 80 years (Mdn = 23, IQR = 22, 31).

Online Panel Recruitment

The social media sample was smaller and contained fewer male participants than was originally anticipated. As such, for the purpose of the wider project, a second sample was recruited directly from the Qualtrics panel provider service. This constituted a convenience sample, as participants were drawn from an opt-in online panel based on availability, without the use of random sampling or nationally representative demographic quotas. The research team requested approximately 300 males and 150 females (with a natural dispersion of gender diverse individuals). The sampling approach was not intended to achieve an exact gender balance, but rather to increase male representation to ensure their experiences could be meaningfully examined.

The process of participant recruitment, data collection and participant reimbursement were facilitated by Qualtrics. Data collection occurred from September 2023 until October 2023, and the total cost per participant was $10.86 AUD (for 460 full responses). Qualtrics provided the research team with incomplete responses and participants who had been screened out. A total of 632 participants clicked into the survey. A number of participants did not consent to the survey (n = 23), had not been to a nightlife venue in the past 12 months (n = 94) or had completed less than 38% of the survey (n = 23). Following the removal of individuals who did not specify their gender and gender-diverse individuals (n = 14), 475 participants (57.3% male) were included in the panel sample. Participants’ ages ranged from 18 to 65 years (Mdn = 25.5, IQR = 23, 29).

Total Sample

As gender quotas were applied to the panel survey, resulting in a sample that was approximately 60% male, this introduced a non-naturally occurring gender distribution. To account for this, analyses were stratified by gender, and no direct gender comparisons were made between samples. Further, as only a low proportion of the panel sample (2.8%) and the social media (5.6%) sample identified as gender diverse (e.g., transgender or non-binary), meaningful analyses with this subgroup could not be conducted. As such, only individuals who identified as male or female were included in the current study. Overall, 940 (63.4% female) participants, whose ages ranged from 18 to 80 years (Mdn = 25, IQR = 22, 30) were included in Study 1.

Measures

Social Media and Panel Survey Tool

The survey instrument comprised of several sections; the items used for the current study are outlined below.

Sociodemographic

Participants were asked about their gender, age, relationship status, sexuality, sexual partners, employment status, annual income and student status. These were included as categorical variables for analyses.

Physical and Verbal Harassment

Participants were asked if they had experienced verbal harassment (e.g., sworn or yelled at) or physical harassment (e.g., pushing, shoving, choking, slapping, punching) in or around nightlife venues in the past 12 months. Dichotomous variables (0 = no, 1 = yes) were created for both physical harassment and verbal harassment.

Sexual Harassment

Participants were asked if they had experienced verbal sexual harassment (e.g., inappropriate sexual jokes, unsolicited requests for sexual favours) or physical sexual harassment (e.g., pinching of the bottom, groping, fondling, kissing) around nightlife settings in the past 12 months. One sexual harassment dichotomous variable (0 = no, 1 = yes) was created. A value of ‘1’ indicated that participants had experienced either physical or verbal sexual harassment (or both) in the past 12 months.

Alcohol Use

Participants were asked about the frequency with which they consumed alcohol in the past 3 months on a 6-point Likert scale. They were also asked how often they engaged in pre-drinking in the past 3 months on a 5-point Likert scale. These were included as categorical variables.

Analytical Plan

Chi-square analyses were conducted to determine if there were significant differences in participant’s sociodemographic characteristics, alcohol use and experiences of harassment across the social media and panel samples. The analyses were stratified by gender, with the p-value for significance set at < .05. Participant numbers differ across analyses due to incomplete responses from some participants. Missing data primarily resulted from a technical survey error that excluded the age question for a subset of panel participants, and from higher dropout rates towards the end of the survey for social media participants, particularly in the section on alcohol use. Little’s Missing Completely at Random test indicated that the data were not missing completely at random; however, follow-up chi-square analyses showed no significant demographic differences (in age, gender or employment status) between those who completed the survey and those who dropped out, suggesting minimal risk of systematic bias.

Results

Table 1 demonstrates the distribution of sociodemographic and nightlife variables across the social media and panel samples for female participants. Chi-square tests for independence revealed a significant difference in all sociodemographic variables between the female panel sample participants and social media participants, except for their studying status. For example, the panel sample had higher proportions of heterosexual and unemployed individuals. No significant differences were found between females’ frequency of nightlife visits, alcohol use and experiences of physical and verbal harassment across the two samples. However, female participants in the social media sample were significantly more likely to report experiencing sexual harassment compared to those in the panel sample.

Table 1.

Comparison of Female Participants From Social Media and Panel Samples Using Chi-Square Analysis.

Characteristics	Social media	Panel	Total	X ²	p	Cramer’s V
Age				9.2	.027	.13
n	359	201	560
18–21	66 (18.4%)	30 (14.9%)	96 (17.1%)
22–25	124 (34.5%)	59 (29.4%)	183 (32.7%)
26–30	84 (23.4%)	71 (35.3%)	155 (27.7%)
31+	85 (23.7%)	41 (20.4%)	126 (22.5%)
Relationship Status				11.69	.003	.14
n	392	201	593
Single	176 (44.9%)	99 (49.3%)	275 (46.4%)
Non-married relationship	173 (44.1%)	64 (31.8%)	237 (40%)
Married	43 (11%)	38 (18.9%)	81 (13.6%)
Sexual Partners				20.8	< .001	.19
n	384	201	585
No sexual partners	96 (25%)	45 (22.4%)	141 (24.1%)
Casual sexual partners	62 (16.1%)	65 (32.3%)	127 (21.7%)
Monogamous sexual partner	226 (58.9%)	91 (45.3%)	317 (54.2%)
Sexuality				22.16	< .001	.19
n	392	203	595
Heterosexual	233 (59.4%)	153 (75.4%)	386 (64.9%)
Homosexual	10 (2.6%)	10 (4.9%)	20 (3.4%)
Bisexual	105 (26.8%)	31 (15.3%)	136 (22.9%)
Other/Prefer Not Specify	44 (11.2%)	9 (4.4%)	53 (8.9%)
Annual Income				20.97	.002	.19
n	392	203	595
Less than $15,000	44 (11.2%)	16 (7.9%)	60 (10.1%)
$15,000–$40,000	84 (21.4%)	36 (17.7%)	120 (20.2%)
$40,000–$65,000	100 (25.5%)	51 (25.1%)	151 (25.4%)
$65,000–$90,000	93 (23.7%)	36 (17.7%)	129 (21.7%)
$90,000–$150,000	53 (13.5%)	40 (19.7%)	93 (15.6%)
$150,000 +	5 (1.3%)	13 (6.4%)	18 (3%)
Prefer not to say	13 (3.3%)	11 (5.4%)	24 (4%)
Employment Status				26.64	< .001	.21
n	392	203	595
Full-time	184 (46.9%)	91 (44.8%)	275 (46.2%)
Part-time	89 (22.7%)	56 (27.6%)	145 (24.4%)
Casual	88 (22.4%)	19 (9.4%)	107 (18%)
Unemployed	31 (7.9%)	37 (18.2%)	68 (11.4%)
Currently Studying				3.84	.05	.08
n	393	203	596
Yes	188 (47.8%)	80 (39.4%)	268 (45%)
No	205 (52.2%)	123 (60.6%)	328 (55%)
Frequency of nightlife visits				4.98	.28	.09
n	393	203	596
A few times per year	114 (29%)	68 (33.5%)	182 (30.5%)
Once a month	98 (24.9%)	44 (21.7%)	142 (23.8%)
2–3 times per month	118 (30%)	51 (25.1%)	169 (28.4%)
Once a week	46 (11.7%)	25 (12.3%)	71 (11.9%)
2+ times per week	17 (4.3%)	15 (7.4%)	32 (5.4%)
Frequency of alcohol consumption				4.30	.51	.09
n	338	172	510
Never	9 (2.7%)	3 (1.7%)	12 (2.4%)
Less than once a month	28 (8.3%)	21 (12.2%)	49 (9.6%)
1–3 times per month	110 (32.5%)	62 (36%)	172 (33.7%)
Once a week	79 (23.4%)	40 (23.3%)	119 (23.3%)
2–3 times a week	92 (27.2%)	39 (22.7%)	131 (25.7%)
4+ times a week	20 (5.9%)	7 (4.1%)	27 (5.3%)
Frequency of Pre-drinking				5.02	.26	.10
n	309	167	476
Never	64 (20.7%)	29 (17.4%)	93 (19.5%)
Occasionally	118 (38.2%)	74 (44.3%)	192 (40.3%)
Half of the time	30 (9.7%)	23 (13.8%)	53 (11.1%)
Most of the time	49 (15.9%)	20 (12%)	69 (14.5%)
Every time	48 (15.5%)	21 (12.6%)	69 (14.5%)
Verbal Harassment (12 months)				2.23	.14	.06
n	391	203	594
Yes	115 (29.4%)	48 (23.6%)	163 (27.4%)
No	276 (70.6%)	155 (76.4%)	431 (72.6%)
Physical Harassment (12 months)				.29	.59	.02
n	368	203	571
Yes	28 (7.6%)	13 (6.4%)	41 (7.2%)
No	340 (92.4%)	190 (93.6%)	530 (29.8%)
Sexual Harassment (12 months)
n	391	203	594	16.42	< .001	.17
Yes	211 (54%)	74 (36.5%)	285 (48%)
No	180 (46%)	129 (63.5%)	309 (52%)

Notes. All frequency data is reported as n (%). The p-value for significance was set at < .05. For readability, n values are bolded.

Table 2 demonstrates the distribution of sociodemographic and non-sociodemographic variables across the social media and panel samples for male participants. Chi-square tests for independence revealed a significant difference in all sociodemographic variables between the male panel sample participants and male social media participants, except for relationship status and studying status. For example, the social media sample had a greater proportion of older adults (31+) and a lower proportion of heterosexual participants compared to the panel sample. No significant differences were found between males’ frequency of nightlife visits, alcohol use and experiences of harassment across the two samples.

Table 2.

Comparison of Male Participants From Social Media and Panel Samples Using Chi-Square Analysis.

Characteristics	Social media	Panel	Total	X ²	p	Cramer’s V
Age				36.53	< .001	.33
n	67	271	338
18–21	5 (7.5%)	57 (21%)	62 (18.3%)
22–25	19 (28.4%)	90 (33.2%)	109 (32.3%)
26–30	15 (22.4%)	93 (34.3%)	108 (32%)
31+	28 (41.8%)	31 (11.4%)	59 (17.5%)
Relationship Status				4.83	.089	.12
n	72	271	343
Single	31 (43.1%)	156 (57.6%)	187 (54.5%)
Non-married relationship	24 (33.3%)	67 (24.7%)	91 (26.5%)
Married	17 (23.6%)	48 (17.7%)	65 (19%)
Sexual Partners				10.25	.006	.18
n	69	266	335
No sexual partners	20 (29%)	81 (30.5%)	101 (30.1%)
Casual sexual partners	14 (20.3%)	100 (37.5%)	114 (34%)
Monogamous sexual partner	35 (50.7%)	85 (32%)	120 (35.8%)
Sexuality
n	72	271	343	17.25	< .001	.22
Heterosexual	44 (61.1%)	215 (79.3%)	259 (75.5%)
Homosexual	17 (23.6%)	21 (7.7%)	38 (11.1%)
Bisexual	7 (9.7%)	15 (5.5%)	22 (6.4%)
Other/Prefer Not Specify	4 (5.6%)	20 (7.4%)	24 (7%)
Annual Income				13.81	.032	.20
n	72	272	344
Less than $15,000	6 (8.3%)	22 (8.1%)	28 (8.1%)
$15,000–$40,000	9 (12.5%)	38 (14%)	47 (13.7%)
$40,000–$65,000	14 (19.4%)	60 (22.1%)	74 (21.5%)
$65,000–$90,000	17 (23.6%)	66 (24.3%)	83 (24.1%)
$90,000–$150,000	12 (16.7%)	56 (20.6%)	68 (19.8%)
$150,000 +	13 (18.1%)	15 (5.5%)	28 (8.1%)
Prefer not to say	1 (1.4%)	15 (5.5%)	16 (4.7%)
Employment Status				9.48	.024	.16
n	72	271	343
Full-time	46 (63.9%)	160 (59%)	206 (60.1%)
Part-time	6 (8.3%)	49 (18.1%)	55 (16%)
Casual	13 (18.1%)	23 (8.5%)	36 (10.5%)
Unemployed	7 (9.7%)	39 (14.4%)	46 (13.4%)
Currently Studying				2.37	.12	.08
n	71	271	342
Yes	25 (35.2%)	123 (45.4%)	148 (43.3%)
No	46 (64.8%)	148 (54.6%)	194 (56.7%)
Frequency of nightlife visits				2.03	.73	.07
n	72	272	344
A few times per year	20 (27.8%)	56 (20.6%)	76 (22.1%)
Once a month	19 (26.4%)	83 (30.5%)	102 (30%)
2–3 times per month	20 (27.8%)	74 (27.2%)	94 (27.3%)
Once a week	8 (11.1%)	36 (13.2%)	44 (12.8%)
2+ times per week	5 (6.9%)	23 (8.5%)	28 (8.1%)
Frequency of alcohol consumption				10.27	.07	.19
n	58	241	299
Never	1 (1.7%)	3 (1.2%)	4 (1.3%)
Less than once a month	6 (10.3%)	30 (12.4%)	36 (12%)
1–3 times per month	16 (27.6%)	91 (37.8%)	107 (35.8%)
Once a week	7 (12.1%)	51 (21.2%)	58 (19.4%)
2–3 times a week	18 (31%)	45 (18.7%)	63 (21.1%)
4+ times a week	10 (17.2%)	21 (8.7%)	31 (10.4%)
Frequency of Pre-drinking				7.26	.12	.16
n	55	234	289
Never	17 (30.9%)	39 (16.7%)	56 (19.4%)
Occasionally	20 (36.4%)	87 (37.2%)	107 (37%)
Half of the time	5 (9.1%)	43 (18.4%)	48 (16.6%)
Most of the time	9 (16.4%)	45 (19.2%)	54 (18.7%)
Every time	4 (7.3%)	20 (8.5%)	24 (8.3%)
Verbal Harassment (12 months)				.56	.45	.04
n	71	272	343
Yes	29 (40.8%)	98 (36%)	127 (37%)
No	42 (59.2%)	174 (64%)	216 (63%)
Physical Harassment (12 months)				.06	.81	.01
n	63	270	333
Yes	12 (19%)	48 (17.8%)	60 (18%)
No	51 (81%)	222 (82.2%)	273 (82%)
Sexual Harassment (12 months)				.82	.37	.05
n	71	272	343
Yes	17 (23.9%)	52 (19.1%)	69 (20.1%)
No	54 (76.1%)	220 (80.9%)	274 (79.9%)

Notes. All frequency data is reported as n (%). The p-value for significance was set at < .05. For readability, n values are bolded.

Study 2

Method

Participants and Procedure

The study compared data from the online panel sample and a patron street-interview sample. In the street-interviews, participants were asked about their experiences of nightlife-related aggression in the past 3 months. To make the data comparable, questions were added to the panel survey regarding participants’ experiences of each form of harassment in the past 3 months. Overall, 698 participants (53.9% male), whose ages ranged from 18 to 65 years (Mdn = 24, IQR 20, 28) were included in Study 2.

Street-Intercept Interviews

Street-interviews were conducted in Geelong, which is the largest regional city in Victoria, Australia. Interviews took place between 10 pm and 3 am on Saturday nights from December 2022 until February 2023. Teams of four or more research assistants gathered a systematic random sample by approaching approximately every third patron who walked by, or who were queueing for a venue. Data were collected via the offline Qualtrics application on iPhone and iPad devices. In total, 305 patrons were approached to be interviewed, with 73 individuals refusing (76.1% participation rate). For the current study, six participants were excluded as they only partially completed an interview and did not complete the non-sociodemographic questions of interest. Following the removal of three additional participants who did not specify their gender, 223 patrons were included in the street-interview sample (53.4% female), with participants’ ages ranging from 18 to 62 years (Mdn = 20, IQR = 19, 24).

Online Panel Recruitment

Details for online panel recruitment are described in the Method for Study 1.

Measures

Street-Intercept Interview Survey Tool

The street-interview survey tool was derived from prior research (Miller et al., 2013).

Sociodemographic Characteristics

Participants were asked to provide their age and which gender they identified as, with both being included in the analyses as categorical variables.

Verbal and Physical Aggression

Participants were also asked if they had experienced any verbal (e.g., arguments or disputes) or physical aggression (e.g., fights) in or around licensed venues in the past 3 months. Dichotomous variables were created for physical aggression and verbal aggression (0 = no, 1 = yes).

Panel Survey Tool

Full details of the panel survey are provided in the Study 1 Method. Panel participants were also asked about their experiences of physical, verbal and sexual harassment in the past 3 months in order to make the data comparable to the street-intercept interviews.

Analytical Plan

Chi-square analyses were conducted to investigate the differences in age and experiences of harassment/aggression in the past 3 months between the panel sample and the patron sample. The analyses were stratified by gender, and the p-value for significance was set at <.05. Participant numbers varied across the analyses due to incomplete responses from some participants.

Results

Table 3 illustrates the distribution of age and experiences of aggression/harassment across the panel sample and the patron sample. For female participants, chi-square tests for independence revealed a significant difference between age and experiences of nightlife-related physical and sexual harassment/aggression in the past 3 months across the two samples. Notably, females in the patron sample were significantly more likely to report experiencing physical and sexual harassment/aggression in nightlife settings compared to panel participants. For male participants, no significant differences were found in experiences of physical, sexual and verbal harassment/aggression across the two samples.

Table 3.

Comparison of Panel and Street-Interview Participants Using Chi-Square Analysis.

Characteristics	Panel	Street-interview	Total	X ²	p	Cramer’s V
Females
Age				72.17	< .001	.48
n	201	118	319
18–21	30 (14.9%)	68 (57.6%)	98 (30.7%)
22–25	59 (29.4%)	29 (24.6%)	88 (27.6%)
26–30	71 (35.3%)	9 (7.6%)	80 (25.1%)
31+	41 (20.4%)	12 (10.2%)	53 (16.6%)
Verbal Harassment/Aggression (3 months)				.87	.35	.05
n	182	115	297
Yes	60 (33%)	32 (27.8%)	205 (69%)
No	122 (67%)	83 (72.2%)	92 (31%)
Physical Harassment/Aggression (3 months)				13.41	<.001	.21
n	202	115	317
Yes	7 (3.5%)	17 (14.8%)	293 (92.4%)
No	195 (96.5%)	98 (85.2%)	24 (7.6%)
Sexual Harassment (3 months)				9.65	.002	.18
n	203	113	316
Yes	63 (31%)	55 (48.7%)	118 (37.3%)
No	140 (69%)	58 (51.3%)	198 (62.7%)

Males
Age				39.26	< .001	.32
n	271	102	373
18–21	57 (21%)	54 (52.9%)	111 (29.8%)
22–25	90 (33.2%)	28 (27.5%)	118 (31.6%)
26–30	93 (34.3%)	15 (7.5%)	108 (28.9%)
31+	31 (11.4%)	5 (4.8%)	36 (9.7%)
Verbal Harassment/Aggression (3 months)				.27	.60	.03
n	188	104	292
Yes	41 (21.8%)	20 (19.2%)	61 (20.9%)
No	147 (78.2%)	84 (80.8%)	231 (79.1%)
Physical Harassment/Aggression (3 months)				.89	.35	.05
n	267	104	371
Yes	41 (21.8%)	12 (11.5%)	53 (14.3%)
No	226 (84.6%)	92 (88.5%)	318 (85.7%)
Sexual Harassment (3 months)				1.28	.26	.06
n	270	102	372
Yes	47 (17.4%)	23 (22.5%)	70 (18.8%)
No	223 (82.6%)	79 (77.5%)	302 (81.2%)

Notes. All frequency data is reported as n (%). The p-value for significance was set at < .05. For readability, n values are bolded.

General Discussion

Research suggests that the methodology employed for participant recruitment can notably influence the characteristics of the obtained sample (Belliveau et al., 2022). The current studies sought to understand the differences in participant characteristics between (a) a panel sample and a social media sample; and (b) a panel sample and a street-interview sample. The findings from Study 1 indicated several of the measured sociodemographic characteristics differed across the panel and social media samples. However, with the exception of females’ experiences of sexual harassment, there were no differences in alcohol use and experiences of harassment. Further, Study 2 found there were no significant differences in experiences of harassment/aggression for males when comparing the street-interview and panel samples. Although female street-interview participants experienced a greater proportion of physical and sexual harassment in the past 3 months compared to female panel participants. Collectively, the findings illustrate that methods of recruitment can influence certain sociodemographic and non-sociodemographic variables of a sample.

Notably, a greater proportion of female social media participants reported experiencing sexual harassment in the past 12 months compared to panel participants. These findings align with prior comparative methodology research, which found that a social media sample reported significantly higher rates of sexual assault compared to a panel sample (Canan et al., 2022). The findings may be attributed to self-selection bias, as participants with personal experience of nightlife sexual harassment may have been more inclined to participate when they see the advertisement materials on social media (Bethlehem, 2010; Canan et al., 2022). This may be less prevalent in panel recruitment, as panel survey participants are typically compensated for completing surveys (Keusch et al., 2014). As such, their motivation for survey completion may be more financially driven as opposed to having a personal connection to the study topic. Similarly, a greater proportion of the female street-interview sample experienced sexual harassment compared to panel participants. This difference may reflect the relatively low frequency of nightlife attendance among female panel participants, resulting in reduced exposure to nightlife environments where harassment is more likely to occur. Collectively, these findings indicate that researchers should be mindful of under-reporting females’ experiences of sexual harassment when utilising panel samples.

In relation to males’ experiences of nightlife harassment, no such differences emerged across the samples. This is noteworthy, as men are often considered less likely to report on sensitive topics in research, particularly in the presence of peers, where social norms around masculinity may discourage disclosure (Robb, 2022). Although patron interviews were typically conducted around peer groups, the consistency in reported experiences of harassment and aggression across methods suggests that social desirability pressures did not substantially influence male participants’ responses in this study. As no differences in harassment reporting were observed across recruitment types, both social media and panel surveys may be appropriate alternatives for engaging men in night-time economy research when patron interviews are not feasible.

Alcohol use and pre-drinking are common predictors and outcome variables in night-time economy and addiction research (e.g., Cameron et al., 2022; Fung et al., 2021). No differences were identified in the frequency of alcohol use and pre-drinking across the social media and panel samples. These findings are consistent with prior gambling addiction research, which found that a Qualtrics panel sample and a social media recruited community sample did not significantly differ on their level of alcohol use problems (Belliveau et al., 2022). Therefore, when street-interviews are not a viable option for researchers examining alcohol use, both panel and online samples may be an appropriate alternative. Further, in accordance with prior research (Belliveau et al., 2022; Boas et al., 2020; Canan et al., 2022), several sociodemographic differences between the social media and panel samples emerged. Age, relationship status, sexual partners, annual income and employment status varied significantly between the two online samples in the current study, although there were some differences across genders. Similar to prior literature, the social media sample had a higher proportion of older individuals compared to the panel sample (Boas et al., 2020). This may be attributed to the greater representation of younger individuals in some crowdsourcing research panels, with younger adults being more likely to provide their email or mobile number to join such panels (Dasgupta et al., 2023). There was also a higher proportion of unemployed females in the panel sample, which may be due to participants joining these platforms as a means of earning some income through survey participation (Keusch et al., 2014).

Another notable difference was that the panel sample had a higher proportion of heterosexual individuals compared to the social media sample. This may, again, reflect self-selection bias, as non-heterosexual individuals, who experience disproportionate rates of sexual harassment (Dawson et al., 2024), may have been more motivated to participate after seeing advertising materials outlining the study’s focus. However, given that approximately only 3.5% of Australians identify as non-heterosexual (Wilson et al., 2020), both online samples in the current study likely over-represented individuals identifying as lesbian, gay and bisexual. The sociodemographic differences across online samples could guide researchers in selecting the most appropriate method for recruiting targeted populations. For example, when recruiting a non-heterosexual sample, researchers may utilise a social media sample given its demonstrated ability to attract higher proportions of non-heterosexual individuals.

Limitations

The findings should be interpreted in light of the studies’ potential limitations. Firstly, we were unable to compare the street-interview sample to the social media sample due to substantial differences in the questions that were asked. Secondly, when enquiring about experiences of physical and verbal harm, the term ‘aggression’ was used in the street-interviews, whereas ‘harassment’ was used in the online survey tool. This difference in wording could have influenced how participants framed their experiences and, in turn, affected reporting rates across samples. While the use of consistent behavioural examples across recruitment types may have helped guide participant understanding, future research should examine how wording may influence participant responses. Thirdly, as the research team requested quotas for male and female participants for the panel sample, we were unable to compare gender distribution across the different samples. Fourthly, in Study 2, an Australia-wide panel sample was compared to a street-intercept sample recruited from one Australian regional city. As street-intercept sampling is subject to geographical and temporal limitations, and behaviours such as alcohol use are typically higher in regional areas compared to metropolitan areas (Redwood et al., 2024), the findings may not reflect national patterns. Future research could address this by recruiting street-intercept samples from diverse nightlife precincts across Australia.

Conclusion

The current study highlights the importance of participant recruitment methods, as they can impact the sociodemographic and non-sociodemographic characteristics of the sample. The findings have implications for researchers in the night-time economy field, as they may assist researchers in selecting the most appropriate participant recruitment technique, given their specific research question. Caution should be taken when utilising panel samples for sexual and physical nightlife harassment research, as these outcomes are likely under-reported in female panel participants. However, for researchers investigating alcohol use and males’ experiences of nightlife-harassment, panel and social media recruitment may be an appropriate and cost-effective alternative to street-interviews. Findings from the current study also highlight the importance of understanding the limitations and possible biases in data due to differing methodologies when used to develop interventions or policy. Knowledge of how each recruitment method shapes results is crucial to developing robust, evidence-based interventions and legislation.

Footnotes

ORCID iDs

Kira Button

Nicholas Taylor

Kerri Coomber

Ethical Considerations

Ethical approvals were granted from Deakin University Human Research Ethics Committee (2022-284; 2022-112).

Consent to Participate

For street-interviews, respondents gave verbal informed consent, and for the online survey’s respondents provided written informed consent.

Consent for Publication

Not applicable.

Funding

The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: Kira Button is supported through an Australian Government Research Training Program Scholarship.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

Data Availability

The ethics approval for the current study does not permit the dataset to be made publicly available.

Author Biographies

Kira Button, BPsych (Hons), is a third-year PhD candidate in the School of Psychology at Deakin University. Her PhD research is centred on understanding and preventing sexual harm within Australian nightlife venues.

Nicholas Taylor, PhD, is a Research Fellow at the National Drug Research Institute and has worked in the field of alcohol policy research since 2015. Much of his research has focused on evaluating the impact of policies and interventions in nightlife settings on alcohol-related violence.

Kerri Coomber, PhD, is a Senior Research Fellow within the Centre for Drug use, Addictive and Anti-social Behaviour Research in the School of Psychology at Deakin University. She has worked within the area of alcohol-related harms and violence for 16 years.

Dominique de Andrade, PhD, is an ARC Discovery Early Career Researcher Awardee (DECRA) with the Griffith Criminology Institute. She leads a program of research that focuses on harm reduction, with an aim to improve health and justice outcomes for vulnerable populations.

Eric Koukounas, PhD, is a Lecturer in the School of Psychology. Dr Koukounas is an active researcher in the areas of human sexuality, violence and aggression and adult psychopathology.

Peter Miller, PhD, is a Professor of Violence Prevention and Addiction Studies at the School of Psychology, Deakin University. Peter has been awarded over $15m in funding and has published over 300 journal articles, books and peer-reviewed reports, as well as completing five of the largest studies ever conducted into alcohol policy, licensed venues and violence.

References

Belliveau

Soucy

K. I.

Yakovenko

(2022). The validity of qualtrics panel data for research on video gaming and gaming disorder. Experimental and Clinical Psychopharmacology, 30(4), 424–431. APA PsycArticles. https://doi.org/10.1037/pha0000575

Bethlehem

(2010). Selection bias in web surveys. International Statistical Review / Revue Internationale de Statistique, 78(2), 161–188.

Boas

T. C.

Christenson

D. P.

Glick

D. M.

(2020). Recruiting large online samples in the United States and India: Facebook, Mechanical Turk, and Qualtrics. Political Science Research & Methods, 8(2), 232–250. https://doi.org/10.1017/psrm.2018.28

Cameron

M. P.

Miller

P. G.

Roskruge

(2022). Pre-drinking behaviour of people in the night-time economy: Evidence from a street-intercept survey in New Zealand. Drug and Alcohol Review, 41(4), 787–794.

Canan

S. N.

Kaplan

A. M.

Jozkowski

K. N.

(2022). Comparing rates of sexual assault between panel quota and social media samples: Findings across sexual orientation categories. Journal of Interpersonal Violence, 37(21–22), NP21386–NP21399.

Coomber

Chikritzhs

Morgan

Lam

Droste

Mayshak

Curtis

Guadagno

Hyder

Gilmore

Peacock

Bruno

Taylor

Miller

P. G.

(2018). Targeting at-risk samples through brief face-to-face interviews in night-time entertainment precincts. Journal of Substance Use, 23(4), 353–357. https://doi.org/10.1080/14659891.2017.1410239

Dasgupta

Perlman

S. E.

Levanon Seligson

Dumas

S. E.

Seil

Bholanath

(2023). Assessing targeted invitation and response modes to improve survey participation in a diverse New York City panel: Healthy NYC. PLoS One, 18(1), e0280911. https://doi.org/10.1371/journal.pone.0280911

Dawson

Burke

Flack

W. F.

Jr O’Higgins

McIvor

MacNeela

(2024). Patterns of sexual harassment: An intersectional approach to reported victimization in a campus climate survey of students at Irish higher education institutions. Violence Against Women, 30(1), 323–344. https://doi.org/10.1177/10778012231203619

De Andrade

Button

Coomber

Taylor

Ferris

Curtis

Miller

P. G

. (2023). Investigating the relationship between inaccurate blood alcohol concentration estimates, harm and other negative consequences. Drug and Alcohol Review, 42(2), 467–475. https://doi.org/10.1111/dar.13569

10.

Fung

E. C.

Santos

M. G. R.

Sanchez

Z. M.

Surkan

P. J.

(2021). Personal and venue characteristics associated with the practice of physical and sexual aggression in Brazilian nightclubs. Journal of Interpersonal Violence, 36(7–8), NP3765–NP3785. https://doi.org/10.1177/0886260518780783

11.

Gelinas

Pierce

Winkler

Cohen

I. G.

Lynch

H. F.

Bierer

B. E.

(2017). Using social media as a research recruitment tool: Ethical issues and recommendations. The American Journal of Bioethics, 17(3), 3–14. https://doi.org/10.1080/15265161.2016.1276644

12.

Graham

Bernards

Clapp

J. D.

Dumas

T. M.

Kelley-Baker

Miller

P. G.

Wells

(2014). Street intercept method: An innovative approach to recruiting young adult high-risk drinkers. Drug and Alcohol Review, 33(4), 449–455. https://doi.org/10.1111/dar.12160

13.

Kapp

J. M.

Peters

Oliver

D. P.

(2013). Research recruitment using Facebook advertising: Big potential, big challenges. Journal of Cancer Education, 28(1), 134–137. https://doi.org/10.1007/s13187-012-0443-z

14.

Keusch

Batinic

Mayerhofer

(2014). Motives for joining nonprobability online panels and their association with survey participation behavior. In Callegaro

Baker

Bethlehem

Göritz

A. S.

Krosnick

J. A.

Lavrakas

P. J.

(Eds.), Online panel research (1st ed., pp. 171–191). Wiley. https://doi.org/10.1002/9781118763520.ch8

15.

Lehdonvirta

Oksanen

Räsänen

Blank

(2021). Social media, web, and panel surveys: Using non-probability samples in social and policy research. Policy & Internet, 13(1), 134–155. https://doi.org/10.1002/poi3.238

16.

Leighton

Kardong-Edgren

Schneidereith

Foisy-Doll

(2021). Using social media and snowball sampling as an alternative recruitment strategy for research. Clinical Simulation in Nursing, 55, 37–42. https://doi.org/10.1016/j.ecns.2021.03.006

17.

Miller

Droste

De Groot

Palmer

Tindall

Busija

Hyder

Gilham

Wiggers

(2016). Correlates and motives of pre-drinking with intoxication and harm around licensed venues in two cities. Drug and Alcohol Review, 35(2), 177–186. https://doi.org/10.1111/dar.12274

18.

Miller

P. G.

Pennay

Jenkinson

Droste

Chikritzhs

Tomsen

Wadds

Jones

Palmer

Barrie

Lubman

(2013). Patron offending and intoxication in night-time entertainment districts (POINTED): A study protocol. The International Journal of Alcohol and Drug Research, 2(1), 69–76. https://doi.org/10.7895/ijadr.v2i1.74

19.

Murdoch

Simon

A. B.

Polusny

M. A.

Bangerter

A. K.

Grill

J. P.

Noorbaloochi

Partin

M. R.

(2014). Impact of different privacy conditions and incentives on survey response rate, participant representativeness, and disclosure of sensitive information: A randomized controlled trial. BMC Medical Research Methodology, 14(1), 90. https://doi.org/10.1186/1471-2288-14-90

20.

Redwood

Saarinen

Ivers

Garne

De Souza

Bonney

Rhee

Mullan

Thomas

S. J.

(2024). Alcohol consumption and health-related quality of life in regional, rural and metropolitan Australia: Analysis of cross-sectional data from the Community Health and Rural/Regional Medicine (CHARM) study. Quality of Life Research, 33(2), 349–360. https://doi.org/10.1007/s11136-023-03522-x

21.

Robb

(2022). ‘Men, we just deal with it differently’: Researching sensitive issues with young men. International Journal of Social Research Methodology, 24(5), 617–625. https://10.1080/13645579.2020.1857974

22.

Sadler

G. R.

Lee

H.-C.

Lim

R. S.-H.

Fullerton

(2010). Research article: Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy: Hard-to-reach populations. Nursing & Health Sciences, 12(3), 369–374. https://doi.org/10.1111/j.1442-2018.2010.00541.x

23.

Shaver

L. G.

Khawer

Aubrey-Bassler

Etchegary

Roebothan

Asghari

Wang

P. P.

(2019). Using Facebook advertising to recruit representative samples: Feasibility assessment of a cross-sectional survey. Journal of Medical Internet Research, 21(8), e14021. https://doi.org/10.2196/14021

24.

Stedman

R. C.

Connelly

N. A.

Heberlein

T. A.

Decker

D. J.

Allred

S. B.

(2019). The end of the (research) world as we know it? Understanding and coping with declining response rates to mail surveys. Society & Natural Resources, 32(10), 1139–1154. https://doi.org/10.1080/08941920.2019.1587127

25.

Vaske

J. J.

Miller

C. A.

Williams

B. D.

Walberg

(2024). Declining hunter and trapper mail survey response rates between 1976 and 2020 in Illinois. Human Dimensions of Wildlife, 1–10. https://doi.org/10.1080/10871209.2024.2354419

26.

Waldron

Grabski

Freeman

T. P.

Mokrysz

Hindocha

Measham

Van Beek

Van Der Pol

Hauspie

Dirkx

Schrooten

Elgán

T. H.

Feltmann

Benedetti

Tomba

G. S.

Fabi

Molinaro

Gripenberg

Van Havere

. . . Curran

H. V.

(2020). How do online and offline sampling compare in a multinational study of drug use and nightlife behaviour? International Journal of Drug Policy, 82, 102812. https://doi.org/10.1016/j.drugpo.2020.102812

27.

Waters

(2015). Snowball sampling: A cautionary tale involving a study of older drug users. International Journal of Social Research Methodology, 18(4), 367–380. https://doi.org/10.1080/13645579.2014.953316

28.

Wilson

Temple

Lyons

Shalley

(2020). What is the size of Australia’s sexual minority population? BMC Research Notes, 13, 1–6.