Abstract
Objectives
Rural communities in the United States are increasingly becoming epicenters of substance use and related harms. However, best practices for recruiting rural people who use drugs (PWUD) for epidemiologic research are unknown, because such strategies were developed in cities. This study explores the feasibility of web- and community-based strategies to recruit rural, young adult PWUD into epidemiologic research.
Materials and Methods
We recruited PWUD from rural Kentucky to participate in a web-based survey about opioid use using web-based peer referral and community-based strategies, including cookouts, flyers, street outreach, and invitations to PWUD enrolled in a concurrent substance use study. Staff members labeled recruitment materials with unique codes to enable tracking. We assessed eligibility and fraud through online eligibility screening and a fraud detection algorithm, respectively. Eligibility criteria included being aged 18-35, recently using opioids to get high, and residing in the study area.
Results
Recruitment yielded 410 complete screening entries, of which 234 were eligible and 151 provided complete, nonfraudulent surveys (ie, surveys that passed a fraud-detection algorithm designed to identify duplicate, nonlocal, and/or bot-generated entries). Cookouts and subsequent web-based peer referrals accounted for the highest proportion of screening entries (37.1%, n = 152), but only 29.6% (n = 45) of entries from cookouts and subsequent web-based peer referrals resulted in eligible, nonfraudulent surveys. Recruitment and subsequent web-based peer referral from the concurrent study yielded the second most screening entries (27.8%, n = 114), 77.2% (n = 88) of which resulted in valid surveys. Other recruitment strategies combined to yield 35.1% (n = 144) of screening entries and 11.9% (n = 18) of valid surveys.
Conclusions
Web-based methods need to be complemented by context-tailored, street-outreach activities to recruit rural PWUD.
Rural communities in the United States, particularly communities in Appalachia, are increasingly recognized as epicenters of substance use and drug-related harms, fueled by opioid injection, syringe sharing, and a historically weak harm-reduction infrastructure. 1,2 Although communities across the rural–urban continuum have been affected by opioid use and related harms, nonmedical prescription opioid use and heroin use have increased dramatically among young adults in rural areas, particularly in Central Appalachia and Kentucky. 3 -8 In a ranking of US counties’ vulnerability to outbreaks of HIV and hepatitis C virus (HCV) infection among people who inject drugs, 43 of the 54 counties that comprise Appalachian Kentucky ranked in the top 5%. 9 The incidence of newly diagnosed HCV infection is now higher among young adults in rural areas than in any other US population. 10 -12
Despite decades of evidence indicating the need for more research on substance use in rural settings, epidemiologic research and methods development have struggled to keep pace. Although innovative methods have been developed and implemented to recruit people who use drugs (PWUD), most methods were created for and implemented in cities and may not succeed in rural areas. 13 -16 For example, targeted sampling, respondent-driven sampling (RDS), and other methods used to recruit PWUD are most effective when population densities are high, applicable service providers are readily present, and public transportation is accessible. 13 -16 Small population size, challenges with transportation, fear of stigma, lack of anonymity, and complex cultural dynamics can create barriers in identifying, recruiting, and collecting data from active, not-in-treatment PWUD living in rural areas. 17 -19
Novel methods for recruitment, particularly among rural young adults, warrant exploration. For example, RDS has been used successfully to recruit samples of rural PWUD by leveraging peer relationships 20 -22 ; however, these studies did not focus explicitly on young adults, a population with a high prevalence of opioid-related harms in rural areas. RDS is a sampling technique involving the recruitment of index participants (ie, “seeds”) who are given a limited number of printed coupons to refer their peers, who are in turn asked to refer their peers, and so on until the target sample size is reached. 14 However, this traditional form of RDS can be logistically complex and expensive and requires printed recruitment coupons, administrative support for coupon tracking, and local study offices.
Alternatively, web-based RDS (WebRDS) is an effective, low-cost method to recruit hard-to-reach populations in cities. 23 -29 WebRDS is identical to traditional RDS, except it uses e-coupons via text, email, or social media instead of paper coupons distributed in person. 25,28 Studies show that web-based methods can be used to recruit hidden populations, including men who have sex with men, people who have tested positive for sexually transmitted infections, people who use marijuana, and young adults. 23 -29 Web-based methods, including WebRDS, have the potential to eliminate logistical barriers in rural areas that present challenges with other strategies. However, web-based recruitment and data collection have not been tested among rural PWUD.
High rates of internet use among rural adults 30 indicate that web-based methods may be feasible, but the feasibility of WebRDS among rural PWUD is unknown. The objectives of our study were to examine the following: (1) which web- and community-based recruitment methods are most successful in enrolling rural, young adult PWUD into an online survey about substance use and sexual risk behavior, considering 2 dimensions of research method success: number screened and efficiency (ie, number eligible among those screened), and (2) to what extent rural, young adult PWUD will use web-based referral to recruit peers to participate.
Methods
The study team recruited young people who use opioids to get high from August 27, 2017, to August 3, 2018, from 5 rural counties in Appalachian Kentucky to participate in a web-based survey about their sexual and drug-related risk behaviors. Three of the 5 counties rank in the top 5% nationally in vulnerability to an outbreak of HIV and hepatitis C infection among people who inject drugs. 9 According to the 2013 Rural–Urban Continuum Codes 31 and 2010 US Census, 32 all 5 counties were classified as nonmetropolitan and/or rural.
Participant eligibility criteria included being aged 18-35, living in the study area, and having used opioids to get high in the past 30 days. The study focused on opioids in response to the increase in opioid-related overdose deaths in Kentucky. People who were deemed eligible via the online eligibility screening tool (described in detail elsewhere 33 ) were directed to an online survey programmed in Surveygizmo (surveygizmo.com) after completing an online consent form. The survey queried drug use, sexual behavior, drug-related harms, and access to substance use treatment and harm-reduction services. Participants received $30 for participation.
Recruitment Methods for WebRDS Seeds
Recruitment of index participants (ie, seeds) for the WebRDS strategy occurred through several outreach strategies (Table), including distribution of flyers by staff members during project cookouts and walks through neighborhoods (ie, street outreach), postings in community venues, postings by staff members at a field office used for a concurrent substance use study (CARE2HOPE), and postings by community coalition members (eg, local stakeholders). Staff members labeled all recruitment materials with a unique code; people were required to enter this code into online eligibility screening surveys to initiate the screener, allowing staff members to track how participants were recruited. We attempted to use Facebook, Craigslist, and local newspaper advertisements during a previous phase of the study involving qualitative data collection, but efforts were unsuccessful in yielding participants and were therefore not used in the online survey phase. Furthermore, we were concerned that online advertisements would increase the vulnerability to fraud (ie, people from outside the study area attempting to participate).
Recruitment strategies for seeds to initiate web-based peer referral in a study of young adults aged 18-35 who use opioids in rural Appalachia, 2017-2018
Cookouts
We held cookouts as low-threshold (ie, easily accessible, not intimidating) events in which community members, including but not limited to PWUD, could join study staff members for a meal, so that the principal investigators and staff members could build relationships with community members on their own terms and learn about the project. In an area where research was relatively new, cookouts allowed community members to choose whether to engage with the project team. Staff members spoke with passersby and store patrons, offered them free food and drinks, passed out flyers, and described the study. All visitors, whether eligible or not, were welcome to eat and speak with staff members. Staff members established a rapport with community members through casual conversation during cookouts, and community members who attended the cookout often informed others about the project. Typically, within a few hours, eligible community members (ie, young adults who use opioids to get high) would visit the cookout and complete the online survey on study-provided tablets or telephones.
Neighborhood walks
Staff members walked through neighborhoods that local partners suggested as having high rates of drug-related harms (eg, overdoses). Most neighborhoods were mobile home parks varying in size from approximately 5 homes to more than 50 homes; some neighborhoods were small apartment complexes housing 20-100 residents. Research staff members called landlords to request permission to distribute flyers.
Community venues
The study team posted flyers in 134 venues (Table). Flyers posted in venues had pull-tabs with study contact information; at the suggestion of local partners, staff members removed a few pull tabs when they hung flyers to help normalize it. Staff members attempted to recruit participants for the qualitative phase of the study that preceded the online survey phase by installing small yard signs, but yard signs were removed frequently by roadside mowing crews; therefore, we did not use them for survey recruitment.
Field office
The CARE2HOPE field office was staffed by community-based personnel who conduct interviewer-administered surveys among adults aged ≥18 who had used opioids and/or injected any drug to get high in the past 30 days. CARE2HOPE participants are recruited from the same 5 counties included in the present study using traditional RDS. Staff members invited CARE2HOPE participants who consented to be contacted for future research to participate when they visited the office.
Community coalitions
The CARE2HOPE study convened community–academic partnership groups composed primarily of local stakeholders from health agencies, substance use disorder treatment agencies, and social service agencies; leaders from faith-based organizations; and people in recovery from substance use disorder. The study team provided coalition members with flyers to distribute or post in their venues and/or give to clients who may be interested.
Web-Based Peer Referral
The online survey asked, “Would you like to help us recruit new people to take part in the study? Recruiters receive $10 for each person they recruit who is eligible and completes the survey, and you can recruit up to 3 participants into the study.” The survey gave participants who said yes instructions on how and who to recruit and asked whether they wanted to receive their e-coupon via text message and/or email. E-coupons included an invitation to participate, a unique referral code, a URL for the screener, and study contact information.
Peers entered the unique coupon codes at the beginning of the screening; staff members used the codes to link participants and their recruits. Participants could refer an unlimited number of peers but were only reimbursed for up to 3 referrals.
Analysis
Fraud detection
Fraud is a well-established threat to validity in web-based research 34 and, therefore, staff members evaluated entries both during and after data collection. During data collection, 49 surveys were completed within two 24-hour periods, and all of these entries requested incentives be mailed to the same address. Once staff members verified that the address was not capable of housing that many people, staff members flagged data as possibly fraudulent (ie, the same person[s] attempting to complete the survey multiple times) and contacted participants who were using that address for verification. Initially, staff members did not withhold compensation from fraudulent entries because the consent form did not describe that possibility. However, the study team made changes to the consent form after the initial uptick in fraud to allow payment to be withheld. Thereafter, staff members contacted participants suspected of fraud to inform them that their entry was flagged and to request they contact staff members to verify their entry to receive compensation, although none did. Staff members identified an additional 18 entries as fraudulent using strategies (described elsewhere 35 ) that were informed by a fraud-detection algorithm used previously in online research. 36 Staff members identified these entries after data collection had ended; as such, these participants were not contacted. Demographic and behavioral characteristics of the final valid sample (n = 151) are described elsewhere. 35
Recruitment method yield and peer referral
We used SPSS version 25 (IBM Corp) to calculate descriptive statistics of the numbers of online screening survey entries, eligible entries, and complete and nonfraudulent eligible surveys for each recruitment method. We used UCINET version 6.664 (Analytic Technologies) and Netdraw version 2.166 (Analytic Technologies) to construct and visualize the web-based peer-referral chains.
The Emory University Institutional Review Board approved all study procedures, and data were protected by a Federal Certificate of Confidentiality.
Results
Recruitment efforts yielded 410 complete online eligibility screening entries, 234 of which were deemed eligible. Of 234 entries that were eligible, 83 entries were invalid (67 were fraudulent and 16 were only partially complete), leaving 151 (64.5%) valid entries (ie, complete and nonfraudulent; Figure 1).
Eligibility of screening entries among young adults aged 18-35 who use opioids to get high for an online survey about substance use, rural Appalachia, 2017-2018.
Yield of Recruitment Strategies
Project cookouts were the most effective strategy in generating entries into the screening process (Figure 2). A total of 54 people, or an average of 9 people per cookout, completed the eligibility screening survey at the event (range, 3-23). People recruited from cookouts generated 98 additional screening entries through peer referral. In total, of the 410 completed online screening entries, 152 (37.1%) were attributable to cookouts or referrals from people recruited at cookouts. However, staff members determined that 65 (42.8%) of the 152 entries from people recruited via cookouts or their peer referrals were fraudulent. The efficiency (ie, percentage of screening entries that met eligibility criteria) of direct recruitment at cookouts was low (35.2%; 19 of 54 people recruited directly at cookouts were eligible), but few (3 of 19) eligible participants were fraudulent. The efficiency of peer referrals from cookout participants was high (92.9%; 91 of 98 peer referrals were eligible); however, 62 of 91 (68.1%) eligible participants were deemed fraudulent. In total, 45 of 151 (29.8%) entries from the final sample were directly or indirectly attributable to the 6 recruitment cookouts (ie, an average yield of approximately 7.5 valid and eligible survey entries per cookout).
Strategies used to recruit young adults aged 18-35 who use opioids to get high for an online survey about substance use, rural Appalachia, 2017-2018. Social service and health agencies include health departments, substance use disorder treatment centers, social service organizations, medical clinics, and a criminal justice facility. Community businesses include gas stations, laundromats, cash advance stores, pawn shops, vape and tobacco shops, hair salons, department store community bulletin boards, and restaurants. Other peer referrals include referred by peers who were recruited through community coalition members or flyers distributed by staff members at medical clinics. Other includes religious organizations, residential areas (apartment complexes), community events (July 4 event), and contacted staff members via email address posted on study website.
Recruitment from the CARE2HOPE field office and subsequent peer referrals combined to be the second most effective method in generating entries into the online screening process and yielding eligible entries, accounting for 114 (27.8%) of 410 total screening entries and 95 of 234 (40.6%) eligible entries, most (92.6%, n = 88) of which were complete and nonfraudulent. Thus, most (58.3%; 88 of 151) of the final, valid sample was directly or indirectly attributable to recruitment from the CARE2HOPE study.
Other recruitment strategies including flyers posted in traditional and nontraditional community venues and referrals from community partners yielded a small proportion (18 of 151, 11.9%) of the final sample, including peer referrals. Most screening entries from these strategies were ineligible.
Web-Based Peer Referral
Of 120 participants who agreed to help recruit peers, 107 (89.2%) reported knowing at least 1 young adult who used opioids in the target area, but only 24 (20.0%) referred a peer to the study. Of 119 seeds, only 14 started a peer-referral chain (Figure 3).
Web-based peer referral chains involved in recruiting young adults aged 18-35 who use opioids to get high for an online survey about substance use, rural Appalachia, 2017-2018. The figure includes unique identification numbers next to each “seed” participant who began a chain and those who are isolates (ie, who did not begin a chain and were not referred by others) to allow for enumeration of chain and isolate types.
Ten participants referred ≥3 people, 4 referred 2 people, and 10 referred 1 person. Of the 83 participants whose surveys were fraudulent or incomplete, 49 (59.0%) used a referral code from someone who also was classified as having a fraudulent or incomplete survey. We found 5 instances in which participants with fraudulent or incomplete surveys referred people into the study who completed valid surveys.
Demographic and Behavioral Characteristics of Sample
In the final sample (n = 151), the average age of participants was 29, most (n = 93, 61.6%) were male, 146 (96.7%) were white, 149 (98.7%) were non-Hispanic, and 102 (67.5%) had completed high school or a general educational development. Most (n = 107, 70.9%) participants lacked transportation (ie, were unable to do something that they needed to because they did not have a way to get there), and 68 (45.0%) had been homeless (ie, living on the street or in a car, park, abandoned building, or shelter) in the past 6 months. Most (n = 114, 75.5%) reported accessing the internet daily, 24 (15.9%) weekly, and 8 (5.3%) at least once in the past 6 months. Only 2 participants reported not accessing the internet at all in the past 6 months.
The most frequently reported opioids used to get high in the past 6 months were heroin (n = 109, 72.2%), followed by buprenorphine or methadone (n = 92, 60.9%), and prescription pain pills (n = 78, 51.7%). In addition, 89 (58.9%) reported methamphetamine use. Most participants (n = 115, 76.2%) reported that they had injected drugs to get high in their lifetime, and 100 (66.2%) had injected drugs in the past 6 months.
Discussion
Best practices for recruiting PWUD in rural areas for epidemiologic research, particularly online research, are not known. This study explored the feasibility of using various outreach and web-based peer-referral strategies to recruit young adults who use opioids in rural Central Appalachia. Although web-based recruitment methods have been used successfully to recruit hard-to-reach urban populations, our study found that web-based methods need to be complemented by a local field office and/or by context-specific, street-outreach activities to recruit people in rural areas who use opioids.
Cookouts, field office recruitment, and related peer referrals accounted for 88% of the final sample. This study provides evidence that internet access and use, which were prevalent in this sample, are necessary but potentially insufficient in isolation to support web-based research among young adults in rural areas who use opioids. Similarly, flyer-based advertisements in community venues yielded few eligible participants despite a large investment of personnel time and travel resources required to post and restock flyers. In this rural setting, participants’ personal or peer contact with study staff members through cookouts and in field offices were critical to successful recruitment. Anecdotal evidence and feedback from field staff members suggest that personal contact with participants increased rapport, trust, and perceptions of study legitimacy and could have contributed to recruitment success. However, further qualitative research is needed to fully understand reasons for the success of cookouts and field office–based recruitment strategies.
Cookouts yielded the most online screening entries, and although many respondents’ entries were ineligible, participant entries that were eligible led to the successful recruitment of many eligible people through peer referral. Although research has highlighted the centrality of cookouts to rural culture, particularly among low-income families, 37 few studies cite the use of cookouts in research in rural areas. Cookouts have been used in research in rural settings to show appreciation or offer social gatherings for participants 38,39 and have been suggested by participants in rural areas as a potentially useful recruitment strategy. 40 However, they are underused as a recruitment method in rural research. Of note, cookouts seemed to have additional, unanticipated, positive effects. Community members who were not using drugs and who would therefore rarely have an opportunity to interact with staff members become acquainted with staff members and shared valuable insights about the community context. Community partners who staffed the local syringe services program also attended portions of the cookouts, which gave community members an opportunity to learn about the program. Finally, the cookouts seemed to contribute to team-building among staff members, to boost morale when they heard community members express appreciation for the team’s desire to improve health in their community, and to increase staff member comfort in interacting with participants and community members in community settings.
Peer referrals from people who attended cookouts yielded a high number of eligible entries, but a few entries led to peer-referral chains that were later identified as cul-de-sacs of fraud (ie, fraudulent participants attempting to refer ineligible people into the study or to reenroll themselves into the study by sending themselves the peer referral code but at a different email address to avoid detection by staff members). Although these referrals appeared to result in referral chains of fraud in which fraudulent people referred many other fraudulent people, the data should be interpreted with caution because chains may be a product of fraudulent participants attempting to refer themselves into the study (ie, sending themselves the referral code but at a different email address to avoid detection by staff members). In total, the fraud detection algorithm detected 67 cases of fraud, accounting for 28.6% of entries that screened eligible. However, 80% of these cases were attributable to an individual or to a small group of people residing in 1 location who repeatedly tried to reenroll themselves or refer friends. Without this isolated pocket of fraud, the prevalence of fraud in this online sample among people who screened eligible would have been 7.6%, which is substantially lower than that found in a study of young men who have sex with men in nearby Central Kentucky (28.7%). 36
Limitations
This study had several limitations. First, the recruitment of index participants via street outreach and convenience sampling was a departure from the recommended strategy for seed selection in traditional RDS. In traditional RDS, seeds should be people who are highly central to the social network from which the sample is obtained, so that if other assumptions are met, population-level estimates can be obtained. 14 Obtaining population estimates was not an objective of this study; therefore, the criteria for seed recruitment were relaxed. Second, because we conducted this study exclusively in a rural population, we could not determine the extent to which face-to-face contact with staff members varies in its importance to recruitment activities according to where a person resides along the rural–urban continuum.
Conclusion
Recruitment strategies for enrolling community-based samples of PWUD have primarily been developed for and tested in population-dense, urban settings; evidence on the feasibility of these strategies in rural settings is lacking. This study revealed that a combination of street outreach via community cookouts, advertisements in a local field office, and web-based peer referral were successful strategies in recruiting young PWUD in rural areas. Recruitment strategies that involved a personal connection via participants’ or their peers’ contact with study staff members through cookouts and in brick-and-mortar field offices were critical to successful recruitment. More research is needed to more fully understand reasons for the success of cookouts and field office–based recruitment strategies and to understand how this success varies across the rural–urban continuum and demographic groups. Although research involving administrative data and recruitment of treatment- and justice-involved PWUD plays an important role in understanding substance use in rural settings, research involving recruitment of community-based samples of PWUD who are not in treatment is essential to fully understand their health and risk behaviors and access to care.
Footnotes
Acknowledgments
The authors thank the study participants for their input, community researchers (Mary Beth Lawson, Travis Green, and Cindy Jolly) for assistance with survey administration and logistics, Nicole Luisi and Danielle Lambert for technical support with survey programming, and Nadya Prood and Paige Higginson-Rollins for study support. The authors also thank Paige Higginson-Rollins for designing the study logo and leading the dissemination of flyers at the beginning of the study and José Bauermeister for his consultation in the study design.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This study was funded by the National Institute on Drug Abuse (R21 DA042727; principal investigators: Cooper and Young). The CARE2HOPE study was supported by the National Institute on Drug Abuse, Centers for Disease Control and Prevention (CDC), Substance Abuse and Mental Health Services Administration (SAMHSA), and the Appalachian Regional Commission (ARC) (UG3 DA044798; principal investigators: Young and Cooper). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, CDC, SAMHSA, or ARC. The Emory Center for AIDS Research (P30 AI050409; principal investigators: del Rio, Curran, Hunter) provided technical support with survey programming.
