Retention Strategies and Factors Associated with Missed Visits Among Low Income Women at Increased Risk of HIV Acquisition in the US (HPTN 064)

Abstract

Women at high-risk for HIV acquisition often face challenges that hinder their retention in HIV prevention trials. These same challenges may contribute to missed clinical care visits among HIV-infected women. This article, informed by the Gelberg-Andersen Behavioral Model for Vulnerable Populations, identifies factors associated with missed study visits and describes the multifaceted retention strategies used by study sites. HPTN 064 was a multisite, longitudinal HIV seroincidence study in 10 US communities. Eligible women were aged 18–44 years, resided in a census tract/zipcode with high poverty and HIV prevalence, and self-reported ≥1 personal or sex partner behavior related to HIV acquisition. Multivariate analyses of predisposing (e.g., substance use) and enabling (e.g., unmet health care needs) characteristics, and study attributes (i.e., recruitment venue, time of enrollment) identified factors associated with missed study visits. Retention strategies included: community engagement; interpersonal relationship building; reduction of external barriers; staff capacity building; and external tracing. Visit completion was 93% and 94% at 6 and 12 months. Unstable housing and later date of enrollment were associated with increased likelihood of missed study visits. Black race, recruitment from an outdoor venue, and financial responsibility for children were associated with greater likelihood of attendance. Multifaceted retention strategies may reduce missed study visits. Knowledge of factors associated with missed visits may help to focus efforts.

Introduction

The HIV epidemic in the United States (US) is not a generalized epidemic; rather, it is concentrated in subpopulations as defined by geography, poverty, race/ethnicity, and transmission mode.¹ The burden of HIV in women has grown substantially over the past 20 years; the proportion of women newly diagnosed with AIDS has risen from 8% in 1985 to more than 23% in 2010.^2,3 Women constitute roughly one quarter of new HIV infections each year in the US; 85% of those infections are acquired through heterosexual transmission.⁴ Substantial racial disparities exist both for new HIV infections and in HIV-related outcomes. In 2010, the rate of new HIV infections (per 100,000 for the general female adult and adolescent population) among black/African-American females was nearly 20 times the rate in white females.⁴ Moreover, black women living with HIV infection face significant barriers to care and treatment⁵ and experience greater AIDS-related mortality than other women.⁴

Inclusion of women and people of color in research studies has been mandated by the National Institutes of Health⁶ and is a key component to ensuring study results that are generalizable to these populations.⁷ However, those considered most at-risk for HIV in the US may also be less likely to enroll in HIV prevention and treatment trials, due in part to restrictive eligibility criteria, researcher bias that certain individuals may be more difficult to retain, cultural and language barriers, and/or a lack of trust on the part of the potential study participants.^{8

–16} Once enrolled, retention of study participants is essential to study success and accurate interpretation of study results. Notably, many of the same factors associated with increased risk of HIV acquisition among US women (e.g., poverty, race, drug and alcohol use, unstable housing, and psychological distress^17
–19) are also associated with missed study visits among HIV-uninfected women in prospective HIV cohort studies.^20
–22 Substantial attrition may lead to loss of study power, bias, and difficulty in interpretation of study results,⁷ further hampering our ability to not only define HIV incidence among women in the US, but to also identify the key behavioral, sexual network, and environmental characteristics that influence women's vulnerability to HIV acquisition. As a result, it is critical to understand the factors that predict missed study visits so that retention efforts may be tailored accordingly.

Prospective observational HIV cohort studies and longitudinal vaccine preparedness studies (observational cohorts which assessed interest in future trials, but did not include vaccine administration as part of the study design) have had varied success in retaining US women, with 12-month retention rates ranging from 67% to 92%.^8,20

–24 Limited data are available regarding factors associated with missed study visits among women enrolled in prospective HIV studies in the US. The most notable example is the Women's Interagency Study (WIHS), a multisite prospective study of women and HIV in the US established in 1993.²⁵ Analysis of the second wave of women enrolled indicated that, among HIV-uninfected participants, temporary housing, depressive symptoms, moderate alcohol consumption, use of crack/cocaine/heroin, and having a primary care provider were associated with missed study visits at 6 and 12 months of follow-up. In contrast, younger age and employment were associated with greater visit attendance. The relationship between recruitment site and visit attendance varied.²⁰ Analysis of past longitudinal vaccine preparedness studies (which assessed interest in future trials, but did not include vaccine administration as part of the study design) enrolling men and women in the US have demonstrated significant associations between gender, enrollment criteria met, study site, frequency of moving, age, education, race/ethnicity, unemployment, and unstable housing with loss to follow-up.^21
–23 Similarly, women enrolled in HIV treatment trials experience higher rates of antiretroviral therapy (ART) discontinuation than men.^14
–16 The most commonly cited reason for discontinuation is loss to follow-up, though the reasons for this loss to follow-up are unclear.^14,16

Many of the same factors associated with missing study visits among women at increased risk of HIV acquisition have also been associated with missing clinical visits among HIV-infected patients. Just as female gender, being a racial/ethnic minority, psychological distress (e.g., depression), drug use, and younger age, have all been associated with missed study visits in prospective HIV cohort studies, these factors have also been associated amongst HIV-infected patients with missing HIV medical appointments.^5,26
–28 Additional factors, including inadequate social support and lack of health insurance, have also been identified as important contributors to missing clinical visits in women living with HIV, however it is not known whether such factors could also play a role in study visit attendance. Missed medical appointments are associated with poorer health outcomes, increased mortality, and higher viral loads, which have important implications both for individual well-being and HIV transmission to uninfected partners.²⁹

The Gelberg-Andersen Behavioral Model for Vulnerable Populations acknowledges that certain populations (e.g., homeless, substance users) may face specific vulnerabilities and competing needs that may lead to significant barriers in obtaining health care. The Gelberg-Andersen Behavioral Model for Vulnerable Populations describes relationships between predisposing, enabling, and need (i.e., illness) factors specific to vulnerable populations. As described by this model, predisposing characteristics include factors that exist prior to the perception of illness and include sociodemographics (e.g., age, gender, and education), as well as variables that reflect vulnerability, such as psychological distress (e.g., depression) and substance use. Enabling characteristics include factors that may serve as facilitators or barriers to care, such as income, health insurance, and competing needs (e.g., food insecurity). This model has been applied successfully to predict health service utilization (HSU) among homeless women,^30,31 and can be extended to women at enhanced risk for HIV in the US, who share many of the characteristics associated with vulnerable populations (e.g., substance use, marginal housing, psychological distress). This model has not previously been used to assess the associations between predisposing and enabling factors and attendance of clinical trial visits.

The HIV Prevention Trials Network Study (HPTN 064), Women's HIV SeroIncidence Study (ISIS) was a multisite prospective observational study designed to estimate the HIV incidence among women living in areas of the US with prevalent HIV and poverty, and to determine the feasibility of retaining women recruited using venue-based sampling.¹⁷ The current analysis is guided by the Gelberg-Andersen Behavioral Model for Vulnerable Populations,³⁰ and (1) explores associations between baseline predisposing factors, enabling factors, study attributes, and study visit attendance, and (2) describes retention strategies used by study sites.

Methods

Eligibility

Study participants were women between the ages of 18–44 years, who resided in either census tracts or zip codes with higher levels of HIV prevalence and poverty than the surrounding tracts. In addition, to be eligible, women had to report at least one personal and/or partner characteristic associated with greater risk of HIV acquisition (e.g., substance use, history of incarceration, exchange of sex for commodities). Women were excluded from participating if they reported a history of previous positive results on an HIV test, current HIV prevention trial enrollment, current or past participation in an HIV vaccine trial, or anticipated absence for more than 2 consecutive months during follow-up. Women were recruited from community locations using venue-based sampling techniques. The overall study design and methodology has been described elsewhere.^17,32

Study procedures

Women were followed for 6 or 12 months, depending on date of enrollment. Participants received HIV rapid testing and audio computer-assisted self-interviews (ACASI) at baseline and at 6-month intervals for up to 12 months. All participants received monthly phone calls to update locator information. Women were compensated for both in-person follow-up visits and phone locator-update calls. The amount of compensation varied by site and was approved by local Institutional Review Boards.

Sites

HPTN 064 participants were enrolled from one of ten study communities across six geographic areas. These sites included: New York City, New York (The Bronx and Harlem), Newark, New Jersey (North and South Newark), Washington, DC, Baltimore, Maryland, North Carolina (Durham and Wake Counties), and Atlanta, Georgia (Atlanta and Decatur). A Certificate of Confidentiality was obtained for the study and study sites obtained local Institutional Review Board approval prior to initiation of study activities.

Definitions of visit nonattendance and retention

The primary outcome of interest for these analyses was study visit attendance. A missed study visit was defined as any in-person study visit that was not completed within the allowable follow-up visit window. Overall study retention was defined as the proportion of women completing the final scheduled in-person study visit within the allowable study window. Using these definitions, a participant enrolled in the 12-month cohort could miss a visit at 6 months and still be considered retained if the 12-month visit was completed. The study visit window was 12 weeks on either side of the visit for ongoing follow-up visits. Participants who completed their final study visit prior to the end of site follow-up activities were considered retained.

Retention strategies

Study sites used a combination of techniques with the goal of maintaining high rates of study visit completion and to prevent loss to follow-up. All sites implemented a core set of retention strategies designed to develop trusting relationships with study communities and the participants themselves and to reduce barriers to visit completion. These strategies were informed by available literature, HPTN and study site best practices, and ethnographic assessments.^{16,20,33
–35} Sites developed additional techniques in response to their participant population (Table 1). Community engagement and retention methods were adapted throughout study implementation in response to observations and ongoing feedback from participants regarding barriers to visit attendance. Strategies were documented throughout the study and aligned with one of five broader categories: community engagement; interpersonal relationship building; reduction of external barriers to participation; staff capacity building; and external tracing approaches.

Table 1.

Retention Strategies Utilized by Each Study Site

	Centralized ^a	Bronx, NY	Harlem, NY	North & South Newark, NJ	Baltimore, MD	Washington, DC	Durham &Raleigh, NC	Atlanta/Decatur & Atlanta, GA
Community engagement
Community Working Group (CWG)	X	X	X	X	X	X	X	X
Community Advisory Board (CAB)	X	X	X	X	X	X	X	X
Community and stakeholder partnerships		X	X	X	X	X	X	X
Study logo and branding	X	X	X	X	X	X	X	X
Volunteering within community						X	X	X
Participating in community events and patronizing businesses (e.g., shopping, eating)		X	X	X	X	X	X	X
Bus ads							X
Flyers			X	X	X	X	X	X
Revisiting recruitment venues		X	X	X	X	X	X	X
Interpersonal relationship building
Locator information forms	X	X	X	X	X	X	X	X
Monthly phone calls	X	X	X	X	X	X	X	X
Reminder letters and postcards		X	X	X	X	X	X	X
Personalized notes (e.g., thank you, birthday, holiday)		X	X	X	X	X	X	X
Social media (e.g., Facebook)						X	X	X
Retention event (e.g., “spa day”, luncheon)				X				X
Branded study items (e.g., key chain, t-shirt)		X		X	X	X	X	X
Reduction of external barriers to participation
Same day screening and enrollment		X	X	X		X	X	X
Home visits		X	X	X	X	X	X	X
Appointments outside traditional work day			X			X	X	X
Short patient visits			X	X	X	X		X
Childcare (informal)		X	X	X	X		X	X
Community-based visit location		X	X	X	X	X	X	X
Mobile van				X	X	X
Toll-free phone line	X	X	X	X	X	X	X	X
Community resource guide	X	X	X	X	X	X	X	X
Visit compensation	X	X	X	X	X	X	X	X
Transportation (e.g., metro/bus card, transportation to/from site)		X		X				X
Staff capacity building
Study and site trainings	X	X	X	X	X	X	X	X
Regular team conference calls	X	X	X	X	X	X	X	X
Retention workshop	X	X	X	X	X	X	X	X
Participant transferred or accepted			X				X	X
“Best Practices” consultation with other sites		X	X	X	X	X	X	X
External tracing
Online prison and jail database search		X		X	X	X	X	X
Online death registry search	X	X	X	X	X	X	X	X
Local newspaper obituary search		X		X	X	X		X
Percent visits completed at 6 months	93	97	90	95	96	94	92	91
Percent visits completed at 12 months	94	95	89	97	98	94	95	90
Percent retained (Final Retention)	92	96	87	94	95	94	94	88

Core strategies/protocol-required.

Community engagement

In accordance with HPTN standard operating procedures, a study-wide Community Working Group (CWG) with representatives from each study site was established early in study development. In addition, each site had its own Community Advisory Board (CAB). The CWG and CABs were consulted on a regular basis (e.g., at least monthly meetings) and reviewed study documents during protocol development, study implementation, and results dissemination. Community collaborators identified potential design pitfalls early in study development during a 3-day facilitated protocol review and provided ongoing expertise and consultation throughout the study. The study logo and associated branding was developed through a community-based logo contest and selected through collaboration between the study protocol and site teams and the CWG.

Prior to study initiation, staff conducted an ethnographic review of census tracts/zip codes included in each study community. Each site developed a tailored approach to conducting ethnography based on consultation with local experts and community advisory boards. Ethnographic data collection activities included walking and driving through potential tracts (e.g., “windshield tours”), informal communication with community partners, advisory boards, residents, and business owners, and in some cases, brief interviews and focus groups with women and other key informants. Through this process, study staff developed ongoing partnerships with local community organizations and key community stakeholders (e.g., community advocates, individuals identified as respected neighborhood members through interactions and communications with community members during ethnographic assessments).³⁶ During both the recruitment and retention stages, site staff maintained an active presence at community events and in day-to-day activities by purchasing bus ads, patronizing community businesses, and volunteering at community events. In addition, some staff lived or worked in the study communities and were already familiar with the area. Site staff also regularly revisited recruitment venues and posted study flyers in order to increase visual reminder cues for participants.

Interpersonal relationship-building

Relationship-building began in the early stages of study interaction and included: collection and regular updates of in-depth locator information (including the use of nicknames and aliases), monthly locator update phone calls, letters, and home and “hang-out” venue visits, birthday, holiday, “we miss you” and thank you cards, study branded items (e.g., t-shirts, pens, key chains), and participant appreciation retention events (e.g., spa days, catered lunches with female speakers). Sites obtained advance permission to contact friends and family, and in some cases obtained permission to contact participants through texting and social media, such as Facebook.

Reduction of external barriers to participation

Study visits were designed with the goal of reducing participant burden, and included pre-planning of study activities to ensure shorter visits (standard follow-up visits lasted between 30 and 45 min), site staff availability on nights and weekends, and informal childcare (e.g., toys and snacks for children), community-based study visit sites, and the use of mobile vans (to locate participants and to conduct follow-up visits). Same day screening and enrollment visits were offered to participants; in some cases, site staff escorted participants from directly from the recruitment venue to the clinic in an effort to reduce barriers to enrollment. Each site developed a list of community resources (e.g., social services, food pantries, woman well-care) that was made available to participants at each visit. Referrals were provided as needed. Participants were compensated for time and travel expenses for each in-person visit and for each monthly phone locator update phone call.

Staff capacity building

HPTN 064 consisted of 10 communities across six geographic regions. A minimum of two staff members were required for all retention activities to ensure the safety of study staff. Study field staff were predominantly women of color and were experienced in community-based approaches to clinical research. Throughout the study, regular calls, meetings, and trainings facilitated ongoing site capacity building and information sharing. In particular, site staff participated in biweekly or monthly (depending on the phase of study implementation) cross-site team calls where they shared successes and challenges, and discussed potential strategies accordingly. All study teams participated in a 1-day Retention Workshop, where they presented actual case studies and discussed approaches for locating difficult to contact participants. Site staff consulted each other over the phone as needed, and in some cases, visited sites with high retention rates to share best practices.

The HPTN 064 study population was highly mobile and moved and changed phone numbers frequently. When possible, study sites transferred responsibility for follow-up of participants who moved from one study catchment area into another. A total of eight inter-site transfers took place throughout the study, limiting loss to follow-up due to out of state relocation.

Participant tracing

Detailed locator information was updated monthly, as dictated by protocol-required phone calls. Locator forms included information on participants' own contact information (e.g., home, phone, email, social media identities), as well as contact information for family, friends, frequent hang-outs, and the names/nicknames to use with each contact. Staff obtained advance permission to contact participants using multiple modalities. Participant tracing activities included use of phone, text-message, mail, online, and in person locator contacts (phone calls, email, social media, visiting participant home and hang-outs), as well as monitoring of online jail and prison databases. In addition, following the completion of participant follow-up, each site searched online death registries, such as the Social Security Death Index (SSDI), to determine if any participants who missed their final study visit were deceased.

Analysis

Continuous variables were summarized using medians, and interquartile ranges. Categorical variables were tabulated using frequency and percentage. Multivariate models of baseline factors that predicted missed follow-up visits at 6 and 12 months were based on log-binomial regression (with robust variances). Covariate selection was informed by the Gelberg-Andersen Behavioral Model for Vulnerable Populations and aligned with enabling and predisposing factors outlined by the model (Table 2),³¹ and review of relevant literature identifying factors associated with missed visits.^8,20,21 The primary outcome (utilization measure) was defined as a missed visit (yes/no) at 6 or 12 months. The following independent variables were assessed for their association with missed study visits: (1) predisposing characteristics (within the last 6 months): demographic variables [age (categorical, 18–26, 27–33, 34+), black race (yes/no), education (categorical, less than high school, high school or greater), symptoms of PTSD, defined as a score of 3 or greater on the Primary Care PTSD screener (yes/no),³⁷ symptoms of depression, defined as a score of 7 or greater on a shortened CES-D scale (yes/no),³⁸ history of childhood abuse (yes/no), current abuse, defined of a report of one or more emotional, sexual, physical abuse or feeling unsafe (yes/no), sex exchange, defined as exchange of sex for drugs, money, shelter, or necessities (yes/no) and concurrency [direct (yes/no), indirect (yes/no)], frequency of drug use (categorical, weekly/daily, monthly/less than monthly, or no use of cocaine, amphetamines, inhalants, sedatives, hallucinogens, and/or opioids), binge drinking (categorical, weekly/daily, monthly/less than monthly, or no consumption of four or more drinks in a given time period)], and current housing (categorical, home rented/owned by participant, home rented/owned by spouse/partner, parent's house/someone else's house or apartment, halfway house/treatment center, homeless shelter, motel/hotel/boarding house, street/park/abandoned building/car, other), food insecurity (yes/no), financial responsibility for children (ordinal, 0, 1, ≥2), social support from one or more friends (yes/no); (2) enabling characteristics (within the last 6 months): forgone care, defined as unmet health or medical care needs (yes/no), and self-reported history of an STI (yes/no), (3) study attributes: recruitment location (categorical, commercial venue, outdoor venue, social service organization, public housing, special event, other public space, spaces that serve alcohol), and date of enrollment relative to site initiation of recruitment (continuous).

Table 2.

Participant Baseline Characteristics (in Previous 6 Months)

Variable	n=2099
Predisposing characteristics
Age
18–26	837 (40%)
27–33	502 (24%)
33+	760 (36%)
Race
Black	1802 (86%)
Non-black	297 (14%)
Education
Less than high school education	777 (37%)
High school or greater	1322 (63%)
Physical, emotional, sexual abuse, and/or feeling unsafe^a	811 (39%)
Childhood abuse^b	934 (44%)
Drug use^c
Non-user	1887 (57%)
Monthly/less than monthly use	379 (18%)
Daily/weekly use	459 (22%)
Alcohol use^d
Non-user	767 (37%)
Monthly/less than monthly use	802 (38%)
Daily/weekly use	498 (24%)
Symptoms indicative of depression³⁸ (score ≥7)^e	692 (33%)
Symptoms indicative of PTSD³⁷ (score ≥3)^f	600 (29%)
Direct concurrency (participant)^g	776 (37%)
Indirect concurrency (partner)	763 (36%)
Sex exchange^h	776 (37%)
Housing stability^l
A home that you own or rent yourself	832 (40%)
A home that your spouse/partner owns or rents	173 (8%)
Parent's house/someone else's house or apartment	707 (34%)
Halfway house or treatment center	43 (2%)
Homeless shelter	167 (8%)
Motel, hotel, boarding house	31 (1%)
Street, park, abandoned building, car, etc.	15 (1%)
Other	101 (5%)
Enabling characteristics
Forgone health careⁱ	417 (20%)
Self-reported STI^j	232 (11%)
Reported food insecurity^k	971 (46%)
Number of children financially responsible for^m
0	974 (47%)
1	465 (22%)
≥2	644 (31%)
Study attributes
Recruitment venue
Retail space	388 (18%)
Outdoor location	1215 (58%)
Social service organization	268 (13%)
Public housing	80 (3%)
Special event	66 (3%)
Public space (e.g., library, court, church, public transport)	69 (3%)
Spaces that serve alcohol	13 (1%)

Missing: n=20 (1%); ^bMissing: n=29 (1%); ^cMissing: n=74 (4%); ^dMissing: n=32 (2%); ^eMissing: n=157 (7%); ^fMissing: n=52 (2%); ^gMissing n=9 (<1%); ^hMissing n=21 (1%); ⁱMissing n=3 (<1%); ^jMissing n=33 (2%); ^kMissing n=27 (1%); ^lMissing n=30 (1%); ^mMissing n=16 (1%).

All variables were assessed for the previous 6 months, with the exception of demographics, current housing situation, and history of childhood abuse. Each covariate was fit into a univariate model and factors that were significant at the 0.1 level were included in a multivariate model. Age, substance use, and recruitment venue were forced into the multivariate model. Relative risks and 95% confidence intervals are reported. All analysis was done with SAS version 9.2 (SAS Inc., Cary, NC).

Results

A total of 2099 participants were enrolled for 6 or 12 months of follow-up. Study participant baseline characteristics are in Table 2. The majority of participants were black (86%). The median age was 29 years. Thirty-seven percent had less than a high school education, and 44% percent reported an annual income of $10,000 or less. Baseline binge drinking and illicit substance use in the last 6 months was high; 62% reported binge drinking and 35% reported illicit substance use (excluding cannabis). Thirty-nine percent reported emotional, physical, or sexual abuse or feeling unsafe, in the past 6 months. Condom use at last sex was low (18%). A total of 472 participants were enrolled in the 6-month cohort and 1627 were enrolled in the 12-month cohort. Participant visit completion was 93% at 6 months and 94% at 12 months. Twenty-seven women did not attend the 6-month visit but completed the 12-month visit. Overall study retention was 92%. Ten participants died during follow-up (0.61% per year). Reasons for missed visits are described in Table 3.

Table 3.

Reported Reasons for Study Visit Nonattendance

	Month 6 visit (n=2099)	Month 12 visit (n=1627)
Missed visit^a	139/2099 (7%)	94/1627 (6%)
Reason visit was missed
Unable to contact participant	102/139 (73%)	64/94 (68%)
Participant refused visit	4/139 (3%)	2/94 (2%)
Participant incarcerated	15/139 (11%)	9/94 (10%)
Participant admitted to a health care facility	2/139 (1%)	3/94 (3%)
Participant withdrew from the study	4/139 (3%)	0/94 (0%)
Other	12/139 (9%)	16/94 (17%)

Participant death was considered an endpoint for the study and is not considered a missed visit.

Baseline predisposing characteristics (within the 6 months prior to enrollment) significantly associated (p<0.05) with an increased likelihood of a missed visit in univariate analysis were indirect (partner) concurrency, symptoms of PTSD, social support, and abuse. Enabling characteristics significantly associated (p<0.05) with an increased likelihood of a missed visit included unstable housing (halfway house/treatment center, shelter, motel, or homeless). Later enrollment in the cohort (study attribute) was also associated with increased likelihood of via missed visit (p<0.05). In contrast, black race (predisposing) and financial responsibility for one or more children in the past 6 months (enabling) were associated with an increased likelihood of visit attendance (p<0.05). In multivariate analysis, unstable housing (specifically halfway house/treatment center or being homeless) and later enrollment remained statistically significantly associated with increased risk of a missed visit, while black race, outdoor recruitment venue, and financial responsibility for one or more children in the past 6 months remained associated with an increased likelihood of visit attendance (Table 4).

Table 4.

Multivariate Analysis of Baseline Characteristics and Visit Nonattendance

Variable	RR [95% CI]	p Value
Predisposing Characteristics
Age
18–26^a
27–33	1.293 [0.875, 1.91]	0.1975
34 +	0.847 [0.553, 1.299]	0.4471
Race
Non-black^a
Black	0.614 [0.428, 0.883]	0.0084^b
Education
Less than high school^a
High school or greater	0.820 [0.610, 1.102]	0.1887
Any indirect concurrency in past 6 months
No^a
Yes	1.243 [0.923, 1.674]	0.1519
Symptoms indicative of PTSD³⁷
No^a
Yes	1.016 [0.724, 1.424]	0.9279
Drug use
Non-user^a
Monthly/less than monthly use	0.838 [0.548, 1.283]	0.4162
Daily/weekly use	0.963 [0.642, 1.444]	0.8557
Physical, emotional, sexual abuse, and/or feeling unsafe
No^a
Yes	1.338 [0.980, 1.827]	0.0665
Childhood abuse
No^a
Yes	1.019 [0.738, 1.406]	0.9103
Housing
Home rent/own self^a
A home that your spouse/partner owns or rents	1.149 [0.605, 2.183]	0.6705
Parents house/someone else's house or apartment	1.190 [0.775, 1.828]	0.4272
Halfway house or treatment center	2.100 [1.024, 4.308]	0.0429^b
Homeless shelter	1.615 [0.921, 2.832]	0.0943
Motel, hotel, boarding house	1.491 [0.489, 4.544]	0.4820
Street, park, abandoned building, car	2.829 [1.094, 7.316]	0.0319^b
Other	1.183 [0.593, 2.358]	0.6332
Enabling characteristics
Number of children financially responsible for
0^a
1	0.397 [0.243, 0.646]	0.0002^c
≥2	0.633 [0.405, 0.990]	0.0449^b
Study attributes
Recruitment venue
Commercial business^a
Park/street location	0.620 [0.420, 0.914]	0.0157^b
Social service organization	0.616 [0.374, 1.015]	0.0572
Public housing	0.354 [0.086, 1.466]	0.1522
Special event	1.023 [0.453, 2.307]	0.9568
Public space	0.678 [0.283, 1.622]	0.3826
Spaces that serve alcohol	1.314 [0.364, 4.743]	0.6769
Time since first participant at site enrolled (+30 days)	1.097 [1.040, 1.157]	0.0007^c

Reference Group ^b p value<0.05, ^c p value<0.001.

Discussion

HPTN 064 was designed to determine the feasibility of recruiting and retaining women at increased risk of HIV acquisition in the United States. One in 300 women in the cohort acquired HIV annually,¹⁷ which is substantially higher than the estimates of HIV incidence in US black women of a similar age,³⁹ demonstrating that the study did recruit and enroll women at risk of HIV. HPTN 064 was the first study to enroll women at risk of HIV acquisition in the US using venue-based sampling (VBS) exclusively.³² Although recruitment of women from community-based settings has been identified as a possible barrier to retention in longitudinal cohort studies, the overall retention rate was 92%, demonstrating that it is possible to retain women at high risk for study discontinuation and at enhanced risk of HIV acquisition. HPTN 064 retention strategies were designed to engage the community, build trust with study participants, and to reduce external barriers to study participation. Site collaborations and capacity-building throughout implementation, combined with appropriate allocation of resources, allowed staff to adapt to retention challenges and develop creative approaches to encourage high rates of study participation. External tracing allowed staff to identify participants who were deceased or incarcerated.

HPTN 064 was an observational cohort study and did not include a control group, limiting our ability to assess casual relationships between retention strategies (positive or negative) and visit attendance. However, the HIV Vaccine Trial Network (HVTN) conducted HVTN 906, a study designed to identify a cohort of US women a high risk of HIV infection during roughly the same time period as HPTN 064. This study was conducted in three urban areas (Philadelphia, Chicago, New York City) and similar to HPTN 064, enrolled women based on geographic, personal, and sex partner risk criteria.²⁴ Both studies found a comparable HIV incidence (0.31% in HVTN 906 vs. 0.32% in HPTN 064).^17,40 HVTN 906 retention strategies have not been described in the literature, however, reported visit completion of women enrolled into HVTN 906 was 86% at 6 months and 83% at 12 months. Comparisons between HPTN 064 and HVTN 906 should be interpreted with caution; however, it is possible that the intensive retention efforts applied in HPTN 064 contributed to the higher rates of visit attendance (93% at 6 months, 94% at 12 months).

There is a growing appreciation of the role of community engagement as a critical element of study implementation, and in recruitment and retention in particular.^41

–45 An early vaccine preparedness study in New York designed to assess the feasibility of recruiting and retaining women at risk of HIV acquisition from heterosexual contact reported 67% retention at 12 months follow-up. The retention strategies reported included collection of locator information, incentives, mail and phone appointment reminders, home visits, and street outreach. This same site later achieved 92% follow-up in a similar cohort at 12 months follow-up,^22,23 which they attributed to enhanced retention strategies, including a focus on community involvement. Community engagement was a cornerstone of the HPTN 064 study and study staff were an active presence in study communities.

Women enrolled in HPTN 064 reported extreme poverty, low education levels, prevalent abuse, and high baseline alcohol and substance use. The confluence of these factors highlights the vulnerability and multilevel needs present among many women living in areas with prevalent HIV and poverty. The Gelberg-Andersen Behavioral Model for Vulnerable Populations provides a valuable framework through which to explore predisposing and enabling characteristics associated with visit attendance. For women who become HIV-infected, these same factors which serve as barriers or facilitators to study visit attendance are likely to also apply to their likelihood of HIV medical care visit attendance.¹⁶ Knowledge of characteristics that predict missed visits, as well as those that are protective, may help staff focus retention efforts and to identify participants in need of enhanced outreach.

Similar to past studies, this study found that less stable housing and non-black race were associated with missed visits.^8,20 In contrast to past studies, age, education, substance and alcohol use were not significantly associated with missed visits in univariate and multivariate analysis. Symptoms of PTSD and history of abuse were significant in univariate analysis, but this relationship faded in multivariate analysis.^8,20,21 These results contrast past research, which has found associations between age, education, substance and alcohol use, psychological distress, and missed study visits. Although the absence of a comparison group makes it difficult to assess causality between retention efforts and study attendance, it is possible that the intensive retention strategies described above reduced attrition among these high risk groups. Housing instability, which remained statistically significant in the multivariate analysis, may be a reflection of the interaction of complex social forces such as substance use, abuse, and financial instability.^46,47 Studies enrolling women who are marginally-housed may consider developing partnerships with and providing referrals to organizations that address housing needs, substance use, etc., while also identifying ways to reach women independent of their address, such as through alternative contacts and/or identification of frequent “hang-out’ spots.

Past studies have not assessed the relationship between recruitment modality (e.g., street outreach, referral) and retention. However, in this study, recruitment at an outdoor venue (park or street location) was associated with an increased likelihood of visit attendance, highlighting that it is possible to retain women recruited from nontraditional venues, and from community-based settings more generally. Women enrolled during early study implementation were more likely to complete study visits than those enrolled later in time. This is likely a reflection of the fact that participants could complete their final study visit up to the date of study closure (HIV seroincidence was the primary endpoint of the study), thereby increasing the length of the final study visit window and the amount of time available to locate difficult to contact participants. Although wide visit windows may not be appropriate for all study designs, this finding highlights the need for maximizing study visit windows when feasible.

A small, but noteworthy proportion of visit nonattendance was due to known incarceration (11% and 10% at 6 and 12 months, respectively). This is a likely under-representation of total incarcerations throughout study implementation as some women were likely incarcerated and released in the time between biannual in-person study visits. Future studies may consider obtaining necessary ethics and funder approvals to contact participants and establishing partnerships with jails and prisons, so that participants can be contacted and, ideally, followed during periods of incarceration. This approach has been identified as an important strategy in achieving high follow-up rates in longitudinal studies of drug abusers.^33,35

Studies suggest that women view monetary compensation as a benefit of study participation and that they believe this compensation should be reflective of the time and risk associated with participation.⁴⁸ The exact compensation varied by site and ranged from 35 to 50 dollars for in-person study visits and 10 to 15 dollars for locator-update phone calls. Fifty-five percent of women reported annual incomes equal to or less than $20,000. Although baseline income and reported food insecurity were not found to be significantly associated with missed visits in this analysis, given the relatively short length of study visits and the minimally-invasive nature of the follow-up study (blood draws and ACASI interviews), it is likely that financial compensation reduced barriers to initial participation and provided some encouragement for continued participation.

Although HIV treatment trials have found associations between being a primary caregiver and study discontinuation,¹⁴ in contrast, this study was the first to identify financial responsibility for children as increasing the probability of visit attendance. Once enrolled, it is possible that women with children may have had additional health motivations to attend study visits, such as an increased interest in HIV voluntary counseling and testing and community referrals, and/or greater reliance on financial compensations.⁴⁹ It is possible that the use of nontraditional working hours, the provision of informal childcare, and community-based clinic locations may have reduced barriers to participation.

However, a focus on financial compensation would minimize the larger potential impact of HPTN 064 on the lives of many of the participants enrolled. Women in treatment trials have cited staff support as an important factor in study completion.¹⁴ Recognizing the geographic diversity and unique life situations of each participant, study sites used a combination of techniques to maintain high rates of study visit completion and to prevent subsequent loss to follow-up, many of which were relational in nature. There were numerous accounts from project staff of how the participants viewed the study as a positive motivation for change in their lives and analysis suggests that women increased condom use over the study follow-up period, despite the fact that it was an observational study.¹⁷ As one participant noted, “I came for the money, but I stayed for the love.”

The findings of this study are subject to limitations. The use of baseline measures as predictors does not allow us to test the associations between changes in predisposing or enabling characteristics and the likelihood of study visit nonattendance over time. As already noted, no comparison group exists, limiting our ability to assess casual relationships between retention strategies (positive or negative) and visit attendance. Furthermore, retention strategies were not assessed for their individual contribution to retention. Future studies would be improved by the addition of a cost-effectiveness component that allows for the assessment of the individual and combined contribution of retention strategies, as well as the relative labor-intensity. In addition, participants were recruited from high poverty areas using venue-based sampling and are not representative of all women living in these communities or all women at-risk for HIV. Findings from this study may not be generalizable to other US states or cities.

In summary, the HPTN 064 study demonstrated that it is possible to achieve high retention rates amongst a population of women at increased risk for study discontinuation; it is likely that the use of a multifaceted approach contributed to this success, although the retention of women in studies is a dynamic, iterative process. Based on our collective experience, we found that the strategies described in this article were associated with outstanding retention rates. Studies enrolling similar populations may consider the following:

1. Engage community in all stages of study development and implementation;

2. Invest extensive face time to develop trusting relationships with study participants early in study participation;

3. Collect detailed locator information which includes permission to contact participants through multiple modalities and update frequently;

4. Be as flexible as possible regarding the day/time of study visits and location, and when possible, visit windows;

5. Develop community partnerships and provide referrals for situations that may affect study participation, but are outside of the scope of the study (e.g., homelessness, substance use);

6. Use multiple retention approaches and modify retention strategies throughout implementation based on site experience;

7. Follow incarcerated participants if protocol allows;

8. Continue tracing efforts for hard to reach participants throughout study implementation;

9. Provide staff with the training and resources needed implement retention strategies;

10. Develop systems to assess the relative cost-effectiveness of different retention strategies.

Such retention efforts require substantial commitment by study funders and research sites and significant human and fiscal resources but such efforts support reaching accurate study results through high retention and may apply more broadly to the retention of similar populations of women in non-HIV related clinical trials as well as in clinical care. Methods to maximize study participant retention must be considered during the study planning process.

Footnotes

Acknowledgments

The authors thank the study participants, community stakeholders, and staff from each study site. In particular, they acknowledge Alexis Amsterdam, Gina Wingood, Jane Bupp, Wairimu Chege, Nirupama Sista, Kathy Hinson, Thomas Coates, Deborah Donnell, Elizabeth DiNenno, Ann O'Leary, Lisa Diane White, Waheedah Shabaaz-El, Quarraisha Abdool-Karim, Sten Vermund, Edward E. Telzak, Rita Sondengam, Cheryl Guity, Avelino Loquere, Jr., Manya Magnus, Gregory Phillips, Christin Root, Valarie Hunter, Ilene Wiggins, Tracey Chambers Thomas, Carolyna Torres, Marta Paez-Quinde, Makisha Ruffin, Genda Dockery, and LeTanya Johnson-Lewis.

Funding: Grant support provided by the National Institute of Allergy and Infectious Diseases, National Institute on Drug Abuse, and National Institute of Mental Health [UM1 AI068619, U01-AI068613, and UM1-AI068613]; Centers for Innovative Research to Control AIDS, Mailman School of Public Health, Columbia University [5U1Al069466]; University of North Carolina Clinical Trials Unit [AI069423]; University of North Carolina Clinical Trials Research Center of the Clinical and Translational Science Award [RR 025747]; University of North Carolina Center for AIDS Research [AI050410]; Emory University HIV/AIDS Clinical Trials Unit [5UO1AI069418], Center for AIDS Research [P30 AI050409], and Clinical and Translational Science Award [UL1 RR025008]; The Terry Beirn Community Programs for Clinical Research on AIDS Clinical Trials Unit [5 UM1 AI069503-07], and; The Johns Hopkins Adult AIDS Clinical Trial Unit [AI069465] and The Johns Hopkins Clinical and Translational Science Award [UL1 RR 25005].

Trial Registration Information: Clinicaltrials.gov, NCT00995176

The views expressed herein are solely the responsibility of the authors and do not necessarily represent the official views of the National Institute of Allergy and Infectious Diseases, the National Institute of Mental Health, the National Institutes of Health, the HPTN, or its funders.

Author Disclosure Statement

No competing financial interests exist.

References

El-Sadr

, Mayer

, Hodder

. AIDS in America—Forgotten but not gone. N Engl J Med, 2010; 362:967–970.

McDavid

, Li

, Lee

. Racial and ethnic disparities in HIV diagnoses for women in the United States. J Acquir Immune Defic Syndr, 2006; 42:101–107.

Centers for Disease Control and Prevention. HIV Surveillance Report, 2011. 2013;23.

Centers for Disease Control and Prevention. Estimated HIV incidence in the United States, 2007–2010. HIV Surveillance Supplemental Report, 2012. December 2012.

Aziz

, Smith

. Challenges and successes in linking HIV-infected women to care in the United States. Clin Infect Dis, 2011; 52:S231–S237.

National Institutes of Health: Office of Extramural Research. NIH Policy and Guidelines on The Inclusion of Women and Minorities as Subjects in Clinical Research – Amended, October, 2001. http://grants.nih.gov/grants/funding/women_min/guidelines_amended_10_2001.htm. Accessed February 8, 2013 .

Shadish

, Cook

, Campbell

. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin, 2001.

Hessol

, Schneider

, Greenblatt

, et al. Retention of women enrolled in a prospective study of human immunodeficiency virus infection: Impact of race, unstable housing, and use of human immunodeficiency virus therapy. Am J Epidemiol, 2001; 154:563–573.

BeLue

, Taylor-Richardson

, Lin

, Rivera

, Grandison

. African Americans and participation in clinical trials: Differences in beliefs and attitudes by gender. Contemp Clin Trials, 2006; 27:498–505.

10.

Smith

, Johnson

, Newman

, Greene

, Johnson

, Rogers

. Perceptions of clinical research participation among African American women. J Womens Health (Larchmt), 2007; 16:423–428.

11.

Humphreys

, Weingardt

, Harris

. Influence of subject eligibility criteria on compliance with National Institutes of Health guidelines for inclusion of women, minorities, and children in treatment research. Alcohol Clin Exp Res, 2007; 31:988–995.

12.

Chandra

, Paul

. African American participation in clinical trials: Recruitment difficulties and potential remedies. Hosp Top, 2003; 81:33–38.

13.

Shaya

, Gbarayor

, Huiwen Keri Yang , Agyeman-Duah

, Saunders

. A perspective on African American participation in clinical trials. Contemp Clin Trials, 2007; 28:213–217.

14.

Squires

, Feinberg

, Bridge

, et al. Insights on GRACE (Gender, Race, And Clinical Experience) from the patient's perspective: GRACE participant survey. AIDS Patient Care STDS, 2013; 27:352–362.

15.

Squires

, Johnson

, Yang

, et al. Comparative gender analysis of the efficacy and safety of atazanavir/ritonavir and lopinavir/ritonavir at 96 weeks in the CASTLE study. J Antimicrob Chemother, 2011; 66:363–370.

16.

Falcon

, Bridge

, Currier

, et al. Recruitment and retention of diverse populations in antiretroviral clinical trials: Practical applications from the gender, race and clinical experience study. J Womens Health (Larchmt)., 2011; 20:1043–1050.

17.

Hodder

, Justman

, Hughes

, et al. HIV acquisition among women from selected areas of the United States: A cohort study. Ann Intern Med, 2013; 158:10–18.

18.

Adimora

, Schoenbach

, Taylor

, Khan

, Schwartz

. Concurrent partnerships, nonmonogamous partners, and substance use among women in the United States. Am J Public Health, 2011; 101:128–136.

19.

Aidala

, Cross

, Stall

, Harre

, Sumartojo

. Housing status and HIV risk behaviors: Implications for prevention and policy. AIDS Behav, 2005; 9:251–265.

20.

Hessol

, Weber

, Holman

, et al. Retention and attendance of women enrolled in a large prospective study of HIV-1 in the United States. J Womens Health (Larchmt), 2009; 18:1627–1637.

21.

Seage

, Holte

, Metzger

, et al. Are US populations appropriate for trials of human immunodeficiency virus vaccine? The HIVNET Vaccine Preparedness Study. Am J Epidemiol, 2001; 153:619–627.

22.

Brown-Peterside

, Chiasson

, Ren

, Koblin

. Involving women in HIV vaccine efficacy trials: Lessons learned from a vaccine preparedness study in New York City. J Urban Health, 2000; 77:425–437.

23.

Brown-Peterside

, Rivera

, Lucy

, et al. Retaining hard-to-reach women in HIV prevention and vaccine trials: Project ACHIEVE. Am J Public Health, 2001; 91:1377–1379.

24.

Koblin

, Metch

, Novak

, et al. Feasibility of identifying a cohort of US women at high risk for HIV infection for HIV vaccine efficacy trials: Longitudinal results of HVTN 906. J Acquir Immune Defic Syndr, 2013; 63:239–244.

25.

Barkan

, Melnick

, Preston-Martin

, et al. The Women's Interagency HIV Study. WIHS Collaborative Study Group. Epidemiology, 1998; 9:117–125.

26.

Toth

, Messer

, Quinlivan

. Barriers to HIV care for women of color living in the Southeastern U.S. are associated with physical symptoms, social environment, and self-determination. AIDS Patient Care STDS., 2013; 27:613–620.

27.

Messer

, Quinlivan

, Parnell

, et al. Barriers and facilitators to testing, treatment entry, and engagement in care by HIV-positive women of color. AIDS Patient Care STDS, 2013; 27:398–407.

28.

Bofill

, Waldrop-Valverde

, Metsch

, Pereyra

, Kolber

. Demographic and psychosocial factors associated with appointment attendance among HIV-positive outpatients. AIDS Care, 2011; 23:1219–1225.

29.

Cohen

, Chen

, McCauley

, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med, 2011; 365:493–505.

30.

Stein

, Andersen

, Robertson

, Gelberg

. Impact of hepatitis B and C infection on health services utilization in homeless adults: A test of the Gelberg-Andersen Behavioral Model for Vulnerable Populations. Health Psychol, 2012; 31:20–30.

31.

Stein

, Andersen

, Gelberg

. Applying the Gelberg-Andersen behavioral model for vulnerable populations to health services utilization in homeless women. J Health Psychol., 2007; 12(5):791–804.

32.

Haley

, Golin

, El-Sadr

, et al. Venue-based recruitment of women at elevated risk for HIV: an HIV prevention trials network study. J Women's Health (in press).

33.

Cottler

, Compton

, Ben-Abdallah

, Horne

, Claverie

. Achieving a 96.6 percent follow-up rate in a longitudinal study of drug abusers. Drug Alcohol Depend, 1996; 41:209–217.

34.

Hunt

, White

. Retaining and tracking cohort study members. Epidemiol Rev, 1998; 20:57–70.

35.

Scott

. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers. Drug Alcohol Depend, 2004; 74:21–36.

36.

Group NCHSPT. Design and integration of ethnography within an international behavior change HIV/sexually transmitted disease prevention trial. AIDS, 2007; 21 Suppl 2:S37–S48.

37.

Prins

, Ouimette

, Kimerling

, et al. The primary care PTSD screen (PC-PTSD): Development and operating characteristics. Primary Care Psychiat, 2004:5.

38.

DiClemente

, Wingood

, Crosby

, et al. A prospective study of psychological distress and sexual risk behavior among black adolescent females. Pediatrics, 2001; 108:E85.

39.

Prejean

, Song

, Hernandez

, et al. Estimated HIV incidence in the United States, 2006–2009. PLoS One, 2011; 6:e17502.

40.

Metch

, Frank

, Novak

, et al. Recruitment of urban US women at risk for HIV infection and willingness to participate in future HIV vaccine trials. AIDS Behav, 2013; 17:760–772.

41.

Corbie-Smith

, Moody-Ayers

, Thrasher

. Closing the circle between minority inclusion in research and health disparities. Arch Intern Med, 2004; 164:1362–1364.

42.

Gilliss

, Lee

, Gutierrez

, et al. Recruitment and retention of healthy minority women into community-based longitudinal research. J Womens Health Gend Based Med, 2001; 10:77–85.

43.

Morin

, Maiorana

, Koester

, Sheon

, Richards

. Community consultation in HIV prevention research: A study of community advisory boards at 6 research sites. J Acquir Immune Defic Syndr, 2003; 33:513–520.

44.

Leonard

, Lester

, Rotheram-Borus

, Mattes

, Gwadz

, Ferns

. Successful recruitment and retention of participants in longitudinal behavioral research. AIDS Educ Prev, 2003; 15:269–281.

45.

Frew

, Hou

, Davis

, et al. The likelihood of participation in clinical trials can be measured: The Clinical Research Involvement Scales. J Clin Epidemiol, 2010; 63:1110–1117.

46.

Boerma

, Weir

. Integrating demographic and epidemiological approaches to research on HIV/AIDS: The proximate-determinants framework. J Infect Dis, 2005; 191:61–67.

47.

McNeil

, Guirguis-Younger

, Dilley

, Aubry

, Turnbull

, Hwang

. Harm reduction services as a point-of-entry to and source of end-of-life care and support for homeless and marginally housed persons who use alcohol and/or illicit drugs: a qualitative analysis. BMC Public Health, 2012; 12:312.

48.

Breitkopf

, Loza

, Vincent

, Moench

, Stanberry

, Rosenthal

. Perceptions of reimbursement for clinical trial participation. J Empir Res Hum Res Ethics, 2011; 6:31–38.

49.

Boehme

, Davies

, Moneyham

, Shrestha

, Schumacher

, Kempf

. A qualitative study on factors impacting HIV care adherence among postpartum HIV-infected women in the rural southeastern USA. AIDS Care., 2014; 26:574–581.