Strategies Associated with Retaining Participants in the Longitudinal National Alzheimer’s Coordinating Center Uniform Data Set Study

Abstract

Background:

Best approaches for retaining research participants in Alzheimer’s disease cohort studies are understudied.

Objective:

Using data from the National Alzheimer’s Coordinating Center Uniform Data Set, we evaluated the associations of unique strategies with participant retention across Alzheimer’s Disease Research Centers and explored potential effect modification by race, ethnicity and diagnostic group.

Methods:

We examined retention at the first follow-up visit among participants enrolled during 2015–2017. Structured surveys ascertained 95 retention tactics among 12 strategies. Strategy-specific summary scores were created based on the number of implemented tactics for each strategy and grouped into tertiles. Generalized estimating equations were constructed to evaluate associations between strategy scores and the odds of retention, controlling for age, sex, education, study partner type, marital status, visit length, battery length, diagnostic group, race and ethnicity. Separate models were stratified by race, ethnicity and diagnostic group. Effect modification was formally tested with interaction terms.

Results:

Among 5,715 total participants enrolled, 4,515 were Non-Hispanic White (79%), 335 were Hispanic/Latino (6%), 651 were Non-Hispanic Black (11%), and 214 were Non-Hispanic Asian (4%). Compared to the lowest tertile of scores, the highest tertile of scores involving improvement in study personnel and communication of study requirements and details were associated with 61% higher odds of retention in fully adjusted models (adjusted Odds Ratios [aOR] = 1.61, 95% Confidence Interval [CI] = 1.05–2.47 and aOR = 1.55, 95% CI = 1.03–2.35, respectively). We did not find evidence for effect modification.

Conclusion:

In the setting of limited resources, specific retention strategies may be more valuable than others.

Keywords

Alzheimer’s disease diagnostic-related group ethnicity patient dropouts patient participation race

INTRODUCTION

Longitudinal cohort studies play an essential role in advancing our understanding of the onset and progression of Alzheimer’s disease (AD) and related dementias (ADRD). Participant attrition, however, poses a challenge for ADRD researchers as it reduces statistical power to detect effects, increases the potential for bias in estimated treatment effects, and limits the generalizability of study findings [1]. In some instances, threats to internal validity may occur through differential loss to follow-up or nonignorable missingness [2, 3], which can then lead to incorrect inferences of observed estimates. Given investments in ADRD cohort studies, identifying effective strategies to improve participant retention is of paramount importance.

Considerable efforts have been made over the years to examine the effectiveness of various retention methods. A previous systematic review [4] of several established retention strategies demonstrated that offering greater amounts of participant incentives (cash or gift) and employing various modes of contact such as reminder letters improves retention. The effectiveness of some approaches was, however, observed to vary depending on the age of the cohort. Some studies have suggested that offering social support to families of participants, particularly those with dementia, could improve retention in AD cohorts [5 –8]. Indeed, the evidence shows that family members become more involved in decisions about medical care over the course of the study period as patients progress with their disease [9]. This suggests that optimal communication about study procedures and support resources, in addition to more intensive follow-up and contact, become increasingly important for retention. A more recent systematic review and meta-analysis [10] of 143 longitudinal studies and 95 different retention strategies found that employing the barrier-reducing strategy of offering alternative methods of data collection to participants was the strongest predictor of retention. Notably, the authors also observed that the greater number of retention strategies used was not associated with improved retention, suggesting that a more parsimonious selection of retention strategies, perhaps based on participant barriers and characteristics, can be both cost-effective and impactful.

With regard to characteristics of participants, there is perhaps no greater need than to improve the retention of racial and ethnic groups who have been historically underrepresented in ADRD research [11]. Of the limited studies in older racial and ethnic groups, findings underscore the importance of maintaining strong and active community relationships [12], fostering cultural competence among study personnel who actively engage with culturally diverse participants and offering social support to families of participants with dementia as well as accessible locations for study assessments [13 –15]. The available evidence, however, is scarce and often lacking the methodological rigor to draw firm conclusions into best strategies [16]. More work is needed to offer additional insight.

Improving retention using effective strategies in AD studies is a national and public health priority. We have previously reported a positive association between the number of employed retention tactics at Alzheimer’s Disease Research Centers (ADRCs) and participant follow-up rates using longitudinal data from the National Alzheimer’s Coordinating Center Uniform Data Set (NACC-UDS) [17]. In the present study, we expand upon these findings to: 1) evaluate the associations of 12 domain-specific retention strategies with participant retention and 2) explore whether the association of a select group of strategies hypothesized to improve participation of special populations varies by race and ethnicity or by diagnostic group.

MATERIALS AND METHODS

Study participants

Initiated in 2005, the NACC-UDS is a publicly accessible, longitudinal database of adults enrolled at ADRCs who have undergone baseline and yearly clinical follow-up assessments [18]. These assessments include standardized procedures to ascertain physical and neurocognitive functioning as well as sociodemographic characteristics. ADRCs were approved by their local Institutional Review Boards and participants provided informed consent at the ADRC where they completed their study visits. The present study used only de-identified participant data and therefore did not meet the definition of human subjects research.

Retention strategies

Details of the procedures involving the survey development and administration have previously been reported elsewhere [17]. Briefly, the data collection instrument assessed the implementation of 95 retention tactics chosen a priori from evidence-based sources [19, 20]. These tactics were adjusted and vetted across the Outreach Recruitment and Engagement and Clinical Core leaders of three different ADRCs [17]. They were then grouped into 12 retention strategies that focused on specific areas of intervention; namely, 1) six “community involvement” tactics, 2) ten “study identity” tactics, 3) seven “study personnel” tactics, 4) nine “study description” tactics, 5) eight “reminder” tactics, 6) six “contact and scheduling methods” tactics, 7) twelve “visit characteristics” tactics, 8) thirteen “benefits of study” tactics, 9) five “financial incentives” tactics, 10) seven “reimbursements or cost coverage” tactics, 11) seven “non-financial incentives” tactics, and 12) five “special tracking methods” tactics. The survey was administered on September 29, 2017 to the leaders of the Outreach, Recruitment and Engagement Cores of each of the (then) 30 NIA-funded ADRCs through a secure web application (REDCap) [21]. Center-level survey data were then linked to individual-level participant data from NACC-UDS. In analysis, we assigned a score of 1 to each tactic if it was implemented or 0 if it was not. We then created a summary score of tactics within each strategy.

Retention

Our primary outcome of interest was participant retention, defined as completing the first follow-up visit within 2 years (730 days) from the initial baseline visit. This 2-year window was specified a priori to provide a conservative timeframe given that participants were anticipated to return for follow-up visits on a yearly basis. To reduce potential cohort effects [22], the current study was restricted to data from 6,155 newly enrolled NACC participants during 2012 – 2015 across 25 ADRCs that completed the survey. Of these, we excluded 1 center with incomplete data on tactics/strategies (n = 60 NACC participants, 1% of total). At the individual level, we excluded 322 NACC participants (5.2%) who were known to be deceased at follow-up. With regards to race and ethnicity, we excluded 18 individuals (< 1%) who were missing information on Hispanic ethnicity and 4 individuals (< 1%) who self-identified as non-Hispanic and were of mixed (unknown) race. Due to limited numbers for meaningful comparisons, we also excluded non-Hispanic participants who self-identified as American Indian or Alaska Native (n = 21, < 1%) and Native Hawaiians or other Pacific Islander (n = 13, < 1%). In sensitivity analyses, exclusion of these participants did not appreciably change our main effect estimates. The final analytic sample was comprised of 5,715 NACC participants from 24 ADRCs.

Covariates

Collected at the baseline visit, individual-level sociodemographic information from the NACC included self-reported age and education in years, sex, study partner type (categorized as spouse/partner/companion, adult child, and other, which included sibling, other relative, friend, neighbor, paid caregiver, or health care provider) and marital status. We a priori categorized educational attainment into five groups: <high school (< 12 y), high school (12 y), some college (13–15 y), college (16 y), and graduate (> 16 y). At the center-level, we gathered information about the average length in hours of study visits and neuropsychological test batteries.

Stratification variables

Participants self-identified their race (Asian, Black, White) and Hispanic ethnicity at the baseline visit. We classified individuals into four mutually exclusive racial and ethnic categories: Non-Hispanic (NH) White, NH Black, NH Asian, and Hispanic/Latino (of any race). Diagnostic classifications of normal, impaired but not mild cognitive impairment (MCI), MCI, and dementia were made at the baseline assessment. For analysis, we collapsed the categories of “impaired but not MCI” and “MCI”.

Statistical analyses

In descriptive analyses, we assessed the distribution of participant characteristics overall and across racial and ethnic groups. For each of the 12 strategies, we calculated strategy-specific scores by summing the number of tactics used within each strategy. We then a priori categorized each strategy-specific score into tertiles based on the observed sample distribution. To examine the association between strategies and retention, we estimated odds ratios in separate models using generalized estimating equations (GEE) with independent working correlation structure and robust variance estimates to account for correlated data within ADRC study sites. All models were adjusted for age, sex, education, study partner type, marital status, visit, and battery length. Because of a previously reported interaction between marital status and sex in our data, we added this interaction term to the adjusted models.

To evaluate whether associations between hypothesized strategies and retention varied across racial and ethnic and diagnostic groups, we stratified regression models by these variables. Based on literature [12 –16], we hypothesized that strategies that focus on community involvement, study personnel characteristics, and adequate communication of study benefits and description would be associated with higher retention of underrepresented racial and ethnic participants and those with more advanced disease. We tested for effect modification via multiplicative interactions in separate multivariable models by adding an interaction term of race/ethnicity^*strategy and diagnostic group^*strategy, where strategy scores were entered as a continuous variable. In exploratory analysis, we used Classification and Regression Trees (CART) to reduce the dimensionality of the tactic-level data and identify tactics that were most predictive of participant retention. This was performed through recursive partitioning of the data, where binary separations of the study sample were created by choosing the optimal predictor and cut-point combination to yield the largest discrimination in mean response values between the resulting subpopulations. Of the 95 tactics in the data set, we excluded 19 tactics from the exploratory CART analysis that were categorized as “other”. Inferential statistical tests were considered statistically significant at a 0.05 threshold. All descriptive and regression-based analyses were conducted with SAS statistical software version 9.4 (Cary, NC, USA). CART analyses were implemented in the R statistical software using the rpart package.

RESULTS

Study participants had an overall mean age of 70 years at enrollment and were 79% NH White (n = 4,515), 6% Hispanic/Latino (n = 335), 11% Black (n = 651), and 4% Asian (n = 214). Figure 1 depicts the distribution of race and ethnicity across the 24 study sites. Over 65% of Asian participants came from 3 study centers. Approximately 60% of Hispanic/Latino participants came from 4 centers and 40% of Black participants came from 3 centers.

Fig. 1

Histogram depicting the proportion of individuals from our study sample (n = 5,715) distributed across the 24 Alzheimer’s disease research centers by race and ethnicity. NH, Non-Hispanic.

We observed differences in the distribution of participant characteristics by race and ethnicity (Table 1). Black participants were more often female and less often married. More than half of all Hispanic/Latino participants had attained a high school education or less (52%) and, compared to the other racial and ethnic groups, included a greater proportion diagnosed with dementia. Black, Hispanic/Latino, and Asian participants less frequently enrolled with a spousal study partner when compared to their NH White counterparts. Retention at the first follow-up visit also varied appreciably by race and ethnicity; Hispanic/Latino and Black participants exhibited the lowest retention rates (62% and 63%, respectively) and NH Whites had the highest rate (74%).

Table 1

Distribution of participant characteristics by race and ethnicity

Participant characteristics	Overall	Race/ethnicity
		NH Whites	Hispanics/ Latinos	NH Blacks	NH Asians
Total N	5715	4515 (79%)	335 (5.9%)	651 (11.4%)	214 (3.7%)
Age at enrollment, mean (SD)	70.1 (4.3)	70.6 (9.8)	69.2 (11.5)	70.7 (8.9)	69.4 (10.2)
Sex (Female), n (%)	3187 (55.8%)	2390 (52.9%)	193 (57.6%)	482 (74%)	122 (57%)
Education level, n (%)
< HS	238 (4.2%)	69 (1.5%)	104 (31%)	57 (8.8%)	8 (3.7%)
HS	883 (15.5%)	642 (14.2%)	73 (21.8%)	148 (22.7%)	20 (9.4%)
Some college	1006 (17.6%)	752 (16.7%)	61 (18.2%)	171 (26.3%)	22 (10.3%)
College	1380 (24.2%)	1161 (25.7%)	40 (11.9%)	112 (17.2%)	67 (31.3%)
Graduate	2167 (37.9%)	1858 (41.2%)	53 (15.8%)	162 (24.9%)	94 (43.9%)
Study partner type, n (%)
Spouse/partner/companion	3369 (59%)	2919 (64.7%)	130 (38.8%)	202 (31%)	118 (55.1%)
Child	1180 (20.7%)	782 (17.3%)	134 (40%)	216 (33.2%)	48 (22.4%)
Other	1041 (18.2%)	716 (15.9%)	65 (19.4%)	217 (33.3%)	43 (20.1%)
Marital status
Married	3788 (66.3%)	3203 (70.9%)	190 (56.7%)	249 (38.3%)	146 (68.2%)
Widowed	726 (12.7%)	507 (11.2%)	53 (15.8%)	138 (21.2%)	28 (13.1%)
Divorced	727 (12.7%	488 (10.8%)	54 (16.1%)	164 (25.2%)	21 (9.8%)
Other	474 (8.3%)	317 (7.1%)	38 (11.4%)	100 (15.5%)	19 (8.9%)
Diagnostic groups
Normal cognition	2392 (41.9%)	1895 (42%)	111 (33.1%)	294 (45.2%)	92 (43%)
Impaired/MCI	1472 (25.8%)	1115 (24.7%)	73 (21.8%)	219 (33.6%)	65 (30.4%)
Dementia	1851 (32.4%)	1505 (33.3%)	151 (45.1%)	138 (21.2%)	57 (26.6%)
Retention rate^*, %	72.5%	74.9%	62.1%	63.3%	68.2%

^* Retention to the first follow-up visit.

Associations between strategies and participant retention are shown in Table 2. In multivariable models, individuals at the highest tertile strategy scores had significantly greater odds of retention when compared to those at the lowest tertile scores for strategies that involved personnel characteristics (adjusted odds ratio [aOR] = 1.61, 95% confidence interval [CI]: 1.05–2.47) and study description (aOR = 1.55, 95% CI: 1.03–2.35). The association involving personnel characteristics had an approximately linear trend across tertiles of scores. Sensitivity analyses showed that no single tactic drove this association. For the strategy involving study identification, we observed that individuals at the highest tertile of scores were less likely to be retained when compared to those at the lowest tertile (aOR = 0.59, 95% CI: 0.37–0.95). No other significant associations were observed.

Table 2

Associations of ADRC-specific retention strategies with participant retention at the first follow-up visit

	Unadjusted odds ratios (ORs)			Adjusted^** odds ratios (ORs)
Strategy scores (tertiles)^*	OR	95% CI	p	OR	95% CI	p
Community involvement
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.51	(0.96 – 2.38)	0.0729	1.36	(0.81 – 2.28)	0.2407
Highest tertile	1.07	(0.72 – 1.59)	0.7483	0.95	(0.59 – 1.54)	0.8295
Study identification
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.34	(0.87 – 2.07)	0.1836	1.24	(0.82 – 1.89)	0.3046
Highest tertile	0.79	(0.56 – 1.13)	0.2026	0.59	(0.36 – 0.95)	0.0315
Personnel characteristics
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.12	(0.68 – 1.84)	0.6596	1.31	(0.82 – 2.10)	0.2582
Highest tertile	1.06	(0.67 – 1.70)	0.7933	1.61	(1.05 – 2.47)	0.0283
Study description
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.56	(0.87 – 2.80)	0.1365	1.94	(1.20 – 3.11)	0.0064
Highest tertile	1.26	(0.82 – 1.94)	0.2825	1.55	(1.02 – 2.35)	0.0381
Reminders
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	0.92	(0.60 – 1.41)	0.6977	0.89	(0.57 – 1.40)	0.6244
Highest tertile	0.80	(0.47 – 1.37)	0.4219	1.14	(0.58 – 2.24)	0.7029
Contact and scheduling
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	0.99	(0.64 – 1.54)	0.9671	0.90	(0.59 – 1.37)	0.6194
Highest tertile	1.29	(0.67 – 2.46)	0.4473	1.15	(0.62 – 2.14)	0.6637
Visit characteristics
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.01	(0.65 – 1.56)	0.9801	0.98	(0.66 – 1.46)	0.9192
Highest tertile	1.23	(0.76 – 2.00)	0.3957	0.92	(0.55 – 1.56)	0.7695
Benefits of study
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.23	(0.69 – 2.18)	0.4791	1.44	(0.79 – 2.62)	0.2374
Highest tertile	0.89	(0.56 – 1.41)	0.6206	1.16	(0.63 – 2.13)	0.6264
Financial incentives
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.14	(0.69 – 1.89)	0.6004	0.87	(0.56 – 1.35)	0.5364
Highest tertile	1.23	(0.83 – 1.84)	0.3048	1.11	(0.70 – 1.74)	0.6609
Reimbursement &coverage
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.19	(0.73 – 1.93)	0.4902	1.19	(0.74 – 1.92)	0.4762
Highest tertile	0.89	(0.57 – 1.37)	0.5948	0.89	(0.57 – 1.37)	0.5828
Non-financial incentives
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	1.15	(0.77 – 1.73)	0.4839	1.02	(0.65 – 1.61)	0.9247
Highest tertile	1.76	(1.06 – 2.91)	0.0275	1.46	(0.81 – 2.62)	0.2080
Special tracking
Lowest tertile	1	(referent)		1	(referent)
Middle tertile	0.92	(0.58 – 1.45)	0.7099	0.85	(0.44 – 1.66)	0.6354
Highest tertile	1.34	(0.78 – 2.30)	0.2855	1.27	(0.67 – 2.40)	0.4683

^* Cut points for tertiles were: (1) Community involvement scores: lowest tertile = 0–2, middle tertile = 3, highest tertile = 4–5; (2) Study identification scores: lowest tertile = 0–3, middle tertile = 4–5, highest tertile = 6–8; (3) Personnel characteristics: lowest tertile = 0–2, middle tertile = 3, highest tertile = 4–5; (4) Study description: lowest tertile = 0–3, middle tertile = 4, highest tertile = 5–7; (5) Reminders: lowest tertile = 0–2, middle tertile = 3, highest tertile = 4; (6) Contact and scheduling: lowest tertile = 0–1, middle tertile = 2, highest tertile = 3–6; (7) Visit characteristics: lowest tertile = 0–6, middle tertile = 7, highest tertile = 8–11; (8) Benefits of study: lowest tertile = 0–4, middle tertile = 5–8, highest tertile = 9–11; (9) Financial incentives: lowest tertile = 0, middle tertile = 1, highest tertile = 2; (10) Reimbursement and coverage: lowest tertile = 0–1, middle tertile = 2, highest tertile = 3–5; (11) Non–financial incentives: lowest tertile = 0–1, middle tertile = 3–4, highest tertile = 5–6; (12) Special tracking: lowest tertile = 0, middle tertile = 1, highest tertile = 2–3. ^** Odds ratios were calculated in separate models and adjusted for age, sex, education, race/ethnicity, study partner type, marital status, visit length, battery length and diagnostic group. An interaction term for marital status^*sex was included in the adjusted models.

Stratified analyses by race and ethnicity are shown in Table 3. Across each ethnoracial group, associations between scores of selected strategies and participant retention were generally positive, albeit not reaching statistical significance for most effect estimates. Although the associations appeared to be stronger for Asians compared to other groups, we did not find evidence for effect modification by race and ethnicity (all p for interactions > 0.05). Sensitivity analyses were conducted in which multivariable models serially excluded the 3 study centers with the highest proportion of Asian participants (≥15%) and showed that effect estimates did not attenuate or appreciably change (not shown). When we considered effect modification by diagnostic group (Table 4), stratum-specific multivariable models showed generally positive associations. We found no evidence of statistical interaction by diagnostic group (all p for interactions > 0.05).

Table 3

Associations of selected retention strategies with participant retention at the first follow-up visit stratified by race and ethnicity

Strategy scores (tertiles)	NH White (n = 4,515)		NH Black (n = 651)		NH Asian (n = 214)		Hispanic/Latino (n = 335)		p for interaction
	OR^*	95% CI	OR^*	95% CI	OR^*	95% CI	OR^*	95% CI
Community involvement									0.1100
Middle versus Lowest	1.33	(0.73 – 2.43)	1.17	(0.69 – 1.98)	18.3	(4.91 – 68.1)	0.96	(0.38 – 2.40)
Highest versus Lowest	0.91	(0.53 – 1.55)	1.02	(0.59 – 1.78)	2.96	(1.00 – 8.81)	0.57	(0.34 – 0.95)
Personnel characteristics									0.1114
Middle versus Lowest	1.21	(0.78 – 1.87)	1.15	(0.69 – 1.91)	5.64	(1.52 – 20.9)	0.65	(0.35 – 1.21)
Highest versus Lowest	1.62	(0.98 – 2.66)	1.70	(1.12 – 2.56)	2.82	(0.94 – 8.46)	0.97	(0.48 – 1.96)
Study benefits									0.0602
Middle versus Lowest	1.30	(0.69 – 2.45)	1.12	(0.56 – 2.22)	2.46	(0.44 – 13.7)	1.74	(0.50 – 6.14)
Highest versus Lowest	1.16	(0.60 – 2.25)	1.13	(0.72 – 1.78)	0.73	(0.16 – 3.33)	0.81	(0.21 – 3.18)
Study description									0.0616
Middle versus Lowest	1.86	(1.08 – 3.21)	1.32	(0.66 – 2.66)	8.91	(2.75 – 28.8)	2.47	(1.02 – 5.98)
Highest versus Lowest	1.54	(0.94 – 2.53)	1.45	(0.72 – 2.93)	3.38	(1.10 – 10.4)	1.53	(0.88 – 2.64)

^*Odds ratios were adjusted for age, sex, education, study partner type, marital status, visit length, battery length and diagnostic group. An interaction term for marital status^*sex was included in the adjusted models.

Table 4

Associations of selected retention strategies with participant retention at the first follow-up visit stratified by diagnostic group

Strategy scores (tertiles)	Cognitively unimpaired (n = 2,392)		MCI/Impaired Not-MCI (n = 1,472)		Dementia (n = 1,851)		p for interaction
	OR^*	95% CI	OR^*	95% CI	OR^*	95% CI
Personnel characteristics							0.1142
Middle versus Lowest	1.53	(0.98 – 2.38)	1.81	(0.97 – 3.40)	0.87	(0.57 – 1.35)
Highest versus Lowest	2.07	(1.52 – 2.80)	1.70	(0.97 – 2.96)	1.14	(0.68 – 1.89)
Study description							0.1016
Middle versus Lowest	2.05	(1.45 – 2.94)	2.78	(1.42 – 5.42)	1.35	(0.75 – 2.40)
Highest versus Lowest	1.88	(1.18 – 2.99)	1.80	(1.04 – 3.10)	1.03	(0.65 – 1.66)
Study benefits							0.4762
Middle versus Lowest	1.48	(0.85 – 2.60)	1.66	(0.83 – 3.30)	1.27	(0.66 – 2.44)
Highest versus Lowest	1.31	(0.71 – 2.47)	1.01	(0.54 – 1.87)	1.06	(0.53 – 2.12)

^*Odds ratios adjusted for age, sex, education, race/ethnicity, study partner type, marital status, visit length, battery length. An interaction term for marital status^*sex was included in the adjusted models.

In exploratory analyses that assessed associations of tactics with participant retention using CART, we found that having off-site locations, a phone tree, a diverse staff, and an active blog best partitioned the data on retention (Fig. 2). When we added the variables for race and ethnicity and diagnostic groups separately to the CART model, off-site locations and phone tree tactics consistently partitioned the data before these variables. In sensitivity analyses using multivariable models that adjusted for the same variables as in the main effect models, we found that off-site locations and phone tree tactics were positively associated with participant retention and did not vary significantly by race and ethnicity or diagnostic groups (not shown).

Fig. 2

Classification and regression tree (CART) results using 76 retention tactics. The data were partitioned on being retained (yes/no) during the two-year follow-up period. The numerators indicate the number of individuals retained or not retained, and the denominators indicate the total number of individuals in the subpopulation.

DISCUSSION

In a sample of 5,715 participants of the NACC-UDS study enrolled across 24 ADRCs, we found that specific strategies focused on study personnel and study description were associated with a higher odds of participant retention. These associations were independent of potential confounding factors and did not vary significantly across racial and ethnic or diagnostic groups. Overall, these findings are consistent with previous data [4, 10] and suggest that, in the setting of limited resources, some retention strategies may be more valuable than others.

The study personnel strategy includes tactics such as maintaining a diverse staff, engaging staff in regular retention trainings, and ensuring that specific staff members work with specific participants over time. The lack of effect modification by race and ethnicity or diagnostic group may suggest that all participants stand to gain in terms of increased retention through implementation of this strategy. Indeed, staff who understand and appreciate the target populations’ cultural practices, beliefs and values is among the most pivotal features of a successful retention strategy [23]. These results are also consistent with previous studies reporting improvements in retention of older underrepresented racial and ethnic participants by consistently using the same interviewers or field staff over time [24, 25]. The current results suggest this tactic may be valuable for retaining all participants.

Our data showed that implementing a strategy that enhances study description, communication of study requirements and details, was associated with improved participant retention. Establishing expectations and helping participants understand and plan for study requirements/burden during recruitment and the informed consent process may be essential for maintaining participant satisfaction. In fact, every ADRC indicated that they employed some of these tactics. Greater variance (and therefore greater opportunity to contribute to the observed effect) was observed in the use of family conferences during recruitment and study participation, using decision aids to facilitate consenting, and incorporating video tools to explain the overall study or specific procedures [17]. Ensuring understanding of the study goals and requirements, perhaps especially after enrollment may be key. For example, in a large longitudinal study of Black women, successful retention strategies involved frequent contact about study procedures with individuals who were more likely to be poor and mobile [26].

We found no evidence for effect modification of associations between selected strategies and participant retention according to diagnostic groups. This was somewhat surprising for the strategy focused on study benefits, given that we expected participants with dementia to differentially benefit from tactics that offer social support services and make study test results such as routine laboratories, neuropsychological measures, and biomarker results more accessible to them and their families. A possible explanation for the lack of interaction is that these benefits were viewed as universally important to participants, regardless of their diagnostic category [27]. Alternatively, some of the tactics with the greatest potential for effect modification (e.g., access to a clinical care or a social worker) were employed by a very small number of ADRCs. It is also possible that, for this population of participants diagnosed with MCI or dementia at baseline, other strategies and tactics were more important for retention. More research is needed to understand the optimal means of achieving successful retention across levels of disease severity.

In exploratory analyses, CART identified holding study visits at off-site locations, presumably ones more convenient to participants, as the tactic most effective (i.e., highest on the regression tree) at separating retained from lost participants. After that, using a phone tree to maintain contact with participants, having a diverse staff and offering an active blog for participants (a study identity tactic) best separated the data. A potential strength in using CART is that it accounted for any potential interactions between the 76 tactics tested. Though exploratory, these results may suggest that the most practical approaches to retention—making visits convenient and ensuring modes of contact—are of greater value than unique tactics within other strategies.

There are several limitations in the present study. Quantification of retention efforts was at the center level, rather than the participant level. Thus, interpreting the value of observed associations to retaining individual participants must be done with caution. The timing, duration, and intensity of exposure to retention tactics at each center were not ascertained. Despite the longitudinal nature of the study, the timing of exposure would be necessary to establish a causal relationship. Additionally, it is possible that differences in duration and intensity of specific strategies could potentially impact participant retention. For example, building and sustaining trusting relationships in the community takes time and considerable investment [28]; thus, a more established strategy of community involvement might lead to better participant engagement than a nascent one. Our survey was limited to binary indications for use of strategies; these types of effects could not be assessed. We only assessed a subset of retention strategies, selected a priori, for potential interaction effects with subgroup categorizations. We did this in an effort to reduce the risk of multiplicity, but it is conceivable that interactions could have existed for other retention strategies. Assessment of potential effect modification by race and ethnicity was also limited by disproportionate contribution of specific racial and ethnic groups by a small number of centers. To explore potential bias resulting from this skew, we conducted sensitivity analyses whereby centers with high proportions of specific racial and ethnic groups were serially omitted in stratified models that assessed interaction by race and ethnicity. Our findings showed consistency in the stratum-specific effect estimates. Another potential limitation was our approach to quantifying strategy exposure using count-based summary scores that gave equal weighting to tactics within each strategy. It is possible that specific tactics within strategies may have carried higher weight and were diluted in the scoring.

Some of the observed associations were counterintuitive, such as greater use of study identification strategy being associated with poorer retention. Included in this strategy are tactics that attempt to create a “study identity” for participants in an effort to retain them; for example, use of a center logo or moniker that is easily recognizable, maintaining a social media presence via Facebook, Twitter, YouTube, or blog, and publishing electronic or print newsletters for participants to read. Maintaining a study website and distributing electronic or print newsletters, were performed by every or nearly ever center [17] and therefore were unlikely to contribute to the unexpected relationship. The remaining tactics included maintaining a blog and social media channels and engaging in live webinars or other remote opportunities for participants to give feedback and have questions answered. These types of activities were increased and may have become essential during the COVID-19 pandemic, which occurred after the collection of the study data. It is also possible that participants may have preferred other more personalized forms of contact to connect with a study [29, 30] and maintain high participant interest. Another possibility is that some participants may not have easy access to online materials and thereby lack opportunities to fully engage through digital communication [31]. Alternatively, since many ADRCs now incorporate a wide variety of potentially burdensome ancillary procedures such as imaging, lumbar puncture, and fibroblast donation for induced pluripotent stem cell generation, we cannot rule out that study identity may become synonymous with research burden for some participants (and lead to dropout). Elucidating why such associations occurred will indeed require further study.

In summary, our study found that the most effective retention strategies were those that focus on improving study personnel through trainings and communicating study description irrespective of race and ethnicity or diagnostic groups. Identifying evidence-based retention strategies can inform national guidelines/policy to improve the conduct of AD studies and help to allocate limited resources more efficiently.

Footnotes

ACKNOWLEDGMENTS

This project was supported by NIA P30 AG066519 and NCATS UL1 TR001414. CRS is supported by a diversity supplement to NIA AG059407 and Alzheimer’s Association research fellowship AARFD-20-682432.

Authors’ disclosures available online ().

References

Galea

, Tracy

(2007) Participation rates in epidemiologic studies. Ann Epidemiol 17, 643–653.

Hernan

, Hernandez-Diaz

, Robins

(2004) A structural approach to selection bias. Epidemiology 15, 615–625.

Howe

, Cole

, Lau

, Napravnik

, Eron

Jr. (2016) Selection bias due to loss to follow up in cohort studies. Epidemiology 27, 91–97.

Booker

, Harding

, Benzeval

(2011) A systematic review of the effect of retention methods in population-based cohort studies. BMC Public Health 11, 249.

Connell

, Shaw

, Holmes

, Foster

(2001) Caregivers’ attitudes toward their family members’ participation in Alzheimer disease research: Implications for recruitment and retention. Alzheimer Dis Assoc Disord 15, 137–145.

Cox

, Ryan

BAM

, Gillen

, Grill

(2019) A preliminary study of clinical trial enrollment decisions among people with mild cognitive impairment and their study partners. Am J Geriatr Psychiatry 27, 322–332.

Nuno

, Gillen

, Dosanjh

, Brook

, Elashoff

, Ringman

, Grill

(2017) Attitudes toward clinical trials across the Alzheimer’s disease spectrum. Alzheimers Res Ther 9, 81.

Kim

, Cox

, Caine

(2002) Impaired decision-making ability in subjects with Alzheimer’s disease and willingness to participate in research. Am J Psychiatry 159, 797–802.

Karlawish

, Casarett

, Propert

, James

, Clark

(2002) Relationship between Alzheimer’s disease severity and patient participation in decisions about their medical care. J Geriatr Psychiatry Neurol 15, 68–72.

10.

Teague

, Youssef

, Macdonald

, Sciberras

, Shatte

, Fuller-Tyszkiewicz

, Greenwood

, McIntosh

, Olsson

, Hutchinson

, Theme

SLS

(2018) Retention strategies in longitudinal cohort studies: A systematic review and meta-analysis. BMC Med Res Methodol 18, 151.

11.

National Strategy for Recruitment and Participation in Alzheimer’s Disease Clinical Research, National Institutes of Health, https://www.nia.nih.gov/research/recruitment-strategy

12.

Wong

, Amano

, Lin

, Zhou

, Morrow-Howell

(2019) Strategies for the recruitment and retention of racial/ethnic minorities in Alzheimer disease and dementia clinical research. Curr Alzheimer Res 16, 458–471.

13.

Ejiogu

, Norbeck

, Mason

, Cromwell

, Zonderman

, Evans

(2011) Recruitment and retention strategies for minority or poor clinical research participants: Lessons from the Healthy Aging in Neighborhoods of Diversity across the Life Span study. Gerontologist 51 Suppl 1, S33–45.

14.

Hinton

, Carter

, Reed

, Beckett

, Lara

, DeCarli

, Mungas

(2010) Recruitment of a community-based cohort for research on diversity and risk of dementia. Alzheimer Dis Assoc Disord 24, 234–241.

15.

Yancey

, Ortega

, Kumanyika

(2006) Effective recruitment and retention of minority research participants. Annu Rev Public Health 27, 1–28.

16.

Gilmore-Bykovskyi

, Jin

, Gleason

, Flowers-Benton

, Block

, Dilworth-Anderson

, Barnes

, Shah

, Zuelsdorff

(2019) Recruitment and retention of underrepresented populations in Alzheimer’s disease research: A systematic review. Alzheimers Dement (N Y) 5, 751–770.

17.

Grill

, Kwon

, Teylan

, Pierce

, Vidoni

, Burns

, Lindauer

, Quinn

, Kaye

, Gillen

, Nan

(2019) Retention of Alzheimer disease research participants. Alzheimer Dis Assoc Disord 33, 299–306.

18.

Morris

, Weintraub

, Chui

, Cummings

, Decarli

, Ferris

, Foster

, Galasko

, Graff-Radford

, Peskind

, Beekly

, Ramos

, Kukull

(2006) The Uniform Data Set (UDS): Clinical and cognitive variables and descriptive data from Alzheimer Disease Centers. Alzheimer Dis Assoc Disord 20, 210–216.

19.

Robinson

, Dennison

, Wayman

, Pronovost

, Needham

(2007) Systematic review identifies number of strategies important for retaining study participants. J Clin Epidemiol 60, 757–765.

20.

Robinson

, Dinglas

, Sukrithan

, Yalamanchilli

, Mendez-Tellez

, Dennison-Himmelfarb

, Needham

(2015) Updated systematic review identifies substantial number of retention strategies: Using more strategies retains more study participants. J Clin Epidemiol 68, 1481–1487.

21.

Harris

, Taylor

, Thielke

, Payne

, Gonzalez

, Conde

(2009) Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42, 377–381.

22.

Keyes

, Utz

, Robinson

, Li

(2010) What is a cohort effect? Comparison of three statistical methods for modeling cohort effects in obesity prevalence in the United States, 1971-2006. Soc Sci Med 70, 1100–1108.

23.

Levkoff

, Sanchez

(2003) Lessons learned about minority recruitment and retention from the Centers on Minority Aging and Health Promotion. Gerontologist 43, 18–26.

24.

Arean

, Alvidrez

, Nery

, Estes

, Linkins

(2003) Recruitment and retention of older minorities in mental health services research. Gerontologist 43, 36–44.

25.

Gauthier

, Clarke

(1999) Gaining and sustaining minority participation in longitudinal research projects. Alzheimer Dis Assoc Disord 13 Suppl 1, S29–33.

26.

Russell

, Palmer

, Adams-Campbell

, Rosenberg

(2001) Follow-up of a large cohort of Black women. Am J Epidemiol 154, 845–853.

27.

Erickson

, Chin

, Johnson

, Gleason

, Clark

(2021) Disclosure of preclinical Alzheimer’s disease biomarker results in research and clinical settings: Why, how, and what we still need to know. Alzheimers Dement (Amst) 13, e12150.

28.

Examining Community-Institutional Partnerships for Prevention Research Group (2006) Building and sustaining community-institutional partnerships for prevention research: Findings from a national collaborative. J Urban Health 83, 989–1003.

29.

Mapetla

, Malahleha

, van Niekerk

, Thindisa

, Mpete

, Ahmed

, Dilraj

(2022) Establishing communication challenges and preferences among clinical trial participants in an under-resourced setting to improve adherence to study visits and participant retention. Clin Trials 19, 81–85.

30.

Hayes

, Sharma

(2021) Applying the integrated marketing communication approach to recruit and retain African American women. Health Promot Perspect 11, 460–466.

31.

Crouch

, Gordon

(2019) Prevalence and factors influencing use of Internet and electronic health resources by middle-aged and older adults in a US health plan population: Cross-sectional survey study. JMIR Aging 2, e11451.