Recruitment challenges associated with a randomised controlled trial within a general telephone counselling service

Abstract

We investigated recruitment into a randomised controlled trial (RCT) following contact with a generalist telephone counselling service. Telephone counsellors were required to ask callers if they would be willing to receive a telephone call to discuss their possible involvement in a trial designed to evaluate the use of a web-based mental health intervention. The five centres involved answered a total of 34,722 counselling calls during the recruitment period. Only 9% of callers were invited by the telephone counsellor to receive the recruitment call and of all callers, only 2.6% agreed. Common reasons reported by the telephone counsellors for not referring eligible applicants to the trial included their view that it was inappropriate to ask the caller, feeling uncomfortable about asking the question, being concerned that the invitation might affect the counselling relationship and forgetting to ask. The presence of an on-site trial manager, regular encouragement and feedback, and tailored and specific recruiter training may improve rates of recruitment in future trials.

Introduction

Recruitment is an important matter in clinical trials, because low entry can compromise trial quality and power, and may extend trial duration and costs.¹ Poor recruitment may render a trial useless. Recruitment to clinical trials from general practice is particularly difficult,² although it is not clear why this is the case. Some evidence suggests that recruitment may be more successful if remuneration is offered.³ Experts in clinical trial management have offered a number of suggestions to improve clinical recruitment including: (1) a simple and coherent research design; (2) the employment of experienced staff and proven practices; (3) engagement with and the provision of feedback to practitioners; and (4) the use of the media.⁴ Some recent work has examined patient preferences for randomisation,⁵ but this area of investigation is relatively new.

To our knowledge there has been no study of recruitment with respect to randomised controlled trials (RCTs) within a community telephone counselling service. In the present study, located within a non-government organization (NGO), we anticipated that recruitment difficulties would arise, as we would be using a diverse, heterogeneous population of volunteers with varying levels of experience with and attitudes towards conducting research. Accordingly, we devised a number of organisational, training and management strategies to assist the volunteers to manage their roles as both telephone counsellors and participant recruiters. Consultation was undertaken to design protocols that would fit as seamlessly as possible within the existing structure. Once these protocols were developed, extensive training was instigated to describe the rationale and procedures of the trial. This included presentations to staff within the organization, at all levels, including directors, centre managers, telephone counselling supervision staff and telephone counsellors. Training emphasised the potential benefits of the trial for both telephone callers (improved outcomes) and for the NGO (quality assurance, improving the service). It also focussed on the technical aspects of recruitment, such as how to introduce the project to callers and how to use the project's online software to record relevant personal information. Evidence describing the needs of callers was provided to telephone recruiters.⁶

The NGO comprised a number of call centres situated in different regions of Australia. The majority of participants were recruited from one of these centres (Centre 1) although the trial also recruited from four other centres in order to increase sample size. A trial manager who coordinated and implemented the project, and liaised between the project researchers and the NGO, was accommodated full-time in Centre 1 over the three-year period of the project. Attempts were made to minimize the load on counsellors by not requiring that they undertake any screening, except assessing callers’ access to the Internet. The trial manager kept in contact with telephone counsellors by visiting the call centres during the recruitment periods, and providing regular feedback about the rate of recruitment to telephone counsellors. Monetary incentives were offered to telephone counsellors to enhance recruitment.

Despite these attempts to facilitate recruitment, the recruitment rate was low. We therefore conducted a survey to determine the reasons that telephone counsellors did not refer eligible callers to the trial and to suggest how these barriers might be targeted in further trials.

Methods

Five telephone counselling centres (Lifeline Australia) participated in the trial. The protocol was approved by the appropriate ethics committee.

At the end of a call to Lifeline's crisis line, the telephone counsellors were required to invite callers to participate in a research project which ‘aimed to improve the telephone service’. The trial aimed to evaluate the effectiveness of a web-based mental health intervention compared to a treatment as usual condition (TAU) involving access to telephone counselling as needed. All callers were eligible to receive an invitation to the project, with the exception of callers with immediate distress, suicidal thoughts, or callers seeking only a specific referral to a health-care provider, or individuals who did not have Internet access. Callers provided their name and telephone number, and the trial manager contacted them directly to discuss the project further.

Centres 1 and 2 recruited participants for 14 months from June 2007 to August 2008. Centre 3 recruited participants for 4 months from May 2008 to August 2008, Centre 4 recruited participants for 3 months from June 2008 to August 2008 and Centre 5 recruited participants for 2 months from mid-November 2008 to mid-January 2009. Telephone counsellors across all work shifts were involved in recruitment except those at Centres 3 and 5, where only telephone counsellors on overnight shifts were involved. These overnight telephone counsellors were part of other trials examining the effect of using paid staff on call answering rates. All other staff were unpaid volunteers.

Following the recruitment phase of the project, telephone counsellors from all centres completed a feedback questionnaire on the recruitment process.

Outcome variables

Telephone counsellors were provided with a list of 14 reasons for not inviting a caller to participate in the study (see Table 1), and asked to what degree each statement applied to them using one of five possible response categories: ‘applied none of the time’, ‘applied a little of the time’, ‘applied some of the time’, ‘applied most of the time’ or ‘applied all of the time’. These response categories were later dichotomised to either ‘applied none of the time’ or ‘applied at least a little of the time, if not more’ (which included the remaining four categories). Response categories were dichotomised in this way to compare the absence of an endorsement of a particular reason versus any form of endorsement, and to ensure sufficient numbers of participants in each category to conduct appropriate analyses.

Table 1

Telephone counsellors’ reasons for non-recruitment

Reason	n	Applied none of the time (%)	Applied at least a little of the time (%)	Overall main reason (%), n = 95
I didn't know how to provide an invitation	85	71	29	1
I felt that I didn't understand the project well enough	87	59	41	5
I felt too uncomfortable or awkward to ask the questions*	88	42	58	6
I did not feel that the callers I spoke to were appropriate for an invitation*	91	6	95	41
I forgot to ask the caller*	90	8	92	25
I forgot to submit the online invitation	86	80	20	1
I don't think it is appropriate to conduct research with Lifeline callers	82	73	27	0
I think the aims of the project are not in line with the aims of the Crisis Line	85	63	37	1
I felt that asking the questions would affect negatively the relationship between me and the caller*	90	38	62	7
I don't think I should be expected to do anything extra in addition to telephone counselling	83	82	18	0
A caller that I asked reacted negatively, so I did not ask any other callers	86	88	12	0
I got tired of asking because none of the callers I asked were interested or had access to the Internet	89	58	42	11
No other telephone counsellors on my shift were asking callers, so neither did I	84	94	6	0
I think the regular referrals that we offer are enough for callers at the moment	82	73	27	0

*Reasons endorsed by more than 50% of respondents

Telephone counsellors were asked to nominate one main reason for not inviting callers: ‘If you have to choose one main reason from the statements listed above what would it be?’

Predictor variables

There were three predictor variables:

A dichotomous variable was created to represent the presence or absence of the on-site manager. There was an on-site manager at Centre 1 and no on-site manager present at Centres 2, 3, 4 and 5;

A length of recruitment period variable was created with two categories: long recruitment period (Centres 1 and 2; 14 months) and short recruitment period (Centres 3, 4 and 5; 2–4 months);

A continuous variable was created based on how long (in years and months) survey respondents had been working as a Lifeline telephone counsellor.

Statistical analysis

Logistic regression was used to examine the relationships between presence of manager, length of recruitment period, and telephone counsellor experience and each of the reasons for non-recruitment. These analyses were only undertaken for those items where at least 25% of the sample nominated that the reason either did or did not apply. This was to ensure the validity of the logistic regression approach and the need to exclude items where sample size differences were too extreme. All three predictor variables were significantly correlated (presence of manager and length of recruitment period: r = 0.68, P < 0.001; presence of manager and telephone counsellor experience: r = 0.33, P = 0.001; length of recruitment period and telephone counsellor experience: r = 0.26, P = 0.009). Initial models examined each predictor separately to determine if the overlap between the combined predictors produced divergent findings. These earlier models were consistent with the final combined model into which all three predictors were entered.

Results

The five centres answered a total of 34,722 counselling calls during the recruitment period. Of this total, 3143 (9%) callers were invited to the project, and of these 1818 expressed some interest (58%), while 29% did not and 13% indicated that they had already been invited during a previous call to the service. Of the 1818 callers who expressed some interest, 1076 (59%) had Internet access and 910 agreed to a call-back by the trial manager for possible recruitment to the trial. Overall this resulted in a recruitment rate of 2.6%.

Centre 1 invited 10.7% of their counselling calls to the project during their recruitment period. Centre 2 invited 6.7% of their counselling calls, Centre 3 invited 10.8% of their counselling calls, and Centres 4 and 5 both invited 5.9% of their counselling calls. The differences between these rates were significant (χ² = 190.8, P < 0.001). The centres with non-paid telephone counsellors recruited 9.3% of their counselling calls, while centres with paid telephone counsellors recruited 6.4% of their counselling calls. These rates of recruitment were significantly different (χ² = 24.7, P < 0.001).

In all, 595 telephone counsellors were involved in the recruitment of callers at some stage during the recruitment period. These represented 233, 60, 3, 273 and 26 counsellors respectively from Centres 1–5.

Reasons for non-recruitment

A total of 95 telephone counsellors (16%) completed the feedback survey, with 59, 15, 1, 11 and 9 from Centres 1–5, respectively. This represented response rates of 25, 25, 33, 4 and 35%, respectively, from the five centres.

The questionnaire responses are summarized in Table 1. Reasons marked with an asterisk were endorsed by more than half of telephone counsellors as reasons for non-recruitment of callers at least a little of the time, if not more. When asked about the single main reason for not inviting callers, 41% of telephone counsellors indicated that they felt it was inappropriate to ask that particular caller at the time. A further 25% indicated that they forgot to ask the caller. Other main reasons are shown in Table 1.

The questionnaire results according to centre characteristics and counsellor experience are summarised in Table 2. The relationship between the presence of the manager, the length of the recruitment period and the extent of telephone counsellor experience and each of the reasons for non-recruitment is summarised in Table 3. When looking at predictors for ‘I didn't know how to provide an invitation’, the only significant predictor was the presence of the manager (OR = 0.20, 95% CI = 0.05–0.78) which resulted in fewer endorsements of this item. When looking at predictors for ‘I think the aims of the project are not in line with the aims of the crisis line’, the single significant predictor was length of recruitment period (OR = 7.5, 95% CI = 1.4–41.1). Telephone counsellors at centres which had a long recruitment period were more likely to report that they thought the aims of the project were not in line with the aims of the crisis line. There were no other significant predictors for any of the other eligible items.

Table 2

Questionnaire results according to centre characteristics and counsellor experience (n = 95)

Predictors	Percentage
On-site manager present	62
Long recruitment period (i.e. 14 months)	74
	Mean (SD)
Length of telephone experience (months)	43 (49)

Table 3

Relationship between presence of on-site manager, length of recruitment period and telephone counsellor experience and the reasons for non-recruitment

	Applied none of the time (%)	Regression coefficient	P value
Provide an invitation	71
On-site manager		−1.61	0.02*
Recruitment period		0.87	0.26
Telephone counsellor experience		−0.02	0.07
Understand the project	59
On-site manager		−0.03	0.96
Recruitment period		−0.62	0.39
Telephone counsellor experience		−0.01	0.39
Uncomfortable to ask questions	35
On-site manager		−0.33	0.62
Recruitment period		0.05	0.94
Telephone counsellor experience		−0.01	0.39
Inappropriate to conduct research with callers	73
On-site manager		−0.37	0.60
Recruitment period		0.57	0.50
Telephone counsellor experience		0.01	0.68
Aims not in line with crisis line	68
On-site manager		−1.19	0.08
Recruitment period		2.01	0.02*
Telephone counsellor experience		0.01	0.09
Questions would impact negatively on relationship	38
On-site manager		0.70	0.28
Recruitment period		−0.69	0.36
Telephone counsellor experience		−0.01	0.56
Got tired of asking	58
On-site manager		1.10	0.13
Recruitment period		0.03	0.97
Telephone counsellor experience		0.01	0.15
Regular referrals are enough	73
On-site manager		1.19	0.21
Recruitment period		−0.28	0.78
Telephone counsellor experience		0.01	0.16

*Significant at P < 0.05

Discussion

In our trial only 9% of callers to the service received an initial invitation to the trial, and only 3% of callers received a follow-up recruitment call. These rates of recruitment into the trial were relatively low compared to other community-based mental health RCTs. For example, a recent trial investigating web-based depression interventions with a community sample received questionnaires from 23% of its initial sample, although less than 15% of these gave consent or were eligible to enter the trial at the next stage.⁷ A trial investigating telephone-based mental health interventions using primary care patients recruited approximately 30% of patients identified from a primary care clinic.⁸ However, a fundamental difference between these trials and our own was the recruiting method. Instead of recruiting participants via mass mailed surveys or identifying them through computerized patient records, we required telephone counsellors to recruit callers during a counselling call. However, there have been few trials within the NGO sector to serve as comparators. Moreover, comparison between recruitment rates is not possible because the studies typically fail to provide data describing the size of the pool from which trial participants are recruited. Finally, the nature of the recruitment samples in community trials has varied markedly across studies (e.g. community participants, medical patients).

Although little is known about recruitment to trials, previous research has shown that Lifeline telephone counsellors are able to achieve reasonable recruitment rates for survey research. In previous surveys profiling the mental health of callers to several different Lifeline centres, telephone counsellors have invited 31%⁶ and 23%⁹ of callers to be involved in these surveys. Furthermore, Burgess and colleagues found that 74% of callers to the Lifeline service expressed an interest in participating in future research trials.⁶ This suggests that the telephone counselling service can be a viable method for recruitment. The lower recruitment rate in the present trial may have been due to one or a combination of factors. First, the offer of a treatment or intervention may have been a critical factor in whether the counsellor was prepared to hand over the caller to the trial manager. Although counsellors may have felt comfortable with survey collection, offering a service to improve mental health may have been a strong barrier to referral. Second, the requirement that the counsellor pass the caller to a new project person may have discouraged recruitment. Third, collecting, recording and handing over the personal information of callers required a role change for counsellors working in an anonymous counselling environment. Hunt and colleagues reported similar difficulties associated with the role change from practitioner to scientist-practitioner in their research with general practitioners (GPs) as trial recruiters.²

Centres 1 and 3 had the highest recruitment rates. This could be attributed to a number of factors. Centre 1 was involved in recruitment for a considerably longer time than other centres, and the project trial manager was physically present at this centre during the recruitment period. Centre 3 had only a few telephone counsellors involved in recruitment. These factors allowed for more frequent and effective communication with telephone counsellors at these centres. Overall, recruitment rates were very similar between centres with paid and non-paid telephone counsellors.

Most callers to the service (91%) were not invited to be involved in the trial. This low rate of recruitment cannot be explained solely by the presence of frequent repeat callers to the service⁶ or suicide-related/immediate high distress in callers, each of which rendered callers ineligible for participation. Rather it would appear that eligible callers were not invited by telephone counsellors to participate. Previous research has shown that callers to the service experience more mental health symptoms relative to the general population⁶ and it would have been expected that an initial invitation to the project would have been more frequently offered.

The main reasons indicated by telephone counsellors for non-recruitment included their perception that it was inappropriate to invite the caller to participate and the counsellors’ forgetfulness. These reasons are similar to those cited in general practice research.^2,3,10,11 This suggests that these reasons should be targeted in any further trials. For example, recruiter training should focus on presenting tailored information about who can and should receive research invitations. The findings also support the importance of ongoing communication between trial managers and recruiters, to enhance engagement and actively encourage recruitment. The current findings also indicate that the presence of an on-site manager helps recruiters to understand the recruitment process and how to provide their callers with research invitations.

Over half of telephone counsellors also indicated that they were uncomfortable asking the recruitment questions and were concerned about the effect on the relationship with the caller and cited these as a reason for non-recruitment at least a little of the time, if not more. Again, similar concerns have also been expressed by GPs when required to recruit patients during their consultations,¹² a finding which supports the suggestions above that the ‘handing over’ of a caller and ‘loss of control’ may be key factors in the observed low recruitment rates.

The findings also indicate that telephone counsellors recruiting for longer periods of time consider that the aims of the project are not in line with the aims of the crisis line. It is unclear why this concern should be greater among counsellors from centres engaged in the practice for a longer period of time. However, it has previously been reported that ‘cultural’ factors commonly affect research in clinical practices and that there is unease with moving from old practices towards newer evidence-based practices.² The longer that research is present in an organization, the greater is the awareness that the processes associated with the research and its outcomes are part of ordinary service delivery. Again, it is possible that the problem may be addressed by better targeted and relevant training, and information that is focused on the potential benefits of the research and how the invitations can be integrated into the current service.

A major limitation of the present study was the low rate of participation by telephone counsellors in the feedback survey. Most of the telephone counsellors (84%) did not answer the survey. This may be due in part to the retirement or departure of telephone counsellors, rather than to non-compliance by those undertaking recruitment. Lifeline has a high turnover rate of volunteer counsellors and this represents another challenge to recruitment through this organisation.

In summary, the present study is one of the first to examine recruitment problems for research conducted in a community-based telephone counselling setting. Despite the low level of recruitment into the trial, the RCT was successfully completed within the NGO. The reasons for low recruitment should be targeted by future trials. Meanwhile, the presence of an on-site trial manager and regular encouragement and feedback may facilitate recruitment in similar trials. Tailored recruiter training focusing on the benefits of the research may also help. Successful RCTs are possible with close collaboration between researchers and the recruiting organization.

Footnotes

Acknowledgements

Helen Christensen and Kathleen Griffiths were supported by NHMRC Fellowships (no. 366781 and 525413, respectively). The study was funded by an ARC Linkage Grant (LP0667970). We thank Lifeline Australia, and in particular Dawn Smith, Trevor Carlyon and Julie Aganoff for their assistance.

References

Yallop

, McAvoy

, Croucher

, Primary health care research – essential but disadvantaged. Med J Aust 2006;185:118–20

Hunt

, Shepherd

, Andrews

. Do doctors know best? Comments on a failed trial. Med J Aust 2001;174:144–6

Pearl

, Wright

, Gamble

, Doughty

, Sharpe

. Randomised trials in general practice – a New Zealand experience in recruitment. N Z Med J 2003;116:U681

Grant

, Mackinnon

, Christensen

, Walker

. Participants' perceptions of motivation, randomisation and withdrawal in a randomised controlled trial of interventions for prevention of depression. J Med Ethics 2009;35:768–73

Hague

, Gebski

, Keech

. Recruitment to randomised studies. Med J Aust 2003;178:579–81

Burgess

, Christensen

, Leach

, Farrer

, Griffiths

. Mental health profile of callers to a telephone counselling service. J Telemed Telecare 2008;14:42–7

Christensen

, Griffiths

, Jorm

. Delivering interventions for depression by using the internet: randomised controlled trial. BMJ 2004;328:265

Simon

, Ludman

, Tutty

, Operskalski

, Von Korff

. Telephone psychotherapy and telephone care management for primary care patients starting antidepressant treatment: a randomized controlled trial. JAMA 2004;292:935–42

Perkins

, Fanaian

. Who calls Lifeline: Baseline study of callers and their needs (Final Report). Centre for Equity and Primary Health Care Research in the Illawarra and Shoalhaven, University of New South Wales, 2004

10.

van der Windt

, Koes

, van Aarst

, Heemskerk

, Bouter

. Practical aspects of conducting a pragmatic randomised trial in primary care: patient recruitment and outcome assessment. Br J Gen Pract 2000;50:371–4

11.

Minas

, Klimidis

, Kokanovic

. Mental health research in general practice. Australas Psychiatry 2005;13:181–4

12.

Mason

, Shaw

, Wiles

, GPs' experiences of primary care mental health research: a qualitative study of the barriers to recruitment. Fam Pract 2007;24:518–25