Caller satisfaction with the Cannabis Information and Helpline

Abstract

We assessed caller satisfaction with an illicit drug helpline in Australia (the Cannabis Information and Helpline, CIH). A 10-min telephone interview was conducted with 200 volunteers who called the service during 2009 (121 called regarding themselves and 79 called regarding another person's cannabis use). Callers were a mean of 43 years of age and typically female (59%). Callers showed high levels of satisfaction with the service: the mean CSQ-8 score was 28.2 (SD = 4.3). Participants who found the service easy to access and those who felt their needs were met reported the highest satisfaction. A linear regression analysis was conducted to identify the significant predictors of participants' satisfaction with the CIH. The ease with which the participant was able to get through to a counsellor (standardized beta = 2.37, P < 0.02) and whether the participant felt that all their needs were met (standardized beta = −4.26, P < 0.001) were the only significant predictors of total satisfaction with the call. Despite the recognition that telephone services are possibly the easiest health-care service to access, ensuring consistent availability and accessibility remains paramount although not easy.

Introduction

Cannabis is the Western world's most frequently used illicit drug, despite links with a range of adverse health effects.^1,2 Although the rate of drug users entering treatment has increased over the last decade in the USA³ and Australia⁴, most people who use cannabis do not seek treatment, despite the published success and availability of several treatments.⁵ On a global scale, improving the availability and accessibility of drug dependence treatment is of paramount importance.⁶ Telephone helplines have improved access to treatment for people experiencing mental health crises.^7,8 They have also evolved into a promising means of providing assistance for people with tobacco dependence⁹ and general substance use concerns.¹⁰

Providing information about treatment options and access to treatments is particularly important in facilitating cannabis treatment seeking.¹¹ In response to a recognized dearth of national drug and alcohol helplines in Australia, the National Cannabis Strategy recommended that a helpline specific to cannabis-related concerns should be provided.¹² In 2008, Australia's first free national Cannabis Information and Helpline (CIH) was established.

However, providing access to information and education on treatment options alone may not suffice. People may need to be satisfied with the service before they will access it and continue with treatment.¹³ The measurement of satisfaction is important when evaluating the quality of medical health-care services¹⁴ and, more recently, substance abuse treatment.¹⁵ Despite the value of assessing client satisfaction, evaluations of telephone counselling services have been based mostly on their face validity and numbers of callers. Given the resources involved in providing telephone counselling services, there is a need to assess the content and level of clients' satisfaction with the services provided.

Satisfaction with drug helplines has rarely been researched due to methodological difficulties such as the desire to maintain anonymity of people who use telephone helplines, and a lack of resources.^16–18 A single study has been published on the helpfulness of American illicit drug helplines.¹⁹ However, this study referred to the opinion of psychologists making scripted calls and not to genuine caller satisfaction. The present study aimed to assess satisfaction with an illicit drug helpline by analysing callers' satisfaction.

Methods

Data were collected, via telephone interviews, from 200 participants who satisfied the study eligibility criteria of being aged 16 years or older and providing consent to be contacted by an independent researcher. The CIH is an Australian national service that provides information, counselling and referral information specific to cannabis-related problems. During the period of the evaluation, the CIH operated from 14:00 to 23:00 six days a week. The CIH routinely collects information about each answered call, including demographic details, aspects of the caller's cannabis use, reasons for calling, what happened during the call and call duration. In addition, the counsellors record which of 15 different counselling components were utilized during the call (such as reflecting, re-framing, grounding, summarizing, using open-ended questions). This CIH Dataset was used to supplement the data that were obtained by the evaluation interviews.

Evaluation interviews

Demographic data were collected using questions selected from the Australian National Minimum Data Set (NMDS), which is employed by clinicians in Australia to provide basic data, such as a client's main source of income, highest level of education and aspects of their living situation.²¹ The interview collected information about: basic demographics of the caller, how the caller heard of the service, perceived alternative help-seeking options, number of treatments previously accessed for cannabis concerns, ease of accessing a counsellor, reason for call, satisfaction with the service and counsellor, if an action plan was developed during the call and implemented thereafter, referral advice and follow-through, and general helpfulness of the call.

Data regarding satisfaction with the service were collected using the Client Satisfaction Questionnaire (CSQ-8, internal consistency coefficient α = 0.93).²² Feelings toward the counsellor were obtained using ten questions regarding empathy and warmth (internal consistency coefficient α = 0.70).²³ In addition to those published scales, other questions were developed specifically for this study. Those items used 11-point Likert scales (where 0 is not at all and 10 indicates the highest level of satisfaction/agreement) to address: ease of getting through to a counsellor; respect shown by the counsellor; perceived helpfulness of the call and any printed materials received; the impact that the call had on the caller's confidence in dealing with the situation discussed during the call; confidence in carrying out a plan of action or contacting a referral; amount of direction that the counsellor expressed in forming the plan of action; and the caller's likelihood of calling the service again.

Procedure

Approval for the study was granted by the appropriate ethics committee. At the end of each CIH call, verbal consent was obtained after the counsellors informed each caller about the nature of the study. The caller's identification code and contact information were then sent to the first author by the CIH.

The first author conducted a telephone interview approximately eight days (SD = 2) after the caller's initial conversation with the CIH to allow time for any printed information to be posted to the caller. The interviews took approximately 10 min to complete and participants were not reimbursed for their participation. Attempts to contact participants (typically two or three attempts were sufficient) ceased when more than three calls per day had been made over three days, or more than 14 days had passed since the individual had first called the CIH. Most participants were interviewed following their first call to the helpline (91%).

Data analysis

The quantitative and qualitative data from the study were analyzed using PASW.²⁴ Qualitative data were coded and organized into categories using the open coding techniques from grounded theory.²⁵ A variable depicting socio-economic status was computed by the addition of three indicators of relative advantage or disadvantage: the presence or absence of employment, tertiary education and property ownership.²⁶ A linear regression analysis was conducted to determine the significant predictors of satisfaction. Visual analysis of box plots revealed several univariate outliers and the Mahalanobis Measure distances revealed two multivariate outliers. As there was no reason to believe that these cases did not reflect real observations, and because their removal did not change the significance of the regression, the full dataset was used.

Results

Of the 2255 genuine calls (excluding nuisance calls or wrong numbers) answered by the CIH during the evaluation period between February and December 2009, 16% (n = 353) of callers were willing to leave their contact details. Of this group, more than half (57%) completed the research interview (n = 200). CIH counsellors recorded the main reasons for refusing to leave contact details for 510 calls; the most common reasons were: desire to end the call before the request could be made (42%); the caller's lack of interest (20%); the caller's concern for anonymity (19%); the caller had already participated in the study (5%); or other reasons (15%). The 200 participants had a mean age of 43 years (SD = 13), and 59% were female. Participants who called regarding their own cannabis use (n = 118) were more likely to be male (57%) and younger (mean of 38 [SD = 11] years) than participants who called regarding the cannabis use of a friend/family member (n = 82; 20% male and 49 [SD = 13] years, respectively). The majority (84%) were born in Australia, 3% were of Indigenous or Torres Strait Islander origin, a proportion similar to the overall Australian population.²⁰ Over half were employed (61%), had a tertiary education (52%) and owned their residence (60%).

Participant profile

Based on 181 recorded cases, the callers came from most Australian states (NSW = 40%; Victoria = 27%; Queensland = 20%; South Australia = 7%; Western Australia = 6%; Tasmania = 0.6%). Less than half of the sample (44%) indicated having used cannabis in the three months prior to interview. This group (n = 88) indicated that they had used cannabis on an average of 67 days (SD = 29) out of the previous 90 days.

Most participants (61%) were seeking assistance for themselves, with 40% calling to obtain assistance for a friend or member of the family. Prior to contacting the helpline, participants had expected the call to provide them with: counselling (53%); information/advice (30%); printed information (23%); someone to talk to (21%); strategies for a specific situation (15%); a referral (9%); or other help (7%). Other participants reported having no expectation of how the service was going to help them (9%). Those participants who were hoping to receive counselling (n = 106) were anticipating this to be for their own cannabis use (47%), or for a friend/family member's cannabis use (53%). Many participants (64%) had consulted with at least one health professional regarding their cannabis use problems before calling the helpline. This subsample (n = 127) reported visiting a: general practitioner (40%), counsellor/psychologist (24%), telephone counsellor (19%), drug treatment service (10%), alternative health/medicine worker (8%), mental health service (6%), hospital worker (3%) and/or social worker (3%).

Ease of access

Most participants were able to get through to a counsellor when they first called (80%). Those participants who were unable to get through on their first attempt (n = 40) tended to leave a message (55%). Participants rated the ease of reaching a counsellor with a mean score of 9.2 out of 10 (SD = 1.5).

Knowledge of CIH and need for assistance

Participants first heard of the CIH telephone number through: an online search (31%); advertising on the radio (23%); the Yellow Pages telephone directory (20%); health professional suggestion or referral (19%); a brochure (7%); or other means (1%). Many participants (48%) were reliant on the helpline at the time of their call, as they did not know of alternative professional help. Participants who were aware of alternative help (n = 104; 52%) identified their first alternative choice as: another helpline (29%), personal research (18%), a general practitioner (16%), face-to-face counselling (15%), specialist drug treatments (10%), colleagues or family (9%), information from brochures (2%) and alternative medicines (1%).

Outcomes from the call

Participants indicated how the call was helpful to them and rated the effect that the call had on their confidence in dealing with any associated problems (see Table 1). Less than half of participants (39%) made a ‘plan of action’ with the counsellor during the call. Those participants (n = 78) rated their confidence in carrying out that plan at a mean rating of 7.7 (SD = 2.5) out of 10. Participants indicated that the counsellor exhibited a small degree of direction in developing the plan, giving a mean rating of 6.9 (SD = 3.0) out of 10 (where 5 indicated a mutual decision).

Table 1

Average helpfulness of the call and related impact on caller's confidence

How was the call helpful?	Number of calls*	Helpfulness rating Mean (SD)	Impact rating Mean (SD)
Reducing cannabis use	27	7.2 (2.5)	5.7 (2.6)
Quitting cannabis use	23	7.9 (2.5)	7.5 (2.6)
Helped with family relationships	27	9.0 (2.0)	7.7 (2.6)
Provided support	51	8.9 (1.6)	7.8 (2.3)
Advice/information	86	8.2 (2.5)	7.1 (3.4)
Other help	17	7.8 (3.2)	6.3 (3.6)
Addressing concern over another person's cannabis use	79	7.4 (3.1)	†
Total	310	8.1	7.0

*Participants could select more than one way in which their call was helpful to them

^†Participants were not asked to rate their confidence that their friend or family member would address their own cannabis use

Fewer than half of the participants (42%) were offered a referral to alternative cannabis treatments during the call. For those offered a referral (n = 83), most (57%) had not made contact in the week after their call. Moreover, these participants indicated that they would most likely not make contact, rating the overall likelihood at an average of 4.3 (SD = 4.0) out of 10. Similar proportions (43%) asked to receive printed materials during the call. From this subsample (n = 85), two thirds (66%) had received the materials, although less than half (46%) had had a chance to read them before the interview. Those who had read the information by the time of interview (n = 26) indicated that the materials were helpful, rating them at an average of 8.3 (SD = 1.8) out of 10.

The different aspects of counselling that were utilized during each call are summarised in Table 2. The counsellors recorded that they utilized a mean of 10.5 (SD = 2.3) components of counselling during the calls.

Table 2

The components of counselling utilized during calls (n = 198). Two counsellors failed to enter data regarding the components of counselling that were utilized

Components of counselling	Percent of calls where component was utilized (%)
Listening and empathising	99
Open-ended questioning	99
Closed-ended questioning	99
Identifying issues or options	95
Reflecting	93
Offering emotional support	92
Exploring options	86
Summarising	81
Facilitating actions	70
Reframing	55
Grounding	49
Paraphrasing	48
Making a contract	41
Challenging	27
Limit-setting	12

Satisfaction with service

The CSQ-8 gives possible satisfaction scores ranging from 8 (lowest satisfaction) to 32 (highest satisfaction). The mean CSQ-8 score given for the helpline service was 28.2 (SD = 4.3) indicating high satisfaction. Approximately one-fifth (22%) of CSQ-8 scores were the highest possible score. No significant difference in CSQ-8 scores was found between those calling regarding their own or a friend/family member's cannabis concern. Participants indicated the likelihood that they would call the helpline back at an average rating of 7.9 (SD = 2.9) out of 10. The majority (91%) indicated that they at least might call back.

Approximately one-quarter (26%) of participants reported an unmet need following their call to the service. Those participants (n = 52) reported: a lack of provided information or support (37%); a need for better availability of other treatment services and/or available medications (17%); the CIH could not offer a call back service (14%); wanting something more formal and more directive (12%); or other grievances such as wanting counsellors to disclose personal information and finding that they were not permitted to (8%).

Perceptions of counsellors' skill

The empathy score that was used to measure satisfaction with counsellors ranged from −15 (demonstrating poor counselling skills) to +15 (demonstrating good counselling skills). Participants gave a mean score of +12.7 (SD = 3.1) indicating that the counsellors demonstrated good counselling. The vast majority (94%) of scores were above a neutral score of 0, and 40% gave the highest possible score. Furthermore, participants reported that counsellors were very respectful with an average respect score of 9.4 (SD = 1.2) out of 10. No significant differences in scores on counsellor skill or respect were found between those calling regarding their own or a friend/family member's cannabis concern.

Predictors of call satisfaction

A linear regression analysis was conducted to identify the significant predictors of participants' satisfaction with the CIH. The dependent variable, total satisfaction, was created in two steps. First, the measure of the perceived quality of the service (the CSQ-8) and perceived quality of the counsellor (the empathy scale) were standardized using Z-scores and, given their high correlation (r = 0.5, P < 0.001), their mean score was computed. Second, as this new variable was skewed positively it was log transformed. In order to preserve zero values the variable was shifted positively by one unit prior to the log transformation.

The regression analysis was designed to determine how aspects relating to the caller and the call itself contributed to predicting total satisfaction. Five predictor variables regarding the call and five predictor variables regarding the participant were included in the analysis. Variables regarding the call included: (1) how easy the participant felt it was to get through to a counsellor; (2) the duration of their call; (3) if the participant was offered a referral; (4) if the participant had an unmet need; and (5) if a plan of action was made. Variables regarding the participant included: (1) age; (2) sex; (3) socio-economic status; (4) the number of different health professionals accessed in the past 12 months; and (5) a dummy-coded variable indicating if the participant used cannabis or not. The final regression model (R² = 0.234, F = 4.3) suggested that the ease with which the participant was able to get through to a counsellor (standardized beta = 2.37, P < 0.02) and whether the participant felt that all their needs were met (standardized beta = −4.26, P < 0.001) were the only significant predictors of total satisfaction with the call.

Discussion

Overall, callers to the CIH were highly satisfied with the information and helpline regardless of whether they were calling for themselves or a friend or family member. This is in contrast to the findings of Hughes et al.,¹⁹ in which psychologists scripted calls to 25 American drug helplines. Hughes et al. rated the helpfulness of each drug helpline for a variety of scripted drug problems and found that only 25% of calls regarding cannabis concerns were thought to be helpful.

An important predictor of substance use treatment outcomes is the client's satisfaction with the service.¹⁵ The present study showed that more than 90% of participants were satisfied with the service and the counsellor, and 17% of participants gave the highest possible overall satisfaction score. Furthermore, the average helpfulness of the call was rated at 8.2 out of 10 and the likelihood of calling the service a second time was rated at 7.9 out of 10. These figures of high satisfaction appear typical of previous similar evaluations of mental health helpline services, such as Lifeline and Kids Helpline.²⁷

Although participants were satisfied with the CIH service, participants did not commonly make contact with referrals during the week before their research interview. In fact, the likelihood of calling back the CIH service was rated higher than the likelihood of following through with referrals. This suggests that service delivery by telephone may be necessary for some people. However, these results are unsurprising given that callers chose a helpline as their source of assistance.

Callers to the CIH were found to be dissimilar to the typical individual who accesses face-to-face or Internet-delivered substance abuse treatment. A study of entrants to Australian government-funded face-to-face cannabis treatments in 2007-2008 found that individuals were typically 20-29 years old and typically male (70%).⁴ An Australian study investigating client characteristics of those engaging in Internet-based, face-to-face and telephone-based substance abuse treatments found that individuals who engaged in Internet-based treatment were more likely to be female (68%) and younger than those accessing other forms of treatment.²⁸ In addition, participants in the present study who called the CIH regarding their own cannabis use problem were likely to be slightly older (mean age of 38 years) than people in the studies above and more likely to be male (57%) than those engaging in Internet services. It should also be noted that the total sample (59% female, mean age of 43 years) was not dissimilar to those who call other telephone counselling services throughout Australia where 61–82% of callers are female, and most callers are over 25 years old.²⁷ These findings demonstrate that different treatment modalities may offer qualities that help engage different groups of users with different demographic profiles.

Our study found that aspects of the call, including counsellor accessibility and helpfulness, rather than aspects of the caller, significantly predicted levels of satisfaction with the telephone service. These results are consistent with those of Bobevski et al. ²⁹ who found that counsellors who were more verbally active, explored all aspects of a problem situation, and attended to the emotional concerns of the caller, were those perceived to be the most helpful. In addition, previous research has found that aspects of individuals accessing face-to-face health-care services, such as demographic characteristics, have not consistently predicted service satisfaction.^14,15 The present findings suggest that telephone counselling services would not only benefit by ensuring that the caller is well attended to and has no unresolved needs, but equally, the service should ensure that each call is answered quickly. Perhaps the biggest problem that faces telephone counselling services is the ability to resource the number of hours of service and the staff required to answer the calls when they are made. This can be seen in the most recent review of Australian helplines in which the proportion of calls that were answered by three of Australia's biggest helpline services ranged from 10% to 38%.²⁷

The present study was not without limitations. First, the sample of two hundred callers was a small proportion of the 2255 genuine calls received by the CIH during the evaluation period and therefore, as is common when evaluating satisfaction with telephone helplines, there may have been bias in the sample of callers who were interviewed.^30,31 This bias favours individuals who showed indices of relative socio-economic advantage compared to the average Australian.²⁶ It is also conceivable that those who were unsatisfied with the CIH may have been less willing to participate in the study. Second, the telephone interview was conducted just over one week after the participant's call to the CIH to allow for printed information to be received by post and may have lost some validity due to the caller's inability to accurately recall details of their call. Nonetheless, this methodology provides the first independent evaluation of caller satisfaction with an illicit drug helpline. The study also found that caller characteristics were not significant predictors of satisfaction. However, the ease with which the call was answered and whether or not all the caller's needs were met did predict satisfaction. Thus, despite the recognition that telephone services are possibly the easiest health-care service to access, ensuring consistent availability and accessibility remains paramount although not easy. Further research is also required to identify the possibility of matching certain user characteristics with different treatment technologies.

References

United Nations Office on Drugs and Crime. World Drug Report 2009. New York: United Nations, 2009

Hall

, Degenhardt

. Adverse health effects of non-medical cannabis use. Lancet 2009;374:1383–91

Substance Abuse and Mental Health Services Administration. Treatment Episode Data Set (TEDS): 1996–2006 – National Admissions to Substance Abuse Treatment Services. Rockville, MD: United States Department of Health and Human Services, 2008

Australian Institute of Health and Welfare. Alcohol and Other Drug Treatment Services in Australia 2007–08: Report on the National Minimum Data Set. Canberra: Australian Government Department of Health and Ageing, 2009

Copeland

. Developments in the treatment of cannabis use disorder. Curr Opin Psychiatry 2004;17:161–7

United Nations Office on Drugs and Crime. Principles of Drug Dependence Treatment: Discussion Paper. See http://www.who.int/substance_abuse/publications/principles_drug_dependence_treatment.pdf (last checked 2 August 2010)

Auerbach

, Kilmann

. Crisis intervention: a review of outcome research. Psychol Bull 1977;84:1189–217

Lester

. The effectiveness of suicide prevention centers: a review. Suicide Life Threat Behav 1997;27:304–10

Stead

, Perera

, Lancaster

. Telephone counselling for smoking cessation. Cochrane Database Syst Rev 2006;3:CD002850

10.

Stichting European Council on Drug Helplines. Report on the 17th data collection. Changes during the second half of 2008. See http://www.fesat.org/uploads/files/Reports/FESAT%2017th%20Monitoring%20Report.pdf (last checked 2 August 2010)

11.

National Cannabis Prevention and Information Centre. Barriers and Facilitators to Cannabis Treatment. See http://ncpic.org.au/assets/uploads/files/barriers-and-facilitators-to-cannabis-treatment.pdf (last checked 2 August 2010)

12.

Australian Institute of Health and Welfare. National Cannabis Strategy 2006–2009. Canberra: AIHW, 2006

13.

Hawkins

, Baer

, Kivlahan

. Concurrent monitoring of psychological distress and satisfaction measures as predictors of addiction treatment retention. J Subst Abuse Treat 2008;35:207–16

14.

Crow

, Gage

, Hampson

, The measurement of satisfaction with health care: implications for practice from a systematic review of the literature. Health Technol Assess 2002;6:1–244

15.

Zhiwei

Zhang

, Gerstein

, Friedmann

. Patient satisfaction and sustained outcomes of drug abuse treatment. J Health Psychol 2008;13:388–400

16.

Ammenwerth

, Brender

, Nykänen

, Visions and strategies to improve evaluation of health information systems. Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. Int J Med Inform 2004;73:479–91

17.

McLellan

, Woody

, Metzger

, Evaluating the effectiveness of addiction treatments: reasonable expectations, appropriate comparisons. Milbank Q 1996;74:51–85

18.

Marinelli-Casey

, Domier

, Rawson

. The gap between research and practice in substance abuse treatment. Psychiatr Serv 2002;53:984–7

19.

Hughes

, Riggs

, Carpenter

. How helpful are drug abuse helplines? Drug Alcohol Depend 2001;62:191–4

20.

Australian Bureau of Statistics. Experimental Estimates of Aboriginal and Torres Strait Islander Australians 2006. See http://www.abs.gov.au/AUSSTATS/abs@.nsf/Lookup/3238.0.55.001Main+Features1Jun%202006?OpenDocument (last checked 2 August 2010)

21.

Copeland

, Conroy

. Australian National Minimum Data Set for clients of alcohol and other drug treatment services: findings of the national pilot and developments in implementation. Drug Alcohol Rev 2001;20:295–8

22.

Attkisson

, Zwick

. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann 1982;5:233–7

23.

Anastasi

. Psychological Testing. New York: Macmillan, 1976

24.

PASW Statistics Release 18.0. Chicago: SPSS, Inc., 2009

25.

Strauss

, Corbin

. Basics of Qualitative Research: techniques and procedures for developing grounded theory. 2nd edn. Thousand Oaks, CA: Sage Publications, Inc., 1998

26.

Adhikari

. Research Paper: Socio-Economic Indexes for Areas (SEIFA): Introduction, Use and Future Directions. Canberra: Australian Bureau of Statistics, 2006

27.

Australian Government Department of Health and Ageing. National review of telecounselling and web counselling services: final report. See http://www.health.gov.au/internet/main/publishing.nsf/content/mental-pubs-n-tele (last checked 24 September 2010)

28.

Swan

, Tyssen

. Enhancing treatment access: evaluation of an Australian web-based alcohol and drug counselling initiative. Drug Alcohol Rev 2009;28:48–53

29.

Bobevski

, Holgate

, McLennan

. Characteristics of effective telephone counselling skills. Br J Guid Counc 1997;25:239–49

30.

Glasgow

, Lando

, Hollis

, McRae

, La Chance

. A stop-smoking telephone helpline that nobody called. Am J Public Health 1993;83: 252–3

31.

Pascoe

. Patient satisfaction in primary health care: a literature review and analysis. Eval Program Plann 1983;6:185–210