Feasibility randomised controlled trial of a self-help acceptance and commitment therapy intervention for grief and psychological distress in carers of palliative care patients

Abstract

We tested the feasibility and preliminary effectiveness of an acceptance and commitment therapy self-help intervention for grief and psychological distress in carers of patients in palliative care. Carers were randomised to the control group, which received treatment as usual, or the intervention group, which received treatment as usual plus an acceptance and commitment therapy–based self-help booklet and telephone support call. Questionnaires were completed at baseline, 1-month post-allocation and 6 months post-loss. Results indicated that the intervention was generally feasible and viewed as acceptable to carers. Preliminary effectiveness analyses showed at least a small effect in acceptance, valued-living, grief and psychological distress.

Keywords

acceptance and commitment therapy carer feasibility palliative care self-help

Introduction

Taking on a caring role for a loved one in palliative care can have a substantial impact on an individual’s well-being. In a recent systematic review of the burden among carers of patients with advanced or terminal illness, most studies identified that carers were overburdened and that this was associated with the development of complications in the grieving process (Delalibera et al., 2015). Carers can experience high levels of psychological distress and anticipatory grief while caring for their loved one, with high levels of anticipatory grief being associated with poor bereavement outcomes such as prolonged grief disorder (PGD; Nanni et al., 2014; Thomas et al., 2014). PGD is a proposed clinical syndrome defined as persistent, debilitating grief reactions post-loss (Prigerson et al., 2009) and is included in the International Classification of Diseases-11 that is due for release in 2018 (Maercker et al., 2013). The psychological challenges associated with bereavement and grief are also relevant to the wider support system of close friends and family, who do not necessarily differ from primary carers in their overall perception of need for support (Burns et al., 2013).

With the aging population, there will be an increasing burden placed on the health system to manage chronic and progressive diseases. Self-help interventions are a feasible option low resource intervention to support carer adjustment. They have been identified in systematic reviews as both effective (Lewis et al., 2012; Musiat and Tarrier, 2014) and cost-effective (Donker et al., 2015; Musiat and Tarrier, 2014; Solomon et al., 2015) for a variety of mental health conditions. To date, no known studies have evaluated a self-help intervention for carers of patients in palliative care.

Research suggests that providing anticipatory support to help carers prepare for the patient’s loss may be beneficial to adjustment during bereavement (Burke et al., 2015; Clark et al., 2011; Weissflog and Mehnert, 2015) and needs further investigation (Schut and Stroebe, 2010). However, reviews have concluded that there is inconsistent evidence on the benefit of psychotherapeutic interventions on carer psychological suffering (Candy et al., 2011; Gauthier and Gagliese, 2012; Harding et al., 2012; Harding and Higginson, 2003; LeMay and Wilson, 2008; Peacock and Forbes, 2003) and for grief in general (Waller et al., 2016). Methodological arguments have been proffered to explain the mostly small to moderate treatment effects and the inconsistency and difficulty in interpreting outcomes from these studies (Currier et al., 2008; Waller et al., 2016). The strongest treatment effects have been found for bereaved individuals who are ‘clinically indicated’ by a clinical screening tool, as compared to selective interventions for populations identified as at risk or universal interventions for all grievers (e.g. Currier et al., 2008).

A promising approach for managing the difficulties experienced by carers of patients in palliative care is acceptance and commitment therapy (ACT). A more detailed explanation for the theoretical and empirical rationale of ACT for carers in palliative care has been provided previously (Davis et al., 2016, 2019). To date, no observational or intervention research has been conducted that examines the potential of ACT among carers of patients in palliative care. ACT is a mindfulness-based therapy that has an increasing evidence base for effectively treating a number of psychological disorders (A-Tjak et al., 2015; Twohig et al., 2015). ACT has also been demonstrated as amendable to the self-help format for depressive and anxiety symptoms and overall psychological well-being (Cavanagh et al., 2014; Fledderus et al., 2012; Forsyth, 2011; Johnston et al., 2010; Lappalainen et al., 2014; Muto et al., 2011; Pots et al., 2016; Rasanen et al., 2016).

Understandably, carers of patients in palliative care can experience thoughts and feelings about their situation as overwhelming and disengage from leading an actively fulfilling life in an attempt to cope. Two highly relevant mechanisms of therapeutic change in ACT include experiential avoidance and engagement in valued behaviour (Davis et al., 2015). Experiential avoidance is the opposite term to our use of acceptance in this article. It describes an unwillingness to remain in contact with unwanted private events (i.e. thoughts, feelings, sensations, memories) and becomes problematic when it is rigidly and pervasively applied to the extent that it impinges on what pursuit of what is important and meaningful in one’s life (Hayes et al., 1999, 2012). In ACT treatment, individuals utilise acceptance and mindfulness skills and commitment and behaviour change processes (Hayes et al., 2011). These enable individuals to develop a different relationship with unwanted private events so that they are no longer perceived as negative experiences to be avoided, but rather as harmless transient psychological events. This permits individuals to engage in valued behaviour in the presence of whatever unwanted private events arise.

Both experiential avoidance and valued-living have been implicated in prolonged grief (PG) symptoms and psychological distress in bereaved populations (Boelen et al., 2010; Davis et al., 2016). Theoretically, lower levels of experiential avoidance would be expected to lead to reduced grief and psychological distress by enabling carers to let their thoughts and feelings be as they are and to explore them with curiosity. This would help make the experience of having unwanted thoughts and feelings more manageable and viewed less negatively, and ultimately less encumbered to engage in what matters most to them. Higher levels of valued-living would likewise be expected to reduce psychological suffering by helping carers engage in positively reinforcing activities that enrich their life and bolster their well-being.

An additional advantage of ACT is that it is transdiagnostic, meaning it is consistent with a universally applied principles approach where treatment principles are applicable to the human condition rather than being limited to psychopathology and thus clinical populations only (Kashdan and Rottenberg, 2010; Sauer-Zavala et al., 2017). In this way, ACT principles are theoretically helpful across the spectrum of grief responses, from ‘normal’ grieving to those that have become problematic for the individual. It also remains to be found whether a transdiagnostic approach such as ACT would prove equally efficacious as an intervention delivered to all carers as for those who are clinically indicated only.

In sum, there is a need for the development of interventions for carers that support their psychological adjustment and can viably be widely distributed and evaluated using rigorous randomised controlled trials (RCTs). We therefore sought to test the feasibility of an ACT self-help intervention for carers of patients in palliative care prior to a main trial to examine effectiveness.

Objectives

The primary objective of this study was to assess feasibility of the intervention for carers of patients in palliative care (Davis et al., 2019). Specifically, we sought to (1) test the feasibility of recruitment, attrition and data collection procedures; (2) determine engagement with the intervention through rates and amounts of protocol completion; and (3) evaluate acceptability of the intervention to carers of patients in palliative care. The secondary objective of this study was to evaluate preliminary effectiveness of the intervention on increasing acceptance and valued-living, while reducing grief and psychological distress, at 1-month follow-up (Davis et al., 2019).

Methods

Our methods have been described in detail in a published protocol (Davis et al., 2019). The essential elements are reported here.

Ethics

Ethical approval was obtained from the Joint University of Wollongong and Illawarra Shoalhaven Local Health District Health and Medical Human Research Ethics Committee (HE13/464). Written informed consent was obtained for all participants prior to participation in accordance with the World Medical Association Declaration of Helsinki – ethical principles for medical research involving human subjects.

Setting

Participants were recruited from two inpatient units within the South Coast of NSW, Australia. One was 15-bed palliative care unit while the other was 24-bed unit for management of both cardiac/respiratory rehabilitation and palliative care. The recruitment period was between March 2014 and August 2016, with follow-up continuing until February 2017.

Participants

Eligible carers were aged 18 years and above and English speaking. Patients were diagnosed with a life-limiting illness and recognised by their treating clinician as being within the last 6 months of life. Carers were identified as the primary informal caregiver who provided most of the informal day-to-day care to the patient, while significant others were identified by the patient or carer as someone who provided informal care, assistance or support and was perceived as being substantially affected by the patient’s situation. For ease of reference, the term ‘carers’ will be inclusive of ‘significant others’ unless otherwise specified.

Sample size

A sample size of 30 participants per arm is recommended for feasibility and pilot studies (Browne, 1995). This number enables calculation of the critical parameters relating to the feasibility outcomes (e.g. attrition rates) and a reasonable indication of the likely sample size required for a larger trial (Arain et al., 2010; Thabane et al., 2010). To account for an estimated 40 per cent attrition rate that had been observed in similar intervention or longitudinal studies with carers (Hudson et al., 2013; Kapari et al., 2010; Steinhauser et al., 2006a), we aimed for 80 carers to complete the trial.

Design and procedure

This study is a two-arm RCT (Phase I/II). Carers and significant others were randomly allocated to the intervention or control group, with carers and significant others from the same social network allocated together. The control group received treatment as usual while the intervention group received treatment as usual plus the booklet and telephone support. Participants were randomised according to computer-generated random numbers performed by a research assistant blinded to the identity of participants. Recruitment packages were pre-randomised in order to minimise the time frame between enrolment into the study and delivery of the intervention, which was particularly important given the patient’s imminent death. All data were collected by the first author (E.L.D.) who was aware of the group allocation of participants. Carers completed a questionnaire at baseline and two follow-up questionnaires at 1 month after group allocation and 6 months post-bereavement.

Recruitment and follow-up

Carers were invited to take part in the study by E.L.D. or clinical staff (i.e. social workers, nurses, registrars and doctors) who were trained as recruiters to the study. Recruiters introduced the study to carers and provided a brochure offering a summary on the study and describing what was involved in participation. If the carer was absent, recruiters would ask patients if they would like to nominate their carer to receive more information about the study.

E.L.D. attended the participating sites weekly and provided any interested carers with a pre-randomised study package containing an information sheet and consent form, baseline questionnaire, reply paid envelope and the self-help booklet if allocated. Carers who were nominated received the study package with a letter of invitation via a family member or post. The self-help booklet was sealed in an envelope and carers had instructions to complete the baseline questionnaire prior to opening it.

The 1-month post-allocation and 6 month post-bereavement questionnaires were posted to participants with a reply paid envelope.

Intervention

Skills-based booklet and telephone support

Based in ACT, the aim of the self-help booklet was to help carers learn skills to manage their difficult thoughts and feelings and pursue values-based action. It contained psychoeducation and experiential mindfulness exercises, of which some were included in an accompanying CD. Consumers and experts in ACT and palliative care were consulted in the development of the booklet and their feedback on drafts was incorporated into the final version.

Carers received a phone call from E.L.D. after 1–2 weeks of receiving the booklet. E.L.D. was a clinical psychology PhD student with training and clinical supervision in ACT and was the primary author of the booklet. The primary aim of the phone call was to provide support in understanding the material and personal application. If a carer became bereaved within 1–2 weeks of receiving the booklet, the phone call was delayed an additional 2 weeks based on current practice of site staff in bereavement follow-up support. If the carer was unable to be contacted, they were sent a letter notifying them of the unsuccessful attempts to contact them and inviting them to contact the research team at their earliest convenience.

Treatment as usual

Psychosocial support was available to all carers before and after the patients’ death. This was primarily provided by social workers, which included psychosocial assessment, counselling, advocacy and assistance in navigating health and community systems and coordinating services. Social workers also delivered a bereavement service, which involved an information pack and follow-up phone call, and they organise an annual Memorial Service for friends and family of patients who died over the past year. Access to a clinical psychologist was also available to carers on an as-needs basis, although availability was limited due to low staffing levels.

Measures

A summary of the outcomes measures collected is provided here with more detail available in the published protocol (Davis et al., 2019).

Feasibility and acceptability outcomes

The movement of carers into and out of the study was recorded to provide response and attrition rates. Questionnaire acceptability was assessed at baseline through four purpose-designed items asking about their experience of completing the questionnaire and whether they would do it again knowing what was asked (1 = strongly disagree to 5 = strongly agree).

At 1-month follow-up, carers in the intervention condition were asked a series of purpose-designed questions to evaluate the completion and acceptability of the self-help booklet and telephone support. They were asked to provide an overall indication of the extent to which they read the booklet (0 = I did not read it to 4 = from start to finish), the extent that they had completed the 10 sections and 14 exercises, and whether they found them helpful (1 = not at all helpful to 5 = very helpful). A total of 10 items assessed their general opinions of the content and helpfulness of the booklet (1 = strongly disagree to 5 = strongly agree). Finally, they were invited to provide qualitative comments on the booklet during the follow-up phone call and in the questionnaire.

Preliminary effectiveness outcomes

Experiential avoidance

The Acceptance and Action Questionnaire–II (AAQ-II; Bond et al., 2011) is the most widely used measure of experiential avoidance. It contains 7 items with statements assessing how participants relate to their thoughts, feelings and memories (1 = never true to 7 = always true). Higher total scores indicate higher levels of experiential avoidance, with scores ranging from 7 to 49.

Valued-living

The Valued Living Questionnaire (VLQ; Wilson et al., 2010) comprises two parts. First, participants rate the importance (1 = not at all to 10 = extremely) of 13 domains of living (e.g. marriage, work, leisure). They then rate how consistently (1 = not at all to 10 = extremely) they have lived in accordance with their values within each domain over the past week. Domains of psychological well-being, financial security/prosperity and autonomy/independence were added to the existing 10 domains in the original scale because they were considered relevant to the sample but not necessarily captured in the existing domains. Valued-living was calculated as the mean consistency score from only those domains rated 5 or above for importance (since these were considered sufficiently important that they would likely want to be pursued). Higher scores indicate higher levels of valued-living, with scores ranging from 13 to 130.

Grief

The PG-13 is a rater-administered application of the diagnostic criteria for PGD in bereaved individuals and was used to measure post-loss grief. It contains 13 items, with the first 11 assessing the severity of a particular set of symptomatic thoughts, feelings and behaviours (1 = not at all to 5 = several times a day/overwhelmingly) and the final 2 items assessing the duration of symptoms (greater than 6 months for PGD) and whether they are associated with significant functional impairment (yes/no). Higher scores indicate higher levels of grief, with scores ranging from 11 to 55. To meet criteria for PGD, respondents must have a total score of 44 or more on the first 11 items and answer in the positive to the final 2 items.

Anticipatory grief was measured using the PG-12 (Jacobsen et al., 2010; Prigerson and Maciejewski, 2006), which is the pre-loss equivalent of the PG-13. The loss referred to is reframed to the patient’s illness and the item assessing duration of symptoms is removed. A total score of 36 or more was used for fulfilling criteria for syndromal levels of anticipatory grief, which is in line with research using the PG-13 to indicate syndromal levels of PGD symptoms among bereaved carers (Guldin et al., 2012; O’Connor et al., 2010). Higher scores indicate higher levels of anticipatory grief, with scores ranging from 11 to 55.

Psychological distress

The Hospital Anxiety and Depression Scale (HADS; Zigmond and Snaith, 1983) consists of 14 items that provide an overall indication of psychological distress, with two 7-item subscales of anxiety and depression (0 = not at all/very seldom to 3 = most of the time/as much as I ever did). Higher scores indicate higher levels of psychological distress, with scores ranging from 0 to 42.

Data analysis

Data were analysed using International Business Machines Statistical Package for Social Sciences V.22.0 (International Business Machines Corporation (IBM Corp.), 2013). This study was primarily concerned with generating descriptive statistics to be used in evaluating the feasibility of the methods.

Descriptive statistics are provided on the number of eligible carers and the proportion approached and enrolled. Retention rates are reported, with reasons for attrition. The feasibility and acceptability of data collection procedures and outcome measures are determined through questionnaire completion rates, identification of patterns of missing data and the questionnaire acceptability items. Common issues in adherence to the recruitment and follow-up protocol are qualitatively reported. Differences between the control and intervention group on demographic variables were examined using chi-square tests and t-tests in conjunction with nonparametric Mann–Whitney U tests (as a precaution with the small sample size).

The acceptability of the intervention was primarily examined through descriptive statistics of the acceptability items collected at 1-month follow-up and from the amount of protocol completion for both the booklet and telephone call. Mean scores of acceptability items showed statistically significant platykurtic kurtosis. Given the kurtosis and multiple comparisons, the conservative approach of using nonparametric Wilcoxon signed rank tests was chosen for exploratory analyses of the acceptability items. Qualitative information collected in the questionnaire and during the telephone call about barriers and facilitators to protocol completion is also reported.

Descriptive statistics are provided for the outcome measure at each time point. Missing values of data were examined. If more than 80 per cent of values were available within a scale for all three time points, the mean of the available data for the individual participant was used (prorated scores). If less than 80 per cent of values were available within a scale, the participant’s responses for that scale across all three time points were deleted. At baseline, 12 cases were prorated for the VLQ, 4 cases prorated for the PG-12 and a maximum of 2 cases prorated for all other scales. At 1-month follow-up, five cases were prorated for the VLQ. At 6-month follow-up, eight cases were prorated for the VLQ and a maximum of one case prorated for all other scales. Finally, eight cases were deleted for the VLQ across all three time points.

Data were also examined for outliers and casewise diagnostics were used to detect any problematic cases, of which none were found. Mauchly’s tests showed that assumptions of sphericity were met, while Levene’s tests showed assumptions of homoscedasticity were met except for grief at 1-month follow-up. Normality plots and statistical tests of normality were inspected. The AAQ at baseline and 6-month follow-up showed statistically significant positive skewness while the HADS total score at 6-month follow-up showed statistically significant positive skewness and leptokurtic kurtosis. The skewness and kurtosis were slight and improved with transformation. However, analyses conducted comparing transformed and non-transformed data resulted in the same pattern and statistical significance of findings and similar magnitudes. Thus, for ease of interpretation, we report the results from the non-transformed analyses. In addition, analyses were conducted using parametric and nonparametric equivalent tests, which similarly produced no differences in the pattern or substance of the findings. Therefore, the non-transformed data and parametric tests were used for ease of interpretation.

Repeated-measures analysis of variance (ANOVA) over baseline to 1-month follow-up was used to examine preliminary effectiveness of the intervention on the outcomes of acceptance, valued-living, grief and psychological distress. The small sample size limited the scope of our analyses; we were only able to analyse intent-to-treat and not per-protocol or according to PGD risk as outlined in the published protocol nor control for demographic predictors (Davis et al., 2019). As a precaution with the small sample size, we also ran nonparametric tests. Mann–Whitney U tests were used to assess for differences between groups on outcomes at each time point. Friedman tests were used to examine change over all three time points within groups and any statistically significant results were followed up with post hoc Wilcoxon signed rank tests.

For all analyses, p-value of <.05 was considered statistically significant. However, given the small sample size, we were primarily concerned with trends and effect sizes to give an indication of the shape and magnitude of any differences found and to inform sample size calculations for a larger main trial. Effect sizes from the group by time interaction in the repeated-measures ANOVAs were calculated using an online calculator that transformed eta squared into Cohen’s d (Lenhard and Lenhard, 2016). Effect sizes for post hoc Wilcoxon signed rank tests were calculated using the formula $r = Z \div \sqrt{N}$ (Field, 2009). Sample size calculations were also made using an online calculator (Kohn et al., 2014).

Results

Demographics

Carers comprised 45 individuals identified as the primary carers and 10 significant others. Table 1 provides details of participant demographic information. The mean age of participants was 58 years and most were female (73%). Most were born in Australia (75%) and had completed a minimum of a diploma qualification (73%). Approximately half indicated they followed no religion, while nearly half identified as Christian. The majority of carers were spouses/partners (38%) or daughters (27%) and provided daily care (80%). Mean duration of care provision was approximately 5 years. There were no statistically significant differences between groups on demographic variables.

Table 1.

Carer demographics (N = 55).

Variable		ControlN = 20	InterventionN = 35	TotalN = 55
Variable		Mean (SD)	Mean (SD)	Mean (SD)
Age (years)^a		57.31 (6.74)	59.03 (15.56)	58.39 (12.90)
Length of relationship with patient (years)		43.02 (16.47)	41.48 (19.56)	42.04 (18.35)
Length of care provision to patient (years)^a		8.24 (11.51)	3.52 (4.84)	5.19 (8.08)
		N (%)	N (%)	N (%)
Gender	Female	13 (65%)	27 (77%)	40 (73%)
Gender	Male	7 (35%)	8 (23%)	15 (27%)
Relationship status	Married/de Facto	13 (65%)	27 (77%)	40 (73%)
	Divorced/separated	5 (25%)	0 (0%)	5 (9%)
	Never married	1 (5%)	5 (14%)	6 (11%)
	Widowed	1 (5%)	3 (9%)	4 (7%)
Education	Year 12 or lower	7 (35%)	15 (44%)	22 (40%)
	TAFE diploma	5 (25%)	13 (38%)	18 (33%)
	Undergraduate	5 (25%)	4 (12%)	9 (16%)
	Postgraduate	3 (15%)	2 (6%)	5 (9%)
Country of birth	Australia	14 (75%)	26 (74%)	41 (75%)
Country of birth	Other	5 (25%)	9 (26%)	14 (25%)
Religion	No religion	10 (50%)	17 (49%)	27 (49%)
	Christian	10 (50%)	16 (46%)	26 (47%)
	Other	0 (0%)	2 (6%)	2 (4%)
Relationship to patient	Spouse/partner	5 (25%)	17 (49%)	22 (40%)
	Daughter	5 (25%)	10 (29%)	15 (27%)
	Sibling	3 (15%)	2 (6%)	5 (9%)
	Friend	1 (5%)	1 (3%)	2 (4%)
	Other	5 (28%)	4 (11%)	9 (16%)
Frequency of care provision	Daily (5–7 days per week)	13 (72%)	31 (89%)	44 (80%)
	Intermittent (2–4 days per week)	5 (28%)	3 (9%)	8 (15%)
	Occasional (1 or less days per week)	0 (0%)	1 (3%)	1 (2%)

TAFE: technical and further education; SD: standard deviation.

Not all values add to 100 per cent due to missing data.

N = 48.

At baseline, 29 per cent (n = 16) carers were identified as showing syndromal levels of anticipatory grief, while 10 per cent (n = 3) met criterion for PGD at 6-month post-loss follow-up. In the month before baseline completion, 29 per cent (n = 16) had accessed support for their emotional or mental health. In the interim periods between questionnaire completion, 14 per cent (n = 6) of carers indicated they had accessed additional emotional support from a mental health worker at 1-month follow-up and 10 per cent (n = 3) at 6-month post-loss follow-up.

Feasibility of recruitment to the trial and attrition

A Consolidated Standards of Reporting Trials (CONSORT) diagram is provided in Figure 1. Over 28 months, 186 carers (142 carer and significant other units) from 457 (31%) eligible carer and significant other units were approached and invited. Of the 186 invited, 106 (57%) provided verbal consent and were randomised (with pre-randomised recruitment packages), with 53 individuals in each group (45 carer and significant other units in the intervention group and 41 carer and significant other units in the control group).

Figure 1.

CONSORT flow diagram.

Recruitment expectations of 80 carers completing the trial (Davis et al., 2019) were not met within the anticipated timeframes, with only 55 carers providing written consent and completing baseline assessments.

Feasibility of data collection procedures

There was a substantial proportion of missing data from the acceptability items from the 1-month follow-up questionnaire for the intervention group. While 24 of the 26 intervention participants indicated how much they had read of the booklet, only 17 responded to the acceptability items of the booklet, 14 to the section that asks carers to indicate which sections they had read and the extent they found it helpful, and 12 to the acceptability items of the phone call. The participants who responded generally indicated they had read the booklet ‘quite thoroughly’.

As described in the ‘Data analysis’ section, there was also a notable proportion of missing or incomplete data for the VLQ compared to the other preliminary effectiveness measures. Participants seemed to leave domains blank that they did not consider relevant to them at the time. For example, ‘Employment’ is less relevant in this older and more frequently retired sample.

Overall, acceptability of the questionnaire was high. Nearly half disagreed or strongly disagreed that it took too long to complete (n = 25, 46%) while less than a quarter agreed or strongly agreed (n = 10, 18%). Similarly, a small minority agreed or strongly agreed the questionnaire was distressing to complete (n = 5, 9%), while the majority disagreed or strongly disagreed (n = 38, 69%). A quarter (n = 15, 27%) agreed that the questionnaire was helpful, while the majority neither agreed nor disagreed with this item (n = 36, 65%). Finally, most agreed or strongly agreed that they would still complete the questionnaire now knowing what was asked (n = 41, 75%).

Protocol completion

Phone contact was attempted for 52 carers, of which 36 were successful and led to discussion of the booklet. Multiple phone calls were sometimes required before carers were able to discuss the booklet; data from the last phone call involving discussion about the booklet was used and the mean duration of the phone calls was 5.27 minutes (standard deviation (SD) = 3.36; range = .50–14.18 minutes).

At the time of the support telephone call (n = 36), most carers had read the booklet only a little (n = 15) or not at all (n = 11). This suggests that for most carers the phone call came at a time when they had had little opportunity to learn about or implement the skills and thus only afforded an opportunity for the provision of general validation and support. By comparison, of the 24 intervention participants were contacted by telephone at 1-month follow-up (n = 26), 10 had read the booklet ‘quite thoroughly’ and only 1 had not read it at all.

Acceptability of intervention

As stated above, 17 intervention participants provided data on the acceptability of the booklet (8 items) and 12 on the phone call (2 items). Based on ratings given by the 12 participants who provided data for both the booklet and phone call, the overall mean of the acceptability items (10 items; range from 1 = strongly disagree to 5 = strongly agree) indicated that the intervention was generally acceptable (M = 3.62, SD = .44). Carers on average appeared to have found the phone call helpful (M = 3.75, SD = .62) and an essential part of the intervention (M = 3.50, SD = .90).

Data on the 24 individual booklet sections were provided by 14 of the 26 intervention participants, with between 8 and 11 participants (different participants across the items) providing ratings of the helpfulness of each section. The overall mean of the helpfulness ratings (24 items; range from 1 = not at all helpful to 5 = very helpful) indicated an intermediate level of helpfulness (M = 3.22, SD = 1.11). A Wilcoxon signed rank test showed that the mean helpfulness ratings of the combined psychoeducation sections (n = 10; median = 3.74, interquartile range (IQR) = 1.13) was significantly higher than the ratings for the mean of the combined exercises (n = 11; median = 3.29, IQR = 1.86; Z = −2.70, p < .01). To examine this more closely we ran Wilcoxon signed rank tests among helpfulness ratings within the psychoeducation and exercise items. There was little difference overall in helpfulness ratings of the psychoeducation sections. Results showed that only the highest and lowest ranked psychoeducation sections were significantly different (Z = −2.12, p = .03), with the higher rating given to the section on how attempts to control unwanted thought and feelings paradoxically leads to an increase in those thoughts and feelings (n = 8; median = 4.00, IQR = 2.00) and the lower rating given to a section on noticing and separating oneself from unwanted thoughts (n = 7; median = 3.00, IQR = 1.00). There was more variation among ratings of the exercises. The two highest and two lowest rated exercises were significantly different from one another (p < .05). The exercise showing the ineffectiveness of thought suppression following imaginal exposure of cutting a lemon (n = 10; median = 3.50, IQR = 3.50) and that which listed everyday mindfulness practices (n = 9; median = 4.00, IQR = 2.00) were rated the highest in helpfulness. The exercise involving a written statement of unwanted feelings that carers would be willing to sit with in order to do a specified valued behaviour (n = 7; median = 2.00, IQR = 2.00) and another that demonstrated that they can choose their behaviour regardless of what their thoughts state (n = 8; median = 2.50, IQR = 1.75) were rated the lowest.

Although it seemed that carers overall could not definitively say whether the booklet helped them or not, most of the 17 participants who provided data on acceptability agreed or strongly agreed that they would recommend the booklet to others (n = 11) and none disagreed. To better understand this issue, we examined the qualitative data that carers provided. For those carers who did not find the booklet helpful, the reasons were most often some variation of ‘it is not for me’, such as feeling like they did not need the support or that it did not provide what they were specifically after. This perhaps helps explain why most carers said they would recommend the booklet to others but were overall ambivalent about its effect on them – they perceived it was not suited to them personally:

I felt the booklet did not apply to me very much. The telephone call was much better to know that the way I am feeling is normal. (Female, 63 years)

I didn’t read all of the booklet as I was tired and stressed when I first received it. I think it may be helpful to some people though. I think I am coping okay. (Female, 79 years)

For those carers who found the booklet helpful, they most often commented on the skills learned while also acknowledging that engaging with the material could be very challenging to the point that they nearly disengaged:

I found the booklet helped me to get through this very bumpy time and with ‘creep up behind you’ emotions and thoughts. (Female, 63 years)

When I started the booklet I found it extremely challenging. Eventually I found most exercises beneficial and definitely gave me another outlook. … It brought out so many emotions to start with that I almost did not complete it. (Female, 37 years)

Preliminary effectiveness outcomes

Means of the main effectiveness outcomes are presented in Table 2 alongside effect size calculations. There were no statistically significant main or interaction effects from the repeated-measures ANOVAs on the main effectiveness outcomes when conducted on baseline to 1-month follow-up (p > .05). Likewise, null results remained (p > .05) when we conducted the same analyses over all three time points. Despite a smaller sample size across the three time points, we would anticipate these comparisons would provide the greatest differences between groups given the intervention participants potentially persisted with the intervention through to 6-month post-loss follow-up. Nonetheless, the effect sizes (see Table 2) were small for acceptance and grief at both 1-month follow-up and by 6-month post-loss follow-up. While the effect size for psychological distress increased from negligible at 1-month follow-up to medium-sized by 6-month post-loss follow-up, the effect size for valued-living decreased from small at 1-month to negligible by 6-month-post-loss follow-up.

Table 2.

Control and intervention group means on effectiveness outcomes with effect sizes.

Measures	Baseline		1-month follow-up		Effect size^a	6-month post-loss follow-up		Effect size^b
Measures	N	M (95% CI)	N	M (95% CI)	d	N	M (95% CI)	d
Experiential avoidance (AAQ-II)	55	19.60 (16.76, 22.44)	44	19.91 (17.40, 22.42)	.24	29	19.41 (15.68, 23.15)	.28
Control	20	20.59 (16.19, 25.00)	18	21.72 (18.01. 25.43)		9	22.44 (14.53, 30.36)
Intervention	35	19.03 (15.19, 22.87)	26	18.65 (15.14, 22.17)		20	18.05 (13.58. 22.52)
Valued-living (VLQ)	47	89.75 (83.01, 96.43)	36	97.35 (90.67, 104.02)	.22	25	102.04 (95.86, 108.22)	<.01
Control	15	85.32 (73.42, 97.22)	13	93.01 (81.08, 104.94)		7	100.61 (87.84, 113.38)
Intervention	32	91.83 (84.42, 100.24)	23	99.80 (91.26, 108.33)		18	102.60 (94.77, 110.42)
Grief (PG-12, PG-13)^c	55	28.40 (25.78, 31.02)	44	36.24 (31.11. 41.38)	.14^d	29	27.00 (22.59, 31.42)	.26^d
Control	20	28.75 (23.94, 33.56)	18	35.95 (25.80, 46.10)		9	26.44 (14.88, 38.01)
Intervention	35	28.20 (24.94, 31.45)	26	36.48 (30.61, 42.35)		20	27.26 (22.50, 32.01)
Psychological distress (HADS)	55	15.63 (13.25, 18.02)	44	14.39 (11.81, 16.96)	<.01	29	10.79 (7.69, 13.90)	.52
Control	20	17.45 (12.73, 22.17)	18	14.50 (10.26, 18.75)		9	14.22 (6.41, 22.03)
Intervention	35	14.60 (11.86, 17.34)	26	14.31 (10.84, 17.78)		20	9.25 (6.05, 12.45)

CI: confidence interval; AAQ-II: Acceptance and Action Questionnaire–II; VLQ: Valued Living Questionnaire; PG: prolonged grief; HADS: Hospital Anxiety and Depression Scale.

Ranges on mean scores are as follows: AAQ-II (reversed) is 7–49, VLQ is 13–130, PG-12 and PG-13 is 11–55 and HADS is 0–42.

Effect size for baseline to 1-month follow-up comparison.

Effect size for comparison across all three time points.

PG-12 mean scores are provided for baseline and PG-13 mean scores for both follow-ups.

Sample sizes vary due to a proportion of carers not bereaved at 1-month follow-up and thus not completing the PG-13. Grief for 1-month follow-up effect size: Control = 12, Intervention = 15. Grief for 6-month post-loss follow-up effect size: Control = 5, Intervention = 11.

Nonparametric Mann–Whitney tests showed no statistically significant differences between groups at any time point (p > .05). Friedman’s test showed that there was a significant time effect for psychological distress in the intervention group ( $χ_{2}^{2} = 13.63$ , p < .01), but not the control group (p > .05). Post hoc Wilcoxon signed rank tests on the intervention group data revealed a significant decrease in psychological distress at 6-month follow-up (median = 7.00, IQR = 10.75) compared to both baseline (median = 12.00, IQR = 14.00; Z = −3.39, p < .01) and 1-month post-allocation (median = 12.50, IQR = 11.50; Z = −3.39, p < .01). The effect sizes for these changes were large, both between baseline and 6-month post-loss (r = .76) and between 1-month and 6-month follow-up (r = .71).

To estimate sample size calculation for a main trial, acceptance was chosen a priori as the primary outcome given it is the key targeted mechanism of therapeutic change. Using the repeated-measures ANOVA effect size results at 1-month follow-up, it was estimated that there would need to be a total sample size of 545 participants (n = 272 control, n = 273 intervention). Estimates for secondary clinical outcomes of grief and psychological distress were also explored. For grief, total sample size estimates ranged from 464 participants (6-month post-loss follow-up effect size) to 1602 participants (1-month follow-up effect size). A calculation can only be made based on the 6-month post-loss follow-up effect size for psychological distress, resulting in an estimated total sample size of 116 participants.

Discussion

We evaluated the feasibility and preliminary effectiveness of an ACT self-help intervention for carers of patients in palliative care. The intervention seemed generally feasible and acceptable to carers, with a sizeable proportion engaging with the booklet and overall indicating that it was understandable. Preliminary effectiveness analyses showed tentative trends for acceptance, valued-living, grief and psychological distress in helpful directions. Effect size calculations from the repeated-measures ANOVA showed that no or small change seemed to take place in outcomes from baseline to 1-month follow-up or across all time points, while a medium change was observed for psychological distress by 6-month post-loss follow-up. Nonparametric effect size calculations conducted on psychological distress in the intervention group were large for comparisons between 6-month post-loss follow-up and the preceding time points. In line with qualitative feedback, the predominant pattern of larger effects occurring by 6-month post-loss might suggest that carers needed more time to engage with the content and exercises of the booklet. The period leading up to 1-month was potentially too challenging to do so given many carers became bereaved within that time. That the intervention seemed to be more effective on psychological distress than grief suggests that the content targets general psychological distress and might need to be modified to target proposed grief specific maintaining mechanisms to a greater extent (e.g. see Boelen et al., 2006).

It is positive that acceptance and, to some extent, valued-living showed at least a small change given they are the proposed mechanisms of therapeutic change and acceptance is the primary outcome for the future main trial. However, the limited extent of change indicates that modifications must be made to the intervention and its implementation in order to achieve the intended effects on outcomes. This is reinforced by the relatively large sample size estimate for a larger main effectiveness trial.

There is a recognised difficulty of recruiting samples within palliative care (e.g. Kars et al., 2015; Schildmann and Higginson, 2011). Recruitment expectations were not met in this study, and three key influential factors impacting recruitment figures were: patient death before carers completed the baseline questionnaire, thereby making them no longer eligible; limited availability of recruiters resulting in an inability to approach all eligible carers; and patient reluctance to nominate their carers and significant others, and likewise carer reluctance in nominating significant others. The primary reason given by patients and carers for the reticence in referring others was to minimise the perceived load of information and tasks on their loved ones. Patients and carers were more comfortable with being given a brochure to pass on, but this was not an effective recruitment strategy (did not result in potential participants contacting the research team). Given that significant others were overall less likely to be present at the sites than the carers, the small number of nominations mostly resulted in proportionately smaller numbers of significant others participating.

Nonetheless, overall attrition throughout the study was comparable to similar studies of carers of patients in palliative care (e.g. Hudson et al., 2005, 2013; Northouse et al., 2007). There was more attrition in the control group at baseline before providing written consent, which appeared mostly attributable to carers in the control group being disproportionately affected by patient death before completing the baseline questionnaire. It is also possible that more carers in the intervention group completed baseline because they needed to do so before opening an envelope to read the booklet and they received the support phone call that acted as a further reminder. Following baseline, the factor most impacting attrition appears to be carer failure to respond to follow-up. Anecdotally, those carers who we were able to contact that never returned post-death questionnaires indicated that there was too much going on in terms of consequent life changes, such as sorting through the deceased’s belongings or moving, and other practical and emotional demands.

Overall, this feasibility RCT has provided direction for necessary improvements to the intervention and its implementation prior to a larger main trial. These improvements as well as strengths and limitations of the current trial are discussed in more detail below.

Strengths and limitations

Limitations

There are a number of limitations to this study that require consideration. The generalisability and reliability of the results are limited by the small sample size. To address this issue, additional recruitment sites from a range of different areas, from both community and inpatient populations, and across different stages in the patient illness trajectory would be beneficial. There was also potential bias introduced through lack of blinding of recruiting research and clinical staff to group allocation. This was compounded by E.L.D. being involved across all aspects of the research, including recruitment, follow-up and provision of telephone support. One option to ameliorate this issue in the future is to use an equally weighted blank notebook in the control recruitment packages and isolating the roles of recruitment and follow-up to the study and delivery of the intervention. The acceptability results about the booklet were also potentially impacted by courtesy bias, thereby providing an inflated estimate of protocol completion and helpfulness. Isolating the research administration and intervention delivery roles, as previously mentioned, might provide some safeguard by creating a sense of independence of the evaluation of the intervention from those who delivered it. Finally, it is important to note that because most carers were not demonstrating clinical levels of grief or psychological distress, it is possible that the responses from a clinical carer population may differ.

Strengths

Despite these limitations, there are also a number of strengths of the current feasibility RCT. As noted earlier, there is a lack of consistent evidence for psychotherapeutic intervention for both carers of patients in palliative care and for grief. The novel application of ACT and self-help to carers and of ACT and self-help to grief in this study represents progress in the development of rigorously evaluated interventions in these domains. Also, the inclusion of a subclinical or mildly symptomatic population has the advantage of improving the generalisability or external validity of the intervention for the majority of the carer population. Finally, the feasibility focus allowed us to identify and more closely examine both strengths and specific usability and implementation issues for future improvement and research.

Implications for future research

This feasibility trial has been valuable in highlighting a number of changes to improve the viability of a larger effectiveness trial of this self-help intervention. In terms of recruitment and attrition, there is a strong need for greater availability of recruiters and a greater number of sites in order to improve recruitment numbers.

It would also likely be of benefit to recruit carers earlier in the patient’s illness trajectory so that there is a greater chance of the carer completing the questionnaire before patient death. It is possible that this would also give carers greater opportunity to engage with the material in a more optimal manner given it would be at a less critical time. Another potential change to increase engagement with the material is to increase the frequency of the telephone support. However, to maintain feasibility in implementing the intervention, the frequency might be based on a plan developed collaboratively with the carer according to their needs and preferences. Not only might increased telephone support help with accountability of using the booklet, but also provide encouragement when the material is emotionally confronting or challenging to the carer. We have also considered screening anticipatory grief to target those carers who are more ‘clinically indicated’ as in need of support and therefore more likely to benefit from the intervention. Given the potential impact of this change on reducing the pool of eligible carers, stratifying random allocation based on levels of anticipatory grief might be a more viable solution and allow us to compare the clinical effect of the intervention across the full presentation without potential selection bias. A trial of the screening process would be a valuable project to undertake prior to implementing a larger effectiveness trial.

Finally, we will carefully consider making indicated changes to the booklet, such as specifically targeting grief maintaining mechanisms and working out ways to improve engagement and helpfulness of the booklet. The acceptability data showed that the booklet would perhaps benefit from a reduction in content. The acceptability data were also valuable in showing that the psychoeducation content tended to be rated higher in helpfulness than the exercises and thus revealed an important focus for our review. The highest rated parts of the booklet related to understanding the unhelpfulness of attempts to control, avoid or get rid of unwanted thoughts and feelings. This might have been experienced as a novel idea for how to relate to unwanted thoughts and feelings and perhaps even liberating. Likewise, the everyday mindfulness practices were rated highly. Meanwhile, the preliminary effectiveness data were valuable in showing us that acceptance, valued-living, grief and psychological distress showed improvements in the intervention group even compared to treatment as usual. Given the need for some revisions to the intervention, prior to a larger effectiveness trial, it may be desirable for a small pilot (case studies) study to test these minor revisions, trial more carer specific outcomes measures (e.g. carer VLQ; Romero-Moreno et al., 2016) and minor revisions to recruitment processes.

Conclusion

Overall, the delivery of an ACT self-help intervention to carers of patients in palliative care seemed to be feasible and well received. The results will assist us in improving the intervention and administration of the trial in preparation for a larger effectiveness trial. As the demand for palliative care rises, equally does the relevance and potential of self-help interventions to offer effective treatment with the flexibility that the context requires (Davidson et al., 2014).

Footnotes

Acknowledgements

The authors gratefully acknowledge the carers and staff of Port Kembla Hospital and the Wollongong Hospital who participated in this study. They also thank Dr Lis Lane for her review of the self-help booklet and the Improving Palliative Care through Clinical Trials (ImPaCCT) group for their review of the protocol. They are immensely grateful to the Illawarra Cancer Carers Inc. for their review and funding of the self-help booklet.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been conducted with the support of the Australian Government Research Training Program Scholarship. The Illawarra Cancer Carers Inc. funded the design and printing costs of the intervention booklet and CD. They have no involvement in the design of the study and collection, analysis and interpretation of data and in writing the manuscript.

References

Arain

Campbell

Cooper

, et al. (2010) What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology 10: 67.

A-Tjak

JGL

Davis

Morina

, et al. (2015) A meta-analysis of the efficacy of acceptance and commitment therapy for clinically relevant mental and physical health problems. Psychotherapy and Psychosomatics 84(1): 30–36.

Boelen

van den Bout

van den Hout

(2010) A prospective examination of catastrophic misinterpretations and experiential avoidance in emotional distress following loss. Journal of Nervous and Mental Disease 198(4): 252–257.

Boelen

van den Hout

van den Bout

(2006) A cognitive-behavioral conceptualization of complicated grief. Clinical Psychology: Science and Practice 13(2): 109–128.

Bond

Hayes

Baer

, et al. (2011) Preliminary psychometric properties of the Acceptance and Action Questionnaire–II: A revised measure of psychological inflexibility and experiential avoidance. Behavior Therapy 42(4): 676–688.

Browne

(1995) On the use of a pilot sample for sample-size determination. Statistics in Medicine 14(17): 1933–1940.

Burke

Clark

Ali

, et al. (2015) Risk factors for anticipatory grief in family members of terminally ill veterans receiving palliative care services. Journal of Social Work in End-of-Life and Palliative Care 11(3–4): 244–266.

Burns

Abernethy

Dal Grande

, et al. (2013) Uncovering an invisible network of direct caregivers at the end of life: A population study. Palliative Medicine 27(7): 608–615.

Candy

Jones

Drake

, et al. (2011) Interventions for supporting informal caregivers of patients in the terminal phase of a disease. Cochrane Database of Systematic Reviews 15(6): CD007617.

10.

Cavanagh

Strauss

Forder

, et al. (2014) Can mindfulness and acceptance be learnt by self-help? A systematic review and meta-analysis of mindfulness and acceptance-based self-help interventions. Clinical Psychology Review 34(2): 118–129.

11.

Clark

Brethwaite

Gnesdiloff

(2011) Providing support at time of death from cancer: Results of a 5-year post-bereavement group study. Journal of Social Work in End-of-Life and Palliative Care 7(2–3): 195–215.

12.

Currier

Neimeyer

Berman

(2008) The effectiveness of psychotherapeutic interventions for bereaved persons: A comprehensive quantitative review. Psychological Bulletin 134(5): 648–661.

13.

Davidson

Yuen

Felton

, et al. (2014) Feasibility assessment of a brief, web-based behavioral activation intervention for adolescents with depressed mood. International Journal of Psychiatry in Medicine 48(1): 69–82.

14.

Davis

Deane

Lyons

(2015) Acceptance and valued living as critical appraisal and coping strengths for caregivers dealing with terminal illness and bereavement. Palliative & Supportive Care 13(2): 359–368.

15.

Davis

Deane

Lyons

(2019) An acceptance and commitment therapy self-help intervention for carers of patients in palliative care: Protocol of a feasibility randomised controlled trial. Journal of Health Psychology 24(5): 685–704. DOI: 10.1177/1359105316679724.

16.

Davis

Deane

Lyons

(2016) Prediction of individual differences in adjustment to loss: Acceptance and valued-living as critical appraisal and coping strengths. Death Studies 40(4): 211–222.

17.

Delalibera

Presa

Barbosa

, et al. (2015) Burden of caregiving and its repercussions on caregivers of end-of-life patients: A systematic review of the literature. Ciencia & Saude Coletiva 20(9): 2731–2747.

18.

Donker

Blankers

Hedman

, et al. (2015) Economic evaluations of internet interventions for mental health: A systematic review. Psychological Medicine 45(16): 3357–3376.

19.

Field

(2009) Non-parametric tests. In: Field

(ed.) Discovering Statistics Using SPSS (3rd edn). London: Sage, pp. 539–583.

20.

Fledderus

Bohlmeijer

Pieterse

, et al. (2012) Acceptance and commitment therapy as guided self-help for psychological distress and positive mental health: A randomized controlled trial. Psychological Medicine 42(3): 485–495.

21.

Forsyth

(2011) Facing the unmet burden of human suffering: Using ACT in a self-help context. In: 5th annual Australian and New Zealand conference for acceptance and commitment therapy. 30 September – 2 October. Brisbane, QLD, Australia.

22.

Gauthier

Gagliese

(2012) Bereavement interventions, end-of-life cancer care and spousal well-being: A systematic review. Clinical Psychology: Science and Practice 19(1): 72–92.

23.

Guldin

M-B

Vedsted

Zachariae

, et al. (2012) Complicated grief and need for professional support in family caregivers of cancer patients in palliative care: A longitudinal cohort study. Supportive Care in Cancer 20(8): 1679–1685.

24.

Harding

Higginson

(2003) What is the best way to help caregivers in cancer and palliative care? A systematic literature review of interventions and their effectiveness. Palliative Medicine 17(1): 63–74.

25.

Harding

List

Epiphaniou

, et al. (2012) How can informal caregivers in cancer and palliative care be supported? An updated systematic literature review of interventions and their effectiveness. Palliative Medicine 26(1): 7–22.

26.

Hayes

Pistorello

Levin

(2012) Acceptance and commitment therapy as a unified model of behavior change. Counseling Psychologist 40(7): 976–1002.

27.

Hayes

Strosahl

Wilson

(1999) Acceptance and Commitment Therapy: An Experiential Approach to Behaviour Change. New York: Guildford Press.

28.

Hayes

Strosahl

Wilson

(2011) Acceptance and Commitment Therapy: The Process and Practice of Mindful Change. New York: Guildford Press.

29.

Hudson

Trauer

Kelly

, et al. (2013) Reducing the psychological distress of family caregivers of home-based palliative care patients: Short-term effects from a randomised controlled trial. Psycho-oncology 22(9): 1987–1993.

30.

Hudson

Aranda

Hayman-White

(2005) A psycho-educational intervention for family caregivers of patients receiving palliative care: A randomized controlled trial. Journal of Pain and Symptom Management 30(4): 329–341.

31.

International Business Machines Corporation (IBM Corp.) (2013) IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp.

32.

Jacobsen

Zhang

Block

, et al. (2010) Distinguishing symptoms of grief and depression in a cohort of advanced cancer patients. Death Studies 34(3): 257–273.

33.

Johnston

Foster

Shennan

, et al. (2010) The effectiveness of an acceptance and commitment therapy self-help intervention for chronic pain. Clinical Journal of Pain 26(5): 393–402.

34.

Kapari

Addington-Hall

Hotopf

(2010) Risk factors for common mental disorder in caregiving and bereavement. Journal of Pain and Symptom Management 40(6): 844–856.

35.

Kars

van Thiel

van der Graaf

, et al. (2015) A systematic review of reasons for gatekeeping in palliative care research. Palliative Medicine 30: 533–348.

36.

Kashdan

Rottenberg

(2010) Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review 30(7): 865–878.

37.

Jarrett

(2014) Sample size calculators. Available at: http://www.sample-size.net/sample-size-means/

38.

Lappalainen

Granlund

Siltanen

, et al. (2014) ACT Internet-based vs face-to-face? A randomized controlled trial of two ways to deliver acceptance and commitment therapy for depressive symptoms: An 18-month follow-up. Behaviour Research and Therapy 61: 43–54.

39.

LeMay

Wilson

(2008) Treatment of existential distress in life threatening illness: A review of manualized interventions. Clinical Psychology Review 28(3): 472–493.

40.

Lenhard

(2016) Calculation of effect sizes. Available at: https://www.psychometrica.de/effect_size.html

41.

Lewis

Pearce

Bisson

(2012) Efficacy, cost-effectiveness and acceptability of self-help interventions for anxiety disorders: Systematic review. British Journal of Psychiatry 200(1): 15–21.

42.

Maercker

Brewin

Bryant

, et al. (2013) Proposals for mental disorders specifically associated with stress in the International Classification of Diseases-11. The Lancet 381(9878): 1683–1685.

43.

Musiat

Tarrier

(2014) Collateral outcomes in e-mental health: A systematic review of the evidence for added benefits of computerized cognitive behavior therapy interventions for mental health. Psychological Medicine 44(15): 3137–3150.

44.

Muto

Hayes

Jeffcoat

(2011) The effectiveness of acceptance and commitment therapy bibliotherapy for enhancing the psychological health of Japanese college students living abroad. Behavior Therapy 42(2): 323–335.

45.

Nanni

Biancosino

Grassi

(2014) Pre-loss symptoms related to risk of complicated grief in caregivers of terminally ill cancer patients. Journal of Affective Disorders 160: 87–91.

46.

Northouse

Mood

Schafenacker

, et al. (2007) Randomized clinical trial of a family intervention for prostate cancer patients and their spouses. Cancer 110(12): 2809–2818.

47.

O’Connor

Lasgaard

Shevlin

, et al. (2010) A confirmatory factor analysis of combined models of the Harvard Trauma Questionnaire and the Inventory of Complicated Grief-Revised: Are we measuring complicated grief or posttraumatic stress? Journal of Anxiety Disorders 24(7): 672–679.

48.

Peacock

Forbes

(2003) Interventions for caregivers of persons with dementia: A systematic review. The Canadian Journal of Nursing Research 35(4): 88–107.

49.

Pots

WTM

Fledderus

Meulenbeek

PAM

, et al. (2016) Acceptance and commitment therapy as a web-based intervention for depressive symptoms: Randomised controlled trial. The British Journal of Psychiatry 208(1): 69–77.

50.

Prigerson

Maciejewski

(2006) Prolonged Grief PG-12. Boston, MA: Dana-Farber Cancer Institute.

51.

Prigerson

Horowitz

Jacobs

, et al. (2009) Prolonged grief disorder: Psychometric validation of criteria proposed for DSM-V and ICD-11. PLoS Medicine 6: e1000121.

52.

Rasanen

Lappalainen

Muotka

, et al. (2016) An online guided ACT intervention for enhancing the psychological wellbeing of university students: A randomized controlled clinical trial. Behaviour Research and Therapy 78: 30–42.

53.

Romero-Moreno

Gallego-Alberto

Marquez-Gonzalez

, et al. (2016) Psychometric properties of the Valued Living Questionnaire adapted to dementia caregiving. Aging & Mental Health 2: 1–8.

54.

Sauer-Zavala

Gutner

Farchione

, et al. (2017) Current definitions of ‘transdiagnostic’ in treatment development: A search for consensus. Behavior Therapy 48(1): 128–138.

55.

Schildmann

Higginson

(2011) Evaluating psycho-educational interventions for informal carers of patients receiving cancer care or palliative care: Strengths and limitations of different study designs. Palliative Medicine 25(4): 345–356.

56.

Schut

Stroebe

(2010) Effects of support, counselling and therapy before and after the loss: Can we really help bereaved people? Psychologica Belgica 50(1–2): 89–102.

57.

Solomon

Proudfoot

Clarke

, et al. (2015) e-CBT (myCompass), antidepressant medication and face-to-face psychological treatment for depression in Australia: A cost-effectiveness comparison. Journal of Medical Internet Research 17(11): e255.

58.

Steinhauser

Clipp

Hays

, et al. (2006) Identifying, recruiting and retaining seriously-ill patients and their caregivers in longitudinal research. Palliative Medicine 20(8): 745–754.

59.

Thabane

Chu

, et al. (2010) A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology 10: 1.

60.

Thomas

Hudson

Trauer

, et al. (2014) Risk factors for developing prolonged grief during bereavement in family carers of cancer patients in palliative care: A longitudinal study. Journal of Pain and Symptom Management 47(3): 531–541.

61.

Twohig

Vilardaga

JCP

Levin

, et al. (2015) Changes in psychological flexibility during acceptance and commitment therapy for obsessive compulsive disorder. Journal of Contextual Behavioral Science 4(3): 196–202.

62.

Waller

Turon

Mansfield

, et al. (2016) Assisting the bereaved: A systematic review of the evidence for grief counselling. Palliative Medicine 30(2): 132–148.

63.

Weissflog

Mehnert

(2015) Family focused grief therapy: A suitable model for the palliative care of cancer patients and their families? Psychotherapie Psychosomatik Medizinische Psychologie 65(11): 434–438.

64.

Wilson

Sandoz

Kitchens

(2010) The Valued Living Questionnaire: Defining and measuring valued action within a behavioral framework. The Psychological Record 60: 249–272.

65.

Zigmond

Snaith

(1983) The Hospital and Anxiety Depression Scale. Acta Psychiatrica Scandinavica 67(6): 361–370.