Web-Based Acceptance and Commitment Therapy for Depressive Symptoms With Minimal Support

Abstract

Low-intensity interventions for people suffering from depressive symptoms are highly desirable. The aim of the present study was to investigate the outcomes of a web-based acceptance and commitment therapy (ACT)–based intervention without face-to-face contact for people suffering from depressive symptoms. Participants (N = 39) with depressive symptoms were randomly assigned to an Internet-delivered acceptance and commitment therapy (iACT) intervention or a waiting list control condition (WLC). Participants were evaluated with standardized self-reporting measures (Beck Depression Inventory [BDI-II], Symptom Checklist–90 [SCL-90], Acceptance and Action Questionnaire [AAQ-2], Five Facet Mindfulness Questionnaire [FFMQ], Automatic Thoughts Questionnaire [ATQ], and White Bear Suppression Inventory [WBSI]) at pre- and post-measurement. Long-term effects in the iACT group were examined using a 12-month follow-up. The iACT program comprised home assignments, online feedback given by master’s-level students of psychology over a 7-week intervention period, and automated email-based reminders. Significant effects were observed in favor of the iACT group on depression symptomatology (between effect sizes [ESs] at post-treatment, iACT/WLC, g = .83), psychological and physiological symptoms (g = .60), psychological flexibility (g = .67), mindfulness skills (g = .53), and frequency of automatic thoughts (g = .57) as well as thought suppression (g = .53). The treatment effects in the iACT group were maintained over the 12-month follow-up period (within-iACT ES: BDI-II, g = 1.33; SCL-90, g = 1.04; ATQF/B [Frequency/Believability], FFMQ, WBSI, AAQ-II, g = .74-1.08). The iACT participants stated that they would be happy to recommend the same intervention to others with depressive symptoms. We conclude that an ACT-based guided Internet-delivered treatment with minimal contact can be effective for people with depressive symptoms.

Keywords

depressive symptoms acceptance and commitment therapy Internet treatment minimal support reminders

Introduction

Depression is a common health condition that affects nearly 350 million people around the world (World Health Organization [WHO], 2008). It is not only highly prevalent but is also associated with poor life quality, higher mortality rate, increased need for health services, and high economic costs (Alonso et al., 2004; Kessler et al., 1994). In particular, major depression incurs high costs for society, largely because it generates substantial indirect costs (von Knorring, Åkerblad, Bengtsson, Carlsson, & Ekselius, 2006). Depressive disorders are projected to be one of the three leading causes of disease burden by the year 2030 (Mathers & Loncar, 2006). Therefore, effective treatments are important to prevent progression of the disorder and decrease the suffering of patients (Ekman, Granström, Omérov, Jacob, & Landén, 2013; Kleine-Budde et al., 2013).

Low-intensity (LI) treatments are one way to increase access to psychological treatment for people suffering from mild to moderate psychological disorders, thereby enabling high intensity cognitive behavior therapy (CBT) to be reserved for individuals with more severe disorders (Bennet-Levy, Richards, & Farrand, 2010). The primary purpose of LI interventions is to increase access to evidence-based psychological treatments to enhance mental health and well-being (Bennet-Levy et al., 2010). LI treatments have several features in common: First, they use fewer resources in terms of therapist time when compared with more conventional psychological treatment. Second, they are delivered and supported by mental health workers without formal professional health qualifications. Third, most of them are based on the principles of CBT, with the aim of teaching specific skills, along with an emphasis on the value of between-session homework, and fourth, LI interventions are delivered in accessible ways and in various forms, including face-to-face, group, email, telephone, and Internet-based treatments (Bennet-Levy et al., 2010).

Delivery of effective early LI interventions via the Internet can both increase their availability and facilitate treatment of high prevalence disorders, such as depression. The literature provides strong evidence for Internet-delivered CBT-based interventions for the treatment of a wide range of psychological disorders, including depression (cf. Johansson & Andersson, 2012; Richards & Richardson, 2012). Similarly, self-help and minimal contact therapies have demonstrated their effectiveness in the treatment of anxiety and subthreshold mood disorders (Cuijpers, Smit, & van Straten, 2007; Newman, Szkodny, Llera, & Przeworski, 2011). Most of these interventions are based on CBT; however, early interventions in the field of acceptance and commitment therapy (ACT) for people at risk of depression have also recently been developed (Bohlmeijer, Fledderus, Rokx, & Pieterse, 2011; Fledderus, Bohlmeijer, Pieterse, & Schreurs, 2012). Despite coming under the umbrella of CBT approaches, and despite sharing several common features, ACT differs in certain aspects from traditional cognitive-behavioral therapy (Zettle, Rains, & Hayes, 2011). For example, ACT emphasizes the function of thoughts, feelings, memories, and sensations that are feared or avoided, and provides a large variety of exercises aimed at changing the effects of these behaviors. These experiential exercises, in particular in the form of audio and video recordings, are well suited for use in web-based interventions. ACT uses mindfulness and acceptance strategies to teach awareness and focus on the present moment, and to develop an attitude of acceptance toward those thoughts and feelings. Another important feature of ACT is a strong emphasis on clarifying personal values and committing to value-based actions to help create a meaningful life (Twohig & Hayes, 2008).

Several studies and meta-analyses have found ACT (Hayes, Strosahl, & Wilson, 1999) to be or likely to be an effective form of treatment for a wide variety of psychological disorders (Hayes, Luoma, Bond, Masuda, & Lillis, 2006; Ruiz 2010, 2012). However, although the number of studies investigating the effects of ACT has increased over the last few years, ACT studies have some methodological limitations (Öst, 2008, 2014) that need to be recognized when discussing the effectiveness of ACT.

There is growing evidence that ACT is effective in reducing depressive symptoms, although further studies are called for (Bohlmeijer et al., 2011; Forman, Herbert, Moitra, Yeomans, & Geller, 2007; Kohtala, Lappalainen, Savonen, & Timo, Tolvanen, 2013; R. Lappalainen et al., 2007; Powers, Zum Vörde Sive Vörding, & Emmelkamp, 2009; Zettle & Rains, 1989).

Studies investigating Internet-delivered acceptance and commitment therapy (iACT) treatments are relatively scarce (Bricker, Wyszynski, Comstock, & Heffner, 2013; Buhrman et al., 2013; Heffner, Wyszynski, Comstock, Mercer, & Bricker, 2013; Hesser et al., 2012). In our previous study (P. Lappalainen et al., 2014), we demonstrated that guided iACT treatment was as effective as a face-to-face ACT intervention in reducing depressive symptoms. The iACT treatment for depression showed equivalent, and in some cases better, post-intervention effects than the face-to-face ACT treatment.

Internet-delivered interventions, in some cases also known as behavior change support systems (Oinas-Kukkonen, 2013), can vary significantly in terms of usage and the extent of the support required (de Graaf et al., 2009). The majority of studies indicate that the effects of Internet-delivered treatment are larger when therapists provide some form of support (Andersson, Carlbring, Ljótsson, & Hedman, 2013; Richards & Richardson, 2012). The between-group effect sizes (ESs) have varied from d = 0.21 to 0.76, depending on the amount of support, indicating that more support leads to greater effects (Johansson & Andersson, 2012). An alternative way of providing support is by issuing periodic prompts to remind and encourage people to adopt healthy behaviors. Periodic prompts can take the form of text messages, email-based reminders, or concise feedback delivered to the participants several times during the intervention (Fry & Neff, 2009; Oinas-Kukkonen & Harjumaa, 2009). According to a review, the use of periodic prompts can be effective in interventions targeting behavior change (Fry & Neff, 2009). In addition, the effectiveness of such prompts can be improved if they are supplemented with personalized content. The timing and content of prompts could also add to the overall effectiveness of the intervention (Schneider, de Vries, Candel, van de Kar, & van Osch, 2013). In future interventions, periodic prompts could lead to more frequent use and increased effectiveness by imitating the characteristics of guided treatments (Andersson et al., 2013).

In our previous study, which included iACT treatment along with two face-to-face sessions, we observed that the intervention was effective in reducing depressive symptoms (P. Lappalainen et al., 2014). In the present study, we aimed to investigate whether a guided iACT-based intervention augmented with email-based reminders for individuals with self-reported depressive symptomatology would be effective without face-to-face contact. Thus, the aim was to replicate our earlier iACT intervention (P. Lappalainen et al., 2014) with the exception that no face-to-face contact was provided, and email-based reminders were added to the intervention. The web program, Good Life Compass, was the same as that used in our previous study, with, however, the difference that the program had been enhanced with videos highlighting the ACT-related processes. Therefore, the present study was designed to explore whether a guided iACT intervention combined with telephone contact at the beginning of the intervention, brief weekly support through the secured Internet platform, and weekly iACT-tailored reminders would be effective in alleviating depressive symptoms.

We compared an online ACT-based intervention group (iACT) with a waiting list control group (WLC) of participants with depressive symptoms, and followed the iACT group over a period of 12 months. The aim of this study was to investigate the effectiveness of the iACT-based treatment augmented with email-based reminders for depressive symptoms without any face-to-face contact. We hypothesized that the ACT-based intervention would show positive effects on the symptoms of depression (Beck Depression Inventory [BDI-II], Symptom Checklist–90 [SCL-90]) and in the ACT-related process measures of psychological flexibility (Acceptance and Action Questionnaire [AAQ-II]), mindfulness (Five Facet Mindfulness Questionnaire [FFMQ]), thought suppression (White Bear Suppression Inventory [WBSI]), and depressive automatic thoughts (Automatic Thoughts Questionnaire [ATQ]).

Method

Recruitment

The study was an outcome of collaboration between the Department of Psychology, University of Jyväskylä, and the Department of Information Processing Science, University of Oulu, Finland. Recruitment began in September 2012 and took place through advertisements in local newspapers in two municipalities located at a distance of about 350 km (217 miles) from each other. In the advertisements, it was stated that the university clinic was seeking participants who were currently experiencing depressive symptoms. In response to the advertisements, 42 potential participants contacted the university clinic at the University of Jyväskylä through email and/or telephone between the dates given. They were initially screened over the telephone using structured interviews.

Procedure

The main inclusion criterion was the presence of self-reported depressive symptoms, which was assessed during a structured interview over the telephone using one question (“Do you feel depressed at the moment?”) in addition to questions elaborated on the basis of the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR; American Psychiatric Association [APA], 2000) criteria for major depressive episode. To be included in the study, the participants had to report depressed mood, or loss of interest or pleasure, and fulfill at least five of the DSM-IV-TR criteria for major depressive episode. In addition, it was required that a stable medication regimen should not be changed during the intervention period. The remaining inclusion criteria were no parallel treatment for depression or other psychological problems, at least 18 years of age, possession of an email account and access to the Internet, and a telephone at home.

On the basis of the pre-intervention interview, 2 participants did not meet the inclusion criteria, and thus were excluded from the study. One participant reported having parallel treatment for depression and the other did not have major depressive episode according to the DSM-IV-TR criteria. A further participant dropped out for unknown reasons before randomization. Thus, a total of 39 participants were randomly assigned to one of two conditions: iACT (n = 19) and a wait-list control group (n = 20). We used the stratified randomization method to achieve a balance in baseline characteristics between the two groups. After all the participants had been identified and assigned into blocks by sex and amount of self-reported depressive symptoms identified during the telephone interviews, a simple randomization procedure was performed. A person unconnected with the research group carried out the randomization. Participants were randomized after having given their oral consent over the telephone. A written consent was obtained after the randomization. Those in the iACT group (n = 19) were randomly divided among student coaches (n = 11), using simple randomization. During the telephone interview, the purpose and procedure of the study were explained to the prospective participants and any questions they had about the study answered. In addition, they were told that they would receive information about the study by regular mail, including a written informed consent form. The study followed the Declaration of Helsinki and was approved by the Ethics Committee of the Central Finland Health Care District (Diary No. 15U/2012).

All participants were assessed at two points. For the pre-measurement, they received the measurement packages, including the informed consent form by regular mail. After they had read the consent document and agreed to participate in the study, they signed the informed consent form and returned it via regular mail together with the completed measurement package. The participants in the intervention group (iACT) were granted access to the web program, Good Life Compass, after they had posted back their respective assessment packages. After having completed the 6-week program, the participants were measured for the second time (post-measurement). At the post-measurement, data were available for 38 participants (dropout rate 2.6%, n = 1). The participant who dropped out was in the treatment group and discontinued after 3 weeks owing to hospitalization (reasons unknown). The control group was granted access to the web program after they had returned both the pre- and post-measurements (the latter after the 7-week waiting list period and after the post-measurements had been received from the iACT group). The follow-up measurement for the intervention group was conducted 12 months post-treatment in December 2013 (n = 12). Despite offering the participants an incentive (three Acer Tablets for three lucky draw winners), we were not able to collect follow-up measurements from 6 participants (reasons unknown). Figure 1 shows the flow of participants.

Figure 1.

Participant flow diagram.

Basic demographics were collected by questionnaire at the beginning of the study in both groups. No significant differences in demographics were observed between the groups. Most of the participants were females (n = 28, 71.8%). Mean age in the intervention group was 50.32 (SD = 12.54) compared with 53.40 (SD = 13.35) in the control group. Both groups were similar in education, employment, medication, and earlier treatment. According to the BDI-II, the majority of the participants (n = 17) in both groups (iACT 89.5%; WLC 85%) had at least mild depression (score of 14 or more on BDI-II scale). Duration of self-reported depressive symptoms varied from 3 weeks to 55 years (iACT, M = 6.08 years, SD = 5.38; WLC, M = 10.03 years, SD = 14.68). A total of 20.5% of all participants were on antidepressant medication. Eleven participants in the iACT group (57.9%) and 12 (60%) in the control group had previously been treated for depression. Participants did not have any prior experience with either the CBT- or ACT-based treatments. Table 1 shows the participants’ characteristics.

Table 1.

Participant Characteristics.

Baseline characteristics	All (n = 39)	iACT (n = 19)	Control (n = 20)
Age M (SD)	51.90 (12.88)	50.32 (12.54)	53.40 (13.35)
Gender
Female	28 (71.8%)	13 (68.4%)	15 (75%)
Men	11 (28.2%)	6 (31.6%)	5 (25%)
Marital status
Single	5 (12.8%)	4 (21.1%)	1 (5%)
Married or living together	23 (59%)	9 (47.3%)	14 (70.0%)
Divorced	8 (20.5%)	5 (26.3%)	3 (15.0%)
Widowed/other	3 (7.7%)	1 (5.3%)	2 (10.0%)
Education^a
Low	2 (5.2%)	1 (5.3%)	1 (5.0%)
Middle	20 (61.3%)	10 (52.7%)	10 (50.0%)
High	17 (43.60%)	8 (42.1%)	9 (40.0%)
Employment
Employed^b	16 (41%)	7 (36.8%)	9 (45%)
Student	2 (5.1%)	1 (5.3%)	1 (5%)
Unemployed	6 (15.4%)	3 (15.8%)	3 (15%)
Retired	11 (28.2%)	5 (26.3%)	6 (30%)
Other^c	4 (10.2%)	3 (15.8%)	1 (5%)
Depression^d
Minimal	5 (12.8%)	2 (10.5%)	3 (15%)
Mild	10 (25.6%)	5 (26.3%)	5 (25%)
Moderate	17 (43.6%)	8 (42.1%)	9 (45%)
Severe	7 (17.9%)	4 (21.1%)	3 (15%)
Medication^e	8 (20.5%)	3 (15.8%)	5 (25%)

Note. iACT = Internet-delivered acceptance and commitment therapy; BDI-II = Beck Depression Inventory.

Low < 9 years of education, middle < 12 years of education, high > 12 years of education.

Full-time, part-time, and entrepreneurs.

The group Others (6, 15.7%) included those being on sick leave, homemakers, or others choosing this option.

Depression scores according to BDI-II: 0-13 minimal, 14-19 mild, 20-28 moderate, 29-63 severe.

Use of antidepressants and/or hypnotics.

Intervention

Master’s-level students of psychology (Department of Psychology, University of Jyväskylä) performed the pre-intervention telephone assessments. During the interview, the student coaches collected general information on the problems reported by the participants to construct a case conceptualization model (functional analysis clinical case model [FACCM]; Haynes & O’Brien, 2000; Haynes, O’Brien, & Keawe’aimoku Kaholokula, 2011). The FACCM is a vector-graphic diagram for visually illustrating and analyzing problem behaviors, the relations between behavioral problems, and the importance, strength, and direction of those relations (Haynes & O’Brien, 2000). The case conceptualization was carried out to provide personalized feedback and to increase the effectiveness of the treatment. The student coaches were under supervision throughout the intervention. The lead supervisor reviewed and approved the case conceptualizations individually, and his or her individual case conceptualization analyses were posted to the participants.

The participants had no face-to-face contact with their coach during the intervention. After sending back their measurement package, the participants received emails with a link to the treatment website, including a login username and a password so that they could access the web program, Good Life Compass, from home. Home assignments and feedback were dispensed through the same portal to which both the participants and the coaches had access. The online intervention consisted of six modules, based on the core processes of ACT, which promote psychological flexibility (see, for example, Hayes et al., 2006). These were Creative Hopelessness and Values (Week 1), Value-Based Actions (Week 2), Contact With the Present Moment (Week 3), Cognitive Defusion (Week 4), Self as Context (Week 5), and Acceptance (Week 6). Each module consisted of self-help texts, videos, and downloadable Mp3 audio files (mindfulness and other experiential exercises) to illustrate the core process focused on that particular week. Table 2 presents the content of the Good Life Compass. A screenshot of the intervention website is provided in Figure 2.

Table 2.

The Content of the Good Life Compass.

Weekly theme (process)	Content of the good life compass assignment of the week	Number of exercises
Week 1: Choose your values!	Text of the week: Creative Hopelessness, Values Video: What are values in life?	13 exercises including 8 audio recordings such as Attending Your Own Funeral, Half a Year to Live, Passengers in the Bus, etc.
Weekly assignment	Value analysis: Reflect and write about your values
Week 2: Take action!	Text of the week: Value-Based Actions	The same exercises as above
	Video: What are value-based actions?
Weekly assignment	Take action: Choose actions based on your values
Week 3: Live here and now!	Text of the week: Be present, live here and now.	12 exercises including 8 audio recordings, such as Follow Your Breath, Mindful Sitting, Mindful Eating, etc.
	Video: What is mindfulness?
Weekly assignment	Practice mindfulness exercises and mindfulness in everyday life; take action: Choose another value-based action
Week 4: Self as Context	Text of the week: The different stories about myself: What story are you telling yourself?	7 exercises including 3 audio recordings, such as Observer, Leaves in the Stream, etc.
	Video: The art of observing
Weekly assignment	Retell your story and write about it; practice experiential exercises; take action: Choose another value-based action
Week 5: Watch your thinking!	Text of the week: Watch your thinking!	6 exercises, including 5 audio recordings, such as Milk, Milk; the Little Man, etc.
	Video: Getting a new perspective
Weekly assignment	Practice exercises related to cognitive defusion
	Take action: Choose another value-based action
Week 6: Accept your feelings and thoughts, accept what you cannot change	Text of the week: What do I need to accept?	3 exercises, including 1 audio recording, such as Stone on a Beach, The Child, etc.
	Video: What is acceptance?
Weekly assignment	Observer-exercise, reflect and write about: What do I need to accept? Take action: Choose another value-based action

Figure 2.

Screenshots from the program Good Life compass illustrating Week 3: Present Moment (text, video, quote, audio exercise: Leaves in the Stream).

All six modules were open to the participants throughout the whole 7-week intervention. However, participants were advised to follow the modules in a given order and to complete each module within a given time period (a week). For the weekly home assignments, the participants were instructed to read the weekly text, watch the weekly video and complete the experiential exercises within a particular module, and to reflect on questions arising out of the weekly process, as well as describe outcomes from specific assignments. For example, in the module “The Present Moment” (Week 3), participants were instructed to watch the video, read the text highlighting mindfulness, conduct daily the mindfulness exercises provided in the program or in their daily lives, report their reflections on the exercises in writing, and save these in their personal folders in a secured encrypted platform. Each week, participants received personalized written feedback through the platform within 3 days from the day they had completed the weekly exercise and saved it in their personal folder. All communication between the coaches and the participants was asynchronous.

Each week, two automated email-based reminders were sent to the participants. The first email explained the theme of that particular week with motivating ACT-based quotes and a URL to the system. The quotes were tailored to each week’s process to remind the participants of the week’s theme (Example: Week 1: Values. In the end it’s not the years in your life that count. It’s the life in your years—A. Lincoln. Dear User, welcome to Good Life Compass! We hope that you are having a good day. This is a gentle reminder to complete the first exercise tomorrow. If you wish to log into the Good Life Compass, please click on the link below and you will be redirected to the Good Life Compass website: [hyperlink]). On clicking the URL, the client was redirected to the system. The second email included the date on which the assignments were to be completed. In a situation where a given client did not complete the weekly exercise, a third reminder was sent out the following day to his or her email address. If the participant did not complete the weekly assignment on receiving these three reminders, the assigned coach contacted her or him via the telephone. Only one participant was contacted via the telephone. All the participants completed all the assignments except for one participant who failed to complete the assignments for Weeks 4, 5, and 6. One participant reported difficulties in accessing the home assignment page. However, this person followed the program from beginning to end.

After completion of the sixth and last module, measurement packages were sent to the participants in both groups (post-measurement). The measurement packages included a survey with 34 questions focusing on their experiences and opinions about the intervention (intervention group). The intervention group participants were asked to fill out the questionnaires and bring them along to a final face-to-face interview with their coaches (1 hr) and to a structured interview conducted by an independent, trained interviewer (1 hr).

Support

Master’s-level student coaches (n = 11) provided support in the form of written feedback through the secured platform. Eight coaches were responsible for two participants and three for one participant each. All the coaches were females with a mean age of 24.2 years (range = 21-33 years). Each coach was able to access her clients’ completed home assignments to monitor their progress. The purpose of the feedback was to encourage the participants to do their exercises and to give suggestions about additional exercises available in the web program. The case conceptualization was used to target the feedback according to the participant’s needs. For instance, if, on the basis of the case conceptualization, a client was considered to be engaged in excessive worrying and rumination, he or she was advised to practice, in addition to that week’s exercises, some of the mindfulness exercises and relevant metaphors included in the program. Coaches were instructed to keep the weekly feedback short, preferably within about 15 min per week.

All the coaches underwent 21 hr of training in ACT. The training included a 7-hr ACT workshop by Dr. Jason Luoma from Portland Psychotherapy, and a 2-day workshop highlighting the general principles of ACT, its core processes and presentation, as well as how to apply a FACCM (Haynes & O’Brien, 2000; Haynes et al., 2011). The students used Finnish versions of the relevant ACT Handbooks (Hayes & Smith, 2008; Lappalainen, et al., 2004) during both training and the actual intervention. They received 2 hr supervision in groups once a week for the first 3 weeks and again during the last week of the intervention. The supervisors were two researchers with significant experience in ACT procedures and ACT-based web interventions. The leading supervisor was an experienced clinician, licensed psychologist and psychotherapist, with nearly 30 years’ experience of clinical work and supervision and 12 years’ experience in ACT clinical practice.

Adherence to the ACT intervention was monitored via a weekly diary. Coaches were required to keep a record of the weekly feedback given to their client(s) during the intervention. In the diaries, coaches were asked to report which ACT processes they had targeted and which additional exercises they had suggested during a given feedback session. The diaries were checked by the supervisor and collected after the treatment period. In this way, the supervisors ensured the consistency of the written feedback with the ACT model.

Measures

Symptom measures

The primary outcome measure was the level of depressive symptoms measured with the BDI-II. The BDI-II contains 21 questions about depressive symptoms and their severity (Beck, Steer, & Brown, 2004). The scale ranges from 0 to 63 (0-13 indicates no or very few depressive symptoms, 14-19 indicates mild depression, 20-28 indicates moderate depression, and 29-63 indicates severe depression). The BDI-II has shown good reliability and validity in both nonclinical and clinical populations, and also high internal consistency (Cronbach’s α = .92; Segal, Coolidge, Cahill, & O’Riley, 2008). The Finnish version of the BDI-II has good sensitivity and specificity to differentiate nondepressed from depressed individuals (Beck et al., 2004).

The secondary outcome measure was the SCL-90 (Holi, Sammallahti, & Aalberg, 1998), which measures psychological and physiological symptoms. We expected the SCL-90 to reflect the general well-being of the participants. The SCL-90 is a self-report checklist with 90 items. Each item is rated on a 5-point scale of distress from 0 (none) to 4 (very much). In a Finnish community sample (n = 337; Holi et al., 1998), the mean general severity index (GSI) was 0.60 (SD = 0.44). Holi et al. (1998), for example, also reported that the internal consistency of the subscales was adequate (Cronbach’s alphas ranged between .77 and .90). In this study, the SCL-90 scores are reported as the GSI.

Process measures

The ACT-specific process measures included assessments of psychological flexibility, mindfulness, thought suppression, frequency, and believability of automatic thoughts. We expected to observe larger changes in these measures in the iACT group than in the wait-list control group.

Psychological flexibility was measured with the Finnish version of the AAQ-II (Bond et al., 2011; Bond & Bunce, 2003; Hayes, Strosahl, Bunting, Twohig, & Wilson, 2004). A 10-question version of the AAQ-II was used in the present study. The AAQ-II measures, on a scale from 1 (never true) to 7 (always true), the participant’s willingness to be in contact with negative private events, acceptance of those events, and whether they can live according to their values. Summing the scores resulted in a total score ranging between 10 and 70, where higher scores indicated a greater level of psychological flexibility.

The FFMQ (Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006) was employed to measure mindfulness. The FFMQ is a self-report inventory for assessing mindfulness skills. In the present study, the total sum score for the FFMQ was used to reflect mindfulness skills, including observing, describing, acting with awareness, nonjudging, and nonreactivity. The FFMQ comprises 39 questions that need to be answered on a scale from 1 (never or very rarely true) to 5 (very often or always true). A higher FFMQ score indicates a greater level of mindfulness skills.

The WBSI (Wegner & Zanakos, 1994) was employed to measure thought suppression. Chronic thoughts suppression is related to obsessive thinking and negative affects associated with depression and anxiety. The 15-item WBSI is scored on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree). The total score can range from 15 to 75. Higher scores on the WBSI indicate greater tendencies to suppress thoughts.

The ATQ (Hollon & Kendall, 1980) measures depressogenic thought frequency and believability (Addis & Jacobson, 1996). The ATQ-F measures the frequency of automatic depressive thoughts, and the ATQ-B measures the believability of (or fusion with) these thoughts. The ATQ-F is scored on a 5-point scale from 1 (the thought does not occur at all) to 5 (the thought occurs all the time). The ATQ-B is scored on a 5-point scale from 1 (I don’t believe at all) to 5 (I believe very strongly). The total score ranges between 30 and 150. The internal consistency of the ATQ-F and ATQ-B has been reported to be high (Cronbach’s α = .94 for depressed population, .89 ≤ α ≤ .91 for nondepressed population), and the measure correlates with the depression estimations given by therapists (Harrel & Ryon, 1983).

At the pre-measurement, the ACT-related process variables correlated highly with each other. For example, the AAQ-II (n = 39) correlated highly with the FFMQ (r = .59), ATQ-F (r = −.65), ATQ-B (r = −.68), and WBSI (r = −.72), thereby indicating a high degree of collinearity between the variables.

Use of the program and satisfaction with the system

After treatment, the participants were asked to report the amount of time they had spent using the system. Using a Visual Analogue Scale (VAS), on a scale from 1 (not useful at all) to 10 (very useful), the participants in the intervention group were asked to evaluate their satisfaction with the treatment (1 = not at all satisfied, 10 = very satisfied). On the same scale of 1 (would not recommend at all) to 10 (would recommend gladly), participants were also asked to rate if they would recommend the same treatment program to other people with depressive symptoms. They were also requested to answer questions in the final interview questionnaire and to comment on the overall usefulness of the intervention.

Statistical analyses

Differences between the intervention groups at the start of the treatment were tested using chi-square and t tests. We examined between-group changes during the active intervention period (pre- to post-) and within-group changes among the iACT participants during the entire study period (pre-, post-, and follow-up measurement) as follows. All the other analyses were calculated using Mplus (Version 7) statistical program (Muthén & Muthén, 1998-2012), which allows all the randomized clients to be included in the analyses. The full information maximum likelihood (FIML) estimation method was used on the assumption that there would be few values “missing at random” (MAR). First, we examined whether the changes in the groups from pre- to post-measurement (comparison between the iACT and the WLC group) differed. Second, we examined whether the within-group changes in the iACT group were significant over the pre- and post-intervention, and follow-up measurements (long-term effects of the iACT). The analyses were as follows. For analyzing and comparing the mean change between the groups across two measurements, the new parameters feature, implemented in Mplus, was used. For analyzing the overall change in treatment groups across three successive measurements, two new parameters measuring the change from the first to second and from the second to third measurement were defined. These two parameters were tested simultaneously using the Wald test. Mean values and standard deviations (SD) were calculated using Mplus to correct the means for missing values.

ESs are reported using Hedges’s g. They were calculated as follows: The between-group ES was calculated after the treatment and at follow-ups by dividing the difference between the iACT group and the WLC group mean by the pooled standard deviation of the two conditions. The within-group ES was calculated for both the post-intervention and follow-up measurements by dividing the mean change from the pre- to the post-measurements by the combined (pooled) standard deviation (SD) and by dividing the mean change from the pre- to the follow-up measurements with the combined SD (Feske & Chambless, 1995; Morris & DeShon, 2002). Because of the small sample size, the possible bias of d was corrected by an unbiased estimate of Hedges’s g (Hedges, 1981; Hedges & Olkin, 1985). Considering a possible lack of pre-test equivalence, adjusted ESs were used. Between-group effects sizes at the post-measurement were adjusted by subtracting the pre-ES from the post-ES, after calculating the pre-ES in the same manner as the post-ES (e.g., Durlak, 2009; Wilson & Lipsey, 2007). A between-group ES of 0.2 was considered small, 0.5 medium, and 0.8 large. A within-group ES of 0.5 was considered small, 0.8 medium, and 1.1 large (Roth & Fonagy, 1996; Öst, 2006).

The clinical significance of change was evaluated using the Jacobson–Truax method of evaluating change in recovering individuals (Lambert & Ogles, 2009). The method comprises two steps. The first calculates the reliable change index (RCI). The RCI equals the difference between a participant’s pre-test and post-test scores, divided by the standard error of the difference. The second step calculates the cutoff C value to find a weighted midpoint between the means of depressed and a nondepressed population (Atkins, Bedics, McGlinchey, & Beauchaine, 2005). These two steps are used to classify individuals into one of four categories: Recovered (the individual has passed Cutoff C and the RCI in the positive direction), Improved (has passed the RCI in the positive direction but not Cutoff C), Unchanged (has passed neither criterion), or Deteriorated (has passed RCI in the negative direction). When calculating the RCI and Cutoff C, the results of Beck’s validation study were used. The mean and standard deviation of the BDI-II for a nondepressed population were 7.65 and 5.9, respectively, and internal consistency was α = .91 (Beck, Steer, Ball, & Ranieri, 1996). The mean and standard deviation of the BDI-II for the participants in this study were 21.36 (M) and SD = 7.43.

Results

Comparison of iACT Treatment Group Versus Wait-List Control Group

Before treatment start, there were no significant differences in any of the symptom measures between the two groups. After treatment, the pre to post changes for the primary outcome measures of depressive symptoms (BDI-II), and for the secondary outcome measures of general psychological symptoms (SCL-90) were significantly different between the groups, in favor of the iACT intervention group. Four of the five process measures—psychological flexibility (AAQ-II), mindfulness (FFMQ), frequency of automatic (or depressive) thoughts (ATQ-F), and thought suppression (WBSI)—showed significantly larger changes in favor of the iACT group. The changes in believability of automatic thoughts (ATQ-B) did not differ significantly between the iACT group and the control group. The between-group ES at post-measurement were large for the primary outcome measure of depression (BDI, g = .83) and medium for experienced psychological and physiological symptoms (SCL-90, g = .60), psychological flexibility (AAQ-2, g = .67), mindfulness (FFMQ, g = .53), frequency of automatic thoughts (ATQ-F, g = .57), and thought suppression (WBSI, g = .53). A small difference between the groups was observed in believability of automatic thoughts (ATQ-B, g = .33). Mean scores, standard deviations, interaction, and within-group main effects are presented in Table 3.

Table 3.

Standardized Mean Scores and Standard Deviation for Depression, Physiological and Psychological Symptoms, Psychological Flexibility, Mindfulness, Automatic Thoughts, and Thought Suppression at Pre, Post, and Follow-Up.

Outcome		Pre M (SD)	Post M (SD)	Follow-up M (SD)	Pre, post change^a estimate p value	Pre, post, follow-up change^b Wald test (df = 2) p value
BDI-II	iACT	22.11 (7.79)	13.34 (6.75)	12.57 (6.15)	5.97 p < .01	42.70 p < .01
BDI-II	Control	20.65 (6.80)	17.85 (7.34)	12.57 (6.15)	5.97 p < .01	42.70 p < .01
SCL-90	iACT	1.02 (0.46)	0.65 (0.44)	0.52 (0.48)	0.31 p < .01	31.47 p < .01
SCL-90	Control	1.04 (0.40)	0.98 (0.54)	0.52 (0.48)	0.31 p < .01	31.47 p < .01
AAQ-II	iACT	39.68 (11.61)	48.35 (9.93)	50.92 (8.64)	8.31 p < .01	32.18 p < .01
AAQ-II	Control	40.65 (13.59)	41.00 (14.21)	50.92 (8.64)	8.31 p < .01	32.18 p < .01
FFMQ	iACT	129.11 (14.26)	138.57 (12.96)	140.74 (13.13)	8.86 p < .05	10.06 p < .01
FFMQ	Control	123.65 (19.00)	124.25 (19.43)	140.74 (13.13)	8.86 p < .05	10.06 p < .01
ATQ-F	iACT	70.21 (25.78)	56.73 (23.30)	50.56 (21.22)	13.88 p < .05	19.39 p < .01
ATQ-F	Control	68.95 (17.50)	69.35 (25.22)	50.56 (21.22)	13.88 p < .05	19.39 p < .01
ATQ-B	iACT	70.68 (25.80)	58.01 (22.99)	52.02 (23.62)	7.65 ns	15.74 p < .01
ATQ-B	Control	71.89 (16.50)	67.32 (23.43)	52.02 (23.62)	7.65 ns	15.74 p < .01
WBSI	iACT	48.53 (10.43)	42.82 (11.67)	36.21 (12.06)	6.81 p < .01	18.37 p < .01
WBSI	Control	49.80 (12.63)	50.90 (13.02)	36.21 (12.06)	6.81 p < .01	18.37 p < .01

Note. BDI-II = Beck Depression Inventory; SCL-90 = Symptom Checklist–90; iACT = Internet-delivered acceptance and commitment therapy; AAQ-II = Acceptance and Action Questionnaire–II; FFMQ = Five Facet Mindfulness Questionnaire; ATQ-F = Automatic Thoughts Questionnaire–Frequency; ATQ-B = Automatic Thoughts Questionnaire–Believability; WBSI = White Bear Suppression Inventory.

Investigates whether groups change differently from pre to post.

Investigates whether the iACT group changes significantly over time (pre, post, follow-up), ns = nonsignificant.

The within-group ES in the iACT group (from pre to post) was large for the primary outcome depressive symptoms (BDI-II, g = 1.18) and medium for psychological and physiological symptoms (SCL-90, g = .81). Small ESs were observed for psychological flexibility (AAQ-II, g = .79), mindfulness (FFMQ, g = .68), frequency and believability of automatic thoughts (ATQ-F, g = .54; ATQ-B, g = .51), and thought suppression (WBSI, g = .51). In the control group, the within-group ESs were very small (pre to post, g = .02-.39).

Treatment Outcome and 12-Month Follow-Up

In the intervention group, the treatment effects lasted over the 12-month follow-up period. Statistically significant changes were observed over time (pre- and post-intervention, and follow-up) in all the symptom and process measurements (see Table 3). Significant change in all measures was observed both during the active intervention period (pre to post; p < .05 to p < .001) and from the pre-intervention to follow-up period (p < .05 to p < .001).

There was a trend toward continued positive change in most measures during the follow-up period. However, a significant decrease was detected in thought suppression (WBSI) from post-intervention to follow-up (p < .05). The within-group ES of the iACT group for depressive symptoms (BDI-II; from pre-intervention to follow-up) was large (g = 1.33). Medium ESs were observed for SCL-90 (g = 1.04), AAQ-II (g = 1.08), FFMQ (g = .83), ATQ-F (g = .82), and WBSI (g = 1.07). The ESs for ATQ-B were small (g = .74).

Clinical significance (Jacobson & Truax, 1991) of depressive mood based on the BDI-II (Beck et al., 2004) was examined at post-intervention and follow-up using the following four categories: (a) Recovered, (b) Improved, (c) Unchanged, (d) Deteriorated. When examining the iACT group (n = 18) at post-intervention, 50% (9) were recovered, 11.1% (2) improved, and 38.9% (7) remained unchanged. Importantly, none of the participants had a deteriorated condition. In the control group (n = 20), 10% (2) of the participants were recovered, 10% (2) improved, and 80% (16) remained unchanged over the same period. Again, none of the participants had a deteriorated condition. Because of missing data in the iACT group (7 missing out of 19), the clinical significance data cannot be reported for the 12-month follow-up.

Use of the Program and Satisfaction With the Program

Almost all the participants completed the iACT program (n = 18). On average, 50% (n = 9) of the participants spent less than 1 hr per week on the program, for 7 participants (38.9%), the time spent was 1 to 2 hr, and for 2 participants (11.1%) 2 or more hours. On a scale from 1 (not at all) to 10 (a very great deal), participants’ mean estimate of how much effort they had put into their home assignments was 7.24 (SD = 1.75, min 4, max 10).

On a scale from 1 (not at all satisfied) to 10 (very satisfied), the mean rating of their satisfaction with the intervention given by the participants in the iACT group (n = 18) was 8.22 (SD = 1.59, min 5, max 10). In general, participants were satisfied with the intervention and gave positive comments, such as, “It gave me many new thoughts and means to cope with life. I realized that I have to take better care of myself. Now I can go on with my life happier, stronger and more satisfied.” On a scale from 1 to 10, participants were also asked whether they would recommend the treatment to other people experiencing depressive symptoms. The mean rating for recommending the intervention was 8.94 (SD = 0.10, min 7, max 10). To the open-ended question about recommending the intervention to other people, most of the participants gave a positive reply. One participant stated, “If I was to decide, this program should be mandatory for everyone!”

Discussion

This study is among the first to investigate ACT-based online treatment for depressive symptoms. Specifically, the objective was to examine whether an ACT-based Web-intervention without face-to-face contact during the course of treatment is effective in alleviating self-reported depressive symptoms. In comparison with a wait-list control group, the present guided iACT with no face-to-face contact produced significant improvements in depressive, psychological and physiological symptoms. Improvements were also observed in ACT-related processes measuring psychological flexibility, mindfulness, thought suppression, and frequency of negative thoughts. The majority of participants expressed satisfaction with the intervention and would recommend it to other people with depressive symptoms.

Efficacy

A large between-group ES for depression symptoms (g = .83) and a medium ES for psychological and physiological symptoms, and for ACT-specific processes such as psychological flexibility, mindfulness, frequency of automatic thoughts, and thought suppression, were observed in favor of the Internet treatment group (iACT). At the 12-month follow-up, positive effects for depressive symptoms and process measures had been maintained in the iACT group. In addition, large within-group ESs (pre-intervention to follow-up) for depressive symptoms and medium ESs for most of the other measures testify to the effectiveness of the iACT treatment. However, the within-group ESs for the process measures were slightly smaller than those in our previous study, which included two face-to-face meetings (P. Lappalainen et al., 2014).

Our results are consistent with the results of a recent meta-analysis by Richards and Richardson (2012), who found between-group ESs of d = .78 for therapist-supported, d = .58 for administrative-supported, and d = .36 for unsupported studies, and with those of an earlier meta-analysis by Andersson and Cuijpers (2009), who found between-group effect sizes of d = .61 for supported and d = .25 for unsupported treatments.

Self-reported duration of illness was not significantly associated with the outcome of the iACT intervention (as measured by the BDI pre–post change, r = −.32, p = .217). This suggests that the effect of the iACT intervention was not associated with the duration of depressive symptoms. The results indicate that iACT may also be effective for those with a long history of self-reported depressive symptoms. However, further research is needed to address this issue.

Use of the Program and Satisfaction With the Program

The findings indicate that our iACT treatment was well-received by the majority of the participants. Only one person (5.3%) dropped out during the treatment period. Most participants used the intervention in a daily life setting and reported that they spent at least 1 to 2 hr per week interacting with the program. The weekly modules provided information, teaching participants new skills to deal with life’s challenges, and motivating them to complete process-related exercises, which many reported to be useful. As one participant stated, “They helped me to realize how much power I had given to my mind and how negatively this had affected me.” Participants also reported having gained several benefits for their health and general well-being. However, the participants assessed as severely depressed at the pre-measurement (BDI > 30, n = 4) were not equally satisfied with the treatment. They had hoped for more support and face-to-face contact, and reported that because of extreme fatigue and distress, they were not able to benefit from the intervention as much as they had expected. Nevertheless, they reported that they had learnt new skills to cope with their thoughts and feelings.

In line with other studies, we conclude that Internet-delivered interventions probably stimulate self-management and empowerment. This was reflected in one participant’s comment: “My life is in my hands, not in other peoples’ hands and particularly not in my mind’s hands.” Although individuals suffering from severe depression were not in focus here, there are indications that the same kind of intervention supplemented by some face-to-face contact might benefit those who suffer from severe depression. This is in line with earlier findings (cf. Bendelin et al., 2011; Lillevoll et al., 2013), indicating that web-based treatments have the ability to motivate, engage, and enable people to work on their own. Severely depressed individuals might have the motivation but, owing to exhaustion and distress, they may not be able to work as independently as those with mild depression. Therefore, further investigation is needed to find out precisely for whom Internet-based ACT treatment is suitable and how much support is needed.

Limitations

The first limitation of the presented study is the relatively small sample size. Therefore, the results cannot be regarded as representative for all people who receive Internet-based treatment for depressive symptoms, and hence should be interpreted with caution. Second, the majority of the participants were female (71.8%), which can also be considered a limitation. Third, the sample itself might be a limitation because recruitment was done through newspaper advertisements. Persons responding to a newspaper advertisement might be more motivated to participate in a new intervention trial than patients receiving treatment in clinical settings, and this might affect the results. Another limitation of the present study concerns the measures. All the measures were self-reports, which could influence the validity of the study. However, self-reports are commonly used and are, perhaps, the only feasible way to evaluate the constructs of interest. A similar, additional limitation is the fact that the participants were not diagnosed. However, they were interviewed through a structured telephone interview elaborated on the basis of the DSM-IV-TR criteria for major depressive disorder. Moreover, the intervention aimed at reaching individuals who had not been diagnosed and, therefore, did not have access to adequate treatment. In addition, except for students of psychology who receive brief training in the ACT model, the results cannot be generalized to experienced therapists conducting treatment for depression. Finally, the use of a wait-list control group can be considered a potential weakness in our study as such a group does not offer a strong test of treatment efficacy.

Clinical Implications and Future Directions

To summarize, this study provides evidence that an iACT-based intervention over 6 weeks combined with weekly therapist feedback can have a positive impact on depressive symptoms, psychological flexibility, and mindfulness skills. Furthermore, iACT is acceptable to clients and can be successfully delivered by novice therapists without formal health care qualifications. This study suggests that iACT-based interventions are potentially useful as an early-stage intervention for people suffering from depressive symptoms. However, further research is needed to investigate the proposed treatment model and its effects in a larger population sample. In particular, the effectiveness of this treatment model needs to be studied in clinical trials. We also suggest that it would be worthwhile to exploring what specific populations could benefit the most from iACT-based treatment. The mechanisms by which low-intensity, Web-based interventions can produce benefit and change also need to be further explored. Future studies could also examine mediators in iACT, thereby contributing knowledge on factors that are important for successful treatments. To date, mediators of change have not been investigated in iACT for depression.

Despite its limitations, the present study adds to the current literature and knowledge base on ACT-based treatments for depressive symptoms. It shows that changes in ACT-specific processes enhancing psychological flexibility are closely associated with positive treatment outcomes and improvements in functioning. This is consistent with previous studies across different conditions (cf. Flaxman & Bond, 2010; Gregg, Callaghan, Hayes, & Glenn-Lawson, 2007; R. Lappalainen et al., 2007; McCracken, 2013; McCracken & Gutiérrez-Martínez, 2011). The study indicates that different aspects of psychological flexibility can be targeted and improved through a relatively short Internet-based ACT treatment without any face-to-face contact.

The present study also exemplifies how LI interventions in health care could be realized in practice, that is, delivery of an Internet-based ACT intervention with limited support and operated by workers without formal health professional qualifications. The study suggests that iACT is a noteworthy means to ensure access to psychological treatment and to enhance patient self-management, empowerment, recovery, and general well-being.

Footnotes

Acknowledgements

We would like to thank Riikkasisko Kirjonen and the entire team of student coaches from the Department of Psychology, University of Jyväskylä, for their valuable contribution to the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Päivi Lappalainen, MA, is currently working on her dissertation on web-based psychological interventions in the Department of Psychology at the University of Jyväskylä, Finland. Her research interests include acceptance and commitment therapy and web and mobile interventions for depression and general well-being.

Sitwat Langrial, PhD, completed his doctorate in the Department of Information Processing Science at the University of Oulu. His research interests include eHealth, human–computer interaction (HCI), and information systems for mental well-being.

Harri Oinas-Kukkonen is a professor of information systems in the Department of Information Processing Science at the University of Oulu, Finland. His current research interests within human–computer interaction and health informatics include user behavior, behavior change, persuasive design, social knowledge, innovation creation, and the next generation of the web.

Asko Tolvanen is professor in the Methodology Center for Human Sciences at the University of Jyväskylä. He is a statistician and is interested in developing and applying structural equation modeling in the context of multilevel and mixture modeling.

Raimo Lappalainen is a professor in clinical psychology and psychotherapy in the Department of Psychology at the University of Jyväskylä, Finland. He is a licensed psychologist and psychotherapist. His current research interests include acceptance and commitment therapy, web/mobile-based interventions, and brief psychological interventions.

References

Addis

M. E.

Jacobson

N. S.

(1996). Reasons for depression and the process and the outcome of cognitive-behavioral psychotherapies. Journal of Consulting and Clinical Psychology, 64, 1417-1424.

Alonso

Angermeyer

M. C.

Bernert

Bruffaerts

Brugha

T. S.

Bryson

. . . Almansa

(2004). Disability and quality of life impact of mental disorders in Europe: Results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatrica Scandinavica. Supplementum, 420(109), 38-46.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Andersson

Carlbring

Ljótsson

Hedman

(2013). Guided Internet-based CBT for common mental disorders. Journal of Contemporary Psychotherapy, 43, 223-233.

Andersson

Cuijpers

(2009). Internet-based and other computerized psychological treatments for adult depression: A meta-analysis. Cognitive Behavior Therapy, 38, 196-205.

Atkins

D. C.

Bedics

J. D.

McGlinchey

J. B.

Beauchaine

T. P.

(2005). Assessing clinical significance: Does it matter which method we use? Journal of Consulting and Clinical Psychology, 73, 982-989.

Baer

R. A.

Smith

G. T.

Hopkins

Krietemeyer

Toney

(2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13, 27-45.

Beck

A. T.

Steer

R. A.

Ball

Ranieri

(1996). Comparison of Beck Depression Inventories-IA and -II in psychiatric outpatients. Journal of Personality Assessment, 67, 588-597.

Beck

A. T.

Steer

R. A.

Brown

G. K.

(2004). BDI II—Beckin depressioasteikko. (The Beck Depression Inventory). Käsikirja, Helsinki: Psykologien Kustannus Oy.

10.

Bendelin

Hesser

Dahl

Carlbring

Zetterqvist Nelson

Andersson

(2011). Experiences of guided Internet-based cognitive-behavioural treatment for depression: A qualitative study. BMC Psychiatry, 11, Article 107.

11.

Bennet-Levy

Richards

D. A.

Farrand

(2010). Low-intensity CBT interventions: A revolution in mental health care. In Bennett-Levy

. (Eds.), Oxford guide to low intensity CBT interventions (pp. 3-18). Oxford, NY: Oxford University Press.

12.

Bohlmeijer

E. T.

Fledderus

Rokx

T. A.

Pieterse

M. E.

(2011). Efficacy of an early intervention based on acceptance and commitment therapy for adults with depressive symptomatology: Evaluation in a randomized controlled trial. Behavior Research and Therapy, 49, 62-67.

13.

Bond

F. W.

Bunce

(2003). The role of acceptance and job control in mental health, job satisfaction, and work performance. Journal of Applied Psychology, 88, 1057-1067.

14.

Bond

F. W.

Hayes

S. C.

Baer

R. A.

Carpenter

K. M.

Guenole

Orcutt

H. K.

. . . Zettle

R. D.

(2011). Preliminary psychometric properties of the Acceptance and Action Questionnaire–II: A revised measure of psychological flexibility and experiential avoidance. Behavior Therapy, 42, 676-688.

15.

Bricker

Wyszynski

Comstock

Heffner

J. L.

(2013). Pilot randomized controlled trial of web-based acceptance and commitment therapy for smoking cessation. Nicotine & Tobacco Research, 15, 1756-1764.

16.

Buhrman

Skoglund

Husell

Bergström

Gordh

Hursti

. . . Andersson

(2013). Guided Internet-delivered acceptance and commitment therapy for chronic pain patients: A randomized controlled trial. Behavior Research and Therapy, 51, 307-315.

17.

Cuijpers

Smit

van Straten

(2007). Psychological treatments of subthreshold depression: A meta-analytic review. Acta Psychiatrica Scandinavica, 115, 434-441.

18.

de Graaf

L. E.

Gerhards

S. A. H.

Arntz

Riper

Metsemakers

J. F.

Evers

S. M.

. . . Huibers

M. J.

(2009). Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: Randomised trial. The British Journal of Psychiatry, 194, 73-80.

19.

Durlak

J. A.

(2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34, 917-928.

20.

Ekman

Granström

Omérov

Jacob

Landén

(2013). The societal cost of depression: Evidence from 10,000 Swedish patients in psychiatric care. Journal of Affective Disorders, 150, 790-797.

21.

Feske

Chambless

D. L.

(1995). Cognitive behavioral versus exposure only treatment for social phobia: A meta-analysis. Behavior Therapy, 26, 695-720.

22.

Flaxman

Bond

F. W.

(2010). A randomised worksite comparison of ACT and stress inoculation training. Behaviour Research and Therapy, 48, 816-820.

23.

Fledderus

Bohlmeijer

E. T.

Pieterse

M. E.

Schreurs

K. M. G.

(2012). Acceptance and commitment therapy as guided self-help for psychological distress and positive mental health: A randomized controlled trial. Psychological Medicine, 42, 485-495.

24.

Forman

E. M.

Herbert

J. D.

Moitra

Yeomans

P. D.

Geller

P. A.

(2007). A randomized controlled effectiveness trial of acceptance and commitment therapy and cognitive therapy for anxiety and depression. Behavior Modification, 31, 772-799.

25.

Fry

J. P.

Neff

R. A.

(2009). Periodic prompts and reminders in health promotion and health behavior interventions: Systematic review. Journal of Medical Internet Research, 11, e16.

26.

Gregg

J. A.

Callaghan

G. M.

Hayes

S. C.

Glenn-Lawson

J. L.

(2007). Improving diabetes self-management through acceptance, mindfulness, and values: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 75, 336-343.

27.

Harrel

T. H.

Ryon

N. B.

(1983). Cognitive-behavioral assessment of depression: Clinical validation of the Automatic Thoughts Questionnaire. Journal of Consulting and Clinical Psychology, 51, 721-725.

28.

Hayes

S. C.

Luoma

J. B.

Bond

F. W.

Masuda

Lillis

(2006). Acceptance and commitment therapy: Model, processes and outcomes. Behaviour Research and Therapy, 44, 1-26.

29.

Hayes

S. C.

Smith

(2008). Vapaudu mielesi vallasta ja ala elää: Hyväksymis- ja omistautumisterapian avulla [Get out of your mind and into your life—The new acceptance and commitment therapy]. Jyväskylä, Finland: Gummerus Kirjapaino Oy.

30.

Hayes

S. C.

Strosahl

K. D.

Bunting

Twohig

Wilson

K. G.

(2004). What is acceptance and commitment therapy? In Hayes

S. C.

Wilson

K. G.

(Eds.), A practical guide to acceptance and commitment therapy (pp. 3-29). New York, NY: Springer.

31.

Hayes

S. C.

Strosahl

K. D.

Wilson

K. G.

(1999). Acceptance and commitment therapy: An experimental approach to behavior change. New York, NY: Guilford Press.

32.

Haynes

S. N.

O’Brien

(2000). Principles and practice of behavioral assessment. New York, NY: Springer.

33.

Haynes

S. N.

O’Brien

W. H.

Keawe’aimoku Kaholokula

(2011). Behavioral assessment and case formulation. Hoboken, NJ: John Wiley.

34.

Hedges

L. V.

(1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.

35.

Hedges

L. V.

Olkin

(1985). Statistical methods for meta-analysis. London, England: Academic Press.

36.

Heffner

J. L.

Wyszynski

C. M.

Comstock

Mercer

L. D.

Bricker

(2013). Overcoming recruitment challenges of web-based interventions for tobacco use: The case of web-based acceptance and commitment therapy for smoking cessation. Addictive Behaviors, 38, 2473-2476.

37.

Hesser

Gustafsson

Lunden

Henrikson

Fattahi

Johnsson

. . . Andersson

(2012). A randomized controlled trial of Internet-delivered cognitive behavior therapy and acceptance and commitment therapy in the treatment of tinnitus. Journal of Consulting & Clinical Psychology, 4, 649-661.

38.

Holi

M. M.

Sammallahti

P. R.

Aalberg

V. A.

(1998). A Finnish validation study of the SCL-90. Acta Psychiatrica Scandinavica, 97, 42-46.

39.

Hollon

S. D.

Kendall

P. C.

(1980). Cognitive self-statements in depression: Development of an Automatic Thoughts Questionnaire. Cognitive Therapy and Research, 4, 384-395.

40.

Jacobson

N. S.

Truax

(1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.

41.

Johansson

Andersson

(2012). Internet-based psychological treatments for depression. Expert Review of Neurotherapeutics, 12, 861-870.

42.

Kessler

R. C.

McGonagle

K. A.

Zhao

Nelson

C. B.

Hughes

Eshleman

. . . Kendler

K. S.

(1994). Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: Results from the National Comorbidity Survey. Archives of General Psychiatry, 51, 8-19.

43.

Kleine-Budde

Müller

Kawohl

Bramesfeld

Moock

Rössler

(2013). The cost of depression—A cost analysis from a large database. Journal of Affective Disorders, 147, 137-143.

44.

Kohtala

Lappalainen

Savonen

Timo

Tolvanen

(2013). A four-session acceptance and commitment therapy based intervention for depressive symptoms delivered by master’s degree level psychology students: A preliminary study. Behavioral and Cognitive Psychotherapies, 11, 1-14.

45.

Lambert

M. J.

Ogles

B. M.

(2009). Using clinical significance in psychotherapy outcome research: The need for a common procedure and validity data. Psychotherapy Research, 19, 493-501.

46.

Lappalainen

Granlund

Siltanen

Ahonen

Vitikainen

Tolvanen

Lappalainen

(2014). ACT Internet-based vs. face-to-face? A randomized controlled trial of two ways to deliver acceptance and commitment therapy for depressive symptoms: An 18-month follow-up. Behavior Research and Therapy, 61, 43-54.

47.

Lappalainen

Lehtonen

Hayes

S. C.

Batten

Gifford

Wilson

K.G.

McCurry

(2004). Hyväksymis- ja omistautumisterapia käytännön terapiatyössä [Applying acceptance and commitment therapy (ACT): A clinical manual]. Tampere, Finland: Domus Print Oy.

48.

Lappalainen

Lehtonen

Skarp

Taubert

Ojanen

Hayes

S. C.

(2007). The impact of CBT and ACT models using psychology trainee therapists: A preliminary controlled effectiveness trial. Behavior Modification, 31, 488-511.

49.

Lillevoll

K. R.

Wilhelmsen

Kolstrup

Sørensen Høifødt

Waterloo

Eisemann

Bech Risør

(2013). Patients’ experiences of helpfulness in guided internet-based treatment for depression: Qualitative study of integrated therapeutic dimensions. Journal of Medical Internet Research, 15, e126.

50.

Mathers

C. D.

Loncar

(2006). Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine, 3, e442

51.

McCracken

L. M.

(2013).Committed action: An application of the psychological flexibility model to activity patterns in chronic pain. The Journal of Pain, 14, 828-835.

52.

McCracken

L. M.

Gutiérrez-Martínez

(2011). Processes of change in psychological flexibility in an interdisciplinary group-based treatment for chronic pain based on acceptance and commitment therapy. Behaviour Research and Therapy, 49, 267-274.

53.

Morris

S. B.

DeShon

R. P.

(2002).Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105-125.

54.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (7th ed.) Los Angeles, CA: Author.

55.

Newman

M. G.

Szkodny

L. E.

Llera

S. J.

Przeworski

(2011). A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: Is human contact necessary for therapeutic efficacy? Clinical Psychology Review, 31, 89-103.

56.

Oinas-Kukkonen

(2013). A foundation for the study of behavior change support systems. Personal and Ubiquitous Computing, 17, 1223-1235.

57.

Oinas-Kukkonen

Harjumaa

(2009). Persuasive systems design: Key issues, process model, and system features. Communications of the Association for Information Systems, 24, 485-500.

58.

Öst

L. G.

(2006). Det empiriska stödet för KBT vid psykiska störningar [The empirical support for CBT in psychiatric disorders]. In Öst

L. G.

(Ed.), KBT: Kognitiv beteendeterapi inom psykiatrin. Stockholm, Sweden: Natur och Kultur.

59.

Öst

L. G.

(2008). Efficacy of the third wave of behavioral therapies: A systematic review and meta-analysis. Behaviour Research and Therapy, 46, 296-321.

60.

Öst

L. G.

(2014). The efficacy of acceptance and commitment therapy: An updated systematic review and meta-analysis. Behaviour Research and Therapy, 61, 105-121.

61.

Powers

M. B.

Zum Vörde Sive Vörding

M. B.

Emmelkamp

P. M. G.

(2009). Acceptance and commitment therapy: A meta-analytic review. Psychotherapy and Psychosomatics, 78, 73-80.

62.

Richards

Richardson

(2012). Computer-based psychological treatments for depression: A systematic review and meta-analysis. Clinical Psychology Review, 32, 329-342.

63.

Roth

Fonagy

(1996).What works for whom? A critical review of psychotherapy research. New York, NY: Guilford Press.

64.

Ruiz

(2010). A review of acceptance and commitment therapy (ACT) empirical evidence: Correlational, experimental psychopathology, component and outcome studies. International Journal of Psychology and Psychological Therapy, 10, 125-162.

65.

Ruiz

(2012). Acceptance and commitment therapy versus traditional cognitive behavioral therapy: A systematic review and meta-analysis of current empirical evidence. International Journal of Psychology and Psychological Therapy, 12, 333-357.

66.

Schneider

de Vries

Candel

van de Kar

van Osch

(2013). Periodic email prompts to re-use an Internet-delivered computer-tailored lifestyle program: Influence of prompt content and timing. Journal of Medical Internet Research, 15, e23.

67.

Segal

D. L.

Coolidge

F. L.

Cahill

B. S.

O’Riley

A. A.

(2008). Psychometric properties of the Beck Depression Inventory-II (BDI-II) among community-dwelling older adults. Behavior Modification, 32, 3-20.

68.

Twohig

M. P.

Hayes

S. C.

(2008). ACT verbatim for depression & anxiety. Oakland, CA: New Harbinger.

69.

von Knorring

Åkerblad

A.-C.

Bengtsson

Carlsson

Å.

Ekselius

(2006). Cost of depression: Effect of adherence and treatment response. European Psychiatry, 21, 349-354.

70.

Wegner

D. M.

Zanakos

(1994). Chronic thought suppression. Journal of Personality, 62, 615-640.

71.

Wilson

S. J.

Lipsey

M. W.

(2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33, 130-143.

72.

World Health Organization. (2008). Depression (A Global Public Health Concern. Developed by Marina Marcus, M. Taghi Yasamy, Mark van Ommeren, and Dan Chisholm, Shekhar Saxena). WHO Department of Mental Health and Substance Abuse. Retrieved from http://www.who.int/mental_health/management/depression/who_paper_depression_wfmh_2012.pdf.

73.

Zettle

R. D.

Rains

J. C.

(1989). Group cognitive and contextual therapies in treatment of depression. Journal of Clinical Psychology, 45, 436-445.

74.

Zettle

R. D.

Rains

J. C.

Hayes

S. C.

(2011). Processes of change in acceptance and commitment therapy and cognitive therapy for depression: A mediational reanalysis of Zettle and Rains (1989). Behavior Modification, 35, 265-283.