Isolating therapeutic procedures to investigate mechanisms of change in cognitive behavioral therapy for depression

Abstract

Background:

Isolating a therapeutic procedure might be a powerful way to enhance our understanding of how cognitive behavior therapy (CBT) works. The present study explored new methods to isolate cognitive procedures and to study their direct impact on hypothesized underlying processes and CBT outcome.

Method:

The effects of a cognitive therapy skill acquisition procedure (n = 36) were compared to no procedure (n = 36) on cognitive therapy skills, dysfunctional thinking, distress, and mood in response to induced distress following a social stress test in healthy participants.

Results:

Participants reported more cognitive therapy skills after the procedure that focused on the acquisition of cognitive therapy skills compared to no procedure, but there were no differences in dysfunctional thinking, distress, and mood between the groups.

Conclusions:

By demonstrating an experimental approach to investigate mechanisms of change, including the pitfalls that come along with it, the present experiment provides a blueprint for other researchers interested in the underlying mechanisms of change in CBT for depression.

Keywords

Cognitive therapy skills depression therapeutic procedures treatment process Trier social stress test

Introduction

Cognitive behavior therapy (CBT) is the most extensively studied and widely recognized treatment for depression (for a review, see Butler, Chapman, Forman, & Beck, 2006; Cuijpers et al., 2013). However, about 47% of the individuals suffering from depression show no response to CBT (Cuijpers et al., 2014) and its effectiveness might have been overestimated (Cuijpers, Smit, Bohlmeijer, Hollon, & Andersson, 2010). Based on these findings, it seems necessary to explore how CBT could be optimized to achieve better treatment outcomes. A first step toward optimizing treatment outcomes would be to understand through which pathways CBT actually works (Hollon et al., 2002; Kazdin, 2007; Kazdin, 2009).

CBT for depression aims to tackle dysfunctional thinking by challenging negative thought patterns. This process is also referred to as cognitive change. Cognitive change can be described as the change of the content and structure of dysfunctional cognitions that are often measured using a self-report questionnaire that focuses on the patient’s dysfunctional beliefs. Accordingly, previous research showed a strong association between CBT and cognitive change and cognitive change and depressive outcomes (Beevers & Miller, 2005; Garrat, Ingram, Rand, & Sawalani, 2007; Lorenzo-Luaces, German, & DeRubeis, 2014; Quilty, McBride, & Bagby, 2008; Segal et al., 2006). However, most of these studies were unable to demonstrate a causal relationship between cognitive change and depressive outcome (Lorenzo-Luaces et al., 2014). Specifically, these studies showed that change in dysfunctional beliefs was associated with a greater reduction in depressive symptoms not only after CBT but also after other forms of psycho- or pharmacotherapy (Cristea et al., 2015; Garratt, Ingram, Rand, & Sawalani, 2007; Lorenzo-Luaces et al., 2014; Quilty et al., 2008). Hence, whether cognitive change is a direct result of cognitive procedures that are responsible for the change in depressive outcomes or a more general epiphenomenon of recovery from depression as a result of treatment remains an important question (Craighead, Sheets, & Bjornsson, 2005; Huibers & Cuijpers, 2015; Strunk, Brotman, & DeRubeis, 2010).

It is also possible that it is not the process of cognitive change but the acquisition or enhancement of cognitive therapy skills is responsible for the effects of CBT (Barber & DeRubeis, 1989; Strunk, DeRubeis, Chiu, & Alvarez, 2007; Strunk, Hollars, Adler, Goldstein, & Braun, 2014). Cognitive therapy skills have been defined as the ability to reevaluate the accuracy of one’s dysfunctional beliefs and engage proactively in pleasurable activities (Strunk et al., 2014). It has been suggested that learning cognitive therapy skills might be the primary process in CBT that subsequently leads to cognitive change. Circumstantial evidence for the hypothesis that cognitive therapy skills might act as a mechanism of change has been provided in a number of studies showing that the frequency and quality of cognitive therapy skills improve during treatment (Hundt, Mignogna, Underhill, & Cully, 2013; Lorenzo-Luaces et al., 2014). Moreover, cognitive therapy skills may explain why provocation of sad mood was associated with increased dysfunctional thinking in patients who received antidepressants compared to patients who received CBT, while both treatments have been associated with cognitive change (Barber & DeRubeis, 1989; Barber & DeRubeis, 2001; Garrat et al., 2007; Segal, Gemar, & Williams, 1999; Segal et al., 2006). However, the exact nature of the association between cognitive procedures, cognitive therapy skills, and treatment outcome is unclear.

At least two reasons might explain why previous studies were unable to confirm hypotheses about mechanisms of change in CBT beyond terms of association. First, the majority of studies were not able to show statistical mediation and simultaneously establish a time line between the mediator (the proxy variable that is often tested to account for the relationship between the independent and dependent variables) and outcome at the same time. This makes it impossible to determine whether change in the mediator actually preceded change in the outcome. Second, although direct manipulation of the mediator is assumed to be one of the most powerful ways to enhance our understanding of how psychotherapy works (Kazdin, 2007, 2009), this method has not been used to investigate mechanisms of change in CBT for depression. In fact, treatment experiments have been more common in the field of anxiety disorders, where the results of early (animal) experiments on the original learning theory are still seen as the theoretical basis for the behavioral treatment of anxiety disorders (Clark, 2004; Lissek et al., 2005; Mineka & Oehlberg, 2008). However, a recent special issue of Behavior Research and Therapy (Craske, 2016) reviewed the latest advances in experimental psychopathology and demonstrated how using an experimental approach might help to understand the processes underlying therapeutic change across a variety of psychological disorders. More specific, one way to increase insights into mechanisms of change might be to make the distinction between therapeutic procedures and change processes, as was recently illustrated with findings on the effects, and modification, of anxiety-linked attentional biases (Macleod & Grafton, 2016). For example, such a distinction might help to identify under what conditions therapeutic procedures will result in change in the hypothesized underlying processes and investigate how existing therapeutic procedures can be optimized (Macleod & Grafton, 2016).

An unprecedented example study in the field of depression was conducted by Teasdale and Fennel (1982). In this study, five patients treated with CBT for their depression received in random order a session focusing on changing their dysfunctional thoughts versus merely exploring their dysfunctional thoughts (without attempting to change thoughts). The results showed that the patients demonstrated a larger improvement in mood and less belief in the dysfunctional thought after a session with cognitive change procedures, compared to a session of only exploring the dysfunctional thought. These findings support the hypothesis that modifying dysfunctional thinking can directly reduce symptoms of depression and might explain how CBT really works. Moreover, it shows that the most straightforward way to investigate the direct role of cognitive change or skills in CBT might be to isolate a cognitive procedure that is expected to lead to immediate changes in the mediator and the subsequent outcome.

The present experiment examined whether we can isolate cognitive procedures to investigate how CBT for depression works. Showing that the cognitive procedure leads to changes in the mediator and the outcome, compared to the control procedure, would strengthen the notion that the hypothesized process functions as a mechanism of change. An illustration of the hypothesized cognitive procedures and processes in CBT for depression is presented in Figure 1.

Figure 1.

Hypothesized cognitive procedures and processes in CBT for depression.

Effects of cognitive therapy skill acquisition on cognitive skills, dysfunctional thinking, and mood following a social stress test

The present experiment compared a procedure that focused on the acquisition of cognitive therapy skills to no procedure on measures of cognitive therapy skills (process), dysfunctional thinking (process), and mood (outcome) in a healthy student sample. As a way to activate dysfunctional thinking, the current experiment used a social stress test. Stress is a phenomenon that is highly related to both depression and anxiety (Hammen, 2005, 2006) and as an earlier experiment reported levels of dysfunctional thinking similar to that of a nondepressed sample after a sad mood induction in a healthy sample (de Graaf, Roelofs, & Huibers, 2009; Renner, Schwarz, Peters, & Huibers, 2014), we expected that for a student sample with low depression severity, a stress test would be more effective in inducing dysfunctional thinking. We expected that participants who received the procedure would report more cognitive therapy skills after the cognitive skill acquisition and show less dysfunctional thinking, less distress, and better mood after exposure to a social stress test than participants that did not receive the procedure.

Method

Design and participants

The present study had a 2 (procedure, between participants) × 3 (time, within participants) design. Participants were randomly assigned to a written cognitive therapy skill acquisition procedure (CTSAP; n = 36) or a control group that did not receive the procedure (n = 36) before exposing them to a social stress test. Seventy-two undergraduate healthy students (9 males and 63 females) ranging in the age from 18 years to 26 years (CTSAP: mean 21.22, SD 4.63 and control: mean 19.56, SD 1.64) participated in this study.

Therapeutic procedure: CTSAP

We developed a short procedure that was based on the protocol of Beck, Rush, Shaw, and Emery (1979). The goal of the procedure was to teach students a small set of cognitive therapy skills to challenge automatic negative thoughts using a written assignment. The first part of the procedure consisted of an introduction on the relation between thinking, behavior and mood, and the definition and examples of automatic negative thoughts and asked the participant to make a list of one’s own automatic negative thoughts that he or she would commonly encounter in daily life. Subsequently, the second part of the procedure focused on teaching participants two different ways to challenge their own automatic negative thoughts. The first technique showed participants that automatic negative thoughts can be challenged by searching for evidence that the automatic negative thought is true or not true, while the second technique focused on questioning the functionality of the thought (“how is having this thought helping you right now?”). After a description of the techniques, participants were asked to apply them to two of their own automatic negative thoughts they had identified during the first part of the procedure and two additional scenarios that included the presence of automatic negative thoughts in a situation similar to the social stress test (i.e., dysfunctional beliefs for scenarios 1 and 2 were “I am a loser” and “I will never be okay,” respectively). The total procedure took a maximum of 60 min.

Procedure

Ethics approval for this study was given by the scientific ethics board of VU Amsterdam, and informed consent was obtained from all participants. Participants were included if they were not in the treatment for a psychological problem as defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV-TR. The experiment took place on two subsequent days and a follow-up 6–8 days later. Phones were turned off during the whole experiment. During Day 1, baseline measures were completed, and the experimental group completed the CTSAP. Afterward, both groups completed the cognitive therapy skill test. Day 2 started with a 15-min relax period. During this period, participants were provided with popular magazines and given the opportunity to bring their own reading. After relaxation, the experimental group received a reminder of the CTSAP (i.e., participants were given six questions to retrieve information from the CTSAP: (1) Give a summary of the CTSAP, (2) “Describe the relationship between behavior, thinking, and emotions,” (3) What is the role of negative thinking in the relation between thinking, behavior, and emotions?, (4) Can you give an example from daily life?, (5) “How would you cope with automatic negative thoughts?,” and (6) “Describe an example of a automatic negative thought you might encounter during a job interview and describe how you would cope with this thought”). Subsequently, participants received a short summary containing the correct answers and examples on these questions and participants from both groups were introduced to the Trier social stress test (TSST). The TSST has repeatedly shown to be a valuable tool for investigating stress responses in a laboratory setting (Birkett, 2011; Gaab et al., 2003; Kirschbaum, Pirke, & Hellhammer, 1993). After relaxing for 15 min, participants were introduced to the TSST by first showing them the interview room. They were then asked to prepare a speech in the next 10 min. After 10 min, they returned to the interview room to give a speech for the next 5 min. The speech was followed by a mental arithmetic task (5 min) in front of a jury consisting of two people. During data collection, three participants dropped out because they felt too stressed to perform the speech. After completing the TSST, participants were asked to complete questionnaires. Six to eight days after the first day, participants returned to the lab to complete follow-up questionnaires and for a debriefing. Participants received course credit or monetary compensation (13 euro) for their participation.

Measures

All measures were completed at baseline, after the TSST, and 6–8 days later.

Process measures

Cognitive therapy skill test

Cognitive therapy skills were tested using two written scenarios that included the presence of a automatic negative thought and asked participants what they would do in these situations. An English version of the cognitive therapy skill test is provided in Appendix 1. For each technique (i.e., searching for evidence and investigating the functionality of the thought) that was used to cope with the automatic negative thoughts in the two hypothetical situations, participants received 1 point (i.e., dichotomous variable: 0 = technique is present, 1 = technique is present) resulting in a minimum score of 0 and a maximum score of 4 (i.e., when both techniques were used for each scenario). Manipulation checks were scored by two independent raters that were blind to condition. In the case of different ratings, a third rater decided on the final rating.

Cognition checklist

The Cognition Checklist (CCL) investigates automatic negative thoughts and cognitions related to anxiety and depression (Steer, Beck, Clark, & Beck, 1994). The scale consists of 26 items that are rated on a 5-point Likert-type scale ranging from 1 = never to 5 = always and is divided in a depression (CCL-D, 14 items) and anxiety (CCL-A, 12 items) subscale. In previous research, average scores of depressed patients were 15.25 (SD = 8.23) and 24.04 (SD = 10.95) on the CCL-A and CCL-D, respectively. The coefficients α for the CCL-D and CCL-A were, respectively, .93 and .91 for the patients with a variety of DSM III diagnoses and convergent, and discriminant validity was supported (Steer et al., 1994).

Outcome measures

Positive Affect and Negative Affect Scale

The Positive Affect and Negative Affect Scale (PANAS) consists of 20 words that reflect positive affect (10 items) and negative affect (10 items) (Watson, Clark, & Tellegen, 1988). Participants are asked to rate each item on a 5-point Likert-type scale from 1 = not at all to 5 = extremely. Means and standard deviations (SD) for depressed patients are 17.1 (SD = 7.2) on the positive affect subscale and 34.3 (SD = 8.7) on the negative affect subscale, respectively (Boon & Peeters, 1999). Cronbach’s α for the Dutch version of the negative and positive affect subscale was .83 and .79, respectively (Peeters, Ponds, & Vermeeren, 1996). Construct validity was supported (Watson et al., 1988).

State-Trait Anxiety Inventory

The State-Trait Anxiety Inventory (STAI) consists of two distinct anxiety concepts: state and trait anxiety (Spielberger, Gorsuch, & Lushene, 1970). For this experiment, we only used the 20-item state anxiety subscale that measures a temporary feeling of tension and autonomic nervous system activation. Items are scored on a 4-point Likert-type scale, ranging from 1 = not at all to 4 = very much. Baseline mean scores of college students vary from 35 to 40 on the state scale and increase to a range of 43 to 60 after experiencing a stressful situation. Cronbach’s α ranged from .82 to .93 (Spielberger et al., 1970). Construct and concurrent validity were supported (Spielberger et al., 1970). Previous studies showed that state anxiety increases during the TSST (Birkett, 2011).

Data analyses

First, means and standard deviations of the baseline measurements were reported. Also, for the process and outcome measures, the Cronbach’s α in the current sample was reported. Second, the differences between the groups on the cognitive therapy skill test were tested using an independent t-test. Third, we tested group by time interactions using repeated-measures analysis of covariance. All analyses were conducted using SPSS (version 21.0; IBM Corp., 2012), and for all analyses, significance was tested at α < .05 (two-tailed). Levene’s test was used to test the homogeneity of variances.

Results

Baseline characteristics

Baseline characteristics can be found in Table 1. Mean and standard deviations for each group before and after the TSST, and at 1-week follow-up can be found in Table 2. Data of one participant were missing in the control group on the PANAS-positive and CCL at baseline. For the measurements after the TSST and at follow-up, data of three participants (two in the control group and one in the experimental group) on the PANAS, data of four participants on the STAI (two in each group), and of five participants (one participant in the experimental group and four in the control group) on the CCL were missing.

Table 1.

Baseline characteristics for each group (means and standard deviations).

	Cognitive therapy skill acquisition (n = 36)	No procedure (n = 35)
Female sex	88.9%	86.1%
Age*	21.22 (4.63)	19.56 (1.64)
Dysfunctional thinking
CCL anxiety	8.63 (6.87)	7.80 (5.69)
CCL depression	7.75 (8.12)	7.05 (5.55)
Mood
Positive mood (PANAS)	23.80 (4.36)	22.80 (3.69)
Negative mood (PANAS)	21.63 (3.81)	20.52 (4.05)
State anxiety (STAI)	33.73 (7.68)	33.29 (6.04)

Note. CCL = Cognition Checklist; PANAS = Positive and Negative Affect Scale; STAI = State-Trait Anxiety Inventory. Groups differed significantly on age at baseline: Participants in the experimental group were somewhat older than participants in the control group: t(43.69) = 2.03, p = .048.

*p < .05.

Table 2.

Means and standard deviations on the process and outcome measures in the experimental and control group.

	Before TSST	After TSST	Follow-up
Dysfunctional thinking—depression
CCL: experimental	7.75 (8.12)	7.44 (9.22)	6.02 (8.03)
CCL: control	7.05 (5.55)	6.70 (6.42)	5.80 (5.28)
Dysfunctional thinking—anxiety
CCL: experimental	8.63 (6.87)	8.28 (7.40)	7.14 (6.14)
CCL: control	7.80 (5.69)	8.25 (6.23)	7.63 (5.90)
Cognitive therapy skills
CTS: experimental	.86 (.76)	NA	NA
CTS: control	.35 (.54)	NA	NA
Mood—positive affect
PANAS: experimental	24.05 (4.15)	24.51 (4.48)	25.45 (4.16)
PANAS: control	22.91 (3.68)	23.52 (4.86)	24.79 (3.60)
Mood—negative affect
PANAS: experimental	21.94 (3.65)	20.91 (4.13)	23.61 (3.85)
PANAS: control	20.62 (4.06)	20.45 (4.00)	22.91 (4.27)
State anxiety
STAI: experimental	33.73 (7.68)	48.26 (11.65)	NA
STAI: control	33.29 (6.04)	48.47 (10.00)	NA

Note. TSST: Trier social stress test; CCL = Cognition Checklist; CTS = Cognitive therapy skills; PANAS = Positive and Negative Affect Scale; NA = not applicable; STAI = State-Trait Anxiety Inventory. None of the differences between the procedures were significant. Note that cognitive therapy skills were only tested before the TSST (but after the skills acquisition procedure) and that state anxiety was not measured at follow-up.

Process measures

Cognitive therapy skill acquisition

The experimental group (M = .86, SD = .76) used significantly more techniques on the cognitive therapy skill test compared to the control group (M = .35, SD = .54), t (68) = 3.20, p < .01.

Dysfunctional beliefs

There were no differences between the experimental group and the control group in change in CCL scores over time (group by time interaction effects, respectively: F(2, 64) = .66; p = .52, F(2, 65) = 1.58; p = .21). Mean and standard deviations for each group can be found in Table 3. Cronbach’s α of the scale during the first measurement was .88. Inspection of the raw data showed no differences that point to clinical relevance.

Table 3.

Considerations for future experimental depression research.

1. Nature of the sample: Experimental depression studies that focus on therapy procedures should include a (sub)clinical depressed sample or induce dysfunctional thinking and mood to allow for change in outcomes and processes.

2. Distinguish between procedures, processes, and mechanisms of change: We suggest that mechanisms of change should be distinguished into treatment procedures and treatment processes. A treatment procedure is a procedure isolated from a complete treatment package that focuses on changing a process that is expected to lead to change in the outcome.

3. Personalize outcome measures: Experimental depression studies could benefit from using personalized outcome measures. First, instead of using standardized self-report questionnaire of cognitive change, change in personal, idiosyncratic dysfunctional beliefs should be measured. Second, research on cognitive therapy skills should test these skills in response to a test that directly challenges the participants to identify and cope with their own dysfunctional thoughts.

4. Distinguish between different levels of dysfunctional thinking: Future studies should investigate in what way cognitive procedures target the different levels of dysfunctional thinking: (1) automatic negative thoughts; (2) statements that include rules, expectations, attitudes, and assumptions; and (3) core structures (schemas).

5. Investigate the doses of cognitive procedures: Experimental depression studies should investigate whether different strengths or intensities of procedures lead to different outcomes in cognitive skills, cognitive change, and depression.

6. Compare cognitive change procedures with noncognitive change procedures: Comparing a cognitive change procedure with a noncognitive change procedure instead of no procedure enables testing whether the effects of a cognitive change procedure are specific for the cognitive change procedure. Showing that the mediator accounts for therapeutic change in one procedure but not in the other, will strengthen the argument that a particular mediator is specific for that procedure.

Outcome measures

State anxiety

The TSST significantly induced distress in both groups, but there were no differences between the experimental and control groups in change of scores on the STAI, time: F(1, 66) = 186.76 p < .001; group by time interaction effect: F(1, 66) = .09; p = .76. Cronbach’s α of the scale during the first measurement was .88.

Mood

There were no differences between the experimental and control group in change of positive and negative affect on the PANAS, group by time interaction effects, respectively: F(2, 66) = .17; p = .84, F(2, 66) = .36; p = .69. Inspection of the raw data showed no differences that point to clinical relevance. Cronbach’s α of the scale during the first measurement was .85 and .82 for the positive and negative scale, respectively.

Discussion

The present experiment was one of the first to examine whether it is feasible to isolate cognitive procedures that are part of the CBT protocol for depression and to investigate whether these would lead to direct changes in the hypothesized underlying processes and outcomes. Results showed that participants who received a CTSAP reported better cognitive therapy skills before attending the social stress test but did not show less distress, dysfunctional thinking, or better mood after the test compared to participants that did not receive the procedure.

Before we conclude that CTSAPs do not lead to cognitive change and (subsequent) depression change, we would like to present several alternative explanations that might explain the lack of significant results.

First, our procedure may not have been sufficiently powerful to activate cognitive change or changes in mood: Participants showed a difference in use of cognitive therapy skills, but the mean level of skill use was low (means below 1) for both groups. This means that, even in the experimental group, a part of the participants did not use any of the skills that were provided during cognitive skill acquisition. The clinical relevance of this difference is therefore limited. Also, although a previous study that showed that cognitive change or better mood may be achieved with very short procedures (i.e., in the study of Teasdale & Fennel, 1982, five depressed patients, already in treatment, showed cognitive change and reduced depression even after a short 30-min cognitive change procedure), this study (Teasdale & Fennel, 1982) used a sample of moderately to severe depressed patients.

Second, our measures might have been insensitive for measuring cognitive change or cognitive therapy skills. In other words, the automatic negative thoughts related to depression and anxiety that are measured with the CCL may not be applicable to each individual. Instead of measuring cognitive change with standardized self-report questionnaires as was done in the current experiment, another strategy may be to measure the participants’ belief in personal, idiosyncratic dysfunctional thinking and related feelings before and after the procedure and use these differences as an outcome measure. Moreover, a stronger test of cognitive therapy skills might be to directly test the use of cognitive therapy skills by asking participants to write down how they would cope with their own dysfunctional beliefs following the social stress test, instead of giving example scenario’s that include automatic negative thoughts and measuring them before the social stress test. In this way, future studies might be able to test whether a procedure focused on cognitive therapy skill acquisition leads to the development of cognitive therapy (CT) skills and decreases dysfunctional thinking that is unique to a given individual. In addition, it seems important to investigate under what conditions cognitive procedures might affect different levels of dysfunctional thinking as these can be distinguished into the following: (1) automatic negative thoughts (i.e., as were used in the present experiment) that can be described as negative statements that are linked to a specific negative situation (e.g., “I guess I will fail the exam.”), (2) statements that include rules, expectations, attitudes, and assumptions that are more general than automatic negative thoughts (e.g., “If I’m not perfect, nobody will like me.”), and (3) core cognitive structures, also called schemas (e.g., “I am a loser.”) (Beck et al., 1979; McBride, Farvolden, & Swallow, 2007). Whereas automatic negative thoughts are easily accessible, assumptions and attitudes are less malleable, and schemas are hypothesized to rely outside of overt awareness and to underlie the presence of the dysfunctional thoughts (McBride et al., 2007; Young, Klosko, & Weishaar, 2003). It is possible that different levels of dysfunctional thinking require a different dose of cognitive skill acquisition (“different strokes for different folks”) to lead to successful use of CT skills and the reduction of dysfunctional thinking.

Third, a sad mood induction might be better suited for investigating the effects of cognitive change procedures on dysfunctional thinking and depressed mood. Although stress is a phenomenon that is highly related to depression (Hammen, 2005, 2006), and the social stress test did lead to an increase in feelings of distress, in contrast to our expectations, it did not lead to higher levels of dysfunctional thinking or worsened mood in a healthy student sample. Future studies should investigate whether sad mood inductions can lead to an increase in idiosyncratic automatic negative thoughts, attitudes, and schemas in healthy samples or use (sub)clinical samples to allow for change in outcomes and processes.

Taken together, our pilot study leads to six important considerations for future experimental depression research, which are presented in Table 3.

Concluding remarks

Against the background that experimental studies on mechanisms of change in cognitive behavioral therapy for depression have been scarce and even lacking, the present pilot study might serve as a first step toward a more experimental approach to investigate mechanisms of cognitive change in the field of depression. We reported on the design and results of a hypothesis-driven experiment as well as on the pitfalls that came along with them. Before we can conclude that cognitive procedures do not lead to change in underlying processes and (subsequent) depression outcomes, more experimental research in this area is highly needed. Future experimental depression treatment research might benefit from the abovementioned recommendations.

Footnotes

Authors’ contributions

Sanne J. E. Bruijniks, Marit Sijbrandij, and Marcus J. H. Huibers contributed to the design. Sanne J. E. Bruijniks conducted the data collection. Sanne J. E. Bruijniks performed data analysis and wrote the manuscript. All authors read, contributed, and approved the final manuscript.

Ethics approval

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, and its most recent revision.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Appendix 1

References

Barber

J. P.

DeRubeis

R. J.

(1989). On second thought: Where the action is in cognitive therapy for depression. Cognitive Therapy and Research, 13, 441–457.

Barber

J. P.

DeRubeis

R. J.

(2001). Change in compensatory skills in cognitive therapy for depression. Journal of Psychotherapy Practice and Research, 10, 8–13.

Beck

A. T.

Rush

A. J.

Shaw

B. F.

Emery

(1979). Cognitive therapy of depression. New York, NY: Guilford.

Beevers

C. G.

Miller

I. W.

(2005). Unlinking negative cognition and symptoms of depression: Evidence of a specific treatment effect for cognitive therapy. Journal of Consulting and Clinical Psychology, 73, 68.

Birkett

M. A.

(2011). The trier social stress test protocol for inducing psychological stress. Journal of Visualized Experiments: JoVE, (56).

Boon

M. T. G.

Peeters

F. P. M. L.

(1999). Affectieve dimensies bij depressie en angst. Tijdschrift voor Psychiatrie, 41, 109–113.

Butler

A. C.

Chapman

J. E.

Forman

E. M.

Beck

A. T.

(2006). The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review, 26, 17–31.

Clark

D. M.

(2004). Developing new treatments: On the interplay between theories, experimental science and clinical innovation. Behaviour Research and Therapy, 42, 1089–1104.

Craighead

W. E.

Sheets

E. S.

Bjornsson

A. S

. (2005). Specificity and nonspecificity in psychotherapy. Clinical Psychology: Science and Practice, 12, 189–193.

10.

Craske

M. G.

(2016). Introduction to special issue: Experimental psychopathology approach to understanding and treating mental disorders. Behaviour Research and Therapy, 86, 1.

11.

Cristea

I. A.

Huibers

M. J.

David

Hollon

S. D.

Andersson

Cuijpers

(2015). The effects of cognitive behavior therapy for adult depression on dysfunctional thinking: A meta-analysis. Clinical Psychology Review, 42, 62–71.

12.

Cuijpers

Berking

Andersson

Quigley

Kleiboer

Dobson

K. S.

(2013). A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Canadian Journal of Psychiatry, 58, 376.

13.

Cuijpers

Karyotaki

Weitz

Andersson

Hollon

S. D.

van Straten

(2014). The effects of psychotherapies for major depression in adults on remission, recovery and improvement: A meta-analysis. Journal of Affective Disorders, 159, 118–126.

14.

Cuijpers

Smit

Bohlmeijer

Hollon

S. D.

Andersson

(2010). Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: Meta-analytic study of publication bias. The British Journal of Psychiatry, 196, 173–178.

15.

De Graaf

L. E.

Roelofs

Huibers

M. J.

(2009). Measuring dysfunctional attitudes in the general population: The dysfunctional attitude scale (form A) revised. Cognitive Therapy and Research, 33, 345–355.

16.

Gaab

Blättler

Menzi

Pabst

Stoyer

Ehlert

(2003). Randomized controlled evaluation of the effects of cognitive–behavioral stress management on cortisol responses to acute stress in healthy subjects. Psychoneuroendocrinology, 28, 767–779.

17.

Garratt

Ingram

R. E.

Rand

K. L.

Sawalani

(2007). Cognitive processes in cognitive therapy: Evaluation of the mechanisms of change in the treatment of depression. Clinical Psychology: Science and Practice, 14, 224–239.

18.

Hammen

(2005). Stress and depression. Annual Review of Clinical Psychology, 1, 293–319.

19.

Hammen

(2006). Stress generation in depression: Reflections on origins, research, and future directions. Journal of Clinical Psychology, 62, 1065–1082.

20.

Hollon

S. D.

Muñoz

R. F.

Barlow

D. H.

Beardslee

W. R.

Bell

C. C.

Bernal

… Sommers

. (2002). Psychosocial intervention development for the prevention and treatment of depression: Promoting innovation and increasing access. Biological Psychiatry, 52, 610–630.

21.

Huibers

M. J. H.

Cuijpers

(2015). Common (Nonspecific) Factors in Psychotherapy. The Encyclopedia of Clinical Psychology, 1–6.

22.

Hundt

N. E.

Mignogna

Underhill

Cully

J. A.

(2013). The relationship between use of CBT skills and depression treatment outcome: A theoretical and methodological review of the literature. Behavior Therapy, 44, 12–26.

23.

IBM Corp. (2012). IBM SPSS statistics for windows, version 21.0. Armonk, NY: IBM.

24.

Kazdin

A. E.

(2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27.

25.

Kazdin

A. E.

(2009). Understanding how and why psychotherapy leads to change. Psychotherapy Research, 19, 418–428.

26.

Kirschbaum

Pirke

K. M.

Hellhammer

D. H.

(1993). The ‘Trier Social Stress Test’–a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28, 76–81.

27.

Lissek

Powers

A. S.

McClure

E. B.

Phelps

E. A.

Woldehawariat

Grillon

Pine

D. S.

(2005). Classical fear conditioning in the anxiety disorders: A meta-analysis. Behaviour Research and Therapy, 43, 1391–1424.

28.

Lorenzo-Luaces

German

R. E.

DeRubeis

R. J.

(2014). It’s complicated: The relation between cognitive change procedures, cognitive change, and symptom change in cognitive therapy for depression. Clinical Psychology Review, 42, 3–15.

29.

MacLeod

Grafton

(2016). Anxiety-linked attentional bias and its modification: Illustrating the importance of distinguishing processes and procedures in experimental psychopathology research. Behaviour Research and Therapy, 86, 68–86.

30.

McBride

Farvolden

Swallow

S. R.

(2007). Major depressive disorder and cognitive schemas. In Riso

L. P.

du Toit

P. L.

Stein

D. J.

Young

J. E.

(Eds.), Cognitive schemas and core beliefs in psychological problems: A scientist-practitioner guide. Washington, D.C.: American Psychological Association.

31.

Mineka

Oehlberg

(2008). The relevance of recent developments in classical conditioning to understanding the etiology and maintenance of anxiety disorders. Acta Psychologica, 127, 567–580.

32.

Peeters

F. P. M. L.

Ponds

R. W. H. M.

Vermeeren

M. T. G.

(1996). Affectiviteit en zeltbeoordeling van depressie en angst. Tijdschrift voor psychiatrie, 38, 240–250.

33.

Quilty

L. C.

McBride

Bagby

R. M.

(2008). Evidence for the cognitive mediational model of cognitive behavioural therapy for depression. Psychological Medicine, 38, 1531–1541.

34.

Renner

Schwarz

Peters

M. L.

Huibers

M. J.

(2014). Effects of a best-possible-self mental imagery exercise on mood and dysfunctional attitudes. Psychiatry Research, 215, 105–110.

35.

Segal

Z. V.

Gemar

Williams

(1999). Differential cognitive response to a mood challenge following successful cognitive therapy or pharmacotherapy for unipolar depression. Journal of Abnormal Psychology, 108, 3.

36.

Segal

Z. V.

Kennedy

Gemar

Hood

Pedersen

Buis

(2006). Cognitive reactivity to sad mood provocation and the prediction of depressive relapse. Archives of General Psychiatry, 63, 749–755.

37.

Spielberger

C. D.

Gorsuch

R. L.

Lushene

R. E.

(1970). STAI. Manual for the state-trait anxiety inventory (self evaluation questionnaire). Palo Alto, CA: Consulting Psychologists Press.

38.

Steer

R. A.

Beck

A. T.

Clark

D. A.

Beck

J. S.

(1994). Psychometric properties of the cognition checklist with psychiatric outpatients and university students. Psychological Assessment, 6, 67.

39.

Strunk

D. R.

Brotman

M. A.

DeRubeis

R. J.

(2010). The process of change in cognitive therapy for depression: Predictors of early inter-session symptom gains. Behaviour Research and Therapy, 48, 599–606.

40.

Strunk

D. R.

DeRubeis

R. J.

Chiu

A. W.

Alvarez

(2007). Patients’ competence in and performance of cognitive therapy skills: Relation to the reduction of relapse risk following treatment for depression. Journal of Consulting and Clinical Psychology, 75, 523.

41.

Strunk

D. R.

Hollars

S. N.

Adler

A. D.

Goldstein

L. A.

Braun

J. D.

(2014). Assessing patients’ cognitive therapy skills: Initial evaluation of the competencies of cognitive therapy scale. Cognitive Therapy and Research, 38, 559–569.

42.

Teasdale

J. D.

Fennell

M. J.

(1982). Immediate effects on depression of cognitive therapy interventions. Cognitive Therapy and Research, 6, 343–351.

43.

Watson

Clark

L. A.

Tellegen

(1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063.

44.

Young

J. E.

Klosko

J. S.

Weishaar

M. E.

(2003). Schema therapy: A practitioner’s guide. New York, NY: Guilford Press.