The Dark-Side of Rule Governed Behavior

Abstract

The current research set out to investigate whether adolescents who self-reported high or low levels of depressive symptomatology would demonstrate differential sensitivity to changing contingencies as a function of accurate/inaccurate (Experiment 1) or pliance/tracking instructions (Experiment 2). Following a screening procedure, students were divided into two groups and instructed on how to respond during a simple learning task. In Experiment 1, we observed a characteristic set of outcomes that were contingent upon the type of instructional control provided and levels of depressive symptomatology reported. Whereas accurate instructions resulted in quick and efficient learning (schedule sensitivity) regardless of depressive symptomatology, inaccurate instructions lead to problematic rule-following in the high depressive symptom group. Experiment 2 revealed that schedule insensitivity effects can be further augmented when participants who report depressive symptoms are equipped with a set of superordinate pliance instructions. In contrast, students in the tracking condition showed increasing adaptation to the changing contingencies throughout the study.

Keywords

rule-governed behavior learning via instructions depression

The ability to generate and apply rules to our own behavior (as well as that of others) is a fundamental avenue through which humans adapt to the world around them. Privately or publically generated rules allow us to set and achieve goals (O’Hora & Maglieri, 2006), delay immediate gratification, and even deal with events before they occur (e.g., “Mow my lawn next month and I will pay you afterwards”; Doll, Jacobs, Sanfey, & Frank, 2009; Hayes, 1993). Rules or instructions allow us to respond to consequences that are extremely abstract in nature (e.g., “Only honest people go to Heaven”) as well as indirectly profit from other people’s experiences. For instance, a person can respond to the rule “If you drink bleach, you will die” without having to engage in the behavior of drinking bleach or of contacting the consequence of dying. More generally, rules such as moral principles, laws, commands, religious prescriptions, norms, and customs serve as the bedrock upon which many social and cultural groups are formed and function (Baumeister, 2008).

Nevertheless, it appears that the adaptive advantages afforded by rule-following come at a real and significant cost—rules have a dark side. Over four decades of research indicates that rules can undermine how sensitive we are to important changes in the world around us and produce undesirable consequences that could have otherwise been avoided (e.g., Baruch, Kanter, Busch, Richardson, & Barnes-Holmes, 2007; Cella, Dymond, & Cooper, 2009; Hayes, Brownstein, Haas, & Greenway, 1986; Hayes, Brownstein, Zettle, Rosenfarb, & Korn, 1986; Rosenfarb, Burker, Morris, & Cush, 1993; Shimoff, Catania, & Matthews, 1981). To illustrate, imagine that some individuals are provided with highly specific instructions on how they should respond during a subsequent learning task (e.g., “Press the button quickly when the light is on” or “Press the button slowly when the light is on”). Others are provided with no instructions so that they have to learn how to respond via trial-and-error. Thereafter, both groups are exposed to a learning task that reinforces low rates (LRs) of key pressing in the presence of one light and HR (high rate) of key pressing in the presence of another light. While specific procedural properties may vary from study to study, a characteristic set of outcomes tends to emerge. On one hand, participants who learn via instructions initially respond with greater speed and accuracy than their uninstructed counterparts. On the other hand, when the above contingencies are reversed those same participants tend to rigidly adhere to previously learned instructions even when doing so places them into contact with an aversive consequence (e.g., loss of money). For example, participants who are instructed to press quickly often fail to adapt their behavior when slow rates of responding are reinforced and vice versa. Thus, it appears that a previously effective rule for responding can quickly come to undermine an individual’s ability to contact important changes in the world around them.

Over the past decade, a growing number of researchers suggest that this capacity to become “locked into” or “stuck” in problematic patterns of rule-following is a characteristic feature of psychopathologies such as addiction (“I need to smoke in order to feel good”), self-harm (“I always cut myself when I do poorly at school”), suicide (“My pain will stop after I kill myself”), and schizophrenia (Monestes, Villatte, Stewart, & Loas, in press; see also Baruch et al., 2007; Luoma, Kohlenberg, Hayes, Bunting, & Rye, 2008; Törneke, Luciano, & Valdivia-Salas, 2008). For example, it may be that gamblers following the rule “My bad luck is bound to change” continue to bet despite the aversive outcomes that result from following that rule (i.e., losing increasing sums of money; Dixon, Hayes, & Aban, 2000). Likewise, a person addicted to alcohol may emit the rule “I will feel better after drinking” and this may be effective in the short run. But when this rule persists over time, drinking continues, social and inter-personal problems fail to go away, and thoughts and feelings about poor life outcomes may actually increase (see Törneke et al., 2008). In other words, a rigid adherence to certain types of rules may have persistent and problematic effects on human behavior. Deploying these rules to avoid or escape aversive thoughts, feelings, and memories may be “successful” for a brief period of time. But in the long run, problematic thoughts and feelings paradoxically increase as contact with the wider environment decreases (see Fletcher & Hayes, 2005; Hayes, Strosahl, & Wilson, 1999). In what follows, we turn our attention to problematic rule-following and its role in a complex and common clinical phenomenon—depression.¹

The Impact of Rule-Following in Depression

A number of clinical theories share the assumption that deficits or inaccuracies in rule-governed behavior play a central role in the development and maintenance of psychopathologies such as depression and anxiety (see Dobson & Dozois, 2010; Hayes et al., 1999). For instance, traditional variants of Cognitive Behavioral Therapy (CBT) implicate specific types of rules (e.g., “cognitive distortions or appraisals”) in depression and argue that a reduction in depressive symptomatology may be achieved through the elimination of such rules and their replacement (via cognitive restructuring) with more accurate rules (e.g., “balanced thoughts”; Beck, 2005; see also Hollon, Stewart, & Strunk, 2006). Others from a more contextual wing of CBT known as Acceptance and Commitment Therapy (ACT) emphasize the need to alter how self- and socially generated rules are experienced to promote desired behavior change and ultimately valued action. From this perspective, “attempts to control unwanted subjective experiences may not only be ineffective but even counterproductive, insofar as they can actually result in a net increase in distress, result in significant psychological costs, or both” (Forman, Herbert, Moitra, Yeomans, & Geller, 2007, p. 775).

Although the theoretical literature linking problematic rule-following to depression has gained momentum in recent years, only a handful of studies have actually sought to submit these assumptions to empirical testing (at least within the contextual behavioral tradition; for a review, see Kanter, Busch, Weeks, & Landes, 2008). In one such study, Rosenfarb and colleagues (1993) exposed a group of depressed and non-depressed individuals to a learning task similar to the one outlined above (i.e., two sets of alternating contingencies that differentially reinforced HRs or LRs of responding). Half of the high and low depression groups were administered instructions that accurately mapped onto the experimental contingencies operating within the first training session. However, these instructions were rendered inaccurate when the contingencies were reversed during the second section of the study (the other half of the participants always had their behavior shaped via trial-and-error learning). Interestingly, the authors found that non-depressed individuals in the rule-governed condition failed to successfully adapt to the changing contingencies whereas their counterparts in the two depressed groups (and the non-depressed contingency-shaped group) managed to do so. In other words, instead of demonstrating a maladaptive adherence to instructions that conflicted with task requirements, depressed participants appeared to abandon inaccurate rules and respond on the basis of the recently reversed contingencies.

More recently, Baruch et al. (2007) examined problematic rule-following in a dysphoric and non-dysphoric undergraduate population. These authors were also interested in whether sensitivity to environmental contingencies could be moderated by different types of instructions known as pliance and tracking (for a detailed overview of these concepts, see Hayes, Barnes-Holmes, & Roche, 2001). On one hand, pliance can be defined as rule-governed behavior that is under the control of consequences mediated by the speaker for a correspondence between the rule and the behavior of interest (Hayes, Zettle, & Rosenfarb, 1989; Törneke et al., 2008; Zettle & Hayes, 1982). This type of rule-following refers to instances where compliance with a rule or instruction is reinforced by the person who delivers the rule (e.g., when a child cleans his or her bedroom after being told by a parent that “You will only get pocket-money once your chores are complete”). On the other hand, tracking can be defined as an instance of rule-governed behavior that is under the control of a correspondence between the rule and the way the world is arranged (Törneke et al., 2008). In this latter case, the consequences of rule-following are not contingent upon coordination between behavior and the specified rule. Rather, rule-following is reinforced by accessing the reinforcer specified by the rule itself (e.g., enjoying a clean room after being told that “Cleaning your room will make you feel great”).

Prior to completing the learning task, students in the Baruch et al. study were asked to select instructions from a container and either read them aloud to the researcher (pliance) or privately to themselves (tracking). Participants in the pliance condition were also informed that the researcher would carefully check their performance at the end of every training session and that they should respond in a particular way during the task. Their counterparts in the tracking condition were simply instructed to try to earn as many points as possible. The authors found that dysphoric students were more sensitive to the contingencies operating within the learning task and less sensitive to ineffective rules compared with their non-dysphoric peers. However, this effect was observed only in the tracking and not in the pliance condition. In line with Rosenfarb et al. (1993), instructions only led to insensitivity effects for non-dysphoric participants (note that a direct comparison between the two studies is difficult given that Baruch et al. did not include a contingency-shaped condition with which the rule-governed condition could be compared).

Before drawing strong conclusions from the above work, several points are worth noting. First, Baruch et al. (2007) and Rosenfarb et al. (1993) both employed an exclusively female undergraduate sample. Thus, it remains to be seen whether males reporting high and low levels of depressive symptomatology would also fall prey to maladaptive patterns of rule-following as well. Given that prevalence and incidence of, as well as morbidity risk for, depressive disorders are generally higher in females relative to males (Nolen-Hoeksema, 2001; Piccinelli & Wilkinson, 2000; Van de Velde, Bracke, Levecque, & Meuleman, 2010), it could be that those insensitivity effects observed in the literature are specific to, or more elaborated in, one gender relative to the other. Second, Baruch and colleagues reinforced alternating rates of responding with access to raffle tickets that could be exchanged for a large yet uncertain sum of money. Given that there was no guarantee that their task performance would result in such a reward, participants may not have placed much value on the consequences for responding. In effect, non-dysphoric participants may not have been insensitive to the contingencies per se, but rather that raffle tickets failed to function as effective reinforcers for this group.

Experiment 1

With the above in mind, Experiment 1 set out to determine whether adolescent males who self-reported high or low levels of depressive symptomatology would show divergent patterns of responding when a set of rules and contingencies were put into competition with one another. Following an initial screening procedure, students were divided into two groups (high vs. low levels of depressive symptoms). Thereafter, half of the participants received accurate instructions on how to respond during a subsequent learning task while the other half received inaccurate instructions that were designed to directly undermine their task performance. Both groups were then exposed to alternating reinforcement contingencies that aimed to establish either HRs or LRs of responding. If excessive rule-following is a functional characteristic of depressive symptomatology, then a number of testable predictions should follow. First, participants in the high symptom group should demonstrate larger insensitivity effects when they are provided with inaccurate (relative to accurate) instructions. Second, participants in the low symptom group should be less susceptible to excessive rule-following and should abandon inaccurate rules with greater speed than those in the high symptom/inaccurate instructions condition. Furthermore, they should also demonstrate increasing sensitivity to changing contingencies across successive blocks of trials. Finally, given that accurate instructions perfectly correlate with the learning task’s alternating contingencies of reinforcement, we do not expect any differences between the high and low depressive symptoms groups when they both receive accurate instruction for responding. However, if a maladaptive adherence to rules is not a keystone of depressive symptomatology, then we should obtain broadly comparable findings with those noted by Rosenfarb et al. (1993) and Baruch et al. (2007).

Method

Participants

To determine eligibility for the study, 172 adolescent males ranging from 15 to 17 years were administered the Inventory for Depressive Symptomatology–Self-Reported (IDS-SR; Rush et al., 1986) as a screening measure. Using recommended cutoff scores (see Rush et al., 2003), the final sample was divided into two groups: a low depressive (IDS scores ≤ 13) and a high depressive symptom group (IDS scores ≥ 41), each consisting of 20 second-level students who agreed to participate in exchange for a small monetary reward. The experiment consisted of a 2 (Instructions: accurate vs. inaccurate) × 2 (Student-Type: high vs. low levels of depressive symptoms) design with both variables manipulated between participants.

Measures

IDS

All participants completed the original version of the IDS-SR consisting of 28 equally weighed items, rated on a 4-point scale (range = 0-3). Items are summed to create a standard total score ranging from 0 to 84, with higher values reflecting increased symptom severity. The IDS-SR has been found to correlate highly with other measures of depressive symptomatology (Rush, Gullion, Basco, Jarrett, & Trivedi, 1996), has acceptable psychometric properties and high internal consistencies (see Rush, Carmody, & Reimitz, 2000 ).

Procedure

Upon arriving at the laboratory, participants were welcomed by the researcher, registered their informed consent, and were seated in front of a computer. They were subsequently informed that they would participate in a short learning task where they could win a small amount of money based on their performance. The experiment took approximately 2 hr to complete.

Overall, the learning phase consisted of three consecutive blocks, each comprised of 16 trials that sought to establish alternating patterns of HRs and LRs of responding. Within each block, participants encountered eight HR and eight LR trials. HR trials provided them with a point every time 36 successive responses were emitted. This schedule of reinforcement was designed to produce high, steady rates of responding with only a brief pause after the delivery of a reinforcer. LR trials provided participants with access to a point if they emitted a single response within an 8 s window. This schedule of reinforcement was designed to produce low, steady rates of responding by imposing a minimum inter-response time. Both HR and LR trials were randomly presented every 2 min with the result that each block of 16 trials required a total of 32 min to complete.

To investigate adaptive versus maladaptive rule-following, we randomly assigned 10 participants who reported high levels and 10 participants who reported low levels of depressive symptoms to the accurate instructions condition. In this case, participants were provided with set of instructions that allowed them to accurately track the alternating response contingencies. For instance, on HR trials, instructions appeared in the middle of the screen stating that they should “Press fast” or “Press fast to earn points.” When those same participants encountered a LR trial on-screen instructions indicated that they should “Press slowly” or “Press slowly to earn points.” Another 10 participants reporting high and 10 reporting low levels of depressive symptoms were randomly assigned to the inaccurate instructions condition. That is, instructions were provided that directly contradicted the reinforcement contingencies mentioned above (i.e., they were told to “Press slowly” or “Press slowly to earn points” on HR trials and “Press fast” or “Press fast to earn points” on LR trials). A 3 min interval followed each consecutive session and participants were encouraged to take a break at these times. Thereafter, the next session of learning began. Following the completion of three successive training sessions, participants were thanked, debriefed, and dismissed.

Results

IDS scores obtained from the high depressive symptoms group were comparable for participants who received accurate (M = 32.2) and inaccurate instructions (M = 36.2). Likewise, scores obtained from the low depressive symptoms group were comparable for participants who received accurate (M = 7.1) and inaccurate instructions (M = 8.7). For each participant, we calculated six scores based on the mean number of responses they emitted during HR and LR trials on Sessions 1, 2, and 3 (see Figure 1). As can be seen in the graph, participants who were provided with accurate instructions produced schedule sensitive performances across all three training sessions - regardless of their depressive symptomatology (i.e., they showed HRs of responding during HR trials and LRs of responding during LR trials). Interestingly, however, inaccurate instructions that directly contradicted the experimental contingencies resulted in different outcomes depending on self-reported depressive symptomatology. On one hand, the behavior of participants reporting low levels of depressive symptoms appeared to be under the control of inaccurate instructions during the first session—with LRs of responding during HR trials and HRs of responding during LR trials. By the second and third sessions, however, they demonstrated a breakdown in maladaptive rule-following (i.e., a reversal in their HR and LR response patterns). On the other hand, students reporting high levels of depressive symptoms demonstrated excessive rule-following across all three experimental sessions. They responded slowly during HR trials and quickly during LR trials despite the aversive consequences that such responding occasioned (see Table 1).

Figure 1.

Mean number of responses emitted during blocks of HR and LR trials for accurate and inaccurate instruction conditions as a function of depressive symptomatology (high vs. low).

Table 1.

Overall Means and Standard Deviations for High Rate and Low Rate Experimental Sessions as a Function of Student-Type (High vs. Low Levels of Depressive Symptoms) and Instruction-Type (Accurate vs. Inaccurate).

Student and instruction type	High rate	Low rate
Dysphoric (Accurate)	587.5 (113.4)	31.8 (6.6)
Dysphoric (Inaccurate)	65.2 (32.8)	429.7 (183.8)
Non-dysphoric (Accurate)	607.3 (99.9)	30.6 (12.7)
Non-dysphoric (Inaccurate)	403 (173.8)	282.5 (269)

To investigate whether self-reported depressive symptomatology influenced rigid rule-following, we submitted the above data to a 6 (Contingencies: HR 1, 2, 3 vs. LR 1, 2, 3) × 2 (Student-Type: high vs. low) × 2 (Instruction-Type: accurate vs. inaccurate) repeated-measures ANOVA. Analyses revealed a significant main effect for Contingency, F(5, 36) = 37.4, p = .001, $η_{p}^{2}$ = .51, as well as a two-way interaction between Contingency and Student-Type, F(5, 36) = 12.4, p = .001, $η_{p}^{2}$ = .26; Contingency and Instruction-Type, F(5, 36) = 77.2, p = .001, $η_{p}^{2}$ = .68; and perhaps most importantly in the current context, a three-way interaction between Contingency, Student-Type, and Instruction-Type, F(5, 36) = 10.4, p = .001, $η_{p}^{2}$ = .26. To specify the obtained three-way interaction, the effects of instructions and reinforcement contingencies were assessed separately for participants with high and low levels of self-reported depressive symptoms.

With respect to the high symptom group, a significant main effect of Contingency was obtained, F(5, 18) = 6.5, p = .001, $η_{p}^{2}$ = .27, which was qualified by a two-way interaction with Instruction-Type, F(5, 18) = 126.8, p = .001, $η_{p}^{2}$ = .88. When participants in the accurate condition encountered HR trials, they responded, on average, at a higher rate relative to when they encountered LR trials, t(9) = 15.5, p = .001. This pattern was reversed for the inaccurate condition who produced—on average—a lower rate of responding on HR relative to LR trials, t(9) = −6.6, p = .001. In other words, the high depressive symptoms group demonstrated clear evidence of maladaptive rule-following, even when those rules were in direct conflict with the reinforcement contingencies available during the learning task.

With respect to the low depressive symptoms group, a significant main effect of Contingency was also obtained, F(5, 18) = 31.6, p = .001, $η_{p}^{2}$ = .64, and this was once again qualified by a two-way interaction with Instruction-Type, F(5, 18) = 13.5, p = .001, $η_{p}^{2}$ = .43. Similar to before, participants with low levels of depressive symptoms who received accurate instructions typically responded at a higher rate on HR relative to LR trials, t(9) = 19.8, p = .001. Moreover, when that same group received inaccurate instructions, they tended to produce higher rates of responding on the HR compared with LR trials, although this difference failed to reach significance (p = .3). Upon closer inspection, it appears that while there was no significant difference between mean rates of responding during HR and LR trials in Session 1 (M = 200.8 vs. 365) and Session 2 (M = 422.5 vs. 136.2), a breakdown in rule-following was evident in Session 3 (M = 585.6 vs. 166.1), t(9) = 3.5, p = .007, such that behavior increasingly came under the control of reinforcement contingencies rather than the inaccurate instructions provided.

Discussion

Overall, our analyses support the idea that a tendency to excessively adhere to maladaptive rules at the expense of effective contact with environmental contingencies may be a functional response pattern characteristic of depressive symptomatology. In Experiment 1, we found that when participants with high levels of (self-reported) depressive symptoms were provided with inaccurate instructions, they showed evidence of persistent rule-following, such that they failed to adapt their behavior to the two reinforcement contingencies operating across three experimental sessions. When participants with low levels of (self-reported) depressive symptoms were provided with inaccurate instructions, they also demonstrated evidence of problematic rule-following. However, they successfully adapted their behavior to the reinforcement contingencies across each successive block of training. Thus, it appears that when a set of instructions directly contradicted the way the environment was arranged, only the low depressive symptoms group abandoned those instructions and adapted to the changing parameters of the experiment.

Interestingly, our results differ dramatically from the outcomes obtained by Rosenfarb et al. (1993) and Baruch et al. (2007) who found that dysphoric individuals demonstrated greater schedule sensitivity and less rule-governed behavior compared with their non-dysphoric counterparts. When reflecting on these findings, it should be noted that the current work differed from the above studies in three important ways. First, while we manipulated the accuracy of instructions between participants, the above studies manipulated the accuracy of instructions within participants (so that previously accurate instructions became inaccurate after a specific number of trials). Second, whereas Baruch and colleagues focused their attention on the interaction between pliance/tracking and accurate/inaccurate instructions, we only examined how instructional accuracy impacts sensitivity to environmental contingencies—regardless of second-order instructional control. Finally, while we employed a simple HR/LR of responding manipulation, Baruch et al. also exposed their participants to a conditional discrimination task known as Matching-to-Sample (MTS).

Experiment 2

With the above in mind, we set out to replicate our previous findings while controlling for several factors that may explain why the current work departs from the existing literature. Similar to before, participants were exposed to a learning task comprised of two alternating sets of contingencies and equipped with either accurate or inaccurate instructions. This time, however, we manipulated instructional accuracy during the task itself so that a set of rules for responding were initially effective but subsequently rendered ineffective. In line with Baruch et al. (2007), we also examined whether schedule sensitivity could be manipulated by providing participants with pliance or tracking instructions that specified different consequences for responding (i.e., second-order instructional control). Finally, we exchanged the HRs and LRs of responding task for an MTS procedure that required participants to relate stimuli on the basis of their similarity or difference from one another.

If excessive rule-following at the expense of effective contact with environmental contingencies is a functional property of depressive symptomatology, then a similar set of outcomes should be observed as in Experiment 1 (i.e., quick adaptation by participants displaying low rather than high levels of depressive symptoms). This effect may be augmented in the pliance condition given that people are continuously reinforced for following rules within the wider social environment (Hayes et al., 1986). If this assumption is correct, and adolescents do have an extensive history of reinforcement for compliance with the instructions of perceived authority figures (e.g., teachers, doctors, police), then it may be that they will continue to follow experimental instructions, even when those instructions are at variance with, or contradict, contingencies of reinforcement operating within the learning task itself (Milgram, 1974). In contrast, a superordinate set of tracking instructions (e.g., “try to earn as many points as possible”) may lead participants to abandon specific task instructions when the latter directly undermine the ability to earn as many points as possible (i.e., tracking should lead to a reduction in the insensitivity effect).

Method

Participants

Two hundred thirty-three adolescent males ranging from 15 to 17 years were administered the IDS as a screening measure. Applying the same set of cutoff scores to this sample as used in Experiment 1 resulted in 51 individuals who were randomly selected to take part in a pre-experimental MTS task to determine whether they were capable of producing the conditional discriminations that would be required in the experiment proper. Based on their performance during the selection phase (see below), 18 individuals who reported high (IDS scores ≥ 41) and low levels of depressive symptoms (IDS scores ≤ 13) were randomly assigned to two different instruction conditions. The study consisted of a 2 (Instruction-Type: pliance vs. tracking) × 2 (Student-Type: high vs. low depressive symptoms) factorial design with both factors manipulated between participants.

Materials

The stimuli employed in the experiment consisted of either letter sets (e.g., “P-,” “-PP,” “P-P”) or nonsense syllables (e.g., “KAC,” “KUC,” “LAC”) presented in uppercase format. Stimuli were presented in black uppercase font against a white screen and were 9 cm wide and 8 cm high. Participants responded using the spacebar and were seated at a distance of approximately 50 cm from a computer screen. Both the learning task and recording of responses were controlled by a personal computer.

Procedure

Overall, the study consisted of three experimental phases that were completed in the following order; a selection phase, provision of (pliance or tracking) instructions followed by the MTS task.

Selection phase

Upon arriving at the laboratory, participants were seated in front of a computer and informed that a number of symbols (e.g., “XY,” “XZ,” and “XX”) or nonsense words (e.g., “SAG,” “SUG,” and “SIG”) would appear on the screen. One of these symbols (sample stimulus) appeared at the upper middle portion of the screen while three other symbols (comparison stimuli) appeared on the lower left, middle, and right of sections of the screen. Their task was to select the comparison stimulus at the bottom of the screen that they considered to be most like the sample stimulus at the top of the screen. If the correct comparison stimulus was selected in the presence of a given sample stimulus, the computer awarded the participant a point while an incorrect response resulted in the loss of a point. As participants continued to gain or lose points across the 10 training trials, a total score was printed on the upper right section of the screen. From those participants who choose the correct comparison stimulus on 8 or more trials, 36 were randomly selected to continue to the next phase of the study. Nine participants with high and nine participants with low levels of depressive symptoms were assigned to the tracking condition while a further nine participants in the high and nine in the low depressive symptoms groups were assigned to the pliance condition. The remaining students were thanked, debriefed, and dismissed.

Pre-task instructions

Following the selection phase, participants were presented with a cloth bag that supposedly contained different types of instructions that they could use to guide their behavior later on in the experiment. Unknown to them, the bag contained an identical set of paper slips stating that they should “Select the comparison stimulus most like a given sample stimulus.” On one hand, participants in the pliance condition were asked to read their instructions aloud in the presence of the experimenter and then return their slip of paper to the cloth bag. On-screen instructions then informed them that they would complete a task similar to the one they had previously encountered. Once again, a point could be earned if the correct stimulus was selected from the available options while a point would be lost if an incorrect stimulus was selected. Participants in the pliance condition were also informed that “I (the researcher) will carefully check your performance at the end of each and every training session” and that “I want you to select the comparison stimulus most like the sample stimulus at the top of the screen,” thereby introducing a verbally implied social contingency into the experimental task. On the other hand, participants in the tracking condition were also asked to select a slip of paper from the cloth bag. Unlike their counterparts in the pliance condition, however, they were simply asked to read the instruction privately before returning it to the bag. From their perspective, the researcher did not know what instruction they had picked. While on-screen instructions informed participants about the consequences of responding (i.e., that they would gain or lose points), there was no indication that their performance would be monitored or that they should respond in accordance with the researcher’s desires. Rather, it was suggested that they “try to earn as many points as possible” as each point could be exchanged for money at the end of the experiment.

Learning phase

After receiving the above instructions, participants were immediately administered the MTS procedure. This task consisted of 4 blocks that each contained 40 trials. Within each block of trials, participants encountered stimuli that were similar but not identical to those used in the earlier selection phase. During the first 2 blocks, participants were provided with accurate instructions that corresponded to the reinforcement contingencies (i.e., they were instructed to select the comparison stimulus that was most like a given sample stimulus). Once again, selecting the correct comparison stimulus from the available options resulted in the on-screen delivery of a point, the total score being incremented, followed by the removal of all stimuli, a 2,000 ms inter-trial interval and the onset of the next trial. However, selecting one of the two incorrect comparison stimuli resulted in the on-screen removal of a point and the deduction of that point from their overall score. All stimuli were then removed from the screen followed by an inter-trial interval and the subsequent trial. During the second two blocks of trials, the scheduled contingencies were reversed such that access to points was now made contingent upon the selection of the comparison stimulus that was least like a given sample stimulus. Critically, the instructions still specified that participants should select the comparison stimulus that was most like a given sample stimulus—thus rendering those instructions inaccurate.

At the end of every block, participants were instructed to report to the experimenter and a 3 min break was administered. Following the completion of four consecutive blocks of trials, participants were thanked, debriefed, and dismissed.

Results

For each participant, we calculated four scores based on the mean number of contingency sensitive responses they emitted during each block of MTS trials (see Table 2). We defined a response as “contingency sensitive” if participants selected the comparison stimulus that was most like a sample stimulus during Blocks 1 and 2 or least like a sample stimulus during Blocks 3 and 4. To aid interpretation, we then converted these raw scores into percentage accuracy values for each block of trials (see Figure 2). As can be seen in the graph, providing students with different types of instructions (pliance vs. tracking) appears to differentially influence their sensitivity to changing contingencies dependent on their self-reported depressive symptomatology. When participants with high and low levels of depressive symptoms encountered instructions that corresponded to experimental contingencies, they responded accurately. That is, they showed evidence of adaptive rule-following. However, when those contingencies were reversed and the instructions held constant—so that they now directly undermined access to a reinforcer—two distinct outcomes were obtained. On one hand, when participants with high levels of depressive symptoms were provided with pliance instructions (Figure 2, left panel), they continue to respond in accordance with the previously correct instructions. This pattern of maladaptive rule-following appeared to persist even though it undermined access to appetitive events in the environment. This did not appear to be the case for participants with high levels of depressive symptoms in the tracking condition (Figure 2, right panel), who showed evidence of adaptation to the reversed contingencies. On the other hand, the behavior of individuals with low levels of depressive symptoms appears to be controlled by task rules during the first two blocks and the experimental contingencies during the final two blocks in both the pliance and tracking conditions.

Table 2.

Mean and Standard Deviations for Experimental Sessions as a Function of Student-Type (Dysphoric vs. Non-Dysphoric) and Instruction-Type (Pliance vs. Tracking).

Student and instruction type	Block 1	Block 2	Block 3	Block 4
Dysphoric (Pliance)	39.7 (.5)	39.7 (.7)	4.1 (8.6)	7.8 (15.3)
Dysphoric (Tracking)	39.9 (.3)	39.9 (.3)	25.6 (12.4)	32.1 (10.5)
Non-dysphoric (Pliance)	40 (0)	40 (0)	16.3 (15.2)	28.8 (15.9)
Non-dysphoric (Tracking)	38.9 (2.1)	40 (0)	24.78 (12.3)	34.6 (4.3)

Figure 2.

Percentage of contingency sensitive responses made by participants who received pliance or tracking instructions as a function of depressive symptomatology (high vs. low).

To determine whether adolescents with different levels of depressive symptoms were differentially sensitive to certain types of instructions, we submitted the above data to a 4 (Contingency: Block 1, 2, 3 vs. 4) × 2 (Instruction-Type: pliance vs. tracking) × 2 (Student-Type; high vs. low) repeated-measures ANOVA, with both factors measured between participants. Analyses revealed a significant main effect for Contingency, F(3, 32) = 71.2, p = .001, $η_{p}^{2}$ = .69; a two-way interaction between Contingency and Student-Type, F(3, 32) = 4.7, p = .004, $η_{p}^{2}$ = .13, Contingency and Instruction-Type, F(3, 32) = 11.5, p = .001, $η_{p}^{2}$ = .26; as well as a three-way interaction between Contingency, Student-Type, and Instruction-Type, F(3, 32) = 3, p = .03, $η_{p}^{2}$ = .09. To specify the obtained three-way interaction, the influence of instructions on MTS performance was assessed separately for the two student groups.

With respect to the high depressive symptoms group, a significant main effect of Contingency was obtained, F(3, 16) = 58.1, p = .001, $η_{p}^{2}$ = .78, which was qualified by a two-way interaction with Instruction-Type, F(3, 16) = 14.6, p = .001, $η_{p}^{2}$ = .48. Follow-up one-way ANOVAs indicated that when task instructions accurately mapped onto the reinforcement contingencies (Blocks 1 and 2), participants responded with a high degree of accuracy—regardless of whether they had earlier received pliance or tracking instructions. However, an important difference did emerge between these groups when reinforcement contingencies were reversed so that instructions now conflicted with task requirements. For instance, during Block 3, participants in the pliance condition appeared to rigidly adhere to the previously correct instructions while their counterparts in the tracking condition showed evidence of adaptation to the reversed contingencies—The difference between the two groups was significant, F(1, 17) = 18.1, p = .001, $η_{p}^{2}$ = .53. A similar pattern of responding also emerged during Block 4, such that providing participants with high levels of depressive symptoms with pliance instructions led to problematic patterns of rule-following while their counterparts in the tracking condition appeared to abandon those rules and come under the control of the experimental contingencies, F(1, 17) = 15.6, p = .001, $η_{p}^{2}$ = .49.

With respect to the low depressive symptoms group, a significant main effect of Contingency was also obtained, F(3, 16) = 22.1, p = .001, $η_{p}^{2}$ = .58; however, this did not interact with Instruction-Type (p = .3). Participants in the pliance and tracking conditions both responded with a high degree of accuracy when task instructions corresponded to the contingencies of reinforcement (Blocks 1 and 2). In addition, both groups showed evidence of a breakdown in rule-following when the contingencies were reversed and instructions held constant. During Block 3, for instance, participants in the pliance and tracking conditions quickly adapted to the reversed contingencies and discarded inaccurate instructions as the basis for responding (no significant difference emerged between the two groups; p = .2). This sensitivity to experimental contingencies was even stronger in Block 4 for both the pliance and tracking conditions (once again, there was no difference between the two groups; p = .3).

Discussion

The results from Experiment 2 extend our earlier findings and indicate that rule-following can in some instances distort and even override contact with important contingencies in the world around us. When participants were equipped with instructions that enabled them to respond accurately on a simple learning task (MTS), they demonstrated evidence of adaptive rule-following (i.e., they selected the correct comparison stimulus in the presence of a given sample stimulus). However, sensitivity to a reversal in reinforcement contingencies appeared to be moderated by an interaction between self-reported depressive symptomatology and prior exposure to a set of superordinate (pliance or tracking) instructions. In particular, we found that when the high depressive symptoms groups were provided with pliance instructions, they were significantly less likely to adopt a new rule for responding (e.g., select the comparison stimulus least like the sample stimulus) when they had been previously reinforced for following another rule (e.g., selecting the comparison stimulus most like the sample stimulus). Unlike their counterparts in the tracking or either of the low depressive symptoms conditions—who modified their behavior in accordance with the changing contingencies—the high depressive symptoms/pliance condition continued to adhere to inaccurate instructions even when such behavior produced undesirable consequences that could have otherwise been avoided (i.e., loss of money).

General Discussion

Over the past two decades, a number of researchers have argued that excessive or problematic rule-following aimed at reducing contact with aversive thoughts, feelings, and memories is a core feature of clinical phenomenon such as depression, anxiety, self-harm, and suicide (Hayes et al., 1999). To submit this assumption to empirical testing, we exposed a group of adolescent males with high or low levels of self-reported depressive symptoms to a simple learning task and provided them with accurate/inaccurate instructions (Experiment 1) or pliance/tracking instructions (Experiment 2). Across two separate studies, we observed a characteristic set of outcomes that were contingent upon the type of instruction provided, levels of depressive symptomatology reported, and the correspondence between a specific instruction and contingencies of reinforcement operating within a learning task. For instance, when a set of instructions signaled the correct rate or type of response to be emitted, participants immediately responded with a high degree of accuracy—regardless of their depressive symptomatology or prior exposure to pliance/tracking instructions. Yet when participants were provided with inaccurate instructions (or the response contingencies were reversed so that previously correct instructions were now rendered incorrect), rule-based insensitivity effects were obtained. Consistent with previous findings in the non-clinical domain (e.g., Hayes et al., 1986), we found that inaccurate instructions undermined contact with important consequences available in the wider environment. However, in our study, this effect was moderated by self-reported depressive symptomatology. In Experiment 1, for instance, individuals with high levels of depressive symptoms showed evidence of persistent and problematic rule-following throughout the learning task while their low depressive symptoms counterparts demonstrated a breakdown in maladaptive rule-following across successive blocks of training trials. The findings from Experiment 2 indicate that this insensitivity effect can be augmented by providing participants with a superordinate set of instructions that emphasize the importance of complying with other instructions later in the study (pliance). This effect can also be eliminated by emphasizing the importance of contacting natural reinforcers available in the environment (tracking). Once again, we found that the influence of pliance or tracking varied as a function of depressive symptomatology. Only the high depressive symptoms group who received pliance instructions adhered to inaccurate instructions throughout the experiment while their high and low depressive symptoms counterparts in the tracking condition did not.

In short, it seems that learning via instructions represents a double-edged sword where humans are concerned. On one hand, accurate instructions can quickly and efficiently increase the rate at which we adapt to a changing world, placing us into contact with important consequences for responding. On the other hand, inaccurate instructions are often “sticky” insofar as they actively impede our ability to contact changing contingencies and the consequences that they afford. The current work suggests that this maladaptive effect of excessive rule-following can be augmented (pliance) or diminished (tracking) when a form of second-order instructional control is introduced and this is particularly evident for individuals reporting high levels of depressive symptomatology.

Interestingly, our findings represent a significant departure from those of Rosenfarb et al. (1993) and Baruch et al. (2007) who both found that dysphoric individuals demonstrated greater sensitivity to environmental contingencies and less rule-governed behavior compared with their non-dysphoric counterparts. In addition, and unlike the current work, Baruch et al. reported no moderating effect of pliance versus tracking on schedule sensitivity. A number of possible explanations present themselves. First and foremost, between-study variations in sampling and screening methods could have played an important role in the differential patterns of rule-following observed both here and elsewhere in the literature. Whereas Baruch et al. utilized clinical cutoffs on the Beck Depression Inventory (BDI) to identify depressed and non-depressed participants, and Rosenfarb et al. employed a multiple gating procedure that involved clinical screening and follow-up diagnostic interviews, we utilized an entirely different inventory (IDS) to classify participants as a function of their self-reported depressive symptomatology. Second, both of the former studies relied on an exclusively female undergraduate sample while we recruited a group of adolescent males. Both gender and developmental factors need to be considered when interpreting any differences between existing studies, especially given that (a) rule-governed behavior may have different developmental implications during childhood, adolescence, and adulthood while (b) depression also has different developmental trajectories for males and females across the life span (e.g., Essau, Lewinsohn, Seeley, & Sasagawa, 2010; Nolen-Hoeksema, 2001; Piccinelli & Wilson, 2000).

Third, while pliance instructions were always delivered by another (graduate) student in the Baruch et al. study, we employed a researcher who was a Catholic priest and teacher at the participant’s secondary school. It may be that this difference in the perceived authority of the rule-giver (teacher vs. student) and their relationship to the rule-follower (student) resulted in greater pliance in the current study relative to previous work in this area—especially for students with high levels of depressive symptoms. Indeed, other aspects of how we arranged the social contingencies (e.g., conducting the study at the student’s school) may have further increased a tendency to comply with instructions when delivered by a strong authority figure. These suggestions are consistent with an extensive body of social-psychological work on obedience highlighting that rule-following (especially pliance) can be moderated by an interaction between salient contextual factors and the individual’s learning history (for a review, see Blass, 1999; Zimbardo, 2007).

While these suggestions may account for differential pliance effects across studies, they do not explain why participants with low levels of depressive symptoms showed schedule sensitivity (rather than insensitivity) in Experiment 1. In explaining this divergence in the literature, it seems important to note that Rosenfarb et al. and Baruch et al. manipulated the relationship between instructions and alternating contingencies within rather than between participants. That is, in their studies, participants were initially exposed to a set of accurate instructions that were rendered inaccurate only when the response contingencies changed later on in the experiment. In contrast, we altered the reinforcement of HRs and LRs of responding in Experiment 1 across every block of trials and equipped half of the participants with instructions that were always accurate and the other half of participants with instructions that were always inaccurate. Thus, it may be that participants with low levels of depressive symptoms are less sensitive to changing contingencies when they have previously contacted a rule that was always correct rather than always incorrect. Put another way, when the contingencies change halfway through an experiment and the previously correct rate or type of response is punished, it may be more difficult to abandon a rule that occasioned effective action in past compared with a rule that never did so. To test this possibility, future work could systematically manipulate the instruction/contingency relationship between as well as within dysphoric and non-dysphoric participants. Given that we have only scratched the surface when it comes to second-order instructional control, these studies could also explore which contextual factors increase or decrease the probability of excessive rule-following.

Limitations and Future Directions

Before proceeding, it is worth considering several points. In both of our experiments, participants were exclusively male and divided into high and low groups on the basis of their self-reported depressive symptomatology. Future work could attempt to generalize our findings through the use of a larger, more gender balanced sample that is selected on the basis of a detailed screening procedure. Likewise, given that research has almost exclusively relied on clinical analog populations such as undergraduate and second-level students, there is a clear need to extrapolate these findings to clinical samples of depressed individuals (see Cella et al., 2009, for preliminary work in this vein). The ubiquity of excessive rule-following or insensitivity to changing contingencies in other clinical domains such as anxiety, obsessive-compulsive disorders, and substance abuse also awaits further empirical inquiry. When carrying out such work, researchers should vary the manner in which instructions are delivered and their precise relationship to experimental contingencies. For instance, it may be that the observed pliance effects could be further augmented by seating an authority figure beside the participant throughout the task or even diminished by employing an automated procedure that delivers the rule on a comparable schedule. Likewise, future work could also examine whether inaccurate instructions that previously occasioned effective action (e.g., accessing money) undermine contingency control to a greater degree than instructions that never occasioned such action in the past. In addition, we did not compare participants who had their behavior instructed with those that were directly shaped via trial-and-error learning. Replications could include such a condition to examine whether sensitivity to changes in the world are better for individuals who are directly shaped versus instructed when the above factors are controlled for.

Throughout the current study, we have assumed that accurate responding in presence of inaccurate instructions reflects a sensitivity to changing environmental contingencies. It may be that these “sensitivity” effects are also rule-governed in the sense that they reflect the formation of a novel set of rules based on the individual’s direct experience (e.g., “The previous rule is incorrect. It seems that I have to select the comparison stimulus that is least like the sample stimulus.”). Follow-up work could test this hypothesis by examining whether responding changes gradually or suddenly when contingencies are repeatedly reversed without notice. A large body of evidence now indicates that while non-humans adjust to alternating contingencies of reinforcement in a gradual manner, verbally trained humans often do so quickly and these differences reflect the deployment of (covert) self-generated rules on the part of the latter group (Shimoff et al., 1981).

It is also worth noting that the foregoing studies (including the current work) focused on how conflicts between rules and contingencies impact behavior in sub-clinical populations. Yet, to the best of our knowledge, no research has ever explicitly tailored the content of those rules (or consequences for responding) to the psychopathology under investigation. While it is certainly interesting that dysphoric and non-dysphoric participants show different patterns of responding when rules and contingencies oppose one another, it remains to be seen whether even more dysfunctional responding emerges when the content of a rule (or consequences of responding) is directly related to the clinical domain of interest. For example, future work could determine whether chronic pain sufferers become “locked into” patterns of maladaptive rule-following when those rules allow them to avoid pain at the expense of contact with other consequences in the environment. The same goes for clinical and sub-clinical populations suffering from anxiety, depression, obsessive-compulsions, or any other psychopathology where rule-following plays a role. Directly manipulating the content of a rule (and the consequences of responding) so that they are either relevant or irrelevant to the individual’s particular psychopathology may provide one interesting way forward.

Finally, research has yet to show that (maladaptive) rule-following in clinical populations is directly aimed at reducing contact with aversive thoughts, feelings, and memories (i.e., experiential avoidance). This is particularly surprising given that this is a central tenant of contextual CBT therapies such as ACT (Hayes et al., 1999). One way of exploring this issue would be to establish aversive or avoidance functions for a set of stimuli and then index whether the obtained insensitivity effects continue to hold for clinical and non-clinical groups.

Conclusion

When taken together, the current work indicates that learning via instructions significantly impacts our ability to flexibly adapt to changing contingencies in the environment—especially for individuals who self-report high levels of depressive symptomatology. While empirical interest in verbal regulation has largely focused on its clinical importance (Törneke et al., 2008), this class of behavior may also have implications for other domains within psychological science. The ability to learn via instructions may unlock new conceptual and theoretical insights into phenomena such as motivation, emotion, persuasion (Smith, De Houwer, & Nosek, 2013), and evaluation (Gast & De Houwer, 2013). Consider, for example, social cognition. A number of functional researchers have recently argued that rules—with sufficient practice—can come to guide our behavior in an “automatic” manner (Hughes, Barnes-Holmes & Vahey, 2012). Future work could subject the environmental regularities that influence “automaticity” to an experimental analysis and, in doing so, clarify its role in adaptive and maladaptive rule-following.

Footnotes

Authors’ Note

The preparation of this article was made possible by a Government of Ireland Research Fellowship to the second author.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Author Biographies

David McAuliffe is a lecturer at University College Cork. His research interest focus on the impact of excessive rule-following on work-place stress among senior and middle management.

Sean Hughes is a Post Doctoral Researcher at NUIM. His research interests include Relational Frame Theory (RFT) and its implications for evaluative learning, rule-governed behavior and implicit cognition.

Dermot Barnes-Holmes is Professor of Psychology at NUIM. His research interests include RFT and its implications for most areas in psychological science including clinical psychology, implicit cognition, social psychology and applied behavior analysis.

References

Baruch

D. E.

Kanter

J. W.

Busch

A. M.

Richardson

J. V.

Barnes-Holmes

(2007). The differential effect of instructions on dysphoric and nondysphoric persons. The Psychological Record, 57, 543-554.

Baumeister

R. F.

(2008). Free will in scientific psychology. Perspectives on Psychological Science, 3, 14-19.

Beck

A. T.

(2005). The current state of cognitive therapy: A 40-year retrospective. Archives of General Psychiatry, 62, 953-959.

Blass

(1999). The Milgram paradigm after 35 years: Some things we now know about obedience to authority. Journal of Applied Social Psychology, 29, 955-978.

Cella

Dymond

Cooper

(2009). Impairment in flexible emotion-based learning in hallucination- and delusion-prone individuals. Psychiatry Research, 170, 70-74.

Dixon

M. R.

Hayes

L. J.

Aban

(2000). Examining the roles of rule-following, reinforcement, and pre-experimental histories on risk-taking behavior. The Psychological Record, 50, 687-704.

Dobson

K. S.

Dozois

D. J.

(2010). Historical and philosophical bases of the cognitive-behavioral therapies. In Dobson

K. S.

(Ed.), Handbook of cognitive behavioral therapies (pp. 3-39). New York, NY: Guilford.

Doll

B. B.

Jacobs

W. J.

Sanfey

A. G.

Frank

M. J.

(2009). Instructional control of reinforcement learning: A behavioral and neurocomputational investigation. Brain Research, 1299, 74-94.

Essau

C. A.

Lewinsohn

P. M.

Seeley

J. R.

Sasagawa

(2010). Gender differences in the developmental course of depression. Journal of Affective Disorders, 127, 185-190.

10.

Fletcher

Hayes

S. C.

(2005). Relational Frame Theory, Acceptance and Commitment Therapy, and a functional analytic definition of mindfulness. Journal of Rational-Emotive & Cognitive-Behavioral Therapy, 23, 315-336.

11.

Forman

E. M.

Herbert

J. D.

Moitra

Yeomans

P. D.

Geller

P. A.

(2007). A randomized controlled effectiveness trial of Acceptance and Commitment Therapy and Cognitive Therapy for anxiety and depression. Behavior Modification, 31, 772-799.

12.

Gast

De Houwer

(2013). The influence of extinction and counterconditioning instructions on evaluative conditioning effects. Learning and Motivation, 44, 312-325.

13.

Hayes

S. C.

(1993). Rule governance: Basic behavioral research and applied implications. Current Directions in Psychological Science, 2, 193-197.

14.

Hayes

S. C.

Barnes-Holmes

Roche

(Eds.). (2001). Relational Frame Theory: A Post-Skinnerian account of human language and cognition. New York, NY: Plenum Press.

15.

Hayes

S. C.

Brownstein

A. J.

Haas

J. R.

Greenway

D. E.

(1986). Instructions, multiple schedules, and extinction: distinguishing rule-governed from scheduled-controlled behavior. Journal of the Experimental Analysis of Behavior, 46, 137-147.

16.

Hayes

S. C.

Brownstein

A. J.

Zettle

R. D.

Rosenfarb

Korn

(1986). Rule-governed behavior and sensitivity to changing consequences of responding. Journal of the Experimental Analysis of Behavior, 45, 237-256.

17.

Hayes

S. C.

Strosahl

Wilson

K. G.

(1999). Acceptance and Commitment Therapy: An experiential approach to behavior change. New York, NY: Guilford.

18.

Hayes

S. C.

Zettle

R. D.

Rosenfarb

(1989). Rule following. In Hayes

S. C.

(Ed.), Rule-governed behavior: Cognition, contingencies, and instructional control (pp. 191-220). New York, NY: Plenum Press.

19.

Hollon

S. D.

Stewart

M. O.

Strunk

D. R.

(2006). Enduring effects for cognitive behavior therapy in the treatment of depression and anxiety. Annual Review of Psychology, 57, 285-315.

20.

Hughes

Barnes-Holmes

Vahey

(2012). Holding on to Our Functional Roots When Exploring New Intellectual Islands: A Voyage through Implicit Cognition. Journal of Contextual Behavioural Science, 1, 17-38.

21.

Kanter

J. W.

Busch

A. M.

Weeks

C. E.

Landes

S. J.

(2008). The nature of clinical depression: Symptoms, syndromes, and behavior analysis. Behavior Analyst, 31, 1-22.

22.

Luoma

J. B.

Kohlenberg

B. S.

Hayes

S. C.

Bunting

Rye

A. K.

(2008). Reducing self-stigma in substance abuse through acceptance and commitment therapy: Model, manual development, and pilot outcomes. Addiction Research & Theory, 16, 149-165.

23.

Milgram

(1974). Obedience to authority. New York, NY: Harper & Row.

24.

Monestes

J. L.

Villatte

Stewart

Loas

(in press). Rule-based insensitivity and delusion maintenance in schizophrenia. The Psychological Record.

25.

Nolen-Hoeksema

(2001). Gender differences in depression. Current Directions in Psychological Science, 10, 173-176.

26.

O’Hora

Maglieri

K. A.

(2006). Goal statements and goal-directed behavior: A relational frame account of goal setting in organizations. Journal of Organizational Behavior Management, 26, 131-170.

27.

Piccinelli

Wilkinson

(2000). Gender differences in depression: Critical review. British Journal of Psychiatry, 177, 486-492.

28.

Rosenfarb

I. S.

Burker

E. J.

Morris

S. A.

Cush

D. T.

(1993). Effects of changing contingencies on depressed and nondepressed individuals. Journal of Abnormal Psychology, 102, 642-646.

29.

Rush

A. J.

Giles

D. E.

Schlesser

M. A.

Fulton

C. L.

Weissenburger

Burns

(1986). The Inventory for Depressive Symptomatology (IDS): Preliminary findings. Psychiatry Research, 18, 65-87.

30.

Rush

A. J.

Carmody

Reimitz

P. E.

(2000). The Inventory of Depressive Symptomatology (IDS): Clinician (IDS-C) and self-report (IDS-SR) ratings of depressive symptoms. International Journal of Methods in Psychiatric Research, 9, 45-59.

31.

Rush

A. J.

Gullion

C. M.

Basco

M. R.

Jarrett

R. B.

Trivedi

M. H.

(1996). The Inventory of Depressive Symptomatology (IDS): Psychometric properties. Psychological Medicine, 26, 477-486.

32.

Rush

A. J.

Trivedi

M. H.

Ibrahim

H. M.

Carmody

T. J.

Arnow

Klein

D. N.

. . . Keller

M. B.

(2003). The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): A psychometric evaluation in patients with chronic major depression. Biological Psychiatry, 54, 573-583.

33.

Shimoff

Catania

A. C.

Matthews

B. A.

(1981). Uninstructed human responding: Sensitivity of low-rate performance to schedule contingencies. Journal of the Experimental Analysis of Behavior, 36, 207-220.

34.

Smith

C. T.

De Houwer

Nosek

B. A.

(2013). Consider the source: Persuasion of implicit evaluations is moderated by manipulations of source credibility. Personality and Social Psychology Bulletin, 39, 193-205.

35.

Törneke

Luciano

Valdivia-Salas

(2008). Rule governed behavior and psychological problems. International Journal of Psychology and Psychological Therapy, 8, 141-156.

36.

Van de Velde

Bracke

Levecque

Meuleman

(2010). Gender differences in depression in 25 European countries after eliminating measurement bias in the CES-D8. Social Science Research, 39, 396-404.

37.

Zettle

R. D.

Hayes

S. C.

(1982). Rule governed behavior: A potential theoretical framework for cognitive behavior therapy. In Kendall

P. C.

(Ed.), Advances in cognitive behavioral research and therapy (pp. 73-118). New York, NY: Academic Press.

38.

Zimbardo

(2007). The Lucifer effect. New York, NY: Random House.