Abstract
Cognitive distortions are an important focus in many investigations and treatments of externalizing problem behavior, such as antisocial, delinquent, and aggressive behavior. Yet the overall strength of the association between cognitive distortions and externalizing behavior is unknown. Furthermore, it is unknown whether interventions can effectively reduce cognitive distortions and subsequently externalizing behavior. To fill these gaps, we conducted a meta-analysis of 71 studies on 20,685 participants. Results showed a medium to large effect size (d = .70) for the association between cognitive distortions and externalizing behavior. Interventions had a small effect (d = .27) on reducing cognitive distortions. In a subset of intervention studies that incorporated both cognitive distortions and externalizing behavior, however, neither cognitive distortions nor externalizing behavior were effectively reduced. Hence, although cognitive distortions are substantially linked to externalizing behavior and interventions can reduce cognitive distortions, a subsequent reduction in externalizing behavior remains to be demonstrated.
As fallible human beings, all of us share the impulse to justify ourselves and avoid taking responsibility for any actions that turn out to be harmful, immoral, or stupid . . . We are justifying behavior that we know is wrong precisely so that we can continue to see ourselves as honest people and not criminals or thieves . . . [But] we can catch ourselves . . . By looking at our actions critically and dispassionately, as if we were observing someone else, we stand a chance of breaking out of the cycle of action followed by self-justification, followed by more [morally wrong] action. We can learn to . . . insert a moment of reflection, and think about whether we really want to . . . hold on to a belief that is unfettered by facts.
In the above-quoted passage, Tavris and Aronson claim that we all share the impulse to excuse or rationalize antisocial actions to avoid responsibility and protect our positive self-perception. Understanding the emergence and maintenance of antisocial actions is important given the widespread and serious negative consequences of socially destructive acts in society (Burfeind & Bartusch, 2011; Loeber & Farrington, 1998). In the present meta-analysis, we will use the term externalizing problem behaviors as an overarching term to refer to the broad range of problem behaviors directed toward damaging others, including antisocial behavior, delinquent behavior, aggressive behavior, externalizing behavior, and bullying behavior (Achenbach, McConaughy & Howell, 1997).
The concept of cognitive distortions plays a key role in the understanding and treatment of externalizing problem behaviors in the field of criminology and corrections. The term cognitive distortions has often been used as a general umbrella term to refer to pseudo-justifications and rationalizations for their deviant behavior, and pro-criminal or offense-supporting attitudes (Maruna & Copes, 2004; Maruna & Mann, 2006; Ó Ciardha & Gannon, 2011). Cognitive distortions may be criminogenic insofar that they may help protect the self from blame or prevent a negative self-concept and thereby reinforce antisocial behavior (Barriga, Landau, Stinson, Liau, & Gibbs, 2000). Many studies have indicated that offenders have elevated levels of cognitive distortions compared with non-offenders, and other studies showed that higher levels of cognitive distortions are related to higher levels of several types of externalizing problem behavior. In spite of these empirical findings, two narrative reviews have challenged the assumption of a strong relationship between cognitive distortions and externalizing problems (Gannon & Polaschek, 2006; Maruna & Copes, 2004; Maruna & Mann, 2006). As such, they also questioned whether cognitive distortions should be an important program target in intervention programs aimed at reducing externalizing behavior.
Given the absence of a meta-analysis on this topic, researchers at present have no definitive information regarding the overall strength of the empirical association between cognitive distortions and externalizing problem behavior. We do not know whether cognitive distortions can be effectively reduced in interventions, nor whether such reductions then diminish externalizing problem behavior. Despite the fact that narrative reviews can be very informative, they lack a systematic and transparent inclusion of studies, which makes them susceptible to bias (Egger & Smith, 1997; Teagarden, 1989). A meta-analysis is a prime vehicle for avoiding such biases by providing transparent specifications. Another important asset is that meta-analyses, in contrast to narrative reviews, provide insight into the strength of a phenomenon by a systematic assessment of the effect sizes of the included studies. Accordingly, we conducted a meta-analysis with the aim to investigate (a) the strength of the association between cognitive distortions and externalizing problem behaviors and (b) the effectiveness of interventions in reducing cognitive distortions, and subsequently, externalizing problem behavior.
Externalizing Problem Behavior
Actions that harm others, by breaking important social or moral norms, are typically referred to as antisocial or externalizing behavior (Barriga, Morrison, Liau, & Gibbs, 2001). Such antisocial actions are sometimes termed externalizing (i.e., outwardly focused or directed) and are considered to indicate under-control, aggressive, or even psychopathic or personality disorder problems (Achenbach, 1993; Kazdin, 1995). Externalizing problem behaviors may be “overt” (directly confrontational, such as frequent fights, arguing, and temper tantrums) or “covert” (indirect or concealed, such as stealing, lying, and setting fires; Loeber & Farrington, 1998). In longitudinal studies (e.g., Loeber, Keenan, Lahey, Green, & Thomas, 1993), overt and/or covert externalizing problem behavior has been found to worsen for some persons (in a common etiological pathway) from early oppositional defiance or frequent tantrums, to conduct disorder, and finally to adult offending. Individuals in this trajectory often become involved at some point with mental health services (at clinics and hospitals) and/or the juvenile or criminal justice system (police, courts, correctional institutions).
Externalizing problem behavior has been assessed in various ways. Common measures include direct observation, self-reports, ratings (by others such as peers, parents, teachers, or treatment specialists), and institutional records (such as incident or truancy reports, suspensions or expulsions, and arrests or convictions). In outcome evaluations of intervention programs for offenders, lower rates of recidivism (re-arrest or re-incarceration for parole violations or for new offenses) are typically interpreted as indicating reductions in antisocial behavior or successful re-entry into the community (Kazdin, 1995).
Cognitive Distortions
The concept of cognitive distortion is multifaceted and hence complex. It is important to note that there is not a shared definition among scholars of the concept of cognitive distortions. In the criminological literature, the term cognitive distortion is often used as “an umbrella term to refer to offence-supportive attitudes, cognitive processing during an offence sequence, as well as post hoc neutralizations or excuses for offending” (Maruna & Mann, 2006, p. 155). Interpretations or beliefs that support illegal offenses have been called pro-criminal attitudes (Banse, Koppehele-Gossel, Kistemaker, Werner, & Schmidt, 2013). In the present study, we focus on self-serving cognitive distortions that function mainly to support offending or externalizing problem behavior, as distinct from the function of self-debasing cognitive distortions that support self-destructive or internalizing problem behavior (Barriga et al., 2000; Beck, 1976, Ellis, 1977). In our literature review that follows, we describe several perspectives on self-serving cognitive distortions including their definition and categorization of cognitive distortions.
How might self-serving cognitive distortions facilitate externalizing problem behavior? Maruna and Mann’s (2006) characterization of excuses as “neutralizations” (see above) refers to the classic term used by Sykes and Matza (1957). This theory assumes that everyone, including juvenile delinquents and other offenders, has some commitment to the norms or values of a given society, and that criminal behavior is typically discrepant from those values and accordingly creates a problem for the offender (Sykes & Matza, 1957). According to Sykes and Matza, offenders often resolve this problem by using “neutralization” techniques (i.e., rationalizations that deny or minimize the normative violations) and thereby enable pseudo-reconciliations between criminals’ societal norms or values and their antisocial behavior. These rationalizations protect individuals from self-blame, and could follow—but also precede—deviant behavior. Sykes and Matza posited five neutralization techniques. “Denial of responsibility” enables delinquents to eschew responsibility for their deviant acts. In “denial of injury,” the delinquent act is viewed as not causing any great harm. “Denial of the victim” either denies the existence of a victim or transforms the victim into an individual deserving injury. Similarly, “condemnation of the condemners” shifts the focus from the delinquent act to motives and behaviors of the ones who reject the delinquent act. “Appeal to higher loyalties,” finally, recasts the deviant act as dedicated service to the gang or other group to which the delinquent belongs.
Intrigued by Sykes–Matza’s and similar analyses (e.g., Samenow, 2004), Gibbs (1987, 1991, 2014) defined cognitive distortions as “inaccurate or rationalizing attitudes, thoughts or beliefs concerning one’s own or other’s behavior” (Gibbs, Potter, & Goldstein, 1995, p. 108). Cognitive distortions are self-serving in that they protect the self from blame and negative self-concept when engaging in antisocial behavior. Cognitive distortions are classified into four categories (Barriga & Gibbs, 1996; Gibbs et al., 1995). “Self-centered” is described as according status to one’s own views and needs to such a degree that the views of others are scarcely considered. Self-centered is considered a primary cognitive distortion that precedes and facilitates antisocial behavior; the ego threats from such behavior are then vitiated through the use of “secondary” cognitive distortions. “Blaming others” attributes blame to external sources and “assuming the worst” refers to the gratuitous attribution of hostile intentions to others or considering social situations as a worst-case scenario. Finally, offenders using “minimizing/mislabeling” reframe their antisocial behavior as causing no real harm or as being acceptable and even admirable. Blaming others, assuming the worst, and minimizing/mislabeling are considered as secondary cognitive distortions that permit people to continue engaging in antisocial behavior (Barriga et al., 2000; Gibbs et al., 1995).
A third perspective on cognitive distortions has been developed by Bandura, Barbaranelli, Caprara, and Pastorelli (1996). The moral disengagement theory posits that people refrain from behaving in ways that will violate their moral standards because it negatively affects their self-concept. Bandura et al. (1996) describe the following eight mechanisms of moral disengagement. By “moral justification,” deviant behavior is made acceptable by portraying it as in service of valued social or moral purposes. Deviant acts can be accorded respectable status with the mechanism “euphemistic language,” and deviant acts can be made to appear of little consequence by “advantageous comparison.” “Displacement of responsibility” can be used to view one’s actions as attributable to social pressure, instead of something for which one is responsible. In the case of “diffusion of responsibility,” people take less responsibility for their actions when performed under group conditions. By “disregarding or distorting consequences,” people avoid facing and minimize the harm they caused others. By “dehumanization,” certain people are robbed of their human qualities, and by “attribution of blame,” offenders view themselves as victims. The actual victims are blamed for bringing suffering on themselves.
Cognitive distortions have also taken a central place in research on sex offending. In a review, Ó Ciardha and Ward (2013) conclude that the concept of cognitive distortions in sexual offender literature is vague, broad, and at worst meaningless. They present the following working definition:
Cognitive distortions in sex offenders are specific or general beliefs/attitudes that violate commonly accepted norms of rationality, and which have been shown to be associated with the onset and maintenance of sexual offending. These beliefs may violate rationality norms in a number of ways—for example, they may be based on sources of evidence that are not considered to be sufficient to ground particular beliefs/attitudes (e.g., the responses of an abused child, the response of fellow offenders, pornography use, etc.). (Ó Ciardha & Ward, 2013, p. 6)
In their review, they describe and appraise several theories on cognitive distortions in the sex offender literature, such as the founding work of Abel et al. (1989), the implicit theory (IT) theory and its successor the judgment model, the schema-based model, the extended mind theory, and discursive psychology perspective (Ó Ciardha & Ward, 2013). Given the limited space, we will only briefly discuss the implicit theories theory of Ward and Keenan (1999) and Ward (2000) as this theory was overall appraised as strongest in the review.
Ward and Keenan (1999) and Ward (2000) presented the idea that child molesters hold a set of beliefs that they referred to as implicit theories (ITs). These ITs are viewed as particular types of schema, which offenders use to explain, predict, and interpret interpersonal phenomena relevant to sexual offending (Ó Ciardha & Ward, 2013). Not all sex offenders have the same problematic ITs; however, five ITs are hypothesized to be prevalent among sexual offenders against children: children as sexual objects (children are capable of desiring and enjoying sex and have adult sexual motivations), nature of harm (sexual molesting is not harmful or is beneficial), uncontrollability (offending behavior is outside the offender’s control), entitlement (the offender’s needs or wants supersede those of others), and dangerous world (the world is a hostile and dangerous place where no one or only children can be trusted) (Ó Ciardha & Ward, 2013). Overall, Ó Ciardha and Ward conclude in their review that progress has been made in theory development of cognitive distortions in sex offender literature, but more empirical support is needed.
Consistent with their multifaceted nature, self-serving cognitive distortions have been assessed through a variety of methods. Mostly used are self-reported semi-structured interviews or questionnaires, but their great disadvantage is the potential for respondents to answer social desirability and with that an underreport of cognitive distortions (Babchishin, Nunes, & Hermann, 2013; Nunes, Firestone, & Baldwin, 2007; Ó Ciardha & Ward, 2013). To overcome this problem, more indirect measures (e.g., response latency following a brief “prime” techniques) aimed to assess cognitive distortions without interference of impression management by the respondent (Babchishin et al., 2013; Nunes et al., 2007; Ó Ciardha, & Ward, 2013). In addition, “surface” cognitive distortions as measured with direct measures may reflect underlying belief systems, attitudinal schemas, or implicit theories or worldviews. Researchers attempt to assess these underlying belief systems using indirect measures. Unfortunately, research to date using indirect methods has been inconclusive (Ó Ciardha & Ward, 2013). Therefore, in the present meta-analysis, we focus on direct measures of cognitive distortions.
Self-report measures that broadly target (often with factor subscales) cognitive distortions have included for example the How I Think questionnaire (Gibbs, Potter, & Barriga, 2001), the Measures of Criminal Attitudes and Associates (Mills, Kroner, & Forth, 2002), and the Psychological Inventory of Criminal Thinking Styles (Walters, 2005, 2011). Self-report cognitive distortion measures specific to types of offenses, such as sexual offenses, have included the Abel-Becker Cognition Scale (Abel et al., 1989) and the Rape Myth Acceptance Scale (Burt, 1980). Although they did not conduct a meta-analysis for assessing the effect size, they found support for the hypothesis that pro-criminal attitudes are related to reoffending (Banse et al., 2013).
Moderators
The association between cognitive distortion and externalizing problem behavior could be affected by certain study, sample, and measurement characteristics. Studies with non-significant findings are often not written down or published and hence they are difficult to retrieve; this phenomenon is known as the “file drawer problem” (Hox, 2010; Lipsey & Wilson, 2001). To cope with and assess this problem, we included unpublished studies wherever possible and investigated whether effect sizes were larger for published studies (cf. Lipsey, 2009; Stams et al., 2006). In addition, we examined the study design as a moderator. The association between cognitive distortions and externalizing problem behavior could be stronger for correlational studies (e.g., Bandura et al., 1996; Barriga, Hawkins, & Camelia, 2008) given that group comparisons (e.g., Lardén, Melin, Holst, & Långström, 2006) do not utilize continuous measures and hence lose information and power (Banse et al., 2013; Markon, Chmielewski, & Miller, 2011).
Furthermore, we examined whether the sample characteristics of age, gender, and ethnicity would moderate the strength of the association between cognitive distortion and externalizing problem behavior. Differences in levels of cognitive distortions and externalizing problem behavior have been established for certain subgroups (e.g., adolescents, males, ethnic minorities) and this might moderate the strength of association for certain subgroups (Bandura et al., 1996; Barriga et al., 2000; Barriga et al., 2001; Bruno, 2010; Burfeind & Bartusch, 2011; Hawkins, Laub, & Lauritsen, 1998; Moffitt, 1993; Moffitt, Caspi, & Sila, 2001; Sampson & Laub, 2003; Wallinius, Johansson, Lardén, & Dernevik, 2011).
The strength of the association between cognitive distortions and externalizing problem behavior could also be influenced by the reporting mode of externalizing problem behavior. Self-report measures of behavior might be more representative of the actual level of externalizing problem behavior because not all offenses and problem behaviors are detected by official documentation, parents, and/or teachers (Achenbach, McConaughy, & Howell, 1987; Brame, Fagan, Piquero, Schubert, & Steinberg, 2004; Kirk, 2006; Thornberry & Krohn, 2000). Therefore, the association between cognitive distortions and externalizing problem behavior might be stronger for self-reported externalizing problem behavior compared with other reporting modes of externalizing problem behavior. In addition, the link between self-reported behavior and cognitive distortions might also be stronger due to shared method variance.
Finally, in the present meta-analysis, we include a broad range of externalizing behavioral problems, such as antisocial behavior, delinquent behavior, aggressive behavior, externalizing behavior, and bullying behavior; therefore, we will explore whether the strength of association between cognitive distortions and externalizing problem behavior is affected by the type of externalizing behavioral problems.
Treatment of Cognitive Distortions
The treatment of cognitive distortions has become an important component in many present-day intervention programs that are aimed at reducing externalizing problem behaviors (Maruna & Copes, 2004; Maruna & Mann, 2006; Ó Ciardha & Gannon, 2011). Previous meta-analyses have shown that cognitive behavior interventions are effective in reducing recidivism (Landenberger & Lipsey, 2005; Lipsey, 2009), but they have not made clear whether success treatment outcomes are a consequence of “cognitive restructuring,” that is, the reframing or correction of cognitive distortions in the treatment which is expected to result in behavioral changes (Maruna & Copes, 2004; Maruna & Mann, 2006).
The inclusion of intervention studies in the present meta-analysis will make an important contribution to clinical practice by investigating whether treatment can reduce cognitive distortions, and whether such reductions can subsequently induce lower levels of externalizing problem behavior. In addition, the inclusion of experimental studies will not only provide input to the treatment of cognitive distortions in clinical practice, but can also provide theoretical insight to the field of cognitive distortions. The inclusion of experimental studies can help ascertain a causal relationship between cognitive distortions and externalizing problem behavior.
In their review, Banse et al. (2013) found that most offender treatment programs tend to reduce pro-criminal attitudes, but that there is no conclusive evidence that intervention programs designed to reduce pro-criminal attitudes are effective in reducing recidivism. Unfortunately, they did not identify the effect size of the impact of intervention programs on pro-criminal attitudes and recidivism.
Research Aims and Hypotheses
This study examined the cognitive distortions literature using a meta-analytic approach for the first time. Our first aim was to investigate the extent to which cognitive distortions are linked to externalizing problem behavior. We hypothesized that there would be a positive association between cognitive distortions and externalizing problem behavior. In addition, we aimed to identify moderators that influenced the strength of the association between cognitive distortions and externalizing problem behavior. We focused on study and sample characteristics (i.e., publication type, design, gender, age, and ethnicity) and measurement characteristics (i.e., type and report mode of externalizing problem behavior). Our second aim was to examine the impact of intervention programs on cognitive distortions and externalizing problem behavior. We hypothesized that intervention programs would effectively reduce cognitive distortions, and subsequently, reduce externalizing problem behavior.
Method
Literature Search
From October 2010 to August 2011, we searched studies via the databases PsycInfo (including Dissertation Abstracts International Section A and B), Scopus, and Medline using specified keyword combinations. For the association between cognitive distortions and externalizing problem behaviors, we used the following keywords referring to cognitive distortions: “Cognitive and Distortions, Moral and Disengag*, Neutrali*, Belief and System, Thinking and Error” in combination with the following keywords for externalizing problem behavior: “Antisocial, Delinq*, Criminal*, Offender*, Aggress*, Externali*.” For the effect size of interventions aimed at reducing cognitive distortions, we added specific intervention-related keywords to the keyword combinations mentioned above: “Intervention, Program*, Treatment, Prevention, Therapy, EQUIP, Cognitive Self-Change, Changing Criminal Thinking, CHANGE, THINK, Thinking for a change, Steps to change, ART, COVA, Enhanced Thinking Skills, Improve*, Reduc*.” In addition to the databases, we checked the reference list of previous reviews on this topic (Gannon & Polaschek, 2006; Maruna & Copes, 2004; Maruna & Mann, 2006), and by searching studies—both published and non-published—from personal libraries of the authors.
Selection Criteria
We used the following selection criteria for studies to assess the association between cognitive distortions and externalizing problem behavior.
1. We placed no restrictions on the year and type of publication (i.e., published or non-published), nor on participants’ age and the severity of externalizing problem behavior that was studied.
2. Studies could be either written in English or Dutch.
3. To be included, studies had to contain a measure of self-serving cognitive distortions in which explicit answers were produced. Measures tapping implicit attitudes utilizing reaction time or other nonverbal indicators were not included in the present meta-analysis.
4. Studies had to include either a correlation coefficient assessing the relationship between measures of cognitive distortions and externalizing problem behavior, or a comparison assessing differences in levels of cognitive distortions between groups with and without externalizing problem behavior.
Whenever multiple studies reported on the same sample, we selected the study with the most detailed results or, when equally detailed, the most recent study. We did so to prevent “double counts” of these samples. Using these selection criteria, 55 studies were included that provided data on the association between cognitive distortions and externalizing problem behavior.
To assess the effect size of treatment on cognitive distortions and externalizing problem behavior, we included studies examining the effectiveness of treatment programs in reducing cognitive distortions. We used the same inclusion and exclusion criteria (Items 1, 2, and 3) as mentioned above. In case of intervention studies (Item 4), studies had to include means and/or F, T, or χ2 values. In addition, based on the Scientific Methods Scale (a system for ranking quality of research designs; Hollin, 2008; Sherman et al., 1997), the intervention studies had to meet the following quality criteria for the evaluation of treatment effectiveness.
5. Studies had to include pre- and posttest measurement of cognitive distortions and, if present, externalizing problem behavior.
6. Studies had to include a treatment group and an appropriate control group; this could be with or without random allocation. The control group could be treatment as usual, placebo, waiting list, or no treatment.
7. Samples sizes of treatment and control groups had to be n > 5.
Using these selection criteria, 18 intervention studies could be included. Although all 18 intervention studies included measures on cognitive distortions, only 9 studies included measures on externalizing problem behavior.
Coding of Studies
Each study was coded using a detailed coding system for recording study, sample, and measurement characteristics. Intercoder reliability was assessed in 43% of the studies (n = 31) and was found to be satisfactory, with an intercoder agreement of 98% and an average Cohen’s Kappa of .80 ranging from .65 to 1.00, p < .001 (Landis & Koch, 1977).
Study and Sample Characteristics
As study characteristics, we coded whether the study was published (1) versus not published (0), and whether the study design was correlational in nature (1) or had a group-comparison design (0). As sample characteristics, we coded gender distribution, age, and ethnic composition. For gender distribution, the category “male” indicated a sample with more than 60% males, with a reference category female/mixed indicating more than 60% females or a mixed sample with 40% to 60% females. For age, the category “youths” indicated a sample with children less than 12 years or adolescents between 12 and 18 years old, with a reference category “adults” indicating a sample with adults on average more than 18 years. For ethnic composition, the dummy category “majority” indicated an ethnic majority sample with more than 60% of the sample belonging to the ethnic majority, with a reference category of ethnic minority or mixed samples indicating samples with more than 40% ethnic minorities (or 40%-60% ethnic minorities).
Measurement Characteristics
As measurement characteristics, we coded the type of reporting mode of externalizing problem behavior. We constructed the dummy category “self-report,” comparing it with all other reporting modes (i.e., official documentation as well as parent, teacher, peer, and other types of ratings) as a reference. The type of externalizing problem behavior has been coded based on the terminology as used by the authors of the study. We constructed dummies with “delinquent behavior,” “aggressive behavior,” “externalizing behavior,” “bullying behavior,” and “other behavior” (i.e., gambling, substance use, cheating), comparing them with a reference category of “antisocial behavior.”
Effect Sizes
We used Cohen’s d as a measure of effect size and used Wilson’s (2005) spreadsheet for the calculation of effect sizes. For those studies that looked at the association between cognitive distortions and externalizing problem behavior expressed as a correlation (r), the r was converted into Cohen’s d. For group-comparison studies, Cohen’s d was calculated by contrasting the mean difference between groups with and without externalizing problem behavior: (Mext − Mno-ext) / SDpooled (see Lipsey & Wilson, 2001). For studies in which no means and standard deviations were reported, Cohen’s d was computed from F- or T-values (see Lipsey & Wilson, 2001).
For intervention studies, we also used Cohen’s d as measure of effect size, representing the difference in improvement (reduction of cognitive distortions) between intervention and control conditions expressed in standard deviation units: (Xpost − Xpre) / SDpooled (see Lipsey & Wilson, 2001). For studies where no means and standard deviations were reported Cohen’s d was computed from the F or T or χ2 values (see Lipsey & Wilson, 2001). We obtained additional statistics from the authors of two intervention studies (Forde, 2005 and Liau et al., 2004) to be able to include them in the meta-analysis. Effect sizes were computed for all studies at immediate posttest. Furthermore, all effect sizes were adjusted for sample size using an inverse variance correction (Lipsey & Wilson, 2001, p. 72), and all pooled standard deviations were adjusted for the sample size of each group (Lipsey & Wilson, 2001, p .198).
Outlier analysis identified three studies with outlying effect sizes of at least two standard deviations above the mean, and these were removed from the sample (Lipsey & Wilson, 2001). The outliers were studies by Broxholme and Lindsay (2003; d = 2.14-2.89), Kubik and Hecker (2005; d = 2.09-2.75), and Wood and Riggs (2009; d = 7.04). The removed Wood and Riggs study could be replaced by Wood (2007) as these two studies used the same sample, and the Wood study did not show an outlying d value. With the exclusion of these studies, the final sample contained 53 studies that assessed the relationship between cognitive distortions and externalizing problem behavior. In Online Appendices A and B, one can find the forest plots displaying the effect sizes of this meta-analysis. The forest plots were constructed using the excel macro by Neyeloff, Fuchs, and Moreira (2012).
Strategy of Analysis
For the first part of our study, it should be noted that several studies included multiple effect sizes for the association between cognitive distortions and externalizing problem behavior. For example, a study could include different effect sizes on the association between different types of externalizing problem behavior (e.g., delinquency, aggression) resulting in multiple effect sizes within a study. Because this resulted in nested data, we used multilevel analysis in HLM 6 (Hox, 2010; Raudenbush, Bryk, & Congdon, 2004). First, we calculated the mean effect size for the association between cognitive distortions and externalizing problem behavior by specifying the separate effect sizes as outcome variables at the first level, and study number was specified at the second level using a random model.
Second, we performed separate multilevel regressions with a random model to examine whether study characteristics (i.e., published and design), sample characteristics (i.e., gender distribution, age, and ethnic composition), and measurement characteristics (i.e., type and reporting mode of externalizing problem behavior) would moderate the effect size. For conducting separate multilevel regressions, we chose to analyze the effect of each moderator, because not all studies reported on all moderators. Combining them into one multiple regression would result in the exclusion of a large number of studies. When a moderator did not vary within a study (e.g., gender distribution), they were entered as Level 2 variables; these were the study and sample characteristics (i.e., publication type and design, gender, age, and ethnicity). The other measurement characteristics were entered as Level 1 variables (i.e., type and report mode of externalizing problem behavior).
Third, with regard to the intervention studies, it was not feasible to use a multilevel approach due to a limited number of studies. Therefore, analyses were conducted in SPSS using Wilson’s mean effect size macro for meta-analyses (Wilson, 2005) using a random model. For intervention studies, when more than one effect size was available in a study, we aggregated the different effect sizes into one single average effect size for the study. Finally, in interpreting the magnitude of effect sizes, we followed formulations by Cohen (1988); effect sizes of d = .20, d = .50, and d = .80 were considered small, medium, and large effects, respectively.
Publication Bias
We addressed the problem of publication bias by calculating Rosenthal’s fail safe N with DeCoster and Iselin’s (2005) macro in which we used the average effect size for each study. The fail safe N represents the number of studies needed with a null result to bring the mean effect size to non-significance (Cooper, Hedges, & Valentine, 2009). After the fail safe N has been calculated, one can judge whether it is realistic to assume that these many unpublished studies exist using Rosenthal’s (1979) threshold level. Rosenthal suggested that the fail safe N may be considered as being unrealistic when it exceeds 5k + 10 (k is the number of studies). This resulted in a threshold level of 275 studies for the first research question and 100 studies for the second research question.
Results
Association Between Cognitive Distortions and Externalizing Problem Behavior
In the first step of this meta-analysis, we examined 53 studies reporting data on 18,544 participants. In Online Appendix A, one can find an overview of the studies included in the meta-analysis and a forest plot displaying the effect sizes. The studies assessed the relationship between cognitive distortions and externalizing problem behavior. This analysis yielded a significant, medium to large mean effect size of d = .70, p < .001, 95% confidence interval (CI) [.59, .81], random model. This indicates that in correlation studies, higher levels of cognitive distortions were related to higher levels of externalizing behavior. In group-comparison studies, (a) offenders reported higher levels of cognitive distortion than non-offenders, and (b) non-offenders with externalizing problem behavior reported higher levels of cognitive distortions than non-offenders without externalizing problem behavior. The fail safe N analysis showed that 30,217 studies with a null result were needed to render the effect size d = .70 non-significant. This number of studies can be considered unlikely to be found in reality, it is far higher than Rosenthal’s (1979) threshold level of 275 studies and thus indicate there is no file drawer problem.
Moderators
Effect sizes were found to be heterogeneous, Q(52) = 224.23, p < .001, which led us to examine the variation in effect sizes among studies using moderator analysis. In Tables 1 and 2, the intercepts show the effect sizes of the reference category of the moderator variables. To retrieve the effect size of the other categories of the variables, one should add the coefficient B to the intercept of the variable. The moderator analyses demonstrated that none of the study or sample characteristics (i.e., published vs. non-published, correlational vs. group-comparison design, gender, age, and ethnic composition) moderated the effect sizes (see Table 1). With regard to measurement characteristics, there were significant difference in effect size based on the reporting mode and type of externalizing problem behavior. More specifically, studies that reported findings on self-reported externalizing problem behavior had a significantly larger effect size (d = .81) compared with other reporting modes (d = .42). Also, analyses of antisocial behavior (d = .98) yielded a higher effect size than did analyses of delinquent and aggressive behavior, respectively (d = .51, d = .63). The association for externalizing and bullying behavior did not differ from antisocial behavior.
Study and Sample Characteristics as Moderators of Effect Size (Random Model)
Note. All separate regressions. K = number of studies in analysis; R = reference category.
p < .05. **p < .01. ***p < .001.
Measurement Characteristics as Moderators of Effect Size (Random Model)
Note. All separate regressions. K = number of studies in analysis; EPB = externalizing problem behavior; R = reference category.
p < .05. **p < .01. ***p < .001.
Treatment of Cognitive Distortions and Externalizing Problem Behavior
In the second step of this meta-analysis, we examined 18 intervention studies that reported data on 2,037 participants. In Online Appendix B, one can find an overview of the studies included in the meta-analysis and a forest plot displaying the effect sizes. The studies focused on sex offender (n = 6), adult offenders (n = 4), youth offenders (n = 3), at risk youth (n = 2), pathological gamblers (n = 2), and offenders with a drug addiction (n = 1). The most frequently studied programs are EQUIP (n = 6) and Stop and Think (n = 2). Almost all programs use cognitive behavioral elements (n = 17); other frequently mentioned elements are social skills training (n = 11), anger management (n = 7), moral reasoning (n = 6), relapse prevention (n = 4), and victim empathy (n = 3). Online Appendix B provides a full overview of the study characteristics of the included intervention studies.
The studies assessed whether cognitive distortions could be effectively reduced and—in a subset of nine studies—whether reductions in cognitive distortions lead to decreases in externalizing problem behavior. This meta-analysis yielded a significant, small effect size of d = .27, p < .05, 95% CI [.05, .50], random model, indicating that the interventions studied overall had a significant but small effect in the reduction of cognitive distortions. The set of effect sizes was homogeneous, Q(17) = 9.21, p = .93, indicating that there was no significant variation in the effect sizes between studies. The fail safe N analysis showed that 144 studies with a null result were needed to render the effect size d = .27 non-significant. Again, this number of studies is unlikely to be found in reality, because it is higher than the Rosenthal’s (1979) threshold of 100 studies for this research question. These results suggest there is no file drawer problem.
As noted in the methods section, not all intervention studies included a measure of externalizing problem behavior. As a consequence, we were forced to base our analysis of whether reductions in cognitive distortions were related to decreases in externalizing problem behavior on a limited sample of nine studies. In this specific subsample, we found a non-significant effect size for the reduction of cognitive distortions, d = .19, p = .23, 95% CI [−.12, .50], random model, and this set of effect sizes was homogeneous, Q(8) = 4.29, p = .83. In addition, we also found a non-significant effect size on the reduction of externalizing problem behaviors in the subsample, d = .05, p = .77, 95% CI [−.26, .35], random model. This set of effect sizes was found to be homogeneous, Q(8) = 3.30, p = .91. Thus, in this subset of studies, we observed that neither cognitive distortions nor externalizing problem behavior was effectively reduced by the interventions implemented. This makes it difficult to answer the question of whether reductions in cognitive distortions lead to lower levels of externalizing behavior; we did not observe a reduction of cognitive distortions in these studies in the first place. Importantly, when analyzed separately the other nine studies (that only included measures of cognitive distortions) did show that the interventions had small to medium effects in reducing cognitive distortions, d = .37, p = .03, 95% CI [.04, .70], random model.
Discussion
The present study provides a meta-analytic overview of extant research concerning cognitive distortions and externalizing problem behavior. The results showed that as hypothesized, higher levels of cognitive distortions are related to higher levels of externalizing problem behavior, and this association was quite strong. Effect sizes were larger for self-reported externalizing problem behavior as opposed to official documentation, parent, peer, or teacher reports. Effect sizes were also larger for bullying, antisocial, and externalizing behavior as opposed to aggressive and delinquent behavior. The meta-analysis also demonstrated that cognitive distortions are treatable to some extent. As expected, we found that intervention programs lead to small reductions in cognitive distortions. However, in the available subsample of studies that included measures of both cognitive distortions and externalizing problem behavior, neither significant reductions in cognitive distortions nor significant reductions in externalizing problem behavior were found. Despite criticisms raised in previous narrative reviews concerning the strength of the link between cognitive distortions and externalizing problem behavior (Maruna & Copes, 2004; Maruna & Mann, 2006), our findings suggest that cognitive distortions do play an important role in externalizing problem behavior, regardless of ethnic background, age, and gender. This role is particularly pronounced when externalizing problem behavior is indexed by individuals’ self-reports, as compared with other reporting modes such as institutional records, teacher, parent, or peer ratings. Shared method variance or differences in reported levels of externalizing behavior by different informants—with individuals’ self-reports yielding higher levels of externalizing problem behavior than other reports—could explain the stronger association for self-reported externalizing problem behavior (Achenbach et al., 1987; Brame et al., 2004; Kirk, 2006).
Even though we found a medium to large association between cognitive distortions and externalizing problem behavior, we also found that cognitive distortions showed a weaker association with delinquent and aggressive behavior than with antisocial, externalizing, and bullying behavior. A potential explanation for these findings is that questionnaires measuring antisocial behavior, general externalizing behavior, and bullying behavior might represent somewhat less severe types of externalizing problem behaviors compared with questionnaires measuring delinquency and aggression. If that is the case, these findings might suggest that cognitive distortions might have less explanatory power for more serious offenders. These findings seem to be in line with scholars who argue that hard core offenders may not need to develop cognitive distortions that strongly because they adapt their attitudes and behavior to the social norms of their criminal subculture, in contrast to more average offenders who are involved in both a criminal subculture and the wider society at the same time (Banse et al., 2013). For these average offenders, the commitment of criminal acts and engagement to widely shared social norms at the same time are irreconcilable, resulting in more psychological discomfort in average offenders; consequently, cognitive distortions are needed to protect the self, more so than for hard core offenders who are exclusively engaged to their criminal subculture (Banse et al., 2013). It was beyond the scope of the present meta-analysis to categorize the measures of externalizing problem behavior in terms of problem behavior severity, because this would demand analysis of the measures at item level. Therefore, it is important that future research examines the role of cognitive distortions in relation to the severity of externalizing problem behavior and to empirically test whether the association between cognitive distortions and externalizing problem behavior is weaker for more serious offenders, as this hypothesis is currently untested.
An important finding of the present meta-analysis is that treatment programs can lead to small reductions in cognitive distortions. In a subset of studies that also measured externalizing problem behavior, however, we found neither reductions in cognitive distortions nor reductions in externalizing problem behavior. Consequently, the question whether reducing cognitive distortions is an effective mediating mechanism for reducing externalizing behavior remains unresolved. An intervention study, that could not be included in this meta-analysis because it did not include a control group, has demonstrated that reductions in cognitive distortions were significantly related to subsequent reductions in recidivism rates (Devlin & Gibbs, 2010). As Kazdin (1995) noted, “a treatment study showing that changes in cognitive processes occur and correlate with changes in treatment outcome . . . would considerably advance the case for that treatment” (p. 79), as well as for the critical role of cognitive change.
In line with this reasoning, it still seems reasonable to hypothesize that successful interventions on cognitive distortions would lead to subsequent decreases in externalizing problem behavior; this hypothesis deserves extensive and thorough testing in the future. To be able to truly test this hypothesis, it is crucial that future treatment studies include both measures of cognitive distortions and externalizing problem behavior, and use high-quality study designs that include pre- and posttest measurements of experimental as well as control groups, preferably with random allocation. The need for high-quality treatment studies is further underlined by the limited number of available intervention studies that could be included in the present meta-analysis. Such intervention studies on cognitive distortions can make an important contribution to finding effective ingredients for the treatment of externalizing problem behavior. Furthermore, it is of great importance that correctional treatment studies examine whether the treatment of cognitive distortion and subsequent reoffending is less or more effective for serious offenders. These outcomes would provide more insight into what works for whom and could have important impact on policy concerning treatment programs in corrections.
In addition, such experimental intervention studies, as well as longitudinal studies, can provide evidence toward resolving the debate regarding temporal ordering: whether cognitive distortions play a role in the emergence of deviant behavior or whether they are merely post hoc cognitive phenomena related to its maintenance (Maruna & Copes, 2004; Maruna & Mann, 2006). Therefore, conducting more experimental intervention and longitudinal research on the causality of the relationship between cognitive distortions and externalizing behaviors will be important. An important longitudinal study on cognitive distortions and externalizing problem behavior demonstrated, in line with our findings, that declines in moral disengagement are associated with declining likelihood of offending, based on self-reports as well as official records (Shulman, Cauffman, Piquero, & Fagan, 2011). This study was not included in our meta-analysis as it was published beyond our search scope.
A limitation of the present study is that the association between cognitive distortions and externalizing problem behavior in this study is strictly correlational by design. Consequently, it is impossible to disentangle the sequence of the relationship between cognitive distortions and externalizing problem behavior. We had hoped that the inclusion of experimental intervention studies would give us more insight into the causality of the relation, but the outcomes of the analysis on a subset of intervention studies that included both cognitive distortions and externalizing problem behaviors did not allow us to draw conclusions on causality.
Conclusion
The present study has made an important contribution to the research on cognitive distortions by giving insight into strength of the association between cognitive distortions and externalizing problem behavior, and the effectiveness of treatment in reducing cognitive distortions and externalizing problem behavior. This meta-analysis demonstrated that higher levels of cognitive distortions are related to higher levels of externalizing problem behavior. Intervention programs demonstrated the potential to reduce cognitive distortions; however, whether a decrease in cognitive distortions successfully leads to the expected decrease in externalizing behavior still needs to be demonstrated in future high-quality treatment studies.
Footnotes
Acknowledgements
We thank Joop Hox for his help with our multilevel analyses, and we thank Gerben Langendijk for his help in the creation of the forest plots. We also thank our reviewers and our colleagues for their feedback that helped us improve the manuscript.
This research was partially supported by the Frentrop Foundation in The Netherlands.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
