Abstract
Cognitive theories of psychopathology posit that maladaptive patterns of cognitions confer elevated risks to individuals in the development of psychological disorders. This meta-analysis examined the extent to which six cognitive vulnerabilities associated with depression (i.e., pessimistic inferential style, dysfunctional attitudes, and ruminative style) and anxiety (i.e., anxiety sensitivity, intolerance of uncertainty, and fear of negative evaluation) were related with one another. A total of 159 effect sizes from 73 articles were obtained to estimate the mean correlations among the vulnerabilities. Results indicated that they were moderately to strongly correlated. Meta-analytic structural equation modeling was applied to evaluate two alternative factor analytic models underlying the associations among the vulnerabilities. A one-factor model provided the best fit to the meta-analytic data, suggesting a common etiologic factor shared among the vulnerabilities. This suggests that the vulnerabilities are not distinct at a broad level and their common core presents an avenue for transdiagnostic interventions.
A major agenda for psychopathology research has been to identify vulnerability factors that contribute to the onset and maintenance of psychological disorders. Cognitive vulnerabilities, in particular, constitute characteristic patterns of encoding, interpretation, and recall of information that confer elevated risks to individuals who possess them (Riskind & Alloy, 2006). In depression and anxiety research, several cognitive vulnerabilities have been identified to be reliable predictors of symptoms, either by themselves or by interacting with life stress. A cognitive vulnerability is often conceptualized as disorder-specific; that is, it is supposed to predict a specific form of psychopathology exclusively and not others. For instance, pessimistic inferential style (Abramson, Metalsky, & Alloy, 1989) has been proposed to confer elevated risk to the development of depressive, but not anxiety, disorders. However, recent empirical evidence suggests that a single cognitive vulnerability can be related to several forms of psychopathology (e.g., Aldao, Nolen-Hoeksema, & Schweizer, 2010; Naragon-Gainey, 2010; Starcevic & Berle, 2006). This raises the question of whether the various cognitive vulnerabilities, as distinct as they may appear to be, are actually associated with one another. If so, what is the degree of overlap among these vulnerabilities? Also, is there a structural model that organizes these vulnerabilities?
The purpose of the current research was to estimate, through meta-analysis, the associations among six cognitive vulnerabilities to depression and anxiety. They were pessimistic inferential style, dysfunctional attitudes, ruminative style, anxiety sensitivity, intolerance of uncertainty, and fear of negative evaluation. 1 These vulnerabilities were chosen because they were well established in the literature, with numerous studies explicating their links to depression and anxiety. After estimating their intercorrelations, a follow-up analysis would be to delineate a structural model that underlies these vulnerabilities. Do these vulnerabilities load onto a unitary factor or onto separate factors? Addressing this question may help inform whether the vulnerabilities share a common core etiologic process versus distinct processes. To our knowledge, this is the first study to examine the structure of vulnerabilities associated with depression and anxiety using a meta-analytic approach.
The six cognitive vulnerabilities (three putatively associated with depression and three with anxiety) are briefly reviewed here. Pessimistic inferential style (also referred to as negative attributional or negative cognitive style) is a tendency to attribute global and stable causes to negative events and to infer subsequent negative implications and self-image from those events (Abramson et al., 1989; Alloy, Abramson, Keyster, Gerstein, & Sylvia, 2008). Individuals who possess this cognitive style are prone to experience hopelessness and depression symptoms (e.g., Hankin, Abramson, Miller, & Haeffel, 2004; Hankin, Farley, & Abela, 2005) and episodes of clinical depression (e.g., Alloy et al., 2000; Alloy et al., 2006) in the presence of negative life events. Individuals who hold dysfunctional attitudes—a network of supposedly depressogenic schemas related to themes of inadequacy, failure, and the need for social approval—set themselves up for a pattern of self-defeating information processing that can lead to depression (Beck, 1983, 1987; Scher, Ingram, & Segal, 2005). These schemas, often activated in the event of negative stressors, increase the risk of developing depression (Alloy et al., 2000; Alloy et al., 2006; Hankin et al., 2004; Ingram, Miranda, & Segal, 1998). A ruminative style is defined as the tendency to direct one’s attention to the causes and implications of one’s negative moods (Nolen-Hoeksema, 1991; Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008). Although the motivation to ruminate is to provide insights into the causes for problems and moods (Lyubomirsky & Nolen-Hoeksema, 1993), ruminating can entrap individuals in a self-perpetuating spiral of perseverative thoughts that exacerbates depressive moods (Lyubomirsky & Nolen-Hoeksema, 1995). Ruminative style as a vulnerability factor to depression has been amply demonstrated in various methodological contexts: using experimental manipulation (Ciesla & Roberts, 2007), experience sampling (Moberly & Watkins, 2008) and in prospective designs (Just & Alloy, 1997; Nolen-Hoeksema, 2000; Spasojevic & Alloy, 2001).
Anxiety sensitivity is defined as the tendency to believe that anxiety or anxiety-related physical arousal can have undesirable and harmful consequences, thus evoking fearful responses to one’s own anxiety symptoms (McNally, 1994; Reiss, 1991). Its role as a vulnerability factor to panic disorders has been established in biological challenge studies (i.e., exposure to carbon dioxide inhalation; Harrington, Schmidt, & Telch, 1996; Zvolensky, Feldner, Eifert, & Stewart, 2001) and prospective studies (Li & Zinbarg, 2007; Schmidt, Lerew, & Jackson, 1997, 1999; Schmidt, Zvolensky, & Maner, 2006). Intolerance of uncertainty is defined as a set of future-oriented negative beliefs that see uncertainty as undesirable, annoying, and something that should be avoided (Dugas, Buhr, & Ladouceur, 2004; Freeston, Rhéaume, Letarte, Dugas, & Ladouceur, 1994; see also Carleton, 2012, for a review), and is postulated to be a cognitive risk factor to excessive and pathological worry, a central feature underlying generalized anxiety disorder (Buhr & Dugas, 2006; Dugas, Letarte, Rhéaume, Freeston, & Ladouceur, 1995). Beyond predicting worry symptoms in nonclinical (Fergus & Wu, 2011; Sexton, Norton, Walker, & Norton, 2003) and clinical (McEvoy & Mahoney, 2012; van der Heiden et al., 2010) samples, intolerance of uncertainty also leads to increased worrying in experimental studies (Buhr & Dugas, 2009; Ladouceur, Gosselin, & Dugas, 2000). Individuals high on the fear of negative evaluation (Watson & Friend, 1969) tend to see themselves as being subjected to other people’s social evaluations, which they imagine to be overly harsh and critical. This apprehension over what others might think of them is predictive of social anxiety/phobia and performance deficits (Carleton, Collimore, & Asmundson, 2007, 2010; Haikal & Hong, 2010; Rapee & Heimberg, 1997).
To date, no systematic review on the associations among the six cognitive vulnerabilities has been done. A reason for this might be that each of the vulnerabilities had been traditionally seen as disorder-specific and researchers were primarily interested in delineating the precise cognitive mechanisms involved in the development of discrete symptom categories. However, in light of several observations, it might be crucial to take a broad-based view of these vulnerabilities rather than treating them as independent discrete variables. The first observation, which psychopathology researchers have long known, is the strong comorbidity between depressive and anxiety disorders (Brown, Campbell, Lehman, Grisham, & Mancill, 2001; Mineka, Watson, & Clark, 1998). Recent evidence even suggests the existence of a general psychopathology factor, further challenging the notion that psychiatric conditions are discrete categories (Caspi et al., 2014).
The second observation is that culminating empirical evidence suggests that symptom specificity in the strictest sense is rather elusive. Although the cognitive vulnerabilities are predictive of the corresponding symptoms that they are theoretically linked to, they also show associations with other symptoms that are unexpected. This pattern of results have been demonstrated in meta-analytic reviews for rumination (Aldao et al., 2010; Olatunji, Naragon-Gainey, & Wolitzky-Taylor, 2013), anxiety sensitivity (Naragon-Gainey, 2010), and intolerance of uncertainty (Gentes & Ruscio, 2011). Furthermore, when the overlap among (a) the cognitive vulnerabilities or (b) the symptom categories were controlled for, cognitive vulnerabilities still retained their associations with several symptom categories (e.g., Fergus & Wu, 2011; Hong, 2013; P. J. Norton & Mehta, 2007). 2
The third observation is the emerging focus on transdiagnostic etiologic processes. Although inferring a causal role for a cognitive vulnerability based on observed symptom specificity or nonspecificity is fraught with difficulties (Garber & Hollon, 1991), the finding that cognitive vulnerabilities predict several comorbid forms of psychopathology is suggestive of a transdiagnostic etiologic process (Caspi et al., 2014; Nolen-Hoeksema & Watkins, 2011). Indeed, ruminative style (McLaughlin & Nolen-Hoeksema, 2011; Nolen-Hoeksema & Watkins, 2011) and intolerance of uncertainty (Carleton, 2012; McEvoy & Mahoney, 2013) have been proposed as transdiagnostic vulnerability factors.
The previously mentioned observations strongly suggest that the cognitive vulnerabilities may be associated among themselves, despite differences in their theoretical bases. If symptoms of depression and anxiety tend to co-occur, it is highly plausible that the vulnerabilities supposedly responsible for those symptoms would also co-occur. Although associations among the depressogenic vulnerabilities have been documented (e.g., Haeffel et al., 2003; Hankin et al., 2005; Hong, 2013), much less is known about the associations among vulnerabilities for both depression and anxiety. Knowing how the various vulnerabilities relate to one another can provide insights into their underlying structure. Do these vulnerabilities form a single overarching factor, or do they branch into separate factors? Addressing this question is important as it might guide current understanding on the etiologic processes of depression and anxiety (e.g., a transdiagnostic versus multiple distinct etiologic pathways).
Drawing parallels from the structure of mood and anxiety disorders, one might extrapolate about the structure of the cognitive vulnerabilities—processes that precipitate those psychopathological symptoms. Watson and colleagues (Watson, 2005; Watson, O’Hara, & Stuart, 2008) have proposed that the mood and anxiety disorders be classified into three classes: (a) the distress disorders (i.e., major depression, dysthymic disorder, generalized anxiety disorder, posttraumatic stress disorder), (b) the fear disorders (i.e., panic disorder, agoraphobia, social and specific phobias), and (c) the bipolar disorders (i.e., bipolar I, bipolar II, cyclothymia). Thus, one structural model of the vulnerabilities might mirror the structure of mood and anxiety disorders proposed by Watson and colleagues. In this model, vulnerabilities could be organized on the basis of how their corresponding symptoms had been categorized. One factor might reflect vulnerabilities associated with the distress disorders (i.e., pessimistic inferential style, dysfunctional attitudes, ruminative style, and intolerance of uncertainty) and the second factor might represent vulnerabilities for fear disorders (i.e., anxiety sensitivity and fear of negative evaluation). (The third category on bipolar disorders was not relevant for this research.) Alternatively, the plausibility of a one-factor model could also be evaluated (see Hong & Paunonen, 2011). Such a model would be consistent with the structural models of psychopathology where mood and anxiety disorders are represented by a higher-order factor of internalizing psychopathology (Brown, Chorpita, & Barlow, 1998; Krueger, 1999; Krueger, Caspi, Moffitt, & Silva, 1998; Vollebergh et al., 2001).
The primary goal of the present research was to determine the degree of overlap among the cognitive vulnerabilities via a meta-analytic approach. A total of 15 summary effect sizes (i.e., correlation coefficients) could be obtained from the six cognitive vulnerabilities. The available literature was expected to vary widely across the various pairwise combinations of vulnerabilities. For example, sizeable bodies of research have accumulated for the associations among pessimistic inferential style, dysfunctional attitudes, and ruminative style, but less research has been conducted for the association, say, between dysfunctional attitudes and anxiety sensitivity. It is surprising that no systematic review of the associations among cognitive vulnerabilities has been conducted (especially for those with sizeable bodies of literature), and the current meta-analytic review sought to address this gap in the literature.
A second objective of this research was to evaluate the structure that underlies these cognitive vulnerabilities. Specifically, two structural models were tested. The one-factor model, labeled common core vulnerability model, posits a single factor underlying the associations among the vulnerabilities. The two-factor model, labeled distress-fear vulnerabilities model, specifies that vulnerabilities for distress disorders form a separate latent factor from the vulnerabilities for fear disorders. Conducting such structural analyses can inform whether these vulnerabilities share a common (versus unique) etiologic mechanism.
Method
Literature search
The literature search for relevant articles was conducted using the PsycINFO and Scopus databases (period ending March 2013). Relevant unpublished dissertations were also searched to minimize publication bias. The cognitive vulnerability constructs and their corresponding search terms (in parentheses) were as follows: pessimistic inferential style (PI; pessimistic inferential style, pessimistic explanatory style, pessimistic attributional style, Cognitive Style Questionnaire, CSQ, Attributional Style Questionnaire); dysfunctional attitudes (DA; dysfunctional attitudes, Dysfunctional Attitudes Scale, DAS), ruminative style (RU; rumination, brooding, ruminative response style, RRS, Response Style Questionnaire, RSQ), anxiety sensitivity (AS; anxiety sensitivity, Anxiety Sensitivity Index, ASI, Anxiety Sensitivity Profile, ASI-3), intolerance of uncertainty (IU; intolerance of uncertainty, Intolerance of Uncertainty Scale, IUS), and fear of negative evaluation (NE; fear of negative evaluation, negative evaluation sensitivity, FNE).
The search was conducted using pairwise combinations of the search terms, which resulted in 1,896 distinct records (i.e., duplicates gathered from the two databases were omitted). In addition, the reference lists of studies were consulted to identify appropriate studies (n = 58). Unpublished data sets were solicited from 18 researchers who had published extensively in some of these cognitive vulnerabilities (i.e., more than three publications in this search). However, only two replied saying that they did not have any unpublished data.
Selection of studies
Inclusion criteria for articles were as follows: (a) selected established operationalizations of the cognitive vulnerability constructs were used (see Table 1), (b) correlation between two vulnerability measures and the sample size were reported, (c) only college student and adult samples were used (i.e., studies using child or adolescent participants were excluded), and (d) articles were published in English. An initial screening of abstracts resulted in 177 articles being retained for further eligibility assessment. Among these articles, 6 were excluded because they were review articles that did not report empirical data, 16 were excluded because they reported data based on child or adolescent samples, and 47 were excluded due to the lack of the appropriate vulnerability measure. A total of 29 articles were identified to have insufficient information for computing the effect size (i.e., Pearson’s correlation). Email requests for additional information were sent out if the article was published within the past 10 years. Among the 24 authors emailed, 5 indicated that they no longer had access to the data and 5 replied with the requested information. Hence, 24 articles were eventually excluded based on this criterion.
Cognitive Vulnerability Constructs and Associated Measures
One assumption underlying meta-analysis is the independence of individual observation; that is, a single study (or sample) should contribute only one effect size. Although a three-level meta-analysis can be used to handle the dependent effect sizes, it is still challenging to integrate the three-level meta-analysis into meta-analytic structural equation modeling. Several strategies were used to ensure the independence of this meta-analytic data set. First, if correlations were reported at the subscale level in a study, then the average correlation was estimated at the scale level for inclusion. For example, some researchers (e.g., Gibb, Alloy, Abramson, Beevers, & Miller, 2004; Hankin, Lakdawalla, Carter, Abela, & Adams, 2007) reported the correlations between subscales of CSQ (i.e., generality of attributions for causes of negative event, negative inferences about the self, and negative consequences following the negative events) and subscales of DAS (i.e., performance evaluation and approval by others). Because this meta-analysis was primarily concerned with the associations among the cognitive vulnerability variables at the scale level, subscale level correlations were averaged, using Fisher’s r-to-Z transformation, to derive an overall effect size (i.e., correlation between CSQ and DAS). Because the degree of correlation among the effect sizes was not reported in the studies, several values (rs = .20, .50, and .80) were used to calculate the sampling variances of the averaged effect size in each study in a sensitivity analysis. The results were nearly identical.
Second, for studies that administered measures to the same sample for more than one time point, only the effect size from the first time point was coded. Third, articles that reported data from the same or overlapping samples were identified. In these cases, the decision rule was to (a) select the article with the largest sample size for inclusion or (b) select published reports over dissertations if the same data were reported. For instance, several articles reported data regarding the association between CSQ and DAS based on data collected from the Cognitive Vulnerability to Depression project. However, only the CSQ-DAS correlation coefficient from Gibb et al. (2004) was included because its sample size was the largest (N = 5,002). A total of 11 articles were excluded on the basis of dependent samples.
Effect size coding
Applying the inclusion and exclusion criteria yielded 73 articles, 84 samples, and 159 independent effect sizes for this meta-analysis. For each of these articles, the effect size, sample size, sample type, cognitive vulnerability measure used, internal consistency reliabilities of the measures, study design, mean (or median) age of sample, and proportion of female participants were coded (see Table 2). The effect size of interest was the Pearson’s correlation coefficient between two cognitive vulnerability measures. The internal consistency reliabilities were recorded so as to estimate the correlation coefficients corrected for unreliability. In the event the coefficient alpha of a measure was not reported in a study, it was estimated using the averaged reliability computed from other studies in the meta-analysis that had administered the same measure. In addition, sample type (clinical versus nonclinical samples), mean age, and percentage female in sample were considered as potential moderator variables that may influence the magnitude of the summary effect sizes. All studies were coded by a trained research assistant. A subset of 30 studies were randomly selected and coded by the first author (R. Y. Hong) to evaluate interrater agreement, which was 97%.
Studies Contributing Effect Sizes to Meta-Analysis; Organized by Cognitive Vulnerability Combination Pairs
Note: For sample type, 1 = college students, 2 = community participants, and 3 = treatment-seeking participants. % female = percentage of female participants in the sample. Refer to Table 1 for full names of cognitive vulnerability measures.
Meta-analytic procedure
Meta-analyses were conducted using the procedures proposed by Lipsey and Wilson (2001). Pearson’s r coefficients coded from the studies were converted to Zr using Fisher’s r-to-Z transformation (Hedges & Olkin, 1985). Each transformed effect size was then weighted by the inverse variance to account for the different sample sizes across studies. The inverse variance weight is defined as the inverse of the squared standard error of Zr. Mean effect sizes and confidence interval values were converted back to r values using the Z-to-r transformation for clarity of presentation.
In addition to the estimation of observed mean effect sizes, mean effect sizes corrected for attenuation were also computed. Because scale reliabilities were coded for each study, each observed correlation was corrected for attenuation due to unreliability. The correction was performed according to the following formula (Lipsey & Wilson, 2001): ρ = r / √(αCV1)(αCV2), where ρ is the corrected correlation, r is the observed correlation, and αCV1 and αCV2 are the internal consistency reliabilities associated with cognitive vulnerability variables. The corresponding inverse variance weights for the corrected correlations were also adjusted using the following formula: w’ = w(αCV1)(αCV2). w’ is the adjusted inverse variance weight, w is the inverse variance weight for the uncorrected correlation, and αCV1 and αCV2 are the reliabilities of the measures. As in the case for the observed mean effect sizes, the corrected correlations were subjected to the same meta-analytic procedures in deriving the summary estimates.
All meta-analytic procedures were conducted using a random effects model. Compared to a fixed effect model, the random effects model assumes that variability between effect sizes is not due to sampling error only—that variability also includes real between-study differences (see Hedges & Vevea, 1998, for relevant theories and examples). Furthermore, random effects models produce more appropriate confidence intervals than those of fixed effect models when the effect sizes are heterogeneous. More important, random effects models allow generalizing the findings beyond the studies included in the meta-analysis, whereas fixed effect models cannot generalize beyond the included studies. Because studies in this meta-analysis were expected to vary in terms of characteristics such as sample type (e.g., college student versus treatment-seeking samples) and mean sample age, a random effects model was more appropriate.
Publication bias
Orwin’s (1983) fail-safe N statistic was examined for each mean effect size to evaluate potential publication bias (i.e., inflation of mean effect size because nonsignificant findings are typically not published and thus excluded from the meta-analysis). The fail-safe N statistic indicates the number of unpublished studies with an average effect size of zero that would be needed for inclusion in the meta-analysis to reduce the observed effect size to a negligible magnitude. In the present analysis, the criterion effect size of r = .10 was used (Orwin, 1983).
Heterogeneity
Heterogeneity of effect sizes were assessed using the Q and I2 indices. The Q statistic tests whether the population effect sizes are the same (homogeneity of effect sizes). A significant Q statistic indicates the presence of heterogeneity. The I2 statistic is the percentage of variance that is attributable to true between-study variation, with 25%, 50%, and 75% considered as thresholds for low, medium, and high heterogeneity, respectively (Higgins, Thompson, Deeks, & Altman, 2003). Substantial heterogeneity of effect sizes indicates that moderators may be used to explain this heterogeneity.
Moderation analysis
One potential categorical moderator for the effect size estimates was sample type—nonclinical versus clinical samples (i.e., the former constituted college student and community samples whereas the latter comprised treatment-seeking participants). A categorical moderation analysis was conducted by deriving separate effect sizes based on subgroups and testing the variability of effect sizes within each subgroup as well as the differences between the subgroups. Moderation is quantified by a significant between-group Q statistic, suggesting that differences among mean effect sizes across subgroups cannot be attributed to solely sampling error. Variation among groups, Q(b), is distributed as a chi-square test with c – 1 degrees of freedom (where c = number of groups).
Two continuous variables, sample mean age and the proportion of females in the sample, were also considered as potential moderators of the summary effect sizes. Moderation analyses were performed using mixed-effects meta-analysis. Weighted method of moments was used to estimate the heterogeneity variances. Due to the exploratory nature of these analyses, only one predictor variable (either age or proportion of females) was used for each meta-regression analysis. Presence of moderation is quantified by (a) an R2 statistic that indicates the amount of between-study variation explained by the regression model to the total variation and (b) statistical significance of the standardized regression coefficient as evaluated using a Z test (Borenstein, Hedges, Higgins, & Rothstein, 2009).
Meta-analytic structural equation modeling
The two-stage structural equation modeling (TSSEM) approach developed by Cheung and Chan (2005, 2009) uses a structural equation model approach to conduct meta-analytic structural equation modeling. This two-stage meta-analytic structural equation modeling approach overcomes some of the limitations of traditional univariate methods. These limitations include the choice of an appropriate sample size, handling of missing correlations, and the correct analysis of correlation matrices using structural equation models (see Cheung & Chan, 2005, for a discussion of these limitations). The random effects extension by Cheung (2014) was used in this study. Correlation matrices were combined into a pooled correlation matrix using a random effects model in the first stage of analysis. In the second stage, confirmatory factor analyses for the models were fitted using weighted least squares with the average correlation matrix and the asymptotic covariance matrix of the pooled correlation matrix (obtained in Stage 1 of analysis incorporating the random effects) as the weight matrix. Several recent applications of the random effects TSSEM have illustrated the usefulness of this approach in testing and comparing theories from a pool of empirical studies (e.g., de Wit, Greer, & Jehn, 2012; Murayama & Elliot, 2012; S. Norton, Cosco, Doyle, Done, & Sacker, 2013). As an illustrative example, S. Norton et al. (2013) applied meta-analytic structural equation modeling to study the factor structure of the Hospital Anxiety and Depression Scale. Ten possible factor structures were identified in the literature. These authors synthesized 28 independent samples from 21 studies with the TSSEM approach and found that the bifactor structure consisting of a general distress factor and two group factors (i.e., anxiety and depression) fitted the data best.
Two sets of data were used in these analyses, one with the correlations corrected for attenuation due to unreliability and one without any corrections. The analysis conducted on the corrected correlation matrices resulted in a nonpositive definite asymptotic covariance matrix in Stage 1; this excluded further analysis in Stage 2. Thus, results reported here were based on analyses conducted on uncorrected correlation matrices. Substantive conclusions have been shown to be relatively unaffected by corrections for attenuation due to unreliability (Michel, Viswesvaran, & Thomas, 2011).
Statistical software
The meta-analysis was conducted using Comprehensive Meta-Analysis Version 2 (Borenstein, Hedges, Higgins, & Rothstein, 2005) and SPSS macros written by Wilson (2010). The TSSEM was conducted using the metaSEM package developed by Cheung (2013) in the R statistical environment (R Development Core Team, 2013).
Results
Mean effect sizes
Table 3 presents the results of the 15 meta-analyses, each summarizing the relation between two cognitive vulnerability variables. As seen, all confidence intervals surrounding the uncorrected mean correlations do not include zero, implying that those correlations are statistically significant at α = .05. These mean effect sizes ranged between .34 (for PI-RU) and .58 (for DA-NE), conforming to magnitudes typically considered as moderate to large (Cohen, 1992). The corrected effect sizes ranged between .40 (for PI-RU and PI-AS) and .64 (for DA-NE). Fail-safe N statistics indicated that all 15 meta-analyses required a larger number of null-effect studies (in comparison to the number of included studies per analysis) to reduce the observed mean effect size to r = .10. Even for the case of RU-NE where there were only three studies, the fail-safe N for that particular meta-analysis was 9. This suggested that it was quite unlikely that the observed mean correlation of .40 would be a spurious finding. Nonindependence of data did not allow for tests of differences in mean effect sizes across the various vulnerability combinations.
Meta-Analyses of the Vulnerability Combinations
Note: k = number of studies; N = aggregate sample size; r = uncorrected mean weighted effect size; CI = confidence interval; ρ = corrected mean weighted effect size; Q = heterogeneity statistic; I2 = true heterogeneity percentage; FSN = Orwin’s fail-safe N, the number of studies with average effect size of 0 necessary to reduce the observed mean effect size to r = .10. See Table 2 for combination names.
p < .05. **p < .01.
Moderation analysis
Heterogeneity of effect sizes, as indexed by the significant Q statistics (p < .01), was observed for following pairwise vulnerability combinations: PI-DA, AS-IU, and AS-NE. The distribution of effect sizes for these three vulnerability combinations was also considered highly heterogeneous based on the benchmark of 75% on the I2 statistic. Therefore, moderation analyses were conducted for these three associations.
Categorical moderator analysis was done to examine whether there were differences in mean observed correlations obtained from nonclinical (i.e., college students and community-dwelling participants) versus clinical (i.e., treatment-seeking participants) samples. One study reported a sample consisting of both normative and clinical participants (Abela, Payne, & Moussaly, 2003); its sample was coded as clinical for the purpose for this moderation analysis. In these random effects models, separate estimates of τ2 by subgroups may not be reliable because of the small number of studies reporting on clinical samples. As recommended by Borenstein et al. (2009, p. 163), the τ2 estimates were pooled across subgroups to derive a more accurate τ2 value that would be applied to both subgroups (clinical and nonclinical).
As seen in Table 4, the mean observed correlation obtained from nonclinical samples did not differ from its counterpart derived from clinical samples for the PI-DA and AS-IU combinations. For the AS-NE association, the Q(b) test of significance yielded a p value of .07, suggesting a trend toward a difference in mean observed effect sizes between nonclinical (r = .48) and clinical (r = .34) samples.
Categorical Moderator Analysis by Sample Type
Note: k = number of studies; r = uncorrected mean weighted effect size; CI = confidence interval; Q(b) = Q test for between-group variance. See Table 2 for combination names.
Table 5 presents the results obtained from the meta-regression analyses using sample mean age and proportion of female in sample as continuous moderators. As before, meta-regression analyses were conducted only for the three vulnerability combinations that had demonstrated substantial heterogeneity (i.e., PI-DA, AS-IU, and AS-NE). Across the three vulnerability combinations, mean age and the proportion of females failed to account for the heterogeneity of effect sizes, with one exception. The proportion of females was a significant moderator for the heterogeneity of effect sizes observed for AS-NE. Specifically, greater representation of females in samples was linked to a larger magnitude of association between anxiety sensitivity and fear of negative evaluation (β = .64, p < .001).
Continuous Moderator Analysis
Note: R2 = proportion of between-study variance accounted for; β = beta coefficient; Z = Z test of significance for β; prop female = proportion of female participants in sample. Separate meta-regression analyses were conducted for age and proportion of females, respectively. See Table 2 for combination names.
p < .001.
Confirmatory factor analysis
The first stage of the TSSEM procedure involved combining correlation matrices (or coefficients, for studies that yielded only one effect size) into a pooled matrix using a random effects model (Cheung, 2014). Although most studies do not provide complete data on the associations among the six vulnerability variables, this procedure was able to accommodate this situation of missing data with maximum likelihood estimation and, at the same time, allow for the total sample size to be used. In contrast to the traditional methods of pooling the correlation matrices where the appropriate sample size was determined in an arbitrary fashion (e.g., using the arithmetic mean, harmonic mean, or total), Cheung and Chan (2005) argued that the TSSEM approach eliminates this ambiguity by using only the total sample size. In the current study, the total sample size was 23,027 across all studies.
The pooled uncorrected correlation matrix can be found in Table S1 in the Supplementary Materials available online. The six cognitive vulnerabilities were moderately to strongly correlated with one another, ranging between .35 and .61. These estimated correlation coefficients were very similar to those presented in Table 3 (column 3). Slight variations in the magnitudes of the correlation coefficients were due to the different methods used in pooling the effect sizes. The coefficients in Table 3 were derived using the univariate method (i.e., using Fisher’s z transformed scores; Hedges & Olkin, 1985), whereas the corresponding coefficients in Table S1 were derived via the multivariate method proposed by Cheung (2014; see also Cheung & Chan, 2005). The latter is considered to be a better approach to fitting structural equation models because it takes into account the dependence among correlation coefficients (Becker, 2000). The I2 on the average correlations ranged from 0% to 85.5%, with a median of 26.7%. These values indicated that a random effects model was more appropriate than a fixed effect model for the data.
Two confirmatory factor analytic models were fitted using the pooled uncorrected correlation matrix in Table S1. The following fit indices were used to evaluate model fit: comparative fit index (CFI), Tucker–Lewis index (TLI), standard root mean square residual (SRMR), and root mean square error of approximation (RMSEA). For good model fit, the value of CFI and TLI should ideally be greater than .95, SRMR should be less than .08, and RMSEA should be less than .06 (Kline, 2010). The one-factor common core vulnerability model yielded excellent fit indices, χ2(9, N = 23,027) = 35.24, p < .01, CFI = .99, TLI = .98, SRMR = .04, RMSEA = .01. The two-factor distress-fear vulnerabilities model also obtained excellent fit indices, χ2(8, N = 23,027) = 35.23, p < .01, CFI = .99, TLI = .98, SRMR = .04, RMSEA = .01. However, the estimated factor correlation in the two-factor model was 1.004, which was theoretically unacceptable.
How should this improper solution (i.e., a Heywood case) of a correlation larger than 1 be explained? Rindskopf (1984) has argued that improper solutions on the correlations may happen if the population correlation is “close to one.” According to Rindskopf, “A correlation greater than .95 is ‘close to one,’ and a correlation greater than .90 often is” (p. 111). Going by this argument, the improper solution could be a reflection of a population correlation that was close to unity. In other words, the data were generated from the one-factor model. Furthermore, according to Chen, Bollen, Paxton, Curran, and Kirby (2001), the occurrence of improper solutions can be due to an instance of sampling fluctuations (i.e., a chance effect) or model misspecification (i.e., an incorrect model). One strategy proposed by Chen et al. was to compare the unconstrained model with the improper solution (i.e., the two-factor model) to the constrained model without the improper solution (i.e., the one-factor model). If the test between these two models is not significant, it may be concluded that the improper solutions are due to sampling fluctuations. They suggested that the constrained model (i.e., one-factor model) “should be interpreted as usual” (Chen et al., 2001, p. 504). The chi-square difference test was not significant, Δχ2(1, N = 23,027) = 0.01, p = .92, suggesting that the improper solution could be attributed to a chance effect. It should also be noted that, given the huge sample size here, the nonsignificant result could not be attributed to a small sample size.
Based on the evidence, the two latent factors of the distress-fear vulnerabilities model were not distinguishable and thus should be regarded as a single factor. Therefore, the common core vulnerability model was selected as the model that best fitted the meta-analytic data. The factor loadings are as follows: pessimistic inferential style (.59), dysfunctional attitudes (.70), ruminative style (.57), anxiety sensitivity (.67), intolerance of uncertainty (.81), and fear of negative evaluation (.64). Intolerance of uncertainty was the cognitive vulnerability that had the strongest factor loading.
Discussion
The present findings indicated that cognitive vulnerabilities were moderately to strongly associated (mean correlations approximately between .35 and .60). Across the 15 vulnerability combinations, 3 summary effect sizes (i.e., PI-DA, AS-IU, and AS-NE) showed some degree of heterogeneity. The effect sizes did not differ as a function of sample type (nonclinical versus clinical), age, or proportion of females within a sample. The only exception was that the proportion of females emerged as a significant moderator for the variation in effect sizes seen for AS-NE (i.e., studies with a higher proportion of females yielded stronger correlations). This is in line with the observation that females are more prone to panic and social anxiety disorders than are males (Kessler et al., 1994; White & Barlow, 2002). Overall, however, effect sizes of various vulnerability combinations did not show substantial heterogeneity. Given the substantial intercorrelations among the vulnerabilities, the possibility of an underlying factorial structure was evaluated. A one-factor model was found to fit the meta-analytically derived data, suggesting the existence of a common core latent factor among the six vulnerabilities.
The six cognitive risk variables have been traditionally seen as disorder-specific vulnerabilities that predict a particular form of psychopathology. However, such a view is increasingly being challenged. The high rates of comorbidity for mood and anxiety disorders (Brown et al., 2001; Mineka et al., 1998) and the lack of symptom specificity (e.g., Aldao et al., 2010; Gentes & Ruscio, 2011; Naragon-Gainey, 2010) have raised serious concerns about the uniqueness of these vulnerabilities. From the perspective of the current findings, it is thus not surprising to find weak evidence of symptom specificity for cognitive vulnerabilities. For example, the strong association between intolerance of uncertainty and depression (Gentes & Ruscio, 2011) could potentially be explained by the former’s links with putative depressogenic vulnerabilities (e.g., dysfunctional attitudes, ruminative style).
Because of the traditional focus on symptom specificity, each of these cognitive vulnerabilities has been typically studied in isolation and issues on how one vulnerability may be the same as, or distinct from, other related vulnerabilities and psychopathology are seldom being addressed. Researchers will need to take on a more broad-based view with regard to cognitive vulnerabilities, recognizing that they can overlap substantially, and find ways to account for this overlap when delineating etiologic processes of psychopathology. This is not a call to abandon research on any single cognitive vulnerability. The existence of a shared component among the vulnerabilities does not necessarily imply that the constituent variables are associated to anxiety and depression in the same way. Each of the individual cognitive vulnerabilities may still possess disorder-specific features that are crucial in understanding the etiology of a particular disorder (see Garber & Hollon, 1991). Comorbidity between anxiety and depression may be best modeled using a combination of shared and distinct vulnerability factors (Mathew, Pettit, Lewinsohn, Seeley, & Roberts, 2011).
The observation that the cognitive vulnerabilities loaded onto a single latent factor suggests that they share a common core. Extracting commonalities across all vulnerabilities may shed light on the nature of this core. First, intolerance of uncertainty had the strongest factor loading—implying that a fundamental fear of the unknown (Carleton, 2012) may feature heavily in this common core. This element of unknown may encompass external environmental uncertainties and threats and an individual’s internally oriented uncertainty about his or her own resources to deal with such threats (cf. Lazarus & Folkman, 1985). A second common feature that permeates the cognitive vulnerabilities, in varying degrees, is a pattern of negative repetitive thinking that individuals find it intrusive and hard to control (McEvoy, Watson, Watkins, & Nathan, 2013; Watkins, 2008). Negative repetitive thinking, as a form of shared common feature, stripped of diagnosis-specific elements, has been found to be predictive of a range of depression and anxiety symptoms (Mahoney, McEvoy, & Moulds, 2012; McEvoy, Mahoney, & Moulds, 2010). A third common element may be the tendency for cognitive distortions (Beck, 1983, 1987; Scher et al., 2005) that often magnify the negative implications of events (e.g., catastrophizing, overgeneralization, exaggeration; see Burns, 1980). Collectively, these common features that are present in most, if not all, cognitive vulnerabilities may constitute a general pattern of negative repetitive thinking pervaded with themes of uncertainty and uncontrollability, often made worse through cognitive distortions. This is somewhat akin to the generalized psychological vulnerability—a general sense of uncontrollability and unpredictability over the external environment and one’s internal emotions—proposed in Barlow’s (2000, 2002) triple vulnerability model of emotional disorders.
An alternative perspective on the common core latent factor draws inspiration from dispositional-trait theories of psychopathology (e.g., Brown et al., 1998; Clark, 2005; Clark, Watson, & Mineka, 1994; Klein, Kotov, & Bufferd, 2011; Watson & Naragon-Gainey, 2014; Zinbarg & Barlow, 1996). Essentially, these theories posit that genetically based dispositions like neuroticism (or negative emotionality) and extraversion (or positive emotionality) confer broad and undifferentiated vulnerability to the development of mood and anxiety disorders. Recent research has shown that cognitive vulnerabilities play the role of intervening variables between broad-distal dispositions and specific-proximal psychopathological symptoms (Fergus & Wu, 2011; Hong, 2013; Hong & Paunonen, 2011; McEvoy & Mahoney, 2012; P. J. Norton & Metha, 2007; P. J. Norton, Sexton, Walker, & Norton, 2005; Sexton et al., 2003; Sutton et al., 2011; van der Heiden et al., 2010; Zinbarg et al., 2010). Hence, the common core would most likely represent the broad dispositional dimension of neuroticism, based on its robust links with the various vulnerabilities. From the standpoint of Barlow’s (2000, 2002) triple vulnerability model, neuroticism would be considered as a generalized biological vulnerability—a genetically based stable disposition to experience negative affect. Indeed, if one were to regard cognitive vulnerabilities as clinical facets of neuroticism (Watson, Kotov, & Gamez, 2006), then neuroticism would constitute this common core underlying emotional disorders (Barlow, Sauer-Zavala, Carl, Bullis, & Ellard, 2014).
The current findings are consistent with the notion of a transdiagnostic etiologic process that underlies emotional disorders. A practical implication is to formulate treatment protocols that target putative transdiagnostic etiologic processes (e.g., repetitive negative cognitions, fear of uncertainty). This could potentially curtail the proliferation of treatment manuals for different disorders, which could lead to redundancy and inefficiencies in clinical practice settings (Craske, 2012). Several transdiagnostic treatment protocols based on cognitive behavioral therapy (CBT) have recently been developed and tested with promising efficacy (e.g., Ellard, Fairholme, Boisseau, Farchione, & Barlow, 2010; P. J. Norton, 2012; P. J. Norton et al., 2013). Such CBT-based transdiagnostic treatments focus on global (as opposed to diagnosis-specific) conceptualizations of anxiety/mood difficulties combined with training for appropriate affective-cognitive responses to perceived threats or dysfunctional beliefs. This may include psychoeducation, self-monitoring, identification of underlying maladaptive beliefs (e.g., uncontrollability), cognitive reappraisal, and graduated exposure to feared cues. The pattern of repetitive and negative cognitions involving uncertainty and uncontrollability that cuts across the six vulnerabilities should lend itself well to the interventions described in these protocols. Similarly, changes in neuroticism have been documented using CBT and pharmaceutical treatments for depression (e.g., Tang et al., 2009) and mindfulness training (Krasner et al., 2009), particularly when specific behaviors deemed relevant to the trait are targeted (Magidson, Roberts, Collado-Rodriquez, & Lejuez, 2013).
This meta-analytic review has several limitations that need to be considered. First, among the 15 vulnerability combinations examined, 9 of them had fewer than 10 studies available for effect size computations. A mean effect size estimated from a small set of studies could pose the problem of unreliability due to sampling error. Although the fail-safe N statistics indicated that the summary correlations were probably robust for most of the vulnerability combinations, future research should examine the relations among vulnerabilities associated with mood and anxiety symptoms more closely. Second, the majority of the studies included in this review relied on nonclinical college student samples obtained in Western cultures. The mean correlation coefficients estimated might be inflated because nonclinical samples were unlikely to have restricted ranges on the measured vulnerabilities compared with clinical samples. In other words, clinical samples might generally endorse the high end of the vulnerabilities, creating range restrictions in the process. The moderation analysis for AS-NE (Table 4) revealed such a trend (see also Gentes & Ruscio, 2011). Although most effect sizes were not heterogeneous, the lack of clinical samples in some of the vulnerability combinations precluded potential moderation analyses by sample characteristics. The lack of differential diagnoses across the clinical samples also precluded moderation analysis by diagnosis. The current findings had limited generalizability to non-Western samples because of the lack of studies conducted in these societies. Third, individual differences in comorbidity profiles were not controlled for and how this might impact the relations among the cognitive vulnerabilities was unclear. Due to the cross-sectional nature of the studies reviewed here, there was a possibility that the comorbid symptoms of depression and anxiety accounted for the covariation among the cognitive vulnerabilities. Although causality cannot be inferred here, we assume that vulnerabilities take casual precedence over symptoms, consistent with most theoretical accounts (e.g., Alloy et al., 2000; Alloy et al., 2006; Buhr & Dugas, 2009; Nolen-Hoeksema et al., 2008).
Psychopathology researchers have long focused on the individuating and unique features associated with supposedly disorder-specific cognitive vulnerabilities. Furthermore, each of these vulnerabilities is often studied in isolation (or with vulnerabilities within the same symptom class), and little work has been done to explore systematically the common features that these vulnerabilities might share with one another. Employing the latest development in meta-analytic structural equation modeling, this meta-analytic review is the first to derive the extent of associations among a broad range of cognitive vulnerabilities implicated in mood and anxiety disorders. Not only do the vulnerabilities show moderate to strong intercorrelations, they can also be organized structurally into a one-factor model with the latent factor depicting a common core. This common core suggests the existence of a shared etiologic mechanism among the vulnerabilities, which in turn can be the target for transdiagnostic interventions.
Footnotes
Acknowledgements
We are grateful to Charlene Chen, Jonathan Phan, Grace Ong, and Yonghao Lim for their research assistance.
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Funding
This research is supported by a research grant (R-581-000-075-133) and a Staff Research Support Scheme fund from the National University of Singapore awarded to Ryan Y. Hong.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
