Abstract
The Cognitive Emotion Regulation Questionnaire–Short form (CERQ-Short; Garnefski & Kraaij, 2006) was developed to assess nine theoretically derived factors of emotion regulation. However, the psychometric properties of this measure have never been studied in a clinical sample. The present study examined the latent factor structure and construct validity of the CERQ-Short in two samples presenting for posttraumatic stress disorder treatment (N = 480). Results indicated that a six-factor solution, rather than the proposed nine factors, was the best-fitting measurement model. The original CERQ-Short factors of acceptance, positive refocusing, other-blame, and self-blame were retained. A novel perseveration factor incorporated both the original rumination and catastrophizing factors and a novel reappraisal factor incorporated items from the original positive reappraisal and putting into perspective factors. The revised six-factor measurement model provided good fit and demonstrated strong construct validity in a second clinical sample. Results support a more parsimonious six-factor CERQ-Short measurement model.
Emotion regulation (ER) is an umbrella label for a heterogeneous group of strategies to attend to, evaluate, and enhance or diminish emotional responding (Gross, 1998). ER strategies are theorized to play critical roles in the development, maintenance, and recovery from a wide range of disorders (Aldao, Nolen-Hoeksema, & Schweizer, 2010; Kring & Sloan, 2009). Beyond the clinical literature, aspects of ER have emerged as salient predictors of an array of constructs ranging from memory performance (e.g., Richards & Gross, 2006) to social competence (e.g., Butler et al., 2003), academic performance (e.g., N. P. Allan & Lonigan, 2011), and overall quality of life (e.g., Gross & John, 2003).
One widely used measure of ER strategies is the Cognitive Emotion Regulation Questionnaire (CERQ; Garnefski, Kraaij, & Spinhoven, 2001). The CERQ was designed to measure nine theoretically distinct strategies: self-blame (Anderson, Miller, Riger, Dill, & Sedikides, 1994); acceptance (Carver, Scheier, & Weintraub, 1989); rumination (Nolen-Hoeksema, Parker, & Larson, 1994); positive refocusing (Endler & Parker, 1990); refocusing on planning (Carver et al., 1989); positive reappraisal (Carver et al., 1989; Spirito, Stark, & Williams, 1988); putting into perspective (S. Allan & Gilbert, 1995); catastrophizing (Sullivan, Bishop, & Pivik, 1995); and other-blame (Tennen & Affleck, 1990). Strategies are anchored to a stressful or traumatic event. In this way, the CERQ was designed to measure how respondents regulate emotional responses to a particular, distressing event (Garnefski et al., 2001). Of note, several of the included subscales have substantial conceptual overlap. For example, the refocus on planning, positive reappraisal, and putting into perspective strategies each reference a cognitive strategy involving shifting appraisal of a situation in a way that reduces negative emotional responding. Consistent with the theoretical basis of the respective strategies, multiple studies have supported the hypothesized associations between CERQ subscales and dimensions of psychopathology (e.g., Garnefski & Kraaij, 2006; Martin, & Dahlen, 2005).
As part of their initial scale validation, Garnefski et al. (2001) hypothesized the 36 included items would represent the nine respective factors. The authors used principal components analysis (PCA) to examine the full measure’s factor structure and found that the anticipated nine factors adequately explained item covariance. Since this study, the factor structure of the full measure has been replicated using confirmatory factor analysis (CFA) in multiple samples (e.g., d’Acremont & Van der Linden, 2007; Jermann, Van der Linden, d’Acremont, & Zermatten, 2006).
More recently, Garnefski and Kraaij (2006) created a reduced version of the measure, the Cognitive Emotion Regulation Questionnaire–Short form (CERQ-Short). This 18-item scale was constructed by selecting two of the original four items from each subscale, selected based on alpha if item deleted analysis. As with the full scale, the authors used PCA to determine the factor structure of the reduced measure and found the original nine factors adequately represented item covariance. Only two additional studies have examined the latent factor structure of the CERQ-Short.
Using CFA, Kiekens, Hasking, and Boyes (2019) found that the nine-factor model provided adequate, but not good fit among a large undergraduate sample. Ireland, Clough, and Day (2017) examined the latent factor structure of the CERQ-Short among a large, nonclinical community sample. These investigators found the nine-factor model provided good fit to the data, with all but one rumination item displaying good factor loadings. However, results from the full CERQ suggested that different items would have been more appropriate for the CERQ-Short based on their stronger factor loadings in this sample, compared with the loadings reported by Garnefski and Kraaij (2006) in the CERQ-Short development study. Interestingly, in the original validation of the CERQ, two items that had been placed a priori within the positive reappraisal subscale loaded more highly on the refocus on planning subscale; they were nevertheless maintained within positive reappraisal for both the full CERQ (Garnefski et al., 2001) and the CERQ-Short (Garnefski & Kraaij, 2006). This variability in factor loadings raises questions regarding the fit of the CERQ-Short across different samples, and whether all items are performing well across studies. Notably, Ireland et al. (2017) also found considerable overlap among the latent factors of the CERQ, including correlations of >.70 between seven different pairs of factors. In particular, Ireland et al. (2017) found strong correlations between the Rumination and Catastrophizing subscales in the CERQ-Short (r = .76). These findings indicate a degree of redundancy among the subscales, which is not surprising given the content of the individual items. For example, positive reappraisal (e.g., “I tell myself that there are worse things in life”) and putting into perspective (e.g., “I think I can learn something from the situation”) items both focus on reinterpreting an experience with a more positive outlook. Likewise, rumination (e.g., “I am preoccupied with what I think and feel about what I have experienced”) and catastrophizing (e.g., “I keep thinking about how terrible it is what I have experienced”) involve perseverative negative interpretation of an experience and associated affect.
The existing evidence for the factor structure of the CERQ-Short has several limitations. First, many of the assumptions about the factor structure of the CERQ-Short rest on PCA (e.g., Garnefski & Kraaij, 2006). PCA is a useful analytic approach for reducing an observed set of highly correlated variables to a more parsimonious number of principal components. In contrast, exploratory factor analysis (EFA) examines the degree to which a given set of indicators measure underlying latent constructs. Unlike EFA, PCA does not differentiate common and unique variance and focuses on the variance of items, rather than the correlations among items (see Brown, 2006, for a discussion). Accordingly, some of the factor analytic evidence for the CERQ-Short factor structure may be misleading. This limitation is of particular concern given the conceptual overlap among some strategies and observed strong associations among theoretically distinct scales. Second, none of the three studies to examine the CERQ-Short factor structure have included clinical samples. This represents a major limitation as the expressed intent of the measure is to examine the degree to which ER strategies represent targets for intervention. Examination of the factor structure of the measure among a clinical population would provide greater information for its use.
The current study had two goals. Given the limited factor analytic evidence regarding the CERQ-Short and the absence of factor structure data using clinical samples, the first goal was to examine the latent factor structure of the CERQ-Short in two clinical samples. Based on the similarity of items across many of the subscales and the previously reported high intercorrelations, we anticipated EFA would produce a more parsimonious model with fewer than nine scales. The second goal of the study was to examine the construct validity of the measurement model produced by EFA using CFA. Given past construct validity evidence for the CERQ-Short (e.g., Garnefski & Kraaij, 2006), we expected the resulting measurement model would demonstrate strong convergent and discriminant validity.
Method
Participants and Procedures
Participants were drawn from two samples. A community clinical sample consisted of all particpants who completed the baseline assessment for two individual treatment studies for posttraumatic stress disorder (PTSD; n = 244). Details regarding participant recruitment for these trials is provided elsewhere (Sloan, Marx, Bovin, Feinstein, & Gallagher, 2012; Sloan, Marx, & Resick, 2016). This sample was racially diverse and the majority of participants identified as female (Table 1); mean age was 41.75 years (SD = 13.31). A second sample consisted of 236 male military veterans recruited for a separate PTSD treatment study; mean age for this sample was 56.20 years (SD = 11.96). Details regarding participant recruitment for this trial is provided elsewhere (Sloan, Unger, & Beck, 2016). All participants who completed the baseline assessment for these trials were included in analyses, regardless of whether they met inclusion criteria. For both samples, after participants completed informed consent procedures, a master’s or doctoral-level assessor conducted an interview consisting of a number of clinician-administered measures, followed by a battery of self-report questionnaires. Both samples were characterized by substantial co-occurring anxiety, mood, and alcohol/substance-use disorders (see Table 1). Study approval was obtained from institutional review boards at the VA Boston Healthcare System for the community sample and both the VA Boston Healthcare System and Providence VA Medical Center for the veteran sample.
Sample Characteristics.
Note. HS = high school; VT = vocational/technical school.
Measures
The CERQ-Short (Garnefski & Kraaij, 2006) is an 18-item self-report measure of multiple ER strategies. Respondents rate how often they use each strategy on a 5-point scale from almost never to almost always. Higher scores on each item indicate more frequent use of the respective strategy.
The Structured Clinical Interview for DSM-IV Disorders (SCID; Spitzer, Williams, Gibbon, & First, 1994) was administered in each trial for exclusion criteria as well as to characterize co-occurring psychopathology in each sample. Interrater reliability was high in each sample (all κs > .8). The SCID was included in this study solely to characterize the sample regarding co-occurring psychopathology.
Construct Validity Measures
The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5; Weathers et al., 2013) is a structured diagnostic interview of PTSD diagnosis and symptom severity. Clinicians rate each symptom on a 5-point scale ranging from absent to extreme. Ratings take into account frequency and intensity of each PTSD symptom. The current study used a CAPS-5 total score, in which all items are summed to reflect overall PTSD symptom severity; higher scores indicate greater severity. The CAPS-5 demonstrated strong internal consistency in the current study (α = .85); interrater reliability for the CAPS-5 was high (κ = .80). The CAPS-5 was included to examine construct validity of the CERQ-Short.
The Beck Anxiety Inventory (BAI; Beck, Epstein, Brown, & Steer, 1988) is a 21-item self-report measure of anxiety symptoms during the past week. Respondents rate the degree to which they have been bothered by each symptom on a 4-point scale ranging from not at all to severely, I could barely stand it. Items are summed to a total score reflecting overall anxiety symptoms, with higher scores indicating greater severity. The BAI has demonstrated strong construct validity (de Beurs, Wilson, Chambless, Goldstein, & Feske, 1997) and excellent internal consistency (α = .92; Beck et al. 1988), including α = .93 in the current study. The BAI was included to test construct validity of the CERQ-Short.
The Beck Depression Inventory–II (BDI-II; Beck, Steer, & Brown, 1996) is a 21-item self-report measure of depression symptoms during the past 2 weeks. Respondents rate the degree to which they have been bothered by each symptom on a 4-point scale. Items are summed to a total score reflecting overall depressive symptoms, with higher scores reflecting greater severity. The BDI-II has demonstrated strong construct validity (Steer & Clark, 1997) and strong internal consistency (α = .92; Dozois, Dobson, & Ahnberg, 1998), including α = .92 in the current study. The BDI-II was included in this study to examine construct validity of the CERQ-Short.
The Wide Range Achievement Test–4 (WRAT-4; Wilkinson & Robertson, 2006) word reading subtest is a measure of reading achievement level. The subtest consists of both letter and word reading. The standardized reading achievement score was used for the current study. The WRAT-4 has demonstrated strong reliability and validity in multiple studies (e.g., Jantz et al., 2015; Sayegh, Arentoft, Thaler, Dean, & Thames, 2014; Wilkinson & Robertson, 2006). The WRAT-4 was included to examine discriminant validity of the CERQ-Short.
The Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) is a brief screening measure of cognitive impairment. This instrument provides brief measure of multiple domains of cognitive functioning, including short-term memory, attention, concentration, and executive functioning. The MoCA has been extensively validated in both veteran (e.g., Waldron-Perrine, & Axelrod, 2012) and nonveteran (e.g., Nasreddine et al., 2005) samples. The MoCA was included to examine discriminant validity of the CERQ-Short.
Data Analytic Strategy
All analyses were conducted using Mplus version 8 (Muthén & Muthén, 1998-2017). In the first step of analyses, we examined the latent factor structure of the CERQ-Short using EFA among the community clinical sample. Oblique (geomin) rotation was used due to well-established correlations among strategies (e.g., Ireland et al., 2017). Solutions of up to 10 factors were examined using eigenvalues, fit statistics, and patterns of factor loadings. Individual items were used as indicators. In the second step of analyses, the model identified through EFA in the community sample was then examined using CFA in the veteran sample. This approach allowed for examination of the novel measurement model in an independent clinical sample. Additionally, construct validity of a resulting measurement model was examined among the veteran sample.
Given the limited number of response options, items were treated as categorical in all models (Flora & Curran, 2004; Wirth & Edwards, 2007) and parameters were estimated with the weighted least squares means and variances adjusted (WLSMV) estimator, which provides a robust χ2. Model fit was evaluated using χ2, Bentler comparative fit index (CFI), Tucker–Lewis index (TLI), and root mean square error of approximation (RMSEA). Fit statistics were collectively evaluated for each model and established criteria used to determine close fit: χ2 p values > .05, CFI and TLI ⩾ 0.95, and lower limit of the RMSEA 95% confidence interval < 0.05 (Bentler, 1990; Brown, 2006; Browne & Cudeck, 1992; Hu & Bentler, 1999; Kline, 2011). Missing data were handled using multiple imputation; consistent with guidelines proposed by Bodner (2008), 24 data sets were imputed.
Results
Factor Structure
In the first step of analyses, EFA was conducted among the community clinical sample. Eigenvalues were above 1.0 for solutions with one through six factors (Table 2), indicating a maximum of six factors should be extracted. Model fit statistics generally supported a six-factor model; both RMSEA and CFI values met or exceeded guidelines for good fit. By comparison, models with one- through five-factor solutions generally provided poor fit to the data as evidenced by all examined fit statics.
Fit Statistics for Exploratory Factor Analysis Among the Community Clinical Sample.
Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; CI = confidence interval; df = degrees of freedom.
p < .05.
Examination of factor loadings within the six-factor solution revealed substantial overlap with the original measure subscales (Table 3). Factors 1, 4, 5, and 6 reflected the exact mapping of items for the original acceptance, positive refocusing, self-blame, and other-blame subscales, respectively. Items 12, 13, and 15 had salient cross-loadings on multiple factors, indicating these items function as indicators of multiple established latent variables. Accordingly, these items were dropped from subsequent analyses. Aside from the three dropped items, all 15 retained items demonstrated salient loadings to only one factor each. Factor 2 consisted of all items from the original rumination and catastrophizing subscales, reflecting a general perseveration factor. Finally, Factor 3 consisted of both original positive reappraisal items and one putting into perspective item. Correlations among the six factors ranged from weak (e.g., r = −.03 between perseveration and reappraisal) to moderate (e.g., r = .35 between acceptance and reappraisal; Table 4). Taken together, this six-factor solution appears to offer a more parsimonious integration of the scales originally postulated.
Standardized Exploratory Factor Analysis Factor Loadings of a Six-Factor Model Among the Community Clinical Sample.
Note. Bolded items indicate salient loadings to respective factors.
p < .05.
Correlations Among Latent Variables From the Exploratory Factor Analysis Six-Factor Model Among the Community Clinical Sample.
p < 05.
In the second step of analyses, both the original nine-factor and novel six-factor measurement models were examined using CFA in the veteran sample. The original nine-factor model provided mediocre fit to the data (χ2 = 211.07, df = 99, p < .001, CFI = .91, TLI = .86, RMSEA = .07, 95% CI [0.06, 0.09]). Review of modification indices indicated allowing Item 16 (“I tell myself that there are worse things in life”) to cross-load onto both putting into perspective and positive reappraisal factors would provide the greatest model improvement. However, allowing any item to cross-load would render the measurement model underidentified due to the limited number of indicators per factor. By comparison, the novel six-factor measurement model provided good fit to the data (χ2 = 119.82, df = 75, p = .001, CFI = 0.95, TLI = 0.93, RMSEA = 0.05, 95% CI [0.03, 0.07]). These models cannot be directly compared as they include a different number of indicators. However, collective review of fit indices indicates the more parsimonious six-factor model provides superior fit to the data. Within the six-factor model, all items had significant loadings to respective factors and correlations among factors ranged from weak (e.g., r = −.02 between positive refocusing and self-blame) to moderate (e.g., r = .48 between perseveration and self-blame; see Figure 1).

Six-factor measurement model of the Cognitive Emotion Regulation Questionnaire–Short form among the veteran sample.
Discriminant and Concurrent Validity
In the final step of analyses, construct validity of the novel six-factor model was examined by correlated CERQ-Short subscales with external correlates (Table 5). First, correlations were examined between CERQ-Short subscales and discriminant validity variables cognitive functioning, reading level and age. Acceptance, reappraisal, and other-blame scales were not associated with any of the discriminant validity measures. Self-blame was significantly associated with MoCA scores, positive refocusing was significantly associated with WRAT scores, and perseveration was significantly associated with MoCA scores, WRAT scores, and age. Each of these associations was such that greater use of these strategies was associated with lower cognitive functioning, lower reading level, and younger age. However, each of these associations was small in magnitude (all rs ⩽ .25).
Fully Standardized Correlations Between Newly Proposed CERQ Subscales With Construct Validity Measures Among the Veteran Sample.
Note. CERQ = Cognitive Emotion Regulation Questionnaire; MoCA = Montreal Cognitive Assessment; WRAT-4 = Wide Range Achievement Test–4; BDI-II = Beck Depression Inventory–II; BAI = Beck Anxiety Inventory; PTSD = posttraumatic stress disorder; DSM-5 = Diagnostic and Statistical Manual of Mental Disorders–Fifth edition; CAPS-5 = Clinician-Administered PTSD Scale for DSM-5.
p < 05.
Next, correlations were examined between CERQ-Short subscales and concurrent validity variables depression, anxiety, and PTSD symptom severity. Acceptance and other-blame were not significantly associated with any measures of psychopathology. Perseveration and self-blame were significantly associated with all three forms of psychopathology such that greater use of these strategies was associated with greater symptom severity. Conversely, positive refocusing was significantly associated with depression and PTSD symptoms, and reappraisal was significantly associated with depression, such that greater use of these strategies was associated with lower symptom severity.
Discussion
This study examined the latent factor structure of the CERQ-Short, a widely used brief self-report measure of ER strategies, among two treatment-seeking PTSD samples. In contrast to the nine-factor measurement model proposed by Garnefski and Kraaij (2006), results of the EFA among a community clinical sample indicated a more parsimonious six-factor model fit the data well. This model replicated in an independent veteran clinical sample and appeared to provide better fit to the data compared with the original nine-factor model. Additionally, each of the six CERQ-Short subscales demonstrated acceptable discriminant validity and strong concurrent validity among the veteran clinical sample.
The newly proposed six-factor model overlaps considerably with the original nine-factor model proposed by Garnefski and Kraaij (2006). This model retains the original acceptance, positive refocusing, other-blame, and self-blame subscales, but combines the rumination and catastrophizing subscales into a perseveration factor, and combines the original positive reappraisal items with a putting into perspective item into a reappraisal factor. The refocus on planning items and the remaining putting into perspective item were dropped due to cross-loading on multiple latent variables. Accordingly, this six-factor model appears to provide a more parsimonious measurement model which accounts for content overlap among items.
This study is consistent with several other factor analytic studies of multiple ER strategies in finding substantial overlap among theoretically distinct constructs (e.g., Lee, Witte, Weathers, & Davis, 2015; Seligowski & Orcutt, 2015). This overlap suggests some hypothesized lower order strategies may be indicators of common underlying latent constructs (see Seligowski & Orcutt, 2015, for a discussion of emotional distancing). Another area of potential overlap is between ER strategies and psychopathology. Some strategies are explicitly described as symptoms of specific disorders (e.g., thought suppression in PTSD), others have been suggested as implicit features of certain disorders (e.g., rumination in depression; see Treynor, Gonzalez, & Nolen-Hoeksema, 2003, for a discussion). However, it is worth noting that all observed associations between CERQ-Short scales and measures of psychopathology (all rs ⩽ .43) were in a range suggestive of related but distinct constructs, rather that reflecting a common construct. Nonetheless, content overlap between measures of ER and psychopathology remains a concern in this literature.
The present study has a number of limitations worth noting. First, all participants were seeking PTSD treatment. While this represents an appropriate clinical sample for examining the psychometric properties of the CERQ-Short given the relevance of ER to PTSD (Seligowski, Lee, Bardeen, & Orcutt, 2015), results may not generalize to clinical samples with other primary disorders or nontreatment-seeking samples. Additional research among samples with a more diverse range of principal diagnoses would provide valuable contributions in this regard. Second, one of the included samples was entirely male. This limitation is of note given established sex differences in ER strategy use (e.g., Gross & John, 2003; Nolen-Hoeksema, 2012). Future research examining the invariance of various measurement models across sex would advance this literature. Third, the present study was cross-sectional and unable to examine the test–retest reliability of the revised measurement model. Such work represents an important line of future inquiry. Fourth, construct validity of the CERQ-Short was only examined in the veteran clinical sample as the two community sample studies did not have any common measures other than the CERQ-Short and each sample separately was underpowered to conduct latent variable analyses. Finally, none of the studies utilized in this article were designed to explicitly examine the psychometric properties of the CERQ-Short. Accordingly, this study was limited in available measures to examine concurrent and discriminant validity, and no measures were included to examine convergent validity. However, it is worth noting this study represents the first psychometric study of the CERQ-Short among a clinical sample.
The CERQ-Short was designed to provide a brief measure of constructs posited to function as pivotal mechanisms in the onset, maintenance, and remission of a broad range of psychopathology (see Garnefski & Kraaij, 2006, for a discussion). However, all of the existing research regarding the psychometric properties of this measure has been conducted in nonclinical samples involving community members and students. The current study advances the literature by examining the latent factor structure of the CERQ-Short in two independent clinical samples. Future work using this measure may benefit from examination of convergent validity and greater examination of discriminant validity (e.g., response bias), examining differential responses to intervention between subscales (i.e., does a particular intervention lead to greater changes in some strategies compared with others) and the differing roles of particular strategies in reducing particular forms of psychopathology. Such work would make the greatest contribution if conducted among clinical samples.
Footnotes
Authors’ Note
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Theses studies were funded by Department of Veterans Affairs (Merit I01 CX000467) and National Institute of Mental Health (R01MH095737, 1R34MH077658-01A2) grants awarded to Dr. Sloan. Dr. Lee is supported by National Institute of Mental Health award #5T32MH019836-16. Dr. Thompson-Hollands is supported by Department of Veterans Affairs (Clinical Sciences Research and Development Service) award # IK2 CX001589.
