Gender Invariance and Psychometric Properties of the Nonproductive Thoughts Questionnaire for Children

Abstract

Perseverative cognitions—rumination and worry—can contribute to mental and somatic health problems in adolescence. Girls usually report stronger rumination or worry than boys even in earlier years across the development age spectrum. Our aim was to test the gender invariance and psychometric properties of the Nonproductive Thoughts Questionnaire for Children (NPTQ-C). Study 1 involved 1,572 students (mean age = 15.39 years, SD = 2.26; 49% boy) recruited representatively from local schools. We applied confirmatory factor analysis (CFA) to test the measurement model of perseverative cognitions. The gender invariance of the measurement model and the convergent validity were analyzed as well with CFA with covariates model. The original one-factor structure of NPTQ-C was supported in our adolescent sample. Gender invariance was also confirmed. The NPTQ-C indicated high convergent validity: showing positive correlations with emotional instability and emotional symptoms. In Study 2 (385 students; mean age = 16.05; SD = 1.23; 34% boy), we tested the construct validity of NPTQ-C. The applied CFA with covariates model supported the construct validity. In summary, NPTQ-C proved to be a good instrument for measuring ruminative and worrying thoughts of adolescents, which reliably measures perseverative cognitions across genders.

Keywords

rumination worry Nonproductive Thoughts Questionnaire for Children gender invariance convergent validity psychometrics

Perseverative cognitions such as rumination and worry have become the main focus of much psychological research over the past three decades (Beckwe & Deroost, 2015; Ottaviani et al., 2016) because of its importance to mental health (Aldao, Nolen-Hoeksema, & Schweizer, 2010; Nolen-Hoeksema, 1991; Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008) and physical health (Brosschot, Gerin, & Thayer, 2006; Jellesma, Verkuil, & Brosschot, 2009; Kökönyei et al., 2015). Ruminative and worrying thoughts reflect the tendency to focus repetitively on negative aspects of oneself and on dysphoric mood states that relate either to past or future events and to failures (Nolen-Hoeksema, 1991; Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008). The repetitive occurrence of these thoughts are considered to play a significant role in internalizing disorders such as major depressive disorder (MDD) and generalized anxiety disorder (GAD; Borkovec, Ray, & Stober, 1998; Hsu et al., 2015; McLaughlin, Borkovec, & Sibrava, 2007). Furthermore, it is important in prolonged stress-related activation, which is also often accompanied with increased negative affects (Brosschot, Pieper, & Thayer, 2005; Verkuil, Brosschot, de Beurs, & Thayer, 2009).

Perseverative thoughts are usually studied in adult samples (Brosschot et al., 2006; Nolen-Hoeksema & Morrow, 1993); however, recent research has aimed to expand the focus to younger populations (Abela, Brozina, & Haigh, 2002; Garnefski, Rieffe, Jellesma, Terwogt, & Kraaij, 2007; Young & Dietrich, 2015). Abela et al. (2002) found that rumination predicted increases in depressive symptoms in third- and seventh-grade (aged 8-12 years) children, through increased levels of hopelessness and low self-esteem (Abela et al., 2002). In another study of 10-year-olds (preadolescents), rumination and worry were also positively correlated with self-reported somatic complaints (Jellesma, 2008).

Rumination and worry are often distinguished from each other and are measured separately as different constructs (Hoyer, Gloster, & Herzberg, 2009; Muris, Roelofs, Meesters, & Boomsma, 2004; Watkins, 2004); however, they share many common characteristics such as self-focused perspective, repetitiveness, or intrusiveness (Fresco, Frankel, Mennin, Turk, & Heimberg, 2002). Rumination and worry are highly correlated factors (Smith & Alloy, 2009) and play overlapping roles in the development and maintenance of MDD and GAD (Segerstrom, Tsao, Alden, & Craske, 2000). These findings provide a justification for the simultaneous measurement of both constructs.

A relatively new instrument the Nonproductive Thoughts Questionnaire for Children (NPTQ-C) was designed to capture simultaneously rumination and worry of children and adolescents (Jellesma, Meerum Terwogt, Reijntjes, Rieffe, & Stegge, 2005). The NPTQ-C was developed by Jellesma et al. (2005) using an elementary school sample consisted of 717 children (394 boys and 323 girls, mean age = 10.3 years). The questionnaire contains 10 items organized in a one-factor structure showing good reliability (Cronbach’s α = .84; Jellesma et al., 2005). Despite this, NPTQ-C has been used in only a few studies (Garnefski et al., 2007; Rieffe et al., 2007) and sometimes with a different name. Rieffe et al. (2007) called it “The Worry Questionnaire.” The reliability of the NPTQ-C was proven to be good in both a Dutch and an English sample. Cronbach alpha values in these cases were found to be above .8 (Garnefski et al., 2007; Rieffe et al., 2007). Initial work by Rieffe et al. (2007) on validity indicated that perseverative cognitions are differently associated with the components of emotional awareness. However, it is clear that work remains to be done to evaluate the psychometric properties and factor structure of the questionnaire.

Previous studies (Nolen-Hoeksema, 2012; Nolen-Hoeksema & Aldao, 2011) revealed gender differences in the tendency to engage in rumination or worry (girls being more like to do this than boys). Other studies have shown that these gender differences are already presented at an early age, for example, in second-grade children (Jose & Brown, 2008; Lopez, Driscoll, & Kistner, 2009) and in 12-year-old preadolescents (Jose & Brown, 2008). These compare well with well-known gender differences in depression and anxiety (McLean, Asnaani, Litz, & Hofmann, 2011; Parker, Fletcher, Paterson, Anderson, & Hong, 2014). Johnson and Whisman (2013), however, argued that findings from a meta-analysis show the effect sizes of the gender differences in rumination to be small, albeit the study was among adults. Despite findings that have shown significant differences in the usage of perseverative cognitions between men and women (Nolen-Hoeksema, 2012), little work has been carried out to test gender invariance in the context of the psychometric properties of NPTQ-C. It is therefore important to determine if the distinct responses given to the items by men and women reflect true gender differences, or are just the product of different meanings ascribed to scale items by different groups (Milfont & Fischer, 2010). This provides the specific rationale for our research.

Two different studies were conducted to answer our hypotheses. The aims of Study 1 were threefold: (a) to test the original one-factor structure in a representative sample of Hungarian adolescents, (b) to test the gender invariances of the NPTQ-C, and (c) to investigate the internal consistency and convergent validity of the questionnaire. The Strength and Difficulties Questionnaire (SDQ; Goodman, Meltzer, & Bailey, 1998) and The Big Five Questionnaire for Children (BFQ-C; Barbaranelli, Caprara, Rabasca, & Pastorelli, 2003) were used to test the latter. The rationale for this was twofold. First, conscious perseverative cognitions (e.g., rumination) have been found to contribute to prolonged physiological and psychological distress (Brosschot et al., 2006) and second, they show a significant relationship with personality traits (e.g., neuroticism; Roelofs, Huibers, Peeters, Arntz, & van Os, 2008; Trapnell & Campbell, 1999). Given that rumination and worry have already been linked to internalizing disorders (Aldao et al., 2010), we expected that NPTQ-C would have a significant positive relationship with the emotional symptoms factor of SDQ. In addition, based on previous empirical and theoretical findings relating to neuroticism and rumination (Nolen-Hoeksema & Davis, 1999; Roelofs et al., 2008), we hypothesized a significant positive relationship between NPTQ-C and emotional instability factor of BFQ-C.

In Study 2, our goal was to test the construct validity of NPTQ-C. The Ruminative Response Scale (RRS; Treynor, Gonzalez, & Nolen-Hoeksema, 2003) was used for this purpose. This instrument measures two distinguishable facets of rumination: brooding and reflective pondering. Brooding is considered to be a maladaptive process and can be described as a tendency to dwell on negative events and dysphoric mood states. Reflective pondering (or reflection), however, seems to be a much more adaptive process, defined as an increased, nonevaluative introspection toward one’s affects and mood states (Treynor et al., 2003).

Given this existing knowledge, we expected to find a strong positive association between NPTQ-C and brooding, and a significant, but moderate relationship with reflective pondering.

Method

Participants and Procedure

Participants for Study 1 were recruited from every primary and high school in one of the largest districts in Budapest, to participate in a large cross-sectional study examining the mental health of adolescents. All schools in the district (7 high schools and 14 primary schools) agreed to participate. The local government facilitated access to schools but did not play any role in the study. From each school, half of the parallel classes were randomly selected. Adolescents completed the questionnaires in the classroom within one class session. The sample characteristics therefore reflect the composition of participating classes. Data collection was supervised by the authors of this study and trained undergraduate students of psychology. No teaching staff were present. Participation in the study was voluntary. All parents and students were issued with written informed consent letters; 141 parents refused permission and 178 students were absent during data collection. Altogether 1,625 youth completed the questionnaires; however, 53 were excluded from the analysis due to missing data. The final representative sample comprised 1,572 students (mean age = 15.39; SD = 2.26 years; 770 boys [49%] and 802 girls [51%]).

In Study 2, adolescents were recruited from 12 Hungarian high schools by a convenience sampling method. The procedure for this study was similar to Study 1: Participants completed self-rated questionnaires in the classroom within one class session on the theme of mental and somatic health. Data collection was supervised by the authors of this study and trained undergraduate students of psychology. No teaching staff were present. Participation in the study was voluntary. All parents and students were issued with written informed consent letters. Altogether 397 youth completed the questionnaires; however, 12 were excluded from the analysis due to missing data. The final sample comprised 385 students (mean age = 16.05; SD = 1.23 years; boys [34%] and girls [66%]).

Both studies were ethically approved by the Institution Review Board of Eötvös Loránd University and the work was carried out in accordance with the Declaration of Helsinki.

Measures

Study 1

Nonproductive Thoughts Questionnaire for Children (Jellesma et al., 2005)

Perseverative traitlike thoughts were assessed by the NPTQ-C. The one-factor scale consists of 10 items such as “I am often worried”; “When I have a problem I can’t stop thinking about it.” Participants answered on a 3-point scale, ranging from 1 to 3 (1 = not true, 2 = sometimes true, 3 = often true) where higher scores reflected more nonproductive thoughts (except one reversed item, e.g., “Problems never worry me. I just solve them.”) and lower scores indicated more maladaptive thoughts. The questionnaire was translated from Dutch to Hungarian by two independent translators and the back-translated version of the questionnaire was approved by Jellesma. The Dutch and English versions are available at http://www.focusonemotions.nl/index.php/worry-rumination-questionnaire-for-children. The internal consistency of the original scale was good (Cronbach’s α = .84; Jellesma et al., 2005).

Strength and Difficulties Questionnaire (Goodman et al., 1998)

Self-reported psychological symptoms were measured by the Hungarian version of the SDQ (Turi et al., 2013). SDQ is a 25-item instrument with 5 items measuring each of the five scales: 1 = emotional symptoms (e.g., “I worry a lot”); 2 = conduct problems (e.g., “I usually do as I am told”); 3 = hyperactivity/inattention (e.g., “I am easily distracted, I find it difficult to concentrate”); 4 = peer relationship problems (e.g., “I get on better with adults”); 5 = prosocial behavior (e.g., “I am helpful if someone is hurt, upset, or feeling ill”). Participants were asked to score the items on a scale of 0 to 2 (0 = not true, 1 = somewhat true, 2 = certainly true). For each subscale, the sum score ranged from 0 to 10. Higher scores indicated more severe problems, except in the “prosocial behavior” scale where higher ratings implied more prosocial behavior. Internal consistency of the Hungarian version was adequate (Cronbach αs were ranging from .55-.74), similar to the original questionnaire (Cronbach αs were ranging from .61-.75; Goodman et al., 1998; Turi et al., 2013). In our study, the internal consistency of the five subscales was good or satisfactory (Cronbach αs were ranging from .45-.68, see Table 4).

The Big Five Questionnaire for Children

This instrument was developed by Barbaranelli, Caprara, and Rabasca (1998) to measure a five-factor model of personality in children and adolescents. The Hungarian version of the self-report BFQ-C was used and reproduced with special permission from the Publisher, OS Hungary Ltd. It consists of 65 items grouped into five factors: energy (e.g., “I like to be with others”), agreeableness (e.g., “I share my things with other people”), conscientiousness (e.g., “If I take an engagement I keep it”), emotional instability (e.g., “I worry about silly things”), and openness (e.g., “I like to know and learn new things”; Barbaranelli et al., 2003; Rózsa, 2005). Participants had to answer whether they agreed with the statements on a 5-point scale ranging from 1 (almost never) to 5 (almost always). The Hungarian adaptation of the questionnaire showed good validity and reliability (Cronbach αs were ranging from .75-.80; Rózsa, 2005). In our study, the internal consistency of all the five subscales was above .80 (see Table 4).

Study 2

Ruminative Response Scale (Treynor et al., 2013)

Rumination and its two facets brooding and reflective pondering were measured by the short version of RRS. The two-factor scale consists of 10 items such as “What am I doing to deserve this?”; “I write down what I am thinking about and analyze it.” Items are rated on a 4-point Likert-type scale of 1 to 4 (1 = never, 2 = sometimes, 3 = often, 4 = always). The sum of the scores were calculated and used in the analysis. The internal consistency of the questionnaire was found to be good (Cronbach α .77), and is adequate in the brooding and reflective pondering subscales (Cronbach αs .68 and .73, respectively).

NPTQ-C described above was also used in Study 2.

Statistical Analysis

Study 1

SPSS 22.0 and Mplus 7.3 software were used for statistical analysis in three steps. First, confirmatory factor analysis (CFA) was applied to test the original one-factor structure of the NPTQ-C. Given the robust deviation from the normal distribution, we used the technique of robust weighted least square (WLSMV) estimation (Brown, 2006; Muthén & Muthén, 1998-2010). To adjust for the hierarchical nature of our data, we defined our data set as type complex. This step allowed us to take into account the nonindependence of our observations. The classes as clusters were incorporated in our analysis.

Multiple criteria were applied to measure the goodness of fit in the CFA. This was done in three ways. First, a satisfactory degree of fit requires the comparative fit index (CFI) and the Tucker–Lewis index (TLI) to be close to .95; the model is rejected if the indices are below .90 (Brown, 2006). The root mean square error of approximation (RMSEA) was also used to assess goodness of fit. RMSEA below .05 indicates an excellent fit: a value around .08 an adequate fit and a value above .10 indicates a poor fit. It should be noted that closeness of model fit using RMSEA (CFI of RMSEA) and confidence intervals for the value of RMSEA are not available under WLSMV estimation. The index, weighted root mean square residual was also used as a method to assess the fitness of the model (Muthén & Muthén, 1998-2010).

A second step of the analysis involved testing the invariance of the factor structure across gender using the in-built measurement invariance test in Mplus 7.3. This allowed us to compare simultaneously chi squares for a configural, metric, and scalar model. In the configural model, the factor loadings and intercepts are unconstrained, except the referent indicator which is fixed at 1.0 in both groups. The means of factors in the group of boys and girls are fixed at 0, and their variances are free to vary (Bowen & Guo, 2012). In the metric model, the factor loadings were equal across groups but the thresholds were allowed to differ between them. This weak or metric invariance tests whether respondents across groups attribute the same meaning to the latent construct under study. In the strong or scalar invariance model, the factor loadings and thresholds are constrained to be equal across groups (Bowen & Guo, 2012). In order to compare nested models using the WLSMV estimator, we used the DIFFTEST procedure within Mplus (Asparouhov & Muthen, 2006) to calculate the adjusted Δχ² test.

A third step involved reliability analysis (Cronbach’s alphas) to analyze the internal consistency of the NPTQ-C. Potential errors in the estimation of reliability were eliminated with the estimation of omega coefficients.

Finally, the convergent validity of the NPTQ-C was tested with CFA with a covariates model to estimate the effect of each covariate on latent variables while controlling for the effect of other covariates.

Study 2

Construct validity of NPTQ-C was tested in Mplus 7.3, with CFA with covariates model controlling for the effects of age and gender.

Results

Study 1

Confirmatory Factor Analysis

The general CFA model indicated good fit to the data (χ² = 169.977; [degrees of freedom] df = 35; p = .0001; CFI = .977; TLI = .970; RMSEA = .050). Although the examination of the modification indices revealed that degree of fit could be increased if correlations of error variances were allowed: between Items 5 (“When I have a problem, I can’t stop thinking about it”) and 7 (“When I have a problem, I think about it all the time”); and between Items 8 (“When I try to deal with a problem, I am worried that I will only make things worse”) and 10 (“When I don’t know exactly what is happening, I usually think that bad things will happen”). Allowance for the aforementioned correlations improved the degree of fit (χ² = 85.156; df = 33; p = .0001; CFI = .991; TLI = .988; RMSEA = .032) of the indices. In this modified model, all of the standardized factor loadings were above .5 as shown in Table 1.

Table 1.

Standardized Factor Loadings From Confirmatory Factor Analysis of NPTQ-C.

Item	Standardized factor loadings
1. I am often worried.	.574*
2. When I have a problem, I often think “why me?”	.533*
3. It is difficult to forget bad things.	.678*
4. Problems never worry me. I just solve them.	−.502*
5. When I have a problem, I can’t stop thinking about it.	.701*
6. When I make a mistake, I am worried about what might happen.	.644*
7. When I have a problem, I think about it all the time.	.779*
8. When I try to deal with a problem, I am worried that I will only make things worse.	.545*
9. When I have a problem, I often think about how bad it is.	.753*
10. When I don’t know exactly what is happening, I usually think that bad things will happen.	.584*

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children.

p < .001.

Measurement of invariance

To determine whether the factor structure, factor loadings, and thresholds of the NPTQ-C are valid in both gender groups, we tested invariances. The one-factor model indicated close fit to the configural model (χ² = 129.150; f = 66; p = .001; CFI = .985; TLI = .979; RMSEA = .035). The weak or metric invariance also indicated good fit to the data (χ² = 144.775; f = 75; p = .001; CFI = .983; TLI = .980; RMSEA = .034). Finally, we also tested the strong or scalar invariance and the indices showed good fit (χ² = 154.759; f = 84; p = .001; CFI = .983; TLI = .982; RMSEA = .033). Although comparing the metric (or weak) and scalar (or strong) invariances to the configural model, we found that there were significant differences between the chi-square statistics (see Table 2).

Table 2.

Testing Measurement Invariance.

	χ²	df	RMSEA	CFI	Δχ²	Δdf	p	ΔRMSEA	ΔCFI
CFA in each group separately
Girls	74.305	33	.040	.978
Boys	58.179	33	.032	.990
Multigroup analyses to test the measurement invariance
Configural invariance	129.150	66	.035	.985	—	—	<.001
Configural versus metric invariance					28.879	9	<.001	−.001	−.002
Metric invariance	144.775	75	.034	.983	15.625	9	<.001
Metric versus scalar invariance					18.553	9	<.05	−.001	<.001
Scalar invariance	154.759	84	.033	.983	9.984	9	<.05

Note. χ² = chi-squared test; RMSEA = root mean squared error of approximation; CFI = comparative fit index; CFA = confirmatory factor analysis; df = degrees of freedom; Δχ² = χ² difference test calculated with DIFFTEST procedure.

Given these results, it is wise to follow recommendations made by Cheung and Rensvold (2002). They argue for analysis of the ΔCFI and ΔRMSEA (although the latter is highly sensitive to sample size and to model complexity) before conclusions about the lack of invariance can be determined. The hypothesized invariance should only be rejected if there is a decrease of .01 or larger in CFI and/or if there is a decrease of .015 or larger in RMSEA (Cheung & Rensvold, 2002; Dimitrov, 2010). Meade, Johnson, and Braddy (2008) recommended stronger criteria (ΔCFI ≤ .002) for measurement invariance.

Table 2 shows the ΔCFI in our study to be −.002 in the metric and <.001 in the scalar model and that the degree of the decrease does not exceed .01. Similarly, ΔRMSEA is below .015. Consequently, based on more liberal fit indices, there is evidence of configural, metric, and scalar invariance of the proposed CFA model across both gender groups.

Reliability

The internal consistency of the NPTQ-C was good (Cronbach’s α = .83), similar to the original version. To eliminate the errors in the estimation of reliability, the omega coefficient was also determined. This also indicated good internal consistency (Ω_total = .85).

Descriptive statistics

Means, standard deviations, Cronbach’s alphas, and effect sizes are shown in Table 3. We found significant differences between boys and girls in the NPTQ-C, and in all scales of SDQ. Gender differences were significant in the three scales of BFQ-C. Although the effect sizes for differences were small in magnitude. One exception was the emotional symptoms scale of SDQ and NPTQ-C where the Cohen’s ds were above 0.6.

Table 3.

Means, Standard Deviations, and Effect Sizes by Gender Group With Cronbach’s Alphas.

Variable (α)	Total sample, M (SD)	Boys, M (SD)	Girls, M (SD)	t (p)	Effect size, Cohen’s d
Nonproductive Thoughts Questionnaire for Children (α = .83)	19.64 (4.55)	18.23 (4.29)	20.99 (4.37)	12.44 (<.001)	0.64
SDQ
Emotional symptoms (α = .68)	3.13 (2.27)	2.35 (1.99)	3.86 (2.29)	13.65 (<.001)	0.70
Conduct problems (α = .45)	2.67 (1.59)	2.72 (1.67)	2.62 (1.52)	1.23 (<.001)	0.06
Hyperactivity/inattention (α = .59)	4.14 (2.09)	3.92 (2.05)	4.35 (2.11)	3.98 (<.001)	0.21
Peer relationship problems (α = .54)	2.13 (1.77)	2.34 (1.84)	1.93 (1.67)	4.55 (<.001)	0.23
Prosocial behavior (α = .64)	7.02 (4.55)	6.60 (1.92)	7.41 (1.78)	8.51 (<.001)	0.44
BFQ-C
Energy (α = .83)	48.46 (7.82)	48.24 (8.17)	48.65 (7.51)	0.89 (>.05)	0.05
Agreeableness (α = .85)	45.56 (8.12)	44.24 (8.07)	46.71 (7.99)	5.26 (<.001)	0.31
Conscientiousness (α = .86)	43.59 (8.47)	43.64 (8.44)	43.54 (8.51)	0.21 (>.05)	0.01
Emotional instability (α = .88)	36.93 (9.89)	35.20 (9.46)	38.48 (10.01)	5.77 (<.001)	0.34
Openness (α = .79)	44.15 (7.83)	44.76 (7.69)	43.62 (7.92)	2.49 (<.05)	0.15

Note. SDQ = Strength and Difficulties Questionnaire; BFQ-C = Big Five Questionnaire for Children.

Table 4 reports correlations between measures. As expected, NPTQ-C showed a significant positive correlation with the emotional symptoms scale of SDQ and with emotional instability contained within BFQ-C (r > .50). The negative correlation seen between perseverative thoughts and agreeableness also proved to be significant (r > −.50).

Table 4.

Bonferroni-Corrected Correlations Between NPTQ-C, SDQ, and BFQ-C Along With Cronbach’s Alphas.

	NPTQ-C	BFQ-C			Emotional instability	Openness	Cronbach’s alpha
	NPTQ-C	Extraversion	Agreeableness	Conscientiousness	Emotional instability	Openness	Cronbach’s alpha
NPTQ-C	—	−.200**	−.067*	−.181**	.525**	−.213**	.83
SDQ
Emotional symptoms	.652**	−.212**	−.038	−.141**	.583**	−.203**	.68
Conduct problems	.226**	.001	−.202**	−.271**	.557**	−.154**	.45
Hyperactivity/inattention	.307**	.011	−.112**	−.402**	.422**	−.272**	.59
Peer relation problems	.218**	−.312**	−.260**	−.107**	.292**	−.095**	.54
Prosocial behavior	.006	.359**	.608**	.390**	−.078**	.319**	.64
Cronbach’s alpha	.83	.83	.85	.86	.88	.79

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children; SDQ = Strength and Difficulties Questionnaire; BFQ-C = Big Five Questionnaire for Children.

p < .05. **p < .001.

Convergent validity

In order to measure convergent validity, we applied CFA with a covariates model, so we could determine the effect of each observed variable on the latent variable while controlling for the effect of other variables. We tested two models. The first model was limited to age and gender. The second model included all variables (gender, age, and the subscales of SDQ and BFQ-C) and these were entered together. This approach allowed us to investigate the impact of the factors of SDQ and BFQ-C on the NPTQ-C. The standardized regression coefficients are shown in Table 5.

Table 5.

Standardized Regression Weights Between NPTQ-C, Subscales of SDQ, and BFQ-C Controlled for Age and Gender.

Explanatory variables	Model 1	Model 2
Gender	.319***	.099***
Age	.199***	.042
SDQ
Emotional symptoms		.464***
Conduct problems		−.118*
Hyperactivity/inattention		.043**
Peer relationship problems		−.070**
Prosocial behavior		.045
BFQ-C
Energy		−.147**
Agreeableness		−.027
Conscientiousness		−.019
Emotional instability		.364***
Openness		−.013
R ²	.140	.581

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children; SDQ = Strength and Difficulties Questionnaire; BFQ-C = Big Five Questionnaire for Children.

*p <.05. **p < .01.***p < .001.

The emotional symptoms factor of the SDQ and the emotional instability factor of BFQ-C showed the strongest positive relation with the NPTQ-C. The conduct problems factor of SDQ and energy factor of BFQ-C also had a significant but weak negative effect on the ruminative/worrying thoughts.

Study 2

Descriptive statistics

Means, standard deviations, Cronbach’s alphas, and effect sizes are shown in Table 6. Significant differences were found between boys and girls in the NPTQ-C, and in both subscales of RRS. The effect sizes for differences in total rumination, brooding, and reflective pondering scales were small in magnitude, except the NPTQ-C where the Cohen’s d was above 0.6, similar to Study 1.

Table 6.

Means, Standard Deviations, and Effect Sizes by Gender Group With Cronbach’s Alphas.

Variable (α)	Total sample, M (SD)	Boys, M (SD)	Girls, M (SD)	t (p)	Effect size, Cohen’s d
NPTQ-C (α = .81)	18.80 (4.19)	17.05 (3.69)	19.71 (4.16)	6.47 (<.001)	0.67
RRS
Brooding (α = .68)	20.20 (5.08)	9.36 (2.75)	10.52 (2.82)	3.90 (<.001)	0.46
Reflective pondering (α = .73)	10.04 (3.15)	9.31 (2.90)	10.43 (3.21)	3.40 (<.001)	0.36
Total (α = .77)	10.12 (2.84)	18.72 (4.62)	20.98 (5.15)	4.28 (<.001)	0.42

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children; RRS = Ruminative Response Scale.

Correlation matrix between measures are shown in Table 7. As expected, NPTQ-C showed a significant positive correlation with all the scales and total scores of RRS.

Table 7.

Bonferroni-Corrected Correlations Between NPTQ-C, Brooding, and Reflective Pondering Subscales of RRS.

	RRS
	NPTQ-C	Brooding	Reflective pondering	Total
NPTQ-C	1.000
RRS
Brooding	0.605**	1.000
Reflective pondering	0.364**	0.418**	1.000
Total	0.561**	0.803**	0.864**	1.000

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children; RRS = Ruminative Response Scale.

p < .001.

Construct validity

In Study 2, we applied CFA with covariates model, to test construct validity of NPTQ-C. We defined the nonproductive thoughts as a latent variable and we investigated the effects of brooding and reflective pondering subscales of RRS on NPTQ-C in the effects of age and gender were controlled for. The standardized regression coefficients are shown in Table 8.

Table 8.

Standardized Regression Weights Between NPTQ-C, Brooding, and Reflective Pondering Subscales of RRS Controlled for Age and Gender.

Explanatory variables	Standardized regression weights
Gender	.194***
Age	.073
RRS
Brooding	.643***
Reflective pondering	.101*
R ²	.582

Note. NPTQ-C = Nonproductive Thoughts Questionnaire for Children; RRS = Ruminative Response Scale.

p < .05. **p < .01. ***p < .001.

While the reflective pondering factor had significant but weak impact on ruminative/worrying thoughts, the brooding factor showed strong positive association with NPTQ-C.

Discussion

The two studies presented here have several important implications. In Study 1, the factor structure, the reliability, and validity of the NPTQ-C measuring perseverative cognitions were supported in a representative adolescent sample. In this study, gender invariance of the NPTQ-C was also confirmed. In addition, the results of Study 2 supported the good construct validity of the questionnaire.

One-Factor Structure of NPTQ-C in Adolescents

Since nonproductive thoughts have significant impact on mental and physical health not only in adulthood but also in early adolescence (Abela, Vanderbilt, & Rochon, 2004; Kökönyei et al., 2015), we tested the factor structure of NPTQ-C in a school-aged sample. The CFA supported the originally proposed one-factor structure of NPTQ-C (Jellesma et al., 2005), showing good fit to our data. Based on modification indices, we allowed the correlations of error variances between Items 5 (“When I have a problem, I can’t stop thinking about it”) and 7 (“When I have a problem, I think about it all the time”), and between the Items 8 (“When I try to deal with a problem, I am worried that I will only make things worse”) and 10 (“When I don’t know exactly what is happening, I usually think that bad things will happen”) in the measurement model.

Gender Invariances

Gender differences in emotion regulation strategies have been well-established (see Nolen-Hoeksema, 2012); however, it is still not clear how these differences may account for gender differences in psychopathology. Before answering this important question, it is essential to explore whether boys and girls attribute the same meanings to questionnaire items and interpret them and the underlying latent construct in the same way. Therefore, testing measurement invariance of a scale is a prerequisite of interpreting any group differences.

Measurement invariances across gender were tested to determine whether the factor structure, the factor loadings, and thresholds of the NPTQ-C are valid in both gender groups. It is meaningful in every instrument to test if the different answers reflect true gender differences or just an artefact because the groups interpret the scale items differently (Milfont & Fischer, 2010). This is particularly important in relation to rumination and worry. Evidence suggests that there are differences in the usage of these maladaptive cognitions between men and women (Nolen-Hoeksema, 2012; Nolen-Hoeksema & Jackson, 2001), albeit the magnitude of these differences in adult samples has been shown to be fairly small (Johnson & Whisman, 2013). Our results indicate that adolescent girls tend to have more perseverative cognitions than boys do; moreover, the magnitude of this difference is not negligible (Cohen’s d = 0.63). This may infer an important moderator role of age in the gender differences found in rumination/worry.

Our results of gender invariance testing provided support for the configural model (invariant factor structure) of NPTQ-C. In contrast, given the chi-square difference statistics, there is a suggestion that invariance is not supported in the metric (variance in the factor loadings) and scalar (variance in the factor loadings and thresholds) models. On this basis, we assume that NPTQ-C shows invariance only at configural level. However, given that the chi-square difference test is often criticized because of its sensitivity to the sample size and to normal distribution (Brown, 2006), additional analyses of other indices is worthwhile (Dimitrov, 2010). Cheung and Rensvold (2002) recommend that ΔCFI, Δstandardized root mean square residual, or ΔRMSEA are investigated before conclusions are made about the lack of invariance. Decreases of .01 or more in CFI across the models provide more certainty that the hypothesis should be rejected (Chen, 2007). In light of these findings, NPTQ-C showed configural-, metric-, and also scalar-level invariances across both gender groups, indicating that NPTQ-C is a reliable instrument, which is not substantially affected by gender bias. Our results seem to concord with Carter’s (2010) research who found evidence of configural gender invariance of five instruments specifically related to worry and rumination, although these conclusions were solely based on chi-square differences, and confined to adult samples (Carter, 2010; Carter & Bates, 2013). Further studies are needed to clarify whether questionnaires measuring rumination and worry show metric or scalar gender invariance by other indices (e.g., CFI), and whether these observations are valid in adolescence and in childhood.

Internal Consistency and Convergent Validity

In terms of internal consistency, the NPTQ-C shows convincing results. Both coefficients (alpha and omega) support the reliability of the scale, which is line with other studies where the internal consistency of NPTQ-C also proved to be good (Jellesma et al., 2005; Rieffe et al., 2007).

Convergent validity of NPTQ-C was explored using CFA with a covariates model, and as expected, significant relationship were found with the emotional symptoms factor of SDQ and emotional instability factor of BFQ-C. According to these results, ruminative and worrying thoughts are associated with certain emotional symptoms such as nervousness or dysphoric mood, and with feelings of anxiety, depression, discontent, or anger. These results are in line with previous studies, emphasizing the link of rumination and worry to internalizing disorders (Aldao et al., 2010), and to some personality traits such as neuroticism (Roelofs et al., 2008).

We found that the conduct problems factor of SDQ had a significant but weak negative effect on NPTQ-C. These findings may contradict a recent study (McLaughlin, Aldao, Wisco, & Hilt, 2014) which pointed out that rumination could be related to transitions between internalizing and externalizing disorders, although the results only related to boys. There are a number of reasons why caution should be applied to the interpretation of the negative relationship between NPTQ-C and the conduct problems factor of SDQ. On the one hand, the internal consistency of the conduct problem subscale is quite low, so its reliability is questionable. On the other hand, other variables, such as emotional symptoms that were controlled for, could easily influence the direction of the relationship between perseverative thoughts and externalizing symptoms.

Furthermore, we measured content-independent perseverative thoughts. Previous studies have found that angry rumination was associated with aggression in questionnaire (Peled & Moretti, 2010) and experimental studies (Denson, Pedersen, Friese, Hahm, & Roberts, 2011). Further studies are needed to investigate the associations between conduct problems and perseverative cognitions. This research would be enhanced by using a diverse range of methods and questionnaires to capture the preservative cognitions construct.

The energy factor of BFQ-C in this study was seen to have a weak negative impact on NPTQ-C. Our results are consistent with the findings of Cox, Enns, Walker, Kjernisted, and Pidlubny (2001), who detected a moderate negative correlation between rumination and extraversion (which is theoretically equivalent to energy; Barbaranelli et al., 1998; Cox et al., 2001). It is true, however, that most of the studies have not found significant relationships between nonproductive thoughts and energy/extraversion traits (Bagby & Parker, 2001; Goldberg, 1992). That said these studies used mainly adult clinical samples (Bagby & Parker, 2001; Cox et al., 2001). Overall studies exploring the association between rumination and personality traits except neuroticism are sparse, especially among school-aged children. Koster, Fang, and Marchetti (2015) suggest that is important to explore the complex dynamics of rumination, since it could lead to social isolation and the avoidance of social stimuli; therefore, lower scores on extraversion could even be a consequence of ruminative tendencies. Further prospective studies are needed to clarify the associations between personality traits and rumination (Koster et al., 2015).

Construct Validity

The design and instruments used in Study 1 did not allow us to test construct validity of the NPTQ-C; therefore, we conducted Study 2, in which construct validity was explored using the RRS (Nolen-Hoeksema, 1991). This widely used self-report measure of rumination was selected intentionally, since it shows good reliability and validity, and it could differentiate between the adaptive (reflective pondering) and the maladaptive (brooding) facets of rumination (Treynor, Gonzalez, & Nolen-Hoeksema, 2003). In addition, instead of relying solely on general correlations, we tested the hypothesized associations using CFA with covariates modelling; therefore, we were able to test the effects of brooding on NPTQ-C while we controlled for the effects of reflective pondering and vice versa.

We found a significant positive relationship with brooding and reflective pondering subscales of RRS, although the strength of the associations between NPTQ-C and these subscales were considerably different. According to our findings, NPTQ-C is weakly associated with the more adaptive self-reflective aspect of rumination; however, it shows a strong positive relationship with maladaptive brooding. These findings are completely in line with our previous expectations, and enhance that the nature of thoughts NPTQ-C measures is rather nonproductive and maladaptive than constructive.

Although the present study does not aim to provide a large conceptual overview referring to the productive/nonproductive or constructive/unconstructive distinction of repetitive thoughts, it is worthwhile to discuss this question briefly. Previous evidence suggests that many theoretically and operationally overlapping forms of repetitive thinking exist, such as rumination, worry, anticipatory stress, perseverative cognitions, rehearsal, reflection, problem solving, or mental stimulation (Martin & Tesser, 1996; Papageorgiou & Wells, 2004; Smith & Alloy, 2009; Watkins, 2008). These processes could have constructive and unconstructive outcomes; consequently, repetitive thinking could have both consequences. However, according to the theoretical framework of Watkins (2008), we could reliably identify the unconstructive forms of repetitive thinking by three properties: (a) negative valence, (b) negative intrapersonal and situational context in which the repetitive thought occurs, and (c) abstract level of construal. Using this framework, Watkins (2008) concluded that while depressive rumination, worry and perseverative cognition could be described by negative valence, negative context, and abstract construal, reflection could be characterized by positive valence, positive context, and abstract construal. For this reason, the former constructs are considered to be unconstructive, but the latter is a potentially constructive form of repetitive thinking.

In light of the above, NPTQ-C seems to measure nonproductive/unconstructive thoughts, since most of its items are negatively valenced (e.g., “ . . . bad things will happen,” “I am worried . . . ”), and the situational context in which the thoughts occur are also negative (e.g., “when I have a problem . . . ,” “when I make a mistake . . . ”). In addition, the statements reflect an abstract level of processing, where the focus is on the desirability and importance of outcomes instead of the feasibility or planning of outcomes (e.g., “ . . . I will only make things worse,” “ . . . why me?”).

In general, our findings concur with previous research that NPTQ-C has good psychometric properties (Jellesma et al., 2005; Rieffe et al., 2007). Specifically, our findings show that NPTQ-C is also a reliable and applicable scale for use in adolescent populations. On the basis of the invariance analysis, we suggest it is also well placed in studies to detect any gender differences that might exist.

Limitations

The study has a number of limitations. First, despite most of the empirical and theoretical findings which distinguish rumination and worry (Fresco et al., 2002; Watkins, 2004), our study measured the two constructs in the same instrument. It could be argued that measuring rumination and worry together was a weakness since most of the empirical and theoretical findings place them as separate concepts. However, given the high correlations between these and their overlapping roles in the development of MDD and GAD (Aldao et al., 2010), we are convinced that this decision is well-established. A second limitation relates to our inability to confirm test–retest reliability as data collection was not repeated.

Conclusion

It is essential to detect the early signs of maladaptive thoughts so that preventive efforts can work toward the avoidance of subsequent disorders including depression, anxiety, or somatic health complaints (Jellesma et al., 2009; Muris et al., 2004). Moreover, a recent systematic review and meta-analysis (Clancy, Prestwich, Caperon, & O’Connor, 2016) identified an indirect pathway from perseverative cognitions to disease outcomes: the health behaviors. Clancy et al. (2016) found that perseverative cognitions were associated with increases in health risk behaviors such as substance use, unhealthy eating, smoking, or alcohol consumption. These findings also highlight the importance of the early mapping of nonproductive thoughts/perseverative cognitions and we think, that the availability of valid and reliable instruments designed specifically for children/adolescents are paramount to this endeavor.

This study draws attention to an under tested measurement, the NPTQ-C. This questionnaire, which was designed to capture ruminative and worrying thoughts of children and adolescents, was found to show good psychometric properties. In addition, it is able to delineate perseverative cognitions across gender if differences exist.

Future work to develop new instruments should take into account the need to understand the content of the perseverative cognitions (Martin & Tesser, 1996; Nolen-Hoeksema et al., 2008) as well as being to measure their prevalence.

Footnotes

Acknowledgements

Gyöngyi Kökönyei and Zsolt Demetrovics acknowledge financial support of the János Bolyai Research Fellowship awarded by the Hungarian Academy of Sciences.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Hungarian Scientific Research Fund (Grant Number: 111938).

References

Abela

J. R. Z.

Brozina

Haigh

E. P.

(2002). An examination of the response styles theory of depression in third- and seventh-grade children: A short-term longitudinal study. Journal of Abnormal Child Psychology, 30, 515-527. doi:10.1023/a:1019873015594

Abela

J. R. Z.

Vanderbilt

Rochon

(2004). A test of the integration of the response styles and social support theories of depression in third and seventh grade children. Journal of Social and Clinical Psychology, 23, 653-674. doi:10.1521/jscp.23.5.653.50752

Aldao

Nolen-Hoeksema

Schweizer

(2010). Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review, 30, 217-237. doi:10.1016/j.cpr.2009.11.004

Asparouhov

Muthen

(2006, May 26). Robust chi square difference testing with mean and variance adjusted test statistics. Mplus Web Notes. Retrieved from https://www.statmodel.com/download/webnotes/webnote10.pdf

Bagby

R. M.

Parker

J. D. A.

(2001). Relation of rumination and distraction with neuroticism and extraversion in a sample of patients with major depression. Cognitive Therapy and Research, 25, 91-102.

Barbaranelli

Caprara

G.V.

Rabasca

(1998). Manuale del BFQ-C Big Five Questionnaire Children. Firenze, Italy: O.S. Organizazioni Speciali.

Barbaranelli

Caprara

G. V.

Rabasca

Pastorelli

(2003). A questionnaire for measuring the Big Five in late childhood. Personality and Individual Differences, 34, 645-664. doi:10.1016/s0191-8869(02)00051-x

Beckwe

Deroost

(2015). Attentional biases in ruminators and worriers. Psychological Research, 80, 952-962. doi:10.1007/s00426-015-0703-8

Borkovec

T. D.

Ray

W. J.

Stober

(1998). Worry: A cognitive phenomenon intimately linked to affective, physiological, and interpersonal behavioral processes. Cognitive Therapy and Research, 22, 561-576. doi:10.1023/a:1018790003416

10.

Bowen

N. K.

Guo

(2012). Structural equation modeling: Pocket guide to social work research methods. New York, NY: Oxford University Press.

11.

Brosschot

J. F.

Gerin

Thayer

J. F.

(2006). The perseverative cognition hypothesis: A review of worry, prolonged stress-related physiological activation, and health. Journal of Psychosomatic Research, 60, 113-124. doi:10.1016/j.jpsychores.2005.06.074

12.

Brosschot

J. F.

Pieper

Thayer

J. F.

(2005). Expanding stress theory: Prolonged activation and perseverative cognition. Psychoneuroendocrinology, 30, 1043-1049.

13.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

14.

Carter

J. A.

(2010). Worry and rumination: Measurement invariance across gender (Doctoral dissertation, All Graduate Theses and Dissertations, Paper 805). Utah State University, Logan. Retrieved from http://digitalcommons.usu.edu/etd/805

15.

Carter

J. A.

Bates

S. C.

(2013). The Penn State Worry Questionnaire: Comparative model fit and invariance across gender. National Forum Journal of Counseling and Addiction, 2(1), 1-25.

16.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 464-504.

17.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. doi:10.1207/s15328007sem0902_5

18.

Clancy

Prestwich

Caperon

O’Connor

D. B.

(2016). Perseverative cognition and health behaviors: A systematic review and meta-analysis. Frontiers in Human Neuroscience, 10, 534. doi:10.3389/fnhum.2016.00534

19.

Cox

B. J.

Enns

M. W.

Walker

J. R.

Kjernisted

Pidlubny

S. R.

(2001). Psychological vulnerabilities in patients with major depression vs panic disorder. Behaviour Research and Therapy, 39, 567-573.

20.

Denson

T. F.

Pedersen

W. C.

Friese

Hahm

Roberts

(2011). Understanding impulsive aggression: Angry rumination and reduced self-control capacity are mechanisms underlying the provocation-aggression relationship. Personality and Social Psychology Bulletin, 37, 850-862. doi:10.1177/0146167211401420

21.

Dimitrov

D. M.

(2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43, 121-149. doi:10.1177/0748175610373459

22.

Fresco

D. M.

Frankel

A. N.

Mennin

D. S.

Turk

C. L.

Heimberg

R. G.

(2002). Distinct and overlapping features of rumination and worry: The relationship of cognitive production to negative affective states. Cognitive Therapy and Research, 26, 179-188. doi:10.1023/a:1014517718949

23.

Garnefski

Rieffe

Jellesma

Terwogt

M. M.

Kraaij

(2007). Cognitive emotion regulation strategies and emotional problems in 9- 11-year-old children: The development of an instrument. European Child & Adolescent Psychiatry, 16(1), 1-9. doi:10.1007/s00787-006-0562-3

24.

Goldberg

L. R.

(1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4, 26-42.

25.

Goodman

Meltzer

Bailey

(1998). The Strengths and Difficulties Questionnaire: A pilot study on the validity of the self-report version. European Child & Adolescent Psychiatry, 7, 125-130.

26.

Hoyer

Gloster

A. T.

Herzberg

P. Y.

(2009). Is worry different from rumination? Yes, it is more predictive of psychopathology! GMS Psycho-Social-Medicine, 6. doi:10.3205/psm000062

27.

Hsu

K. J.

Beard

Rifkin

Dillon

D. G.

Pizzagalli

D. A.

Bjorgvinsson

(2015). Transdiagnostic mechanisms in depression and anxiety: The role of rumination and attentional control. Journal of Affective Disorders, 188, 22-27. doi:10.1016/j.jad.2015.08.008

28.

Jellesma

F. C.

(2008). Somatic complaints in childhood: How they are related to children’s emotional and social functioning (Doctoral thesis). Retrieved from http://www.acqol.com.au/publications/resources/thesis-jellesma-f.pdf

29.

Jellesma

F. C.

Meerum Terwogt

Reijntjes

A. H.

Rieffe

C. J.

Stegge

(2005). De vragenlijst Non-Productieve Denkprocessen voor Kinderen [Nonproductive Thoughts Questionnaire for Children: Worrying and rumination]. Kind en adolescent, 26, 171-177.

30.

Jellesma

F. C.

Verkuil

Brosschot

J. F.

(2009). Postponing worrisome thoughts in children: The effects of a postponement intervention on perseverative thoughts, emotions and somatic complaints. Social Science & Medicine, 69, 278-284. doi:10.1016/j.socscimed.2009.04.031

31.

Johnson

D. P.

Whisman

M. A.

(2013). Gender differences in rumination: A meta-analysis. Personality and Individual Differences, 55, 367-374. doi:10.1016/j.paid.2013.03.019

32.

Jose

P. E.

Brown

(2008). When does the gender difference in rumination begin? Gender and age differences in the use of rumination by adolescents. Journal of Youth and Adolescence, 37, 180-192. doi:10.1007/s10964-006-9166-y

33.

Kökönyei

Jozan

Morgan

Szemenyei

Urban

Reinhardt

Demetrovics

(2015). Perseverative thoughts and subjective health complaints in adolescence: Mediating effects of perceived stress and negative affects. Psychology & Health, 30, 969-986. doi:10.1080/08870446.2015.1007982

34.

Koster

E. H. W.

Fang

Marchetti

(2015). Self-regulation through rumination: Consequences and mechanisms. In Gendolla

G. H. E.

Tops

Koole

S. L.

(Eds.), Handbook of biobehavioral approaches to self-regulation (pp. 371-383). New York, NY: Springer.

35.

Lopez

C. M.

Driscoll

K. A.

Kistner

J. A.

(2009). Sex differences and response styles: Subtypes of rumination and associations with depressive symptoms. Journal of Clinical Child & Adolescent Psychology, 38, 27-35. doi:10.1080/15374410802575412

36.

Martin

L. L.

Tesser

(1996). Some ruminative thoughts. In Wyer

R. S.

(Ed.), Ruminative thoughts: Advances in social cognition (Vol. IX, pp. 1-48). Mahwah, NJ: Lawrence Erlbaum.

37.

McLaughlin

K. A.

Aldao

Wisco

B. E.

Hilt

L. M.

(2014). Rumination as a transdiagnostic factor underlying transitions between internalizing symptoms and aggressive behavior in early adolescents. Journal of Abnormal Psychology, 123, 13-23. doi:10.1037/a0035358

38.

McLaughlin

K. A.

Borkovec

T. D.

Sibrava

N. J.

(2007). The effects of worry and rumination on affect states and cognitive activity. Behavior Therapy, 38, 23-38. doi:10.1016/j.beth.2006.03.003

39.

McLean

C. P.

Asnaani

Litz

B. T.

Hofmann

S. G.

(2011). Gender differences in anxiety disorders: Prevalence, course of illness, comorbidity and burden of illness. Journal of Psychiatric Research, 45, 1027-1035. doi:10.1016/j.jpsychires.2011.03.006

40.

Meade

A. W.

Johnson

E. C.

Braddy

P. W.

(2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592.

41.

Milfont

T. L.

Fischer

(2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3, 111-121.

42.

Muris

Roelofs

Meesters

Boomsma

(2004). Rumination and worry in nonclinical adolescents. Cognitive Therapy and Research, 28, 539-554. doi:10.1023/B:COTR.0000045563.66060.3e

43.

Muthén

L. K.

Muthén

B. O.

(1998-2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

44.

Nolen-Hoeksema

(1991). Responses to depression and their effects on the duration of depressive episodes. Journal of Abnormal Psychology, 100, 569-582.

45.

Nolen-Hoeksema

(2012). Emotion regulation and psychopathology: The role of gender. Annual Review of Clinical Psychology, 8, 161-187. doi:10.1146/annurev-clinpsy-032511-143109

46.

Nolen-Hoeksema

Aldao

(2011). Gender and age differences in emotion regulation strategies and their relationship to depressive symptoms. Personality and Individual Differences, 51, 704-708. doi:10.1016/j.paid.2011.06.012

47.

Nolen-Hoeksema

Davis

C. G.

(1999). “Thanks for sharing that”: Ruminators and their social support networks. Journal of Personality and Social Psychology, 77, 801-814.

48.

Nolen-Hoeksema

Jackson

(2001). Mediators of the gender difference in rumination. Psychology of Women Quarterly, 25, 37-47. doi:10.1111/1471-6402.00005

49.

Nolen-Hoeksema

Morrow

(1993). Effects of rumination and distraction on naturally-occurring depressed mood. Cognition and Emotion, 7, 561-570. doi:10.1080/02699939308409206

50.

Nolen-Hoeksema

Wisco

B. E.

Lyubomirsky

(2008). Rethinking rumination. Perspectives on Psychological Science, 3, 400-424. doi:10.1111/j.1745-6924.2008.00088.x

51.

Ottaviani

Thayer

J. F.

Verkuil

Lonigro

Medea

Couyoumdjian

Brosschot

J. F.

(2016). Physiological concomitants of perseverative cognition: A systematic review and meta-analysis. Psychological Bulletin, 142, 231-259.

52.

Papageorgiou

Wells

(2004). Nature, functions and beliefs about depressive rumination. In Papageorgiou

Wells

(Eds.), Depressive rumination: Nature, theory and treatment (pp. 3-20). Chichester, England: John Wiley.

53.

Parker

Fletcher

Paterson

Anderson

Hong

(2014). Gender differences in depression severity and symptoms across depressive sub-types. Journal of Affective Disorders, 167, 351-357. doi:10.1016/j.jad.2014.06.018

54.

Peled

Moretti

M. M.

(2010). Ruminating on rumination: Are rumination on anger and sadness differentially related to aggression and depressed mood? Journal of Psychopathology and Behavioral Assessment, 32, 108-117. doi:10.1007/s10862-009-9136-2

55.

Rieffe

Terwogt

M. M.

Petrides

K. V.

Cowan

Miers

A. C.

Tolland

(2007). Psychometric properties of the Emotion Awareness Questionnaire for children. Personality and Individual Differences, 43, 95-105. doi:10.1016/j.paid.2006.11.015

56.

Roelofs

Huibers

Peeters

Arntz

van Os

(2008). Rumination and worrying as possible mediators in the relation between neuroticism and symptoms of depression and anxiety in clinically depressed individuals. Behaviour Research and Therapy, 46, 1283-1289. doi:10.1016/j.brat.2008.10.002

57.

Rózsa

(2005). A BFQ-C pszichometriai jellemzőinek hazai vizsgálata, a normák kialakítása [Testing of the psychometric properties of BFQ-C and forming of norms]. In Barbaranelli

Caprara

G. V.

Rabasca

(Eds.), BFQ-C Big Five Questionnaire for Children, Kézikönyv (pp. 63-67). Budapest, Hungary: OS Hungary.

58.

Segerstrom

S. C.

Tsao

J. C. I.

Alden

L. E.

Craske

M. G.

(2000). Worry and rumination: Repetitive thought as a concomitant and predictor of negative mood. Cognitive Therapy and Research, 24, 671-688. doi:10.1023/a:1005587311498

59.

Smith

J. M.

Alloy

L. B.

(2009). A roadmap to rumination: A review of the definition, assessment, and conceptualization of this multifaceted construct. Clinical Psychology Review, 29, 116-128. doi:10.1016/j.cpr.2008.10.003

60.

Trapnell

P. D.

Campbell

J. D.

(1999). Private self-consciousness and the five-factor model of personality: Distinguishing rumination from reflection. Journal of Personality and Social Psychology, 76, 284-304. doi:10.1037/0022-3514.76.2.284

61.

Treynor

Gonzalez

Nolen-Hoeksema

(2003). Rumination reconsidered: A psychometric analysis. Cognitive Therapy and Research, 27, 247-259. doi:10.1023/A:1023910315561

62.

Turi

Gervai

Aspan

Halasz

Nagy

Gadoros

(2013). [Validation of the Hungarian version of the Strengths and Difficulties Questionnaire in an adolescent clinical population]. Psychiatria Hungarica, 28, 165-179.

63.

Verkuil

Brosschot

J. F.

de Beurs

D. P.

Thayer

J. F.

(2009). Effects of explicit and implicit perseverative cognition on cardiac recovery after cognitive stress. International Journal of Psychophysiology, 74, 220-228. Retrieved from http://www.sciencedirect.com/science/article/pii/S0167876009002232

64.

Watkins

(2004). Appraisals and strategies associated with rumination and worry. Personality and Individual Differences, 37, 679-694. doi:10.1016/j.paid.2003.10.002

65.

Watkins

E. R.

(2008). Constructive and unconstructive repetitive thought. Psychological Bulletin, 134, 163-206. doi:10.1037/0033-2909.134.2.163

66.

Young

C. C.

Dietrich

M. S.

(2015). Stressful life events, worry, and rumination predict depressive and anxiety symptoms in young adolescents. Journal of Child and Adolescent Psychiatric Nursing, 28, 35-42. doi:10.1111/jcap.12102