Structure,Reliability,Measurement Stability,and Construct Validity of the Polish Version of the Revised Child Anxiety and Depression Scale

Abstract

The Revised Child Anxiety and Depression Scale (RCADS) is a self-report questionnaire that aims to assess symptoms of anxiety and depressive disorders in children. Two studies were conducted to evaluate the psychometric properties of the Polish version of the RCADS. Study 1 was conducted to analyze the structural validity and reliability of the RCADS scores and Study 2 assessed the longitudinal measurement of stability over time. Data were collected from a community sample of 501 children and adolescents aged 8 to 14 years in Poland. The original 47-item version of the Polish RCADS was compared with two shortened versions: 30- and 20-item versions. Overall results revealed support for the structural and construct validity, reliability, and stability of the Polish version of the RCADS.

Keywords

RCADS construct structure validity stability

Anxiety disorders are one of the most common mental health conditions among children and adolescents that occur in more than one third of community-based adolescents and nearly half of clinically referred youth (Higa-McMillan, Francis, Rith-Najarian, & Chorpita, 2016). The prevalence of depression and anxiety symptoms among Polish children and adolescents is 25% to 28% and such reported prevalence significantly exceeds the extent of cases registered in treatment services (Braddick, Carral, Jenkins, & Jané-Llopis, 2009).

Self-report assessments are particularly important to identify a wide range of psychological problems in youth, especially when compared with other informants’ reports (such as parents, caregivers, or teachers) that indicate relative insensitivity in detecting these symptoms or until they start causing significant interference with academic performance and social functioning (Muris & Meesters, 2002). In general, internalizing symptoms are more difficult to observe by others when compared with externalizing symptoms. Children themselves have been deeper access to their inner states and thus have been found to be more accurate reporters of internalizing symptoms (Kösters, Chinapaw, Zwaanswijk, Van der Wal, & Koot, 2015). Researchers agree that “children report internalizing symptoms more frequently than adults while parents and teachers rather report externalizing symptoms” (Ederer, 2004, pp. 127-128). Moreover, children’s reports provide useful information about the individual experiences of different emotions, and they have therefore been recommended for researchers and clinicians for their utility in providing direct insight into the inner psychological world of children (Stone, Otten, Engels, Vermulst, & Janssens, 2010).

Cross-cultural studies have shown that depending on cultural context, adolescents report different levels of anxiety and depressive symptoms. There is therefore a need to develop scales that are valid and could be used for cross-cultural comparisons (Stevanovic et al., 2016). In Poland, there is a limited number of anxiety and depression measures for children and adolescents, especially self-report measures. For instance, a Polish version of the State–Trait Anxiety Inventory for Children (Spielberger, 1973) was developed by Sosnowski, Iwaniszczuk and Spielberger (1989) to measure state and trait anxiety in children between the ages of 10 and 14 years. However, the stability of the scale is low, especially for state anxiety; the scale is also quite difficult to obtain (a psychologist’s license is needed).

The Revised Child Anxiety and Depression Scale

The Revised Child Anxiety and Depression Scale (RCADS) is a 47-item, self-report questionnaire designed to assess anxiety and depressive symptoms in youth. The RCADS includes six Diagnostic and Statistical Manual of Mental Disorders, Fourth edition (DSM-IV)–based subscales: separation anxiety disorder (SAD), generalized anxiety disorder (GAD), panic disorder (PD), social phobia (SP), obsessive–compulsive disorder (OCD), and major depressive disorder (MDD). Scores are aggregated into the following two total domains: Total Anxiety Scale (sum of the five anxiety subscales) and Total Internalizing Scale (sum of all six subscales). The RCADS scales were originally developed using items from the Spence Children’s Anxiety Scale (Spence, 1998), along with items representing DSM-IV major depression symptoms and new items related to general anxiety and negative affect (DSM-IV; American Psychiatric Association, 1994). Studies in community and clinical samples have demonstrated strong psychometric properties for the RCADS as a favorable measure of major depression and anxiety disorders among youth (Chorpita, Moffitt, & Gray, 2005; Chorpita, Yim, Moffitt, Umemoto, & Francis, 2000). A strong advantage of the RCADS is its ability to screen for both anxiety and depressive symptoms. This feature makes the RCADS a valuable instrument from a clinical and prevention perspective because children with anxiety problems are more prone to also exhibit depression symptoms (Angold, Costello, & Erkanli, 1999) as their comorbidity is common (Brady & Kendall, 1992; Cummings, Caporino, & Kendall, 2014).

The RCADS can be completed via self-report (by children and adolescents) or observers report (by parents or caregivers). The only difference between the self-rated and other informant-rated version is a grammatical change from the first to the third person. Originally, the RCADS was developed in English but has since been translated into other languages: Spanish, Chinese, Danish, Dutch, French, Greek, Korean, Norwegian, Persian, Swedish, and Urdu. A Polish adaptation of the RCADS (self-report and parent-report) has been prepared recently. All translations are available for researchers and clinicians to download free of charge from the Internet (http://www.childfirst.ucla.edu/Resources.html); however, the use of norms and interpretation of t scores should be done cautiously with non-English versions, as research is still underway on these instruments (Chorpita, Ebesutani, & Spence, 2015).

Current Issues Regarding RCADS Reliability and Structure

One of the main issues regarding the RCADS is its length. Although longer instruments generally provide the most reliable estimates of true scores at a given point in time, many conditions are better served by briefer instruments that provide a reasonable estimation of either clinical status or change over time. Screening large numbers of youth to determine risk for impairment can be efficiently accomplished using a brief measure, often followed by a longer measure for the subset of youth who score positive for risk. Likewise, measuring change over time when the feedback interval is brief (e.g., each week) is far more feasible with briefer instruments. Moreover, when constructing measures, researchers are encouraged to minimize the inclusion of redundant items that have synonymous meaning but were reworded, for example, “I worry that bad things will happen to me” and “I worry that something bad will happen to me.” Thus, the development of briefer versions is in high demand because it reduces the burden on children and adolescents in addition to shortening the overall administration time (Ebesutani et al., 2012).

Development of Brief Versions of the RCADS

To date, several attempts have been made to shorten the length of the RCADS while maintaining its good psychometric properties. Muris, Meesters, and Schouten (2002) reduced the 47-item RCADS to a 25-item version. However, the shortened version did not include the Obsessive–Compulsive scale due to a problem with inconsistent loadings. Only two out of five items loaded onto the Obsessive–Compulsive scale, and therefore this dimension was discarded. As Ebesutani et al. (2012) noted, the elimination of the Obsessive–Compulsive scale runs counter to the original design strategy of the RCADS, which was to achieve a wide content diversity in the assessment of anxiety-related symptoms. The authors, therefore, pursued a slightly different strategy, using community and clinical samples. Specifically, they sought to reduce the 37-item Total Anxiety scale to 15 items (based on items that loaded on the general Anxiety factor), while maintaining the original 10-item MDD scale as in the original version. Based on the Schmid–Leiman exploratory analytic procedure—which models a broad, general factor (Anxiety) and specific content domains—Ebesutani et al. (2012) supported retaining three items from each of the five anxiety domains, along with retaining the original 10 items for the depression domain. This led to the 25-item shortened RCADS, yielding the following the scales: Total Anxiety, Depression, and Total Anxiety and Depression scales.

Authors of the translated versions of the RCADS have also made an attempt to shorten the measure. A 30-item version of the Spanish RCADS was developed by Sandin, Chorot, Valiente, and Chorpita (2010) which included five items per each of the six scales. The authors stressed that the reduced version should be composed of an equal number of items per scale to maximize the likelihood of equal reliabilities. A high correlation with the original 47-item version was found (r = .98) which means that both versions measured the same construct.

Although briefer measures serve an important function in assessment, researchers have to consider the challenges that can appear while constructing shorter measures (e.g., having too few items per scale can degrade reliability to unacceptable levels; Emons, Sijtsma, & Meijer, 2007). Therefore, the items and scales selection should be done with caution and followed by thorough analysis.

The Present Study

In the current research, we assessed psychometric properties of the Polish version of the RCADS in two studies.

Study 1 focused on the structural validity and reliability of the Polish version of the RCADS. We compared the original 47-item version (Chorpita et al., 2000) and two shortened, that is, 30-item (Sandin et al., 2010), and 20-item versions of the RCADS. In the 20-item version, the anxiety scales contained the same set of 15 items as in the shortened 25-item version by Ebesutani et al. (2012), but the MDD scale contained 5 items as in the 30-item version of the RCADS by Sandin et al. (2010).

Study 2 aimed to test whether the Polish version of the RCADS measures symptoms of anxiety consistently across three assessment waves, over a period of 18 months. We hypothesized that the results obtained with the Polish version of the RCADS would be stable over time and the changes measured with the RCADS will reflect true changes in the levels of anxiety and depression among children.

Study 1

This study aimed to assess the structural validity and reliability of the Polish version of the RCADS by analyzing and comparing different models (i.e., the original 47-item and the two shortened versions: 30-item and 20-item). For each of the models, we ran a mixed bifactor exploratory structural equation modeling (bi-ESEM) and confirmatory factor analysis (CFA) where anxiety items freely loaded on the general factor (i.e., “Broad Anxiety”). The general factor comprised five anxiety domains (SAD, GAD, PD, SP, and OCD), and these five domains were also specific factors in the bifactor model. The depression items were specified to load only on the MDD factor (see Reise, Moore, & Haviland, 2010).

Participants and Procedure

The study was conducted in a community sample of 501 Caucasian, native Polish children and adolescents (55% females) between 8 and 14 years of age (M_age = 10.89, SD_age = 1.41). The current sample comprised the following age distribution: 8-year-olds (8%), 9-year-olds (17%), 10-year-olds (21.5%), 11-year-olds (25%), 12-year-olds (23%), 13-year-olds (5%), and 14-year-olds (0.5%). Data from one child were missing. First, the institutional review board at the Psychology Institute, Cardinal Stefan Wyszyński University in Warsaw reviewed this project and provided ethical approval and permission to implement it. A random selection of public and private secondary schools based in central Poland was made. The school principals received letters stating that their school was invited to take part in a research project that aimed to assess problem behavior in children and youth. They were informed that participation in this study was voluntary and anonymous and that the results would be analyzed for scientific purposes only. Participants did not receive any payment, however, as a form of gratification, we offered workshops on psychological topics as well as a general study report. With the principals’ consent to participate, the main researcher met with parents at parent–teacher meetings during which parents received a study description with a written consent form to agree or disagree for their children to participate in this study. The positive response rate was 78%. Children were asked to provide oral consent. They were informed that participation is voluntary, anonymous, and they had the right to refuse or defer from the study at any time. Data were collected during standard, daily classes in groups of 15 to 20 children using a paper and pencil form. To standardize the procedure, the researchers (native Polish speakers) went to each school to collect the data themselves. The main teacher and two researchers were available during the data collection to ensure that the instructions and statements were clear to the participants. After giving the instruction, participants were asked not to share their answers or consult them with other classmates to ensure independence between individuals’ scores.

Measures

The Revised Child Anxiety and Depression Scale

(Chorpita et al., 2000). The RCADS is a 47-item, self-report measure of depression and anxiety symptoms in children that consists of six dimensions: SAD, GAD, PD, SP, OCD, and MDD. Scores are aggregated into two total domains: a Total Anxiety Scale (sum of the five anxiety subscales), and a Total Anxiety and Depression Scale (sum of all six subscales). Children rate how often each item applies to them on a 4-point Likert-type scale (0 = never, 1 = sometimes, 2 = often, 3 = always).

The adaptation procedure to create the Polish version of the RCADS was carried out according to the International Test Commission guidelines for translating and adapting tests in cross-cultural research (Brislin, 1986; Hambleton, 2005) which included (1) forward translation of all items from English to Polish, (2) consultation of the results in a group of developmental and cross-cultural psychologists regarding the linguistic, developmental, and cultural suitability of the questionnaire, (3) independent back-translation of all items from Polish to English, (4) submission of the back-translation to the original RCADS authors, (5) discussing authors’ comments and suggestions, introducing all recommended modifications, (6) back-translation of the modified items and repetition of Steps 4, 5, and 6 until a final version of the translated RCADS was agreed on. During the development of the Polish version of the RCADS, we made an effort to capture anxiety and depression symptoms in the most effective way possible. Except for the aforementioned standard linguistic procedures, we discussed the items in a group of developmental and clinical psychologists and asked a sample of children for a cognitive debriefing.

Results

Model Fit of the Original 47-Item and Two Shortened Versions of the RCADS

Mixed bifactor ESEM and CFA analyses were run using Mplus version 7.2 (Muthén & Muthén, 2012) with a general factor (i.e., “Broad Anxiety”) and a separate depression (MDD) factor. Since the RCADS has a 4-point Likert-type scale that produces ordinal responses, we treated the data as categorical (Rhemtulla, Brosseau-Liard, & Savalei, 2012) and we used the bigeomin orthogonal rotation method. A substantive higher item loading on the “Broad Anxiety” factor (over the specific domain) indicates that the particular item serves as a better indicator of the general anxiety domain rather than the specific anxiety domain. Inversely, a higher loading on the specific anxiety domain indicates that the particular item specifically measures the content of the intended specific anxiety domain and is relatively independent of the general domain. Results of the mixed bifactor ESEM and CFA solutions are presented in Table 1.

Table 1.

Mixed Bifactor ESEM and CFA Solutions for the Original 47-Item Version the RCADS.

Scale	Item	Abbreviated item content	Bifactor ESEM						CFA
Scale	Item	Abbreviated item content	G	F1	F2	F3	F4	F5	MDD
SAD
	5	Fears being alone at home	0.57*	−0.25*
	9	Fears being away from parents	0.49*	−0.32*
	17	Scared to sleep alone	0.62*	−0.58*
	18	Trouble going to school	0.55*	−0.28*
	33	Afraid of being in crowded places	0.65*	−0.22*
	45	Worries in bed at night	0.65*	−0.51*
	46	Scared to sleep away from home	0.50*	−0.14	0.33*
GAD
	1	Worries about things	0.43*		0.45*
	13	Worries something awful will happen to family	0.52*		0.26*
	22	Worries bad things will happen to self	0.72*		0.10	−0.52*
	27	Worries something bad will happen to self	0.79*		0.10	−0.17*
	35	Worries about what will happen	0.63*		0.25*
	37	Thinks about death	0.56*	0.23*	0.06
PD
	3	When has a problem, stomach feels funny	0.34*		0.34*	0.01
	14	Suddenly has trouble breathing for no reason	0.54*			0.40*
	24	When has a problem, heart beats really fast	0.51*			0.12*
	26	Suddenly trembles or shakes for no reason	0.63*			0.28*
	28	When has a problem, feels shaky	0.54*			0.26*
	34	Suddenly feels really scared for no reason	0.74*			0.02		−0.22*
	36	Suddenly becomes dizzy of faint for no reason	0.57*			0.26*
	39	Heart suddenly beats too quickly for no reason	0.65*			0.41*
	41	Worries will suddenly get scared for no reason	0.71*			0.07		−0.33*
SP
	4	Worries when does poorly at things	0.39*				0.36*
	7	Scared to take a test	0.47*				0.45*
	8	Feels worried when someone angry	0.47*				0.20*
	12	Worries will do badly at school work	0.59*				0.52*
	20	Worries might look foolish	0.61*	0.19*			0.07
	30	Worries about mistakes	0.56*				0.43*
	32	Worries what others think	0.43*	0.50*			−0.02
	38	Afraid to talk in front of class	0.44*	0.27*			0.02
	43	Afraid of looking foolish in front of people	0.55*	0.57*			−0.06
OCD
	10	Bothered by bad or silly thoughts or images	0.51*					0.35*
	16	Keeps checking if things are done right	0.45*		0.22*			−0.09
	23	Cannot get bad or silly thoughts out of head	0.54*					0.53*
	31	Has to think special thoughts to stop bad events	0.58*					0.12*
	42	Has to do things over and over again	0.50*				0.21*	0.02
	44	Has to do things just right to stop bad events	0.50*		0.17*			−0.08
MDD
	2	Feels sad or empty							0.68
	6	Nothing is much fun anymore							0.36
	11	Has trouble sleeping							0.43
	15	Has problems with appetite							0.33
	19	Has no energy for things							0.41
	21	Is tired a lot							0.26
	25	Cannot think clearly							0.56
	29	Feels worthless							0.69
	40	Feels like does not want to move							0.41
	47	Feels restless							0.75

Note. RCADS = Revised Child Anxiety and Depression Scale; ESEM = exploratory structural equation modeling; CFA = confirmatory factor analysis; MDD = major depressive disorder; SAD = separation anxiety disorder; GAD = generalized anxiety disorder; PD = panic disorder; SP = social phobia; OCD = obsessive–compulsive disorder; G = general “broad anxiety” factor. Items that appear in the shortened 20-item version are bolded.

The results revealed that all items had positive and significant loadings on the “Broad Anxiety” factor, ranging from 0.34 to 0.79. Most items also loaded on the hypothesized specific factors; some items, however, cross-loaded on other factors not consistent with the original theory.

SAD items

Although all SAD items had positive loadings on the “Broad Anxiety” factor, most SAD items had negative loadings on the corresponding specific factor. This may mean that the SAD items reflect generalized anxiety rather than a separation anxiety among Polish children. Moreover, Item 46 (“I would feel scared if I had to stay away from home overnight”) could be less applicable to younger children who typically do not stay away from home overnight. That item may, therefore, be referring more to generalized anxiety, leading to the cross-loading on the GAD factor.

GAD items

Item 22 (“I worry that bad things will happen to me”) and Item 27 (“I worry that something bad will happen to me”) had cross-loadings on the PD scale. Due to their synonymous meaning, one of these items could be discarded. Item 37 (“I think about death”) had a cross-loading on the SAD scale because the child’s potential suicidal ideation was potentially associated with the separation anxiety from their significant others following their death.

PD items

Item 34 (“I suddenly feel really scared for no reason”) and Item 41 (“I worry that I will suddenly get scared for no reason”) relate to sudden and unfounded fears and both had negative cross-loadings on the OCD scale. Item 3 (“When I have a problem, my stomach feels funny”) relates to somatic symptoms occurring in anxiety and this item had a cross-loading on GAD.

SP items

Almost half of the SP items (i.e., 20, 32, 38, 43) had cross-loadings on the SAD scale. As Ebesutani et al. (2012) noted, the social anxiety items could be split into two related factors: “perfectionism” which includes Items 4, 7, 8, 12, 30 (which are related to individual high demands, fears regarding poor school performance, and “social humiliation”) with Items 20, 32, 38, 43 (which are related to interpersonal difficulties and fears regarding poor public performances and judgements from other people). Both may relate to social rejection and separation anxiety.

OCD items

Based on the results we obtained, the OCD factor could be divided into two categories: obsessions (Items 10, 23, 31) that relate to recurrent, intrusive thoughts, urges, or images and compulsations (Items 16, 42, 44) that relate to repetitive behaviors that an individual feels driven to perform in response to an obsession or according to rules that must be applied rigidly. This division into two OCD categories is consistent with the current DSM diagnostic criteria for this disorder. Moreover, Item 42 (“I have to do things over and over again”) had a cross-loading on the SP scale because children who suffer from compulsations may avoid social contacts due to the fear of being judged or rejected.

MDD items

Only Item 21 (“I am tired a lot”) had a factor loading less than 0.30, which could be due to a subjective interpretation as children often understood tiredness in common sense (e.g., tiredness after coming back from school, rather than in terms of experiencing constant fatigue as characterized by MDD).

Although we found some cross-loadings at the specific factor level, the presented mixed bi-ESEM and CFA model with the general “Broad Anxiety” factor and the separate depression (MDD) factor fit the data very well in all three versions of the RCADS (i.e., original 47-item χ² = 1596.44, p < .001, comparative fit index [CFI] = 0.930, Tucker–Lewis index [TLI] = 0.912, root mean square error of approximation [RMSEA] = 0.041, 90% confidence interval [CI: 0.038, 0.045], shortened 30-item χ² = 626.80, p < .001, CFI = 0.953, TLI = 0.929, RMSEA = 0.048, 90% CI [0.043, 0.053] and shortened 20-item χ² = 241.07, p < .001, CFI = 0.961, TLI = 0.928, RMSEA = 0.051, 90% CI [0.043, 0.060]). These results therefore support that all three versions of the Polish RCADS measure the total anxiety and depression domains well.

Reliability

The composite reliability based on Bagozzi formula with scores above 0.6 indicating acceptable reliability were used (Bagozzi, 1994). Composite reliabilities based on the RCADS primarily unidimensional constructs (i.e., Total Anxiety and Total Depression) were calculated which provide more accurate reliability estimates from the bifactor perspective (e.g., Reise, Scheines, Widaman, & Haviland, 2013). Composite reliabilities greater than 0.80 were achieved in all three versions of the RCADS (i.e., original 47-item version: 0.96 for the “Broad Anxiety” and 0.90 for the MDD; shortened 30-item version: 0.94 for the “Broad Anxiety” and 0.84 for the MDD; shortened 20-item version: 0.91 for the “Broad Anxiety” and 0.82 for the MDD). The reliabilities surpassed the cutoffs for good reliability and suggest that over 80% of the variance of the Anxiety Total and Depression Total composite scores could be attributed to the variance on the “Broad Anxiety” and MDD factors. Consequently, this supports the scoring of items based on a primarily unidimensional scoring framework rather via multidimensional scoring procedures (e.g., Ebesutani et al., 2012; Reise et al., 2010).

Intercorrelations Between Different Versions of the RCADS

In addition to the structure assessment, we tested the validity of the proposed 20-item version of the RCADS. We correlated the 20-item version with the two existing versions: the 30-item version developed by Sandin et al. (2010) and the original 47-item version developed by Chorpita et al. (2000). Results of intercorrelations (along with means and standard deviations) are presented in Table 2.

Table 2.

Intercorrelations and Descriptives Among the Original 47-Item, 30-Item, and 20-Item Versions of the RCADS.

	RCADS-47						RCADS-30						RCADS-20
	SAD	GAD	PD	SP	OCD	MDD	SAD	GAD	PD	SP	OCD	MDD	SAD	GAD	PD	SP	OCD	MDD
RCADS-47
SAD	—
GAD	.57	—
PD	.55	.67	—
SP	.47	.64	.60	—
OCD	.55	.63	.63	.59	—
MDD	.43	.54	.67	.54	.56
RCADS-30
SAD	.96	.55	.51	.44	.50	.41	—
GAD	.57	.98	.62	.63	.64	.50	.54	—
PD	.57	.56	.88	.52	.55	.61	.49	.52	—
SP	.32	.59	.55	.89	.51	.52	.29	.55	.47	—
OCD	.54	.58	.60	.55	.97	.53	.48	.59	.56	.46	—
MDD	.46	.50	.60	.52	.58	.87	.44	.48	.57	.45	.57	—
RCADS-20
SAD	.87	.44	.48	.42	.48	.38	.78	.44	.53	.33	.47	.39	—
GAD	.51	.93	.64	.56	.55	.50	.48	.85	.55	.54	.51	.45	.41	—
PD	.51	.49	.82	.53	.51	.57	.42	.46	.95	.47	.52	.53	.50	.49	—
SP	.25	.55	.48	.82	.44	.43	.25	.51	.36	.82	.40	.37	.19	.48	.35	—
OCD	.45	.54	.53	.50	.83	.46	.41	.53	.45	.42	.73	.45	.42	.49	.43	.36	—
MDD	.46	.50	.60	.52	.58	.87	.44	.48	.57	.45	.57	—	.39	.45	.53	.37	0.58	—
M	0.41	0.98	0.56	1.11	0.81	0.70	0.45	1.04	0.38	0.96	0.82	0.63	0.33	0.78	0.39	1.18	0.62	0.58
SD	0.46	0.55	0.45	0.52	0.53	0.39	0.49	0.58	0.47	0.57	0.56	0.45	0.50	0.67	0.53	0.63	0.58	0.58

Note. RCADS = Revised Child Anxiety and Depression Scale; SAD = separation anxiety disorder; GAD = generalized anxiety disorder; PD = panic disorder; SP = social phobia; OCD = obsessive–compulsive disorder; MDD = Major Depression Disorder. All correlations are statistically significant (p < .01). Correlations with corresponding scales from other versions are in boldface are statistically significant. (p < .01). Correlations with corresponding scales from other versions are in boldface.

As expected, the highest correlations were found for the corresponding scales in all three versions of the RCADS. Overall, high correlations were found between the original 47-item and the two shortened versions: 30-item version (.87-.98) and the 20-item version (.82-.93), suggest that all versions assess the same construct. Moreover, the reduction of items from 10 to 5 in the MDD scale did not lead to scale violations. The original 10-item MDD scale achieved high correlations with the 5-item MDD scale (r = .87). Therefore, it can be concluded that alongside with the reduction of the scales’ item numbers, we did not compromise its theoretical validity by reducing the number of MDD scale items to three.

Study 2

Study 2 aimed to assess the longitudinal measurement stability of the Polish version of the RCADS by comparing different models (i.e., the original 47-item and the two shortened versions: 30-item and 20-item) across three waves at 6-month intervals. The examination of measurement stability over time is necessary for understanding whether observed changes in anxiety symptom scores across time reflect true changes in symptoms or are due to changes in measurement properties of the instrument (Mathyssek et al., 2013).

Participants and Procedure

The study was conducted in a community subsample of 75 children and adolescents (48% females) between 10 and 13 years of age (M_age = 11.24, SD_age = 0.68) who were selected from the participants who took part in Study 1. The current sample comprised the following age distribution: 10-year-olds (12%), 11-year-olds (53%), 12-year-olds (33%), and 13-year-olds (2%).

As in Study 1, ethical approval for this study was granted by the Institutional Review Board at the Psychology Institute, Cardinal Stefan Wyszyński University in Warsaw. The school principals received a letter stating that their school was invited to take part in a longitudinal research project that aimed to assess problem behavior in children and youth. The main difference between Study 1 and Study 2 was that the latter study had a longitudinal character. During the entire 18-month period, three assessments were carried out. The researchers returned to the schools on three separate occasions at 6-month intervals to ask participants to complete the RCADS again.

Measures

The RCADS (Chorpita et al., 2000) was applied, similarly as in Study 1.

Results

We tested longitudinal measurement stability using the latent growth curve (LGC) model to assess the equality of construct measurement across three testing occasions at 6-month intervals and whether the RCADS measured symptoms of anxiety consistently across time. While assessing the LGC model fit, we relied on chi-square (Kline, 2011), and on CFI and RMSEA indices due to a small sample size. Results of the LGC models for all three versions of the RCADS are presented in Table 3. With the exception for the OCD scale in the 20-item version, all models fit the data well. The highest initial level in means was obtained for SP and the lowest for SAD and PD that could be subject to further interpretation.

Table 3.

Model Fit Indices, Means, Variances, and Correlations Between Initial Level With Change in Three Versions of the RCADS.

	Model fit indices				Means		Variances		Correlation between initial level with change
	χ²	p	CFI	RMSEA	Initial level	Change	Initial level	Change	Correlation between initial level with change
RCADS-47
SAD	0.038	.846	1.000	0.000	1.304*	−0.044	0.088*	0.017	−0.657
GAD	3.737	.053	0.926	0.192	1.893*	0.072	0.161*	0.062	−0.653
PD	6.085	.048	0.950	0.166	1.531*	−0.030	0.073	0.020	−0.081
SP	3.616	.057	0.959	0.188	2.175*	0.012	0.190*	0.062	−0.519
OCD	0.081	.776	1.000	0.000	1.869*	−0.126*	0.028	0.008	0.836
MDD	0.044	.834	1.000	0.000	1.746*	−0.062	0.082	0.012	−0.294
RCADS-30
SAD	0.097	.756	1.000	0.000	1.373*	−0.053	0.136*	0.037	−0.573
GAD	4.496	.034	0.900	0.217	1.968*	−0.101	0.176*	0.067	−0.677
PD	5.039	.025	0.930	0.234	1.334*	−0.057	0.125*	0.038	−0.572
SP	4.310	.116	0.973	0.125	2.006*	0.037	0.224*	0.090*	−0.365
OCD	1.232	.267	0.991	0.056	1.887*	−0.143	0.060	0.019	−0.094
MDD	0.073	.788	1.000	0.000	1.723*	−0.082	0.151*	0.029	−0.501
RCADS-20
SAD	2.831	.092	0.910	0.157	3.537*	−0.114	0.471	0.071	−0.139
GAD	0.109	.741	1.000	0.000	5.522*	−0.127	1.215	0.357	−0.126
PD	0.078	.781	1.000	0.000	3.945*	−0.140	0.461	0.217	−0.070
SP	2.155	.142	0.981	0.125	6.914*	0.018	2.846*	0.781	−0.910
OCD^a
MDD	0.073	.788	1.00	0.000	8.613*	−0.412	3.780*	0.098	−0.501

Note. RCADS = Revised Child Anxiety and Depression Scale; SAD = separation anxiety disorder; GAD = generalized anxiety disorder; PD = panic disorder; SP = social phobia; OCD = obsessive–compulsive disorder; MDD = major depression disorder; CFI = comparative fit index; RMSEA = root mean square error of approximation.

Model was not identified.

p < .001.

We also assessed factor loadings of the RCADS at three assessments (T1, T2, and T3) presented in Table 4. Most indicators were significantly related to the latent construct of anxiety and depression and demonstrated similar factor patterns across time. However, some problems were found, for example, with Item 8 from SP scale that achieved negative loadings on its respective scale in T2 and T3. Moreover, the majority of the OCD items and half of the MDD items achieved factor loadings <0.50 at T1. In other words, the temporal variation in the measurement properties may question its stable relationship of these items to their underlying construct. Thus, the changes in the observed scores should be interpreted with caution and explained from both a developmental and methodological point of view.

Table 4.

Standardized Factor Loading Estimates Across Three Assessment Waves.

Scale	Item number and description		T1	T2	T3
SAD
	5	Fears being alone at home	0.75	0.00	0.75
	9	Fears being away from parents	0.67	0.42	0.75
	17	Scared to sleep alone	0.65	0.84	0.92
	18	Trouble going to school	0.45	0.97	1.13
	33	Afraid of being in crowded places	0.24	0.55	0.16
	45	Worries in bed at night	1.05	0.8	0.5
	46	Scared to sleep away from home	0.55	0.51	0.75
GAD
	1	Worries about things	0.35	0.58	0.09
	13	Worries something awful will happen to family	0.24	0.45	0.4
	22	Worries bad things will happen to self	0.76	0.74	0.4
	27	Worries something bad will happen to self	0.76	0.77	0.77
	35	Worries about what will happen	0.52	0.78	0.61
	37	Thinks about death	0.53	0.85	0.52
PD
	3	When has a problem, stomach feels funny	0.29	0.58	0.79
	14	Suddenly has trouble breathing for no reason	0.83	0.84	0.8
	24	When has a problem, heart beats really fast	0.61	0.53	0.34
	26	Suddenly trembles or shakes for no reason	0.7	0.95	0.77
	28	When has a problem, feels shaky	0.4	0.63	0.66
	34	Suddenly feels really scared for no reason	0.71	0.93	0.78
	36	Suddenly becomes dizzy of faint for no reason	0.63	0.77	0.8
	39	Heart suddenly beats too quickly for no reason	0.57	0.68	0.88
	41	Worries will suddenly get scared for no reason	0.71	0.88	0.88
SP
	4	Worries when does poorly at things	0.52	0.31	0.46
	7	Scared to take a test	0.65	0.67	0.79
	8	Feels worried when someone angry	0.57	−0.04	−0.17
	12	Worries will do badly at school work	0.63	0.77	0.76
	20	Worries might look foolish	0.93	0.90	1.03
	30	Worries about mistakes	0.34	0.89	0.73
	32	Worries what others think	0.58	0.23	0.58
	38	Afraid to talk in front of class	0.3	0.62	0.66
	43	Afraid of looking foolish in front of people	0.63	0.84	0.75
OCD
	10	Bothered by bad or silly thoughts or images	0.36	0.64	0.43
	16	Keeps checking if things are done right	0.33	0.03	0.38
	23	Cannot get bad or silly thoughts out of head	0.1	0.39	0.79
	31	Has to think special thoughts to stop bad events	0.5	0.72	0.57
	42	Has to do things over and over again	0.38	0.51	0.41
	44	Has to do things just right to stop bad events	0.34	0.18	0.15
MDD
	2	Feels sad or empty	0.77	0.83	0.87
	6	Nothing is much fun anymore	0.28	0.68	0.66
	11	Has trouble sleeping	0.29	0.46	0.66
	15	Has problems with appetite	−0.01	0.27	0.37
	19	Has no energy for things	0.62	0.49	0.52
	21	Is tired a lot	0.47	0.76	0.7
	25	Cannot think clearly	0.35	0.7	0.57
	29	Feels worthless	0.54	0.6	0.53
	40	Feels like does not want to move	0.39	0.72	0.74
	47	Feels restless	0.76	0.84	0.44

Note. SAD = separation anxiety disorder; GAD = generalized anxiety disorder; PD = panic disorder; SP = social phobia; OCD = obsessive–compulsive disorder; MDD = major depression disorder.

Normative Data

We provided normative data in Table 5 based on the current sample (N = 501), including means and standard deviations for the original 47-item version and the two shortened, that is, 30-item and 20-item versions of the RCADS.

Table 5.

Ranges, Means, and Standard Deviations for the 47-Item, 30-Item, and 20-Item RCADS Subscales by Gender.

Scale	Boys			Girls			Total
Scale	N	M (SD)	Range	N	M (SD)	Range	N	M (SD)	Range
RCADS-47
SAD	211	0.40 (0.49)	0.00-2.00	263	0.42 (0.44)	0.00-2.14	474	0.41 (0.46)	0.00-2.14
GAD	215	0.98 (0.58)	0.00-2.83	250	0.98 (0.53)	0.00-2.50	465	0.98 (0.55)	0.00-2.83
PD	205	0.53 (0.43)	0.00-2.44	257	0.59 (0.45)	0.00-2.44	462	0.56 (0.45)	0.00-2.44
SP	209	1.06 (0.49)	0.00-2.44	255	1.16 (0.53)	0.00-2.67	464	1.11 (0.52)	0.00-2.67
OCD	204	0.83 (0.54)	0.00-2.50	257	0.79 (0.53)	0.00-2.00	461	0.81 (0.53)	0.00-2.50
MDD	210	0.69 (0.38)	0.00-2.20	250	0.71 (0.41)	0.00-2.30	460	0.70 (0.39)	0.00-2.30
RCADS-30
SAD	213	0.43 (0.48)	0.00-2.00	263	0.43 (0.50)	0.00-2.40	476	0.45 (0.49)	0.00-2.40
GAD	215	1.05 (0.60)	0.00-2.80	250	1.04 (0.56)	0.00-2.40	465	1.04 (0.58)	0.00-2.80
PD	214	0.34 (0.43)	0.00-2.60	263	0.41 (0.50)	0.00-2.60	477	0.00-2.60	0.00-2.60
SP	210	0.90 (0.51)	0.00-2.20	259	1.00 (0.62)	0.00-2.80	469	0.96 (0.57)	0.00-2.80
OCD	205	0.82 (0.57)	0.00-3.00	261	0.82 (0.56)	0.00-2.40	466	0.82 (0.56)	0.00-3.00
MDD	215	0.61 (0.43)	0.00-2.00	253	0.64 (0.47)	0.00-2.00	468	0.63 (0.45)	0.00-2.00
RCADS-20
SAD	215	0.31 (0.54)	0.00-2.33	265	0.34 (0.46)	0.00-2.33	480	0.33 (0.50)	0.00-2.33
GAD	218	0.97 (0.69)	0.00-3.00	262	0.96 (0.64)	0.00-3.00	480	0.96 (0.66)	0.00-3.00
PD	215	0.36 (0.48)	0.00-3.00	264	0.42 (0.56)	0.00-2.67	479	0.39 (0.53)	0.00-3.00
SP	215	1.11 (0.59)	0.00-2.67	262	1.25 (0.66)	0.00-3.00	477	1.18 (0.63)	0.00-3.00
OCD	215	0.67 (0.59)	0.00-2.33	261	0.58 (0.56)	0.00-2.33	476	0.62 (0.58)	0.00-2.33
MDD	215	0.61 (0.43)	0.00-2.00	253	0.64 (0.47)	0.00-2.00	468	0.58 (0.58)	0.00-3.00

Discussion

The aim of this article was to investigate the psychometric properties of the Polish version of the RCADS based on the results obtained from two studies (one cross-sectional and one longitudinal sample) that assessed its structural validity, reliability, test stability, and longitudinal change.

In Study 1, we took a mixed analytic approach (i.e., exploratory bifactor and confirmatory modeling) to examine the degree to which each anxiety indicator represented the “Broad Anxiety” factor and their respective specific domain. The results supported the existence of a broad anxiety factor that comprises all anxiety scales. Although several cross-loadings appeared in our results, they do not undermine the usefulness of the RCADS, but rather, highlight the comorbidity of symptoms across the specific types of anxiety disorders. The present results are in line with previous studies by Ebesutani et al. (2012) who demonstrated that the broad anxiety factor accounts for the variability in scores across the anxiety items. Thus, researchers are advised to interpret the total scores from a measure rather than specific anxiety subscales because items appear to serve as better indicators of the broad anxiety.

We also introduced a shortened 20-item version of the Polish RCADS. The scales achieved good levels of reliabilities and high correlations with the scales of the original 47-item version of the RCADS suggest that both versions assess the construct in a similar manner. Therefore, the shortened 20-item version is a reliable and structurally valid measure.

The close association between “Broad Anxiety” and MDD (r = .67) supports the comorbidity between these two disorders. However, the items could be still differentiated by the symptomatology as suggested in the tripartite model of these two disorders as proposed by Clark and Watson (1991). This tripartite model suggests that depression and anxiety share a high negative affect vulnerability, and depression is specifically characterized by the absence of positive affect, whereas anxiety is specifically characterized by high physiological hyperarousal. From a methodological point of view, a possible explanation could be that the items might perform differently in different populations (e.g., younger vs. older groups of children). Moreover, depression has a low prevalence in the age category of our sample, therefore the MDD scale may, in fact, be reflecting anxiety symptoms rather than symptoms of depression, similarly as it was claimed in the study by Kösters et al. (2015). Also since our data are derived from a random sample, it is possible that among the nonclinical sample we selected, there were children who should have had clinical diagnoses of anxiety or depression. From a diagnostic point of view, a possible explanation could be the symptoms characteristics of both disorders. Moreover, as pointed out by some researchers (e.g., Trent et al., 2012), the RCADS anxiety and depression items may be associated with differential item functioning across various cultures. For instance, Items 42 and 44 in the OCD scale may represent systematic differences in cultural norms (e.g., due to superstitions) and behavioral reaction styles. Therefore, cultural differences should be taken into account when interpreting the scores obtained from the RCADS-specific scales.

In Study 2, we aimed to check whether the Polish version of the RCADS questionnaire measures symptoms of anxiety and depression consistently across time. Our findings partly confirmed our hypothesis regarding the measurement equality and are in line with a recent study by Kösters et al. (2015) showing the RCADS’ validity through its sensitivity in detecting a change in symptoms across an intervention program for childhood anxiety and depression. Thus, the RCADS is capable of identifying children who may need help such as those participating in preventive programs. Similar to Mathyssek et al.’s (2013) study, we found that the RCADS measured most anxiety and depression symptoms in a similar manner across time. However, we found some problems with the stability of SP, OCD, and MDD items. A possible explanation is that the SP symptoms are related to social interaction and performance anxiety that become more important in adolescence. The OCD symptoms have been also problematic in previous studies, however, this scale has been recommended to be included to increase the diversity of the total anxiety construct assessed with the RCADS. More MDD symptomsappear while entering adolescence because there is more pressure on academic achievement and increased fear from being judged by others, for example, “I feel afraid that I will make a fool of myself in front of people.” Taking into consideration the problems we faced, longitudinal changes of the particular indicators should be interpreted with special caution. Moreover, rather than interpreting the specific subscale scores, results of the present study and the bifactor model suggest that interpretations regarding anxiety should be based primarily on the general “Broad Anxiety” score (as this score appears to be the most reliable index of one’s overall anxiety-related psychological health).

Conclusion and Limitations

To summarize, our data provided support for the psychometric soundness of the Polish version of the RCADS as a valuable instrument that screens for five specific anxiety disorders and yields overall anxiety and depression scores. This study also supported the development of a briefer, 20-item version that contains highly informative items and allows for administration in a relatively short period of time. However, special caution should be placed while interpreting the results because fewer items might not fully capture anxiety and depression symptoms. Second, relying on self-report questionnaires may lead to mono-method bias. Third, we introduced a translation of the RCADS to a population that is ethnically homogeneous. It is therefore relatively unknown whether these results generalize to other samples. Further research including clinical samples, data obtained from other-informants and cross-cultural testing are needed to support the diagnostic usefulness of the instrument.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work of I. Skoczeń was supported by doctoral scholarship ETIUDA 2016/20/T/HS6/00030 from the Polish National Science Centre, and the work of M. Rogoza was supported by Diamont Grant 0082/DIA/2012/41 from the Polish Ministry of Science and Higher Education.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Angold

Costello

E. J.

Erkanli

(1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40, 57-87. doi:10.1111/1469-7610.00424

Bagozzi

R. P.

(1994). Structural equation models in marketing research: Basic principles. In Bagozzi

R. P.

(Ed.), Principles of marketing research (pp. 317-385). Oxford, England: Blackwell.

Braddick

Carral

Jenkins

Jané-Llopis

(2009). Child and adolescent mental health in Europe: Infrastructures, policy and programmes. Retrieved from http://ec.europa.eu/health/ph_determinants/life_style/mental/docs/camhee_infrastructures.pdf

Brady

E. U.

Kendall

P. C.

(1992). Comorbidity of anxiety and depression in children and adolescents. Psychological Bulletin, 111, 244-255. doi:10.1037/0033-2909.111.2.244

Brislin

R. W.

(1986). The wording and translation of research instruments. In Lonneri

W. J.

Berry

J. W.

(Eds.), Cross-cultural research methodology series: Field methods in cross-cultural research (pp. 137-164). Beverly Hills, CA: Sage.

Chorpita

B. F.

Ebesutani

Spence

S. H.

(2015). Revised Children’s Anxiety and Depression Scale: User’s guide. Retrieved from http://www.childfirst.ucla.edu/RCADSUsersGuide20150701.pdf

Chorpita

B. F.

Moffitt

C. E.

Gray

(2005). Psychometric properties of the Revised Child Anxiety and Depression Scale in a clinical sample. Behaviour Research and Therapy, 43, 309-322. doi:10.1016/j.brat.2004.02.004

Chorpita

B. F.

Yim

Moffitt

Umemoto

L. A.

Francis

S. E.

(2000). Assessment of symptoms of DSM-IV anxiety and depression in children: A revised child anxiety and depression scale. Behavior Research and Therapy, 38, 835-855. doi:10.1016/S0005-7967(99)00130-8

10.

Clark

L. A.

Watson

(1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100, 316-336. doi:10.1037//0021-843x.100.3.316

11.

Cummings

C. M.

Caporino

N. E.

Kendall

P. C.

(2014). Comorbidity of anxiety and depression in children and adolescents: 20 Years after. Psychological Bulletin, 140, 816-845. doi:10.1037/a0034733

12.

Ebesutani

Reise

S. P.

Chorpita

B. F.

Ale

Regan

Young

. . . Weisz

J. R.

(2012). The Revised Child Anxiety and Depression Scale-short version: Scale reduction via exploratory bifactor modeling of the broad anxiety factor. Psychological Assessment, 24, 833-845. doi:10.1037/a0027283

13.

Ederer

E. M.

(2004). Mental health problems in young children: Self-reports and significant others as informants. Psychology Science, 46, 123-140. Retrieved from http://www.pabst-publishers.de/psychology-science/supp01-2004/ps_supp1_2004_123-140.pdf

14.

Emons

Sijtsma

Meijer

(2007). On the consistency of individual classification using short scales. Psychological Methods, 12, 105-120. doi:10.1037/1082-989X.12.1.105

15.

Hambleton

R. K.

(2005). Issues, designs and technical guidelines for adapting tests into multiple languages and cultures. In Hambleton

R. K.

Merenda

P. F.

Spielberger

C. D.

(Eds.), Adapting psychological and educational tests for cross-cultural assessment (pp. 3-38). Mahwah, NJ: Lawrence Erlbaum.

16.

Higa-McMillan

H. K.

Francis

S. E.

Rith-Najarian

Chorpita

B. F.

(2016). Evidence base update: 50 Years of research on treatment for child and adolescent anxiety. Journal of Clinical Child & Adolescent Psychology, 45, 91-113. doi:10.1080/15374416.2015.1046177

17.

Kline

R. B.

(2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.

18.

Kösters

M. P.

Chinapaw

M. J. M.

Zwaanswijk

Van der Wal

M. F.

Koot

H. M.

(2015). Structure, reliability, and validity of the Revised Child Anxiety and Depression Scale (RCADS) in a multi-ethnic urban sample of Dutch children. BMC Psychiatry, 15, 132. doi:10.1186/s12888-015-0509-7

19.

Mathyssek

C. M.

Olino

T. M.

Hartman

C. A.

Ormel

Verhulst

F. C.

Van Oort

F. V. A.

(2013). Does the Revised Child Anxiety and Depression Scale (RCADS) measure anxiety symptoms consistently across adolescence? The TRAILS study. International Journal of Methods in Psychiatric Research, 22, 27-35. doi:10.1002/mpr.1380

20.

Muris

Meesters

(2002). Symptoms of anxiety disorders and teacher-reported school functioning of normal children. Psychological Reports, 91, 588-590. doi:10.2466/pr0.2002.91.2.588

21.

Muris

Meesters

Schouten

(2002). A brief questionnaire of DSM-IV-defined anxiety and depression symptoms among children. Clinical Psychology & Psychotherapy, 9, 430-442. doi:10.1002/cpp.347

22.

Muthén

L. K.

Muthén

B. O.

(2012). Mplus user’s manual (7th ed.). Los Angeles, CA: Muthén & Muthén.

23.

Reise

Moore

Haviland

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223891.2010.496477

24.

Reise

S. P.

Scheines

Widaman

K. F.

Haviland

M. G.

(2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73, 5-26. doi:10.1177/0013164412449831

25.

Rhemtulla

Brosseau-Liard

P. E.

Savalei

(2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373. doi:10.1037/a0029315

26.

Sandin

Chorot

Valiente

R. A.

Chorpita

B. F.

(2010). Development of a 30-item version of the Revised Child Anxiety and Depression Scale. Revista de Psicopatología y Psicología Clínica, 15, 165-178. Retrieved from http://www.childfirst.ucla.edu/Sandin%20et%20al%20(2010).%20RCADS-30.pdf

27.

Sosnowski

Iwaniszczuk

Spielberger

C. D.

(1989). Polska adaptacja Inwentarza Stanu i Cechy Leku dla dzieci (STAIC) [Polish Adaptation of the State-Trait Anxiety Inventory for Children (STAIC)]. Studia Psychologiczne, 27, 67-81.

28.

Spence

S. H.

(1998). A measure of anxiety symptoms among children. Behavior Research and Therapy, 36, 545-566. doi:10.1016/S0005-7967(98)00034-5

29.

Spielberger

C. D.

(1973). Manual for the State-Trait Anxiety Inventory for Children. Palo Alto: Consulting Psychologists Press.

30.

Stevanovic

Bagheri

Atilola

Vostanis

Stupar

Moreira

. . . Ribas

(2016). Cross-cultural measurement invariance of the Revised Child Anxiety and Depression Scale across 11 world-wide societies. Epidemiology and Psychiatric Sciences. Advance online publication. doi:10.1017/S204579601600038X

31.

Stone

Otten

Engels

R. C. M. E.

Vermulst

A. A.

Janssens

J. M. A. M

. (2010). Psychometric properties of the parent and teacher versions of the Strengths and Difficulties Questionnaire for 4- to 12-years old: A review. Clinical Child and Family Psychology Review, 13, 254-274. doi:10.1007/s10567-010-0071-2

32.

Trent

L. R.

Buchanan

Ebesutani

Ale

C. M.

Heiden

Hight

T. L.

. . . Young

(2012). A measurement invariance examination of the Revised Child Anxiety and Depression Scale in a Southern sample: Differential item functioning between African American and Caucasian Youth. Assessment, 20, 175-187. doi:10.1177/1073191112450907