Examining the Screen for Child Anxiety-Related Emotional Disorder-71 as an assessment tool for anxiety in children with high-functioning autism spectrum disorders

Abstract

The psychometric properties of a questionnaire developed to assess symptoms of anxiety disorders (SCARED-71) were compared between two groups of children: children with high-functioning autism spectrum disorder and comorbid anxiety disorders (ASD-group; n = 115), and children with anxiety disorders (AD-group; n = 122). Anxiety disorders were established with a semi-structured interview (ADIS-C/P), using child- as well as parent-report. Internal consistency, construct validity, sensitivity, specificity, and discriminant validity of the SCARED-71 was investigated. Results revealed that the psychometric properties of the SCARED-71 for the ASD-group were quite comparable to the AD-group, however, the discriminant validity of the SCARED-71 child-report was less in the ASD-group. Raising the parental cutoffs of the SCARED-71 resulted in higher specificity rates, which suggests that research should focus more on establishing alternative cutoffs for the ASD-population.

Keywords

ASD anxiety self-report child-parent agreement

Introduction

Autism spectrum disorders (ASD) are characterized by varying degrees of impairment in communication skills, social interactions, and restricted, repetitive, and stereotyped patterns of behavior (American Psychiatric Association (APA), 2000). An additional aggravating factor consists of the fact that youth with ASD often endorse (multiple) comorbid psychiatric disorders, of which anxiety disorders (ADs) appear to be among the most common (e.g. De Bruin et al., 2007; Leyfer et al., 2006; Simonoff et al., 2008). ADs are found to affect 11%–84% of the children with ASD (White et al., 2009b), with a meta-analytic estimate of nearly 40% of youth with ASD reaching clinical anxiety levels (Van Steensel et al., 2011). Higher levels of anxiety are also found in youth with ASD when compared to typically developing children (e.g. Gillot et al., 2001; Kim et al., 2000), and compared to various clinical groups such as youth with specific language impairment (Gillot et al., 2001), attention deficit hyperactivity disorder (ADHD) (Gadow et al., 2009; Van Steensel et al., 2012), conduct disorder (Green et al., 2000), and intellectual disabilities (Brereton et al., 2006).

Questionnaires are a widely used and studied method to assess anxiety symptoms, or screen for ADs, in typically developing populations. In fact, the majority of studies examining anxiety in youth with ASD have relied on the use of such instruments to examine anxiety in ASD (e.g. Van Steensel et al., 2011). The applicability and psychometric properties of such instruments, however, have rarely been studied in the ASD population. To the authors’ knowledge, there are only three studies that have explored the use of anxiety questionnaires developed for the general population in youth with ASD (Blakeley-Smith et al., 2012; Mazefsky et al., 2011; White et al., 2011), which we will discuss next.

The study of Mazefsky et al. (2011) compared the Revised Children’s Manifest Anxiety Scale (RCMAS; Reynolds and Richmond, 1985), completed by 38 adolescents (10–17 years) with high-functioning ASD, to a parental diagnostic interview. It was found that individuals with ASD were able to report about their anxiety to some degree, however, not sufficiently for clinical diagnostic purposes (Mazefsky et al., 2011). That is, their self-reports may not reach thresholds developed for screening instruments, while their parents do rate the problems to meet clinical diagnosis. The internal consistency of the RCMAS in this study was .88. The sensitivity and specificity of the RCMAS was .33 and .88, respectively. An alternative cutoff was developed by the authors by lowering the threshold for child-report, and sensitivity was increased to .52 (Mazefsky et al., 2011).

The study of White et al. (2011) investigated the psychometric properties of the Multidimensional Anxiety Scale for Children–Child and Parent Version (MASC-C/P; March et al., 1997) in a sample of 30 adolescents (12–17 years) with high-functioning ASD. Cronbach’s alpha was .90 and .92 for child- and parent-reports, respectively. The authors found little support for inter-rater agreement; child- and parent-report were not significantly correlated. In addition, the authors raised concerns about the validity of self-report measures in youth with ASD; only 23% of the adolescents with ASD reported clinically elevated anxiety scores while all were diagnosed with ADs (White et al., 2011). According to White et al. (2011) these findings suggest that adolescents with ASD might under-report their anxiety problems due to a lack of insight, a different perspective about their own anxiety symptoms, and/or an unwillingness to truthfully describe the extent of their problem.

The study of Blakeley-Smith et al. (2012) investigated the use of self-report in 63 verbally fluent children (8–14 years) with ASD by examining the child–parent agreement. The Screen for Child Anxiety-Related Emotional Disorder (SCARED) (Birhamer et al., 1999), containing 41 items and five subscales (panic disorder, generalized anxiety, separation anxiety, social anxiety, and school avoidance), was used to assess anxiety. Child–parent agreement was moderate for the SCARED total score (r = .52), and fair to substantial child–parent agreement (r = .34–.71) was found across the SCARED subscales.

In conclusion, all three studies report high internal consistencies of the questionnaires developed for typically developing children in the ASD samples; however, the results with respect to validity are rather mixed. The study of White et al. (2011) and Mazefsky et al. (2011) questioned the validity of self-report in adolescents with ASD, while the study of Blakeley-Smith et al. (2012) concluded that the use of self-report may be just as problematic in ASD samples as in non-ASD samples. A direct comparison to non-ASD groups, however, was not made. The aim of this study was to evaluate the psychometric properties of SCARED-71 (Bodden et al., 2009) in children with high-functioning ASD and comorbid ADs, and compared these properties to those of children with ADs (without ASD). If it can be established that the SCARED-71 is a valid and reliable instrument to detect clinical anxiety in children with ASD, this instrument may be a valuable tool for clinicians to assess (or screen for) ADs.

Method

Participants

Participants were 237 families of (a) children with high-functioning ASD and comorbid ADs (n = 115; further referred to as ASD-group) and (b) children with ADs (n = 122; further referred to as AD-group). All 237 children participated, and 229 mothers and 180 fathers participated. All families were referred to the same secondary-care community mental health-care centers. Diagnosis of ASD, and the presence of an anxiety disorder, was established by a multidisciplinary team within the mental health-care centers. As part of the research measures, the Autism Diagnostic Interview–Revised (ADI-R) (Lord et al., 1994) was administered in 90 cases; 98% was found to meet the ADI-R cutoff for the social domain, while 90% and 70% were found to meet the ADI-R cutoffs for the communicative and the repetitive domains, respectively. The presence of at least one AD was confirmed by the Anxiety Disorder Interview Schedule–Child and Parent Version (ADIS-C/P; Silverman and Albano, 1996). Compared to the AD-group, the ASD-group was represented with more boys (90 vs 62), and the children in that group were significantly younger (M = 11.37 vs M = 12.79; age range = 7–18 years). Educational levels were not found to differ between groups. For more details about the participants, see the study by Van Steensel et al. (2012).

Assessments

ADIS-C/P

The ADIS-C/P (Silverman and Albano, 1996) was developed to assess childhood psychiatric disorders based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) in children aged 7–17 years. It possesses good psychometric properties (Silverman et al., 2001). The interview initially screens for DSM-IV symptoms of disorders, followed by an assessment on whether the symptoms lead to significant impairment. This impairment is rated on a scale from 0 to 8 (a score ≥ 4 warrants a final diagnosis). In this study, a total anxiety severity score was calculated by summing up the severity ratings of all anxiety disorders. This approach, which combines the number and severity of anxiety disorders, has been used by other studies to evaluate the effectiveness of treatment (Hudson et al., 2009; Simon et al., 2011). The ADIS-C/P has also been used to assess anxiety disorders, and to evaluate treatment effectiveness for anxiety, in youth with ASD (e.g. Reaven et al., 2012; White et al., 2009a; Wood et al., 2009). The presence of ADs in the ASD- and AD-groups according to the ADIS-C/P, as well as information about the child–parent agreement (calculated with kappa-coefficients), is displayed in Table 1. The ADIS-P was administered to the parents who participated in the research. The number of families in which both parents participated was 172. In about 70%–80% of these cases, the ADIS-P was administered to both parents. In about 90% of the other cases, the mothers were the informants of the ADIS-P.

Table 1.

Presence of anxiety disorders (ADIS-C/P) according to child- and parent-report.

	ASD-group					AD-group
	Parent (n = 115)		Child (n = 113)			Parent (n = 122)		Child (n = 121)

	n	%	n	%	κ	n	%	n	%	κ
Separation anxiety disorder	30	26.1	21	18.6	.52	27	22.1	30	24.8	.56
Social anxiety disorder	63	54.8	45	39.8	.59	59	48.4	55	45.5	.70
Specific phobia	96	83.5	86	76.1	.67	77	63.1	73	60.3	.65
Generalized anxiety disorder	58	50.4	43	51.3	.53	68	55.7	52	43.0	.55
Obsessive–compulsive disorder	14	12.2	10	8.8	.63	14	11.5	18	14.9	.64
Panic disorder	6	5.2	3	2.7	.65	10	8.2	13	10.7	.57
Agoraphobia	7	6.1	7	6.2	.85	20	16.4	23	19.0	.75
Post-traumatic stress disorder	7	6.1	5	4.4	.82	10	8.2	10	8.3	.67

ADIS-C/P: Anxiety Disorder Interview Schedule–Child and Parent; ASD-group: children with ASD and comorbid anxiety disorders (two ADIS child-report missing); AD-group: children with anxiety disorders.

κ = kappa coefficient of child–parent agreement.

SCARED-71

The Dutch version of the SCARED (Bodden et al., 2009) was completed by both parents and children and was used to measure anxiety symptoms. The questionnaire consists of 71 items, each scored on a 3-point scale for its occurrence (0 = almost never, 1 = sometimes, and 2 = often). A total anxiety score can be obtained as well as subscale scores for separation anxiety disorder (12 items; e.g. “I am afraid to be alone at home”), social anxiety disorder (9 items; e.g. “I am shy with people I don’t know well”), specific phobia (15 items; e.g. “I am afraid of heights”), generalized anxiety disorder (9 items; e.g. “I worry about the future”), obsessive–compulsive disorder (9 items; e.g. “I doubt whether I really did something”), panic disorder (13 items; e.g. “when frightened, I feel like passing out”), and post-traumatic stress disorder (4 items; e.g. “I try not to think about a very aversive event I once experienced”).

Psychometric properties of the SCARED-71 were examined by Bodden et al. (2009) in a sample of 138 clinically anxious children and 38 typically developing children, aged 8–18 years. Internal consistencies of the SCARED-71 were excellent; .95, .96, and .95, for child-, mother-, and father-reports, respectively (see also Table 2 for the internal consistencies of the SCARED-71 scales as reported by Bodden et al., 2009). The SCARED-71 was able to discriminate between clinically anxious and nonanxious children, and cutoffs for the total score and most subscales were established (see also Table 4 for the cutoffs and corresponding sensitivity and specificity as reported by Bodden et al., 2009).

Table 2.

Cronbach’s alphas for SCARED-71 child-, mother-, and father-reports.

	Sample of Bodden et al. (2009)			ASD-group			AD-group
	Child (n = 176)	Mother (n = 174)	Father (n = 155)	Child (n = 109)	Mother (n = 111)	Father (n = 88)	Child (n = 112)	Mother (n = 111)	Father (n = 87)
SCARED-71 total score	.95	.96	.95	.92	.94	.95	.94	.93	.94
Separation anxiety disorder	.81	.86	.84	.73	.79	.82	.77	.74	.73
Social anxiety disorder	.85	.91	.89	.80	.83	.85	.87	.90	.87
Specific phobia^a	.64–.87	.66–.92	.67–.93	.78	.85	.85	.80	.78	.78
Generalized anxiety disorder	.87	.88	.86	.86	.88	.88	.83	.85	.87
Obsessive–compulsive disorder	.74	.77	.70	.62	.70	.77	.78	.73	.78
Panic disorder	.88	.88	.86	.77	.83	.87	.86	.86	.84
Post-traumatic stress disorder	.82	.84	.80	.75	.77	.79	.78	.82	.77

SCARED-71: Screen for Child Anxiety-Related Emotional Disorder-71; ASD-group: children with ASD and comorbid anxiety disorders; AD-group: children with anxiety disorders.

Bodden et al. (2009) reported internal consistencies for animal phobia, blood-injection-injury phobia, and situational–environmental phobia separately but not for the total specific phobia scale.

Procedure

Families were asked to participate in a longitudinal study, which examines the treatment of anxiety disorders in children with and without ASD. The results described here are based on pretreatment measurements. Inclusion criteria for the study were as follows: (a) having at least one anxiety disorder and (b) at least one parent willing to participate. Exclusion criteria were the following: (a) IQ level below 70 (IQ must have been estimated to be above 70 based on school performance; in case of doubt, an IQ test was administered), (2) nontreated psychotic disorder, (3) acute suicidal risk, and (4) current sexual or physical abuse. The study was approved by a Medical Ethical Committee and informed consent was gathered. Assessments took place at the mental health-care center or at the families’ home. Children were instructed to fill in the questionnaires themselves; however, they were told that they could ask the administrator, or their parents (when completing the questionnaires at home), if they did not understand the questions. Overall, children seemed to be able to fill in the questionnaire themselves and only occasionally clarification was asked. Considering the administration of the ADIS-C, all children had sufficient language abilities to answer the questions of the interview and no modifications were made on forehand. However, if needed, clarifications/adaptations to the interview, such as the use of more concrete and direct questions, more simple language (e.g. questions without a denial), and the use of more examples, were made.

Seven mental health-care centers throughout the Netherlands participated, and each center had an administrator who worked and/or conducted research within that center. The administrators were independent of the staff who initially established the diagnoses. Interrater reliability for the diagnoses in the current sample was not specifically assessed. However, administrators were properly trained for the ADIS-C/P (see Bodden et al., 2009, for a description of the training procedure), and interrater reliability within our research group was found to be high (e.g. Bodden et al., 2009; Simon et al., 2011). With respect to the ADI-R, the administrators were trained by the first author of the study (who is certified for the administration of the interview), and they had to achieve a reliability of at least 80%.

Data and analyses

All ADIS-P interviews were completed; however, three ADIS-C interviews (1.3%) were not completed. SCARED-71 reports were not complete for 16 children (6.8%), 7 mothers (3.1%), and 5 fathers (2.8%). Families with or without missing data did not differ with respect to the child’s gender, age, or group (ASD or AD-group).

Cronbach’s alpha was calculated to examine the internal consistency. Construct agreement between the two anxiety measures (ADIS-C/P and SCARED-71) and among respondents (children, mothers, and fathers) was examined with Pearson’s correlations. The equality between two correlations was examined with the Z-statistic. The sensitivity and specificity of the SCARED-71 were evaluated by comparing the SCARED-71 cutoffs (established by Bodden et al., 2009) to the ADIS-C/P diagnoses. Since Bodden et al. (2009) only reported cutoffs for a combined parent version of the SCARED-71, the reports of mothers and fathers were averaged for further analyses. Sensitivity and specificity were not evaluated for obsessive–compulsive disorder and post-traumatic stress disorder because Bodden et al. (2009) did not establish cutoffs for these subscales. In addition, sensitivity and specificity for panic disorder were not evaluated due to the small number of children with ASD meeting criteria for this disorder (see Table 1). Receiver operating characteristic (ROC) analyses were used to examine alternative cutoffs.

The discriminant validity of the SCARED-71 was examined with analyses of variance (ANOVAs) by comparing the SCARED-71 subscales scores of children with a particular anxiety disorder (as established by the ADIS-C/P) to children without that anxiety disorder. Nonparametric tests were also applied when sample sizes of children with a particular anxiety disorder were small (e.g. panic disorder, post-traumatic stress disorder). The nonparametric analyses yielded similar results, and therefore, the results of the parametric tests are reported. Post hoc, we examined which items of the SCARED-71 subscales were able to discriminate between children with ASD with and without a particular AD. This was done only for those subscales that were found to have sufficient discriminant validity in the previous analyses (ps < .05). The analyses conducted were exploratory, to examine whether items of a particular subscale would be more or less useful to reliably assess anxiety in ASD. To examine this issue, several multivariate analyses of variance (MANOVAs) were conducted. All items of a particular SCARED-71 subscale were included as the dependent variables, while the presence/absence of that anxiety disorder was entered as the fixed factor. If a MANOVA yielded significance, each item was evaluated for its discriminate ability.

Results

Internal consistency

Internal consistencies of the SCARED-71 are displayed in Table 2. Cronbach’s alpha for the SCARED-71 total score was high (.92–.95) for both groups (ASD- and AD-group) and for all respondents (children, mothers, and fathers). Cronbach’s alpha was acceptable (≥ .70) for all subscales with the exception of one. In the ASD-group, Cronbach’s alpha for child-reported obsessive–compulsive symptoms was found to be .62. Within the ASD-group, internal consistencies of the SCARED-71 child-report were somewhat lower than those of the SCARED-71 parent-report.

Construct validity

Correlations between the ADIS-C/P and the SCARED-71 within respondents, and correlations between respondents for the same anxiety measure were all positive and reached significance (Table 3). The correlation between the ADIS-C and the SCARED-71 child-report in the AD-group was somewhat higher compared to the correlation between those measures in the ASD-group (r = .57 vs. r = .38; Z = 1.85; p = .06). Correlations between the ADIS-P and the SCARED-71 parent-report were medium and did not differ significantly between the ASD- and AD-groups (ps > .10).

Table 3.

Correlations between anxiety measurements (ADIS-C/P and SCARED-71) and between respondents (children, mothers and fathers), for the ASD-group (presented in bold) and the AD-group.

ASD-group AD-group	ADIS child	ADIS parent	SCARED child	SCARED mother	SCARED father
ADIS child	—	.58***	.38***	.31**	.20
ADIS parent	.63***	—	.24*	.47***	.42***
SCARED child	.57***	.38***	—	.39***	.41***
SCARED mother	.32**	.36***	.56***	—	.75***
SCARED father	.20	.38***	.52***	.74***	—

ADIS: Anxiety Disorder Interview Schedule; SCARED: Screen for Child Anxiety-Related Emotional Disorder; ASD-group: children with ASD and comorbid anxiety disorders; AD-group: children with anxiety disorders.

p < .001; **p < .01; ***p < .05.

Correlations between children and parents for the ADIS report were found to be large in both groups and did not significantly differ between groups (p > .10). Correlations between children and parents for the SCARED-71 were medium in the ASD-group (r = .39/.41) and large in the AD-group (r = .56/.52); however, the child-parent correlations did not differ significantly (ps > .10).

Sensitivity and specificity

Sensitivity for the SCARED-71 child-report was found to be .81 for the ASD-group and .79 for the AD-group (Table 4). This means that 19% of the ASD group and 21% of the AD-group were not identified by the SCARED-71 as clinically anxious. For the parent-report, 5% of the ASD group and 9% of the AD-group were not identified as clinically anxious (sensitivity of .95 and .91, respectively).

Table 4.

Examining the sensitivity and specificity of the SCARED-71 for child- (ASD (n = 109)/AD (n = 112)) and parent-report (ASD (n = 114)/AD (n = 113)) using the SCARED-71 cutoffs developed by Bodden et al. (2009).

	Sample of Bodden et al. (2009)			ASD-group		AD-group
	Cutoff	Sensitivity	Specificity	Sensitivity	Specificity	Sensitivity	Specificity
Child
Total score	30	.78	.76	.81	NA	.79	NA
Separation anxiety	8	.80	.75	.67	.63	.70	.53
Social anxiety	8	.71	.77	.77	.61	.85	.75
Specific phobia	8	.72	.64	.74	.56	.57	.64
Generalized anxiety	8	.62	.69	.72	.68	.85	.75
Parent
Total score	21	.92	.92	.95	NA	.91	NA
Separation anxiety	8	.79	.72	.90	.52	.86	.48
Social anxiety	7	.83	.78	.87	.39	.84	.67
Specific phobia	7	.74	.78	.78	.67	.65	.73
Generalized anxiety	8	.75	.80	.84	.54	.71	.69

ASD-group: children with ASD and comorbid anxiety disorders; AD-group: children with anxiety disorders (without ASD); NA: not applicable (i.e. all children had at least one anxiety disorder according to ADIS-C/P); ADIS-C/P: Anxiety Disorder Interview Schedule–Child and Parent.

Depending on the subscale, 67%–90% of the children with ASD were correctly classified by the SCARED-71 cutoffs as having that same ADIS-C anxiety disorder (for the AD-group, this was 57%–86%; Table 4). This means that 10%–33% of the children with ASD were “false negatives” (for the AD-group: 14%–43%), that is, the children were not detected by the SCARED-71 as having a particular anxiety disorder, although ADIS report yielded a diagnosis. The sensitivity rates for parent-report were somewhat higher in the ASD-group compared to the AD-group for all SCARED-71 scales (Table 4).

Depending on the subscale, 39%–78% of the children with ASD were correctly classified by the SCARED-71 as not having a particular anxiety disorder (for the AD-group, this was 48%–75%; Table 4). These results indicate that 22%–61% were “false positives” (for the AD-group: 25%–52%), that is, being falsely identified by the SCARED-71 as having a particular anxiety disorder. Compared to the AD-group, the specificity rates for separation anxiety disorder were somewhat higher in the ASD-group. Specificity rates were lower in the ASD-group with respect to social anxiety disorder, specific phobia, and generalized anxiety disorder.

Alternative cutoffs for the ASD-group

ROC-analyses revealed that alternative cutoffs did not result in a better fit with respect to the SCARED-71 child-report. An alternative cutoff for parent-report about symptoms of specific phobia did not result in a more optimal fit, however, as raising the other parental cutoffs did: (1) raising the cutoff for separation anxiety from 8 to 10 resulted in a sensitivity of .80 and a specificity of .67, (2) raising the cutoff for social anxiety from 7 to 9 resulted in a sensitivity of .77 and a specificity of .71, and (3) raising the cutoff for generalized anxiety from 8 to 9 resulted in a sensitivity of .71 and a specificity of .68.

Discriminant validity

Based on the child-report, the SCARED-71 was able to discriminate participants with ASD with a particular AD from participants with ASD without that disorder for four out of the seven scales (Table 5). For three scales (obsessive–compulsive disorder, panic disorder, and post-traumatic stress disorder), SCARED-71 subscale scores did not differ significantly between those with and without that particular anxiety disorder. Of note, all SCARED-71 subscales scores differed significantly between children with and without a particular anxiety disorder in the AD-group (ps < .05).

Table 5.

Discriminant validity of the SCARED-71 subscales for the ASD-group.

	Child (n = 109)							Parent (n = 114)
	+		−					+		−
	M	SD	M	SD	F	ES	p	M	SD	M	SD	F	ES	p
SAD	10.44	5.10	6.69	4.01	11.98	.89	.001	13.20	5.10	8.22	4.42	29.66	1.16	.001
SOC	10.33	4.10	6.28	3.49	29.96	1.08	<.001	11.27	3.61	7.60	3.50	29.86	1.03	<.001
SPH	11.39	5.76	6.60	5.16	13.90	.85	<.001	11.56	5.91	6.36	5.98	11.66	.88	.001
GAD	10.08	4.06	5.44	4.22	30.74	1.11	<.001	10.91	3.42	7.05	4.28	27.98	1.00	<.001
OCD	6.78	1.20	6.40	3.42	.11	.12	.742	7.21	3.23	5.83	3.27	2.19	.42	.142
PAN	8.00	3.00	5.04	3.99	1.62	.75	.206	9.17	4.20	5.60	4.24	4.02	.84	.047
PTSD	3.75	2.06	2.53	2.28	1.10	.53	.297	5.13	1.43	2.61	2.00	29.66	1.28	.001

SCARED-71: Screen for Child Anxiety-Related Emotional Disorder-71; ASD-group: children with ASD and comorbid anxiety disorders; ES: effect size (Cohen’s d); SAD: separation anxiety disorder; SOC: social anxiety disorder; SPH: specific phobia; GAD: generalized anxiety disorder; OCD: obsessive–compulsive disorder; PAN: panic disorder; PTSD: post-traumatic stress disorder.

: children with the anxiety disorder listed in the first column; −: children without the anxiety disorder listed in the first column.

Based on the parent-report, the only SCARED-71 subscale that was not able to discriminate effectively between children with ASD with and without a particular AD was the subscale obsessive–compulsive disorder (Table 5). In the AD-group, a nonsignificant result was found for post-traumatic stress disorder (p > .10).

Post hoc, the discriminate ability of the specific items of the SCARED-71 subscales was examined for the ASD-group. Considering child-report, it was found that the MANOVA did not reach significance for specific phobia. For separation anxiety, 7 of the 12 items did not discriminate effectively (ps > .05) between children with ASD with and without this anxiety disorder. However, all items measuring social anxiety and generalized anxiety had sufficient discriminate ability (ps < .05). Considering parent-report, MANOVA did not yield a significant result for panic disorder, and 8 of the 15 items measuring specific phobia did not have sufficient discriminate ability. Only 3 of the 12 items measuring separation anxiety and 1 of the 4 items measuring post-traumatic stress disorder were not able to discriminate between children with ASD with and without those anxiety disorders. All items of social anxiety and generalized anxiety were found to have sufficient discriminate ability (ps < .05). A list of the items that were (not) found to effectively discriminate between children with and without a particular anxiety disorder can be obtained from the first author.

Discussion

The aim of this study was to examine the psychometric properties of a questionnaire developed for typically developing children to assess anxiety in an ASD sample. Overall, internal consistency of the SCARED-71 was found to be good and evidence was found for construct validity (i.e. correlations within respondents between the ADIS-C/P and the SCARED-71 were all medium and reached significance). Although child–parent agreement tended to be lower in the ASD-group, the correlations did not differ significantly in magnitude from the AD-group. These findings support the notion of Reaven et al. (2012) that the possible problems associated with the use of self-report in children with ASD may not be different from those in non-ASD samples.

In contrast to the White et al. (2011) study, we found that about 80% of the children with ASD and comorbid anxiety disorders rated themselves to have clinical levels of anxiety (i.e. meet thresholds indicative for clinical anxiety), suggesting that children with ASD are able to self-report reliably about their anxiety levels. However, it is important to add here that the discriminating ability of the SCARED-71 child-report was found to be less in the ASD-group compared to parent-report in the ASD-group, and compared to child-report in the AD-group. However, note also that for the subscales with insufficient discriminate ability, the sample sizes of children with ASD meeting criteria for these anxiety disorders was rather small. This might reflect a lack of power rather than children being unable to report about these symptoms. Alternatively, children with ASD may have more trouble reporting about symptoms of obsessive–compulsive disorder, post-traumatic stress disorder, and panic disorder because these symptoms rely more on the identification and expression of internal experiences and thoughts (obsessive–compulsive disorder and post-traumatic stress disorder) as well as on having sufficient awareness of bodily sensations (panic disorder).

Parent-report was also not able to differentiate between children with and without obsessive–compulsive disorder in the ASD-group. This finding might reflect problems with symptom overlap between the two disorders (see e.g. Wood and Gadow, 2010). However, one might expect similar problems with social anxiety disorder. On the contrary, post hoc analyses revealed that all items of the social anxiety subscale were able to discriminate between children with ASD with and without this disorder.

Compared to the AD-group, sensitivity of the SCARED-71 parent-report was somewhat higher, however, at the cost of specificity. It may be that (1) parents overreport anxiety symptoms because of the considerable overlap between anxiety and ASD-symptoms or (2) that the scores reflect a true heightened level of anxiety in ASD because anxiety may be (partly) inherent to ASD (e.g. Sukhodolsky et al., 2008; White et al., 2009b). Nevertheless, raising the SCARED cutoffs for parent-report resulted in a better fit with acceptable sensitivity and specificity. These findings suggest that cutoffs developed for the typically developing population may not automatically be applicable to the ASD population and research should focus more on establishing alternative cutoffs for the ASD population.

One major limitation of this study is the selection of participants. That is, all the children who participated were referred to mental health centers and had at least one anxiety disorder. This selection has a number of disadvantages. First, we were not able to examine the specificity rate of the SCARED-71 total score. It might be that the SCARED-71 is sensitive for clinical anxiety also in children with ASD, however, that the specificity is lower (leading to high rates of false positives). Second, clinically anxious children (with and without ASD) may have higher anxiety symptoms on every subscale regardless of whether they have a particular anxiety disorder. This may also lead to lower specificity. Third, children who participated were—at least to some extent—aware of their anxiety problems, which may have increased child–parent agreement. For example, Nauta et al. (2004) demonstrated that child–parent agreement is higher in a clinical group compared to a group of typically developing children. A further limitation of the study is that we used the ADIS-C/P as the “standard” to measure anxiety disorders in ASD. This instrument is developed for typically developing children and—although applied in research to assess anxiety disorders in youth with ASD (e.g. Reaven et al., 2012; White et al., 2009a; Wood et al., 2009)—its validity or reliability in the ASD population is unknown. Finally, IQ was assumed to be above 70 based on clinical judgment (and in case of doubt, an IQ test was conducted). As a consequence, we cannot describe exactly how high functioning our ASD sample is, nor could we examine the possible influence of IQ on, for example, child–parent agreement.

Overall, the findings of this study suggest that the SCARED-71 may be used to assess anxiety in children with ASD as it has demonstrated acceptable psychometric properties. In addition, this questionnaire has the advantage to capture the symptoms of all anxiety disorders. Therefore, it might be suitable as a screening tool for anxiety disorders in children with ASD. However, replication of the findings, research examining its use for community samples of youth with ASD, and research comparing children with ASD with and without anxiety disorders is needed.

Footnotes

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

American Psychiatric Association (APA) (2000) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR). Washington, DC: American Psychiatric Association.

Birhamer

Brent

Chiapetta

. (1999) Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED). Journal of the American Academy of Child and Adolescent Psychiatry 38: 1230–1236.

Blakeley-Smith

Reaven

Ridge

. (2012) Parent-child agreement of anxiety symptoms in youth with autism spectrum disorders. Research in Autism Spectrum Disorders 6: 707–716.

Bodden

DHM

Bögels

Muris

(2009) The diagnostic Utility of the Screen for Child Anxiety Related Emotional Disorders-71 (SCARED-71). Behaviour Research and Therapy 47: 418–425.

Brereton

Tonge

Einfield

(2006) Psychopathology in children and adolescents with autism compared to young people with intellectual disability. Journal of Autism and Developmental Disorders 36: 863–870.

De Bruin

Ferdinand

Meester

. (2007) High rates of psychiatric co-morbidity in PDD-NOS. Journal of Autism and Developmental Disorders 37: 877–886.

Gadow

DeVincent

Schneider

(2009) Comparative study of children with ADHD only, autism spectrum disorder + ADHD, and chronic multiple tic disorder + ADHD. Journal of Attention Disorders 12: 474–485.

Gillot

Furniss

Walter

(2001) Anxiety in high-functioning children with autism. Autism 5: 277–286.

Green

Gilchrist

Burton

. (2000) Social and psychiatric functioning in adolescents with Asperger Syndrome compared with Conduct Disorder. Journal of Autism and Developmental Disorders 3: 279–293.

10.

Hudson

Rapee

Deveney

. (2009) Cognitive-behavioral treatment versus an active control for children and adolescents with anxiety disorders: a randomized trial. Journal of the American Academy for Child and Adolescent Psychiatry 48: 533–544.

11.

Kim

Szatmari

Bryson

. (2000) The prevalence of anxiety and mood problems among children with autism and Asperger Syndrome. Autism 4: 117–132.

12.

Leyfer

Folstein

Bacalman

. (2006) Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. Journal of Autism and Developmental Disorders 36: 849–861.

13.

Lord

Rutter

Le Couteur

(1994) Autism Diagnostic Interview - Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders 24: 659–685.

14.

March

Parker

JDA

Sullivan

. (1997) The Multidimensional Anxiety Scale for children (MASC): factor structure, reliability, and validity. Journal of the American Academy of Child and Adolescent Psychiatry 36: 554–565.

15.

Mazefsky

Kao

Oswald

(2011) Preliminary evidence suggesting caution in the use of psychiatric self-report measures with adolescents with high-functioning autism spectrum disorders. Research in Autism Spectrum Disorders 5: 164–174.

16.

Nauta

Scholing

Rapee

. (2004) A parent report measure of children’s anxiety: psychometric properties and comparison with child-report in a clinic and normal sample. Behaviour Research and Therapy 42: 813–839.

17.

Reaven

Blakeley-Smith

Culhane-Shelburne

. (2012) Group cognitive behavior therapy for children with high-functioning autism spectrum disorders and anxiety: a randomized trial. Journal of Child Psychology and Psychiatry 53: 410–419.

18.

Reynolds

Richmond

(1985) Revised Children’s Manifest Anxiety Scale Manual. Los Angeles, CA: Western Psychological Services.

19.

Silverman

Albano

(1996) Anxiety Disorders Interview Schedule for DSM-IV Child Version, Child Interview Schedule. San Antonio, TX: The Psychological Corporation.

20.

Silverman

Saavedra

Pina

(2001) Test–retest reliability of anxiety symptoms and diagnoses with the anxiety disorders interview schedule for DSM-IV: child and parent versions. Journal of the American Academy for Child and Adolescent Psychiatry 40: 937–944.

21.

Simon

Bögels

Voncken

(2011) Efficacy of child-focused and parent-focused interventions in a child anxiety prevention study. Journal of Clinical Child and Adolescent Psychology 40: 204–219.

22.

Simonoff

Pickles

Charman

. (2008) Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry 47: 921–929.

23.

Sukhodolsky

Scahill

Gadow

. (2008) Parent-rated anxiety symptoms in children with pervasive developmental disorders: frequency and association with core autism symptoms and cognitive functioning. Journal of Abnormal Child Psychology 36: 117–128.

24.

Van Steensel

FJA

Bögels

De Bruin

(2012) Psychiatric comorbidity in children with autism spectrum disorders: a comparison with children with ADHD. Journal of Child and Family Studies. Epub ahead 1 May 2012. DOI: 10.1007/s10826-012-9587-z.

25.

Van Steensel

FJA

Bögels

Dirksen

(2012) Anxiety and Quality of Life: clinically anxious children with and without autism spectrum disorders compared. Journal of Clinical Child and Adolescent Psychology. Epub ahead of print 09 July 2012. DOI:10.1080/15374416.2012.698725.

26.

Van Steensel

FJA

Bögels

Perrin

(2011) Anxiety disorders in children and adolescents with Autistic Spectrum Disorders: a meta-analysis. Clinical Child and Family Psychology Review 14: 302–317.

27.

White

Ollendick

Scahill

. (2009a) Preliminary efficacy of a cognitive-behavioral treatment program for anxious youth with autism spectrum disorders. Journal of Autism and Developmental Disorders 39: 1652–1662.

28.

White

Oswald

Ollendick

. (2009b) Anxiety in children and adolescents with autism spectrum disorders. Clinical Psychology Review 29: 216–229.

29.

White

Schry

Maddox

(2011) Brief report: the assessment of anxiety in high-functioning adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders. Epub ahead of print 27 August 2011. DOI: 10.1007/s10803-011-1353-3.

30.

Wood

Gadow

(2010) Exploring the nature and function of anxiety in youth with autism spectrum disorders. Clinical Psychology: Science and Practice 17: 281–292.

31.

Wood

Drahota

Sze

. (2009) Cognitive behavioral therapy for anxiety in children with autism spectrum disorders: a randomized, controlled trial. Journal of Child Psychology and Psychiatry 50: 224–234.