Exploring the agreement between questionnaire information and DSM-IV diagnoses of comorbid psychopathology in children with autism spectrum disorders

Abstract

Autism spectrum disorders are often comorbid with other psychiatric symptoms and disorders. However, identifying psychiatric comorbidity in children with autism spectrum disorders is challenging. We explored how a questionnaire, the Child Behavior Check List, agreed with a Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM-IV)-based semi-structured interview, the Kiddie-Schedule for Affective Disorders and Schizophrenia for School-Age Children (Kiddie-SADS). The sample comprised 55 children and adolescents (age 6 to 18 years) with autism spectrum disorders, including the main autism spectrum disorder subgroups and the broad range of cognitive and language functioning. High rate of psychopathology was found both through questionnaire and interview assessment. Using predefined Child Behavior Check List cutoffs, we found good agreement between the Child Behavior Check List and the Kiddie-SADS for identifying attention deficit/hyperactivity disorder, depressive disorders, and oppositional defiant disorder. However, overall the specificity of the Child Behavior Check List was low. The Child Behavior Check List was not useful for identifying anxiety disorders. The Child Behavior Check List may capture core symptoms of autism spectrum disorders as well as comorbid psychopathology, and clinicians should be aware that the Child Behavior Check List may be unspecific when used in children and adolescents with autism spectrum disorders.

Keywords

autism spectrum disorders children Child Behavior Check List DSM-IV disorders psychiatric comorbidity

Introduction

Comorbid psychiatric symptoms and disorders are common in children and adolescents with autism spectrum disorder (ASD). Prevalence rates of 70% for any disorder are frequently reported, and attention deficit/hyperactivity disorder (ADHD), anxiety disorders, and behavioral problems are particularly common (Brereton et al., 2006; Gjevik et al., 2011; Hartley et al., 2008; Simonoff et al., 2008). Furthermore, psychiatric comorbidity may add to the overall impairment in children with ASD (Leyfer et al., 2006; Sukhodolsky et al., 2008), has negative impact on intervention (Antshel et al., 2011), and represents distress and worries to caregivers (Herring et al., 2006; Lecavalier et al., 2006). However, studies indicate that comorbid psychopathology in children with ASD is treatable. Cognitive behavioral therapy (CBT) has shown favorable outcome for anxiety (Reaven et al., 2012), methylphenidate is shown to reduce ADHD symptoms (Posey et al., 2007), and antipsychotics are shown to have effect on behavioral problems (McPheeters et al., 2011). Thus, it is recommended that the clinical evaluation of children with ASD includes the assessment of comorbid psychopathology (Mattila et al., 2010; Simonoff et al., 2008). In parallel, Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM-IV) restrictions for diagnosing comorbid psychiatric disorders were controversial, and Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition (DSM-5) allows for more comorbid disorders in the presence of ASD (APA, 2013).

For successful treatment and good clinical care, it is essential that clinicians can identify and describe comorbid psychiatric disorders in children with ASD. However, intellectual disability and language impairment often make the diagnostic assessment difficult. Furthermore, there is overlap of symptoms between ASD and several childhood disorders, for example, repetitive behavior and restricted interests and obsessive–compulsive disorder (OCD), impairment of social interaction and social phobia, and unusual and stereotyped interests and behavior and symptoms of a psychotic disorder. Moreover, specialized instruments for assessing psychiatric comorbidity in children with ASD are hardly developed, and the psychometric properties of common childhood diagnostic tools when used in children with ASD are not well studied (Pandolfi et al., 2009; Witwer et al., 2012). Thus, there are no “gold-standard,” and questionnaires and interviews for general child population are commonly used.

The Child Behavior Check List (CBCL) is a parent-rated questionnaire assessment of child emotional and behavioral problems. It includes empirically derived syndrome scales measuring general psychopathology, and DSM-oriented scales constructed by expert agreement to be consistent with DSM-IV disorders. The CBCL is commonly used in child psychiatric practice and research, with well-documented psychometric properties in the general and clinical child population (Achenbach and Rescorla, 2001). The CBCL is also used to assess comorbid psychopathology in children with ASD, and high scores are reported on the Withdrawn, Social-, Attention-, Anxiety-, and Thought Problems scales (Hartley et al., 2008; Hurtig et al., 2009; Kanne et al., 2009; Kuusikko et al., 2008). Other studies have found the CBCL useful for identifying children with ASD (Biederman et al., 2010; Mazefsky et al., 2011; Ooi et al., 2011). However, these studies did not rule out the presence of comorbid psychopathology when studying how well the CBCL identified ASD.

The Kiddie-Schedule for Affective Disorders and Schizophrenia for School-Age Children (Kiddie-SADS) is a semi-structured interview designed to assess DSM-IV disorders. It consists of a screening interview covering the range of disorders and supplement interviews with questions related to the specific disorders. The Kiddie-SADS is widely used in child psychiatric practice and research, and has well-established psychometric properties (Kaufman et al., 1997). The Kiddie-SADS is also used in studies to assess the comorbid psychiatric disorders in children with ASD, and rates for any DSM-IV disorder ranging from 72% to 100% are reported (Gjevik et al., 2011; Joshi et al., 2010; Mattila et al., 2010; Mukaddes et al., 2010).

Psychiatric assessment is preferably based on the combined use of questionnaires and interviews. Diagnostic interviews capture symptoms as defined by the diagnostic categories and provide clinicians with information to determine whether a disorder is present. However, only symptoms above diagnostic threshold are recorded. Furthermore, interviews are cost and time consuming and require well-trained clinicians. Questionnaires identify the full range of psychiatric symptoms, and are easy and cost-effective screens to identify children likely to have a DSM-IV disorder. However, questionnaire assessment may be more vulnerable to errors based on the informants’ understanding and response to the questionnaire items.

The CBCL questionnaire and the Kiddie-SADS interview are two tools commonly used complementarily, and studies support the utility of the CBCL in identifying DSM-IV disorders in the general child psychiatric population, and comorbid DSM-IV disorders in children with ADHD (Achenbach and Rescorla, 2001; Bellina et al., 2013; Biederman et al., 2008; Krol et al., 2006). Information from the CBCL and the Kiddie-SADS is often what clinicians have available on comorbid psychopathology in children with ASD.

The aim of this study was to explore how the CBCL questionnaire agrees with the DSM-IV-based Kiddie-SADS interview in identifying comorbid psychiatric symptoms and disorders in children and adolescents with ASD, and to discuss the usefulness of combining these two tools in clinical practice.

Methods

Study participants

The study sample included 55 children and adolescents with ASD, all students registered at a school for children with ASD in Norway. The school provides a curriculum specialized for children with ASD based on eclectic teaching principles. A total of 26 children were students at the school. The rest were students at regular schools enrolled in an ASD special-education counseling program (n = 26), or on a waiting list for the program (n = 3). These 55 children represented a subsample of 71 participants in an earlier study assessing comorbid DSM-IV disorders in children with ASD (Gjevik et al., 2011). CBCL and Kiddie-SADS information were available in 55 participants, corresponding to 77% of the initial study population. All participants were previously diagnosed with ASD. However, a diagnostic confirmation was obtained using Autism Diagnostic Interview–Revised (ADI-R, Lord et al., 1997), conducted by a clinical psychologist certified as an ADI-R interviewer for research purpose. The participants had not previously undergone systematic assessment of comorbid psychopathology. The sample consisted of 46 boys and 9 girls; the male-to-female ratio was 5:1. A total of 42 children (76%) were Caucasian and 13 (24%) were non-Caucasian, mainly Asian. The mean age was 11.9 years (SD = 3.2, range, 6.3 to 17.9). A total of 33 children (60%) were <12 years, and 22 (40%) were ≥12 years. Nonverbal intellectual ability was assessed by the Leiter International Performance Scale–Revised (Leiter-R, Roid and Miller, 1997), and scores were available in 54 children. Mean Leiter-R score was 64.9 (SD = 29.2, range 30 to 129). A total of 16 children (30%) had Leiter-R score <40, 6 (11%) had scores 40 to 54, 10 (18%) had scores 55 to 69, and 22 (41%) had scores ≥70. Based on all available clinical information, the sample was diagnosed according to DSM-IV criteria for pervasive developmental disorders (PDDs). In all, 36 children had autistic disorder, 11 had Asperger’s disorder, and 8 had PDD–not otherwise specified. Sixteen children (28%) received medication: anticonvulsants for epilepsy (n = 8), antipsychotics for aggressive behavior (n = 6), and stimulants for ADHD symptoms (n = 2). Two children used both anticonvulsants and antipsychotics. None received psychiatric therapy for a comorbid disorder.

Measures

The CBCL (Achenbach and Rescorla, 2001), age 6 to 18 years, Norwegian translation, was used to assess comorbid psychiatric symptoms. The CBCL includes 112 questionnaire items; each rated on a three-point scale, and combined to form CBCL scales. The CBCL syndrome scales are developed through factor analysis of symptoms in the general child population, while the DSM-oriented scales are constructed by expert agreement to be consistent with DSM-IV disorders. The eight syndrome scales cover the following domains: Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Social Problems, Thought Problems, Attention Problems, Rule-Breaking Behavior, and Aggressive Behavior. Three broadband scales, Internalizing, Externalizing, and Total Problems, are formed from the syndromes scales. Six DSM-oriented scales cover Affective Problems, Anxiety Problems, Somatic Problems, ADHD Problems, Oppositional Defiant Problems, and Conduct Problems. Gender and age standardized T-scores for each CBCL scale are converted from raw scores and calculated by a computer program. A T-score of 50 represents average scores in typically developing children of the same age and gender, and each 10 points represents one standard deviation. Scores above borderline and clinical cutoffs indicate scores of concern. We used American cutoffs where borderline and clinical cutoffs are 60 and 64, respectively, on the broadband scales, and 65 and 70 on the Syndrome scales and DSM-oriented scales, respectively. Norway lacks national CBCL norms, but studies have shown that Norwegian children score low compared to American children and that Norway belongs to the so-called low-scoring societies (Kornor and Jozefiak, 2012). Thus, borderline cutoffs may be the most appropriate in Norwegian child populations. In this study, we primarily explored the agreement between the DSM-oriented scales and interview-identified DSM-IV disorders. Development of the Norwegian CBCL included a translation and back-translation. New questionnaire items were not added. The psychometric properties of the Norwegian CBCL have been evaluated in previous studies, and show acceptable criterion-related validity (sensitivity 71%, specificity 92%) and good internal consistency (α ≥ 0.8) for the broadband scales (Kornor and Jozefiak, 2012; Novik, 1999). The CBCL was used without modifications to the questionnaire items, and were filled out shortly, within days or 1 week, before the Kiddie-SADS interviews were conducted. The internal consistency (Cronbach’s alpha values) of the CBCL scales in the total sample ranged from 0.58 for the Withdrawn/Depressed scale, 0.59 for Social Problems, 0.63 for Anxiety Problems, 0.67 for Thought Problems, 0.67 for ADHD, 0.67 for Rule-Breaking Behavior, 0.68 for Somatic Problems, 0.75 for Affective Problems, 0.75 for Attention Problems, 0.75 for Somatic Complains, 0.76 for Oppositional Defiant Problems, 0.77 for Anxious/Depressed, 0.83 for Conduct Problems, 0.85 for Internalizing Problems, 0.86 for Aggressive Behavior, and 0.88 for Externalizing Problems. The values ranged from modest to good, but were lower than those provided by the CBCL manual (0.65 to 0.97).

The Kiddie-SADS (Kaufman et al., 1997) was used to assess DSM-IV disorder, and was conducted with one or both parents as reporters. The Kiddie-SADS consists of a lifetime and a present version, and present disorders were assessed in this study. The Kiddie-SADS screening interview was conducted with all parents and the six supplement interviews when indicated from the screening. Two child psychiatrists (E.S. and E.G.), with years of experience with children with ASD, performed the interviews. The Kidde-SADS was used without modifications, and DSM-IV criteria were used to establish diagnoses. DSM-IV exclusionary criteria for social phobia, separation anxiety, generalized anxiety, and ADHD in the presence of ASD were not applied. Otherwise, DSM-IV criteria were followed firmly. Social phobia was only diagnosed if there were expressed or observable symptoms of anxiety, and OCD was only diagnosed when there was anxiety and distress associated with repetitive thoughts and behaviors. The clinicians were blind to the CBCL results when the interviews were conducted. Inter-rater agreement on the interviews was assessed in the initial study population of 71 children and was based on 18 interviews. A total of 22 diagnoses were established on these interviews. There was full agreement on 19 diagnoses (86%), with disagreement on tic disorder in two children and ADHD in one child (Gjevik et al., 2011).

Statistical approach

As a measure of agreement between CBCL scales, at borderline and clinical cutoffs, and interview-identified DSM-IV disorders, we calculated kappa measure of agreement values. We also calculated (1) sensitivity—among the children who had a DSM-IV diagnosis, the number who had CBCL score above borderline and clinical cutoffs, and (2) specificity—among the children who did not have a DSM-IV diagnosis, the number who had CBCL score below borderline and clinical cutoffs. In addition, we conducted receiver operating characteristic (ROC) analyses and calculated area under the curve (AUC) values based on these. CBCL T-scores were used for the statistical analyses. Data from age and gender subgroups were combined, as a T-score, across subgroups, represents the same deviance in symptom load from average score in the general population of the child’s age and gender. Statistics were performed using PASW version 18.

Results

CBCL scale scores

The scales with the highest number of children in the clinical rage were the Total Problems scale (35 children, 64%), the Thought Problems scale (26 children, 47%), the Internalizing Problems scale (23 children, 42%), and the Attention Problems scale (22 children, 40%). The number of children with score in the normal, borderline, and clinical ranges, respectively, on the CBCL scales are presented in Table 1.

Table 1.

CBCL T-scores in 55 children and adolescents with ASD.

	Normal range, N (%)	Borderline range, N (%)	Clinical range, N (%)
Syndrome scales
Anxious/Depressed	40 (73)	7 (13)	8 (15)
Withdrawn/Depressed	24 (44)	19 (35)	12 (22)
Somatic Complaints	46 (84)	3 (5)	6 (11)
Social Problems	22 (40)	19 (35)	14 (25)
Thought Problems	17 (31)	12 (22)	26 (47)
Attention Problems	20 (36)	13 (24)	22 (40)
Rule-Breaking Behavior	45 (82)	6 (11)	4 (7)
Aggressive Behavior	33 (60)	11 (20)	11(20)
Internalizing Problems	26 (47)	6 (11)	23 (42)
Externalizing Problems	24 (44)	10 (18)	21 (38)
Total Problems	12 (22)	8 (15)	35 (64)
DSM-oriented scales
Affective Problems	31 (56)	9 (16)	15 (27)
Anxiety Problems	30 (55)	9 (16)	16 (29)
Somatic Problems	48 (87)	3 (5)	4 (7)
ADHD Problems	33 (60)	15 (27)	7 (13)
Oppositional Defiant Problems	37 (67)	12 (22)	6 (11)
Conduct Problems	31 (56)	18 (33)	6 (11)

CBCL: Child Behavior Check List; ASD: autism spectrum disorder; ADHD: attention deficit/hyperactivity disorder; DSM: Diagnostic and Statistical Manual of Mental Disorders.

Prevalence of DSM-IV disorders

A total of 40 children and adolescents (73%) were diagnosed with at least one comorbid DSM-IV disorder. The most prevalent diagnostic groups were anxiety disorders (24 children, 44%) and ADHD (17 children, 31%). Prevalence of comorbid DSM-IV disorders is presented in Table 2.

Table 2.

Prevalence of comorbid DSM-IV disorders in 55 children with ASD.

	N	%
Any psychiatric disorder	40	73
Any anxiety disorder	24	44
Specific phobia	17	31
Social phobia	5	9
OCD	5	9
Any ADHD	17	31
ADHD, inattentive type	11	20
ADHD, combined type	3	5
ADHD, hyperactive type	3	5
Any depressive disorder	6	11
Depressive disorder, NOS	5	9
Major depressive disorder	1	2
Any tic disorder
Chronic motor or vocal tics	8	15
ODD	4	7

DSM-IV: Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition; ADHD: attention deficit/hyperactivity disorder; NOS: not otherwise specified; ODD: oppositional defiant disorder; OCD: obsessive–compulsive disorder.

In all, 23 children were diagnosed with one comorbid psychiatric disorder, 12 children with two disorders, 3 children with three disorders, and 1 child with four disorders. Three children had more than one anxiety disorder.

Agreement between CBCL scales and DSM-IV disorders

Sensitivity and specificity of the CBCL scales are presented in Table 3. The CBCL showed good agreement with the Kiddie-SADS for identifying depressive disorders, ADHD, and ODD. Specificity was good for depressive disorders, but low for ADHD and ODD. The CBCL had low sensitivity and specificity for identifying anxiety disorders. Kappa values for the agreement between CBCL scales and Kiddie-SADS ranged from 0 to 0.5, and AUC values based on ROC analysis ranged from 0.7 to 0.9 (Table 3). Overall, the CBCL showed better agreement for identifying DSM-IV disorders at borderline than clinical cutoffs, although lowering cutoffs decreased specificity. The distribution of children with comorbid DSM-IV disorder, versus those with no disorder, on the CBCL scales is illustrated in Figures 1 to 4.

Table 3.

Agreement between CBCL scales and Kiddie-SADS identified DSM-IV disorders.

	Sensitivity (%)	Specificity (%)	Kappa	AUC for ROC analysis
Attention Problems				0.7
Cutoff 65	94	50	0.3
Cutoff 70	59	68	0.3
ADHD Problems				0.7
Cutoff 65	65	71	0.3
Cutoff 70	12	87	0.0
Affective Problems				0.9
Cutoff 65	100	63	0.3
Cutoff 70	100	82	0.5
Withdrawn/Depressed				0.7
Cutoff 65	83	47	0.1
Cutoff 70	50	82	0.2
Anxiety Problems				0.7
Cutoff 65	50	58	0.2
Cutoff 70	25	68	0.0
Oppositional Defiant Problems				0.8
Cutoff 65	75	71	0.2
Cutoff 70	50	92	0.3
Aggressive Behavior				0.9
Cutoff 65	75	63	0.1
Cutoff 70	75	84	0.4
Rule-Breaking Behavior				0.8
Cutoff 65	50	84	0.2
Cutoff 70	25	94	0.2
Internalizing Problems				0.7
Cutoff 60	65	59	0.2
Cutoff 64	62	76	0.4
Externalizing Problems				0.9
Cutoff 60	100	47	0.1
Cutoff 64	75	65	0.1
Total Problems				0.7
Cutoff 60	85	40	0.3
Cutoff 64	67	47	0.1

CBCL: Child Behavior Check List; SADS: Schedule for Affective Disorders and Schizophrenia for School-Age Children; DSM-IV: Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition; AUC: area under the curve; ROC: receiver operating characteristic.

Agreement between CBCL scales and Kiddie-SADS identified DSM-IV disorders: Attention Problems and ADHD problems versus ADHD, Affective Problems and Withdrawn/Depressed versus any depressive disorder, Anxiety Problems versus any anxiety disorder, Oppositional Defiant Problems, Aggressive Behavior and Rule-Breaking Behavior versus ODD, Internalizing Problems versus any anxiety or any depressive disorder, Externalizing Problems versus ODD, and Total Problems versus any psychiatric disorder.

Figure 1.

Using borderline cutoff, the CBCL Attention Problem scale identified 16 of 17 children (sensitivity 94%) with ADHD diagnosis. The ADHD Problems scale identified 11 (sensitivity 65%). In all, 19 and 11 children had scores above borderline cutoffs on the Attention Problems and the ADHD Problems scales, respectively, but no ADHD diagnosis.

Figure 2.

The CBCL Affective Problem scale identified all six children (sensitivity 100%) with depressive disorder, and the Withdraw/Depressed scale identified five (sensitivity 83%). A total of 18 and 26 children had scores above borderline cutoffs on the Affective Problem and the Withdraw/Depressed scales, respectively, but no depressive disorder.

Figure 3.

The CBCL Anxiety Problem scale identified 12 of 24 children (sensitivity 50%) with an anxiety disorder. Thirteen children had Anxiety Problems scores above borderline cutoff, but no anxiety disorder.

Figure 4.

The CBCL Oppositional Defiant Problems scale identified three of four children with ODD (sensitivity 75%). Fifteen children had scores above borderline cutoff, but no ODD diagnosis.

Discussion

In this explorative study of 55 children and adolescents with ASD, we found that both the CBCL and the Kiddie-SADS identified high rates of comorbid psychopathology. When comparing the two assessment tools, we found good agreement for identifying children with ADHD, depressive disorders, and ODD. However, overall, the specificity for the CBCL scales was low. The CBCL was not useful for identifying anxiety disorders. A high number of children had CBCL Thought Problems scores in the borderline and clinical range, while none were diagnosed with a psychotic disorder.

Few studies have explored the agreement between questionnaire and interview information of comorbid psychopathology in children with ASD. In line with our results, Pandolfi et al. (2012) found that CBCL showed moderate to high sensitivity, but low specificity, for identifying DSM-IV-defined anxiety disorders, ADHD, and ODD. Furthermore, they reported modest support for the Thought Problems scale as a measure of comorbid symptomatology.

It is relevant to ask whether we could have expected better agreement between the CBCL and the Kiddie-SADS. The CBCL DSM-oriented scales are constructed to overlap DSM-IV criteria, and good agreement could be expected. The syndrome scales, however, may tap different aspects of psychopathology when captured by the DSM categories, and poorer agreement is reasonable.

For ADHD, we found best agreement between the CBCL Attention Problems scale and interview diagnosis. We would expect best agreement with the DSM-oriented ADHD scale. However, these findings are plausible as ADHD inattentive type was the most prevalent ADHD subtype in our sample. A predominance of the ADHD inattentive type in children with ASD has also been reported by others (Gadow et al., 2006). Thus, the preferable use of the Attention Problems scales to identify ADHD may also apply to other ASD samples. The specificity Attention Problems scale was low, which could reflect that items such as “acts young,” “confused,” “day dreams,” “poor school,” and “stares” are unspecific for ADHD but also capture symptoms closely related to core ASD symptoms.

For depressive disorders, we found good agreement between the Affective Problems scale and the Kiddie-SADS. The agreement between the Withdrawn/Depressed scale and Kiddie-SADS was poorer, and the Withdrawn/Depressed scale showed only modest internal consistency. The Withdrawn/Depressed scale and the Kiddie-SADS may tap slightly different aspects of depression, with the Kiddie-SADS mainly the defining symptoms, such as depressed mood and lack of joy and interest, while the Withdrawn/Depressed scale also taps systems of social withdrawal and isolation associated with depression. However, several authors have recognized a risk of double scoring of ASD symptoms on the Withdrawn/Depressed scale, and excluded items on the scale when assessing comorbid psychopathology (Hurtig et al., 2009; Kuusikko et al., 2008).

We found poor agreement between the Anxiety Problems scales and anxiety disorders, and the Anxiety Problems scales showed modest internal consistency. One reason for the poor agreement could be that mainly specific phobia was diagnosed by the interview, while the Anxiety Problems scale captures more generalized aspects of anxiety. Furthermore, OCD and social phobia were identified by the interview, while symptoms of these disorders are not included on the Anxiety Problems scale.

The CBCL scales assessing behavioral problems all identified a greater number of children than the four with an ODD diagnosis. Rating scale studies have consistently reported high rates of behavioral problems in children with ASD (Brereton et al., 2006; Kanne and Mazurek, 2011), while findings in studies assessing DSM-IV-defined ODD/CD disorder are more inconsistent (De Bruin et al., 2007; Gjevik et al., 2011; Leyfer et al., 2006; Simonoff et al., 2008). This could indicate that the ODD diagnosis does not capture the essential characteristics of behavioral problems in children with ASD. On the other hand, the CBCL may overestimate behavioral problems. The CBCL includes items such as “breaking rules,” “not showing guilt,” “destroying others,” and “running away from home,” but it is questionable whether children with ASD exhibit the quality of the behavior as conceived in an ODD/CD diagnosis.

Taken together, our finding suggests that CBCL scales have some utility in identifying comorbid psychopathology in children with ASD. The usefulness was largest for ADHD and depressive disorders, while the CBCL was of little value for anxiety disorders and thought disorders. Furthermore, even for ADHD and ODD, the specificity of CBCL was low, and the use of CBCL should be followed by a diagnostic interview.

The internal consistency of some of the CBCL scales was low in our sample, raising questions concerning the reliability of the CBCL scales as a measure of comorbid psychopathology. It is important that these limitations are known when the CBCL is used in clinical practice.

In this study, we used the Kiddie-SADS interview as a “gold-standard” and calculated sensitivity and specificity of the CBCL relative to Kiddie-SADS diagnosis. However, Kidde-SADS also has limitations. Kiddie-SADS diagnoses are based on parent report, and do not represent clinical psychiatric diagnosis. Furthermore, the psychometric properties of the Kiddie-SADS interview when used to assess comorbid psychiatric disorders in children with ASD are not well studied. However, Leyfer et al. (2006) developed their autism comorbidity interview (ACI), based on minor modifications to Kiddie-SADS, and found that the ACI provided reliable DSM-IV diagnosis that was valid based on clinical psychiatric diagnosis. Furthermore, studies suggest that comorbid psychiatric disorders in children with ASD are phenotypically similar to conventional DSM-IV disorders (Lecavalier et al., 2009, 2011). Thus, we argue that for the purpose of exploring the clinical usefulness of diagnostic measures to identify psychiatric comorbidity in children with ASD, the DSM-based Kiddie-SADS interview is an appropriate comparison measure.

This study has several limitations. Sample size was relatively small, and replication in lager samples is needed. We used CBCL T-scores for statistical analyses, but further studies should use raw scores. Moreover, we used predefined borderline and clinical cutoffs on the CBCL scales. Other cutoffs may be more appropriate in children with ASD, but Figures 1 to 4 illustrate that these were not easily detectable in our data. Further studies should utilize ROC analyses within larger ASD samples, including age- and gender-specific subgroups, and determine optimal CBCL cutoffs. Interview and questionnaire assessments were based only on information from parents. Many children in our sample had intellectual disability and poor language skills, and only parent information was available from all participants. Furthermore, teacher reports were not obtained. However, in the interviews with the parents, we made sure that the questions also addressed the child’s behavior and performance in school. A number of children in our sample had moderate or severe intellectual disability, and the diagnostic challenges and inconsistency between questionnaire and interview assessment may in part be due to these intellectual difficulties, rather than to the ASD symptomatology per se. However, our sample represents the broad range of children with ASD, with regard to language and intellectual ability, seen in clinical practice. Most studies assessing psychiatric comorbidity in children with ASD have focused on the intellectually high-functioning children, and several authors have underlined the need for more research in clinically diverse samples (Pandolfi et al., 2012; Witwer et al., 2012).

Conclusion

This study is one of very few comparing questionnaire information and DSM-IV diagnoses of comorbid psychopathology in children and adolescents with ASD. We found good agreement between the CBCL and the Kiddie-SADS for identifying ADHD, depressive disorders, and ODD. However, overall, the specificity for the CBCL scales was low. We did not find the CBCL useful for identifying anxiety disorders and thought disorders. The CBCL may capture core symptoms of ASD as well as comorbid psychopathology, and clinicians should be aware that CBCL scales may be unspecific when used in children and adolescents with ASD.

Footnotes

Acknowledgements

We thank Mr Gunnar F. Lothe, University of Oslo, Oslo, Norway, for help with the figures.

Funding

The current project was funded by the University of Oslo, Oslo University Hospital, and the Research Council of Norway (#213694).

References

Achenbach

Rescorla

(2001) Manual for ASEBA-School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families.

American Psychiatric Association (APA) (2013) Diagnostic and Statistical Manual of Mental Disorders (DSM-5). Washington, DC: APA.

Antshel

Polacek

McMahon

. (2011) Comorbid ADHD and anxiety affect social skills group intervention treatment efficacy in children with autism spectrum disorders. Journal of Developmental and Behavioral Pediatrics 32(6): 439–446.

Bellina

Brambilla

Garzitto

. (2013) The ability of CBCL DSM-oriented scales to predict DSM-IV diagnoses in a referred sample of children and adolescents. European Child & Adolescent Psychiatry 22(4): 235–246.

Biederman

Ball

Monuteaux

. (2008) CBCL clinical scales discriminate ADHD youth with structured-interview derived diagnosis of oppositional defiant disorder (ODD). Journal of Attention Disorders 12(1): 76–82.

Biederman

Petty

Fried

. (2010) Child behavior checklist clinical scales discriminate referred youth with autism spectrum disorder: a preliminary study. Journal of Developmental and Behavioral Pediatrics 31(6): 485–490.

Brereton

Tonge

Einfeld

(2006) Psychopathology in children and adolescents with autism compared to young people with intellectual disability. Journal of Autism and Developmental Disorders 36(7): 863–870.

De Bruin

Ferdinand

Meester

. (2007) High rates of psychiatric co-morbidity in PDD-NOS. Journal of Autism and Developmental Disorders 37(5): 877–886.

Gadow

DeVincent

Pomeroy

(2006) ADHD symptom subtypes in children with pervasive developmental disorder. Journal of Autism and Developmental Disorders 36(2): 271–283.

10.

Gjevik

Eldevik

Fjaeran-Granum

. (2011) Kiddie-SADS reveals high rates of DSM-IV disorders in children and adolescents with autism spectrum disorders. Journal of Autism and Developmental Disorders 41(6): 761–769.

11.

Hartley

Sikora

McCoy

(2008) Prevalence and risk factors of maladaptive behaviour in young children with Autistic Disorder. Journal of Intellectual Disability Research 52(10): 819–829.

12.

Herring

Gray

Taffe

. (2006) Behaviour and emotional problems in toddlers with pervasive developmental disorders and developmental delay: associations with parental mental health and family functioning. Journal of Intellectual Disability Research 50(Pt 12): 874–882.

13.

Hurtig

Kuusikko

Mattila

. (2009) Multi-informant reports of psychiatric symptoms among high-functioning adolescents with Asperger syndrome or autism. Autism 13(6): 583–598.

14.

Joshi

Petty

Wozniak

. (2010) The heavy burden of psychiatric comorbidity in youth with autism spectrum disorders: a large comparative study of a psychiatrically referred population. Journal of Autism and Developmental Disorders 40(11): 1361–1370.

15.

Kanne

Abbacchi

Constantino

(2009) Multi-informant ratings of psychiatric symptom severity in children with autism spectrum disorders: the importance of environmental context. Journal of Autism and Developmental Disorders 39(6): 856–864.

16.

Kanne

Mazurek

(2011) Aggression in children and adolescents with ASD: prevalence and risk factors. Journal of Autism and Developmental Disorders 41(7): 926–937.

17.

Kaufman

Birmaher

Brent

. (1997) Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. Journal of the American Academy of Child and Adolescent Psychiatry 36(7): 980–988.

18.

Kornor

Jozefiak

(2012) CBCL: Child Behavior Checklist, 2-3, 4-18, 11/2-5 and 6-18. In: Kornor

(ed.) PsykTestBARN.

19.

Krol

De Bruyn

Coolen

. (2006) From CBCL to DSM: a comparison of two methods to screen for DSM-IV diagnoses using CBCL data. Journal of Clinical Child and Adolescent Psychology 35(1): 127–135.

20.

Kuusikko

Pollock-Wurman

Jussila

. (2008) Social anxiety in high-functioning children and adolescents with Autism and Asperger syndrome. Journal of Autism and Developmental Disorders 38(9): 1697–1709.

21.

Lecavalier

Gadow

DeVincent

. (2009) Validation of DSM-IV model of psychiatric syndromes in children with autism spectrum disorders. Journal of Autism and Developmental Disorders 39(2): 278–289.

22.

Lecavalier

Gadow

DeVincent

. (2011) Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism 15(5): 527–543.

23.

Lecavalier

Leone

Wiltz

(2006) The impact of behaviour problems on caregiver stress in young people with autism spectrum disorders. Journal of Intellectual Disability Research 50(Pt 3): 172–183.

24.

Leyfer

Folstein

Bacalman

. (2006) Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. Journal of Autism and Developmental Disorders 36(7): 849–861.

25.

Lord

Pickles

McLennan

. (1997) Diagnosing autism: analyses of data from the Autism Diagnostic Interview. Journal of Autism and Developmental Disorders 27(5): 501–517.

26.

McPheeters

Warren

Sathe

. (2011) A systematic review of medical treatments for children with autism spectrum disorders. Pediatrics 127(5): e1312–e1321.

27.

Mattila

Hurtig

Haapsamo

. (2010) Comorbid psychiatric disorders associated with Asperger syndrome/high-functioning autism: a community- and clinic-based study. Journal of Autism and Developmental Disorders 40(9): 1080–1093.

28.

Mazefsky

Anderson

Conner

. (2011) Child behavior checklist scores for school-aged children with autism: preliminary evidence of patterns suggesting the need for referral. Journal of Psychopathology and Behavioral Assessment 33(1): 31–37.

29.

Mukaddes

Herguner

Tanidir

(2010) Psychiatric disorders in individuals with high-functioning autism and Asperger’s disorder: similarities and differences. World Journal of Biological Psychiatry 11(8): 964–971.

30.

Novik

(1999) Validity of the child behaviour checklist in a Norwegian sample. European Child & Adolescent Psychiatry 8(4): 247–254.

31.

Ooi

Rescorla

Ang

. (2011) Identification of autism spectrum disorders using the Child behavior checklist in Singapore. Journal of Autism and Developmental Disorders 41(9): 1147–1156.

32.

Pandolfi

Magyar

Dill

(2009) Confirmatory factor analysis of the child behavior checklist 1.5-5 in a sample of children with autism spectrum disorders. Journal of Autism and Developmental Disorders 39(7): 986–995.

33.

Pandolfi

Magyar

Dill

(2012) An initial psychometric evaluation of the CBCL 6-18 in a sample of youth with autism spectrum disorders. Research in Autism Spectrum Disorders 6(1): 96–108.

34.

Posey

Aman

McCracken

. (2007) Positive effects of methylphenidate on inattention and hyperactivity in pervasive developmental disorders: an analysis of secondary measures. Biological Psychiatry 61(4): 538–544.

35.

Reaven

Blakeley-Smith

Culhane-Shelburne

. (2012) Group cognitive behavior therapy for children with high-functioning autism spectrum disorders and anxiety: a randomized trial. Journal of Child Psychology and Psychiatry and Allied Disciplines 53(4): 410–419.

36.

Roid

Miller

(1997) Leiter International Performance Scale–Revisited: Examiners Manual. Wood Dale, IL: Stoelting Co.

37.

Simonoff

Pickles

Charman

. (2008) Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry 47(8): 921–929.

38.

Sukhodolsky

Scahill

Gadow

. (2008) Parent-rated anxiety symptoms in children with pervasive developmental disorders: frequency and association with core autism symptoms and cognitive functioning. Journal of Abnormal Child Psychology 36(1): 117–128.

39.

Witwer

Lecavalier

Norris

(2012) Reliability and validity of the children’s interview for psychiatric syndromes—parent version in autism spectrum disorders. Journal of Autism and Developmental Disorders 42(9): 1949–1958.