Multiple Informant Agreement on the Anxiety Disorders Interview Schedule in Youth with Autism Spectrum Disorders

Abstract

Objective:

The purpose of this study was to examine child, parent, and clinician's consensus agreement on the Anxiety Disorders Interview Schedule, Child and Parent versions (ADIS-C/P) in a sample of children and adolescents with autism spectrum disorders (ASD).

Method:

Youth with ASD (n=85; age range=7–17 years) and their parents were each administered the ADIS-C/P by a trained clinician. Consensus diagnoses were determined in a clinical conference using best estimate procedures that incorporated all available information.

Results:

Children and youth with ASD diagnoses generally showed poor diagnostic agreement with parents and clinical consensus, whereas parents showed good-to-excellent diagnostic agreement with clinical consensus diagnoses. Diagnostic agreement between parents and consensus was moderated by the specific ASD diagnosis. Otherwise, the pattern of relationships did not systematically differ as a function of age or externalizing comorbidity.

Conclusions:

These data suggest that parent and youth agreement regarding the presence of clinical levels of anxiety is markedly poor among youth with ASD. Additionally, clinicians are likely to base their diagnostic impressions on parent report, placing minimal emphasis on child report.

Introduction

C linically significant anxiety is common among youth with autism spectrum disorders (ASD), substantially contributing to functional impairment (Ben-Sasson et al. 2008; Kelly et al. 2008; Sukhodolsky et al. 2008) and complicating diagnostic assessment (Wood and Gadow 2010). Unfortunately, assessment strategies for anxiety in youth with ASD are understudied. For example, diagnostic instruments developed and validated in neurotypical youth are being used to assess anxiety in youth with ASD without an adequate understanding of their utility and psychometric properties.

In ascertaining psychiatric diagnoses in children and youth, a multi-informant approach is recommended (Kendall et al. 2000; Crozier et al. 2011). However, the ability of children and youth with ASD and their parents to accurately identify anxiety symptoms has yet to be adequately examined. In addition to the usual confounds that influence parent–child agreement (e.g., Can parents report on child internalizing symptoms? Do parents overreport symptoms so the child is "accepted" for treatment? Do children provide socially desirable responses?) (Kendall and Flannery-Schroeder 1998), other factors relevant to ASD may influence children's ability to respond. For example, children and youth with ASD may lack insight into their symptoms; may have varying levels of social motivation and relatedness; and may have difficulty considering abstract concepts, language, and emotional states. Conversely, parents may struggle to understand and report on children's internal symptoms in the presence of communication differences, and/or may confuse core symptoms of ASD with anxiety in their children (e.g., describing stereotyped behaviors and fixated interests as anxiety-driven rituals and fear-based behavior). These obstacles represent a conundrum for the assessing clinician as parent–child disagreement is common in typically developing youth (Grills and Ollendick 2003), a finding that has been corroborated by our clinical experiences working with youth with ASD.

Although few in number, studies examining multi-informant agreement in neurotypical youth have shown variable agreement. Grills and Ollendick (2003) showed variable informant agreement in 165 typically developing youth seen in a clinic setting. Parent–child agreement across diagnoses were generally poor (κ ranging from 0.09 to 0.37); child–consensus agreement was poor for all categories except separation anxiety disorder (SAD) and generalized anxiety disorder (GAD) (κ ranging from 0.28- to 0.50); and parent–consensus agreement varied from poor to good (κ ranging from 0.37 to 0.70). Choudhury et al. (2003) showed poor parent–child agreement for anxiety disorders among 45 typically developing youth and their parents for principal diagnosis (κ ranging from 0.22 to 0.31) and presence of diagnosis anywhere in the clinical profile (κ ranging from 0.04 to 0.23). Using American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders, 3rd ed. (DSM-III) criteria, Rapee et al. (1994) showed poor parent–child agreement for primary diagnosis (κ ranging from 0.11 to 0.44) and presence of an anxiety diagnosis (κ ranging from 0.16 to 0.35). We are aware of only one study examining respondent agreement for non-autistic symptom severity in youth with ASD. In a sample of 177 youth with ASD (aged 3–18 years; 27% autism, 73% Asperger's disorder and pervasive developmental disorder–not otherwise specified [PDD-NOS]), Kanne et al. (2009) showed weak-to-moderate associations between subscales on the Teacher Report Form and the parent-rated Child Behavior Checklist that correspond to Using American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV) diagnoses (r=0.08–0.49). For example, parent and teacher correspondence on the DSM Anxiety Problems, Affective Problems, Somatic Problems, and Attention Deficit Hyperactivity Problems subscales were r=0.14, 0.08, 0.49, and 0.30, respectively (Kanne et al. 2009).

Certain variables among typically developing children have been suggested to moderate informant agreement, namely age and gender. Older age has been associated with improved agreement for certain diagnoses in some studies (Rapee et al. 1994; Choudhury et al. 2003), but has generally not been strongly related to agreement. Findings regarding gender have been mixed, with some evidence for better agreement for females with social phobia (Choudhury et al. 2003), as well as males with social phobia and males with SAD (Grills and Ollendick 2003), whereas others have found no difference (Rapee et al. 1994). Our clinical experience with youth with ASD suggests that other clinical variables may hold particular relevance, including presence of a disruptive behavior disorder and the child's specific ASD diagnosis. Theoretically, disruptive behavior has been hypothesized to be a proxy for limited insight, motivation, and willingness to report symptoms and engage in the assessment process (Storch et al. 2008). Our work with disruptive youth – neurotypical or with ASD – suggest that presence of this comorbidity is associated with more limited overall symptom endorsement by the child. The exact autism spectrum diagnosis may relate to agreement by virtue of disorder-specific differences in verbal ability, cognitive capacities (e.g., abstract thinking abilities), and social relatedness/motivation that may be related to improved rapport with the examiner or lessened social anxiety that may contribute to inhibited response patterns (e.g., child less likely to present self in an overly positive manner).

Given the increasing number of youth being diagnosed with ASD (i.e., 1 out of every 91 children and 1 in 58 boys (Fombonne 2005; Centers for Disease Control 2009; Kogan et al. 2009; Kim et al. 2011), together with prevalence of clinically significant anxiety in as many as 80% of youth with ASD (Muris et al. 1998; Leyfer et al. 2006; Sukhodolsky et al. 2008), it is critical to improve our understanding of the nature of informant agreement and factors that may moderate such agreement. With this in mind, the current study examines multiple informant agreement among youth with ASD presenting for inclusion in psychosocial treatment studies targeting anxiety. We had two primary research questions. First, we sought to examine the level of agreement among children, parents, and the resulting consensus clinician diagnoses that represent diagnoses based on a synthesis of all available information (Leckman et al. 1982). We expected child–parent and child–consensus agreement to be poor, whereas parent–consensus agreement was expected to be good. There are multiple bases for these expectations: 1) children and youth with ASD may exhibit a number of cognitive and interpersonal features that prevent accurate reporting of their symptoms (e.g., reduced insight, expressive language, self-reflection about emotional states), 2) children's anxiety symptoms will manifest themselves outwardly enabling parents to provide more accurate assessments, and 3) in the absence of clearly articulated child-reports of anxiety symptoms, clinicians will more heavily weigh ratings upon parent response. Second, we wanted to explore if agreement differed as a function of several theoretically or clinically relevant variables, namely age, specific ASD diagnosis, and disruptive behavior disorder comorbidity. The findings from the present investigation have clear implications: should children or parents be “better” reporters of certain diagnoses or overall, this may inform the clinician of how best to weigh information. Integrating data from multiple respondents is a complicated endeavor, and if certain factors are associated with poor agreement, it may be advisable to individualize assessments to reduce assessment burden.

Method

Participants were 85 children and early adolescents, age range 7–17 years, who were being screened for possible inclusion in studies examining cognitive-behavioral therapy for anxiety in children with ASD (participant demographics are in Table 1). Anxiety symptoms and diagnoses were assessed at an initial study visit. Autism spectrum disorder diagnosis was established at a second screening visit through administration of the Autism Diagnosis Interview-Revised (ADI-R) (Lord et al. 1994) and Autism Diagnostic Observation Schedule (ADOS)-Module 3 (Lord et al. 1999) by a certified rater, for those youth who exhibited elevated anxiety on the Pediatric Anxiety Rating Scale (PARS) >13 total score (RUPP 2002) and a diagnosis of GAD, obsessive-compulsive disorder (OCD), SAD, and/or social phobia. For those not meeting this anxiety threshold (i.e., did not have one of these diagnoses; n=9), autism diagnoses were determined using best estimate procedures that incorporated all available information including clinical interviews with the parent(s) and child, observation of the child in the context of an ∼3 hour assessment, completed forms as part of the respective study, and past records review. In such cases, clinical consensus on ASD diagnosis between two clinical psychologists was required to for the case to be included in the present report. There were no instances of disagreement regarding the presence of ASD or specific ASD diagnosis. Children had an intelligence quotient (IQ) ≥70 as assessed either at the time of the assessment or if the family had documentation of an accepted IQ test within the past 2 years. If applicable, youth were stable on any psychotropic medications prior to presentation (8 weeks for an antidepressant, 6 weeks for an antipsychotic).

Table 1.

Demographic and Clinical Characteristics of the Study Sample

Variable
Age (mean, SD)	10.34 (2.21)
Gender (n, % male)	65 (76.5%)
Ethnicity
Caucasian (n, %)	69 (81.2%)
Hispanic (n, %)	12 (14.1%)
Asian (n, %)	3 (3.5%)
African-American (n, %)	1 (1.2%)
ASD diagnosis
Autistic disorder (n, %)	29 (34.1%)
Asperger's disorder (n, %)	25 (29.4%)
Pervasive developmental disorder, not otherwise specified (n, %)	31 (36.5%)
Anxiety severity on Pediatric Anxiety Rating Scale (mean, SD)	16.53 (2.96)
On stable dose of psychotropic medication (n, %)	50 (58.8%)

SD=standard deviation; ASD=autism spectrum disorder.

Measures

Anxiety Disorder Interview Schedule, Child and Parent Versions (ADIS-C/P)

The ADIS-C/P (Silverman and Albano 1996) are clinician-administered, semistructured interviews that assesses for the presence and severity of DSM-IV anxiety disorders as well as dysthymia, major depression, attention-deficit/hyperactivity disorder (ADHD), conduct disorder, and oppositional defiant disorder (ODD). Each diagnosis is assigned a clinician's severity rating (CSR), a 0–8 rating of symptom severity and functional impairment. A minimum CSR of 4 is required to assign a particular diagnosis. The primary diagnosis is that with the highest clinician's severity rating. The ADIS-C/P yields separate diagnoses and CSRs based on the child and parent interviews. After diagnoses were separately derived from the Child and Parent Versions, the interviewer then made a composite diagnosis using recommended guidelines (Silverman and Albano 1996). Excellent validity properties have been reported (Wood et al. 2002). The ADIS-C does not assess child reports of externalizing diagnoses.

Autism Diagnostic Observation Schedule

The ADOS (Lord et al. 1999) is a clinician-administered semistructured assessment of autism spectrum disorder symptoms. Excellent inter-rater reliability, test–retest reliability, and internal consistency have been reported, as well as strong diagnostic sensitivity and specificity (Lord et al. 1999).

Autism Diagnosis Interview-Revised

The ADI-R (Lord et al. 1994) is a semistructured parent interview that evaluates the child's developmental history as well as current presenting symptoms. The ADI-R has shown strong internal consistency and adequate discriminant validity and diagnostic sensitivity and specificity (Lord et al. 1994, 1997; Mildenberger et al. 2001).

Procedures

Written informed consent and assent from the parent and child were obtained prior to starting study procedures. The ADIS-C/P were administered to children and parents by the same interviewer on a single day. The interviewer assigned DSM-IV diagnoses based on information gleaned from each respondent. Thereafter, consensus diagnoses were established in a clinical team conference that included two experienced clinical psychologists (EAS and ABL) and the interviewer. During this meeting, data from the individual child and parent interviews were discussed, along with other relevant information from measures not included in this article (e.g., PARS; RUPP 2002), to determine consensus diagnoses for the child. All interviews were audiotaped, and the team had access to such tapes to help determine consensus diagnoses. Within 10 days after completing both assessment points, participants returned for a baseline assessment prior to randomization to study treatments. At this assessment, measures not germane to this investigation were conducted.

Interviews were conducted by one of three clinicians with experience in assessing childhood anxiety in youth with and without ASD. Training consisted of didactic presentation, discussion, video exemplars, and calibration with criterion videotapes. As part of study procedures, ongoing reliability checks were in place for the ADIS-C/P and other clinician-administered measures that included the blind review of 20% of interviews to assess inter-rater reliability and rater drift.

Analytic plan

To evaluate inter-rater agreement on the ADIS-C/P, Cohen's κ (Cohen 1960) was used, which corrects for chance agreement among raters. According to criteria established by Mannuzza et al. (1989), values of κ<0.40 are considered poor, 0.40–0.60 are considered fair, 0.60–0.74 are considered good, and a κ>0.74 is considered to be indicative of excellent inter-rater agreement. We considered the agreement between raters on diagnoses of SAD, social phobia, specific phobia, GAD, OCD, unipolar depressive disorders (major depressive disorder and dysthymic disorder), and ODD.

The moderating influence of age, ASD diagnosis, and externalizing diagnosis was examined by evaluating the agreement among rating parties (i.e., parent, child, and consensus) after splitting the sample into groups for each moderating variable. Decisions on group membership followed precedents established in previous research on inter-rater agreement with the ADIS-C/P (Choudhury et al. 2003; Grills and Ollendick, 2003), where age was split into groups of younger (ages 7–10; n=45) and older youth (ages 11–18; n=40); ASD diagnosis was divided into autistic disorder (n=29), Asperger's disorder (n=25), and PDD-NOS (n=31); and those children with an ADIS-C/P disruptive behavior disorder diagnosis (i.e., ODD or conduct disorder) were grouped as externalizing (n=30) whereas all others were categorized as non-externalizing (n=55).

In order to evaluate if child endorsement of diagnosis added any information above and beyond parental report in making a consensus diagnosis, a series of logistic regressions were run for each individual anxiety disorder diagnosis, with parent and child diagnostic endorsement (via the ADIS-C/P) as the predictors and consensus diagnosis as the criterion. Given the sparse distribution of some of the outcome data (i.e., given the strong parental agreement with clinician, for most diagnoses there were relatively few instances where parent and consensus disagreed on diagnostic status), exact logistic regression was used (Derr 2009). This procedure relies on exact conditional inference and is preferable to maximum likelihood estimation in traditional logistic regression in situations with skewed or sparse distributions of categorical outcomes. Results were evaluated with exact p-values. Because of perfect agreement between parent and consensus diagnosis on the presence of any anxiety disorder in the diagnostic profile, exact logistic regressions were not considered for this diagnostic condition.

Results

Overall sample

Results of analyses of inter-rater agreement between child, parent, and consensus are in Table 2. As expected, child agreement with parent and consensus was generally poor, except for agreement between child and consensus on depressive disorder diagnoses, which was fair (κ=0.45). In contrast, parent and consensus agreement across diagnoses was uniformly excellent (i.e., κ>0.74). When considering relative risk ratios in the context of each clinical diagnosis evaluated, youth were 32–76% as likely to support individual diagnoses relative to parents or clinicians. In contrast, parents were anywhere from equally likely to 30% more likely than clinicians to endorse diagnoses. Only 49 of 85 youth found agreement, with either consensus or parents, on the presence of any anxiety disorder, whereas 80 of 85 parents agreed with clinicians on the presence of any anxiety disorder.

Table 2.

Agreement Between Parent, Child, and Consensus on the ADIS-C/P for Children with ASD as Measured by Cohen's κ

	Child-Parent						Child-Consensus						Parent-Consensus
Diagnosis	κ	++	+−	−+	− −	RR	κ	++	+−	−+	− −	RR	κ	++	+−	−+	− −	RR
Separation anxiety disorder	0.20	13	10	21	41	0.68	0.35^**	16	7	18	44	0.68	0.80^**	30	4	4	47	1.00
Social phobia	0.07	22	4	44	15	0.39	0.19^**	25	1	41	18	0.39	0.80^**	63	3	3	16	1.00
Specific phobia	0.23^*	24	5	31	25	0.51	0.21^*	21	8	27	29	0.60	0.78^**	47	8	1	29	1.15
Generalized anxiety disorder	0.08	13	2	49	21	0.24	0.11	14	1	48	22	0.24	0.76^**	58	4	4	19	1.00
Obsessive compulsive disorder	0.23^*	11	6	23	45	0.50	0.27^**	11	6	20	48	0.55	0.83^**	29	5	2	49	1.10
Any anxiety disorder	0.16^**	49	0	31	5	0.61	0.16^**	49	0	31	5	0.61	1.00^**	80	0	0	5	1.00
Depressive disorders	0.36^**	4	2	9	70	0.46	0.45^**	4	2	6	73	0.60	0.85^**	10	3	0	72	1.30
Oppositional defiant disorder^a	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	0.89^**	26	2	2	55	1.00

Note: ++=Agreement by both parties on presence of diagnosis; +−=First informant in pair endorses diagnosis, no endorsement by second informant; −+=Second informant in pair endorses diagnosis, no endorsement by first informant; − −=Agreement by both parties of no endorsement of diagnosis; RR=risk ratio evaluating the probability of the endorsement of diagnosis by first informant in pair relative to endorsement by second informant in pair.

The ADIS-C/P does not incorporate the assessment of oppositional defiant disorder with youth.

ADIS-C/P=Anxiety Disorders Interview Schedule, Child and Parent versions; ASD=autism spectrum disorder.

p<0.05.

p<0.01.

Moderators of diagnostic agreement

Parent–child

Inter-rater agreement between parent and child in the presence of the hypothesized moderators is found in Table 3. Overall, agreement was poor across all levels of the potential moderating variables examined, with the only exceptions being a good level of agreement for OCD in youth with Asperger's disorder, and fair levels of agreement for OCD and depressive disorders in non-externalizing children, and for SAD in youth with Asperger's disorder. Age did not have systematic effects on diagnostic agreement, although older children agreed more frequently with parents on diagnoses of separation anxiety disorder, specific phobia, and OCD than did younger children. Parent–child agreement on depressive disorders was higher for both age groups than for anxiety disorders, but was still poor overall. ASD diagnosis type did not show a systematic pattern of effects on diagnosis; whereas agreement was better for SAD for youth with Asperger's disorder relative to other ASD diagnoses, it was still in the low end of the “fair” range. Non-externalizing youth had better agreement with parents on diagnoses of specific phobia, OCD, and depressive disorders relative to externalizing youth.

Table 3.

Agreement Between Parent and Child on the ADIS-C/P for Children with ASD by Age, Specific ASD Diagnosis, and Externalizing Status as Measured by Cohen's κ

Diagnosis	Ages 7–10	Ages 11–17	Autistic disorder	Asperger's disorder	PDD-NOS	Non-Externalizing	Externalizing
Separation anxiety disorder	0.06	0.31^*	0.04	0.42^*	0.15	0.18	0.08
Social phobia	0.09	0.05	0.03	0.09	0.06	0.06	0.09
Specific phobia	0.09	0.33^*	0.22	0.14	0.27	0.39^**	−0.10
Generalized anxiety disorder	0.03	0.14	0.12	0.11	−0.01	0.16	−0.04
Obsessive compulsive disorder	0.11	0.33^*	−0.01	0.72^**	0.07	0.40^**	−0.07
Any anxiety disorder	0.11	0.22^*	NA^a	0.30^*	0.16	0.21^*	NA^c
Depressive disorders	0.29	0.39^**	0.27	0.39^*	NA^b	0.51^**	0.14

All youth with autistic disorder were classified by parents as having an anxiety disorder.

No youth with PDD-NOS were classified by parents as having a unipolar depressive disorder.

All parents of children and youth with externalizing diagnoses reported their child as having at least one anxiety disorder.

ADIS-C/P=Anxiety Disorders Interview Schedule, Child and Parent versions; ASD=autism spectrum disorder; PDD-NOS=pervasive developmental disorder–not otherwise specified.

p<0.05

p<0.01

Child–consensus

Table 4 presents inter-rater agreement between child and consensus in the presence of the hypothesized moderators. Agreement between child and consensus over both age groups was largely poor, except for a diagnosis of depressive disorders in adolescents (which was fair). Diagnostic agreement was largely poor among all ASD diagnosis groups, with the exceptions of OCD in children with Asperger's disorder and depressive disorders for youth with autistic disorder (which were in the “good” range); there was otherwise no discernible pattern of group differences. With regard to externalizing child behavior, agreement for children and youth in the non-externalizing group were in the “fair” range on a diagnosis of OCD and in the “good” range for depressive disorders (both diagnoses were higher on agreement relative to externalizing children and youth).

Table 4.

Agreement Between Child and Consensus on the ADIS-C/P for Children with ASD by Age, Specific ASD Diagnosis, and Externalizing Status as Measured by Cohen's κ

Diagnosis	Ages 7–10	Ages 11–17	Autistic disorder	Asperger's disorder	PDD-NOS	Non-Externalizing	Externalizing
Separation anxiety disorder	0.12	0.64^**	0.20	0.42^*	0.41^*	0.31^*	0.29
Social phobia	0.21^*	0.16	0.29^*	0.09	0.16	0.23^*	0.12
Specific phobia	0.06	0.33^*	0.05	0.22	0.33^*	0.38^**	−0.10
Generalized anxiety disorder	0.05	0.18^*	0.15	0.09	0.10	0.16	0.04
Obsessive compulsive disorder	0.29^*	0.26	0.16	0.72^**	−0.06	0.45^**	−0.03
Any anxiety disorder	0.11	0.22^*	NA^a	0.30^*	0.16	0.21^*	NA^c
Depressive disorders	0.37^*	0.49^**	0.65^**	0.39^*	NA^b	0.64^**	0.19

All youth with autistic disorder were classified by parents as having an anxiety disorder.

No youth with PDD-NOS were classified by parents as having a unipolar depressive disorder.

All parents of children and youth with externalizing diagnoses reported their child as having at least one anxiety disorder.

ADIS-C/P=Anxiety Disorders Interview Schedule, Child and Parent versions; ASD=autism spectrum disorder; PDD-NOS=pervasive developmental disorder–not otherwise specified.

p<0.05.

p<0.01.

Parent–consensus

Results from analyses of inter-rater agreement between parent and consensus when considering the hypothesized moderators can be found in Table 5. Agreement between parent and consensus was in the “excellent” or “good” range at all moderator levels for age and externalizing diagnosis. Parents of children with externalizing diagnoses showed somewhat lower agreement than those with non-externalizing children, but all agreement levels were still in the “excellent-to-good” range. With regard to ASD diagnosis, diagnostic agreement was excellent for all anxiety disorders for youth with Asperger's disorder and PDD-NOS (with the exception of SAD for PDD-NOS, which was at the high end of the “good” range), whereas diagnostic agreement for autistic disorder was only in the “fair-to-good” range for all anxiety disorders, and in the “poor” range for depressive disorders. However, diagnostic agreement was excellent for youth with autistic disorder on the diagnosis of ODD.

Table 5.

Agreement Between Parent and Consensus on the ADIS-C/P for Children with ASD by Age, Specific ASD Diagnosis, and Externalizing Status as Measured by Cohen's κ

Diagnosis	Ages 7–10	Ages 11–17	Autistic disorder	Asperger's disorder	PDD-NOS	Non-Externalizing	Externalizing
Separation anxiety disorder	0.87^**	0.70^**	0.70^**	1.00^**	0.74^**	0.78^**	0.80^**
Social phobia	0.81^**	0.78^**	0.52^**	1.00^**	0.82^**	0.78^**	0.84^**
Specific phobia	0.69^**	0.85^**	0.57^**	0.84^**	0.93^**	0.85^**	0.65^**
Generalized anxiety disorder	0.77^**	0.75^**	0.55^**	0.88^**	0.85^**	0.76^**	0.63^**
Obsessive compulsive disorder	0.79^**	0.85^**	0.66^**	1.00^**	0.84^**	0.84^**	0.80^**
Any anxiety disorder	1.00^**	1.00^**	NA^a	1.00^**	1.00^**	1.00^**	NA^d
Depressive disorders	0.79^**	0.86^**	0.37^*	1.00^**	NA^b	0.84^**	0.87^**
Oppositional defiant disorder	1.00^**	0.76^**	0.92^**	0.91^**	0.86^**	NA^c	0.63^**

All youth with autistic disorder were classified as having an anxiety disorder by both parents and clinical consensus.

No youth with PDD-NOS were classified as having a unipolar depressive disorder by either parents or clinical consensus.

By definition, no children or youth with oppositional defiant disorder would be considered non-externalizing.

All parents of children with externalizing diagnoses reported their child as having at least one anxiety disorder.

ADIS-C/P=Anxiety Disorders Interview Schedule, Child and Parent versions; ASD=autism spectrum disorder; PDD-NOS=pervasive developmental disorder–not otherwise specified.

p<0.05.

p<0.01.

Evaluating additive value of child report in making consensus diagnoses

Parent report predicted consensus diagnosis via exact logistic regression for all anxiety diagnoses and a diagnosis of a depressive disorder at the p<0.01 level. Child report only added information above and beyond parent report for diagnoses of SAD (p<0.01) and social phobia (p<0.01). Child report did not add statistically significant increases in predictive information above parent report for predicting consensus diagnoses of specific phobia (p=1.00), GAD (p=0.27), OCD (p=0.41), and unipolar depressive disorders (p=0.59).

Discussion

The present study provides the first evaluation of child–parent–consensus agreement on the presence of diagnostic levels of psychiatric disorders in youth with high functioning ASD. Not surprisingly, and consistent with findings in youth without ASD that showed κ ranging from 0.09 to 0.37 across diagnoses (Grills and Ollendick 2003) and κ ranging from 0.22 to 0.31 for prinicpal diagnosis (Choudhury et al. 2003), child–parent agreement on the presence of most anxiety disorders was poor. Our data suggest youth with ASD underreport anxiety symptoms compared with their parents; the probability of child endorsement of diagnoses was 32–76% less than for their parents for the five anxiety disorders assessed on the ADIS-C/P. Qualitatively, the most common patterns were: 1) child did not endorse the disorder/parent endorsed the disorder and 2) both agreed upon the absence of the disorder. In very few cases did the child endorse a particular disorder in the absence of parental endorsement. Notably, when examining the presence of any ADIS-C/P anxiety disorder, only 49 out of 85 youth agreed with parents regarding the presence of an anxiety disorder. Therefore, as hypothesized, consensus ADIS-C/P diagnoses corresponded closely with parent ADIS-C/P ratings but not with child ratings.

Generally, these results in a sample of youth with ASD parallel findings by Grills and Ollendick (2003) obtained in youth without ASD. Specifically, child–parent and child–consensus agreement was poor in both samples (κ ranging from 0.28 to 0.50 in Grills and Ollendick [2003]). However, in contrast to the Grills and Ollendick (2003) non-ASD sample in which κ for parent–consensus agreement exceeded 0.60 (“good” agreement) for only one anxiety disorder (GAD), κ exceeded 0.74 (“excellent” agreement) for all anxiety disorders in our sample. Data from Grills and Ollendick (2003) as well as others (Orvaschel et al. 1981; Jensen et al. 1988a; Silverman and Eisen 1992) suggest that parent–consensus agreement appeared higher for externalizing diagnoses than for internalizing diagnoses. In our sample, however, parent–consensus agreement was consistently strong for anxiety and depressive disorders and ODD. These findings highlight the emphasis raters placed on parent reports in youth with ASD, as many children reported minimal symptoms. When comparing these data to findings in typically developing youth, it is possible that lower levels of parent–consensus agreement is the result of reduced emphasis on parent perspectives of their child's internalizing symptoms and/or that parents may struggle to report on their child's anxiety/depressive symptoms.

Contrary to our expectations and findings in typically developing youth (Jensen et al. 1988b; Rapee et al. 1994; Dadds et al. 1998), neither age nor externalizing symptomatology consistently moderated agreement between raters. There were some limited findings; for example, children with externalizing diagnoses showed worse agreement with clinicians and parents on diagnoses of specific phobia, OCD, and depressive disorders than did their non-externalizing counterparts. This may be because internalizing symptoms are overshadowed by concurrent disruptive behavior, and/or a lack of insight into their behavior among youth with ASD and comorbid disruptiveness, which manifests in reduced reporting. Overall, there may be explanations for the general lack of moderation. One is that all participants were diagnosed with ASD and may have had poor insight into their psychiatric symptoms; in this way, an ASD diagnosis is the principal moderator, and there is no little additional moderation to be observed among other variables given this interference with symptom insight.

However, specific ASD diagnosis showed particular moderation effects for parent-consensus agreement on comorbid diagnoses. Specifically, agreement for anxiety and depressive disorders were lower for youth with autistic disorder relative to other ASD diagnoses, whereas agreement on ODD remained excellent. This may be because there is difficulty differentiating anxiety from core ASD symptoms (e.g., repetitive behaviors) in youth with autistic disorder and/or ascertaining the cause of avoidance behavior in youth with more pronounced ASD symptoms, as it can be difficult to discern whether such avoidance is the product of anxiety or stereotyped habits. However, a diagnosis of ODD requires no such insight into the etiology of behavior and overlaps less with anxiety; therefore, such externalizing behavior may be readily recognized regardless of ASD presentation.

Limitations

There are a number of limitations inherent in the present study. First, there was no independent criterion for diagnostic status. Although the clinician-rated consensus represents information gathered from all available sources (observation, available records, rating scales), the consistent and primary sources of information were the ADIS-C/P reports. Consequently, the consensus diagnosis is not an independent criterion, as independent clinician evaluations were not conducted. Second, although parent report appears to be the basis for the consensus diagnostic formulation and consequently appears to be the “better” report, these data cannot establish the internal state of the child (Flanery 1990). Autism spectrum presentation contributes to a foggy lens through which assessment of anxiety must be established. Inherent difficulties with emotion recognition combined with struggles in articulating relevant details pertaining to the experience of internal emotional experiences necessitate information from other sources (e.g., parents), but parent report remains no more than an inference. However, parents may also err in the opposite direction, underestimating the intensity of child internalizing states (Kurdek and Berg 1987). Third, the marked frequency of child rejection of symptoms in contrast to parent reports is striking. Although awareness/insight/articulation may explain the discrepancy, it is possible that youth with ASD are denying symptoms for social conformity or other intentional reasons (e.g., to truncate the interview, avoid cognitive dissonance, offset embarrassment/discomfort). In short, the reason for the parent–youth discrepancy cannot be explained based on the present data. Finally, parents may overreport symptoms. Based on our clinical experiences with these youth, parents often present as distressed and desperately seeking care for their children. It is not uncommon for parents to describe idiosyncratic behavior, social avoidance, and routinized behavior as anxiety, wherein these behaviors are better understood as manifestations of ASD (Wood and Gadow 2010).

Conclusions and Clinical Significance

Study findings have clear implications for the assessment of anxiety in youth with ASD. Because psychosocial and pharmacological therapies for anxiety in ASD are increasingly being tested, it is critical to provide data regarding the psychometric properties of relevant clinician-administered assessments. These data suggest that parents are the “better” reporters of information pertaining to diagnostic formulation overall; or, at least that clinicians should weigh parental report more heavily. Given this, time-consuming assessments focused on the child with ASD may be minimized so as to reduce assessment burden without losing any particularly valuable contributions. Second, the structured interview report may not be optimal for assessing anxiety in youth with ASD. Concreteness and limited insight inherent in ASD, combined with anxiety, may necessitate a more flexible approach to presenting and interpreting questions about internal subjective states (e.g., fears, worries) which can be difficult using a structured interview. In other words, other techniques (e.g., observation, role playing) or presenting queries in the context of specific life examples personally relevant to these youth may be more useful in establishing diagnostic criteria for anxiety disorders. Anecdotally, many children and youth with ASD are less likely to answer a general question about worry, e.g., “Do you worry about your family's health?” but may respond differently to a specific question, “Do you sometimes worry that your mom has cancer?” The difficulty can lie in determining the ideal wording for most children and youth with ASD.

Notably, this study highlights some of the challenges of identifying and establishing the correct anxiety disorder diagnoses among youth with ASD. First, youth with ASD are likely to underreport symptoms of anxiety and unipolar mood disorders. Consequently, the burden is placed on the parent to recognize these diagnoses and to communicate observations to the clinician. Our data seem to suggest that this is particularly difficult for those youth with autistic disorder relative to Asperger's disorder and PDD-NOS. Second, integrating data from multiple respondents in youth with anxiety and unipolar depressive disorders is a complicated endeavor, in which a uniform approach to integrating family diagnostic reports is lacking. For adolescents without ASD who have internalizing disorders, structured diagnostic interviews with the adolescents are generally considered sufficient (Hope et al. 1999). Conversely, our data suggest that with few exceptions, structured diagnostic interviews with the parents are sufficient for youth with internalizing disorders and comorbid ASD. In this way, a diagnosis of ASD may moderate the relative weighting of information by reporting parties in establishing a consensus diagnosis. Further research with other multimethod, multisource assessments (including observation, teacher report, and rating scales by multiple informants) may also further our understanding as to the information most relevant to establishing accurate diagnostic impressions of non-ASD Axis I diagnoses in youth with ASD (Jensen et al. 1999). Whereas a parsimonious algorithm for all children and youth with internalizing disorders remains elusive, these data exemplify substantial differences in the relative contributions of parents and children to the diagnostic formulations of common comorbid psychiatric conditions in the context of ASD.

Footnotes

Disclosures

Dr. Storch has received grant funding in the last 3 years from the National Institutes of Health, Agency for Healthcare Research and Quality, All Children's Hospital Research Foundation, Centers for Disease Control, National Alliance for Research on Schizophrenia and Affective Disorders, International OCD Foundation, Tourette Syndrome Association, Janssen Pharmaceuticals, and Foundation for Research on Prader–Willi Syndrome. He receives a textbook honorarium from Springer publishers, American Psychological Association, and Lawrence Erlbaum. Dr. Storch has been an educational consultant for Rogers Memorial Hospital. He is a consultant for Prophase, Inc. and is on the Speaker's Bureau and Scientific Advisory Board for the International OCD Foundation. He receives research support from the All Children's Hospital Guild Endowed Chair. Dr. Ehrenreich May receives grant funding from the National Institutes of Health. Dr. Wood receives grant funding from the National Institutes of Health. Ms. Jones, Mr. De Nadai, and Ms. Arnold have no disclosures to report. Dr. Lewin receives grant funding from the National Institutes of Health, Agency for Healthcare Research and Quality, National Alliance for Research on Schizophrenia and Affective Disorders, and International OCD Foundation. He is a consultant for Prophase, Inc. Dr. Murphy has received research support in the past 3 years from National Institutes of Health, Forest Laboratories, Janssen Pharmaceuticals, Endo, Obsessive International OCD Foundation, Tourette Syndrome Association, All Children's Hospital Research Foundation, Centers for Disease Control, and National Alliance for Research on Schizophrenia and Affective Disorders. Dr. Murphy is on the Medical Advisory Board for Tourette Syndrome Association. She receives a textbook honorarium from Lawrence Erlbaum.

Acknowledgments

The contributions of Sarah Gunderson, Jessica Morgan, Amy Parks, Psy.D., and Jennifer Park are gratefully acknowledged.

References

Ben-Sasson

, Cermak

, Orsmond

, Tager-Flusberg

, Kadlec

, Carter

. Sensory clusters of toddlers with autism spectrum disorders: Differences in affective symptoms. J Child Psychol Psychiatry, 49:817–825. 2008.

Centers for Disease Control:. Prevalence of the autism spectrum disorders–autism and developmental disabilities monitoring (ADDM) network. MMWR Surveill Summ, 58:1–20. 2009.

Choudhury

, Pimentel

, Kendall

. Childhood anxiety disorders: Parent–child (dis)agreement using a structured interview for the DSM-IV. J Am Acad Child Adolesc Psychiatry, 42:957–964. 2003.

Cohen

. A coefficient of agreement for nominal scales. Educ Psychol Meas, 20:37–46. 1960.

Crozier

, Gillihan

, Powers

. Issues in differential diagnosis: Phobias, phobic conditions. Handbook of Child and Adolescent Anxiety Disorders. McKay

, Storch

E.A.

New York: Springer, 2011; 7–22.

Dadds

, Perrin

, Yule

. Social desirability and self-reported anxiety in children: An analysis of the RCMAS Lie scale. J Abnorm Child Psychol, 26:311–317. 1998.

Derr

. Performing Exact Logistic Regression within the SAS System–Revised 2009. Cary, NC: SAS Institute, 2009.

Flanery

. Methodological, psychometric considerations in child reports. Through the Eyes of a Child: Obtaining Self-Reports from Children and Adolescents. LaGreca

A.M.

Boston: Allyn & Bacon, 1990; 57–82.

Fombonne

. Epidemiology of autistic disorder and other pervasive developmental disorders. J Clin Psychiatry, 66,Suppl 10:3–8. 2005.

10.

Grills

, Ollendick

. Multiple informant agreement and the anxiety disorders interview schedule for parents and children. J Am Acad Child Adolesc Psychiatry, 42:30–40. 2003.

11.

Hope

, Adams

, Reynolds

, Powers

, Perez

, Kelley

. Parent vs. self-report: Contributions toward diagnosis of adolescent psychopathology. J Psychopath Behav Assess, 21:349–363. 1999.

12.

Jensen

, Rubio–Stipec

, Canino

, Bird

, Dulcan

, Schwab–Stone

, Lahey

. Parent and child contributions to diagnosis of mental disorder: Are both informants always necessary? J Am Acad Child Adolesc Psychiatry, 38:1569–1579. 1999.

13.

Jensen

, Traylor

, Xenakis

, Davis

. Child psychopathology rating scales and interrater agreement: I. Parents' gender and psychiatric symptoms. J Am Acad Child Adolesc Psychiatry, 27:442–450. 1988a.

14.

Jensen

, Xenakis

, Davis

, Degroot

. Child psychopathology rating scales, interrater agreement: II. Child and family characteristics. J Am Acad Child Adolesc Psychiatry, 27:451–461. 1988b.

15.

Kanne

, Abbacchi

, Constantino

. Multi-informant ratings of psychiatric symptom severity in children with autism spectrum disorders: The importance of environmental context. J Autism Dev Disord, 39:856–864. 2009.

16.

Kelly

, Garnett

, Attwood

, Peterson

. Autism spectrum symptomatology in children: The impact of family and peer relationships. J Abnorm Child Psychol, 36:1069–1081. 2008.

17.

Kendall

, Chu

, Pimental

, Choudhury

. Treating anxiety disorders in youth. Child and Adolescent Therapy. Kendall

P.C.

New York: Guilford, 2000; 235–290.

18.

Kendall

, Flannery–Schroeder

. Methodological issues in treatment research for anxiety disorders in youth. J Abnorm Child Psychol, 26:27–38. 1998.

19.

Kim

, Leventhal

, Koh

, Fombonne

, Laska

, Lim

, Cheon

, Kim

, Lee

, Song

, Grinker

. Prevalence of autism spectrum disorders in a total population sample. Am J Psychiatry, 168:904–912. 2011.

20.

Kogan

, Blumberg

, Schieve

, Boyle

, Perrin

, Ghandour

, Singh

, Strickland

, Trevathan

, van Dyck

. Prevalence of parent-reported diagnosis of autism spectrum disorder among children in the US, 2007. Pediatrics, 124:1395–1403. 2009.

21.

Kurdek

, Berg

. Children's Beliefs About Parental Divorce Scale: Psychometric characteristics and concurrent validity. J Consult Clin Psychol, 55:712–718. 1987.

22.

Leckman

, Sholomskas

, Thompson

, Belanger

, Weissman

. Best estimate of lifetime psychiatric diagnosis: A methodological study. Arch Gen Psychiatry, 39:879–883. 1982.

23.

Leyfer

, Folstein

, Bacalman

, Davis

, Dinh

, Morgan

, Tager–Flusberg

, Lainhart

. Comorbid psychiatric disorders in children with autism: Interview development and rates of disorders. J Autism Dev Disord, 36:849–861. 2006.

24.

Lord

, Pickles

, McLennan

, Rutter

, Bregman

, Folstein

, Fombonne

, Leboyer

, Minshew

. Diagnosing autism: Analyses of data from the Autism Diagnostic Interview. J Autism Dev Disord, 27:501–517. 1997.

25.

Lord

, Rutter

, DiLavore

, Risi

. Autism Diagnostic Observation Schedule. Los Angeles: Western Psychological Services, 1999.

26.

Lord

, Rutter

, Le Couteur

. Autism diagnostic interview–revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord, 24:659–685. 1994.

27.

Mannuzza

, Fyer

, Martin

, Gallops

, Endicott

, Gorman

, Liebowitz

, Klein

. Reliability of anxiety assessment. I. Diagnostic agreement. Arch Gen Psychiatry, 46:1093–1101. 1989.

28.

Mildenberger

, Sitter

, Noterdaeme

, Amorosa

. The use of the ADI-R as a diagnostic tool in the differential diagnosis of children with infantile autism and children with a receptive language disorder. Eur Child Adolesc Psychiatry, 10:248–255. 2001.

29.

Muris

, Steerneman

, Merckelbach

, Holdrinet

, Meesters

. Comorbid anxiety symptoms in children with pervasive developmental disorders. J Anxiety Disord, 12:387–393. 1998.

30.

Orvaschel

, Weissman

, Padian

, Lowe

. Assessing psychopathology in children of psychiatrically disturbed parents: A pilot study. J Am Acad Child Psychiatry, 20:112–122. 1981.

31.

Rapee

, Barrett

, Dadds

, Evans

. Reliability of the DSM-III-R childhood anxiety disorders using structured interview: Interrater and parent–child agreement. J Am Acad Child Adolesc Psychiatry, 33:984–992. 1994.

32.

RUPP:. The Pediatric Anxiety Rating Scale: Development and psychometric properties. J Am Acad Child Adolesc Psychiatry, 41:1061–1069. 2002.

33.

Silverman

, Albano

. The Anxiety Disorders Interview Schedule for DSM-IV-Child and Parent Versions. San Antonio, TX: Graywinds Publications, 1996.

34.

Silverman

, Eisen

. Age differences in the reliability of parent and child reports of child anxious symptomatology using a structured interview. J Am Acad Child Adolesc Psychiatry, 31:117–124. 1992.

35.

Storch

, Merlo

, Larson

, Geffken

, Lehmkuhl

, Jacob

, Murphy

, Goodman

. Impact of comorbidity on cognitive-behavioral therapy response in pediatric obsessive-compulsive disorder. J Am Acad Child Adolesc Psychiatry, 47:583–592. 2008.

36.

Sukhodolsky

, Scahill

, Gadow

, Arnold

, Aman

, McDougle

, McCracken

, Tierney

, Williams White

, Lecavalier

, Vitiello

. Parent-rated anxiety symptoms in children with pervasive developmental disorders: Frequency and association with core autism symptoms and cognitive functioning. J Abnorm Child Psychol, 36:117–128. 2008.

37.

Wood

, Gadow

. Exploring the nature and function of anxiety in youth with autism spectrum disorders. Clin Psychol: Res Pract, 17:281–292. 2010.

38.

Wood

, Piacentini

, Bergman

, McCracken

, Barrios

. Concurrent validity of the anxiety disorders section of the Anxiety Disorders Interview Schedule for DSM-IV: Child and parent versions. J Clin Child Adolesc Psychol, 31:335–342. 2002.