Demographic and clinical correlates of autism symptom domains and autism spectrum diagnosis

Abstract

Demographic and clinical factors may influence assessment of autism symptoms. This study evaluated these correlates and also examined whether social communication and interaction and restricted/repetitive behavior provided unique prediction of autism spectrum disorder diagnosis. We analyzed data from 7352 siblings included in the Interactive Autism Network registry. Social communication and interaction and restricted/repetitive behavior symptoms were obtained using caregiver-reports on the Social Responsiveness Scale. Demographic and clinical correlates were covariates in regression models predicting social communication and interaction and restricted/repetitive behavior symptoms. Logistic regression and receiver operating characteristic curve analyses evaluated the incremental validity of social communication and interaction and restricted/repetitive behavior domains over and above global autism symptoms. Autism spectrum disorder diagnosis was the strongest correlate of caregiver-reported social communication and interaction and restricted/repetitive behavior symptoms. The presence of comorbid diagnoses also increased symptom levels. Social communication and interaction and restricted/repetitive behavior symptoms provided significant, but modest, incremental validity in predicting diagnosis beyond global autism symptoms. These findings suggest that autism spectrum disorder diagnosis is by far the largest determinant of quantitatively measured autism symptoms. Externalizing (attention deficit hyperactivity disorder) and internalizing (anxiety) behavior, low cognitive ability, and demographic factors may confound caregiver-report of autism symptoms, potentially necessitating a continuous norming approach to the revision of symptom measures. Social communication and interaction and restricted/repetitive behavior symptoms may provide incremental validity in the diagnosis of autism spectrum disorder.

Keywords

autism spectrum disorder autism symptoms diagnosis prediction

Introduction

Autism symptoms vary widely both across individuals meeting categorical diagnosis and the remaining population (Constantino and Todd, 2003, 2005; Piven et al., 1997). Accurately quantifying autism symptom levels is crucial for advancing assessment, autism spectrum disorder (ASD) diagnosis, and research. Unfortunately, there is limited consensus on how best to measure autism symptoms, with several different approaches and instruments available for clinical and research settings (Lord et al., 2002; Mayes et al., 2009). Caregiver-report remains an important part of most assessment strategies (Mildenberger et al., 2001; Risi et al., 2006). This is driven, in part, by difficulties with observing the full assortment and range of autism symptoms in an office visit. For example, qualitative difficulties in reciprocal interaction with peers are often determined through caregiver-report rather than direct observation. Other symptoms can be inconsistent over time and situation (e.g. difficulties with transitions/schedule change, hand flapping when very excited), and therefore, may not be observed in a brief encounter. Additionally, questions remain concerning how to optimally quantify caregiver symptom ratings. These questions include, but are not limited to, whether to adjust symptom reports for relevant demographic and clinical factors and whether to evaluate specific symptom domains or global symptom severity (Constantino and Gruber, 2005). This study examined these issues by analyzing the demographic (age, sex, race/ethnicity, and birth order) and clinical (attention deficit hyperactivity disorder (ADHD), any anxiety disorder, and intellectual disability) correlates of caregiver-reported autism symptoms and the relationship between symptoms and ASD diagnosis.

Several motivations exist for evaluating the correlates of autism symptoms. An important theoretical motivation is to determine whether autism symptom domains are truly distinct. If distinct patterns of correlates across symptom domains are identified, this may imply the need for separate consideration of these domains rather than only global symptom severity. Latent variable modeling studies have identified two core symptom domains related to social communication and interaction (SCI) and restricted/repetitive behavior (RRB) (Frazier et al., 2008, 2012; Snow et al., 2009). Other minor age- or measure-specific domains, such as imagination/play, have also been identified but are poorly replicated (Frazier et al., 2008). Earlier studies suggested a single global autism severity factor, often strongly associated with social communication behavior (Constantino et al., 2004). These studies, however, included fewer RRB symptoms or mixed autism symptoms with adaptive behavior (Szatmari et al., 2002), thereby pushing findings toward a single factor. Furthermore, proposed Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) criteria contain separate SCI and RRB symptom criteria (American Psychiatric Association, 2011), consistent with the notion that these are distinct, but correlated, domains. This study evaluates the issue of domain-specific versus global symptom severity by separately examining demographic and clinical correlates of SCI and RRB symptom ratings and by assessing whether distinct domains yield incremental validity over global autism severity in the prediction of ASD diagnosis.

A practical reason for examining demographic and clinical correlates is that they may be important factors to consider when developing or revising measures. Age, sex, cognitive functioning, and comorbid behavior problems have all been found to influence autism symptom reports (Hus et al., 2013; Mayes and Calhoun, 2010), although the specific patterns have been sample- and measure-dependent. For example, results from initial studies of the Social Responsiveness Scale (SRS) suggested that it may be relatively independent of age and intelligence quotient (IQ), but not sex (Constantino and Gruber, 2005), with males showing higher symptom levels. Individuals with comorbid psychopathology have also been found to show higher symptom levels (Hus et al., 2013). The SRS contains items beyond social behavior, including items tapping RRB content. This provided an opportunity to separately evaluate the effects of demographic and clinical factors on SCI and RRB domains. Furthermore, this study extends recent work examining both ASD-affected individuals and their siblings (Hus et al., 2013) by evaluating the influence of demographic and clinical factors before and after accounting for ASD diagnosis. Some factors, particularly internalizing and externalizing behavior problems, may be confounded with ASD diagnosis. If these factors are significant after accounting for ASD diagnosis, adjustment for these factors—either by direct adjustment or through the creation of separate norms—may be needed to enhance the specificity of autism symptom measurement.

This study

The first aim of this study was to examine demographic and clinical correlates of SCI and RRB symptoms before and after accounting for ASD diagnosis. Evaluating the influence of demographic and clinical correlates after accounting for ASD diagnosis is crucial because it may be that the relationship between these factors and autism symptoms is largely driven by the presence of categorical ASD. In other words, ASD may “travel with” or induce a range of disability. If true, controlling for these factors would actually remove variance in the measure associated with autism phenotype—an undesirable effect for a measure of autism symptoms. Thus, the chosen analytic approach represents the first comprehensive attempt to apportion overlap between categorical ASD and other factors into (a) overlap that is a result of the presence versus absence of ASD diagnosis and (b) extraneous influences of unrelated psychopathological traits on symptom measurement.

Not surprisingly, ASD diagnosis was expected to be the primary driver of SCI and RRB symptoms. However, we also hypothesized that age, sex, comorbid ADHD symptoms, anxiety disorder diagnosis, and the presence of intellectual disability would be significant predictors of SCI and RRB symptoms, even after accounting for ASD diagnosis. The second aim of this study was to determine whether caregiver-reported SCI and RRB symptoms, considered separately, provided incremental validity in the prediction of ASD diagnosis. Caregiver-reported SCI and RRB symptoms were expected to make significant unique contributions to the prediction of clinical ASD diagnosis beyond global symptom severity.

Method

Participants

Data were obtained from the Interactive Autism Network (IAN; www.ianproject.org), an internet-based registry for families with one or more ASD-affected children (IAN Data Export ID: IAN_DATA_2010-07-06). Families were eligible for enrollment in IAN if the parent or legal guardian who provided information was English speaking, the family lived in the United States, and their child was diagnosed with an ASD by a professional. To be included in this study, caregivers must have reported ASD symptom data for at least one ASD-affected child. A subset of IAN caregivers (usually the mother) completed the SRS, as this measure was added to the database approximately 9 months after the initial launch of IAN. The present analyses are based on the SRS subset of IAN, which includes all registered families for whom at least one child (ASD-affected or non-ASD sibling) had completed SRS data.

The SRS subsample of IAN has previously been shown to be fairly representative of the larger IAN registry (Kalb et al., 2010). Relative to the larger IAN registry, the SRS subsample is slightly older, due to the fact that the instrument is completed only for individuals aged 4 and older, and has a higher proportion of White/non-Hispanic individuals. Additionally, the SRS sample includes fewer nonverbal individuals, as some SRS items are geared toward verbal individuals. Caregivers tend to leave these items incomplete or simply do not complete the instrument. Previous examinations of these subsample differences indicated that the magnitude of the effects ranged from very small to small (all standardized mean differences < 0.20) (Frazier et al., 2012). IAN—while an internet registry—is fairly representative of the larger population of US families affected by ASD, with the primary exception of over-sampling non-Hispanic Whites and higher socioeconomic status (SES) demographics.

Procedures

Informed consent was obtained from parents/guardians for all participants prior to entry into the IAN data collection. The procedures of IAN and this study were reviewed and approved by the institutional review boards of Kennedy Krieger Institute and Cleveland Clinic, respectively. Demographics, comorbid clinical symptoms and diagnoses, and autism symptoms were provided by caregivers as part of participation in IAN.

ASD diagnosis

The vast majority of clinical ASD diagnoses (93%) were provided by a doctoral-level professional or team. A substantial proportion of youth (74.9%) were diagnosed using the Autism Diagnostic Interview–Revised, the Autism Diagnostic Observation Schedule, or both. Of these, 98.7% scored in the ASD-affected range for one or both instruments. A recent study of youth from the IAN registry by Lee et al. (2010) supported the validity of clinical ASD diagnoses. In this study, randomly selected verbal youth with a score >12 on the Social Communication Questionnaire were evaluated using the Autism Diagnostic Interview–Revised and by an expert clinician’s observation. All but a handful of youth (2%) were confirmed to have a Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision; DSM-IV-TR) clinical diagnosis of ASD (autistic disorder, pervasive developmental disorder–not otherwise specified (PDD-NOS), or Asperger’s disorder). Thus, IAN caregiver-reported clinical ASD diagnoses serve as an excellent proxy for diagnoses based on semistructured clinical interview and/or observation. ASD diagnosis was coded by collapsing all DSM-IV-TR autism spectrum diagnoses into a single category (ASD vs non-ASD) following Centers for Disease Control and Prevention (CDC, 2012) epidemiologic surveillance protocols.

Demographic and clinical factors were also reported for siblings without a clinical ASD diagnosis. Many of these siblings have caregiver-reported psychiatric or developmental diagnoses (Constantino et al., 2010). For this reason, we refer to these siblings as non-ASD siblings.

Measures

SRS

Autism symptom data were obtained using the SRS (Constantino and Gruber, 2005). The SRS is a 65-item, ordinally scaled (1 = “not true” to 4 = “almost always true”) quantitative assessment of the severity of autism traits. It is one of the most frequently used quantitative measures of autism symptoms, showing strong measurement properties in healthy and autism-affected samples, is one of the few measures with separate norms by sex, and is frequently incorporated in research to evaluate global autism trait severity (Constantino and Gruber, 2005). The SRS total raw score is typically reported in research, and the sex-adjusted T-score is available for clinical interpretation. In addition to the total score, SRS items are divided conceptually into five subscales: Social Cognition, Social Awareness, Social Communication, Social Motivation, and Autistic Mannerisms. The first four subscales tap SCI symptoms, while the last subscale includes symptoms consistent with the RRB domain. For this study, raw scores for SRS social subscales were summed to create an aggregate score for the SCI symptom domain. The raw score for the subscale autistic mannerisms was used to evaluate the RRB domain. This scoring was based on a recent latent variable modeling study of the SRS (Frazier et al., 2012). Internal consistency reliability was excellent for the SRS total raw score (α = 0.99), SCI domain (α = 0.98), and RRB domain (α = 0.94).

Demographic and clinical covariates

Demographics included age, sex, race/ethnicity (coded White non-Hispanic, other race/ethnicity), and birth order. Clinical covariates included caregiver-reported clinical diagnoses (any ADHD, anxiety disorder, or intellectual disability). These covariates were derived from specific questions included in the IAN questionnaire completed by caregivers. Questions included the following: “Has (child name) ever been diagnosed with or received treatment for ADHD or attention deficit disorder (ADD)?” “Has (child name) ever been diagnosed with or received treatment for an anxiety disorder?” and “Has (child name) ever been diagnosed with mental retardation?” Caregivers responded “yes” or “no” to each question. In the case of ADHD, a comorbid diagnosis is typically not permitted by DSM-IV-TR but is often given in practice by clinicians who wish to describe significant comorbid ADHD symptoms. For ease of communication and to maintain consistency with other caregiver-reported diagnoses, we refer to this ADHD symptom pattern as a dichotomously coded comorbid ADHD diagnosis (ADHD vs non-ADHD). Caregiver-reports of any anxiety disorder or intellectual disability diagnosis were also coded as single categories (anxiety disorder vs no anxiety or intellectual disability vs no intellectual disability). These three caregiver-reported clinical diagnoses should be viewed only as proxies for actual clinical diagnoses or symptom patterns but serve the useful purpose of estimating the effects of externalizing (ADHD) and internalizing (anxiety) behavior or lower cognitive level (intellectual disability) on autism symptoms. The estimates obtained for these correlates should be viewed as lower bound estimates of the true relationship with autism symptoms because caregiver-reported clinical diagnoses may be less reliable than those based on semistructured interviews or observations.

Analytic plan

Independent sample t-tests and Pearson’s chi-square were used to compare sample characteristics across ASD-affected and non-ASD siblings. Nonparametric equivalents produced the same pattern of results, and therefore, parametric statistics are reported to facilitate effect size calculation for future meta-analyses.

Missing data were anticipated for several of the variables included in this study. For this reason, missing data analyses focused on the pattern and quantity of missing data. The empirically recommended approach of generating multiple imputations was used (Schafer and Graham, 2002). To account for missing data, three imputed data files were created, and analyses were computed on these files and the original data set.

Correlates of autism symptom domains (Aim 1)

To examine aim 1, four mixed-effects regression models were estimated. The first two models included age, sex, race/ethnicity, birth order, any ADHD diagnosis, any anxiety disorder diagnosis, and a diagnosis of intellectual disability. SCI and RRB symptoms served as dependent variables in separate analyses. The second two models were similar to above except that clinical ASD diagnosis (ASD vs non-ASD) was added as a covariate. Follow-up regression analyses examined interactions between clinical ASD diagnosis and covariates to examine whether the covariate effects differed across ASD and non-ASD groups.

Mixed-effects regressions modeled individual- and family-level variables while accounting for the nested nature of the sample, with many families contributing multiple siblings. Model fit was considered by iteratively examining alternative covariance structures (Bryk and Raudenbush, 1992; Kreft, 1995; Peugh and Enders, 2005). Final models were estimated using restricted maximum-likelihood estimation and were presented based on fixed effects and scaled identity covariance structures, which fit comparably and yielded a highly similar pattern of results to other more complex structures (autoregressive, diagonal, etc.). Significant covariates are described as predictors of autism symptoms only in the statistical sense because all information was cross-sectional. Follow-up regression analyses examined whether significant covariate main effects were modified by ASD diagnosis. In these post hoc analyses, only the individual covariate, ASD diagnosis, and their interaction were included.

Incremental validity of SCI and RRB symptoms (Aim 2)

To examine aim 2, we first computed receiver operating characteristic curves for caregiver-reported SRS total scores and SCI and RRB symptoms. Clinical ASD diagnosis was the state (outcome) variable for each curve. Predictive efficiency was determined using areas under the curve (AUCs). Second, we computed a hierarchical logistic regression model with SRS total scores entered in step 1 and either SCI or RRB symptoms entered in step 2. This evaluated whether specific domains may improve prediction beyond global autism symptom severity. Third, we computed two hierarchical logistic regression models with SCI and RRB symptoms alternating between step 1 and step 2. These models directly test whether separate autism symptom domains provide incremental validity in the prediction of ASD diagnosis. It is important to note that these analyses do not speak to the absolute efficiency of SRS-derived measures in predicting ASD diagnosis because caregivers provided symptoms with knowledge of the clinical diagnosis. Instead, these analyses are meant to examine the relative and incremental validity of SRS-derived measures.

Sensitivity analyses

Follow-up sensitivity analyses examined whether the pattern of results differed when using empirically derived diagnostic classifications (Frazier et al., 2012) as an alternative diagnostic criterion to caregiver-reported ASD diagnosis. These empirically derived diagnostic classifications were computed based on the results of factor mixture model analyses of SRS item packets in Mplus. Factor mixture models examine whether both latent categories and latent dimensions are needed to most accurately characterize covariances between symptoms. Empirical classifications are helpful for evaluating whether results from primary analyses were due to caregiver knowledge of the ASD diagnosis when reporting symptoms.

All analyses were computed in IBM SPSS version 19. Statistical significance was determined using p < 0.001. This conservative alpha level was used to reduce potential for Type 1 error because the magnitude of effects for significant predictors is most relevant. Effect magnitude was quantified for all variables using the familiar correlation (r) metric. To quantify magnitude for each covariate, F-statistics were converted to r. These r values represent effect magnitude of each variable after accounting for all other variables in the model, analogous to partial r coefficients. For this study, partial correlation (r_ab.c) values less than 0.04 were described as very small, 0.04–0.10 small, 0.10–0.30 small to medium, and 0.30–0.50 medium to large, although it should be recognized that the latter three conventions are typically used for bivariate correlations and may be conservative for partial correlations (Cohen, 1987).

Results

Sample description

Table 1 presents demographic and clinical characteristics for ASD-affected and non-ASD siblings. ASD-affected siblings were younger and more likely to be early in the birth order. These differences may reflect the decreased likelihood of having additional children after having a child with ASD (stoppage). ASD-affected siblings were also more likely to be male, reflecting the male predominance in ASD; more likely to have a comorbid ADHD diagnosis, an anxiety disorder, or intellectual disability diagnosis; and had substantially higher SRS scores. Intellectual disability is clearly underreported in this sample relative to clinical expectations in individuals with ASD, reinforcing the notion that caregiver-reports of comorbidities are only proxies for actual clinical observations.

Table 1.

Demographic and clinical characteristics of ASD-affected and non-ASD siblings.

	Non-ASD	ASD	t/χ² (p)
N	2848	4504
Age (SD)	9.5 (4)	8.8 (4)	7.98 (<0.001)
Female (%)	53.1	17.3	1040.55 (<0.001)
Birth order (SD)	1.7 (1)	1.8 (1)	6.17 (<0.001)
White non-Hispanic (%)	86.9	86.2	0.76 (0.384)
Any ADHD (%)	12.3	32.4	373.47 (<0.001)
Any anxiety (%)	5.8	20.8	304.18 (<0.001)
Intellectual disability (%)	0.5	8.8	220.96 (<0.001)
SRS total raw (SD)	24.2 (24)	105.9 (30)	122.67 (<0.001)
SRS total T-score (SD)	46.6 (12)	86.1 (15)	114.80 (<0.001)

SRS: Social Responsiveness Scale; SD: standard deviation; ADHD: attention deficit hyperactivity disorder; ASD: autism spectrum disorder.

SRS total raw score ranges from 0 to 195. Any ADHD, any anxiety disorder, and intellectual disability are caregiver-reported clinical diagnoses.

Inspection of the means and standard deviations for SRS-derived scores reveals a positively skewed but wide distribution of autism symptoms in non-ASD siblings with central tendency only slightly below the population normative mean (T = 46.6 vs T = 50). ASD-affected siblings also showed a wide but elevated distribution of symptoms. These wide symptom ranges are ideal for examining the effects of demographic or clinical correlates on autism symptom domains.

Missing data

Small proportions of data were missing for four variables: comorbid ADHD diagnosis (2.3%), any anxiety disorder (2.3%), intellectual disability (2.4%), and birth order (2.7%). This resulted in a very small percent of overall missing data across the 11 primary variables included in the analyses (0.9%). Correlations between missingness and imputed values were very small and nonsignificant (all pooled r < 0.03, p > 0.05), indicating that data approximate the missing at random assumption. Analyses across three imputed data sets were highly similar to each other and to analyses of the original data—there were no changes in the significance of parameter estimates across the original or imputed data, and all parameter estimates were stable (within ±0.02). These results suggest that the effects of missing data are not likely to be substantial. For simplicity, results are presented for the original nonimputed data.

Power

Power to detect significant covariate effects in aim 1 was computed assuming N = 7352, alpha level of p < 0.001 (two-tailed), and a range of covariate effects—partial correlations r_ab.c = 0.03–0.10. Power to detect significant covariate effects was very good (Power > 0.84) for effect sizes r_ab.c = 0.05 and larger. Power to detect very small covariate effects of r_ab.c = 0.04 was modest (Power = 0.55). Low power for very small covariate effects was less problematic because these effects may not be of conceptual or practical significance. Power for aim 2 was expected to be comparable to aim 1 given the similar nature of models.

Correlates of SCI and RRB symptoms

Without ASD diagnosis

Initial mixed-effects regression analyses examined the influence of demographic and clinical correlates on SCI and RRB without including ASD diagnosis. Age, sex, comorbid ADHD diagnosis, any anxiety disorder diagnosis, and intellectual disability were significant unique predictors of SCI and RRB scores (smallest F(1, 7134) = 252.17, p < 0.001). Birth order was a marginally significant predictor of RRB scores (F(1, 7134) = 10.44, p = 0.001) but did not significantly predict SCI scores (F(1, 7134) = 2.16, p = 0.142). Race/ethnicity was not a significant predictor of SCI or RRB scores (largest F(1, 7134) = 3.14, p = 0.076).

With ASD diagnosis

The next set of mixed-effects regression analyses examined correlates after including ASD diagnosis as an additional covariate. ASD diagnosis was the largest predictor of both SCI (total R² = 0.69, p < 0.00001; ASD diagnosis unique predictive component R² = 0.45; F(1, 7133) = 10074.95, p < 0.001) and RRB scores (total R² = 0.66, p < 0.00001; ASD diagnosis unique predictive component R² = 0.41; F(1, 7133) = 8780.14, p < 0.001), with very large effect sizes for both domains (SCI: r = 0.67; RRB: r = 0.64). After including ASD diagnosis, demographic and other clinical factors tended to show much smaller influences on SCI (unique R² = 0.02) and RRB symptoms (unique R² = 0.02). The largest remaining effects for both domains were observed for any ADHD diagnosis, any anxiety disorder diagnosis, or intellectual disability (r = 0.11–0.13; all p < 0.001; SCI largest F(1, 7133) = 114.40, p < 0.001 was for intellectual disability; RRB largest F(1, 7133) = 106.96, p < 0.001 was for any anxiety disorder diagnosis). Age and sex showed slightly different patterns across SCI and RRB symptoms. For SCI, but not RRB, age trended toward statistical significance (r = 0.04, F(1, 7133) = 9.91, p = 0.002). For RRB, but not SCI, sex was a significant unique predictor (r = 0.06, F(1, 7133) = 32.49, p < 0.001). Race/ethnicity was not a significant predictor of SCI or RRB after considering ASD diagnosis (largest F(1, 7133) = 3.00, p = 0.083). Intercepts and regression parameters are presented in Supplementary Table 1.

Effect magnitude

Figures 1 and 2 present the magnitude (r) of unique predictive effects on SCI and RRB scores, with and without simultaneously covarying for ASD diagnosis. The largest effects were seen for age, sex, any ADHD diagnosis, any anxiety disorder, and intellectual disability before and after covarying for ASD diagnosis. Sex showed a very large relationship with SCI and RRB scores before accounting for ASD diagnosis, but this relationship was considerably smaller after including ASD diagnosis. A dramatic decrease is easily observed in the magnitude of most covariate effects after including ASD diagnosis.

Figure 1.

Magnitude (r) of unique predictive effects for SCI, with and without ASD diagnosis as a covariate.

Figure 2.

Magnitude (r) of unique predictive effects for RRB, with and without ASD diagnosis as a covariate.

Interactions with ASD diagnosis

Follow-up regression analyses examined interactions between demographic/clinical covariates and ASD diagnosis. Table 2 presents SCI and RRB raw scores at representative levels of each covariate, separately for ASD and non-ASD youth. SCI scores decreased across age groups in both ASD-affected and non-ASD youth (interaction F(4, 7126) = 0.47, p = 0.759). RRB scores remained fairly stable across age groups in both ASD-affected and non-ASD youth (interaction F(4, 7126) = 0.57, p = 0.688). The presence of caregiver-reported ADHD, anxiety, and intellectual disability resulted in substantial increases in the level and variability of SCI and RRB symptoms in the non-ASD youth; increases were smaller but still significant and sizeable in ASD-affected youth (smallest interaction F(1, 7132) = 12.13, p < 0.001 for intellectual disability with SCI scores). The only exception was a nonsignificant trend for the interaction of ASD diagnosis and intellectual disability on RRB scores (F(1, 7132) = 2.76, p = 0.097). No significant interactions were observed for sex, race/ethnicity, or birth order (largest F(2, 7130) = 4.88, p = 0.008 for birth order with SCI scores). Intercepts and regression parameters are presented in Supplementary Table 1.

Table 2.

SCI and RRB raw scores at representative levels of demographic and clinical covariates, separately for ASD and non-ASD siblings.

	SCI		RRB
	Non-ASD	ASD	Non-ASD	ASD
	M (SD)	M (SD)	M (SD)	M (SD)
Age, years
4–6	33.03 (32.91)	94.81 (38.21)	6.42 (9.02)	21.86 (10.47)
7–9	32.67 (29.81)	92.97 (30.38)	6.65 (8.17)	22.33 (8.33)
10–13	32.04 (28.08)	93.94 (26.98)	6.49 (7.70)	22.48 (7.40)
13–15	31.68 (26.29)	92.49 (25.25)	6.31 (7.21)	22.25 (6.92)
16–18	31.18 (24.45)	91.15 (23.24)	5.85 (6.70)	21.73 (6.37)
Sex
Female	31.28 (36.70)	93.82 (26.80)	6.13 (10.07)	21.56 (7.35)
Male	33.68 (35.18)	94.13 (42.79)	6.91 (9.65)	22.68 (11.74)
Race/ethnicity
White/non-Hispanic	32.32 (40.95)	93.85 (40.95)	6.61 (11.23)	22.41 (11.23)
Other	34.13 (26.34)	92.70 (25.96)	6.94 (7.22)	21.64 (7.12)
Birth order
1	32.15 (34.09)	91.48 (37.15)	6.47 (9.36)	21.89 (10.20)
2	32.30 (32.92)	95.09 (31.28)	6.43 (9.03)	22.34 (8.59)
3+	34.97 (27.51)	97.53 (26.72)	6.96 (7.55)	22.70 (7.34)
ADHD symptoms
Not reported	28.13 (42.66)	91.84 (41.08)	5.53 (11.75)	21.62 (11.32)
Reported	45.83 (25.52)	94.76 (30.85)	9.02 (7.03)	22.60 (8.50)
Anxiety diagnosis
Not reported	27.94 (41.58)	90.21 (41.01)	5.26 (11.42)	21.14 (11.26)
Reported	46.52 (23.50)	95.93 (28.17)	9.71 (6.45)	22.95 (7.74)
Intellectual disability
Not reported	26.24 (33.92)	87.57 (34.13)	4.88 (9.31)	20.56 (9.37)
Reported	58.49 (22.60)	99.13 (23.09)	10.73 (6.20)	23.70 (6.34)

SCI: social communication and interaction; RRB: restricted/repetitive behavior; SD: standard deviation; ADHD: attention deficit hyperactivity disorder; ASD: autism spectrum disorder.

SCI total raw scores range from 0 to 159 and RRB total raw scores range from 0 to 36. Age: 4–6 (n = 2806), 7–9 (n = 1860), 10–12 (n = 1293), 13–15 (n = 950), 16–18 (n = 443) years; male: n = 5060, Female: n = 2292; White non-Hispanic: n = 6357, other race/ethnicity: n = 995; Birth order: 1 (n = 3429), 2 (n = 2460), 3+ (n = 1265); ADHD diagnosis: reported (n = 1767), not reported (n = 5417); anxiety diagnosis: reported (n = 1073), not reported (n = 6108); intellectual disability: reported (n = 400), not reported (n = 6777).

Incremental prediction of ASD diagnosis

Figure 3 presents receiver operating characteristic curves for SRS total, SCI, and RRB scores in predicting clinical ASD diagnosis. AUC values were quite strong, all falling above 0.95, which is considered very strong discrimination of ASD versus non-ASD siblings. The AUCs were also highly similar for SRS total, SCI, and RRB scores (all AUC 95% CIs overlap; SRS total raw score = 0.969, 95% confidence interval (CI) = 0.964–0.973; SCI raw score = 0.966, 95% CI = 0.961–0.970; RRB raw score = 0.967, 95% CI = 0.963–0.971). Correlations between SCI and RRB scores were generally high but slightly lower in ASD-affected siblings relative to non-ASD siblings (ASD: r = 0.79, non-ASD: r = 0.86). These high but imperfect relationships indicate that the SCI and RRB domains may show incremental validity in predicting ASD diagnosis. Consistent with this hypothesis, SCI and RRB scores provided small, but highly significant, improvements in the prediction of ASD diagnosis beyond SRS total scores (Step 1—SRS total scores entered: R = 0.902, R² = 0.814, Wald χ²(1) = 6733.70, p < 0.0001; Step 2—SCI/RRB scores added: smallest ΔR = 0.071, ΔR² = 0.005, Wald χ²(1) = 68.15, p < 0.0001). Additionally, SCI and RRB scores displayed incremental validity when considered iteratively in the prediction of ASD diagnosis (smallest ΔR = 0.12, ΔR² = 0.015, Wald χ²(1) = 206.95, p < 0.0001; final model constant = −4.15, standard error (SE) = 0.110; SCI B = 0.052, SE = 0.003; RRB B = 0.183, SE = 0.013).

Figure 3.

Receiver operating characteristic curves for SRS total raw, SCI raw, and RRB raw scores show close overlap in the prediction of ASD diagnosis.

Sensitivity analyses

Empirically derived diagnostic classifications (ASD and non-ASD) were highly correlated, but not redundant, with caregiver-reported clinical ASD diagnosis (r = 0.90). Using empirical classifications did not substantively change the pattern (magnitude or significance) of findings. As with caregiver-reported clinical diagnoses, empirical classifications result in similar AUC values for SRS total, SCI, and RRB scores (all AUC 95% CIs overlap; SRS total raw score = 0.977, 95% CI = 0.974–0.981; SCI raw score = 0.975, 95% CI = 0.972–0.979; RRB raw score = 0.974, 95% CI = 0.970–0.977). Both SCI and RRB scores provided small, but highly significant, improvements in the prediction of empirical classifications beyond SRS total scores (smallest ΔR = 0.06, ΔR² = 0.004, χ²(1) = 53.09, p < 0.0001). Additionally, SCI and RRB scores provided incremental validity when considered iteratively in the prediction of empirical classifications (smallest ΔR = 0.11, ΔR² = 0.013, χ²(1) = 174.98, p < 0.0001).

Discussion

ASD diagnosis was the strongest individual correlate of SCI and RRB symptoms, uniquely accounting for the majority of variance in both symptom domains. This observation indicates that in this sample, the SRS showed good discrimination between typical and pathological levels of SCI and RRB symptoms. Mean scores of ASD and non-ASD youth were consistent with mean scores in previous SRS studies (Constantino and Gruber, 2005). This implies that caregiver knowledge of ASD diagnosis did not substantially alter the spread of scores between ASD and non-ASD siblings and, therefore, may not account for the present findings. Additionally, the consistency of findings using clinical ASD diagnoses and empirical classifications further confirms previous observations that categorical ASD diagnosis is strongly associated with quantitative autism symptom measurement (Constantino et al., 2003a).

The next strongest correlates of SCI and RRB symptoms were the presence of any ADHD, anxiety disorder, or intellectual disability. Youth with any of these caregiver-reported diagnoses had meaningfully higher SCI (5–12 raw score points higher on average) and RRB symptoms (2–3 raw score points higher) after accounting for ASD diagnosis. In particular, the presence of ADHD, anxiety, or intellectual disability in non-ASD youth increased SCI and RRB symptom ratings. Symptom increases in non-ASD youth increased overlap between ASD and non-ASD cases, making discrimination more challenging. At a theoretical level, this overlap likely reflects genetic relationships between ASD and other development problems (Constantino et al., 2003b; Lichtenstein et al., 2010; Lundstrom et al., 2011). In this sense, overlap may be unavoidable, reflecting the true nature of the genetic relationships between different forms of developmental psychopathology, rather than a limitation of the instrument. At a practical level, overlap raises the problem of clinically discriminating categorical ASD from non-ASD. Because internalizing and externalizing behaviors, as well as global ability level, may influence caregiver-reports of autism symptoms, clinicians collecting parent-report measures in youth with comorbid neuropsychiatric disorders or symptom patterns would be wise to consider these influences in their score interpretation. Future revisions of the SRS should consider examining these and possibly other factors as norms are developed. Identifying and removing items that are most sensitive to comorbid symptom patterns and do not represent core ASD symptoms have the potential to substantially reduce score overlap. Overlap also suggests the practical implication that proposed DSM-5 diagnostic criteria should consider permitting comorbid diagnoses in individuals who show high levels of both autism and co-occurring developmental psychopathology.

Age, sex, and birth order effects—while weaker and more variable across SCI and RRB domains—also influenced symptoms. Younger, male, and later born siblings tended to have higher symptom levels. The pattern is consistent with previous longitudinal data suggesting improvements in ASD symptoms with development (Charman et al., 2005; Moss et al., 2008) and literature suggesting higher levels of autism traits in the male population (Constantino and Todd, 2003). The observation of later born siblings having greater symptoms may reflect greater sensitivity to atypical development in caregivers with larger family sizes, where the majority of children do not have ASD. Results for sex were consistent with literature suggesting higher levels of autism traits in the male population (Constantino and Todd, 2003). Interestingly, there were no sex differences for RRB among ASD cases. Previous studies suggest that females with ASD show reduced RRB symptoms relative to males with ASD (Lord et al., 1982; Mandy et al., 2012a), although differences appear to be less prominent in young children (Carter et al., 2007). The lack of differences for RRB in the present sample may be due to under-sampling of high-functioning females with ASD rather than true equivalence between sexes (Frazier et al., 2012). Future replication studies with careful sampling of high-functioning females are needed to ensure the stability and generalizability of age, sex, and birth order effects. If replicated, revisions of the SRS would benefit from separate or continuous norming approaches that account for these factors (Hus et al., 2013). These revisions may also consider generating indices that specifically evaluate SCI and RRB symptoms and can be linked to proposed DSM-5 criteria.

This study also presented the unique opportunity to examine the impact of clinical and demographic factors on autism symptom measurement before and after accounting for categorical ASD diagnosis. Recent work by Hus et al. (2013) identified substantial influences of behavior problems and cognitive impairment on SRS scores before accounting for categorical ASD diagnosis. One interpretation of these relationships is that external factors “cause” higher autism symptom levels. The present results suggest an alternative explanation—that behavior problems and cognitive disability increase caregiver-reports of autism symptoms in unaffected siblings and that the presence of a categorical ASD diagnosis induces a range of behavioral disability. The implication is that overlap between autism-related traits and other psychopathological traits can be theoretically apportioned into (a) overlap that is a result of the presence versus absence of an ASD diagnosis and (b) extraneous influences of unrelated psychopathological traits on symptom measurement. Practically, this implies that in some instances, controlling for behavioral disability would obscure autism symptom measurement, particularly in research attempting to link behavior with underlying genetic or neurobiologic variation. In situations where a clinician or researcher is interested in discriminating individuals with high autism symptom levels from other forms of psychopathology, then it would be reasonable to consider adjustment for these other psychopathological constructs.

The SRS has typically been interpreted using a single score reflecting global autism symptom/trait severity. Yet, in this study, SCI and RRB symptoms were imperfectly correlated (particularly in ASD-affected youth) and had a slightly different pattern of relationships with age and sex. Furthermore, SCI and RRB scores produced incremental validity over the SRS total raw score in spite of the already strong predictive validity of the total score. These findings, coupled with previous studies of autism symptom structure (Frazier et al., 2008; Mandy et al., 2012b), provide preliminary validation of proposed DSM-5 symptom domains and support separation of SCI and RRB. Future revisions of the SRS and other symptom measures—and current use of symptom measures in research—may benefit from the development and use of subscales tapping these distinct domains. Additional studies with raters blinded to diagnosis are needed to further establish the incremental validity of SCI- and RRB-derived indices from the SRS and their correspondence to proposed DSM-5 domains. These studies will be helpful for developing evidence-based medicine procedures (e.g. cut scores, diagnostic likelihood ratios) for using SCI- and RRB-derived measures in screening for ASD.

Limitations and future directions

The primary limitations of this study were simultaneous ascertainment of the SRS and ASD diagnoses, reliance on caregiver-reported clinical ASD diagnoses and comorbidities, and use of sibling data from an internet-based registry. Simultaneous collection of the SRS and ASD diagnoses from caregivers may have exaggerated the validity of the SRS. As a result, the effect sizes for ASD diagnosis in predicting SCI and RRB symptoms may be larger than the true effect size. Similarly, AUC values for SCI and RRB symptoms may be inflated. However, the results of sensitivity analyses using empirical classifications were highly similar and produced comparable effect sizes. This suggests that exaggeration cannot completely explain the very large effects observed for ASD diagnosis relative to other demographic and clinical factors. Furthermore, the previous validation study of ASD diagnoses in IAN (Lee et al., 2010), and the relative similarity between SRS scores in this study and in previous SRS validation work (Constantino et al., 2003a; Constantino and Gruber, 2005), indicates that the present findings are not substantially misestimated. Additional studies with independent diagnoses, although expensive and laborious to collect on a large scale, will be needed to confirm the present results.

Caregiver-reports of ADHD, anxiety disorder, or intellectual disability diagnoses are weak proxies for diagnoses based on semistructured research interviews. As a result, these effects are biased. Specifically, they almost certainly under-estimate the true relationships between internalizing and externalizing behavior, cognitive ability, and autism symptoms. Therefore, it is possible that these clinical factors may influence autism symptoms beyond the level observed in this study. Clearly, direct assessment of clinical diagnoses or the broader psychopathology constructs will be crucial for determining the exact magnitude of effects on SCI and RRB domains. Regardless, the present results indicate that the effects of these clinical correlates are not trivial and likely reduce the purity of caregiver-reports of autism symptoms.

The IAN internet registry does not include data from families unaffected by ASD. Saturation of ASD-affected cases may have enhanced the effect of ASD diagnosis in predicting SRS-derived autism symptom domains. However, the opposite is also possible; non-ASD siblings may provide a more stringent comparison than healthy individuals from the general population. Furthermore, the national scope and large size of the IAN registry make it preferable for quantifying the effects of multiple factors on autism symptom reports, covering the full range of autism symptom severity.

In spite of these limitations, the present findings indicate that quantitative, caregiver-reported autism symptoms are heavily influenced by the presence of an ASD diagnosis and, therefore, are excellent additions to the screening and assessment of ASD. Careful clinical assessments of ASD will benefit from considering the impact of demographic and other clinical factors on these caregiver-reports. Clinicians may also consider SCI and RRB symptoms separately, as these domains appear to have incremental validity in the prediction of ASD over global symptom alone. Future studies are needed to determine whether separate symptom domains should be used to track treatment response. Longitudinal studies will be valuable for further refining our understanding of demographic and clinical influences on SCI and RRB symptoms throughout development.

Footnotes

Acknowledgements

The authors wish to acknowledge the important contribution of the participants with autism, their siblings, and their families.

Conflict of interest

The contents are solely the responsibility of the authors and do not necessarily represent the official views of NIH.

Funding

This work was made possible by the Case Western Reserve University/Cleveland Clinic CTSA grant (grant number UL1 RR024989) provided by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health.

Dr Thomas W Frazier has received federal funding or research support from, acted as a consultant to, received travel support from, and/or received a speaker’s honorarium from the Simons Foundation, Forest Laboratories, Ecoeos, IntegraGen, Shire Development, Bristol-Myers Squibb, National Institutes of Health, and the Brain and Behavior Research Foundation. Dr Eric A Youngstrom has received travel support from Bristol-Myers Squibb and consulted with Lundbeck. Mrs Rebecca Embacher receives research support from IntegraGen. Dr Antonio Y Hardan has received research support from Forest and Bristol-Myers Squibb and is currently a consultant for IntegraGen. Dr Robert L Findling receives or has received research support from, acted as a consultant to, received royalties from, and/or served on a speaker’s bureau for Abbott, Addrenex, Alexza, American Psychiatric Press, AstraZeneca, Biovail, Bristol-Myers Squibb, Dainippon Sumitomo Pharma, Forest Laboratories, GlaxoSmithKline, Guilford Press, Johns Hopkins University Press, Johnson & Johnson, KemPharm Lilly, Lundbeck, Merck, National Institutes of Health, Neuropharm, Novartis, Noven, Organon, Otsuka, Pfizer, Physicians’ Post-Graduate Press, Rhodes Pharmaceuticals, Roche, SAGE, Sanofi-Aventis, Schering-Plough, Seaside Therapeutics, Sepracore, Shionogi, Shire, Solvay, Stanley Medical Research Institute, Sunovion, Supernus Pharmaceuticals, Transcept Pharmaceuticals, Validus, WebMD and Wyeth. Dr John N Constantino receives royalties from Western Psychological Services for the commercial distribution of one of the metrics used in this study, the Social Responsiveness Scale. Dr Charis Eng is a co-PI on a sponsored research contract from IntegraGen and is an unpaid member of the External Scientific Advisory Boards of Ecoeos and GenomOncology, and of the Genomic Medicine Advisory Board of Complete Genomics, Inc. Dr Paul Law has no conflicts of interest to disclose.

References

American Psychiatric Association. (2000) Diagnostic and statistical manual of mental disorders. Washington, DC: American Psychiatric Association.

American Psychiatric Association (2011) DSM-5 Development: 299.00 Autistic disorder. Available at: http://www.dsm5.org/ProposedRevisions/Pages/proposedrevision.aspx?rid=94

Bryk

Raudenbush

(1992) Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: SAGE.

Carter

Black

Tewani

. (2007) Sex differences in toddlers with autism spectrum disorders. Journal of Autism and Developmental Disorders 37: 86–97.

Centers for Disease Control and Prevention (CDC) (2012) Prevalence of autism spectrum disorders—autism and developmental disabilities monitoring network, 14 sites, United States, 2008. Morbidity and Mortality Weekly Report 61: 1–19.

Charman

Taylor

Drew

. (2005) Outcome at 7 years of children diagnosed with autism at age 2: predictive validity of assessments conducted at 2 and 3 years of age and pattern of symptom change over time. Journal of Child Psychology and Psychiatry and Allied Disciplines 46: 500–513.

Cohen

(1987) Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Erlbaum.

Constantino

Gruber

(2005) Social Responsiveness Scale: Manual. Los Angeles, CA: Western Psychological Services.

Constantino

Todd

(2003) Autistic traits in the general population: a twin study. Archives of General Psychiatry 60: 524–530.

10.

Constantino

Todd

(2005) Intergenerational transmission of subthreshold autistic traits in the general population. Biological Psychiatry 57: 655–660.

11.

Constantino

Davis

Todd

. (2003a) Validation of a brief quantitative measure of autistic traits: comparison of the social responsiveness scale with the autism diagnostic interview–revised. Journal of Autism and Developmental Disorders 33: 427–433.

12.

Constantino

Gruber

Davis

. (2004) The factor structure of autistic traits. Journal of Child Psychology and Psychiatry 45: 719–726.

13.

Constantino

Hudziak

Todd

(2003b) Deficits in reciprocal social behavior in male twins: evidence for a genetically independent domain of psychopathology. Journal of the American Academy of Child and Adolescent Psychiatry 42: 458–467.

14.

Constantino

Zhang

Frazier

. (2010) Sibling recurrence and the genetic epidemiology of autism. American Journal of Psychiatry 167: 1349–1356.

15.

Frazier

Youngstrom

Kubu

. (2008) Exploratory and confirmatory factor analysis of the autism diagnostic interview–revised. Journal of Autism and Developmental Disorders 38: 474–480.

16.

Frazier

Youngstrom

Speer

. (2012) Validation of proposed DSM-5 criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry 51: 28–40.e3.

17.

Hus

Bishop

Gotham

. (2013) Factors influencing scores on the social responsiveness scale. Journal of Child Psychology and Psychiatry and Allied Disciplines 54: 216–224.

18.

Kalb

Law

Landa

. (2010) Onset patterns prior to 36 months in autism spectrum disorders. Journal of Autism and Developmental Disorders 40: 1389–1402.

19.

Kreft

IGG

(1995) Hierarchical linear models: problems and prospects. Journal of Educational and Behavioral Statistics 20: 109–113.

20.

Lee

Marvin

Watson

. (2010) Accuracy of phenotyping of autistic children based on Internet implemented parent report. American Journal of Medical Genetics. Part B: Neuropsychiatric Genetics 153: 1119–1126.

21.

Lichtenstein

Carlstrom

Rastam

. (2010) The genetics of autism spectrum disorders and related neuropsychiatric disorders in childhood. American Journal of Psychiatry 167: 1357–1363.

22.

Lord

Rutter

DiLavore

. (2002) Autism Diagnostic Observation Schedule: ADOS Manual. Los Angeles, CA: Western Psychological Services.

23.

Lord

Schopler

Revicki

(1982) Sex differences in autism. Journal of Autism and Developmental Disorders 12: 317–330.

24.

Lundstrom

Chang

Kerekes

. (2011) Autistic-like traits and their association with mental health problems in two nationwide twin cohorts of children and adults. Psychological Medicine 41: 2423–2433.

25.

Mandy

Chilvers

Chowdhury

. (2012a) Sex differences in autism spectrum disorder: evidence from a large sample of children and adolescents. Journal of Autism and Developmental Disorders 42: 1304–1313.

26.

Mandy

Charman

Skuse

(2012b) Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry 51: 41–50.

27.

Mayes

Calhoun

(2010) Impact of IQ, age, SES, gender, and race on autistic symptoms. Research in Autism Spectrum Disorders 5: 749–757.

28.

Mayes

Calhoun

Murray

. (2009) Comparison of scores on the checklist for autism spectrum disorder, Childhood Autism Rating Scale, and Gilliam Asperger’s Disorder Scale for children with low functioning autism, high functioning autism, Asperger’s disorder, ADHD, and typical development. Journal of Autism and Developmental Disorders 39: 1682–1693.

29.

Mildenberger

Sitter

Noterdaeme

. (2001) The use of the ADI-R as a diagnostic tool in the differential diagnosis of children with infantile autism and children with a receptive language disorder. European Child & Adolescent Psychiatry 10: 248–255.

30.

Moss

Magiati

Charman

. (2008) Stability of the autism diagnostic interview–revised from pre-school to elementary school age in children with autism spectrum disorders. Journal of Autism and Developmental Disorders 38: 1081–1091.

31.

Peugh

Enders

(2005) Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models. Educational and Psychological Measurement 65: 717–741.

32.

Piven

Palmer

Jacohbi

. (1997) Broad autism phenotype: evidence from a family history study of multiple-incidence autism families. American Journal of Psychiatry 154: 185–190.

33.

Risi

Lord

Gotham

. (2006) Combining information from multiple sources in the diagnosis of autism spectrum disorders. Journal of the American Academy of Child and Adolescent Psychiatry 45: 1094–1103.

34.

Schafer

Graham

(2002) Missing data: our view of the state of the art. Psychological Methods 7: 147–177.

35.

Snow

Lecavalier

Houts

(2009) The structure of the autism diagnostic interview–revised: diagnostic and phenotypic implications. Journal of Child Psychology and Psychiatry and Allied Disciplines 50: 734–742.

36.

Szatmari

Merette

Bryson

. (2002) Quantifying dimensions in autism: a factor-analytic study. Journal of the American Academy of Child and Adolescent Psychiatry 41: 467–474.