Do the Autism Spectrum Quotient (AQ) and Autism Spectrum Quotient Short Form (AQ-S) Primarily Reflect General ASD Traits or Specific ASD Traits? A Bi-Factor Analysis

Abstract

In the current study, we fit confirmatory bi-factor models to the items of the Autism Spectrum Quotient (AQ) and Autism Spectrum Quotient Short Form (AQ-S) in order to assess the extents to which the items of each reflect general versus specific factors. The models were fit in a combined sample of individuals with and without a clinical diagnosis of autism spectrum disorders. Results indicated that, with the exception of the Attention to Details factor in the AQ and the Numbers/Patterns factors in the AQ-S, items primarily reflected a general factor. This suggests that when attempting to estimate an association between a specific symptom measured by the AQ or AQ-S and some criterion, associations will be confounded by the general factor. To resolve this, we recommend using a bi-factor measurement model or factor scores from a bi-factor measurement whenever hypotheses about specific symptoms are being assessed.

Keywords

bi-factor autism spectrum quotient psychometric confounding

It is now widely accepted that autism spectrum disorders (ASD) are most appropriately described not in terms of a single unitary impairment but in terms of multiple related traits. This multidimensionality of symptoms is reflected in current and historical diagnostic criteria, in psychometric measures of ASD symptoms, and in theoretical treatments of ASD and its etiology. Until recently, diagnosis of ASD was based on impairments in three domains sometimes referred to as the “classical triad” of autism. These domains were social, communication, and restricted repetitive activities (American Psychiatric Association, 1994). In current diagnostic criteria, the social and communication domains have been combined into a single dimension, but restricted repetitive behaviors are considered sufficiently distinct to be maintained as a separate dimension (American Psychiatric Association, 2013). Other associated features potentially include sensory symptoms, executive functioning and attention deficits, a local processing style, and a range of medical and psychological comorbidities (Bauman, 2010; Ben-Sasson et al., 2009; Buck et al., 2014; Happé, Booth, Charlton, & Hughes, 2006; Happé & Frith, 2006; Matson & Shoemaker, 2009).

With regard to the core features of ASD, although the extent of their interrelation and its cause are debated, it is generally assumed that they are all positively associated (Murray, McKenzie, Kuenssberg, & O’Donnell, 2014; Williams & Bowler, 2014). To some, this covariation may be indicative of a general latent ASD factor that underlies the set of more specific symptoms associated with the disorder. For example, Mandy, Charman, Puura, and Skuse (2014, p. 45) describe ASD as “currently conceptualized as a behavioral syndrome, whereby a cluster of observable characteristics is posited as the manifestation of the latent ASD disease entity.”

It could be argued that this view is also implicit in studies which have modelled ASD symptoms and behaviors as involving some higher order ASD factor, in addition to multiple specific first-order factors (e.g., Hoekstra et al., 2011; Kuenssberg, Murray, Booth, & McKenzie, 2014). The ability to represent the covariation among ASD symptoms as a general latent ASD factor does not, however, suggest that this latent factor need be anything more than a statistical entity or psychometric convenience (e.g., see van der Maas et al., 2006), and in the current study, we use the term “general ASD factor” to mean variance shared among most or all symptoms (i.e., a general factor in a statistical but not necessarily causal sense). There is an important theoretical distinction between a statistical general factor and a reified general factor that represents the common cause of multiple symptoms. The latter interpretation requires much stronger assumptions about the causal structure underlying ASD symptoms. Nevertheless, the presence of a statistical general factor highlights the fact that whenever individual symptoms are measured, this measurement is likely to reflect not only symptom-specific variance (represented by the specific factors) but also variance that is shared with other symptoms of ASD (represented by the general factor).

This covariation among ASD traits creates a challenge with respect to investigating the causes and consequences of specific ASD symptoms because when attempting to measure specific symptoms of ASD, the systematic variance in that measure may only partially reflect the specific symptom of interest. In addition, it may also reflect substantial variance due to a general ASD factor. When the shared variance among ASD symptoms is not accounted for, it can confound associations between specific symptoms and some criterion which can then reflect not only an association due to the specific symptom but also due to a general ASD factor. For some purposes, this conflation of general and specific variance may have important implications for tests of theoretical hypotheses. Many hypotheses in ASD research refer to specific symptoms and thus require an estimate that is unconfounded by general ASD variance. One example is the hypothesis that positive symptoms of schizophrenia should correlate negatively with social impairments of ASD (Russell-Smith, Maybery, & Bayliss, 2011). In this case, a negative association with social impairment could be masked if the general factor of ASD is positively associated with these symptoms.

A solution which has been employed in other areas of psychopathology is to use a bi-factor measurement model (Reise, Morizot, & Hays, 2007). An example of the bi-factor structure is shown in Figure 1. The bi-factor model represents items within a questionnaire as being influenced by a general factor (influencing all or most items) and a specific factor (influencing a specific subset of items reflecting some specific trait of interest). The specific factors are all orthogonal to the general factor, and they are often also set orthogonal to one another, but this is not necessary from a statistical point of view, merely conventional and often providing some interpretational benefits.

Figure 1.

Example of a bi-factor model.

The bi-factor model allows an estimate of the extent to which items reflect covariation due to specific symptoms versus a general factor and by the same token helps identify items that are good measures of a specific factor versus a general factor. Bi-factor models have been used in this way in inventories measuring a range of clinical constructs, such as attention-deficit-hyperactivity disorder (Willoughby & Blanton, 2015), oppositional defiant disorder (Burke et al., 2014), and general psychopathology (e.g., Caspi et al., 2014). For example, Simms, Grös, Watson, and O’Hara (2008) used a bi-factor model to assess the extent to which general and specific factors influenced responses to the Inventory of Depression and Anxiety symptoms (Watson et al., 2007). They found that the shared item variance was attributable approximately equally to a general factor and the relevant specific factors. However, it is not uncommon for a general factor to dominate even purportedly multidimensional inventories. For example, Watkins, Canivez, James, James, and Good (2013) fit a bi-factor model to the Weschler Intelligence Scale for Children Fourth UK edition (Wechsler, 2003) in a sample of children referred for the assessment of learning difficulties. They found that the general factor accounted for almost two thirds of the common item variance.

To date, few studies have addressed the question of the extent to which the items in commonly used inventories in ASD research reflect general versus specific factors. Snow, Lecavalier, and Houts (2009) fit a bi-factor model to the items of the Autism Diagnostic Interview-Revised (Lord, Rutter, & Le Couteur, 1994) in individuals with a clinical diagnosis of pervasive developmental disorder but did not interpret the solution, reporting that it did not yield better fit than an oblique model. Lecavalier, Gadow, DeVincent, Houts, and Edwards (2009), for the same reason, did not report the solution for a bi-factor model fit to the items of the Early Childhood Inventory-4 (Gadow & Sprafkin, 2000) and the Childhood Symptom Inventory-4 (Gadow & Sprafkin, 2000) in a sample of children referred to a developmental disability clinic.

However, one possibility is that in using only clinically diagnosed or referred individuals, item intercorrelations were attenuated due to range restriction, reducing the variance common to all items and thus any support for a general factor. Another study, which used a general population sample including individuals with a self-reported diagnosis of ASD, reported that a bi-factor model fit best among those assessed (Posserud, Breivik, Gillberg, & Lundervold, 2013). It was fit to a 7-item screening tool, the “autism self-report for adolescents and adults,” derived from Asperger’s syndrome (AS; and high-functioning autism [HFA]) diagnostic interview (Gillberg, Gillberg, Råstam, & Wentz, 2001). Although it was the best fitting model, several core symptoms had only low loadings on the general factor.

In the current study, we fit a bi-factor confirmatory factor model in a popular measure of ASD traits—the autism spectrum quotient (AQ; Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001) as well as its abridged version, the autism spectrum quotient short form (AQ-S; Hoekstra et al., 2011)—and evaluate the extent to which the specific symptom areas measured by each reflect the intended specific factors versus a general factor of ASD.

Method

Participants

Data came from four (one clinical sample, three control samples) sources which have been separately used in previous studies but were combined into a single data set for the current study. From the original samples, cases were removed if they (a) were under the age of 18 years, (b) had no responses to any of the 50 AQ items, or (c) were participants recruited for a control sample and self-reported a diagnosis of ASD or intellectual disability. This resulted in a total sample of 562 respondents (204 male, 357 female; 1 not reported) with a mean age of 30.6 years (SD = 11.8 years), ranging from 18 to 69 years. Specific details for each of the four constituent samples are provided below.

Clinical Sample

One hundred and forty-seven participants were drawn from a clinical sample. These individuals all had a clinical diagnosis of either AS or HFA. AS was defined as meeting the criteria for HFA but with no history of delay in language, while HFA was defined as meeting the criteria for autism but with normal intellectual functioning. Data were obtained from case notes from a specialist Regional ASD Consultancy service and clinical psychology services in Scotland. Diagnoses were made with reference to Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.) criteria by an experienced clinician and used clinical interview: informant interview where available and individual assessment such as neuropsychological assessment where indicated. Prior to finalizing a diagnosis, each case was discussed at a multidisciplinary clinic. Participants were primarily male (n = 107, 73%) and had a mean age of 33.5 years (SD = 10.7 years; range = 18-62 years). The sample has been used in several previous studies (Booth et al., 2013; Kuenssberg et al., 2014; Murray, Booth, McKenzie, Kuenssberg, & O’Donnell, 2014; Murray, McKenzie et al., 2014).

Control Samples

One hundred and sixty-four participants came from a previous psychometric study of the AQ (Murray, Booth, et al., 2014). These participants were primarily female (n = 125, 76%). They had a mean age of 30.1 years (SD = 11.3), ranging from 18 to 65 years.

Data for 97 participants came from an ongoing study of emotion recognition and ASD traits which have been previously used in a study by Murray, McKenzie, et al. (2014). These participants were recruited online and from the university community, and included 27 males, 69 females, and 1 participant who described their gender as “other.” The mean age of the sample was 31.1 years (SD = 12.5, range = 18-68).

Last, 154 participants came from an ongoing study of sex differences in ASD traits in the general population. These have also been previously used in the study by Murray, McKenzie, et al. (2014). These participants were recruited online and were composed of 123 females and 31 males. The mean age of the sample was 28.0 (SD = 12.4, range = 18-69).

Ethical approval was obtained from the relevant ethics committee in all cases. We analyzed data from both clinically diagnosed individuals and individuals with no clinical diagnosis of ASD together because the development of the AQ was based on the premise that it is possible to measure ASD traits on a continuum from very extreme levels to levels found in the general population, with clinically diagnosed individuals merely being at the extreme end of this continuum. Indeed, in practice, the AQ and its derivatives are used in both clinical and general population samples. Furthermore, restricting analyses to either those who have a clinical diagnosis of ASD or those who do not could lead to a downward bias in the parameters of a bi-factor model due to variance restriction (Murray, McKenzie, et al., 2014).

Measures

Autism Spectrum Quotient

The AQ is a 50-item questionnaire developed to measure ASD traits in adults of normal intellectual ability (Baron-Cohen et al., 2001). The areas measured by the inventory were chosen based on the classical triad of ASD as well as commonly associated features. The AQ is organized into 5 subscales, each with 10 items. These subscales aim to measure Social Skills, Attention Switching, Attention to Detail, Imagination, and Communication. Abbreviated item contents are provided in Table 1. The phrasing of items is such that they reflect behavioral tendencies and preferences rather than symptoms or impairments. Thus, it is most accurately characterized as a measure of autistic traits.

Table 1.

Standardized factor loadings of the bi-factor CFA (WLSMV) of the AQ50 (n = 562).

Item	Abbreviate item content	Factor loadings						Explained variance
Item	Abbreviate item content	General	Social skills	Attention switching	Attention to detail	Communication	Imagination	Total	General to total ratio
AQ1	Prefer to do things on own	.46	.43					.40	.54
AQ11	Find social situations easy	.87	.27					.83	.91
AQ13	Prefer library to party	.59	.46					.56	.62
AQ15	Drawn to people over things	.74	.20					.59	.94
AQ22	Hard to make new friends	.81	.10					.67	.99
AQ36	Can infer thoughts/feelings from faces	.77	−.25					.65	.91
AQ44	Enjoy social occasions	.80	.51					.89	.71
AQ45	Difficulty inferring intentions	.79	−.27					.70	.89
AQ47	Enjoy meeting new people	.77	.41					.77	.78
AQ48	Good diplomat	.64	−.10					.42	.98
AQ2	Prefer to do things same way	.56		.46				.53	.60
AQ4	Get so absorbed	.55		.14				.32	.94
AQ10	Keep track of several conversations	.70		−.11				.50	.98
AQ16	Very strong interests	.46		.33				.32	.66
AQ25	Not upset by disruption of daily routine	.53		.47				.50	.55
AQ32	Multitasking easy	.62		−.06				.38	.99
AQ34	Enjoy doing things spontaneously	.70		.26				.56	.88
AQ37	Can switch back after interruption	.60		.04				.36	1.00
AQ43	Like to plan activities carefully	.51		.39				.42	.63
AQ46	Anxiety in new situations	.66		.22				.48	.90
AQ5	Notice small sounds	.28			.29			.16	.48
AQ6	Notice information such as number plates	.30			.63			.49	.19
AQ9	Fascinated by dates	.29			.60			.45	.19
AQ12	Notice details others don’t	.23			.60			.41	.13
AQ19	Fascinated by numbers	.30			.61			.47	.19
AQ23	Notice patterns all the time	.33			.66			.55	.20
AQ28	Concentrate on whole picture over details	.58			.24			.39	.86
AQ29	Not good at remembering phone numbers	.07			.48			.24	.02
AQ30	Don’t notice small changes	−.11			.29			.10	.12
AQ49	Not good at remembering dates of birth	−.05			.44			.19	.02
AQ7	Inadvertently impolite	.63				.42		.57	.69
AQ17	Enjoy social chitchat	.78				−.37		.74	.81
AQ18	Difficult for others to get word in edgeways	.19				.48		.27	.13
AQ26	Don’t know how to maintain conversation	.82				−.07		.68	.99
AQ27	Easy to read between the lines	.73				.17		.55	.95
AQ31	Can’t tell if someone is bored listening	.62				.20		.42	.91
AQ33	Can take turns talking on phone	.59				.18		.38	.91
AQ35	Last to understand point of joke	.59				.27		.42	.82
AQ38	Good at social chitchat	.88				−.38		.92	.85
AQ39	Go on about the same thing	.61				.60		.73	.51
AQ3	Hard to create picture in mind	.38					.49	.38	.38
AQ8	Easy to imagine characters in a story	.53					.50	.52	.53
AQ14	Making up stories is easy	.24					.54	.35	.17
AQ20	Working out character intentions in story	.62					.25	.45	.86
AQ21	Don’t enjoy fiction	.37					.38	.28	.49
AQ24	Prefer theatre to museum	.42					.04	.18	.99
AQ40	Enjoyed pretend play as child	.55					.36	.43	.71
AQ41	Collect information about categories of things	.61					−.07	.38	.99
AQ42	Difficult to imagine being someone else	.62					.19	.41	.91
AQ50	Easy to play pretending games with children	.59					.40	.51	.69

Note. CFA = confirmatory factor analysis; WLSMV = weighted least squares mean and variances; AQ = autism spectrum quotient. Values in boldface are nonsignificant at p < .05.

In this study, respondents were offered a four point response scale from Strongly Agree to Strongly Disagree. Some items are phrased in a “forward” direction where choosing Strongly Agree suggests higher levels of autistic traits, and some are phrased in a “reverse” direction where selecting Strongly Disagree suggests higher levels of autistic traits. For this study, all items were (re-)coded in the direction of higher scores reflecting higher autistic trait levels. Many previous studies have collapsed the 4-point scale of the items into a dichotomous “0” versus “1” scoring scheme; however, this generally entails a loss of information, and in the current study, we scored the items using a “1” to “4” scoring scheme to maximize reliability. All participants in the current sample were administered the full AQ. To disambiguate the AQ from its short form AQ-S in the current study, we henceforth refer to the full 50-item AQ as the “AQ50.”

Autism Spectrum Quotient Short Form

We also present analyses of the abridged version of the AQ50, the AQ-S (Hoekstra et al., 2011). The AQ-S contains a subset of 28 items judged to provide the best measures of ASD traits from the AQ50 while still ensuring that all key content areas are adequately represented. The process by which the items of the AQ-S were selected from the full AQ50 is described in full in Hoekstra et al. (2011). Briefly, in a sample of Dutch controls (no ASD diagnosis), exploratory factor analyses were conducted and potentially problematic items identified on the basis of content (e.g., an item very similar in content or phrasing to another item) and the factor analysis results (e.g., low loadings on the relevant factor). After selecting an optimal factor structure, a number of potentially problematic items were excluded. A series of confirmatory factor analyses guided further refinements. Finally, confirmatory factor analyses in two independent samples (a further sample of Dutch controls and a sample of English controls) were used to verify that the factor structure developed in the initial sample provided a good representation of the item covariances in these new samples, supporting its generalizability.

The set of items selected using these methods comprised 28 of the original 50 and could be represented in terms of 5 specific dimensions of Social Skills, Routine, Switching, Imagination, and Numbers/Patterns. A higher order Social Behavior dimension underlying all of these dimensions, except Numbers and Patterns, was also supported. The Numbers/Patterns was relatively independent of all other dimensions and correlated with the Social Behavior factor at only r = .2.

Statistical Procedure

We estimated bi-factor confirmatory factor models based on previous psychometric evidence on the structure of the AQ50 and the AQ-S. Specifically, we estimate models closely related to the structures presented in Baron-Cohen et al. (2001) for the AQ50, and Hoekstra et al. (2011) for the AQ-S.

Baron-Cohen et al. (2001) originally developed the AQ50 and suggested a 5-factor structure with 10 items measuring each factor. The factors were labelled Social Skills (Items 1, 11, 13, 15, 22, 36, 44, 45, 47, 48), Attention Switching (Items 2, 4, 10, 16, 25, 32, 34, 37, 43, 46), Attention to Detail (Items 5, 6, 9, 12, 19, 23, 28, 29, 30, 49), Communication (Items 7, 17, 18, 26, 27, 31, 33, 35, 38, 39), and Imagination (Items 3, 8, 14, 20, 21, 24, 40, 41, 42, 50). For the AQ50, we specified a bi-factor model with a general factor loading on all items and five specific factors with item loadings as described above.

Hoekstra et al. (2011) developed a higher order model for the AQ-S in which a second-order Social Behavior factor underlay four specific factors labelled Social Skills (Items 1, 11, 13, 15, 22, 44, 46, 47), Routine (Items 2, 25, 34, 46), Switching (Items 4, 10, 32, 37), and Imagination (Items 3, 8, 14, 20, 36, 42, 45, 50). In addition, the general Social Behavior factor was allowed to correlate with a specific Numbers/Patterns factor (Items 6, 9, 19, 23, 41). We evaluated a bi-factor structure based on, but not corresponding exactly to, this model. Specifically, we specified a model in which all items loaded on both a general and specific factor, and these factors were mutually uncorrelated. The Hoekstra et al. (2011) study would imply that the items from the Numbers/Patterns factor should not load on the general factor. In this study, we allowed them to load on the general factor so that we could evaluate all items for general versus specific factor variance.

It is common practice to compare the fit of a bi-factor model to a higher order model; however, in the context of the current study where our goal was to estimate the extent to which AQ50 and AQ-S items reflect general versus specific factor variance, a higher order model would provide no additional information. Furthermore, previous studies have suggested that whenever there are unmodelled complexities in a psychometric model (e.g., cross-loadings or correlated residuals), these kinds of model comparisons may be biased in favor of the bi-factor model, which, being the more general model, is relatively less sensitive to misspecification in terms of its impact on model fit (Murray & Johnson, 2013). However, we did compare the fit of the bi-factor model with a correlated first order factor model, which includes only specific factors and no general factor. This provided a test of whether a general factor was supported.

Models were estimated using weighted least squares mean and variances estimation in Mplus 6.11 (Muthén & Muthén, 1998-2013). Weighted least squares mean and variance was used to account for the ordered categorical response format of the scale. By default, this method uses pairwise deletion to deal with missingness, and given the low rate (0.54%) of missing item responses and high (>98% in all cases) covariance coverage, this was judged a reasonable strategy. Latent factor variances were fixed to 1 for scaling and identification purposes.

Model fit was evaluated based on the comparative fit indices (CFI), Tucker Lewis Index (TLI), root mean square error of approximation (RMSEA), and the weighted root mean square residual (WRMR). For the CFI and TLI, values of >.90 to .95, and for RMSEA, values of <.08 were taken as indicative of good fit (Hu & Bentler, 1999; Schermelleh-Engel, Moosbrugger, & Muller, 2003).

Strength of General and Specific Factors

As an index of the extent to which the items reflected a general versus specific factors, we computed the explained common variance (ECV) statistic (see Reise, 2012). This is computed as a ratio of the variance explained by the general factor to the variance explained by the general plus specific factors. Higher values of ECV suggest that a higher proportion of the common items variance is inventory-wide, rather than specific to a set of items in a subscale.

We also estimated the reliability of the individual subscales, both before and after controlling for the general factor. Similarly, we estimated both the overall reliability of the total scales and the reliability of the total scales that were attributable to the general factor alone (i.e., controlling for the specific factors). All of these reliability indices are variants on McDonald’s (1999) ω statistic, which estimates the proportion of variance in a scale (or subscale) that is attributable to a given factor or set of factors. The denominator is the total scale or relevant subscale variance, and the numerator is the variance due to the relevant factor(s). For example, a measure of the reliability of a scale due to the general factor is

ω_{h} = \frac{{(\sum^{​} λ_{i G})}^{2}}{{(\sum^{​} λ_{i G})}^{2} + {(\sum^{​} λ_{i S 1})}^{2} + {(\sum^{​} λ_{i S 2})}^{2} + {(\sum^{​} λ_{i S 3})}^{2} + \dots {(\sum^{​} λ_{i S P})}^{2} + \sum^{​} θ_{i}^{2}},

where $λ_{i G}$ represents general factor loading for item i, $λ_{i S 1}$ . . . $λ_{i S P}$ represent the specific factor loadings for item i on specific factors 1-P, and $θ_{i}^{2}$ represents the error variance for item i. When referring to scale or subscale reliability due to both the general factor and the relevant specific factor, we use “omega total” or $ω_{T}$ , and when referring to scale or subscale reliability due only to the general factor (for total scales) or the relevant specific factor (for subscales), we use “omega hierarchical” or $ω_{h}$ . Both ECV and the $ω$ indices were computed from the parameter estimates obtained from the bi-factor model.

Practical Implications

Although confirmatory factor analyses and indices derived from these analyses provide valuable information about the extent to which specific items and tests as a whole reflect general versus specific factors, they do not necessarily lead directly to recommendations about whether individual subscale or total scores are appropriate for use in practical applications. We, therefore, estimated additional indices to inform practical recommendations (Reise, Bonifay, & Haviland, 2013).

First, we estimated a measure of worst-split half reliability, β (Revelle, 1979) for the total scales and all specific subscales. Revelle (1979) has argued that lower level subscales are usefully combined into larger higher level scales, and when doing so increases β. Therefore, comparing β for the individual subscales of the AQ50 and AQ-S and the inventories as a whole can help determine the level of aggregation of items that is most appropriate. The β index was computed from the original item covariance matrix for each inventory using the “psych” package in R statistical software (Revelle, 2015; R Core Team, 2014 ).

Second, we evaluated whether use of subscale scores would be appropriate using the methodology proposed by Haberman (2008). Haberman (2008) noted that a scale total score is sometimes a better estimator of the true score for a construct measured by a subscale than is the subscale score itself. This can happen if the subscales contributing to a total score are highly correlated, and the reliability of the total score sufficiently exceeds that of the subscale. In this situation, using subscale scores is not recommended. To assess whether this is the case for any of the AQ50 and AQ-S subscales, the proportional reduction in mean square error (PRMSE) with respect to the construct measured by the relevant subscale was computed for both the total score and the subscale score. If the former exceeds the latter, then subscales scores should not be used. PRMSE values were computed using the “sirt” package in R statistical software (Robitzsch, 2015).

Last, to assess the potential impact of confounding of specific factor associations with the general factor, we evaluated the extents to which each of the specific factors of the AQ50 and AQ-S were predicted by sex both before and after controlling for the general factor. In the “uncontrolled” models, we estimated a first-order correlated factors model and regressed each of the factors on sex. In the “controlled” models, we estimated a bi-factor measurement model, which includes a general factor that accounts for the shared variance across all items. Each of the specific factors, which are free of general factor variance, were regressed on sex. We also estimated 95% confidence intervals for each effect.

Results

AQ50

Model fit for the bi-factor model was good according to all model fit indices (χ² = 3646.24 (1,125), p < .001; RMSEA = .06; WRMR = 1.72; CFI = .92; TLI = .91) and better than that of the first order correlated factors model (χ² = 4744.05 (1,165), p < .001; RMSEA = .07; WRMR = 2.11; CFI = .88; TLI = .88), which would be considered to provide poor fit according to the CFI and TLI. These results supported the inclusion of the general factor in the AQ50.

Standardized factor loadings, total item variance explained, and the proportion of total variance attributable to the general factor for the AQ50 are provided in Table 1.

General factor loadings ranged from |.05| to |.88|, with the general factor accounting for between 0.2% and 100% of the total explained item variance. Two items (29 and 49) had nonsignificant general factor loadings, and the general factor loadings for two items (30 and 49) were negative, despite all items being coded in the same direction.

Specific factor loadings were quite variable across items. For four of the five specific factors, there was at least one item which had either a nonsignificant or negative loading. Only for the Attention to Detail specific factor did all items have significant loadings in the expected direction. The items in this factor had some of the lowest general factor loadings, suggesting they capture variability that is somewhat independent of that in the remainder of the AQ50.

These patterns of factor loadings are reflected in the factor reliabilities as estimated by the $ω$ values. As can be seen from Table 2, the general factor showed very good reliability according to the $ω_{h}$ (.91), but the values for four of the specific factors were very low, ranging from .05 to .24. Attention to Details, however, displayed moderate unique reliability (.67). Overall, the ECV value (.70) suggested that most of the variance shared among items was inventory-wide rather than subscale specific.

Table 2.

ECV, Omega Total, Omega Hierarchical, Beta, and PRMSE for the AQ50 and AQ-S.

	ECV	ω _T	ω _h	β	PRMSE total score	PRMSE modified total score	PRMSE subscale score
AQ50	.70
General		.97	.91	.83	—	—	—
(excluding Attention to Details items)		(.97)	(.80)	(.89)
Social Skills		.94	.05	.83	.84	.89	.91
Attention Switching		.87	.10	.74	.85	.85	.84
Attention to Details		.81	.67	.59	.33	—	.75
Communication		.91	.05	.73	.88	.91	.86
Imagination		.85	.24	.71	.78	.80	.81
AQ-S	.63
General		.96	.86	.83	—	—	—
(excluding Number/Patterns items)		(.96)	(.71)	(.87)
Social Skills		.93	.27	.84	.79	.85	.90
Routine		.82	.18	.72	.77	.77	.76
Switching		.82	.19	.71	.73	.73	.73
Imagination		.87	.21	.71	.76	.76	.82
Numbers/Patterns		.83	.62	.70	.38	—	.77

Note. ECV = explained common variance; PRMSE = proportional reduction in mean square error. For the AQ50, the modified total score excludes the Attention to Detail items, and for the AQ-S, it excludes the Numbers/Patterns items.

As the items in the Attention to Details subscale appeared to be relatively independent of the remaining items of the AQ50, an estimate of worst-split half reliability for the AQ50 total score was obtained both with (Total Score) and without these items (Modified Total Score). Based on a comparison of β values for the AQ50 total scales and the subscales and Revelle’s (1979) suggestion that a good heuristic for when scales should be aggregated is when it results in an increase in this index, the appropriate level of aggregation would appear to be the total scale level. This was true irrespective of whether the Attention to Detail items were included in the total scale or not but more so when they were not.

Again, given that the Attention to Detail items did not appear to be strongly related to a general factor in the AQ50, we also computed PRMSE for each subscale based on a total score, both including and excluding these items; however, this made little difference to results. For the Social Skills, Attention to Detail, Attention Switching, and Imagination subscales, subscale score PRMSE was greater than or effectively equal to total score PRMSE, suggesting that the use of subscale scores is supported for these constructs. For the Communication subscale, however, total score PRMSE was greater than subscale PRMSE, suggesting that this construct may be better predicted using AQ50 scores than subscale scores.

Taking all results together, with the exception of the Communication subscale, using both subscale and total scores appears to be appropriate in the AQ50. A modified total score that excludes the Attention to Detail items should be preferred to a total score of all 50 items because these items appear to reflect a relatively distinct construct compared with the rest of the items of the AQ. In addition, as the $ω_{h}$ values show, subscale scores should be used in the knowledge that much of the reliability of these subscales is due to a general factor, rather than subscale-specific factors.

Autism Spectrum Quotient Short Form

The initial model specification for the AQ-S yielded a Heywood case: a negative residual variance estimate for item 32. We dealt with this by constraining this residual variance to a small positive value (.01). Parameter estimates suggested that this item was strongly related to both a general and specific factor. Inspecting these parameters and the content of the item (referring to multitasking ability) suggested that there was no reason not to retain the item in spite of it initially yielding an out-of-range parameter estimate. The model with this additional constraint showed good to excellent model fit according to all indices (χ² = 973.13 (321), p < .001; RMSEA = .06; WRMR = 1.27; CFI = .96; TLI = .95). Model fit for the first order correlated factors model was reasonable (χ² = 1537.95 (339), p < .001; RMSEA = .08; WRMR = 1.67; CFI = .93; TLI = .92). While the raw fit may suggest this model to be plausible by the cutoff values highlighted above, the difference in fit between the bi-factor and the correlated factors model was reasonably large (ΔRMSEA = .02; ΔCFI = −.03; ΔTLI = −.03). Again, these results support the inclusion of the general factor in the AQ-S.

Standardized factor loadings, total item variance explained, and the proportion of total variance attributable to the general factor for the AQ-S are displayed in Table 3.

Table 3.

Standardized Factor Loadings of the Bi-Factor CFA (WLSMV) of the AQ-S (n = 562).

Item		Factor loadings						Explained variance
Item		General	Social skills	Routine	Switching	Imagination	Numbers/patterns	Total	General to total ratio
AQ1	Prefer to do things on own	.40	.49					.40	.40
AQ11	Find social situations easy	.81	.42					.83	.79
AQ13	Prefer library to party	.51	.54					.55	.47
AQ15	Drawn to people over things	.72	.30					.61	.86
AQ22	Hard to make new friends	.78	.25					.66	.91
AQ44	Enjoy social occasions	.69	.64					.88	.54
AQ47	Enjoy meeting new people	.70	.53					.78	.63
AQ46	Anxiety in new situations	.66	.17	.20				.50	.87
AQ2	Prefer to do things same way	.59		.37				.48	.72
AQ25	Not upset by disruption of daily routine	.55		.52				.57	.54
AQ34	Enjoy doing things spontaneously	.74		.25				.61	.89
AQ4	Get so absorbed	.53			.17			.31	.91
AQ10	Keep track of several conversations	.72			.16			.54	.95
AQ32	Multitasking easy	.60			.80			.99	.36
AQ37	Can switch back after interruption	.61			.24			.43	.86
AQ3	Hard to create picture in mind	.38				.57		.47	.31
AQ8	Easy to imagine characters in a story	.53				.51		.54	.53
AQ14	Making up stories is easy	.26				.52		.34	.20
AQ20	Working out character intentions in story	.61				.24		.43	.87
AQ36	Can infer thoughts/feelings from faces	.75				.13		.58	.97
AQ42	Difficult to imagine being someone else	.62				.18		.42	.92
AQ45	Difficulty inferring intentions	.77				.13		.62	.97
AQ50	Easy to play pretending games with children	.61				.27		.45	.84
AQ6	Notice information such as number plates	.29					.62	.46	.18
AQ9	Fascinated by dates	.27					.69	.55	.13
AQ19	Fascinated by numbers	.28					.72	.60	.13
AQ23	Notice patterns all the time	.31					.64	.50	.19
AQ41	Collect information about categories of things	.58					.32	.44	.77

Note. AQ = autism spectrum quotient; AQ-S = autism spectrum quotient short form; CFA = confirmatory factor analysis; WLSMV = weighted least squares mean and variances. All loadings are significant at p < .05.

General factor loadings for the AQ-S items were all significant and positive and ranged from .26 to .81. The general factor accounted for between 13% and 97% of the total explained item variance. The ECV for the AQ-S was .63.

As with the AQ50, there was some degree of variability in the magnitudes of the specific factor loadings in the AQ-S; however for the AQ-S, all specific factor loadings were (a) significant and (b) positive, as would be expected. This pattern resulted in $ω_{h}$ values for the specific factors ranging from .18 to .62 (see Table 2). Again, the general factor showed excellent reliability $ω_{h} =$ 86.

Analogous to the AQ50, we computed worst-split half reliability based on a total score of the 28 AQ-S items and a modified total score that excluded the low general factor loading Numbers/Patterns items. The resulting β values suggested that the appropriate level of aggregation was the total scale level excluding the Numbers/Patterns items.

The PRMSE values suggested that for the Social Skills, Imagination, and Numbers/Patterns subscales, the subscale score provided the best prediction for the subscale constructs. In the case of Switching, the PRMSE values were identical, and for Routine, PRMSE was slightly higher for the total score.

Collectively, the results from the AQ-S mirror the conclusions of the AQ50. The $ω_{h}$ estimates suggest that much of the subscale variance is in fact attributable to the general factor, and based on the β estimates, the general factor appears to be the best level of aggregation. However, the PRMSE analysis indicated that the subscale scores remain plausible.

Criterion Associations

Table 4 contains the parameter estimates from the multiple indicators and multiple causes models for the effect of sex on each of the specific factors estimated from both a first-order and bi-factor measurement model. The effects reported are interpreted as the standard deviation change in the specific factors for being male (= 0) versus female (= 1). As can be seen in Table 4, women consistently score lower on all specific factors across both models.

Table 4.

Standardized Coefficients for the Effect of Sex on Specific Factor Estimates From First-Order and Bi-Factor MIMIC Models (n = 561).

Specific factor	First order		Bi-factor
Specific factor	Estimation	95% CI	Estimation	95% CI
AQ50
Social Skills	−0.58	−0.74 to −0.42	−0.61	−0.89 to −0.34
Attention Switching	−0.71	−0.87 to −0.55	−1.30	−1.51 to −1.11
Attention Details	−0.39	−0.57 to −0.22	−0.30	−0.50 to −0.11
Communication	−0.67	−0.83 to −0.52	−0.68	−0.87 to −0.48
Imagination	−0.74	−0.91 to −0.58	−0.92	−1.14 to −0.70
AQ-S
Social Skills	−0.52	−0.68 to −0.35	−0.76	−1.02 to −0.51
Routine	−0.51	−0.70 to −0.33	−0.93	−1.23 to −0.62
Switching	−0.83	−0.99 to −0.66	−1.33	−1.56 to −1.12
Imagination	−0.71	−0.87 to −0.55	−1.04	−1.27 to −0.82
Numbers/Patterns	−0.55	−0.72 to −0.38	−0.52	−0.71 to −0.33

Note. MIMIC = multiple indicators and multiple causes; AQ = autism spectrum quotient. Coefficients are standardized with respect to the latent variable only. Coefficients may be interpreted as the standard deviation difference for being male (= 0) versus female (= 1).

For both the AQ-S and the AQ50, in general, the estimates of the effects of sex on the specific factors are stronger in the bi-factor models. Exceptions to this are the Numbers and Patterns factor in the AQ-S and the Attention to Details and Communications factors of the AQ50. One interpretation here is that general variance is masking stronger sex effects; however, caution is required as the confidence intervals for sex effects in the bi-factor models are much wider. The consistency in the direction of effects suggests that in this instance at least, general factor variance is not hugely distorting covariate effects—but the presence of any difference does suggest attention should be paid to this issue if a study’s intention is to understand the effects of external factors on specific factors, not the general factor.

Discussion

In the current study, we used a bi-factor confirmatory factor model to assess the extent to which the items of the full AQ (AQ50) and its short form, the AQ-S, reflect a general factor of ASD versus factors reflecting more specific symptoms. For the AQ50, the majority of shared item variance was attributable to the general factor. The main exceptions were the items of the Attention to Details subscale, which tended to have lower general factor loadings but higher specific factor loadings. Similarly, for the AQ-S, with the exception of the items in the Numbers/Patterns scale, item covariance primarily reflected a general factor rather than specific factors. Overall, in both inventories, shared item variance was to a large extent inventory-wide rather than subscale-specific.

Several authors have previously discussed the implications of the extent to which items in an inventory reflect general factor as opposed to specific factor variance. One implication concerns the appropriate level of analysis, in particular, whether the strength of the general factor is such that items from different subscales should be combined into a single scale. Revelle (1979) argued that combining items from scales would be justified if it resulted in an increase in the worst split half reliability, β. For the AQ50, this heuristic suggested it should be scored in two parts: a modified scale that includes all AQ50 items except those from the Attention to Details subscale and a separate Attention to Details subscale. On the other hand, the PRMSE values suggested that if subscale scores were desired, it would generally be appropriate to use subscale scores, except for the Communication construct, which is likely to be better estimated using the AQ50 total score.

For the AQ-S, Revelle’s heuristic suggested that items should be organized into a general scale comprising the items of the Social Skills, Routine, Switching, and Imagination items and a separate Numbers/Patterns subscale. However, the PRMSE values indicated that only the Attention Switching and Routine constructs would be expected to be better predicted by the AQ-S general scale score than by their respective subscale scores. The use of subscale scores were supported for the Social Skills, Numbers/Patterns, and Imagination constructs.

If using subscale scores from the AQ50 or AQ-S, it is important to note that a large part of their variance and reliability is due to a general factor. Reise, Moore, and Haviland (2010) noted that using subscales as measures of specific factors may be misleading when their systematic variance is mostly due to a general factor. Similarly, DeMars (2013) noted that the subscales may appear to be highly reliable but will not show distinct correlations with external criteria because of confounding with the general factor. Therefore, it will be difficult to establish discriminant validity of specific symptom subscales as well as specific correlates of the symptoms they are assumed to measure. These kinds of considerations apply equally when the general factor is of interest. For example, the presence of a subscale structure when left unmodelled creates issues such as violations of local independence and attendant overestimates of the reliability of the scale as a measure of a general ASD factor (e.g., Braeken, 2011).

In the current study, in both the AQ-S and the AQ50, the sizes of the sex difference in specific factors depended on whether or not the general factor was controlled for but the direction of the effect did not. Specifically, males scored higher on all specific factors, both with and without controlling for the general factor. Thus, the case of sex differences is not one in which substantive conclusions would be strongly affected by the failure to control for general factor variance. The most notable differences were in the Attention Switching factor of the AQ50 and in the Routine, Switching, and Imagination factors of the AQ-S. Here, the sex difference was larger when controlling for the general factor. The most likely explanation is that controlling for the general factor revealed normative sex differences, which were otherwise conflated with the extent to which individuals exhibit general autistic-like tendencies.

One potential solution to confounding with the general or specific factors is to use a bi-factor measurement model for ASD inventories to obtain estimates of reliability and correlations with external criteria for the specific and general factors. Less ideal but also defensible is to use factor scores estimated from the bi-factor model that represent scores on specific symptoms controlling for general ASD variance, as well as general ASD scores controlling for specific symptoms (DeMars, 2013). These approaches may be useful in the context of testing theories, which predict a correlation between a specific ASD symptom and some external variable. Such specific theoretical mappings have, for example, been drawn between possible emotion recognition deficits and social symptoms (Dawson, Webb, & McPartland, 2005) and between restrictive repetitive activities and frontal lobe functions (Lopez, Lincoln, Ozonoff, & Lai, 2005). These associations will be more difficult to assess when items reflect not only the specific symptom with which the hypothesis is concerned but also inventory-wide variance. In this situation, it will not be possible to be sure that an observed correlation is due only to the specific symptom of interest. Similarly, when a negative association between a specific symptom and some external criterion is predicted but the association with the general factor of ASD is positive, that negative association may be masked.

In some contexts, it will not be necessary or appropriate to attempt to separate out general and specific ASD variance. One example is when the test is being used for prediction (Revelle & Zinbarg, 2009). For example, when screening for or diagnosing ASD (i.e., predicting ASD status) where the goal is simply to identify an individual with overall high levels of ASD symptoms, it may matter little if item scores reflect both general and specific traits. Here, an amalgamated estimate will generally identify individuals who may meet diagnostic criteria for ASD irrespective of the factor structure thought to be the best representation of the inventory. If the underlying factor structure is not relevant, the AQ-S total score may be preferred over the full AQ50, because, in spite of being 22 items longer, the AQ50 had a total score reliability that was only marginally higher than that of the AQ-S. Similarly, the Numbers/Patterns items could be omitted without adverse impact on total score reliability. Thus, participant fatigue can be reduced by administering a briefer inventory with little detriment to the reliability achieved.

As noted above, there were some exceptions to the overall tendency for the general factor to dominate item variance. In the AQ50, the Attention to Detail items were an exception to this trend, as were the Numbers/Patterns items in the AQ-S. In fact, these are essentially the same items. That these items reflected a large proportion of shared variance independent of the general factor is consistent with previous research that has suggested that the items measure a construct relatively distinct from the other constructs captured by the items of the AQ (e.g., Hoekstra et al., 2011; Stewart & Austin, 2009). For example, Hoekstra et al. (2011) found that it was possible to include a higher order factor in a CFA model of the AQ-S, but the Numbers/Patterns did not fit within this factor, and the factor correlation between the Numbers/Patterns factor and the higher order ASD factor was only .2. Thus, evidence is accumulating that the items measured by the Numbers/Patterns factor reflect a relatively distinct attribute that may not represent a core feature of ASD.

Limitations and Future Directions

In terms of study limitations, while we were able to include individuals with a very broad range of ASD traits levels (from clinically diagnosed to nonclinical levels), our sample was not population representative. Individuals with low intellectual functioning were not represented in the current sample, as the AQ was designed for individuals of normal intellectual ability. In addition, we had only self-report measures, which is suboptimal, given the possibility that the autistic traits may be associated with accuracy of self-reports (Johnson, Filliter, & Murphy, 2009).

In addition, our methodology was unable to probe the cause of inventory-wide variance. While it could reflect a causal general ASD factor, it could also reflect local interactions between symptoms or a range of other causal structures that are statistically indistinguishable (e.g., see van der Maas et al., 2006). It is conceivable that common method variance was at least partly responsible for the observation that items tended to reflect a general ASD factor more than specific factors. That is, the general ASD variance could, to some extent, have a methodological root rather than a substantive root, and in fact, the bi-factor model has sometimes been recommended as a means of partialling out “nuisance” or “method” variance common to items in an inventory (Maydeu-Olivares & Coffman, 2006). One mitigating factor is that items of the AQ50 and AQ-S are keyed in both a forward and reverse direction, limiting the amount of common variance due to individual differences in acquiescent response styles. Nonetheless, there remain other sources of common variance of nonsubstantive origin. These could include, for example, common variance due to implicit theories about autistic behaviors tending to go together which could result in a respondent who holds such an implicit theory answering items more similarly than is merited based on their actual behavior (e.g., Lahey et al., 2015). Similarly, it could reflect individual differences in the tendency to portray oneself in a socially desirable manner (e.g., Lahey et al., 2012). Finally, it could reflect context effects of other items, whereby responses to previous items are used as a source of information in constructing answers to subsequent items, inflating their similarity (Harrison, McLaughlin, & Coalter, 1996). For this reason, it is also important to note that results obtained using the AQ-S could have been affected by the fact that the 28 items were completed in the context of the full 50-item AQ, not in isolation. An important future direction will be to determine why such a strong general factor is found in the AQ50 and AQ-S and, in particular, the extent to which it reflects various influences such as measurement artefacts, sets of shared etiological factors, and local interactions between different symptoms.

Finally, it should be noted that the strength of the general factor in an inventory is inexorably linked to the breadth of behaviors and symptoms covered by the set of items. A strong general factor will tend to be in evidence when items are all very similar to one another (e.g., when some items are mere paraphrases) and measure a narrow construct. The AQ50 and AQ-S are not exhaustive in the features of ASD that they cover, and they focus on trait-like behaviors; however, within this, they appear to be reasonably diverse in content and don’t include many obviously highly redundant items that could inflate item intercorrelations. In terms of the most important areas not represented in the AQ, items focusing on stereotyped behaviors and other features commonly associated with individuals of a low level of functioning are generally absent. Their exclusion is a result of the deliberate targeting of the AQ and AQ-S to individuals of normal intellectual functioning (Baron-Cohen et al., 2001); however, it does limit the relevance of the inventories to the large proportion of individuals with ASD who would be classified as low functioning.

Conclusion

With the exception of the Attention to Details and Numbers/Patterns factor of the AQ50 and AQ-S, respectively, the items of both inventories appear to primarily reflect a general ASD attribute rather than specific symptoms. This suggests that caution is due when attempting to estimate the relation of a specific symptom with some external criterion that may, as a result of this inventory-wide shared variance, be confounded. In some circumstances, when an association with a specific symptom is of interest, a bi-factor measurement model can be used to separate out general and specific factor variance.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Baron-Cohen

Wheelwright

Skinner

Martin

Clubley

(2001). The autism-spectrum quotient (AQ): Evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31, 5-17.

Bauman

M. L.

(2010). Medical comorbidities in autism: Challenges to diagnosis and treatment. Neurotherapeutics, 7, 320-327.

Ben-Sasson

Hen

Fluss

Cermak

S. A.

Engel-Yeger

Gal

(2009). A meta-analysis of sensory modulation symptoms in individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, 1-11.

Booth

Murray

A. L.

McKenzie

Kuenssberg

O’Donnell

Burnett

(2013). Brief report: An evaluation of the AQ-10 as a brief screening instrument for ASD in adults. Journal of Autism and Developmental Disorders, 43, 2997-3000.

Braeken

(2011). A boundary mixture approach to violations of conditional independence. Psychometrika, 76, 57-76.

Buck

T. R.

Viskochil

Farley

Coon

McMahon

W. M.

Morgan

Bilder

D. A.

(2014). Psychiatric comorbidity and medication use in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44, 3063-3071.

Burke

J. D.

Boylan

Rowe

Duku

Stepp

S. D.

Hipwell

A. E.

Waldman

I. D.

(2014). Identifying the irritability dimension of ODD: Application of a modified bifactor model across five large community samples of children. Journal of Abnormal Psychology, 123, 841-851.

10.

Caspi

Houts

R. M.

Belsky

D. W.

Goldman-Mellor

S. J.

Harrington

Israel

. . . Moffitt

T. E.

(2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2, 119-137.

11.

Dawson

Webb

S. J.

McPartland

(2005). Understanding the nature of face processing impairment in autism: Insights from behavioral and electrophysiological studies. Developmental Neuropsychology, 27, 403-424.

12.

DeMars

C. E.

(2013). A tutorial on interpreting bifactor model scores. International Journal of Testing, 13, 354-378.

13.

Gadow

K. D.

Sprafkin

(2000). Early Childhood Inventory-4 screening manual. Stony Brook, NY: Checkmate Plus.

14.

Gillberg

Råstam

Wentz

(2001). The Asperger syndrome (and high-functioning autism) Diagnostic Interview (ASDI): A preliminary study of a new structured clinical interview. Autism, 5, 57-66.

15.

Haberman

S. J.

(2008). When can subscores have value?. Journal of Educational and Behavioral Statistics, 33, 204-229.

16.

Happé

Booth

Charlton

Hughes

(2006). Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder: Examining profiles across domains and ages. Brain and Cognition, 61, 25-39.

17.

Happé

Frith

(2006). The weak coherence account: Detail-focused cognitive style in autism spectrum disorders. Journal of Autism and Developmental Disorders, 36, 5-25.

18.

Harrison

D. A.

McLaughlin

M. E.

Coalter

T. M.

(1996). Context, cognition, and common method variance: Psychometric and verbal protocol evidence. Organizational Behavior and Human Decision Processes, 68, 246-261.

19.

Hoekstra

R. A.

Vinkhuyzen

A. A.

Wheelwright

Bartels

Boomsma

D. I.

Baron-Cohen

. . . van der Sluis

(2011). The construction and validation of an abridged version of the autism-spectrum quotient (AQ-Short). Journal of Autism and Developmental Disorders, 41, 589-596.

20.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.

21.

Johnson

S. A.

Filliter

J. H.

Murphy

R. R.

(2009). Discrepancies between self- and parent-perceptions of autistic traits and empathy in high functioning children and adolescents on the autism spectrum. Journal of Autism and Developmental Disorders, 39, 1706-1714.

22.

Kuenssberg

Murray

A. L.

Booth

McKenzie

(2014). Structural validation of the abridged Autism Spectrum Quotient–Short Form in a clinical sample of people with autism spectrum disorders. Autism, 18, 69-75.

23.

Lahey

B. B.

Applegate

Hakes

J. K.

Zald

D. H.

Hariri

A. R.

Rathouz

P. J.

(2012). Is there a general factor of prevalent psychopathology during adulthood? Journal of Abnormal Psychology, 121, 971-977.

24.

Lahey

B. B.

Rathouz

P. J.

Keenan

Stepp

S. D.

Loeber

Hipwell

A. E.

(2015). Criterion validity of the general factor of psychopathology in a prospective study of girls. Journal of Child Psychology and Psychiatry, 56, 415-422.

25.

Lecavalier

Gadow

K. D.

DeVincent

C. J.

Houts

Edwards

M. C.

(2009). Deconstructing the PDD clinical phenotype: Internal validity of the DSM-IV. Journal of Child Psychology and Psychiatry, 50, 1246-1254.

26.

Lopez

B. R.

Lincoln

A. J.

Ozonoff

Lai

(2005). Examining the relationship between executive functions and restricted, repetitive symptoms of autistic disorder. Journal of Autism and Developmental Disorders, 35, 445-460.

27.

Lord

Rutter

Le Couteur

(1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659-685.

28.

Mandy

Charman

Puura

Skuse

(2014). Investigating the cross-cultural validity of DSM-5 autism spectrum disorder: Evidence from Finnish and UK samples. Autism, 18, 45-54.

29.

Matson

J. L.

Shoemaker

(2009). Intellectual disability and its relationship to autism spectrum disorders. Research in Developmental Disabilities, 30, 1107-1114.

30.

Maydeu-Olivares

Coffman

D. L.

(2006). Random intercept item factor analysis. Psychological Methods, 11, 344-362.

31.

McDonald

R. P.

(1999). Test theory: A uniﬁed treatment. Mahwah, NJ: LEA.

32.

Murray

A. L.

Booth

McKenzie

Kuenssberg

O’Donnell

(2014). Are autistic traits measured equivalently in individuals with and without an autism spectrum disorder? An invariance analysis of the Autism Spectrum Quotient Short Form. Journal of Autism and Developmental Disorders, 44, 55-64.

33.

Murray

A. L.

Johnson

(2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41, 407-422.

34.

Murray

A. L.

McKenzie

Kuenssberg

O’Donnell

(2014). Are we under-estimating the association between autism symptoms? The importance of considering simultaneous selection when using samples of individuals who meet diagnostic criteria for an autism spectrum disorder. Journal of Autism and Developmental Disorders, 44, 2921-2930.

35.

Muthén

L. K.

Muthén

B. O.

(1998-2013). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

36.

Posserud

M. B.

Breivik

Gillberg

Lundervold

A. J.

(2013). ASSERT—The autism symptom self-report for adolescents and adults: Bifactor analysis and validation in a large adolescent population. Research in Developmental Disabilities, 34, 4495-4503.

37.

R Core Team (2014). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/

38.

Reise

S. P.

(2012). Invited paper: The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696.

39.

Reise

S. P.

Bonifay

W. E.

Haviland

M. G.

(2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140.

40.

Reise

S. P.

Moore

T. M.

Haviland

M. G.

(2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559.

41.

Reise

S. P.

Morizot

Hays

R. D.

(2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19-31.

42.

Revelle

(1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14, 57-74.

43.

Revelle

Zinbarg

R. E.

(2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145-154.

44.

Revelle

. (2015) psych: Procedures for personality and psychological research. Evanston, IL: Northwestern University. Retrieved from https://cran.r-project.org/web/packages/psych/index.html

45.

Robitzsch

. (2015). sirt: Supplementary item response theory models. Retrieved from http://CRAN.R-project.org/package=sirt

46.

Russell-Smith

S. N.

Maybery

M. T.

Bayliss

D. M.

(2011). Relationships between autistic-like and schizotypy traits: An analysis using the Autism Spectrum Quotient and Oxford-Liverpool Inventory of Feelings and Experiences. Personality and Individual Differences, 51, 128-132.

47.

Schermelleh-Engel

Moosbrugger

Muller

(2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research, 8, 23-74.

48.

Simms

L. J.

Grös

D. F.

Watson

O’Hara

M. W.

(2008). Parsing the general and specific components of depression and anxiety with bifactor modeling. Depression and Anxiety, 25, E34-E46.

49.

Snow

A. V.

Lecavalier

Houts

(2009). The structure of the Autism Diagnostic Interview-Revised: Diagnostic and phenotypic implications. Journal of Child Psychology and Psychiatry, 50, 734-742.

50.

Stewart

M. E.

Austin

E. J.

(2009). The structure of the Autism-Spectrum Quotient (AQ): Evidence from a student sample in Scotland. Personality and Individual Differences, 47, 224-228.

51.

van der Maas

H. L.

Dolan

C. V.

Grasman

R. P.

Wicherts

J. M.

Huizenga

H. M.

Raijmakers

M. E

. (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 113, 842-861.

52.

Watkins

M. W.

Canivez

G. L.

James

Good

(2013). Construct validity of the WISC–IVUK with a large referred Irish sample. International Journal of School & Educational Psychology, 1, 102-111.

53.

Watson

O’Hara

M. W.

Simms

L. J.

Kotov

Chmielewski

McDade-Montez

E. A.

. . . Stuart

(2007). Development and validation of the Inventory of Depression and Anxiety Symptoms (IDAS). Psychological Assessment, 19, 253-268.

54.

Wechsler

(2003). Wechsler intelligence scale for children (4th ed.). San Antonio, TX: Psychological Corporation.

55.

Williams

D. M.

Bowler

D. M.

(2014). Autism spectrum disorder: Fractionable or coherent? Autism, 18, 2-5.

56.

Willoughby

M. T.

Blanton

Z. E.

(2015). Replication and external validation of a bi-factor parameterization of attention deficit/hyperactivity symptomatology. Journal of Clinical Child & Adolescent Psychology, 44, 68-79.