The SWAN Captures Variance at the Negative and Positive Ends of the ADHD Symptom Dimension

Abstract

Objective: The Strengths and Weaknesses of ADHD Symptoms and Normal Behavior (SWAN) Rating Scale differs from previous parent reports of ADHD in that it was designed to also measure variability at the positive end of the symptom spectrum. Method: The psychometric properties of the SWAN were tested and compared with an established measure of ADHD, the Disruptive Behavior Rating Scale (DBRS). Results: The SWAN demonstrates comparable validity, reliability, and heritability to the DBRS. Furthermore, plots of the SWAN and DBRS reveal heteroscedasticity, which supports the SWAN as a preferred measure of positive attention and impulse regulation behaviors. Conclusion: The ability of the SWAN to measure additional variance at the adaptive end of the ADHD symptom dimensions makes it a promising tool for behavioral genetic studies of ADHD.

Keywords

ADHD psychometrics measurement invariance

The Strengths and Weaknesses of ADHD Symptoms and Normal Behavior (SWAN; Swanson et al., 2006) Rating Scale is a relatively new parent-report measure of symptoms associated with ADHD. Unlike other established ADHD questionnaires, such as the Disruptive Behavior Rating Scale (DBRS; Barkley & Murphy, 1998), the SWAN was constructed with the intent of capturing variability at the good ends of attention and activity regulation as well as at the symptomatic levels of these dimensions. For example, the SWAN asks how well the child “listens when spoken to” (attention) or “awaits turn” (activity regulation).

The overall goals of this study were to evaluate the psychometric properties of the SWAN and compare it with the DBRS, because the SWAN could be useful in research aimed at understanding strengths as well as weaknesses in attention and impulse control. Contemporary behavioral genetic and phenotypic studies of ADHD conceptualize the diagnosis as an extreme phenotype that is not etiologically distinct from the spectrum of symptoms that fall within the normal range (Pennington, 2002). However, it is possible that the spectrum of symptoms that are behaviorally adaptive does not share a common etiology with the disorder. Measuring variance in both the positive as well as the negative ends of the symptom dimensions can address this question.

Background

ADHD is one of the most common neurodevelopmental disorders, affecting approximately 4% of 4- to 8-year-olds in the general population (Centers for Disease Control, 2005). Current diagnostic guidelines include two correlated symptom dimensions of inattention (I) and hyperactivity/impulsivity (HI). A child with ADHD may be elevated on one or both symptom dimensions, resulting in three subtypes of ADHD: inattentive, hyperactive-impulsive, and combined. To meet diagnostic criteria for ADHD, symptoms must be present before age 7 and occur in more than two settings, be inconsistent with developmental stages, and cause impairments in adaptive functioning (American Psychiatric Association [APA], 1994, 2000). Parent- and teacher-report measures, such as the DBRS, typically show internal and external validity for symptoms of inattention and hyperactivity/impulsivity (e.g., Bussing et al., 2008; Friedman-Weieneth, Doctoroff, Harvey, & Goldstein, 2009). However, these same scales do not capture variation at the positive end of the two symptom dimensions (Polderman et al., 2007). Studies of genetic etiologies for ADHD have found that heritability for these symptoms is consistent across the continuum of severity (Stevenson et al., 2005). However, without a scale that measures variance at the adaptive end of the phenotype, it is difficult to estimate the degree to which these genetic factors are shared at both ends of the symptom dimensions.

The SWAN Scale (Swanson et al., 2006) was developed with the goal of detecting variance at the negative and adaptive ends of the two symptom dimensions. The SWAN differs from other ADHD measures in its balanced Likert-type scale that includes positively phrased, Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) anchored items (see instrument descriptions below). However, despite this apparent face validity, few studies have examined its psychometric properties, particularly whether the SWAN truly captures meaningful variance at the favorable end of the inattention_and_HI distributions. Polderman et al. (2007) plotted SWAN scores with scores on the Attention Problems scale of the Child Behavior Checklist (CBCL; Achenbach, 1991) to test for heteroscedasticity or a difference in variances between the two scales. Whereas the SWAN showed variance at the positive ends of inattention and HI levels, CBCL “positive” scores were truncated at 0. However, this study did not test the external validity of the SWAN scores. Moreover, unlike the SWAN, the CBCL ADHD Scale comprises 11 items that do not map directly onto DSM-IV ADHD criteria, but instead tap global symptoms of inattention and hyperactivity/impulsivity. For example, Item 10 on the CBCL reads, “Can’t sit still, restless, or hyperactive,” whereas the DSM-IV criteria and the SWAN identify nine specific behaviors that fall under the umbrella term hyperactive. So, for both these reasons, the Polderman et al. results are not conclusive.

Young, Levy, Martin, and Hay (2009) found that the SWAN identified a distribution of ADHD subtypes consistent with the Rasch model, suggesting consistency in the relative predictive strength of individual items on the measure; however, the authors did not correct for inflation due to familial correlations. Finally, Hay, Bennett, Levy, Sergeant, and Swanson (2007) compared the SWAN with ADHD items on the DSM-IV-oriented Australian Twin Behavior Rating Scale in a group of Australian twins, aged 6 to 17 years. This study confirmed that the SWAN provides a more normal distribution as compared with the skew and kurtosis found in a 0 to 3 rating scale similar to the DBRS, but other validity tests were not conducted.

In studies of genetic and environmental influences on inattention and impulse control, the ability to measure individual differences across the whole distribution of these symptom dimensions would be useful. Thus, the goal of the current study is to examine the psychometric properties of the SWAN to determine whether it can be useful in future research. Specifically, the goals of the study were to examine the following:

internal validity of the SWAN as measured by internal consistency and test–retest reliability, exploratory factor analyses, and tests of normality;

convergent validity of the SWAN with the DBRS at the symptom severity level, via sensitivity/specificity to identify clinically elevated children, and by estimating heritability for symptoms of ADHD; and

external validity of the SWAN, as compared with the DBRS, as measured by phenotypic and genetic correlations with a cognitive latent factor and ratings of interference with daily functioning.

Method

Participants

Participants in this study were 379 monozygotic (MZ) and 372 same-sex dizygotic (DZ) twin pairs (1,502 participants in total) from the Colorado Twin Registry in the United States (65%) and the National Health and Medical Research Council’s Australian Twin Registry (35%). This unselected population sample is part of the International Longitudinal Twin Study (ILTS) of early reading and attention development (e.g., Samuelsson et al., 2005; Willcutt et al., 2007). This study is ongoing, and data are not currently available from Australia for the post-fourth-grade time point. At the time of the initial assessment with the SWAN, all participants had just completed kindergarten, with ages ranging from 64 to 92 months (M = 75.7 months). Mean years of the participants’ parents’ education was 14. Zygosity was confirmed with DNA collected via cheek swabs or, in a small number of cases, by selected items from the questionnaire by Nichols and Bilbro (1966).

Instruments

DBRS

The DBRS is a caregiver-report questionnaire comprising 18 items targeting symptoms of inattention and HI, and 8 items measuring conduct disorder, the latter of which were not included in this study. The inattention and HI items map onto the criteria for ADHD as outlined by the DSM-IV (APA, 1994). The DBRS is scored on a 4-point Likert-type scale, with anchors 0 = not at all, 1 = sometimes, 2 = often, and 3 = very often. The two-factor structure of the DBRS ADHD Scales (i.e., inattention and hyperactivity/impulsivity) was established by DuPaul, Anastopoulos, et al. (1998) using a normative sample of 4,666 participants ranging from 4 to 20 years old (M = 9.57). Subsequent studies have further supported this structure in preschool-aged children (e.g., Friedman-Weieneth et al., 2009). Adequate reliability, as well as convergent and discriminant validity of the DBRS, have been established for parent and teacher ratings (DuPaul, Power, McGoey, Ikeda, & Anastopoulos, 1998; Erford, 1997).

SWAN

The SWAN Scale is a more recently developed measure. The wording of the SWAN items differs from that of the DBRS in that the questions are phrased relative to normal behavior expectations, rather than with a particular focus on deficits. For example, Item 3 on the SWAN reads, “Compared to other children, how well does this child . . . Listen when spoken to directly?” whereas the corresponding DBRS item reads, “Does not seem to listen when spoken to directly.” The SWAN is rated on a balanced 7-point Likert-type scale, with anchors of 0 = far below, 1 = below, 2 = slightly below, 3 = average, 4 = slightly above, 5 = above, and 6 = far above. Despite frequent use of the SWAN in recent research (e.g., Cornish, Wilding, & Hollis, 2008), there exist few independent studies of the measure’s psychometric properties, including incremental ability to predict above-average cognitive performance.

Interference ratings

As part of the DBRS, parents also rated the degree to which symptoms of inattention and HI interfere with social interactions, academic performance, daily responsibilities, and recreational activities. These ratings were made on a 0 to 3 Likert-type scale identical to the DBRS and were completed concurrently with DBRS ratings.

Comprehensive Test of Phonological Processing (CTOPP)

The Rapid Naming subtests from the CTOPP (Wagner, Torgesen, & Rashotte, 1999) were used as measures of naming speed, which previous research has shown to load strongly on a general processing speed factor also defined by perceptual speed tasks that do not require naming (McGrath et al., 2011). Subtests administered at each time point were chosen according to developmental level. At post kindergarten, participants completed two Rapid Color Naming, two Rapid Digit Naming, and two Rapid Letter Naming tests. At post first and post fourth grades, participants completed two Rapid Digit and two Letter Naming subtests. Raw end times for individual subtests were log transformed and summed to create rapid naming composites, where higher values reflected worse performance.

Procedure

Procedures for the ILTS have been described in detail elsewhere (Samuelsson et al., 2005; Willcutt et al., 2007), so the following description focuses only on the assessments relevant to this article. Participants were assessed using rapid naming measures at the end of kindergarten and first grade in the Australian and U.S. samples, and also at the end of fourth grade in the U.S. sample. One-hour testing sessions were performed at home or at school in the United States, and at school in Australia. Parent ratings of ADHD symptoms were obtained at kindergarten, first, second, and fourth (the United States only) grades while individual testing was completed in the United States and via mail in Australia.

Analyses were performed with SPSS 18.0 and MPlus 6.0. Inattention, hyperactivity/impulsivity, and total ADHD composite scores were created, consistent with procedures used in previous studies of similar rating scales (e.g., Pelham, Gnagy, Greenslade, & Milich, 1992; Willcutt et al., 2007). On both measures, Items 1 to 9 are intended to correspond with symptoms of inattention; Items 10 to 18 tap symptoms of HI. SWAN items were reverse scored (i.e., far below = 7 and far above = 1) to match the directionality of DBRS symptomatic scores.

Results

Internal Validity

Internal consistency was estimated for both measures by calculating Cronbach’s alpha for scale and total scores across the four time points: post kindergarten, post first, post second, and post fourth grades. The DBRS Scales were found to demonstrate positively skewed distributions at all test occasions, with the majority of participants receiving low scores, consistent with previous research. Hence, these values were logarithm transformed prior to analyses (Willcutt et al., 2007). Mean Cronbach’s alpha for the SWAN Scales was nearly identical to that of the DBRS (α = .88).

Low-bound estimates of test–retest reliability for SWAN and DBRS subscales were calculated by correlating scale scores at each time point for MZ twins. As described in previous literature (Samuelsson et al., 2005), these coefficients are underestimates of the true reliability due to nonshared environmental factors that create differences between individuals in the twin pair. Nonetheless, both measures demonstrated good reliability, although the mean SWAN coefficient was significantly higher than the mean DBRS coefficient (p < .01). Reliability for the SWAN ranged from .72 to .90 (M = 0.82); DBRS coefficients ranged from .58 to .81 (M = 0.71). The Inattention Scale showed significantly lower test–retest reliability than the other scales for both the SWAN and DBRS at all time points except post fourth grade. Reliability did not vary significantly by time point for the SWAN; however, the post-fourth-grade reliability coefficients were significantly lower than those at post kindergarten for the DBRS.

Longitudinal stability was calculated for both measures by correlating scale and total scores across the four time points: post kindergarten, post first, post second, and post fourth grades. Pearson product–moment correlation coefficients for the SWAN Scales were acceptable and comparable with DBRS coefficients at all time intervals, and ranged from r = .57 to r = .75 (M = 0.67), p < .001; DBRS coefficients ranged from r = .51 to r = .76 (M = 0.66), p < .001. The DBRS coefficients were slightly lower than values previously published by DuPaul, Power, McGoey, Ikeda & Anastopoulos (1998), who found inattention r = .78, HI r = .86, and total r = .85. However, the time interval between reports in the DuPaul et al. study was only 4 weeks, while the current study averaged longitudinal stabilities across kindergarten and Grades 1, 2, and 4. To our knowledge, there are no published longitudinal stability values for the SWAN.

Factor analysis was performed using SPSS 18.0. A two-factor, maximum-likelihood extraction model with Varimax rotation was tested on SWAN and DBRS scores, with a random selection of one twin from each pair at post second grade (n = 532). Post-second-grade data were used because we expect symptoms of ADHD to become more stable with age and because only U.S. data were available at post fourth grade. Overall, the items loaded as expected for both measures, with Items 1 to 9 loading together as an inattention factor and Items 10 to 18 loading onto the hyperactivity/impulsivity factor (see Table 1). Items 10 and 11 (fidgets and leaves seat in classroom) cross-loaded in both models. The SWAN showed additional cross-loading of Items 3 (listens when spoken to) and 16 (reflects on questions, controls blurting out answer). On average, SWAN factor loadings were higher than DBRS loadings (SWAN M = 0.74; DBRS M = 0.63). The higher loading of SWAN items on the two factors would be consistent with it having a more differentiated Likert-type scale (7 points vs. 4 on the DBRS). Similarly, the two-factor model accounted for 74% of the variance in the SWAN and 64% of the variance in the DBRS. However, the DBRS showed a better fit with this model, χ² = 490, df = 118, compared with the SWAN, χ² = 866, df = 118.

Table 1.

Factor Loadings for Two-Factor Varimax Rotation Model

	DBRS		SWAN
	Inattention	HI	Inattention	HI
Attention to details	.639	.103	.743	.296
Sustaining attention	.675	.265	.766	.391
Listening when spoken to	.543	.381	.664	.495
Following through on instructions	.750	.171	.735	.371
Organizing tasks	.732	.183	.805	.287
Avoiding tasks requiring mental effort	.623	.094	.721	.290
Losing things necessary for tasks	.555	.257	.800	.291
Distractibility	.707	.294	.675	.375
Forgetfulness	.676	.256	.766	.277
Fidgeting	.436	.502	.433	.711
Remaining seated in class	.426	.468	.443	.721
Running, climbing	.310	.517	.345	.684
Playing quietly	.303	.621	.285	.829
On the go	.230	.619	.316	.853
Excessive talking	.057	.656	.192	.806
Waiting to answer questions	.089	.654	.437	.552
Waiting turn	.209	.717	.377	.734
Interrupting, intruding on others	.216	.701	.347	.688

Note: DBRS = Disruptive Behavior Rating Scale; SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale; HI = Hyperactivity/Impulsivity.

To test the difference in fit of the two-factor structure, we used MPlus Version 6.0 to run a confirmatory factor analysis with the default geomin rotation. Maximum likelihood factor analysis was performed, and the Inattention and HI Scales were allowed to correlate within measures. As in the exploratory Varimax rotation, the factor loadings on both the inattention and HI factors were significantly higher on the SWAN as compared with the DBRS (see Table 1). Constraining the path estimates of corresponding items to be equal across measures resulted in a significantly worse fit for both scales—Inattention: Δχ² = 45.83, Δdf = 8, p < .001; HI: Δχ² = 64.57, Δdf = 8, p < .001—confirming that the measures do not show weak factorial invariance. These results suggest that the variance at the favorable end of the SWAN item scores is being used in a meaningful way.

Histogram plots revealed evident skew and kurtosis for the DBRS Scales (Inattention skew: 0.95-1.13, kurtosis: 0.74-1.70; HI skew: 1.37-1.56, kurtosis: 2.25-2.55). SWAN plots were within the normal range (Inattention skew: –0.29-0.09, kurtosis: 0.05-0.65; HI skew: –0.33-0.04, kurtosis: –0.25-0.55). A bivariate scatterplot of SWAN and DBRS total Scale scores at Time 1 is shown in Figure 1 and is representative of plots at all four time points. As we would expect, the shape of the scatterplot demonstrates heteroscedasticity, meaning that there is more variance around the regression line at the low end of the DBRS Scale. Whereas the DBRS attributes scores of 0 to participants who are free of negative symptoms (x-axis), SWAN ratings of these same participants capture variability among their scores (y-axis). A child who demonstrates good attention and regulation of activity level will have a summed Total scale score of 0 on the DBRS. In contrast, this same child will receive scores between 1 and 4 (in our reverse-scored data set) for each item on the SWAN, and will thus have a Total scale score between 18 and 72. The graph shows us that the SWAN is more sensitive to individual differences at the good ends of the symptom phenotypes than is the DBRS.

Figure 1.

Heteroscedasticity of SWAN and DBRS Total scores at post kindergarten

To further test this point, we compared MZ and DZ correlations on ADHD subscale scores for probands scoring at the favorable versus unfavorable halves of the distributions (determined by a median split). Because we observed greater variance in the SWAN than in the DBRS at the favorable ends of the ADHD dimensions, we likewise expected to find greater differences between the MZ and DZ correlations in the SWAN in the favorable halves. In contrast, we expected comparable differences between the MZ and DZ correlations for the unfavorable halves. However, the results showed that at both the favorable and symptomatic halves of the dimensions, the SWAN had greater and more significant differences (Table 2).

Table 2.

Differences Between MZ and DZ Correlations by Median Split for ADHD Scales

	SWAN r_MZ − r_DZ	DBRS r_MZ − r_DZ
Post kindergarten
Good attention	.492	.432
Bad attention	.290	.221
Good HI	.283	.293
Bad HI	.421	.338
Good total	.412	.340
Bad total	.309	.214
Post first grade
Good attention	.420	.327
Bad attention	.479	.362
Good HI	.279	.098
Bad HI	.457	.503
Good total	.403	.231
Bad total	.464	.395
Post second grade
Good attention	.312	.383
Bad attention	.470	.295
Good HI	.232	.128
Bad HI	.772	.050
Good total	.245	.469
Bad total	.712	.319
Post fourth grade
Good attention	.438	.351
Bad attention	.516	.291
Good HI	.336	.249
Bad HI	.437	.354
Good total	.454	.094
Bad total	.554	.093
Mean
Good attention	.416	.373
Bad attention	.439	.292
Good HI	.283	.192
Bad HI	.522	.311
Good total	.379	.284
Bad total	.510	.255

Note: MZ = monozygotic; DZ = dizygotic; SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale; DBRS = Disruptive Behavior Rating Scale; HI = Hyperactivity/Impulsivity.

Convergent Validity

Currently, measures like the DBRS, which map directly onto the DSM-IV criteria for ADHD, are considered the gold standard in clinical assessment of ADHD. Thus, it is useful to test whether the SWAN Scale scores are related to the DBRS scores. The DBRS Scale distributions violated assumptions of normality, and the logarithm transformed DBRS Scale scores exhibited a curvilinear relationship with the SWAN Scale scores. Thus, rather than performing linear correlations, we first performed regression analyses using the log transformed DBRS values and a quadratic variable (log₁₀DBRS)², to determine the amount of variance shared by the models, as measured by R (Table 3). The SWAN and DBRS share a moderate amount of variance (M R = .57).

Table 3.

SWAN and DBRS Scale and Diagnostic Agreement

	Inattention	HI	Combined
Post kindergarten
R	.607	.481	.593
Cramer’s V scale agreement	.516	.475	.502
Cramer’s V diagnostic agreement	.379	.435	.563
Post first grade
R	.624	.476	.607
Cramer’s V scale agreement	.461	.475	.512
Cramer’s V diagnostic agreement	.433	.259	.427
Post second grade
R	.657	.475	.617
Cramer’s V scale agreement	.471	.509	.521
Cramer’s V diagnostic agreement	.297	.37	.367
Post fourth grade
R	.649	.454	.597
Cramer’s V scale agreement	.486	.469	.578
Cramer’s V diagnostic agreement	.256	.563	.456

Note: SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale; DBRS = Disruptive Behavior Rating Scale; HI = Hyperactive-Impulsive.

Both the SWAN and DBRS Scales are ordered categorical variables and, as such, do not have a defined metric. Therefore, to estimate the degree of correlation between the two measures, we used a measure of association (Cramer’s V) that is analogous to a Pearson correlation but does not assume a linear relationship. Like a correlation, Cramer’s V ranges from −1 to 1, with a value of 0 indicating no association and a value of 1 indicating a perfect association. To calculate Cramer’s V, we first computed the chi-square for the difference between the expected number of individuals at each level of the SWAN and DBRS and the observed number of individuals in a contingency table. Then, we entered that value into Equation 1:

Cramer's V = SQRT (χ^{2} / (n (k - 1))),

where n equals the number of participants and k equals the number of rows or columns, whichever is smaller.

Cramer’s V statistics were calculated at concurrent time points to estimate the association between the two measures (Table 3). Total scale scores consistently showed the strongest relationship (M Cramer’s V = 0.53), and Inattention and HI Scales demonstrated moderate cross-measure associations (M = 0.48).

Next, we truncated the SWAN item-level scores to match the DBRS Scales. So, using our reverse-scored data set, a SWAN score of 1 to 4 was coded as a 0 in the truncated variable, 5 was coded as 1, 6 as 2, and 7 as 3. Because both scales now violated the assumption of normality, we again used Cramer’s V to estimate the agreement between the composite scale scores at each time point. Cramer’s V values were similar to those achieved using the nontruncated SWAN scores: M total V = 0.56, M inattention V = 0.50, M HI V = 0.50.

Between-measure diagnostic agreement for ADHD subtypes was also calculated using Cramer’s V statistic. In accordance with DSM-IV diagnostic guidelines, participants were classified as ADHD-inattentive type if six or more items on the Inattentive Scale were endorsed, hyperactive type if six or more HI items were endorsed, and combined type if six or more items on each scale were endorsed. An item was considered “endorsed” on the DBRS if it was scored as a 2 (often) or 3 (very often). As mentioned previously, SWAN scores were reversed to match directionality of the DBRS. On the SWAN, items were considered endorsed for scores of 5 (slightly below), 6 (below), or 7 (far below). We found slightly improved categorical agreement when we counted these three ratings as endorsements on the SWAN, as opposed to only counting the two highest ratings, as is done on the DBRS. Cramer’s V statistics for categorical comparisons across measures were similar to the continuous values: M inattentive = 0.34, M hyperactive-impulsive = 0.41, M combined = 0.45 (Table 3). Cramer’s V agreement for any ADHD diagnosis was 0.50.

Another way to examine categorical correlations is to calculate sensitivity and specificity for the SWAN, using the DBRS as the gold standard. For detection of all ADHD subtypes, the SWAN showed moderate sensitivity (M = 58%) and excellent specificity (M = 98%). However, it should be noted that given the imperfect reliability of the DBRS in this sample (categorical test–retest reliability ∼.60), it is both a mathematical and theoretical fallacy to assume that the DBRS is in fact the “gold standard.” Error in both measures will lower the apparent sensitivity of the SWAN, and with this in mind, we consider the observed sensitivity values to be acceptable, overall. Table 4 reports the number of individual cases identified by both measures, the SWAN only, and the DBRS only at each time point. Overall, the SWAN identified more cases than did the DBRS, and raw agreement rates were comparable at each time point.

Table 4.

Case Agreement Between SWAN and DBRS as a Function of Subtype and Grade Level

	ADHD-I	ADHD-HI	ADHD-C	Any ADHD
Post kindergarten
Both	4	7	3	16
SWAN only	11	8	4	21
DBRS only	3	9	1	11
Post first grade
Both	6	3	4	18
SWAN only	12	12	10	29
DBRS only	4	5	2	6
Post second grade
Both	5	5	3	22
SWAN only	14	7	13	25
DBRS only	8	9	1	9
Post fourth grade
Both	4	4	3	16
SWAN only	18	3	11	27
DBRS only	5	3	0	3
Mean
Both	5	5	3	18
SWAN only	14	8	10	26
DBRS only	5	7	1	7

Note: SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale; DBRS = Disruptive Behavior Rating Scale; HI = Hyperactive-Impulsive; C = combined.

Heritability

It is extensively documented that symptoms of inattention and HI are heritable, with one meta-analysis of 20 twin studies finding a mean heritability of 0.76 for total ADHD scores (Faraone, Spencer, Aleardi, Pagano, & Biederman, 2004). Thus, testing whether SWAN scores are heritable and genetically correlated with DBRS scores are two important convergent validity tests. We used univariate ACE models to estimate genetic (A), shared environment (C), and nonshared environment (E) factors for DBRS and SWAN separately for each grade level using Mx (Neale, Boker, Xie, & Maes, 2006). Likelihood-based 95% confidence intervals were used to estimate the significance of genetic and environmental estimates, with the inclusion of 0 signifying a nonsignificant factor. Nonsignificant paths were then dropped from the model, and difference in chi-squares was used to confirm a nonsignificant loss in model fit. As before, the DBRS measures were log transformed to minimize skew. The effects of age (age and age squared) and gender were regressed out prior to genetic analyses, and the residuals were used.

We used an ACE correlated factors model. Although statistically equivalent to a Cholesky decomposition, the latent A, C, and E factors in this model are linked by a correlation r rather than common paths as in a Cholesky decomposition. This model was used to investigate the degree to which shared risk factors account for the comorbidity between SWAN and DBRS. The correlation r measures the degree of shared risk. Correlations can range from −1 to 1, indicating the extent to which the same genes influence both traits irrespective of their specific heritabilities. Similarly, this model allowed us to test the extent to which shared and nonshared environmental factors are correlated between these measures. Nonsignificant factors were dropped from the model and the difference in chi-squares was measured to confirm a nonsignificant loss in model fit. The results from a reduced AE model are presented in Table 5 because shared environment could be dropped without a significant loss in fit across all grades and measures. Heritabilities for SWAN were similar across subtypes and grade levels, ranging from .69 to .89. Moreover, these estimates are similar to previous findings for ADHD scores.

Table 5.

Univariate Estimates of A and E and Genetic and Environmental Correlations as a Function of Grade Level

Grade	Component	Inattention	Hyperactivity-impulsivity	Total score
SWAN univariate values
Post kindergarten	A	.75 (.69-.79)	.84 (.80-.87)	.83 (.79-.86)
	E	.25 (.21-.32)	.16 (.13-.20)	.17 (.14-.21)
Post first grade	A	.80 (.75-.84)	.89 (.87-.91)	.89 (.87-.91)
	E	.20 (.16-.25)	.11 (.09-.13)	.11 (.09-.13)
Post second grade	A	.77 (.72-.81)	.89 (.86-.91)	.85 (.82-.88)
	E	.23 (.19-.28)	.11 (.09-.14)	.15 (.12-.19)
Post fourth grade	A	.69 (.59-.76)	.86 (.82-.89)	.82 (.77-.86)
	E	.31 (.24-.41)	.14 (.11-.18)	.18 (.14-.28)
DBRS univariate values
Post kindergarten	A	.71 (.65-.76)	.79 (.74-.82)	.81 (.78-.84)
	E	.29 (.24-.35)	.21 (.18-.26)	.19 (.16-.22)
Post first grade	A	.64 (.60-.70)	.79 (.75-.83)	.77 (.72-.81)
	E	.36 (.30-.43)	.21 (.17-.25)	.23 (.19-.28)
Post second grade	A	.57 (.49-.64)	.77 (.71-.81)	.71 (.65-.76)
	E	.43 (.36-.52)	.23 (.19-.29)	.29 (.24-.35)
Post fourth grade	A	.52 (.39-.61)	.70 (.63-.76)	.65 (.57-.72)
	E	.49 (.39-.61)	.30 (.24-.37)	.35 (.28-.43)
SWAN and DBRS common variance
Post kindergarten	A	.53 (.47-.59)	.56 (.49-.62)	.54 (.48-.56)
	E	.50 (.41-.58)	.66 (.58-.72)	.61 (.53-.68)
Post first grade	A	.56 (.50-.61)	.60 (.53-.65)	.56 (.57-.71)
	E	.42 (.32-.52)	.67 (.60-.74)	.59 (.50-.66)
Post second grade	A	.50 (.43-.56)	.60 (.53-.66)	.53 (.46-.59)
	E	.57 (.47-.65)	.69 (.62-.75)	.65 (.57-.71)
Post fourth grade	A	.48 (.40-.55)	.54 (.43-.62)	.49 (.40-.57)
	E	.64 (.53-.72)	.72 (.67-.80)	.73 (.65-.79)

Note: DBRS = Disruptive Behavior Rating Scale; HI = Hyperactivity/Impulsivity; SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale. 95% confidence interval in parentheses. Across all measures and time points, shared environment in the univariate models could be dropped from the model without a significant loss in fit.

Univariate A and E estimates for the DBRS from best-fitting models at each grade level are presented in Table 5. Heritabilities were similar across subtypes and grade levels, ranging from .52 to .79. These estimates are similar to findings for SWAN scores.

Bivariate heritabilities (h² _xy ) are presented at the bottom of Table 5, which were moderate across time and type (i.e., inattention, hyperactivity-impulsivity), ranging from .48 to .60. Nonshared environment correlations were moderate to large across time and subscale, ranging from .42 to .72.

Finally, we calculated the proportion of variance in the phenotypic correlations due to shared genetic factors (pg) across grade levels. The formula for the pg correlations for the SWAN and DBRS Scales is as follows:

p g_{x y} = {h^{2}}_{x y} / p_{x},

where h² _xy represents the bivariate heritability for the SWAN (x) and DBRS (y) and p_xy represents the phenotypic correlation between the two measures. For each subscale, pg averaged across time points ranged from .78 to .90. The results show that genetic factors shared by the SWAN and DBRS account for a large amount of variance in the phenotypic correlation of the two measures.

External Validity

Of primary interest in this study is whether the SWAN offers improved ability over DSM-IV-based ADHD measures to measure the full range of behavior, including performance at the good ends of the two ADHD symptom dimensions. Thus, our analyses of external validity focused on testing whether the SWAN offers incremental external validity over the DBRS for predicting good cognitive performance and less behavioral interference.

To answer these questions, we performed linear regressions to determine whether the SWAN could predict cognitive performance and behavioral interference over and above the DBRS. We tested these regressions both predictively and concurrently. Using the “step” option in SPSS 6.0, DBRS Inattention was entered first, followed by DBRS HI, and then each SWAN Scale, one at a time. Table 6 shows the results of Rapid Naming scores at post first grade regressed on the postkindergarten ADHD Scales. Adding the SWAN Inattention Scale resulted in a significant R² change (.068), and Rapid Naming scores were best predicted by SWAN Inattention Scale scores (β = .34) when all predictors were included in the model. The results of rapid naming regressions at post fourth grade (where only data from the United States are available) were similar and therefore not reported in detail here.

Table 6.

Hierarchical Regressions of External Validators on Postkindergarten ADHD Subscales

		Standardized β	t	p	R
Model: Post-first-grade rapid naming on postkindergarten ADHD
1	DBRS inattention	.234	5.597	.000	.234
2	DBRS inattention	.274	5.513	.000
	DBRS HI	−.074	−1.481	.139	.242
3	DBRS inattention	.075	1.312	.190
	DBRS HI	−.060	−1.255	.210
	SWAN inattention	.324	6.472	.000	.356
4	DBRS inattention	.070	1.179	.239
	DBRS HI	−.049	−0.825	.410
	SWAN inattention	.340	4.961	.000
	SWAN HI	−.023	−0.340	.734	.356
Model: Post-second-grade interference on postkindergarten ADHD
1	DBRS inattention	.265	4.876	.000	.265
2	DBRS inattention	.117	1.848	.066
	DBRS HI	.266	4.198	.000	.345
3	DBRS inattention	.083	1.077	.282
	DBRS HI	.270	4.238	.000
	SWAN inattention	.051	0.760	.448	.348
4	DBRS inattention	.115	1.419	.157
	DBRS HI	.209	2.687	.008
	SWAN inattention	−.031	−0.341	.733
	SWAN HI	.119	1.338	.182	.355

Note: DBRS = Disruptive Behavior Rating Scale; HI = Hyperactivity/Impulsivity; SWAN = Strengths and Weakness of ADHD and Normal Behavior Scale.

In contrast, hierarchical regressions of post-second-grade interference scores on ADHD subscales revealed that the SWAN Scales explained very little variance in behavioral functioning when DBRS scores had already been entered (ΔR² = .005). We chose to use post-second-grade data for this analysis as it represented the longest, more stringent predictive time interval where both U.S. and Australia data were available. The DBRS HI Scale was the best predictor of behavioral functioning (β = .209) when all four variables were included in the model (Table 6).

Finally, post-fourth-grade Rapid Naming (RAN) scores were regressed onto DBRS and truncated SWAN Scale scores separately, to compare the amount of variance accounted for by each measure. The truncated SWAN Scale scores accounted for more variance than did the DBRS scores, with a mean R² value of .056 for the SWAN total score and .037 for the DBRS.

Discussion

We examined the psychometric properties of a new measure of ADHD symptoms, the SWAN Rating Scale. The SWAN demonstrates internal validity comparable with the DBRS, as measured by internal consistency, longitudinal stability, and factor structure. The similar factor structures also provide evidence of convergent validity between the SWAN and DBRS, in that the two questionnaires are measuring overlapping constructs. Furthermore, the significantly higher factor loadings on the SWAN indicate that the variance measured by this scale at the good ends of inattention and HI is related to the original two factors, and thus meaningful.

All tests of convergent validity indicate that the SWAN and DBRS are measuring similar constructs. Linear regressions established a moderate amount of shared variance between the SWAN and DBRS subscales. Continuous and categorical agreement between the measures was estimated using Cramer’s V statistic and found to be moderate. One possible confound in comparing the measures is that the SWAN has a longer Likert-type scale. With extended Likert-type scales, reporters may be less likely to endorse extreme scores. However, Table 4 demonstrates that, on average, the SWAN identified more cases than did the DBRS, so it does not appear that this potential confound was a factor in this sample. Although agreement between the two measures is not perfect, we expected the SWAN to identify different cases than the DBRS because the measures are in fact partly independent, and both are further limited by their own imperfect reliabilities. Furthermore, categorical correlations are always expected to be lower than continuous associations. Taken together, these tests support the use of the SWAN for clinical purposes as well as research in the diagnosis of ADHD subtypes.

Bivariate heritability analyses demonstrated that shared genetic influences account for a large proportion of the phenotypic correlation between the measures. However, the genetic correlation is lower than we expected, and it is not due to a difference in genetic influences specifically at the favorable ends of the scales. Rather, the results suggest that the SWAN is measuring some genetic factors that are distinct from those measured by the DBRS on both ends of the dimensions. This supports the SWAN as a distinct and useful measure of variance not captured by the DBRS.

Our primary question in this study was whether the SWAN has additional information about children performing at the favorable ends of inattention and HI. As predicted, the SWAN does have variance at the positive ends of Inattention, HI, and Combined subscales, as demonstrated by heteroscedasticity with the DBRS Scale. Furthermore, hierarchical regressions of a measure of cognitive functioning showed the SWAN Inattention Scale to be the only independent predictor of cognitive performance, indicating that this subscale in particular is capturing meaningful variance over and above those of the DBRS. Finally, the differences between MZ and DZ correlations were greater for the SWAN at both ends of the symptom dimensions, suggesting that the SWAN offers increased variance for the full spectrum of behaviors.

However, the SWAN did not offer incremental validity for prediction of interference. This may be related to two limitations: (a) the interference ratings were collected at the same time as the DBRS ratings, thus they may be biased to be more closely related to the DBRS scores and (b) the interference ratings were made on a 0 to 3 Likert-type scale, with no variance on the “good” end of interference, so the SWAN’s ability to capture variance at this good end was likely not maximized during these analyses.

Research to date has struggled to identify consistently strong cognitive correlates of ADHD (Nigg, Willcutt, Doyle, & Sonuga-Barke, 2005). The current study suggests that skewed measures of ADHD symptomatology, such as the DBRS, may be inadvertently limiting the potential for those correlations. In contrast, the SWAN provides sufficient variance at both ends of the inattention and HI spectrums to capture a statistical relationship. Thus, future investigations of candidate cognitive endophenotypes of ADHD would benefit from use of the SWAN as a behavioral measure.

Overall, the SWAN appears to be a useful tool for examining the etiology of the two ADHD symptom dimensions, both at the adaptive and impaired ends of performance. Given the increased range and complexity of SWAN scores, future work could include development of normative scores for diagnosis of ADHD. This would be an improvement over the symptom-count method employed by the DBRS and other DSM-IV–based measures for prediction of cognitive as well as behavioral functioning. Normative scores would also serve to strengthen the SWAN as a research tool and would provide a rich set of data with which to examine candidate cognitive endophenotypes for adaptive functioning correlates of development of inattention and HI.

Footnotes

Authors’ Note

We are grateful to the many twins, their families, and the twins’ teachers for their participation and to the Colorado and Australian Twin Registries for their assistance. The Australian Twin Registry is supported by an Enabling Grant from the National Health and Medical Research Council administered by the University of Melbourne.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is being conducted with the support of the Research Council of Norway (154715/330), the Swedish Research Council (345-2002-3701), University of Stavanger, Australian Research Council (A79906201, DP0770805, and DP0663498), and National Institutes of Health (2 P50 HD27802 and 1 R01 HD38526).

Bios

Anne B. Arnett, MA, is a clinical psychology PhD student in the Developmental Cognitive Neuroscience track at the University of Denver.

Bruce F. Pennington, PhD, is a developmental neuropsychologist who applies genetic and neuropsychological research methods to understand comorbidity among dyslexia, ADHD, and autism. He is a John Evans Professor of Psychology at the University of Denver, where he heads the Developmental Cognitive Neuroscience program.

Angela Friend, PhD, is a cognitive psychologist with a specialty in behavioral genetics. She is currently a postdoctoral fellow at the University of Colorado at Denver.

Erik G. Willcutt, PhD, is an associate professor in the Department of Psychology at the University of Colorado. He completed his PhD in clinical child psychology in 1998 at the University of Denver.

Brian Byrne, PhD, is emeritus professor of psychology at the University of New England, Australia. He cofounded the International Longitudinal Twin Study of Literacy, Language, and Attention with Dr. Richard K. Olson.

Stefan Samuelsson, PhD, is a professor at Linköping University, Sweden, and at the National Centre for Reading Education and Reading Research at Stavanger University, Norway. His research interests include behavioral genetics and reading development as well as reading difficulties.

Richard K. Olson, PhD, is a developmental psychologist who cofounded the Colorado Reading Project in 1979 and the Colorado Learning Disabilities Research Center in 1990. He is well known for his research on phonological and orthographic processing deficits in children with reading disabilities.

References

Achenbach

T. M.

(1991). Manual for the child behavior Checklist/4-18 and 1991 profile. Burlington: University of Vermont.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.

Barkley

Murphy

(1998). Attention deficit hyperactivity disorder: A clinical workbook (2nd ed.). New York, NY: Guilford.

Bussing

Fernandez

Harwood

Hou

Garvan

C. W.

Eyberg

S. M.

Swanson

J. M.

(2008). Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms: Psychometric properties and normative ratings from a school district sample. Assessment, 15, 317-328.

Centers for Disease Control. (2005). Mental health in the United States: Prevalence of diagnosis and medication treatment for attention-deficit/hyperactivity disorder—United States, 2003. Morbidity and Mortality Weekly Report, 54, 842-847.

Cornish

K. M.

Wilding

J. M.

Hollis

(2008). Visual search performance in children rated as good or poor attenders: The differential impact of DAT1 genotype, IQ, and chronological age. Neuropsychology, 22, 217-225.

DuPaul

G. J.

Anastopoulos

A. D.

Power

T. J.

Reid

Ikeda

M. J.

McGoey

K. E.

(1998). Parent ratings of attention deficit/hyperactivity disorder symptoms: Factor structure and normative data. Journal of Psychopathology and Behavioral Assessment, 20, 83-102.

DuPaul

G. J.

Power

T. J.

McGoey

K. E.

Ikeda

M. J.

Anastopoulos

A. D.

(1998). Reliability and validity of parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms. Journal of Psychoeducational Assessment, 16, 55-68.

10.

Erford

B. T.

(1997). Reliability and validity of scores on the Disruptive Behavior Rating Scale—Teacher version (DBRS-T). Educational and Psychological Measurement, 57, 329-339.

11.

Faraone

S. V.

Spencer

Aleardi

Pagano

Biederman

(2004). Meta-analysis of the efficacy of methylphenidate for treating adult attention-deficit/hyperactivity disorder. Journal of Clinical Psychopharmacology, 24, 24-29.

12.

Friedman-Weieneth

Doctoroff

G. L.

Harvey

E. A.

Goldstein

L. H.

(2009). The Disruptive Behavior Rating Scale—Parent version (DBRS-PV): Factor analytic structure and validity among young preschool children. Journal of Attention Disorders, 13, 42-55.

13.

Hay

Bennett

K. S.

Levy

Sergeant

Swanson

(2007). A twin study of attention-deficit/hyperactivity disorder dimensions rated by the strengths and weaknesses of ADHD-symptoms and normal-behavior. Biological Psychiatry, 61, 700-705.

14.

McGrath

Pennington

B. F.

Shanahan

Santerre-Lemmon

Barnard

Willcutt

E. G.

. . . Olson

R. K.

(2011). A multiple deficit model of reading disability and attention-deficit/hyperactivity disorder: Searching for a shared cognitive deficit. Journal of Child Psychology and Psychiatry, 52, 547-557.

15.

Neale

M. C.

Boker

S. M.

Xie

Maes

H. H.

(2006). Mx: Statistical modeling manual. Richmond: Department of Psychiatry, Medical College of Virginia.

16.

Nichols

R. C.

Bilbro

W. C.

(1966). The diagnosis of twin zygosity. Acta Genetica, 16, 265-275.

17.

Nigg

Willcutt

E. G.

Doyle

A. E.

Sonuga-Barke

E. J.

(2005). Causal heterogeneity in attention-deficit/hyperactivity disorder: Do we need neuropsychologically impaired subtypes? Biological Psychiatry, 57, 1224-1230.

18.

Pelham

W. E.

Gnagy

E. M.

Greenslade

K. E.

Milich

(1992). Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 31, 210-218.

19.

Pennington

B. F.

(2002). The development of psychopathology: Nature and nurture. New York, NY: Guilford.

20.

Polderman

T. J. C.

Derks

E. M.

Hudziak

J. J.

Verhulst

F. C.

Posthuma

Boomsma

D. I.

(2007). Across the continuum of attention skills: A twin study of the SWAN ADHD Rating Scale. Journal of Child Psychology and Psychiatry, 48, 1080-1087.

21.

Samuelsson

Byrne

Quain

Wadsworth

Corley

DeFries

J. C.

. . . Olson

(2005). Environmental and genetic influences on prereading skills in Australia, Scandinavia, and the United States. Journal of Educational Psychology, 97, 705-722.

22.

Stevenson

Asherson

Hay

Levy

Swanson

Thapar

Willcutt

(2005). Characterizing the ADHD phenotype for genetic studies. Developmental Science, 8(2), 115-121

23.

Swanson

Shuck

Mann

Carlson

Hartman

Sergeant

. . . McCleary

(2006). Categorical and dimensional definitions and evaluations of symptoms of ADHD: The SNAP and SWAN Rating Scales. Irvine: University of California.

24.

Wagner

R. K.

Torgesen

J. K.

Rashotte

C. A.

(1999). The comprehensive test of phonological processes (CTOPP). Austin, TX: PRO-ED.

25.

Willcutt

E. G.

Betjemann

R. S.

Wadsworth

S. J.

Samuelsson

Corley

DeFries

J. C.

. . . Olson

R. K.

(2007). Preschool twin study of the relation between attention-deficit/hyperactivity disorder and prereading skills. Reading and Writing, 20, 103-125.

26.

Young

Levy

Martin

N. C.

Hay

D. A.

(2009). Attention deficit hyperactivity disorder: A Rasch analysis of the SWAN Rating Scale. Child Psychiatry and Human Development, 40, 543-559.