The Structure of Cognitive Abilities in Youths With Manic Symptoms

Abstract

This study compared the structure of cognitive ability (specifically, verbal/crystallized [Gc] and visual-spatial ability [Gv]), as measured in the Wechsler Intelligence Scale for Children, in youth with manic symptoms with a nationally representative group of similarly aged youth. Multigroup confirmatory factor analysis found the majority of the estimated parameters were invariant between the groups, although there was a difference in the intercepts for the Similarities subtest and difference in unique variances for the Picture Completion, Comprehension, and Picture Arrangement subtests. Thus, although there are many neurological changes associated with manic symptoms, the structure of verbal/crystallized and visual-spatial abilities appear relatively robust and invariant. As Gc and Gv are the primary domains on all the Wechsler Intelligence Scales for Children, results also indicate that clinical interpretation of the Wechsler scales may be appropriate to measure cognitive performance in youths with manic symptoms.

Keywords

mania bipolar disorder cognitive ability Wechsler Intelligence Scale for Children factorial invariance children and adolescents

Bipolar disorder among children and youth has become an important focus of clinical research. The onset of this disorder in children and adolescents causes not only disruptions in social (Goldstein, Miklowitz, & Mullen, 2006) and family functioning (Lewinsohn, Klein, & Seeley, 2000) but also a tremendous challenge in terms of academic functioning (Doyle et al., 2005; Henin et al., 2007; Lagace, Kutcher, & Robertson, 2003).

As the onset of a disorder on the bipolar spectrum appears to be accompanied by many neurobiological and neurochemical changes (Berns & Nemeroff, 2003), an obvious follow-up question is if the structure of cognitive abilities also changes. Although there is a reasonable body of literature regarding cognitive abilities in adults with bipolar disorder (for meta-analyses, see Arts, Jabben, Krabbendam, & van Os, 2008; Robinson et al., 2006), less research has been available regarding cognitive profiles of youth with this disorder until recently, and the research that has been published is often equivocal (e.g., Dickstein et al., 2004; Doyle et al., 2005; Glahn et al., 2005; for a review and meta-analysis, see Joseph, Frazier, Youngstrom, & Soares, 2008). There are some differences that appear more pronounced during acute mood states, and others that appear to persist during remission and which may be trait markers of illness. However, meta-analyses have identified some emerging patterns. More specifically, youth with this disorder may manifest the greatest impairments in the areas of verbal memory, attention, and working memory (Cohen’s [1988] ds = .60-.77), in addition to other deficits with small-to-medium effect sizes, for example, executive functioning, visual memory, and verbal fluency (Joseph et al., 2008). Some of the deficits in executive function appear to be specific to bipolar disorder and not attributable to attention-deficit/hyperactivity disorder or other comorbid conditions (Walshaw, Alloy, & Sabb, 2010).

Importantly, despite indications that deficits may exist for some neuropsychological domains, many research groups have failed to find differences in Full Scale Intelligence Quotient (FSIQ) scores between youth with a bipolar diagnosis and their nondiagnosed peers (Dickstein et al., 2004; Dickstein et al., 2005; Lagace et al., 2003; McClure et al., 2005; Rucklidge, 2006). When studies have found differences, the youth with bipolar disorder usually scored within the average range, although lower than the comparison subjects (Doyle et al., 2005; Meyer et al., 2004; Olvera, Semrud-Clikeman, Pliszka, & O’Donnell, 2005; Voelbel, Bates, Buckman, Pandina, & Hendren, 2006). In one instance, the healthy comparison group had a mean FSIQ one standard deviation above the mean, clearly creating an artifact of lower FSIQ in the bipolar group (Voelbel et al., 2006). However, in another study, inpatient youth with bipolar disorder did have strikingly lower FSIQ scores and Performance IQ scores when compared with inpatient youth with attention-deficit/hyperactivity disorder, oppositional defiant disorder, or conduct disorder (McCarthy et al., 2004), with scores from most bipolar cases falling in the below average to borderline range.

Despite these relatively recent investigations into the varying cognitive abilities in youths with bipolar disorder or different degrees of manic symptoms, there are still many unanswered questions regarding the underlying structure of cognitive abilities. Specifically, nowhere in the literature does anyone address the question of whether manic symptoms is associated with change the overall structure of cognitive abilities in youths, or if there are just mean differences, with the overall structure remaining the same. The distinction becomes important when, say, comparing youths with learning disabilities to their typically developing peers. Studies that have compared students with learning disabilities versus the general school-age population often show that students with learning disabilities tend to have lower scores on measures of cognitive ability (Fiorello et al., 2007). However, there are not just mean differences between the groups, as the few studies that have compared the factor structure between these groups have also shown differences, indicating that the ways in which knowledge is processed, remembered, and used is different in those with learning disabilities than in the general population (Pomplun & Omar, 2001; see also Hale & Fiorello, 2004). As yet, there is no study that has examined if the same holds true in youth with manic symptoms. It is plausible that mania might degrade performance in some areas, but be neutral or even enhance performance in other areas. These contrasting influences would change the relationship of the scales measuring each domain—both to each other and to any latent factors.

Comparing the structure of cognitive abilities across groups can be done through a process called multigroup factor analysis (MGCFA; Widaman & Reise, 1997). The goal of multigroup factor analysis is to see if the parameter estimates for the factor-analytic model (e.g., factor loadings, variances) are the same across multiple groups. Scholars investigate factorial invariance under two scenarios: investigating measurement invariance (MI) or factor-analytic structural equation invariance (FASEI; Bontempo & Hofer, 2007; Vandenberg & Lance, 2000). Although the methods used in both scenarios can be equivalent, the reason for choosing a particular method is usually quite different. Researchers conduct MI studies in test development as part of the test validity studies, and the major purpose is to investigate possible bias in the instrument. Consequently, they typically conduct the analyses at the item level and usually involve only one instrument. FASEI studies, on the other hand, are usually interested in more substantive theoretical issues or, perhaps, developing better models for the construct of interest. Researchers usually conduct FASEI studies with aggregated scores (e.g., subtests, index scores) and can involve one or multiple instruments.

The current study compares the cognitive ability in youth with manic symptoms with their similarly aged peers without such symptoms. It is a FASEI study, as the purpose is to assess whether the structure of cognitive ability is the same in the two groups of youths and not to assess for bias in a specific measurement instrument.

Method

Sample

Two hundred forty-eight youth were admitted to a university hospital children’s psychiatric inpatient facility for the first time between September 1987 and December 1993. Irrespective of admission symptoms, a school psychologist administered a Wechsler Intelligence Scale for Children (WISC) to all admitted youth unless he/she had had recent testing in school or the psychologist was not available on admission. A psychologist interviewed the youth and their parents with the Yale Version of the Schedule for Affective Disorders and Schizophrenia for Children (Orvaschel, Puig-Antich, Chambers, Tabrizi, & Johnson, 1982) within 2 weeks of admission. The psychologist doing the interviews was blind to preadmission data. Eighty-six of these youth either reported having experienced manic symptoms and/or had parents who reported the youth had manic symptoms sufficient to warrant diagnosis of a manic episode, and thus Bipolar I disorder. All patients met with a board-certified child psychiatrist, who confirmed the diagnoses. The clinical interview and observation on the inpatient unit ruled out instances where mania might have been substance induced or because of a general medical condition. Prior medications were discontinued at admission so the staff could understand the degree to which visible psychopathology was because of the child’s condition versus a response to unsuccessful treatment when they were hospitalized. Psychoeducational testing was done at that time. Although the Diagnostic and Statistical Manual of Mental Disorders (DSM) has gone through several revisions through the period during which data were collected, it is important to note that the symptoms and definitions of a manic episode and Bipolar I disorder remained consistent across DSM-III (American Psychiatric Association, 1980), DSM-III-R (American Psychiatric Association, 1987), and DSM-IV (American Psychiatric Association, 1994). Of those 86 with mania, 81 had subtest scores available on a WISC. The mean age for the group was 8.62 years (SD = 1.95), and 89% of them were male. Eighty-three percent were Caucasian, 12% were African American, and the other 5% were Hispanic. The patients meet criteria for the following diagnoses, although they were not necessarily given a diagnosis: attention deficit/hyperactivity disorder, 81.50%; conduct disorder, 58.8%; oppositional defiant disorder, 86.4%; schizophrenia, 4.9%; depression (any type), 43.2%; anxiety (any type), 16.0%; pervasive developmental disorder, 3.7%; learning disorder (any type), 13.6%; language disorder, 19.8%; IQ ≤ 70,¹ 4.9%.

For more information about the clinical sample see other work by Carlson and colleagues (Carlson & Kelly, 1998; Carlson & Youngstrom, 2003).

Instrument

The 81 youth with manic symptoms (YMS) had scores on the following WISC subtests: Information, Similarities, Arithmetic, Vocabulary, Comprehension, Digit Span, Picture Completion, Picture Arrangement, Block Design, Object Assembly, and Coding. The model of cognitive ability used in this study for the WISC subtests follows the results from Phelps, McGrew, Knopik, and Ford (2005), whose model, in turn, follows from Carroll’s (1993) work, the most comprehensive work on the structure of cognitive abilities to date (Jensen, 2004). In Phelps et al.’s (2005) model, the Vocabulary, Information, Similarities, and Comprehension subtests are measures of verbal/crystallized intelligence (Gc). Block Design, Picture Completion, Picture Arrangement, and Object Assembly subtests are measures of visual-spatial ability (Gv), although it is important to note that these tests are also time limited. The Digit Span, Arithmetic, and Coding subtests each measure separate abilities (short-term memory, quantitative knowledge, and processing speed, respectively). Because the protocol measure did not gather any additional data on these other abilities, and the use of single indicator latent variables is tenuous (Brown, 2006), we did not use scores on these subtests for the current analysis. A pictorial representation of this study’s model of cognitive ability is given in Figure 1.

Figure 1.

The Verbal/Crystallized (Gc)–Visual-Spatial (Gv) factor model for the Wechsler Intelligence Scale for Children–Third Edition

The WISC is particularly apt for assessing invariance of Gc and Gv abilities. All WISC editions published to date have multiple subtests that measure these constructs. As a result, these factors are well specified and there is a large accumulated body of research with these constructs measured via the WISC subtests in both clinical and nonclinical samples. It is the third most commonly administered test by practicing clinical psychologists (behind only the Wechsler Adult Intelligence Scale and the Minnesota Multiphasic Personality Inventory—both of which are used primarily with adults; Camara, Nathan, & Puente, 2000), and it is the most commonly used test in the assessment of adolescents (Archer & Newsom, 2000). Moreover, the Wechsler family of cognitive ability scales (which include the WISC, the Wechsler Adult Intelligence Scale, and the Wechsler Preschool and Primary Scale of Intelligence) are the most commonly taught tests of cognitive ability in both clinical and school psychology programs (Alfonso, Oakland, LaRocca, & Spanakos, 2000). More than 60% of these programs see the WISC as “essential” to training (Belter & Piotrowski, 2001), and roughly 90% of all training programs include the WISC in their core assessment training (Childs & Eyde, 2002). Thus, using the WISC when measuring Gc and Gv abilities maximizes generalizability and clinical relevance of the study.

WISC editions

In 1991, the hospital where the YMS were admitted moved from administering the WISC–Revised (WISC-R; Wechsler, 1974) to administering the WISC–Third Edition (WISC-III; Wechsler, 1991a) following its publication, consistent with ethical guidelines for practice. Of the 81 YMS, 38 have scores on the WISC-R and 43 have scores on the WISC-III. The covariance-and-means matrices of the two groups were compared for equivalence for the eight WISC variables included in Figure 1 and were found not to be different (Sörbom, 1974), so they were combined into one YMS group for all other analyses (see Table 1). Table 2 provides descriptive statistics for the WISC subtests in the combined YMS group.

Table 1.

Fit Statistics for Youth With Manic Symptoms on the Wechsler Intelligence Scale for Children Combined Data

n	χ²	df	p	CFI	TLI	RMSEA	SRMR
81	29.85	36	.76	1.000	1.030	.000	.090

Note. CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

Table 2.

Correlation Matrix (With Standard Deviations on Diagonal) and Means for Wechsler Intelligence Scale for Children Subtests in Youth With Manic Symptoms

Subtest	1	2	3	4	5	6	7	8
1.Information	3.08
2.Similarities	.72	3.55
3.Vocabulary	.66	.77	3.20
4.Comprehension	.65	0.74	.75	2.87
5.Picture Completion	.40	.56	.57	.58	2.51
6.Picture Arrangement	.29	.35	.39	.46	.52	3.98
7.Block Design	.36	.46	.49	.33	.43	.47	3.37
8.Object Assembly	.38	.42	.49	.40	.49	.47	.62	3.15
Mean	10.09	12.07	10.25	9.96	10.90	11.24	10.30	10.44

For the reference group, the WISC-III standardization sample was chosen, as opposed to another WISC edition (e.g., WISC-R), because its norms were developed close in time to when the YMS data were collected, thus lessening concerns about Flynn effect artifacts (Beaujean, Sheng, & Qiu, 2010; Flynn, 2007). Moreover, using Cohen’s (1988) monikers, the correlations between the WISC-R and WISC-III subtests are large for Gc (Information, 80; Similarities, .74; Vocabulary, .77; Comprehension, .67) and Gv (Picture Completion, .57; Picture Arrangement, .42; Object Assembly, .58; Wechsler, 1991b). The WISC-III norming data (correlations, means, and standard deviations) are divided by age groups, so the current analysis used the 9-year old sample (Wechsler, 1991b) as it is closest in age to the average age of the YMS data. There were 200 children assessed at this age, with 50% of them being male. The norming samples were selected to represent the population of children in the United States in the late 1980s and thus are diverse with respect to geographic location, race, and socioeconomic status (Wechsler, 1991b).

Testing for Invariance

Testing for invariance is a multistep procedure, testing more restrictive models at each step (Widaman & Reise, 1997). When testing for factorial invariance, there are multiple ways to examine if the data are consistent with the increasingly restrictive models, but Byrne and Stewart (2006) suggest two sets of criteria. The first (“traditional perspective”) examines the change in chi-square values (Δχ²) across nested models. If, as the models grow more restrictive, the Δχ² values do not significantly change, this is evidence that more restrictive model fits data as well as the less restrictive model; thus, the more restrictive model should be favored over the less restrictive one.

The use of Δχ² values has been criticized because of their sensitivity to sample size (Cheung & Rensvold, 2002). Recently, Cheung and Rensvold (2002) and Meade, Johnson, and Braddy (2008) provided evidence that some alternative fit indices were not prone to this problem. Specifically, both studies found that the comparative fit index (CFI; Bentler, 1990) and McDonald’s (1989) Noncentrality Index (Mc) were robust across a variety of sample sizes. Moreover, for both statistics, the changes in these values from one model nested in another (ΔCFI, ΔMc, respectively) were relatively uncorrelated with the change in values from another nested-model comparison. This is important because it means that the test of invariance between one set of models is not dependent on a test of invariance for another set of models. Thus, Byrne and Stewart’s (2006) second line of criteria (“practical perspective”) recommends that invariance can be based on two criteria: (a) the multigroup factor model exhibits an adequate fit to the data and (b) the change in values for fit indices (e.g., ΔCFI, ΔMc) is negligible.

Fit indices used for this study

Based on Byrne and Stewart’s (2006) recommendation, this study used two sets of fit indices: one to assess overall model fit and the other to assess change in model fit between two models. As Hu and Bentler (1999) recommend, we used multiple fit indices for both. For overall model fit, we included the root mean square error of approximation (RMSEA; Browne & Cudeck, 1993), CFI, Mc, and the standardized root mean square residual (SRMR; Bentler, 1995). These indices were chosen as they represent a variety of fit criteria and they tend to perform well in evaluating different models (Marsh, Hau, & Grayson, 2005). In addition, we also inspected each model’s χ² value and its associated p value (Barrett, 2007). For both overall model fit as well as change in model fit, we looked for patterns in the fit statistics and judged acceptance/rejection of the specific model based on the majority of the indices.

For this study’s criteria of overall model-data fit, we used the following: (a) χ² p values greater than .02 (to correct the traditional cutoff of .05 for multiple comparisons but at the same time not be overly restrictive on power; see also Yu, 2002), (b) RMSEA values less than .075 (halfway between .050 and .100; Chen, Curran, Bollen, Kirby, & Paxton, 2008), (c) SRMR values less than .080 (Hu & Bentler, 1999; Sivo, Xitao, Witta, & Willse, 2006), (d) CFI values greater than .960 (Yu, 2002), and (e) Mc values greater than .900 (Marsh, Hau, & Wen, 2004; Sivo et al., 2006).²

To test the change in fit between nested models, we used the ΔCFI, ΔMc, and the Δ χ²’s associated p value. As with assessing overall fit, we used p values greater than .02 as evidence of invariance for the new constrictions between the two models. Cheung and Rensvold (2002) suggested .01 as the threshold of ΔCFI and .02 as the threshold for ΔMc. Meade et al. (2008), however, suggested more restrictive values of .002 for ΔCFI and .007 for ΔMc (based on having 2 factors and 8 indicators).³ As this issue is not yet resolved, we considered both values for the ΔCFI and ΔMc, with values less than Meade et al.’s (2008) criteria indicating stronger evidence of invariance than values only meeting Cheung and Rensvold’s (2002) criteria.

Data inspection

When using latent variable models, or at least estimating their parameters by maximum likelihood, a key assumption is that the data come from a multivariate normal distribution (Kline, 2004). Using the methods outlined in DeCarlo (1997), we investigated the data for univariate and multivariate normality. No variable departed from univariate normality using D’Agostino and Pearson’s (1973) normality test, and the variable set exhibited multivariate normality using Small’s (1980) test. Consequently, we estimated all models in MPlus (version 5.1) using the maximum likelihood estimator.

Results

Invariance Analysis

Before testing for invariance, Meade et al. (2008) recommend first assessing factor model fit for each group separately. The results are in the first two rows in Table 3 (Models 1a and 1b). All the fit indices for the Gc-Gv model were within the limits specified in the Methods section for both samples, except the RMSEA for YMS data. Consequently, there appeared to be enough evidence to support assessing for invariance using the two-factor model.

Table 3.

Fit Statistics for Invariance Assessment

Model		n	χ²	df	p	Δχ²	Δdf	p	CFI	ΔCFI	RMSEA	SRMR	Mc	ΔMc
1a	YMS	81	29.19	19	.06	—	—	—	0.971	—	0.081	0.047	0.938	—
1b	WISC-III	200	24.21	19	.19	—	—	—	0.989	—	0.037	0.029	0.987	—
2	Configural Invariance	281	53.40	38	.05	—	—	—	0.985	—	0.054	0.035	0.973	—
3	Metric Invariance	281	66.01	44	.02	12.61	6	.05	0.979	0.006	0.060	0.058	0.961	0.011
4	Scalar Invariance	281	109.11	50	.00	43.10	6	.00	0.942	0.037	0.092	0.067	0.900	0.062
4a	Scalar Invariance^a	281	75.94	49	.01	9.93	5	.08	0.974	0.005	0.063	0.061	0.953	0.008
5	Invariant Unique Variance	281	94.90	57	.00	18.96	8	.02	0.963	−0.011	0.069	0.076	0.935	0.018
5a	Invariant Unique Variance^b	281	76.93	54	.02	0.99	5	.96	0.978	0.004	0.055	0.062	0.960	−0.007
6	Factor Variance Invariance	281	80.03	56	.02	3.10	2	.21	0.977	−0.001	0.055	0.074	0.958	0.002
7	Invariant Factor Covariance	281	80.42	57	.02	0.39	1	.54	0.977	0.000	0.054	0.073	0.959	−0.001
8	Equal Factor Means	281	84.00	59	.02	3.58	2	.17	0.976	−0.001	0.055	0.078	0.956	0.003

Note. YMS = youth with manic symptoms; CFI = comparative fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; Mc = McDonald’s Noncentrality Index; Δ = change in value between competing models.

Similarities intercepts free.

Picture Completion, Comprehension, and Picture Arrangement unique variances are free.

In the next step, we assessed for configural invariance, which examines whether the factor model is the same across both groups. Given the results from the individual group analysis, it was not surprising that the Gc-Gv model fit the (combined) data well, as shown by all the models’ overall model fit values (i.e., χ², RMSEA, SRMR, and Mc) meeting the prespecified criteria (see Table 3, Model 2).

The third step involved assessing for metric invariance, which we did by examining if the factor loadings were the same across both groups. The results (Table 3, Model 3) indicate that the overall model fit the data relatively well, as the fit statistics’ values all met the cutoff criteria. For the change-in-model fit statistics, the p value for the Δχ² was greater than the cutoff and the ΔCFI and ΔMc both meet the Cheung and Rensvold (2002) criteria but not the Meade et al. (2008) criteria. Consequently, while some indices were not within limits specified in the Methods section, the majority of them were within the limits; thus, there appeared to be enough evidence to continue the invariance assessment.

In the next model, we constrained all the indicator variables’ intercepts (i.e., the scales’ origins) across groups. The results are in Table 3, under Model 4. The fit indices for the overall model satisfy the a priori criteria, but all three change-in-model fit values indicated this model did not fit the data as well as the previous (i.e., metric invariance) model. We examined the groups’ subtests’ mean scores and found the largest discrepancy for the Similarities tasks, so allowed it to vary between groups. We then tested this revised model (Model 4a) and found both the overall fit and change-in-fit statistics indicated this model fit the data relatively well, although the ΔCFI and ΔMc values exceeded the Meade et al. (2008) criteria.

The next step involved examining if any of the subtests’ unique (residual) variances were invariant across the two groups. The results (Model 5) of almost all the overall and change-in-fit indices (the exceptions were the SRMR and ΔMc using Cheung and Rensvold’s, 2002, criteria) indicated this model poorly fit the data. We then examined each subtests’ variability and found that Picture Completion, Comprehension, and Picture Arrangement had the largest between-groups differences (i.e., all >2 standard score units). Subsequently, we allowed them to vary between groups, and almost all the resulting fit indices from this ameliorated Model 5, Model 5a, indicated this model fit the data relatively well (the exception was the ΔCFI value using the Meade et al., 2008, criterion).

For the last step we assessed whether the factor variances, covariances, and means were invariant between the two groups. These steps were carried out separately (Models 6, 7, and 8, respectively, in Table 3), but converge in showing that there was no between-group difference in these values as both the overall model fit statistics and the change-in-fit statistics are all within the predefined criteria. The parameter estimates from the final model (Model 8) are in Table 4.

Table 4.

Estimates of Intercepts, Loadings, and Unique Variances of the Partial Invariance Model

	Intercept	Factor Loading	Unique Variance
Information	10.08	2.40	3.73
Similarities		2.65	2.51
YMS	11.90
WISC-III	10.37
Vocabulary	9.93	2.63	2.36
Comprehension	10.01	2.07
YMS			2.51
WISC-III			4.83
Picture Arrangement	10.39	1.84
YMS			10.36
WISC-III			8.32
Block Design	10.02	2.46	5.35
Picture Completion	10.40	2.03
YMS			2.90
WISC-III			5.64
Object Assembly	10.27	2.38	4.87

Note. YMS = youth with manic symptoms; WISC-III = Wechsler Intelligence Scale for Children–Third Edition. Parameter estimates are shown for Model 8 (see Table 3), where all factor variances are 1, all factor means are 0, and the Gv-Gc covariance is .76. The estimated values for the YMS and 9-year-old norming group from the WISC-III groups are shown separately if invariance constraints are not imposed.

Discussion

The purpose of this study was to compare the structure of Gc and Gv abilities of youth with manic symptoms (YMS) and similar-aged peers from the general population. Using the four Gv and four Gc subtests on the WISC-III (Wechsler, 1991a), a group of YMS were compared with a similarly aged group from the norming sample of the WISC-III (9-year-old sample). The comparison was conducted by examining invariance in the mean and covariance structure between the two groups via a multiple-group confirmatory factor analysis.

The results indicated that the Gv-Gc factor structure (see Figure 1) and factor loadings are invariant across both groups. Moreover, for all subtests except Similarities, the intercepts were the same between both groups. Alternatively stated, all mean differences in the subtest scores (except for the Similarities) are accounted for by the Gv and Gc factor means (Bontempo & Hofer, 2007).

When we examined the unique variances, five of the eight subtests were equal across groups; the three that were not were Picture Completion, Comprehension, and Picture Arrangement. For Picture Arrangement the YMS group exhibited more unique variance (10.36 vs. 8.32), but the norming group showed more for the other two subtests (2.51 vs. 4.83 and 2.90 vs. 5.64 for Comprehension and Picture Completion, respectively). Last, we assessed invariance at the factor level and found, when the noninvariant intercept and unique variances were unconstrained, the factor variances, covariance, and means were equivalent across both groups.

Byrne, Shavelson, and Muthén (1989) first labeled the situation where most, but not all, of the parameter estimates across groups are the same as partial measurement invariance. However, as we conducted the current study under a factor-analytic structural equation invariance framework instead of measurement invariance framework (Bontempo & Hofer, 2007), we interpret the findings from a different perspective than Byrne et al. (1989). First, while the Gc and Gv constructs are the same between the two groups, their ability to predict subtest scores somewhat differs. More specifically, for a given factor score, the predicted subtest score is the same in both the YMS and the norming groups for all the WISC-III subtests except Similarities. For this subtest, a given score on the Gv factor will produce a higher predicted score for the YMS group than the norming group. As this difference in predicted Similarities scores will be constant across all Gv factor scores, this situation is akin to an ANCOVA with equal slopes but different intercepts (see Figure 2A). Second, because there were unique variance differences between the two groups on the Picture Completion, Comprehension, and Picture Arrangement subtests, there will be differences in the accuracy of the Gv and Gc factors to predict these subtests’ scores. For example, because the norming group has a larger unique variance for Picture Completion than the YMS group, we would expected that for a given Gc factor score the predicted Picture Completion score would be more accurate for the YMS group than the norming group. This situation is akin to heterogeneity of error variance in an ANCOVA model, where for a given value of the predictor variable there is more variability in the outcome variable in one group than another group. We depict this situation graphically in Figure 2B.

Figure 2.

Graphical depiction of measurement invariance

The results from this current study indicate that despite the many neurobiological and neurochemical changes that co-occur in people with a bipolar diagnosis (Chang et al., 2005; DelBello, Zimmerman, Mills, Getz, & Strakowski, 2004; Goodwin & Jamison, 2007), the general structure of their overall cognitive abilities, at least in their youth, does not change. This finding is reassuring from a clinical perspective, because it indicates that one of the most widely used tests in clinical assessment (i.e., the WISC) appears to remain internally valid when working with youths potentially affected by bipolar disorder. Given the importance of recommending and implementing appropriate educational accommodations for youth with bipolar disorder (Fristad, Verducci, Walters, & Young, 2009; Papolos & Papolos, 2002; Pavuluri, O’Connor, Harral, Moss, & Sweeney, 2006), it is reassuring that a widely used measure of cognitive ability appears to maintain largely invariant measurement properties, even when used with youth experiencing severe enough disturbance to warrant psychiatric hospitalization.

Limitations

There are three major limitations in the current study. The first is that the YMS sample is small (n = 81). Although this sample was representative of the population of youth with mania who are admitted to the hospital where the sample was drawn, we should be cautious about generalization to all youth with mania, especially as the demographics for this sample (i.e., race, sex, age) might not match the general YMS population. Even though the comparison group had 200 people, the size of the overall combined sample (n = 281) could be a reason why we did not find significant departures from invariance at some levels (e.g., loadings, factor variances; Meade et al., 2008). Larger and more representative clinical samples would enable us to make statements that are more definitive. The second major limitation is the use of older versions of the WISC as the measure of cognitive ability. The Wechsler intelligence instruments are among the most commonly used instruments in applied psychology, but they only measure a few areas of cognitive ability, with Gv and Gc being the two major areas, although the fourth edition of the WISC does have fewer subtests measuring Gv than its predecessors (Kaufman, Flanagan, Alfonso, & Mascolo, 2006). Future studies should look to use either a larger battery of cognitive ability batteries or should use a battery designed to measure different areas of cognitive ability, such as the Woodcock-Johnson–Third Edition (Woodcock, McGrew, & Mather, 2001) or Differential Ability Scales–Second Edition (Elliot, 2007). Although medications had been discontinued on admission and for the baseline time during which testing was done, data were not available about the medications taken by each youth prior to admission. Many of the medications used to manage bipolar disorder are known to influence performance on neurocognitive tests. Because the sample is older and the children young, the medication exposure of participants was lower than it would be in a similar cohort today. Medication effects on short- and long-term cognitive performance remain an important topic of inquiry.

Overall, practitioners and consumers of testing reports—including educators, youths, and their families—can take some comfort in the fact that the major factors underlying the WISC appear to have fundamentally equivalent factor validity in youth with mania as compared with a nationally representative sample of youths. Because the current findings are from an inpatient sample, findings from other studies are likely to be similar (or to have even smaller differences) when factor validity is examined in less severely affected samples, such as outpatient or primary care settings. Present findings also enhance confidence in the validity of research finding using the WISC, as its measurement properties appeared robust across the constructs between clinical and community samples.

Footnotes

Declaration of Conflicting Interests

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Funding

The authors received no financial support for the research and/or authorship of this article.

Notes

References

Alfonso

V. C.

Oakland

T. D.

Larocca

Spanakos

(2000). The course on individual cognitive assessment. School Psychology Review, 29, 52-64.

American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author.

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., revised). Washington, DC: Author.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.) Washington, DC: Author.

Archer

R. P.

Newsom

C. R.

(2000). Psychological test usage with adolescent clients: Survey update. Assessment, 7, 227-235. doi:10.1177/107319110000700303

Arts

Jabben

Krabbendam

Van Os

(2008). Meta-analyses of cognitive functioning in euthymic bipolar patients and their first-degree relatives. Psychological Medicine, 38, 771-785. doi:10.1017/S0033291707001675

Barrett

(2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42, 815-824. doi:10.1016/j.paid.2006.09.018

Beaujean

A. A.

Sheng

Qiu

(2010, April). Examining the Flynn effect in the Wechsler Intelligence Scale for Children. Paper presented at the meeting of the Society for Applied Multivariate Research, Dallas, TX.

Belter

R. W.

Piotrowski

(2001). Current status of doctoral-level training in psychological testing. Journal of Clinical Psychology, 57, 717-726. doi:10.1002/jclp.1044

10.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246. doi:10.1037/0033-2909.107.2.238

11.

Bentler

P. M.

(1995). EQS structural equations program manual. Encino, CA: Multivariate Software.

12.

Berns

G. S.

Nemeroff

C. B.

(2003). The neurobiology of bipolar disorder. American Journal of Medical Genetics Part C: Seminars in Medical Genetics, 123C, 76-84. doi:10.1002/ajmg.c.20016

13.

Bontempo

D. E.

Hofer

S. M.

(2007). Assessing factorial invariance in cross-sectional and longitudinal studies. In Ong

A. D.

van Dulmen

(Eds.), Handbook of methods in positive psychology (pp. 153-175). New York, NY: Oxford University Press.

14.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

15.

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.

16.

Byrne

B. M.

Shavelson

R. J.

Muthén

B. O.

(1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. doi:10.1037/0033-2909.105.3.456

17.

Byrne

B. M.

Stewart

S. M.

(2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13, 287-321. doi:10.1207/s15328007sem1302_7

18.

Camara

W. J.

Nathan

J. S.

Puente

A. E.

(2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141-154. doi:10.1037/0735-7028.31.2.141

19.

Carlson

G. A.

Kelly

K. L.

(1998). Manic symptoms in psychiatrically hospitalized children—What do they mean? Journal of Affective Disorders, 51, 123-135. doi:10.1016/S0165-0327(98)00211-0

20.

Carlson

G. A.

Youngstrom

E. A.

(2003). Clinical implications of pervasive manic symptoms in children. Biological Psychiatry, 53, 1050-1058. doi:10.1016/S0006-3223(03)00068-4

21.

Carroll

J. B.

(1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.

22.

Chang

Karchemskiy

Barnea-Goraly

Garrett

Simeonova

D. I.

Reiss

(2005). Reduced amygdalar gray matter volume in familial pediatric bipolar disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 565-573. doi:10.1097/01.chi.0000159948.75136.0d

23.

Chen

Curran

P. J.

Bollen

K. A.

Kirby

Paxton

(2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36, 462-494. doi:10.1177/0049124108314720

24.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233-255. doi:10.1207/S15328007SEM0902_5

25.

Childs

R. A.

Eyde

L. D.

(2002). Assessment training in clinical psychology doctoral programs: What should we teach? What do we teach? Journal of Personality Assessment, 78, 130-144. doi:10.1207/S15327752JPA7801_08

26.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah, NJ: Erlbaum.

27.

D’Agostino

Pearson

E. S.

(1973). Tests for departure from normality. Empirical results for the distributions of b2 and √b1. Biometrika, 60, 613-622. doi:10.1093/biomet/60.3.613

28.

Decarlo

L. T.

(1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-307. doi:10.1037/1082-989X.2.3.292

29.

Delbello

M. P.

Zimmerman

M. E.

Mills

N. P.

Getz

G. E.

Strakowski

S. M.

(2004). Magnetic resonance imaging analysis of amygdala and other subcortical brain regions in adolescents with bipolar disorder. Bipolar Disorders, 6, 43-52. doi:10.1046/j.1399-5618.2003.00087.x

30.

Dickstein

D. P.

Garvey

Pradella

A. G.

Greenstein

D. K.

Sharp

W. S.

Castellanos

F. X.

. . . Leibenluft

(2005). Neurologic examination abnormalities in children with bipolar disorder or attention-deficit/hyperactivity disorder. Biological Psychiatry, 58, 517-524. doi:10.1016/j.biopsych.2004.12.010

31.

Dickstein

D. P.

Treland

J. E.

Snow

McClure

E. B.

Mehta

M. S.

Towbin

K. E.

. . . Leibenluft

(2004). Neuropsychological performance in pediatric bipolar disorder. Biological Psychiatry, 55, 32-39. doi:10.1016/S0006-3223(03)00701-7

32.

Doyle

A. E.

Wilens

T. E.

Kwon

Seidman

L. J.

Faraone

S. V.

Fried

. . . Biederman

(2005). Neuropsychological functioning in youth with bipolar disorder. Biological Psychiatry, 58, 540-548. doi:10.1016/j.biopsych.2005.07.019

33.

Elliot

C. D.

(2007). Differential ability scales (2nd ed.). San Antonio, TX: Psychological Corporation.

34.

Fiorello

C. A.

Hale

J. B.

Holdnack

J. A.

Kavanagh

J. A.

Terrell

Long

(2007). Interpreting intelligence test results for children with disabilities: Is global intelligence relevant? Applied Neuropsychology, 14, 2-12. doi:10.1080/09084280701280338

35.

Flynn

J. R.

(2007). What is intelligence? Beyond the Flynn effect. New York, NY: Cambridge University Press.

36.

Fristad

M. A.

Verducci

J. S.

Walters

Young

M. E.

(2009). Impact of multifamily psychoeducational psychotherapy in treating children aged 8 to 12 years with mood disorders. Archives of General Psychiatry, 66, 1013-1021. doi:10.1001/archgenpsychiatry.2009.112

37.

Glahn

D. C.

Bearden

C. E.

Caetano

Fonseca

Najt

Hunter

. . . Soares

J. C.

(2005). Declarative memory impairment in pediatric bipolar disorder. Bipolar Disorders, 7, 546-554. doi:10.1111/j.1399-5618.2005.00267.x

38.

Goldstein

T. R.

Miklowitz

D. J.

Mullen

K. L.

(2006). Social skills knowledge and performance among adolescents with bipolar disorder. Bipolar Disorders, 8, 350-361. doi:10.1111/j.1399-5618.2006.00321.x

39.

Goodwin

F. K.

Jamison

K. R.

(2007). Manic depressive illness: Bipolar disorders and recurrent depression (Vol. 1, 2nd ed.). New York, NY: Oxford University Press.

40.

Hale

J. B.

Fiorello

C. A.

(2004). School neuropsychology: A practitioner’s handbook. New York, NY: Guilford Press.

41.

Henin

Mick

Biederman

Fried

Wozniak

Faraone

S. V.

. . . Doyle

A. E.

(2007). Can bipolar disorder-specific neuropsychological impairments in children be identified? Journal of Consulting and Clinical Psychology, 75, 210-220. doi:10.1037/0022-006X.75.2.210

42.

L.-T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118

43.

Jensen

A. R.

(2004). Obituary: John Bissell Carroll. Intelligence, 32, 1-5. doi:10.1016/j.intell.2003.10.001

44.

Joseph

M. F.

Frazier

T. W.

Youngstrom

E. A.

Soares

J. C.

(2008). A quantitative and qualitative review of neurocognitive performance in pediatric bipolar disorder. Journal of Child and Adolescent Psychopharmacology, 18, 595-605.

45.

Kaufman

A. S.

Flanagan

D. P.

Alfonso

V. C.

Mascolo

J. T.

(2006). Test review: Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV). Journal of Psychoeducational Assessment, 24, 278-295. doi:10.1177/0734282906288389

46.

Kline

R. B.

(2004). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

47.

Lagace

D. C.

Kutcher

S. P.

Robertson

H. A.

(2003). Mathematics deficits in adolescents with bipolar I disorder. American Journal of Psychiatry, 160, 100-104. doi:10.1176/appi.ajp.160.1.100

48.

Lewinsohn

P. M.

Klein

D. N.

Seeley

J. R.

(2000). Bipolar disorder during adolescence and young adulthood in a community sample. Bipolar Disorders, 2, 281-293. doi:10.1034/j.1399-5618.2000.20309.x

49.

Marsh

H. W.

Hau

K.-T.

Grayson

(2005). Goodness of fit in structural equation models. In Maydeu-Olivares

McArdle

J. J.

(Eds.), Contemporary psychometrics: A festschrift for Roderick P. McDonald (pp. 275-340). Mahwah, NJ: Lawrence Erlbaum.

50.

Marsh

H. W.

Hau

K.-T.

Wen

(2004). In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320-341. doi:10.1207/s15328007sem1103_2

51.

McCarthy

Arrese

Mcglashan

Rappaport

Kraseski

Conway

. . . Tucker

(2004). Sustained attention and visual processing speed in children and adolescents with bipolar disorder and other psychiatric disorders. Psychological Reports, 95, 39-47. doi:10.2466/PR0.95.5.39-47

52.

McClure

E. B.

Treland

J. E.

Snow

Dickstein

D. P.

Towbin

K. E.

Charney

D. S.

. . . Leibenluft

(2005). Memory and learning in pediatric bipolar disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 461-469. doi:10.1097/01.chi.0000156660.30953.91

53.

McDonald

R. P.

(1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6, 97-103. doi:10.1007/bf01908590

54.

Meade

A. W.

Johnson

E. C.

Braddy

P. W.

(2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592. doi:10.1037/0021-9010.93.3.568

55.

Meyer

S. E.

Carlson

G. A.

Wiggs

E. A.

Martinez

P. E.

Ronsaville

D. S.

Klimes-Dougan

. . . Radke-Yarrow

(2004). A prospective study of the association among impaired executive functioning, childhood attentional problems, and the development of bipolar disorder. Developmental Psychopathology, 16, 461-476. doi:10.1017/S095457940404461X

56.

Olvera

R. L.

Semrud-Clikeman

Pliszka

S. R.

O’Donnell

(2005). Neuropsychological deficits in adolescents with conduct disorder and comorbid bipolar disorder: A pilot study. Bipolar Disorders, 7, 57-67. doi:10.1111/j.1399-5618.2004.00167.x

57.

Orvaschel

Puig-Antich

Chambers

Tabrizi

M. A.

Johnson

(1982). Retrospective assessment of prepubertal major depression with the Kiddie-SADS-E. Journal of American Academy of Child and Adolescent Psychiatry, 21, 392-397. doi:10.1016/S0002-7138(09)60944-4

58.

Papolos

D. F.

Papolos

(2002). The bipolar child: The definitive and reassuring guide to childhood’s most misunderstood disorder (2nd ed.). New York, NY: Broadway Books.

59.

Pavuluri

M. N.

O’Connor

M. M.

Harral

E. M.

Moss

Sweeney

J. A.

(2006). Impact of neurocognitive function on academic difficulties in pediatric bipolar disorder: A clinical translation. Biological Psychiatry, 60, 951-956. doi:10.1016/j.biopsych.2006.03.027

60.

Phelps

Mcgrew

K. S.

Knopik

S. N.

Ford

(2005). The general (g), broad, and narrow CHC stratum characteristics of the WJ III and WISC-III tests: A confirmatory cross-battery investigation. School Psychology Quarterly, 20, 66-88. doi:10.1521/scpq.20.1.66.64191

61.

Pomplun

Omar

M. H.

(2001). Score comparability of a state reading assessment across selected groups of students with disabilities. Structural Equation Modeling: A Multidisciplinary Journal, 8, 257-274. doi:10.1207/S15328007SEM0802_5

62.

Robinson

L. J.

Thompson

J. M.

Gallagher

Goswami

Young

A. H.

Ferrier

I. N.

. . . Moore

P. B.

(2006). A meta-analysis of cognitive deficits in euthymic patients with bipolar disorder. Journal of Affective Disorders, 93, 105-115. doi:10.1016/j.jad.2006.02.016

63.

Rucklidge

J. J.

(2006). Impact of ADHD on the neurocognitive functioning of adolescents with bipolar disorder. Biological Psychiatry, 60, 921-928. doi:10.1016/j.biopsych.2006.03.067

64.

Sivo

S. A.

Xitao

Witta

E. L.

Willse

J. T.

(2006). The search for “optimal” cutoff properties: Fit index criteria in structural equation modeling. Journal of Experimental Education, 74, 267-288. doi:10.3200/JEXE.74.3.267-288

65.

Small

N. J. H.

(1980). Marginal skewness and kurtosis in testing multivariate normality. Journal of the Royal Statistical Society Series C (Applied Statistics), 29(1), 85-87.

66.

Sörbom

(1974). A general method for studying differences in factor means and factor structure between groups. British Journal of Mathematical and Statistical Psychology, 27, 229-239.

67.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70. doi:10.1177/109442810031002

68.

Voelbel

G. T.

Bates

M. E.

Buckman

J. F.

Pandina

Hendren

R. L.

(2006). Caudate nucleus volume and cognitive performance: Are they related in childhood psychopathology? Biological Psychiatry, 60, 942-950. doi:10.1016/j.biopsych.2006.03.071

69.

Walshaw

Alloy

Sabb

(2010). Executive function in pediatric bipolar disorder and attention-deficit hyperactivity disorder: In search of distinct phenotypic profiles. Neuropsychology Review, 20, 103-120. doi:10.1007/s11065-009-9126-x

70.

Wechsler

(1974). Wechsler intelligence scale for children (Revised ed.). New York, NY: Psychological Corporation.

71.

Wechsler

(1991a). Wechsler intelligence scale for children (3rd ed.). San Antonio, TX: Psychological Corporation.

72.

Wechsler

(1991b). Wechsler intelligence scale for children-third edition examiner’s manual. San Antonio, TX: Psychological Corporation.

73.

Widaman

K. F.

Reise

S. P.

(1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In Bryant

K. J.

Windle

West

S. G.

(Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association.

74.

Woodcock

R. W.

Mcgrew

K. S.

Mather

(2001). Woodcock-Johnson tests of cognitive abilities (3rd ed.). Itasca, IL: Riverside.

75.

C.-Y.

(2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. Available from ProQuest Dissertations and Theses database. (Document No. AAT 3066425)