WAIS-IV and Clinical Validation of the Four- and Five-Factor Interpretative Approaches

Abstract

The fourth edition of the Wechsler Adult Intelligence Scale (WAIS-IV) is a revised and substantially updated version of its predecessor. The purposes of this research were to determine the constructs measured by the test and the consistency of measurement across large normative and clinical samples. Competing higher order WAIS-IV four- and five-factor models were analyzed using the WAIS-IV’s sample of 1,800 normative adults and 411 clinical adults. When all 15 WAIS-IV subtests were considered, both four- and five-factor models were suitable, but the five-factor model provided a better fit. The WAIS-IV PRI differentiated into two composites as follows: POI(Gv) consisting of Block Design, Visual Puzzles and Picture Completion; and FRI(Gf) consisting Matrix Reasoning, Arithmetic and Figure Weights. The five-factor solution included Quantitative Reasoning (RQ), consisting of Arithmetic and Figure Weights, as a narrow ability subsumed under FRI(Gf). Arithmetic, Vocabulary, and Figure Weights subtests had the highest g loadings. Cancellation had the lowest g loading. The WAIS-IV generally demonstrated full factor invariance between clinical and nonclinical samples.

Keywords

measurement invariance MG-MACS Wechsler scales

Wechsler tests are by far the most popular tests of intelligence in the world (Camara, Nathan, & Puente, 2000; Georgas, van de Vijver, Weiss, & Saklofske, 2003; Lichtenberger & Kaufman, 2009). The fourth edition of the Wechsler Adult Intelligence Scale (WAIS-IV; Wechsler, 2008a, 2008b) is a revised and updated version of its predecessor. It provides a general intelligence composite score and four scores corresponding to first-order factors: Verbal Comprehension (VCI), Perceptual Reasoning (PRI), Working Memory (WMI), and Processing Speed (PSI).

Seventy years ago, Wechsler (1939) included aspects from two primary theories of intelligence, the views of Spearman and Thorndike, and defined intelligence in practical terms as

the aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his [or her] environment. It is global because it characterizes the individual’s behavior as a whole; it is an aggregate because it is composed of elements or abilities which, though not entirely independent, are qualitatively differentiable. (Wechsler, 1939; 1944, p. 3)

Wechsler clearly acknowledged the existence of a global intelligence and that global intelligence is composed of qualitatively different abilities. Wechsler’s foresight is indeed relevant and practical in light of contemporary views on intelligence, which generally support a hierarchical model of cognitive abilities. General intelligence (g) tends to emerge whenever a sufficient number of cognitively complex variables are analyzed (Carroll, 1993). During Wechsler’s lifetime, he expressed the concern that measuring abilities that are too narrow could be impractical, and instead embraced the use of broader domain or composite scores that are supported by both factor-analytic research and clinical utility (Cohen, 1957, 1959; Kaufman, 1975; Wechsler, 1958). The various tasks of cognitive ability that he selected, including measures of verbal comprehension, perceptual organization, working memory, and processing speed—although not known by these names at the time—continue to align with current research into the structure of cognitive abilities (see, for example, Kane, Hambrick, & Conway, 2005) and recent theories of intelligence (Carroll, 1993; Sternberg, 2000).

Among the various cognitive theories, the Cattell-Horn-Carroll model (CHC; Carroll, 1993, 2005; Schnieder & McGrew, in press) is often considered a suitable framework for exploring and comparing the nature of cognitive instruments (Keith & Reynolds, 2010). Resembling Wechsler’s hierarchical view but with more expanded details, the CHC model locates cognitive abilities into three structural levels. On top of the CHC hierarchy is g. In the middle level are 7 to 10 broad abilities [i.e., crystallized intelligence (Gc), fluid intelligence (Gf), quantitative knowledge (Gq), short-term memory (Gsm), long-term retrieval (Glr), visual processing (Gv), auditory processing (Ga), processing speed (Gs), reading and writing ability (Grw), and decision-reaction time-speed (Gt)]. There are more than 70 narrow abilities on the base level, and extension of the model is ongoing (McGrew, 2005; McGrew & Flanagan, 1998; Schneider & McGrew, in press).

Wechsler’s tests have gone through progressive revisions to update the theoretical foundations based on cumulative research findings in neuropsychology, executive functions, and working memory (Coalson, Raiford, Saklofske, & Weiss, 2010; Weiss, Saklofske, Coalson, & Raiford, 2010a). In particular, the most notable change involves the evolution of the composites/index scores. The emergence of the four-index structure (verbal comprehension, perceptual organization, working memory, and processing speed) can be traced to the WISC-III (Wechsler, 1991). Since fluid reasoning has been emphasized as a key aspect of cognitive functioning (Carroll, 1997; Cattell, 1943, 1963; Cattell & Horn, 1978; Sternberg, 1995, 2000), one of the continuous efforts made has been to incorporate new subtests for enhancing the scale’s measurement capabilities of fluid reasoning, including new subtests such as Matrix Reasoning (Wechsler, 1997), Picture Concepts (Wechsler, 2002, 2003), and Figure Weights (Wechsler, 2008b). The increased emphasis on fluid reasoning was also represented by renaming of the Perceptual Organization Index as the Perceptual Reasoning Index in the WISC-IV (Wechsler, 2003) and WAIS-IV (Wechsler, 2008a). The PRI is currently a measure of fluid reasoning, spatial processing, attentiveness to detail, and visual–motor integration (Wechsler, 2008a). As the Wechsler tests continue to evolve, further changes to PRI seem relevant. That is, is there a need to further differentiate the components inside PRI? Would it be clinically advantageous to regroup PRI into two separate factors: fluid reasoning and perceptual organization?

Several studies have explored the WAIS-IV factor model. Bensen, Hulac, and Kranzler (2010) argued for the clear superiority of a CHC derived five-factor model in which Matrix Reasoning, Figure Weights, and Arithmetic loaded on a fifth factor they termed fluid reasoning. Ward, Bergman, and Herbert (2011) proposed a modification of the WAIS-IV four factor model in which residual factors were formed for visual-spatial organization (Block Design, Visual Puzzles, and Picture Completion) and quantitative reasoning (Figure Weights and Arithmetic). Ward and colleagues argued that the modified model was theoretically consistent with the original WAIS-IV model and that there was no compelling statistical reason to prefer the Bensen et al. five-factor model over a modified four-factor model. Conceptually, such a model is equivalent to one with correlated first-order visual-spatial and quantitative reasoning factors.

When comparing models, another important validity issue is measurement invariance. Invariance is a fundamental property for any measure (Drasgow, 1984, 1987; Horn & McArdle, 1992; Vandenberg & Lance, 2000). It assumes that the test measures the same constructs in different groups. Meaningful comparisons can only be made if the measures are comparable (Chen, Sousa, & West, 2005). In empirical settings, the Wechsler Intelligence Scales are often used as part of a diagnostic assessment (Rabin, Barr, & Burton, 2005; Weiss, Saklofske, Coalson, & Raiford., 2010b). Implicit in this common practice is the assumption that WAIS-IV index scores and subtests results have the same meaning for adults in both normative and clinical populations. That is, equivalence is assumed to hold for underlying theoretical structures, factor patterns, and magnitudes of factor loadings, and the same subtest intercepts given the same latent means for the underlying factors. Measurement equivalence of WAIS-IV across large normative and clinical samples has never been reported.

This research employed a large sample with a large degree of variation to investigate construct validity of the WAIS-IV. The purpose of this study was threefold. First, the constructs underlying the WAIS-IV were investigated by comparing its current four-factor structure to a hypothesized five-factor structure. Second, abilities measured by subtests and possible cross-loadings were tested and verified for both structures. Finally, measurement invariance was evaluated to test whether the 15 subtests in WAIS-IV measure latent abilities in the same way for adults in the normative sample and those in the clinical sample. We used WAIS-IV data from the normative and clinical samples to examine invariance of these models with a multigroup higher order confirmatory analysis of mean and covariance structures (MG-MACS). This analysis will subsequently be referred to as the “clinical invariance” analyses.

Method

Participants

We used the WAIS-IV standardization sample as our nonclinical sample. It includes the responses of 1,800 normal adults 16 to 69 years old. Standardization cases from age 70 to 90 were excluded because they did not take all 15 WAIS-IV subtests. This normative sample was carefully selected by the WAIS-IV to match the U.S. 2005 census for demographics, such as region, gender, education, and ethnicity. The overall mean Full-scaled IQ (FSIQ) was 100 (SD = 15). For all 15 subtests, ranges for means were 9.97 to 10.09, standard deviations, 2.88 to 3.12, skewness −0.27 to 0.56, and kurtosis −0.53 to 1.94. Mean age was 37.2 (SD = 17.3). A more detailed description of this normative sample is in the WAIS-IV technical manual (Wechsler, 2008b).

We also used the WAIS-IV clinical sample. This heterogeneous group included 411 adults with various clinical diagnoses. These adults had mild or moderate intellectual disability (25.3%); borderline intellectual functioning (6.6%); reading or math learning disabilities (18.2%); ADHD (10.7%); traumatic brain injury (5.4%); autistic or Asperger’s Disorder (13.6%); major depressive disorder (9.0%); mild cognitive impairment (4.1%); probable Mild dementia or Alzheimer’s (1.9%); and other disabilities (5.2%). The mean FSIQ in the overall clinical sample was 80.7 (SD = 20.4). For all 15 subtests, descriptive statistics were means, 6.39 to 7.75; standard deviations, 3.28 to 3.87; skewness, 0.10 to 0.58; and kurtosis, −0.98 to 0.30. The average age (31.3) was similar to that of the normative sample (SD = 16.5). A detailed description of this clinical sample is in the WAIS-IV technical manual (Wechsler, 2008b).

Instrumentation

The WAIS-IV has 10 core subtests (Similarities [SI], Vocabulary [VC], Information [IN], Block Design [BD], Matrix Reasoning [MR], Visual Puzzles [VP], Digit Span [DS], Arithmetic [AR], Coding [CD], Symbol Search [SS]), and five supplemental subtests (Comprehension [CO], Figure Weights [FW], Picture Completion [PC], Letter-Number Sequencing [LN], and Cancellation [CA]).

Analysis

We analyzed all subtests of the WAIS-IV. Tests for the higher order confirmatory factor structure were based on analysis of mean and covariance structure models using LISREL 8.8 (Jöreskog & Sörbom, 2006). Both four- and five-factor models with hypothesized cross loadings were tested individually.

In addition to testing a higher order g, the initial four-factor structure specified four verbal comprehension subtests (SI, VC, CO, IN) on the first factor, five perceptual reasoning subtests (BD, MR, VP, FW, PC) on the second factor, three working memory subtests (DS, LN, AR) on the third factor, and three processing speed subtests (CD, SS, CA) on the fourth factor. This second-order model was defined as the initial four-factor model (Model A1 in Table 1).

Table 1.

Hypotheses Testing the Four-Factor Structure.

Model	χ²	df	Δχ²	Δdf ^a	p	CFI	RMSEA	RMSEA 90%CI	SRMR	AIC	aBIC
Calibration (n = 1,200)
A1. Initial four-factor model (AR on WMI)	610.93	86	—	—	—	.98	.071	[.066, .077]	.041	678.93	743.99
A2. Arithmetic on PRI	639.70	86	—	—	—	.98	.073	[.068, .079]	.039	707.70	772.77
A3. Arithmetic on WMI, VCI	511.84	85	99.09	1	<.0001	.98	.065	[.059, .010]	.037	581.84	648.82
A4. Arithmetic on WMI, PRI	522.18	85	88.75	1	<.0001	.99	.065	[.060, .071]	.038	592.18	659.16
A5. Arithmetic on PRI, WMI, VCI	481.67	84	129.26	2	<.0001	.99	.063	[.057, .068	.036	553.67	622.56
A6. Similarities on VCI, PRI	600.39	85	10.54	1	.001	.98	.071	[.066, .077]	.041	670.39	737.37
A7. Figure weights on PRI, WMI	571.20	85	39.73	1	<.0001	.98	.069	[.064, .074]	.040	641.20	708.18
A8. Picture Completion on PRI, VCI	613.31	85	2.38	1	.1228	.98	.072	[.067, .077]	.041	683.31	750.29
A9. Coding on PSI, WMI	594.71	85	16.22	1	<.0001	.98	.071	[.065, .076]	.041	664.71	731.69
A10. Symbol Search on PSI, PRI	610.93	85	0.00	1	1.0000	.98	.072	[.067, .077]	.041	680.93	747.91
A11. Cancellation on PSI, WMI	611.89	85	0.96	1	.3272	.98	.072	[.067, .077]	.041	681.89	748.87
A12. Cancellation on PSI, PRI	610.84	85	0.09	1	.7642	.98	.072	[.067, .077]	.041	680.84	747.82
A13. correlated error between AR and FW	563.51	85	47.42	1	<.0001	.98	.069	[.063, .074]	.041	633.51	700.49
Validation in combination (n = 600)
B1. AR,FW,SIM,CD cross loaded & correlated error between AR and FW	219.54	80	—	—	—	.99	.054	[.046, .063]	.040	299.54	348.43
Final overall testing (total N = 1,800)
C1. initial four-factor model	828.46	86	—	—	—	.98	.069	[.065, .074]	.040	896.46	975.29
C2. AR cross loaded	638.29	84	190.17^b	2	<.0001	.99	.061	[.056, .065]	.035	710.29	793.76

Note. ^aCompared to the Model A1, unless otherwise noted. ^bCompared to Model C1.

Source: Data and table copyright Pearson 2013.

The initial five-factor model specified a higher order g and five first-order factors. This initial model was based on evaluation of subtest content and previous research (Benson, Hulac, & Kranzler, 2010; Keith et al., 2006; Lichtenberger & Kaufman, 2009). This model specified four subtests (SI, VC, CO, IN) on the verbal comprehension factor (VCI/Gc), three subtests (BD, VP, PC) on the perceptual organization factor (POI/Gv) factor, three subtests (MR, FW, AR) on the fluid reasoning factor (FRI/Gf), two subtests (DS, LN) on the working memory factor (WMI/Gsm), and three subtests (CD, SS, CA) on the processing speed factor (PSI/Gs). The five-factor model includes both Wechsler-related and CHC-related names to allow comparability. Compared to the four-factor model, this five-factor model split the five perceptual reasoning subtests into fluid reasoning and perceptual organization factors and specified the Arithmetic subtest on the fluid reasoning factor (Model A1 in Table 2).

Table 2.

Hypotheses Testing the Five-Factor Structure.

Model^C	χ²	df	Δχ²	Δdf ^a	p	CFI	RMSEA	RMSEA 90%CI	SRMR	AIC	aBIC
Calibration (n = 1,200)
A1. Initial five-factor model (AR on FRI)	401.86	85	—	—	—	.99	.056	[.050, .061]	.035	471.86	538.84
A2. POI and FRI combined	639.70	86	237.84	1	<.0001	.98	.073	[.068, .079]	.039	707.70	772.77
A3. correlated error between FRI and POI	398.12	84	3.74	1	.0531	.99	.056	[.050, .061]	.035	470.12	539.01
A4. Arithmetic on WMI	489.26	85	—	—	—	.99	.063	[.058, .068]	.038	559.26	626.24
A5.Arithmetic on FRI, WMI	362.58	84	39.28	1	<.0001	.99	.053	[.047, .058]	.033	434.58	503.47
A6.Arithmetic on FRI, VCI	394.25	84	7.61	1	.0058	.99	.056	[.050, .061]	.034	466.25	535.14
A7.Arithmetic on WMI, VCI	394.82	84	7.04	1	.0079	.99	.056	[.050, .061]	.033	466.82	535.71
A8.Arithmetic on FRI, WMI, VCI	347.25	83	54.61	2	<.0001	.99	.052	[.046, .057]	.032	421.25	492.06
A9. Similarities on VCI, FRI	397.34	84	4.52	1	.0335	.99	.056	[.050, .061]	.035	469.34	538.23
A10. Figure weights on FRI, POI	385.66	84	16.20	1	<.0001	.99	.055	[.049, .060]	.035	457.66	526.55
A11. Matrix Reasoning on FRI, POI	381.88	84	19.98	1	<.0001	.99	.054	[.049, .060	.035	453.88	522.77
A12. Block Design on POI, FRI	399.77	84	2.09	1	.1483	.99	.056	[.051, .062]	.034	471.77	540.66
A13. Picture Completion on POI, VCI	400.70	84	1.16	1	.2815	.99	.056	[.051, .062]	.034	472.70	541.59
A14. Coding on PSI, WMI	388.97	84	12.89	1	.0003	.99	.055	[.050, .061]	.035	460.97	529.86
A15. Symbol Search on PSI, POI	394.24	84	7.62	1	.0058	.99	.056	[.050, .061]	.035	466.24	535.13
A16. Cancellation on PSI, WMI	401.30	84	0.56	1	.4543	.99	.056	[.051, .062]	.034	473.30	542.19
A17. Cancellation on PSI, POI	396.90	84	4.96	1	.0259	.99	.056	[.050, .061	.034	468.90	537.79
A18. testing RQ as a narrow ability under FRI	397.75	84	4.11	1	.0426	.99	.056	[.050, .061]	.035	469.75	538.64
Validation in combination (n = 600)
B1. AR,FW, SI, MR, CA cross loaded/RQ under FRI	176.88	78	—	—	—	.99	.046	[.037, .055]	.033	260.88	312.21
Final overall testing (total N = 1,800)
C1. initial five-factor model	549.68	85	—	—	—	.99	.055	[.051, .060]	.034	619.68	700.83
C2. AR,FW,MR cross loaded/RQ under FRI	431.35	81	118.33^b	4	<.0001	.99	.049	[.045, .054]	.033	509.35	599.78

Compared to the Model A1, unless otherwise noted. ^bCompared to Model C1. ^CVCI(Gc); FRI(Gf); POI(Gv); WMI(Gsm); PSI(Gs).

Source: Data and table copyright Pearson 2013.

Alternative competing models of both structures were also tested. Possible cross-loadings and correlated errors were investigated for selected subtests thought to measure multiple abilities or that showed split loadings in previous studies. Parameters statistically nonsignificant or having values too trivial to be practically meaningful were deleted. For all analyses, a calibration-validation approach was used where two thirds of the normative sample (n = 1,200) was randomly selected as the calibration sample to test hypotheses and the remaining third (n = 600) was used to cross-validate the results of calibration analyses. Any statistically significant factor loading greater than .10 was explored in the calibration phase, and the cut point was set at .20 in the cross-validation phase to reduce the loadings to nontrivial ones only. Once a best-fitting solution from each of the four- and the five-factor models was calibrated and validated, final parameters were retested using the entire sample (n = 1,800).

Clinical invariance of each final validated model was examined. Six levels of nested models were tested to investigate the degree of invariance. Each level had more constraints than the previous level (Byrne & Stewart, 2006; Chen et al., 2005; Keith & Reynolds, in press; Meredith, 1993). The initial and weakest level was configural invariance. It assumed the same number of factors and the same overall factor pattern across groups. The second level was first-order factor loading invariance, also called metric (or weak factorial) invariance. Loadings of subtests on factors were constrained so that factor loadings were equal across groups. When the factor loadings are equal, scales of latent variables are the same for both groups and the unit of measurement is identical. That is, for each unit change in latent variable, scores on subtests change by the same amount in both groups. The third level was intercept invariance, or also known as scalar (strong factorial) invariance. In this level of invariance, any group differences in subtest means are a result of true mean differences in latent factors. Subtests have the same intercepts across groups given the same latent means for an underlying factor. To examine whether “all group differences on the measured variables are captured by, and attributable to, group differences on the common factors” (Widaman & Reise, 1997, p. 296), we tested invariance of residuals, also called strict factorial invariance. These residuals are a combination of subtest-specific unique variance and random measurement errors. The fifth level was second-order factor loading invariance. This level assumed first-order latent factors show the same amount of change in each group for the same amount of increase in g. Finally, we tested for invariance of disturbances (factor unique variance) of first-order factors. Although residual/disturbance invariance is not fundamentally crucial for measurement invariance, it provides substantial information about human cognitive abilities across groups. The scale of latent factors was identified by fixing a factor loading of each factor to one.

Multiple indices of model fit were used to evaluate and compare the various models in this study (Bentler & Bonett, 1980; Hu & Bentler, 1998, 1999; Kline, 2005; Marsh, Balla, & McDonald, 1988). Single models were evaluated using the comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). An RMSEA less than .05 corresponded to a good fit, and .08 was considered an acceptable fit (McDonald & Ho, 2002). For completeness, we included the 90% confidence interval for RMSEA. SRMR values less than .08 were considered acceptable (Hu & Bentler, 1999). A value of .95 served as the cutoff for acceptable fit on all indices ranging from zero to 1, with 1 indicating a perfect fit (Hoyle & Panter, 1995; Hu & Bentler, 1999; Kline, 2005). Change in chi-square (Δχ²) was used to evaluate competing, nested models (Bentler & Bonett, 1980). The Akaike information criterion (AIC) and sample size adjusted Bayesian Information Criterion (aBIC) were also used for comparisons of nonnested models (Kaplan, 2000; Loehlin, 2004), with smaller values indicating a better fit. Comparatively, aBIC has a greater reward for parsimony than does the AIC.

To determine evidence of invariance, since there is little consensus concerning the most appropriate criterion (Byrne & Stewart, 2006), two perspectives were evaluated for invariance analyses: (a) the traditional perspective based on Δχ² and (b) the practical perspective based on differences in comparative fix index (ΔCFI). When evaluating the traditional perspectives, given the large sample and the number of comparisons being made, we used a more strict definition of statistical significance of Δχ² (p < .001). Comparatively, the Δχ² test is known to be sensitive to sample size and moderate discrepancies from normality (Kline, 2005; West, Finch, & Curran, 1995). Therefore, Cheung and Rensvold (2002) recommended ΔCFI as superior to Δχ² for tests of invariance because it is independent of both model complexity and sample size and because it is not correlated with the overall fit measures. “A value of ΔCFI smaller than or equal to –.01 indicates that the null hypothesis of invariance should not be rejected” (Cheung & Rensvold, 2002, p. 251).

Results

The Most Appropriate Four-Factor Structure

All examinations of the four-factor structure are shown in Table 1. In the calibration phase, goodness-of-fit indexes reported for the initial four-factor model (Model A1) were within the acceptable range, showing the four-factor structure fit data well. Each hypothesis was checked individually for cross-loadings and error correlations to determine whether subtests were pure or mixed measures of the four latent factors. Six originally unspecified parameters were found to yield statistically significant improvement in model fit: Arithmetic was found a mixed measure of WMI, PRI, and VCI. Loadings of Arithmetic on each of these three factors were .36, .29, and .20, respectively. The Δχ² between Model A5 and A1 was statistically significant, suggesting that, allowing Arithmetic to have cross-loadings on all three factors substantially improved model fit. Three other subtests also showed significant cross-loadings: Similarities loaded mainly on VCI (.74) and slightly on PRI (.10); Figure Weights was cross-loaded on PRI (.51) and WMI (.29); Coding mainly measured PSI (.69) but also some WMI (.12). Errors of Arithmetic and Figure Weights were found to show a trivial, but statistically significant, correlation (.10).

Validation analyses tested all six parameters in Model B1, but only Arithmetic showed statistically and practically meaningful cross-loadings (Table 1). Arithmetic was cross-loaded on WMI, PRI, and VCI (factor loadings were .39, .20, and .25 respectively). PRI showed the highest g loading across four first-order factors.

This validated model with cross-loaded AR was retested using the entire sample of 1,800 adults (Model C2). This validated model had a far superior fit compared to the initial four-factor model (Model C1). Goodness-of-fit indexes for both models were within ideal range.

Standardized estimates for Model C2 are shown in Figure 1. All 15 subtests loaded strongly on corresponding factors. Consistent with literature, Arithmetic was a mixed measure of WMI, PRI, and VCI (factor loadings were .36, .27, and .22 respectively). Across all four first-order factors, PRI had the highest g loading (.91). All parameter estimates were reasonable and theoretically sound.

Figure 1.

Standardized estimation for the final validated four-factor model.

The Most Appropriate Five-Factor Structure

Examinations of five-factor models are shown in Table 2. In the calibration phase, the initial five-factor model (Model A1) showed a good fit to the data, suggesting it was an appropriate model for interpreting WAIS-IV results. Each hypothesis of cross-loadings and related errors was individually tested before being cross-validated in combination. Calibration analyses supported separating FRI(Gf) and POI(Gv). Arithmetic was a mixed measure of FRI(Gf), WMI(Gsm), and VCI(Gc) (factor loadings were .39, .30, and .15 respectively). Four other subtests showed significant split-loadings above the .10 cutoff established for the calibration phase: Similarities on VCI(Gc) (.79) and FRI(Gf) (.11); Figure Weights on FRI(Gf) (.58) and POI(Gv) (.21); Matrix Reasoning on FRI(Gf) (.55) and POI(Gv) (.22); and Cancellation on PSI(Gs) (.48) and POI(Gv) (.11). Allowing a narrow ability Quantitative Reasoning (RQ) under FRI improved the model fit as well (cf. Ward et al., 2011).

When validated in combination and allowing a narrow ability RQ under FRI(Gf), only Arithmetic, Figure Weights, and Matrix Reasoning showed robust split loadings. When tested with the entire sample (n = 1,800), goodness-of-fit indexes for Model C2 suggested a good fit to the nonclinical adults (CFI = .99, RMSEA = .049, SRMR = .033). With four extra parameters allowed, Model C2 showed significant improvement compared to the initial five factor model (Model C1) in which no cross loadings were specified, Δχ² = 118.34(4), p < .0001.

Standardized estimates for Model C2 are shown in Figure 2. All 15 subtests loaded strongly on corresponding factors. Three subtests showed salient split loadings. Arithmetic was, as expected, a complicated measure of Quantitative Reasoning (RQ) and WMI(Gsm) (factor loadings were .62, and .23 respectively). Figure Weights measured both RQ and POI(Gv) (factor loadings were .53 and .29 respectively). Matrix Reasoning mainly measured FRI(Gf) and some POI(Gv) (factor loadings were .48 and .27 respectively). The highest g loading was on FRI(Gf) (.99).

Figure 2.

Standardized estimation for the final validated five-factor model.

Which Fit Better: Four- or Five-Factor Model?

Since values of Models C1 and C2 in Table 1 and Models C1 and C2 in Table 2 were within the ideal range, both four-factor and five-factor models provide a good fit to the data. Therefore, either of these models could serve as a basis for WAIS-IV score interpretation. Side-by-side comparison of two structures revealed the five-factor model showed a better fit. When comparing initial models (Model C1) from both sets, the aBIC values for four- and five-factor C1 models were 975.29 and 700.83, respectively. When cross-loadings were relaxed, the five-factor Model C2 still showed an overall better fit than the four-factor Model C2 (aBIC values were 793.76 and 599.78, respectively).

For these two C2 Models with cross-loadings, subtest loadings on g are shown in Table 3. Estimated values from both models were similar. Arithmetic, Vocabulary, and Figure Weights had the highest loadings on g. The Cancellation subtest had the lowest loading on g.

Table 3.

Loading of WAIS-IV Subtests on the Second-Order g Factor.

Subtest	Loading on g
Subtest	Four-factor model^a	Five-factor model^b
Similarities	.664	.672
Vocabulary	.721	.730
Information	.640	.648
Comprehension	.672	.681
Block design	.701	.656
Matrix reasoning	.655	.694
Visual puzzles	.692	.648
Figure weights	.701	.728
Picture completion	.546	.494
Digit span	.680	.689
Arithmetic	.712	.763
Letter-number sequencing	.640	.648
Symbol search	.545	.529
Coding	.545	.536
Cancellation	.386	.375

Model C2 in Table 1; ^bModel C2 in Table 2.

Source: Data and table copyright Pearson 2013.

Invariance Between Normative and Clinical Samples in Validated Four-Factor Model

The relatively large variation in sample sizes between the normative and clinical samples would result in excessive power in the standardization group. To prevent findings from being so heavily weighted toward the normative sample, we compared the clinical group (n = 411) with a randomly selected representative sample (n = 411) from the normative group.

Clinical invariance analyses for the final cross-validated four-factor model (Model C2 in Table 1) are reported in Table 4. Variance-covariance matrices were first constrained to be equal across normative and clinical samples (Model 1). This constrained model fit the data adequately (CFI = .99; RMSEA = .057), suggesting fairly invariant WAIS-IV subtest covariance patterns in adults. Since any factor structure is derived from these variance-covariance matrices, if the WAIS-IV measures the same constructs, factor structure between the normative and clinical samples should be similar.

Table 4.

Invariance Analyses of Four-Factor Model (Model C2 in Table 1) of Normative and Clinical Adults.

Model	χ²	df	CFI	RMSEA	RMSEA 90%CI	SRMR	AIC	aBIC	Model comparison	ΔCFI	Δχ²	Δdf	p

Phase I: Baseline model fit for each group
Normative (n = 411)	196.55	84	.99	.057	[.047, .068]	.041	268.55	298.98
Clinical (n = 411)	488.29	84	.98	.108	[.099, .120]	.035	560.29	590.72
Phase II: Measurement Invariance across groups
Model 1 Equality of variance-covariance matrices	281.30	120	.99	.057	[.049, .066]	.190	521.30	705.63
Model 2 configural	684.84	168	.98	.087	[.080, .093]	.035	888.84	1045.52	—	—	—	—	—
Model 3 first-order loadings	714.41	181	.98	.085	[.078, .091]	.045	892.41	1029.12	3 vs. 2	.00	29.57	13	.005
Model 4 first-order loadings and subtest intercepts	764.70	192	.98	.085	[.079, .092]	.051	920.70	1040.52	4 vs. 3	.00	50.29	11	.000
subtest intercepts-SS Freed	747.40	191	.98	.084	[.078, .091]	.049	905.40	1026.75		.00	32.99	10	.000
subtest intercepts-BD Freed	742.39	190	.98	.084	[.078, .091]	.049	902.39	1025.28		.00	27.98	9	.001
Model 5 first-order loadings, subtest intercepts, and subtest residual variances	775.63	205	.98	.082	[.076, .089]	.050	905.63	1005.48	5 vs. 4	.00	33.24	15	.004
Model 6 first-order loadings, subtest intercepts, residual variances, and second-order loadings	777.68	208	.98	.082	[.076, .088]	.051	901.68	996.92	6 vs. 5	.00	2.05	3	.562
Model 7 first-order loadings, subtest intercepts, residual variances, second-order loadings, and disturbances of first-order factors	785.42	212	.98	.081	[.075, .087]	.052	901.42	990.51	7 vs. 6	.00	7.74	4	.102

Note. Compare model fit with previous model, unless noted otherwise.

Source: Data and table copyright Pearson 2013.

First, the configural model (Model 2) provided an acceptable fit to the data.¹ Normative and clinical adults shared the same WAIS-IV first- and second-order four-factor patterns and corresponding subtests loaded onto the same factors. With the factor pattern established, we imposed cross-group constraints on first-order factor loadings. The fit was not substantially reduced (Model 3). The ΔCFI was 0. Using a .001 criterion, the Δχ² did not fit significantly worse than the configural model. This means that the subtests measure the same latent factors in both groups. Next, we constrained subtest intercepts to be equal. To properly identify this model, the means of first-order factors in the normative group were fixed to zero, but those in the clinical group were freed. Thus factor means for the clinical group represent mean differences. All corresponding first-order intercepts were constrained to be equal. The addition of intercept constraints significantly reduced fit according to the Δχ² but not according to ΔCFI and other indices. Sources of misfit were the intercepts of the Symbol Search and Block Design subtests. When these two parameters were freed, model fit improved substantially and no longer differed significantly compared to the first-order factor loading invariant model. Next, when subtest residuals and structural parameters (second-order loadings, first-order unique variances) were constrained as equal between groups in steps. There was no result in deterioration of fit. In fact, aBIC generally improved, suggesting these steps achieved measurement invariance.

The small size of the indicators of noninvariance for Symbol Search and Block Design should be interpreted in light of the complexity of the model and strictness of the test. We conclude that the WAIS-IV shows acceptable levels of invariance among factors between the normative and clinical groups. Differences in subtest scores on the WAIS-IV are generally due to latent constructs, and the test is not biased by clinical state². Those who take a more strict approach may consider that the group differences in means of the Symbol Search and Block Design subtests are not fully attributed to the mean differences in the model-specified latent factors. These differences could be a result of unmodeled minor factors. Estimated Symbol Search intercepts were 10.10 and 10.95 for normative and clinical adults, respectively. Estimated Block Design intercepts were 10.07 and 10.39, respectively. Given the same levels of Processing Speed and Perceptual Reasoning, the clinical group scored slightly higher on these two intercepts, showing that these tasks are slightly easier for this group than would be expected based on their Processing Speed and Perceptual Reasoning abilities. Again, these differences may be the result of these subtests measuring different narrow abilities compared to other subtests on these factors.

Invariance Between Normative and Clinical Samples in the Validated Five-Factor Model

Following a similar procedure, factorial invariance of the final validated five-factor model (Model C2 in Table2) was assessed. Fits of the various models are shown in Table 5. Evaluation of various indices, such as ΔCFI and aBIC, generally supported full measurement invariance. The five-factor model demonstrated good levels of factorial invariance between normative and clinical adults.³ When the more strict Δχ² criterion was considered, only one parameter was revealed as noninvariant: the Symbol Search intercept was freely estimated as 9.99 and 10.62 for the normative and clinical samples, respectively. Again, the clinical group was slightly higher on this intercept. Group difference on the Symbol Search subtest might not be fully attributed to the mean differences in the model specified latent factors.

Table 5.

Invariance analyses of five-factor model (Model C2 in Table 2) of normative and clinical adults.

Model	χ²	df	CFI	RMSEA	RMSEA 90%CI	SRMR	AIC	aBIC	Model comparison^a	ΔCFI	Δχ²	Δdf	p

Phase I: Baseline model fit for each group
Normative (n = 411)	165.97	81	.99	.051	[.040, .062]	.042	243.97	276.94
Clinical (n = 411)	480.83	81	.98	.110	[.100, .120]	.037	558.83	591.80
Phase II: Factor Invariance across groups
Model 1 equality of variance/covariance matrices	281.30	120	.99	.057	[.049, .066]	.190	521.30	705.63
Model 2 configural	646.80	162	.99	.085	[.079, .092]	.037	862.80	1028.70	—	—	—	—	—
Model 3 first-order loadings	670.90	174	.98	.083	[.077, .090]	.047	862.90	1010.37	3 vs. 2	.01	24.10	12	.020
Model 4 first-order loadings and subtest intercepts	706.76	183	.98	.084	[.077, .090]	.048	880.76	1014.40	4 vs. 3	.00	35.86	9	.000
Subtest intercept-SS Freed^b	697.91	183	.98	.083	[.076, .089]	.046	871.91	1005.55		.00	27.01	9	.001
Model 5 first-order loadings, subtest intercepts, and subtest residual variances	732.62	198	.98	.081	[.075, .087]	.047	876.62	987.22	5 vs. 4	.00	34.71	15	.003
Model 6 first-order loadings, subtest intercepts and residual variances, and higher-order loadings	746.31	203	.98	.081	[.075, .087]	.052	880.31	983.23	6 vs. 5	.00	13.69	5	.018
Model 7 first-order loadings, subtest intercepts and residual variances, second-order loadings, and disturbances of higher-order factors	764.04	209	.98	.080	[.074, .087]	.054	886.04	979.74	7 vs. 6	.00	17.73	6	.007

Compare model fit with previous model, unless noted otherwise. ^bThe model could not be estimated without an additional constraint. We fixed the latent mean of the PSI factor to the value found in the precious model.

Source: Data and table copyright Pearson 2013.

Primary and secondary interpretations of each WAIS-IV subtests are shown in Table 6 based on the pattern of factor loadings observed.

Table 6.

Primary and Secondary Abilities Measured by WAIS-IV Subtests.

Subtest	Primary interpretation	Secondary interpretation
Similarities	Verbal comprehension/crystallized intelligence (Gc)
Vocabulary	Verbal comprehension/crystallized intelligence (Gc)	g
Comprehension	Verbal comprehension/crystallized intelligence (Gc)
Information	Verbal comprehension/crystallized intelligence (Gc)
Block design	Perceptual organization/visual processing (Gv)
Matrix reasoning	Fluid reasoning/fluid intelligence (Gf)	Perceptual organization/visual processing (Gv)
Visual puzzles	Perceptual organization/visual processing (Gv)
Figure weights	Fluid reasoning/quantitative reasoning (RQ)	Perceptual Organization/visual processing (Gv), g
Picture completion	Perceptual organization/visual processing (Gv)
Digit span	Working memory/short-term memory (Gsm)
Arithmetic	Fluid reasoning/quantitative reasoning (RQ)	g, Working memory/short-term memory (Gsm), Verbal comprehension/crystallized intelligence (Gc)
Letter-number sequencing	Working memory/short-term memory (Gsm)
Symbol search	Processing speed (Gs)
Coding	Processing speed (Gs)
Cancellation	Processing speed (Gs)

Source: Data and table copyright Pearson 2013.

Discussion

This study is important because it is the first study to evaluate the clinical validity of an alternative five-factor measurement model for WAIS-IV. The four-factor interpretive approach is the model published with the test’s technical manual (Wechsler, 2008). The five-factor model reorganizes the 15 WAIS-IV subtests into five factors with the additional factor measuring fluid reasoning.

The first major set of findings in this study is that both four- and five-factor models fit the data well and provide meaningful strategies for interpreting WAIS-IV scores. Fit statistics were within the ideal range for both models. Comparatively, the five-factor model fit better, providing psychometric support for separating the WAIS-IV PRI into two composites as follows: POI(Gv) consisting of Block Design, Visual Puzzles, and Picture Completion; and FRI(Gf) consisting Matrix Reasoning, Arithmetic, and Figure Weights. Our five-factor solution included Quantitative Reasoning (RQ), consisting of Arithmetic and Figure Weights, as a narrow ability subsumed under FRI(Gf).

The second and most important set of finding is that both 4 and 5 factor models derived from the normative data provide a good fit to the clinical data. Invariance analyses generally supported invariance of both models between normative and clinical groups. The WAIS-IV 15-subtests demonstrated the same underlying theoretical latent constructs, strength of relations among factors and subtests, validity of each of the first-order factors, and the same subtest intercepts and communalities regardless of clinical status. Thus each model demonstrated nearly full factorial invariance between clinical and nonclinical samples. This means that whichever model is used, the WAIS-IV subtests have the same meaning regardless of clinical status.

The clinical sample scored slightly higher on both the Symbol Search and Block Design subtests in the four-factor model, and slightly higher on Symbol Search in the five-factor model. Although the strict criterion identified these two subtest intercepts as noninvariant in the clinical sample, the small magnitude of the differences identified likely have little clinical meaning given model complexity and the strictness of parameter constraints, and do not jeopardize overall WAIS-IV invariance. One probable reason for the finding is that these subtests also measure some degree of narrow abilities not modeled by the factor structures shown. In practice, however, factor loading invariance is often seen as the most important step in invariance testing, whereas complete intercept invariance is hard to fulfill (Reynolds & Keith, in press) and subtest intercept variation does occur (Cooke, Kosson, & Michie, 2001; Immekus & Maller, 2010; Maller & French, 2004). Thus we believe that this variation does not preclude the usefulness of these subtests for measuring the proposed latent abilities in clinical samples.

A third set of major findings concerns clarification of multiple abilities measured by some subtests as evidenced by cross loadings on more than one factor, and confirmation of subtest g loadings. As expected, the Arithmetic, Figure Weight, and Matrix Reasoning subtests measured multiple abilities. Arithmetic, Vocabulary, and Figure Weights had the highest g loadings. Cancellation had the lowest g loading.

The Arithmetic subtest likely requires examinees to integrate a complex mix of abilities in both adult (Benson et al., 2010; Bowden, Saklofske, & Weiss, 2011; Bowden, Weiss, Holdnack, & Lloyd, 2006; Lichtenberger & Kaufman, 2009; Ward et al., 2011) and child populations (Chen, Keith, Chen, & Chang, 2009; Keith et al., 2006). In this study, Arithmetic loaded mainly on working memory (.36), with some perceptual reasoning (.27) and verbal comprehension (.22) in the four-factor model. In the five-factor model, Arithmetic mainly measured quantitative reasoning (.62), which is a narrow ability under fluid reasoning, and some working memory (.23). These results are consistent with empirical findings presented by Keith, who found loadings for Arithmetic on Gf, Gsm, and Gc as .34, .31, and.19, respectively (Lichtenberger & Kaufman, 2009, p. 32).

Our findings are also consistent with current research into the theoretical structure of intelligence, which documents considerable shared variance between working memory and fluid reasoning (Conway, Cowan, Bunting, Therriault, & Minkoff, 2002; de Jong & Das-Smaal, 1995; Engle, Tuholski, Laughlin, & Conway, 1999; Fry & Hale, 1996, 2000; Kane et al., 2005). Specifically, the cognitive control mechanisms involved in working memory have been identified as the source of the link between working memory and fluid intelligence (Engel de Abreu, Conway, & Gathercole, 2010). This potentially explains the cross loading of the Arithmetic subtest on FRI(Gf) and WMI(Gsm), and the movement of Arithmetic from the WMI to the FRI in the five-factor solution. The minor loading of Arithmetic on VCI(Gc) in the four-factor model is readily explained by the verbal nature of the Arithmetic word problems.

Two other subtests, Figure Weights and Matrix Reasoning, measured mixed abilities in the five-factor model. Figure Weights loaded on perceptual reasoning in the four-factor model. In the five-factor model, Figure Weights measured mainly quantitative (fluid) reasoning (.53) and some perceptual organization (.29). In this subtest, the examinee is asked to view a scale with missing weights and to select the option that keeps the scale balanced (Wechsler, 2008a). Based on the response processes required by this task, it seems reasonable for this subtest to involve fluid reasoning, working memory, visual processing, and the narrow ability of quantitative reasoning. Matrix Reasoning loaded on the PRI in the four-factor model. In the five-factor model, Matrix Reasoning measured both FRI(Gf) and POI(Gv) (factor loadings were .48 and .27, respectively), a result also observed by Keith et al. (2006).

The finding that some subtests cross load on more than one factor does not necessarily indicate lack of content validity. Cognitive abilities are interrelated in nature. Moreover, data from Figures 1 and 2 show that for every subtest the factor loading for each main ability was clearly higher than its loading for secondary abilities. The point of the discussion regarding cross loadings is to clarify the constructs measured by the subtests relative to the factors so as to inform clinical interpretation of unusual patient profiles. Proposed interpretive hypotheses for each subtest based on its primary and secondary factor loadings are shown in Table 6 above. Given consistent subtest scores within indexes, the column labeled “primary interpretation” is probably the best interpretation of constructs measured by each subtest. When inconsistencies are found, or when examiners wish to test specific hypotheses, the abilities listed in the column “secondary interpretation” may be worth considering.

Overall, our findings have important applied implications for clinical interpretation of WAIS-IV results. Acceptability of the four- and five-factor models in both the normative and clinical samples suggests that these are useful and complementary models for interpreting WAIS-IV findings. For adults with consistent subtest scores within each of the four WAIS-IV composites, the current four WAIS-IV Index scores constitute an appropriate level of interpretation. For adults with discrepant subtest scores within some of the four composites, the five-factor model suggests a likely interpretive reorganization. For those patients who present subtest scatter within the PRI or WMI composite, our findings suggest that a common pattern may be consistencies between POI(Gv) subtests (i.e., Block Design, Visual Puzzles, and Picture Completion), and between FRI(Gf) subtests (i.e., Matrix Reasoning, Arithmetic, and Figure Weights)—but inconsistencies across these two factors. Furthermore, when adults present subtest scatter within the FRI(Gf) factor, our data suggest interpretation of a narrow ability under FRI(Gf) known as Quantitative Reasoning (RQ) (i.e., Arithmetic and Figure Weights). Furthermore, our data suggest that this interpretative approach should be equally applicable for clinical patients and examinees from the general population.

While current results provide solid information for understanding the structure behind the 15 WAIS-IV subtests between normative and clinical adults, it is nonetheless important to note that this study focused primarily on the broad factors and generally did not tap the domains of narrow abilities. Joint CFA with other measures may provide a more complete picture of all abilities measured.

The choice of four- or five-factor models is somewhat controversial in the Wechsler literature. For example, based on the U.S. adult data, Keith reported that “The WAIS-IV four-factor structure fits better than the CHC model when Arithmetic is excluded” (Lichtenberger & Kaufman, 2009, p. 32). Chen et al. (2009) also reported that the superiority of the five-factor model over the four-factor model is somewhat less salient in a population of Asian children: “Taiwanese children revealed more strongly correlated fluid reasoning and visual-spatial processing factors than do American children. The correlation between visual-spatial processing and working memory was also higher (Chen et al., 2009. p. 100).” Therefore, many others variables, such as culture and subtest combination, should be considered when studying factor structures.

In conclusion, our results support the model-data fit for both four- and five-factor WAIS-IV models. Both models explained the data well. Thus both are plausible and have psychometric merit. Results also confirmed findings by Benson et al (2010): that the five-factor model fit better than did the four-factor model for the 15-subtest set, validating the approach for separating FRI(Gf) and POI(Gv) and for loading Arithmetic primarily on fluid reasoning factor. Moreover, factor invariance of the WAIS-IV between normative and clinical samples was also supported. The subtests generally appear to measure the same abilities in both normative and clinical adults, supporting meaningful comparisons of WAIS-IV between these two samples. As Prifitera, Weiss, Saklofske, and Rolfhus (2005) suggested, while factor analysis is a useful tool for informing the best way to interpret relations among subtests, clinical utility should always be considered when selecting factors. Accumulating and balancing validity evidence from psychometric and clinical perspectives should continue.

Footnotes

Declaration of Conflicting Interests

Drs. Weiss, Zhu, and Chen were involved in the research and development of the WAIS-IV and WISC-IV as employees of Pearson, which is the publisher of numerous psychological tests including the Wechsler scales.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Benson

Hulac

D. M.

Kranzler

J. H.

(2010). Independent examination of Wechsler Adult Intelligence Scale–Fourth edition (WAIS-IV): What does the WAIS-IV measure? Psychological Assessment, 22(1), 121-130.

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Bowden

S. C.

Saklofske

D. H.

Weiss

L. G.

(2011). Invariance of the measurement model underlying the Wechsler Adult Intelligence Scale-IV in the United States and Canada. Educational and Psychological Measurement, 71, 186-199.

Bowden

S. C.

Weiss

L. G.

Holdnack

J. A.

Lloyd

(2006). Age-related invariance of abilities measured with the Wechsler Adults Intelligence Scale-III. Psychological Assessment, 18, 334-339.

Byrne

B. M.

Stewart

S. M.

(2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13, 287-321.

Camara

W. J.

Nathan

J. S.

Puente

A. E.

(2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141-154.

Carroll

J. B.

(1993). Human cognitive abilities: A survey of factor analytic studies. New York, NY: Cambridge University Press.

Carroll

J. B.

(1997). The three-stratum theory of cognitive abilities. In Flanagan

D. P.

Genshaft

J. L.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 122-130). New York, NY: Guilford.

Carroll

J. B.

(2005). The three-stratum theory of cognitive abilities. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 69-76). New York, NY: Guilford.

10.

Cattell

R. B.

(1943). The measurement of adult intelligence. Psychological Bulletin, 40, 153-193.

11.

Cattell

R. B.

(1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1-22.

12.

Cattell

R. B.

Horn

J. L.

(1978). A check on the theory of fluid and crystallized intelligence with description of new subtest design. Journal of Educational Measurement, 15, 139-164.

13.

Chen

Keith

Chen

Chang

(2009). What does the WISC-IV measure? Validation of the scoring and CHC-based interpretative approaches. Journal of Research in Education Sciences, 54(3), 85-108.

14.

Chen

F. F.

Sousa

K. H.

West

S. G.

(2005). Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12, 471-492.

15.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.

16.

Coalson

D. L.

Raiford

S. E.

Saklofske

D. H.

Weiss

L. G.

(2010). Advances in the assessment of intelligence. In Weiss

L. G.

Saklofske

D. H.

Coalson

D. L.

Raiford

S. E.

(2010). WAIS-IV clinical use and interpretation: Scientist–practitioner perspectives. San Diego, CA: Academic Press.

17.

Cohen

(1957). The factorial structure of the WAIS between early adulthood and old age. Journal of Consulting Psychology, 21, 283-290.

18.

Cohen

(1959). The factorial structure of the WISC at ages 7-6, 10-6, and 13-6. Journal of Consulting Psychology, 23, 285-299.

19.

Conway

A. R. A.

Cowan

Bunting

M. F.

Therriault

D. J.

Minkoff

S. R. B.

(2002). A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence, 30, 163-183.

20.

Cooke

D. J.

Kosson

D. S.

Michie

(2001). Psychopathy and ethnicity: Structural, item, and test generalizability of the Psychopathy Checklist-Revised (PCL-R) in Caucasian and African American participants. Psychological Assessment, 13, 531-542.

21.

de Jong

P. F.

Das-Smaal

E. A.

(1995). Attention and intelligence: The validity of the star counting test. Journal of Educational Psychology, 87(1), 80-92.

22.

Drasgow

(1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are central issues. Psychological Bulletin, 95, 134-135.

23.

Drasgow

(1987). Study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology, 72, 19-29.

24.

Engel

Abreu

P. M. J.

Conway

A. R. A.

Gathercole

S. E.

(2010). Working memory and fluid intelligence in young children. Intelligence, 38, 552-561.

25.

Engle

R. W.

Tuholski

S. W.

Laughlin

J. E.

Conway

A. R. A.

(1999). Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General, 128, 309-331.

26.

Fry

A. F.

Hale

(1996). Processing speed, working memory, and fluid intelligence: Evidence for a developmental cascade. Psychological Science, 7, 237-241.

27.

Fry

A. F.

Hale

(2000). Relationships among processing speed, working memory, and fluid intelligence in children. Biological Psychology, 54, 1-34.

28.

Georgas

van de Vijver

F. J. R.

Weiss

L. G.

Saklofske

D. H.

(2003). A cross-cultural analysis of the WISC-III. In Georgas

Weiss

L. G.

van de Vijver

F. J. R.

Saklofske

D. H.

(Eds.), Culture and children’s intelligence: Cross-cultural analysis of the WISC-III (pp. 278-313). San Diego, CA: Academic Press.

29.

Horn

J. L.

McArdle

J. J.

(1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117-144.

30.

Hoyle

R. H.

Panter

A. T.

(1995). Writing about structural equation models. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp.158-176). Thousand Oaks, CA: SAGE.

31.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.

32.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

33.

Immekus

J. C.

Maller

S. J.

(2010). Factor structure invariance of the Kaufman Adolescent and Adult Intelligence Test across male and female samples. Educational and Psychological Measurement, 70, 91-104.

34.

Jöreskog

K. G.

Sörbom

(2006). LISREL 8.8 statistical program. Lincolnwood, IL: Scientific Software.

35.

Kane

M. J.

Hambrick

D. Z.

Conway

A. R. A.

(2005). Working memory capacity and fluid intelligence are strongly related constructs: Comment on AcKerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1), 66-71.

36.

Kaplan

(2000). Structural equation modeling: Foundations and extensions. Thousand Oaks, CA: SAGE.

37.

Kaufman

A. S.

(1975). Factor analysis of the WISC-R at 11 age levels between 6½ to 16½ years. Journal of Consulting and Clinical Psychology, 43(2), 135-147.

38.

Keith

T. Z.

Reynolds

M. R.

(2010). CHC and cognitive abilities: What we’ve learned from 20 years of research. Psychology in the Schools, 47, 635-650

39.

Keith

T. Z.

Reynolds

M. R.

(2012). Using confirmatory factor analysis to aid in understanding the constructs measured by intelligence tests. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed.). New York, NY: Guilford.

40.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

41.

Lichtenberger

E. O.

Kaufman

A. S.

(2009). Essentials of WAIS-IV Assessment. New York, NY: John Wiley.

42.

Loehlin

J. C.

(2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th rev. ed.). Mahwah, NJ: Lawrence Erlbaum.

43.

Maller

S. J.

French

B. F.

(2004). Universal nonverbal intelligence test factor invariance across deaf and standardization samples. Educational and Psychological Measurement, 64, 647-660.

44.

Marsh

H. W.

Balla

J. R.

McDonald

R. P.

(1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391-410.

45.

McDonald

R. P.

M. R.

(2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7(1), 64–82.

46.

McGrew

K. S.

(2005). The Cattell-Horn-Carroll theory of cognitive abilities: Past, present, and future. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 136-181). New York, NY: Guilford Press.

47.

McGrew

Flanagan

(1998). The intelligence test desk reference: Gf-Gc cross-battery assessment. Boston, MA: Allyn & Bacon.

48.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.

49.

Prifitera

Weiss

L. G.

Saklofske

D. H.

Rolfhus

(2005). The WISC-IV in the clinical assessment context. In Prifitera

Saklofske

D. H.

Weiss

L. G.

(Eds.), WISC-IV clinical use and interpretation: Scientist-Practitioner perspectives (pp.3-32). San Diego, CA: Academic Press.

50.

Rabin

L. A.

Barr

W. B.

Burton

L. A.

(2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA division 40 members. Archives of Clinical Psychology, 20, 33-65.

51.

Reynolds

M. R.

Keith

T. Z.

(2013). Measurement and statistical issues in child assessment research. In Saklofske

D. H.

Reynolds

C. R.

Schwean

(Eds.), The Oxford handbook of child and adolescent assessment.

52.

Schneider

W. J.

McGrew

K. S.

(2012). The Cattell-Horn-Carrol model of intelligence. In Flanagan

D. P.

Harrison

P. L.

(Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed.). New York, NY: Guilford.

53.

Sternberg

R. J.

(1995). In search of the human mind. Fort Worth, TX: Harcourt Brace College.

54.

Sternberg

R. J.

(2000). Handbook of Intelligence. New York, NY: Cambridge University Press.

55.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-69.

56.

Ward

L. C.

Bergman

M. A.

Herbert

K. R.

(2011). WAIS-IV subtest covariance structure: Conceptual and statistical considerations. Psychological Assessment, Advance online publication. doi:10.1037/a0025614

57.

Wechsler

(1939). The measurement of adult intelligence. Baltimore, MD: Williams &Wilkins.

58.

Wechsler

(1944). The measurement of adult intelligence (3rd ed.). Baltimore, MD: Williams & Wilkins.

59.

Wechsler

(1958). The measurement and appraisal of adult intelligence (4th ed.). Oxford, UK: Williams & Wilkins.

60.

Wechsler

(1991). Wechsler Intelligence Scale for Children (3rd ed.). San Antonio, TX: Psychological Corporation

61.

Wechsler

(1997). Wechsler Adult Intelligence Scale (3rd ed.). San Antonio, TX: Psychological Corporation.

62.

Wechsler

(2002). Wechsler Preschool and Primary Scale of Intelligence (3rd ed.). San Antonio, TX: Psychological Corporation.

63.

Wechsler

(2003). Wechsler Intelligence Scale for Children (4th ed.). San Antonio, TX: Psychological Corporation.

64.

Wechsler

(2008a). Manual for the Wechsler Adult Intelligence Scale–Fourth Edition. San Antonio, TX: Pearson.

65.

Wechsler

(2008b). Wechsler Adult Intelligence Scale–Fourth Edition technical and interpretive manual. San Antonio, TX: Pearson.

66.

Weiss

L. G.

Saklofske

D. H.

Coalson

D. L.

Raiford

S. E.

(2010a). Theoretical, empirical and clinical foundations of the WAIS-IV index scores. In Weiss

L. G.

Saklofske

D. H.

Coalson

D. L.

Raiford

S. E.

(Eds.), WAIS-IV Clinical Use and Interpretation: Scientist–practitioner perspectives. San Diego, CA: Academic Press.

67.

Weiss

L G.

Saklofske

D. H.

Coalson

Raiford

S. E.

(Eds.) (2010b). WAIS-IV clinical use and interpretation. San Diego, CA: Academic Press.

68.

West

S. G.

Finch

J. P.

Curran

P. J.

(1995). Structural equation models with nonnormal variables. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56-75). Thousand Oaks: SAGE.

69.

Widaman

K. F.

Reise

S. P.

(1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In Bryant

K. J.

Windle

West

S. G.

(Eds.), The science of prevention (pp. 281-324). Washington, DC: American Psychological Association.