Orthogonal Higher Order Structure of the WISC-IV Spanish Using Hierarchical Exploratory Factor Analytic Procedures

Abstract

As recommended by Carroll, the present study examined the factor structure of the Wechsler Intelligence Scale for Children–Fourth Edition Spanish (WISC-IV Spanish) normative sample using higher order exploratory factor analytic techniques not included in the WISC-IV Spanish Technical Manual. Results indicated that the WISC-IV Spanish subtests were properly aligned with theoretically proposed factors; however, application of the Schmid and Leiman procedure found that the g factor accounted for large portions of total and common variance, whereas the four first-order factors accounted for small portions of total and common variance. Implications for clinical interpretation of the measurement instrument are discussed.

Keywords

WISC-IV Spanish exploratory factor analysis intelligence Schmid–Leiman procedure

The Wechsler Intelligence Scale for Children–Fourth Edition Spanish (WISC-IV Spanish; Wechsler, 2005a) is a translation and adaptation of the WISC-IV (Wechsler, 2003a) for use with Spanish-speaking children and adolescents aged 6 to 16 years. It features 10 core and four supplemental subtests, and provides four index scores as well as an overall full scale IQ score (FSIQ) that represents general intellectual ability. According to Harris, Muñoz, and Llorente (2008), the WISC-IV Spanish was developed to meet the needs of examinees learning English as a second language and in the process of acculturating to the United States. It should be noted that although the WISC-IV was recently revised, the WISC-IV Spanish has yet to be updated and continues to be widely utilized by practitioners (Sotelo-Dynega & Dixon, 2014).

According to the Technical Manual (Wechsler, 2005b), an explicit goal in developing a Spanish language version of the WISC-IV was to create a measurement instrument that explicitly reflected the content, structure, and theoretical foundations of the WISC-IV (attempts to develop a Spanish version of the WISC-III were never completed). To that end, WISC-IV Spanish users are advised in the Technical Manual to become familiar with the rationale for the revisions made to the WISC-IV from the previous iteration of the measurement instrument. According to the Technical Manual (Wechsler, 2003b), a major goal of the WISC-IV revision was to update the instrument’s theoretical foundation to better reflect advances in psychometric theory, including elements of Carroll’s (1993) three-stratum theory of cognitive abilities.

Despite being criticized as atheoretical, subsequent factor analytic research (e.g., Keith, Fine, Taub, Reynolds, & Kranzler, 2006; Weiss, Keith, Zhu, & Chen, 2013) indicated that the WISC-IV measures several constructs consistent with the Cattell–Horn–Carroll (CHC) model of intelligence, including crystallized ability, fluid reasoning, and processing speed (PS). Accordingly, users have been encouraged (e.g., Flanagan & Kaufman, 2009) to adopt a CHC-based interpretive scheme for WISC-IV scores that places primary emphasis on evaluation of the profile of obtained first-order factor scores. As a result of the structural congruence between the measures, these results and interpretive recommendations are thought to be germane for the WISC-IV Spanish (San Miguel Montes, Allen, Puente, & Neblina, 2010).

The structure of the WISC-IV Spanish was examined using extensive exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) supporting four first-order factors consistent with the WISC-IV (Wechsler, 2005b). However, the factor solution initially examined in exploratory analyses appeared to be forced to comport with publisher theory as no empirical support for a four-factor extraction was provided. Inexplicably, higher order factor analysis to verify and describe the hierarchical structure of the WISC-IV Spanish was not conducted despite the implied influence of Carroll’s (1993) higher order model. According to Carroll (1993, 1995), subtest performance on tests of cognitive ability reflects the combination of both first-order and second-order factors. As a result, he argued that variance from the higher order factor must be extracted first to residualize the lower order factors, leaving them orthogonal to the higher order dimension. Variability associated with a higher order factor is accounted for before interpreting variability associated with lower order factors, resulting in variance being apportioned correctly to higher order and lower order dimensions. To accomplish this, Carroll (1995) recommended second-order factor analysis of first-order factor correlations followed by a Schmid–Leiman transformation (Schmid & Leiman, 1957). This process transforms an oblique solution “into an orthogonal solution which not only preserves the desired interpretation characteristics of the oblique solution, but also discloses the hierarchical structuring of the variables” (Schmid & Leiman, 1957, p. 53).

Also missing from the Technical Manual were empirical support for a four-factor extraction, proportions of variance accounted for by the higher order FSIQ and the four first-order factors, higher order subtest loadings, and subtest specificity estimates. As a result, clinicians do not have the information necessary for determining the relative importance of the WISC-IV Spanish factor and subtest scores relative to the FSIQ score, which can result in overinterpretation of lower order factors at the expense of the higher order factor (Carretta & Ree, 2001). Following the recommendations of Carroll (1995), the present study used exploratory factor analytic procedures followed by the Schmid and Leiman (1957) orthogonalization procedure to better clarify the hierarchical factor structure and allocation of reliable score variance in WISC-IV Spanish as illustrated by Watkins (2006) and Canivez, Watkins, and Dombrowski (2015).

Method

The WISC-IV Spanish was standardized on a stratified nationally representative sample of 500 Spanish-speaking children aged 6 to 16 years closely approximating the 2001 U.S. Census on age, sex, and parent education level. The WISC-IV Spanish has 10 core subtests (M = 10, SD = 3) that combine to form four-factor composites (M = 100, SD = 15): Verbal Comprehension (VC), Perceptual Reasoning (PR), PS, and Working Memory (WM). The FSIQ (M = 100, SD = 15) is based on the sum of scores from the 10 core subtests.

As recommended by Gorsuch (2003), multiple criteria for determining the number of factors to retain were examined. Consistent with best practice recommendations in the professional literature (e.g., Frazier & Youngstrom, 2007), these procedures included Horn’s parallel analysis (HPA; Horn, 1965) and minimum average partial (MAP; Velicer, 1976). HPA was generated using the Monte Carlo Principle Component Analysis (PCA) for Parallel Analysis program (Watkins, 2000) with 100 replications to produce stable estimates. MAP procedures were conducted using O’Connor’s (2000) SPSS syntax. The intercorrelation matrix of the 10 core subtests for the WISC-IV Spanish normative sample of 500 children and adolescents (Wechsler, 2005b) was then subjected to EFA as described in the Technical Manual (Wechsler, 2005b), that is, principal-axis extraction, followed by oblique rotation. The resulting factor correlations were subjected to second-order EFA and orthogonalized using the Schmid and Leiman (1957) procedure as programmed in the MacOrtho program (Watkins, 2004). Omega-hierarchical and omega-subscale coefficients (Reise, 2012) were also estimated with the Omega program (Watkins, 2013), which is based on the tutorial by Brunner, Nagy, and Wilhelm (2012) and the work of Zinbarg, Revelle, Yovel, and Li (2005) and Zinbarg, Yovel, Revelle, and McDonald (2006).

Results

Parallel analysis (Horn, 1965) and MAP (Velicer, 1976) criteria suggested that one factor be retained for the WISC-IV Spanish normative sample. In sum, none of the extraction criteria supported the retention of four factors, as suggested by the WISC-IV Spanish Technical Manual, with the exception of publisher theory. Despite factor extraction criteria suggesting extraction of only one factor, four factors were extracted to comport with the proposed test structure indicated in the Technical Manual. It is believed that these methods provide for a direct examination of publisher theory with results that are potentially relevant for clinicians who utilize the WISC-IV Spanish in practice.

Schmid–Leiman variance partitioning EFA results are presented in Table 1. Correlations between the four first-order factors from the promax rotation (k = 4) ranging from .62 to .74 suggested the presence of a higher order dimension requiring additional explication. The WISC-IV Spanish general factor accounted for between 25% and 47% of the variance in the core subtests. The VC factor accounted for an additional 15% to 26% of the variance in the three VC subtests. Beyond g, the PR factor accounted for between 4% and 17% of the variance in the three PR subtests, the PS factor provided 11% to 14% of the variance in the PS subtests, and the WM factor contributed 9% of the variance in its two subtests. As a consequence, the general factor accounted for the greatest amount of total (41.3%) and common variance (73.2%). Altogether, the general and first-order factors accounted for 56.6% of the total variance, leaving 43.4% unique variance (combination of specific and error variance). Omega-hierarchical coefficient for general intelligence (.833) was high, supporting scale interpretation but omega-subscale coefficients for the four WISC-IV Spanish group factors ranged from .120 (WM) to .280 (VC) and likely contained too little true score variance for clinical interpretation (Reise, 2012; Reise, Bonifay, & Haviland, 2013).

Table 1.

Sources of Variance in the WISC-IV Spanish for Ages 6 to 16 (N = 500): 10 Core Subtests According to an Orthogonalized (Schmid & Leiman, 1957) Higher Order Factor Model.

Subtest	WISC-IV Spanish group factors										h ²	u ²
	General		VC		PR		PS		WM
	b	S ²	b	S ²	b	S ²	b	S ²	b	S ²
Vocabulary	.682	.465	.513	.263	−.024	.000	−.028	.000	.045	.002	.730	.270
Comprehension	.613	.376	.494	.244	−.061	.000	.022	.000	.017	.000	.621	.379
Similarities	.720	.518	.392	.154	.128	.016	.011	.000	−.037	.000	.689	.311
Matrix reasoning	.718	.516	−.012	.000	.410	.168	−.031	.000	.023	.001	.684	.316
Block design	.661	.437	−.056	.000	.359	.129	.035	.001	.027	.001	.568	.432
Picture concepts	.658	.433	.160	.026	.208	.043	.084	.007	−.031	.000	.509	.491
Coding	.498	.248	.024	.001	−.022	.000	.378	.143	.000	.000	.392	.608
Symbol search	.584	.341	−.029	.000	.079	.006	.335	.112	.021	.000	.460	.540
Digit span	.636	.404	.057	.003	.003	.000	.075	.006	.302	.091	.505	.495
Letter–number Sequencing	.634	.402	.080	.006	.077	.006	−.044	.000	.297	.088	.502	.498
% total variance	41.3		7.0		3.7		2.7		1.8		56.6	43.4
% common variance	73.2		12.3		6.5		4.8		3.2		100
ω_h/s	.833		.280		.150		.179		.120

Note. Bold denotes theoretically consistent factor loadings. WISC-IV = Wechsler Intelligence Scale for Children–Fourth edition; VC = Verbal Comprehension Index; PR = Perceptual Reasoning Index; PS = Processing Speed Index; WM = Working Memory Index; b = standardized loading of subtest on factor; S² = variance explained; h² = communality; u² = uniqueness; ω_h = Omega Hierarchical (general); ω_s = Omega Subscale (group factors).

Discussion

Although the first-order factor structure of the WISC-IV Spanish was presented in the Technical Manual (Wechsler, 2005b), its hypothesized higher order structure was not investigated. Because first-order factors are abstractions of measured variables, interpreting a second-order factor on the basis of the relationships between these variables can be misleading because performance on any cognitive subtest reflects a mixture of both general and first-order factors (Watkins & Beaujean, 2014). As a result, Carroll (1995) insisted that it is necessary to decompose variance into components that can be sourced more appropriately to higher and lower order dimensions. Given the fact that the Technical Manual encourages users to interpret the WISC-IV Spanish predominately at the first-order factor level, it is vital that the higher order structure be examined to determine how well the scores measure those traits (i.e., American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).

The application of the Schmid and Leiman (1957) transformation to the WISC-IV Spanish normative sample demonstrated that variance for each of the core subtests can be decomposed into multiple components. The most important of these components was a higher order latent dimension (g). General cognitive ability accounted for more variance in each of the WISC-IV Spanish subtests than any of the proposed first-order factors. Across all subtests, the combination of g and uniqueness overshadowed the contributions made by the four first-order factors (see Table 1).

Notably, although subtest loadings were generally consistent with their theoretically assigned factors, the four-factor model appears to be overfactored (e.g., Frazier & Youngstrom, 2007). Specifically, once the reliable variance attributable to the higher order dimension was partialed out, Letter–Number Sequencing failed to load saliently (e.g., ≥.30) on the WM factor. These results suggest that WM may not be a viable latent dimension on the WISC-IV Spanish because a common factor cannot be produced from a singlet loading of one manifest variable (Preacher & MacCallum, 2003).

The results from the present investigation are relatively consistent with examinations of the higher order structure of the WISC-IV (e.g., Canivez, 2014; Watkins, 2006). As observed in the present investigation, Canivez (2014) and Watkins (2006) both found that the general factor accounted for the largest portions (> 70%) of common variance in the WISC-IV subtests in both normative and referred samples. Interestingly, both studies provided evidence for the viability of the WM factor. Whereas Canivez (2014) found that the subtests theoretically assigned to the WM factor aligned with that factor when estimating a direct hierarchical model via CFA, Watkins (2006) found that both of those measures loaded saliently on that dimension after higher order variance partitioning via the Schmid–Leiman procedure in the normative sample. Given the fact that the WISC-IV normative sample (N = 2,200) was significantly larger than the present sample (N = 500), it is certainly possible that the weak WM loadings produced for the WISC-IV Spanish are an artifact of sampling error (MacCallum, Widaman, Preacher, & Hong, 2001).

Given the hierarchical nature of the WISC-IV Spanish, additional research estimating rival measurement models through CFA such as a bifactor structure and computing model-based reliability coefficients (e.g., Reise et al., 2013) would be beneficial. Nevertheless, it is believed that the present results are a first step in examining the higher order structure of the WISC-IV Spanish and will be a useful criterion to compare against future results produced from alternate methods (e.g., CFA). In addition, these results may be instructive for guiding the development of future iterations of the WISC-IV Spanish.

To conclude, our results diverge from the four-factor structure posited in the WISC-IV Spanish Technical Manual and therefore suggest caution in interpretation of first-order factors until additional research has been conducted. As a consequence, it is recommended that users of the WISC-IV Spanish focus most, if not all, of their interpretive weight on the FSIQ as g accounts for the majority of common variance in the WISC-IV Spanish subtests, and too little true score variance is contained in the group factors. Thus, clinicians who wish to interpret beyond the FSIQ must account for its effects at every level of the WISC-IV Spanish or risk overinterpretation (Glutting, Watkins, Konold, & McDermott, 2006). These results suggest that if assessment and interpretation of first-order dimensions are of critical importance, the test authors will likely need to increase the number of subtests estimating those dimensions to increase the amount of variance at that level of measurement (Canivez, 2011).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Brunner

Nagy

Wilhelm

(2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846. doi:10.1111/j.1467–6494.2011.00749.x

Canivez

G. L.

(2011). Hierarchical factor structure of the Cognitive Assessment System: Variance partitions from the Schmid–Leiman (1957) procedure. School Psychology Quarterly, 26, 305-317. doi:10.1037/a0025973

Canivez

G. L.

(2014). Construct validity of the WISC-IV with a referred sample: Direct versus indirect hierarchical structures. School Psychology Quarterly, 29, 38-51. doi:10.1037/spq0000032

Canivez

G. L.

Watkins

M. W.

Dombrowski

S. C.

(2015). Factor structure of the Wechsler Intelligence Scale for Children–Fifth Edition: Exploratory factor analyses with the 16 primary and secondary subtests. Psychological Assessment. Advance online publication. doi:10.1037/pas0000238

Carretta

T. R.

Ree

M. J.

(2001). Pitfalls of ability research. International Journal of Selection and Assessment, 9, 325-335. doi:10.1111/1468-2389.00184

Carroll

J. B.

(1993). Human cognitive abilities: A survey of factor-analytic studies. New York, NY: Cambridge University Press.

Carroll

J. B.

(1995). On methodology in the study of cognitive abilities. Multivariate Behavioral Research, 30, 429-452. doi:10.1207/s15327906mbr3003_6

Flanagan

D. P.

Kaufman

A. S.

(2009). Essentials of WISC-IV assessment (2nd ed.). Hoboken, NJ: John Wiley.

10.

Frazier

T. W.

Youngstrom

E. A.

(2007). Historical increase in the number of factors measured by commercial tests of cognitive ability: Are we overfactoring? Intelligence, 35, 169-182. doi:10.1016/j.intell.2006.07.002

11.

Glutting

J. J.

Watkins

M. W.

Konold

T. R.

McDermott

P. A.

(2006). Distinctions without a difference: The utility of observed versus latent factors from the WISC-IV in estimating reading and math achievement on the WIAT-II. Journal of Special Education, 40, 103-114. doi:10.1177/00224669060400020101

12.

Gorsuch

R. L.

(2003). Factor analysis. In Schinka

J. A.

Velicer

F. F.

(Eds.), Handbook of psychology, Vol. 2: Research methods in psychology (pp. 143-164). Hoboken, NJ: John Wiley.

13.

Harris

J. G.

Muñoz

M. R.

Llorente

A. M.

(2008). Considerations in using the WISC-IV with Hispanic children. In Prifiteria

Saklofske

D. H.

Weiss

L. G.

(Eds.), WISC-IV clinical assessment and intervention (2nd ed., pp. 409-442). San Diego, CA: Academic Press.

14.

Horn

J. L.

(1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179-185. doi:10.1007/BF02289447

15.

Keith

T. Z.

Fine

J. G.

Taub

G. E.

Reynolds

M. R.

Kranzler

J. H.

(2006). Higher order, multisample, confirmatory factor analysis of the Wechsler Intelligence Scale for Children–Fourth edition: What does it measure? School Psychology Review, 35, 108-127. Available from http://www.nasponline.org

16.

MacCallum

R. C.

Widaman

K. F.

Preacher

K. J.

Hong

(2001). Sample size in factor analysis: The role of model error. Multivariate Behavioral Research, 36, 611-637. doi:10.1207/S15327906MBR3604_06

17.

O’Connor

B. P.

(2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavioral Research Methods, 32, 396-402. doi:10.3758/BF03200807

18.

Preacher

K. J.

MacCallum

R. C.

(2003). Repairing Tom Swift’s electric factor analysis machine. Understanding Statistics, 2, 13-43. doi:10.1207/S15328031US0201_02

19.

Reise

S. P.

(2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi:10.1080/00273171.2012.715555

20.

Reise

S. P.

Bonifay

W. E.

Haviland

M. G.

(2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. doi:10.1080/00223891.2012.725437

21.

San Miguel Montes

L. E.

Allen

D. N.

Puente

A. E.

Neblina

. (2010). Validity of the WISC-IV Spanish for a clinically referred sample of Hispanic children. Psychological Assessment, 22, 465-469. doi:10.1037/a0018895

22.

Schmid

Leiman

J. M.

(1957). The development of hierarchical factor solutions. Psychometrika, 22, 53-61. doi:10.1007/BF02289209

23.

Sotelo-Dynega

Dixon

S. G.

(2014). Cognitive assessment practices: A survey of school psychologists. Psychology in the Schools, 51, 1031-1045. doi:10.1002/pits.21800

24.

Velicer

W. F.

(1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 31, 321-327. doi:10.1007/BF02293557

25.

Watkins

M. W.

(2000). Monte Carlo PCA for parallel analysis [Computer software]. State College, PA: Ed & Psych Associates.

26.

Watkins

M. W.

(2006). Orthogonal higher order structure of the Wechsler Intelligence Scale for Children–Fourth edition. Psychological Assessment, 18, 123-125. doi:10.1037/1040-3590.18.1.123

27.

Watkins

M. W.

(2013). Omega. [Computer software]. Phoenix, AZ: Ed & Psych Associates.

28.

Watkins

M. W.

Beaujean

A. A.

(2014). Bifactor structure of the Wechsler Preschool and Primary Scale of Intelligence–Fourth edition. School Psychology Quarterly, 29, 52-63. doi:10.1037/spq0000038

29.

Wechsler

(2003a). Wechsler Intelligence Scale for Children–Fourth edition. San Antonio, TX: Psychological Corporation.

30.

Wechsler

(2003b). Wechsler Intelligence Scale for Children–Fourth edition: Technical and interpretive manual. San Antonio, TX: Psychological Corporation.

31.

Wechsler

(2005a). Wechsler Intelligence Scale for Children–Fourth edition, Spanish. San Antonio, TX: Psychological Corporation.

32.

Wechsler

(2005b). Wechsler Intelligence Scale for Children–Fourth edition, Spanish manual. San Antonio, TX: Psychological Corporation.

33.

Weiss

L. G.

Keith

T. Z.

Zhu

Chen

(2013). WISC-IV and clinical validation of the four-and five-factor interpretive approaches. Journal of Psychoeducational Assessment, 31, 114-131. doi:10.1177/0734282913478032

34.

Zinbarg

R. E.

Revelle

Yovel

(2005). Cronbach’s α, Revelle’s β, and McDonald’s ω_Η: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. doi:10.1007/s11336–003–0974–7

35.

Zinbarg

R. E.

Yovel

Revelle

McDonald

R. P.

(2006). Estimating generalizability to a latent variable common to all of a scale’s indicators: A comparison of estimators for ω_h. Applied Psychological Measurement, 30, 121-144. doi:10.1177/0146621605278814