Abstract
In this commentary, we reviewed two clinical validation studies on the Wechsler Scales conducted by Weiss and colleagues. These researchers used a rigorous within-battery model-fitting approach that demonstrated the factorial invariance of the Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV) and Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) across populations. Beyond the important finding of invariance across populations, however, these studies did not provide much additional clarification about exactly what is and what is not measured by the WISC-IV and WAIS-IV vis-à-vis an overarching empirically validated theory of the structure of cognitive abilities. To that end, we argued that a better understanding of the cognitive constructs (both broad and narrow) measured by the Wechsler scales or invariance across selection of variables is necessary and will require theory-driven joint or cross-battery confirmatory factor analysis. Recommendations for conducting this type of research were offered within the context of the Cattell-Horn-Carroll theory of cognitive abilities.
We reviewed two studies that examined four- and five-factor Wechsler Intelligence Scale for Children–Fourth Edition (WISC-IV) and Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) measurement models and their invariance across normal and clinical samples (Weiss, Keith, Zhu, & Chen, 2013a, 2013b). In our opinion, the major contribution of these well-designed and executed studies is the finding that the four- and five-factor models were invariant across samples. The findings are of particular import because these measures are so widely used, and often with examinees who may experience symptoms similar to those encountered in the clinical samples. The rigorous model-fitting approach used to test for measurement invariance by the researchers is welcome (Meredith, 1993). In all, the authors should be congratulated for putting together two fine studies. In fact, beyond the typical nitpicking, there is not much for us to critique with regard to the studies themselves or to the interpretations drawn from them. The studies, however, stimulated some thoughts among us, and our intention is to share some of these thoughts in hopes of opening up some dialogue about how to continue to improve the understanding and assessment of human cognitive abilities.
In this commentary, we argue two primary points. First, an examination of the structural validity of the Wechsler Scales (and similar batteries) via within-battery factor analysis is limited by the subtests contained within the battery, which in turn limits further understanding of the constructs measured by these batteries (although we were glad to see that the authors included the supplemental tests in their analysis). Theory-driven joint or cross-battery confirmatory factor analysis (CB-CFA) will be necessary to understand more clearly the constructs measured by cognitive batteries. Second, CB-CFAs should more routinely model the narrow abilities that are measured by the Wechsler scales, particularly in light of their importance in explaining variance in academic skills above and beyond the variance accounted for by g (see Flanagan, Ortiz, & Alfonso, 2013). Throughout this commentary, we opine with regard to what cognitive constructs are and are not measured by the Wechsler Scales and recommend that these instruments be supplemented with measures of other cognitive constructs to assess more completely the breadth of abilities that are important in understanding learning and achievement.
Clinical Validation of the Four- and Five-Factor Interpretive Approaches for the Wechsler Scales
According to the WISC-IV Technical and Interpretive Manual (Wechsler, 2003), four factors underlie the WISC-IV, namely Verbal Comprehension (VC), Perceptual Reasoning (PR), Working Memory (WM), and Processing Speed (PS). In independent factor analyses of the WISC-IV standardization sample, five factors were modeled (e.g., Keith, Fine, Taub, Reynolds, & Kranzler, 2006). For example, Keith and colleagues split the PR factor into separate Fluid Reasoning (Gf; Matrix Reasoning and Picture Concepts) and Visual Processing (Gv; Block Design and Picture Completion) factors (see also Chen, Keith, Chen, & Chang, 2009; Reynolds, Keith, Flanagan & Alfonso, 2013). Although Keith and colleagues found both the WISC-IV four-factor model and the Cattell-Horn-Carroll (CHC) five-factor model equally plausible, they recommended the five-factor model because it was based on a well-supported theory. The purpose of the WISC-IV study conducted by Weiss and colleagues was to determine whether the four- and five-factor models were invariant across normal and clinical samples. They found that both models fit the data well, reported one model was not superior to the other, and concluded both models were invariant across samples.
Similarly, the WAIS-IV Technical and Interpretive Manual (Wechsler, 2008) states that four factors underlie the WAIS-IV, namely VC, PR, WM, and PS. In independent factor analyses of the WAIS-IV standardization data, Benson, Hulac, and Kranzler (2010) and Ward, Bergman, and Hebert (2012) found that a CHC five-factor model fit the data better than a four-factor model whereas Canivez and Watkins’ (2010) research supported the four-factor model. Like the WISC-IV study, the purpose of the WAIS-IV study conducted by Weiss and colleagues was to determine whether the four- and five-factor models were invariant across normal and clinical samples. Similar to Benson et al. and Ward et al., they found that the five-factor model fit the data better, providing support for separating the PR factor into separate Fluid Reasoning (FR; Matrix Reasoning, Arithmetic, and Figure Weights) and Perceptual Organization (PO; Block Design, Visual Puzzles, and Picture Completion) factors (see also Niileksela, Reynolds, & Kaufman, 2012). Moreover, because both the four- and five-factor models provided a good fit to the clinical data, Weiss and colleagues concluded that the WAIS-IV subtests have the same meaning regardless of clinical status.
Overall, it is important to know that differences in scores on the Wechsler scales across normative and combined clinical samples are due to differences in the common factors, and hence the test is not biased. The current studies provide an important contribution in this area, as measurement invariance, which requires both mean and covariance structures, should be applied within a model-fitting framework across groups as performed in these two studies (Meredith, 1993); it is simply not sufficient to exclude means or to factor analyze data within groups. It is not only insufficient to use exploratory factor analysis (EFA) to analyze data separately within clinical groups as is sometimes done, but EFA may produce misleading findings. For example, if subtest scores are used in part to determine clinical group status, the correlational structure among those subtests will be altered. Because EFA is most often based on correlations and not covariances, the findings from EFAs have to be interpreted with great caution because they may be due to statistical artifacts (Nesselroade & Thompson, 1995).
To reiterate, the Weiss et al. (2013a, 2013b) studies were important because they demonstrated a critical issue related to factorial invariance, invariance across populations. Beyond the findings of invariance across populations, however, the Weiss et al. studies did not provide much additional clarification about exactly what is and what is not measured by the WISC-IV and WAIS-IV vis á vis an overarching theory of the structure of cognitive abilities. In other words, the within-battery factor analyses conducted by Weiss and his colleagues could not fully resolve the controversies surrounding these batteries, such as how the Arithmetic subtest should be interpreted or issues related to the potential bifurcation of the Perceptional Reasoning Index. What could not be fully addressed, although it has been addressed to some extent across Wechsler revisions that include new subtests, is invariance related to selection of variables (Mulaik, 2010; Thurstone, 1947). Would the constructs measured by Arithmetic be the same as reported in these studies if additional measures of quantitative reasoning and applied mathematical thinking were included in the analysis? Would the same four or five Wechsler common factors arise if subtests from different batteries were factor analyzed with the Wechsler subtests? Would a PR factor split be more apparent if additional measures of fluid reasoning and visual-spatial thinking from other batteries were included in a factor analysis? Weiss and colleagues were constrained in their analysis and understanding of the WISC-IV and WAIS-IV because they had to rely solely on the subtests that comprise these batteries. A better understanding of the constructs measured by the Wechsler scales may never be fully accomplished via within-battery factor analysis.
Within-Battery Factor Analyses of the Wechsler Scales: Are They Still Necessary?
The primary structural validity tool of the Wechsler scales has been factor analysis. The earlier versions of the Wechsler Scales were evaluated using EFA (e.g., Blaha & Wallbrown, 1996; Cohen, 1959; Donders, 1993)—a method that identifies the factor structure of a cognitive battery by allowing the data to “speak for themselves,” particularly when there is no a priori theory specified (Carroll, 1993; Flanagan, McGrew, & Ortiz, 2000). By contrast, CB-CFA typically examines the extent to which an a priori hypothesized factor structure or model fits the data, as well as how the fit of the model compares to alternative models, and when used in a somewhat exploratory manner, how the model might be modified to fit the data better (Keith & Reynolds, 2012). About 15 years ago, Keith (1997) demonstrated the prominence of theory-based test development and research activities, and he and his colleagues (and other researchers) have used CHC theory-driven CFA to analyze the standardization data of the Wechsler scales (e.g., Keith et al., 2006; Keith & Witta, 1997; Weiss et al., 2013a, 2013b). Much has been learned about the Wechsler scales based on these analyses, and all for the better.
For the most part, the CHC theory-driven confirmatory factor analyses of the Wechsler scales can be categorized as within-battery. Within-battery factor analysis is confined to the tests from a single cognitive battery (e.g., the 15 subtests of the WISC-IV). Joint or cross-battery factor analysis (CB-FA) includes tests from more than one cognitive battery (e.g., the WISC-IV and the Woodcock-Johnson III, Tests of Cognitive Abilities [WJ III; Woodcock, McGrew, & Mather, 2001]). In CB-FA, the 15 WISC-IV subtests (along with WJ III tests, for example) are allowed, or are specified, to load on the various cognitive factors included in the theory.
The preponderance of structural within-battery validity studies summarized by Flanagan and colleagues (2000) and highlighted in Table 1, support the internal validity of the Wechsler scales. The movement away from the original Verbal and Performance scales in the direction of the four-factor structure (VC, PR, WM, PS) is supported by a substantial body of systematic within-battery factor analysis research conducted over the past 20 to 25 years (see Appendix A in Flanagan et al., 2000; see also Flanagan & Kaufman, 2009; Lichtenberger & Kaufman, 2013). Nevertheless, the ever changing interpretation of the original Freedom from Distractibility (FFD) tests (viz., arithmetic) and the current PR tests (e.g., do they measure Gv or Gf?) is concerning (Flanagan et al., 2000; McGrew, 1997). For example, within-battery factor analyses of the various Wechsler scales have shown that Arithmetic has changed its factor allegiance from the original Verbal factor to that of FFD, WM, and more recently Fluid Reasoning (Gf). Given that Arithmetic migrates to various factors, perhaps it should be a supplemental subtest on the WAIS-IV, as it is on the WISC-IV.
Within-Battery and Cross-Battery Factor Analyses of the Child and Adult Versions of the Wechsler Scales.
Note: VIQ = Verbal Intelligence Quotient; PIQ = Performance Intelligence Quotient; VC(I) = Verbal Comprehension (Index); PO(I) = Perceptual Organization (Index); FD(I) = Freedom from Distractibility (Index); PSI = Processing Speed Index; WM(I) = Working Memory (Index); Gc = Crystallized Intelligence; Gf = Fluid Reasoning; Gsm = Short-term Memory; Gs = Processing Speed; Gv = Visual Processing; Gq = Quantitative Knowledge; RQ = Quantitative Reasoning.
As demonstrated by Woodcock (1990), it is clear that the structural validity of the Wechsler scales can be improved on through the application of theory-based cross-battery factor analysis research (see also Flanagan et al., 2000; Keith, 2005; Keith & Reynolds, 2010, 2012; Reynolds et al., in press). Furthermore, the CHC-based CB-CFAs of the Wechsler scales conducted by Phelps, McGrew, Knopik, and Ford (2005) shed light on the constructs measured and not measured by these scales (see Table 1). For example, in the CHC-driven CB-CFAs, markers for at least seven or eight broad CHC abilities were included, representing the breadth of cognitive abilities specified by CHC theory (Woodcock, 1990). In addition to Gc, Gv, Gf, Gsm, and Gs, markers for Long-term Storage and Retrieval (Glr), Auditory Processing (Ga), and Quantitative Knowledge (Gq) were specified. As expected, Arithmetic loaded on Gq (not Gf or Gsm) in both Woodcock’s and Phelps and colleagues’ studies (see also Golay, Reverte, Rossier, Favez, & Lecerf, 2012).
Nevertheless, as a result of the within-battery WISC-IV study conducted by Weiss et al. in this issue (2013b), questions about what the Arithmetic subtest measures remain. To date, the WISC-IV has only been included in one CB-CFA (Reynolds et al., in press). Unfortunately, the study conducted by Reynolds and colleagues did not include Gq markers. CB-CFAs with sufficient markers of Gq (i.e., tests of math knowledge and tests of math achievement), Quantitative Reasoning (i.e., tests involving reasoning both inductively and deductively with numbers), Gf (i.e., tests of induction and tests of general sequential reasoning not involving numbers), and Short-term Memory (Gsm; tests of memory span and tests of working memory) will therefore be needed to best understand how the Arithmetic subtest should be interpreted. Even better, maybe it is time to take Arithmetic into the laboratory. Examinees might be asked what type of strategies or thought processes they used while performing the Arithmetic subtest. It might be that Arithmetic loads on different factors for different people because people use different types of strategies while performing this multifaceted task (Keith & Reynolds, 2010).
Noteworthy is the fact that the extant within-battery and cross-battery factor analysis research and the results of content validity (e.g., expert consensus) studies were used by Flanagan and colleagues (2013) to inform how the Wechsler subtests should be classified according to the broad and narrow abilities they measure. These CHC classifications are found in Table 2. The results of Weiss and colleagues (2013a, 2013b) provide additional validity support for these Wechsler CHC classifications and did not suggest the need to alter the classifications in any way.
Broad and Narrow CHC Ability Representation on the Wechsler Scales Based on Within- and Cross-Battery Factor Analyses and Expert Consensus.
Note: CHC classifications are from Flanagan, Ortiz, and Alfonso (2013) based on their review of the literature and expert consensus studies. Gf = Fluid Reasoning; Gc = Crystallized Intelligence; Gv = Visual Processing; Gsm = Short- term Memory; Glr = Long-term Storage and Retrieval; Ga = Auditory Processing; Gs = Processing Speed; RQ = Quantitative Reasoning; I = Induction; VL = Lexical Knowledge; K0 = General (verbal) Knowledge; Vz = Visualization; CF = Flexibility of Closure; MW = Working Memory; MS = Memory Span; P = Perceptual Speed; R9 = Rate-of-Test-Taking.
Although replication of findings is necessary, it is our opinion that within-battery factor analysis of the WISC-IV and WAIS-IV is no longer necessary. Improvements with regard to how the Wechsler subtests should be interpreted will require theory-driven CB-CFAs (or something more novel), not within-battery factor analysis. In this regard, we agree with Dr. Richard Woodcock who, at the inaugural meeting of the Richard Woodcock Institute for Advancement of Contemporary Cognitive Assessment, stated that test authors and publishers have a responsibility to provide the results of CB-FAs in their test manuals (Woodcock, 2012). Although we also believe that independent researchers will ultimately provide the peer reviewed research that is necessary to support or refute validity findings published in test manuals and to further inform interpretation of cognitive ability tests. In addition, when cross-battery data are gathered, there is a greater likelihood of obtaining information about the CHC narrow abilities that are measured by cognitive batteries. This information is important because there is a mounting body of research demonstrating the critical role of narrow cognitive abilities in the prediction of specific academic skills (see Flanagan et al., 2013; Flanagan, Ortiz, Alfonso, & Mascolo, 2006; and McGrew & Wendling, 2010 for summaries).
Do the Wechsler Scales Measure CHC Narrow Abilities Important for Learning and Achievement?
Yes, but not as many as other batteries. A considerable amount of research has demonstrated an empirical relationship among cognitive abilities, neuropsychological processes, and specific academic skills (see Flanagan et al., 2006; Fletcher, Lyon, Fuchs, & Barnes, 2007; Hale & Fiorello, 2004; and McGrew & Wendling, 2010 for summaries). Much of the recent research on cognitive-academic relationships has been interpreted within the context of CHC theory (e.g., Flanagan et al., 2013) and with specific instruments developed from CHC theory (e.g., McGrew & Wendling, 2010). In addition, statistical analyses, such as structural equation modeling, have been used to understand the extent to which specific cognitive abilities (both broad and narrow) explain variance in academic skills above and beyond the variance accounted for by g (e.g., Floyd, McGrew, & Evans, 2008; Juarez, 2012; McGrew, Flanagan, Keith, & Vanderwood, 1997; Vanderwood, McGrew, Flanagan, & Keith, 2002).
For example, in the area of reading, narrow abilities in seven broad CHC domains appear to be important to assess in individuals who struggle to read (McGrew & Wendling, 2010). Specifically, narrow abilities subsumed by Gc (language development, lexical knowledge, listening ability, general information), Gsm (memory span, working memory), Ga (phonetic coding), Glr (associative memory, naming facility, meaningful memory), and Gs (perceptual speed) are related significantly to reading achievement. Furthermore, developmental results suggest that the Ga, Gs, and Glr relations with reading are strongest during the early elementary school years, after which they systematically decrease in strength (e.g., Flanagan et al., 2006; McGrew, 1993). In contrast, the strength of the relations between Gc abilities and reading achievement increases with age (McGrew & Wendling). The Gv abilities of orthographic processing and visual memory are also related to reading achievement (e.g., Berninger, 1990). Narrow Fluid Reasoning (Gf) abilities appear related primarily to reading comprehension from childhood to young adulthood. Finally, related to Gf, the role of executive functions in reading achievement (particularly reading comprehension) has been demonstrated in the neuropsychology literature (e.g., McCloskey, Whitaker, Murphy, & Rogers, 2012), and is consistent with neuropsychological evidence (Decker, Hill, & Dean, 2007).
In a school setting, in particular, cognitive batteries are administered to students who experience difficulties in the learning process (e.g., students who are suspected of having a specific learning disability). In light of the cognitive-achievement relations research, practitioners seek to measure the cognitive abilities and processes that are most closely associated with the academic areas in which a student has difficulty. Table 3 shows the narrow abilities purportedly measured by the WISC-IV (and Wechsler Individual Achievement Test, Third Edition, Pearson, 2009; WIAT-III) that are related significantly to reading achievement. In addition, this table shows the narrow abilities related to reading that are measured by another popular cognitive battery—the WJ III (cognitive and achievement)—as well as a popular neuropsychological battery—the NEPSY-II (Korkman, Kirk, & Kemp, 2007). As may be seen in this table, the WJ III and NEPSY-II measure about one and one half as many narrow abilities important for reading achievement than the WISC-IV/WIAT-III. Three main conclusions may be drawn from this observation. First, modeling of CHC narrow abilities should be part of a systematic program of Wechsler-based CHC CB-CFA research. Second, future editions of the WISC-IV (and WIAT-III) should measure a greater breadth of narrow abilities. Third, the Wechsler scales should be conormed with other tests to allow for a greater breadth of narrow abilities to be measured.
Abilities and Processes Related to Reading Achievement Measured by Subtests on Popular Batteries.
Note: Dashes indicate that there are no direct measures of the ability or process in the corresponding battery. All Gc narrow abilities involve LD or Language Development.
Conclusions and Recommendations
The Weiss et al. (2013a, 2013b) studies of the WISC-IV and WAIS-IV provided important information about the factorial invariance of these instruments, specifically invariance across populations. In this commentary we argued, however, that a better understanding of the constructs measured by the Wechsler scales, or invariance across selection of variables will require CB-FA. Interestingly, we were only able to locate 14 CB-FAs (mostly confirmatory) in the peer-reviewed literature. Nevertheless, these studies should be used as a guide for future CB-CFA research, as there is much to learn from what has already been accomplished.
Table 4 provides a summary of the 14 studies that used CB-CFA with ability subtests since 1990. This table reports the major findings of each study, as well as a “Comments” column that highlights the unique contribution each study offered to the literature. The majority of these studies used Fluid-Crystallized (Gf-Gc) or CHC-driven CB-CFA to evaluate the structural validity of intelligence and cognitive ability batteries. Several conclusions can be drawn from the information presented in this table.
A Description of Cross-Battery Factor Analysis Studies.
The authors reanalyzed data from Keith and Novak (1987).
First, the CB-CFAs conducted since 1990 included a total of 20 cognitive batteries. However, 11 of the 14 studies included either the WJ-R or WJ III. Only one study included the WISC-IV (Reynolds et al., in press) and only one study included the WAIS-IV (Holdnack, Zhou, Larrabee, Mills, & Salthouse, 2011). Because the WJ-R and WJ III batteries were developed from Gf-Gc and CHC theory, respectively, the structure of these batteries guided nearly all CB-CFAs included in Table 4 and few models tested alternative explanations of the WJ factor structure. These studies were necessary to derive a better understanding of the broad cognitive abilities measured by commonly used cognitive batteries, including the Wechsler scales, but the CB-CFAs conducted to date are too WJ “heavy.”
Second, nine CHC broad cognitive abilities were represented among the CB-CFA studies, including Gf, Gc, Gv, Gsm, Glr, Ga, Gs, Gq, and Grw. The number of CHC broad ability factors representing the best fitting models in these studies ranged from five to nine, with Gf, Gc, Gsm, Gv, and Gs among the most frequently represented. The broad abilities of Gq and Grw were not included in most CB-CFAs because subtests that assess these abilities are typically measured by achievement batteries. The broad abilities of Ga and Glr were represented by only a few subtests in CB-CFAs because these abilities are not typically measured by a majority of intelligence batteries, including the Wechsler scales. Therefore, it is recommended that future Wechsler-based CB-CFAs include tests from batteries on which Ga and Glr are well represented.
Third, with few exceptions (e.g., Phelps et al., 2005), nearly all of the CB-FAs included in Table 4 modeled broad cognitive abilities. There needs to be more modeling of narrow abilities in light of their importance in understanding learning and in explaining performance in specific academic skills. Greater focus on narrow cognitive abilities may result in better diagnostic assessments (see Table 3).
Fourth, the sample sizes included in the analyses ranged from 53 to 4,261 (median sample size = 145), with the majority of samples ranging from 100 to 200 participants. Overall, the sample size to variable ratio is somewhat low for the type of analyses that were conducted. It is very likely that sample sizes are generally small because cross-battery studies are quite time consuming. That is, cross-battery studies require data from at least two batteries. Giving two complete (or even partial) batteries to a sample of individuals typically requires administration of about 20 subtests at a minimum. Conducting this type of study, therefore, is onerous for researchers and participants alike. Also, in many instances, it may be more desirable to administer subtests from multiple batteries to the same subjects, particularly to test models that include narrow abilities. In order to circumvent the limitations of CB-FAs, Keith and Reynolds (2010) suggested the use of reference variable methodology, which takes advantage of planned missingness (McArdle, 1994). This methodology combines data from different studies, given an overlapping series of tests, referred to as the reference variables (Keith & Reynolds; Reynolds et al., in press). To assist in designing future CB-CFAs, including those using the reference variable methodology, we compiled a table that includes all of the subtests (from Table 4) that have been included in CB-FAs and the number of times each subtest was included in an analysis (e.g., once, twice, three, or more times). This information (found in Table 5) can be used to identify the subtests from various batteries that have the most consistent construct validity support and therefore, can be used confidently as reference variables.
CHC Broad Ability Classifications of Cognitive Ability Subtests Based on Cross-Battery Factor Analyses.
Note: Subtests in bold, uppercase type indicate that they were included in at least three cross-battery factor analyses that yielded consistent results. Thus, there is little doubt regarding the CHC broad abilities they measure. Subtests in bold, lowercase type indicate that they were included in two cross-battery factor analyses that yielded consistent results. Thus, it is likely that they are measures of the CHC broad abilities listed. Subtests in lowercase type indicate that they were included in only one cross-battery factor analysis or results of multiple analyses were inconsistent. Subtests in italics indicate that they were not included in any cross-battery factor analysis to date and thus, CHC broad ability classifications are based on expert consensus and within-battery factor analyses. WISC-IV = Wechsler Intelligence Scale for Children-Fourth Edition (Wechsler, 2003); WAIS-IV = Wechsler Adult Intelligence Scale-Fourth Edition (Wechsler, 2008); WPPSI-III = Wechsler Preschool and Primary Scale of Intelligence-Third Edition (Wechsler, 2002); WNV = Wechsler Nonverbal Scale of Ability (Naglieri & Wechsler, 2006); KABC-II = Kaufman Assessment Battery for Children-Second Edition (Kaufman & Kaufman, 2004); WJ III NU COG = Woodcock-Johnson III, Normative Update, Tests of Cognitive Abilities (Woodcock, McGrew, & Mather, 2001, 2007); WJ III NU DS = Woodcock-Johnson III, Normative Update, Disagnostic Supplement (Woodcock, Schrank, & McGrew, 2004, 2007); WMS-IV = Wechsler Memory Scale–Fourth Edition (Pearson, 2009); SB5 = Stanford-Binet Intelligence Scales-Fifth Edition (Roid, 2003); DAS-II = Differential Ability Scales-Second Edition (Elliott, 2007). KAIT = Kaufman Adolescent and Adult Intelligence Test (Kaufman & Kaufman, 1993); CAS = Cognitive Assessment System (Naglieri & Das, 1997); D-KEFS = Delis-Kaplan Executive Function System (Delis, Kaplan, & Kramer, 2001); DTLA-4 = Detroit Test of Learning Aptitude-Fourth Edition (Hammill, 1998). Gf = Fluid Intelligence; Gc = Crystallized Intelligence; Gv = Visual Processing; Gsm = Short-term Memory; Glr = Long-term Storage and Retrieval; Ga = Auditory Processing; Gs = Processing Speed; Gq = Quantitative Knowledge.
The WNV has not been included in any cross-battery factor analyses to date. However, tests from this battery (e.g., Object Assembly, Picture Arrangement) were included in this type of analysis when they were on the WISC-III.
The DTLA-4 has not been included in any cross-battery factor analyses to date. However, the tests listed in the DTLA-4 row are the ones from the DTLA-3 that were included in this type of analysis.
Table 5 lists the current editions of the batteries that were included in CB-FAs. Subtests of the latest editions of cognitive batteries are included in Table 5 and are listed in the column of the broad ability presumed to be measured by the subtest. For example, the Wechsler Vocabulary subtest is listed in the Gc column, as this subtest measures a narrow Gc ability, namely lexical knowledge. Subtests printed in bold, uppercase type indicate that they were included in at least three CB-FAs that yielded consistent results. In general, the more times a subtest was included in CB-FA and the more consistently the subtest was found to measure a particular broad ability, the more confidence one can have in that classification. For example, the Wechsler Block Design subtest has consistently loaded on a Gv factor in more than three CB-FAs and therefore, we can be confident that the primary broad ability measured by this subtest is Gv.
Subtests printed in bold, lowercase type in Table 5 were included in two CB-FAs that yielded consistent results. KABC-II Atlantis and Rebus (listed under Glr) are in this category. It is very likely that these subtests (bold, lowercase) are measures of the CHC broad abilities listed. Subtests printed in lowercase type were included in only one CB-FA or results of multiple analyses were inconsistent. The Wechsler arithmetic subtest falls into this category for the latter reason. Subtests printed in italics in Table 5 were not included in any CB-FA analysis to date and thus, CHC broad ability classifications are based on expert consensus and within-battery factor analyses. Several Wechsler subtests fall into this category, such as Figure Weights and Word Reasoning. Although these cognitive tests have been classified according to the CHC broad abilities they measure via within-battery FA and content analysis using multiple experts as raters (e.g., Flanagan et al., 2013), the construct validity of these subtests will be enhanced when they are included in CHC designed CB-CFAs.
The results of CB-FAs summarized in Table 5 provide a strand of validity evidence for the CHC broad ability constructs underlying currently used cognitive ability tests. The subtests printed in bold uppercase letters should be used as markers of their respective broad CHC abilities in reference variable studies. Overall, Table 5 may be used to understand the CHC broad abilities measured by cognitive ability subtests and as a blueprint for designing future Wechsler-based (and other) CB-CFAs. The narrow ability classifications provided in Table 2 and in other sources (e.g., Flanagan et al., 2013; Miller, 2013) should be used to guide the design of Wechsler-based (and other) CB-CFAs that model narrow abilities.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
