The consensus of most intelligence scholars is that the Flynn effect (FE) is real, IQ test batteries are now routinely restandardized on a regular basis. A cornerstone in Flynn’s explanation of the FE is his analysis of select Wechsler subtest scores across time. The featured articles by Kaufman and Zhou, Zhu, and Weiss question whether Flynn’s arguments are grounded in the unproven assumption that similarly named Wechsler scores measure the same constructs across editions. Kaufman raises the issue by means of a detailed task analysis of changes in test administration and scoring directions for similarly named tests across different Wechsler editions. The author applauds Zhou et al. for bringing methodological rigor to the comparison of similarly named Wechsler Performance composite scores across time. Unfortunately, both Kaufman and Zhou et al. inadvertently perpetuate some of Flynn’s incorrect interpretations of select Wechsler measures (Similarities and Performance tests) as measures of the novel abstract problem solving that characterizes fluid intelligence (Gf). The author presents empirical Wechsler subtest g-loadings based on seven Wechsler joint- or cross-battery factor analyses (with other cognitive batteries). The results suggest that the extant Wechsler FE data and its system of interpretations, hypotheses, and resultant theory are held together by multiple anchors, a number that, in the words of Kaufman, are “seriously coated in rust.” The author briefly discusses the theory, tools, and technologies that currently exist to place a more reasonable degree of order in the house built by Flynn.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. ( 1999). Standards for educational and psychological testing. Washington, DC: Author.
2.
Beaujean, A., & Osterlind, S.J. (2008). Using item response theory to assess the Flynn Effect in the National Longitudinal Study of Youth 79 Children and Young Adults Data. Intelligence, 36, 455-463.
3.
Carroll, J.B. ( 1993). Human cognitive abilities: A survey of factor analytic studies. New York, NY: Cambridge University Press.
4.
Cummins, J. ( 1979). Cognitive/academic language proficiency, linguistic interdependence, the optimum age question and some other matters. Working Papers on Bilingualism, 19, 121-129.
5.
Dorans, N.J., Pommerich, M., & Holland, P.W. ( 2007). Linking and aligning scores and scales. New York, NY: Springer.
6.
Embretson, S.E. ( 1996). The new rules of measurement. Psychological Assessment, 8, 341-349.
7.
Embretson, S.E., & Hershberger, S.L. (1999). The new rules of measurement . Mahwah, NJ: Lawrence Erlbaum.
8.
Flanagan, D.P., McGrew, K.S., & Ortiz, S.O. ( 2000). The Wechsler Intelligence Scales and Gf-Gc theory: A contemporary approach to interpretation. Boston, MA: Allyn & Bacon.
Flynn, J.R. ( 2006). Tethering the elephant: Capital cases, IQ, and the Flynn effect. Psychology, Public Policy, and Law, 12, 170-189.
11.
Flynn, J.R. ( 2007). What is intelligence? Beyond the Flynn effect. New York, NY: Cambridge University Press.
12.
Flynn, J.R. ( 2009). The WAIS-III and WAIS-IV: Daubert motions favor the certainly false over the approximately true. Applied Neuropsychology , 16, 98-104.
13.
Flynn, J.R., & Weiss, L.G. ( 2007). American IQ gains from 1932 to 2002: The WISC subtests and educational progress. International Journal of Testing , 7, 209-224.
14.
Hunt, E. ( 2000). Let’s hear it for crystallized intelligence. Learning and Individual Differences, 12, 123-129. Jensen, A.R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
15.
Kaufman, A.S. ( 2009). IQ testing 101. New York, NY: Springer.
16.
Kaufman, A.S. ( 2010). "In what way are apples and oranges alike?" A critique of Flynn’s interpretation of the Flynn effect. Journal of Psychoeducational Assessment, 28, 382-398.
17.
Livingston, S. ( 2004). Equating test scores (without IRT). Princeton, NJ: Educational Testing Service.
18.
Marshalek, B., Lohman, D.F., & Snow, R.E. ( 1983). The complexity continuum in the Radex and hierarchical models of intelligence. Intelligence, 7, 107-127.
19.
McGrew, K.S. ( 1997). Analysis of the major intelligence batteries according to a proposed comprehensive Gf-Gc framework. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 151-179). New York, NY: Guilford Press.
20.
McGrew, K.S. ( 2005). The Cattell-Horn-Carroll theory of cognitive abilities . In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (2nd ed., pp. 136-181). New York, NY: Guilford Press.
21.
McGrew, K. ( 2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research [Editorial]. Intelligence, 37, 1-10.
22.
McGrew, K. ( 2010, February). AP101 Brief # 6b supplement: Summaries of Wechsler CHC test classifications. Retrieved from http://www.iqscorner.com/2010/02/ap101-brief-6b-supplement-summaries-of.html
23.
McGrew, K., Dailey, D., & Schrank, F. ( 2007). Woodcock-Johnson III/Woodcock-Johnson III Normative Update score differences: What the user can expect and why (Woodcock-Johnson III Assessment Service Bulletin No. 9). Rolling Meadows, IL : Riverside.
24.
McGrew, K., & Flanagan, D. ( 1998). The Intelligence Test Desk Reference (ITDR): Gf-Gc cross-battery assessment. Boston, MA: Allyn & Bacon.
25.
McGrew, K.S., Werder, J.K., & Woodcock, R.W. (1991). The WJ-R technical manual . Rolling Meadows, IL: Riverside .
26.
McGrew, K.S., & Woodcock, R.W. (2001). Technical manual. Woodcock-Johnson III. Rolling Meadows, IL: Riverside .
27.
Roberts, R.D., Markham, P.M., Matthews, G., & Zeidner, M. ( 2005). Assessing intelligence: Past, present, and future. In O. Wilhelm & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 333-360). Thousand Oaks, CA: SAGE.
28.
Rodgers, J.L. ( 1999). A critique of the Flynn Effect: Massive IQ gains, methodological artifacts, or both?Intelligence, 26, 337-356.
29.
Sattler, J. ( 2001). Assessment of Children: Cognitive Applications- 4th edition . San Diego, CA: Jerome M. Sattler, Publisher, Inc.
30.
Snow, R.E., Lohman, D.F., Marshalek, B., Yalow, E., & Webb, N. ( 1977). Correlational analyses of reference aptitude constructs (Technical Report No. 5). Stanford, CA: Aptitude Research Project, School of Education, Stanford University.
31.
Stankov, L. ( 2000). Structural extensions of a hierarchical view on human cognitive abilities. Learning and Individual Differences, 12, 35-51.
32.
Stankov, L. ( 2005). g factor: Issues of design and interpretation. In O. Wilhelm & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 279-293). Thousand Oaks, CA: SAGE.
33.
Wechsler, D. ( 2004). WISC-IV integrated technical and interpretation manual . San Antonio, TX: Pearson.
34.
Wechsler, D. ( 2008). WAIS-IV technical and interpretation manual. San Antonio, TX: Pearson.
35.
Weiss, L.G. ( 2007). Response to Flynn. WAIS-III technical report. San Antonio, TX: Harcourt Assessments.
36.
Wicherts, J.M., Dolan, C.V., Hessen, D.J., Oosterveld, P., vanBaal, G.C.M., Boomsma, D.I., & Span, M.M. ( 2004). Are intelligence tests measurement invariant over time? Investigating the nature of the Flynn effect. Intelligence , 32, 509-537.
37.
Wilhelm, O. ( 2005). Measuring reasoning ability. In O. Wilhelm & R. W. Engle (Eds.), Handbook of understanding and measuring intelligence (pp. 373-392). Thousand Oaks, CA: SAGE.
38.
Woodcock, R.W. ( 1978). Development and standardization of the Woodcock-Johnson Psycho-Educational Battery. Hingham, MA: Teaching Resources.
39.
Woodcock, R.W. ( 1990). Theoretical foundations of the WJ-R measures of cognitive ability. Journal of Psychoeducational Assessment, 8, 231-258.
Woodcock, R.W., McGrew, K.S., & Mather, N. ( 2001). Woodcock-Johnson III. Rolling Meadows, IL: Riverside.
43.
Zhou, X., Zhu, J., & Weiss, L.G. ( 2010). Peeking inside the "black box" of the Flynn effect: Evidence from three Wechsler instruments. Journal of Psychoeducational Assessment, 28, 399-411.
