Journal of Psychoeducational Assessment

Flynn wrote a book devoted to the Flynn effect, featuring his theoretical explanation of why the intelligence of worldwide populations has apparently increased from generation to generation. The essence of his theorizing is that because of the societal impact of scientific technology, people of today are much more guided by abstract, rather than concrete, approaches to problem solving. He bases his theory in large part on gains on specific tasks, most notably Raven’s matrices, Wechsler Intelligence Scale for Children (WISC) Performance subtests, and, most important, on WISC Similarities. The gains on these separate tasks over more than half a century (1947-2002) are striking. However, Flynn failed to take into account the sweeping changes in test content, administration procedures, and scoring guidelines when the 1974 WISC-R was developed from the 1949 WISC. These substantial changes challenge the meaningfulness of comparing children’s performance in 1947 with their performance in 2002 on Similarities and other WISC subtests— and therefore challenge Flynn’s explanation of the effect that bears his name.

This study investigated the Wechsler Performance IQ (PIQ) or Perceptual Reasoning Index (PRI)/ Perceptual Organization Index (POI) change over time and its relation to ability levels. PIQ or PRI/ POI was analyzed because of the known sensitivity of nonverbal scales to the Flynn effect. Scores were analyzed using two methods. First, analysis of covariance was applied to the combination of four representative samples of individuals who were administered the following pairs of Wechsler batteries in counterbalanced order: Wechsler Preschool and Primary Scale of Intelligence—Revised (WPPSI-R) and WPPSI-III (N = 174), Wechsler Intelligence Scale for Children — Third Edition (WISC-III) and WISC-IV (N = 239), Wechsler Adult Intelligence Scale—Revised (WAIS-R) and WAIS-III (N = 191), and WAIS-III and WAIS-IV ( N = 240). Second, equal percentile equating was applied to each of the samples independently. Although the two methods produced different patterns of results, both methods showed some evidence of variation in the magnitude of the Flynn effect across ability levels. These results call into question the practice of adjusting IQs based on an average expected Flynn effect in routine clinical evaluations.

Despite Kaufman, Raven’s Progressive Matrices and the Wechsler subtest Similarities are tests whose gains call for special explanation. The spread of “scientific spectacles” is the key, but its explanatory potential has been exhausted. Three trends force us to look elsewhere: (a) gains on Wechsler subtests such as Picture Arrangement, (b) gains in developed nations persisting into the 21st century, (c) the growing gap between the active vocabularies of parents and their children.

The Flynn effect probably has multiple causes, and the target essays in this issue have expanded the number of possible causes behind it. This essay deals primarily with a different question: How important is IQ in the current world and should it perhaps be understood also in conjunction with a consideration of some kind of ethical intelligence?

While the magnitude of the Flynn effect is well established (approximately 3 points a decade on the Wechsler scales), the causes behind it are still unknown and hotly debated. Kaufman argues that, because of the administrative and scoring changes that occurred with the introduction of the Wechsler Intelligence Scale for Children—Revised, Flynn’s interpretation of the effect is not appropriate. Although agreeing that these changes account for some aspects of rising IQ, this study questions the impact of these administrative/scoring changes to account for most of the impact, given the heavy documentation of the Flynn effect on multiple IQ tests and norms over time and around the world. The authors also add to the discussion led by Zhou, Zhu, and Weiss by stressing the importance of examining the role of individual differences within the Flynn effect to understand fully the exact nuances and cause(s) of it.

The consensus of most intelligence scholars is that the Flynn effect (FE) is real, IQ test batteries are now routinely restandardized on a regular basis. A cornerstone in Flynn’s explanation of the FE is his analysis of select Wechsler subtest scores across time. The featured articles by Kaufman and Zhou, Zhu, and Weiss question whether Flynn’s arguments are grounded in the unproven assumption that similarly named Wechsler scores measure the same constructs across editions. Kaufman raises the issue by means of a detailed task analysis of changes in test administration and scoring directions for similarly named tests across different Wechsler editions. The author applauds Zhou et al. for bringing methodological rigor to the comparison of similarly named Wechsler Performance composite scores across time. Unfortunately, both Kaufman and Zhou et al. inadvertently perpetuate some of Flynn’s incorrect interpretations of select Wechsler measures (Similarities and Performance tests) as measures of the novel abstract problem solving that characterizes fluid intelligence (Gf). The author presents empirical Wechsler subtest g-loadings based on seven Wechsler joint- or cross-battery factor analyses (with other cognitive batteries). The results suggest that the extant Wechsler FE data and its system of interpretations, hypotheses, and resultant theory are held together by multiple anchors, a number that, in the words of Kaufman, are “seriously coated in rust.” The author briefly discusses the theory, tools, and technologies that currently exist to place a more reasonable degree of order in the house built by Flynn.

IQ test scores should be corrected for high stakes decisions that employ these assessments, including capital offense cases. If scores are not corrected, then diagnostic standards must change with each generation. Arguments against corrections, based on standards of practice, information present and absent in test manuals, and related issues, ignore expert consensus about the assessment of intellectual disabilities and the acceptance of the Flynn effect in the field. Most psychometric concerns about correction are based on validity studies with small subgroups and do not reflect sufficient effort to estimate the precision of the Flynn estimate. We computed a confidence interval for the Wechsler PIQ across four validity studies that shows a SEM of about 1 around a mean of about 3 points per decade. A meta-analytic weighted mean of the 14 studies in Flynn (2009) is 2.80 (2.50, 3.09), close to Flynn’s (2009) unweighted average (2.99). More psychometric research would be helpful, but this level of precision supports the Flynn adjustment of 3 points per decade.

Atkins v. Virginia (2002) dramatically raised the stakes for mental retardation in capital punishment cases, but neither defined this condition nor imposed uniform standards for its assessment. The basic premise that mean IQ scores shift over time enjoys wide recognition, but its application— including the appropriateness of characterizing it in terms of an allegedly predictable “Flynn effect”— is frequently debated in the course of death penalty litigation. The scientifically and ethically sound approach to this issue is to report IQ scores as obtained and be prepared to address those factors that might affect their reliability. Altering the IQ scores themselves is insufficiently supported by professional literature, legal authority, or prevailing standards of practice.

The Flynn Effect is a well documented phenomenon demonstrating score increases on IQ measures over time that average about 0.3 points per year. Normative adjustments to scores derived from IQ measures normed more than a year or so prior to the time of testing an individual have become controversial in several settings but especially so in matters of death penalty litigation. Here we make the argument that if the Flynn Effect is real, then a Flynn Correction should be applied to obtained IQs in order to obtain the most accurate estimate of IQ possible. To fail to provide the most accurate estimate possible in matters that are truly life and death decisions seems wholly indefensible.

Flynn has proposed a grand integrative theory, which he calls “scientific spectacles,” to explain the phenomenon of rising IQ scores across multiple decades known as the Flynn effect (FE). In his theory, he purports that modern society has placed increasing value and emphasis on the application and education of scientific principles—which include abstract fluid reasoning—and this has mirrored larger score increases in the abstract fluid reasoning tasks included as part of most intelligence tests. He highlights huge gains in the Wechsler Similarities subtest over time as one linchpin supporting his theory that fluid reasoning is rising faster than other aspects of intelligence, but further points to large increases in Wechsler performance tasks (i.e., perceptual organization and reasoning) relative to crystallized tasks, and to the many studies of Raven’s Progressive Matrices, which is widely held to be a fairly pure measure of fluid reasoning. As with any new theory of this magnitude, Flynn’s grand proposal has invited numerous criticisms that must now be addressed—and this is as it should be in the spirit of scientific progress. Each of these issues is multilayered and has many legitimate perspectives. The present article attempts to carefully consider each issue and perspective in sequence.

In the first article of this special issue of the Journal of Psychoeducational Assessment, I critiqued Flynn’s theoretical explanation of the Flynn effect because he depended too heavily on an apparently huge generational gain on the WISC Similarities subtest; I claimed he was comparing apples with oranges because that subtest changed too much when the WISC was first revised. Four sets of esteemed researchers were invited to respond to my article and also to an article by Zhou, Zhu, and Weiss—Flynn, Sternberg, McGrew, and Ceci and Kanaya. Flynn disagrees strongly with my critique of his theory, a theory that posits striking generational shifts from concrete (utilitarian) thinking to a kind of fluid reasoning that Flynn nicknames “scientific spectacles.” In this final article of the special issue, I respond to Flynn’s claims, and also to the points made by the other invited respondents. In addition, I respond to the range of opinions expressed by the scholars who were invited to write an essay on whether or not IQs should be adjusted for the Flynn effect in capital punishment cases (Fletcher et al., Hagan et al., and Reynolds et al.). Ultimately, I disagree with Flynn’s explanation of the Flynn effect, but I agree with his position that IQs should be adjusted for the effect in death penalty cases.