Use of Brief Intelligence Tests in the Identification of Giftedness

Examples of Brief Measures of Intelligence

Given the state of the art of evaluation of ability, professionals considering the use of a short form for screening or identification purposes have several options available, including those that have been formally published. This section will overview the characteristics of some published forms used in school-based settings. Tests such as the Stanford–Binet and the Differential Abilities Scale have suggested abbreviated forms nested within their available protocols and manuals.

The Kaufman Brief Intelligence Test (KBIT; Kaufman & Kaufman, 1990) was one of the first short forms tied to a measure designed to assess intelligence based on theory—the Kaufman Assessment Battery for Children (K-ABC; Kaufman & Kaufman, 1983). Constructed to measure Luria’s theory of brain organization, the K-ABC and its derivative, the KBIT, attempted to assess both verbal and nonverbal skills. The revised Kaufman Brief Intelligence Test-Second Edition (KBIT-2; Kaufman & Kaufman, 2004b), resulting from the Kaufman Assessment Battery for Children-Second Edition (K-ABC-II; Kaufman & Kaufman, 2004a), is now grounded in CHC theory, but it can also be interpreted according to Luria’s neuropsychological model in cases where the child displays a diverse cultural or educational background (Kaufman, Kaufman, Kaufman-Singer, & Kaufman, 2005). Both versions of the KBIT have three subtests—two verbal (verbal knowledge and riddles) and one nonverbal subtest (matrices). After examining the three subtests separately, the KBIT-2 was standardized on a sample of 2,120 children using random sampling methods to reflect demographic data from the 2001 U.S. Census (Kaufman & Kaufman, 2004b). The lowest split-half coefficient for different age groups was .78; the rest fell in the .80s and .90s.

Validity for the use of the KBIT to estimate cognitive ability has been determined through a variety of methods. This includes cross-battery comparisons with the WISC-III in different populations of students (Canivez, 1996; Grados & Russo-Garcia, 1999; Thompson, Browne, Schmidt, & Boer, 1997). The generally positive results indicate that the KBIT was a good predictor of WISC-III scores and measured the common underlying construct of ability. Bain and Jaspers (2010) noted in their review of the KBIT-2 that previous studies demonstrated moderate correlations between the KBIT and the WRAT3 (Wide Range Achievement Test-3; Wilkinson, 1993). They also reported that the KBIT-2 may underestimate the scores of gifted students aged between 7 and 16 for the nonverbal portion of the measure.

After many years of various researchers and clinicians using shortened forms of the Wechsler scales on the basis of altered standardization processes, the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999) was published. Intended for a broader age range than its age-specific predecessors, the WASI is normed for those aged between 6 and 89 years. This age range may make the WASI a more appealing instrument for the identification of exceptional students than short forms of the WISC-IV because the addition of harder items for measures of knowledge and reasoning as well as visual reasoning and problem solving reduce problems with subtest ceiling effects. In addition, the extended age range (6 to 89) makes it appealing for those assessing adolescents where the overlap of age levels in different versions of tests can make battery selection difficult.

The WASI is comprised of four subtests: vocabulary, similarities, matrix reasoning, and block design (Wechsler, 1999). Vocabulary and similarities make up a verbal IQ (VIQ), whereas matrix reasoning and block design make up the performance IQ (PIQ). Full-scale scores can be derived from all four subtests (FSIQ-4) or only from vocabulary and matrix reasoning (FSIQ-2). Three of the four subtests require the individual to provide open-ended responses or develop a structure in space as opposed to selecting from a multiple-choice format. This can provide insight into individual characteristics and problem-solving behaviors.

Normed on a nationally representative sample of 2,245 individuals, the two-subtest version of the WASI received reliability coefficients of .93 for children and .96 for adults, whereas the four-subtest version received reliability coefficients of .96 for children and .98 for adults (Wechsler, 1999). When compared with WAIS-III FSIQ scores, the FSIQ-4 was found to correlate at .92 and the FSIQ-2 at .87 (Axelrod, 2002). Several studies have found support for the construct validity of the WASI (Canivez, Konold, Collins, & Wilson, 2009; Ryan et al., 2003). Ryan and Brown (2005) reported that the reliability of the WASI was satisfactory. It should be noted that both the WASI and the KBIT-2 have subtests unique to them that are not part of their parent batteries. This added dimension is especially important to researchers who may be concerned with the prediction problems on test correlations previously discussed by Kaufman and Kaufman (2004b). The second edition of the WASI has an anticipated release date in the fall of 2011. At the time of this writing, little information pertaining to its development or standardization is available. However, given the history of the Wechsler Series and the high quality of typical normative samples, it is anticipated that the WASI-II will have standardization data set consistent with the demographic trends of the most recent census estimates and adequate sample sizes at all ages.

It would appear then that greater attention is now being given to the makeup of abbreviated forms of intelligence not only to improve their predictive validity but also to offer insight into factors related directly to educational performance. This awareness of test utility is pertinent in cases where short forms screen or identify giftedness because improved testing techniques may well maximize the information obtained for the students assessed.

Limitations of Using Short Forms

Although short forms have the benefit of time savings, Thompson, Howard, and Anderson (1986) provided evidence that altering the order of subtest administration, as is done in a shortened form, may reduce the ability of the short form to accurately estimate the individual’s full-scale score. Kaufman and Kaufman (2001) also questioned the validity of short forms that use only one out of two or one out of three items such as in the Satz and Mogel format to reduce test length. They argued that this changes the test-taking procedure by increasing the slope of item difficulty. Changes in the structure of the original test to create the abbreviated form may compromise the quality of the performance observed as well as the data obtained in the evaluation. In particular, the greater distance in difficulty between adjacent items on a subtest may make it that much more difficult for clinicians to discern where problems begin to emerge in regards to the difficulty of items. It is interesting to note that examiners using the current version of the Stanford-Binet (Roid, 2003) may experience a similar sense when testing children on different testlets. ¹

The current literature regarding the measurement of intelligence in samples of gifted and high-ability students has several limitations. Most brief and full-battery cognitive measures do not include information regarding the specificity or sensitivity of tests in classifying individuals for gifted programs or extensive information regarding the criteria for selection, goals of the program, or program philosophy. As a result it is difficult to gauge whether mean scores obtained in classification samples reflect the psychometric properties of the test or the selection criteria and philosophy of the program.

Related to this problem is the reporting of means and standard deviations of persons in gifted samples on measures of cognitive ability. Although this practice is consistent with the broader practice in psychology and education of reporting means and standard deviations, it ignores the basic principles as to why and when means are appropriate. Other measures of central tendency such as interpolated medians may be more helpful to report alongside of means and standard deviations. This would be particularly helpful given the fact that these samples are oftentimes likely to be skewed in distribution (Gagné, 2004).

Predictive Validity

The degrees to which abbreviated or brief measures of intelligence estimate a full-scale measure have largely been discussed relative to their utility. These included the high degree of intercorrelations that will be seen between the brief and composite measure and the effects of introducing an alteration of test order in administration (Kaufman & Kaufman, 2004b; Thompson et al., 1986). Additional concerns arise when assessing individuals at the extreme tail of the bell curve, as is the case in identifying exceptionally able students (McIntosh, Dixon, & Pierson, in press; Ziegler & Ziegler, 2009). In more specific terms, individuals who are administered measures of cognitive ability and who are gifted will tend to score lower than their actual level of innate measurement as a function of measurement error. So children who are tested every year or 2 years as part of an ongoing identification strategy may have years where measurement error incorrectly classifies them as not gifted. This last concern is a primary argument against the use of procedures involving the averaging the repeated measurements of cognitive abilities or the need for multiple-stage evaluation procedures in the identification of gifted children where they need to pass each level. This view may seem counterintuitive to those familiar to the advice given by Gallagher (1994) and Gridley, Norman, Rizza, and Decker, (2003), who advocate the use of multiple tests in assessment. It is critical for parents and teachers to understand that performances may dip below cutoff scores on similar measures of intelligence such as repeated testing on a different cognitive measure as a function of the limitations of our measures and not students or programs.

As noted earlier, the very fact that the measures are brief forms raises concerns about their validity. By definition, when a brief measure is used the observed sample of behavior will be shorter and provides the individual fewer opportunities to demonstrate superior performance. This may not be a primary concern for those researchers interested in profiles or psychometric questions regarding giftedness, but it may be to professionals involved in psychological service delivery or educational planning. Gridley and colleagues (2003) provided an outline of characteristics of intellectually gifted and talented students who may be observed during the testing session.

Cutoff Scores and Their Recommended Uses

Local schools and educational agencies often have differing criteria and methods for the identification of giftedness. As a result, programs in areas with lower population density may have a lower cutoff or minimum obtained score that they have selected as necessary to enter the only gifted and talented program in an area. In contrast, programs in urban settings or with large catchment areas involving state math and science academies may have much higher criteria for entry. As a result, gifted and talented programs in low-population-density areas may serve and identify a more diverse group of gifted students because they use lower entry criteria for a variety of measures. Those in the high-density or large catchment groups may instead offer services to a more focused and extremely talented group.

Because of this variability in identification procedure, a wide range of possible cutpoints or criteria can be applied to connote giftedness. The widespread adoption and recognition by researchers of the utility of the CHC theory of intelligence advocates its use in the screening and identification of gifted students. Most of the major test instruments available have guides on how to interpret test data according to the CHC theory. The use of the multiple strata of intelligence found in the theory allows for the identification of a richer, more diverse group of gifted students than would have been possible through screening procedures or identification procedures using full-scale scores only. In addition, it is suggested by Gridley et al. (2003) and McIntosh and Dixon (2005) that decisions regarding eligibility involve the use of the confidence intervals of scores to better capture the scope of the student’s performance.

For practical purposes, the choice of cutoff score point defining gifted performance will often be tied to the size of the desired program in an educational setting, but several factors should be considered when determining what the point or criteria should be. For example, the Flynn effect (Flynn, 1984) will cause some students with lower levels of ability to “float” into programs over time. Measurement and the tendency for test measurement to move toward the mean will force students who are gifted to score lower on selected tests at different points in time depending on the measures used and the match to their particular pattern of strengths. Multiple methods and multiple informants may improve the quality and ecological validity of such high-stakes decision making.

Use of Brief Measures With Minorities When Identifying Giftedness

The issue of diversity in the assessment and identification of giftedness is one of primary importance for societal as well as educational need. There is a long history of research that has identified the manner in which traditional measures of cognitive ability and intelligence can and have been misapplied when working with minority students (Jacob, Decker, & Hartshorne, 2010). Terman (1922) recognized that in the identification of gifted students certain ethnic groups were being selected at greater-than-expected rates whereas others were being selected at lower-than-expected rates.

Modern measures of cognitive assessment strive to minimize differences between ethnic and gender groups on measures and oftentimes view differences on items as measurement artifacts that should be minimized or removed. Contemporary batteries of cognitive assessment frequently include sample responses for words from Spanish as recognition of the growing need to have that information available to an examiner (Wendling, Mather, & Schrank, 2009). Cross and Coleman (2005) also point to the ability of objective measures of intelligence to be useful in helping to identify students who might otherwise be overlooked. As a result, short measures may introduce perspectives to interpretation of performance that might minimize bias toward the majority culture. For instance, the KBIT-2 (Kaufman & Kaufman, 2004b) can be interpreted using Luria’s model of neuropsychological development that deemphasizes the crystallized or culturally derived component of ability. Another brief measure that has a relatively large normative matched to the demographics of the United States and available for ages 4:0 to 21:11 is the Wechsler Nonverbal Scale (WNV). The WNV may be of particular benefit in the evaluation of individuals who are English language learners (extensive review of the WNV can be found in Naglieri & Brunnert, 2009). Preliminary validity studies reported in the manual use categories such as good or better ability to discriminate between gifted and nongifted sample of students as other, more comprehensive, measures of intelligence. More research with diverse samples using both the KBIT-2 and the WNV is needed.

Construct Validity of Brief Measures

The current generation of brief measures of cognitive ability has increased potential and clinical as well as educational utility for the evaluation of intelligence in individuals. It is anticipated that as time and the science of psychology continues to develop so will the ability to predict and discriminate between groups and utilize time and personnel optimally. Unlike previous generations of brief IQ measures, the current generation for the most part is well grounded in theory (albeit not gifted theory). Test developers have taken a keen eye to the changing demographics of the U.S. population and are working to incorporate better tests to serve diverse populations. Although previous generations of tests may have been atheoretical or designed to support competing models of intelligence, the preeminent theory driving their development today is the CHC model. It is unfortunate that tests based on the CHC model have not resolved all issues associated with evaluating a gifted population. In more specific terms, no current battery has shown a mean value above 125 when test publishers validated the measure. Linking all of our tests and measures to one theory ties all of our measures to its limitations and reduces our conceptualization of intelligence beyond CHC.

List of Recommendations

The authors encourage those interested in integrating brief measures into the evaluation and selection of gifted individuals to review the work of Newton, McIntosh, Williams, and Youman (2008) and the highly regarded chapter by Thompson and Morris (2008). In brief, it is recommended that programs work in accordance with the guidelines of the American Psychological Association (2002) and the National Association of School Psychologists (2010) codes of ethics and consider all high-stakes decision making within the context of the error ranges of the instruments being employed. Similarly, schools and psychologists should choose measures that are clearly linked to gifted and intelligence theory. Tests, whether brief or full batteries, should be chosen based on adequate normative information for the sample that matches the target client group being assessed. When using brief cognitive measures, any identification plan should include supplementary measures and techniques to identify other potential areas of giftedness in line with the program’s philosophy and objectives.

Careful consideration should also be given to who administers cognitive measures. Schools should choose personnel who are familiar with the personality characteristics of gifted students that may lead to lowered estimates of cognitive ability on IQ measures. Schools should administer brief IQ measures only to those students who are not suspected of having a learning disability or ADHD (attention-deficit hyperactive disorder). Finally, the decision to use one or more cognitive measures in the evaluation and identification of giftedness should take into consideration the likelihood that one or more measures may end up with a lower-than-anticipated obtained score as a function of test theory and not as an innate ability on the part of the individual.

Although some brief measures such as the KBIT-2 may be administered by individuals with less training in psychological testing, we would advise against this practice. Individuals with limited training in testing may not have sufficient knowledge or experience to recognize anxiety, low motivation, cultural variations, or other factors that may lead to an underestimation of ability. We also recommend that the results of brief measures be considered in the context of the individuals cultural experiences, personal history, and psychometric properties of the instrument.