Assessment Practices and the Underrepresentation of Minority Students in Gifted and Talented Education

Defining Giftedness

Much of the controversy in this area stems from a fundamental misunderstanding of the construct of giftedness. That is, giftedness is perceived to be a trait that some individuals possess and some do not. Moreover, the trait is presumed to exist in equal proportions across all demographic groups, leading to the assumption that any gifted program that does not reflect the demographic makeup of the district or school is somehow biased. However, whether in athletics, academics, the performing arts, or any other endeavor, giftedness is not about the potential that you have in the domain; rather, giftedness is the manifestation of that potential through actual accomplishments in the real world. In other words, giftedness is not about who you are but what you do (see Pfeiffer article in this issue). The importance of accomplishment as the hallmark of giftedness is succinctly stated in a position paper on the website of The National Association for Gifted Children (NAGC):

Gifted individuals are those who demonstrate outstanding levels of aptitude (defined as an exceptional ability to reason and learn) or competence (documented performance or achievement in top 10% or rarer) in one or more domains. Domains include any structured area of activity with its own symbol system (e.g., mathematics, music, language) and/or set of sensorimotor skills (e.g., painting, dance, sports). . . . Exceptionally capable learners are children who progress in learning at a significantly faster pace than do other children of the same age, often resulting in high levels of achievement. (NAGC, n.d.)

As this quotation makes clear, individuals who are labeled gifted are at the upper end of the distribution of achievement or potential for achievement (i.e., aptitude) in any domain (cf. Worrell, 2003, 2009b).

It is worth noting that this definition is quite broad and does not prescribe the criteria that we should use for identification. The definition also includes academic and nonacademic domains. Given that (a) regular public schools are typically focused on and best suited for developing academic gifts and (b) the identification of nonacademic gifts (e.g., music, dance, basketball) does not rely on psychoeducational assessments, this article will focus on academic giftedness.

Assessing for Placement in GATE Programs

Are Standardized Tests Biased Toward Underrepresented Minority Groups?

In measurement terms, this type of bias (Ford, 1998) refers to systematic error in test scores on the basis of group membership. In other words, group membership (e.g., being African American or Hispanic/Latino) introduces construct-irrelevant variance into the scores, with the scores reflecting the examinee’s demographic background in addition to or rather than the construct being assessed (e.g., reading, mathematics, intelligence). The American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NMCE; 1999) have outlined several sources of bias in tests, including those related to content and responses as well as issues in prediction and selection.

Content-related bias refers to whether questions or instructions from tests are unfair for a specific group (or groups). For instance, ethnic minority students may be less familiar with the content of items on a test than their majority peers, may provide incorrect answers that would be considered correct in the context of their culture, or may have simply not been afforded the opportunity to learn the test’s content (Reynolds & Carson, 2005). Today, there are many sophisticated procedures for detecting bias at the item level or differential item functioning (DIF), and these procedures are routinely used to exclude problematic items from tests. With regard to structural bias, factor analyses of the instruments reveal similar internal structures across ethnic groups, supporting the hypothesis that these tests are tapping the same construct across ethnic groups (e.g., Kush et al., 2001; Roid, 2003). A third area of potential bias is predictive. All things being equal, tests should be able to accurately predict performance in the construct that they measure. In statistical terms, regression equations relating to the test and the construct being assessed should not differ between groups if no bias exists (AERA et al., 1999), and examinations of the major standardized instruments have found no evidence of bias against the underrepresented groups (Frisby & Braden, 1999).

In sum, standardized tests of intelligence effectively measure the constructs that they purport to measure across all ethnic and racial groups and across all achievement levels. To the extent that these measures predict school performance, it would be unreasonable to exclude them for the purpose of gifted identification. We do claim, however, that neither are the constructs assessed by these tools the sole predictors of giftedness or academic achievement nor should they be the sole criteria for gifted identification. Instead, we argue that multiple sources of evidence should always be used in making decisions (Worrell, 2009a), a position recommended by several prominent organizations in the field (e.g., AERA et al., 1999; NAGC, 2008; National Research Council, 2002).

The Importance of Multiple Sources of Information

Given the broad view of giftedness introduced earlier in the article and the validity evidence supporting the use of IQ and other tests in making predictions about academic performance, how do we explain the underrepresentation of certain groups in GATE programs? Research clearly demonstrates that complex outcomes such as academic achievement are determined by multiple factors, including socioeconomic status, teacher qualifications and effectiveness, and academic engagement inside and outside of the classroom (Brody, 1997; Ceci & Williams, 1997; Neisser et al., 1996), and there is a considerable amount of information about these variables across racial/ethnic groups.

For example, the percentages of fourth graders—and by extension all students—eligible for free and reduced-price lunch are substantially higher for African Americans (74%), American Indians (68%), and Hispanic Americans (77%) than for Asian Americans (34%) and European Americans (29%), and these data reflect the differences in child poverty in the United States (Aud, Fox, & KewalRamani, 2010). Asian Americans have the lowest number of absences in school and spend more hours on homework than other groups, resulting in more time engaged with academic materials. More than 50% of African American and Hispanic American students who are employed work more than 20 hr per week, a variable related to lower academic performance. Higher percentages of African American and Hispanic American students are threatened or injured with weapons on school property than their Asian American and European American peers, and African Americans and Hispanic Americans are more likely to be taught by teachers with lower qualifications (Aud et al., 2010). Given these data and their relationship to academic achievement, one can predict with a fair amount of certainty which groups of students are likely to have lower achievement scores, and the patterns that these data predict are the patterns that currently exist.

There are also other several theoretical models that explicitly link ethnic minority status to academic achievement. For example, Ogbu and Simons (1998) contended that some minority groups (e.g., African Americans, American Indians) actively resist doing well in school, which they equate with historic oppression and current discrimination. As doing well in school is considered acting White (Ogbu, 2004), these researchers argue that substantial proportions of students choose to disengage from schooling. Ford, Grantham, and Whiting (2008) provided empirical support for this thesis; they reported that African American students in regular and gifted education associated acting White with being achievement oriented and acting Black with poor academic performance.

Others (e.g., McKown & Weinstein, 2003; Steele & Aronson, 1995) have suggested that the negative societal views about African Americans’ intellectual capacity results in a stereotype threat that is activated in testing situations and results in poor performance on the part of this group. Some proponents of this position (e.g., Walton & Spencer, 2009) argue that this psychological threat “causes measures of academic performance to underestimate the true intellectual ability and potential of ethnic minority students” (p. 1137). These theories suggest that many minority students may feel that they have to choose between (a) doing well in school (an academic identity) or (b) being an authentic member of their racial/ethnic group (a cultural identity).

Despite the failure to replicate the seminal research on teacher expectation effects (Rosenthal & Jacobson, 1968), more recent research in this area (e.g., McKown & Weinstein, 2002; Weinstein, 2002) suggests that teacher expectation effects may be more potent for students from ethnic minority backgrounds. In a comprehensive review of this literature, Jussim and Harber (2005) concluded that although teacher expectation effects are small and typically accurate, these effects are larger (a) in classrooms where teachers engage in high differential treatment of students, (b) for low-achieving students from low-socioeconomic-status backgrounds, and (c) for African American students. Although the applicability of some of these theories in real-world settings remains contentious—for example, how much variance, if any, do stereotype threat and acting White account for in academic performance—these models suggest that at least some of the variance in the academic performance of ethnic minorities and the concomitant underrepresentation in GATE programs is not related to assessment practices but to individual psychological factors including choice and anxiety induced both by internal (e.g., stereotype threat) and external (e.g., teacher expectations) factors.

Alternative Assessment Tools

Several researchers have suggested alternative strategies for identifying gifted students to increase the representation of minority and low-SES students in these programs. Suggested alternatives include nonverbal measures of cognitive ability, performance tasks, multiple indices, and local norms. In support of the nonverbal test option, Naglieri and Ford (2003) reported that minorities and European American samples had similar scores on the Naglieri Nonverbal Ability Test (NNAT; Naglieri, 1997), including at the upper end of the distribution which is used to identify students for GATE programs. However, given that the achievement gap is still with us (Aud et al., 2010), tests that predict academic performance should reflect this gap, and the absence of a gap raises predictive validity questions about NNAT scores. Lohman (2005c) also suggested that the NNAT normative sample was not representative of the U.S. population and even less representative of the racial/ethnic subgroups. Moreover, as verbal reasoning is an important component of school performance, tests assessing only nonverbal reasoning are not the most useful for identifying students who will perform well on verbal reasoning tasks in regular education or gifted programs (Lohman, 2005d).

Performance tasks—that is, tasks in which students are asked to solve functional problems, usually requiring multiple steps—are a useful addition to the gifted assessment repertoire. They are domain-specific and can help to identify the ability that will be enriched or accelerated in gifted programming. Moreover, they assess the fluency and complexity of responses, encourage the use of metacognition, require a dynamic assessment approach on behalf of the examiner and have been shown to be useful in identifying groups of ethnic minorities and low-income students (VanTassel-Baska, Johnson, & Avery, 2002). However, there is little evidence that these tasks add incremental validity beyond commonly used IQ tests and indicators of previous achievement (e.g., GPA).

This is not to say we advocate against using alternative measures as part of a comprehensive identification plan. Indeed, one of the most effective ways of improving ethnic minority representation in GATE programs is to reduce the number of false negative errors by casting the widest selection net possible. Hypothetically, if districts were to eliminate the use of IQ tests for identifying gifted students, domain-specific achievement tests would be the best logical alternative because they are the best predictors of future performance in their domains.

The third alternative involves combining scores from multiple tests of intelligence. Lohman (2005a, 2005b) provided a comprehensive rationale for this practice along with guidelines for combining scores from different tests. Lohman also made a compelling argument for crafting local norms for tests of intelligence and achievement. He argued that individual schools—and we would add especially schools in which gifted students are underrepresented—“rarely replicate the nation in their distribution of ability or achievement” (Lohman, 2005b, p. 13). Thus, he suggested that students’ scores should be compared “only to the scores of other students who share similar learning opportunities and background characteristics” (Lohman, 2005a, p. 349), and decisions about identification and acceleration for gifted placement be “made using local [i.e., school or district] norms” (Lohman, 2005b, p. 14). Of course, the idea of using local norms is controversial and the decision about whether to use them may more appropriately rest with policy makers than assessment researchers. The role of the assessment researcher will be to determine if using local norms consistently increases the identification of minority students who are successful in gifted programs.

Minority Students in a Gifted Program

As noted above, the points of view with regard to minority underrepresentation may best be reconciled by educational policy rather than assessment practice. Although it is important to acknowledge that there will be underrepresentation of minority students in GATE programs until the achievement gap goes away, at the same time, we must also recognize that equitable representation in these programs is a worthy goal that we need to work toward, reflected in the following policy recommendation by Lohman (2005a): “Because the discrepancy between potential and accomplishment will be greatest for those who have had the fewest opportunities, consider weighting accomplishment more heavily for advantaged students and potential for students whose educational opportunities have been more limited” (p. 354). In this section, we highlight one attempt to operationalize Lohman’s recommendation and show that neither is the process easy nor does it result in equal outcomes.

Data

Figure 1 consists of a summary of student academic and socioeconomic profiles. The top two graphs in Figure 1 are based on the California Standards Tests. The left graph has standard score means—the scores range from 150 to 600—and the graph on the right has proficiency means, which range from 1 (far below basic) to 5 (advanced). As is evident, the scores for the African American, Chicano, and Hispanic/Latino groups are lower on average than the scores for the other groups. Indeed, the Asian and European American students’ mean falls between proficient and advanced, as does the English/language arts mean for the African Americans, and the mean mathematics scores for African Americans was at the advanced level. All the other means fall between basic and proficient. Not surprisingly, Chicano and Hispanic/Latino students have substantially lower English/language arts scores than mathematics scores, and substantially lower English/language arts scores than all other groups. The other academic input—the incoming grade point average—is presented in the left column in the bottom-right graph. The underrepresented students have mean GPAs around 3.5, whereas the GPAs of the other groups are in the 3.7-to-3.8 range.

OPEN IN VIEWER

Figure 1.

California Standards Test standard scores (SS) and proficiency levels (PL), parental income and education levels, and school and summer program GPAs by racial/ethnic group

The bottom-left graph contains markers of context and opportunity. The three bars in this graph indicate family income (1 = less than US$10,000, 5 = US$40,000 to US$60,000, 8 = over US$100,000) and mother and father’s education (1 = elementary school, 5 = some college, 7 = bachelor’s degree, 9 = graduate or professional degree), respectively. As can be seen, the differences here are similar to those in national data sets (e.g., Aud et al., 2010). The Hispanic groups have family incomes of less than US$40,000 on average, with both parents having, on average, high school diplomas. African Americans and Chinese Americans form a second tier, with incomes in the US$60,000 to US$80,000 bracket and education levels around the associate’s to bachelor’s degree, although the slightly higher levels and disproportionate size of the Chinese American group (half of the sample) suggests that this group is different from the African American cohort. The other groups report average incomes of more than US$80,000, and education levels in the graduate school range for both parents.

In the bottom-right graph, we also report the summer program GPA for the groups. With the exception of the Indian Americans, there is a drop between incoming GPA and summer program GPA, and this decrease is most pronounced for the three underrepresented groups. On one hand, the drop is not surprising, given that the program is more competitive than their home schools. However, there are probably other factors at work. Given the differences in income and education levels of parents, it is probable that the underrepresented students come from poorer districts, have families that are less able to assist with homework, and may even have to work more during the summer in addition to attending summer school (cf. Aud et al., 2010).

These data show that even in gifted identification based on multiple academic indicators and without the use of strict cut scores, there are differences in the academic profiles of the students from different racial/ethnic groups that reflect the differences evident in the achievement gap. The differences in socioeconomic status (i.e., parental education and income) among the groups were even starker than the differences in academic attainment.

Conclusion

These data tell us several things. First, the achievement gap continues to play an important role in whether students qualify for GATE programs, whether one looks at standardized achievement test scores, GPAs, or variables related to socioeconomic status such as parental education and income levels. Despite making concerted efforts to recruit students from underrepresented populations, the data show that they continue to be underrepresented and that on average, their achievement is lower across multiple indicators. Moreover, if we were not using the comprehensive review in keeping with Lohman’s (2005a) suggestion, and instead using strict cut scores on the measures of potential and accomplishment regardless of context, the scores of the underrepresented group members who would be in the program might be higher, but there would be even starker disproportionality than the one that currently exists.

Although assessment instruments are convenient targets to vilify, there is no compelling evidence suggesting that they play a major role in underrepresentation in GATE programs. Thus, blaming tests for underrepresentation is neither useful nor accurate and distracts us from focusing on the underlying cause, the achievement gap. The reasons for this gap are complex and not fully understood. We do, however, concur with GATE educators and measurement experts who suggest that GATE identification should use multiple indicators of talent. In the context of gifted identification, it is far more harmful to produce false negatives than false positives. This attitude is reflected in best-practice guidelines for identification as well. In 2008, the NAGC adopted a policy asserting that (a) no single measure should be used to make identification or placement decisions, (b) testing situations should not hinder students’ performance, and (c) multiple measures and different types of indicators from multiple sources be used to assess and serve gifted students. Equity, defined as proportional representation in gifted education by racial/ethnic group, is a noble aspiration for the field. However, it will not be fully achieved until the achievement gap itself is eliminated, an outcome that will only be evident in valid assessment data.