Gifted Identification With Aurora

Abstract

The current need to think globally over the long term has necessarily altered perspectives in education. Expectations for educational outcomes have increased over the last century for all students and, in many societies, diverse educational programs are available for those who are deemed to be particularly gifted. However, many children are likely to have intellectual gifts that are not easily revealed by the narrow range of cognitive and academic assessments typically employed by schools and, as a result, their abilities and potentials may remain unrealized. Who are these children and how can we best identify them? Assessment procedures designed for the identification of gifted students have become more refined in many cases, but there is an ever-present danger of having too narrow a perspective. This study demonstrates how an assessment approach based on a theory of intelligence encompassing multiple abilities may be useful in broadening the scope of conceptions and measures of giftedness.

Keywords

assessment giftedness ability successful intelligence achievement

Within the systems of education that have evolved in developed countries during the last century, there has been an increasingly concentrated pursuit of intellectual giftedness. This is partially a result of the desire to be efficient in identifying and developing intellectual potential and partially a result of the desire to maintain a competitive position in the global hierarchy of intellectual and economic power¹ and influence (Mandelman, Tan, Aljughaiman, & Grigorenko, 2010). Thus there has been continuing inquiry into what intellectual giftedness is, as evidenced by the steady flow of theories, models, and assessments concerning intelligence and giftedness (Heller, Monks, Subotnik, & Sternberg, 2000; Sternberg, 2004; Sternberg & Davidson, 2005).

The identification of intellectual giftedness has become focused most intensively on the school setting, with the intention of recognizing potential early and nurturing it to fruition. The larger question that looms concerns the overall purpose or goals of this exercise. What began as a very direct task of finding students in need of special support in school (Binet & Simon, 1905; Jarvin & Sternberg, 2003) has blossomed into a broader multifaceted inquiry on intelligence and intellectual giftedness (Balchin, Hymer, & Matthews, 2009; Plucker & Callahan, 2008). The goal of this inquiry is to realize students’ potentials beyond school performance, throughout the entire life span. Yet the handful of longitudinal studies carried out thus far on life span outcomes of identification has left many questions unanswered as to how best to ensure the identification and realization of intellectual potential and the selection of the most powerful and effective criteria to use for the assessment of giftedness (Benbow & Lubinski, 1996; Heller, 1991, 1996; Subotnik & Arnold, 1994).

Some theories of cognitive abilities have recognized the multiplicity of related but distinct human abilities that might play an important role in achieving success. One theory that has taken such an approach, Sternberg’s theory of successful intelligence, defines intelligence as the balanced system of abilities that allows one to adapt to, shape, and select environments so as to accomplish one’s goals, within the context of one’s culture or society (Sternberg, 1999). According to this conception of intelligence, analytical, creative, and practical abilities play relatively independent and important roles in intellectual functioning and successful outcomes in life. Successfully intelligent people need creative abilities to generate new ideas and to cope with relative novelty, analytical abilities to ascertain the value of their new ideas and their coping strategies, and practical abilities to put their ideas into practice and to persuade others of the value of those ideas. Abilities are viewed as forms of developing expertise (Sternberg, 1998). That is, they are not fixed or static, but rather develop through interactions with the environment. People start off as novices in the use of abilities, and then progress through various stages as they develop expertise in their actions, based on the use of these developing abilities. Thus abilities are modifiable and dynamic. Various sources of data support the theory in numerous domains, ranging from adaptive functioning in jobs to enhancing educational-placement and college-admissions procedures (Sternberg, 2009, 2010; Sternberg, Jarvin, & Grigorenko, 2011; Sternberg & The Rainbow Project Collaborators, 2006).

In the work reported here, we were interested in how the use of a newly developed assessment tool based on Sternberg’s theory of successful intelligence, the Aurora Battery, would allow for the identification of gifted children. In particular, we investigated the extent to which the set of children identified as gifted through the theory of successful intelligence would overlap with the set of children identified through conventional theories and conventional assessments. Aurora was designed to assess a set of abilities that may lead to excellence not only within but also beyond the confines of the system of schooling—in the everyday world and in the long term, after the school years are over. Thus it has the potential to identify those students who are routinely missed when only traditional methods are used to identify gifted children (Chart, Grigorenko, & Sternberg, 2008).

Method

Participants

The participants were drawn from a large sample of British school students recruited through schools located in a town in England. This town has a largely monocultural, White population of 90,000 inhabitants, and is among the 10% of the most disadvantaged areas (out of 354) in the United Kingdom. Unemployment is above both the regional and national averages, and life expectancy is below the national average. The town’s children are served by 30 primary schools, six secondary schools, and four providers of further education. Only 6% of students in the town who leave school at age 16 directly enter paid employment. Post-age-16 education participation rates remain low compared with national figures, and many young school leavers remain largely dependent on state benefits. Levels of educational aspirations and achievement and expectations of life success are such that many children are likely to be failing to realize their true potential.

The sample from this population (n = 426 children) that we discuss here was composed of fourth (n = 52), fifth (n = 276), and sixth (n = 98) graders. The age of the participants ranged from 8.42 to 13.08 years (M = 10.27, SD = 1.19; 52.8% girls). The schools were recruited through the cooperation of the local authority. The data collection was carried out as a townwide initiative, with the approval and support of the city’s education authority, each school’s head teacher, and every participating teacher. In the introductory statements given to classes on the days of administration, students were informed that participation was voluntary, and they were given the option to engage in self-study activity rather than complete the tests.

Measures

The Aurora battery

Aurora is a set of assessments. One of these assessments, Aurora-a (for augmented), hereafter, Aurora—the battery’s paper and pencil test and our main focus in this article—was developed to assess analytical, creative, and practical abilities in a group or classroom setting. It consists of 17 subtests: six analytical, five creative, and six practical. The instrument is characterized by variation in its types of item formats (multiple choice; short answer; and open-ended items, which are scored by trained raters²). The subtests were designed to assess abilities across and between stimulus domains (six verbal-Words, five numerical-Numbers, and six figural-Images subtests) and item formats such that a balanced range of opportunities could be offered to demonstrate various abilities within and across domains (Chart et al., 2008; Tan et al., 2009). To illustrate Aurora items, consider the following examples:

From Metaphors Words-Analytical

Directions: Sometimes people compare things that seem very different. Below are sentences that compare things, but the sentences aren’t finished. Finish the sentences by explaining how the first thing is like the second thing.

Example:

Homework is like health food because

it is good for you, even though you might not like it!

In this example, homework and health food are being compared; the possible response presented in the examples is that they are both good for you even though you may not like them.

From Headlines Words-Practical

Directions: Below are newspaper headlines that have two meanings. One meaning is serious and tells what the newspaper story is really about. The other meaning is silly. The example shows the headline with both its serious and silly meanings. For each question, figure out the SILLY meaning and write it in the box in your own words.

Example:

Headline: Fish Biting off Florida Coast

Serious meaning: People are catching a lot of fish in the water near Florida.

Silly meaning: Some fish are actually biting the land in Florida!

In this example, students are asked to consider the headline, “Fish Biting off Florida Coast,” and try to see how it may be read in two ways: a way that conveys news, giving real and practical information, and a way that only says that seems highly unlikely.

For each of the 17 Aurora-a battery subtests, a total score was obtained using multifaceted Rasch modeling as implemented in FACETS (Linacre, 2009). The data presented in this article are based on analyses performed on the scores that passed psychometric quality control.³ To ensure stable ability-parameter estimation, the scores for the reported sample were calculated with reference to the larger sample (n = 3,501) of English and American schoolchildren.⁴ The resulting logit ability estimates with age regressed out were standardized within the sample for each subtest. The three ability scores were derived by averaging scores on the subtests within the analytical, creative, and practical ability tests. The three domain scores were calculated as averages of the subtests within the Words, Numbers, and Images domains. The scores were then rescaled to have a sample mean of 100 and a standard deviation of 15.

Key Stage 1 and 2 tests (henceforth, KS1 and KS2)

The Key Stage 1 and Stage 2 tests are part of the English system of academic tracking; at the end of particular educational “stages,” all students are assessed (using various combinations of paper and pencil tests and teacher assessments) to indicate whether or not they are working at the expected age-based level for each of a number of academic subjects. The first “key stage” encompasses Years 1 and 2 (Y1 and Y2; equivalent to U.S. Grades K-1). At the end of Y2, students take their first set of key stage assessments, KS1. The KS1 assessments consist of paper and pencil tests of Reading, Writing, and Mathematics, and teacher assessments of Speaking and Listening and Science. The second key stage covers Years 3, 4, 5, and 6 (U.S. Grades 2-5); thus at the end of Y6, they take the KS2 tests. KS2 contains tests of Reading, Writing, English, Mathematics and Science (DirectGov, 2009). Each year the tests are rewritten under the auspices of the British Government; the content of the tests is changed annually and we are unaware of any publicly available reports on their psychometric properties.

The English National Curriculum is expressed in 10 levels for each subject. KS1 scores for each subject are classified as below expectation (Level 1), expected (2), beyond expectation (3), and exceptional (4). Cutoff scores for each Level change from year to year. KS2 scores for each subject are classified as below expectation (2, 3), expected (4), beyond expectation (5) and exceptional (6). As the levels are essentially criterion-referenced measures, a given standard of performance should be scored at the same level irrespective of the child’s age or phase of schooling. To achieve comparability, the scores were standardized within KS1 and KS2. The KS Total score was obtained by averaging the standardized scores on the reading, writing, mathematics, and science measures.

The MidYIS

The MidYIS (Middle Years Information System; Center for Evaluation and Monitoring, 2010), designed to be taken by students on entry to secondary school (during Term 1 of Year 7 or right at the end of Year 6), is best described as a baseline assessment of developed ability and aptitude for learning. The four sections of the MidYIS are vocabulary, mathematics, skills (proofreading and perceptual speed and accuracy), and nonverbal tasks (primarily visual spatial reasoning and logical thinking). The questions are a mixture of multiple choice and free response and are scored by computer (N. Forster, personal communication, January 11, 2011; A. Shields, personal communication, March 24-25, 2009). The MidYIS was developed by, and is administered through, the Center for Evaluation and Monitoring (CEM) at Durham University. According to the CEM, vocabulary and mathematics scores in particular are predictive of student achievement (as reflected in students’ scores on the standardized national achievement—Key Stage—tests).

The MidYIS Overall score, which we used for comparison to Aurora’s scores, is calculated by adding the weighted raw scores for the vocabulary, mathematics, nonverbal, and skills sections. This figure is then standardized to have a national mean of 100 and a standard deviation of 15 (Centre for Evaluation and Monitoring, 2010).

Both the KS and MidYIS scores were obtained through the local authority, with the agreement of the schools involved.

Procedures

Aurora administration

Aurora-a was administered to all classes in multiple sessions, one session per day. Data from Year 7 (U.S. Grade 6) children were collected in three sessions of 1 hr each. The three sessions took place generally within 2 weeks, sometimes occurring on consecutive days, in other cases with days in-between, according to each school’s convenience. The battery was split into 3 packets—A, B, and C. The order of the subtests in the packets was counterbalanced across two versions, 1 and 2. Data from Years 6 and 5 (U.S. Grades 5 and 4) were collected in six sessions of 45 min each. These sessions took place over the course of 3 days. Two consecutive 45-min sessions occurred in one day, with a 15-min break between sessions. The test battery was thus split into 6 packets (A, B, C, D, E, and F). The order of the test packets in all data collections was counterbalanced across schools.

Identifying Giftedness With Aurora

To identify gifted students using Aurora-a, we chose a 90th-percentile cutoff criterion for each of the ability and domain total scores. This procedure reflected Government guidance in England that schools should identify approximately 10% of students as gifted and talented (Tan et al., 2009). Thus for the purposes of our investigation, a child was considered analytically, creatively, or practically gifted if she was in the top 10% in analytical, creative, or practical ability, respectively. The same procedure was used for the domain scores. Overall giftedness status was determined by taking into account all three Aurora ability scores using two different approaches. Combined giftedness status was achieved if a child performed in the top 10% on at least one of the analytical, creative, or practical ability scores (i.e., was considered gifted, as stipulated above). In addition to that, an average ability index was calculated by averaging the three scores (with the same 90th-percentile cutoff criterion used).

Identifying Giftedness With KS Tests and the MidYIS

KS and MidYIS scores were used to identify gifted students as follows: A child was identified as gifted if she obtained a “beyond expectation” score on each of the available KS measures, namely, Reading, Writing, Mathematics, and Science (roughly 9% of children were selected with this method). We also used a 90th-percentile criterion for giftedness identification using the MidYIS Total score.

Results

Aurora, KS, and the MidYIS: Descriptive Statistics, Intercorrelations, and Regression Analyses

To explore the relationship between Aurora ability and content-domain estimates, and KS and MidYIS achievement scores, we conducted a correlational analysis (see Table 1). Aurora ability and domain scores were positively correlated with each other; yet, these correlations (ranging from .50 to .59 for ability estimates, and from .42 to .56 for domain estimates) were not perfect, suggesting that the battery is successfully measuring performance on positively correlated but relatively independent abilities, and within relatively independent domains.

Table 1.

Descriptive Statistics and Intercorrelations Between Study Measures

	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. Aurora analytical	100.00	15.00
2. Aurora creative	100.00	15.00	.51
3. Aurora practical	100.00	15.00	.59	.50
4. Aurora words	100.00	15.00	.68	.69	.74
5. Aurora numbers	100.00	15.00	.63	.74	.64	.56
6. Aurora images	100.00	15.00	.74	.54	.65	.48	.42
7. KS reading	0.00	1.00	.49	.53	.49	.64	.46	.36
8. KS writing	0.00	1.00	.39	.45	.39	.56	.38	.25	.76
9. KS math	0.00	1.00	.44	.35	.52	.43	.48	.36	.59	.53
10. KS science	0.00	1.00	.38	.35	.39	.48	.30	.31	.62	.55	.56
11. KS total	0.00	.84	.51	.50	.54	.63	.48	.38	.89	.85	.80	.82
12. MidYIS vocabulary	104.22	12.61	.50	.53	.58	.67	.50	.40	.63	.43	.41	.50	.58
13. MidYIS math	107.79	14.20	.54	.46	.62	.53	.58	.47	.51	.44	.58	.40	.57	.53
14. MidYIS nonverbal	101.97	13.26	.48	.37	.55	.43	.41	.54	.41	.37	.43	.35	.46	.48	.42
15. MidYIS skills	105.97	13.35	.33	.49	.38	.49	.39	.29	.53	.54	.34	.40	.54	.42	.42	.32
16. MidYIS total	106.35	12.79	.61	.57	.69	.69	.62	.50	.65	.50	.56	.51	.66	.88	.87	.60	.48

Note. N = 426 for intercorrelations between Aurora scales, n = 359 for the intercorrelations between Aurora and KS, n = 209 for the intercorrelations between Aurora and the MidYIS, and n = 208 for the intercorrelations between KS and MidYIS. The KS scores were standardized within KS1 and KS2 collections for the purpose of correlational analysis reported in this table. The KS Total score represents the average of the standardized KS subscales scores. All reported coefficients are significant at p < .01.

The second notable feature of the correlations is that all of the Aurora ability estimates were positively and significantly related to the KS (ranging from .35 to .54, median = .45) and MidYIS scores (ranging from .33 to .69, median = .53). Furthermore, in the context of hierarchical regression analyses (see Table 2), with demographic characteristics entered in the first and the Aurora scores—in the second blocks, Aurora abilities predicted 20% to 56% of the variance in conventional achievement scores.

Table 2.

Hierarchical Regression Analyses Predicting Conventional Achievement Scores Based on Aurora Ability and Domain Scores

	Demographic indicators^a				Aurora abilities					Aurora domains
	Age	G			A	C	P			W	N	I
	β(p)^b	β(p)	F(p)^c	R ²	β(p)	β(p)	β(p)	F(p)	R ²	β(p)	β(p)	β(p)	F(p)	R ²
KS reading	−.02 (ns)	−.07 (ns)	3.31 (.04)	.01	.21 (.00)	.31 (.00)	.22 (.00)	43.82 (.00)	.37	.52 (.00)	.15 (.00)	.06 (ns)	53.07 (.00)	.42
KS writing	−.04 (ns)	−.12 (.01)	6.09 (.00)	.03	.16 (.01)	.26 (.00)	.17 (.00)	26.19 (.00)	.26	.49 (.00)	.12 (.03)	−.02 (ns)	35.04 (.00)	.32
KS math	−.02 (ns)	.08 (ns)	.57 (ns)	.00	.19 (.00)	.09 (ns)	.37 (.00)	31.65 (.00)	.30	.21 (.00)	.31 (.00)	.15 (.00)	29.73 (.00)	.29
KS science	−.04 (ns)	.04 (ns)	.64 (ns)	.00	.18 (.00)	.16 (.00)	.21 (.00)	19.07 (.00)	.20	.43 (.00)	.01 (ns)	.11 (.03)	23.51 (.00)	.24
KS total	−.04 (ns)	.02 (ns)	1.62 (ns)	.00	.22 (.00)	.25 (.00)	.29 (.00)	46.26 (.00)	.39	.49 (.00)	.17 (.00)	.09 (.05)	53.85 (.00)	.42
MidYIS vocabulary	−.00 (ns)	.06 (ns)	.23 (ns)	.00	.16 (.03)	.29 (.00)	.33 (.00)	30.81 (.00)	.41	.58 (.00)	.13 (.05)	.08 (ns)	39.17 (.00)	.47
MidYIS math	.10 (.05)	.04 (ns)	2.34 (ns)	.01	.22 (.00)	.14 (.03)	.41 (.00)	34.27 (.00)	.44	.26 (.00)	.33 (.00)	.19 (.01)	32.27 (.00)	.42
MidYIS nonverbal	.05 (ns)	.01 (ns)	.72 (ns)	.00	.21 (.01)	.07 (ns)	.38 (.00)	20.75 (.00)	.32	.18 (.01)	.11 (ns)	.40 (.00)	21.21 (.00)	.33
MidYIS skills	.08 (ns)	−.13 (.03)	4.64 (.01)	.03	.02 (ns)	.36 (.00)	.19 (.02)	16.17 (.00)	.27	.36 (.00)	.15 (.05)	.05 (ns)	15.64 (.00)	.26
MidYIS total	.04 (ns)	.05 (ns)	1.02 (ns)	.00	.22 (.00)	.24 (.00)	.43 (.00)	55.76 (.00)	.56	.48 (.00)	.27 (.00)	.16 (.00)	58.56 (.00)	.57

Note. G = Gender (0 = Girls, 1 = Boys); A = Analytical, C = Creative, P = Practical; W = Words, N = Numbers, I = Images.

Two sets of regression analyses with KS and MidYIS scores were carried out, one—with all Aurora abilities, and the other—with all Aurora domain scores. Each of the regressions had two blocks—(1) demographic indicators and (2) Aurora scores (either abilities or domains). For demographics, only results for regression analyses with Aurora abilities are shown; the results for analyses with Aurora domains are virtually indistinguishable, with negligible fluctuations in values.

Coefficients are shown for full regressions.

F and R² (adjusted) are shown for each regression step. N = 359-360 for the regression analyses for Aurora and KS, n = 209-215 for the regression analyses for Aurora and the MidYIS.

For the Aurora domain scores, the correlations were similar (.25-.64, median = .43 for KS and .29-.69, median = .50 for MiYIS). Likewise, hierarchical regression analyses (with demographics entered first, and Aurora scores—second) indicated that Aurora domains predicted 24% to 57% of the variance in KS and MidYIS scores. Together, these findings provided preliminary evidence that all of the targeted abilities and domains are related to academic success, following the predictions of the theory (Sternberg, 1999) and the design of the Aurora battery (Chart et al., 2008; Tan et al., 2009).

Sensitivity and Specificity of Aurora Ability and Domain Estimates

We investigated the extent to which Aurora converges in identifying gifted children previously identified with the KS or the MidYIS measures through a set of contingency analyses, which allowed us to build indices of Sensitivity (i.e., percentage identified as gifted by Aurora and the MidYIS/KS) and Specificity (i.e., percentage identified as nongifted by the respective measures). Table 3 displays a summary of these analyses.⁵

Table 3.

A Summary of Sensitivity and Specificity of Aurora Ability and Domain Scores

		KS				The MidYIS
		Sensitivity	Specificity	χ²(1)	p	Sensitivity	Specificity	χ²(1)	p
Aurora
Ability	Analytical	12%	91%	.33	> .05	14%	93%	1.58	< .05
	Creative	20%	94%	11.12	< .001	19%	92%	3.01	> .05
	Practical	18%	93%	5.91	< .05	33%	93%	15.93	< .001
	Average^a	18%	93%	5.91	< .05	24%	91%	4.67	< .05
	Combined^b	40%	81%	10.96	< .001	52%	84%	15.25	< .001
Domain	Words	22%	93%	5.91	< .05	48%	94%	37.83	< .001
	Numbers	18%	92%	4.93	< .05	24%	92%	5.81	< .05
	Images	12%	90%	.25	> .05	5%	89%	.76	> .05

Note. N = 359 for KS, n = 215 for the MidYIS. The index of Sensitivity was calculated as % identified as gifted by both measures, the index of Specificity was % identified as nongifted by both measures.

The child was assigned gifted status on the Average index if he or she performed in the top 10% on the average of the Analytical, Creative, and Practical ability scores.

The child was assigned gifted status on the Combined index if he or she performed in the top 10% on at least one of the Analytical, Creative, or Practical abilities.

With respect to the KS, separate Aurora ability estimates had an average sensitivity of 17%, convergently identifying about one fifth of those determined to be gifted with the KS measures. The creative ability index had the highest sensitivity (20%). When the three ability measures were combined, Aurora convergently identified 40% of the KS-gifted children, χ²(1) = 10.96, p < .001. The average ability score on Aurora, however, identified only 18% of those identified as gifted with the KS, χ²(1) = 5.91, p < .05.

Sensitivity was higher for the MidYIS, with an average index of 22% for the separate ability scores, and a combined sensitivity of 52%, χ²(1) = 15.25, p < .001; the largest contribution was from practical ability (33%). Of note also is that practical giftedness was the only ability that was consistently significantly associated with giftedness as identified with the KS, χ²(1) = 5.91, p < .05, and the MidYIS, χ²(1) = 15.93, p < .001. The average ability index had a sensitivity of 24%, χ²(1) = 4.67, p < .05.

With respect to domains, verbal giftedness as identified with the Aurora Words scale had the highest sensitivity to both the KS, 22%, χ²(1) = 5.91, p < .05, and the MidYIS, 48%, χ²(1) = 37.83, p < .001. In sum, these results suggest that Aurora identifies a significant proportion, but far from all of those students that may be identified as gifted with achievement or ability measures such as the KS tests and MidYIS. Such an outcome is certainly desired as Aurora seeks to identify gifts in domains that are not directly tapped by the other two measures.

Our analysis of the specificity of Aurora ability and domain estimates, when compared with both the KS and the MidYIS, indicated an average specificity of 92.67% and 91.67% for ability and domain estimates, respectively. When combined, Aurora ability scores convergently identified as nongifted 81% and 84% of those identified as nongifted with the KS and MidYIS, respectively (the specificity was 93% and 91% for the average Aurora ability estimates with respect to the KS and MidYIS). Taken together with the results of the sensitivity analysis reported above, our findings suggest that although Aurora convergently identifies a substantial proportion of both those identified as gifted and nongifted with other measures, the overlap is not perfect; that is, Aurora also identifies an additional set of analytically, creatively, and practically gifted children, as well as children gifted in the verbal, numerical, and figural domains.⁶

Supporting these conclusions are the results of the set of binary logistic regression models aimed at predicting the KS/MidYIS giftedness identification status from Aurora ability and domain scores. A total of four regression models were fit (two per KS/MidYIS total scores each) with gender and age entered in the first step, and Aurora ability or domain estimates in the second step. With regard to KS gifted status, demographic characteristics entered in the first step did not predict giftedness status significantly better than the null model, χ²(2) = 1.55, p > .05; Nagelkerge R² = .01; gender B = .24, p > .05; age B = –.15, p > .05. Adding Aurora ability estimates improved the fit of the model, χ²(3) = 36.70, p < .001; Nagelkerge R² = .18, and resulted in significant B coefficients for creative and practical ability estimates (B = .05, p <.01, and B = .04, p < .01, respectively); surprisingly, analytical ability was not a significant predictor of KS giftedness identification status (B = –.007, p > .05). However, a comparison of observed versus predicted values indicated that the model had a 99.7% specificity but only 6% sensitivity. The fitting of the domain model showed similar results, χ²(3) = 35.66, p < .001; Nagelkerge R² = .18, with verbal and numerical domain scores significant predictors of giftedness status (B = .05 and .03, p < .01 and < .05, respectively); there were nonsignificant contribution from the figural domain (B = .002, p > .05). The model, again, had a specificity of 99.7% and a low sensitivity of 2%.

The models fitted to the MidYIS data showed a similar pattern of results. The demographics did not predict gifted status significantly better than the null model, χ²(2) = .77, p > .05; Nagelkerge R² = .01; gender B = .27, p > .05; age B = .60, p > .05. Adding the Aurora ability scores improved the fit of the model, χ²(3) = 24.90, p < .001; Nagelkerge R² = .24 but resulted in significant coefficients only for practical ability (B = .06, p < .05), whereas analytical and creative ability did not contribute to the prediction significantly (B = .02, B = .03, p > .05, respectively). The model had a 99% specificity and a sensitivity of 4.8%. The fitting of the domain model, again, showed similar results, χ²(3) = 36.92, p < .001; Nagelkerge R² = .34, with verbal domain score a significant predictor of giftedness status (B = .14, p <. 001), and no contribution from numerical (B = .01, p > .05) or figural domain scores (B = –.003, p > .05). The model had a specificity of 99% and a sensitivity of 14.3%. Thus the results of the binary logistic regression analysis showed that Aurora ability and domain scores significantly yet dissimilarly predicted the giftedness identification status obtained using the KS and MidYIS measures.

What follows is a closer look at Aurora’s gifted in terms of ability and domain profiles. That is, does the set of children identified as gifted by Aurora reflect particular profiles of analytical, practical, and creativity abilities, or domain-specific profiles?

Ability Profiles of Children Identified as Gifted With Aurora

To characterize the ability profiles of children identified as gifted with the Aurora Battery, we analyzed the data using a Q-factor analysis technique, which has been previously suggested as a powerful descriptive statistical technique for the in-depth qualitative examination of gifted children (Thompson, 2010). The use of Q-factor analysis allowed us to investigate empirically distinguishable ability profiles in the sample. Separate analyses were carried out on overlapping six subsamples of children identified as gifted in analytical, creative, and practical ability, and in the words, numbers, and images domains. Principal-components analyses extraction with promax rotation were used to analyze a transposed matrix of the ability and domain scores. Pattern matrices were used to determine the profile most characteristic of each child on the basis of the absolute values and their signs. Because loadings could be positive or negative in sign, each factor represented two potential profiles – the main one, and the opposite profile.

The results of the six Q-factor analyses that were performed are summarized in Table 4. For each ability and content domain, two factors were extracted with eigenvalues larger than 1.00, which explained 100% of the variance in the scores. First we examined the profiles of children identified as gifted in one of the three main abilities that Aurora is designed to assess (analytical, creative, and practical). The exemplar profiles are plotted on Figures 1 and 2. These profiles illustrate the diversity of the patterns of strengths and weaknesses in the various domains and abilities found in gifted children identified with Aurora.

Table 4.

A Summary of Aurora Q-factor Analyses

Identified as gifted with Aurora in		Factor	% variance explained	Profile	n holding the profile	% out of 42
Ability
	Analytical	1	73.41%	1	28	67%
		2	26.59%	2	14	33%
	Creative	1	63.57%	3	24	57%
		1		4	1	3%
		2	36.43%	5	9	21%
		2		6	8	19%
	Practical	1	62.21%	7	16	38%
		1		8	5	12%
		2	37.79%	9	20	48%
		2		10	1	2%
Domain
	Words	1	58.67%	11	24	57%
		1		12	1	3%
		2	41.33%	13	8	19%
		2		14	9	21%
	Numbers	1	68.80%	15	28	67%
		2	31.20%	16	6	14%
		2		17	8	19%
	Images	1	68.93%	18	24	57%
		2	31.07%	19	16	38%
		2		20	2	5%

Note. N = 42 for each ability and domain subgroups. There was only one profile per the first and the second factor for those identified as gifted in Analytical ability, and one profile for the first factor for those identified in the Numbers and Images domains.

Figure 1.

Examples of Aurora ability profiles of children identified as gifted in each of the abilities measures by Aurora

Figure 2.

Examples of Aurora ability profiles of children identified as gifted in each of the domains measures by Aurora

Ability Profiles of Children Identified as Gifted With KS and MidYIS Tests

To investigate the ability profiles of children identified as gifted with the KS tests and MidYIS, two additional Q-factor analyses following the same procedure were performed on the subsamples of children identified as gifted with the KS (n = 50) and the MidYIS (n = 21).

With regard to the ability profiles, the analysis of the KS data revealed two main factors, accounting for 55.51% and 44.49% of variance, respectively, indicating the existence of a total of four ability profiles (Figure 3).

Figure 3.

Examples of Aurora ability profiles of children identified as gifted with KS (top row) and MidYIS (bottom row)

With regard to the domain profiles, the Q-factor analysis of the domain scores of children identified as gifted with KS has revealed two profile factors, accounting for 57.05% and 42.95% of variance, respectively (see Figure 4). Overall, this set of analyses demonstrated a substantial overlap in the domain performance profiles of children identified with the two conventional assessment methods. We must note that all of these profiles were derived from the analysis of the Aurora domain scores, which also identified some profiles “missed” by the KS and the MidYIS, that is, profiles (11) and (13), both with strong verbal components.

Figure 4.

Examples of Aurora domain profiles of children identified as gifted with KS (top row) and MidYIS (bottom row)

Together, the results of this set of Q-factor analyses revealed striking similarities in the ability and domain profiles of children identified as gifted with the KS and MidYIS. Half of the ability profiles mostly contrast creative ability with practical and analytical ability and have been evident in previous analyses of profiles of children identified as gifted with Aurora. The other half of the ability profiles were relatively flat, identifying children with no apparent weaknesses and strengths in the patterns of their ability scores. The domain profiles either emphasized relative similarities in verbal, numerical, and figural domains, or contrasted numerical with verbal and figural scores. The ability and domain profiles for children identified as gifted with Aurora exhibited, in general, greater between-ability and between-domain variance. This heterogeneity illustrates that Aurora is able to identify children with ability and domain profiles “missed” by the KS and MidYIS, such as children with high practical and creative abilities, and lower analytical ability, or children exhibiting a clear hierarchy of abilities (i.e., high analytical, lower creative, and even lower practical ability) as well as children with apparent strengths in the verbal domain.

Discussion

We have presented here the first application of a newly developed theory-based assessment for identifying gifted children. Based on Sternberg’s theory of successful intelligence, the Aurora Battery is designed to measure three correlated, but distinct abilities, analytical, creative, and practical, across verbal, numerical, and visual-spatial domains. Our investigation was based on the analyses of the relationships between scores obtained with Aurora and other assessments commonly used to identify gifted children in school settings in the United Kingdom. The analyses revealed that Aurora ability and domain scores were all substantially and positively related to children’s achievement scores on the KS and MidYIS assessments, as shown by both correlation and regression analyses. Yet our findings also demonstrated that when high performers are considered, Aurora and conventional achievement-oriented assessments tended to spotlight rather different children, with an average overlap of only ~10% to 20%. In other words, designation as gifted appears to depend on the instruments and the identification criteria used.

The larger question that this battery attempts to address pertains to the need to develop identification instruments that, at least partially, map onto the demands of the real world for innovation and the delivery of practical solutions to real problems. Although any system can always use a great analytical thinker, it appears that what the world really needs now are human talents that are empowered not only to analyze and remember but also to apply and create. Here we have shown that we can identify children in middle school whose profiles appear to be suitable, at least cognitively, to become innovators and practical problem solvers, and these students are typically not identified by more traditional achievement-oriented assessments. It seems to us that in today’s world of knowledge economies, the depleted ozone layer, nuclear energy crisis, and ongoing ethnic and religious conflicts, recognizing the potential of children who may be capable of addressing such questions is important. The Aurora assessment proved to be time-consuming both in its administration and in its scoring. We would argue that such investment is necessary if meaningful judgments are to be made about giftedness in relation to a wider repertoire of abilities than is normally the case in schools. Of key importance, however, is the use to which information from measures such as Aurora will be put. There is little point in undertaking highly elaborate assessments if subsequent findings are not employed in a meaningful fashion that makes a real difference to the educational experiences of the students concerned. Further research will examine findings from the subtests, both together and in isolation, to see how test data may be employed by teachers not merely for the purposes of identification but also for planning appropriate curricula and optimal learning opportunities.

The full Aurora Battery, which includes a teacher rating scale, parent interview, and self-assessment scale, has been designed to identify those children who have strengths in original and flexible thinking as well as those whose knowledge and abilities may be exerted most adaptively in everyday life situations. Therefore, the predictive quality of Aurora’s indicators has a long-term aspect that is potentially unique, though, as yet, unknown. Future longitudinal studies on gifted students, using such measures, should consider also following a set of students who have not been identified as gifted, assessing both short-term academic outcomes as well as long-term future contributions to society, such as eminence (Subotnik & Rickoff, 2010). However, prediction of those gifted across analytical, creative, and practical domains is likely to be greater where teachers adopt approaches in the classroom that reflect the theory of successful intelligence (Sternberg, 2010). For this reason, controlled comparisons are less likely to be illuminating where the educational diet is restricted to narrow academic fare with a strong emphasis on rote learning. Despite the best intentions of Aurora, children whose nascent abilities and future potentials are not developed in school using curricula that foster creative and practical skills may never fully demonstrate their true capabilities. Such an outcome may be unfortunately common for children from economically disadvantaged communities, whose talents and abilities are more likely to be masked by low expectations, diminished aspirations, and family histories of educational underachievement. Thus it is important not only to identify children based on their multifarious abilities but also necessary to teach abilities, both diversifying and enhancing human potential across multiple abilities and within multiple domains, in anticipation of the challenges of future labor markets that, even now, far more than analytical, are capitalizing on creative and practical intellectual gifts.

Footnotes

Acknowledgements

We wish to thank Karen Jensen Neff and Charles Neff, who supported the Aurora Project. Without their encouragement and help, this work would have never happened.

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The authors received no financial support for the research, authorship, and/or publication of this article.

Notes

References

American Competitiveness Initiative. (2006). American competitiveness initiative: Leading the world in innovation. Washington, DC: Domestic Policy Council Office of Science and Technology.

Balchin

Hymer

Matthews

D. J.

(Eds.). (2009). The Routledge international companion to gifted education. Abingdon, UK: Routledge.

Benbow

C. P.

Lubinski

(Eds.). (1996). Intellectual talent. Baltimore, MD: Johns Hopkins University Press.

Binet

Simon

(1905). Méthodes nouvelles pour le diagnostic du niveau intellectuel des anormaux [New methods for the diagnosis of the intellectual level of subnormals]. L’Année Psychologique, 11, 191-244.

Centre for Evaluation and Monitoring. (2010). MidYIS assessments. Retrieved from http://www.midyisproject.org

Chart

Grigorenko

E. L.

Sternberg

R. J.

(2008). Identification: The Aurora Battery. In Plucker

J. A.

Callahan

C. M.

(Eds.), Critical issues and practices in gifted education (pp. 345-365). Waco, TX: Prufrock Press.

De Jong

Linacre

(1993). Rasch estimation methods, statistical independence, and global fit. Retrieved from http://www.rasch.org/rmt/rmt72n.htm

DirectGov. (2009). National curriculum teacher assessments and key stage tests. Retrieved from http://www.direct.gov.uk/en/Parents/Schoolslearninganddevelopment/ExamsTestsAndTheCurriculum/DG_10013041

Heller

K. A.

(1991). The nature and development of giftedness: A longitudinal study. European Journal for High Ability, 2, 174-188.

10.

Heller

K. A.

(1996). The nature and development of giftedness: A longitudinal study. In Cropley

A. J.

Dehn

(Eds.), Fostering the growth of high ability: European perspectives (pp. 41-56). Norwood, NJ: Ablex.

11.

Heller

K. A.

Monks

F. J.

Subotnik

Sternberg

R. J.

(Eds.). (2000). The international handbook of giftedness and talent. New York, NY: Elsevier.

12.

Jarvin

Sternberg

R. J.

(2003). Alfred Binet’s contributions to educational psychology. In Zimmerman

B. J.

Schunk

D. H.

(Eds.), Educational psychology: A century of contributions. Mahwah, NJ: Lawrence Erlbaum.

13.

Linacre

J. M.

(2009). Facets: Rasch measurement computer program (version 3.65.0). Chicago, IL: Winsteps.

14.

Mandelman

S. D.

Tan

Aljughaiman

A. M.

Grigorenko

E. L.

(2010). Intellectual giftedness: Economic, political, cultural, and psychological considerations. Learning and Individual Differences, 20, 287-297.

15.

Plucker

J. A.

Callahan

C. M.

(Eds.). (2008). Critical issues and practices in gifted education. Waco, TX: Prufrock Press.

16.

Sternberg

R. J.

(1998). Abilities are forms of developing expertise. Educational Researcher, 27(3), 11-20.

17.

Sternberg

R. J.

(1999). The theory of successful intelligence. Review of General Psychology, 3, 292-316.

18.

Sternberg

R. J.

(2009). The Rainbow and Kaleidoscope Projects: A new psychological approach to undergraduate admissions. European Psychologist, 14, 279-287.

19.

Sternberg

R. J.

(2010). Assessment of gifted students for identification purposes: New techniques for a new millennium. Learning and Individual Differences, 20, 327-336.

20.

Sternberg

R. J.

(Ed.). (2004). Handbook of intelligence. Cambridge, UK: Cambridge University Press.

21.

Sternberg

R. J.

Davidson

J. E.

(Eds.). (2005). Conceptions of Giftedness. New York, NY: Cambridge University Press.

22.

Sternberg

R. J.

Jarvin

Grigorenko

E. L.

(2011). Explorations of giftedness. New York, NY: Cambridge University Press.

23.

Sternberg

R. J.

, & The Rainbow Project Collaborators. (2006). The Rainbow Project: Enhancing the SAT through assessments of analytical, practical, and creative skills. Intelligence, 34, 321-350.

24.

Subotnik

Arnold

K. D.

(1994). Beyond Terman: Contemporary longitudinal studies of giftedness and talent. Norwood, NJ: Ablex.

25.

Subotnik

Rickoff

(2010). Should eminence based on outstanding innovation be the goal of gifted education and talent development? Implications for policy and research. Learning and Individual Differences, 20, 358-364.

26.

Tan

Aljughaiman

A. M.

Elliott

J. G.

Kornilov

S. A.

Ferrando-Prieto

Bolden

D. S.

. . . Grigorenko

E. L.

(2009). Considering language, culture, and cognitive abilities: The international translation and adaptation of the Aurora Assessment Battery. In Grigorenko

E. L.

(Ed.), Multicutural psychoeducational assessment (pp. 443-468). New York, NY: Springer.

27.

Thompson

(2010). Q-technique factor analysis as a vehicle to intensively study especially interesting people. In Thompson

Subotnik

(Eds.), Methodologies for conducting research on giftedness (pp. 33-52). Washington, DC: APA.