Abstract
This longitudinal study examined the outcomes of Project Excite on reducing minority students’ achievement gaps in STEM over 14 years. Project Excite was designed to provide intensive supplemental enrichment and accelerated programming for high-potential, underrepresented minority students from third through eighth grades to better prepare them for advanced math and science courses in high school. This study compared the performance of Project Excite participants with that of students from their local school districts and the state on the Illinois Standards Achievement Test, the Explore test, the Measures of Academic Progress, and on rates of placement in above-grade-level math courses in ninth grade. Project Excite participants consistently outperformed their Black, Latino, and low-income peers, and they came close to the performance levels of White, Asian, and non-low-income students. They were more likely to be placed in above-grade-level math courses than their minority peers in ninth grade.
The achievement gaps between minority students, particularly African American or Latino students, and their nonminority peers, and between low- and higher-income students continue to hold strong, despite concerted efforts to narrow them by national and local policy makers, district and school administrators, and a wide range of leaders. The most recent data released by the National Assessment of Educational Progress (NAEP), which measure the performance of students across the country at Grades 4, 8, and 12 on mathematics and reading achievement tests, reveal that the Black–White, Hispanic–White, and free and reduced lunch–nonfree and reduced lunch achievement gaps in mathematics or reading at Grades 4 and 8 remained relatively stable from 2013 to 2015 (NAEP, 2015).
Longitudinal data over the past 20 years (Rampey, Dion, & Donahue, 2009) further suggest that while average scores on both mathematics and reading assessments have generally improved since 1990 for all groups, the gaps between the average scores of Black, Hispanic, and low-income students and those of White students and higher income students have remained relatively static. For example, though the Black–White and Hispanic–White achievement gaps in both math and reading at the fourth- and eighth-grade levels have narrowed since the first assessment year for NAEP (1990), the changes over recent years have been minimal (Hemphill & Vanneman, 2011; NAEP, 2015).
Achievement Gaps and Socioeconomic Status (SES)
Research suggests that the minority achievement gaps exist across all socioeconomic levels, and may even be larger in higher socioeconomic levels. “ . . . African Americans, Latinos, Native Americans, and English Language Learners are underrepresented among the top 1%, 5% and 10% of students at every level of the education system from kindergarten through graduate and professional school” (Olszewski-Kubilius & Clarenbach, 2012, p. 6). While overall, the average mathematics and reading NAEP scores were higher for students whose family income was above the threshold to qualify for the National School Lunch Program (NSLP) than for those whose family income was below that threshold, the Black–White and Hispanic–White achievement gaps were also greater among students who did not qualify for the NSLP (National Center for Education Statistics, 2013). These larger Black–White and Hispanic–White achievement score gaps among students who did not qualify for NSLP compared with those who did were seen in mathematics and reading at both fourth- and eighth-grade levels (National Center for Education Statistics, 2013). Moreover, the Black–White and Hispanic–White math achievement score gaps among higher income, non-NSLP-eligible students increased from 2003 to 2009, while it decreased for lower income, NSLP-eligible students during this same time period.
The Achievement Gaps Through the School Years
There is, moreover, a considerable amount of research that indicates achievement gaps begin early and widen through the school years (e.g., Fryer & Levitt, 2004; Lockwood, 2007; Rathbun, West, & Germino-Hausken, 2004; Reardon & Galindo, 2009; Thompson & O’Quinn, 2001). Children whose mothers had higher levels of education and had not received welfare assistance had higher levels of math knowledge (numbers and shape, relative size, ordinal sequence, addition and subtraction) and reading skills (letter and sound recognition) on entry into kindergarten (West, Denton, & Germino-Husken, 2000). Similarly, large gaps exist in science knowledge at kindergarten between lower and higher SES students, largely as a result of differential, informal exposure to the natural world, and these gaps persist through elementary school (Morgan, Farkas, Hillemeier, & Maczuga, 2016). Furthermore, national data indicate the Black–White and Hispanic–White gaps in mathematics achievement test scores, in general, tend to increase through the eighth grade (West et al., 2000) and are significantly larger in math than in reading/prereading proficiency (Fryer & Levitt, 2004; Rathbun et al., 2004). Wyner, Bridgeland, and Diiulio (2009) reported that over a million children who qualify for free and reduced lunch start school with achievement levels in the top quartile. However, a greater percentage of these children will fall out of this high-achieving group by Grade 5 compared with their more advantaged peers.
The Achievement Gaps at the Highest Levels of Achievement
Plucker and colleagues brought attention to an often overlooked aspect of the achievement gap—excellence gaps or disparities among students reaching the highest levels of achievement (Plucker, Burroughs, & Song, 2010; Plucker, Hardesty, & Burroughs, 2013). They found that children who are eligible for free and reduced lunch, who are second language learners, or who are Black, Hispanic, or Native American are much less likely to score at high levels of achievement when measured via NAEP or state-level assessments. While NAEP data indicate that greater percentages of all racial groups completed more challenging curriculum levels in 2009 than in 2005, only White and Asian groups exhibited significant increases in the percentage of students who completed a rigorous curriculum by the time they graduated from high school. 1 The percentage of White students who completed a rigorous curriculum increased from 11% to 14% during this time, while the rate of Black and Hispanic students who completed a similar curriculum remained stagnant (at 6% for Blacks and 8% for Hispanics). Unsurprisingly, taking high-level mathematics courses was correlated with higher scores on the NAEP mathematics assessment, a greater number of Advanced Placement math courses taken, and higher SAT scores.
Loveless (2014) suggested that the often criticized practice of using tracked classes in math may actually be a route to closing achievement gaps. He reported that low-income, Black and Hispanic middle school students are less likely to have “tracked” advanced math classes in their schools than students who are socioeconomically better off, White or Asian. Additionally, 25% of lower income, Black and Hispanic students were in a course lower than Algebra in Grade 8 despite scoring at the 90th percentile or above in math on NAEP. This research suggests that achievement gaps exist and persist at the highest levels of achievement and performance and are perhaps compounded as minority students are less likely than their White counterparts to have access to more advanced courses, particularly in STEM areas.
Bromberg and Theokas (2013) found that the minority gaps in math achievement, in particular, were likely to be greater among high achievers in both Grades 4 and 8. Additionally, Reardon (2008) claimed that the minority achievement gaps in both mathematics and reading test scores increase at a greater rate between kindergarten and fifth grade among students who enter kindergarten with high levels of skill than among students who enter with lower levels of skill. Indeed, according to Reardon, the gaps grow approximately twice as fast for students who enter kindergarten with scores 1 standard deviation above the mean as for students who enter kindergarten with scores 1 standard deviation below the mean. These findings have enormous implications for students who demonstrate early high potential, both with respect to their continued academic performance in later elementary years and middle school and their long-term performance and course selection, access to elite colleges, and employment in jobs with the highest skill demands and the highest pay. This is particularly striking when considering the National Science Foundation (1986) finding that while college-bound Blacks expressed an interest in majoring in math or science, only a small portion of them actually did.
The Impetus for Project Excite
Project Excite responded to a call for increasing access to academic supplemental programs to close the achievement gap, beginning in the early primary years and through middle school. In 1999, the year before Project Excite began, the College Board published a report titled Reaching the Top: A Report of the National Task Force on Minority High Achievement. The report highlighted a number of findings and provided recommendations in response. First, “Getting off to a good start in elementary school puts children on track to be good students in high school” (College Board, 1999, p. 8). Another finding was that Black, Hispanic, and Native American students at virtually all socioeconomic levels do not perform nearly as well on standardized tests as their White and Asian counterparts. Significantly, some of the largest of these gaps are often found at middle and professional class levels, at least when they are measured by the education of students’ parents. (College Board, 1999, p. 9)
Finally, the College Board (1999) Task Force highlighted that “many high-achieving students from all racial and ethnic groups are beneficiaries of extensive formal and informal supplementary educational opportunities over time, many of which are provided directly or paid for by their parents” (p. 25). The College Board Task Force recommended that an important step to closing the achievement gap at the highest levels of achievement would be to establish and increase access to supplementary educational programs beginning in the primary grades. Recent research by Schmidt, Burroughs, Zoido, and Houang (2015) buttresses the argument for expanded learning opportunities to address achievement gaps. These authors found that one third of the relationship between SES and literacy can be accounted for by differences in opportunities to learn, on the basis of PISA data in an international study of the role of schooling in perpetuating inequality. According to the National Summer Learning Institute, about two thirds of the achievement gap between lower and higher income students in the ninth grade is attributable to disparities in access to summer learning opportunities during elementary and middle school (National Summer Learning Association, n.d.). Research has also demonstrated that closing the excellence gap by eighth grade would safeguard the future professional well-being of underrepresented minorities. Eliminating the test score gap that arises by the end of middle school may be “a critical component of reducing racial wage inequality” (Fryer & Levitt, 2004, p. 447). In sum, research supports the idea that intervention in the early primary grades by means of supplemental educational programs could have long-ranging and cumulative effects, especially for underrepresented students who have the potential to achieve at the highest levels.
Project Excite
Project Excite was developed in 2000 by Northwestern University’s Center for Talent Development (CTD) of the School of Education and Social Policy, in collaboration with Evanston/Skokie School District 65 and Evanston Township High School (ETHS) District 202, two large, racially and socioeconomically diverse suburban school districts north of Chicago. In school districts 65 and 202, African American and Latino students comprise a high percentage of student enrollment. Together, the percentage of African American and Latino students (47.5%) is higher than that of White students (43.4%) at ETHS. However, African American and Latino students in these school districts have been consistently underrepresented in advanced mathematics and science classes. For example, in the first semester of 2013-2014, only 55.6% of ETHS Grades 9 to 12 Black/African American students and 66.1% of all ETHS Hispanic/Latino students were enrolled in one or more honors or Advanced Placement courses, compared with 92.5% of Grades 9 to 12 White students (ETHS, 2014b).
Project Excite was designed to support and enhance minority students’ interest and performance in math and science through extensive supplemental programing, with the ultimate goal of preparing participants for advanced-level math and science coursework in high school. Project Excite can be considered as an exemplary, front-loading program for bridging the gap in the readiness of some culturally, linguistically, and ethnically diverse (CLED) gifted students, nurturing their talents, preparing them for advanced-level courses or programs, and improving their representation and successful participation in gifted programs (Adams & Chandler, 2013; Briggs, Reis, & Sullivan, 2008).
Over the years, Project Excite’s programming components have changed, evolving through two stages. In its first stage, from 2000 to 2013, Project Excite identified high-potential CLED third graders and provided them with enrichment and accelerated learning opportunities through eighth grade. Participating students attended Project Excite activities after school, during weekends, or in the summer in addition to their normal schooling. The specific goals of Project Excite during this stage were to help participating students successfully complete at least Algebra 1 and gain considerable laboratory science experience before entering ninth grade.
Overall, the first stage of Project Excite provided approximately 445 required and 180 optional hours of Saturday, summer and after-school activities that were designed to prepare students for advanced-level math and science coursework in high school. Specifically, the number of required learning hours for Grades 3 through 8 was 15, 40, 40, 145, 80, and 125 hours, respectively (see Lee, Olszewski-Kubilius, & Peternel, 2009, for a detailed description of Project Excite program activities by timing and grade level). All activities supplemented rather than supplanted regular school learning. The specific program activities varied to some degree by grade level.
In the third grade, Project Excite students attended 10 afterschool classes at ETHS and a 6-week, half-day summer program on reading and math at the CTD. Students were routinely divided into small groups facilitated by ETHS students, often former Project Excite students, who served as teaching assistants. In the fourth grade, students attended 16 Saturday classes that provided an integrated mathematics and science course with a focus on mathematical problem solving, science experimentation, and the designing of experiments. They also attended a 6-week, half-day summer program for reading and math. In the fifth grade, students attended an 8-week Saturday enrichment session at CTD in the fall and an 8-week session during the winter. They chose math, science, or technology classes that matched their abilities based on their own interests. Students also attended an 8-week (once a week) summer program held in conjunction with a local library that focused on literacy skills.
In the sixth grade, students took self-contained classes focused on prealgebra concepts and algebraic thinking. They also attended CTD’s Apogee program, a 3-week all-day summer program, with a residential add-on component, focusing on enrichment classes in math or science. Additionally, they participated in Northwestern University’s Midwest Academic Talent Search, through which they took the Explore test, a test typically given to eighth-grade students, as an above-grade-level assessment. In the seventh and eighth grades, students attended eight Saturday sessions at the CTD in the winter. In the summer, seventh-grade students attended a special 3-week, 4-hours-per-day laboratory science class segmented into 1 week of biology, chemistry, and physics. Students in eighth grade attended a specially designed 4-week, all-day bridge program that consisted of instruction in math, focusing on ensuring that students attained a firm foundation for geometry or Algebra 2 when entering ninth grade. The bridge program also served as an early introduction to the high school’s policies and procedures, giving students an edge when the school year began. They also had access to one-on-one or small group tutoring with undergraduates of local universities.
Project Excite also hosted regular parent meetings for each grade level to ensure parents remained informed about the program components and expectations, and organized one to three parent educational events annually for the education and support of parents. Parents were invited to attend parent seminars or workshops held on Saturdays in conjunction with the Saturday Enrichment Program at CTD. Additionally, many families participated in CTD’s annual weekend family conference in the summer, “Opportunities for the Future,” which provided sessions for parents, enrichment classes for younger students, and workshops on college and career for older students.
Starting in the fall of 2013, Project Excite entered its second stage, wherein a high school component was added to provide substantial continued support for Project Excite students in Grades 9 to 12 in order to help students maintain high academic achievement, make appropriate college choices, and become strong candidates for entrance to prestigious institutions of higher education.
The Current Study
The current study examines the academic outcomes of the first stage of Project Excite (2000-2013). It addresses the following four research questions:
Method
Participants
Project Excite Students
Between 2000 and 2013, Project Excite recruited 14 cohorts of students. The first cohort of third graders entered Project Excite in 2000 and graduated from high school in 2010. The most recent cohort of third graders (i.e., Cohort 14) was recruited in fall 2013. The number of students in each cohort varied, ranging from 19 to 30. In total, the participants of the current study consisted of 361 students from Cohorts 1 to 14.
Project Excite students were identified from five participating elementary schools in District 65. In these five elementary schools, all third graders from underrepresented minority groups were invited to test to determine their eligibility for Project Excite. Selection of students was based on performance at or above the 75th percentile on the Naglieri Nonverbal Ability Test (NNAT) and on a reading or math subtest of the Iowa Tests of Basic Skills. Students whose scores fell below this level but who had strong, positive recommendations from classroom teachers regarding work habits, achievement, ability, and interest were also admitted on a case-by-case basis. The identification procedures for Project Excite changed over the 14 years of project implementation. Most significantly, initially only mathematics and NNAT scores were taken into consideration, but reading scores were incorporated into the selection procedures beginning with Cohort 4 students as we learned that reading ability was equally important to success in STEM classes. Family income was not a criterion for selection because research shows that even minority students from advantaged families underperform as compared with their White counterparts (Miller, 2004). However, at least 80% of students were from low-income families.
Student attrition over the course of the program was anticipated, as Project Excite was a 6-year intervention in its first stage. Among the 361 recruited students across the 14 cohorts, 76.5% of the participants who started at the third grade completed or are currently participating in the program. Approximately 23.5% of students left the program before ninth grade, including 10.2% who moved out of the district or transferred into local private schools as a result of parents’ better understandings of their child’s educational needs. The 10.2% mobility rate of Project Excite students was higher than that of District 65 (6%) but slightly lower than that of the State of Illinois (12.8%) in the 2013-2014 school year. Considering that all students were minorities, and most were from low-income or socioeconomically disadvantaged backgrounds, the 10.2% mobility rate is on par with research that shows mobility for minority and low-income students is the highest among all students (Fong, Bae, & Huang, 2010). The remaining 13.3% dropped out of the program for various reasons. Specifically, 3% left due to transportation challenges, conflicts with other family activities such as church, or significant family health problems; 8.3% left the program for unknown reasons; and 1.9% left the program because students lacked interest or motivation to continue.
Comparison Students
Although we were not able to construct a true comparison group for Project Excite students, we compared Project Excite students with students in their home K-8 district (District 65) and with students in the high school district (District 202), which Project Excite students entered after eighth grade. In addition, we also compared Illinois Standards Achievement Test (ISAT) math and reading median scores of Project Excite with those of the State of Illinois.
Evanston/Skokie School District 65
District 65 is a school district headquartered in Evanston that serves both Evanston and part of Skokie, two suburban cities north of Chicago. According to its 2013-2014 enrollment statistics, District 65 enrolled 7,116 students in the K-8 grade levels, 45% White, 25% Black, 18% Hispanic, 8% multiracial, and 4% Asian. Thirty-seven percent (2,697) of students qualified for free/reduced-price lunch programs. Of those students, 32.6% (2,323) qualified for the free-lunch program, indicating a higher level of poverty. The percentage of low-income students at each of the District 65 schools ranges from 18% to 62%. About 11.8% (838) of students were English Language Learners.
Evanston Township High School (ETHS) District 202
ETHS is a comprehensive high school (Grades 9-12) in Evanston, nationally ranked in the top 2 to 3 percentage of high schools. According to its 2013-2014 School Statistical Report, ETHS enrolled 3,120 students, 43.3% White, 30.9% African American/Black, 16.6% Hispanic/Latino, 4.8% multiracial, 3.9% Asian, 0.3% American Indian, and 0.1% Native Hawaiian. About 41.4% of the students were from low-income backgrounds. Its 4-year graduation rate was 85.6% for the Class of 2013 and its college attendance rate was 76% for the Class of 2011 (ETHS, 2014b).
Achievement Outcomes
The current study analyzed three standardized achievement data, including ISAT scores, Explore scores, and Measures of Academic Progress (MAP) scores (Table 1). It also compared the proportions of Project Excite participants and District 202 ninth graders placed on grade-level versus above-grade-level math course. In addition, we reported on the college placement statistics of the first five cohorts of Project Excite after they graduated from high school.
Achievement Data Analyzed in the Current Study.
Note. ISAT = Illinois Standards Achievement Test; ETHS = Evanston Township High School; MAP = Measures of Academic Progress.
The mean ISAT math and reading scores of District 65 were available only for the years 2009-2013, reported in the district’s 2013 Accountability & Achievement Report. ISAT math and reading scores (i.e., third-eighth grades) of Project Excite Cohorts 4 to 13 who entered the program between 2003 and 2013 were available to us. No scores were available for Cohorts 1 to 3 (i.e., the earliest three cohorts) nor for students of Cohort 14 who entered the program in fall 2013. We averaged the ISAT scores within each grade level across the cohorts for which scores were available. bCohorts 1 to 5 of Project Excite students graduated from high school in 2010-2014, respectively. Detailed information regarding the college placement of ETHS graduates during this period was not available publically.
Illinois Standards Achievement Test (ISAT) Scores
ISAT reading and math scores were collected for Project Excite students from Grades 3 through 8. The ISAT is used to measure student achievement in mathematics and reading in third through eighth grades and science in fourth through seventh grades. ISAT tests were usually administered in March or April. Under the 2014 ISAT scales, all third- through eighth-grade students have a minimum ISAT math score of 120, and maximum scores ranging from 341 to 410, depending on grade levels; all third through eighth graders have a minimum ISAT reading score of 120, and maximum scores ranging from 329 to 379.
We obtained District 65 students’ ISAT math and reading scores from the District 65’s Accountability & Achievement Report for the 2013 academic year, which reported ISAT math and reading mean scores for District 65 students averaged across 2009 to 2013 for Grades 3, 4, 6, 7, and 8, respectively. For example, the third-grade mean ISAT math score represented the average ISAT math scores of all third graders within the district from 2009 to 2013. The report also presented District 65’s ISAT math and reading scores by major ethnic groups (i.e., White, Black, Latino, Asian, and multiracial) in each grade level. It also provided the median ISAT (math and reading) scores of students in Grades 3, 5, and 8 in District 65 and the State of Illinois, as well as the median scores for the major ethnic groups (i.e., White, Black, Hispanic, Asian, and multiracial).
Explore Scores
Explore is a curriculum-based test, designed as a precursor to the American College Testing, whose purpose is to assess students’ levels of knowledge and skills in the areas of math, science, reading, and English prior to entry into high school. Explore subject scores range from 1 to 25. The composite score is the average of test scores in math, science, reading, and English.
Project Excite Cohort 1 students took the Explore test during the 2005-2006 school year when they were in eighth grade. Cohort 8 students took the test during the 2012-2013 school year as eighth graders. Most of the Project Excite cohorts’ scores were documented as percentile ranks rather than scale scores. Scores of some cohorts were missing. To compare the scores of Excite students with those of the ETHS students, which were reported as scale scores, we converted Project Excite students’ percentile ranks into scale scores. Because the American College Testing normed Explore test in 2005 and renormed it in 2010, we applied the 2005 norm scale to the conversion of the test scores obtained before 2011 and applied the 2010 norm scale to that of the test scores obtained in 2011 and thereafter.
Until 2015, all eighth-grade students in Districts 65 took the Explore test for placement into classes for their freshmen year at ETHS. The 2013-2014 Annual Report on Student Achievement (ETHS, 2014a) provided the Explore math, science, reading, English, and composite scores of ETHS ninth graders who took the Explore test in the eighth grade. With available scores, we were able to match Project Excite Cohorts 1, 2, 3, 5, 6, 7, and ETHS students who took EXPLORE on math, science, and reading scores; match Excite Cohorts 5 to 7 and ETHS students on English scores; and match Excite Cohorts 1 to 3 and 6 to 8 with ETHS students on composite scores.
Measures of Academic Progress (MAP) Scores
A third measure of achievement performance was the percentage of students at or above the college and career readiness standards (CCRS) using the national 50th percentile as the benchmark on the MAP tests. MAP is a norm-referenced, computerized, adaptive achievement test of student growth over time in mathematics and reading. In order to track student growth, MAP is often administered to students in Grades 1 through 8 two or three times throughout the school year: in the beginning (fall), middle (winter), and end of the school year (spring). We obtained the percentage of District 65 students at or above CCRS on MAP math and reading from the District’s Accountability & Achievement Report for the 2013 academic year (Evanston/Skokie School District 65, 2014). This report presented data for the spring of 2010 through 2013, combining students in third through eighth grades in each year. We combined Project Excite students across Grades 3 through 8 and calculated the percentage of students who were at or above CCRS using the national 50th percentile as the benchmark for springs for 2010-2013.
Ninth-Grade Math Placements
A fourth measure of academic achievement was the percentage of Project Excite ninth graders placed in above-grade-level math courses, compared with their peers at ETHS. Taking regular/honors Algebra 1 courses in Grade 9 was categorized as at-grade-level math placement. Taking regular/honors Algebra 2, regular/honors Geometry, and any math courses above Algebra 2 was categorized as above-grade-level math placement. We calculated the percentages of at- and above-grade-level math placements for each of Cohorts 1 to 9 and the average percentage across the cohorts. We compared this information with that of all of ETHS’s ninth-grade students, and African American and Latino in ninth grade during the 2013-2014 school year.
College Placements
The last academic achievement measure was participating students’ college placement after high school. We collected this information for Cohorts 1 to 5 of Project Excite students who entered the program between 2000 and 2004 and graduated from high school between 2010 and 2014. We categorized the colleges/schools into two groups: 4-year colleges and community colleges/technical schools. To identify whether a 4-year college was ranked in the top 50 of U.S. colleges, we referred to the 2014 U.S. News & World Report’s Top National Universities Rankings and Top National Liberal Arts Colleges Rankings (“The Best Colleges and Universities Rankings,” 2013). We calculated the percentage of students in each category. We were not able to compare this information with that of ETHS students, as the latter was not publicly accessible.
Results
Effect Sizes
We calculated Hedges’s g to indicate the difference between the performance of Project Excite students and their comparison groups. Hedges’s g is a standardized mean difference. A Hedges’s g was calculated by dividing the mean score difference between Project Excite and a comparison group (e.g., District 65 White or overall students) with the average standard deviation of the two groups. The standard deviations of all comparison groups were not reported in any of the reports on which we relied in gathering the aggregated mean or median scores and no individual-level comparison group data were accessible to us. To overcome this problem, we therefore assumed equal variances (standard deviations) for Project Excite students and their related comparison groups. For example, the standard deviation of District 65 Grade 3 ISAT math mean score was assumed to be equal to that of Project Excite third graders, which was 30.34. Due to this limitation, effect sizes in the current study should be only considered as a rough estimation of the effect at most.
Illinois Standards Achievement Test (ISAT) Scores
ISAT Mean Scores
Tables 2 and 3 show the descriptive information of ISAT math and reading scores and related effect sizes.
Comparing Project Excite With District 65: Illinois Standards Achievement Test (ISAT) Math.
For both math and reading, the total number of third-, fifth-, and eighth-grade students in District 65 was reported in the district’s report, but the total number of fourth-, sixth-, and seventh-grade students was not reported. In general, the number of students decreased from third to eighth grades in the district. We assumed the total number of students was 716, the mean number of third-grade and fifth-grade students in the district. Similarly, the respective total number of District 65 sixth- and seventh-grade students was assumed as the mean number of fifth- and eighth-grade students. bThe standard deviations of District 65 ISAT scores were not reported in the district’s 2013 Accountability & Achievement Report. We assumed equal variances (i.e., standard deviations) within a same grade level between scores of Project Excite students and those of District 65. For example, the standard deviation of Project Excite third-grade ISAT math was 30.34; we therefore assumed the standard deviation of District 65 third-grade ISAT math was also 30.34. cThe 95% confidence interval (CI) of each Hedges’s g. dThe number of cohorts with available ISAT scores varied by grade level. For example, third-grade ISAT math and reading scores were available for Cohorts 4 to 13, but not for Cohort 10; seventh-grade ISAT math and reading scores were available for Cohorts 4 to 5 and Cohorts 7 to 9; eighth-grade scores were only available for Cohorts 6 to 8. eThe mean scores of District 65 students were obtained from the district’s 2013 Accountability & Achievement Report. This report presented the mean score of each grade level by year with zero decimal. We calculated the mean score of each grade across 2009-2013 and reported the outcome with one decimal.
Comparing Project Excite With District 65: Illinois Standards Achievement Test Reading.
Math
Project Excite students in third grade (M = 231.35) performed almost equally well as District 65 students in third grade overall (M = 231.4) on ISAT math, g = −0.001, 95% confidence interval [CI −0.16, 0.15]. It is important to note that third-grade ISAT scores were collected approximately 6 months after Project Excite students entered the program, still the early stage of a 6-year intervention. Therefore, it is reasonable to use the third-grade ISAT scores as the baseline achievement data of Project Excite students and their comparison peers in District 65. Although Project Excite students were relatively high achievers selected at the third grade from a pool of minority students, they were not the highest achievers, especially compared with their White or Asian peers. Most of the students would not have qualified for a typical gifted program that used achievement at the 90th or 95th percentile to qualify. The result that the mean ISAT math scores of Project Excite third-grade students were equivalent to those of District 65 students overall suggests that the average Project Excite student was comparable to the average District 65 student at the beginning of the project.
However, at each grade level from fourth through eighth, the mean ISAT math scores of Project Excite students (Ms ranging from 254.73 to 301.38) were consistently significantly higher than that of District 65 students (Ms ranging from 231.4 to 291.8), with effect sizes ranging from 0.30 to 0.53. We were 95% confident that the effect sizes fell within the range as indicated by respective associated 95% CIs (see Table 2). From third to fourth grades, fourth to fifth grades, fifth to sixth grades, sixth to seventh grades, and seventh to eighth grades, Project Excite students’ ISAT gain score was 23.38, 17.77, 11.83, 9.13, and 7.92, respectively. This led to a total gain score of 70.03. For District 65 students overall, the respective gain score was 16.4, 14.8, 10.8, 8.8, and 9.6, with a total gain score 60.4. On average, Project Excite gained 14.1 points each year, whereas District 65 students overall gained 12.08 points each year. Thus, Project Excite students gained 9.63 more points in average than did District 65 students overall on ISAT math in a 6-year period. Taken together, results showed that the ISAT math scores of Project Excite participants continually improved from fourth to eighth grades after the first year in the project.
Reading
As Table 3 shows, the mean ISAT reading scores of Project Excite students in third grade (M = 218.36) was lower, although not significantly, than that of District 65 students in third grade overall (M = 221.2), g = −0.11, 95% CI [−0.27, 0.43]. This suggests that Project Excite students’ reading achievement in the beginning of the project was comparable to average third graders in District 65. However, at each grade level from fourth through eighth grade, Project Excite students (Ms ranging from 238.40 to 269.09) consistently and significantly outscored District 65 students overall (Ms ranging from 233.6 to 260.0). Respective effect sizes ranged from 0.21 to 0.51 and we were 95% confident that the effect sizes fell within the range as indicated by each 95% CI. From third to eighth grades, Project Excite students achieved a total gain score of 50.73, comparing with 38.3 of District 65 students overall. Thus, Project Excite students gained 11.93 more points than did District 65 students overall on ISAT math in a 6-year period. On average, Project Excite gained 10.15 points each year, comparing an average of 7.76 points for District 65 students overall. Project Excite students significantly improved their reading performance after their first year in the project through eighth grade. Figure 1 shows their growth pattern from third to eighth grades in comparison with District 65 students overall, as measured by ISAT math and reading mean scores.
Growth pattern of mean scores on Illinois Standards Achievement Test (ISAT) math (left) and reading (right).
ISAT Median Scores
To supplement the above analysis on ISAT mean scores, we conducted one-sample median tests (the Wilcoxon signed-rank tests) to assess whether the median scores (i.e., the observed median) of third-, fifth-, and eighth-grade students in Project Excite (i.e., the sample) were significantly different from those of their peers in District 65 and the State of Illinois students (i.e., a known median) as reported in the Evanston/Skokie School District 65’s 2013 Accountability & Achievement Report.
Table 4 presents the results of median equality tests on the ISAT math scores. In third grade, students of Project Excite Cohort 13 (median = 241) performed almost equally as well as District 65 students in third grade overall (median = 232), z = 1.55, p = .121. However, their median ISAT math score (median = 241) was significantly lower than that of District 65 Asian (median = 251) and White (median = 247) third grade (z = −3.12, −2.17, p = .002, .03, respectively).
Tests of Median Equality Between Project Excite and District 65/the State of Illinois: Illinois Standards Achievement Test (ISAT) Math.
We conducted tests of median equality between the median ISAT math and readings scores (see Table 5) of third-, fifth-, and eighth-grade students in District 65 and the State of Illinois in 2013 and those of Project Excite Cohorts 13, 11, and 8 students who were in third, fifth, and eighth grades in 2013, respectively. bA positive z score along with a significant p value indicates that Excite students’ median score was statistically higher than the median score of the comparison group. A negative z score along with a significant p value indicates that Excite students’ median score was statistically lower than the median score of the comparison group.
In fifth grade, students of Project Excite Cohort 11 (median = 280) significantly outperformed District 65 students in fifth grade overall (median = 267), z = 3.03, p = .002. Project Excite students appeared to make large gains in fifth grade, demonstrated by the fact that they performed equally well as Asian (median = 280), White (median = 280), and multiracial (median = 280) students, z = 1.05, p = .293 for all three related tests. In eighth grade, students of Project Excite Cohort 8 (median = 296) were significantly outperformed by District 65 White students in the eighth grade (median = 314), z = −2.48, p = .013. They performed almost equally well with District 65 Asian (median = 305) and multiracial students (median = 296), z = −0.67, 0.99, p = .501, 0.325, respectively. However, they again significantly outperformed District 65 students in eighth grade overall (median = 289), z = 2.46, p = .014. These outcomes suggested that the math achievement gap between Project Excite and District 65 White, Asian, and overall students was reduced after third grade, as measured by ISAT math median scores.
Project Excite students consistently and significantly outperformed all subgroups of the State of Illinois on ISAT math, including White (median = 226, 249, 279 in third, fifth, and eighth grades, respectively), multiracial (median = 221, 244, 274), and students overall (median = 217, 241, 272), with the exception of Asian students. They performed almost equally as well as Asian students in third and eighth grades (median = 241, 300, respectively; z = −0.74, 0.12, p = .458, .904, respectively) and they outperformed Asian students in fifth grade (median = 267; z = 3.03, p = .002). These results can be partially attributed to the fact that District 65’s education performance in general is higher than that of the State of Illinois.
Results of one sample median equality tests on ISAT reading scores (see Table 5) revealed a similar pattern to what was found from the analyses on ISAT median math scores. Taken together, the median equality test results showed that across third, fifth, and eighth grades, the achievement gap between Project Excite students, District 65, and the State of Illinois’s White and Asian Students was eliminated.
Tests of Median Equality Between Project Excite and District 65/the State of Illinois: Illinois Standards Achievement Test Reading.
Explore Scores
Table 6 shows descriptive statistics and effect sizes for Explore math, science, reading, English, and composite scores, comparing Project Excite eighth graders and those of ETHS students.
Math, Reading, Science, English, and Composite: Explore Scores.
Project Excite students’ Explore math, reading, science scores were averaged across Cohorts 1 to 3 and 5 to 7 who took the Explore tests in eighth grade during the 2005-2008 school years and 2009-2012 school years, respectively. We kept two decimal places for all calculated values. bExplore math, reading, science, English, and composite scores of Evanston Township High School (ETHS) ninth-grade students were retrieved from the ETHS Annual Report on Student Achievement which reported aggregated student achievement data. The mean scores for ETHS students and the subgroup of ETHS students were reported with one decimal place in the ETHS Annual Report on Student Achievement. cThe Explore English scores of Project Excite Cohorts 5 to 7 students were available to us. These students took Explore English tests during the 2009-2012 school years, respectively. To ensure comparability, the average mean Explore English scores of these students were compared with those of ETHS students who took the Explore during the respective comparable school years. dThe Explore Composite scores of Project Excite Cohorts 1 to 3 and 6 to 8 were available to us. These students took Explore tests during the 2005-2008 and 2010-2013 school years, respectively. To ensure comparability, the average mean Explore Composite scores of these students were compared with those of ETHS students who took the Explore tests during the respective comparable school years.
Math
Project Excite students in eighth grade performed almost equally well on Explore math (M = 19.74) to their peers who took Explore tests in eighth grade and then entered ETHS in ninth grade (M = 19.8), g = −0.02, 95% CI [−0.54, 0.50]. However, Project Excite students in eighth grade significantly outperformed ETHS Black (M = 15.03) and Latino (M = 16.06) students, g = 1.58, 1.21, 95% CI [1.02, 2.13], [0.63, 1.78], respectively. When ETHS ninth-grade students were grouped into low-income and non-low-income students, Project Excite students outperformed ETHS low-income students (M = 15.1), g = 1.55, 95% CI [1.00, 2.09]; their mean score was slightly higher but not statistically significantly than that of ETHS non-low-income counterparts (M = 19.3), g = 0.15, 95% CI [1.00, 2.09].
Reading
Project Excite students in eighth grade (M = 16.84) performed significantly better on Explore reading than their Black (M = 13.6) and low-income (M = 13.8) counterparts in ETHS, g = 0.75, 0.71, 95% CI [0.22, 1.28], [0.18, 1.23], respectively. Their Explore reading mean score was lower than that of ETHS White (M = 18.8) and non-low-income students (M = 18.1), and higher than that of Latino students (M = 14.5), but none of these comparisons was statistically significant.
Science
Project Excite eighth graders’ mean score on Explore science (M = 19.41) was lower but not statistically significantly than that of ETHS White (M = 20.2) and non-low-income students (M = 19.8). However, they performed significantly better on Explore science than their Black (M = 16.1), Latino (M = 16.6), and low-income (M = 16.0) counterparts in ETHS, g = 1.26, 1.07, 1.30, 95% CI [0.72, 1.80], [0.50, 1.63], [0.77, 1.84], respectively.
English
Similar to the case of Explore science, the Explore English mean score of Project Excite students in eighth grade (M = 18.91) was not statistically significantly lower than that of ETHS White (M = 20.0) and non-low-income students (M = 19.4). However, they performed significantly better than ETHS Black (M = 14.1), Latino (M = 15.2), and low-income (M = 14.1) students, g = 1.38, 1.06, 1.38, 95% CI [0.81, 1.94], [0.49, 1.63], [0.83, 1.93], respectively.
Composite Scores
The Explore mean composite score of Project Excite students in eighth grade (M = 18.14) was significantly lower than that of ETHS White students (M = 29.7), g = −0.62, 95% CI [−1.12, −0.11]. Project Excite eighth-grade students performed slightly better, although not significantly, than ETHS overall (M = 17.4). However, they performed significantly better than ETHS Black (M = 14.7) and Latino students (M = 15.2), g = 0.29, 1.36, 1.15, 95% CI [0.81, 1.94], [0.49, 1.63], [0.83, 1.93], respectively.
Taken together, as Figure 2 shows graphically, Explore scores suggest that there was still an achievement gap between Project Excite eighth graders (who were primarily low-income Black and Latino students) and their White and non-low-income counterparts in ETHS. However, Project Excite students in eighth grade consistently and significantly outscored their Black, Latino, and low-income peers in ETHS on all subjects and composite scores. They also significantly outperformed ETHS students overall on Explore composite scores.
Explore mean scores of Project Excite and Evanston Township High School.
MAP Scores
Table 7 shows the percentages of Project Excite and District 65 students who met or exceeded the CCRS 50th percentile benchmark on MAP scores during spring 2010 through spring 2013. The average percentages of Project Excite students meeting or exceeding the CCRS 50th percentile benchmark on MAP math (92%) and reading (91.2%) across the 4 years were significantly higher than those of District 65 White (math 66.8%, reading 80.5%), Asian (57.9%, 65.6%), multiracial (46.3%, 60.9%), all students (42.4%, 55.4%), Hispanic (20.7%, 32.2%), and Black (14%, 27.2%), p < .001, for all related one-sample tests of proportions (one-tailed). The difference was statistically significant in each respective year, p < .001 (one-tailed). Although one needs to take into consideration the small group sizes of Project Excite, these outcomes at least showed that Project Excite performed impressively well on MAP math and reading as a group.
Percentage of Project Excite and District 65 Students at or Above CCRS on MAP a : Math and Reading.
Note. CCRS = college and career readiness standards; MAP = Measures of Academic Progress.
We obtained the percentage of District 65 students at or above CCRS on MAP math and reading from the District’s 2013 Accountability & Achievement Report. This report presented data for the springs of 2010 through 2013, combing students in third through eighth grades in each year. bDistrict 65 used the national 50th percentile as the CCRS benchmark. cResults from each year are from spring MAP testing.
Ninth-Grade Math Placements
Table 8 show the percentages of at- and above-grade-level math placements in the ninth grade for Project Excite Cohorts 1 to 9, compared with ETHS Students in the 2013-2014 school year. Although the percentage of students who were placed at an above-grade-level math course in ninth grade varied somewhat by cohort, the percentages exceeded that for the district overall and showed an upward trend across years. An average of 76% of Excite students in Cohorts 1 to 9 were placed in an above-grade-level math course (Algebra 2 honors, Geometry honors) in their high school freshmen year, which was not statistically higher than of all ETHS students (72%), z = 1.09, p = .138 (one-tailed one-sample test of proportion), but was statistically higher than that of African American and Latino students (50%), z = 6.35, p < .001.
Ninth-Grade Math Placements of Project Excite Cohorts 1 to 9 and Evanston Township High School (ETHS) Students in 2013-2014 School Year.
Note. All values are in percentage.
This information was based on all ETHS ninth-grade students in the 2013-2014 school year. bProject Excite Cohorts 1 to 9 were in ninth grade during 2006-2014, respectively. That is, Cohort 1 students were in ninth grade in 2006, Cohort 2 students were in ninth grade in 2007, and Cohort 9 students were in ninth grade in 2014.
Across Cohorts 1 to 9, the average percentage of Project Excite students being placed in on grade-level math courses (e.g., Algebra 1) in ninth grade was 23%, which was not statistically different from that of all ETHS students (28%), z = −1.36, p = .174 (two-tailed one-sample test of proportion), but was statistically lower than that of African American and Latino students (50%), z = −6.59, p < .001 (one-tailed one-sample test of proportion).
Together, ninth-grade math placement data showed that the proportion of Project Excite participating students being placed on advanced above-grade-level math courses in ninth grade after exiting the project at end of eighth grade was almost as high as that of ETHS students overall, and was significantly higher than that of their African American and Latino peers. Project Excite improved the representation of minority students in advanced tracks in their high school.
College Placements
Although the first stage of Project Excite focused on Grades 3 through 8, we collected Cohorts 1 to 5 college placement information after their high school graduation. The percentage of students for whom there was college matriculation information available ranged from 37% to 60% for the first five cohorts, with an average of 53%. Some students left the district, others attended schools other than ETHS for high school, and some students did not provide college matriculation information.
Table 9 presents the college placement information of Project Excite students. Among them, 84.5% (49 out of 58) entered a 4-year college, 27.6% (16 out of 58) enrolled a 4-year college that ranked in the top 50 U.S. colleges based on 2014 U.S. News & World Report rankings, and 15.5% (9 out of 58) entered in community colleges or technical schools. The percentages of students attending more selective institutions increased and the percentage of students attending community colleges and technical schools decreased with later cohorts of students. We attribute this partly to the change in our selection procedures and to a greater focus in our parent and student workshops on college guidance and counseling. The college placement information of other students in ETHS was not accessible for comparison purposes. However, the 84.5% college enrollment of Project Excite first five cohorts is substantially higher than the sum (72%) of the national average college enrollment of Black (37.3%, the average of 3 years) and Hispanic students (34.7%, the average of 3 years) between 2010 and 2012, based on the information in the Digest of Education Statistics 2013 (Snyder & Dillow, 2015).
Project Excite Students College Placement.
These colleges were ranked as top colleges in the U.S. News & World Report 2014 edition of the National Liberal Arts Colleges Rankings and Top National Universities Rankings.
Summary of Findings
Project Excite was designed to address the math and science achievement gaps between minority and nonminority students within a large, diverse, suburban school district. Students were identified as having potential for future high achievement in the STEM field based on above average, but not stellar performance on standardized tests. They were given significant amounts of supplemental, outside-of-school programming designed to boost their achievement and prepare them for placement in advanced STEM courses in high school. The current study examined the academic outcomes of Project Excite, particularly their ISAT math and reading scores, Explore test scores, MAP scores, high school freshmen math placement, and college placement. The results of the study revealed six findings:
The data suggest that Project Excite reduced the math and science achievement gap between disadvantaged high-potential minority and high-achieving majority (i.e., White and Asian) students, prepared them for above-grade-level placement in math, and improved their representation and participation in advanced courses in high school.
There was a consistent improvement among participants in multiple subjects (e.g., math, science, and reading) as shown by the overall positive trend in their test scores. Students’ achievement scores were comparable to those of students overall in District 65 when they began the program in third grade, but consistently surpassed their peers in both math and reading after fourth grade. In comparison with District 65 students as a whole, Project Excite students made larger achievement gains in science, reading, and math throughout their time in the program.
Although the achievement levels of different participant cohorts varied to some degree when they first entered the project and during the early years of their participation, the differences gradually diminished after Grade 5.
Although there was no significant difference in the achievement performances of different cohorts, the achievement levels of more recent cohorts appear to be higher than those of the earlier ones, perhaps attributable to the fact that, starting with Cohort 4, Project Excite selected students on both math and reading achievement scores.
Participants were well prepared for high school, suggested by their Explore scores which exceeded those of Black and Latino students and were comparable to the scores of White students, the highest scoring group in the district.
Participants’ college choices ranged but there was a notable trend toward the selection of more academically selective institutions of higher education and away from community colleges and technical schools as time passed.
Discussion
Implications for CLED Programming Practices
Findings of this longitudinal study of Project Excite suggest that front-loading programs (i.e., identifying high-potential children and providing learning opportunities for advanced work prior to formal identification) can be quite useful for nurturing and identifying the talents of high-potential minority students and increase their overall representation in advanced courses (Adams & Chandler, 2013; Olszewski-Kubilius & Clarenbach, 2012). As a front-loading program for gifted CLED students, Project Excite has many of the features put forward by Briggs et al. (2008) and Olszewski-Kubilius and Clarenbach (2012) for the identification and participation of CLED students in gifted programs. We highlight several features below.
Modified Identification Procedures
Briggs et al. (2008) found three main categories of identification strategies used in these programs based on their study of 25 promising programs for CLED gifted students: (a) employing alternative pathways for identification, such as using different assessments tools or eliminating formal identification procedures; (b) early identification, usually at the primary grade level but as early as preschool; and (c) including information about broader perspectives of performance, such as classroom observations or student work portfolios. As described earlier, Project Excite selected high-potential CLED students in third grade using multiple criteria, including nomination from their schools or parents, recommendations from their teachers regarding students’ work habits, achievement, ability, and interests, and performance on criterion-referenced tests (e.g., the ISAT) and/or norm-referenced (e.g., the Iowa Tests of Basic Skills; the NNAT). These identification strategies allowed Project Excite to be more inclusive of high-potential CLED students at an early age who otherwise might have not been identified or had their potential developed.
In addition, following the recommendations for best practice of Olszewski-Kubilius and Clarenbach (2012) in a review of exemplary programs for underrepresented gifted students, Project Excite used a lower threshold (i.e., 75th percentile) for students. The lower threshold is in keeping with the concept of “local norms” or using cutoffs that take into consideration students’ previous opportunities to learn. Finally, rather than relying exclusively on a nomination process by teachers with little or no training in gifted education, which has been shown to miss many qualified students (Ford, Grantham, & Milner, 2004), Project Excite invited all minority students from the five participating elementary schools to test for entrance into the program.
Front-Loading
Front-loading is a programming strategy that identifies high-potential children at an early age and provides them with opportunities for advanced work prior to formal identification for gifted programs. Its main goal is to bridge the gap in the readiness of some CLED students, nurture their abilities, and prepare them for success in advanced programs. Project Excite supported students from Grades 3 through 8 and aimed to prepare them for advanced-level coursework (especially in math and science) in high school. In total, a typical participating student engaged in approximately 445 required and 180 optional hours of learning through after-school, Saturday, and school break time (i.e., fall, winter, and summer) during the 6-year program from Grades 3 through 8. Assuming students spent 5 hours on learning activities during a normal school day, completing the required 445 hours in Project Excite means that students engaged in additional learning for an equivalent of almost 90 school days (445/5 = 89), not counting the 180 optional hours. In addition, Project Excite intervened early and provided support for a 6-year duration. Such intensive, front-loading programming is a feature that distinguishes Project Excite from many other similar programs (see Adams & Chandler, 2013; Briggs et al, 2008; Olszewski-Kubilius & Clarenbach, 2012, for descriptions of other programs).
Curriculum
Project Excite implemented a continuum of curriculum and learning opportunities, focusing on the needs of students at a particular time in their development. The after-school experiences for third graders were designed to stimulate excitement for STEM, as well as to give students a preview of the large high school they would attend, familiarity and confidence with high school lab spaces, and opportunities to interact with science and math teachers who would potentially teach them in the future. The summer programs for third and fourth graders were designed to intervene early to make sure that basic literacy and math skills were solid, ensuring a high level of academic progress in the later grades. Other project components consisted of early enrichment designed to prepare students for acceleration in mathematics during middle school. In addition, though all of the courses and program components took place outside of the students’ regular school day, they were designed to articulate and support in-school learning. For example, the Saturday classes for Project Excite students in sixth grade focused on algebraic thinking as students’ prepared to take exams at the end of the school year that would determine their math placement in seventh grade. Some of the curricula were designed to fill in for gaps in the school curriculum, for example, providing significant science laboratory experience in Grades 7 and 8.
Leveling the Playing Field
Tutoring with undergraduates provided access to supports that other more advantaged families garner on their own as well as giving Project Excite students some mentoring by older, successful students. Parent workshops delivered tacit knowledge about courses and advanced high school programs that more advantaged families might have ready access to. In these ways, lower income families had access to similar supports and information as more advantaged families.
Parent–Home Connection
Project Excite put considerable efforts into building connections between families and the program (Olszewski-Kubilius, Lee, Ngoi, & Ngoi, 2004). For example, it hosted regular parent meetings for the education and support of parents, conducted parent workshops exclusively for Excite families and involves Excite families in other CTD programs such as its Saturday speaker program and its annual summer family conference, “Opportunities for the Future.” In addition, it involved parents as volunteers to support or organize family gatherings such as picnics and excursions to plays and museums. In the later years of the program, parents formed their own parent group, ran their own events and identified topics and speakers for parent workshops. Such efforts helped build important bridges between the program and the families. Translators were used to enable Spanish-speaking families to benefit from the parent workshops and meetings. Over time, project coordinators and staff became trusted collaborators with parents in supporting their children’s educational paths and aspirations. Parents turned to staff for help with other children and or assistance in negotiating other issues affecting their child’s achievement or school adjustment.
Program Evaluation
Evaluation is an essential component of successful programs for gifted CLED students. Project Excite employed various efforts to track and document students’ progress in the program. For example, it collected multiple achievement data across the years, such as scores in ISAT, Explore, MAP tests, and information on ninth-grade math placement and college placement after high school. It also regularly conducted parent, student, and teacher surveys. In addition, Project Excite conducted and published a series of studies to examine important aspects of the project, such as students’ performance in math and science, the experience and perceptions of participants and their parents, and factors contributing to or hindering academic success of students (e.g., Cockrell & Olszewski-Kubilius, 2012; Lee et al., 2009; Lee, Olszewski-Kubilius, & Peternel, 2010; Olszewski-Kubilius & Clarenbach, 2014). Such concerted efforts in program evaluation and research helped amass a wealth of information on the students’ growth and rich knowledge to further refine the program.
Future Improvement for Project Excite
As discussed above, the comprehensive and intensive features of Project Excite make it exceptional among similar programs for CLED students. However, there is always room for improvement. While Project Excite moved away from relying exclusively on teacher referrals for student nomination and instituted a more inclusive strategy, teacher buy-in for the program and support for Excite students within schools is still important. This could be enhanced if teachers saw themselves as partners with Excite administrators in identifying and providing services to academically talented CLED students. One way to accomplish this is to provide training and helpful tools for partnering school personnel to enable them to better identify and support students. Project U-STARS~PLUS has developed student observational tools (i.e., the Teachers’ Observation of Potential in Students) and professional development modules (Coleman & Shah-Coltrane, 2011), which have been widely used with demonstrated results (Harradine, Coleman, & Winn, 2014) to help teachers recognize and nurture students’ potential in their classrooms. There are other validated instruments available for identifying academic potential of students from low-income families as well such as the HOPE Scale (Peters & Gentry, 2010).
Second, Project Excite needs to continue to explore the best integration of multiple achievement assessments for identifying eligible students. Like many other gifted programs, Project Excite chose to integrate the NNAT in its identification process because there was some earlier evidence supporting its use as a more appropriate measure of general ability (e.g., Naglieri & Ronning, 2000) that would be at least as effective as other ability tests that contain both verbal and nonverbal components (e.g., Naglieri & Ford, 2003). However, some recent research suggests that the nonverbal battery of Form 6 of the Cognitive Abilities Test (CogAT6) produces smaller gaps between minority groups such as Black, Hispanic, multiracial, and non-Asian English language learners than the NNAT does (e.g., Giessman, Gambrell, & Stebbins, 2013; Lohman, Korb, & Lakin, 2008). Also, following the recommendations of Lohman et al. (2008), using both the NNAT and achievement test scores resulted in a group of students whose aptitudes better matched the STEM program that was the focus of Project Excite.
Third, as described earlier, Project Excite added a high school component and began to serve students from third grade through high school starting in 2013. The high school component addresses important issues arising from the past 14 years of practice. Particularly, although most participants performed quite well during third through eighth grades with the support of Project Excite, some were not able to maintain their achievement after they entered high school, with the ninth grade being an exceptionally challenging and critical transition year. Some students encountered difficulty in selecting colleges or planning for future careers and they often lacked sufficient help and support from their family and school. The high school component aims to provide substantial continued support for students, so that they can maintain their achievement through Grades 9 to 12, and become strong candidates for entrance into selective institutions of higher education. It is reasonable to expect that this high school component can further leverage the comprehensiveness and intensity of Project Excite to better serve high-potential CLED students.
Fourth, Project Excite administrators need to remain flexible, responsive, and open to continuous improvement and making changes that would increase students’ participation and retention in the program. As mentioned earlier, the overwhelming majority of Project Excite students (76.5%) completed or are actively participating in the program, 23.5% of students left the program at some point, and only 1.9% left the program citing lack of interest or motivation to continue. This strong retention rate speaks to the fact that Project Excite met the needs of disadvantaged minority students who were eager for a program that would help them strengthen their academic abilities. The retention rate is also attributable to the fact that Project Excite has remained flexible, responsive, and has continuously sought further improvement, which has also led to some programming changes over the years. For example, Project Excite adjusted selection criteria to include reading scores to ensure the readiness of recruited students. Additionally, an invitation to apply for Project Excite was extended to all minority students in the third grade in participating schools rather than relying on teacher recommendations. The thinking was that families who sought out the opportunity, and who made the effort to bring their children for testing on a Saturday would likely be willing and able to support their participation in Saturday, summer and after-school classes. This has generally been the case.
Furthermore, Project Excite administrators increased the number of parent workshops on the ways that families could support their children and help them navigate critical issues such as peer influences, racial identity, and noncognitive aspects of high achievement, such as motivation, grit, and self-control. As students and family situations change over time, affected by significant events such as job losses, family instability, and health issues, Project Excite administrators need to continue to grapple with the level and kinds of support they can reasonably provide to students and families to prevent the attrition of students with issues that significantly affect their academic achievement.
There are also other innovative ways to improve Project Excite in the future. For example, it can consider providing a curriculum for improving students’ executive functioning skills, which have been linked to many important aspects of children and adolescents’ functioning including academic achievement, school behaviors, self-regulation, social–emotional development, and physical well-being (e.g., Best, Miller, & Naglieri, 2011; Fuhs, Nesbitt, Farran, & Dong, 2014; Langberg, Dvorsky, & Evans, 2013; Young et al., 2009). Another promising area is to nurture students’ leadership skills. With the newly added high school component, Project Excite has the potential to involve students in the CTD’s Civic Leadership Institute, a 3-week summer service-learning program for high school students that combines hands-on education and service related to addressing significant societal problems.
The Role of Gifted Programming in Closing the Achievement Gap
As a field, gifted education has not been perceived as having anything to contribute to closing achievement gaps, and in some cases, has been accused of contributing to them (Sapon-Shevin, 1994). This longitudinal study of Project Excite adds to the growing research literature on the role of gifted programming and practices in addressing achievement and excellence gaps. Recent research has shown that advanced curriculum designed specifically for gifted learners can be used successfully with a broad range of students in Title 1 schools when implemented with appropriate scaffolding strategies (Bland, Coxon, Chandler, & VanTassel-Baska, 2010; Gavin, Casa, Adelson, Carroll, & Sheffield, 2009; Horn, 2015; Reis, McCoach, Little, Muller, & Kaniskan, 2011; VanTassel-Baska, Bracken, Brown, & Feng, 2005, 2009). Achievement gains for students as a result of access to advanced, challenging curricula enable more students to qualify for gifted programs and reach higher levels of achievement commensurate with their abilities. Gifted education specialists have increasingly come to realize the role that problem-based learning, enrichment, and other gifted education strategies can play in addressing broad educational issues.
Limitations
Project Excite was conceived as an outreach project designed to address the achievement gap of a large, racially and socioeconomically diverse suburban school system. Its origins included a concerted effort on the part of administrators from both the elementary and high school districts and faculty at Northwestern University to take action to address a major, complex problem. Most of the effort and all of the funds were used to implement the program. Data were collected as regularly and as systematically as possible to better inform the design of the Excite program, given the many changes to personnel within the districts and changing criteria for accelerated math programs over the course of 14 years. The educational component of Project Excite was tweaked and refined as a result. However, basic research was not a primary focus of this collaborative effort. Therefore, a major limitation of this study is that it lacks a true comparison group and it relied on comparing the achievement data of Project Excite students with their peers in their local school, school districts, or the State of Illinois.
Second, we lacked disaggregated student achievement data for the comparison group and only had mean or median scores reported on school or districts achievement reports. As a result, data analyses were limited to median equality tests and computation of effect sizes assuming equal variance of two comparing groups. Third, there were no true baseline data to allow us to clearly assess whether Project Excite students were equivalent to their comparing peers prior to the program. We thus tentatively used the third-grade ISAT math and reading scores as the baseline data. Fourth, we were not be able to collect any achievement test scores for Project Excite students during high school since they exited from the program after eighth grade. As a result, we could not assess their actual learning performance during this stage. Last, there were only descriptive statistics for Project Excite students’ college placement because no such data for the comparison students were publically accessible.
It is relevant to note that, despite these limitations, this study was able to use multiple outcome data including standardized achievement test scores (i.e., ISAT math and reading scores, Explore math, reading, science, English, and composite scores, MAP scores), ninth-grade math placement, and college placement information. Most important, results of our analyses were fairly consistent across the scores of the three different standardized achievement tests. Furthermore, all the data points, taken collectively, inform us about the outcomes of Project Excite.
Directions for Future Research
The current study points to several possible directions for future research on Project Excite and other similar programs for high-potential CLED students. First, research is needed to focus more specifically on the various components of intervention programs such as Project Excite. What components or processes are especially critical and effective? Although several studies have been conducted on Project Excite (e.g., Lee et al., 2009, 2010), the role and efficacy of particular program components has not been studied to date. Second, more research is needed on the processes underlying the impact of the program. For example, does identification for a special program galvanize parents and teachers to raise expectations thereby affecting student achievement? Does success in a challenging academic program involving other talented peers significantly affect the self-concepts, expectations, and academic-related decisions of students and families so that they continue to seek advanced courses and programs? Does support from a wide ranging set of peers (e.g., beyond one’s home, school, district, or country) reinforce motivation and commitment to achievement and help students better benchmark their progress? Does access to “near-peers” cultivate motivation and perseverance? Does contact with exceptional teachers increase interest and thus motivation to study? Do challenging yet supportive supplementary programs engender growth mind-sets and other beliefs conducive to high achievement? Understanding how programs such as Project Excite influence students and families would require longer term studies that include measurement of other psychological constructs such as self-concept, social support, sense of belonging, and others. Such research would greatly inform the design of future interventions aimed at achievement gaps.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.