Hidden Progress of Multilingual Students on NAEP

Abstract

Using National Assessment of Educational Progress data from 2003 to 2015, this brief describes changes in the reading and mathematics performance of multilingual students—defined as students who report a primary home language or languages other than English. Although all students’ scores improved, multilingual students’ scores improved two to three times more than monolingual students’ scores in both subjects in Grades 4 and 8. There was little evidence that these trends were explained by cohort changes in racial/ethnic, socioeconomic, or regional composition. These promising trends are obscured when researchers and policymakers focus only on scores for students currently classified as English learners.

Keywords

achievement gap bilingual/bicultural correlational analysis educational policy NAEP regression analyses secondary data analysis

For many years, media reports and introductions to research articles have noted the persistently large achievement gaps between English learners (ELs) and their monolingual peers on state and national tests (e.g., Carnoy & García, 2017; Maxwell, 2013). For instance, a recent report on key trends on the National Assessment of Educational Progress (NAEP) highlighted “the stall of non-English speakers” over the past 15 years (Carnoy & García, 2017, p. 1). This implies that the U.S. educational system has made little progress in meeting ELs’ needs. Another (perhaps unintended) implication is that ELs’ language challenges set them up for academic failure regardless of the education they receive, an interpretation that reinforces deficit thinking about these learners’ potential (e.g., Hakuta, 2011).

However, researchers have recently argued that focusing on the scores of only those students currently classified as ELs can produce misleading conclusions about whether educational systems are getting better or worse in serving these learners (e.g., Hopkins, Thompson, Linquanti, Hakuta, & August, 2013; Kieffer, Lesaux, & Snow, 2007; Saunders & Marcelletti, 2013). Specifically, researchers have demonstrated how the difference between ELs and non-ELs in content-area test performance is a “gap that can’t go away” (Saunders & Marcelletti, 2013). By definition, ELs are not yet proficient in listening, speaking, reading, or writing English, and their language skills impact their performance on content-area assessments administered in English (Hopkins et al., 2013). However, once students attain English proficiency and are thus able to perform at higher levels on these content-area tests, students exit the EL subgroup.

As shown in Figure 1, current ELs are a subgroup of the broader population of multilingual students, which also includes former ELs (i.e., students who have reached standards for English proficiency) and “never EL” multilingual students (i.e., students who speak another language or languages at home but are classified as proficient in English when entering school).¹ Researchers have also referred to this same broader population as language minority learners (Kieffer et al., 2007). As multilingual students move up the grades, more students attain English proficiency and move into the former EL category; recent estimates from different states suggest that about 25% to 50% of students who enter kindergarten as ELs have been reclassified before Grade 4 and 70% to 85% before Grade 8 (Kieffer & Parker, 2016; Thompson, 2015; see also Conger, 2009). Thus, the current EL group in these grades consists of those students who—by definition—have relatively low English test scores, because either they have recently immigrated to the United States or they have struggled to acquire English language and literacy skills in U.S. schools. Prior research shows that even in states with large numbers of immigrants, students who have been classified as ELs in U.S. schools for many years far outnumber recent immigrants, especially at the secondary level (Olsen, 2010). If U.S. schools are getting much better at serving multilingual students, it is likely that more students will be reclassified, but it is unlikely that the test scores of those who remain as current ELs will change. For instance, if schools make major improvements in serving EL students in the primary grades, leading to more students becoming English proficient and academically successful by Grade 4, those gains will not show up in Grade 4 NAEP trends for ELs (because the students who have attained English proficiency will no longer be counted as ELs).

Figure 1.

Diagram illustrating how the population of multilingual students changes as increasing numbers of current English learners become former English learners.

To address this problem, researchers have recommended tracking the performance of the stable “ever EL” group, which includes both current and former ELs (i.e., the dashed-line ovals in Figure 1), (Thompson, Rew, Martinez, & Clinton, 2017). While useful for state data, this approach is not currently possible with NAEP scores,² but NAEP scores can be disaggregated for multilingual learners. Data suggest that current and former ELs comprise the substantial majority of multilingual learners, while never ELs constitute a small minority.³ Thus, by analyzing multilingual learners’ performance on NAEP, we are primarily (if not exclusively) analyzing the performance of current and former ELs. The smaller group of never ELs include second- or third-generation immigrants and recent immigrants from countries where multiple languages including English are commonly spoken (e.g., former colonized countries such as India, the Philippines, Nigeria, and Kenya or European countries such as Sweden or the Netherlands).

Focusing on the broader population of multilingual students (i.e., the broadest circles in Figure 1) also has the advantage of aligning more closely with how we disaggregate data to examine other achievement gaps, including by race/ethnicity or income. Students are initially classified as ELs after scoring below an established threshold on an English language proficiency assessment. Thus, the present system of analyzing achievement only for students currently classified as ELs is analogous to focusing only on the achievement of Black students who perform below proficiency in the current year or at some grade in the past rather than the entire demographic group that shares the disadvantages of historic oppression and exclusion.

An additional reason for analyzing outcomes for the broad group of multilingual learners is because of differences across states and districts in the home language survey questions and initial English language proficiency (ELP) assessments used to identify ELs at school entry and differences in the assessments and criteria used to exit students from EL services. Given these differences, the current, former, and never EL groups are defined differently in different locations (e.g., Linquanti, Cook, Bailey, & MacDonald, 2016). Focusing on outcomes for the broader multilingual learner group increases the comparability of the subgroup across locations.

NAEP allows researchers to use student-report data on home language use to estimate the achievement of the broader population of multilingual students (i.e., the broadest circles in Figure 1). By evaluating trends in achievement over time for all multilingual students—not only those currently classified as ELs—we will be better equipped to evaluate efforts to promote equity in educational outcomes across language differences defined broadly.

Current Study

This study uses straightforward and simple descriptive analyses of publicly available NAEP data to explore whether and how much U.S. multilingual students’ achievement improved between 2003 and 2015. Multilingual students were identified based on students reporting that “people in their home talk to each other in a language other than English . . . most or all of the time” (NCES, n.d.)—regardless of whether they fell into the current EL, former EL, or never EL categories. Monolingual students were identified as students who reported that people in their home never spoke a language other than English at home. (For sample descriptives, see Supporting Online Materials [SOM] Table S1–S3 available on the journal website.) Multilingual and monolingual students’ achievement in mathematics and reading was compared in Grades 4 and 8 in each NAEP testing year. Supporting analyses investigated whether trends found could be explained by differences between cohorts in their socioeconomic, racial/ethnic, and regional composition. Analyses were conducted using the NCES Data Explorer (available at http://nces.ed.gov/nationsreportcard/naepdata/) to account for features of the NAEP data set and facilitate easy replication and application of this approach to future data for the United States, specific states, and large city districts.

Results

Between 2003 and 2015, NAEP achievement differences between monolingual and multilingual students have narrowed by 24% and 27% in reading and 37% and 39% in math, in Grades 4 and 8 respectively, as shown in Figure 2 (all ps < .001). When converted into grade equivalents, these changes indicate that multilingual students are about one-third to one-half of a grade level closer to their monolingual peers in 2015 than they were in 2003 (see SOM Table S4 available on the journal website). Although monolinguals’ scores have increased significantly over time, multilingual students’ scores have increased substantially more across both grades and subjects—nearly twice as much in Grade 4 in both reading and math, more than three times as much in Grade 8 reading, and more than twice as much in Grade 8 math (Figure 2). Across each test administration, multilingual students’ scores were consistently increasing or stable, with the exception of Grade 8 scores between 2013 and 2015. (See SOM Tables S4 and S5, available on the journal website, and related text for additional details.)

Figure 2.

Changes in National Assessment of Educational Progress (NAEP) scores in fourth- and eighth-grade reading and mathematics for monolingual and multilingual students from 2003 to 2015, based on authors’ calculations using data available at http://nces.ed.gov/nationsreportcard/naepdata/.

To shed light on how this progress can be obscured when focusing only on the performance of the subset of current ELs, we also investigated the trends of current EL scores as contrasted with those of multilingual and monolingual students. As shown in Figure 3, scores for the subgroup of current ELs demonstrated small and inconsistent increases in each grade and subject, but these were all substantially smaller than increases for all multilingual learners and similar to or smaller than increases for monolinguals (for additional details, see SOM Tables S6 and S7 available on the journal website).

Figure 3.

Changes in National Assessment of Educational Progress (NAEP) scores in fourth- and eighth-grade reading and mathematics for current English learners, as contrasted with monolingual and multilingual students, from 2003 to 2015, based on authors’ calculations using data available at http://nces.ed.gov/nationsreportcard/naepdata/

Because NAEP is cross-sectional, changes in the multilingual-monolingual achievement differences may be due to changes in the composition of one or both groups across cohorts. Although all such cohort effects cannot be captured, multiple regression analyses were used to investigate whether the trends found were explained by confounding differences in racial/ethnic composition and socioeconomic status (as indicated by national school lunch program eligibility for both grades and student-reported parental education for Grade 8). As shown in Table 1, when controlling for these factors, achievement differences were much smaller, but trends over time were similar. The monolingual-multilingual difference declined notably over time even among students from similar backgrounds, as indicated by changes in adjusted standardized differences between 2003 and 2015 (.06 to –.13 SDs) demonstrating similar magnitudes to unadjusted standardized differences (–.10 to –.15 SDs). Results were also similar when controlling for region of the country. (For additional details, see SOM Table S8 available on the journal website.)

Table 1

Standardized Differences in National Assessment of Educational Progress (NAEP) Scores Between Monolingual and Multilingual Students

		Unadjusted		Adjusted
		Standardized Difference Between Monolingual and Multilingual Students	Change in Standardized Differences Since 2003	Standardized Difference Between Monolingual and Multilingual Students	Change in Standardized Differences Since 2003
Grade 4 reading	2003	0.45***		0.11***
	2005	0.40***	−0.05	0.05***	−0.06
	2007	0.40***	−0.05	0.05**	−0.06
	2009	0.40***	−0.05	0.06***	−0.04
	2011	0.39***	−0.06	0.07***	−0.04
	2013	0.37***	−0.08	0.03*	−0.08
	2015	0.34***	−0.10	0.04*	−0.06
Grade 8 reading	2003	0.47***		0.12***
	2005	0.43***	−0.04	0.09***	−0.03
	2007	0.47***	−0.01	0.11***	−0.01
	2009	0.42***	−0.05	0.04*	−0.08
	2011	0.38***	−0.09	0.04	−0.08
	2013	0.39***	−0.09	0.04*	−0.08
	2015	0.34***	−0.13	0.001	−0.12
Grade 4 mathematics	2003	0.42***		0.10*
	2005	0.36***	−0.06	0.03*	−0.07
	2007	0.36***	−0.06	−0.002	−0.10
	2009	0.35***	−0.07	0.01	−0.09
	2011	0.34***	−0.08	0.03*	−0.07
	2013	0.32***	−0.10	0.02	−0.08
	2015	0.27***	−0.15	−0.02	−0.12
Grade 8 mathematics	2003	0.43***		0.10*
	2005	0.39***	−0.03	0.08*	−0.02
	2007	0.38***	−0.05	0.02	−0.08
	2009	0.38***	−0.05	0.01	−0.09
	2011	0.34***	−0.09	0.001	−0.10
	2013	0.33***	−0.10	−0.005	−0.11
	2015	0.28***	−0.15	−0.03	−0.13

Note. Scores are unadjusted and adjusted for race/ethnicity and free-reduced lunch status in Grade 4 and adjusted for race/ethnicity, free-reduced lunch status, and student-reported parental education level in Grade 8, and change in standardized differences since 2003.

p < .05. **p < .01. *** p < .001.

Discussion and Conclusions

Contrary to the dominant perception that multilingual students have demonstrated little academic progress in recent years, our findings indicate that this population demonstrated substantially better reading and math achievement in 2015 than 2003. Multilingual students’ scores have improved two to three times as much as monolingual students’ scores, narrowing the gap by 24% to 39% (or one-third to one-half of a grade level), depending on grade and subject. When contrasted with the flat trends in current ELs’ scores (Figure 3; Carnoy & García, 2017), these results also illustrate how focusing exclusively on the current EL subgroup can obscure the progress that educational systems make in moving students toward English proficiency and higher levels of academic achievement (Hopkins et al., 2013; Thompson et al., 2017). In addition, by using a simple approach that can be implemented with the user-friendly and publicly available NAEP Data Explorer, we provide a model for how states and large districts can investigate their own trends for multilingual learners.

Although these data indicate that multilingual students are achieving at higher levels now than in the past, they shed little light on why this is the case. Between 2003 and 2015, a wide variety of changes in policies and practices affecting multilingual students have been implemented. The time period studied corresponds to the era of No Child Left Behind (NCLB), with a myriad of associated changes in accountability and instruction that likely affected multilingual students. Indeed, research suggests that one of the most commonly reported effects of NCLB implementation was increased attention to the performance and needs of ELs (e.g., Jennings & Rentner, 2006). However, research also suggests many potentially negative effects of NCLB on ELs’ achievement, such as inappropriate use of assessments and incentivizing the exclusion of low-scoring students (e.g., Darling-Hammond, 2007). Other observed impacts, such as narrowing of the curriculum to the detriment of science, social studies, the arts, and social-emotional learning (Darling-Hammond, 2007), may have constrained students’ educational opportunities without negatively affecting reading and math scores on a test in English. Meanwhile, a variety of policy and practice changes at the state and district levels may also be associated with closing of the achievement gap for multilingual students. For example, during these years, dual language immersion programs, which have shown positive effects on content-area achievement for ELs (Steele et al., 2017; Valentino & Reardon, 2015), expanded rapidly in many states (U.S. Department of Education, 2015). In addition, multiple states expanded certification requirements for teachers of ELs (Education Commission of the States, 2014), and this additional preparation may have better prepared teachers to meet multilingual learners’ needs (Loeb, Soland, & Fox, 2014; Master, Loeb, Whitney, & Wyckoff, 2016).

Future research that directly evaluates the relationship between such policy and practice changes and multilingual student achievement is needed. Nonetheless, the substantial recent progress of multilingual students demonstrated here suggests that the bundle of various changes that occurred have together been more beneficial than harmful. As schools begin to implement the Every Student Succeeds Act, evaluating new policy changes affecting multilingual students will be essential. This study also sheds light on important limitations in NAEP. NAEP variables did not allow us to describe trends specifically for the ever EL subgroup—the stable group of both current and former ELs (see the dashed-line ovals in Figure 1)—to capture the effects of EL policies and practices more precisely (Thompson et al., 2017). In addition, the multilingual students did not include all current ELs because a small portion reported speaking a language other than English less than “most or all of the time” (see SOM Table S1 and related text, available on the journal website). NAEP would be substantially improved by collecting information on students’ former EL status and reporting an aggregated ever EL category. This is not common practice in many states and districts so may require local changes to data management in addition to changes to NAEP. Despite these limitations, by documenting the substantial recent progress of multilingual students, this study challenges the dominant storyline emphasizing the underachievement of these learners and the failures of educational systems to meet their needs.

Supplemental Material

EdR777740_Online_Supplement – Supplemental material for Hidden Progress of Multilingual Students on NAEP

Supplemental material, EdR777740_Online_Supplement for Hidden Progress of Multilingual Students on NAEP by Michael J. Kieffer and Karen D. Thompson in Educational Researcher

Footnotes

Notes

ORCID iD

Michael J. Kieffer

Authors

MICHAEL J. KIEFFER, EdD, is an associate professor at New York University Steinhardt School of Culture, Education, and Human Development, 239 Greene Street, New York, NY 10008; michael.kieffer@nyu.edu . His research focuses on the language and literacy development of students from linguistically diverse backgrounds.

KAREN D. THOMPSON, PhD, is an assistant professor in the College of Education at Oregon State University, 301B Furman Hall, Corvallis, OR 97331; karen.thompson@oregonstate.edu . Her research explores how policy, curriculum and instruction, and teacher education intersect to shape the classroom experiences of multilingual students in K–12 schools.

References

Carnoy

García

(2017). Five key trends in U.S. student performance. Washington, DC: Economic Policy Institute. Retrieved from http://www.epi.org/files/pdf/113217.pdf

Conger

(2009). Testing, time limits, and English learners: Does age of school entry affect how quickly students can learn English? Social Science Research, 38, 383–396.

Darling-Hammond

(2007). Race, inequality and educational accountability: The irony of “No Child Left Behind.” Race Ethnicity and Education, 10(3), 245–260.

Education Commission of the States. (2014). 50-state comparison: What ELL training, if any, is required for general classroom teachers? Retrieved from http://ecs.force.com/mbdata/mbquestNB2?rep=ELL1415

Jennings

Rentner

D. S.

(2006). Ten big effects of the No Child Left Behind Act on public schools. Phi Delta Kappan, 88(2), 110–113.

Hakuta

(2011). Educating language minority students and affirming their equal rights research and practical perspectives. Educational Researcher, 40(4), 163–174.

Hopkins

Thompson

K. D.

Linquanti

Hakuta

August

. (2013). Fully accounting for English learner performance: A key issue in ESEA reauthorization. Educational Researcher, 42(2), 101–108.

Kieffer

M. J.

Lesaux

N. K.

Snow

C. E.

(2007). Promises and pitfalls: Implications of No Child Left Behind for identifying, assessing, and educating English language learners. In Sunderman

(Ed.), Holding NCLB accountable: Achieving accountability, equity, and school reform (pp. 57–74). Thousand Oaks, CA: Corwin Press.

Kieffer

M. J.

Parker

C. E.

(2016). Patterns of English learner student reclassification in New York City public schools (REL 2017–200). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands.

10.

Linquanti

Cook

Bailey

MacDonald

(2016). Moving toward a more common definition of English learner: Collected guidance for states and multi-state assessment consortia. Washington, DC: Council of Chief State School Officers.

11.

Loeb

Soland

Fox

(2014). Is a good teacher a good teacher for all? Comparing value-added of teachers with their English learners and non-English learners. Educational Evaluation and Policy Analysis, 36(4), 457–475.

12.

Master

Loeb

Whitney

Wyckoff

(2016). Different skills? Identifying differentially effective teachers of English lan guage learners. The Elementary School Journal, 117(2), 261–284.

13.

Maxwell

L. A.

(2013, November 11). ELLs and nation’s report card: No change in reading and math performance. Education Week. Retrieved from http://blogs.edweek.org/edweek/learning-the-language/2013/11/ells_and_nations_report_card_n.html

14.

National Center for Educational Statistics. (2016). Table 204.27. English language learner (ELL) students enrolled in public elementary and secondary schools, by grade, home language, and selected student characteristics: Selected years, 2008–09 through fall 2014. Retrieved from https://nces.ed.gov/programs/digest/d16/tables/dt16_204.27.asp

15.

National Center for Educational Statistics. (n.d.). NAEP Data Explorer. Retrieved from https://nces.ed.gov/nationsreportcard/naepdata/dataset.aspx.

16.

Olsen

(2010). Reparable harm: Fulfilling the unkept promise of educational opportunity for California’s long term English learners. Long Beach, CA: Californians Together.

17.

Saunders

W. M.

Marcelletti

D. J.

(2013). The gap that can’t go away: The catch-22 of reclassification in monitoring the progress of English learners. Educational Evaluation and Policy Analysis, 35(2), 139–156.

18.

Steele

J. L.

Slater

R. O.

Zamarro

Miller

Burkhauser

Bacon

(2017). Effects of dual-language immersion programs on student achievement: Evidence from lottery data. American Educational Research Journal, 54(Suppl. 1), 282S–306S.

19.

Thompson

K. D.

(2015). English learners’ time to reclassification: An analysis. Educational Policy, 31, 330–363.

20.

Thompson

K. D.

Rew

Martinez

M. I.

Clinton

(2017, June 16). Understanding student outcomes by using the “Ever English learner” category. Retrieved from https://ies.ed.gov/blogs/research/post/understanding-outcomes-for-english-learners-the-importance-of-the-ever-learner-category

21.

U.S. Census Bureau. (2015). Characteristics of people by language spoken at home, 2011–2015 American Community Survey 5-year estimates. Retrieved from https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_15_5YR_S1603&prodType=table

22.

U.S. Department of Education, Office of English Language Acquisition. (2015). Dual language education programs: Current state policies and practices. Washington, DC: Author.

23.

Valentino

R. A.

Reardon

S. F.

(2015). Effectiveness of four instructional programs designed to serve English learners: Variation by ethnicity and initial English proficiency. Educational Evaluation and Policy Analysis, 37(4), 612–637.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB