Abstract
As state and federal policymakers continue to adopt more centralized policies, it is increasingly important to understand how policies, particularly those designed to enhance education for underserved students, are implemented locally. We employ a mixed-methods sequential explanatory design to investigate the implementation of one policy, that which guides the process of reclassifying English learners (ELs) as English proficient in Texas. First, we use event history analysis to determine whether the likelihood of being reclassified differs across the state for similar ELs. Second, we utilize interview and observation data from eight schools to unpack how practitioners implement reclassification policy. We find differences in the hazard rate of reclassification across the state, which is linked to practitioners’ understanding of the policy.
On the surface, it seems that standardization efforts may lead to more uniform practices. However, extant research around education policy implementation indicates that the loose coupling inherent in the American system of federalism affects how local policy implementers such as school administrators and teachers adopt and adapt policy handed down to them from state and federal governments (Weick, 1977). This allows local policy implementers to enact realities that they can comprehend, and, oftentimes, these local comprehensions differ from those of state and federal policymakers (Fuhrman, Clune, & Elmore, 1991; Marsh & Odden, 1991; McDonnell, 1991; Porter, Fusarelli, & Fusarelli, 2015; Werts & Brewer, 2015). Drawing upon this and other research, a newfound interest has emerged related to the role and influence of the locality in American public education (Crowson & Goldring, 2009).
Although many state and federal policies once tried to tighten the loosely coupled system by implementing stricter sets of state and federal controls, more recent policy reform efforts have recognized local policy implementers as partners in national- and state-level education reform. Thus, the locality has, in many ways, become the front-line decision maker, shaping, massaging, and adjusting policies to reflect the local context while reflecting national and state agendas, priorities, and goals. In essence, this movement has afforded LEAs the freedom to implement these policies and meet central education goals in the way they deem most suitable for the surrounding community. However, it remains unclear whether individuals at the local level possess the necessary knowledge, expertise, and capacity to effectively interpret and implement state and federal policy on the ground. This is particularly concerning when considering policies that address new issues unfamiliar to LEAs. For example, as demographics across the nation have shifted, LEAs have found themselves implementing education policies that seek to address the intersection of three very different issues: poverty, educational equity, and English language acquisition.
Take, for example, federal policy such as the No Child Left Behind Act (NCLB, 2002) that prompted states to identify and annually assess ELs to monitor their progress toward acquiring English proficiency. Although NCLB required states to carry out these actions, it was left to the discretion of the states to determine how these requirements would be met. In response, some states enacted policies that guided LEAs’ annual assessment of EL students and set specific reclassification criteria that applied to all districts, whereas others opted to allow districts to determine their own reclassification criteria. With the recent enactment of the Every Student Succeeds Act (ESSA, 2015), however, each state is required to standardize its criteria for identifying and reclassifying EL students. Despite this push for standardization, local decision making will likely still factor heavily into reclassification decisions. For example, if states indicate that classroom teachers’ evaluations of EL student performance should factor into the reclassification decision-making process, it remains unclear how much weight teachers’ assessments should carry. Thus, there is evidence of both an increase in the centralization of policies and a prominent role that LEAs play in policy implementation. Yet, as previously mentioned, it is unclear whether individuals at the local level possess the necessary knowledge and expertise to effectively make these decisions in a way that maximizes ELs’ educational opportunities and outcomes.
As policymakers at the federal and state levels continue to simultaneously adopt more centralized policies (e.g., educator evaluation systems, school accountability systems) and embrace the role of the locality in shaping policy to its context, it is increasingly important to better understand how policies are implemented at the local level. We argue that this is particularly important for policies that seek to expand educational opportunities for historically marginalized and underserved groups of students.
Through the study of the implementation of one statewide policy—that which guides the process of exiting students from EL status and reclassifying them as English proficient in Texas—this research investigates the following questions:
This article makes several key contributions. First, we shed light on the reclassification process itself. Prior research has provided convincing evidence that EL reclassification is a critical juncture in a student’s educational trajectory (e.g., Callahan, 2005; E. Flores, Painter, & Pachon, 2009; Parrish, Merickel, Perez, & Linquanti, 2006; Silver, Saunders, & Zarate, 2008). However, very few studies have assessed variation in this process across different contexts (Cimpian, Thompson, & Makowski, in press; Estrada & Wang, 2013; L. E. Hill, Weston, & Hayes, 2014). Moreover, even less is known about how educators implement reclassification policy on the ground and how the reclassification decision-making process may vary across different contexts. As a second contribution, we provide deeper insight into the front lines of the policy implementation process by unpacking the variation that exists in how school-level policy implementers understand and implement policy.
Theoretical Framework and Empirical Evidence
Drawing upon rational choice theory, many conventional accounts of how local actors respond to policy assume that local actors understand policy messages as policymakers intend and choose rationally between following or ignoring them (Spillane, 2009). However, research has shown that this is simply not the case: Policy is not normative and, although educators frequently adhere to and work hard to implement higher level policies (e.g., Firestone, Fitz, & Broadfoot, 1999; H. C. Hill, 2001), implementation does not always proceed in ways consistent with original intentions (e.g., Marsh & Odden, 1991; McDonnell, 1991; Porter et al., 2015; Werts & Brewer, 2015).
Although policy implementers may believe they are implementing the law as originally intended, oftentimes, due to competing demands as well as personal and institutional constraints, they tend to focus on technical aspects of implementation (e.g., Ball, 1990; Cohen, 1990; Weimers, 1990; Wilson, 1990). Moreover, elements such as policy implementers’ knowledge, skills, backgrounds, motivation, will, and problem framing, as well as institutional contexts such as size, capacity, competing priorities, and institutional leadership influence the policy implementation process (e.g., Coburn, 2001; Cohen, 1990; McLaughlin & Elmore, 1982; Spillane, 1996, 1998, 2000). As a result, many policy implementers engage in letter of the law implementation, which equates to literal, strict implementation of what is formally written, irrespective of mitigating contextual factors (Garcia, Chen, & Gordon, 2014). Because policymakers do not always know or understand the general meaning, purpose, and intent of a law, as opposed to its literal content, the spirit of the law—that is, the “general purpose and intent of the law” (Garner, 2009) and the “social and moral census of the interpretation of the letter of law” (Gordon, 2011, p. 4)—and the underlying goals of the policy often do not materialize during implementation.
Research suggests that policy outcomes are profoundly influenced by education practitioners, often termed “street-level bureaucrats” (Lipsky, 1969), who often lack adequate resources to meet ambitious policy expectations but who, nonetheless, are able to devise solutions to satisfy competing policy demands. For example, Texas policymakers may expect a statewide EL reclassification policy to result in equitable reclassification of ELs based on the same standards across the state. Such an accomplishment—equitable reclassification—requires extensive investment of time and effort by educators at the state, regional, and local level as well as by students. As Lipsky (1980) contended, however, educators often lack the resources, time, technology, skills, knowledge, and/or expertise necessary to accomplish such policy goals. As a result, educators often adopt practices that they believe are appropriate within their context (e.g., Marsh & Odden, 1991; McDonnell, 1991; Porter et al., 2015; Werts & Brewer, 2015).
Researchers have also drawn upon cognitive theory to show that street-level bureaucrats often mediate the way in which policy is implemented (Cohen & Weiss, 1993; Spillane, Reiser, & Reimer, 2002). For example, Spillane and Zeuli (1999) examined how teachers came to understand and implement a policy that reformed local mathematics curriculum standards and found that the majority of teachers interpreted new curriculum standards through the lens of their current practice; thus, similar to Lipsky’s (1980) street-level bureaucrats, the understanding these teachers constructed did not necessarily reflect the fundamental changes in practices pressed by reformers.
Similarly, the Educational Policy and Practice Study’s (EPPS) research on mathematics and language arts standards in California, Michigan, and South Carolina found that local school teachers who encountered the same policy texts constructed different understandings of the policy’s message (Cohen & Barnes, 1993; Jennings, 1992; Spillane, 1996, 1998). This research also emphasized the power of local actors and their ability to construct different understandings of policy through a process by which they draw upon what they already believe (Spillane, Reiser, & Gomez, 2006).
Each of the previously described studies contributed to the body of education policy implementation research that supports Lipsky’s (1969) concept of street-level bureaucracy. They also provided substantial insight into the policy implementation process. However, much of this research has focused on how teachers understand and implement education reform (e.g., a new curriculum) in their classroom. Very little research has examined how reforms that aim to provide equitable educational opportunities for underserved students are understood and implemented by teachers, administrators, and parents outside the classroom. As the EL student population continues to grow, it will be important to understand the unique needs of these students and how districts, schools, and educators interact with policies that affect ELs. Moreover, little research has examined the ways in which a group of individuals, rather than a single person, coconstructs an understanding of and implements an education policy. Although some research has surfaced around school-level group decisions related to special education placement (e.g., Mehan, Hertweck, & Meihls, 1986), this research was conducted before the current high-stakes accountability environment surfaced. Thus, this research, which examines a group’s interpretation and implementation of EL reclassification policy, will contribute to a more current discussion of organizational-level policy interpretation and implementation.
Background: EL Reclassification
ELs are students, whose difficulties in speaking, reading, writing, or understanding the English language may be sufficient to deny the individual the ability to meet the State’s proficient level of achievement on State assessments, the ability to successfully achieve in classrooms where the language of instruction is English, the opportunity to participate fully in society. (ESSA, 2015)
One of the primary goals of any program that serves ELs is for students to acquire a sufficient level of English proficiency such that they no longer require language supports. Title III of NCLB required all school districts that receive federal funds to submit an annual evaluation describing the progress that ELs are making in acquiring English proficiency. ESSA further requires that states build English proficiency into their accountability systems under Title I, which intensifies pressure to ensure ELs acquire English proficiency. One of the most common educational milestones used to measure ELs’ progress is reclassification (Linquanti, 2001; Robinson, 2011; Thompson, 2015; Umansky & Reardon, 2014). Reclassification is a landmark event for EL students because it signifies a shift in educational experiences. In addition to the withdrawal of English language development services, reclassified students’ “[t]eachers change, peers change, course content changes, instructional techniques change, access to resources changes, and assessment changes” (Umansky, 2012, p. 31).
Increasing attention has been paid to the risks and benefits associated with being classified as an EL and being reclassified as English proficient. Researchers have shown that reclassifying an EL student too late may have detrimental effects such as higher dropout rates, restricted access to honors and college preparatory coursework, decreased rates of college enrollment, and a greater likelihood of needing remedial coursework in college (Callahan, 2005; Cummins, 1980, 1981; Estrada, 2014; S. M. Flores & Drake, 2014; Harklau, 2002; Kanno & Kangas, 2014; Parrish et al., 2006; Silver et al., 2008). Conversely, if ELs are reclassified prematurely when they are still in need of language supports, they are similarly at risk of academic failure (Linquanti, 2001; Robinson, 2011). Thus, the time at which an EL is reclassified is undoubtedly a critical juncture in a student’s academic trajectory.
Under NCLB, decisions regarding how to determine whether a student is ready to exit EL status were left up to states. As such, there is substantial variability in terms of how states go about reclassifying students (Kim & Herman, 2009). As Abedi (2008) noted, one would expect that a student who is classified as an EL in one state would carry the same classification in another state; however, this is not always the case. This variation may be due to differences in EL reclassification requirements (Linquanti, 2001; Robinson, 2011). For example, Kim and Herman (2009) examined reclassification standards across three states, and found that there was variability in the stringency of these standards. Within-state variation has also been examined. Using data from California, Estrada and Wang (2013) compared the reclassification criteria used in two case study districts, and L. E. Hill et al. (2014) surveyed districts across one state about their reclassification criteria. Both found that the districts that layered on additional reclassification criteria beyond objective state assessments (e.g., teacher recommendations and writing samples) and/or set higher assessment score requirements resulted in lower rates of reclassification. Employing data from two unnamed states, Cimpian et al. (in press) found considerable variability in reclassification criteria across districts within each state and used regression discontinuity design to gauge the effect of different district reclassification policies on subsequent achievement and graduation outcomes.
Although these studies make important contributions to the literature on the variability of reclassification policies, they do not speak to variation in how policy is enacted. There is no extant research that actually observes educators implementing reclassification policy and making reclassification decisions in schools. This is an important gap in the literature, particularly given the fact that ESSA is requiring states to design policy that standardizes their reclassification criteria. Given this new requirement, there will likely be a decline in within-state reclassification criteria variation. However, we assert that even as states centralize their reclassification requirements, we will continue to see variation within states because of how educators on the front lines go about implementing reclassification policy. Thus, it will be increasingly important to document the differences in reclassification policy implementation, and unpack what shapes uneven implementation. To help close this gap, we turn to Texas, a state that has had longstanding statewide reclassification policy.
Context of the Study: Texas
Texas is second only to California in terms of the number and proportion of ELs enrolled in public schools in the state (NCES, 2014). The percentage of EL students enrolled in Texas more than doubled between 1979 and 2005 (United States v. State of Texas, 2008). As of the 2014–2015 school year, 18.1% of students in Texas were identified as ELs (Texas Education Agency [TEA], 2016).
Since 1981, state legislation has required that every district in Texas that serves EL students establish a Language Proficiency Assessment Committee (LPAC) to annually review and monitor the progress of EL students’ English proficiency and academic achievement and decide when to ultimately reclassify them as English proficient (Texas Commissioner’s Rules, §89.1220, 1996). The LPAC is composed of a bilingual educator, a transitional language educator (such as a bilingual or English as a second language [ESL] teacher), a parent of an EL (who is not an employee of the school district), and a campus administrator (Texas Education Code [TEC], §29.063, 1996). In addition, it is the district’s and/or school’s prerogative to add other members to the committee beyond those required by the state. Individual teachers of students being considered for reclassification may or may not be in the committee. Although other members of school staff (classroom teachers, aides, etc.) may help to provide information on EL students’ academic and English proficiency progress, key decisions regarding EL reclassification are ultimately made by the members of the LPAC.
Under the letter of the law—that is, “the formal code, rule, regulation or principle that must be followed according to governmental mandates or policies” (Gordon, 2011, p. 4)—Texas LPACs must convene each spring to review the files of EL students to determine whether students are ready to exit EL status and be reclassified. Although state law mandates LPACs monitor all EL students, the impetus behind the LPAC goes much deeper, extending “beyond the responsibilities established under [state policy]” (TEA, 2015). The spirit of the law—again, the “social and moral census of the interpretation of the letter of law” (Gordon, 2011, p. 4)—as enumerated by the TEA (2015) is that, As an advocate for the ELL, the LPAC becomes the voice that initiates, articulates, deliberates, and determines the best instructional program for the student. It functions as a link between the home and the school in making appropriate decisions regarding placement, instructional practices, assessment, and special programs that impact the student. (p. 7)
When making reclassification decisions, Texas LPACs are required to take into account performance on a series of English language proficiency and academic assessments. Although the state English language arts assessment must be taken into account for students in all grades and subjects for which the test is administered, the English proficiency assessments used for reclassification decisions are selected at the district level from a list of state-approved tests that measure listening, speaking, reading, and writing. 1 Although English proficiency assessment performance criteria for reclassification are explicit (e.g., “satisfactory” level on the district-selected English proficiency assessment, “proficient” or “highly proficient” level on the state English language arts assessment), there is little guidance as to the weight that these objective evaluations should carry in the reclassification process. In addition, documentation provided by the TEA indicates that teachers’ recommendations, known as the “Subjective Teacher Evaluation,” which is based on “[a]ssessments, anecdotal notes, portfolios, etc.,” should factor into reclassification decisions (TEA, 2010, p. 73). Again, however, neither is there guidance as to how much weight subjective teacher evaluations should carry in the reclassification process nor is there information about how teachers’ evaluations of their EL students should be conveyed to LPAC members.
Research Methodology
This study employs a mixed-methods sequential explanatory design (Creswell & Clark, 2007; Ivankova, Creswell, & Stick, 2006; Teddlie & Tashakkori, 2009). In this design, quantitative and qualitative data are collected and examined in two distinct phases, eventually being connected through analysis and by integrating results (Ivankova et al., 2006). This design provides an opportunity to gain a more nuanced understanding of the underlying process of reclassification policy implementation by analyzing multiple data sources to investigate the research questions in a complementary fashion (Greene & Caracelli, 1997).
The first phase of this study focuses on utilizing quantitative data to gauge whether there is evidence of differential policy implementation related to the reclassification of ELs across the state of Texas. Specifically, we investigate whether the likelihood of being reclassified varies for similar EL students in different parts of the state of Texas. We assert that if students with the same academic and English proficiency scores and the same demographic characteristics experience different likelihood of reclassification depending on the region in which they live, this provides evidence of differential policy implementation across the state.
The second phase illuminates quantitative findings by employing qualitative data to understand how practitioners in eight schools implement reclassification policy on the ground and unpacks reasons as to why practitioners in these schools approach, interpret, and implement this policy in different ways.
Research Phase 1: Quantitative Event History Analysis (EHA)
Data
Since 1990, Texas has been collecting student-level data through the Public Education Information Management System (PEIMS), which has resulted in the Texas Schools Microdata Panel (TSMP), a confidential database containing annual data for more than 11 million K–12 students and more than 400,000 public school teachers and administrators. These data include detailed information on student demographics and school composition, in addition to information on students’ educational profile, and performance on assessments. The TSMP contains encrypted student identifiers, which enable researchers to link data files from year to year to construct a panel data set and conduct longitudinal student-level analyses.
Sample
The data set constructed for this analysis includes the first-grade cohort in Texas public schools during the 2002–2003 academic year. 2 It is particularly important to study ELs through a cohort analysis that begins when students enter school because of the instability in the EL subgroup (Abedi, 2008; Saunders & Marcelletti, 2012). Unlike other traditionally underperforming subgroups (e.g., economically disadvantaged, racial minorities, students with special needs), there is systematic fluctuation in this group; students who are identified as ELs in a given year may no longer be members of that subgroup in subsequent years because they have been reclassified. Therefore, cross-sectional comparisons are particularly ill suited for studying ELs.
The panel data set constructed contains a minimum of two and up to seven records per student, one for each year of data (2002–2003 through 2008–2009). Students who were not present in the TSMP in a given year (meaning they did not attend public schools within the state of Texas that year) were not permitted to reenter the sample because there is no way to determine whether or not they were reclassified while they were being educated elsewhere. The data set was then restricted to only include the last 5 years of data because these are the only years with achievement data since statewide assessments begin in third grade. 3 In an effort to minimize sample bias, students who were missing test scores in a given year (but not all years) were included in the analysis for the years that they had complete testing information. The final sample consisted of 55,763 EL students.
Variables
The dependent variable is expressed as a conditional failure rate or hazard rate, which is a latent variable of the underlying risk process for reclassification. The hazard rate is conditional because it gives the rate at which students are reclassified (failed to survive) by time t given that the student had not been reclassified (survived) until t. The data utilized to estimate the hazard rate are dichotomous variables for whether each student was reclassified in the spring of a particular academic year. Each student has a value of 0 for each year that the student remains classified as an EL and a value of 1 for the year that the student is reclassified.
The independent variable of interest is an indicator of which Education Service Center (ESC) region a student’s school is located within. ESC region is a categorical variable that indicates the ESC region in which each student’s school is located (see Figure 1). The state of Texas has 20 ESC regions that serve several purposes, one of which is to help school districts implement policies adopted by the Texas legislature and the commissioner of education. This variable contains 20 categories, one for each region. Each region is incorporated in the analysis as a separate dummy variable (1 = yes, 0 = no), with Region 1 serving as the reference group because this region serves a greater proportion of ELs than any other region in Texas.
Texas Education Service Center regions.
We also include an extensive series of control variables to account for differences between students who live in these different regions. Our goal is to compare the hazard rate of reclassification across different regions of Texas for similar students. Thus, we include a series of student demographic and educational characteristics as well as school environment variables. Descriptions of the control variables are included in Table 1 and descriptive statistics are presented in Table 2.
Control Variable Descriptions
Note. TAKS = Texas Assessment of Knowledge and Skills; EL = English learner; LPAC = Language Proficiency Assessment Committee; TELPAS = Texas English Language Proficiency Assessment System; ESL = English as a second language.
Descriptive Statistics for Control Variables in Event History Analysis Analysis, Time 1 (2005)
Note. The number of observations begins at 48,875, which reflects the number of students included in the first year of the analysis. This number is less than the total sample of 55,763 because there are a number of students who were missing TAKS or TELPAS scores in 2005. This is in part due to retention in 2003 and 2004. Students who had been retained in 2003 or 2004 had not yet reached third grade, the first grade during which the TAKS test is given. Thus, they entered the sample the following year. TAKS = Texas Assessment of Knowledge and Skills; TELPAS = Texas English Language Proficiency Assessment System; ESL = English as a second language; EL = English learner.
Method
The analytic approach centers on EHA. This method models the relationship between multiple covariates and the probability that an EL is reclassified as English proficient at a particular point in time. EHA focuses on modeling the processes that may lengthen or shorten the amount of time that passes before a crucial event occurs (Yamaguchi, 1991). The particular event of interest in this analysis is reclassification. Any EL who has not been reclassified at a given time period is considered to be at risk of experiencing the event. After the student experiences reclassification, he is no longer at risk, so the remainder of the periods are coded as missing and removed from the data set.
EHA offers a number of advantages over traditional logistic regression techniques (Bennett, 1999; Box-Steffensmeier & Jones, 2004). First, in contrast to logistic regression, which only predicts whether or not an event occurred, EHA allows for the examination of both the occurrence and the timing of events (Mokher, 2008). This is particularly useful for examining the research question of interest because it permits the examination not only of whether or not students exit EL status, but when reclassification occurs during ELs’ educational careers. Second, logistic regression commonly omits cases that do not experience the event by the end of the observation period, which may result in sample bias (Mokher, 2008). In EHA, students who have not experienced the event of interest by the end of the observation period are known as censored observations. One of the distinct advantages of this method is that it is able to use information from both censored and noncensored cases to predict the risk of an event occurring at a specific point in time, thereby generating unbiased parameter estimates (Singer & Willett, 2003). Finally, EHA allows for periods of nonobservance, which means that students are included in the analysis even if they are not included in the data for all years. This allows for the inclusion of students who were present in Texas public schools for several years but not the complete 8 years of observation. It also facilitates the inclusion of students who have missing data for a period of time (e.g., a missing assessment score) to be included in years when the data are complete.
EHA centers on two key distributional functions, the survivor function and the hazard function. The survivor function, S(t), is a nonincreasing function that estimates the probability that individual i will survive (or fail to experience the event of interest) longer than time t (Box-Steffensmeier & Jones, 2004). In this study, the survival function is the probability that an EL student will remain classified as EL beyond a given academic year. The second important distributional function, the hazard function, estimates the instantaneous rate of change in the probability of experiencing the event of interest during time t, conditional upon remaining in the risk set of those who are eligible to have an event at that point in time (Box-Steffensmeier & Jones, 2004). For the purposes of this analysis, the hazard function can be interpreted as an indicator of how the risk of being reclassified shifts over time for those students who have not been reclassified. The multivariate model determines how the explanatory variables influence the hazard rate.
Instead of a highly parameterized model, this study uses an unrestricted approach to time in which the hazard rate is permitted to vary by year. This approach is the discrete-time analog to the Cox proportional hazards model, which has become more conventional in social science research (Box-Steffensmeier & Jones, 2004). The particular specification of the hazard function in this analysis is a discrete-time proportional hazard model, in which time is divided into discrete units rather than being continuous. In this analysis, the discrete-time unit is academic years because ELs will either be reclassified or not during the spring of each school year. Time is measured in discrete units as the number of academic years since 2004–2005 (t) until an EL student (i) is reclassified as English proficient. See Appendix (available in the online version of the journal) for more information on how the baseline hazard specification was selected.
The discrete-time proportional hazards model makes use of a complementary log–log link function to determine the effect of covariates on the hazard rate. The discrete-time proportional hazards model estimated for the reclassification of ELs is
where λ
it
is the hazard rate of reclassification of individual i in year t,
Research Phase 2: Qualitative Analysis
The second phase of the study utilizes qualitative data to unpack quantitative results and examine how the EL reclassification process varies in practice across places where there is quantitative evidence of differential hazard rates of reclassification.
Data
To capture and understand how LPAC members approach, interpret, and implement reclassification policy, we conducted qualitative case studies (Yin, 2009) of eight elementary schools across three ESC regions in Texas. Each case study included (a) observations of end-of-year LPAC meetings when EL student files are reviewed and considered for reclassification and (b) semistructured focus group interviews with LPAC members.
Observations and focus group interviews with LPAC members occurred in May and June of 2014. We directly observed end-of-year LPAC meetings take place on school grounds. As direct observers, we watched rather than took part in the meeting. We strove to be as unobtrusive as possible so as not to bias the observation. We took detailed field notes during the observations, recording not only the processes, procedures, and tools that were used during the LPAC meeting but also the ambiance and physical set up of the room in which the LPAC meeting took place and the level of involvement of each of the LPAC members. Following observations, we conducted focus group interviews with LPAC members, which allowed us to ask questions or obtain clarity around what we observed during LPAC meetings. We used a semistructured protocol that included open-ended questions, which was designed to elicit rich information about the ways LPAC members understood and implemented EL reclassification policy as well as how reclassification decisions were made at the local level. Focus group interviews were digitally recorded and transcribed for analysis. Focus group participants were provided with a modest incentive (US$25 gift card) to thank them for speaking with us.
Although these case studies are not broadly generalizable, our observations and focus group interviews allowed us to generate new hypotheses and begin to theorize about relationships significant to understanding reclassification policy implementation variations that were largely invisible in our quantitative analysis (Hartley, 1994).
Sample
We purposefully selected schools on several characteristics. First, we selected schools in different ESC regions based on quantitative Phase 1 results in an effort to maximize reclassification policy implementation variation. Second, we chose schools that had variation in a number of characteristics that we posited might affect the way reclassification policy is implemented on the ground, including percentage of EL students, percentage of economically disadvantaged students, average teacher tenure, and charter school status. Our sample is summarized in Table 3.
Qualitative Sample Summary
Note. In two schools (Spruce and Rodriguez), only one LPAC member was available to stay and participate in our focus group interview following our observation of the LPAC. In both cases, it was the administrator on the LPAC who was interviewed. Source of total student enrollment, percent EL, percent economically disadvantaged, average teacher tenure data is Texas Education Agency Academic Performance Report System, 2013–2014. We have rounded student enrollment, percent EL, percent economically disadvantaged, and average teacher tenure so as to protect anonymity of our participating schools. ESC = English as a second language; LPAC = Language Proficiency Assessment Committee; EL = English learner; TPS = traditional public school.
Method
Our analysis utilized pattern coding to discern patterns of thought, action, and behavior among LPAC members who were part of the observation and/or focus group (Yin, 2009). To explore the research questions, we established and applied a baseline a priori framework during the first round of coding the data. For example, prior literature on policy implementation and reclassification prompted us to code for LPAC members’ orientations toward or philosophies about reclassification policy (e.g., Anagnostopoulos & Rutledge, 2007; Coburn, 2006; Estrada & Wang, 2013); their working knowledge, schemas, and problem framing (e.g., Anagnostopoulos & Rutledge, 2007; Coburn, 2006); and how LPAC members incorporated data in the reclassification decision-making process (Estrada & Wang, 2013). Utilizing this framework, we coded LPAC observation field notes and focus group transcripts using the constant comparative method (Patton, 2001). This process was both iterative and theory driven, and reflected inductive and deductive analysis (Strauss & Corbin, 1990).
Throughout the collection and analysis of data, we employed recommended techniques for establishing trustworthiness to minimize bias and avoid errors while conducting qualitative research (Lincoln & Guba, 1985). For example, we assured names of LPAC members, school names, and district names were kept confidential; used peer debriefing to test working hypotheses; and triangulated data collected from multiple sources (observations and focus groups) as well as between two investigators.
Study Limitations
There are several limitations worth noting. First, although we believe a mixed-methods approach offers many advantages, there are also trade-offs when using this design. Because we use the quantitative results to inform our sample for the qualitative analysis, our two data sources are not concurrent. Second, we acknowledge limitations to the quantitative data. Specifically, we must constrain the EHA when students are in third grade because this is the grade level state achievement assessments begin, which means that students reclassified during first and second grade are not incorporated into the analysis. Third, we recognize that our sample of eight case study schools is small and only studies schools in three regions, limiting both our ability to comment on how local policy implementers make sense of reclassification policy across the entire state of Texas and our ability to make generalized comparisons within or across schools, districts, or regions.
Findings
In this section, we describe the quantitative results on whether there is evidence of differential policy implementation regarding EL reclassification and qualitative findings on how educators collectively approach, interpret, and implement EL reclassification policy.
Evidence of Differential EL Reclassification Policy Implementation
Results from the discrete-time proportional hazards model are presented in two formats in Table 4: the first with the raw coefficients and standard errors, and the second with the exponentiated coefficient to ease interpretation. A one-unit change in a covariate corresponds with an estimated change in the hazard rate by exp(coefficient); the idea is that the coefficient has a multiplicative effect on the hazard rate.
Event History Analysis Results: Time to Reclassification
Note. In all, 9,127 students had not been reclassified by the final year of the analysis and are considered right-censored. ESC = Education Service Center; TAKS = Texas Assessment of Knowledge and Skills; TELPAS = Texas English Language Proficiency Assessment System; ESL = English as a second language; EL = English learner.
p < .05. **p < .01. ***p < .001.
Interpretation of the results will focus on the exponentiated coefficients in the final model, which are displayed in the far right column of Table 4. With this transformation, a coefficient that is greater than one indicates that a particular covariate is associated with an increase in the hazard rate of reclassification, whereas a coefficient less than one corresponds to a decrease in the hazard rate. Specifically, a one-unit increase in X corresponds to a predicted 100 × (exp(β) − 1) percent change in the hazard rate of reclassification. Figure 2 displays a plot of exponentiated coefficients and 95% confidence intervals, which allows for the comparison of the hazard rate of reclassification across regions.
95% confidence interval plot of Education Service Center region exponentiated coefficients.
Regional covariates are all compared with Region 1, which consists of the Rio Grande Valley and is based in Edinburg, Texas. Interestingly, a number of regions have statistically significant differences in the hazard rate of reclassification than Region 1. Regions 3, 4, 15, and 19 (areas surrounding Victoria, Houston, San Angelo, and El Paso, respectively) have a positive and statistically significant influence on the hazard rate of reclassification, indicating that similar students in these regions are more likely to be reclassified by the end of seventh grade than students in Region 1. Conversely, Regions 5, 6, 7, 8, 10, 12, 13, 18, and 20 (areas surrounding Beaumont, Huntsville, Kilgore, Mt. Pleasant, Richardson, Waco, Austin, Midland, and San Antonio) were negatively related to the hazard rate of reclassification, suggesting that EL students in these regions are less likely to be reclassified than students in Region 1. As an example of how to interpret these results, students who attend schools in Region 19 (around El Paso) are nearly twice (1.92 times) as likely to be reclassified as their peers who attend schools in Region 1. In other words, two students who perform at the same academic and English proficiency levels and are identical on a whole host of demographic and educational characteristics have notably different hazard rates of reclassification depending on where they attend school in Texas.
This analysis finds substantial variation in the hazard rate of reclassification for similar EL students between the different ESC regions. This confirms that schools in Texas implement reclassification policy differently across ESC regions, suggesting that local policy implementers exert much influence over the process at the local level. However, this quantitative analysis does not help explain why these differences may exist. Thus, we turn to the findings from the second phase of the study, which employs qualitative data to examine variation in reclassification policy implementation.
EL Reclassification Policy Implementation in Practice
Across the eight schools observed, educators on the school-level LPAC committees went about organizing the EL reclassification decision-making process in strikingly different ways. Although one might expect that LPAC demands would vary by the sheer number of EL students in the school, the work of the LPAC also varied along other meaningful dimensions. As illustrative examples of the extensive variation in the EL reclassification process, we begin by providing descriptions of the LPAC meeting at three case study schools, one in each ESC region included in this study. We then present broader thematic findings regarding how LPAC members understand and implement reclassification policy. Although this portion of our study qualitatively examines how EL reclassification practices vary across regions, it should be noted that this analysis is not meant to explain the hazard rates of reclassification from the quantitative analysis. Instead, the field data offer an opportunity to develop informed hypotheses about how policy implementation differences may shape the rate of reclassification that could serve as the foundation of future research.
Case Study 1: Cooper Elementary School 4
At Cooper E.S., approximately 85% of the student body are ELs, all of whom are native Spanish speakers. Located in Region 1 along the border with Mexico, this school like others in the region serves substantial numbers of EL students. In the first phase of this study, Region 1 was used as the reference group in the quantitative analysis and can be classified as region with medium hazard rate of reclassification.
LPAC members met over several school days in a small conference room in the main office. They were surrounded by stacks of student record folders, one for each EL student. The assistant principal began the meeting by asking each LPAC member to read aloud a specific assessment score while the assistant principal hand wrote the scores onto a form for each student. For example, one teacher was asked to say the reading achievement score, while another was responsible for reading aloud the English language proficiency scores in each language domain. In self-described “assembly line” fashion, LPAC members went around the table, each reading the respective scores for one student. If scores were missing, the “LPAC clerk,” an administrative assistant, was summoned to the room via walkie-talkie and tasked with investigating the missing information. This time-intensive process focused exclusively on completing the required score form documentation and obtaining LPAC members’ signatures.
Upon completing the forms for each grade level, the assistant principal read aloud the names of students who had met objective state-set exit criteria. Interestingly, this list had been derived before scores had been filled out by hand, suggesting that filling in a score form for each student was solely to ensure that the school was in compliance with district and state regulations, but was not in fact being used to determine eligibility for reclassification.
Before the meeting, students’ classroom teachers had been asked to complete a district-designed subjective evaluation form for each student, prompting teachers to reflect on students’ grades and linguistic supports, and provide other “anecdotal notes or comments.” The LPAC reviewed teachers’ comments on each student who had met the objective assessment exit criteria, and there was little, if any, disagreement between the objective assessment data and teachers’ subjective evaluations. Thus, all students who had met objective assessment criteria were reclassified.
Students who had not met objective assessment criteria were not discussed by the LPAC at all. In fact, the assistant principal commented, Ideally, we would be able to discuss every child; but, realistically, with the amount of time we have, we haven’t been able to do that . . . I am curious how it works in other places and if there is a better way.
Case Study 2: Rodriguez Elementary School
Rodriguez E.S. serves more than 200 EL students (close to 50% of the student population), the vast majority of whom are Spanish speaking. This school is located in ESC Region 19, which includes school districts in the El Paso metropolitan area. Quantitative findings indicate that this is a high hazard rate of reclassification region.
The LPAC meeting at Rodriguez E.S. took place over the course of one school day in a spare classroom. The space had been set up to facilitate the meeting. Small tables were clustered together to create a large conference table close to a projector screen. Although an assistant principal and parent representative were around the table for the entire day, grade-level teachers rotated in and out of the meeting when their students were up for review. For example, when the LPAC considered third-grade EL students, all third-grade teachers were present to discuss the needs of their students during their 45-minute planning period. Once every third-grade student was discussed, the third-grade teachers exited and a new set of teachers came in to discuss the next grade level of EL students.
In this LPAC meeting, the assistant principal announced each student’s name, often alongside comments from teachers such as “she is such a hard worker” or “he just came to the district this year and is living with his grandmother.” Simultaneously, the student’s performance report was projected on a screen with the student’s performance on all state-set exit criteria prepopulated onto the form. The assistant principal then highlighted important information from the student’s performance report, directing attention to the set of academic achievement and English proficiency scores the state mandates must be met for a student to be reclassified. Teachers focused on the report on the screen and at times interjected to correct an error on the report (e.g., a score that had been entered incorrectly or a last name misspelled). The assistant principal then asked the student’s teacher, “Is there anything we can do for this student to make him/her more successful?” probing further about the student’s progress over the course of the school year, classroom disposition, use of linguistic accommodations, and, on occasion, social/emotional health and well-being. Teachers paused to discuss surprising scores, particularly focusing on why a student may have scored lower than expected, making statements such as “I’m pretty sure he didn’t take his medication the day we tested because these scores do not reflect his actual abilities” and “this assessment was given online and he is still learning how to use a computer.” Teachers also frequently provided specific recommendations for both classroom and testing accommodations for the students to receive in the next school year while the assistant principal took detailed notes on her laptop. For example, teachers made recommendations about specific students who could benefit from regularly having access to a bilingual dictionary, attending afterschool tutorials, and a need for next year’s teacher to be regularly in contact with parents to prevent absences. Although many students had not met state-set criteria to be reclassified and, thus, were not exited from EL status, all EL students’ files were individually reviewed and discussed.
Case Study 3: Arbor Elementary School
Arbor E.S. serves more than 150 EL students (approximately one third of the student population), most of whom are Spanish speaking. This school is located in ESC Region 10, which services school districts in the Dallas metropolitan area. The first phase of this study found this region to have a low hazard rate of reclassification.
The Arbor E.S. LPAC meeting took place one morning around a small table in the assistant principal’s office. The composition of the committee remained the same for the duration of the meeting and consisted of the school’s principal, assistant principal, a content-area specialist, a bilingual teacher, and a parent representative.
Prior to the meeting, a hard copy of each student’s performance report prepopulated with student scores had been printed out. In addition, in cases where students had met state-set assessment criteria for reclassification, that child’s classroom teacher had been asked to provide written comments about whether or not he or she believed the child was in fact ready to be reclassified. The LPAC began the meeting by reviewing files for all students who had met state-set criteria. The assistant principal read aloud each student’s name and scores to the rest of the group, and then read aloud comments provided by that child’s classroom teacher. Interestingly, comments provided by teachers often seemed disconnected from determining readiness for reclassification. For example, one teacher expressed concern about reclassifying a child because he had not yet been able to demonstrate effective leadership skills. Another teacher recommended against reclassifying a student because she was shy and introverted. Despite this apparent disconnection, the LPAC weighted teachers’ recommendations heavily and did not reclassify any students for whom teachers had raised concerns.
Upon completing reviews of students who had met state-set reclassification criteria, the LPAC turned to the files of students who had not met these criteria. The assistant principal read each student’s name aloud and quickly passed the student’s performance report to the LPAC member sitting to her left, who quickly glanced at the form and passed it along to the next LPAC member. During this process, the assistant principal stated, “If you have any questions or disagree [that the student should remain classified as an EL], let me know.” This process ensued with little to no discussion of any student. It took approximately 5 minutes to “review” the files for more than 50 EL students in two grade levels.
These contrasting vignettes of LPAC meetings demonstrate the variation in how LPAC meetings are conducted, reflecting deep differences in how reclassification policy is implemented at the school level. In particular, there were clear differences in terms of (a) the meeting activities that constituted reviewing students for reclassification, (b) the role of technology in facilitating the meeting, and (c) the data sources that were used and weighted in the decision-making process. We have summarized these differences across all our case study schools in Table 5 and we describe key differences below.
Case Study Data: Meeting Activities, Use of Technology, and Data Sources
Note. ESC = Education Service Center; EL = English learner.
Meeting Activities
Although all the schools in our sample conducted an end-of-year LPAC meeting, activities that took place during these meetings varied substantially across schools. One key dimension of variation is the extent to which LPACs spent their meeting time filling in score forms for each student. In three schools (Cooper, Maverick, and Antonio), LPACs spent meeting time filling in a score form by hand for each student. This process was very mechanical and was devoid of any substantive discussion. It was also tremendously time consuming, especially because each of these schools had hundreds of EL students. Conversely, in five schools (Arbor, Spruce, Martinez, Sage, and Rodriguez), reports were prepopulated before the meeting. However, schools that had prepopulated forms utilized their meeting time differently. In Martinez, Sage, and Rodriguez, LPACs reviewed prepopulated forms and discussed the progress made by each individual EL student, regardless of whether students had met objective assessment exit criteria. LPACs in these three schools discussed English proficiency growth over time, provided anecdotal information on individual students, and considered the extent to which students would continue to need linguistic accommodations on assessments the next school year. In the other two schools with prepopulated forms (Arbor and Spruce), LPAC meetings were brief encounters where members gathered to quickly review and discuss only students who had met the objective assessment exit criteria and sign forms.
Role of Technology
LPAC committees made different use of technology to facilitate their meetings. Four of the five schools with prepopulated score forms had used the mail merge feature of Microsoft Excel to merge data into individual score reports (score forms at Martinez had been filled out by hand before the meeting). Doing so freed up time—LPACs did not have to spend their entire meeting filling out forms. However, as discussed above, LPACs at Martinez, Sage, and Rodriguez capitalized on this freed up time when they would have been filling out forms to actually discuss each individual student and consider how to best support students the following year, whereas Arbor and Spruce just ended meetings earlier.
In addition, some LPACs chose to use technology to display data as a way of facilitating and focusing discussion. At Sage and Rodriguez, prepopulated score reports from each student were displayed on a screen one at a time and no paper score reports were passed around the table during this discussion. This allowed all LPAC members to focus on each student’s data simultaneously and seemed to spur a more in-depth discussion of each student. It also saved time because papers were not being passed around the table for review.
Finally, in one school, Rodriguez, the assistant principal on the LPAC utilized her laptop to take notes on the discussion about each student. She was interested in recording ideas around supports and services the LPAC recommended for each student the subsequent academic year, noting how important this was, particularly for students transitioning from elementary to middle school, because she could email this document to the middle school administrators and teachers who provide English language development services.
Data Sources and Weighting
All LPACs utilized state-standardized reading and writing academic achievement data and state-approved English proficiency assessment data. Scores on these assessments were a central part of the reclassification decision-making process—LPACs did not reclassify any students who had not met these criteria in any of our sample schools. However, LPACs did vary in the extent to which they included input from each students’ classroom teacher and the weight this information carried in the decision-making process. Half of the schools, Arbor, Spruce, Cooper, and Martinez, incorporated written input from classroom teachers when students met exit criteria. At Cooper and Martinez, teachers’ written input served as a way of confirming objective assessment scores. These LPACs reviewed teachers’ input, and in all cases, teachers’ evaluations aligned with objective assessment scores. LPAC members reported that it was rare that teachers’ subjective evaluations conflicted with objective assessment scores, but if they did, they would defer to the objective assessment scores to determine whether a student should be reclassified. Conversely, when classroom teachers at Arbor and Spruce recommended against reclassifying a student despite the fact that the student had met all objective assessment criteria, the teacher’s evaluation always trumped objective assessment data even when teachers expressed concerns about aspects of students’ learning that were not related to English proficiency (e.g., introverted personality, lack of leadership skills, disciplinary infractions).
LPACs at Sage and Rodriguez incorporated classroom teachers’ subjective input in the meeting in a different way. These LPACs relied exclusively on objective assessments when making the decision to reclassify a student. As one Sage LPAC member explained, they used objective assessment data to decide because, “it is not opinions. It is based on results. It is based on information.” However, although teachers’ evaluations did not factor into the reclassification decision itself, their input was incorporated into the process as a way of unpacking and contextualizing objective assessment scores and planning for the supports, services, and accommodations students would receive the subsequent school year.
In line with Estrada and Wang’s (2013) study, our case studies suggest that the use of subjective data in the reclassification decision-making process is not inherently positive or negative, but how these data are incorporated has the potential to expand or constrain educational opportunity for EL students. This highlights the critical need for those implementing the policy on the ground to understand the policy’s underlying purpose so that they can integrate subjective data into the process in a way consistent with the underlying goals of the reclassification policy.
The qualitative portion of our study sets out to examine how the EL reclassification process varies in practice across regions with different reclassification hazard rates. One limitation of our study is that we are unable to concretely link any of the three different reclassification practices discussed above to the rate of reclassification. We can, however, offer some hypotheses about how these differences may shape the rate of reclassification that could be examined in subsequent research. First, it appears that when subjective teacher evaluations are prioritized over objective data sources, students may be less likely to be reclassified. Only two schools in our sample, Arbor and Spruce, prioritized teacher evaluations over objective assessment data, and both these schools were located in a region with a low rate of reclassification. This makes intuitive sense; if students are required to meet additional criteria, they are going to be less likely to be reclassified.
Perhaps less intuitive is the possible relationship that exists between the rate of reclassification and the other two differences in how LPAC meetings were conducted: (a) meeting activities and (b) the role of technology. The three schools where LPACs spent their meeting time discussing individual students were located in regions with a medium or high rate of reclassification. Technology entered the equation in two of these schools, freeing up time that would have otherwise been spent filling out score forms and providing data displays that helped guide and focus discussions on individual students. Although LPACs in these schools discussed each individual student and used technology to do so, neither appeared to shape its decision to reclassify students that year. However, it is possible that these discussions and technology influenced the rate of reclassification over time. As illustrated in the case study of Rodriguez above, when this school’s LPACs discussed individual students, they talked about not only whether or not to reclassify a student that year but also what services, supports, and accommodations should be provided to that individual student the subsequent year. It may be the case that the EL students in these schools receive services that are better aligned to meet their individual needs the following school year, thereby influencing the subsequent year’s likelihood of reclassification. LPACs across all eight case study schools indicated that the way they conduct their LPAC meetings had changed little over time, meaning that in schools where we saw discussions of individual students and saw technology integrated into the process, these practices had become routine and had occurred in previous years. Thus, there may be a cumulative effect of these practices on the rate of reclassification.
Constructing Policy Understanding
In conjunction with our observations, focus group interviews helped shed light on why these tangible differences in reclassification policy implementation occurred. We found that LPACs had divergent orientations toward reclassification and that they had profoundly different knowledge of the purpose of reclassification policy.
Orientation Toward Reclassification
Although all schools were operating under the same statewide reclassification criteria, LPACs constructed different understandings of the consequences of reclassification. LPACs at Martinez, Sage, and Rodriguez expressed a belief that objective state reclassification criteria are rigorous, setting a challenging bar for students to meet. LPAC members at Sage Elementary made comments such as “The criteria are high. I mean out of [more than 250 EL students] we have 38 that are eligible to exit next year. So the standard is high,” “It is not easy to pass this test,” and “If they can pass it, they are ready.” An LPAC member at Rodriguez Elementary saw meeting these criteria as a major “milestone” for their EL students and confirmation of a job well done. They seemed eager to reclassify students who had met these criteria and did not have doubts about these students’ ability to succeed.
LPACs at Cooper, Maverick, and Antonio took a much more neutral stance on reclassification. For instance, one Cooper Elementary LPAC member explained that their principal told them “We aren’t going to worry about whether or not they exit—consider what the child needs.” Reclassification decisions in these schools were made in a very matter-of-fact fashion. LPAC members in these schools seemed to see reclassification as just part of the educational process for ELs as opposed to a major milestone.
Conversely, LPACs at Arbor and Spruce approached reclassification with caution and seemed to view reclassification as a risky event that might result in student failure. They often seemed tentative about reclassifying students who had met state-set reclassification criteria, even for students with high English proficiency and academic achievement assessment scores. They tended to approach state-set criteria with skepticism and layered on additional data, which was often subjective and disconnected from English proficiency, to inform reclassification decisions.
Knowledge of Policy Purpose
Our data illustrate that policy implementation differences were linked to how LPAC members collectively understood the underlying purpose of reclassification policy. When asked about the purpose of policies related to EL reclassification, LPACs at Martinez, Sage, and Rodriguez focused on the ways that reclassification meetings can be used as an important tool to review and monitor the progress of EL students and make data-driven decisions around the services to be provided to help EL students succeed. For example, one LPAC member at Sage said, the most important thing is that we are monitoring progress . . . that we are making sure that we are showing gains and sometimes students regress and we have to go back and teach some of the standards.
These LPACs deeply understood the spirit of the reclassification policy—it serves as a mechanism to transform the way in which schools approach monitoring progress to ensure equitable educational opportunities for all EL students. Thus, they structured their reclassification meetings in a way that mirrored this knowledge, focusing more on a discussion of the progress each individual student had made and supports and services that could be provided to help students be more successful in the future as opposed to getting paperwork signed and filed.
Although these LPACs believed the EL reclassification policy could be used to foster something greater than another procedure that must be carried out, other LPACs’ understanding of the purpose of the policy did not reflect this fundamental, transformative aspect of the reclassification policy. Individuals in these schools understood the EL reclassification policy as chiefly involving technical processes put in place to ensure schools remain in compliance with state law. LPACs at Arbor, Spruce, Cooper, Maverick, and Antonio emphasized the importance of being in compliance with policy. For example, one Cooper LPAC member indicated that the purpose of the reclassification policy is “to make decisions for these kiddos, follow through with state mandates.” Similarly, a member of the Spruce LPAC indicated that the LPAC must “be very aware of the changes because if you don’t know them, you can be out of compliance for a lot of things.” This compliance orientation toward the policy manifested in the actions of the LPAC. For example, the LPAC at Arbor Elementary read students’ names aloud and rapidly passed prepopulated forms around the table to comply with the requirement that every student be reviewed individually. This focus on completing very specific tasks to check off boxes was pronounced in schools where LPAC members believed the purpose of the policy was to demonstrate compliance with state policy.
Discussion and Implications
At both the state and national level, government has sought to ensure educational equity for ELs by adopting policies guiding reclassification. However, the way these policies are implemented on the ground varies considerably, arguably influencing their potential to have the desired impact on students. The results from our study have implications for both how EL students are served as well as the broader policy implementation literature.
Implications for ELs
Our findings suggest that similar EL students receive different treatment across the state. Students with the same academic achievement and English proficiency performance levels may remain classified as an EL for a longer period of time in a low hazard rate of reclassification region such as Region 10, while exiting EL status more rapidly in a high hazard region such as Region 19. This signifies that these students would receive English language development services for different periods of time depending not on English proficiency level, but on how reclassification policy is implemented at their school. This begs a question about whether schools in some regions are reclassifying students too early, thereby withdrawing language services prematurely, or whether schools in other regions are reclassifying students too late, possibly preventing students from accessing advanced coursework and important peer networks. Although our study cannot answer this question, it certainly warrants further investigation.
Cimpian et al. (in press) have begun to quantitatively compare how reclassified students in different districts across two states are faring academically. Their novel use of regression discontinuity design to measure the effect of being reclassified helps determine whether reclassification is systematically occurring too early or too late in different districts. However, much more qualitative work is needed to understand why some students experience academic success upon reclassification, whereas others decline academically upon being reclassified. While reclassification timing is arguably important, so too is the actual change in services and student experiences that takes place upon reclassification. For example, it may be the case that EL students in some schools are effectively barred from enrolling in advanced coursework until they are reclassified, whereas in other schools, ELs are encouraged as much as their non-EL peers to take advanced courses. If the latter is the case, we would expect reclassification to have less of an effect because it does not result in access to something that was previously denied. Similarly, in some schools, reclassification may not immediately signal the withdrawal of all language development services; schools may continue to provide reclassified ELs with English language development supports or tutoring to ensure a smooth transition, which would also shape the impact of reclassification. Ultimately, the effect of reclassification is largely a factor not only of timing but also of the services students receive before they are reclassified, as well as the changes that occur in services postreclassification. At present, very little is known about what is bookending the actual reclassification decision and how this interacts with reclassification timing. That is, future research needs to quantitatively and qualitatively examine the extent to which the effect of reclassification is informed by the types and quality of services being provided before being reclassified, and the specific educational changes that occur upon being reclassified.
Differences in hazard rate of reclassification also have consequences for comparing the performance of ELs across the state and holding schools accountable, a key provision of federal EL policy under ESSA. In regions with a low rate of reclassification, the threshold for English proficiency is likely higher, indicating that a greater portion of higher performing students remain classified as ELs, whereas the opposite is likely the case in regions where ELs have a higher hazard rate of reclassification. This complicates the comparison of the EL subgroup because, in regions with a lower hazard rate of reclassification, ELs will likely appear to be faring better not necessarily because schools in the region are doing a better job of serving these students, but because of how reclassification policy is implemented, particularly what data are part of the decision-making process, and how data are weighted.
The way that LPACs implemented the EL reclassification policy may also have implications for educational equity. For example, ELs in schools that only review the files of students that meet the state-specific reclassification criteria may not receive equitable educational opportunities if their language and academic progress is never discussed in a meaningful way. However, ELs in schools that review every EL, regardless of academic and English language proficiency test performance, may receive the necessary and adequate supports and, thus, a more equitable educational experience. Moreover, LPACs that rely heavily on data that have nothing to do with English proficiency (e.g., teacher input on students’ personality traits, leadership skills, behavior) to make reclassification decisions may be well intentioned, but may restrict educational opportunity for ELs who are in fact proficient in English. These students may be required to continue enrolling in an English language development course and may effectively be barred from enrolling in advanced coursework.
Implications for Policy Implementation
Our study builds upon the policy implementation literature by understanding how LPACs interpret and implement policy and how these two processes interact. LPACs resoundingly agreed that they do not interpret policy: “We don’t play any role in interpreting state policy—not at all,” stated one LPAC member from Sage Elementary. However, our study demonstrates that LPACs do in fact interpret state policy because they construct different understandings of the underlying purpose of the policy that affect implementation. LPACs in some schools understood the true spirit of the law, deliberately advocating for ELs. Even though policy implementation was not uniform in these schools, it was evident that the modus operandi of these LPACs was to implement the spirit of the law. In contrast, other schools often lost sight of the overarching goal and purpose of the policy when implementing it within their own context. In these schools, the administrative tasks embedded in the policy often took precedent over meaningful tasks that spoke to the true spirit of the law. In essence, our study highlights the fine line that exists between adapting policy to fit local context while continuing to reflect the goals of the law and adapting policy to the extent that it may actually undermine the law’s underlying purpose.
What can local and state governments do to encourage and support policy understanding and implementation in line with the spirit of the law? Our case studies point to two areas in which we believe spirit of the law implementation can be encouraged while still ensuring compliance with the letter of the law. First, a number of LPACs found efficiencies within their LPAC process that then allowed them to focus on the core pieces of the policy, serving as “the voices that initiate, articulate, deliberate, and determine the best instructional program for the students” (TEA, 2015, p. 7). For example, some LPACs realized efficiencies through the use of technology, allowing basic technologies to facilitate the data and documentation compliance elements of the EL policy. This efficiency allowed these LPACs to allocate more time to implementing the core of the law and focus on discussing every single student’s progress and their needs. Thus, in many ways, building capacity to use technology may help schools comply with the administrative aspects of the law, while also affording LPACs the time to actually implement the spirit of the law.
Second, we found that LPACs that reflected spirit of the law implementation took purposeful steps to structure their meetings in ways that demonstrated a more nuanced understanding of the policy. For example, administrators that served on these LPACs had clear meeting agendas, established roles and responsibilities for each LPAC member prior to the meeting, and set up the room to facilitate conversation among participants. In schools that reflected letter of the law implementation, LPAC meetings occurred in an ad hoc fashion and were murkier and lacked definition. As a result, LPAC meetings transpired without specific tasks occurring, often resulting in important policy implementation elements being overlooked. In particular, core elements of the policy such as discussions on the progress of each individual EL student seemed to elude the members of the LPAC in favor of a focus on administrative tasks required to demonstrate compliance.
These findings suggest that it is important to help policy implementers deepen their understanding of the policy. One way to do so would be to provide professional development that introduces LPAC members to meeting activities and structures that align with the spirit of the policy. As one LPAC member at Cooper Elementary said, It would be good to share best practices at the district level. I don’t know that we’ve had that conversation on exactly how you [make reclassification decisions] . . . we’ve never talked about it. There is always so much about the requirements and the PowerPoints from TEA that we really don’t talk . . . about how you should set up and do it. That would be a good conversation I would be interested in having.
As ESC regions and districts work to train LPAC members, they could discuss the different ways that LPACs go about their work to increase awareness of the different approaches schools take and provide explicit examples and counterexamples of how to structure LPAC meetings. They could also purposefully target school leaders to build this capacity. Across our case studies, it was evident that the school administrator on the LPAC played a prominent role in setting the meeting agenda, integrating technology into the process, and making decisions around how data would be used in the reclassification decision-making process.
Conclusion
Our findings indicate that there are systematic differences between regions that affect similar students’ likelihood of reclassification. ELs who live in different ESC regions of Texas have statistically significant and meaningful differences in the hazard rate of reclassification, controlling for their performance on English proficiency and achievement assessments as well as student characteristics. This is particularly noteworthy given the fact that Texas is a state that had already taken steps to specify clear reclassification standards and procedures, now a requirement for all states under ESSA. Our qualitative findings suggest that differences in hazard rate of reclassification are due, in part, to educators on the school-level committees emulating Lipsky’s (1969, 1980) concept of street-level bureaucrats, bringing strikingly different perspectives, knowledge bases, and technology to the decision-making process around EL reclassification. As more states are prompted to move toward more standardized reclassification policies under ESSA, it will be important for states and school districts to help build capacity to implement these new policies in a way that aligns with the spirit of the policy.
Our study demonstrates that how local policy implementers make sense of and operationalize reclassification policy shapes their meeting agendas, the extent to which they employ technology, and how they include and weight objective and subjective data. Nonetheless, this is an area that merits additional research, especially as it pertains to how reclassification policy implementation affects access to educational opportunities for ELs. In particular, further research is necessary to understand the relationships between specific EL reclassification practices, rates of reclassification, and, most important, educational outcomes for EL students. Moreover, as states and districts begin to respond to ESSA’s mandate requiring greater standardization of criteria for reclassifying EL students, a future line of inquiry could examine how schools’ reclassification processes change over time. Research exploiting policy changes under ESSA holds potential for better understanding the relationship between reclassification and student performance.
Footnotes
Acknowledgements
The authors are grateful to their colleagues, in particular, Ellen Goldring, Stella Flores, Ron Zimmer, George Bunch, and Will Doyle, for their feedback and support. They want to thank those who provided data access and assistance, including the members of the Texas Advisory Board and the staff of the University of Texas at Dallas Educational Research Center, including Rodney Andrews, Greg Branch, Leigh Hausman, Mark Lu, Nidhi Mehrotra, Janie Jury, and Katie Patterson. They very much appreciate the extremely thorough and helpful feedback they received from two anonymous reviewers. They also wish to recognize and thank the anonymous participants in their study for taking the time to speak with them.
Authors’ Note
The conclusions of this research do not necessarily reflect the opinions or official position of the Texas Education Agency, the Texas Higher Education Coordinating Board, or the State of Texas. All opinions expressed in this article represent those of the authors and not necessarily the institutions with which the authors are affiliated. All errors are solely the responsibility of the authors.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support was provided to Madeline Mavrogordato and Rachel White by the Education Policy Center, Department of Educational Administration, and College of Education at Michigan State University. Funding for Madeline Mavrogordato was provided in part by a grant from the U.S. Department of Education’s Institute of Education Sciences (IES) to Vanderbilt’s ExpERT program for doctoral training (R305B080025).
Notes
Authors
MADELINE MAVROGORDATO is an assistant professor of K–12 educational administration in the College of Education at Michigan State University. She utilizes mixed-methods to investigate how district, state, and federal policies; systems of school governance; and school leadership practices affect educational outcomes for underserved students, particularly English learners and immigrants.
RACHEL S. WHITE is a doctoral candidate in education policy at the Michigan State University College of Education. Her research interests include education governance, politics, and policy implementation.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
