Abstract
Educators raise concerns about what happens to students when they are exposed to new or new-to-school teachers. However, even when teachers remain in the same school they can switch roles by moving grades and/or subjects. We use panel data from New York City to compare four ways in which teachers are new to assignment: new to teaching, new to district, new to school, or new to subject/grade. We find negative effects of having a churning teacher of about one third the magnitude of the effect of a new teacher. However the average student is assigned to churning teachers four times more often than to new teachers, and historically underserved students are slightly more likely to be assigned to churning teachers.
Keywords
Introduction
We find that in any given year, students are nearly four times more likely to be assigned to a teacher who has undergone a within-school assignment switch than a teacher who is new to teaching. We also document that, on average in New York City each year, over 40% of teachers are new to their post in one of the following ways: new to the profession, new to New York City (transferred from another district), new to their school, or in the same school but new to their subject/grade assignment. Given this notable rate at which teachers are new to their positions in some way, we use a variety of fixed effects approaches to estimate the link between student achievement and these various forms of being new to one’s job assignment. We particularly focus on within-school switches given that we find that over half of all switches are of this type and we know so little about how students are affected by it.
Background
As with most professions, on average teachers exhibit returns to experience particularly during the early career (Atteberry, Loeb, & Wyckoff, 2015; Boyd, Lankford, Loeb, Rockoff, & Wyckoff, 2008; Clotfelter, Ladd, & Vigdor, 2007; Harris & Sass, 2011; Ost, 2009; Papay & Kraft, 2015; Rivkin, Hanushek, & Kain, 2005; Rockoff, 2004). Teachers likely improve over time because they gain familiarity and fluency both with the act of teaching itself, as well as the interpersonal demands of the profession. However, many factors are correlated with how much teachers improve over time, including prior training and pathway into the profession (Boyd, Grossman, Lankford, Loeb, & Wyckoff, 2009; Kane, Rockoff, & Staiger, 2008), on-the-job professional development (Yoon, 2007), the strength of school leadership (Boyd et al., 2011; Grissom, 2011), the quality of professional networks within schools (Atteberry & Bryk, 2010), the effectiveness of grade-level peers (Jackson & Bruegmann, 2009), and school socioenvironmental factors including trust, peer collaboration, and shared decision making (Bryk & Schneider, 2002; Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010; Kraft & Papay, 2014). Developing access to many of these resources—or reaping the benefits of them—often takes time. Trust, for instance, is an iterative and long-term discernment process through which actors judge one another’s intentions and worthiness of trust (Bryk & Schneider, 2002). When teachers are brand new to the profession, to a school, or even to a particular working group within a school, they may need to reestablish their connection to these resources. Along those same lines, Ronfeldt, Loeb, and Wyckoff (2013) hypothesized that the negative relationship they observe between high rates of new-to-school teachers and achievement could be explained by the disruption of working norms. Given that teacher improvement may be associated with these local conditions, we therefore begin by considering the reasons that teachers switch schools and roles, potentially disrupting their development.
Why might teachers switch jobs within schools? First, teachers may be relatively more effective in one position than another, and either school leaders or the teachers themselves may seek to optimize the matches of teachers to jobs. Second, some jobs may simply be more appealing, and teachers may vie for these positions. Finally, new demands such as differential enrollments across student cohorts, new courses, or difficulty hiring for particular positions may necessitate reassignment even if neither leaders nor teachers would otherwise seek such reassignment.
Of these three reasons, the first—more optimal matching—might lead to improved outcomes. Either principals or teachers might instigate these changes. In order for principals to reassign teachers strategically, they must understand differences in the quality of their teachers and be able to act on that knowledge. Extant research provides evidence that many principals do have the ability to discern differences in teacher quality (Jacob & Lefgren, 2008; Rockoff, Staiger, Kane, & Taylor, 2012) and, furthermore, that some principals actively use reassignments strategically to achieve their goals (Chingos & West, 2011; Cohen-Vogel, 2011; Grissom, Kalogrides, & Loeb, 2014). These authors conclude that school leaders are attempting to better match teachers to available vacancies. For example, teachers report that principals are more involved in the assignment of teachers to tested grades than to other grades, and teachers whose students have lower test score gains are more likely to move away from tested grades (Grissom et al., 2014). The other two reasons for within-school churn—teachers seeking more desirable positions or due to other changes in the school—do not necessarily have benefits for students.
One can think of “newness” on a continuum. One’s job can be entirely new (as is the case in the first year in the profession), the job assignment can be virtually identical from one year to the next, or it can be somewhere in the middle with some aspects of the job—but not others—new to the individual at a given point in time. Changes in the “what” and “where” of a job may reintroduce some newness back into the work.
Whereas most research on teacher experience has examined the effect on students of having a teacher who is new to the profession (see Hanushek & Rivkin, 2006, for a review), teachers who are new to a district or school might also face challenges. When a teacher moves to a new school to teach the same class, many aspects of the work will remain the same, including the developmental age of the students and the general curricular content. However, the teacher may need to make meaningful changes to instructional materials either to suit a new population of students, or to integrate with the general strategies that are used in the new school. Further, the social norms of the school are new to her, and it may require time and energy to learn how to navigate a new system and/or work with new colleagues. Surprisingly little evidence exists on the impact of being assigned to a new-to-school teacher. Because being new to school involves less unfamiliarity than being new to the profession, the average effect of a cross-school reassignment on student achievement may be negative, but less so than the effect of being a first year teacher.
Similarly, being switched to a new assignment within the same school may also reintroduce some novelty into the work of a teacher. Sometimes moving involves a grade-only shift (e.g., teaching third grade to fourth grade), a subject switch (e.g., switching from teaching social studies to English Language Arts [ELA]), or both (e.g., fifth-grade math to eighth-grade science). Being new to one’s specific job assignment within the same school may also be challenging for teachers, though perhaps less so than being new to the profession, the district, or the school. Whereas such a teacher would continue to possess institutional knowledge and working relationships within the school, the teacher may need to become familiar with a new grade-level or subject-specific curriculum. She may also find herself working with a new set of grade- or subject-specific colleagues. On a daily basis, a new-to-assignment teacher may need to create new lesson plans and/or use existing materials that were previously unfamiliar. The “newness” of these annual within-school switches may cause teachers to be temporarily less effective, and students assigned to switching teachers may exhibit lower achievement than had they been assigned to a teacher who taught in the exact same school–subject–grade the previous year.
We therefore hypothesize that the most challenging form of being new to assignment is being entirely new to the profession, followed by teachers who are new to the district (but not to teaching) and cross-school moves, and finally we hypothesize that within-school reassignments are negative but less so than the other forms. It is worth noting, however, that even if within- and between-school reassignments are initially associated with decrements to student achievement in the year of the switch, it is possible that the teachers are ultimately moving into positions that suit them better (i.e., the optimal matching scenario). If this were true, then we would expect that teachers’ effectiveness in years following a reassignment would rise above their observed effectiveness in the year(s) prior to the move. Initial decrements to effectiveness may be outweighed by longer term student achievement improvements if teachers are systematically moving into positions in which they excel—a possibility we also explore in this article.
To better understand within-school churning, this study addresses three research questions:
Data and Sample
The data for this analysis are administrative records from a range of databases provided by the New York City Department of Education (NYCDOE) and the New York State Education Department (NYSED). It is worth noting that the New York City context—though important in its own right—may not be representative of other districts nationwide (a potential limitation we explore in greater detail in the conclusion). The NYCDOE data include information on teacher race, ethnicity, experience, school assignment, links to the students and classroom(s) in which the teacher taught each year, 1 and student achievement data. 2 The student data also include measures of sex, ethnicity, free-lunch status, special-education status, number of absences and suspensions in each year for each student who was active in any of Grades 3 through Grade 8 in a given year.
The NYSED also collects information from all public education employees through an annual survey and maintains a database called the Personnel Master File (PMF) which records information about job assignments, percentage of time allocated to each position, annual salary, age, gender, and experience. The PMF covers the time period from 1974 to 2010 (with the exception of the 2003 school year) and contains unique employee identifiers that can be linked to data on student achievement and schools in New York City.
Defining teacher transitions can be difficult because often researchers do not have complete information on the set of vacancies that need to be filled each year. Instead, we observe a series of yearly snapshots of teacher job placements at a given point in time based on the New York State PMF files. 3 We describe our approach in detail in Appendix A (available in the online version of the journal), but briefly summarize it here. When a teacher is classified as having a different subject–grade–school assignment in a given year than in the previous year, we refer to this as a “switch” or “reassignment.” We focus on four mutually exclusive switch types: (a) teachers who are new to their position because they are entirely new to the profession; (b) teachers who are new to New York City but not new to the profession; (c) teachers who appear in a different New York City school in year y versus y − 1; and (d) within-school switches—teachers who are in the same school but in a different subject 4 and/or grade from year y − 1 to year y. Many teachers, especially those in middle school, have multiple assignments. To be classified as experiencing a within-school switch, the teacher must have a different primary (i.e., greatest percentage of their time) subject- and/or grade-level assignment than the previous year in the same school (see Appendix A [available in the online version of the journal] for a complete discussion of how primary subject and grades were identified, as well as complications arising from ambiguous or missing information).
Population and Analytic Sample
The overall population for this article is the set of New York City employees who were ever classroom teachers in traditional public schools (i.e., noncharters) between 1974 and 2010 (271,492 unique teachers with over 2.4 million teacher-year observations—see row 1 on the left-hand side of Table 1). When examining impacts on student outcomes, we narrow the focus to teachers linked to student achievement outcomes—that is, those present in 1999 through 2010 in Grades 3 through Grades 8 (179,037 unique teachers with 1 million teacher-year observations—right-hand side of Table 1).
Sample Size Comparisons Based on Missing Teacher Switching Data
In our analyses, we exclude data from 2003 and 2004 due to an idiosyncratic problem with the teacher PMF file in 2003 (row 2 of Table 1). We also must limit the sample to the set of person-years in which we can observe an employee’s switch status. To identify a switch in a given school year, we must observe the subject or assignment type for person p in years y (current) and y − 1 (prior), the grade level (if applicable) in both years, the school of record in both years, each person’s current years of experience to identify teachers who are new, and years of experience within the district to identify teachers who are new to New York City. We are missing data on subject and/or grade assignment data for a subset of observations in the PMF (see row 3 of Table 1). Finally, as alluded to above, a teacher’s primary teaching assignment can be ambiguous, because her time may be divided equally among several classrooms. In these cases, it is not possible to determine whether a genuine switch has occurred since a single, definitive subject–grade assignment cannot be identified, and we lose some additional observations (see row 4 of Table 1). 5 In sum, due to these various data limitations, we lose a total of 18.7% of the teacher-year observations in the 1974+ sample; however, that translates into only 1.3% of the unique teachers in that sample as most teachers had at least one observed switch. In the 1999+ sample, we lose 22.6% of teacher-years to these various data limitations (the loss of 2003 and 2004 is disproportionately felt in this time frame), but again only 2% of the unique teachers from this time period (see row 4 of Table 1). 6
Methods
Research Question 1
For our first research question, we present descriptive statistics about the frequency of switch types across teacher-years. We also examine the timing of within-school switches throughout the average teacher’s early career. This allows us to determine whether being reassigned within schools is something that only some teachers experience or that virtually all teachers undergo, and whether it tends to happen more than once in the career. This will be germane to a subsequent analysis in which we examine the impact of a teacher’s initial switch on not only next year’s outcomes, but also for subsequent years before she switches a second time.
Research Question 2
For our second research question, we assess whether students who belong to historically underserved groups (i.e., non-White, low socioeconomic status, nonnative English speakers) are more likely to be assigned to teachers who are new to subject–grade, school, district, or the profession. An existing body of research has shown that students have differential access to teachers of differing levels of experience, value-added scores, and qualifications (Clotfelter, Ladd, & Vigdor, 2005; Goldhaber, Lavery, & Theobald, 2015; Isenberg et al., 2013; Kalogrides & Loeb, 2013; Kalogrides, Loeb, & Béteille, 2013). As some of this sorting exists within schools as well (see, for example, the work by Kalogrides & Loeb, 2013, in particular), one might also expect to see uneven assignment to teachers who are new to the profession/district/school/assignment, both within and between schools. Should we subsequently find that switching has a negative impact on student achievement, the answer to this question would provide evidence on the equality of educational opportunities within and across schools.
We are also interested in whether teachers who are new to their assignment in a given year tend to have other characteristics (in terms of the students they serve, their own characteristics, or the kinds of schools they work in) that might bias estimates of the effect of being new to assignment on student achievement if not accounted for in the estimation approach. It is difficult to establish a causal link between switching behaviors (new to teaching, a school, or a subject–grade assignment) and student achievement as many factors could be associated with both switching and student achievement. A few examples may prove useful here. For students within the same schools, teachers with more seniority often have more discretion in terms of the kinds of students and classes they teach. If more senior teachers can select to work with less challenging students and are also less likely themselves to change assignments, more challenging students may be systematically more likely to be exposed to switching teachers who are in turn more likely to be novice. At the teacher level, principals may try to move their struggling teachers around to find a better “fit.” Again, here we can imagine how a selection problem arises if struggling teachers also tend to experience more switching. In this scenario, reassignments would appear to be associated with lower student performance, but in fact the prior low performance is the cause of the reassignment, not the effect. Finally, at the school level, we know from prior work that teachers tend to leave schools serving disadvantaged and minority students at higher rates (Boyd, Lankford, Loeb, & Wyckoff, 2003). When teachers leave at higher rates, schools are likely to have to move teachers around and hire more novice teachers to replace them. Switch rates thus may be higher in schools serving historically underserved students, but it is often difficult to disentangle the impact of the switching itself from the fact that it happens more in schools that are likely to have lower student achievement for reasons unrelated directly to the churning. We explore these hypotheses to examine whether students, teachers, or schools might “select into” within-school churn at higher rates.
To estimate individual students’ probabilities of being assigned to a teacher who is new to her primary school–subject–grade assignment in a given year, we run three separate linear probability models for teacher-year level binary outcomes for each of four specific teacher switch types: (a) Teacher p switches subject–grade within same school or not (
We predict students’ assignment to teachers undergoing each of these four kinds of switches as a function of a vector of time-invariant student-level characteristics (
In the same vein, we explore whether certain kinds of teachers are more likely to churn (or be churned). We focus on within-school churns (
We predict a teacher’s probability of churning as a function of a set of time-invariant teacher-level characteristics (
Finally, we explore the possibility that certain kinds of schools engage in more teacher within-school churning than others. We calculate the churn rate for each school in each year (i.e., the percentage of the faculty in the given year who were teaching in the same school but in a different subject or grade in the previous year). Because churn rates in a given year may be somewhat unstable, we take the mean for each school across 3 years (2006–2007 through 2008–2009) and predict this mean within-school churn rate as a function of average school characteristics during the same time period. We can see whether, for instance, schools serving disadvantaged populations have less stability in teaching assignments from one year to the next. Again this is relevant for thinking about what potential confounding factors may be associated with both the treatment of interest (switching into a new assignment) and the outcome, student achievement.
Research Question 3
Ultimately, we are interested in whether the pervasive phenomenon of teacher reassignments—the four kinds of switches—appear to have a positive or negative impact on student achievement. Here we necessarily restrict our analysis to teacher-year observations linked to student achievement, and as such the sample now is limited to observations from 1999 to 2010 and in Grades 3 to 8. Recall that sample sizes are reported separately for this group in the right panel of Table 1, and rates of switching in lower panel of Table 2.
Frequency of Switching Overall, by Type of Switch and by Type of Within-School Switch
Note. NYC = New York City.
As previously stated above, establishing a causal link between switching and student achievement is difficult because students, teachers, and schools do not randomly experience reassignments. Many confounding factors may be associated with switching behavior and student achievement.
For these reasons, we take a number of different approaches to estimating the association between student achievement outcomes and teacher switching behaviors, in an effort to eliminate potential unobserved confounding factors. We begin with a basic education production function, in which all observable characteristics of students, classrooms, teachers, and schools are directly controlled.
In Equation (3),
The main predictors of interest are a set of four key dummy variables, which indicate the kind of teaching assignment switch a teacher experienced in a given year, if any. The first,
Though we have controlled for many factors that might confound the estimated impact of switching, we remain concerned that other unobserved factors may be associated with both switching behaviors and student achievement. We therefore also introduce a number of fixed effects to further isolate the switching behavior. For instance, in one specification we replace the teacher time-invariant characteristics with teacher fixed effects so that the coefficients on the switching predictors of interest become within-teacher estimates. That is, we examine whether student achievement scores appear to be lower for the same teacher in the years that she experiences a given switch, as compared with that same teacher in another year in which a switch did not occur. One might be concerned, for instance, that less effective teachers are more likely to be churned within school. The teacher fixed effects allow us to try to separate a teacher’s latent (time invariant) effectiveness from the act of switching. This is one of the preferred specifications, as we will see some evidence that assignment to particular positions within a school might be related to teacher characteristics. However, it is of course possible that some teacher-level confounders—such as teaching effectiveness—depend on circumstances that fluctuate from year to year and therefore would not be captured by the teacher fixed effects.
We also run the model with student, school, school × grade, and school × year fixed effects. Each of these has its own logic, isolating a source of variation that can be exploited to rule out a certain set of unobserved potential confounders. The student fixed effects, for instance, can eliminate any unobserved time-invariant student characteristics as a potential confounding factor for the analysis by examining how a given student performs in years in which his or her teacher experienced a switch versus years in which the student had a teacher who did not switch. This is a useful approach if we find that students are nonrandomly sorted to switching teachers, particularly if that sorting occurs among students within the same school. The student fixed effects approach remains vulnerable to unobserved, endogenous, time-varying factors.
The school fixed effects approach, on the other hand, makes comparisons among switching teachers within the same school. This is also a potentially compelling specification because teachers working within the same school are generally exposed to the same leadership, building-level assignment policies, student composition, and so on. However the school fixed effects do not account for time-varying characteristics of the school, nor any important within-school variation, for example across grades. We therefore also run School × Grade and School × Year fixed effects specifications, which further limit the within-school comparisons to particular grades, or particular years (to rule out, for instance, the possibility that some secular trends in the teacher labor market may confound the analysis).
Results
Research Question 1: How Often Do Teachers Switch School-, Subject-, and Grade-Level Assignments?
The movement of teachers to new teaching assignments is substantial (Table 2). Furthermore, the magnitude of the phenomenon is relatively consistent between the full 1974 to 2010 sample (upper panel of Table 2) and the restricted sample of teacher-year observations tied to student achievement in 1999 to 2010 (lower panel of Table 2). On average, 41.5% of teachers are switching in some way—either new to the profession, district, school, or their subject–grade assignment—each year (among the 1974–2010 sample). Of those switches, there are four mutually exclusive types of switches: (a) 15.4% are new teachers, (b) 6.2% are new to New York City but not teaching, (c) 24.9% are cross-school movers, and (d) the clear majority of switches (53.5%) take place within the same school. Thus, about a quarter of all teachers churn every year within their school into new subject–grade assignments. We can further break down the fourth group, within-school churns, into three subtypes: within-school subject switch only, grade switch only, or both. Here we find that most switches are across grade levels (68.3%), with the remaining 13.0% and 18.7% subject-only switches and both-grade-and-subject switches, respectively.
Switching of any kind is less frequent in elementary schools (36.2%), and somewhat more frequent in high schools (46.9%) than in middle schools (44.4%). Within-school churning is particularly prevalent in high schools, with 59.6% of all switches occurring within school. Although the within-school churn rate has fluctuated modestly over time, varying between 43% and 63% over the 36 years in the analytic sample (not shown, available upon request), it has always been the most dominant form of switching. Overall, within-school churn is approximately twice as likely as cross-school reassignments each year, yet to date very little attention has been paid to its frequency or impact.
In the lower panel of Table 2, we examine whether overall switching patterns are similar among the subset of teacher-years for whom we can conduct the achievement analyses for Research Question 3. By definition, the achievement analysis is limited to 1999 to 2010 and teachers linked to students (Grades 3 through 8). Overall, patterns are quite similar, with few notable exceptions: There appears to be a higher rate of new-to-profession teachers in the more recent achievement subsample (22.6%), and a corresponding lower rate of cross-school switches (16.0%). However, the overall within-school churn rate is quite similar (55.5% of all switches are within school). There are some differences in the kinds of within-school switches that are most common by elementary versus middle school as well; subject switches are more common in middle schools than in elementary schools, as one would expect. However subject-only and grade-only switches do occur in both elementary and middle school settings.
In describing the overall phenomenon of within-school churn, one natural question is whether this reshuffling occurs simply as a result of teachers departing from the school the previous year. Indeed, the correlation between the rate of teacher exits from a school and the subsequent year’s within-school churn is 0.45, which suggests that prior year departures tend to lead to current year teacher switches. That said, shuffling cannot be purely accounted for by new vacancies: For every teacher exit from a school last year, there are on average 4.3 teachers who switch assignments within school the following year (Figure 1). Therefore, replacing departing teachers is not a matter of simply moving or hiring one other teacher. Although most of the school-year observations are clustered near the median of 3.38 switches per exit, the spread in Figure 1 illustrates that some schools experience much greater switching than others. This provides some preliminary evidence that schools may engage in teacher reassignments differently from one another.
Distribution of ratio of this year’s switches to last year’s departures, across school-years.
Most teachers who remain in the system for multiple years will experience a switch. To report on the differential frequencies of switching, we examine the first 15 years of teachers’ careers to explore if they are switched, and if so how often. In Table 3, when we examine teachers during their first 2 years (row 1), about 76% have not yet experienced a within-school switch from year 1 to year 2, though about 24% do. In the second row, which examines teachers throughout the first 4 years of experience, we see that the number of teachers who have not yet churned within school drops to about 46.7%. So already by the fourth year of the career, teachers are more likely to have experienced a within-school churn than not. As teachers continue their career, they become even more likely to experience at least one (if not more) within school churns. Indeed, among teachers who are observed throughout the first 15 years, only 10.6% have never been churned within their school, whereas 53.8% of those teachers will have already experienced three or more churns. This suggests that, although there may be a small group of teachers who do not experience churn, most experience churn early in their career and more than one time. We also calculate for each teacher the average number of years between within-school switches. The mean is one switch every 5 years, the standard deviation is about 4, and to give a sense of the variability across teachers, the 10 to 90 range is once every 2 to 11 years. This corroborates the main takeaways from Table 3: The average teacher will experience multiple within-school switches if they remain in the district for long enough, but there are some teachers who experience less switching than others—a phenomenon we subsequently attempt to explore as a function of observed teacher covariates.
Percent of Teachers Who Experience 0, 1, 2, or 3+ Within-School Churns, Within Given Periods of Their Career
Note. Each row is limited to the set of teachers who are observed at least in their first X years of teaching, and the columns capture the number of switches (0, 1, 2, or 3+) that have occurred within those first X years.
Research Question 2: Are Students Who Belong to Historically Underserved Groups More Likely to Be Assigned to Switching Teachers?
Student-Level Analysis
Overall, there is some modest evidence that non-White, low socioeconomic status, and ELL students may be more likely to be assigned to switching teachers, in some cases even within the same school. In Table 4, we present results across eight models (each of the four switch types, both with and without school fixed effects). The constant in the model represents the probability of being assigned to a teacher experiencing the given switch type for a male, White student who is not FRPL eligible, who is not ELL and does speak English at home, with no prior-year absences and suspensions, and with average prior achievement (in other words, a relatively advantaged student). In column 1 for instance, we see that such a student has an 18% chance of being assigned to a teacher who is experiencing a within-school churn. The coefficients on each student characteristic represent a difference in probability of being assigned to a reassigned teacher in a given year relative to that more advantaged peer. The statistical significance levels are somewhat difficult to interpret given the very large sample sizes of students; therefore, for dummy predictors we focus on coefficients that represent at least a 1 percentage point difference in probability. Black students and Hispanic students are both about 3 points more likely to be assigned to a within-school churned teacher (column 1), and ELL-designated students are 5.4 percentage points more likely to be assigned to such a teacher. The magnitude of these coefficients is large relative to the constant, roughly a 20% increase for Black and Hispanic students and a 30% increase for ELL students. In column 2, we add the school fixed effects and generally find that most of the associations are no longer meaningfully large (i.e., smaller than a 1 percentage point change). The one exception to this pattern is that the ELL finding persists within schools (4.6 percentage points). It is possible this reflects the difficulty of recruiting and retaining ELL teachers, so ELL students may be more subject to staff instability than other students even within the same school.
Predicting Students’ Conditional Probability of Assignment to Switching Teachers, Based on Full Vector of Student Characteristics
Note. NYC = New York City; ELL = English language learner.
p < .05. **p < .01. ***p < .001.
Transfers between schools are less frequent than within-school switching and appear to have little association with student attributes (columns 3 and 4 of Table 4). Black and Hispanic students continue to exhibit a 1 to 3 percentage point higher probability of being assigned to a teacher who is new to the school, but those associations are not present once we add school fixed effects in column 4. Unlike in columns 1 and 2, the coefficients on the ELL predictor in columns 3 and 4 are not meaningfully large. Overall, there seem to be fewer differences across student demographics—both within and between schools—in terms of probability of being assigned to a new-to-school teacher than we saw for probability of being assigned to a churning teacher. However, there may also be a small, negative correlation between student prior achievement and probability of being assigned to a new-to-school teacher.
In columns 5 and 6, we examine predictors of assignment to a “new-to-district” teacher, but we find that this is both a relatively infrequent event and that there are few meaningful predictors of being assigned to such a teacher. Finally, in columns 7 and 8, we see that Black and Hispanic students have about a 3% higher probability of being exposed to brand new teachers, relative to an estimated constant of 7.1 percentage points (column 7, Table 4). A few other characteristics play a role here as well; students eligible for free lunch have a 1.6 percentage point higher chance of encountering a new teacher, whereas an increase in student achievement of one standard deviation reduces the likelihood of having a new teacher by 2.2 percentage points. In addition, the coefficient on students’ ELL designation in the new teacher model (β = 0.021 in column 7) goes in the opposite direction from the within-school churn model (column 1), suggesting that ELL students are slightly less likely to be exposed to new teachers.
Once school fixed effects are added (column 8), most of the differences observed in column 8 are quite small. The coefficients on ELL (β = 0.023) and prior year test scores (β = 0.019) persist within schools, suggesting that ELL students and students with lower test scores are less likely to have a new teacher when compared with similar students within the same school.
Taken together, these results suggest that historically underserved students may have somewhat higher probabilities of being assigned to within-school switching teachers, even when controlling for all other observed covariates and, in some cases, even when limiting comparisons to students in the same school. However, the magnitude of these differences is typically small. The largest estimated coefficient is about a 5 percentage point difference. These multivariate models set the stage for the fixed effects models employed to estimate the impact of switching on student achievement. 9
Teacher-Level Analysis
The analysis above suggests why it is important to account for observable student characteristics that may be both associated with assignments to teachers who churn, as well as student achievement. In the same vein, we explore whether female and minority teachers with different pathways into the profession, less experience, or lower value-added scores may be more likely to churn (or be churned).
In Table 5, we present results from three versions of Equation (2), in which we predict probability of experiencing a within-school churn
Predicting Teachers’ Probability of Within-School Reassignment, Based on Teacher Characteristics
Note. VA = Value Added; Std verb SAT = Standardized Verbal Scholastic Aptitude Test; Std math SAT= Standardized Math Scholastic Aptitude Test; TFA= Teach for America.
p < .05. **p < .01. ***p < .001.
Omitted categories include female teachers, White teachers, and teachers who attended an undergraduate institution that was “not” competitive and entered teaching through a traditional “college-recommended” pathway. The value-added score is the mean of math and ELA value-added scores (when both are available in the same year) from the year preceding the switch.
We are also interested in whether a teacher’s probability to be churned was related to his or her value-added scores in the year preceding the observation; however, only approximately 15% of the sample of all employees possesses these value-added scores. In columns 4 through 6, we added prior-year value-added scores
Controlling for other factors, there are some systematic differences in teachers’ propensities to be switched to a new assignment in their same school; however, the magnitude of these differences is typically not large. For instance, we see in column 1 that, although the conditional probability of a within-school switch is statistically different for male and female teachers, the difference is about half a percentage point (β = 0.006**). Again, for dummy predictors we choose to focus on relationships that are least 1 percentage point different in magnitude. When not including school fixed effects, Black and Hispanic teachers are 2 to 2.7 percentage points more likely to experience a within-school switch, and although the magnitude diminishes when we include school fixed effects (column 2), they do not disappear. In terms of teacher preparation, SAT scores are not a strong predictor, but we do see some 1-point differential probabilities by competitiveness of undergraduate institution (which persist in column 3 when school fixed effects are included). There are also some differences in conditional propensity to switch by teacher pathway: TFA teachers are 3.7 percentage points less likely to be switched than teachers entering the profession through traditional pathways (omitted category), whereas those entering through other (e.g., alternative certification) or unknown pathways are slightly more likely to be switched within school. Again, the findings on teacher pathway variables persist in the school fixed effects model, but are somewhat more muted. Finally, we see that there is a statistically significant but substantively weak, negative relationship between experience and switching (β = −0.001** in column 1), which suggests that, conditional on all other observed covariates, more veteran teachers are slightly less likely to be reassigned than similar teachers with fewer years of experience (results are similar when we include school fixed effects in column 3). It is interesting to note, however, that when we replace the time-invariant teacher covariates with the teacher fixed effects in column 2, the coefficient on years of experience reverses direction, though it remains substantively small (β = 0.004*** in column 2). Overall, we also note that readily available teacher covariates account for a small portion of the variance in probability of switching: The adjusted R2 from these models ranges from 0.002 (without fixed effects) to 0.118 (with teacher fixed effects).
Finally, we repeat these three models by adding teacher prior value-added (see Table 5, columns 4–6). Recall that these models are now essentially restricted to Grades 4 to 8 math and ELA teachers, by virtue of including value-added scores. Prior value-added scores are a significant predictor of propensity to churn: The higher one’s value-added, the less likely they are to churn (β = 0.072*** in column 4), even when comparing teachers in the same school (β = −0.070*** in column 6). It is interesting to note, however, that when we examine the results from the model that predicts outcomes by prior value-added scores with teacher fixed effects included in the model, no relationship persists. In other words, value-added scores do not appear to predict why the same teacher is assigned to switch assignments within school in some years but not others. Columns (4) through (6) that include prior value-added have only slightly higher adjusted R2 values (0.005 without teacher fixed effects and .150 with teacher fixed effects) than models presented in columns (1) through (3) without value-added.
Taken together, these results suggest that teachers may be systematically targeted for reassignment both within- and between schools. Teacher race/ethnicity is a persistent predictor of propensity to be reassigned in all models. The relationship between years of experience and reassignment depends on whether looking within or across teachers, and whether one also controls for prior value-added. Prior value-added is also related to propensity to be reassigned, except when looking within teacher. The covariates in Table 5 will be included as controls in the subsequent models used to isolate exogenous variation in reassignments, so we do not have to be concerned specifically about these factors biasing our estimates. However, we are concerned that, if teachers are systematically reassigned based on the things we do observe, there may be other teacher-level endogenous variables that we do not observe that cannot be included directly. For this reason, teacher fixed effects may prove a particularly important specification of models used to link reassignment to impacts on student achievement.
School-Level Analysis
We find some evidence that schools that serve higher percentages of Black students, ELLs, or students with higher rates of suspension or absenteeism also tend to exhibit more within-school churn (see Table 6). For instance, a 1 percentage point increase in the number of Black students in the school is associated with a 0.037 percentage point increase in the churn rate (statistically, but likely not substantively significant). It does appear that, conditional on other school factors, schools with high rates of absenteeism and suspensions exhibit great within-school switch rates.
Three-Year Average Within-School Churn Rate, as a Function of Average School Characteristics
Note. ELL = English language learner.
p < .05. **p < .01. ***p < .001.
Predictors are school-level 3-year means (2007–2009), expressed as percentage points on a scale of 0 to 100. The outcome is the 3-year mean churn rate in the school (2007–2009), also expressed as percentage on a scale of 0 to 100.
Overall, there is some evidence that historically underserved groups of students are more likely to be assigned to switching teachers (even within the same school), certain kinds of teachers are more likely to be switched, and certain schools may experience greater degrees of switching; however, these relationships tend to be weak. These findings have two potential implications. The first is that it may be difficult to isolate the impact of churning from the fact that this behavior appears to be nonrandom—an issue we take up in the next section. The second implication is that, if we do find evidence of negative impacts of these various forms of being new to one’s assignment, some students may be more likely to experience those negative effects.
Research Question 3: What Is the Impact on Students of Being Assigned to Switching Teachers?
Switching teacher assignments negatively affects student achievement across all four types of switches. Table 7 presents results for student achievement outcomes in math (top panel) and ELA (bottom panel). Given that the conceptual model suggests that “newness” and “unfamiliarity” might be the primary mechanism driving a negative impact of switching, the relative magnitude of the results seems reasonable: Brand new teachers are new to all aspects of their assignments—the job itself, the school, the colleagues, as well as the specific class itself. Therefore, we are not surprised that achievement is lowest when assigned to a brand new teacher. Teachers who are moving across districts or schools, on the other hand, are confronting new circumstances and social norms, but they are not new to the act of teaching and thus we would expect the negative impact of this form of “newness” would be relatively less strong than being completely new. Finally, teachers who churn within the same school are not new to the school culture, but their particular subject–grade assignment, responsibilities, and immediate subject- or grade-level assignments have changed. The results suggest that the more aspects of one’s subject–grade–school assignment are unfamiliar, the more negative the impact of the reassignment. Results are relatively consistent across all model specifications with various fixed effects in math. For instance, the coefficient on the indicator for within-school churn is consistently between −0.010 and −0.018 and statistically significant in all models. Though the magnitude of these effects is small (on average, about a quarter of the size of the effect of having a new teacher), keep in mind that nearly four times more teachers are new to assignment than new to the profession each year. Indeed about a quarter of all teachers are reassigned within school each year, thus making the aggregate effect on the distribution of student achievement notable. Results are also negative for ELA outcomes (lower panel of Table 7); however, the coefficients on the within-school churn variable are closer to −0.004 to −0.11 (and not statistically significant in the model with teacher fixed effects).
The Impact of Four Switch Types on Student Math and ELA Achievement, Across Model Specifications
Note. ELA = English Language Arts; NYC = New York City.
p < .05. **p < .01. ***p < .001.
All models shown here have time-varying and time-invariant student characteristics, aggregated time-varying classroom covariates, teacher time-invariant characteristics and time-varying years of experience, and school time-invariant and time-varying characteristics (except when collinear with the relevant fixed effects).
It is worth noting that the estimates of within-school switching are smallest in models that include teacher fixed effects, conditional on teacher experience. This may reflect the fact that teachers are not equally likely to experience a switch, and there may be unobservable differences among them, conditional on the covariates included in these models. 10 As models that include teacher fixed effects ensure that estimates are not confounded with unobservable (time invariant) teacher characteristics, some may prefer estimates from this model. However, from a policy perspective, it may not be entirely desirable to isolate the effect of switching from the characteristics of teachers who switch: From the student’s perspective, this phenomenon is quite pervasive, and their experience of the switching teacher includes that teacher’s other qualities.
Recall that about 20% of the person-years in the data set do not have a clear “primary” subject–grade level assignment. We conduct a bounding exercise related to these ambiguous teacher-year observations and find that our results are robust to the various assumptions one could make about the status of those unknown cases (see Appendix D, available in the online version of the journal, for descriptive of approach and presentation of results).
Is It Harder to Switch Subjects, Grades, or Both?
To further probe the nature of the negative impact of within-school churning, we hypothesized that switches might be more challenging for teachers when they were more dissimilar to the prior year assignment. For instance, it might be the case that it is more difficult to switch both subjects and grades simultaneously rather than just switching one or the other. To explore this, we further subdivided the within-school churn indicator into three distinct subcategories (a) a within-school switch of subject only (grade remained the same), (b) a within-school switch of grade only (subject remained the same), and (c) a within-school switch of both subject and grade. In essence, we ran Equation (3) with six dummy variable predictors of interest rather than four, in which the indictor of within-school churn
Of the within-school switches, 71% were a grade switch only, 14% were a subject switch only, and 15% were both (refer back to Table 1). Although it is straightforward to think about scenarios in which teachers switch grades only, it may be less clear what kinds of transitions are captured by the “subject-only” switch category—that is, teachers remaining in the same grade and school but teaching a different subject. Indeed, this is the least common form of within-school switch. Many of the subject-only switches are characterized by teachers who were assigned to a grade-specific “English as a Second Language” classroom, or a “Special Education” classroom in the previous year but now are in ELA, math, or elementary (i.e., whole classroom) positions in the current year. We also see teachers who were previously teaching a nontested subject to a specific grade (e.g., fine arts, science, foreign language, or social studies) who now primarily teach math, ELA, or elementary students in the current year. One might be concerned that subject-only switches only occur in some grades, thus limiting those analyses to specific grade levels. However, subject-only switchers are approximately evenly distributed across grades, with the exception of Grade 6, which has about twice as many subject-only switchers as any other grade.
Switching both subjects and grades at the same time is more difficult than just switching one or the other. Table 8 presents the results for this analysis for math achievement outcomes for just three specifications of the model—with teacher (M2), school (M3), or student fixed effects (M6)—for the sake of parsimony. According to the model with student fixed effects (final column), switching both subject and grade is associated with a −0.023 decrease in student achievement, whereas switching subjects only was associated with a −0.010 decrease, and switching grades only was associated with a −0.019 decrease. Results for Model 2 (teacher fixed effects) and Model 3 (school fixed effects) also show that switching both subject and grade may be slightly more negative than switching only one or the other, though the magnitude of all coefficients is again smallest in the teacher fixed effect specification. Taken together, these findings suggest that the phenomenon may operate in a way that is consistent with a conceptual frame of newness—when both subject and grade level are new, the challenge of teaching may be greater when either the approximate age or the subject matter has not changed.
Effects of Different Kinds of Within-School Switches: Subject Only, Grade Only, or Both
Note. NYC = New York City; ELA = English Language Arts.
p < .05. **p < .01. ***p < .001.
Is the Impact of Switching Temporary?
When thinking further about our descriptive findings that teachers appear to be reassigned within their school multiple times during their career, we wondered about whether the impact of switches might be temporary—that is, strongest in the year in which the teacher was new to the school, subject, and/or grade. We imagine three possible scenarios for what we might observe. First, it is possible that switching teachers may have a temporary cost in terms of teacher impacts on student achievement in the year of the switch, but ultimately these switches might lead teachers to find a better fit between their own strengths and their teaching assignment. In this scenario, we would expect to find that student achievement scores drop in the year of the switch itself; however, in subsequent years the teacher’s students’ scores would exceed preswitch levels. A second possibility is that switches are less strategic and more random. In this case, we would expect to find that scores drop in the year of the switch, but in postswitch years teachers simply revert back to their preswitch achievement levels. In other words, there is nothing about the switch experience that systematically improves the teacher’s ability to improve student learning. The third possibility is that switching is a negative experience with lasting negative impacts on teachers. If this were the case, we would expect to find that, after student test scores drop in the year of a switch, they do not return to preswitch levels afterwards.
To examine these competing hypotheses about the lasting impacts of switching behavior, we use the education production function framework from Equation (3) but change the coding scheme to reflect whether each student was assigned to a teacher who switched (a) in the current year, (b) last year, (c) 2 years ago, or (d) 3 or more years ago. The omitted category then becomes expected achievement outcomes for students in years that predate the first reassignment. Furthermore, we limit the sample here to the set of teacher-year observations that occur 1 year prior to a teacher’s first within-school switch and 1 year before a second switch occurs. Because teachers switch many times in their career on average, midcareer years can ambiguously be classified as either post- one switch, but simultaneously pre- the next switch. Imagine, for instance, that a teacher is reassigned within the school in both her third and fifth years on the job. The fourth year could be considered the year after the first switch, but also the year before the next switch. Limiting the sample in this way allows us to isolate a subset of teacher-year observations in which the temporal pattern of switching is unambiguous; however, it also narrows the focus to the effects of the first time a teacher is switched.
Results in Table 9 differ somewhat depending on model specification. As before, we see that there is a negative decrement to student achievement in the year a teacher is reassigned. However, the coefficients on years subsequent to the switch are less consistent across models. Although the coefficients tend to be positive, suggesting that the teachers’ students are performing better than they had in the year before the switch occurred, those differences are significant only in the models with School × Grade, School ×Year, and student fixed effects. In this temporal exploration, the specification with teacher fixed effects is perhaps most straightforward in terms of thinking about a teacher’s pattern of switch behavior from one year to the next. In that version of the model (column 2), there do not appear to be any statistically significant differences between preswitch and postswitch student outcomes. The lack of positive increases postswitch suggests that—however decisions are made about shuffling teachers within the same school—these movements do not appear to match teachers to subject–grade assignments in which they are more effective.
The Temporal Impact of Within-School Switching on Math Achievement
Note. Sample: Teacher-year observations that occur 1 year prior to a teacher’s first within-school switch and one before a second switch occurs. All models shown here have time-varying and time-invariant student characteristics, aggregated time-varying classroom covariates, teacher time-invariant and time-varying characteristics, and school time-invariant and time-varying characteristics (except when collinear with the relevant fixed effects). The constant in these models is not directly interpretable given the variety of time-varying and time-invariant student, teacher, classroom covariates, and the hold-one-out reference categories for the relevant teacher fixed effects.
p < .05. **p < .01. ***p < .001.
Conclusions
This article documents a phenomenon that most practitioners understand but that education researchers have largely ignored: the incredible prevalence of annual within-school reassignments to new teaching positions. We have situated this phenomenon within a larger body of work that examines other instances in a teacher’s career when he or she is new to their teaching assignment—either in the first year on the job, new to the district, or when teachers move across schools from one year to the next. All of these switch types share a common theme—it is more difficult to be effective at complex tasks when the task or context is unfamiliar. We contribute to this body of work by documenting that within-school switches in New York City are twice as common as between-school switches, and nearly four times more likely as being new to the profession. We also find that there is a modest negative impact of being assigned to teachers when they are new to teaching, the district, the school, or their subject–grade assignment. The relative negative impact of these phenomena follows a pattern that suggests that the more “new” the teaching assignment is, the more challenging the teaching may be in a given year: The impact on student achievement is most negative when students are assigned to brand new teachers, followed by teachers who are new to the district or school, and finally (least strongly but still negative) to teachers who are in the same school but new to their subject and/or grade.
The estimated impact of within-school churn is not large in absolute terms. However, given that about a quarter of all teachers each year are churning within the same school, these small negative decrements add up: The estimated impact of churning is, on average, about a quarter of the size of the impact of being assigned to a brand new teacher—a phenomenon that has received a great deal of attention in the field. However, in any given year, more than nearly four times as many students will be assigned to a churning teacher than a new teacher, in essence quadrupling the overall impact on the distribution of student achievement. Stated another way, the average student only encounters one brand new teacher between Grades 3 through 8, but four or five churning teachers in the same time frame. Furthermore, we find some evidence that some schools experience more of this churn than others, and one might be concerned that schools serving disadvantaged populations of students are also the schools most likely to have instability in their teacher assignments. Our analysis also suggests that even within the same schools, historically underserved student groups may be more likely to be assigned to churning teachers than their more privileged counterparts: While the average student has about a 24% chance of being assigned to a churning teacher in any given year, a White, male student who is not FRPL eligible, is not an ELL student, and has not been suspended only has an 18% chance of being assigned to a churning teacher. Taken together, the results of the current article suggest a widespread and understudied phenomenon that negatively affects the students of almost all teachers at some point in their career, and disproportionally affects disadvantaged students.
It is important to acknowledge that this article focuses on a particular context: New York City. It is not necessarily the case that findings regarding the frequency or impact of switching would be similar in smaller or less urban districts. Although we do not have access to achievement outcome data outside of New York City, we do possess information about the teaching assignments of teachers across the entire state since 1974. We therefore calculate the average within-school switch rate for each district in New York State. In Figure 2, we present the distribution of those within-school switch rates across districts to see where New York City falls. One can see that New York City’s reported within-school switch rate (vertical line at 22% per year) is toward the high end; however, the average within-school switch rate is about 15% among districts with at least 10,000 students. It turns out that 23 other New York State districts have a higher average within-school rate than New York City, though the majority have lower rates of within-school movements. We further hypothesized that teacher movements might be less frequent in smaller districts and rural districts. In Table 10, we therefore also present the average district-level rates of new-to-profession, new-to-district, new-to-school, and new-to-assignment (i.e., within-school) switches for other New York state districts of different sizes and different geographic types (city, suburban, town, and rural). For reference, the New York City rates are reported at the bottom of Table 10. Indeed, it is the case that fewer switches occur in rural districts than in New York City. However it appears that switch rates in other non–New York City districts that are large and urban exhibit are nearly as high as in New York City. Our findings may therefore generalize more to these kinds of environments, rather than smaller districts in towns or rural areas. However in most kinds of districts shown in Table 10, between 30% and 40% of all teachers experience some kind of switch every year. This suggests that these movements affect districts of all size, though perhaps to a lesser degree than in New York City. A brand new article that examines the frequency specifically of grade switching (both within and across schools) in a large California district was recently published (Blazar, 2015). 11 Findings from that article are consistent with ours with regard to the surprising frequency of assignment switching (in their case, particularly grade switching). This suggests that assignment instability is a prevalent phenomenon outside the New York setting.
Distribution of within-school switch rates across New York state districts with at least 10,000 students (outside of New York City).
Number (Percentage) of Switch Types in Other (Non-NYC) New York State Districts, by Average District Size and Urbanicity
Note. NYC = New York City.
This article generates several questions. Although we conclude that the average impact of within-school churn appears to be negative, it is not clear whether that average effect is a relatively accurate description of the effect in all places, or instead whether the impact varies dramatically perhaps from one school to the next. We hypothesized that some teacher reassignment could be beneficial for students if these decisions are made strategically to optimize what and where teachers teacher. However, in the current data we have no way to differentiate discretionary movements intended to either improve student outcomes (e.g., I think teacher A will work more effectively with older students) or to satisfy teacher requests for certain types of students or subject matter from unavoidable staffing driven movements (e.g., the need to replace exiting teachers or there are more fourth graders this year than last year and so we need to move some teacher into fourth grade). One might hypothesize that some school leaders may develop strategies around reallocating teachers that benefit students. Again, this is difficult to observe in the current data, as we have relatively shallow insight into how individual schools are managed. In results not shown here, we conducted preliminary analyses to explore whether the impact of churn was different for schools in the top and bottom third of distributions on various student characteristics (i.e., schools in the top third of math performance vs. the bottom third). In none of these top- versus bottom-third comparisons were the impacts of churn positive, nor were the group differences statistically significant from one another. The lack of differential impact across these groups is only a first step toward trying to identify places where within-school reassignments are conducted in strategic ways that benefit students. Administrative data alone provides relatively blunt ways of characterizing schools, and these demographic dimensions may fail to help us account for any variability in the effect of churn across schools. Future work in this area might generate and test hypotheses for school characteristics that could cause or support beneficial within-school churn.
We end with a final word on the policy implications of the current analyses. Of course, it is impractical to imagine that within-school churn can or should be eliminated by policy. Indeed, it is an unavoidable artifact of such a large system that instability can and will occur. The current findings do highlight just how much of that switching is taking place on an annual basis: A full 40% of all teachers are new to the district, the school, or their subject–grade each year, and half of those switches occur within school. If our findings are corroborated in other districts, it may be the case that school administrators should recognize that reassigning a teacher will have a small, negative impact on students, and that exposing students to high doses of this churning could more meaningfully influence their achievement. This recognition may cause schools and districts to temper the level of discretionary churning. Future research could collect more nuanced data to classify different types of churning and better understand whether discretionary churning benefits students.
Footnotes
Acknowledgements
We are grateful to the New York City Department of Education and the New York State Education Department for the data employed in this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We appreciate financial support from the National Center for the Analysis of Longitudinal Data in Education Research (CALDER). CALDER is supported by IES Grant R305A060018. Support has also been provided by IES Grant R305B100009 to the University of Virginia. The views expressed in the article are solely those of the authors. Any errors are attributable to the authors.
Notes
Authors
ALLISON ATTEBERRY is an assistant professor at the School of Education, University of Colorado Boulder. She specializes in evaluating the effects of policies and interventions that are intended to provide effective teachers to historically underserved student populations.
SUSANNA LOEB is the Barnett Family Professor of Education at Stanford University. She specializes in education policy with a focus on school governance and finance and educator labor markets.
JAMES WYCKOFF is a Curry Memorial Professor at the Curry School of Education, University of Virginia. His research focuses on teacher labor markets and policies intended to improve teacher quality and student outcomes.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
