Abstract
The purpose of this article is to bring attention to the illustrative power and capacity of qualitative longitudinal research within the context of the urban educational “reform churn.” In this article, we draw upon longitudinal ethnographic data collected over 3 years in four low-performing, urban secondary schools in Buffalo, New York, to illuminate the power of the methodology behind our findings. Our research provides an example of the ways in which qualitative longitudinal research can reveal the temporal mechanisms of stabilization and destabilization that develop within urban schools and districts in and through time.
When you get into a repetitive failure cycle people begin to believe that they cannot control or they cannot impact the outcomes for which they have been asked to work. That creates kind of a mentality of this can’t be done. So what we have been able to do, I have seen a cultural shift in that. We are not where we want to be, but because we have been making improvements, because we have been achieving and gone from the second lowest performing school to the seventh out of sixteen in the area, there is starting to be a light again in the mindset of our faculty, in our administration, and even in our students as well. They are seeing it in data and the data is showing what we are doing is working.
The historical and contemporary landscape of American urban educational reform reflects a pattern of unrelenting, continuous change—what Diane Ravitch (2011) refers to as the “constant reform churn” (p. 224). This constant upending churn prevents often well-conceived reform from taking root and preserves a kind of systemic inertia—one that effectively churns inequality. Urban schools, 1 in particular, have been the focus of continued, often rapid reform efforts that have left many students, a large percentage of whom are low-income students of color, with even fewer opportunities and stagnated outcomes—further depressing the achievement of underserved students and widening extant gaps (Carter & Welner, 2013). Because reforms have been proposed at a particularly rapid rate within urban contexts (Henig, Hula, Orr, & Pedescleaux, 2001), as well as within districts or schools that are characteristic of those located within larger cities (see Milner, 2012a), urban schools continue to be most subjected to the pendulous nature of reform. As a consequence, urban schools in particular are left scrambling to adapt to newly imposed changes, meet newly established aims, and produce meaningful gains under unrealistic expectations and time constraints, only to be faced with calls for subsequent reform. Although not entirely predictable, this pattern of churning reform—often influenced by larger social and political changes—demands that educational researchers account for the temporal realities of urban schooling, particularly as they play out in the lives of those most affected by reform efforts.
Within urban districts and schools, time itself and the stabilizations or destabilizations that occur over time become critical to our understanding of the complex spaces within which reform is implemented. Because of the temporal patterns specific to the cycle of educational reform that largely affect urban schools, the absence of time as a dominant feature of our research can lead to an incomplete picture, or worse, misleading accounts of success and/or failure within schools, districts, and communities—accounts that assign blame and fail to critically examine the broader social, historical, cultural, and economic contexts. The consequences of our research practices as they inform or misinform policy can serve to reinforce current patterns and exacerbate educational inequalities. This is especially important within urban contexts where there is a notable crisis relative to the “failure of research, theory, and policy to make significant strides in impacting urban schools” (Milner & Lomotey, 2014, p. xvii).
The purpose of this article is to bring attention to the illustrative power and capacity of qualitative longitudinal research within the context of the urban educational “reform churn.” In this article, we draw upon empirical data collected within “urban emergent” (Milner, 2012a) secondary schools to make a methodological argument for designing research that has the potential to advance the field of urban education. Although researchers have made significant contributions to our understanding of urban educational reform through the use of longitudinal approaches, not enough attention has been paid to the ways in which and the extent to which methodological choices led to the findings of these contributions. In other words, qualitative longitudinal research has yielded important data, but we know little about why this methodological research is well suited to the study of the educational reform churn and how others might go about replicating such a design within similar contexts. By drawing upon longitudinal ethnographic data (a form of qualitative longitudinal research) collected over 3 years in four low-performing, urban secondary schools in Buffalo, New York, 2 we take the reader in and through the educational reform churn to illuminate the power of the methodology behind our findings. Our research provides an example of the ways in which qualitative longitudinal research can reveal the temporal mechanisms of stabilization and destabilization that develop within urban schools and districts in and through time. We hope that by spotlighting the case as well as the methods behind it, we can encourage more researchers to take up qualitative longitudinal efforts when studying urban school reform.
We begin this article with a discussion of the conceptual framework of the temporal patterns of the urban educational reform churn and follow with a brief discussion of existing longitudinal approaches that have enhanced our understanding of educational reform. We then use our research within four Buffalo public schools as an example of the illustrative power of qualitative longitudinal research. We conclude with a discussion of this particular methodology in relation to the overwhelming evidence that suggests that its benefits supersede the challenges and limitations associated with this approach.
Framing the Urban Educational Reform Churn: Why Time Is of the Essence
In echoing Ravitch’s notion of “reform churn,” Cuban (2010) writes: “For the past half-century, urban districts striving to ‘fix the system’ have been afflicted with constant policy churn and personnel turnover” (p. 155). Similarly, Elmore (2004) describes local reform initiatives as “typically characterized by volatility—jumping nervously from one reform idea to the next over relatively short periods of time—and superficially—choosing reforms that have little impact on instruction or student learning and implementing them in shallow ways” (p. 2). Together, these insights project a consistent image of reform churn—a cycle of ineffectual, often failed efforts with “a ‘new’ reform always ready to replace an older discarded reform” (Peck & Reitzug, 2014, p. 12).
Reasons for this trend are numerous and are situated within a complicated nexus of social, cultural, economic, political, and historical factors that can undercut even the most promising of reform efforts. Factors include, but are not limited to the growing disinvestment in public education and the professional development of teachers and administrators, changes in superintendents and school boards, and what Darling-Hammond (2010) calls “long-standing traditions, standard operating procedures, and expectations . . . sometimes, the expectation that the students who have traditionally failed should continue to do so . . .” (p. 265).
Within this context, time is of the essence—as “short timelines for producing demonstrable improvements put a premium on swift and confident action” (Honig, 2013, p. 3). For example, “superintendents’ average tenure is between eighteen months and three years, and school board membership turns over regularly” (Elmore, 2004, p. 2). As Hess (1999) explains, “The short-term emphasis of urban district management, aggravated by rapid leadership turnover and overburdened reform agendas, make the thorny problems of implementation and fostering long-term organizations development nearly impossible” (p. 152). In these ways, the temporal pattern of urban education reform not only sparks and sustains many cycles of “failure” but also allows the blame for those failures to be placed on particular individuals and groups, rather than larger systemic issues such as institutional racism and poverty.
Sudden and dramatic educational reform measures, such as school turnaround efforts that too often lead to the same conditions they sought to ameliorate, largely affect urban public schools (Henig et al., 2001; Peck & Reitzug, 2014; Trujillo, 2012) wherein educational reform legitimated by neoliberal logic is pushed forward at an accelerated pace (Buras, Randels, ya Salaam, & Students at the Center, 2010; Lipman, 2011, 2013; Oakes, 2015; Saltman, 2007). This neoliberal logic is particularly harmful to our nation’s urban schools as it operates under a set of corporate, market-based values that favor the dismantling and privatization of public schools, school choice, increased reliance upon standardized testing and accountability measures, and “shock therapy” (Johnson, 2012) or rapid turnaround strategies for schools deemed failing. Rather than strengthening the democratic purpose of public education, this logic intensifies competition for resources and creates a seemingly meritocratic system wherein those who succeed work harder while those who fail are undeserving (Costigan, 2013). In doing so, neoliberalism intensifies the educational reform churn within urban schools.
In the midst of this reform churn, methodological decisions and practices matter a great deal. Studying the temporal mechanisms of stabilization and destabilization that affect particular schools and subsequent reform efforts demand approaches that ground research in institutional culture and the lives of those most affected by reform, serving to “texturize what statistical data reveal” (Scott & Solyom, 2011, p. 12) over time. Ideally, such an approach would be both deeply qualitative and longitudinal.
However, as reforms continue to churn, it is more common to see qualitative studies, including ethnographies, present “snap-shots” based on the short term—a kind of “freezing . . . characteristic of ethnographies conducted at one point in time” (Weis, 2004, p. 190). While we gain important insights from these studies, they neglect the critical element of time within spaces that often experience rapid and radical change. The element of time strongly defines the nature of urban education reform and the lives affected by these ebbs, flows, and upending waves of change. For example, in between moments characterized by patterns of intense change, we may see periodic lulls wherein deeper instability becomes cloaked by temporary stasis. In other words, a snapshot of a school in the early stages of reform may provide a misleading sense of stability and sustained positive change, while a longer lens may reveal the consequence for schools caught up in reform churn—the great potential to look drastically different from one year to the next. Thus, the element of time becomes central to understanding the inability to sustain reform efforts within the larger political, economic, and social context. However, it is not just the passage of time that is important to analysis but also the accumulations, recursions, connections, relations, and stabilizations and destabilizations that develop within urban schools and districts as time passes.
Within the space of urban education reform, qualitative longitudinal research allows for researchers to document change over time and more fully assess the “success” or “failure” of the reform as it plays out on-the-ground, in actual practice and in the lives of students, families, teachers, and school personnel. It permits a deep analysis of the structural drivers at work while examining the ways in which policy edicts—often created by those most removed from the schools and communities in which they are implemented—are enacted. This kind of methodological approach carries great potential to combat current and popular discourses that blame individuals (school and district leaders), groups (students, parents, teachers), and schools for academic “failures” (Aviles de Bradley, 2015) by highlighting the fissures between unrealistic expectations and practice.
A Review of Longitudinal Research on Urban Educational Reform
Qualitative longitudinal research is essential if we are to uncover the structural sources of reform inequity. As Carter and Welner (2013) argue, shortsighted studies of educational reform may “cause us to condemn our schools and teachers unfairly” (p. 67). While a number of studies on urban educational reform rely upon quantitative longitudinal research designs to assess and evaluate efforts over time (see, for example, Balfanz, Herzog, & Mac Iver, 2007; Burks & Hochbein, 2015; Stuit, 2010), they reveal little about how and why reform is either successful or ineffective over time within urban districts and schools.
Although fewer in number, longitudinal designs that have incorporated qualitative methods have exposed a great deal about how and why things change, often drastically, over time as a consequence of educational reform (Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010; Darling-Hammond et al., 2005; Datnow, Borman, Stringfield, Overman, & Castellano, 2003; Stringfield & Yakimowski-Srebnick, 2005). In this section, we highlight selected studies that demonstrate the complexity of urban educational reform and its variability year over year, providing clear and compelling qualitative evidence and the need for more longitudinal research in reform conversations.
In a study of 13 schools over 4 years of comprehensive school reform efforts (CSE), Datnow et al. (2003) used both qualitative and quantitative methods to uncover “the conditions at the state and district level that facilitated or inhibited the implementation of the reforms” and better understand “how the reforms helped or hindered educators’ efforts to address the linguistic needs of their student populations and to foster positive cultural identities for their students” (p. 151). Specifically, from interviews with school staff, researchers were able to elicit in-depth responses to how reforms were either ineffective or sustained over time. Interestingly, in combination with observation data, researchers also revealed that although two schools were purportedly part of the Coalition of Essential Schools, reform efforts were “virtually absent in practice” (Datnow et al., 2003, p. 153).
Studies by Stringfield and Yakimowski-Srebnick (2005), Bryk et al. (2010), and Darling-Hammond et al. (2005) also examine district-level reform efforts over time in Baltimore, Chicago, and San Diego, respectively. In these studies, the combination of longitudinal quantitative and qualitative data reveals emergent complexities relative to context that helped researchers to better understand why certain reform efforts were deemed unsuccessful in improving some schools and successful in others. In addition, these studies recognize the fragility of reform arguing that future forces could threaten to destabilize current efforts that were seemingly successful.
These studies are important because they each make the case for longitudinal designs that fully account for change over time. Darling-Hammond et al. (2005) demonstrate the importance of methodological depth and breadth in studying the complexity of change within and through time in urban schools subject to reform initiatives, while Bryk et al. (2010) argue for careful consideration of the potential effects produced by the “underlying temporal dynamics of school improvement” (p. 90). The authors acknowledged that changes over time (e.g., a change in administration) could produce misclassification in their prediction of reform success or failure. For example, “a school that appeared weak organizationally in 1994 could subsequently take off if, for example, a new principal was appointed” (p. 90). In this case, data collected at a particular point in time, though accurate, would be a poor indicator of the effects of reform in the long term.
Although this review is not an exhaustive list, the notable studies above indicate the importance of qualitative longitudinal research in capturing the patterns, changes, deeper meanings, and layers of context along a necessary temporal continuum. Each study improves our understanding of urban school reform and makes the case for the need to elicit rich, textural, qualitative data over time relative to the ways in which reform plays out on the ground. We further contribute to this conversation by offering a description of how the longitudinal and qualitative components of the research design were able to yield more meaningful findings. In the following, we use our research to take the reader in and through the educational reform churn to illuminate the power of the methodology behind our findings.
A Longitudinal Ethnography of Buffalo Public Schools
In this section, we provide details on our research methodology. 3 Because history and context are so integral to longitudinal ethnography, we begin with a description of Buffalo and its public schools, the majority of which might be called “urban emergent” schools (Milner, 2012a). These types of urban schools face many of the same challenges experienced by urban “intensive schools” (schools located within the largest, most densely populated metropolitan areas within the United States), but on a smaller scale. We then discuss our longitudinal ethnographic approach and the methodological choices made in the design phase of this study. We also aim to provide sufficient detail relative to what we did, why we did it, and how we did it, in the interest of transparency and potential replication for future research on urban education and urban educational reform.
Buffalo and Its Public Schools
Buffalo’s most recent history has been defined by economic decline, population loss, and segregation. It has been consistently ranked among the poorest of America’s large cities with just fewer than a third of the population living in poverty. Main Street, the dividing line, separates the almost exclusively Black East Side (97% African American) from the significantly more diverse West Side, resulting in Buffalo designation as one of America’s most segregated cities. The West Side is home to a burgeoning refugee and immigrant community (Burmese, Sudanese, among others), a multigenerational Puerto Rican community, and an affluent White community.
There is evidence to suggest that Buffalo’s economic situation is improving and that its demographics are changing. Recent and significant capital investment by the State, coupled with local efforts, has brought renewed interest to the city. Early indicators suggest that the city population may be stabilizing—possibly even increasing, mirroring national trends. As Buffalo attempts to rewrite its past there have been renewed efforts to reform schools. In particular, private–public partnerships between foundations, private organizations and not-for-profits have been established with the expressed purpose of bring back and keeping middle-class (White) families into the city schools.
Buffalo has a relatively long history with reform efforts. In fact, at one time, Buffalo was a model for school reform—“School Integration in Buffalo Is Hailed as a Model for U.S.” boldly proclaimed the New York Times in 1985. 4 Relying upon a system of magnet schools and bussing, the Buffalo Public School (BPS) District abandoned its system of neighborhood schools and adopted a choice model in an effort to increase access to high-quality education for all students in integrated spaces. By most accounts, the efforts were successful—test scores improved across the district and school segregation decreased—so successful, in fact, that by 1987 (only 7 years after court-mandated integration began), court oversight was lifted. Today, the choice model remains, however schools have returned to preintegration segregation levels and educational quality and access vary dramatically across the district. In fact, as of 2014, almost 80% of the schools in the district were now considered “failing” by the New York’s Department of Education. The few high schools that remain off this list overwhelmingly tend to be criterion-based schools that select students based on rigorous admissions tests—and, not coincidentally, are whiter and less impoverished than the other schools and the district as a whole. The result, then, is a highly tiered system in which only those with relative advantage benefit while the remainder of students suffers the consequences of cumulative disadvantage.
Not unlike other districts grappling with the social and structural challenges facing urban intensive and urban emergent schools, the BPS District has attempted a number of reforms to improve school outcomes, the most recent include adopting the small schools model (College Board, in particular) and creating science, technology, engineering and math (STEM)-focused schools and programs. As a form of school reform, STEM initiatives have been extremely popular (Means, Confrey, House, & Bhanot, 2008; National Research Council, 2011), touted as providing solid pathways for low-income students of color. In the larger comparative study, we were particularly interested to see how such reforms were enacted, how local contexts shaped reform efforts, and the effects of the reforms on students in and through time as they transitioned from one grade to next and, upon graduation, into various postsecondary destinations. Given the complexities of Buffalo’s history and contemporary position (e.g., residential and institutional segregation, concentrated urban poverty, “failing” urban schools, rapid reform efforts), such a study required a deeper, ethnographic approach that allowed for a better understanding of those most affected by reform and their relationship to reform changes over time.
Research Design
Although approaches to qualitative longitudinal research can vary (Corden & Millar, 2007; Holland, Thomson, & Henderson, 2006), our study is defined by the “deliberate way in which temporality is designed into the research process making change a central focus of analytic attention” (Thomson, Plumridge, & Holland, 2003, p. 185). As opposed to studies that engage in longitudinality by way of follow-up studies or “revisits” (Burawoy, 2003; Lareau, 2011; Weis, 2004), our initial study design incorporated a longitudinal focus across transitional points in time (i.e., from one school year to the next, from high school to postsecondary pathways, and from the early stages of school reform through its deterioration and recalibration). Our research design is distinguished by the three elements that Vallance (2005) argues constitute longitudinal qualitative research:
Firstly, the research question is that which is longitudinal in its intention, secondly that the sample is consistent with longitudinal requirements and thirdly that the means of analysis explicitly addresses changes over time for individuals in such a manner as to describe meaningful relationships between the changes and the maturation or change of time in itself. (p. 4)
To answer research questions that included observing change over time (relative to schools and the lives of students), the design of this project required ethnographers to closely follow groups of selected students within their respective schools from the early stages of their high school careers through to graduation.
Specifically, researchers spent 3 years within four nonselective, low-performing, urban public high schools in Buffalo, New York (for an extensive discussion of the larger comparative study that includes urban public schools in Denver, CO, please see Eisenhart et al., 2015; Weis et al., 2015). Because of the study’s focus on STEM-based educational reform initiatives, two of the four schools are “STEM-focused” and two of the four have a traditional comprehensive high school structure. Data reported in this article are drawn from the four Buffalo public schools. Two schools, Broadway Science and Lincoln, are large comprehensive high schools, while Global Horizons and STEM Academy are College Board Schools. 5 Broadway and STEM Academy are district designated STEM-focused schools. 6 At the time of the study, all schools had a reported free and reduced lunch rate of at least 75% and less than 10% of the student population was White.
Sources of Data
In the case of our particular research design, longitudinal ethnography demanded conducting and replicating intensive ethnographic methods over time. Thus, research for this project included intensive participant and nonparticipant observation in schools and classrooms; in-depth interviews with focal students, parents, teachers, counselors, and principals; and descriptive surveys of students (in the first and third year of the project). At each school, approximately 12 focal students were selected from the relevant population of sophomore students based on their location in the top 20% of their class and met the following criteria: (a) 10th-grade standing, (b) ranked in the top 20% of their high school class in math and science, and (c) stated some interest in pursuing STEM at the postsecondary level. We assembled a list of students meeting these criteria and worked with each school to create a final sample of student participants who were the most proficient in STEM subjects relative to the remaining 80% at their school, who agreed to participate in the study, and who met all consenting criteria. We were most interested in the top 20% given our focus on high school STEM preparation and postsecondary transitions and STEM reform. It is ostensibly the case that the top 20% would be best positioned for this transition and the greatest beneficiary of access to high-level STEM programs. Given that the four schools under study did not rely upon selection criteria for admission, we wanted to focus on the most academically competitive students.
A total of 54 focal students are included in this sample, along with parents (27), teachers (2-3 per school), school counselors (1-3 per school), and administrators (1 per school). Each focal student was interviewed twice per year over 3 years. Parents were interviewed twice. In addition, researchers interviewed at least one science teacher and one math teacher at each school (once per year), and at least one school counselor at each school (once each year). Administrators were interviewed once. In addition to interview data, researchers spent more than 300 hours in each school engaged in participant and nonparticipant observations. Researchers would visit classrooms, observe counselor meetings, attend parent events, and a range of other extracurricular activities. Researchers also visited students’ homes. Finally, official school documents (e.g., official student transcripts that provided data on actual courses taken, grades, and standardized test scores) and other materials (e.g., classroom handouts, letters home, lists of course offerings, website materials) were also collected and analyzed.
Data Analysis
Interview data were transcribed and, with the assistance of HyperRESEARCH, initially coded deductively using codes consistent with the definitions of opportunity structure and the participants’ perceptions of opportunity structure that framed the study (Miles, Huberman, & Saldana, 2014, p. 81). Examples of codes include course offerings, requirements, views of school, STEM, among others. Next, general coding categories were narrowed inductively (using codes originating in the data) into subcategories. In all, approximately 200 codes were agreed upon. Data segments were then double- and triple-coded, as relevant, and consolidated under a set of subcodes (e.g., future plans, math courses, science courses, among numerous others). A quarter of all interviews were read individually by team members. Coding schema emerged from joint discussions of the individual codes. Upon agreement, codes were then applied to the remaining interview data.
We engaged a similar process of coding and analysis for observation data. We first deductively coded the data and then relied upon inductive coding. In addition to triangulating methods (e.g., published course offerings in school documents), research team members independently examined the same data, which were then checked by team members for consensus. We also entered school documents in a spreadsheet so we could compare, for example, published accounts of course offerings with student transcripts.
In the following section, we highlight findings drawn primarily from student, teacher, and counselor interviews and observations to advance our methodological conclusions concerning qualitative longitudinal research. Specifically, we focus upon examples of curricular and structural change that occurred in all four schools during the 3 years we were immersed in our sites. Data suggest that efforts and reforms to improve opportunities and outcomes for students resulted in the failure to sustain these efforts over time, conclusions we could not have drawn without long-term, on-the-ground immersion within these contexts. These findings, though disappointing, are not necessarily surprising considering the arguments for how difficult it is for urban districts to sustain reform efforts beyond a single year (Peck & Reitzug, 2014). However, such findings would not have been apparent without a qualitative longitudinal framework—one that provided both longitudinal breadth and qualitative depth.
The Dizzying Nature of Change: Turnaround and Turnover
“Turnaround” policies will result in a churn in school leadership, in teaching staff, and even in the closing and reopening of schools themselves. This will all happen disproportionately in low-income communities of color. (Carter & Welner, 2013, p. 223)
With each new reform effort comes the promise of positive change. Struggling schools and the communities in which they are situated have reason for a renewed sense of hope for improved opportunities and outcomes as schools open with new names, new resources, and assurances of innovative programs and rigorous curricula. At various points in our study, all four schools demonstrated signs of renewed hope and potential following increased investment and funding for new facilities, aesthetic improvements, innovative curricular changes, and support services and programming; however, over the course of 3 years within these schools, that hope and potential waned as supports and programs eroded, and curricular offerings devolved into a set of less intellectually demanding courses with low expectations. Supported by existing literature, these changes altered the culture of schools and affected students, teachers, staff, and administrators in significant ways (Liou & Rotheram-Fuller, 2019; Milner, 2012b).
In the years leading up to our research, the four schools within our study were each part of significant reform efforts. All four schools had previously been “SURR” schools (Schools Under Registration Review), meaning the State Education Department deemed them as failing and the schools were required to institute school improvement plans. STEM Academy, formerly a “failing” vocational high school, closed and reopened as a College Board School with a STEM focus. Global Horizons, formerly a “failing” comprehensive school, phased out and then also reopened as a College Board School.
Both larger, comprehensive schools within our study, Broadway Science Academy (BSA) and Lincoln High School, had also been “failing” schools, and each had undergone significant internal transformations. Most notably, at the start of our research in 2010, each school had recently established curricular changes and brought in new school leadership. At Lincoln High School, an academy system was put into place, which was already starting to deteriorate by the second year of the project and the administration responded by reorganizing the academies. At BSA, new high-level science tracks were developed in collaboration with a nearby research university, which by Year 3 of our study had been completely dismantled. In addition to these internal changes, external partners were brought in to work with particular schools to further reform efforts. Partnerships with national organizations such as Talent Search and College Board, as well as local efforts driven by governmental “Promise” grants were also cultivated. School personnel were tasked with turning around schools through new models while managing multiple partnerships. In the next sections, we highlight specific barriers to implementing reform to illuminate the complexity of urban spaces and the temporal mechanisms of destabilization.
A Revolving Door
Not unlike other urban districts across the nation experiencing similar reform churn, Buffalo public schools continued to suffer from high rates of teacher and administrator turnover. During the study, the BPS District saw two different superintendents (and the campaign to force the second one out was underway), and three of the four schools saw principal turnover. As argued by Hill, Campbell, and Harvey (2000), leadership turnover brings about “frequent and abrupt changes of improvement strategy and equally abrupt abandonment of half-accomplished reforms” (p. 84). Not surprisingly, these consequences were evident across schools. For example, in the first year of our study, after years of falling into a “repetitive failure cycle,” Lincoln High School seemed to be on the upturn. Under Dr. Romero’s new direction and leadership (in his second year when our research began), Lincoln saw notable improvements and faculty and student morale seemed relatively high. In the second year of our research, an interview with Dr. Romero reveals his vision of progress:
We are trying to build a foundation to re-culture the school from the bottom, from the entry point. The biggest impact will be viewed four years after that but we have already noticed, based on some of our changes, our juniors this year came in with our new administrative team and our changes that we made and they are at a much stronger level so our graduation rate next year will improve. We are seeing the impact.
Already?
Yes. We are in year three and in year four we are going to see that impact and in year six and seven we are going to see even a greater of an impact.
I remember at some point, maybe last year, you talked about how you wanted to at least make sure you are here for five because that is the amount of time you thought it might take to actually see some of these changes.
Yes, that is what the research shows, that it takes a minimum three but it really takes about five to re-culture a school. It is much more evident at the high school level than it is at the elementary level for many reasons, but it does take about five years. I would at least like to stay here for five years.
Amy: And you plan to see it through?
Yes, absolutely.
However, in what would have been his fourth year at Lincoln, the principal was reassigned to a new position within the district, again handing the school over to new leadership. In fact, several new leaders cycled in and out that year undermining Lincoln’s forward progress. The following year, one of Lincoln’s outside collaborating organizations—which had increased both their involvement in the school following Dr. Romero’s departure and their public rhetoric regarding public school failure and the need for more private partnerships—hand-selected an administrator from outside of BPS. 7 His tenure lasted only 1 year and was, by all accounts, unsuccessful. In 2015, the New York State Department of Education publically noted Lincoln’s inability to improve and was cited in the local news as having referred to the school as being “out of time.” Although the school continues to operate in 2016, it does so under threats of closure, and most recently, fought against plans for a takeover by charters.
Similarly, other schools in the study experienced principal turnover. Broadway Science hired a new principal during our second year of study (who is no longer there). During his tenure, we saw the deterioration of the university-based, high-status STEM track (key to the school’s overall reform effort)—a program that had experienced some success in providing pathways for students into STEM majors at the local university—only to be replaced by a vocational STEM track that prepared students for minimum wage work (Weis et al., 2015). Both the principals at Global Horizons and STEM Academy had been replaced, as had other key personnel, given turnaround requirements associated with Race to the Top, when we withdrew from the field in 2013.
Not surprisingly, given the extent of administrative change, all four schools demonstrated high levels of teacher turnover, which can have an enormous effect upon schools and their students (Simon & Johnson, 2013). In his third year at Lincoln High School, Derek (a student) talked about the difficulties associated with teacher turnover.
Tell me about your academic year.
Uh, rocky year . . . changing teachers from the beginning and then things like that.
Right. I remember that.
Yes.
Now tell me about how that class [Algebra II] ended up.
Overall it—it was pretty good. Uh, mainly because there—there’re a lot of seniors in there and so, you kind of need that class to graduate, you know? So it wasn’t as bad as I thought it was going to turn out with the transition of the new teacher and things like that.
Good, okay.
But still, you know, once you become comfortable with that—with that teacher and, you know, it—
Yeah, it’s tough changing teachers.
It is tough, yeah.
So, now did you end up—did, uh, the particular teacher who was there when I sat in at the beginning, did she stay?
No, she didn’t.
She didn’t stay?
She was moved somewhere else.
Okay, then—so was that the third teacher that you had?
No, that was . . . she was, she was the second because the first one, uh, was out on leave.
Okay.
And never, she never came in.
Oh, I see. So, she—
So she was the fill-in for the original teacher, uh, but then she was, she moved.
Okay, okay, and then who did you end up . . . ?
Now we have, uh, a guy, Mr. Kenner.
Derek, like so many of the students in our study, felt the difficulties associated with teacher turnover. As Simon and Johnson (2013) argue, “stability in staffing is especially important for low-income students who research suggests are especially dependent upon their teachers” (p. 2). Chronic teacher turnover is, unfortunately, not uncommon among urban districts and schools that struggle with resources and meeting progress measures (Darling-Hammond, 2013). Allensworth, Ponisciak, and Mazzeo (2009) note that many schools serving the most financially disadvantaged students lose more than half of their teaching staff every 5 years. These patterns of “chronic turnover exact instructional, financial and organizational costs that destabilize learning communities and directly affect student learning” (Simon & Johnson, 2013, p. 6).
With a new administrative team and continuous teacher turnover, the struggle to move forward at the Buffalo schools was seemingly thwarted, in part, by the pendulum swing of reform efforts. Given the longitudinal design of our study, we were able to observe the turnover in staff and its effects on students, other staff members, and overall morale.
Counselors: Overburdened and Underprepared
School counselors play a critical role in facilitating students’ postsecondary trajectories, a role that is essential to the success of school reform efforts in Buffalo, reforms grounded in creating greater access to college. However, counselors within schools primarily serving low-income students of color experience significant challenges and constraints in performing their jobs (Perna et al., 2008), often due to the litany of responsibilities with which they are tasked—addressing emotional needs of students, administrative work, data collection, scheduling, career and college preparation, and so forth. Research documents the extent to which counselors are overwhelmed by these and many other issues and are “often overburdened and undervalued by schools, particularly in institutions with a high percentage of underrepresented students” (Corwin, Venegas, Oliverez, & Colyar, 2004, p. 447; McDonough, Korn, & Yamasaki, 1997; Perna et al., 2008).
Despite caseloads smaller than the state average (NACAC, 2015), field notes and interviews demonstrate that the counselors in the four Buffalo schools continually reported feeling burnt out and unsupported by administration. In addition, a general lack of experience pervades the sample. Of the 11 counselors across the four schools, eight had fewer than 5 years of experience. Furthermore, many shared that their counselor training left them unprepared for many of their required tasks. Tabitha is emblematic of the sample when she says:
Well, you know, I went to [University] and I think any counseling program actually is probably like this, they focus on counseling. A lot of it is counseling. I received zero training about credits, about college prep, anything. I mean, I received career information, you know, how to do career and interest inventories, blah blah blah, but nothing about colleges, which is helping kids apply for college, especially this population because there’s so much hand-holding that has to be done. And the credits are probably like the biggest part of my job. That kind of stuff.
Drawing upon the case of Global Horizons, we demonstrate the ways in which school counseling exerted a destabilizing effect upon reform efforts, an effect we would not have observed had we left the field after the first year. Field notes from Year 1 document a seemingly well-functioning counseling office. Tabitha and Benjamin shared the counseling responsibilities at Global Horizons and received intensive support and professional development from the College Board. Principal Mark Sexton is recorded on multiple occasions, praising the capability of the counseling staff. While both counselors indicated that their days were generally overwhelming—“Really chaos. Really, you walk . . . from the moment I walk in there’s always somebody sitting in my office—you’ve seen that. There’s always somebody in my office,” Tabitha reports—neither hinted at what we were to find in Years 2 and 3.
Our extended stay in the field allowed us to observe the breakdown of counseling support. In 3 years, we witnessed three different counseling arrangements. In Year 1, Tabitha and Benjamin split the school equally by alphabet. In Year 2 of the study, Tabitha was given sole responsibility of the high school while Benjamin assumed responsibility for Grades 5 through 8. In Year 3, this arrangement continued, except that Benjamin took over high school test administration. The following year, Tabitha left, and Benjamin again split the counseling load with a new counselor, who was in her first full-time position. Simultaneously, College Board pulled back its support. Counselor received intermittent support in the second year of our study and none in Year 3.
The shifting counseling arrangements contributed to disorganization and the lack of a unified counseling approach harmed student progress. Poor communication between the counselors allowed for items to fall through the cracks. For example, many students, like Jelani, told us “I don’t know [who my counselor is] because they all call me up, like they want to talk to me and stuff.” Others, like Dannika complained that “she [Tabitha] is always losing stuff. She’s so disorganized” when discussing her college applications. Perhaps the most egregious issue that arose because of counselor disorganization was an incident in the fall of 2012. While Tabitha managed the high school and the college applications process, Benjamin was tasked with test administration. After students took the PSAT, Benjamin failed to submit the tests to the College Board. Tabitha, who was responsible for pushing into classes to explain test results and provide support for improvement, was baffled when official results had not arrived. Only when she reached out to the College Board was she informed that tests were never submitted (Field notes, December 2012).
Scheduling also served to destabilize reform efforts. For students to be successful, it is necessary that they take courses sequentially. Tabitha’s account demonstrates some of the typical scheduling challenges we observed in the later years of our study:
So then I have to make sure that all those kids get into the courses that they’re supposed to be in. Um, and then at the beginning of the year, of course, there are always like brand new kids that are coming in because they’re [District] just, you know, shoveling and dumping ’em into the building. And then kids who are like, “Oh, Miss, I already took this one,” and I didn’t—because our computer system is still not accurate, so I’ll have to go through and be like, “Oh, you’re right, it wasn’t reflecting this.” Okay, fine. So I do that. That takes me through probably November.
Several factors undermined the course taking patterns of students. District allotment of teachers is contingent upon student numbers; thus, it was often several weeks into the school year before the school was staffed appropriately—resulting in multiple revisions to the master schedule, sometimes lasting until November (Years 2 and 3). As a small school, there were also fewer sections of required courses, which meant sometimes students could not get into the sequential courses they need. Yet, these reasons cannot fully account for the patterns of haphazard scheduling we uncovered as we remained in the school. Here, Rafael recounts his schedule:
Yeah, so last year, my exact schedule was I had, for a while, Trigonometry but then they took me out of that. So then I had geometry and algebra. So in the beginning I had three math classes and two English; one senior and English 1. I had Global 9 and what other classes are there? I had that science one, the first one, what is it . . . ?
Rafael, like many students and their needs, fell through the cracks, cracks that only became visible to us the longer we remained in the field. We were able to observe changes in the ways counseling was provided less support and how organizational changes undermined programmatic stability. Furthermore, given the relationships we were able to form through sustained time in the field, we were provided greater access, which allowed us to observe that which was hidden to us in our first few months.
A Culture of Continuous Curricular Change
Because changes in the number and quality of curricular offerings, including the quality of advanced-level courses, are not always recorded or visible on paper, the full scope of the consequences for the student lives affected also remain invisible without a longer-term, qualitative approach to research. Even as quantitative longitudinal work provides evidence of the erosion of programs or courses over time and the “failure” to implement reform, this kind of research does little to explain why the erosion of opportunities occurs and what mechanisms precipitate these changes—both of which require better understanding as we work to improve reform implementation.
For example, in spite of district intentions to increase opportunities for low-income students, our study’s findings reveal that these efforts stalled or collapsed across the four sites over the course of 3 years, leaving low-income and minority students with even fewer opportunities than before the implementation of these reform efforts (see Eisenhart et al., 2015; Weis et al., 2015, for an extended discussion of this point). For example, at the start of our research, Global Horizons appeared to be a well-functioning school, despite the low student test scores. What is not evident on paper but only clear through sustained time in the field is why this model collapsed at Global Horizons. Course offering lists and school rhetoric about all students being college-bound disguise what observations, teacher and student interviews, and student transcript data reveal. While the school touted accelerating students in math and science in eighth grade, it neglected to publicize that many students were repeating the courses they had taken during their early high school years and failed—a trend that was common in all four schools. Several focal students—students who were at the top of their graduating classes—were not taking any math or science senior year because there were simply not enough of them to warrant opening a class (despite course offerings listed in official school documents), or in some cases, they were retaking previously failed classes so they could graduate. 8 Given that New York State does not mandate 4 years of math and science for graduation, there was no requirement for the school to offer these courses senior year.
The Case for Qualitative Longitudinal Research
Destabilizing effects, such as those discussed above, lead to higher rates of remediation, attrition and dropout (Darling-Hammond, 2013) as the continuous nature of school policy and reform directly links to the social reproduction cycle of students, families, and communities. The case of school reform as it transpired across the four schools in Buffalo is representative of the reform “churn” continually inflicted upon “failing” urban districts across the United States. Each element we describe above—the revolving door of school staff, counseling inexperience, teacher job insecurity, and curricular changes—all worked to undermine initial progress.
Our use of longitudinal ethnography was critical to understanding the complexity of the ways in which various factors undercut reform initiatives, despite outward commitments to turnaround schools for the better. While perhaps some of the undercutting issues could be observed through a short-term “snapshot” analysis of school documents, as is demonstrated by our findings, an ethnographic longitudinal approach allowed for us to better understand the mechanisms through which reform efforts played out and the resulting impacts on various stakeholders, in and through time. For example, being immersed within schools across key transitional points in time allowed us to see and better understand the mechanisms of destabilization that together, worked to dismantle once promising reform efforts. Importantly, immersion in and through time also allowed us to see and better understand the consequences for students, families, teachers, and staff relative to a complex system that prevents positive sustained change from taking shape. In this study, the revolving door of school staff, counseling inexperience, teacher job insecurity, and curricular change all contributed to upholding a cycle of reform churn that relies upon unrealistic expectations and the perceived failures of individuals/groups to meet those expectations.
More specifically, had we terminated our research after Year 1, we may have drawn the erroneous conclusion that the College Board model was successful at Global Horizons, or that improvements at Lincoln would sustain themselves, or that STEM reform efforts to enhance particular opportunities for students at STEM Academy would come to fruition, or that the same efforts at Broadway Science would be lasting. Within each of the four schools, time gave way to the erosion of curricular opportunities and dismantling of programmatic initiatives. Had we departed the field after 1 year, we would not have observed the scheduling obstacles and the extent to which they act as barriers to student success. Again, immersion in sites over time revealed egregious scheduling errors that placed some students in classes they had already taken and passed or in auditoriums, where they sat for months until their schedules were resolved. Had we not remained in the schools over time, we might have missed the deleterious effects of constant turnover and uncertainty on morale. For example, in looking back to the opening quotation to this article, we see evidence of improvement, promise, and hope in the words of Lincoln’s principal. Indeed, evidence of high morale was further supported by the voices of teachers, students, and staff during the first year of the project. By Year 3, however, we observed a full reversal of this attitude and its consequences following the principal’s departure. Overall, had we not followed students and remained deeply situated within these schools in and through time, crossing over transitional spaces and witnessing the nature and consequence of educational reform, we might have missed the ways in which policy and social process largely functioned to reproduce rampant inequality.
The dizzying nature of repeated demands for change within schools sets the foundation for the inability of schools to sustain reform efforts. Not unlike other reform efforts that have fallen short (e.g., No Child Left Behind), low-income students within these low-performing schools were poorly served by various initiatives that were meant to facilitate greater opportunity and improve outcomes. A longitudinal analysis in the absence of qualitative depth (or vice versa) would not have allowed us to see whether reform was lasting or, and more importantly, how hidden mechanisms functioned to destabilize, erode, and undercut reform efforts. Furthermore, our continuous presence in the school over the course of 3 years provided deeper insights that are crucial to discussions of policy and reform. If “efforts at comprehensive school reform are time-consuming and difficult, and proceed with uneven success across schools” (Rowan, Camburn, & Barnes, 2004, p. 2), then our methods need to account for this, lending support to the need for more qualitative longitudinal research of urban education reform.
Conclusion and Implications
Carter and Welner (2013) argue that
the problem [of ineffective reform and increased inequality] is not a lack of good ideas; the problem is a lack of sustained attention to good effective ideas that compel us to work together as a nation, as communities, and as fellow citizens. (p. 224)
We, too, believe that a commitment to sustained attention to the implementation of reform, particularly reform that considers the temporal, social, cultural, economic, and political context can in fact provide greater opportunity, improve outcomes, and generate a more democratic education for all. Qualitative longitudinal research provides us with the intellectual tools to examine the complex picture of urban education and educational reform.
However, like any methodology, qualitative longitudinal research is not without limitations. Qualitative longitudinal research is time-consuming, can be expensive, requires a great deal of foresight and planning, and presents the issue of navigating an “ocean of data” (Saldana, 2003, p. 5). Rather than a dismissal of the power of this approach, the absence of qualitative longitudinal research, however, may be more an outcome of the challenges associated with longitudinal work (e.g., greater costs associated with being in the field), as well as increasing neoliberal pressures placed upon academics to produce research and publish with great speed and efficiency. Indeed, as educational researchers, we must recognize that our practices are subject to the very same neoliberalism that drives, shapes, and perpetuates the educational reform churn—its effects permeate and are felt throughout all corners and levels of the system of education. As Jeannie Oakes (2015) reminds us, “we must always keep in mind that the structures and cultures of universities (and philanthropies as well) operate in concert with other structures that serve to perpetuate current practices, norms, politics and power dynamics” (p. 133).
As we work within these conditions, the consequences of our research practices as they inform or misinform educational policy and reform efforts have perhaps even greater potential to reinforce current patterns and exacerbate educational inequalities. While we recognize the difficulties facing educational researchers working within the same neoliberal climate wherein research is conceived, proposed, funded, conducted, and disseminated, we believe research practices should not mimic the same lack of sustained attention demonstrated by educational reform. For example, if we superimpose deadlines onto research that fit our needs rather than reflect the full scope of the phenomenon we wish to study, we risk producing analyses that misrepresent and misinform. In other words, when research is shortsighted, we risk misunderstanding the ways in which an unjust system remains intact. Peck and Reitzug (2014) write that “[t]ime is of the essence when it comes to our collective responsibility to provide a high-quality, meaningful education to urban students who are not currently experiencing it” (p. 30). Undeniably, time, in so many ways, is of the essence. Injustice can easily hide within temporal space, masquerading as school, administrative, teacher, or student failure, rather than reveal itself as a pattern and product of systemic inequity. As the consequences of the perpetual reform churn continue to intensify within urban spaces, we cannot afford to ignore the qualitative and temporal features of educational reform in our research.
Footnotes
Acknowledgements
We are grateful to all members of research teams led by Lois Weis at the University at Buffalo and Margaret Eisenhart at the University of Colorado Boulder. The authors are especially grateful to Lois Weis and Margaret Eisenhart for extensive feedback on earlier versions of this manuscript and to H. Richard Milner and three anonymous reviewers for their incredibly insightful suggestions.
Authors’ Note
All errors rest with the authors.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded the National Science Foundation (awards DRL 1008215 to Lois Weis and DRL 1007964 to Margaret Eisenhart).
