Abstract
Instead of having to comply with rigid requirements of the No Child Left Behind era, states and districts now have greater latitude to decide for themselves which evidence-based programs are best suited to address their students’ needs. However, as Maria Ferguson explains, evidence-based school improvement is not as easy as it sounds. Not all states have the capacity or expertise to assist districts in determining which programs are most suitable. When the Every Student Succeeds Act (ESSA) is up for reauthorization in two years, policy makers will need to consider whether states and districts have what they need to follow its mandates.
As states take on more responsibility for identifying evidence-based solutions, they need capacity and support.
When former President Barack Obama signed the Every Student Succeeds Act (ESSA) into law in December 2015, a rare gift was bestowed upon state and local education leaders. After many years of having to comply with rigid and often frustrating federal requirements for identifying and (hopefully) improving low-performing schools, education leaders now had more authority and greater latitude over once highly regulated decisions. States are now required to identify three fairly straightforward categories of low-performing schools, and school district leaders are able to decide for themselves which evidence-based programs are best suited to address their students’ needs.
The opportunity for choice
Although districts are free to select their own interventions, any programs they choose must be selected from a list of state-approved vendors whose products comply with ESSA’s four “tiers” of evidence, which range from ”strong evidence” of effectiveness (Tier 1) to “likely to improve student outcomes” (Tier 4). In a nutshell, the top three tiers require interventions to demonstrate a statistically significant effect on student or other relevant outcomes. Tier 4 was developed for those interventions that do not yet have an evidence base that qualifies them for Tiers 1-3. Because Tier 4 is speculative in nature and requires only a research-based rationale that the intervention is likely to improve student outcomes, some worry that it gives education leaders carte blanche to use interventions that are easier to implement but less likely to demonstrate impact (more on that later).
With an eye toward innovation, ESSA also states that districts using Tier 4 interventions ”should make an effort” to study the effects of the intervention, with the goal of producing more promising evidence of effectiveness. It will be interesting to see how many districts actually do that because “should make an effort” is usually an open invitation to not do anything (at least that is how it plays out in my house when I tell my kids they should make an effort to clean their rooms). Having said that, it would be extremely helpful to the field if some districts did study the effects of the Tier 4 intervention(s) they use. The field is eager for more research on innovative school improvement strategies, and many communities, especially those in rural areas, are short on research related to their unique student populations.
While ESSA’s more autonomous approach to school improvement is clearly a welcome change from the structure and compliance mind-set that characterized No Child Left Behind, implementation studies from various groups (including the Center on Education Policy [CEP], the organization I lead) show that evidence-based school improvement is not as easy as it sounds. In addition to the lists of approved vendors, states have adopted a wide variety of approaches to help school districts determine which school improvement strategies are best for their students. These are at different stages of implementation, and their effectiveness is not yet clear. However, recent reports from the Council of Chief State School Officers (CCSSO, 2019), the Collaborative for Student Success (Ross, 2018), and CEP (Rentner, Kober, & Braun, 2019) all identify significant challenges with capacity and a general lack of familiarity with how to assess needs, select an appropriate intervention, and use data effectively.
The capacity challenge
Most of what we know about state and local efforts to meaningfully engage in ESSA’s evidence-based school improvement requirements underscores a familiar problem in education reform: the capacity issue. It is important to note that, in most instances, state and local concerns about capacity are not just about funding. Because state education agencies (SEAs) and school districts vary greatly in size, scope, budget, and expertise, they face differing limitations when implementing new requirements. In addition to inequitable funding formulas, capacity concerns include too few or insufficiently trained staff, limited internal and external resources, and a lack of engaged partners within the community.
Despite these concerns, an increasingly complex set of responsibilities and requirements has been laid at the feet of state and local education leaders over the last decade, with little regard to their overall capacity. When the Obama administration introduced its Race to the Top grants in 2009, questions and concerns about capacity began with the application process, which was complicated, labor intensive, and focused heavily on the collection and use of data. After the grants were awarded, many states still struggled to meet their ambitious goals. Implementation studies from multiple sources highlighted state and local capacity as a significant hurdle toward progress.
Adapting to a new normal
It is hard to criticize the intentions behind ESSA’s evidence-based approach to school improvement. For too long, policy makers and the public have lamented the resources spent on strategies that yielded little or no results. Decisions about policy and practice should be rooted in evidence, but state and local leaders must have the access, training, and support they need to actually use it effectively. The state and district leaders CEP interviewed contrasted their ”lean” staffs with the “big lift” of ESSA’s evidence requirements. SEAs are being asked to play a new kind of role, one that is more focused on supporting and guiding district and school leaders about evidence use and school improvement. For many state department employees, that kind of support and assistance is a far cry from their prior compliance mind-set. State officials, many with little or no expertise in research methods, data management, or the multitude of new products and interventions that are being marketed to them, are trying to figure out how to provide guidance and technical assistance, a job that was once managed at least in part by the U.S. Department of Education.
District leaders are also adapting to ESSA, trying to understand the research behind their state’s ”vetted vendor” list of school improvement interventions. Districts are also having to rethink long-standing vendor relationships to ensure that the interventions they use meet ESSA’s criteria.
Still, state departments and district leaders seem to be slowly adapting to the new normal of school improvement. CCSSO’s report points out that supporting district and school capacity to collect, analyze, and use data was a major priority for states; and CEP’s report highlights state efforts to help districts select appropriate interventions that comply with ESSA’s requirements. The Collaborative for Student Success has also looked at how states and districts are addressing equity.
Moving forward, both federal and state leaders need to acknowledge how challenging evidence-based school improvement can be. In two short years, ESSA will again be ready for reauthorization. While it’s hard to imagine Congress doing anything in a timely manner anymore, policy makers will eventually need to consider what aspects of the law seem to be working versus those that appear ineffective. At the top of their minds should be these questions: How does the capacity of state and local education agencies affect school improvement? Do states and districts have the access and expertise they need to effectively use evidence to support decision making and practice?
It is pointless to keep expecting state and local leaders to take on new and more complicated responsibilities without at least considering whether they have the capacity and expertise to be successful. Having a realistic understanding of what state and local leaders are able to accomplish does not have to mean lowering expectations. Nobody wants to make excuses for education leaders who are not prioritizing their students. But states and school districts stand a far better chance of making progress if they are encouraged to set ambitious goals and are given the support needed to achieve them.