School Effectiveness Factors and Student Achievement: A Longitudinal Study in an Urban School District

Abstract

In this study we explore the longitudinal impact of effective school factors on student achievement over a five-year period in an urban school district. By collecting teacher survey data from urban schools in a Midwest school district and analyzing the survey data alongside student achievement, we identified several factors associated with student achievement growth. Our findings reveal that Shared Vision and Goals is positively associated with student achievement growth in math and reading. Furthermore, Student Behavior Expectations and Behavior Management Practices are positively associated with growth in reading achievement. Our research contributes to the understanding of effective school factors from a longitudinal perspective, particularly in an urban school setting.

Keywords

urban education school improvement students change

Efforts to improve educational performance often revolve around two pertinent questions: “What does an effective school look like?” and “How do schools enhance their effectiveness?” (Harris & Bennett, 2004). The dialog surrounding effective schools has spanned over 70 years, blossoming with the effective school movement. Discussions typically concentrate on the characteristics or current state of effective schools. However, numerous scholars, including Creemers and Kyriakides (2010), Reynolds (2010), and Verachtert et al. (2009), have posited that the focus should be on the evolving nature of school effectiveness, thereby revealing trends of development. They argue that this perspective could yield a more comprehensive understanding of how school effectiveness factors impact school outcomes.

Although much of the research on school effectiveness has been broad in scope, understanding school effectiveness factors in the context of urban schools poses unique challenges. Urban schools, with their distinct socioeconomic and cultural landscapes, necessitate a nuanced exploration of effectiveness, particularly given the disparities in resources and outcomes often observed in such settings. This study aimed to investigate how school effectiveness factors are associated with student achievement over a 5-year period in an urban setting. What follows is a review of the literature regarding the topic.

Factors of School Effectiveness

Given that school effectiveness is usually defined as the performance of a school regarding student achievement, most measures of school effectiveness focus on student academic performance (Leithwood & Louis, 2021). However, the factors (or a combination of factors) contributing to school effectiveness appear to have a wide range of definitions and measurement approaches. Some studies have focused on specific school effectiveness factors. For example, based on national level data, Seashore Louis et al. (2010) found a positive and indirect relationship between school leadership and student achievement. Boyle et al. (2005) explored the impact of professional development, revealing coaching and inquiry to be the two most effective professional development activities.

Conversely, some scholars argue that since factors typically interact with each other when impacting school effectiveness, it is more appropriate to assess a collection of factors influencing school effectiveness rather than isolating individual ones. For example, in a meta-analysis of 155 studies on school effectiveness from 1984 to 2005, Scheerens (2013) revealed several factors, including, for example, cooperation among staff, orderly school climate, monitoring, curriculum quality, homework, learning time, parental involvement, achievement orientation, and educational leadership. The outcome variables tended to center on student achievement. The analysis showed that the highest effect sizes were related to curriculum quality and teaching time (Fischer’s z = 0.15), closely followed by school climate factors, including orderly climate and achievement orientation (Fischer’s z = 0.14). Creemers and Kyriakides (2010) found that factors such as quality of teaching practice, school policy for teaching, teacher collaboration, partnership policy, provision of resources and evaluation of school learning environment were related to student achievement. Meta-analyses also found that various aspects of principal leadership (Wu & Shen, 2022) and teacher leadership (Shen et al., 2020) are positively associated with student achievement.

School Effectiveness in a Developing Perspective

Unlike the static perspective of school effectiveness, the construct of school effectiveness from a developing perspective offers a more dynamic, longitudinal approach to understanding educational performance. According to Reynolds et al. (2014), the use of longitudinal data and a repeated measure in studies is still relatively rare but has gained traction. Most previous studies have applied a repeated measure on the dependent side. For example, Creemers and Kyriakides (2010) had two data collection points to study about the changes in selected schools on a follow-up test compared to the original data collected 4 years earlier. They found that 25% of the schools showed an improvement in their school effectiveness status such as math and language. They further argue that one-time observation of the impact of school effectiveness is not enough. Vanlaar et al. (2016) collected data at the beginning and the end of a school year in six European countries to investigate how teacher and school factors affected the achievement growth of low- and high-performing students. The findings show that all teacher-related factors, such as teacher collaboration and partnership, collected from a student survey, were positively related to student math and science achievement.

A few studies have applied repeated measures on the independent side. Although Verachtert et al. (2009) did not discuss what specific factors affect the student achievement, they found that transitioning to a different educational level (e.g., from elementary to middle school) reduced the achievement gap between high and low socioeconomic students, the school effectiveness factors did not narrow the achievement gap within each educational level.

Studies that used repeated measures on both independent and dependent sides were extremely rare, particularly those considering various school effectiveness factors. Hallinger and Heck (2010), for instance, constructed growth models on the association between school improvement climate, collaborative leadership and student achievement. Their analysis of data collected at four time points in 4 years showed that the growth in collaborative leadership was both directly and reciprocally related to the growth in school improvement climate and indirectly related to the growth of student achievement. The study had applied repeated measures on both the independent and dependent sides, an approach that our study utilized. Notably, most studies on school effectiveness factors do not distinguish between urban and non-urban schools. In the following section, we review the literature focusing on urban schools.

School Effectiveness in the Urban Setting

Urban Schools: A Unique Context and A Tougher Environment

Urban schools indeed hold a unique place in the educational landscape, marked by both their challenges and their potential for innovation (Abdul-Jabbar & Kurshan, 2015). However, these schools face significantly tougher challenges than their non-urban counterparts, especially in terms of school effectiveness. One of the unique aspects of urban schools is the diversity and dynamism of their student populations. This diversity is a double-edged sword: while it enriches the educational environment, providing students with a more diversified perspective due to the diversity of teachers (Shen et al., 2003) and students (Bonner et al., 2018), it also contributes to the complexity of the challenges faced by these schools. For instance, principals, teachers, and students in urban schools face a more complex environment that encompasses socio-political intricacies (Yavuz & Gülmez, 2018). Instead of placing blame on families and urban societies for the underperformance of students, researchers such as Thrupp have argued that urban schools should proactively respond to these complexities, transforming challenges into opportunities for growth and learning (Thrupp, 1998; Thrupp & Lupton, 2006).

Urban school principals uniquely navigate maintaining positive relationships with their teachers (Ainscow & West, 2006), adapting to constantly changing demographics (Cuban, 2001; Klocko & Justis, 2019), and striving to meet escalating educational goals (Jackson, 2005; May, 2023). These challenges are emblematic of the urban school environment, demanding innovative solutions and a deep understanding of the community and its needs. Teachers in urban schools face their own set of challenges, including a high turnover rate (Carver-Thomas & Darling-Hammond, 2017) and a more demanding work environment (Shann, 1998). Factors contributing to these challenges include, among others, the mismatch between teachers’ and students’ racial and cultural background (Easton-Brooks, 2021; Sleeter, 2017), difficulty of fostering positive teacher–student relationships amid the presence of gang-related issues (Duncan-Andrade, 2007), and navigating hostile racial climates (Kohli, 2018). Despite these challenges, urban teachers are positioned to make a greater impact on their students, providing them with unique learning experiences shaped by the urban context (Knoblauch & Chase, 2015). Students in urban schools experience significant cultural and ethical disparities, which can contribute to a larger achievement gap. However, this also provides a unique opportunity for these schools to become centers of cultural competence and social awareness, as they address and bridge these disparities (Boutte, 2012; Rivera-McCutchen, 2021).

Although urban schools face formidable challenges in creating an equitable, safe, and conducive learning climate, their unique context also positions them as hubs of diversity, innovation, and community engagement. In the face of these inherent complexities and challenges, there is a need for a responsive and adaptive approach to identifying the factors of urban school effectiveness.

Studies of Factors of Urban School Effectiveness

The unique challenges in urban schools naturally lead to an inquiry into the possible differences in school effectiveness factors in urban and non-urban schools. For example, Hallinger and Liu (2016) discovered that teachers’ professional development was less associated with principal leadership in urban schools than in rural ones. However, very few studies have focused on urban school effectiveness to offer frameworks or factors for consideration. Two reports, in particular (Gates et al., 2014; New Leaders for New Schools, 2009), shed light on the impact of the New Leaders Program on both principal and student performance in urban settings. This program was anchored in the Urban Excellence Framework, encompassing five principal work categories: learning and teaching, developing a shared vision, fostering a school-wide culture, supporting learning-oriented operations and systems, and individual leadership. The research indicated that students in schools led by New Leaders principals were twice as likely to enhance their performance over the previous year compared to peers in other schools within the same district. Such progress was noted in both K-8 and high school settings. The gains were especially pronounced in reading as opposed to math at the high school level. Nevertheless, while these findings underscore the importance of leadership in urban schools, they did not account for achievement growth over extended periods or consider other influential factors pertaining to teachers and students.

Summary: The Research Gap in Factors of Urban School Effectiveness

Despite numerous studies on school effectiveness factors and their relationship with school outcomes, notably student achievement, there is a scarcity of studies examining how changes in school effectiveness factors over time correspond to changes in student achievement. There are even fewer studies with repeated measures on both school effectiveness factors and student achievement in urban settings. In-depth studies, such as Scheerens’s (2013) meta-analysis, have delineated certain factors. However, studies focusing on the evolving nature of school effectiveness, especially in urban schools, have been scant. Distinct challenges intrinsic to urban schools, as highlighted by researchers such as Yavuz and Gülmez (2018) and Hallinger and Liu (2016), underline the necessity of a longitudinal approach in studying school effectiveness in an urban setting.

This study attempts to fill this gap by investigating potential factors of school effectiveness and their correlation with student achievement in an urban setting over a 5-year period. We seek to answer: How is the change in factors of school effectiveness associated with the overall trajectory of student achievement in math and reading in an urban school setting over a 5-year period?

Study Design

Our research is embedded in an urban school environment, focusing on data spanning multiple years to trace longitudinal changes in both the school effectiveness factor and student achievement. This multi-year analysis allows us to observe any trends that might be obscured in a cross-sectional study. The primary focus is on the value added to the growth of school-level student achievement by factors of school effectiveness. These factors represent different facets of the measurement of school effectiveness confirmed by confirmatory factor analysis (CFA). More details on the factors and measurement properties are provided below.

Sample

Our sample comprises schools within an urban district in a mid-western state. Over a 5-year span, teacher surveys were conducted, amassing a total of 916 responses. Given the transient nature of the teaching profession, tracking the change in the perception of specific teachers in specific schools over time was not feasible. As a result, we aggregated the teacher responses to the school level, facilitating an examination of general trends and patterns at the school level rather than individual levels. The linear growth model was, therefore, applied at the school level exclusively. A total of six schools had consistent data available for the entire 5-year period. The sampled schools included three elementary schools, two middle schools, and one high school.

Data Collection

There were two primary sources of data for the study. The first source of data was teacher perceptions, collected through an extensive survey administered to all teachers from 2018 to 2022. The survey was designed to garner teachers’ ratings of school effectiveness factors, thus providing a unique perspective on the internal workings of the schools. The second source of data was the measurement of school-level student achievement, which was retrieved from the Department of Education of the state. These official records provided a concrete representation of student performance over the years. It should be noted that the data set for 2020 was missing due to the disruption caused by the COVID-19 pandemic.

Data Analysis

Based on the content analysis of the teacher survey items, we conducted confirmatory factor analysis (CFA) to test our hypothesized measurement model and establish the fit of our selected items to the underlying school effectiveness factors. Due to the categorical nature of the data, we used WLSMV as the estimator. CFA was conducted in Mplus 8.1 (Muthén & Muthén, 2017) and confirmed seven factors of school effectiveness: (a) Shared Vision and Goals, (b) Collegial Trust and Openness, (c) Principal Leadership and Support, (d) Instructional Supports for Struggling Students, (e) Student Behavior Expectations, (f) Behavior Management Practices, and (g) Professional Development and Collaboration. The model fit indices were CFI = 0.952, TLI = 0.947, SRMR = 0.061, and RMSEA = 0.077, which suggested an acceptable level of model fit. Please see Table 1 and Figure 1 for the factors and items as well as the CFA results.

Table 1.

Factors and Items.

Factors and items	Abbreviation
Shared vision and goals	SVG
At this school, we have a common understanding of the objectives we’re trying to achieve with students.	SVG1
Goals and priorities for this school are clear.	SVG2
Most teachers at this school have values and philosophies of education that are similar to my own.	SVG3
Most of my colleagues share a focused vision for student learning.	SVG4
Once we start a new program, we follow up to make sure that it’s working.	SVG5
This school generally chooses only those school improvement opportunities that fit with our improvement goals and strategies.	SVG6
Collegial trust and openness	CTO
Teachers in this school are comfortable discussing beliefs about teaching and learning.	CTO1
Teachers in this school are willing to question one another’s views on issues of teaching and learning.	CTO2
Teachers in this school trust each other.	CTO3
Principal leadership and support	PLS
The principal has confidence in the expertise of the teachers.	PLS1
The principal actively monitors the quality of teaching at this school.	PLS2
The principal sets high expectations for student learning.	PLS3
The principal understands how children learn.	PLS4
The principal carefully tracks students’ academic progress.	PLS5
The principal has my trust.	PLS6
The principal takes a personal interest in the professional development of teachers.	PLS7
The principal presses teachers to implement what they have learned in professional development.	PLS8
The principal looks out for the personal welfare of the teachers.	PLS9
The principal places the needs of children ahead of personal and political interests.	PLS10
The principal at this school is an effective manager who makes the school run smoothly.	PLS11
Instructional supports for struggling students	ISSS
I have received adequate training on effective instructional practices for teaching struggling students.	ISSS1
I have adequate instructional time to effectively identify and address the needs of struggling students in my classes.	ISSS2
I have the supports (resources, staff, etc.) to support struggling students in my classes.	ISSS3
Student behavior expectations	SBE
My school has defined a set of clear expectations for student behavior.	SBE1
My school has clearly defined consequences for students who do not meet expectations for appropriate behavior.	SBE2
Behavior management practices	BMP
My school’s schoolwide procedures for managing behavior are adequate for my classroom.	BMP1
Behavior expectations are explicitly taught to students.	BMP2
Nearly all staff in my school model expected student behaviors.	BMP3
Nearly all staff in my school positively reinforce (e.g., verbally acknowledge, reward) expected student behaviors.	BMP4
Professional development and collaboration	PDC
Review student achievement data to guide my instructional planning	PDC1
Read about educational research findings or best practices	PDC2
Consult with other teachers about challenges I am facing in the classroom	PDC3
Share the content of my lesson plans with other teachers	PDC4
Observe another teacher teaching	PDC5

Figure 1.

Confirmatory factor analysis.

Following the identification of the school effectiveness factors based on CFA, we performed a longitudinal analysis using a linear growth model. This model allows us to examine the trajectory of student achievement growth over the 5 years (four data points) and identify how growth in factors of school effectiveness is associated with the growth in student achievement. The linear growth model was conducted in hierarchical linear model (HLM) 8.2 (Raudenbush & Congdon, 2021).

Modeling Method

We employed a two-level HLM with time-varying covariates to account for the longitudinal nature of the data (multiple time points for each school) for both the independent variable (a school effectiveness factor) and dependent variable (student achievement).

At Level 1, we model the within-school change over time, with the equation:

\begin{array}{l} {Achievement}_{ti} = π_{0 i} + π_{1 i} * ({Year}_{ti}) + π_{2 i} * ({Factor}_{ti}) + \\ π_{3 i} * ({Year}_{ti} \times {Factor}_{ti}) + e_{ti} \end{array}

Achievement_ti is the achievement score for time t for school i. Here, π_0i is the initial status or intercept for time t. π_1i is the rate of change or slope for time t, π_2i is the effect of Factor_ti on the achievement score, and π_3i is the interaction term capturing how the effect of Factor_ti on the achievement score changes over time (Year_ti × Factor_ti). The term e_ti is the within-group error for school i at time t, accounting for unexplained variability within each group.

At Level 2, we model the between-school differences in the change over time and the impact of factors on school effectiveness, using the following equations:

π_{0 i} = β_{00} {+ r}_{00}

π_{10} = β_{10} {+ r}_{10}

π_{20} = β_{20}

π_{30} = β_{30}

Here, β₀₀ is the average initial status across all groups. β₁₀ represents the average effect of time on achievement across all groups (e.g., mean linear growth rates), β₂₀ is the average effect of Factor_ti on the achievement score across all years, and β₃₀ is the average effect of the interaction between Factor_ti and Year_ti across all years.

This two-level model allows us to assess, among others, the effect of changes in school effectiveness factors on changes in student achievement over time. By doing so, our findings provide a more nuanced and detailed understanding of school effectiveness.

Findings

Descriptive Statistics

Table 2 and Table 3 show the data statistics, including student performance in math and reading, as well as the seven CFA identified factors. For math, the percentage of students being proficient or advanced in math ranged from 1.58% to 20.33%, with an average of 8.66% and a standard deviation of 5.27. For reading, the percentages ranged from 5.77% to 38.50%, with a mean of 17.75% and a standard deviation of 8.02. For the school effectiveness factors, each variable is rated on a 4-point scale, with higher numbers indicating more positive teacher perceptions. The average scores ranged from 2.42 (Student Behavior Expectations) to 3.18 (Professional Development and Collaboration), suggesting overall positive views of these factors. However, there is some variability in these perceptions, as indicated by standard deviations ranging from 0.17 (Collegial Trust and Openness) to 0.36 (Student Behavior Expectations).

Table 2.

Descriptive Statistics for the Whole Sample.

	N	Minimum	Maximum	Mean	Std. deviation
Math (%)	24	1.58	20.33	8.66	5.27
Reading (%)	24	5.77	38.50	17.75	8.02
Shared vision and goals	24	2.50	3.62	2.91	0.24
Collegial trust and openness	24	2.61	3.24	2.89	0.17
Principal leadership and support	24	2.42	3.85	3.12	0.32
Instructional supports for struggling students	24	2.33	2.98	2.67	0.21
Student behavior expectations	24	1.48	3.00	2.42	0.36
Behavior management practices	24	2.25	3.22	2.86	0.23
Professional development and collaboration	24	2.78	3.58	3.18	0.18

Table 3.

Descriptive Statistics by Year.

	N	Minimum	Maximum	Mean	Std. deviation
2017–2018
Math (%)	6	1.58	20.33	9.90	6.85
Reading (%)	6	5.77	38.50	18.65	12.52
Shared vision and goals	6	2.50	3.11	2.87	0.23
Collegial trust and openness	6	2.62	3.18	2.89	0.22
Principal leadership and support	6	2.99	3.38	3.14	0.16
Instructional supports for struggling students	6	2.33	2.65	2.48	0.12
Student behavior expectations	6	1.48	2.52	2.15	0.41
Behavior management practices	6	2.25	3.06	2.72	0.28
Professional development and collaboration	6	3.04	3.38	3.22	0.11
2018–2019
Math (%)	6	2.29	19.13	9.32	5.93
Reading (%)	6	9.91	25.89	17.16	6.79
Shared vision and goals	6	2.63	3.62	3.05	0.34
Collegial trust and openness	6	2.80	3.24	2.98	0.17
Principal leadership and support	6	2.57	3.85	3.24	0.45
Instructional supports for struggling students	6	2.43	2.96	2.73	0.20
Student behavior expectations	6	2.04	3.00	2.50	0.36
Behavior management practices	6	2.53	3.22	2.89	0.25
Professional development and collaboration	6	3.03	3.58	3.30	0.20
2020–2021
Math (%)	6	3.40	13.36	7.48	4.05
Reading (%)	6	8.84	26.29	17.00	5.88
Shared vision and goals	6	2.58	3.09	2.90	0.18
Collegial trust and openness	6	2.61	3.02	2.85	0.16
Principal leadership and support	6	2.42	3.36	2.96	0.33
Instructional supports for struggling students	6	2.42	2.98	2.79	0.20
Student behavior expectations	6	2.00	2.96	2.58	0.34
Behavior management practices	6	2.72	3.18	2.98	0.16
Professional development and collaboration	6	2.85	3.30	3.14	0.17
2021–2022
Math (%)	6	3.15	17.33	7.92	4.95
Reading (%)	6	9.26	27.20	18.21	7.51
Shared vision and goals	6	2.65	3.09	2.80	0.16
Collegial trust and openness	6	2.72	3.07	2.83	0.13
Principal leadership and support	6	2.73	3.61	3.15	0.31
Instructional supports for struggling students	6	2.33	2.89	2.66	0.22
Student behavior expectations	6	2.17	2.83	2.44	0.25
Behavior management practices	6	2.62	3.08	2.85	0.17
Professional development and collaboration	6	2.78	3.26	3.07	0.19

Growth Model Results

Table 4 shows the results based on growth modeling. The value added by a school effectiveness factor to the linear growth of student achievement was assessed for both math and reading. For math, the factor Shared Vision and Goals had a statistically significant positive association with Math achievement growth (value added = 4.70, SE = 1.90). The results suggest that a 1-unit increase on Shared Vision and Goals was associated with an increase of 4.70 percentage points in the rate of growth of the math proficiency rate. The other factors showed non-significant but positive effects.

Table 4.

Growth Model Results.

Math	Math		Reading
School effectiveness factors	Value added	SE	Value added	SE
Shared vision and goals	4.70*	1.90	6.44*	1.82
Collegial trust and openness	0.24	1.85	0.29	1.45
Principal leadership and support	2.74	1.67	0.56	1.79
Instructional supports for struggling students	0.79	1.69	1.51	4.33
Student behavior expectations	1.66	1.20	3.54*	1.52
Behavior management practices	1.69	2.56	5.20*	1.97
Professional development and collaboration	4.33	2.69	7.61	4.92

Note: *: p<0.05.

In terms of reading achievement growth, Shared Vision and Goals was again found to be a statistically significant positive factor (value added = 6.44, SE = 1.82). This result indicated that a 1-unit increase in Shared Vision and Goals was associated with a 6.44 percentage point increase in the rate of growth in the reading proficiency rate. Student Behavior Expectations had a statistically significant positive association (value added = 3.54, SE = 1.52). A 1-unit increase on Student Behavior Expectations was associated with an increase of 3.54 percentage points in the rate of growth for the reading proficiency rate. Interestingly, Behavior Management Practices also showed a statistically significant positive association with growth in proficiency rate in reading (value added = 5.20, SE = 1.97). A 1-unit increase in Behavior Management Practices was associated with 5.20 percentage point increase in the growth rate of the reading proficiency rate. Combining the findings on Student Behavior Expectations and Behavior Management Practices, the finding seemed to suggest that better student discipline practices were associated with a positive growth rate in reading proficiency. Other factors including Collegial Trust and Openness, Principal Leadership and Support, Instructional Supports for Struggling Students, and Professional Development and Collaboration showed non-significant, but positive, associations with Reading achievement growth.

Discussion

Our findings provide valuable insights into factors of school effectiveness that contribute significantly to student achievement growth over time in different subjects, in urban settings. Urban schools, often faced with unique challenges shaped by their unique socio-economic and cultural contexts, require tailored strategies to improve student achievement and drive school effectiveness. In what follows, we discuss some key points indicated by the findings.

First, notably, Shared Vision and Goals played a significant role in value-added growth in both math and reading in the sampled schools. Urban schools, often fragmented by diverse cultural backgrounds and socio-economic disparities, can particularly benefit from a shared vision and collective goals. This underscores the importance of school administrators, teachers, and other stakeholders to come up with a unified vision and collective goals in driving student achievement, a finding that aligns with the literature (Lomos et al., 2011; Neuman & Simmons, 2000).

Second, for growth in reading achievement, the significant value-added growth associated with Student Behavior Expectations and Behavior Management Practices suggests the importance of student behavior in general. Urban schools tend to face more challenges in student discipline issues (Anyon et al., 2016; Duncan-Andrade, 2007; Hirschfield & Celinska, 2011). Therefore, in urban schools fostering positive student behavior becomes even more pivotal. Generally, it is more difficult to improve reading achievement than math achievement (Shen et al., 2020; Supovitz, 2002). However, our study illustrates the importance of paying attention to student discipline issues when striving to improve student reading achievement.

Third, the juxtaposition of the statistically significant and non-significant school effectiveness factors seems to suggest that for this urban district at this stage of the improvement cycle, the emphasis should be on the “first things first” types of school effectiveness factors. The three statistically significant positive school effectiveness factors relate to vision and goals as well as discipline. These two aspects seem to be among the “first things first.” These “first things first” school effectiveness factors seem to be the significant levers for inducing positive change in this urban district at this stage of the improvement cycle. Although this set of school effectiveness factors is a good guide, more precise guidance is needed for schools in different contexts and at different stages in the cycles of school improvement.

Finally, the study seems to point to the need to continue to study school effectiveness factors longitudinally in an urban context. In the literature, there is support for all seven factors identified in this studied. However, three out of the seven factors were found to be statistically significant positive predictors, that is, (a) Shared Vision and Goals, (b) Student Behavior Expectations, and (f) Behavior Management Practices. The other four factors had positive correlation with student achievement, but not statistically significant: (a) Collegial Trust and Openness, (b) Principal Leadership and Support, (d) Instructional Supports for Struggling Students, and (d) Professional Development and Collaboration. Is this pattern of the findings for this study due to (a) the urban context, (b) the stage of the school improvement in this particular urban district, or the combination of both the urban context and the stage of the school improvement? When more research is conducted by taking into account the dimensions of geographic location (i.e., urban school vs. others) and time (i.e., longitudinal data), our field will gain more nuanced understanding of school effectiveness factors and provide more precise guidance to improve urban and other schools that are various stage of school improvement process. A list of school effectiveness factors is a good guide, and more precise guidance are needed for schools in different contexts at different cycles of school improvement.

The study is not without limitations. The first limitation is the small sample size and the still limited time span. A larger sample with a longer time span can support more robust modeling and powerful testing capabilities. Although not all factors reached statistical significance, they do show positive associations with growth in both math and reading, suggesting potential areas for further exploration, particularly given the small sample size and still limited longitudinal perspective (i.e., 5 years with four data points). The second limitation is that we did not have non-urban schools and schools at various stage of continuous school improvement in the sample. By having schools with variations along the dimensions of (a) urban versus non-urban and (b) at various stage of continuous school improvement, more direct comparisons could be made along these two dimensions and different sets of school effectiveness factors could possibly be found for urban and non-urban schools at various stages of continuous school improvement. These two limitations could point to directions for future research on this topic.

The results of this study and future similar studies can guide education stakeholders in prioritizing areas for school improvement efforts in an urban school setting. Although many factors had been highlighted in the school effectiveness literature, our longitudinal study found that these three factors are particularly important in an urban setting: (a) Shared Vision and Goals, (b) Student Behavior Expectations, and (c) Behavior Management Practices. The study points to the need for more research and the importance of using a longitudinal dataset in assessing the impact of school effectiveness factors in urban school contexts.

Footnotes

Acknowledgements

The authors acknowledge and are grateful for the teacher survey data provided by American Institutes of Research. All possible errors in data analyses, findings and interpretations are the authors’ responsibility.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The research was supported, in part, by the W.K. Kellogg Foundation.

ORCID iD

Yunzheng Zheng

Author Biographies

Dr. Yunzheng Zheng is a postdoctoral research associate in Western Michigan University. He received his Ph.D. degree in Educational Leadership from Western Michigan University in 2022, He has published 8 papers in professional journals, and has 3 other papers under review.

Dr. Jianping Shen is the John E. Sandberg Professor of Education and the Gwen Frostic Endowed Chair in Research Innovation in Dept. of Educational Leadership, Research and Technology at Western Michigan University. He earned a Ph.D. in educational leadership from University of Washington. He has published more than 100 peer-reviewed journal articles and several books.

Dr. Megan Russell Johnson is a program officer at the W.K. Kellogg Foundation in Battle Creek, Michigan. In this role, she works in the foundation’s priority place of Michigan to support thriving children, working families and equitable communities. Dr. Johnson received her Ph.D. degree in education foundations and policy from the University of Michigan in Ann Arbor.

Dr. Huilan Y. Krenn is the Director of Learning & Impact in W.K. Kellogg Foundation. She holds a Ph.D. degree in evaluation, research and measurement and a master’s degree in educational leadership, both from Western Michigan University. She published widely on evaluation-related topics.

Kimberly Carter, Ed.D., is Superintendent of Battle Creek Public Schools. A long-time educator and leader in public education, she was a teacher, principal, and assistant superintendent.

References

Abdul-Jabbar

Kurshan

(2015). Educational ecosystems: A trend in urban educational innovation. Penn GSE Perspectives on Urban Education, 12(1), n1.

Ainscow

West

(2006). Improving urban schools: Leadership and collaboration. McGraw-Hill Education.

Anyon

Gregory

Stone

Farrar

Jenson

J. M.

McQueen

Downing

Greer

Simmons

(2016). Restorative interventions and school discipline sanctions in a large urban school district. American Educational Research Journal, 53(6), 1663–1697. https://doi.org/10.3102/0002831216675719

Bonner

P. J.

Warren

S. R.

Jiang

Y. H.

(2018). Voices from urban classrooms: Teachers’ perceptions on instructing diverse students and using culturally responsive teaching. Education and Urban Society, 50(8), 697–726. https://doi.org/10.1177/0013124517713820

Boutte

G. S.

(2012). Urban schools: Challenges and possibilities for early childhood and elementary education. Urban Education, 47(2), 515–550. https://doi.org/10.1177/0042085911429583

Boyle

Lamprianou

Boyle

(2005). A longitudinal study of teacher change: What makes professional development effective? Report of the second year of the study. School Effectiveness and School Improvement, 16(1), 1–27. https://doi.org/10.1080/09243450500114819

Carver-Thomas

Darling-Hammond

(2017). Teacher turnover: Why it matters and what we can do about it. Learning Policy Institute. https://learningpolicyinstitute.org/media/174/download?inline&file=Teacher_Turnover_REPORT.pdf

Creemers

B. P. M.

Kyriakides

(2010). Explaining stability and changes in school effectiveness by looking at changes in the functioning of school factors. School Effectiveness and School Improvement, 21(4), 409–427. https://doi.org/10.1080/09243453.2010.512795

Cuban

(2001). Leadership for student learning: Urban school leadership—Different in kind and degree. Institute for Educational Leadership.

10.

Duncan-Andrade

(2007). Gangstas, wankstas, and ridas: Defining, developing, and supporting effective teachers in urban schools. International Journal of Qualitative Studies in Education, 20(6), 617–638. https://doi.org/10.1080/09518390701630767

11.

Easton-Brooks

(2021). Ethnic-matching in urban schools. In Millner

H. R.

Lomotey

(Eds.), Handbook of urban education (pp. 234–252). Routledge.

12.

Gates

S. M.

Hamilton

L. S.

Martorell

Burkhauser

Heaton

Pierson

(2014). Preparing principals to raise student achievement: Implementation and effects of the new leaders program in ten districts (Research report). RAND Corporation.

13.

Hallinger

Heck

R. H.

(2010). Collaborative leadership and school improvement: Understanding the impact on school capacity and student learning. School Leadership and Management, 30(2), 95–110. https://doi.org/10.1080/13632431003663214

14.

Hallinger

Liu

(2016). Leadership and teacher learning in urban and rural schools in China: Meeting the dual challenges of equity and effectiveness. International Journal of Educational Development, 51, 163–173. https://doi.org/10.1016/j.ijedudev.2016.10.001

15.

Harris

Bennett

(2004). School effectiveness and school improvement: Alternative perspectives. Continuum.

16.

Hirschfield

P. J.

Celinska

(2011). Beyond fear: Sociological perspectives on the criminalization of school discipline. Sociology Compass, 5(1), 1–12. https://doi.org/10.1111/j.1751-9020.2010.00342.x

17.

Jackson

(2005). Leadership for urban public schools. Educational Forum, 69(2), 192–202.

18.

Klocko

Justis

R. J.

(2019). Leadership challenges of the rural school principal. The Rural Educator, 40(3), 23–34. https://doi.org/10.35608/ruraled.v40i3.571

19.

Knoblauch

Chase

M. A.

(2015). Rural, suburban, and urban schools: The impact of school setting on the efficacy beliefs and attributions of student teachers. Teaching and Teacher Education, 45, 104–114. https://doi.org/10.1016/j.tate.2014.10.001

20.

Kohli

(2018). Behind school doors: The impact of hostile racial climates on urban teachers of color. Urban Education, 53(3), 307–333. https://doi.org/10.1177/0042085916636653

21.

Leithwood

Louis

K. S.

(Eds.) (2021). Organizational learning in schools. Taylor & Francis.

22.

Lomos

Hofman

R. H.

Bosker

R. J.

(2011). Professional communities and student achievement – A meta-analysis. School Effectiveness and School Improvement, 22(2), 121–148. https://doi.org/10.1080/09243453.2010.550467

23.

May

J. J.

(2023). The market-driven age of education: Challenges of urban school leadership. Mid-Western Educational Researcher, 20(4), 5.

24.

Muthén

(Eds.). (2017). Mplus user’s guide (8th ed.). Author.

25.

Neuman

Simmons

(2000). Leadership for student learning. Phi Delta Kappan, 82(1), 9–12. https://doi.org/10.1177/003172170008200105

26.

New Leaders for New Schools. (2009). Principal effectiveness: A new principalship to drive student achievement, teacher effectiveness and school turnarounds with key insights from the urban excellence framework [Tm]. Executive summary. ERIC Clearinghouse.

27.

Raudenbush

S. W.

Congdon

R. T.

(2021). HLM 8: Hierarchical linear and nonlinear modelling. Scientific Software International, Inc.

28.

Reynolds

(2010). School effectiveness. A&C Black.

29.

Reynolds

Sammons

De Fraine

Van Damme

Townsend

Teddlie

Stringfield

(2014). Educational effectiveness research (EER): A state-of-the-art review. School Effectiveness and School Improvement, 25(2), 197–230. https://doi.org/10.1080/09243453.2014.885450

30.

Rivera-McCutchen

R. L.

(2021). “We don’t got time for grumbling”: Toward an ethic of radical care in urban school leadership. Educational Administration Quarterly, 57(2), 257–289. https://doi.org/10.1177/0013161x20925892

31.

Scheerens

(2013). The use of theory in school effectiveness research revisited. School Effectiveness and School Improvement, 24(1), 1–38. https://doi.org/10.1080/09243453.2012.691100

32.

Seashore Louis

Dretzke

Wahlstrom

. (2010). How does leadership affect student achievement? Results from a national US survey. School Effectiveness and School Improvement, 21(3), 315–336. https://doi.org/10.1080/09243453.2010.486586

33.

Shann

M. H.

(1998). Professional commitment and satisfaction among teachers in urban middle schools. Journal of Education and Research, 92(2), 67–73. https://doi.org/10.1080/00220679809597578

34.

Shen

Wegenke

G. L.

Cooley

V. E.

(2003). Has the public teaching force become more diversified? National and longitudinal perspectives on gender, race, and ethnicity. Educational Horizons, 81(3), 112–118.

35.

Shen

Reeves

Zheng

Ryan

Anderson

(2020). The association between teacher leadership and student achievement: A meta-analysis. Educational Research and Reviews, 31, 1–19. https://doi.org/10.1016/j.edurev.2020.100357

36.

Sleeter

C. E.

(2017). Critical race theory and the whiteness of teacher education. Urban Education, 52(2), 155–169. https://doi.org/10.1177/0042085916668957

37.

Supovitz

J. A.

(2002). Developing communities of instructional practice. Teachers College Record, 104(8), 1591–1626. https://doi.org/10.1111/1467-9620.00214

38.

Thrupp

(1998). Exploring the politics of blame: School inspection and its contestation in New Zealand and England. Comparative Education, 34(2), 195–209. https://doi.org/10.1080/03050069828270

39.

Thrupp

Lupton

(2006). Taking school contexts more seriously: The social justice challenge. British Journal of Educational Studies, 54(3), 308–328. https://doi.org/10.1111/j.1467-8527.2006.00348.x

40.

Vanlaar

Kyriakides

Panayiotou

Vandecandelaere

McMahon

De Fraine

Van Damme

(2016). Do the teacher and school factors of the dynamic model affect high- and low-achieving student groups to the same extent? A cross-country study. Research Papers in Education, 31(2), 183–211. https://doi.org/10.1080/02671522.2015.1027724

41.

Verachtert

Van Damme

Onghena

Ghesquière

(2009). A seasonal perspective on school effectiveness: Evidence from a Flemish longitudinal study in kindergarten and first grade. School Effectiveness and School Improvement, 20, 215–233. https://doi.org/10.1080/09243450902883896

42.

Shen

(2022). The association between principal leadership and student achievement: A multivariate meta-meta-analysis. Educational Research and Reviews, 35(4), 1–17. https://doi.org/10.1016/j.edurev.2021.100423

43.

Yavuz

Gülmez

(2018). Preparing perform and impact ready school leaders for improving urban school success. Research in Educational Administration and Leadership, 3(1), 88–120. https://doi.org/10.30828/real/2018.1.4