Modeling the Effects of Educational Resources on Student Achievement

Abstract

The convergence of standards, accountability, and school finance policies necessitates a systematic rethinking about how state-level resource allocation policies can be created to distribute resources in a manner that provides equal educational opportunities for all students. Given the demand for policymakers to distribute adequate resources to improve schools’ capacities to increase student learning, there is a need for evidence detailing the effects of those educational resources on student achievement. Therefore, the purpose of this study was to discern the effects of educational resources on student achievement using structural equation modeling. Using data from a southeastern state in the United States, the authors offer resource allocation policy recommendations that align with the state’s constitutional obligation to provide equality of educational opportunity, particularly for students living in poverty.

Keywords

education finance accountability policy student achievement structural equation modeling

Introduction

Current federal and state accountability policies mandate that all students achieve proficiency of academic standards as measured by statewide testing systems to ensure that students have attained the necessary knowledge to participate in economic and political life (Kress, Zechmann, & Schmitten, 2011). At the forefront of these policies are states’ roles in providing systems of public education that achieve equality of educational opportunities for all students, regardless of their economic backgrounds or learning needs (Goertz, 2001; Weiss, Knapp, Hollweg, & Burrill, 2001). Wong and Nicotera (2007) described the purpose of educational accountability policy as embodying the logic of focusing reform efforts and resources toward improving instructional practices to increase student achievement. With the emphasis on schools’ requirements to continuously raise student achievement, state-level policymakers must make strategic resource allocation decisions to assist schools in meeting their desired student learning goals. To make these decisions, leaders and policymakers need reliable evidence of the effects of specific educational resources on student achievement.

Scholars have recognized the interconnection between the notion of equal opportunity as expressed in accountability and school finance policies (Rebell, 2009; Ryan, 2008; Superfine, 2009). Others have called for the alignment of accountability policies with state finance systems to allocate resources toward student learning goals (Adams, 2008; Verstegen, 2002). These scholars have promoted the alignment of accountability standards and education finance policies as requisite to meet state constitutional clauses that mandate an adequate education for all students. If the goal of current educational accountability policy is for all students to reach proficiency, then states must provide adequate educational resources to meet the learning needs of all students.

Many state finance systems, though, are currently premised on notions of horizontal equity, which seek to distribute comparable funding amounts to school districts regardless of differences in student demographics (Verstegen & Knoeppel, 2012). Furthermore, Adams (2008) noted that the dollars allocated to school districts are rarely traceable to individual students’ learning needs. With the evolution of accountability policies that emphasize improving students’ academic achievement, state finance systems must allocate resources toward meeting specific learning goals. This repurposing of resources with research-based practices that are linked to student achievement may result in significant improvements in student learning. Verstegen (2002) also recognized the inherent linkage between finance and accountability policies, noting that determining an adequate education required the alignment of resources with student achievement results, “With the national emphasis on teaching all students to high standards, new models of state finance systems are needed that align school funding more closely to standards based reform aimed at high outcomes for all children and youth” (p. 749).

The linkage between accountability standards and education finance policy also has implications for school finance litigation (Paris, 2010). Ryan (2008) noted that the connection between standards and testing within school finance litigation has dominated the discourse in education law and policy, particularly in adequacy-based cases. Moreover, Superfine (2009) echoed this sentiment by arguing that the evolution of school finance litigation from equity to adequacy has led to judicial interpretations of laws and evidence concerning standards, testing, and accountability. Kentucky’s landmark school finance case, Rose v. Council for Better Education (1989), marked the convergence of school finance policy, academic content standards, and accountability policy. The adequacy-based case resulted in the Kentucky Supreme Court determining that the state education system was unconstitutional and in need of a complete overhaul. The Kentucky Supreme Court required the legislature to fully fund education so that students could attain competencies in the state’s defined content areas, which would be measured by the state’s standardized exams.

In other adequacy-based school finance cases, plaintiffs have attempted to link state finance models with standards and accountability to discern whether an adequate education was provided to all students. For instance, plaintiffs in Colorado’s Lobato v. State (2009) argued that the constitutional mandate for a thorough and uniform education system was not met, stating that “the state violated the education clause by failing to provide sufficient funds to enable the school districts to satisfy both the content standards and performance objectives in the education reform legislation” (p. 8). The Colorado Supreme Court also noted that “education reform statutes with proficiency targets and content standards . . . may also be used to help evaluate the constitutionality of the legislature’s actions” (p. 15). Also, North Carolina’s Supreme Court decision in Leandro v. State (1997) supported the linkage between state accountability and school finance policies. The court held that “standards adopted by the legislature are factors that may be considered on remand to the trial court for its determination as to whether any of the state’s children are being denied their right to a sound basic education” (p. 4). Moreover, the court suggested that student achievement results could be used to discern whether students are receiving an adequate education:

Another factor that may properly be considered in this determination is the level of performance of the children of the state and its various districts on standard achievement tests. In fact, such output measurements may be more reliable than measurements of input such as per-pupil funding or general educational funding provided by the state. (p. 4)

Although the courts in Colorado and North Carolina have adopted the notion that student achievement outcomes may be traced to resource allocation patterns within states, state education finance systems remain detached from their accountability policies. Specifically, education finance systems were not designed to allocate resources on the basis of students’ differential learning needs. Rather, state finance systems were typically created to distribute similar funding levels to school districts for them to maintain a uniform educational program. Perhaps more research is required to discern the types, intensity, and combination of resources necessary to be allocated to meet state accountability goals. The emergence of standards, accountability, and school finance in education necessitates a systematic rethinking about policy changes geared toward achieving desired student learning goals. Evidence detailing the effects of educational resources on student achievement may provide state and district policymakers with the tools to allocate resources that improve schools’ capacities to foster enhanced student learning.

Literature Review

The current educational accountability context necessitates accurate and reliable information for leaders and policymakers to guide strategic resource allocation decisions toward improved student learning. In particular, scholars have argued that states ought to provide adequate educational resources to ensure equal educational opportunities that meet the learning needs of all students. Scholars have conducted a considerable amount of research investigating the effects of educational resources on student achievement. Most notably, they have made significant efforts in publishing their studies within the controversial “Does Money Matter?” debate, ranging from the 1980s into the late 1990s. Researchers who took part in the scholarly dialogue sought to provide evidence to answer the following question: Do variations in educational resources significantly affect variations in student achievement? In addition, research on the relationship between educational resources and student achievement has served an increasing role in school finance litigation, being used as direct evidence in as few as 30 cases (Rebell, 2009).

The conceptual models that guide these studies assume a systematic relationship between inputs (i.e., educational resources) and outputs (i.e., student achievement outcomes). Inputs are often comprised of student and community characteristics that are measured using independent variables such as household income, percentage of students qualifying for free/reduced lunch, and the educational attainment of the surrounding community. Inputs also include characteristics of schooling such as teacher quality indicators, class sizes, access to technology, and per-pupil expenditures. Outputs may be defined as “the result of the initial and continuing influences on individual student background as modified by the schooling process” (Greene, Huerta, & Richards, 2007, p. 53). The dependent variables typically used to measure educational productivity include student performance on standardized exams, graduation rates, and measures that indicate students’ practice of democratic citizenship. To situate these studies within the overall context of the present study, articles have been divided into two sections based on whether they found significant effects of educational resources on student achievement.

Educational Resources Do Not Significantly Affect Student Achievement

Much of the research on the effects of educational resources on student achievement was sparked by the findings from the 1966 seminal report headed by James S. Coleman, Equality of Educational Opportunity. Coleman et al.’s (1966) study was conducted to assess the progress of the Civil Rights Act of 1964 for the U.S. Department of Health, Education, and Welfare. Using production functions and subsequent descriptive statistics, Coleman and his colleagues reported that student, family, and peer characteristics were more deterministic of students’ achievement than schools. As K. Alexander (1998) stated, Coleman’s findings promoted the notion that “the public schools themselves have little discernible value in enhancing student achievement, the most effective forces being those external to the public schools” (p. 239). Because of the political and social consequences and controversies associated with the findings of the report, research on the effects of resources/spending on student achievement increasingly became prevalent among scholars across the United States.

Hanushek (1981, 1989, 1991, 1996, 1997) conducted the most widely disseminated research on the topic. Hanushek (1981, 1989, 1991) used vote counting methods to synthesize production function studies to estimate the effects of resources on student achievement. These studies primarily comprised of variables including expenditures per pupil, teacher quality indicators, and student–teacher ratio measures and found that those resources had little effects in significantly predicting student achievement. Hanushek (1989) detailed the findings of 152 studies that examined the effects of educational resources on student achievement, finding “no strong evidence that teacher-student ratios, teacher education, or teacher experience have the expected positive effects on student achievement” (p. 47). Hanushek concluded that no conclusive evidence exists to support funding increases to education as a means to improve student performance.

Moreover, Hanushek (1997) updated his research using 377 studies, for which 96 of those studies use value-added modeling to control for student demographic characteristics. After analyzing teacher–pupil ratio, teacher education, salary, and experience, and per-pupil expenditures, he found that there was little evidence to support increasing the amount and intensity of educational resources to promote student achievement. Okpala (2002) also conducted multiple production functions using data from a single district in North Carolina. The researcher examined the effects of school and student/family characteristics on fourth-grade students’ reading and mathematics scores for 3 consecutive school years. The analysis yielded findings that students’ socioeconomic backgrounds were significant predictors of student achievement. In particular, the percentage of students on free or reduced lunch program (β = −.377) and percentage of parents with post–high school education (β = .35) were consistently the strongest predictors of student achievement. Measurements of teacher quality, class size, and expenditures per pupil were found to be insignificant predictors of student achievement.

Educational Resources Do Significantly Affect Student Achievement

In response to the research proclaiming that educational resources do not significantly affect student achievement, the Journal of Education Finance released a 1994 special edition titled, “Further Evidence on Why and How Money Matters in Education.” The edition featured methodological critiques on the research designs that had been used in studies that found non-significant relationships between resources and achievement (Fortune & O’Neil, 1994; Greenwald, Hedges, & Laine, 1994) and offered improved methods, showing significant relationships between educational resources and student achievement (Cooper et al., 1994; Greenwald et al., 1994). Furthermore, other scholars have conducted complex empirical research and have found that specific resources, mainly teacher quality indicators, do have positive effects on variations in student achievement (Archibald, 2006; Ferguson, 1991; Krueger, 2002; Sanders, 1998).

As a research team, Greenwald, Hedges, and Laine (Greenwald et al., 1994; Hedges, Laine, & Greenwald, 1994; Laine, Greenwald, & Hedges, 1996) conducted a series of studies examining the effects of educational resources on student achievement using Hanushek’s (1981, 1989, 1991) data. Greenwald et al. (1994) explained that Hanushek’s (1981, 1989, 1991) inclusion of certain studies in his meta-analysis was not justified or valid and that vote counting yields more conservative findings than other meta-analytic methods. After applying more stringent standards to the inclusion of studies from Hanushek’s data, the authors used a combined significance test meta-analysis method and found that resource inputs of teacher education and salary, administrative inputs, and teacher–pupil ratio have statistically reliable and positive relationships with student achievement. Using an effect magnitude analysis, the authors concluded that a US$500 increase in per-pupil expenditures would yield a 0.7 standard deviation increase in student achievement.

Ferguson (1991) examined the impact of schooling on student achievement using school districts in Texas and found that the differences in the quality of schooling, as measured by teacher quality, class size, and student characteristics, accounted for between one third and two thirds of the variation in students’ test scores. Using multiple regression techniques, the author found that teacher quality, as measured by teachers’ performances on a statewide recertification exam, explained between 20% and 25% of the variation across school districts’ test scores. In his conclusion, Ferguson suggested that money does matter when it is used on high-quality teachers, particularly in schools with lower socioeconomic status. This finding is similar to the research synthesized by Sanders (1998), who claimed that the single largest predictor of gains in student achievement was the quality of teachers in schools.

Cooper et al. (1994) noted that the majority of studies that attempt to estimate the effects of educational resources on student achievement relate inputs to outputs without an understanding of the variations in expenditures within schools and classrooms. Given these limitations, the researchers developed a micro-finance model to trace funds in New York City from the district to 84 high schools and ultimately their classrooms. The researchers were able to cluster groups based on their socioeconomic status and then include expenditures on instruction (their measure of classroom expenditures) and teachers’ years of experience in their multiple regression analysis. The researchers found that per-pupil dollars spent on direct instruction (β = .18) and teachers’ years of experience (β = .11) significantly affected academic achievement, as measured by high schools’ combined average math and verbal Scholastic Aptitude Test (SAT) scores. Furthermore, the model explained 65% of the variance in school’s average SAT scores.

Using a canonical analysis, Knoeppel, Verstegen, and Rinehart (2007) examined the effects of school resources on student achievement. Whereas production functions estimate the effects of multiple independent variables on one dependent variable, a canonical analysis accommodates multiple independent variables and multiple dependent variables. The authors used a host of independent variables, including per-pupil expenditures, student–teacher ratio, days of school, average teacher salary, and a measure of local wealth. In addition, a series of dependent variables were used, including students’ performance on the Iowa Test of Basic Skills (ITBS), schools’ graduation rates, college plans, and voter participation. The analysis yielded average teacher salary (β = .878) and local wealth (β = .349) as the two inputs with the largest effects on student achievement.

Critique of Research Method

Since the 1960s, researchers have used quantitative methods to estimate the effects of educational resources on student achievement. Scholars have used methods that range from simple regressions and production functions to multivariate regression statistics (Verstegen & King, 1998). In addition, economists have informed the research on the relationship between resources and student achievement using economic efficiency frontiers such as Stochastic Frontier Analysis (SFA) and Data Envelopment Analysis (DEA; Rolle, 2005; Ruggiero, 2001, 2007). Despite diverse attempts by scholars in education and economics disciplines to determine the effects of educational resources on student achievement, results remain mixed. Although the general narrative in education finance is that money matters when it is spent on resources that positively affect student achievement and “as long as the resources reach schools, classrooms, teachers and pupils” (Cooper et al., 1994, p. 86), scholars have yet to model a holistic depiction of the direct and indirect links between resources and achievement.

Monk (1990) described some of the limitations of production functions. The author found that production functions produce limited results, particularly because they can only account for a single dependent variable. Conceptually, production functions and other regression models are limited in their abilities to fully depict the effects of resources on student achievement because they account for the variations of student achievement using school- and district-level variables. These variables do not measure variations at the classroom level. Furthermore, researchers have had difficulties in attaining precise measurements of variables. For instance, Rivkin, Hanushek, and Kain (2005) attempted to estimate the effects of teacher quality and class size on student achievement in Texas. They found that little variation in teacher quality was explained by experience and education; however, even when using experience and education as proxies for teacher quality, they found that teachers affected variations in student achievement more than class size.

Verstegen and King (1998) conducted a review of the literature on production functions used to discern the effects of educational resources on student achievement. Despite concluding that educational resources do significantly affect student achievement, the authors noted limitations of production function methods, stating, “Production equations are limited to the degree that they model only the quantitative contributions of resources while leaving aside more qualitative aspects of how resources are deployed in the classroom” (p. 261). Furthermore, the authors described four methodological approaches that could significantly improve production function research:

(1) if individual children and classrooms were the unit of observation rather than the school, district, or state (if other variables could be specified appropriately at that level); (2) if outputs were expressed in terms of progress or longitudinal growth instead of achievement at one point in time; (3) if resources were identified as those available to a specific child rather than by average resources in a classroom, school or district; and (4) if processes were to include the quality, content, and intensity of student–teacher interactions and time on task. (p. 259)

Along the lines of these methodological critiques for discerning the deployment of resources in classrooms and their effects on student achievement, Greene et al. (2007) suggested using the actual instructional resources that are purchased with educational dollars rather than expenditure amounts as independent variables to accurately depict their effects on student achievement. The use of actual instructional resources as variables would result in a more accurate depiction of the educational interactions between teachers and students. Cohen, Raudenbush, and Loewenberg Ball (2003) argued

for a model in which the key causal agents are situated in instruction; achievement is their outcome. Conventional resources can enable or constrain the causal agents in instruction, thus moderating their impact on student achievement. (p. 119)

Multilevel statistical models have been hypothesized and tested to clarify the effects of resources on achievement (Archibald, 2006; Borman & Dowling, 2010; Odden, Borman, & Fermanich, 2004). These authors posited that educational resources are nested within classrooms and schools and thus should be examined using hierarchical linear modeling to explain variances at the different levels. Whereas other scholars have found that family background and socioeconomic indicators explain a large proportion of the variance in student achievement, Borman and Dowling (2010) found that about 40% of the variance in achievement was attributed to differences at the schooling levels. Thus, multilevel models are better suited to disentangle the effects of resources on achievement.

Although the use of multilevel modeling has provided useful estimates, the methods used in the current line of research have not accounted for moderating or mediating variables. In addition, hierarchical linear modeling does not account for latent variables (e.g., independent variables with similar characteristics). Certainly, the analysis of variables nested in schools and classrooms is appropriate for research of this type. However, more research is needed to capture the direct and indirect effects between resources and student achievement. Therefore, the purpose of this study is to test the effects of educational resources on student achievement using structural equation modeling (SEM) as a means of clarifying the discourse within the “Does Money Matter?” debate. If SEM is found to be a viable method by the researchers, then multilevel SEM may be the next step in this line of research.

Theoretical Framework

The concept of educational adequacy is inherent in an ideal education system that achieves equality of opportunity for all students. An adequate education system is one that provides the appropriate amount and type of resources and services so that all students have equal opportunities to achieve their learning goals. Many scholars have provided in-depth definitions of adequacy that vary in degree of complexity (N. A. Alexander, 2004; Baker, 2005; King, Swanson, & Sweetland, 2005; Ladd, 2008; Verstegen, 2002). Ladd (2008) noted that adequacy requires the differential treatment of students with different needs. In addition, she argued that adequacy also entails sufficiency of resources to meet the learning needs of all students. Adequacy has also been characterized as vertical equity in the ideal (King et al., 2005); schools with larger percentages of students who require differential services should receive sufficient funding to teach those students at high levels (Toutkoushian & Michael, 2007). The achievement of educational adequacy is contingent on the type, amount, and intensity of resources that are needed to provide the necessary instructional conditions and opportunities for all students to achieve at high levels. Furthermore, N. A. Alexander (2004) and Baker (2005) both recognized that adequacy is also influenced by the type and difficulty of outcomes that are determined for the students. The shape of educational adequacy, then, may be directly influenced by the goals of education accountability policies established at the federal, state, and district levels.

Method

The present study attempted to advance scholarship on the effects of educational resources on student achievement by using SEM. Whereas past researchers have analyzed the effects of resources on student achievement using multivariate techniques with multiple independent variables, SEM is a “sophisticated statistical method for testing complex causal models in which the dependent and independent variables are latent” (Vogt, 2005, p. 313). Because SEM requires an a priori approach to statistical testing, it lends itself well to the analysis of data for inferential purposes (Byrne, 2012).

The method is comprised of three aspects that together set it apart from other causal modeling techniques. First, SEM allows researchers to combine similar independent variables into theoretical constructs, or latent variables, which are unobservable factors that influence multiple observed variables and account for the correlations among those variables (Brown, 2006). Moreover, “it is a construct . . . a theoretical entity inferred from a pattern of relations among observed variables” (Vogt, 2005, p. 313). Second, the causal process under examination is represented by a series of regression equations that are simultaneously conducted by the researcher. Third, the regression equations “can be modeled pictorially to enable a clearer conceptualization of the theory under study” (Byrne, 2012, p. 3). In addition to the foundational aspects of SEM, the statistical approach accounts for a multitude of characteristics essential to causal modeling, including endogenous and exogenous variables, recursive and non-recursive models, and reflective and formative models. These aspects allow the researcher to test complex causal models that otherwise cannot be calculated using regression methods or other multivariate techniques (Byrne, 2012).

Before the SEM is conducted, the latent variables must be tested using confirmatory factor analysis (CFA) and the model must be determined to be testable using model identification standards and goodness-of-fit indices (Byrne, 2012). Model identification refers to whether there is a sufficient number of degrees of freedom in the model, thus, helping the researcher determine “whether or not there is a unique set of parameters consistent with the data” (Byrne, 2012, p. 32). If the model is overidentified, meaning that the number of parameters in the model is less than the number of sample moments, the model is sufficient. Because the variables are non-normal, a robust maximum likelihood (MLR) estimator was used to test the model fit (Byrne, 2012). In addition, goodness-of-fit indices, such as the comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR), should be used to evaluate whether the model fits the data. Brown (2006) offered guidelines for interpreting these goodness-of-fit indices. He noted that comprehensive evaluations of cutoff criteria for these indices found that adequate model fit is “obtained in instances where (1) SRMR values are close to .08 or below; (2) RMSEA values are close to .06 or below; and (3) CFI and TLI values are close to .95 or greater” (p. 87). However, Brown did note that other scholars have found that CFI and TLI values between .90 and .95 reflect an adequate model fit. Moreover, grand mean centering was conducted to reduce the likelihood of multicollinearity (Heck & Thomas, 2009).

Data and Variables

The present study was conducted using data from elementary schools in a state located in the Southeastern United States. During the 2012-2013 school year, the state was comprised of more than 650 public elementary schools, more than 300 public middle schools, and more than 215 public high schools. Existing data for the 2012-2013 school year were obtained from the state’s Department of Education website. In particular, data were collected from published files including data that were used for elementary schools’ report cards. The initial sample of elementary schools was more than 650; however, that was reduced to 470 to maintain some form of uniformity in the sample—only elementary schools that tested Grades 3 through 5 were included in the study.

Various school-level independent variables were used to model and predict the effects of educational resources on student achievement. As can be seen in the conceptual model in Figure 1, the observed variables were situated within latent variables. The student characteristics latent variable was comprised of the poverty index, the percentage of students with disabilities other than speech, the percentage of students eligible for gifted and talented, the percentage of students retained, and the percentage of students older than usual for grade. In addition, the second latent variable, instructional condition, was comprised of the schools’ student–teacher ratio, the percentage of expenditures for instruction, and the number of professional development days per teacher. The third latent variable, personnel, was comprised of schools’ average teacher salary, percentage of teachers with advanced degrees, percentage of teachers with continuing contracts, the percentage of returning teachers, and each principal’s years in the school. The student achievement dependent variable was the State’s 2013 Elementary and Secondary Education Act (ESEA) Waiver Index score, which is a composite index score that is calculated for each public school in the state. The score is comprised of multiple achievement indicators from the state’s standardized tests. In particular, the index score includes weighted measures of achievement in English language arts (ELA), mathematics, science, and social studies. Each school’s score is weighted by the percentage of students tested on the assessments and ranges from zero to 100.

Figure 1.

Conceptual model to be tested.

Findings

Descriptive Statistics

The mean, standard deviation, variance, minimum value, and maximum value for all 15 variables used in the analysis are displayed in Table 1. The total sample size for the study was 470 elementary schools that tested Grades 3 through 5. Six of the variables were missing data as reported by the state Department of Education; however, Mplus, the statistical software used for this study, accommodates missing data (Muthén & Muthén, 1998-2012). Moreover, the data were relatively normal; however, to improve the reliability of the subsequent analysis, a MLR estimator was used to test the model fit.

Table 1.

Descriptive Statistics for All Variables.

	N	M	SD	Variance	Minimum	Maximum
Poverty index	470	74.98	20.41	416.70	14.12	99.79
% gifted and talented	470	8.67	5.66	32.13	0.0	41.90
% special education	470	12.74	3.79	14.40	4.40	28.10
% students retained	470	1.266	1.22	1.49	0.00	7.00
% students older than grade	470	2.21	1.64	2.7	0.00	10.50
% returning teachers	456	87.65	5.68	32.259	57.40	98.40
Average teacher salary	470	48,309.31	3,478.15	12,097,584.84	37629	57928
% teachers with adv. degree	470	65.56	11.07	122.63	22.20	91.30
% teachers with continuing contract	470	82.52	10.75	115.59	48.80	100
Principal’s years at school	467	5.79	4.83	23.33	0.00	29.00
Professional development days per teacher	467	11.63	5.38	28.92	1.00	31.00
Student–teacher ratio	453	19.836	2.95	8.73	2.30	31.20
% expenditures for instruction	466	68.3	5.48	30.08	48.00	84.00
Total school enrollment	470	524.39	203.33	41,345.65	88.00	1,372.00
2013 ESEA waiver index score	470	83.54	16.05	257.64	10.5	100.00

Note. ESEA = Elementary and Secondary Education Act.

CFA

The conceptual model was overidentified, with 53 degrees of freedom and 51 free parameters, indicating that the model was sufficient for analysis. Model fit indices yielded confirming results. The CFI and TLI were either greater than or equal to the .90 standard, at .93 and .90, respectively. The RMSEA and SRMR were found to be .06 and .05, respectively. Because the model fit indices indicated confirming results, the model was deemed appropriate for the SEM. The standardized estimates for the model are detailed in Table 2. For the student characteristics latent variable, schools’ poverty indices and percentage of students eligible for gifted and talented had the largest loadings of .836 and −.873, respectively. Schools’ percentage of students older than usual for grade had a moderate loading of .631. In addition, three of the observed variables within the personnel latent variable had moderate to large loadings. Schools’ percentage of teachers on continuing contracts, average teacher salary, and percentage of returning teachers had loadings of .631, .748, and −.603, respectively. For the instructional condition latent variable, the schools’ student–teacher ratios had a moderate loading of .578. Higher absolute values of loadings suggest that those variables are the distinguishing features of the latent variables.

Table 2.

Standardized Estimates From the CFA.

	Estimate	SE	p value
Student characteristics
Poverty index	.836	.03	<.001
% gifted and talented	−.873	.03	<.001
% special education	.381	.05	<.001
% students retained	.275	.04	<.001
% students older than grade	.631	.04	<.001
Personnel
% returning teachers	−.603	.06	<.001
Average teacher salary	.748	.06	<.001
% teachers with adv. degree	.474	.06	<.001
% teachers with continuing contract	.693	.05	<.001
Principal’s years at school	.208	.05	<.001
Instructional condition
Professional development days per teacher	−.031	.07	.682
Student–teacher ratio	.578	.16	<.001
% Expenditures for instruction	−.217	.07	.003

Note. CFA = confirmatory factor analysis.

SEM

The model that was tested using SEM is displayed in Figure 1. The model was overidentified, with 62 degrees of freedom and 57 free parameters, and found to be sufficient to examine the model fit indices. Model fit indices suggested a proximate adequate fit. The CFI was found to be .93 and the TLI was .90; the RMSEA was .06 and the SRMR was .05. The model predicted 35.2% of the variation in the schools’ 2013 waiver index score, 19.6% of the variation in the personnel latent variable, and 31.3% of the variation in the instructional condition latent variable. Similar to the CFA results, the student characteristics and personnel latent variables had the strongest loadings (see Table 3). For the student characteristics latent variable, schools’ poverty indices and percentage of students eligible for gifted and talented had the largest loadings of .843 and −.871, respectively. Schools’ percentage of students older than usual for grade had a moderate loading of .613. In addition, schools’ percentage of students with disabilities other than speech and the percentage of students retained had the smallest loadings of .375 and .259, respectively.

Table 3.

Standardized Estimates for the Structural Model.

	Estimate	SE	p value	R ²
2013 Waiver Index Score				.352
Student characteristics
Poverty index	.843	.02	<.001	.711
% gifted and talented	−.871	.03	<.001	.758
% special education	.375	.05	<.001	.141
% students retained	.259	.04	<.001	.067
% students older than grade	.613	.04	<.001	.375
Personnel				.196
% returning teachers	−.611	.05	<.001	.374
Average teacher salary	.714	.05	<.001	.510
% teachers with adv. degree	.441	.05	<.001	.194
% teachers with continuing contract	.721	.05	<.001	.519
Principal’s years at school	.208	.05	<.001	.043
Instructional condition				.313
Professional development days per teacher	−.013	.08	.873
Student–teacher ratio	.580	.15	<.001	.336
% expenditures for instruction	−.225	.07	.003	.051

Note. R² is reported for variables with p values less than .05.

For the personnel latent variable, schools’ percentage of teachers on continuing contracts, average teacher salary, and percentage of returning teachers had moderate to large loadings of .721, .714, and −.611, respectively. Schools’ percentage of teachers with advanced degrees and principals’ years in the schools had the smallest loadings of .441 and .208. Two of the observed variables loaded significantly on the instructional condition latent variable with schools’ student–teacher ratio at .580 and percentage of expenditures for instruction at −.225. The instructional condition latent variable should be interpreted with caution because the loadings vary considerably.

A visual representation of the regression estimates for the post hoc SEM can be viewed in Figure 2. Significant weightings were marked with an asterisk. For the standardized direct effects between the latent variables, student characteristics (β = −.512) and personnel (β = .243) significantly predicted schools’ achievement as measured by their 2013 ESEA waiver index score (R² = .352). The instructional condition latent variable was not a significant predictor of the waiver index score. In addition, the personnel latent variable served as a moderating variable between student characteristics and the ESEA waiver index score. Student characteristics negatively predicted (β = −.440) personnel, which positively predicted (β = .243) the schools’ ESEA waiver score. The total indirect effect of student characteristics on schools’ ESEA waiver score was −.12. This was determined by multiplying the regression coefficient for personnel on student characteristics and the ESEA waiver score on personnel. The student characteristics latent variable was also a positive and significant predictor (β = .560) of schools’ instructional conditions. The standardized regression coefficients also serve as measures of effect size. When compared with standards in the social sciences (Cohen, 1988), the effect sizes for student characteristics on achievement and student characteristics on personnel were moderate and the effect size for personnel on student achievement was small. However, the size of the effect of personnel on student achievement was large relative to the findings of studies within the “Does Money Matter?” debate.

Figure 2.

Standardized estimates.

Discussion

With the increase of educational policies centered on improving students’ academic achievement toward specific student learning goals, reliable models that explain the relationship between educational resources and student achievement has become even more necessary. In particular, the alignment between accountability and school finance policies lends itself to the use of research on the effects of resources on achievement; findings relate with school finance litigation purposes (Rebell, 2009) and could be used to inform resource allocation policies to direct specific resources to school districts and schools to improve student learning. Since the 1960s, researchers have investigated the relationship between educational resources and student achievement and have found mixed results (Archibald, 2006; Coleman et al., 1966; Cooper et al., 1994; Greene et al., 2007; Greenwald et al., 1994; Hanushek, 1981, 1989, 1991, 1997; Knoeppel et al., 2007; Okpala, 2002). Whereas some researchers have found significant relationships between variations in resources, such as teacher qualifications and per-pupil expenditures, and variations in student achievement (Archibald, 2006; Cooper et al., 1994; Greenwald et al., 1994; Knoeppel et al., 2007), other researchers have found non-significant effects (Coleman et al., 1966; Hanushek, 1981, 1989, 1991; Okpala, 2002). As a result of differences in these findings, scholars have yet to model a holistic representation of the relationship between resources and achievement.

SEM was found to be a plausible method that improved the reliability of the findings when modeling the effects of educational resources on student achievement. In particular, the use of SEM comprised of multiple latent variables and direct and indirect effects between those variables. This allowed researchers to obtain more precise estimated effects between resources and achievement. Moreover, the findings of the present study suggest that traditional economic analytical frameworks that represent “inputs and outputs” may limit the potential for researchers to account for unique effects of educational resources on student achievement because they do not accommodate mediating and moderating variables.

The model explained 35.2% of the variation in the schools’ measure of student achievement, 19.6% of the variation in the schools’ measure of personnel, and 31.3% of the variation in the instructional condition latent variable. A finding consistent with the literature was a significant negative direct effect between the students’ characteristics in the schools and the schools’ overall achievement levels. In addition, a key finding of the study was that the model provided empirical evidence that confirms Sanders’s (1998) suggestion that after controlling for student demographics, “the single largest factor affecting academic growth of populations of students is differences in effectiveness of individual classroom teachers” (p. 27). Still, further research could help strengthen this line of research. In accordance with research using multilevel analyses (Archibald, 2006; Borman & Dowling, 2010; Odden et al., 2004), the present model should be examined within a multilevel framework to ensure the reliability and validity of the results.

Notwithstanding this limitation, the findings from the present study also serve as a basis for recommendations for resource allocation policy. To some degree, the strategic deployment of resources based on students’ economic circumstances and needs would partially fulfill states’ obligations to provide equality of educational opportunity. A significant policy implication emerged from the analysis: Modifications should be made to the current state finance formula to lessen the effects of students’ characteristics on student achievement. The state does not distribute additional funds to schools based on their poverty index; 37 states included weightings for low-income or compensatory education (Verstegen, 2011). In this study, schools’ poverty indices were found to have a large loading for the student characteristics latent variable, which was a significant and negative predictor of personnel and student achievement. Because the state does not distribute additional funds to schools based on each school’s poverty index, an additional student weighting could be established that provides additional funds for schools to invest in school personnel. The additional poverty weighting would direct funds to school districts to provide them the capacities to devise programs or structures that have been proven to recruit, retain, and train teachers and administrators to work in schools with students living in poverty. For instance, funds could be allocated specifically to schools for them to use to incentivize teachers and administrators to work in schools with higher poverty indices. Based on the findings of this model, funds for teacher incentive strategies would make more sense than additional funding toward instructional condition variables given the non-significant relationship between schools’ instructional conditions and student achievement measures.

The model could then serve as a tool to evaluate the effectiveness of funding formula policy changes. For instance, with the new student weighting policy, if the redistribution of funds based on schools’ poverty indices were to be effective in providing adequate educational services that are directed toward increasing student achievement, then the estimated effects of students’ characteristics on achievement should decrease and the effects of personnel and the instructional condition on achievement should increase. Judgments could then be made about whether the state is furthering its attempts to provide a just system of education premised on the notions of adequacy and equality of educational opportunity. With this policy in place, the state may further its efforts in providing an adequate education for all students so that they have fair opportunities to learn at high levels.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Matthew R. Della Sala is an assistant professor of educational studies at Purdue University. His research interests center on education finance and policy with attention to how state-level policies affect states’ progress in providing educational systems that achieve broader philosophical concepts such as equity, adequacy, justice, and equality of educational opportunity.

Robert C. Knoeppel is an associate professor and chair of the Faculty of Leadership, Counselor Education and Human & Organizational Development at Clemson University. His research interests include the intersection of school finance and education accountability policy.

Russ Marion is a professor in educational leadership at Clemson University.

References

Adams

J. E.

Jr. (2008). Funding student learning: How to align education resources with student learning goals (A report prepared for the School Finance Redesign Project). Retrieved from http://www.crpe.org/sites/default/files/pub_sfrp_wrkgrp_oct08_0.pdf

Alexander

(1998). Money matters: Commentary and analyses. Journal of Education Finance, 24, 237-242.

Alexander

N. A.

(2004). Exploring the changing face of adequacy. Peabody Journal of Education, 79(3), 81-103.

Archibald

(2006). Narrowing in on educational resources that do affect student achievement. Peabody Journal of Education, 81(4), 23-42.

Baker

B. D.

(2005). The emerging shape of educational adequacy: From theoretical assumptions to empirical evidence. Journal of Education Finance, 30, 259-287.

Borman

Dowling

(2010). Schools and inequality: A multilevel analysis of Coleman’s equality of educational opportunity data. The Teachers College Record, 112, 1201-1246.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

Byrne

(2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

10.

Cohen

D. K.

Raudenbush

S. W.

Loewenberg Ball

(2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25, 119-142.

11.

Coleman

J. S.

Campbell

E. Q.

Hobson

C. J.

McPartland

Mood

A. M.

Weinfeld

F. D.

York

R. L.

(1966). Equality of educational opportunity. Washington, DC: U.S. Government Printing Office.

12.

Cooper

B. S.

Sarrel

Darvas

Alfano

Meier

Samuels

Heinbuch

(1994). Making money matter in education: A micro-financial model for determining school-level allocations, efficiency, and productivity. Journal of Education Finance, 20, 66-87.

13.

Ferguson

R. F.

(1991). Paying for public education: New evidence on how and why money matters. Harvard Journal of Legislation, 28, 465-498.

14.

Fortune

J. C.

O’Neil

J. S.

(1994). Production function analysis and the study of educational funding equity: A methodological critique. Journal of Education Finance, 20, 21-46.

15.

Goertz

M. E.

(2001). Standards-based accountability: Horse trade or horse whip? In Fuhrman

S. H.

(Ed.), From the capitol to the classroom: Standards-bassed reform in the United States (pp. 39-59). Chicago, IL: University of Chicago Press.

16.

Greene

G. K.

Huerta

L. A.

Richards

(2007). Getting real: A different perspective on the relationship between school resources and student outcomes. Journal of Education Finance, 33, 49-68.

17.

Greenwald

Hedges

L. V.

Laine

R. D.

(1994). When reinventing the wheel is not necessary: A case study in the use of meta-analysis in education finance. Journal of Education Finance, 20, 1-20.

18.

Hanushek

E. A.

(1981). Throwing money at schools. Journal of Policy Analysis and Management, 1, 19-41.

19.

Hanushek

E. A.

(1989). The impact of differential expenditures on school performance. Educational Researcher, 18(4), 45-51.

20.

Hanushek

E. A.

(1991). When school finance “reform” may not be good policy. Harvard Journal of Legislation, 28, 465-498.

21.

Hanushek

E. A.

(1996). School resources and student achievement. In Burtless

(Ed.), Does money matter? The effect of school resources on student achievement and adult success (pp. 43-73). Washington, DC: Brookings Institution Press.

22.

Hanushek

E. A.

(1997). Assessing the effects of school resources on student performance: An update. Educational Evaluation and Policy Analysis, 19, 141-164.

23.

Heck

R. H.

Thomas

S. L.

(2009). An introduction to multilevel modeling techniques (2nd ed.). New York, NY: Routledge.

24.

Hedges

L. V.

Laine

R. D.

Greenwald

(1994). Does money matter? A meta analysis of studies of the effects of differential school inputs on student outcomes. Educational Researcher, 23(3), 5-14.

25.

King

R. A.

Swanson

A. D.

Sweetland

S. R.

(2005). Designing finance structures to satisfy equity and adequacy goals. Education Policy Analysis Archives, 13(15), 1-26.

26.

Knoeppel

R. C.

Verstegen

D. A.

Rinehart

J. S.

(2007). What is the relationship between resources and student achievement: A canonical analysis. Journal of Education Finance, 33, 183-202.

27.

Kress

Zechmann

Schmitten

J. M.

(2011). When performance matters: The past, present, and future of consequential accountability in public education. Harvard Journal on Legislation, 48, 185-234.

28.

Krueger

A. B.

(2002). Understanding the magnitude and effect of class size on student achievement. In Mishel

Rothstein

(Eds.), The class size debate (pp. 7-35). Washington, DC: Economic Policy Institute.

29.

Ladd

H. F.

(2008). Reflections on equity, adequacy, and weighted student funding. Education Finance and Policy, 3, 402-423.

30.

Laine

R. D.

Greenwald

Hedges

L. V.

(1996). Money does matter: A research synthesis of a new universe of education production function studies. In Picus

L. O.

Wattenbarger

J. L.

(Eds.), Where does the money go? Resource allocation in elementary and secondary schools (pp. 44-70). Thousand Oaks, CA: Corwin Press.

31.

Leandro v. State, 346 N.C.336; 488 S.E.2d 249 (1997).

32.

Lobato v. State, 218 P.3d 358 (2009).

33.

Monk

D. H.

(1990). Educational finance: An economic approach. New York, NY: McGraw-Hill.

34.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.

35.

Odden

Borman

Fermanich

(2004). Assessing teacher, classroom, and school effects, including fiscal effects. Peabody Journal of Education, 79(4), 4-32.

36.

Okpala

C. O.

(2002). Educational resources, student demographics and achievement scores. Journal of Education Finance, 27, 885-907.

37.

Paris

(2010). Framing equal opportunity: Law and politics of school finance reform. Stanford, CA: Stanford University Press.

38.

Rebell

M. A.

(2009). Courts & kids: Pursuing educational equity through the state courts. Chicago, IL: University of Chicago Press.

39.

Rivkin

S. G.

Hanushek

E. A.

Kain

J. F.

(2005). Teachers, schools, and academic achievement. Econometrica, 73, 417-458.

40.

Rolle

(2005). Rethinking educational productivity and its measurement: A discussion of stochastic frontier analysis within a budget-maximizing framework. In Stiefel

Schwartz

A. E.

Rubenstein

Zable

(Eds.), Measuring school performance and efficiency: Implications for practice and research (2005 Yearbook of the American Education Finance Association) (pp. 185-204). Larchmont, NY: Eye on Education.

41.

Rose v. Council for Better Education, 790 S.W.2d 186 (1989).

42.

Ruggiero

(2001). Determining the base cost of education: An analysis of Ohio school districts. Contemporary Economic Policy, 19, 268-279.

43.

Ruggiero

(2007). Measuring the cost of meeting minimum educational standards: An application of data envelopment analysis. Education Economics, 15, 1-13.

44.

Ryan

J. E.

(2008). Standards, testing, and school finance. Texas Law Review, 86, 1223-1262.

45.

Sanders

W. L.

(1998). Value added assessment. School Administrator, 11(55), 24-27.

46.

Superfine

B. M.

(2009). Deciding who decides questions at the intersection of school finance reform litigation and standards-based accountability policies. Educational Policy, 23, 480-514.

47.

Toutkoushian

R. K.

Michael

R. S.

(2007). An alternative approach to measuring horizontal and vertical equity in school funding. Journal of Education Finance, 32, 395-421.

48.

Verstegen

D. A.

(2002). Financing the new adequacy: Towards new models of state education finance systems that support standards based reform. Journal of Education Finance, 27, 749-781.

49.

Verstegen

D. A.

(2011). A quick glance at school finance: A 50 state survey of school finance policies. Retrieved from http://schoolfinancesdav.wordpress.com/

50.

Verstegen

D. A.

King

R. A.

(1998). The relationship between school spending and student achievement: A review and analysis of 35 years of production function research. Journal of Education Finance, 24, 243-262.

51.

Verstegen

D. A.

Knoeppel

R. C.

(2012). Financing education: A survey of finance policies and programs across the 50 states. Journal of Education Finance, 38, 145-166.

52.

Vogt

W. P.

(2005). Dictionary of statistics and methodology: A nontechnical guide for the social sciences (2nd ed.). Thousand Oaks, CA: Sage.

53.

Weiss

I. R.

Knapp

M. S.

Hollweg

K. S.

Burrill

(Eds.). (2001). Investigating the influence of standards: A framework for research in mathematics, science, and technology education. Washington, DC: National Academies Press.

54.

Wong

K. K.

Nicotera

(2007). Successful schools and educational accountability: Concepts and skills to meet leadership challenges. Boston, MA: Pearson.