There has been a recent state-level emphasis on monitoring student outcomes to develop comprehensive school accountability. Such monitoring systems can include content standards and benchmarks to measure progress, statewide assessment instruments, and school report card data that policy makers, school personnel, and parents can also use to compare schools. To be fair, school comparisons should somehow take into consideration differences in communities and the background characteristics of students who live in these communities. Little previous research has examined what report card information might be used to identify school indicators that are related to student achievement and improvement gains across school settings with diverse student composition. The purpose of this study is to present an approach to statewide school comparison that focuses on the value-added effects of report card indicators of elementary schools’ educational environments on school achievement and school improvement after making school-level adjustments for student differences.
Get full access to this article
View all access options for this article.
References
1.
Barr, R. , & Dreeban, R. (1983). How schools work. Chicago: University of Chicago Press.
2.
Bosker, R. , Kremers, E., & Lugthart, E. (1990). School and instructional effects on mathematics achievement. School Effectiveness and School Improvement, 1, 213-248.
3.
Bosker, R. , & Scheerens, J. (1989). Issues in the interpretation of the results of school effectiveness research. International Journal of Educational Research, 13, 213-248.
4.
Brookover, W. , & Lezotte, L. (1977). Changes in school characteristics coincident with changes in student achievement. East Lansing: Michigan State University Press.
5.
Bryk, A. , & Raudenbush, S. (1992). Hierarchical linear models. Newbury Park, CA: Sage.
6.
Bryk, A. , Raudenbush, S., & Congdon, R. (1996). Hierarchical linear modeling with the HLM/2L and HLM/3L Programs. Chicago: Scientific Software.
7.
Burns, R. , & Mason, D. (1998). Class formation and composition in elementary schools. American Educational Research Journal, 35(4), 739-772.
8.
Burstein, L. (1980). The analysis of multilevel data in educational research and evaluation. Review of Research in Education, 8, 158-233.
9.
Burstein, L. (1992). The IEA study of mathematics III: Student growth and classroom processes. Oxford, U.K.: Pergamon.
10.
Chou, C. P., Bentler, P, & Pentz, M. (1998). A two-stage approach to multilevel structural equation models: Application to longitudinal data. Unpublished manuscript.
11.
Creemers, B. (1994). The effective classroom. London: Cassell.
12.
Darling-Hammond, L. (1994). Performance-based assessment and educational equity. Harvard Educational Review, 64(1), 5-29.
13.
de Leeuw, J. , & Kreft, I. (1986). Random coefficient models for multilevel analysis. Journal of Education Statistics, 11, 57-85.
14.
Edmonds, R. (1979). Effective schools for the urban poor. Educational Leadership, 37, 15-24.
15.
Evans, R. (1999, February 3). The great accountability fallacy. Education Week, 28(21), 35-35, 52-52.
16.
Goldstein, H. (1987). Multilevel models in educational and social research. London: Arnold.
17.
Gray, J. , & Simes, N. (1991, August). The stability of school effects over time. Paper presented at the annual conference of the British Educational Research Association, Nottingham, England.
18.
Gross, S. (1993). Early mathematics performance and achievement: Results of a study within a large suburban school system. Journal of Negro Education, 62(3), 269-287.
19.
Guiton, G. , & Oakes, J. (1995). Opportunity to learn and conceptions of educational equality. Educational Evaluation and Policy Analysis, 17(3), 323-336.
20.
Hallinger, P. , & Heck, R. (1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32, 5-44.
21.
Hallinger, P. , & Heck, R. (1998). Exploring the principal’s contribution to school effectiveness: 1980-1995. School Effectiveness and School Improvement, 9, 157-191.
22.
Hallinger, P. , & Murphy, J. (1986). The social context of effective schools. American Journal of Education, 94(3), 328-355.
23.
Hawai‘i Department of Education (1996). Effective schools survey. Honolulu, HI: Department of Education.
24.
Heck, R. , Larsen, T., & Marcoulides, G. (1990). Instructional leadership and school achievement: Validation of a causal model. Educational Administration Quarterly, 26, 94-125.
25.
Heck, R. , & Marcoulides, G. (1996). School culture and performance: Testing the invariance of an organizational model. School Effectiveness and School Improvement, 7, 76-95.
26.
Heck, R. , & Mayor, R. (1993). School characteristics, school academic indicators, and student outcomes: Implications for policies to improve schools. Journal of Education Policy, 8(2), 143-154.
27.
Heck, R. , & Thomas, S. (2000). An introduction to multilevel modeling techniques. Mahwah, NJ: Lawrence Erlbaum.
28.
Hill, P. , & Rowe, K. (1996). Multilevel modelling in school effectiveness research. School Effectiveness and School Improvement, 7, 1-34.
29.
Hox, J. (1995). Applied multilevel analysis (2nd ed.). Amsterdam: TT-Publikaties.
30.
Hu, L. , & Bentler, P. (1995). Evaluating model fit. In R. Hoyle (Ed.), Structural equation models: Concepts, issues, and applications (pp. 76-99). Newbury Park, CA: Sage.
31.
Jackson, K. (1982). Achievement differences in mathematics: A search for causal explanations. Paper presented at the annual meeting of the National Council of Black Studies, Chicago.
32.
Jerald, C. & Boser, U. (1999). Taking stock. Education Week, 28(17), 81-97.
33.
Jöreskog, K. , & Sörbom, D. (1999). LISREL user’s guide. Chicago: Scientific Software.
34.
Kahne, J. (1994). Democratic communities, equity, and excellence: A Deweyan reframing of educational policy analysis. Educational Evaluation and Policy Analysis, 16, 233-248.
Kaplan, D. (1998). Methods for multilevel data analysis. In G. A. Marcoulides (Ed.) Modern methods for business research. Mahwah, NJ: Lawrence Erlbaum.
37.
Kaplan, D. , & Elliott, P. (1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation Modeling, 4(1), 1-23.
38.
Lee, V. , & Bryk, A. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62, 172-192.
39.
Leithwood, K. (1994). Leadership for school restructuring. Educational Administration Quarterly, 30, 498-518.
40.
Marcoulides, G. , & Hershberger, S. (1997). Multivariate statistical methods: A first course. Mahwah, NJ: Lawrence Erlbaum.
41.
McDonnell, L. (1995). Opportunity to learn as a research concept and a policy instrument. Educational Evaluation and Policy Analysis, 17(3), 305-322.
42.
Monk, D. , & Plecki, M. (1999). Generating and managing resources for school improvement. In J. Murphy & K. Seashore-Louis (Eds.), Handbook of research on educational administration (2nd ed.). San Francisco: Jossey-Bass, 491-510.
43.
Mortimore, P. (1993). School effectiveness and the management of effective learning and teaching. School Effectiveness and School Improvement, 4, 290-310.
44.
Mortimore, P. , Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School matters. Wells, England: Open Books.
45.
Muthén, B. , Huang, L., Jo, B., Khoo, S., Goff, G., Novak, J., & Shih, J. (1995). Opportunity-to-learn effects on achievement: Analytical aspects. Educational Evaluation and Policy Analysis, 17(3), 371-403.
46.
Muthén, L. , & Muthén, B. (1998). Mplus user’s guide. Los Angeles: Muthén & Muthén.
47.
National Council on Education Standards and Testing . (1992). Raising standards for American education. Washington, DC: Author.
48.
National Center for Educational Statistics (Report No. NCES 98-015). (1997). Digest of education statistics, 1997. Washington, DC: U.S. Department of Education.
49.
Nuttall, D. , Goldstein, H., Prosser, R., & Rasbash, J. (1989). Differential school effectiveness. International Journal of Educational Research, 13(7), 769-776.
50.
Oakes, J. (1989). What educational indicators? The case for assessing the school context. Educational Evaluation and Policy Analysis, 11(2), 181-199.
51.
Olson, L. (1999). Moving beyond test scores. Education Week, 28(17), 67-73.
52.
Raizen, S. (1993). Approaches to the science curriculum for grades K-12. In S. Fitzsimmons & L. Kerpelman (Eds.), Pre-college teacher enhancement in science and mathematics: Status, issues, and problems. Report submitted to the National Science Foundation. Cambridge, MA: Abt Associates.
53.
Raudenbush, S. (1989). The analysis of longitudinal, multilevel data. International Journal of Educational Research, 13, 721-740.
54.
Raudenbush, S. , & Bryk, A. (1986). A hierarchical model for studying school effects. Sociology of Education, 59, 1-17.
55.
Raudenbush, S. , & Sampson, R. (1999). Assessing direct and indirect effects in multilevel designs with latent variables. Sociological Methods & Research, 28(2), 123-153.
56.
Reynolds, D. , & Cuttance, P. (Eds.). (1992). School effectiveness: Research, policy, and practice. London: Cassell.
57.
Reynolds, D. , & Packer, A. (1992). School effectiveness and school improvement in the 1990’s. In D. Reynolds & P. Cuttance (Eds.), School effectiveness: Research, policy, and practice. (pp. 171-188). London: Cassell.
58.
Salganik, L. (1994). Apples and apples: Comparing performance indicators for places with similar demographic characteristics. Educational Evaluation and Policy Analysis, 16(2), 125-142.
59.
Sammons, P. , Nuttall, D., & Cuttance, P. (1993). Differential school effectiveness: Results of a reanalysis of the Inner London Education Authority’s Junior School Project data. British Educational Research Journal, 19(4), 381-405.
60.
Sammons, P. , Nuttall, D., Cuttance, P., & Thomas, S. (1995). Continuity of school effects: A longitudinal analysis of primary and secondary school effects on GCSE performance. School Effectiveness and School Improvement, 6, 285-307.
61.
Scheerens, J. (1993). Basic school effectiveness research: Items for the research agenda. School Effectiveness and School Improvement, 4, 17-36.
62.
Scheerens, J. , Vermeulen, C., & Pelgrum, W. (1989). Generalizability of instructional and school effectiveness indicators across nations. International Journal of Educational Research, 13(7), 789-799.
63.
Seltzer, M. (1995). Furthering our understanding of the effects of educational programs via a slopes-as-outcomes framework. Educational Evaluation and Policy Analysis, 17(3), 295-304.
64.
Seltzer, M. , Frank, K., & Bryk, A. (1994). The metric matters: The sensitivity of conclusions about growth in student achievement to choice of metric. Educational Evaluation and Policy Analysis, 16(1), 41-50.
65.
Tate, R. , & Wongbundhit, Y.(1983). Random versus nonrandom coefficient models for multi-level analysis. Journal of Educational Statistics, 8, 103-120.
66.
Viadero, D. (1999). Setting the bar: How high?Education Week, 28(17), 21-26.
67.
Wang, J. (1998). Opportunity to learn: The impacts and policy implications. Educational Evaluation and Policy Analysis, 20(3), 137-156.
68.
Wiley, D. , & Yoon, B. (1995). Teacher reports on opportunity to learn: Analyses of the 1993 California Learning Assessment System (CLAS). Educational Evaluation and Policy Analysis, 17(3), 355-370.
69.
Willett, J. , & Sayer, A. (1996). Multilevel models from a multiple group structural equation perspective. In G. Marcoulides & R. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 125-158). Mahwah, NJ: Lawrence Erlbaum.
70.
Willms, J. , & Kerckhoff, A. (1995). The challenge of developing new educational indicators. Educational Evaluation and Policy Analysis, 17(1), 113-131.
71.
Willms, J. , & Raudenbush, S. (1989). A longitudinal hierarchical linear model for estimating school effects and their stability. Journal of Educational Measurement, 26(3), 209-232.
72.
Witte, J. , & Walsh, D. (1990). A systematic test of the effective schools model. Educational Evaluation and Policy Analysis, 12(2), 188-212.
