The Multilevel Crossed Random Effects Growth Model for Estimating Teacher and School Effects: Issues and Extensions

Abstract

This article examines the multilevel linear crossed random effects growth model for estimating teacher and school effects from repeated measurements of student achievement. Results suggest that even a small degree of unmodeled nonlinearity can result in a substantial upward bias in the magnitude of the teacher effect, which raises concerns about its appropriateness for estimating teacher effects. To address this issue, a piecewise linear crossed random effect growth model is proposed. A comparison with the linear growth form shows that the piecewise specification provides more accurate estimates of teacher effects when achievement growth departs from linear growth across grade levels or over summer, which are prevalent conditions. Fitted examples using nationally representative data and Bayesian estimation methods are provided.

Keywords

crossed random effects teacher effects multilevel growth model piecewise summer learning value added Bayesian

Get full access to this article

View all access options for this article.

References

Alexander, K.L. , Entwisle, D.R. , & Olsen, H.R. ( 2001). Schools, achievement, and inequality: A seasonal perspective . Educational Evaluation and Policy Analysis, 23, 171-191.

Bauer, D.J. , & Cai, L. ( 2009). Consequences of unmodeled nonlinear effects in multilevel models. Journal of Educational and Behavioral Statistics, 34, 97-114.

Browne, W.J. ( 2003). MCMC estimation in MlwiN, Version 2.0. London: Centre for Multilevel Modeling.

Browne, W.J. , & Draper, D. ( 2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1, 473-549.

Burkam, D.T. , Ready, D.D. , Lee, V.E. , & Logerfo, L.F. ( 2004). Social-class differences in summer learning between kindergarten and first grade: Model specification and estimation. Sociology of Education, 77, 1-31.

Clayton, D. , & Rasbash, J. ( 1999). Estimation in large crossed random effect models by data augmentation. Journal of the Royal Statistical Society: Series A, 162, 425-436.

Draper, D. ( 2008). Bayesian multilevel analysis and MCMC. In J. de Leeuw & E. Meijer (Eds.), Handbook of multilevel analysis (pp. 77-140). New York: Springer.

Garner, C.L. , & Raudenbush, S.W. (1991). Neighborhood effects on educational attainment: A multilevel analysis. Sociology of Education, 64, 251-262.

Gelman, A. , Carlin, J. , Stern, H. , & Rubin, D.B. ( 2004). Bayesian data analysis (2nd ed.). Boca Raton, FL: Chapman & Hall.

10.

Goldstein, H. ( 1987). Multilevel covariance component models. Biometrika, 74, 430-31.

11.

Goldstein, H. ( 2003). Multilevel statistical model (3rd ed.). London: Edward Arnold.

12.

National Center for Educational Statistics. ( 2002). User’s guide to the longitudinal kindergarten-first grade public-use data file. Washington, DC: U.S. Department of Education.

13.

Nye, B. , Konstantopoulos, S. , & Hedges, L.V. ( 2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237-257.

14.

Palardy, G.J. , & Rumberger, R.W. (2008). Teacher effectiveness in the first grade: The importance of background qualifications, attitudes, and instructional practices for student learning. Educational Evaluation and Policy Analysis, 30, 111-140.

15.

Raudenbush, S.W. ( 1993). A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research. Journal of Educational and Behavioral Statistics, 18, 321-350.

16.

Raudenbush, S.W. ( 2008). Many small groups. In J. de Leeuw & E. Meijer (Eds.), Handbook of multilevel analysis (pp. 207-236). New York: Springer.

17.

Rowan, B. , Correnti, R. , & Miller, R.J. ( 2002). What large-scale survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104, 1525-1567.

18.

Seltzer, M.H. , Wong, W.H. , & Bryk, A.S. ( 1996). Bayesian analysis in applications of hierarchical models: Issues and methods. Journal of Educational and Behavioral Statistics, 21, 131-167.

19.

Spiegelhalter, D.J. , Best, N.G. , Carlin B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society: Series B, 64, 583-640.

20.

Willett, J.B. ( 1988). Questions and answers in the measurement of change. Review of Research in Education, 15, 345-422.