Journal of Educational and Behavioral Statistics

This article proposes a multilevel model for the assessment of school effectiveness where the intake achievement is a predictor and the response variable is the achievement in the subsequent periods. The achievement is a latent variable that can be estimated on the basis of an item response theory model and hence subject to measurement error. Ignoring covariate measurement error leads to biased parameter estimates. To address this problem, a likelihood-based measurement error adjustment for multilevel models is proposed. In particular, the method deals with a covariate measured with error that has a random coefficient. An application to educational data from the Italian region of Lombardy illustrates the method.

Manuscript received January 21, 2009 Revision received November 24, 2009 Accepted January 17, 2010

In randomized control trials (RCTs) in the education field, the complier average causal effect (CACE) parameter is often of policy interest, because it pertains to intervention effects for students who receive a meaningful dose of treatment services. This article uses a causal inference and instrumental variables framework to examine the identification and estimation of the CACE parameter for two-level clustered RCTs. The article also provides simple asymptotic variance formulas for CACE impact estimators measured in nominal and standard deviation units. In the empirical work, data from 10 large RCTs are used to compare significance findings using correct CACE variance estimators and commonly used approximations that ignore the estimation error in service receipt rates and outcome standard deviations. The key finding is that the variance corrections have very little effect on the standard errors of standardized CACE impact estimators. Across the examined outcomes, the correction terms typically raise the standard errors by less than 1%, and change p values at the fourth or higher decimal place.

Manuscript received April 16, 2010 Revision received April 26, 2010 Accepted May 24, 2010

Research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Many of these designs involve two levels of clustering or nesting (students within classes and classes within schools). Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies with other studies and to combine information across studies in meta-analyses. This article addresses the problem of defining effect sizes in designs with two levels of clustering and computing estimates of those effect sizes and their standard errors from information that is likely to be reported in journal articles. Five effect sizes are defined corresponding to different standardizations. Estimators of each effect size index are also presented along with their sampling distributions (including standard errors).

School accountability decisions based on standardized tests hinge on the degree of alignment of the test with the state’s standards documents. Yet, there exist no established criteria for judging strength of alignment. Previous measures of alignment among tests, standards, and teachers' instruction have yielded mixed results that are difficult to interpret and to compare across studies. This article reports findings from a simulation study to determine critical values for Porter’s alignment index, suitable for hypothesis testing at alpha levels of .05 and .10, as is familiar to many scholars. It then reexamines the results of previous alignment studies and demonstrates how these findings may be compared through the use of these critical values. The tables of critical values will be useful to researchers or policymakers who seek to judge strength of alignment of standards, assessments, and instruction. The article also describes directions for future research in establishing objective criteria for the interpretation of alignment indices.