Effect Sizes in Three-Level Cluster-Randomized Experiments

Research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Many of these designs involve two levels of clustering or nesting (students within classes and classes within schools). Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies with other studies and to combine information across studies in meta-analyses. This article addresses the problem of defining effect sizes in designs with two levels of clustering and computing estimates of those effect sizes and their standard errors from information that is likely to be reported in journal articles. Five effect sizes are defined corresponding to different standardizations. Estimators of each effect size index are also presented along with their sampling distributions (including standard errors).