Effect Sizes and their Intervals: The Two-Level Repeated Measures Case

Abstract

Probability coverage for eight different confidence intervals (CIs) of measures of effect size (ES) in a two-level repeated measures design was investigated. The CIs and measures of ES differed with regard to whether they used least squares or robust estimates of central tendency and variability, whether the end critical points of the interval were obtained using a theoretical or an empirical sampling distribution, and whether the ESs used a pooled or nonpooled estimate of error variability. These intervals were compared when data were obtained from both normal and nonnormal distributions and when the population magnitude of effect, size of sample, and variance heterogeneity were varied. Itwas found that the ESs and intervals that used robust estimators and critical values were obtained through a bootstrap method better at controlling the probability coverage (i.e., within [.925, .975]).

Keywords

effect size confidence intervals robust procedure

Get full access to this article

View all access options for this article.

References

Algina, J. , & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63, 537-553.

Bird, K. D. (2002). Confidence intervals for effect sizes in analysis of variance. Educational and Psychological Measurement, 62, 197-226.

Bradley, J. V. (1978). Robustness? British Journal of Mathematical& Statistical Psychology, 31, 144-152.

Cumming, G. , & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-574.

Glass, G. V , McGaw, B. ,& Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

Hoaglin, D. C. (1985). Summarizing shape numerically: The g- and h- distributions. In D. C. Hoaglin , F. Mosteller , & J. W. Tukey (Eds.), Exploring data tables, trends, and shapes(pp. 461-513). New York: John Wiley.

SAS Institute Inc. (1999). SAS/IML user’s guide, version 8. Cary, NC: Author.

Steiger, J. H. ,& Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L. Harlow , S. Mulaik ,& J. H. Steiger (Eds.), What if there were no significance tests?(pp. 221-257). Hillsdale, NJ: Lawrence Erlbaum.

Wilcox, R. R. (1997). Introduction to robust estimation and hypothesis testing. San Diego: Academic Press.

10.

Wilcox, R. R. (2001). Fundamentals of modern statistical methods. New York: Springer.

11.

Wilcox, R. R. (2003). Applying contemporary statistical techniques. San Diego, CA: Academic Press.

12.

Wilcox, R. R. ,& Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8, 254-274.

13.

Wilkinson, L. ,& the APATask Force on Statistical Inference . (1999). Statistical methods in psychology journals. American Psychologist, 54, 594-604.