Small group investigators have been plagued by the problem of observational dependency. This problem exists when data collectedfrom members of the same group are more similar to each other than they are to data collected from another small group receiving identical treatment. Observational dependency can result in inflated Type I error rates. This study demonstrates the effect of different levels of observational dependency on Type I error rates for ANOVA and introduces an alternative statistical procedure to address the problem. Bootstrapping is shown to be superior to ANOVA in minimizing the effect of Type I error rates due to observational dependency.
Get full access to this article
View all access options for this article.
References
1.
Addleman, S. (1970). Variability of treatment and experiment units in the design and analysis of experiments. Journal of the American Statistical Association, 65, 1095-1108.
2.
Anderson, L. , & Ager, J. (1978). Analysis of variance in small group research. Personality and Social Psychology Bulletin, 4(2), 341-345.
3.
Burlingame, G. , Kircher, J., & Taylor, S. (1994). Methodological considerations in group psychotherapy research: Past, present and future practices. In A. Fuhriman & G. Burlingame (Eds.), Handbook of group psychotherapy and counseling. New York: Wiley.
4.
Crits-Christoph, P. , & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative psychotherapies. Journal of Consulting and Clinical Psychology, 59(1), 20-26.
5.
Cronbach, L. (1976). Research on classrooms and schools, Formulation of questions, design, and analysis. Stanford, CA: Stanford University Press.
6.
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7, 1-26.
7.
Efron, B. (1982). The jackknife, the bootstrap, and other resampling plans. Philadelphia: Society for Industrial and Applied Mathematics.
8.
Efron, B. (1988). Bootstrap confidence intervals: Good or bad?Psychological Bulletin, 102, 293-296.
9.
Efron, B. , & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical analysis. Statistical Science, 1, 54-77.
10.
Glass, G. , & Stanley, J. (1970). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice-Hall.
11.
Hopkins, K. (1982). The unit of analysis: Group means versus individual observations. American Educational Research Journal, 19, 5-18.
12.
Kaul, T. , & Bednar, R. (1986). Experiential group research: Results, questions, and suggestions. In S. Garfield & A. Bergin (Eds.), Handbook of psychotherapy and behavior change (3rd ed., pp. 671-714). New York: Wiley.
13.
Kenny, D. , & Judd, C. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422-431.
14.
Kenny, D. , & La Voie, L. (1985). Separating individual and group effects. Journal of Personality and Social Psychology, 48(2), 339-348.
15.
Kirk, R. (1982). Experimental design: Proceduresfor the behavioral sciences. Belmont, CA: Brooks/Cole.
16.
Koomen, W. (1982). A note on the analysis of group data. European Journal of Social Psychology, 12, 297-300.
17.
Lindquist, E. (1940). Statistical analysis in educational research. New York: Houghton-Mifflin.
18.
Lissitz, R. , & Chardos, S. (1975). A study of the effect of the violation of the assumption of independent sampling upon the Type I error rate of the two-group t-test. Educational and Psychological Measurement, 35, 353-359.
19.
Morran, D. , Robison, F., & Hulse-Killacky, D. (1990). Group research and the unit of analysis problem: The use of ANOVA designs with nested factors. Journal for Specialists in Group Work, 15(1), 10-14.
20.
Page, E. (1975). Statistically recapturing the richness within the classroom. Psychology in the Schools, 12, 339-344.
21.
Rasmussen, J. L. (1987). Estimating correlation coefficients: Bootstrap and parametric approaches. Psychological Bulletin, 101, 136-139.
22.
Schiffenbauer, A. , Schulman, R., & Poe, D. (1978). Anested analysis for data collected from groups: Making crowding research more efficient. Environment and Behavior, 10, 127-132.
23.
Simon, J. L. , & Bruce, P. (1991). Resampling: A tool for everyday statistical work. Chance: New Directions for Statistics and Computing, 4(1), 22-31.
24.
Strube, M. (1988). Bootstrap Type I error rates for the correlation coefficient: An examination of alternative procedures. Psychological Bulletin, 104(2), 290-292.
25.
Winer, B. (1971). Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.
