Random Measurement Error Does Not Bias the Treatment Effect Estimate in the Regression-Discontinuity Design

Abstract

This article examines the regression-discontinuity (RD) design when there is random measure ment error and a treatment interaction effect. Two simulation issues -the specification of the pretest-posttest functional form and the choice of the point-of-estimation of the treatment effect- are examined Traditionally, an interaction effect in the general linear model has been con structed after centering the true scores by subtracting their mean. However, because the RD design has traditionally estimated the treatment effect at the cutoff, one is liable to obtain an apparently biased treatment effect that is actually attributable to the misspecification with regard to the point-of-estimation. Formulas are provided that allow one to control exactly in simulations the magnitude of a treatment effect at any point-of-estimation. These formulas can also be used for simulating the randomized experimental (RE) case where estimation is not at the pretest mean.

Get full access to this article

View all access options for this article.

References

Boruch, R.F. 1975. Coupling randomized experiments and approximations to experiments in social program evaluation. Sociological Methods & Research 4:31-53.

————. 1978. Double pretests for checking certain threats to the validity of some conventional evaluation designs, or stalking the null hypothesis. Northwestern University. Typescript.

Campbell, D.T. 1969. Reforms as experiments. American Psychologist 24:409-29.

Cappelleri, J.C. 1991. Cutoff-based designs in comparison and combination with randomized clinical trials. Ph.D. diss., Cornell University , Ithaca, NY.

Cappelleri, J.C. , W. Trochim , T.D. Stanley , and C. Reichardt. 1991. Random measurement error does not bias the treatment effect estimate in the regression-discontinuity design: I. The case of no interaction. Evaluation Review 15:395-419.

Carmines, E.G. , and R.A. Zeller. 1979. Reliability and validity assessment . Beverly Hills, CA: Sage.

Cochran, W.G. 1969. The use of covariance in observational studies. Journal of the Royal Statistical Society Series C 18:270-75.

————. 1970. Some effects of errors in measurement on linear regression . Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability 1:527-39.

Cronbach, L.J. 1987. Statistical tests for moderator variables: Flaws in analysis recently proposed. Psychological Bulletin 102:414-17.

10.

Darlington, R.B. 1990. Regression and linear models. New York : McGraw-Hill.

11.

Goldberger, A.S. 1972. Selection bias in evaluating treatment effects: The case of interaction. University of Wisconsin, Institute for Research on Poverty, Madison. Mimeo.

12.

Jaccard, J. , R. Turrisi , and C.K. Wan. 1990. Interaction effects in multiple regression , Newbury Park, CA: Sage.

13.

Mohr, L.B. 1988. Impact analysis for program evaluation. Chicago: Dorsey.

14.

Peters, C.C. , and W.R. Van Voorhis . 1940. Statistical procedures and their mathematical bases. New York: McGraw-Hill.

15.

Porter, A.C. 1967. The effects of using fallible variables in the analysis of covariance. Ph.D. diss., University of Wisconsin, Madison.

16.

Porter, A.C. , and T.R. Chibucos. 1974. Selecting statistical strategies. In Evaluating educational programs and products, edited by G. D. Borich. Englewood Cliffs, NJ: Educational Technology Publication.

17.

Reichardt, C.S. 1979. The statistical analysis of data from nonequivalent group designs. In Quasi-experimentation: Design and analysis issues for field settings, edited by T. D. Cook and D. T. Campbell. Boston: Houghton-Mifflin.

18.

Rubin, D.B. 1977. Assignment to treatment group on the basis of a covariate . Journal of Educational Statistics 2:1-26.

19.

Stanley, T.D. , and A. Robinson. 1990. "Second best" evaluation design: Regression discontinuity or abbreviated time series? Paper presented at the annual conference of the American Evaluation Association , Washington, DC, October.

20.

Tallmadge, G.K. , and C.T. Wood. 1978. User's guide: ESEA Title I evaluation and reporting system. Mountain View, CA: RMC Research Corporation.

21.

Thorndike, R.L. 1951. Reliability. In Educational measurement , edited by E. F. Lindquist. Washington, DC: Americal Council on Education.

22.

Trochim, W. 1984. Research design for program evaluation: The regression discontinuity approach. Beverly Hills, CA: Sage.

23.

—. 1990. Cutoff assignment strategies for enhancing randomized clinical trials (RCTs. Paper presented at the annual conference of the American Evaluation Association, Washington, DC, October.

24.

Trochim, W. , and J. Davis. 1986a. Computer simulation for program evaluation . Evaluation Review 10:609-34.

25.

—. 1986b. Computer simulation of human service program evaluations . Computers in Human Services 1 (4): 17-38.

26.

Trochim, W. , and C.H. Spiegelman . 1980. The relative assignment variable approach to selection bias in pretest-posttest group designs. Proceedings of the Social Statistics Section, American Statistical Association. Alexandria, VA: American Statistical Association.