The Excluded Variable Problem in Nonrandomized Control Group Designs

Abstract

A solution to the problem of specification error due to excluded variables in statistical models of treatment effects in nonrandomized (nonequivalent) control group designs is presented. The solution relies upon longitudinal observation involving at least two pre tests. The assumptions of a single equation probit model for selection bias, structural invariance, and a first order autoregressive pattern are sufficient to identify the model and treatment effect—with one overidentifying restriction. The estimate of the treatment effect is adjusted for an unmeasured variable that influences both the dependent variable and the selection of cases into treatment and control groups. A small simulation suggests that the use of a ML estimation program such as LISREL, which assumes multivariate normality and linearity, may provide reasonable estimates of treatment effects.

Get full access to this article

View all access options for this article.

References

Barnow, B.S. (1973) "The effects of head start and socioeconomic status on cognitive development of disadvantaged children." Ph.D. dissertation, University of Wisconsin.

Bentler, P.M. and J.A. Woodward (1978) "A head start reevaluation: positive effects are still not yet demonstrable." Evaluation Q. 2: 493-510.

Campbell, D.T. and R.F. Boruch (1975) "Making the case for randomized assignment to treatments by considering alternatives six ways in which quasi-experimental evaluations tend to underestimate effects," in C. A. Bennett and C. A. Lumsdane (eds.) Evaluation and Experiment: Some Critical Issues in Assessing Social Programs . New York: Academic.

Cook, T. D. and D. T. Campbell [eds.] (1979) Quasi-Expenmentation: Design & Analysis Issues for Field Settings. Chicago: Rand McNally.

Dwyer, J.H. (1983) Statistical Models for the Social and Behavioral Sciences. New York: Oxford Univ. Press.

Heckman, J. (1979) "Sample bias as a specification error." Econometrica (January): 153-162.

Joreskog, K.G. and D. Sorbom (1981) LISREL V: analysis of linear structural relationships by maximum likelihood and least squares methods Research Report 81-8, University of Uppsala, Sweden.

Kenny, D.A. (1975) "A quasi-experimental approach to assessing treatment effects in the nonequivalent control group design." Psych. Bull. 82: 345-362

Magidson, J. (1977) "Towards a causal model approach for adjusting for pre-existing differences in the nonequivalent control group situation: a general alternative to ANCOVA." Evaluation Q. 1: 399-420.

10.

Muthen, B. and K.G. Joreskog (1983) "Selectivity problems in quasi-experimental studies ." Evaluation Rev. 7: 139-174.

11.

Reichardt, C.S. (1979) "The statistical analysis of data from nonequivalent group designs," in T. D. Cook and D. T. Campbell (eds.) Quasi-Experimentation: Design & Analysis for Field Settings. Chicago: Rand McNally.

12.

Sorbom, D. (1981) "Structural equation models with structured means," in K. G. Joreskog and H. Wold (eds.) Systems Under Indirect Observation: Causality, Structure, Prediction. Amsterdam: North-Holland.