Differences in causal estimates from longitudinal analyses of residualized versus simple gain scores: Contrasting controls for selection and regression artifacts
Restricted accessOtherFirst published online March, 2010
Differences in causal estimates from longitudinal analyses of residualized versus simple gain scores: Contrasting controls for selection and regression artifacts
This study estimates the causal effects of six corrective actions for children's problem behaviors, comparing four types of longitudinal analyses that correct for pre-existing differences in a cohort of 1,464 4- and 5-year-olds from Canadian National Longitudinal Survey of Children and Youth (NLSCY) data. Analyses of residualized gain scores found apparently detrimental effects of all corrective actions by parents and professionals on subsequent antisocial behavior and hyperactivity. In contrast, analyses of simple gain scores found only apparently beneficial effects. Temporally reversed analyses yielded the same pattern of results, consistent with selection biases and regression artifacts, not with unidirectional causal effects. The findings were similar for corrective actions by professionals (e.g., Ritalin, psychotherapy) and by parents (physical and nonphysical punishment, scolding/yelling, “hostile-ineffective” parenting). Longitudinal analyses should check for similar artifacts by implementing temporally-reversed analyses and by determining whether causally relevant coefficients would replicate without artifacts biased in their favor.
Boyle, M.H., Offord, D.R., Racine,Y.A., Szatmari, P., & Sanford, M. ( 1993). Evaluation of the revised Ontario Child Health Study scales. Journal of Child Psychology & Psychiatry & Allied Disciplines, 34, 189-213.
2.
Campbell, D.T., & Boruch, R.F. ( 1975). Making the case for randomized assignment to treatments by considering the alternatives: Six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects. In C.A. Bennett & A.A. Lumsdaine (Eds.), Evaluation and experiment: Some critical issues in assessing social programs (pp. 195-296). New York: Academic Press.
3.
Campbell, D.T., & Kenny, D.A. ( 1999). A primer on regression artifacts. New York: Guilford Press.
4.
Cohen, P. ( 1991). A source of bias in longitudinal investigations of change . In L.M. Collins & J.L. Horn (Eds.), Best methods for the analysis of change (pp. 18-33). Washington, DC: American Psychological Association.
5.
Eyberg, S.M., Nelson, M.M., & Boggs, S.R. ( 2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology, 37, 215-237.
6.
Freedman, D.A. ( 2006). Statistical models for causation: What inferential leverage do they provide? Evaluation Review, 30, 691-713.
7.
Gershoff, E.T. ( 2002). Corporal punishment by parents and associated child behaviors and experiences: A meta-analytic and theoretical review. Psychological Bulletin, 128, 539-579.
8.
Graham, J.W., Cumsille, P.E., & Elek-Fisk, E. ( 2003). Methods for handling missing data. In J.A. Schinka & W.F. Velicer (Eds.), Handbook of psychology: Research methods in psychology (Vol. 2, pp. 87-114). Hoboken, NJ : Wiley.
9.
Gunnoe, M.L., & Mariner, C.L. ( 1997). Toward a developmental-contextual model of the effects of parental spanking on children’s aggression. Archives of Pediatrics and Adolescent Medicine, 151, 768-775.
10.
Haviland, A.M., Nagin, D.S., & Rosenbaum, P.R. (2007). Combining propensity score matching and group-based trajectory analysis in an observational study. Psychological Methods, 12, 247-267.
11.
Heckman, J., Ichimura, H., Smith, J., & Todd, P. ( 1998). Characterizing selection bias using experimental data . Econometrica, 66, 1017-1098.
12.
Heckman, J.J. ( 1979). Sample selection bias as a specification error. Econometrica, 47, 153-161.
13.
Hill, N.E., & Tyson, D.F. ( 2009). Parental involvement in middle school: A meta-analytic assessment of the strategies that promote achievement. Developmental Psychology, 45, 740-763.
14.
Jensen, P.S., Hinshaw, S.P., Swanson, J.M., Greenhill, L.L., Conners, C.K., Arnold, L.E., et al. (2001). Findings from the NIMH multimodal treatment study of ADHD (MPA): Implications and applications for primary care providers . Journal of Developmental and Behavioral Pediatrics, 22(1), 60-73.
15.
Jöreskog, K.G., & Sörbom, D. (1979). Advances in factor analysis and structural equation models. Cambridge, MA: Abt Books.
16.
Keppel, G., & Wickens, T.D. ( 2004). Design and analysis (4th ed.). Upper Saddle River, NJ: Pearson Education International.
17.
Lambert, E.W., & Bickman, L. ( 2001, March). Risk adjusted mental health outcomes: Ritual or solution. Paper presented at the 14th Annual Research Conference, A System of Care for Children’s Mental Health: Expanding the Research Base, University of South Florida, Tampa.
18.
Larzelere, R.E., Kuhn, B.R., & Johnson, B. ( 2004). The intervention selection bias: An underrecognized confound in intervention research. Psychological Bulletin, 130, 289-303.
19.
Lord, F.M. ( 1967). A paradox in the interpretation of group comparisons . Psychological Bulletin, 68, 304-305.
20.
Pomerantz, E.M., & Eaton, M.M. ( 2001). Maternal intrusive support in the academic context: Transactional socialization processes. Developmental Psychology, 37, 174-186.
Rothman, K.J., & Greenland, S. ( 1998). Modern epidemiology (2nd ed.). Philadelphia, PA: Lippincott-Raven .
23.
Rubin, D.B. ( 1978). Bayesian inference for causal effects: The role of randomization . Annals of Statistics, 6, 34-58.
24.
Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
25.
Special Surveys Division. (2002). National longitudinal survey of children and youth. Ottawa, ON: Statistics Canada.
26.
Statistics Canada. (2001/2002 ). NLSCY Data Users Guide. Ottawa, ON: Author.
27.
Statistics Canada. (2008). National longitudinal survey of children: Overview of survey instruments. Retrieved January 6, 2009, from http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?lang=eng&catno=89F0078X
28.
Tremblay, R.E., Pihl, R.O., Vitaro, F., & Dobkin, P.L. ( 1994). Predicting early onset male antisocial behavior from preschool behavior. Archives of General Psychiatry, 51, 732-739.
29.
Westinghouse Learning Corporation & Ohio University. ( 1969). The impact of Head Start: An evaluation of the effects of Head Start on children’s cognitive and affective development. Report presented to the Office of Economic Opportunity Pursuant to Contract of B89-4536, Vols. 1and2. Athens, OH: Authors.
