Bridging observational studies and randomized experiments by embedding the former in the latter

Abstract

Consider a statistical analysis that draws causal inferences from an observational dataset, inferences that are presented as being valid in the standard frequentist senses; i.e. the analysis produces: (1) consistent point estimates, (2) valid p-values, valid in the sense of rejecting true null hypotheses at the nominal level or less often, and/or (3) confidence intervals, which are presented as having at least their nominal coverage for their estimands. For the hypothetical validity of these statements, the analysis must embed the observational study in a hypothetical randomized experiment that created the observed data, or a subset of that hypothetical randomized data set. This multistage effort with thought-provoking tasks involves: (1) a purely conceptual stage that precisely formulate the causal question in terms of a hypothetical randomized experiment where the exposure is assigned to units; (2) a design stage that approximates a randomized experiment before any outcome data are observed, (3) a statistical analysis stage comparing the outcomes of interest in the exposed and non-exposed units of the hypothetical randomized experiment, and (4) a summary stage providing conclusions about statistical evidence for the sizes of possible causal effects. Stages 2 and 3 may rely on modern computing to implement the effort, whereas Stage 1 demands careful scientific argumentation to make the embedding plausible to scientific readers of the proffered statistical analysis. Otherwise, the resulting analysis is vulnerable to criticism for being simply a presentation of scientifically meaningless arithmetic calculations. The conceptually most demanding tasks are often the most scientifically interesting to the dedicated researcher and readers of the resulting statistical analyses. This perspective is rarely implemented with any rigor, for example, completely eschewing the first stage. We illustrate our approach using an example examining the effect of parental smoking on children’s lung function collected in families living in East Boston in the 1970s.

Keywords

Experimental design observational studies causal inference environmental epidemiology parental smoking lung function Rubin Causal Model (RCM)

Get full access to this article

View all access options for this article.

References

Dockery

Pope

3rd Xu

et al.

An association between air pollution and mortality in six U.S. cities. N Engl J Med 1993; 329: 1753–1759.

Bell

Peng

Dominici

. The exposure-response curve for ozone and risk of mortality and the adequacy of current ozone regulations. Environ Health Perspect 2006; 114: 532–536.

Schwartz

. Air pollution and blood markers of cardiovascular risk. Environ Health Perspect 2001; 109: 405–409.

Corbo

Agabiti

Pistelli

et al.

Parental smoking and lung function: misclassification due to background exposure to passive smoking. Respir Med 2007; 101: 768–773.

Holland

. Statistics and causal inference (with discussion). J Am Stat Assoc 1986; 81: 945–970.

Angrist

Imbens

Rubin

. Identification of causal effects using instrumental variables (with discussion). J Am Stat Assoc 1996; 91: 444–472.

Imbens

Rubin

Rubin causal model. In: Durlauf

Blume

(eds). The New Palgrave Dictionary of Economics 2008; Vol 7. 2ed. New York: Palgrave McMillan, pp. 255–262.

Imbens

Rubin

. Causal inference for statistics, social, and biomedical sciences: an introduction, New York, NY: Cambridge University Press, 2015.

Rubin

. Comment: Neyman (1923) and causal inference in experiments and observational studies. Stat Sci 1990; 5: 472–472.

10.

Rubin

. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol 1974; 66: 688–688.

11.

Pearl

. Causal diagrams for empirical research. Biometrika 1995; 82: 669–688.

12.

Rubin

. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat Med 2007; 26: 20–20.

13.

Rubin

. For objective causal inference, design trumps analysis. Ann Appl Stat 2008; 2: 808–808.

14.

Fisher

. Statistical methods for research workers. 1st ed, Edinburgh: Oliver and Boyd, 1925.

15.

Fisher

. Design of experiments, Edinburgh: Oliver and Boyd, 1935.

16.

Kempthorne

. The design and analysis of experiments, Robert Krieger Publishing Company, 1952.

17.

Cochran

Cox

. Experimental design, Wiley Classics Library, 1957.

18.

Cochran

. The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 1968; 24: 295–314.

19.

Cochran

Rubin

. Controlling bias in observational studies: a review. Sankhya: Ind J Stat 1973; 35: 417–446.

20.

Rubin

. The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics 1973; 29: 184–203.

21.

Rubin

. Matched sampling for causal effects, New York, NY: Cambridge University Press, 2006, pp. 489–489.

22.

Tager

Weiss

Rosner

et al.

Effect of parental cigarette smoking on the pulmonary function of children. Am J Epidemiol 1979; 110: 15–26.

23.

Kahn

. An exhalent problem for teaching statistics. J Stat Educ 2005; 13.

24.

Box

GEP

Hunter

et al.

Statistics for experimenters: an introduction to design, data analysis, and model building, New York: Wiley, 1978, pp. 653–653.

25.

Rosenbaum

Rubin

. Constructing a control group using multivariate matched sampling incorporating the propensity score. Am Stat 1985; 39: 33–38.

26.

Sekhon

. Multivariate and propensity score matching software with automated balance optimization: the matching package for R. J Stat Software 2011; 42.

27.

Hansen

Olsen Klopfer

. Optimal full matching and related designs via network flows. J Computat Graph Stat 2006; 15: 609–627.

28.

Carles

Charles

Heude

et al.

Joint Bayesian weight and height postnatal growth model to study the effects of maternal smoking during pregnancy. Stat Med 2017; 36: 3990–4006.

29.

Rubin

. Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc 1979; 74: 318–328.

30.

Gutman

Rubin

. Robust estimation of causal effects of binary treatments in unconfounded studies with dichotomous outcomes. Stat Med 2013; 32: 1795–1814.

31.

Rubin

. Multivariate matching methods that are equal percent bias reducing, I: Some examples. Biometrics 1976; 32: 109–120.

32.

Andridge

Little

. A review of hot deck imputation for survey non-response. Int Stat Rev 2010; 78: 40–64.

33.

Iacus SM, King G and Porro G. Causal inference without balance checking: coarsened exact matching. Political analysis 2011: mpr013.

34.

Althauser

Rubin

. The computerized construction of a matched sample. Am J Sociol 1970; 76: 325–346.

35.

Morgan

Rubin

. Rerandomization to improve covariate balance in experiments. Ann Stat 2012; 40: 1263–1282.

36.

Ahmed

Husain

Love

et al.

Heart failure, chronic diuretic use, and increase in mortality and hospitalization: an observational study using propensity score methods. Eur Heart J 2006; 27: 1431–1439.

37.

Rubin

. Bayesian inference for causal effects: the role of randomization. Ann Stat 1978; 6: 34–58.

38.

Brillinger

Jones

Tukey

. Report of the Statistical Task Force to the Weather Modification Advisory Board. In: Anonymous The management of weather resources, Volume II 1978, pp. 25, F-5. .

39.

Wang

Wypij

Gold

et al.

A longitudinal study of the effects of parental smoking on pulmonary function in children 6–18 years. Am J Respir Crit Care Med 1994; 149: 1420–1425.

40.

Rubin

. “Author’s reply” to letter by I. Shrier re: The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials (2007). Stat Med 2008; 27: 2741–2742.

41.

Cochran

Moses

Mosteller

. Planning and analysis of observational studies, New York, NY: Wiley, 1983, pp. 145–145.

42.

Rubin

. More powerful randomization-based p-values in double-blind trials with non-compliance. Stat Med 1998; 17: 371–385.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB