Problems with inferring causal relationships from nonexperimental data are
briefly reviewed, and four broad classes of methods designed to allow estimation
of and inference about causal parameters are described: panel regression,
matching or reweighting, instrumental variables, and regression discontinuity.
Practical examples are offered, and discussion focuses on checking required
assumptions to the extent possible.
AbadieA., DrukkerD., Leber HerrJ., and ImbensG. W.2004. Implementing matching estimators for average
treatment effects in Stata. Stata Journal4: 290–311.
2.
AbadieA., and ImbensG. W.2006. On the failure of the bootstrap for matching
estimators. NBER Technical Working Paper No. 325. http://www.nber.org/papers/t0325/.
3.
AndersonT., and RubinH.1949. Estimation of the parameters of a single
equation in a complete system of stochastic equations.
Annals of Mathematical Statistics20: 46–63.
4.
AngristJ. D., ImbensG. W., and RubinD. B.1996. Identification of causal effects using
instrumental variables. Journal of the American
Statistical Association91: 444–472.
5.
AngristJ. D., and
KruegerA. B.1991. Does compulsory school attendance affect
schooling and earnings?Quarterly Journal of Economics106: 979–1014.
6.
AutorD. H., KatzL. F., and KerneyM. S.2005. Rising wage inequality: The role of composition and
prices. NBER Technical Working Paper No. 11628. http://www.nber.org/papers/w11628/.
7.
AzevedoJ. P.2005. dfl: Stata module to estimate DiNardo, Fortin, and
Lemieux coun-terfactual kernel density. Statistical Software
Components S449001, Boston College Department of
Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s449001.html.
8.
BakerM., BenjaminD., and StangerS.1999. The highs and lows of the minimum wage effect:
A time-series cross-section study of the Canadian law.
Journal of Labor Economics17: 318–350.
BaumC. F., SchafferM., and StillmanS.2007. Enhanced routines for IV/GMM estimation and
testing. Stata Journal7: 465–506.
11.
BaumC. F., SchafferM., StillmanS., and WigginsV.2006. overid: Stata module to calculate tests of
overidentifying restrictions after ivreg, ivreg2, ivprobit, ivtobit, and
reg3. Statistical Software Components S396802,
Boston College Department of Economics.
Downloadable from http://ideas.repec.org/c/boc/bocode/s396802.html.
12.
BeckerS., and CaliendoM.2007. Sensitivity analysis for average treatment
effects. Stata Journal7: 71–83.
13.
BeckerS. O., and IchinoA.2002. Estimation of average treatment effects based
on propensity scores. Stata Journal2: 358–377.
14.
BlackS.1999. Do better schools matter? Parental valuation
of elmentary education. Quarterly Journal of
Economics114: 577–599.
15.
BlinderA. S.1973. Wage discimination: Reduced form and
structural estimates. Journal of Human
Resources8: 436–455.
16.
BoundJ., JaegerD., and BakerR.1995. Problems with instrumental variable estimation
when the correlation between the instruments and the endogenous explanatory
variables is weak. Journal of the American
Statistical Association90: 443–450.
17.
CardD. E.1995a. Using geographic variation in college
proximity to estimate the return to schooling. In
Aspects of Labour Economics: Essays in Honour of John
Vanderkamp, ed. ChristofidesL., GrantE. K., and
SwindinskyR.Toronto, Canada: University of
Toronto Press.
18.
CardD. E.1995b. Earnings, schooling, and ability
revisited. Research in Labor Economics14: 23–48.
19.
CardD. E.1999. The causal effect of education on
earnings. Handbook of Labor Economics3: 1761–1800.
20.
CardD. E.2001. Estimating the return to schooling: Progress
on some persistent econometric problems.
Econometrica69: 1127–1160.
21.
ChengM., JianqingF., and MarronJ. S.1997. On automatic boundary
corrections. Annals of Statistics25: 1691–1708.
22.
CochranW., and RubinD. B.1973. Controlling bias in observational
studies. Sankhyā35: 417–446.
DiNardoJ., FortinN. M., and
LemieuxT.1996. Labor market institutions and the distribution
of wages, 1973–1992: A semiparametric approach.
Econometrica64: 1001–1044.
25.
DiNardoJ., and LeeD.2002. The impact of unionization on establishment closure:
A regression discontinuity analysis of representation elections.
NBER Technical Working Paper No. 8993. http://www.nber.org/papers/w8993/.
26.
DiPreteT., and GanglM.2004. Assessing bias in the estimation of causal
effects: Rosenbaum bounds on matching estimators and instrumental variables
estimation with imperfect instruments. Sociological
Methodology34: 271–310.
27.
FanJ., and GibelsI.1996. Local Polynomial Modelling and Its
Applications.New York: Chapman &
Hall.
28.
FisherR. A.1918. The causes of human
variability. Eugenics Review10: 213–220.
29.
FisherR. A.1925. Statistical Methods for Research Workers.Edinburgh: Oliver &
Boyd.
30.
GlazermanS., LevyD. M., and MyersD.2003. Nonexperimental versus experimental estimates
of earnings impacts. Annals of the American Academy
of Political and Social Science589: 63–93.
31.
GoldbergerA. S., and DuncanO. D.1973. Structural Equation Models in the Social
Sciences.New York: Seminar
Press.
32.
GomulkaJ., and SternN.1990. The employment of married women in the United
Kingdom, 1970–1983. Econometrica57: 171–199.
33.
GrilichesZ., and HausmanJ. A.1986. Errors in variables in panel
data. Journal of Econometrics31: 93–118.
34.
GutierrezR. G., LinhartJ. M., and
PitbladoJ. S.2003. From the help desk: Local polynomial
regression and Stata plugins. Stata Journal3: 412–419.
35.
HahnJ., ToddP., and van der
KlaauwW.2001. Identification and estimation of treatment
effects with a regression-discontinuity design.
Econometrica69: 201–209.
36.
HallA. R., RudebuschG. D., and WilcoxD. W.1996. Judging instrument relevance in instrumental
variables estimation. International Economic
Review37: 283–298.
37.
HardinJ. W.,
SchmiedicheH., and CarrollR. J.2003. Instrumental variables, bootstrapping, and
generalized linear models. Stata Journal3: 351–360.
38.
HeckmanJ., IchimuraH., and ToddP.1997. Matching as an econometric evaluation
estimator: Evidence from evaluating a job training program.
Review of Economic Studies64: 605–654.
39.
HeckmanJ. J., and
VytlacilE.2004. Structural equations, treatment effects, and
econometric policy evaluation. Econometrica73: 669–738.
40.
HollandP. W.1986. Statistics and causal
inference. Journal of the American Statistical
Association8: 945–960.
41.
ImaiK., and van DykD. A.2004. Causal inference with general treatment
regimes: Generalizing the propensity score. Journal
of the American Statistical Association99: 854–866.
42.
ImbensG.2004. Nonparametric estimation of average treatment
effects under exogene-ity: A review. Review of
Economics and Statistics86: 4–29.
43.
ImbensG. W., and
LemieuxT.2007. Regression discontinuity designs: A guide to
Practice. NBER Technical Working Paper No. 13039. http://www.nber.org/papers/w13039/.
44.
JannB.2005a. jmpierce: Stata module to perform Juhn–Murphy-Pierce
decomposition. Statistical Software Components S448803, Boston
College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s448803.html.
45.
JannB.2005b. oaxaca: Stata module to compute decompositions of
outcome differentials. Statistical Software Components S450604,
Boston College Department of Economics.
Downloadable from http://ideas.repec.org/c/boc/bocode/s450604.html.
46.
JuhnC., MurphyK. M., and PierceB.1991. Accounting for the slowdown in black-white
wage convergence. In Workers and Their Wages:
Changing Patterns in the United States, ed.
KostersM., 107–143.
Washington, DC: American
Enterprise Institute.
47.
JuhnC.1993. Wage inequality and the rise in returns to
skill. Journal of Political Economy101: 410–442.
48.
KleibergenF., and SchafferM.2007. Ranktest: Stata module to test the rank of a matrix
using the Kleibergen–Paap rk statistic.Boston College Department of Economics,
Statistical Software Components S456865. Downloadable from http://ideas.repec.org/c/boc/bocode/s456865.html.
49.
LeeD. S.2001. The electoral advantage to incumbency and voters’
valuation of politicians’ experience: A regression discontinuity analysis of
elections to the U.S. House. NBER Technical Working Paper No. 8441.
http://www.nber.org/papers/w8441/.
50.
LeeD. S.2005. Training, wages, and sample selection: Estimating
sharp bounds on treatment effects. NBER Technical Working Paper No.
11721. http://www.nber.org/papers/w11721/.
51.
LeeD. S., and CardD.2006. Regression discontinuity inference with specification
error. NBER Technical Working Paper No. 322. http://www.nber.org/papers/t0322/.
52.
LeibbrandtM., LevinsohnJ., and McCraryJ.2005. Incomes in South Africa since the fall of
apartheid. NBER Technical Working Paper No. 11384. http://www.nber.org/papers/w11384/.
53.
LeuvenE., and SianesiB.2003. psmatch2: Stata module to perform full Mahalanobis
and propensity score matching, common support graphing, and covariate
imbalance testing.Boston College Department of Economics,
Statistical Software Components. Downloadable from http://ideas.repec.org/c/boc/bocode/s432001.html.
54.
LudwigJ., and MillerD. L.2005. Does head start improve children's life chances?
Evidence from a regression discontinuity design. NBER Technical
Working Paper No. 11702. http://www.nber.org/papers/w11702/.
55.
MachadoJ., and MataJ.2005. Counterfactual decompositions of changes in
wage distributions using quantile regression.
Journal of Applied Econometrics20: 445–465.
56.
ManskiC.1995. Identification Problems in the Social
Sciences.Cambridge, MA: Harvard University
Press.
57.
McCraryJ.2007. Manipulation of the running variable in the
regression discontinuity design: A density test. NBER Technical
Working Paper No. 334. http://www.nber.org/papers/t0334/.
58.
MikushevaA., and PoiB. P.2006. Tests and confidence sets with correct size
when instruments are potentially weak. Stata
Journal6: 335–347.
59.
MorganS. L., and
HardingD. J.2006. Matching estimators of causal effects:
Prospects and pitfalls in theory and practice.
Sociological Methods and Research35: 3–60.
60.
NanniciniT.2006. sensatt: A simulation-based sensitivity analysis for
matching estimators.Boston College Department of Economics,
Statistical Software Components. Downloadable from http://ideas.repec.org/c/boc/bocode/s456747.html.
61.
NelsonC., and StartzR.1990. Some further results on the exact small sample
properties of the instrumental variable estimator.
Econometrica58: 967–976.
62.
NeymanJ.1923. Roczniki Nauk Roiniczych (Annals of
Agricultural Sciences) Tom X: 1–51 [In Polish].
Translated as “On the application of probability theory to agricultural
experiments. Essay on principles. Section 9,” by D. M. Dabrowska and T. P. Speed
(Statistical Science 5: 465–472, 1990).
NicholsA., and RaderK.2007. Spending in the districts of marginal incumbent
victors in the House of Representatives. Unpublished working
paper.
65.
NicholsA., and SchafferM. E.2007. Cluster–robust and GLS corrections.
Unpublished working paper.
66.
OaxacaR.1973. Male–female wage differentials in urban labor
markets. International Economic Review14: 693–709.
67.
PoiB. P.2006. Jackknife instrumental variables estimation in
Stata. Stata Journal6: 364–376.
68.
RosenbaumP. R.2002. Observational Studies. 2nd ed.
New York:
Springer.
69.
RosenbaumP. R., and RubinD. B.1983. The central role of the propensity score in
observational studies for causal effects.
Biometrika70: 41–55.
70.
RothsteinJ.2007. Do value-added models add value? Tracking fixed
effects and causal inference. Unpublished working
paper.
71.
RubinD. B.1974. Estimating causal effects of treatments in
randomised and non-randomised studies. Journal of
Educational Psychology66: 688–701.
72.
RubinD. B.1986. Statistics and causal inference: Comment:
Which ifs have causal answers. Journal of the
American Statistical Association81: 961–962.
73.
RubinD. B.1990. Comment: Neyman (1923) and causal inference in
experiments and observational studies. Statistical
Science5: 472–480.
74.
SchafferM., and StillmanS.2006. xtoverid: Stata module to calculate tests of
overidentifying restrictions after xtreg, xtivreg, xtivreg2, and
xthtaylor. Statistical Software Components S456779, Boston College
Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s456779.html.
75.
SchafferM.2007. xtivreg2: Stata module to perform extended IV/2SLS,
GMM and AC/HAC, LIML, and k-class regression for panel-data
models. Statistical Software Components S456501, Boston College Department
of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s456501.html.
76.
ShadishW. R., CookT. D., and
CampbellD. T.2002. Experimental and Quasi-Experimental Designs for
Generalized Causal Inference.Boston: Houghton
Mifflin.
77.
SimpsonE. H.1951. The interpretation of interaction in
contingency tables. Journal of the Royal Statistical
Society, Series B13: 238–241.
78.
SpenceM.1973. Job market signaling.
Quarterly Journal of Economics87: 355–374.
79.
StockJ. H., and YogoM.2005. Testing for weak instruments in linear IV
regression. In Identification and Inference for
Econometric Models: Essays in Honor of Thomas Rothenberg, ed.
AndrewsD. W. K., and
StockJ. H.,
80–108.
Cambridge: Cambridge
University Press.
80.
StoneM.1974. Cross-validation and multinomial
prediction. Biometrika61: 509–515.
81.
StoneM.1977. Asymptotics for and against
cross-validation. Biometrika64: 29–35.
82.
StuartE. A., and RubinD. B.2007. Best practices in quasiexperimental designs:
Matching methods for causal inference. In Best
Practices in Quantitative Social Science, ed.
OsborneJ.Thousand Oaks, CA:
Sage.
83.
van der KlaauwW.2002. Estimating the effect of financial aid offers
on college enrollment: A regression discontinuity approach.
International Economic Review43: 1249–1287.
84.
WooldridgeJ. M.2002. Econometric Analysis of Cross Section and Panel
Data.Cambridge, MA: MIT
Press.
85.
YuleG. U.1903. Notes on the theory of association of
attributes in statistics. Biometrika2: 275–280.
86.
YunM.-S.2004. Decomposing differences in the first
moment. Economics Letters82: 275–280.
87.
YunM.-S.2005a. Normalized equation and decomposition analysis:
Computation and inference. IZA Discussion Paper No. 1822. ftp://ftp.iza.org/dps/dp1822.pdf.
88.
YunM.-S.2005b. A simple solution to the identification
problem in detailed wage decompositions. Economic
Inquiry43: 766–772.