Getting the Within Estimator of Cross-Level Interactions in Multilevel Models with Pooled Cross-Sections: Why Country Dummies (Sometimes) Do Not Do the Job

Abstract

Multilevel models with persons nested in countries are increasingly popular in cross-country research. Recently, social scientists have started to analyze data with a three-level structure: persons at level 1, nested in year-specific country samples at level 2, nested in countries at level 3. By using a country fixed-effects estimator, or an alternative equivalent specification in a random-effects framework, this structure is increasingly used to estimate within-country effects in order to control for unobserved heterogeneity. For the main effects of country-level characteristics, such estimators have been shown to have desirable statistical properties. However, estimators of cross-level interactions in these models are not exhibiting these attractive properties: as algebraic transformations show, they are not independent of between-country variation and thus carry country-specific heterogeneity. Monte Carlo experiments consistently reveal the standard approaches to within estimation to provide biased estimates of cross-level interactions in the presence of an unobserved correlated moderator at the country level. To obtain an unbiased within-country estimator of a cross-level interaction, effect heterogeneity must be systematically controlled. By replicating a published analysis, we demonstrate the relevance of this extended country fixed-effects estimator in research practice. The intent of this article is to provide advice for multilevel practitioners, who will be increasingly confronted with the availability of pooled cross-sectional survey data.

Keywords

cross-level interaction pooled cross-section country fixed effects fixed effects multilevel analysis within effect

1. Introduction

Comparing individuals across national contexts allows social scientists to get a glimpse of how social, economic, cultural, and institutional contexts influence individual outcomes. Such issues are at the heart of the social sciences. Traditionally, such comparisons were based on few countries, and inference about contextual effects was based on theoretical argumentation and corresponding case selection (Przeworski and Teune 1970:31–32). Since the early 2000s and because of the growing availability of large international surveys, multilevel modeling has become an increasingly prominent statistical technique in comparative social science. Instead of treating countries as cases, it allows researchers to statistically model contextual effects on individual outcomes on the basis of a sample¹ of countries (Bryan and Jenkins 2016:3). In the Journal of European Social Policy (JESP), we find 46 analyses of multilevel data, published through March 2016, of which about 90 percent use countries as the relevant contextual units.² Even in the American Sociological Review (ASR), a journal in which most analyses are traditionally based on data from the United States, we find 58 analyses of multilevel data, published until April 2016, of which about one quarter use countries as contextual units.³

However, it did not take long before some researchers warned against the use of multilevel models with cross-national survey data (Bryan and Jenkins 2016; Stegmueller 2013). Although in principle multilevel models allow controlling for any observed variable, be it an individual-level or a contextual-level variable, in research designs with countries as higher level units, the number of these is typically rather small. Therefore, the ability to control for any systematic differences between countries is practically limited (small-N problem). In addition, scholars argued that there might be numerous unobserved country-level characteristics that result in biased estimates if they are ignored in the statistical model (Jaeger 2013:156).

Social scientists propose solving this dilemma by estimating context effects on the basis of within-country variation over time (see Fairbrother 2014 for a detailed discussion; see Jaeger 2013, Wulfgramm 2014, and te Grotenhuis et al. 2015 for research examples). The idea to identify an effect solely by within-unit variation stems from the analysis of longitudinal data, and it is regularly applied to individual panel data, where this technique is known as the fixed-effects (FE) estimator (Allison 2009). Similarly, we can obtain within-country estimators of contextual effects by pooling international cross-sectional surveys across time.

Generally, within-cluster estimation has become a key analytical strategy for causal inference in the social sciences (for example, see the overview on causal methods by Gangl 2010). Reference measures within clusters provide better counterfactuals than measures between clusters and therefore are more appropriate for assessing causal effects. This intuitive insight is formally rooted in the homogeneity within clusters, where observations usually have several characteristics in common. Such cluster-specific characteristics are automatically held constant if observations within clusters are compared. Therefore, estimators based solely on intracluster covariation automatically control for all unobservable and observable characteristics at the cluster level (and any additional higher clusters).

In the context of panel data, for example, single observations (level 1) are nested within individuals (level 2), which allows the estimation of within-unit effects (e.g., of marital status) that cannot be biased by systematic differences between individuals (level 2 clusters). In a multilevel model with pupils at level 1, nested in classes at level 2, nested in schools at level 3, pupil-level effects (e.g., of ethnic background) may be estimated from within-class variation only, and the resulting estimate will be independent of differences between classes (level 2 clusters) and schools (level 3 clusters). In a multilevel model with families as clusters (level 2), within-family differences between siblings (level 1) can be used to control individual-level determinants (e.g., playing tennis) for family-level effects, and so forth. Although the concept of within effects is very general, in this article we focus on one specific type: in three-level data with persons (level 1) nested in country-occasions (level 2), nested in countries (level 3), within-country differences can be used to identify the effects of time-varying institutional, cultural, or structural macro variables net of unobserved heterogeneity between countries.

A simple way to obtain this within-country estimator is to add a set of country dummies to the empirical model (compare te Grotenhuis et al. 2015). This approach, often called country FE (cFE), is equivalent to group-mean-centering all variables at the level of countries. A more complex approach was described by Fairbrother (2014), who demonstrated how country-level effects can be decomposed into their within and between components, where the within effects replicate the cFE estimates.

Although the methodological discourse has clearly outlined the merits of within-cluster estimation in comparisons across contexts, our review of published multilevel analyses reveals that contemporary research into context-level effects typically uses simple cross-sectional multilevel models. In our review of the JESP, about 74 percent of published multilevel analyses are based on cross-sectional data without repeated measurements on the context level. In the ASR, we find that about 40 percent of the multilevel analyses are based on cross-sectional data. Obviously, a reason for not using repeated cross-sectional data could be the lack of availability. However, although many researchers use survey data from multiple years (26 percent in JESP, 60 percent in ASR), the longitudinal variation is typically not exploited for causal inference: in our analysis of JESP publications, only 20 percent of the analyses based on data from multiple time points systematically use the within-context variation over time.⁴ In the ASR, we find the figure to be about 10 percent⁵ (see also Schmidt-Catran and Fairbrother 2016:34). Overall, we find that only 5 of 95 studies (about 5 percent) use a within-country estimator. (Compare Tables A1 and A2 in the Appendix.)

Nevertheless, as we argue above, social scientists are beginning to understand the merits of estimating within-country effects, and we believe that the increasing availability of repeated large-scale international surveys will further increase the number of publications using a within-country estimator.⁶ Given this foreseeable development, which we strongly support, we feel the urge to demonstrate that the standard cFE approach does not yield a genuine within estimator of cross-level interaction effects. Before we go into the details of this issue, we briefly discuss the substantive importance of cross-level interactions.

Although the sheer number of multilevel analyses published since the early 2000s clearly indicates the enormous interest in contextual effects, much of this research is in fact interested in the interactions between individual- and contextual-level characteristics. In our review of multilevel analyses published in the JESP and the ASR, we find that almost half of the studies (43 percent and 40 percent, respectively) are substantially interested in cross-level interactions (see again Tables A1 and A2 in the Appendix). Consider, for example, the evaluation of policies (such as antidiscrimination laws), in which researchers are typically much more interested in the question of how such policies affect differences between groups of individuals (such as men and women) than in the total effect of a policy on an outcome variable (such as wages). That is, rather than being interested in the total effect of an antidiscrimination policy on wages, we want to know how this policy affects wage disparities between men and women. In other words, social scientists are typically interested in differences between social groups, and when it comes to contextual variables, they are often interested in their impact on these group differences.

In a statistical model, this substantial interest is reflected in a cross-level interaction effect (e.g., between antidiscrimination law and gender): $wag e_{ji} = α_{j} + β_{1} gnd r_{ji} + β_{2} polic y_{j} + β_{3} (gnd r_{ji} * polic y_{j})$ , where j indicates countries and i individuals. Depending on the perspective of the researcher, this specification either allows treating the policy as a moderator of the gender effect (conditional effect of gender: $\frac{\partial wage}{\partial gndr} = β_{1} + β_{3} polic y_{j}$ ) or treating gender as a moderator of the policy effect (conditional effect of policy: $\frac{\partial wage}{\partial policy} = β_{2} + β_{3} gnd r_{ji}$ ). Of course, both perspectives tell the same story with regard to policy-related gender differences as they are both implicit to the same interaction effect $β_{3}$ .

In the sections that follow, we formally prove that such cross-level interactions in cFE models (and in context FE models in general) are not genuine within estimators. Section 2 embeds our considerations in a general discussion of the cFE design, first retrieving the (well-known) properties of cFE models for main effects of context variables and then moving on to cFE models with cross-level interactions in Section 3. We show that the standard way to specify the cFE model is likely to yield biased estimates of cross-level interactions and present specifications that produce genuine within estimators and provide unbiased estimates of cross-level interactions in the presence of unobserved effect heterogeneity between countries. Section 4 presents the results of a series of Monte Carlo simulations that support our formal econometric considerations. Section 5 replicates the analysis of an article published in JESP to demonstrate the relevance of our claim for research practice. In conclusion, Section 6 discusses the practical implications of our considerations and relates our findings to other discourses on effect heterogeneity in cross-country analyses.

2. The Country FE Model for Pooled Cross-Sections

Consider a cross-sectional survey of individuals (i = 1, . . . , n) that is collected in multiple countries (j = 1, . . . , N) and repeated at regular intervals (t = 1, . . . , T). Examples of such data sets include the European Social Survey, the World Values Study, the European Values Study, and the International Social Survey Programme. National surveys such as the General Social Survey are obviously equally relevant if the contextual units are, for example, states. When data from multiple time points are pooled together, the data can be viewed as having a three-level structure with individuals nested in country-waves (samples) and these country-waves nested in countries (Schmidt-Catran and Fairbrother 2016). A corresponding statistical model can be written as

y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{2 j} + v_{0 j} + u_{0 jt} + e_{jti},

(1)

where the individual outcome at level 1, $y_{jti}$ (such as wage), is dependent on an individual-level characteristic, $x_{jti}$ (such as gender), a time-varying country-level variable (level 2), $z_{1 jt}$ (such as antidiscrimination law), and also a time-constant country-level variable (level 3), $z_{2 j}$ (such as legal tradition).⁷ $v_{0 j}$ is an unobserved effect at level 3, and $u_{0 jt}$ is an unobserved effect at level 2. Finally, $e_{jti}$ is random variation at level 1. (Note that equation 1 presents a statistical model of the data-generating process [DGP], not a random effects [RE] model.)

In research concerned with the direct effects of context-level variables, the coefficients $β_{2}$ and $β_{3}$ are of major interest. A serious problem in such research is the existence of unobserved systematic differences between countries that are related to the observed variables in the model. That is, $v_{0 j}$ may contain correlated unobserved heterogeneity, which one needs to control for in order to obtain unbiased effects. As stated above, a recently suggested solution to this issue is the introduction of cFE. That is, the unobserved effects in $v_{0 j}$ are estimated as FE:

y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + \sum_{j = 1}^{N - 1} γ_{j} c_{j} + u_{0 jt} + e_{jti},

(2)

where $\sum_{j = 1}^{N - 1} γ_{j} c_{j}$ is a set of dummies ( $c_{j} = c_{1}, \dots, c_{N - 1}$ ) with corresponding coefficients $γ_{j}$ . This set of dummies partials out any differences between the j clusters, thereby replacing $v_{0 j}$ . At the same time, it makes estimating the effect of time-constant country-level variables impossible (therefore $z_{2 j}$ is omitted from equation 2).⁸ The effect of a time-varying cluster-level variable ( $z_{1 jt}$ ), in contrast, can still be estimated and is now controlled for any differences between clusters. In statistical terms, $β_{2}$ is now identified solely from variation in $z_{1 jt}$ within clusters over time. In the studies reviewed for this article, 94 percent of the key independent (context-level) variables are inherently time varying and thus may potentially allow such estimation of within effects.

Any textbook on panel data analysis demonstrates that the dummy variable approach, which is typically called the least squares dummy variables (LSDV) estimator, is equivalent to a model that uses demeaned variables (e.g., Wooldridge 2015:484):

(y_{j t i} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{j t i} - {\bar{x}}_{j}) + β_{2} (z_{1 j t} - {\bar{z_{1}}}_{j}) + β_{3} (z_{2 j} - {\bar{z_{2}}}_{j}) + (v_{0 j} - {\bar{v_{0}}}_{j}) + (u_{0 j t} - {\bar{u_{0}}}_{j}) + (e_{j t i} - {\bar{e}}_{j}) .

(3)

This formulation nicely demonstrates how the cFE model controls for unobserved heterogeneity and why this implies that $β_{3}$ can no longer be estimated: for all measurements jti, it is $z_{2 j} - {\bar{z_{2}}}_{j} = 0$ , and $v_{0 j} - {\bar{v_{0}}}_{j} = 0$ . Thus equation (3) reduces to

(y_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + (u_{0 jt} - {\bar{u_{0}}}_{j}) + (e_{jti} - {\bar{e}}_{j}) .

(4)

Alternatively, the within estimator can also be specified in an RE framework, a strategy that is rooted in a formulation by Mundlak (1978). It is often introduced in the general literature on multilevel modeling (Snijders and Bosker 1999; see also Hoffman 2015 in the context of longitudinal data structures) and discussed by Fairbrother (2014) in the context of cross-country analysis. The core idea of this formulation is to disentangle within and between effects by integrating cluster-specific means as additional independent variables into the model. That is, instead of eliminating between-context variation from the data via demeaning, as in equation (3), the model controls for the between-context effects via their inclusion into the fixed part of the model:

y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{2 j} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + v_{0 j} + u_{0 jt} + e_{jti} .

(5)

Note that equation (5) now describes an RE model applied to the statistical model of the DGP in equation (1). In this specification, between-context covariation of x and $z_{1}$ with y is completely absorbed by the coefficients of cluster-specific mean variables. Therefore, $β_{1}$ and $β_{2}$ in equation (5) replicate the coefficients from the FE models in equations (2) and (4). In contrast to the FE models, the RE model has the advantage of providing coefficients that capture between-country effects of time-varying and time-constant country-level variables.

The practical relevance of estimating within effects of country-level variables (as in equations 2, 4, and 5) was demonstrated by te Grotenhuis et al. (2015). Focusing on the effects of social security and religious involvement, they showed that within-country estimates substantially differ from conventional RE estimators. Consequently, they diagnosed the latter to be biased by correlated unobserved heterogeneity and advised researchers to use within-country estimators if possible. If the researcher’s interest lies in cross-level interactions, however, the specification of a within estimator with desirable statistical properties is not as straightforward: the procedures described in equations (2), (3), and (5) do not yield a within-country estimator in this case, as we discuss in detail in the next section.

3. The Country FE Model with Cross-Level Interactions

If the research interest is not in the direct effect of a contextual variable (such as antidiscrimination law) on the dependent variable (such as wages) but is rather in its moderating influence on the effect of an individual-level characteristic (such as gender)—or vice versa, if the interest is not in the direct effect of an individual-level variable but in its moderating impact on the effect of a contextual variable—then a cross-level interaction term constitutes the main independent variable in the empirical model.

In such cases, a within estimator of a cross-level interaction should technically measure how changes in, for example, antidiscrimination laws within countries are related to within-country variation in, for example, the gender wage-gap. However, the simple cFE approach from equation (2), suggested in methodological papers and applied in several research papers, does not yield such a within-country estimator of interaction effects: the introduction of cluster dummies absorbs differences in levels but not differences in slopes. That is, the cFE model controls for heterogeneity in the level of $y_{jti}$ between clusters, but it does not control for the effect heterogeneity between them. The same is obviously true for the alternative specifications of the within estimator, as shown in equations (3) and (5).

Considering the example from above, wage disparities between men and women may depend not only on (time-varying) antidiscrimination laws but also on unobservable country-specific (time-constant) gender norms. If gender norms are correlated with antidiscrimination laws, which seems like a reasonable assumption, the conditional effect of gender ( $\frac{\partial wage}{\partial gndr} = β_{1} + β_{3} polic y_{j}$ ) estimated by cFE will not only reflect the moderating effect of antidiscrimination law but also the moderating impact of societal gender norms. Similarly, if the distribution of gender in the working population depends on gender norms, which again seems like a reasonable assumption, the conditional effect of antidiscrimination law ( $\frac{\partial wage}{\partial policy} = β_{2} + β_{3} gnd r_{ji}$ ) will not only reflect the moderating effect of gender but also the moderating impact of societal gender norms.

This phenomenon is somewhat similar to the specification problems of regular cross-sectional models with interaction terms. Assume that we are interested in the impact that a moderator variable $z$ has on the effect of $x$ . Assume, furthermore, that $z$ is highly correlated with the unobserved moderator $w$ , such that the coefficient of the interaction term $zx$ will be biased, as it transports the influence that $w$ has on the effect of $x$ . In this case, the introduction of $w$ into the model alone will not disentangle the moderating effect of $w$ from the coefficient of the interaction term $zx$ . In addition, the interaction term $wx$ needs to be introduced to control the moderating effect of $z$ on $x$ for the moderating impact of $w$ on $x$ .

Transferring these mechanisms to pooled cross-sectional data, cluster dummies take the role of $w$ . If the cluster variable of interest ( $z$ ) is correlated with several cluster unobservables ( $w$ ), which influence the effect of the level 1 variable $x$ , the introduction of cluster dummies alone will not yield a genuine within estimate of the interaction term $zx$ . In contrast, the estimated interaction effect will transport the effects of unobserved heterogeneity between clusters. To hold country-specific impacts on the effects of the individual-level variables constant, interactions between the dummy variables and the individual-level variable of interest must be specified. If, instead, the individual-level variable x is correlated with cluster unobservables (w), which moderate the effect of the cluster-level variable (z), interactions between the dummies and the cluster variable must be specified to control for the effect heterogeneity (in z). If both variables that constitute the interaction term of interest are moderated by unobserved correlated variables, each must be interacted with the set of dummies.

The persistence of country-specific effect heterogeneity as a confounder in cFE designs can also be illustrated via some basic transformations of the standard cFE model. We start with the basic DGP from equation (1), expanding it with a cross-level interaction of a time-varying country-level variable and an individual-level variable ( $z_{1 jt} x_{jti}$ ). Furthermore, we allow the unobservables at the country level to moderate the effects of $x_{jti}$ and $z_{1 jt} .$ (For ease of presentation, we drop $z_{2 j}$ from the DGP):

y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} (z_{1 jt} x_{jti}) + v_{0 j} + v_{1 j} x_{jti} + v_{2 j} z_{1 jt} + u_{0 jt} + e_{jti},

(6)

where $v_{1 j}$ and $v_{2 j}$ represent unobserved effect heterogeneity between countries (of x and z₁, respectively).

If country dummies are specified in a regression-based estimation of equation (6), then $v_{1 j}$ and $v_{2 j}$ are not eliminated from the error term. This is shown via transformations of this model. As we discussed in the previous section, the introduction of country dummies is equivalent to demeaning all variables on the country level:

\begin{matrix} (y_{jti} - {\bar{y}}_{j}) = & β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} (z_{1 jt} x_{jti} - {\bar{z_{1} x}}_{j}) \\ + (v_{0 j} - {\bar{v_{0}}}_{j}) & + (v_{1 j} x_{jti} - {\bar{v_{1} x}}_{j}) + (v_{2 j} z_{1 jt} - {\bar{v_{2} z_{1}}}_{j}) + (u_{0 jt} - {\bar{u_{0}}}_{j}) + (e_{jti} - {\bar{e}}_{j}), \end{matrix}

(7)

with ${\bar{v_{1} x}}_{j}$ and ${\bar{v_{2} z_{1}}}_{j}$ each representing the country-specific mean values of the products $v_{1 j} x_{jti}$ and $v_{2 j} z_{1 jt}$ , respectively. They can alternatively (or, rather, more exactly) be written as ${\bar{v_{1 j} x_{jti}}}_{j}$ and ${\bar{v_{2 j} z_{1 jti}}}_{j}$ .

As $v_{1 j}$ , $v_{2 j}$ and $v_{0 j}$ are constant within clusters j, for each jti it is⁹

v_{0 j} - {\bar{v_{0}}}_{j} = 0,

(8.1)

\begin{matrix} {\bar{v_{1 j} x_{jti}}}_{j} = \frac{\sum_{t = 1}^{T} \sum_{i = 1}^{n} v_{1 j} x_{jti}}{Tn} = \frac{v_{1 j} \sum_{t = 1}^{T} \sum_{i = 1}^{n} x_{jti}}{Tn} \\ = v_{1 j} \frac{\sum_{t = 1}^{T} \sum_{i = 1}^{n} x_{jti}}{Tn} = v_{1 j} {\bar{x}}_{j}, \end{matrix}

(8.2)

and

\begin{matrix} {\bar{v_{2 j} z_{1 jti}}}_{j} = \frac{\sum_{t = 1}^{T} \sum_{i = 1}^{n} v_{2 j} z_{1 jti}}{Tn} = \frac{v_{2 j} \sum_{t = 1}^{T} \sum_{i = 1}^{n} z_{1 jti}}{Tn} \\ = v_{2 j} \frac{\sum_{t = 1}^{T} \sum_{i = 1}^{n} z_{1 jti}}{Tn} = v_{2 j} {\bar{z_{1}}}_{j} . \end{matrix}

(8.3)

Therefore, the model in equation (7) reduces to

\begin{matrix} (y_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} (z_{1 jt} x_{jti} - {\bar{z_{1} x}}_{j}) \\ + v_{1 j} (x_{jti} - {\bar{x}}_{j}) + v_{2 j} (z_{1 jt} - {\bar{z_{1}}}_{j}) + (u_{0 jt} - {\bar{u_{0}}}_{j}) + (e_{jti} - {\bar{e}}_{j}) . \end{matrix}

(9)

Equation (9) shows that the error term is independent of cluster-specific heterogeneity ( $v_{0 j}$ ) and, consequently, the estimators of $β_{1}$ and $β_{2}$ are controlled for unobserved heterogeneity. However, contrary to the idea of within estimation, the error term is not free of cluster-specific heterogeneity in the effects of $x_{jti}$ and $z_{1 jt}$ , as it still contains the components $v_{1 j}$ and $v_{2 j}$ . In other words, $v_{1 j}$ and $v_{2 j}$ are unobserved country-level moderators of the effect of $x_{jti}$ on $y_{jti}$ and of $z_{1 jt}$ on $y_{jti}$ , and they remain uncontrolled after introducing cFE. If these unobserved moderators correlate with the observed moderators, the estimated interaction effect $β_{3}$ will be biased.

The insight that differences across clusters in the effects of predictors may exist in hierarchical frameworks is not new: although not directly introduced as a means to control for effect heterogeneity, Hoffmann (2015:494) explicitly discussed the systematic specification of such between-cluster differences in effects on the basis of cluster-dummy interactions. However, as our review of empirical and methodological articles shows, specifically in cross-country research, such a specification is not yet acknowledged. To obtain a genuine within estimator $β_{3}$ that does not exhibit country effect heterogeneity, this heterogeneity must be systemically specified in the fixed part of the model. On the basis of the LSDV approach, this systematic specification works via introducing interactions of the cluster-dummies with the variable that exhibits correlated effect heterogeneity. Thereby country-specific effect heterogeneity in $x_{jti}$ and $z_{1 jt}$ is shifted from the error term to the systematic part of the model, and the terms $v_{1 j} x_{jti}$ and $v_{2 j} z_{1 jt}$ from the DGP in equation (6) are replaced:

y_{j t i} = β_{0} + β_{1} x_{j t i} + β_{2} z_{1 j t} + β_{3} (z_{1 j t} x_{j t i}) + \sum_{j = 1}^{N - 1} γ_{0 j} c_{j} + \sum_{j = 1}^{N - 1} γ_{1 j} (c_{j} x_{j t i}) + \sum_{j = 1}^{N - 1} γ_{2 j} (c_{j} z_{1 j t}) + u_{0 j t} + e_{j t i}

(10)

The model in equation (10) controls not only for unobserved heterogeneity between countries but also for unobserved effect heterogeneity in the effects of $x_{jti}$ and $z_{1 jt}$ .¹⁰ We call this model the country FE and slopes model (cFES). Whether it is necessary to control for effect heterogeneity in both $x_{jti}$ and $z_{1 jt}$ to obtain unbiased estimates of $β_{3}$ is an empirical question. First, it may be that there is no effect heterogeneity at all with respect to one of the involved variables. Second, and more important, with many individual-level variables it is quite likely that their country-level means are (almost) equal across countries. This implies that there cannot be a (strong) correlation between the individual-level variable and an unobserved country-level moderator, and therefore it is not necessary to control for effect heterogeneity in $z_{1 jt}$ to obtain an unbiased estimate of the interaction effect $β_{3}$ . Under such conditions, the more parsimonious specification, which leaves $v_{2 j} z_{1 jt}$ unspecified and, thus, in the error term, may be preferred.

The cFES model can also be specified via transformations of the involved variables, as in the classic FE approach (equation 3), or through an effect decomposition in an RE framework (as in equation 5). In these frameworks, the estimator derived in equation (10) can be replicated by additional transformations that are not part of the standard FE or RE model. Remember that the standard FE model simply demeans all variables in the model; that is, the interaction term is generated first and then demeaned (compare equation 4). To control for effect heterogeneity in $x_{jti}$ , the variable $z_{jt}$ must be demeaned before entering the interaction term and before demeaning the complete function. If we let $(z_{1 jt} - {\bar{z_{1}}}_{j})$ be ${\overset{\cdot\cdot}{z}}_{1 jt}$ , then a model controlling for effect heterogeneity in $x_{jti}$ is (for reasons of simplicity we switch from a model including the random part to a model of expected values)

({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} ({\overset{\cdot\cdot}{z}}_{1 jt} x_{jti} - {\bar{{\overset{\cdot\cdot}{z}}_{1} x}}_{j}) .

(11.1)

Equivalently, controlling for effect heterogeneity in $z_{1 jt}$ is achieved with the following model:

({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} (z_{1 jt} {\overset{\cdot\cdot}{x}}_{jti} - {\bar{z_{1} \overset{\cdot\cdot}{x}}}_{j}),

(11.2)

where ${\overset{\cdot\cdot}{x}}_{jti}$ is $(x_{jti} - {\bar{x}}_{j})$ . Finally, controlling for effect heterogeneity in both $x_{jti}$ and $z_{1 jt}$ is done by demeaning both variables before they enter the interaction and then demeaning the complete function:

({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} ({\overset{\cdot\cdot}{z}}_{1 jt} {\overset{\cdot\cdot}{x}}_{jti} - {\bar{{\overset{\cdot\cdot}{z}}_{1} \overset{\cdot\cdot}{x}}}_{j}) .

(11.3)

By demeaning the components before they enter the interaction term, they become orthogonal to any (country-level) variable that is constant over time (compare Balli and Sørensen 2013).

In the RE framework, we can replicate the cFES models from equation (10) via the following specification:

y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{1 jt} x_{jti} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + β_{6} {\bar{z_{1}}}_{j} x_{jti} + β_{7} {\bar{x}}_{j} z_{1 jt} + v_{0 j},

(12)

where the term $β_{6} {\bar{z_{1}}}_{j} x_{jti}$ controls for effect heterogeneity in $x_{jti}$ and the term $β_{7} {\bar{x}}_{j} z_{1 jt}$ for effect heterogeneity in $z_{1 jt}$ . In a similar context, this was discussed by Hoffman (2015:chap. 11), who suggested that between-cluster heterogeneity in lower level effects can be controlled for by interacting cluster-specific means with the lower level variables that exhibit effect heterogeneity. The interaction term $z_{1 jt} x_{jti}$ will provide a genuine within estimate because the interaction effects of ${\bar{z_{1}}}_{j} x_{jti}$ and ${\bar{x}}_{j} z_{1 jt}$ measure how the effect of $x_{jti}$ , respectively $z_{1 jt}$ , depends on between-country differences in ${\bar{z_{1}}}_{j}$ , respectively ${\bar{x}}_{j}$ , thereby partialing out the correlated between-country differences in the effects of $x_{jti}$ and $z_{1 jt}$ . In the Online Appendix to this article,¹¹ we demonstrate how to apply these different specifications to real survey data using Stata syntax.¹²

Before demonstrating the properties of the cFES model by means of simulations in the next section, we briefly discuss unobserved effects at the level of country-waves ( $u_{0 jt}$ ). It is obvious from equation (2) that the cFE model controls for unobserved heterogeneity at the level of countries but does not control for such heterogeneity at the level of country-years ( $u_{0 jt}$ remains part of the equation). In other words, the model will yield unbiased results only if there are no omitted correlated time-varying country-level variables. The same is true for the cFES model in its various forms (equations 10 to 12). However, the fact that $u_{0 jt}$ remains part of the equation indicates not only that there is a source of potential bias in coefficients but also that there might be statistical dependencies left in the data that we need to account for to obtain unbiased standard errors. In a strict FE framework, as in equations (10) and (11), the only way to account for such dependencies would be to use cluster robust standard errors, and we recommend doing so. In an RE framework, as in equation (12), these dependencies could be accounted for via an additional random intercept at the level of country-years, and we recommend doing so (see also Schmidt-Catran and Fairbrother 2016). For the simulations that follow, however, we estimate any RE model with a random intercept only at the level of countries, in order to have a specification that is comparable with the FE models.

4. Monte Carlo Simulations

We simulated data using the following basic DGP:

\begin{matrix} y_{jti} = β_{0} + β_{1} x_{1 jti} + β_{2} z_{1 jt} + β_{3} x_{1 jti} z_{1 jt} + β_{4} z_{2 j} + β_{5} x_{1 jti} z_{2 j} \\ + β_{6} z_{1 jt} z_{2 j} + v_{0 j} + v_{1 j} x_{1 jti} + v_{2 j} z_{1 jt} + u_{0 jt} + u_{1 jt} x_{1 jti} + e_{jti} . \end{matrix}

This DGP corresponds to a three-level model with random intercepts and random slopes at both higher levels. We see that $y_{jti}$ is dependent on an individual-level variable, $x_{1 jti}$ , a country-level variable that varies over time, $z_{1 jt}$ , and a second country-level variable that is time-invariant, $z_{2 j}$ . Both country-level variables interact with the individual-level variable, $x_{1 jti}$ . At each higher level, the random part of the model contains a random intercept ( $v_{0 j}$ and $u_{0 jt}$ ) and a random slope of $x_{1 jti}$ and $z_{1 jt}$ ( $v_{1 j}$ , $u_{1 j t}$ , and $v_{2 j}$ ). $e_{jti}$ is an idiosyncratic error. The random intercepts and the idiosyncratic error ( $v_{0 j}, u_{0 jt}, e_{jti}$ ) have a standard normal distribution; these are uncorrelated with each other and also with all variables from the fixed part of the model $(x_{1 jti}, z_{1 jt,} z_{2 j}$ ). The random slopes ( $v_{1 j}, v_{2 j}, u_{1 jt})$ are also uncorrelated with one another and with any other variable from the DGP but with a smaller standard deviation (SD = .25) than we used for the random variation in levels. All variables from the fixed part $(x_{1 jti}, z_{1 jt,} z_{2 j}$ ) are also generated from standard normal distributions, but $z_{1 jt}$ consists of two components: one country-level component ( $z_{1 j})$ and one country-year-level component $z_{1 jt}'$ ( $z_{1 jt}$ = $z_{1 j} + z_{1 jt}'$ ), where each of the components of $z_{1 jt}$ has a standard normal distribution. Likewise, $x_{1 jti}$ is made of three components, one at each level: $x_{1 jti}$ = $x_{1 j} + x_{1 jt}' + {x ″}_{1 jti}$ . This procedure allows us to specify correlations with $x_{1 jti} and z_{1 jt}$ at the level of countries (see below). We simulate data for 30 countries, 10 years, and 100 individuals within each country-year.

The true parameters of the DGP are as follows:

\begin{matrix} β_{0} = 0; β_{1} = 1; β_{2} = 1; β_{3} = - 1; β_{4} = 1; \\ depending on condition : \\ β_{5} = 0 or - 1; β_{6} = 0 or - 1 . \end{matrix}

The variable $z_{2 j}$ takes the role of an unobserved confounder. That is, we omit it from the estimated models. What varies between the single DGPs are (1) the correlations between the unobserved country-level variable $z_{2 j}$ and the observed country-level components $z_{1 j} and x_{1 j}$ and (2) the size of the interaction effects $β_{5} and β_{6}$ . $Corr (z_{1 j}, z_{2 j})$ and $Corr (x_{1 j}, z_{2 j})$ each take the values 0 and .6, and $β_{5} and β_{6}$ each take the values 0 and –.1. The conditions in which $β_{5}$ and $β_{6}$ equal zero generate scenarios in which there is no unobserved interaction, while there is one if they do not equal zero. In conditions in which observed and unobserved moderators are correlated, an unobserved interaction is expected to bias the interaction effect of the observed variables when using the standard cFE model.

We fit three sets of models to each generated data set. In the first set, we employ the LSDV approach and run four different specifications: (1) a model in which we include simple country dummies (the cFE model, as in equation 2), (2) a model in which we control for effect heterogeneity in $x_{1 jti}$ , (3) a model in which effect heterogeneity in $z_{1 jt}$ is controlled for, and (4) a model in which effect heterogeneity in both variables is controlled (as in equation 10).

The second set of models uses (extended) FE transformations. Within this set we use the same four specifications as in the first set: (1) a model that uses the standard approach (as in equation 4), (2) a model that controls for effect heterogeneity in $x_{1 jti}$ (as in equation 11.1), (3) a model that controls for effect heterogeneity in $z_{1 jt}$ (as in equation 11.2), and (4) a model that controls for effect heterogeneity in both variables (as in equation 11.3).

The third set of models uses the RE approach and consists of the same four specifications as the other two sets (as in equations 5 and 12). As noted above, we include in these RE models only a random intercept at the country level and no additional random effects. In sum, each simulation consists of 192 estimated models (2×2×2×2 conditions, with 12 models fitted to each [three sets and four specifications]). We ran a total of 1,000 simulations. Table A3 presents the DGPs and the specifications of the 12 models fitted to the generated data.

Figure 1 shows the average estimates of the interaction effect between $x_{1 jti}$ and $z_{1 jt}$ ( $β_{3}$ ) across 1,000 simulations. The y-axes give the estimated parameter for $β_{3}$ , where the true value is $- 1$ , and the interval shown around the average estimates is the space covered by the middle 95 percent of estimates. Each subgraph presents the three sets of models with the four specifications described above. The different conditions in the DGP define the rows and columns of the figure so that each subgraph presents the results for a specific combination of conditions.

Figure 1.

Results of simulation study for $β_{3} .$

The four subgraphs in the upper left corner (1, 2, 5, 6) present conditions in which there is no unobserved interaction effect ( $β_{5} = 0$ and $β_{6} = 0$ ). As expected, the standard specifications of all three modeling approaches obtain unbiased effects under these conditions. Unsurprisingly, the cFES models also yield unbiased estimates. The first three subgraphs in the third row (9, 10, 11), as well as the first three in the third column (3, 7, 11), present conditions in which there are unobserved moderators but they are not correlated with the variables in the model. Again, all specifications provide unbiased estimates.

The interesting conditions are depicted in the last column and row. The first three graphs in the last row (13, 14, 15) show conditions in which an unobserved interaction between $x_{1 jti}$ and $z_{2 j}$ is present ( $β_{5} = - 1)$ and the observed variable $z_{1 jt}$ is correlated with the unobserved moderator ( $Corr (z_{1 j}, z_{2 j}) = . 6$ ). In other words, there is effect heterogeneity in $x_{1 jti}$ that correlates with $z_{1 jt}$ . The standard specifications (model 1 within each set) yield biased results, while the cFES models controlling for effect heterogeneity in $x_{1 jti}$ (models 2 and 4) give unbiased estimates, as expected. Obviously, the cFES models controlling for effect heterogeneity in $z_{1 jt}$ but not in $x_{1 jti}$ (models 3) provide biased results as well. Equivalently, the first three subgraphs in the last column (4, 8, 12) present conditions in which there is effect heterogeneity in $z_{1 jt}$ and the unobserved moderator correlates with $x_{1 jti} .$ The standard specification and the cFES models controlling for effect heterogeneity in $x_{1 jti}$ (models 1 and 2) provide biased estimates, while the models controlling for effect heterogeneity in $z_{1 jt}$ give unbiased estimates.

Finally, subgraph 16 shows the simulation results under the condition that the effects of both variables constituting the interaction term are moderated by an unobserved variable. This unobserved moderator correlates with the two observed variables. In this case, only the cFES models accounting for effect heterogeneity in both variables ( $x_{1 jti}$ and $z_{1 jt})$ provide unbiased results.

We do not discuss the results with regard to the other parameters in detail here. Figure OA1 in Online Appendix B shows the average estimates of the parameters $β_{1}$ and $β_{2} .$ As expected, all specifications provide unbiased estimates of the main effects of $x_{1 jti}$ and $z_{1 jt}$ , because these are correlated only with time-constant country-level variables, which are controlled for by simple cFE. We also do not go into the details regarding differences in point estimates between the three modeling approaches. As later discussed in note 15, the LSDV approach, the FE approach with transformations, and the RE approach give exactly the same results only if the data are perfectly balanced. This is demonstrated in Online Appendix C. In our simulations, this condition was not met due to the random sampling. On average, the point estimates from the LSDV approach deviated from the FE-transformation estimates by about 3.2 percent and the estimates from the hybrid models by about 1.6 percent.

After showing formally and by means of simulations that the simple cFE model yields biased estimates of cross-level interaction effects in the presence of (correlated) unobserved effect heterogeneity at the country level, we now turn to an empirical example from published research in order to demonstrate the relevance of this issue for applied research.

5. Empirical Example: The Impact of Active Labor Market Policies on Subjective Well-Being of the Unemployed

We reproduce the results of a published study (Wulfgramm 2014)¹³ that focuses on cross-level interaction effects and uses a standard cFE model. We compare its coefficients with results from an extended FE model with additional country-interactions (cFES), as introduced in Section 3. The analyses are based on four waves of the European Social Survey (Norwegian Centre for Research Data 2002, 2004, 2006, 2008; see Table 2 in Wulfgramm 2014 for details).

The author hypothesized that the detrimental effect of individual unemployment on subjective well-being depends on the configuration of country-specific labor market policies. Specifically, it is expected that generous benefit systems weaken the negative effect of unemployment on life-satisfaction (Wulfgramm 2014:261, H1). The author expressed her concerns about unobserved country-level effects that “might lead to endogeneity problems” (p. 263). As the country-level variables of interest (labor market policies) change over time, the author consequently introduces within-country estimation as the solution to the endogeneity problems: “Therefore, models that include country FE are estimated. . . . Thus, changes in the severity of the life satisfaction effect of unemployment can be traced back to policy changes within countries across time” (p. 263).

In light of the discussion above, this statement is statistically not correct: As we have shown, the standard cFE-coefficient of cross-level interactions is not a genuine within estimator. Contrary to the claims, this coefficient does not exclusively measure the effects of changes in policies over time within countries, but it uses between-country variance and, therefore, is subject to heterogeneity bias. This bias might, for example, stem from the influence of country-specific work ethics, which impact the detrimental psycho-emotional effects of unemployment (Stavrova, Schlösser, and Fetchenhauer 2011). To the extent that such ethics are correlated with unemployment policies or unemployment risks, their moderating impact will be transported in the standard cFE interaction coefficients.

The consequences of leaving effect heterogeneity uncontrolled using cFE are relevant for the conclusions of the study. Table 1 displays three different specifications of the key model in Wulfgramm (2014). Column 1 shows the figures from an RE model of the main effects and the cross-level interaction. Column 2 presents the coefficients from an equivalent model with country dummies (cFE), used by the author to demonstrate the robustness of findings. Both figures stem from the original article (models 2 and 7) and are reproduced exactly in our replication. Column 3 shows results from our replication of the cFE-model using the cFES model to control for effect heterogeneity in both variables constituting the interaction term (individual unemployment and unemployment benefit generosity).¹⁴ We also estimated models controlling for effect heterogeneity in only one of the two variables. This analysis shows that controlling for effect heterogeneity in unemployment benefit generosity leads to similar results as simple cFE, while controlling for effect heterogeneity in individual unemployment leads to the interaction coefficient changing as drastically as with full cFES.

Table 1.

Analysis of Life Satisfaction: Replication and Extension of Analyses by Wulfgramm (2014)

	(1) RE	(2) cFE	(3) cFES
Dependent Variable: Life Satisfaction	Coefficient (\|t\|)	Coefficient (\|t\|)	Coefficient (\|t\|)
Level 1 variables
Main activity (reference: paid work)
Unemployment	–1.500*** (33.5)	–1.510*** (8.4)	–.887*** (6.7)
Level 2 variables
ALMP expenditure	.000 (.1)	–.001 (1.1)	–.001 (1.2)
Unemployment benefit generosity	.006* (2.1)	.005** (2.9)	.004 (.0)
Ln GDP per capita	.820** (3.1)	.230 (.3)	1.255* (2.1)
Social expenditure	–.020 (1.4)	–.020 (1.3)	.007 (.4)
Unemployment rate	–.060*** (6.3)	–.070*** (5.8)	–.070*** (7.1)
Cross-level interaction: unemployment*
Unemployment benefit generosity	.013*** (9.6)	.014** (3.3)	–.011 (1.1)
Constant	–1.820 (.7)	4.430 (.6)	–6.701 (1.1)
Method	MLM (RE)	OLS (cFE)	OLS (cFES)
N1 (persons)	107,973	107,973	107,973
N2 (country-years)	72	72	72
N3 (countries)	21	21	21

Note: Level 1 control variables as in Wulfgramm (2014). ALMP = active labor market policy; cFE = country fixed effects; cFES = country fixed effects and slopes; GDP = gross domestic product; MLM = multilevel modeling; OLS = ordinary least squares; RE = random effects.

p < .05, **p < .01, and ***p < .001 (two-tailed tests).

The interaction effect relating to the key hypothesis changes substantively in size and significance once the genuine within estimator we propose is employed: although the coefficient of the simple cFE model suggests that increasing benefit generosity weakens the detrimental effect of unemployment significantly (by +0.014 scale points per standard deviation), the cFES model provides an insignificant negative effect. Hence, the findings originally reported in Wulfgramm (2014) are subject to a heterogeneity bias; once country effect heterogeneity is accurately controlled for by the specification we propose, the study’s null hypothesis can no longer be rejected.¹⁵

Despite this conclusion, it is important to note that Wulfgramm’s (2014) original study is one of only five studies examined in our literature survey that attempted to hold country effects constant. Furthermore, the original findings still reveal relevant and substantial cross-level associations between countries. However, the author’s claim that results are robust against a within-country specification is not correct. Hence, a more defensive interpretation of the results with regard to causal mechanisms (possibly connected with a discussion on the limited potential of four-wave macro-data to reveal complex within associations) seems to be appropriate.

6. Discussion

Because of strong investments in the international research data infrastructure since the early 2000s, social scientists have access to surveys that are collected in many countries and repeated at regular intervals. The structure underlying this survey data allows estimation of within-country effects of time-varying country-level characteristics by means of cFE. Although the general statistical properties of cFE models are increasingly appreciated in social science, the specification of within estimators of cross-level interaction effects has not yet received special attention. Through formal and econometric arguments, we have shown that the standard cFE approach does not yield a genuine within estimator of cross-level interaction terms. Consequently, this standard approach leads to biased coefficients if correlated country-level moderators remain unobserved or unspecified. This has also been verified through Monte Carlo simulations. By replicating the analysis from Wulfgramm (2014), we emphasize that employing the extended cFES estimator is essential for practitioners concerned with causal interpretations of cross-level interactions.

We proposed models that control for effect heterogeneity of one or both interacted variables. Although only the latter qualifies as a genuine within estimator, it may be that a more parsimonious specification is appropriate in some research situations. In our replication, it turned out that controlling for effect heterogeneity in the individual-level variable (unemployment) had a huge effect on the estimated interaction effect, while controlling for effect heterogeneity in the country-year-level variable did not provide substantially different results compared with standard cFE. We believe that this situation is common with cross-country data because the correlation between a micro-level variable and an unobserved country-level moderator will be low in many cases. In such situations, controlling for country effect heterogeneity in the individual-level variable may be sufficient.

Although our paper focuses on repeated cross-sectional international survey data, the technical issues demonstrated here also apply to other types of data with similar multilevel structures. In our applications, we referred to individuals (level 1) nested in time-specific entities (level 2) that are embedded in countries (level 3). Obviously, these models can be applied to data from repeated national surveys, where subnational regions would be the relevant contextual entities, as well as any other data that have a hierarchical structure and a longitudinal dimension (such as individuals in neighborhoods, pupils in schools). A structurally identical design is employed when fitting models to cross-sectional international survey data, where the relevant contextual characteristics are measured at the level of subnational regions. Here, individuals (level 1) are nested within regions (level 2) that are nested in countries (level 3). In such designs, researchers might introduce country FE in order to control for unobserved heterogeneity between countries (for an example, see Finseraas 2012). The estimated effect of contextual-level variables measured at the regional level then is also a within-country estimator.

Generally, our study can be related to the methodological literature encouraging more attention to effect heterogeneity in multilevel models. Heisig, Schaeffer, and Giesecke (2017), for example, reveal that the specification of cluster-specific effect heterogeneity as random variance (random slopes) may lead to smaller and more accurate standard errors in multilevel applications—specifically if countries are the clusters. Our work shows that in addition to efficiency gains, the specification of effect heterogeneity can also eliminate biases that stem from unobservable context-level moderators. To realize this potential, however, effect heterogeneity must be specified in the fixed part of the model and not as random variance. Accounting for effect heterogeneity in terms of random slopes will not entirely partial out unobserved moderation effects.

Should empirical practitioners interested in cross-level interactions therefore use the suggested cFES estimation by default? If they use repeated cross-context data with the aim to improve causal interpretations based on within estimation, the answer has to be yes. Any model without a systematic specification of cluster-level effect heterogeneity, such as simple cFE or an estimation with random slopes, would be inconsistent with the outlined motive and would not fully eliminate the effect of correlated unobservable context-level moderators from cross-level interaction estimates.

There are, however, certain limitations of the cFES model that practitioners should be aware of. First, it does not control for country-year (level 2) specific effect heterogeneity. This reflects a general limitation of within estimators: they allow us to partial out effects above the lowest level of variation of an independent variable, but not at this level itself because identification would become impossible. Thus, the cFES model controls for unobserved time-constant moderators but not for unobserved moderation effects at the level of country-years.

For the same reason, characteristics that do not vary within countries cannot be tested as moderators in a cFES specification. Variables that exhibit only small within-country variance can technically be identified, but the estimates may be very imprecise. Although we refrain from formulating a specific rule or threshold on sufficient levels of within-cluster variation, a careful decision on whether to use solely within-cluster variation is a sensible strategy made in each research situation.

Supplemental Material

Schmidt-Catran_crosslevel_online_appendix – Supplemental material for Getting the within Estimator of Cross-level Interactions in Multilevel Models with Pooled Cross-sections: Why Country Dummies (Sometimes) Do Not Do the Job

Supplemental material, Schmidt-Catran_crosslevel_online_appendix for Getting the within Estimator of Cross-level Interactions in Multilevel Models with Pooled Cross-sections: Why Country Dummies (Sometimes) Do Not Do the Job by Marco Giesselmann and Alexander W. Schmidt-Catran in Sociological Methodology

Footnotes

Appendix

Table A3.

DGPs and Fitted Models in the Simulation

DGPs	LSDV Models	FE Transformation Models	RE Models
$y_{jti} = 1 x_{1 jti} + 1 z_{1 jt} - 1 x_{1 jti} z_{1 jt} + 1 z_{2 j} + β_{5} x_{1 jti} z_{2 j} + β_{6} z_{1 jt} z_{2 j} + v_{0 j} + v_{1 j} x_{1 jti} + v_{2 j} z_{1 jt} + u_{0 jt} + u_{1 jt} x_{1 jti} + e_{jti}$ , with $β_{5}$ taking the values 0 and −1 $β_{6}$ taking the values 0 and −1 $Corr (z_{1 j}, z_{2 j})$ taking the values 0 and .6 $Corr (x_{1 j}, z_{2 j})$ taking the values 0 and .6	(1) Standard model: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} x_{jti} z_{1 jt} + \sum_{j = 1}^{N - 1} γ_{0 j} c_{j} + e_{jti}$	(1) Standard model: $(y_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} (z_{1 jt} x_{jti} - {\bar{z_{1} x}}_{j}) + e_{jti}$	(1) Standard model: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{1 jt} x_{jti} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + β_{6} {\bar{z_{1}}}_{j} x_{jti} + β_{7} {\bar{x}}_{j} z_{jt} + v_{0 j} + e_{jti}$
	(2) Controlling effect heterogeneity in x: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} x_{jti} z_{1 jt} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{0 j} c_{j} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{1 j} (c_{j} x_{jti}) + e_{jti}$	(2) Controlling effect heterogeneity in x: $({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} ((z_{1 jt} - {\bar{z_{1}}}_{j}) x_{jti} - {\bar{(z_{1 jt} - {\bar{z_{1}}}_{j}) x}}_{j})$ $+ e_{jti}$	(2) Controlling effect heterogeneity in x: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{1 jt} x_{jti} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + β_{6} {\bar{z_{1}}}_{j} x_{jti} + v_{0 j} + e_{jti}$
	(3) Controlling effect heterogeneity in z: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} x_{jti} z_{1 jt} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{0 j} c_{j} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{2 j} (c_{j} z_{jt}) + e_{jti}$	(3) Controlling effect heterogeneity in z: $({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} (z_{1 jt} (x_{jti} - {\bar{x}}_{j}) - {\bar{z_{1} (x_{jti} - {\bar{x}}_{j})}}_{j})$ $+ e_{jti}$	(3) Controlling effect heterogeneity in z: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{1 jt} x_{jti} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + + β_{7} {\bar{x}}_{j} z_{jt} + v_{0 j} + e_{jti}$
	(4) Controlling effect heterogeneity in x and z: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} x_{jti} z_{1 jt} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{0 j} c_{j} + {\sum^{¯}}_{j = 1}^{N - 1} γ_{1 j} (c_{j} x_{jti}) + {\sum^{¯}}_{j = 1}^{N - 1} γ_{2 j} (c_{j} z_{jt}) + e_{jti}$	(4) Controlling effect heterogeneity in x and z: $({\hat{y}}_{jti} - {\bar{y}}_{j}) = β_{0} + β_{1} (x_{jti} - {\bar{x}}_{j}) + β_{2} (z_{1 jt} - {\bar{z_{1}}}_{j}) + β_{3} ((z_{1 jt} - {\bar{z_{1}}}_{j}) (x_{jti} - {\bar{x}}_{j}) - {\bar{(z_{1 jt} - {\bar{z_{1}}}_{j}) (x_{jti} - {\bar{x}}_{j})}}_{j}) + e_{jti}$	(4) Controlling effect heterogeneity in x and z: $y_{jti} = β_{0} + β_{1} x_{jti} + β_{2} z_{1 jt} + β_{3} z_{1 jt} x_{jti} + β_{4} {\bar{x}}_{j} + β_{5} {\bar{z_{1}}}_{j} + β_{6} {\bar{z_{1}}}_{j} x_{jti} + β_{7} {\bar{x}}_{j} z_{jt} + v_{0 j} + e_{jti}$

Note: DGP = data-generating process; FE = fixed effects; LSDV = least squares dummy variables; RE = random effects.

Acknowledgements

We thank David Brady for sharing his ideas on estimation issues in cross-country analyses and Matthew C. Mahutga for critical comments.

Notes

Author Biographies

Marco Giesselmann is an assistant professor for quantitative methods at the University of Bielefeld and a research associate at the Research Data Center of the Socio-Economic Panel Study (SOEP). At the present time, he primarily works on the methods and mechanics of panel data analysis. His applied research is mainly concerned with poverty dynamics and life course sociology. Combining these strands, he recently published, together with David Brady, Ulrich Kohler, and Anke Radenacker, “How to Measure and Proxy Permanent Income” (Journal of Economic Inequality, 2018).

Alexander W. Schmidt-Catran is a professor of sociology and quantitative methods at Goethe-University Frankfurt. His methodological work focuses on modeling techniques for comparative survey data and is published in journals such as Demography and the European Sociological Review. He is also the author of a textbook on the analysis of panel data, and his substantive research focuses on immigration and welfare states. He recently published related articles in the American Sociological Review, the Journal of European Social Policy, and Comparative Political Studies.

References

Allison

Paul D.

2009. Fixed Effects Regression Models. Thousand Oaks, CA: Sage.

Andress

Hans-Jürgen

Golsch

Katrin

Schmidt

Alexander W.

2013. Applied Panel Data Analysis for Economic and Social Surveys. Berlin, Germany: Springer.

Balli

Hatice Ozer

Sørensen

Bent E.

2013. “Interaction Effects in Econometrics.”Empirical Economics45(1):583–603.

Brady

David

Baker

Regina S.

Finnigan

Ryan

. 2013. “When Unionization Disappears: State-level Unionization and Working Poverty in the United States.”American Sociological Review78(5):872–96.

Brady

David

Finnigan

Ryan

. 2014. “Does Immigration Undermine Public Support for Social Policy?”American Sociological Review79(1):17–42.

Bryan

Mark L.

Jenkins

Stephen P.

2016. “Multilevel Modelling of Country Effects: A Cautionary Tale.”European Sociological Review32(1):3–22.

Edlund

Jonas

Lindh

Arvid

. 2015. “The Democratic Class Struggle Revisited: The Welfare State, Social Cohesion, and Political Conflict.”Acta Sociologica58(4):311–28.

Fairbrother

Malcolm

. 2013. “Rich People, Poor People, and Environmental Concern: Evidence across Nations and Time.”European Sociological Review29(5):910–22.

Fairbrother

Malcolm

. 2014. “Two Multilevel Modeling Techniques for Analyzing Comparative Longitudinal Survey Datasets.”Political Science Research and Methods2(1):119–40.

10.

Finseraas

Henning

. 2012. “Poverty, Ethnic Minorities among the Poor, and Preferences for Redistribution in European Regions.”Journal of European Social Policy22(2):164–80.

11.

Gangl

Markus

. 2010. “Causal Inference in Sociological Research.”Annual Review of Sociology36:21–47.

12.

Heisig

Jan Paul

Schaeffer

Merlin

Giesecke

Johannes

. 2017. “The Costs of Simplicity: Why Multilevel ModelsMay

Benefit from Accounting for Cross-cluster Differences in the Effects of Controls.”

American Sociological Review82(4):796–827.

13.

Hoffman

Lesa

. 2015. Longitudinal Analysis: Modeling Within-person Fluctuation and Change. New York: Routledge.

14.

Jaeger

Mads M.

2013. “The Effect of Macroeconomic and Social Conditions on the Demand for Redistribution: A Pseudo Panel Approach.”Journal of European Social Policy23(2):149–63.

15.

Mewes

Jan

. 2014. “Gen(d)eralized Trust: Women, Work, and Trust in Strangers.”European Sociological Review30(3):373–86.

16.

Mundlak

Yair

. 1978. “On the Pooling of Time-series and Cross Section Data.”Econometrica46:69–85.

17.

Norwegian Centre for Research Data. 2002. “European Social Survey Round 1 Data.” Data File Edition 6.4. Bergen: Norwegian Centre for Research Data.

18.

Norwegian Centre for Research Data. 2004. “European Social Survey Round 2 Data.” Data File Edition 3.4. Bergen: Norwegian Centre for Research Data.

19.

Norwegian Centre for Research Data. 2006. “European Social Survey Round 3 Data.” Data File Edition 3.5. Bergen: Norwegian Centre for Research Data.

20.

Norwegian Centre for Research Data. 2008. “European Social Survey Round 3 Data.” Data File Edition 4.3. Bergen: Norwegian Centre for Research Data.

21.

Przeworski

Adam

Teune

Henry

. 1970. The Logic of Comparative Social Inquiry. New York: Wiley-Interscience.

22.

Schmidt-Catran

Alexander W.

2016. “Economic Inequality and Public Demand for Redistribution: Combining Cross-sectional and Longitudinal Evidence.”Socio-economic Review14(1):119–40.

23.

Schmidt-Catran

Alexander W.

Fairbrother

Malcolm

. 2016. “The Random Effects in Multilevel Models: Getting Them Wrong and Getting Them Right.”European Sociological Review32(1):23–38.

24.

Schmidt-Catran

Alexander W.

Spies

Dennis

. 2016. “Immigration and Welfare Support in Germany.”American Sociological Review81(2):242–61.

25.

Snijders

Tom A. B.

Bosker

Roel J.

1999. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Thousand Oaks, CA: Sage.

26.

Stavrova

Olga

Schlösser

Thomas

Fetchenhauer

Detlef

. 2011. “Are the Unemployed Equally Unhappy All around the World? The Role of the Social Norms to Work and Welfare State Provision in 28 OECD Countries.”Journal of Economic Psychology32(1):159–71.

27.

Stegmueller

Daniel

. 2013. “How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches.”American Journal of Political Science57(3):748–61.

28.

te Grotenhuis

Manfred

Scholte

Marijn

de Graaf

Nan Dirk

Pelzer

Ben

. 2015. “The between and within Effects of Social Security on Church Attendance in Europe 1980–1998: The Danger of Testing Hypotheses Cross-nationally.”European Sociological Review31(5):643–54.

29.

Wooldridge

Jeffrey M.

2015. Introductory Econometrics: A Modern Approach. 5th ed.Mason, OH: South-Western.

30.

Wulfgramm

Melike

. 2014. “Life Satisfaction Effects of Unemployment in Europe: The Moderating Influence of Labour Market Policy.”Journal of European Social Policy24(3):258–72.

31.

Ziller

Conrad

. 2015. “Ethnic Diversity, Economic and Cultural Contexts, and Social Trust: Cross-sectional and Longitudinal Evidence from European Regions, 2002–2010.”Social Forces93(3):1211–40.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.46 MB