Abstract
Replications of published studies are of vital importance in any science, and especially in the social sciences, which seem to be particularly vulnerable to issues of reproducibility. Therefore, we gratefully acknowledge Auspurg, Brüderl, and Wöhler’s replication of our own study and thank them for their rigorous reanalysis of our data and models. Needless to say, we are also pleased that the replication of our models worked out smoothly. 1
In our initial study (Schmidt-Catran and Spies 2016, henceforth SCS), we analyzed the relationship between immigration and native Germans’ support of welfare between 1994 and 2010, testing for conflict and protection arguments. Whereas the latter assumes natives react to immigration with increased demand for social assistance, because they see immigrants as a threat to their own economic well-being, the conflict thesis states that natives’ reactions to immigration are driven by cultural concerns, eventually leading to lower support for redistributive policies. Using individual- and regional-level data and differentiating between cross-sectional and longitudinal effects of the latter, we found (1) a negative effect of the share of foreigners on German natives’ support for welfare and (2) interaction effects between the share of foreigners and the regional unemployment rate, which supports the conflict hypothesis. We concluded “that the relationship between migration and welfare-critical attitudes among the native population is not restricted to the United States but can also be identified in Germany” (SCS 2016:256)—and thus also in a European setting.
Auspurg and colleagues’ main concern is that our data violated the parallel trends assumption, which is implicit in our model specification. Once one drops the parallel trends assumption—that is, time trends are allowed to vary between eastern and western regions—the former significant effects of (1) and (2) become drastically smaller and lose their significance.
After learning of Auspurg and colleagues’ replication, we carefully reexamined our data and models. We have to conclude that Auspurg and colleagues’ methodological claims are solid, and we are happy to admit this: once non-parallel time trends are explicitly modeled, the negative main effect of the share of foreigners disappears and the interaction effects between the share of foreigners and the regional unemployment rate—our core theoretical argument—also fail to reach common levels of statistical significance.
In the following, we take the opportunity offered by the ASR editors and respond to Auspurg and colleagues’ replication. In general, we would like to relate our (now insignificant) findings to the still rapidly expanding scholarly debate on the effects of ethnic diversity/immigration on welfare support, and we would like to address two technical points, which we believe are important for the wider research community working with similar modeling approaches.
To start with our view on the literature (for recent reviews, see Brady and Finnigan 2014; Steele 2016; Stichnoth and Van der Straeten 2013), we would like to differentiate between three strongly interrelated groups of studies with different regional focuses or analytic approaches. The first concerns the effects of racial diversity in the United States, and here the evidence seems very clear: racially motivated beliefs (micro level) are strongly and negatively correlated with White Americans’ support of welfare (micro level). Also, in more racially heterogeneous contexts (macro level), individuals’ support for welfare is lower, leading many scholars to conclude that the residual character of the U.S. welfare state is strongly due to the society’s racial diversity (see, e.g., Fox 2004; Gilens 1999).
Second, other studies are interested in the effects of immigration (macro level) on welfare support (micro level) outside the U.S. context. Our initial study is one example of this, and Auspurg and colleagues cite others. Here, the reported results are indeed mixed. In our view, such mixed results are not surprising, as the studies differ greatly in the contextual level analyzed (some are international-comparative, others focus on local or regional differences inside one or more countries), the time period of the analysis, the measurement of welfare support, and so on—a picture comparable to the literature on the effects of contact with minorities on minority-related attitudes (Pettigrew and Tropp 2006). Based on such ambiguous results alone, we would follow Auspurg and colleagues and be very hesitant to conclude that the U.S. experience is somehow reflected in other countries.
However, none of the studies cited by Auspurg and colleagues control for immigration-related attitudes (micro level) or for their potential cross-level interaction with (actual or perceived) immigration. This is done by the third group of studies, and here the empirical evidence is—in our reading—again very clear: immigration-related attitudes (e.g., racism, ethnopluralism, cultural concerns about immigration) are related to low levels of general support for welfare (Finseraas 2008; Schmidt and Spies 2014; Senik, Stichnoth, and Van der Straeten 2008), lower support for some distinct social policies (Goldschmidt 2015), and especially welfare chauvinism (Larsen 2011). Only the perceived economic threat of immigration appears to increase, rather than reduce, individuals’ general support for welfare (Finseraas 2008). In summary, we see strong evidence that immigration—via its effects on cultural attitudes—exerts a significant effect on welfare attitudes outside the U.S. context.
How do our (now insignificant) findings on Germany relate to this broader picture? We would like to stress that we did not select Germany by chance, but for its theoretically derived status as being a least-likely case for any negative effects of immigration on natives’ support for welfare. Given Germany’s work-related welfare regime, its highly assimilationist integration policy, and (until recently) its highly restrictive immigration policy, we would not have expected any negative effects of immigration on German natives’ support for welfare, in the East or the West, as both regions have shared this institutional setting for nearly 30 years.
Auspurg and colleagues’ replication of our data teaches us that there are indeed no significant effects, but it also shows that the directions of the now insignificant negative effects are still as reported in our initial study (see Table 1 in Auspurg and colleagues). Auspurg and colleagues’ Figure C1 in their online supplement provides a nice comparison of the original and replication results. The general patterns of the conditional effects are identical overall, particularly for the interaction between the share of foreigners and the unemployment rate, 2 which misses the common criterion of significance by only a small margin. One could argue that we used the entire population of regions (“Raumordnungsregionen” [RORs]), 3 and that p-values are irrelevant in such a census-setting, but we do not wish to defend our initial results with such purely statistical arguments. However, we do stand by the substantive conclusion that there is no evidence in Germany that immigration causes an overall increase in welfare support, contra the protection hypothesis. Given that studies on other least-likely cases do find a negative effect of immigration (macro level) on natives’ support for welfare (on Sweden, see Dahlberg, Edmark, and Lundqvist 2012; Eger 2010), and the overwhelming evidence for a link between anti-immigrant sentiments and low welfare support on the micro level (see above), we conclude that the debate on the effects of immigration is anything but solved, but we still see stronger evidence for conflict arguments.
Before concluding this reply, we would like to address two technical points, which are not relevant for the substantive conclusion, but we believe they are important for the wider research community working with similar modeling approaches. First, Auspurg and colleagues claim to have improved our specification by using a different de-meaning procedure for the within-between decomposition (see Part B in their online supplement). When working with pooled cross-sectional data, a decomposition of context-level effects into their within- and between-components requires one to subtract the unit-specific mean of context-level variables from the original variables and to then enter this mean into the equation (compare Fairbrother 2014; Schmidt-Catran, Fairbrother, and Andreß 2019). The following is a simple version of this model:
This is a three-level model with individuals i nested in time points t nested in contextual units j (e.g., regions or countries). x is an individual-level variable and cannot be decomposed, because data at the individual-level are cross-sectional. The effect of variable z, a context-level variable, is decomposed into its within- and between-effects, that is, a longitudinal and a cross-sectional component. This is possible because the context-level data have the structure of a panel dataset, with repeated measures of the contextual units j. In this sense, the within-effect is identical to estimates from a fixed-effects panel model; they both rely only on variation within units over time.
Auspurg and colleagues argue that one should calculate the mean and the subtraction in the raw context-level data and then merge it with the individual-level data. We merged the data first and then performed the calculations necessary for the decomposition. We believe it is correct to do so. This becomes clear when thinking about the analogy to a fixed-effects panel data model. The fixed-effects model eliminates any between-unit variation from the data by subtracting the mean of each variable from the original values, just as in the equation above. Mathematically, this means the mean of the term
Second, in the section on robustness checks, Auspurg and colleagues argue that it would be a better test of our hypothesis to identify the interaction of the share of foreigners and the unemployment rate by using only between-variation from the unemployment rate. From a theoretical point of view, this argument seems plausible because we expect the negative effect of immigration to be particularly strong in regions with (continuous) economic problems. However, from a causal identification standpoint, Giesselmann and Schmidt-Catran (2018) argue that using only within-variation is the better test—following the basic causal identification logic of fixed-effects estimation. For a detailed discussion of these technical issues, we refer readers to Giesselmann and Schmidt-Catran (2018).
To summarize, we acknowledge that the explicit modeling of non-parallel time trends leads to a necessary correction of our initial results. While we agree with many of the methodological arguments brought forward by Auspurg and colleagues, we do not share their general picture of the literature and the classification of our initial and corrected results on Germany within it. In terms of future studies—also but not exclusively on immigration effects on support for welfare—we believe that using within-context variation from pooled cross-sectional data is a promising and increasingly used strategy (see the discussion in Schmidt-Catran et al. 2019), but it shares all its assumptions with the standard fixed-effects approach. The technique is an important improvement over reliance on pure cross-sectional variation, but this does not mean it can be applied without caution. Any assumption implicit to the fixed-effects model is also implicit to the within-between decomposition of pooled cross-sectional multilevel data. Auspurg and colleagues’ contribution demonstrates this very well, and we hope their statistical argument will be noted by the wider research community using such models.
