These Are Not the Effects You Are Looking for: Causality and the Within-/Between-Persons Distinction in Longitudinal Data Analysis

Helpful further readings

Maybe because of psychology’s fraught relationship with causality (Grosz et al., 2020), the literature on the many models discussed in the field—such as varieties of change-score models, cross-lagged models, and latent-curve models—is unfortunately not always transparent with respect to the assumptions under which these models can successfully identify causal effects. However, more recently, researchers have tried to bridge the gap between longitudinal data modeling in psychology and causal inference.

Gische et al. (2021) introduced graphical causal models for researchers familiar with structural equation modeling and the cross-lagged panel design, and Voelkle et al. (2018) provided a more general discussion of the role of time for understanding psychological mechanisms, which they quite explicitly described as a sequence of causal effects. Usami et al. (2019) provided a discussion of the causal assumptions underlying popular models. Both Andersen (2021) and Lüdtke and Robitzsch (2022) discussed different classes of longitudinal models with respect to the conditions under which they recover the (cross-lagged) causal effects of interests and conditions under which they will result in equivalent results. Finally, Zyphur et al. (2020) developed a comprehensive general cross-lagged panel model as a generic approach to translate assumptions into a statistical model.

Of course, other fields have also tackled the issue of causal inference with longitudinal data. For example, the sociologists Elwert and Pfeffer (2019) developed an approach that uses future values of the independent variable to detect and reduce omitted variable bias. In epidemiology, the particularly promising approach of marginal-structural models (Cole & Hernán, 2008; Robins et al., 2000) has been developed. These models implement a multistep estimation procedure to control for time-varying confounding variables (Williamson & Ravani, 2017). The promise of such models for causal inference in psychology, however, has not yet been well recognized (Lüdtke & Robitzsch, 2020; Usami, 2020). Thoemmes and Ong (2016) provided an introduction to marginal-structural models in combination with inverse probability weighting as a means for third-variable adjustment in longitudinal data, including annotated SPSS and R code for psychologists. The tutorial by Bray et al. (2006) showcased an implementation in SAS and highlighted how this method can, unlike other common methods, successfully adjust for time-varying confounders. Finally, VanderWeele et al. (2020) developed a comprehensive template for so-called outcome-wide longitudinal designs in which the goal is to identify the causal effects of an independent variable on a number of outcome variables and longitudinal data are leveraged to reduce concerns about confounding.

Additional advantages of longitudinal data

Aside from causal identification in the narrow sense (getting rid of confounding), which often focuses on average effects, longitudinal data may also enhance causal inference for other reasons. First, longitudinal data can improve the understanding of how causal effects unfold over time (Voelkle et al., 2018). Second, they may provide the means to actually estimate individual-level causal effects. Causal effects may vary between individuals, and researchers can take into account such between-persons variability of causal effects with longitudinal data. The optimal approach to identify such effects are experiments in which one observes individuals repeatedly in different experimental conditions (see Fine Point 2.1, Hernán & Robins, 2020, p. 16), and there have been some recent methodological developments to obtain a better causal estimate with this type of design (Schmiedek & Neubauer, 2020). Doing this with observational longitudinal data once again requires more and stronger assumptions, and this is an important avenue for future methodological work.

Setting the analysis goal

Researchers should start by explicitly spelling out the theoretical estimand of interest in precise terms that exist outside of any statistical model (Lundberg et al., 2020). At this point, it may become clearer whether the research question targets causal quantities—but even noncausal endeavors, such as “description of developmental trajectories,” require conceptual clarity. This estimand in combination with the additional assumptions researchers are (or are not) willing to make determine which research design is appropriate, be it experimental or nonexperimental, cross-sectional or longitudinal, needing many time points or not.

What does such a well-defined estimand look like? Psychologists like to make claims about broad concepts (Yarkoni, 2020) and to address broad research questions (“What is the interplay between talkativeness and happiness?”). However, from a causal-inference perspective, things need to be broken down and taken more slowly (see also Rohrer et al., 2022). For example, a more tractable research question may concern the effect of being continuously talkative (vs. continuously untalkative) for a certain defined amount of time on well-being immediately after the episode. Formalization, for example with the help of the potential-outcomes model, makes it explicit that causal effects are defined by contrasts of specific treatments on specific outcomes. Treatments may be time-varying—there are many different sequences of talkativeness and untalkativeness that one could contrast to learn something about the effects of talkativeness on happiness (for an introduction to time-varying treatments, see Hernán & Robins, 2020, Chapter 19)—and outcomes can be evaluated at different points in time.

Thus, there is no such thing as “the” effect of talkativeness on happiness; there are many different possible theoretical estimands that may all be deemed informative with respect to the overarching question of whether and how talkativeness affects happiness. This nuance may get lost if researchers simply apply an out-of-the-box model, which may target a different estimand than the one they actually have in mind. Furthermore, insufficient clarity about estimands may lead to ostensible contradictions between empirical studies that in reality target different estimands.

Once researchers try to be more precise about the causal effects they have in mind, they may run into deeper issues. Theories in psychology are often quite underspecified—a topic that was addressed in a recent special issue of the journal Perspectives on Psychological Science (Volume 16, Issue 4, July 2021)—and may thus make only vague predictions about which causal effects are to be expected. But even if theories were more precise, causal effects involving psychological variables pose conceptual obstacles. Although these are neither unique nor central to the within/between distinction and thus outside of the focus of the present article, we provide more details in Box 4 for interested readers.

Identification strategy and design parameters

Once researchers have settled on an estimand, they can start thinking about appropriate identification strategies. Accessible articles provide some guidance on this step (Foster, 2010a; Grosz et al., 2020). Considerations may include whether a sufficiently targeted intervention is available to manipulate talkativeness (but see Eronen, 2020) and thus whether an experiment is plausible, whether a suitable natural experiment may exist (e.g., a situation that affects talkativeness in a plausibly random manner), and which time-invariant or time-varying confounders are deemed relevant.

If within-persons data turn out to be a productive way forward—for example, because time-invariant confounders are deemed particularly relevant and because researchers can assume that there is relevant within-persons variability in the independent variable of interest—the causal angle can clarify specific design parameters. Consideration of potential time-varying confounders tells researchers what needs to be measured. Consideration of the precise definition of the causal effect of interest tells researchers which time lag between assessments is sensible.

Discussions of the appropriate time lag in psychology often focus on attempts to uncover the true underlying dynamic system (Haslbeck & Ryan, 2021), which is, of course, unknown. Hence, one might conclude that the narrowest possible sampling is desirable because it still allows one to estimate effects with a wider lag (e.g., in the crudest case, one might just drop the measurement points in between). In practice, when it comes to time lags, pragmatic concerns need to be considered as well. High-frequency sampling can overburden participants, and there is a very real possibility that the assessment interferes with the causal system of interest. Self-reporting positive affect 100 times a day may influence mood; filling out a personality questionnaire over and over again may change the way individuals answer the items. Thus, the smallest possible time lag is not always advisable. But if researchers decide that they want to investigate a relatively well-defined specific causal effect, such as “the immediate effect of picking up one’s smartphone on well-being” or “the effect of cumulative smartphone usage over the course of a day on well-being on the next day,” the research question already implies how data need to be collected. ⁴

Statistical estimation

If the theoretical estimand is set, the aim of the statistical analysis is to provide an actual empirical estimate. We have already extensively referred to the longitudinal modeling literature above and will thus just briefly emphasize a central concern. Psychological researchers have often relied on out-of-the-box longitudinal models such as cross-lagged panel models, which could, in principle, be applied to any pair of variables. Such default solutions have multiple shortcomings. First, we note that currently, the psychological literature on within-persons associations is not well integrated with the causal-inference literature—thus, for at least some of the out-of-the-box solutions, it is unclear or at least untransparent which causal effect is targeted by the analysis. Second, the causal webs linking different sets of variables can look very different, and a model that is not tailored to the specific underlying causal web cannot recover the causal effects of interest. Third, many published implementations of these models pay less attention to including measured (time-invariant and time-varying) confounders, further limiting the chances that the causal effects of interest will be recovered.

Interpreting model results

Finally, once the model has been estimated, how does one interpret it? In the psychological literature, model coefficients associated with particular paths are often treated as the relevant analysis output. But even if researchers are lucky and their coefficients correctly identify the causal processes of interest, they may still not provide a straightforward answer to their (causal) research question. For example, the model-implied effects of X at a given point in time on Y at a given point in time may vary between individuals because of either systematic heterogeneity (e.g., interactions) or nonlinearity (for a discussion of why in nonlinear models, “everything interacts,” see Rohrer & Arslan, 2021). Furthermore, in a cross-lagged panel model, the effect of a predictor on the outcome at a later time point can be the sum of both direct and indirect effects (Lüdtke & Robitzsch, 2022) so that multiple coefficients need to be added up when contrasting different states of the world.

Here again, causal thinking can clarify how to summarize effects in complex models in an interpretable manner. We can once again consider a hypothetical intervention and use the model to predict how it would affect individuals’ outcomes at a point in time we consider relevant. Gische et al. (2021) demonstrated how to work with such hypothetical interventions in the context of cross-lagged panel models and how to calculate both average and person-specific effects.

The general framework for using models to make predictions about the effects of various interventions are so-called marginal effects. Marginal effects have received comparatively little attention in psychology outside of methods journals; they are more common in, for example, sociology (e.g., Mize et al., 2019), possibly because the statistical software Stata, popular in that field, makes calculating them quite easy (Williams, 2012). However, there are now packages available that allow researchers to calculate marginal effects in R using structural equation models (Mayer, 2019; Mayer et al., 2016) and a vast number of other model classes, including multilevel models (Arel-Bundock, 2022; Lenth, 2022). As far as we know, no comprehensive primer to marginal effects for psychologists has been published to date, but recent blog posts have tried to provide a gentle introduction (Heiss, 2022; Rohrer, 2022).

Should researchers just do description instead?

Having read so far, readers may feel that inferring causal effects from (nonexperimental) within-persons data in an overwhelming task and may instead prefer a “descriptive” approach. Indeed, many longitudinal analyses claim to be descriptive in nature, although that term may be used in an ambiguous manner. This may partly be a strategic move to avoid the heightened scrutiny that results from overtly causal claims (Alvarez-Vargas et al., 2020; Grosz et al., 2020). We do believe that descriptive research is currently undervalued in psychology (see e.g., Scheel et al., 2020). But many models in psychology are too complex to produce good descriptions (Foster, 2010b), and this holds true for longitudinal models in which the explanation for how coefficients behave quickly turns opaque.

An actually informative descriptive analysis should involve much more basic description than researchers routinely encounter in studies analyzing longitudinal data. For example, a fruitful first step to describe associations in longitudinal data may consist of a fixed-effects model in its most basic specification or the equivalent multilevel model with within-persons centering. This tells the strength of the contemporaneous association between the variables after removing stable between-persons differences in the level of the predictor and the outcome. With sufficient data points per individual, researchers may even simply calculate the bivariate association for every individual in isolation. As we discussed earlier, any association in such analyses may still be confounded and thus does not necessarily provide a convincing causal estimate, but at least researchers will have narrowed down the range of confounders while sticking with estimates that still have a straightforward descriptive interpretation. Reporting results from such analyses before moving on to more complex models also mirrors established practices in cross-sectional studies in which authors routinely report bivariate correlations before moving on to more complex regression analysis.

In contrast, it is not easy to apply the “standard” longitudinal models for descriptive purposes. The literature is filled with many related “state-of-the-art” longitudinal models (e.g., the autoregressive-latent-trajectory model, Bollen & Curran, 2004; the random-intercept cross-lagged panel model, Hamaker et al., 2015; the stable trait/autoregressive trait/state model, Kenny & Zautra, 1995; the dual-change score model, McArdle & Hamagami, 2001), and these in turn can usually be specified in multiple ways. Assuming researchers chose the right model that correctly reflects the data-generating mechanism, they could elegantly capture the underlying causal within-persons dynamics. But in reality, researchers do not know which model generated the observed data, and presented with a daunting number of different models and little guidance on which nuances matter and which do not, they may resort to the standard that is accepted in the field—and that might not be optimal, as is known from the story of cross-lagged panel models (Hamaker et al., 2015). Trying to uncover the complete causal dynamics of a system is a more ambitious task than identifying a specific causal effect; once researchers fail to uncover the complete dynamics, the interpretation of any specific component of the model becomes questionable.

Putting causal inference upfront, researchers are still confronted with a challenging task, but one that is potentially more tractable because there is at least a clear circumscribed analysis goal: recovery of a specific causal effect. This also opens the possibility to use available experimental evidence as a benchmark to evaluate observational longitudinal analysis—if a longitudinal model implies certain effects that contradict existing evidence from intervention studies targeting similar cause and effect, this can at least be taken as a warning sign (for an implementation of this logic, see Wan et al., 2021). The choice of the data-analytic model matters only insofar as it should map onto the assumptions about the underlying causal web that researchers are willing to make. These assumptions will often be strong and potentially unrealistic—there is no free lunch—but at least researchers are actually tackling the question of interest.

Consider, for example, the debate surrounding the age trajectory of happiness (Galambos et al., 2020). This actually seems to be one of the easier questions one could answer with longitudinal data, yet it has spawned a bloated literature and lots of confusion about how to specify the model. If one tackles the problem from a causal-inference perspective, as demonstrated by Kratz and Brüderl (2021), it becomes clear that some analytic decisions are just wrong (e.g., statistical adjustment for mediators, which makes sense only if researchers are trying to address a different research question), whereas others hinge on additional assumptions (e.g., about the existence and shape of period and cohort effects). This does not mean that the debate is automatically settled, but at least one can pinpoint where exactly analysts disagree and how to make progress on the research question.

We believe that a better understanding of causal inference and how it can be enhanced with the help of within-persons data has the potential to clarify other debates in psychological research as well, resulting in an overall improvement of the quality of our inferences.