The impact of ignoring lags on developmental science: A re-analysis of meta-analyses using lag as moderator

Abstract

Longitudinal studies provide developmental science with invaluable information about how variables and the associations between variables change across time, but typically give limited attention to the length of time over which that change occurs. The present study re-analyzed data from previously published meta-analyses of longitudinal data across a broad range of developmental science to ascertain how lag may have impacted coefficients of stability (k_{meta-analyses} = 6, k_studies = 157) and prediction (k_{meta-analyses} = 15, k_studies = 270). We additionally analyzed how average participant age interacts with lag to test how the impact of lag might change across the lifespan. Findings indicate that conventional lags (e.g., 6 months, 12 months) were used at extremely high rates: More than 75% of lags were selected based on convention. Linear and nonlinear models indicated that lag moderated stability and predictive associations, although the significance, magnitude, and direction of this impact changed depending on the phenomenon under investigation. Average participant age interacted with lag in certain cases, providing a possibility for more time-specific developmental theory. However, these results should not be considered conclusive due to the high number of conventional lags in our sample, which likely restricted both variability in lags and the length of those lags. Future longitudinal studies should measure phenomena at varying lags, and future meta-analysts should consider both lag and average participant age when synthesizing longitudinal research. Both practices would enable developmental science to determine the interval over which a phenomenon occurs and facilitate advancements in developmental theory.

Keywords

Longitudinal study meta-analysis development temporal design methodology

Developmental science inherently involves the study of change across the lifespan. Longitudinal data are, therefore, essential to evaluate developmental change. Card and Little (2007) found that 41% of studies published in high-impact developmental journals are longitudinal. Despite the importance of longitudinal studies, little attention has been paid to lag, the time between longitudinal measurements. Multiple sources assert that lag should be selected for theoretical reasons wherever possible (Collins, 2006; Little, 2013). Incorrect choice of lag length may impact the magnitude and even the direction of the results found in longitudinal studies (Pelz & Lew, 1970; Selig et al., 2012). Choice of lag, therefore, is foundational to developmental research.

Researchers should choose lag based on theory regarding the anticipated rate of change and functional form of a change process (Collins, 2006). However, developmental theories are often silent about how they expect a change process to unfold (Card, 2019). That said, it is widely theorized that different processes will occur at different ages and rates across the lifespan (Collins, 2006). Some developmental periods may be particularly sensitive for certain phenomena (Woodard & Pollak, 2020). Some research suggests that optimal lag, defined as the lag required to yield the maximum effect of X predicting Y at a later time while statistically controlling for initial values of Y, may correspond with when a change process occurs (Dormann & Griffin, 2015; Selig et al., 2012). Such results could empirically inform us when a process changes (Card, 2019). Understanding how developmental processes vary across time would enable researchers to create more complex theories and design more precise studies, leading to more accurate developmental research.

Impact of Lag

In longitudinal research, measurements are taken of the same individuals at two or more time points. This design allows for the quantification of two key parameters: stability and cross-lag prediction. Stability refers to the association between initial and later levels of the same variable. Its examination allows researchers to ascertain whether interindividual differences in a phenomenon are consistent (stability) or changing (instability) over time (Bornstein et al., 2017). Understanding whether a phenomenon demonstrates stability or instability also allows inferences to be drawn about differences between various developmental periods (Card, 2019). For example, periods where change is rapid may show lower levels of stability, whereas higher levels of stability might characterize those with less rapid change.

Cross-lag prediction involves using initial levels of one variable to predict later levels of another, separate variable. This is a common method for approximating causality, where experimentally manipulating participants is impossible or unethical (Little et al., 2009). In many cases, researchers will statistically control for initial levels of the dependent variable so that the independent variable predicts the change (i.e., instability) in the dependent variable across time. Cross-lag predictions provide crucial information about developmental change, particularly regarding ages or developmental periods where causal processes occur more strongly or rapidly than others. In sum, both stability and cross-lag prediction provide researchers with essential information about developmental change, hence the high frequency of longitudinal studies noted above.

Lag is likely to differentially impact stability and cross-lag prediction. Stability is prone to decrease as lag length increases. Little (2013) illustrated how decay in stability coefficients is expected over longer lags where the change process follows a simplex structure. Specifically, stability is expected to follow a pattern of exponential decrease equivalent to $r_{t} = {(r_{1})}^{t}$ , where $r_{t}$ represents predicted stability over t units of time, $r_{1}$ indicates stability over one unit of time, and $t$ is equivalent to the lag under examination. For example, where stability is found to be .50 over a 1-month lag, stability over 2 months will be equivalent to $r_{2} = {(. 50)}^{2}$ = .25, and so forth. Simplex processes assume that the rate of change examined is steady. Therefore, simplex structures might be expected to apply to some developmental periods more than to others (e.g., those where change is expected to be consistent versus those when change is either accelerating or decelerating). Other factors external to an individual might also render simplex structures inapplicable, such as transition periods.

Selecting a suboptimal lag to measure a causal process is also likely to influence coefficients of cross-lag prediction. Given conceptual reasons to expect variation in stability and cross-lag prediction depending on lag, it might be questioned whether some lags could be thought of as optimal, that is, providing the strongest magnitude of effect for a particular phenomenon. In a series of simulations examining the impact of lag, Pelz and Lew (1970) found that cross-lag coefficients are likely to approach zero at longer lags and that the direction of cross-lag coefficients may even reverse. Cole and Maxwell (2003) found an even more complex situation, where incorrect lags (relative to a known “correct” lag) could either positively or negatively bias cross-lag coefficients. Specifically, where a variable is highly stable across time, coefficients of cross-lag prediction are likely to be overestimated if too long of a lag is chosen. Conversely, where levels of a variable demonstrate low stability, cross-lag coefficients will be underestimated when lags are too long. These findings were later supported by results from Cain et al. (2018) which indicate that power was lower where lag was mis-specified. Studies examining the impact of lag using primary data also indicate that selecting the incorrect lag to evaluate a predictive association will bias results (Collins & Graham, 2002; Selig et al., 2012). However, these studies are limited in number, making the actual impact that incorrect lags have had on developmental science unclear. It is further unclear how the effect of lag on cross-lag associations might change depending on participant age.

Optimal Lags

In the previous section, we raised the concept of an optimal lag, and described previous evidence of the impact of selecting suboptimal lags. Since every phenomenon in developmental science will take some amount of time to unfold, it can be argued that an optimal lag likely exists over which to measure it. This is directly related to the theory that every cause will take some time to effect (Gollob & Reichardt, 1987). Recognizing the challenges of inferring causality from non-experimental designs, one can still consider optimal lags without necessarily specifying causal relations.

Dormann and Griffin (2015) defined optimal lag as “the lag that is required to yield the maximum effect of X predicting Y at a later time while statistically controlling for prior values of Y.” This definition is particularly applicable when examining bivariate or multivariate relations involving predictor and outcome variables in correlational studies. In such instances, the maximum effect between predictor and outcome indicates the correct lag at which to measure an association. This definition is particularly useful where there are discrete and non-recurring instances of predictor and outcome or when a researcher desires to make broad statements about the timeframe over which a process unfolds (Selig et al., 2012). Statistically, it aligns most with a lagged regression or cross-lag panel model framework.

For example, if a researcher were trying to examine the effect of ibuprofen on headaches, they should not measure the association over a 2-week lag. They would find no effect and conclude that ibuprofen does not reduce headaches. To assess the effect of ibuprofen, a researcher would need to measure at approximately a 10-min lag. In this example, we might expect a peak in the efficacy of ibuprofen somewhere between 30 min and 2 hr after taking it. However, not all developmental processes might have this simple quadratic function, like the effect of ibuprofen. For this reason, we need to explore the impact of lag as both linear and nonlinear functions.

In addition to this first definition of optimal lag, a second definition exists. Specifically, optimal lag is the range of lags required to assess changes in a phenomenon accurately. It aligns with recommendations made in Collins (2006), which states that number of measurements and length of lags should correspond with hypothesized functional form. If change is not examined at the correct lag, it may be impossible to assess the functional form that best fits the phenomenon under examination. This, in turn, would make it impossible to understand the development of the phenomenon over time.

Lag and Age

The impact of lag becomes more complex with the addition of age. A foundational concept in developmental science is that individuals change as they age. This change can occur in terms of mean level changes; for example, intellectual ability increases throughout childhood development. Another potential change is that predictive relations are stronger at some ages than others; for example, parenting practices might be more impactful on child behavior at some ages than others. Finally, a less-often considered form of developmental change is that processes might occur faster at some ages. For example, peer rejection or victimization might have a more immediate impact on early adolescents than on other ages. This type of developmental change in the speed of processes is rarely theorized or examined but is detectable by considering the interaction of lag and age.

Methods of Accounting for Lag

Although most developmental research appears to give little attention to the potential impact of lag on findings, some methodologies do consider the impact of lag. Some researchers advocate including different lags within a study (either within or between participants) to understand a variable’s predictive effects, a method commonly called variable-lag design (McArdle & Woodcock, 1997). A different but related approach recommends increasing the number of measurements within the entire sample (Collins & Graham, 2002; Dormann & Griffin, 2015; Timmons & Preacher, 2015). However, there are significant drawbacks to using either of these approaches. Researchers may not always have the financial resources to collect data at the desired frequency. Additional time points also increase participant burden. Alternatively, there may be logistical reasons why collecting data at the desired rate is impossible. For example, school administrators may only permit a researcher to collect data at certain times.

Although most longitudinal studies report a single lag, participants often complete measurement occasions at varying times in practice. Selig and colleagues (2012) proposed the lag as moderator (LAM) approach, which capitalizes on naturally occurring inter-individual variability in lag to model the impact of different lags between two measurement occasions. They illustrated this approach using existing variation in the Early Head Start Research and Evaluation Study, finding significant linear and exponential lag moderation. However, two potential drawbacks exist for this approach. First, a primary study’s naturally occurring range of lags may not be wide enough to detect moderation. Second, it relies on researchers recording the required data (i.e., exact lag length for each participant). Neither of these conditions will likely be met consistently, which potentially limits the use of LAM.

Card (2019) proposed lag as moderator meta-analysis (LAMMA), which uses between-study variability to assess the impact of lag on longitudinal effect sizes. LAMMA has several advantages over primary longitudinal studies: It allows for increased variability in lag length, includes larger sample sizes and accordingly increased statistical power, and consists of a more heterogeneous overall sample, leading to greater potential for generalizability. It is also not constrained by the same logistical considerations as primary longitudinal studies. Given these considerations, using LAMMA provides a unique opportunity to examine the impact of lag on developmental science. LAMMA may provide researchers with empirical reasons for choosing a particular lag in cases where theory is nonspecific about when a phenomenon will occur, or the functional form-related change processes are likely to follow. This is particularly true if the lag moderation employed in LAMMA were combined with analyses examining average participant age moderation and age interactions with lag.

Present Study

Despite methods to account for lag, it is unclear that primary studies have considered lag as an essential design feature of longitudinal studies. It is probable that lack of consideration of lag has impacted results in developmental science: Longitudinal studies of similar phenomena often pick arbitrary (e.g., 1 year) lags. It is unclear what, if any, effect this may have had on research findings. However, developmental scientists will need to understand the impact of lag in conjunction with age as a fundamental step toward solving this problem. Therefore, the present study has two aims: First, to reanalyze previously published meta-analyses in developmental science using the LAMMA framework to assess the impact of lag on effect sizes related to a wide range of developmental phenomena. We will do this by meta-analytically assessing the impact of lag length as a moderator on longitudinal effect sizes. Second, to evaluate the effect of lag in conjunction with average age from reanalysis of these multiple meta-analyses to understand this interaction across numerous, different phenomena. We then use information from these aims to make recommendations about how we, as a field, might take steps toward more thoughtful consideration of lag when planning and drawing conclusions from developmental research.

Method

Literature Search and Study Selection

We searched PsycINFO using the keywords “longitudinal study” or “prospective study” and “meta-analysis” or “quantitative review.” This literature search was conducted in January 2019. Studies were deemed eligible for inclusion if they were meta-analyses of longitudinal data, included study-specific effect sizes (stability or cross-lag) and lag information, reported lag as chronological time, and examined topics that could be broadly categorized as developmental. We chose to survey meta-analyses of longitudinal studies because they granted us the ability to broadly assess how lag may be impacting a variety of areas of study in developmental science. This would not have been possible if we had focused solely on a single topic area. Developmental meta-analyses were defined as investigating change in a normative (i.e., not hospitalized or incarcerated) population. This search strategy yielded 16 meta-analyses for inclusion (Figure 1). Given that this study was a secondary analysis, it did not require Institutional Review Board approval.

Figure 1.

PRISMA Diagram of the Literature Search Strategy.

Coding of Studies

Several descriptive characteristics were coded from each meta-analysis, including author, title, year published, and journal title, as well as whether the meta-analysis synthesized longitudinal studies. This last characteristic was not only an inclusion criterion, but allowed us to assess how frequently meta-analyses of longitudinal data are conducted. In addition, we coded stability, cross-lag, or both from each meta-analysis as well as lag length in months. This information was used to conduct LAMMA analyses. Finally, we coded the sample size, which was used for the differential weighting of studies, and average participant age at Time 1. Complete information (including author, title, year, and so forth) from all the studies was coded twice, first in June 2019 and second in January 2020. Both authors participated in the coding, and consulted with one another where questions or disagreement existed. Reliability was high (97% agreement) between the two instances of coding (Card, 2012).

Data Analytic Strategy

We completed the following steps for each included meta-analysis and data from each meta-analysis were analyzed separately. Stability and cross-lag were calculated as a correlation (Pearson’s r) between Time 1 and Time 2. Prior to analysis, effect sizes were transformed to Fisher’s Zr scores (Card, 2012). In all other ways, data were left as provided by the authors of the original meta-analyses. For example, if the author controlled for initial levels of a variable at Time 1 then those steps remained in place. This ensured that any coding decisions that they felt necessary given their expertise related to the phenomenon they were studying remained in place.

LAMMA analyses were conducted using fixed effects models due to the small sample size of some of our included meta-analyses. Results for mixed effects models are shown in the Supplemental Material. We assessed linear, quadratic, and exponential models of the impact of lag as each of these functional forms has been shown to model lag moderation in past research (Card, 2019; Selig et al., 2012).

Linear Models

Linear models were assessed as described in Card (2019), by differentially weighting studies using sample size and then employing lag as a moderator, as illustrated by the equation below:

{ES}_{i} = b_{0} + b_{1} {Lag}_{i} + ε_{i}

(1)

The coefficients b₀, which represents the intercept, and b₁, which represents the predicted change (slope) in stability or cross-lag (ES_i) per unit change in Lag_i, are estimated from this model. Error terms are captured in the percent heterogeneity unaccounted for by the model. The magnitude of the $b_{1}$ coefficient depends upon the unit in which lag is measured. For example, where lag is given in months, the output of the model will indicate the expected amount of change per additional month of lag (whereas if lag length was measured in years, this value would be 12 times larger, though it would have the same direction and level of statistical significance). All our analyses used months to capture change in the smallest unit we were reasonably able to discern. Following this step, average participant age and age interacting with lag were introduced as additional moderating variables. Average participant age was centered prior to analysis.

Quadratic Functional Form

Quadratic models of lag may be assessed using the following predictive equation (Card, 2019):

{ES}_{i} = b_{0} + b_{1} ({Lag}_{i} - \bar{Lag}) + b_{2} {({Lag}_{i} - \bar{Lag})}^{2} + ε_{i}

(2)

To analyze the quadratic models, we centered lag $({Lag}_{i} - \bar{Lag})$ to reduce nonessential collinearity. This term was then squared to create quadratic lag ${({Lag}_{i} - \bar{Lag})}^{2}$ . Following these steps, we regressed effect sizes (Zr) onto centered and quadratic lag, respectively. This model yielded three pieces of information. These included the intercept $b_{0}$ , which represents the predicted value of stability or cross-lag where lag is equal to the average lag. We also estimated parameters for centered lag $b_{1}$ and quadratic lag $b_{2}$ . To assess the interaction of average participant age with quadratic lag, average age was centered then multiplied, creating centered and quadratic lag by average age interaction terms.

Exponential Functional Form

As mentioned earlier, developmental processes do not necessarily operate like ibuprofen, and therefore quadratic functions might not best model all developmental phenomena. We derived methods of constructing this model using equations found in Selig et al. (2012): Longitudinal effect sizes (Zr) were transformed using a logarithmic (base 10) transformation. To run an exponential LAMMA model, we applied a log(10) transformation to the dependent variable and then ran standard linear meta-analysis regression.

{ES}_{i} = b_{0} + b_{1} \times * \log ({Lag}_{i}) + ε_{i}

(3)

After being altered by the log transformation, any dataset that follows an exponential functional form will appear linear, and will therefore test as significant in a linear analysis. As with a linear model, two main coefficients are yielded from an exponential model. These include the intercept $b_{0},$ and the coefficient $b_{1}$ , which is applied to lag in the exponent. Predicted values yielded from this model were back-transformed using an exponential function, which moved them back into their original unit. Average participant age interaction analyses were conducted by multiplying a centered age term with centered lag. All analyses were conducted using the metafor package in R (Viechtbauer, 2010). For more information on how we ran exponential analyses please see section A of the Supplemental Material. These materials also provide information on how LAMMA analyses work generally.

Results

A total of 1,167 potential meta-analyses were reviewed for inclusion. Of these, 1,151 were excluded for the following reasons: 971 were not meta-analyses of longitudinal data, 251 did not utilize developmental data, and 110 did not report study-specific longitudinal effect sizes or lag information. A final sample of 16 meta-analyses was included in this study. Of these, 6 meta-analyses (k_studies = 157, N = 45,443) met inclusion criteria for stability, while 13 (k_studies = 267, N = 190,082) met criteria for cross-lag prediction (Figure 1). Two meta-analyses in our cross-lag sample investigated two different cross-lag associations. These were analyzed individually, yielding a final sample of 15 meta-analyses investigating cross-lag predictive associations. Meta-analyses from the stability sample examined phenomena such as peer victimization, attachment, and school competence (Table 1). The cross-lag sample investigated predictive relationships between constructs, including emotional problems predicting school attainment, internalizing problems predicting peer victimization, and maltreatment predicting antisocial behavior (Table 2). Please see the Supplemental Material for a reference list of all the included meta-analyses.

Table 1.

Stability Models.

First author (year)	k	N	Phenomenon	M _age	% CLL	Linear model	Quadratic model		Exponential model
First author (year)	k	N	Phenomenon	M _age	% CLL	Linear β	Linear β	Quadratic β	Log β
Fraley (2002)	27	1,410	Attachment	1.0	81	−.0002	−.0055	.0004**	.0002
La Paro (2000)	78	8,908	School competence	4.5	100	.0027	.0023	0	.0031*
Larzelere (2018)	13	24,009	Externalizing problems	4.4	100	−.0049***	−.0027***	.0002***	−.0070***
Pouwles (2016)	18	6,254	Peer victimization	10.9	78	−.0087***	−.0030	−.0004*	−.0099***
Smith (2016)	11	1,758	Depression	28.8	18	−.0023***	−.0131***	.0002	−.0026***
Smith (2018)	10	3,104	Anxiety	23.9	50	−.0032***	.0021	−.0001***	−.0038***

Note. *p < .05, **p < .01, ***p < .001. Phenomenon = constructs at T1 predicting constructs at T2; M_age = mean age (years) at T1; CLL = convenience lag length; β = change per month. See the Supplemental Material for a reference list of all included meta-analyses.

Table 2.

Cross-Lag Predictive Models.

First author (year)	Phenomenon						Linear model	Quadratic model		Exponential model
First author (year)	k	N	T₁	T₂	M _age	% CLL	Linear β	Linear β	Quadratic β	Log β
Braga (2017)	21	15,504	Maltreatment	Antisocial behavior	13.7	90	−.001	−.001	0	−.002***
Colonnesi (2010)	18	580	Pointing	Language	1.2	11	.001	−.017	−.001
Huang (2015)	54	12,323	Attributional style	Depression	18.2	26	−.001	−.002	0	−.007***
Koletić (2017)	8	7,860	Explicit material	Behavior	14.7	100	−.005**	−.090***	.007***	−.004*
Larzelere (2018)	13	24,009	Spanking	EP	4.4	92	.001*	.001	0	.005***
Reijntjes (2010)	15	12,361	PV	IP	10.3	82	−.002	−.003	.001*	−.002
	11	10,307	IP	PV	10.3	87	.002	−.001	.001*	.024***
Reijntjes (2011)	10	5,825	PV	EP	8.2	80	.002	.004	−.001	−.004
	8	4,494	EP	PV	8.2	88	−.003	−.010***	.004***	−.008***
Riglin (2014)	26	25,088	Emotional support	School attainment	13.7	96	.003***	.003***	0	0
Smith (2016)	11	1,758	Perfectionism	Depression	28.8	18	.001*	.002	0	.003***
Smith (2018)	10	3,104	Perfectionism	Anxiety	23.9	60	−.001	−.010	−.001	−.007
Valentine (2004)	45	25,091	Self-belief	Academic achievement	11.5	77	.002	0
Wardle (2011)	20	19,621	Stress	Weight	35.5	80	−.001*	−.0001*	0

Note. *p < .05, **p < .01, ***p < .001.Phenomenon = constructs at T1 predicting constructs at T2; M_age = mean age (years) at T1; CLL = convenience lag length; β = change per month; IP = internalizing problems; EP = externalizing problems; PV = peer victimization. See the Supplemental Material for a reference list of all included meta-analyses.

Conventional Lag

To estimate the percentage of lags not chosen for theoretical reasons, we operationalized conventional lags as bi-yearly intervals (6, 12, 24 months, etc.). While it may be that some of these lags were chosen for theoretical reasons (or that other lags were selected based on convention), we reasoned that bi-yearly intervals are more likely to make sense societally (e.g., school semesters, funding year) than developmentally. Therefore, we considered it unlikely that intervals of half a year or a year have been frequently chosen following developmental theory and more likely that these intervals were selected based on convention.

Both the stability and cross-lag samples were found to contain high numbers of conventional lags. Most stability meta-analyses (k = 5) contained at least 50% conventional lags, and the majority (k = 4) contained at least 75% conventional lags. Two stability meta-analyses included only conventional lags (Table 1; Figure 2). In the cross-lag sample, most meta-analyses (k = 12) also contained at least 50% conventional lags. The majority (k = 11) also contained at least 75% conventional lags (Table 2; Figure 3).

Figure 2.

Predicted Change in Effect Size with Lag for Stability Models.

Figure 3.

Predicted Change in Effect Size with Lag for Cross-Lag Models.

LAMMA Analyses

Each meta-analysis in the stability (k = 6) and cross-lag samples (k = 15) was analyzed individually. We assessed linear models, then quadratic and exponential models. In the cross-lag sample, lag moderation models for one meta-analysis could not be run because all included studies used only one lag.

Stability Analyses

Linear Models

Results for the LAMMA model indicated that stability was lower in studies that used longer time lags. Particularly, results indicated a significant decrease in four cases out of the six cases examined. Fixed effects models including average participant age showed significance in four cases (k = 3 decrease, k = 1 increase; see Table 3). Average participant age terms consistently indicated that as average age increased effect sizes generally decrease.

Table 3.

Stability Models for Age Interacting with Lag.

First author (year)	Linear model				Quadratic model					Exponential model
First author (year)	N	Linear β	Age β	Lag × age β	Linear β	Quadratic β	Age β	LLag × age β	QLag × age β	Log β	Age β	Log lag × age β
La Paro (2000)	1,410	.1113	−.0067	.1340	.1645	−.4543	−.0212	.0608	.5403	.1301	.0065	.0570
Larzelere (2018)	8,908	−.0033***	−.0042	−.0016***	−.0024***	−.0003***	−.0164**	.0003	0	−.0048***	−.0015	−.0026***
Pouwles (2016)	24,009	.0170***	−.0462***	−.0064***	.0168***	.0002	−.0460***	−.0069***	0	.0012*	−.0394***	−.0051***
Smith (2016)	6,254	−.0109***	−.0002	.0004	−.0101	−.0008	−.0090	−.0007	.0001	−.0099***	.0003	.0003
Smith (2018)	1,758	−.0015**	−.0050*	.0002***	.0062**	−.0002*	.0022	.0002	0	−.0017**	−.0031	.0002***

Note. *p < .05, **p < .01, ***p < .001. β indicates change per month. See the Supplemental Material for a reference list of all included meta-analyses.

Quadratic Models

To reduce nonessential collinearity, lag and average participant age were centered prior to analysis. However, there were still several models that initially displayed problems. In these cases (k = 3), studies with outliers (extreme lags) were removed from the models to reduce skewness. Outliers were identified as being more than two standard deviations away from the mean.

Quadratic models for stability were less consistent than the linear models. Based on plots of predicted effect sizes for the quadratic models (Figure 2), half of the models predicted decrease in effect sizes over longer lags, while the rest predicted that effect sizes would increase until a certain time point before decreasing. Stability models indicated complex patterns of results for quadratic lag interactions with average participant age. Interaction terms were mixed in direction, though linear interactions were consistently stronger than quadratic interactions (Table 3). In most cases, average participant age and interaction terms did not yield statistically significant results.

Exponential Models

Analyses for stability (Table 1) indicated exponential decrease in the majority (k = 4) of cases. Models indicated significance in most cases of exponential lags (k = 5). The addition of average participant age and average age interacting with lag reversed the direction of the exponential models. These models indicated exponential decrease, but the addition of average participant age and average age interaction altered models so that increase was indicated in the majority of cases (Table 3).

Cross-Lag Analyses

Linear Models

The linear model results for the cross-lag meta-analyses were inconsistent in direction: Results were significant in five cases, with two exhibiting decreases and three having increased cross-lag effects with increased lag. Average participant age interaction terms varied in direction, with decrease indicated in two cases and increase indicated in three cases (Table 4).

Table 4.

Cross-Lag Prediction Models for Age Interacting with Lag.

First author (year)	Linear model				Quadratic model						Exponential model
First author (year)	N	Linear β	Age β	Lag × age β	Linear β	Quadratic β	Age β	LLag × age β	QLag × age β	Log β	Age β	Log lag × age β
Braga (2017)	15,504	−.0001	−.0006	.0001	−.0003	0	.0039	.0003	0	.0003	.0175***	.0003***
Huang (2015)	580	.0010	.0048*	.0003	−.0001	−.0003	.0069	.0004	0	.0085***	.0042	−.0004
Koetić (2017)	12,323	−.0663***	−.1806***	−.0569***	−.0894***	.0066***	−.0326	0	0	−.0079***	−.053***	0
Larzelere (2018)	7,860	.0007	.0049	0	.0003	0	.0024	−.0004	0	.0043***	.0216***	−.0008***
Reijntjes (2010)	24,009	−.0009	.0057	.0051***	−.0003	−.0002	.0091	.0053***	.0001	.0001	.0207***	.0163***
Reijntjes (2010)	12,361	.0018	.0110*	.0039***	−.0022	.0003	−.0166	.0028***	.0007***	.0212***	−.0456***	.0029*
Reijntjes (2011)	10,307	.0083	.0226	.0031	−.0009	.0008	.0206	−.0028	−.0007	.0001	.0406	.111
Riglin (2014)	5,825	.0028***	.0116*	−.0002	.0031***	0	.0168**	0	0	.0006	.0604***	−.0016***
Smith (2016)	4,494	.8950	.0015	.0006	−.0593	−.0623	−.0161	−.0121	.0628	.385***	.0005	−.0157**
Smith (2018)	25,088	−.0001	.0039	0	.0036	0	.0168	−.0005	0	−.0033***	.0185***	.0002***

Note. *p < .05, **p < .01, ***p < .001. β indicates change per month. See the Supplemental Material for a reference list of all included meta-analyses.

Quadratic Models

As with the quadratic models for stability, lag and average participant age were centered to reduce nonessential collinearity prior to analysis. For cross-lag, three models included outlier lags which were removed from models to reduce skewness. Outliers were again identified as being more than two standard deviations away from the mean.

The cross-lag models varied in similar ways to the stability models. While some models predicted an increase (k = 3), others predicted an increase followed by a decrease (k = 2). Still others predict that cross-lag coefficients will decrease (k = 3) or decrease before increasing (k = 5). Including average participant age interactions did not yield statistically significant results (Table 4).

Exponential Models

As with the stability models, cross-lag analyses (Table 2) also generally indicated a decreasing pattern (k = 9). Analyses were significant for the majority cases (k = 7). The addition of average participant age and average age interaction terms increased the number of analyses that were statistically significant (Table 4).

Discussion

The present re-analysis represents the first effort to systematically investigate the impact of lag on stability and cross-lag associations in longitudinal research in developmental science. It is also the first article to combine LAMMA with average participant age analyses to test a method that will help developmental researchers to detect average age differences in the rates of processes to better construct developmental theory. To accomplish this, we utilized a broad range of existing meta-analyses of various phenomena, which, taken together, provide a window into how lag may impact a broad field of developmental science. Results presented in this article may help guide how future primary longitudinal studies are designed so that developmental scientists begin to consider the questions of lag and how to choose lag across the lifespan more carefully. This article also serves as a call for more meta-analytic synthesis of longitudinal developmental research, capitalizing on existing between-study variation to advance understanding of the impact of lag in specific research areas.

Prevalence of Longitudinal Meta-Analyses

Our literature search indicated that longitudinal meta-analyses make up a relatively small proportion of the meta-analyses published yearly. As noted earlier, Card and Little (2007) surveyed five high-impact developmental journals to assess the prevalence of longitudinal studies in developmental science. These journals included Developmental Psychology, Child Development, Journal of Research on Adolescence, International Journal of Behavior Development, and Merrill-Palmer Quarterly. We reviewed our literature search to establish the prevalence of longitudinal meta-analyses in these same journals. Results from our search indicate that from 2013 to 2018, these journals collectively published 85 meta-analyses. Of these, 11.8% (k = 10) were meta-analyses of longitudinal data. This percentage is relatively small compared to 41% of longitudinal studies published yearly in these journals (Card & Little, 2007), indicating that relatively few meta-analyses take advantage of the wealth of longitudinal data available. There is enormous untapped potential to use meta-analysis to synthesize longitudinal data, which comprises almost half of published developmental research (Card & Little, 2007), to assess how developmental phenomena change over time. Furthermore, LAMMA provides a methodological approach allowing developmental scientists to synthesize these longitudinal studies to assess how lag may impact their field of study and create more appropriate designs for primary longitudinal studies. Combining LAMMA with average participant age interaction analyses may also help researchers better assess what lags might be appropriate for different age groups.

Impact on Stability

Models of the impact of lag consistently indicated a linear decrease in stability. These results provide empirical verification with previous literature predicting that stability coefficients will decrease at increased lag (Little, 2013). The pattern of results from quadratic stability models was less consistent across meta-analyses. Our models generally predicted that stability would either decrease or else increase then decrease. Exponential models also generally predicted decrease. Closer examination of the quadratic and exponential stability models indicates that the latter instances may be driven by either a lack of studies with short lags or a small number of studies with extremely long lags, further highlighting the need for future research to investigate lags not chosen by convention.

Including average participant age and average age interacting with lag improved some models but not others. Compatible with simplex structures (Little, 2013), the linear and exponential models, in particular, showed improvement. This may have occurred for two reasons. First, it could be that the phenomenon investigated in those models behaves similarly across the average ages included in those meta-analyses. Alternatively, it could also be the preponderance of conventional lags in these data clouded the results. Regardless, our results suggest that including average age interactions in future LAMMA analyses of stability may be valuable and support the development of theory.

These results indicate that stability generally, though not exclusively, follows a decay across longer time spans consistent with a simplex structure. Future meta-analyses of particular constructs might note when stability follows this pattern and at what ages or conditions they do not. The synthesis of stability estimates in developmental research also guides future primary work in that area (i.e., to study interindividual change in a phenomenon, researchers need to study it long enough for sufficient instability). In terms of guiding theory, the impact of lag on stability might reduce characterization of a quality as more trait or state by reaching different conclusions across different periods.

Impact of Lag on Predictive Relations

Universal conclusions about the impact of lag on predictive relationships are more complicated, given the variety of phenomena examined across the meta-analyses. These include diverse topics such as spanking predicting externalizing problems, pointing predicting language development, peer victimization predicting internalizing problems, and self-belief predicting academic achievement. Accordingly, the impact of lag varied across these meta-analyses, including linear increases and decreases and complex nonlinear patterns such as an increase followed by a decrease or a decrease followed by an increase. Multiple possible interpretations exist that may explain these models, and it is unlikely that any one interpretation could be applied across phenomena and development across the lifespan. A model showing a decrease might illustrate a case where a process occurred relatively quickly, and the effect diminished over the range of lags typically studied. Following this logic, it may be that any models showing an increase have not yet been measured at their actual lag interval because effect sizes should rise until the effect reaches a maximum.

Limitations and Future Directions

Although our results indicated that lag might impact the effects found in longitudinal studies, several limitations likely affected our results. First, the high number of conventional lags present in nearly every meta-analysis may have restricted the range of available data on lags, leading to attenuation of results. The limited variability in lags used by most studies also led to the inclusion of only a few longer lags, which may also have overly influenced results. Second, the majority of meta-analyses contained a scarcity of longer lags. This meant that some results might be influenced by a small number of high leverage, long lag, effect sizes and that we may not have sufficient data to accurately describe how many longitudinal processes operate across various lags. Third, many of the included meta-analyses contained relatively small numbers of studies. Results might be more precise in LAMMA analyses including larger numbers of studies. Finally, for the average participant age analyses, it may be that not a wide enough range of ages was included or that some age groups (e.g., college students) were over-sampled in some cases. For some phenomena, studying a limited age range might make theoretical sense, but for many others, there is a need for deliberate downward or upward extensions. So, although it must be acknowledged that findings of lag and the interaction of lag and age were inconsistent in the meta-analyses considered here, the potential limitations mentioned above might be the source of this inconsistency. With deliberate efforts to investigate a wider variety of lag in longitudinal studies over a broader age of the life course, we have an opportunity to gain a more nuanced understanding of stability and predictive relations in developmental science.

The above limitations indicate the next steps to be taken to understand how lag impacts developmental phenomena. Future researchers may wish to conduct studies utilizing shorter and longer lags than previously studied, and certainly not consistently rely on lags chosen based on convention and convenience. Researchers conducting primary studies could also record the precise date each participant’s data were collected, rather than simplifying the structure into a single time point (e.g., Time 1, Time 2, etc.). This would allow analyses of the impact of lag using the LAM method proposed by Selig et al. (2012). This basis of primary research not driven by conventional lags could then be combined using LAMMA to generalize across multiple studies and draw precise conclusions about the impact of lag on developmental phenomena. Taken together, these steps will allow researchers to conclusively understand the impact lag and use of conventional lags has had on developmental research.

Conclusion

The present study assessed the impact of lag on longitudinal studies in developmental science. Our first finding was the scarcity of synthesis of longitudinal studies relative to the common use of longitudinal designs in development science. We also found a high rate of conventional lags among studies. The overwhelming use of bi-yearly lags suggests that most researchers are not making deliberate design choices about the length of follow-up. Despite the preponderance of conventional lags, we found evidence for lag moderation of both stability and predictive effects across a range of developmental phenomena and instances of further moderation by average age to suggest age differences in the speed of these processes. We hope these results make clear the importance of designing primary studies selecting lags determined by theory or prior evidence or intentionally introducing variability in lag. For future research synthesis, we hope this work prompts greater use of meta-analysis to aggregate longitudinal results and the application of LAMMA models to investigate the interval over which to study any given process. Using this information, comparisons could be made across phenomena, allowing researchers to create more nuanced developmental theories about how, when, and how quickly processes occur.

Supplemental Material

sj-docx-1-jbd-10.1177_01650254241247155 – Supplemental material for The impact of ignoring lags on developmental science: A re-analysis of meta-analyses using lag as moderator

Supplemental material, sj-docx-1-jbd-10.1177_01650254241247155 for The impact of ignoring lags on developmental science: A re-analysis of meta-analyses using lag as moderator by Rachel M. Taylor and Noel A. Card in International Journal of Behavioral Development

Research Data

sj-docx-2-jbd-10.1177_01650254241247155 – Supplemental material for The impact of ignoring lags on developmental science: A re-analysis of meta-analyses using lag as moderator

Supplemental material, sj-docx-2-jbd-10.1177_01650254241247155 for The impact of ignoring lags on developmental science: A re-analysis of meta-analyses using lag as moderator by Rachel M. Taylor and Noel A. Card in International Journal of Behavioral Development

This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Rachel M. Taylor

Supplemental Material

Supplemental material for this article is available online.

References

Bornstein

M. H.

Putnick

D. L.

Esposito

(2017). Continuity and stability in development. Child Development Perspectives, 11(2), 113–119. https://doi.org/10.1111/cdep.12221

Cain

M. K.

Zhang

Bergeman

C. S.

(2018). Time and other considerations in mediation design. Educational and Psychological Measurement, 78(6), 952–972. https://doi.org/10.1177/0013164417743003

Card

N. A.

(2012). Applied meta-analysis for social science research. Guilford Press.

Card

N. A.

(2019). Lag as moderator meta-analysis: A methodological approach for synthesizing longitudinal data. International Journal of Behavioral Development, 43(1), 80–89. https://doi.org/10.1177/0165025418773461

Card

N. A.

Little

T. D.

(2007). Longitudinal modeling of developmental process. International Journal of Behavioral Development, 31(4), 297–302. https://doi.org/10.1177/0165025407077750

Cole

D. A.

Maxwell

S. E.

(2003). Testing mediational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112, 58–577. https://doi.org/10.1037/0021-843X.112.4.558

Collins

L. M.

(2006). Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annual Review of Psychology, 57, 505–528. https://doi.org/10.1146/annurev.psych.57.102904.190146

Collins

L. M.

Graham

J. W.

(2002). The effect of timing and spacing of observations in longitudinal studies of tobacco and other drug use: Temporal design considerations. Drug and Alcohol Dependence, 68, 85–96. https://doi.org/10.1016/s0376-8716(02)00217-x

Dormann

Griffin

M. A.

(2015). Optimal time lags in panel studies. Psychological methods, 20(4), 489. https://doi.org/10.1037/met0000041

10.

Gollob

H. F.

Reichardt

C. S.

(1987). Taking account of time lags in causal models. Child Development, 58(1), 80–92.

11.

Little

T. D.

(2013). Longitudinal structural equation modeling. Guilford Press.

12.

Little

T. D.

Card

N. A.

Preacher

K. J.

McConnell

(2009). Modeling longitudinal data from research on adolescence. In Lerner

R. M.

Steinberg

(Eds.), Handbook of adolescent psychology (3rd ed., pp. 15–54). Wiley.

13.

McArdle

J. J.

Woodcock

R. W.

(1997). Expanding test–retest designs to include developmental time-lag components. Psychological Methods, 2(4), 403–435. https://doi.org/10.1037/1082-989X.2.4.403

14.

Pelz

D. C.

Lew

R. A.

(1970). Heise’s causal model applied. In Borgatta

E. F.

Bohrnstedt

(Eds.), Sociological methodology (pp. 28–37). Jossey-Bass.

15.

Selig

J. P.

Preacher

K. J.

Little

T. D.

(2012). Modeling time-dependent association in longitudinal data: A lag as moderator approach. Multivariate Behavioral Research, 47(5), 697–716. https://doi.org/10.1080/00273171.2012.715557

16.

Timmons

A. C.

Preacher

K. J.

(2015). The importance of temporal design: How do measurement intervals affect the accuracy and efficiency of parameter estimates in longitudinal research? Multivariate Behavioral Research, 50, 41–55. https://doi.org/10.1080/00273171.2014.961056

17.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://www.jstatsoft.org/v36/i03/

18.

Woodard

Pollak

S. D.

(2020). Is there evidence for sensitive periods in emotional development? Current Opinion in Behavioral Sciences, 36, 1–6. https://doi.org/10.1016/j.cobeha.2020.05.004

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB

0.10 MB