Why Are There Different Age Relations in Cross-Sectional and Longitudinal Comparisons of Cognitive Functioning?

Cross-sectional comparisons involving different people at each age typically reveal nearly monotonic declines in several measures of cognitive functioning beginning when people are in their 20s (Salthouse, 2009; Schaie, 2013). However, age trends in longitudinal comparisons involving the same people at different ages often reveal maintained, or even increased, levels of performance (e.g., Ronnlund & Nilsson, 2006; Ronnlund, Nyberg, Backman, & Nilsson, 2005; Salthouse, 2009; Schaie, 2013).

Two major hypotheses have been proposed to explain the discrepant age trends in cross-sectional and longitudinal comparisons. One hypothesis is that cross-sectional comparisons are misleading because people of different ages may differ in factors other than age that could contribute to different levels of cognitive performance. Because it has been difficult to identify all of the relevant characteristics that could differ across people of different ages, birth year (or birth cohort) is often used as a proxy for generation-specific influences on cognitive functioning. This interpretation is frequently expressed in the form of a proposal that cross-sectional age differences are confounded by cohort differences (e.g., Schaie, 2013).

A second hypothesis that has been proposed to account for the discrepant age trends in cross-sectional and longitudinal designs is that longitudinal comparisons are distorted because performance on a second occasion can be influenced by experience with the tests on the first occasion. That is, because cognitive testing can be reactive, such that performance on a subsequent assessment is often better than it would have been without an initial assessment, this interpretation postulates that longitudinal comparisons may yield misleading age trends because they are affected by practice, or retest, effects.

Although the relation between age and cognition is one of the most fundamental issues in the field of cognitive aging, surprisingly little research has directly investigated the relative contributions of cohort and experience on age trends in measures of cognitive functioning. However, two methods originally proposed by Warner Schaie have been used in a number of studies to investigate experience effects in individuals from the same birth cohorts. The two methods are schematically illustrated in Figure 1 in the context of predicting the cognitive score at the second occasion (Time 2, T2). Notice that a minimum of three samples of research participants are involved in each type of analysis, consisting of one longitudinal sample tested at two occasions (S1a and S1b) and two separate samples with the same birth years as the longitudinal sample who are tested in the same years, respectively, as the first (S2) and second (S3) longitudinal occasions. For concreteness, specific test years and ages are indicated in the example, but the methods are applicable across a wide range of test years and ages.

OPEN IN VIEWER

Fig. 1.

Schematic illustration of methods that can be used to estimate age-related change without influences of prior test experience.

In the twice-minus-once-tested method, the performance of a new sample of participants at T2 (S3) serves as an estimate of the score on the second occasion that would have been expected in the longitudinal participants if they had no prior test experience. However, because people who are retested on a second occasion often have higher scores on the first occasion than those who do not return, the S3 score is adjusted for selectivity of the longitudinal sample (S1a) relative to the total sample at the first occasion (i.e., S1a + S2). That is, the difference between the score of the longitudinal sample (S1a) and the score of the total sample (S1a and S2) at the initial occasion is added to the score of the S3 sample to account for the selectivity of returning participants.

The quasi-longitudinal method is based on the difference in scores between two new samples of participants (S2 and S3) from the same birth years who are tested in different years, when they are at different ages. This difference, which can be considered an estimate of the change expected without prior test experience, is subtracted from the Time 1 (T1) score of the longitudinal sample to link the predicted T2 score to the longitudinal sample.

Note that the two methods rely on the same logic of estimating experience-independent change from the difference between separate samples of adults who are tested in different years. Unlike cross-sectional comparisons, both methods involve comparisons of people from the same birth cohort, and thus the comparisons are not distorted by generation-specific effects; unlike longitudinal comparisons, the methods provide estimates of change without prior test experience because the comparisons are based on each participant’s initial assessment. If the samples in different years are assumed to be comparable in relevant respects, both methods can be viewed as providing estimates of the age trends that would have resulted if there were no confounds of either cohort differences or test-experience effects.

The twice-minus-once-tested procedure has been used in several studies (e.g., Ronnlund & Nilsson, 2006; Ronnlund et al., 2005; Salthouse, 2009, 2010; Schaie, 1988). The estimates of change without experience varied across studies, possibly because they involved different measures of cognitive functioning and different intervals between assessments (i.e., 2–3 years in the Salthouse reports, 5 years in the Ronnlund studies, and 7 years in the Schaie study). However, in nearly every case, the estimates of cognitive change without prior experience were less positive than the observed longitudinal changes.

Quasi-longitudinal comparisons were apparently first reported by Schaie (Schaie, Labouvie, & Buech, 1973; Schaie & Strother, 1968), who suggested that the results were similar to those from longitudinal comparisons. However, Schaie’s interpretations were challenged by Horn and Donaldson (1976) and by Salthouse (1991), who argued that the quasi-longitudinal results more closely resembled the trends in cross-sectional comparisons than those in longitudinal comparisons. Additional studies using a similar rationale were reported by Arenberg (1978) and Kaufman (2001, 2013). The analyses by Kaufman (2013) were particularly interesting because they capitalized on the samples used to establish the norms for different versions of the Wechsler Adult Intelligence Scale (WAIS) test batteries (Wechsler, 1997, 2008) to estimate age-related cognitive change without prior experience. To illustrate, the WAIS-III test battery was normed in 1995 and the WAIS-IV test battery was normed in 2007; therefore, individuals born between 1941 and 1950 had a median age of 49.5 in 1995 and a median age of 61.5 in 2007. Comparing the index scores of individuals in these groups (as in Table 7.9 in Kaufman, 2013) thus provides an estimate of the age-related change that might be expected across the 12-year interval without prior test experience. The major result of the Kaufman analyses was that the age gradients in the quasi-longitudinal comparisons closely mirrored the gradients from traditional cross-sectional comparisons.

Application of the two methods in the same data set can be illustrated with published results on a composite memory measure from the Betula Project (Ronnlund et al., 2005). This project is particularly relevant in the current context because cross-sectional assessments with independent samples were collected in 1989 and 1994 and a longitudinal follow-up of the original sample was carried out in 1994. Episodic memory was assessed in the Betula Project with locally developed tests of recall and recognition of information from sentences describing actions, some of which were performed by participants. An episodic-memory-factor score was derived from the individual tests, and the scores in all samples were converted into t-score units based on the means and standard deviations at the first occasion (T1).

Table 1 summarizes relevant results for adults born in 1949 who were tested in 1989, when they were 40 years of age, and in 1994, when they were 45 years of age. Note that the cross-sectional comparisons revealed a moderately large age difference favoring the younger group (i.e., age differences of −0.99 in 1989 and −2.40 in 1994), but a positive difference (i.e., +2.08) in the longitudinal comparison. The predicted T2 scores from the twice-minus-once-tested and quasi-longitudinal methods are presented in Table 2, along with the estimates of change without experience obtained by subtracting the predicted T2 value from the observed T1 value. The analyses in this example are crude because they are based on only two measurement occasions, the total sample at T1 was used to approximate the S2 sample, and group means were used rather than scores of individual participants. Under these simplifying conditions, the estimates of change without experience were identical (i.e., −1.26) for the twice-minus-once-tested and quasi-longitudinal methods, and both were negative—and thus in the opposite direction of the longitudinal changes (i.e., + 2.08).

OPEN IN VIEWER

Table 1.

Estimated Change Without Experience With Data From Ronnlund, Nyberg, Backman, and Nilsson (2005)

	Cross-sectional		Longitudinal
Sample age	1989	1994	1989	1994
Age 40	56.40 (S2)	57.54	56.41 (S1a)	—
Age 45	55.41	55.14 (S3)	—	58.49 (S1b)
Difference	−0.99	−2.40	+2.08

Note: The data presented here are drawn from Table 3 in Ronnlund, Nyberg, Backman, and Nilsson (2005).

OPEN IN VIEWER

Table 2.

Illustration of Methods to Estimate Change Without Experience, Using Data From Ronnlund, Nyberg, Backman, and Nilsson (2005)

Method	Formula for estimation	Predicted T2 score	Estimated longitudinal T1-to-T2 change (56.41)
Twice-minus-once-tested method	S3 + (S1a − [S1a + S2]/2)	55.14 + (56.41 − [56.41+ 56.40]/2) = 55.15	−1.26
Quasi-longitudinal method	S1a − (S2 − S3)	56.41 − (56.40 − 55.14) = 55.15	−1.26

Note: The data presented here are drawn from Table 5 in Ronnlund, Nyberg, Backman, and Nilsson (2005). T1 = Time 1; T2 = Time 2; S1a = longitudinal sample tested at Time 1; S1b = longitudinal sample tested at Time 2; S2 = independent sample tested only at Time 1; S3 = independent sample tested only at Time 2.

Results with the two methods can also be examined with data from the Virginia Cognitive Aging Project (VCAP), an ongoing longitudinal study of cognitive functioning involving multiple tests of multiple cognitive abilities (Salthouse, 2009, 2010, in press). As of December 2013, over 2,300 individuals have participated in at least two occasions, with an average interval between occasions of approximately 3 years. A unique feature of VCAP is that new samples of participants spanning a wide age range have been recruited every year, which allows quasi-longitudinal estimates of change without experience to be derived from regression equations relating age to cognitive performance for individuals within the same birth years (Salthouse, 2013). The recruiting methods were similar in each test year, and scaled scores on the Wechsler tests were also used as covariates in the regression equations to enhance the comparability of samples in different test years.

Analyses in the current article were based on composite scores formed from the average of z scores (based on the distribution at T1) from three different types of tests (recalling a list of unrelated words, recalling details of a story, and remembering arbitrary associations between unrelated words). Means of the observed longitudinal scores at T1 and T2 and the predicted score at T2 from the two methods of estimating change without experience are reported in Figure 2.

OPEN IN VIEWER

Fig. 2.

Mean composite memory scores on the first (Time 1, T1) and second (Time 2, T2) longitudinal occasions and predicted T2 scores based on the twice-minus-once-tested and quasi-longitudinal procedures. The dashed line indicates the cross-sectional age trends based on between-cohort comparisons, and the three sets of solid lines projecting from each T1 longitudinal data point show within-cohort comparisons.

Three major results shown in this figure should be noted. First, as is typically found, the cross-sectional age trends (indicated by the dashed line) were negative beginning from the youngest age. Second, the longitudinal changes were positive at young ages and gradually became more negative with increased age. And third, age trends of the estimates of change without prior test experience from individuals within the same birth cohorts were more similar to the cross-sectional age trends than to the longitudinal age trends.

Because participants in VCAP perform tests assessing five distinct cognitive domains, results of the twice-minus-once-tested and quasi-longitudinal methods were examined in different cognitive domains (Salthouse, in press). Results with measures of reasoning, spatial visualization, and perceptual speed resembled those shown in Figure 2, but the age trends with vocabulary were positive until participants were in their 70s, and there was little effect of prior test experience on vocabulary scores at any age.

To summarize, results across several studies indicate that longitudinal comparisons of cognitive functioning are influenced by test-experience effects and that when those effects are eliminated, the age trends closely resemble cross-sectional age trends. Because the relevant samples in the twice-minus-once-tested and quasi-longitudinal samples are from the same birth years, the age trends found using these methods cannot be attributed to birth-cohort factors.

Although these results are more consistent with the hypothesis that longitudinal comparisons are distorted by experience effects than with the hypothesis that cross-sectional comparisons are distorted by birth-year-cohort effects, those are not the only interpretations of the different age trends in cross-sectional and longitudinal comparisons. For example, because the people at each age in cross-sectional comparisons are different, observed differences in cognitive functioning could be attributable to characteristics of the individuals other than age, such as quality or quantity of education or exposure to different cultural experiences. This possibility should continue to be explored, but it is important that researchers investigate potentially relevant characteristics directly instead of relying on birth year as a proxy for possible differences in people of different ages.

Because sociocultural changes can affect performance on cognitive tests (as exemplified by, e.g., the Flynn effect), age comparisons in cognitive functioning could also be influenced by period effects. With regard to time-related improvements in cognitive performance, it is sometimes suggested that cross-sectional comparisons are not meaningful because older adults at the later occasion may not have been equivalent to the current sample of young adults when they were at that age. However, the critical issue with respect to the interpretation of age differences in cognition is not whether there have been time-related effects on cognitive performance, but whether those effects varied as a function of age. That is, if the period effects were similar at all ages, then relative age comparisons would still be meaningful despite time-related shifts in the absolute level of performance. In contrast, the validity of most longitudinal comparisons could be compromised by time-lag effects because some of the observed change in the score from T1 to T2 may be attributable to period effects and not to maturational changes of primary interest. The observed longitudinal contrasts might be adjusted for period effects if time-lag data are available at different ages, but there have been few attempts of this type (see Salthouse, 1991).

Although the results reported here suggest that the mean age trends in longitudinal comparisons with certain cognitive abilities are distorted by prior test experience, the findings should not be interpreted as diminishing the importance of longitudinal studies. In fact, longitudinal data are absolutely essential for examining change within the same individual and for identifying relations of other variables with change. However, longitudinal changes may be more accurately estimated and may potentially yield stronger correlations with various lifestyle or neurobiological variables if influences of test experience on cognitive change are removed with methods such as those described here.

In conclusion, several factors likely contribute to different age trends in cross-sectional and longitudinal comparisons. The groups in cross-sectional comparisons could differ in aspects related to cognitive functioning, and time-related effects associated with the period of measurement could influence longitudinal comparisons. Nevertheless, the results summarized in this report indicate that longitudinal comparisons can be affected by test experience, and the similar age trends in between-cohort and within-cohort comparisons suggest that birth cohort is not a critical factor contributing to the differences in at least some cognitive abilities.