Successive Relearning: An Underexplored but Potent Technique for Obtaining and Maintaining Knowledge

Efficacy of Successive Relearning in Promoting Retention

In our research, we have studied the efficacy of SR using the following method. In an initial practice session, participants are asked to retrieve from memory information that they have just studied. Feedback is provided, and the session for a given participant ends when that participant reaches a predetermined level of retrieval success. Participants then try to relearn the same content in one or more spaced practice sessions.

For example, in one experiment (Rawson et al., 2018), college students initially studied 48 Lithuanian-English word pairs (e.g., “pyragas – cake”). Next, they were tested on each pair (“pyragas – ?”) and were then shown the correct response if they did not correctly recall it. They continued until they reached a criterion level of recall for each pair (either one or three correct recalls) during the initial practice session. Beginning a week later, the students completed four relearning sessions, each separated by a week from the previous one. Each session began with a cued-recall test (“pyragas – ?”), and all items were relearned to a criterion of one correct recall. Because each relearning session began with a test, the impact of each relearning session on retention could be evaluated after a 1-week delay. Figure 1 shows key outcomes collapsed across the learning conditions during the initial practice sessions (criterion of one or three correct recalls). Retention a week after initial learning (leftmost bar) further emphasizes Bahrick’s (1979) claim: Even though participants correctly recalled all items during the initial practice session, they forgot much of what they had learned in the span of a week. More important, the impact of SR is reflected in the boosts to retention that occurred after each SR session. Just a single SR session substantially boosted retention. Each additional SR session also boosted retention, and three sessions produced a noteworthy level of knowledge maintenance a week after practice.

OPEN IN VIEWER

Fig. 1.

Cued-recall performance 1 week after no relearning sessions (i.e., initial learning only) versus one, two, or three relearning sessions. Error bars represent standard errors of the mean. Cohen’s d values are effect-size estimates for no relearning sessions versus one, two, or three relearning sessions; they show the difference in accuracy from the initial practice session in terms of the number of standard deviations (e.g., the d of 1.82 indicates that the percentage of correct responses following one relearning session was 1.82 SD higher than the percentage of correct responses at the end of the initial practice session). Adapted from Figure 2 of Rawson et al. (2018).

One might argue that simply recalling items more times is better for retention regardless of when those correct recalls happen, but evidence indicates otherwise: One-week retention was better when students had recalled items correctly one time in each of three spaced sessions than when they had correctly recalled each item three times during a single session (68% vs. 26%, respectively; Rawson et al., 2018). Moreover, retention a week after initial practice was better when students had recalled items three times versus only one time during that session; however, this effect of initial criterion level was no longer significant after three SR sessions. So, engaging in more (successful) retrieval practice during the initial study session did not ensure better retention in the long term. That is, if students need to practice initially learned content during another spaced session so as to achieve a higher learning objective, then spending more time to initially learn the content to a higher criterion (overlearning) may not be an efficient use of time, because the subsequent spaced relearning can override the initial benefits of overlearning (for another demonstration of a relearning-override effect, see Vaughn et al., 2016).

Although these and other outcomes demonstrate the promise of SR for boosting students’ achievement, a perennial concern is that effects demonstrated in the laboratory will not always generalize to classrooms (for detailed discussion, see Dunlosky et al., 2018). Thus, before the use of SR is prescribed, its impact should be explored in contexts where students use it to learn course-related material. For our first classroom investigation (Rawson et al., 2013), we experimentally assigned college students enrolled in introductory psychology a subset of course concepts to learn using SR. In each practice session, students were shown question prompts for the assigned concepts (e.g., “What is the definition of episodic memory?”) one at a time. Following each prompt, they attempted to retrieve the meaning of the concept and were then shown the correct answer. That is, they were instructed to retrieve the meaning, or gist, of the definition and not merely regurgitate the definition verbatim. In the initial practice session, students continued practice until they had correctly recalled the definition of each concept three times. During two subsequent relearning sessions spaced a few days apart, the students practiced the same concepts until each definition was successfully recalled one time. The last practice session occurred a couple days before an in-class exam comprising multiple-choice questions, many of which required the application of the practiced concepts.

Did SR practice produce better performance on the exam compared with students’ own study methods (i.e., business as usual)? As shown in Figure 2, exam performance was more than 10% (a full letter grade) higher for questions tapping concepts that were practiced using SR than for questions tapping concepts students studied on their own. This improvement occurred on questions that involved more than just recognizing the meaning of the concepts, which is consistent with the instructions to remember the meaning of the definitions. One concern with students’ business-as-usual approach to studying for exams (which often involves cramming the night before) is that it likely will not produce long-term retention. To evaluate this possibility, we also administered delayed tests in which students attempted to recall the meaning of each concept. As shown in Figure 2, a business-as-usual approach left students struggling to define the concepts just 3 days after the exam. In contrast, using SR supported a relatively high level of retention even 24 days after the course exam.

OPEN IN VIEWER

Fig. 2.

Undergraduates’ knowledge of introductory psychology concepts that they either learned using successive relearning or studied on their own (business as usual), tested during an in-class exam, 3 days later, and 24 days later. Error bars represent standard errors of the mean. Adapted with permission from “The Power of Successive Relearning: Improving Performance on Course Exams and Long-Term Retention,” by K. A. Rawson, J. Dunlosky, and S. M. Sciartelli, 2013, Educational Psychology Review, 25(4), Figs. 3 and 4 (https://doi.org/10.1007/s10648-013-9240-4). Copyright 2013 by Springer Nature.

Since this first evaluation of SR in a classroom setting, it was shown to improve students’ performance on an in-class exam in biopsychology (Janes et al., 2020) and their retention of key terms in another introductory psychology course (Higham et al., 2021). In the latter case, students’ use of SR across a semester also decreased their reported anxiety about using it (as compared with restudying the key terms, i.e., using their usual study methods) and led them to realize that SR was a more effective approach to learning than is restudying (Higham et al., 2021). Unfortunately, only a few studies have investigated the efficacy of SR (for a complete list of citations, see Table 1), despite the promise these studies have demonstrated.

OPEN IN VIEWER

Table 1.

Published Reports on Investigations of Successive Relearning

Publication	Participants	Materials	Outcome measure
Bahrick (1979)	Undergraduates	Foreign-language vocabulary	Cued recall
Bahrick (1979)	Undergraduates	Chinese characters	Performance on multiple-choice questions
Bahrick et al. (1993)	Bahrick family	Foreign-language vocabulary	Cued recall
Bahrick and Hall (2005)	Undergraduates	Foreign-language vocabulary	Cued recall
Dunlosky et al. (in press)	Eighth graders	Statistics definitions a	Cued recall
Higham et al. (2021)	Undergraduates	Introductory psychology definitions b	Cued recall
Janes et al. (2020)	Undergraduates	Biopsychology concepts	Performance on an in-class exam
Rawson and Dunlosky (2011)	Undergraduates	Introductory psychology definitions a	Cued recall
Rawson and Dunlosky (2013)	Undergraduates	Introductory psychology definitions a	Cued recall
Rawson et al. (2020)	Undergraduates	Probability concepts	Problem solving
Rawson et al. (2013)	Undergraduates	Introductory psychology definitions a	Performance on an in-class exam
Rawson et al. (2018)	Undergraduates	Foreign-language vocabulary	Cued recall
Schuchard et al. (2020)	Adults with aphasia	Proper nouns	Cued recall
Vaughn et al. (2016)	Undergraduates	Foreign-language vocabulary	Cued recall

a

For successive-relearning practice of sentence-length definitions in these investigations, participants were instructed to recall the meaning of the concept on cued-recall trials.

b

For successive relearning in this investigation, students were instructed to recall terms from a definition.

Successive Relearning: Alternative Questions for Cognitive-Education Research

Investigating SR has shifted our approach to conducting cognitive-education research, and doing so expanded the scope of research questions in a manner that better aligned our research outcomes with real learning objectives. That is, a great deal of cognitive-education research evaluates whether one technique boosts learning relative to another; this approach is exemplified by studies in which participants attempt to learn educationally relevant material using one of two strategies during a fixed amount of time within a single learning session prior to a final test at the end of the same session. Even when one technique produces better performance than the other, the absolute magnitude of performance often is well below what is demanded for passing courses and does not approach the level of competence that is required in many contexts. By contrast, SR is aimed at helping students to meet their real-world learning objectives.

If the research focus is on whether students obtain and maintain knowledge to achieve learning objectives, aspects of experimental control may need to be relaxed. For instance, cognitive-education research typically involves a single practice session in which the amount of time allotted for study is held constant across students and across to-be-learned items. However, students differ in the amount of practice they will require to reach a criterion level of performance (e.g., Zerr et al., 2018), and some will require more time than others (and more than one session) to reach a learning objective; the amount of time required to meet an objective will also differ across items. When using SR, each student continues testing (with feedback) until each item is correctly recalled, and thus the amount of time is tailored rather than fixed across students and items. Tailoring time spent by using SR may help reduce individual differences in learning that arise as a result of cognitive constraints (e.g. variations in working memory), background knowledge, differences in item difficulty, and so forth.

Moreover, time on task becomes a key variable of interest for investigations of SR and introduces additional questions with practical relevance: What are the costs in time for using SR? And when is that time well spent? These questions pertain to how efficiently SR improves learning, and we suspect that students consider such cost-benefit analysis when making decisions about which strategies to use when studying. To answer these questions about efficiency, one can look at the time required to reach criterion in a given practice session and how much that session improves the maintenance of knowledge.

For instance, consider outcomes from one of our experiments (Rawson & Dunlosky, 2011, Experiment 3). During an initial learning session, all the students practiced recalling the meanings of concepts from introductory psychology. Depending on their group assignment, students subsequently completed one, two, three, four, or five relearning sessions for those concepts; successive sessions were separated by a few days. In the relearning sessions, students continued to practice until they correctly recalled the meaning of each concept one time. Concepts assigned to a control condition were not practiced in any session, so that performance could be evaluated for concepts that students learned on their own. One month after the last relearning session, all the students completed a retention test that involved retrieving the meaning of the concepts. The left panel of Figure 3 presents the time required to reach criterion during each session. Students used almost 5 min per concept to reach criterion in the initial learning session. The good news is that relearning was much more rapid: Less than 2 min of practice was needed per concept to achieve correct recall of the definition in the first relearning session. Even less time was required to achieve criterion in subsequent relearning sessions (e.g., less than a minute per concept by the fifth relearning). However, even though the time cost associated with later relearning sessions was modest, performance on the 1-month retention test (right panel of Fig. 3) suggests that it was not time well spent. That is, the incremental benefit to long-term retention diminished across the relearning sessions, such that students who completed four or five relearning sessions did not retain the knowledge appreciably better than students who had completed fewer relearning sessions.

OPEN IN VIEWER

Fig. 3.

Results from a study investigating the efficiency of successive relearning among undergraduates in an introductory psychology course (Rawson & Dunlosky, 2011). Following an initial learning session, students completed one, two, three, four, or five relearning sessions for concepts assigned to the successive-relearning condition; students studied concepts in the business-as-usual (BAU) control condition on their own. The left panel shows the average amount of time the students needed to practice each concept until they could correctly recall its definition during the initial learning session and in each relearning session. The right panel (adapted from Fig. 9 of Rawson & Dunlosky, 2011) shows the students’ performance on a test of their recall of the definitions 1 month after the last practice session as a function of how many relearning sessions had been completed; 1-month recall of concepts in the BAU control condition is also shown. Error bars represent standard errors of the mean.

Our tentative conclusion from this evidence was that engaging in more than three relearning sessions was not time well spent (Rawson & Dunlosky, 2011), although this conclusion is limited to the undergraduates, materials, and schedule of practice involved in this experiment. For instance, SR sessions were separated by only a few days, and the retention test occurred a month after a final session; if an expanding schedule (i.e., with intervals between sessions increasing across sessions) had been used with an even longer delay before the final test, then more than three sessions may have further boosted long-term retention (e.g., Cepeda et al., 2008). When SR is used to promote achievement within a classroom, however, the number of possible sessions may be constrained by the class structure (e.g., fewer sessions will be feasible when exams occur every other week). Accordingly, we suspect that any single recommendation (e.g., “three sessions are enough”) would not be widely generalizable or meet the constraints of all contexts in which SR could be used.

Returning to a larger point illustrated by this example, we emphasize that students rarely will attain a real-world learning objective with a fixed amount of time during a single session. However, they also have limited time to spend studying, so the cost-benefit trade-off (time spent during practice vs. how much practice boosts retention) should be considered when identifying the optimal use of SR—or when prescribing the use of any learning technique more generally.

Future Directions

Table 1 lists all the published reports on investigations of SR, and in every case, SR boosted performance on the relevant criterion test (although in one context, the benefit of SR was unimpressive; see Rawson et al., 2020). What is most notable in this table is the homogeneity of participants, to-be-learned materials, and outcome measures used in SR investigations. Thus, further research is needed to better establish the durability and efficiency of the effects of SR for other populations, content, and outcomes. Identifying how much SR practice is enough (for any given combination of students, materials, and criterion tests) to ensure that students maintain their knowledge or can quickly relearn it when forgetting eventually occurs will also be important for establishing how to optimize students’ use of SR.

Other important issues pertain to understanding (a) the active ingredients responsible for SR and (b) the mechanisms that produce SR gains. Prior research has established that spacing of relearning sessions is an active ingredient; for example, when students practice until they correctly recall to-be-learned items three times, their retention is superior if they meet this learning criterion across three spaced sessions (one correct recall per session) rather than during a single session (e.g., Rawson et al., 2018). Some evidence suggests that retrieval practice (until a criterion is met) is also an active ingredient; for instance, when Higham et al. (2021) equated time on task between a spaced-restudy condition and a spaced-SR condition, retention was superior for the latter (see also Rawson et al., 2013). Even so, time on task was not perfectly controlled in these experiments, so it is an open question as to whether spaced restudy will be as effective as retrieval practice to criterion for helping students achieve their learning objectives. Equating time on task will be challenging, however, given that restudy may not be as engaging as retrieval practice; equating for functional time on task (vs. nominal time) may be undermined by lapses of attention during fixed amounts of study time.

Concerning the mechanisms that produce SR gains, an important direction for future research is undoubtedly to explore whether the benefit of combining spaced practice and retrieval practice (to criterion) using SR is merely a sum of their individual benefits or if the underlying mechanisms of the two techniques interact such that their combined benefits are greater than the sum of their individual benefits (i.e., the benefits are superadditive). Such investigations may be premature now, however, because the theoretical mechanisms are currently being debated for spaced practice and underdeveloped for retrieval practice. Moreover, such theory development will need to be guided by empirical investigations (beyond the very few studies now available) that firmly establish the active ingredients of SR across many different contexts (e.g., restudy may be sufficient to support learning of paired associates, whereas retrieval practice may be a necessary ingredient for learning more complex, semantic content) and that systematically explore its effects using different schedules of practice, materials, and so forth. Our current emphasis is squarely on developing a larger database on SR to ground subsequent theory development.

Other directions include investigating factors that may further enhance the effectiveness of SR. For example, one possibility is that the diminishing returns from additional relearning sessions illustrated in Figure 3 may be overcome if the interval between sessions is increased (e.g., an extra “booster” session later in the semester may further enhance students’ maintenance of knowledge over even long-time intervals). Individual differences involving cognitive architecture and motivation will likely influence various aspects of SR (e.g., how quickly the criterion is reached, rate of forgetting between sessions; see Zerr et al., 2018). If so, investigating these factors will reveal moderators of the costs and benefits of SR and will provide insight about how to further improve its effects. A final direction will involve evaluating the degree to which using SR to reach criterion performance on tasks that require more than retrieving concepts (e.g., model-based reasoning, inductive generalizations) successfully yields long-term retention of the underlying skills (e.g., Rawson et al., 2020). Given that few investigations of SR have been conducted, we expect that almost any efforts toward exploring its generalizability will be valuable for developing evidence-based prescriptions about when SR will provide the best bang for the buck for helping students efficiently obtain and maintain knowledge.

Recommended Reading

Bahrick, H. P. (1979). (See References). A classic introduction to the maintenance of knowledge and methodological approaches to investigating it.

Higham, P. A., Zengel, B., Bartlett, L. K., & Hadwin, J. A. (2021). (See References). Presentation of an innovative approach to investigating the impact of successive relearning on multiple outcome measures in a classroom setting.

Rawson, K. A., Dunlosky, J., & Janes, J. L. (2020). (See References). A report on our first successive-relearning study focused on students’ ability to solve math problems, which demonstrated only meager benefits to performance and relatively low levels of retention.