Abstract
Retrieval practice (e.g., testing) has been shown to facilitate long-term retention of information. In two experiments, we examine whether retrieval practice also facilitates use of the practised information when it is needed to solve analogous problems. When retrieval practice was not limited to the information most relevant to the problems (Experiment 1), it improved memory for the information a week later compared with copying or rereading the information, although we found no evidence that it improved participants’ ability to apply the information to the problems. In contrast, when retrieval practice was limited to only the information most relevant to the problems (Experiment 2), we found that retrieval practice enhanced memory for the critical information, the ability to identify the schematic similarities between the two sources of information, and the ability to apply that information to solve an analogous problem after a hint was given to do so. These results suggest that retrieval practice, through its effect on memory, can facilitate application of information to solve novel problems but has minimal effects on spontaneous realisation that the information is relevant.
The goal of education should not simply be to prepare students to demonstrate their knowledge on a memory test. Rather, education should also prepare students to use their knowledge to solve problems both inside and outside the classroom. To access knowledge when needed to solve a problem, students must form strong memory representations that persist over time. In addition, the use of knowledge to solve problems requires students to understand what they are learning rather than merely memorise isolated facts (e.g., Bransford & Stein, 1984; Nguyen & McDaniel, 2016). Thus, the best educational practices are likely those that both promote the formation of strong memory traces while also facilitating comprehension, integration, and application of what has been learned (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). The purpose of this study is to examine whether one often-used technique for strengthening memory—retrieval practice (i.e., testing)—also improves the ability to apply that information to solve a problem.
Testing is most often used in the classroom as a way to assess student knowledge, but retrieval practice can also be used to facilitate memory formation, memory integration, and memory maintenance (e.g., Brown, Roediger, & McDaniel, 2014; Roediger & Karpicke, 2006). Students who take a test have better long-term retention of material than students who restudy the same material (see Rowland, 2014 for a recent meta-analysis). Indeed, Dunlosky et al. (2013) identified testing as one of only two commonly used educational practices with strong empirical support for improving student learning across a variety of domains, populations, and situations. In the present experiments, we consider whether testing (i.e., retrieval practice) makes information more available in a way that might make it easier to use in solving an analogous problem.
Analogical problem solving occurs when problem solvers apply a previously learned solution to a novel problem (Gentner & Smith, 2013). In the classic demonstration of analogical problem solving, Gick and Holyoak (1980) presented participants with Duncker’s (1945) radiation problem, in which a tumour must be destroyed with a ray while sparing the healthy tissue surrounding the tumour. One solution is to bombard the tumour with several lower intensity rays that are positioned around the tumour. Gick and Holyoak presented participants with a story describing an analogous solution, in which a military general splits his army into smaller forces that each approach a fortress via a different road to avoid detonating mines. The authors found that people could use the source story to solve the problem, although they often required a hint that there was a connection. Gick and Holyoak concluded that using a source to solve a problem requires both (1) that the source be accessible in memory and (2) that the connection between the source and the problem be noticed.
Whether participants notice the connection between the source and the problem depends on the extent to which they have formed a schema of the story that allows them to ignore its surface features. For example, understanding the military general story as being about forces that are divided and re-converged rather than being about an army and a fortress helps participants see the connection to the medical problem about rays and tumours. Schema formation can be encouraged in several ways. For instance, participants form a stronger schema after reading two source stories than after reading only one and, as a consequence, have more success generating the analogous solution to a problem (Gick & Holyoak, 1983). Kurtz and Loewenstein (2007) showed that encountering two analogous problems and directly comparing them can have an even stronger effect than encountering and comparing two source stories. Monaghan et al. (2015) found that participants who slept after encountering a source were more adept at using the source to solve the problem than participants who did not sleep before attempting the problem. Interestingly, however, the advantage offered by sleep was not a result of improved memory for the source content; rather, the authors argue that sleep caused consolidation of the source content into a schema that was more easily applied to the novel problem.
Thus, to improve the likelihood that a source will be applied to solve an analogous problem, it is important to consider interventions that might both make the source more accessible in memory and improve the participants’ schematic understanding of the source. Moreover, because it is difficult to anticipate what problems students will one day need to solve with the information they are learning, it is worthwhile to explore whether there are activities or strategies that can be used during information presentation (and not just during recall) that will increase the likelihood that the information can be used when it is needed. We consider retrieval practice as one such possible intervention because it has been identified as a fruitful technique for improving memory that takes relatively little class time (e.g., McDaniel, Roediger, & McDermott, 2007) and that has the potential to benefit information transfer (e.g., Butler, 2010).
Retrieval practice (e.g., testing) likely facilitates long-term memory through both indirect and direct effects (Rowland, 2014). Testing may improve memory indirectly by providing an opportunity to assess current knowledge and revisit material missed on the test or by encouraging more time and attention to each piece of information. However, testing still benefits memory even when no opportunity for additional restudy is provided after the test (see Thomas & McDaniel, 2013) and even when the amount of time engaged in retrieval practice versus rereading is held constant (see Roediger & Karpicke, 2006), suggesting that testing also has a direct effect on memory. Several theoretical accounts have been proposed for how testing might directly strengthen memory (see Karpicke, 2017 for a review), including the ideas that testing provides an additional episodic context (Karpicke, Lehman, & Aue, 2014) or provides opportunities for additional semantic elaboration (Carpenter, 2009) that are helpful for long-term memory retrieval. Regardless of how and why testing benefits memory, the superior memory produced by testing should increase the availability of that information to use at a critical moment to solve a problem.
Although the effect of retrieval practice on memory is well established, the extent to which tested information can be transferred to a new question or task has received less study (see Carpenter, 2012 for a review). Some studies indicate that the effects of retrieval practice remain robust even when the format of questions is not identical between the initial and final tests (e.g., Kang, McDermott, & Roediger, 2007; Rohrer, Taylor, & Sholar, 2010). Furthermore, final performance on a test appears better after retrieval practice even when the test requires inferences that go beyond the original information (Karpicke & Blunt, 2011) or domain (Butler, 2010) studied. Such evidence suggests that retrieval practice is a powerful tool that promotes more than just rote memorisation. However, whether retrieval practice can promote analogical problem-solving—a task that requires application of the practised material in both a new domain and a new format—is unknown.
For retrieval practice to facilitate analogical problem solving, its effects must go beyond memory to actually promote schematic understanding of the practised information. There is some reason to believe this could be possible. In some contexts, testing has been shown to reduce surface processing and promote semantic processing of verbal materials (Verkoeijen, Bouwmeester, & Camp, 2012). This emphasis on semantic processing can lead to better organisation of the information in memory. For example, Zaromb and Roediger (2010) presented participants with words from several different categories (e.g., animals, fruits). Participants who repeatedly retrieved the items were more likely to cluster their responses into categories compared with participants who repeatedly studied unorganised lists. Similarly, Congleton and Rajaram (2012) suggest that testing promotes relational processing of associations between items in a set. To recall the information, participants must focus not on individual items but on how the items are related to one another so that they can be retrieved effectively. The act of retrieving allows participants to discover the relations between items that are not as readily apparent from rereading the unorganised list of items.
Thus, when applied to processing multiple stories that share a schematic structure, retrieval practice may make the structure more apparent. As participants retrieve each story, they focus not just on the individual events that are happening in the story but on how those events are related and supporting the overall narrative of the story. As has been shown with word lists (e.g., Verkoeijen et al., 2012), testing may help participants to look beyond the specific surface features of the source stories and more readily identify the underlying schematic structure. Furthermore, when participants retrieve two source stories that both share the same schematic structure, retrieving the second story after the first may further highlight the schematic structure they share. As a result, practice retrieving the story could, in addition to improving long-term memory, also promote the formation of a schema about the essential story elements that can then be more readily applied to a novel problem that shares a similar schema.
However, not all research regarding the effects of testing on relational understanding has been as promising. For example, Peterson and Mulligan (2013) found that participants who practised retrieving cue-target pairs that rhymed were less likely to notice conceptual relationships between targets than participants who reread the cue-target pairs. As a result, testing actually had a negative effect on memory because participants focused more attention on individual items (e.g., the cue-target pairs) than they did on the underlying categorical structure of the targets, which is what would have been helpful on the final test. Rawson, Wissman, and Vaughn (2015) were unable to replicate this negative testing effect but were also unable to provide unequivocal evidence that testing improves relational understanding. Thus, whether testing affects relational processing of information is unclear.
In sum, even if retrieval practice improves memory for a source story, it may not lead to more success with related problems if it does not also build understanding of how the essential elements in the story are related to one another. To investigate this issue, the present pair of experiments explores the relations between retrieval practice of source stories, memory for the stories, ability to identify schematic similarities between the stories, and ability to use the information to generate an analogous solution to novel problems.
Experiment 1
In Experiment 1, participants read two source stories with identical schematic structure about a protagonist who solves a problem by dividing and conquering. Participants were randomly assigned to one of three study activities. One group of participants reread each story (reread condition), whereas another group retrieved the stories from memory (retrieval practice condition). We also included a third group that copied the stories verbatim (copy condition) to discern if systematic re-exposure to each event in the story and engagement in the same output activity (e.g., typing) but without the difficulty of retrieving the information from memory would have the same effects as retrieval practice.
Participants in all three conditions returned to the lab a week later and were presented with two problems that could be solved by using the same schematic solution presented in the stories. We compared participants’ ability to generate the solution suggested by the stories, their memory for the stories, and their ability to identify the schematic similarities between the stories across the three conditions.
We predicted that practice retrieving the stories during the first session would improve memory for the stories a week later compared with rereading or copying the stories. Furthermore, we examined whether retrieval practice would also encourage recognition that the stories shared schematic similarities because the relations between the events in the stories might become more apparent during retrieval than during rereading. Finally, we explored whether retrieval practice would affect success on the analogous problems either before or after a hint was given to use the stories to generate a solution to the problems.
Method
Participants
Undergraduate students were recruited from general psychology courses (N = 141) to complete two sessions a week apart. Only individuals who completed both sessions were included in the analyses (N = 116, 70 females, 46 males; mean of age [Mage] = 19.25 years; standard deviation [SD] = 1.18 years). Of the final sample, 63% identified as Caucasian, 17% as Asian, 6% as Hispanic, 4% as Biracial, 5% as African American, 2% as African, and 3% as Other or did not respond. The methods were approved by the Institutional Review Board at Kalamazoo College.
Materials
Stories
Four stories were taken from Gick and Holyoak (1980). Each story described the common theme of overcoming a central target (e.g., a fortress) by deconstructing larger units (e.g., armies) into smaller pieces that simultaneously converge on the target. Table 1 outlines the plot events in two of the stories.
Events in two of the stories.
Indicates an event that is critical to the schematic similarity between the stories and the problems.
Problems
All participants were prompted to solve the radiation problem (Duncker, 1945) and the Engineering problem (adapted from Anolli, Antonietti, Crisafulli, & Cantoia, 2001). In the radiation problem, a patient has an inoperable gastrointestinal tumour, rendering the use of X-rays necessary to save the patient’s life. To solve the problem, participants have to overcome two obstacles: (1) using high-intensity beams of X-rays will damage healthy surrounding tissue and (2) using a low-intensity beam of X-ray will not eradicate the tumour. One plausible solution is to use several low-intensity beams that surround the patient and target the tumour simultaneously, each from a different direction, combining with enough intensity to damage the tumour but without enough intensity to individually damage the healthy tissue they pass through. This solution is consistent with the theme of deconstructing larger units and simultaneously converging on the target as described in the four stories.
The same theme was present in the Engineering problem (adapted from Anolli et al., 2001), which described an engineer trying to supply water to an artificial lake using canals while accounting for two factors: (1) during the rainy season, a large canal is likely to flood the surrounding area and (2) during the dry season, a small canal is likely to dry up too much to adequately supply the artificial lake. One solution would be to construct several small canals that surround and feed into the lake.
The problems were printed on separate sheets of paper with space for participants to write their solutions. For both problems, we examined whether participants generated the solution analogous to that used by the protagonists in the stories (i.e., the convergence solution).
Procedure
Session 1
Participants were tested individually and were told that their ability to study and remember information was of interest. After signing the consent form, participants saw one of the four stories on a computer screen and were given as much time as they needed to read the story.
After reading the first story, participants were randomly assigned to one of the three study activities: reread (n = 40), copy (n = 38), or retrieval practice (n = 38). In the reread condition, participants were instructed to spend as long as they wanted rereading the story. In the retrieval practice condition, participants were told they would have an opportunity to practice recalling the story. The file containing the original story was closed and participants were asked to type as much of the story as they could remember into a blank Word document. In the copy condition, participants were told they would have an opportunity to study the story by copying it. A blank Word document was opened alongside the original story, and participants were asked to type verbatim of the story into the blank document. In all three conditions, the amount of time actually spent by participants as they reread, copied, or retrieved was recorded. We did not require that participants in the three conditions engage with the stories for the same amount of time to more closely approximate what students naturally do when choosing to reread, copy, or practice retrieving material. Participants then followed the same protocol for a second story, so that each participant was exposed to two of the four stories (which were counterbalanced and presented an approximately equal number of times across participants).
The first session took an average of 15 min. At the end, participants were reminded to return to the lab a week later at the same time. They were asked not to discuss the stories with anyone else in the interim, as their memory for the stories would be of interest. In this way, participants expected that their memory for the stories would be tested, much as students expect their memory for classroom information to be tested.
Session 2
Participants returned to the same lab a week later, a retention interval for which testing is known to produce superior memory as compared with rereading (e.g., Roediger & Karpicke, 2006). Participants were first asked whether they were willing to help with the development of materials for a different study about problem solving. All agreed. They were then given one of the two problems and 3 min to devise possible solutions. After attempting the first problem, participants were given an initial 3-min attempt with the second problem. The order of the problems was counterbalanced across participants.
Throughout all problem-solving attempts, participants were encouraged to generate as many solutions as possible. If participants generated a solution other than the convergence solution, the experimenter encouraged them to keep thinking of alternate solutions. When participants generated impossible solutions (e.g., cut the tumour out), the experimenter provided a brief explanation of why the solution would not work (e.g., the problem states that the tumour is inoperable) and then encouraged them to keep thinking about it. When participants generated a correct solution other than the convergence solution (e.g., send the ray down the patient’s oesophagus), they were congratulated on finding a correct solution and told that we were interested in how many correct solutions they could generate, so they should continue to think about it. This occurred only one or two times across all conditions.
Following the first problem-solving attempts, participants completed a free recall of the stories from the previous session. The title of the story from the previous week (e.g., “the general”) was given, and participants were instructed to type as much of the story as they could recall, improvising if they could not remember the exact words. After doing this for both stories, participants described as many similarities between the two stories as possible. They typed their list of similarities from memory, without visual access to their recollections of the stories. Note that while participants may have had the radiation and engineering problems active in their working memory from the first problem-solving attempt, no instruction was given to compare the stories with the problems during this stage in the procedure, only the two stories to one another. Participants were given as much time as they wished for each step in this part of the procedure.
After completing the free recall and story similarities tasks, participants were given a second attempt to solve each problem for which the convergence solution was not generated during the first attempt. Participants were again given 3 min to create additional solutions. If participants were still unable to generate the convergence solution, they were given a hint (i.e., “Try to think about how the stories may help you solve the problem”) and a third 3-min attempt to solve the problem. The same protocol was followed for the second problem: a second unprompted attempt, followed by a hint and a third attempt if needed.
Participants were debriefed on the purpose of the study and the nature of the connection between the stories and the radiation and engineering problems. The second session ranged from 25 min for participants who generated the convergence solutions to the problems during the first attempts to 45 min for participants who did not generate the convergence solutions at all.
Coding
We coded participants’ responses from the second session to evaluate their memory for the stories, their ability to identify the schematic similarities between the stories, and their success at applying the stories to generate the analogous convergence solution to the problems.
Memory for story events
In total, 14 events were identified in each story, 6 of which were critical to understanding the schematic overlap with the problems (see Table 1 for the events in two of the stories). Research assistants read each participant’s recollection from the second session and coded whether each of the 14 events from the original story was mentioned. Each story was coded by one of four research assistants. To calculate reliability, a fifth research assistant then coded data from 12 randomly chosen participants for each story. The coders agreed on whether each event was mentioned in 89% of cases (N = 672; Cohen’s κ = .78). The codes assigned by the original coder were used in all analyses.
Identification of schematic similarities
According to Gick and Holyoak (1983), the schematic overlap between the stories is captured by three specific similarities between them. First, both stories involve the convergence of forces from different directions. Second, both stories involve multiple small forces. Third, both stories have a similar setup (e.g., both involve central targets, sufficient means to overcome the target, and a hurdle that prevents the direct application of the means to the target). One research assistant coded each similarity noted by each participant for whether it was one of these three schematic similarities or not (e.g., “both stories involved a male protagonist” is not one of the three schematic similarities relevant to the problems). A second assistant then coded a random subset of 24 participants (20% of the data). Reliability for coding each noted similarity as schematic versus not schematic was 94% (Cohen’s κ = .76). We then used the codes assigned by the first coder to count the number of schematic similarities that were noted on each participant’s generated list of similarities (0-3).
Note that although the two problems (Engineering and Radiation) also share these same three schematic similarities, participants were not asked to compare the stories to the problems during the story similarities task, only the stories to one another. Thus, this measure specifically assesses how well participants recognised the important similarities between the stories—the information that was retrieved, copied, or reread the previous week. This measure is likely related to participants’ schematic understanding that the problems are also related; that is, participants who understand the stories’ schematic connection to the problems can likely also identify the key similarities between the stories. However, it is also possible that participants can identify the schematic similarities between the stories when they are asked to, without also realising that the problems are related. Thus, this measure is an index of how well participants are able to notice the key similarities in the information in their memory when prompted and not a measure of how well they can apply those similarities to the problems.
Results
We first examined performance during the study activity of the first session. We then analysed whether study activity (retrieval practice, copying, or rereading) affected participants’ memory and ability to identify schematic similarities between the stories a week later. Finally, we examined whether success on the analogous problems was affected by study activity and whether it was related to memory for the stories or ability to identify the schematic similarities between them.
Study activity during Session 1
Participants spent less time rereading the stories (mean [M] = 61.60 s, SD = 24.81) than they spent copying (M = 347.78 s, SD = 91.08) or retrieving (M = 241.04 s, SD = 103.94) the stories, F(2, 113) = 126.57, p < .001,
Memory of the stories
Consistent with previous demonstrations of the testing effect, we predicted that participants who practised retrieving the stories would have better memory for the stories a week later compared with participants who reread the stories. As shown in Figure 1, this hypothesis was supported by a univariate analysis of variance (ANOVA), F(2, 113) = 3.49, p = .034,
The average number of events recalled (out of 14) during the second session of Experiment 1 as a function of study activity.
Identification of stories’ schematic similarities
We next addressed whether retrieval practice also affected ability to identify schematic similarities between the stories. Perhaps unsurprisingly given that participants had to identify the similarities between the stories from memory, better memory for the stories was associated with greater ability to identify schematic similarities between the stories, r(114) = .336, p < .001. However, there was no effect of study activity on the number of schematic similarities identified by participants, F(2, 113) = 1.99, p = .14. Although participants who practised retrieving had better memory for the stories, the superior memory did not help them identify the schematic similarities between them (M = 0.58, SD = 0.76) compared with copying (M = 0.95, SD = .99) or rereading (M = 0.95, SD = 1.04).
Problem success
Problem success was considered during each attempt in a separate logistic regression model. In each model, we included study activity (dummy coded with retrieval practice as the reference group) and problem (Engineering as the reference group) as fixed factors and included a random intercept for participant. 1 All analyses were run using the lme4 package in R (Bates, Maechler, Bolker, & Walker, 2015).
The results of all models are shown in Table 2. There were no effects of study activity during any of the three attempts. Retrieval practice did not help participants spontaneously access the stories (Attempt 1), spontaneously notice their connection to the problems (Attempt 2), or apply the stories to the problem when given a hint (Attempt 3).
Results of logistic regression models predicting problem success during each attempt of Experiment 1.
SE: standard error.
p < .05; **p < .001.
To explore the relevance of memory for the stories and ability to identify their schematic similarities on problem success, we ran a second analysis for each problem attempt that included average number of events recalled, average number of schematic similarities identified, and problem as fixed factors, and a random intercept for participant. As shown in Table 2, ability to identify schematic similarities between the stories was positively related to problem success before the hint (during Attempts 1 and 2). This suggests that participants’ ability to identify the schematic similarities between the stories is related to their ability to see that the problems are also related before a hint is given. Memory for the events in the stories was positively related to problem success after the stories had been recalled (during Attempts 2 and 3).
Discussion
Participants who practised retrieving the stories during the first session had significantly better memory for the stories a week later than did participants who reread the stories. This is a replication of the testing effect and adds to the growing body of literature suggesting that retrieval practice results in better long-term retention than rereading (e.g., McDaniel et al., 2007; Roediger & Karpicke, 2006; Rowland, 2014). In the present data, this could be due to either the more effortful retrieval involved in retrieval practice or the increased amount of time participants spent retrieving compared with rereading.
Although retrieval practice clearly improved memory for the stories a week later, we found no evidence that retrieval practice helped participants identify the schematic similarities between the stories or use the stories to successfully solve the analogous problems during any of the three attempts. Thus, retrieval practice as implemented here did not encourage participants to notice the common relational structure of the material more so than rereading. However, in this study, participants engaged in retrieval practice by typing as much as they could recall from each story. Such free recall instructions made no attempt to focus participants’ attention on the most important elements during retrieval. Indeed, retrieval practice did not improve participants’ memory specifically for the events that were most critical to the schematic overlap with the problems. Perhaps retrieval practice could highlight the relational structure of the material effectively if the practice were more focused on the critically relevant pieces of information. For example, van Eersel, Verkoeijen, Povilenaite, and Rikers (2016) found that transfer of practised information occurs more readily when it includes focused exposure to the key information that is to be transferred. Experiment 2 explores the possibility that specifically recalling the critical analogy-relevant details across two stories may draw participants’ attention to the similarities in those details, thereby improving understanding of the relational structure of the stories, memory of the stories, and success with the analogous problems.
Experiment 2
In Experiment 2, we implemented a 2 (study activity: reread vs retrieval practice) × 2 (information focus: critical vs noncritical) between-subjects design. The method followed Experiment 1, except that the study activity during the first session focused on only part of the stories. Specifically, in the retrieval practice conditions, participants were asked short-answer questions that encouraged them to practice retrieving details of the story that would be either critical for solving the problems the following week or noncritical (i.e., irrelevant) to the problems. In the reread conditions, participants reread either the critical or noncritical information from the stories. As in Experiment 1, we measured participants’ success at solving two analogous problems, their memory for the original stories, and their ability to identify the schematic similarities between the stories a week later.
We predicted that retrieval practice with the critical story elements would enhance memory for the critical story information a week later, improve identification of the schematic similarities between the stories, and facilitate success with the analogous problems compared with retrieval practice of the noncritical story elements.
Method
Participants
An initial sample of 105 undergraduate students was recruited from general psychology courses. Only participants who completed both sessions were included (N = 93; 50 females, 43 males; Mage = 18.77 years; SD = 1.10 years). Of the included sample, 62% identified as Caucasian, 14% as Asian, 12% as biracial, 6% as African American, and 6% as Hispanic.
Materials
The same stories and problems were used as in Experiment 1. For each story, five critical details and five noncritical details were chosen for presentation during the study period. For each detail, a sentence restating the detail was written for presentation in the study conditions, and a question requiring a short answer that would describe the same detail was written for presentation in the retrieval practice conditions. Table 3 provides examples of each type of statement and question. The five statements (reread conditions) or questions (retrieval practice conditions) were presented in the same order they appeared in the story.
Examples of statements and questions used in the critical and noncritical conditions of Experiment 2.
Procedure
Session 1
Participants were randomly assigned to one of four conditions in a 2 × 2 between subjects design, yielding four conditions: reread-noncritical details (n = 24), reread-critical details (n = 22), retrieval practice–noncritical details (n = 24), and retrieval practice–critical details (n = 23). Participants were instructed to take as much time as they needed to read each story. After reading the first story, they were given a list of statements to reread (in the reread conditions) or a list of questions to answer by typing a response after each question (in the retrieval practice conditions). Depending on the participant’s assigned condition, the list of statements or questions contained either critical or noncritical details. After rereading the details or answering the questions about the first story, the participant completed the same procedure for the second story. Participants were allowed to spend as much time as they wanted rereading the statements or answering the questions for each story, and the times were recorded. No feedback was given about participants’ answers in the retrieval practice conditions.
Session 2
The method in the second session was identical to that in Experiment 1. Data were coded following the same procedures as in Experiment 1, except that we considered how many of the critical and noncritical details participants recalled of the 10 (5 critical; 5 noncritical) that had been used during the study activities of the first session.
Results
Study activity during session 1
Participants spent more time reviewing the stories when they answered questions than when they reread statements, regardless of whether the details they were focused on were critical or noncritical. A 2 (study activity: reread vs retrieval) × 2 (information focus: critical vs noncritical) ANOVA on study times revealed main effects of study activity, F(1, 89) = 219.80, p < .001,
Participants’ ability to answer the five questions correctly was high during the first session, with participants answering more critical questions (M = 4.43, SD = .57) correctly than noncritical questions (M = 4.02, SD = .77), t(45) = 2.082, p = .04, d = 0.61.
Memory of the stories
Participants’ memory in the second session for critical and noncritical details across the four conditions is shown in Figure 2. Retrieval practice improved the retention of the specific information that was practised. Data were analysed in a 2 (study activity: retrieval practice vs reread) × 2 (information focus: critical vs noncritical) × 2 (recalled detail type: critical vs noncritical) repeated measures ANOVA with recalled detail type as a repeated measure. Although participants in the retrieval practice conditions tended to remember a higher number of the details overall than participants in the reread conditions, this effect of study activity was not quite significant, F(1, 89) = 3.53, p = .06,
The average number of (a) critical details and (b) noncritical details recalled (out of 5) in Session 2 of Experiment 2 as a function of study activity (reread vs retrieval practice) and information focus (critical vs noncritical) during Session 1.
In addition, there was a significant effect of information focus, F(1, 89) = 11.05, p = .001,
Identification of stories’ schematic similarities
Participants’ ability to identify schematic similarities between the stories was positively correlated with their memory for the critical story details, r(90) = .621, p < .001, but not with their memory for noncritical details, r(90) = .15, p = .16. Furthermore, practice retrieving critical details—which led to the best memory for critical information—also led to the best identification of schematic similarities, as shown in Figure 3. A 2 (study activity: reread vs retrieval practice) × 2 (type of information: critical vs noncritical) factorial ANOVA revealed a significant information focus × study activity interaction, F(1, 88) = 4.55, p = .036,
The average number of schematic similarities identified (out of 3) by participants during the second session of Experiment 2 as a function of their study activity and information focus during Session 1.
The question remains whether retrieval practice of critical information promoted the identification of schematic similarities directly, for example, by highlighting the structural overlap between the stories, or indirectly through its effect on memory. That is, because participants were identifying the schematic similarities from memory, it is possible that those who had the best memory for the critical information in the stories were better able to use the memory to identify the similarities. We ran a second analysis comparing the number of identified schematic similarities across study activity and information focus conditions while controlling for memory of critical events. In this model, there were no significant effects of information focus or study activity (all
Problem success
As in Experiment 1, we analysed success with the problems during each of the three attempts with separate logistic regression models that included study activity (reread as reference group), information focus (critical as reference group), and their interaction as fixed factors, as well as problem (engineering as reference). A random intercept was included for participant. We also ran a second model for each attempt that included average number of critical story events recalled, number of schematic similarities identified, and problem as fixed factors, and a random intercept for participant. All models were run using the lme4 package in R, and the results are shown in Table 4.
Results of logistic regression models predicting problem success during each attempt of Experiment 2.
SE: standard error.
p < .05; **p < .001.
First attempt
Participants were more likely to succeed on the engineering problem than on the radiation problem during the first attempt. Furthermore, as shown in the leftmost panel in Figure 4, participants who were focused on critical information during study were more likely to succeed during the first attempt than participants who were focused on noncritical information. This effect was strongest for those who reread (rather than retrieved) the critical information. It appears that being focused on critical information during study improved the likelihood of being able to spontaneously notice the relevance of the information to a new problem.
The proportion of participants who solved the Radiation problem (top) and Engineering problem (bottom) correctly during each of the three attempts across the four conditions of Experiment 2, as a function of study activity and information exposure during the first session.
However, this effect is not the result of changes in memory or ability to identify schematic similarities between the stories, as problem success during the first attempt was unrelated to these factors (see Table 4). This is in contrast to Experiment 1, where success during the first problem-solving attempt was related to how many of the schematic similarities participants could identify when prompted. Thus, it is curious that participants who reread the critical information had such high success with the problems during the first attempt, as their success does not appear related to better memory for the stories or better understanding of their relational structure. Nonetheless, it is clear that retrieval practice did not facilitate successful spontaneous transfer to the problems.
Second attempt
There were no effects of study activity, information focus, or their interaction and no effect of problem during the second attempt. However, paralleling the findings from Experiment 1, there was a significant positive effect of schematic similarities identified. For every additional schematic similarity that participants identified between the stories, their odds of solving the problems in the second attempt increased by 2.88.
Third attempt
After being given a hint to use the stories to solve the problems, practice retrieving the critical information increased the odds of success on the problem compared with rereading the critical information (see rightmost panel in Figure 4), whereas practice retrieving the noncritical information had no effect. Furthermore, the average number of critical details recalled about the stories significantly predicted problem success during the third attempt. There was no effect of number of schematic similarities identified.
To see whether the effect of study activity was driven by its effect on memory, we ran an additional logistic model that included study activity, information exposure, their interaction, problem, and memory for critical details as fixed factors (and participant as a random intercept). This model revealed a marginally significant effect of memory for critical details, coefficient = 0.77, standard error (SE) = 0.40, Z = 0.91, p = .056, and no effects of anything else. Thus, the effect of our manipulations (study activity and information focus) during the final problem attempt can be explained by their effect on memory for the critical details. Once memory for critical details is accounted for (which is enhanced with practice retrieving the critical information), there is no additional effect of retrieval practice or information exposure on one’s ability to apply the stories to solve the problems.
Discussion
The results of Experiment 2 further demonstrate the testing effect; retrieval practice significantly improved participants’ memory a week later for the details that were practised compared with rereading those details. This is consistent with our findings from Experiment 1 and many other investigations of the testing effect (e.g., Roediger & Karpicke, 2006; see Rowland, 2014). In addition, we found that practice retrieving the critical story information also improved participants’ ability to apply the stories to solve a problem once they were given a hint that there was a connection to be made and that this effect was due to improvement in participants’ memory for the critical story details.
Moreover, retrieval practice with the critical story elements not only benefitted participants’ memory for that information but also helped them recognise the schematic similarities between the two stories. There are two possible explanations for this effect. First, it is possible that answering a series of questions that highlight similar features across two stories improves the chances that the similarities will be noticed, even more so than reading the list of similar features. Under this view, participants who practised retrieving critical story information may have formed a better understanding of the schematic similarities between the stories during the first session. Alternatively, improvement in ability to identify schematic similarities may be a downstream consequence of the superior memory developed by retrieval practice with the critical information. Because we did not show participants the stories while they identified the similarities between them, the task was likely easier for participants who had better memory for the critical story information. As a result, participants who practised retrieving the critical story elements may have been better able to identify schematic similarities between the stories, not because their study activity the week before had better highlighted the similarities but because their memory for the information in the stories they needed to draw on was now better.
The available evidence strongly favours the second interpretation. Specifically, when memory for critical details is accounted for, there is no remaining relation between study activity and identification of schematic similarities. Moreover, those who practised retrieving the critical story information did not demonstrate greater spontaneous success with the problems the following week, as would be expected if the act of retrieving the critical information had improved their understanding of the similarities between the stories during the first session. In fact, during the first problem solving attempt, those who had practised retrieving critical information seemed to be less likely to spontaneously access the stories and notice their connection compared with those who reread the critical information. Thus, results from this study do not support the prediction that retrieval practice with critical information facilitates schematic understanding independently from its effects on memory.
General discussion
Across two experiments, we examined whether retrieval practice of source stories would improve the likelihood that participants could use the stories a week later to solve analogous problems. In both experiments, we found that retrieval practice benefitted memory for the stories (Experiment 1) or for the specific information that was practised (Experiment 2) a week later compared with rereading. This effect could be due to increased time spent engaged with the story information, more effortful engagement with the story information, or the combination of both in the retrieval practice condition. Although our results cannot speak to why retrieval practice is beneficial for memory, they do suggest that instructing students to practice retrieval during studying likely encourages a variety of desirable outcomes—including both increased time and increased effort—that benefit long-term retention.
Our results also demonstrate that retrieval practice, through its improvement of memory, can have other important effects for cognition. Specifically, in Experiment 2, practice retrieving the critical story information helped participants identify the similarities in the stories when they were asked to do so and helped them apply the stories to the problems after a hint was given, and these effects were driven by the increase in memory for the critical story information. Thus, it appears that retrieval practice is beneficial for long-term memory, and these effects on memory can have downstream effects that make it easier for participants to use the information when asked to do so.
At the same time, our results also suggest that the benefits of testing may be limited. Specifically, we found no evidence that retrieval practice, even with only the most critical story information, encouraged participants to spontaneously realise the underlying similarity in the schematic structure of the stories was related to the problems. In neither experiment did participants who had practised retrieving the stories show more spontaneous success with the problems a week later, which would be expected if practice retrieving the stories had improved their schematic understanding of the stories even before they were asked to reflect on the similarities. Thus, retrieval practice, at least as implemented here, does not appear to improve relational understanding independently from its effects on memory.
Of course, there are ways to implement retrieval practice that yield larger effects on memory than what we used here. For example, the effect of testing on long-term retention is stronger when participants receive feedback than when they do not (e.g., Kang et al., 2007). Likewise, the effect of testing is stronger when participants have been given multiple tests compared with a single retrieval attempt (e.g., Roediger & Karpicke, 2006). Similarly, other methods of implementing retrieval practice may be more likely to directly promote schematic understanding. For example, asking participants to recall both stories at the same time may encourage them to cluster the story events around the common similarities. Testing participants on the story similarities rather than on the details of each story is another possibility. Nonetheless, the present studies suggest that the implementation of retrieval practice used here is not strong enough to generate schematic understanding that can be spontaneously applied to solve analogous problems.
At first glance, our results appear at odds with some recent studies suggesting that testing can benefit transfer of information. Butler (2010) found that participants who practised retrieving information repeatedly were able to apply the information to answer questions even in a different domain. For example, participants could use their tested information about how bat and bird wings differ to answer questions about the design of military aircraft. However, Butler always gave participants a hint that their knowledge was relevant. Thus, his results are comparable with our findings in the final problem-solving attempt of Experiment 2, in which participants who had practised retrieving the critical story information were better able to apply the information to solve the problems once they had been given a hint to do so. In contrast, we know of no previous studies that have examined whether retrieval practice might influence participants’ ability to spontaneously realise the connection between the information and new problems.
Our results suggest that recognising that learned information is relevant to solving a novel problem relies on more than just good memory for the information. Rather, as has been suggested in previous work (e.g., Monaghan et al., 2015), spontaneous transfer seems to rest heavily on understanding that the schematic structure of the information in memory is shared by the problems. Measuring whether participants have a shared schema for the stories and the problems is difficult, as asking participants to report how the problems and stories are related would likely (1) clue participants into the possibility that they could be related and (2) cause them to discover the similarities in the act of describing them. This is why we asked participants only to reflect on the similarities between the two stories, and not between the stories and the problems. Presumably, the more a participant understands the similarities between the two stories, the more likely he or she will be to see that the similarities are also shared with the problems. Indeed, we found in both experiments that ability to identify schematic similarities between the stories was related to participants’ ability to solve the problems before a hint was given, lending credibility to the claim that understanding the schematic similarities between the stories is at least related to understanding that those similarities are also shared by the problems. However, better measures of schematic understanding between the stories and the problems could be insightful in future research using this paradigm.
Finally, it should be noted that when the findings are considered across both experiments presented here, retrieval practice was only effective at promoting transfer to the problem (with a hint) when the practice was focused on the critical information from the stories. Asking participants to practice retrieving the entire stories in Experiment 1 did not improve their memory specifically for the critical information, or their ability to apply the information to the problems, either with or without a hint, even though retrieval practice of the entire stories also included practice with the critical information. Therefore, it was not simply retrieving the critical story information that was important for facilitating far transfer of the information in Experiment 2; it was retrieving the critical story information by itself, unencumbered by extraneous detail. This is likely because asking questions about only the critical information in Experiment 2 focused participants’ attention on that information and created better overall understanding of the critical narrative of the stories. Thus, it appears that retrieval practice is most effective at promoting application of the information when it is combined with techniques that assure students are understanding what they are learning.
In conclusion, the present pair of experiments addressed whether retrieval practice is an effective means by which to improve the likelihood that learned information can be used when needed to solve an analogous problem. We found that retrieval practice clearly benefitted long-term memory and, when combined with a focus on only the most critical information, had the extra effect of making the information more accessible for use in identifying schematic similarities and applying to the problems when a hint was given. However, retrieval practice fell short of improving spontaneous realisation that the information was relevant to a problem. Thus, it appears that retrieval practice alone does not ubiquitously improve all educational outcomes; nonetheless, in conjunction with making sure that learners understand the most important aspects of what they are learning, it is certainly a helpful tool for improving long-term retention.
Footnotes
Acknowledgements
We thank Olivia Finkelstein, David Graham, Lizzie Kinney, Katie Mattison, Jeffery Washington, and Lia Williams for their assistance with data collection and coding.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.