Abstract
Feedback, or information given to learners regarding their performance, is found to facilitate second language (L2) learning. Research also suggests that the timing of feedback (whether it is provided immediately or after a delay) may affect learning. The purpose of the present study was to identify the optimal feedback timing for L2 vocabulary learning. This study differs from previous feedback timing studies in two important respects. First, unlike some previous studies, feedback timing was not confounded with lag to test (interval between the last encounter with a given item and the posttest). Second, in order to test the view that delayed feedback may be particularly effective when learners make few errors during learning, the present study manipulated the frequency of practice to influence learning phase performance. In this study, 98 Japanese college students studied 16 English–Japanese word pairs. Immediate feedback was given immediately after each response, whereas delayed feedback was withheld until all target items were practised. Learning was measured by posttests administered immediately, 1 week, and 4 weeks after the treatment. Results suggested that when lag to test is controlled, feedback timing may have little effect on L2 vocabulary learning regardless of the frequency of errors during learning.
I Introduction
Research suggests that feedback, which is defined as information given to learners regarding their performance, facilitates second language (L2) learning (e.g., Lee, 2013; Li, 2010; Lyster, Saito, & Sato, 2013). The role of corrective feedback, feedback provided in response to learner errors, has attracted particular attention from researchers investigating conversational interaction (e.g., Li, 2010; Lyster et al., 2013; Rolin-Ianziti, 2010) and writing (e.g., Evans, Hartshorn, McCollum, & Wolfersberger, 2010; Lee, 2013). Not only SLA (second language acquisition) researchers but also cognitive psychologists have examined the effects of feedback on learning (e.g., Butler, Karpicke, & Roediger, 2007; Kulik & Kulik, 1988; Metcalfe, Kornell, & Finn, 2009). In cognitive psychology, feedback typically refers to the provision of the correct answer following a learner’s response. Note that while corrective feedback is provided only in response to learner errors (e.g., Li, 2010; Lyster et al., 2013), feedback can be given after successful as well as unsuccessful performance.
The cognitive psychology literature suggests that the timing of feedback may affect learning. For instance, suppose that the learner was asked to translate a Swahili word chakura (‘food’) into English. Would it be more effective to give the correct answer immediately after the response (immediate feedback) or 1 week after (delayed feedback)? Empirical studies have yielded mixed results regarding the effects of immediate and delayed feedback (e.g., Butler et al., 2007; Kulik & Kulik, 1988; Metcalfe et al., 2009). Furthermore, previous studies on feedback timing may suffer from several limitations (see below). Due to the inconsistent findings and limitations of the previous research, it is not clear whether immediate or delayed feedback is more effective for L2 learning. The present study compared the effects of immediate and delayed feedback on L2 learning while addressing the limitations of previous studies. Findings of this study may be useful because they may allow us to determine whether immediate or delayed feedback should be used in order to optimize L2 learning.
II Review of literature
1 Theoretical background
The timing of feedback is regarded as a factor that may affect L2 acquisition (e.g., DeKeyser, 2007; Evans et al., 2010; Lee, 2013; Quinn, 2013; Sheen, 2012). DeKeyser (2007), for instance, notes that feedback should not be too immediate or too delayed. Evans et al. (2010) and Lee (2013) argue that written corrective feedback should be provided immediately, although their claims are not based on empirical evidence. Some cognitive psychology studies, however, suggest that delaying feedback may increase learning (e.g., Butler et al., 2007; Carpenter & Vul, 2011; Roediger, Agarwal, Kang, & Marsh, 2010). The superiority of delayed over immediate feedback is referred to as the ‘delay-retention effect’ (e.g., Metcalfe et al., 2009; Mory, 2004).
The delay-retention effect may be accounted for by the distributed practice effect and interference theory. The distributed practice effect refers to the phenomenon where larger spacing leads to better long-term retention than shorter spacing or no spacing at all (e.g., Cepeda et al., 2009; Cepeda, Pashler, Vul, Wixted, & Rohrer, 2006; Cepeda, Vul, Rohrer, Wixted, & Pashler, 2008). Because delayed feedback is given after a greater delay than immediate feedback, the distributed practice effect predicts that delaying feedback may facilitate learning (e.g., Butler et al., 2007; Metcalfe et al., 2009). Interference theory also predicts an advantage of delayed over immediate feedback (e.g., Butler et al., 2007; Carpenter & Vul, 2011; Mory, 2004). Suppose learners made errors (e.g., *chakura = ‘drink’) during learning. (Note that throughout this study, ‘errors’ refer only to incorrect responses [i.e., errors of commission] and do not include blank responses [i.e., errors of omission]; Metcalfe et al., 2009.) When the correct answer (chakura = ‘food’) is given immediately after the response, learners may confuse their error with the correct response and might learn false information (e.g., Butler et al., 2007; Carpenter & Vul, 2011; Mory, 2004). In contrast, when feedback is given after a delay, learners’ errors may be forgotten by the time they receive feedback. As a result, their error is less likely to interfere with the correct response, which may increase learning (e.g., Butler et al., 2007; Carpenter & Vul, 2011; Mory, 2004).
The theory of errorless learning (e.g., Skinner, 1954), on the other hand, predicts an advantage of immediate feedback (e.g., Butler et al., 2007; Metcalfe et al., 2009). According to this theory, immediate feedback may be more effective because unless feedback is given immediately after the response, errors might be consolidated, and learners might learn false information. Note that predictions based on interference theory and the theory of errorless learning both rest on the assumption that learners make errors during learning. If learners produce only few errors, both theories may be irrelevant (see below).
2 Empirical evidence
Let us now consider empirical evidence regarding the effects of immediate and delayed feedback. Because none of the previous feedback timing studies examined L2 learning (unpublished conference presentations – such as Quinn, 2013; Sheen, 2012 − will not be discussed in detail in this study), this section will review previous studies that investigated the learning of materials other than L2 such as first language (L1) vocabulary or reading materials. Empirical studies have yielded mixed results regarding the effects of immediate and delayed feedback. Kulik and Kulik (1988), for instance, conducted a meta-analysis of 53 experiments on feedback timing and found that although 26 of them observed the delay-retention effect, the other 27 failed to do so. At least three explanations have been offered for the inconsistent results. First, Kulik and Kulik (1988) point out that studies conducted in laboratory settings are more likely to observe a significant delay-retention effect than those conducted in actual classroom settings. Butler et al. (2007) and Roediger et al. (2010) speculate that the setting of the experiment (laboratory or classroom) may interact with the delay-retention effect probably because participants may process feedback differently in laboratory and classroom studies. Specifically, Butler et al. and Roediger et al. point out that delayed feedback may not be studied as carefully as immediate feedback in classroom studies. Laboratory studies, in contrast, usually require learners to study feedback for a fixed amount of time in both immediate and delayed feedback conditions, ensuring that both types of feedback are processed equally carefully. Because learners may pay more attention to delayed feedback in laboratory studies than in classroom studies, laboratory studies may be more likely to produce a significant delay-retention effect (Butler et al., 2007; Roediger et al., 2010). A possible interaction between the delay-retention effect and experimental settings may partially account for the mixed results of previous studies.
Second, Metcalfe et al. (2009) point out that the effects of immediate and delayed feedback may be conditional upon the frequency of errors produced during learning. As noted above, interference theory predicts an advantage of delayed over immediate feedback, whereas the theory of errorless learning predicts a superiority of immediate feedback. It should be noted that both predictions are based on the assumption that learners make errors during learning and may be irrelevant when only few errors are produced. The distributed practice effect, in contrast, may be observed for both correct and incorrect responses. As a result, when learners produce few errors during learning, a significant delay-retention effect may be observed due to the distributed practice effect. If learners make many errors, in contrast, the beneficial effects of delaying feedback (larger spacing and less interference) might be offset by the risk of not correcting an error immediately, and a delay-retention effect may not be observed (Metcalfe et al., 2009). The inconsistent findings of previous studies may be attributed in part to a possible interaction between the delay-retention effect and the frequency of errors during learning.
Two experiments conducted by Metcalfe et al. (2009) suggest that the delay-retention effect may interact with the frequency of errors during learning. In their Experiment 1, 27 American grade school children studied 72 low-frequency English words. At the beginning of the treatment, participants were presented with a target word followed by its definition. After all target words were introduced, participants practised retrieval. More specifically, they were presented with a definition and asked to type the corresponding target word. Delayed feedback was provided 1 or 4 days after the response, whereas immediate feedback was given on the same day. In Experiment 2, 20 Columbia University students studied 75 low-frequency English words. Immediate feedback was given on the same day as the initial treatment session, whereas delayed feedback was given 3.85 days after on average. Metcalfe et al. found a larger delay-retention effect in their Experiment 1 than in their second experiment. Metcalfe et al. argue that the results might have been caused by a difference in the frequency of errors during learning. Participants in their second experiment produced more errors (61% of all responses) during the treatment than in their Experiment 1 (40%). Because delayed feedback may be particularly effective when learners make few errors during learning, Metcalfe et al.’s Experiment 1 might have produced a larger delay-retention effect than in their Experiment 2. One limitation of their study, though, is that although their first experiment was conducted with grade school children, the participants in Experiment 2 were university students. As a result, the results of their experiments may be at least partly attributed to the difference in the age of participants rather than differential learning phase performance.
Third, the inconsistent results of existing studies may be partially due to methodological differences. Previous studies differ in the operationalization of immediate and delayed feedback (Roediger et al., 2010). For instance, in Carpenter and Vul (2011), immediate feedback was given immediately after each response, and delayed feedback was given 3 seconds after. In Phye and Baller (1970), in contrast, immediate feedback was provided after 30 minutes, and delayed feedback was provided 2 days after the response. This implies that what is classified as immediate feedback in some studies may qualify as delayed feedback in others. Earlier studies also differ in the materials. The materials used by previous studies include L1 vocabulary, word-number pairs (e.g., right-12), trigram pairs (e.g., NHK-RCX), L2-trigram pairs (e.g., chakura-NHK), face-name pairs, reading materials, motor skills, programming languages, mathematics, chemistry, and psychology (see Kulik & Kulik, 1988, for a review). These methodological differences could partially be responsible for the inconsistent results of previous studies as well.
3 Limitations of previous studies
Previous feedback timing studies not only report inconsistent results but also suffer from at least three limitations. First, none of the previous feedback timing studies examined L2 learning. Thus, it is unclear to what extent their findings may be applicable to L2 learning. Second, some earlier studies did not control for lag to test. Lag to test refers to an interval between the last encounter with a given item and the test (e.g., Cepeda et al., 2008; Metcalfe et al., 2009; Rohrer, Taylor, Pashler, Wixted, & Cepeda, 2005). For instance, if a posttest is given 24 hours after the last encounter with a given item (lag to test is 24 hours) instead of 1 hour after (lag to test is 1 hour), memory performance will naturally be worse. In some previous studies, delayed feedback was associated with greater lag to test than immediate feedback. Let me illustrate this point by using Butler et al. (2007, Experiment 2) as an example. Butler et al. examined the effects of immediate and delayed feedback on the retention of reading materials. Their experiment was conducted over a period of 8 days. On Day 1, 40 American undergraduate students read 12 passages and answered multiple-choice comprehension questions. Feedback was given immediately after each response on Day 1 in the immediate feedback condition, while it was provided on Day 2 in the delayed feedback condition. The posttest was administered on Day 8. Butler et al. (2007) found that delayed feedback led to a higher posttest score (70%) than immediate feedback (60%). Based on their finding, Butler et al. argue that delaying feedback may increase long-term retention.
Metcalfe et al. (2009), however, point out that Butler et al.’s (2007) finding may be at least partly attributed to lag to test rather than feedback timing per se. Specifically, in the immediate feedback condition in Butler et al., participants received feedback on Day 1, and the posttest was conducted on Day 8. Hence, there was a lag of 7 days between feedback and the posttest. In the delayed feedback condition, however, there was a lag of only 6 days between feedback and the posttest. The confounding of feedback timing and lag to test is problematic because a shorter lag to test generally leads to better memory performance than a longer lag (e.g., Cepeda et al., 2008; Metcalfe et al., 2009; Rohrer et al., 2005). Feedback timing was confounded with lag to test in other existing studies as well (e.g., Kulhavy & Anderson, 1972; O’Neill, Rasor, & Bartz, 1976; Swindell & Walls, 1993).
Third, as described above, Metcalfe et al. (2009) observe that delayed feedback may be particularly effective when learners make few errors during learning. This suggests that in order to obtain a comprehensive picture regarding the effects of immediate and delayed feedback, it may be useful to manipulate the frequency of errors during learning. Yet, a possible relationship between the delay-retention effect and learning phase performance has not been explored thoroughly in the existing literature. Metcalfe et al.’s (2009) study constitutes the only exception. However, the results of their study may be at least partly attributed to the difference in the age of participants (i.e., grade school vs. college students) rather than the difference in the proportions of errors per se. It would be useful to examine a possible relationship between the delay-retention effect and learning phase performance without confounding the frequency of errors with the age of participants.
III Present study
This study differs from previous feedback timing studies in three important respects. First, because none of the previous studies on feedback timing examined L2 learning, it is unclear to what extent their findings may be applicable to L2 learning. With this limitation in mind, the present study compared the effects of immediate and delayed feedback on L2 vocabulary learning. Vocabulary rather than syntax or phonology was chosen as a focus of this study because L1 vocabulary research has observed a significant delay-retention effect (Metcalfe et al., 2009), suggesting that feedback timing may affect vocabulary learning. In contrast, because none of the previous feedback timing studies looked into the learning of syntax or phonology either in L1 or L2, it is not yet clear whether the learning of these aspects may be affected by feedback timing. Hence, the present study examined L2 vocabulary learning. Investigating the effects of feedback timing on L2 vocabulary learning may also have pedagogical value because previous SLA (e.g., Ellis & He, 1999; Lyster et al., 2013) and cognitive psychology studies (e.g., Metcalfe & Kornell, 2007; Pashler, Cepeda, Wixted, & Rohrer, 2005) have shown that feedback may increase vocabulary learning.
Second, because feedback timing was confounded with lag to test, the results of earlier studies may be at least partly attributed to lag to test rather than feedback timing per se. In order to address this limitation, immediate and delayed feedback in this study were controlled for lag to test. Third, in order to test the view that delayed feedback may be particularly effective when learners make few errors during learning (Metcalfe et al., 2009), the present study set out to examine a possible relationship between the delay-retention effect and learning phase performance. Unlike Metcalfe et al. (2009), where the frequency of errors during learning was confounded with the age of participants, the effects of these two factors were isolated in this study.
In order to control feedback timing and the frequency of errors during learning, this study was conducted in a computer-assisted language learning (CALL) environment. At the beginning of the treatment, Japanese learners of English were presented with an L2 (English) target word together with its L1 (Japanese) meaning (e.g., mane = たてがみ). From the second encounter, learners practised retrieval. In other words, participants were presented with a Japanese word and asked to type the corresponding English translation (e.g., たてがみ = ____?). Retrieval was followed by either immediate or delayed feedback. In the former, feedback was provided immediately after each retrieval attempt. In the latter, feedback was withheld until all target items were practised. The frequency of errors during learning was manipulated by using four levels of retrieval frequency: one, three, five, and seven. Retrieval frequency refers to the number of retrieval attempts (e.g., たてがみ = ____?) during the learning phase. For instance, if learners practise retrieval five times, the retrieval frequency is five. Because repeated retrieval may lead to more successful learning phase performance than fewer retrievals (e.g., practising retrieval seven times may lead to better learning phase performance than practising retrieval three times), manipulating retrieval frequency may allow us to investigate a possible relationship between the feedback timing effect and learning phase performance. The research question of this study is as follows: Is delayed feedback more effective than immediate feedback for L2 vocabulary learning when lag to test is controlled and learners make few errors during learning?
IV Method
1 Participants
The participants were 98 first-year engineering students (aged 15–16) from three EFL classes at a technical college in Gifu, Japan. They had been studying English for at least 3 years. Students were given participant information sheets and asked to sign consent forms if they chose to participate.
2 Experimental design
There were three independent variables in the current study. The first independent variable was the timing of feedback: immediate and delayed. The second independent variable was the retrieval frequency (i.e., number of retrieval attempts during the learning phase): one, three, five, and seven. The third independent variable was the retention interval (i.e., interval between the treatment and posttest): immediate, 1-week delayed, and 4-week delayed posttests. Retrieval frequency was a between-participant variable, and the timing of feedback and retention interval were within-participant variables. The dependent variable was the number of correct responses on the posttest.
Table 1 summarizes the design of the current study. The participants were randomly assigned by a computer program to one of the four groups: Retrieval 1, 3, 5, and 7. Although no data were available regarding their English proficiency, an analysis of learning phase performance suggested that the four retrieval frequency groups might not have differed significantly from each other in their ability to learn L2 vocabulary (see Nakata, 2013, for details). The Retrieval 7 group consisted of 26 participants, and each of the remaining three groups consisted of 24 participants. 1 The imbalance in the number of participants was caused by the absence of participants. The participants in each group were randomly divided into two subgroups: Subgroups X and Y. Sixteen target word pairs were also divided into two sets of eight items: Sets A and B (see below). The two subgroups of participants in each group studied both sets of word pairs under different feedback conditions (immediate or delayed), thus counterbalancing the effects of target items (Table 1).
Design of the present study.
Note. Each item set consisted of 8 items.
3 Target and filler items
Sixteen English–Japanese word pairs (e.g., mane – たてがみ) were used as target items. Words that are outside the most frequent 9,000 word families in the British National Corpus frequency lists (Nation, 2006) were chosen because the target items needed to be unfamiliar to participants. The 16 word pairs were divided into Sets A and B so that the learning difficulty would be distributed as evenly as possible, Set A: billow, gouge, grig, jibe, levee, loach, toupee, and urn; Set B: apparition, citadel, dally, husk, mane, mirth, rue, and warble. Learning difficulty was operationally defined as the pretest and posttest scores in a similar previous experiment, where another group of 95 Japanese college students studied the same 16 English–Japanese word pairs (Nakata, 2013). Although the two sets may not be completely equivalent in their difficulty, it was judged that a possible difference, if any, might not have a major effect on the results of the present study because effects of target items would be counterbalanced across participants (Table 1).
The number of target items was set to 16 (eight items per feedback type) based on the results of pilot studies conducted with 53 Japanese learners of English with a similar learning profile as the participants in this study. One may argue that the relatively small number of items may reduce the potential of showing a difference between immediate and delayed feedback. Despite this potential disadvantage, it was decided to set the number of target items to 16 for two reasons. First, previous studies on word-pair learning found significant differences between conditions using a relatively small number of items per condition such as two, four, or six (e.g., Cull, Shaughnessy, & Zechmeister, 1996; Karpicke & Roediger, 2007). These studies suggest that even if the number of target items is set to eight per condition, we may still be able to observe a significant feedback timing effect provided that such an effect exists. Second, even if only 16 target items are used, it may not considerably decrease the probability of detecting a significant effect of feedback timing because the cell size is relatively large (98). For the above two reasons, the number of target items was set to 16.
Three additional items were used as filler items: tyro, valor, and lava. They were chosen based on the same criterion as the target items. Filler items were studied and tested like target word pairs, but were excluded from analysis. To ensure that the filler items would be treated in the same way as the target items, the participants were not informed about any differences between items. Filler items were studied at the beginning and end of the treatment and used as primacy and recency buffers (e.g., Karpicke & Roediger, 2007). In other words, when items are presented in series, the first and last several items tend to be remembered better than the middle ones, a phenomenon known as serial position effects. The primacy and recency buffers were included to reduce these effects on target items. The same target and filler items were used during the pretest, treatment, and posttest.
4 Procedure
The experiment was conducted during regular class hours with a computer program developed by the author. The experiment consisted of three sessions.
a Session 1
Participants received explanations about the computer program and practised using it with three sample word pairs. After the practice, productive and receptive pretests were given in that order. The pretest was followed by the treatment, where participants studied 19 English–Japanese word pairs (including three filler items) using a computer program. After the treatment, participants answered 10 two-digit additions (e.g., 53 + 49 = ?) as a filler task. The immediate posttest was given after the filler task.
b Sessions 2 and 3
In order to measure retention, the delayed posttest was administered 1 and 4 weeks after the treatment.
5 Treatment
Table 2 presents the overview of the treatment in the present study. The treatment consisted of the initial presentation, retrieval phases (e.g., R1, R2, and R3 in Table 2), delayed feedback phases (e.g., D1, D2, and D3 in Table 2), and final review. In the initial presentation, the English and Japanese words were presented simultaneously for 8 seconds per word pair (e.g., mane = ‘たてがみ’). Each word pair was presented only once in the initial presentation. The initial presentation was followed by a series of alternating retrieval and delayed feedback phases. There were one, three, five, and seven sets of retrieval and delayed feedback phases for the Retrieval 1, 3, 5, and 7 groups, respectively.
Overview of the treatment.
Note. a R: retrieval phase; D: delayed feedback phase.
In the retrieval phase, participants were presented with a Japanese word and asked to type the corresponding English translation (e.g., たてがみ = ____?). Each word pair appeared only once in each retrieval phase. Participants were allowed to take as much time as they needed to type a response. For items assigned to the immediate feedback condition, feedback was provided to the participants immediately after each response. The target English word, Japanese translation, and learners’ response were given in the feedback window. Feedback also indicated whether the response was correct, partially correct, or incorrect. Partially correct responses (e.g., apparation, appartion, and applition for apparition) were defined as those that would be awarded 0.75 using a lexical production scoring protocol (e.g., Barcroft, 2007). Delayed feedback was not given until the end of each retrieval phase (e.g., D1, D2, and D3 in Table 2), where feedback for all eight delayed feedback items was presented one at a time. Both immediate and delayed feedback were shown for 5 seconds per response.
The retrieval and delayed feedback phases were followed by the final review, where the English and Japanese words were presented simultaneously for 5 seconds per word pair. The final review was included to control for lag to test (see Section II) in the immediate and delayed feedback items. As each retrieval phase was followed by a delayed feedback phase (Table 2), without the final review, the delayed feedback items would be clustered at the end of the treatment and have a shorter interval to the posttest than the immediate feedback items. The inclusion of the final review ensured that the immediate and delayed feedback items were controlled for lag to test. Because it may be common for learners to review what they will be tested on shortly before a test (Kornell, 2009), the inclusion of the final review may also reflect authentic learning and increase ecological validity. The three filler items were used as primacy and recency buffers (e.g., Karpicke & Roediger, 2007) and studied at the beginning and end of the treatment in all four groups.
6 Dependent measures
a Pretest
Immediately before the treatment, productive and receptive tests were given in that order as the pretest. In the former, participants were presented with a Japanese word and asked to type the corresponding English translation (e.g., たてがみ = ____?). In the latter, they translated target English words into Japanese (e.g., mane = ____?). The order of items in the test was randomized anew for each participant to minimize the potential of an order effect. In the productive pretest, it was necessary to prevent participants from providing synonyms for a target word because if participants produced hair for the target word mane, for instance, it would not be clear whether or not they were familiar with mane. In order to prevent learners from providing synonyms, one letter in the target word and the number of letters in the word were given as a hint (e.g., _ _ n _ for mane) together with the Japanese translation in the productive pretest. The hints were determined so as to minimize effects that they may have on performance on the receptive pretest, which was administered immediately after the productive pretest (see Nakata, 2013, for the protocol to determine the hints).
b Posttest
Immediate, 1-week delayed, and 4-week delayed posttests were administered in the present study. At each test administration, productive and receptive tests were given in that order. Although the target items were practised in a productive format during the treatment, learning was measured by receptive as well as productive tests. This is because research suggests that administering multiple types of tests may be useful because it may give us a more comprehensive picture of lexical development than a single test (see Webb, 2012, for a review). The posttests were exactly the same as the pretests except that the hints (e.g., _ _ n _ for mane) were not given in the productive posttest. This is because in the posttest, learners were instructed to produce only English words that were studied during the treatment and informed that giving a synonym for target words would be marked as incorrect. The delayed posttests were administered without prior notice so that participants would not review the target words during the period between the treatment and delayed posttests. As in the pretest, the order of items in the test was randomized anew for each participant to minimize the potential of an order effect. The immediate and the two delayed posttests were exactly the same except the item order.
7 Scoring
Responses on the pretest and posttest were scored using the following two procedures: strict and sensitive. In the productive test, in the strict scoring method, only responses without any misspellings were scored as correct. In the sensitive scoring procedure, responses that would be awarded 0.75 using a lexical production scoring protocol (e.g., Barcroft, 2007) were also scored as correct (e.g., apparation, appartion, and applition for apparition). In the receptive test, in the strict scoring method, responses were scored as incorrect if (1) they were of a wrong part of speech (e.g., 後悔 [noun] for rue) or (2) an intransitive verb was given for a transitive verb (e.g., 調和させる [transitive] for jibe) and vice versa. In the sensitive scoring system, the above two kinds of responses were both marked as correct.
V Results
1 Pretest
None of the participants exhibited prior knowledge of any of the target words on the productive pretest. When collapsed across the four retrieval frequency groups, the average receptive pretest score (SDs in parentheses) was 0.04 (0.20) and 0.07 (0.26) out of 8 with strict scoring and 0.05 (0.22) and 0.08 (0.28) out of 8 with sensitive scoring in the immediate and delayed feedback conditions, respectively. These findings suggest that participants had little or no prior knowledge of the target items.
2 Learning phase data
First, let us investigate how much spacing intervened between the retrieval attempt and feedback in the delayed feedback condition. The retrieval attempt and delayed feedback were separated by 95.61 (45.76), 91.05 (21.32), 93.88 (15.89), and 90.72 (21.79) seconds on average in the Retrieval 1, 3, 5, and 7 groups, respectively (SDs in parentheses). When collapsed across the four groups, the mean interval duration was 92.77 (28.12) seconds. No statistically significant difference was found among the four groups in their average spacing, F (3, 97) = 0.17, p = .919, producing no effect size (η2 < .01). As the difference is relatively small, it may be possible to assume that the four retrieval frequency groups had roughly equivalent spacing between the retrieval attempt and delayed feedback.
In order to examine a possible relationship between the delay-retention effect and learning phase performance, the frequency of errors during learning was analysed. (Note that ‘errors’ here refer only to incorrect responses [i.e., errors of commission] and do not include blank responses [i.e., errors of omission].) The analysis showed that 36.2% (22.3%), 24.4% (17.6%), 18.1% (8.8%), and 16.6% (14.0%) of responses during the treatment were errors (SDs in parentheses) in the Retrieval 1, 3, 5, and 7 groups, respectively. The difference among the four groups was statistically significant, F (3, 97) = 7.28, p < .001, η2= .04. The results suggest that repeated retrieval led to better learning phase performance as intended. This enables us to test the view that delayed feedback may be particularly effective when learners make few errors during learning (e.g., Metcalfe et al., 2009). The results also indicate that the proportion of errors in this study (16.6%−36.2%) was lower than in Metcalfe et al.’s (2009) experiments (Experiment 1: 40%, Experiment 2: 61%). Hence, this study might produce a larger delay-retention effect compared with Metcalfe et al.
3 Posttest performance
Tables 3 and 4 summarize the immediate and delayed posttest results as a function of feedback timing and retrieval frequency. Cronbach’s alpha was .84 or higher (.84–.91) for all dependent measures, indicating good reliability. The productive and receptive test scores were analysed by a three-way mixed design 2 (feedback timing: immediate / delayed) × 4 (retrieval frequency group: 1 / 3 / 5 / 7) × 3 (retention interval: immediate / 1-week delayed / 4-week delayed) ANOVA. As some items were answered correctly on the receptive pretest, the pretest scores were subtracted from the posttest scores, and gains were analysed when examining the receptive test results. Table 5 shows the results of the ANOVAs. (In the present study, retrieval frequency was manipulated in order to examine whether the proportion of errors during learning, which may be a function of retrieval frequency, might interact with the feedback timing effect [see Section III]. Because the question of whether repeated retrieval may increase learning is outside the scope of this study, the main effect of retrieval frequency will not be investigated.) The table indicates the following three things. First, the main effect of feedback timing was not significant on either the productive or receptive posttest regardless of the scoring procedure. This suggests that the timing of feedback had little effect on learning when collapsed across the retrieval frequency groups and the retention intervals. Second, the interaction between feedback timing and the retention interval was significant on the receptive posttest with both strict and sensitive scoring, but not on the productive posttest irrespective of the scoring system. Third, none of the other interactions involving feedback timing were significant on any of the dependent variables.
Average number of correct responses on the productive posttest (standard deviations in parentheses).
Note. The maximum score is 8 for each cell.
Average number of correct responses on the receptive posttest (standard deviations in parentheses).
Note. The maximum score is 8 for each cell.
Results of three-way ANOVAs for the posttest scores.
As the interaction between feedback timing and the retention interval proved significant on the receptive posttest, the simple main effect of feedback timing was tested to investigate where the significant differences lay. Testing the simple main effect of feedback timing allows us to examine whether any difference existed between immediate and delayed feedback at each retention interval. The results of the simple main effect tests are summarized in Table 6. When testing the simple main effect, the type I error rate was controlled using Ryan’s method for multiple comparisons. The table shows that when collapsed across the four retrieval frequency groups, immediate feedback significantly outperformed delayed feedback with strict scoring on the 1-week delayed receptive posttest (p = .031). However, despite statistical significance, only a very small effect size was observed (d = 0.14, partial η2 = .05), and the difference in the mean gains between immediate (6.09) and delayed feedback (5.83) was small. The finding is also supported by the relatively narrow confidence intervals of difference in the mean gains: [0.03, 0.51], which indicates that there is a 95% chance that the difference in the mean gains between the two feedback conditions lay somewhere between 0.03 and 0.51. The simple main effect of feedback timing was not significant in all other cases (p ≤ .123), yielding very small effect sizes (0.02 ≤ d ≤ 0.12, partial η2 ≤ .02). Overall, although a statistically significant effect was found, given the small effect size and narrow confidence intervals of difference, it might be reasonable to assume that feedback timing had little effect on posttest results irrespective of the retrieval frequency or retention interval. The statistical significance may be partially due to the relatively large cell size (98).
Results of simple main effect of feedback timing on the receptive posttest.
Note. CI of diff: 95% confidence intervals of difference in the mean gains from the pretest to the posttest between the immediate and delayed feedback conditions. df = (1, 97). Effect sizes (d) of 0.20, 0.50, and 0.80 are indicative of small, medium, and large effects, respectively (Cohen, 1988).
VI Discussion
The purpose of this study was to identify the optimal feedback timing for L2 vocabulary learning. Unlike some previous studies, the immediate and delayed feedback conditions in this study were controlled for lag to test. In order to examine a possible relationship between the delay-retention effect and the frequency of errors during learning, retrieval frequency was also manipulated. The results of this study suggested that when lag to test is controlled, feedback timing may have little effect on learning regardless of the frequency of errors during the learning phase. Although the experimental settings in this study were closer to those in laboratory studies, which are more likely to find the superiority of delayed over immediate feedback than classroom studies (Butler et al., 2007; Kulik & Kulik, 1988; Roediger et al., 2010; see Section II), a significant delay-retention effect was not observed. Taken together, the results suggest that the benefits of delayed feedback at the intervals used in this study may be limited as far as L2 vocabulary learning is concerned.
It should be noted that the immediate posttest scores of the Retrieval 5 and 7 groups neared the ceiling in both feedback conditions (Tables 3 and 4). Hence, the lack of a significant feedback timing effect in these two groups on the immediate posttest may be partly ascribed to a possible ceiling effect. On the delayed posttests, however, neither a ceiling nor floor effect was observed. The lack of statistical significance on the delayed posttest, therefore, seems to indicate that feedback timing may have little effect on long-term retention. Because from a pedagogical perspective, scores on the delayed posttest may be more important than those on the immediate posttest, the present study may nonetheless have pedagogical value despite the possible ceiling effect in the Retrieval 5 and 7 groups on the immediate posttest.
1 Theoretical implications
As discussed in Section II, there exist conflicting views about the effectiveness of immediate and delayed feedback. On one hand, delayed feedback is considered more effective because it may introduce larger spacing as well as cause less interference than immediate feedback. On the other hand, according to the theory of errorless learning, feedback needs to be given immediately after retrievals because otherwise, learners’ errors might be consolidated. The present study did not find any significant difference between the two types of feedback. The lack of a significant feedback timing effect was caused possibly because the beneficial effects of delaying feedback (larger spacing and less interference) might have been offset by the risk of not correcting an error immediately. It should be noted, however, that a significant delay-retention effect was not observed in this study although the proportion of errors (16.6%−36.2%) was lower than in Metcalfe et al.’s (2009) experiments (40%−61%). The interaction between feedback timing and the retrieval frequency group was not significant either although repeated retrieval was associated with better learning phase performance. The findings seem to be inconsistent with the observation that when learners make few errors during learning, delayed feedback may be more effective because of the distributed practice effect (Metcalfe et al., 2009). The results may suggest that the findings of Metcalfe et al.’s experiments may be at least partly attributed to the difference in the age of participants (i.e., grade school vs. college students) rather than the difference in the frequency of errors per se.
Alternatively, the conflicting results might be due in part to at least two methodological differences between the present study and Metcalfe et al. (2009). First, while Metcalfe et al. investigated the learning of L1 vocabulary, the present study looked into L2 vocabulary learning. Second, although delayed feedback was given after a delay of 1 day or longer in Metcalfe et al., it was provided 92.77 seconds on average after the response in this study (see Section V.2). Because larger spacing generally leads to better long-term retention than shorter spacing (distributed practice effect; e.g., Cepeda et al., 2006, 2008, 2009), a significant delay-retention effect might have been observed in this study if the delayed feedback condition had used much larger spacing. Future research may provide delayed feedback after a longer delay in order to explore this possibility.
2 Pedagogical implications
Pedagogically, the results imply that either immediate or delayed feedback may be used in L2 teaching and learning. Because learners generally prefer immediate over delayed feedback, the use of immediate feedback may have a positive effect on learners’ motivation and might be more desirable. Karpicke, Smith, and Grimaldi (2009), for instance, surveyed 103 American college students and found that when learning from paper-based flashcards, 91% of them confirm the correct answer immediately after retrieval attempts, which is equivalent to receiving immediate feedback. Feedback timing may also be determined based on practical considerations. For instance, immediate feedback may be used when learning vocabulary from paper-based flashcards because it may be easier to implement manually than delayed feedback. Immediate feedback may also be preferable when correcting written work because it may speed up the revision process, creating more opportunities for learners to revise their writing and receive teacher feedback (e.g., Evans et al., 2010). In conversational interaction, however, immediate feedback may be avoided because providing corrective feedback during communicative activities may potentially inhibit learners’ willingness to speak up (Rolin-Ianziti, 2010). Delayed feedback may also be more feasible for tests that require marking by teachers. Nonetheless, because this study investigated only L2 vocabulary learning in a CALL environment, the issue of whether the results of this study also apply to the learning of other aspects of L2 (e.g., syntax or phonology) or other instructional activities (e.g., face-to-face interaction or writing) awaits future research.
VII Conclusions and directions for further research
The purpose of this study was to identify the optimal feedback timing for L2 vocabulary learning while controlling the lag to test and manipulating the frequency of errors during learning. The results of this study suggested that when lag to test is controlled, feedback timing may have little effect on learning regardless of the frequency of errors during the learning phase. At the same time, the present study may suffer from some limitations. First, because both immediate and delayed feedback conditions in this study included the final review, it is possible that the effects of feedback timing might have been overshadowed by the final review. Future research may manipulate the presence or absence of the final review in order to explore this possibility. (The author is grateful to an anonymous reviewer for pointing this out.) Another limitation may be a possible ceiling effect on the immediate posttests in the Retrieval 5 and 7 groups (Tables 3 and 4). The nonsignificant feedback timing effect in these two groups on the immediate posttest may be partly ascribed to a possible ceiling effect. Third, the retention interval was a within-participant variable in this study, and each participant sat posttests at three retention intervals: immediately, 1 week, and 4 weeks after the treatment. Because correct responses in the productive test were used as cues in the receptive test, and correct responses in the receptive test were used as cues in the productive test, earlier tests might have affected performance on later tests. In future research, in order to reduce potential learning effects from posttests, the retention interval may be manipulated between participants (e.g., Cepeda et al., 2008; Karpicke & Roediger, 2007; Rohrer et al., 2005).
Footnotes
Acknowledgements
This article is based on part of the author’s doctoral dissertation, which was submitted to Victoria University of Wellington in 2013. I am very grateful to Stuart Webb, Paul Nation, Rod Ellis, Jan Hulstijn, and Kazuya Saito for their invaluable advice and Tomohiro Tsuchiya for his cooperation with data collection.
Funding
This work was supported by Faculty Research Grants from Victoria University of Wellington (grant number 109605).