How to Make Computer-Based Feedback More Productive: The Power of Erroneous Solutions

Abstract

This research aims to expand our understanding of how to facilitate student feedback engagement processes in a computer-based formative assessment environment. In the present research, we designed a new type of elaborated feedback in terms of combining the correct solution and the erroneous solution, and the erroneous solution matched the student’s initial answer. Furthermore, we analyzed whether this feedback had a stronger positive effect than the other three types of feedback containing different complexities of correct information (i.e., Knowledge of Correct Response, Problem-Solving Cues, or Complete Correct Solutions). As predicted, students who received correct and erroneous solutions experienced more positive feedback perceptions, perceived lower extraneous cognitive load and higher germane cognitive load, and achieved higher transfer performance. This research is one of the first that provides empirical evidence for the positive impact of incorporating students’ errors into the feedback design, and this novel insight can extend current theories on how to optimize feedback design to promote students’ active processing and use of feedback.

Keywords

computer-based feedback learning from errors erroneous solutions feedback perception cognitive load transfer performance

Introduction

In a computerized learning environment, feedback is regarded as an effective means to foster student learning. Feedback specifically assists learners by identifying knowledge gaps and providing guidance for future development, thereby stimulating learners’ self-regulated learning and improving academic performance (Narciss, 2013). Despite the potential advantages, research has indicated that feedback varies in its effectiveness, and even informative feedback can be unproductive for learning (see Cai et al., 2023; Mertens et al., 2022; Van der Kleij et al., 2015 for reviews). One of the important reasons is that there are individual differences in learners’ engagement with feedback (Narciss et al., 2022; Winstone et al., 2019). It may not be the case that all learners would actively process and act upon the feedback they receive. Especially in a computerized environment, it is easy for learners to roughly read feedback or simply ignore feedback (Winstone et al., 2017). As such, identifying design principles for optimizing feedback continues to be a theoretically and pedagogically important endeavor.

In the present study, we were inspired by productive failure research and erroneous worked-example research, and explored how the feedback content in the computer-based mathematics formative assessment should be designed to most effectively support learners’ problem-solving processes. Specifically, we aimed to investigate whether pairing the correct solution with the erroneous solution that matched the learner’s error would lead learners to experience more positive feedback perceptions, lower ineffective cognitive load, and achieve better academic performance than directly providing different complexities of correct problem-solving information.

Impact of Computer-Based Feedback Content on Learning

Feedback content refers to the type and amount of information provided after a learner’s response (Mory, 2004). Depending on feedback content, feedback can be divided into simple feedback and elaborated feedback (EF) (Shute, 2008). Simple feedback provides verification information (e.g., “Right or Wrong”) or the correct answer (e.g., “The correct answer is C”). EF provides additional information of correct responses beyond verification or the correct answer. Elaboration can take many forms, including cues, explanations, examples and problem-solving strategies.

From a cognitive viewpoint, feedback is considered a type of information or scaffold necessary for the correction of erroneous responses (Anderson et al., 1971; Kulhavy, 1977). Computer-based feedback can not only indicate the location and type of errors in time, but also provide information for error correction and schema reconstruction (Narciss, 2008). A large amount of research has investigated how computer-based feedback content influences learning outcomes (see Cai et al., 2023; Mertens et al., 2022; Van der Kleij et al., 2015 for reviews). This line of research has identified learning benefits from the use of elaborated feedback, including suggestions for improvement (Pardo et al., 2019), explanations (Gong et al., 2019; Lin et al., 2013), and examples (Finn et al., 2018). Simple feedback may only work on lower-order learning outcomes, such as recall test performance and recognition test performance. For example, a recent meta-analysis (Mertens et al., 2022) regarding the computer-based feedback effects for lower-order learning outcomes revealed the largest positive effect size for elaborated feedback (g = 0.71), followed by a medium effect size for correct answer feedback (g = 0.49) and a small effect size for verification feedback (g = 0.24); for higher-order learning outcomes, results revealed a medium effect size for elaborated feedback (g = 0.46) and nonsignificant effect sizes for verification feedback (g = 0.14) and correct answer feedback (g = 0.22).

However, some studies have found that elaborated feedback is unproductive for learning, or that it only promotes learning under specific conditions. For example, research conducted in a computer-assisted biological formative assessment showed that elaborated feedback (concise explanation) improved retention performance only when learners thoroughly used it, otherwise elaborated feedback had no advantage over no feedback (Maier et al., 2016). The authors ascribed the results to the influence of learners’ intrinsic motivation and ability. Only learners with high intrinsic motivation and the ability to accurately understand feedback messages will mindfully process elaborated feedback. In a computer-based mathematics homework scenario, Fyfe (2016) considered the role of learners’ prior knowledge and found that elaborated feedback (explanation and worked-example) significantly improved learning scores for learners with low prior knowledge but not for those with high prior knowledge. This can be explained by the fact that learners with sufficient prior knowledge can easily correct errors using verification information or correct answers, while elaborated feedback may lead to their redundant processing. Golke and colleagues (2015) compared (meta)cognitive prompts feedback with error explanation feedback, but did not find any positive effects on text comprehension performance. One explanation for this missing effect is that learners often overlook the feedback information provided by computer systems.

Taken together, these studies suggest that providing elaborated feedback does not always facilitate learning, and learners must engage more deeply with the elaborated feedback in a way that prompts them to identify misconceptions and revise incorrect mental models. As such, it is important to optimize feedback content to promote learners’ active processing and use of feedback. Taking this idea as a starting point, we particularly focused on whether including erroneous solutions in feedback content would enhance the positive effects of elaborated feedback. To our knowledge, no experimental research has yet investigated the effect of combining correct and erroneous solutions in the context of computer-based feedback learning, where learners usually complete test questions first and then receive different types of feedback.

Potential Positive Effects of Erroneous Solutions

The studies mentioned above have designed various types of feedback to improve learning in computer-based learning contexts. However, in most studies, correct feedback information (e.g., cues, explanations, and suggestions) is provided directly to learners after they give incorrect responses (see Cai et al., 2023; Mertens et al., 2022; Van der Kleij et al., 2015 for reviews). A prominent disadvantage of this feedback design is that it completely treats the learner as a passive receiver of feedback. Passive learning may not be the best way for learners to acquire knowledge, as passive teaching activities are often easy to stimulate superficial learning (Chi & Wylie, 2014). Therefore, simply providing learners with correct feedback information does not necessarily lead to learning benefits. Instead, effective interventions need to be considered to promote learners’ active processing and use of feedback. According to concept change theory (Vosniadou & Verschaffel, 2004), when learners are dissatisfied with their existing concept, they will engage more actively in knowledge reorganization activities to obtain correct knowledge. Learners’ dissatisfaction can be triggered by comparing their errors to correct solutions, which could help them be aware of knowledge gaps, and further put effort into revising mental models. Therefore, including erroneous solutions in feedback content may encourage learners’ engagement with feedback. Similarly, a review of errorful learning research summarized different views on the importance of designing errors as intentional learning events (Metcalfe, 2017). For example, some researchers believe that a learner’s wrong answer can act as stepping stone to guide them to the correct answer without producing interference effects (Carpenter, 2011; Metcalfe & Miele, 2014). The reconsolidation theory suggests that learners are more likely to generate correct answer when the error is retrieved in conjunction with the new stimulus, as contrasted to when the preexisting error is not evoked and only the correct answer is provided (Lee, 2008). Indeed, there is evidence that incorporating typical errors into instruction yields greater learning benefits than simply providing learners with correct problem-solving information.

Learning from errors has proven to be an effective instructional approach in many situations. A related area of research focused on the “productive failure approach” (Kapur, 2012, 2014; Kapur & Bielaczyc, 2012). This instructional design involves a problem-solving activity in which learners usually generate erroneous or incomplete solutions for novel problems, followed by an instruction phase in which learners would receive different instructional strategies. Related research has been devoted to examining the impact of instruction characteristics on the effectiveness of productive failure. For instance, in a study on the concept of variance, Loibl and Rummel (2014) investigated how the timing of instruction (before or after problem-solving) and form of instruction (with or without erroneous solutions) affected learners’ conceptual knowledge. They found a large effect size for the form (d = 0.9), favoring instruction with erroneous solutions, and a medium effect size for the timing (d = 0.6), favoring instruction after problem-solving. In another study on fractions, Loibl and Leuders (2019) further examined the role of comparison prompts when learners worked with correct and erroneous solutions, and found that learners who were prompted to compare erroneous solutions to correct solutions had better performance on the conceptual knowledge test. Similarly, after the problem-solving activity, Heemsoth and Heinze (2016) compared the effects of prompted reflection on one’s own errors to reflection on correct solutions, and found that the former had more positive effects on procedural and conceptual knowledge of fractions. To sum up, these studies suggest that incorporating erroneous solutions into instruction is a promising approach that can help learners elaborate on the rationale behind their errors and detect erroneous problem-solving steps.

Another research topic related to learning from errors is called “erroneous worked-examples”. This instructional design involves an initial worked-examples learning phase, in which learners first need to learn correct or erroneous worked-examples prepared by instructors, and then complete problem-solving tasks. Related research indicated that showing pre-designed errors in worked-examples could facilitate learning (e.g., Beege et al., 2021; Durkin & Rittle-Johnson, 2012; Richey et al., 2019). In a programming scenario, Beege and colleagues (2021) found that erroneous worked-examples containing syntactic errors led to higher error correction accuracy and learning performance than correct worked-examples. Durkin and Rittle-Johnson (2012) found that comparing correct and erroneous worked-examples was more conducive to the knowledge acquisition of fractions than only learning correct worked-examples. The underlying mechanism for the positive effects of erroneous worked-examples may be that the cognitive conflicts caused by errors encourage learners to reflect and elaborate on errors, which in turn would promote knowledge acquisition.

Research related to refutation texts has also confirmed the benefits of learning from errors (Schroeder & Kucera, 2022). A refutation text states a common misconception about a subject, refutes it, and provides an explanation of the correct conception (Tippett, 2010). There is substantial empirical evidence that refutation texts can effectively address scientific misconceptions (see Schroeder & Kucera, 2022; Zengilowski et al., 2021 for reviews). For instance, the recent meta-analysis showed that refutation text was associated with a moderate effect (g = 0.41) compared to non-refutation text conditions, and the positive effect was robust in a range of contexts (Schroeder & Kucera, 2022). It is obvious that refutation text is effective in facilitating knowledge revision processes and the development of an integrated mental model.

In conclusion, research on “learning from errors” has shown that incorporating errors into instruction phase can promote learners to engage more actively in subsequent learning activities and further achieve better academic performance. However, the previous research also has some critical shortcomings. First, in some studies on productive failure, learners were instructed to compare erroneous solutions with correct solutions within delayed whole-class discussions (e.g., Kapur, 2012, 2014; Loibl & Rummel, 2014). Notably, according to the error perseveration hypothesis, if the learner’s initial error is not remembered as an error in time, it is likely to reappear as a retention test error (Kulhavy & Anderson, 1972). Delayed feedback was more likely to result in retention of initial errors than immediate feedback (Clariana et al., 2000). Additionally, whole-class discussions can trigger individual cognitive processes only to a certain extent. Computer-based learning environments can easily address these barriers by providing learners with immediate one-on-one instruction. Second, in previous studies on productive failure and erroneous worked-examples, the erroneous solutions did not match each student’s initial answers or their own errors (e.g., Beege et al., 2021; Durkin & Rittle-Johnson, 2012; Loibl & Rummel, 2014; Richey et al., 2019). The fuzzy link between the erroneous solutions and learners’ own misconceptions may interfere with learners’ schema construction and even lead to erroneous schemas (Beege et al., 2021; Loibl & Leuders, 2019). In this vein, incorporating errors into instructional design can be ineffective. Consequently, the present research aims to provide each learner with immediate feedback containing the erroneous solution that matches their own error, and to test whether this feedback design can promote learning more than the feedback containing only correct problem-solving information.

Research Questions and Hypothesis

In computer-based feedback research, the feedback content usually only contains correct problem-solving information, which is not enough to successfully motivate learners to actively and deeply process feedback. However, the existing experimental research has focused considerably less on how to optimize feedback content to encourage learners to be proactive receivers of feedback. To address this gap, we designed a new type of elaborated feedback in terms of combining the correct solution and the erroneous solution (CES). Different from productive failure research and erroneous worked-examples research, in the present study, the erroneous solution matched the error generated by each student. In general, this study aimed to investigate whether CES was more conducive to promoting mathematics learning for learners with low and moderate knowledge than the feedback containing only correct problem-solving information (Knowledge of Correct Response, KCR; Problem-Solving Cues, PSC; or Complete Correct Solutions, CCS). According to concept change theory (Vosniadou & Verschaffel, 2004), active and deep learning is more likely to occur when learners engage in comparing their errors to correct solutions. Empirical evidence in the area of “learning from errors” suggests that exposing learners to erroneous solutions can enhance their learning. Therefore, we hypothesized:

Hypothesis 1

In terms of feedback perceptions, compared with CCS, PSC, and KCR groups, the CES group would perceive higher usefulness and acceptance of feedback, have a higher willingness to use feedback to improve performance, and experience more positive emotions.

Hypothesis 2

In terms of cognitive load, compared with CCS, PSC, and KCR groups, the CES group would perceive more germane cognitive load and less extraneous cognitive load.

Hypothesis 3

In terms of learning outcome, compared with CCS, PSC, and KCR groups, the CES group would have better transfer performance.

Method

Participants and Design

To ensure that learners had the demand for learning feedback content, 111 junior high school students with low and moderate prior knowledge (the pre-test score ≤ 6, Median_pre-test = 5) were selected through a knowledge pre-test of “Quadratic Radical”. Their mean age was 14.29 years (SD = .69) and 53 of them were girls.

This study adopted a one-factor between-subjects design with 28 participants in the CCS group, 28 in the CES group, 28 in the PSC group, and 27 in the KCR group. The dependent variables were feedback perceptions, cognitive load, and transfer performance.

Instructional Materials

The computer-based assessment covered the topic of “Quadratic Radical”. The feedback learning phase consisted of 10 multiple-choice items (α = .63) addressing the concept and properties of the quadratic radical, and the simplification and calculation of the quadratic radical. Each item had four choices—one correct answer and three high-frequency wrong answers. Before the formal experiment, a pilot study was conducted with 116 junior high school students who had learned the quadratic radical. Students completed a test consisting of 40 subjective calculation items with moderate difficulty, which were determined by two junior high school mathematics teachers and a lecturer in psychology. Students were required to write down the problem-solving steps as clearly and in detail as possible. The lecturer in psychology evaluated each student’s answers and collected their high-frequency erroneous problem-solving steps. Finally, 30 items with difficulty coefficient between 0.40 and 0.60 were selected as the formal experimental materials (10 items for prior knowledge test, 10 items for feedback learning, and 10 items for knowledge post-test). These items were designed as multiple-choice items in the formal experiment. For the items in the feedback learning phase, the collected high-frequency wrong answers were designed as three distracters in the multiple-choice items, and the corresponding erroneous problem-solving steps were designed as erroneous solutions in CES. That is to say, each of the three distracters has its own corresponding erroneous solution.

In the feedback learning phase, when learners gave an incorrect answer, they would receive one of four types of feedback according to the experimental conditions. In the present study, CCS showed the correct solution; CES combined correct and erroneous solution, and the erroneous solution matched the learner’s initial answer (see Figure 1); PSC simply provided a cue for problem-solving (e.g., “Please calculate from left to right according to the multiplication and division rules of the quadratic radical”); KCR directly provided the correct answer (e.g., “The correct answer is D”).

Figure 1.

Examples of the CES (left) and CCS (right) versions.

Assessment Instrument

Prior Knowledge test

Ten multiple-choice items were used to assess learners’ domain-specific knowledge of “Quadratic Radical”. Learners received one point for each correct answer, and they could achieve a maximum of 10 points for the prior knowledge test.

Feedback Perception

Feedback Perceptions Questionnaire (Strijbos et al., 2010, 2021) was used to assess how learners perceived the feedback. Six items measured learners’ perceived usefulness and acceptance of feedback (α = .84, e.g., “I would consider this feedback useful”), three items measured learners’ willingness to improve (α = .85, e.g., “I would be willing to use feedback to improve my performance”), three items measured learners’ positive emotions (α = .87, e.g., “I felt satisfied when I received the feedback”), and three items measured learners’ negative emotions (α = .76, e.g., “I felt frustrated when I received the feedback”). The items were rated on an 11-point scale ranging from 0 (completely disagree) to 10 (completely agree).

Cognitive Load

For the cognitive load, we adopted the Cognitive Load Scale compiled by Leppink et al. (2014). Four items measured learners’ intrinsic cognitive load (α = .83, e.g., “The questions in the computer-based assessment system are very complex”), four items measured extraneous cognitive load (α = .81, e.g., “The explanations and instructions in the computer-based assessment system are very unclear”), and five items measured germane cognitive load (α = .91, e.g., “I invest a very high mental effort during this activity in enhancing my knowledge and understanding”). The items were rated on an 11-point scale ranging from 0 (completely disagree) to 10 (completely agree).

Transfer Test

The transfer test consisted of 10 multiple-choice items covering the same topics as the items in the feedback learning phase (α = .60). It aimed to measure learners’ ability to apply knowledge to solve problems in new situations. Learners got one point for each correct answer, with a maximum score of 10 points for the transfer test.

Research Procedure

The experimental procedure is illustrated in Figure 2. The experiment was conducted in a computer-based assessment system. Before the formal experiment, participants were required to read a brief introduction of the experimental procedure and sign the informed consent. Next, participants were asked to log into the learning system with their experimental number and demographic information, and then they were randomly assigned to four experimental conditions. The formal experiment consisted of four phases: the pre-test, feedback learning, questionnaire measurement, and transfer test. First, in the pre-test phase, participants completed the prior knowledge test. Second, in the feedback learning phase, each question was accompanied by immediate feedback. If participants gave a correct answer, the system would only provide verification feedback through smiley emoji (i.e., “”). If participants gave a wrong answer, the system would not only provide verification feedback through frustrated emoji (i.e., “”), but also provide one of four types of feedback (CES, CCS, PSC, or KCR) according to different experimental conditions. As long as learners think they have mastered the feedback, they can click the “Next Question” button to proceed to the next question. After the feedback learning phase, participants took a one-minute break and then worked on the cognitive load questionnaire and feedback perceptions questionnaire. Finally, participants answered the transfer test. The whole experiment lasted about 70 min.

Figure 2.

Research procedure.

Statistical Analyses

All the statistical analyses were conducted with SPSS 21.0. First, we used the Chi-square test and one-way ANOVA to test whether there were differences in gender and age across different conditions. Then, one-way ANCOVAs with feedback content as the experimental factor were used to calculate its effects on dependent variables (feedback perceptions, cognitive load, and transfer performance), and prior knowledge was considered as the covariate to control for the initial differences between learners. Post hoc comparisons were performed using LSD.

Results

Preliminary Analyses

Results of the Chi-square test showed no significant difference in gender (χ²₍₃₎ = 6.70, p > .1). Results of the one-way ANOVA showed no significant differences in age (F (3, 107) = 1.31, p > .1). Assumptions of homogeneity of variance matrices were met, as Levene’s Tests of homogeneity were not significant (p > .05). All variables were normally distributed (|skew and kurtosis| < .94) as recommended by Tabachnick and Fidell (2013). As with the outliers, researchers proposed that in large samples (N > 80), the standard scores (Z-score) above 4 can be viewed as outliers (Liu, 2019). Based on this standard, there were no outliers in our samples. Table 1 provides an overview of the means and standard deviations of all variables.

Table 1.

Means and standard deviations of variables for four groups.

	CES	CCS	PSC	KCR
	M (SD)	M (SD)	M (SD)	M (SD)
Perceived usefulness and acceptance of feedback	8.60 (1.87)	7.64 (1.93)	7.42 (2.15)	7.13 (2.03)
Willingness to improve	8.44 (2.07)	7.44 (2.16)	7.49 (2.31)	6.68 (2.54)
Positive emotions	7.52 (2.81)	6.65 (2.56)	7.29 (2.53)	6.16 (2.35)
Negative emotions	2.39 (.88)	3.69 (1.09)	3.65 (1.12)	4.81 (1.35)
Intrinsic cognitive load	4.26 (1.29)	5.04 (1.89)	5.13 (1.84)	5.24 (1.49)
Extraneous cognitive load	3.01 (.88)	4.32 (1.52)	4.96 (1.71)	5.51 (1.68)
Germane cognitive load	7.78 (1.86)	6.56 (2.24)	6.55 (2.05)	6.41 (1.73)
Transfer performance	5.96 (2.22)	4.32 (2.09)	3.82 (1.72)	3.85 (1.73)

Feedback Perceptions

For perceived usefulness and acceptance of feedback, the four groups differed significantly (F (3, 106) = 2.82, p < .05, η²_p = .074). Post hoc tests (see Figure 3) showed that the CES group reported higher perceived usefulness and acceptance than the CCS (p = .077, d = .51), PSC (p < .05, d = .59), and KCR groups (p < .01, d = .75), which did not differ significantly from each other.

Figure 3.

Results of the perceived usefulness and acceptance for the four groups.

For the willingness to improve, the four groups differed significantly (F (3, 106) = 2.75, p < .05, η²_p = .072). Post hoc tests (see Figure 4) showed that the CES group had a higher willingness to use feedback to improve performance than the KCR group (p < .01, d = .76). Other differences were not significant.

Figure 4.

Results of the willingness to improve for the four groups.

For the positive emotions when receiving feedback, there were no significant differences among the four groups (F (3, 106) = 1.57, p = .202).

For the negative emotions when receiving feedback, the four groups differed significantly (F (3, 106) = 21.39, p < .001, η²_p = .377). Post hoc tests (see Figure 5) showed that the CES group experienced lower negative emotions than the CCS (p < .001, d = 1.31), PSC (p < .001, d = 1.25), and KCR groups (p < .001, d = 2.13). The CCS group (p < .001, d = .91) and the PSC group (p < .001, d = .94) experienced lower negative emotions than the KCR group. Other differences were not significant.

Figure 5.

Results of the negative emotions for the four groups.

Cognitive Load

For the intrinsic cognitive load, there were no significant differences among the four groups (F (3, 106) = 2.03, p = .114).

For the extraneous cognitive load, the four groups differed significantly (F (3, 106) = 14.53, p < .001, η²_p = .291). Post hoc tests (see Figure 6) showed that the CES group perceived lower extraneous cognitive load than the CCS (p < .01, d = 1.05), PSC (p < .001, d = 1.43), and KCR groups (p < .001, d = 1.87). The CCS group perceived lower extraneous cognitive load than the KCR group (p < .01, d = 0.74). Other differences were not significant.

Figure 6.

Results of the extraneous cognitive load for the four groups.

For the germane cognitive load, the four groups differed significantly (F (3, 106) = 2.88, p < .05, η²_p = .075). Post hoc tests (see Figure 7) showed that the CES group reported higher germane cognitive load than the CCS (p < .05, d = .59), PSC (p < .05, d = .63), and KCR groups (p < .05, d = .76). Other differences were not significant.

Figure 7.

Results of the germane cognitive load for the four groups.

Transfer Performance

For the transfer performance, the four groups differed significantly (F (3, 106) = 9.10, p < .001, η²_p = .205). Post hoc tests (see Figure 8) showed that the CES group outperformed the CCS (p < .01, d = .76), PSC (p < .001, d = 1.08), and KCR groups (p < .001, d = 1.06), which did not differ significantly from each other.

Figure 8.

Results of the transfer performance for the four groups.

Discussion

The goal of this study is to incorporate the erroneous solutions that match learners’ self-generated errors into feedback content to facilitate active and deep learning in a computer-based mathematics formative assessment environment. Our findings revealed that low and moderate knowledge learners in the CES group reported more positive feedback perceptions than those in the other feedback groups (i.e., CCS, PSC, KCR). Furthermore, providing CES had more positive effects on reducing learners’ extraneous cognitive load and improving their germane cognitive load and transfer performance. Taken together, these findings support the theoretical considerations on the role of actively involving learners in the feedback process and the role of incorporating learners’ errors into feedback design.

Firstly, for feedback perceptions, we found that compared to CCS, PSC, and KCR groups, the CES group reported higher perceived usefulness and acceptance of feedback, greater willingness to implement feedback, and fewer negative emotions, validating hypothesis 1. These findings were similar to prior findings (Corbalan et al., 2010; Gong et al., 2019; Wang et al., 2019), which revealed the positive effect of computer-based elaborated feedback on stimulating learners’ positive feedback perception (perceived usefulness of feedback). The present study goes beyond prior research by designing the CES, and further investigating how different types of feedback affect learners’ willingness to implement feedback and their emotions after receiving feedback. From the perspective of “learning from errors” (Metcalfe, 2017; Ohlsso, 1996), CES would be efficient in helping learners figure out the errors timely and accurately, which could promote their understanding and use of the feedback content, increase their confidence in correcting errors, and reduce their negative emotions after wrong reactions. However, PSC and KCR provided very limited information (simple cues or the correct answer) in the present study, which made it hard for low and moderate knowledge learners to adequately understand and use the feedback information. Regarding the CCS, although it provided the complete correct solutions, this feedback design that directly presented the correct problem-solving information may not be enough to stimulate active learning. In addition, based on CCS alone, low and moderate knowledge learners can easily understand why an answer is correct, but they may not fully understand the reason for their errors (Heemsoth & Heinze, 2016; Loibl & Rummel, 2014). Thus, compared to the other three types of feedback, the CES was more conducive to learners’ understanding and mastery of the feedback content, which led to more positive feedback perceptions.

Secondly, this study found that the CES group perceived lower extraneous cognitive load and higher germane cognitive load, verifying hypothesis 2. These findings indicate that CES has more advantages in reducing the extraneous cognitive load brought by instructional design and increasing the cognitive resources for modifying mental models. These positive effects on cognitive load can be explained from the perspective of information processing (Mayer, 2014). Specifically, in the information selection phase, the erroneous solutions in the CES can direct learners’ attention to the error location in time and avoid wasting excessive cognitive resources for identifying key information. In the information organization and integration phase, CES can more clearly guide learners to invest cognitive resources to establish connections between misconceptions and correct knowledge, which is conducive to constructing correct schemas. However, the present study did not find an advantage of CES in reducing intrinsic cognitive load. One potential reason may be that intrinsic cognitive load is mainly caused by the complexity of the learning content and learners’ prior knowledge (Sweller et al., 2019), and instructional interventions, such as the feedback in this study, can hardly change the intrinsic cognitive load.

Finally, this study found that the CES group had better transfer performance, supporting hypothesis 3. Previous studies in the erroneous worked-examples domain (Booth et al., 2013; Durkin & Rittle-Johnson, 2012) have found that providing erroneous worked-examples before learners attempt to solve a task, rather than just providing correct worked-examples, could significantly improve knowledge transfer. Compared with these studies, the innovation of the present study lies in that we used erroneous solutions in the feedback learning phase after task-solving. Furthermore, to take full advantage of the CES, we ensured that the erroneous solutions matched the learner’s incorrect answers. The positive role of CES in promoting transfer performance can be explained from the following two aspects. For one thing, compared with passive learning of correct problem-solving information, receiving CES can trigger cognitive conflict and make learners aware of the existence of knowledge gaps, which would encourage learners to actively compare correct and erroneous solutions, and stimulate their elaboration on errors. For another, regarding the positive effects of CES on extraneous and germane cognitive load in the present research, it can be inferred that CES has advantages in facilitating the selection, organization, and integration of information, which enables learners to gain a deeper understanding of the feedback content and ultimately facilitate the transfer and application of learning content.

Research Contribution

To improve learners’ engagement with feedback, the present study drew on the research findings in the field of “learning from errors” to optimize the design of feedback content, and confirmed the positive effects of CES on enhancing low and moderate knowledge learners’ positive feedback perceptions and learning performance, as well as reducing ineffective cognitive load. Theoretically, the present study provides a new perspective on the issue of how learners can be supported to deeply and actively engage with the feedback process to generate new knowledge. Our findings also have practical implications for designing interventions to make feedback content more useable in computerized learning environments. In the future, instructional designers need to be noted that only providing low and moderate knowledge learners with correct task-related information, especially simple verification feedback, may not be enough to lead to successful learning. Instead, integrating erroneous solutions generated by learners into feedback design can make computer-based feedback more productive. By comparing correct and erroneous problem-solving information, learners can accurately identify and locate the error, which in turn contributes to error correction and better learning outcomes. Therefore, the computerized environments, such as computer-based formative assessment systems, should make full use of technical advantages to systematically collect, analyze and summarize learners’ typical errors in various subject areas, and use these errors to customize feedback.

Limitations and Future Work

Some limitations also merit discussion. Firstly, individual differences in feedback learning need to be further explored. Winstone et al. (2019) used the questionnaire method to examine the associations between individual difference variables and feedback perceptions, and found that mastery approach goals and conscientiousness could significantly predict perceived utility of feedback, volition to implement feedback, and self-efficacy for using feedback. Therefore, it would be a very interesting issue for further research to investigate learners with which specific characteristics are more likely to mindfully process feedback and the mechanisms that lead to such individual differences.

Secondly, considering the long duration of this experiment and the frequent head movements of learners in the learning process, eye-tracking technology is not used to collect learners’ attention data in the present experiment. After learners finished working on feedback learning tasks, we collected their feedback perceptions data through the self-reported method. Although these self-reported data can explain to some extent how learners read, evaluate and use feedback, there are still some limitations in revealing the process of feedback engagement. For example, our data does not provide enough information on what strategies learners use to process feedback. Recent research combining online and offline measures (eye tracking, think aloud, keystroke logging, and text analyses) revealed that learners had three feedback processing strategies: superficial processing, local processing, and deep processing (Bouwer & Dirkx, 2023). Therefore, future research should consider adopting appropriate online and offline measures to comprehensively examine learners’ feedback processing characteristics.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Humanity and Social Science Youth Foundation of Ministry of Education of China (23YJC190027), the Philosophy and Social Science Planning Project of Henan Province (2022CJY047), the National Natural Science Foundation of China (61877025), the Humanity and Social Science Foundation of Ministry of Education of China (23YJA190003), and the Teaching Reform Research Project of Henan University (HDXJJG2022-105).

ORCID iD

Zhen Wang

Author Biographies

Zhen Wang is a lecturer in the School of Psychology at Henan University. Her research interests include feedback learning, the design of pedagogical agents, and online video-based learning.

Xinrui Pei is a master student in the School of Psychology at Henan University. Her research interests include feedback learning and emotional design.

Hejie Zhu is a master student in the School of Psychology at Henan University. Her research interests include peer feedback and writing learning.

Shaoying Gong is a professor in the School of Psychology at Central China Normal University. Her research interests include feedback learning, the design of pedagogical agents, creativity, and teacher education.

Enguo Wang is a professor in the School of Psychology at Henan University. His research interests include learning disability, developmental coordination disorder, and cognitive neuroscience.

References

Anderson

R. C.

Kulhavy

R. W.

Andre

(1971). Feedback procedures in programmed instruction. Journal of Educational Psychology, 62(2), 148–156. https://doi.org/10.1037/h0030766

Beege

Schneider

Nebel

Zimm

Windisch

Rey

G. D.

(2021). Learning programming from erroneous worked-examples. Which type of error is beneficial for learning? Learning and Instruction, 75, Article 101497. https://doi.org/10.1016/j.learninstruc.2021.101497

Booth

J. L.

Lange

K. E.

Koedinger

K. R.

Newton

K. J.

(2013). Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction, 25, 24–34. https://doi.org/10.1016/j.learninstruc.2012.11.002

Bouwer

Dirkx

(2023). The eye-mind of processing written feedback: Unraveling how students read and use feedback for revision. Learning and Instruction, 85(1), Article 101745. https://doi.org/10.1016/j.learninstruc.2023.101745

Cai

Gui

Mao

Wang

Hao

Fan

Tai

R. H.

(2023). The effect of feedback on academic achievement in technology-rich learning environments (TREs): A meta-analytic review. Educational Research Review, 39(7), Article 100521. https://doi.org/10.1016/j.edurev.2023.100521

Carpenter

S. K.

(2011). Semantic information activated during retrieval contributes to later retention: Support for the mediator effectiveness hypothesis of the testing effect. Journal of Experimental Psychology. Learning, Memory, and Cognition, 37(6), 1547–1552. https://doi.org/10.1037/a0024140

Chi

M. T. H.

Wylie

(2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823

Clariana

R. B.

Wagner

Murphy

L. R.

(2000). Applying a connectionist description of feedback timing. Educational Technology Research & Development, 48(3), 5–21. https://doi.org/10.1007/BF02319855

Corbalan

Paas

Cuypers

(2010). Computer-based feedback in linear algebra: Effects on transfer performance and motivation. Computers & Education, 55(2), 692–703. https://doi.org/10.1016/j.compedu.2010.03.002

10.

Durkin

Rittle-Johnson

(2012). The effectiveness of using incorrect examples to support learning about decimal magnitude. Learning and Instruction, 22(3), 206–214. https://doi.org/10.1016/j.learninstruc.2011.11.001

11.

Finn

Thomas

Rawson

K. A.

(2018). Learning more from feedback: Elaborating feedback with examples enhances concept learning. Learning and Instruction, 54, 104–113. https://doi.org/10.1016/j.learninstruc.2017.08.007

12.

Fyfe

E. R.

(2016). Providing feedback on computer-based algebra homework in middle-school classrooms. Computers in Human Behavior, 63(2), 568–574. https://doi.org/10.1016/j.chb.2016.05.082

13.

Golke

Dorfler

Artelt

(2015). The impact of elaborated feedback on text comprehension within a computer-based assessment. Learning and Instruction, 39, 123–136. https://doi.org/10.1016/j.learninstruc.2015.05.009

14.

Gong

S. Y.

Han

Y. S.

Wang

Liu

X. X.

(2019). Effects of complexity of computer-based feedback and prior knowledge on learning. Studies of Psychology and Behavior, 17(6), 765–772. https://doi.org/10.3969/j.issn.1672-0628.2019.06.007

15.

Heemsoth

Heinze

(2016). Secondary school students learning from reflections on the rationale behind self-made errors: A field experiment. The Journal of Experimental Education, 84(1), 98–118. https://doi.org/10.1080/00220973.2014.963215

16.

Kapur

(2012). Productive failure in learning the concept of variance. Instructional Science, 40(4), 651–672. https://doi.org/10.1007/s11251-012-9209-6

17.

Kapur

(2014). Productive failure in learning math. Cognitive Science, 38(5), 1008–1022. https://doi.org/10.1111/cogs.12107

18.

Kapur

Bielaczyc

(2012). Designing for productive failure. The Journal of the Learning Sciences, 21(1), 45–83. https://doi.org/10.1080/10508406.2011.591717

19.

Kulhavy

R. W.

(1977). Feedback in written instruction. Review of Educational Research, 47(2), 211–232. https://doi.org/10.3102/00346543047002211

20.

Kulhavy

R. W.

Anderson

R. C.

(1972). Delay-retention effect with multiple-choice tests. Journal of Educational Psychology, 63(5), 505–512. https://doi.org/10.1037/h0033243

21.

Lee

J. L. C.

(2008). Memory reconsolidation mediates the strengthening of memories by additional learning. Nature Neuroscience, 11(11), 1264–1266. https://doi.org/10.1038/nn.2205

22.

Leppink

Paas

Van Gog

Van der Vleuten

C. P. M.

van Merriënboer

J. J.

(2014). Effects of pairs of problems and examples on task performance and different types of cognitive load. Learning and Instruction, 30, 32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001

23.

Lin

L. J.

Atkinson

R. K.

Christopherson

R. M.

Joseph

S. S.

Harrison

C. J.

(2013). Animated agents and learning: Does the type of verbal feedback they provide matter? Computers & Education, 67, 239–249. https://doi.org/10.1016/j.compedu.2013.04.017

24.

Liu

H. Y.

(2019). Advanced statistics for psychology. China Renmin University Press.

25.

Loibl

Leuders

(2019). How to make failure productive: Fostering learning from errors through elaboration prompts. Learning and Instruction, 62, 1–10. https://doi.org/10.1016/j.learninstruc.2019.03.002

26.

Loibl

Rummel

(2014). Knowing what you don’t know makes failure productive. Learning and Instruction, 34, 74–85. https://doi.org/10.1016/j.learninstruc.2014.08.004

27.

Maier

Wolf

Randler

(2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85–98. https://doi.org/10.1016/j.compedu.2015.12.002

28.

Mayer

R. E.

(2014). Cognitive theory of multimedia learning. In Mayer

R. E.

(Ed.), The Cambridge handbook of multimedia learning (2ed., pp. 43–71). Cambridge University Press.

29.

Mertens

Finn

Lindner

M. A.

(2022). Effects of computer-based feedback on lower- and higher-order learning outcomes: A network meta-analysis. Journal of Educational Psychology, 114(8), 1743–1772. https://doi.org/10.1037/edu0000764

30.

Metcalfe

(2017). Learning from errors. Annual Review of Psychology, 68(1), 465–489. https://doi.org/10.1146/annurev-psych-010416-044022

31.

Metcalfe

Miele

D. B.

(2014). Hypercorrection of high confidence errors: Prior testing both enhances delayed performance and blocks the return of the errors. Journal of Applied Research in Memory and Cognition, 3(3), 189–197. https://doi.org/10.1016/j.jarmac.2014.04.001

32.

Mory

E. H.

(2004). Feedback research review. In Jonassen

(Ed.), Handbook of research on educational communications and technology (pp. 745–783). Erlbaum.

33.

Narciss

(2008). Feedback strategies for interactive learning tasks. In Spector

J. M.

Merrill

M. D.

van Merrienboer

J. J. G.

Driscoll

M. P.

(Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 125–144). Lawrence Erlbaum Associates.

34.

Narciss

(2013). Designing and evaluating tutoring feedback strategies for digital learning environments on the basis of the interactive tutoring feedback model. Digital Education Review, 23(1), 7–26. https://doi.org/10.1344/DER.2013.23.7-26

35.

Narciss

Prescher

Khalifah

Korndle

(2022). Providing external feedback and prompting the generation of internal feedback fosters achievement, strategies and motivation in concept learning. Learning and Instruction, 82(2), Article 101658. https://doi.org/10.1016/j.learninstruc.2022.101658

36.

Ohlsson

(1996). Learning from performance errors. Psychological Review, 103(2), 241–262. https://doi.org/10.1037/0033-295X.103.2.241

37.

Pardo

Jovanovic

Dawson

Gasevic

Mirriahi

(2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592

38.

Richey

J. E.

Andres-Bray

J. M. L.

Mogessie

Scruggs

Andres

J. M.

Star

J. R.

Baker

R. S.

McLaren

B. M.

(2019). More confusion and frustration, better learning: The impact of erroneous examples. Computers & Education, 139(2), 173–190. https://doi.org/10.1016/j.compedu.2019.05.012

39.

Schroeder

N. L.

Kucera

A. C.

(2022). Refutation text facilitates learning: A meta-analysis of between-subjects experiments. Educational Psychology Review, 34(2), 957–987. https://doi.org/10.1007/s10648-021-09656-z

40.

Shute

V. J.

(2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

41.

Strijbos

J. W.

Narciss

Dünnebier

(2010). Peer feedback content and sender’s competence level in academic writing revision tasks: Are they critical for feedback perceptions and efficiency? Learning and Instruction, 20(4), 291–303. https://doi.org/10.1016/j.learninstruc.2009.08.008

42.

Strijbos

J. W.

Pat-El

Narciss

(2021). Structural validity and invariance of the feedback perceptions questionnaire. Studies In Educational Evaluation, 68(7), Article 100980. https://doi.org/10.1016/j.stueduc.2021.100980

43.

Sweller

van Merriënboer

J. J. G.

Paas

(2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5

44.

Tabachnick

B. G.

Fidell

L. S.

(2013). Using multivariate statistics (6th ed.). Pearson.

45.

Tippett

C. D.

(2010). Refutation text in science education: A review of two decades of research. International Journal of Science and Mathematics Education, 8(6), 951–970. https://doi.org/10.1007/s10763-010-9203-x

46.

Van der Kleij

F. M.

Feskens

R. C.

Eggen

T. J. H. M.

(2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511. https://doi.org/10.3102/0034654314564881

47.

Vosniadou

Verschaffel

(2004). Extending the conceptual change approach to mathematics learning and teaching. Learning and Instruction, 14(5), 445–451. https://doi.org/10.1016/j.learninstruc.2004.06.014

48.

Wang

Gong

S. Y.

X. E.

(2019). Elaborated feedback and learning: Examining cognitive and motivational influences. Computers & Education, 136, 130–140. https://doi.org/10.1016/j.compedu.2019.04.003

49.

Winstone

N. E.

Hepper

E. G.

Nash

R. A.

(2019). Individual differences in self-reported use of assessment feedback: The mediating role of feedback beliefs. Educational Psychology, 41(7), 844–862. https://doi.org/10.1080/01443410.2019.1693510

50.

Winstone

N. E.

Nash

R. A.

Rowntree

Parker

(2017). It’d be useful, but I wouldn’t use it: Barriers to university students’ feedback seeking and recipience. Studies in Higher Education, 42(11), 2026–2041. https://doi.org/10.1080/03075079.2015.1130032

51.

Zengilowski

Schuetze

B. A.

Nash

B. L.

Schallert

D. L.

(2021). A critical review of the refutation text literature: Methodological confounds, theoretical problems, and possible solutions. Educational Psychologist, 56(3), 175–195. https://doi.org/10.1080/00461520.2020.1861948