The Positive Effect of Intrinsic Feedback on Motivational Engagement and Self-Efficacy in Information Literacy

Types of Feedback and the Influence of Feedback

Students can get feedback from both external and internal sources, which are called extrinsic and intrinsic feedback. Extrinsic feedback takes the form of external comment on an action, which the teacher has traditionally provided, whereas intrinsic feedback is an inherent consequence of an action and arises from students’ engagement in a task or activity (Hounsell, 2003). The trouble is that most teachers have difficulty finding time to give all students the feedback they need, when they need it (Andrade, 2007). This is especially true in large classes, where there are too many students, and teachers may feel that giving students feedback is a large burden. Prior studies have suggested that students themselves can be excellent sources of feedback (Andrade, 2007; Andrade & Valtcheva, 2009). Student self-assessment under the right conditions can provide accurate, useful information to promote learning, because it involves students in thinking about the quality of their own work rather than relying on teachers as the sole source of evaluative judgment (Andrade & Valtcheva, 2009).

There have been studies which explored the influence of feedback on various students’ learning outcomes and reported that feedback can have either positive or negative effects. For example, one review of the literature on classroom formative assessment (Black & William, 1998) revealed that feedback produced significant benefits in learning and achievement across all content areas, knowledge and skill types, and levels of education in schools. Other findings indicated that students who received feedback were more accurate in their self-evaluative judgments than students who received neither type of feedback (Labuhn et al., 2010).

Research on Formative Assessment’s Effects on Students’ Outcomes

A number of studies demonstrated that using rubrics as a formative assessment tool has a positive effect on learning and performance (Panadero & Jonsson, 2013), and can be an effective tool for making learning goals and evaluation criteria explicit for both teachers and students (Andrade & Boulay, 2003; Andrade & Du, 2007). Reitmeier, Svendsen, and Vrchota (2004) found that food science students who used rubrics for repeated teacher-, self-, and peer-assessment of oral communication skills had a higher presentation score than those who did not use them. Similarly, undergraduate and graduate business students in Bolton’s (2006) study asserted that rubrics enabled them to engage in identifying critical issues in an assignment, including reducing uncertainty, doing more meaningful work, determining the amount of effort needed for an assignment, and evaluating their own performances to get immediate feedback. On the contrary, some studies showed no differences between the quality of work done by students who work with rubrics and those who work without them. For example, Green and Bowser’s (2006) study showed that there were no differences between students’ scores on the literature review that they wrote with rubrics and the one they wrote without them. Panadero and Jonsson (2013) reviewed the studies on the formative use of rubrics and found that although few in number, the studies indicated that rubrics might be valuable in supporting student learning, at least in combination with different meta-cognitive activities (such as self-regulation, self- or peer-assessment). They suggested further investigation to assess the effectiveness of rubrics as tools for self-assessment (Panadero, Tapia, & Huertas, 2012).

To our knowledge, no study has examined the influence of intrinsic feedback from using rubrics as a self-assessment tool on a variety of cognitive and behavioral learning outcomes in the area of information literacy. In Thailand, where we conducted the study, universities require undergraduate students to acquire information literacy skills, which are framed by the 21st century’s global information literacy skills. According to Information Literacy Competency Standards for Higher Education (The Association of College and Research Libraries, 2000), information literacy is “a set of abilities requiring individuals to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information” (p. 2). Information literacy

forms the basis for lifelong learning. It is common to all disciplines, to all learning environments, and to all levels of education. It enables learners to master content and extend their investigations, become more self-directed, and assume greater control over their own learning. (The Association of College and Research Libraries, 2000, p. 2)

Even though universities expect undergraduate students to be information literate, studies have shown that students commonly have trouble effectively accessing information they need (e.g., formulating keyword searches in unfamiliar content areas (Bos et al., 2000) and seem to rely on one-keyword searches to find the answers to all their questions (Timmers & Glas, 2010). Accordingly, we addressed the question of whether using rubrics for student self-assessment can have a positive effect on their information literacy skills, and foster self-regulated learning in which they have control over their metacognition, motivation, and behavior.

Information Literacy–Related Outcomes

We measured information literacy–related outcomes in terms of the following constructs:

Research Question

We designed our investigation to answer the following research question:

Does formal classroom use of a rubric as a tool for self-assessment influence cognitive and behavioral outcomes measured as “motivational engagement in information literacy,” “self-efficacy in information literacy,” and “information-seeking behavior”? We collected data for our study from first-year undergraduate students enrolled in an Information Literacy Skills course within a general education program, at an institution of higher education in Thailand.

Research has shown that students are more likely to report a higher sense of self-efficacy as they develop objectivity in relation to standards and expectations (Marsh & Yeung, 1997; Nicol & Macfarlane-Dick, 2006). The rationale for investigating the benefits of using the course rubric as a formal self-assessment tool is that the internal feedback they gain from using a rubric as a self-assessment tool contributes to student self-reflection, self-assessment, and self-regulation (Sadler, 1998). It is known that when students monitor their work, and take an active role in the management of their own learning, internal feedback becomes an element of motivational engagement and self-regulated learning (Butler & Winne, 1995). We therefore hypothesized that formalized classroom use of a course rubric to self-assess individual progress will have positive effects on cognitive and behavioral outcomes.

Method

Experimental Design

All participants from the six classes completed the prequestionnaire in the first week of the information literacy skills course. We treated three classes as a control group and the other three as an experimental group. A teacher assigned students to work in groups (three students per group) for a project on reviewing the literature related to topics they had chosen, and writing a review article. Students in both control and experimental groups received feedback from their teacher. The teachers for both groups worked together to establish and ensure consistency in the criteria used to give feedback. Students in the experimental group took part in a formalized classroom activity in which they used rubric as a tool for guiding student self-assessment (see Figure 1).

OPEN IN VIEWER

Figure 1.

Experimental procedure.

In the experimental group, students got a rubric with information regarding its meaning, which can help students to self-assess their own work (see Figure 2). To address students’ understanding, the teacher then gave students an example of the work and asked all of them to practice by assessing and scoring this work. This way students could test whether they had understood how to use a rubric. Then they used the rubric to self-assess their own work during the class period. After getting feedback, both groups of students revised and resubmitted their complete projects again.

OPEN IN VIEWER

Figure 2.

A rubric given to students for self-assessment.

Analysis

When conducting the quasi-experimental research, potential bias—which is called “selection bias”—can occur. There is a possibility that those who are eligible or chosen to participate in the experimental group are systematically different from those who are in the control group. Observed differences between the two groups may be due—in full or in part—to an imperfect match rather than intervention. Therefore, to ensure that the average characteristics of both groups were similar, we used the propensity score matching (PSM) approach in this study (White & Sabarwal, 2014). We used the PSM technique to select control cases, who matched with treated cases based on controlled background covariates. We estimated the participation equation by using preintervention values. We treated motivational goal orientation as a controlled covariate because the literature states that it can moderate students’ cognitive and affective outcomes in a real classroom situation (Dweck & Leggett, 1988; Panadero et al., 2012).

To perform the PSM procedure, we used the MatchIt in R package. We matched treatment cases (n = 263) with their nearest “neighbors” in the control group (n = 321; those with the closest propensity scores). After performing PSM, the matched cases numbered 526 (n = 263 for the control group and n = 263 for the experimental group). Results from the PSM procedure indicated a good improvement in all controlled covariates (see Table 1).

OPEN IN VIEWER

Table 1.

Summary Statistics of Matching Success.

Controlled covariates	Mean differences		Percentage of balance improvement
	Before matching	After matching
Mastery goal orientation	−.0098	−.0076	22.2917
Performance approach goal orientation	−.1448	−.0485	66.5283
Performance avoidance goal orientation	−.2798	−.0970	65.3507

We then calculated means and standard deviations for all variables of the pre- and posttest in both the control and experimental groups. To see the changes within the control group, we performed a paired samples t test to reveal the mean difference of each variable (posttest minus pretest) and determine whether the difference was statistically significant. We carried out the same process for the experimental group. Then to investigate whether the changes in the experimental group were significantly different from the changes in the control group, we performed t test analysis again, and computed effect sizes, Cohen’s d values, to measure the strength of the t test result. According to Cohen’s (1988) definition, a d value of 0.2 indicates a small effect size, 0.5 indicates a medium effect size, and 0.8 indicates a large effect size.

Results

Table 2 presents mean and standard deviation values of the pre- and post-survey for both the control and experimental groups and the t values which indicate whether mean differences between pre- and post-survey are statistically significant. The last two columns indicate the differences in the changes between groups and the effect sizes.

OPEN IN VIEWER

Table 2.

Means, Standard Deviations, and the Differences of Pre- and Post-Survey of the Control and Experimental Groups and the Group Difference.

IL-related outcomes	Control group (n = 263)					Experimental group (n = 263)					Group difference
	Pre		Post		t	Pre		Post		t	t	Cohen’s d
	M	SD	M	SD		M	SD	M	SD
Motivational goal orientation (scale 1-5)
Mastery	4.12	0.39	4.04	0.42	−3.23**	4.11	0.40	4.12	0.39	0.37	2.57*	0.22
Performance approach	3.28	0.65	3.27	0.72	−0.29	3.23	0.67	3.23	0.73	−0.07	0.16	0.01
Performance avoidance	2.66	0.57	2.72	0.69	1.37	2.56	0.61	2.57	0.69	0.05	−0.94	0.08
Motivation engagement in IL (scale 1-9)
Search	7.69	1.14	7.78	1.04	1.05	7.87	1.09	8.04	0.90	2.27*	0.74	0.06
Evaluation	7.50	1.26	7.71	1.06	2.80**	7.59	1.09	7.76	1.02	2.45*	−0.41	0.03
Processing	7.19	1.63	7.45	1.32	2.36*	7.24	1.61	7.69	1.25	4.73***	1.31	0.11
Communication and dissemination	8.20	0.99	8.25	0.93	0.92	8.40	0.78	8.53	0.61	2.70**	0.86	0.09
Self-efficacy in IL (scale 1-9)
Search	6.40	1.25	6.54	1.21	1.56	6.07	1.31	6.47	1.13	5.10***	2.23*	0.19
Evaluation	5.71	1.24	6.22	1.07	6.36***	5.52	1.31	6.41	1.17	10.69***	3.26**	0.29
Processing	4.81	1.64	5.37	1.58	5.18***	4.04	1.79	5.74	1.51	14.24***	7.12***	0.62
Communication and dissemination	5.74	1.35	6.27	1.06	6.41***	5.45	1.42	6.37	1.06	12.25***	3.34**	0.30
Information-seeking behavior (scale 1-4)
Applying search strategies	2.63	0.46	2.81	0.46	5.39***	2.55	0.54	2.79	0.44	7.52***	1.28	0.11
Evaluating information on websites	2.51	0.58	2.76	0.56	6.41***	2.54	0.69	2.75	0.57	5.56***	−0.93	0.06
Referring to information	2.30	0.54	2.56	0.63	6.15***	2.11	0.59	2.75	0.64	13.98***	6.01***	0.52

Note. IL = information literacy.

*

p < .05. **p < .01. ***p < .001.

Discussion

This study aimed to understand whether formalized classroom use of a course rubric to self-assess individual progress can promote students’ information literacy–related outcomes. The results showed that overall, either by using a rubric (experimental group) or without using one (control group), students’ outcomes improved. However, our comparison of the control and experimental groups found significant differences between the groups in changes in self-efficacy beliefs. On average, students in the experimental group reported significantly higher increases in self-efficacy in information literacy than the students in the control group. As the conclusion of the review by Kluger and DeNisi (1996) stated, feelings of self-efficacy are important mediators in feedback situations. Feedback is effective to the degree to which it directs information to enhance self-efficacy and to enable more effective self-regulation. When students have the meta-cognitive skills of self-assessment, they can evaluate their levels of understanding, their effort and strategies used on tasks, their attributions and opinions of others about their performance, and their improvement in relation to their goals and expectations (Hattie & Timperley, 2007).

According to self-efficacy theory (Bandura, 1982), mastery learning experiences are the major source for higher self-efficacy. Self-assessment is an example of mastery learning experience. Feedback, internal or external, can identify “the discrepancy between current and desired understanding” (Hattie & Timperley, 2007, p. 86) to help the student bridge the gap between the two. When students use a rubric to assess their learning, it gives them an opportunity to monitor their task engagement and assess progress toward goals (Schunk, 2003). Research has shown that this kind of mastery experience helps students develop positive beliefs about their capabilities. Feedback through formative assessment tools, such as a rubric, helps a student to recognize his or her strengths and weaknesses over time and become a stronger, more self-regulated learner. Our findings on the benefits of rubrics are consistent with the evidence from prior research (Andrade & Boulay, 2003; Andrade & Du, 2007; Andrade & Valtcheva, 2009; Panadero et al., 2012). Andrade and Du (2007), for example, reported that undergraduate students feel more motivated and confident about their work when using criteria-referenced self-assessment.

The findings highlighted important implications for teachers, as self-efficacy is an important motivational construct which can influence individual choices, goals, emotional reactions, effort, ability to cope, engagement, persistence, and achievement (Gist & Mitchell, 1992). For example, students who feel efficacious about learning generally expend greater effort and persist longer than students who doubt their capabilities, especially when they encounter difficulties (Bandura, 1982). As this study has proven self-efficacy to be malleable and that one can increase it by using rubrics as self-assessment tools, recognizing and increasing students’ self-efficacy could eventually lead them to work harder, in worse conditions than their counterparts with lower self-efficacy (Bandura, 1982). The benefits of self-assessment may be even more significant in Thai classrooms, where the class size tends to be large and students have limited opportunities for one-on-one interaction with the teacher. Thus, using rubrics as the students’ self-assessment method is feasible and powerful in practice. Teachers can provide rubrics to students not only for use in self-assessment but also for peer-assessment.

This study has limitations. First, we collected data using self-reported methodology rather than explanatory methodology. The findings, therefore, do not establish cause-and-effect relationships between self-assessment and students’ information literacy–related outcomes. Second, we did not collect data over time on a single variable. We used a cross-sectional methodology to study differences in mean scores among the experimental and control groups on several measures. We suggest researchers use longitudinal methodology to learn more about cause-and-effect relationships between the use of self-assessment and changes in self-efficacy beliefs. Third, low alpha values on the mastery goal orientation scale and the regulation activities when applying search strategies scale are obvious limitations to the present study. Finally, there were two different teachers who participated in this study. It is possible that differences between the two teachers (in teaching style, experience, and teaching quality, etc.) may have unavoidably affected students’ pre–post intervention changes differently.

Conclusion

The findings reported in this study have clear implications for both theory and practice. Using a rubric as students’ self-assessment tool provides intrinsic feedback so that they can evaluate their levels of understanding, understand their current situations and the designated goals, and evaluate their effort and strategies used on tasks which can boost their self-efficacy in learning tasks. Teachers can apply rubrics in classrooms for both self-assessment and peer-assessment purposes, especially in classes with high levels of enrollment.