Scaffolding self-regulated learning through student-generated quizzes

Self-regulated learning

Self-regulated learning describes the role of students in regulating and managing their own learning process. While there is no simple or straightforward definition of the construct (Boekaerts and Corno, 2005), self-regulated learning is generally understood to include the thoughts, feelings, and actions related to the students’ attainment of personal goals (Zimmerman, 2000). There are four core assumptions that most self-regulated learning models share (Pintrich, 2004). First, a self-regulated learning approach assumes that learners are active agents that construct meaning, goals, and strategies. Second, learners have the potential to monitor, control, and regulate their learning, specifically they can regulate their cognition, motivation, and behavior. Third, the learning process can be measured against a goal or standard such that learners can know whether progress is being made. Finally, self-regulated learning assumes that learners mediate the learning process as it relates to personal characteristics and to the classroom environment. In short, this self-regulated learning is an active, dynamic, and complex process.

A self-regulated learning–informed approach to teaching requires a comprehensive perspective of students, including an awareness of cognitive, motivational, affective, and social contextual factors (Pintrich, 2004; Vermunt and Vermetten, 2004). Cognitive processing activities are related to learning outcomes such as knowledge and understanding and, up until the mid-1980s, were the focus of the scholarship on teaching and learning. It is now understood, however, that cognition is only one aspect of the overall learning process. Affective processing activities also influence the overall learning process. For example, students’ emotions and affective states may positively or negatively influence their ability to learn (see, for example, Boekaerts, 1995). Social context also matters, and students may seek belonging, social support, safety, entertainment, and self-determination goals simultaneously as they pursue more cognitive goals (Boekaerts, 2006). Self-regulation activities can mediate cognitive and affective learning and, consequently, are an important part of the overall learning process (Vermunt and Vermetten, 2004). While the dynamics of self-regulation may shift contextually, students generally employ one of the three regulation strategies: (a) students self-regulate their learning, (b) students allow the learning environment (including the teacher) to regulate their learning, or (c) students lack both self-regulation and external regulation.

The development of self-regulated learning skills can be either top-down or bottom-up (Boekaerts and Corno, 2005). Top-down self-regulation occurs when students autonomously steer their learning process toward their goals. This behavior is typically motivated by students’ personal interest, values, expected satisfaction, and rewards. Bottom-up self-regulation occurs when the environment provides the reward structures that promote self-regulated learning. The role of the teacher and of the learning environment, then, can play a key role in students’ development of self-regulated learning skills. For example, self-regulation can be promoted by allowing students to become part of the decision-making process (Ratelle et al., 2007; Taylor and Ntoumanis, 2007) and by providing tasks that are relevant, meaningful, and that encourage effective learning strategies (Alexander and Wade, 2000).

Instructional choices and self-regulated learning

Instructional choices can activate self-regulated learning to varying degrees (Stefanou et al., 2013; Vermunt and Vermetten, 2004). On the positive side, teachers can facilitate students’ use of new learning and thinking strategies, a process described as constructive friction (Trigwell et al., 1999), and these new skills can be incorporated into the students’ repertoire of self-directed skills. On the negative side, destructive friction occurs when teachers take responsibility for learning activities that the student was previously performing without aid or when the teacher introduces learning activities that are too advanced for the students. In destructive friction, the teacher’s behavior can decrease the student’s use of independent learning strategies (Clark, 1990; Lunenberg and Volman, 1999).

A pedagogical strategy can be both constructive and destructive, depending on the students in the classroom. For example, problem-based learning (PBL) is a structured pedagogical activity designed to increase students’ critical thinking skills and ability to work in a team. Students engage in individual study and, then, in team-based activities. Usually, these team-based activities focus on the complex nuances of a problem and require deeper thought. The effectiveness of a team is related in part to how individual students engage in the self-regulated learning activities of time planning and self-monitoring. In one study of 165 first-year psychology students, those students with more skills in these areas (time planning and self-monitoring) were more efficient in individual study time, better prepared for group time, and scored higher on cognitive tests (Van Den Hurk, 2006). This suggests that the friction created by a learning strategy such as PBL can be constructive for some students and destructive for others based on students’ self-regulation skills. This finding underscores the role of the instructor in scaffolding low-risk opportunities for students to learn self-regulated learning strategies.

Student-generated quizzes and self-regulated learning

One low-risk way to scaffold student self-regulation is to invite students to generate questions for an upcoming quiz or exam. The pedagogical strategy of having students generate quiz questions first emerged in the literature in the 1980s (see, for example, Green, 1997; Rakes and Smith, 1987). The process of creating quiz questions can be a fun, role-reversing activity that supports higher-order thinking regarding the material (Bates et al., 2013) and the students’ own learning process (Warburton and Volet, 2012). Rather than just memorize or apply the material, students must think about how to evaluate their knowledge of the material (for more information, see Bloom, 1956). In short, they must think strategically about what is important to remember or take away from the material. This activity provides what Hogan and Pressley (1997) describe as scaffolding: instructional support that encourages students to function at their highest cognitive capacity. In short, it is an assessment strategy that promotes learning (Boud and Falchikov, 2005).

With supportive scaffolding, students can produce high-quality questions (Bates et al., 2013; Chin and Brown, 2002; Gillespie, 1990). In one experiment, a series of activities were designed to help students of two classes learn how to write high-quality questions. These activities included a quiz designed to demonstrate poorly written questions, a self-diagnosis quiz to help students see the difference between an orientation to learning and an orientation to performance, a question template that encouraged higher-order thinking, and an example question that set a high standard for creativity and complexity (Bates et al., 2013). More than 600 student-generated questions were reviewed and evaluated the questions based on three criteria: (a) the cognitive challenge (based on Bloom’s taxonomy), (b) the quality of the explanation and solution provided for each question, and (c) the quality of question overall (that is, clarity, reasoning, etc.). The students’ questions, scholars concluded, were of high quality. More than 75% of the questions encouraged learning beyond the simple recall classification, approximately 60% were classified as being of high or outstanding quality in regard to response, and 75% met the overall quality criteria. One reason for the quality, it is suggested, is that ownership over questions encourages students to be more thorough.

The process of developing questions may support the low-performing students in particular. In one study, 68 students who had completed a series of four examinations were surveyed, one of which included both lecturer- and student-generated questions (Jobs et al., 2013). Students were asked to state (a) the number of questions they had constructed, (b) the number of hours they had studied, and (c) the amount of time they had spent using variable learning methods. High- and low-performing students were identified based on student responses to lecturer-generated questions. It was found that the low-performing students had statistically significant better results when answering student-generated questions (+19% for low performers vs +6.1% for high performers). There is some question about how to interpret these results. For example, the authors suggested that writing questions calls for a more intense approach to learning because the question writer must reflect on their own perspective. However, the authors also surmised that low-performing students might have just memorized the question bank, resulting in a higher test score but lower long-term understanding.

One way to help students connect the lessons from one learning activity to their overall learning process is to create opportunities for self-reflection. One study asked 94 first-year students to work in self-selected teams of 2–4 to produce a five-question self-test for the class (Warburton and Volet, 2012). Students were instructed to create matching, short answer, true/false, and/or multiple choice questions and to use at least one image. Afterward, students wrote a 150- to 200-word reflection piece describing the extent to which they felt that they had achieved the learning objectives of the group assignment. More than 80% of the students in this study indicated that this activity enhanced their learning of the content and 67% indicated that the activity helped them develop learning skills. Comments also reveal that some students were developing an awareness of their learning processes (metacognition) and that targeted assessment strategies were a useful tool in enhancing this awareness. This awareness was likely prompted and supported by the reflective activity.

Clearly, there are many potential pedagogical benefits to having students develop quiz questions, and there is, in any case, a need for further study. In particular, it is still unclear how students perceive this experience. The purpose of this study, therefore, is to identify students’ perceptions about the experience of developing questions for a quiz or examination. In particular, the study addressed the following questions:

Methodology

Findings

Perceived benefits

Students reported that quiz-development activities improved their learning (74%) and allowed for student-centered learning (25%). Many students liked the idea that they might know the quiz questions in advance (13%). Others believed that this activity produced better quiz questions (11%) in that the questions would not be tainted by the professor’s bias and would contain more contemporary wording. Some students also indicated that this process helped them take the perspective of the professor and develop empathy for that position (6%), which is an important finding for a leadership class. Finally, some students indicated that it was a good opportunity to challenge their peers (4%), another interesting finding for a leadership class. It appears that for at least some of the (presumably high-performing) students, developing quiz questions was a way of exercising scholastic leadership. The perceived benefits can be seen in Figure 1, and selected quotes are presented in Table 1.

OPEN IN VIEWER

Figure 1.

Perceived benefits.

OPEN IN VIEWER

Table 1.

Perceived benefits quote matrix.

Perceived benefit	Sample quotes
Improved learning (74%)	“Really allowed for us to think about what we had learned and reflect back on the topics we have covered.” “Made us think deeply about how concepts could be applied.” “Loved it! Helped us engage in the quiz and actively participate and as a result we did better on the quizzes.”
Empowered student-centered learning (25%)	“Also we were able to aid the class overall, we were able to contribute to our own learning.” “It allowed us to determine what we believed was important, which I think is beneficial for learning.”
Produced better quizzes (11%)	“Some professors may be biased and focus on one or the other but different students will focus on all the parts of the class.” “Students are thinking differently about the material than a professor, so it is easier as a student to understand.” “I think a major benefit is it allows for more contemporary wording of questions, and questions that students actually find relevant.”
Fostered empathy for the role of the professor (6%)	“Gave us a chance to understand how it can be difficult to write good questions.” “These activities allowed us to take a step back and put ourselves in Dr. [Name Redacted] shoes when we were to create quiz questions. It challenged us to see the material in a new light.”
Provided opportunity to challenge classmates (4%)	“Helped me challenge myself and my peers.” “The other benefit was being able to be creative with the questions and trying to get other students to think a little bit more about what we were learning.”

Perceived drawbacks

In total, 46% of the students did not or could not identify a single drawback (13% did not respond; 33% wrote “none”). In total, 39% were concerned that the process did not produce the best questions. A limited number of other students found it hard to write the questions (4%), did not like or see value in the activity (4%), wanted more time to produce the questions (3%), or were concerned it took up valuable class time (3%). The perceived drawbacks can be seen in Figure 2. Responses from the 39% of students who were concerned the process did not produce the best questions were analyzed. These students expressed the following concerns: 39% worried the questions were or could be too difficult, 21% worried the questions were or could be too easy, 25% were concerned the questions might not be the best representation of the material, and 11% worried the questions would be poorly written (Table 2). It is interesting to note that all student concerns can be mitigated in the instructors’ introduction to the activity. For example, the instructor can explain that the instructor will curate the questions to ensure high quality, and the instructor can explain the pedagogical value of this activity.

OPEN IN VIEWER

Figure 2.

Perceived drawbacks.

OPEN IN VIEWER

Table 2.

Perceived drawbacks quote matrix.

Perceived drawbacks	Process did not produce the best questions
Questions were/could be too difficult (43%)	“One of the drawbacks to this was that some of the questions that people made were either difficult or challenging to answer. I think that can be attributed to each person’s individual level of what is difficult and what is not difficult.” “We almost turned against ourselves because we would create really difficult and creative questions.”
Questions might not be the best representation of the material (25%)	“The questions may not have been a representation of the most important things.” “The questions could literally go in any direction and may not be in the direction that is most beneficial for the quizzes.”
Questions were/could be too easy (21%)	“I was tempted to submit very easy questions at the beginning, and I’m sure that some of my peers were as well.” “These activities often made the quizzes easier than you probably wanted in my opinion.”
Questions might not be written well (11%)	“The only drawback of these activities would be that some of the questions that students came up with were difficult to answer or comprehend.”

Preference for activity

Students were asked their preference for the three different versions of the quiz-development activity: in class individual, in class team, and online individual. Students identified clear benefits to all three options (Table 3). However, students preferred the team-based version of this activity preferable to working individually (Table 3). In total, 51% of the students also perceived the team-based version to be more conducive to learning (Figure 3). It is unclear from the data whether high- and low-performing students differed in regard to their preference.

OPEN IN VIEWER

Table 3.

Preference for activity quote matrix.

Activity	Preference	Sample quote
Students worked in teams to create 2–3 higher-order quiz questions.	42%	“I preferred this activity because working in teams allowed us to discuss the material more in-depth, which led to a deeper understanding.”
Each student submitted one quiz question on an index card.	34%	“It took some pressure off of what other students would think of my questions. It really forced me to work independently and come up with my own questions. I had to know the material.”
Each student submitted one or more quiz questions via canvas as extra credit.	24%	“This allowed for the least social loafing and groupthink, so each quiz idea was original and more material was reviewed for the quiz.”

OPEN IN VIEWER

Figure 3.

Student perceptions on which was more helpful for learning: working in teams or as individuals.

Influence on study habits

In total, 47% of the students reported that this process had not changed how they study. In total, 35% reported that there had been a change, and 18% of the students did not state. One of the students who indicated that this process had not affected their studying wrote, “I never really considered it a form of studying, but now that you mention it, it does sound like a great way to review the material.” This quote indicates that the survey itself served as a sort of reflection exercise that, in a small way, may have scaffolded the progression of the students’ self-regulated learning skills.

OPEN IN VIEWER

Figure 4.

Student perceptions on whether these activities influenced how they approached studying.

Discussion and conclusion

This study examined the perceptions of undergraduate students about their experience of developing questions for a quiz. Students reported that the experience of developing questions improved their learning and, in particular, empowered student-centered learning. They generally preferred a team-based process over creating questions as individuals. Students also reported some concerns with the activity, including a concern that the process did not produce the best question. About one-third of the students reported that the quiz-development activities changed their study habits. Overall, they felt this sort of activity should be repeated in the future.

One of the core assumptions behind most self-regulated learning models is that learning can be measured such that students assess whether progress has been made (Pintrich, 2004). However, in most learning environments, the assessment of learning typically remains the domain of the instructor. The dynamic is that students learn and teachers assess. This unilateral dynamic, however, may inadvertently reinforce a teacher-directed approach to learning rather than facilitate self-regulated learning. The process of developing a quiz or exam question invites the student to temporarily put themselves in what is normally the teacher’s position and ask, “What is important for me to learn in this module?” This question is at its essence an act of self-regulated learning.

The process of developing quiz questions is, for students, an activity of bottom-up (that is, environment supported; Boekaerts and Corno, 2005) self-regulated learning. Instructors must be careful when assigning activities that require or favor self-regulated learning as not all students will have the same skill set. In any given class, some students will have more of those skills than others. Active learning activities can potentially teach students self-regulated learning skills; however, they can also favor students who have already developed those skills (Van Den Hurk, 2006). Thus, it is critical teachers identify low-risk, non-graded opportunities for students to practice these skills. Developing quiz questions is one example of a low-risk opportunity. Doing this in groups can provide additional support to students. In particular, when conducted in pairs or small groups, this activity can facilitate peer-to-peer feedback which is an important part of the overall feedback available to students in the classroom (Pokorny and Pickford, 2010).

Self-regulated learning requires self-reflection, and, as such, it is important to allow students to practice reflecting on their learning activities. This is critical to students’ full absorption of the learning. In one study (Warburton and Volet, 2012), developing quiz questions helped students become aware of their learning process; however, this awareness came after the activity had been completed, and the students were writing a 200-word reflection piece. In the study described in this article, the survey offered an unplanned opportunity for reflection. At least one student’s response suggested that without the survey, they would not have considered how this activity might be used to improve their study habits.

As with all studies, there are some limitations. This study was conducted in one undergraduate, social science class in a large, state-funded university in one particular country/cultural context. The findings may or may not be generalizable to other disciplines or to other types of universities or contexts. Additionally, this study focused on students’ perceptions, and perceptions may or may not be reality. For example, there may be a difference in whether students perceived this activity changed their study habits and whether or not their study habits actually changed. Also, this study investigated students’ responses to three different quiz-development activities, and this medley of activities may have influenced their reactions. For example, responses to the question on perceived benefits may have been directed to one specific activity rather than the general category, and the order of activities may have influenced students’ perceptions. That said, the focus on perspectives does provide an important lens through which to see this sort of activity, and the findings do suggest that this activity is worth experimenting with in different settings.

Future research should consider how students’ perceptions relate to students’ actual self-regulated learning skills. For example, are students with higher self-regulated learning skills more likely to enjoy this activity and how, if at all, does this activity influence self-regulated learning in future classes. Additionally, it is important to know the extent to which quiz-development activities contribute to student learning. We know from other studies that quiz-development activities improve the grades of low-performing students more than high-performing students, but it is unclear whether it is because low-performing students gained a deeper understanding of the material or whether they just memorized possible quiz questions (Jobs et al., 2013). Finally, this study investigated one low-risk, ungraded procedure. It is likely that many low-risk activities are needed to develop self-regulated learning. Future studies should identify and investigate other low-risk opportunities.

Quiz-development activities are a valuable way to scaffold self-regulated learning. Students have the capacity to develop high-quality questions and, with the support of reflective exercises, increase their metacognition regarding their learning. This study extends research of quiz-development activities by incorporating an important dimension: student perceptions. Students in this study articulated clear benefits to these activities, and the few concerns they reported could be mitigated through instructions. Even with a low-risk activity such as this, clear instruction can both assuage the students’ concerns and provide the sense of structure for which students often feel a deep need (Knowles, 1975). Instructors are encouraged to experiment with low-risk opportunities to help students develop self-regulated learning skills and to provide opportunities for reflection that help students consolidate their newfound perspective.