Supporting the Self-Regulated Learning of Primary School Students With a Performance-Based Assessment Platform for Programming Education

Abstract

This study implemented and evaluated the innovative use of a performance-based assessment platform to support the development of self-regulated learning (SRL) in senior primary students as they completed programming tasks. We embedded SRL support features into a performance-based assessment platform as scaffolding to help the students implement problem-solving strategies. A mixed-methods approach was adopted to evaluate the intervention. The students’ perceptions of their SRL skills after working through the programming tasks were measured by a survey of 45 students. The quantitative results suggested that the students benefited from the performance-based assessment platform in developing their SRL skills. A thematic analysis of interview data from 20 students further indicated that the embedded SRL scaffolding and automatic marking function helped them to solve the programming tasks. The results demonstrate that a well-designed performance-based assessment platform with embedded SRL support can be an effective tool for developing students’ SRL. The qualitative results further revealed that algorithmic thinking is an aspect of programming for which students need more SRL support.

Keywords

programming education scaffolding performance-based assessment primary students self-regulated learning

Introduction

Research has shown that students’ academic success is positively related to their self-regulated learning (SRL) skills (Broadbent & Poon, 2015; Cho & Shen, 2013; Samruayruen et al., 2013). For successful learning, students need to be proficient in SRL strategies such as setting learning goals, managing learning time, switching learning strategies, and seeking help when they get stuck in the learning process. Therefore, supporting students in developing these proficiencies helps them to achieve academic success (Hao & Tsikerdekis, 2019; Tabuenca et al., 2015; Zimmerman & Schunk, 2013). However, supporting students from kindergarten through to Grade 12 to become self-regulated learners is challenging, and self-regulation in the learning process requires extra attention beyond the learning content (Ayres & Paas, 2009; Chiu, 2021; Rivers et al., 2022).

Studies have revealed that SRL ability is closely related to programming ability (Bergin et al., 2005; Loksa & Ko, 2016). Students who use more metacognitive and resource management strategies in their learning might perform better in programming, and students equipped with better programming abilities might be more capable of deploying SRL strategies. Three popular tools that have been implemented to support SRL are prompts, feedback, and integrated support systems. Although these tools have been found effective, each has limitations. The effectiveness of prompts is related to learners’ prior knowledge, as learners with low prior knowledge benefit less from prediction-based prompts. Feedback is usually applied to support SRL in combination with prompts and does not have a significant effect on learning outcomes on its own. In addition, the integrated support system is only effective when learners use the tools and support provided. Of the three phases of SRL – forethought, performance, and self-reflection – the performance phase requires students to monitor their performance and to adjust their learning behaviour and strategies accordingly. Applying performance-based assessment in the performance phase of SRL is therefore a promising approach to supporting SRL development. However, to the best of our knowledge, there have been no studies of the application of performance-based assessment to support SRL.

Literature Review

Self-Regulated Learning

SRL is a set of active and volitional behaviours in which learners purposefully pursue predetermined goals through an iterative process (Pintrich, 2000; Winne, 2019; Zimmerman, 2000). Several fundamental models of self-regulation have been developed over recent decades (Bembenutty et al., 2013; Vohs, 2013; Zimmerman & Schunk, 2011). One widely applied model is Zimmerman’s (2000) three-phase SRL model, which comprises the phases of forethought, performance, and self-reflection. In the forethought phase, self-regulated learners set goals and plan their learning accordingly. In the performance phase, learners work on tasks while continuously monitoring their performance and modifying their strategies. For example, they might modify their strategies by assessing the difficulty of various tasks, findings ways of breaking down the problems, and determining when to seek help. In the reflection phase, learners reflect on their learning process and cognitive strategies. In this study, we adopted Zimmerman’s three-phase SRL model to guide us in the design and evaluation of the SRL support features to embed on the performance-based assessment platform.

Scaffolding for SRL

SRL support helps learners achieve academic success, and its importance is demonstrated by studies that have shown that learners who lack SRL support may not regulate their learning to obtain an adequate level of understanding when studying complex topics (Azevedo & Hadwin, 2005; Shin & Song, 2022; Wang et al., 2022). Various forms of computer-based scaffolding have been proposed, such as prompts, feedback, and integrated support systems. The use of prompts is an instructional strategy that encourages students to carry out specific self-regulation activities while performing a task (Bannert, 2009; Engelmann et al., 2021). Feedback is a method of promoting reflection based on a learner’s current learning status (Biesinger & Crippen, 2010; Wäschle et al., 2014). For the purposes of SRL, timely feedback enables learners to monitor their performance on demand and adjust their learning strategies accordingly. Integrated support systems are sets of features that are embedded in online learning environments to support various SRL strategies (Chen et al., 2014; Delen et al., 2014; Kim & Pedersen, 2011). For example, Delen et al. (2014) enhanced a video-watching environment with tools to support note-taking, searching for additional resources, and self-evaluation with reflective prompts. As integrated support systems may include prompts, feedback, and other SRL tools to implement various SRL strategies on an online platform, integrated support is a more versatile approach to supporting SRL. Integrated support systems are usually designed according to the learning context, the intended SRL strategies, and the features of the online platform.

Performance-based Assessment

Performance-based assessment, also called authentic assessment, is a testing method that includes a problem-solving process (Katz & Slomka, 2000; Messick, 1994; Torrance, 1995). It requires students to complete a task (such as creating a product or answering a question) on the assumption that specific, measurable attributes can be extracted from the outcome of the task to evaluate the extent of students’ learning toward the expected skill or knowledge (Gresse Von Wangenheim et al., 2021; Liu et al., 2002; Salma & Prastikawati, 2021; Xing et al., 2021). In the performance phase of SRL (Zimmerman & Moylan, 2009), it is vital for learners to effectively assess their current performance to determine whether to maintain or change their strategy (Panadero et al., 2018; Zimmerman, 2002). Because of this close connection between performance monitoring and strategy selection, performance-based assessment can be beneficial in fostering students’ SRL and evaluating their abilities in an authentic context (Ali & Hanna, 2022; Çakıroğlu et al., 2021; Van der Graaf et al., 2021). Performance-based assessment can motivate students’ learning, stimulate the expression of their creativity, and provide SRL opportunities by helping students link the desired outcomes to the regulation of their learning behaviour.

Computational thinking (CT) is an analytical thinking process that uses fundamental computing concepts and practices for problem-solving. A widely applied framework for assessing CT is that proposed by Brennan and Resnick (2012), which assesses CT from three dimensions: concepts, practices, and perspectives. However, it can be difficult to assess all three of these dimensions with a single measure. Accordingly, Koh et al. (2014) developed a cyberlearning tool for assessing CT that provided teachers with real-time patterns, on the basis of which they could make effective instructional decisions. Choi et al. (2017) designed a puzzle-solving program for teaching algorithm design. Çoban and Korkmaz (2021) developed an interactive online performance-based assessment tool to measure CT practice skills. In this study, we embedded SRL support functions into a performance-based platform for developing CT concepts and practices.

Although performance-based assessment is widely applied in training and fostering skills in various disciplines, there have been few studies of the use of performance-based assessment as scaffolding to develop SRL. Although the performance phase of SRL is a promising scenario for implementing performance-based assessment, careful design consideration must be given to how to enable students to monitor their performance effectively and efficiently. In this study, we explored the potential of a performance-based assessment platform enhanced with SRL support scaffolding and an automatic marking function to support SRL development.

Proposed Performance-based Assessment Platform with SRL Support for Programming Education

For this study, a set of SRL support functions were embedded into the EasyCode (Kong & Liu, 2020) performance-based assessment platform for developing CT (Wing, 2006). EasyCode provides a block-based programming environment and an online ‘judge’ that gives real-time feedback on the code submitted by students. It enables researchers and practitioners to design and administer programming tasks for performance-based assessment. In the performance phase of SRL, students work on specific tasks and need to monitor their own performance. Therefore, timely feedback on students’ tasks is important in this phase to help them reflect on their progress. EasyCode provides an environment for students to perform SRL while tackling the assessment tasks. An automatic marking feature was integrated into the performance-based assessment platform in this study to further support the students’ SRL in relation to their problem-solving process. Figure 1 presents a screenshot of the EasyCode user interface with the embedded SRL support features. The user interface consists of a series of foldable buttons and content containers. The left side of the window mainly provides information, with the work area representing an input-output programming model on the right. The following sections describe in detail the SRL support components that were embedded into the platform for this study.

Figure 1.

The User Interface of EasyCode with SRL Support Features Embedded.

Simplified Toolbox Support in EasyCode

Students with limited programming experience may be perplexed at the beginning of a problem-solving process. In particular, some students may find the choice of programming blocks overwhelming when attempting to construct a solution. The EasyCode platform thus provides a support mode, as shown in Figure 2. When this is active, the default toolbox in the programming workspace is replaced by a simplified toolbox that contains only the blocks needed to construct a solution. Support mode therefore enables students to implement a bottom-up strategy to solve the programming task. Students can find clues in the blocks provided by guessing the purpose of each and then construct a solution by arranging and connecting the blocks so that they function as intended.

Figure 2.

User Interface with Support Mode Active, Providing Only the Needed Blocks.

Prompt Support in EasyCode

Problem-solving prompts can be accessed for each programming task using the foldable ‘Tips’ button in the workspace, as shown in Figure 3. The tips aim to help students understand the goal of the task and identify the key concepts. They offer some background on the problem, including explanations of specific terms and notes about certain boundary cases.

Figure 3.

Prompts for SRL Support in the Form of ‘Tips’.

Prior Knowledge Acquisition Support in EasyCode

The programming tasks were categorised into three levels of difficulty, with Level 3 being the most difficult. Three Level 3 tasks were developed for this study, for which the task descriptions and model answers are shown in Table 1. These three tasks are complex for senior primary students because they require knowledge of CT concepts, such as data structures, CT practices such as testing and debugging, and algorithmic thinking. Therefore, 19 simpler tasks at Levels 1 and 2 were also included to equip students with the prerequisite knowledge. Each Level 3 programming task and its Level 1 and 2 subtasks were organised in a hierarchical structure to provide scaffolding for students’ problem-solving. To illustrate the structure, Figure 4 shows the relationship between the Level 3 task Triangle Classifier and the relevant subtasks that provide the prerequisite knowledge.

Table 1.

Descriptions and Model Answers for the Level 3 Programming Tasks.

Level 3 Task	Model Answer
Favourite Food: Write a program that can print the person’s favourite food according to the input name.
Triangle classifier: Write a program that can determine the type of a triangle according to the three input angles.
Sequential search: Given a list of numbers and a number, write a program to determine whether the number is in the list.

Figure 4.

Hierarchical Structure of the Triangle Classifier Programming Task. Note: The connections between tasks represent prerequisites. The arrows in the figure point from a task that provides prerequisite knowledge to a more difficult task.

It is not uncommon for students to get stuck in solving a programming problem. There are many ways in which a student can find it hard to move on: for example, they may not fully understand the programming task, they may understand the task but be unable to find a way to solve it, or they may submit a solution to the platform but not obtain the expected outcome. In this study, we provided scaffolding in the performance-based assessment platform to support students in finding a way out. Figure 5 shows the flow for acquiring prior knowledge of related tasks.

Figure 5.

The Acquisition of Prior Knowledge of Related Tasks for Scaffolding the Triangle Classifier Task.

If they find that they are stuck in solving a higher-level task, students can press the ‘Prior knowledge acquisition’ button to preview the related sub-tasks. Pressing one of the subtasks then reveals its problem description. Students can then attempt to solve the related subtasks in an environment like that shown in Figure 1. After attempting a subtask, students can press the ‘Prior knowledge acquisition’ button again for further support if needed; otherwise, they can press the ‘Next Challenge’ button to proceed and are then guided back to the higher-level task.

This study attempted to support students’ SRL in programming education through a performance-based assessment platform with embedded SRL support functions that uses automatic marking to provide timely feedback. The platform enables programming tasks to be offered in a structured manner, permits scaffolding in the learning process, and provides SRL support via a focused design. In addition to platform-based SRL support, the students were also provided with mentor support based on the data captured by the system. In evaluating the system, we aimed to answer the following research questions (RQs). RQ1: To what extent did the performance-based assessment platform and mentor support help the students to develop SRL skills? RQ2: How were the students’ SRL skills enhanced through the performance-based assessment platform and mentoring support?

Methodology

Research Design

This study explored the potential of using a performance-based assessment platform to support students’ SRL and took a mixed-methods approach to evaluate its effectiveness. To answer RQ1, we administered a survey to measure students’ SRL skills after they had used the platform to complete the three programming tasks. To answer RQ2, and to better understand students’ SRL needs, we interviewed the students about their experiences after they had used the assessment platform.

Programming Competition and Participants

We held a competition for senior primary students to motivate them to solve programming problems on the performance-based assessment platform. In the competition, the students were given three weeks to complete three Level 3 programming tasks on the EasyCode platform. The three Level 3 tasks were designed at different levels of difficulty but carried equal weight. Students received awards if they solved one or more tasks, and could set their learning goals at solving one, two, or all three tasks. They could adopt the strategy of attempting the easier tasks first or that of attempting the difficult tasks first. If the students encountered difficulties in solving a programming task, they could deploy various strategies. They could follow our guidance on the platform to tackle the less demanding sub-tasks at Level 2 and, if needed, at Level 1 before returning to solve the Level 3 task; they could join our support sessions and ask questions; they could directly submit their solutions to the platform and obtain automatic marking feedback; and they could switch from one Level 3 task to another Level 3 task if they wished. They were thus given options to execute different strategies. All of the SRL support offered by the platform was accessible to the students during the competition. At the first tutorial, the participating students were introduced to the platform’s SRL features. Another tutorial was held in the middle of the competition, when students could ask questions and obtain appropriate support from the tutor. We conducted post-competition surveys to measure the students’ self-evaluated SRL skills and conducted focus group interviews to solicit the students’ in-depth views on their experience of SRL. Eighty-eight 10- to 11-year-old students from 15 primary schools enrolled in the competition. Only 45 students logged into the platform, of whom 32 completed at least one of the required programming tasks.

Measures and Instruments

The platform automatically assesses students’ solutions using a black-box test design. For each programming task, a set of test cases and expected outputs were prepared to test the functionality of the students’ programs without consideration of their internal structures and workings. The program output was checked against the expected output. To validate the effectiveness of the automatic marking, we employed two researchers with computer science backgrounds to examine and rate the students’ programming solutions independently using a 4-point scale with the rubric shown in Table 2.

Table 2.

Rubric for the Manual Marking of Students’ Programming Solutions, with Examples.

Score	Criteria	Example
0	Incorrect
1	Correct solution with minor problems (Insignificant logical error, dangling code blocks, and unnecessary complexity)
2	Correct solution
3	Correct solution with outstanding features (Fewer intermediate variables, a compact design, and extra consideration of boundary cases.)

Note: The examples were selected from real solutions submitted by students.

We designed a survey to check the students’ perceptions of their SRL skills and administered it to the participants after the competition. The survey instrument was designed in three steps. First, we conducted a literature review to form initial ideas about the major components needed for the survey. We referred to guides on incorporating SRL in online learning environments (Wong et al., 2019) and on measuring self-regulation skills with an online questionnaire (Barnard-Brak et al., 2010), and reviewed the major features of the performance-based assessment platform (Kong & Liu, 2020). These served as the primary guidelines for developing the instrument. Second, we compiled the items for the survey instrument and produced an initial draft. Third, a professor of computer science and five research staff members scrutinised and revised the instrument.

The survey instrument comprises 12 items on five sub-scales: environment structuring, goal setting, time management, help seeking, and self-evaluation. The students were asked to indicate their level of agreement with each item on a 5-point scale (1 = strongly disagree; 5 = strongly agree). For example, one item was ‘I have the autonomy to achieve the goal in my way’. The students’ responses to this survey were taken as reflecting their self-perceived level of SRL skills. An alpha value of .90 should be seen as having an excellent internal consistency (George & Mallery, 2006). The instrument’s reliability in this study, with a Cronbach’s alpha of 0.948, was therefore considered outstanding.

Post-competition Interview

Sampling Procedures

Upon completion of the three-week programming competition, 20 of the 32 students who finished at least one programming task were invited for a follow-up interview regarding their SRL experience. The interviews lasted for an average of 60 minutes each and were conducted in five focus groups. The research staff were trained to conduct the interviews following a standard procedure designed for this study. All of the interviews were conducted online using video conferencing software and were recorded with the consent of the students and their guardians.

Interview Design

A semi-structured interview design was used to elicit a more profound understanding of the students’ perceptions of their SRL skills. According to Weiss (1995), interviews allow students to give an in-depth description of their experiences, thus allowing researchers to obtain a deeper understanding of students’ views. A semi-structured interview guide was developed for the interviews in this study, with questions focused on SRL-related issues. The interview questions were designed to elicit in-depth narratives of the students’ experiences and thoughts during the competition. There were six questions in the interview guide: (1) How did you get started in the competition? (2) Tell me about the plan/strategy you used to help you solve the problem. How did you allocate your time to do the task? (3) Which problem did you deal with first, and what steps did you take to solve it? (4) When you encountered a challenge in the problem-solving process, what did you do to resolve the situation? (5) For those unsolved issues, what may have prevented you from finding a solution? (6) If you were to join a similar contest again in the future, what would you do to improve your performance? All of the interviews were conducted in Cantonese and videorecorded. The recorded interviews were transcribed by a member of the research staff. All of the personal information was removed from the data to ensure the anonymity of the interviewees.

Thematic Analysis

We conducted a thematic analysis of the interview data to extract the students’ views of SRL. Thematic analysis is a robust qualitative approach that can effectively remove susceptibility to reliability issues (Yu, 2005). It attempts to find patterns, themes, and relationships between phenomena based on the frequency of phrases and words (Guest et al., 2014). The interview data were analysed in the following steps: (1) the interview transcripts were read through to form an initial impression of the collected data; (2) the data were coded systematically; (3) the codes were sorted and searched for potential patterns and categories; (4) themes relevant to the research questions were identified; and (5) a relationship model was devised to organise the themes for explaining observations. The initial coding was conducted by two members of the research team working independently. After any disagreement between these two coders was resolved, the final codes were generated. The open codes and themes were then developed and further revised based on a review of the interview transcripts to generate a reliable thematic map that provided a coherent narrative for the study.

Results

This section presents the results, including the students’ performance on the programming tasks, their self-reported SRL skills after the competition, and the thematic analysis of the interview data.

Students’ Performance in Programming Problem-Solving

The solutions submitted by the students were marked automatically by the online platform. Of the 45 students who completed the competition, 32 (71.11%) passed the automatic marking on at least one Level 3 programming task, as shown in Table 3. There were 72 solutions that passed the automatic marking.

Table 3.

Number of Students Passing the Platform’s Automatic Marking for One or More Level 3 Tasks.

Result	Number of Students
Passed automatic marking on
- Three Level 3 programming tasks	15
- Two Level 3 programming tasks	10
- One Level 3 programming task	7
Did not pass any Level 3 programming tasks	13
Total	45

Note: 32 students passed the automatic marking on at least one Level 3 task.

The online platform automatically marked the programming tasks submitted to it using a set of black box testing data that checked the solutions against the expected output. Given that the automatic marking could overlook logical errors in the submitted code, we employed two computer science experts to rate the 72 submitted solutions independently using the rubric in Table 2. An interrater reliability analysis was performed using the intraclass correlation coefficient (ICC). As shown in Table 4, the ICC was 0.850 (p < 0.001) with a 95% confidence interval from 0.771 to 0.903, suggesting substantial agreement between the two raters.

Table 4.

Intraclass Correlation Coefficient of the Manual Marking Result.

	Intraclass Correlation	95% Confidence Interval		F Test with True Value 0
	Intraclass Correlation	Lower Bound	Upper Bound	Value	df1	df2	Sig.
Single Measures	0.850	0.771	0.903	12.343	71	71	0.000

The final scores of the manual marking were determined by the two raters working together to resolve disagreements case by case until a consensus was reached (Table 5). Eight solutions were automatically marked as correct but rated as incorrect (scored 0) when marked manually. This discrepancy was caused by the limited coverage of the black box testing, with the manual markers identifying paths that were not detected in the automatic testing process. A comparison with the manual scores indicated that the accuracy of the automatic marking was 88.88%. Most of the submitted solutions were therefore marked correctly by the automatic marking function, fulfilling the need for real-time feedback. From the manual marking, the students could obtain a maximum score of nine.

Table 5.

Example Discussion to Reach a Consensus on Different Scores.

Students’ Submitted Code for the Sequential Search Task:

Rater 1 Score: 1	The code generates the output as expected.
Rater 2 Score: 0	The first if-statement is unnecessary and unreasonable.
Final consensus	Rater 1’s score was adopted because the submitted code generates the correct output. The unnecessary if-statement does not cause a significant problem in the programming logic that harms the accuracy of the program’s final output.

Note: This table provides an example of the discussion on how a final consensus was reached on the score for a student’s solution. To reach a final score, the two raters discussed their reasons for assigning the respective mark.

The manual marking revealed that the programming tasks allowed the students to demonstrate not only their ability to solve problems but also their creativity in resolving them. For example, Figure 6 shows one student’s solution to the Favourite Food task. The student constructed a list of lists by mapping the original two lists (marked by the red frame). Figure 7 is a student’s solution for the Triangle Classifier task. This student applied a modular design, using a procedure (marked by the red frame) to first find the largest angle in a triangle using linear traversal and then to determine the triangle type accordingly.

Figure 6.

A Student’s Solution for the Favourite Food Task.

Figure 7.

A Student’s Solution for the Triangle Classifier Task.

Students’ Self-rated SRL Skills

Table 6 presents the descriptive statistics of the students’ SRL survey. The overall mean score was 3.930, indicating that the students generally rated their SRL skills at a moderately high level. The mean values for the items on the environment structuring subscale were relatively high, falling between 3.933 and 4.333, implying that the students were able to construct a stable learning environment for SRL, avoiding constraints on time, network, and physical location. Comparatively high mean values between 3.888 and 4.000 were also observed for time management, reflecting the students’ awareness of the need to manage their time to keep up with their overall learning progress. The lowest mean score (3.377) was for the final item of the help-seeking subscale, concerning the students’ ability to solve programming tasks independently. This shows that the students required more support for problem-solving during the programming process. The standard deviations for the students’ SRL skills were close to 1 for all of the items, suggesting a comparatively high variation in the students’ perceptions of their SRL skills. Some of the students showed high confidence in their SRL skills after the competition, but others felt that they had not mastered the SRL skills and needed further support.

Table 6.

Means and Standard Deviations for the SRL Student Survey.

Item	Students Who Solved at Least One Problem (n = 32)	Students Who Failed to Solve Any Problems (n = 13)	All students (n = 45)
Item	Mean (SD)	Mean (SD)	Mean (SD)
Environment structuring
I can arrange an appropriate time and location to learn on the EasyCode platform directly.	4.031 (0.932)	3.692 (0.947)	3.933 (0.939)
I have suitable computers/mobile devices and Internet access to support my learning on the EasyCode platform.	4.343 (1.003)	4.307 (0.854)	4.333 (0.953)
Goal setting
I am clear about how to make use of the EasyCode platform to achieve my goals.	3.937 (1.075)	4.076 (0.862)	3.977 (1.011)
I know how to set up sub-goals to achieve my main goal.	3.968 (0.897)	3.923 (0.954)	3.955 (0.903)
I receive guidance from teachers/parents on how to achieve the goals in stages.	3.968 (1.031)	4.076 (0.640)	4.000 (0.929)
I have the autonomy to achieve the goals in my way.	3.968 (0.897)	3.846 (0.898)	3.933 (0.889)
Time management
I can schedule enough time to work with EasyCode for learning/assessment.	4.031 (0.999)	3.538 (0.967)	3.888 (1.005)
I make sure I keep up with the overall progress of learning/assessment in EasyCode.	4.125 (0.906)	3.692 (0.854)	4.000 (0.904)
Help seeking
I can find someone to help me when I need to.	4.000 (1.107)	4.076 (0.640)	4.022 (0.988)
I use the support mode/tips/prior knowledge acquisition functions for help when I have difficulties in the coding process.	3.968 (0.999)	4.307 (0.751)	4.066 (0.939)
I prefer using support mode/tips/prior knowledge acquisition before asking for help from other people to complete the coding tasks.	3.843 (0.987)	4.00 (0.707)	3.888 (0.910)
Ultimately, I try to complete the coding tasks independently without using support mode/tips/prior knowledge acquisition.	3.375 (1.237)	3.384 (1.192)	3.377 (1.211)
Self-evaluation
I carry out the coding tasks on the EasyCode platform repeatedly until I fully understand those tasks.	3.687 (0.965)	3.538 (0.776)	3.644 (0.908)
I understand the goal of coding tasks on the EasyCode platform is for me to understand computational thinking concepts and practices.	4.031 (1.031)	3.923 (0.759)	4.00 (0.953)
Total	3.948 (0.829)	3.884 (0.610)	3.930 (0.766)

Students’ Perceptions of SRL When Using the Performance-based Assessment Platform.

We also analysed the survey responses by splitting the data according to the number of Level 3 problems solved by each student to reveal any differences between the more and less competent students. The overall mean (3.948) of the students who successfully solved at least one Level 3 problem was nearly the same as that of the students who failed to solve any Level 3 problems (3.884), and the difference was not statistically significant (Mann–Whitney U = 186, n₁ = 32, n₂ = 13, p > 0.05). There were also no statistically significant differences between the scores on any of the items between these two groups. This result indicates that even students who failed to solve any Level 3 problems acquired some SRL skills from the competition. In summary, all of the students benefited meaningfully from the programming competition in the development and application of their SRL.

The Six Themes and the Relationship Model

To address RQ2, the interview transcripts were analysed and open coding was carried out. A thematic analysis based on open coding shed light on the students’ SRL development in the process of attempting the programming tasks and on the effectiveness of the SRL support provided. The open codes were determined by two coders from the research team, who independently labelled the content of the interview transcripts. The coding was then carefully examined and discussed between the two coders to generate the final 24 open codes, which were then organised into six themes: foresight, time management, difficulties in programming, problem-solving, help seeking, task strategies, and reflection. Table 7 shows the 24 open codes with their frequencies reported in parentheses. We formulated the relationship model shown in Figure 8 following the three-phase SRL model. The foresight theme is related to the forethought phase of SRL, the reflection theme is related to the self-reflection phase of SRL, and the remaining themes are related to the performance phase of SRL. The task strategies that the students applied in their SRL process played a central role in their development of SRL skills. How the students tackled the difficulties they experienced in programming problem-solving is reflected in the task strategies and help-seeking themes, both of which illustrate students’ SRL development. In addition, the foresight, time management, and reflection themes provide insights into the forethought and self-reflection phases of students’ SRL.

Table 7.

Open Codes and Frequency of the Corresponding Themes Collected from the Interview Data.

Theme 1	Theme 2	Theme 3	Theme 4	Theme 5	Theme 6
Foresight	Time management	Difficulty in programming problem-solving	Help seeking	Task strategy	Reflection
Goal setting (68)	Time estimation (12)	Adaptation of programming interface (18)	Support from the platform (70)	Assess difficulty (53)	Improvement in programming (39)
Planning (28)	Fixed timetable (10)	Implementation of solution (17)	Support from other people (33)	Design before programming (42)	Psychological gain (21)
	Time allocation between tasks (9)	Understanding tasks (29)	Public online sources (11)	Getting unstuck (19)	Changing strategy and performance enhancement (41)
	No specific arrangement (7)		Avoid support (11)	Transfer of learning (39)	Interactive learning environment (61)
				Problem-solving order (23)
				Sufficient testing and debugging (17)
96	38	64	125	193	162

Note: The frequency of each code is presented in parentheses. The bottom row of the table gives the overall frequency of each theme.

Figure 8.

Relationships Between the Six Themes Based on a Thematic Analysis of the Interview Data.

A theme is a pattern that captures a significant or interesting aspect of the data. To generate themes, we sorted and examined the created codes to identify patterns. In the interview, students were given an opportunity to describe their experiences in SRL, such as how they decided on which question to start, how they allocated time to each question, and so on. The interview data reveal the students’ SRL process in detail, and the themes that emerged from the data mainly reflect their behaviour, thinking, and experiences in SRL. Table 8 shows the frequency with which each of the themes was raised in the five interviews. All six themes were captured in the interviews, but the most popular were Themes 4, 5, and 6: help seeking, task strategy, and reflection.

Table 8.

Frequency of the Themes Raised in the Five Interviews.

Theme			Interview
	1	2	3	4	5
Foresight	20	19	23	18	16
Time management	9	6	6	3	14
Difficulty in programming problem-solving	19	18	12	8	7
Help seeking	41	21	15	29	19
Task strategy	26	33	22	71	41
Reflection	41	32	30	38	21

The Thematic Map

To provide a more comprehensive picture of the students’ perceptions on their development of SRL skills during the competition, we formulated a relationship model of the themes. Figure 9 shows a thematic map of the 24 open codes based on the six themes. In this section, we discuss in detail the key ideas of the interviewees on each of the six themes of SRL.

Figure 9.

Thematic Map Based on Axial Coding. Note. The size of each rectangle in the model represents the frequency of the codes. The arrows show the relationships between the codes based on the thematic analysis of the interview transcripts.

Foresight

This theme refers to the students’ goals in the learning process and how they planned to achieve their goals. The interviewed students said that they had the clear goal of completing the Level 3 tasks. In addition, they were motivated by the certificate of merit for completing the competition. For instance, student #1 commented, ‘I have a basic knowledge of programming. I consolidated the groundwork through this competition. Therefore, I expected to be able to win a prize’. Students also reported their preferences in task selection, with most saying that they adopted an easiest-first strategy. As student #2 stated, ‘Choose whichever is easy to get started’. A few applied a hardest-first strategy to take on more challenge up front. For example, student #3 said, ‘Usually, I look at all the problems, then start with the most difficult one. After that, I attempt other problems. I feel comfortable doing it that way’.

Time Management

This theme refers to the students’ scheduling and pacing control. Most of the students felt that they had sufficient time to work on the tasks, which is consistent with the SRL survey result. Nevertheless, most admitted that they did not limit their time on a task. They stated that when they encountered questions beyond their competence, they would spend a few days reflecting on their solutions and ideas before completing the task. As student #4 mentioned, ‘If I get stuck solving a programming problem, I attempt other tasks and then tackle those problems again a day later. I review my thoughts and examine whether I have formulated a clearer idea’. The students also reported that their problem-solving strategy influenced the time they allocated to each question. Students adopting the easiest-first strategy specified that they preferred to leave more time to work on the more difficult tasks. In contrast, students choosing the hardest-first approach felt that solving the difficult questions first would offer greater insights and knowledge and thus enable them to complete the more straightforward questions more quickly. As student #4 said, ‘Usually, I will attempt the more challenging tasks first. I find that the harder tasks provide more information and ideas. … Maybe it is because more challenging tasks provide more knowledge’.

Difficulty in Problem-solving During Programming

This theme refers to the issues that students encountered when programming. Most of the students reported that they could understand the task descriptions clearly, as the input/output examples helped them form a clear programming goal: as student #1 stated, ‘I could understand because the problem description stated the information clearly, and I could find the necessary items accordingly’. However, some of the students said they had difficulty understanding some of the abstract or technical terms, such as ‘data structure’ or ‘mathematical definition’. For instance, student #5 stated, ‘I found that some task descriptions were unclear to me. For example, I did not know the terms “sequential search”, “elements in a list”, or “number list”’.

Another reported difficulty was converting ideas into code. The students stated that their programming errors were sometimes caused by a gap between their ideas and their implementation of these ideas. For instance, student #3 said, ‘I supposed that the task would be easy to solve at the beginning. I just needed to put the two large blocks together. However, the platform informed me that my program had failed, and I realised that I had missed the middle part in the connected blocks. That missing part was the most important one. I thought my idea was correct, but my program did not have the necessary code to implement my idea’. Some students also admitted that they encountered difficulties adapting to the new programming environment. They failed to find the blocks providing the function they wanted and were unable to understand the blocks’ functions from the descriptions. As student #6 said, ‘I thought I understood the task, as it was similar to the previous problem. Nevertheless, I noticed something different there. It seemed that some names of the blocks had changed. I think the programming environment there looked like App Inventor, except some wordings were different there, and I did not fully understand them’.

Help Seeking

This theme refers to how the students sought help to overcome difficulties in addressing the programming tasks. Most of the students reported that they first turned to platform-based support when they reached an impasse. The first step was often to find clues in the simplified toolbox provided in support mode. For instance, student #7 said, ‘I noticed that there were fewer blocks in support mode. I tried to find clues using these blocks to solve the problem’. Some students also tried to find clues in the Level 1 and 2 tasks provided in the prior knowledge acquisition function, which they found helpful in tackling the Level 3 questions. For example, student #8 stated, ‘I started by solving the Level 3 questions. I worked with the related questions, which provided me with prior knowledge when I did not know how to solve the Level 3 questions. It was useful to learn the prior knowledge. These tasks were related to solving the Level 3 questions. They could help me to solve them. These tasks for learning prior knowledge helped me figure out how to combine them and solve the Level 3 questions’. Some students also said they sought help from teachers, who still play an essential role in SRL, and from peers and mentors. When they failed to understand or devise a solution, they turned for help to their teachers, whose role was to assist in pace control, time management, and evaluating the learning difficulties, as reported by student #7: ‘Our teachers went through the problems together with us after class. We all agreed that these tasks are relatively easy, and we estimated the time needed to solve them. When I got stuck in some tasks, I discussed them on Zoom immediately with my peers and teachers. I could solve all these questions in one evening’. The students viewed the mentor support session we provided as additional support, especially when they could not receive help from peers and teachers, and some were able to achieve a breakthrough by joining the support session. As student #9 confirmed, ‘With the support of the mentor in the support session, I found out I needed to use one more block. This enabled me to solve the problem’.

Task Strategy

This theme refers to the strategies the students applied in the problem-solving process. Apart from assessing the difficulty of the problem and deciding the task sequence, learning transfer was a common strategy reported by the students. Most claimed that they used the knowledge and experience they had obtained from using other platforms to answer the questions in this programming environment. For instance, student #10 said, ‘Because I know how to code in Scratch, I noticed that the code blocks here were like those in Scratch. I took advantage of this prior knowledge and was able to formulate ideas to solve the tasks’. Other strategies that the students used when they got stuck in the programming process included rewriting the code, verifying their programs on a case-by-case basis, and replacing specific components with others that shared similar functions. For example, student #7 stated, ‘When I got stuck, I examined all the blocks in the workspace again. Then I replaced some blocks and tested whether the program worked as expected’. The students generally felt that debugging and testing helped them achieve breakthroughs. For example, student #1 stated, ‘I discovered the problems by comparing the output with the tested result and revised my code accordingly’.

Reflection

This theme refers to the students’ reflections on their performance, actions, feelings, and gains. Most of the students perceived the website as a learning platform rather than an entertainment playground, with EasyCode having fewer distracting elements than other programming platforms. The simple interface enabled them to concentrate on the tasks. As student #11 stated, ‘EasyCode has a simple and clear interface compared to the Scratch programming environment. Scratch has more graphics and gaming features, allowing us to explore. To me, Scratch is more for entertainment, and EasyCode is more for learning. Because it provided feedback and support, EasyCode provided an environment for me to solve problems. I could concentrate on solving the problem in EasyCode because there were not so many distractions’. The students also generally believed that they improved their programming skills by taking part in the competition, which they viewed as an opportunity to accumulate and apply the programming experience and knowledge they had acquired from using other platforms. In addition to improving their programming, some of the students highlighted their increased persistence and confidence when facing difficulties. When asked what actions they could have taken to improve their performance, most of the students said they would revise their previous task strategies, switch the order in which they solved problems, go through the problem descriptions in detail, and review computing concepts before starting to program.

Discussion

In this study, we attempted to support senior primary students’ SRL through a performance-based assessment platform in the context of programming education. Students were required to finish programming tasks on a performance-based assessment platform with SRL support. More than 70% of the students successfully solved at least one of the difficult programming tasks. As our interest was in the development of SRL skills, we conducted a survey to measure the students’ SRL skills after they used the platform. The survey results showed that the students’ SRL skills were at a moderately high level after they had used the platform with SRL scaffolding to solve programming tasks. This suggests that the SRL support provided on the platform helped them to develop SRL skills. The survey results also showed that the students were able to construct a suitable learning environment, allocate time between learning tasks, and monitor their learning progress using the SRL support features.

A more comprehensive view of how the performance-based assessment platform enhanced students’ SRL skills was obtained from a thematic analysis of the interview data. We generated six themes from the coded interview data according to the behaviour described by the students, with the organising framework of the three-phase SRL model. The foresight theme relates to the forethought phase of SRL. We found that the students set goals when they started work in the competition setting. We identified four themes related to the performance phase of SRL: time management, difficulty in programming problem-solving, help seeking, and task strategy. The students modified their task strategies, managed their time, and decided when and how to seek help according to the difficulties they encountered in the SRL process. The reflection theme relates to the self-reflection phase of SRL. Students reflected that the embedded SRL scaffolding on the platform was helpful. The simplified toolbox helped the students find solutions to the programming tasks, and the prior knowledge acquisition function offered them an opportunity to attain the knowledge they needed to solve the tasks. These features provided strategic scaffolding for the students’ SRL.

Strategic scaffolds focus on identifying and selecting the necessary information, evaluating the available resources, and relating new knowledge to existing experience and knowledge (Hannafin et al., 1999; Lajoie et al., 2001). Research has found that scaffolds can support SRL processes (Dabbagh & Kitsantas, 2005; Shapiro, 2008). The theoretical foundation of scaffolding is that it can guide students on an as-needed basis and fade away as students’ abilities increase (Hogan & Pressley, 1997). Properly designed scaffolding can help achieve advanced understanding beyond students’ current abilities (Jackson et al., 1994; Simons & Klein, 2007). In this study, the simplified toolbox essentially narrowed down the resources needed for the tasks by helping the students to select appropriate blocks. When the students tried to find clues from the blocks provided in the simplified toolbox, they evaluated how each block could be used and tried to construct a solution based on the available resources. The prior knowledge acquisition function helped the students to identify and understand the knowledge required for the next level of problem-solving. It prepared students with the knowledge to solve difficult programming tasks and enabled them to relate this acquired knowledge to more complex scenarios. The results of the thematic analysis suggest that the students benefited from these strategic scaffolds.

Algorithmic thinking is a problem-solving skill that empowers students to design an action sequence to form a solution to a programming problem, and is an important skill for solving programming tasks. The performance-based platform broke complex programming tasks down into simpler tasks and allowed the students to apply either a top-down or a bottom-up problem-solving approach. However, neither approach guaranteed they would solve the problems successfully. Some of the students finished all of the Level 1 and 2 tasks set by the prior knowledge acquisition function but still failed to solve the related Level 3 programming task. It seems that the students needed additional support to connect the subtasks to the solution of the more complex programming problems. In other words, regardless of whether they adopt a top-down or bottom-up approach, the students needed more guidance on developing algorithms to solve the programming problems.

Mentor support, which aims to help students function independently via a mentor–protégé interaction, plays a key role in SRL (Schunk & Mullen, 2013; Tammy & Lillian, 2011; Zhang & Lin, 2021). The students’ feedback on the SRL experience was more positive when they received support from a mentor. Although students are responsible for decision-making in the SRL process, mentor support can help them make wiser decisions and thus lead to better learning outcomes.

Conclusion and Implications

This study made an innovative attempt to use a performance-based assessment platform to support the development of senior primary students’ SRL. We embedded a series of SRL support features into a platform for solving programming tasks. The effectiveness of our SRL support was evaluated with a mixed-methods approach. The results from the SRL survey indicated that students rated their SRL skills at a moderately high level after working with the platform. At the interviews, most of the students reported that the support features embedded in the platform facilitated their SRL processes in solving the programming tasks. The interview data enabled us to identify the SRL strategies that the students used in solving the programming tasks. These strategies included ways of tackling difficult programming tasks, ways of designing solutions before programming, and changing the order of solving problems with different difficulty levels. This study provides evidence that a performance-based assessment platform with strategic scaffolds and automatic marking feedback can support students’ programming problem-solving processes and suggests that well-designed performance-based assessment can be applied as an effective tool in SRL.

From the thematic analysis of the interview data, we gained a deeper understanding of the students’ difficulties and the support they needed in the SRL process. Although the support functions provided in the performance-based assessment platform gave the students an environment in which they could implement various SRL strategies, they needed further support in algorithmic thinking to come up with solutions. The students also needed guidance on pace control and understanding the blocks in the programming environment and tutorials on using the platform for SRL. Although the survey showed that the students were aware of the need to manage their time, we noticed from the interviews that most of the students lacked a concrete plan for their work outside of allocating time to tasks according to difficulty level. Regarding pace control, most of the students attempted to finish all of the tasks in a single session, which may not be a good practice given the spaced learning effect (Dempster, 1988), which suggests that a learning process that occurs in a single session is less effective than a process spaced out over time.

Another problem that the students encountered was how to use the blocks in the programming environment. Although all of the platform features were explained to the students at the beginning of the competition, some students still did not know when or how to obtain SRL support for problem-solving. Some tried to guess from the descriptions and others turned to the online examples; most referred to their knowledge of other block-based programming environments. Nevertheless, for students with limited experience, programming in a new environment is challenging. Therefore, there is a need to provide students with better support for understanding the command blocks in the platform, such as easier access to explanations of the command blocks and illustrations of how the command blocks work. A demonstration of how to use the SRL support available in the performance-based assessment platform would have been helpful for these students.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

The submitted work is original and has not been published elsewhere in any form or language, partially or in full. The authors have included the ethical approval and all of the relevant declaration statements in the submission of this manuscript.

Informed Consent

The authors have obtained the approval of the ethics committee of the University for research involving humans and the informed consent of the human participants in this study.

Data Availability

The datasets generated during and/or analysed in the current study are available from the corresponding author on reasonable request.

ORCID iD

Siu-Cheung Kong

Author Biographies

Siu-Cheung Kong is Professor of the Department of Mathematics and Information Technology and Director of Centre for Learning, Teaching and Technology at The Education University of Hong Kong. His research interests include pedagogy in digital classroom, teacher development, mathematics education, computational thinking education, STEM education, artificial intelligence literacy, and policy on digital technology in education.

Bowen Liu received his bachelor's degree from Jinan University. During his bachelor's study, he was an exchange student at Aalborg University in Demark. He obtained his Ph.D. from the Education University of Hong Kong. His research interest includes image processing, computer vision, and learning analytics.

References

Ali

A. D.

Hanna

W. K.

(2022). Predicting students’ achievement in a hybrid environment through self-regulated learning, log data, and course engagement: A data mining approach. Journal of Educational Computing Research, 60(4), 960–985. https://doi.org/10.1177/07356331211056178

Ayres

Paas

(2009). Interdisciplinary perspectives inspiring a new generation of cognitive load research. Educational Psychology Review, 21(1), 1–9. https://doi.org/10.1007/s10648-008-9090-7

Azevedo

Hadwin

A. F.

(2005). Scaffolding self-regulated learning and metacognition – Implications for the design of computer-based scaffolds. Instructional Science, 33(5–6), 367–379. https://doi.org/10.1007/s11251-005-1272-9

Bannert

(2009). Promoting self-regulated learning through prompts. Zeitschrift Für Pädagogische Psychologie, 23(2), 139–145. https://doi.org/10.1024/1010-0652.23.2.139

Barnard-Brak

Paton

V. O.

Lan

W. Y.

(2010). Profiles in self-regulated learning in the online learning environment. The International Review of Research in Open and Distributed Learning, 11(1), 61. https://doi.org/10.19173/irrodl.v11i1.769

Bembenutty

Kitsantas

Clearly

T. J.

(Eds), (2013). Applications of self-regulated learning across diverse disciplines: A tribute to Barry J. Zimmerman. Information Age Publishing.

Bergin

Reilly

Traynor

(2005). Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the 2005 International Workshop on Computing Education Research, ACM, pp. 81–86. https://doi.org/10.1145/1089786.1089794

Biesinger

Crippen

(2010). The effects of feedback protocol on self-regulated learning in a web-based worked example learning environment. Computers & Education, 55(4), 1470–1482. https://doi.org/10.1016/j.compedu.2010.06.013

Brennan

Resnick

(2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 Annual Meeting of the American Educational Research Association, 1, Vancouver, 2012, p. 25.

10.

Broadbent

Poon

W. L.

(2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007

11.

Çakıroğlu

Ü.

Çevik

İ.

Köşeli

Aydın

(2021). Understanding students’ abstractions in block-based programming environments: A performance based evaluation. Thinking Skills and Creativity, 41, 100888. https://doi.org/10.1016/j.tsc.2021.100888

12.

Chen

C.-M.

Wang

J.-Y.

Chen

Y.-C.

(2014). Facilitating English-language reading performance by a digital reading annotation system with self-regulated learning mechanisms. Journal of Educational Technology & Society, 17(1), 102–114.

13.

Chiu

T. K. F.

(2021). Student engagement in K–12 online learning amid COVID-19: A qualitative approach from a self-determination theory perspective. Interactive Learning Environments, 1–14. https://doi.org/10.1080/10494820.2021.1926289

14.

Cho

M.-H.

Shen

(2013). Self-regulation in online learning. Distance Education, 34(3), 290–301. https://doi.org/10.1080/01587919.2013.835770

15.

Choi

Lee

(2017). Puzzle based algorithm learning for cultivating computational thinking. Wireless Personal Communications, 93(1), 131–145. https://doi.org/10.1007/s11277-016-3679-9

16.

Çoban

Korkmaz

Ö.

(2021). An alternative approach for measuring computational thinking: Performance-based platform. Thinking Skills and Creativity, 42, 100929. https://doi.org/10.1016/j.tsc.2021.100929

17.

Dabbagh

Kitsantas

(2005). Using web-based pedagogical tools as scaffolds for self-regulated learning. Instructional Science, 33, 513–540. https://doi.org/10.1007/s11251-005-1278-3

18.

Liew

Willson

(2014). Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education, 78, 312–320. https://doi.org/10.1016/j.compedu.2014.06.018

19.

Dempster

F. N.

(1988). The spacing effect: A case study in the failure to apply the results of psychological research. American Psychologist, 43(8), 627–634. https://doi.org/10.1037/0003-066X.43.8.627

20.

Engelmann

Bannert

Melzner

(2021). Do self-created metacognitive prompts promote short- and long-term effects in computer-based learning environments? Research and Practice in Technology Enhanced Learning, 16(1), 3. https://doi.org/10.1186/s41039-021-00148-w

21.

George

Mallery

(2006). SPSS for windows step-by-step: A simple guide and reference, 14.0 update. Allyn & Bacon.

22.

Gresse Von Wangenheim

Alves

N. da C.

Rauber

M. F.

Hauck

J. C. R.

Yeter

I. H.

(2021). A proposal for performance-based assessment of the learning of machine learning concepts and practices in K–12. Informatics in Education, 21(3), 479–500. https://doi.org/10.15388/infedu.2022.18

23.

Guest

MacQueen

K. M.

Namey

E. E.

(2014). Applied thematic analysis. SAGE Publications, Inc.

24.

Hannafin

Land

Olive

(1999). Open learning environments: Foundations, methods, and models. In Reigeluth

C. M.

(Ed.), Instructional design theories and models: A new paradigm of instructional theory (2, pp. 115–140). Taylor and Francis.

25.

Hao

Tsikerdekis

(2019). How automated feedback is delivered matters: Formative feedback and knowledge transfer. Proceedings of 2019 IEEE Frontiers in Education Conference, 1-6. https://doi.org/10.1109/FIE43999.2019.9028686

26.

Hogan

Pressley

(Eds.), (1997). Scaffolding student learning: Instructional approaches and issues. Brookline Books.

27.

Jackson

S. L.

Stratford

S. J.

Krajcik

Soloway

(1994). Making dynamic modeling accessible to precollege science students. Interactive Learning Environments, 4(3), 233–257. https://doi.org/10.1080/1049482940040305

28.

Katz

L. J.

Slomka

G. T.

(2000). Achievement testing. In Goldstein

Hersen

(Eds.), Handbook of psychological assessment (pp. 149–182). Elsevier. https://doi.org/10.1016/B978-008043645-6/50085-9

29.

Kim

H. J.

Pedersen

(2011). Advancing young adolescents’ hypothesis-development performance in a computer-supported and problem-based learning environment. Computers & Education, 57(2), 1780–1789. https://doi.org/10.1016/j.compedu.2011.03.014

30.

Koh

K. H.

Basawapatna

Nickerson

Repenning

(2014). Real time assessment of computational thinking. In Proceedings of 2014 IEEE symposium on visual languages and human-centric computing, Melbourne, Australia, 2014, pp. 49–52. https://doi.org/10.1109/VLHCC.2014.6883021

31.

Kong

S. C.

Liu

(2020). A performance-based assessment platform for developing computational thinking concepts and practices: EasyCode. Bulletin of the Technical Committee on Learning Technology, 20(2), 8.

32.

Lajoie

S. P.

Guerrera

Munsie

S. D.

Lavigne

N. C.

(2001). Constructing knowledge in the context of bioworld. Instructional Science, 29(2), 155–186. https://doi.org/10.1023/A:1003996000775

33.

Liu

C.-C.

Chen

G.-D.

Wang

C.-Y.

C.-F.

(2002). Student performance assessment using Bayesian network and web portfolios. Journal of Educational Computing Research, 27(4), 437–469. https://doi.org/10.2190/EL2A-TQF1-PYVM-N51V

34.

Loksa

A. J.

(2016). The role of self-regulation in programming problem solving process and success. In Proceedings of the 2016 ACM Conference on International Computing Education Research, ACM, pp. 83–91. https://doi.org/10.1145/2960310.2960334

35.

Messick

(1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13–23. https://doi.org/10.3102/0013189X023002013

36.

Panadero

Andrade

Brookhart

(2018). Fusing self-regulated learning and formative assessment: A roadmap of where we are, how we got here, and where we are going. The Australian Educational Researcher, 45(1), 13–31. https://doi.org/10.1007/s13384-018-0258-y

37.

Pintrich

P. R.

(2000). The role of goal orientation in self-regulated learning. In Boekaerts

Zeidner

Pintrich

P. R.

(Eds.), Handbook of self-regulation (pp. 451–502). Elsevier. https://doi.org/10.1016/B978-012109890-2/50043-3

38.

Rivers

D. J.

Nakamura

Vallance

(2022). Online self-regulated learning and achievement in the era of change. Journal of Educational Computing Research, 60(1), 104–131. https://doi.org/10.1177/07356331211025108

39.

Salma

Prastikawati

E. F.

(2021). Performance-based assessment in the English learning process: Washback and barriers. Getsempena English Education Journal, 8(1), 164–176. https://doi.org/10.46244/geej.v8i1.1305

40.

Samruayruen

Enriquez

Natakuatoong

Samruayruen

(2013). Self-regulated learning: A key of a successful learner in online learning environments in Thailand. Journal of Educational Computing Research, 48(1), 45–69. https://doi.org/10.2190/EC.48.1.c

41.

Schunk

D. H.

Mullen

C. A.

(2013). Toward a conceptual model of mentoring research: Integration with self-regulated learning. Educational Psychology Review, 25(3), 361–389. https://doi.org/10.1007/s10648-013-9233-3

42.

Shapiro

A. M

(2008). Hypermedia design as learner scaffolding. Educational Technology Research and Development, 56(1), 29–44. https://doi.org/10.1007/s11423-007-9063-4

43.

Shin

Song

(2022). The effects of self-regulated learning support on learners’ task performance and cognitive load in computer programing. Journal of Educational Computing Research, 60(6), 1490–1513. https://doi.org/10.1177/07356331211052632

44.

Simons

K. D.

Klein

J. D.

(2007). The impact of scaffolding and student achievement levels in a problem-based learning environment. Instructional Science, 35(1), 41–72. https://doi.org/10.1007/s11251-006-9002-5

45.

Tabuenca

Kalz

Drachsler

Specht

(2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education, 89, 53–74. https://doi.org/10.1016/j.compedu.2015.08.004

46.

Tammy

D. A.

Lillian

T. E.

(Eds.), (2011). The Blackwell handbook of mentoring: A multiple perspectives approach. John Wiley & Sons.

47.

Torrance

(Ed.), (1995). Evaluating authentic assessment: Problems and possibilities in new approaches to assessment. Open University Press.

48.

Van der Graaf

Lim

Fan

Kilgour

Moore

Bannert

Gasevic

Molenaar

(2021). Do instrumentation tools capture self-regulated learning? In LAK21: 11th International Learning Analytics and Knowledge Conference, ACM, pp. 438–448. https://doi.org/10.1145/3448139.3448181

49.

Vohs

K. D.

(Ed.), (2013). Handbook of self-regulation: Research, theory, and applications (2nd ed.). Guilford Press.

50.

Wang

Peng

Wang

(2022). A curation activity-based self-regulated learning promotion approach as scaffolding to improving learners’ performance in STEM courses. Journal of Educational Computing Research, 60(4), 843–876. https://doi.org/10.1177/07356331211056532

51.

Wäschle

Lachner

Stucke

Rey

Frömmel

Nückles

(2014). Effects of visual feedback on medical students’ procrastination within web-based planning and reflection protocols. Computers in Human Behavior, 41, 120–136. https://doi.org/10.1016/j.chb.2014.09.022

52.

Weiss

R. S.

(1995). Learning from strangers: The art and method of qualitative interview studies (1st ed.). Free Press.

53.

Wing

J. M.

(2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

54.

Winne

P. H.

(2019). Paradigmatic dimensions of instrumentation and analytic methods in research on self-regulated learning. Computers in Human Behavior, 96, 285–289. https://doi.org/10.1016/j.chb.2019.03.026

55.

Wong

Baars

Davis

Van Der Zee

Houben

G.-J.

Paas

(2019). Supporting self-regulated learning in online learning environments and MOOCs: A systematic review. International Journal of Human–Computer Interaction, 35(4–5), 356–373. https://doi.org/10.1080/10447318.2018.1543084

56.

Xing

Chen

Huang

Chao

Massicotte

Xie

(2021). Automatic assessment of students’ engineering design performance using a Bayesian network model. Journal of Educational Computing Research, 59(2), 230–256. https://doi.org/10.1177/0735633120960422

57.

(2005). A focused conversation model in consumer research: The incorporation of group facilitation paradigm in in-depth interviews. Asia Pacific Advances in Consumer Research, 6, 337–344.

58.

Zhang

Lin

C.-H.

(2021). Effects of community of inquiry, learning presence and mentor presence on K–12 online learning outcomes. Journal of Computer Assisted Learning, 37(3), 782–796. https://doi.org/10.1111/jcal.12523

59.

Zimmerman

B. J

(2000). Attaining self-regulation. In Boekaerts

Zeidner

Pintrich

P. R.

(Eds.), Handbook of self-regulation (pp. 13–39). Elsevier. https://doi.org/10.1016/B978-012109890-2/50031-7

60.

Zimmerman

B. J

(2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2

61.

Zimmerman

B. J.

Moylan

A. R.

(2009). Self-regulation: Where metacognition and motivation intersect. In Hacker

D. J.

Dunlosky

Graesser

A. C.

(Eds.), Handbook of metacognition in education (1st ed., pp. 311–328). Elsevier.

62.

Zimmerman

B. J.

Schunk

D. H.

(Eds.), (2011). Handbook of self-regulation of learning and performance. Routledge.

63.

Zimmerman

B. J.

Schunk

D. H.

(2013). Reflections on theories of self-regulated learning and academic achievement. In Zimmerman

B. J.

Schunk

D. H.

(Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 289–307). Lawrence Erlbaum Associates Publishers.