Toward improving student motivation and performance in introductory programming learning by Scratch: The role of achievement emotions

Abstract

This study investigated the differences in programming novices’ intrinsic motivation and performance within a Scratch-based programming learning environment using a pretest–posttest intervention design. Specifically, this study aimed to examine what and how achievement emotions were aroused to impact learners’ academic success during Scratch implementation processes by employing regression analyses. One hundred and seventy-two undergraduates (Mage = 20.44, SDage = 1.09, 55.8% female) were voluntarily recruited to participate in a Scratch-based instruction workshop during a 6-week period. The workshop was structured with regard to preparation, Scratch-based programming learning and projects, and assessment activities and questionnaire implemented both before the instructional period and after the workshop. The results suggested that the Scratch-based intervention significantly increased students’ overall motivation and examination performance. Concerning achievement emotions, the outcome emotion of hopelessness first negatively predicted students’ motivation and performance. Anxiety then replaced hopelessness as a negative predictor, while enjoyment was the strongest predictor of motives and performance. Taking eight emotions together, the arousal of activity and outcome emotions within visual programming environments facilitated students’ emotion dynamics that impacted motivation and performance. The discussion and contributions concerning the mechanism behind the effectiveness of visualization and the emotion-performance relationship are presented herein.

Keywords

Programming learning Scratch motivation performance achievement emotions

Introduction

Computer programming learning benefits higher-order competency development, such as computational thinking, problem-solving, creative thinking, and systematic experimentation, which is essential at various levels in the digital age.^1–4 However, learning a first programming language may challenge students. A longitudinal study by Connolly et al.⁵ found that low retention rates in computing courses are evident because learning programming may cause some computing students a lack of confidence and anxiety. Furthermore, related difficulties have been observed, including the poor ability to convert programming skills into practice,^3,6 the inadequate ability to understand the abstraction and complexity of programming concepts,^3,4,7 and the insufficient ability to deal with syntactic errors within complex logical structures.⁴ These challenges may lead to students’ negative attitudes toward programming and lower motivation levels, which may in turn significantly influence students’ learning achievement.^4,6,8,9

Previous research has highlighted the potential benefits of visual programming environments (VPEs) for motivating novice programmers.^2,6,10 Such software allows users to test and edit possibilities through materializing algorithm processes^11,12 and develop problem-solving skills concerning programming issues.³ Scratch is one of the most widely used VPE tools.^13,14 Previous studies^3,8,15 have indicated that Scratch implementation effectively increases students’ motivation and continuous desire to learn to code. The reasons behind this enhanced motivation could be the instruction period with a visualized, entertaining algorithm process⁸ and game design functionalities that encourage pleasant and comfortable rather than boring and difficult feelings during learning.^4,8 Although game design and visual programming structure have been previously emphasized, research into how emotions such as enjoyment are aroused to impact students’ learning by using Scratch in programming courses is quite limited.

To our knowledge, some studies have addressed beginner programmers’ emotions, such as enjoyment/interest, playfulness, and anxiety, in association with their performance and attitude toward programming^1,4,16–18; however, there are no studies that have included positive and negative activities and outcome achievement emotions, as suggested by Pekrun and his associates.^19–21 Furthermore, Scherer et al.²² recommended that the empirical evidence associated with the Scratch language and the mechanisms behind the effectiveness of visualization still need to be understood, particularly considering the potential of motivational and attitudinal factors in explaining learning mechanisms. To bridge this research gap, this study investigated the differences in students’ intrinsic motivation and performance with Scratch using a pretest-posttest intervention design and applied Pekrun's control-value theory of achievement emotions to understand the mechanism behind how visualization facilities affect and thereby impact learning performance.

This research can contribute to the literature by producing evidence on the actual effects of a Scratch-based intervention on the enhanced motivation and performance of students by using a pretest-posttest research design. Notably, drawing on studies of Pekrun's achievement emotions, this study offers insights into Scratch research in that we study how students’ perceptions of the visualized, entertaining algorithm process influence the performance of their achievement emotional experience, suggesting that the potential of achievement emotions in explaining the mechanism behind visualization intervention is effective.

Literature review

Scratch programming

Scratch is a visual programming language developed by the Media Lab of Massachusetts Institute of Technology.^1,23 Although Scratch was initially developed for children, it is often employed regardless of age, background, or interests and has become one of the most commonly used VPE tools.^4,13 In contrast to traditional textual languages, Scratch enables students to learn algorithms and programming logic with a simple interface and drag-and-drop structure without focusing on syntax, which renders the process more attractive by designing important and unique content⁸ and supports a more intuitive and engaging experience.²⁴ Specifically, Scratch provides an interactive media-rich environment for learners to create animated stories and games, leading to creativity and programming skill development in an exciting, fun way.^4,8,24

The advantages of Scratch for programming novices are evidential. Many researchers have identified positive results in learning, including programming knowledge and achievement,^8,23,25 computational thinking skills,^23,25 a positive attitude toward programming,^4,6,13,23 and learning motivation.⁸ Scherer et al.²² meta-analyzed previous studies and found that interventions focusing on visualization through Scratch are especially effective because visual programming languages may reduce the cognitive load associated with reading, understanding, and creating code. The authors also suggested that Scratch accommodates different projects, allowing for different interests and learning styles that may positively influence learners’ attitudes, resulting in better performance. Scratch shows its potential benefits of making computer programming accessible and appealing to students for academic success.

Achievement emotions

Emotions affect a wide range of cognitive processes associated with learning because emotions consume cognitive resources and then influence cognitive problem-solving, motivation, the self-regulation of learning, and the resulting achievement.^26,27 Positive and negative affect are two constructs generally used in prior educational studies in which the concept of test anxiety has gained popularity in the decades.^19,27 Achievement emotions are emotions that are both positive and negative and concern competence-relevant activities or outcomes.^19,20 Although previous research has mainly focused on achievement outcome-related emotions (e.g. success or failure), Pekrun et al.^19,20 suggested considering both activity- and outcome-related achievement emotions in addressing the effects of discrete emotions on students’ engagement and academic performance.

Drawing on control-value theory, Pekrun and his associates proposed a three-dimensional taxonomy of achievement emotions, namely, distinguishing activity versus outcome emotions of the object focus of achievement emotions; grouping positive and pleasant versus negative and unpleasant affects according to their valence; and differentiating activating versus deactivating effects according to their degree of activation. In the outcome focus category, positive emotions include hope, pride, gratitude, and relief, while negative emotions include anxiety, hopelessness, shame, disappointment, and sadness. Concerning activity emotions, positive emotions include enjoyment and relaxation, while negative emotions include boredom, anger, and frustration.^19,20,27,28 Additionally, Bieleke et al.²⁹ introduced a measurement framework to evaluate students’ emotional experiences in different educational settings (i.e. class-, learning-, and test-related situations). This study adopted the three-group taxonomy of achievement emotions suggested by Pekrun et al.¹⁹ Activity emotions include the positive emotion of enjoyment and the negative emotions of boredom and anger; positive outcome emotions include hope and pride; and negative outcome emotions include anxiety, hopelessness, and shame. Congruent with Tempelaar et al.³⁰ research, this study focused solely on learning-related emotions, leaving class- and test-related emotions beyond the scope of the study.

Emotions can be aroused and modulated by different control and value appraisals of situational perception, personal competencies, causal expectations, and outcome values.^27,28 Specifically, one should avoid distinguishing learners’ positive affect as good and negative affective experiences as bad. Positive emotions are sometimes detrimental, and negative emotions, such as anxiety and anger, are beneficial to learning under certain circumstances, suggesting that recent research evidence may be generally biased toward negative emotions and that ambivalent, inconsistent effects of activating negative emotions exist.^19,21,27,31 For example, Pekrun et al.¹⁹ identified the differential links between students’ achievement goals, their discrete emotions, and performance; the results support the existence of positive versus negative and activating versus deactivating effects of emotions compared to the general positive and negative affect observed in prior research. Artino Jr and Jones II³² observed that enjoyment in online learning contexts is a positive predictor of elaboration and metacognition that and unexpectedly, frustration, which is a negative activity emotion, positively predicts metacognition learning behavior. Camacho-Morles et al.³³ reviewed 68 studies focusing on activity-related emotions. The authors confirmed the positive association between enjoyment and academic achievement and the negative relations between anger and boredom and student performance. In contrast, no relationship between performance and frustration was found to exist.

Emotions are more complex than researchers have assumed, and a full range of students’ achievement emotions should be addressed more thoroughly.^21,26 The effects of activity emotions and the subject course of programming rather than traditional subject domains such as mathematics and literacy^34,35 should be researched further. This study bridged these gaps by focusing on both activity- and outcome-related emotions in programming educational contexts.

Motivation, achievement emotions, and programming learning

Motivation can be described as the inner power of people that drives them to take action toward a goal; it is differentiated into extrinsic and intrinsic motivation.^36,37 Extrinsic motivation refers to doing something due to a separable outcome, whereas intrinsic motivation is defined as pursuing an activity because of interest or enjoyment.^38,39 Both motivations may promote performance gains, but only intrinsic motivation can improve students’ psychological well-being and enhance creativity and learning outcomes.³⁷ Thus, this study focuses on the intrinsic motivation construct. Following Zainuddin,³⁹ this study specifies the relationships between competence, autonomy, relatedness needs, and intrinsic motivation. Competence is the need to be effective in one's pursuits and interactions with the environment, while autonomy is the need to experience behavior as self-determined by the self, and relatedness is the need to be emotionally connected with others.^37,39–41 Students with high levels of the three intrinsic needs are considered intrinsically motivated students in a Scratch-based programming learning environment.

Student motivation plays an essential role in effective programming learning; motivation is significantly influenced by attitude, expectation, and task difficulty in programming learning settings.^9,16,42 Students who have negative attitudes toward learning programming may be less motivated to overcome obstacles when facing difficulties.^4,8 Negative perceptions about barriers to classical textual programming learning facilitate abecedarians’ negative affects, such as anxiety and failure, both before and during the learning stages.⁴ Given the numerous difficulties observed in programming instruction processes, negative beliefs and affect may be aroused to undermine interest and intrinsic motivation.

In this regard, recent studies have identified the advantages of Scratch visualization in not only increasing students’ positive perceptions toward learning programming concepts but also motivating them and engaging them in learning to gain academic success. For example, Yukselturk and Altiok⁴ noted that Scratch effectively helps respondents’ anxiety change into feelings of relief and accomplishment, leading to positive attitudes toward programming. Erol and Kurt⁸ pointed out that the visual structure of Scratch makes the algorithm process more comprehensible and entertaining, thereby increasing learner motivation and achievement. Tsai³ also suggested that design-based learning in a VPE environment can allow self-efficacy learners to sidestep the obstacles of complex syntax, reduce the frustration and barriers of learning, and thus increase intrinsic motivation and performance.

Moreover, some studies have examined the relationships between VPE tools and student motivation from an emotional perspective. The affective predictors of increased motivation and performance include playfulness,¹ happiness,¹⁷ relief,⁴ and enjoyment;¹⁸ emotions negatively affecting achievement include interest¹⁸ and anxiety,^1,4,17 whereas no significant effects of emotions are portrayed as interests/enjoyment and pressure/tension.¹⁶ However, as mentioned earlier, achievement emotions are more complex than recent evidence shows and should be addressed more thoroughly in educational settings.^21,26 Therefore, research into the relationships between motivation, emotions, and programming learning—considering activity- and outcome-relevant achievement emotions—may aid in understanding the mechanisms behind the effectiveness of Scratch visualization and programming novices’ motivational factors and performance outcomes.

Research questions

Drawing upon the literature discussed above, this study predominantly emphasized the role of achievement emotions in programming learning with Scratch to understand the effectiveness of visualization and the emotion-performance relationship. The following research questions guided the investigation:

RQ1: Do students increase their motivation level when integrating a Scratch-based intervention into introductory programming learning?

RQ2: Do students increase their examination performance level when integrating Scratch-based intervention into introductory programming learning?

RQ3: How do achievement emotions influence students’ programming learning and increase their motivation and achievement levels in a Scratch-based programming learning environment?

Methodology

Participants

This study's participants consisted of 172 undergraduates in Taiwan during the 2022 academic year. The sample included 96 females (55.8%) and 76 males (44.2%). The mean age of the participants was 20.44 years (SD = 1.09, range 19–23). Most respondents (86%) reported not receiving computer- or programming-related courses and extracurricular activities before participating in this workshop. This study recruited volunteers for a six-lesson introductory programming training workshop with the Scratch visual language. Prior to starting this workshop, participants were instructed to complete assessments and questionnaires during the first class held before the instructional period and after the workshop; they were instructed to learn and practice programming assignments and develop animation projects. The participants agreed to take part in the workshop in a completely voluntary manner; they were not provide with either course credit or extra credit for their participation, and they completed informed consent forms prior to the study.

Procedure

The workshop's visual programming environment (VPE) tool was Scratch 3.0. Scratch is a visual, block-based programming language used by novices to learn computer-related concepts; it also provides an interactive media-rich, user-friendly environment for users to develop their projects collaboratively in the form of animations, games, and stories.^3,13,23,25 The lessons, including lectures and practice, were delivered in a computer classroom.

A schedule of the preparation, intervention, and assessment activities covered in the six-lesson workshop is outlined in Table 1. The time allocation and duration of each session sometimes varied and depended on the included intervention activities and concepts. In the first lesson, students took a computer pretest and answered survey questionnaires; orientation activities were carried out to increase the level of recognition between the instructor and participants; and the instructor delivered lectures concerning basic computer-related concepts and an introduction to programming. A short review of lesson 1 and an introduction to the functionalities of varying blocks of Scratch were conducted in the second lesson. Afterward, teaching-learning activities concerning the three basic programming concepts of sequence, loop, and condition were introduced in the third and fourth lessons. Later, in the last two lessons, students developed their final animation projects and completed questionnaires and posttest examinations. The written assessment examination comprised 20 multiple-choice and five short-answer questions. Figure 1 shows an example project.

Figure 1.

Example of a Scratch project.

Table 1.

Instruction content.

Le	Intervention content	Period	Duration
1	Assessments: Pretest and questionnaires Students’ orientation Lecture: Introduction to basic computer-related concepts and programming	One week	Three hours
2	Review of Lesson 1 Lecture: Introduction to Scratch applications	One week	Three hours
3	Review of Lesson 2 Lecture: Introduction to the topics of sequence and loop and practice using Scratch	One week	Four hours
4	Review of Lesson 3 Lecture: Introduction to the topic of condition and practice using Scratch	One week	Four hours
5–6	Review of Lessons 3 and 4 Project task: Developing the final animation projects Assessments: Posttest and questionnaires	Two weeks	Six hours

During the learning period, the lecturer tried to recognize each participant, make participants familiar with others, and manage the class atmosphere as a friendly rather than competitive learning environment. The instructor gave participants in-class assignments or practice to ensure their understanding of the topics they had been taught rather than allow students to see how they performed relative to others. Furthermore, the instructor reviewed the previous lesson before the class and left extra time afterward for students who demonstrated learning obstacles related to the content. This approach aimed to help every learner participate in learning and practicing Scratch programming without showing his or her instantaneous performance for appraisals in classes. The questionnaires and written test papers were anonymous and recorded with students’ serial numbers.

Measures

This study employed two survey questionnaires and one written examination. The questionnaires used a 7-point Likert-type agreement response scale, and all item descriptions were adapted to the Scratch programming learning experience. The first questionnaire was a short version of the Achievement Emotions Questionnaire (AEQ-S) developed by Bieleke et al.²⁹ which is used to assess students’ achievement emotions. The AEQ-S provides a reliable instrument with short scales and asks respondents to describe how they feel in class, during a course, and while taking tests. This study mainly focused on Scratch learning-related achievement emotions. Participants responded using a scale ranging from 1 (strongly disagree) to 7 (strongly agree). Following Pekrun et al.,¹⁹ this study considered three emotion categories consisting of eight discrete emotions, namely, negative outcome emotions (anxiety, hopelessness, and shame), activity emotions (boredom, anger, and enjoyment), and positive outcome emotions (hope and pride).

The negative outcome emotion scales assessed anxiety (four items; e.g. “I get tense and nervous while studying Scratch programming”), hopelessness (three items; e.g. “I feel helpless while studying Scratch programming”), and shame (four items; e.g. “I feel ashamed while studying Scratch programming”). The activity emotion scales assessed boredom (four items; e.g. “Studying for my courses with Scratch programming bores me”), anger (four items; e.g. “Studying Scratch programming makes me irritated”), and enjoyment (four items; e.g. “I enjoy the challenge of learning the material concerning Scratch programming”). The positive outcome emotion scales assessed hope (four items; e.g. “I feel confident when studying Scratch programming”) and pride (four items; e.g. “I feel proud of myself while studying Scratch programming”).

The second motivation questionnaire was adapted from Zainuddin.³⁹ It consisted of fifteen items based on the three intrinsic needs: five items on competence (e.g. “I am more competent at learning and mastering new skills while learning programming by Scratch”), five items on autonomy (e.g. “I am more able to control the learning environment while learning programming by Scratch”), and five items on relatedness (e.g. “I am more able to interact with peers in class while learning programming by Scratch”). The Cronbach's alpha values of achievement emotions and motivation reported above exceeded the acceptable cutoff of 0.7.⁴³ Except for the pretest factor loading value for one item of hopelessness, which was 0.46, all item factor loadings surpassed 0.7 and were significant.⁴⁴ Table 2 shows the reliability and validity of the measurement items before and after the intervention.

Table 2.

Reliability and validity of the measurement items of the pretest and posttest.

		Pretest		Posttest
Variables	Number of items	Cronbach's alpha	Factor loading range	Cronbach's alpha	Factor loading range
1. Anxiety	4	0.91	0.85–0.93	0.91	0.87–0.91
2. Hopelessness	3	0.94	0.93–0.95	0.96	0.94–0.96
3. Shame	4	0.92	0.84–0.93	0.96	0.93–0.96
4. Boredom	4	0.95	0.90–0.95	0.97	0.95–0.97
5. Anger	4	0.94	0.90–0.94	0.96	0.94–0.96
6. Enjoyment	4	0.93	0.89–0.93	0.93	0.89–0.93
7. Hope	4	0.90	0.79–0.94	0.93	0.88–0.94
8. Pride	4	0.86	0.76–0.91	0.87	0.72–0.92
9. Competence	5	0.93	0.85–0.93	0.97	0.92–0.95
10. Autonomy	5	0.92	0.79–0.91	0.94	0.88–0.92
11. Relatedness	5	0.91	0.82–0.91	0.94	0.86–0.93

Finally, the test concerning basic programming concepts consisted of 20 multiple-choice items and five short answer questions. An example is as follows: “In Scratch, how do you make a sprite appear again after it has been hidden? Options: 1. ‘show’ block; 2. ‘repeat’ block; 3. ‘wait’ block; 4. ‘forever’ block.” Another example is as follows: “What does the ‘repeat until’ block do in Scratch? Options: 1. It runs a block of code once; 2. It runs a block of code a specified number of times; 3. It runs a code block until a specified condition is true; 4. It stops running a block of code.” An example of the short-answer question is as follows: “What does the “if… then… else” block do in Scratch? Please provide an example.” The examination test was adopted to evaluate students’ performance using a 100-point scale during the posttest period.

Results

Descriptive statistics and Pearson correlations

Table 3 presents the descriptive statistics and correlation analysis results of the posttest values of all the assessed emotions, motivation, and the performance variables. Despite the anger emotion, in line with Pekrun et al.,¹⁹ an examination of the correlations revealed that the motivational effects of the discrete emotions were individually significant and in the expected direction. Negative outcome emotions such as anxiety, hopelessness, and shame were negatively correlated with motivation and examination performance. Concerning activity emotions, boredom had a negative influence on motivation and achievement, while enjoyment had a positive effect on intrinsic needs and performance. The positive outcome emotions of hope and pride were expected to be positive predictors of motives and performance attainment.

Table 3.

Descriptive statistics and correlation results for posttest results.

Variables	1	2	3	4	5	6	7	8	9	10	11	12
1. Anxiety
2. Hopelessness	0.82**
3. Shame	0.88**	0.85**
4. Boredom	0.78**	0.90**	0.84**
5. Anger	0.90**	0.73**	0.86**	0.75**
6. Enjoyment	−0.19*	−0.25**	−0.23**	−0.26**	−0.15*
7. Hope	−0.21**	−0.22**	−0.17*	−0.21**	−0.12	0.88**
8. Pride	−0.13	−0.22**	−0.12	−0.20**	−0.08	0.77**	0.80**
9. Competence	−0.23**	−0.29**	−0.25**	−0.30**	−0.16*	0.44**	0.45**	0.38**
10. Autonomy	−0.21**	−0.33**	−0.28**	−0.31**	−0.10	0.67**	0.68**	0.57**	0.61**
11. Relatedness	−0.20*	−0.24**	−0.22**	−0.26**	−0.14	0.82**	0.79**	0.77**	0.50**	0.77**
12. Examination performance	−0.19*	−0.26**	−0.19*	−0.23**	−0.09	0.83**	0.86**	0.81**	0.47**	0.78**	0.91**
Mean	3.73	3.45	3.50	3.33	3.53	4.99	4.89	4.94	5.35	5.05	5.11	67.27
SD	1.45	1.47	1.46	1.54	1.46	0.95	0.97	0.97	1.12	0.96	0.97	13.24

Note: *p < 0.05, **p < 0.01 (two-tailed)

Analyses of motivation and examination performance

The paired sample t test examined the motivational and performance changes between the pretest and posttest scores. Table 4 shows that overall motivation significantly changed due to the Scratch-based intervention in programming learning (t = – 3.83, p < 0.001). Competence and autonomy need significantly increased (t = – 6.11, p < 0.001, t = – 3.20, p < 0.01, respectively), whereas relatedness motive did not show a significant difference between the pretest and posttest scores (t = – 0.56, p > 0.05). Table 4 shows that the Scratch–based intervention improved students’ examination performance (t = – 5.74, p < 0.001).

Table 4.

The paired sample t test results for motivation and performance.

Variables	Groups	Mean	SD	t value	p value	_η2
Overall Motivation	Pretest	4.81	0.88	−3.83	0.000***	0.04
Overall Motivation	Posttest	5.17	0.89
Competence	Pretest	4.65	1.01	−6.11	0.000***	0.10
Competence	Posttest	5.35	1.12
Autonomy	Pretest	4.72	0.97	−3.20	0.002**	0.03
Autonomy	Posttest	5.05	0.96
Relatedness	Pretest	5.05	0.88	−0.56	0.577	0.00
Relatedness	Posttest	5.11	0.97
Examination performance	Pretest	59.07	13.23	−5.74	0.000***	0.08
Examination performance	Posttest	67.27	13.24

Note: * p < 0.05, ** p < 0.01, *** p < 0.001

Hierarchical linear regression analyses of achievement emotions

This study performed a hierarchical linear regression analysis to examine the effects of achievement emotions on overall motivation and performance variables. Following Pekrun et al.¹⁹ achievement emotions can be categorized into three groups: negative outcome emotions (anxiety, hopelessness, and shame), activity emotions (boredom, anger, and enjoyment), and positive outcome emotions (hope and pride). The analysis thus comprised the three models. First, three negative outcome emotions were entered into the regression as a block. Second, the independent variables of boredom, anger, and enjoyment were simultaneously entered into the regression to examine the effect of the activity emotion category on overall motivation and performance. Finally, two positive outcome emotions were entered as a block.

Moreover, in line with Artino Jr and Jones II,³² the gender and age of the respondents were found to significantly influence neither overall motivation nor achievement performance in this study. Thus, these two variables were not retained in the regression analyses. The variance inflation factors of the analyses in this study ranged from 1.09 to 8.73; these values are lower than 10, suggesting that the multicollinearity issue was not a problem in the current study.⁴³ The emotion categories (i.e. negative outcome, activity, and positive outcome) are presented as signified by the values of the incremental F-statistic. The values of the β's signified the direct effects of the discrete emotions.

Model 1 in Table 5 shows that the block of the three negative outcome emotions was significantly identified as the predictor of overall motivation (F = 5.78, p < 0.01). Hopelessness was negatively correlated with students’ motivation level (β = - 0.37, p < 0.05), whereas anxiety and shame showed no significant β-values. Consequently, among the three negative outcome emotions, hopelessness rather than anxiety and shame was found to explain the negative effects on motivation. After activity emotions were entered (see Model 2 in Table 5), the regression result for the effect of the block of activity emotions on motivation was significant (F = 122.43, p < 0.001). The activity emotion category impacted motivational needs by an additional 63% variance. The β-value of anxiety showed a significant negative effect (β = - 0.27, p < 0.05), and the β-value of enjoyment exhibited a significant positive effect (β = 0.81, p < 0.001). In contrast, the β-values of other discrete emotions were nonsignificant. In this case, the relationships between achievement emotions and overall motivation showed increased complexity. The negative predictor of learner motivation changed from hopelessness to anxiety, and the positive activity emotion of enjoyment was expected to highly enhance motivation. In the final model shown in Table 5, it can be seen that the relationship between the incremental F value for the block of positive outcome emotions and overall motivation was significant (F = 20.43, p < 0.001); adding hope and pride to the model explained an additional 6% of the variance. The significant, influential emotions became enjoyment (β = 0.35, p < 0.001), hope (β = 0.36, p < 0.001), and pride (β = 0.19, p < 0.01). This result showed that positive outcome emotions’ motivational effects were significantly activated during the study. Specifically, compared to negative affect, the results implied that positive emotions such as enjoyment, hope, and pride could more significantly increase learner motivation. The overall model effect was R² = 0.76.

Table 5.

Hierarchical regression analysis for emotions and overall motivation.

Independent variable entered	Model 1	Model 2	Model 3
Independent variable entered	β	β	β
Anxiety	−0.06	−0.27*	−0.13
Hopelessness	−0.37*	−0.18	−0.12
Shame	0.14	0.19	0.06
Boredom		0.02	0.00
Anger		0.17	0.12
Enjoyment		0.81***	0.35***
Hope			0.36***
Pride			0.19**
Δ F for model	5.78**	122.43***	20.43***
F for regression	5.78**	70.37***	70.32***
Δ R²	0.09	0.63	0.06
Adjusted R²	0.08	0.71	0.76

Note: * p < 0.05, ** p < 0.01, *** p < 0.001

Table 6 shows the hierarchy regression analyses for the relationships between achievement emotions and examination performance. Similarly, when examining negative outcome emotions, the results showed a significant effect of the negative outcome emotion category on the performance dependent variable (F = 4.31, p < 0.01), and hopelessness was a significant negative predictor of academic outcome (β = - 0.36, p < 0.05). In Model 2 in Table 6, adding the activity emotion block to the model revealed the significant effects of activity emotions on performance (F = 124.05, p < 0.001) and explained an additional 64% of the variance. Anxiety became a negative predictor of performance (β = - 0.26, p < 0.05), whereas anger (β = 0.27, p < 0.05) and enjoyment (β = 0.82, p < 0.001) were both positive predictors of examination outcome. Finally, both the incremental F value for the block of positive outcome emotions (F = 38.45, p < 0.001) and the overall influence of emotions on performance was significant (F = 84.76, p < 0.001), thereby indicating the significant effects of activity and positive outcome emotions. Specifically, students’ improvements in examination tests were found to be achieved by emotions of anger (β = 0.23, p < 0.05), enjoyment (β = 0.25, p < 0.01), hope (β = 0.38, p < 0.001), and pride (β = 0.30, p < 0.001), leading to the overall model effect being high R² = 0.80.

Table 6.

Hierarchical regression analysis for emotions and performance.

Independent variable entered	Model 1	Model 2	Model 3
Independent variable entered	β	β	β
Anxiety	0.02	−0.26*	−0.12
Hopelessness	−0.36*	−0.17	−0.08
Shame	0.10	0.09	−0.08
Boredom		0.05	0.02
Anger		0.27*	0.23*
Enjoyment		0.82***	0.25**
Hope			0.38***
Pride			0.30***
Δ F for model	4.31**	124.05***	38.45***
F for regression	4.31**	68.91***	84.76***
Δ R²	0.07	0.64	0.09
Adjusted R²	0.06	0.70	0.80

Note: * p < 0.05, ** p < 0.01, *** p < 0.001

Discussion

The effect of scratch on motivation and performance

The results of the paired sample t test shown in Table 4 indicated that participants who finished the six-lesson training workshop were more motivated than they were during the pretest stage. RQ1 was thus answered. This increase from the scratch-based intervention significantly differentiated the outcomes of students’ overall motivation, competence need, and autonomy need but not their relatedness need. These findings are parallel to those of prior related studies concerning the positive relationship between activities that use Scratch and participants’ motivation.^3,8,15 Specifically, we presented the effects of the different motivation types derived from a Scratch-based intervention in a more nuanced way rather than using only one specific aspect of motivation, such as intrinsic motivation, as is often found in the previous literature.

The increased competence level suggested that participants felt competent and efficient in regard to programming achievements. It might be that an intuitive, user-friendly programming environment makes it relatively easy for learners to perform tasks while reducing the level of frustration derived from programming syntax and logic.^3,24 An increase in autonomy need revealed that students feel autonomous and independent when learning programming concepts based on their interests and values. This could be because Scratch makes the algorithm process visualizable and supports animated project creation, which lowers the barriers to the compilation process and increases the level of student participation.^3,8,45 It is essential to note that the positive outcomes of overall motivation and autonomy need appeared but had small effect sizes (η² = 0.04 and 0.03, respectively); thus, this study suggests carefully addressing the visualization causal effect on motivation. Furthermore, Scratch did not increase students’ relatedness motives; this is because most participants were observed to perform learning tasks individually, not collaboratively, leading to no changes in their relatedness need.

The Scratch-based intervention was found to lead to better learning achievements. Learners’ posttest scores were relatively higher than those at the pretest stage (see Table 4). These results supported RQ2. This finding could be attributed to visualization reducing the cognitive challenges related to understanding and creating code.²² The instruction process using blocks benefits the learning of programming logic through a constructive approach⁸ that offers an engaging experience^4,24 and an easier method,³ which in turn enhances learners’ performance. Our results are consistent with past studies concerning Scratch's effectiveness in increasing students’ understanding of algorithm processes and programming concepts.^3,8,22

The effect of achievement emotions on motivation and performance

This study has established evidence supporting the significant relationships between achievement emotions and students’ overall motivation and examination performance (see the results shown in Tables 5 and 6). RQ3 was thus answered. Sequentially considering the negative outcome, activity, and positive outcome emotion categories, the regression findings suggested that participants’ emotional changes are complex and dynamic during the learning process.³² The results also confirmed previous research in which the appearance of discrete emotions was found to likely depends on learners’ perceived control and the value of the learning context and material.²⁷

Emotional change dynamics may result from the subjective importance of activity- and outcome-related emotions in a Scratch-based programming learning context. First, the arousal of hopelessness in regression Model 1 added empirical evidence supporting the negative relations of abecedarians’ belief about programming difficulties, incapability, and lower comfort levels to their learning perceptions and subsequent consequences.^3,42,46 If anticipated learning failures occur, then students who likely posited their inability to engage in programming logic will exhibit hopelessness that will in turn negatively affect their motives and performance. Consequently, educators’ effort to minimize the novices’ negative outcome emotions is recommended to motivate their continuous learning desire.^5,27

Second, after adding activity emotions to the model, negative outcome emotion anxiety replaced hopelessness to show students’ anticipated failure in learning (β's = −0.26 — −0.27); activity emotion enjoyment became the strongest positive predictor of the regression models (β's = 0.81—0.82). Learner anxiety was found to be related less closely to motivation and achievement than enjoyment, suggesting that positive affect exhibited higher effects than negative affect. This study confirmed that the more general emotions students judge and experience under Scratch learning mechanisms are anxiety and enjoyment.^1,4,17,18 Moreover, outcome and activity emotions could be aroused and simultaneously affect learners’ motives and performance enhancement, suggesting that activity emotions in achievement settings are equally relevant to outcome emotions.^21,27 Achievement emotion changes may thus provide the potential to explain the mechanisms behind the effectiveness of Scratch visualization.²²

Third, when the eight abovementioned emotions were considered together, the positive outcome emotions of hope and pride significantly predicted learner motivation and performance. In line with previous studies,^19,21,32 our results suggested that the notions of “positive” emotions, regardless of the objective focus of Scratch activities (activity emotions) or learners’ perceived control of examinations (outcome emotions); our results also predicted higher motivation and achievement levels through substantial physiologically activating effects. Specifically, the coefficient for pride in predicting performance (β's = 0.30) was higher than that in predicting motivation (β's = 0.19). The outcome emotion of pride was shown to relate more strongly to the case of learners who mastered the examination than to the case of increasing perceptions about programming. It could be that Scratch effectively helped students gain performance success and causal attribution of achievement to their ability or effort, thus inducing pride.^4,27 The findings extended the emotion-performance link not only to the subject course of programming rather than the traditional mathematics and literacy subjects but also to a full range of students’ affective experiences (i.e. activity- and outcome-related emotions) that extended beyond outcome emotions (e.g. success or failure).^21,26,27,47

Last, this study found that anger was uncorrelated with performance at the bivariate level. However, it positively predicted examination performance when the activity emotion block of enjoyment, boredom, and anger entered the regression. As seen in Artino Jr and Jones II,³² anger can be a suppressor variable, increasing the predictive validity of another variable or set of variables by its inclusion in a regression model.⁴⁸ Therefore, rather than solely considering anger's regression coefficient, this study suggested combining anger and enjoyment to address the combined effects of activity emotions.^48,49 Concerning activating negative emotions, scholars^19,28 noted that these emotions may evoke intrinsic needs to invest the effort to avoid failure. Thus, the visual structure and interactive media-rich functionality, both of which reduce beginners’ cognitive barriers and support an engaging experience,^8,24 could enable students to control and value Scratch tasks and put more effort into avoiding failure. Moreover, the underlying functional mechanisms concerning activating negative emotions are more complex.^27,28 Pekrun and Stephens²⁸ noted that the effects of negative activating emotions on motivation and effort depend on the situation-dependent and person-specific balance of the different mechanisms. Artino Jr and Jones II³² also indicated that cognition, affect, and behavior are dynamic elements that work in context. Accordingly, this study suggested carefully considering Scratch's activity tasks and environments and individuals’ beliefs and expectancies when interpreting the effects of activating negative predictors such as anger.

Conclusion

This study investigated whether a Scratch-based intervention in programming learning can promote student motivation and examination performance. The results showed that the use of Scratch effectively increased participants’ overall motivation and performance levels. When simultaneously considering eight emotions, the results revealed participants’ complex, dynamic affective reactions during the learning processes, showing substantial effects on educational outcomes.

Research and practical contributions

The present investigation offers theoretical and empirical contributions. First, the results concerning increased competence and autonomous motivation are a theoretical lens through which to account for the underlying motivational processes that shape how and whether Scratch works. This research added a more nuanced assessment to extant studies regarding the motivational effects of Scratch in programming learning.^3,8,15 Additionally, this study presented evidence of learners’ affective experience dynamics within a Scratch-based programming learning environment. The results extended the emotion-performance link to not only the subject course of programming rather than the traditional mathematics and literacy subjects but also a full range of activity and outcome emotions beyond success and failure and test anxiety.^21,26,27,47 Finally, the observed significant effects of different motivational types and emotional changes may deepen the knowledge of novice programmers’ differentiated interpretations and reactions to Scratch-based interventions. The findings added evidence to Scratch's enjoyable and appealing environment for achievement attainment.³¹ Further, they explained the mechanisms behind the effectiveness of Scratch visualization, which demands further research.²²

This study also yields practical implications. Given the significant motivational effects of emotions on the desired outcomes, educators’ efforts to minimize students’ maladaptive emotions and foster their adaptive emotions in programming learning are recommended. Emotion-performance links could be more complex and change over time,^19,27,32 and visualization is differentially effective across student samples.²² Therefore, this study suggests considering how other educational issues within visual programming environments affect what and how emotions will likely be aroused to facilitate positive consequences. These issues may include (1) learner characteristics such as individual self-efficacy, effort, achievement goals, and the value of academic success and failure^3,19,28,46; (2) curriculum and pedagogical issues concerning task and activity design, instructors’ delivery, examination assessments, and strategies for learning^{3,16,22,28,32}; and (3) shaping programming learning environments to improve students’ control and value-related appraisals for increased levels of adaptive achievement emotions and decreased levels of maladaptive emotions.^4,19,27,28

Limitations and future work

This study has several limitations. First, this research employed a pretest-posttest intervention design and recruited volunteer participants to identify the effectiveness of Scratch. Participants’ demographics, academic backgrounds, and contextual variables were not carefully considered. Second, Scratch was the only VPE language used in this study to understand the motivational effects of participants’ achievement emotions. Third, although this study focused on eight activity and outcome emotions, knowledge of emotions with varying degrees of activating and deactivating effects, particularly in regard to activating negative emotions, still needs to be addressed.^21,27,28

The recommendations for future studies are as follows. First, the results of this study suggest carefully aligning the study design features of rigorous experimental designs, different specific groups of students’ characteristics, instructional conditions, and measurements to reduce the possible methodological bias and understand the actual intervention effects.²² Second, researchers could consider other similar tools, such as Alice and App Inventor, which previous studies^1,3,23 have indicated to be effective in programming learning, to continuously monitor emotion dynamics and their impact on learner behavior outcomes under visualization interventions. Third, future studies could consider variables concerning regulation, treatment, mechanisms of learning, and the redesign of achievement settings to investigate how to promote adaptive achievement emotions and prevent maladaptive emotions that increase students’ interests and help them achieve academic development in programming learning.²⁷ Finally, mixed-method research strategies and experimental research are highly recommended to examine the different dynamics between individual cognition, emotions, and behaviors within the examined context, thereby advancing the related knowledge to varying degrees of activating and deactivating effects, particularly in the category of activating negative emotions.^28,32

Footnotes

Abbreviations

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Science and Technology Council, Taiwan (grant number MOST 109-2511-H-344-001, MOST 110-2511-H-344-001-MY3).

ORCID iD

Tienhua Wu

Author biographies

Fu-Hsiang Wen is a Lecturer at the Department of Management at the Air Force Institute of Technology in Kaohsiung, Taiwan. He is a PhD candidate at the PhD Program in Engineering Science and Technology College of Engineering, National Kaohsiung University of Science and Technology in Kaohsiung, Taiwan. His current research interests focus on network and information security, computer programming education, and information security management systems.

Tienhua Wu is a Professor at the Department of Management at the Air Force Institute of Technology in Kaohsiung, Taiwan. Her research interests include entrepreneurship and business education, information security and programming education, innovation and entrepreneurship, risk management, sustainable supply chain management, and military logistics management.

Wei-Chih Hsu is a Professor at the Department of Computer and Communication Engineering at the National Kaohsiung University of Science and Technology in Kaohsiung, Taiwan. With a rich academic background and years of research experience, he focuses on smart technology, particularly the application and development of emerging technologies such as artificial intelligence, the Internet of Things (IoT), information security, and blockchain. In addition to research, he is also dedicated to nurturing the next generation of tech professionals and has accumulated extensive experience in teaching.

References

Chang

. Effects of using Alice and Scratch in an introductory programming course for corrective instruction. J Educ Comput Res 2014; 51: 185–204.

Ching

Hsu

Baldwin

. Developing computational thinking with educational technologies for young learners. TechTrends 2018; 62: 563–573.

Tsai

. Improving students’ understanding of basic programming concepts through visual programming language: the role of self-efficacy. Comput Human Behav 2019; 95: 224–232.

Yukselturk

Altiok

. An investigation of the effects of programming with Scratch on the preservice IT teachers’ self-efficacy perceptions and attitudes towards computer programming. Br J Educ Technol 2017; 48: 789–801.

Connolly

Murphy

Moore

. Programming anxiety amongst computing students—a key in the retention debate? IEEE Trans Educ 2008; 52: 52–56.

Cheng

. Exploring factors influencing the acceptance of visual programming environment among boys and girls in primary schools. Comput Human Behav 2019; 92: 361–372.

Erodogan

Aydin

Kabaca

. Exploring the psychological predictors of programming achievement. J Instr Psychol 2008; 35: 264.

Erol

Kurt

. The effects of teaching programming with Scratch on pre-service information technology teachers’ motivation and achievement. Comput Human Behav 2017; 77: 11–18.

Law

Lee

. Learning motivation in e-learning facilitated computer programming courses. Comput Educ 2010; 55: 218–228.

10.

Hsu

Rice

Dawley

. Empowering educators with Google's android app inventor: an online workshop in mobile app design. Br J Educ Technol 2012; 43: E1–E5.

11.

Baldwin

Kuljis

. Learning programming using program visualization techniques. J Comput Inf Technol 2000; 8: 285–291.

12.

Powers

Gross

Cooper

, et al. Tools for teaching introductory programming: what works? In: Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education. Association for Computing Machinery; 2006: 560–561.

13.

Marcelino

Pessoa

Vieira

, et al. Learning computational thinking and Scratch at distance. Comput Human Behav 2018; 80: 470–477.

14.

Resnick

Maloney

Monroy-Hernández

, et al. Scratch: programming for all. Commun ACM 2009; 52: 60–67.

15.

Ouahbi

Kaddari

Darhmaoui

, et al. Learning basic programming concepts by creating games with scratch programming environment. Procedia Soc Behav Sci 2015; 191: 1479–1482.

16.

Facey-Shaw

Specht

Van Rosmalen

, et al.

Do badges affect intrinsic motivation in introductory programming students?

Simul Gaming 2020; 51: 33–54.

17.

Giannakos

Jaccheri

Leftheriotis

. Happy girls engaging with technology: assessing emotions and engagement related to programming activities. In: Learning and collaboration technologies. Designing and developing novel learning experiences. New York, NY: Springer, 2014, pp.398–409.

18.

Kanaparan

Cullen

Mason

. Effect of self-efficacy and emotional engagement on introductory programming students. Australas J Inf Syst 2019; 23: 24.

19.

Pekrun

Elliot

Maier

. Achievement goals and achievement emotions: testing a model of their joint relations with academic performance. J Educ Psychol 2009; 101: 115–135.

20.

Pekrun

Frenzel

Goetz

, et al. The control-value theory of achievement emotions: an integrative approach to emotions in education. In: Schutz

Pekrun

(eds) Emotion in education. San Diego, CA: Academic Press, 2007, pp.13–36.

21.

Pekrun

Goetz

Titz

, et al. Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educ Psychol 2002; 37: 91–105.

22.

Scherer

Siddiq

Viveros

. A meta-analysis of teaching and learning computer programming: effective instructional approaches and conditions. Comput Human Behav 2020; 109: 106349.

23.

Papadakis

Kalogiannakis

Zaranis

, et al. Using Scratch and App inventor for teaching introductory programming in secondary education. A case study. Int J Technol Enhanc Learn 2016; 8: 217–233.

24.

Papadakis

Kalogiannakis

. Evaluating a course for teaching introductory programming with Scratch to pre-service kindergarten teachers. Int J Technol Enhanc Learn 2019; 11: 231–246.

25.

Piedade

Dorotea

. Effects of Scratch-based activities on 4th-grade students’ computational thinking skills. Inform Educ 2022. doi:10.15388/infedu.2023.19

26.

Patulny

Bellocchi

Olson

, et al. Emotions in late modernity. Oxon: Taylor & Francis, 2019.

27.

Pekrun

. Achievement emotions: a control-value theory perspective. In: Patulny

Bellocchi

Olson

, et al. (eds) Emotions in late modernity. Oxon: Taylor & Francis, 2019, pp.142–157.

28.

Pekrun

Stephens

. Achievement emotions in higher education. In: Smart

(ed.) Higher education: handbook of theory and research. New York, NY: Springer, 2010, pp.257–306.

29.

Bieleke

Gogol

Goetz

, et al. The AEQ-S: a short version of the achievement emotions questionnaire. Contemp Educ Psychol 2021; 65: 101940.

30.

Tempelaar

Niculescu

Rienties

, et al. How achievement emotions impact students’ decisions for online learning, and what precedes those emotions. Internet High Educ 2012; 15: 161–169.

31.

Loderer

Pekrun

Lester

. Beyond cold technology: a systematic review and meta-analysis on emotions in technology-based learning environments. Learn Instr 2020; 70: 101162.

32.

Artino

Jr Jones

II . Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning. Internet High Educ 2012; 15: 170–175.

33.

Camacho-Morles

Slemp

Pekrun

, et al. Activity achievement emotions and academic performance: a meta-analysis. Educ Psychol Rev 2021; 33: 1051–1095.

34.

Tan

. Relationships between achievement emotions, motivation and language learning strategies of high, mid and low English language achievers. J Lit Art Stud 2017; 7: 770–778.

35.

Peixoto

Sanches

Mata

, et al. How do you feel about math?” Relationships between competence and value appraisals, achievement emotions and academic achievement. Eur J Psychol Educ 2017; 32: 385–405.

36.

Hanus

Fox

. Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput Educ 2015; 80: 152–161.

37.

Ryan

Deci

. Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 2000; 25: 54–67.

38.

Ryan

Deci

. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55: 68–78.

39.

Zainuddin

. Students’ learning performance and perceived motivation in gamified flipped-class instruction. Comput Educ 2018; 126: 75–88.

40.

Mekler

Brühlmann

Tuch

, et al. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput Human Behav 2017; 71: 525–534.

41.

Sailer

Hense

Mayr

, et al. How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. Comput Human Behav 2017; 69: 371–380.

42.

Alturki

. Measuring and improving student performance in an introductory programming course. Inform Educ 2016; 15: 183–204.

43.

Hair

Black

Babin

, et al. Multivariate data analysis. New Jersey: Prentice-Hall, 2006.

44.

Fornell

Larcker

. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res 1981; 18: 39–50.

45.

Papadakis

Kalogiannakis

Orfanakis

, et al. The appropriateness of Scratch and app inventor as educational environments for teaching introductory programming in primary and secondary education. Int J Web Based Learn Teach Technol 2017; 12: 58–77.

46.

Bergin

Reilly

. Predicting introductory programming performance: a multi-institutional multivariate study. Comput Sci Educ 2006; 16: 303–323.

47.

Pekrun

. The control-value theory of achievement emotions: assumptions, corollaries, and implications for educational research and practice. Educ Psychol Rev 2006; 18: 315–341.

48.

Pandey

Elliott

. Suppressor variables in social work research: ways to identify in multiple regression models. J Soc Social Work Res 2010; 1: 28–40.

49.

Maassen

Bakker

. Suppressor variables in path models: definitions and interpretations. Sociol Methods Res 2001; 30: 241–270.