Examining the Effect of the Task-Technology Fit of Game Mechanisms on Learning Outcomes in Online Gamification Platforms

Abstract

The designs of gamification platforms are diverse and constantly evolving. Excessive use of various game mechanisms in learning platforms can distract from the learning process. However, the fit of game mechanisms is still uncertain. Thus, this study investigates the effect of achieving fit when implementing game mechanisms on learning outcomes by applying the well-known task-technology fit theory (TTF). TTF is frequently employed to improve fit between tasks to be completed and the technology applied. The findings indicate that achieving gamification fit can reduce the cognitive load of students and result in enhanced learning performance in terms of learning outcomes. Data collected from 266 participants were analyzed using the technique of the partial least squares to validate the developed research model. The findings of this study can aid educators and educational technology designers in identifying the design mechanisms and characteristics that can be used to ensure design fit on gamification platforms.

Keywords

gamification game mechanism task-technology fit learning performance cognitive load

Introduction

In this era, gaming has become a crucial aspect of human culture and society (Bozkurt & Durak, 2018; Krath et al., 2021). Several studies have demonstrated that games promote individual engagement, motivation, and productivity (Alsawaier, 2018; Kiron et al., 2020; Liu et al., 2021; Xu et al., 2021; Yu et al., 2021). In the educational context, the use of a gaming approach to learning is referred to as game-based learning or gamification (Lester et al., 2023). Game-based learning involves developing a sense of accomplishment through the use of game content and play involving problem solving and challenges (Dahalan et al., 2023; Qian & Clark, 2016). Gamification focuses on applying game elements and mechanics to the development of learning processes in a nongame context (Deterding et al., 2011). In addition, the emergence of game-based approaches in non-game contexts has led to the concept of serious games, which involves the use of digital games for educational purposes using full-fledged games (Gurbuz & Celik, 2022; Krath et al., 2021). The distinction lies in the fact that serious learning applies to only digital games, whereas gamification and game-based learning involves the methodologies employed (Roedavan et al., 2021).

Franzwa et al. (2014) summarized three game mechanisms for designing gamification platforms that are fun, reduce frustration, and maintain a balance between enjoyment and learning: feedback in the form of rewards, instructional information that is aligned with the game’s environment and story, and the delivery of instructions in short cycles. Based on these three concepts, our study aims to investigate the fit design of implementing reward mechanisms, visual programming learning environments, and flexible learning environments. We used the theory of task-technology fit (TTF) to evaluate design fit, as TTF has been extensively used to evaluate how technology can enhance performance and match tasks to technology characteristics. Thus, the purpose of this study is to explore how applied game mechanics are employed and how performance is generated through gamification.

Among various forms of gamification, programming has become a topic of worldwide discourse due to the emergence of the digital economy, which has made it a must for professional success (Melro et al., 2023). In the context of programming instruction, gamification platforms are designed in various ways (Kafai & Burke, 2016). For instance, Scratch uses a block-based environment (Topalli & Cagiltay, 2018), Algotaurus uses a mini-language microworld programming environment (Krajcsi et al., 2021), Programmer Adventure Land uses an adventure-based programming environment (Chang et al., 2020), and many other computer programs designed to promote gamification provide various learning environment features.

We utilize samples from HackerRank and Grasshopper, two widely used and easily accessible, free programming learning platforms. Both platforms utilize three distinct game mechanisms with distinct strategies. Table 1 illustrates the game mechanisms utilized by HackerRank and Grasshopper, as well as the strategies utilized by both platforms. Based on three game mechanisms, reward mechanisms are designed to balance fun and learning; the visual programming environment determines the suitability of gamification for a game’s environment and learning context; and flexible learning environments ensure that students are not frustrated by the number of instructions provided.

Table 1.

Differences between HackerRank and Grasshopper.

	HackerRank	Grasshopper
Reward mechanism	Extrinsic rewards (badge, leaderboard)	Intrinsic rewards (progress track/lock-unlock, animation level completion)
Visual programming environment	Text-based	Puzzle-based
Flexible learning environment	Daily challenge	Continuously

Nevertheless, the purpose of this study is not to determine which of the evaluated platforms is more advanced but rather to investigate the relationship between the application of game mechanisms with various strategies and students’ perceptions of the TTF of gamification platforms. Hence, the following research questions are addressed:

RQ1.

Does the student’s perspective of the game mechanism influence the student’s perspective on TTF?

RQ2.

Does the student’s perspective on TTF have a substantial impact on the desired learning outcomes?

Theoretical Background

Gamification Design Research

A systematic review by Tay et al. (2022) confirmed that research on gamification design is still popular. From 30 studies in the last ten years, they concluded that three characteristics are mainly observed in research: teaching strategies, game design mechanisms, and research findings that could influence the implementation of gamification learning platforms. Meanwhile, of the three characteristics, research related to the design of gamification for computer programming tends to involve observations of how game design mechanisms, i.e., the application of game design, influence the outcomes of gamification, as summarized in Table 2.

Table 2.

Previous Studies On Design and Outcome Evaluation.

Focus of Study	Measured Variable	Students’ Outcomes Evaluation	Reference
Factor influencing the use of gamification learning platform	ease of use; usefulness; enjoyment; competence; motivation	actual use	Zhang et al. (2022)
Balancing learning and enjoyment in serious games	self-efficacy; experiential and instrumental attitude	continuance intention	Rosenthal and Ratan (2022)
Investigating the perceived player experience and short-term learning of the text-based java programming serious game	focused attention; relevance; confidence; satisfaction; usability; social interaction; challenge; and fun	perceived learning outcomes	Tsiotras and Xinogalos (2021)
Learning programming by creating games through the use of structured activities in secondary education	students’ knowledge, skills and performance; students’ attitudes; genders; programming knowledge; suitability from teacher perspective	pre and post-test	Seralidou and Douligeris (2021)
Influence of problem-based learning games on effective computer programming learning in higher education	enjoyment; motivation; interface design	satisfaction; pre and post-test	Chang et al. (2020)
Investigating the effect of implementing game elements on motivation, engagement and learning outcomes	digital game elements (aesthetics, narrative, game mechanics)	engagement emotional cognitive; intrinsic motivation; user disposition; learning outcomes	(Alexiou & Schippers, 2018)
Influence of several game element on behavioral outcomes	impact of implement basic game element	behavioral outcomes	(Puritat, 2019)

As presented in Table 2, From the previous studies, the aims of those previous studies are balancing enjoyment and learning and/or optimize outcomes. Nevertheless, the results of the investigation efforts into the design of game mechanisms for gamification remain inconclusive. Consequently, this study aims to explore the design fit of game mechanisms by utilizing the TTF theory.

Task–Technology Fit (TTF) Theory

Goodhue and Thompson (1995) introduced the TTF model to evaluate the suitability of tasks and technology characteristics as indicators of the success of information systems. According to TTF theory, the success of an information system depends on whether the system’s performance meets the user’s task requirements. Therefore, task performance will improve if the utilization of information systems can assist users in achieving task requirements. In the education context, TTF has been used to evaluate information systems such as massive open online courses (MOOCs) (Khan et al., 2018; Kim & Song, 2022; Shanshan & Wenfei, 2022; Wu & Chen, 2017), online learning (Isaac et al., 2019; Lin et al., 2021; Zhu et al., 2022), learning management systems (Navarro et al., 2021), and digital textbook services (Omotayo & Haliru, 2020; Rai & Selnes, 2019). Research confirms that when the task fits the technology, it improves learner performance. As demonstrated by the studies in Table 3, TTF has not been utilized to assess the fit of game mechanic design. Accordingly, this study uses TTF theory to measure the influence of an appropriate game mechanism design on learning tasks and learning outcomes.

Table 3.

Previous Studies Using TTF Theory for Evaluating Gamification.

Focus of Study	Variables Affect TTF	Variable Affected by TTF	Reference
Factors influencing students' adoption intention of brain–computer interfaces in a game-learning context	game characteristics of brain-computer interface; technology characteristics of brain-computer interface	behavioral intention	Wang et al. (2022)
Investigating students' perceptions of usefulness and ease	—	platform usage; perceived usefulness; perceived ease of use	Idowu et al. (2021)
Investigating students' perceptions of usefulness and ease	—	perceived usefulness; perceived ease of use	Vanduhe et al. (2020)
Factors influencing online game-based learning effectiveness	game quality, information quality	knowledge improvement, usage, perceived usefulness	Mosiane and Brown (2020)

Cognitive Load Theory

Cognitive load is connected to working memory’s capacity to process information (Sweller, 1988; 2010). As a theory of instructional design, working memory limitations should be considered when designing learning instruction (Sweller, 1988; Van Merrienboer & Sweller, 2005). It is believed that cognitive load is a crucial consideration in designing instructional learning, as a lack of consideration for the limitations of working memory could reduce the effectiveness of learning (Hanham et al., 2017). This theory has been widely adopted and refined as an explanation of how the brain processes information.

According to the literature, the subjective questionnaire is the most often employed method among the numerous approaches (Ayres & Sweller, 2005; Korbach et al., 2017; Leppink et al., 2013). In previous studies, cognitive load has been measured in various ways due to its evolution. The three most common dimensions consist of extraneous cognitive load, which is caused by ineffective instructional feature designs; intrinsic cognitive load, which depends on the complexity and characteristics of learning materials; and germane cognitive load, which is caused by instructional feature designs that promote learning (Chandler & Sweller, 1991; Sweller, 2010). In addition, Paas and Van Merriënboer (1994) introduce a causal dimension that reflects the interactions between the individual, tasks, and performance. This causal dimension describes the measurable aspects of mental load, mental effort, and performance. Based on this framework, mental load reflects task demands (a task-based dimension), and mental effort reflects cognitive resources allocated to fulfill task demands (a learner-based dimension). However, cognitive load can be generally split into two groups: learning-relevant (productive) cognitive load processes and learning-irrelevant (unproductive) cognitive load processes (Kalyuga & Singh, 2016). Productive cognitive load comprises all cognitive processes required to fulfill learning objectives, such as providing instructions as needed. Meanwhile, unproductive cognitive load process occurs when the design instructions distract students during the learning process.

Krell (2017) examined instruments for measuring mental load and mental effort based on complexity levels. According to this research, mental load and mental effort are positively correlated and enhance student performance. In addition, Wu (2018) and Yang et al. (2023) demonstrated that gamification can facilitate students' comprehension of the material presented on learning platforms and reduce their cognitive load by enhancing playful interactions and increasing engagement. Therefore, based on the findings, this study aims to examine the effect of TTF on cognitive load based on mental load, mental effort, and learning performance.

Theoretical Model and Hypothesis Development

Reward Mechanism

Reward mechanisms have been scientifically shown to reinforce learning outcomes (Howard-Jones & Jay, 2016; Pierce et al., 2003). The use of rewards has been shown to increase students' motivation to continue challenging tasks and produce positive feedback (Nadolny et al., 2020). In reinforcement learning problems, cumulative rewards affect the learning outcome (Silver et al., 2021). Reward mechanisms can be divided into two categories: intrinsic and extrinsic rewards (Singh et al., 2010). Intrinsic rewards are rewards that stimulate users’ intrinsic motivation to continue completing challenges, such as the lock and unlock challenge, until successful completion. On the other hand, extrinsic rewards are rewards that offer explicitly beneficial outcomes, such as badges and leaderboards. Singh et al. (2010) indicate that both extrinsic and intrinsic rewards have a close relationship with motivation and the improvement of fit in the learning environment. The achievement of optimal fit between rewards and individuals results in positive performance and behavioral outcomes (Drugan, 2019; Grunitzki et al., 2017; Sequeira et al., 2014). However, the relationship between the implementation of rewards and the achievement of fit in TTF remains unknown. However, it should be emphasized that using only these two types of rewards to predict student learning performance is insufficient. Therefore, the purpose of this study was to determine whether reward mechanisms are closely related to students’ perspectives on the TTF when utilizing gamification platforms. Thus, the following hypothesis is proposed:

H1:

The student’s perspective of the reward mechanism is positively related to the student’s perspective on the TTF of gamification.

Visual Programming Environment

Gamification commonly implements a visual programming environment, especially for computer programming. Previous studies have examined the effect of applying visual learning environments, such as block-based learning, text-based learning (Broll et al., 2018; Weintrop & Wilensky, 2019), and puzzle-based learning (Hsu & Wang, 2018). The findings demonstrate that a visual programming environment increases the enjoyment and effectiveness of learning. Hence, based on previous findings, this study proposes the following hypothesis:

H2:

The student’s perspective of the visual programming environment is positively related to the student’s perspective on the TTF of gamification.

Flexible Learning Environment

Lock-and-unlock features in gamification are commonly used as features that provide a challenge to learners, also known as a flexible learning environment. A flexible learning environment involves providing technological support for the learning process by giving freedom of choice about where, when, and how learning occurs (Jiang et al., 2021). Moreover, previous studies have specifically discussed flexible learning and indicated that it positively affects student satisfaction and performance (Li & Wong, 2018; Sun et al., 2008). Accordingly, the following hypothesis is proposed:

H3:

The student’s perspective of the flexible learning environment is positively related to the student’s perspective on the TTF of gamification.

Task–Technology Fit and Learning Outcomes

The significance of TTF in gamification design must be demonstrated by analyzing the connection between obtaining fit in gamification design and learning outcomes. In the context of gamification for programming learning, interface features of programming tools such as coding screens, drag-and-drop options, and button colors may reduce the efficiency of the learning process (Asai et al., 2019; Çakiroğlu et al., 2018; Tsai, 2019). Therefore, by analyzing the TTF of how game mechanisms are employed, it is essential to ensure that game mechanisms can reduce cognitive load and enhance learning performance.

This study uses cognitive load and performance as indications of the effectiveness of gamification design. This approach is derived from a previous study conducted by Krell (2017), which validates the correlation between mental load and mental effort as well as how these variables can be used to predict performance. In addition, Unal and Topu (2021) confirmed that when students perceive a decrease in cognitive load, their learning performance improves. As a result, this study develops the following hypothesis:

H4:

The student’s perspective of TTF in gamification is positively related to the student’s learning performance.

H5:

The student’s perspective of TTF in gamification is negatively related to the student’s cognitive load.

H6:

Reducing the cognitive load of students enhances their learning performance.

Additionally, Vilhunen et al. (2022) suggested using the pretest score as a control variable that influences the posttest score, as the measure of learning performance, can prevent spurious affect and misspecification. Therefore, this study used prior knowledge as a control variable for learning performance in the proposed research model.

Figure 1 illustrates the comprehensive framework of the hypotheses that we aim to examine. The theoretical model was based on the recent literature on gamification design and TTF theory. Students' perceptions of the reward mechanism, visual programming environment, and flexible learning environment were used to assess the TTF of the design resulting from the implementation of the game mechanism. Furthermore, the effect of TTF achievement on gamification design is measured by learning outcomes based on cognitive load and learning performance.

Figure 1.

Game mechanism, task-technology fit, and learning outcomes: a theoretical model.

Material and Methods

This study proposes a new model to investigate the significance of achieving TTF in gamification to optimize benefits by demonstrating their influence on learning outcomes. We studied HackerRank and Grasshopper, two popular open-access learning platforms for programming. Both learning platforms apply the same game mechanism, such as a reward mechanism, visual programming environment, and flexible learning environment with their distinctive features. HackerRank applies a badge and leaderboard for the reward mechanism, while Grasshopper shows a progress track and an animation indicating that the student has completed the task successfully. For the visual programming environment, HackerRank utilizes a conventional text-based approach. According to a study by Weintrop and Wilensky (2019), a text-based approach makes it easier for students to transfer their skills to a professional programming environment. Meanwhile, Grasshopper provides a block-based visual learning environment, which is believed to make programming easier to learn (Bak et al., 2020; Cakiroglu et al., 2021). Another difference is a flexible learning environment. HackerRank provides one challenge for each daily topic, in which learners are free to choose the challenge they want to complete first. On the other hand, Grasshopper requires students to accomplish one challenge before advancing to the next, with various simple challenges. From the variation in differences provided by the two platforms, we aim to reveal whether these differences have a significant effect on the TTF of game mechanisms and learning outcomes.

Our experiment was conducted in four phases. First, students completed a pretest to determine their prior knowledge condition score. Second, we randomly assigned participants to one of two gamification platforms. When utilizing the gamification platform, they were asked to accomplish four fundamental challenges within a week: arithmetic operators, conditional statements (if-else), loops, and arrays. In the third phase, participants were asked to complete a posttest to assess their acquired knowledge as their learning performance. Participants then completed a questionnaire to determine the students’ perspectives on the gamification platform used.

Table 4 presents the measurement items from the previous study that were adjusted to increase the reliability and validity of the measurements. Except for learning performance, each item was evaluated using a five-point Likert scale (1 = strongly disagree, 5 = strongly agree). Learning performance was assessed by conducting posttests designed by the researchers based on the tasks and learning objectives as well as a pretest as the control variable. To gauge students' prior knowledge, a pretest featuring ten questions on the fundamentals of JavaScript, such as arithmetic operators, conditional statements (if-else), loops, and arrays, was included. Meanwhile, the posttest consisted of ten questions to assess the learning outcomes after using the gamification platform. The pretest and posttest were multiple-choice with a five-minute time limit. We evaluated the internal consistency of the pretest and posttest using Cronbach’s alpha coefficient, where the value is 0.70 and a standard above 0.6 is considered acceptable (Churchill Jr, 1979; Lee & Hallak, 2018; Rahimnia & Hassanzadeh, 2013).

Table 4.

Measurement Items.

Construct	Indicator	Instrument	References
Reward mechanisms (RM)	RM1	The game gave me a lot of rewards for getting things right	McKernan et al. (2015)
	RM2	I received praise and recognition from the game for doing the right thing
	RM3	Getting things right in the game felt rewarding
Visual programming environment (VP)	VP1	The visualization of HackerRank/Grasshopper helps me understand the programming concepts	Moons and De Backer (2013)
	VP2	The visualization of HackerRank/Grasshopper helps me understand the exercises
	VP3	The visualization of HackerRank/Grasshopper is easy to understand
	VP4	The visualization of HackerRank/Grasshopper makes the lessons and the exercises more fun
	VP5	The visualization of HackerRank/Grasshopper is definite added value to the programming course
	VP6	I Find it time-consuming to use the visualization during the exercises
Flexible learning environment (FL)	FL1	Gives me greater control over my work to learn using HackerRank/Grasshopper	Drennan et al. (2005)
	FL2	Saves me time to learn using HackerRank/Grasshopper
	FL3	HackerRank/Grasshopper improves the quality of my programming skills
	FL4	HackerRank/Grasshopper enhances my effectiveness as a learner
Task-Technology Fit (TTF)	TTF1	HackerRank/Grasshopper fits well with the way I like to learn and study	Isaac et al. (2019); McGill and Hobbs (2008)
	TTF2	HackerRank/Grasshopper is compatible with all aspects of my learning
	TTF3	HackerRank/Grasshopper is suitable for helping me improve my programming skills
	TTF4	HackerRank/Grasshopper is necessary to improve my programming skills
	TTF5	HackerRank/Grasshopper is easy to use
	TTF6	HackerRank/Grasshopper is user friendly
	TTF7	It is easy for me to become more skillful at using HackerRank/Grasshopper
	TTF8	The output from HackerRank/Grasshopper is presented in a useful format for me
	TTF9	HackerRank/Grasshopper is easy to learn
Mental load (ML)-Cognitive load	ML1	The learning content in HackerRank/Grasshopper was difficult for me	Hwang et al. (2013)
	ML2	I had to put a lot of effort into answering the questions in HackerRank/Grasshopper
	ML3	It was troublesome for me to answer the questions in HackerRank/Grasshopper
	ML4	I Felt frustrated answering the questions in HackerRank/Grasshopper
	ML5	I Did not have enough time to answer the questions in HackerRank/Grasshopper
Mental effort (ME)-Cognitive load	ME1	During the learning activity, the way of instruction or learning content presentation caused me a lot of mental effort	Hwang et al. (2013)
	ME2	I need to put lots of effort into completing the learning tasks or achieving the learning objectives in HackerRank/Grasshopper
	ME3	The instructional way in HackerRank/Grasshopper was difficult to follow and understand

Preliminary Analysis

Data were collected from the informatics engineering departments of two Indonesian universities, where all students were enrolled in an introductory computer programming course. A total of 266 students who chose to participate were randomly divided into two groups. One group was assigned to HackerRank, while the other was assigned to Grasshopper. Of the participants, 78.20% were male and 21.80% were female. The majority of participants were 18–22 years old (95.49%), with 2.26% under 18 years old and 2.26% over the age of 22.

We evaluated the data using an independent t test to determine the difference in the students' perceptions between the two groups, as shown in Table 5. The results demonstrate that the distribution of prior knowledge between the two randomly divided groups is equal. In addition, we compared the three game mechanisms employed by HackerRank and Grasshopper from the learner’s perspective. Accordingly, there are significant differences between the three game mechanisms on both gamification platforms, including the students’ perspectives on TTF and cognitive load.

Table 5.

Independent t test Result.

	Group	Mean	SD	t value	p value
Prior knowledge (PK)	HackerRank	69.30	21.56	0.48	0.63
	Grasshopper	68.12	18.50
Learning performance (LP)	HackerRank	82.97	17.81	0.18	0.86
	Grasshopper	82.61	15.25
Reward Mechanism (RM)	HackerRank	3.70	0.84	4.23	0.00*
	Grasshopper	4.10	0.73
Visual programming environment (VP)	HackerRank	3.70	0.89	7.60	0.00*
	Grasshopper	4.43	0.67
Flexible learning environment (FL)	HackerRank	3.57	0.90	4.36	0.00*
	Grasshopper	4.02	0.77
Task-technology fit (TTF)	HackerRank	3.52	0.90	7.19	0.00*
	Grasshopper	4.26	0.76
Mental load (ML)	HackerRank	2.93	0.80	8.37	0.00*
	Grasshopper	2.11	0.80
Mental effort (ME)	HackerRank	2.94	0.90	9.51	0.00*
	Grasshopper	1.91	0.86

*p < 0.05, significant.

To test our hypotheses, we employed the technique of component-based structural equation modeling, namely the partial least squares (PLS). We employed three primary measures to test the convergent validity of the model including factor loading, composite reliability, and average extracted variance (AVE). As shown in Table 6, all factor loadings were greater than 0.7 (Teo, 2009). Items with a value less than 0.7 were removed to meet the standard. For consistency, we used composite reliability with an acceptable value greater than 0.7 (Nunnally, 1967; Rahimnia & Hassanzadeh, 2013). Moreover, for the discriminant validity, the AVE value is above 0.5, and AVE’s square roots are greater than any other construct (Fornell & Larcker, 1981). We also evaluated the Cronbach’s alpha of our model. In accordance with Churchill Jr (1979), Rahimnia and Hassanzadeh (2013) and Lee and Hallak (2018), a Cronbach’s alpha value higher than 0.60 indicates acceptable validity. Thus, based on the standard measurement model, Table 5 shows that this construct has convergent validity and meets the standard. Table 7 demonstrates that the AVE is higher than any other correlation, which can be used to validate the discriminant validity.

Table 6.

Measurement Model.

	Indicator	Factor Loading	Cronbach’s Alpha	Average Variance Extracted (AVE)	Composite Reliability (CR)
Reward mechanism	RM1	0.83	0.77	0.68	0.86
	RM2	0.76
	RM3	0.88
Visual programming environment	VP1	0.87	0.86	0.78	0.91
	VP2	0.90
	VP3	0.88
Flexible learning environment	FL2	0.87	0.84	0.76	0.90
	FL3	0.86
	FL4	0.88
Task-technology fit	TTF1	0.83	0.85	0.76	0.91
	TTF5	0.90
	TTF9	0.88
Mental load	ML1	0.76	0.67	0.60	0.82
	ML3	0.72
	ML5	0.84
Mental effort	ME1	0.92	0.83	0.85	0.83
Mental effort	ME3	0.92

Removed indicators below 0.7: VP4, VP5, VP6; FL1; TTF2, TTF3, TTF4, TTF6, TTF7, TTF8, ML2, ML4, ME2.

Table 7.

Discriminant Validity.

	RM	VP	FL	TTF	ML	ME	PK	LP
Reward mechanism	0.82
Visual programming environment	0.71	0.88
Flexible learning environment	0.78	0.77	0.87
Task-technology fit	0.66	0.82	0.77	0.87
Mental load	−0.32	−0.48	−0.35	−0.49	0.78
Mental effort	−0.24	−0.43	−0.22	−0.42	0.75	0.92
Prior knowledge	0.11	0.10	0.09	0.13	0.04	0.08	1
Learning performance	0.07	0.12	0.04	0.07	−0.06	−0.04	0.53	1

Results of the Analysis of the Structural Model

The outcomes of the hypothesis analysis are presented in Figure 2. Excluding the reward mechanism, this study indicates that game mechanisms, such as the visual programming environment and the flexible learning environment, positively affect students' perceptions of TTF in gamification. However, according to the results of this study, the reward mechanism implemented in both gamification platforms has no significant impact on students' perceptions of the TTF. This indicates that while Hypothesis 1 is rejected, Hypotheses 2 and 3 are supported. Moreover, our research revealed that TTF negatively affects cognitive load but does not directly affect learning performance. This finding highlights the impact of achieving TTF through the gamification platform on learning outcomes. Meanwhile, game mechanisms and instructional content reduce students' cognitive load, which improves learning performance. Thus, Hypothesis 4 is rejected, as there is no significant direct affect between the TTF and learning performance. However, Hypotheses 5 and 6 are supported, as the findings demonstrate that TTF negatively affects cognitive load and that cognitive load is negatively correlated with learning performance.

Figure 2.

Hypothesis testing results.

According to the coefficient of determination findings, the R square of the TTF is 0.72, indicating that ensuring the TTF of implementing game mechanisms is essential, as it could explain 72% of the variance in the TTF of gamification. Although the implementation of the reward mechanism’s (β = 0.01, t = 0.18) effect on TTF is insufficiently significant, the total effects for the visual programming environment (β = 0.56, t = 10.18) and the flexible learning environment (β = 0.33, t = 4.53) indicate that both game mechanisms are relevant. The TTF effect (β = −0.49, t = 8.75) explained 24% of the variance in cognitive load, whereas the combination of the TTF effect (β = −0.07, t = 1.01) and the cognitive load effect (β = −0.12, t = 2.01) explained 30% of learning performance.

Discussion

The findings of this study reveal that reward mechanisms implemented in both gamification platforms do not substantially affect students’ perspectives toward TTF. Previous studies have declared that reward mechanisms enhance student behavior, motivation, and engagement (Chen et al., 2015; Hanus & Fox, 2015). However, Hanus and Fox (2015) found that the availability of reward mechanisms in gamification affects student motivation, which may be split into intrinsic and extrinsic. Intrinsic motivation arises when students complete assignments according to their own desires, whereas extrinsic motivation occurs when students perform tasks in exchange for rewards. In addition, McKernan et al. (2015) found that the number of rewards was relevant to how positively students rated their overall gaming experience. Nevertheless, there was no correlation between the number of rewards and learning outcomes. From these two studies, the following recommendations have been suggested: (a) the presence of a reward system can motivate students to complete learning tasks, but (b) extrinsic motivation can result in students completing tasks without acquiring new knowledge. Thus, based on the two gamification platform assessment results, the number of rewards delivered had no significant impact on learning outcomes or student perceptions of TTF. However, eliminating the reward mechanism can reduce students' motivation to learn through gamification (Lepper et al., 1973). Consequently, this study suggests that the reward mechanism is still a game mechanism that must be integrated into gamification.

The second finding relates to the visual programming environment, which positively influences students' perspectives on the TTF of the gamification platform. This result is consistent with the previous findings Unal and Topu (2021) and Palazzo et al. (2020). Both of these studies found that the effect of visual learning facilitates realizing learning programming objectives. In the context of learning programming, extensive information and complex rules can lower students' success in acquiring knowledge (Kelleher & Pausch, 2005; Yukselturk & Altiok, 2017). Visual learning environments are vital for simplifying the concept of learning and facilitating students' knowledge acquisition. Thus, the previous research supports our second hypothesis, which highlights that the visual programming environment is an essential component and that the visual programming environment positively affects students' perspectives on the TTF of gamification.

To establish successful gamification, it is crucial to provide learners with an environment that is flexible to their needs, as demonstrated by the third finding. Previous research has demonstrated that the implementation of a flexible learning environment can have a favorable effect on learning outcomes (Mavridis et al., 2017). Students who engage in flexible learning are better able to adapt to new challenges. The extent to which students perceive that they are participating in a flexible learning environment is one factor that can help them prepare for the upcoming task. Thus, the third hypothesis is supported.

Furthermore, by demonstrating the effect of TTF on learning outcomes, the results of our study highlight the importance of achieving a fit between gaming and learning tasks. The results of the insignificant influence of TTF on learning outcomes are consistent with the findings of Lowyck et al. (2004) and Wu et al. (2013) that environmental elements on learning platforms, such as instructional strategies, do not have a direct effect on learning outcomes; rather, they are mediated by the cognitive processes and activities of the learners. This explains why the TTF associated with the applied game mechanism does not have a significant direct effect on learning performance but has an indirect effect on it through the mediation of cognitive load. In addition, consistent with the results of Blikstein et al. (2014) and Song et al. (2021), who showed that there was no significant correlation between students' success and failure on a number of tasks, students' success on these tasks was not always indicative of their performance. Moreover, there was no significant difference between the two groups in terms of learning performance. In line with a previous study, there were no significant differences in learning performance between groups across learning environments (Tugtekin & Odabasi, 2022; Unal & Topu, 2021). This indicates that both gamification platforms can provide equal performance. The results of previous studies are relevant to our findings, which suggest that a well-designed learning environment can enhance student performance by reducing cognitive load. As a result, the gamification platform meets its objective of making programming education more accessible to students by simplifying the complex concepts of programming. Therefore, Hypothesis 4 is rejected, and Hypothesis 5 is accepted.

Meanwhile, it has been demonstrated that reducing cognitive load, particularly extraneous cognitive load, has a significant impact on improving student learning performance. Several previous studies have demonstrated that reducing cognitive load using a well-designed learning environment can improve student memory and learning performance (Krell, 2017; Park et al., 2011; Scheiter et al., 2009; Wang et al., 2020). Thus, Hypothesis 6 in this study is significantly supported and consistent with the previous findings.

Conclusions and Implications

This study aims to investigate the significance of the application of game mechanisms in online learning gamification platforms that implement flexible learning processes for supporting learning in computer programming skills. The results indicate that game mechanisms play a significant role in enhancing TTF in such online gamification platforms. Interestingly, it is found that the reward mechanism implemented by both gamification platforms investigated does not have a significant influence on the TTF. This indicates that while rewards may be one of the essential elements of gamification, they may not be the most influencing factor in determining the overall design fit of online gamification platforms that are developed for supporting learning. To conclude, the findings of this study indicate that when students’ cognitive loads are reduced as a result of the appropriate design of online gamification learning platforms, their learning performance is likely to be enhanced.

Theoretical Implications

This study has provided a theoretical foundation for future research into the implementation of game mechanisms on gamification platforms. Researchers can improve and enhance gamification experiences by knowing how game mechanisms and learning platform goals interact. The results of this study can also be used to guide the design of future gamification platform activities, providing educators with the materials they need to develop effective learning environments that optimize the benefits of gamification. Using TTF theory, this study suggests that gamification platform design should prioritize achieving a design fit between game mechanisms and learning environments. Although TTF has been frequently used in previous studies, these findings contribute to demonstrating the significance of measuring TTF in gamification based on the applicable game mechanism.

The findings demonstrate that reward mechanisms have no substantial impact on the TTF of gamification platforms. This has implications for developers, who should consider focusing on the design mechanisms of visual learning environments and flexible learning environments when attempting to create a fit design of gamification platforms, particularly for computer programming learning. Additionally, this finding has implications for educators, who may need to adjust their teaching strategies to account for the increased importance of visual and flexible learning environments in gamification platform design for computer programming learning. It also suggests that educators may need to prioritize teaching students how to use these mechanisms effectively to ensure that their gamification platform designs fit the tasks they present. Ultimately, this study contributes to the development of a more comprehensive understanding of gamification platform design and its implications for educational practice.

Moreover, although several studies have revealed that gamification is effective since it results in higher learning scores (Mavridis & Tsiatsos, 2017; Yang et al., 2018), the results of learning performance in this study cannot be predicted unless the effect of stimuli on cognitive load is identified. Therefore, to determine the impact of achieving fit on performance, the effect on cognitive load should be determined beforehand to predict the results of learning performance. This suggests that the design of gamification, along with other aspects such as cognitive load, must be considered when evaluating its effectiveness. This implies that educators should be mindful of the design of their learning platform activities and the specific ways in which students experience the learning process to ensure that their students are best able to benefit from learning platforms. Furthermore, this research indicates that the evaluation of gamification should go beyond simple measures of learning performance and include a more holistic approach to assessing student experience and performance.

Practical Implications

The practical implications of this study are in improving the quality of gamification by revealing the influencing factors and accurate variables for measuring the TTF of gamification. The findings from this study highlight the importance of achieving a good TTF between game mechanisms and the learning environment when designing a gamification platform. As Qian and Clark (2016) concluded, effective learning is based on game design. Therefore, to create an effective gamification that balances gameplay and learning, academics and practitioners should consider the visual programming environment and flexible learning environment.

In an educational context, the employment of animation can have a powerful and positive impact on student learning. By introducing animation into the design of gamification, educators could create a more engaging and interactive learning environment for students while also accomplishing the intended purpose of gamification. This could ultimately lead to increased active learning and higher learning outcomes. In addition, level stages that must be completed before moving on to a more difficult level generate a sense of accomplishment and advancement that will encourage them to continue advancing and broaden their perspective of the gamification that achieves fit.

Although this study demonstrated that the reward mechanism does not significantly affect the TTF of gamification design, the absence of a reward mechanism has a negative impact on student behavior (Lepper et al., 1973). In addition, our findings revealed that the different approaches to reward mechanisms do not interfere with students’ learning and do not affect the balance of gameplay design of the learning environment in gamification. The results of this study suggest that when designing gamification, various reward mechanisms should not be prioritized over the perspectives of the students. Moreover, to accurately measure the quality of gamification, educators should focus on assessing students' cognitive performance rather than using a score on learning performance. This approach can provide a more meaningful and reliable assessment of the efficacy of gamification. Furthermore, by focusing on cognitive performance, educators can gain insight into the effectiveness of gamification approaches and make more informed decisions about how to optimize gamification for their students.

Limitations and Future Work

This research was limited to an exploration of computer learning through gamification for the JavaScript programming language. Furthermore, we rely on two gamification platform environments. Both platform environments use the three primary game mechanisms investigated in our study. Future studies should be conducted by extending our research model to further investigate the effect of task and technology characteristics on a good fit. Thus, future research can build more mechanisms to be applied to gamification and assess how these applications alter the design fit to give students a rewarding and successful learning experience.

In addition, according to the study, further research is needed to identify relevant constructs and indicators that can be used to determine performance. Predicting performance as a learning outcome demands precise constraints and indicators. These conditions support the idea that obtaining a good fit does not directly correlate with improving learner performance scores but correlates with cognitive load. Furthermore, Qian and Clark (2016) indicated that games that provide content with a simple design, such as quizzes or practical approaches, do not interest learners. Therefore, additional research on the relationship between good fit and learner engagement is required since increasing engagement is impossible if there is no interest in gamification.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Ministry of Science and Technology, Taiwan (MOST 109-2511-H-006-006-MY3).

ORCID iD

Wei-Tsong Wang

Data Availability Statement

The author provides the data set used upon reasonable request.

Author Biographies

Wei-Tsong Wang received his PhD in information science from the State University of New York at Albany, USA. His current research interests include e-learning, game-based learning, the behaviours of e-commerce consumers, user acceptance of information technology, and knowledge management. His works have appeared/are to be appeared in journals such as Computers in Human Behavior, Computers and Education, Journal of Educational Technology and Society, Journal of Educational Computing Research, and Education and Information Technologies, among others.

Mega Kartika Sari is currently a PhD student in information management at National Cheng Kung University, Taiwan, and is doing her thesis on the gamification area in order to improve learning technology as a support system. Her main research interests are game-based learning, e-learning support systems, and data mining.

References

Alexiou

Schippers

M. C.

(2018). Digital game elements, user experience and learning: A conceptual framework. Education and Information Technologies, 23(6), 2545–2567. https://doi.org/10.1007/s10639-018-9730-6

Alsawaier

R. S.

(2018). The effect of gamification on motivation and engagement. The International Journal of Information and Learning Technology, 35(1), 55–79. https://doi.org/10.1108/IJILT-02-2017-0009

Asai

Phuong

D. T. D.

Harada

Shimakawa

(2019). Predicting cognitive load in acquisition of programming abilities. International Journal of Electrical & Computer Engineering (2088–8708), 9(4), 3262–3271. https://doi.org/10.11591/ijece.v9i4.pp3262-3271

Ayres

Sweller

(2005). The split-attention principle in multimedia learning. The Cambridge Handbook of Multimedia Learning, 2, 135–146.

Bak

Chang

B. M.

Choi

(2020). Smart block: A visual block language and its programming environment for IoT. Journal of Computer Languages, 60, 19. https://doi.org/10.1016/j.cola.2020.100999

Blikstein

Worsley

Piech

Sahami

Cooper

Koller

(2014). Programming pluralism: Using learning analytics to detect patterns in the learning of computer programming. Journal of the Learning Sciences, 23(4), 561–599. https://doi.org/10.1080/10508406.2014.954750

Bozkurt

Durak

(2018). A systematic review of gamification research: In pursuit of homo ludens. International Journal of Game-Based Learning, 8(3), 15–33. https://doi.org/10.4018/IJGBL.2018070102

Broll

Lédeczi

Á.

Zare

D. N.

Sallai

Völgyesi

Maróti

Brown

Vanags

(2018). A visual programming environment for introducing distributed computing to secondary education. Journal of Parallel and Distributed Computing, 118(1), 189–200. https://doi.org/10.1016/j.jpdc.2018.02.021

Cakiroglu

Cevik

Koseli

Aydin

(2021). Understanding students' abstractions in block-based programming environments: A performance based evaluation. Thinking Skills and Creativity, 41, 12. https://doi.org/10.1016/j.tsc.2021.100888

10.

Çakiroğlu

Ü.

Suiçmez

S. S.

Kurtoğlu

Y. B.

Sari

Yildiz

Öztürk

(2018). Exploring perceived cognitive load in learning programming via Scratch. Research in Learning Technology, 26, 1888. https://doi.org/10.25304/rlt.v26.1888

11.

Chandler

Sweller

(1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4), 293–332. https://doi.org/10.1207/s1532690xci0804_2

12.

Chang

C.-S.

Chung

C.-H.

Chang

J. A.

(2020). Influence of problem-based learning games on effective computer programming learning in higher education. Educational Technology Research and Development, 68(5), 2615–2634. https://doi.org/10.1007/s11423-020-09784-3

13.

Chen

Burton

Mihaela

Whittinghill

(2015). Cogent: A case study of meaningful gamification in education with virtual currency. International Journal of Emerging Technologies in Learning, 10(1), 39–45. https://doi.org/10.3991/ijet.v10i1.4247

14.

Churchill

G. A.

Jr (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73. https://doi.org/10.1177/002224377901600110

15.

Dahalan

Alias

Shaharom

M. S. N.

(2023). Gamification and game based learning for vocational education and training: A systematic literature review. Education and Information Technologies. https://doi.org/10.1007/s10639-022-11548-w

16.

Deterding

Dixon

Khaled

Nacke

(2011). From game design elements to gamefulness: Defining “gamification”. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, Tampere, Finland, 2011. https://doi.org/10.1145/2181037.2181040

17.

Drennan

Kennedy

Pisarski

(2005). Factors affecting student attitudes toward flexible online learning in management education. The Journal of Educational Research, 98(6), 331–338. https://doi.org/10.3200/JOER.98.6.331-338

18.

Drugan

M. M.

(2019). Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms. Swarm and Evolutionary Computation, 44, 228–246. https://doi.org/10.1016/j.swevo.2018.03.011

19.

Fornell

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104

20.

Franzwa

Tang

Johnson

Bielefeldt

(2014). Balancing fun and learning in a serious game design. International Journal of Game-Based Learning, 4(4), 37–57. https://doi.org/10.4018/ijgbl.2014100103

21.

Goodhue

D. L.

Thompson

R. L.

(1995). Task-technology fit and individual performance. Mis Quarterly, 19(2), 213–236. https://doi.org/10.2307/249689

22.

Grunitzki

da Silva

B. C.

Bazzan

A. L.

(2017). A flexible approach for designing optimal reward functions. AAMAS.

23.

Gurbuz

S. C.

Celik

(2022). Serious games in future skills development: A systematic review of the design approaches [review]. Computer Applications in Engineering Education, 30(5), 1591–1612. https://doi.org/10.1002/cae.22557

24.

Hanham

Leahy

Sweller

(2017). Cognitive load theory, element interactivity, and the testing and reverse testing effects. Applied Cognitive Psychology, 31(3), 265–280. https://doi.org/10.1002/acp.3324

25.

Hanus

M. D.

Fox

(2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152–161. https://doi.org/10.1016/j.compedu.2014.08.019

26.

Howard-Jones

P. A.

Jay

(2016). Reward, learning and games. Current Opinion in Behavioral Sciences, 10, 65–72. https://doi.org/10.1016/j.cobeha.2016.04.015

27.

Hsu

C.-C.

Wang

T.-I.

(2018). Applying game mechanics and student-generated questions to an online puzzle-based game learning system to promote algorithmic thinking skills. Computers & Education, 121, 73–88. https://doi.org/10.1016/j.compedu.2018.02.002

28.

Hwang

G.-J.

Yang

L.-H.

Wang

S.-Y.

(2013). A concept map-embedded educational computer game for improving students' learning performance in natural science courses. Computers & Education, 69, 121–130. https://doi.org/10.1016/j.compedu.2013.07.008

29.

Idowu

Nat

Kissi

P. S.

(2021). Student perception of usefulness and ease using Kahoot, a free web-based tool in a tertiary education setting. Acta Scientiarum-Technology, 43, 47347. https://doi.org/10.4025/actascitechnol.v42i1.47347

30.

Isaac

Aldholay

Abdullah

Ramayah

(2019). Online learning usage within Yemeni higher education: The role of compatibility and task-technology fit as mediating variables in the IS success model. Computers & Education, 136, 113–129. https://doi.org/10.1016/j.compedu.2019.02.012

31.

Jiang

Cheng

Wang

Xie

A. N.

Fitzgerald

S. R.

(2021). Twelve tips for teaching medical students online under COVID-19. Medical Education Online, 26(1), 1854066. https://doi.org/10.1080/10872981.2020.1854066

32.

Kafai

Y. B.

Burke

(2016). Connected gaming: What making video games can teach us about learning and literacy. Mit Press.

33.

Kalyuga

Singh

A.-M.

(2016). Rethinking the boundaries of cognitive load theory in complex learning. Educational Psychology Review, 28(4), 831–852. https://doi.org/10.1007/s10648-015-9352-0

34.

Kelleher

Pausch

(2005). Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers. ACM Computing Surveys, 37(2), 83–137. https://doi.org/10.1145/1089733.1089734

35.

Khan

I. U.

Hameed

Islam

Sheikh

Khan

S. U.

(2018). Predicting the acceptance of MOOCs in a developing country: Application of task-technology fit model, social motivation, and self-determination theory. Telematics and Informatics, 35(4), 964–978. https://doi.org/10.1016/j.tele.2017.09.009

36.

Kim

Song

H.-D.

(2022). Examining the influence of teaching presence and task-technology fit on continuance intention to use MOOCs. The Asia-Pacific Education Researcher, 31(2), 395–408. https://doi.org/10.1007/s40299-021-00581-x

37.

Kiron

Adaji

Long

Vassileva

(2020). Engaging students in a peer-quizzing game to encourage active learning and building a student-generated question bank. Electronic Journal of E-Learning, 18(3), 235–247.

38.

Korbach

Brünken

Park

(2017). Measurement of cognitive load in multimedia learning: A comparison of different objective measures. Instructional Science, 45(4), 515–536. https://doi.org/10.1007/s11251-017-9413-5

39.

Krajcsi

Csapodi

Stettner

(2021). Algotaurus: An educational computer programming game for beginners. Interactive Learning Environments, 29(4), 634–647. https://doi.org/10.1080/10494820.2019.1593862

40.

Krath

Schürmann

von Korflesch

H. F. O.

(2021). Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning. Computers in Human Behavior, 125, 106963. https://doi.org/10.1016/j.chb.2021.106963

41.

Krell

(2017). Evaluating an instrument to measure mental load and mental effort considering different sources of validity evidence. Cogent Education, 4(1), 1280256. https://doi.org/10.1080/2331186x.2017.1280256

42.

Lee

Hallak

(2018). Investigating the moderating role of education on a structural model of restaurant performance using multi-group PLS-SEM analysis. Journal of Business Research, 88, 298–305. https://doi.org/10.1016/j.jbusres.2017.12.004

43.

Lepper

M. R.

Greene

Nisbett

R. E.

(1973). Undermining children's intrinsic interest with extrinsic reward: A test of the “overjustification” hypothesis. Journal of Personality and Social Psychology, 28(1), 129–137. https://doi.org/10.1037/h0035519

44.

Leppink

Paas

Van der Vleuten

C. P.

Van Gog

Van Merriënboer

J. J.

(2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1

45.

Lester

Skulmoski

G. J.

Fisher

D. P.

Mehrotra

Lim

Lang

Keogh

J. W. L.

(2023). Drivers and barriers to the utilisation of gamification and game-based learning in universities: A systematic review of educators' perspectives [review; early access]. British Journal of Educational Technology, 23, 13311. https://doi.org/10.1111/bjet.13311

46.

K. C.

Wong

B. Y. Y.

(2018). Revisiting the definitions and implementation of flexible learning. In Innovations in Open and Flexible Education (pp. 3–13), Springer. https://doi.org/10.1007/978-981-10-7995-5_1

47.

Lin

C.-L.

Jin

Y. Q.

Zhao

S.-W.

Y.-S.

(2021). Factors influence students’ switching behavior to online learning under COVID-19 pandemic: A push–pull–mooring model perspective. The Asia-Pacific Education Researcher, 30(3), 229–245. https://doi.org/10.1007/s40299-021-00570-0

48.

Liu

Y. C.

Wang

W.-T.

Lee

T.-L.

(2021). An integrated view of information feedback, game quality, and autonomous motivation for evaluating game-based learning effectiveness. Journal of Educational Computing Research, 59(1), 3–40. https://doi.org/10.1177/0735633120952044

49.

Lowyck

Elen

Clarebout

(2004). Instructional conceptions: Analysis from an instructional design perspective. International Journal of Educational Research, 41(6), 429–444. https://doi.org/10.1016/j.ijer.2005.08.010

50.

Mavridis

Katmada

Tsiatsos

(2017). Impact of online flexible games on students' attitude towards mathematics. Educational Technology Research and Development, 65(6), 1451–1470. https://doi.org/10.1007/s11423-017-9522-5

51.

Mavridis

Tsiatsos

(2017). Game-based assessment: Investigating the impact on test anxiety and exam performance. Journal of Computer Assisted Learning, 33(2), 137–150. https://doi.org/10.1111/jcal.12170

52.

McGill

T. J.

Hobbs

(2008). How students and instructors using a virtual learning environment perceive the fit between technology and task. Journal of Computer Assisted Learning, 24(3), 191–202. https://doi.org/10.1111/j.1365-2729.2007.00253.x

53.

McKernan

Martey

R. M.

Stromer-Galley

Kenski

Clegg

B. A.

Folkestad

J. E.

Rhodes

M. G.

Shaw

Saulnier

E. T.

Strzalkowski

(2015). We don’t need no stinkin’badges: The impact of reward features and feeling rewarded in educational games. Computers in Human Behavior, 45, 299–306. https://doi.org/10.1016/j.chb.2014.12.028

54.

Melro

Tarling

Fujita

Kleine Staarman

(2023). What else can Be learned when coding? A configurative literature review of learning opportunities through computational thinking. Journal of Educational Computing Research, 61(4), 901–924. https://doi.org/10.1177/07356331221133822

55.

Moons

De Backer

(2013). The design and pilot evaluation of an interactive learning environment for introductory programming influenced by cognitive load theory and constructivism. Computers & Education, 60(1), 368–384. https://doi.org/10.1016/j.compedu.2012.08.009

56.

Mosiane

Brown

(2020). Factors influencing online game-based learning effectiveness. Electronic Journal of Information Systems Evaluation, 23(1), 79–95. https://doi.org/10.34190/EJISE.20.23.1.006

57.

Nadolny

Valai

Cherrez

N. J.

Elrick

Lovett

Nowatzke

(2020). Examining the characteristics of game-based learning: A content analysis and design framework. Computers & Education, 156, 103936. https://doi.org/10.1016/j.compedu.2020.103936

58.

Navarro

M. M.

Prasetyo

Y. T.

Young

M. N.

Nadlifatin

Redi

A. A. N. P.

(2021). The perceived satisfaction in utilizing learning management system among engineering students during the COVID-19 pandemic: Integrating task technology fit and extended technology acceptance model. Sustainability, 13(19), 10669. https://doi.org/10.3390/su131910669

59.

Nunnally

J. C.

(1967). Psychometric theory. Mc-Graw-Hill.

60.

Omotayo

F. O.

Haliru

(2020). Perception of task-technology fit of digital library among undergraduates in selected universities in Nigeria. The Journal of Academic Librarianship, 46(1), 102097. https://doi.org/10.1016/j.acalib.2019.102097

61.

Paas

F. G.

Van Merriënboer

J. J.

(1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6(4), 351–371. https://doi.org/10.1007/BF02213420

62.

Palazzo

Spampinato

Kavasidis

Giordano

Schmidt

Shah

(2020). Decoding brain representations by multimodal learning of neural activity and visual features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11), 3833–3849. https://doi.org/10.1109/TPAMI.2020.2995909

63.

Park

Moreno

Seufert

Brünken

(2011). Does cognitive load moderate the seductive details effect? A multimedia study. Computers in Human Behavior, 27(1), 5–10. https://doi.org/10.1016/j.chb.2010.05.006

64.

Pierce

W. D.

Cameron

Banko

K. M.

(2003). Positive effects of rewards and performance standards on intrinsic motivation. The Psychological Record, 53(4), 561–578. https://doi.org/10.1007/BF03395453

65.

Puritat

(2019). Enhanced knowledge and engagement of students through the gamification concept of game elements [article]. International Journal of Engineering Pedagogy, 9(5), 41–54. https://doi.org/10.3991/ijep.v9i5.11028

66.

Qian

Clark

K. R.

(2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58. https://doi.org/10.1016/j.chb.2016.05.023

67.

Rahimnia

Hassanzadeh

J. F.

(2013). The impact of website content dimension and e-trust on e-marketing effectiveness: The case of Iranian commercial saffron corporations. Information & Management, 50(5), 240–247. https://doi.org/10.1016/j.im.2013.04.003

68.

Rai

R. S.

Selnes

(2019). Conceptualizing task-technology fit and the effect on adoption–A case study of a digital textbook service. Information & Management, 56(8), 103161. https://doi.org/10.1016/j.im.2019.04.004

69.

Roedavan

Pudjoatmodjo

Siradj

Salam

Hardianti

B. Q. D.

(2021). Serious game development model based on the game-based learning foundation [article]. Journal of Ict Research and Applications, 15(3), 291–305. https://doi.org/10.5614/itbj.ict.res.appl.2021.15.3.6

70.

Rosenthal

Ratan

R. A.

(2022). Balancing learning and enjoyment in serious games: Kerbal Space Program and the communication mediation model. Computers & Education, 182, 104480. https://doi.org/10.1016/j.compedu.2022.104480

71.

Scheiter

Gerjets

Vollmann

Catrambone

(2009). The impact of learner characteristics on information utilization strategies, cognitive load experienced, and performance in hypermedia learning. Learning and Instruction, 19(5), 387–401. https://doi.org/10.1016/j.learninstruc.2009.02.004

72.

Sequeira

Melo

F. S.

Paiva

(2014). Learning by appraising: An emotion-based approach to intrinsic reward design. Adaptive Behavior, 22(5), 330–349. https://doi.org/10.1177/1059712314543837

73.

Seralidou

Douligeris

(2021). Learning programming by creating games through the use of structured activities in secondary education in Greece. Education and Information Technologies, 26(1), 859–898. https://doi.org/10.1007/s10639-020-10255-8

74.

Shanshan

Wenfei

(2022). Understanding the impact of quality elements on MOOCs continuance intention. Education and Information Technologies, 27(8), 10949–10976. https://doi.org/10.1007/s10639-022-11063-y

75.

Silver

Singh

Precup

Sutton

R. S.

(2021). Reward is enough. Artificial Intelligence, 299, 103535. https://doi.org/10.1016/j.artint.2021.103535

76.

Singh

Lewis

R. L.

Barto

A. G.

Sorg

(2010). Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2(2), 70–82. https://doi.org/10.1109/TAMD.2010.2051031

77.

Song

Hong

E. Y.

(2021). Applying computational analysis of novice learners' computer programming patterns to reveal self-regulated learning, computational thinking, and learning performance. Computers in Human Behavior, 120, 106746. https://doi.org/10.1016/j.chb.2021.106746

78.

Sun

P.-C.

Tsai

R. J.

Finger

Chen

Y.-Y.

Yeh

(2008). What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183–1202. https://doi.org/10.1016/j.compedu.2006.11.007

79.

Sweller

(1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1016/0364-0213(88)90023-7

80.

Sweller

(2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5

81.

Tay

Goh

Y. M.

Safiena

Bound

(2022). Designing digital game-based learning for professional upskilling: A systematic literature review. Computers & Education, 184(93), 104518. https://doi.org/10.1016/j.compedu.2022.104518

82.

Teo

(2009). Modelling technology acceptance in education: A study of pre-service teachers. Computers & Education, 52(2), 302–312. https://doi.org/10.1016/j.compedu.2008.08.006

83.

Topalli

Cagiltay

N. E.

(2018). Improving programming skills in engineering education through problem-based game projects with Scratch. Computers & Education, 120, 64–74. https://doi.org/10.1016/j.compedu.2018.01.011

84.

Tsai

C.-Y.

(2019). Improving students' understanding of basic programming concepts through visual programming language: The role of self-efficacy. Computers in Human Behavior, 95, 224–232. https://doi.org/10.1016/j.chb.2018.11.038

85.

Tsiotras

Xinogalos

(2021). Investigating the perceived player experience and short-term learning of the text-based java programming serious game “rise of the java emperor. Informatics in Education, 20(1), 153–170. https://doi.org/10.15388/infedu.2021.08

86.

Tugtekin

Odabasi

H. F.

(2022). Do interactive learning environments have an effect on learning outcomes, cognitive load and metacognitive judgments? Education and Information Technologies, 27(5), 7019–7058. https://doi.org/10.1007/s10639-022-10912-0

87.

Unal

Topu

F. B.

(2021). Effects of teaching a computer programming language via hybrid interface on anxiety, cognitive load level and achievement of high school students. Education and Information Technologies, 26(5), 5291–5309. https://doi.org/10.1007/s10639-021-10536-w

88.

Vanduhe

V. Z.

Nat

Hasan

H. F.

(2020). Continuance intentions to use gamification for training in higher education: Integrating the technology acceptance model (TAM), Social motivation, and task technology fit (TTF). IEEE Access, 8, 21473–21484. https://doi.org/10.1109/ACCESS.2020.2966179

89.

Van Merrienboer

J. J.

Sweller

(2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0

90.

Vilhunen

Turkkila

Lavonen

Salmela-Aro

Juuti

(2022). Clarifying the relation between epistemic emotions and learning by using experience sampling method and pre-posttest design [original research]. Frontiers in Education, 7, 826852. https://doi.org/10.3389/feduc.2022.826852

91.

Wang

Fang

(2020). Learning performance and behavioral patterns of online collaborative learning: Impact of cognitive load and affordances of different multimedia. Computers & Education, 143, 103683. https://doi.org/10.1016/j.compedu.2019.103683

92.

Wang

Y.-M.

Wei

C.-L.

Wang

M.-W.

(2022). Factors influencing students' adoption intention of brain–computer interfaces in a game-learning context. Library Hi Tech. (ahead-of-print) https://doi.org/10.1108/LHT-12-2021-0506

93.

Weintrop

Wilensky

(2019). Transitioning from introductory block-based and text-based environments to professional programming languages in high school computer science classrooms. Computers & Education, 142, 103646. https://doi.org/10.1016/j.compedu.2019.103646

94.

Chen

(2017). Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior, 67, 221–232. https://doi.org/10.1016/j.chb.2016.10.028

95.

Lowyck

Sercu

Elen

(2013). Task Complexity, Student Perceptions of Vocabulary Learning in EFL, and Task performance. British Journal of Educational Psychology, 83(1), 160–181. https://doi.org/10.1111/j.2044-8279.2011.02061.x

96.

Y. L.

(2018). Gamification design: A comparison of four m-learning courses [article]. Innovations in Education and Teaching International, 55(4), 470–478. https://doi.org/10.1080/14703297.2016.1250662

97.

Lio

Dhaliwal

Andrei

Balakrishnan

Nagani

Samadder

(2021). Psychological interventions of virtual gamification within academic intrinsic motivation: A systematic review. Journal of Affective Disorders, 293(3), 444–465. https://doi.org/10.1016/j.jad.2021.06.070

98.

Yang

J. C.

Lin

M. Y. D.

Chen

S. Y.

(2018). Effects of anxiety levels on learning performance and gaming performance in digital game-based learning. Journal of Computer Assisted Learning, 34(3), 324–334. https://doi.org/10.1111/jcal.12245

99.

Yang

Q. F.

Lian

L. W.

Zhao

J. H.

(2023). Developing a gamified artificial intelligence educational robot to promote learning effectiveness and behavior in laboratory safety courses for undergraduate students [Article]. International Journal of Educational Technology in Higher Education, 20(1), 31. https://doi.org/10.1186/s41239-023-00391-9

100.

Z. G.

Gao

M. L.

Wang

L. F.

(2021). The effect of educational games on learning outcomes, student motivation, engagement and satisfaction. Journal of Educational Computing Research, 59(3), 522–546. https://doi.org/10.1177/0735633120969214

101.

Yukselturk

Altiok

(2017). An investigation of the effects of programming with Scratch on the preservice IT teachers’ self‐efficacy perceptions and attitudes towards computer programming. British Journal of Educational Technology, 48(3), 789–801. https://doi.org/10.1111/bjet.12453

102.

Zhang

Wong

G. K. W.

Chan

P. C. F.

(2023). Playing coding games to learn computational thinking: What motivates students to use this tool at home? Education and Information Technologies, 28(1), 193–216. https://doi.org/10.1007/s10639-022-11181-7

103.

Zhu

Geng

Disney

Pan

(2023). Changes in university students’ behavioral intention to learn online throughout the COVID-19: Insights for online teaching in the post-pandemic era. Education and Information Technologies, 28(4), 3859–3892. https://doi.org/10.1007/s10639-022-11320-0