Abstract
Background
Gamification research and practice have gained popularity, but there is a lack of experimental and field evidence regarding the effectiveness of specific design elements and their use contexts especially in developing countries.
Aim
This research aimed to measure the effect of gamifying digital learning environments and compare the effects of badges and leaderboards on the development of cognitive and achievement motivation of elementary school students.
Method
A gamified environment focused on the circulatory system and employing interactive storytelling was developed. It was tested through an in-field experiment at an elementary school in Egypt. The research sample was of 30 students in the 5th grade, divided into two equal experimental groups (badges - leaderboard).
Results
The results indicated an increase in cognitive and achievement motivation among students in both experimental groups. However, there were no significant differences in the effects of badges and leaderboards on the cognitive and achievement motivations of the participants in the two experimental groups.
Conclusions
These findings suggest that the effects of gamification are more likely due to the holistic design and novelty, rather than the specific use of a specific game element.
Introduction
Motivation, or its lack thereof, can often be a problem for learners, especially when they do not find the purpose of a learning activity to be clear or easily meaningful to them. This is especially true while presenting scientific concepts to primary school pupils as students at that age may not always understand the relevance of the material they are learning and may need additional support to connect it to real-world experiences. Traditional didactic methods are not necessarily compatible with the nature of these scientific concepts, nor the nature of this century, and many pupils tend to find it challenging to grasp these concepts or perceive their importance (Aura et al., 2021a; Botha & Herselman, 2015). The resulting lack of motivation and engagement can lead to difficulty in understanding and retaining information thereby leading to failure in attaining the desired educational goals and a poor acquisition of scientific concepts, as has been confirmed by the pilot study which was a part of this research. Previous studies have explored simplifying education content, learning exercises, matching demand levels to student skills and more, through the use of different emerging game-based methods, including games, fantasy, gamification (Aura et al., 2022; Freina and Ott, 2015; Majuri et al., 2018) serious games (Ullah et al., 2022), game-based learning (Pan et al., 2021), storification, playfulness (Järvelä et al., 2022), and many such labels .
Across game-based approaches to education, existing research has indicated that there are cultural and contextual differences in the perception and reception of game-based designs in education, including but not limited to gamification design (Hassan, 2018; Klock et al., 2020; Iten and Petko, 2016). However, it remains that the majority of current research is primarily western-centric, providing limited insights on the reception of gamification in, for example, Middle Eastern and developing countries, as well as which gamification designs / elements would resonate most with students in different cultures. Furthermore, gamification can be particularly valuable in educational contexts characterized by limited resources and classroom management challenges, as is often the case in developing countries (Aura et al., 2021a), where classrooms are significantly big, and the ratio of students to teachers is high. Gamification, compared to serious games, can be particularly easier and quicker to implement, giving it a slight advantage in contexts with limited resources, if it is indeed able to positively impact learning sustainably. In such situations, employing an engaging tool, such as gamification, can help with classroom management and with fostering learner engagement and appreciation of the classroom, potentially freeing teachers to other educational tasks. It, hence, becomes crucial that we evaluate different gamification designs that can appeal to different learners (Gasland, 2011), especially in developing countries, such as in Egypt, the context of this particular study. By doing so, we aim to contribute to the development of optimal solutions tailored to the needs and circumstances of different educational contexts.
Previous research on gamification in developing countries (Al-Hosni et al., 2023; al-Juraywi, 2019; Alfulaih, 2018; Alabbasi, 2018) has studied its use in learning and teaching generally, and its impact on acquiring scientific concepts, academic achievement, creative thinking, and motivation. These studies have shown that gamification has a positive impact on these skills and enhances learning. However, Alabbasi (2018) found that some students felt negatively effects from certain gaming elements. There remains a need for further research. Hence, this experiment is conducted to determine an effective reward for a particular game whether badges or a leaderboard. Accordingly, the aim of this study is to investigate: What are the effects of gamification on the learning and achievement motivations of primary school students in a developing, Middle Eastern country? More specifically, this research employs field experiments to investigate: What is the impact of utilizing badges vs. leaderboards on the cognitive and achievement motivation of students? To address this question, we gamified education by developing a game tailored to the school's requirements, and the aim is to evaluate which reward is better, badges or leaderboard.
Background
Gamification in Education
The ubiquity of information and communication technology (ICT), and our increasing reliance on it, is reshaping the way teaching and learning takes place. Today's learners are digital natives with distinct learning styles, attitudes, and expectations for education. The educational landscape is rapidly changing, and utilizing modern teaching approaches and techniques reinforced by ICT can facilitate active learning and new pedagogical paradigms. Gamification is one of such approaches that in recent years, has become a trend in not only education (Aura et al., 2022; Majuri et al., 2018), but in production (Wrmelink et al., 2018), business and management (Vesa et al., 2017) and even government communication (Harviainen & Hassan, 2019; Hassan et al., 2019), just to name but a few application areas.
Gamification, as briefly mentioned in the introduction, has many definitions. It has been defined as the use of game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems (Kapp, 2012). It has also been defined as using game elements and game design techniques in non-game contexts (Deterding et al., 2011). Or it has been defined as an overall holistic approach to design, akin to serious games (Hamari, 2019). Emphasis has also been especially put on defining and measuring gamification on the basis of the gameful experience it aims to give rise to, rather than on the basis of the elements it employs or the context of its use (Landers, 2019). Across these definitions, there is consensus on gamification being an old practice since the stone ages, where gatherers would play games to pass time while doing their field work. But the specific label of “gamification” has only recently appeared, arguably, around the year 2010, with strong roots in marketing, such as in connection to loyalty cards and membership rewards (Nelson, 2012). Please see (Jagoda, 2020) for a longer treatise of the origins of gamification.
Furthermore, it has been challenging to distinguish gamification from other game-based approaches. Different definitions for these different approaches to education exist, grounding them in different traditions. Nonetheless, there remains a significant overlap and the terms can often be used interchangeably or are defined in practically interchangeable ways across different researchers and research traditions (Aura et al., 2022). For example, the game-based approach we discuss later in this manuscript can be defined as serious gaming according to some definitions (Ullah et al., 2022), game-based learning according to other definitions (Pan et al., 2021), or as gamification (Deterding et al., 2011). We adopt the term “gamification” as it has been posited an umbrella term for these different approaches (Hamari, 2019; Landers, 2019; Nelson, 2012). The use of gamification as an anchor term in this research is to simultaneously discuss a holistic game design - in the spirit of serious games or game-based learning - but also to evaluate the specific use of badges and leaderboards, most readily associated in the readers’ minds with gamification than with serious games.
The high visibility of gamification nowadays is believed to have been caused by many overlapping factors, including cheaper technology, increased personal data tracking possibilities, high-profile successes that gathered attention, and the legitimization of games in society (Deterding et al., 2011). Gamification may also have increased in popularity for how it could be used to “engage” workers and consumers, leading to exploitation and burnout, which adversely contribute to neoliberal goals (Rey, 2012). Hence, the term has grown to garner considerable negativity and criticism over the years (Bogost, 2014; Kapp, 2012) as well as considerable popularity in research and practice (Koivisto and Hamari, 2019) positively and negatively.
Gamification has long been used in early childhood development, as seen with how giving a 'golden star' to students is a familiar feature in many classrooms. Most traditionally, gamification employs game design and elements, such as badges and leaderboards to make non-gaming activities as engaging and enjoyable as many games are (Deterding et al., 2011; Huotari & Hamari, 2017). It can make learning activities more active, meaningful, and participatory (Majuri et al., 2018) and is now a fast-growing part of many classroom and teaching activities in close connection with labels like serious games, educational games and gameful design amongst others (Aura et al., 2022). Gamified education can be a method that encourages a sense of pride and achievement for learners of all ages (Aura et al., 2022; Majuri et al., 2018) and is often used by educators to create engaging experiences that can motivate learners (Aura et al., 2021b). Research has demonstrated that gamified education has a notable impact on students' academic achievement, motivation, thinking skills and understanding of concepts in comparison with non-gamified learning methods (Bai et al., 2020; Zhan et al., 2022). There is also evidence that supports the notion that gamification is effective in enhancing cognitive, motivational, and behavioral learning outcomes (Sailer & Homner, 2020). Incorporating game elements into education not only facilitates the acquisition of new knowledge, but also promotes the retention of the learned material, which is a fundamental objective of any educational approach (Putz et al., 2020).
Gamification can effectively enhance learning motivation and achieve these outlined positive outcomes by utilizing well-designed incentives, stimuli, and rewards (Dicheva et al., 2015). Different types of rewards, including surprise, random, and revolving rewards, can create varying levels of anticipation and increase learners' engagement but the successful implementation of motivational strategies must ensure that the rewards hold significant meaning for learners. Rewards without meaningful value fail to provide the desired motivation and are not truly rewarding (Rigby, 2015). Hence, the importance of finding rewards that suit learners best as is the objective of this study.
In addition, the use of games in the learning process helps in the development and application of acquired information through experiences of fun, fantasy, control, challenge, curiosity and competition, which can foster intrinsic motivation (Cordova & Lepper, 1996; Malone, 1981). Gamification can also make understanding and achieving the steps in an activity easier by providing users with frequent feedback and reinforcement by gaining incentives or achievement symbols (Hassan et al., 2019; 2020) as learners move towards higher goals. Hence, gamification often requires the breaking down of long-term success goals into many smaller goals, allowing learners to focus on the next step in the chain and get immediate feedback as they complete each activity (Hamari et al., 2018). The quick nature of feedback to learners is one of the most compelling arguments for the use of gamification in education (Aura et al., 2022), which is an essential component of education even without any attempts to incorporate game design. Learners can be further reinforced with feedback mechanisms by using elements of game design. Designers can develop continuous informational feedback for learners with self-selection of exercises, visual cues, frequent question-and-answer activities, a progress bar, or carefully placed comments. Informative feedback differs from simple rewards because unlike points and grades, which simply tell learners whether they are right or wrong, informational feedback provides immediate explanations as to why a response is right or wrong (Hassan et al., 2019).
However, gamification also can have many negative effects on learners. Toda and colleagues (2018) identified four general areas of negative effects of gamification in the educational context which are indifference, loss of performance, fostering of undesirable behavior, and declining (novelty) effects. First, indifference is observed when gamification is reported to have no impact, whether positive or negative, on learners. Second, loss of performance stems from tasks and scenarios in which gamification disrupts or impedes the students' learning process. Third, undesired behavior happens when gamification results in an undesired impact (either positive or negative) on the learning environment in which it was implemented, either due to inadequate planning or a lack thereof. Fourth, declining effects (novelty effects) are connected to the gradual waning of motivation and engagement caused by the employed gamification. For instance, students may initially begin the activity with high motivation due to the novelty of gamification, but this motivation diminishes over time.
Negative effects of gamification can occur because of the lack of frameworks of gamification planning and deployment and/or the absence of educational theories to enhance gamification framework such as motivational design theories which can increase engagement and motivation of learners (Attali, and Arieli-Attali, 2015; De-Marcos, et al., 2014). Therefore, these instructional design theories are needed to help build gamification strategies in educational context which may have positive impacts on learners.
Motivation in Education and Gamification
Motivation is a critical factor in psychology, as it is the driving force that compels individuals to pursue certain things while avoiding others. It is responsible for driving and shaping our behavior, from the initiation of actions to their direction, maintenance, and termination (Burton, Westen & Kowalski, 2015). Motivation therefore also plays a vital role in the learning process, as it directs learners towards specific goals and determines the level of effort and energy, they invest in achieving them (Bakar, 2014). Studies indicate that motivated learners tend to exhibit higher levels of engagement (Alemayehu & Chen, 2021), persistence (Vollmeyer & Rheinberg, 2000), and academic achievement (Amrai et al., 2011) than their less motivated peers.
Various theories have been influential in shaping the design of game-based and gamified learning environments and are also frequently referenced in this context. These theories include self-determination theory (SDT), behavior reinforcement theory, goal-setting theory, need achievement theory, flow theory, social comparison theory, and personal investment theory (PIT) (Koivisto and Hamari, 2019; Richter, et al., 2015; Landers et al., 2015). While each of these theories offer valuable insights into the development of an effective and engaging learning experience, the focus of this study primarily revolves around the significance of the self-determination theory.
Self-determination theory (Deci & Ryan, 1985) has emerged as a foundational concept in comprehending human motivation. The theory distinguishes between two fundamental forms of motivation: intrinsic and extrinsic motivation. Intrinsic motivation refers to the psychological processes that foster a sense of competence and self-determination in an individual, driving them to engage in a behavior for the inherent satisfaction and personal fulfillment (Ryan & Deci, 2002; Vallerand, 2001). Extrinsic motivation, on the other hand, involves engaging in behavior for reasons external to the activity itself, such as the pursuit of rewards, reinforcement, or the avoidance of punishment (Vallerand, 1997).
Intrinsic motivation offers numerous benefits to learners (Deci & Ryan, 1985). It encompasses internal factors that drive learners to engage in activities, such as their interests, enjoyment, and excitement (Niemiec & Ryan, 2009). These motivators can stem from a genuine fascination with the subject matter, a sense of its relevance to life and the world, a feeling of accomplishment in mastering it, or a personal calling to it. Ormrod (1995) further explains that learners who are intrinsically motivated approach their tasks with enthusiasm, curiosity, and a genuine desire to acquire knowledge. In contrast, learners driven by extrinsic motivation tend to focus primarily on meeting the minimum academic requirements.
The form of intrinsic motivation described above, which is characterized by a proactive attitude towards learning and purposeful striving towards understanding, is often referred to as cognitive motivation. This type of motivation can be identified and fostered by cultivating students’ enjoyment, curiosity, appropriate challenge, perseverance, and engagement in the learning process (Gottfried, 1990; Klausmeier, 1985). It holds particular significance in the educational context as cognitive motivation directs students to pursue knowledge driven by their own convictions, leading to a deeper and more enduring understanding of the subject matter (Locke & Latham, 2006, p. 2). The cognitive motivation to learn can be divided into three categories, as indicated by (Vallerand et al., 1992):
1.
2.
3.
Hypothesis Development
The self-determination theory emphasizes that individuals are more likely to become self-motivated when engaging in activities that provide intrinsic rewards and that are accompanied by feelings of autonomy, purpose, mastery, pleasure and satisfaction (Rigby, 2015). Conversely, activities driven by extrinsic motivation often do not sustain long term engagement (Ingledew & Markland, 2008; Murcia et al., 2008). However, the theory suggests that learning can represent a behavioral habit, wherein initially extrinsic motives can become internalized and aligned with oneself (Deci & Ryan, 1985). This internalization process can lead to sustained engagement in the activity, especially when the extrinsic motives are compatible with and complementary to one’s needs and priorities, resulting in positive experiences.
The integration of gamification, which leverages both internal and external stimuli, has been found to effectively influence learner’s behavior by fulfilling their basic psychological needs for autonomy, competence, and relatedness, (Ryan, Rigby, and Przybylski, 2006). Manipulating learners' abilities to fulfill these psychological needs within the context of game-based learning, has been associated with improved performance and higher levels of motivation and enjoyment (Sheldon & Filak, 2008). Furthermore, the fulfillment of these needs has been linked to enhanced task performance and efficiency.
It is important to note that applications of gamification can target both intrinsic and extrinsic motivations. For example, on a psychology focused social networking site, where learners could earn badges by completing knowledge tests. This addressed both extrinsic motivation (badges as rewards) and intrinsic motivation (fulfillment of psychological needs for competence and relatedness) (Landers & Callan, 2011). Such examples demonstrate how gamification may leverage both internal and external stimuli to effectively influence learners’ behavior (Landers, 2019, p. 179).
Badges and leaderboards are some of the most commonly used mechanics in gamification, capable of influencing both intrinsic and extrinsic motivations when well designed (Ibanez et al., 2014; Na and Han, 2023). In the educational context, previous research has shown that gamification interventions incorporating badges and leaderboards have a tendency to impact learning outcomes (see Aura et al., 2021a; Majuri et al., 2018 for reviews). Studies also suggest that both badges and leaderboards can positively impact motivation and engagement (Mekler et al., 2017; Landers, R.N., & Landers, A.K., 2014; Hamari et al., 2014; Denny, 2013). But relatively less research has comparatively examined badges against leaderboards, specifically in a non-western context where one of the elements could be received differently due to cultural differences or unique cultural significances as outlined in the introduction of this paper. Further investigation is needed to determine the specific effects and potential advantages of each mechanic, especially because previous research has indicated that the outcomes from and reception of gamification can depend on the context (culture) in which it is implemented (Hassan et al., 2018), hence, existing findings cannot be fully applicable to non-western or developing countries. Based on this, the following hypotheses are proposed:
There is a statistically significant difference at (0.05) in the cognitive motivation of the pupils in the first experimental group post using badges in an educational game.
There is a statistically significant difference at (0.05) in the cognitive motivation of the pupils in the second experimental group post using leaderboards in an educational game.
There is a statistically significant difference at (0.05) in the achievement motivation of the pupils in the first experimental group post using badges in an educational game.
There is a statistically significant difference at (0.05) in the achievement motivation of the pupils in the second experimental group post using leaderboards in an educational game.
As highlighted earlier, cognitive motivation plays a significant role in the educational context. However, it remains essential to understand the specific effects of badges and leaderboards on knowledge, achievement and the overall educational experience, given this non-western context for the same reasons outlined with the previous hypotheses. Existing research has explored the positive impact of badges on learner engagement, participation, self-directed learning and learner autonomy (Sailer et al., 2017; Denny et al., 2018). Similarly, the effects of leaderboards have been examined in terms of increasing motivation, effort in learning activities (Dicheva et al., 2015) and promoting social interaction and collaboration among students (Landers et al., 2015). However, there is limited peer-reviewed research comparing the effects of badges and leaderboards in the context of Middle Eastern or developing countries, as previously outlined in the introduction. Therefore, based on the available evidence, following hypotheses are proposed:
There is a statistically significant difference at (0.05) in the cognitive motivation of the pupils in the first and second experiment groups post measurement due to the main effect of the experimental treatments (badges - leaderboard)
There is a statistically significant difference at (0.05) in the achievement motivation of the pupils in the first and second experiment groups post measurement due to the main effect of the experimental treatments (badges - leaderboard)
Method and Data
This research employed an action design research approach (Sein et al., 2011) to develop and evaluate two prototypes of a game aimed at teaching primary school students about the circulatory system. Both prototypes were identical except for the type of in-game reward system employed. The first prototype utilized badges as rewards, while the second prototype incorporated leaderboard placement as a reward. A field experiment was conducted simultaneously to evaluate the effectiveness of both prototypes, and pre and post use surveys were administered for further assessment. The action research approach facilitated collaboration with the research participants, allowing the creation of valuable work in partnership with them.
Research Participants
The study participants were 5th grade primary school pupils from Egypt (N = 30), with an age range of 9 to 11 years of age. This research took place in Egypt, led by an Egyptian researcher, associated with an Egyptian university, and the research involved cooperations with Finnish researchers. Due to the lack of established research ethics guidelines in Egypt, or at least we failed to identify relevant ones, we utilized non-formal, social guidelines that were communicated to us by local academics and the school administration. Evaluating this research against the guidelines of the Finnish Advisory Board on Research Integrity (TENK), we further find that this research did not require ethical clearances as it took place at a school, under the direct supervision and approval of the school administration and teachers, and it does not involve the administration of significant stimuli or sensitive data collection. The students who participated in this study were asked to volunteer to participate in this research, without any consequences befalling them. Given the novelty of the gamification, there was significant excitement to participate in the research among the students and no one declined to participate. Close attention was also paid to non-verbal signs of consent and we did not see signs for non-consensual participation to the best of our abilities. The school’s science teachers also showed interest and actively collaborated in overcoming any obstacles encountered during the research. The principal expressed a particular interest in the outcomes of the study and requested to receive the findings upon completion of the research. It was confirmed that all participants had prior familiarity with computers, the internet and games through their computer teachers and an introductory session conducted at the beginning of the experiment.
Procedure
Summary of the experimental groups and treatments.
Before the commencement of the experiment, an introductory session was conducted with the pupils to familiarize them with the research purpose and provide a general overview of the learning game they would be using. Visual aids, such as pictures and models, were presented to enhance their understanding. It was observed that almost all, if not all, participants exhibited great interest and enjoyment during this session. Additionally, during this session, the students were queried about their familiarity with digital games, particularly those whose design was similar to that of the game understudy, and it was found that without exception all students were well acquainted with such games.
Subsequently, individual accounts were created for each student in the two experimental groups, and the relevant account details (student name, entry email, password and the web address of the learning environment) were printed on cards and distributed to the school administration for further dissemination to the students.
The school’s Facebook page was used as a technical support channel to address any queries or technical issues raised by the students. The primary researcher visited the school every Tuesday throughout the five-week application period to document the experiment’s progress, engage with students and assess their experiences thus far. Data collection for post measurements was conducted during the sixth week over a five-day period, allowing for the absence of some students from school.
Research Material / Game Design and Game Play
To further gamify education, we embarked on developing a game aimed at helping students learn about circulatory systems in the science curriculum. Collaborating with a multidisciplinary team—including an instructional designer, illustrators, a developer, a voice-over artist, and a tester—we designed the game. Upon recommendations from science teachers and considering target students’ penchant for watching cartoons, the researchers opted for a cartoon-themed game. Leveraging the lead researcher's experience in instructional design and game design, he crafted the game's scenario and storyboard. Collaborating closely with illustrators, the lead author ensured that characters were designed to reflect Egyptian children, and the laboratory environment mirrored their actual school labs. Additionally, a fantasy character, representing a drop of blood, was created to guide the learning journey and explain the content as shown in figure 1. Furthermore, the cartoon voices were selected to perform in the game to engage students throughout the game and also employed clear Arabic language voices. Post-storyboarding, discussions with developers ensued. Following the game's development, accurate testing was conducted by the tester to identify and resolve any technical issues. Main characters and laboratory background in game.
The game developed for this research focused on teaching students about the circulatory system, which was part of the science curriculum at the school in the second semester of the 5th grade. This challenging topic was identified by the science teachers at the school. The content for the game was also provided by them. Figure 2 provides an overview of the game’s levels, which were identical for both experimental groups, except for the assessment stage, where one group was assessed and rewarded through badges and the other through leaderboard placement. Overview of the game’s levels.
The game is divided into six locked levels except the first one. After the student logs in, a short-animated video introduction begins to show the student the usage and goals of the game. Watching the introductory video is mandatory during the first log-in but becomes optional thereafter. Next, the main menu displays locked game levels that unlock as the learner progresses, aligning with the educational objectives specified by the science teachers. The student starts the game with the first unlocked level. Upon completing a level, there is an interactive exercise to open the next level. The researchers and teacher were keen to employ formative evaluation to gauge students' progress toward educational goals. This involved a series of questions following each level, designed to ascertain the degree of accomplishment of the lesson objectives. These questions serve as a measure for obtaining achievements and rewards and were crafted with input from science teachers. The learners earn a specific number of points from the exercise. To advance to the next level, they must achieve a minimum score of at least 80% of the maximum possible score for that level. If a learner scores below 80% in either treatment group, they must repeat the level. In such cases, the questions in the level are changed every time they repeat the level to promote learning instead of memorizing correct answers for level completion. Learners also have the option to repeat a level as many times as they wish, with their highest score being considered by the game’s reward system rather than their most recent score. The score is displayed within a frame in the middle of the interface after finishing the exercise. During the exercise screen, responses are checked promptly and displayed at the top of the exercise interface as immediate feedback. Also, the points are fixed in their designated place on the interface. In the badge’s treatment group, learners receive bronze badges for scoring 80% to 90% of the possible level score, silver badges for scoring 90% or more, and gold badges for achieving a perfect score (100%). In the leaderboard treatment group, learners are ranked on the leaderboard based on their scores in a level.
Figure 3 showcases screenshots from the game. Screenshots from the game.
Figure 4 illustrates the possible badges that could be earned by learners in the first experiment group. Different badges were earnable in the game. They were designed to reflect notable achievements connected to the subject being learned. For example, a distinct badge featuring a heart was designed to represent proficiency in understanding how the heart works in the circulatory system. To earn this badge, students are required to achieve a minimum of 80% in the corresponding exercise. Additionally, several badge tiers have been introduced for each badge: bronze, silver, and gold. Upon completing all levels, a cup badge is unlocked, also available in bronze, silver, and gold variants. The type of cup badge awarded is contingent on the student's overall performance, with cup badge (bronze, silver, and gold). The possible badges in the game.
Measurements
Two scales were developed and utilized in this research to measure the cognitive and achievement motivation of the primary stage pupils. The cognitive motivation scale consisted of 31-items and the achievement motivation scale consisted of 12-items, both scored on a 5-point Likert-type response format (strongly disagree, disagree, neutral, agree, or strongly agree). All the items were positively worded, so the scale scores were calculated by summing the scores on individual items.
Cognitive Motivation Scale
The Cognitive Motivation Scale was developed as part of the research, by drawing upon the existing literature that dealt with the concept of cognitive motivation (Valle, A. et al., 2003). Cognitive motivation is commonly conceptualized to encompass three dimensions (Beswick, 2017; Valle et al., 2003; Crawford et al., 2021): 1) Tendency to Knowledge, reflecting the desire to acquire scientific knowledge and connect it to one’s life and other concepts. 2) Curiosity: indicating a strong desire to learn more about the subject matter measured by the ability to respond to any interesting or surprising questions about the circulatory system beyond the information covered in the game understudy. 3) Raising Questions, capturing the inclination to ask questions to gain deeper understanding of the topic, for example, more explanations and a practical understanding of the circulatory system. In the context of this study, the focus was on the circulatory system and its functions, which were covered in the game.
Achievement Scale
The achievement scale aimed to measure the desire for high achievements and the desire to seek information (Atkinson & Feather, 1966). However, since these aspects could not be directly measured in the game used in the study, alternative means were developed based on previous literature, to estimate the research participants' achievement orientation (Auvinen et al., 2015; Hakulinen et al., 2015).
The measurement items of both scales can be found in Appendix A. The pre and post experiment measurements were conducted in Arabic.
Validity
To improve the validity and reliability of the developed scales, they were reviewed by experts in educational science. Suggestions provided by the experts were incorporated and the scales were translated into Arabic and linguistically adapted to suit the age group of the research participants. Arabic language teachers and translators also evaluated the translated versions to ensure the clarity and comprehensibility of the items for the students (research participants). Some items were eliminated if they were redundant or due to their lack of conformity to the research sample or learning subject. After these revisions, the cognitive motivation scale consisted of 31 items, with a total possible score ranging between 31 to 155. The achievement motivation scale had 12 items with a total possible score ranging between 12 to 60.
To confirm the validity of the scales, Pearson's simple linear correlation coefficient was calculated to examine the relationship between each individual item and its corresponding dimension, as well as the overall degree of the scale. The results (see Appendix B) indicate that all correlation coefficients are statistically significant, demonstrating the consistency of all dimensions of the scales.
Reliability
The reliability of the cognitive motivation scale was assessed by examining the correlation coefficient between the individual items and their corresponding dimension. The Spearman-Brown coefficient yielded a value of 0.85, indicating satisfactory internal consistency. Furthermore, Cronbach's alpha coefficient for the total scale was calculated to be 0.82, suggesting a high level of reliability. The Gutman coefficient also demonstrated favorable reliability with a value of 0.84. Similarly, for the achievement motivation scale, the stability factor Spearman-Brown coefficient was found to be 0.788, while Cronbach’s alpha coefficient for the total scale was 0.781, indicating good internal consistency. The Gutman coefficient for the achievement motivation scale was 0.786, indicating sound reliability. These results (see Appendix B) provide evidence of the scales’ reliability and support their use in the research context.
Results and Analysis
This study was conducted to investigate the effects of gamification on the cognitive and achievement motivations of primary school students in a developing country. The study employed the Mann-Whitney and Wilcoxon tests to examine the presence of a statistically significant difference between the pre and post test results. The independent variable in this study was the type of in-game reward (badges or leaderboard), while the dependent variables were cognitive motivation and achievement motivation. As suggested in the literature, the alpha was established a priori at the 0.05 level, meaning that the employed gamified designs have a positive influence on the dependent variables. The following sections discuss in detail the findings of this study.
Group 1, Badges
Wilcoxon test results indicating the difference between the mean of ranks significant of group one in terms of Pre and post results of cognitive motivation.
For cognitive motivation, the z-value was found to be -3.409 (for all dimensions), which falls well beyond the critical values (±1.96 for α = 0.05, n = 15 and ±2.58 for α = 0.01, n = 15). This indicates a statistically significant difference (p < 0.01) in the pre and post test results. The negative sign of the z-values suggests that the group with higher scores predominantly come from the post tests. Furthermore, the effect size for cognitive motivation was calculated to be 0.55, indicating a large effect. The results support the hypothesis (H1) that there is a statistically significant difference (p < 0.05) in the cognitive motivation of the students post using badges as an incentive in an educational game.
Wilcoxon test results, indicating the difference between the mean of ranks significant of group one in terms of Pre and post results of achievement motivation.
Group 2, Leaderboards
Wilcoxon test results indicating the difference between the mean of ranks significant of group two in terms of Pre and post results of cognitive motivation.
For cognitive motivation, the z-value was found to be -3.409 (for all dimensions), which falls well beyond the critical values (±1.96 for α = 0.05, n = 15 and ±2.58 for α = 0.01, n = 15). This indicates a statistically significant difference (p < 0.01) in the pre and post test results. The negative sign of the z-values suggests that the group with higher scores predominantly come from the post tests. Furthermore, the effect size for cognitive motivation was calculated to be 0.55, indicating a large effect. The results support the hypothesis (H2) that there is a statistically significant difference (p < 0.05) in the cognitive motivation of the students post using leaderboards as an incentive in an educational game.
Wilcoxon test results indicating the difference between the mean of ranks significant of group two in terms of Pre and post results of achievement motivation.
Comparing Experiment Groups
Comparison of Blake’s Modified Gain Ratio between experiment groups for the cognitive motivation.
Comparison of Blake’s Modified Gain Ratio between experiment groups for the achievement motivation.
In order to compare the influence of badges and leaderboards on the experimental groups, the post test scores of both groups for cognitive and achievement motivation were analyzed using the Mann-Whitney test
Mann-Whitney test results indicating the difference between the mean of ranks of the two experimental groups grades in the post-measurement of cognitive motivation.
Mann-Whitney test results indicating the difference between the mean of ranks of the two experimental groups in the post-measurement of achievement motivation.
The findings from both tables indicate that the effects of badges and leaderboards were remarkably similar. The calculated z-values for both cognitive and achievement motivation were not statistically significant (p > 0.05), suggesting that there was no noteworthy distinction in the effects of badges and leaderboards on the cognitive and achievement motivations of the participants in the two experimental groups.
Discussion
The aim of this study was to comparatively examine the effects of badges and leaderboards on the cognitive and achievement motivations of pupils at a primary school level. As such, this is one of the few studies that explored gamification incentives with children, in a developing country with mixed perceptions towards games and using a field experiment that can be considered action research. The result of this study provides support to H1-6. Based on that, the following observations can be made:
Theoretical Implications
Badges (Boticki et al., 2015; Davis & Klein, 2015) and leaderboards (Landers & Landers, 2014; Shi et al., 2014) are often considered incentives that motivate learners to perform missions/actions in games or in real life. The present study adds to the existing body of literature on gamification in (science) education, particularly in primary education.
Globally, there seems to be a digital divide in the adoption of gamification in education. Research indicates that the majority of the studies in gamified education have been conducted in the USA, Canada, Spain and Germany, while many developing countries have limited studies (Nadi-Ravandi & Batooli, 2022). This disparity can contribute to the reluctance of educational institutes in developing countries to adopt innovative educational approaches, including gamification. While the precise reasons for this reluctance were not within the scope of our study, potential factors may include resource limitations, a reliance on traditional teaching methods, religiousness, negative attitude towards games or doubts about the effectiveness of gamification. This study, therefore, highlights the positive reception of this approach to education in the described local context, and the potential benefits of gamification in the localized context, particularly when individualized attention from teachers may be scarce, thereby aiming to bridge this divide and encourage the adoption of innovative methods in developing countries.
Consistent with previous studies (Hursen & Bas, 2019; Park & Kim, 2021; Janković et al., 2023), the results highlight the positive impact of gamification incentives on students’ motivation and engagement in science education. The integration of badges and leaderboards within the learning environment provided a meaningful incentive for students to actively participate in learning activities and fostered a sense of competence and autonomy in their learning processes. This aligns with the theoretical framework of self-determination theory, which emphasizes the importance of intrinsic motivation and perceived control in driving optimal learning outcomes. The accounts received from the science teachers involved in the study further support the notion that gamification incentives had a lasting impact on students’ cognitive motivation, as also evidenced by their increased curiosity, engagement with scientific content and inclination to seek explanations even outside the game. This aligns with the positive opinions expressed by students and parents regarding the usage of gamification in science education (Hursen & Bas, 2019). These observations support hypotheses H1 and H2, which proposed that badges and leaderboards would have a positive effect on the cognitive motivation of students. However, it is important to note that the effectiveness of gamification in promoting motivation is contingent upon a sound pedagogical rationale (Alt, 2023).
The students’ enthusiastic display of earned badges and their eagerness to showcase their leaderboard placement to the lead researcher (someone who could be thought of as an outsider with perceived importance) during weekly visits to the school throughout the field experiments suggest a desire for social recognition and a sense of achievement. These behaviors are consistent with the psychological effects associated with gamification, such as the desire for social reciprocity and the signaling of achievement (Tamborini et al., 2010). Moreover, the students’ enjoyment of the game and their extended hours of gameplay further support the notion that the external rewards stimulated internal motivations. This suggests that over time, a level of internalization potentially occurred as the external motivators helped foster intrinsic motivations to some extent. In a related study (Ioannou & Kyza, 2017), the analysis of the data explored the relationship between gamification and students’ learning gains and motivation. The results indicated that the gamified approach contributed to an increase in students’ extrinsic motivation and improvement in the subject knowledge. However, the impact on intrinsic motivation was limited, except for the enhancement of interest. These findings align with the present study, suggesting that gamification interventions, specifically through badges and leaderboards, can effectively enhance students’ motivation and promote learning outcomes. These observations also support hypotheses H3 and H4, which proposed that badges and leaderboards would have a positive effect on the achievement motivation of students.
While this study was conducted in a developing, Middle Eastern country, it is noteworthy that the observed outcomes align closely with those reported in the existing gamification literature from more developed regions. No significant differences were discerned between the results from this study and those reported in the literature. This highlights the potential universality and generalizability of gamification's impact on student motivation, regardless of the country's developmental context. It suggests that the positive effects of gamification, as evidenced by enhanced cognitive and achievement motivations, may transcend geographical boundaries and educational systems. This observation encourages the consideration of gamification as a viable and effective educational strategy in diverse global settings, thereby reinforcing the potential for its widespread adoption to enhance student motivation and engagement.
The design of badges in this study offered students a relatively easy means to showcase their achievements. Moreover, badges were earned based on individual progress and skill development, irrespective of how one compared to others, allowing for a sense of accomplishment that could be attained and maintained more readily compared to leaderboards. However, it is important to note that within the badge system, a hierarchy of difficulty existed, with silver and gold badges presenting more challenging goals for the students. The hierarchical structure aligned well with Csikszentmihalyi’s flow theory (Csikszentmihalyi, 2014), wherein the optimal experience arises from matching challenge levels with an individual’s skills. More difficult goals, in terms of more challenging levels or the potential to earn gold badges, where not available to learners unless they attained easier goals first, in terms of easier levels and silver badges. The progression in the difficulty of levels and of earning silver or gold badges might have contributed to flow experiences and increased challenges among students as their skills developed, however, we can’t ascertain that as we did not directly measure students’ flow experiences. Notably, students frequently reported, “I got the silver badge, but I will improve my level to get gold.” This demonstrated that the varying difficulty levels of the badges not only provided appropriate challenges but also motivated students to enhance their learning and skills.
It is interesting to note that the number of studies focusing on gamification in primary education is significantly lower compared to those in secondary and higher education (Aura et al., 2022). This disparity may be attributed to the fact that science education is more restricted at the primary level than language and math (Kalogiannakis et al., 2021). Therefore, educators should carefully design and implement gamification strategies that align with educational objectives and foster a supportive learning environment. These findings collectively suggest that gamification holds promise as an effective tool to enhance motivation and engagement in primary school education, particularly in the domain of science.
Practical Implications
The present research offers valuable insights into the design and utilization of badges and leaderboards with young primary school pupils. During the course of the study, pictures of the badges earned by students were occasionally posted on the school’s social media Facebook page to encourage participation in the experiment. This practice augmented the social signaling effects of badges. However, doing the same with leaderboards posed more challenges, as it would have required obtaining consent from multiple students before sharing such posts and potentially infringed on their privacy. Furthermore, posting a social ranking of students through leaderboards could potentially lead to bullying or hyper competitiveness, which are highly undesirable outcomes. While the results of this study did not show any significant difference between the use of badges and leaderboards previous studies do suggest that the individualistic and competitive nature of leaderboards might serve to adversely influence students’ ability to attain the desired educational outcomes (Huang et al., 2020). It might also be that the novelty of both badges and leaderboards contributed to their positive impact, fostering curiosity and enthusiasm among students rather than leading to adverse effects like hyper competitiveness. Further research is needed to explore these potential factors long term and their implications for gamification in developing country settings. In this regard, badges appeared to be a more suitable incentive in this particular school environment. It is therefore necessary to exercise caution when selecting rewards in educational settings so as to avoid unintended, adverse and potentially negative results (Landers et al., 2015).
Based on the findings of this research which support hypotheses H5 and H6, we can recommend the conscious use of leaderboards and badges in educational games, in-line with previous studies, with a particular focus on developing countries and resource limited settings. They might be especially useful with the latter when teacher resources are limited, and teachers are not available to help each student individually become interested in the subject matter. To sustain engagement, we recommend the use of mystery badges, where the students are not fully aware of when they will earn a badge, but they can still expect when they would generally receive a badge (see Nagle et al., 2014).During the experiment, the lead researcher requested the developer to lock and obscure the shape of the final level badges to make it unrecognizable to the learners. This was under the assumptions that obscuring the badges could further motivate the students to discover what those badges are and why they were “mysterious” and obscure. During school visits, students from the group with badges continued to inquire about the last level, when it would become accessible, and what the badges would represent it. It was evident to the researcher that the students' curiosity was increasing because of this mystery. Fully random reward schedules can be associated with addictive behavior, hence, designers should be responsible in their implementation of such a tactic and to give players a general idea of when to expect a badge without precision in expectations (Chang, 2022) and perhaps only use such a tact in the context of a finite game, where play is going to end at some point, rather than in an infinite open world game. Additionally, careful attention should be given to the aesthetics of the badges, aiming for designs that are professional and visually pleasing to the user (Jylhä and Hamari, 2019). The design of the badges in this study was inspired by the circulatory system, the subject matter of the game, and unique badges was reserved for particularly challenging tasks. While the personal considerations and individual differences in the reception and perception of badges and leaderboards were beyond the scope of this study, they should be taken into account in future research.
We recommend highlighting users’ achievements to themselves and to others, but it is important to use a combination of motivational tools, especially when utilizing leaderboards, to ensure that users who do not attain high ranks still experience a sense of achievement. Immediate feedback is essential as supported by previous research (Hassan et al., 2019). Therefore, designers should approach the overall design experience holistically, considering how different components complement each other, rather than viewing badges or leaderboards as isolated additions to existing solutions. Failing to achieve a cohesive fit between the design and the users can result in ineffective gamification efforts, where sustained engagement over the long term is not achieved.
Conclusion and Future Directions
Based on the findings of this study, it is evident that gamification can have a positive impact on students' learning outcomes, including their achievements, engagement, and motivation throughout the learning process. These results lead to the conclusion that incorporating game-like elements into teaching and learning can be highly beneficial for students’ learning and should be widely adopted by educators. However, it is important to note that when incorporating a leaderboard, it should be implemented in conjunction with other elements to ensure that learners who are unable to reach the top positions still feel motivated to continue their participation and learning journey. Likewise, when designing badges, it is crucial to create badges for difficult and complex tasks, while taking into consideration the individual differences among learners. Furthermore, future studies should focus on investigating gamification in primary education, as the majority of existing research has primarily targeted high school and university students. Additionally, there is a need for further exploration of other gamification elements such as feedback and learning experience systems. Moreover, it is essential to conduct research that delves into learners’ attitude towards gamified learning, as well as their levels of satisfaction and enjoyment with such approaches.
Supplemental Material
Supplemental Material - Effects of Gamification on Motivations of Elementary School Students: An Action Research Field Experiment
Supplemental Material for Effects of Gamification on Motivations of Elementary School Students: An Action Research Field Experiment by Mohammed Mohammed, Amal Fatemah, and Lobna Hassan in Simulation & Gaming
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Liikesivistysrahasto (grant number 22-12430).
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
