An Integrated View of Information Feedback,Game Quality,and Autonomous Motivation for Evaluating Game-Based Learning Effectiveness

Abstract

Critical factors that influence the value of digital game-based learning (DGBL) for enhancing learning effectiveness remain debatable. Based on the self-determination theory (SDT), people can be autonomously motivated to adopt DGBL to enhance learning effectiveness if their basic psychological needs are satisfied. Additionally, studies that simultaneously examine the effects of two critical factors of information feedback and game quality on students’ autonomous motivation and learning effectiveness in DGBL contexts are missing in the literature. This study integrates information feedback, game quality, and SDT to develop a research model for comprehending the effectiveness of DGBL systems. Data collected from 383 respondents was analyzed to validate our research model. The results showed that information feedback, game quality, and autonomous motivation significantly influenced students’ learning effectiveness and their continuance intention to use DGBL systems. Additionally, autonomous motivation indirectly influenced continuance intention via learning effectiveness. Implications for theory and for practice are discussed accordingly.

Keywords

digital game-based learning systems self-determination theory autonomous motivation information feedback game quality

Introduction

The rapid development of digital game-based learning (DGBL) systems show a great potential for educators to use educational games to support learning and teaching activities (Tsai et al., 2016). According to self-determination theory (SDT), people learn via games because games motivate them to learn by fulfilling their basic psychological needs, including autonomy and competence (Burgers et al., 2015; Park, Kim, et al., 2019). Individuals are more often encouraged to perform a specific behavior more by intrinsic-oriented motivation (i.e., autonomous motivation) than by extrinsic-oriented motivation (i.e., controlled motivation). While some academics claim that DGBL systems motivate people, improves their learning skills, and enhances learning effectiveness in the contexts of higher education (e.g., Santhanam et al., 2016) and K-12 education (e.g., Young et al., 2012), others have argued that DGBL systems do not provide learners with time for self-reflection in the contexts of higher education (van der Meij et al., 2011) because it is unclear if they are focusing on being entertained or educated. Consequently, more empirical studies are required to investigate the effectiveness of DGBL systems and the continuity of the use of the DGBL systems (i.e., continuance intention) (Girard et al., 2013). The discussion above implies that it is inconclusive regarding the effects of the use of educational games on learning performance in the literature. Prior research indicates that it is inconclusive regarding whether or not the use of technological tools (e.g., DGBL systems) is associated with the learners’ motivation, which is critical in terms of enhancing learners’ engagement in DGBL systems for learning purposes (Wouters et al., 2013). Proulx et al. (2017) point out that a primary reason for the inconclusive results regarding the relationship between DGBL systems and motivation is that the effectiveness of applying DGBL systems to various educational contexts is greatly dependent on various human-related and game-related factors, such as learners’ diversity, the characteristics of the course instructors, and gaming mechanisms. SDT asserts that the psychological need for autonomy (i.e., autonomous motivations) is likely to be inherent to all individuals, and the effect of one’s sense of autonomy on a particular behavior tends to be universal across individual differences in demographic and social factors (Deci & Ryan, 2000). Consequently, SDT has its merits in understanding the internal driving force of a particular individual behavior, such as their involvement in DGBL-related activities.

Some studies have considered various game elements or information feedback as an identical set of key variables for comprehending the effectiveness of DGBL systems in terms of enhancing learners’ learning performance and their intention to continue to the systems (Boyle et al., 2011; Damnjanovic et al., 2015; Göksün & Gürsoy, 2019; Hong et al., 2009; Johnson & Mayer, 2010; Moizer et al., 2019; Ninaus et al., 2019; Panigrahi et al., 2018; Park, Liu, et al., 2019; Tsai et al., 2016; Wu et al., 2012). For example, well-designed DGBL systems can shorten the feedback cycles to allow learners to maintain engagement and to easily evaluate their own capabilities as a result of their use of the DGBL systems (Su & Cheng, 2015). However, these studies have revealed that game-quality factors and information feedback represent essentially different dimensions of DBGL and should thus be investigated simultaneously but independently. Prompt and relevant information feedback offered by DGBL systems is considered a critical and concrete learning mechanics of DGBL systems and game-quality-related factors resemble the game mechanics of DGBL systems, both of which can influence the effectiveness of DGBL systems in terms of facilitating learning (Proulx et al., 2017). Therefore, in this study, information feedback and game quality have been adopted simultaneously as mutually independent antecedents influencing the tendency towards autonomous motivation, which is represented by the relative autonomy index (RAI), as implied in prior studies (Park, Kim, et al., 2019; Park, Liu, et al., 2019). We designed a DGBL system and developed a research model from an integrated perspective of SDT, game quality, and information feedback in order to advance our understanding of the relationships between intrinsic (i.e., autonomous) motivation and the learning effectiveness of learners.

To be specific, the primary research questions (RQs) of this study are as follows:

RQ1: How do the information feedback and game quality of DGBL systems contribute to the development of learners’ autonomous motivation?

RQ2: How the learners’ autonomous motivation influences their learning effectiveness and intention to continue to use DGBL systems in the context of higher education?

To answer our research questions, we developed a theoretical model in which information feedback and game quality were adopted as independent variables to examine their effects on autonomous (intrinsic) motivation, which is represented by the RAI, and students’ learning effectiveness and continuance intention to use DGBL systems. In this study, the subject of database normalization was used as an example to develop a web-based DGBL system that provided high-quality information feedback to its learners and to validate our research model. The concept of normalization is crucial to designing appropriate table structures of a relational database. While universal guidelines for database normalization at different levels are available, challenges remain for instructors to help students understand the concepts of normalization, as there exist guidelines rather than specific procedures for normalization processes. Students need to develop relevant higher-level cognitive reasoning skills in order to appropriately master those guidelines to construct a digital database with appropriate table structures, which has always been a challenge for both instructors and students in traditional classroom/lab-based educational settings. Prior research indicates that e-learning tools are effective in terms of enhancing students’ learning performance on the subject of database normalization because it allows the instructors to demonstrate the normalization process to students in an interactive manner, such as using interactive problem-based learning methods (Hoque et al., 2012; Kung & Tung, 2006; Zhang, 2005). It is thus worthwhile to investigate how DGBL systems can be of assistance to resolve this challenge.

Theoretical Background and Related Studies

Digital Game-Based Learning

DGBL systems can enhance learning achievement and motivation for K-12 students (Chen & Wang, 2018; Rosas et al., 2003) and lifelong learners (Moizer et al., 2019; Sharples, 2000). Papastergiou (2009) claimed that the educational objectives and the focal subjects of DGBL defined with reference to learners’ needs in order to make them perceive learning via the DGBL to be easier, more interesting and enjoyable. Furthermore, prior research indicate that DGBL systems can significantly promote learning in various contexts, including higher education settings (Ninaus et al., 2019; Park, Kim, et al., 2019).

DGBL systems integrate game content with gameplay in order to facilitate students’ acquisition of knowledge and skills, enabling them to have a greater sense of achievement and increased learning motivation. Sharples (2000) also believed that DGBL systems could enhance learning effectiveness and motivation in higher education. However, to make it compelling, learning and gameplay must be tightly integrated.

Information Feedback

In this study, information feedback refers to the corresponding information that DGBL systems offer to learners based on the systems’ evaluation regarding the outcomes of the learners’ decisions made during DGBL-supported learning processes. Previous studies have claimed that information feedback can be categorized into three types: outcome feedback, feed forward, and cognitive feedback (Cohen et al., 2016; Gonzalez, 2005).

First, outcome feedback is also called the knowledge of results. It stands for decision outcomes and the process of verification (Narciss, 2008). It is important in terms of offering learners a continued challenge to achieve learning goals (Göksün & Gürsoy, 2019; Park, Liu, et al., 2019). Next, feed forward is a process wherein learners are provided with information about a required task in advance, in order to improve the quality of their decisions (Björkman, 1972). Feed forward is important because it enables learners to correctly implement a task (Chenoweth et al., 2004). Finally, cognitive feedback refers to the information feedback that is offered for the purpose of explaining how the decision is made (Chenoweth et al., 2004) and is useful when completing complex learning tasks (Lindell, 1976).

Game Quality

Good game design of DGBL systems can lead to more effective learning (Qian & Clark, 2016). It has a direct effect on learners’ performance (Osman & Bakar, 2012). A good game design enables learners to engage in reflection (Johnson & Mayer, 2010). However, previous studies have pointed out that games may distract learners and lower their rate of participation (Young et al., 2012). Consequently, game quality is positively associated with learners’ learning motivations and learning effectiveness. Therefore, in the present research, we reviewed the literature on game characteristics/elements, as presented in Table 1, and identified four primary categories of game quality: rules/goals, challenges, control, and competition (Garris et al., 2002; Hou & Li, 2014; Jung et al., 2010; Kim & Shute, 2015; Moizer et al., 2019; Santhanam et al., 2016; Tan et al., 2016; Yang & Chang, 2013).

Table 1.

Summary of the Categories of Game Quality.

Game-quality dimensionLiterature	Fantasy	Goal/rule	Sensory stimuli	Challenge	Mystery	Control	Competition
Garris et al. (2002)	✓	✓	✓	✓	✓	✓
Hou & Li (2014)		✓		✓
Johnson & Mayer (2010)		✓		✓
Jung et al. (2010)		✓		✓			✓
Kim & Shute (2015)				✓		✓
Moizer et al. (2019)				✓		✓
Ranchhod et al. (2014)		✓				✓	✓
Santhanam et al. (2016)							✓
Tan et al. (2016)		✓		✓		✓
Yang & Chang (2013)						✓

Prior research points out that if the three game elements of rules/goals, challenges, and competition are included in learning games, learners would have better learning results. This is because that these three game elements meet the learners’ intrinsic needs for competence, thus enhancing learners’ intrinsic motivation and improve learning effectiveness (Jung et al., 2010; Tan et al., 2016). Additionally, prior studies indicate that the element of competition serve as a source of motivation and challenge that draw attention of students in learning environments (Malone, 1981; Santhanam et al., 2016), It can thus help encourage the learners to actively participate in the learning games for learning (Demetrovics et al., 2011; Liu et al., 2013; Yee, 2006). Ranchhod et al. (2014) also argue that competition (engagement of a player against himself/herself, computer, or other players) is an inherent element of digital educational games that motivate the players to make decisions in order to achieve their final objectives. Based on the propositions of the prior studies above, the element of competition is considered to be one of the key elements of game quality in the current study.

The elements of fantasy (imaginary context or entities), sensory stimuli (dramatic visual and/or auditory stimuli), and mystery (optimal level of information complexity) (Garris et al., 2002) are not included in the current study for the following reasons. First, endogenous fantasy (fantasy that is directly related to learning content) that is preferred in learning games (Garris et al., 2002; Rieber, 1996) is difficult to achieve due to the abstract essence of the learning materials (i.e., normalization of digital database) of the current study. Second, while the existence of sensory stimuli is common in digital games to facilitate players’ intoxicated and exciting senses, we consider this factor to be more important in entertaining-driven digital games. Additionally, over-dramatic sensory stimuli may appear to be strange and create undesirable distraction to the student users of DGBL systems (Malone & Lepper, 1987). Finally, the concept of mystery is incorporated in the concept of information feedback that we included and investigated in more details in the current study. The four primary categories of game quality of rules/goals, challenges, control, and competition are discussion further in the following sections.

Rules/Goals

This refers to the clear and specific rules and goals by which feedback is generated. Rules and goals allow learners to perceive feedback while involved in gameplay. In addition, they enhance motivation as players try to reach their goal (Moizer et al., 2019; Su & Cheng, 2015).

Challenge

Challenge denotes the ideal level of trial. In other words, game players are challenged by gaming activities that employ progressive difficulty levels. Each of those levels should be neither too easy nor too difficult to accomplish and is associated with valued personal abilities that the game players desire to acquire.

Control

The term means the authority of regulating, managing, directing and commanding the progress of the game is given to learners. When learners have such authority, their learning motivation tends to be increased.

Competition

Competition occurs when more than two learners fight for superiority or victory (Liu et al., 2013). Prior studies have shown that learners competing with one another during gameplay are better motivated, enabling them to be more engaged in game activities (Demetrovics et al., 2011; Liu et al., 2013; Yee, 2006). Competition is a source of motivation and a sense of challenge in games (Malone, 1981). Numerous academics have argued that learners are most engaged in game activity when they intend to compete with one another.

Self-Determination Theory

Deci and Ryan (1985) proposed SDT to investigate the formation and the effects of individual motivation based on the concept of autonomy. The central premise of SDT is a controlled-to-autonomous motivation continuum. Based on this continuum, different types of motivation result in different efforts, outcomes, and levels of intention to continue to use in the future (Deci & Ryan, 1985; Ryan & Deci, 2000; Ryan et al., 2011). The key concepts of SDT can be categorized as described below.

Intrinsic Motivation

Intrinsic motivation refers to inherently autonomous motivation. In other words, people engage in activities because they find them interesting and decide to do so by themselves. They receive enjoyment from involving themselves in their chosen activities. This type of motivation is invariably self-determined (i.e., autonomously driven) (Gagné & Deci, 2005).

Extrinsic Motivation

Extrinsic motivation is also called controlled motivation. People behave a certain way because of some external purposes, such as the pursuit of a reward or compliment, or avoidance of punishment. Extrinsic motivation is not invariably controlled; it is possible to internalize extrinsic motivation (Roca & Gagné, 2008).

Amotivation

People refrain from engaging in certain activities because of a lack of motivation, intention, or purpose (i.e., amotivation). They do not understand the value or meaning of the focal behavior.

SDT proposes a continuum from controlled-to-autonomous motivation to highlight the importance of distinguishing different types of personal motivation. In other words, when people are autonomously motivated to learn, they may internalize the motivational process. SDT denotes that autonomous motivation and controlled motivation may take effects on a particular individual behavior at the same time. Based on the controlled-to-autonomous motivation continuum, individuals have different levels of autonomous motivation (i.e., the RAI) regarding a behavior dependent on the levels of different types of motivation that they exhibit. Therefore, academics can investigate why individuals perform a particular behavior by using the RAI to measure the extent to which people are motivated to engage in the behavior. In other words, if students/learners are more autonomously motivated to learn, they tend to possess a higher level of the RAI (Gagné & Deci, 2005; Ryan et al., 2006). The RAI refers to the degree to which individuals are autonomously motivated to exhibit a specific behavior. It represents individuals’ existing level of autonomous motivation. The formula for the RAI is as follows (Cockrell & Stone, 2010; Gagné & Deci, 2005; Wang, 2016):

\begin{matrix} RAI = [(2 * intrinsicmotivation) + identifiedregulation] - [introjectedregulation \\ + (2 * externalmotivation)] \end{matrix}

Based on the controlled-to-autonomous continuum of SDT (Gagné & Deci, 2005; Ryan & Connell, 1989; Ryan et al., 2011), the types of motivations that are used to calculate the RAI are introduced below. First, controlled/external motivation means that one is motivated by external rewards or punishments). Second, moderately controlled motivation/introjected regulation indicates that one is motivated by self or others in order to avoid undesirable implicit consequences, including guilt and disapproval of others. Third, moderately autonomous motivation/identified regulation shows that one is motivated by the conscious value of the behavior and is thus willing to accept the responsibility for regulating the behavior. Finally, intrinsic motivation/inherently autonomous motivation represents that one engages in a particular behavior because the behavior is itself fun and interesting (i.e., inherent satisfaction).

Because the RAI can clearly represent the degree of autonomous motivation regarding a particular behavior, it has been applied in various fields (Cockrell & Stone, 2010; Gagné & Deci, 2005). The more autonomous the motivation is to engage in a particular behavior, the higher the RAI is. As a result, integrating the construct of autonomous motivation (i.e., the RAI) into our research model will help to explain the overall quality of motivation in terms of the degree to which one is autonomously motivated (Cockrell & Stone, 2010; Gagné, 2009).

Learners exhibit significantly lower intrinsic and higher extrinsic motivation when learning in a traditional educational environment, as compared to game-based learning situations (Tüzün et al., 2009). Moreover, intrinsic motivation (i.e., inherently autonomous motivation) has a greater influence than does extrinsic (i.e., controlled) motivation, in terms of facilitating a focal individual behavior (Wang & Hou, 2015).

Intrinsic motivation has a positive effect on the continuance intention (Burgers et al., 2015; Ryan et al., 2006). Previous studies have found that intrinsic motivation has a significant effect on learning. SDT posits that intrinsic motivation can be elicited to maintain and enhance learning. In contrast, it can also be decreased or diminished (Ryan & Deci, 2000). Conversely, the more extrinsic motivation is internalized, the more autonomous the learner will be (Gagné & Deci, 2005). Ryan and Deci (2000) claimed that satisfying the basic needs for autonomy, competence, and relatedness of individuals are crucial to motivating them to perform a particular behavior. If these basic needs are satisfied, the intrinsic motivation of the individual will be enhanced.

Learning Effectiveness

Learning effectiveness is defined as the extent to which learners acquire the knowledge, abilities, or skills that they intend to learn (Hu et al., 2007). Learning effectiveness can be influenced by environmental (e.g., features of DGBL systems) and motivational factors (Hu & Hui, 2012; Shakroum et al., 2018). Therefore, learning effectiveness can be considered to be an ultimate learning outcome (Piccoli et al., 2001) and be adopted to evaluate how DGBL-supported learning activities help students achieve their learning goals and to predict their intention to continue to use DGBL systems (Y. M. Huang, 2019; Su & Cheng, 2015).

Kirkpatrick (1998) and Kirkpatrick and Kirkpatrick (2009) proposed four levels of learning effectiveness: the reaction, learning, behavior, and result. Each level presents different information and can be connected and evaluated. First, reaction refers to the learner’s degree of satisfaction with and preference for a particular learning course. In other words, we can understand learners’ thoughts and level of satisfaction with regards to a particular learning purpose, content, and approach. This also represents how learners react emotionally to a particular course. Reaction focuses simply on the actual perceptions of learners. If learners have a positive reaction to a course, they are more likely to learn (Kirkpatrick & Kirkpatrick, 2009).

Second, learning denotes that learners improve their skills, knowledge, and abilities through program training. It also measures the acquisition and understanding of skills and knowledge. Instructors can evaluate learning using tests, interviews, practice, and performance. Kirkpatrick and Kirkpatrick (2009) claimed that it is impossible to expect a change in learners’ real-life behavior, as a result of the improved skills, knowledge, and ability that are achieved through participating in learning activities if learners cannot learn what they expect to learn in an effective way.

Third, behavior refers to the extent to which learners’ behavior is changed after participating in the educational activities. By the end of the training program in academic educational contexts, the behavior may reflect the degree to which learners have internalized what is learned and apply the knowledge and skills learned to real-world situations.

Finally, results refer to the final outcomes that take place because the learners attended the educational program. Final outcomes are relevant to the level of improvement of learners’ understanding of the focal learning subjects, which is the main reason for attending the educational program.

Research Model and Hypotheses

The DGBL System Developed

To achieve our research purposes, a DGBL system that focused on instructing the concept of normalization of an electronic database management course was developed. This system was composed of three primary modules. First, the normalization instruction module includes a description of the rules and goals of the system and instructional messages of normalization concepts (as the representation of feed forward). Second, after reviewing the information offered in the first module, the users begin the procedures of the gaming module, in which they were asked to complete some tests regarding the concepts of normalization using the gaming components, which resembled the famous video game of Super Mario Bros developed by Nintendo Corporation in the 1980s. The game was designed by referring to the Super Mario Bros for the purpose of including some important game mechanisms (e.g., role play, rewards, story, challenge, and competition) that students were familiar with in order to make them consider the game to be fun (Proulx et al., 2017). Consequently, students were more likely to think of the participation in the DGBL activities designed for the current study to be relevant to the fulfillment of their psychological needs for autonomy, competence, and relatedness that can significantly influence their autonomous motivation, as specified by SDT (Proulx et al., 2017). The gaming module also included an automatic evaluation mechanism that determined the users’ test scores (as the representation of outcome feedback). In addition to presenting the test score to the users, the automatic evaluation mechanism provided the users personalized instructions of the normalization concepts based on the errors that they made in the test (as the representation of cognitive feedback). The users who got a score lower than 60 would be asked to take the test again after reviewing the information feedback offered by the module. Finally, the scoring summary and ranking module presented the scores of all the users with rankings for the current users’ references to allow them to evaluate how well they learned about normalization compared to their peers. The users were then asked to complete the survey for the current study at the end of the game. Figures 1 presents the process flow diagram of the DGBL system, and Figures 2 to 5 illustrate some snapshots of the game that we developed.

Figure 1.

Process Flow of the Developed DGBL System.

Figure 2.

Snapshot of Feed Forward.

Figure 3.

Snapshot of Test Using Gaming Components.

Figure 4.

Snapshot of Cognitive Feedback.

Figure 5.

Snapshot of Outcome Feedback.

The Research Model

To explore the answers to the research questions presented above, we employed the concepts of information feedback, game quality, and self-determination theory. These were used to construct the research model presented in Figure 6. In our research model, the constructs of information feedback and game quality are treated as the direct antecedents to both learning effectiveness and students’ autonomous motivation (i.e., the RAI) with regards to using DGBL systems. Additionally, we assumed that students’ autonomous motivation would have a direct and positive effect on their learning effectiveness. We also considered learning effectiveness and students’ autonomous motivation to be critical factors directly influencing students’ intention to continue to use DGBL systems for their learning.

Figure 6.

The Research Model of DGBL Systems.

Boyle et al. (2011) claim that when learners are provided with immediate feedback in DGBL systems, they learn more effectively. In addition, Burgers et al. (2015) found that feedback increases learners’ motivation and satisfaction and has a positive effect on learning effectiveness. Hattie and Timperley (2007) argue that feedback facilitates learning. If teachers provide feedback, it reduces the gap between actual performance and perfect attainment. It is also one of the key factors facilitating learning effectiveness.

Prior studies have shown that feedback plays an important role in learning effectiveness. Not providing learners with appropriate information feedback has a negative effect on learning effectiveness. Moreno (2004) argued that if we provide learners with cognitive feedback, they are more likely to engage, promoting deeper learning than if they were only provided with outcome feedback. Law and Chen (2016) demonstrated that both outcome and cognitive feedback increase learners’ learning effectiveness. Athanasopoulos and Hyndman (2011) found that outcome feedback improved performance. Therefore, we develop the following hypothesis:

H1a. Information feedback has a positive effect on learning effectiveness.

Ryan and Deci (2000) claimed that enhancing intrinsic motivation has a positive effect on autonomy. In other words, it maintains intrinsic motivation, causing learners to feel satisfied. Providing learners with feedback makes them feel more confident, eliciting a sense of competence and enhancing their intrinsic motivation. Mekler et al. (2017) argued that providing feedback satisfies learners’ need for competence; Akbari et al. (2015) found that providing feedback increases autonomy. Burgers et al. (2015) determined that positive feedback satisfies a need for autonomy and sense of competence, and may have a positive effect on learners’ intrinsic motivation. Therefore, we develop the following hypothesis:

H1b. Information feedback has a positive effect on the RAI.

We intend to focus on the quality of the game design. The literature has demonstrated that game quality has a positive effect on learning effectiveness. Osman and Bakar (2012) found that game design impacts learners’ performance. Jung et al. (2010) argue that if learners are provided with three game elements – goals, competition, and challenge – while conducting their tasks, they will have better learning performance. Santhanam et al. (2016) showed that competition has a significant effect on learning performance. Tan et al. (2016) found that clear goals, a sense of control, and a challenging exchange are crucial to game design. Well-designed e-learning tools, such as serious games, enable learners to achieve better learning effectiveness (Ding & Er, 2018). Therefore, we develop the following hypothesis:

H2a. Game quality has a positive effect on learning effectiveness.

Ryan and Deci (2000) showed that enhancing intrinsic motivation has a positive effect on autonomy, competence, and relatedness. In other words, when learners are satisfied, they maintain intrinsic motivation. Moreover, well-designed game elements can enhance learners’ sense of autonomy, competence, and relatedness, thus improving their intrinsic motivation (Pe-Than et al., 2014). Autonomy is a feeling of power in which learners believe they have the ability to make decisions for themselves (Ryan & Deci, 2000). Learners’ perception of autonomy increases when control is added. Jung et al. (2010) argued that providing learners with digital games that are of high quality in terms of the four game elements, including rule/goal, competition, challenge, and control, while they are conducting learning tasks can satisfy their need for autonomy and competence. Therefore, we develop the following hypothesis:

H2b. Game quality has a positive effect on the RAI.

Research indicates that learners in a DGBL environment tend to have higher levels of autonomous motivation and better learning performance (Barzilai & Blau, 2014; Chen & Law, 2016). Consequently, autonomous motivation is crucial to facilitating learning effectiveness. Ryan and Deci (2000) have argued that enhancing autonomous motivation has a positive effect on autonomy, competence, and relatedness. In other words, when learners feel satisfied, their autonomous motivation is enhanced. Autonomous motivation encourages learners’ to learn actively, which, in turn, leads to an increase in their learning effectiveness. Therefore, we develop the following hypothesis:

H3a. The RAI has a positive effect on learning effectiveness.

Prior studies have implied that intrinsic motivation has a positive effect on the intention to continue to use DGBL systems (Burgers et al., 2015; Ryan et al., 2006). If learners’ intrinsic motivation is enhanced, their overall autonomous motivation will increase. In addition, this will have an impact on their RAI. Intrinsic motivation is a determinant of the intention to continue to use in DGBL systems. In other words, if learners have high levels of intrinsic motivation, they are more likely to perform autonomously. This has a positive effect on their intention to continue using the DGBL systems. Therefore, we develop the following hypothesis:

H3b. The RAI has a positive effect on the intention to continue to use DGBL systems.

Chiu and Wang (2008) have shown that performance expectancy has a positive effect on the intention to continue learning. Mohammadyari and Singh (2015) have argued that we can predict individuals’ intention to continue to use DGBL systems by establishing a level of learning effectiveness. Learning effectiveness has a positive effect on learners’ intention to continue using the DGBL systems. When learners receive more from their system, they are more likely to use that system again. Therefore, we develop the following hypothesis:

H4. Learning effectiveness has a positive effect on the intention to continue to use DGBL systems.

Research Methods

Development of Measures

From the existing literature, we identified 67 items for the 16 first-order constructs of the research model, and we refined these items to make them fit within the specific context of this study. We piloted all of the survey items with 35 individuals who were undergraduate or graduate students in a university and did not have any experience with database management-related courses. We then examined the internal consistency and reliability of the survey items via a Cronbach’s alpha coefficient analysis, concluding that the survey items included in the final questionnaire (see Appendix) were highly reliable; the individual Cronbach’s alpha coefficients for all of the first-order constructs were greater than the recommended level of 0.7 (ranging from 0.71 to 0.93). A seven-point Likert scale ranging from strongly disagree (1) to strongly agree (7) was used to measure the survey items.

Data Collection

To solicit a pool of respondents who would be as close as possible to the general population of potential survey respondents, the invitations to participate included the URL of the online questionnaire and were posted on the Facebook student groups of eight universities located in the southern region of Taiwan. These were used to invite college students who did not have experience in taking database management-related courses to participate in the survey. The survey respondents were entered into a lottery for 100 gift certificates for a chain of convenience stores; these served as prizes to increase the response rate. Respondents were asked to use the DGBL system that we developed and evaluate the survey questions based on their experience using the system. A total of 433 responses were eventually received. We later discarded 50 responses, mainly due to the respondents’ failure to properly answer the reverse questions included in the survey or obvious systematic answers. This process eventually yielded 383 valid responses, for a valid return rate of 88.45%.

To evaluate the impact of potential non-response bias on the collected data, independent sample t tests were used to compare the profiles of early and late respondents in terms of gender, age, perceived usefulness of database management courses, and level of relevance of the respondents’ academic major to the study of database management. The results indicated that there were no statistically significant differences between these two data sets (with 228 and 155 respondents, respectively) in terms of gender (p = 0.20), age (p = 0.32), perceived usefulness of database management courses (p = 0.43), and level of relevance (p = 0.48). Thus, we believe that there was no serious non-response bias. We also used independent sample t tests to test the homogeneity of the first-order constructs included in our research model, with regards to the level of relevance of the respondents’ academic majors to database management courses. The results showed that there were no significant differences in terms of most of the constructs of interest, including game quality (p = 0.79), information feedback (p = 0.47), learning effectiveness (p = 0.64), and the RAI (p = 0.36). Given these results, we determined that it was appropriate to use the 383 valid survey responses as a single dataset in this study.

Among our respondents, 48.8% were females, and more than 89% were under 26 years of age. In addition, more than 75% of our respondents believed that acquiring knowledge of database management was important, considering their individual university majors.

Data Analysis of the Measurement Model

To examine the multiple relationships among the constructs hypothesized in our theoretical model, the technique of component-based structural equation modeling, which is also referred to as the partial least squares (PLS) method, was adopted for data analysis in this study. The SPSS 20.0 and SmartPLS 3.3 software packages were used to perform our data analysis procedures.

We first examined the measurement model for our research framework, checking the reliability of the constructs of interest by evaluating their individual Cronbach’s alpha coefficients. The convergent and discriminant validity were also verified based on the criteria suggested in prior studies (e.g., Fornell & Larcker, 1981; Hair et al., 2010). An initial examination of the measurement model indicated an unsatisfactory measurement, with the problems mostly related to the low factor loadings of individual measures or constructs of interest failing to pass an examination of their discriminant validity. To resolve those problems, we discarded seven measures: GQCha1, IFC2, LER4, LER5, LEL4, LEB4, and LERes4. We then examined the adequacy of the measurement model based on the remaining measures.

The results indicated that all of the constructs of interest had an appropriate level of reliability because all of their individual Cronbach’s alpha coefficients were greater than the recommended level of 0.7 (see Table 2). Additionally, all of the factor loadings of the remaining items were statistically significant and higher than the recommended 0.6. The values of the composite reliability (CR) statistics were also greater than the recommended 0.6. Finally, the average variance extracted (AVE) estimates for all of the constructs were greater than the recommended 0.5. Thus, we concluded that the measurement model had an adequate level of convergent validity (see Table 2).

Table 2.

Convergent Validity of the Measurement Model.

Construct	Indicator	Factor loading^a	Cronbach’s alpha	Composite reliability (CR)	Average variance extracted (AVE)
Game quality – rule/goal (GQR)	GQR1	0.78	0.83	0.88	0.60
	GQR2	0.83
	GQR3	0.67
	GQR4	0.80
	GQR5	0.77
Game quality – competition (GQCom)	GQCom1	0.87	0.90	0.93	0.77
	GQCom2	0.88
	GQCom3	0.88
	GQCom4	0.90
Game quality – challenge (GQCha)	GQCha2	0.74	0.80	0.87	0.63
	GQCha3	0.82
	GQCha4	0.83
	GQCha5	0.77
Game quality – control (GQCon)	GQCon1	0.84	0.73	0.85	0.65
	GQCon2	0.69
	GQCon3	0.70
Information feedback – cognitive feedback (IFC)	IFC1	0.77	0.80	0.87	0.64
	IFC3	0.66
	IFC4	0.87
	IFC5	0.87
Information feedback – feed forward (IFF)	IFF1	0.86	0.73	0.83	0.56
	IFF2	0.62
	IFF3	0.83
	IFF4	0.65
Information feedback – outcome feedback (IFO)	IFO1	0.75	0.72	0.84	0.65
	IFO2	0.87
	IFO3	0.79
Learning effectiveness – reaction (LER)	LER1	0.88	0.84	0.91	0.76
	LER2	0.89
	LER3	0.85
Learning effectiveness – learning (LEL)	LEL1	0.89	0.92	0.95	0.86
	LEL2	0.92
	LEL3	0.92
Learning effectiveness – behavior (LEB)	LEB1	0.86	0.91	0.94	0.80
	LEB2	0.93
	LEB3	0.91
	LEB5	0.87
Learning effectiveness – result (LERes)	LERes1	0.92	0.91	0.94	0.80
	LERes2	0.84
	LERes3	0.90
	LERes5	0.91
RAI – external motivation (RE)	RE1	0.83	0.78	0.86	0.61
	RE2	0.87
	RE3	0.73
	RE4	0.66
RAI – introjected regulation (RInt)	RInt1	0.83	0.89	0.93	0.76
	RInt2	0.87
	RInt3	0.90
	RInt4	0.87
RAI – identified regulation (RIde)	Ride1	0.84	0.89	0.92	0.75
	Ride2	0.89
	Ride3	0.88
	Ride4	0.84
RAI – intrinsic motivation (RIn)	RIn1	0.88	0.95	0.96	0.86
	RIn2	0.995
	RIn3	0.95
	RIn4	0.93
Continuance intention	CI1	0.93	0.85	0.91	0.77
	CI2	0.93
	CI3	0.76

^aAll factor loadings of the individual items are statistically significant (p < 0.01).

The discriminant validity of the measurement model was verified by comparing the AVE estimates of the constructs to the squared correlations among those constructs. The results presented in Table 3 indicated that the squared correlations were smaller than the corresponding AVE statistics, meaning that the constructs were more closely related to their own measures than to those of the other constructs (Fornell & Larcker, 1981). We thus concluded that the measures used exhibited adequate discriminant validity.

Table 3.

Discriminant Validity for the Measurement Model.

Construct	1.	2	3	4.	5.	6.	7.	8.	9.	10.	11.	12.	13.	14.	15.	16.
1. GQR	0.60
2. GQCom	0.23	0.77
3. GQCha	0.52	0.28	0.63
4. GQCon	0.33	0.17	0.26	0.65
5. IFC	0.35	0.10	0.24	0.15	0.64
6. IFF	0.34	0.12	0.26	0.17	0.43	0.56
7. IFO	0.27	0.08	0.21	0.12	0.41	0.52	0.65
8. LER	0.49	0.24	0.43	0.28	0.34	0.41	0.34	0.76
9. LEL	0.38	0.21	0.31	0.22	0.33	0.34	0.23	0.49	0.86
10. LEB	0.44	0.23	0.40	0.26	0.29	0.41	0.31	0.55	0.54	0.80
11. LERes	0.42	0.22	0.39	0.24	0.28	0.43	0.33	0.54	0.51	0.75	0.80
12. RE	0.21	0.11	0.21	0.11	0.04	0.05	0.04	0.20	0.12	0.18	0.16	0.61
13. RInt	0.06	0.06	0.08	0.04	0.01	0.01	0.01	0.09	0.03	0.09	0.07	0.53	0.76
14. RIde	0.32	0.17	0.27	0.17	0.19	0.19	0.19	0.32	0.23	0.32	0.32	0.28	0.18	0.75
15. RIn	0.36	0.32	0.44	0.24	0.22	0.32	0.25	0.50	0.47	0.59	0.54	0.24	0.13	0.35	0.86
16. CI	0.30	0.19	0.31	0.17	0.26	0.22	0.18	0.41	0.49	0.50	0.47	0.18	0.25	0.10	0.46	0.77

Note. Diagonals represent the AVEs, and the other matrix entries represent the squared factor correlations. Boldface highlights the diagonals.

The constructs of game quality, information feedback, and learning effectiveness were modeled as second-order formative constructs, formed as the weighted sum of their individual first-order reflective constructs. Consequently, an examination of the weights in the principal component analysis was recommended (Diamantopoulos & Winklhofer, 2001; Petter et al., 2007). The results of this analysis showed that the weights were all significant (see Table 4). Moreover, the correlations among all first-order reflective constructs (ranging from 0.17 to 0.86) were smaller than the cutoff value of 0.9, indicating that substantial collinearity was not present (Hair et al., 2010). Furthermore, because excessive multicollinearity in the formative constructs might destabilize the model, a variance inflation factor (VIF) test was used to examine whether the indicators for the construct of trust exhibited significant multicollinearity (Hair et al., 2010; Petter et al., 2007). The results indicated that the VIFs for the first-order indicators of game quality, information feedback, and learning effectiveness were smaller than the cutoff value of 5 (Hair et al., 2010), as presented in Table 4. Thus, it was determined that high multicollinearity was not an issue. Furthermore, the content validity levels of the constructs of game quality, information feedback, and learning effectiveness were all assessed. As shown in Table 4, the magnitude of the error terms of the first-order indicators of these three second-order formative constructs were small, and all of the indicator coefficients were significant, demonstrating that these three second-order constructs were well described by their individual first-order indicators, and no further action was required (Diamantopoulos, 2006; Petter et al., 2007). Table 5 presents the descriptive statistics for each of the constructs in our research model.

Table 4.

Weight and VIF of Formative Indicators.

2nd order construct	1st order construct	VIF	Standard error	Weight (t value)
Game quality (GQ)	Rule/Goal	2.29	0.01	0.31 (29.11)
	Competition	1.45	0.01	0.25 (29.68)
	Challenge	2.25	0.01	0.25 (31.77)
	Control	1.50	0.01	0.19 (24.07)
Information feedback (IF)	Cognitive feedback	1.94	0.01	0.36 (33.47)
	Feed Forward	2.37	0.01	0.35 (37.30)
	Outcome feedback	2.28	0.01	0.27 (31.87)
Learning effectiveness (LE)	Reaction	2.60	0.01	0.23 (43.10)
	Learning	2.52	0.01	0.20 (35.53)
	Behavior	4.62	0.01	0.28 (40.79)
	Result	4.34	0.01	0.28 (38.98)

Table 5.

Descriptive Statistics of the Constructs of Interest.

Construct*	Mean	Standard deviation
Game quality	5.58	0.77
Information feedback	5.36	0.79
Learning effectiveness	5.52	0.88
RAI – external motivation	3.85	1.03
RAI - introjected regulation	2.93	1.18
RAI – identified regulation	4.88	1.12
RAI – intrinsic motivation	5.43	1.12
Continuance intention	5.18	1.13

*A total of 60 items.

Structural Model and Hypotheses Testing

The PLS technique was used to examine the structural model. To evaluate its quality, we checked the goodness-of-fit by adopting the following examinations. We first evaluated the standardized root mean square residual (SRMR) of the structural model in order to assess the potential model misspecification issue. We found the SRMR of our structural model to be 0.08, which was not greater than the recommended cutoff of 0.08. In addition, as indicated in Figure 7, the explained variance (R²) values for our three endogenous variables were equal to or greater than 0.25, except for that of autonomous motivation (i.e., the RAI), which was 0.21. This indicated that our structural model exhibited moderate predictive accuracy. Finally, we checked the Stone-Geisser Q² values for the constructs of autonomous motivation (i.e., the RAI), learning effectiveness, and continuance intention, in order to evaluate the predictive relevance of the proposed path model to these endogenous constructs. Hair et al. (2013) suggested that Q² values of 0.02, 0.15, and 0.35 should be considered, respectively, small, medium, and large levels of predictive relevance for an endogenous construct. The Q² values for our three endogenous constructs were 0.20 (autonomous motivation), 0.68 (learning effectiveness), and 0.56 (continuance intention). We thus determined that our path model had medium to large predictive relevance to the endogenous constructs of interest. In conclusion, the results of the above goodness-of-fit assessment indicate an adequate fit for our structural model.

Figure 7.

Results of Hypotheses Testing of the Research Model.

The structural model for this study was considered adequate. Thus, we tested our hypotheses via a bootstrapping procedure. The standardized path coefficients and t values, significance of the paths, and coefficients of determination (R²) for each endogenous construct in the research model are summarized in Figure 6.

Hypotheses H1a and H1b were supported. These results indicate that information feedback had a significant and directly positive effect on DGBL system users’ autonomous motivation and learning effectiveness. Hypothesis H2a was supported, but Hypothesis H2b was not. This means that game quality had a significant and directly positive effect on DGBL system users’ learning effectiveness, but not on their autonomous motivation. Hypothesis H3a was supported, but Hypothesis H3b was rejected. These results indicated that DGBL system users’ autonomous motivation had a directly positive effect on their learning effectiveness, but not on their intention to continue to use DGBL systems. Finally, hypothesis H4 was supported, indicating that DGBL system users’ learning effectiveness had a directly positive effect on users’ intention to continue to employ DGBL systems to support their learning activities, as was expected. These findings also showed that information feedback, game quality, and autonomous motivation positively influenced DGBL system users’ intention to continue using DGBL, via mediation of learning effectiveness. Our research results also indicated that the independent variables explained a significant portion of the variance in the three dependent variables. The independent variables in the proposed research model explained 21% of the variance in DGBL system users’ autonomous motivation, 70% in learning effectiveness, and 58% in DGBL continuance intention.

To avoid the potential presence of common method bias, we randomly arranged our individual dependent and independent variable items in our research model, and intentionally presented the measures of the dependent variables behind (rather than in front of) those of the independent variables (Podsakoff et al., 2003). In addition, we performed a Harman’s single-factor analysis procedure to statistically examine the magnitude of common method variance (CMV) among our first-order constructs. The data for all of the 60 remaining survey items (after the previously-described item deletion process) were included in an exploratory factor analysis using an unrelated factor solution. We extracted multiple factors from this analysis, accounting for 68.92% of the variance in the data. Moreover, there was no single factor that accounted for the majority of the covariance. The first factor extracted from this procedure accounted for 39.38% of the variance in the data. We thus determined that severe common method bias was unlikely to be present, based on the Harman’s single-factor test.

Discussion

First, we found that information feedback has an effect on autonomous motivation (i.e., the RAI) and learning effectiveness. These results are consistent with those of Akbari et al. (2015), Burgers et al. (2015) and Law and Chen (2016). In other words, when the information feedback provided by the DGBL systems can fulfill learners’ basic psychological needs, their learning effectiveness can be enhanced as a result of the use of DGBL systems. Consequently, developers of DGBL systems can provide learners with appropriate feedback based on their performance during different processes of DGBL-enabled learning activities. Those information feedbacks can help the learners better understand their learning goals, their own progress on achieving this goal, and understand how to bridge the gap between the current and the desired learning outcomes. This can then satisfy their basic psychological needs, which, in turn, contribute to the increase in their learning effectiveness.

Second, we found that game quality has a significant positive effect on learners’ learning effectiveness, but not on their autonomous motivation. These results are consistent with those of Santhanam et al. (2016) and Tan et al. (2016), but are not identical to those of Jung et al. (2010) and Pe-Than et al. (2014). These findings indicate that if we include critical game elements, including rules/goals, competition, challenge, and control in the DGBL systems for learners, the learners are more likely to actively engage in learning activities supported by the DGBL systems. In other words, DGBL systems that have a high level of game quality can provide learners with a clear direction of learning and the associated information for learning, and the learners are thus more likely to sense a strong association between their learning objectives and the DGBL systems, and then learn more effectively. Additionally, a plausible explanation of the insignificant effect of game quality on the RAI is that DGBL systems with a high degree of game quality can make learners consider it useful to use the systems and then to encourage them to engage actively in the programmed learning procedures. However, those systems might not be able to autonomously motivate the learners regarding studying the focal learning subject if they are not developed in a way that makes learners to perceive using the systems to learn the focal learning subjects to be inherently interesting and fun. To conclude, developers of game-based learning system can pay attention to these key game-quality elements to design DGBL systems to satisfy the basic psychological needs of learners, which, in turn, can improve learning effectiveness as well.

Third, according to our results, autonomous motivation has a direct effect on learning effectiveness, which is consistent with the propositions of Tüzün et al. (2009), Barzilai and Blau (2014), and Chen and Law (2016). In other words, while learners are autonomously motivated to use DGBL systems, they are more likely to perceive a higher degree of their learning effectiveness. Therefore, the developers of DGBL systems can design system procedures that can satisfy learners’ psychological needs to enhance their autonomous motivation to use the systems, which, in turn, can contribute to the improvement of learners’ perceived learning effectiveness.

Finally, we found that learning effectiveness has a significant positive effect on the continuance intention regarding DGBL systems, which is consistent with the results of Chiu and Wang (2008) and Mohammadyari and Singh (2015). If learners receive benefits from DGBL systems, they are more likely to continue to use the systems to support their learning efforts. On the other hand, the results showed that autonomous motivation does not have a direct significant effect on the continuance intention regarding using DGBL systems. This implies that because the primary objective of DGBL system users is to acquire critical knowledge and skills from the courses offered by DGBL systems, the key to encourage them to continue to use the DGBL systems is the improvement in their learning effectiveness. In other words, when learners perceive that they cannot obtain the expected learning effectiveness by using the DGBL systems, they will not continue to use the systems, even if they find the use of the DGBL systems to be enjoyable and inherently fun. In summary, we found that learning effectiveness is the key determinant of the learners’ intention to continue to use DGBL systems.

Conclusion

Implications for Theory

The first primary theoretical implication of this study is that, to the best of our knowledge, this study is among the first group of studies of DGBL systems that applied the construct of autonomous (relative to controlled) motivation (using the RAI as its surrogate) to the investigation of the effects of DGBL systems on students’ learning effectiveness. Investigating the RAI enables us to encompass the controlled-to-autonomous continuum of individuals’ motivation, and thus allows us to account for overall motivation quality which can enhance the prediction of students’ learning effectiveness and their behavioral intention to continue to use DGBL systems to support their learning initiatives (Wang, 2016). Our model explains 70% of the variance of students’ learning effectiveness and 58% of the variance of students’ intention to continue to use DGBL systems, showing support for the significance of adopting autonomous motivation proposed by SDT to investigate the key determinants of students’ learning performance in the context of DGBL systems.

The second implication is that we identified information feedback as a prerequisite for promoting a high degree of students’ autonomous motivation (i.e., the RAI) regarding using DGBL systems to enhance their learning effectiveness, while game quality was directly associated with learning effectiveness but not with autonomous motivation. The confirmation of the causal paths among information feedback, game quality, autonomous motivation and learning effectiveness implies that information feedback is an essential ingredient for enhancing learning effectiveness and continuance intention regarding DGBL systems, because it satisfies students’ needs for autonomy and competence to better learn the knowledge of interest. Such a perception will not only lead the students to actively engage in the learning procedures enabled by DGBL systems, but it will also raise their degree of autonomous motivation (i.e., the RAI) to work with DGBL systems in order to achieve the predefined learning goals. These findings can provide researchers with insights into developing a structured research map for further investigation of the relationships among various kinds of information feedback, various levels of autonomous motivations, and students’ learning effectiveness in diverse e-learning contexts.

Finally, although there have been studies that have adopted SDT to investigate students’ e-learning performance and behaviors, they mostly focus on primary or secondary school students or professional workers, rather than students of higher education (Burgers et al., 2015; Chen & Law, 2016; Roca & Gagné, 2008; Tüzün et al., 2009). In this study, the confirmation of autonomous motivation’s direct positive impact on learning effectiveness and indirect positive impact on continuance intention regarding using DGBL systems has extended the application of SDT to studies that are related to game-based learning in the context of higher education.

Implications for Practice

While we devote efforts into developing high-quality DGBL systems, one of the most important things to keep in mind is whether the systems developed can significantly enhance users’ learning effectiveness. Additionally, prior studies argue that if learners exhibit a higher degree of autonomous motivation, they are more likely to actively engage in the learning procedures offered by DGBL systems in order to enhance their learning effectiveness (Barzilai & Blau, 2014; Chen & Law, 2016). Consequently, learners can be encouraged to continue using DGBL systems because of the perception of the positive association between system use and the enhancement of learning effectiveness. Therefore, to encourage learners to be autonomously motivated to learn via using DGBL systems for achieving desirable learning effectiveness, developers of the DGBL systems must pay attention to the design of the messages conveyed in the gaming processes, such as messages concerning feed forward (e.g., gaming instructions, learning purposes, and introduction of focal learning subjects) and cognitive feedback (e.g., explanations, hints, and instructions related to students’ errors identified during the gaming processes). Guiding students to comprehend the learning materials through well-organized and personalized information feedbacks can enhance students’ inherent autonomous motivation to learn and allow them to have timely and useful information to correct discrepancies between their understanding of the focal learning subjects and what the subjects are truly about. This can, in turn, enhance students’ learning effectiveness and their intention to continue using the systems for learning purposes in the future.

It is also important that the developers of DGBL systems ensure the quality of the systems by incorporating the four critical elements of game quality investigated in this study in their system design efforts, including, rule/goal, competition, challenge, and control. Developers of DGBL systems should make the rules and goals of those DGBL systems clear to the users at the beginning of the game, and offer users with certain degree of control over the progress of the game. Additionally, they can enrich the elements of challenging DGBL systems by including functions that motivate users to think deeper on how the knowledge learned can be applied to the tasks assigned by DGBL systems (e.g., in-game exercises that are designed based on real-world cases). Furthermore, they can incorporate the element of competition of DGBL systems by designing functions that allow users to be aware of the progress of other users while playing the game (e.g., real-time contest) to create an environment for productive competition. The measures of the DGBL system development discussed above can satisfy users’ psychological needs for autonomy (having control over the game), competence (handling challenges well), and relatedness (competing with others in harmony to build positive relationships), and eventually lead to better learning effectiveness, which, in turn, increase the users’ intention to continue to use DGBL systems in the future.

Limitations and Future Research Directions

This study has a number of limitations. First, this study focuses on a single course, namely database management. The research results may not be applicable to other courses. Future research projects can be conducted by extending our research model to further investigate the effects of DGBL systems on the student learning effectiveness regarding other academic subjects/courses.

Additionally, this study adopted a quasi-experimental design to investigate how the use of DGBL systems may affect students’ learning effectiveness and whether or not the influences of DGBL systems on students’ learning effectiveness can be attributed to the associations among the constructs of interest that we hypothesized in the contexts of higher education. The contribution of the current study may be limited in terms of providing insights into whether the application of DGBL systems made a significant difference in students’ learning effectiveness compared to the traditional lecture-based learning or some other learning methods. Future research projects that focus on investigating the indicated issue by adopting an experimental-design research are encouraged.

Furthermore, while a number of studies investigate issues that are related to game players’ motivation and gaming experience of non-educational digital games from the perspectives of the types of players (e.g., Ryan et al., 2006; Yee, 2006), which is beyond the scope of the current study. Future studies can focus on similar research topics by adopting frameworks for categorizing game players, such as Bartle’s (1996) taxonomy of game players. Finally, although we took dimensions of game quality into considerations when developing our measures for this construct, we did not exhaust all game-quality factors. For example, Garris et al. (2002) propose six primary categories of characteristics that are critical to the determination of game quality, which include fantasy, rule/goal, challenge, control, sensory stimuli, and mystery. Future studies can take into considerations other game-quality elements to investigate the effectiveness of DGBL systems.

Appendix.
List of Survey Items by the First-order Constructs Adopted.

Information Feedback – Cognitive Feedback (IFC) (Maier et al., 2016)

IFC1: I read the feedback in detail.

IFC2: I was only interested if my answer was right or wrong. (discarded)

IFC3: I am not interested in the feedback. (reverse coded)

IFC4: Further information in the feedback was good.

IFC5: The feedback was very helpful.

Information Feedback – Feed Forward (IFF) (Self-developed with reference to Faber et al., 2017)

IFF1: Information that is related to the subject of normalization provided by the DGBL system before the test was helpful.

IFF2: Instructions of normalization provided by the DGBL system before the test was not helpful. (reverse coded)

IFF3: Instructions of normalization provided by the DGBL system before the test help me develop a basic understanding of the subject of normalization.

IFF4: The information that is related to the subject of normalization provided by the DGBL system before the test made me aware of which parts of the concept of normalization of which I should improve my understanding.

Information Feedback – Outcome Feedback (IFO) (Self-developed with reference to Faber et al., 2017)

IFO1: The feedback provided by the DGBL system based on my test results did not make me better understanding the subject of normalization. (reverse coded)

IFO2: The feedback provided by the DGBL system based on my test results made me better comprehend the subject of normalization.

IFO3: The feedback provided by the DGBL system based on my test results made me aware of which parts of the subject of normalization of which I should improve my understanding.

Game Quality – Rule/Goal (GQR) (Tan et al., 2016)

GQR1: The learning objectives and goals of the DGBL system were clear to me when I used it.

GQR2: The DGBL system helped me link new knowledge and skills with the subject of normalization.

GQR3: I found the contents of the DGBL system relevant to the things I encounter in my life.

GQR4: I learned new concepts and skills from the DGBL system.

GQR5: I understand the instructions and the contents of the DGBL system.

Game Quality – Competition (GQCom) (Demetrovics et al., 2011)

GQCom1: I can compete my knowledge with others’ when using the DGBL system.

GQCom2: I like to win when using the DGBL system.

GQCom3: It is important for me to complete the test with a score that is higher than others’ when using the DGBL system.

GQCom4: When using the DGBL system, I feel pleasure if my test score ranking is high.

Game Quality – Challenge (GQCha) (Merikivi et al., 2017)

GQCha1: Using the DGBL system challenges me. (discarded)

GQCha2: Using the DGBL system provides a good test of my skills.

GQCha3: Using the DGBL system challenges me to perform the best of my ability.

GQCha4: Using the DGBL system stretches my capabilities to the limits.

GQCha5: Using the DGBL system makes me think.

Game Quality – Control (GQCon) (Kim & Shute, 2015)

GQCon1: It was easy to learn the DGBL system controls.

GQCon2: The controls of the DGBL system were intuitive.

GQCon3: It was easy to remember the game controls when I wanted to do something in the DGBL system.

Learning Effectiveness – Reaction (LER) (Chrysafiadi & Virvou, 2013)

LER1: The DGBL system meets my expectation.

LER2: The DGBL system helps me understand the logic of database normalization.

LER3: I think that the DGBL system is useful as an educational tool.

LER4: I think that the use of the DGBL system is a waste of time. (discarded)

LER5: I understand the concept of database normalization after using the DGBL system. (discarded)

Learning Effectiveness – Learning (LEL) (C. F. Huang et al., 2015)

LEL1: I think DGBL system can make the course more interesting.

LEL2: I think DGBL system is worth trying.

LEL3: I think DGBL system is meaningful.

LEL4: I think everyone can complete his or her own tasks on DGBL system if he or she studies hard. (discarded)

Learning Effectiveness –Behavior (LEB) (Chrysafiadi & Virvou, 2013)

LEB1: The DGBL system affects positively my perception about database management.

LEB2: The DGBL system draws my interest on database management.

LEB3: The DGBL system motivates me to be involved in database management.

LEB4: The DGBL system helps me understand the subject of database management. (discarded)

LEB5: The DGBL system motivates me to use other DGBL systems.

Learning Effectiveness – Result (LERes) (Chrysafiadi & Virvou, 2013)

LERes1: The DGBL system makes me understand better the logic of database normalization.

LERes2: The DGBL system helps me to learn other database-related courses.

LERes3: The DGBL system helps me in my studies.

LERes4: The DGBL system helps me understand better other lessons of database management. (discarded)

LERes5: The DGBL system helps me in the elaboration of tasks and activities considering your studies of database management.

RAI – External Motivation (RE) (Cockrell & Stone, 2010; Ryan & Connell, 1989)

I use the DGBL system …

RE1: Because I will get in trouble if I do not.

RE2: Because it is what I am supposed to do.

RE3: Because I will get a reward for doing so.

RE4: Because It would harm my relationships with others if I do not.

RAI – Introjected Regulation (RInt) (Cockrell & Stone, 2010; Ryan & Connell, 1989)

I use the DGBL system …

RInt1: Because I want the instructors to think I am a good student.

RInt2: Because I feel bad about myself if I do not.

RInt3: Because I want others to like me.

RInt4: Because it bothers when I do not.

RAI – Identified Regulation (RIde) (Cockrell & Stone, 2010; Ryan & Connell, 1989)

I use the DGBL system …

RIde1: Because I want to know what others are doing.

RIde2: Because it is important to me to share what I learned with others

RIde3: Because I think I can offer others information that can help them.

RIde4: Because I believe it is an important personal attribute to share what I know with others.

RAI – Intrinsic Motivation (RIn) (Cockrell & Stone, 2010; Ryan & Connell, 1989)

I use the DGBL system …

RIn1: Because it is fun.

RIn2: Because I enjoy doing it.

RIn3: Because I feel happy when doing it.

RIn4: Because it is satisfying to do it.

Continuance Intention (CI) (Mohammadyari & Singh, 2015)

CI1: I intend to continue using DGBL systems, rather than discontinue their use.

CI2: My intentions are to continue using DGBL systems for my learning than use any alternative means.

CI3: If I could, I would like to discontinue my use of DGBL systems.

Footnotes

Acknowledgments

The authors thank the survey respondents for providing valuable data. Additionally, the authors thank the Editor and anonymous reviewers for their valuable feedback on this paper.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Ministry of Science and Technology, Taiwan (grant number: MOST 105-2511-S-006-001-MY2).

Author Biographies

Yi Chun Liu is currently an associate professor in the Department of Applied Foreign Languages at Chia Nan University of Pharmacy & Science, Tainan, Taiwan. She received her PhD in Teaching English as a Second Language from Texas A&M University, USA. Her research interests include Computer-Assisted Language Learning, E-learning, and English for Specific Purposes (ESP).

Wei-Tsong Wang received his PhD in Information Science from the State University of New York at Albany, USA. His current research interests include e-learning, game-based learning, behaviors of e-commerce consumers, user acceptance on information technology, and knowledge management. His works have appeared/are to be appeared in journals such as Computers and Education, Journal of Educational Technology and Society, Information and Management, Decision Sciences, Decision Support Systems, Journal of Organizational Computing and Electronic Commerce, Information Systems Frontiers, and Journal of Organizational and End User Computing, among others.

Tzu-Lien Lee received her MS in Information Management from National Cheng Kung University, Taiwan. Ms. Lee’s areas of interests include e-learning, knowledge management, and e-commerce. She is currently working for a leading company of global navigation and geographic information systems as an e-service application developer.

ORCID iD

Wei-Tsong Wang

References

Akbari

Pilot

Simons

P. R. J.

(2015). Autonomy, competence, and relatedness in foreign language learning through facebook. Computers in Human Behavior, 48, 126–134. https://doi.org/10.1016/j.chb.2015.01.036

Athanasopoulos

Hyndman

R. J.

(2011). The value of feedback in forecasting competitions. International Journal of Forecasting, 27(3), 845–849. https://doi.org/10.1016/j.ijforecast.2011.03.002

Bartle

(1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD Research, 1(1), 21–2020. http://mud.co.uk/richard/hcds.htm

Barzilai

Blau

(2014). Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences. Computers & Education, 70, 65–79. https://doi.org/10.1016/j.compedu.2013.08.003

Björkman

(1972). Feedforward and feedback as determiners of knowledge and policy: Notes on a neglected issue. Scandinavian Journal of Psychology, 13(1), 152–158. https://doi.org/10.1111/j.1467-9450.1972.tb00063.x

Boyle

Connolly

T. M.

Hainey

(2011). The role of psychology in understanding the impact of computer games. Entertainment Computing, 2(2), 69–74. https://doi.org/10.1016/j.entcom.2010.12.002

Burgers

Eden

van Engelenburg

M. D.

Buningh

(2015). How feedback boosts motivation and play in a brain-training game. Computers in Human Behavior, 48, 94–103. https://doi.org/10.1016/j.chb.2015.01.038

Chen

C. H.

Law

(2016). Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation. Computers in Human Behavior, 55, 1201–1212. https://doi.org/10.1016/j.chb.2015.03.010

Chenoweth

Dowling

K. L.

Louis

R. D. S.

(2004). Convincing DSS users that complex models are worth the effort. Decision Support Systems, 37(1), 71–82. https://doi.org/10.1016/S0167-9236(03)00005-8

10.

Chen, Y. H. & Wang, C. H. (2018). Learner presence, perception, and learning achievements in augmented-reality-mediated learning environments. Interactive Learning Environments, 26(5), 695–708. https://doi.org/10.1080/10494820.2017.1399148

11.

Chiu

C. M.

Wang

E. T.

(2008). Understanding web-based learning continuance intention: The role of subjective task value. Information & Management, 45(3), 194–201. https://doi.org/10.1016/j.im.2008.02.003

12.

Chrysafiadi

Virvou

(2013). PeRSIVA: An empirical evaluation method of a student model of an intelligent e-learning environment for computer programming. Computers & Education, 68, 322–333. https://doi.org/10.1016/j.compedu.2013.05.020

13.

Cockrell

R. C.

Stone

D. N.

(2010). Industry culture influences pseudo-knowledge sharing: A multiple mediation analysis. Journal of Knowledge Management, 14(6), 841–857. https://doi.org/10.1108/13673271011084899

14.

Cohen

Brinkman

W. P.

Neerincx

M. A.

(2016). Effects of different real-time feedback types on human performance in high-demanding work conditions. International Journal of Human-Computer Studies, 91, 1–12. https://doi.org/10.1016/j.ijhcs.2016.03.007

15.

Damnjanovic

Jednak

Mijatovic

(2015). Factors affecting the effectiveness and use of Moodle: Students’ perception. Interactive Learning Environments, 23(4), 496–514.

16.

Deci

E. L.

Ryan

R. M.

(1985). The general causality orientations scale: Self-determination in personality. Journal of Research in Personality, 19(2), 109–134. https://doi.org/10.1016/0092-6566(85)90023-6

17.

Deci

E. L.

Ryan

R. M.

(2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/S15327965PLI1104_01

18.

Demetrovics

Urbán

Nagygyörgy

Farkas

Zilahy

Mervó

Reindl

Ágoston

Kertész

Harmath

(2011). Why do you play? The development of the motives for online gaming questionnaire (MOGQ). Behavior Research Methods, 43(3), 814–825. https://doi.org/10.3758/s13428-011-0091-y

19.

Diamantopoulos

(2006). The error term in formative measurement models: Interpretation and modeling implications. Journal of Modelling in Management, 1(1), 7–17. https://doi.org/10.1108/17465660610667775

20.

Diamantopoulos

Winklhofer

H. M.

(2001). Index construction with formative indicators: An alternative to scale development. Journal of Marketing Research, 38(2), 269–277. https://doi.org/10.1509/jmkr.38.2.269.18845

21.

Ding

(2018). Determinants of college students' use of online collaborative help‐seeking tools. Journal of Computer Assisted Learning, 34(2), 129–139. https://doi.org/10.1111/jcal.12221

22.

Faber

J. M.

Luyten

Visscher

A. J.

(2017). The effects of a digital formative assessment tool on mathematics achievement and student motivation: Results of a randomized experiment. Computers & Education, 106, 83–96. https://doi.org/10.1016/j.compedu.2016.12.001

23.

Fornell

C. R.

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312

24.

Gagné

(2009). A model of knowledge-sharing motivation. Human Resource Management, 48(4), 571–589. https://doi.org/10.1002/hrm.20298

25.

Gagné

Deci

E. L.

(2005). Self‐determination theory and work motivation. Journal of Organizational Behavior, 26(4), 331–362. https://doi.org/10.1002/job.322

26.

Garris

Ahlers

Driskell

J. E.

(2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441–467. https://doi.org/10.1177/1046878102238607

27.

Girard

Ecalle

Magnan

(2013). Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3), 207–219. https://doi.org/10.1111/j.1365-2729.2012.00489.x

28.

Göksün

D. O.

Gürsoy

(2019). Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Computers & Education, 135, 15–29. https://doi.org/10.1016/j.compedu.2019.02.015

29.

Gonzalez

(2005). Decision support for real-time, dynamic decision-making tasks. Organizational Behavior and Human Decision Processes, 96(2), 142–154. https://doi.org/10.1016/j.obhdp.2004.11.002

30.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Pearson Education.

31.

Hair

J. F.

Hult

G. T. M.

Ringle

C. M.

Sarstedt

(2013). A primer on partial least squares structural equation modeling (PLS-SEM). Sage Publications.

32.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

33.

Hong

J. C.

Cheng

C. L.

Hwang

M. Y.

Lee

C. K.

Chang

H. Y.

(2009). Assessing the educational values of digital games. Journal of Computer Assisted Learning, 25(5), 423–437. https://doi.org/10.1111/j.1365-2729.2009.00319.x

34.

Hoque

A. S.

Islam

M. M.

Hossain

Md, I.

Ahmed

M. F.

(2012). Problem-based e-learning and evaluation system for database design and programming in SQL. International Journal of e-Education, e-Business, e-Management and e-Learning, 2(6), 537–542. https://doi.org/10.7763/IJEEEE.2012.V2.180

35.

Hou

H. T.

M. C.

(2014). Evaluating multiple aspects of a digital educational problem-solving-based adventure game. Computers in Human Behavior, 30, 29–38. https://doi.org/10.1016/j.chb.2013.07.052

36.

P. J. H.

Hui

(2012). Examining the role of learning engagement in technology-mediated learning and its effects on learning effectiveness and satisfaction. Decision Support Systems, 53(4), 782–792.

37.

P. J.-H.

Hui

Clark

T. H. K.

Tam

K. Y.

(2007). Technology-assisted learning and learning style: A longitudinal field experiment. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 37(6), 1099–1112.

38.

Huang

C. F.

Nien

W. P.

Yeh

Y. S.

(2015). Learning effectiveness of applying automated music composition software in the high grades of elementary school. Computers & Education, 83, 74–89. https://doi.org/10.1016/j.compedu.2015.01.003

39.

Huang

Y. M.

(2019). Examining students’ continued use of desktop services: Perspectives from expectation-confirmation and social influence. Computers in Human Behavior, 96, 23–31.

40.

Johnson

C. I.

Mayer

R. E.

(2010). Applying the self-explanation principle to multimedia learning in a computer-based game-like environment. Computers in Human Behavior, 26(6), 1246–1252. https://doi.org/10.1016/j.chb.2010.03.025

41.

Jung

Schneider

Valacich

(2010). Enhancing the motivational affordance of information systems: The effects of real-time performance feedback and goal setting in group collaboration environments. Management Science, 56(4), 724–742. https://doi.org/10.1287/mnsc.1090.1129

42.

Kim

Y. J.

Shute

V. J.

(2015). The interplay of game elements with psychometric qualities, learning, and enjoyment in game-based assessment. Computers & Education, 87, 340–356. https://doi.org/10.1016/j.compedu.2015.07.009

43.

Kirkpatrick

D. L.

(1998). Evaluating training programs: The four levels (2nd ed.). Berrett-Koehler.

44.

Kirkpatrick

D. L.

Kirkpatrick

J. D.

(2009). Evaluating training programs: The four levels. Berrett-Koehler.

45.

Kung

H. J.

Tung

H. L.

(2006). An alternative approach to teaching database normalization: A simple algorithm and an interactive e-learning tool. Journal of Information Systems Education, 17(3), 315–325. https://aisel.aisnet.org/jise/vol17/iss3/10

46.

Law

Chen

C. H.

(2016). Promoting science learning in game-based learning with question prompts and feedback. Computers & Education, 103, 134–143. https://doi.org/10.1016/j.compedu.2016.10.005

47.

Lindell

M. K.

(1976). Cognitive and outcome feedback in multiple-cue probability learning tasks. Journal of Experimental Psychology: Human Learning & Memory, 2(6), 739–745. https://doi.org/10.1037/0278-7393.2.6.739

48.

Liu

Santhanam

(2013). Digital games and beyond: What happens when players compete. MIS Quarterly, 37(1), 111–124. https://doi.org/10.25300/MISQ/2013/37.1.05

49.

Maier

Wolf

Randler

(2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85–98. https://doi.org/10.1016/j.compedu.2015.12.002

50.

Malone

T. W.

(1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 5(4), 333–369. https://doi.org/10.1207/s15516709cog0504_2

51.

Malone

T. W.

Lepper

M. R.

(1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In Snow

R. E.

Farr

M. J.

(Eds.), Aptitude, learning, and instruction (volume 3): conative and affective process analyses (pp. 223–253). Lawrence Erlbaum Associates, Publishers.

52.

Mekler

E. D.

Brühlmann

Tuch

A. N.

Opwis

(2017). Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Computers in Human Behavior, 71, 525–534. https://doi.org/10.1016/j.chb.2015.08.048

53.

Merikivi

Tuunainen

Nguyen

(2017). What makes continued mobile gaming enjoyable? Computers in Human Behavior, 68, 411–421. https://doi.org/10.1016/j.chb.2016.11.070

54.

Mohammadyari

Singh

(2015). Understanding the effect of e-learning on individual performance: The role of digital literacy. Computers & Education, 82, 11–25. https://doi.org/10.1016/j.compedu.2014.10.025

55.

Moizer

Lean

Dell’Aquila

Walsh

Keary

A. A.

O'Byrne

Di Ferdinando

Miglino

Friedrich

Asperges

Sica

L. S.

(2019). An approach to evaluating the user experience of serious games. Computers & Education, 136, 141–151. https://doi.org/10.1016/j.compedu.2019.04.006

56.

Moreno

(2004). Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instructional Science, 32(1/2), 99–113. https://doi.org/10.1023/B:TRUC.0000021811.66966.1d

57.

Narciss

(2008). Feedback strategies for interactive learning tasks. In Spector

J. M.

Merrill

M. D.

van Merrienboer

J. J. G.

Driscoll

M. P.

(Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 125–144). Lawrence Erlbaum Associates.

58.

Ninaus

Greipl

Kiili

Lindstedt

Huber

Klein

Karnath

H. O.

Moeller

(2019). Increased emotional engagement in game-based learning - A machine learning approach on facial emotion detection data. Computers & Education, 142, 103641–103610. https://doi.org/10.1016/j.compedu.2019.103641

59.

Osman

Bakar

N. A.

(2012). Educational computer games for Malaysian classrooms: Issues and challenges. Asian Social Science, 8(11), 75–84. https://doi.org/10.5539/ass.v8n11p75

60.

Panigrahi

Srivastava

P. R.

Sharma

(2018). Online learning: Adoption, continuance, and learning outcome: A review of literature. International Journal of Information Management, 43, 1–14.

61.

Papastergiou

(2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12. https://doi.org/10.1016/j.compedu.2008.06.004

62.

Park

Kim

M. Y.

(2019). Learning to be better at the game: Performance vs. completion contingent reward for game-based learning. Computers & Education, 139, 1–15. https://doi.org/10.1016/j.compedu.2019.04.016

63.

Park

Liu

M. Y.

Santhanam

(2019). GAMESIT: A gamified system for information technology training. Computers & Education, 142, 103643–103619. https://doi.org/10.1016/j.compedu.2019.103643

64.

Pe-Than

E. P. P.

Goh

D. H. L.

Lee

C. S.

(2014). Making work fun: Investigating antecedents of perceived enjoyment in human computation games for information sharing. Computers in Human Behavior, 39, 88–99. https://doi.org/10.1016/j.chb.2014.06.023

65.

Petter

Straub

Rai

(2007). Specifying formative constructs in information systems research. MIS Quarterly, 31(4), 623–656. https://doi.org/10.2307/25148814

66.

Piccoli

Ahmad

Ives

(2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly, 25(4), 401–426. https://doi.org/10.2307/3250989

67.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J. Y.

Podsakoff

N. P.

(2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. The Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

68.

Proulx

J. N.

Romero

Arnab

(2017). Learning mechanics and game mechanics under the perspective of self-determination theory to foster motivation in digital game based learning. Simulation & Gaming, 48(1), 81–97. https://doi.org/10.1177/1046878116674399

69.

Qian

Clark

K. R.

(2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58. https://doi.org/10.1016/j.chb.2016.05.023

70.

Ranchhod

Gurău

Loukis

Trivedi

(2014). Evaluating the educational effectiveness of simulation games: A value generation model. Information Sciences, 264, 75–90.

71.

Rieber

L. P.

(1996). Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research and Development, 44(2), 43–58.

72.

Roca

J. C.

Gagné

(2008). Understanding e-learning continuance intention in the workplace: A self-determination theory perspective. Computers in Human Behavior, 24(4), 1585–1604. https://doi.org/10.1016/j.chb.2007.06.001

73.

Rosas

Nussbaum

Cumsille

Marianov

Correa

Flores

Grau

Lagos

López

Rodriguez

Salinas

(2003). Beyond Nintendo: Design and assessment of educational video games for first and second grade students. Computers & Education, 40(1), 71–94. https://doi.org/10.1016/S0360-1315(02)00099-4

74.

Ryan

R. M.

Connell

J. P.

(1989). Perceived locus of causality and internalization: Examining reasons for acting in two domains. Journal of Personality and Social Psychology, 57(5), 749–761. https://doi.org/10.1037/0022-3514.57.5.749

75.

Ryan

R. M.

Deci

E. L.

(2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68

76.

Ryan

R. M.

Lynch

M. F.

Vansteenkiste

Deci

E. L.

(2011). Motivation and autonomy in counseling, psychotherapy, and behavior change: A look at theory and practice. The Counseling Psychologist, 39(2), 193–260. https://doi.org/10.1177/0011000009359313

77.

Ryan

R. M.

Rigby

C. S.

Przybylski

(2006). The motivational pull of video games: A self-determination theory approach. Motivation and Emotion, 30(4), 344–360. https://doi.org/10.1007/s11031-006-9051-8

78.

Santhanam

Liu

Shen

W. C. M.

(2016). Research note—Gamification of technology-mediated training: Not all competitions are the same. Information Systems Research, 27(2), 453–465. https://doi.org/10.1287/isre.2016.0630

79.

Shakroum

Wong

K. W.

Fung

C. C.

(2018). The influence of gesture-based learning systems (GBLS) on learning outcomes. Computers & Education, 117, 75–101.

80.

Sharples

(2000). The design of personal mobile technologies for lifelong learning. Computers & Education, 34(3-4), 177–193. https://doi.org/10.1016/S0360-1315(99)00044-5

81.

C. H.

Cheng

C. H.

(2015). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning, 31(3), 268–286. https://doi.org/10.1111/jcal.12088

82.

Tan

J. L.

Goh

D. H. L.

Ang

R. P.

Huan

V. S.

(2016). Learning efficacy and user acceptance of a game-based social skills learning environment. International Journal of Child-Computer Interaction, 9-10, 1–19. https://doi.org/10.1016/j.ijcci.2016.09.001

83.

Tsai

M. J.

Huang

L. J.

Hou

H. T.

Hsu

C. Y.

Chiou

G. L.

(2016). Visual behavior, flow and achievement in game-based learning. Computers & Education, 98, 115–129. https://doi.org/10.1016/j.compedu.2016.03.011

84.

Tüzün

Yılmaz-Soylu

Karakuş

İnal

Kızılkaya

(2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 52(1), 68–77. https://doi.org/10.1016/j.compedu.2008.06.008

85.

van der Meij

Albers

Leemkuil

(2011). Learning from games: Does collaboration help? British Journal of Educational Technology, 42(4), 655–664. https://doi.org/10.1111/j.1467-8535.2010.01067.x

86.

Wang

W. T.

(2016). Examining the influence of the social cognitive factors and relative autonomous motivations on employees' knowledge sharing behaviors. Decision Sciences, 47(3), 404–436. https://doi.org/10.1111/deci.12151

87.

Wang

W. T.

Hou

Y. P.

(2015). Motivations of employees’ knowledge sharing behaviors: A self-determination perspective. Information and Organization, 25(1), 1–26. https://doi.org/10.1016/j.infoandorg.2014.11.001

88.

Wouters

van Nimwegen

van Oostendorp

van der Spek

E. D.

(2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311

89.

W. H.

Hsiao

H. C.

P. L.

Lin

C. H.

Huang

S. H.

(2012). Investigating the learning-theory foundations of game-based learning: A meta-analysis. Journal of Computer Assisted Learning, 28(3), 265–279. https://doi.org/10.1111/j.1365-2729.2011.00437.x

90.

Yang

Y. T. C.

Chang

C. H.

(2013). Empowering students through digital game authorship: Enhancing concentration, critical thinking, and academic achievement. Computers & Education, 68, 334–344. https://doi.org/10.1016/j.compedu.2013.05.023

91.

Yee

(2006). Motivations for play in online games. CyberPsychology & Behavior: The Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society, 9(6), 772–775. https://doi.org/10.1089/cpb.2006.9.772

92.

Young

M. F.

Slota

Cutter

A. B.

Jalette

Mullin

Lai

Simeoni

Tran

Yukhymenko

(2012). Our princess is in another castle: A review of trends in serious gaming for education. Review of Educational Research, 82(1), 61–89. https://doi.org/10.3102/0034654312436980

93.

Zhang

(2005). Interactive multimedia-based e-learning: A study of effectiveness. American Journal of Distance Education, 19(3), 149–162. https://doi.org/10.1207/s15389286ajde1903_3