How Different Categories of Gamified Stimuli Affect Massive Open Online Courses Continuance Intention and Learning Performance? Mediating Roles of Internal Experiences

Abstract

Drawing on gamification literature, this study develops a research model to examine whether gamification features as environmental stimulus antecedents to learners’ organismic experiences in using massive open online courses (MOOCs) can affect their response on MOOCs and learning outcomes. The proposed research framework, rooted in the stimulus-organism-response model, provides a strong foundation for understanding these hypothesized relationships. Sample data for this study were collected from learners who had experience in taking the gamified MOOCs provided by the MOOC platform launched by a well-known university in Taiwan, and 307 usable questionnaires were analyzed using structural equation modeling. This study verified that three types of gamification features including achievement-related gamification features, immersion-related gamification features, and social interaction-related gamification features positively influenced learners’ internal experiences in using MOOCs (i.e., cognitive involvement, flow experience, and social presence), which jointly expounded their continuance intention of MOOCs, and this in turn enhanced their perceived impact on learning. Overall, this study’s results offered enough evidence to strongly support all of the hypothesized links and the research model. Besides, the results of the mediation analysis confirmed that learners’ internal experiences and continuance intention of MOOCs fully mediated the effects of their perceived gamification features on perceived impact on learning.

Keywords

massive open online courses gamification features internal experiences continuance intention perceived impact on learning

Nowadays, the movement of massive open online courses (MOOCs) has been regarded as an educational innovation that overcomes the structural limitations (e.g., high cost, limited space, low accessibility, and time restrictions) of traditional educational institution settings (Aparicio et al., 2019; Lambert, 2020). MOOCs deliver high-quality contents from some of the world’s best universities for free to allow a large quantity of learners to access the online learning activities from anywhere with a computer and an Internet connection (Dai, Teo, & Rappa, 2020; Lambert, 2020). That is, MOOC allows distance learners who have learning desires to use the highest quality educational resources to free learn with low restrictions and high effectiveness (Tsai et al., 2018; Aparicio et al., 2019). Hence, distance learning via MOOCs has been seen of as a most effective, flexible, and affordable way of knowledge acquisition. As compared to the traditional teacher-centered instructing approach, the model of MOOCs is a student-centered learning approach, and pays more attention to learners’ initiative and autonomy that they can customize their learning goals according to their own situations, organize the learning structure, and choose learning courses they need (Lambert, 2020; Zhao et al., 2020).

Up to now, while MOOCs, as a new mode of online learning, have gained increasing attention from both academia and industry, there still exist a prevailing problem, high dropout rates and low completion rates when comparing enrollments with the number of students that completed these courses (El Kabtane et al., 2019; Dai, Teo, & Rappa, 2020; Zhao et al., 2020). Thus, the measurement of MOOCs’ effectiveness may depend on its continued usage (Wu & Chen, 2017; Zhao et al., 2020), and it is crucial to explore the underlying mechanisms that motivate learners to continue using MOOCs, which is a major research topic to this study and it is of great importance to online education.

Noteworthily, gamification is one of the most vital strategies that have attracted the attention of MOOC practitioners during the recent years, because they believe that it is more useful to transform learning resources to be more game-like to increase learners’ motivation to engage in MOOCs (Aparicio et al., 2019; Poondej & Lerdpornkulrat, 2020). Thus, gamification features may be introduced to MOOCs as the environmental stimuli of improving learners’ motivation and usage intention towards using MOOCs (Aparicio et al., 2019; Cheng, 2021). To date, there is still a lack of knowledge regarding how gamification features influence learners’ MOOCs continuance intention after having initially accepted it, while MOOCs have been regarded as a more innovative tool for delivering information and knowledge within educational sectors (Aparicio et al., 2019). Further, little is known about whether learners’ perceptions towards using gamification features in MOOCs and their response in MOOCs usage can enhance their perceived learning outcomes (Bai et al., 2020). Thus, the foregoing are the major research challenges to this study, and these are the reasons why this study explores how different categories of gamified stimuli affect MOOCs continuance intention and learning performance. The stimulus-organism-response (S-O-R) model, proposed by Mehrabian and Russell (1974), has been widely applied to the environmental features of various domains on users’ continuance intention of the information system (IS)/information technology (IT) to explain their decision-making processes. Hence, this study’s purpose is to propose a research model based on the S-O-R model to examine whether gamification features as environmental stimulus antecedents to learners’ internal experiences in using MOOCs including cognitive involvement, flow experience, and social presence can affect their continuance intention of MOOCs and perceived impact on learning.

Theoretical Background

Prior studies have extended the technology acceptance model (TAM) with a range of external factors to explain the adoption of MOOCs; however, the simplicity of the TAM has been frequently criticized, because it only concerns the effects of short-term beliefs and attitude on MOOC continuance intention (Arpaci et al., 2020; Wu & Chen, 2017). Besides, while the expectation-confirmation model has been proven to be a reliable model and has good predictive validity for MOOCs continuance intention (Alraimi et al., 2015; Dai, Teo, Rappa, & Huang, 2020), and the task-technology fit model has been used to study antecedents of MOOCs continuance intention (Huang et al., 2017; Wu & Chen, 2017), the two models cannot examine internal experiences (Alraimi et al., 2015; Dai, Teo, Rappa, & Huang, 2020; Zhao et al., 2020). Thus, the foregoing models provide little assistance in capturing the expectation of learners’ intrinsic motivation in MOOCs usage, which may be a key user belief that affects learners’ MOOCs continuance intention (Alraimi et al., 2015; Dai, Teo, Rappa et al., 2020; Zhao et al., 2020). Compared with the foregoing models employed in MOOC settings, the S-O-R model may provide a theoretically better justified explanation of MOOCs continuance intention via examining the impacts of both unique environmental features and internal experiences (Shao & Chen, 2021; Zhao et al., 2020). Following a review of prior studies, this study attempts to build a solid research model based on the S-O-R model to examine whether gamification features as antecedents to learners’ internal experiences in using MOOCs (i.e., cognitive involvement, flow experience, and social presence) can affect their continuance intention of MOOCs and perceived impact on learning. Each concept and theory is elaborated on below.

The Outline of Massive Open Online Coursess

The Features of Massive Open Online Coursess

The term “MOOC” was used for the first time by Georges Siemens and Dave Cormier and was coined in 2008, MOOCs can allow a large quantity of learners aiming to take a course or to be educated to access online courses all over the world regardless of their educational background, physical locations, and time constraints (Fianu et al., 2020; Lin & Kao, 2018). The MOOC refers to an online course that allows an unlimited number of learners from all over the world to learn the massive breadth and depth of open and online materials over the Internet, and it seeks to bring a different type of pedagogical content to a heterogeneous audience (Aparicio et al., 2019; Lambert, 2020; Lin & Kao, 2018). With the rapid development of MOOCs, many educational institutions have launched high-quality MOOCs to fit around online learners on a variety of topics and at a diverse range of educational levels. Generally, MOOCs are classified into connectivist MOOCs (cMOOCs) and extended MOOCs (xMOOCs), for most courses provided by Coursera, Open2Study, EdX, Udacity, and other MOOC platforms, and these platforms are each associated with highly regarded institutions of higher education (Alraimi et al., 2015; Wu & Chen, 2017; Zhao et al., 2020).

Previous Studies of Massive Open Online Courses

To date, research scopes in the context of the MOOCs field are constantly evolving. Prior studies present a holistic picture of MOOCs literature from 2008 to 2021, outlining a diversity of research topics on MOOCs including research areas of interests (Liyanagunawardena et al., 2013), research themes, trends of development, methodological approaches, learning and teaching aspects (Deng et al., 2019), an extensive variety of course content, teaching and learning methods, and assessment strategies for MOOCs (Yousef & Sumner, 2021). Besides, some researchers focus on financial aspects and business models of MOOCs (Aparicio et al., 2019; Dellarocas & Van Alstyne, 2013), and aspects of success, such as on engagement (Shao & Chen, 2021), adoption and continuous usage of MOOCs (Alraimi et al., 2015; Dai, Teo, Rappa, & Huang, 2020; Wu & Chen, 2017), and course completion (Li et al., 2018). However, there is still a lack of knowledge regarding the underlying mechanisms that motivate learners to continue using MOOCs and enhance their learning performance, which is a major research challenge to this study and it is of great importance to online education.

Massive Open Online Courses in Taiwan

Since 2013, the injection of MOOCs into traditional education ecosystem has become a new trend in Taiwan. Given that a significant number of online learners are using various MOOC platforms, MOOCs have gradually influenced Taiwan’s educational ecosystem (Yang et al., 2016). Recently, Taiwan’s government employs MOOCs as a means to revolutionize Taiwan’s education and improve pedagogical methods and structures from a series of projects named “Taiwan MOOCs” that Taiwan’s Ministry of Education launched to various industries in Taiwan, and such a Taiwan’s MOOCs movement has devoted to the development of MOOCs via providing free MOOCs platforms and has spread all over Taiwan from higher education to lifelong learning (Hsu et al., 2018; Yang et al., 2017). Currently, the spread of MOOCs in Taiwan has resulted in new pedagogical concepts and has provided the Taiwan’s educational ecosystem with numerous advantages (Hsu, 2021; Yang et al., 2017).

Gamification in Education

Gamification has left its mark on the face of education during the recent years. Gamification is a non-game mechanism that incorporates game design elements and creates aesthetics, intending to motivate a better user experience (Deterding et al., 2011; Hunicke et al., 2004). To date, gamification gradually acquires popularity in the context of online learning, and it has been regarded as a key competitive advantage in educational institutions (Aparicio et al., 2019). In the context of the educational sector, gamification can be used as a practical tool that incorporates game elements and gameful experience into the learning processes to motivate learners to participate in learning activities and enhance their learning performance (Dichev & Dicheva, 2017; Metwally et al., 2021). With gamification being a crucial factor in the educational sector, there is rising evidence to suggest that gamification is increasingly being accepted as an effective teaching strategy utilized to motivate and enhance learners’ learning experiences, and indicate that such a strategy can facilitate continuous improvements not only in learners’ motivation of performing desired learning tasks but also in their learning engagement (Zainuddin et al., 2020). However, there is little consensus about whether it contributes to improved learning performance (Bai et al., 2020), which is a main research topic to this study and it is of great importance to online education.

Gamification and Massive Open Online Courses

In the educational context, one of the most prominent research types in the gamification field is gamification applications in the development of MOOCs (Kasurinen & Knutas, 2018). Recently, employing the gamification in developing MOOCs has shown potential benefits in evoking learners’ learning motivation and engagement to improve such current MOOCs’ problems, such as high dropout rates and low completion rates of MOOCs (Khalil et al., 2018; Saputro et al., 2019). Many existing efforts have been made to empirically investigate the influences of gamification on MOOCs usage, and indicate that gamification is one of the most innovative pedagogical strategies that can motivate learners to active learning in MOOCs, and allow them to achieve their own learning goals and improve the effectiveness of their MOOCs usage (Kasurinen & Knutas, 2018; Khalil et al., 2018; Saputro et al., 2019). However, while extant studies clarify the positive effects of gamification on various learning activities within MOOCs, yet the question remains which are the most effective types of gamification features that can be used in MOOCs to drive learners’ learning motivation and enhance their MOOCs continuance intention and learning performance (Chang & Wie, 2016; Khalil et al., 2018), which is also a primary research topic to this study and it is of great significance for the MOOCs development.

The S-O-R Model

The S-O-R model is a theoretical framework which states that some environmental stimuli (S) affect individuals’ internal/organismic states (O), resulting in their specific behavioral responses (R) (Zhang et al., 2014; Zhao et al., 2020). The S-O-R model provides an underlying mechanism to examine the effects of external environmental stimuli on individuals’ virtual experiences and in turn on their responses, and it mainly emphasizes the vital role of individual internal processing in response to environmental stimuli (Triantoro et al., 2019; Shao & Chen, 2021). Stimulus refers to the factors or environmental cues that can trigger or affect individuals’ internal/organismic states (Mehrabian & Russell, 1974). In the MOOCs setting, stimuli are defined as the interactive features of MOOCs platforms with which learners interact with other participants by using such features provided by the platforms (Shao & Chen, 2021). Organism refers to the intrinsic states of cognition, emotion, feeling, perception, and physiology, which mediates the relationship between environmental stimuli and individuals’ responses in the S-O-R model (Mehrabian & Russell, 1974). In the context of MOOCs, organism reflects learners’ internal experiences in the process of using MOOCs, which in turn facilitates their continuance intention (Shao & Chen, 2021; Zhao et al., 2020). Response refers to an outcome of an attitude/a behavior, representing individuals’ final psychological reactions in response to a specific environmental stimulus (Mehrabian & Russell, 1974). In the MOOCs learning environment, response represents learners’ psychological reactions such as attitudes/behaviors to the MOOCs platform (Zhao et al., 2020).

Gamification Features as Environmental Stimuli (S)

In MOOCs learning environments, learners get information and knowledge via instructors, MOOCs education models, and MOOCs platforms (Fianu et al., 2020; Lambert, 2020), thus, the infrastructure of a course and its set of environmental features may act as stimuli (S) to affect learners’ internal/organismic states (O). Prior studies on MOOCs usage have confirmed that one of the most pivotal environmental stimuli of MOOCs usage, namely, gamification, because gamification captures crucial aspects of learners’ interactions with MOOCs learning environments (Aparicio et al., 2019; Poondej & Lerdpornkulrat, 2020). Hence, this study focuses on environmental stimuli, namely, gamification features.

Gamification is defined as the use of game design elements in non-game contexts (Deterding et al., 2011, p. 10). Further speaking, gamification is a specific kind of the game design process applying elements and characteristics of game design to influence individuals’ self-awareness and behaviors towards achieving specific goals, which can be used in non-game environments (Aparicio et al., 2019; Deterding et al., 2011). While most of the prior studies ignore the particular impacts of various gamification mechanisms on behavioral intention and regard gamification as a first-order construct comprising of multiple elements (e.g., Aparicio et al., 2019; Rodrigues et al., 2016), it is noteworthy that the impacts of multiple game design mechanisms on users’ internal states and behavioral outcomes may differ across various IS/IT usage contexts (Zhang et al., 2021). Hence, to uncover how and to what degree different gamification mechanisms can contribute to learners’ internal states and behavioral outcomes in the context of the MOOCs usage, this study divides gamification into achievement-related gamification features (ArGF), immersion-related gamification features (IrGF), and social interaction-related gamification features (SIrGF) in accordance with Xi and Hamari’s (2020) and Bitrián, Buil, & Catalán’s (2021) works. ArGF can attempt to enhance users’ sense of achievement in a manner in which each user can challenge themselves by accomplishing given milestones or reaching higher levels, and this game element category includes badges, challenges, goals, leaderboards, medals, missions, points, progress bars, rankings, etc. (Bitrián et al., 2021; Koivisto & Hamari, 2019; Xi & Hamari, 2020; Zhang et al., 2021). IrGF can try to immerse users in self-directed inquisitive activity, perceptions of escaping the real world in new virtual locations, new roles-playing, or the game story world, and this game element category includes avatars, customization, narratives, profiles, role-play mechanics, and storytelling, etc. (Bitrián et al., 2021; Koivisto & Hamari, 2019; Xi & Hamari, 2020). SIrGF can be used to enable users’ social interaction, and this game element category includes cooperation, competition, group, social networking features, and team. (Bitrián et al., 2021; Jang et al., 2018; Koivisto & Hamari, 2019; Xi & Hamari, 2020).

Distance Learners’ Experiences as Their Internal/Organismic States (O)

Previous studies on online learning usage have confirmed the crucial roles of three types of online learners’ internal/organismic states in determining their behavior, namely, cognitive involvement, flow experience, and social presence (Reychav & Wu, 2015; Zhao et al., 2020). Thus, this study includes the foregoing three kinds of online learners’ internal/organismic states in the research model.

Involvement refers to a person’s perceived relevance of the object based on inherent needs, values, and interests (Zaichkowsky, 1985, p. 342). Cognitive involvement is a motivational state that influences the degree of users’ informational processing activities elicited by the IS/IT usage (Kim & Sung, 2009), and it is defined as users’ perceived cognitive relevance of using the IS/IT in a social networking site based on cognitive needs, values, and interests (Huang, 2012; Kang et al., 2015; Zaichkowsky, 1994). MOOCs platforms can effectively involve learners who tend to search for learning resources more extensively, explore new stimuli for higher intention to interact with the platforms, and thus involvement may be seen in the relevance of the information presented on the platforms, and it may also reflect the level of learners’ motivation in using the MOOCs platforms to produce knowledge. That is why this study focuses on the role of cognitive involvement.

Flow experience is grounded on the flow theory introduced by Csikszentmihalyi (1975), and it is described as an individual’s inner experience in which the individual is completely engaged in her/his current activity (Guo & Poole, 2009). Flow experience refers to a holistic consciousness/sensation that people feel when they are totally absorbed, immersed, and involved in an activity (Csikszentmihalyi, 1975). In the MOOCs context, flow experience has been constructed by cognitive concentration, temporal dissociation, perceived control, and perceived enjoyment (Alraimi et al., 2015; Zhao et al., 2020), and prior studies have provided evidence for flow experience being a crucial construct for understanding the “gameful” experience in gamification and how it influences learners’ learning responses (Aparicio et al., 2019; Aşıksoy, 2018).

Social presence can be regarded as a mechanism of a computer-mediated environment that facilitates the communication and interaction of social cues between virtual participants (Hassanein & Head, 2006). Social presence refers to the feeling communication exchanges are sociable, warm, personal, sensitive, and active or the degree to which a medium is perceived as conveying the presence of communicating participants (Short et al., 1976, p. 65). Social presence emphasizes an individual’s psychological experience of interpersonal connections with others within a medium (Gefen & Straub, 2004). In the MOOCs context, social presence can be defined as MOOCs learners’ psychological experience of interpersonal connections and interactions with other learners or instructors within a MOOCs learning environment (Castellanos-Reyes, 2021; Zhao et al., 2020).

Distance Learners’ Massive Open Online Courses Continuance Intention as Their Behavioral Responses (R)

To date, the issue of high dropout rates undoubtedly represents an unprecedented challenge for MOOCs practitioners (Dai, Teo, & Rappa, 2020; El Kabtane et al., 2019). However, existing studies that capture learners’ intention to participate in MOOCs are too rough to explain their behavioral response to MOOCs adoption situations (Wu & Chen, 2017). Accordingly, it is particularly worth mentioning that learners’ continuance intention of MOOCs is a crucial means of measuring their behavioral response to MOOCs usage (Wu & Chen, 2017; Zhao et al., 2020). Hence, this study uses distance learners’ continuance intention of MOOCs as a proxy for their behavioral response to MOOCs usage in the research model. Continuance intention is defined as the behavior of a user to continue using a service after accepting it (Bhattacherjee, 2001).

Perceived Impact on Learning as the Performance Impact

The performance impact assessment of the IS/IT usage is a crucial topic for organizations if they intend to know whether the IS/IT investments are successful, however, relatively little attention has been paid to the performance impact of the IS/IT usage, because most efforts for prior studies have still regarded usage as an outcome in essence (Arning & Ziefle, 2007). Performance impact refers to the accomplishment of a portfolio of tasks by an individual, and higher performance implies some mix of improved efficiency, improved effectiveness, and/or higher quality (Goodhue & Thompson, 1995, p. 218). In the e-learning context, performance impact may relate to the impacts on students’ perceptions of learning outcomes among others, thus, the measure of students’ perceived impact on learning can be used as a proxy for their perceived performance impact (McGill & Klobas, 2009). Definitely, perceived impact on learning is defined as the perceptions of learning results of a learner using an online learning tool to accomplish her/his learning activities (Cheng, 2019; McGill & Klobas, 2009).

Hypotheses Development and Research Model

Based on the S-O-R framework, this study’s research model presents the effects of gamification features (i.e., ArGF, IrGF, and SIrGF) on learners’ continuance intention of MOOCs and their perceived impact on learning are mediated via their internal experiences (i.e., cognitive involvement, flow experience, and social presence) elicited by MOOCs. The hypotheses with related inferences are respectively proposed and elaborated on below.

Gamification Features and Distance Learners’ Experiences

ArGF

ArGF can be designed in a manner in which users can challenge themselves by achieving given milestones/reaching higher levels, and make users experience feelings of competence, because these features can continuously provide users with informational feedback (Bitrián et al., 2021; Zhang et al., 2021); and these features are commonly tied to a more cognitive style and behavior (Bitrián et al., 2021; Xi & Hamari, 2020; Zhang et al., 2021). Thus, this study posits that learners’ perceived ArGF in MOOCs can influence their cognitive involvement elicited by MOOCs. Hence, this study hypothesizes:

ArGF will positively affect cognitive involvement elicited by MOOCs.

IrGF

Learners usually have higher intrinsic motivation towards using e-learning platform in the gamified environment (Aşıksoy, 2018), and gamification strategies can increase learners’ perceived enjoyment of the learning experience in using MOOCs (Aparicio et al., 2019). Further, if users perceive greater IrGF designed on social network games, they may express higher flow experience elicited by using such games (Chang, 2013). Thus, this study posits that learners’ perceived IrGF in MOOCs can influence their flow experience elicited by MOOCs. Hence, this study hypothesizes:

IrGF will positively affect flow experience elicited by MOOCs.

SIrGF

The introduction of social interaction-related features into the virtual gamified systems can help users to communicate and exchange information with more other users online, which increases their sense of relatedness (Bitrián et al., 2021; Hassan et al., 2019; Xi & Hamari, 2020). That is, users’ sense of relatedness will arise when users compete, cooperate, and interact with other users via SIrGF developed on the virtual systems (Bitrián et al., 2021; Xi & Hamari, 2020). Consequently, it can be referred that if there are more interactions with SIrGF, learners can easily get/share information/knowledge about MOOCs from/with others online which can make learners perceive a higher level of social presence. Thus, this study posits that learners’ perceived SIrGF in MOOCs can influence their social presence elicited by MOOCs. Hence, this study hypothesizes:

SIrGF will positively affect social presence elicited by MOOCs.

Distance Learners’ Experiences and Massive Open Online Courses Continuance Intention

Cognitive Involvement

Users who are more cognitively involved will experience a stronger sense of flow due to their increased interests towards the usage of online environments for available information they need (Novak et al., 2000; Huang, 2012). Thus, this study posits that learners’ cognitive involvement elicited by MOOCs has a positive effect on their flow experience elicited by MOOCs. Hence, this study hypothesizes:

Cognitive involvement will positively affect flow experience elicited by MOOCs.

Users who experience social presence induced by social interactions with others via employing the online advertisement will feel more emotionally comfortable, such feelings are usually based on users’ motivations for their perceived cognitive involvement associated with the online advertisement usage (Fortin & Dholakia, 2005; Campbell et al., 2010). Thus, this study posits that learners’ cognitive involvement elicited by MOOCs has a positive effect on their social presence elicited by MOOCs. Hence, this study hypothesizes:

Cognitive involvement will positively affect social presence elicited by MOOCs.

If users may reveal motivations towards the virtual environment usage for the available information they need, thus their perceived cognitive involvement elicited by the virtual environment will be enhanced, and this will further promote their usage intention of the virtual environment (Drossos et al., 2014; Jiang et al., 2010). Thus, this study posits that learners’ cognitive involvement elicited by MOOCs can influence their continuance intention of MOOCs. Hence, this study hypothesizes:

H6: Cognitive involvement will positively affect continuance intention of MOOCs.

Flow Experience

Learners’ level of intrinsic motivation towards the MOOCs usage has a positive impact on their continuance intention of MOOCs (Alraimi et al., 2015; Tsai et al., 2018). Further, learners who experience a sense of flow have a strong desire to continue using MOOCs (Zhao et al., 2020). Thus, this study posits that learners’ flow experience elicited by MOOCs can influence their continuance intention of MOOCs. Hence, this study hypothesizes:

Flow experience will positively affect continuance intention of MOOCs.

Social Presence

When a learner strongly perceives the social presence of other learners, the learner is regarded as highly making much of the role that other learners participating in online learning activities, and may feel more emotionally enjoyable and further concentrate on such learning activities (Fu et al., 2009), which can ultimately lead to a stronger sense of flow experience within online learning environments (Kiili, 2005a, 2005b). Further, online learners with social presence elicited by MOOCs are likely to develop a higher degree of engagement in their MOOCs learning activities, thus they further become deeply involved, engaged, and engrossed in the social interactions via their MOOCs learning activities (Zhao et al., 2020). Thus, this study posits that learners’ social presence elicited by MOOCs has a positive effect on their flow experience elicited by MOOCs. Hence, this study hypothesizes:

Social presence will positively affect flow experience elicited by MOOCs.

Users’ perceived social presence has a positive impact on their usage intention of online environments (Cheung et al., 2011; Hassanein & Head, 2006). Further, users’ level of social presence will positively affect their desire to continue using online environments (Pavlou et al., 2007; Smith & Sivo, 2012). Thus, this study posits that learners’ social presence elicited by MOOCs can influence their continuance intention of MOOCs. Hence, this study hypothesizes:

Social presence will positively affect continuance intention of MOOCs.

Distance Learners’ Massive Open Online Courses Continuance Intention and Perceived Impact on Learning

In the e-learning context, students’ increased usage of the learning management system can cause enhancements in their perceived impact on learning (McGill & Klobas, 2009). Further, students’ continuance intention of MOOCs can positively result in their perceived impact on learning (Cheng, 2021). Thus, this study posits that learners’ continuance intention of MOOCs has a positive effect on their perceived impact on learning. Hence, this study hypothesizes:

H10: Continuance intention of MOOCs will positively affect perceived impact on learning.

Research Model

Drawing on the S-O-R framework and the gamification literature described above, this study develops a research model to explain learners’ continuance intention of MOOCs and their perceived impact on learning. Specifically, while this study only hypothesized a direct relationship between the perception of a class of gamification features (i.e., environmental stimuli) and one type of internal experiences (i.e., internal/organismic states) based on the available literature, this study still tested if learners could experience more than one type of internal experiences from all the investigated classes of gamification features that they perceived, because this was to ensure the identification of all possible relationships. Furthermore, in order to verify the mediating effects, this study also tested some unexpected direct effects between gamification features and perceived impact on learning. The research model used in this study is depicted in Figure 1.

Figure 1.

The research model. Note. Dotted arrows represent unexpected direct effects.

Methodology

Measurement Instruments

Based on the in-depth literature review, the different items of each construct were identified to design the draft questionnaire. In this study, responses to the items in ArGF, IrGF, SIrGF, cognitive involvement, flow experience, social presence, continuance intention, and perceived impact on learning were translated into Chinese via a standard back-translation process and were measured on a 7-point Likert scale ranging from 1 (= “strongly disagree”) to 7 (= “strongly agree”) with 4 labeled as neutral. Based on Cronbach’s (1951) views, items chosen for the constructs were adapted and revised from prior relevant studies and existing validated scales, where they had been shown to exhibit strong content validity. This study’s draft questionnaire includes nine parts, on eight constructs and participants’ demographics.

Pre-test

A pre-test was conducted with 50 undergraduate students who had experience in taking the gamified MOOCs provided by the MOOC platform launched by a well-known university in Taiwan in the voluntary and anonymous way during the Spring Semester 2021, and this selected university has implemented MOOCs for more than 3 years at the time of the study. Participants were requested to identify ambiguities in the meanings, and the questionnaire was revised based on their comments. Thus, according to Cronbach’s (1971) views, items chosen for the constructs in this study had strong face validity. The Cronbach’s α values (ranging from 0.83 to 0.91) exceeded requirements for exploratory research, indicating a satisfactory reliability level (Hair et al., 2010; Nunnally, 1978). The subjects who had participated in the pre-test were excluded from the final data collection and subsequent study. The final questionnaire includes nine parts, on eight constructs and participants' demographics. The final items of eight constructs are listed in Table 1 along with their sources.

Table 1.

Construct Measurement and Sources.

Construct	Item	Measure	Source
ArGF	ArGF1	“Points” can entice me to learn in the MOOCs learning environment	Koivisto and Hamari (2019)
	ArGF2	“Badges/medals” can make me feel more likely to do actions to obtain them in the MOOCs learning environment	Xi and Hamari (2020)
	ArGF3	“Progress bars” can motivate me to achieve my learning activities in the MOOCs learning environment	Bitrián et al. (2021)
	ArGF4	“Leaderboards/rankings/challenges” can assist me in enhancing the motivation for learning in the MOOCs learning environment	Zhang et al. (2021)
IrGF	IrGF1	“Avatars/profiles” can enable me to interact with life-like characters in the MOOCs learning environment	Koivisto and Hamari (2019)
	IrGF2	“Customization” can engage me in MOOCs by appealing to my emotions in the MOOCs learning environment	Xi and Hamari (2020)
	IrGF3	“Narrative/storytelling” can stimulate my thinking development in the MOOCs learning environment	Bitrián et al. (2021)
	IrGF4	“Role-play” can engage me in higher-order thinking in the MOOCs learning environment
SIrGF	SIrGF1	“Cooperation” can promote cooperative learning between those engaged in the MOOCs learning environment	Jang et al. (2018)
	SIrGF2	“Competition” can be regarded as a critical mechanism to boost effort-based learning and attention in the MOOCs learning environment	Koivisto and Hamari (2019)
	SIrGF3	“Social networking features” can encourage me to form an emotional investment in the MOOCs learning environment	Xi and Hamari (2020)
	SIrGF4	“Teams” can help me to better collaborate when organized with teams in the MOOCs learning environment	Bitrián et al. (2021)
CogInv	CogInv 1	Using MOOCs is important in my learning	Zaichkowsky (1994)
	CogInv 2	Using MOOCs is relevant in my learning	Huang (2012)
	CogInv 3	Using MOOCs is valuable in all aspects of my learning	Reychav and Wu (2015)
FE	FE1	I am absorbed in what I am doing while using MOOCs	Alraimi et al. (2015)
	FE2	I am often unable to keep track of the passage of time while using MOOCs	Zhao et al. (2020)
	FE3	I feel I have perceived control when I am using MOOCs
	FE4	I find using MOOCs to be enjoyable
SP	SP1	I feel comfortable interacting with instructors and other learners in the MOOCs learning environment	Pavlou et al. (2007)
	SP2	I feel that my point of view was acknowledged by other participants while using MOOCs	Cheung et al. (2011)
	SP3	I feel comfortable participating in discussions via the usage of MOOCs	Zhao et al. (2020)
	SP4	There is a sense of an online community when I am using MOOCs
ConInt	ConInt1	I intend to continue using MOOCs in the future	Bhattacherjee (2001)
	ConInt2	I will use MOOCs on a regular basis in the future	Wu and Chen (2017)
	ConInt3	I will frequently use MOOCs in the future	Zhao et al. (2020)
	ConInt4	My intentions are to continue using MOOCs than use any alternative means
PIoL	PIoL1	MOOCs have a large, positive impact on the effectiveness of my learning	Goodhue and Thompson (1995)
	PIoL2	MOOCs have a large, positive impact on the productivity of my learning	McGill and Klobas (2009)
	PIoL3	MOOCs are an important and valuable aid to me in the performance of my learning	Cheng (2019)
	PIoL4	MOOCs help me to improve the quality of my learning

Note. MOOC = massive open online courses; ArGF = Achievement-related gamification features; IrGF = Immersion-related gamification features; SIrGF = Social interaction-related gamification features; CogInv = Cognitive involvement; FE = Flow experience; SP = Social presence; PIoL = Perceived impact on learning.

Research Context and Sampling Process

This study tested the research model and hypotheses via using a cross-sectional questionnaire survey. This study’s sampling frame was taken from among learners who had experience in taking the gamified MOOCs provided by the MOOC platform launched by a well-known university in Taiwan. This selected university is recognized as an outstanding university with quality MOOCs in Taiwan and have adopted the MOOC platform to implement MOOCs for more than 3 years at the time of the study. All the MOOCs are video-based learning courses, follow a syllabus, and consisted of a learning manual, a set of slides, video, quizzes, and forums, and aimed at all of students enrolled at this university and other interested learners.

Data Collection

The target participants of this study were those with experience in taking the gamified MOOCs provided by the MOOC platform launched by the sample university that this study chose. This study’s data collection was performed via the MOOC platform launched by the sample university. The MOOC platform manager helped this study to distribute survey questionnaires via emails to learners who ever registered on the MOOC platform during the Spring Semester 2021. This study especially stated that the purpose of the survey is for academic research and invited target participants’ participation in the email, and further specified that the participants’ responses would be completely anonymous, confidential, and voluntary, and the obtained data from the participants were used only for this study. This study collected data from the middle to the end of the Spring Semester 2021. Overall, 317 participants responded to the survey, of whom 312 gave consent for their data to be used in this study and five withdrew. Consequently, a total of 312 questionnaires were started, of which 5 were considered problematic due to partial portions of missing data, after data-screening, 307 usable questionnaires were retained for further analysis in this study.

Non-Response Bias

T-tests were used to test non-response bias of the sample between early and late wave returned surveys, late wave responses being treated as a proxy for non-responses (Armstrong & Overton, 1977). In this study, 220 usable responses were received in the early wave and 87 in the late wave, the mean differences between the two groups with respect to demographic variables were tested using an unpaired t-test, and no significant differences were observed at the 0.05 level. Hence, the non-response bias is not a serious problem in this study, and the final sample of 307 usable responses can be considered to be representative of the population.

Data Analysis

This study adopted a quantitative empirical approach using cross-sectional questionnaire survey data to examine whether gamification features (i.e., ArGF, IrGF, and SIrGF) as antecedents to learners’ experiences (i.e., cognitive involvement, flow experience, and social presence) affected their continuance intention of MOOCs and perceived impact on learning. In order to analyze the data, this study followed a two-step method for structural equation modeling (SEM) approach recommended by Anderson and Gerbing (1988); first, confirmatory factor analysis (CFA) was used to develop the measurement model; next, to explore the causal relationships among all constructs, the structural model for the research model was tested by using SEM. Subsequently, the bootstrapping method was employed to examine the mediating roles within the research model. The statistical analysis software packages used to perform these analyses were AMOS 5.0 (SPSS, Inc., Chicago, Illinois, United States) and SPSS 8.0 (SPSS, Inc., Chicago, Illinois, United States).

Results

Demographics of the Usable Responses

Three hundred seven valid samples remained as the final sample data in the study. Overall, 60.3% of the respondents were male, while 39.7% were female. Most of the respondents were aged between 20 and 25 years (68.1%). The majority of the respondents were undergraduate students (77.5%). In terms of experience in using MOOCs, 98.4% reported logging into the MOOC platform at least once a week, while only 7.5% of respondents had more than 3 years of experience in using MOOCs; besides, 75.9% of the respondents had enrolled in one or two gamified MOOCs. Demographics of the usable responses are depicted in Table 2.

Table 2.

Demographics of the Usable Responses.

Demographics	Number (n = 307)	%
Gender
Female	122	39.7
Male	185	60.3
Age
Under 20	62	20.2
20–25	209	68.1
above 25	36	11.7
Educational level
Students
Undergraduate students	238	77.5
Master students	39	12.7
PhD students	9	2.9
Work
Bachelor	9	2.9
Master	6	2.0
PhD	6	2.0
Duration of using MOOCs (year)
Under 1	168	54.7
1–3	116	37.8
above 3	23	7.5
Frequency of using MOOCs
Several times a month	5	1.6
Once a week	51	16.6
Several times a week	178	58.0
Once a day	66	21.5
Several times a day	7	2.3
How many gamified MOOCs have you enrolled in?
1–2	233	75.9
3–4	71	23.1
5 or more	3	1.0

Note. MOOC = massive open online courses.

Normality Test

The sample size of 307 is more than the minimum size of 200 required for SEM analysis, thus normality should be tested because it is one of the basic assumptions required in order to carry out SEM analysis (Kline, 2011). The skewness and kurtosis values of all observed variables were used to assess the univariate normality of the data distribution. The absolute values of the skewness and kurtosis of all the items should be less than 3 and 8 respectively (Dai, Teo, & Rappa, 2020; Kline, 2011). In this study, the absolute values of the skewness and kurtosis of all the items were within 0.083–0.996 and 0.378–1.788, respectively. After this, multivariate normality was supported by comparison results, and Mardia’s coefficient (Mardia, 1970) were used to test this situation and the coefficient should be smaller than the formula p (p + 2), where p is the number of observed variables (Bollen, 1989; Chen, 2016). Thus, 591.199 in this study, was below the computed result of 1023 from the formula p (p + 2). Two test results all showed that the data were normally distributed.

Results of Structural Equation Modeling Analysis

Measurement Model

To assess the measurement model, three types of analyses were conducted in this study. First, indicator reliability and construct reliability were used in this study to measure the reliability of all constructs. Indicator reliability was measured by standardized factor loadings and squared multiple correlation (SMC) values, results exhibited that the standardized factor loadings of the items on the latent constructs were all above 0.7 and SMC values for all items were greater than 0.5, indicating good indicator reliability of the items (Hair et al., 2010; Nunnally, 1978). Further, construct reliability was tested by Cronbach’s α coefficient and composite reliability (CR) for each construct, results showed that the Cronbach’s α coefficient for each construct exceeded the 0.7 cut-off value as recommended by Hair et al. (2010), and the CR for each construct was greater than the benchmark of 0.7 (Hair et al., 2010; Nunnally, 1978), indicating good internal consistency and reliability of all constructs. The results of the reliability test are shown in Table 3.

Table 3.

Results of Confirmatory Factor Analysis, Reliability Test, and Validity Analysis.

ConstructItem	Estimate	T-value	Standardized Factor Loadings	Squared Multiple Correlation	Composite Reliability	Average Variance Extracted	Cronbach’s α
ArGF					0.927	0.761	0.945
ArGF1	1.000	—^a	0.905	0.820
ArGF2	0.997	27.638	0.928	0.862
ArGF3	0.949	25.106	0.888	0.778
ArGF4	1.008	26.506	0.900	0.809
IrGF					0.925	0.755	0.939
IrGF1	1.000	—^a	0.905	0.818
IrGF2	0.987	26.019	0.912	0.830
IrGF3	1.020	23.935	0.896	0.803
IrGF4	1.053	25.505	0.903	0.815
SIrGF					0.920	0.741	0.931
SIrGF1	1.000	—^a	0.869	0.759
SIrGF2	0.973	20.788	0.853	0.727
SIrGF3	1.078	25.516	0.930	0.865
SIrGF4	1.087	23.682	0.897	0.805
CogInv					0.913	0.777	0.925
CogInv1	1.000	—^a	0.916	0.838
CogInv2	1.020	33.295	0.930	0.859
CogInv3	1.002	29.517	0.906	0.815
FE					0.923	0.749	0.937
FE1	1.000	—^a	0.915	0.839
FE2	1.015	28.308	0.920	0.850
FE3	1.047	25.566	0.888	0.789
FE4	1.041	27.095	0.907	0.822
SP					0.922	0.746	0.935
SP1	1.000	—^a	0.897	0.805
SP2	1.008	23.808	0.895	0.803
SP3	1.088	28.889	0.947	0.897
SP4	1.080	27.678	0.933	0.870
ConInt					0.935	0.781	0.950
ConInt1	1.000	—^a	0.926	0.850
ConInt2	1.029	31.555	0.945	0.888
ConInt3	1.045	28.971	0.917	0.841
ConInt4	1.017	29.166	0.923	0.847
PIoL					0.945	0.810	0.961
PIoL1	1.000	—^a	0.927	0.861
PIoL2	1.033	31.700	0.939	0.883
PIoL3	1.077	29.300	0.919	0.846
PIoL4	1.059	30.796	0.933	0.868

Note. ArGF = Achievement-related gamification features; IrGF = Immersion-related gamification features; SIrGF = Social interaction-related gamification features; CogInv = Cognitive involvement; FE = Flow experience; SP = Social presence; PIoL = Perceived impact on learning.

^aThe loading was fixed.

Next, convergent validity and discriminant validity were used in this study to assess the validity of all constructs. To evaluate the convergent validity, this study first examined the average variance extracted (AVE), and AVE values for all constructs exceeded the minimum acceptable values of 0.5 (Fornell & Larcker, 1981; Hair et al., 2010), indicating a good convergent validity level (see Table 3); next, according to Anderson and Gerbing’s (1988) rule, the results of CFA demonstrated that the t-value of every item exceeded the 1.96 value (p < 0.05), so the evidence significantly supported the presence of good convergent validity (see Table 3). Discriminant validity was confirmed by comparing the AVE for each construct with the squared correlation for each pair of constructs (Fornell & Larcker, 1981). The results of CFA proved that the AVE of each construct was greater than the squared correlation for each pair of constructs, indicating that each construct was distinct and the presence of discriminant validity was supported (see Tables 3 and 4). Besides, convergent validity and discriminant validity were also assessed by using principal components analysis with varimax rotation. As noted in Table 5, eight components were extracted, accounting for 88.9% of the total variance. Convergent and discriminant validity can be established by the factor loadings greater than 0.7 and the cross-loadings lower than 0.4 (Hair et al., 2010). The results verified that all loadings were above 0.7, and the cross-loading scores were all below 0.4 (see Table 5). Thus, convergent validity and discriminant validity are demonstrated.

Table 4.

Discriminant Validity for the Measurement Model.

Construct	Variance
Construct	ArGF	IrGF	SIrGF	CogInv	FE	SP	ConInt	PIoL
ArGF	0.761
IrGF	0.027	0.755
SIrGF	0.100	0.053	0.741
CogInv	0.628	0.389	0.415	0.777
FE	0.396	0.561	0.368	0.437	0.749
SP	0.381	0.331	0.618	0.465	0.345	0.746
ConInt	0.112	0.054	0.062	0.308	0.496	0.533	0.781
PIoL	0.138	0.045	0.056	0.137	0.133	0.127	0.518	0.810

The italic values along the diagonal line are the AVE values for the constructs, and the other values are the squared correlations for each pair of constructs.

Table 5.

Results of Factor Structure Matrix of Loadings and Cross-loadings.

Construct	Component
Item	1	2	3	4	5	6	7	8
ArGF
ArGF1	0.197	0.176	0.925	0.151	0.163	0.167	0.141	0.101
ArGF2	0.206	0.149	0.916	0.156	0.205	0.157	0.106	0.071
ArGF3	0.176	0.220	0.880	0.174	0.170	0.163	0.076	0.060
ArGF4	0.136	0.183	0.851	0.182	0.171	0.162	0.063	0.070
IrGF
IrGF1	0.233	0.167	0.198	0.888	0.170	0.152	0.118	0.113
IrGF2	0.228	0.183	0.154	0.871	0.186	0.175	0.107	0.115
IrGF3	0.134	0.192	0.215	0.906	0.147	0.157	0.143	0.133
IrGF4	0.215	0.147	0.205	0.851	0.168	0.165	0.091	0.061
SIrGF
SIrGF1	0.183	0.113	0.153	0.162	0.194	0.153	0.815	0.062
SIrGF2	0.213	0.216	0.183	0.183	0.162	0.163	0.788	0.083
SIrGF3	0.220	0.168	0.163	0.191	0.163	0.154	0.859	0.218
SIrGF4	0.195	0.181	0.230	0.151	0.176	0.159	0.795	0.130
CogInv
CogInv1	0.185	0.200	0.170	0.173	0.216	0.156	0.086	0.879
CogInv2	0.206	0.156	0.183	0.157	0.188	0.180	0.104	0.898
CogInv3	0.175	0.151	0.228	0.171	0.196	0.176	0.085	0.861
FE
FE1	0.145	0.211	0.149	0.146	0.880	0.153	0.141	0.113
FE2	0.178	0.153	0.167	0.163	0.817	0.152	0.106	0.105
FE3	0.145	0.192	0.176	0.171	0.835	0.151	0.047	0.062
FE4	0.123	0.160	0.186	0.156	0.862	0.168	0.086	0.070
SP
SP1	0.181	0.201	0.161	0.156	0.156	0.835	0.123	0.118
SP2	0.223	0.151	0.208	0.161	0.178	0.788	0.094	0.086
SP3	0.173	0.156	0.158	0.163	0.175	0.871	0.095	0.050
SP4	0.160	0.213	0.201	0.176	0.149	0.812	0.118	0.053
ConInt
ConInt1	0.213	0.930	0.191	0.159	0.167	0.150	0.108	0.126
ConInt2	0.195	0.905	0.167	0.168	0.216	0.161	0.098	0.111
ConInt3	0.178	0.901	0.152	0.157	0.201	0.171	0.068	0.108
ConInt4	0.196	0.888	0.163	0.173	0.203	0.163	0.052	0.123
PIoL
PIoL1	0.929	0.213	0.213	0.176	0.183	0.174	0.141	0.123
PIoL2	0.909	0.196	0.227	0.149	0.206	0.181	0.136	0.095
PIoL3	0.939	0.234	0.154	0.176	0.238	0.175	0.130	0.070
PIoL4	0.917	0.218	0.192	0.203	0.194	0.161	0.123	0.051

The italic values are factor loadings that are significant and greater than 0.7.

Third, the overall fit indices of measurement model were chi-square (χ²) = 803.002, df = 406, χ²/df = 1.978, goodness-of-fit index (GFI) = 0.923, adjusted GFI (AGFI) = 0.906, incremental fix index (IFI) = 0.956, Tucker-Lewis index (TLI) = 0.950, comparative fit index (CFI) = 0.956, and root mean square error of approximation (RMSEA) = 0.057, as all conformed the guidelines suggested by Bagozzi and Yi (1988) and Hair et al. (2010), that is, χ²/df < 3, GFI >0.9, AGFI >0.8, IFI >0.9, TLI >0.9, CFI >0.9, RMSEA <0.08. Thus, the results of CFA showed that the indices were over their respective common acceptance levels.

Common Method Bias

If the self-report questionnaires are employed to collect data at the same time from the same source, a common method bias (CMB) should be a concern (Podsakoff et al., 2003). Harman’s single-factor test is performed to estimate the existence of the CMB (Harman, 1976; Podsakoff et al., 2003). Following Sanchez et al. (1995) work, a CFA method to Harman’s single-factor test was used to measure the CMB. This study employed CFA to examine the fit of a single-factor model (where all items were loaded on a single factor) and an eight-factor model. The results confirmed that the fit indices of the eight-factor model (i.e., χ² = 803.002, df = 406, χ²/df = 1.978, GFI = 0.923, AGFI = 0.906, IFI = 0.956, TLI = 0.950, CFI = 0.956, and RMSEA = 0.057) were better than those of the single-factor model (i.e., χ² = 2191.681, df = 434, χ²/df = 5.050, GFI = 0.730, AGFI = 0.691, IFI = 0.829, TLI = 0.817, CFI = 0.829, and RMSEA = 0.115). According to the criteria recommended by Sanchez et al. (1995), this study certified that there was no CMB in the data.

Multicollinearity

Before the evaluation of the structural model, this study assessed the model for multicollinearity issues. The variance inflation factor (VIF) value is a common judgment of multicollinearity, and a VIF value should be smaller than or equal to 5 (Hair et al., 2010). This study’s results showed that the VIF values of the seven independent variables ranged from 1.118 to 2.878, confirming that multicollinearity was not a serious problem in this study.

Structural Model

To examine the path relationship and explanatory power of the research model, this study further tested the structural model for the research model depicted in Figure 1. The bootstrapping procedure with 5000 subsamples was employed to calculate the statistical significance of the parameter estimates, which enables us to derive valid t-values (Preacher & Hayes, 2008; Zhang et al., 2021). The overall fit indices for the structural model were as follows: χ² = 931.888, df = 409, χ²/df = 2.278, GFI = 0.901, AGFI = 0.880, IFI = 0.938, TLI = 0.930, CFI = 0.938, and RMSEA = 0.065. Following Bagozzi and Yi (1988) and Hair et al. (2010), the results of CFA showed that the fit indices for this structural model were quite acceptable. Further, properties of the causal paths, including standardized path coefficients (β), t-values, and explained variances (R²), are shown in Figure 2. This study’s results strongly supported the research model with all hypothesized links being significant, and thus all hypotheses were demonstrated. As for the five endogenous constructs, the explained variances (R²) of cognitive involvement, flow experience, social presence, continuance intention, and perceived impact on learning were 0.655, 0.837, 0.694, 0.844, and 0.753, respectively. The hypothesis testing results are presented in Table 6 and Figure 2.

Figure 2.

Results of structural modeling analysis. Note. 1. Standardized path coefficients are reported (t-values in parentheses). 2. Absolute t-value > 1.96, p < .05; absolute t-value > 2.58, p < .01; absolute t-value > 3.29, p < .001.

Table 6.

The Results of Hypothesis Testing.

Hypothesis/Unexpected Effect		Standardized Path Coefficient	T-value	Significance	Support
H1	ArGF → CogInv	0.695	15.227	p < .001	Yes
Unexpected effect	ArGF → FE	0.248	4.550	p < .001
Unexpected effect	ArGF → SP	0.185	3.939	p < .001
Unexpected effect	ArGF → ConInt	0.047	0.739	p > .05
Unexpected effect	ArGF → PIoL	0.100	1.545	p > .05
Unexpected effect	IrGF → CogInv	0.223	4.153	p < .001
H2	IrGF → FE	0.497	11.715	p < .001	Yes
Unexpected effect	IrGF → SP	0.151	3.427	p < .001
Unexpected effect	IrGF → ConInt	0.043	0.723	p > .05
Unexpected effect	IrGF → PIoL	0.067	0.981	p > .05
Unexpected effect	SIrGF → CogInv	0.260	6.430	p < .001
Unexpected effect	SIrGF → FE	0.167	3.546	p < .001
H3	SIrGF → SP	0.545	11.310	p < .001	Yes
Unexpected effect	SIrGF → ConInt	0.009	0.187	p > .05
Unexpected effect	SIrGF → PIoL	0.026	0.460	p > .05
H4	CogInv → FE	0.293	4.567	p < .001	Yes
H5	CogInv → SP	0.345	4.819	p < .001	Yes
H6	CogInv → ConInt	0.130	1.968	p < .05	Yes
Unexpected effect	CogInv → PIoL	0.073	1.118	p > .05
H7	FE → ConInt	0.360	5.537	p < .001	Yes
Unexpected effect	FE → PIoL	0.018	0.159	p > .05
H8	SP → FE	0.157	2.510	p < .05	Yes
H9	SP → ConInt	0.441	6.970	p < .001	Yes
Unexpected effect	SP → PIoL	0.063	0.916	p > .05
H10	ConInt → PIoL	0.373	5.812	p < .001	Yes

Note. ArGF = Achievement-related gamification features; IrGF = Immersion-related gamification features; SIrGF = Social interaction-related gamification features; CogInv = Cognitive involvement; FE = Flow experience; SP = Social presence; PIoL = Perceived impact on learning.

Mediation Analysis

In order to better understand the mediating effects in the research model, this study examined whether the central variables of the research model could act as mediators in the model. Following Preacher and Hayes’ (2008) work, the bootstrapping procedure with 5000 subsamples was used to test the mediating effects, because this approach had the highest power and the best error control, as well as the 95% confidence interval (CI) for the mediating variables. For each bootstrapping sample, the study estimated the direct effects and indirect effects using AMOS. If the bootstrapping CI between the lower and upper bounds for the total, direct, or indirect effect does not contain zero, the total, direct, or indirect effect is significantly different from zero with a 95% CI (Preacher & Hayes, 2008; Hayes, 2009). Essentially, full mediation indicates that the inclusion of the mediator makes the direct effect between the independent variable and dependent variable insignificant whereas partial mediation indicates that direct and indirect effects are significant (Zhao et al., 2021).

As shown in Table 7, mediation analyses in this study were tested using the bootstrapping procedure with 95% CI of bias-corrected percentile method and 95% CI of percentile method. Gamification features (i.e., ArGF, IrGF, and SIrGF) had indirect impacts on continuance intention of MOOCs (i.e., zero was not included between the lower and upper values of the two methods for indirect effects) and all direct impacts of gamification features on continuance intention of MOOCs were insignificant (i.e., zero was included between the lower and upper values of the two methods for all direct effects), thus the results of the mediation analysis confirmed that distance learners’ experiences (i.e., cognitive involvement, flow experience, and social presence) fully mediated the effects of gamification features (i.e., ArGF, IrGF, and SIrGF) on continuance intention of MOOCs. Further, distance learners’ experiences (i.e., cognitive involvement, flow experience, and social presence) and continuance intention of MOOCs fully mediated the effects of gamification features (i.e., ArGF, IrGF, and SIrGF) on perceived impact on learning, and continuance intention of MOOCs fully mediated the effects of distance learners’ experiences (cognitive involvement, flow experience, and social presence) on perceived impact on learning.

Table 7.

Results of the Bootstrapping Procedure for Testing the Mediation Effects.

Influential Path Standardized Effect	Standardized Path Coefficient						Mediation Result
	95% CI of Bias-Corrected Percentile Method			95% CI of Percentile Method
	Lower	Upper	Two-Tailed Test p-value	Lower	Upper	Two-Tailed Test p-value
Gamification features → MOOCs continuance intention						Mediation result: Full mediation
ArGF → ConInt
Direct effect	−0.024	0.330	p > .05	−0.017	0.344	p > .05	Full mediation
Indirect effect	0.346	0.625	p < .001	0.323	0.600	p < .001
Total effect	0.460	0.763	p < .001	0.454	0.756	p < .001
IrGF → ConInt
Direct effect	−0.174	0.085	p > .05	−0.192	0.075	p > .05	Full mediation
Indirect effect	0.217	0.658	p < .001	0.210	0.634	p < .001
Total effect	0.202	0.510	p < .001	0.198	0.503	p < .001
SIrGF → ConInt
Direct effect	−0.126	0.149	p > .05	−0.129	0.147	p > .05	Full mediation
Indirect effect	0.291	0.613	p < .001	0.280	0.603	p < .001
Total effect	0.310	0.587	p < .001	0.304	0.579	p < .001
Gamification features → perceived impact on learning						mediation result: Full mediation
ArGF → PIoL
Direct effect	−0.088	0.338	p > .05	−0.095	0.326	p > .05	Full mediation
Indirect effect	0.313	0.722	p < .001	0.294	0.707	p < .001
Total effect	0.435	0.742	p < .001	0.432	0.741	p < .001
IrGF → PIoL
Direct effect	−0.093	0.339	p > .05	−0.099	0.323	p > .05	Full mediation
Indirect effect	0.078	0.513	p < .05	0.051	0.478	p < .05
Total effect	0.203	0.496	p < .001	0.195	0.490	p < .001
SIrGF → PIoL
Direct effect	−0.153	0.215	p > .05	−0.161	0.208	p > .05	Full mediation
Indirect effect	0.165	0.567	p < .001	0.153	0.547	p < .001
Total effect	0.221	0.530	p < .001	0.213	0.522	p < .001
Distance learners’ experiences → perceived impact on learning mediation result: Full mediation
CogInv → PIoL
Direct effect	−0.034	0.287	p > .05	−0.045	0.279	p > .05	Full mediation
Indirect effect	0.051	0.415	p < .05	0.020	0.387	p < .05
Total effect	0.326	0.641	p < .001	0.322	0.636	p < .001
FE → PIoL
Direct effect	−0.447	0.364	p > .05	−0.430	0.375	p > .05	Full mediation
Indirect effect	0.060	0.436	p < .05	0.027	0.367	p < .05
Total effect	0.066	0.479	p < .05	0.060	0.471	p < .05
SP → PIoL
Direct effect	−0.043	0.470	p > .05	−0.061	0.430	p > .05	Full mediation
Indirect effect	0.059	0.430	p < .05	0.051	0.420	p < .05
Total effect	0.088	0.617	p < .01	0.083	0.613	p < .01

Discussions

Gamification is widely regarded as a crucial technique for motivating users towards further preferred behaviors. However, there has been a lack of empirical evidence on whether and how different categories of gamification features may be able to affect learning outcome. Thus, this study identifies and defines three categories of gamification features including ArGF, IrGF, and SIrGF, and examines how they influence perceived impact on learning in the context of MOOCs learning environments. This study obtains valuable research results by uncovering the full mediators of distance learners’ experiences (i.e., cognitive involvement, flow experience, and social presence) and continuance intention of MOOCs between gamification features (i.e., ArGF, IrGF, and SIrGF) and perceived impact on learning. Overall, this study’s empirical results support all of the proposed hypotheses and the research model respectively accounts for 84.4% and 75.3% of the variance in learners’ continuance intention of MOOCs and perceived impact on learning via positioning key constructs as drivers (see Figure 2). Detailed discussions for the key research findings of this study are proposed below.

To date, extant research (e.g., Aparicio et al., 2019) has still failed to pay attention to whether and how different categories of learners’ gamification features can affect their MOOCs continuance intention via internal experiences. This study especially echoes Xi and Hamari’s (2020) and Bitrián et al.’s (2021) calls for strengthened empirical research on multi-dimensional gamification constructs in MOOCs usage settings. This study verified that learners’ perceived gamification features (i.e., ArGF, IrGF, and SIrGF) in MOOCs positively affected their internal experiences (i.e., cognitive involvement, flow experience, and social presence) elicited by MOOCs, which jointly expounded their continuance intention of MOOCs. Hence, this study’s results are consistent with Xi and Hamari’s (2020) findings that ArGF and SIrGF are the most powerful features of gamified experiences within the usage of online platforms. Next, this study confirms that Zhang et al.’s (2021) views that ArGF are beneficial to increasing users’ internal experiences (i.e., perceived enjoyment and social interaction), which in turn can significantly increase their usage intention of online platforms. Besides, this study’s results also supports Bitrián et al.’s (2021) research that have found that the most predominant game features are ArGF while they positively affect users’ engaging experiences (i.e., users’ engagement) via the simultaneous promotion of their needs, and the inclusion of IrGF is also worthwhile. Hence, this study’s results reveal that if learners can interact with various types of gamification features including ArGF, IrGF, and SIrGF in the MOOCs learning environment, this can drive their internal experiences (i.e., cognitive involvement, flow experience, and social presence), and trigger their MOOCs continuance intention, which can further increase their perceived impact on learning. The results first imply that interacting with gamification features within a virtual environment can assist in keeping users engaged in an interested and challenging self-directed activity, and further drive their intrinsic motivations including cognition, emotion, and social connectedness (Bitrián et al., 2021; Xi & Hamari, 2020; Zhang et al., 2021). In the learning context, learners’ intrinsic motivations can facilitate them to participate in learning activities away from any external compulsion and without expectations of rewards (Ryan & Deci, 2000), which may further enhance their learning achievements (Pérez-López & Contero, 2013). Hence, this study’s results further imply that interacting with gamification features via using an online learning tool for motivating learners to perform their desired learning tasks can enhance such a tool continuance intention and increase learners’ learning performance (Bai et al., 2020), and learners who complete course-related tasks tend to perform better than learners who postpone completing them (Bai et al., 2020; Michinov et al., 2011). Unquestionably, this study’s results just mirror the foregoing views through the lens of intrinsic motivation theory. Further, this study proved that learners’ continuance intention of MOOCs had a positive direct impact on their perceived impact on learning. The result supports Aparicio et al.’s (2019) study that has found that learners’ MOOCs usage is a direct antecedent of their performance impact. This study’s results further reveal that learners’ perceived gamification features (i.e., ArGF, IrGF, and SIrGF) in MOOCs show the full-mediation impacts on their perceived impact on learning via their internal experiences (i.e., cognitive involvement, flow experience, and social presence) and continuance intention of MOOCs. Hence, the result implies that gamification features can play the critical roles as determinants to drive the transforming process of “internal experiences → continuance intention → perceived impact on learning” in MOOCs settings.

However, in this study, there are three findings in particular that would be valuable to explore in greater depth. Detailed discussions for the three worthwhile research findings of this study are respectively proposed and elaborated on below.

ArGF and Distance Learners’ Experiences

This study’s results reveal that learners’ perceived ArGF in MOOCs have a positive and evidently larger direct impact on their cognitive involvement elicited by MOOCs than on their flow experience and social presence, and 95% CIs of “ArGF→CogInv” do not overlap with those of “ArGF→FE” and “ArGF→SP” (see Table 8). Hence, it is not surprising to verify a positive connection between learners’ perceived ArGF and their cognitive involvement elicited by MOOCs as hypothesized (H1), and, arguably, learners’ interaction with ArGF in MOOCs is positively and more strongly associated with cognitive involvement elicited by MOOCs than with flow experience and social presence. The results imply that the designs of ArGF in MOOCs often do not intentionally attempt to provide learners with flow experience and social presence elicited by MOOCs, while learners who use ArGF applications may also experience flow and social presence. According to the results of examining the structural model for the research model depicted in Figure 1, the second item pertaining to the ArGF construct has the highest standardized factor loadings (=0.925), hence, to evoke learners’ cognition in the context of their MOOCs usage, and further boost their MOOCs continuance intention and perceived impact on learning, MOOCs developers and educational practitioners are recommended to make effective utilization of the ArGF, which is particularly manifested in badges/medals upgrading, for example, MOOCs developers and educational practitioners may provide learners with higher-level badges/medals to encourage them to learn via the level-up mechanism, and they will have a chance to exchange senior badges/medals for exclusive rewards, such as “energy value.”

Table 8.

Results of the Bootstrapping Procedure between Gamification Features and Distance Learners’ Experiences.

Influential Path	Standardized Path Coefficient
	β	95% CI of Bias-Corrected Percentile Method			95% CI of Percentile Method
	β	Lower	Upper	Two-Tailed Test p-value	Lower	Upper	Two-Tailed Test p-value
ArGF → CogInv	0.695	0.546	0.809	p < .001	0.545	0.808	p < .001
ArGF → FE	0.248	0.082	0.432	p < .01	0.078	0.424	p < .01
ArGF → SP	0.185	0.046	0.352	p < .01	0.041	0.343	p < .01
IrGF → CogInv	0.223	0.066	0.366	p < .01	0.063	0.361	p < .01
IrGF → FE	0.497	0.388	0.698	p < .001	0.381	0.691	p < .001
IrGF → SP	0.151	0.051	0.266	p < .01	0.052	0.267	p < .01
SIrGF → CogInv	0.260	0.127	0.401	p < .001	0.116	0.398	p < .001
SIrGF → FE	0.167	0.065	0.293	p < .01	0.054	0.281	p < .01
SIrGF → SP	0.545	0.409	0.684	p < .001	0.406	0.681	p < .001

IrGF and Distance Learners’ Experiences

This study’s results reveal that learners’ perceived IrGF in MOOCs have a positive and evidently larger direct impact on their flow experience elicited by MOOCs than on their cognitive involvement and social presence, and 95% CIs of “IrGF→FE” do not overlap with those of “IrGF→CogInv” and “IrGF→SP” (see Table 8). Hence, it can be showed that learners’ perceived IrGF are positively connected with their flow experience elicited by MOOCs as hypothesized (H2), and learners’ interaction with IrGF in MOOCs is positively and more strongly associated with flow experience elicited by MOOCs than with cognitive involvement and social presence. The results imply that the designs of IrGF in MOOCs can often be immersive enough to evoke genuine emotional experiences (e.g., excitement, enthusiasm, and passion) of learners’ MOOCs usage, and this can further spur learners to explore and immerse themselves in the usage of MOOCs rather than facilitate them to enhance their sense of accomplishment and develop close social relationships with others during the MOOCs learning. According to the results of examining the structural model for the research model depicted in Figure 1, the second item pertaining to the IrGF construct has the highest standardized factor loadings (=0.908), hence, to induce learners’ emotion in the context of their MOOCs usage, and further promote their MOOCs continuance intention and perceived impact on learning, MOOCs developers and educational practitioners are suggested to think how to design various types of IrGF, which is especially manifested in customization, for example, MOOCs developers and educational practitioners may share in goal-setting to help learners develop motivation and reliability, or engage in self-assessment to assist learners in developing self-reflective abilities.

SIrGF and Distance Learners’ Experiences

This study’s results reveal that learners’ perceived SIrGF in MOOCs have a positive and evidently larger direct impact on their social presence elicited by MOOCs than on their cognitive involvement and flow experience, and 95% CIs of “SIrGF→SP” do not overlap with those of “SIrGF→CogInv” and “SIrGF→FE” (see Table 8). Hence, it can be certified that learners’ perceived SIrGF are positively associated with their social presence elicited by MOOCs as hypothesized (H3), and learners’ interaction with SIrGF in MOOCs is positively and more strongly connected with social presence elicited by MOOCs than with cognitive involvement and flow experience. The results imply that the designs of SIrGF in MOOCs can intrinsically motivate them to communicate and exchange information with others and mainly afford learners a sense of belonging to the MOOCs learning group, while learners who use SIrGF applications may also acquire a multifaceted experience of cognitive processes and an emotional experience from the winning state during the MOOCs learning. According to the results of examining the structural model for the research model depicted in Figure 1, the third item pertaining to the SIrGF construct has the highest standardized factor loadings (=0.934), hence, to encourage learners’ social connectedness in the context of their MOOCs usage, and further promote their MOOCs continuance intention and perceived impact on learning, MOOCs developers and educational practitioners are advised to consider launching various types of SIrGF, which is principally manifested in social networking features, for example, MOOCs developers and educational practitioners may launch the MOOC-centered learning communities via using social networking features to induce learners to participate in MOOCs, and further help them communicate and exchange information, and share ideas and views with more other learners, and facilitate learners’ perception of belonging sense.

Research Contributions and Theoretical Implications

As shown in Figure 2, the research model has good explanatory power and provides academics and practitioners with an all-round understanding of learners’ continuance intention of MOOCs and perceived impact on learning. This study makes three contributions to the extant literature. Detailed theoretical implications are proposed below.

As the first major theoretical contribution of this study, this study analyzes the impacts of the three types of gamification features including ArGF, IrGF, and SIrGF in MOOCs settings, and establishes a theoretical link between gamification features, internal experiences, and continuance intention of MOOCs. While previous studies have mostly considered social influences, media influences, or MOOCs features (e.g., Alraimi et al., 2015; Wu & Chen, 2017; Zhao et al., 2020), there is a dearth of evidence on the role of gamification features in triggering learners’ continuance intention of MOOCs and learning outcomes (Aparicio et al., 2019; Bai et al., 2020). Particularly, most studies (e.g., Hassan et al., 2019; Rodrigues et al., 2016) only consider gamification as a uni-dimensional construct, and extant research (e.g., Aparicio et al., 2019) has still failed to investigate whether and how different categories of gamification features may be able to affect continuance intention of MOOCs and learning outcome. Drawing upon the S-O-R model, this study decomposed gamification features into ArGF, IrGF, and SIrGF, and examined the relationships between different gamification features with three kinds of internal experiences (i.e., cognitive involvement, flow experience, and social presence), which could help explain the mechanisms of what kinds of the gamification features affected internal experiences and were more appropriate for enhancing continuance intention of MOOCs. This study implies that the research findings can reinterpret the role of gamification features including ArGF, IrGF, and SIrGF in the context of MOOCs continuance intention from the cognitive, affective, and social interaction theoretical perspectives.

Second, while educators always try to find ways to help learners acquire better performance impact via using the e-learning system (Ngan et al., 2018), to date, many research efforts for the e-learning system usage models still regard usage as an end in itself, and researchers also place considerably less focus on evaluating the performance impact of the e-learning system usage, which is a key topic for educational institutions if they intend to know whether the e-learning system implementations are successful (Cheng, 2019; McGill & Klobas, 2009). Hence, a further contribution of this study to the MOOCs literature is that it places considerably more emphasis upon the outcome of learners’ continuance intention of MOOCs in understanding learners’ perceived impact on learning greatly driven by their gamification features, internal experiences, and continuance intention of MOOCs, thus a positive assessment of performance impact should be taken into account in the theoretical development of learners’ continuance intention of MOOCs to acquire a more robust analysis.

Finally, the contribution of this study to the gamification literature is that it successfully uncovers the full mediating effects of learners’ internal experiences (i.e., cognitive involvement, flow experience, and social presence) and continuance intention of MOOCs in regard to how learners’ perceived gamification features (i.e., ArGF, IrGF, and SIrGF) affect their perceived impact on learning. More importantly, given the lack of further discussion around the role of gamification features in learners’ perceived impact on learning, the “gamification features → internal experiences → continuance intention → learning performance” model that this study proposed and validated is crucial for understanding the process of how gamification features can be used to drive learners’ perceived impact on learning in the MOOCs settings. Hence, this study first implies that designing various types of gamification features (i.e., ArGF, IrGF, and SIrGF) can be regarded as a significant contributing factor to the success of MOOCs. Not only can it encourage learners’ MOOCs continuance intention driven by their internal experiences, but it can further enhance learners’ learning performance. It is of particular interest to note that if learners may feel to the presence of the participants’ interactions in MOOCs learning environments, thus they will be intrinsically motivated, and they eventually tend to complete MOOCs, which is considered as a major challenge encountered in the MOOCs development (El Kabtane et al., 2019; Wu & Chen, 2017). Hence, this study’s results further imply that if learners can be intrinsically motivated by their interactions with gamification features, and thus this will trigger their MOOCs continuance intention, which can increase their learning performance, thereby reducing dropout rates.

Practical Implications and Suggestions

This study’s results can enhance the understanding for the impacts of gamified antecedents including ArGF, IrGF, and SIrGF to learners’ internal experiences on their continuance intention of MOOCs and perceived impact on learning in MOOCs learning settings, and provide educational practitioners who are faced with the challenge of learners’ discontinuing their MOOC usage with implications. Detailed practical implications are proposed below.

First, this study’s results reveal that learners’ perceived ArGF in MOOCs are the most crucial antecedents that can make the strongest direct effect on their cognitive involvement elicited by MOOCs and the most effective game features are those oriented towards ArGF in MOOCs (see Tables 7 and 8). The results imply that learners’ perceived ArGF in MOOCs can overwhelmingly lead them to experience feelings of competence, as this type of gamified features continuously provide them with a multifaceted experience of cognitive processes. Based on the potential of the ArGF, most gamified MOOCs have already included points, badges, and leaderboards, the so called PBL triad. To effectively evoke learners’ MOOCs continuance intention and boost their perceived impact on learning, in this regard, this study suggests that MOOCs developers and educational practitioners should consider designing MOOCs that further contain challenges, progress bars, or rankings, and provide learners with instantaneous feedback to enable them to monitor their progress and results, if they can reach higher levels that feature MOOCs learning activities with increased difficulty, thus they will perceive that their capacities are evolving.

Next, this study’s results reveal that learners’ perceived IrGF in MOOCs are the most overwhelming antecedents that can make the largest direct impact on their flow experience elicited by MOOCs (see Table 8). The results imply that learners’ perceived IrGF in MOOCs may vitally make them enthusiastically immerse themselves in the human-system interactions, this can be immersive enough to evoke genuine emotional experiences such as excitement, enthusiasm, and passion, and can further encourage learners to be more engaged in their continued MOOCs usage, which can be useful for enhancing their learning outcomes. Hence, to induce learners’ MOOCs continuance intention and promote their perceived impact on learning, this study suggests that MOOCs developers and educational practitioners may think how to design various types of IrGF (e.g., avatars, role-play mechanics, storytelling, narrative structures, and customization) to provide learners with immersive learning scenarios embedded in MOOCs to engage themselves in self-directed inquisitive behavior and empower them to pursue meaning via their actions.

Third, this study’s results reveal that learners’ perceived SIrGF in MOOCs are the most pivotal antecedents that can make the greatest direct influence on their social presence elicited by MOOCs (see Table 8). The results imply that learners’ perceived SIrGF in MOOCs may crucially make them experience social relatedness to achieve their common goals via competing with other learners, introducing social networking features to gamified MOOCs, or cooperating with other learners which can create a sense of belonging to a learning team within the MOOCs learning environment. Thus, to trigger learners’ MOOCs continuance intention and enhance their perceived impact on learning, this study suggests that MOOCs developers and educational practitioners may consider launching MOOC-centered learning communities via using social networking features to induce learners to participate in MOOCs, and develop various types of SIrGF embedded in MOOCs to enable learners to induce their classmates to join such learning communities, and to share their achievements publicly to facilitate participants’ perception of connectedness including care, recognition, and belonging sense, which will bring more learners to participate in MOOCs.

However, it is especially worth mentioning that the IS/IT developers should always remind themselves that their gamification efforts cannot be too complicated or overwhelm their users with a plethora of features since many users have no game experience (Bitrián et al., 2021; Xi & Hamari, 2020; Zhang et al., 2021). In the MOOCs learning settings, if there is more gamification features (e.g., interactivity, functionality) than the learners can take, the MOOCs may be unable to keep learners’ attention on learning for a long time. That is, MOOCs developers should ensure that their MOOCs design is easy for learners to pick up, but challenging enough for them to keep coming back. Hence, this study suggests that MOOCs developers and educational practitioners should be cautious about the level of the gamification features to allow for the constraints of gamification in the learners’ experience.

Conclusions

Drawing on gamification literature, this study develops a research model to examine whether gamification features as environmental stimulus antecedents to learners’ organismic experiences in using MOOCs can affect their responses on MOOCs and learning outcomes. The proposed research model, rooted in the S-O-R framework, provides a strong foundation for understanding these relationships, and the empirical analysis is conducted using SEM by taking cross-sectional questionnaire survey data, and with a quantitative empirical approach. This study verified that all of the gamification features including ArGF, IrGF, and SIrGF positively influenced learners’ internal experiences in using MOOCs (i.e., cognitive involvement, flow experience, and social presence), which jointly expounded their continuance intention of MOOCs, and this in turn enhanced their perceived impact on learning. Overall, these results offer enough evidence to strongly support all of the hypothesized links and the research model that this study proposed. Besides, the results of the mediation analysis confirmed that learners’ internal experiences and continuance intention of MOOCs fully mediated the effects of their perceived gamification features on perceived impact on learning. Accordingly, the “gamification features → internal experiences → continuance intention → learning performance” model this study proposed and validated is crucial for understanding the process of how gamification features can be utilized to drive learners’ perceived impact on learning within the MOOCs learning environment.

Limitations and Further Research

While this study provides academics and practitioners with salient theoretical and practical implications, there are some limitations that may open avenues for further research. First, there might still be a possibility of CMB due to the limitation of self-reported measures, while all this study’s efforts to statistically check CMB. Further research may combine with qualitative methods against data collected from different sources to obtain more complete interpretations of learners’ MOOCs learning behaviors and outcomes. Next, this study measured the gamification features including ArGF, IrGF, and SIrGF via investigating how participants would evaluate the degree of the effect of the gamification elements on motivating their usage of gamified MOOCs, because it was difficult for participants to answer the frequency at which they interacted with each gamification element. Further research may measure the gamification features via investigating the frequency at which participants interact with each gamification element. Third, this study measured learners’ learning performance via the subjective measure of perceived impact on MOOCs learning. Further research may examine whether these learners actually enhanced their learning outcomes within the MOOCs learning environment via additional measures such as student grades and qualitative data. Fourth, this study’s sampling frame was taken from among learners who had experience in taking the gamified MOOCs provided by the MOOC platform launched by a well-known university in Taiwan, which tends to be a collectivistic culture, thus there might be some cultural influences on participants’ behaviors. Further research may conduct the comparative studies among different cultures to increase the generalizability of the findings. Fifth, the data were only collected based on the gamified MOOCs in this study. Future research may apply this study’s research model using other gamified services (e.g., gamified mobile applications, gamification in healthcare, or gamification for financial services) as their research contexts. Sixth, this study’s data were collected using a cross-sectional questionnaire survey. Thus, it would be desirable if further research may use longitudinal data to capture the dynamic process of learners’ continuance intention of MOOCs and perceived impact on learning in the long term. Finally, learners’ perceived interactions within MOOCs will eventually encourage them to complete MOOCs, which is regarded as a main problem in MOOCs (El Kabtane et al., 2019; Wu & Chen, 2017). Thus, this study implies that interacting with gamification features to intrinsically motivate learners can drive their continuance intention of MOOCs, and further enhance their perceived impact on learning, thereby reducing dropout rates. Hence, this study’s results may be of interest to the general public. However, the completion rates for most MOOCs remain low (Dai, Teo, & Rappa, 2020; El Kabtane et al., 2019; Zhao et al., 2020). Given such a unique feature of MOOCs, it will also be interesting for further research to explore whether learners’ perceptions of gamification features in MOOCs can enhance MOOCs completion rates.

Footnotes

Acknowledgments

The author would like to thank the Editor and anonymous reviewers for their insightful comments and valuable suggestions.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Yung-Ming Cheng

Author Biography

Yung-Ming Cheng is a Professor in the Department of Business Administration at Chaoyang University of Technology, Taiwan. He obtained his Bachelor’s degree from National Chiao Tung University, Taiwan, and He received his Master’s and PhD degrees from National Cheng Kung University, Taiwan. His research interests include technology acceptance and adoption, information systems management and applications, international entry mode strategies, competitive strategies in high-technology industries, electronic commerce, service quality and service recovery, health care management, and international human resource management. His research has been published in journals such as Information Systems Journal, Internet Research, Journal of Enterprise Information Management, Information Technology & People, Journal of Educational Technology & Society, Aslib Journal of Information Management, Education + Training, Nurse Education Today, Kybernetes, The Journal of Nursing Research, Canadian Journal of Administrative Sciences, Benchmarking, Journal of Asia Business Studies, Asia Pacific Management Review, Journal of Systems and Information Technology, International Journal of Web Information Systems, International Journal of Commerce & Management, and others. Yung-Ming Cheng can be contacted at: ymcheng@mail.cyut.edu.tw

References

Alraimi

K. M.

Ciganek

A. P.

(2015). Understanding the MOOCs continuance: The role of openness and reputation. Computers & Education, 80(January), 28–38. https://doi.org/10.1016/j.compedu.2014.08.006

Anderson

J. C.

Gerbing

D. W.

(1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411

Aparicio

Oliveira

Bacao

Painho

(2019). Gamification: A key determinant of massive open online course (MOOC) success. Information & Management, 56(1), 39–54. https://doi.org/10.1016/j.im.2018.06.003

Armstrong

J. S.

Overton

T. S.

(1977). Estimating non-response bias in mail surveys. Journal of Marketing Research, 14(3), 396–402. https://doi.org/10.1177/002224377701400320

Arning

Ziefle

(2007). Understanding age differences in PDA acceptance and performance. Computers in Human Behavior, 23(6), 2904–2927. https://doi.org/10.1016/j.chb.2006.06.005

Arpaci

Al-Emran

Al-Sharafi

M. A.

(2020). The impact of knowledge management practices on the acceptance of massive open online courses (MOOCs) by engineering students: A cross-cultural comparison. Telematics and Informatics, 54(November), 101468. https://doi.org/10.1016/j.tele.2020.101468

Aşıksoy

(2018). The effects of the gamified flipped classroom environment (GFCE) on students’ motivation, learning achievements and perception in a physics course. Quality & Quantity: International Journal of Methodology, 52(1), 129–145. https://doi.org/10.1007/s11135-017-0597-1

Bagozzi

R. P.

(1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. https://doi.org/10.1007/bf02723327

Bai

Hew

K. F.

Huang

(2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30(June), 100322. https://doi.org/10.1016/j.edurev.2020.100322

10.

Bhattacherjee

(2001). Understanding information systems continuance: An expectation–confirmation model. MIS Quarterly, 25(3), 351–370. https://doi.org/10.2307/3250921

11.

Bitrián

Buil

Catalán

(2021). Enhancing user engagement: The role of gamification in mobile apps. Journal of Business Research, 132(August), 170–185. https://doi.org/10.1016/j.jbusres.2021.04.028

12.

Bollen

K. A.

(1989). Structural equations with latent variables. John Wiley & Sons.

13.

Campbell

D. E.

Wright

R. T.

Clay

P. F.

(2010). Deconstructing and operationalizing interactivity: An online advertising perspective. Journal of Information Technology Theory and Application, 11(4), 29–53.

14.

Castellanos-Reyes

(2021). The dynamics of a MOOC’s learner-learner interaction over time: A longitudinal network analysis. Computers in Human Behavior, 123(3), 106880. https://doi.org/10.1016/j.chb.2021.106880

15.

Chang

C. C.

(2013). Examining users’ intention to continue using social network games: A flow experience perspective. Telematics and Informatics, 30(4), 311–321. https://doi.org/10.1016/j.tele.2012.10.006

16.

Chang

J. W.

Wie

H. Y.

(2016). Exploring engaging gamification mechanics in massive online open courses. Journal of Educational Technology & Society, 19(2), 177–203.

17.

Chen

S.-Y.

(2016). Using the sustainable modified TAM and TPB to analyze the effects of perceived green value on loyalty to a public bike system. Transportation Research Part A: Policy and Practice, 88(June), 58–72. https://doi.org/10.1016/j.tra.2016.03.008

18.

Cheng

Y.-M.

(2019). How does task-technology fit influence cloud-based e-learning continuance and impact? Education + Training, 61(4), 480–499. https://doi.org/10.1108/et-09-2018-0203

19.

Cheng

Y.-M.

(2021). Can gamification and interface design aesthetics lead to MOOCs’ success? Education + Training, 63(9), 1346–1375. https://doi.org/10.1108/et-09-2020-0278

20.

Cheung

C. M. K.

Chiu

P. Y.

Lee

M. K. O.

(2011). Online social networks: Why do students use Facebook? Computers in Human Behavior, 27(4), 1337–1343. https://doi.org/10.1016/j.chb.2010.07.028

21.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/bf02310555

22.

Cronbach

L. J.

(1971). Test validation. In Thorndike

R. L.

(Ed.), Educational measurement (2nd ed., pp. 443–507). American Council on Education.

23.

Csikszentmihalyi

(1975). Beyond boredom and anxiety: Experiencing flow in work and play. Jossey-Bass Publishers.

24.

Dai

H. M.

Teo

Rappa

N. A.

(2020a). Understanding continuance intention among MOOC participants: The role of habit and MOOC performance. Computers in Human Behavior, 112(November), 106455. https://doi.org/10.1016/j.chb.2020.106455

25.

Dai

H. M.

Teo

Rappa

N. A.

Huang

(2020b). Explaining Chinese university students’ continuance learning intention in the MOOC setting: A modified expectation confirmation model perspective. Computers & Education, 150(1), 103850. https://doi.org/10.1016/j.compedu.2020.103850

26.

Dellarocas

Van Alstyne

(2013). Money models for MOOCs. Communications of the ACM, 56(8), 25–28. https://doi.org/10.1145/2492007.2492017

27.

Deng

Benckendorff

Gannaway

(2019). Progress and new directions for teaching and learning in MOOCs. Computers & Education, 129(February), 48–60. https://doi.org/10.1016/j.compedu.2018.10.019

28.

Deterding

Dixon

Khaled

Nacke

(2011). From game design elements to gamefulness: Defining gamification. In Lugmayr

E. A.

(Ed.), Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9–15), Tampere, Finland, September 29–30, 2011. ACM. https://doi.org/10.1145/2181037.2181040

29.

Dichev

Dicheva

(2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14(9), 1–36. https://doi.org/10.1186/s41239-017-0042-5

30.

Drossos

D. A.

Kokkinaki

Giaglis

G. M.

Fouskas

K. G.

(2014). The effects of product involvement and impulse buying on purchase intentions in mobile text advertising. Electronic Commerce Research and Applications, 13(6), 423–430. https://doi.org/10.1016/j.elerap.2014.08.003

31.

El Kabtane

El Adnani

Sadgal

Mourdi

(2019). Augmented reality-based approach for interactivity in MOOCs. International Journal of Web Information Systems, 15(2), 134–154. https://doi.org/10.1108/ijwis-04-2018-0033

32.

Fianu

Blewett

Ampong

G. O.

(2020). Toward the development of a model of student usage of MOOCs. Education + Training, 62(5), 521–541. https://doi.org/10.1108/et-11-2019-0262

33.

Fornell

Larcker

D. F.

(1981). Evaluating structural equation models with unobservable and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312

34.

Fortin

D. R.

Dholakia

R. R.

(2005). Interactivity and vividness effects on social presence and involvement with a web-based advertisement. Journal of Business Research, 58(3), 387–396. https://doi.org/10.1016/s0148-2963(03)00106-1

35.

F.-L.

Y.-L.

H.-C.

(2009). An investigation of coopetitive pedagogic design for knowledge creation in web-based learning. Computers & Education, 53(3), 550–562. https://doi.org/10.1016/j.compedu.2009.01.004

36.

Gefen

Straub

D. W.

(2004). Consumer trust in B2C e-commerce and the importance of social presence: Experiments in e-products and e-services. Omega, 32(6), 407–424. https://doi.org/10.1016/j.omega.2004.01.006

37.

Goodhue

D. L.

Thompson

R. L.

(1995). Task-technology fit and individual performance. MIS Quarterly, 19(2), 213–236. https://doi.org/10.2307/249689

38.

Guo

Y. M.

Poole

M. S.

(2009). Antecedents of flow in online shopping: A test of alternative models. Information Systems Journal, 19(4), 369–390. https://doi.org/10.1111/j.1365-2575.2007.00292.x

39.

Hair

J. F.

Jr. Black

W. C.

Babin

B. J.

Anderson

R. E.

(2010). Multivariate data analysis: A global perspective (7th ed.). Pearson Education.

40.

Harman

H. H.

(1976). Modern factor analysis (3rd ed. revised). The University of Chicago Press.

41.

Hassan

Dias

Hamari

(2019). How motivational feedback increases user’s benefits and continued use: A study on gamification, quantified-self and social networking. International Journal of Information Management, 46(June), 151–162. https://doi.org/10.1016/j.ijinfomgt.2018.12.004

42.

Hassanein

Head

M. M.

(2006). The impact of infusing social presence in the web interface: An investigation across product types. International Journal of Electronic Commerce, 10(2), 31–55. https://doi.org/10.2753/jec1086-4415100202

43.

Hayes

A. F.

(2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420. https://doi.org/10.1080/03637750903310360

44.

Hsu

J.-Y.

Chen

C.-C.

Ting

P.-F.

(2018). Understanding MOOC continuance: An empirical examination of social support theory. Interactive Learning Environments, 26(8), 1100–1118. https://doi.org/10.1080/10494820.2018.1446990

45.

Hsu

Y.-P.

(2021). Technological innovation and massive open online courses (MOOCs) in Taiwan higher education. In Hou

A.Y.-C.

Chiang

T.-L.

Chan

S.-J.

(Eds.), Higher education in Taiwan (pp. 183–197). Springer-Verlag. https://doi.org/10.1007/978-981-15-4554-2_10

46.

Huang

(2012). Online experiences and virtual goods purchase intention. Internet Research, 22(3), 252–274. https://doi.org/10.1108/10662241211235644

47.

Huang

Zhang

Liu

(2017). Antecedents of student MOOC revisit intention: Moderation effect of course difficulty. International Journal of Information Management, 37(2), 84–91. https://doi.org/10.1016/j.ijinfomgt.2016.12.002

48.

Hunicke

LeBlanc

Zubek

(2004). Mda: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI (pp. 17–22), July 25–26, 2004. San Jose McEnery Convention Center.

49.

Jang

Kitchen

P. J.

Kim

(2018). The effects of gamified customer benefits and characteristics on behavioral engagement and purchase: Evidence from mobile exercise application uses. Journal of Business Research, 92(November), 250–259. https://doi.org/10.1016/j.jbusres.2018.07.056

50.

Jiang

Chan

Tan

B. C.

Chua

W. S.

(2010). Effects of interactivity on website involvement and purchase intention. Journal of the Association for Information Systems, 11(1), 34–59. https://doi.org/10.17705/1jais.00218

51.

Kang

J.-Y. M.

Mun

J. M.

Johnson

K. K. P.

(2015). In-store mobile usage: Downloading and usage intention toward mobile location-based retail apps. Computers in Human Behavior, 46(May), 210–217. https://doi.org/10.1016/j.chb.2015.01.012

52.

Kasurinen

Knutas

(2018). Publication trends in gamification: A systematic mapping study. Computer Science Review, 27(February), 33–44. https://doi.org/10.1016/j.cosrev.2017.10.003

53.

Khalil

Wong

de Koning

Ebner

Paas

(2018). Gamification in MOOCs: A review of the state of the art. In Proceedings of the 2018 IEEE global engineering education conference (EDUCON) (pp. 1629–1638), Santa Cruz de Tenerife, Canary Islands, April 17–20, 2018. IEEE.

54.

Kiili

(2005a). Content creation challenges and flow experience in educational games: The IT-emperor case. The Internet and Higher Education, 8(3), 183–198. https://doi.org/10.1016/j.iheduc.2005.06.001

55.

Kiili

(2005b). Digital games-based learning: Towards an experiential gaming model. The Internet and Higher Education, 8(1), 13–24. https://doi.org/10.1016/j.iheduc.2004.12.001

56.

Kim

Sung

(2009). Dimensions of purchase-decision involvement: Affective and cognitive involvement in product and brand. Journal of Brand Management, 16(8), 504–519. https://doi.org/10.1057/bm.2008.39

57.

Kline

R. B.

(2011). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.

58.

Koivisto

Hamari

(2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45(April), 191–210. https://doi.org/10.1016/j.ijinfomgt.2018.10.013

59.

Lambert

S. R.

(2020). Do MOOCs contribute to student equity and social inclusion? A systematic review 2014–18. Computers & Education, 145(February), 103693. https://doi.org/10.1016/j.compedu.2019.103693

60.

Wang

Tan

S. C.

(2018). What makes MOOC users persist in completing MOOCs? A perspective from network externalities and human factors. Computers in Human Behavior, 85(August), 385–395. https://doi.org/10.1016/j.chb.2018.04.028

61.

Lin

F.-R.

Kao

C.-M.

(2018). Mental effort detection using EEG data in E-learning contexts. Computers & Education, 122(July), 63–79. https://doi.org/10.1016/j.compedu.2018.03.020

62.

Liyanagunawardena

T. R.

Adams

A. A.

Williams

S. A.

(2013). MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distributed Learning, 14(3), 202–227. https://doi.org/10.19173/irrodl.v14i3.1455

63.

Mardia

K. V.

(1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519–530. https://doi.org/10.1093/biomet/57.3.519

64.

McGill

T. J.

Klobas

J. E.

(2009). A task–technology fit view of learning management system impact. Computers & Education, 52(2), 496–508. https://doi.org/10.1016/j.compedu.2008.10.002

65.

Mehrabian

Russell

J. A.

(1974). An approach to environmental psychology. The MIT Press.

66.

Metwally

A. H. S.

Nacke

L. E.

Chang

Wang

Yousef

A. M. F.

(2021). Revealing the hotspots of educational gamification: An umbrella review. International Journal of Educational Research, 109(August), 101832. https://doi.org/10.1016/j.ijer.2021.101832

67.

Michinov

Brunot

Le Bohec

Juhel

Delaval

(2011). Procrastination, participation, and performance in online learning environments. Computers & Education, 56(1), 243–252. https://doi.org/10.1016/j.compedu.2010.07.025

68.

Ngan

O. M. Y.

Tang

T. L. H.

Chan

A. K. Y.

Chen

D. M.

Tang

M. K.

(2018). Blended Learning in anatomy teaching for non-medical students: An innovative approach to the health professions education. Health Professions Education, 4(2), 149–158. https://doi.org/10.1016/j.hpe.2017.11.001

69.

Novak

T. P.

Hoffman

D. L.

Yung

Y. F.

(2000). Measuring the customer experience in online environments: A structural modeling approach. Marketing Science, 19(1), 22–44. https://doi.org/10.1287/mksc.19.1.22.15184

70.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). McGraw-Hill.

71.

Pavlou

P. A.

Liang

Xue

(2007). Understanding and mitigating uncertainty in online exchange relationships: A principal-agent perspective. MIS Quarterly, 31(1), 105–136. https://doi.org/10.2307/25148783

72.

Pérez-López

Contero

(2013). Delivering educational multimedia contents through an augmented reality application: A case study on its impact on knowledge acquisition and retention. Turkish Online Journal of Educational Technology, 12(4), 19–28.

73.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J.-Y.

Podsakoff

N. P.

(2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879

74.

Poondej

Lerdpornkulrat

(2020). Gamification in e-learning: A moodle implementation and its effect on student engagement and performance. Interactive Technology and Smart Education, 17(1), 56–66. https://doi.org/10.1108/itse-06-2019-0030

75.

Preacher

K. J.

Hayes

A. F.

(2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40(3), 879–891. https://doi.org/10.3758/brm.40.3.879

76.

Reychav

(2015). Are your users actively involved? A cognitive absorption perspective in mobile training. Computers in Human Behavior, 44(March), 335–346. https://doi.org/10.1016/j.chb.2014.09.021

77.

Rodrigues

L. F.

Oliveira

Costa

C. J.

(2016). Does ease-of-use contributes to the perception of enjoyment? A case of gamification in e-banking. Computers in Human Behavior, 61(August), 114–126. https://doi.org/10.1016/j.chb.2016.03.015

78.

Ryan

R. M.

Deci

E. L.

(2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020

79.

Sanchez

J. I.

Korbin

W. P.

Viscarra

D. M.

(1995). Corporate support in the aftermath of a natural disaster: Effects on employee strains. Academy of Management Journal, 38(2), 504–521. https://doi.org/10.5465/256690

80.

Saputro

R. E.

Salam

Zakaria

M. H.

Anwar

(2019). A gamification framework to enhance students’ intrinsic motivation on MOOC. Telkomnika, 17(1), 170–178. https://doi.org/10.12928/telkomnika.v17i1.10090

81.

Shao

Chen

(2021). Understanding individuals’ engagement and continuance intention of MOOCs: The effect of interactivity and the role of gender. Internet Research, 31(4), 1262–1289. https://doi.org/10.1108/intr-10-2019-0416

82.

Short

Williams

Christie

(1976). The social psychology of telecommunications. Wiley.

83.

Smith

J. A.

Sivo

S. A.

(2012). Predicting continued use of online teacher professional development and the influence of social presence and sociability. British Journal of Educational Technology, 43(6), 871–882. https://doi.org/10.1111/j.1467-8535.2011.01223.x

84.

Triantoro

Gopal

Benbunan-Fich

Lang

(2019). Would you like to play? A comparison of a gamified survey with a traditional online survey method. International Journal of Information Management, 49(3), 242–252. https://doi.org/10.1016/j.ijinfomgt.2019.06.001

85.

Tsai

Y.-H.

Lin

C.-H.

Hong

J.-C.

Tai

K.-H.

(2018). The effects of metacognition on online learning interest and continuance to learn with MOOCs. Computers & Education, 121(June), 18–29. https://doi.org/10.1016/j.compedu.2018.02.011

86.

Chen

(2017). Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Computers in Human Behavior, 67(February), 221–232. https://doi.org/10.1016/j.chb.2016.10.028

87.

Hamari

(2020). Does gamification affect brand engagement and equity? A study in online brand communities. Journal of Business Research, 109(March), 449–460. https://doi.org/10.1016/j.jbusres.2019.11.058

88.

Yang

S. J. H.

Huang

J. C. H.

Huang

A. Y. Q.

(2017). MOOCs in Taiwan: The movement and experiences. In Jemni

Kinshuk Khribi

M. K.

(Eds.), Open education: From OERs to MOOCs (pp. 101–116). VerlagSpringer.

89.

Yang

S. J. H.

Huang

J. C. H.

Huang

A. Y. Q.

Chen

I. Y. L.

(2016). MOOCs for K-12 and higher education in Taiwan. In 2016 5th IIAI international congress on advanced applied informatics (IIAI-AAI), Kumamoto, Japan, 10–14 July 2016. IEEE. https://doi.org/10.1109/IIAI-AAI.2016.42

90.

Yousef

A. M. F.

Sumner

(2021). Reflections on the last decade of MOOC research. Computer Applications in Engineering Education, 29(4), 648–665. https://doi.org/10.1002/cae.22334

91.

Zaichkowsky

J. L.

(1985). Measuring the involvement construct. Journal of Consumer Research, 12(3), 341–352. https://doi.org/10.1086/208520

92.

Zaichkowsky

J. L.

(1994). The personal involvement inventory: Reduction, revision, and application to advertising. Journal of Advertising, 23(4), 59–70. https://doi.org/10.1080/00913367.1943.10673459

93.

Zainuddin

Chu

S. K. W.

Shujahat

Perera

C. J.

(2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30(June), 100326. https://doi.org/10.1016/j.edurev.2020.100326

94.

Zhang

Gupta

Zhao

(2014). What motivates customers to participate in social commerce? The impact of technological environments and virtual customer experiences. Information & Management, 51(8), 1017–1030. https://doi.org/10.1016/j.im.2014.07.005

95.

Zhang

Shao

Feng

(2021). Gamification and online impulse buying: The moderating effect of gender and age. International Journal of Information Management, 61(December), 102267. https://doi.org/10.1016/j.ijinfomgt.2020.102267

96.

Zhao

Yang

Shu

Liu

(2021). Sustainability orientation, the adoption of 3D printing technologies, and new product performance: A cross-institutional study of American and Indian firms. Technovation, 101(March), 102197. https://doi.org/10.1016/j.technovation.2020.102197

97.

Zhao

Wang

Sun

(2020). Technological environment, virtual experience, and MOOC continuance: A stimulus–organism–response perspective. Computers & Education, 144(January), 103721. https://doi.org/10.1016/j.compedu.2019.103721