The Role of Creative Communications and Gamification in Student Engagement in Higher Education: A Sentiment Analysis Approach

Abstract

This article aims to explore gamification tools in services and higher education environments and their role in creating student engagement. The research adopts a qualitative method based on quasi-experimental design. Participants from a higher education institution are exposed to gamification activities during a full semester. Researchers use the sentiment analysis (SA) technique based on a text-mining approach to analyse the data. Findings reveal that participants perceive gamification in service settings as a useful tool. The global SA reveals a positive sentiment about the gamification approach that contributes to increasing participants’ engagement. This study’s novelty arises from quasi-experimental research to measure gamification activities’ impact on students’ engagement, measured through SA of their opinions.

Keywords

Gamification gamification-based tools engagement behaviours student engagement Kahoot!

Introduction

This article aims to understand the role of gamification in student engagement in a higher education setting, following recent calls for further research about the role of technology development on stakeholders engagement (Leonidou et al., 2018; Loureiro et al., 2020c; Watson et al., 2018) and customer engagement (Bilro & Loureiro, 2020; Hollebeek et al., 2020). Student engagement may be defined as behavioural and psychological involvement in learning activities (Appleton et al., 2008). The literature already addresses engaged students as an active part in the process of searching, sharing and learning new skills and theories about a topic (e.g., Damnjanovic et al., 2020; Iosup & Epema, 2014), especially when using gamification tools in educational environments (e.g., Faiella & Ricciardi, 2015; Galbis-Córdova et al., 2017; Looyestyn et al., 2017). However, pieces of evidence about how the use of gamification can successfully create engagement in participants of a service environment, such as higher education service, are still scarce (Kayimbaşioǧlu et al., 2016; Martín-Gutiérrez et al., 2017).

The academic discussion on the topic has focused the attention on gamification as a way to increase attachment (Oleksy & Wnuk, 2017), to build co-creation experiences (Nobre & Ferreira, 2017), to foster the intention of engagement and brand attitude (Lim & Puspita, 2020; Yang et al., 2017) or to develop interaction and participation at higher education environments (Piñeiro-Otero & Costa-Sánchez, 2015). Contrariwise, several authors have discussed the subversive ludification of society that gamification may represent (e.g., Fuchs, 2018; Woodcock & Johnson, 2017). The adoption and practice of game-based techniques or gamification in higher educational environments may promote positive behaviour changes, leading to increased motivation, relevance or immersion (Kapp, 2012; Kapp & Coné, 2012). However, gamification’s use as a tool to leverage student engagement in services is still an under-explored topic (e.g., Leclercq et al., 2018; Rosado-Pinto & Loureiro, 2020).

This article adopts a quasi-experimental approach to test the causal effects between an intervention and its outcome (Harris et al., 2006), aiming to answer the following research question: Does gamification tools influence students’ engagement in services, such as higher education settings? Specifically, this research aims to shed light on gamification’s implications in students’ engagement in higher education environments, analysing the data through a sentiment analysis (SA) technique based on a text-mining approach. For that, authors lay hands to stimulus-organism-response (SOR) theoretical framework (Eroglu et al., 2003; Mehrabian & Russell, 1974; Roschk et al., 2017). This research shows gamification’s potential outside online environments and in distinct practical application areas, collecting evidence of its positive impact in service settings. The findings unveil that students understood the experience as an educational element, and only after as entertainment. The findings also show that our participants’ overall experience was positive or very positive and that participants feel engaged by using gamification tools in service settings such as educational environments.

The article’s contribution is twofold. First, the findings add to the existing stakeholder and engagement literature, namely on services and technology transformation topics. Second, the authors claim that services, specifically higher education institutions (HEIs), can adopt new technological approaches such as gamification to improve students’ service (i.e., education). As far as authors are aware, this article is one of the pioneers to further understand students’ engagement process through technological tools such as gamification in an extended period (such as a semester) in an actual/physical environment.

The remainder of this article is structured as follows. The article reviews the relevant literature on gamification in educational settings and student engagement in the next section. Next, the authors present the research methodology and the results of our study. Finally, we discuss the findings and this article’s theoretical and managerial implications, along with research limitations and opportunities for further research.

Theoretical Background

Gamification and Its Use in Educational Settings

The gamification concept is still in its early days and open to a more established definition (Hamari et al., 2014). The Oxford dictionary describes gamification as applying typical game playing elements (such as competition with others, point scoring, among others) to distinct fields (Oxford, 2019). Literature defines gamification as the use of game design elements in non-game contexts (Deterding et al., 2011; Priya & Kalpana, 2014), by incorporating game elements into non-game settings, or by using game mechanics, dynamics and frameworks to encourage specific behaviours (Ip et al., 2011; Pagowsky, 2012; Sheldon, 2012; Stott & Neustaedter, 2013). Additionally, other authors consider gamification more broadly and describe it as the concept of using game-based processes and game thinking to involve people, to motivate actions, to promote learning processes, or to solve problems (Chaudhary, 2010; Kapp, 2012).

In this article, the authors adopt the definition of Deterding et al. (2011, p. 11), who claim that gamification is ‘the use of game design elements in non-game contexts’. It is one of those non-game contexts where gamification techniques are evolving is the educational setting, specifically the Higher Education (e.g., Faghihi et al., 2014; Galbis-Córdova et al., 2017). In this article, we also accept Robson and colleagues’ definition of gamification: ‘The application of lessons from the gaming domain to change behaviours in non-game situations’ (Robson et al., 2015, p. 2). Both definitions point out how to use gamification in different surroundings, such as HEI.

Various studies have observed participants’ behavioural changes in gamified environments (Barata et al., 2017; Koivisto & Hamari, 2014; Turan et al., 2016), with several others reinforcing the positive aspects of gamification’s application at the HEI level (Subhash & Cudney, 2018; Yildirim, 2017). Still, since its beginning, gamification has sparked controversy between academics and practitioners devoted to human-computer interaction (Mahnič, 2014; Woodcock & Johnson, 2017). This controversy is reflected in some literature addressing the topic, putting in evidence that gamification’s effect on motivation or participation is lower than the expectations created by the hype (Broer, 2014; Fuchs, 2018), or not showing a positive relationship between gamification tools usage and improved success (Frost et al., 2015). Although gamification in educational settings is a growing phenomenon, the literature reveals that not sufficient evidence exists to support the long-term benefits of gamification in educational contexts (Dichev & Dicheva, 2017) and that the knowledge about the gamification outcome in an educational context is still scarce (Dichev & Dicheva, 2017; Dicheva et al., 2015). However, the levels of motivation and engagement among individuals joining gamified scenarios seem to increase, leading to improved performance and positive results (Looyestyn et al., 2017).

This study intends to contribute to the ongoing debate shedding light on gamification’s role in students’ engagement. For that, authors lay hands to SOR theoretical framework (Eroglu et al., 2003; Roschk et al., 2017) initially proposed by (Mehrabian & Russell, 1974). The framework in use represents the role of stimuli (i.e., in-class gamification activities) in influencing consumers’ emotional and cognitive states (i.e. the organism), which, in turn, result in approach or avoidance behaviour (i.e. student engagement).

Students’ Engagement in Gamified Environments

The literature already shows the relevance of distinct stakeholders’ engagement, such as consumers, shareholders, providers or students, from various viewpoints (Loureiro et al, 2020c), and engagement between companies and consumers (e.g., Brodie et al., 2011; Hollebeek et al., 2014; Loureiro et al., 2020b; Sprott et al., 2009). The topic of engagement is also discussed and studied in other areas such as psychology, sociology or education (Garczynski et al., 2013; Loureiro et al., 2020a; Morimoto & Friedland, 2013). Literature defines student engagement as a behavioural and psychological involvement in learning activities (Appleton et al., 2008). Student engagement is also seen as a multidimensional construct, comprising three dimensions: cognitive engagement, behavioural engagement and emotional/affective engagement (Fredricks et al., 2004; Jimerson et al., 2003).

First, cognitive engagement can be recognised as expending additional effort to understand multifaceted concepts and/or overcome upscaled expertise (Finn & Zimmer, 2012; Fredricks et al., 2004). Diverse authors evidence the relevance of the cognitive engagement dimension in many online contexts (e.g., Burkey, 2019; Ghosh, 2019; Putman et al., 2012; Zhu, 2006). Cognitive engagement in an online context can be perceived as the attention and effort students apply to interact with peers and tutors in discussions, comments or posts (Bilro et al., 2018; Putman et al., 2012). Moreover, it includes students’ upscale expertise, such as analysing, reviewing or reasoning (Putman et al., 2012; Shukla & Sharma, 2018; Zhu, 2006). Second, behavioural engagement can usually be perceived in technological contexts based on discussions or replies (Amegbe et al., 2017; Bilro & Loureiro, 2020; Leclercq et al., 2018). The literature debates if the number of interactions among peers is a measure of behaviour engagement (Goggins & Xing, 2016) and the linkage between discussions among students and achievements (Ramos & Yudko, 2008). Lastly, the literature defines emotional engagement as the students’ psychological response to academic settings, such as boredom or enjoyment from the learning activities (Finn & Zimmer, 2012). The relationship between tutors and students, the connection between students and their peers, and students’ interests or enjoyment in making part of the online discussions also influences this dimension (Fredricks et al., 2004).

Methods

Qualitative research based on a quasi-experimental design is adopted in this study, as we aim to demonstrate causality between an intervention and its outcome (Harris et al., 2006). To operationalise the study, we conduct a quasi-experimental design without a control group (Shadish et al., 2002) and expose 91 undergraduate students to various gamification moments during a full semester in a specific curricular unit. Participants were all management degree students, ranging from 19 to 25 years old, and gender balance was achieved (57.14% female). We have built and assess a services scenario using a gamification tool, Kahoot!, ‘a game-based learning platform that makes it easy to create, share and play fun learning games or trivia quizzes in minutes. Users can play Kahoot! on any mobile device or computer with an internet connection.’ (Kahoot!, 2019). During the semester, researchers ask participants to use Kahoot! in class, as final modules quizzes, as an assessment tool for other colleagues’ group presentations or others. At no moment during the semester was explained to students why they were using Kahoot!. The goal was explained only at the end of the semester, and students opinions were collected. Researchers collected the data through one single open-ended question made available on Qualtrics, assessing their opinion about the advantages and disadvantages of this type of gamification-based tool: ‘What is your opinion about the use of Kahoot! inside the classroom? Which are the main advantages and disadvantages that you see in using this type of Gamification tools?’ Participants were first asked to provide informed consent, fill out a pre-experiment questionnaire and anonymity was guaranteed. Students access the form during a class through their own devices, answering individually. Researchers also conducted a post-experimental debrief to participants, as we are dealing with experience-based learning activities (Stewart, 1992). From a total of 91 participants, it was possible to collect 73 valid answers after blank responses or other non-suitable answers were eliminated (80.2% success rate). The sample size is considered adequate (Simmons et al., 2013) and in line with the current practise (e.g., Javornik, 2016; Zhao & Patrick Rau, 2020).

To extract, examine and transpose the vast amount of information into valuable knowledge, it became necessary to proceed with a text mining technique (Fan et al., 2006; Költringer & Dickinger, 2015; Srivastava & Sahami, 2009). Researchers performed a SA procedure to understand the participants’ sentiments and attitudes towards the gamification moments. Data and text mining techniques allow researchers to analyse information and process non-structured text to find relevant knowledge decoded into actionable information (Fan et al., 2006; Zhang et al., 2009). The text mining techniques usually include distinct actions such as text clustering, topic extraction, text categorisation, among others (Li & Wu, 2010). For this study, we use the MeaningCloud software, a powerful tool to extract meaningful knowledge from all types of unstructured content, allowing researchers to perform text analysis, text classification or SA (MeaningCloud, 2019b). MeaningCloud SA API uses semantic approaches based on advanced natural language process (NLP) in all aspects of morphology, syntax, semantics and pragmatics (MeaningCloud, 2019b). Distinct research has opted for this software in different scientific domains (e.g., Bilro et al., 2019; Kaur & Chopra, 2016; Martínez et al., 2016; Segura-Bedmar et al., 2015). This tool can analyse a vast amount of data through NLP. NLP aid computers to comprehend the human’s natural language, allowing machines to interpret the relevant elements of the human language sentence and produces an interpretation of the text so it can be analysed (Godbole et al., 2010; Loureiro et al., 2020b; Mostafa, 2013). To assess the validity and reliability of the text mining outcomes, the authors lay the hands into existing literature measuring and comparing MeaningCloud tool results. Previous studies highlight that MeaningCloud presents high validity and reliability when compared with several other SA tools (van Aggelen, 2015). van Aggelen (2015) argues that the operation made by MeaningCloud of decomposing the text provides concurrently valid and consistent SA, as the outcomes agree with other concurring tools, with only minor deviations. Moreover, literature also put in evidence that when compared with other tools, the results obtained by MeaningCloud are very good, presenting a high percentage of correctly classified text, with minor detected errors (Gonzalez-Marron et al., 2017).

Results

The data analysis was made using distinct text-mining procedures. The first procedure is topic analysis, and topic extraction was made from the data on MeaningCloud. Table 1 shows the 10 most mentioned topics discussed by the students in their responses.

Table 1.

Ten Most Mentioned Topics

Topic	Frequency
Class	58
Student	35
Classroom	29
Match	21
Subject	10
Competition	10
Kahoot	10
People	10
Question	9
Interactive	9

Source: Authors’ own.

The main topics in students’ answers are connected to the classes themselves, such as class, student, classroom or subject. These are followed by topics connected to match, competition or interactive. These results are in line with the expected outcome. Therefore, students perceive gamification firstly as an educational tool, and only after that, they mention the facets more connected to the game or the competition itself. Based on these results, it is possible to argue that students understood the experience they have had as an element of education, and only after as an element of entertainment.

The second procedure is to analyse the most mentioned words. To analyse and group the set of words most used, we resort to a WordCloud, a visual illustration of text data, used to portray keyword metadata or visualise the free form of text. For this purpose, HTML5 WordCloud was used. The respondents’ most mentioned word is ‘class’, followed by ‘student’ (see Figure 1).

Figure 1.

Source: Authors’ own.

Other words, such as game, competition, or winner, are also highly ranked. However, as seen in Figure 1, respondents focus their opinion on words connected to education, such as class, students, or subjects. Again, respondents seem to understand that the purpose of this type of tool is related to education rather than entertainment.

Moving on in our text-mining analysis, we perform the third procedure through deep categorisation of the available text. Deep categorisation ‘assigns one or more categories of a predefined taxonomy to different snippets of a text. By applying a powerful semantic rule technology, it provides maximum accuracy in the classification while allowing the fastest and most efficient definition of models’ (MeaningCloud, 2019a). The deep categorisation reveals that the respondents’ main categories are education, educational assessment, technology and computing, video gaming and mobile games. Again, the text categories are in line with the previous analysis, with a clear focus on educational environments (Table 2).

Table 2.

Text Categories

Categories
Education
Business and Finance > Industries
Technology and Computing > Consumer Electronics
Science > Environment
Education > Educational Assessment
Business and Finance > Business
Video Gaming
Video Gaming > Mobile Games
Technology and Computing > Computing

Source: Authors’ own.

To complement the analysis, researchers use a fourth text-mining procedure as it became relevant to carry out a cluster analysis, to classify our object (in this case, the text corpus) into related groups that are similar to each other (i.e., clusters) (Li & Wu, 2010; Punj & Stewart, 1983). The software creates clusters and attributes a score to each cluster to identify the clusters with higher scores. Due to the large group of clusters that emerged, researchers decide to use only the clusters with higher scores, establishing a cut-off for clusters scoring 150 or higher (see Table 3).

Table 3.

Word Clusters

Cluster	Score
Good tool	602.60
Good way	602.55
Paying attention	531.20
Knowledge	370.08
Game	284.24
Fun way to learn	255.64
Stop paying attention	250.24
Outweigh the disadvantages	245.31
Wi-Fi	244.80
Feel	243.10
Motivation	231.84
Good idea	231.03
Competitive environment	198.76
Way of learning	188.90
Serious	174.45
Interactive way	168.70
Limited time	154.72
Way to teach	154.28

Source: Authors’ own.

The clusters with the highest scores are ‘Good tool’ (602.60), ‘Good way’ (602.55) and ‘Paying attention’ (531.20). We can argue that students understand this type of gamification tools as a good approach to adopt in educational environments, namely inside classrooms. Students also highlight that it is a fun way to learn (255.64) and outweighs the disadvantages (245.31). Some students also refer to a more practical aspect of using this type of technology in class: internet Wi-Fi should always be available to not detract from the overall experience (244.80).

Finally, to understand the sentiment level of each participant in the study it is performed a global SA, which intends to map the overall sentiments expressed in the text (i.e., participants’ answers) (Cambria et al., 2013; Liu, 2015). Through the MeaningCloud software, it is possible to analyse the sentiments expressed in each comment or answer and attribute a polarity scale for each one, from Positive + (P+) to Negative + (N+). As seen in Table 4, most answers express positive sentiments (72.60%), and only 8.22 % of the answers reveal negative sentiments.

Table 4.

Sentiment Analysis Results

	Sum	%	P–N %
P+	8	10.96	72.60
P	45	61.64
NEU\NONE	14	19.18	19.18
N	6	8.22	8.22
N+	0	0.00
Total	73	100.00	100.00

Source: Authors’ own.

Based on these results, most participants have a positive sentiment about this practice in higher educational settings. Moreover, less than 10% of respondents revealed negative sentiments towards the experience, and around 20% reveal neutral sentiments. Based on these results, we argue that this group of participants’ overall experience is positive or very positive and that participants feel engaged by using gamification tools in service settings such as educational environments.

Discussion

This study explores if gamification tools influence students’ engagement in services, specifically in Higher Education Settings. The literature already points out some behavioural changes observed in participants of gamified environments, namely online (e.g., Chaudhary, 2010; Koivisto & Hamari, 2014; Turan et al., 2016). With this research, this article contributes to the ongoing discussion by putting in evidence the potential of gamification outside online environments and in distinct areas of practical application, collecting evidence of its possible positive impact in service settings. Moreover, it is possible to realise that students perceive gamification as an educational tool, and only after that as a game or a competition itself. The findings show that students understand the experience as an educational element, and only after as entertainment. Based on the results, it is also possible to argue that students understand this type of gamification tools as a good approach to implement in educational environments, namely inside classrooms.

This study also shed light on the impact of gamification activities on student engagement, which can be seen as a behavioural and psychological involvement in learning activities (Appleton et al., 2008). Our findings show that participants’ overall experience is positive or very positive and feel engaged using gamification tools. Additionally, by resorting to SA, it is possible to find and measure the participants’ general sentiment about the gamification approach to support the learning process. SA is an exciting way to facilitate understanding of how participants feel about something, about their emotional states and their opinions, answers or comments expressed freely, without limitations or restrictions (Altrabsheh et al., 2014; Liu, 2015). Most of our participants’ sentiments expressed (72.60%) were positive, revealing that this type of gamification tool and its direct application can contribute to greater interest and motivation.

Conclusions and Implications

Theoretical Contributions

This article proposes to study how gamification’s technology-based approach could influence students’ engagement with the contents made available in services. Using a gamification tool, we have built and assess a services scenario to collect students’ motivations, interests and engagement in higher education settings. The findings may contribute to theory development and improvement of services and HEI.

First, this article underlines that gamification influences students’ engagement in services, such as higher education settings. When asked, students have reported high levels of engagement behaviours when taking part in the Kahoot! activities. However, the implementation of gamification-based activities within an educational environment requires special attention to ensure the best possible outcome from the whole experience for participants and providers. Following Huang and Soman (2013), caution should be made about the preparatory actions to be taken, such as the definition of objectives, the understanding of the target audience, or identifying the available resources that could support the application of gamification elements.

Second, this study uses a novel approach by applying the S-O-R framework to explain the role of gamification in student engagement in higher education settings, particularly the use of Kahoot! as a gamification tool. The student engagement, grounded in the S-O-R environmental psychology framework, posits how environmental stimuli are perceived and processed, ultimately impacting attitudinal and behavioural responses (Donovan & Rossiter, 1982; Mehrabian & Russell, 1974). Our study empirically extends the SOR framework to the study of students’ engagement. Research employing this framework suggests that technological experiences resulting from controlled stimuli may be critical to engagement (Mosteller & Mathwick, 2014). Psychological engagement, defined as a cognitive and affective commitment to a functional relationship (Mollen & Wilson, 2010), may manifest from the gamification process experienced.

Managerial Implications

From a managerial perspective, students’ engagement in gamification activities shows relevant managerial implications due to the interest and behavioural engagement students reveal in these activities. First, this study aims not to analyse the gamification tool but highlight the importance of this type of gamification-based tool to stimulate participants and increase their interest in the contents made available through an active and engaging offer. Kahoot! is a straightforward and interactive online tool, which can be considered an entertainment tool. Managers should bear in mind that it can be a powerful tool to stimulate participation and interest when used in a service setting. In our study, participants consider it a ‘good tool’ and a ‘good way to pay attention’ and, most of all, a ‘fun way to learn’ which is also an exciting condition to promote participant satisfaction and engagement (Borrás-Gené et al., 2019). Second, practitioners (either college administrations, lecturers or others) may understand and promote this type of tools as a leverage to create participation and engagement in their audiences. Gamification providers, such as firms, brands or HEI, need to master the best way to apply gamification elements in a service setting properly and, above all, to be aware of this technology-based approach advantages or disadvantages.

Limitations and Future Research

This article focuses on gamification tools’ effect on students’ engagement in a service setting such as HEI. Although this research is based on a rigorous method and data analysis, unveiling relevant findings that broaden literature knowledge, is not without its limitations. First, limitations arise from using a quasi-experimental design, which has recognised boundaries (Harris et al., 2006), such as the lack of randomisation or the presence of temporal confounders. Second, while data mining and text mining have many potential benefits and values, there are still some technical limitations to their capabilities. Data and text mining can analyse and assign a comment to one group of ideas (e.g., topic analysis, clusters) based on its content characteristics (word similarities defined in the NLP). However, that information can contain multiple cognitive aspects or contradictory categorisation cues that may result in missing data assigned to specific groups of ideas (Seifert, 2004). Third, using a single gamification-based tool (Kahoot!) can also be perceived as a limitation. Other gamification tools could be used, and/or a mix of tools and methods could contribute in an aggregated way to influence the participant’s engagement process. Since applying a single gamification-based tool has shown positive results in participants’ engagement, it can be relevant to test different approaches and scenarios, seeking to identify additional effective methods that may contribute to participants’ interest and engagement.

Consistent with our research, future researchers may consider building on the S-O-R framework applied to student engagement to investigate the structural paths linking environmental stimuli, psychological processes and engagement-related responses. Moreover, further research can also be performed by comparing distinct types of gamification tools with the same sample to understand who ensures the best learning outcomes. Finally, future research may also use different university courses in distinct cultural contexts to strengthen the findings and consider students’ emotional, cognitive and social dimensions and their role in these gamification educational approaches.

Code Availability

MeaningCloud 3.4.2.0.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

References

Altrabsheh

, Cocea

, & Fallahkhair

(2014). Sentiment analysis: Towards a tool for analysing real-time students feedback. Proceedings of the 26th International Conference on Tools with Artificial Intelligence (pp. 419–423). https://doi.org/10.1109/ICTAI.2014.70

Amegbe

, Owino

J. O.

, & Kerubo

O. L.

(2017). Behavioural responses to corporate image building through social media advertising: A study among Nairobi students. Journal of Creative Communications, 12(3), 223–238. https://doi.org/10.1177/0973258617725607

Appleton

J. J.

, Christenson

S. L.

, & Furlong

M. J.

(2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369–386. https://doi.org/10.1002/pits.20303

Barata

, Gama

, Jorge

, & Gonçalves

(2017). Studying student differentiation in gamified education: A long-term study. Computers in Human Behavior, 71, 550–585. https://doi.org/10.1016/j.chb.2016.08.049

Bilro

R. G.

, & Loureiro

S. M. C.

(2020). A consumer engagement systematic review: Synthesis and research agenda. Spanish Journal of Marketing-ESIC, 24(3), 283–307. https://doi.org/10.1108/SJME-01-2020-0021

Bilro

R. G.

, Loureiro

S. M. C.

, & Ali

(2018). The role of website stimuli of experience on engagement and brand advocacy. Journal of Hospitality and Tourism Technology, 9(2), 204–222. https://doi.org/10.1108/JHTT-12-2017-0136

Bilro

R. G.

, Loureiro

S. M. C.

, & Guerreiro

(2019). Exploring online customer engagement with hospitality products and its relationship with involvement, emotional states, experience and brand advocacy. Journal of Hospitality Marketing and Management, 28(2), 147–171. https://doi.org/10.1080/19368623.2018.1506375

Borrás-Gené

, Martínez-Núñez

, & Martín-Fernández

(2019). Enhancing fun through gamification to improve engagement in MOOC. Informatics, 6(28), 2–20.

Brodie

R. J.

, Hollebeek

L. D.

, Jurić

, & Ilić

(2011). Customer engagement: Conceptual domain, fundamental propositions, and implications for research. Journal of Service Research, 14(3), 252–271. https://doi.org/10.1177/1094670511411703

10.

Broer

(2014). Gamification and the trough of disillusionment. In Butz

, Koch

, & Schlichter

(Eds.), Mensch & Computer 2014: Workshopband (pp. 389–395). De Gruyter Oldenbourg.

11.

Burkey

(2019). Total recall: How cultural heritage communities use digital initiatives and platforms for collective remembering. Journal of Creative Communications, 14(3), 235–253. https://doi.org/10.1177/0973258619868045

12.

Cambria

, Schuller

, Xia

, & Havasi

(2013). New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 28(2), 15–21. https://doi.org/10.1109/MIS.2013.30

13.

Chaudhary

A. G.

(2010). Educational gaming: An effective tool for learning and social change in India. Journal of Creative Communications, 5(3), 135–152. https://doi.org/10.1177/0973258612471244

14.

Damnjanovic

, Proud

, & Milosavljevic

(2020). Mentoring development at student international business case competitions. EuroMed Journal of Business. https://doi.org/10.1108/EMJB-12-2018-0092

15.

Deterding

, Dixon

, Khaled

, & Nacke

(2011). From game design elements to gamefulness: Defining gamification. Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments (pp. 9–11). https://doi.org/10.1145/2181037.2181040

16.

Dichev

, & Dicheva

(2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14(9), 1–36. https://doi.org/10.1186/s41239-017-0042-5

17.

Dicheva

, Dichev

, Agre

, & Angelova

(2015). Gamification in education: A systematic mapping study. Educational Technology & Society, 18(3), 75–88. https://doi.org/10.1109/EDUCON.2014.6826129

18.

Donovan

R. J.

, & Rossiter

J. R.

(1982). Store atmosphere: An environmental psychology approach. Journal of Retailing, 58(1), 34–57.

19.

Eroglu

S. A.

, Machleit

K. A.

, & Davis

L. M.

(2003). Empirical testing of a model of online store atmospherics and shopper responses. Psychology and Marketing, 20(2), 139–150. https://doi.org/10.1002/mar.10064

20.

Faghihi

, Brautigam

, Jorgenson

, Martin

, Brown

, Measures

, & Maldonado-Bouchard

(2014). How gamification applies for educational purpose specially with college algebra. Procedia Computer Science, 41, 182–187. https://doi.org/10.1016/j.procs.2014.11.102

21.

Faiella

, & Ricciardi

(2015). Gamification and learning: A review of issues and research. Journal of E-Learning and Knowledge Society, 11(3), 13–21. https://doi.org/10.20368/JE-LKS.V11I3.1072

22.

Fan

, Wallace

, Rich

, & Zhang

(2006). Tapping the power of text mining. Communications of the ACM, 49(9), 76–82. https://doi.org/10.1145/1151030.1151032

23.

Finn

J. D.

, & Zimmer

K. S.

(2012). Student engagement: What is it? Why does it matter? In Christenson

S. L.

, Reschly

A. L.

, & Wylie

(Eds.), Handbook of research on student engagement (pp. 97–131). Springer.

24.

Fredricks

J. A.

, Blumenfeld

P. C.

, & Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

25.

Frost

R. D.

, Matta

, & MacIvor

(2015). Assessing the efficacy of incorporating game dynamics in a learning management system. Journal of Information Systems Education, 26(1), 59–70.

26.

Fuchs

(2018). Subversive gamification. In Cermak-Sassenrath

(Ed.), Playful disruption of digital media (pp. 181–191). Springer. https://doi.org/10.1007/978-981-10-1891-6_12

27.

Galbis-Córdova

, Martí-Parreño

, & Currás-Pérez

(2017). Higher education students’ attitude towards the use of gamification for competencies development. Journal of E-Learning and Knowledge Society, 13(1), 129–146.

28.

Garczynski

A. M.

, Waldrop

J. S.

, Rupprecht

E. A.

, & Grawitch

M. J.

(2013). Differentiation between work and nonwork self-aspects as a predictor of presenteeism and engagement: Cross-cultural differences. Journal of Occupational Health Psychology, 18(4), 417–429. https://doi.org/10.1037/a0033988

29.

Ghosh

(2019). Analysing the engagement and attitude of elderly towards digital platforms in India. Journal of Creative Communications, 14(3), 214–234. https://doi.org/10.1177/0973258619872085

30.

Godbole

, Bhattacharya

, Gupta

, & Verma

(2010). Building re-usable dictionary repositories for real-world text mining. Proceedings of the 19th ACM International Conference on Information and Knowledge Management (p. 1189). https://doi.org/10.1145/1871437.1871588

31.

Goggins

, & Xing

(2016). Building models explaining student participation behavior in asynchronous online discussion. Computers and Education, 94, 241–251. https://doi.org/10.1016/j.compedu.2015.11.002

32.

Gonzalez-Marron

, Mejia-Guzman

, & Enciso-Gonzalez

(2017). Exploiting data of the Twitter social network using sentiment analysis. In Sucar

, Mayora

, & Munoz de Cote

(Eds.), Applications for future Internet (pp. 35–38). Springer. https://doi.org/10.1007/978-3-319-49622-1

33.

Hamari

, Koivisto

, & Sarsa

(2014). Does gamification work? A literature review of empirical studies on gamification. Proceedings of the Annual Hawaii International Conference on System Sciences (pp. 3025–3034). https://doi.org/10.1109/HICSS.2014.377

34.

Harris

A. D.

, Mcgregor

, Perencevich

E. N.

, Furuno

J. P.

, Zhu

, Peterson

D. E.

, & Finkelstein

(2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal of American Medical Informatics Association, 13, 16–23. https://doi.org/10.1197/jamia.M1749

35.

Hollebeek

L. D.

, Clark

M. K.

, Andreassen

T. W.

, Sigurdsson

, & Smith

(2020). Virtual reality through the customer journey: Framework and propositions. Journal of Retailing and Consumer Services, 55. https://doi.org/10.1016/j.jretconser.2020.102056

36.

Hollebeek

L. D.

, Glynn

M. S.

, & Brodie

R. J.

(2014). Consumer brand engagement in social media: Conceptualization, scale development and validation. Journal of Interactive Marketing, 28(2), 149–165. https://doi.org/10.1016/j.intmar.2013.12.002

37.

Huang

W. H.-Y.

, & Soman

(2013). A practitioner’s guide to gamification of education. Rotman School of Management.

38.

Iosup

, & Epema

(2014). An experience report on using gamification in technical higher education. Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE’14; pp. 27–32). https://doi.org/10.1145/2538862.2538899

39.

, Law

, & Lee

H. A.

(2011). A review of website evaluation studies in the tourism and hospitality fields from 1996 to 2009. International Journal of Tourism Research, 13(3), 234–265. https://doi.org/10.1002/jtr.815

40.

Javornik

(2016). ‘It’s an illusion, but it looks real!’ Consumer affective, cognitive and behavioural responses to augmented reality applications. Journal of Marketing Management, 32(9–10), 987–1011. https://doi.org/10.1080/0267257X.2016.1174726

41.

Jimerson

S. R.

, Campos

, & Greif

J. L.

(2003). Toward an understanding of definitions and measures of school engagement and related terms. The California School Psychologist, 8(1), 7–27. https://doi.org/10.1007/bf03340893

42.

Kahoot! (2019). Kahoot! | Learning games | Make learning awesome! https://kahoot.com/

43.

Kapp

K. M.

(2012). The gamification of learning and instruction. John Wiley & Sons.

44.

Kapp

K. M.

, & Coné

(2012). What every chief learning officer needs to know about games and gamification for learning. White Paper. Institute for Interactive Technologies.

45.

Kaur

, & Chopra

(2016). Comparison of text mining tools. 2016 5th International Conference on Reliability, Infocom Technologies and Optimization, ICRITO 2016: Trends and Future Directions (pp. 186–192). https://doi.org/10.1109/ICRITO.2016.7784950

46.

Kayimbaşioǧlu

, Oktekin

, & Haci

(2016). Integration of gamification technology in education. Procedia Computer Science, 102. https://doi.org/10.1016/j.procs.2016.09.460

47.

Koivisto

, & Hamari

(2014). Demographic differences in perceived benefits from gamification. Computers in Human Behavior, 35, 179–188. https://doi.org/10.1016/j.chb.2014.03.007

48.

Költringer

, & Dickinger

(2015). Analysing destination branding and image from online sources: A web content mining approach. Journal of Business Research, 68(9), 1836–1843. https://doi.org/10.1016/j.jbusres.2015.01.011

49.

Leclercq

, Hammedi

, & Poncin

(2018). The boundaries of gamification for engaging customers: Effects of losing a contest in online co-creation communities. Journal of Interactive Marketing, 44, 82–101. https://doi.org/10.1016/j.intmar.2018.04.004

50.

Leonidou

, Christofi

, Vrontis

, & Thrassou

(2018). An integrative framework of stakeholder engagement for innovation management and entrepreneurship development. Journal of Business Research, 1–14. https://doi.org/10.1016/j.jbusres.2018.11.054

51.

, & Wu

D. D.

(2010). Using text mining and sentiment analysis for online forums hotspot detection and forecast. Decision Support Systems, 48(2), 354–368. https://doi.org/10.1016/j.dss.2009.09.003

52.

Lim

, & Puspita

(2020). Apple’s tale: The engagement and sense-making of Apple’s story-based advertisements. Journal of Creative Communications, 15(3), 255–268. https://doi.org/10.1177/0973258620924948

53.

Liu

(2015). Sentiment analysis: Mining opinions, sentiments, and emotions. Cambridge University Press.

54.

Looyestyn

, Kernot

, Boshoff

, Ryan

, Edney

, & Maher

(2017). Does gamification increase engagement with online programs? A systematic review. PLOS One, 12(3), e0173403. https://doi.org/10.1371/journal.pone.0173403

55.

Loureiro

S. M. C.

, Bilro

R. G.

, & Angelino

F. J. A.

(2020a). Virtual reality and gamification in marketing higher education: A review and research agenda. Spanish Journal of Marketing—ESIC. https://doi.org/10.1108/SJME-01-2020-0013

56.

Loureiro

S. M. C.

, Bilro

R. G.

, & Japutra

(2020b). The effect of consumer-generated media stimuli on emotions and consumer brand engagement. Journal of Product & Brand Management, 29(3), 387–408. https://doi.org/10.1108/JPBM-11-2018-2120

57.

Loureiro

S. M. C.

, Romero

, & Bilro

R. G.

(2020c). Stakeholder engagement in co-creation processes for innovation: A systematic literature review and case study. Journal of Business Research, 119, 388–409. https://doi.org/10.1016/j.jbusres.2019.09.038

58.

Mahnič

(2014). Gamification of politics: Start a new game. Teorija in Praksa, 51(1), 143–161.

59.

Martín-Gutiérrez

, Mora

C. E.

, Añorbe-Díaz

, & González-Marrero

(2017). Virtual technologies trends in education. Eurasia Journal of Mathematics, Science and Technology Education, 13(2), 469–486. https://doi.org/10.12973/eurasia.2017.00626a

60.

Martínez

, Martínez

J. L.

, Segura-Bedmar

, Moreno-Schneider

, Luna

, & Revert

(2016). Turning user generated health-related content into actionable knowledge through text analytics services. Computers in Industry, 78, 43–56. https://doi.org/10.1016/j.compind.2015.10.006

61.

MeaningCloud (2019a). Deep categorization | MeaningCloud. https://www.meaningcloud.com/products/deep-categorization

62.

MeaningCloud (2019b). Our company | MeaningCloud. https://www.meaningcloud.com/our-company

63.

Mehrabian

, & Russell

J. A.

(1974). An approach to environmental psychology. MIT Press.

64.

Mollen

, & Wilson

(2010). Engagement, telepresence and interactivity in online consumer experience: Reconciling scholastic and managerial perspectives. Journal of Business Research, 63(9–10), 919–925. https://doi.org/10.1016/j.jbusres.2009.05.014

65.

Morimoto

S. A.

, & Friedland

L. A.

(2013). Cultivating success: Youth achievement, capital and civic engagement in the contemporary United States. Sociological Perspectives, 56(4), 523–546. https://doi.org/10.1525/sop.2013.56.4.523

66.

Mostafa

M. M.

(2013). More than words: Social networks’ text mining for consumer brand sentiments. Expert Systems with Applications, 40(10), 4241–4251. https://doi.org/10.1016/j.eswa.2013.01.019

67.

Mosteller

, & Mathwick

(2014). Reviewer online engagement: The role of rank, well-being, and market helping behavior. Journal of Consumer Marketing, 31(6/7), 464–474. https://doi.org/10.1108/JCM-05-2014-0974

68.

Nobre

, & Ferreira

(2017). Gamification as a platform for brand co-creation experiences. Journal of Brand Management, 24(4), 349–361. https://doi.org/10.1057/s41262-017-0055-3

69.

Oleksy

, & Wnuk

(2017). Catch them all and increase your place attachment! The role of location-based augmented reality games in changing people–place relations. Computers in Human Behavior, 76, 3–8. https://doi.org/10.1016/j.chb.2017.06.008

70.

Oxford (2019). The use of elements of game-playing in another activity, usually in order to make that activity more interesting. Gamification at Oxford Dictionary.

71.

Pagowsky

(2012). Etcv 524 Literature review: Motivation in gamified learning scenarios. University of Arizona.

72.

Piñeiro-Otero

, & Costa-Sánchez

(2015). ARG (Alternate reality games). Contributions, limitations, and potentialities to the service of the teaching at the university level. Comunicar, 22(44), 141–148. https://doi.org/10.3916/C44-2015-15

73.

Priya

, & Kalpana (2014). Gamification: The best approach for student engagement. International Journal of Management, 5(2), 976–6502.

74.

Punj

, & Stewart

D. W.

(1983). Cluster analysis in marketing research: Review and suggestions for application. Journal of Marketing Research, 20(2), 134–148. https://doi.org/10.2307/3151680

75.

Putman

S. M.

, Ford

, & Tancock

(2012). Redefining online discussions: Using participant stances to promote collaboration and cognitive engagement. International Journal of Teaching and Learning in Higher Education, 24(2), 151–167.

76.

Ramos

, & Yudko

(2008). ‘Hits’ (not ‘discussion posts’) predict student success in online courses: A double cross-validation study. Computers and Education, 50(4), 1174–1182. https://doi.org/10.1016/j.compedu.2006.11.003

77.

Robson

, Plangger

, Kietzmann

J. H. J.

, McCarthy

, & Pitt

(2015). Is it all a game? Understanding the principles of gamification. Business Horizons, 58(4), 411–420. https://doi.org/10.1016/j.bushor.2015.03.006

78.

Rosado-Pinto

, & Loureiro

S. M. C.

(2020). The growing complexity of customer engagement: A systematic review. EuroMed Journal of Business, 15(2), 167–203. https://doi.org/10.1108/EMJB-10-2019-0126

79.

Roschk

, Loureiro

S. M. C.

, & Breitsohl

(2017). Calibrating 30 years of experimental research: A meta-analysis of the atmospheric effects of music, scent, and color. Journal of Retailing, 93(2), 228–240. https://doi.org/10.1016/j.jretai.2016.10.001

80.

Segura-Bedmar

, Martínez

, Revert

, & Moreno-Schneider

(2015). Exploring Spanish health social media for detecting drug effects. BMC Medical Informatics and Decision Making, 15(2), S6. https://doi.org/10.1186/1472-6947-15-S2-S6

81.

Seifert

J. W.

(2004). Data mining an overview. Analyst in information science and technology policy resources, science, and industry division. The Library of Congress.

82.

Shadish

W. R.

, Cook

T. D.

, & Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

83.

Sheldon

(2012). The multiplayer classroom: Designing coursework as a game. In Course Technology. Cengage Learning. https://doi.org/10.1111/teth.12095

84.

Shukla

, & Sharma

(2018). Emotions and media multitasking behaviour among Indian college students. Journal of Creative Communications, 13(3), 197–211. https://doi.org/10.1177/0973258618790794

85.

Simmons

J. P.

, Nelson

L. D.

, & Simonsohn

(2013). Life after p-hacking. Meeting of the Society for Personality and Social Psychology (pp. 17–19). https://doi.org/10.2139/ssrn.2205186

86.

Sprott

, Czellar

, & Spangenberg

(2009). The importance of a general measure of brand engagement on market behavior: Development and validation of a scale. Journal of Marketing Research, 46(1), 92–104. https://doi.org/10.1509/jmkr.46.1.92

87.

Srivastava

A. N.

, & Sahami

(2009). Text mining: Classification, clustering, and applications. Chapman & Hall/CRC.

88.

Stewart

L. P.

(1992). Ethical issues in postexperimental and postexperiential debriefing. Simulation & Gaming, 23(2), 196–211. https://doi.org/10.1177/1046878192232007

89.

Stott

, & Neustaedter

(2013). Analysis of gamification in education. Carmster.Com (pp. 1–8).

90.

Subhash

, & Cudney

E. A.

(2018). Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87(May), 192–206. https://doi.org/10.1016/j.chb.2018.05.028

91.

Turan

, Avinc

, Kara

, & Goktas

(2016). Gamification and education: Achievements, cognitive loads, and views of students. International Journal of Emerging Technologies in Learning, 11(7), 64–69. https://doi.org/10.3991/ijet.v11i07.5455

92.

Van Aggelen

J. M.

(2015). Concurrent validity and consistency of social media sentiment analysis tools.

93.

Watson

, Wilson

H. N.

, Smart

, & Macdonald

E. K.

(2018). Harnessing difference: A capability-based framework for stakeholder engagement in environmental innovation. Journal of Product Innovation Management, 35(2), 254–279. https://doi.org/10.1111/jpim.12394

94.

Woodcock

, & Johnson

M. R.

(2017). Gamification: What it is, and how to fight it. The Sociological Review, 66(3), 542–558. https://doi.org/10.1177/0038026117728620

95.

Yang

, Asaad

, & Dwivedi

(2017). Examining the impact of gamification on intention of engagement and brand attitude in the marketing context. Computers in Human Behavior, 73, 459–469. https://doi.org/10.1016/j.chb.2017.03.066

96.

Yildirim

(2017). The effects of gamification-based teaching practices on student achievement and students’ attitudes toward lessons. Internet and Higher Education, 33, 86–92. https://doi.org/10.1016/j.iheduc.2017.02.002

97.

Zhang

, Zeng

, Li

, Wang

F. Y.

, & Zuo

(2009). Sentiment analysis of chinese documents: From sentence to document level. Journal of the American Society for Information Science and Technology, 60(12), 2474–2487. https://doi.org/10.1002/asi.21206

98.

Zhao

, & Patrick Rau

P. L.

(2020). Merging and synchronizing corporate and personal voice agents: Comparison of voice agents acting as a secretary and a housekeeper. Computers in Human Behavior, 108(March), 106334. https://doi.org/10.1016/j.chb.2020.106334

99.

Zhu

(2006). Interaction and cognitive engagement: An analysis of four asynchronous online discussions. Instructional Science, 34(6), 451–480. https://doi.org/10.1007/s11251-006-0004-0