From Gaming to Computational Thinking: An Adaptive Educational Computer Game-Based Learning Approach

Abstract

Educational games have been increasingly used to improve students’ computational thinking. However, most existing games have focused on the theoretical knowledge of computational thinking, ignoring the development of computational thinking skills. Moreover, there is a lack of integration of adaptivity into educational computer games for computational thinking, which is crucial to addressing individual needs in developing computational thinking skills. In this study, we present an adaptive educational computer game, called AutoThinking, for developing students’ computational thinking skills in addition to their conceptual knowledge. To evaluate the effects of the game, we conducted an experimental study with 79 elementary school students in Estonia, where the experimental group learned with AutoThinking, while the control group used a traditional technology-enhanced learning approach. Our findings show that learning with the adaptive educational computer game significantly improved students’ computational thinking related to both conceptual knowledge and skills. Moreover, students using the adaptive educational computer game showed a significantly higher level of interest, satisfaction, flow state, and technology acceptance in learning computational thinking. Implications of the findings are also discussed.

Keywords

adaptive educational computer game computational thinking adaptive learning adaptive gameplay experimental research

Computational Thinking (CT) is characterized as a cognitive ability enabling people to develop computational solutions for a current problem (Wing, 2006). It is also seen as a mental ability of applying computer science’s reasoning processes to STEM (science, technology, engineering, and mathematic) domains, and further applying it to different problems and activities in everyday life (Wang, 2016). Moreover, the influence of CT has gone even beyond STEM, for instance in medicine, digital humanities, computational finance, archaeology, economics, etc. (Wing, 2011). For these reasons—the fact that CT denotes a general and applicable problem-solving strategy for a wide range of domains—Wing (2008) refers to it as one of the main and fundamental 21st century skills. Research has shown that teaching CT can improve students’ analytical skills, and acquiring CT abilities could be seen as a factor of their academic success (e.g., Haddad & Kalaani, 2015). Therefore, similar to numeracy and literacy, CT is recognized not only as an important competence for computer scientists, but for every person, which is why it should be taught and acquired in early education. To this end, it has frequently been reported by several studies that the integration of CT into curricula benefits both the cognitive and non-cognitive aspects of learning (e.g., Boucinha et al., 2019; Brown et al., 2014; Malva et al., 2020; Repenning et al., 2015). Consequently, several reformations of educational programs have taken place on different educational levels all over the world in order to integrate CT into official curricula (e.g., Brown et al., 2014; Perković et al., 2010; Waterman et al., 2019).

However, there exists some challenge in promoting computational thinking in educational practice. Research has shown pupils’ negative attitudes toward learning CT, which in turn result in impeding its proper development (e.g., Yardi & Bruckman, 2007). To overcome such challenges, different approaches have been applied in order to make CT more available and also engaging to learners. Educational computer games (henceforth referred to as educational games) can be one prospective solution. Educational games have gained researchers’ attention as they have been shown to be effective learning tools that both engage and motivate students (e.g., Acquah & Katz, 2020). In addition to that, findings from research suggest that educational games can also improve students’ learning achievements (e.g., Hwang et al., 2015; Partovi & Razavi, 2019). For example, Partovi and Razavi (2019) studied the effectiveness of game-based learning on elementary students’ academic achievement and motivation to learn science. Their findings indicated that students who learned through gameplay had significantly better academic achievement and motivation to learn science than those who learned with traditional approaches.

There are several educational games that aim to foster students’ CT, such as CodeSpell (Esper et al., 2014) and MiniColon (Ayman et al., 2018); however, they mostly focus on helping students gain theoretical concepts, such as sequence and conditional logic, as well as enhancing their learning motivation, while the provision of prompts to enhance students’ CT skills (e.g., pattern generalization and debugging) are generally ignored (Hooshyar et al., 2019a; Kazimoglu et al., 2011, 2012; Zhang & Nouri, 2019; Zhao & Shute, 2019). One possible reason is that it is usually easier to promote and directly assess CT concepts than CT skills (Lye & Koh, 2014). This can result in unequal development of different aspects of CT (less development of CT skills), situating CT development in a dilemma. Games that aim to teach CT skills, on the other hand, should offer opportunities to practice the theoretical knowledge through gameplay. Hence, we must differentiate between games that aim to teach applied knowledge and skills, and those that merely support reinforcing theoretical knowledge.

In addition, the existing games mainly follow predefined and rigid computer-assisted instruction concepts, without incorporating personalization and adaptation in the games that would suit the individual needs of the player. Ignoring adaptivity, however, results in impeding the full educational potential of computer games (e.g., Hooshyar et al., 2019b; Kickmeier-Rust et al., 2011). Taking into account the importance and relevance of CT in society, as well as the existing gaps in CT game research, in this study, we propose a new approach, CT-oriented adaptive educational computer game-based learning—to promote learners’ CT knowledge and skills. To evaluate the performances of the proposed approach, an adaptive educational game, called AutoThinking, was developed; moreover, an experiment was conducted to answer the following research questions:

Do the students learning with the proposed approach have better CT knowledge (including conceptual knowledge and skills) than those learning with the traditional technology-enhanced learning approach?

Do the students learning with the proposed approach have higher interest in learning CT than those learning with the traditional technology-enhanced learning approach?

Do the students learning with the proposed approach have higher satisfaction in learning CT than those learning with the traditional technology-enhanced learning approach?

Do the students learning with the proposed approach have higher technology acceptance than those learning with the traditional technology-enhanced learning approach?

Do the students learning with the proposed approach have higher flow states than those learning with the traditional technology-enhanced learning approach?

Related Research

Computational Thinking

During the past half century, several researchers have proposed and highlighted ideas revolving around the notion of applying knowledge and skills used in computer science to other fields and its possible potential for all learners (e.g., Papert, 1990; Wilensky, 2001). Even though this could be considered as the motive behind the movement of computational thinking (CT), for the first time, Wing (2006) employed the term “computational thinking” to characterize the thinking process of computer scientists while trying to solve a problem. She argued that CT consists of a set of competencies that, besides other essential competencies such as numeracy and literacy, should be acquired and learned by everyone in early education. According to Cuny et al. (2010, p. 1), CT is “the thought processes involved in formulating problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent.”

In spite of the long history of research in the area of CT, there is still no solid consensus among researchers on the definition of CT and what processes or competencies it is comprised of (e.g., Guzdial, 2008; Lee et al., 2011; Shute et al., 2017). An overview of the emergence of CT definitions and skills has been given by Palts and Pedaste (2020b). For instance, Wing (2006) defined six main dimensions of CT: problem formulation, abstraction, problem reformulation, problem decomposition, automation, and systematic testing. Moreover, according to her argument, all core skills involving problem solving with systematic and logical thinking, as well as engineering and mathematical thinking are to be incorporated into CT. Furthermore, Denning (2009) argues that CT is not all about computer science or ways to reduce poor retention or the high dropout rates of students in computer science. He believes CT is about seven different views or categories, namely coordination, communication, computing, recollection, design and assessment, and automation. Ater-Kranov et al. (2010), by evaluating prospective students and academics, studied the importance of various computation thinking skills and concluded that the application of abstractions to solve a problem, along with algorithmic thinking are the key competencies of CT. Unlike Wing’s (2006) argument, Ater-Kranov et al. (2010) found that since complex CT can also occur spontaneously, engineering and mathematical thinking are not essential parts of CT. In another study, Brennan and Resnick (2012) state that CT includes several concepts and skills the practice of which results in the development of CT knowledge. According to their study, CT concepts include sequences, loops, parallelism, events, conditionals, operators, and data. In addition, they named several practices or skills of CT, among them abstracting and modularizing, pattern recognition, debugging, and simulation. Shute et al. (2017) state that CT is comprised of several main facets, namely decomposition, abstraction, algorithm design, debugging, iteration, and generalization. Even though there is no solid consensus on the different components of CT, there are four different computational practices (or skills) and three concepts— constituting overall knowledge of CT—that several studies agree on. The skills consist of problem solving or algorithmic thinking, building algorithms, debugging, and simulation, while the concepts are sequence, conditional logic, and loop logic. In brief, algorithmic thinking includes identifying and decomposing the problem; building algorithms involves the development of procedures one step at a time to solve the problem in hand, including creating efficient and repeatable patterns; debugging entails investigating the problems and errors that might have occurred in the algorithm; simulation deals with demonstration of the developed algorithm, or testing the solution before actually applying it; sequence is about the logical steps that are organized in a correct order to successfully complete a task; conditional logic includes decision-making based on pre-set conditions or criteria that support the result of multiple outcomes; and finally loop logic is about repeating the same sequence of actions multiple times. These skill sets and concepts might have been termed differently by different researchers, but their importance as core elements of CT is becoming ubiquitously accepted one way or another (Brennan & Resnick, 2012; Hooshyar et al., 2019a; Kazimoglu et al., 2012; Lee et al., 2011; Tsarava et al., 2017; Wing, 2006; Zhang & Nouri, 2019; Zhao & Shute, 2019).

Educational Games Targeting Computational Thinking

Several learning environments use coding in order to teach CT to students, as computer programming and CT share overlapping aspects in terms of cognitive skills (Grover & Pea, 2013). These environments mostly use block-based and visual programming, or they adapt game design principles in order to reduce the complexity associated with programming language syntax. This is usually done by simplifying it down to drag-and-drop interactions in order to offer simpler solutions than writing syntax. Some examples of these environments are Scratch (Resnick et al., 2009; Zhang & Nouri, 2019), Snap! (Harvey & Mönig, 2010), and Blockly (Fraser, 2013; Luo et al., 2020). Even though the research has shown some success in improving learners’ motivation in programming and CT with these environments, they fall short when it comes to promoting deeper learning (e.g., Brennan & Resnick, 2012; Meerbaum-Salant et al., 2011). One reason is that players still get distracted and overwhelmed by syntax that is given in different forms (e.g., blocks), even though the main focus of CT should be conceptualization and the basic thought processes of solving problems, not coding. Therefore, the alignment of these environments with CT skills is partial and incomplete. Moreover, these environments are often called games for fostering CT, yet they cannot be reckoned as educational games as they lack several relevant elements of educational games such as timely feedback, supporting engagement, enhancing retention, and incentives (e.g., Kazimoglu et al., 2012; Kickmeier-Rust et al., 2011).

Other educational games that develop different skills and are proven to be effective tools for supporting learning have also been used for improving learners’ CT (e.g., Weintrop & Wilensky, 2012). Usually, educational games that aim to foster CT use motivating contexts to engage players in the process of developing solutions to a problem. In contrast to block-based and visual programming environments (or design-based learning environments), these educational games have the capacity to support more purposeful learning due to different game elements (e.g., Land, 2000). For instance, Eagle and Barnes (2009), Esper et al. (2014), and Ayman et al. (2018) developed Wu's Castle, CodeSpell, and MiniColon for teaching early programming and supporting knowledge of CT, respectively. While a number of studies have reported these games as having a positive impact on learners’ programming skills and CT, they are not fully aligned with CT. The reason for this is that these games are using a text-based programming language, which requires learners’ focused attention on syntax details (Zhao & Shute, 2019). More importantly, some of these games put more emphasis on promoting abstract and conceptual knowledge of CT, providing fewer opportunities to develop CT skills (Hooshyar et al., 2019a; Kazimoglu et al., 2011, 2012; Zhang & Nouri, 2019). To promote CT skills, Zhao and Shute (2019) developed a video game for fostering students’ CT, particularly their CT skills. Evaluation of their game revealed that playing the game, called Penguin Go, for even less than 2 hours significantly improved the students’ CT skills, whereas no influence was found on their attitudes toward CT. Besides, there are some other educational games, based on block-based programming languages, that implicitly target promoting CT, for instance, RoboBuilder (Weintrop & Wilensky, 2012), LightBot (Gouws et al., 2013), and the games at Code.org. Studies evaluating the influence of these games on improving students’ CT are limited. In other words, studies reporting the effects of these games on learners are preliminary or qualitative with small sample sizes (Giannakoulas & Xinogalos, 2018; Kazimoglu et al., 2012; Weintrop & Wilensky, 2012).

On the other hand, to a large extent, the educational games aimed at fostering CT tend to neglect adaptation and personalization to individual needs. In other words, such games follow unadaptable and rigid computer-assisted instruction concepts, resulting in limiting the full educational potential of computer games (Hooshyar et al., 2019a; Kickmeier-Rust et al., 2011). In brief, research has shown promising results concerning the application of educational games to CT; however, there is still some room for improvement in such games. In order to improve on the existing games, we developed an adaptive CT game, called AutoThinking, which engages users with individually tailored gameplay and a learning process that helps to foster learners’ CT concepts and their skills.

AutoThinking

Overview of the Game

AutoThinking¹ is an adaptive educational game developed for promoting students’ CT (Hooshyar et al., 2019a). The game uses icons rather than syntax of computer programming languages with the purpose of excluding syntactical errors. AutoThinking, to the best of our knowledge, is the first adaptive educational game developed for promoting CT that includes adaptivity in both the gameplay and the learning process. In a novel way, it promotes four CT skills: 1) problem identification and decomposition (algorithmic thinking); 2) algorithm building (pattern recognition and generalization); 3) debugging; and 4) simulation. In addition to that, it fosters three CT concepts: 1) sequence; 2) conditional; and 3) loop (for more details, see Hooshyar et al., 2019a).

AutoThinking currently consists of three levels in which a player should, in the role of a mouse, develop different types of strategies and solutions to complete the levels, while collecting as many cheese pieces and scoring as much as possible, and at the same time escaping from two cats in the maze. Players can develop up to 20 solutions for clearing all 76 cheese pieces in the maze. During the gameplay, the player receives more points for solutions that involve various CT concepts or skills, and for traversing non-empty tiles. Note that the player is provided with various options to develop different types of solutions; for example, it is possible to use “function” to save various patterns, and if necessary apply or generalize them in different situations of the game. According to the suitability of the solution for the current state of the maze, if necessary, the player is adaptively given various types of feedback (textual, graphical, or video) and hints.

Several activities and features in the AutoThinking game are designed and embedded to target and promote different CT skills and concepts. A “solution bar” is created in order to help the player develop different solutions for different situations of the maze using sequence of proper actions (targeting both problem-solving and sequence). “Function” is designed to encourage the player to construct generalizable patterns where they can be used in different situations of the game (targeting algorithmic thinking and pattern recognition skill). The vertical bar with 10 squares in Figure 2 shows the “solution bar,” while the horizontal bars with 10 squares indicate “function.” The “loop” button helps run the same sequence of actions multiple times (practicing the loop concept). The “conditional” button enables the player to decide based on certain conditions that support the result of multiple outcomes (practicing conditional concept). The “debug” button enables the player to monitor the solution algorithm and possibly detect and solve any potential errors in its logic (practicing debugging skill). And finally, the “simulation” button allows the players to simulate their solution before actually executing it in order to observe the outcome of their solution regardless of intervention of other variables in the game, such as the cats’ movements and cheese pieces (practicing run time mode or simulation skill). The two rows of buttons at the bottom right in Figure 2 show “loop” (indicated by ‘q’), “conditional” (indicated by ‘w’), “debug” (indicated by ‘e’), and “simulation” (indicated by ‘s’).

Bayesian Network in AutoThinking

The framework of AutoThinking is illustrated in Figure 1, showing how adaptivity takes place both in gameplay and in the learning process. As shown in Figure 1, after the player executes a solution, the game data are inputted to the Bayesian Network (BN) algorithm which has been developed by experts in the field (for more details, see Hooshyar et al., 2019a). Accordingly, the BN decides which algorithm the cat should follow for the current solution of the player and, if necessary, what kind of feedback or hint should be provided to the player (see the following sections).

Figure 1.

Framework of AutoThinking.

Adaptivity in Gameplay

During gameplay, one of the cats moves intelligently according to the quality of the developed solution by the player. To do so, it considers whether the solution has the potential to gain a high enough score, whether it is risky in terms of the mouse getting caught by cats, and if players used proper CT skills or concepts in their developed solution according to the current state of the maze. Accordingly, a decision-making technique used in the game—provided by a probabilistic model, Bayesian Network, that automatically assesses players’ skills—regulates the movement of the cat by switching between the following algorithms:

Random: the cat decides to move randomly without iteration through the maze

Provocative: the cat decides to move provocatively by going close to the mouse (up to one tile away), not to catch it, and come back

Aggressive: the cat decides to move aggressively with the aim of catching the mouse (by finding the shortest distance from the mouse)

Lenient: the cat decides not to get closer than six tiles away from the mouse

Note that the cat decides to choose a more appropriate algorithm to use for its movements according to both the short-term and long-term solutions of the player. In other words, it considers both the current developed solution as well as previous solutions developed by the player. More explicitly, if a player underperforms in loop logic in several previous episodes of the game and in the current episode performs highly, the decision-making algorithm would probably calculate a lower probability for his/her loop logic competency (and accordingly for other competencies) than those players that performed highly both in the previous and current episodes of the game. However, another cat still moves at random with repetition based on the number of commands placed in the solution bar, making AutoThinking an unpredictable game that always provides the player with a new situation that might not have been faced by previous players.

Adaptivity in Learning

While playing the game, the automatic short- and long-term assessment of the players enables the game to provide them with timely feedback and hints. Observe that the short-term assessment is based on the current solution developed by a player, whereas the long-term assessment considers all previous solutions developed by the player. According to the current state of the maze and the player’s skill level (long- and short-term), the game offers textual, graphical, or video feedback about CT concepts and skills that are embedded in the gameplay. It also highlights some of the game features or buttons as hints, enabling players to improve their solutions according to both the hints and the feedback. For instance, Figure 2A illustrates a solution developed by a player, and Figure 2B shows feedback and hints generated after clicking on the “debug” button for that specific situation of the game. Apart from the feedback and hints, whether the player opts to run the solution or to revise, the cat movement will be regulated by switching between the four algorithms explained previously (i.e., random, provocative, aggressive, and lenient).

Figure 2.

(A) a solution developed by a player, and (B) textual feedback on the right side and graphical feedback on the left side, along with hints generated for the solution.

This phase of adaptivity takes place at two different times, before or after running the solution. Regarding the former, once players have developed their solution they can use the “debug” button—which activates the probabilistic model used for decision-making—to see the estimation of the suitability of their solution in the form of timely adaptive feedback or hints. More explicitly, if the developed solution is estimated as satisfactory, players receive feedback indicating that their solution is a successful one and they can continue; otherwise they may receive another type of feedback guiding them to think about revising their solution before running it. Doing so provides the player with a chance to, if necessary, change and improve their solution so as to have a more optimum solution. Alternatively, concerning the latter situation, the player can skip using the “debug” option and directly “run” the game (after developing the solution). This results in timely adaptive feedback or hints, after running the game, which would help the player to know about the shortcomings and mistakes in previous solutions and possible ways to overcome them. Such adaptivity—which aims to foster both learners’ problem-solving (algorithmic thinking) and pattern recognition skills—individually supports learners in developing the most optimum solution for the problem in hand.

Method

Participants

A total of 79 fifth graders aged 11 and 12 from four classes of an elementary school in Estonia took part in this experiment. A typical class size of an elementary school in Estonia usually varies from 15 to 25 students, which was also the case of this study. We randomly assigned two classes of students as the control group (20 girls and 23 boys) and the other two as the experimental group (16 girls and 20 boys). Participants had no prior programming experience as the informatics curriculum only includes basic computer skills (e.g., editing text, finding information, internet security, etc.). The students in fifth grade have an informatics lesson once a week. Additionally, all lessons were carried out by the same teacher so that we could avoid the possible effect of different teachers on the results. Observe that we informed students that it was not compulsory for them to participate in our study and their participation would have no effect on their final grades. To conduct this study, we obtained ethical approval from the ethical committee of (removed for blind review) (issue date: November 18, 2019; Application Registration Number: 298/T-7). Before starting the experiment, all students and their parents signed consent forms to participate in our study.

Procedure

The experimental process followed in this study is illustrated in Figure 3. Firstly, students in both groups spent about 30 minutes answering the pre-test for measuring computation thinking knowledge (conceptual knowledge and skills). Secondly, the control group took part in a 60-75-minute session on the basic and essential CT content (including both concepts and skills) from a teacher using a PowerPoint presentation with Multimedia (traditional technology-enhanced learning). All topics taught by the teacher in the control group were in line with those covered by the AutoThinking game (e.g., sequence, conditional and loop logic, problem decomposition, etc.). In addition to the teacher’s lecture, the activities of the control group session included watching educational videos, discussion, and performing tasks. Students in the experimental group spent 60–75 minutes using the adaptive educational game (AutoThinking) for learning basic CT subjects. The CT skills and concepts used for teaching students in both groups were identical, including algorithmic thinking, pattern recognition, debugging, and simulation for skills, and sequence, loop and conditional for concepts. Finally, when the learning activity was finished, the students answered the post-test, motivation and technology acceptance questionnaires (30–45 min).

Figure 3.

Experimental Procedure.

Instruments

Instruments of measurement employed in this experiment include a pre-test and post-test, post questionnaire of interest in learning, learning satisfaction, technology acceptance, and flow states. The pre- and post-test aim to assess participants’ CT knowledge (including conceptual knowledge and skills) before and after the experiment. The tests include nine multiple-choice questions and one fill-in-the-blank question. Both the pre- and post-test were adopted from a validated instrument for assessing CT developed by González (2015) and were augmented and examined by two experienced teachers and researchers with over 5 years’ experience of teaching and researching CT and computer science courses. An example of a question used in the pre- and post-test is presented in Figure 4. This sample question shows how students’ different CT concepts and skills are simultaneously targeted. More specifically, to correctly answer this question, problem solving or algorithmic thinking, building algorithms (including creating efficient and repeatable patterns), and simulation skills are required concurrently with knowledge of sequence and loop.

Figure 4.

Example of Question From the Pre- and Post-Test, “Which Solution Takes the Boat to the Island, While Collecting the Most Coins?.”

To measure students’ learning interest, we adapted the validated questionnaires developed by Hwang and Chang (2011). The questionnaire of interest in learning consists of five items where participants had to indicate their answer on a 5-point Likert scale. Some examples of statements in this questionnaire are: “Learning more about CT is interesting” and “Other lessons do not attract me as much as the lesson about computational thinking.”

The questionnaire of learning satisfaction, adapted from Chu et al. (2010a), includes six items where participants had to indicate their answer on a 5-point Likert scale. Some examples of this questionnaire include: “I like to learn more with this learning approach” and “Learning with this approach is more challenging and interesting than learning with other traditional approaches.”

The questionnaire of technology acceptance, adapted from Hwang et al. (2013), includes eight items on the same 5-point Likert scale, comprising four questions for “Perceived usefulness” and another four for “Perceived ease of use.” Some examples of the items in the technology acceptance questionnaire are: “It was not difficult for me to learn (for the experimental group: “to play the game”; for the control group: “through PowerPoint with multimedia”), “The (for the experimental group: “game”; for the control group: “PowerPoint with multimedia”) was helpful for me in acquiring new knowledge,” and “Learning with (for the experimental group: “the game”; for the control group: “PowerPoint”) is more useful than the traditional computer-assisted learning approaches.”

The questionnaire of flow states, adapted from Pearce et al. (2005), includes six items where participants had to indicate their answer on a 5-point Likert scale. Some examples of the statements in this questionnaire include: “I was absorbed intensely by the (for the experimental group: “game”; for the control group: “PowerPoint with multimedia”) and its challenges” and “During the learning activity (for the experimental group: “and gameplay”), time seemed to pass fast.”

The Cronbach’s alpha values for the questionnaire of interest in learning, learning satisfaction, and flow states are .77, .84, and .91, respectively. Additionally, the Cronbach’s alpha values for the perceived ease of use and usefulness questionnaires (both dimensions of technology acceptance) are .84 and .78, respectively.

Data Analysis Methods

Various data analysis methods were used to analyze the collected data. These include descriptive statistics for describing the basic features of the data; one-way analysis of covariance (ANCOVA)—with the pre- and post-test scores of CT knowledge as the covariate and dependent variable, respectively, and learning approach as the independent variable—for evaluating the effect of the teaching approaches on enhancing students’ CT knowledge and skills; and independent sample t-tests to investigate the differences in the student interest, satisfaction, flow states, and technology acceptance when learning CT of the two groups.

Results

CT Knowledge and Skills

The first research question seeks to examine the possible effect of the AutoThinking game in regard to enhancing students’ CT knowledge (including conceptual knowledge and skills). To this end, we collected and used the students’ pre- and post-test scores. We analyzed data from 79 students (36 in the experimental and 43 in the control group).

According to the ANCOVA result, shown in Table 1, the mean and adjusted mean score of the control group is 8.18 and 8.26, respectively, whereas the mean and adjusted mean score of the experimental group is 9.00 and 8.90, respectively. What is more, the result reveals that both groups’ post-test scores are significantly different when the effect of their pre-test scores was excluded (F = 6.66, p = 0.01 < 0.05). This finding shows the effectiveness of the AutoThinking game compared to the traditional technology-enhanced learning approach in improving students’ CT knowledge.

Table 1.

ANCOVA Analysis of CT Knowledge.

Group	N	Mean	SD	Adjusted mean	SE	F
Experimental group	36	9.00	1.06	8.90	0.18	6.66*
Control group	43	8.18	1.31	8.26	0.16

*p<0.05.

Learning Interest

To answer the second research question aimed at better understanding students’ interest in learning CT with the adaptive educational game, we used data from the post-questionnaires collected from both groups of students. We excluded data from one and five students in the experimental and control group, respectively, as they did not fully answer the questionnaires. Given the exclusion, we analyzed data from 73 students (35 in the experimental and 38 in the control group). Based on our findings, shown in Table 2, students in the experimental group perceived greater interest in learning CT than the control group, with average ratings of 4.20 and 3.90, respectively. Comparison of both groups reveals that this difference is statistically significant (t = 2.52, p < 0.05), implying that the experimental group students who learned with the AutoThinking game showed more interest in learning CT than the control group.

Table 2.

Result of t-Test for the Interest in Learning Post-Questionnaire of Both Groups.

Group	N	Mean	S.D.	t
Experimental group	35	4.20	0.54	2.52*
Control group	38	3.90	0.47

*p < 0.05.

Learning Satisfaction

To answer the third research question aimed at better understanding students’ learning satisfaction degrees with the adaptive educational game, we used data from the post-questionnaires collected from both groups of students. Similar to analysis of learning interest, to analyze the degree of learning satisfaction, we used data from 73 students. According to our results, shown in Table 3, students in the experimental group perceived higher learning satisfaction than the control group, with average ratings of 4.47 and 4.13, respectively. Comparison of both groups reveals that this difference is statistically significant (t = 2.66, p < 0.05), implying that the experimental group students who learned with the AutoThinking game had a higher degree of learning satisfaction with CT.

Table 3.

The t-Test Result for the Learning Satisfaction Post-Questionnaire of Both Groups.

Group	N	Mean	S.D.	t
Experimental group	35	4.47	0.50	2.66^*
Control group	38	4.13	0.59

*p < 0.05.

Technology Acceptance

To answer the fourth research question, which aims at studying students’ perceptions of the use of the AutoThinking game, we used data from the post-questionnaires collected from both groups of students. The data show students’ perceptions of the ease of use and perceived usefulness of the technology used in their classes. In other words, it shows how students accept and use technology in their learning process. Similar to previous analysis, to analyze the degree of technology acceptance, we used data from 73 students. According to our results, shown in Table 4, we found that students in both groups perceived technology usefulness positively, with an average rating of 4.35 for the experimental group and 4.06 for the control group. The mean scores for perceived ease of use were 4.30 and 4.11 for the experimental and control groups, respectively. The comparison of the two groups shows that the ratings for perceived usefulness for the experimental group students are significantly higher than those of the control group (t = 2.00, p < 0.05). This finding implies that students who learned with the adaptive educational game believed in the potential of the adaptive educational game for enhancing their CT knowledge.

Table 4.

The t-Test Result for the Technology Acceptance Post-Questionnaire of Both Groups.

Dimension	Group	N	Mean	S.D.	t
Usefulness	Experimental group	35	4.35	0.61	2.00*
	Control group	38	4.06	0.59
Ease of use	Experimental group	35	4.30	0.55	1.29
	Control group	38	4.11	0.68

*p < 0.05.

Flow States

Flow theory refers to a psychological state that learners can experience when they are involved in completing tasks that challenge their skills, resulting in a deeper level of learning. In the context of the present study, a flow state implies putting players or learners in a situation with an enjoyable challenge at an appropriate level of difficulty. To answer the fifth research question aimed at studying the possible effect of the AutoThinking game on promoting students’ flow states compared to the traditional technology-enhanced learning approach, we used data from post-questionnaires collected from both groups of students. According to our results, shown in Table 5, the mean scores were 4.58 and 4.07 for the experimental and control groups, respectively. The comparison of the two groups reveals that the experimental group students showed a significantly higher flow state than the control group students (t = 3.50, p < 0.05). This finding suggests that the balance between the task challenges and students’ level of knowledge was facilitated by the adaptive educational game.

Table 5.

Result of t Test for the Flow States Post-Questionnaire of Both Groups.

Group	N	Mean	S.D.	t
Experimental group	35	4.58	0.43	3.50*
Control group	38	4.07	0.74

*p < 0.05.

Discussion

CT Knowledge and Skills

According to our findings, regarding the first research question, AutoThinking significantly improved the students’ CT knowledge, and the game did have a positive effect on the development of their CT knowledge. This involves both CT conceptual knowledge and skills. In brief, students’ different skills and conceptual knowledge of CT were targeted. That is, out of 10 items in the pre- and post-test, three targeted all four CT skills (algorithmic thinking, pattern recognition, simulation and debugging), three focused on three skills, and the other four targeted two skills. On the other hand, out of 10 items in the pre- and post-test, two targeted all three concepts (sequences, conditional logic, and loop logic), five focused on two concepts, and the other three targeted one concept. This implies that the students’ CT skills and conceptual knowledge were simultaneously assessed, and according to the results, the students who learned with the AutoThinking game appeared to have superior improvement in the growth of their CT knowledge and skills compared to those who learned with the traditional approach.

Although there are a few studies reporting the effectiveness of an educational game for improving students’ CT, they mostly employed a one-group pretest-posttest design (e.g., Zhao & Shute, 2019), and did not compare educational games with traditional technology-enhanced learning approaches for improving students’ CT (including both conceptual knowledge and skills). Even though further research is required to find the main reason behind the significant improvement in the CT knowledge of students who learned with AutoThinking, one possible reason could be the adaptivity feature of the game that supports individual needs during both the gameplay and the learning process. Students could, in a timely manner and according to their level of skills, receive feedback, hints, and learning materials to help them succeed in the game. More explicitly, the game adaptively supports students by offering different types of feedback, tutorials, and hints on various CT concepts and skills (e.g., sequence, loop logic, algorithmic thinking, and debugging). Furthermore, the game offers adaptivity in the gameplay that could be a reason for the students’ retention, motivation, and success. Providing students with different types of adaptivity during the gameplay and learning could both facilitate the learning process and improve students’ motivation to learn CT. From the perspective of metacognition theory (Desautel, 2009), which refers to the importance of guiding students to identify their own learning status and to find alternative ways to solve problems, the provision of feedback, hints, and learning materials enabled the students to reflect on their gaming and learning status, which encouraged them to not only think in depth, but also to reconsider whether there were better ways to deal with the problems. To this end, several researchers have also reported that adaptivity and personalization in educational games facilitate reaching their full educational potential (e.g., Hooshyar et al., 2019b; Hwang et al., 2010b; Kickmeier-Rust et al., 2011; Sampayo-Vargas et al., 2013).

Learning Interest and Satisfaction

Regarding the second research question, we found that the increase in learning interest of the students using the game was statistically significantly higher compared to those who learned with the traditional approach. This shows that even though CT has recently been added to the formal school curricula (it is a fairly new subject), and several researchers have found that many students show negative behavior toward learning computer science-based subjects in early education (e.g., Yardi & Bruckman, 2007), our game could successfully improve students’ interest in learning CT. In general, this shows that learning CT through gameplay interests students more than the traditional teaching approaches. In regard to the third research question, our findings show that students learning with the adaptive educational game show a higher learning satisfaction with CT than students who learned with the traditional technology-enhanced learning approach. One reason for this improvement could be that elementary school students often prefer to learn through games rather than using traditional approaches. Our findings regarding interest in learning CT and learning satisfaction are in line with those reported by researchers in other disciplines (e.g., Hwang et al., 2015; Sharma et al., 2019).

Technology Acceptance and Flow States

Concerning students’ technology acceptance, the focus of the fourth research question, we found that students in both groups perceived technology usefulness positively. The comparison of the two groups shows that the ratings for perceived usefulness of the experimental group students are significantly higher, while there are no statistically significant differences between the perceived ease of use of the two groups. This finding implies that students who learned with the adaptive educational game believed in the potential of the adaptive educational game for enhancing their CT knowledge. Feedback collected during the experiment also confirms that the students mostly perceived AutoThinking as a very useful approach for learning CT that is easy to use. For example, one of the students stated that “AutoThinking is a very good and useful game for learning computational thinking.” Another student mentioned that “the lesson in which we used the game was nice and fun.” This finding indicates that students usually prefer learning CT through gameplay than through traditional teaching approaches.

Finally, regarding the fifth research question, we found that the experimental group students showed higher flow states than the control group. From the perspective of flow theory (Killi et al., 2012), when learners are situated in an activity that makes them experience full involvement and enjoyment, they are likely to pay full attention to the activity and temporarily forget their anxiety or sorrows. Gameplay has been recognized by several researchers as a potential approach to situating people in a flow state (Hamari et al., 2016). This finding suggests that the balance between the task challenges and students’ level of knowledge was facilitated by the adaptive educational game. This finding is in line with those reported by previous researchers in other disciplines (e.g., Chang et al., 2012; Liu et al., 2011).

Limitations and Future Work

A few limitations of this study could be regarded as directions for future research. For example, the sample size of our study was rather small and the experiment condition was limited to one school and a single age group. Various experimental settings can be used to study the effectiveness of the game under different conditions, for example, using a larger sample of participants, participants of different age groups, and providing more time for students to play the game. Another limitation of our study was neglecting the game data collected during the gameplay. This could help us understand how students learn and employ CT skills and concepts in different situations of the game. Furthermore, regarding the transferability of the gained CT knowledge, it would be interesting to design different situations, in different domains other than CT, and possibly investigate how students apply their gained CT knowledge to solve various problems in different contexts.

Conclusions

In this study, a CT-oriented adaptive educational computer game-based learning approach to promoting learners’ CT knowledge and skills was implemented. The experimental results show that the proposed approach improved students’ conceptual knowledge and CT skills as well as their learning interest, satisfaction, flow states, and technology acceptance when learning computational thinking. The findings reveal that the integration of adaptivity into educational computer games can improve students’ computational thinking not only in relation to conceptual knowledge but also CT skills, in addition to improving student interest, satisfaction, flow states, and technology acceptance when learning computational thinking.

This study has some implications for educational researchers and practitioners. First, both CT conceptual knowledge and skills should be targeted together as they supplement each other. One challenge in targeting and promoting CT skills is to assess these skills. Our findings showed that automatic assessment of students’ CT skills would help to improve their skills, besides CT concepts. Therefore, educators should pay more attention to the assessment and promotion of students’ CT skills in relation to their conceptual knowledge. Second, individual needs of learners should be taken into account when promoting CT. To address such needs, educational computer games can be a great solution. More explicitly, our findings showed that engaging students with individually tailored gameplay could support improving their CT knowledge and skills. Thus, educators should pay more attention to individual students’ needs while promoting their CT knowledge and skills. Third, employing educational computer games in fostering CT skills and knowledge should be considered more seriously. As supported by the findings of this study, educational computer games can not only engage and motivate students in learning CT, but can also support improving their CT knowledge and skills. Therefore, educators should be more open to employing educational computer games in their teaching.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the University of Tartu ASTRA Project PER ASPERA, financed by the European Regional Development Fund; the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01405) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation); and the Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques).

ORCID iDs

Danial Hooshyar

Margus Pedaste

Yeongwook Yang

Note

Author Biographies

Danial Hooshyar received his Ph.D. degree from Faculty of Computer Science at University of Malaya, Malaysia. He is currently a senior research fellow in the Centre for Educational Technology at the University of Tartu in Estonia, and a research professor in Department of Computer Science and Engineering at Korea University in South Korea. His research interest includes AI in education, adaptive educational systems, and educational technology.

Margus Pedaste received his masters degree in biology education, and Ph.D. in biology and earth science education from the University of Tartu, Tartu, Estonia. He is currently a Professor of Educational Technology at the University of Tartu, Tartu, Estonia. His research interests include effect of educational technologies, virtual and augmented reality, science education, teacher education and agency.

Yeongwook Yang received his masters degree from computer science education, and Ph.D. from department of computer science and engineering at Korea University, Seoul, South Korea. He is currently a Senior Researcher of University of Tartu, Tartu, Estonia. His research interests include information filtering, recommendation system, educational data mining, and deep learning.

Liina Malva received her masters degree in educational sciences, and is currently pursuing Ph.D. at the University of Tartu, Tartu, Estonia. She is currently a Junior Research Fellow in Teacher Education at the University of Tartu, Tartu, Estonia. Her research interests include teachers’ knowledge, inclusive education, and teacher education.

Gwo-Jen Hwang is a chair professor at the Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology as well as Department of Medical Humanities and Education, School of Medicine, Kaohsiung Medical University. His research interests include mobile learning, digital game-based learning, flipped classroom and AI in education.

Minhong Wang (Maggie) received her Ph.D. degree in information systems from City University of Hong Kong. She is currently a Professor and Director of the Laboratory for Knowledge Management & E-Learning (KM&EL Lab) in the Faculty of Education, The University of Hong Kong. She is also Eastern Scholar Chair Professor at East China Normal University. Her research focuses on technology-enhanced learning, creative thinking, and complex problem solving.

Heuiseok Lim received the B.S., M.S., and Ph.D. degrees in computer science and engineering from Korea University, Seoul, South Korea, in 1992, 1994, and 1997, respectively. He is currently a Professor with the Department of Computer Science and Engineering, Korea University. His research interests include natural language processing, machine learning, and artiﬁcial intelligence

Dejan Delev received his masters degree from educational technology at University of Tartu, Tartu, Estonia. He is currently working as an educational technologist at an elementary school in Estonia. His research interests include educational technology, teacher education and learning analytics.

References

Acquah

E. O.

Katz

H. T.

(2020). Digital game-based L2 learning outcomes for primary through high-school students: A systematic literature review. Computers & Education, 143, 103667. https://doi.org/10.1016/j.compedu.2019.103667

Ater-Kranov

Bryant

Orr

Wallace

Zhang

(2010, October). Developing a community definition and teaching modules for computational thinking: Accomplishments and challenges. In Proceedings of the 2010 ACM conference on Information technology education (pp. 143–148). ACM. https://doi.org/10.1145/1867651.1867689

Ayman, R., Sharaf, N., Ahmed, G., & Abdennadher, S. (2018, November). MiniColon; teaching kids computational thinking using an interactive serious game. In Joint International Conference on Serious Games (pp. 79–90). Springer, Cham.

Basawapatna

A. R.

Koh

K. H.

Repenning

(2010, June). Using scalable game design to teach computer science from middle school to graduate school. In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education (pp. 224–228). New York: ACM. https://doi.org/10.1145/1822090.1822154

Boucinha

R. M.

Barone

D. A. C.

Reichert

J. T.

Brackmann

C. P.

Schneider

A. M.

(2019). Relationship between the learning of computational thinking and the development of reasoning. International Journal of Advanced Engineering Research and Science, 6(6), 623–631. https://doi.org/10.22161/ijaers.6.6.71

Brennan

Resnick

(2012, April). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association, Vancouver, Canada (Vol. 1, p. 25).

Brown

N. C.

Sentance

Crick

Humphreys

(2014). Restart: The resurgence of computer science in UK schools. ACM Transactions on Computing Education (TOCE), 14(2), 9. https://doi.org/10.1145/2602484

Chang

K.-E.

L.-J.

Weng

S.-E.

Sung

Y.-T.

(2012). Embedding game-based problem-solving phase into problem-posing system for mathematics learning. Computers & Education, 58(2), 775–786. https://doi.org/10.1016/j.compedu.2011.10.002

Chu

H. C.

Hwang

G. J.

Tsai

C. C.

Tseng

J. C.

(2010a). A two-tier test approach to developing location-aware mobile learning systems for natural science courses. Computers & Education, 55(4), 1618–1627.

10.

Cuny

Snyder

Wing

J. M.

(2010). Demystifying computational thinking for non-computer scientists [Unpublished Manuscript]. http://www.Cs.Cmu.Edu/∼CompThink/Resources/TheLinkWing.Pdf

11.

Denning

P. J.

(2009). The profession of IT Beyond computational thinking. Communications of the ACM, 52(6), 28–30.

12.

Desautel

(2009). Becoming a thinking thinker: Metacognition, self-reflection, and classroom practice. Teachers College Record, 111(8), 1997–2020.

13.

Eagle

Barnes

(2009). Experimental evaluation of an educational game for improved learning in introductory computing. ACM SIGCSE Bulletin, 41(1), 321–325. https://doi.org/10.1145/1508865.1508980

14.

Esper

Foster

S. R.

Griswold

W. G.

Herrera

Snyder

(2014). CodeSpells: Bridging educational language features with industry-standard languages. In Proceedings of the 14th Koli calling international conference on computing education research (pp. 05–14). New York: ACM. https://doi.org/10.1145/2674683.2674684

15.

Fraser

(2013). Blockly . https://developers.google.com/blockly/:Google

16.

Giannakoulas

Xinogalos

(2018). A pilot study on the effectiveness and acceptance of an educational game for teaching programming concepts to primary school students. Education and Information Technologies, 23(5), 2029–2052. https://doi.org/10.1007/s10639-018-9702-x

17.

González

M. R.

(2015). Computational thinking test: Design guidelines and content validation. In Proceedings of EDULEARN15 conference (pp. 2436–2444). Barcelona, Spain.

18.

Gouws

L. A.

Bradshaw

Wentworth

(2013, July). Computational thinking in educational activities: An evaluation of the educational game light-bot. In Proceedings of the 18th ACM conference on Innovation and technology in computer science education (pp. 10–15). ACM. https://doi.org/10.1145/2462476.2466518

19.

Grover

Pea

(2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051

20.

Guzdial

(2008). Education paving the way for computational thinking. Communications of the ACM, 51(8), 25–27. https://doi.org/10.1145/1378704.1378713

21.

Haddad

R. J.

Kalaani

(2015, March). Can computational thinking predict academic performance? In 2015 IEEE integrated STEM education conference (pp. 225–229). IEEE. https://doi.org/10.1109/ISECon.2015.7119929

22.

Hamari

Shernoff

D. J.

Rowe

Coller

Asbell-Clarke

Edwards

(2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170–179. https://doi.org/10.1016/j.chb.2015.07.045

23.

Harvey

Mönig

(2010). Bringing “no ceiling” to Scratch: Can one language serve kids and computer scientists. Proc. Constructionism, 1–10.

24.

Hooshyar

Lim

Pedaste

Yang

Fathi

Yang

(2019a, December). In Lisbet Rønningsbakk, Ting-Ting Wu, Frode Eika Sandnes,Yueh-Min Huang (Eds.), AutoThinking: An adaptive computational thinking game. In International conference on innovative technologies and learning (pp. 381–391). Springer.

25.

Hooshyar

Yousefi

Lim

(2019b). A systematic review of data-driven approaches in player modeling of educational games. Artificial Intelligence Review, 52(3), 1997–2017.

26.

Hwang

G. J.

Chang

H. F.

(2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56(4), 1023–1031.

27.

Hwang

G. J.

Chiu

L. Y.

Chen

C. H.

(2015). A contextual game-based learning approach to improving students' inquiry-based learning performance in social studies courses. Computers & Education, 81, 13–25.

28.

Hwang, G.-J., Kuo, F.-R., Yin, P.-Y., & Chuang, K.-H. (2010). A heuristic algorithm for planning personalized learning paths for context-aware ubiquitous learning. Computers & Education, 54(2), 404–415.

29.

Hwang

G. J.

Yang

L. H.

Wang

S. Y.

(2013). A concept map-embedded educational computer game for improving students' learning performance in natural science courses. Computers & Education, 69, 121–130.

30.

Kazimoglu

Kiernan

Bacon

MacKinnon

(2011). Understanding computational thinking before programming: Developing guidelines for the design of games to learn introductory programming through game-play. International Journal of Game-Based Learning (IJGBL), 1(3), 30–52. https://doi.org/10.4018/ijgbl.2011070103

31.

Kazimoglu

Kiernan

Bacon

Mackinnon

(2012). A serious game for developing computational thinking and learning introductory computer programming. Procedia-Social and Behavioral Sciences, 47, 1991–1999. https://doi.org/10.1016/j.sbspro.2012.06.938

32.

Kickmeier-Rust

M. D.

Mattheiss

Steiner

Albert

(2011). A psycho-pedagogical framework for multi-adaptive educational games. International Journal of Game-Based Learning (IJGBL), 1(1), 45–58. https://doi.org/10.4018/978-1-4666-1864-0

33.

Kiili

De Freitas

Arnab

Lainema

(2012). The design principles for flow experience in educational games. Procedia Computer Science, 15, 78–91. https://doi.org/10.1016/j.procs.2012.10.060

34.

Kuruvada

Asamoah

Dalal

Kak

(2010). Learning computational thinking from rapid digital game creation (pp. 31–36). https://corescholar.libraries.wright.edu/infosys_scm/3

35.

Land

S. M.

(2000). Cognitive requirements for learning with open-ended learning environments. Educational Technology Research and Development, 48(3), 61–78. https://doi.org/10.1007/BF02319858

36.

Lee, I., Martin, F., Denner, J., Coulter, B., Erickson, W. A. J., Malyn-Smith, J., & Werner, L. (2011). Computational thinking for youth in practice. Acm Inroads, 2(1), 32–37. https://doi.org/10.1145/1929887.1929902

37.

Liu

C. C.

Cheng

Y. B.

Huang

C. W.

(2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57(3), 1907–1918. https://doi.org/10.1016/j.compedu.2011.04.002

38.

Luo

Antonenko

P. D.

Davis

E. C.

(2020). Exploring the evolution of two girls’ conceptions and practices in computational thinking in science. Computers & Education, 146, 103759. https://doi.org/10.1016/j.compedu.2019.103759

39.

Lye

S. Y.

Koh

J. H. L.

(2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51–61. https://doi.org/10.1016/j.chb.2014.09.012

40.

Malva

Hooshyar

Yang

Pedaste

(2020, July). Engaging Estonian primary school children in computational thinking through adaptive educational games: A qualitative study. In 2020 IEEE 20th international conference on advanced learning technologies (ICALT) (pp. 188–190). IEEE.

41.

Meerbaum-Salant

Armoni

Ben-Ari

(2011, June). Habits of programming in scratch. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (pp. 168–172). Darmstadt, Germany. https://doi.org/10.1145/1999747.1999796

42.

Palts

Pedaste

(2020b). A model for developing computational thinking skills. Informatics in Education, 19(1), 113–128.

43.

Papert

(1990). Children, computers and powerful ideas. BasicBooks.

44.

Partovi

Razavi

M. R.

(2019). The effect of game-based learning on academic achievement motivation of elementary school students. Learning and Motivation, 68, 101592. https://doi.org/10.1016/j.lmot.2019.101592

45.

Pearce

J. M.

Ainley

Howard

(2005). The ebb and flow of online learning. Computers in Human Behavior, 21(5), 745–771. https://doi.org/10.1016/S0747-5632(04)00036-6

46.

Perković

Settle

Hwang

Jones

(2010, June). A framework for computational thinking across the curriculum. In Proceedings of the fifteenth annual conference on innovation and technology in computer science education (pp. 123–127). ITiCSE ’10: New York. https://doi.org/10.1145/1822090.1822126

47.

Repenning

Webb

D. C.

Koh

K. H.

Nickerson

Miller

S. B.

Brand

Horses

I. H. M.

Basawapatna

Gluck

Grover

(2015). Scalable game design: A strategy to bring systemic computer science education to schools through game design and simulation creation. ACM Transactions on Computing Education (TOCE), 15(2), 11. https://doi.org/10.1145/2700517

48.

Resnick

Maloney

Monroy-Hernández

Rusk

Eastmond

Brennan

Millner

Rosenbaum

Silver

Silverman

(2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67. https://doi.org/10.1145/1592761.1592779

49.

Sampayo-Vargas

Cope

C. J.

Byrne

G. J.

(2013). The effectiveness of adaptive difficulty adjustments on students' motivation and learning in an educational computer game. Computers & Education, 69, 452–462. https://doi.org/10.1016/j.compedu.2013.07.004

50.

Sharma

Papavlasopoulou

Giannakos

(2019). Coding games and robots to enhance computational thinking: How collaboration and engagement moderate children’s attitudes? International Journal of Child-Computer Interaction, 21, 65–76. https://doi.org/10.1016/j.ijcci.2019.04.004

51.

Shute

V. J.

Sun

Asbell-Clarke

(2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003

52.

Tsarava

Moeller

Pinkwart

Butz

Trautwein

Ninaus

(2017, October). Training computational thinking: Game-based unplugged and plugged-in activities in primary school. In European conference on games based learning (pp. 687–695). Academic Conferences International Limited.

53.

Wang

P. S.

(2016). From computing to computational thinking. Chapman and Hall/CRC. https://doi.org/10.1201/9781315115320

54.

Waterman

K. P.

Goldsmith

Pasquale

(2019). Integrating computational thinking into elementary science curriculum: An examination of activities that support students’ computational thinking in the service of disciplinary learning. Journal of Science Education and Technology, 29(1), 53–64. https://doi.org/10.1007/s10956-019-09801-y

55.

Weintrop

Wilensky

(2012, August). RoboBuilder: A program-to-play constructionist video game [Paper presentation]. In Proceedings of the Constructionism 2012 Conference. Athens, Greece.

56.

Wilensky

(2001, August). Modeling nature’s emergent patterns with multi-agent languages. In Proceedings of EuroLogo (pp. 1–6). Linz, Austria.

57.

Wing

J. M.

(2006). Computational thinking. Communications of the ACM, 49(3), 33–35.

58.

Wing

J. M.

(2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725.

59.

Wing

J. M.

(2011). Research notebook: Computational thinking—What and why. The Link Magazine, 6.

60.

Yardi

Bruckman

(2007, September). What is computing? Bridging the gap between teenagers' perceptions and graduate students' experiences. In Anderson, R., Fincher, S., & Guzdial, M. (Eds.) Proceedings of the third international workshop on computing education research (pp. 39–50). ACM: New York.

61.

Zhang

Nouri

(2019). A systematic review of learning computational thinking through Scratch in K-9. Computers & Education, 141, 103607. https://doi.org/10.1016/j.compedu.2019.103607

62.

Zhao

Shute

V. J.

(2019). Can playing a video game foster computational thinking skills? Computers & Education, 141, 103633. https://doi.org/10.1016/j.compedu.2019.103633