Integrating Artificial Intelligence and Computational Thinking in Educational Contexts: A Systematic Review of Instructional Design and Student Learning Outcomes

Abstract

A growing body of research is focusing on integrating artificial intelligence (AI) and computational thinking (CT) to enhance student learning outcomes. Many researchers have designed instructional activities to achieve various learning goals within this field. Despite the prevalence of studies focusing on instructional design and student learning outcomes, how instructional efforts result in the integration of AI and CT in students’ learning processes remains unclear. To address this research gap, we conducted a systematic literature review of empirical studies that have integrated AI and CT for student development. We collected 18 papers from four prominent research databases in the fields of education and AI technology: Web of Science, Scopus, IEEE, and ACM. We coded the collected studies, highlighting the students’ learning processes in terms of research methodology and context, learning tools and instructional design, student learning outcomes, and the interaction between AI and CT. The integration of AI and CT occurs in two ways: the integration of disciplinary knowledge and leveraging AI tools to learn CT. Specifically, we discovered that AI- and CT-related tools, projects, and problems facilitated student-centered instructional designs, resulting in productive AI and CT learning outcomes.

Keywords

artificial intelligence computational thinking instructional design learning outcomes systematic review

Introduction

Artificial intelligence (AI), since its introduction in the mid-20^th century, has evolved at an unprecedented pace, transforming from theoretical concepts to major technological advances. The field of AI is rooted in the development of digital computers and computer sciences. John McCarthy, one of the founders of the AI discipline, proposed that “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it” (McCarthy et al., 2006, p. 12). He further described AI development as an attempt “to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves” (McCarthy et al., 2006, p. 12). AI involves the development of intelligent machines that can perform tasks that typically require human intelligence, such as learning, reasoning, problem-solving, perception, and natural language processing (Choi, 2023). AI applications span industries and everyday life (Gansser & Reich, 2021), and integrating AI technology, concepts, and skills in education has received increasing attention worldwide. This trend reflects a growing interest in and the need to prepare schoolchildren for a modern, AI-powered society.

Numerous educational researchers have utilized AI in their studies, suggesting that applying AI in education has the potential to improve the quality of education administration, enhance instruction, and foster student learning outcomes (Chen, Lai, & Lin, 2020). For instance, Ingkavara et al. (2022) developed a personalized learning system to support students’ learning experiences based on their individual needs, abilities, and learning styles. Students reported better self-regulated online learning in the study. Conati et al. (2021) conducted a case study with intelligent tutoring systems that provided students with explicit explanations of AI-driven hints and feedback. The study provided initial evidence regarding the potential value of personalized explainable AI. Vittorini et al. (2021) developed an automated grading system, which saved correction time and provided more support for students to solve exercises. The system not only helped students achieve higher exam grades but also assisted professors in identifying correction errors. Lee et al. (2021) developed deep neural network models that can predict student learning outcomes by using video learning behaviors as input. The systems proved helpful in identifying underperforming students and providing them with additional assistance.

Further, AI-integrated studies have adopted different instructional designs to improve student learning outcomes. Available examples include gamified learning (Cheah, 2021), story-based learning (Fang et al., 2023), case study (Conati et al., 2021), and peer learning (McLaren et al., 2010). The integration of AI and computational thinking (CT) represents an emerging area of interest for educational researchers (Hsu et al., 2023; Huang & Qiao, 2022). Originating from Wing (2006), CT is a problem-solving technique that is increasingly considered a critical skill for success in today’s digital world. CT draws on concepts from computer science, mathematics, and other disciplines (Ng & Cui, 2021; Voogt et al., 2015; Weintrop et al., 2016). It involves breaking down complex problems into smaller, more manageable parts, and using algorithms and logical reasoning to analyze and solve them (Brennan & Resnick, 2012). AI and CT were integrated across various subjects, such as STEAM-related domains (Estevez et al., 2019; Gadanidis, 2017; Huang & Qiao, 2022) and social science areas (Jiang et al., 2023). Researchers have observed that the integration of AI and CT in education can help students develop critical thinking, problem-solving, and technology skills that are essential in the 21st century (Huang & Qiao, 2022; Lin & Chen, 2020).

Studies integrating AI and CT have shown a mutual relationship between the two domains regarding their relevant tools, concepts, and competencies (Hsu et al., 2023; Wang et al., 2022). For example, AI making engages students’ CT practices (Hsu & Chen, 2022), the involvement of CT deepens AI disciplinary knowledge (Estevez et al., 2019; Shamir & Levin, 2021), and the application of AI and CT in the learning environment improves students’ learning performance (Wang et al., 2022). Nevertheless, while CT seems to be constructive in learning some AI-related themes, it cannot be concluded that CT-facilitated AI learning activities would be superior to other instructional approaches in fostering student learning outcomes or that all domains of CT are generally suitable for integration with AI elements (Gadanidis, 2017).

Further, as the thinking patterns for operating AI and CT are potentially different, learners may experience some challenges when solving problems in AI- and CT-integrated learning environments (Hsu & Chen, 2022; Lai et al., 2021). Overall, investigations of the interrelationships and components of overlap between AI and CT in educational practices are underdeveloped. In particular, educators have emphasized instructional design as a vital influencing factor for student learning outcomes (Morrison et al., 2019). An investigable question would be how to effectively design AI-relevant activities to foster student development in CT and how to appropriately design CT-rich activities to enhance student learning in AI.

Existing Reviews, the Present Review, and Research Questions

Some existing reviews have focused on the utilization of cutting-edge technologies for student learning outcomes, of which AI and CT were investigated separately (Grover & Pea, 2013; Kafai & Proctor, 2022; Roll & Wylie, 2016). However, few reviews have discussed the integration or interplay between AI and CT. The most relevant work was contributed by Gadanidis (2017) and Zeng (2013). Specifically, Gadanidis (2017) constructed a theoretical framework regarding the intersection of AI, CT, and mathematics education based on relevant literature for young students from the sociocultural perspective. However, this study did not use a systematic review methodology; therefore, the representativeness of the studies reviewed is noted, and its disciplinary focus on mathematics education has yet to address the interdisciplinary nature of AI and CT. In an opinion paper recognizing overlaps between CT and AI thinking, Zeng (2013) proposed that AI thinking should go beyond what CT offers and that, although connected, AI thinking and AI encompass different scopes. Therefore, future research should address the multifaceted AI thinking fostered by a variety of tool-based instructional designs. While “AI goes beyond the logic- and algorithmic-based approach used by traditional CT […] AI technology is based on some of the fundamental concepts of CT, such as abstractions, pattern recognition, and algorithmic thinking” (Zerega & Milrad, 2023, p. 213). This suggests an intrinsic relationship between AI and CT, which may become crucial in examining how AI and CT can be effectively incorporated into K-12 education to maximize learning outcomes. Among the many critical roles that systematic reviews serve, this paper provides syntheses of the state of knowledge in a field and thereby enabling the identification of priorities for future research (Page et al., 2021) and informing practice (Munn et al., 2018). In particular, we aim to gain an in-depth understanding of the existing empirical studies of educational practices that explicitly connect AI and CT while designing learning activities for student development.

Our systematic review serves the dual purpose of (ⅰ) reviewing the characteristics of teaching and learning design in the environment of the collected studies, including their research methodology, learning contexts, educational tools, learning theories, instructional designs, and student learning outcomes; and (ⅱ) analyzing and discussing the integration of AI and CI based on the interventions of the extant empirical studies. To achieve these purposes, we reviewed the studies that reported details of the instructional designs, student learning processes, and outcomes in the learning activities with AI and CT integrated. Correspondingly, we posed the following research questions (RQs) in the study:

(1) In what educational contexts are AI and CT integrated?

(2) What tools and instructional designs are used for the integration of AI and CT in educational contexts?

(3) What are the learning outcomes of educational activities that integrate AI and CT?

(4) How do AI and CT interact in student development?

Methodology

We followed the guidance of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) to conduct the systematic review (Moher et al., 2010). This section reports the methods used in the paper collection and data analysis.

Paper Collection

Following the suggestions of our research team (e.g., consisting of four members with expertise in instructional design and educational technology) and a university librarian, we adopted four representative databases in educational technology and AI, including Web of Science (WOS), Scopus, IEEE, and ACM. The search keywords were designed collaboratively with the research team and the university librarian. We also referred to previously published studies, such as Zawacki-Richter et al. (2019) and Tang et al. (2021), at this stage. We used the “all fields” option of the databases, which allowed the keyword search to involve the title, abstract, keywords, and full text of the documents. We used the key words “computational thinking” AND several word strings (“artificial intelligence” OR “machine intelligence” OR “intelligent support” OR “intelligent virtual reality” OR “chat bot*” OR “machine learning” OR “automated tutor” OR “personal tutor*” OR “intelligent agent*” OR “expert system*” OR “neural network*” OR “natural language processing” OR “chatbot*” OR “intelligent system*” OR “intelligent tutor*”). Further, we developed a series of inclusion and exclusion criteria for the paper selection phase (Table 1). These criteria were co-designed by our research team to ensure that the studies collected were relevant to our research goal of understanding the integration of AI and CT in instructional activities for student learning outcomes.

Table 1.

Inclusion and Exclusion Criteria for Paper Selection.

Criterion	Inclusion Criteria	Exclusion Criteria
Article attribute	Peer reviewed journal papers	Non-peer-reviewed papers, book chapters, conference papers/proceedings, unpublished works
Language	English	Papers written in other languages
Education setting	Non-special education	Special education for disabled or ability impaired students
Research context	Integrated AI and CT for student development	No AI and CT integration in the learning environment; activities not for student development
Study attribute	Empirical studies implemented instructional activities/interventions	Opinion papers, literature review, meta-analysis, conceptual study, book review
Research focus	Student learning in the AI- and CT-integrated learning environment	Studies involved AI or CT, but AI and CT were not integrated in the learning environment
Participants	Has learners participated in the interventions	Do not involve research participants

The paper search step was conducted in early March 2023. To cover as many relevant studies as possible, we did not set constraints on the publication date, research location, or cultural background of the studies. Initially, we retrieved 257 papers from WOS, 4996 papers from Scopus, 812 papers from IEEE, and 734 papers from ACM. By selecting the “English” and “article” options to fit the paper selection criteria, we identified 86 studies from WOS, 2032 papers from Scopus, 136 papers from IEEE, and 89 papers from ACM for further screening. We exported the article title, authors, keywords, abstract, and journal name from the four databases in CSV format. We then aggregated the four documents into one Excel spreadsheet. We removed duplications, checked the titles and abstracts of the articles, and kept 38 papers for further selection. After a full-text review, 16 papers were retained. Subsequently, we manually searched the reference lists of the retained studies using the keywords “artificial intelligence” and “computational thinking.” The manual search resulted in the identification of two additional papers. Eventually, we obtained 18 papers for the review. The entire paper selection process is represented in Figure 1.

Figure 1.

PRISMA flow chart for paper collection.

Dataset and Data Analysis

We categorized the basic attributes of the empirical studies, including their research methods, sample size, intervention time, and data collection method, as done by previous researchers (Liang et al., 2021; Ye et al., 2023). Regardless of our qualitative-oriented research questions to investigate the attributes of the studies that have integrated AI and CT, mixed method studies were predominantly represented in our sample, while qualitative methods were used the least (e.g., with 44.44% mixed method, 38.89% quantitative, and 16.67% qualitative). Over half of the studies involved relatively large sample sizes with more than 31 learners (66.67%). Most studies organized their interventions into multiple sessions (77.78%), of which 64.29% were short-term (e.g., less than 10 weeks). Further, 55.56% of the collected studies reported the duration of their interventions, of which 60% lasted more than 5 hours. Additionally, questionnaires/surveys (83.33%) and tests (50%) were found to be the most frequently used data collection methods to reveal or evaluate students’ learning in the AI- and CT-integrated studies, followed by interviews (27.78%) and video recordings (22.22%).

We conducted qualitative analysis combined with deductive and inductive coding to analyze data, aiming at identifying and refining all possible categories from the empirical studies to achieve saturation of the codes. A two-level coding scheme was co-constructed in response to the first three research questions. The process started with the research team reading the 18 articles and proposing the first level of the coding scheme to the research group for discussion and confirmation. Regarding the research context (RQ1), the coding categories included the education level of learners, educational settings, and discipline context. Regarding the research interventions (RQ2), the coding categories included applied AI algorithms, constructed AI artifacts, adopted CT platform, and instructional design. Concerning the learning outcomes (RQ3), the coding categories included CT-related learning outcomes, AI-related learning outcomes, affective or psychological states, and contemporary competencies. Further, the first and second authors coded five studies (27.78% of the sample studies) and developed the second level of the coding scheme independently. They then discussed the coding results with other team members to clarify the meaning of the codes and improve the items that were relevant to the RQs by adding, removing, and adjusting the codes. Eventually, the two-level coding scheme and the corresponding definitions of the codes were created and finalized (as shown in Appendix 1).

Regarding RQ4, our research team adopted constant comparative analysis (Strauss & Corbin, 1990) to help us generate systematic findings where “data are continually compared with each other to allow categories to emerge and for relationships between these categories to become apparent” (Harding, 2006, p. 131). The constant comparative method has its roots in, but is not limited to, classical grounded theory (Glaser & Strauss, 2017); it is often used in the analytical process of comparing similarities and differences between different data, which can be very useful for exploring questions of the interplay and relationships in a systematic review. Thus, following the guidance of constant comparative method, the two authors first adopted open coding to generate initial codes—by extracting information from the empirical studies that were relevant to the AI- and CT-integrated instructional activities and student learning outcomes. Later, axial coding and selective coding were utilized to generate upper-level categories to connect the initial codes.

After designing the coding scheme, the two authors coded the collected studies, yielding an acceptable interrater agreement of over 80% (Belur et al., 2021). The two authors held group meetings to address any conflict. When difficulties in decision making were encountered, they approached other experts in AI and CT for suggestions and discussed further until a consensus was reached.

Results

This section reports the review findings. We begin by revealing the educational context of the studies that have integrated AI and CT, including learners’ education levels, educational settings, and the discipline context of the collected studies. We then describe the pedagogical perspective of the collected studies, including the tools and instructional designs utilized. We also discuss the learning outcomes of the reviewed studies, as well as the AI and CT integrations.

Educational Context of AI and CT Integration

Education Levels and Settings

We observed that more studies were implemented in higher education contexts (55.56%), followed by the elementary education level (22.22%). This phenomenon may be driven by the urgency to equip higher education students with AI- and CT-related literacy and competencies to become successful in the 21st century when they enter the job market (Khan et al., 2022), as well as by the development of more user-friendly instructional tools to scaffold students’ early exposure to AI and CT during their critical cognitive developmental period (Lefa, 2014). Among the collected studies, Lin et al. (2021) invited participants from higher professional education. This indicates the possibility of designing AI- and CT-integrated activities for learners with varying knowledge and skill backgrounds. Their instructional activities are supported by courses that combine AI and the internet of Things (IoT) (i.e., AIoT), together with augmented reality (AR) technology, to enhance learners’ CT competence in an information professional context.

In particular, Lin et al. (2021) investigated how different prior knowledge situations of AIoT and AR influence learners’ AIoT-facilitated CT competence training. They revealed that learners’ prior knowledge is relevant to cognitive usefulness, which further affects their acceptance of the technology learning system. Further, the researchers considered learners’ problem solving in the actual application environment of their designs. The comprehensive problems encountered during the construction of different Arduino coding applications (e.g., smart systems for agriculture, homes, campuses, lighting, or transportation) afforded diverse learning tasks involving AIoT and CT, linked learners’ prior knowledge with the learning activities, and built relatedness between their life experiences and the learning content. The relevant study showed the promise of utilizing AI- and CT-integrated activities to achieve learning outcomes for learners from different knowledge backgrounds. Nevertheless, the review findings raised questions surrounding problem-solving in the AI- and CT-integrated learning environment that predominantly emphasized creating simulations to present comprehensive authentic situations. Hence, we highlight the need for educational activities that evoke innovative solutions to solve authentic problems with AI and CT integrations.

We assessed the educational settings of the collected studies to understand whether AI- and CT-integrated instruction were applied in formal, non-formal, or informal education contexts (Eshach, 2007). Among the collected studies, 83.33% were conducted in formal settings, and 16.67% were implemented in non-formal learning environments (which were all mixed method studies with short-term interventions less than 10 weeks). In particular, the non-formal settings were contextualized in extra-curricular workshops (Estevez et al., 2019; García et al., 2020) and home-based distant learning (Shamir & Levin, 2021). Although AI- and CT-integrated activities are the most voluntary form of learning that is closest to daily life in informal learning (Eshach, 2007), no study has been conducted in daily life environments that are relatively spontaneous and unstructured compared. This indicates a lack of AI and CT integration in informal learning contexts.

Discipline Context

We explored whether the instructional activities were implemented in a disciplinary or an interdisciplinary context (English, 2016). We categorized half of the interventions as interdisciplinary, with concepts from different disciplines closely linked (n = 9). Some possible combinations drew on AI, CT, IoT, and STEM/STEAM-related domains. For instance, the interdisciplinary practices of AI, CT, IoT, and AR were represented in Lin et al.’s (2021) study. Similarly, Estevez et al. (2019) organized Scratch workshops, during which mathematics knowledge underlaid coding exercises to introduce some basic mechanisms of AI systems (e.g., k-means, artificial neural networks) to high school students. Jiang et al. (2023) conducted their interdisciplinary study in a journalism class, in which students refined machine learning models to classify people’s reviews of ice cream stores, enriching STEAM practice in a broader sense.

The remaining half of the studies (n = 9) reported on the integration of AI and CT in the disciplinary context, with students learning concepts and skills from different disciplines separately. We found that AI (n = 5) and programming (n = 4) were the two dominant courses adopted by the researchers for learners at different education levels. For instance, Shamir and Levin (2022) designed a machine learning (ML) curriculum to guide elementary school students from understanding ML to creating rule-driven ML systems. Hsu et al. (2022) implemented a conversational AI curriculum, in which middle school students prepared knowledge to create conversational AI projects. Notably, in these studies, learners developed CT competencies while constructing AI projects/artifacts. Furthermore, Lin and Chen (2020) organized the “Program Logic Thinking Education” course using the deep learning recommendation system to introduce programming concepts and cultivate student CT with image-based programming. Ríos Félix et al. (2020) designed an intelligent learning environment that supports emotion recognition with ML for students to learn the main concepts of CT by operating the visual programming tool Blockly. In these cases, AI platforms were adopted to facilitate the learning environment, and all the programming courses utilized block-based programming tools for CT purposes. The popularity of visualizing programming indicates researchers’ efforts to present operations and knowledge regarding AI and CT in concrete ways.

The research findings showed that most AI- and CT-integrated studies have involved elements from other domains beyond the two target disciplines. Our systematic review showed the popularity of the interdisciplinary context, as AI and CT were integrated across various subjects, such as science and social studies (Estevez et al., 2019; Huang & Qiao, 2022; Jiang et al., 2023), with evidence of a mutual relationship regarding their relevant tools, concepts, and competencies (Hsu et al., 2023; Wang et al., 2022). Overall, our investigation of the educational contexts that integrated AI and CT showed that existing studies have not achieved balanced development. Researchers are encouraged to implement more empirical studies to inform effective integration of students from secondary school levels in informal learning environments.

Pedagogical Perspective of AI and CT Integration

Adopted AI algorithms and constructed AI artifacts

As a component of pedagogical design, the AI algorithms adopted in the collected studies were examined. ML-related algorithms are most predominantly used (66.67%), followed by natural language processing (22.22%), Bayesian networks, and k-means (both 5.56%). ML falls under the umbrella of AI as a specialized field, and deep learning is a specialized area of ML. Further, deep learning algorithms and neural networks differ in the number of layers in the network (e.g., the neural network has a few layers, while a deep learning algorithm needs to have more than three layers to be considered “deep”). The reviewed studies involved different subcategories of ML-related algorithms, including conventional ML (n = 8) and deep learning/neural networks (n = 7). The limited types of ML tools utilized indicate the insufficient development of the research area and the need to adopt other AI algorithms, such as data mining, evolutionary algorithms, search and optimization, and fuzzy set theory, as well as other ML tools, such as decision trees (Liang et al., 2021).

Some of the collected studies also reported AI artifacts constructed by students (55.56%). Among these studies, classifier and recognition systems were the most frequently reported artifacts (n = 8). A range of categories were formed based on the function of the systems, including text classification (Jiang et al., 2023), voice/speech recognition (Hsu & Chen, 2022; Hsu et al., 2022, 2023), object recognition (Hsu et al., 2021), and facial recognition (Huang & Qiao, 2022). Other AI artifacts created include systems of AI agents (Shamir & Levin, 2021), AI painting, self-driving cars (Huang & Qiao, 2022), and AIoT projects (Chen, Lai, & Lin, 2020). The popularity of constructing recognition systems echoes the generally adopted ML-related algorithms, indicating that there are relatively more experience and resources available for learners to use ML tools to create recognition systems in different contexts.

CT Tools

CT activities do not merely rely on digital tools. Our review found that some researchers utilized unplugged activities to equip students with CT competencies and facilitate learners’ AI-related learning tasks (Chen, Lai, & Lin, 2020; Silapachote & Srisuphab, 2017). e.g., instead of using computing machines, Silapachote and Srisuphab (2017) structured their CT course by involving mental tools (e.g., twist competitive games) to develop learners’ metacognitive skills for AI problem solving. However, more researchers adopted plugged CT activities in their studies, of which block-based programming was the most popular category (61.11%), followed by text-based programming (11.11%).

Block-based programming enables students to work on the screen of digital equipment using the mouse or their hands. The visual script blocks that appear on the screen can be dragged and dropped to create programs. This approach also naturally and intuitively provides visual cues on how to operate program commands, lowering the threshold for programming (Weintrop, 2016). Commonly used block-based programming platforms include Scratch (Estevez et al., 2019; García et al., 2020; Shamir & Levin, 2021, 2022), MIT App inventor (Hsu & Chen, 2022; Hsu et al., 2021, 2022, 2023), and Blockly (Ríos Félix et al., 2020). Lin and Chen (2020) introduced an image-based programming system as a unique visual tool for AI and CT integration. The system involved AR- and deep-learning-facilitated recommendations and was specially designed for learners from different learning areas. The study showed that using images helped learners overcome their learning challenges of being non-programming majors and therefore encouraged them to explore CT (Lin & Chen, 2020).

Text-based programming is demanding because it requires students to understand the syntax language for coding. Researchers have adopted Arduino coding as the programming platform for their VR/AR integrated AIoT courses, but learners must have adequate knowledge of IoT and VR/AR applications (Lai et al., 2021; Lin et al., 2021). Due to the high requirements, participants from higher education levels were recruited for the studies. The instructors also established collaborations with different stakeholders to capture comprehensive situations while implementing cross-domain learning tasks. For example, school professors and industry experts were invited to jointly score learners’ final reports (Lai et al., 2021) or review their final project presentations (Lin et al., 2021).

Our research revealed findings regarding the characteristics of CT tools. The application of CT tools is relevant to learners’ attributes, including their education level, major background, and prior knowledge in programming. For example, considering that the participants all have learned programming blocks with micro: bit, Hsu et al. (2021) adopted MIT App Inventor for students’ mobile app development. Thus, the researchers facilitated elementary school students’ practice in CT by involving them in robot car-making and CT board game playing activities. AI was embedded in the activities in the MIT App Inventor through some of its additional features, such as audio classification and personal image classifiers, which are related to ML (Hsu & Chen, 2022; Hsu et al., 2021). The researchers indicated that CT tools could be adopted to foster understanding AI and cultivate AI making. Nevertheless, these studies have not been extended to investigate AI from a thinking pattern perspective.

Instructional Design and Learning Theories

We identified the commonly adopted instructional approaches in the collected studies and grouped them into categories according to the structure and organization of the learning tasks and learning processes (Figure 2). Regarding different task structures, the representative instructional design approaches for AI- and CT-integrated learning activities included project-based learning (61.11%) and problem-based learning (16.67%). In terms of fostering the learning process, there were game-based learning (11.11%), gamification (5.56%), and inquiry-based learning (5.56%). In this review, we analyzed the characteristics of the dominant instructional design approaches and assessed how they occasioned the learning activities.

Figure 2.

Instructional approaches of collected studies.

Project-based learning aims to create CT- or AI-related artifacts in the form of completing projects. This instructional design strategy has been used by Shamir and Levin (2022), who implemented four learning modules for students to learn ML, from introducing ML and practicing ML processes to constructing data-driven and rule-driven ML systems. In the study, the learners worked individually or in groups to make final projects (e.g., creating their own ML system, artificial neural network) with the designed ML modeling toolkit, which used Scratch as the interface. While completing the project, learners may encounter different challenges related to AI or CT. To meet the project requirements, they will need to use their knowledge and skills and work with their peers to make plans and decisions.

We further highlighted the adoption of problem-based instructional design in learning activities, as CT is a universal problem-solving skill applicable in different contexts (Wing, 2008). The learning objective of the relevant studies was to enable learners to adopt AI- and CT-related knowledge and skills, which were developed in the intervention activities, for algorithmic problem-solving (Li, 2019; Lin & Chen, 2020). A specific problem-based learning approach was exemplified by Silapachote and Srisuphab (2017), who integrated elements from AI into the university introductory computing curriculum (e.g., by using mental tools rather than computers) and provided practical exercises in AI (e.g., in the form of AI puzzles and games) that required learners’ collaborative problem solving beyond CT-related competencies. The problem-based learning approach empowered learners through active engagement and autonomous hands-on explorations.

A common feature shared by the instructional design approaches in the collected studies was student-centeredness. Students were regarded as active learners and problem solvers in technology-enhanced learning environments. The reviewed studies were mostly founded on learning theories related to constructivism (Estevez et al., 2019; Hsu et al., 2022, 2023; Silapachote & Srisuphab, 2017) and constructionism (Shamir & Levin, 2021, 2022). Although learning by making is the core advocate of constructionism (Papert & Harel, 1991), the specific making activities in the collected studies vary. For example, learners are creating CT- or AI-related artifacts in project-based learning, and they are producing solutions with CT or AI tools in problem-based learning. Further, though less frequently reported, game-related instructional design and inquiry-based learning are possible approaches to achieve specific objectives of cultivating student learning motivation (Hsu & Chen, 2022; Ríos Félix et al., 2020) and providing guidance for exploration (Estevez et al., 2019).

Learning Outcomes Associated with AI and CT

Integrating AI and CT indicates researchers’ assumption that composite activities can bring advantages that merely focusing on one of the two domains may not. Accordingly, an essential aspect of interest to educators is how students’ AI and CT learning interact in the learning environment and the corresponding learning outcomes of the integration. Unlike the intuitive and straightforward perceptions that the interactions of AI and CT are bidirectional and can cultivate student learning, the current review reports more granular results—regarding the complex integration of AI and CT and the challenges when coordinating the AI and CT modes of learning and thinking. We further examine the conceptual frameworks adopted to characterize AI- and CT-related learning outcomes and discuss how students learned in the contexts.

AI Learning Outcomes

Our discussion of AI learning outcomes concentrates on diverse perspectives. In the collected studies, we analyzed the reciprocal impact of AI/CT integration, in which students’ engagement in AI- and CT-integrated tasks contributed to the development of their AI literacy, and we identified the mechanisms through which this learning took place. As an essential AI-related learning outcome, AI literacy associated with the integrated learning environment was reported by the reviewed studies. We identified four aspects of AI literacy—knowing and understanding AI, using and applying AI, evaluating and creating AI, and AI ethics—as synthesized by Ng et al.’s (2021) exploratory review, and we highlighted how CT contributed to this development.

Two of the reviewed studies reported learners’ achievements in knowing and understanding AI. Estevez et al. (2019) utilized a program scaffold to conduct experiments on the fundamental mechanisms of AI systems. After the experiment, the students gained a comprehensive understanding of the basic principles and functioning of two widely used AI algorithms—k-means and artificial neural networks. Shamir and Levin (2021) developed a programmable learning environment to enable elementary school students to create their own AI agents. Ultimately, the innovative learning environment helped enhance students’ comprehension of machine learning through the construction of neural networks. Both studies used Scratch to simulate AI mechanisms, through which learners conducted CT tasks and understood the workings of AI algorithms.

In a few of the reviewed studies, using and applying AI was reported as the learning outcome. For instance, Silapachote and Srisuphab (2017) incorporated components derived from AI into their computing courses. During the course, students made efforts to think computationally to overcome unseen challenges in solving AI problems. The practical AI exercises enhanced students’ comprehension of CT and further developed their competencies in collaborative problem-solving. However, in the study, adopting AI was rarely the main goal of the learning activities; instead, developing CT was the core of the designed intervention.

The majority of the reviewed studies involved evaluating and creating AI as their learning outcomes (72.22%). In these studies, CT practices supported learners’ construction of AI, and CT exploration was the form of learning students performed. In some of the studies, learners further evaluated the effectiveness of the AI platforms they used, such as AIoT (Lai et al., 2021; Lin et al., 2021) and Chatbot (Li, 2019). According to Bloom’s taxonomy (Silapachote & Srisuphab, 2017), creating and evaluating belong to higher-level cognitive skills. Therefore, CT practices serve as the scaffold as well as the process to construct learners’ experience, so that they can build higher-level skills regarding AI artifacts/systems.

Although not purposely examined, AI ethics were mentioned by Estevez et al. (2019), who found that learners expressed more worries about the misuse of data and its consequences for privacy and personal liberty after the experiment, and by García et al. (2020), who proposed ethical issues as a concern to deal with in the introduction of AI content to schools. A common point shared by the studies was the adoption of CT platforms, such as Scratch (Estevez et al., 2019) and LearningML (García et al., 2020), to scaffold students’ learning activities. These simulation platforms not only enable learners to experience AI but also inspire their moral thinking about the new technology.

CT Learning Outcomes

Half of the studies (50%) reported the definitions or frameworks they adopted for CT. The most frequently utilized was Brennan and Resnick’s (2012) framework. We focused on the first two dimensions of the framework (e.g., CT concepts and practices), as CT perspectives were the least reported with limited information available.

A series of CT concepts prevalently reported in the reviewed studies (16.67%), including loops, conditionals, and sequences, were related to logic thinking. When engaging in AI- and CT-integrated learning tasks, students usually need to understand the learning platforms before utilizing them to design their own systems or artifacts. In addition to helping to understand the making platforms, CT concepts can assist learners in achieving specific designs of the application and exploring solutions for comprehensive problems. For example, the CT concepts of loops, conditionals, and sequences were highlighted when students created voice assistant applications (VA) through the MIT App Inventor (Hsu et al., 2023). By operating these CT concepts, learners can ensure the function of the AI application—if the users mishear or misunderstand a question, the application repeats the questions. Further, in the IoT course instructed by Lai et al. (2021), which involved IoT sensing, plot-based VR, and the AIoT platform, the CT concepts of sequences, loops, events, parallelism, conditionals, and operators were presented as learners engaged in logical thinking of solutions for the problems in different fields simulated by VR. Further, Hsu et al. (2022) reported that the students engaged with CT concepts of events, conditionals, data, sequences, loops, parallelism, and operators while they created their conversational AI agents with MIT App Inventor.

Researchers have considered learner’s logical thinking related to CT concepts in creating learning environments that integrate AI and CT (Hsu et al., 2022, 2023; Lai et al., 2021). However, the concepts were reported in groups from a general perspective for evaluation purposes. Further, they have not been examined in students’ learning processes. We suggest highlighting the adoption details of CT concepts from an individual perspective. Furthermore, we observed that various CT concepts can provide insights for future researchers to extend the application of CT concepts, such as variables and subroutines (Ye et al., 2023), to learning environments that integrate AI and CT.

Most of the reviewed studies reported students’ CT practiced in the integrated learning environment (88.89%). Based on the empirical studies, we grouped and synthesized the CT practices using a diagram that illustrates multiple levels representing their composition and interaction in learning environments that integrate AI and CT (see Figure 3). Overall, we identified system thinking, system practices, and generic skills as the three main dimensions of CT practices.

Figure 3.

Diagram of CT practices.

System thinking, which helps in analyzing and understanding complex systems, involves the competencies of examining problems from a systematic perspective. Such practices include analyzing an intricate system in its entirety, comprehending the interconnectedness within the system, employing hierarchical thinking, conveying knowledge about a system, defining systems, and handling intricacy. Based on the practices reported in the empirical studies, we further divided system thinking into three sub-categories: decomposition, abstraction, and algorithmic thinking. Specifically, decomposition means dividing a complex problem into smaller, more manageable problems. It can also be applied to AI-related tasks, in which comprehensive problems are broken down by considering the relationships between parts and wholes. For example, in the AIoT Maker course developed by Chen (2020a), learners employed decomposition to understand various critical problems in earthquake relief. They broke down the comprehensive problems into simpler questions, such as how to collect information about the earthquake scene and how to choose a location for the disaster relief center. By analyzing the sub-topics, the complex problems became simplified and more manageable issues.

Abstraction can also aid in the problem-solving process by selectively retaining key information and identifying commonalities within the problem (Jiang et al., 2023). Researchers have reported various examples of abstraction practices. For instance, pattern recognition was involved in Ríos Félix et al.’s (2020) intelligent learning environment. In the study, the students abstracted the general properties of a given pattern by modifying the properties of the 3D objects with sliders, irrespective of the object composition or structure. Lin et al. (2021) represented pattern generalization as students recognized the shared characteristics or elements across different patterns in AIoT learning—they planned their own AR sensors after observing with the available AR apps. Moreover, Silapachote and Srisuphab (2017) identified multiple layers of abstraction among the practical implementations of their study, which structured an introductory engineering course on CT by solving AI problems.

Decomposition and abstraction include breaking down a complicated problem into smaller parts and focusing on essential elements that help to make the problem easier to understand. Algorithmic thinking goes further to facilitate the process of simplification by designing a concise algorithm to address the problem effectively. The reviewed studies defined algorithmic thinking as the ability to comprehend, utilize, evaluate, and create algorithms in an AI-integrated learning environment (Chen, Lai, & Lin, 2020; Huang & Qiao, 2022). Algorithmic thinking helps establish the guidelines and sequence of tasks required to address problems, outlining the actions that can be employed when different problems arise. An alternative concept, algorithmic design, devises instruction flows that can efficiently tackle comparable problems and be employed repeatedly (Chen, 2020b). In this regard, algorithmic design is closely related to system automation, which highlights the significance of data structures, flows of controls, and recursive procedures (Silapachote & Srisuphab, 2017).

Unlike system thinking, which represents concepts from a general perspective, system practices indicate more specific programming competencies. Various system practices emerged from empirical studies that have incorporated AI-related elements. For example, in Jiang et al.’s study (2022), data modeling was regarded as an engaging activity for students to enhance their understanding and reasoning skills regarding the operational principles of AI technologies (e.g., by constructing machine learning models). In the study, debugging was recognized as a CT process essential for data modeling. Further, Hsu et al. (2023) defined syntax, testing, and iteration as the competencies constructed by learners’ CT practice while designing the AI-related voice assistants. e.g., the designed intervention may not necessarily achieve better learning outcomes (e.g., AI concepts and anxiety) than the control group.

The other dimension of CT practices we captured was generic skills, indicating that CT practices are among the social cognitive skills that are applicable, regardless of the learning environment and tool adoption. Researchers identified creativity, cooperativity, and critical thinking as the composition of CT practices as learners engage in problem-solving, specifically in the design and evaluation of an AR-facilitated deep learning recommendation system in teaching CT (Lin & Chen, 2020) or the integration of AI education with the STEAM model to cultivate learners’ CT (Huang & Qiao, 2022). These generic skills showed that learners in the AI- and CT-integrated learning environments interacted with the learning platforms to think creatively and critically, as well as to collaborate with other people to solve problems. Moreover, these skills are higher-order 21st century thinking skills that can be transferred to any problem-solving situation beyond AI and CT integrations (Weng et al., 2022, 2023).

Overall, we synthesized a diagram to present the composition of CT practices. These practices are interconnected and work simultaneously, and they are the CT learning outcomes developed together with the AI-related learning outcomes. For instance, in García et al.’s (2020) preliminary intervention to evaluate the feasibility of the LearningML course, the learners developed a series of CT practices listed in the CT framework (Brennan & Resnick, 2012). Additionally, they achieved various AI-related learning outcomes through finishing practical AI projects, including their understanding of AI in defining its concept and AI ethics regarding its potential dangers. These outcomes show that CT practices are involved in facilitating AI artifact making and AI element adoption. Accordingly, the AI- and CT-integrated learning environment provides students with explicit opportunities to engage in CT practices as they manipulate programming operations that directly impact the design of the artifact.

Integration of AI and CT Learning Outcomes

Over half of the studies reported efforts to design AI activities in parallel with their CT activity design (55.56%). This indicates that a considerable portion of researchers decided to design activities simultaneously for AI and CT rather than facilitating AI and CT independently or regarding the knowledge and skills from one area as the prerequisite for learners to achieve the learning outcomes of the other domain. In these studies, the elements of AI and CT were intertwined and mutually reinforced. As understanding how AI works in integrated studies can provide insights for future instructional designers who desire to foster student development, in what follows, we summarize two AI strategies—one of the contributions of our review study.

The most frequently adopted strategy in the collected studies is the use of AI platforms to facilitate the acquisition of students’ CT competencies. Lin and Chen (2020) and Ríos Félix et al. (2020) were elaborated on relevant examples. In both studies, the researchers reported learners’ achievement in CT competencies with AI scaffolders (e.g., recommendation learning systems and intelligent learning environments). We considered AI making as another strategy to achieve CT practice goals. For instance, in Hsu and Chen’s study (2022), students were guided to create their own AI apps on smartphones using the MIT App Inventor. In other words, while performing AI-making activities, the learners practiced CT skills and therefore performed better, for example, in learning engagement. Overall, although the number of supporting cases was limited, using AI platforms and AI making has shown their potential in facilitating learners’ CT competencies.

Furthermore, the reviewed studies revealed a tendency to connect AI with CT frameworks. Two types of AI frameworks were mentioned in the reviewed studies: extended and self-created. Specifically, Van Brummelen et al. (2019) recommended incorporating AI concepts and practices into Brennan and Resnick’s (2012) CT framework. Aligned with this recommendation, the extended AI framework has been adopted in different contexts. e.g., Hsu et al. (2022) used the extended framework while elaborating on students’ activities in their conversational AI curriculum. They reported the students’ learning in AI-related concepts (e.g., classification, prediction, and generation), practices (e.g., training, testing, and validating), and perspectives (e.g., project evaluation). The same AI framework was utilized by García et al. (2020), and Hsu et al. (2023).

By contrast, Shamir and Levin (2022) self-developed a CT-ML framework while teaching elementary school students machine learning. The researchers also utilized three dimensions to define the CT-ML framework: computational concepts, computational practices, and computational perspectives (Brennan & Resnick, 2012). Nevertheless, the items developed all presented the features of ML. For example, computational practices were grouped into machine training practices and machine validating practices. Here, computational perspectives were framed as myself, the world around me, and an awareness of bias to reflect how learners perceive themselves and their surroundings while creating ML artifacts with Scratch. The two types of AI frameworks showed how the researchers connected AI with CT frameworks and identified the foundational role of Brennan and Resnick’s (2012) CT framework in the development of AI frameworks. However, the frameworks available for research reference are limited, and insufficient attempts have been made to construct innovative AI frameworks.

Discussion and Conclusion

AI focuses on the development of intelligent machines capable of performing tasks that typically require human intelligence. It encompasses various techniques and is utilized to solve complex problems and automate tasks across a wide range of domains. CT is a cognitive process that involves abstracting, algorithm thinking, decomposition, and utilizing programming tools to address various intricate issues. When learners participate in activities that integrate AI and CT, they not only create physical CT or AI artifacts but also develop understandings of diverse elements associated with AI and CT as they construct meaning during the making process. Although there have been encouraging results regarding the influence of implementing AI and CT activities on student learning, there is still a lack of consensus on how to integrate these two fields effectively and meaningfully in a cohesive manner. This systematic review has provided specific examples to clarify the educational settings, tools utilized, pedagogical designs, learning outcomes, and interactions explored in empirical studies focusing on AI- and CT-integrated learning activities. Collectively, the reviewed studies demonstrated that regardless of the learning context, the integration of CT and AI should not be treated as an additional component of AI learning. Instead, it is significant to approach research and practice from a conceptual standpoint to fully comprehend and optimize how CT can effectively enhance AI learning.

Implications

Our research examined the impact of AI- and CT-integrated instruction on student development by considering four key elements: educational context, instructional design, learning outcomes, and the interaction between AI and CT. Based on the analysis of each of these aspects, our review indicates several implications of AI and CT integration. Notably, it is evident that activities integrating AI and CT can be developed to cater to learners with diverse levels of knowledge and skills, and the integration is especially relevant within interdisciplinary and disciplinary contexts. This means that the integration of AI and CT can be carried out at different educational levels, across or within a variety of disciplines. Thus, different stakeholder groups should drive the implementation of AI and CT in K-12 education in terms of curriculum development, resource support, and so on. Moreover, project-based learning aligns well with activities that integrate AI and CT. The integration of AI and CT has been shown to provide valuable insights into enhancing students’ AI literacy (Ng et al., 2021) as well as CT concepts and practices (Chen, Lai, & Lin, 2020; Hsu et al., 2023; Jiang et al., 2023; Lai et al., 2021).

Furthermore, AI has the potential to fulfill dual roles in the advancement of CT competencies—by utilizing AI platforms to create CT artifacts and by employing AI making in the practical implementation of CT. The integrated activities provide students with opportunities to enhance and utilize their AI knowledge and skills in conjunction with their CT expertise. Specifically, we observed the implementation of linking AI with CT frameworks (Jiang et al., 2023). These features have important implications for organizing AI learning activities in computationally enhanced ways. Embracing them can lead to transformative changes in the structure of AI curricula and instructional approaches. Accordingly, we emphasize the potential for reorienting the pedagogical perspective of AI instruction within CT environments. Further, we underline the reciprocal support and opportunities provided by AI and CT activities and highlight the potential for designing CT instruction centered on AI acquisition for the concurrent development of AI and CT knowledge and skills.

Future Directions

Nevertheless, our review highlights the existing gaps in the research on AI and CT integration in multiple areas. The first is the integration of AI and CT learning activities in informal education settings at the secondary education level. This gap highlights the crucial need for teachers in relevant education settings to undergo professional development, ensuring that they possess the essential competency to integrate AI and CT mechanisms appropriately. Second, there is a need for experiments that encourage students to find solutions to authentic problems. Researchers could take advantage of problem solving in learning environments and, more importantly, be aware of the role of real-world problem solving in supporting AI and CT integrations. Third, researchers should design empirical studies that investigate and illuminate the various ways in which AI and CT learning can be mutually developed. This includes exploring the potential for integrating CT principles into other domains of AI rather than ML. It is crucial to examine the lessons that can be derived from AI- and CT-integrated instruction in different contexts, specifically how CT can facilitate AI learning and vice versa.

In conclusion, this study paves the way for future research on the seamless and efficient integration of AI and CT in education. We support increased research on child-friendly AI and programming instructional designs and tools for primary education, aiming to foster the early integration of AI and CT. Given the nature of AI content, its integration with CT appears to be more inherent and prevalent in higher education compared to K–12 education. Unlike K–12 AI education, which often follows a more conventional instructional approach, undergraduate AI courses are frequently observed to incorporate the components of CT. Despite the prevalent integration of AI and CT in higher education, limited research has offered a detailed analysis of the relationship between CT and AI learning, as reported by Gadanidis (2017) and Zeng (2013). The research conducted by these scholars aligns with the interplay identified in the current review. This indicates that future research has the potential to provide further insights into how the interactions between AI and CT may manifest in different or similar ways in both educational contexts.

Our study enriches the understanding of technology-enhanced learning environments regarding the involvement of AI and CT in instructional activities for student learning outcomes. It also provides a clearer and more explicit identification of the integration between AI and CT during student learning processes. Notably, there are some limitations in this study that require attention from researchers. For instance, the reviewed studies may not provide a complete representation of the current state of empirical studies on AI and CT integration. All of the reviewed studies came from the four most reliable educational technology research databases, ensuring high quality. However, it is possible that important studies available in other databases were overlooked. Additionally, there was a time lag in this study. Although we did not set specific time constraints during the search, there was an endpoint to the search phase. As a result, newly published studies after that period were not included in the review study, limiting the available data resources. Future researchers can build upon these limitations and explore relevant issues in depth.

Footnotes

Authors Contributions

Xiaojing Weng: Conceptualization, Methodology, Formal analysis, Writing - Review & Editing, Validation. Project administration.

Huiyan Ye: Conceptualization, Methodology, Formal analysis, Writing - Review & Editing, Validation.

Yun Dai: Validation, Writing - Review & Editing.

Oi-lam Ng: Conceptualization, Methodology, Validation, Project administration, Supervision.

Brief description: A growing body of research is focusing on integrating artificial intelligence (AI) and computational thinking (CT) to enhance student learning outcomes. Many researchers have designed instructional activities to achieve various learning goals within this field. Despite the prevalence of studies focusing on instructional design and student learning outcomes, how instructional efforts result in the integration of AI and CT in students’ learning processes remains unclear. To address this research gap, we conducted a systematic literature review of empirical studies that have integrated AI and CT for student development. We collected 18 papers from four prominent research databases in the fields of education and AI technology: Web of Science, Scopus, IEEE, and ACM. We coded the collected studies, highlighting the students’ learning processes in terms of research methodology and context, learning tools and instructional design, student learning outcomes, and the interaction between AI and CT. The integration of AI and CT occurs in two ways: the integration of disciplinary knowledge and leveraging AI tools to learn CT. Specifically, we discovered that AI- and CT-related tools, projects, and problems facilitated student-centered instructional designs, resulting in productive AI and CT learning outcomes.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Xiaojing Weng

Huiyan Ye

Yun Dai

Oi-lam Ng

Data Availability Statement

The data is available in the databases stated in manuscripts, including Web of Science, Scopus, IEEE, and ACM.

Appendix 1

Table A1 Table A1.

Coding definitions.

Theme of the code	First-level code	Second-level code	Definition
Research context	Education level of learners Education settings Disciplinary context	Kindergarten Elementary education Lower secondary education Higher secondary education Higher education Graduate school Formal Non-formal In-formal Disciplinary AI course Disciplinary programing course Interdisciplinary	Preschool or early childhood Primary school, grade 1 to grade 6 Middle school, grade 7 to grade 9 High school, grade 10 to grade 12 Undergraduate, post-secondary degree Postgraduate stage Structured learning, follows a curriculum, and leads to certifications Organized learning, doesn’t have a fixed curriculum or formal recognition Unstructured learning, unplanned, and happens naturally in everyday life An educational program or course explores AI concepts, techniques, and applications relevant to AI An educational program or course that aims to teach students programming languages, techniques, and practices that are relevant to programming Integrating related concepts and skills from multiple disciplines to enhance the acquisition of knowledge and skills (English, 2016)
Research interventions	Adopted CT platform Applied AI algorithm Constructed AI artifacts Instructional design Learning theories	Unplugged activity Text-based programming Block-based programming Tangible programming Conventional machine learning Deep learning/neural networks Natural language processing K-means Bayesian networks Recommendation system Classifier Recognition system AI painting Self-driving cars AIoT AI agents Project-based learning Problem-based learning Inquiry-based learning Game-based learning Gamification Constructionism Constructivism Combination Other theories	Offline, hands-on learning activity without the use of digital devices Use text-based coding languages and syntax to program and create tasks for computers to perform Use visual blocks or graphical representation of the code to program and create tasks for computers to perform Use physical objects or manipulatives to represent different programming concepts and create programs A field of AI where algorithms learn from data to make predictions or decisions A subset of machine learning uses neural networks to model complex patterns in data Uses AI to understand and process human language for communication and tasks A clustering algorithm that partitions data into K clusters based on similarity Graphical models expressing probabilistic relationships between variables An information filtering technology that suggests items based on users’ past preferences and behavior patterns An algorithm that categorizes data into predefined classes or labels based on features; for examples, see section “adopted AI algorithms and constructed AI artifacts” A recognition system identifies and interprets patterns or objects in data to make informed decisions; for examples, see section “adopted AI algorithms and constructed AI artifacts” Use AI to generate or enhance artistic creations, such as paintings and drawings Use AI to enable autonomous driving capabilities AIoT stands for artificial intelligence of things, where AI is integrated with the internet of things devices Software entities that perceive their environment and act autonomously to achieve specific goals An instructional method that engages students in practical projects to deepen understanding and develop skills An instructional method that encourages students to learn by actively exploring and solving real-world challenges and problems An educational approach that encourages students to ask questions, investigate, and discover knowledge through exploration An educational method that utilizes games to enhance engagement, motivation, and learning outcomes in students Applying game elements, such as rewards and challenges, to non-game contexts to boost engagement and motivation Learning through hands-on creation, building, and active engagement with materials (Papert & Harel, 1991) Learners construct knowledge through active engagement and personal meaning-making experiences (Barrouillet, 2015) Use both constructionism and constructivism As specified in the studies
Learning outcomes	AI-related learning outcomes CT-related learning outcomes	Know and understand AI Use and apply AI Evaluate and create AI AI ethics CT concepts CT practices CT perspectives	For details, see section “AI learning outcomes” For details, see section “CT learning outcomes”
Integration of AI and CT	For details, see section “integration of AI and CT learning outcomes”

Author Biographies

Xiaojing Weng is a Research Assistant Professor in the Department of Curriculum and Instruction, The Education University of Hong Kong. She has obtained the PhD degree in education from The Chinese University of Hong Kong, the Master of Science in educational technology from The University of Hong Kong. Her research focuses on technology-enhanced teaching and learning, instructional design, and 21st-century skills.

Huiyan Ye is a PhD candidate in the Department of Curriculum and Instruction at The Chinese University of Hong Kong, where she served as a research assistant before her PhD study. Her research interests focus on technologies for mathematics teaching and learning. Particularly, she is interested in the integration of computational thinking and mathematics education.

Yun Dai is an Assistant Professor of Curriculum and Instruction at the Chinese University of Hong Kong (CUHK). Before joining the CUHK in 2019, she worked as Postdoc Fellow in engineering education at the University of Southern California. Her research interests broadly revolve around artificial intelligence (AI) as a subject matter in the K-12 education and as a learning tool in higher education. Her current research focuses on: (1) teaching and learning artificial intelligence in the K-12 context, (2) personal epistemology of STEM teachers, and (3) generative AI in higher education.

Oi-Lam Ng is an Associate Professor in the Department of Curriculum and Instruction at The Chinese University of Hong Kong. Her research interests include technology innovations in mathematics education, language and mathematics discourse, teacher noticing, and STEM education. Particularly, she is interested in advancing a Papert-inspired conception of “learning as Making,” and the new opportunities it entails for engaging learners in constructionist practices with emergent technologies (3D printing, coding etc).

References

Barrouillet

(2015). Theories of cognitive development: From Piaget to today. Developmental Review, 38, 1–12. https://doi.org/10.1016/j.dr.2015.07.004

Belur

Tompson

Thornton

Simon

(2021). Interrater reliability in systematic review methodology: Exploring variation in coder decision-making. Sociological Methods & Research, 50(2), 837–865. https://doi.org/10.21428/cb6ab371.b8fc485a

Brennan

Resnick

(2012). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American educational research association (1, p. 25).

Cheah

C. W.

(2021). Developing a gamified AI-enabled online learning application to improve students’ perception of university physics. Computers and Education: Artificial Intelligence, 2, 100032. https://doi.org/10.1016/j.caeai.2021.100032

Chen

Lin

(2020a). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510

* Chen

S. Y.

Lai

Y. H.

Lin

Y. S.

(2020b). Research on head-mounted virtual reality and computational thinking experiments to improve the learning effect of AIoT maker course: Case of earthquake relief scenes. Frontiers in Psychology, 11, 1164. https://doi.org/10.3389/fpsyg.2020.01164

Choi

(2023). GeoAI: Integration of artificial intelligence, machine learning, and deep learning with GIS. Applied Sciences, 13(6), 3895. https://doi.org/10.3390/app13063895

Conati

Barral

Putnam

Rieger

(2021). Toward personalized XAI: A case study in intelligent tutoring systems. Artificial Intelligence, 298, 103503. https://doi.org/10.1016/j.artint.2021.103503

English

L. D.

(2016). STEM education K-12: Perspectives on integration. International Journal of STEM Education, 3(3), 1–8. https://doi.org/10.1186/s40594-016-0036-1

10.

Eshach

(2007). Bridging in-school and out-of-school learning: Formal, non-formal, and informal education. Journal of Science Education and Technology, 16, 171–190. https://doi.org/10.1007/s10956-006-9027-1

11.

* Estevez

Garate

Graña

(2019). Gentle introduction to artificial intelligence for high-school students using scratch. IEEE Access, 7, 179027–179036. https://doi.org/10.1109/access.2019.2956136

12.

Fang

D. T. K.

Leung

J. K. L.

Chu

S. K. W.

(2023). A systematic review of artificial intelligence technologies used for story writing. Education and Information Technologies, 28, 1–37. https://doi.org/10.1007/s10639-023-11741-5

13.

Gadanidis

(2017). Artificial intelligence, computational thinking, and mathematics education. The International Journal of Information and Learning Technology, 34(2), 133–139. https://doi.org/10.1108/ijilt-09-2016-0048

14.

Gansser

O. A.

Reich

C. S.

(2021). A new acceptance model for artificial intelligence with extensions to UTAUT2: An empirical study in three segments of application. Technology in Society, 65, 101535. https://doi.org/10.1016/j.techsoc.2021.101535

15.

* García

J. D. R.

Moreno-León

Román-González

Robles

(2020). LearningML: A tool to foster computational thinking skills through practical artificial intelligence projects. Revista de Educación a Distancia (RED), 20(63), 1–37. https://doi.org/10.6018/red.410121

16.

Glaser

Strauss

(2017). Discovery of grounded theory: Strategies for qualitative research. Routledge.

17.

Grover

Pea

(2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189x12463051

18.

Harding

(2006). Grounded theory. In Jupp

(Ed.), The Sage dictionary of social research methods (pp. 131–132). Sage.

19.

* Hsu

T. C.

Abelson

Lao

Tseng

Y. H.

Lin

Y. T.

(2021). Behavioral-pattern exploration and development of an instructional tool for young children to learn AI. Computers and Education: Artificial Intelligence, 2, 100012. https://doi.org/10.1016/j.caeai.2021.100012

20.

* Hsu

T. C.

Abelson

Van Brummelen

(2022). The effects on secondary school students of applying experiential learning to the Conversational AI Learning Curriculum. International Review of Research in Open and Distributed Learning, 23(1), 82–103. https://doi.org/10.19173/irrodl.v22i4.5474

21.

* Hsu

T. C.

Chang

Lin

Y. W.

(2023). Effects of voice assistant creation using different learning approaches on performance of computational thinking. Computers & Education, 192, 104657. https://doi.org/10.1016/j.compedu.2022.104657

22.

* Hsu

T. C.

Chen

M. S.

(2022). The engagement of students when learning to use a personal audio classifier to control robot cars in a computational thinking board game. Research and Practice in Technology Enhanced Learning, 17(1), 1–17. https://doi.org/10.1186/s41039-022-00202-1

23.

* Huang

Qiao

(2022). Enhancing computational thinking skills through artificial intelligence education at a STEAM high school. Science & Education, 33, 383–403. https://doi.org/10.1007/s11191-022-00392-6

24.

Ingkavara

Panjaburee

Srisawasdi

Sajjapanroj

(2022). The use of a personalized learning approach to implementing self-regulated online learning. Computers and Education: Artificial Intelligence, 3, 100086. https://doi.org/10.1016/j.caeai.2022.100086

25.

* Jiang

Qian

Tang

Yalcinkaya

Rosé

C. P.

Chao

Finzer

(2023). Examining computational thinking processes in modeling unstructured data. Education and Information Technologies, 28(4), 4309–4333. https://doi.org/10.1007/s10639-022-11355-3

26.

Kafai

Y. B.

Proctor

(2022). A revaluation of computational thinking in K–12 education: Moving toward computational literacies. Educational Researcher, 51(2), 146–151. https://doi.org/10.3102/0013189x211057904

27.

Khan

Sarwar

Chen

T. B.

Khan

(2022). Connecting digital literacy in higher education to the 21st century workforce. Knowledge Management & E-Learning, 14(1), 46–61. https://doi.org/10.34105/j.kmel.2022.14.004

28.

* Lai

Y. H.

Chen

S. Y.

Lai

C. F.

Chang

Y. C.

Y. S.

(2021). Study on enhancing AIoT computational thinking skills by plot image-based VR. Interactive Learning Environments, 29(3), 482–495. https://doi.org/10.1080/10494820.2019.1580750

29.

Lee

C. A.

Tzeng

J. W.

Huang

N. F.

Y. S.

(2021). Prediction of student performance in massive open online courses using deep learning system based on learning behaviors. Educational Technology & Society, 24(3), 130–146. https://www-jstor-org-443.web.bisu.edu.cn/stable/27032861

30.

Lefa

(2014). The piaget theory of cognitive development: An educational implications. Educational Psychology, 1(1), 1–8.

31.

* Li

(2019). Experience report: Explorable web apps to teach AI to non-majors. Journal of Computing Sciences in Colleges, 34(4), 128–133. https://dl.acm.org/doi/10.5555/3344013.3344035

32.

Liang

J. C.

Hwang

G. J.

Chen

M. R. A.

Darmawansah

(2021). Roles and research foci of artificial intelligence in language education: An integrated bibliographic analysis and systematic review approach. Interactive Learning Environments, 31(7), 4270–4296. https://doi.org/10.1080/10494820.2021.1958348

33.

* Lin

P. H.

Chen

S. Y.

(2020). Design and evaluation of a deep learning recommendation based augmented reality system for teaching programming and computational thinking. IEEE Access, 8, 45689–45699. https://doi.org/10.1109/access.2020.2977679

34.

* Lin

Y. S.

Chen

S. Y.

Tsai

C. W.

Lai

Y. H.

(2021). Exploring computational thinking skills training through augmented reality and AIoT learning. Frontiers in Psychology, 12, 640115. https://doi.org/10.3389/fpsyg.2021.640115

35.

McCarthy

Minsky

M. L.

Rochester

Shannon

C. E.

(2006). A proposal for the dartmouth summer research project on artificial intelligence, august 31, 1955. AI Magazine, 27(4), 12. https://doi.org/10.1609/aimag.v27i4.1904

36.

McLaren

B. M.

Scheuer

Mikšátko

(2010). Supporting collaborative learning and e-discussions using artificial intelligence techniques. International Journal of Artificial Intelligence in Education, 20(1), 1–46. https://doi.org/10.3233/JAI-2010-0001

37.

Moher

Liberati

Tetzlaff

Altman

D. G.

Prisma Group . (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007

38.

Morrison

G. R.

Ross

S. J.

Morrison

J. R.

Kalman

H. K.

(2019). Designing effective instruction. John Wiley & Sons. https://doi.org/10.4135/9781071801017.n4

39.

Munn

Peters

M. D.

Stern

Tufanaru

McArthur

Aromataris

(2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18, 1–7. https://doi.org/10.1186/s12874-018-0611-x

40.

D. T. K.

Leung

J. K. L.

Chu

S. K. W.

Qiao

M. S.

(2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041

41.

O. L.

Cui

(2021). Examining primary students’ mathematical problem-solving in a programming context: Towards computationally enhanced mathematics education. ZDM–Mathematics Education, 53, 847–860. https://doi.org/10.1007/s11858-020-01200-7

42.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Moher

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, 1–9. https://doi.org/10.1186/s13643-021-01626-4

43.

Papert

Harel

(1991). Situating constructionism. Constructionism, 36(2), 1–11.

44.

Ríos Félix

J. M.

Zatarain Cabada

Barrón Estrada

M. L.

Favela Vara

(2020). An intelligent learning environment for computational thinking. Computación Y Sistemas, 24(3), 1199–1210. https://doi.org/10.13053/cys-24-3-3480

45.

Roll

Wylie

(2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26, 582–599. https://doi.org/10.1007/s40593-016-0110-3

46.

Shamir

Levin

(2021). Neural network construction practices in elementary school. KI-Künstliche Intelligenz, 35(2), 181–189. https://doi.org/10.1007/s13218-021-00729-3

47.

Shamir

Levin

(2022). Teaching machine learning in elementary school. International Journal of Child-Computer Interaction, 31, 100415. https://doi.org/10.1016/j.ijcci.2021.100415

48.

Silapachote

Srisuphab

(2017). Engineering courses on computational thinking through solving problems in artificial intelligence. International Journal of Engineering Pedagogy, 7(3), 34. https://doi.org/10.3991/ijep.v7i3.6951

49.

Strauss

Corbin

(1990). Basics of qualitative research. Sage publications.

50.

Tang

K. Y.

Chang

C. Y.

Hwang

G. J.

(2021). Trends in artificial intelligence-supported e-learning: A systematic review and co-citation network analysis (1998–2019). Interactive Learning Environments, 31(4), 2134–2152. https://doi.org/10.1080/10494820.2021.1875001

51.

Van Brummelen

Shen

J. H.

Patton

E. W.

(2019). The popstar, the poet, and the grinch: Relating artificial intelligence to the computational thinking framework with block-based coding. Proceedings of International Conference on Computational Thinking Education, 3, 160–161.

52.

Vittorini

Menini

Tonelli

(2021). An AI-based system for formative and summative assessment in data science courses. International Journal of Artificial Intelligence in Education, 31(2), 159–185. https://doi.org/10.1007/s40593-020-00230-2

53.

Voogt

Fisser

Good

Mishra

Yadav

(2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20, 715–728. https://doi.org/10.1007/s10639-015-9412-6

54.

Wang

C. J.

Zhong

H. X.

Chiu

P. S.

Chang

J. H.

P. H.

(2022). Research on the impacts of cognitive style and computational thinking on college students in a visual artificial intelligence course. Frontiers in Psychology, 13, 864416. https://doi.org/10.3389/fpsyg.2022.864416

55.

Weintrop

Beheshti

Horn

Orton

Jona

Trouille

Wilensky

(2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25, 127–147. https://doi.org/10.1007/s10956-015-9581-5

56.

Weng

Cui

O. L.

Jong

M. S.

Chiu

T. K.

(2022). Characterizing students’ 4C skills development during problem-based digital making. Journal of Science Education and Technology, 31(3), 372–385. https://doi.org/10.1007/s10956-022-09961-4

57.

Weng

O. L.

Chiu

T. K.

(2023). Competency development of pre-service teachers during video-based learning: A systematic literature review and meta-analysis. Computers & Education. https://doi.org/10.1016/j.compedu.2023.104790

58.

Wing

J. M.

(2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

59.

Wing

J. M.

(2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical & Engineering Sciences, 366(1881), 3717–3725. https://doi.org/10.1109/ipdps.2008.4536091

60.

Liang

O. L.

Chai

C. S.

(2023). Integration of computational thinking in K-12 mathematics education: A systematic review on CT-based mathematics instruction and student learning. International Journal of STEM Education, 10(1), 1–26. https://doi.org/10.1186/s40594-023-00396-w

61.

Zawacki-Richter

Marín

V. I.

Bond

Gouverneur

(2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0

62.

Zeng

(2013). From computational thinking to AI thinking [A letter from the editor]. IEEE Intelligent Systems, 28(06), 2–4. https://doi.org/10.1109/mis.2013.141

63.

Zerega

Milrad

(2023). Computational thinking & artificial intelligence in K-12 education: Two distinct but still complementary worlds. In International conference in methodologies and intelligent systems for techhnology enhanced learning (pp. 207–218): Springer Nature Switzerland.