Abstract
The programming curriculum is crucial in today’s digital age, but a common issue is its focus on technical training while neglecting essential skills enhancement. Teachers lack structured and practical guidance on instructional methods, and the wide range of students’ foundational abilities makes it difficult to cater to personalized learning needs. The advent of Generative Artificial Intelligence (GAI) technology has brought significant changes to education, with its personalized dialogue features offering effective technical support to address these challenges in programming education. This study develops a GAI-supported programming learning approach, applied in a programming course at a vocational school in China, to investigate the effects of various learning approaches on students’ learning outcomes, motivation, self-efficacy, and 5C competencies among different learning styles. The study involved 46 learners in the experimental group using GAI-assisted learning and 46 in the control group using traditional methods. Results showed that the experimental group significantly outperformed the control group in learning outcomes, motivation, self-efficacy, and 5C competencies. Moreover, students with a diverging learning style in the experimental group showed notably higher motivation than others. Additionally, creativity levels among students with diverging and assimilating learning styles in the experimental group were substantially greater than those in the control group.
Keywords
Introduction
Computer science (CS), and particularly programming, has traditionally been viewed as a domain exclusively for IT professionals. In recent years, however, this perception has evolved (Nouri et al., 2020). Programming education is now recognized as a vital part of K-12 education and is receiving increased attention globally. Despite its growing importance, current programming instruction often emphasizes technology skills alone, neglecting the development of essential competencies. Teachers mainly focus on teaching fundamental knowledge and the application of technology, with lessons largely centred on rote learning and replication of basic concepts. This results in students being passive learners, with few opportunities for hands-on practice or critical thinking. Consequently, current programming education does not effectively foster the development of core competencies in students. Although numerous researchers have examined this problem, most studies concentrate on the macro-level construction of the teaching environment. Furthermore, the practical guidance offered in existing literature is often insufficient, making implementation challenging for educators. From the learners’ perspective, programming education today shows significant disparities in individual learning conditions and lacks personalization. Beginners often perceive programming concepts as abstract and difficult, leading to a decline in learning interest (Robins, 2019). Therefore, it is imperative to design a personalized learning approach that not only promotes the development of core competencies and increases motivation but is also easy for educators to implement, addressing the limitations of current programming education.
On November 30, 2022, OpenAI, an American artificial intelligence research lab, introduced ChatGPT, a generative AI chatbot powered by a large language model (LLM) based on GPT-3.5. This release represented a major milestone in the development of generative artificial intelligence (GAI; Jeon & Lee, 2023). GAI, as an AI technology, has distinct generative capabilities that allow it to autonomously create various forms of content, such as text, images, and videos, tailored to users’ personalized requests (T. Wu et al., 2023). These generative capabilities provide GAI with significant potential in the educational sector, positioning it as a transformative tool for teaching, learning, and communication. Educators can leverage GAI to deliver more tailored and adaptive learning experiences during instructional activities, while students can engage with GAI for real-time, personalized, and deeper knowledge acquisition. However, despite the many opportunities GAI brings to education, it also presents several challenges, such as issues related to ethics and academic integrity (Ahmad et al., 2023). As this technology continues to evolve, there is both an opportunity and a challenge in seamlessly integrating GAI into educational practices.
Generative AI (GAI) possesses the capacity for autonomous learning and content generation, mimicking human cognitive processes to produce coherent and natural language independently. Its applications extend across multiple domains, from text generation to coding, demonstrating significant potential for enhancing programming education (Phung et al., 2023). GAI’s flexibility allows it to provide technical support for individualized instruction tailored to specific learner needs (G. J. Hwang & Chen, 2023), addressing the current gap in personalized programming education. Furthermore, by interacting with GAI, students can actively participate in knowledge construction, which not only increases their motivation but also effectively nurtures critical thinking, creativity, and other essential competencies (Shanto et al., 2024). In essence, GAI offers innovative solutions to the existing challenges in programming education. This study, therefore, seeks to utilize GAI technology to develop a personalized learning approach that significantly enhances students’ core competencies, providing practical guidance for educational implementation and helping the education sector adapt to the evolving demands of the GAI-driven teaching landscape.
Literature Review
The literature review section begins by outlining the primary challenges currently faced in programming education, setting a problem-focussed foundation for the subsequent research. It then presents Generative Artificial Intelligence (GAI) technology as a potential solution, summarizing its current applications within the educational field and highlighting its potential for driving educational innovation. Considering the substantial influence of learning styles on educational outcomes, the review also includes an overview of learning style models. This serves as a basis for a detailed investigation into the effects of GAI-assisted programming learning on students with varying learning styles in the subsequent research.
The Current State of Programming Learning
Programming learning is the process through which learners develop foundational knowledge of programming languages, computational thinking, algorithm design, and software development skills through structured educational activities and hands-on experiences. It has gained global recognition, with several countries incorporating programming as a compulsory part of the K-12 curriculum to foster students’ core competencies (Perera et al., 2021; Popat & Starkey, 2019).
To investigate the factors influencing programming learning outcomes, several studies have examined Python programming instruction in higher education (Wang et al., 2017), focussing on the fundamental principles of Python language teaching. By assessing students’ learning outcomes, these studies have highlighted that performance differences are influenced by variables such as gender, class standing, and attendance. Additionally, in an effort to further improve the effectiveness of programming learning, scholars have introduced innovative teaching methods and strategies. For instance, one study compared the impact of serious game-based instruction with traditional teaching methods in programming education (Kroustalli & Xinogalos, 2021). The results indicated that serious game-based instruction was more effective than conventional lecture-based programming education, significantly enhancing the efficiency of learning fundamental concepts and boosting student engagement. Furthermore, other researchers have proposed strategies to optimize programming learning from various angles, such as cognitive development (Whalley & Kasto, 2014) and debugging practices (Fitzgerald et al., 2008).
However, there are still several challenges in the implementation of programming learning. Unlike other disciplines, programming learning exhibits significant disparities in students’ foundational levels, particularly at the high school stage. While some students may already possess advanced programming skills, others may be encountering programming concepts for the first time. This diversity necessitates that programming education be adaptable to student groups with varying learning styles and background knowledge (Rovshenov & Sarsar, 2023). Unfortunately, traditional programming teaching methods often fall short in providing sufficient personalized instruction to cater to the individual needs of each student. Additionally, programming concepts are inherently abstract, making it challenging for students to develop an interest in programming courses and, consequently, difficult to sustain their motivation. With information technology courses now part of entrance examinations, programming education has predominantly adopted lecture-based methods, focussing on the mastery of basic knowledge. This approach leaves little room for active thinking, which is not conducive to the development of students’ core competencies. Therefore, future research should aim to innovate programming learning methods, balancing the emphasis on learning outcomes with skill development, enhancing students’ motivation and initiative, and achieving personalized programming education.
The rapid advancement of information technology has created opportunities for transformative changes in this field. The rise of generative artificial intelligence has made personalized learning feasible (Baidoo-Anu & Owusu Ansah, 2023; Farrokhnia et al., 2024; Rahman & Watanobe, 2023) and is anticipated to act as an instructional support tool for educators (Grassini, 2023; Polverini & Gregorcic, 2024). Consequently, this development introduces a novel approach to traditional programming education.
The Application of Generative Artificial Intelligence in Education
Generative artificial intelligence (GAI) has significantly impacted the education sector. Several scholars (Farrokhnia et al., 2024; Murugesan & Cherukuri, 2023; Parse, 2023) have examined the opportunities and challenges posed by GAI from a theoretical standpoint. They highlight that GAI can enhance information accessibility, support personalized and complex learning experiences, and alleviate teaching workloads. However, they also note drawbacks such as a lack of deep comprehension, challenges in evaluating the quality of generated responses, and potential risks of bias and discrimination.
Some researchers (Yilmaz & Karaoglan Yilmaz, 2022a, 2022b) propose integrating generative artificial intelligence (GAI) into education to enhance personalized learning experiences. To further inform practice, several scholars (Kasneci et al., 2023; Megahed et al., 2024; Murugesan & Cherukuri, 2023) have outlined the roles of GAI in teaching practices from both teacher and student perspectives. From the teachers’ standpoint, GAI can support instructional design, efficiently retrieve educational resources, organize key concepts systematically, and facilitate personalized instruction for learners. From the students’ perspective, GAI can act as a learning companion, helping to explain knowledge points, answer questions, summarize information, conduct interactive assessments, and more. Overall, the current theoretical framework for applying GAI to enhance educational practices is extensive. Conducting a systematic review of GAI’s functions and value will provide crucial support for the development of future models.
The application of generative artificial intelligence (GAI) in education spans multiple disciplines, including pharmaceutical education (Cain et al., 2023), chemical education (Humphry & Fuller, 2023), science education (Cooper, 2023), medical education (Lee, 2024), writing training (Yan, 2023), engineering education (Sánchez-Ruiz et al., 2023), and art education (Saurini, 2023). In the domain of programming education, GAI has also demonstrated significant potential. GAI can boost the productivity and creativity of software developers, providing programmers with continuous learning opportunities through support in code management, generation, and programming tutoring. Moreover, some researchers (Vukojičić & Krstić, 2023) have investigated the effectiveness of GAI in programming education, noting that GAI-supported learning can significantly enhance students’ coding proficiency, improve the quality of their explanations, and deepen their understanding of standard solutions. Additionally, in programming learning, GAI functions as a sustainable teaching tool, offering teachers valuable assessment and grading guidance while helping students to understand and optimize their solutions (Wieser et al., 2023). However, the systematic implementation and evaluation of this technology in programming education are still in their infancy, and further exploration is needed to understand its impact on learning methods and teaching outcomes when integrated with generative artificial intelligence.
Learning Style
The term “learning styles” refers to the idea that individuals vary in the instructional or study methods that are most effective for them; optimal instruction involves diagnosing individuals’ learning styles and adapting instruction accordingly (Pashler et al., 2008). In the field of education, learning styles have been a topic of significant interest, with numerous studies examining the influence of different learning styles on learning outcomes (Moser & Zumbach, 2018; Soflano et al., 2015; Van Waes et al., 2014). This study uses learning styles as an independent variable to explore how learning methods impact students with different learning styles, aiming to validate the generalizability of this model and identify which learning styles may require additional support or adjustments to ensure that all students benefit from the instruction. While many learning style models have been proposed, only a few have been widely implemented in computer-supported learning environments and have had their reliability and validity tested (Soflano et al., 2015). These include the Honey & Mumford model (Honey et al., 1986), Reid model (Reid, 1987), Felder-Silverman model (Felder & Silverman, 1988), and Kolb model (D. A. Kolb, 1984). Among them, the Kolb model, which is based on experiential learning theory, is easier to measure compared to other mainstream models and is suitable for assessing larger groups, making it the most widely used learning style model currently.
This model (D. A. Kolb & Kolb, 2013) outlines two interrelated methods for acquiring experience: Concrete Experience (CE) and Reflective Observation (RO), as well as two dialectically related processes for transforming experience: Abstract Conceptualization (AC) and Active Experimentation (AE). By integrating these elements, the model identifies four distinct learning styles: Diverging, Assimilating, Converging, and Accommodating (Cavanagh et al., 1995). In this study, the Kolb Learning Style Inventory will be employed as a research tool to examine how learners with different learning styles perform under various learning approach supports.
Research Questions
Learning motivation refers to the dynamic tendency that initiates and sustains students’ learning behaviours, propelling them towards specific academic goals. Self-efficacy is an individual’s subjective speculation and judgement about their ability to successfully accomplish a certain behaviour and achieve expected goals, based on factors such as their own experiences. The integration of new technologies into learning significantly influences students’ learning achievements, motivation, and self-efficacy (Iku-Silan et al., 2023; H. Li et al., 2021; Lin et al., 2021; T.-T. Wu et al., 2024). Additionally, innovative learning approaches have proven effective in enhancing students’ creative thinking, communication skills, and problem-solving abilities. Research has found that enabling students to learn in real or simulated problem-based scenarios can significantly enhance their problem-solving abilities and communication skills (Aslan, 2021). Meanwhile, M. Li et al. (2023) adopted an Augmented Reality (AR)-based motivational learning approach, which effectively improved students’ creative thinking. Additionally, Jauhiainen et al. introduced ChatGPT as an auxiliary tool in a class, successfully stimulating students’ creative thinking and aiding them in better problem-solving (Jauhiainen & Guerra, 2023). Building on previous studies, Lai and Hwang et al. (2014a) summarized the core competencies students need into five dimensions: collaboration, communication, complex problem-solving, metacognitive awareness, and creativity, collectively known as the 5C competence. In this study, learning outcomes (learning achievements, learning motivation, self-efficacy) and the 5C competence will be utilized as dependent variables to thoroughly investigate the effects of generative learning supported by GAI on students.
This study primarily aims to explore the following inquiries:
Does GAI-assisted programming learning exhibit a significant difference in the learning achievements compared to traditional programming learning?
Does GAI-assisted programming learning exhibit a significant difference in the learning motivation compared to traditional programming learning?
Does GAI-assisted programming learning exhibit a significant difference in the self-efficacy compared to traditional programming learning?
Does GAI-assisted programming learning exhibit a significant difference in the 5C competence compared to traditional programming learning?
Do students’ learning styles (such as diverging) and learning approaches (GAI-assisted programming learning or traditional programming learning) have an interaction effect on learning outcomes and 5C competence?
GAI-assisted Programming Learning Approach
System Structure
Figure 1 illustrates the structure of the GAI-assisted learning system. In the teacher terminal, GAI provides pre-class design functionalities, supporting teachers in developing pre-class materials and instructional plans. After class, GAI assists teachers in intelligently designing questions for after-class exercises based on students’ actual performance. On the student terminal, GAI functions as a dedicated in-class assistant, offering solutions, stimulating ideas, answering queries, and debugging code. After class, students can engage with GAI for personalized review and reinforcement through interactive assessments.
Structure of the GAI-assisted learning system.
GAI-PL Learning Approach
Fiorella and Mayer’s SOI framework divides the generative learning process into three stages. The first stage is selection, which involves directing attention to relevant instructional information (McCrudden & Rapp, 2017). At this stage, the effectiveness of the information is crucial; if students focus too much on interesting but non-essential details, it may disrupt the coherence and fluency of the educational content (Park et al., 2015; Sanchez & Wiley, 2006). The second stage is organization, which focuses on clarifying the relationships between fragments of information within instructional material. In this phase, learners encode and store information based on the connections between new content and their prior knowledge, facilitating long-term memory retention (Dunlosky et al., 2013). The third stage is integration, which builds upon the organization stage by encouraging learners to link instructional information with their existing knowledge framework. This model highlights the learner’s active role and the significance of cognitive processing. GAI can enhance learners’ understanding and organization of information by providing diverse learning resources and interactive methods. The integration of these two elements can lead to more effective teaching outcomes.
Based on the concept of generative learning and SOI framework, we proposed a learning approach for GAI-assisted programming learning (GAI-PL; see Figure 2).
The GAI-PL learning approach.
From a macro perspective, the learning process is structured into three continuous phases: before-class preview, during-class exploration, and after-class consolidation, aligning with the three stages of generative learning. This structure guides learners through the processes of selection, organization, and integration to facilitate knowledge construction. As the core technological support of this model, GAI is positioned at the centre, functioning as both an assistant and a learning partner. It offers robust technical support for both teachers’ instructional activities and students’ learning processes, aiming to address the challenges faced in traditional generative learning.
The GAI-PL approach is described as follows: The pre-class preparation phase aligns with the initial stage of generative learning—selection. In this phase, knowledge selection involves three steps: needs assessment, setting clear objectives, and fostering interest. Initially, teachers utilize GAI to thoroughly analyse the learning needs of students. By integrating learning data and conducting personalized student interviews, GAI accurately identifies and summarizes specific information about students’ needs. Following this, GAI assists teachers in defining clear instructional goals for the lesson, ensuring that students engage in focussed learning. This step sets a clear direction for knowledge selection. Finally, GAI helps create a realistic instructional scenario that guides the lesson design, stimulating students’ curiosity and eagerness to learn, thereby encouraging them to actively engage in selecting knowledge. The in-class exploration phase corresponds to the second step of generative learning—organization. In this stage, students progress through three steps: acquiring new knowledge, reviewing experiences, and forming connections. This enables them to encode and store new knowledge by linking it to existing knowledge. Initially, during the new knowledge acquisition stage, GAI maximizes its personalized teaching capabilities by establishing a one-on-one interactive model with students. It not only provides real-time responses and accurate Q&A support but also offers simplified explanations for difficult concepts and recommends additional learning resources, helping students overcome obstacles and achieve effective understanding and preliminary internalization of new content. The post-class consolidation phase corresponds to the third stage of generative learning—integration. At this point, students engage in knowledge transfer and reflective assessment to deeply integrate knowledge. During knowledge transfer, GAI supports teachers in designing personalized assignments based on student learning data from class. Students interact with GAI through tailored assessments, allowing it to intelligently generate varied exercises that address individual knowledge gaps, thus promoting flexible knowledge transfer and personalized reinforcement. Assessment and reflection are critical for integrating knowledge. GAI acts as an instructional assistant by posing thought-provoking questions that guide students in reflecting deeply on their learning journey. It also analyses learning data and homework completion comprehensively to produce teaching evaluation reports. Additionally, GAI facilitates students’ independent and effective learning reflection by providing self-assessment tools, encouraging them to examine their learning outcomes scientifically and promoting deeper knowledge integration and skill enhancement. The specific methods of integrating GAI into programming instruction and their corresponding instructional processes are illustrated in Figure 3.
Instructional process design supported by GAI.
Experimental Design-Method
To address ethical considerations, the research included several key measures. First, an ethical issue application for this study was submitted to the Institute ethics committee and was approved. This approval ensured that the research adhered to ethical guidelines and minimized the risk of harm to participants. Additionally, written informed consent was obtained from all participants and their guardians before starting the experiment. This consent process was crucial in ensuring that participants were fully aware of the study’s purpose and their rights, thereby mitigating any potential harm.
Participants
This study is a quasi-experimental research conducted in the field of education. Two parallel teaching classes were randomly selected from a secondary vocational school in Zhejiang Province, and pre-tests and post-tests were administered to them. The two classes shared the same major, grade level, instructors, and teaching content, and the teaching experiment was carried out in the same computer lab to control for confounding variables. Based on the results of the pre-test, it was found that there were no significant differences between the two classes in terms of academic performance, learning motivation, self-efficacy, and 5C competencies. The first class, consisting of 46 students (37 males and 9 females), was assigned as the GAI-assisted programming learning (GAI-PL) group, while the second class, also with 46 students (33 males and 13 females), was designated as the traditional programming learning (TPL) group. The age range of students in both groups was between 17 and 18 years. In addition, all participants completed a learning styles questionnaire. The results showed that in the TPL group, there were 14 divergers, 11 accommodators, 7 convergers, and 14 assimilators. In contrast, the GAI-PL group had 6 divergers, 9 accommodators, 12 convergers, and 19 assimilators.
Learning Environment
Based on the implementation requirements and the characteristics of the GAI-assisted programming learning approach, the experimental group in this study was provided with both offline and online learning environments. The offline environment took place mainly in a computer lab, where each student had access to an internet-enabled computer. For the online environment, the study utilized the “ERNIE Bot” GAI platform (as illustrated in Figure 4). ERNIE Bot is a generative AI platform with a broad knowledge base across different fields, capable of engaging in natural language conversations with users and offering relevant information and recommendations based on user queries. This platform fulfills the tool requirements for various learning stages in the study, acting as both an instructional assistant and a learning companion. Students can interact with ERNIE Bot to access needed information and receive tailored solution suggestions through real-time dialogue.
The interface and functions of ERNIE Bot.
Furthermore, both the experimental and control groups utilized the Mythware Classroom Management System (Mythware CMS) during classroom sessions. This software provides comprehensive support for teachers and students, enhancing the effectiveness of classroom instruction and learning. It includes various features such as screen broadcasting, projection sharing, assignment submission, and resource distribution, all of which facilitate interactive teaching in both online and offline environments. These capabilities allow for a more engaging and dynamic classroom experience, ensuring efficient communication and collaboration between teachers and students.
Experimental Procedure
Figure 5 presents the experimental procedure. The learning content for both the experimental and control groups was identical, sourced from the third unit of the “Introduction to Python Programming” course. The curriculum covered fundamental Python syntax, including sequential, selection, and loop structures, as well as their comprehensive application. By the end of the course, students were expected to choose appropriate syntax structures for different tasks and write simple Python programmes to solve real-world problems. The experiment spanned 9 weeks, conducted in a computer lab, with both groups having four 40-min class periods per week. During the first week, all students completed a learning styles questionnaire administered by their teachers. Following this, they took a pre-test designed to evaluate their prior programming knowledge, and they filled out pre-questionnaires to measure their learning motivation, self-efficacy, and 5C competencies.
Experimental procedure.
In the second week, the GAI-PL group was introduced to the methods and strategies of using GAI, providing a foundation for their subsequent generative learning with the technology. During the third to fifth weeks, the GAI-PL group applied the GAI-assisted programming learning approach, while the TPL group followed the traditional programming learning approach. Additionally, the teacher and several randomly selected students from the GAI-PL group participated in interviews to share their learning experiences. In the ninth week, both groups completed the post-test and post-questionnaires to assess their learning outcomes and gather data on their motivation, self-efficacy, and 5C competencies.
Measuring Instruments
Both groups, following their respective learning approaches, were required to complete pre- and post-tests to measure their learning achievements. Additionally, they completed post-questionnaires aimed at assessing their learning motivation, self-efficacy, and 5C competencies.
The pre-test consisted of 10 multiple-choice questions, 10 fill-in-the-blank questions, 2 programming fill-in-the-blank questions, and 1 programming task. Its purpose was to evaluate the learners’ prior knowledge before they studied the syntax structures covered in Python’s second unit. Example questions included: “What type of programming language is Python?” and “Which of the following options is incorrect regarding the naming rules for variables?”
The post-test consisted of 20 multiple-choice questions, 1 programming error correction question, and 5 programming fill-in-the-blank questions. An example question was: “Debug the ‘Fibonacci sequence’ programme in Python.” The purpose of this test was to evaluate students’ learning achievements following the experiment. Both the pre- and post-tests were based on past Python exam questions from vocational schools and were compiled by the research assistant and the teacher.
The motivation questionnaire primarily assessed the changes in students’ intrinsic and extrinsic motivation before and after the experiment. The content was selected from the motivation scale developed by G.-J. Hwang et al. (2013). The wording of the questionnaire was modified to align with the context of vocational education, making it suitable for the cognitive abilities of vocational students. Experts were invited to review the questionnaire, and after a small-scale pilot test, it was put into use. An example item from the questionnaire is: “In studying this course, I prefer challenging content because it allows me to learn new things.” The Cronbach’s α for the questionnaire was .70, indicating acceptable internal consistency.
The self-efficacy questionnaire was selected from the self-efficacy scale developed by Pintrich (1991) to evaluate the impact of the new learning approach on students’ confidence in completing tasks. The wording of the questionnaire was modified to align with the context of vocational education, making it suitable for the cognitive abilities of vocational students. Experts were invited to review the questionnaire, and after a small-scale pilot test, it was put into use. It comprised 8 questions, each rated on a five-point Likert scale. An example item from the questionnaire is: “I am confident that I can learn and overcome the most difficult part of the course.” The Cronbach’s α for this questionnaire was .88, indicating high internal consistency.
The 5C competence questionnaire was selected from the 5C competence scale developed by Lai and Hwang (2014b) to comprehensively evaluate the impact of GAI on developing students’ future-oriented core competencies. The wording of the questionnaire was modified to align with the context of vocational education, making it suitable for the cognitive abilities of vocational students. Experts were invited to review the questionnaire, and after a small-scale pilot test, it was put into use. It included five questions for each of the following areas: collaboration, communication, critical thinking, meta-cognitive awareness, and creativity tendency. The Cronbach’s α for the questionnaire was .87, indicating high internal reliability. According to exploratory factor analysis, the KMO value of the questionnaire exceeds 0.833, and the Bartlett’s Test of Sphericity reaches a significant level, indicating good validity of the questionnaire.
The learning style measurement tool utilized in this study was the A. Y. Kolb and Kolb (2005) Learning Style Inventory (LSI), which comprised 12 questions rated on a four-point Likert scale. Each question presented four options that represented different learning tendencies: concrete experience (CE), reflective observation (RO), abstract conceptualization (AC), and active experimentation (AE). Participants ranked the options in each question according to how closely they matched their own learning experiences.
Results
To validate the effectiveness of the GAI-PL approach, the study assessed its impact on students’ programming learning outcomes and the development of their 5C competencies, using learning styles and instructional methods (TPL and GAI-PL) as independent variables. The investigation specifically focussed on evaluating students’ learning achievements, motivation, self-efficacy, and 5C competencies.
Analysis of Learning Achievements
A two-way ANCOVA was performed to evaluate students’ learning achievements. The pre-test scores were used as a covariate, while the learning approach and learning styles served as independent variables, and the post-test scores were the dependent variable.
Table 1 provides a descriptive analysis of students’ learning achievements, confirming that the assumption of homogeneity of regression slopes was not violated (F = 1.644, p > .05). As shown in Table 2, there was no significant interaction between learning styles and learning approaches (F = 0.58, p > .05). Additionally, the results indicated that learning styles did not have a significant effect on students’ learning achievements (F = 1.08, p > .05). However, a main effect was found for learning approaches on post-test scores (F = 4.00, p < .05, η2 = 0.046), suggesting that the method of instruction influenced learning outcomes.
Descriptive Data of Students’ Learning Achievements.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Learning Achievements.
p < .05.
Specifically, the adjusted mean and standard deviation for the experimental group were 82.80 and 9.80, respectively, while for the control group, they were 76.30 and 15.00. This indicates that students who used the GAI-PL learning approach achieved higher learning outcomes compared to those who followed the TPL method.
Analysis of Learning Motivation
The homogeneity of regression remained unviolated (F = 1.72, p > .05). Table 3 presents an overview of the descriptive data related to learning motivation. Table 4 reveals that no significant interaction between learning approaches and the learning styles (F = 0.53, p > .05, η2 = 0.019). When analysing the main effects, both the learning approaches (F = 4.85, p < .05, η2 = 0.055) and learning styles (F = 3.71, p < .05, η2 = 0.118) exhibited differences in learners’ motivation.
Descriptive Data of Students’ Learning Motivation.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Learning Motivation.
p < .05.
The adjusted means and standard deviations for the experimental group were 4.02 and 0.65, respectively, while those for the control group were 3.80 and 0.66. Notably, students utilizing the GAI-PL learning approach demonstrated significantly higher learning motivation compared to those using the TPL approach. Additionally, divergers, with an adjusted mean of 4.22 and a standard deviation of 0.55, exhibited superior learning motivation.
Analysis of Self-Efficacy
The homogeneity of regression was upheld (F = 0.58, p > .05). Table 5 provides a comprehensive overview of the descriptive data pertaining to learners’ self-efficacy. Table 6 reveals that there was no significant interaction between learning approaches and learning styles (F = 0.69, p > .05, η2 = 0.024). Nevertheless, it remains evident that learning approaches exert a noteworthy influence on learners’ self-efficacy. (F = 33.39, p < .001, η2 = 0.287).
Descriptive Data of Students’ Self-efficacy.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Self-efficacy.
p < .001.
Specifically, the learners’ adjusted mean values for their post-test ratings of self-efficacy were 3.78, with a standard deviation of 0.58 for learners employing the GAI-PL approach, whereas those adhering to the TPL approach demonstrated a mean value of 3.13 (SD = 0.58). This highlights the potential of the GAI-PL approach to enhance learners’ self-efficacy.
Analysis of 5C Competence
To explore students’ 5C Competence, the collaborative aspect was initially analysed. The assumption of homogeneity of regression was upheld (F = 0.88, p > .05). Following the exclusion of pre-communication scores’ effects, no significant interaction emerged between learning approaches and learning styles (F = 0.80, p > .05). Furthermore, no notable variations were observed in the impact of learning styles on learners’ communication skills (F = 0.35, p > .05). This suggests that learning approaches influenced learners’ communication skills (F = 9.10, p < .05, η2 = 0.099). Post-test communication ratings adjusted mean values were 3.81 (SD = 0.64) for GAI-PL learners and 3.39 (SD = 0.64) for TPL learners, indicating the GAI-PL approach’s ability to enhance communication skills.
The homogeneity of regression for complex problem-solving skills was confirmed (F = 1.60, p > .05), allowing the use of a two-way ANCOVA. Descriptive data are provided in Table 7. According to Table 8, the learning approaches affected learners’ complex problem-solving skills (F = 25.24, p < .001, η2 = 0.233). The adjusted mean post-test ratings for complex problem-solving were 3.97 (SD = 0.48) for the GAI-PL group and 3.62 (SD = 0.74) for the TPL group, highlighting the GAI-PL approach’s ability to enhance these skills.
Descriptive Data of Students’ Complex Problem-solving Skills.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Problem-solving Skills.
p < .001.
The homogeneity of regression for creativity skills was confirmed (F = 1.13, p > .05), supporting the application of two-way ANCOVA. Descriptive data on learners’ creativity skills are shown in Table 9. According to Table 10, learning approaches significantly influenced students’ creativity (F = 11.55, p < .05, η2 = 0.122). Additionally, a significant interaction between learning approaches and learning styles was identified (F = 6.89, p < .001, η2 = 0.199). Subsequently, a simple main effect analysis was carried out, with results presented in Table 11 and Figure 6.
Descriptive Data of Students’ Creativity.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Creativity.
p < .01. ***p < .001.
Results of the Simple Main Effect Analysis of Students’ Creativity.
p < .01. ***p < .001.
Interaction between learning styles and learning approaches on students’ creativity.
As shown in Table 11, significant differences in creativity were found among learners with different learning styles in the GAI-PL group (F = 3.74, p < .05, η2 = 0.119) and TPL group (F = 3.30, p < .05, η2 = 0.107). Furthermore, significant differences in creativity were observed between divergers and assimilators with different learning approaches (F = 19.72, p < .001, η2 = 0.192; F = 14.71, p < .001, η2 = 0.150). Specifically, divergers in the GAI-PL group (adjusted M = 4.20; SD = 0.55) exhibited higher creativity than those in the TPL group (adjusted M = 3.39; SD = 0.60). Similarly, assimilators in the GAI-PL group (adjusted M = 4.02; SD = 0.70) demonstrated greater creativity than those in the TPL group (adjusted M = 3.44; SD = 0.60).
The homogeneity of regression for communication ability was confirmed (F = 0.98, p > .05). Table 12 presents the descriptive data for learners’ communication skills. According to the analysis in Table 13, an effect of learning approaches on students’ communication skills was found (F = 4.70, p < .05, η2 = 0.054). The adjusted mean post-test communication scores were 4.00 (SD = 0.61) for the GAI-PL group and 3.75 (SD = 0.57) for the TPL group, indicating the GAI-PL approach’s effectiveness in enhancing communication skills. No significant interaction between learning approaches and learning styles was identified (F = 0.48, p > .05, η2 = 0.017), and learning styles did not have a significant impact on learners’ communication abilities (F = 0.69, p > .05, η2 = 0.024).
Descriptive Data of Students’ Communication Ability.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Communication Ability.
p < .01.
The homogeneity of regression for meta-cognitive awareness was confirmed (F = 0.74, p > .05). Table 14 displays the descriptive data for students’ meta-cognitive awareness. According to the analysis in Table 15, no significant interaction was found between learning approaches and learning styles (F = 0.41, p > .05, η2 = 0.015), and no significant effect of learning styles on meta-cognitive awareness was observed (F = 0.99, p > .05, η2 = 0.035). However, an impact of learning approaches on meta-cognitive awareness was identified (F = 4.89, p < .001, η2 = 0.056). The GAI-PL approach (adjusted M = 3.65; SD = 0.53) proved more effective in enhancing learners’ meta-cognitive awareness compared to the TPL approach (adjusted M = 3.36; SD = 0.62).
Descriptive Data of Students’ Meta-cognitive Awareness.
M = mean; SD = standard deviation; n = number of participants.
Two-way ANCOVA Analysis of Students’ Meta-cognitive Awareness.
p < .01.
Analysis of Interview Results
The interviews indicated that teachers held a positive view of the pedagogical innovation introduced by GAI-PL in vocational programming education, noting that the integration of GAI significantly improved the efficiency of the teaching process. Teachers highlighted that “in the lesson preparation phase, GAI supports the teaching design by generating engaging scenarios and step-by-step classroom exercises, which substantially enhance both the efficiency and quality of lesson planning.” During the lessons, teachers acknowledged that “GAI provides students with a variety of personalized learning prompts, enabling effective differentiated instruction.” However, they also pointed out some practical challenges, mentioning that “the autonomous use of GAI in vocational classrooms may somewhat affect the efficiency of classroom management.”
Student interviews addressed two main aspects: learning experience and learning tools. In terms of learning experience, most students in the GAI-PL group adapted quickly to the new approach, stating that it stimulated their interest in learning more effectively than traditional methods. Regarding the learning tools, some students highlighted benefits such as “assisting with code ideas,”“efficiently correcting code errors,” and “providing timely responses.” However, a significant number of students felt that GAI was not intelligent enough, often offering rigid responses, and when it failed to deliver direct and accurate feedback, they would lose patience in further inquiries.
Discussion
Discussions on Learning Achievements
Interviews from the GAI-PL group revealed that when students encounter challenging concepts during the process of acquiring new knowledge, GAI provides prompt and targeted explanations. Its enhanced summarization and targeting capabilities, compared to traditional search engines, facilitate effective knowledge construction and help minimize unnecessary cognitive load (T.-T. Wu et al., 2024). Furthermore, GAI can generate detailed and engaging examples based on students’ requests, which deepens their understanding of concepts. In programming contexts, GAI supports students who have weaker foundations by offering programming ideas, ensuring they do not fall behind the class pace. Additionally, GAI assists in optimizing and debugging code after students complete it, allowing them to strengthen their coding skills and establish better programming habits. During the after-class consolidation phase, GAI enables students to integrate knowledge through interactive and personalized testing, reinforcing autonomous learning. According to the SOI model, the selection stage involves directing attention to relevant instructional information. GAI effectively supports this stage by providing targeted explanations and examples that help students focus on essential information. The organization stage involves clarifying the relationships between fragments of information within instructional material. GAI aids this stage by generating detailed examples and facilitating the connection between new content and students’ prior knowledge. The integration stage involves linking instructional information with students’ existing knowledge framework. GAI supports this stage by providing interactive and personalized testing, which helps students integrate new knowledge into their existing knowledge structure. In contrast, while the TPL group benefits from the teacher’s guidance, it predominantly focuses on the transmission and assimilation of knowledge. Traditional approaches struggle to accommodate individual student differences, often lacking the specificity needed for personalized learning. Moreover, traditional information technology tools used in teaching may not efficiently address each student’s questions in real-time, and teachers face challenges in managing and responding to individual needs within limited class periods. GAI, acting as a personal assistant for each student, effectively overcomes these limitations. The differences between the GAI-PL and TPL approaches, especially in terms of personalization and responsiveness, likely contribute to the significant improvement in learning achievements observed in the GAI-PL group.
Discussions on Learning Motivation and Self-efficacy
The GAI-PL group’s learning motivation and self-efficacy were significantly higher than those of the TPL group, a finding consistent with Yilmaz and Karaoglan Yilmaz (2023). Firstly, this approach emphasizes students’ active participation in the learning process. Through interaction with GAI, students can receive targeted learning advice and answers, helping them better understand and master knowledge, thereby enhancing their learning confidence. Finally, from the perspective of generative learning, the GAI-PL approach guides students through the process of selecting, organizing, and integrating information for learning. This approach encourages students to continually refine their knowledge structures and gain a sense of mastery over knowledge.
Discussions on 5C Competence
In terms of 5C competence, the GAI-PL approach significantly improves students’ collaboration, communication, complex problem-solving, creativity, and metacognitive awareness compared to the traditional TPL approach. During the learning process, students can access GAI for assistance at any time, fostering their habit of actively seeking help and collaborating. In the after-class consolidation phase, interactive testing with GAI further enhances their collaboration and communication skills. Moreover, GAI positively impacts students’ creative thinking and reflective abilities during classroom activities (Essel et al., 2024). By engaging with GAI, students learn to approach problems from diverse perspectives, expand their thinking boundaries, and strengthen their creativity and innovation awareness. Additionally, GAI’s immediate feedback and personalized guidance provide students with deeper insights into their learning progress, promoting the development of metacognitive awareness. After receiving feedback, students are encouraged to reflect on their learning processes, identify weaknesses, and make appropriate adjustments. According to the SOI model, the integration stage involves linking instructional information with students’ existing knowledge framework. GAI supports this stage by providing immediate feedback and personalized guidance, which helps students integrate new knowledge into their existing knowledge structure and develop metacognitive awareness.
In the GAI-PL group for learning styles, divergers show significantly higher motivation compared to other learners, and both divergers and assimilators demonstrate greater creativity with the GAI-PL approach than with the TPL approach. Previous studies indicate that divergent thinking is a crucial component of creativity, and divergers are often imaginative and creative (Chen et al., 2024), enjoying new experiences and engaging more with diverse learning tasks and challenges. The GAI-PL approach aligns with these traits, effectively boosting their motivation and creativity. In contrast, traditional approaches often focus on knowledge transmission. Additionally, divergers possess strong communication skills (Kolb et al., 2005), and the GAI-PL approach’s emphasis on human-computer interaction enhances these skills, further fostering creativity. Assimilators, known for their ability to plan, organize, analyse, and integrate new information systematically, benefit from the GAI-PL approach’s emphasis on generative learning, which aligns with their needs and stimulates their creativity in practice.
Conclusion
This study introduces a GAI-assisted programming learning (GAI-PL) method and applies it within the context of programming education in a vocational school. To thoroughly assess and compare the learning outcomes of students with different learning styles, a two-factor quasi-experimental design was implemented. The findings reveal that, compared to the traditional programming learning (TPL) group, the GAI-PL group exhibited significant advantages in several key areas, including learning achievements, motivation, self-efficacy, collaboration skills, complex problem-solving abilities, creativity, communication skills, and meta-cognitive awareness. Notably, within the GAI-PL group, divergers displayed significantly higher motivation compared to other learners. Additionally, there was a significant interaction between the learning approach and learning styles in terms of creativity, with divergers and assimilators using the GAI-PL approach demonstrating higher levels of creativity than their counterparts in the TPL approach.
Research Limitations
In this study, we developed a generative learning approach supported by generative artificial intelligence (GAI), demonstrating its effectiveness in enhancing students’ learning outcomes and 5C competencies. This marks a significant advancement in the application of GAI within the educational field. However, there are several limitations. The experimental intervention had a short duration and lacked a delayed post-test to assess long-term effects. Moreover, the study exclusively utilized ERNIE Bot as the GAI tool, which may have constrained the generalizability of the research. Additionally, the sample size and diversity of this study were limited, as it primarily involved vocational school students from specific regions. Furthermore, the gender imbalance present in the random sampling process was not thoroughly addressed during data analysis, potentially affecting the representativeness of the findings. Besides, the study focussed solely on the application of the approach in programming learning, leaving its effects in other subjects unexplored. Although the sample size and diversity of this study are restricted, mainly involving vocational school students from specific regions, which may limit the generalizability of the results to some extent, the study still holds representative significance for the following reasons. Firstly, the study focuses on the programming course in a vocational school, where students share relatively homogeneous learning environments and challenges, allowing for in-depth insights into the effects of GAI-supported programming learning in this specific educational context. Secondly, the rigorous quasi-experimental design, comprehensive assessment of multiple dimensions, and the theoretical foundation of generative learning theory and SOI framework ensure the reliability and validity of the research findings within the scope of this study. Lastly, as an exploratory study, this research lays the foundation for subsequent larger-scale studies, providing valuable references and directions for further exploration of the applicability of GAI-supported programming learning in different educational settings.
Future research should aim to expand the sample size and diversity, address gender balance issues, and conduct more comprehensive analyses across different subjects and educational levels to validate and broaden the applicability of the approach.
Implications
The findings of this study offer valuable insights for educators, educational institutions, and technology developers, highlighting the potential of Generative Artificial Intelligence (GAI) in enhancing programming education. For educators, the integration of GAI into programming courses can significantly enhance students’ learning outcomes, motivation, self-efficacy, and 5C competencies. Educators can leverage GAI to provide personalized learning experiences, addressing the diverse needs of students with different learning styles. GAI’s ability to offer real-time feedback and interactive assessments can help teachers monitor student progress more effectively and adjust their teaching strategies accordingly. Moreover, GAI can serve as a valuable tool for scaffolding students’ learning, particularly for those who struggle with abstract programming concepts, by providing targeted explanations and examples.
For educational institutions, incorporating GAI into curricula can foster a more inclusive and effective learning environment. By adopting GAI-supported learning approaches, institutions can better prepare students for the demands of the digital age, equipping them with essential 21st-century skills. Additionally, GAI can enhance the efficiency of instructional processes, allowing teachers to focus on more complex and personalized aspects of teaching while GAI handles routine tasks such as answering common questions and providing basic explanations. This can lead to more productive classroom interactions and improved learning outcomes across the board.
For technology developers, the results underscore the importance of developing GAI tools that are specifically tailored for educational purposes. Technology developers should focus on creating GAI platforms that are user-friendly, reliable, and capable of providing high-quality educational content. These platforms should be designed to support various learning activities, from pre-class preparation to post-class consolidation, and should be adaptable to different educational contexts and student needs. Furthermore, developers should prioritize the ethical use of GAI in education, ensuring that their tools promote academic integrity and protect students’ privacy.
Footnotes
Acknowledgements
We acknowledge the support received from the National Natural Science Foundation of China (Grant No. 62077043).
Ethical Considerations
This study was approved by the Ethics Review Committee of the College of Education, Zhejiang University of Technology, Zhejiang Province, China.
Consent to Participate
Written informed consent has been obtained from the participants and their guardians.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (Grant No. 62077043).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.