Visual Programming Environments and Computational Thinking Performance of Fifth- and Sixth-Grade Students

Abstract

Currently, many countries actively cultivate students to develop computational thinking ability. Many visual programming environments (VPEs) and physical robot courses have been integrated into computational thinking learning in the elementary education stage. This study explores the relationship between the programming learning environment (including VPE, physical robots, and no experience) and the computational thinking ability of higher-grade elementary school students of different genders. The results show that learning through VPE or physical robots can help students improve their computational thinking ability and that students learn better via physical robots. In addition, among the four dimensions of computational thinking ability, most students are weak in algorithm design. In terms of gender, no differences exist in computational thinking ability. Further analysis reveals that female students have better decomposition performance in VPE learning, while male students have better algorithm design performance.

Keywords

computational thinking visual programming environments (VPEs)robotics

In 2006, Wing, a scholar at Carnegie Mellon University in the United States, proposed the concept of computational thinking, which is applying the basic abilities of computer science to solve problems, design systems, and understand human behavior (Wing, 2006). In addition to the abilities of reading, writing, and mathematics, computational thinking is a basic ability for everyone, not exclusively for computer scientists. Computational thinking uses inductive, embedded, transformed, or simulated methods to turn complex problems into simple daily problems. Computational thinking is also a problem-solving ability for system design via abstraction and decomposition (García-Peñalvo, 2018; Wing, 2008).

Since computational thinking has been proposed, many advanced countries in the world, such as the United States (CSTA, 2011), the United Kingdom (Department for Education, 2013), Estonia (HITSA, 2015), Australia (Australian Curriculum, Assessment, Reporting Authority, 2013), Germany (Brinda et al., 2009), and Israel (Gal-Ezer & Harel, 1999), have begun to integrate such thinking into educational frameworks and have trained subject teachers. These countries are aware of the importance of computer science and have begun to integrate computer science into the curriculum. In Taiwan, information technology and life technology have been moved from the original learning area to a new separate learning area, the technology area, in 12-year compulsory education to enhance the scientific and technological capabilities of students, and it has also been formally included in the curriculum (NAER, 2018).

Further changes in education have been observed. Many countries have begun to train subject teachers and include computational thinking-related courses in the curriculum so that students can learn computational thinking skills from a young age. The common method of developing computational thinking ability from a young age is through coding (Buitrago Flórez et al., 2017; García-Peñalvo et al., 2018). However, Costelloe (2004) and Powers et al. (2007) noted that students who are new to programming often experience difficulties and frustration in terms of syntax and design. The traditional user interface of the compiler requires students to learn many commands, making most students feel bored and lacking interest in learning (Mannila et al., 2006).

Therefore, visual programming environments (VPEs) and interactive toys were developed by research units and manufacturers to enhance students' interest in learning. VPE involves visualization of the operational process of programs, which avoids the need for students to transform their ideas into program code. Therefore, students can more effectively build programs to control computer behavior (Repenning et al., 2010). Many VPEs have been developed around the world, such as code.org, Kodu, Blockly, and CodeMonkey. Among them, Scratch, developed by MIT, is the most widely used VPE around the world. Physical robots can be controlled via VPE design programs, and many types of computational thinking physical robots are available, such as mBot, Zenbo, Dash & Dot, SPRK+, Micro:bit, and Wedo 2.0.

The VPE and physical robots mentioned above have been integrated into computational thinking courses. Both have demonstrated positive learning performance in related research. For example, in terms of VPE, Sáez-López et al. (2016) analyzed 107 primary school students from five schools in Spain and concluded that students in grades 5 to 6 experience significant improvements in performance and attitude towards programming concepts, logic, and computational practices after their two-year course in Scratch. These students also maintained motivation and enthusiasm for learning by creating their own programs. Calao et al. (2015) also trained computational thinking in sixth-grade students through Scratch and explored whether these problem-solving abilities could be applied to various types of problems. The results showed that the experimental group, trained in Scratch, had a statistically significant increase in the understanding of mathematical processes. Fewer studies on the learning effects of computational thinking compared to VPE are available for physical robots, but a tendency of improved performance has been observed. For example, Chou (2018) conducted a study on mBot in 30 fifth-grade elementary school students. The students in the experimental group, using mBot, showed improved knowledge of coding and problem-solving skills. Morze et al. (2017) explored the issues of educational robots as a learning topic in elementary school. The results indicated that programmable bricks provide opportunities for children to create their own products and learn robot programming, help students develop problem-solving skills and improve academic performance.

Although both learning environments mentioned above are effective, “whether students have to learn computational thinking through VPE and physical robots” and “whether students who are learning through physical robots have better performance than those through VPE” remain unclear. The goal of this study is to understand whether VPE and the physical robot learning environment contribute to their improvement in computational thinking. The first objective is to explore the effectiveness of computational thinking ability for higher-grade elementary school students with different programming learning environments, including no visual programming experience (NoVPE Group), basic visual programming experience (VPE Group), and visual programming and physical robotic control experience (VPERobot Group).

In addition, the issue of gender differences has been widely discussed in many coding and robot studies (Cameron et al., 2018; Sandygulova & O'Hare, 2018; Showkat & Grimm, 2018). Although previous studies have found that in kindergarten, boys perform better than girls in robot- and programming-related fields, Sullivan and Bers (2013) studied whether kindergarten boys and girls have the same performance in a range of architectural and programming tasks as TangibleKrobot programs. The results showed that boys had higher average scores than girls on more than one-half of the tasks, but few statistically significant differences in scores were found. Boys scored significantly higher than girls in only two areas, properly attaching robotic materials and programming using ifs. Milto et al. (2002) taught the fundamentals of engineering through LEGO and ROBOLAB and asked students to work in groups and participate in design competitions. The results showed that both female and male students enjoyed this competition, while male students were more confident in their abilities. In addition, male students paid more attention to competition than female students, and males learned to work together in a small group to make the most progress in their tasks. Kalelioğlu (2015) studied the influence of computational thinking of female and male students on problem-solving skills by means of teaching methods on the website of code.org. The study found no difference in problem-solving skills between male and female primary school students. The responsive ability of female students in problem solving increased slightly. In addition, both male and female students had positive attitudes towards coding.

According to the studies above, when students are engaged in learning activities such as coding or robots, both males and females have positive attitudes towards these learning activities. Even if males and females had differences in abilities in terms of program logic, problem solving or teamwork, they did not show their dislike of these activities. Although some researchers have studied the differences in learning activities, such as coding or robots, among students of different genders, the gender differences in the performance of computational skills for students are less discussed. Therefore, the second objective of this study is to examine whether gender differences exist in the ability to learn computational thinking and which specific abilities are affected.

Finally, a set of assessments is required to discuss different programming learning environments and the computational thinking ability differences of students of different genders. Regarding evaluating students’ computational thinking ability, some scholars are working to develop a computational thinking assessment framework for students. For example, Brennan and Resnick (2012) developed a computational thinking assessment framework that analyzed the computational thinking ability of youth (productions on a specific topic) based on concepts, practice and viewpoints via three methods—the analysis of productions on a specific topic, interviews for production, and design scenario. Werner et al. (2012) used 3D game records on the website alice.org to assess students’ computational thinking performance. Selby et al. (2014) developed a computational thinking framework based on qualitative observations to assess whether students were able to apply the concept of computational thinking (including abstraction, decomposition, algorithmic thinking, evaluation, and generalization). Koh et al. (2014) developed a set of visualized analyses for the computational thinking ability of the REACT system based on game learning and simulation semantic analysis. Chen et al. (2017) designed scenario questions and open questions to assess students’ computational thinking ability. According to the diverse assessment methods for computational thinking ability mentioned above, developing an assessment framework is the first step in assessing computational thinking ability. Other methods, such as individual productions on a specific topic and qualitative analysis, help to understand the development of diverse abilities because of the long duration required to complete a work and the analysis of work via video recording, but this method is time consuming. Record analysis helps to analyze student behavior and the effectiveness of learning more quickly but is often restricted by the system, and the system may not be available for all teachers. Therefore, according to the discussions above, this study designed a set of single-choice questions to assess computational thinking ability.

On the basis of the above discussion, the purpose of this study is to design a computational thinking assessment based on a set of single-choice questions to explore

The relationship between the programming learning environment (including VPE, physical robots, and no experience).

The computational thinking ability of higher-grade elementary school students of different genders.

Methodology

Participations and Procedure

This study applied purposive sampling to select fifth- and sixth-grade students in Taiwan. The students in this study were divided into three groups, the NoVPE Group, VPE Group and VPERobot Group. The following is an explanation for the three groups:

NoVPE Group: Students who have not used or studied visual programming environments.

VPE Group: Students have used or studied related software in visual programming environments, such as Scratch, code.org, Kodu, Blockly, and CodeMonkey.

VPERobot Group: Students have used physical robots, such as mBot, Zenbo, Dash & Dot, SPRK+, Micro:bit, and Wedo 2.0. These physical robots must be controlled in visual programming environments.

Each group included 60 students for a total of 180 students. For the second objective of the study, the author sampled 30 male students and 30 female students from each group, for a total of 90 male students and 90 female students (such as Table 1). The research process included (a) design assessment tools (literature review, content design for the assessment and expert validity analysis), (b) sample selection, (c) testing and (d) data analysis.

Table 1.

Grouping and Number of Participations.

Group	Male students	Female students	Total students
NoVPE Group	30	30	60
VPE Group	30	30	60
VPERobot Group	30	30	60
Total students	90	90	180

Instruments

This study referred to the descriptions of computational thinking ability from several studies (e.g., CSTA, 2011; Google, 2010; Grover & Pea, 2013; Wing, 2006). After discussion with elementary school teachers, the study focused on comprehension ability and application because the subjects were higher-grade elementary school students. The classification of computational thinking proposed by Google (2010) is simpler and more suitable for the learning level of higher-grade elementary school students than is the classification proposed by Wing (2006). Therefore, this study divided computational thinking ability into four assessment dimensions—decomposition, pattern recognition, abstraction and algorithm design. The descriptions of each dimension are as follows:

Decomposition: Decompose a complex problem into many small problems, which makes it easier to understand, handle, and maintain.

Pattern recognition: Find similarities between different problems.

Abstraction: Focus on important information and ignore irrelevant details.

Algorithm design: Develop steps or rules to solve the problem.

Next, according to the four dimensions of decomposition, pattern recognition, abstraction and algorithm design, the author designed five questions for each dimension, with one point for each question, for a total of 20 points. Next, the study adopted the Delphi method (Hasson et al., 2000) and conducted expert validity analysis by inviting three teachers with experience in promoting computational thinking in elementary schools. The design of the questions was revised based on feedback from these teachers. This step was conducted in three rounds. In the first round, the descriptions of questions and options should align with the cognitive ability of higher-grade elementary school students, especially considering the issue of urban-rural disparity; second, the descriptions of questions and options should consider whether the students have learning environment in the software or hardware of computational thinking; third, each question in the dimensions of computational thinking should be diverse and should not be just a difference in the difficulty. In the second round, the three experts reviewed the suggestions for revisions of others and proposed amendments in text and presentation format. Finally, in the third round, only a few words were corrected. After the revisions were completed according to the teachers' suggestions, the computational assessment was developed. One sample question for each dimension is shown in Table 2.

Table 2

. Sample Questions for Computational Thinking Ability Assessment.

Dimensions	Sample question
Decomposition	（）4. How many triangles are there in the picture below? (1) 2 (2) 3 (3) 4 (4) 5 (5) 6
Pattern recognition	（）9. Please find the same phrases in the following narrative, which of the following answer is correct.There are two dogs kept in my backyard.There are five fishes kept in my family tank.There is a bird kept in my family cage.(1)_________ kept in my family.(2) There is a_________ kept in my family _________.(3) There is a _________ kept_________ in my family _________.(4) _________ kept in my family_________.(5) None of the above.
Abstraction	（）15. If you want to draw a train by imagination, which of the following is not necessary? Windows (2) Chimneys (3) Wheels (4) Cars (5) Rails
Algorithm design	（）19. Mr. Robot has a new instruction E, and the content of this instruction is to execute command A first and then execute command C. Brackets () and a number are added after command E to indicate the number of times that command E is repeatedly executed. For example: E(1) = AC, E(2) = ACAC, E(3) = ACACAC. If using the E() command, which of the following can draw a square?(1) E(0) (2) E(1) (3) E(2) (4) E(3) (5) E(4)

Analysis Methods

To examine the objectives of the study, the author used SPSS software for statistical analysis. For the first objective, the relationship between the programming learning environment (including VPE, physical robots, and no experience), descriptive statistics, and ANOVA are adopted. For the second objective, the computational thinking ability of elementary school students of different genders, descriptive statistics, and t-tests are adopted.

Results

After the computational thinking ability assessment was completed, the study conducted an assessment of 180 higher-grade elementary school students in Taiwan based on the research design. The results of these 180 students on four dimensions of computational thinking ability are shown in Table 3. The students had the highest average score in pattern recognition (M = 3.66, SD = 1.30) and the lowest average score in algorithm design (M = 2.02, SD = 1.34). This result showed that students' ability to abstract concepts of programming needs to be strengthened. Because algorithm design is the final result of computational thinking, teachers must pay more attention to helping students develop such skills.

Table 3.

The Scores on Four Dimensions of Computational Thinking Ability.

Dimension	Total score	NoVPE Group		VPE Group		VPERobot Group		Total
Dimension	Total score	M	SD	M	SD	M	SD	M	SD
Decomposition	5	2.55	1.11	3.77	1.16	3.57	1.28	3.29	1.29
Pattern recognition	5	3.13	1.52	3.85	.95	3.98	1.23	3.66	1.30
Abstraction	5	2.57	1.23	3.4	.91	3.32	1.02	3.09	1.12
Algorithm design	5	1.23	1.05	2.13	1.17	2.68	1.38	2.02	1.34
Total performance	20	9.48	3.35	13.15	2.35	13.55	3.52	12.06	3.60

Further analysis compared the NoVPE Group, VPE Group and VPERobot Group: the ranking of the average scores was VPERobot Group (M = 13.55, SD = 3.52) > VPE Group (M = 13.15, SD = 2.35) > NoVPE Group (M = 9.48, SD = 3.35). The preliminary descriptive analysis showed that students with a programming learning environment had higher computational thinking abilities than those with no learning environment. Overall, the average pattern recognition score was the highest (M = 3.66, SD = 1.30), while the algorithm design score was the lowest (M = 2.02, SD = 1.34).

The study analyzed the performance difference among the students with experience in programming learning and those with no experience via ANOVA. Table 4 shows the results of the homogeneity test of variance. The variances of composition, abstraction and algorithm design are not statistically significant, indicating no significant difference in deviation among these three groups. However, the result for pattern recognition was statistically significant, indicating that the two groups had significant differences in performance. For the post hoc test, the study applied the LSD method on decomposition, abstraction and algorithm design and the Dunnett T3 test on pattern recognition because of the sample deviation.

Table 4.

Homogeneity of Variance Hypothesis Test for Computational Thinking Ability.

Dimension	Levene Statistic	df1	df2	Sig.
Decomposition	.784	2	177	.458
Pattern recognition	7.856	2	177	.001
Abstraction	2.726	2	177	.068
Algorithm design	2.344	2	177	.099
Total performance	4.682	2	177	.010

As shown in Table 5, a statistically significant difference was observed for students with and without the program learning environment on the four dimensions of computational thinking. The rankings of performance in each dimension were the same—VPE Group > NoVPE Group and VPERobot Group > NoVPE Group—indicating that the students who learned the visual program software or hardware showed improvements in computational thinking ability. In addition, in the area of algorithm design, the average score of the VPERobot Group was significantly higher than that of the VPE Group. Post hoc interviews indicated that when students designed various types of physical robots, their basic knowledge came from the fundamentals of VPE. In other words, spending more learning time on programming was one factor that affected academic performance. In addition, when designing movements for a physical robot, the robot provides instant and physical responses to their designs, in contrast to VPE. When students can see the results of their design, they can easily put the concept of virtual programming into practice (Costa et al., 2015; Morze et al., 2017).

Table 5.

ANOVA Table of Computational Thinking Ability.

Dimension		Sun of Squares	df	Mean Square	F	Sig.
Total	2318.328	179
Decomposition	Between Groups	51.078	2	25.539	18.204	.000	VPE > NoVPE (1.217) VPERobot > NoVPE (1.017)
	Within Groups	248.317	177	1.403
	Total	299.394	179
Pattern recognition	Between Groups	25.078	2	12.539	7.939	.000	VPE > NoVPE (.717) VPERobot > NoVPE (.850)
	Within Groups	279.567	177	1.579
	Total	304.644	179
Abstraction	Between Groups	25.278	2	12.639	11.292	.000	VPE > NoVPE (.833) VPERobot > NoVPE (.750)
	Within Groups	198.117	177	1.119
	Total	223.394	179
Algorithm design	Between Groups	64.300	2	32.150	22.001	.000	VPE > NoVPE (.900) VPE Robot > No VPE (1.450) VPERobot > NoV PE (.550*)
	Within Groups	258.650	177	1.461
	Total	322.950	179
Total performance	Between Groups	602.844	2	301.422	31.100	.000	VPE > NoVPE (3.667) VPERobot > NoVPE (4.067)
	Within Groups	1715.483	177	9.692

The improvement of computational thinking skills through different learning environments enhances students’ problem-solving skills. Kalelioglu et al. (2016) and Román-González et al. (2017) also pointed out that CT is fundamentally linked with general mental ability, such as inductive reasoning and spatial and verbal abilities. This corroborates the conceptualization of CT as a problem-solving ability. Thus, learning through VPE or VPERobot environments is one of the ways to improve computational thinking ability.

Next, we explored the gender differences of students with learning program experience. Table 6 shows that the average scores of female students on each of the four dimensions were slightly higher than those of male students. Both female and male students scored highest on pattern recognition and lowest on algorithm design.

Table 6.

Overall Gender Difference in Scores of Computational Thinking Ability.

Dimensions	Total scores	Male students (n=90)		Female students (n=90)
Dimensions	Total scores	M	SD	M	SD
Decomposition	5	3.14	1.27	3.44	1.31
Pattern recognition	5	3.59	1.41	3.72	1.19
Abstraction	5	2.94	1.24	3.24	.96
Algorithm design	5	2.16	1.49	1.88	1.17
Total performance	20	11.83	4.05	12.29	3.08

Furthermore, the study conducted a t-test of gender differences on the four dimensions of computational thinking (Table 7). For the Levene test, no statistically significant difference was observed for degradation, pattern recognition, abstraction and algorithm design, indicating that sample deviation was not obvious. However, the total scores were statistically significant, indicating significant differences among sample deviations. Therefore, when conducting a t-test without the assumption of equal variance in each group, no statistically significant gender differences were observed in overall computational thinking ability or in the scores for the four dimensions. In other words, there was no gender difference in computational thinking ability.

Table 7.

t-Test for Gender Differences in Computational Thinking Ability.

Dimension	Levene's Test for Equality of Variances		t-Test for Equality of Means
	F	Sig.	t	df	Sig. (2-tailed)
Decomposition	.001	.977	−1.562	178	.120
Decomposition			−1.562	177.831	.120
Pattern recognition	3.873	.051	−.685	178	.494
Pattern recognition			−.685	172.985	.495
Abstraction	2.362	.126	−1.813	178	.072
Abstraction			−1.813	167.793	.072
Algorithm design	4.962	.027	1.391	178	.166
Algorithm design			1.391	168.427	.166
Total performance	7.161	.008	−.848	178	.397
Total performance			−.848	166.173	.397

In addition to the overall computational thinking ability of male and female students, the study further explored the gender differences among the NoVPE Group, VPE Group and VPERobot Group. The results are shown in Tables 8 and 9. Female students in the NoVPE group scored significantly higher than male students (t = −2.091, p = .041). In the VPE group, the female students scored significantly higher overall than the male students (t = −2.838, p = .006), but in terms of algorithm design, the male students scored significantly higher than the female students (t = 2.036, p = .046). Finally, no gender difference was observed in the VPERobot group. On the basis of the above results, after learning through VPE, female students had a stronger ability to decompose, while male students had a stronger ability to perform algorithm design. Decomposition involves whole-to-detail thinking ability, while algorithm design is an ability of abstraction to a physical image. The possible reason is that female students have an advantage in problem solving in a problem situation with a social condition (Tarampi et al., 2016), and many studies have pointed out that male students are better in logical thinking (Hutchins et al., 2017). A gender difference was observed for these two abilities; therefore, teachers should pay attention to gender differences in their future teaching. Additionally, students of different genders should be included in each group during group activities. Similar to the results of the study by Zhan et al. (2015), the arrangements of two males and two females (2 M2F) and four females (4F) will improve group performance. Speck (2003) also suggested equal numbers of male and female students in groups because the communication style of female members improves team coherence.

Table 8.

Gender Differences in Grouping and Computational Thinking Ability Scores.

	Gender	NoVPE Group		VPE Group		VPERobot Group
	Gender	Mean	SD	Mean	SD	Mean	SD
Decomposition	Male (n=30)	2.57	1.22	3.37	1.19	3.50	1.23
Decomposition	Female (n=30)	2.53	1.01	4.17	0.99	3.63	1.35
Pattern recognition	Male (n=30)	2.73	1.55	4.07	0.98	3.97	1.27
Pattern recognition	Female (n=30)	3.53	1.41	3.63	0.89	4.00	1.20
Abstraction	Male (n=30)	2.30	1.34	3.40	1.04	3.13	1.07
Abstraction	Female (n=30)	2.83	1.05	3.40	0.77	3.50	0.94
Algorithm design	Male (n=30)	1.27	1.05	2.43	1.31	2.77	1.65
Algorithm design	Female (n=30)	1.20	1.06	1.83	0.95	2.60	1.07
Total performance	Male (n=30)	8.87	3.45	13.27	2.69	13.37	4.20
Total performance	Female (n=30)	10.10	3.18	13.03	1.99	13.73	2.73

Table 9.

t-Test for Gender Differences in Grouping and Computational Thinking Ability.

		Levene's Test for Equality of Variances		t-Test for Equality of Means
		F	Sig.	t	df	Sig. (2-tailed)
NoVPE Group	Decomposition	2.318	.133	.115	58	.909
	Decomposition			.115	55.962	.909
	Pattern recognition	.951	.334	−2.091	58	.041
	Pattern recognition			−2.091	57.455	.041
	Abstraction	3.166	.08	−1.712	58	.092
	Abstraction			−1.712	54.882	.093
	Algorithm design	.136	.713	.245	58	.808
	Algorithm design			.245	57.988	.808
	Total performance	.016	.899	−1.44	58	.155
	Total performance			−1.44	57.606	.155
VPE Group	Decomposition	5.989	.017	−2.838	58	.006
	Decomposition			−2.838	56.078	.006
	Pattern recognition	0	.987	1.793	58	.078
	Pattern recognition			1.793	57.466	.078
	Abstraction	1.142	.29	0	58	1
	Abstraction			0	53.522	1
	Algorithm design	3.576	.064	2.036	58	.046
	Algorithm design			2.036	53	.047
	Total performance	5.329	.025	.382	58	.704
	Total performance			.382	53.439	.704
VPERobot Group	Decomposition	.111	.74	−.4	58	.69
	Decomposition			−.4	57.447	.69
	Pattern recognition	.056	.814	−.104	58	.917
	Pattern recognition			−.104	57.82	.917
	Abstraction	.15	.7	−1.408	58	.164
	Abstraction			−1.408	56.96	.164
	Algorithm design	7.592	.008	0.463	58	.645
	Algorithm design			0.463	49.649	.645
	Total performance	4.766	.033	−.401	58	.69
	Total performance			−.401	49.792	.69

Conclusion and Suggestion

An assessment was designed and conducted to evaluate the performance based on a set of multiple choice questions to understand the gender differences in the program learning environment and computation thinking ability. The results showed that learning through VPE or physical robot programming can help students improve their computational thinking ability. In addition, among the four dimensions of computation thinking ability, most students are weakest in algorithm design. Finally, no gender difference exists in learning program experience, which is the same conclusion as a previous study (i.e., Kalelioğlu, 2015; Sullivan & Bers, 2013). Moreover, no gender difference was observed in computational thinking ability. Further analysis indicated that in the group with no programming learning environment, the female students had better performance on pattern recognition, while male students had better performance on algorithm design.

The conclusions based on the above results are as follows. First, on the basis of the results of different learning programs, the training effect may influence performance (Özsoy & Ataman, 2017). Allowing students to practice solving different tasks helps to improve computational thinking ability. Therefore, parents or schools should integrate computational thinking software or hardware into learning at the elementary school stage. Second, according to the international curriculum design provided on the websites, step-by-step tasks are designed for most courses, such as moving from point A to point B. Because these tasks are simple and lack problem-solving scenarios linked to daily life, students have difficulty developing computational thinking ability on the four dimensions. Additionally, this problem causes students to have poorer ability in terms of algorithm design (Selby, 2015). Therefore, in addition to teaching programming syntax to build blocks of programs, teachers should design course activities from the level of leaning syntax to the level of problem solving. In other words, we suggest that teachers might follow four steps—composition, pattern recognition, abstraction, and algorithm design—when teaching programming and then let students build the blocks of the program according to the flow chart of the algorithm. Third, following the previous suggestion, students have better performance on decomposition and pattern recognition and worse performance on abstraction and algorithm design. The first two are more concrete abilities, and the latter two are more abstract abilities. Therefore, when teachers teach in programming, they should pay attention to the difficulties encountered by students in abstraction and algorithm design and provide more guidance and explanation.

Fourth, when students learn programming, those with a physical robot learning environment perform better in terms of algorithm design than those with a VPE learning environment. Because students have limited concentration when learning, the instant and physical feedback provided by a robot is important. Teachers should design diverse activities to attract students’ attention and enhance their learning motivation (Shinozawa et al., 2005). Fifth, in terms of gender differences, no statistically significant difference in computational thinking ability is observed. Among the VPE students, the female students perform better on decomposition, while male students perform better on algorithm design, indicating that gender differences are not obvious in terms of overall computational thinking ability, although some differences may result from the training environment. Therefore, teachers should treat male and female students equally and divide students equally by gender into groups for group activities, which will help the progress of activities (e.g., Speck, 2003; Su et al., 2019; 2020; Zhan et al., 2015).

Finally, in terms of the limitations of this study, first, this study included the same number of students (and equal numbers of students of each gender) in each learning environment group for the statistical analysis. More samples are recommended to conduct statistical analysis in future studies. Second, this study analyzes whether exposure to VPE and physical robots affects computational thinking ability. In fact, most students who learn with physical robots also learn through VPE, which makes it difficult to distinguish the performance differences between the two groups. This study does not explore the performance difference based on learning duration. Therefore, the results should be considered carefully. Third, although the assessment conducted in this study was designed by the author with consideration of reliability and validity, it may not be well designed. For example, the author divides computational thinking ability into four dimensions—decomposition, pattern recognition, abstraction and algorithm design—but some scholars have proposed more dimensions of ability (e.g., CSTA, 2011; Grover & Pea, 2013; Su & Chen, 2020; Su & Wu, 2020; Wing, 2006). Therefore, future studies may consider more dimensions of ability when designing an assessment.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the Ministry of Science and Technology of Taiwan under contract numbers MOST 108-2511-H-153-009, MOST 108-2511-H-153-010, MOST 108-2511-H-019-002, MOST 108-2511-H-019-003, MOST 109-2511-H-019-004-MY2, and MOST 109-2511-H-019-001.

ORCID iD

Yu-Sheng Su

Author Biographies

Dr. Sheng-Yi Wu is currently an associate professor of Department of Science Communication at National Pingtung University, Taiwan.

Dr. Yu-Sheng Su is currently an assistant professor of Department of Computer Science and Engineering at National Taiwan Ocean University, Taiwan.

References

Australian Curriculum, Assessment, Reporting Authority. (2013). Draft Australian curriculum: Technologies. http://consultation.australiancurriculum.edu.au/Static/docs/Technologies/Draft%20Australian%20Curriculum%20Technologies%20-%20February%202013.pdf

Brennan

Resnick

(2012, April). New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 annual meeting of the American Educational Research Association, Vancouver, Canada (pp. 1–25).

Brinda

Puhlmann

Schulte

(2009). Bridging ICT and CS: Educational standards for computer science in lower secondary education. ACM SIGCSE Bulletin, 41(3), 288–292.

Buitrago Flórez

Casallas

Hernández

Reyes

Restrepo

Danies

(2017). Changing a generation’s way of thinking: Teaching computational thinking through programming. Review of Educational Research, 87(4), 834–860.

Calao

L. A.

Moreno-León

Correa

H. E.

Robles

(2015). Developing mathematical thinking with scratch. In Design for teaching and learning in a networked world (pp. 17–27). Springer.

Cameron

Millings

Fernando

Collins

E. C.

Moore

Sharkey

A., . . .

Prescott

(2018). The effects of robot facial emotional expressions and gender on child–robot interaction in a field study. Connection Science, 30(4), 343–361.

Chen

Shen

Barth-Cohen

Jiang

Huang

Eltoukhy

(2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175.

Chou

P. N.

(2018). Skill development and knowledge acquisition cultivated by maker education: Evidence from Arduino-based educational robotics. EURASIA Journal of Mathematics, Science and Technology Education, 14, 1–9.

Costa

Lehmann

Dautenhahn

Robins

Soares

(2015). Using a humanoid robot to elicit body awareness and appropriate physical interaction in children with autism. International journal of social robotics, 7(2), 265–278.

10.

Costelloe

(2004). Teaching programming: The state of the art. Department of Computing, Institute of Technology Tallaght, Dublin 24. CRITE Technical Report.

11.

CSTA. (2011). CSTA K–12 computer science standards. The ACM K-12 Education Task Force. http://www.csta.acm.org/Curriculum/sub/CurrFiles/CSTA_K-12_CSS.pdf

12.

Department for Education. (2013). National curriculum in England: Computing programmes of study. https://www.gov.uk/government/publications/national-curriculum-in-england-com puting-programmes-of-study

13.

Gal-Ezer

Harel

(1999). Curriculum and course syllabi for a high-school program in computer science. Computer Science Education, 9(2), 114–147.

14.

García-Peñalvo

F. J.

(2018). Computational thinking. IEEE Revista Iberoamericana de Tecnologías del Aprendizaje (IEEE RITA), 13(1), 17–19. https://doi.org/10.1109/RITA.2018.2809939

15.

García-Peñalvo

F. J.

Reimann

Maday

(2018). Introducing coding and computational thinking in the schools: The TACCLE 3–coding project experience. In Computational thinking in the STEM disciplines (pp. 213–226). Springer.

16.

Google. (2010). Exploring computational thinking. https://www.google.com/edu/computational-thinking/

17.

Grover

Pea

(2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.

18.

Hasson

Keeney

McKenna

(2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008–1015.

19.

HITSA. (2015). Programming at schools and hobby clubs. http://www.innovatsioonikeskus.ee/en/programming-schools-and-hobby-clubs

20.

Hutchins

N. M.

Zhang

Biswas

(2017). The role gender differences in computational thinking confidence levels plays in STEM applications. In The International Conference on Computational Thinking Education (CTE’17) (pp. 34–38). The Education University of Hong Kong, Hong Kong.

21.

Kalelioğlu

(2015). A new way of teaching programming skills to K-12 students: Code.org. Computers in Human Behavior, 52, 200–210.

22.

Kalelioglu

Gulbahar

Kukul

(2016). A framework for computational thinking based on a systematic research review. Baltic Journal of Modern Computing, 4(3), 583–596.

23.

Koh

K. H.

Basawapatna

Nickerson

Repenning

(2014, July). Real time assessment of computational thinking. In 2014 IEEE symposium on visual languages and human-centric computing (VL/HCC) (pp. 49–52). IEEE.

24.

Mannila

Peltomaki

Salakoski

(2006). What about a simple language? Analyzing the difficulties in learning to program. Computer Science Education, 16(3), 211–227.

25.

Milto

Rogers

Portsmore

(2002). Gender differences in confidence levels, group interactions, and feelings about competition in an introductory robotics course. In 32nd Annual frontiers in education (Vol. 2, pp. F4C-F4C). IEEE.

26.

Morze

Smyrnova-Trybulska

Zuziak

Kommers

Gladun

(2017). Robotics in primary school in the opinion of prospective and in-service teachers. A comparison study. International Journal of Continuing Engineering Education and Life Long Learning, 27(4), 318–338.

27.

NAER. (2018). Directions governing for 12-year basic education curriculum. https://www.naer.edu.tw/files/15-1000-14113,c639-1.php?Lang = zh-tw

28.

Özsoy

Ataman

(2017). The effect of metacognitive strategy training on mathematical problem solving achievement. International Electronic Journal of Elementary Education, 1(2), 67–82.

29.

Parmar

Babu

S. V.

Lin

Jörg

D'Souza

Leonard

A. E.

Daily

S. B.

(2016, August). Can embodied interaction and virtual peer customization in a virtual programming environment enhance computational thinking? In Research on equity and sustained participation in engineering, computing, and technology (RESPECT), 2016 (pp. 1–2). IEEE.

30.

Powers

Ecott

&. Hirshfield

(2007). Through the looking glass: Teaching CS0 with Alice. In Proceedings of the 38th SIGCSE technical symposium on computer science education, 39 1, pp. 213–217.

31.

Repenning

Webb

Ioannidou

(2010, March). Scalable game design and the development of a checklist for getting computational thinking into public schools. In Proceedings of the 41st ACM technical symposium on computer science education (pp. 265–269). ACM.

32.

Román-González

Pérez-González

J. C.

Jiménez-Fernández

(2017). Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior, 72, 678–691.

33.

Sáez-López

J. M.

Román-González

Vázquez-Cano

(2016). Visual programming languages integrated across the curriculum in elementary school: A two year case study using “Scratch” in five schools. Computers & Education, 97, 129–141.

34.

Sandygulova

O’Hare

G. M.

(2018). Age-and gender-based differences in children’s interactions with a gender-matching robot. International Journal of Social Robotics, 10(5), 687–700.

35.

Selby

C. C.

(2015, November). Relationships: Computational thinking, pedagogy of programming, and Bloom's Taxonomy. In Proceedings of the workshop in primary and secondary computing education (pp. 80–87). ACM.

36.

Selby

Dorling

Woollard

(2014). Evidence of assessing computational thinking. http://eprints.soton.ac.uk/372409/1/372409EvidAssessCT.pdf

37.

Shinozawa

Naya

Yamato

Kogure

(2005). Differences in effect of robot and screen agent recommendations on human decision-making. International Journal of Human-Computer Studies, 62(2), 267–279.

38.

Showkat

Grimm

(2018, June). Identifying gender differences in information processing style, self-efficacy, and tinkering for robot tele-operation. In 2018 15th international conference on ubiquitous robots (UR) (pp. 443–448). IEEE.

39.

Speck

B. W.

(2003). Fostering collaboration among students in problem‐based learning. New Directions for Teaching and Learning, 2003(95), 59–65.

40.

Y. S.

Chen

H. R.

(2020). Social Facebook with big six approaches for improved students’ learning performance and behavior: A case study of a project innovation and implementation course. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.01166

41.

Y. S.

Chou

C. H.

Chu

Y. L.

Yang

Z. Y.

(2019). A finger-worn device for exploring Chinese printed text with using CNN algorithm on a micro IOT processor. IEEE Access, 7, 116529–116541. https://doi.org/10.1109/ACCESS.2019.2936143

42.

Y. S.

Lin

C. L.

Chen

S. Y.

Lai

C. F.

(2019). Bibliometric study of social network analysis literature. Library Hi Tech. https://doi.org/10.1108/LHT-01-2019-0028

43.

Y. S.

C. F.

W. C.

Lee

I. H.

Lin

C. P.

(2020). Applying deep learning algorithms to enhance simulation of large-scale groundwater flow in IOTS. Applied Soft Computing, 92, 106298. https://doi.org/10.1016/j.asoc.2020.106298

44.

Y.S.

S.Y.

(2020). Applying data mining techniques to explore users behaviors and viewing video patterns in converged IT environments. Journal of Ambient Intelligence and Humanized Computing. https://doi.org/10.1007/s12652-020-02712-6

45.

Sullivan

Bers

M. U.

(2013). Gender differences in kindergarteners’ robotics and programming achievement. International Journal of Technology and Design Education, 23(3), 691–702.

46.

Tarampi

M. R.

Heydari

Hegarty

(2016). A tale of two types of perspective taking: Sex differences in spatial ability. Psychological science, 27(11), 1507–1516.

47.

Werner

Denner

Campe

Kawamoto

D. C.

(2012, February). The fairy performance assessment: Measuring computational thinking in middle school. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (pp. 215–220). ACM.

48.

Wing

(2006). Computational thinking. Communications of the ACM, 49(3), 33–36.

49.

Wing

J. M.

(2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725.

50.

Zhan

Fong

P. S.

Mei

Liang

(2015). Effects of gender grouping on students’ group performance, individual achievements and attitudes in computer-supported collaborative learning. Computers in Human Behavior, 48, 587–596.