Educational Robots Improve K-12 Students’ Computational Thinking and STEM Attitudes: Systematic Review

Abstract

Due to the interdisciplinary nature of robotics, more and more attention has been paid to its effectiveness in the field of education in recent years. This systematic review evaluated existing studies in improving K-12 students’ computational thinking and STEM attitudes. Research articles published between 2010 and 2019 were collated from major databases according to six criteria, and 17 studies were eligible. A meta-analysis was conducted to evaluate the effectiveness of educational robots in terms of standardized mean differences (SMD) or mean differences (MD) of test scores as outcome measures. The overall effect size was medium (SMD = 0.46, 95% CI: 0.23–0.69). Subgroup analysis found that some groups to have better effectiveness. Specifically, the effect of STEM attitudes (SMD = 0.01) was smaller than computational thinking (SMD = 0.48). Educational robots had more significant effect on boys (MD = 0.39) than girls (MD = 0.27). The effect in primary school (SMD = 0.27) was higher than in middle school (SMD = 0.04), and the effect was great on short-term instruction with educational robots (SMD = 0.35). Based on these results, the study makes some recommendations for educators about strengthening the influence of educational robots on STEM attitudes, improving the persistence of their learning effects, and further exploring their application models.

Keywords

educational robots K-12 computational thinking STEM attitudes systematic review

Introduction

So far, with the gradual maturity of robot technology, various countries have paid more and more attention to the application and practice of robots in the field of education (Benitti, 2012). With their openness, friendly interaction, and popularity of open-source programming technology, educational robots have become an important tool for cultivating students’ innovative practice, analysis, and problem-solving skills, which are widely used in STEM, sociology, dance, music, art, and other disciplines. The current research focuses are mainly as follows: 1) exploring the role of robotics education in promoting the development of students’ computational thinking (Lee et al., 2011), collaboration (Shimada et al., 2012), critical thinking (Blanchard et al., 2010), spatial ability (Julià & Antolí, 2015), and other abilities; 2) discussing the basic concept of robotics education and its educational value, such as the fact that educational robots has not only been extolled for its role in learning but has also been identified as a pathway to broaden participation in STEM and STEM-related careers (Sheridan et al., 2013); 3) designing different robot teaching tools such as LEGO, the ROBOLAB software, and their application environment, including science and life technology courses (Shih et al., 2013), theme camps, or extracurricular activities combined with various robot competitions.

Many scholars and researchers strongly believe that students’ STEM attitudes are particularly important, and are closely related to their future investment in the STEM field (Bonvillian, 2002). At the same time, computational thinking, as part of the core competencies of the 21st century, is also playing an increasingly important role in K-12 education (V. Barr & Stephenson, 2011; Wing, 2006). Taking robot and related technology as the teaching project construction platform has been proven to be effective in cultivating compound, innovative, and personalized talents (Joshi, 2014; Nourbakhsh et al., 2015). This goal coincides with the idea of developing students’ STEM literacy and computational thinking, both aimed at fostering interdisciplinary and innovative thinking (Bers et al., 2014; Kopcha et al., 2017). Accordingly, it seems that robotics has become an excellent tool for teaching and learning and a compelling topic for students of all ages. Many studies have been devoted to studying the impact of educational robots on students’ STEM attitudes and computational thinking.

However, due to insufficient sample size, a non-standardized data analysis process, different learning environments, and other practical reasons, the influence of robotics education on the learning of K-12 education students has not been unified in the existing research. Studies have shown that educational robots do not have a positive effect on student learning outcomes (Nugent et al., 2010), and may even have a potentially negative impact on students’ STEM learning (Barak & Assal, 2018). Furthermore, the effect of educational robots on students’ STEM attitudes and computational thinking may be affected by a variety of variables, such as gender, teaching experiment period, grade level, and so on.

Therefore, this study attempts to answer the following two key research questions:

Compared with traditional teaching methods, is the educational robots more helpful to improve students’ computational thinking and STEM attitudes due to its friendly human–computer interaction and other characteristics?

To what extent are educational robots outcomes moderated by: (a) gender, (b) teaching experiment period, and (c) grade level?

In order to solve the above problems, the study used meta-analysis methods to explore the impact of educational robots on students’ STEM attitudes and computational thinking, as well as the differences in impact on different dimensions, including gender, teaching experiment period, and grade level. The purpose of the research was to analyze the relevant research published in the past decade, grasp the development direction of educational robots, clarify the road map for future research, and provide a reference for the future development of robotics education.

Literature Review

Educational Robots

Robotics allows students to learn not only about STEM concepts but also about STEM’s interdisciplinary nature, encouraging students to work collaboratively (Yuen et al., 2014), so it has been used as an emerging approach to engaging K-12 students in learning science, technology engineering, and mathematics, which has tremendous potential to improve classroom teaching (Benitti, 2012; Kim et al., 2015; Kucuk & Sisman, 2017). One of the reasons for the growing use of robots in education is that students can directly interact with a robot and observe the immediate effffects of their interactions on the robot’s behavior (Bers et al., 2014; Stoeckelmayr et al., 2011). And robotics is widely used as an authentic and kinesthetic tool to improve children’s problem-solving skills, reinforcing science applications and concepts (Karp & Maloney, 2013). Educational robots is often presented as a platform for achieving the following three main objectives (Altin & Pedaste, 2013; Chung et al., 2014; Kim et al., 2015; Sullivan & Bers, 2016; Ucgul & Cagiltay, 2014): (1) to teach science, technology, engineering, and mathematics; (2) to develop broad learning skills such as scientiﬁc inquiry, engineering design, problem-solving, creative thinking, and teamwork; (3) to foster students’ motivation to engage in science and technology, to connect the lessons with the real world, and to reduce psychological or cultural barriers in dealing with these subjects, for example among girls or students from underprivileged communities. Gradually, the robot becomes a tool, at the service of teachers and students, to develop skills and promote the acquisition of content and competencies practically in all curricular areas (Mubin et al., 2013).

STEM Attitudes and Measurement

According to psychological research, attitudes can be viewed as individual beliefs about the attributes of a particular object (Fishbein & Ajzen, 1975), which are made up of emotion, cognition, and behavioral tendency (Myers, 1993). Therefore, STEM attitudes can be considered a three-part response generated by learners in the STEM learning process. To further clarify the connotation of STEM attitudes in the field of education, many researchers have carried out relevant studies. According to the Concerns-Based Adoption Model (Anderson, 2002) and the Taxonomy of Education Objectives Handbook II (Kraftwohl et al., 1964), Mahonry (2009) proposed that STEM attitudes can be divided into four dimensions: consciousness, perception ability, value, and expectation. Guzey determined that the student’s STEM attitude scale should include three dimensions: STEM learning, STEM employment, and STEM social impact (Guzey et al., 2014). Eric Wiebe used S-STEM survey to measure student attitudes toward science, mathematics, engineering/technology, and 21st century skills (Unfried et al., 2015). This scale has been widely recognized and applied in domestic and foreign research.

In a STEM attitudes instrument, STEM attitudes, defined by Eric Wiebe (Unfried et al., 2015), is a composite of both self-efficacy and expectancy-value beliefs in STEM (Eccles & Wigfield, 2002). Self-efficacy is the belief in one’s ability to complete tasks or influence events that have an impact on one’s life (Bandura, 1986). Numerous researchers have found links between domain-specific self-efficacy and academic outcomes (Pajares & Usher, 2008; Simon et al., 2015). Research has also shown that youths’ goals for STEM learning, their self-efficacy, and the value that they assign to STEM tasks and activities are likely to influence their level of engagement (National Research Council, 2007). Expectancy-value theories posit that individuals regularly assess the likelihood of attaining specific goals and appraise the value gained or lost from such attainment (Eccles & Wigfield, 2002). The theory helps frame both self-efficacy in terms of expectancies of success in a particular academic domain and outcome expectancy in terms of the value of this academic subject area to future goals. Previous research on STEM education found that students’ attitudes toward STEM are critically associated with their later engagement in STEM fields. Similarly, outcome expectancies for success and the value a student associates with them have been found to directly influence performance, persistence, and choice in (academic) tasks (Eccles & Wigfield, 2002; Guo et al., 2015).

Computational Thinking and Measurement

In the literature there are multiple definitions of computational thinking (CT) and several suggestions about which skills and abilities are relevant to it. CT was initially defined by Seymour Papert as the ability to think computationally (Papert, 1996). The International Society for Technology in Education defined CT as a problem-solving process, which includes asking questions, logically analyzing data, abstracting data, formulating solutions, analyzing and implementing possible solutions, and promoting solutions to problems (D. Barr et al., 2011; Ioannou et al., 2011). At first, CT is the process of thinking like a computer scientist when encountering problems to be solved (Wing, 2006). Then Jeannette M. Wing defined CT as the thought process involved in formulating problems and their solutions, so that the solutions are represented in a form that can be effectively carried out by an information-processing agent (Wing, 2011). Later, five components of CT were outlined by Selby and Woollard (2013): decomposition, generalization, abstraction, evaluation, and algorithmic thinking. These five factors are explained as follows: (1) decomposition is the skill of breaking complex problems into simpler ones that may be more easily solved; (2) generalization is replacing multiple entities which perform similar functions with a single construct to reduce complexity; (3) abstraction is removing characteristics or attributes from an object or entity in order to reduce it to a set of fundamental characteristics, which leaves out the irrelevant details to reduce complexity; (4) evaluation is identifying which problem-solving method is appropriate; (5) algorithmic thinking is a way to identify the steps needed to solve a problem (National Research Council, 2010; Selby, 2012; Selby & Woollard, 2013; Wing, 2011). In conclusion, CT is human thought, focused on formulating and solving problems from a computational viewpoint. But CT is considered not just limited to the field of computer science, as it can be transferred to any domain, such as art, mathematics, biology, engineering, and so on (National Research Council, 2010). The Carnegie Mellon University Center for Computational Thinking greatly emphasizes the importance of CT for all disciplines by stating that without CT skills, it is almost impossible to study in any academic field. And Jeannette M. Wing stated this vision for the 21st century: CT will be a fundamental skill used by everyone in the world (Wing, 2008).

Educational Robots and STEM Attitudes

It is widely recognized that educational robots have significant positive effects on students’ STEM learning. A functional learning environment in the context of educational robots requires the application of multiple STEM principles through building and programming a robot to solve an authentic problem (Kopcha et al., 2017). Furthermore, as they see how knowledge becomes functional during problem-solving, this environment can help motivate students because they contextualize learning. Barak and Assal (2018) found that all students in the experimental class showed high motivation to learn robotics and STEM subjects, in which robotics provides a very rich and attractive learning environment for STEM education. Robots used in the teaching of programming can provide interesting opportunities and authentic practice situations with immediate feedback (Spolaôr & Benitti, 2017). Research suggests that tangible learning experiences improve motivation and overall interest in STEM-related subject matter (Bers et al., 2014; Ucgul & Cagiltay, 2014). These experiences can be obtained by programming a tangible robot, as robotics has long been leveraged as a more concrete approach to teaching computer programming (Ching et al., 2019). Similarly, Sáez-López et al. (2019) emphasize the effectiveness of introducing robotics and visual programming based on active methodologies in primary education to gain a high degree of student participation.

On the contrary, robots may have a potentially negative effect on students’ STEM learning. When young children lack the relevant knowledge in STEM, it must be realized that there is a danger of engaging students in activities that could be considered “doing without learning” (Barak & Assal, 2018). Barak and Assal found that among 32 junior high school students, only 20% of the students stated that they took on the highest-level project, while the others chose projects that were close to the exercises or problem-solving tasks they had already completed, which may indicate the inapplicability of the robotics course in classroom teaching for most students’ learning level. It has been also pointed out that learners often require additional instruction to gain some knowledge and skills related to robotics before being able to cope with complex projects, which may increase the difficulty of teaching and students’ understanding. Under these circumstances, this study intends to use meta-analysis to comprehensively analyze 17 experimental or quasi-experimental research papers from 2010 to 2019, focusing on exploring the effect of educational robots on students’ STEM learning attitudes, as well as differences in the effects on different dimensions.

Educational Robots and Computational Thinking

Educational robots is being introduced in many schools as an innovative learning environment, enhancing and building higher-order thinking skills and abilities, and helping students solve complex problems (Blanchard et al., 2010). Educational robots is also seen as a tool for advancing CT, coding, and engineering (Eguchi, 2014a, 2014b). Eguchi (2014a, 2014b) uses Piaget’s constructivism as a lens for the study of CT through robotics. The theory posits that knowledge is not passively received by a student, but it is actively built up in the mind of the learner while one interacts with the environment and with physical artifacts. Piaget holds the view that knowledge is gained merely through experience (Piaget, 1964). Educational robots can be used as a tool that offers opportunities for students to engage in and develop CT skills, because children can directly interact with a robot and observe the immediate effects of their interactions on the robot’s behavior (Bers et al., 2014; Kazakoff & Bers, 2012; Stoeckelmayr et al., 2011) And robots are used in programming education, as abstract concepts can be implemented visually and physically so that students can more easily understand computer-science concepts (Noh & Lee, 2019). Many studies have thus shown that education in a programming language and programming a robot does foster students’ abstract understanding and CT.

Despite the sufficient studies on CT and educational robots, there has been very little work connecting the two (Berland & Wilensky, 2015; Ioannou & Makridou, 2018). This paper investigates the possible impacts that the implementation of educational robots activities might have had on the development of students’ computational skills in the past ten years.

Moderator Variables

Gender

In the present study, it has been proved that girls’ STEM interests tend to be lower and decline more rapidly than boys’ during high school (Brotman & Moore, 2008). There is also evidence that the proportion of young men interested in STEM remains unchanged whereas the proportion of young women decreases (Sadler et al., 2012). Jackson et al. (2019) found that the interest gains following participation in the robotics lessons were the same for both gender and lesson type. However, some mixed or anecdotal evidence has showed the success of robotics programs in influencing girls’ engineering perceptions (Hendricks et al., 2012). Moreover, some other findings observations regarding robotics and gender noted that educational robots might contribute to decreasing gender gaps in STEM fields (Diekman et al., 2011; Terry et al., 2011). By using expectancy-value theory, evidence of gender disparities in STEM fields has been parsed (Jackson et al., 2019), which may also explain the different effects of robots on the interests of students of different genders. Therefore, the study speculated that gender may be a moderator of the relationship between educational robots and STEM attitudes.

In terms of CT, most research has shown that boys are traditionally more familiar with technology (Papastergiou, 2009) and require less time to reach a given level of CT than girls do (Atmatzidou & Demetriadis, 2016). Moreover, boys are generally higher than girls in these aspects: computer experience (Papastergiou, 2009), participation in programming and assembling robotics activities (Rusk et al., 2008), and attitude (Baser, 2013). However, some studies found that the difference in CT was not significant between girls and boys (Durak & Saritepeci, 2018; Noh & Lee, 2019). Specifically, Atmatzidou and Demetriadis (2016) reported that after an 11-week robotics-supported lesson, gender difference was not statistically significant in the final scores of CT.

In conclusion, as gender is an important variable in robotics education, the study systematically examined gender differences in educational robots courses in this study.

Grade Level

Several studies showed that children as young as four years old were able to engage in CT activities using a robotics curriculum (Bers et al., 2014; Kazakoff & Bers, 2012; Sullivan & Bers, 2016). Researchers have also strongly argued for the importance of integrating CT into the education of students starting from early childhood (Angeli & Valanides, 2019; Bers et al., 2014; Botički et al., 2018). Age relevant differences also appear when analyzing students’ scores in the various specific dimensions of the CT skills model (Atmatzidou & Demetriadis, 2016). At the same time, studies have shown that students’ attitudes toward STEM-related fields decline as grade levels increase (George, 2006), and the effect of educational robotics is different between students (Shan et al., 2019). However, the lack of age-appropriate robotics curricula presents a barrier to the application and spread of educational robots in K-12 education (D. Barr et al., 2011; Khanlari, 2016; Kopcha et al., 2017). As a result, the study set grade level as a variable of the relationship among educational robots, STEM attitudes, and CT.

Teaching Experiment Period

The learning time has been found to be related to achievement in subject areas (Hofferth & Sandberg, 2001; Parkerson et al., 1984; Reynolds & Walberg, 1992). Previous research also suggest a causal relationship between learning time and learning effect (Fredrick & Walberg, 1980; Parkerson et al., 1984; Slavin, 1995). For instance, CT skills in most cases need time to fully develop (students’ scores improve significantly towards the end of the activity; Atmatzidou & Demetriadis, 2016). Nugent et al. (2010) found that longer intervention led to significantly greater learning in subjects compared to a control group not receiving the instruction, whereas short-term intervention primarily impacted students’ attitude and motivation. In this study, learning time refers to teaching time or learning time provided by the school, which is an external variable (Quilter & Chester, 2001). The purpose was to determine whether experimental teaching periods incorporating robotics influences students’ learning effects.

Methods

In the study, Review Manager 5.3 was used for meta-analysis. The meta-analysis procedures followed several key steps: (1) choose all possible keywords and screen potential studies using preset criteria; (2) code all qualified studies; (3) calculate effect sizes for further combined analyses; and (4) carry out comprehensive statistical analyses between effects and study features.

Literature Search

In order to limit the number of papers included and to collect as many studies as possible, this study selected “Web of Science”, “ERIC”, “IEEE”, “Science Direct”, “Springer Link”, and other domestic and foreign journals, as well as nearly 10 years of proceedings of the well-known International Conference on Educational Technology, “AERA”, “AECT”, and other papers as the scope of literature search. The period searched was limited to January 1, 2010–December 31, 2019. The retrieval method combined subject words and free words. The specific keywords included “educational robots”, “teaching robotics”, “LEGO”, “primary school”, “middle school”, “K-12”, “STEM attitude”, and “computational thinking”.

Eligibility Criteria

In order to further narrow the scope of research, and to ensure the consistency of the studies included and the rigor of the research results in the meta-analysis research, the following six criteria had to be met: (1) the research theme must be the influence of educational robots on students’ CT or STEM attitudes; (2) the type of research must be experimental or quasi-experimental research, excluding secondary data analysis and literature review; in addition, the research must include the experimental group and the control group, or the experiment includes pre-test and post-test; (3) the educational robots in the study is only used as a teaching tool to assist classroom teaching; (4) the study reports sufficient statistical information to estimate the effect value of the student’s CT or STEM attitudes, such as the average (M) and standard deviation (SD), t value, p value, and other data. If the study only provided descriptive statistics or p-values and no appropriate statistics, it was excluded; (5) the research subjects must be concentrated on primary and middle school students; (6) the research publication date must be between 2010 and 2019.

Initially, 1,411 articles were selected through keywords and abstracts, and then the above six criteria were used for step-by-step selection. Finally, as can be seen in Figure 1, 17 studies were included for further analysis.

Figure 1.

Flow of Study Analysis Through Different Phases of the Meta-Analysis.

Literature Coding

In order to analyze the effect value of educational robots on the learning skills of elementary and middle school students, it was necessary to encode the data of the 17 original studies included. The contents of the paper coding included author, year of publication, sample size, gender of subject, experimental period, grade level, and outcome variables (see Table 1). In this study, the sample size of the study was divided into three types: the small scale is less than 50 people, the medium scale is 50 to 100 people, and the large scale is more than 100 people. The grade level was divided into primary and middle school. The experimental period was divided into three periods: less than 4 weeks, 4 weeks to 11 weeks, and more than 11 weeks. And the outcome variables included CT and STEM attitudes. Among them, the effect value of the paper was coded according to each independent sample. If a paper reported multiple independent samples at the same time, it was coded separately to produce multiple independent effect quantities.

Table 1.

Literature Coding.

Author (year)	Sample size (CG/EG)	Gender (yes/no)	Grade level	Experimental period	Outcome variables
Atmatzidou & Demetriadis (2016)	75/75	yes	middle school	11 weeks	CT
Angeli & Valanides (2019)	13/13	yes	primary school	1 week	CT
Angeli & Valanides (2019)	12/12	yes	primary school	1 week	CT
Ching et al. (2019)	14/14	no	primary school	8 weeks	STEM attitudes
Nugent et al. (2010)	141/147	no	middle school	1 week	STEM attitudes
Sáez-López et al. (2019)	36/93	no	primary school	2 months	STEM attitudes
Welch & Huffman (2011)	58/58	no	middle school	3 months	STEM attitudes
Leonard et al. (2016)	20/20	no	middle school	3 years	STEM attitudes
Barak & Assal (2018)	32/32	no	middle school	2 weeks	STEM attitudes
Shih et al. (2013)	53/53	no	primary school	3 months	STEM attitudes
Gaudiello & Zibetti (2016)	25/25	no	primary school	8 weeks	STEM attitudes
Jackson et al. (2019)	153/278	no	middle school	1 year	STEM attitudes
Berland & Wilensky (2015)	44/34	no	middle school	2 weeks	CT
Merino-Armero et al. (2018)	27/26	yes	primary school	1 weeks	CT
Noh & Lee (2019)	155/155	yes	primary school	11 weeks	CT
Ioannou & Angeli (2016)	127/113	no	primary school	16 weeks	CT
Constantinou & Ioannou (2018)	16/16	no	middle school	3 months	CT
Atmatzidou & Demetriadis (2014)	35/35	no	middle school	11 weeks	CT

Publication Bias

Meta-analysis usually detects the publication bias of sample data. Publication bias means that the published literature does not systematically and comprehensively represent the overall research that has been completed in the field (Rothstein et al., 2005). Publication is selective to some extent, and if the positive results obtained from the study are statistically significant, it is easier or faster to publish (Thornton & Lee, 2000). In the study, in order to ensure the scientific nature and reliability of the research results, a funnel plot was used to further test publication bias. Funnel plots are mainly identified by visual observation of publication bias, which take effect size as abscissa, ordinate as standard error, and two oblique lines as 95% confidence interval (Schulz et al., 1995). Ideally, 95% of the points should fall in this interval, and the dispersion obtained by small samples is large. Therefore, the points are usually at the bottom of the funnel plot, while the dispersion of large samples is small and at the top. Under normal circumstances, it should be small at the top and large at the bottom.

Quality Assessment

Using the Cochrane Handbook for Systematic Reviews of Interventions, researchers assessed the risk of bias in all the included studies (Noyes et al., 2008), including: (1) whether the randomization method was used; (2) whether to use allocation to hide; (3) whether the subjects were blinded; (4) whether to use blind method for the evaluation of outcome indicators; (5) whether the results were fully reported; (6) whether there was selective reporting of results; and (7) whether there were other sources of bias. Researchers cross-checked and discussed with other researchers (the third party) in case of any differences.

The bias risk assessment results of the included studies are shown in Figures 2 and 3. (1) four studies were non-probabilistic and intentional in this research. While the others were randomized controlled trials, only two studies pointed to specific randomized methods. (2) except for one study, the method of allocation concealment was not mentioned in the studies. (3) considering the characteristics of the teaching method itself, we believe that none of the studies adopted the blind method, but the outcome indicators will not be affected by the lack of the blind method. (4) none of the 15 studies included mentioned the absence of data, and only two reported low attrition and survey response rates, which may affect the final survey results. (5) all studies were assessed as having a low risk of selective reporting. (6) one study suggested that the non-randomized controlled trials may affect the results, and the other studies did not mention whether there was a significant risk of bias.

Figure 2.

Risk of Bias Graph.

Figure 3.

Risk of Bias Summary.

Results

Effect Size Calculations

Effect size is the most important statistical value of meta-analysis, which represents the standardized measurement and can indicate the strength and direction of the relationship between variables. Correlation coefficients are used as effect size to comprehensively integrate the relationship between educational robots and students’ learning effect. Some of the papers do not directly report the correlation coefficient but report F-value and t-value, referring to the conversion method recommended by Wang et al. (2013), that is,

r = {[t^{2} /(t^{2} + df)]}^{1/2},df = n_{1} + n_{2} - 2; r = {[F /(F + df)]}^{1/2},df = n_{1} + n_{2} - 2

Then the Pearson correlation coefficient r value of each study was transformed into Fisher’s Z. The basic data was sorted out, and further analysis was carried out.

Model Selection and Heterogeneity

The effect amount of meta-analysis includes two parts: the real effect amount and the error. The error will cause the effect amount to be partially false. If the false part exceeds the statistical range, the effect amount obtained is heterogeneous, and heterogeneity testing is required. Two tests are discussed here because Revman only provides the Q test and I² test. The Q test is based on the test of total variation and mainly refers to the P-value. If the P-value was larger than 0.1, there was no heterogeneity; if the P-value was less than 0.1, there was heterogeneity. The I² test mainly reflects the proportion of the real variation of the effect quantity in the total variation. According to previous views, 25%, 50%, and 75% of the I² value can be regarded as the limit of low, medium, and high heterogeneity, respectively (Higgins et al., 2003). If there was no significant heterogeneity difference among the results, the fixed effect model was adopted. On the contrary, the random effect model was used for meta-analysis. It is necessary to further analyze the sources of heterogeneity, using subgroup analysis or sensitivity analysis to explore the sources of heterogeneity (Borenstein et al., 2009). According to the above judgment criteria, Q = 138.28, P = 0.000 < 0.001, and the I² value is 86%, indicating that there is high heterogeneity in this study.

Overall Effect Size

The overall effect of all the studies according to the learning effect of the students is shown in Figure 4. Both the fixed effect model and the random effect model have reached a statistically significant level (P < 0.00001). According to the homogeneity test results above, the random effect model was selected for analysis in this study. The overall effect was 0.46 (SMD = 0.46, 95% CI: 0.23–0.69, 17 studies, effective sample size was 2,757 students). According to the classification standard of the effect value proposed by Cohen, an effect value of about 0.2 is generally considered to have a small impact, that of about 0.5 has a moderate effect, and 0.8 a large effect (Cohen, 1977). It can be seen that the overall effect of educational robots on students’ learning has a medium promotion effect, which helps to improve the learning effect of students.

Figure 4.

Forest Plot of Overall Effects on Students’ Learning Effects.

Sensitivity Analysis

This study examines the heterogeneity of the relationship between educational robots and students’ learning effects. As presented in Figure 4, the heterogeneity test showed that I² = 86% and P < 0.000, indicating that there was considerable heterogeneity, so it should be eliminated before analysis. In this study, studies with significant differences were deleted, and changes of combined effect size were observed. After removing two studies’ data (Berland & Wilensky, 2015; Merino-Armero et al., 2018), the heterogeneity of students’ learning effects was reduced to 58%, p = 0.002 < 0.00001. This indicates that regardless of the degree of heterogeneity, the learning effects of students were significantly related to the educational robots. Compared with previous studies, the heterogeneity was greatly reduced, so the analysis results can be adopted.

Subgroup Analysis

After the sensitivity analysis, the heterogeneity test showed that there was still moderate heterogeneity in the effect value of educational robots on the overall effect of students, so there may be adjustment variables. Subgroup analysis is one of the most commonly methods to deal with heterogeneity. Generally, it can be grouped according to study protocol, study quality, race, publication time, or other factors. Therefore, this study further analyzed the sources of heterogeneity through subgroup analysis, focusing on the adjustment effect of the type of outcome variable, the research subject’s gender, the experimental period, and the grade level on the student’s learning effects.

The Impact of Educational Robots on Different Learning Effects

In order to further explore the impact of educational robots on different learning effects, this study divided the outcome variations into CT and STEM attitudes. Meta-analysis results showed that the influence of educational robots on students’ CT and STEM attitudes was quite different. Specifically, the influence of educational robots on students’ CT (SMD = 0.48, 95% CI: 0.32–0.64, I² = 24%, 6 studies, effective sample size was 930 students) was significantly higher than the effect on STEM attitudes (SMD = 0.01, 95% CI: -0.08–0.10, I² = 17%, 9 studies, effective sample size was 2,675 students), as shown in Figure 5.

Figure 5.

The Effect of Educational Robots on Students’ Different Learning Effects.

The Impact of Educational Robots on Students of Different Genders

Due to the lack of research on the impact of educational robots on STEM attitudes of students of different genders, only four studies analyzed the gender differences in the impact of educational robots on students’ CT. Subgroup analysis showed that educational robots have a higher impact on the CT of boys (MD = 0.39, 95% CI: 0.32–0.47, I² = 43%, 4 studies, effective sample size was 493 students) than girls (MD = 0.27, 95% CI: 0.14–0.40, I² = 53%, 4 studies, effective sample size was 440 students), as shown in Figure 6.

Figure 6.

The Effect of Educational Robots on Students of Different Genders.

This is consistent with previous research results. Eccles’s (1994) expectancy-value model maintains that cultural milieu plays a role in the development of everyone’s interests, goals, and expectations in academic and vocational pursuits; here, gender stereotypes are particularly relevant. In adolescence, girls report hearing sexist comments about STEM abilities (Brown & Leaper, 2010) and hold the most negative STEM attitudes. In society, there is not only a lack of positive intervention, but also a large number of negative effects. There is a “negative stereotype” for girls, which causes them to feel less able to study STEM. In addition, because boys are traditionally more familiar with technology, the advantages of boys in science and technology are gradually highlighted, and it takes them less time to reach a given level of CT than girls.

The Impact of Educational Robots on Students in Different Grade Levels

This study analyzed the differences in the impact of educational robots on student learning effects in different grade levels. The grade level was divided into two subgroups: primary school and middle school. Meta-analysis results showed that educational robots had a higher impact on primary school students’ learning effects (SMD = 0.27, 95% CI: 0.08–0.45, I² = 43%, 7 studies, effective sample size was 1,015 students) than middle school students (SMD = 0.04, 95% CI: –0.08–0.17, I² = 46%, 8 studies, effective sample size was 2,488 students), as shown in Figure 7.

Figure 7.

The Effect of Educational Robots on Students’ Different Grade Levels.

According to the STEM pipeline theory, the development process of students from the initial acceptance of STEM education to the end of their STEM career is likened to a narrower and narrower pipeline (Maltese & Tai, 2011). Statistics from the National Longitudinal Study of Education in the United States show that brain drain occurs at almost every junction in the pipeline, and college entrance pressure reduces students’ positive attitude towards things other than exam content (Cannady et al., 2014). The self-determination theory also holds that the less stress the students suffer, the better their autonomy, concentration, and interest involvement are (Ryan & Deci, 2000). The level of difficulty of studying and the pressure of entering college also influence the effect of robotics education on students’ learning. Studies have shown that educational robots have positive and significant effects at each stage of K-12, but the effect is the best in primary school (Zhou et al., 2019).

In addition, primary school robotics education is mainly focused on stimulating interest and cultivate creativity, while secondary school robotics education is mainly focused on carrying out robot competitions (Wang et al., 2017). Primary school students’ thoughts are more active, their imagination is richer, and their academic burden is relatively lighter. In recent years, many STEM laboratories established in elementary schools have used robots to conduct STEM courses, developed students’ CT through programming and other tasks, and regularly held activities such as summer camps with the theme of robots. For middle school students, some schools and parents attach importance to children’s performance in competitions and neglect the advantages of robotics education in cultivating students’ CT, problem-solving ability, and other aspects (Hou, 2016). Therefore, compared with the middle school stage, the primary school stage has lower learning content, activity forms, and learning requirements, and the primary school students have higher learning interest and participation. This shows that elementary school is an important stage for students to develop CT and STEM attitudes by using robots.

The Effect of Educational Robots on Different Experimental Periods

In order to explore the influence of educational robots on students’ learning effects in different experimental periods, this study divided the experimental periods of the sample literature into three groups of less than 4 weeks, 4–11 weeks, and more than 11 weeks for analysis. The combined effect values of each group were shown in the Figure 8. The effect of educational robots in different experimental periods varied, indicating that the influence of educational robots on the learning effect of students had a certain correlation with the experimental period. Specifically, studies with an experimental period of less than 4 weeks had the highest effect value (SMD = 0.35, 95% CI: 0.15–0.55, I² = 16%, 3 studies, effective sample size was 516 students). For studies with an experimental period of 4–11 weeks, the effect value was 0.28 (SMD = 0.28, 95% CI: 0.10–0.47, I² = 29%, 6 studies, effective sample size was 821 students). For studies with an experimental period of more than 11 weeks, the effect value was 0.04 (SMD = 0.04, 95% CI: –0.13–0.21, I² = 68%, 6 studies, effective sample size was 2,268 students).

Figure 8.

The Effects of Educational Robots on Different Experimental Periods.

With the extension of the teaching experiment period, the influence of educational robots on the learning effects of students gradually weakened, indicating that too long an experimental period will reduce the application effect of educational robots. The reason for this phenomenon may be that with the extension of the application time of educational robots, students’ familiarity with robots increases, freshness is greatly reduced, learning enthusiasm lessens, and participation is reduced. So the improvement of learning effects does not occur. This is basically similar to the conclusions of existing studies (Chauhan, 2017).

Publication Bias

The funnel plot for the theoretical scores of the 17 studies is shown in Figure 9. The funnel plot’s shape is symmetrical and basically distributed on both sides of the total effect. There was no significant publication bias indicated in the main analysis, and the overall results of our meta-analysis can be acceptable. However, a small number of points fall outside the two oblique lines, indicating that there may be heterogeneity between the included studies. This conclusion is consistent with the heterogeneity test results and has been further analyzed in combination with the forest map data.

Figure 9.

Funnel Plot of the Studies Included.

Discussion

This study made a quantitative analysis of 17 studies on the influence of educational robots on students’ learning effects by using meta-analysis and made an objective evaluation on the effect of educational robots. In general, the combined effect value of the influence of educational robots on students’ learning effect was 0.46, which indicated that educational robots had a medium degree of positive influence on students’ skill development and can better promote students’ skill development. On the other hand, the data analysis of this study also showed that robots need to be improved in promoting middle school students’ learning, training students’ STEM attitudes, and maintaining the impact of educational robots on students’ learning effects.

Improve the Role of Educational Robots in Promoting Students’ STEM Attitudes

From the perspective of the impact of educational robots on students’ different learning skills, robot teaching has a moderate impact on the development of CT, but it cannot significantly improve students’ STEM attitudes. This is consistent with previous research results, which requires further attention. DeWitt et al. (2013) found that students’ interest and attitudes towards STEM decrease with age. In order to fully tap the potential of educational robots to improve students’ attitudes toward STEM, the following measures can be taken. First, set up experimental courses to train students’ operational ability. For example, almost all robot courses in the United States have experimental classes. Students use various tool platforms to assemble physical robots or program and debug robot programs. Studies have shown that the substantiality of, interaction with, and interest in robots can transform obscure STEM concepts into practical problems that can be operated, which greatly improves students’ attitudes towards STEM (Carstro et al., 2018). The classroom is guided by practical problems or tasks, allowing students to learn modeling and algorithms, and to study how algorithms are applied in these systems. In this process, students’ interest towards STEM is stimulated and they experience a sense of success. In this way, students’ STEM interest and STEM literacy, such as CT and complex problem-solving skills, can be cultivated. Second, focus on the connection between the robot curriculum and the real STEM field. When carrying out STEM-related courses based on robots, the engineering and technical problems in real life should be simplified and modeled. For example, robots are designed to provide reconnaissance and assistance in natural and man-made disasters such as the deep ocean accidents, earthquake-stricken areas, and war zones. In addition, the teaching content is related to STEM career content through gamification, such as fun programming to understand related careers in engineering, eliminating students’ stereotypes of scientists, technicians, and engineers.

Strengthen the Persistence of the Influence of Educational Robots on Learning Effects

The results show that the effect of educational robots on students’ learning decreases with the extension of the experimental period, which indicates that in the process of the application and practice of the educational robot, its effect on students’ learning needs to be further strengthened. To do this, first introduce emerging artificial intelligence technology into robotics education. With the development of technology, the combination of educational robots and other emerging technological achievements, such as virtual reality, 3D printing, voice interaction, and so on, can deepen students’ understanding of related theories and technologies and enhance students’ interest and experience. In addition, the combination also brings more opportunities for the application of robot-assisted education in the future, and its educational auxiliary functions also tend to be diversified and humanized (Wu & Li, 2018). But students’ acceptance and ease of use of robot technology should be taken into account in practice, and blind application of emerging technologies will increase students’ cognitive load and reduce their interest in learning (Gong et al., 2020). Second, design continuous and hierarchical learning content for K-12 students at all stages. As students move to higher grade levels, based on the zone of proximal development theory, robotics education should gradually increase the difficulty of learning tasks to ensure that teaching content is comparable to students’ current cognitive level (Chaiklin, 2003) and that learning is always built on what students have learned before, rather than spending a lot of valuable instructional time on repetition. Then students continue to challenge themselves in the process of completing tasks, gaining a sense of freshness and accomplishment.

Actively Explore the Application and Practice of Educational Robots in Middle Schools

The integration of artificial intelligence and education will gradually become a future trend and an important driving force for education, and further strengthening the application of artificial intelligence in middle schools is an effective way to cultivate students’ skills for the 21st century (Khanlari, 2013). In response, some actions should be taken. First, strengthen the characterization of robotics education. For learners with strong cognitive ability, the engineering design education concept can be integrated in practice; for learners with weak cognitive ability, the education concept of gamification learning can be integrated. In addition, because attitudes towards STEM learning in middle school decreases for girls, robot technology can be used as an intermediary to integrate art and STEM subjects, through design tasks that enhance students’ STEM attitudes and self-confidence. Second, carry out student-centered team tasks. With the enhancement of middle school students’ autonomy and initiative, group activities should be carried out and tasks of varying difficulty should be assigned according to students’ abilities, so as to stimulate their creativity and enhance their ability to cooperate. Third, enhance teacher training with the theme of robotics education. Lack of suitable teachers will become one of the important factors affecting the development of robotics education (Williams et al., 2012), and teachers’ information literacy also has a direct impact on the effect of robot-assisted teaching (Smith & Sivo, 2012). It is worth noting that teachers’ ability with regard to robot teaching not only refers to their ability to use robots but also includes their ability to flexibly use robot teaching to improve the classroom environment, enhance students’ learning effectiveness, and build a good classroom learning ecology. Therefore, in terms of teacher training for teachers to improve their robot teaching application capabilities, the course content should be oriented to practical knowledge and limit unnecessary theoretical knowledge, increasing interaction between teachers and students, and classroom practice in order to ensure that teachers’ teaching ability can be fully exerted.

Conclusion and Further Studies

This study used the meta-analysis method to investigate the impact of educational robots on students’ CT and STEM attitudes in the past ten years and put forward some suggestions to improve the application effect of educational robots, hoping to provide references for the research of educational robots in the field of education. But there are still some deficiencies in this study, such as the number of studies examined is insufficient, and the moderator variables such as teaching experiment period and grade level have not been divided in more detail. Subsequent studies can be combined with more relevant studies to analyze the effects in more detail. In addition, in view of the influence of other latent variables such as teaching methods, learning environment, and robot type on students’ learning effects, further attention and in-depth analysis are needed. In the future, there will be more empirical research to verify the improvement of students’ multiple abilities. The research theme and conclusions of this research will be further supplemented and refined.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ronghua Luo

Author Biographies

Yanjun Zhang had been awarded Phd by the College of Education of Zhejiang University in China, and had finished the post-doctoral research in the College of Teacher Education of Zhejiang Normal University. Dr Zhang is now an associate professor in Zhejiang Normal University, focusing on STE(A)M education, international and comparative education, with a new research of “Comparison on the STEM Attitudes of Zhejiang and Jiangsu High School Students”.

Ronghua Luo is a postgraduate student in College of Teacher Education, Zhejiang Normal University, whose research focuses on comparative education and STEM education.

Yijin Zhu is a postgraduate student in College of Teacher Education, Zhejiang Normal University, who mainly studies comparative higher education.

Yin Yuan is a postgraduate student in College of Teacher Education, Zhejiang Normal University who mainly engages in doing comparative education research.

References

Altin

Pedaste

(2013). Learning approaches to applying robotics in science education. Journal of Baltic Science Education, 12(3), 365–377. http://oaji.net/articles/2015/987-1425808570.pdf

Anderson

S. E.

(2002). Understanding teacher change: Revisiting the concerns based adoption model. Curriculum Inquiry, 27(3), 331–367. https://doi.org/10.1111/0362-6784.00057

Angeli

Valanides

(2019). Developing young children's computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy. Computers in Human Behavior, 105,105954. https://doi.org/10.1016/j.chb.2019.03.018

Atmatzidou

Demetriadis

(2014). How to support students' computational thinking skills in educational robotics activities. In D. Alimisis & M. Moro (Eds.), International workshop teaching robotics (pp. 43–50). ResearchGate. https://www.researchgate.net/publication/265208083

Atmatzidou

Demetriadis

(2016). Advancing students’ computational thinking skills through educational robotics: A study on age and gender relevant differences. Robotics and Autonomous Systems, 75, 661–670. https://doi.org/10.1016/j.robot.2015.10.008

Bandura

(1986). Social foundations of thoughts and action: A social cognitive theory. Prentice Hall. https://doi.org/10.1017/S0813483900008238

Barak

Assal

(2018). Robotics and STEM learning: Students’ achievements in assignments according to the p3 task taxonomy—Practice, problem solving, and projects. International Journal of Technology and Design Education, 28(1), 121–144. https://doi.org/10.1007/s10798-016-9385-9

Barr

Harrison

Conery

(2011). Computational thinking: A digital age skill for everyone. Learning & Leading with Technology, 38(6), 20–23. https://files.eric.ed.gov/fulltext/EJ918910.pdf

Barr

Stephenson

(2011). Bringing computational thinking to K-12: What is involved and what is the role of the computer science education community? ACM Inroads, 2(1), 48–54. https://doi.org/10.1145/1929887.1929905

10.

Baser

(2013). Attitude, gender and achievement in computer programming. Online Submission, 14(2), 248–255. https://doi.org/10.5829/idosi.mejsr.2013.14.2.2007

11.

Benitti

F. B. V.

(2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58(3), 978–988. https://doi.org/10.1016/j.compedu.2011.10.006

12.

Berland

Wilensky

(2015). Comparing virtual and physical robotics environments for supporting complex systems and computational thinking. Journal of Science Education and Technology, 24(5), 628–647. https://doi.org/10.1007/s10956-015-9552-x

13.

Bers

M. U.

Flannery

Kazakoff

E. R.

Sullivan

(2014). Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers & Education, 72, 145–157. https://doi.org/10.1016/j.compedu.2013.10.020

14.

Blanchard

Freiman

Lirrete-Pitre

(2010). Strategies used by elementary schoolchildren solving robotics-based complex tasks: Innovative potential of technology. Procedia-Social and Behavioral Sciences, 2(2), 2851–2857. https://doi.org/10.1016/j.sbspro.2010.03.427

15.

Bonvillian

W. B.

(2002). Science at a crossroads. Technology in Society, 24(1–2), 27–39. https://doi.org/10.1016/S0160-791X(01)00040-9

16.

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2009). Introduction to meta-analysis. John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470743386

17.

Botički

Pivalica

Seow

(2018). The use of computational thinking concepts in early primary school. In S-C. Kong, D. Andone, G. Biswas, T. Crick, H.U. Hoppe, T-C. Hsu, R. H. Huang, R. K. Li, C-K. Looi, J. Sheldon, J-L. Shih, K-F. Sin, M. Tissenbaum & J.Vahrenbold (Eds.) Proceedings of the International Conference on Computational Thinking Education 2018 (pp.8--13). The Education University of Hong Kong. file:///C:/Users/14206/Downloads/930955.CTE2018-Boticki_Pivalica_Seow_final_camera_ready.pdf

18.

Brotman

J. S.

Moore

F. M.

(2008). Girls and science: A review of four themes in the science education literature. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 45(9), 971–1002. https://doi.org/10.1002/tea.20241

19.

Brown

C. S.

Leaper

(2010). Latina and European American girls’ experiences with academic sexism and their self-concepts in mathematics and science during adolescence. Sex Roles, 63, 860–870. https://doi.org/10.1007/s11199-010-9856-5

20.

Cannady

M. A.

Greenwald

Harris

K. N.

(2014). Problematizing the STEM pipeline metaphor: Is the STEM pipeline metaphor serving our students and the STEM work force? Science Education, 98(3), 443–460. https://doi.org/10.1002/sce.21108

21.

Carstro

Cecchi

Valentente

Buselli

Salvini

Dario

(2018). Can educational robotics introduce young children to robotics and how can we measure it. Journal of Computer Assisted Learning, 34, 970–977. https://doi.org/10.1111/jcal.12304

22.

Chaiklin

(2003). The zone of proximal development in Vygotsky’s analysis of learning and instruction. Cambrige University Press. https://doi.org/10.1017/CBO9780511840975.004

23.

Chauhan

(2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30. https://doi.org/10.1016/j.compedu.2016.11.005

24.

Ching

Y. H.

Yang

Wang

Baek

Swanson

Chittoori

(2019). Elementary school student development of stem attitudes and perceived learning in a stem integrated robotics curriculum. TechTrends, 63(5), 590–601. https://doi.org/10.1007/s11528-019-00388-0

25.

Chung

C. C.

Cartwright

Cole

(2014). Assessing the impact of an autonomous robotics competition for STEM education. Journal of STEM Education: Innovations and Research, 15(2), 24–34. http://www.mcole-psy.com/mcole/Cole-publications/ChungCartwrightCole2014.pdf

26.

Cohen

(1977). Statistical power analysis for the behavioral sciences. Academic Press. https://doi.org/10.1016/C2013-0-10517-X

27.

Constantinou

Ioannou

(2018). Development of Computational Thinking Skills through Educational Robotics. In V. Dimitrova, S. Praharaj, M. Fominykh, & H. Drachsler (Eds.) EC-TEL Practitioner Proceedings 2018: 13th European Conference On Technology Enhanced Learning Leeds (pp.1--11). University of Leeds. https://doi.org/10.1016/j.robot.2015.10.008

28.

Dewitt

Osborne

Archer

Dillon

Willis

Wong

(2013). Young children's aspirations in science: The unequivocal, the uncertain and the unthinkable. International Journal of Science Education, 35(6), 1037–1063. https://doi.org/10.1080/09500693.2011.608197

29.

Diekman

A. B.

Clark

E. K.

Johnston

A. M.

Brown

E. R.

Steinberg

(2011). Malleability in communal goals and beliefs influences attraction to stem careers: Evidence for a goal congruity perspective. Journal of Personality and Social Psychology, 101(5), 902. https://doi.org/10.1037/a0025199

30.

Durak

H. Y.

Saritepeci

(2018). Analysis of the relation between computational thinking skills and various variables with the structural equation model. Computers & Education, 116, 191–202. https://doi.org/10.1016/j.compedu.2017.09.004

31.

Eccles

J. S.

(1994). Understanding women’s educational and occupational choices: Applying the Eccles et al. model of achievement-related choices. Psychology of Women Quarterly, 18, 585–610. https://doi.org/10.1111/j.1471-6402.1994.tb01049.x

32.

Eccles

J. S.

Wigfield

(2002). Motivational beliefs, values and goals. Annual Review of Psychology, 53(1), 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153

33.

Eguchi

(2014a). Educational robotics for promoting 21st century skills. Journal of Automation Mobile Robotics and Intelligent Systems, 8(1), 5–11. https://doi.org/10.14313/JAMRIS_1-2014/1

34.

Eguchi

(2014b). Learning experience through robocupjunior: Promoting stem education and 21st century skills with robotics competition. In M. Searson & M. Ochoa (Eds.), Proceedings of society for information technology & teacher education international conference (pp. 87–93). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/130716/

35.

Fishbein

Ajzen

(1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison-Wesley. https://www.researchgate.net/publication/233897090

36.

Fredrick

W. C.

Walberg

H. J.

(1980). Learning as a function of time. The Journal of Educational Research, 73(4), 183–194. https://doi.org/10.1080/00220671.1980.10885233

37.

Gaudiello

Zibetti

(2016). Learning by robotics: The impact of educational robots on learning. In I. Gaudello & E. Zibetti (Eds.) Learning robotics, with robotics, by robotics: Educational robotics (Vol. 3, pp. 87--141). John Wiley & Sons, Inc. https://doi.org/10.1002/9781119335740.ch3

38.

George

(2006). A cross-domain analysis of change in students’ attitudes toward science and attitudes about the utility of saience. International Journal of Science Education, 28(6), 571–589. https://doi.org/10.1080/09500690500338755.

39.

Gong

L. L.

Zhao

Liu

Wang

Y. Y.

(2020). Recent progress and enlightenment of robotics education research in K-12 field in the past decade: Based on systematic literature review method. Distance Education in China, 40(03), 25–34. https://doi.org/10.13541/j.cnki.chinade.2020.03.004

40.

Guo

Marsh

H. W.

Morin

A. J.

Parker

P. D.

Kaur

(2015). Directionality of the associations of high school expectancy-value, aspirations, and attainment: A longitudinal study. American Educational Research Journal, 52(2), 371–402. https://doi.org/10.3102/0002831214565786

41.

Guzey

S. S.

Harwell

Moore

(2014). Development of an instrument to assess attitudes toward science, technology, engineering, and mathematics (STEM). School Science & Mathematics, 114(6), 271–279. https://doi.org/10.1111/ssm.12077

42.

Hendricks

C. C.

Alemdar

Ogletree

T. W.

(2012). The impact of participation in vex robotics competition on middle and high school students' interest in pursuing stem studies and stem-related careers. American Society for Engineering Education. https://doi.org/10.18260/1-2–22069

43.

Higgins

J. P.

Thompson

S. G.

Deeks

J. J.

Altman

D. G.

(2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557

44.

Hofferth

S. L.

Sandberg

J. F.

(2001). How American children spend their time. Journal of Marriage and Family, 63(2), 295–308. https://doi.org/10.1111/j.1741-3737.2001.00295.x

45.

Hou

(2016). Current situation and thinking of robot education in schools. Software Guide, 15(12), 85–97. https://doi.org/1672-7800(2016)12-0085-02.

46.

Ioannou

Angeli

(2016). A framework and an instructional design model for the development of students' computational and algorithmic thinking. In Smith (Ed.) MCIS 2016 proceedings (p.19). AIS Electronic Library . http://aisel.aisnet.org/mcis2016/19

47.

Ioannou

Bennett

Repenning

Koh

K. H.

Basawapatna

(2011). Computational thinking patterns [Paper presentation]. American Educational Research Association Annual Meeting 2011, Division C, New Orleans, LA, United States. https://www.learntechlib.org/p/108975/

48.

Ioannou

Makridou

(2018). Exploring the potentials of educational robotics in the development of computational thinking: A summary of current research and practical proposal for future work. Education and Information Technologies, 23(6), 2531–2544. https://doi.org/10.1007/s10639-018-9729-z

49.

Jackson

Mentzer

Kramer-Bottiglio

(2019). Pilot analysis of the impacts of soft robotics design on high-school student engineering perceptions. International Journal of Technology and Design Education, 29(5), 1083–1104. https://doi.org/10.1007/s10798-018-9478-8

50.

Joshi

S. S.

(2014). Development and implementation of a MATLAB simulation project for a multidisciplinary graduate course in autonomous robotics. Computer Applications in Engineering Education, 12(1), 54–64. https://doi.org/10.1002/cae.20001

51.

Julià

Antolí

J. S.

(2015). Spatial ability learning through educational robotics. International Journal of Technology and Design Education, 26(2), 185–203. https://doi.org/10.1007/s10798-015-9307-2

52.

Karp

Maloney

(2013). Exciting young students in grades K-8 about STEM through an afterschool robotics challenge. American Journal of Engineering Education, 4(1), 39–54. https://doi.org/10.19030/ajee.v4i1.7857

53.

Kazakoff

Bers

(2012). Programming in a robotics context in the kindergarten classroom: The impact on sequencing skills. Journal of Educational Multimedia and Hypermedia, 21(4), 371–391. https://doi.org/10.1002/j.2168-9830.2012.tb00055.x

54.

Khanlari

(2013). Effects of robotics on 21st century skills, European Scientific Journal, 9(27), 27–36. https://doi.org/10.19044/esj.2013.v9n27p%25p

55.

Khanlari

(2016). Teachers’ perceptions of the benefits and the challenges of integrating educational robots into primary/elementary curricula. European Journal of Engineering Education, 41(3), 320–330. https://doi.org/10.1080/03043797.2015.1056106

56.

Kim

Yuan

Hill

R. B.

Doshi

Thai

C. N.

(2015). Robotics to promote elementary education pre-service teachers' stem engagement, learning, and teaching. Computers & Education, 91, 14–31. https://doi.org/10.1016/j.compedu.2015.08.005

57.

Kopcha

Mcgregor

Shin

Qian

Choi

Hill

Mativo

Choi

(2017). Developing an integrative stem curriculum for robotics education through educational design research. Journal of Formative Design in Learning, 1(1), 31–44. https://doi.org/10.1007/s41686-017-0005-1

58.

Kraftwohl

D. R.

Bloom

B. S.

Masia

B. B.

(1964). Taxonomy of educational objectives, the classification of educational goals, Handbook II: Affective domain. McKay. https://www.uky.edu/∼rsand1/china2018/texts/Bloom%20et%20al%20-Taxonomy%20of%20Educational%20Objectives.pdf

59.

Kucuk

Sisman

(2017). Behavioral patterns of elementary students and teachers in one-to-one robotics instruction. Computers & Education, 111, 31–43. https://doi.org/10.1016/j.compedu.2017.04.002

60.

Lee

Martin

Denner

Coulter

Allan

Erickson

Malyn-Smith

Werner

(2011). Computational thinking for youth in practice. ACM Inroads, 2(1), 32–37. https://doi.org/10.1145/1929887.1929902

61.

Leonard

Buss

Gamboa

Mitchell

Fashola

O. S.

Hubert

Almughyirah

(2016). Using robotics and game design to enhance children's self-efficacy, STEM attitudes, and computational thinking skills. Journal of Science Education & Technology, 25(6), 1–17. https://doi.org/10.1007/s10956-016-9628-2

62.

Mahonry

M. P.

(2009). Students’ attitudes toward STEM: Development of an instrument for high school STEM-based programs. Journal of Technology Studies, 36(1), 24–34. https://doi.org/10.21061/jots.v36i1.a.4

63.

Maltese

A. V.

Tai

R. H.

(2011). Pineline persistence: Examining the association of educational experiences with earn degrees in STEM among U.S. students. Science Education, 95(5), 877–907. https://doi.org/10.1002/sce.20441

64.

Merino-Armero

J. M.

González-Calero

J. A.

Cózar-Gutiérrez

Villena-Taranilla

(2018). Compuputational thinking initiation: An experience with robots in primary education. Journal of Research in Science, Mathematics and Technology Education, 1(2), 181–206. https://doi.org/10.31756/jrsmte.124

65.

Mubin

Stevens

C. J.

Shahid

Al Mahmud

Dong

J. -J.

(2013). A review of the applicability of robots in education. Journal of Technology in Education and Learning, 1(209-0015), 13. https://doi.org/10.2316/Journal.209.2013.1.209-0015

66.

Myers

D. G.

(1993). Behavior and attitudes. In D. G. Myers (Ed.), Social psychology: Fourth edition (pp.120). McGraw-Hill.

67.

National Research Council. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. National Academies Press. www.nationalacademies.org/cosepup

68.

National Research Council. (2010). Report of a workshop on the scope and nature of computational thinking. National Academies Press. http://www.nap.edu/catalog/12840.html

69.

Noh

Lee

(2019). Effects of robotics programming on the computational thinking and creativity of elementary school students. Educational Technology Research and Development, 68(1), 463–484. https://doi.org/10.1007/s11423-019-09708-w

70.

Nourbakhsh

I. R.

Crowley

Bhave

Hamner

(2015). The robotic autonomy mobile robotics course: Robot design, curriculum design and educational assessment. Autonomous Robots, 18(1), 103–127. https://doi.org/10.1023/B:AURO.0000047303.20624.02

71.

Noyes, J., Popay, J., Pearson, A., Hannes, K. & Booth, A. (2008). Qualitative Research and Cochrane Reviews. In J. P. Higgins & S. Green (Eds.) Cochrane handbook for systematic reviews of interventions (version 5.1.0, pp. 571-591). John Wiley & Sons. https://doi.org/10.1002/9780470712184.ch20

72.

Nugent

Barker

Grandgenett

Adamchuk

V. I.

(2010). Impact of robotics and geospatial technology interventions on youth stem learning and attitudes. Journal of Research on Technology in Education, 42(4), 391–408. https://doi.org/10.1080/15391523.2010.10782557

73.

Pajares

Usher

E. L.

(2008). Self-efficacy, motivation, and achievement in school from the perspective of reciprocal determinism. Advances in Motivation and Achievement, 15, 391–423. https://doi.org/10.1016/S0749-7423(08)15012-9

74.

Papastergiou

(2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12. https://doi.org/10.1016/j.compedu.2008.06.004

75.

Papert

(1996). An exploration in the space of mathematics education. International Journal of Computers for Mathematical Learning, 1(1), 95–123. https://doi.org/10.1007/BF00191473

76.

Parkerson

J. A.

Lomax

R. G.

Schiller

D. P.

Walberg

H. J.

(1984). Exploring causal models of education achievement. Journal of Educational Psychology, 76(4), 638–646. https://doi.org/10.1037/0022-0663.76.4.638

77.

Piaget

(1964). Part I: Cognitive development in children: Piaget. development and learning. Journal of Research in Science Teaching, 2(3), 176–186. https://doi.org/10.1002/tea.3660020306

78.

Quilter

S. M.

Chester

(2001). The relationship between web-based conferencing and instructional outcomes. International Journal of Instructional Media, 28(1), 13–22. http://web.b.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=1&sid=7281af38-3482-4db9-9cf7-b390b72b79a9%40pdc-v-sessmgr06

79.

Reynolds

A. J.

Walberg

H. J.

(1992). A structural model of science achievement and attitude: An extension to high school. Journal of educational Psychology, 84(3), 371. https://doi.org/10.1037/0022-0663.84.3.371

80.

Rothstein

H. R.

Sutton

A. J.

Borenstein

(2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.) Publication bias in meta-analysis (pp. 261--286). John Wiley & Sons, Ltd. https://doi.org/10.1002/0470870168

81.

Rusk

Resnick

Berg

Pezalla-Granlund

(2008). New pathways into robotics: Strategies for broadening participation. Journal of Science Education and Technology, 17(1), 59–69. https://doi.org/10.1007/s10956-007-9082-2

82.

Ryan

R. M.

Deci

E. L.

(2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68

83.

Sadler

P. M.

Sonnert

Hazari

Tai

(2012). Stability and volatility of stem career interest in high school: A gender study. Science Education, 96(3), 411–427. https://doi.org/10.1002/sce.21007

84.

Sáez-López

J.-M.

Sevillano-García

M.-L.

Vazquez-Cano

(2019). The effect of programming on primary school students’ mathematical and scientific understanding: Educational use of mbot. Educational Technology Research and Development, 67(6), 1405–1425. https://doi.org/10.1007/s11423-019-09648-5

85.

Schulz

K. F.

Chalmers

Hayes

R. J.

& Altman, D. G. (1995). Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Journal of the American Medical Association, 273(5), 408–412. https://doi.org/10.1001/jama.1995.03520290060030

86.

Selby

C. C.

(2012). Promoting computational thinking with programming. In M. Knobelsdorf & R. Romeike (Eds.) Proceedings of the 7th workshop in primary and secondary computing education (pp. 74–77). ACM. https://doi.org/10.1145/2481449.2481466

87.

Selby

C., & Woollard

(2013). Computational thinking: The developing definition. In J. D. Dougherty, K. Nagel, A. Decker, & K. Eiselt (Eds.) Special Interest Group on Computer Science Education (pp. 1--12). ACM. http://eprints.soton.ac.uk/id/eprint/356481

88.

Shan

J. H.

Gong

L. L.

Yan

H. B.

(2019). The impact of educational robots on student learning outcomes: A meta-analysis based on 49 experimental or quasi-experimental research papers. China Educational Technology, 40(05), 76–83. https://doi.org/1006-9860(2019)05-0076-08

89.

Sheridan

K. M.

Clark

Williams

(2013). Designing games, designing roles: A study of youth agency in an urban informal education program. Urban Education, 48(5), 734–758. https://doi.org/10.1177//00 42085913491220

90.

Shih

B. Y.

Chen

T. H.

Wang

S. M.

Chen

C. Y.

(2013). The exploration of applying LEGO NET in the situated science and technology learning. Journal of Baltic Science Education, 12(1), 73–91. http://oaji.net/articles/2015/987-1425757780.pdf

91.

Shimada

Kanda

Koizumi

(2012). How can a social robot facilitate children’s collaboration? In S. S. Ge, O. Khatib, J-J. Cabibihan, R. Simmons, & M-A. Williams (Eds.) International conference on social robotics (pp. 98–107). Springer. https://doi.org/10.1007/978-3-642-34103-8_10

92.

Simon

R. A.

Aulls

M. W.

Dedic

Hubbard

Hall

N. C.

(2015). Exploring student persistence in stem programs: A motivational model. Canadian Journal of Education/Revue Canadienne De l’éducation, 38(1), 1–27. https://journals.sfu.ca/cje/index.php/cje-rce/article/view/1729

93.

Slavin

R. E.

(1995). A model of effective instruction. The Educational Forum, 59(2), 166–176. https://doi.org/10.1080/00131729509336383

94.

Smith

J. A.

Sivo

S. A.

(2012). Predicting continued use of online teacher professional development and the influence of social presence and sociability. British Journal of Educational Technology, 43(6), 871–882. https://doi.org/10.1111/j.1467-8535.2011.01223.x

95.

Spolaôr

Benitti

F. B. V.

(2017). Robotics applications grounded in learning theories on tertiary education: A systematic review. Computers & Education, 112, 97–107. https://doi.org/10.1016/j.compedu.2017.05.001

96.

Stoeckelmayr

Tesar

Hofmann

(2011). Kindergarten children programming robots: A first attempt. In C. Ishii, G. Capi, & A. Olaru (Eds.) Proceedings of 2nd international conference on robotics in education (pp. 185–192). INNOC - Austrian Society for Innovative Computer Sciences. https://www.researchgate.net/publication/265092506

97.

Sullivan

Bers

M. U.

(2016). Robotics in the early childhood classroom: Learning outcomes from an 8-week robotics curriculum in pre-kindergarten through second grade. International Journal of Technology and Design Education, 26(1), 3–20. https://doi.org/10.1007/s10798-015-9304-5

98.

Terry

B. S.

Briggs

B. N.

Rivale

(2011). Work in progress: Gender impacts of relevant robotics curricula on high school students' engineering attitudes and interest. In B. S. Terry, B. N. Briggs, & S. Rivale (Eds.) Frontiers in education conference (pp. 1--3). IEEE. https://doi.org/10.1109/FIE.2011.6143090

99.

Thornton

Lee

(2000). Publication bias in meta-analysis: Its causes and consequences. Journal of Clinical Epidemiology, 53(2), 207–216. https://doi.org/10.1016/S0895-4356(99)00161-4

100.

Ucgul

Cagiltay

(2014). Design and development issues for educational robotics training camps. International Journal of Technology and Design Education, 24(2), 203–222. https://doi.org/10.1007/s10798-013-9253-9

101.

Unfried

Faber

Stanhope

D. S.

Wiebe

(2015). The development and validation of a measure of student attitudes toward science, technology, engineering, and math (s-stem). Journal of Psychoeducational Assessment, 33(7), 622–639. https://doi.org/10.1177/0734282915571160

102.

Wang

Chen

Yang

Gao

(2013). Meta-analysis of the relationship between sensation seeking and internet addiction. Advances in Psychological Science, 21(10), 1720–1730. https://doi.org/10.3724/SP.J.1042.2013.01720

103.

Wang

Sun

S. N. S.

(2017). The development status and trend of robot education in Japan. Modern Educational Technology, 27(4), 5–11. https://doi.org/10.3969/j.issn.1009-8097.2017.04.001

104.

Welch

Huffman

(2011). The effect of robotics competitions on high school students' attitudes toward science. School Science and Mathematics, 111(8), 416–424. https://doi.org/10.1111/j.1949-8594.2011.00107.x

105.

Williams

Igel

Poveda

Kapila

Iskander

(2012). Enriching K-12 math education using legos [Paper presentation]. 2011 ASEE Annual Conference & Exposition Proceedings, Vancouver, British Columbia, Canada. https://doi.org/10.18260/1-2–17910

106.

Wing

J. M.

(2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

107.

Wing

J. M.

(2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences, 366(1881), 3717–3725. https://doi.org/10.1098/tsta.2008.0118

108.

Wing

J. M.

(2011). Research notebook: Computational thinking—what and why. The Link Magazine, p. 6. https://www.cs.cmu.edu/∼CompThink/resources/TheLinkWing.pdf

109.

Y. H.

T. T.

(2018). Current situation, practice, reflection and prospect of robot education from the perspective of machine intelligence. Journal of Distance Education, 36(4), 79–87. https://doi.org/10.15881/j.cnki.cn33-1304/g4.2018.04.010

110.

Yuen

T. T.

Boecking

Stone

Tiger

E. P.

Gomez

Guillen

Arreguin

(2014). Group tasks, activities, dynamics, and interactions in collaborative robotics projects with elementary and middle school children. Journal of STEM Education, 15(1), 39–45. file:///C:/Users/14206/Downloads/article_148284%20(2).pdf

111.

Zhou

Han

X. J.

(2019). A meta-analysis of the effects of educational robots on students’ learning. Modern Distance Education Research, 31(3), 96–105. https://doi.org/10.3969/j.issn.1009-5195.2019.03.011