The Development and Validation of the Robotics Learning Self-Efficacy Scale (RLSES)

Abstract

Robotics education has gradually been emphasized in contemporary school curricula; however, assessment tools for robotics learning are still limited. Based on Bloom’s Taxonomy of educational objectives, this study aimed to develop the Robotics Learning Self-Efficacy Scale (RLSES) with a two-level construct of five dimensions for assessing students’ self-efficacy for learning robotics. A total of 181 elementary, junior high and senior high school students (5th–12th graders) with robotics learning experience were selected as the sample of this study. A questionnaire including 32 candidate items designed for the initial version of the RLSES was administered to the sample. An exploratory factor analysis was conducted and, finally, 16 items were drawn for the final RLSES under five subscales (Comprehension, Practice, Analysis, Application, and Collaboration), with a total explained variance of 85.28%. The Cronbach’s alpha reliability was .97 for the overall scale, ranging from .87 to .95 for the subscales. The inter-correlation analysis showed evidence of discriminant validity. Regression analysis results supported that Practice and Comprehension self-efficacy were significant predictors of Analysis, Application, and Collaboration self-efficacy, confirming the two-level (2 × 3) construct of the RLSES. Significant differences among school levels were found and are discussed.

Keywords

robotics learning self-efficacy evaluation instrument scale validation

Background

The Rationale of Robotics Learning

Robotics learning has drawn increasing attention in education due to the cutting-edge artificial intelligence technology developments during the last decade. Recent educational studies have reported some impacts of robotics on student learning and school curricula in several ways. First, robotics learning can enhance students’ creativity, critical thinking, and problem-solving abilities (Afari & Khine, 2017), as well as enhance their technology literacy (Alimisis, 2013; Slangen et al., 2011). This benefit is parallel with the educational goals of 21st century competencies for all citizens in many countries (Ananiadou & Claro, 2009). Second, integrating robotics technology into school curricula has the potential to enhance students’ learning motivation and learning achievements (Mubin et al., 2013), especially for lower achieving learners’ science learning motivation (Hu & Tsai, 2019). Third, robotics learning is becoming one of the main streams in technology education in which robotics has been regarded as learning content involving the design of robotic components, power, sensors, control, and intelligence.

Robotics learning is usually delivered via hands-on and minds-on learning contexts which support constructivist or social constructivist learning. During the process of designing and making a robot, students may understand more about the cutting-edge technology and relevant knowledge (Aristawati et al., 2018; Slangen et al., 2011). Additionally, robotics learning provides an authentic interdisciplinary learning context, such as a STEAM curriculum, for students to learn science, mathematics, technology, engineering, and art design in an integrated and meaningful way. Through the real-world hands-on and minds-on problem-solving learning experience, students may find it easier to build, test, and revise a model of the abstract conceptual knowledge learnt in traditional classrooms. In addition, collaborative or cooperative learning is usually implemented in robotics learning curricula, because social scaffoldings are important for such problem-solving tasks (Gomoll et al., 2018; Jordan & McDaniel, 2014; Menekse et al., 2017). Therefore, robotics learning is regarded as a perfect learning context for delivering constructivist learning (Alimisis, 2013; Sung et al., 2017; Toh et al., 2016) or social-constructivist learning (Mubin et al., 2013). Although many efforts have been made by researchers to design or implement robotics curricula, the educational goals of robotics learning have not yet been thoroughly analyzed in the literature.

Educational Goals for Robotics Learning

The educational goals of robotics learning may be analyzed by Bloom et al.’s (1956) Taxonomy of educational goals in the cognitive, psychomotor, and affective domains. Regarding the cognitive domain, six levels were classified as knowledge, comprehension, application, analysis, synthesis, and evaluation, and were further revised as the following six verbs: remember, understand, apply, analyze, evaluate, and create (Anderson et al., 2001). As for the psychomotor domain, seven levels, namely perception, set, guided response, mechanism, complex overt response, adapting, and originating, were proposed (Simpson, 1966). Regarding the affective domain, five levels of learning goals were proposed: receiving, responding, valuing, organizing, and characterizing (Krathwohl et al., 1964). Gagne and Briggs (1974) further revised Bloom’s three domains into the five domains of intellectual skills, cognitive strategies, verbal information, motor skills, and attitude. The three domains (the cognitive, skill, and affective domains) usually serve as a guideline for teachers to design the instructional objectives of a course.

The educational goal of robotics learning has not been specifically defined or conveyed in the literature; however, some studies have provided some features of the framework. According to Toh et al. (2016), robotics learning positively impacts child development in four dimensions: cognitive, conceptual, language, and social/collaborative skills. Also, in robotics learning contexts, students have more chances to practice problem-solving skills and apply them in real-world contexts (Afari & Khine, 2017; Alimisis, 2013). Moreover, not only can the teamwork collaborations be improved in robotics learning contexts, but also students can become more confident in solving problems, sharing ideas, and becoming active learners (Chalmers, 2018). These features imply that robotics learning can be profiled from the cognitive, affective, and skill domains, suggesting that the educational goal of robotics learning can be grounded on the framework of Bloom’s Taxonomy (Bloom et al., 1956).

Based on all of the above literature, this study proposes that the educational goals of robotics learning should include the following five aspects: (a) conceptual understanding, (b) technical performance, (c) application, (d) analysis, and (e) collaboration. The conceptual understanding aspect involves the comprehension of robotics-relevant conceptual knowledge in science, technology, engineering, design or mathematics. The technical performance aspect involves the real-world task or practice of making a robot. The application aspect refers to generalizing the problem solution or approach for designing or making a robot into another similar problem context. The analysis aspect refers to comparing or evaluating the advantages and disadvantages of several problem solutions; and finally the collaboration aspect refers to the collaborative learning skills utilized in a teamwork context for solving robotics problems. The above five aspects can be regarded as the educational goals of robotics learning which should guide the related curriculum and instructional designs; furthermore, they can serve as guidelines for developing assessment tools for robotics learning.

Assessment of Robotics Learning Self-Efficacy

Self-efficacy refers to an individual’s perception of his/her own abilities in conducting a specific task, and usually relates to personal prior experience in the task performance accomplishments (Bandura, 1977). Learners with higher self-efficacies generally have higher learning motivations because they are usually more persistent when facing frustrations, and have more confidence in accomplishing a difficult task (Bandura, 1992). Prior studies have provided tremendously significant evidence regarding the positive correlations between self-efficacy and learning outcomes in academic achievements in general or in specific learning domains (see the review of Tsai et al., 2011). Therefore, self-efficacy is usually measured in post-tests in educational studies, serving as an indirect assessment of students’ learning performances or achievements (Müller & Seufert, 2018; Sun & Yeh, 2017). Sometimes, however, self-efficacy is assessed in a pre-test in educational studies to diagnose students’ prior proficiencies before entering any instructional treatment, serving as a baseline for measuring learning improvement (Chen, 2017; Hsu et al., 2019).

Self-efficacy can be task-general or task-specific (see Tsai et al., 2011), and it is usually task-specific in ICT-related learning domains. For example, internet self-efficacy indicates individuals’ perceptions of their own abilities of (or self-confidence in) using the Internet (Tsai & Tsai, 2010), while computer programming self-efficacy refers to individuals’ self-confidence in conducting a computer programming task (Tsai et al., 2019). To the best of our knowledge, robotics learning self-efficacy has not been reported nor discussed in the prior literature, which may be due to the lack of an assessment tool. Developing a valid and reliable assessment tool is important for implementing and conducting research about robotics curricula and instruction.

Since self-efficacy is a psychological state that can only be perceived by the learners themselves, it is usually assessed using self-reported scales. The above five aspects (i.e., comprehension, practice, application, analysis, and collaboration) of the educational goals for robotics learning may serve as a conceptual framework for the assessment of students’ robotics learning self-efficacy. According to Bloom’s hierarchical framework of educational goals (Anderson et al., 2001; Bloom et al., 1956), comprehension (or understanding) and practice (or setting) are usually regarded as the goals for development in the beginning level of the cognitive and skill domains; applying and analyzing are usually regarded as the goals for the development in the higher level of the cognitive domain; and collaboration can be regarded as the goal for the development in the higher level metacognitive, affective, and interpersonal skill domains (Roberts, 2004; Tsai, 2009). Therefore, this current study further proposed that, when learning about robotics, students’ self-efficacies in the comprehension and practice dimensions may be the two prerequisite conditions of their self-efficacies in the application, analysis, and collaboration dimensions.

In sum, a valid and reliable assessment tool for students’ robotics learning self-efficacy is required before educational researchers can examine students’ learning outcomes in robotics-related curricula (Toh et al., 2016). With a convenient scale to diagnose students’ robotics learning self-efficacy before entering a robotics classroom, teachers can adopt or arrange appropriate teaching approaches or materials for student learning in the curriculum. The score obtained before the curriculum can also serve as a baseline to measure how students improve in such a curriculum. As robotics technology is increasingly impacting human society and school curricula, there is a need to develop a valid and reliable scale for computer educational researchers and instructors to assess students’ robotics learning self-efficacy.

Purpose

Based on the above literature, this study aimed to develop and validate a self-reported scale, the Robotics Learning Self-Efficacy Scale (RLSES), to assess students’ self-efficacy for robotics learning. According to Bloom’s Taxonomy for educational objectives (Anderson et al., 2001; Bloom et al., 1956), a five-dimensional framework (Practice, Comprehension, Application, Analysis, and Collaboration) was proposed for the construct of the RLSES. A two-level relational construct was further examined according to the following three hypotheses (shown in Figure 1):

Figure 1.

The Hypotheses to Examine the Framework of the Two-Level Construct of the RLSES.

H1: In robotics learning, both the Practice (H1a) and the Comprehension (H1b) self-efficacies can significantly predict the Application self-efficacy.

H2: In robotics learning, both the Practice (H2a) and the Comprehension (H2b) self-efficacies can significantly predict the Analysis self-efficacy.

H3: In robotics learning, both the Practice (H3a) and the Comprehension (H3b) self-efficacies can significantly predict the Collaboration self-efficacy.

Method

Development of the RLSES Items

Based on Bloom’s Taxonomy for educational objectives (Anderson et al., 2001; Bloom et al., 1956) and the above literature review, this study argued that the educational goals of robotics learning should be discussed and assessed in the five domains: conceptual understanding, technical performance, application, analysis, and collaboration. Thus, the initial version of the RLSES was developed based on the following five-dimensional framework: Comprehension, Application, Analysis, Practice, and Collaboration. To build up a pool of candidate items, six to eight candidate items were developed for each dimension via related components of robotics learning (Altin & Pedaste, 2013; Benitti, 2012). All of the candidate items were developed by two robotics education researchers, and verified by one middle school mathematics teacher, one middle school social science teacher, and one elementary school computer teacher. Each item was revised until a consensus had been reached among the school teachers. Finally, 32 candidate items were included in the initial version of RLSES. All items were designed to be evaluated using a 5-point Likert scale, ranging from 1 (not confident at all) to 5 (very much confident).

Data Collection

To examine the validity of the initial version of RLSES, a convenient sample of 181 students from northern and central Taiwan were recruited for testing in this study. The sample included 44 elementary school students (5th and 6th graders, aged from 11 to 12 years old), 86 junior high school students (7th to 9th graders, aged from 13–15 years old), and 51 senior high school students (10th to 12th graders, aged from 16–18 years old). The sample was selected from two elementary schools, two junior high schools and one senior high school in which robotics-embedded curricula had been implemented. Only students with robotics learning experience of more than one semester were selected as the sample of this study. All of them had robotics learning experience in the school curriculum which required them to collaboratively conduct a robotics hands-on project with peers. Some of them also had extra robotics learning experience from after-school activities.

Data Analysis

For the purpose of this study, first of all, an explorative factor analysis (EFA) using the principle component method with a varimax rotation was conducted using the initial RLSES of 32 candidate items in order to draw the final items and factors of the RLSES. After the final items were drawn and the factors (or subscales) were confirmed from the EFA, the Cronbach’s alpha coefficient was examined for the overall scale and for each subscale for analyzing the reliability of the RLSES. In addition, Pearson’s correlation analyses were conducted to examine the inter-correlations among the subscales and the evidence of discriminant validity. Furthermore, three regression analyses were conducted to examine the three hypotheses proposed in this study. Finally, regarding students’ RLSES scores, ANOVAs with post-hoc comparisons were conducted to examine the differences among the three school levels.

Results

Explorative Factor Analysis

This study utilized exploratory factor analysis (EFA) to explore the construct of the RLSES. Before conducting the analysis, the Kaiser–Meyer–Olkin (KMO) value was .96 and the Barlett’s test of Sphericity was significant (Barlett’s = 3053.12, p < .000), suggesting that it was appropriate to conduct an EFA analysis. Therefore, an EFA with the principal component method was conducted for the 32 candidate items designed for the original version of the RLSES. In this study, the items with a factor loading above 0.5 were retained in each subscale. Finally, with a total explained variance of 85.28%, a total of 16 items were drawn for the final version of the RLSES under a construct of five factors (see Table 1).

Table 1.

Results of EFA and Cronbach’s Alpha Reliabilities for the Final RLSES.

Item	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5
Factor 1: Practice (PR), reliability α = .92
PR 1	.71	.30	.32	.34	.06
PR 2	.68	.35	.18	.27	.32
PR 3	.76	.26	.33	.16	.26
PR 4	.72	.36	.19	.23	.34
Factor 2: Application (AP), reliability α = .95
AP 1	.40	.71	.23	.29	.25
AP 2	.39	.77	.28	.22	.20
AP 3	.33	.77	.25	.22	.33
Factor 3: Collaboration (CL), reliability α = .91
CL 1	.25	.25	.58	.29	.34
CL 2	.39	.39	.72	.32	.21
CL 3	.26	.26	.78	.26	.30
Factor 4: Comprehension (CP), reliability α = .87
CP 1	.25	.36	.34	.72	.16
CP 2	.39	.38	.39	.64	.07
CP 3	.22	.11	.17	.74	.43
Factor 5: Analysis (AN), reliability α = .92
AN 1	.43	.34	.28	.27	.60
AN 2	.33	.44	.35	.32	.58
AN 3	.28	.35	.38	.26	.65
Eigenvalue	11.15	0.83	0.62	0.54	0.51
% of Variance	69.68	5.17	3.89	3.38	3.16
Total variance explained = 85.28%Overall reliability of Cronbach’s alpha = .97 (N = 181)

Note. Factor loadings higher than 0.5 are all marked in bold.

Table 1 summarizes the results of the EFA with the final 16 items categorized under the five subscales: Practice (4 items), Application (3 items), Collaboration (3 items), Comprehension (3 items), and Analysis (3 items). The Cronbach’s alpha reliability of the final RLSES was .97 for the overall scale, and ranged from .87 to .95 for the five subscales. This suggests that the RLSES has very good internal consistency.

According to the EFA results, the final 16 items (see Table 2) drawn for the RLSES can be categorized in five subscales with the following definitions:

Table 2.

The Items Drawn for the Five Subscales in the Final Version of RLSES (16 Items).

Subscale item no.	Item
Practice
PR1	I can assemble a robot step by step.
PR2	I know how to use the tools of making a robot.
PR3	I can build up a circuit for a robot.
PR4	I can make a robot.
Application
AP1	I can propose the ideas of using robots to solve problems.
AP2	I can apply robotics knowledge in my daily life.
AP3	I can make use of a robot to solve a problem.
Collaboration
CL1	I can discuss easily with peers how to make robots.
CL2	I can present my ideas clearly in a robotics classroom.
CL3	I can express my opinions freely in a robotics classroom.
Comprehension
CP1	I am clear about the learning concepts of robotics.
CP2	I can answer teachers’ questions in a robotics classroom.
CP3	I can link robotics concepts with other learning subjects.
Analysis
AN1	I can evaluate several robotic solutions for solving a problem.
AN2	I can demonstrate an idea by making a robot.
AN3	I can think of a robotics problem from different angles.

Practice: assessing students’ confidence in doing a hands-on robotics task. An example item is, “I can assemble a robot step by step.”

Application: measuring students’ confidence in applying their robotics knowledge to solve problems. An example item is, “I can make use of a robot to solve a problem.”

Collaboration: evaluating students’ confidence in collaborating with peers while learning robotics in classrooms. An example item is, “I can discuss easily with peers how to make robots.”

Comprehension: assessing students’ confidence in comprehending the conceptual knowledge while learning robotics. An example item is, “I am clear about the learning concepts of robotics.”

Analysis: measuring students’ confidence in analyzing the problems relating to robotics while learning robotics. An example item is, “I can think of a robotics problem from different angles.”

Inter-Correlation Analyses

In this study, Pearson’s correlation analyses were conducted to examine the inter-correlations among the RLSES sub-scales. Table 3 shows that all of the sub-scores are significantly correlated (p < .001) with the correlation coefficients, ranging from 0.73 to 0.82. To examine the discriminant validity, the reliability Cronbach’s alpha of each subscale (ranging from .87 to .95) is shown in bold on the diagonal of Table 4. It is clear that the correlation coefficient between any subscale and another is lower than its Cronbach’s alpha coefficient, providing evidence of the discriminant validity (Gaski & Nevin, 1985) for the RLSES.

Table 3.

Inter-Correlations Among the Scores of the RLSES Subscales (N = 181).

	1	2	3	4	5
1. Practice (PR)	.92
2. Application (AP)	.81***	.95
3. Collaboration (CL)	.77***	.77***	.91
4. Comprehension (CP)	.75***	.73***	.79***	.87
5. Analysis (AN)	.81***	.81***	.82***	.78***	.92

Note. ***p < .001; Cronbach’s alpha reliability of each subscale is marked in bold on the diagonal.

Table 4.

Results of Regression Analyses and Hypotheses Testing (N = 181).

Predicted	Predicting	SE	β	T	Adjusted R²	Hypothesis testing
Application	Practice	0.06	0.61	9.77***	0.69	H1a supported
	Comprehension	0.07	0.28	4.48***		H1b supported
	Constant	0.17		1.95
Analysis	Practice	0.06	0.51	8.63***	0.72	H2a supported
	Comprehension	0.06	0.39	6.61***		H2b supported
	Constant	0.16		0.84
Collaboration	Comprehension	0.07	0.49	7.86***	0.69	H3a supported
	Practice	0.07	0.40	6.51***		H3b supported
	Constant	0.17		0.96

Note. ***p < .001.

Regression Analyses

To examine the hypotheses proposed in this study, three stepwise regression analyses were conducted, and the results are summarized in Table 4. The table shows that, with an explained variance of 69%, the Application score was significantly and positively predicted by the Practice score (β = 0.61, p < .001) and the Comprehension score (β = 0.28, p < .001), supporting both the H1a and H1b hypotheses.

Additionally, the Analysis score was significantly and positively predicted by the Practice (β = 0.51, p < .001) and the Comprehension (β = 0.39, p < .001) scores, with an explained variance of 72%. Therefore, the hypotheses H2a and H2b were supported. Finally, with a 69% explained variance, the Collaboration score was significantly and positively predicted by the Comprehension (β = 0.49, p < .001) and the Practice (β = 0.40, p < .001) scores, supporting the H3a and H3b hypotheses.

In sum, all the hypotheses H1, H2, and H3 proposed in this study were supported, confirming the two-level relationships among the five factors of the RLSES. That is, in robotics learning contexts, students’ Practice self-efficacy and Comprehension self-efficacy were both significant predictors of their Application, Analysis, and Collaboration self-efficacies. All the significant relationships found in the regression analyses are illustrated in Figure 2.

Figure 2.

The Two-Level Construct Confirmed for the Framework of the RLSES.

Students’ Scores on RLSES

To understand students’ scores on each dimension of the RLSES, the descriptive data are summarized in Table 5. It shows that, for all the subscales, the values of kurtosis and skewness are within an acceptable range of –1 to 1 (Chan, 2003). Therefore, the distribution of each RLSES sub-score can be regarded as a normal curve. Meanwhile, the item means of the sub-scores range from 3.49 (for the Practice subscale) to 3.57 (for the Comprehension subscale) on a 5-point scale. This indicates that the overall sample had about a medium level of the RLSES in each dimension.

Table 5.

Students’ Scores on Each Subscale of the RLSES (N = 181).

	Min	Max	Item mean	SD	Kurtosis	Skewness
Practice	1.00	5.00	3.49	1.02	–0.24	–0.39
Application	1.00	5.00	3.53	1.04	–0.32	–0.33
Collaboration	1.00	5.00	3.55	1.07	–0.23	–0.49
Comprehension	1.00	5.00	3.57	0.98	–0.61	–0.21
Analysis	1.00	5.00	3.50	1.05	–0.41	–0.38

This study further explored the group differences on the students’ RLSES scores among the three school levels: elementary, junior high, and senior high school students. Table 6 summarizes the results of the ANOVAs among the groups along with the post-hoc Scheffé comparison results. The ANOVA results showed that there were significant differences among the sub-groups in all five sub-scores (p < .001 for the Practice, Application, and Collaboration scores; p < .01 for the Analysis score; p < .05 for the Comprehension score). The Scheffé tests further showed that, for all RLSES sub-scores, no significant differences were found between the Elementary group and the Junior High group. However, both the Elementary and the Junior High groups had significantly higher Practice, Application, and Collaboration scores than the Senior High group. Moreover, the Junior High group scored significantly higher than the Senior High group on the Comprehension and Analysis subscales. In sum, these findings suggest that the Elementary group and the Junior High group had significantly higher self-efficacies in robotics learning than the Senior High group. This interesting finding is worth further discussion.

Table 6.

Results of the ANOVAs on the RLSES Sub-Scores Among the Groups by School Levels.

Subscale	(1)Elementary (n = 44)		(2) Junior high (n = 86)		(3) Senior high (n = 51)
Subscale	Mean	SD	Mean	SD	Mean	SD	F	p	Scheffé test
Practice	3.60	0.99	3.76	0.95	2.93	0.97	12.27	.000***	(1)>(3) (2)>(3)
Application	3.61	0.97	3.82	0.94	2.96	1.03	12.67	.000***	(1)>(3) (2)>(3)
Collaboration	3.70	1.02	3.78	0.99	3.04	1.08	9.00	.000***	(1)>(3) (2)>(3)
Comprehension	3.57	0.94	3.75	0.98	3.27	0.94	4.01	.020*	(2)>(3)
Analysis	3.50	1.06	3.76	1.00	3.06	1.00	7.70	.001**	(2)>(3)

Note. *p < .05. **p < .01. ***p < .001.

Discussion

Framework of the Factors of the RLSES

The EFA results suggested that the RLSES developed in this study had good validity and reliabilities with the following five subscales (or factors): practice, comprehension, application, analysis, and collaboration. This means that robotics learning self-efficacy can be assessed via five dimensions including hands-on practice, conceptual understanding, conceptual application, problem analysis, and team-work collaboration. The EFA results also confirmed the multi-domain framework for robotics learning self-efficacy. That is, it was composed of the self-efficacy from the cognitive domain (comprehension, application, and analysis), the affective domain (collaboration), and the psychomotor domain (practice), which is basically parallel to Bloom’s Taxonomy (Anderson et al., 2001; Bloom et al., 1956). This result provides some insights about the conceptions of robotics learning. That is, robotics learning should be regarded as a learning task which simultaneously involves competencies in the cognitive, affective, and psychomotor skill domains. Robotics learning requires not just hands-on skills, but also the conceptual knowledge of robotics and the social skills to share and construct knowledge (Chalmers, 2018; Menekse et al., 2017; Toh et al., 2016).

Additionally, the regression analyses results supported the two-level relational framework proposed for the construct of the RLSES. The first level consists of the Practice and Comprehension factors, while the second level consists of the Analysis, Application, and Collaboration factors. Each first-level factor was able to predict each second-level factor, suggesting that all the second-level factors are determined by the first-level factors. In other words, practicing and comprehension are the basic and essential elements of robotics learning (Slangen et al., 2011; Tsai et al., 2020; Tsai & Wang, 2020). Moreover, practice and comprehension are the preconditions or the foundations of analysis, application, and collaboration in robotics learning. This could suggest that, in robotics-related curricula, both hands-on (practice) and minds-on (comprehension) learning experiences are required for the development of the higher level cognitive strategies (application and analysis) as well as the affective and interpersonal skills (collaboration). This relational framework could be applied to robotics-related curriculum designs. Future studies may reconfirm this relationship using more advanced statistical analysis approaches such as SEM analyses with a larger sample size.

The Role of School Levels in Students’ RLSES Scores

Regarding the students’ scores of RLSES, this study found some interesting results among the three school levels: elementary, junior high, and senior high school. First, no significant differences were found between the elementary and the junior high school students for any of the dimensions of the RLSES. This indicates that the elementary and junior high school students had about the same levels of self-efficacies for learning robotics. This might be due to the two groups of participants being recruited from the same curriculum project in which robotics was integrated into the STEAM curriculum in the same way.

In this study, the significant differences among the groups mainly came from the senior high school students. That is, the senior high school students had significantly lower robotics learning self-efficacies that the elementary school students and the junior high school students in the following three dimensions: Practice, Application, and Collaboration. They also scored significantly lower than the junior high school students for the Analysis and Comprehension self-efficacies. One possible reason for the differences could that the senior high school students were recruited from a traditional robotics curriculum in a high school computer course, rather than from an integrated STEAM curriculum. Recent literature has reported that STEAM curricula can facilitate deeper learning (Quigley et al., 2020), reinforce students’ technological skills (Conde, 2020), and increase students’ learning motivation (Herro et al., 2018; Lin & Tsai, 2021). In order to reach a confident conclusion, more studies are needed to examine the impacts of STEAM curricula on students’ robotics learning self-efficacy. Other factors such as the senior high school students’ attitudes and motivation of learning robotics as well as the teachers’ instructional approaches could be responsible for the differences observed in this study, and are also worth further examination.

Applications and Future Studies

With good validity and reliabilities, the RLSES developed in this study can be applied to all students in robotics-related curricula above the elementary school upper division levels, for both research and teaching practices. In the past years, many robotics curricula have been successively developed and implemented. Although learning variables such as learning interest (Gomoll et al, 2018), computational thinking (Bers et al., 2014; Tsai et al., 2020; Witherspoon et al., 2017), and collaborative skills (Gomoll et al., 2018; Menekse et al., 2017) have been examined and reported in the literature, robotics learning self-efficacy has not yet been deeply examined. Since self-efficacy has been consistently reported to be significantly correlated with students’ learning achievement or learning performance in many subject learning domains (Bandura, 1977; Tsai et al., 2011), teachers may consider assessing students’ robotics learning self-efficacy as an indirect assessment. With high reliability, this scale is convenient and reliable for instructors and researchers to diagnose or assess students’ self-confidence in learning robotics before or after a curriculum.

Many future studies can be conducted using the RLSES developed in this study. For example, a confirmatory factor analysis (CFA) can be conducted to reconfirm the validity and reliability of this scale for different student groups. Comparisons among school levels can also be reexamined in the future. Researchers can further use the RLSES to examine the robotics curriculum designs on students’ robotics learning. In addition, the relationships between robotics learning self-efficacy and learning performance can be examined. Teachers’ robotics learning self-efficacy may also be examined and compared with students’ self-efficacy. The roles of demographic variables such as gender, learning experience, as well as socio-economic status can also be examined. Finally, since robotics learning is usually involved in problem-solving contexts, the roles of computational thinking (García-Peñalvo & Mendes, 2018; Leonard et al., 2016; Tsai et al., 2020) and design thinking dispositions (Tsai & Wang, 2020) in robotics learning self-efficacy may also be explored in the future.

Conclusion

Robotics learning is increasingly emphasized in contemporary school curricula; however, assessment tools for robotics learning are still limited in the literature. This study, based on Bloom’s Taxonomy of educational goals (Anderson et al., 2001; Bloom et al., 1956), developed the Robotics Learning Self-Efficacy Scale (RLSES) to assess students’ self-efficacy for robotics learning. Moreover, a five-factor (practice, comprehension, application, analysis, and collaboration) two-level (basic level vs. advanced level) framework of the RLSES construct was hypothesized and validated in this study. With good validity and reliability, this scale can be applied to all students from the elementary school upper division levels. School teachers can use it to quickly diagnose students’ self-confidence in robotics learning before and after instruction, and researchers can use this scale to conduct further validation analyses or to investigate the roles that more learning variables play in robotics learning. This scale may contribute more research insights and instructional innovations for future robotics-related curricula.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Ministry of Science and Technology in Taiwan (MOST 109-2511-H-003-019-MY3 and MOST 109-2511-H-003-018-MY3) and the ‘Institute for Research Excellence in Learning Sciences’ of National Taiwan Normal University (NTNU) from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan.

ORCID iDs

Meng-Jung Tsai

Ching-Yeh Wang

Author Biographies

Meng-Jung Tsai is a distinguished professor in the Program of Learning Sciences, National Taiwan Normal University. Her research centered at computer information literacy and technology-enhanced science learning, especially focusing on students’ learning strategies and information processing in digital learning contexts. She has been using eye-tracking technology to examine learners’ visual attention patterns in multimedia problem solving, online information search, and game-based learning environments. Recently, she is interested in exploring computational thinking and design thinking dispositions in young students’ self-efficacies of computer programming and robotics learning in STEAM curricula.

Ching-Yeh Wang is currently a postdoctoral research fellow in the Program of Learning Sciences, National Taiwan Normal University. He is interested in using eye-tracking technology to explore students’ online information reading such as conflicting information about socio-scientific issues and online multimedia recipe reading. He is also interested in exploring effective factors in digital learning such as self-efficacy, learning attitudes, and interests.

An-Hsuan Wu is a doctoral student at Graduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology. She is interested in examining visual patterns in E-book reading, multimedia learning, and socio-scientific-issue reading. She is currently working on investigating online controversial information evaluation using eye-tracking technology.

Chun-Ying Hsiao is a master student at Graduate Institute of Information and Computer Education, National Taiwan Normal University. She is interested in students’ computational thinking in robotics learning. She is also interested in students’ cognitive process in digital learning environment.

References

Afari

Khine

M. S.

(2017). Robotics as an educational tool: Impact of Lego Mindstorms. International Journal of Information and Education Technology, 7(6), 437–442. https://doi.org/10.18178/ijiet.2017.7.6.908

Alimisis

(2013). Educational robotics: Open questions and new challenges. Themes in Science and Technology Education, 6(1), 63–71. http://earthlab.uoi.gr/theste

Altin

Pedaste

(2013). Learning approaches to applying robotics in science education. Journal of Baltic Science Education, 12(3), 365.

Ananiadou

Claro

(2009). 21st century skills and competences for new millennium learners in OECD countries (OECD Education Working Papers, No. 41). OECD Publishing. https://dx-doi-org.web.bisu.edu.cn/10.1787/218525261154

Anderson

L. W.

Krathwohl

D. R.

Bloom

B. S.

(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. Longman.

Aristawati

F. A.

Budiyanto

Yuana

R. A.

(2018). Adopting educational robotics to enhance undergraduate students’ self-efficacy levels of computational thinking. Journal of Turkish Science Education, 15, 42–50. https://doi.org/10.12973/tused.10255a

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191

Bandura

(1992). Exercise of personal agency through the self-efficacy mechanism. In Schwarzer

(Ed.), Self-efficacy: Thought control of action (pp. 3–38). Hemisphere Publishing Corp.

Benitti

F. B. V.

(2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58(3), 978–988. https://doi.org/10.1016/j.compedu.2011.10.006

10.

Bers

M. U.

Flannery

Kazakoff

E. R.

Sullivan

(2014). Computational thinking and tinkering: Exploration of an early childhood robotics curriculum. Computers & Education, 72, 145–157. https://doi.org/10.1016/j.compedu.2013.10.020

11.

Bloom

B. S.

Engelhart

M. D.

Furst

E. J.

Hill

W. H.

Krathwohl

D. R.

(1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. David McKay Company.

12.

Chalmers

(2018). Robotics and computational thinking for primary school. International Journal of Child-Computer Interaction, 17, 93–100. https://doi.org/10.1016/j.ijcci.2018.06.005Get rights and content

13.

Chan

Y. H.

(2003). Biostatistics 101: Data presentation. Singapore Medical Journal, 44(6), 280–285. http://www.smj.org.sg/article/biostatistics-101-data-presentation

14.

Chen

I. S.

(2017). Computer self-efficacy, learning performance, and the mediating role of learning engagement. Computers in Human Behavior, 72, 362–370.

15.

Conde

M. Á.

Rodríguez-Sedano

F. J.

Fernández‐Llamas

Gonçalves

Lima

García‐Peñalvo

F. J.

(2020). Fostering STEAM through challenge-based learning, robotics, and physical devices: A systematic mapping literature review. Computer Applications in Engineering Education. Advance online publication. https://doi.org/10.1002/cae.22354

16.

Gagne

R. M.

Briggs

L. J.

(1974). Principles of instructional design. Holt, Rinehart & Winston.

17.

García-Peñalvo

F. J.

Mendes

A. J.

(2018). Exploring the computational thinking effects in pre-university education. Computers in Human Behavior, 80, 407–411. https://doi.org/10.1016/j.chb.2017.12.005

18.

Gaski

J. F.

Nevin

J. R.

(1985). The differential effects of exercised and unexercised power sources in a marketing channel. Journal of Marketing Research, 22(2), 130–142. https://dx-doi-org.web.bisu.edu.cn/10.2307/3151359

19.

Gomoll

Šabanović

Tolar

Hmelo-Silver

C. E.

Francisco

Lawlor

(2018). Between the social and the technical: Negotiation of human-centered robotics design in a middle school classroom. International Journal of Social Robotics, 10(3), 309–324. https://doi.org/10.1007/s12369-017-0454-3

20.

Herro

Quigley

Jacques

L. A.

(2018). Examining technology integration in middle school STEAM units. Technology, Pedagogy and Education, 27(4), 485–498. https://doi.org/10.1080/1475939X.2018.1514322

21.

Hsu

C.-Y.

Chiou

G.-L.

Tsai

M.-J.

(2019). Visual behavior and self-efficacy of game playing: An eye movement analysis. Interactive Learning Environments, 27(7), 942–952. https://doi.org/10.1080/10494820.2018.1504309

22.

S. H.

Tsai

M.-J.

(2019). Development and effects of an interdisciplinary robotics STEAM curriculum: A case study of a rural junior high school in Taiwan. International Journal on Digital Learning Technology, 11(4), 51–75 (in Chinese language). https://doi.org/10.3966/2071260X2019101104003

23.

Jordan

M. E.

McDaniel

R. R.

Jr. (2014). Managing uncertainty during collaborative problem solving in elementary school teams: The role of peer influence in robotics engineering activity. Journal of the Learning Sciences, 23(4), 490–536. https://doi.org/10.1080/10508406.2014.896254

24.

Khine

M. S.

(2017). Robotics in education: Redesigning the learning experience. Springer. https://doi.org/10.1007/978-3-319-57786-9

25.

Krathwohl

D. R.

Bloom

B. S.

Masia

B. B.

(1964). Taxonomy of educational objectives: The classification of educational goals. Handbook 2: Affective domain. Mckay.

26.

Leonard

Buss

Gamboa

Mitchell

Fashola

O. S.

Hubert

Almughyirah

(2016). Using robotics and game design to enhance children’s self-efficacy, STEM attitudes, and computational thinking skills. Journal of Science Education and Technology, 25(6), 860–876. https://doi.org/10.1007/s10956-016-9628-2.

27.

Lin

C. L.

Tsai

C. Y.

(2021). The effect of a pedagogical STEAM model on students’ project competence and learning motivation. Journal of Science Education and Technology, 30(1), 112–113. https://doi-org.ezproxy.lib.ntust.edu.tw/10.1007/s10956-020-09885-x

28.

Menekse

Higashi

Schunn

C. D.

Baehr

(2017). The role of robotics teams’ collaboration quality on team performance in a robotics tournament. Journal of Engineering Education, 106(4), 564–584. https://doi.org/10.1002/jee.20178

29.

Mubin

Stevens

C. J.

Shahid

Al Mahmud

Dong

J. J.

(2013). A review of the applicability of robots in education. Technology for Education and Learning, 1(1), 7. https://doi.org/10.2316/Journal.209.2013.1.209-0015

30.

Müller

N. M.

Seufert

(2018). Effects of self-regulation prompts in hypermedia learning on learning performance and self-efficacy. Learning and Instruction, 58, 1–11.

31.

Quigley

C. F.

Herro

Shekell

Cian

Jacques

(2020). Connected learning in STEAM classrooms: Opportunities for engaging youth in science and math classrooms. International Journal of Science and Mathematics Education, 18(8), 1441–1463. https://doi.org/10.1007/s10763-019-10034-z

32.

Roberts, N. (2004). Public deliberation in an age of direct citizen participation. American Review of Public Administration, 34(4), 315–353. https://doi.org/10.1177/0275074004269288

33.

Simpson

E. J.

(1966). The classification of educational objectives, psychomotor domain. Illinois Teacher of Home Economics, 10, 110–144. https://eric.ed.gov/?id=ED010368

34.

Slangen

van Keulen

Gravemeijer

(2011). What pupils can learn from working with robotic direct manipulation environments. International Journal of Technology and Design Education, 21(4), 449–469. https://doi.org/10.1007/s10798-010-9130-8

35.

Sun

J. C. Y.

Yeh

K. P. C.

(2017). The effects of attention monitoring with EEG biofeedback on university students’ attention and self-efficacy: The case of anti-phishing instructional materials. Computers & Education, 106, 73–82.

36.

Sung

Ahn

J. H.

Kai

S. M.

Black

(2017). Effective planning strategy in robotics education: An embodied approach. In Resta

Smith

(Eds.), Proceedings of society for information technology & teacher education international conference (pp. 1065–1071). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/177387/

37.

Toh

L. P. E.

Causo

Tzuo

P. W.

Chen

I. M.

Yeo

S. H.

(2016). A review on the use of robots in education and young children. Educational Technology & Society, 19(2), 148–163. https://www-jstor-org-443.web.bisu.edu.cn/stable/jeductechsoci.19.2.148

38.

Tsai

C.-C.

Chuang

S. C.

Liang

J.-C.

Tsai

M.-J.

(2011). Self-efficacy in internet-based learning environments: A literature review. Educational Technology & Society, 14(4), 222–240. https://www-jstor-org-443.web.bisu.edu.cn/stable/10.2307/jeductechsoci.14.4.222

39.

Tsai

M.-J.

(2009). The model of strategic E-Learning: Understanding and evaluating students' e-learning from metacognitive perspectives. Educational Technology & Society, 12(1), 34–48. https://www-jstor-org-443.web.bisu.edu.cn/stable/10.2307/jeductechsoci.12.1.34

40.

Tsai

M.-J.

Liang

J.-C.

Hsu

C.-Y.

(2020 ). The computational thinking scale (CTS) for computer literacy education. Journal of Educational Computing Research. Advance online publication. https://doi.org/10.1177/0735633120967326

41.

Tsai

M.-J.

Tsai

C.-C.

(2010). Junior high school students’ internet usage and self-efficacy: A re-examination of the gender gap. Computers & Education, 54(4), 1182–1192. https://doi.org/10.1016/j.compedu.2009.11.004Get rights and content

42.

Tsai

M.-J.

Wang

C.-Y.

(2020 ). Assessing young students’ design thinking disposition and its relationship with computer programming self-efficacy. Journal of Educational Computing Research. Advance online publication. https://doi.org/10.1177/0735633120967326

43.

Tsai

M.-J.

Wang

C.-Y.

Hsu

P.-F.

(2019). Developing the computer programming self-efficacy scale (CPSES) for computer literacy education. Journal of Educational Computing Research, 56(8), 1345–1360. https://doi.org/10.1177/0735633117746747

44.

Witherspoon

E. B.

Higashi

R. M.

Schunn

C. D.

Baehr

E. C.

Shoop

(2017). Developing computational thinking through a virtual robotics programming curriculum. ACM Transactions on Computing Education, 18(1), 1–20. https://doi.org/10.1145/3104982