Collaborative Robotics,More Than Just Working in Groups

Abstract

The purpose of this study was to determine what collaborative interventions produce positive effects for students working on collaborative robotics projects for science process skills, collaborative problem-solving, and learning motivation. In addition, the study examined the impact students’ prior robotics experience had on science process skills, collaborative problem-solving, and learning motivation. The results indicated experience level and collaboration interventions can have impacts on students. Assigned Group Roles had positive effects on students’ motivation and collaborative problem-solving. Experience level also had effects upon student learning motivation and collaborative problem-solving with the Novice status associated with higher levels as compared with students who had more experience. A collaboration intervention was identified that has the potential to produce positive effects for students in collaborative robotics projects as well as assist classroom educators in the purposeful design of collaborative robotics projects with scientifically based strategies to improve the attitudinal outcomes for students of various robotics experience.

Keywords

collaborative robotics learning motivation collaborative problem-solving science process skills

Introduction

Robotics has been used for educational purposes since the 1980s (Bers, 2010; Castledine & Chalmers, 1993; Chambers, Carbonaro, Rex, & Grove, 2007; Papert, 1993). Educational robotics provides a fun and developmentally appropriate way to teach technology and engineering to students of all ages (Bers, 2010; Slangen, Keulen, & Gravemeijer, 2010; Sullivan & Bers, 2016). A variety of content areas, as well as social skills, can also be taught using educational robotics (Eguchi, 2012; Grandgenett, Ostler, Topp, & Goeman, 2012; Hwang & Wu, 2014; Sullivan & Bers, 2016). Typical goals for these learning tools include generating student interest in technology through robotic activities or lessons and engaging students in learning while teaching difficult or abstract concepts through nontraditional methods (Eguchi, 2012). Learning with robotics can facilitate student collaboration, problem-solving, and critical thinking (Jordan & McDaniel, 2014; Mills, Chandra, & Park, 2013). Furthermore, robotics instruction can reflect real-world research where complex problems are solved in collaboration with others (Karahoca, Karahoca, & Uzunboylub, 2011; Mills et al., 2013; Papert, 1993; Robinson, 2005). The potential benefits for educational robots move beyond classroom academics if students are able to develop real-world problem-solving skills (Mills et al., 2013; Papert, 1993; Sullivan, 2008). This allows them to make connections between abstract content areas through concrete hands-on robotics, negotiate and interact in collaborative problem-solving environments, and develop skills that benefit them in a variety of real-world situations, all within the educational setting (Mills et al., 2013; Papert, 1993; Sullivan, 2008). Collaborative robotics projects have the potential to use robotics as an educational tool that combines hands-on learning in a collaborative environment and provides the opportunity for students to develop learning motivation, collaborative problem-solving, and scientific process skills.

While it is common for students working with robotics to have challenges with programming and the mechanics (Ucgul & Cagiltay, 2014), lack of successful collaboration may limit beneficial learning outcomes and motivational benefits from the robotics projects. Difficulties with sharing ideas and equally dividing the workload during the hands-on experience seem to limit the potential for positive benefits for students. In addition, different ability and experience levels can make it challenging to comfortably share ideas within a group. However, for educational purposes, collaborative robotics projects can be used to build the skills of each student in all areas of the project, designing, building, and programming whether it is an existing strength or not. The collaborative environment is a necessity in robotics instruction due to available resources in the engineering lab and is even recommended for group work (Eguchi, 2012; Mills et al., 2013; Yuen et al., 2014). Nevertheless, it seems group collaboration sometimes leaves students with fewer participation opportunities, less learning motivation, or with a less than enjoyable experience.

Learning motivation during collaborative projects may vary depending upon the group and its collaborative success. Past experiences in the elementary engineering lab indicated that groups that struggle to work collaboratively seem to have a more negative view of collaborative projects. Students may have not felt as though their ideas were listened to, or may have not had an opportunity to work on an area of the project that interested them. Perhaps collaboration strategies can be implemented that would promote a more productive collaboration process to aid students in achieving learning objectives and increasing benefits from the collaborative nature of robotics instruction. Eguchi (2012) notes the introduction of robots alone cannot influence students’ minds or directly influence their learning; therefore, changes have to be made in the learning environment to support the collaborative robotics projects.

The goal of this study was to implement collaborative instructional strategies, interventions for supporting group work to improve student learning motivation, collaborative problem-solving, and science process skills, when designing, building, and programming robotics solutions. While motivation is a complex construct, for the purpose of this study, learning motivation was defined as motivation for students to take an active role in and a responsibility for their own learning (Petre & Price, 2004). This may also include developing an interest in a subject area that previously had been of little or no interest. Collaborative problem-solving was defined as activities identifying problems and developing solutions through testing, improving, and using the collective ideas of a group (Barak & Zadok, 2007). Science process skills are defined as a set of skills used to systematically identify and answer scientific questions (Benitti, 2012). Experience levels were defined as novice level having no prior experience with collaborative robotics projects and experienced level having experienced at least one prior collaborative robotics project. Research questions in this study are the following: When controlling for students’ pretest scores, what are the effects of collaboration interventions and prior student experience with collaborative robotics on students’ (a) learning motivation, (b) problem-solving, and (c) science process skills?

Literature Review

Learning Motivation in Robotics

The hands-on nature and materials in the elementary engineering lab seem to provide sufficient motivation to some students (Petre & Price, 2004). However, researchers continually turn to instructional methods that involve a problem, meaningful units, and are cross-curricular to motivate and engage other students (Blumenfeld et al., 1991). Increases in student motivation are associated with constructivist and problem-based learning (Barak & Zadok, 2007; Bers, 2010; Eguchi, 2012; Papert, 1993; Somyürek, 2014). Moreover, integrated robotics projects combine educational tools with experiential learning to promote an increase in student learning motivation for Science, Technology, Engineering, and Mathematics (STEM) subjects (Blanchard, Freiman, & Lirrete-Pitre, 2010; Mohr-Schroeder, Little, & Schroeder, 2014; Petre & Price, 2004; Somyürek, 2014; Ucgul & Cagiltay, 2014). The role of facilitator becomes critical in robotics projects to help sustain student motivation and promote the desired higher level learning (Blumenfeld et al., 1991). When students have the necessary collaborative provisions to create a supportive classroom environment during robotics projects, the results can positively impact student learning motivation (Yuen et al., 2014). Student motivation contributes to successfully learning and retaining the content (Mohr-Schroeder et al., 2014). The use of robotics in the classroom has the potential to motivate students to learn; however, it must be noted that the introduction of robotics alone does not guarantee positive student impacts (Eguchi, 2012). Indeed, the teacher must shift to a new role and implement appropriate instructional practices to support the learning process, collaboration, and to promote the benefits of student learning and motivation (Blumenfeld et al., 1991; Yuen et al., 2014).

With teachers taking on new roles as facilitators in the shifting learning environment, students are also asked to take on new roles as collaborators. The new roles allow students to take a more active role in their own knowledge development when participating in constructivist and problem-based learning, which increases motivation (Barker & Ansorge, 2007; Blanchard et al., 2010; Chambers et al., 2007; Hwang & Wu, 2014; Papert, 1993). Not only do students have the opportunity to be active learners, they also have the potential to positively impact student learning and motivation through peer collaboration (Blumenfeld et al., 1991; Eguchi, 2012; Robinson, 2005). Small group settings for robotics necessitates social and peer interaction. With scaffolding and direction for students on successful collaboration, robotics has the potential to promote quality social interactions, which support successful collaboration and increase motivation (Denis & Hubert, 2001; Hwang & Wu, 2014). If the nature of robotics is not motivating enough for students, the collaborative nature of robotics projects, constructivist learning, and problem-based learning may offer social interactions that support and maintain student motivation (Slangen et al., 2010; Yuen et al., 2014). The collaborative robotics projects in the elementary engineering lab integrate characteristics of constructivist and problem-based learning that have been shown to motivate student learning and engage students in the learning process.

Collaborative Problem-Solving in Robotics

While there are opportunities in the elementary engineering lab to practice problem-solving individually, the fourth- and fifth-grade robotics projects rely heavily on collaborative problem-solving. Collaborative problem-solving is defined by Mills et al. (2013) as a process where peers construct new knowledge together that neither of them had prior knowledge of before working together. By practicing collaborative problem-solving in real-world applications with peer and teacher support, students may improve their problem-solving skills and possibly transfer learning across content areas (Jordan & McDaniel, 2014; Mills et al., 2013; Petre & Price, 2004). Research suggests that problem-solving engages students in the learning process, as opposed to more passive learning, and is essential for developing real-life skills (Eguchi, 2012; Jordan & McDaniels, 2014; Mills et al., 2013; Papert, 1993; Somyürek, 2014). Using the design process for identifying a problem and developing a robotic solution can increase students’ use of critical thinking and problem-solving skills (Barak & Zadok, 2007; Barker & Ansorge, 2007; Castledine & Chalmers, 1993; Jordan & McDaniel, 2014). In addition, researchers argue that acquiring problem-solving and critical thinking skills is essential for students’ futures (Castledine & Chalmers, 1993). Nelson (2012) emphasized there is more than content preparation needed to implement robotics projects, some of which are less tangible. If these less tangible preparations are addressed through the implementation of effective strategies, additional student benefits can be achieved with skills-transfer (Nelson, 2012).

Understanding the characteristics of collaborative problem-solving will assist in determining what scaffolds, less tangible preparations, would best support student learning. Because collaborative problem-solving is a form of problem-solving requiring peer interactions (Mills et al., 2013), students need to ask questions, gather information, and reflect on what they have learned to solve a problem (Somyürek, 2014). Although the process may seem simple enough, it requires complex skills (Somyürek, 2014). In fact, problem-solving is a complex phenomenon that uses both conscious and unconscious processes as well as combinations of explicit knowledge and intuition (Barak & Zadok, 2007). However, the process can be simplified for students by breaking problem-solving into a series of steps (Mills et al., 2013). Another potential support for collaborative problem-solving is the use of language to promote the collaboration and the development of newly coconstructed knowledge (Mills et al., 2013). The potential benefits for collaborative robotics projects to facilitate group work, problem-solving, and critical thinking may be supported through language scaffolds (Hwang & Wu, 2014; Mills et al., 2013). While collaborative problem-solving may prove to be a challenge for students because of the complexities, experiences with collaborative problem-solving allow students to develop group solutions to meet the common group goal (Denis & Hubert, 2001; Jordan & McDaniel, 2014; Mills et al., 2013). However, the uncertainties of collaborative problem-solving may also create barriers to students’ development of solutions (Jordan & McDaniels, 2014). Jordan and McDaniels (2014) determine that teacher and peer support is critical for managing the uncertainties of collaborative problem-solving. Students’ ability to reflect on and relate problem-solving strategies in relation to real-world contexts could boost confidence levels in the subject area (Castledine & Chalmers, 1993). Students may need the boost in confidence to feel comfortable sharing ideas and assisting with the project.

The collaborative problem-solving and the group work required with robotics projects may also make it more difficult to identify individual student progress. Teachers must closely monitor individual students, their understanding, and their performance to support successful projects (Eguchi, 2012). Papert (1993) identifies ways in which to guide student thinking but cautions that problem-solving cannot be as simple as memorizing a procedure, such as a math algorithm, because the variety of problems are always changing. In addition, Papert (1993) reminds us that students do not have to give up old methods to learn new ones. Furthermore, structured thinking is powerful thinking and is not a skill that all students develop when left to construct their own knowledge (Papert, 1993). The role of the teacher, as facilitator, then, is to develop the proper balance as well as organize the instructional plan to support the learning process. Providing students with the means to comfortably share and participate within the group can make collaborative problem-solving a better experience for all involved.

Science Process Skills in Robotics

While instruction in the elementary engineering lab focuses on engineering, the use of cross-curricular instruction is also a priority to develop connections for the students between the general classroom and the engineering lab. Nelson (2012) identified the scientific method and engineering as primary rationales for STEM education and robotics integration. Fortunately, integration of science skills fits easily within engineering and robotics instruction (Eguchi, 2012; Papert, 1993). For instance, robotics helps students master various concept areas, depending upon how the robotic instruction is developed. As a result, multiple student benefits are possible by using robotics as cross-curricular activities (Eguchi, 2012). The structure of collaborative robotics projects and robotic materials provide an opportunity to focus student learning on engineering (Bers, 2010; Jordan & McDaniel, 2014; Petre & Price, 2004; Ringwood, Monaghan, & Maloco, 2005; Yuen et al., 2014). With the cross-curricular potential of robotics, the use of a collaborative robotics project can easily combine science and engineering in addition to creating connections between the elementary engineering lab and the general classroom. Research identified three main skills developed through the use of robotics: thinking skills, science process skills/problem-solving skills, and social interaction/group work skills (Benitti, 2012; Hwang & Wu, 2014; Mills et al., 2013; Ringwood et al., 2005; Slangen et al., 2010). These skills are applicable and valuable for engineering, science, and the general classroom setting. In addition to the identified three main skills, Benitti (2012) noted robotics activities required the use of thinking skills and scientific reasoning. Furthermore, Sullivan (2008) maintained that an appropriate open-ended instructional approach, in conjunction with the use of robotics, promotes the use of thinking and science process skills, as well as increased systems understanding.

Science requires students to use language as a component of critical thinking and is necessary to understand and identify solutions for problems (Mills et al., 2013). Sullivan (2008) suggests that the process of debugging a program is an ideal format for teaching science process skills. Students generate hypotheses about what would work in the program, test it, and receive immediate feedback. The feedback starts as an iterative cycle of observation, hypothesis generation, testing of the hypothesis, and evaluation of the solution (Sullivan, 2008). This not only is an appropriate format for teaching science process skills but also emphasizes the real-world process scientists engage while offering a different exposure to science because typical classroom lessons do not have the iterative feedback loops (Sullivan, 2008). Sullivan (2008) makes clear connections between the scientific process and the engineering design process by arguing that students must control variables and change only one variable at a time. This is key in the use of the scientific method and science process skills. A study by Somyürek (2014) indicates that during robotics instruction, students learn by designing and programming robots to solve problems. They use scientific skills such as making predictions, generating a hypothesis, conducting experiments or tests, and presenting their results. Robotics provides a hands-on method for teaching critical thinking, science process skills, and support for learning abstract concepts (Eguchi, 2012; Papert, 1993; Slangen et al., 2010; Sullivan, 2008). Although the learning objectives for the collaborative robotics project may not connect specifically with science standards, implementation scaffolds and problem-based learning could assist the students in achieving similar beneficial learning outcomes.

Prior Experience in Robotics

Barak and Zadok’s (2007) comparison between expert and novice robotics problem solvers highlights the importance of experience level in effective design and problem-solving. While this is a factor that is out of the teacher’s control, it is an important factor to consider for structure of the instruction. Although some differentiation was provided in the past in the elementary engineering lab, based upon experience level, it may not have been sufficient to promote student success in collaborative robotics projects. Eguchi (2012) contends that it is very important to provide inexperienced students with supports, and in fact, emphasized the essential teacher’s role as facilitator to support inexperienced students. Teachers should provide modeling, guiding, and project planning and assist with the necessary skills and thought processes for students to successfully complete robotics projects (Barak & Zadok, 2007; Eguchi, 2012).

Each year the robotics project is slightly different in the elementary engineering lab with an emphasis on a different problem, that is, landing or mars, solving community problems, or creating robots that can create art. Fourth-grade students have not experienced a collaborative robotics group projct. They have worked with a partner in programming the robot but have not had to design, build, and program a working solution. Fifth-grade students experienced collaborative robotics project as fourth graders. The fifth-grade students have more prior knowledge and experiences to draw upon to help them successfully complete their robotics projects. However, students new to the school as fifth graders may not have collaborative robotics experience. Therefore, novice-level students have not completed a collaborative robotics project, while experienced-level students have completed at least one. With low turnover in the elementary grades, most groups had the same experience levels with only a few groups in fifth grade having students with mixed experience levels. Even groups with the same experience level would have had a variety of ability levels. The past instructional practices and the design of the study may not provide enough support for the inexperienced students because they are receiving the same interventions as the fifth-grade students and no other scaffolds.

The lack of additional supports for inexperienced students may impact potential benefit for the fourth-grade students in the study. According to Barak and Zadok (2007), students with varying experience levels tend to approach robotics problems from different perspectives. Inexperienced students may have difficulty in describing problems, which can hamper the success of the group in developing solutions, while experienced students may be able to use collaborative techniques such as being able to redescribe or redefine problems (Barak & Zadok, 2007; Blanchard et al., 2010). If a description of the problem is an issue, robotics vocabulary instruction or modeling may be beneficial to inexperienced students to alleviate possible barriers to problem-solving issues. Furthermore, modeling of appropriate discussion could provide support for inexperienced students to overcome any differences in perspectives. Another concern with the differences in experience levels is that novices may rely more on trial and error where experts use domain-specific strategies (Barak & Zadok, 2007; Papert, 1993). Blanchard et al. (2010) demonstrate that experienced students still may use trial and error but may also use it more efficiently base upon their prior knowledge. Experts are able to develop chunks of specialized knowledge that are transferable, while novices tend to memorize small disconnected facts (Barak & Zadok, 2007). The expert use of knowledge may allow for shortcuts or efficiency in problem-solving rather than having to follow a specific method from start to finish (Barak & Zadok, 2007; Blanchard et al., 2010). It is recommended that students within a group have similar levels of expertise so that one group member is not an expert, and thereby supporting the coconstruction of knowledge rather than expert to novice transmission of information, as in a teacher-directed situation (Barak & Zadok, 2007; Mills et al., 2013). The Mills et al. (2013) study also demonstrates that novices with no prior knowledge of building or programming a robot continue to improve their speed of problem-solving. Can this increase in speed or success of problem-solving be maintained at the experienced level? Perhaps novice students require more support and scaffolds to develop a successful collaborative process and support the growth of all students in collaborative problem-solving, learning motivation, and scientific process skills within the robotics project.

Method

Participants

The participants in this study consist of fourth- and fifth-grade students of six classes at a school in Eagle, Idaho, USA. Of the 91 fourth-grade students and 88 fifth-grade students the study started with, 42% were female and 58% were male. They range in age from 8 to 11. They attended engineering for a 1-hour class each week. The engineering teacher was also the primary researcher and had been teaching robotics in the elementary engineering classroom since 2012. Fourth-grade students have not experienced the robotics project in the elementary engineering lab. Therefore, the fourth-grade students were identified as the Novice in regard to completing a robotics project. Fifth-grade students who attended the school during the previous year have completed a similar project. Student groups within each classroom were established with random assignments. The number of groups with four student members totaled 33 groups at the beginning of the study and 35 upon completion of the study. Group size consisted primarily of four students with a few groups of three. Prior to the start of the study, permission for the study was secured through the school district and institutional review board approval. Institutional review board approval included an informed consent letter being sent home to all the parents of students involved in the study. The purpose of the study was shared with all the students emphasizing the process as a learning experience for the teacher/researcher rather than an assessment of the students. Identities of the students were also protected with a coded system that provided anonymity.

Robot Activities

Each group did robotics activities with LEGO MINDSTORMS EV3. The Mindstorms robotics platform uses LEGO pieces for construction along with a brick for controlling the robot. Programming is completed using a drag-and-drop software. Students in fourth- and fifth-grade continued to work on their programming skills with a partner at the beginning of the school year, advancing to the sensors and more complex programming, to refresh and build upon their skills from the previous year of instruction before starting the collaborative robotics project. Student participants completed their collaborative robotics project over 16 weeks for 1 hour each week. The progression from basic programming to complex designs and builds followed Ucgul and Cagiltay’s (2014) recommendation that content be organized from simple to complex. The learning sequence provided the necessary experiences and prior knowledge for students to take on the more complex collaborative robotics projects that required students to design, build, and program a robotics solution. Collaborative robotics projects were designed to reinforce and assist with learning in the general classroom by targeting science, math, language, and engineering standards. The use of the robots promoted the cross-curricular aspect of STEM in real-world practices. The use of the Engineering Notebooks and the presentation of their group solution to their peers reinforced language standards, promoted student reflection, and coconstruction of knowledge in group projects. Engineering Notebooks are used for all projects in the elementary engineering classroom to record relevant information and progress notes. The process of identifying problems and developing solutions promoted science process skills and engineering practices. The collaborative robotics project required students to use what they have learned about robots and fostered the development of new knowledge to design, build, and program an original solution to a problem they identified based upon the criteria given. Students had the option of using instructions to design the base of their robotic solutions but were required to add an original working part or modifications.

Interventions

The first collaborative support used classroom discussions including establishing expectations and developing a safe environment for sharing ideas. Additional supports were included throughout the project to reinforce the expectations and promote effective discussions both within the group and as a whole class. Restating someone else’s thoughts, increasing wait time, partner talk, and encouraging many contributors were strategies used for the intervention, modeled after the classroom strategies in Classroom Discussions Using Math Talk to Help Children Learn (Chapin, O’Connor, & Canavan Anderson, 2012). Whole class discussions provided the teacher time to model strategies and assist students in developing the strategies, with redirection if necessary. The teacher also had the opportunity to monitor group discussions by visiting groups during the class time. Checking in with each group allowed the teacher to assess how the group discussions were progressing and provided opportunities for additional modeling and redirecting within a small group setting rather than the whole class (Mills et al., 2013). The goal was to model and promote collaborative discussion techniques and assist students in implementing these strategies in their collaborative robotics groups.

The second intervention was assigning group roles within the project that rotated throughout the course of the project, giving all the group members equal opportunity to develop a variety of skills. The roles consisted of a time manager, materials manager, project manager, and data manager. The time manager assisted the group in monitoring the time they had available in comparison with the tasks to be accomplished for the class time. The materials manager was responsible for gathering and maintaining the materials needed for the project, the robotics kit, and other miscellaneous items. The project manager was responsible for the big picture of the project and identifying tasks that needed to be accomplished for the completion of the project. The data manager recorded any data and notes pertaining to the project. The assigned roles with specific jobs, in addition to the scheduled rotations, were designed to promote a process for the equal distribution of work and the opportunity for all students to develop the skills from all the roles. Groups with only three students combined two of the roles. It was controlled not to play the role in the assigned group role.

The third intervention was the previous instructional practice. Two classrooms, one fourth- and one fifth-grade classroom, did not receive the structured classroom discussion nor use assigned group roles but rather continued with the previous instructional practice as outlined in Table 1. Although the teacher/researcher did not bring in aspects from the first two interventions, aspects such as discussion and group roles could develop if occurring naturally through student interaction. In addition, groups that were struggling received extra time from the teacher/researcher to help students identify challenges and potential solutions rather than implement interventions that were not appropriate for the group.

Table 1.

Collaborative Robotics Learning Sequence.

Lesson	Activities
1	Complete pretest, introduce challenge, students brainstorm problems to solve, and identify possible solutions to the problem the group chooses
2	Groups receive robotics kits, inventory kits, continue developing problem and solution
3	Begin designing and building robotic solution
4	Continue designing and building robotic solution
5	Continue designing and building robotic solution
6	Begin programming and testing
7	Continue testing and improving program and design
8	Continue testing and improving program and design
9	Continue testing and improving program and design, develop student-generated evaluation rubrics
10	Continue testing and improving program and design, finalize student-generated evaluation rubrics
11	Continue testing and improving program and design, begin preparing presentation
12	Continue testing and improving program and design, continue preparing presentation
13	Continue testing and improving program and design, finalize presentation
14	Finalize and practice project demonstration and presentation
15	Present and demonstrate robotic solution
16	Wrap-up, students evaluate themselves and their fellow group members, complete posttests, inventory robotics kits

Students were given the basic criteria for the robotics project, brief reminders, and potential tasks that needed to be accomplished for each class, and any questions were addressed. They were reminded to share the work equally, make sure all group members got equal turns building and programming the robot, and given reinforcements for these aspects as needed. Not all students were able to program or build on the robot at the same time. Therefore, these activities rotated through all group members in a method established by each group with the understanding that the time should be divided equally among the group members and everyone should work on the programming or building within each class time. It is possible, even likely, discussions and group roles took place in all student groups. However, the additional supports and scaffolds from the teacher/researcher were only provided in the appropriate intervention.

Measures

Robotics Expo 2012 [Pre CEENbot]—Adapted 2015 Student Survey

Students also completed the Robotics Expo 2012 [Pre CEENbot] (Grandgenett, Chen, & Timms, 2010)—Adapted 2015 Student Survey before and after the robotics project to assess student learning motivation and collaborative problem-solving. It consisted of 38 questions using a 5-point Likert scale to have participants self-assess for collaborative problem-solving and motivation.

Sample survey question for collaborative problem-solving:

I am able to help my group to accomplish the task within the allocated time frame.

Collaborative problem-solving had 22 questions at 5 points each question for a possible full score of 110 points. Motivation had 16 questions at 5 points each question for a possible full score of 80 points.

Sample survey question for learning motivation:

It is important for me to learn how to conduct a scientific investigation.

The same items were used twice for the pre- and posttest as well. The items were shuffled to avoid possible memorized answers. The survey had a Cronbach’s alpha reliability coefficient of .94 (Grandgenett et al., 2010). In the present study, the 22 questions for collaborative problem-solving had a Cronbach’s alpha reliability coefficient of .87. The 16 questions for learning motivation had a Cronbach’s alpha reliability coefficient of .88.

Fowler Science Process Skills Assessment

The Fowler Science Process Skills Assessment (Fowler, 1990) was administered. This assessment consists of two sets of test: a pretest of Form A and a posttest of Form B. An administrator read the problem loud while students look at the paper.

Sample Fowler Prompt for Science Process Skills:

Today you are going to complete a short exercise to see how well you can design an experiment. Look at your paper while I read the problem aloud:

(Form A) Are earthworms attracted to light? In other words, do earthworms like light?

Tell how you would test this question. Be as scientific as possible when you write about your test.

Write down the steps you would take to find out if earthworms like light. You may draw a picture of your experiment, but you still need to explain your experimental design in words.

Students wrote down the steps and explained their experimental design for the given problem. Two teachers marked an answer sheet of one student. A score of 0 on an item indicated no evidence and a score of 3 indicated strong evidence. The mean of the two scores was used as the final score. It could range from 0 to 45 points. The validity and the reliability of the Fowler Science Process Skills Assessment as a measure of science process skills has been previously established (Callahan, Hunsaker, Adams, Moore, & Bland, 1995; Fowler, 1990; Mallozzi & Heilbronner, 2013). Furthermore, the intrarater reliability Pearson product-moment correlation coefficient of .89 for the pretest and .91 for the posttest was comparable with other science performance assessments (Callahan et al., 1995).

Research Design

Figure 1 illustrates the research design for this study. Each of the experience levels has groupings for the three levels of intervention. This creates six groups of participants, Novice/Classroom Discussion, Novice/Assigned Group Roles, Novice/Usual Instructional Practices, Experienced/Classroom Discussion, Experienced/Assigned Group Roles, and Experienced/Usual Instructional Practices. The Novice groups were the three fourth-grade classes, while the Experienced groups were the three fifth-grade classes. The elementary engineering teacher/researcher worked with all grade levels, classes, and groupings.

Figure 1.

Research design.

Statistical Analysis

Data analysis was performed in SPSS 21 to determine answers to the main research questions. To determine possible interactions, 2 × 3 analysis of covariance (ANCOVAs) were used for both experience levels and the three interventions while controlling for students’ pretest scores. The ANCOVA procedure was applied because three dependent variables—science process skills, collaborative problem-solving, and learning motivation—were not correlated with one another in pretest scores. The Pearson’s correlation coefficients for learning motivation and collaborative problem-solving, for learning motivation and science process skills, and for collaborative problem-solving and science process skills were .45, −.01, and .03, respectively. Pretest scores for science process skills, collaborative problem-solving, and learning motivation were used as covariates. Tukey honest significant difference followed the results from the ANCOVAs to determine which differences were significantly different.

Results

Research Question 1

Table 2 indicates the means of pre- and postscores for three interventions by experience levels. Classroom Discussion had a premean score of 66.91 and a postmean score of 65.33. Usual Instructional Practices had a premean score of 67.51 and a postmean score of 65.25. Only Assigned Group Roles indicated an increase in postmean scores, which was 1 point increase from 68.14 to 69.14. When examining the scores by intervention condition and experience level, both Experienced Classroom Discussion and Usual Instructional Practices had greater than 4 point decrease. Therefore, only the intervention of Assigned Group roles in the Experienced students demonstrated an increase in the postscore for motivation according to Table 2.

Table 2.

Means, and Standard Deviations of Motivation Scores.

Experience level	Interventions	N	Pretest		Posttest
Experience level	Interventions	N	M	SD	M	SD
Experienced	Classroom Discussion	25	68.24	6.20	63.56	9.08
	Assigned Group Roles	26	66.65	8.09	66.70	7.22
	Usual Instructional Practices	25	66.84	8.86	61.93	9.89
	Total	76	67.24	7.72	64.06	8.73
Novice	Classroom Discussion	28	65.57	6.06	67.10	6.48
	Assigned Group Roles	27	69.63	7.35	71.57	7.00
	Usual Instructional Practices	23	68.17	8.30	68.56	6.68
	Total	78	67.79	7.24	69.08	6.72
Total	Classroom Discussion	53	66.91	6.13	65.33	7.78
	Assigned Group Roles	53	68.14	7.72	69.14	7.11
	Usual Instructional Practices	48	67.51	8.58	65.25	8.29
	Total	154	67.52	7.48	66.57	7.73

The means of the postsurvey for the Novice demonstrated over 1 point increase, while the Experienced demonstrated a decrease of more than 3 points. In addition, the postmean score for the Novice (69.08) was more than 5 points higher than the Experienced (64.06). The Novice Classroom Discussions and Assigned Group Roles for student learning motivation demonstrated increases for postmean scores. Therefore, at the Novice level, all three interventions produced a positive impact on student motivation as indicated in Table 2.

To test the differences discovered in Table 2 for statistical significance, a 2 × 3 ANCOVA was applied with the premotivation scores as covariate. The ANCOVA indicated statistically significant main effects of the interventions for motivation after controlling for students’ pretest scores, F(2, 135) = 5.24, p = .006, p < .05, as seen in Table 3. The ANCOVA also indicated statistically significant main effects of the experience level after controlling for students’ pretest scores for motivation, F(1, 135) = 24.97, p = .000, p < .05. There was no interaction between intervention and experience level, F(2, 135) = .77, p = .463, p > .05.

Table 3.

ANCOVA for Posttotal Motivation.

Source	Type III sum of squares	df	MS	F	Sig.
Corrected model	3744.17^a	6	624.03	13.03	.000
Intercept	1714.65	1	1714.65	35.81	.000
Pretotal motivation	1704.56	1	1704.56	35.60	.000
Interventions	502.02	2	251.01	5.24	.006
Experience level	1195.42	1	1195.42	24.97	.000
Interventions × Experience Level	74.15	2	37.07	0.77	.463
Error	6464.12	135	47.88
Total	642431.00	142
Corrected total	10208.29	141

Note. ANCOVA = analysis of covariance.

^aR²= .367 (adjusted R²= .339).

In examining the motivation levels by intervention and experience level, it can be concluded that there is a statistically significant main effect for the interventions and experience levels. Tukey honest significant difference post hoc analysis in Table 4 indicated that the mean motivation score for Assigned Group Roles were statistically different in motivation, p = .011, p < .05 than Classroom Discussion and Usual Instructional Practices, p = .004, p < .05. Taken together, these results indicate that the use of Assigned Group Roles had an effect on student motivation. Specifically, students in Assigned Group Roles had higher postmean motivation scores when controlling for students’ premotivation scores than the students in Usual Instructional Practices with a mean difference of 4.27 and Classroom Discussion with a mean difference of 3.65.

Table 4.

Post Hoc Test for Posttotal Motivation.

(I) Intervention	(J) Intervention	Mean difference (I−J)	SE	Sig.^a	95% Confidence interval for difference^a
(I) Intervention	(J) Intervention	Mean difference (I−J)	SE	Sig.^a	Lower bound	Upper bound
Classroom Discussion	Assigned Group Roles	−3.65*	1.408	.011	−6.430	−0.860
Classroom Discussion	Usual Instructional Practices	0.62	1.445	.668	−2.237	3.477
Assigned Group Roles	Classroom Discussion	3.65*	1.408	.011	0.860	6.430
Assigned Group Roles	Usual Instructional Practices	4.27*	1.442	.004	1.414	7.116
Usual Instructional Practices	Classroom Discussion	−0.62	1.445	.668	−3.477	2.237
Usual Instructional Practices	Assigned Group Roles	−4.27*	1.442	.004	−7.116	−1.414

Note. Based on estimated marginal means.

^aAdjustment for multiple comparisons: least significant difference (equivalent to no adjustments).

*The mean difference is significant at the .05 level.

Research Question 2

Table 5 indicates the mean pre- and posttest scores for collaborative problem-solving. Table 5 identifies Classroom Discussion and Assigned Group Roles as having a positive effect with increased postmean scores. Classroom Discussion increased from 89.60 to 90.32, while Assigned Group Roles increased from 92.29 to 93.07. Although these increases are less than 1 point, Usual Instructional Practices demonstrated a decrease of over 2 points from 92.61 to 89.35. Experienced Classroom Discussion, Novice Classroom Discussion, and Novice Assigned Group Roles demonstrated a positive impact on collaborative problem-solving. The increase is greatest for Novice Assigned Group Roles. Experienced Assigned Group Roles was less than .1 point of a decrease. Experienced Usual Instructional Practices experienced the largest decrease with 5.68 points difference between the pre- and postmean scores for collaborative problem-solving.

Table 5.

Means, and Standard Deviations of Collaborative Problem-Solving Scores.

Experience level	Interventions	N	Pretest		Posttest
Experience level	Interventions	N	M	SD	M	SD
Experienced	Classroom Discussion	25	89.36	8.78	90.08	8.82
	Assigned Group Roles	26	90.62	9.68	90.54	6.65
	Usual Instructional Practices	27	90.57	9.10	84.89	9.98
	Total	78	90.18	9.19	88.50	8.48
Novice	Classroom Discussion	29	89.83	7.73	90.57	8.49
	Assigned Group Roles	26	93.96	10.51	95.59	8.60
	Usual Instructional Practices	25	94.65	9.44	93.80	8.60
	Total	80	92.81	9.23	93.32	8.56
Total	Classroom Discussion	54	89.60	8.26	90.32	8.66
	Assigned Group Roles	52	92.29	10.10	93.07	7.63
	Usual Instructional Practices	52	92.61	9.27	89.35	9.29
	Total	158	91.50	9.201	90.91	8.53

The postmean scores for the Novice demonstrated a slight increase for collaborative problem-solving from 92.81 to 93.32. The postmean scores for the Experienced indicated a decrease from 90.18 to 88.50. The Experienced only had positive impact for Classroom Discussion, while the Novice had positive impact for both Classroom Discussion and Assigned Group Roles.

To test these differences statistically, a 2 × 3 ANCOVA was applied with the precollaborative problem-solving scores as covariate. The ANCOVA indicated statistically significant main effects of interventions after controlling for students’ pretest scores for collaborative problem-solving, F(2, 140) = 5.09, p = .007, p < .05, as seen in Table 6. The ANCOVA also indicated statistically significant main effects of the experience levels after controlling for students’ pretest scores for collaborative problem-solving, F(2, 140) = 18.51, p = .000, p < .05. There was no statistically significant interaction between intervention and experience level, F(2, 140) = 2.35, p = .099, p > .05.

Table 6.

ANCOVA for Posttotal Collaborative Problem-Solving.

Source	Type III sum of squares	df	MS	F	Sig.
Corrected model	4835.97^a	6	805.99	17.49	.000
Intercept	3032.65	1	3032.65	65.81	.000
Pretotal collaborative problem-solving	2803.29	1	2803.29	60.83	.000
Interventions	469.09	2	234.55	5.09	.007
Experience level	852.93	1	852.93	18.51	.000
Interventions × Experience Level	216.24	2	108.12	2.35	.099
Error	6451.93	140	46.09
Total	1227867.00	147
Corrected total	11287.89	146

Note. ANCOVA = analysis of covariance.

^aR²= .428 (adjusted R²= .404).

Examining the collaborative problem-solving levels by intervention and experience level, it can be concluded that there is a statistically significant main effect for the interventions and experience level. A post hoc analysis for the three interventions using Tukey HSD was performed to determine which differences were statistically significant. Table 7 indicate that the mean collaborative problem-solving score for Assigned Group Roles were statistically different than Usual Instructional Practices, p = .014, p < .05. Taken together, these results indicate that the use of Assigned Group Roles has an effect on collaborative problem-solving. Specifically, students in Assigned Group Roles had higher postmean collaborative problem-solving scores when controlling for students’ precollaborative problem-solving scores than students in both Classroom Discussion with a mean difference of 2.06 and Usual Instructional Practices with a mean difference of 3.98. However, there was only a statistically significant difference between Assigned Group Roles and Usual Instructional Practices.

Table 7.

Post Hoc Test for Posttotal Collaborative Problem-Solving.

(I) Interventions	(J) Interventions	Mean difference (I−J)	SE	Sig.^a	95% Confidence interval for difference^a
(I) Interventions	(J) Interventions	Mean difference (I−J)	SE	Sig.^a	Lower bound	Upper bound
Classroom Discussion	Assigned Group Roles	−2.06	1.53	.182	−5.09	0.97
Classroom Discussion	Usual Instructional Practices	1.92	1.57	.223	−1.19	5.03
Assigned Group Roles	Classroom Discussion	2.06	1.53	.182	−0.97	5.09
Assigned Group Roles	Usual Instructional Practices	3.98*	1.60	.014	0.81	7.15
Usual Instructional Practices	Classroom Discussion	−1.92	1.57	.223	−5.03	1.19
Usual Instructional Practices	Assigned Group Roles	−3.98*	1.60	.014	−7.15	−0.81

Note. Based on estimated marginal means.

^aAdjustment for multiple comparisons: least significant difference (equivalent to no adjustments).

*The mean difference is significant at the .05 level.

Table 8.

Means, and Standard Deviations of Science Process Skills Scores.

Experience level	Interventions	N	Pretest		Posttest
Experience level	Interventions	N	M	SD	M	SD
Experienced	Classroom Discussion	26	2.12	1.28	2.89	1.67
	Assigned Group Roles	29	3.21	1.74	4.17	1.49
	Usual Instructional Practices	28	2.71	1.70	3.00	1.46
	Total	83	2.68	1.57	3.35	1.54
Novice	Classroom Discussion	31	1.77	1.06	2.79	1.29
	Assigned Group Roles	31	1.48	1.03	2.72	1.33
	Usual Instructional Practices	30	1.80	1.42	2.83	1.29
	Total	92	1.68	1.17	2.78	1.30
Total	Classroom Discussion	57	1.95	1.17	2.84	1.48
	Assigned Group Roles	60	2.35	1.39	3.45	1.41
	Usual Instructional Practices	58	2.26	1.56	2.92	1.38
	Total	175	2.19	1.37	3.07	1.42

Research Question 3

Table 9 indicates that all interventions for all the experience levels had increases in the postmean scores for science process skills with relation to the premean scores. However, higher gains were seen within the Novice with all the postmean scores showing an increase of at least 1 point from 1.68 to 2.78. Assigned Group Roles also demonstrated an increase in postmean scores for science process skills from 2.35 to 3.45 as seen in Table 8.

Table 9.

ANCOVA for Posttotal Science Process Skills.

Source	Type III sum of squares	df	MS	F	Sig.
Corrected model	120.71^a	6	20.12	12.49	.000
Intercept	199.64	1	199.64	123.89	.000
Pretotal scientific process skills	75.55	1	75.55	46.88	.000
Interventions	7.19	2	3.60	2.23	.111
Experience level	0.40	1	0.40	0.25	.619
Interventions × Experience Level	6.95	2	3.48	2.16	.119
Error	270.72	168	1.61
Total	2027.00	175
Corrected total	391.43	174

Note. ANCOVA = analysis of covariance.

^aR²= .308 (adjusted R²= .284).

To test these differences statistically, a 2 × 3 ANCOVA was applied with prescience process skills score as the covariate. The ANCOVA indicated no statistically significant main effects of interventions after controlling for students’ pretest scores for science process skills, F(2, 168) = 2.23, p = .11, p > .05, as seen in Table 9. The ANCOVA also indicated no statistically significant main effects of the experience levels after controlling for students’ prescience process skills scores, F(2, 168) = .248, p = .619, p > .05. There was no statistically significant interaction between intervention and experience level, F(2, 168) = 2.16, p = .119, p > .05.

Discussion

The findings for the collaborative robotics project demonstrated that Assigned Group Roles with an increase produced statistically significant positive effects on measured student motivation in relation to Classroom Discussion and Usual Instructional Practices, which both demonstrated decreases from pre- to postmean scores. Only Assigned Group Roles produced statistically significant positive effects on measured collaborative problem-solving in relation to Usual Instructional Practices. The Novice experience level was associated with statistically significant positive effects on measured student motivation and collaborative problem-solving in relation to the Experienced level. All interventions demonstrated positive effects on science process skills but not at a statistically significant level.

The Assigned Group Roles intervention included teacher reinforcement and a document outlining each role, Time Manager, Materials Manager, Project Manager, and Data Manager, as well as detailed job responsibilities. In the present study, the roles were also designed to balance quiet and dominant group members, as encouraged by Somyürek (2014). Rotating the roles weekly was established to further promote equal distribution of the work responsibilities. The rotation of the roles provided each student in the group to experience the different roles rather than remaining in a single role the entire project. This may have helped students develop empathy and understanding of the various roles, as well as, an understanding of their own strengths and weaknesses. The study’s positive results support Yuen et al.’s (2014) emphasis of structure during collaborative group work. The four roles included some responsibilities that were the same, such as researching, designing, building, and programming. In addition to each role having identical responsibilities, each role had unique responsibilities created to develop and promote collaboration between the four roles. The division and sharing of job responsibilities created structure for the Assigned Group Roles that supports Eguchi (2012) and Mills et al. (2013) recommendations for collaborative robotics work. The roles were developed for educational purposes to develop a wide range of skills in each student. Students were required to work on all areas of the collaborative robotics project rather than focusing solely on one area or on any areas of strength they may have had. While this use of group roles is contrary to Yuen et al. (2014) recommendations of focusing on students’ areas of strength, the roles are used to promote new skills, as well as, developing existing areas of strength. Yuen et al. (2014) addressed collaborative robotics in an informal educational setting and had different instructional purposes for group roles. Furthermore, the rotation of the roles may have increased motivation because students did not become complacent in one role and created a stronger support for collaborative problem-solving. The positive results indicate that structure of the roles provided the necessary support and scaffold to meet the needs of the classroom setting while developing motivation and collaborative problem-solving.

The Novice experience level was associated with a clear positive effect on motivation and collaborative problem-solving. Interestingly, research with experience levels in robotics suggests that the Novice students would have needed more supports than more experienced robotics students to recognize the same benefits (Eguchi, 2012), and yet, the Novice demonstrated more positive effects. Although the positive effects for the Novice were less significant, the Experienced had a decrease. Perhaps the novelty of the collaborative robotics project could be responsible for higher postmean motivation scores. However, it is difficult to identify potential reasons for the differences in postmean collaborative problem-solving scores. Postmean scores for collaborative problem-solving should have been higher for the Experienced, and yet, the Experienced students showed a decrease in postmean scores, while the Novice students had a slight increase. Perhaps additional supports and interventions need to be identified to continue the growth of these skills for Experienced students. In addition, students had to self-evaluate on the Robotics Expo 2012 [Pre CEENbot]—Adapted 2015 Student Survey measure. Students with experience with collaborative robotics projects may have overestimated their ability to complete the task in groups and had more realistic understanding during the postevaluation.

The lack of any statistically significant difference in postmean scores for science process skills for interventions and experience level may be due to the overall low scores. With the potential for 45, student scores ranged from 0 to 9. The low scores may not have allowed for large enough difference to successfully identify a statistical difference between the interventions. The Fowler Science Process Skills assessment was designed for middle school students. While some middle schools include fifth-grade students, the instrument may not have been the most appropriate measure for students in fourth and fifth grades. The potential issue with the instrument appropriateness may have made it difficult to identify effects of the interventions or experience levels.

Conclusion

While the general method of instruction for collaborative robotics projects had promoted student benefits, there was a need to evaluate the general method of instruction and determine if collaborative interventions could improve student outcomes. In addition, with students using a form of robotics as early as first grade and the Mindstorms as early as third grade, understanding how to support the developing experience levels is critical for ongoing robotics instruction and development of an appropriate scope and sequence.

The results indicate that the Assigned Group Roles had a positive effect on student motivation and collaborative problem-solving. While Assigned Group Roles had not been previously used in the elementary engineering classroom for previous projects, it will be used in future instruction and adapted for use with other collaborative projects. Not all groups used the roles as effectively, and some still struggled with the challenge of sharing the work and staying on task. However, the structure offered with the Assigned Group Roles seemed to provide students with appropriate guidance to stay on task if desired. Furthermore, the structure provided the teacher with appropriate guidance to assist students who may have been off task with a simple means of redirection. In the present study, Assigned Group Roles was the only intervention that had printed guidelines for students to keep in their Engineering Notebooks. Perhaps a printed guideline for classroom discussions could be included in the intervention to offer appropriate sentence starters, questions, and reminders about classroom interaction expectations to better support that collaboration strategy.

Robotics experience level also played an important part in student motivation and collaborative problem-solving. The status of Novice was associated with positive effects and consistently outperformed the status of Experienced. Understanding the supports needed for the varying experience levels is important to implement the appropriate instructional practices. It appears that additional supports are necessary for the Experienced students for them to continue to make gains in learning motivation and collaborative problem-solving. Postmean motivations scores showed only an increase in the Experienced for Assigned Group Roles, and the increase was minimal. Only Classroom Discussion from the Experienced had increased postmean scores for collaborative problem-solving; however, not at a significant level. Overall, additional investigation into supports for Experienced students is warranted.

The identification of successful collaboration intervention for collaborative robotics projects fills a practical and growing need for robotics integration into the educational setting. In fact, the relevance of the study to student learning also added to the rigor (Reeves, 2011). This study was relevant to teaching, learning, and educational outcomes, as it addresses the educational needs of learners, practitioners, designers, and society by promoting collaborative problem-solving skills and learning motivation (Reeves, 2011). Furthermore, the study has the potential to provide instructional insights for various problem-based collaborative environments.

The study was conducted in an authentic contextual setting to determine how best to support collaborative robotics projects. Additional studies could be conducted to determine how to develop classroom discussions to produce a more consistent benefit to students. Using claims, evidence, and reasoning in discussions to construct explanations may support the collaborative nature of the robotics project by reducing the potential for personal opinion to enter the discussions. Furthermore, the claims, evidence, and reasoning discussion format supports the systematic scientific approach desired for scientific process skills. Further research could also be conducted to improve upon the Assigned Group Roles. Perhaps it would be possible to improve the size of the increase in postmean scores by combining the two interventions. Future research has the potential to benefit not only robotics instruction but collaborative learning as well.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Kellie Taylor has been the Kindergarten through Fifth Grade Engineering Teacher at Galileo STEM Academy since 2012. She began her teaching career in 2004 and earned her masters and doctorate in educational technology while teaching fulltime. Kellie has shared her work in the classroom with robotics and 3D printing at various conferences and teacher workshops since 2013. She has been recognized as a 2014 State Finalist for the Presidential Award for Excellence in Mathematics and Science Teaching and received the 2016 K-6 Industry’s Excellent Educators Dedicated to STEM award for her work integrating industry into the classroom and providing real-world science experiences. Kellie continues to grow professionally by attending conferences and educator institutes. Furthermore, while developing her professional abilities and the abilities of her students, Kellie works to conduct research on best practices for STEM implementation and collaborative STEM projects in the elementary classroom.

Youngkyun Baek has been a professor of educational technology at Boise State University since 2011. Before joining Boise State, he also was a professor at Korea National University of Education, beginning in 1991. He teaches “Digital Engagement for Learning,” “Digital Game Design for K-12 Classrooms,” “Advanced Educational Game Design,” and “Autonomous Robotics for Teaching and Learning.” Dr. Baek has published several editions on game-based learning and written articles on digital and mobile learning games. His research interests are digital game design, mobile learning games, and new media & computational thinking. Currently, he is developing Robotics and Games for computational thinking enhancement of students.

References

Barak

Zadok

(2007) Robotics projects and learning concepts in science, technology and problem solving. International Journal of Technology and Design Education 19(3): 289–307. doi:10.1007/s10798-007-9043-3.

Barker

B. S.

Ansorge

(2007) Robotics as means to increase achievement scores in an informal learning environment. Journal of Research on Technology in Education 39(3): 229–243.

Benitti

F. B. V.

(2012) Exploring the educational potential of robotics in schools: A systematic review. Computers & Education 58(3): 978–988. doi:10.1016/j.compedu.2011.10.006.

Bers

M. U.

(2010) The TangibleK robotics program: Applied computational thinking for young children. Early Childhood Research & Practice 12(2): 1–20.

Blanchard

Freiman

Lirrete-Pitre

(2010) Strategies used by elementary school children solving robotics-based complex tasks: Innovative potential of technology. Procedia – Social and Behavioral Sciences 2(2): 2851–2857. Retrieved from http://doi.org/10.1016/j.sbspro.2010.03.427.

Blumenfeld

P. C.

Soloway

Marx

R. W.

Krajcik

J. S.

Guzdial

Palincsar

(1991) Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist 26(3 & 4): 369–398.

Callahan

C. M.

Hunsaker

S. L.

Adams

C. M.

Moore

S. D.

Bland

L. C.

(1995) Instruments used in the identification of gifted and talented students, Charlottesville, VA: The National Research Center on the Gifted and Talented (NRCG/T), University of Virginia.

Castledine

Chalmers

(1993) Lego robotics: An authentic problem solving tool? Design and Technology Education: An International Journal 16(3): 19–27.

Chambers

J. M.

Carbonaro

Rex

Grove

(2007) Scaffolding knowledge construction through robotic technology: A middle school case study. Electronic Journal for the Integration of Technology in Education 6: 55–70.

10.

Chapin

S. H.

O’Connor

Canavan Anderson

(2012) Classroom discussions using math talk to help students learn (2nd ed.), Sausalito, CA: Math Solutions.

11.

Denis

Hubert

(2001) Collaborative learning in an educational robotics environment. Computers in Human Behavior 17(5–6): 465–480. doi:10.1016/S0747-5632(01)00018-8.

12.

Eguchi

(2012) Educational robotics theories and practice: Tips for how to do it right. In: Barker

Nugent

Grandgenett

Adamchuk

(eds) Robots in K-12 education: A new technology for learning, Hershey, PA: Information Science Reference, pp. 1–30. doi:10.4018/978-1-4666-0182-6.ch001.

13.

Fowler

(1990) The diet cola test. Science Scope 13(4): 32–34.

14.

Grandgenett, N., Chen, B., & Timms, M. (2010). Silicon prairie initiative for robotics in information technology SPIRIT (Project Final Evaluation Report). Omaha, NE: National Science Foundation.

15.

Grandgenett

Ostler

Topp

Goeman

(2012) Robotics and problem-based learning in STEM formal educational environments. In: Barker

Nugent

Grandgenett

Adamchuk

(eds) Robots in K-12 education: A new technology for learning, Hershey, PA: Information Science Reference, pp. 94–119. Retrieved from http://doi.org/10.1017/CBO9781107415324.004.

16.

Hwang

W.-Y.

S.-Y.

(2014) A case study of collaboration with multi-robots and its effect on children’s interaction. Interactive Learning Environments 22(4): 429–443. Retrieved from http://doi.org/10.1080/10494820.2012.680968.

17.

Jordan

M. E.

McDaniel

R. R.

(2014) Managing uncertainty during collaborative problem solving in elementary school teams: The role of peer influence in robotics engineering activity. Journal of the Learning Sciences 23(4): 490–536. Retrieved from http://doi.org/10.1080/10508406.2014.896254.

18.

Karahoca

Uzunboylub

(2011) Robotics teaching in primary school education by project based learning for supporting science and technology courses. Procedia Computer Science 3: 1425–1431. Retrieved from http://doi.org/10.1016/j.procs.2011.01.025.

19.

Mallozzi

Heilbronner

N. N.

(2013) The effects of using interactive student notebooks and specific written feedback on seventh grade students’ science process skills. Electronic Journal of Science Education 17(3): 1–23.

20.

Mills

Chandra

Park

(2013) The architecture of children’s use of language and tools when problem solving collaboratively with robotics. Australian Educational Research 40(3): 315–337. doi:10.1007/s13384-013-0094-z.

21.

Mohr-Schroeder

M. J.

Little

D. L.

Schroeder

D. C.

(2014) Developing middle school students’ interests in STEM via summer learning experiences: See blue STEM camp. School Science and Mathematics 114(6): 291–301. Retrieved from http://doi.org/10.1111/ssm.12079.

22.

Nelson, C. A. (2012). Generating transferable skills in STEM though educational robotics. In B. Barker, G. Nugent, N. Grandgenett, & V. Adamchuk (Eds.), Robots in K-12 education: A new technology for learning (pp. 1–30). Hershey, PA: Information Science Reference. http://doi.org/10.1017/CBO9781107415324.004.

23.

Papert

(1993) Mindstorms children, computers, and powerful ideas, 2nd ed. New York, NY: Basic Books.

24.

Petre

Price

(2004) Using robotics to motivate “back door” learning. Education and Information Technologies 9(2): 147–158. doi:10.1023/B:EAIT.0000027927.78380.60.

25.

Reeves

T. C.

(2011) Can educational research be both rigorous and relevant? Educational Designer 1(4): 1–24.

26.

Ringwood

J. V.

Monaghan

Maloco

(2005) Teaching engineering design through Lego® Mindstorms™. European Journal of Engineering Education 30(1): 91–104. doi:10.1080/03043790310001658587.

27.

Robinson

(2005) Robotics-driven activities: Can they improve middle school science learning? Bulletin of Science, Technology & Society 25(1): 73–84. Retrieved from http://doi.org/10.1177/0270467604271244.

28.

Slangen

Keulen

Gravemeijer

(2010) What pupils can learn from working with robotic direct manipulation environments. International Journal of Technology and Design Education 21(4): 449–469. Retrieved from http://doi.org/10.1007/s10798-010-9130-8.

29.

Somyürek

(2014) An effective educational tool: Construction kits for fun and meaningful learning. International Journal of Technology and Design Education 25: 25–41. doi:10.1007/s10798-014-9272-1.

30.

Sullivan

Bers

M. U.

(2016) Robotics in the early childhood classroom: Learning outcomes from an 8-week robotics curriculum in pre-kindergarten through second grade. International Journal of Technology and Design Education 26: 3–20. Retrieved from http://doi.org/10.1007/s10798-015-9304-5.

31.

Sullivan

F. R.

(2008) Robotics and science literacy: Thinking skills, science process skills and systems understanding. Journal of Research in Science Teaching 45(3): 373–394. doi:10.1002/tea.

32.

Ucgul

Cagiltay

(2014) Design and development issues for educational robotics training camps. International Journal of Technology and Design Education 24(2): 203–222. doi:10.1007/s10798-013-9253-9.

33.

Yuen, T. T., Boecking, M., Tiger, E. P., Gomez, A., Guillen, A., Arreguin, A., & Stone, J. (2014). Group tasks, activities, dynamics, and interactions in collaborative robotics projects with elementary and middle school children. Journal of STEM Education: Innovations and Research, 15(1), 39–46.