Effect of Initial Design Experience on Students’ Development of Scientific Understanding

Abstract

Background: In countries that endorse science, technology, engineering, and mathematics (STEM) education as an educational movement, design-based learning is deemed a pedagogical approach. Purpose: Because the integration of the engineering design process and scientific investigation has been an issue in science education, this study examined the effect of initial design experience on students’ understanding of pulleys. Methodology/Approach: We used quasi-experimental research, comparing two experimental groups with one control group in terms of their understanding of pulleys, measured by 10 multiple-choice conceptual questions. We used a one-way ANOVA and Tukey's HSD tests to compare the groups prior to and after design-based learning. Paired-sample t-tests were used to compare the pre- and postscores of each group. Findings/Conclusions: The results indicate that, while the experimental groups did not have a better understanding of pulleys than the control group did in the pretest, they outperformed the control group in the posttest. Moreover, although the control group did not demonstrate a significant improvement, the experimental groups did so with large effect size. Implications: Based on the premises of experiential education, it is suggested that, for design-based activities to be effective, students should design using their prior knowledge before engaging in scientific investigation.

Keywords

design-based learning experiential learning pulleys STEM education

Due to economic pressure and concerns regarding human wellbeing, many countries, including Thailand, have embraced science, technology, engineering, and mathematics (STEM) education as part of their primary and secondary education curricula. Their aim is to increase the workforce in STEM-related careers as well as to promote STEM literacy among citizens (Li et al., 2020; Promboon et al., 2018). In this regard, the engineering design process has been recognized as a new pedagogical approach to promoting STEM learning. Through the engineering design process, students are challenged to collaboratively design artifacts, systems, or solutions intended to address real-world problems using scientific, mathematical, and technological knowledge, skills, and dispositions (Kelly & Knowles, 2016). A key premise inherent in this pedagogical approach is that direct experience of the engineering design process may facilitate students’ meaningful learning of STEM in a more integrated way than traditional instruction. Arguably, this premise aligns with the philosophy of experiential education.

According to Itin (1999),
experiential education is a holistic philosophy where carefully chosen experiences supported by reflection, critical analysis, and synthesis, are structured to require the learner to take initiative, make decisions, and be accountable for the results, through actively posing questions, investigating, experimenting, being curious, solving problems, assuming responsibility, being creative, constructing meaning, and integrating previously developed knowledge. (pp. 93-94)

Thus, experiential education is not limited to field-based education, outdoor education, adventure education, and service learning. Rather, “it is the learning and teaching process that defines whether a learning experience is experiential” (Chapman et al., 1992, p. 20, italics in original). As design-based learning provides students direct experience with and opportunities to reflect on what they are learning (Ellefson et al., 2008), it can be considered a kind of experiential learning. Given its experiential dimension (Eyler, 2009; Hall & Miro, 2016), it is evident that design-based learning can promote students’ learning (Kolodner et al., 2003; Korur et al., 2017).

Models of design-based learning in science education structure design experience and scientific investigation differently to promote students’ science learning. For example, Fortus et al. (2004) encourage students to conduct background research (e.g., literature search and scientific investigation) before using what they learn to solve engineering problems. In contrast, Apedoe et al. (2008) have students design and test prototypes before generating questions or hypotheses for scientific investigation based on the results. Lewis (2006) thus identifies two approaches to design-based learning in science education: design-through-science and science-through-design with the “effect (of both being) the same” (p. 269) in the sense that both approaches mimic complementarities between science and design in the real world.

Regardless of the approach, design-based learning has its roots in a constructionist perspective on learning (Jun et al., 2017). This perspective holds that students actively make meanings as a result of interactions between their prior knowledge and new experiences, especially when such experience involves creating things (Ke, 2014). By creating things, students’ prior knowledge is not only cognitively activated but also physically augmented; thus, it can be empirically tested. Particularly when the process of creating and testing things occurs in social contexts where students share and discuss various ideas with appropriate scaffolding provided by a teacher (Gomez Puente et al., 2013), this experience is likely to foster students’ meaningful learning. Research indicates that design-based learning can facilitate students’ conceptual learning of science (Kolodner et al., 2003; Korur et al., 2017) to an even greater extent than inquiry-based learning (Marulcu & Barnett, 2016; Mehalik et al., 2008). Given the positive outcomes of design approaches based on both design through science and science through design (Apedoe et al., 2008; Fortus et al., 2004), it is unclear which approach results in learning becoming more meaningful for students.

This quasi-experimental study was aimed to address the question of whether initial design experience better facilitates students’ understanding of scientific concepts when they subsequently engage in scientific investigation. By letting students experience the designing process before engaging in scientific investigation, educators may activate students’ prior knowledge so that students can then perform scientific investigations in a meaningful way. Given limited class time and a large amount of content in many countries’ curricula, such experience can be excluded if it does not contribute to students’ science learning. Bearing in mind the fact that engineering and science are related yet different disciplines (Antink-Meyer & Brown, 2019), this study could provide insight into how the engineering design process is best incorporated with scientific investigation in ways that are meaningful to students. Moreover, given that experiential education can enhance STEM education (Collins et al., 2020; Morrison & Bartlett, 2009), this study could broaden understanding of how experiential education can advance design-based learning in particular.

Design-Based Learning

According to Ellefson et al. (2008), design-based learning is “a type of problem-based learning, in which students work together in teams to solve a problem” (p. 292). However, design-based learning is slightly more specific than problem-based learning “because students work together specifically to create or design a new invention/prototype” (ibid, p. 292). Moreover, rather than focusing on solving general problems, design-based activities focus on solving other peoples’ (e.g., users or customers’) problems under given constraints and based on certain criteria (Dym et al., 2005). By highlighting the engineering design process, design-based learning has become a pedagogical approach to STEM education. This is because the nature of STEM can be reflected by the nature of engineering (Quinn et al., 2020) and the defining aspect of engineering is design (Pleasants & Olson, 2019). As a result, research in STEM at the primary and secondary education levels has encouraged design-based learning (Thibaut et al., 2018). Other labels that can be used to refer to design-based learning in science education include “design-based science” (Fortus et al., 2004) and “learning by design” (Kolodner et al., 2003).

There are various models for design-based learning in science education. In their model, Fortus et al. (2004) illustrate five stages: identifying and defining contexts, performing background research, developing personal and group ideas, constructing 2D and 3D artifacts, and testing to gain feedback. According to Lewis’s (2006) classification, this model fits the design-through-science approach because it highlights “how science becomes the vehicles for prompting design” (p. 269). In this model, students are expected to acquire scientific ideas through background research (e.g., scientific investigation) that is subsequently used in designing artifacts. As students often have misconceptions that are resistant to change (Chinn & Malhotra, 2002), they may ignore scientific ideas and, instead, rely on misconceptions in designing artifacts (Schauble et al., 1991), resulting in what is called “design fixation” (Schut et al., 2020). Thus, to support students’ conceptual change toward scientific ideas, Schnittka and Bell (2011) recommend demonstrations targeting students’ misconceptions before students begin to design artifacts.

To address misconceptions that play a role in design-based learning, some researchers allow students to obtain design experience while conducting scientific investigation. For example, Kolodner et al. (2003) propose a model that comprises two interacting cycles, namely the design/redesign cycle and the investigate/explore cycle. Students can move between these two cycles to concurrently engage in the engineering design process and scientific investigation if necessary. In this model, design experience can lead to scientific investigation, the results of which can in turn be used in the designing process. Likewise, in Apedoe et al.’s (2008) model, consisting of seven stages—creating designs, evaluating outcomes, generating reasons, testing ideas, analyzing results, generalizing results, and connecting to big ideas—students are encouraged to initially design products to be tested before using the results of such testing to generate questions or hypotheses for scientific investigation. According to Lewis’s (2006) classification of design-based learning, these models fit the science-through-design approach because “the design process … is used as the vehicle for teaching science concepts” (p. 268).

The Institute for the Promotion of Teaching Science and Technology (2015), Thailand's national organization driving STEM education, describes the engineering design process as a key pedagogical model comprising six steps: problem identification; related information search; solution design; planning and development; testing, evaluation, and design improvement; and presentation. This model, which features a design-through-science approach (Lewis, 2006), has been introduced to science teachers across Thailand. However, as Promboon et al. (2018) caution, no model of STEM education should be viewed as a one-size-fits-all; developing alternative models of STEM education is necessary. Research is thus needed to identify which aspects of each model are crucial in promoting students’ science learning. Given the positive effects of design-based learning on students’ science learning (Apedoe et al., 2008; Fortus et al., 2004; Kolodner et al., 2003), it is necessary to determine whether initial design experience, which appears in some models, is necessary in developing students’ scientific understanding. Thus, the research question guiding this study is as follows:
Do students with initial design experience perform better in acquiring scientific understanding through a design-based learning activity than those without initial design experience?

Methods

Addressing the research question ideally requires true experimental research to compare at least two equivalent groups of students who have and do not have initial design experience. A quasi-experimental research approach was instead utilized in this study because such true experimental research is not possible in many educational settings where students are already assigned classes (Gopalan et al., 2020). According to Chiang et al. (2015), in a quasi-experimental study, researchers can use nonequivalent groups of students with certain strategic designs (e.g., pretest-posttest design and interrupted time series design) to ensure that such groups of students are as similar as possible. In this study, different classes of students at the same age and grade level were asked to complete the pretest. We used the results to assign each class to control or experimental groups. After observing the teacher's fidelity to each pedagogical approach and controlling time spent on the approaches, we asked the students to complete the posttest. The results were compared with those of the pretest to make a valid inference regarding the experimental treatment.

Context

This study was approved by the IRB at the University of Phayao (number: 2/048/62). It took place during January and February 2021 in a secondary school in a rural area with a total of 401 students from seventh to 12^th grades. The school is surrounded by rice fields, though it is about 30 kilometers from the city. Given this distance, which takes about 40 min to drive by car, parents with sufficient incomes (e.g., businessmen and governmental officers) prefer to send their children to the more privileged school located in the city, leaving students from local families with lower incomes (e.g., agricultural families and laboring families) to study at the local school. Forty-two teachers worked at this school at the time of the study. Of these teachers, there were five science teachers, including the second author, who has a background in physics. He has 24 years of teaching experience in total and 12 years of teaching experience at lower secondary levels. Due to his interest in STEM education and design-based learning, he voluntarily participated in the study. When this study was conducted, he was responsible for teaching science to one seventh-grade class and three eighth-grade classes, as well as to teaching physics to one 10^th-grade class and one 12^th-grade class. His three eighth-grade classes participated in this study.

Activities

Based on an initial discussion with the teacher, we decided to develop a lesson on pulleys for all three eighth-grade classes. The reasons underlying this decision were, first, that pulleys are commonly used by local people; thus, this could help the students see how science is relevant to their everyday life (Tekbiyik, 2015). Second, students at this age often have misconceptions about pulleys (Sullivan et al., 2017), such as “the more pulleys there are in a setup, the easier it is to pull to lift a load”; “the longer the string is a pulley setup, the easier it is to pull to lift a load”; and “having more pulleys in a pulley setup reduces the amount of work” (Myneni & Narayanan, 2012, p. 75). This provides an opportunity to investigate the role of these misconceptions in students’ designing process. Third, given that “pulleys are … often challenging for … students to set up on their own” (Sullivan et al., 2017, p. 1579), there is no study examining how design-based learning may help students develop scientific understandings of pulleys. Most studies compare the effect of physical or virtual experiments on students’ understanding (e.g., Chini et al., 2012).

Apedoe et al.’s (2008) model was used as a framework for the lesson. We chose this model because it makes explicit that students have initial design experience. Thus, students in the experimental groups learned about pulleys by creating designs, evaluating outcomes, generating reasons, testing ideas, analyzing results, generalizing the results, and then connecting to big ideas. Students in the control group followed six steps: problem identification; related information search; solution design; planning and development; testing, evaluation, and design improvement; and presentation, as described by the Institute for the Promotion of Teaching Science and Technology (2015). We chose this model because it has been introduced across the country and the teacher was quite familiar with it. Both the experimental and control groups spent eight periods of 50 min each on the design-based learning. Table 1 describes the details of both design-based activities.

Table 1.
A comparison of design-based activities for the experimental and control groups.

Activity for the experimental groups Activity for the control group

(1) Creating designs: Students were introduced to a situation where an elder has difficulty lifting a bucket of water from a well. Then, groups of four to six students were challenged to design and create prototype pulley setups using their prior knowledge to help the elder lift a given object. (1) Problem identification: Students were introduced to a situation where an elder has difficulty lifting a bucket of water from a well. Then, they discussed how to help the elder in a condition with no electricity. The teacher led the students to the idea of using a pulley to help the elder lift the bucket of water. Thereafter, they were introduced to more specific problems with how a pulley works and how a pulley setup is designed.

(2) Evaluating outcomes: Each group of students evaluated the performance of their pulley setup by using a balance to measure force used to lift the object. Each group of students then shared the results, which were recorded by the teacher on the blackboard for discussion. (2) Related information search: Groups of four to six students were introduced to scientific investigation, which was conducted in a structured manner to explore how different setups of pulley(s) (e.g., one fixed pulley, one moveable pulley, and both fixed and moveable pulleys) affect the force used to lift the object. They then reached the conclusion that a moveable pulley reduces the force when compared to the weight of the object, while a fixed pulley does not.

(3) Generating reasons: By comparing the results on the blackboard, students shared their ideas about what might cause the difference in the force used to lift the same object. (3) Solution design: Each group of students was challenged to use what they had learned from scientific investigation to draw a design of a prototype pulley setup that would lift a given object. They could search for additional information on the Internet.

(4) Testing ideas: With scaffolding provided by the teacher, each group of students examined the different ways a pulley could be set up (e.g., fixed pulley versus movable pulley). This led to scientific investigation, which was conducted in a structured manner, to explore how the different setups affect the force used to lift the object. (4) Planning and development: Each group of students constructed a real setup based on their drawing. They could change their ideas if they wished to do so.

(5-6) Analyzing and generalizing results: Each group of students analyzed and discussed the results to conclude that a moveable pulley reduces the force when compared to the weight of the object, while a fixed pulley does not. With this conclusion, each group of students could improve their prototype pulley setup. (5) Testing, evaluation, and design improvement: Each group of students tested and evaluated the performance of their design. They were encouraged to repeatedly improve the performance of their respective designs.

(7) Connecting to big ideas: The teacher used a virtual simulation to demonstrate how pulleys work. This simulation was connected to the idea of exploiting mechanical advantage using pulleys. The students also had an opportunity to solve physics problems. (6) Presentation: Each group of students presented their design and its performance. Then, the teacher used a virtual simulation to demonstrate how pulleys work. This simulation was connected to the idea of exploiting mechanical advantage using pulleys. The students also had an opportunity to solve physics problems.

As can be seen in Table 1, the activities were similar in the context of the engineering problem, the use of virtual demonstrations, and the calculation used to solve physics problems. Moreover, in designing the pulley setup, the same set of materials and equipment was provided. The key difference between the two design-based activities was the sequence of design experience and scientific investigation. Students in the experimental groups designed and tested a prototype of the pulley setup based on their prior knowledge before they engaged in scientific investigation into how pulleys work, the results of which were then used to improve the prototype. In contrast, students in the control group engaged in scientific investigation shortly after the engineering problem was presented. Thereafter, they were encouraged to use the results of scientific investigation to design, create, and test a prototype pulley setup in an iterative manner. This allowed us to examine whether initial design experience is necessary to facilitate students’ scientific understanding of pulleys.

Instrument

To examine the role of initial design experience in students’ scientific understanding of pulleys, we used a set of conceptual questions from the literature (Chini, 2006; Sullivan et al., 2017) as the pretest and the posttest. While the possible conceptual questions covered a range of scientific concepts, such as force, work, potential energy, and mechanical advantage, only questions associated with force and work were selected, resulting 12 questions (see Table 2). Six questions related to force were selected because the students can learn this concept through kinesthetic experiences (Sullivan et al., 2017). Six questions related to work were selected because this concept represents more abstract ideas (e.g., potential energy) than the concept of force (Gire et al., 2010), meaning the students may not simply learn work through kinesthetic experiences (Chini et al., 2012). Thus, comparing the students’ learning of these concepts could illustrate the potential role of initial design experience in facilitating students’ understanding of these two scientific concepts.

Table 2.
The structure of the conceptual test.

Item Description Concepts Item difficulty

Force Work

1 Students are asked to choose which of two setups, one with a fixed pulley and one with a moveable pulley, requires less force to lift the same object (Picture provided). ✓ 0.38

2 Students are asked whether a setup consisting of two pulleys, a fixed one and a moveable one, requires less or more force to lift an object when compared with no pulleys (Picture provided). ✓ 0.45

3 Given the same string for a pulley, students are asked whether an object can be lifted higher or lower if they use a moveable pulley instead of a fixed one (No picture). ✓ 0.28

4 Students are asked whether the amount of force required to lift an object will be more or less if they use a moveable pulley instead of a fixed one (No picture). ✓ 0.48

5* Students are asked whether a moveable pulley or a fixed pulley requires less work to lift the same object to the same position from the ground (No picture). ✓ 0.10

6* Given two setups, one using a moveable pulley only and one using both a fixed pulley and a movable pulley, students are asked to choose the setup that requires less force to lift the same object (Picture provided). ✓ 0.17

7 Students are asked to determine the work required to lift a 10-Newton weight using a moveable pulley to a height of 0.05 m above the ground while the position of the pulley also rises to a position 0.1 m higher than its original position (Picture provided). ✓ 0.52

8 Two persons individually attempt to lift the same object. One uses a setup consisting of a fixed pulley and a movable pulley, while the other does not use this setup. Students are asked which person used less work to achieve the same task (Picture provided). ✓ 0.52

9 A person lifts the same object using the same fixed pulley twice to reach different positions—the second attempt achieves a higher position than the first attempt. Students are asked to compare the work used in these two attempts (Picture provided). ✓ 0.62

10 Given two setups, one using a fixed pulley only and one using both a fixed pulley and a movable pulley to individually lift the same object to the same position, students are asked to compare the work used by these two setups (Picture provided). ✓ 0.48

11 Given two setups, one using a fixed pulley and one using two fixed pulleys, students are asked to compare the amount of force required to lift the same object (Picture provided). ✓ 0.24

12 Given two setups, one using a fixed pulley and one using a moveable pulley, students are asked to compare the amount of force required to lift the same object (Picture provided). ✓ 0.34

Note. Items with asterisk were excluded after the trial test due to their difficulty.

The selected questions were translated into Thai and sent to three physics educators to check their validity and readability. After we revised the questions based on the educators’ comments, 29 seventh-grade students in a school located in the same province answered them. The reason for choosing seventh-grade students was that we were interested in determining whether the test could detect naïve understandings related to pulleys, as reported in the literature (Myneni & Narayanan, 2012). Thus, slightly younger students who had not studied pulleys were deemed appropriate. The results indicated that item 5 and item 6 were too difficult, as their item difficulties were lower than 0.20 (see Table 2). Thus, these items were excluded from the study, resulting in 10 questions. The reliability of the test as calculated by Cronbach’s alpha was about 0.60, which was lower than the standard value of 0.70 (Morgan et al., 2013), perhaps due to the fact that the test measures two concepts (i.e., force and work). However, the reliability for each concept was about 0.71, which was acceptable for educational research.

Data Collection

We administered the test to 61 eighth-grade students from three classes with a similar gender ratio (seven males and eight girls in the first class, nine males and 11 females in the second class, and 12 males and 14 girls in the third class) in December 2020. As is presented in the results, the first class had the highest mean score, which was followed by the second class and the third class, respectively. Their mean scores are not surprising given that Thai schools assign students to classes based on their academic achievements. With a hypothesis that initial design experience might be useful to students’ development of their scientific understanding, the first class was assigned to be the control group, while the other two classes were assigned to be the experimental groups. After the eight 50-min periods of design-based activities lasting about three weeks, each of which had two successive periods and one single period (see Table 3 for each class's schedule), all three classes completed the test again on the same day. We also observed and recorded the teacher's classroom practices to ensure his fidelity to each approach to design-based learning.

Table 3.
Each class's schedule.

Class Group Schedule Each Week

1st class Control Two successive periods on Tuesday morning and a single period on Wednesday afternoon

2nd class Experimental Two successive periods on Tuesday morning and a single period on Thursday morning

3rd class A single period on Tuesday afternoon and two successive periods on Thursday morning

Data Analysis

We used descriptive statistics to analyze the students’ scores on the test prior to and after the design-based activities, calculating the mean scores and standard deviation of each class. Then, inferential statistics were used to examine whether there was difference between the mean scores of each class. As guided by Morgan et al. (2013), we used a one-way ANOVA for both the pretest and the posttest if the basic assumptions of equal variances on the dependent variable and of normal distribution of the dependent variance for each group were achieved. If a difference between mean scores was observed, Tukey HSD post hoc multiple comparison tests were used to determine the source of the difference. Independent t-tests were also used to confirm the differences between each of the three classes. Moreover, we used paired-sample t-tests to examine whether each class improved their understanding of pulleys after the design-based activity. If so, we calculated Cohen's d-value of effect size to determine how much the design-based activity affected students’ understanding of pulleys. In all cases of inferential analyses, significance at 0.05 was used.

Results

The descriptive analyses indicated that the three classes of students had different levels of understanding in the pretest, as can be seen in Figure 1. Specifically, the first class had a mean score of 3.60 (SD = 1.06), the second class had a mean score of 3.05 (SD = 1.43), and the third class had a mean score of 2.31 (SD = 1.19). Since the basic assumptions of a one-way ANOVA were achieved in the pretest data, the one-way ANOVA was used to determine whether these three classes significantly differed in their initial understanding of pulleys. The result indicated a significant difference between the classes, F(2) = 5.43, p = .007, effect size (η2) = 0.16. In this regard, Tukey's HSD tests suggested that the only notable difference was between the first class and the third class (p = .006). Independent t-tests confirmed that the first class performed significantly better than the third class, t(39) = 3.48, p < .001, Cohen's d effect size = 1.13, while the differences between the first class and the second class and between the second class and the third class were not significant, t(33) = 1.25, p = .219 and t(44) = 1.92, p = .062, respectively.

Figure 1.
Each class’s mean scores on the pretest and the posttest.

As previously mentioned, we assigned the first class to be the control group and the other two classes to be the experimental groups based on the results of the pretest. This means that the first class engaged with the design-through-science model (Institute for the Promotion of Teaching Science and Technology, 2015). The second and third classes engaged with the science-through-design model (Apedoe et al., 2008). The decision to assign the classes in this manner was based on Promboon et al.’s (2018) recommendation regarding the improvement of STEM education in Thailand, which states that it is crucial to seek an alternative model of design-based learning that may be more effective in facilitating students’ STEM learning than the existing model. When considering Apedoe et al.’s (2008) model as an alternative approach to design-based learning, it is important to ensure that this model is superior to the one that has been nationally promoted.

Following the design-based learning activity, all three classes improved their understanding of pulleys. Specifically, the first class had a mean score of 4.07 (SD = 2.09), the second class had a mean score of 5.70 (SD = 2.62), and the third class had a mean score of 6.42 (SD = 2.04). Since the basic assumptions of a one-way ANOVA were also achieved in the posttest data, the one-way ANOVA was used to determine whether these three classes differed in their understanding of pulleys following the design-based activity. The result indicated a significant difference between the classes, F(2) = 5.21, p = .008, effect size (η2) = 0.15. As in the pretest, Tukey’s HSD tests revealed a significant difference between the first class and the third class (p = .006) and a considerable but insignificant difference between the first class and the second class (p = .095). In contrast to the pretest, independent t-tests revealed that the third class performed significantly better than the first class in the posttest, t(39) = 3.53, p < .001, Cohen's d effect size = 1.14). This is also the case for the second class when compared to the first class, t(33) = 1.99, p < .028, Cohen's d effect size = 0.68).

We also compared each class's scores between the pretest and the posttest using paired-sample t-tests. The results indicated that the first class did not significantly improve their understanding of pulleys, t(14) = 0.83, p = .209. However, the second and the third classes performed significantly better in the posttest than in the pretest, t(19) = 3.72, p < .001 and t(25) = 9.47, p < .001, with Cohen's d effect sizes of 0.83 and 1.86, respectively. Moreover, when considering the scores for each concept, the first class significantly improved their understanding of force alone, t(14) = 2.93, p = .005, with Cohen's d effect size = 0.76. In contrast, the second class significantly improved their understanding of both force, t(19) = 3.80, p < .001, with Cohen's d effect size = 0.85, and work, t(19) = 2.30, p = .017, with Cohen's d effect size = 0.51. This was also true for the third class, t(25) = 7.39, p < .001, with Cohen's d effect size = 1.45 for force and t(25) = 5.47, p < .001, with Cohen's d effect size = 1.07 for work.

Discussion

Due to increasing interest in design-based learning as a pedagogical approach to STEM education, how to best integrate the engineering design process and scientific investigation in meaningful ways in the classroom has become a notable issue. Perhaps because engineering is often viewed as a discipline that applies scientific knowledge to solving human problems (Ladachart et al., 2020), some people believe that students learn best through performing scientific investigation and applying the results thereof to the engineering design process (Institute for the Promotion of Teaching Science and Technology, 2015); this is the design-through-science approach (Lewis, 2006). In contrast, because engineers not only use knowledge from other disciplines but also produce new knowledge in their own discipline (Pleasants & Olson, 2019), some people believe students can construct new knowledge through the engineering design process (Ellefson et al., 2008); this is the science-through-design approach (Lewis, 2006). While positive effects of design-based learning on students’ science learning have been reported (e.g., Apedoe et al., 2008; Fortus et al., 2004; Kolodner et al., 2003), studies comparing such an effect between the two design-based approaches are still rare.

This quasi-experimental study demonstrates that, in the context of design-based learning about pulleys, students develop scientific understanding via the science-through-design approach better than via the design-through-science approach. In other words, they learn scientific concepts better when they have an opportunity to design based on prior knowledge before conducting scientific investigation. Without this initial design experience, they seem to struggle in terms of understanding how scientific investigation can meaningfully solve engineering problems. This result supports Apedoe and Schunn’s (2013) study, which indicated that students do not typically use strategies commonly seen in scientific investigation when engaging in the engineering design process. Rather, it is adaptive growth, where students attempt a better outcome if successful and try the same method or something more basic if unsuccessful, that is “most useful for articulation of explicit design principles (i.e., scientific concepts) as well as leading to design success” (p.786). Unlike the trial-and-error method naturally used by young students (Park et al., 2018), the adaptive growth strategy requires more reflection on and reasoning about design experience.

Thus, with the insight that reflecting on design experience plays a vital role in learning scientific concepts during design-based activities (Puntambekar & Kolodner, 2005), the results of this study can be explained. Initial design experience afforded the students in the experimental groups a unique opportunity on which they could reflect to develop a better scientific understanding, unlike the students in the control group. With initial design experience, the experimental students’ prior knowledge was likely activated, which made the subsequent scientific investigation more meaningful to them. This explanation is sensible given research by Chase et al. (2019) demonstrating that students presented contrasting cases of design products are more likely to notice critical aspects of those design products. The critical aspects that are noticed can then be the focal point when the students engage in scientific investigation (Malkiewich & Chase, 2019). Without the focal point framed by the initial design experience before scientific investigation, students have more difficulty comprehending scientific concepts (Wecker et al., 2013).

This explanation is also supported by the data collected through classroom observation. Among all groups of students, only one group of students in the third class used a moveable pulley in their initial design. This caused them to need less force to lift the object when compared with their classmates. The teacher was thus able to explicitly make a comparison between this group's pulley setup and the ones designed by the other groups. This use of contrasting cases (Chase et al., 2019) might have allowed students in the third class to notice for themselves how moveable pulleys reduce the force required to lift an object. As none of students from the other two classes used a moveable pulley in their initial design, it was the teacher who introduced to them that setup before encouraging them to scientifically investigate whether a moveable pulley requires the same force as a fixed pulley. This experiential difference could contribute to the third-class students’ greatest improvement in their understanding of pulleys.

Despite studies reporting positive effects of the design-through-science approach (e.g., Fortus et al., 2004), students in the first class of this study, who conducted a scientific investigation before engaging in the engineering design process, exhibited no significant improvement in their understanding of pulleys. This result is supported by some studies indicating that this approach alone is not sufficient for students to overcome their misconceptions and develop scientific understandings (Dankenbring & Capobianco, 2016). Therefore, additional strategies should be integrated to enhance its effectiveness, such as demonstrations targeting students’ misconceptions (Schnittka & Bell, 2011) and communicative scaffolding (Chusinkunawut et al., 2021). Based on the premises of experiential education, which highlights the crucial role of experience and reflection in learning (Joplin, 1981), this study highlights how it is more beneficial for students to have initial experience of designing solutions to an engineering problem before engaging in scientific investigation. In this process, opportunities to reflect on design experience as well as scientific investigation are crucial.

This study also emphasizes the importance of how experiences are structured in design-based learning. Based on an experiential perspective, learning can be considered as “the process whereby knowledge is created through the transformation of experience” (Kolb, 2015, p. 49). Given this definition, experience is necessary for students to construct knowledge. However, simply providing students experience with what they are expected to learn is not sufficient. The construction of knowledge requires students to grasp the meaning of experience via apprehension (e.g., feeling the weight of the object being lifted) and/or comprehension (e.g., symbolizing the effort force and the object's weight with arrows and numbers) and to then transform such grasped experience via intention (e.g., reflecting on what they believe about how a pulley works) and/or extension (e.g., manipulating a pulley in different ways to empirically test a working hypothesis). It is through this process that an abstract concept as a form of knowledge is derived from and continuously modified by experiences (Kolb, 2015).

When comparing the control group and the experimental groups, Apedoe et al.’s (2008) model of design-based learning seemed to provide a more complete and connected process of grasping and transforming design-based experience than the Institute for the Promotion of Teaching Science and Technology’s (2015) model. While the control group began their learning process by comprehending the abstract concepts of force and work based on scientific investigation, the experimental group began with the apprehension of their effort force and the object's weight during their initial attempt to design a setup of pulleys. As Gire et al. (2010) highlight, with this kinesthetic experience facilitating their understanding of pulleys, the experimental groups had a foundation for subsequently comprehending the concepts of force and work in the scientific investigation. Without this initial kinesthetic experience, the control group struggled to comprehend the abstract concepts. Based on the experiential perspective, initial design experience is thus beneficial in allowing students to grasp and transform experience into scientific understanding.

Implications

This study has implications for the implementation of design-based learning in science education. Most fundamentally, it confirms previous studies that conclude that design-based learning can facilitate students’ development of scientific understanding (Apedoe et al., 2008; Chusinkunawut et al., 2021; Fortus et al., 2004; Kolodner et al., 2003; Korur et al., 2017; Marulcu & Barnett, 2016; Mehalik et al., 2008; Schnittka & Bell, 2011). However, to what extent this is true depends on how the engineering design process and scientific investigation are structured. Based on the results of this study, it is more beneficial to begin design-based activities by providing students an early opportunity to design based on their prior knowledge. Thus, students’ prior knowledge is activated before they meaningfully engage in scientific investigation. With initial design experience, it is more likely that design-based learning becomes experiential (Chapman et al., 1992) before scientific concepts are introduced via scientific investigation. This implication can provide guidance for alternative design-based approaches to STEM education (Promboon et al., 2018).

Limitations

This study is subject to certain limitations. First, it was conducted in a single rural school in Thailand. Thus, its results may not fully be generalizable to different schools in other contexts, even in the same country. Second, as the primary instrument used in this study was a conceptual test in a multiple-choice format, its results may be explained by initial design experience contributing to students’ development of scientific understanding. It is not possible, however, to exactly identify what experiences and strategies the students gained and utilized when they were engaging in the engineering design process and scientific investigation. Thus, to gain better insight into the students’ experiences, reflection, and reasoning, it is recommended that qualitative data such as students’ discussions and interviews also be collected and analyzed in future research. Third, given the limited number of classes in each grade in the school, an imbalance between the experimental and control groups was inevitable, despite little gender bias. Therefore, future research aiming to verify and elaborate on the results of this study should ensure more balance between the experimental and control groups.

Conclusion

Research in STEM education recommends design-based learning as a pedagogical approach to facilitating students’ learning (e.g., scientific understanding). However, approaches to design-based learning can vary depending on how the engineering design process and scientific investigation are integrated to best create meaningful learning for students. Based on an experiential perspective on learning, this study makes the following contribution to the literature: Design-based learning can be more effective in facilitating conceptual learning of scientific concepts when students have an opportunity to initially design solutions to an engineering problem based on their prior knowledge than when students are first exposed to relevant concepts through scientific investigation. While it may not immediately lead to design success, initial design experience can serve as an experiential resource that students can use to transform to develop scientific understanding during design-based activities. Without access to such a resource, students may struggle to comprehend scientific concepts.

Activity for the experimental groups	Activity for the control group
(1) Creating designs: Students were introduced to a situation where an elder has difficulty lifting a bucket of water from a well. Then, groups of four to six students were challenged to design and create prototype pulley setups using their prior knowledge to help the elder lift a given object.	(1) Problem identification: Students were introduced to a situation where an elder has difficulty lifting a bucket of water from a well. Then, they discussed how to help the elder in a condition with no electricity. The teacher led the students to the idea of using a pulley to help the elder lift the bucket of water. Thereafter, they were introduced to more specific problems with how a pulley works and how a pulley setup is designed.
(2) Evaluating outcomes: Each group of students evaluated the performance of their pulley setup by using a balance to measure force used to lift the object. Each group of students then shared the results, which were recorded by the teacher on the blackboard for discussion.	(2) Related information search: Groups of four to six students were introduced to scientific investigation, which was conducted in a structured manner to explore how different setups of pulley(s) (e.g., one fixed pulley, one moveable pulley, and both fixed and moveable pulleys) affect the force used to lift the object. They then reached the conclusion that a moveable pulley reduces the force when compared to the weight of the object, while a fixed pulley does not.
(3) Generating reasons: By comparing the results on the blackboard, students shared their ideas about what might cause the difference in the force used to lift the same object.	(3) Solution design: Each group of students was challenged to use what they had learned from scientific investigation to draw a design of a prototype pulley setup that would lift a given object. They could search for additional information on the Internet.
(4) Testing ideas: With scaffolding provided by the teacher, each group of students examined the different ways a pulley could be set up (e.g., fixed pulley versus movable pulley). This led to scientific investigation, which was conducted in a structured manner, to explore how the different setups affect the force used to lift the object.	(4) Planning and development: Each group of students constructed a real setup based on their drawing. They could change their ideas if they wished to do so.
(5-6) Analyzing and generalizing results: Each group of students analyzed and discussed the results to conclude that a moveable pulley reduces the force when compared to the weight of the object, while a fixed pulley does not. With this conclusion, each group of students could improve their prototype pulley setup.	(5) Testing, evaluation, and design improvement: Each group of students tested and evaluated the performance of their design. They were encouraged to repeatedly improve the performance of their respective designs.
(7) Connecting to big ideas: The teacher used a virtual simulation to demonstrate how pulleys work. This simulation was connected to the idea of exploiting mechanical advantage using pulleys. The students also had an opportunity to solve physics problems.	(6) Presentation: Each group of students presented their design and its performance. Then, the teacher used a virtual simulation to demonstrate how pulleys work. This simulation was connected to the idea of exploiting mechanical advantage using pulleys. The students also had an opportunity to solve physics problems.

Item	Description	Concepts	Item difficulty
1	Students are asked to choose which of two setups, one with a fixed pulley and one with a moveable pulley, requires less force to lift the same object (Picture provided).	✓		0.38
2	Students are asked whether a setup consisting of two pulleys, a fixed one and a moveable one, requires less or more force to lift an object when compared with no pulleys (Picture provided).	✓		0.45
3	Given the same string for a pulley, students are asked whether an object can be lifted higher or lower if they use a moveable pulley instead of a fixed one (No picture).		✓	0.28
4	Students are asked whether the amount of force required to lift an object will be more or less if they use a moveable pulley instead of a fixed one (No picture).	✓		0.48
5*	Students are asked whether a moveable pulley or a fixed pulley requires less work to lift the same object to the same position from the ground (No picture).		✓	0.10
6*	Given two setups, one using a moveable pulley only and one using both a fixed pulley and a movable pulley, students are asked to choose the setup that requires less force to lift the same object (Picture provided).	✓		0.17
7	Students are asked to determine the work required to lift a 10-Newton weight using a moveable pulley to a height of 0.05 m above the ground while the position of the pulley also rises to a position 0.1 m higher than its original position (Picture provided).		✓	0.52
8	Two persons individually attempt to lift the same object. One uses a setup consisting of a fixed pulley and a movable pulley, while the other does not use this setup. Students are asked which person used less work to achieve the same task (Picture provided).		✓	0.52
9	A person lifts the same object using the same fixed pulley twice to reach different positions—the second attempt achieves a higher position than the first attempt. Students are asked to compare the work used in these two attempts (Picture provided).		✓	0.62
10	Given two setups, one using a fixed pulley only and one using both a fixed pulley and a movable pulley to individually lift the same object to the same position, students are asked to compare the work used by these two setups (Picture provided).		✓	0.48
11	Given two setups, one using a fixed pulley and one using two fixed pulleys, students are asked to compare the amount of force required to lift the same object (Picture provided).	✓		0.24
12	Given two setups, one using a fixed pulley and one using a moveable pulley, students are asked to compare the amount of force required to lift the same object (Picture provided).	✓		0.34

Class	Group	Schedule Each Week
1st class	Control	Two successive periods on Tuesday morning and a single period on Wednesday afternoon
2nd class	Experimental	Two successive periods on Tuesday morning and a single period on Thursday morning
3rd class	A single period on Tuesday afternoon and two successive periods on Thursday morning

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Thailand Science Research and Innovation and the University of Phayao, (grant number RSA6180010).

ORCID iD

Luecha Ladachart

Author Biographies

Luecha Ladachart is an associate professor in science education. He works at the School of Education, University of Phayao. His research interests are STEM education, especially design-based learning.

Visit Radchanet is a science teacher at Thumpinwittayakom School. He has 24 years of teaching science at lower secondary levels. His interests are in STEM education and design-based learning.

Wilawan Phothong is an assistant professor of educational technology at the School of Education, University of Phayao, Thailand. She is interested in computational and innovative thinking.

References

Antink-Meyer

Brown

R. A.

(2019). Nature of engineering knowledge: An articulation for science learners with nature of science understandings. Science & Education, 28(3-5), 539–559. https://doi.org/10.1007/s11191-019-00038-0

Apedoe

X. S.

Reynolds

Ellefson

M. R.

Schunn

C. D.

(2008). Bringing engineering design into high school science classrooms: The heating/cooling unit. Journal of Science Education and Technology, 17(5), 454–465. https://doi.org/10.1007/s10956-008-9114-6

Apedoe

X. S.

Schunn

C. D.

(2013). Strategies for success: Uncovering what makes students successful in design and learning. Instructional Science, 41(4), 773–791. https://doi.org/10.1007/s11251-012-9251-4

Chapman

McPhee

Proudman

(1992). What is experiential education? Journal of Experiential Education, 15(2), 16–23. https://doi.org/10.1177/105382599201500203

Chase

C. C.

Malkiewich

Kumar

A. S.

(2019). Learning to notice science concepts in engineering activities and transfer situations. Science Education, 103(2), 440–471. https://doi.org/10.1002/sce.21496

Chiang

I. A.

Jhangiani

R. S.

Price

P. C.

(2015). Research methods in psychology (2^nd Canadian edition). BCcampus.

Chini

J. J.

(2006). Comparing the scaffolding provided by physical and virtual manipulatives for students’ understanding of simple machines. Kansas State University.

Chini

J. J.

Madsen

Gire

Rebello

N. S.

Puntambekar

(2012). Exploration of factors that affect the comparative effectiveness of physical and virtual manipulatives in an undergraduate laboratory. Physical Review Special Topics – Physics Education Research, 8(1), 010113. https://doi.org/10.1103/PhysRevSTPER.8.010113

Chinn

C. A.

Malhotra

B. A.

(2002). Children’s responses to anomalous scientific data: How is conceptual change impeded? Journal of Educational Psychology, 94(2), 327–343. https://doi.org/10.1037/0022-0663.94.2.327

10.

Chusinkunawut

Henderson

Nugultham

Wannagatesiri

Fakcharoenphol

(2021). Design-based science with communication scaffolding results in productive conversations and improved learning for secondary students. Research in Science Education, 51(4), 1123–1140. https://doi.org/10.1007/s11165-020-09926-w

11.

Collins

M. A.

Totino

Hartry

Romero

V. F.

Pedroso

Nava

(2020). Service-learning as a lever to support STEM engagement for underrepresented youth. Journal of Experiential Education, 43(1), 55–70. https://doi.org/10.1177/1053825919887407

12.

Dankenbring

Capobianco

B. M.

(2016). Examining elementary school students’ mental models of sun-earth relationships as a result of engaging in engineering design. International Journal of Science and Mathematics Education, 14(5), 825–845. https://doi.org/10.1007/s10763-015-9626-5

13.

Dym

C. L.

Agogino

A. M.

Eris

Frey

D. D.

Leifer

L. J.

(2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94(1), 103–120. https://doi.org/10.1002/j.2168-9830.2005.tb00832.x

14.

Ellefson

M. R.

Brinker

R. A.

Vernacchio

V. J.

Schunn

C. D.

(2008). Design-based learning for biology: Genetic engineering experience improves understanding of gene expression. Biochemistry and Molecular Biology Education, 36(4), 292–298. https://doi.org/10.1002/bmb.20203

15.

Eyler

(2009). The power of experiential education. Liberal Education, 95(4), 24–31. https://www.aacu.org/publications-research/periodicals/power-experiential-education.

16.

Fortus

Dershimer

R. C.

Krajcik

Marx

R. W.

Mamlok-Naaman

(2004). Design-based science and student learning. Journal of Research in Science Teaching, 41(10), 1018–1110. https://doi.org/10.1002/tea.20040

17.

Gire

Carmichael

Chini

J. J.

Rouinfar

Rebello

Smith

Puntambekar

(2010). The effects of physical and virtual manipulatives on students’ conceptual learning about pulleys. Proceedings of the 9^th International Conference of the Learning Sciences. 937–943. Chicago: International Society of the Learning Sciences. https://dl.acm.org/doi/10.5555/1854360.1854480.

18.

Gomez Puente

S. M.

van Eijck

Jochems

(2013). A sampled literature review of design-based learning approaches: A search for key characteristics. International Journal of Technology and Design Education, 23(3), 717–732. https://doi.org/10.1007/s10798-012-9212-x

19.

Gopalan

Rosinger

Ahn

J. B.

(2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243. https://doi.org/10.3102/0091732X20903302

20.

Hall

Miro

(2016). A study of student engagement in project-based learning across multiple approaches to STEM education programs. School Science and Mathematics, 116(6), 310–319. https://doi.org/10.1111/ssm.12182

21.

Institute for the Promotion of Teaching Science and Technology (2015, March 4). Basic knowledge about STEM education . http://www.stemedthailand.org/wp-content/uploads/2015/03/newIntro-to-STEM.pdf.

22.

Itin

C. M.

(1999). Reasserting the philosophy of experiential education as a vehicle for change in the 21^st century. Journal of Experiential Education, 22(2), 91–98. https://doi.org/10.1177/105382599902200206

23.

Joplin

(1981). On defining experiential education. Journal of Experiential Education, 4(1), 17–20. https://doi.org/10.1177/105382598100400104

24.

Jun

Han

Kim

(2017). Effect of design-based learning on improving computational thinking. Behaviour and Information Technology, 36(1), 43–53. https://doi.org/10.1080/0144929X.2016.1188415

25.

(2014). An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing. Computer and Education, 73(1), 26–39. https://doi.org/10.1016/j.compedu.2013.12.010

26.

Kelly

T. R.

Knowles

J. G.

(2016). A conceptual framework of integrated STEM education. International Journal of STEM Education, 3, 11. https://doi.org/10.1186/s40594-016-0046-z

27.

Kolb

D. A.

(2015). Experiential learning: Experience as the source of learning and development (2^nd edition). Pearson Education, Inc.

28.

Kolodner

J. L.

Camp

P. J.

Crismond

C. D.

Fasse

Gray

Holbrook

Puntambekar

Ryan

(2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting learning by design^TM into practice. The Journal of the Learning Sciences, 12(4), 495–547. https://doi.org/10.1207/S15327809JLS1204_2

29.

Korur

Efe

Erdogan

Tunc

(2017). Effects of toy crane design-based learning on simple machines. International Journal of Science and Mathematics Education, 15(2), 251–271. https://doi.org/10.1007/s10763-015-9688-4

30.

Ladachart

Phothong

Suaklay

Ladachart

(2020). Thai Elementary science teachers’ images of engineer(s) at work. Journal of Science Teacher Education, 31(6), 631–653. https://doi.org/10.1080/1046560X.2020.1743563

31.

Lewis

(2006). Design and inquiry: Bases for an accommodation between science and technology education in the curriculum? Journal of Research in Science Teaching, 43(3), 255–281. https://doi.org/10.1002/tea.20111

32.

Wang

Xiao

Froyd

(2020). Research and trends in STEM education: A systematic review of journal publications. International Journal of STEM Education, 7, 11. https://doi.org/10.1186/s40594-020-00207-6

33.

Malkiewich

L. J.

Chase

C. C.

(2019). Focusing processes: Potential pathways for transfer of science concepts from an engineering task. International Journal of Science Education, 41(11), 1475–1495. https://doi.org/10.1080/09500693.2019.1613583

34.

Marulcu

Barnett

(2016). Impact of an engineering design-based curriculum compared to an inquiry-based curriculum on fifth graders’ content learning of simple machines. Research in Science and Technological Education, 34(1), 85–104. https://doi.org/10.1080/02635143.2015.1077327

35.

Mehalik

M. M.

Doppelt

Schuun

C. D.

(2008). Middle-school science through design-based learning versus scripted inquiry: Better overall science concept learning and equity gap reduction. Journal of Engineering Education, 97(1), 71–85. https://doi.org/10.1002/j.2168-9830.2008.tb00955.x

36.

Morgan

G. A.

Leech

N. L.

Gloeckner

G. W.

Barrett

K. C.

(2013). IBM SPSS for introductory statistics: use and interpretation. Routledge.

37.

Morrison

Bartlett

R. V.

(2009). STEM As a curriculum: An experiential approach. Education Week, 23(4), 28–31. https://www.edweek.org/teaching-learning/opinion-stem-as-a-curriculum/2009/03.

38.

Myneni

L. S.

Narayanan

N. H.

(2012). ViPS: An intelligent tutoring system for exploring and learning physics through simple machines. Proceedings of the 4^th International Conference on Computer Supported Education. 73–82. Porto: SCITEPRESS. https://www.scitepress.org/Link.aspx?doi=10.5220/0003924700730082.

39.

Park

D. Y.

Park

M. H.

Bates

A. B.

(2018). Exploring young children’s understanding about the concept of volume through engineering design in a STEM activity: A case study. International Journal of Science and Mathematics Education, 16(2), 275–294. https://doi.org/10.1007/s10763-016-9776-0

40.

Pleasants

Olson

J. K.

(2019). What is engineering? Elaborating the nature of engineering for K-12 education. Science Education, 103(1), 145–166. https://doi.org/10.1002/sce.21483

41.

Promboon

Finley

F. N.

Kaweekijmanee

(2018). The evolution and current status of STEM education in Thailand: Policy directions and recommendations. In Fry

G. W.

(Ed.), Education in Thailand: an old elephant in search of a new mahout (pp. 423–459). Springer. https://doi.org/10.1007/978-981-10-7857-6_17.

42.

Puntambekar

Kolodner

J. L.

(2005). Toward implementing distributed scaffolding: Helping students learn science from design. Journal of Research in Science Teaching, 42(2), 185–217. https://doi.org/10.1002/tea.20048

43.

Quinn

C. M.

Reid

J. W.

Gardner

G. E.

(2020). S + T + M = E as a convergent model for the nature of STEM. Science & Education, 29(4), 881–898. https://doi.org/10.1007/s11191-020-00130-w

44.

Schauble

Klopfer

L. E.

Raghavan

(1991). Students’ transition from an engineering model to a science model of experimentation. Journal of Research in Science Teaching, 28(9), 859–882. https://doi.org/10.1002/tea.3660280910

45.

Schnittka

Bell

(2011). Engineering design and conceptual change in science: Addressing thermal energy and heat transfer in eighth grade. International Journal of Science Education, 13(1), 1861–1887. https://doi.org/10.1080/09500693.2010.529177

46.

Schut

Klapwijk

Gielen

van Doorn

de Vries

(2020). Uncovering early indicators of fixation during the concept development state of children’s design processes. International Journal of Technology and Design Education, 30(5), 951–972. https://doi.org/10.1007/s10798-019-09528-2

47.

Sullivan

Gnesdilow

Puntambekar

Kim

J.-S.

(2017). Middle school students’ learning of mechanics concepts through engagement in different sequences of physical and virtual experiments. International Journal of Science Education, 39(12), 1573–1600. https://doi.org/10.1080/09500693.2017.1341668

48.

Tekbiyik

(2015). The real life application of pulleys in a competitive environment. Teaching Science, 61(1), 18–26. https://search.informit.org/doi/10.3316/INFORMIT.081672443440422.

49.

Thibaut

Ceuppens

de Loof

de Meester

Goovaerts

Struyf

Pauw

J. B.

Dehaene

Deprez

de Cock

Hellinckx

Knipprath

Langie

Struyven

van de Velde

van Petegem

Depaepa

(2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3(1), 1–12. https://doi.org/10.20897/ejsteme/85525

50.

Wecker

Rachel

Heran-Dorr

Waltner

Wiesner

Fischer

(2013). Presenting theorical ideas prior to inquiry activities fosters theory-level knowledge. Journal of Research in Science Teaching, 50(10), 1180–1206. https://doi.org/10.1002/tea.21106

Item	Description	Concepts		Item difficulty
Item	Description	Force	Work	Item difficulty
1	Students are asked to choose which of two setups, one with a fixed pulley and one with a moveable pulley, requires less force to lift the same object (Picture provided).	✓		0.38
2	Students are asked whether a setup consisting of two pulleys, a fixed one and a moveable one, requires less or more force to lift an object when compared with no pulleys (Picture provided).	✓		0.45
3	Given the same string for a pulley, students are asked whether an object can be lifted higher or lower if they use a moveable pulley instead of a fixed one (No picture).		✓	0.28
4	Students are asked whether the amount of force required to lift an object will be more or less if they use a moveable pulley instead of a fixed one (No picture).	✓		0.48
5*	Students are asked whether a moveable pulley or a fixed pulley requires less work to lift the same object to the same position from the ground (No picture).		✓	0.10
6*	Given two setups, one using a moveable pulley only and one using both a fixed pulley and a movable pulley, students are asked to choose the setup that requires less force to lift the same object (Picture provided).	✓		0.17
7	Students are asked to determine the work required to lift a 10-Newton weight using a moveable pulley to a height of 0.05 m above the ground while the position of the pulley also rises to a position 0.1 m higher than its original position (Picture provided).		✓	0.52
8	Two persons individually attempt to lift the same object. One uses a setup consisting of a fixed pulley and a movable pulley, while the other does not use this setup. Students are asked which person used less work to achieve the same task (Picture provided).		✓	0.52
9	A person lifts the same object using the same fixed pulley twice to reach different positions—the second attempt achieves a higher position than the first attempt. Students are asked to compare the work used in these two attempts (Picture provided).		✓	0.62
10	Given two setups, one using a fixed pulley only and one using both a fixed pulley and a movable pulley to individually lift the same object to the same position, students are asked to compare the work used by these two setups (Picture provided).		✓	0.48
11	Given two setups, one using a fixed pulley and one using two fixed pulleys, students are asked to compare the amount of force required to lift the same object (Picture provided).	✓		0.24
12	Given two setups, one using a fixed pulley and one using a moveable pulley, students are asked to compare the amount of force required to lift the same object (Picture provided).	✓		0.34