Effects of Immersion on Knowledge Gain and Cognitive Load in Additive Manufacturing Process Education

Abstract

Although the additive manufacturing (AM) market continues to grow, industries face barriers to AM adoption due to a shortage of skilled designers in the workforce that can apply AM effectively to meet this demand. This shortage is attributed to the high cost and infrastructural requirements of introducing high- barrier-to-entry AM processes such as powder bed fusion (PBF) into in-person learning environments. To meet the demands for a skilled AM workforce, it is important to explore other mediums of AM education, such as computer-aided instruction (CAI) and virtual reality (VR), which can increase access to hands-on learning experiences for inaccessible AM processes. However, limited work compares virtual and physical AM instruction or explores how the differences in immersion and presence between mediums can affect the knowledge gained and the mental effort exerted when learning about different AM processes. To address this gap in the literature, this research evaluates the use of CAI, VR, and in-person instruction in AM process education when learning about material extrusion (ME) and PBF. Our findings show that the differences in immersion and presence between CAI, VR, and in-person instruction do not have a statistically significant effect when learning about ME, but do have a significant effect when learning about PBF. Specifically, we found that VR generally yields equivalent effects in knowledge gain and cognitive load to in-person PBF education while offering advantages in both metrics over CAI learning. The findings from this work thus have significant implications for using VR as an alternative to in-person training to improve designer development in process-centric AM education of typically high-barrier-to-entry AM processes.

Introduction

The additive manufacturing (AM) industry expanded by nearly 7.5% to roughly $12.8 billion within the year 2020¹ with a 2 × growth forecasted to roughly $37.2 billion for 2026.² This continued market growth is driven by the demand for rapid design and manufacturing of complex products by leveraging AM capabilities in geometrical, hierarchical, functional, and material complexity. This can be observed in expert projections that suggest that by 2030, manufacturing of less critical spare parts will be primarily driven by AM and a significant amount of AM products will leverage capabilities in multimaterial fabrication and product development with embedded electronics.³ Although the demand for AM continues to grow, there is a deficit of designers and engineers in the workforce suited to meet this demand and apply the technology to different product design opportunities.^4,5 Inadequate in-house AM and design for additive manufacturing (DfAM) knowledge due to this deficit of designers presents a barrier to the integration of AM^6,7 within organizations.

Therefore, the future workforce must be equipped with the skills and knowledge in AM and DfAM to meet this growing demand for AM and drive future innovation in industrial product development.

Design and process-centric AM education can help prepare the AM workforce⁸ and empower designers to innovate with AM. The process-dependent nature of DfAM and applying AM in product development^9–11 indicate that in-depth process-centric education for the full range of AM processes can complement the growth of DfAM intuition and improve a designer's versatility with AM. However, observable barriers to entry faced by AM systems (e.g., cost, safety, required infrastructure^12–14) inhibit designers from accessing knowledge for AM processes like powder bed fusion (PBF) within educational institutions and communities. There is a need to provide accessible and in-depth education on the range of AM systems, and there is an opportunity to do so by leveraging virtual mediums such as computer-aided instruction (CAI) and virtual reality (VR). This research is thus motivated to explore this opportunity and address this inaccessibility to AM knowledge to improve the design capabilities of the future AM-driven design and engineering workforce.

Simulation and gaming-structured CAI has historically addressed this need and enhanced different learning outcomes,^15,16 including declarative and procedural knowledge in science, engineering, and manufacturing^17–20 that typically require in-person instruction. While nonimmersive virtual tools like CAI can potentially benefit AM education, research shows that enhancing immersion and presence can improve the experience and its outcomes.^21–25 This is because the characteristics of the media, tools, and human-related factors, such as spatial perception and reasoning and psychomotor skills, strongly influence the design, learning, or engineering experience.²⁶ There is, therefore, an opportunity to explore immersive VR in addition to CAI as a tool for AM education.

Past work indicates that VR improves the development of declarative and procedural knowledge,²⁷ cognitive and affective skills,²⁸ and memory recall²⁹ compared to CAI. Immersive technology is already driving industry uses of VR in engineering and manufacturing to support decision-making and enable innovation³⁰ by enhancing engineering education,²¹ allowing engineers to make fewer mistakes in procedural manufacturing and assembly tasks²⁴ compared to in-person product assembly and take lesser task completion times compared to both CAI and in-person^24,25 conditions. Literature even shows early promise in developing designer intuition in design and process-centric AM concepts^31,32 using VR.

There are mixed effects of VR technology in science and education^33–37 that highlight how the environmental and pedagogical conditions of learning strongly affect the learning experience. However, inductive learning techniques such as task-based and problem-based learning in engineering^38,39 are well suited for AM education⁴⁰ and present pedagogical frameworks that lean toward procedural and declarative learning experiences that are ideal within immersive learning. Past work also suggests that immersion and presence have mixed influences in the observed cognitive load as influenced by the manual operations required during the experience.^41–46 Different cognitive load aspects affect learning,⁴⁷ including variations in modality between learning mediums of varying immersion.⁴⁸ Therefore, it is crucial to gauge how immersive experiences specifically for AM education can affect cognitive load to better understand the simultaneous effects on learning. Collectively, these observations from the literature strengthen the need to compare mediums of varying immersion and presence on the specific application of AM education to expand the existing knowledge bases in both AM and VR.

Immersion and presence in virtual environments give users a “vivid illusion of reality”^27,49 where the reality of the physical world exhibits the highest levels of immersion and presence. Virtual realities are a collaboration of immersion and presence^28,49 surrounding users in a digital space that mimics the sensory elements of the physical reality and are thus measured as the extent to which the virtual environment can surround users to simulate immersion and presence. Traditional computer displays typically fall under nonimmersive VR, and head-mounted displays (HMDs) fall under immersive VR.^28,50

Although past work indicates that there may be differences in educational effect specifically due to immersion or presence or both,^34,36 this research does not differentiate the three mediums specifically between immersion and presence and assumes an overall change in both from CAI to VR to REAL. For further sake of clarity, this research simplifies the objective and subjective relationship between immersion and presence and henceforth refers to both solely using the term immersion with the following distinctions between the studied mediums: CAI = nonimmersive virtual medium (i.e., a flat computer screen), VR = immersive virtual medium (i.e., an HMD with controllers), REAL = immersive physical medium (i.e., the physical world).

Literature shows that the immersion of a medium strongly influences the learning and the mental effort experienced during an educational experience; however, limited work in the supportive knowledge for AM and DfAM³² investigates how the medium in which a designer learns about AM affects their education. New knowledge on how the mediums affect the AM educational experience can be leveraged to further improve industrial product development processes by better training and equipping designers for the AM-driven product demands in the workforce. This research, therefore, aims to address this gap in the literature by exploring the following key research questions:

RQ1: How do the differences in immersion between CAI, VR, and REAL mediums affect knowledge gain when learning about ME and PBF?

We hypothesize that the PBF group will generally show higher knowledge gains than the material extrusion (ME) group.²⁰ For both AM processes, learning through VR and REAL will yield higher knowledge gains than will learning through CAI with identical trends observed between the two immersive conditions.³² This is expected due to the effects of the varying capabilities offered by the conditions during the procedural learning experience: capabilities such as interactivity, immersion, psychomotor coordination, memory recall,²⁹ and spatial perception and reasoning.²⁶

RQ2: How do the differences in immersion between CAI, VR, and REAL mediums affect cognitive load when learning about ME and PBF?

We hypothesize that the PBF group will generally show similar cognitive load trends to the ME group.²⁰ For both AM processes, learning through VR and REAL will yield lower cognitive load trends than will learning through CAI with identical trends observed between the two immersive mediums.^41,45 This is also expected due to the effects of the varying capabilities offered by the conditions which affect the difficulty of navigating the learning environment and conducting self-learning actions within the environmental restrictions; specifically, due to the changes in difficulty of processing task-related information and performing manual operations^41,42,45,46 with the change in immersion.

Materials and Methods

Participants in this research were first-year undergraduate students recruited from an introduction to engineering design course at an R1 university. Volunteers were first informed of their rights and options as per IRB protocol before conducting the study. This information included reassurances that their participation would be anonymous but they may choose to opt out of participating or releasing their data in this research if they experience physical, mental, or ethical discomfort of any kind and that their participation (or lack of) would not affect their academic standing. Participants in the VR condition were reminded to use these rights should they experience nausea, dizziness, or sickness when using the VR equipment. Those who opted in to participate were provided an online Qualtrics survey that they completed on their PCs. Participants volunteered as groups during class time or independently outside of class time and were assigned to one of the three conditions (i.e., either CAI, VR, or REAL) for one of the two AM processes (i.e., either ME or PBF) by a study coordinator. Balancing the number of data points between all the conditions was also handled by the study coordinator.

During the study, participants shared information about their background and interests in AM (see Assessing the Participants' Backgrounds section) and a pre-post assessment of their AM process knowledge and cognitive load (see Measuring Knowledge Gain and Cognitive Load section) from our 13-min intervention (see Completing the Tutorial and Intervention section). This section elaborates on the specifics of the designed experimentation.

Assessing the participants' backgrounds

Participants first shared their interest and motivation regarding learning about AM and using AM. They indicated their agreement to the posed questions on interest and motivation on a 5-point likert scale that ranged from strongly agree to strongly disagree.⁵¹ They also shared their awareness of the overall AM technology. Collectively, the data on interest, motivation, and AM awareness helped strengthen the statistical analysis of the results of knowledge gain and cognitive load by authenticating the participant's engagement in the study and accounting for prior knowledge that could affect the findings. Participants in the CAI and VR conditions also shared their comfort levels in working with or interacting with 3D models (i.e., virtual objects) within their specific conditions. Awareness of interaction in CAI and VR was also recorded on a 5-point Likert scale that covered identical options in each topic.⁵¹ Before moving on to the experiment, participants completed the prequiz⁵² for their assigned AM process, data from which were used with the postquiz data to assess knowledge gain.

Completing the tutorial and intervention

The designed experiment included a custom 4-min tutorial for the assigned condition that instructed them on how to navigate and interact within their condition. Participants in the CAI, VR, and REAL conditions practiced performing tasks and completing objectives identical to the upcoming intervention to familiarize themselves with the capabilities and limitations of their medium. The tutorial, therefore, instructed participants on tasks that required familiarity in picking up and moving objects and navigating within a bounded space. Completing the tutorial session was followed by the 13-min intervention for the assigned AM process where they learned about the AM process and completed tasks to reinforce their learning. Participants assigned to the CAI condition were directed to the tutorial and intervention in the survey on their computers. Those assigned to the VR and REAL conditions were directed to designated study zones where they were provided the equipment and tools needed to complete the exercise.

Participants in the VR condition were given a wired HTC Vive headset and a pair of wireless controllers. Participants in the REAL condition were directed to the physical objects and machines and were instructed to follow along with the audio playing on a device next to the machine. All conditions were designed to foster the same level of involvement during testing while allowing free interaction with the machines, objects, and environment to the extent permitted within the given medium. The virtual environments for the CAI (Fig. 1a) and VR (Fig. 1b) conditions were designed as web applications using Unity: a cross-platform game engine popularly used to design virtual experiences, and included virtual parts and machines to interact with. The design of the REAL condition (Fig. 1c) included physical parts and machines where the physical parts were manufactured using the specific AM process the participants learned about and underwent no postprocessing to specifically highlight the effects of the manufacturing process.

FIG. 1.

Showcasing a participant completing a 60-s task of loading material into the AM machine to highlight the experimental design setup across the conditions and between the AM processes. (a) CAI interaction using a computer mouse and keyboard for task completion. (b) VR interaction using a VR controller and physical movement for task completion. (c) REAL interaction using physical movement for task completion. AM, additive manufacturing; CAI, computer-aided instruction; VR, virtual reality.

Educational concepts from a functional decomposition framework (Fig. 2) were used as the pedagogical foundation for the intervention to provide an on-par comparison between the AM processes when observing knowledge gain and cognitive load.

FIG. 2.

Highlighting the concepts derived from the functional classification framework that are used to design the educational experiences and define the relevant tasks.

Based on the functional classification framework by Williams et al,⁵³ this decomposition framework focused on five key process-centric concepts: (i) material identification and storage, (ii) supplying material to the system, (iii) patterning material or energy, (iv) creating primitives, and (v) generating support structures. Figure 1 illustrates a 60-s-long sample task performed during the intervention for the different conditions and AM processes where participants were verbally instructed about the raw material used for the AM process and were then encouraged to load the material into the machine given sufficient time to attempt the task on their own. All tasks were similarly associated with each concept⁵⁴ scripted specifically to the LulzBot Taz 6 for the ME condition and the Xact Metal XM200C for the PBF condition. To focus on how variations between the conditions influenced the difficulty in performing tasks during the intervention, tasks between the ME and PBF conditions were designed to be of identical conceptual difficulty as per the decomposition framework derived from past work by Williams et al.⁵³

All tasks were constrained to those that would be safe and permitted in a typical in-person learning environment with physical machines. To ensure the safety of the participants in the REAL condition, the physical machine for the ME group was not powered and the physical machine for the PBF group only housed the skeletal structure of the machine with functionality suitable for safe demo purposes. In addition, the PBF group handled a powder-like substitute to teach participants about the raw material for the PBF process.

Measuring knowledge gain and cognitive load

Paired data from a pre-and post-quiz assessment were used to measure knowledge gain as the difference in quiz scores. One quiz variant for each AM process was designed, and participants completed the quiz specific to their assigned process⁵² before and after the intervention. The questions in the quiz were formulated using the same terminology as used in the intervention. All the questions were objective, single-answer, or multiple-answer type questions to ensure simplicity in calculating the quiz scores and knowledge gained through the change in quiz scores. Every question offered an “I don't know” option to minimize the probability that students would try to guess the correct answer. No negative scoring was done, and all questions were worth a maximum of one point. Certain concepts required adding additional questions to the quiz to ensure that all the relevant elements of the concepts were tested; therefore, the number of questions differed between the two conditions (i.e., ME had 10 and PBF had 9). Pre- and post-quizzes were tallied and normalized where normalization entailed that the entire set of scores was rescaled between 0 and 1 for both the quizzes using the min-max feature scaling approach. Statistical analysis for knowledge gain was performed on the normalized scores.

Participants reported their cognitive load using the Workload Profile Assessment (WPA) tool⁵⁵ by sharing the mental effort they exerted during the learning experience. Participants scored each of the eight workload profile dimensions (i.e., the perceptual, response, spatial, verbal, visual, auditory, manual, and speech) independently between 0 and 10 to represent their exerted mental effort. They received a textual and audio description of each dimension to review, along with an example of how cognitive resources for each dimension might be applied to a relatable task to better assess their cognitive load.

Results

This research collected a sample size of 237 data points with the distribution shown in Table 1.

Table 1.

Showing the Distribution of Participants Across the Conditions and Additive Manufacturing Processes

	CAI	VR	REAL
ME	79	18	13
PBF	82	21	24

CAI, computer-aided instruction; ME, material extrusion; PBF, powder bed fusion; VR, virtual reality.

From this participant pool, we collected demographic data, knowledge gain data, and cognitive load data and report these collective data and the results from its analyses while maintaining all outliers. To account for the complexity of the repeated measures experimental setup and the presence of multiple dependent and independent variables in its statistical analysis, this research uses linear regression modeling (lm) for the demographic and cognitive load data and linear mixed-effects regression modeling (lmer) for the knowledge gain (i.e., pre-post quiz) data. A 95% confidence interval was generally used to determine statistical significance (i.e., p < 0.05); however, certain trends around the 95% interval are discussed as emerging trends and not statistically significant under the discretion of this research. The assumptions for linear regression and linear mixed-effects regression modeling were checked for violations using the Peña and Slate⁵⁶ and the Loy and Hofmann⁵⁷ procedures, respectively. This research did not find any observable violations and relies on the acceptable range for the robustness of lms and lmers in its reported findings.

Demographic analysis of the participants

Regressing the interest and motivation levels on the centered process (ME = −0.5, PBF = 0.5; between-subjects variable) showed no observable statistically significant difference between participants assigned to both the AM processes in interest and motivation. However, regressing the interest and motivation levels on the centered condition (CAI = −0.5, VR = 0, REAL = 0.5; between-subjects variable) showed a significant effect within conditions in interest and motivation such that participants generally reported higher interest and motivation in AM as the condition changed from CAI to VR to REAL (for interest to learn AM: b = 0.306, F(1, 233) = 8.085 [t(233) = 2.843], p = 0.005, for interest to use AM: b = 0.287, F(1, 233) = 5.395 [t(233) = 2.323], p = 0.021, for motivation to learn AM: b = 0.414, F(1, 233) = 9.086 [t(233) = 3.014], p = 0.003, for motivation to use AM: b = 0.404, F(1, 233) = 8.515 [t(233) = 2.918], p = 0.004). As shown in Figure 3, many participants agreed or strongly agreed that they were interested and motivated to learn about and use AM within each condition and each AM process.

FIG. 3.

Showcasing the reported interest and motivation to learn and use AM across the conditions and AM processes.

These levels of interest and motivation indicate that participants were authentically engaged with the study and thus strengthen the authenticity of the data collected for knowledge gain and cognitive load.

Regressing the distributions of the prior awareness in AM on the centered condition (CAI = −0.5, VR = 0, REAL = 0.5; between-subjects variable) and process (ME = −0.5, PBF = 0.5; between-subjects variable) showed no observable statistically significant difference between the conditions, b = 0.147, F(1, 233) = 1.087 [t(233) = 1.043], p = 0.298, or between the AM processes, b = 0.02, F(1, 233) = 0.024 [t(233) = 0.156], p = 0.876. As shown in Figure 4a, this means that participants' perceived awareness with general AM across the conditions and AM processes was generally identical and therefore was not accounted for as a variable of interest in later analyses.

FIG. 4.

Showcasing the (a) prior awareness in AM and (b) comfort levels with VR and CAI across the conditions and AM processes.

Regressing the distributions of the prior comfort with interaction in CAI and VR on the centered condition (CAI = −0.5, VR = 0.5; between-subjects variable) and process (ME = −0.5, PBF = 0.5; between-subjects variable) showed a significant difference between the conditions, b = −0.987, F(1, 233) = 31.223 [t(233) = −5.588], p < 0.001, but not between the AM processes, b = 0.229, F(1, 233) = 1.681 [t(233) = 1.296], p = 0.196. This means that participants in the CAI condition generally had a significantly higher comfort with CAI technology than did participants in the VR condition with VR technology. This can be observed in Figure 4b where a significantly higher number of participants reported that they had never worked with VR before the study indicating that they were novices to VR.

These results were expected as this research worked with primarily first-year undergraduate students from an engineering design course at an R1 university who would have completed some CAI course requirements, and likely not have completed any VR course work. While the varying comfort levels between CAI and VR could influence the study, with the limited scope in mind for this work, we acknowledge the limitation of not accounting for technology comfort levels which will be considered as an opportunity for future work.

Effects on knowledge gain by immersion for the different AM processes

Figure 5 shows the key results of the analysis of knowledge gain for each AM process across each condition.

FIG. 5.

Showcasing the distribution of quiz scores and the net knowledge gain as affected by the three conditions between the two AM processes.

For this analysis, quiz score (collapsed pre- and post-quiz scores) was regressed on the centered variables for condition (CAI = −0.5, VR = 0, REAL = 0.5; between-subjects variable), process (ME = −0.5, PBF = 0.5; between-subjects variable), quiz time (prequiz = −0.5; postquiz = 0.5; within-subjects variable) and the interaction of these three variables (condition × process × quiz) as the covariates. This analysis also included a by-subject random intercept and a by-subject random slope for the quiz variable, utilized restricted maximum likelihood estimation to iteratively modify the parameter estimates to minimize the log-likelihood function, and evaluated this model with the Kenward and Rogers (KR) adjustment.⁵⁸ The following results reported from the analysis focus on each detailed effect when controlling for all other main effects and interactions in the model.

Process-wise comparison of knowledge gain across the conditions

To understand the differences in the knowledge gain between the conditions and AM processes, we estimated the two-way interaction between condition and quiz time, b = 0.2, F(1, 233) = 21.65 [t(233) = 4.653], p < 0.001, and process and quiz time, b = 0.323, F(1, 233) = 67.504 [t(233) = 8.216], p < 0.001. These results show that the knowledge gain significantly differed between the conditions and the AM processes. Specifically, participants in PBF generally experienced a higher knowledge gain than participants in ME. Participants also generally experienced a higher knowledge gain as the condition changed from CAI to VR to REAL.

Conducting pairwise-comparison analyses for process within each condition provided specific insight into the differences in knowledge gain between the AM processes for each condition. Results showed that knowledge gain significantly differed when comparing ME to PBF in the CAI (b = 0.25, F(2, 231) = 43.56 [t(231) = 6.6], p < 0.001), VR (b = 0.38, F(2, 231) = 24.01 [t(231) = 4.9], p < 0.001), and REAL (b = 0.36, F(2, 231) = 18.49 [t(231) = 4.3], p < 0.001) conditions. This means that the participants experienced a higher knowledge gain for PBF than for ME in each condition.

Conducting additional pairwise-comparison analyses for condition within each process provided further insight into the differences in knowledge gain between the conditions for each process. As shown in Figure 5 for the ME process, knowledge gain did not significantly differ in comparisons between CAI to VR (b = 0.051, F(2, 231) = 0.64 [t(231) = 0.8], p = 0.424) and VR to REAL (b = 0.093, F(2, 231) = 1.21 [t(231) = 1.1], p = 0.292), but showed an emerging trend between CAI to REAL (b = 0.144, F(2, 231) = 4 [t(231) = 2], p = 0.048). However as shown in Figure 5 for the PBF process, knowledge gain significantly differed in comparisons between the CAI to VR (b = 0.178, F(2, 231) = 9 [t(231) = 3], p = 0.003) and CAI to REAL (b = 0.253, F(2, 231) = 20.25 [t(231) = 4.5], p < 0.001), but not in the comparison between VR to REAL (b = 0.075, F(2, 231) = 1 [t(231) = 1], p = 0.299). This means that the participants did not experience a statistically significant difference in knowledge gain in ME between CAI, VR, and REAL; however, they did experience a higher knowledge gain in PBF as the condition changed from CAI to VR or REAL.

Analyses supporting the observed knowledge gain results

The main analysis showed a significant effect of the quiz time on quiz scores such that on collapsing the condition and process categories, participants generally scored higher in the postquiz than in the prequiz, b = 0.424, F(1, 233) = 464.312 [t(233) = 21.548], p < 0.001. As can be observed in Figure 5, this means that participants generally experienced a statistically significant knowledge gain (i.e., the difference between prequiz and postquiz scores) because the postquiz scores are generally higher than the prequiz scores across the conditions and AM processes.

To evaluate whether scores specifically improved significantly for participants in each condition and for each process, we examined the simple effects of the quiz time for each condition and AM process by recentering condition and process around each level in the variable and then performing the analysis using those variables in turn. Condition was recentered around CAI (CAI = 0, VR = 0.5, REAL = 1), VR (CAI = −0.5, VR = 0, REAL = 0.5), and REAL (CAI = −1, VR = −0.5, REAL = 0), respectively, and process was recentered around PBF (PBF = 0, ME = 1) and ME (PBF = −1, ME = 0), respectively. These analyses provided insight into whether there was a significant knowledge gain for participants in each condition or only for one of the conditions and in each process or only for one of the processes. The simple effects of quiz time (Table 2) showed that participants in each condition and process scored significantly higher on the postquiz than on the prequiz.

Table 2.

Highlighting the Simple Effects from the Pre- to Post-Quiz Comparisons Across the Conditions and Additive Manufacturing Processes

Process	Condition	Estimate	F	t Ratio	p
PBF	REAL	0.7	198.81	14.1	<0.001
PBF	VR	0.62	139.24	11.8	<0.001
PBF	CAI	0.44	278.89	16.7	<0.001
ME	REAL	0.34	25	5	<0.001
ME	VR	0.24	18.49	4.3	<0.001
ME	CAI	0.19	51.84	7.2	<0.001

These results can also be observed in Figure 5 which shows that the postquiz scores are much higher than the prequiz scores for all the conditions and processes, therefore suggesting that the knowledge gain was significant across the board.

To better understand how the trends in quiz scores contributed to the significance of the observed knowledge gain, we also evaluated the simple effects of process and condition at each quiz time (prequiz and postquiz). We did so by recentering quiz time around prequiz (prequiz = 0, postquiz = 1) and postquiz (prequiz = −1, postquiz = 0), respectively, and then performing the analysis with those variables in turn. This allowed us to understand whether the participants differed from one another in prequiz, postquiz, or both, between the conditions and processes. The simple effects analysis of condition and process at each quiz time (Table 3) shows that prequiz scores for each process were not significantly different between the conditions (Table 3a); however, prequiz scores for ME were significantly higher than prequiz scores for PBF in each condition (Table 3b).

Table 3.

Highlighting the Simple Effects from the Comparisons Between Conditions and Additive Manufacturing Processes at Different Quiz Times

Process	Comparison	Quiz	Estimate	F	t Ratio	p
(a) Quiz scores compared between CAI, VR, and REAL for each AM process
ME	CAI to VR	Post	0.04	0.49	0.7	0.437
ME	CAI to VR	Pre	≈0	≈0	≈0	0.982
ME	CAI to REAL	Post	0.16	4.84	2.2	0.022
ME	CAI to REAL	Pre	0.02	0.09	0.3	0.695
ME	VR to REAL	Post	0.11	1.69	1.3	0.184
ME	VR to REAL	Pre	0.02	0.09	0.3	0.735
PBF	CAI to VR	Post	0.14	5.29	2.3	0.018
PBF	CAI to VR	Pre	−0.03	0.49	−0.7	0.47
PBF	CAI to REAL	Post	0.24	19.36	4.4	<0.001
PBF	CAI to REAL	Pre	≈0	≈0	≈0	0.935
PBF	VR to REAL	Post	0.1	1.96	1.4	0.141
PBF	VR to REAL	Pre	0.03	0.25	0.5	0.597

Condition	Quiz	Estimate	F	t Ratio	p
(b) Quiz scores compared between ME and PBF for each condition
CAI	Pre	−0.22	49	−7	<0.001
VR	Pre	−0.26	15.21	−3.9	<0.001
REAL	Pre	−0.25	12.96	−3.6	<0.001
CAI	Post	0.02	0.36	0.6	0.532
VR	Post	0.11	1.96	1.4	0.139
REAL	Post	0.1	1.44	1.2	0.21

AM, additive manufacturing.

This means that participants in the ME group generally had more prior knowledge of ME than did participants in the PBF group had of PBF.

Table 3 and Figure 5 also show that the postquiz scores did not significantly differ between the AM processes for each condition (Table 3b) suggesting that participants in both AM processes generally ended up with equivalent knowledge within each condition. However, Table 3a shows that postquiz scores were significantly impacted by the conditions within each AM process.

For the ME process, Table 3a and Figure 5 show that the postquiz scores did not significantly differ between CAI to VR and VR to REAL, but they did significantly differ between CAI to REAL. This means that participants in CAI, VR, and REAL ended up with equivalent knowledge in ME except when comparing CAI to REAL where participants from the REAL condition gained more knowledge than did participants from the CAI condition.

For the PBF process, Table 3a and Figure 5 show that the postquiz scores did not significantly differ between VR to REAL, but they did significantly differ between CAI to VR and CAI to REAL. This means that participants when learning about PBF ended up with equivalent knowledge between VR and REAL, but ended up with higher knowledge from the VR and REAL than from CAI.

Effects on cognitive load by immersion for the different AM processes

Table 4 and Figure 6 show key results of the analysis of cognitive load for each AM process across each condition.

FIG. 6.

Showcasing the distribution of reported cognitive load as affected by the three conditions between the two AM processes.

Table 4.

Highlighting the Cognitive Load Experienced by Participants for Each Dimension Due to the Condition and Process Variables

Dimension	Estimate	F	t Ratio	p
(a) General cognitive load as affected by the condition variable
Perceptual	−0.14	0.01	−0.31	0.752
Response	−0.5	1.34	−1.15	0.247
Spatial	−1.46	11.06	−3.32	<0.001
Verbal	0.01	≈0	0.03	0.971
Visual	0.23	0.24	0.49	0.623
Auditory	−0.08	0.03	−0.17	0.863
Manual	−0.55	1.15	−1.07	0.284
Speech	−0.74	3.61	−1.9	0.058

Dimension	Estimate	F	t Ratio	p
(b) General cognitive load as affected by the process variable
Perceptual	0.71	3.01	1.73	0.083
Response	1.12	7.84	2.8	0.005
Spatial	0.37	0.86	0.93	0.353
Verbal	1.19	7.16	2.67	0.008
Visual	0.7	2.63	1.62	0.106
Auditory	1.16	6.95	2.63	0.008
Manual	0.09	0.04	0.2	0.835
Speech	0.22	0.37	0.61	0.538

For this analysis, cognitive load was regressed on the centered variable for condition (CAI = −0.5, VR = 0, REAL = 0.5; between-subjects variable) and process (ME = −0.5, PBF = 0.5; between-subjects variable) and the interaction of these two variables (condition × process) as the covariates. This analysis also included a by-subject random intercept, utilized restricted maximum likelihood estimation to iteratively modify the parameter estimates to minimize the log-likelihood function, and evaluated this model with the KR adjustment.⁵⁸ The following results reported from the analysis focus on each detailed effect when controlling for all other main effects and interactions in the model.

As can be observed from Figure 6, the main analysis showed no statistically significant effect on the overall cognitive load by condition such that on collapsing the process categories, participants generally reported equivalent cognitive load for each of the WPA dimensions between CAI, VR, and REAL. However, an exception in the analysis shows a significant effect of condition on the spatial dimension (Table 4a) where participants generally reported a significantly lower spatial cognitive load with the change in condition. The main analysis also showed no statistically significant effect on the overall cognitive load by process such that on collapsing the condition categories, participants generally reported equivalent cognitive load for each of the WPA dimensions between ME and PBF. However, the analysis shows a significant effect on the response dimension (Table 4b) where participants generally reported a significantly higher response, verbal, and auditory cognitive load for PBF than for ME.

To further understand the trends in the cognitive load between the conditions and AM processes, we conducted pairwise-comparison analyses between the different levels in condition and process. Table 5 shows that cognitive load generally did not significantly differ between the ME and PBF processes for the CAI and VR conditions, but response, verbal, visual, and auditory cognitive load in PBF was significantly higher than in ME from the REAL condition.

Table 5.

Highlighting the Cognitive Load Comparison Between Material Extrusion and Powder Bed Fusion Experienced by Participants for Each Dimension from Each Condition

Dimension	Condition	Estimate	F	t Ratio	p
Perceptual	CAI	0.3	0.56	0.75	0.454
Perceptual	VR	−0.27	0.1	−0.33	0.74
Perceptual	REAL	1.71	3.86	1.96	0.05
Response	CAI	0.15	0.17	0.41	0.679
Response	VR	−0.48	0.38	−0.61	0.537
Response	REAL	3.12	13.8	3.71	<0.001
Spatial	CAI	−0.02	≈0	−0.05	0.956
Spatial	VR	−0.21	0.07	−0.27	0.786
Spatial	REAL	1.07	1.59	1.26	0.207
Verbal	CAI	0.02	≈0	0.04	0.96
Verbal	VR	0.77	0.76	0.87	0.384
Verbal	REAL	2.56	7.35	2.71	0.007
Visual	CAI	−0.3	0.52	−0.72	0.471
Visual	VR	0.07	≈0	0.09	0.926
Visual	REAL	2.11	5.23	2.28	0.023
Auditory	CAI	0.2	0.23	0.48	0.627
Auditory	VR	0.23	0.07	0.26	0.79
Auditory	REAL	2.57	7.7	2.77	0.005
Manual	CAI	0.08	0.03	0.19	0.847
Manual	VR	−1.38	2.26	−1.5	0.133
Manual	REAL	1.12	1.28	1.13	0.257
Speech	CAI	−0.13	0.14	−0.38	0.702
Speech	VR	−0.25	0.13	−0.36	0.718
Speech	REAL	0.78	1.08	1.04	0.299

Figure 6 also shows that cognitive load for the ME process generally did not significantly differ between the conditions but spatial and response cognitive load from the REAL condition was significantly higher than cognitive load from the CAI condition. For the PBF process, however, cognitive load generally did significantly differ between the conditions in various pairwise comparisons. As shown in Figure 6, participants reported significantly lower spatial cognitive load from VR than from CAI and significantly lower response and auditory cognitive load from VR than from REAL. In addition, participants reported significantly higher auditory, verbal, and visual cognitive load from REAL than from CAI.

Discussion

The findings highlighted in Results section present key implications for the proposed research questions in this work. The following section elaborates on the interpretation behind the observed results and their underlying mechanisms.

RQ1: How do the differences in immersion between CAI, VR, and REAL mediums affect knowledge gain when learning about ME and PBF?

Our collective findings in Effects on Knowledge Gain by Immersion for the Different AM Processes section reaffirm the existing knowledge gap in process-centric AM among designers and indicate that while any medium of instruction from this research can yield significant knowledge gains, differences in immersion between the conditions strongly affect knowledge gain when comparing learning between the different AM processes. Our analysis identified statistically significant differences in knowledge gain between the studied AM processes such that designers learning about PBF generally experienced a higher knowledge gain than designers learning about ME (32.3% higher). This trend was observed while accounting for the significantly higher prequiz knowledge in the ME group than in the PBF group with identical postquiz knowledge in both AM process groups (see Analyses Supporting the Observed Knowledge Gain Results section).

Paired with the findings on perceived prior awareness in AM from Demographic Analysis of the Participants section, these results indicate that there exists a knowledge gap among designers between ME and PBF with designers having more knowledge in more accessible processes like ME than knowledge in less accessible processes like PBF.

The results and analysis in Process-Wise Comparison of Knowledge Gain Across the Conditions section further identified a statistically significant effect of immersion on knowledge gain. Designers generally experienced a higher knowledge gain as the condition linearly changed from CAI to VR to REAL (20% higher). This implies that increased immersion can increase the knowledge gained from process-centric AM education. Specifically, however, designers did not experience a significant difference in knowledge gain across the mediums when learning about ME, but designers did show significantly higher knowledge gains when learning about PBF through VR and REAL than through CAI with no statistically significant difference in learning between VR and REAL. This means that immersion does not have a significant effect when learning about typically accessible AM processes like ME, but does have a significant effect when learning about typically inaccessible AM processes like PBF.

This finding suggests that VR education can yield equivalent knowledge gains to REAL education while bypassing restrictions in introducing process-centric AM education for high-barrier-to-entry systems like PBF. VR instruction may hence offer industries an alternative to in-person education with higher knowledge gains during designer development than CAI of typically high-barrier-to-entry AM processes.

RQ2: How do the differences in immersion between CAI, VR, and REAL mediums affect cognitive load when learning about ME and PBF?

Our collective findings in Effects on Cognitive Load by Immersion for the Different AM Processes section indicate that the differences in immersion generally do not strongly affect the mental effort experienced when comparing learning between different AM processes, but specifically have significant impacts within the different medium and AM process pairwise combinations. This research limits the discussion of its findings to sight and motor-sensory information (i.e., limited to perceptual, response, spatial, visual, and manual cognitive load) as the verbal, auditory, and speech cognitive load dimensions were attributed to the design of the experimentation and not inherent to the mediums themselves. The analysis in Effects on Cognitive Load by Immersion for the Different AM Processes section identified that designers generally experienced a significantly higher response processing cognitive load when learning about PBF than when learning about ME (11.2% higher). Specifically, however, Table 5 and Figure 6 show that the general trends in cognitive load observed are strongly influenced from REAL learning.

Similarly influenced emerging and significant trends from REAL learning were observed in perceptual and visual processing cognitive load, respectively (Table 5). These findings indicate that designers found learning about PBF to require more mental effort than learning ME when through REAL learning, but found virtual learning (i.e., through CAI and VR) about the two AM processes to require identical mental effort. For industries, this implies that virtual instruction, immersive or nonimmersive, may yield lower mental effort exertion when learning about typically inaccessible and functionally complex AM processes like PBF.

The results and analysis in Effects on Cognitive Load by Immersion for the Different AM Processes section also found that designers experienced a significantly lower spatial processing cognitive load (14.6% lower) as the medium of instruction changed from CAI to VR to REAL. Figure 6 shows that compared to CAI learning, designers specifically experienced a significantly lower spatial cognitive load from REAL learning in the ME group and from VR learning in the PBF group with comparable effects between the two immersive mediums. Additional emerging and significant trends observed in Figure 6 for perceptual, response, and visual cognitive load support the finding that adding immersion to the learning experience can lower the mental effort exerted by designers during certain learning experiences. Specifically, our findings indicate that as the AM process changes from a functionally less complex process like ME to a more complex process like PBF, designers require less mental effort from immersive mediums than nonimmersive mediums.

This implies that designers may benefit more from immersive instruction than nonimmersive instruction to lower exerted mental effort when learning about typically inaccessible and functionally complex AM processes like PBF.

Conclusion

The goal of this research was to identify the effects of immersion in the learning experience and study how immersion in different mediums of instruction (i.e., CAI, VR, REAL) affects the knowledge gain and the mental effort experienced when learning about different AM processes (i.e., ME, PBF). This research measured the pre-and post-quiz scores to study the knowledge gained from the experience and measured cognitive load using the WPA tool to study the mental effort experienced.

The results in Results section indicate that immersion does not have a significant effect when learning about easily accessible and functionally less complex AM processes like ME, but does have a significant effect when learning about less accessible and functionally more complex AM processes like PBF. Immersion (virtual or physical) does not significantly affect knowledge gain when learning about ME; however, immersive mediums yield higher knowledge gains than nonimmersive mediums when learning about PBF. Specifically, VR provides comparable knowledge gain of PBF concepts to REAL instruction while presenting a significant advantage in knowledge gain over CAI. Furthermore, physical immersion yields lower response and spatial cognitive load than immersive and nonimmersive virtual instruction when learning about ME but yields higher response and visual cognitive load when learning PBF. Adding immersion to virtual instruction using VR further yields a lower spatial cognitive load when learning about PBF.

The findings from this work have significant implications for using VR instruction to offer improved designer development in process-centric AM education as an alternative to in-person education and bypassing the restrictions in introducing process-centric AM education for high-barrier-to-entry systems like PBF.

While the findings from this research highlight significant differences between the three mediums (CAI, VR, REAL) in the knowledge gain and cognitive load when learning about different AM processes, these findings need to be considered with certain limitations of this work.

First, regarding the broader scope of this interdisciplinary work, this work limited its scope to observe large effect sizes within rudimentary AM learning experiences inspired by inductive teaching techniques often used in task- and problem-based learning in engineering and AM education^38–40; however, this work did not use any specific pedagogical framework to design its intervention and teaching experiences. Future work needs to account for the effects of different learning styles and teaching methods on knowledge gain and experiential cognitive load. In addition, the correlated effect of cognitive load on learning also needs to be further evaluated by considering how different cognitive load aspects affect learning,⁴⁷ including how variations in modality and cueing across mediums of varying immersion affect learning.⁴⁸ Finally, the designed virtual experiences, while identical to each other, were not validated by standards or specifications from literature.⁵⁹

Experiences within different forms of VR systems that vary in perceived immersion, interaction, sensory feedback, device modalities, and other VR specifications⁵⁹ may yield different learning and cognitive effects. Future work aims to check the different noninvestigated items considered in the design of VR experiences on their effects on AM learning and experiential cognitive load. Even within the specifics of this research, several limitations should be addressed in future work. This work did not filter participants by prior knowledge or ensure that prior knowledge was homogeneous before studying the effects of the intervention on knowledge gain and cognitive load. Rather, this work was aware of the limited formal AM education that participants had before the study and hypothesized that the level of prior knowledge would be different between the two AM processes (i.e., ME and PBF) but homogeneous across the three conditions (i.e., CAI, VR, REAL) within each process.

Although results from the prequiz indicated homogeneity in prior knowledge among the participants, future work should control for prior knowledge as a main independent variable and study how prior AM expertise can influence knowledge gain and cognitive load. Similarly, participants in the virtual conditions had significantly different comfort levels within their respective mediums; specifically, participants had a significantly higher comfort with CAI interaction than with VR interaction. Future work can account for such differences in skill and comfort on their effects on knowledge gain and cognitive load during learning. Knowledge gain in this work was measured using a pre-post quiz assessment and thus assumed to be short term and linear in nature. This approach, however, limits the information collected as knowledge gain and does not assess other learning aspects such as transference and long-term retention. Future work could consider expanding the scope of defining knowledge gain and reassess the effects of immersion on AM education.

In addition, the data collected were unevenly distributed and much smaller in size in the VR and REAL conditions. This is because a majority of this research was conducted during the COVID pandemic and, as such, volunteers leaned toward virtual and remote participation than in-person participation. Future work can expand the current data set to further improve the resolution and power of these findings by collecting data from a larger and more evenly distributed sample of participants. Furthermore, this work did not investigate the novelty of VR over CAI learning to understand why VR learning yielded significant differences from CAI learning but identical effects to REAL learning for PBF. Future work aims to conduct in-depth qualitative studies, including think-aloud exercises, interviews, and analysis of video and screen recordings, to understand why learning experiences may yield the observed outcomes from this work. New knowledge from such future work can aid industries and further empower their designers to meet AM-driven product design needs for a range of AM processes.

Footnotes

Acknowledgments

The authors thank Dr. Stephanie Cutler for her guidance and advice and Xact Metal for providing the 3D data for their powder bed fusion machine and their continued assistance with this research initiative.

Authors' Contributions

J.M.: Conceptualization, Methodology, Investigation, Formal Analysis, Writing—Original draft; S.M.: Funding acquisition, Writing—Review and editing, Supervision; T.S.: Funding acquisition, Writing—Review and editing, Supervision; N.M.: Resources, Funding acquisition, Writing—Review and editing, Supervision.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This research was conducted through the support of the National Science Foundation under grant number 2021267. Any opinions, findings, and conclusions expressed in this article are those of the authors and do not necessarily reflect the views of the NSF.

References

Wohlers

, Campbell

, Diegel

, et al. Wohlers Report 2021: 3D Printing and Additive Manufacturing Global State of the Industry. Wohlers Associates: Fort Collins, CO, USA; 2021.

Roberts

, Bartkova

Additive Manufacturing Trend Report. Hubs, Inc.: Amsterdam, The Netherlands; 2021.

Jiang

, Kleer

, Piller

. Predicting the future of additive manufacturing: A Delphi study on economic and societal implications of 3D printing for 2030. Technol Forecast Soc Change, 2017; 117:8497; doi: 10.1016/j.techfore.2017.01.006

Ford

, Despeisse

. Additive manufacturing and sustainability: An exploratory study of the advantages and challenges. J Clean Prod, 2016; 137:15731587; doi: 10.1016/j.jclepro.2016.04.150

Pei

, Monzón

, Bernard

(eds.) Additive Manufacturing Developments in Training and Education. Springer International Publishing: Cham; 2019.

Huang

, Leu

. Frontiers of Additive Manufacturing Research and Education. University of Florida: Gainesville, FL, USA; 2014.

Thomas-Seale

LEJ

, Kirkman-Brown

, Attallah

, et al. The barriers to the progression of additive manufacture: Perspectives from UK industry. Int J Prod Econ, 2018; 198:104118; doi: 10.1016/j.ijpe.2018.02.003

Simpson

, Williams

, Hripko

. Preparing industry for additive manufacturing and its applications: Summary & recommendations from a National Science Foundation workshop. Addit Manuf, 2017; 13:166178; doi: 10.1016/j.addma.2016.08.002

Ponche

, Kerbrat

, Mognol

, et al. A novel methodology of design for additive manufacturing applied to additive laser manufacturing process. Robot Comput Integr Manuf, 2014; 30(4):389398; doi: 10.1016/j.rcim.2013.12.001

10.

Kumke

, Watschke

, Hartogh

, et al. Methods and tools for identifying and leveraging additive manufacturing design potentials. Int J Interact Des Manuf, 2018; 12(2):481493; doi: 10.1007/s12008-017-0399-7

11.

Meisel

, Williams

. An investigation of key design for additive manufacturing constraints in multimaterial three-dimensional printing. J Mech Des, 2015; 137(11):111406; doi: 10.1115/1.4030991

12.

Huang

, Leu

, Mazumder

, et al. Additive manufacturing: Current state, future potential, gaps and needs, and recommendations. J Manuf Sci Eng, 2015; 137(1):014001; doi: 10.1115/1.4028725

13.

Rayna

, Striukova

. From rapid prototyping to home fabrication: How 3D printing is changing business model innovation. Technol Forecast Soc Change, 2016; 102:214224; doi: 10.1016/j.techfore.2015.07.023

14.

Smith

, Pollard

. The role of computer simulations in engineering education. Comput Educ, 1986; 10(3):335340; doi: 10.1016/0360-1315(86)90002-3

15.

Merchant

, Goetz

, Cifuentes

, et al. Effectiveness of virtual reality-based instruction on students learning outcomes in K-12 and higher education: A meta-analysis. Comput Educ, 2014; 70:2940; doi: 10.1016/j.compedu.2013.07.033

16.

Makransky

, Petersen

. Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Comput Educ, 2019; 134:1530; doi: 10.1016/j.compedu.2019.02.002

17.

Deshpande

, Huang

. Simulation games in engineering education: A state-of-the-art review. Comput Appl Eng Educ, 2011; 19(3):399410; doi: 10.1002/cae.20323

18.

de Jong

, Linn

, Zacharia

. Physical and virtual laboratories in science and engineering education. Science, 2013; 340(6130):305308; doi: 10.1126/science.1230579

19.

Ouyang

S-G

, Wang

, Yao

J-Y

, et al. A unity3D-based interactive three-dimensional virtual practice platform for chemical engineering. Comput Appl Eng Educ, 2018; 26(1):91100; doi: 10.1002/cae.21863

20.

Mathur

, Miller

, Simpson

, et al. Analysis of the Knowledge Gain and Cognitive Load Experienced Due to the Computer-Aided Instruction of Additive Manufacturing Processes. In: Volume 4: 18th International Conference on Design Education (DEC): Virtual. 2021; p. V004T04A013. [Online].

21.

Alhalabi

Virtual reality systems enhance students achievements in engineering education. Behav Inf Technol, 2016; 35(11):919925; doi: 10.1080/0144929X.2016.1212931

22.

Wolfartsberger

Analyzing the potential of virtual reality for engineering design review. Autom Constr, 2019; 104:2737; doi: 10.1016/j.autcon.2019.03.018

23.

Guo

, Zhou

, Chen

, et al. Using virtual reality to support the products maintainability design: Immersive maintainability verification and evaluation system. Comput Ind, 2018; 101:4150; doi: 10.1016/j.compind.2018.06.007

24.

Abidi

, Al-Ahmari

, Ahmad

, et al. Assessment of virtual reality-based manufacturing assembly training system. Int J Adv Manuf Technol, 2019; 105(9):37433759; doi: 10.1007/s00170-019-03801-3

25.

Bharathi

AKBG

, Tucker

. Investigating the Impact of Interactive Immersive Virtual Reality Environments in Enhancing Task Performance in Online Engineering Design Activities. In: Volume 3: 17th International Conference on Advanced Vehicle Technologies; 12th International Conference on Design Education; 8th Frontiers in Biomedical Devices. Boston, Massachusetts, USA; 2015; p. V003T04A004.

26.

Buxton

Sketching User Experiences: Getting the Design Right and the Right Design. Elsevier/Morgan Kaufmann: Amsterdam Boston; 2007.

27.

Radianti

, Majchrzak

, Fromm

, et al. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput Educ, 2020; 147:103778; doi: 10.1016/j.compedu.2019.103778

28.

Jensen

, Konradsen

. A review of the use of virtual reality head-mounted displays in education and training. Educ Inf Technol, 2018; 23(4):15151529; doi: 10.1007/s10639-017-9676-0

29.

Krokos

, Plaisant

, Varshney

. Virtual memory palaces: Immersion aids recall. Virtual Reality, 2019; 23(1):115; doi: 10.1007/s10055-018-0346-3

30.

Berg

, Vance

. Industry use of virtual reality in product design and manufacturing: A survey. Virtual Reality, 2017; 21(1):117; doi: 10.1007/s10055-016-0293-9

31.

Ostrander

, Ryan

, Dhengre

, et al. A Comparative Study of Virtual Reality and Computer-Aided Design to Evaluate Parts for Additive Manufacturing. In: Volume 2A: 45th Design Automation Conference. Anaheim, California, USA; 2019; p. V02AT03A029.

32.

Ostrander

, Tucker

, Simpson

, et al. Evaluating the use of virtual reality to teach introductory concepts of additive manufacturing. J Mech Des, 2020; 142(5):051702; doi: 10.1115/1.4044006

33.

Hamilton

, McKechnie

, Edgerton

, et al. Immersive virtual reality as a pedagogical tool in education: A systematic literature review of quantitative learning outcomes and experimental design. J Comput Educ, 2021; 8(1):132; doi: 10.1007/s40692-020-00169-2

34.

Grassini

, Laumann

, Rasmussen Skogstad

. The use of virtual reality alone does not promote training performance (but sense of presence does). Front Psychol, 2020; 11:1743; doi: 10.3389/fpsyg.2020.01743

35.

Markowitz

, Laha

, Perone

, et al. Immersive virtual reality field trips facilitate learning about climate change. Front Psychol, 2018; 9:2364; doi: 10.3389/fpsyg.2018.02364

36.

Makransky

, Andreasen

, Baceviciute

, et al. Immersive virtual reality increases liking but not learning with a science simulation and generative learning strategies promote learning in immersive virtual reality. J Educ Psychol, 2021; 113(4):719735; doi: 10.1037/edu0000473

37.

Klippel

, Zhao

, Oprean

, et al. The value of being there: Toward a science of immersive virtual field trips. Virtual Reality, 2020; 24(4):753770; doi: 10.1007/s10055-019-00418-5

38.

Boud

, Feletti

The Challenge of Problem-Based Learning. Routledge, New York, NY, USA; 1997.

39.

Duch

, Groh

, Allen

. The Power of Problem-Based Learning: A Practical How To for Teaching Undergraduate Courses in Any Discipline. Stylus Publishing, LLC.: Sterling, VA, USA; 2001.

40.

Prabhu

, Miller

, Simpson

, et al. Complex solutions for complex problems? Exploring the role of design task choice on learning, design for additive manufacturing use, and creativity. J Mech Des, 2020; 142(3):031121; doi: 10.1115/1.4045127

41.

Starkey

, McKay

, Hunter

, et al. Piecing together product dissection: How dissection conditions impact student conceptual understanding and cognitive load. J Mech Des, 2018; 140(5):052001; doi: 10.1115/1.4039384

42.

Pontonnier

, Dumont

, Samani

, et al. Designing and evaluating a workstation in real and virtual environment: Toward virtual reality based ergonomic design sessions. J Multimodal User Interfaces, 2014; 8(2):199208; doi: 10.1007/s12193-013-0138-8

43.

Wismer

, Reinerman-Jones

, Teo

, et al. A Workload Comparison During Anatomical Training with a Physical or Virtual Model. In: Augmented Cognition: Users and Contexts. (Schmorrow DD and Fidopiastis CM. eds.) Springer International Publishing: Cham; 2018; pp. 240252.

44.

Makransky

, Terkildsen

, Mayer

. Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn Instr, 2019; 60:225236; doi: 10.1016/j.learninstruc.2017.12.007

45.

Armougum

, Orriols

, Gaston-Bellegarde

, et al. Virtual reality: A new method to investigate cognitive load during navigation. J Environ Psychol, 2019; 65:101338; doi: 10.1016/j.jenvp.2019.101338

46.

Frederiksen

, Sørensen

SMD

, Konge

, et al. Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: A randomized trial. Surg Endosc, 2020; 34(3):12441252; doi: 10.1007/s00464-019-06887-8

47.

Sweller

, van Merriënboer

JJG

, Paas

. Cognitive architecture and instructional design: 20 Years later. Educ Psychol Rev, 2019; 31(2):261292; doi: 10.1007/s10648-019-09465-5

48.

Tabbers

, Martens

, Merriënboer

JJG

. Multimedia instructions and cognitive load theory: Effects of modality and cueing. Br J Educ Psychol, 2004; 74(1):7181; doi: 10.1348/000709904322848824

49.

Slater

, Wilbur

. A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence Teleoperators Virtual Environ, 1997; 6(6):603616; doi: 10.1162/pres.1997.6.6.603

50.

Freina

, Ott

. A Literature Review on Immersive Virtual Reality in Education: State of the Art and Perspectives. In: The 11th International Scientific Conference eLearning and Software for Education Bucharest, April 23–24, 2015; pp. 101007; doi: 10.12753/2066-026X-15-020

51.

Mathur

Participant Background Questions. Made by Design Laboratory: University Park, Pennsylvania, USA; 2022. Available from: https://sites.psu.edu/madebydesign/files/2022/10/Participant-Background-Questions.pdf [Last accessed: January 26, 2023].

52.

Mathur

AM Process Quizzes. Made by Design Laboratory: University Park, Pennsylvania, USA; 2017. Available from: https://sites.psu.edu/madebydesign/files/2017/07/AMCAI-Quiz-Questions.pdf [Last accessed: January 26, 2023].

53.

Williams

, Mistree

, Rosen

. A functional classification framework for the conceptual design of additive manufacturing technologies. J Mech Des, 2011; 133(12); doi: 10.1115/1.4005231

54.

Mathur

Intervention Script. Made by Design Laboratory: University Park, Pennsylvania, USA; 2017. Available from: https://sites.psu.edu/madebydesign/files/2017/07/AMXR-Script.pdf [Last accessed: January 26, 2023].

55.

Tsang

, Velazquez

. Diagnosticity and multidimensional subjective workload ratings. Ergonomics, 1996; 39(3):358381; doi: 10.1080/00140139608964470

56.

Peña

, Slate

. Global validation of linear model assumptions. J Am Stat Assoc, 2006; 101(473):341354; doi: 10.1198/016214505000000637

57.

Loy

, Hofmann

. HLMdiag: A suite of diagnostics for hierarchical linear models in R. J Stat Soft, 2014; 56(>5); doi: 10.18637/jss.v056.i05

58.

Kenward

, Roger

. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 1997; 53(3):983; doi: 10.2307/2533558

59.

Fuchs

, Moreau

, Guitton

Virtual Reality: Concepts and Technologies. CRC Press: Boca Raton, FL, USA; 2011.