Abstract
A problem-based immersive virtual environment (IVE) about bushfire safety was developed as a learning tool for children aged 10–12. Its effectiveness was assessed in relation to children’s ability to determine how to be safer in a bushfire incident. A series of experiential activities were developed in the IVE with digital storytelling and two-stage embedded assessments providing children with an opportunity to engage with tasks and solve problems while receiving feedback on their performance. Changes from pre- to postsurvey results showed positive learning outcomes as evidenced by significant improvements in children’s knowledge of bushfire safety and confidence in their ability to contribute to decisions during a bushfire incident. The significant change in children’s knowledge as well as their performance at two-stage embedded assessments was independent of their gender, background knowledge and perceived ability in responding to bushfire hazards. This suggests that when appropriately designed and implemented within educational settings, immersive virtual learning tools can effectively engage children and enhance learning outcomes associated with bushfire safety. The paper also argues that such immersive problem-based learning can improve self-efficacy amongst children in relation to coping with a bushfire situation. Implications of the findings are also discussed.
Keywords
Introduction
The frequency and intensity of bushfires is increasing due to climate change effects and the number of people likely to be affected is expected to rise (Bardsley et al., 2018). Peek (2008) contends that children are one of the most vulnerable demographic groups to bushfire disasters and they need to be empowered to understand this natural phenomenon and how they can be safer in the event of a fire, especially children living in peri-urban and other bushfire prone areas. However, for a variety of legitimate reasons, much bushfire education targets adults. Wisner et al. (2018) highlight that some parents make conscious decisions to not involve their children in preparation activities because they are limited by their own fear about the vulnerability of their children. McNeill and Ronan (2017) reported that parents with young children (under twelve) plan less for bushfire threat than childless families because they perceive they lack the ability and time for preparation. Peek (2008) argues there are two main reasons children are at higher risk of harm from bushfires than adults: they are psychologically vulnerable due to their shorter span of experience and time to build resilience skills; and they are physically less strong. Another important barrier impacting children’s vulnerability is the fact they have limited knowledge and understanding of bushfire behaviour and protective response. Towers (2015) studied children’s knowledge of bushfire hazards and preparation in bushfire prone areas and found that children lacked knowledge and understanding of appropriate sheltering strategies and elements of fire behaviour. Children had a tendency to shelter passively and did not have a clear understanding of the mechanisms through which people and properties could be affected by ember attack and radiant heat. Children also showed a tendency to apply their knowledge from the context of house fire safety to bushfire safety. This can be quite dangerous as there are many differences, such as recognizing the protective value of the home in a bushfire situation. Towers (2015) suggests that children have limited and misleading knowledge of fire behaviour in the context of bushfire, which adds to their vulnerability.
Nevertheless, vulnerability of children and their need of support does not mean that children need to be passive victims of bushfires disasters. Children have the capacity to develop particular skills in responding to the disasters (Wisner et al., 2018). An important strategy for reducing this vulnerability is to equip children with the required knowledge and develop particular skills that assist in responding to bushfire threats (Towers, 2012). In recent years, building children’s resilience to natural disasters through education has received notable attention among researchers (Masten, 2021; Mohammadinia et al., 2018; Treichel, 2020). In addition to knowledge and skill provision for children, educational programmes in school settings have substantial potential to influence household motivation for preparation activities for two reasons: children have the capacity to transfer their knowledge of bushfire safety messages to their families, and the ability to influence their parents’ attitude and behaviour toward preparation (Towers, 2012). Empowering children through bushfire educational programmes not only requires delivering the required knowledge and skills to children but also helping them to improve their self-efficacy beliefs (Towers, 2015). This is considered a key predictor of behavioural responses and preparation intentions in the area of natural disasters (Solberg et al., 2010). Bandura (1997) defined self-efficacy as a personal judgement on how well or poorly a person could perform in a given situation based on their skills and the circumstance.
De et al. (2020) argue that current approaches at conventional disaster educational programmes for children are not sufficiently engaging for the diversity of children that is typical in most classrooms. Although conventional communication methods may shape children’s bushfire risk perception, Kazanidis et al. (2020) argue they are not effective in changing self-efficacy and motivating children to become more prepared because students do not generally see the personal relevance. Towers (2015) states that ‘Bushfire education that advances children’s knowledge will require learning activities that encourage each child to voice their perspectives’ (p. 187).
Given the significant variation in knowledge and beliefs about bushfires, as well as life experiences and attitudes among children, a method that is interactive, adaptable and engaging seems a prudent way forward.
Virtual reality (VR) technologies have been found to be useful in teaching elementary school children (Falloon, 2020; Hooshyar et al., 2021; Kurniasih et al., 2020; Nand et al., 2019; Yu et al., 2021). Recent years also have seen an increase in the application of immersive virtual environments (IVEs) for educational purposes among children at elementary level (Dickes et al., 2019; Mead et al., 2019; Merchant, 2012). This is in part due to improved technology, diverse applications and decreased cost (Klippel et al., 2019). IVEs are able to engage students in constructing knowledge and enhancing critical thinking skills. Simultaneously, IVEs can improve student motivation and engagement which in turn provides opportunities for the transfer of knowledge and skills to real-world situations (Falloon, 2020; Mead et al., 2019; Moysey & Lazar, 2019; Zeng et al., 2020). IVE-based instructions have the potential to facilitate education and transferring skills to a wide range of students with diverse backgrounds (Merchant, 2012). Limited work to date has focused on the use of IVEs in educating children about fire safety (Ahmad et al., 2019; Çakiroğlu & Gökoğlu, 2019; Ericson, 2007; Smith & Ericson, 2009), and there is a need to further explore the potential of IVEs in bushfire education programmes targeting children.
Literature Review
Application of VR in Education
The educational potential of VR has been recognised across many discipline areas (Shin, 2017; Vesisenaho et al., 2019). Chang et al. (2020) found that implementation of VR improved students’ learning achievement in natural science, enhanced their learning motivation, self-efficacy and problem-solving. An increase in competence among elementary students was also reported by Sun et al. (2021) following a VR experience. However, Pellas et al. (2020) suggest further research is required to understand the educational value of applying VR in STEM education. Three main uses of virtual worlds in educational environments are in designing experiential spaces, communication spaces and simulating physical spaces (Hew & Cheung, 2010). Integration of immersive technologies such as head-mounted displays (HMDs) to virtual worlds increased the ability of individuals to actively interact with simulated environments while experiencing high levels of presence (Gao et al., 2019). According to Klingenberg et al. (2020), higher levels of immersion can lead to enhanced performance and self-efficacy. A review study on the application of HMDs in post-secondary level education and skill training found, in comparison to traditional methods, immersive VR is a more convenient, engaging and interactive approach to learning (Concannon et al., 2019). Other studies have also reported on the value of IVEs to enhance learning (Dawley & Dede, 2013; Liu et al., 2020; Parong & Mayer, 2018; Shi et al., 2019). While the application of VR technology in knowledge and skill acquisition for children has increased in recent years (Hu et al., 2016), few studies have explored the influence of using HMDs for elementary and middle school students’ learning performance (Southgate et al., 2019). Research that has been conducted showed promise in the use of immersive VR in elementary education with IVEs outperforming desktop virtual environment in terms of learning outcomes (Kozhevnikov et al., 2013; Webster, 2016). Makransky et al. (2020) suggest this improvement is due to a more innovative learning experience and greater student motivation. In the area of science and natural resource management, IVEs have been found to influence both academic performance and engagement across a diversity of age ranges (Mayer, 2014). For example, Liu et al. (2020) reported improvement among sixth grade students using IVEs compared to students exposed to traditional teaching methods and Chen et al. (2020) reported the same among tenth grade students. Educational virtual environments designed to educate children about environmental science are also increasing (Dawley & Dede, 2013), demonstrating the value of immersive learning technologies in primary and middle school education (Dickes et al., 2019). Two examples of such environments are EcoXPT (Dede et al., 2017) and EcoMOD (Dickes et al., 2019) which are both designed to educate children about ecosystem science by engaging them as ecosystem scientists who observe or collect data over time within virtual ecosystems. Dickes et al. (2019) have reported on the application of IVEs in elementary or middle schools environmental science, and Chang et al. (2020) used IVEs to teach geology. According to Klippel et al. (2019), IVEs using a head-mounted device are superior to traditional field trips.
Digital games can also facilitate improved learning outcomes for children. For example, Sardone and Devlin-Scherer (2009) found that games increase interest in learning and facilitate problem-solving among students in grades 6–12. Shi et al. (2019) reported significant improvements in students’ learning motivation among seventh grade students who played a game-based immersive learning environment. A study by Ronelus (2016) comparing the effectiveness of a role-playing educational game with a traditional text-based instructional approach showed that the virtual educational intervention was more effective in helping learners to develop mastery of science content, scientific reasoning abilities, knowledge transfer and academic self-efficacy for fifth grade students compared to more traditional pedagogical approaches.
However, not all studies in educational settings have found immersive VR environments to be more effective. For example, the results of a study by Makransky et al. (2019) showed that despite students feeling more present with the IVE environment, those who participated in a simulated course in science via HMDs learned less than the students who learned via a desktop display. In another study, Parong and Mayer (2018) compared the application of an IVE to teach college students biology with a group taught using an off-the-shelf instructional lesson. They found that despite lower engagement and lower motivation, participants of a self-directed study with PowerPoint on a desktop computer outperformed the immersive virtual group. The authors suggested two reasons to explain the difference in effectiveness, first, the IVE group might have been distracted from the learning content with irrelevant visual information; second, participants of self-directed study with PowerPoint were able to allow time for reflection and processing the previous information, while VR participants were being continuously presented the new information. The differences in findings highlight the importance of any educational programme being driven by sound pedagogical principles and learning outcomes and not simply the application of new technology. A review study by Jensen and Konradsen (2018) suggested that using HMDs are particularly effective where understanding spatial information, or visual scanning of an environment is required during the activities or in cases where emotional responses might be evoked, or affect users’ responses.
Self-Efficacy, Gender Differences and Performance in VR
Virtual environments enable students to engage in problem-based and experimental learning activities while increasing their engagement and self-efficacy beliefs (Clarke-Midura & Dede, 2010). The application of VR in education is mainly due to its ability to increase student engagement, motivation (Makransky & Lilleholt, 2018) and self-efficacy (Chen et al., 2014). People with stronger sense of self-efficacy are more motivated and are more likely to persist in doing a task and consequently have higher chance of success in tasks compared to individuals with lower self-efficacy beliefs (Webb-Williams, 2014). According to Bandura (1997), the primary source of self-efficacy is mastery experiences, defined as an individuals’ level of success in their past performances. Watching the level of success or failures of oneself or peers (vicarious experiences), social persuasion and physiological or affective states are other sources of self-efficacy. Immersive experiences can influence the sources of self-efficacy (Hsieh et al., 2008; Ketelhut, 2007; Rosenberg-Kima et al., 2008).
Self-efficacy is an important factor in self-directed learning experiences (Dede et al., 2017). Application of immersive VR in science learning enhances students’ self-efficacy in science classroom (Cheng & Tsai, 2019). Makransky et al. (2020) reported that simulated immersive environments increased students’ self-efficacy and interest in lab safety. Past studies showed participation in IVE-based activities increases student’ scientific inquiry self-efficacy regardless of their performance within the virtual environment (Chen et al., 2014; Meluso et al., 2012). However, Bradley et al. (2015) examined possible changes to self-efficacy among middle school students after experiencing a virtual game-based assessment in science and found that while self-efficacy beliefs did increase for those students who gained high scores, it decreased for students who gained low scores in the module. Bradley et al. (2015) argued that such differences could be related to the nature of the tasks; for example, the time constraints and the nature of the feedback provided during IVE-based assessment might have accounted for the differences. The authors further suggest that provision of constructive feedback and multiple opportunities to achieve mastery might help those who fail to succeed during their first attempts in conducting the tasks to maintain their self-efficacy beliefs.
The potential of virtual environments in improving students’ self-efficacy in science has been explored through different studies. For example, improved self-efficacy was one of the aims associated with EcoMUVE, as an immersive virtual ecology curriculum (Chen et al., 2014). Improvements in self-efficacy have flow-on effects in terms of student engagement and performance. Students who had higher levels of science self-efficacy before EcoMUVE showed more interest in experiencing the virtual environment. Initial self-efficacy of learners often predicts their performance at IVE-based assessment (Ketelhut, 2007; Nelson & Ketelhut, 2008). Chen et al. (2014) and Reilly et al. (2020) reported statistically significant improvements in self-efficacy exhibited by middle year students (year 6 and year 7, respectively) in relation to science. The authors believed that the students’ self-efficacy increased because they were able to master the subject matter as they were provided with the required information through their experience in EcoMUVE. Mastery is defined as the ability to apply knowledge and skills in real world contexts. A study by Chen et al. (2016) stated that the existence of positive emotions such as excitement and other affective states enhanced students’ self-efficacy. According to Metcalf et al. (2014), taking the role of ecosystem scientists within the virtual environment helped students to strengthen their belief in their ability to solve science problems. Ketelhut (2007) reported that while students’ initial self-efficacy influenced performance, after repeated experiences within the virtual curriculum, learners identified with their role as a scientist who conducts successful experiments, and this improved their self-efficacy. In addition to improvements in self-efficacy, the EcoMUVE experience has been associated with improved knowledge in several studies (Metcalf et al., 2011, 2013). IVEs which are designed to promote learning and involve mastering tasks in personally relevant, realistic situations based on sound pedagogy have the ability to enhance knowledge motivation and self-efficacy of the learners. It is likely that female and male students evaluate their self-efficacy beliefs differently (Pajares et al., 1999). A study by Webb-Williams (2017) examined sources of self-efficacy beliefs among male and female students aged between 10 and 12. For children with similar levels of academic performance, the most important source of self-efficacy for males was found to be a mastery experience while girls were more influenced by the combination of vicarious experience and affective states.
The role of gender in science self-efficacy beliefs among middle school students has found to be negligible in some studies (Chen & Usher, 2013; Kıran & Sungur, 2012; Usher & Pajares, 2008), yet significant in others (Bradley et al., 2015; Huang, 2013; Nietfeld et al., 2014; Webb-Williams, 2014). Some studies report higher self-efficacy in males and others in females (Huang, 2013). The study by Bradley et al. (2015) revealed minor differences between the self-efficacy and performance of male and female students, as did work by Nietfeld et al. (2014). Webb-Williams (2014) reported significant differences in self-efficacy beliefs of primary school children aged between 10 and 12 years based on gender, with boys performing more poorly and exhibiting a lower sense of self-efficacy than girls. Lukosch et al. (2017) reported female participants outperformed male participants in virtual planning tasks. Chen et al. (2019) examined gender differences in learning outcomes in digital game-based instruction of primary school students and reported improvements in female students’ learning outcome was less compared to males after experiencing the virtual learning environment. Another study by Chen et al. (2021) found male students had higher learning performance and learning efficacy compared to female counterparts. On the other hand, Huang et al. (2018) reported no significant differences in learning in a virtual learning system on the basis of gender. Similarly, Rodríguez-Andrés et al. (2016) found no differences in performance of male and female children when conducting a short-term memory assessment using a 3D virtual environment.
There is likely also a relationship between gender and the type of task students are engaged in. Chen et al. (2021) reported that male students at elementary school level have higher self-efficacy and interest toward technology compared to female students. Whereas Boghi et al. (2006) found females outperformed males on memory tasks. Cui et al. (2021) reported higher levels of immersion and engagement for female students than males in a role-playing educational game–based virtual environment suggesting that story-based games may be more effective for female students. Males have often been found to outperform females in terms of spatial tasks (Habig, 2020), suggesting a need to consider gender in IVEs that inherently include some degree of visualization. According to Fokides (2017), gender contributes to shaping the learning process and its role in VR environments is understudied. He highlights the mixed results regarding the effect of gender on learning in virtual environments at primary school level as evidence for the need for greater attention which is also suggested by others (Boghi et al., 2006; Bressler et al., 2019; Cui et al., 2021).
Theoretical Background
Instructional designs which are based on constructivist learning theory provide opportunities for learners to construct knowledge from their own experiences through engaging in purposeful learning activities (Oyelere et al., 2020; Saenz, 2015; Statti & Villegas, 2020). Problem-based learning (PBL), as a constructivist learning model, is an instructional method which focuses on applying knowledge and skills to solve problems in which students work as problem solvers in the context similar to one they might encounter in real life (Russell, 2016). The virtual PBL model uses a PBL model as a template to design the virtual scenarios in which learners get engaged in purposeful virtual activities in an attempt to address a real-world issue (Jonassen, 2000). According to Russell (2016), a virtual PBL model not only has the benefits of IVEs such as high levels of engagement in a simulated real-life situation but is also underpinned by elements of instructional design and sound understanding of cognitive processes. Examples of cognitive processes embedded in PBL design include motivation to solve problems and analytical skills. Learning assessment and feedback processes are also incorporated in the Virtual PBL model (Russell, 2016). Feedback is a key element of any assessment and can provide a positive influence on learners’ performance (Oliver, 2017). According to Gresalfi et al. (2009), PBL followed by immediate feedback is an effective learning strategy. It can promote the ability to think critically and creatively (Imelda & Anzelina, 2020; Tajudin & Chinnappan, 2016). In the 1950s and 1960s, a group of educators and researchers developed Bloom’s Taxonomy, a model describing cognitive processes of learning and developing mastery of content (Huitt, 2011). This was an important step in articulating the need for educators to facilitate ways to encourage students to think more deeply, rather than just being able to recall or memorise facts. The Revised Bloom’s Taxonomy defines two key dimensions: low order thinking skills which includes memorizing, understanding and applying; and higher order thinking skills (HOTS) which involves analysing, evaluating and creating (Anderson et al., 2001). Analysing the information, making comparisons and solving problems as elements of higher order thinking are important factors in designing assessments since they facilitate developing the capability of an individual to think critically (Ping et al., 2017; Syarifah et al., 2019). HOTS impacts learners’ decision-making, and the ability to transfer knowledge into different situations (Yurniwati & Utomo, 2020). Hence, knowledge transfer requires some elements of higher order thinking including analysing and evaluating.
‘Anchored instruction theory’ helps to facilitate the knowledge transfer which takes place after experiencing the meaningful environment. During the process of knowledge transfer, students apply the acquired knowledge to solve problems within a realistic, meaningful context (Pellegrino & Brophy, 2008). Involving users in interesting virtual worlds similar to real-world events through digital storytelling stimulates learning (Berki & Mystakidis, 2018). According to Gunbas (2012), in a digital environment, students would solve problems using data embedded within the context of the narrative. Stories are a valuable technique in facilitating problem solving because they affect students’ understanding of a problem, motivation to solve a problem and can provide a sense of joy associated with the process of finding the solution. Importantly, stories can influence ability to remember sequences of events or processes (Gunbas, 2012), a valuable skill in bushfire preparation. While there are inherent advantages of IVEs, the previous research all points to the need for IVEs to be based on sound pedagogy as that, rather than the technology itself, is what leads to successful learning outcomes. This means educators must consider the wider educational environment and how the IVE is embedded in it, as well as the constructive alignment of the IVE. Constructive alignment assumes constructivism is used as a framework in the instructional design but also that the objectives, assessments and teaching methods are aligned to maximise the learning potential (Biggs, 1996). How we constructed such an environment is described below.
The Present Study
Constructivist learning theory and PBL as an instructional design based on this theory guided the theoretical framework for the design of the virtual bushfire learning tool. Digital storytelling was also incorporated in the design of the learning tool to facilitate PBL. The two-stage embedded assessment within the virtual learning environment was aligned with the key learning objectives and applied Bloom’s Taxonomy in its design.
Aim of the Study and Research Questions
The aim of the study was to develop and assess the capacity of a virtual PBL environment and its two-stage embedded assessments, as a learning tool designed to educate children (aged 10–12) about bushfire safety. The hypothesis of the study was that all students regardless of their gender, prior knowledge and self-efficacy will be effectively engaged and learn from the problem-based virtual immersive experience. The following six research questions were developed to investigate the hypothesis and assess the educational capacity of the designed learning tool. • Did the students perform differently between stage 1 and 2 of two-stage IVE-embedded assessments? • Did the male and female students perform differently in the two-stage IVE-embedded assessments? • Did the designed learning tool improve student’s knowledge of bushfire safety? • Did the designed learning tool improve student’s self-efficacy belief related to their ability to plan for bushfire safety? • Is there a relationship between students’ performance in the two-stage IVE-embedded assessment and their prior knowledge of bushfire safety? • Is there a relationship between students’ performance in the two-stage IVE-embedded assessment and their prior self-efficacy belief in relation to bushfire safety planning?
Methodology
Design of the Learning Tool
It was considered important to make students aware of the best approach to protect themselves in response to an imminent bushfire threat where they do not have the option of leaving as Towers (2015) noted that elementary school students lacked knowledge about bushfire safety. He argued that students are not aware of the criteria for appropriate sheltering and do not differentiate between a house fire and bushfire. To achieve this, we recognised students first needed to understand some basic information about how bushfires affect buildings. We also wanted to provide students with the skills needed to critically analyse a situation and create logical solutions so they could understand the value in forward planning. As such, four key learning objectives formed the basis of the VR educational tool design: 1. The students understand the mechanisms through which bushfire affects buildings. 2. The students can apply the acquired knowledge to identify a safe sheltering location from bushfire. 3. The students can demonstrate ability in analysing options for flexible bushfire planning. 4. The students can design a fire-wise property layout.
Students’ learning outcomes were measured through (i) their performance in two-stage IVE-embedded assessments; (ii) their responses to knowledge and self-efficacy beliefs in relation to bushfire safety in a post VR survey compared to a survey administered before the experience; and (iii) their responses to the IVE-embedded survey at the end of the learning experience.
The study used three phases of PBL and assessments to engage students in an innovative virtual environment. Our tool was based on individual learning but was based on the design template for 3D problem-based collaborative virtual learning environment developed by Russell (2016). Figure 1 shows the three phases of the learning tool designed based on the template of the virtual PBL model. The three distinct phases of the study are as follows: Three phases of the learning tool designed based on the virtual PBL model template (Russell, 2016).
Phase 1 built knowledge about fire behaviour during bushfires. The informatory scenes presented (1) the mechanisms through which bushfire affects buildings (ember attack and radiant heat), (2) how their adverse effect can be reduced in real-life situations and (3) the value of appropriate landscape design in providing safety from bushfires. All contributes to children’s understanding of the importance of bushfire behaviour knowing that fire behaves differently in bushfires comparing to house fires. Benchmark activities aimed to empower children with fire safety knowledge; during ‘fire triangle’ activity, children learned by experimenting the role of three key elements (oxygen, fuel and heat) that are the basis of any fire. Children also were involved in an activity to locate the items in the backyard which might catch fire during ember attack in bushfires. These activities aimed to help children realise they can contribute to reducing fire risk by understanding the fundamental cause of fire and the ways to prevent ignition of flammable materials within the context of bushfire safety.
Phase 2 activities helped children to develop skills in the problem by applying their knowledge of bushfire protection and preparation from phase 1 in an imaginary story-based scenario ‘You and Bella’.
Phase 3 involved students in using their knowledge and skills from phase 1 and 2 to find a solution to the problem (how to be safer in a bushfire situation). The second stage embedded assessments required students to (1) transfer knowledge from previous stages into a more complex situation and (2) design a bushfire safer property (long-term solution to reducing bushfire risk).
Two principles of the anchored instructional theory are used in the design of the learning tool, the narrative and embedded data design principles. The narrative was used in the storyline of ‘You and Bella’, which was designed based on a plausible and relevant story. Children were told they would be looking after a friend’s dog named Bella, which created a meaningful context for problem-solving activities. The emotional connection many children experience with animals (Hawkins et al., 2017) was used to enhance a sense of maturity and responsibility. The story was also designed to arouse a sense of pride and accomplishment among children stemming from saving Bella’s life from bushfire, considering the fact that the emotional state of the learner can influence learning (Dengel & Magdefrau, 2020). The embedded data used at all three phases of the study provides all the necessary information, such as clues and feedback, which students require during the preparation activities and embedded assessments (sheltering and design activities). In order to successfully conduct the two-stage embedded assessment, users needed to apply both low and HOTS. In the first stage assessment, participants were involved in a story-based scenario and were required to complete activities which represented an application of low order thinking. In the second stage assessment, participants were required to analyse the information, to think critically and creatively to be able to successfully complete the activities, which included evaluating a safer place to shelter and creating a fire-wise property layout. These activities demonstrated the application of HOTS. A simple system of adaptive feedback was also applied to construct the two-stage embedded assessment at IVE. The feedback system was specifically designed based on the learners’ responses, helping learners to identify their own misunderstanding and providing immediate feedback on learners’ performance which motivates students to continue exploring the virtual environment.
Creation of the Virtual Scenarios
Sequence of scenes and the purpose of each scene in the design of the virtual PBL tool of the study.
Note. HOTS: higher order thinking skills.
Development of the Virtual Environment
Unity 3D game engine was employed to build the VR environment. In the experiments, Oculus Rift Consumer Version 1 of VR HMDs was employed to display and immerse participants in the virtual environments.
Participants
Participants were recruited from two sources: Mawson Lakes School and a major public event ‘Science Alive’, both located in Adelaide, South Australia. Ethics approval was provided from both the SA Education Department and the University of South Australia’s research institution. The school was not located in a fire prone area, a deliberate decision while testing the potential of the tool, as we did not want to negatively impact children who may have experience with bushfires. All members of four year 7 classes were invited to participate. All students were advised of what was involved and asked specifically about bushfire experience. No students had bushfire experience previously. A total of 76 students from Mawson Lakes School participated in the exercise. An additional 66 children who were attending the ‘Science Alive’ event participated, giving a total of 142 participants (75 males, 59 females and 8 unclassified). All students were in the age range of 10–12 year old.
Survey Development
The data were collected using multiple instruments: a short pre- and postsurvey, questions within the two-stage embedded assessments within the VR experience and a VR embedded survey at the conclusion of the experience.
Pre- and PostSurveys
Pre- and postsurveys were used to record changes in the knowledge and self-efficacy beliefs of students in relation to bushfire protection. These surveys were deliberately designed to be short so as to allow the rotation of all students through the experience in a set lesson time and comprised four questions that included multiple choice as well as true and false responses. The questions were related to two key concepts: knowledge and self-efficacy. Additional control questions were also included to allow for a more meaningful comparison about the effect of the IVE learning environment. These control questions were based on information that was not presented in the IVE, so it was expected that response to these questions should not statistically change from pre- and postsurveys. Control questions were related to the nature of the help that children would expect to receive from emergency services or other sources when a bushfire occurs close to the place where they were staying. A question about knowledge was designed based on the suggestion from Towers (2015) who stated that children lacked knowledge of sheltering strategies and elements of fire behaviour in the context of bushfire and focused primarily on behaviour if a bushfire threat was imminent. Children selected one option among a series of multiple-choice options as follows: (a) ‘The best choice would be to stay in a safe place within the house, especially if the house is made of non-flammable materials such as bricks’; (b) ‘The best choice would be to jump in a swimming pool or dam if you have one, or get in the bath’ and (c) ‘When the fire comes very close, the best choice would be to go outside and wait for someone to help’. Those who selected the correct response (a) were considered knowledgeable about bushfire safety and the rest were considered lacking that knowledge.
Children’s self-efficacy was tested by questioning their confidence to make decisions which would help to protect them and others against bushfire threat. We did not use existing self-efficacy scales (Chen et al., 2001; Sherer et al., 1982) to measure self-efficacy beliefs of children before experiencing the learning tool for several reasons. From a pragmatic perspective, it was essential that we kept the time the students spent completing surveys to a minimum in order to allow all students to experience the IVE, which was the key feature of interest for both students and teachers. From a theoretical perspective we believe our decision to minimise self-efficacy questions was justified on the basis of three main reasons: (1) self-efficacy is task-specific, meaning that the measures need to be tailored to the requirements of the given task and applying instruments from other fields might not reflect self-efficacy belief for the specific task in hand (Webb-Williams, 2017); (2) the importance of planning to minimise risk associated with bushfires is not tangible for many children at this age range (10–12 year old) and (3) measuring self-efficacy associated with an intangible concept might result in over-reporting or under-reporting of the level of self-efficacy. Therefore, it was decided that the focus would be on a simple and tangible item, whether they felt they had the ability to make decisions that would help themselves, their pets and family to be safe from a bushfire. This was a simple true or false response with those selecting true considered to have a self-efficacy belief and those selecting false considered to lack a self-efficacy belief.
Two-Stage Embedded Assessments
Virtual reality facilitates and engages elements of higher order thinking in elementary education (Abdullah et al., 2021; Jonassen, 2000). The first stage of the two-stage IVE-embedded assessments was incorporated in the ‘You and Bella’ scenario and comprised two activities. During the first part of this stage, children were asked to do an activity by looking around a back veranda and finding six items that needed to be removed immediately to make the surrounding area safer. Figure 2 depicts VR scenes related to this gamification activity. An activity to identify six flammable items in the backyard followed by feedback.
Users received immediate feedback after finding each dangerous item via a voice-over that explained why the item was dangerous and needed to be removed. The second activity involved the children exploring four different areas of the property and analysing their potential as suitable places to shelter themselves and Bella – the dog – from a bushfire. At each location, the voice-over asks whether that location on the house plan is safe to shelter in, then they receive feedback based on their response to each question. To emphasise the contextual nature of a bushfire, more than one appropriate location is provided and feedback stresses why in different scenarios one area might be safer than another. Figure 3 depicts VR scenes related to the sheltering activity which was designed to assess students’ ability to apply knowledge as an element of a lower order thinking. An activity to identify a suitable place to shelter from bushfire followed by feedback.
The second stage included three multiple-choice questions followed by a design activity. According to Scully (2017), multiple-choice questions have the capacity to assess higher order thinking skills with the exception of creativity. One of the characteristics of the questions designed based on HOTS is that there might be more than one accepted solution for the task in hand (Lei, 2010). This feature was considered in designing the activities particularly in layout design exercise. The first question focuses on what participants thought was the main purpose of doing the task ‘You and Bella’. In the second question, a more complicated house plan than the ones presented in previous scenes was provided for participants to choose the safest location (Plan A) to shelter from a bushfire. To emphasise the importance of always having an alternative in a fire situation, the next question asked respondents about their Plan B for sheltering option. The final question of the assessment, a layout design activity, comprised two main parts. First, respondents were presented with six paired options related to a variety of choices that could be used to construct a fire-safer home and needed to choose the appropriate option in each pair. Options related to typical infrastructure they might find see in a backyard, such as outdoor furniture, fences, doors, windows and landscaping elements. In each case, the choice consisted of typical options, for example, the choice regarding outdoor furniture included wooden or metal tables. Then, participants were provided with the information about the prevailing wind direction and adjacent land use of a property and asked to apply their knowledge in the design of the layout of a fire-wise garden by choosing the most appropriate location for five features: a paved entertaining area, a swimming pool, a concrete driveway, a vegetable garden and stone paving. Higher order thinking was primarily evaluated using the design exercise; however, the other questions also required some degree of higher order thinking to successfully answer the question (e.g. deciding on plan A and B for the sheltering activity).
Embedded Survey
A survey at the end of the learning tool consisted of six Likert-scale questions ranging from 1 (strongly disagree) to 5 (strongly agree) that investigated students’ perceptions of their accomplishment within the learning environment, the value of the learning experience and the transferability of the acquired knowledge to a real-life situation.
Scoring Students’ Performance in the Two-Stage IVE Embedded Assessments
Students’ scores in the two stages were scored separately to allow for comparison between lower and higher order thinking skills. Students’ performance was assessed using a simple scoring system. The first stage assessment comprised two activities. In both activities, every correct response was given one point. The first activity in the second phase assessment, ‘selection of design materials’, was also a lower order thinking task, so this was also scored at one point per correct item. The sum of the scores for the first (maximum score = 6) and second activity (maximum score = 4), of the first stage assessment and ‘selection of design materials’ (maximum score of 6) of the second stage assessment considered students’ first stage assessment scores, meaning the total possible score for assessing lower order thinking skills was 16 points. The second stage of the assessment comprised four questions/activities, and the scoring followed the same simple scoring of the first stage assessment with the exception of the ‘layout design’ exercise where multiple alternative solutions were considered appropriate. The criteria for scoring elements of layout design was that every reasonable response was allocated one point but a location that would increase the chance of the house being affected by bushfire was given zero points. A position that was illogical from a practical perspective (e.g. a driveway that does not connect with the main road) was also awarded zero points. The maximum number of points a student could gain in ‘layout design’ activity was 5. The rest of the questions were given a score of 1 for each correct answer (maximum 3), giving a total possible score for the second stage assessment of 8. The scores were normalised by subtracting each score from the minimum score and dividing by the scores range between maximum and minimum score. Therefore, the normalised score values were between 0 and 1. The normalisation process allowed comparison of score values of the various assessments with each other.
Procedure
After consulting with the four year 7 teachers at a local elementary school, it was determined the VR experiment would be conducted in their STEM area, which opens on to four classrooms. This allowed for small groups of students to leave class and participate in the experiment and then return to class with minimal disruption. Students were divided into groups of four, and at any one time, four students were completing surveys and another four were in the VR headsets. A brief introduction was provided for classes so students were aware it was about fire and they understood there was no problem if they chose not to participate. No students chose not to participate. Knowledge, self-efficacy and control questions were administered twice: before experiencing the learning tool (pre-test) and immediately after the experience (post-test). After completing the paper-based presurvey, students were assisted in putting on the HMDs and provided some basic orientation before being immersed in the virtual environment of the designed tool. It took approximately 25 min on average for the users to complete the learning process. Users were able to choose the items by focussing their gaze on the item they wished to select. This triggered a blue circle to begin filling in to record the item as the user response. They were able to change their response any time before the circle completely filled (approximately 2 seconds). Appropriate feedback for both correct and incorrect responses was provided through an adaptive embedded feedback system. Paper-based post-test surveys, identical to the pre-test survey, were administered to the participants after they had completed their experience.
Analysis
Data from 131 participants were used for analysis since 11 among 142 missing most of the questions in pre–post IVE surveys.
Data were analysed using standard descriptive statistics, a chi-square test of independence, paired samples t-tests, independent samples t-tests and McNemar’s tests with the assistance of statistical software SPSS. Paired t-tests and McNemar’s tests were used to compare students’ pre- and post-test scores. A chi-square test of independence was employed to assess whether there is an association between categorical (nominal) variables. The independent samples t-tests were used to evaluate whether students’ average two-stage embedded assessment scores differ significantly across two groups. Descriptive statistics were used to assess Likert-scale items. Data from both stages of the IVE-embedded assessments, the knowledge and self-efficacy belief of bushfire safety from pre-test survey, as well as IVE-embedded survey results were used for the analysis. The two-stage embedded assessments and students’ response about their confidence in making a wise decision to protect from a bushfire were assessed for normality, and they were deemed to be reasonably normal, with the absolute value of skewness being less than .75. A mild kurtosis was observed for the second stage of the IVE-embedded assessment with the absolute value of 1.19. While this may be considered to violate strict rules of normality, it is within more liberal rules suggested by Tabachnick & Fidell ( 2013) who recommend an overall absolute kurtosis score of 1.5 or less for the normality.
Results
Analysing the students’ performance in the two-stage IVE-embedded assessments revealed that, as expected, students’ scores were significantly higher at stage 1 assessment (lower order thinking) than stage 2 assessment (higher order thinking) (t(130) = 10.111, p < .001) with a large effect size (Cohen’s d =1.20 95% CI [.96–1.44]) (Cohen, 1988). Mean value of the normalised scores for the stage 1 assessment were M = .81, SD = .17 and for the stage 2 assessment were M =.57, SD = .23.
Students’ performance in the two-stage IVE-embedded assessments were also examined on the basis of gender. Mean value of the normalised scores for male students were M = .81, SD = .15 for stage 1 and M = .59, SD = .22 for stage 2 and for female students were M = .79, SD = .19 for stage 1 and M = .55, SD = .23 for stage 2, respectively. The result of the independent t-test analysis revealed no significant difference between male and female students in performing both embedded assessment tasks (stage one: p = .512, stage two: p = .278).
The effectiveness of the designed learning tool in increasing knowledge or self-efficacy beliefs of bushfire safety was also evaluated. The hypothesis was that the designed learning tool improves student’s knowledge of bushfire safety and their belief that they personally could contribute positively to decisions that influence their safety in a bushfire incident. Pre- and postsurvey results for knowledge and self-efficacy were examined. A total of 35% of students reported they have knowledge about bushfire safety for pre-tests and a total of 87%, for post-tests, respectively. As for the self-efficacy, a total of 77% of students indicated that they are confident in their ability to contribute to decisions during a bushfire incident for pre-tests and a total of 90%, for post-tests, respectively. Since both knowledge and self-efficacy were dichotomous variables, a McNemar’s test was used to evaluate the differences between post-tests and pre-tests results. The analysis showed that after experiencing the learning tool, both self-efficacy and knowledge of the students improved significantly (self-efficacy: Chi-Square = X 2 (1, N = 118) = 17.47, p < .001, knowledge: Chi-Square = X 2 (1, N = 131) = 68.01, p < .001). As a result, a total of 91% of children who lacked the knowledge and 67% who lacked perceived self-efficacy before learning experience, improved their knowledge and perceived ability in responding to bushfire hazards after experiencing the learning tool.
Two control questions were also analysed to see if changes had occurred between pre- and postsurvey results. These questions were related to communication during a bushfire and evacuation procedures. Neither of these two elements were addressed in the learning tool, so it was expected there should be no change between the pre- and postresults. A McNemar’s test was used to evaluate the differences between post-tests and pre-tests results for these control questions. The analysis showed that changes to the responses after experiencing the learning environment were not statistically significant (p = .302 and p = .230). This supports that the significant changes to the knowledge and self-efficacy beliefs are due to the participation in the learning experience.
Furthermore, the authors examined a potential association between students’ performance in the two-stage IVE-embedded assessment and their prior knowledge and self-efficacy beliefs of bushfire safety.
Results of the student performance at stage 1 and 2 show that students’ learning achievement was not related to students’ prior knowledge of bushfire safety. Mean value and standard deviation for students with knowledge of bushfire safety was higher at stage 1 (M = .83, SD = .15) than stage 2 (M = .54, SD = .23). Students who lacked bushfire safety knowledge had a lower score (M = .79, SD = .17) for stage 1 compared to those with knowledge, but higher result in stage 2 (M = .58, SD = .22). However, results of the independent samples test indicated that there was no statistically significant difference between students’ stage 1 and 2 scores in the group who had prior knowledge of bushfire safety compared to the group who lacked such a knowledge (stage 1: p = .239, stage 2: p = .370). In other words, the students with prior knowledge of bushfire safety did not have a significantly better learning achievement than those lacking bushfire safety knowledge prior to the experience.
Results of the student performance at stage 1 and 2 show that students’ learning achievement was not related to student self-efficacy belief of their ability to plan for bushfire safety prior to the experience. Mean value and SD for students with self-efficacy belief were M = .81; SD = .16 for stage 1 and M = .58; SD = .24 for stage 2, respectively. The scores for students lacking a self-efficacy belief were M = .81; SD = .19 for stage 1 and M = .53; SD = .15 for stage 2, respectively. The results of the independent samples test indicated that there was no statistically significant difference between students’ stage 1 and 2 scores in a group who had prior self-efficacy belief related to their ability to plan for bushfire safety compared to the group of students who lacked such a belief (stage 1: p = .975, stage 2: p = .249). In other words, the students with a prior self-efficacy belief regarding their ability to make decisions that would contribute to their safety during a bushfire did not have a significantly better learning achievement than those lacking that self-efficacy belief.
In addition, a chi-square test was used to evaluate a potential association between students’ pre-knowledge and pre-self-efficacy beliefs as well as students’ post-knowledge and post-self-efficacy beliefs of bushfire safety. The result indicated that for both pre- and postsurveys, students’ self-efficacy beliefs were not related to their knowledge of bushfire safety (pre: p = .894, post: p = .749).
Students’ responses to IVE embedded survey in relation to the effectiveness of the learning tool (n = 131).
Note. IVE: immersive virtual environment.
The last question in the IVE-embedded survey deals with the self-efficacy of the participants and asked participants to rate on a 5-point Likert scale about their confidence in making a wise decision to protect from a bushfire. Results showed that over 80% of participants agreed or strongly agreed that they feel more confident that they would be able to calmly evaluate the options and make a wise decision to protect from a bushfire if it occurred in an area where they were staying. We examined whether students who have a self-efficacy belief related to their ability to plan for bushfire safety and the students who do not have such a belief before experiencing the learning tool responded to this question differently. The mean value and standard deviation for students with a self-efficacy belief was higher (M = 4.24, SD = .74) than students lacking a self-efficacy belief (M = 3.74, SD = .86). Results of the independent samples test indicated that there was a statistically significant difference between students who had a prior self-efficacy belief of their ability to plan for bushfire safety and the group of students who lacked such a belief in responding to the question (stage 1: t(116) = 2.989, p = .003) with a moderate to large effect size (Cohen’s d = .65 95% CI [.22–1.09]) (Cohen, 1988). The results mean that students who had prior self-efficacy are more confident in their abilities in making bushfire safety decisions than students who lacked self-efficacy prior to the learning experience.
In summary, the results suggest that students’ perceived knowledge and self-efficacy increased by virtue of the learning tool, regardless of their gender. Moreover, students’ perception of their accomplishment, learning experience and transferability of the acquired knowledge was positive.
Discussion
The effectiveness of a virtual PBL environment and its two-stage embedded assessments, as a learning tool, in educating bushfire safety to children (aged 10–12) was investigated. The results showed that students’ performance in a two-stage IVE-embedded assessment was not related to their prior knowledge or self-efficacy beliefs regarding bushfire protection and was not gender dependent. The results support the hypothesis that all the students regardless of their gender, prior knowledge and self-efficacy were effectively engaged and learned from this problem-based virtual immersive experience.
Chen et al. (2014) and Meluso et al. (2012) also showed student’ scientific inquiry self-efficacy increased regardless of their performance within the virtual environment. The results, however, were in contrast with the study of Bradley et al. (2015) which found that self-efficacy and performance were related during a virtual game–based assessment. Such a contradictory result may be explained by the nature of the IVE. First, the VR bushfire learning tool and its two-stage embedded assessment with immediate adaptive feedback, and multiple opportunities to achieve mastery strengthened students’ self-efficacy regardless of their performance during the VR embedded assessment. Secondly, the focus of the learning tool was to empower children regarding their ability to prepare and respond to bushfires (i.e. self-efficacy) rather than performance in the assessments.
The results were consistent with findings of Ronelus (2016) who demonstrated that immersive educational environments can help students develop knowledge and self-efficacy. Our results indicated that children’s knowledge significantly increased from pre- to postsurveys. This is consistent with previous literature which demonstrated that participating in IVE-based learning environments increase students’ knowledge (Mead et al., 2019; Metcalf et al., 2011, 2013; Moysey & Lazar, 2019; Ronelus, 2016). Similarly, children’s self-efficacy belief in their ability to plan for bushfire safety increased from pre- to postsurveys, which is consistent with findings from previous research (Chen et al., 2014, 2016; Meluso et al., 2012; Reilly et al., 2020). For instance, a study by Ronelus (2016) showed an increase in fifth grade students’ self-efficacy beliefs related to science after participating in IVE-based learning environments. Comparing the results from a self-efficacy question in the IVE-embedded survey to the pre- and postsurvey results revealed that students who had prior self-efficacy were more confident about evaluating the options and making a wise decision to protect themselves from a bushfire than students who lacked self-efficacy prior to the learning experience. This suggests that the manner in which the IVEs are embedded in other classroom activities are important because other activities schools do to build confidence in children’s ability will have a positive flow-on effect.
A significant increase in children’s self-efficacy belief in their ability to plan for bushfire safety means that most of the children (67%) who believed that they were too young to be able to make decisions to enhance their safety and the safety of others changed their mind after the learning experience. Considering the fact that children’s perceived experience with the learning tool was positive (as evidenced by embedded survey results), it can be inferred that the primary source for enhanced self-efficacy was perceived mastery of experience as explained by Bandura (1997). Since self-efficacy changes in response to different experiences (Webb-Williams, 2017), it can be concluded that participants’ self-efficacy beliefs increased based on their perceived success in their learning experience. However, students who had prior self-efficacy were more confident about their ability in making a wise decision to protect themselves from a bushfire than students who lacked self-efficacy belief prior to the learning experience. Since the performance of these two groups were not significantly different at the two-stage embedded assessment, then the difference in their perceived ability in making a wise decision to protect themselves from a bushfire might be related to other sources of self-efficacy beliefs rather than mastery of experience. Social persuasion as a source of self-efficacy (Bandura, 1997) might explain the difference between reported self-efficacy levels of the two groups after the learning experience. Social persuasion was used in the IVE to encourage students via adaptive feedback system within the IVE experience to make wise decisions and try again when they made mistakes (over 67% of students positively evaluated receiving feedback during the learning exercises). It is also likely that the primary self-efficacy of students was related to multiple sources (Webb-Williams, 2017) independent from the IVE, such as encouragement from their parents or local community to get involved in bushfire preparation and decision-making activities.
Self-efficacy of students increased regardless of their gender, and there were no statistical differences between male and female students’ self-efficacy before or after the learning intervention. Considering Webb-Williams (2017) found that the most important source of self-efficacy for female students in the age range of 10–12 years was the combination of vicarious experience and affective states, and it could be inferred that these two sources of self-efficacy affected students’ self-efficacy in the study. As discussed by Metcalf et al. (2014), taking roles within the virtual environment helps students to strengthen the belief that they have the ability to solve problems. In our study, children took the role of a landscape designer to design a fire safe property which likely helped them to increase their self-efficacy. Chen et al. (2016) contend that positive emotions during the VR experience enhance students’ self-efficacy. In our study, over 84% of participants felt proud that they saved Bella from a bushfire which acted as a positive emotion in influencing students’ affective states and consequently their self-efficacy. It can be inferred that the reason for the increase in self-efficacy of all students regardless of their gender or performance is related to all four sources of self-efficacy. The combination of VR embedded assessment with story-based role-playing, combined with adaptive feedback and positive emotions of pride, was able to promote self-efficacy among all students.
Watching peers is an example of a vicarious experience that can be effective in enhancing self-efficacy among children, especially females (Webb-Williams, 2017). Therefore, future educational programmes which aim at empowering children in terms of disaster resilience should facilitate other sources of self-efficacy beliefs (e.g. watching peers and social persuasion) rather than just mastery of experience. One example that could enhance self-efficacy is using a collaborative virtual environment in which students experience the learning environment in groups where they can learn from each other. This could work well in school settings where teachers might struggle to support individual students’ experience at IVE. Moreover, combining VR learning experiences with other pedagogies that more easily allow for observation of peers and social percussion could be another useful approach.
This study suggests that a focus on building student’s confidence in their ability to help out in a fire situation is needed. This might be best achieved by focussing on specific tasks, for example, in this study, knowing how to select an appropriate shelter or wetting and rolling towels to block smoke. An IVE could also help children to prioritise the tasks that need to be done. Building the capacity of children to help in a bushfire incident would not only assist adults in remaining calm and focused but would also provide some relief to adults during an incident as an occupied focussed child is more likely to remain calm.
Interestingly, unlike many studies that suggest fire is quite a gendered issue, with women reporting significantly less prepared (Molan & Weber, 2021), women more concerned about fires (Weber et al., 2019) and women predicting the next fire to be sooner than predictions of their male counterparts (Bardsley et al., 2018), this study found no significant difference between male and female students in their perceived ability in responding to bushfire hazards and performing embedded assessment tasks. This may suggest a need for further research into what intervening factors post-adolescence create such gendered differences in response to fire in adult populations.
Approaches to appropriately engage and involve children in disaster preparation and risk reduction activities are necessary given the expected number of catastrophic fire events is predicted to increase. Findings of the current study have important implications for the design of effective interventions about bushfire safety for children. It is particularly important for the programmes to empower children by allowing them to independently explore, make decisions and solve problems to be able to successfully transfer and apply the acquired knowledge into the real-life situation. This is not to suggest we want children to be over-confident in their ability to deal with a bushfire, but the reality is in most situations they will not be alone and a child that can think rationally and logically will be a great advantage to any family dealing with such a situation. If they are alone, knowledge and a belief that they can make sound decisions to make themselves safer would be invaluable. The results of current study make a contribution to the existing body of research analysing the effectiveness of IVE-based instructions for children’s education at elementary level.
Ethical Considerations, Limitations of the Study and Future Work
Ethical considerations that have been noted in previous immersive virtual learning environments include (1) a feeling of short-term physical discomfort, such as difficulty in balancing and differentiating distance, (2) psychological risk if children are immersed long term, (3) reduced cognitive and affective ability to adjust him/herself to the immersive experiences and (4) presence of false memory when children cannot differentiate between the virtual experience and reality during extended period of time (1 week) after experiencing the immersive environment (Southgate et al., 2019). None of these effects were reported in this study.
The key limitation of this work is related to two factors: the survey design and the lack of comparison. Although the pre- and postsurveys included two control questions, it is recommended that future studies use control groups which enable deeper examination of the related factors in the learning process. Additionally, self-efficacy, as explained in the methodology section, did not use existing scales to measure self-efficacy beliefs of children before experiencing the learning tool. While this would be much more time consuming for children to complete, given the results of this study, further attention to the construct of self-efficacy is warranted.
The time constraints in conducting the VR sessions with children at both formal and informal settings was a limiting factor. Experiencing the virtual learning tool and its two-stage embedded assessment took approximately 25 min of students’ time. Therefore, a very short pre- and postintervention questionnaires were administered which limited the opportunity to include more rigorous measures for knowledge and self-efficacy. Our desire to control as many variables as possible in the sample (e.g. same year level and same school) meant our potential sample was too low to enable desired comparisons. Ideally, we would have liked to have been able to compare the IVE participants to students who had engaged in an interactive South Australian Country Fire Service led programme with a fire-fighter and students who were involved in a more traditional classroom session. The desire of students and interest from the school in being involved in the IVE was also an important consideration.
Technology-based learning tools can enhance HOTS at the elementary school level (Abdullah et al., 2021). Our results showed that most of the children were good at content knowledge mastery as evidenced by children’s scores at stage 1 assessment but not particularly strong using HOTS as evidenced by children’s scores in the stage 2 assessment. This is consistent with the results of previous studies (Haleva et al., 2021). Time restrictions did not allow us to measure any changes to these skills in this study. However, if students’ performance at HOTS improves after the learning experience, then it would provide further evidence for the effectiveness of virtual PBL environments such as our learning tool. There are other factors which might have affected HOTS, such as individual learning styles that were not addressed in this study. Di et al. (2019) suggest a student’s learning style can strongly influence their HOTS, and as such, future research should consider the effect of other individual factors such as learning style in IVE-based learning environments.
Examining the extent to which learning is transferred from a virtual learning environment to the real-world situation helps to more accurately assess the effectiveness of a learning tool (Falloon, 2020). The majority of participants in our study believed that they applied their knowledge to solve a problem similar to real-life situation, and they felt more confident about their abilities in responding to bushfire hazards. However, due to time restrictions and the fact that we were not using a population in a bushfire risk zone, the transferability of the knowledge beyond the simulated environment of the learning tool was not examined. Follow-up research should track children over a longer period of time to examine whether children discussed the implementation of preparation activities with their parents or others at the household level.
Conclusion
The present study drew from the existing literature on bushfire education for children and took a novel approach to train children about bushfire safety using an IVE PBL tool. The significance of this study is twofold. First, the findings of the study revealed promising results of VR-based instruction to effectively engage children in the context of bushfire safety regardless of their prior knowledge, self-efficacy beliefs or gender. Second, the experiential and PBL activities during the VR experience combined with digital immersive storytelling was found to be effective in increasing children’s knowledge and developing their self-efficacy beliefs within the context of bushfire protection.
The capacity of children as potential contributors to bushfire safety at a household level should be valued. Applying innovative and effective learning tools which focus on increasing perceptions of self-efficacy, such as our designed learning tool, provide children with knowledge and skills they can transfer to real-life situations in which successful preparation and protective response to bushfire can save lives. Education programmes focussing on increasing children’s perceptions of their abilities to respond to bushfire hazards contribute to the preparedness decisions and actions at a household level. The importance of this cannot be overstated.
Footnotes
Acknowledgements
The authors would like to acknowledge SA Department for Education, Mawson Lakes School and Mr Steve Hill.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
