Virtual Reality simulation and Video Lecture are Equally Effective in Abscess Incision Training: A Multicentre Randomised Controlled Trial

Abstract

Introduction

The aim of the study was to evaluate undergraduate medical students’ knowledge and self-confidence to perform abscess incision after virtual reality-simulation or video lecture. Pre and post-knowledge assessment was conducted.

Materials and Methods

This was a multicentre randomised controlled trial from October 2021 to January 2022. The study was carried out at two universities providing medical education. Undergraduate medical students were randomized into either fully immersive head mounted display virtual reality-simulation (VR) or video lecture. Pre- and post-learning surveys probed self-confidence and student’s own assessment of competence. Participants were third-, fourth- and fifth-year medical students from the University of Eastern Finland and the University of Helsinki. All students were sent an email informing them of the opportunity to participate in this voluntary study. All willing students were included in the study.

Results

42 medical students participated in the study. Most were fourth year medical students, 52% of the participants had previously used VR in some context and the mean age of participants was around 25 years. Both methods increased knowledge and self-confidence equally. In VR subgroup, those with technical support available experienced greater learning than those without.

Conclusions

Interestingly, in the present study, both VR and video lecture increased knowledge and professional self-confidence equally.

Keywords

virtual reality video head mounted display randomized controlled trial simulation gamification

Introduction

Virtual reality (VR) can potentially improve efficiency of medical education (Barteit et al., 2021). This technology offers a patient-safe way to train healthcare students and professionals, especially in surgical procedures. Virtual reality is not yet in routine use, despite the first medical education VR publications dating back to the 1990s (Satava, 1993; Noar, 1991). Currently, immersive VR includes vision, hearing, and haptics sensations, thus giving a real-life impression. Compared with more traditional learning /teaching methods, VR learning was effective with immersive VR compared to video (Lohre et al., 2020). Students experienced more joy of learning in VR, which can also explain the results (Lohre et al., 2020).

The differences between technologies in medical education have been studied limitely. According to Raleigh, different teaching methods should be compared (Raleigh et al., 2018). Integrating virtual reality to medical education also requires further research.

The aim of this study was to compare medical students’ knowledge and self-confidence after fully immersive head mounted display (HMD) VR-simulation compared to traditional video lectures. There is a paucity in the data regarding HMD VR in medical education (Barteit et al., 2021), (Lohre et al., 2020), Ojala et al., 2022, Barré et al., 2019, Tursø-Finnich et al., 2022^. In this study, we focused on fully immersive HMD VR. To the best of our knowledge, there are no studies comparing the learning outcomes between HMD based VR-simulation and video teaching in undergraduate medical student education. We hypothesised based on previous publications, that highly interactive and fully immersive VR-simulation provides better learning outcomes than video lecture (Miller, 1990).

Materials and Methods

This study was a multicentre randomised controlled trial. The participants of this study consisted of third-, fourth- and fifth-year medical students from the University of Eastern Finland and the University of Helsinki. All students were sent an e-mail informing them of the opportunity to participate in this voluntary study. In the e-mail, students were told that the research topic was related to surgery and teaching technology. All willing students were included in the study; however, individuals with epilepsy were advised not to participate. The study was conducted from October 2021 to January 2022.

All students gave informed consent. The topic of the study was not given to the students in advance. They booked 60-minute slots in the booking calendar for the study. Upon arrival at the research site, the students took one envelope from the research box, which randomised the participants into either the VR-simulation group or the video group. After this, the students read the instructions for participating in the study from the research computer. At this point, students were informed that the study was about the incision of subcutaneous abscess in the knee area. The students completed the pre-questionnaire form, after which they completed the exercise one time (VR-simulation or Video), followed by the post-activity questionnaire.

The questionnaire was based on our previous study, where we explored the teacher's impact in VR-education (Raleigh et al., 2018). We developed a questionnaire measuring student’s self-confidence and a survey on the student’s own assessment of competence; these tasks were rated on a scale from 0 to 10 (0 = worst, 10 = best). In these queries, each numerical answer option included a sentence describing competence or self-confidence in the answer option. The students made assessments before and after the exercise; the same form also had multiple-choice task (in what order what is done, score 0-5) to map the knowledge before and after the exercise. So, the 3 categories of outcomes that was looked were; a) self-confidence in performing, b) self-assessment of competence and c) Test of knowledge of steps on incision and drainage.

The VR-simulation software was manufactured by VR4Healthcare Ltd. Oculus Rift S was used as the VR glasses. The software was in English, which is not the native language of Finns. The VR group had an initial introduction to the use of the device; it took 20 minutes to play the introduction in the virtual world. In the VR-simulation of the abcess incision, the students wore the VR glasses on their face, after which they were able to move freely in the virtual world. The students also heard the space sounds and received habitable feedback when they touched anything. In the VR-simulation, steps can be taken in the order participants want, but there is also the possibility of doing incorrect things. The software did not advise during the VR-simulation.

The video lecture of the abcess incision was created by the authors in Finnish and lasted for about two minutes. After the exercise, the students completed a questionnaire. Some of the students had technical support available. Technical support meant that the student knew the person to ask for technical assistance, if necessary, who was in the same room or apartment.

The collected data included whether the student learned something new, whether VR adds value to medical education, or whether VR was useful when incising an abscess. These tasks were rated on a scale from 0 to 10 (0 = worst, 10 = best). Statistical analysis was performed using IBM SPSS Statistics 25.0 software. We used Fisher’s exact test to compare nominal data and the Mann-Whitney U-test for nonparametric data. Mann-Whitney U-test was also used when comparing nonparametric data changes between the two groups. Results were considered statistically significant at p-values < 0.05.

According to the 488/1999 Finnish Medical Act (amendment 295/2004, 794/2010), research related to the development of teaching quality does not require permission from the research ethics committee.

Results

In total, 42 medical students participated in the study. Two participants from the VR group did not complete the study, one due to nausea and the other due to technical problems. The final study group therefore included 40 participants: 23 in the VR group and 17 in the video group. The groups were comparable in their basic characteristics (Table 1). In short, the majority of the participants were fourth-year medical students, 52% of the participants had previously used VR in some context and the mean age of participants were 25 years. All students had previously used video education.

Table 1.

Baseline characteristics of the study participants.

	VR (N = 23)	Video (N = 17)	p-value
Age, mean ± SD, years	25.8 ± 5.0	25.1 ± 3.1	0.678
Sex, female (%)	14 (61)	9 (53)	0.749
Right-handed (%)	22 (96)	16 (94)	1.000
4th year medical student (%)	18 (78)	13 (76)	0.833
Have used VR before (%)	12 (52)	9 (53)	1.000
Have used videos in learning before (%)	23 (100)	17 (100)	1.000

Comparing the pre and post questionnaire on knowledge and self-confidence, both groups showed an increase from baseline values (Table 2). The learning value was slightly greater in the video group as participants evaluated that they learned something new (0-10) 4.8 ± 2.7 in the VR group and 5.1 ± 2.5 in the video group, p = 0.725 and evaluated that exercise added value to the teaching (0-10) as 6.0 ± 3.2 in the VR group and 6.6 ± 1.9 in the video group, p = 0.808 (Table 2). Students’ assessment of their own knowledge (0-10) before the test 5.4 ± 1.4 (VR) and 4.9 ± 2.8 (Video) was greater than their professional self-confidence to perform the procedure, 4.3 ± 1.7 (VR) and 4.1 ± 2.6 (Video) (Table 2). After the study, knowledge ((0-10) was also assessed to be increased more 7.2 ± 2.2 (VR) and 8.3 ± 1.5 (Video) than professional self-confidence (0-10) 6.3 ± 2.1 (VR) and 6.9 ± 1.5 (Video) (Table 2).

Table 2.

Comparison of pre and post exercise surveys.

	VR	Video	p-value
Preliminary survey
Before exercise I know how to incise an abscess (0-10), mean ± SD	5.4 ± 1.4	4.9 ± 2.8	0.948
Before exercise I dare to open an abscess (0-10), mean ± SD	4.3 ± 1.7	4.1 ± 2.6	0.794
Pre exercise level test score, mean ± SD (0-5)	3.7 ± 2.0	3.8 ± 1.6	0.764
Learning outcomes
After exercise I know how to incise an abscess (0-10), mean ± SD	7.2 ± 2.2	8.3 ± 1.5	0.120
After exercise I dare to open an abscess (0-10), mean ± SD	6.3 ± 2.1	6.9 ± 1.5	0.401
Learning value
Learned something new (0-10), mean ± SD	4.8 ± 2.7	5.1 ± 2.5	0.725
Added value to teaching (0-10), mean ± SD	6.0 ± 3.2	6.6 ± 1.9	0.808
Post exercise level test score, mean ± SD (0-5)	4.6 ± 1.2	4.7 ± 0.8	0.698
Change from baseline scores
I know how to incise an abscess, mean change	+ 1.8 ± 2.1	+ 3.4 ± 2.6	0.071
I dare to open an abscess, mean change	+ 2.0 ± 2.3	+ 2.9 ± 2.3	0.242
Knowledge Level test score, mean change	+ 0.9 ± 2.0	+ 0.8 ± 1.6	1.000

Virtual reality and video training were physically, mentally and educationally suitable for the majority (83%-100%) of the participants (Table 3). Technical usability (0-10) 4.9 ± 2.3 (VR), 9.2 ± 1.3 (Video), p < 0.001 and usefulness (0-10) 6.8 ± 2.4 (VR), 7.5 ± 1.8 (Video), p = 0.432 of the exercises were better in the video group (Table 3). Technical support was available to 65% of participants in both groups (Table 3).

Table 3.

Technical evaluation and the suitability of the teaching methods.

	VR	Video	p-value
Technical usability
The exercise fit educationally, yes (%)	19 (83)	17 (100)	0.123
The exercise fit physically, yes (%)	21 (91)	15 (88)	1.000
The exercise fit mentally, yes (%)	21 (91)	17 (100)	0.499
Technical support was available, yes (%)	15 (65)	11 (65)	1.000
Usefulness of exercise, mean ± SD	6.8 ± 2.4	7.5 ± 1.8	0.432
Usability of the exercise, mean ± SD	4.9 ± 2.3	9.2 ± 1.3	< 0.001

For subgroup analysis in the VR group, those with technical support had greater learning than those without. In this VR subgroup, the knowledge level test change was also greater (1.3 ± 2.1) than in the video group 0.8 ± 1.6 (Table 2 & 4). When technical support was available, students in the VR group also evaluated (0-10) that VR would add more value 6.5 ± 3.3 (technical support) compared to those without technical support 5.0 ± 3.0, p = 0.182 (Table 4).

Table 4.

Learning outcomes of VR subgroups: with and without technical support.

	With technical support (n = 15)	Without technical support (n = 8)	p-value
After exercise I know how to incise an abscess, mean ± SD	7.7 ± 1.8	6.3 ± 2.8	0.201
After exercise I dare to open an abscess, mean ± SD	6.5 ± 2.0	6.0 ± 2.4	0.646
Learned something new, mean ± SD	5.3 ± 2.7	3.9 ± 2.5	0.252
Added value to teaching, mean ± SD	6.5 ± 3.3	5.0 ± 3.0	0.182
Post exercise level test score, mean ± SD (0-5)	4.7 ± 1.3	4.5 ± 0.9	0.324
Change from baseline scores
I know how to incise an abscess, mean change	+ 2.4 ± 2.0	+ 0.6 ± 1.8	0.071
I dare to open an abscess, mean change	+ 2.3 ± 2.1	+ 1.4 ± 2.7	0.265
Knowledge Level test score, mean change	+ 1.3 ± 2.1	+ 0.3 ± 1.7	0.265

Discussion

Herein, we compared video lecture to fully immersive HMD VR-simulations for abscess incision. We did not compare or evaluate improvements of surgical skills. We found that both methods increased undergraduate medical students’ knowledge and self-confidence in performance equally, contrary to previous publications (Kyaw et al., 2019; Yoganathan et al., 2018). Also according to Miller´s pyramid, more active VR-simulation should have been a more effective teaching method (Miller, 1990). The difference between the outcomes found in our study may be due to the lack of technical support in the VR group. Only a half of the participants had used VR before, this affected our results. It should be noted that when undergraduate student’s learning is evaluated using new technology, the student’s initial knowledge level may be too low, and thus the technology does not add value. Inexperienced students do not benefit from exercises that are too complex (Dankbaar et al., 2016). Our pre-level test showed that students had sufficient knowledge of the subject. Virtual reality education is not suitable for everyone, due to the side effects of dizziness/nausea that were recorded also in this current study (Taylor et al., 2022; Birrenbach et al., 2021).

The increase in knowledge in the VR group was slightly better, but only when technical support was available. Students requested also longer time for VR simulation, mainly to master the VR technological competence. The need for teacher or technical support is evident when VR technology is new, as only about half of the participants in both groups had previously used VR in either entertainment or teaching; this has also been stated in previous publications (Ojala et al., 2022). The teacher’s role is not merely a technical support, but with verbal guidance it is possible to provide additional information. The usability of the video was considered better, and students felt that the video lecture added more knowledge. Video teaching is routine for current medical students, and this was reflected in our results. The short duration of the video may lead to biased results, especially when participants only perform the exercise once. A short teaching session, such as the video in this study, can have a good immediate learning result.However, if the knowledge test were repeated a month later, the VR group may perform better since they have already practiced the procedure in VR. Based on the recent study, integrating virtual reality into medical education may require a technical advisor.

The haptics' weakness can affect the VR learning experience, especially in surgical procedures. The most important lesson in simple surgical HMD VR exercises is to learn the procedure. Simulators, on the other hand, primarily seek realistic haptics.

An abscess incision is one of the most common surgical procedures performed by a general practitioner. Pedagogically, training of abscess incision has been difficult, and traditionally, phantoms or cadavers have been used (Fitch et al., 2008). The drawback of phantoms is that the abscess drainage can only be done once or at least the cover must be replaced. Video is a good method for teaching abscess drainage (Fitch et al., 2008). However, the procedure itself cannot be done hands-on, which would allow for the information to be better retained. The advantages of VR include repetition of the procedure, so the learner knows how to act correctly in different situations.

Limitations and Suggestions for Further Future Research

The weakness of this study was the low number of participants. The initial email did not specify the topic, leading some students to skip the exercise. No validated survey was suitable for the study, which forced us to create our own. The limitation of the study is also the fact that practical learning (technical skills) learning in the VR had to be measured by the student’s experienced learning and knowledge, which does not correspond to the learning of technical skills. This can therefore affect the results of the study. It is possible that the students would have done the procedure in the real life before the study, but that was not asked in the survey. Objective structured assessment of technical skill (OSATS) did not fit because no surgical/technical skills were measured. In the future, a survey should be created that would be suitable for evaluating learning in simulations. Another drawback was the weak tactile feedback of technology, i.e., haptics. Madan showed that VR training without haptic feedback is not realistic; however, they did not have fully immersive HMD VR (Madan et al., 2005), the degree of fidelity or realism does not alter the effectiveness in training (Grober et al., 2004). The strengths of this study include multicentre RCT, which measured both pre and post scores and assessed teaching in a multidimensional way: self-confidence, knowledge, and usability.

Conclusions

In the present study, both VR and video education increased knowledge and professional self-confidence in abscess incision training. However, our hypothesis that VR is better as an active learning method was not confirmed. In the future, research into learning using new technologies should consider that the level of knowledge or skills has been measured or assessed in advance. In this way, studies are comparable when sources of error due to new technology and the student´s low knowledge level can be distinguished.

Supplemental Material

Supplemental Material - Virtual Reality simulation and Video Lecture are Equally Effective in Abscess Incision Training: A Multicentre Randomised Controlled Trial

Supplemental Material for Virtual Reality simulation and Video Lecture are Equally Effective in Abscess Incision Training: A Multicentre Randomised Controlled Trial by Henrik Nuutinen, Ville Bister, Virve Koljonen in Simulation & Gaming

Footnotes

Authors Contribution

All authors participated in the design and analysis of the study, all authors also wrote and revised the manuscript. All authors approved the manuscript.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Henrik Nuutinen, is shareholder of VR4Healthcare.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Henrik Nuutinen

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Henrik Nuutinen, MD, PhD is a clinical lecturer and interested in technology and university teaching. In addition working with skin cancers patients.

Ville Bister, MD, PhD is a clinical lecturer, interested in software development and university teaching. Also working in Helsinki University Hospital trauma unit.

Virve Koljonen, Professor of Plastic Surgery at University of Helsinki, enthusiastic about teaching medical students.

References

Barteit

Lanfermann

Bärnighausen

Neuhann

Beiersmann

. Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games. 2021 Jul 8;9(3):e29080. https://doi.org/10.2196/29080

Barré

Michelet

Truchot

Jolivet

Recanzone

Stiti

Tesnière

Pourcher

. Virtual Reality Single-Port Sleeve Gastrectomy Training Decreases Physical and Mental Workload in Novice Surgeons: An Exploratory Study. Obes Surg. 2019 Apr;29(4):1309-1316. https://doi.org/10.1007/s11695-018-03680-9

Birrenbach

Zbinden

Papagiannakis

Exadaktylos

Müller

Hautz

Sauter

. Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial. JMIR Serious Games. 2021 Oct 8;9(4):e29586. https://doi.org/10.2196/29586

Dankbaar

Alsma

Jansen

van Merrienboer

van Saase

Schuit

. An experimental study on the effects of a simulation game on students' clinical cognitive skills and motivation. Adv Health Sci Educ Theory Pract. 2016 Aug;21(3):505-521. https://doi.org/10.1007/s10459-015-9641-x

Fitch

Manthey

McGinnis

Nicks

Pariyadath

. A skin abscess model for teaching incision and drainage procedures. BMC Med Educ. 2008 Jul 3;8:38. https://doi.org/10.1186/1472-6920-8-38

Grober

Hamstra

Wanzel

Reznick

Matsumoto

Sidhu

Jarvi

. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg. 2004 Aug;240(2):374-381. https://doi.org/10.1097/01.sla.0000133346.07434.30

Kyaw

Saxena

Posadzki

Vseteckova

Nikolaou

George

Divakar

Masiello

Kononowicz

Zary

, et al. Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019 Jan 22;21(1):e12959. https://doi.org/10.2196/12959

Lohre

Bois

Pollock

Lapner

McIlquham

Athwal

Goel

. Effectiveness of Immersive Virtual Reality on Orthopedic Surgical Skills and Knowledge Acquisition Among Senior Surgical Residents: A Randomized Clinical Trial. JAMA Netw Open. 2020 Dec 1;3(12):e2031217. https://doi.org/10.1001/jamanetworkopen.2020.31217

Miller

. The assessment of clinical skills/competence/performance. Acad Med. 1990 Sep;65(9 Suppl):S63-S67. https://doi.org/10.1097/00001888-199009000-00045

10.

Madan

Frantzides

Tebbit

Quiros

. Participants' opinions of laparoscopic training devices after a basic laparoscopic training course. Am J Surg. 2005 Jun;189(6):758-761. https://doi.org/10.1016/j.amjsurg.2005.03.022

11.

Noar

. Endoscopy simulation: a brave new world? Endoscopy. 1991 May;23(3):147-149. https://doi.org/10.1055/s-2007-1010643

12.

Ojala

Sirola

Nykopp

Kröger

Nuutinen

. The impact of teacher's presence on learning basic surgical tasks with virtual reality headset among medical students. Med Educ Online. 2022 Dec;27(1):2050345. https://doi.org/10.1080/10872981.2022.205034

13.

Raleigh

Wilson

Moss

Reineke-Piper

Walden

Fisher

Williams

Alexander

Niceler

Viera

Zakrajsek

. Same Content, Different Methods: Comparing Lecture, Engaged Classroom, and Simulation. Fam Med. 2018 Feb;50(2):100-105. https://doi.org/10.22454/FamMed.2018.222922

14.

Satava

. Virtual reality surgical simulator. The first steps. Surg Endosc. 1993 May-Jun;7(3):203-205. https://doi.org/10.1007/BF00594110

15.

Taylor

Dyer

Al-Azzawi

Smith

Nzeako

Shah

. Extended reality anatomy undergraduate teaching: A literature review on an alternative method of learning. Ann Anat. 2022 Jan;239:151817. https://doi.org/10.1016/j.aanat.2021.151817

16.

Tursø-Finnich

Jensen

Konge

Thinggaard

. Virtual Reality Head-Mounted Displays in Medical Education-A Systematic Review. Simul Healthc. 2022 Feb 8;6:11. https://doi.org/10.1097/SIH.0000000000000636

17.

Yoganathan

Finch

Parkin

Pollard

. 360° virtual reality video for the acquisition of knot tying skills: A randomised controlled trial. Int J Surg. 2018 Jun;54(Pt A):24-27. https://doi.org/10.1016/j.ijsu.2018.04.002

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB