Role of immersive technologies in healthcare education during the COVID-19 epidemic

Impact and adaptions COVID-19

The substantial spread of the Coronavirus strain COVID-19 was declared a pandemic by the World Health Organisation on 11th March 2020. ¹ Globally, behavioural changes were implemented to prevent its spread and were based on planned responses to pandemic situations, with restrictions having an undetermined timescale given the novel approach of the global response. In 2009, the Institute of Medicine developed Crisis Standard of Care (CSC) principles surrounding pre-emptive crisis arrangements. These were based on regional coordination, interdisciplinary collaboration, and specific approaches for the management of resource and personnel shortage. ² These responses to the outbreak were implemented alongside actions by Health Education England to stop planned rotations and face-to-face training courses. Elective procedures were postponed after recommendations by esteemed societies while emergency surgery was reserved only for cases which could not be treated conservatively, and in many were led by consultant surgeons rather than trainees. Laparoscopic surgery has been identified as an aerosol generating procedure and is therefore rarely undertaken during the COVID-19 crisis. These changes had a significant impact on surgical training.

Similarly, education has been suspended in schools, universities, and in hospital settings. The COVID-19 pandemic has presented unique challenges to all types of learning and training for all levels. It has stimulated discussion among educators to consider alternative innovative methods for rapid adaptation, and to ensure modifications to education and training are not forgotten following the COVID-19 crisis. Besides the need to replace the lost training opportunities during the COVID-19 pandemic, learning had to be delivered to upskill medical students and refresher courses had to be provided to retired healthcare professionals so that they may safely join the NHS workforce in a professional capacity.

To achieve such adaptions in education and training delivery during the pandemic, the utilisation of immersive technologies is essential. These technologies consist of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) and provide Omni-Learning. ³ Omni-learning is the ability to learn anywhere, anytime, with anyone. It can utilize communication and educational technologies to expand and expedite learning. A brief overview of the more effective and/or evidence based XR products for skills training is presented in this paper, and their immediate use and mass uptake is encouraged. They have direct benefits for schools, universities, and hospitals alike if implemented during the COVID-19 restrictions and in the future.

Technical skills training

A plethora of technical-related XR educational packages have been created that typically surround tools, anatomy, and surgical/medical procedures. The application of AR characteristically utilises holograms projected into the real-world environment. Additionally, holoportation ⁴ allows the transmission of volumetric three-dimensional

(3D) objects into a holographic representation which can be viewed by a remote holographic headset user. ⁵ AR glasses such as the Microsoft HoloLens can be used as they are portable and intuitive. ⁶ During and after the COVID-19 crisis high-fidelity holographic equipment may be accessible and practical. However, incorporation of the technology is limited by cost. For mass access to immersive technology, a cost-effective option for technical skills training is use of VR headsets. VR headset sales have been forecasted to exceed 36.7 million units in 2023. ⁷ Its diverse function and practicality can be implemented in healthcare to fulfil an array of educational requirements. Applications can be easily created, downloaded, and updated. Popular headsets are the HTC Vive or the low-cost option Oculus GO. ⁸

COVID-19 adaptations have reduced the experiences of surgical procedures for trainees. This can prevent improvements in patient data interpretation such as imaging, planning, and decision-making processes. However, several products have empirical support for their ability to increase students’ and surgeons’ understandings of such dynamics. Built specifically for home usage, emergent companies allow environment familiarization and include content mapped to the Accreditation Council for Graduate Medical Education. ⁹ , ¹⁰ A patient’s body parts can be presented in 3-Dimentions (3 D) before and after surgery. Such patient-specific 3 D models with the use of VR or MR can be shared with trainees to continue their exposure and improvement in clinical understanding. ¹¹ , ¹² For example, Parkhomenko et al compared VR headset usage to present computerised tomography (CT) scans relating to a complex endourological procedure of Percutaneous Nephrolithotomy. ¹³ Amongst other benefits the interactive VR data improved urologists' understanding of the renal anatomy. Medical Augmented Intelligence ¹⁴ and Holoeyes ¹⁵ have also created several AR/VR solutions which permit trainees to explore patients’ CT and magnetic resonance imaging (MRI) data. They can be combined with a medically accurate representation of the human body and create patient data to be shared within fifteen minutes. ¹⁴ During prolonged periods of limited access to surgeries, these applications allow educators to counter the loss of real-life exposure, and provide the potential for online assessment as a secondary backup should if typical assessment methods not be deemed feasible.

Being lower fidelity, therefore accessible to a larger number of users, augmented mobile applications should be more utilised. This includes smart phones capable of exploiting camera functionality to project digital content on a physical print out. For example, users can scan the relevant sheet of paper to produce a superimposed, detailed, 3 D anatomical image that is locked into a real-world spatial location. High-powered mobile devices support a recently created advanced 3 D anatomy platform ¹⁶ and enable this method to be used at home. Additionally, tools let tutors create content in PDF formats and share to students as interactive learning material by the same image recognition method. ¹⁷ This technique assists students to understand the material by adding such explanatory visual content. This strategy is designed to be a compensatory technique reinforcing self-study, and medical/surgical content can be added (Figure 1).

Non-technical skills and team training

Non-Technical Skills (NTS) training consists of situational awareness, decision-making, teamwork and communication, and leadership. NTS has had less integration with technology but can be trained remotely when immersive technologies are included. ¹⁸ However, the authors were not able to find an available AR application for home NTS training during the COVID-19 lockdown. Current and emerging VR applications have substantial support for their validity and include Human Factors-based methods for design and training delivery. ¹⁹ Designed for distance learning and cognitive skills acceleration a novel ‘NTS-VR’ training application based on previous research ²⁰ can modify understanding of important cues in VR scenarios (Figure 2). It has embedded feedback from experienced tutors and has 360-degree theory videos, with ambisonic sound. Such VR applications during a pandemic can allow engaging education to continue from home, and the transfer of skills into real-life has been suggested to be increased compared to paper/PC materials. ²¹ For communication skills, Oxford Medical Simulation has created a virtual patient with real-time adaptive responses that allows trainees to improve their consultation and prognostic skills. ²² The aforementioned VR applications provide simulations of a variety of healthcare scenarios and are downloadable with no restriction on practice times.

OPEN IN VIEWER

Figure 1.

Mobile based augmented projection of a 3 D asset to paper. Surgical tools, anatomy, and patient 3d models can be presented with high detail. Image from Human Anatomy Atlas. ¹⁷

OPEN IN VIEWER

Figure 2.

Virtual reality based Non-Technical Skills app that allows users to identify important cues relating to patient safety, and identify the underpinning factors. ²⁰

Many applications for NTS training are under development and require more empirical testing. There is caution in the usage of material which does not suggest validity in observed improvements towards the desired educational benefits. More evaluation may be required before mass usage, and the current user should understand limitations in efficacy. An example may be an increase in confidence that is not matched by an increase in knowledge or technical skills outcome as negative consequences in select settings may occur. ¹¹ , ²¹

Other educational delivery methods during and after COVID-19

For non-VR or AR training, a primary delivery method of educational content can be accessed via a programme named e-Learning for Home (e-LfH). ²³ In partnership with the NHS, Health Education England had created the e-LfH as a PC based tool that pools together all available online content. It has added more resources relating to the changes in practice due to COVID-19 impact and has aided users to receive continuous development remotely. Medtronic also has an online catalogue of material, albeit primarily oriented around surgical procedures. ²⁴ They have developed several PC and mobile based applications that allow remote training of a growing catalogue of interactive step-by-step 3 D procedures like transperineal MRI-targeted prostatic biopsy and catheterization.

As these applications can support mobile browsers, it is noteworthy that such companies may adapt their platforms to deliver both technical and NTS skills by adding mobile-based VR support and materials. Mobile applications can be delivered in an immersive format by adding it to a VR ‘headset frame’, such as the Google Cardboard. This variation of VR headset can cost less than $10 and the immersion is formed from the mobile device screen presented in a landscape and stereoscopic layout. There is typically no ability to interact with applications except eye gaze and head movements. Other PC-based methods of education for basic theory and knowledge can be fulfilled by video-assisted teaching. Indeed, video conferencing was one of the most important technologies for teaching in 2019. Multiple platforms are available to facilitate web conferencing and use features like whiteboards, screen sharing, discussion and voting. ²⁵ This has allowed video consultations for patients followed by assessment.

Virtual clinics, theatres and wards

It is possible for real-time training inside the operating room to continue during the COVID-19 crisis. Students do not have to be present in theatre to gain a surgical perspective if VR and AR are used. Using Mobile Augmented Reality, remote users can view a surgical procedure and can interact with the video feed. Students can draw shapes onto the video stream displayed which can be shown to all connected colleagues. This allows question and answer type teaching with all users sharing the same perspective, visual drawings, and audio communications. The role of VR in theatre started with 360-degree recordings with such cameras as the Insta360 Pro. However, as iterations of VR cameras improved live streaming was reliable, secure, and supported by educational councils. ²⁶ Giblib created an application to livestream VR and was available on mobile app stores in late 2019. VR 360-degree cameras can stream to a portable headset, mobile, and PC screens. For example, Cartucho et al verified that 3 D organ models, volumetric data, and tissue morphology can be presented in AR. ²⁷ With the interaction of the material, the participating surgeons suggested the benefits for intra-operative surgical guidance. This can be useful in education as the surgeon’s views can be shared live via Wi-Fi and ever-increasing wireless data transfer means, therefore students are able to have a first-person surgical perspective and understand how the imaging data can be useful for planning. Questions can also be asked in real-time which allows more students and trainees to attend procedures they may never have had access to in their conventional training due to the limited size and number of available operating theatres, and other logistical matters such as having to take consent from patients to be present.

There are many 360-degree videos of wards and recording of various practices. However, interactive ward rounds that include expert-level feedback in a constructive manner in VR are less common. A VR recording was created of a 5-day urology boot camp with use of actors and high-fidelity simulated wards. This allowed staff to observe and give feedback to students. ²⁸

In summary, an unknown time remains on the disruption from COVID-19 on the functioning and education of healthcare professionals. Both short- and long-term adaptations need to be implemented if they have not already. The aforementioned tools and technologies are viable short-term methods for a large portion of surgical and medical personnel (Table 1). However, if COVID-19 continues to restrict learning opportunities these may also be adapted for long-term, and likely a new and permanent manner of technology usage that is accelerated, trialled and implemented secondary to the current educational restrictions that is coupled with a bourgeoning demand for skilled clinicians. More awareness and incorporation of new technologies in education during a pandemic could reduce the effects on training from lockdown related consequences. Those with evidence that their technologies can improve technical and non-technical skills, immediate use may be the best option to reduce time training loss. Transferability of skills back into real setting could have higher levels for the XR applications designed to support this knowledge transfer (Figure 3).^29–35

OPEN IN VIEWER

Table 1.

Immersive technologies in healthcare education: COVID-19 and beyond.

Taught skill	Method	Advantage	Disadvantage	Example
Technical skills
Anatomy	AR headsets	• Realistic• Peripherals can be added for increased interaction• Pre-op preparation for rare conditions	• Cost• Less options available due to decreased funding and research	• Medical Augmented Intelligence • Holoeyes
	AR phone apps	• Utilise ever-improving phone CPU power• Lower fidelity• Accessible to more users	• Limited peripherals available	• Scanning a sheet of paper to produce a superimposed, 3D anatomical image locked into a real-world spatial location
	VR	• Lower cost• More research done	• Less interaction• Some users experience nausea• Less realistic than AR alternative	• Presenting CT scans relating to a complex endourological procedure of Percutaneous Nephrolithotomy
Medical/surgical  procedures	AR	• Realistic• Peripherals can be added for increased interaction	• Cost• Less options available due to decreased funding and research
	VR	• Lower cost• More research done	• Less interaction• Some users experience nausea• Less realistic than AR alternative
	PC and mobile-based applications	• Many cases available• Step-by-step 3D procedures		• Medtronic
Non-technical skills and team training
	VR	• Lower cost• Can modify understanding of important cues in VR scenarios• Simulation of difficult situations in a safe and supervised space• Embedded feedback from experienced tutors• 360-degree theory videos• Ambisonic sound	• Less interaction• Some users experience nausea• Less realistic than AR alternative	•  ‘NTS-VR’ training application
Communication	AR	• Realistic• Peripherals can be added for increased interaction	• Cost• Less options available due to decreased funding and research	• No examples found by authors
	VR	• Real-time adaptive responses• Allows for consultation and prognostic skills training• Downloadable with no restriction on practice times	• Less interaction• Less realistic than AR alternative• Limitations in efficacy remain	• Oxford Medical Simulation
Other educational delivery methods during and after COVID-19
General  educational  content	PC based tool	• Continuous updates and development remotely	• Less interaction• Some users experience nausea• Less realistic than AR alternative	• e-Learning for Home (e-LfH)
	Video conferencing (for medical education or clinician-patient)	• Whiteboard feature• Screen sharing• Discussion • Voting
Real-time training  from the Operating  Room (OR)	VR/AR	• Trainees are not confined to the OR space• Interaction with procedure is possible• First-person perspective possible• Surgeries can be recorded		• Cameras such as the Insta360 Pro for 360-degree recordings
	Headset, mobile, and PC screens			• Giblib

OPEN IN VIEWER

Figure 3.

Diagram showing suggested XR technologies and apps for a small portion of healthcare roles.