Educational Effects of Training With a Virtual Reality-Based Objective Structured Clinical Examination Software

Introduction

Virtual reality (VR) is a technology that creates simulated environments resembling the real world and is widely applied across various fields and industries. ¹ The effectiveness of VR as an educational and training tool is well recognized, as it enhances learning by enabling users to become fully immersed in an interactive, virtual environment that closely mirrors the real world. ² Furthermore, VR facilitates interaction within these virtual settings through specialized headsets, controllers, and hands-on experiential learning. Accordingly, VR is considered highly effective in medical education.^3,4 It is used for anatomy education, practicing surgical procedures, and training in cardiopulmonary resuscitation and advanced life support.^5-8

Hiroshima University has contributed to these advancements by developing an interventional radiology (IR) simulator that employs a VR system (the VR-IR simulator). ⁹ The VR-IR simulator serves as a platform for training students to perform IR procedures within virtual environments. Mitani et al ⁹ reported that the VR-IR simulator has been used to train medical students in endovascular procedures.

We, along with BeRISE Corp, recently co-developed a virtual reality-based objective structured clinical examination practice software (VR OSCE^TM) (Figure 1A), ¹⁰ which is designed as an educational tool to facilitate social distancing. VR OSCE^TM enables medical students to practice both medical interviews and physical examinations. Tasks include inspection, auscultation of cardiac and pulmonary sounds, and neurological examinations using a percussion device (Figure 1B and C). This software allows learners to select interview items based on the patient’s age, chief complaints, and medical history. Using VR goggles (Pico Neo3), students can engage in a highly realistic and safe virtual environment^11,12 to improve their clinical skills through repeated practice. VR OSCE^TM covers 20 medical scenarios selected according to the foundational diseases outlined in the Model Core Curriculum for Medical Education, as established by the Japan Society for Medical Education. ¹³ This curriculum ensures that medical students acquire the essential knowledge and skills required for medical practice in Japan.OPEN IN VIEWER

In this study, we assess the effectiveness of VR OSCE^TM for the clinical training of medical students. By comparing student skill evaluations before and after VR OSCE^TM training, this study aims to evaluate the impact of VR-based learning on clinical skills development.

Materials and Methods

This prospective, pre- and post-intervention study was designed to evaluate the effectiveness of VR OSCE^TM and was conducted and reported in accordance with the STROBE statement (Supplementary Material 1). ¹⁴ A priori power analysis was conducted using G*Power statistical packages (G*Power Ver. 3.1.9.2, Universität Düsseldorf, Düsseldorf, Germany) ¹⁵ for a paired t-test (two-tailed). With an effect size of d = 0.8, an alpha level of 0.05, and a power of 0.80, the minimum required sample size was determined to be 15 participants. ¹⁶ Medical students from Hiroshima university who had passed the pre-OSCE in their fourth year and provided written informed consent were eligible for inclusion, while those who failed were excluded. Consequently, twenty fourth- and fifth-year medical students at Hiroshima University were randomly selected and enrolled. Participants received an advanced briefing, and practical training was conducted at the university’s training center. The study spanned the period from April to September 2023. Ethics approval for this study was obtained from Hiroshima University (approval E2023-0030).

The study was structured as follows (Figure 2). VR OSCE^TM covers 20 medical scenarios categorized into three difficulty levels (A: easy, B: normal, C: hard). First, each medical student randomly selected a scenario, limited to the easiest difficulty level (Level A). Scenarios were randomized using a shuffled list of Level A scenarios. Using VR OSCE^TM, students conducted medical interviews and performed physical examinations on simulated patients. Students then presented their findings regarding the patients, including differential diagnoses. Through an external monitor, two instructors evaluated their performance based on five assessment criteria using standard OSCE evaluation protocols. Second, a third instructor, distinct from the previous two instructors, provided training using VR OSCE^TM. During this phase, students engaged in two scenarios at any difficulty level and received feedback from the instructor. Third, students chose another easy scenario that they did not previously complete. They conducted medical interviews once again, performed physical examinations, and delivered presentations. The same two instructors scored this session using the same criteria as those in the initial assessment.OPEN IN VIEWER

The figure outlines the process in which students conducted medical interviews and physical examinations of simulated patients using VR OSCE^TM, followed by presentations. Performance was assessed by two instructors via an external monitor (first assessment). After undergoing training on two additional scenarios with a different instructor, students completed another scenario and their performance was re-evaluated by the same instructors (second assessment) as those in the initial assessment.

The following five evaluation criteria were used: (a) safety checks, including patient verification and hygiene; (b) patient care such as greeting and conversation; (c) medical interviews and physical examinations focused on the primary complaint; (d) presentation skills, including differential diagnosis and diagnostic reasoning; and (e) overall performance, capturing general impressions (Figure 3).OPEN IN VIEWER

Student performance was assessed based on five evaluation items using serial OSCE assessment criteria.

The evaluation criteria were rated on a 10-point scale, with scores defined as follows: 3 points represent OSCE borderline performance, 6 points represent the OSCE passing threshold, 8 points reflect the proficiency expected at graduation, and 10 points correspond to internship-level competency.

After the second VR session, all 20 medical students were surveyed to evaluate the perceived effectiveness of VR OSCE^TM. The survey included items rated on a 5-point Likert scale (Questions 1–9) and closed-ended yes/no questions (Questions 10–12) (see Supplementary Figure 1). Any adverse events related to the training were also recorded.

Statistical analysis was performed using R (version 3.2.3; R Foundation for Statistical Computing, Vienna, Austria). Scores before and after training were compared statistically. After confirming normality using the Shapiro–Wilk test, a paired t-test was used to assess differences in each criterion between pre- and post-training evaluations. Statistical significance was defined at p < 0.05 for all analyses.

Results

The results are presented in Figure 4. Scores for (a) safety checks, including patient verification and hygiene (Pre: 6.0 ± 1.9, Post: 8.6 ± 1.8) and (b) patient care such as greeting and conversation (Pre: 4.6 ± 1.8, Post: 7.8 ± 1.1) were close to the OSCE passing threshold prior to training. By contrast, scores for (c) medical interviews and physical examinations focused on the primary complaint (Pre: 4.1 ± 1.7, Post: 7.3 ± 1.2) and (d) presentation skills, including differential diagnosis and diagnostic reasoning (Pre: 3.5 ± 1.3, Post: 7.0 ± 1.6), showed significant improvement. Initially, these scores were below the OSCE borderline performance level but surpassed the passing threshold after training. Scores for (e) overall performance, capturing general impressions (Pre: 3.8 ± 1.3, Post: 7.2 ± 1.5), mirrored these improvements. The increase in post-training scores across all parameters was statistically significant (p < 0.01).OPEN IN VIEWER

In the survey evaluating VR OSCE^TM, all participants recommended its inclusion in OSCE^TM certification, and they expressed high satisfaction with its use in clinical clerkship (Table 1), because VR OSCE^TM allows trainees to experience realistic OSCE. Furthermore, no adverse events such as VR sickness or vomiting were reported by any of the 20 participants.

OPEN IN VIEWER

Table 1.

Survey Results of VR OSCE^TM Evaluation

Item	Mean ± SD	Range
Likert scale items (1–9)
1	4.5 ± 0.60	3–5
2	4.8 ± 0.41	4–5
3	4.5 ± 0.69	3–5
4	4.5 ± 0.60	3–5
5	3.9 ± 0.75	3-5
6	4.4 ± 0.81	3–5
7	4.6 ± 0.60	3–5
8	5.0 ± 0.22	4–5
9	5.0 ± 0.22	4–5
Closed-ended yes/no questions (10–12)
10	100 ± 0.0	1
11	90 ± 0.3	0–1
12	100 ± 0.0	1

Discussion

The Coronavirus Disease 2019 pandemic, first identified in December 2019, ¹⁷ significantly affected medical education, particularly clinical clerkships. To maintain social distancing, ¹⁸ opportunities for patient care and bedside teaching were restricted, thereby limiting medical students’ ability to develop essential OSCE examination skills. Tzeng et al ¹⁹ reported that the pandemic adversely affected medical students’ clinical training, resulting in diminished OSCE skills and scores. In Japan, medical students must pass a national OSCE following their clinical clerkship to obtain a medical license within a six-year curriculum that includes approximately two years of clinical participation. At our hospital, restrictions similar to those mentioned above reduced students’ opportunities to practice history taking, physical examinations, and medical communication.

As a response, simulation-based education in the medical field has become increasingly significant.^20,21 Hiroshima University developed VR OSCE^TM to address the aforementioned challenges while also minimizing infection risk. VR OSCE^TM offers a simulated environment in which students can practice medical interviews and physical examinations in a virtual consultation room, providing realistic, repeatable practice opportunities. This study is the first to examine the educational effects of training with VR OSCE^TM on medical students.

Although students participated in only two VR training sessions with feedback between pre- and post-evaluation, VR OSCE^TM demonstrated a substantial educational impact (Figure 4). Two main factors likely contributed to this result. The first factor is the immersive nature of VR, which provides a heightened sense of realism and engagement and is beneficial for education.^3,4 Although other virtual patient simulators such as Body Interact® have been reported for medical training, ²² VR OSCE^TM is unique as a VR-based system specifically designed for OSCE training. In fact, many students reported that VR OSCE™ is an effective training tool that allows them to experience real OSCE settings. One student noted, “The VR OSCE provided an environment that closely mirrored real OSCE conditions, including the clinical atmosphere, structured workflow, and sense of time pressure during patient encounters.”

The second factor relates to VR OSCE^TM’s game-like features. During VR training, patient responses are displayed at a normal conversational pace and diagnostic tools replicate real-life timing. Students must manage their time within the 15-minute limit, navigating the interview and examination under realistic time constraints. This gameplay element enhances motivation and aligns with the growing use of gamification as an effective learning strategy, while also promoting engagement and a constructive competitive environment. ²³

The primary feature of VR OSCE^TM is Excel-based scenario customization (Figure 1D). In contrast to conventional simulation software, which primarily offers preset cases, VR OSCE^TM enables learners to design and share structured clinical scenarios incorporating differential diagnoses and patient profiles. This customizability facilitates active learning, encouraging students to apply their knowledge and enhancing their educational experience.

This study has some limitations. First, VR OSCE^TM was developed as practice software aligned with OSCE components and timing constraints, and it currently does not include advanced diagnostic elements such as blood tests or imaging studies. Second, VR OSCE^TM lacks voice recognition and automated assessment capabilities, making instructor and student feedback essential to optimize learning. Future developments will aim to address these limitations and enhance the software’s functionality. Additionally, this study was limited to VR-based scenarios. We therefore plan a case-control study comparing students trained with VR OSCE^TM and those without such training, evaluating real OSCE performance using predefined outcome measures, including clinical performance scores, time efficiency, and communication skills.

Although the required sample size based on a priori power analysis was achieved, the small number of participants remains a limitation of this study. Furthermore, since this study was conducted at a single institution, the generalizability of our findings may be limited. The two instructors assessed the same participants during pre- and post-training, which may have introduced recall bias, as familiarity with participants’ prior performance could have influenced subsequent scoring. To address these limitations, future studies should include larger, multi-institutional cohorts to improve generalizability. The use of blinded or independent evaluators may help reduce recall bias, and incorporating objective performance metrics from VR systems could enhance the reliability of assessments.

Conclusions

This study comprehensively evaluated the effectiveness of training with VR OSCE^TM as a learning tool to enhance clinical skills. The results revealed that medical students who underwent VR OSCE^TM training demonstrated improved performance in medical interviews and physical examinations compared with that before training. Digital simulation–based education using VR has the potential to transform medical curricula by providing immersive, standardized, and repeatable training environments that enhance clinical skill acquisition, decision-making, and learner assessment.

Supplemental Material

Supplemental Material