Resident Perceptions of Virtual Reality Versus Dry Lab Simulation for Advanced Shoulder Arthroscopy Resident Training

Introduction

Over 600 orthopaedic surgery residents graduate each year who are expected to be proficient in the operating room with procedural skills and techniques. ¹ Traditional orthopaedic surgical training relies on a mentor-apprentice model with increasing surgical experience gained as the resident becomes more efficient. This mentor-apprentice model is the standard training method at most institutions and appears to be preferred by residents.^2,3 However, it is inefficient in terms of time and cost.^4-6 Resident work restrictions, time and cost restraints, as well as an increased focus on patient safety have pushed the need for alternative methods of surgical training that meets the learning needs of the current generation of residents.^7,8

Knee and shoulder arthroscopies account for four of the top six most common procedures performed by applicants for part II of the American Board of Orthopaedic Surgery’s certification examination. ⁹ Arthroscopic training, as with any surgical training, requires repetition. Increasing complexity of arthroscopic procedures and long learning curves for arthroscopy add to the difficulty of depending solely the mentor-apprentice model alone.^10,11 With the recent healthcare shift towards patient safety and cost effectiveness, as well as with residents’ perception of inadequate arthroscopy training, use of simulator training offers an attractive supplement to standard surgical case repetition. ¹² Arthroscopy simulation has evolved to include cadaver models, plastic dry lab models, and computerized simulation. Both dry lab and virtual reality arthroscopic simulators have been validated in terms of improving surgical skills.^13-17

Additional studies have attempted to identify what training methods are best. However, many of these studies include only novice arthroscopists and compare no training to some structured simulation training.^18-23 Reports that pool data from several studies conclude that simulation training is beneficial, but are unable to define what training modality is most beneficial for more real-life procedures.^24-27 The utility of dry lab vs virtual reality simulation is important to differentiate due to the cost of maintaining virtual reality training platforms. Additionally, the utility of simulation training in general should focus on various learners’ needs as skill progress over residency. We aimed to examine how residents perceived virtual reality simulator training relative to a second training modality, the dry lab, for gaining skill required to complete a complex arthroscopic shoulder procedure. We hypothesized that both simulation methods would be well received but that the virtual reality simulator would be more preferred training modality compared to the dry lab.

Methods

This study was approved by the Institutional Review Board. Orthopaedic Surgery residents at a single training program were recruited to participate in this arthroscopic training study. Informed consent was obtained from each resident participant. The study period covered 2 years and included 2 consecutive incoming intern classes. Of 45 eligible residents, 40 enrolled and 38 completed the study. Of the completers, 24 were junior residents and 14 were senior residents. The 2 non-completers were withdrawn due to unavailability during the second cadaver surgery secondary to clinical rotations off site. Upon enrollment, residents were instructed on the steps of one common arthroscopic shoulder procedure, namely an anterior labral repair, followed by an orientation to 2 available simulators for shoulder arthroscopy training.^28,29 One simulator was a common dry lab model (SawBones®, Vashon,WA); and the second was a virtual reality simulator (Insight Arthro VR, Immersion, Cleveland, OH). The instruction was conducted by a board certified orthopaedic surgeon fellowship trained in sports medicine.

The participating residents then performed the arthroscopic labral repair as instructed on a cadaver upper extremity in a mobile bioskills lab (The Surgical Training Institute, LLC, Camas, WA). Prior to the initiation of the lab, staff surgeons created the necessary portals and an anterior labral tear if not already present in each specimen. Following this initial cadaver procedure, each resident was randomized to participate in arthroscopy training using either the dry lab model or the virtual reality simulator. The post graduate year (PGY) participants were randomized separately to assure an even number of junior and senior level residents trained on both available simulators.

The participating residents then practiced on their respective simulators the same arthroscopic shoulder procedure for four weeks at minimum three times per week. After the practice period was completed, each resident performed the arthroscopic labral repair at a second cadaver lab. Following the second cadaver lab, the participants were surveyed on their prior arthroscopy and simulator experience, their subspecialty interest, and their opinion on use of simulation in their residency training, including which simulator was preferred.

Additionally, the residents were given a brief Student Assessment of the Learning Gains (SALG) questionnaire modified to include questions specific to arthroscopy training. ³⁰ The SALG format has been validated by comparison to one-to-one student interviews about classroom perceptions and reliably established by administrating the instrument in multiple comparable educational settings. To account for assessment of arthroscopic training, the instrument was modified with questions pertaining to concept understanding, skill abilities, learning attitudes, and perceived success learning new tasks and administered using a five-point Likert scale. The skills abilities specifically were modeled after the Arthroscopic Surgical Skills Evaluation Tool (ASSET) which has been validated for grading arthroscopic performance.^31-33

Attitudes and preferences pertaining as assessed by the questionnaires were compared between participating residents who were randomized to train on the dry lab model vs the virtual reality simulator using ordered logistic regression and chi-squared analysis. Additional comparisons were made between junior (PGY1, 2, and 3) residents and senior (PGY 4 and 5) residents controlling for randomization group. The individual SALG questionnaire items were reduced using factor weights to four separate latent variables (understanding, skills, attitudes, and learned tasks) and comparisons of the factor weights made between resident and randomization group using Mann Whitney U tests. Participants with incomplete data were dropped from the data set. Analysis was performed using Stata 14.2/IC (College Station, TX).

Results

Learner preferences for new skills at baseline were largely not different between junior and senior level residents. Senior residents were 4.7 times more likely to report learning gains via staff discussion pre- and post-operatively and tended to report learning gains with staff discussion during surgery. Learning preferences were not affected by participating in the simulator training. (Table 1).

OPEN IN VIEWER

Table 1.

Odds of Senior Residents Versus Junior Residents and dry Lab Trainers Versus Virtual Reality Trainers of Reporting Higher Levels of Use for Choice Learning Modalities in Odds Ratios [95% Confidence Interval], n = 38.

	Seniors vs Juniors		Dry Lab Trainers vs Virtual Reality Trainers
	OR	95% CI	OR	95% CI
What RESOURCES do you use to help you learn a new procedure?
Primary text books	.85	[.25, 2.83]	.51	[.15, 1.71]
Journal articles	2.71	[.78, 9.44]	.72	[.21, 2.45]
On-line video material	.94	[.27, 3.29]	.55	[.16, 1.88]
Staff interaction before and after cases	4.70*	[1.05, 21.08]	.75	[.20, 2.77]
Staff interaction during surgery	3.60	[.63, 20.61]	.55	[.12, 2.51]
Interaction with fellow residents	.64	[.18, 2.27]	1.04	[.30, 3.65]
Simulator modality	.68	[.20, 2.27]	.52	[.16, 1.74]

Fifteen of the 24 (62.5%) of junior residents expressed preference for arthroscopy training using the virtual reality simulator; while 8 of 14 (57.1%) of senior residents expressed preference for the dry lab (P = .043). Simulator preference was not influenced by self-reported prior arthroscopy experience (P = .196), prior simulator experience (P = .304), or the simulator to which each resident was randomized for the study (P = .153). Thirty-four of the 38 (89.5%) participating residents reported desiring a greater amount of simulation for surgical skills, including arthroscopy, as part of their residency training. This was not different based on resident level (P = .225). While 30% of residents reported being interested in sports medicine as a future subspeciality, desired subspecialty did not affect simulator preference (P = .231)

Senior residents were 82%–91% less likely to report gains in their skills of management of the arthroscopic view, maneuver their camera, maneuver their instruments, and bimanual dexterity. Overall, senior residents tended to report less gains compared to junior residents but the remaining reported gains were not statistically different. Participants who trained in the dry lab simulator were 71% and 72% less likely to report increased gains in their understanding of arthroscopy in general and their attitude towards arthroscopy, respectively, compared to participants who trained using the virtual reality simulator. (Table 2) Factor weights analysis indicated no difference in weighted sums of the four learning experiences latent variables between junior and senior residents; however, residents that trained using the virtual reality trainer had a higher weighted sum median (P = .0494) for attitude towards arthroscopy. (Figures 1 and 2).

OPEN IN VIEWER

Table 2.

Odds of Senior Residents Versus Junior Residents and dry Lab Trainers Versus Virtual Reality Trainers of Reporting Higher Levels of Gains in Odds Ratios [95% Confidence Interval], n = 38.

	Seniors vs Juniors		Dry Lab Trainers vs Virtual Reality Trainers
	OR	95% CI	OR	95% CI
What gains did you make in UNDERSTANDING based on this training?
Concepts of shoulder arthroscopy	.50	[.15, 1.65]	.29*	[.09, .98]
Applying simulation skills to cadaver	.48	[.14, 1.60]	.54	[.17, 1.74]
Basic arthroscopy skills	.30	[.08, 1.05]	.41	[.12, 1.36]
Applying goals of test procedure	.57	[.17, 1.86]	.58	[.18, 1.89]
Applying simulation skill to surgery	.55	[.16, 1.88]	.37	[.11, 1.21]
What gains did you make in SKILLS base on the following abilities?
Perform surgery safely	.49	[.15, 1.62]	.38	[.12, 1.24]
Manage arthroscopic view	.09***	[.02, .35]	.37	[.11, 1.23]
Maneuver the camera	.07***	[.02, .32]	.30	[.09, 1.03]
Maneuver the instruments	.18**	[.05, .66]	.46	[.14, 1.51]
Use both hands in coordination	.14**	[.04, .56]	.36	[.11, 1.20]
Maintain orderly flow of procedure	.63	[.19, 2.08]	.63	[.20, 2.03]
Complete the procedure	.89	[.28, 2.83]	.61	[.19, 1.90]
Perform without staff intervention	1.63	[.50, 5.29]	.69	[.22, 2.15]
How to the training affect your ATTITUTE towards:
Simulation as a training tool	.77	[.23, 2.54]	.31	[.09, 1.01]
Arthroscopy in general	.28	[.15, 1.80]	.28*	[.08, .98]
What aspects of the training helped you assimilate LEARNED TASKS?
Using a simulator in general	.75	[.22, 2.53]	.59	[.18, 1.96]
Frequency of training	.57	[.16, 2.03]	.63	[.19, 2.10]
Specific tasks assigned during training	.78	[.23, 2.61]	1.06	[.32, 3.51]

OPEN IN VIEWER

Figure 1.

Comparison of factor weights of the four learning experiences variables surveyed by the SALG between junior and senior residents.

OPEN IN VIEWER

Figure 2.

Comparison of factor weights of the four learning experiences variables surveyed by the SALG between virtual reality and dry lab simulator trainers; *P < .05.

Discussion

This research compared resident training preference following virtual reality vs dry lab simulation training for an advanced shoulder arthroscopy repair procedure. Residents in junior vs senior year groups reported different preferences. Given the high number of repetitions required to master any particular surgical procedure and the advancements in what procedures can be accomplished by arthroscopic means, surgical simulation is a reasonable augment to traditional surgical training. Different types of simulation can develop skills and meet the learning needs of residents as skills naturally progress during training years.

A large majority of residents in this study reported wanting more simulation training. This is consistent with prior reports of orthopaedic residents desiring simulation opportunities; and this has important training-needs implications as residents have reported not being prepared for independent arthroscopic performance leading up to and after residency graduation.^10,34,35 In one survey, 70% of graduating residents reported feeling ill-prepared for independent arthroscopy procedures. ³⁴ Our findings confirm the desire for more simulation training across all year-groups and despite ultimate subspecialty interest. Simulator training has been shown to improve resident arthroscopic performance, particularly for novice arthroscopists.^18-23 Improvements have also been shown across residency year groups as measured by the ASSET. ³⁶ Given that residents express desire for simulation training to help fill training gaps and simulation training has been shown to be beneficial, residency programs should consider implementing surgical simulation into their training routines.

Criticism of simulator training points to the ceiling effect learners experience once basic skills are grasped.^18,20 This is particularly true of virtual reality simulation which provides the benefit of simulated haptic feedback but with programming that is often limited to skills translatable to diagnostic arthroscopy. However, these basic skills are still required and may explain our finding that junior residents preferred virtual reality simulator. Junior residents also indicated, regardless of simulator used, greater gains in skills for basic instrument and camera dexterity. The virtual reality simulator, despite limitations in simulated procedures, still allows for basic skills acquisition, the perceived priority for junior residents. The dry lab simulator probably provides less tactile learning but does offer the benefit to learners by handling real surgical instruments and implants. In this way, the dry lab simulator theoretically is more useful for training specific procedures such as the labral repair studied here. Our finding that senior residents preferred the dry lab simulator, as well as reported high learning gains by staff interaction before and after operative cases, highlights the priorities for more advanced training needs. The implication for training programs is that one type of simulator may not meet the needs of both junior and senior level residents.

One surprising finding in this study was that learner understanding of arthroscopy and attitudes towards arthroscopy were higher as measured by individual Likert score responses and attitude latent variable factor weights for residents who trained using virtual reality simulator. It is unlikely that this finding is solely influenced by the residents in our program who expressed interest in sports medicine. Despite simulator preference for the requested task and the limitations of virtual reality simulation, this finding highlights that virtual reality training is enjoyable to the learner. Prior reports from medical and nursing education confirm that virtual simulation is “fun” and thus theoretically a more effective tool for teaching/training than less engaging methods.^37-39 Additionally, as demonstrated by Palumbo et al., training tools that enhance the visual experience of a procedure can also help procedural learners with understanding the procedure. ⁴⁰ Arthroscopy requires a particular skillset and may be difficult and frustrating for less inclined learners. Virtual reality simulation may be a useful tool especially for these residents who would benefit from basic arthroscopic skill repetition outside of the stresses of live surgery. For programs without access to virtual reality simulation year-round, simulation arthroscopy courses have also been shown to be beneficial and could help programs fill simulation gaps. ⁴¹

One limitation of this study is the small sample size, though the number of study completers was higher than many other published reports. This limitation is common among similar studies. In 2 recent pooled analyses, the sample size limitation is evident. Polce et al. conducted a meta-analysis of 24 studies including a total of 494 simulator training participants, indicating that an average of 20.5 participants per study included in the pooled data. ²⁴ Similary, Frank et al. reported on 57 studies involving 1698 participants, an average of 30 participants per study. Despite the limited sample sizes and overall study heterogeneity, these pooled analyses suggest a benefit to simulation training even if a specific training modality cannot be recommended using these data. Because learning preferences are not the focus of these pooled analyses it is difficult to estimate an appropriate sample size to power comparisons of individual learning preferences.

In conclusion, surgical simulation is a desired and potentially useful adjunct to arthroscopic training. Both junior and senior resident report wanting more simulation training. The training and educational preferences identified herein highlight how residents’ learning needs evolve over their training years. Even if different styles of simulation confer benefits towards developing different skills, programs should consider offering some simulation access.