Abstract
Objective
Mastoidectomy is a cornerstone in the surgical management of middle and inner ear diseases. Unfortunately, training is challenged by insufficient access to human cadavers. Three-dimensional (3D) printing of temporal bones could alleviate this problem, but evidence on their educational effectiveness is lacking. It is largely unknown whether training on 3D-printed temporal bones improves mastoidectomy performance, including on cadavers, and how this training compares with virtual reality (VR) simulation. To address this knowledge gap, this study investigated whether training on 3D-printed temporal bones improves cadaveric dissection performance, and it compared this training with the already-established VR simulation.
Study Design
Prospective cohort study of an educational intervention.
Setting
Tertiary university hospital, cadaver dissection laboratory, and simulation center in Copenhagen, Denmark.
Methods
Eighteen otorhinolaryngology residents (intervention) attending the national temporal bone dissection course received 3 hours of mastoidectomy training on 3D-printed temporal bones. Posttraining cadaver mastoidectomy performances were rated by 3 experts using a validated assessment tool and compared with those of 66 previous course participants (control) who had received time-equivalent VR training prior to dissection.
Results
The intervention cohort outperformed the controls during cadaver dissection by 29% (P < .001); their performances were largely similar across training modalities but remained at a modest level (~50% of the maximum score).
Conclusion
Mastoidectomy skills improved from training on 3D-printed temporal bone and seemingly more so than on time-equivalent VR simulation. Importantly, these skills transferred to cadaveric dissection. Training on 3D-printed temporal bones can effectively supplement cadaver training when learning mastoidectomy.
Keywords
Mastoidectomy is a fundamental procedure in the surgical management of middle and inner ear disease. The key anatomic landmarks include the facial nerve and chorda, dura, sigmoid sinus, lateral semicircular canal, and incus. Excellent anatomic understanding and technical skills are required to perform the procedure safely, and high-quality training is therefore pivotal.1,2 An important component for such high-quality training is repeated deliberate practice in a safe, simulation-based environment before real-life surgery. 3 The gold standard for simulation-based training of mastoidectomy is human cadavers, but their availability is increasingly scarce. 1 Consequently, other options are needed for simulation-based training of mastoidectomy as a supplement to traditional cadaver dissection training.
These options include virtual reality (VR) simulation and training on physical models, such as temporal bones of plaster or plastic. Mastoidectomy VR simulation is the most frequently used alternative to cadaveric dissection, 4 and most educational evidence is therefore centered on this training modality. 5 Overall, VR simulation training is effective in the early stages of training,6,7 but the learning curve seems to plateau after relatively few procedures at an unsatisfactory level.8,9 This is a problem since trainees should reach the highest possible level before progressing to cadavers or patient surgery, and it emphasizes the need for supplemental training that can efficiently elevate trainees’ skills beyond this learning plateau.
Three-dimensional (3D) printing/additive manufacturing is an emerging technology that has great potential for surgical simulation training. 10 Using 3D-printed temporal bone models for training is now feasible as they are increasingly available and affordable. 11 These models potentially enable realistic training of physical aspects of the procedure due to a more lifelike drilling experience than VR simulation—for example, by featuring real drills and an operating microscope. 12 A large number of studies have reported on the development and manufacturing of 3D-printed models for temporal bone training, but there is limited educational evidence on their use. Specifically, most evaluations rely on participants’ subjective opinions of the models 13 : a measure that is poorly correlated with actual surgical skills.14,15 Consequently, it remains largely unknown whether 3D-printed temporal bone models can be used to improve surgical performance.
A key concept in simulation-based training is transfer: skills learned in one context should transfer to another (eg, VR simulation to cadaver or real-life patient surgery). 16 Learning skills during simulation that do not benefit or transfer to patients would render such training irrelevant. VR training of mastoidectomy transfers to improved cadaver dissection, but knowledge on transfer from 3D-printed models is scarce, 13 despite their perceived potential for training, and only a single study has evaluated the use of 3D-printed models as preparation for cadaveric dissection. 17 Consequently, further knowledge on transfer is needed to elucidate whether 3D-printed temporal bone models can actually aid trainees’ skills acquisition and to plan the best possible training curricula in an era of limited cadaver dissection. Therefore, understanding the relative effectiveness of different training modalities is highly useful for training institutions wishing to acquire new training options for trainees.
We hypothesized that 3D-printed models are an effective training tool in early acquisition of novices’ basic mastoidectomy skills and that skills transfer to increased performance during cadaveric dissection. In this study, we compared training on 3D-printed models with VR simulation (the current gold standard for precadaver training) and evaluated the effects on performance (ie, transfer to cadaver dissection).
Materials and Methods
Study Design
This was a prospective cohort study of an educational intervention.
Participants and Setting
Eighteen otorhinolaryngology (ORL) residents from 8 of 10 Danish ORL training institutions were enrolled in relation to the annual Danish national temporal bone dissection course at Rigshospitalet, Copenhagen, Denmark, in January 2021 (intervention). The performances of the intervention cohort were compared with the historical performances of 66 ORL residents receiving comparable amounts of VR simulation training prior to cadaveric dissection at previous national temporal bone dissection courses held 2014 to 2017 and 2019.7,18,19 Figure 1 shows the trial flow diagram. Simulation training took place at the Copenhagen Academy for Medical Education and Simulation, Rigshospitalet. Upon enrollment, participants signed informed consent and completed a background questionnaire.
Study flow diagram.
Training Interventions
The intervention cohort was given 3 hours to perform 3 anatomic mastoidectomies on identical 3D-printed temporal bone models ( Figure 2 ) following the principles of directed, self-regulated learning. 20 Consequently, training was performed independently without hands-on instructor feedback or guidance. During training, participants had access to a printed dissection manual with step-by-step instructions, including pictures of the 3D-printed model drilled correspondingly and, on request, a perfectly drilled model. Otosurgical drills (Midas Rex Legend EHS Stylus Motor; Medtronic) with a standard array of drill bits and an operating microscope (Leica M320 F12; Leica Microsystems) were provided for each participant. Suction and irrigation were not used, and plastic drill dust was manually removed.
Views of the 3D-printed model. Left: frontal view. A yellow wire represents the facial nerve. Right: posterior view. The pink silicone layer represents the dura and the blue layer, the sigmoid sinus.
Similar to the intervention cohort, the historical controls receiving training on a VR simulator were asked to independently complete at least 3 anatomic mastoidectomies (ie, without any instructor guidance). All historical controls were given 3 hours of training, except for the 2014 cohort, which received 2 hours due to study conditions. Throughout the VR training, participants had access to an on-screen step-by-step guide of the dissection procedure as well as simulator-integrated “green lighting” during the first procedure. 21 Green lighting is a tutoring function that highlights the bony structures that need to be removed in the current step of the procedure. The dissection manual provided for the intervention group and the on-screen step-by-step guide provided for the historical controls included similar steps and written instructions. Examples of final-product performances on a 3D-printed model and the VR simulator are provided in Figure 3 .
Example of final-product performance on a 3D-printed model (left) and in the virtual reality simulator (right).
Simulation Models
The VR simulator used for training was the Visible Ear Simulator (VES) developed at our institution. VES is an academic freeware simulator for temporal bone surgical training and includes several cases of temporal bone models based on cryosection and cone beam computed tomography (CBCT) scans. 22 In this study participants trained on the case VES. The simulator uses the Geomagic Touch haptic device (3D Systems) to enable force feedback of drilling during simulation.
The 3D-printed model developed for this study was created with a CBCT data set entitled Delta.23,24 The VES Delta data set was translated to STL format (Standard Tessellation Language) through standard settings in the MeshLab software. 25 Next, the model was printed on a readily available fused deposition modeling (FDM) 3D printer (Ender-3 Pro; Creality) with LayBrick filament (CC Products)—a composite material with finely milled chalk in a thermoplastic matrix, making the material resemble bone. The chalk makes the composite material more brittle than other standard materials for 3D printing and prevents it from melting easily during drilling. A colored silicone layer was applied after printing to represent the dura and sigmoid sinus. The facial nerve was represented by a yellow wire inserted into the fallopian canal ( Figure 2 ). The Delta case is based on a CBCT data set that is more suitable for STL file translation and is largely identical to the standard VES case (based on cryosections) used for VR simulation. 24 The printer settings are presented in Table 1 .
Printer Settings.
Outcomes
After simulation training, the intervention cohort and the historical controls performed an anatomic mastoidectomy on a human cadaveric temporal bone with a time limit of 1 hour. No instruction or guidance was available for the cadaver dissection besides technical assistance (eg, changing drill bits). All dissections were rated by 3 expert raters.
The primary outcome was performance during cadaveric dissection, and the final drilling result (final-product performance) was assessed via a 26-item modified Welling Scale. 26 The secondary outcome was performance during simulation-based training (ie, performance on the 3D-printed temporal bone models), as rated by 3 experts blinded to participant and procedure number.
Sample Size
Sample size calculations for repeated measurements are not well defined. However, comparable studies have suggested that 16 participants would be needed in each arm to demonstrate a 10% improvement in final-product score. 19 Eighteen participants attended the mandatory residency dissection course in 2021 and represent a sample of convenience deemed sufficient for the study purpose.
Statistical Methods
Statistical analyses were performed with SPSS version 27 for Mac (IBM). Due to repeated measurements (multiple raters for each performance), we used linear mixed models (LMMs) for analysis of performances following the principles outlined by Leppink. 27 The LMM for cadaveric dissection performance included training modality (3D-printed model or VR simulation) and rater as fixed effects. The LMM for 3D-printed model performance included the session as a repeated measurement and the rater as a fixed factor. No interactions were found. Pearson’s r was used for correlation analysis between cadaveric and 3D-printed model performances. P values <.05 were considered statistically significant.
Ethics
The Regional Ethical Committee of the Capital Region of Denmark deemed this educational trial exempt (H-20078830). Written consent was obtained from participants. Participation was voluntary and no compensation was given.
Results
The cohorts had comparable baseline characteristics (age, sex, ORL experience, and previous training; Table 2 ).
Participant Characteristics.
The intervention cohort outperformed the historical controls by 29% during cadaver dissection (mean difference, 3.1 points [95% CI, 1.7-5.0]; P < .001). The intervention cohort had a cadaveric dissection performance mean score of 13.7 points (95% CI, 12.3-15.1) as compared with the control group’s mean score of 10.6 points (95% CI, 9.9-11.3). A box plot of the cadaveric performance is presented in Figure 4 .
Box plot of cadaveric dissection performance for the intervention and historical controls. Values are presented as median (line), interquartile range (box), and 95% CI (error bars). Outliers (dots) are cases with values 1.5 to 3 times the interquartile range.
In their training performances on 3D-printed temporal bone models, participants scored a mean 13.9 points (95% CI, 13.1-14.7), which was almost identical with the subsequent cadaveric dissection (13.6 points (95% CI, 12.3-15.1). Performance score did not increase during the participants’ 3 repetitions (P = .10). There was a moderate correlation between 3D-printed model and dissection performance (r = 0.49, P < .001).
Discussion
In this study, we found that training with 3D-printed models improved novices’ subsequent first cadaver dissection performance by 29% more than VR simulation. There was a moderate correlation between final-product scores of cadaveric dissection and performance on the 3D-printed models.
Three-dimensional printing provides new possibilities for training technical skills, and temporal bone surgery is no exception.10,13 The expected potential of 3D-printed models for temporal bone training is supported by the establishment of a 3D-Printed Temporal Bone Working Group by the American Academy of Otolaryngology–Head and Neck Surgery, which has the purpose of gathering knowledge on the subject. 28 There are several considerations when creating a suitable temporal bone model with 3D printing, as the various print technologies have different strengths and limitations. Choice of print technology affects the price of the model but also the material available for printing, the technical expertise necessary for in-house printing, and postprocessing. In this study, models were printed on a commercially available and inexpensive desktop FDM printer (US $300). FDM printers work by depositing thermoplastic material layer by layer, enabling the formation of mastoid air cells. Previous studies have reported how FDM-printed models melted during drilling 28 and therefore suggested using resin-based printers. However, resin-based printers are more costly and are unable to accurately form the mastoid air cells and hollow structures inside the bone model. 29 In contrast to previously reported FDM-print models,28,30-34 our model was printed with a composite material composed of a finely milled chalk in a thermoplastic matrix. This results in a more bone-like material and prevents it from agglomerating when drilling, providing a more realistic drilling experience. The material for each printed model is inexpensive (US $5) and thus a cost-effective alternative to cadaveric dissection. However, it takes approximately 16 hours to print a single temporal bone model, excluding time for maintenance of the printer. This emphasizes that production of 3D-printed models requires committed time and technical knowhow. While mimicking bone properties closely, the present models lack realistic representation of soft tissue, such as the facial nerve, dura, and sigmoid sinus,13,28 and require manual postprocessing. This includes applying a silicone layer to the dural and sigmoid side of the model and inserting a yellow wire to represent the facial nerve. Ultimately, the resolution of an FDM-based 3D print is currently insufficient for representing the chorda, and despite being present, the ossicles are not accurately replicated as with more costly high-resolution print technologies. 35
Unlike these limitations of 3D-printed temporal bone models, VR simulators provide a realistic visual representation of soft tissue structures. Nevertheless, improved graphic fidelity (ie, visual lifelikeness) does not necessarily result in higher educational effectiveness. 14 For some skills, enhancing haptic fidelity can create a closer link between the simulated task and the clinical task than visual accuracy. 36 This 3D-printed model for temporal bone training may serve as a relevant example: despite poor representation of soft tissue (ie, visual lifelikeness), drill and microscope handling more accurately mimics the physical variables in a real-life surgical setting (higher haptic fidelity of drill-to-bone interaction). In our study, all participants were novices at handling the surgical instruments and performing the mastoidectomy. As training on 3D-printed models yielded better results than VR simulation, haptic fidelity mattered more than visual lifelikeness for the initial learning of mastoidectomy procedures. However, this study concerns basic surgical skills acquired after only 3 repetitions, leading to a mean dissection score of 10 to 14 out of a possible 26 for a perfect mastoidectomy with posterior tympanotomy. The remaining high-end points would reward the proper and safe thinning of bone covering soft structures at risk without undue exposure. This requires the surgeon to take advantage of visual cues of the soft tissue vascular surface patterns identified through a thin layer of intact transparent bone. The graphic fidelity of the printed model does not provide such visual cues because neither soft tissue surfaces nor bone transparency is well reproduced. For this reason, the model-to-patient task alignment of the training of such precision skills is most likely better in VR simulation than in 3D-printed simulation. Nevertheless, because these tasks represent a higher skills level than attained after the few repetitions completed in our trial, they do not affect this study’s result; rather, they might come into play in training of experienced trainees. Moreover, if the model is to be used for patient-specific presurgical planning/rehearsal, the visual lifelikeness seems decisive, as accurate representation of anatomic structures (eg, facial recess) is essential.
Participants had 1 hour to complete a cadaveric mastoidectomy, and the final-product performance was rated with a 26-item modified Welling Scale, 26 which is a well-established and reliable assessment tool for mastoidectomy performance.37,38 The tool uses binary scoring of items that reflect specific areas. Consequently, trainees who complete more of the procedure achieve higher scores. When performance is time limited, improvements might not reflect the true ability in a scenario with unlimited time. Therefore, the advantage of training on 3D-printed models is potentially explained by increased time efficiency during cadaver dissection rather than actual superiority as compared with VR simulation. However, time is a key factor in surgical training because operating room and supervisor time during training is costly. 39 Consequently, this assessment of skills reflects a real-world scenario, which is desirable when evaluating training interventions and their clinical implementation. 40
A limitation of our study is the prospective design. Despite the cohorts’ comparable baseline characteristics (eg, similar age and clinical experience) and the standardization of interventions and assessment, the nature of a prospective cohort study remains a threat to internal validity. 41 A single cohort in the historical controls was given only 2 hours of VR training prior to cadaveric dissection 18 ; nonetheless, this cohort had the highest scoring performances among the historical cohorts. This highlights how performances vary from year to year despite participants having comparable clinical experience and the same amount of training. To minimize the effect of this individual variation, we pooled all data from previous participants receiving comparable VR simulation. However, since the intervention group in this study consists of just a single cohort, the relative effect of the intervention as compared with the control might be over- or underestimated. Regardless, the positive effect of training on 3D-printed models prior to cadaveric dissection seems substantial, and this study adds valuable new knowledge to the field of temporal bone surgical training. Most previous studies on 3D-printed models for temporal bone surgical training have based conclusions about effectiveness solely on participants’ opinions on the training. 13 This provides insufficient evidence and is considered outdated in surgical educational research. 15 A single study found that participants training with 3D-printed models performed better during subsequent cadaveric dissection as compared with no training. 17 However, training was not performed in a single-site standardized setting, and assessments were done by a single rater. In contrast, our study uses standardized data collection and modern educational research methodology (eg, multiple raters), and it features comparison of training on 3D-printed models with another intervention, the already established VR simulation. Such “comparative effectiveness” studies are strongly recommended in surgical educational research, as comparing an intervention with no intervention adds little evidence, since even minor training or inefficient training is practically always better than no training. 42
Regardless of training modality, our results demonstrate that the novices’ skill level from this short intervention remains insufficient for performing a safe mastoidectomy because only 50% of the maximum score (ie, a perfect performance) was attained. This might suggest that a comprehensive training curriculum featuring a combination of VR simulation, 3D-printed models, cadaver dissection, and gradual supervised surgery is needed to safely and efficiently learn the procedure. Future research should investigate whether a combination of training modalities can accelerate and extend learning, including how to best integrate them in a comprehensive training program.
This is the first study demonstrating that 3D-printed models improve cadaver dissection performance and compare favorably with VR simulation during cadaveric dissection. Regardless of modality, implementation in the surgical curriculum should be based on strong educational evidence. This study represents an important step in this process by adding evidence on 3D-printed models for mastoidectomy training.
Conclusion
Mastoidectomy skills training with 3D-printed temporal bone models transfers to cadaveric dissection, substantially improving novices’ performance. Despite the remaining room for improved visual lifelikeness, 3D-printed models are an excellent and inexpensive supplement during initial mastoidectomy training and enable training of tactile aspects as real-life instruments are used. However, it is important to continue the process of collecting strong educational evidence through rigorous methods to establish the best use and implementation of 3D-printed models in the temporal bone surgical curriculum.
