Abstract
Otolaryngology surgical education continues to evolve where trainees increasingly use videos to learn technical skills. Trainees commonly use YouTube, but no study to date has evaluated the educational quality (EQ) of otologic surgical videos on YouTube. We aim to assess the EQ of cholesteatoma surgical videos. Cholesteatoma surgical videos were queried using YouTube search terms, assessed using LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS), a validated assessment tool for publication, and categorized into low (0-6), medium (7-12), and high (13-18) EQ groups. In total, 74 videos were identified (mean LAP-VEGaS score = 9.6 ± 4.0) and 44.6% had medium EQ. Videos commonly lacked graphic aids to highlight anatomy (71.6%) and postprocedural outcomes (68.9%). LAP-VEGaS scores were greater in videos originating from US surgeons compared to non-US surgeons (12.4 ± 3.4 vs 8.0 ± 3.5; P < .001). Our study highlights that otolaryngology trainees may experience difficulty finding high-EQ cholesteatoma surgery videos on YouTube. Areas for improved EQ content are discussed.
Level of evidence: IV
Keywords
Surgical education in otolaryngology has rapidly progressed over the past decades. Trainees have traditionally used written resources to learn technical skills but are faced with work-hour restrictions. 1 To augment the surgical learning experience, videos are now increasingly used, driven by technological advances and ease of accessing online information. 2 YouTube, the world’s largest video-sharing platform, offers trainees access to a large number of surgical videos.3-5
The ability for trainees to easily identify high-quality educational content is unclear given YouTube’s proprietary search algorithm, 5 which likely promotes video exposure based upon noneducational factors. While prior studies in the general surgery literature have assessed YouTube videos for educational quality (EQ),2,6-8 none have focused on otologic surgical videos. This study aims to explore the EQ of cholesteatoma surgical YouTube videos, a procedure that otolaryngology trainees may encounter early in training and may find challenging due to altered anatomy from chronic disease.
Methods
Search Strategy
This study was deemed exempt from review by the Massachusetts Eye and Ear Institutional Review Board because it involves publicly available videos. To simulate an Internet search by a trainee in “real-world” settings (eg, limited time), a YouTube search according to “relevance” with a cache-cleared browser on January 1, 2021, was performed using “cholesteatoma surgery,” “cholesteatoma procedure,” “cholesteatoma removal,” and “cholesteatoma endoscopic surgery.” The first 50 results from each keyword query were evaluated. Videos were included if cholesteatoma operative footage and English commentary were present. Videos that were non-English were excluded.
Video Evaluation
Video characteristics (eg, view count), presence of auditory commentary and subtitles, and the surgeon’s country of origin were recorded. Each video was graded using the LAParoscopic surgery Video Educational GuidelineS (LAP-VEGaS), a validated surgical video assessment tool for presentation and publication 9 previously used to assess pyeloplasties, 8 appendectomies, 7 and neck dissections. 10 Nine items are assessed and scored from 0 (not present) to 2 (extensively presented) 9 and used to categorize videos into low (0-6), medium (7-12), and high (13-18) EQ groups.
The LAP-VEGaS scoring was performed by 2 reviewers on a subset of 38 videos. 11 The intraclass correlation coefficient was excellent with a value of 0.92 (95% CI, 0.84-0.96; P < .0001), and 1 reviewer then scored the remaining videos.
Statistical Analysis
Categorical variables were summarized by frequency and percentage. Continuous variables were summarized by mean and standard deviation. The Mann-Whitney U test was performed to compare LAP-VEGaS scores between US and non-US-based surgeons. Spearman correlation was performed to evaluate the association between LAP-VEGaS scoring and video characteristics. Statistical analyses were performed using SPSS Statistics. P values <.05 were considered statistically significant.
Results
Video Characteristics
The search query identified 74 videos that met inclusion criteria. The mean view count, video age, and length were 16,141, 4.6 ± 3.2 years, and 14′33′′± 21′9′′, respectively ( Table 1 ). Auditory commentary was present in 60.8% of videos, and 63% did not originate from the United States.
Cholesteatoma Surgical Video Characteristics.
Less than 100% due to missing data.
Educational Assessment
The mean LAP-VEGaS score was 9.6 ± 4.0, with 19 (25.7%), 33 (44.6%), and 22 (29.7%) videos being categorized into low, medium, and high EQ, respectively. Overall, videos most commonly lacked graphic aids to highlight pertinent anatomy (71.6%), information on postprocedural outcomes (68.9%), and surgical positioning (48.6%). Most videos had clear image quality (91.9%), thoroughly displayed surgical procedure steps (62.2%), and had English commentary (58.1%). LAP-VEGaS scores were significantly greater in videos originating from US surgeons compared to non-US surgeons (12.4 ± 3.4 vs 8.0 ± 3.5; P < .001) ( Table 2 ). LAP-VEGaS scores were positively correlated with higher video like count (r = 0.43; P < .001) and longer video length (r = 0.25; P = .033).
Educational Assessment of Cholesteatoma Surgical Videos.
Abbreviation: LAP-VEGaS, LAParoscopic surgery Video Educational GuidelineS.
Discussion
Our study is the first to assess the EQ of otologic surgical videos from YouTube. In our simulated search as an otolaryngology trainee, most cholesteatoma surgery videos had average EQ. Videos often had high image quality and thoroughly presented surgical steps, with videos from US authors having higher EQ.
Developing useful videos for trainees is a challenging process as it involves balancing time spent producing videos 12 while maximizing EQ. Our study identifies common areas for improvement, such as using graphic aids and discussion of postsurgical outcomes. Given the regular use of cameras in otologic procedures (endoscopy, microscopy), 13 numerous opportunities exist to record educational content. In diseases with chronic destructive processes, such as cholesteatoma, 14 distinguishing anatomical structures may be challenging. Therefore, embedding videos with arrows or creating representative diagrams may benefit the viewer. 15
The findings from this study and others10,16 highlight the difficulty of trainees to reliably identify high-EQ otolaryngology surgical content on YouTube. Higher LAP-VEGaS scores were found to be positively correlated with greater like counts, which may be a helpful indicator of high-EQ videos. Nonetheless, this platform does not require peer review prior to publishing and may lead to decreased EQ. Currently, few surgical video resources are peer reviewed or created by academic professional societies such as the American Academy of Otolaryngology–Head and Neck Surgery and Headmirror. 17 While these atlases use YouTube for video sharing, our search rarely found these videos, highlighting that trainees may need to independently search for these. Emerging journal formats allowing users to submit surgical videos for peer review 18 may help maintain academic rigor by following outlined criteria.
This study has several limitations. First, authors may not have been aware of LAP-VEGaS criteria when creating their videos. Nonetheless, the high number of videos with average EQ suggests the need for otolaryngologists to publish more content, potentially in a peer-reviewed format. Second, the LAP-VEGaS criteria were originally created to assess laparoscopic surgery, where elements (eg, port placement) may be less relevant to otologic surgery. However, most elements have strong overlap with otologic surgery in assessing EQ in intraoperative and perioperative settings. Next, LAP-VEGaS criteria, which overall are a suitable fit for cholesteatoma surgery by providing several opportunities to create annotations and commentary, consider author and institution names and intraoperative steps as having the same educational value, which may be less beneficial when learning technical skills. A future study may revise the scoring system to award additional points for technical skills to promote thorough explanations. Finally, the decision to review the first 50 videos of keyword searches may have prevented analysis of high-EQ videos appearing in later search results. Nonetheless, we intended that our search strategy would reflect those of the majority of otolaryngology trainees with time constraints.
Conclusion
This is the first study to evaluate the EQ of otologic surgical videos on YouTube. Otolaryngology trainees may experience difficulty finding high-EQ content on YouTube for cholesteatoma surgery.
Author Contributions
Disclosures
Footnotes
This article was presented as a poster presentation at the 125th AAO-HNSF 2021 Annual Meeting & OTO Experience; October 5, 2021; Los Angeles, California.