Comparison of Online Learning Video Platforms Regarding Laparoscopic Adrenalectomy: YouTube and WebSurg

Abstract

Background:

The internet is a widely used source for receiving medical information. Nevertheless, the quality of data on online platforms is still questioned. Our aim was to compare the laparoscopic adrenalectomy videos published on the two online platforms, WebSurg^® (WS) and YouTube^® (YTb) regarding the quality of data, educational power, and accuracy of the sources.

Materials and Methods:

Most viewed popular visual content returned by YTb in reply to the keyword “laparoscopic adrenalectomy” were involved in the research protocol. The quality of the data, educational power, and the accuracy and reliability of the sources were assessed by applying new scoring systems. A novel scoring method estimated technical quality. The 10 most viewed videos in the WS were compared with 10 YTb videos with the highest technical scores.

Results:

Scoring systems measuring the quality of data and educational power of the mostly viewed WS visual content revealed higher scores than those of the most technically qualified 10 YTb videos (10 videos having the highest laparoscopic adrenalectomy scoring system scores [LASS-S]; P = .021; P = .003; P = .025, respectively). Also, there was no significant difference between the top 10 most viewed WS videos and 10 YTb videos having the highest LASS-S in terms of LASS-S (P = .34).

Conclusions:

Although WS videos were passed through a professional reviewing process, and were mostly provided by academicians, they remained below the expected quality. The presented research showed that attentively chosen YTb content on laparoscopic adrenalectomy is almost as accurate as WS content. Notwithstanding all its limitations, the informative power or potential of YTb should not be neglected.

Introduction

In the past two decades, visual learning resources have become preferable to traditional approaches such as publishings and meetings^1,2; thus, the websites that are quickly reachable by computer users have globally expanded. This event was titled “Participator and Social Web” by Tim O'Reilly and Dale Dougherty in 2004.³ Currently, one of this event's most famous models is YouTube^® (YTb), a fast-growing quickly accessible online site that reaches more than two billion clicks everyday.^4,5 As with all internet users, YTb's reputation improved among medical professionals and surgeons presented more attention in this excellent and endless data source, which provides enhanced visible and audible data than just texts.⁶ However, the data providers' heterogeneity and unconfirmed data have caused the dissemination of inaccurate or misleading information also regarding health.^5,7

WebSurg^® (WS) is the other archetype of this snowballing online phenomenon. This professional surgical education website defines itself as a virtual university, providing visual and audible data on novel surgical techniques. WS runs various high-resolution surgical education media and professionally focuses on continuous education/training of surgeons. The number of WS users has grown 20 times from 2004 to 2010. In the same period, WS website clicks increased 8 times, and video views has grown 33 times. This rapid progress shows that this virtual university allures surgeons' notice.^2,8

For the past three decades, adrenalectomy by the laparoscopic approach has been widely accepted as the gold-standard method for most of the adrenal masses among surgeons. This minimally invasive technique's advantages are shortened hospitalization time, swift recovery of the patient, and improvement of the patient's satisfaction. So, the request to obtain up-to-date knowledge about laparoscopic adrenalectomy has immediately ascended⁹; thus, YTb and following, WS's reputation have increased.^8,10 However, there are questions regarding the accuracy of the online data that provide visible and auditory interaction.

In the presented research, our principal aim is to make a comparative analysis of the accuracy of the 10 most viewed laparoscopic adrenalectomy videos on WS, with the 10 most reliable laparoscopic adrenalectomy videos aired on YTb. And further, we tried to answer the question, “Have WS and YTb provided satisfactory technical data for the education of surgeons?”

Materials and Methods

We conducted an online search on YTb's searching engine, typing the keyword “laparoscopic adrenalectomy” on December 1, 2020, and sorted the content considering its popularity. The most viewed 100 videos aired on YTb were evaluated. We included the first 10 videos having the highest laparoscopic adrenalectomy scoring system score (LASS-S) in the study protocol. Concurrently, we did the same research on WS by lining up the content in terms of the view rate. The first 10 most popular visual content viewed was analyzed.

Classification of videos according to source and content

All videos were classified under two main topics based on their source and content (Table 1).

Table 1.

Classification of YouTube Video According to Source and Content

According to source	According to content
Medical	Surgical technique
Nonmedical	Information about disease or surgery
	Lecture
	Other

Evaluation of the visual content

Three endocrine surgeons, performing adrenalectomy by laparoscopic approach in routine practice, evaluated the visual content. We applied known evaluation systems shown in Table 2.^11–14 We also used the LASS, which was created by the same three endocrine surgeons, in the evaluation of the medical and technical quality of data (This scoring scheme was altered from the guidelines of the American Association of Clinical Endocrinologists/American Association of Endocrine Surgeons and Society of American Gastrointestinal and Endoscopic Surgeons; Table 3).^15,16

Table 2.

Assessment Methods of the Videos

VPI: To assess both the view and the like ratio of the videos, we preferred the VPI. The formula calculated VPI: like ratio × view ratio/100

DISCERN-Q: To define the quality of the data and suggested treatment options, we practiced the DISCERN-Q, which was evolved by experts at Oxford University in the United Kingdom. This questionnaire system ranges between 0 and 80 points and has three parts, including 16 questions, each question is rated on a five-point scale ranging from No to Yes; 80: the quality criterion has been wholly fulfilled; 32–64: the quality criterion has been partially fulfilled; 16: the quality criterion has not been fulfilled at all.

Global quality score: It is a five-point scale defined by Bernard et al. that was applied to evaluate the educational power of each video (1—poor quality, very unlikely to be of any use to patients; 2—poor quality but some information present, of very limited use to patients; 3—suboptimal flow, some information covered but important topics are missing, somewhat useful to patients; 4—good quality and flow, most important topics covered, useful to patients; 5—excellent quality and flow, highly useful to patients).¹³

JAMA-S: The accuracy, utility, and reliability of each video source were evaluated according to JAMA-S 1 point: insufficient data about video source, 2–3: partially sufficient data about video source, 4: completely sufficient data about video source.

DSCq, DISCERN questionnaire; JAMA-S, Journal of American Medical Association benchmark criteria; VPI, video power index.

Table 3.

Laparoscopic Adrenalectomy Scoring System

A. Preoperative evaluation
1. Was the age of the patient specified on video?
2. Was the gender of the patient specified on video?
3. Was the operation side of the patient specified on video?
4. Was operation indication specified on video?
5. Was the patient's comorbid diseases stated on video?
6. Was preoperative abdominal imaging findings specified on video?
7. Was preoperative laboratory findings specified on video?
8. Was information about patient's previous surgery history stated?

B. During surgery
1. Was the port locations specified on video?
2. Was the diameters of the ports specified on video?
3. Was the patient's position specified on video?
4. Was the adrenal vein seen on video?
5. Was the kind of energy device for vascular ligation/dissection specified on video?
6. Was ligation method of adrenal vein specified on video?
7. Was the usage or not usage of drain specified on video?

C. After surgery
1. Was whether the postoperative complication developed or not specified on video?
2. Was the hospitalization period or discharge time specified on video?
3. Was the oral diet start day specified on video?

Questions were answered as yes or no, yes = 1 point, no = 0 point. Based on the Sleeve Gastrectomy Scoring System checklist, each of the 18 criteria was given 1 point if presented orally or written in the video.

Statistical analysis

GNU Octave 4.3.0 + 2020 (PA, USA) computer application was utilized for statistical evaluation. Descriptive analytical systems (mean, standard deviation, median, first quadrant, the third quadrant, frequency, percentage, minimum, and maximum) were applied to evaluate research data. Average distributions of quantitative data were tested with the Anderson–Darling test. Two-sample t-test was practiced in the comparative analysis of two groups of quantitative variables with the normal distribution. Wilcoxon–Mann–Whitney test was applied in the comparison of the two groups of quantitative variables without normal distribution. The Kruskal–Wallis test and Dunn's nonparametric comparison were made for two-way intergroup quantitative variables with no normal distribution. Fisher's exact test and Fisher–Freeman–Halton exact correlation analysis were used to evaluate the relationships between quantitative variables. Statistical significance was accepted as P < .05.

Results

Evaluation of YTb data

In evaluating 100 visual content obtained from YTb, 67% are classified as the medical source. Surgical technique (69%) is the most dominant video content (11% information about disease or surgery, 10% other, and 10% lecture). DISCERN questionnaire (DSCq), Journal of American Medical Association benchmark criteria (JAMA-S), Global quality score (GqSc), and LASS scores of medical sourced visual content were higher than nonmedical sources (P < .001, P < .001, P < .001, P < .001, respectively). However, the video power index score (VPI-S) of nonmedical videos was higher (P < .001). DSCq, JAMA-S, GqSc scores of the visual content focusing surgical technique were higher (P < .001, P < .001, P < .001, P < .001, respectively; Table 4).

Table 4.

Comparison of Scores According to Descriptive Characteristics of 100 YouTube Videos

	VPI	DSCq	JAMA-S	GqSc	LASS
Video source
Medical	131.86 (2.00, 2938.86)	15 (10, 20)	2 (1, 3)	3 (2, 4)	6 (4, 9)
Others	290518.51 (2014.00, 297675.64)	5 (5, 5)	0 (0, 0)	1 (1, 1)	0 (0, 0)
P	<.001^*	<.001^*	<.001^*	<.001^*	<.001^*
Video content
Surgical technique	10.15 (1.28, 146.09)	10 (5, 15)	1 (1, 2)	2 (1, 3)	5 (4, 8)
Lecture	6950.84 (2113.86, 137876.81)	5 (5, 15)	0 (0, 0.5)	1 (1, 2)	0 (0, 0)
Information about disease or surgery	11349.90 (1931.56, 227036.71)	5 (5, 5)	0 (0, 0)	1 (1, 1)	0 (0, 0)
Other	802910.45 (742.30, 38,910,820)	5 (5, 5)	0 (0, 0)	1 (1, 1)	0 (0, 0)
P	<.001^*	<.001^*	<.001^*	<.001^*	<.001^*

Kruskal–Wallis test, reported as median (first quartile and third quartile) ^*P < .05.

DSCq, DISCERN questionnaire; GqSc, global quality score; JAMA-S, Journal of American Medical Association benchmark criteria; LASS, laparoscopic adrenalectomy scoring system; VPI, video power index.

Comparative analysis of the content on YTb and WS

The 10 videos with the highest educational/technical scores out of 100 most viewed videos on YTb were selected. In addition, the most popular 10 videos on WS were selected. The 10 videos of WS were uploaded by the academic staff affiliated with a university, whereas 9 of 10 videos of YTb with the highest LASS scores were uploaded by the university academic staff and the remaining 1 was uploaded by a physician. No significant difference was found between the videos of the two platforms regarding the source (P = .078). The content of the videos selected from YTb and WS was all related to the surgical technique. The view, like, and dislike mean values of the two groups were similar (P = .597, P = .0335, P = .076, respectively; Table 5).

Table 5.

Comparison of Descriptive Characteristics According to Website

	WebSurg (n = 10)	First 10 YouTube (n = 10)	P
Video source, n (%)
Academic	10 (100)	9 (90)	.078^a
Physician	0 (0)	1 (10)
Video content, n (%)
Surgical technique	10 (100)	10 (100)	–
View
Mean ± SD	6449.4 ± 3285.97	6895.6 ± 8696.3	.597^b
Median (Q1, Q3)	4862 (4006, 4762)	3717.4 (267, 8854)
Like
Mean ± SD	57 ± 27.08	39.9 ± 48.87	.335^b
Median (Q1, Q3)	54.3 (43, 67)	15.6 (3, 72)
Dislike
Mean ± SD	0 ± 0	3.3 ± 5.79	.076^b
Median (Q1, Q3)	0 (0, 0)	0.5 (0, 3)

Fisher's exact test.

Mann–Whitney U test.

Q1, first quartile; Q3, third quartile; SD, standard deviation.

DSCq, JAMA-S, and GqSc of WS visual content were higher than the most technically qualified 10 YTb videos (10 videos having the highest LASS scores) (P = .021; P = .003; P = .025, respectively; Table 6). Also, there is no significant difference between the top 10 most viewed WS videos and 10 YTb videos having the highest LASS scores in terms of LASS scores (P = .34; Table 6).

Table 6.

Comparison of Scores of Videos According to Website

	WebSurg (n = 10)	First 10 YouTube (n = 10)	P
VPI
Mean ± SD	5867.46 ± 1295.53	11720.08 ± 6710.85	.584^a
(minimum–maximum)	(3047–8203)	(2906–19,021)
DSCq
Mean ± SD	47.2 ± 7.82	39.7 ± 4.5	.021^b,^*
(minimum–maximum)	(35–60)	(20–45)
JAMA-S
Mean ± SD	2.9 ± 0.2	1.9 ± 0.88	.003^b,^**
(minimum–maximum)	(2–4)	2 (1–3)
GqSc
Mean ± SD	3.7 ± 0.2	2.9 ± 0.6	.025^b,^*
(minimum–maximum)	(2–5)	3 (1–3)
LASS
Mean ± SD	7.4 ± 1.4	8.2 ± 2.1	.34^b
(minimum–maximum)	(6–11)	(6–13)

Mann–Whitney U test.

Student's t-test.

P < .05; ^**P < .01.

DSCq, DISCERN questionnaire; GqSc, global quality score, JAMA-S, Journal of American Medical Association benchmark criteria; LASS, laparoscopic adrenalectomy scoring system; SD, standard deviation; VPI, video power index.

Discussion

With the standardization of laparoscopic adrenalectomy in adrenal diseases, videos uploaded to the online video platform are gradually increasing. This study is the first one to evaluate the quality of the visual content of videos in the online video sites about laparoscopic adrenalectomy in the training of surgeons.

The presented research showed that attentively chosen YTb content considering its medical and technical quality on laparoscopic adrenalectomy is almost as accurate as WS content.

Keelan et al. was the first researcher focused on the accuracy and quality of visual content in the online world.¹⁷ Since this early research, many studies focused on the quality and correctness of medical content aired on YTb. Most of the investigations also revealed that this online content's quality and accuracy were unsatisfactory.^18–23

The insufficient information about the references and sources of these visual and auditory content in YTb and the absence of a professional/scientific analysis of the data's accuracy pushed researchers to suspect the educational proficiency of YTb content.^24,25 Another common point of previous studies is that YTb users more prefer less reliable visual content. In addition, the open-access policy of YTb makes it challenging to achieve the desired quality and accuracy level.^2,6,26,27

Similar to the other studies,^26,28 VPI scores of the medical videos on YTb was lower than those of the other sourced videos (P < .001). The reason might be the other sourced videos being more popular on social media. DSCq, JAMA-S, GqSc, and LASS median scores of video contents were significantly higher in terms of the medical videos (P < .001 in all comparisons). In addition, DSCq, JAMA-S, GqSc, and LASS median scores of the medical videos focusing on the surgical technique were higher than other medical videos (P < .001 in all comparisons).

In this study, in the comparison of the two websites regarding the view, like, and dislike mean and median values, no significant difference was found (P = .597, P = .335, P = .076, respectively). The LASS scores of WS and YTb content on laparoscopic adrenalectomy were 7.4/18 and 8.2/18, respectively (P = .34). The close rates might be related to the fact that all the selected 10 WS and 9 out of 10 YTb videos with the highest LASS scores evaluated have been prepared by the academicians affiliated with a university. But the weak LASS scores in these two online video platforms are the indirect indicators that the data expected from YTb and WS is less than half.

In similarity with the outcomes of the presented research, Ferhatoglu et al. concluded in their study regarding the comparison of the two websites in sleeve gastrectomy that videos published on YTb and prepared by physicians affiliated with a university had a higher educative quality.² Various investigations have highlighted that online data announced by university affiliated medical doctors are more trustworthy.^2,6,25–32 The DSCq, JAMA-S, and GqSc scores of WS were higher than those of the YTb. We believe that the open-access nature of YTb makes it impossible to comply with the academic publishing standards completely, and the intellectual filters of WS create this difference. After previous analyses, we observed that the WS visual content on laparoscopic adrenalectomy is more trustworthy and academic than YTb videos. There may be several potential reasons why the videos on YTb are insufficient for being educative in terms of the quality of data, educational power, and the reliability and accuracy of the sources. First, it is evident that most of the video uploaders aim to present an actual surgery; this behavior for advertising purposes complicates to discover high-class complete practice videos. Furthermore, medical doctors provided less than half of the visual content, suggesting that the residual data may have been deceptive for educative purposes.

The open-access nature of YTb and the absence of an academic review process make it challenging to apply the identical techniques to WS during the evaluation. Unlike YTb, all content aired on WS is uploaded by the medical doctors after a logical filtration process. Considering the heterogeneity of the data providers of YTb, the application of the same evaluation criteria would not be plausible to analyze both YTb and WS data. This heterogeneity is the main shortcoming of our research. Aiming for the most popular laparoscopic adrenalectomy videos on WS can be regarded as an extra weakness; however, we intended to estimate the most-watched content by medical doctors/surgeons, not the visual content being evaluated having the few viewers. Investigating the nonpopular videos would not assist the idea of our research.

The main limitation of our study is that the scoring systems used are subjective scoring systems, although videos are evaluated by experienced endocrine surgeons. In addition, changing the keywords related to the subject on the website or adding new keywords may result with different sorting schemes with different results.

Conclusion

We concluded that the educational quality should be improved in both of the online video platforms, YTb and WS. WS videos filtered by a professional review system and frequently served by physicians affiliated with a university were below the anticipated quality. The other result of this research is that delicately chosen YTb content is almost as qualified as WS data. Notwithstanding all its limitations, the educational power of YTb should not be neglected. Both platforms' primary deficiencies are the absence of data regarding the pre- and postoperative periods.

Footnotes

Authors' Contributions

All authors have made substantial contributions to this article being submitted for publication. All authors critically reviewed the article and approved the final form.

Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

References

Schmidt

, Shi

, Sethna

. Use of streaming media (YouTube^®) as an educational tool for surgeons—A survey of AAFPRS members. JAMA Facial Plast Surg, 2016; 18:230–231.

Ferhatoglu

, Kartal

, Filiz

Aİ

, Kebudi

. Comparison of new era's education platforms, YouTube^® and WebSurg^®, in sleeve gastrectomy. Obes Surg, 2019; 29:3472–3477.

Web 2.0 [Internet]. San Francisco

: Wikimedia

Foundation

, Inc., 2015 [ cited

April 3

, 2014]. http://en.wikipedia.org/wiki/Web_2.0 (last accessed November 30, 2020).

Snelson

. YouTube across the disciplines: A review of the literature. MERLOT J Online Learn Teach, 2011; 7:159–169.

Pandey

, Patni

, Singh

, Sood

, Singh

. YouTube as a source of information on the H1N1 influenza pandemic. Am J Prev Med, 2010; 38:1–3.

Yilmaz Ferhatoglu

, Kudsioglu

. Evaluation of the reliability, utility, and quality of the information in cardiopulmonary resuscitation videos shared on open access video sharing platform YouTube. Australas Emerg Care, 2020; 23:211–216.

Sood

, Sarangi

, Pandey

, Murugiah

. YouTube as a source of information on kidney stone disease. Urology, 2011; 77:558–562.

Mutter

, Vix

, Dallemange

, Perretta

, Leroy

, Marescaux

. WebSurg: An innovative educational web site in minimally invasive surgery—Principles and results. Surg Innov, 2011; 18:8–14.

Mellon

, Sethi

, Sundaram

. Laparoscopic adrenalectomy: Surgical techniques. Indian J Urol, 2008; 24:583–589.

10.

Barry

, Marzouk

, Chulak-Oglu

, Bennett

, Tierney

, O'Keeffe

. Anatomy education for the YouTube generation. Anat Sci Educ, 2016; 9:90–96.

11.

Erdem

, Karaca

. Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube^®. Spine (Phila Pa 1976), 2018; 43:E1334–E1339.

12.

DISCERN. Quality Criteria for Consumer Health Information. www.discern.org.uk (last accessed October 30, 2012).

13.

Bernard

, Langille

, Hughes

, Rose

, Leddin

, Veldhuyzen van Zanten

. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol, 2007; 102:2070–2077.

14.

Silberg

, Lundberg

, Musacchio

. Assessing, controlling and assuring the quality of medical information on the internet: Caveant lector et viewor—Let the reader and viewer beware. JAMA, 1997; 277:1244–1245.

15.

Stefanidis

, Goldfarb

, Kercher

, Hope

, Richardson

, Fanelli

. Guidelines for the minimally invasive treatment of adrenal pathology, 2013. https://www.sages.org/publications/guidelines/guidelines-for-the-minimally-invasive-treatment-of-adrenal-pathology/ (last accessed October 15, 2020).

16.

Zeiger

, Thompson

, Duh

, Hamrahian

, Angelos

, Elaraj

, et al.; American Association of Clinical Endocrinologists; American Association of Endocrine Surgeons. American Association of Clinical Endocrinologists and American Association of Endocrine Surgeons Medical Guidelines for the Management of Adrenal Incidentalomas: Executive summary of recommendations. Endocr Pract, 2009; 15:450–453.

17.

Keelan

, Pavri-Garcia

, Tomlinson

, Wilson

. YouTube as a source of information on immunization: A content analysis. JAMA, 2007; 298:2482–2484.

18.

Madathil

, Rivera-Rodriguez

, Greenstein

, Gramopadhye

. Healthcare information on YouTube: A systematic review. Health Informatics J, 2015; 21:173–194.

19.

Pathak

, Poudel

, Karmacharya

, Pathak

, Aryal

, Mahmood

, et al. YouTube as a source of information on Ebola virus disease. N Am J Med Sci, 2015; 7:306–309.

20.

Erdem

, Sislik

. The reliability of bariatric surgery videos in YouTube platform. Obes Surg, 2018; 28:712–716.

21.

Akgun

, Karabay

, Kocabay

, Kalayci

, Oduncu

, Guler

, et al. Learning electrocardiogram on YouTube: How useful is it?. J Electrocardiol, 2014; 47:113–117.

22.

Nason

, Kelly

, Burke

, Aslam

, Giri

, et al. YouTube as an educational tool regarding male urethral catheterization. Scand J Urol, 2015; 49:189–192.

23.

Basch

, Zybert

, Reeves

, Basch

. What do popular YouTube videos say about vaccines?. Child Care Health Dev, 2017; 43:499–503.

24.

Hayanga

, Kaiser

. Medical information on YouTube. JAMA, 2008; 299:1424–1425.

25.

Bezner

, Hodgman

, Diesen

, Clayton

, Minkes

, Langer

, et al. Pediatric surgery on YouTube™: Is the truth out there?. J Ped Surg, 2014; 49:586–589.

26.

Ferhatoglu

, Kartal

, Ekici

, Gurkan

. Evaluation of the reliability, utility, and quality of the information in sleeve gastrectomy videos shared on open access video sharing platform YouTube. Obes Surg, 2019; 29:1477–1484.

27.

Mangan

, Cakir

, Yurttaser Ocak

, Tekcan

, Balci

, Ozcelik Kose

. Analysis of the quality, reliability, and popularity of information on strabismus on YouTube. Strabismus, 2020; 28:175–180.

28.

Citgez

, Aygün

, Yigit

, Uludag

. Evaluation of the reliability, utility, and quality of the information in transoral endoscopic thyroidectomy vestibular approach videos shared on open access video sharing platform YouTube. J Laparoendosc Adv Surg Tech A, 2021; 31:1279–1285.

29.

Sorensen

, Pusz

, Brietzke

. YouTube^® as an information source for pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr Otorhinolaryngol, 2014; 78:65–70.

30.

Lee

, Seo

, Hong

. YouTube^® as a source of patient information on gallstone disease. World J Gastroenterol, 2014; 20:4066–4070.

31.

Adorisio

, Silveri

, De Peppo

, Ceriati

, Marchetti

, De Goyet Jde

. YouTube and pediatric surgery. What is the danger for parents?. Eur J Pediatr Surg, 2015; 25:203–205.

32.

Sahin

, Sahin

, Schwenter

, Sebajang

. YouTube^® videos as a source of information on colorectal cancer: What do our patients learn?. J Cancer Educ, 2019; 34:1160–1166.