Abstract
BACKGROUND:
Videos uploaded to YouTube do not go through a review process. The educational aspect of these videos may be insufficient for patellofemoral pain syndrome (PFP).
OBJECTIVE:
To examine the reliability and educational quality of PFP videos on YouTube.
METHODS:
A standard search was performed in the YouTube database using the following terms: patellofemoral pain syndrome/anterior knee pain syndrome/anterior knee pain/patellofemoral pain. For each search term, the top 50 videos based on “relevance” assignment of YouTube’s algorithm were included in the examination. The remaining 96 videos after exclusion were included in the study. The educational quality and reliability of videos was analyzed using DISCERN, JAMA (The criteria of Journal of the American Medical Association), GQS (Global Quality Score) and PFPSS (Patellofemoral Pain Specific Score).
RESULTS:
According to PFPSS, 81.2% of the videos were evaluated as low and very low quality. According to the DISCERN classification, 74.9% of the videos were evaluated as poor and very poor. According to GQS, 59.4% of the videos had scores of 2 or less, which were considered poor quality. According to JAMA, 41.7% of the videos scored 2 and below.
CONCLUSIONS:
The information content of YouTube videos is inadequate. Video design should be created to be understandable by patients and to attract their attention while making these videos.
Introduction
The term patellofemoral pain syndrome (PFP) is commonly used to describe the condition of anterior knee pain. Patellofemoral pain syndrome is the most commonly diagnosed condition in patients under 50 years of age with knee complaints [1].
Inclusion and exclusion criteria.
YouTube is one of the most popular online social platforms where 30 million active users and 5 billion videos are watched daily [2]. As the availability of the Internet increases, the number of patients who use online videos as a source of information about their disease or to search for possible treatment options also increases [2]. However, since there is no referee evaluation, the accuracy and reliability of content uploaded to this platform can be questioned [3]. There are many studies on the reliability of these videos in orthopedic diseases due to the possibility that they may affect the relationship between the patient and the physician. In these studies, YouTube videos about hallux valgus, distal biceps tendon rupture, rotator cuff tears, developmental dysplasia of the hip, femoroacetabular impingement, shoulder instability, low back pain, carpal tunnel syndrome surgery and total hip replacement were evaluated. In these studies, videos were reported as insufficient resources to inform patients [2, 4, 5, 6, 7, 8, 9, 10, 11, 12].
Multimodal interventions have the strongest evidence in the long-term management of PFP. It usually consists of quadriceps and gluteal exercises, stretching, massage, patellar mobilization and patellar kinesio taping. Adapting this multimodal approach with appropriate patient education is essential for effective PFP management [13]. Providing patient education in clinical practice is an important issue. When the information is provided verbally only, the retention of the information becomes weak [13]. It has been shown that the use of a published educational brochure may be insufficient, as education is key in the treatment of PFP. To fill this gap, the use of a multimedia website (more guidance on useful exercises, ideally in video format) can help guide treatment about the management and prognosis of the disease [14]. A website containing a comprehensive education in various multimedia formats to facilitate self management of PFP may be effective in reducing pain, pain catastrophising and kinesiophobia; and improving physical function and quality of life in people with PFP. Providing a comprehensive multimedia information portal may greatly improve access to appropriate and effective education and selfmanagement strategies for people with PFP [15]. When all these are taken into consideration, it can be concluded that patients can be educated about PFP through Youtube videos and can use it in the self-management of the disease.
Although PFP is one of the very common outpatient diseases, there is still no study on the reliability of YouTube videos in this regard. In this study, we aimed to examine the reliability of PFP videos on YouTube. As with other orthopedic diseases, we hypothesized that these videos were inadequate in informing the patient.
Patellofemoral pain syndrome specific score (PFPSS)
Patellofemoral pain syndrome specific score (PFPSS)
On May 8, 2021, using Google Chrome (version 92.0.4515.159 – 64 bit) with cache cleared and cookies deleted, a standard search was performed on the YouTube database using the following terms: “patellofemoral pain syndrome”, “anterior knee pain syndrome”, “anterior knee pain” and “patellofemoral pain”. For each search term, the top 50 videos based on “relevance” assignment of YouTube’s algorithm were included in the examination (200 videos in total) [16, 17]. Videos of more than one episode were considered as one piece and there was no limit to the length of the videos. A YouTube™ account has been created for study purposes and all received video links have been listed following removal of copies. After 104 videos were excluded according to inclusion and exclusion criteria, 96 videos were included in the study (Fig. 1).
The following video features are listed for each included YouTube video: (1) title, (2) video duration, (3) views, (4) video source/uploader, (5) content type, (6) number of days since upload, (7) view rate (views/day), (8) likes, (9) dislikes, (10) likes rate (like
For PFP-related information, we have established an assessment criterion (referred to as the “PFP – Specific Score” or PFPSS) to better assess the educational content. PFPSS consists of 20 items based on the guidelines published by the American Academy of Orthopedic Surgeons, a method that has been proven to be applicable in the literature [16, 18]. Orthoinfo is a website created by the American Academy of Orthopedic Surgeons. The purpose of this site is to inform patients about diseases of the musculoskeletal system and their treatments. Considering this website’s patient information about PFP, PFP was analyzed under five headings, scored as sub-items and named PFPSS. PFPSS specifically evaluates information about (1) patient presentation and symptoms, (2) general information about PFP, (3) diagnosis and evaluation, and (4) treatment options (Table 1). The maximum possible score is 25, with a higher score indicating better quality of PFP special education. This scoring system was grouped as very high (25–21 points), high (20–16 points), moderate (15–11 points), low (10–6 points), and very low (5–0 points) quality [17].
Other scoring systems
The criteria of Journal of the American Medical Association (JAMA) were used to assess video accuracy and reliability [19]. This system consists of 4 separate criteria, each of which is assigned a score of 1, and provides a non-specific assessment of source reliability. A score of 4 indicates higher source accuracy and reliability, while a score of 0 indicates poor source accuracy and reliability. Although not validated, these criteria have previously been widely used in the literature to assess the reliability of online resources [16, 17].
Three different scoring systems were used to assess the informative value of PFP videos. The Global Quality Score (GQS) provides a non-specific assessment of educational value through 5 criteria [17, 20]. GQS has a maximum score of 5 indicating high quality of education. The DISCERN score was developed in Oxford, United Kingdom to assess written health information. The original DISCERN consists of 16 questions scored from 1 to 5 and total scores ranging from 6 to 80; higher scores indicate better quality (Table 4) [21]. According to the score ranges obtained in discern; “very poor” (16–28 points), “poor” (29–41 points), “fair” (42–54 points), “good” (55–67 points), and “excellent” (68–80 points) were evaluated.
DISCERN was developed from 1996 to 1997 by the British Library at the University of Oxford and the NHS Executive Research and Development Program in the United Kingdom [22]. The tool was created in recognition of the need for a general set of quality criteria for written consumer health information, not video evaluation [23]. Another tool, JAMA, was created to evaluate online websites, not videos [23]. Although these two scoring systems have been used in studies evaluating YouTube videos; They may not be sufficient for the evaluation of YouTube videos [23]. The Global Quality Score (GQS) was defined by Bernard et al. This scoring system was used to evaluate the educational value of each video [24]. PFPSS was designed for this study. However, its success and continuity in evaluating youtube videos has not yet been investigated. Considering all these, all 4 scoring systems were used in the evaluation of PFP videos. Thus, it was aimed to reduce the false and/or incomplete results that may arise due to the use of a single scoring system. At the same time, the correlation between the results obtained according to these scoring systems was evaluated statistically. Thus, while evaluating PFP videos, it was aimed to observe whether the scoring systems showed similar features. Even if the quality of the videos is high compared to the scoring, the number of views and likes of the patients may differ, so the correlation between the parameters of the videos and the scoring was also evaluated. Moreover, since the number of views of the videos may reflect the preferences of the video viewer, the correlation between the video features and the scoring was evaluated. Accordingly, there is a statistically significant positive weak relationship between GQS and JAMA (
The videos included in the study were determined by the non-observer author. The links to these videos were tabularized and presented to the observers. Observers were trained on scoring prior to evaluation. The evaluation and scoring of the videos according to DISCERN, GQS, JAMA, and PFPSS were performed blindly by two observers. Interobserver agreement was evaluated as very good in terms of JAMA (ICC: 0.890), GQS (ICC: 0.898), DISCERN (ICC: 0.995) and PFPSS (ICC:0.988) (
Statistical analysis
Frequency distribution of video source and content
Frequency distribution of video source and content
Descriptive statistics on obtained data
Examination of the correlation between JAMA, GQS, DISCERN, PFPSS and video duration, number of views, number of days after upload, number of likes, like rate, VPI values
Data were analyzed with IBM SPSS Statistics, version 20 (SPSS, Chicago, IL, USA). Categorical variables are presented as relative frequencies with percentages. One-way analysis of variance (ANOVA) tests (for normally distributed data) and Kruskal-Wallis tests (for non-normally distributed data) were used to determine whether the video reliability and quality differed based on video source and video content. The Mann-Whitney U test was used to determine the group causing the difference. For video quality assessment, agreement between the two reviewers was analyzed using the Interclass Correlation Coefficient (ICC) followed by the 95% confidence interval reported within parentheses. ICC values less than 0.5 were categorized as poor reliability, values between 0.5 and 0.75 as moderate reliability, values between 0.75 and 0.9 as good reliability, and values above 0.90 as excellent reliability. Spearman’s rank correlation coefficient was used to analyze the relationships between usefulness scores generated for each video and their corresponding technical characteristics. Categorical data were reported as frequencies and relative frequencies, while continuous data were reported as means. Where applicable standard deviations follow in parentheses. Numerical results were rounded to one decimal place.
Comparison of JAMA, GQS, DISCERN and PFPSS values by video source
Comparison of JAMA, GQS, DISCERN and PFPSS values by video source
The distribution of videos by video source/uploader and content type is summarized in Table 2. Quantitative data and descriptive statistics of the videos included in the study are summarized in Table 3. According to the DISCERN classification, 27 (28.1%) of the videos were very poor, 45 (46.9%) were poor, 19 (19.8%) were fair, 4 (4.1%) were good, and 1 (1.1%) was excellent. According to PFPSS, 46 (47.9%) of the videos were evaluated as very low quality, 32 (33.3%) as low quality, 14 (14.6%) as moderate, 4 (2.1%) as high quality. According to GQS, the videos scored 2 and below at a rate of 59.4%. According to JAMA, 41.7% of the videos scored 2 and below.
The statistical relationship between the quantitative data of the videos and JAMA, GQS, DISCERN and PFPSS is summarized in Table 4. Accordingly, there is a statistically significant positive weak relationship between video duration and GQS (
There was no statistical difference between the content type of the videos regarding DISCERN (
The important findings of this study were that YouTube videos about PFP had insufficient quality and education (JAMA mean: 2.67, DISCERN mean: 34.89, GQS mean: 2.42, PFPSS mean: 6.11). At the same time, the positive correlation between the video duration and GQS, DISCERN and PFPSS can be interpreted in the direction of increasing the instructional time as the video duration increases. According to PFPSS, %81.2 of the videos were evaluated as low and very low quality. According to the DISCERN classification %74.9 of the videos were evaluated as poor and very poor. According to GQS, 59.4% of videos had scores of 2 or less, which were considered poor quality. According to JAMA, 41.7% of the videos scored 2 and below. At the same time, GQS, DISCERN and PFPSS scores of the videos shared by those who were physicians were significantly higher than those who were non-physicians. On the contrary, the video posts of non-physicians received a statistically significant like. This may be because physicians use more scientific and boring language for patients. Physicians’ use of a more understandable language in videos for patients and the use of a non-boring and patient-oriented narration may provide a solution to this situation.
Video-based information is expected to become the main source of information for patients in the near future [2, 25]. For this reason, doctors should be aware of this inadequacy in the applications of patients who have been informed in advance or who have received self-education [2]. Social media is increasingly used by patients to investigate their health status, and there are many medical-oriented videos on YouTube [4]. Therefore, the quality and instructiveness of YouTube videos, one of the social media platforms associated with patient education, have been investigated in the literature [5, 6, 7, 8, 26]. It has been found in these studies that youtube videos about rotator cuff, hip dysplasia, shoulder instability and femoroacetabular impingement do not have sufficient information content. Similar to these studies, the educational quality of the PFP videos on YouTube was insufficient. Doctors, in order to prevent patients from being misguided when sharing about diseases; first, it should give general information about the disease, the second should present the treatment options, and the third should indicate the place of the treatment described in the video in the treatment scheme, the fourth is to inform that the treatment mentioned in the video is not the only option and the treatment may vary according to the patient.
Healthcare professionals and organizations should be more careful when recording videos and uploading them to online platforms. Since these videos can reach a wide variety of audience, their content should provide more information about the possible complications of a treatment and other treatment methods [10]. Although 46.9% of the videos in this study included information about the disease, more than half of the video uploaders were non-physicians. Even in the videos shared by the physicians, the highest scores were 8.98 for PFPSS and 41.38 for DISCERN. If a video uploader has posted a video to describe an isolated topic related to PFP (e.g., related to surgical treatment, exercises, or complaints) it may provide sufficient information about part of the topic. However, they who mentioned part of the topic can get a low score from this scoring system. For this reason, the results we obtained should not be interpreted as the videos giving false information. However, the fact that patients have partial information about PFP and are unaware of some treatments and complications may adversely affect the physician-patient joint decision-making process. This situation can be interpreted as the necessity of preparing the videos by official institutions or resources that can provide high quality and adequate information on this subject and making them available to the patients as part of patient’ education.
In previous studies, the scores of the videos uploaded by physicians were higher than those who were non-physicians [2, 26, 27]. In other studies, it has been reported that videos provided by physicians are less popular and their content is very complex or less attractive to patients [28]. The results of our study were similar to these studies. This may be since the videos uploaded by physicians are more scientific and difficult for patients to understand. Patients may prefer videos that are more understandable and not boring. From these findings, it can be concluded that while the information content of the videos is reached to a sufficient level, they should be prepared in a format that the patients can understand and that is not boring.
In a study by Hatipoğlu and Gaş, a positive correlation was reported between video duration and likes [30]. In this study, there is a positive relationship between video duration and video quality. However, there was no correlation between liking and video duration. These findings can be interpreted as an increase in video duration as a result of the fact that the videos related to PFP contain certain information.
The shortcomings of this study can be listed as follows: 1- YouTube videos are presented by artificial intelligence. For this reason, the videos we encounter during the scan may not reflect the most viewed videos by the patients. On the other hand, in order to reduce the possibility of this happening, we searched by deleting cookies and using normal search. 2- We searched for certain keywords selected by us. Different keywords used by patients can also be found. This may not fully reflect the videos that patients encounter in their searches. 3- Only the first 50 videos were evaluated for each keyword. Again, this can be interpreted as only a part of the video pool being evaluated. However, the same method was used in many studies [16, 17]. It was stated that many of the viewers watched the first 50 videos more frequently. 4- PFPSS has not been used in the literature before and has been designed by us for this study. Therefore, there is no study on the reliability of this scoring. However, there was a positive correlation between PFPSS and GQS and DISCERN, as well as excellent interobserver agreement. Considering this, we believe that PFPSS is a scoring with good interobserver agreement and adequate evaluation of patellofemoral pain syndrome. However, the success of PFPSS as a scoring system in the evaluation of videos related to PFP patients can be demonstrated by new studies. 5- YouTube is a dynamic platform where new videos are constantly uploaded. For this reason, the videos evaluated in this study reflect the video content at a certain time. While a video containing only surgical treatment for medical professionals contains a lot of medical information about surgery, it is likely to get insufficient scores from the scoring. However, this study researched the informativeness of videos as a source of patient information. Therefore, this deficiency will not affect the results of the research. However, it should not be deduced from these results that these videos give false information.
This study evaluates a final score of the information presented regarding PFP. As a result, videos that evaluated part of the PFP were rated low. When the videos included in the study are evaluated, it can be concluded that the information tried to be given is not wrong, but incomplete. It was explained in many videos, focusing mainly on a single treatment method. This resulted in insufficient scores in the scoring. It is not wrong for other branches of medicine to describe specific treatments about PFP. However, this may lead to the perception that it is the only or the best treatment for the patient. For this reason, it is the duty of health professionals to mention that there are other treatments in the disease, to be informed that the treatment to be applied may change, even if information about only a treatment is given. The findings of this study can be interpreted as follows: 1) Patient training videos for PFP should be prepared by associations or universities in order not to adversely affect the physician-patient relationship, 2) Patients who request resources should be directed to these videos by physicians, 3) Videos should be modified at certain intervals considering the feedback of the patients and the current literature, and 4) Visual aid should be used as much as possible, language should contains terms as less as possible to prevent patients to be bored while watching these videos.
Conclusion
The information content of YouTube videos is inadequate. Videos evaluating the disease as a whole should be prepared by competent associations/institutions and made available to patients. The design should be created to be understandable by patients, to attract their attention and high quality while making evaluating and treating videos.
Footnotes
Acknowledgments
The authors have no acknowledgments.
Conflict of interest
The authors declare that they have no conflict of interest.
