The quality and reliability of short videos about amblyopia on Bilibili and TikTok: A cross-sectional study

Data sources and search strategy

This study employed a cross-sectional design to evaluate the quality and reliability of amblyopia-related short videos on two major Chinese platforms, Bilibili (https://www.bilibili.com) and TikTok (https://www.douyin.com). On October 1, 2025, the Chinese keyword “弱视” (amblyopia) was used as the primary search term. To minimize bias from personalized recommendation algorithms, all searches were conducted using newly created accounts in incognito mode to ensure that historical data or personalization did not influence the results. At the initial retrieval stage, the top 150 videos from the default search results on each platform were collected.

Inclusion and exclusion criteria

Inclusion criteria: Videos directly related to amblyopia, including information on etiology, clinical features, diagnosis, treatment, or prevention. Exclusion criteria: 1) Duplicate content or re-uploaded videos; 2) Videos unrelated to amblyopia or those that merely mention the condition without providing substantial information; 3) Advertisements or promotional material; 4) Private or inaccessible videos.

Data extraction and classification

For each video meeting the inclusion criteria, the following variables were systematically extracted: platform source (Bilibili or TikTok), upload time, video duration (in seconds), likes, comments, shares, and collections. Video content was categorized into etiology, symptoms, diagnosis, treatment, and prevention. The etiology of amblyopia refers to the common causes, such as refractive errors, strabismus, and congenital conditions. ¹⁶ The symptoms involve the main clinical features, including visual impairment, differences in vision between the eyes, and strabismus. ¹⁷ Diagnosis includes the techniques used to detect amblyopia, such as visual acuity tests and cover tests. ¹⁸ Treatment focuses on the methods available to manage amblyopia, including corrective lenses, patch therapy, and vision training. ¹⁹ Prevention emphasizes early screening and intervention strategies to identify and address amblyopia in children, aiming to prevent its progression. ¹ Uploaders were grouped as follows: specialist group (ophthalmologists and related medical professionals), non-specialist healthcare practitioner group (healthcare providers from other disciplines), and individual user group.

Video quality assessment

To assess the reliability and quality of the collected short videos, the Global Quality Score (GQS) ²⁰ and the modified DISCERN (mDISCERN) ²¹ tool were used. The GQS used a 5-point Likert scale to assess the overall quality of the videos, with scores ranging from 1 (very poor) to 5 (very good). This scale considered aspects such as the professionalism of the video, the comprehensiveness of the information, clarity of presentation, and the viewers’ ability to understand the content. The mDISCERN tool was employed to evaluate whether each video met the following criteria: clarity, relevance, traceability, robustness, and fairness. Each criterion was scored as “yes” (1 point) or “no” (0 points), with a cumulative score ranging from 0 to 5 points. The detailed criteria for these two scales are presented in Tables 1 and 2. Both GQS and mDISCERN are general video quality assessment tools. They effectively evaluate the overall quality, reliability, clarity, and comprehensibility of health information, supporting their suitability for assessing the included amblyopia-related videos. All assessments were conducted independently by two assessors with an ophthalmology background. Before evaluation, both received standardized training to unify criteria and minimize bias. The training covered detailed scoring rules for core dimensions, including clinical features, etiology, diagnosis, treatment, and prevention, as well as guidance to reduce subjective bias. Regular calibration was performed during the evaluation process. Discrepancies were resolved by consulting a third senior evaluator. Inter-rater reliability was assessed using Cohen’s Kappa coefficient.

OPEN IN VIEWER

Table 1.

The Global Quality Score (GQS) quality criteria.

Item features	Points
Poor quality; poor flow of the videos; most information missing; not at all useful for patients	1
Generally poor quality; some information listed, but many important topics missing; of very limited use to patients	2
Moderate quality; suboptimal flow; some important adequately discussed, but other information poorly discussed; somewhat useful for patients	3
Good quality and generally good flow; most of the relevant information listed, but some topics not covered; useful for patients	4
Excellent quality and flow; very useful for patients	5

OPEN IN VIEWER

Table 2.

The modified DISCERN (mDISCERN) quality criteria.

Reliability score
1. Is the video clear, concise, and understandable?
2. Are valid sources cited?
3. Is the content presented balanced and unbiased?
4. Are additional sources of content listed for patient reference?
5. Are areas of uncertainty mentioned?

Statistical analysis

Descriptive statistics were used to summarize basic video characteristics. Non-normally distributed data are presented as median and interquartile range (IQR), and categorical variables as frequencies and percentages. Two group comparisons were conducted using the Mann–Whitney U test, and comparisons among three or more groups were performed with the Kruskal–Wallis H test. Spearman’s rank correlation coefficient was used to assess associations between video quality scores (GQS and mDISCERN) and engagement metrics (likes, comments, shares, and collections). A two-sided p < 0.05 was considered statistically significant. All statistical analyses and plotting were performed in R version 4.3.2.

Video characteristics

After applying the exclusion criteria, a total of 199 videos meeting the inclusion criteria were analyzed, including 100 from Bilibili and 99 from TikTok. The detailed selection process is shown in Figure 1, and the video characteristics are summarized in Table 3. The median engagement metrics were as follows: likes 91.00 (IQR: 16.50–557.50), comments 19.00 (IQR: 3.00–184.00), collections 86.00 (IQR: 14.00–344.00), and shares 37.00 (IQR: 4.25–204.75). The median video duration was 134.00 seconds (IQR: 67.00–307.50). Regarding video quality, the median GQS score was 3.00 (IQR: 2.00–3.00), and the median mDISCERN score was also 3.00 (IQR: 2.00–3.00). Inter-rater consistency was excellent, with Cohen’s Kappa values of 0.88 for GQS and 0.89 for modified DISCERN scores.OPEN IN VIEWER

OPEN IN VIEWER

Table 3.

General characteristics, quality, and reliability of the videos.

Variables	Total (n = 199)
General information
Video length (s), M (Q1, Q3)	134.00 (67.00, 307.50)
Likes, M (Q1, Q3)	91.00 (16.50, 557.50)
Collections, M (Q1, Q3)	86.00 (14.00, 344.00)
Comments, M (Q1, Q3)	19.00 (3.00, 184.00)
Shares, M (Q1, Q3)	37.00 (4.25, 204.75)
Video content
Etiology	79.00 (39.70%)
Clinical features	127.00 (63.82%)
Diagnosis	84.00 (42.21%)
Treatment	176.00 (88.44%)
Prevention	81.00 (40.70%)
Video quality
GQS score, M (Q1, Q3)	3.00 (2.00, 3.00)
mDISCERN score, M (Q1, Q3)	3.00 (2.00, 3.00)

Video content

Treatment was the most frequently discussed topic, appearing in 176.00 videos (88.44%), followed by clinical features in 127.00 videos (63.82%). Etiology, diagnosis, and prevention were discussed in 79.00 (39.70%), 84.00 (42.21%), and 81.00 (40.70%) videos, respectively (Table 3).

Cross-platform comparison

There were significant differences in video duration and engagement metrics between the two platforms (Table 4). Bilibili was dominated by long-form videos, with a median duration of 302.00 seconds (IQR: 162.75–900.25), whereas TikTok featured shorter videos with a median duration of 74.00 seconds (IQR: 38.50–114.50). However, TikTok demonstrated considerably higher user engagement across likes, comments, collections, and shares. In terms of video content, both platforms showed relatively similar content coverage (Figure 2). Figure 3(a) and (b) show the comparisons of GQS and mDISCERN scores between the two platforms. There were no significant differences in either GQS or mDISCERN scores between the platforms (p > 0.05). The median GQS score on Bilibili was 3.00 (IQR: 2.00–3.00), and the median mDISCERN score was 3.00 (IQR: 2.00–3.00). Detailed parameters are presented in Table 4.

OPEN IN VIEWER

Table 4.

General information, quality, and reliability scores of amblyopia videos on TikTok and Bilibili.

Variables	Bilibili (n =100)	TikTok (n =99)	P
General information
Video length(s), M (Q1, Q3)	302.00 (162.75, 900.25)	74.00 (38.50, 114.50)	<.001
Likes, M (Q1, Q3)	22.00 (6.00, 95.00)	419.00 (81.50, 1221.00)	<.001
Collections, M (Q1, Q3)	38.00 (10.00, 166.00)	169.00 (46.00, 452.00)	<.001
Comments, M (Q1, Q3)	3.50 (1.00, 23.25)	126.00 (16.00, 296.50)	<.001
Shares, M (Q1, Q3)	10.00 (1.00, 43.50)	124.00 (31.50, 451.50)	<.001
Video content
Etiology	39.00 (39.00%)	40.00 (40.40%)	-
Clinical features	59.00 (59.00%)	68.00 (68.69%)	-
Diagnosis	39.00 (39.00%)	45.00 (45.45%)	-
Treatment	85.00 (85.00%)	91.00 (91.92%)	-
Prevention	24.00 (24.00%)	57.00 (57.58%)	-
Video quality
GQS score, M (Q1, Q3)	3.00 (2.00, 3.00)	2.00 (2.00, 3.50)	0.559
mDISCERN score, M (Q1, Q3)	3.00 (2.00, 3.00)	3.00 (2.00, 3.00)	0.852

OPEN IN VIEWER

Figure 2.

Information about amblyopia-related video content from TikTok and Bilibili.

OPEN IN VIEWER

Figure 3.

(a) Comparison of GQS scores between TikTok and Bilibili platforms. (b) Comparison of mDISCERN scores between TikTok and Bilibili platforms.

Comparison among different uploaders

Figure 4(a) shows the overall distribution of uploader categories. Specialists accounted for 34.67%, individual users for 43.72%, and non-specialists for 21.61%. Figure 4(b) compared the proportions of different uploader types on TikTok and Bilibili. Individual users made up the largest proportion of uploaders, followed by specialists, with non-specialists representing the smallest group. Both TikTok and Bilibili exhibited similar distribution patterns. By analyzing the relationship between video data and uploader category, we found no statistically significant differences in engagement metrics (likes, comments, collections, and shares) among videos uploaded by specialists, non-specialists, and individual users (Table 5). However, when analyzing video quality, we found that videos uploaded by specialists had significantly higher quality, with a median GQS score of 4.00 (IQR: 3.00–4.00) and a median mDISCERN score of 3.00 (IQR:3.00–4.00). Figure 5(a) illustrates the distribution of GQS scores across different uploader types. Specialists scored significantly higher than other uploader groups in both GQS (p < 0.05). Figure 5(b) shows the distribution of mDISCERN scores. In addition, specialists also achieved the highest scores on the mDISCERN scale (p < 0.05).OPEN IN VIEWER

OPEN IN VIEWER

Table 5.

Characteristics, quality, and reliability of amblyopia videos by different uploaders on TikTok and Bilibili.

Variables	Non-specialists (n = 43)	Specialists (n = 69)	Individual users (n = 87)	P
Video length (s), M (Q1, Q3)	167.00 (63.00, 302.00)	99.00 (64.00, 330.00)	143.00 (80.50, 278.50)	0.709
Likes, M (Q1, Q3)	94.00 (12.00, 1171.50)	76.00 (25.00, 500.00)	97.00 (13.00, 419.50)	0.744
Collections, M (Q1, Q3)	129.00 (10.00, 681.50)	111.00 (19.00, 297.00)	64.00 (15.50, 264.50)	0.659
Comments, M (Q1, Q3)	14.00 (3.00, 162.00)	18.00 (3.00, 126.00)	31.00 (2.50, 265.00)	0.346
Shares, M (Q1, Q3)	71.00 (4.50, 247.00)	41.00 (8.00, 205.00)	25.50 (3.25, 136.50)	0.439
GQS score, M (Q1, Q3)	2.00 (2.00, 3.00)	4.00 (3.00, 4.00)	2.00 (2.00, 3.00)	<.001
mDISCERN score, M (Q1, Q3)	2.00 (2.00, 3.00)	3.00 (3.00, 4.00)	2.00 (2.00, 3.00)	<.001

OPEN IN VIEWER

Figure 5.

(a) Violin plots illustrating the distribution of GQS scores among different uploader groups. (b) Violin plots illustrating the distribution of mDISCERN scores among different uploader groups.

Correlation analysis between video features and quality

Figure 6 presents the correlation analysis between video characteristics and quality metrics. A strong positive correlation was found between GQS and mDISCERN scores (p < 0.001), indicating a high level of consistency between the two quality assessment tools. Video duration showed significant positive correlations with both quality scores, with a stronger association with GQS (r = 0.32) than with mDISCERN (r = 0.25). However, none of the engagement metrics (likes, comments, collections, shares) showed significant correlations with either GQS or mDISCERN scores(p > 0.05).OPEN IN VIEWER

Systemic gaps in content coverage

Analysis of amblyopia-related video content revealed a clear imbalance in topic coverage. Treatment was the most frequently discussed theme, whereas coverage of etiology, diagnosis, and prevention was notably insufficient. Most videos provided only brief introductions to rehabilitation methods, lacking in-depth explanations of pathogenesis, standardized diagnostic criteria, and long-term outcomes. This imbalance may result from creators’ tendency to focus on content that is practical and easy to present, while giving less attention to more abstract or complex topics such as etiology, diagnosis, and prevention. Furthermore, the limited duration of short videos may contribute to selective presentation of content. ²² Such systematic information gaps may pose serious public health risks: insufficient knowledge of etiology can delay early treatment; inadequate understanding of diagnostic criteria may result in misinterpretation of screening results, missing optimal intervention opportunities; and lack of prognostic information can reduce treatment adherence and rehabilitation effectiveness, potentially causing strabismus, visual impairment, or loss of stereopsis.^23–25 Previous studies have shown that the completeness of health information is positively associated with public disease awareness and treatment adherence. ²⁶ Based on these findings, platforms are encouraged to establish systematic content review standards and promote the production of educational videos that comprehensively cover the entire disease process, including prevention, etiology, diagnosis, treatment, and rehabilitation, to ensure the thorough dissemination of essential medical information. In addition, short video creators should prioritize diagnostic and preventive content to improve the scientific accuracy and completeness of their videos.

Platform characteristics and content quality

This study identified significant differences in the dissemination of amblyopia-related content between Bilibili and TikTok. Bilibili predominantly features long-form videos, which are conducive to systematic explanations, yet exhibit relatively low levels of user interaction. Conversely, TikTok’s shorter, more concise, and fast-paced videos elicit substantially higher user engagement across metrics such as likes, comments, favorites, and shares. Notably, both platforms demonstrate comparable levels of scientific reliability and overall content quality, generally at a moderate level. ²⁷ These differences appear to stem primarily from variations in platform content mechanisms, user demographics, and modes of content presentation. To enhance the effectiveness of amblyopia-related science communication, it is recommended that platforms optimize content formats, for instance, by incorporating chapter segmentation or interactive elements in long-form videos and providing appropriate scientific substantiation in short form videos. ²⁸ Furthermore, content creators should be encouraged to translate complex concepts into accessible and comprehensible formats while maintaining scientific rigor, thereby leveraging the high interactivity characteristic of short videos without compromising informational completeness.

Deterministic influence of uploader professional background

Analysis of uploader composition revealed a similar distribution pattern of amblyopia-related video creators across both platforms, with individual users accounting for the largest proportion (43.72%), followed by medical specialists (34.67%) and non-specialists (21.61%). Quality assessment demonstrated significant differences in content quality among creators with varying professional backgrounds. Videos produced by specialists achieved the highest median GQS (4.00) and mDISCERN (3.00) scores, underscoring the decisive influence of professional qualifications on content reliability. This finding aligns with previous studies on diseases such as monkeypox ²⁹ and chronic obstructive pulmonary disease, ³⁰ which reported that healthcare professionals adhering to evidence-based principles are more likely to produce scientifically standardized and credible content. The disparity in content quality may primarily stem from non-professional creators’ limited medical knowledge and their tendency to simplify or sensationalize information to attract viewer engagement, thereby compromising scientific accuracy. ³¹ Nevertheless, even specialist-generated videos exhibited only moderate overall quality (median GQS and mDISCERN = 3.00), indicating substantial room for improvement in professional health communication. Therefore, it is recommended that platforms strengthen medical content review and professional verification systems, implement algorithmic mechanisms to promote high quality evidence-based materials, and encourage broader participation of healthcare professionals in short video production. Meanwhile, viewers should be guided to prioritize medically verified sources and remain cautious when interpreting information from non-professional creators. ³²

Analysis of the correlation between video metrics and scientific content

Notably, correlation analysis revealed strong positive relationships among the engagement metrics, indicating that videos performing well in one dimension of interaction tend to perform well across other dimensions, which is consistent with previous research.^27,33 However, high levels of engagement do not necessarily correspond to increased scientific rigor or content quality. This may be attributed to users’ preference for videos that are entertaining or easily understandable rather than those with the highest scientific accuracy. ³² Therefore, it is recommended that platforms incorporate professional content quality assessment mechanisms into their recommendation algorithms and establish medical content review and traffic support systems to promote the dissemination of high quality science communication. At the same time, users should be fully aware that high engagement does not equate to high quality or highly reliable medical information. ³⁴

Although the videos included in this study were designed for the general public and primarily educational, some medical terminology was inevitably used. ³⁵ Such terms are relatively understandable to professionals with an ophthalmic background but may pose comprehension challenges for patients without medical training, particularly for concepts such as “refractive error” and “amblyopia”. ³⁶ To enhance public accessibility and educational effectiveness, we strongly recommend that video creators use plain language and provide clear explanations when introducing medical terms, thereby helping patients accurately understand and utilize the health information.^37–39

This study systematically evaluated amblyopia-related short videos on Bilibili and TikTok, revealing that platform characteristics, uploaders professionalism, and content focus significantly affect the quality of health information. Videos produced by medical professionals demonstrated higher reliability, whereas content uploaded by non-professionals lacked scientific rigor,^34,40 although overall quality still has room for improvement. ⁴¹ Platform differences were evident, with Bilibili videos being longer and TikTok videos exhibiting higher user engagement,^33,42 yet overall content quality was comparable. Content analysis identified systematic gaps, including insufficient coverage of etiology, diagnosis, and prevention,^43,44 while treatment-related information accounted for the largest proportion. These findings provide empirical support for optimizing health information dissemination strategies.

This study has several limitations. First, this study focused solely on domestic platforms, which limits the external generalizability of the findings. Additionally, most of the videos are in Chinese, which may hinder understanding for non-Chinese-speaking readers. Second, although validated scoring scales were used, the evaluation process was inherently subjective, as the observers were not blinded to the uploader’s identity, which may have introduced bias. Third, although validated general quality scoring systems were used, these tools were not specifically designed for amblyopia-related content, which represents a limitation. The study also relied on a fixed set of search terms, which may restrict the comprehensiveness of retrieved videos and limit broader applicability. Future studies should include more diverse and commonly used search terms, such as lay expressions equivalent to “lazy eye” in different languages, and may benefit from developing more disease-specific scoring criteria. Furthermore, the ophthalmic background of the assessors may have introduced potential bias into the evaluation. Future studies could employ dedicated evaluation tools that explicitly assess plain-language use, avoidance of excessive medical jargon, and public understandability, thereby better ensuring health information is accessible and comprehensible to the general population. Finally, as a cross-sectional study, it does not capture the dynamic changes in video quality and reliability over time.