Abstract
Background
The frailty syndrome is among the most prevalent geriatric syndromes, while social media has become a pivotal place for retrieving health information.
Objective
The objectives of this study are to investigate the quality of frailty-related videos on major Chinese social media platforms and examine the correlation of the quality with user engagement.
Methods
Collect the videos about frailty from TikTok, Bilibili, and Xiaohongshu. Document the general characteristics, uploader information, and content features of each video. Evaluate the quality of each video with the Journal of the American Medical Association (JAMA) benchmark criteria, the Global Quality Score (GQS), modified DISCERN (mDISCERN), and the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V).
Results
We examined 126 videos in current study. Overall, quality was not promising with a mean JAMA score of 1.1 (SD=0.8), GQS of 2.8 (SD=1.0), mDISCERN of 3.0 (SD=0.8), PEMAT-understandability of 76.5% (SD=15.6%), and PEMAT-actionability of 49.7% (SD=40.2%). Among the platforms, Bilibili had the highest quality videos, and Xiaohongshu had the lowest videos quality. Videos produced by organizations, non-profit groups, medical-related personnel, certified authors, expert monologue, and question & answer is better. The correlation between video quality and user engagement metrics was negligible.
Conclusions
The video quality on social media platforms remains inadequate, offering limited utility to users. Frequently, the viewers cannot precisely determine if content from videos is valid or not. First, uploaders need to optimize video quality and second the oversight of platforms should be strengthened to improve public health literacy and raise awareness.
Introduction
Frailty is a state characterized by decreased capacity to sustain the body’s internal balance and an increased vulnerability to stressors. 1 Robust evidence indicates that frailty substantially increases the probability for falls, disability, surgical complications, neurological decline, and overall death in older adults.2–4 Meta-analysis has revealed that more than 60% of older adults exhibit some degree of frailty. 5 With its evidently age-related prevalence, and in a context of rapid acceleration of worldwide population aging, frailty is now deemed a major epidemiological burden worldwide. 6 Dietary intervention, regular physical activity, and engagement in intellectual leisure activities have been proven to be effective in slowing down the advancement of frailty and even reversing it.7–9
Efficient prevention and treatment of frailty are dependent on proper understanding into the condition by the general population, especially among the older adults and their caretakers. The concept of frailty is yet inadequately recognized. Most older adults and their relatives perceive it as a part of the normal process of aging, 10 consequently missing the ideal window for treatment. Moreover, some even label it by linking it with reliance, erosion of identity, and social exclusion. 11 This profound deficiency of acquaintance has resulted into low rates of appropriate identification and delayed actions. Consequently, enhancing public understanding about the science of frailty ought to proceed hand in hand with clear, practical guidance on how to act.
Videos on social media platforms have become a central component of health education because they are easily disseminated, spread rapidly, encourage interaction, and leverage people’s short free moments.12–14 Currently, many people find themselves predominantly depending on digital platforms as a means of sharing and acquiring health information and knowledge. 15 According to the 55th-Statistical Report on China’s Internet Development published by China Internet Network Information Center, the number of internet users has already reached 1.108 billion and elevated the national internet penetration rate to 78.6%. 16 The fastest-growing group of social media users today comprises older adults. Social media provides them with access to health information and health care services, which facilitates the improvement of their general health management.17–19
YouTube is the world’s most popular video platform, but it is not directly available in China. Instead, other platforms like TikTok (its Chinese version, Douyin), Bilibili, and Xiaohongshu represent key social media sites. 20 These attract a great range of users, including many older adults and families, and TikTok alone claims over 740 million average monthly active users. 21 It produces heavy traffic among users, which contributes to the dissemination of health information. The content review system is rather permissive, and hence a great deal of unverified, unilateral, or even totally erroneous health information flows freely, involving possible risks to public health awareness.22–24 Frailty misinformation can result in a range of poor health decisions, including delayed treatment, misuse of medications, unnecessary expenditure on products that do not work, and potential serious harm to a person’s health. Moreover, the role of health information producers is frequently mixed and there is an evident disparity in professional credentials, motivations, and accessible assets. In theory, medical professionals tend to publish data supported by research driven medicine to enhance the community’s health information. Non-profit organizations as well as news generally highlight wide accessibility and societal benefit. For profit organizations characteristically accentuate business promotion: building up a professional brand image and drawing potential clients. Individual creation is based upon personal experiences or commercial collaboration with scientific precision and rigors which remain often not systematically confirmed. Hence, the quality evaluation of videos originating from various sources not merely elucidates the ecology of online health data but also offers a foundation for the public to seek reliable resources. Therefore, when situated into framework, the accuracy and caliber of frailty-related content should be highly assessed on social media.
So far, only one study published in 2021 commenced appraised the quality of the videos on frailty, concentrating on YouTube exclusively. 25 With a continuously expanding number of internet users, research into medical and health data across major Chinese social media forums persists scarce. 26 It is within backdrop that this study commenced on an in-depth evaluation of frailty-related videos shared on TikTok, Bilibili, and Xiaohongshu. We required to examine the quality of the videos related to frailty on China’s predominantly employed video-sharing platforms with recommendations for ways to increase the accuracy and reach of the online content for health.
Methods
Setting and study design
The Cross-sectional content analysis study was Video filtering process and inclusion results. (a) Flowchart of filtering frailty videos for analysis. (b) Trend chart of the number of frailty videos released by the three platforms. (c) Comparison of content comprehensiveness between three platforms. (d) GQS, JAMA, and mDISCERN scores on three platforms. (e) PEMAT-U, and PEMAT-A scores on three platforms.
The inclusion criteria were: (1) videos needed to concentrate on frailty, and (2) they needed to be in Chinese. Exclusion criteria included the following: (1) redundant videos if two videos were equivalent in content or derived from the same primary material, the version uploaded by a authenticated account was kept; if neither account was verified, the video with the earlier uploading date was kept; (2) irrelevant videos-content was categorized into the following sections: ①definition, ②epidemiology, ③risk factors, ④pathophysiology, ⑤negative impacts, ⑥screening, and ⑦management. Videos not encompassing any of these defined aspects were considered irrelevant. (3) advertisements.
Collection of video features
The following data were be systematically collected for all videos included in the analysis: general characteristics of the video, uploader information, and content attributes.
The general characteristics of the video include: days since upload, video duration (seconds), and engagement metrics (number of likes, collections, comments, and shares). Theoretically, in social media, these participation indicators represent the user’s recognition strength (likes), practical value perception (collections),discussion depth (comments), and communication willingness (shares) of the content. In addition, these indicators are the core parameters in platform suggestion models, directly affecting content exposure. However, algorithm driven dissemination does not consistently align with the genuine quality of a video. Therefore, it is essential to investigate the relationship between video quality and interaction indicators. This helps determine whether audience engagement behaviors can serve as reliable filter for data constancy.
Uploader information includes: uploader, number of uploader fans, uploader identity, and certification status. The uploader identities include the self-media of medical-related personnel, the self-media of other individuals, for-profit organizations, non-profit organizations, and news agencies. Among them, the self-media of medical-related personnel and the self-media of other individuals belong to individual users, while others belong to organizational users.
The content attributes include: video content and presentation format. The presentation formats include: expert monologue, question & answer, animation, slideshow narration, photo slideshow, TV show/movie clips.
Quality assessment
The quality of the video content was assessed using four commonly employed tools: the Journal of American Medical Association (JAMA) benchmark criteria, the Global Quality Score (GQS), modified DISCERN (mDISCERN), and the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V).
The JAMA scale comprises of four items (see Supplementary Table 1); each item received one point if satisfied, resulting to a total score that ranged between 0 to 4. Higher scores indicate greater reliability.30,31 The GQS is used broadly to assess the health information quality on digital platforms (Supplementary Table 2). The tool assesses overall quality and flow, possessing achieved broad acknowledgment.32,33 Scores range from a 5-point Likert format in which higher scores denote more rigorous endorsements of the video. The mDISCERN tool is adapted from the DISCERN instrument (Supplementary Table 3), it assists consumers and healthcare professionals in evaluating the quality of health-related information.34,35 This scale possesses a total of five queries which grade 1 point each, for a possible rating ranging starting at 0 until 5. 36 The PEMAT-A/V tool was used to evaluate understandability (PEMAT-U) and actionability (PEMAT-A) (Supplementary Table 4).37,38 This comprises of a total of 17 items, including 13 items assessing understandability and 4 items for assessing actionability. A higher score indicates that the material is easier to understand or more operable.
Two researchers (HP and ZW) independently executed those assessments using any discrepancies resolved via discussion alongside a third researcher (L-YB).
Ethical considerations
All publicly available online videos referenced in this manuscript were used in strict accordance with their respective platform’s terms of use. This study utilized publicly available data from short video sharing platforms and did not involve any human or animal experiments; therefore, no ethical review was required. Additionally, all personally identifiable information in this study has been anonymized. The research complies fully with the ethical standards for publication.
Statistical analysis
All statistical analyses were conducted using IBM SPSS Statistics 29.0 for data processing and GraphPad Prism 9 and OriginPro 2024 software for graphical representations. The normality of quantitative data distributions was assessed using the Shapiro-Wilk test. All quantitative data were presented as mean ± standard deviation (SD), median (interquartile range [IQR]; minimum-maximum). Categorical data were summarized as frequencies and percentages. For comparisons between two groups in quantitative data, either the Student’s t-test or the Mann-Whitney U test was employed. For comparisons among three or more groups, one-way analysis of variance (ANOVA) or the Kruskal-Wallis test was utilized. When significant differences were observed across groups, the Dunn’s test was conducted to identify which specific groups differed from each other. Categorical data were analyzed using the Chi-square test, or Fisher’s exact test as appropriate for multi-group comparisons. Spearman correlation analysis got employed to assess the relationships among independent variables. Positive associations were defined as r > 0, whereas negative associations were defined as r < 0. The subsequent categories were employed to characterize the strength of the correlation: poor (|r| ≤ 0.2), weak (0.2 < |r| ≤ 0.4), moderate (0.4 < |r| ≤ 0.6), strong (0.6 < |r| ≤ 0.8), or very strong (|r| > 0.8). 33 Spearman correlation analysis is suitable for analyzing non normally distributed data and can accurately assess the link between video quality and engagement metrics. Inter-rater reliability for JAMA, GQS, mDISCERN, PEMAT-A, and PEMAT-U scores was evaluated using the intraclass correlation coefficient (ICC), with values interpreted as poor (<0.5), moderate (0.5-0.75), good (0.75-0.90), or excellent (>0.90). 39 Statistical significance was set at P < 0.05.
Results
Overview of the screening process for frailty videos
A sum of 126 videos were incorporated in analysis, of which 80 came from TikTok, 20 from Bilibili, and 26 from Xiaohongshu (Figure 1(a)).
Video features
The earliest year when frailty videos were uploaded was 2020, while the Xiaohongshu platform started uploading such videos two years later (Figure 1(b)). Bilibili’s video duration is higher than TikTok and Xiaohongshu (P<0.001). Frailty videos on TikTok received more likes (P<0.001), generated more comments (P<0.001), and were shared more frequently (P=0.005) compared to other platforms. In contrast, Bilibili videos were more likely to be collected (P<0.001) (Table 1).
TikTok’s uploaders have the highest average number of fans (P<0.001), and the participation of organizational users in this platform is relatively high (P=0.004). Overall, the highest number of videos were posted by medical-related personnel on self-media, with doctors accounting for 90.2% of uploads. Certified users account for 46.8% of the total, with TikTok having the highest uploader authentication rate at 70.0% (P<0.001).
Detailed characteristics of three different platforms.
SD, standard deviation; JAMA, journal of American medical association; GQS, global quality score; mDISCERN, modified DISCERN; PEMAT-U, patient education materials assessment tool - understandability; PEMAT-A, patient education materials assessment tool - actionability.
aMann-Whitney U test.
bChi-square test.
cFisher’s exact test.
Video quality and reliability assessment
The JAMA score for all videos is 1.1 (SD 0.8), GQS score is 2.8 (SD 1.0), mDISCERN score is 3.0 (SD 0.8), PEMAT-U score is 76.5% (SD 15.6), and PEMAT-A score is 49.7% (SD 40.2) (Figure 1(d) and (e)). Videos on Bilibili consistently demonstrated superior quality, including JAMA, GQS, mDISCERN, and PEMAT-U, when compared to those on TikTok and Xiaohongshu (all P < 0.05; Table 1).
Organizational users had higher JAMA and PEMAT-A scores than individual users (P=0.009, Figure 2(a); P=0.002, Figure 2(e), respectively). The JAMA score of videos uploaded by non-profit organizations is higher than that of self-media of other individuals (P=0.019, Figure 3(a)). Compared with self-media of medical-related personnel, the self-media of other individuals have lower mDISCERN scores (P=0.032, Figure 3(c)). The PEMAT-A score of non-profit organizations is better than that of self-media of other individuals (P=0.005, Figure 3(e)). The JAMA score, GQS score, mDISCERN score, PEMAT-U score, and PEMAT-A score of certified authors were significantly higher than those of non certified authors (P<0.001, Figure 4(a); P=0.037, Figure 4(b); P=0.004, Figure 4(c); P=0.039, Figure 4(d); P=0.026, Figure 4(e), respectively). The presentation format of expert monologue and question & answer is easier to understand than slideshow narration (P=0.010, P=0.035, Figure 5(d), respectively). Quality scores among different uploader categories. (a) JAMA score, (b) GQS score, (c) mDISCERN score, (d) PEMAT-U score and (e) PEMAT-A score. Quality scores among different uploader identities. (a) JAMA score, (b) GQS score, (c) mDISCERN score, (d) PEMAT-U score and (e) PEMAT-A score. Quality scores among different certification status. (a) JAMA score, (b) GQS score, (c) mDISCERN score, (d) PEMAT-U score and (e) PEMAT-A score. Quality scores among different presentation formats. (a) JAMA score, (b) GQS score, (c) mDISCERN score, (d) PEMAT-U score and (e) PEMAT-A score.
In fact, the two researchers demonstrated substantial agreement across all the evaluation measures: the JAMA score (ICC 0.988, 95% CI 0.983-0.992), GQS score (ICC 0.916, 95% CI 0.882-0.940), mDISCERN score (ICC 0.950, 95% CI 0.930-0.965), PEMAT-U score (ICC 0.943, 95% CI 0.920-0.960), and PEMAT-A score (ICC 0.980, 95% CI 0.972-0.986).
Engagement metrics analysis
Comparison of engagement metrics across different platforms, uploaders, and presentation styles.
Video quality demonstrated consistent but weak associations with video features (Figure 6). JAMA score was positively correlated with the certification status (r=0.350, P<0.001), PEMAT-A with uploader category (r=0.280, P=0.001), and mDISCERN score with the number of uploader fans (r=0.269, P=0.002). Both the GQS (r=0.540, P<0.001) and PEMAT-U scores (r=0.260, P=0.003) showed positive correlations with video duration. Notably, engagement metrics such as collections and shares were weakly correlated with quality across all scales (r=0.177-0.383), reinforcing that popularity does not guarantee reliability. The correlation between video quality and video features.
Discussion
The study examined the content of frailty-associated videos on three primary social media platforms within China and evaluated its overall quality. Additional assessments were executed based on the features of the uploader and the format of presentation of the videos.
Upload platform and uploader
These findings indicate that videos on frailty first emerged in 2020, likely due to the limited public understanding regarding this condition, as its symptoms are often observed just as a marker of normal aging. During 2020, the COVID-19 outbreak began, that raised novel attention towards health information. During periods of domestic isolation, the use of brief-video platforms grew sharply.40,41 Brief videos are in line with contemporary user habits, enabling popular scientific content to be circulated in a more vivid manner. 42 Consequently, TikTok serves as one of the main brief-video platforms which attracts a broad variety of producers who engage in scientific communication. On TikTok, the creator authentication proportion is the highest, attaining 70%, in concurrence with other studies. 33 The platform implements strict supervision of medical content, and certified accounts frequently enjoy better preferences regarding traffic. Conversely, users on Xiaohongshu rely on “real-life experiences” to a greater extent than official confirmation, which may explain why this constitutes the minimum verification rate among the platforms.
Video content
Regarding to content distribution, management-related subjects constitute the largest fraction of all three platforms, consistent with the results of Natalia. 25 Within the administration categories, physical activity and dietary guidance are the most dominant themes. Both of these measures are, indeed, the best-studied approaches for halting or even restoring frailty. 43 Aerobic exercise and muscle strengthening training can improve mobility and prevent falls. Nutritional strategies, such as supplementing vitamin D and protein, have a positive effect on improving the daily activities and quality of life of older adults. 44
Video quality
Current study used four different assessment tools to assess video quality, and in general, the videos were average and variable. The average JAMA score of all frailty videos on the three platforms is lower than YouTube, and the GQS score of Tiktok and Xiaohongshu videos is lower than YouTube. 25 The mean mDISCERN score of all videos is higher than that of pediatric pneumonia and gastric cancer videos.29,45 The PEMAT-U rating was greater than that of mitral valve regurgitation videos, conversely the PEMAT-A rating demonstrated the opposite trend. 46 In the current study, the rating for PEMAT-U was greater than that of the PEMAT-A. This was not a distinct finding as these studies have shown uniform outcomes in several other similar studies46–48 which indicating a deeper and broader structural issue in the creation of health science public education content.
It is essential for videos to show the older adults how to do things: (1) Enhance the compliance and precision of interventions. People have a tendency to depend on information that is clear, understandable, and can readily be converted into applicable action. 49 When health information was simple to understand and omits practical guidance, it poses a notable potential risk. People stay unsure of where to start, and poor adherence on health recommendations follows. It can also cause the viewers experiencing uncertain and helpless. (2) Reduce safety risks and prevent accidental injuries. Unclear guidance may lead to improper execution of health behaviors. A video that merely states “do exercise” but fails to explain “how to start exercising safely” is not only useless but may even increase the risk of injury due to improper exercise. People with low health literacy rely more on clear instructions, and the lack of actionable information has a greater impact on them, exacerbating health inequality. 50 If information isn’t actionable so users have to search for additional resources themselves and increase the duration and effort it takes to find what they require. (3) Enhance the self-efficacy of older adults and their ability to manage their health independently. Clear “how-to” guidance effectively boosts older adults’ confidence in executing health behaviors. Breaking down complex suggestions into specific steps can transform abstract goals into feasible actions, fostering the subjective initiative and enabling them to shift from passive recipients to active managers. Therefore, the core task for creators moving forward is to effectively transform health information into low-threshold, easy-to-follow action steps that older adults can easily understand and imitate, thereby substantially enhancing the actionability of the disseminated content.
Videos on Bilibili are of highly excellent quality and, in most instances, comprise proper references. As the platform utilizes long-form videos, data is able to be presented more systematically. Data from organizations is typically published after it proceeds via internal review. Medical personnel contribute authoritative content, while certified authors align videos with their professional reputation. Expert monologue videos get normally delivered by experts directly, with concentrated content, robust practical value, and high precision. On the other hand, question & answer videos cater to real viewers’ queries, rendering them highly targeted; their overall quality is generally superior.
Engagement metrics
In this study, there was only a weak correlation between most engagement metrics and video quality. This is consistent with prior study and possesses several significant practical implications. 47 It reinforces our concern within health communication regarding frailty, and the popularity of videos is frequently determined by the platform’s algorithms. So these participants engagements neither of which is a reliable indicator of scientific quality. Users’ likes, comments and sharing behaviors are more likely to be driven by emotional appeal, presentation form, creator’s popularity or other irrational factors, rather than based on a prudent assessment of its scientific accuracy and practicality. In He’s research, video quality is even negatively correlated with interactive behavior. 29 The “quality–interaction” gap indicates that lacking adequate guidance, the public possesses a difficult period recognizing and promoting high-quality health content themselves. Thus, one-sided and insufficiently highly applicable content may get excessive attention. For this reason, it is unsafe to obtain knowledge about health exclusively from “popular” videos, whether via user likes or automated suggestions.
Recommendations based on this study
Frailty prevention and supervision may be reinforced by increasing public awareness of it. For numerous individuals, social media videos have been a primary origin of medical data. Therefore, an effective approach to extend the scope and influence of frailty-associated video content involves optimizing their accessibility by utilizing large subscriber certified producers develop high-quality videos which provide precise medical understanding and enhance public health literacy. Furthermore, the duration is important, too. Excessively short videos generally cannot provide sufficient information, while too lengthy can scarcely be followed by viewers and may result in them losing attention. That is why it is pertinent to modify video duration depending upon the complexity of the content presented so that clarity and viewer engagement are assured. The public expects to obtain practical guidance derived from health data. Whenever the content remains clear, open, and easy to implement, it establishes trust within the platform or alternatively creator and encourages more beneficial practices. Its enhancing the practical serviceability of the knowledge on frailty remains highly significant as this would be more effectively serve the efficacy of public health communication and subsequently enhance improved general public health outcomes. The dissemination of false data online is an urgent issue that requires immediate consideration. 51 All platforms must reinforce creator verification to set up better content-surveillance systems that will aid avoid the circulation of deceptive or harmful content. As a result, users should develop strong analytical skills, 52 focusing on superior sources while seeking for health information-particularly videos published by authenticated producers, so that the data on which they base their decisions is precise and they are capable to make knowledgeable health choices.
Advantages and limitations
Withinn this study, the quality of frailty-associated videos among three major Chinese websites was evaluated using four evaluation instruments. Moreover, the correlation among video quality and user-participation metrics has been analyzed, which provided recommendations based on the results. Nevertheless, the study still has multiple constraints. Firstly, the most significant limitation of current study is the sampling approach. We concentrated on the top 100 algorithm guided videos per term on each site. This may introduce bias through the algorithms employed by these platforms to order videos because they are designed to enhance interaction and not necessarily display the highest quality or most educational content. Because of numerous high-quality and less well-known videos may not have been incorporated in our analysis. Our findings therefore describe widely viewed, high exposure frailty content and should not be interpreted as reflective of all relevant content on digital platforms. Secondly, the information provided on the platforms is partial, the correlations among the quantity of views and dislikes (ratings with thumbs down) and video authenticity and quality might not be examined. Thirdly, the current study investigated only Chinese-language videos from three primary sites, so the results are unable to be generalized to other platforms or languages. Lastly, video quality was evaluated mainly based on the personal opinion of the researchers. Although consistency among their evaluations was extremely high, some degree of bias necessarily persists.
Conclusions
Among the most popular frailty-related videos on the three major social media platforms are mainly uploaded by medical professionals, and almost half of the uploaders had verified credentials. Frailty management was the most frequently addressed topic, and the format of expert discourse was prevalent. In general, the standard of videos was inadequate, with Bilibili presenting comparatively high within selected collection of top videos and not applicable to non-curated content. The production quality and content-design aptitudes of many creators still require substantial improvement. There is a discrepancy between the strong comprehensibility of video content and its limited realism while viewers may grasp the knowledge relating to frailty, measures implemented for management are usually unclear. Consequently, the development of practical feasibility in educational videos needs to be highlighted. The platform should augment the scrutiny of information to ensure precision in all health-related content.
Supplemental material
Supplemental material - TikTok, Bilibili, and Xiaohongshu as sources of information on frailty: Cross-sectional content analysis study
Supplemental material for TikTok, Bilibili, and Xiaohongshu as sources of information on frailty: Cross-sectional content analysis study by Ping Hou, Wen Zhao, Pei Duan, Yangming Cheng, Yixuan Yao, Arzoo Nazir, and Yongbing Liu in DIGITAL HEALTH.
Footnotes
Acknowledgments
We extend our gratitude to all video creators whose publicly shared content contributed to this study. Their efforts provided invaluable data for our analysis.
Author contributions
HP participated in study conception, design, video scoring, data collection, data analysis, and manuscript drafting. ZW participated in video scoring, data collection, and manuscript drafting. DP participated in video screening and data analysis. C-YM participated in video screening. Y-YX participated in video screening. L-YB participated in study conception, design, video scoring, and manuscript revision.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work was supported by: National Social Science Fund of China (24BRK008); Innovation and entrepreneurship training program for students of Yangzhou University, China (XCX20250943).
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Declaration of generative AI in scientific writing
This study did not employ artificial intelligence for data analysis or research conclusions. DeepSeek was solely utilized for English grammar correction of the manuscript.
Supplemental material
Supplemental material for this article is available online.