Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study

Abstract

Purpose

Breast cancer, the most common cancer in women globally, highlights the need for patient education. Despite many breast cancer discussions on TikTok, their scientific evaluation is lacking. Our study seeks to assess the content quality and accuracy of popular TikTok videos on breast cancer, to improve the dissemination of health knowledge.

Methods

On August 22, 2023, we collected the top 100 trending videos from TikTok's Chinese version using “breast cancer/breast nodule” as keywords. We noted their length, TikTok duration, likes, comments, favorites, reposts, uploader types, and topics. Four assessment tools were used: Goobie's six questions, the Patient Educational Material Assessment Tool (PEMAT), the Video Information and Quality Index (VIQI), and the Global Quality Score (GQS). These instruments evaluate videos based on content, informational integrity, and overall quality.

Results

Among the 100 videos, content quality was low with Goobie's questions mostly scoring 0, except for management at 1.0 (QR 1.0). PEMAT scores were moderate: 54.1 (QR 1.6) for sum, 47.0 (QR 18.8) for PEMAT-A, and 52.3 (QR 11.7) for PEMAT-U. Regarding the quality of information, the VIQI (sum) median was 14.1 (QR 0.2). Additionally, the median GQS score was 3.5 (QR 0.1). Medical professionals’ videos focused on breast cancer stages, while patient videos centered on personal experiences. Patient videos had lower content and overall quality compared to those by medical professionals (PEMAT, GQS: P < 0.001, P = 0.004) but received more comments, indicating higher engagement (all P < 0.05).

Conclusion

TikTok's breast cancer content shows educational potential, but while informational quality is moderate, content quality needs improvement. Videos by medical professionals are of higher quality. We recommend increased involvement of healthcare professionals on TikTok to enhance content quality. Non-medical users should share verified information, and TikTok should strengthen its content vetting. Users must scrutinize the credibility of health information on social platforms.

Keywords

Breast cancer TikTok video quality assessment social media short video apps

Introduction

Given the prominence of breast cancer as the most prevalent cancer among women globally and its considerable health implications,¹ understanding the potential of online platforms to enhance patient education is imperative.² The rise of the internet and social media has underscored the importance of online healthcare education.³ Platforms like TikTok, launched by Chinese developers in 2017 and achieving over two billion downloads,⁴ have become powerful mediums for disseminating health-related content,⁵ including popular breast cancer awareness videos.^6,7 This aligns with the increasing digital health literacy among Chinese users⁸ and the growing demand for digital health resources. Digital education on platforms like TikTok has shown benefits in improving emotional cognition and social life quality for patients.⁹ TikTok has a strong influence in China and even around the world. Its outstanding feature is that it has a powerful content creation tool, which attracts a large number of ordinary users who have the desire to express themselves to participate in the creation, which is more conducive to us retrieving the information we want.

However, the use of social media for medical information distribution faces significant challenges. The lack of professional oversight and the potential for misinformation are major concerns. Previous research has found errors and misleading information on various social media platforms.^10–14 This is compounded by the Echo Chamber Effect, where users may engage with and reinforce their preexisting beliefs, potentially exacerbating misinformation impacts.^15,16 Additionally, the credibility of online health materials can be difficult for users, especially those with lower health literacy, to assess.¹⁷

Based on the above concerns, we aim to address the following two questions: First, how is the quality of breast cancer-related videos on TikTok? Second, what is the relationship between different uploaders and the content they upload? So, we employed four validated scoring instruments to evaluate the content, informational value, and overall quality of breast cancer-related information on TikTok. These tools include Goobie et al.'s six questions,^10,18 the Patient Education Material Assessment Tool (PEMAT),^19,20 the Video Information and Quality Index (VIQI),^18,21 and the Global Quality Score (GQS).^22–24 These instruments provide comprehensive assessments, evaluating content quality, information flow, accuracy, and overall utility to patients. This approach helps in identifying the effectiveness of TikTok videos in delivering accurate and useful health education, aiming to mitigate the challenges posed by the platform's open content environment.

Methods

Search strategy and information extraction

On August 22, 2023, we conducted a search on TikTok (Chinese mainland version) using the keywords “breast cancer/breast nodule”. We included all videos relevant to breast cancer posted in Chinese and excluded irrelevant videos, duplicates, commercial promotions, and videos without audio or substantive content. We selected the top 100 videos from the “comprehensive ranking” for analysis (the workflow is shown in Figure 1). TikTok offers three sorting options: comprehensive ranking, latest releases, and most likes. We chose the comprehensive ranking as it is the default option, which most users typically follow. This selection was based on two observations: most relevant videos appeared within the top 100 search results, and beyond this, relevance noticeably declined (Figure 2 shows the correlation curve of the top 300 videos with the search target). Additionally, ordinary users prefer viewing initial search results due to their tendency towards minimal effort when using search engines.²⁵

We systematically collected and recorded essential details of each video, including its URL, topic, publication date, uploader type and verification status, duration on TikTok, thumbs-up, comments, reposts, and favorites count, in an Excel spreadsheet (Microsoft Corporation).

Quality assessment

Two experienced breast surgery doctors (QY and LJ) assessed the video links presented in a tabular format. Before rating, they reviewed the scoring criteria for each scale. Independently, they viewed the videos to assign ratings and classify them according to source and content. In cases of discordant scores, a comprehensive discussion with an additional observer (PB) was conducted to achieve consensus. Video quality was evaluated based on three aspects: content quality, information quality, and overall quality. Content quality focuses on the reliability and completeness of the video content. Information quality pertained to how smoothly and accurately the information was conveyed. Overall quality synthesized these aspects to evaluate the video's usefulness to the audience.

Figure 1.

The workflow of the study.

Figure 2.

The number of videos that comply with our criteria in the top 300 most popular videos.

To assess content quality, we utilized six questions from Goobie et al. and the PEMAT. To evaluate information quality, we employed the VIQI scale. The GQS tool was used to assess the overall quality of the videos. The details of each scale are depicted in the supplementary material. Below are detailed descriptions of each tool's scoring methods and characteristics:

Six questions from Goobie et al:^10,18 The scale evaluates the degree to which the video addresses six fundamental aspects: disease definition, signs/symptoms, risk factors, evaluation, management, and outcomes. Ratings were assigned based on content comprehensiveness: 0 points (no content), 0.5 points (limited content), 1 point (incomplete content), 1.5 points (substantial content), and 2 points (comprehensive content).

Patient Education Material Assessment Tool (PEMAT):^19,20 This scale was designed to evaluate audiovisual materials. It consists of 17 questions: 13 assessing the understandability of health information and 4 evaluating the actionability of video recommendations. Each question is scored as “Agree = 1” or “Disagree = 0,” and the total score, understandability score, and actionability score are calculated as “Total Score/Total Possible Score × 100.”

Video Information and Quality Index (VIQI):^18,21 This index assesses the information quality of a video using four components: VIQI 1 (Information Flow), VIQI 2 (Information Accuracy), VIQI 3 (Quality: based on the inclusion of images, animations, interviews, captions, and summaries), and VIQI 4 (Precision: consistency between the video title and content). Each component is rated on a Likert scale ranging from 1 to 5.

Global Quality Scale (GQS):^22–24 Designed to assess the quality of information on websites and rate videos based on their reliability and content. Scores range from 1 to 5, indicating poor to excellent video quality, with higher scores representing more reliable and comprehensive videos.

Statistical analysis

For statistical analysis, we used the Kolmogorov–Smirnov test for normality when the sample size was greater than 50 and the Shapiro–Wilk test when the sample size was less than 50.^26,27 Mean ± standard deviation and min-max were employed for normally distributed continuous variables, while median ± quartile range (QR) was used for non-normally distributed variables. The Kruskal–Wallis test was used to compare multiple non-normally distributed groups, and the Bonferroni post-hoc test assessed differences between two non-normally distributed groups. The interclass correlation coefficient (ICC) was used to assess the consistency of scoring by the two raters.^28,29 The inter-rater agreement represents how consistent two breast surgeons (QY and LY) are in scoring each evaluation criterion for each video. We opted for the one-way random effects, absolute agreement, multiple raters/measurements model, with the calculation formula being (MSR = mean square for rows; MSW = mean square for residual sources of variance). Statistical significance was defined at a P-value < 0.05.

Results

Video characteristics

Uploader types: The TikTok platform authenticates users’ professional credentials and affiliations, allowing us to categorize video uploaders into five groups (Figure 3A): breast cancer (BC) patients (20%), modern medicine (MM) professionals (60%), traditional Chinese medicine (TCM) practitioners (14%), news media workers (3%), and science bloggers (3%). Table 1 shows the proportion of content shared by different uploaders. MM professionals covered all topics but mainly focused on breast cancer stages, with “Risk factors/Prevention” being the most shared topic (28.3%). BC patients mostly shared survivorship and illness experiences. TCM practitioners primarily disseminated information on risk factors/prevention.

Video topics: Content creators addressed five main domains (Figure 3B): risk factors/prevention (25%), symptoms/detection (16%), treatment (18%), recurrence/prognosis (18%), and survivorship/experience sharing (23%). Videos were classified by their primary emphasis if multiple themes were present.

2.1. Risk factors/prevention: predominantly shared by MM practitioners (n = 18, 72%), followed by TCM practitioners (n = 4, 16%) and patients (n = 3, 12%).

2.2. Symptoms/detection: mainly contributed by MM practitioners (n = 12, 75%), with smaller contributions from TCM practitioners, patients, science bloggers, and news media workers (each n = 1, 6.25%).

2.3. Treatment: led by MM practitioners (n = 12, 66.7%), followed by TCM practitioners and patients (each n = 3, 16.7%).

2.4. Recurrence/prognosis was primarily shared by MM practitioners (n = 14, 77.8%), with contributions from TCM practitioners and patients (each n = 2, 11.1%).

2.5. Survivorship/experience sharing: involves various professionals, including patients (n = 11, 47.8%), MM practitioners (n = 5, 21.7%), TCM practitioners (n = 3, 13%), science bloggers (n = 2, 8.7%), and news media workers (n = 2, 8.7%).

Video features: Table 2 presents the fundamental characteristics of all the videos collected. We found that the length of videos varies from 11 s to 1777 s with a median length of 109 s. To mitigate potential confounding effects from publication time on other metrics, we further standardized the values for thumbs-ups, comments, favorites, and reposts by dividing them by the respective number of days since publication.

Video quality

Table 3 presents the quality assessment scores of the 100 TikTok videos related to breast cancer.

Content quality: the Goobie scale's six questions examine the breadth and depth of disease information covered. Except for Goobie 5 (management) with a median score of 1.0(QR 1.0), all other indicators had a median score of 0. The PEMAT scale assessed video content for actionability and understandability, with median scores of 54.1 (QR 1.6) for the sum, 47.0 (QR 18.8) for actionability (PEMAT-A), and 52.3 (QR 11.7) for understandability (PEMAT-U).

Information quality: the VIQI scale's median score was 14.1 (QR 0.2), with the highest performance in video title and content consistency (VIQI4) at a median score of 4.0 (QR 0.8) and information accuracy (VIQI2) at 3.9 (QR 0.8).

Overall video quality: the median GQS was 3.5 (QR 0.1).

Figure 3.

Distribution of breast cancer-related video topics and uploaders on TikTok. (A) Distribution of the videos based on uploaders. (B) Distribution of the videos based on topics.

Table 1.

The percentage of each topic shared by different uploaders.

	BC patient (N = 20)N(%)	MM professional (N = 60)N(%)	News media worker (N = 3)N(%)	TCM practitioner (N = 14)N(%)	Science blogger (N = 3)N(%)
Risk factors/Prevention (N = 25)	3 (15.0%)	17 (28.3%)	0 (0.0%)	5 (35.7%)	0 (0.0%)
Symptoms/Detection (N = 16)	1 (5.0%)	12 (20.0%)	1 (33.3%)	1 (7.1%)	1 (33.3%)
Treatment (N = 18)	3 (15.0%)	12 (20.0%)	0 (0.0%)	3 (21.4%)	0 (0.0%)
Recurrence/Prognosis (N = 18)	2 (10.0%)	14 (23.3%)	0 (0.0%)	2 (14.3%)	0 (0.0%)
Survivorship/Experience sharing (N = 23)	11 (55.0%)	5 (8.3%)	2 (66.7%)	3 (21.4%)	2 (66.7%)

Table 2.

Characteristics of videos about breast cancer on TikTok.

Characteristics	Median (QR)	Min–Max
Video length (second)	109 (77.1)	11–1777
Duration on TikTok (days)	425 (568)	1–1360
Thumbs-up	3834 (17,494)	15–40,700
Thumbs-up/day	39 (122)	0–7679
Comments	440 (1514)	0–61,000
Comments/day	3 (11)	0–504
Favorites	348 (1689)	0–81,000
Favorites/day	3 (30)	0–903
Reposts	473 (2646)	0–106,000
Reposts/day	4 (25)	0–2751

Table 3.

Quality assessment of videos about breast cancer on TikTok.

		Median (QR)	Min–Max
Content quality assessment
	GOOBIE1(definition) GOOBIE2(symptoms) GOOBIE3(risk factors) GOOBIE4(evaluation) GOOBIE5(management) GOOBIE6(outcomes)	0.0 (0.0) 0.0 (1.0) 0.0 (0.5) 0.0 (1.0) 1.0 (1.0) 0.0 (1.0)	0–2.0 0–2.0 0–2.0 0–2.0 0–2.0 0–2.0
	PEMAT (sum)^a	54.1 (1.6)	23.5–82.4
	PEMAT-A	47.0 (18.8)	0–75.0
	PEMAT-U	52.3 (11.7)	28.6–85.7
Information quality assessment
	VIQI1^b	3.1 (0.0)	2.0–4.0
	VIQI2	3.9 (0.8)	2.0–5.0
	VIQI3	2.9 (0.8)	2.0–4.0
	VIQI4	4.0 (0.8)	3.0–5.0
	VIQI (sum)	14.1 (0.2)	11.0–18.0
Overall quality assessment
	GQS^c	3.5 (0.1)	2.0–5.0

^aPatient Educational Material Evaluation Tool.

^bVideo Information and Quality Index.

^cGlobal Quality Score.

Association between video features and characteristics with uploaders

Table 4 shows the associations between different types of uploaders, video characteristics, and video quality. Figure 4 highlights statistically significant disparities, including:

Video length: news media posted the shortest videos (median 56 s, QR 30 s), while science bloggers posted the longest (median 216 s, QR 116 s).

Duration on TikTok: TCM practitioners had the shortest median duration (31 days, QR 491), whereas news media workers had the longest (788 days, QR 433).

Engagement metrics: Science bloggers had significantly more likes, comments, favorites, and reposts.

Content quality: BC patients, MM practitioners, news media workers, and TCM practitioners scored highest on Goobie 5 (management), while science bloggers scored highest on Goobie 4 (evaluation).

Figure 4.

The analysis of comments, reposts, thumbs-up, PEMAT-A, GQS and PEMAT-sum was based on different TikTok video uploaders, which were statistically different. When the distribution of the data is concentrated, the box plot is a straight line. (A) The number of comments. (B) The number of reposts. (C) The number of thumbs-up. (D) The score of PEMAT-A. (E) The score of GQS. (F) The score of PEMAT-sum. BCP: Breast Cancer Patient; MMP: Modern Medicine Professional; NMW: News Media Worker; SB: Science Blogger; TCMP: Traditional Chinese Medicine Practitioner.

Table 4.

Detailed characteristics of videos based on uploaders.

		BC patient (N = 20)Mean (SD)/Median (QR)	MM professional (N = 60)Mean (SD)/Median (QR)	News media worker (N = 3)Mean (SD)/Median	TCM practitioner (N = 14)Mean (SD)/Median (QR)	Science blogger (N = 3)Mean (SD)/Median	P-value
Common characteristics of videos
	Video length (second)	148 (84)^a	60 (47)	56 (30)	74(50)	216 (116)	0.006
	Duration on TikTok (day)	436 (234)^b	422(573)	788 (433)	31 (491)	63	0.1
	thumbs-up	17,500 (75,655)	2166 (10,368)	39,791 (40,518)	1146 (7949)	206,000 (130,543)	<0.001
	thumbs-up/day	61 (209)	26 (88)	59 (58)	62 (145)	2871 (2949)	0.02
	Comments	1422 (4892)^c	201 (767)	2001 (1445)	100 (843)	16,217 (5828)^d	<0.001
	Comments/day	7 (12)	2 (6)	4 (3)	4.5 (15)	241 (163)	0.09
	Favorites	1052 (2202)	142 (886)^e	1916 (1763)	102 (1125)	29,625 (44,497)	0.002
	Favorites/day	4 (11)	2 (5)	4 (3)	8 (23)	52	0.10
	Reposts	1050 (2649)	202 (2102)	12,650 (10,243)	210 (1769)	47,243 (52,312)	<0.001
	Reposts/day	4 (12)	2.5 (5)	28 (27)	4.5 (20)	91^f	0.04
Content quality assessment
	GOOBIE1GOOBIE2GOOBIE3GOOBIE4GOOBIE5GOOBIE6	0 (0)0 (1)0 (0.8)0.5 (1.5)1 (1.0)0 (1)	0 (0)0 (1)0 (1)0 (1)1 (1)0 (0.8)	0000.3 (0.6)0.8 (0.8)0	0 (0.3)0 (0.5)0 (0.3)0 (0.5)1 (1.3)0 (1.0)	00.8 (0.8)0.5 (0.9)1.2 (0.3)10	0.60.60.30.10.60.8
	PEMAT (sum)	46.2 (6.5)^g	59 (15)	54.9	54.2 (13.0)	53	0.003
	PEMAT-A	33.8 (21.3)	50 (16.8)	50 (0)	50 (13)	50	0.001
	PEMAT-U	46.4 (7.0)^h	57 (14)	52.4 (8.9)	50.5 (13.6)	50	0.09
Information quality assessment
	VIQI1	3 (0)	3.1 (0)	3	3 (1)	3	0.6
	VIQI2	4 (0)	3.9 (0)	3 (0.8)	4 (1)	4 (0.7)	0.03
	VIQI3	3 (1)ⁱ	2.7 (1)	3	3 (1)	3	0.08
	VIQI4	4 (1)^j	4.3 (1)	4	4 (1)	4	0.03
	VIQI (sum)	14 (2)^k	14 (2)	13	14.1 (2)	14.7 (1.5)	0.03
Overall quality assessment
	GQS	3.0 (1)^l	3.7 (1)	3	3.7 (0.9)	4	0.004

^aCompared with modern medicine practitioner, P = 0.04.

^bCompared with traditional Chinese medicine practitioner and modern medicine practitioner, P = 0.02 and 0.001, respectively.

^cCompared with traditional Chinese medicine practitioner, modern medicine practitioner, P = 0.009 and <0.001, respectively.

^dCompared with modern medicine practitioner, P = 0.04.

^eCompared with breast cancer patients, P = 0.03–.

^fCompared with traditional Chinese medicine practitioner, P = 0.04.

^gCompared with modern medicine practitioner, P < 0.001.

^hCompared with modern medicine practitioner, P = 0.02.

ⁱCompared with modern medicine practitioner, P = 0.05.

^jCompared with modern medicine practitioner, P = 0.006.

^kCompared with modern medicine practitioner, P < 0.001.

^lCompared with modern medicine practitioner, P = 0.004.

Pairwise comparisons revealed

Content quality: MM videos scored higher than patient videos in PEMAT and PEMAT-U (P < 0.001 and P = 0.02, respectively).

Information quality: Patient videos had significant differences with MM practitioners in VIQI3, VIQI4, and VIQIsum (P = 0.05, P = 0.006, and P < 0.001, respectively).

Overall quality: BC patients’ median scores were significantly lower than MM practitioners (P = 0.004).

Association between video features and characteristics with video topics

Table 5 compares the differences in features and quality across various video themes, while Figure 5 graphically illustrates items with significant differences.

Interaction metrics: “Symptoms/Detection” videos received the most thumbs-ups, reposts, comments, and favorites.

Content quality (Goobie/PEMAT): Scores were largely similar across “Goobie1” through “Goobie6”, with a slight statistical difference in “Goobie4” (P = 0.046). Except for Goobie5 (evaluation), which had a median score of 1.0, most other topics had a median score of 0. PEMAT evaluation: “Symptoms/Detection” videos achieved the highest overall scores, with a total score of 64 (IQR 11.77), understandability score of 50 (IQR 25), and actionability score of 57.1 (IQR 12.0).

Information quality (VIQI): “Risk Factors/Prevention” videos scored highest in “VIQIsum” with a total of 14.2 (QR 1.2), significantly higher than “Survivorship/Experience Sharing” (P = 0.01).

Overall quality (GQS): “Symptoms/Detection” videos scored highest at 3.8 (±0.8). Median scores for “Survivorship/Experience Sharing” were significantly lower than those for “Risk Factors/Prevention” and “Symptoms/Detection” (P < 0.001 and P = 0.03, respectively).

Figure 5.

The analysis of comments, reposts, thumbs-up, PEMAT-A, PEMAT-U, and PEMAT-sum, which were statistically different. When the distribution of the data is concentrated, the box plot is a straight line. (A) The number of comments. (B) The number of reposts. (C) The number of thumbs up. (D) The score of PEMAT-A. (E) The score of PEMAT-U. (F) The score of PEMAT-sum. R/P: Recurrence/Prognosis; RF/P: Risk Factors /Prevention; S/D: Symptom/Detection; S/ES: Survivorship/Experience Sharing; T: Treatment.

Table 5.

Detailed characteristics of videos based on types.

		Risk factors/Prevention (N = 25) Mean(SD)/Median (QR)	Symptom/Detection (N = 16) Mean(SD)/Median (QR)	Treatment (N = 18) Mean(SD)/Median (QR)	Recurrence/Prognosis (N = 18) Mean(SD)/Median (QR)	Survivorship/Experience sharing (N = 23) Mean(SD)/Median (QR)	P-value
Common characteristics of videos
	Video length (second)	66 (58)	83 (44)	78 (50)	70 (64)	70 (162)	0.9
	Duration on TikTok (day)	326 (634)	464 (604)	168 (483)a	190 (501)	190 (385)	0.03
	thumbs-up	2470 (20,780)	5470 (50,658)	1045 (7255)	2813 (9520)	2813 (102,243)b	0.01
	thumbs-up/day	41 (10)	32 (116)	68 (118)	27 (82)	17 (223.6)	0.7
	Comments	201 (1452)	388 (2881)	123 (805)	334 (755)	334 (7645)c	0.007
	Comments/day	3 (10)	3 (4)	5 (15)	2 (7)	2.24 (16.8)	0.6
	Favorites	256 (2109)	878 (3396)	161 (466)	348 (775)	348 (2168)	0.2
	Favorites/day	5 (22)	4 (7)	7.5 (30)	2 (8)	2.5 (12.7）	0.5
	Reposts	384 (4459.5)	1289 (4778)	135 (580)	369 (1371)	369 (2920)	0.049
	Reposts/day	5 (16)	3.5 (9)	2 (7)	3 (4)	2.78 (42)	0.3
Content quality assessment
	GOOBIE1 (definition)	0 (0.5)	0 (0)	0 (0)	0 (1)	0 (0)	0.2
	GOOBIE 2 (symptoms)	0 (1)	0 (1)	0 (0)	0 (1)	0 (1)	0.3
	GOOBIE 3 (risk factors)	0 (0)	0 (0)	0 (1)	0 (1)	0 (1)	0.2
	GOOBIE 4 (evaluation)	0 (1.5)	1 (1.5)	0 (0.125)	0 (1)	0 (1.5)	0.046
	GOOBIE 5 (management)	1 (1)	1 (0.88)	1 (0.75)	1 (1)	1 (1)	0.3
	GOOBIE 6 (outcomes)	0 (1)	0 (0.38)	0 (0.5)	0 (1)	0 (0.5)	0.6
	PEMAT (sum)	59 (11.8)	64 (11.77)	51.6 (14.0)	55.2 (9.3)	44.8 (9.0)d	<0.001
	PEMAT-A	50 (0)f	50 (25)	50 (25)	50 (13)	50 (25)e	0.001
	PEMAT-U	57 (14.28)	57.1 (12.0)	51.2 (15.0)	50 (16)	50 (14)g	<0.001
Information quality assessment
	VIQI1	3 (0)	3 (0)	3 (0)	3 (0)	3 (0)	0.6
	VIQI2	4 (0)h	4 (0)	4 (1)	4 (0)	3 (1)	0.02
	VIQI3	3 (1)	3 (0)	3 (1)	2 (1)	3 (1)	0.2
	VIQI4	4 (1)	4 (1)	4 (1)	4 (1)	4 (0)i	0.047
	VIQI(sum)	14.2 (1.2)	14.1 (1.5)	14.0 (1.9)	13.7 (1.9)	12.9 (1.1)j	0.03
Overall quality assessment
	GQS	4 (1)	4 (1)	3 (1)	3.5 (1)	3 (0)k	0.002

^aCompared with treatment, P = 0.03.

^bCompared with Treatment and Recurrence/prognosis, P = 0.03 and 0.02, respectively.

^cCompared with Risk factors/Prevention and Treatment, P = 0.009 and 0.03, respectively.

^dCompared with Risk factors/Prevention, Symptoms/Detection and Recurrence/prognosis, P = 0.02, 0.02 and P < 0.001, respectively.

^eCompared with Risk factors/Prevention and Symptoms/Detection, P = 0.03 and 0.004, respectively.

^fCompared with Treatment, P = 0.04.

^gCompared with Risk factors/Prevention, Symptoms/Detection,and Recurrence/Prognosis, P < 0.001 and P = 0.002, respectively.

^hCompared with Survivorship/Experience sharing, P = 0.02.

ⁱCompared with Risk factors/Prevention and Treatment, P = 0.02 and 0.02, respectively.

^jCompared with Risk factors/Prevention, P = 0.01.

^kCompared with Risk factors/Prevention and Symptoms/Detection, P < 0.001 and P = 0.03, respectively.

Notably, “Survivorship/Experience Sharing” videos consistently received the lowest scores across multiple quality assessment indicators.

Discussion

General discussion

Breast cancer, with its increasing incidence rate but abundant treatment options, and relatively good prognosis, has gradually gained attention from researchers and patients. This study explored breast cancer-related content on TikTok, a popular social media platform in China. By analyzing the top 100 most popular videos, we gained insights into the state of educational content about breast cancer on TikTok, along with differences in uploader types and discussion topics.

Findings concerning general characteristics of videos

In our analysis, the median video duration was 109 s (IQR 77.1), aligning with research that indicates videos under one minute attract larger audiences.⁵ These videos received a median of 3834 likes (IQR 17,494), 440 comments (IQR 1514), and 348 favorites (IQR 1689), showing TikTok's potential for spreading breast cancer awareness and enhancing health literacy.^30,31 However, the significantly lower number of comments compared to likes suggests that TikTok's health communication is mainly one-directional,⁵ indicating a need for more interactive dialogue.³² To maximize TikTok's benefits, analyzing audience characteristics such as age, gender, and location can help create targeted videos,^5,33 fostering active health-related discussions.

Findings concerning video quality

The majority of videos reviewed had some scientific value but often presented fragmented and limited information with considerable repetition. The scientific value varied depending on the uploader's background. Videos by medical professionals and science bloggers typically relied on evidence-based medicine, while those by patients and news media were more anecdotal and potentially subjective. Some videos had exaggerated titles, which could be misleading.

We assessed the videos using four evaluation tools. Content quality, as measured by the Goobie scale, was generally low, highlighting the difficulty of providing comprehensive explanations in short video formats.²¹ The PEMAT scale showed low overall scores, with actionability (PEMAT-A) significantly lower than understandability (PEMAT-U), indicating a focus on theoretical knowledge rather than practical guidance. Previous studies have shown that users prefer easily understandable information and engaging videos.⁵ To make videos more appealing and understandable, uploaders often use animations and subtitles. The VIQI scale suggested decent information quality, while the median GQS score of 3.5 (±0.1) indicated moderate overall quality of the videos.

Findings concerning video topics and uploaders

Our analysis revealed a wide range of breast cancer-related videos, broadly categorized into five themes. Risk factors/prevention were the most shared topics, signaling their growing importance in disease management. Survivorship/experience sharing was also prominent, providing emotional support and stress relief to patients.³⁴ These topics generated the highest number of comments, indicating strong user engagement. The remaining topics covered treatment, recurrence/prognosis, and symptoms/screening, addressing various stages of breast cancer progression.

In terms of uploader types, modern medicine (MM) professionals, particularly breast or oncology specialists, were the primary uploaders (60%), focusing mainly on risk factors/prevention, recurrence/prognosis, symptoms/detection, and treatment. Content related to survivorship/experience sharing was less frequent (8.3%). Only one uploader was an official hospital account, indicating a lack of engagement from official healthcare organizations in China on social media.⁵

Breast cancer patients were the second largest group of uploaders (20%), primarily sharing their medical experiences and recovery stories (55%). Despite scoring lower on content quality (PEMAT) and overall quality (GQS), their videos received more likes, comments, and shares compared to those by MM professionals and TCM practitioners. TCM practitioners comprised the third largest group of uploaders (14%), with their content scoring well in quality assessments. News media workers and science bloggers made up the smallest proportion of uploaders (3% each), but science bloggers, with their large fan base, received significantly higher engagement metrics, highlighting the potential influence of social media influencers in health education.³⁵

We recommend that more medical professionals and non-profit healthcare institutions engage on TikTok, incorporate trending topics, and collaborate with influential organizations and individuals to amplify their promotional impact.³⁶ Maintaining attention primarily relies on video quality, which will ensure the extensive dissemination of their messages and influence health-related behaviors.

Comparison with other relevant studies

Furthermore, we reviewed studies on the quality of breast cancer videos across various languages and social media platforms. A study published in 2024 assessing information related to breast cancer surgery on TikTok confirmed that information about breast cancer surgery is widely shared on the platform, with the most common type of content creators being patients (77.3%).³⁷ Audience engagement was high. However, educational content was rare (6.5%), indicating a relative lack of objective and educational content provided by physician creators. A cross-sectional content analysis⁶ of 100 videos on TikTok about breast cancer from the USA revealed that videos shared by breast cancer patients or their loved ones made up the predominant proportion (81%). This is similar to our findings, where the proportion of videos shared by breast cancer patients was second only to those by modern medicine practitioners. Patients and their supporters often share their experiences on public platforms, seeking support and comfort from the community. They also convey breast cancer risk factors through personal stories in a relatable manner, which tends to be more persuasive and elicit greater public empathy. Consequently, despite the average quality of patient-shared videos being subpar in our study, they received more comments and thumbs-up.

In 2020, Chinese researchers analyzed the quality of 170 breast cancer-related videos on the Xigua video platform and found that a significant 62.4% of the videos contained misleading information.³⁸ The following year, a study of 101 YouTube videos concerning metastatic breast cancer in young women indicated that while the videos were rated satisfactory in terms of understandability and moderate in actionability, they scored low on reliability and overall quality, with some exhibiting commercial bias. This suggests that the full potential of online resources for patient support remains untapped.³⁹ In 2023, another study assessed the top 100 videos about breast cancer on both TikTok and Bilibili, concluding that although the quality and reliability of videos on Bilibili were superior to those on TikTok (P = 0.002 and P = 0.001, respectively), both platforms’ offerings were generally of poor overall quality. However, videos uploaded by physicians covering disease knowledge, prevention, and treatment were found to be of higher quality.⁷

Limitations

Our study has several limitations. First, our cross-sectional design, which captures a “snapshot” of data, may not reflect the dynamic nature of TikTok content. Second, selecting only a few tags and the top 100 most popular videos could introduce bias and overlook less popular videos with different qualities. Third, our search was limited to the Chinese version of TikTok, without considering content on other language platforms or other popular social media platforms in China. Fourth, owing to limitations in the TikTok system settings, we were unable to access important indicators such as the “number of views” for each video, which could provide insights into their level of dissemination. Additionally, the absence of a “dislike/disagree” function in TikTok prevented us from directly observing conflicting opinions among users. Finally, while user comments can indicate trust levels towards video content, it's important to remember that credibility is based on perceptions, not necessarily objective quality.⁴⁰

Conclusion

In conclusion, our study highlights a substantial discourse on breast cancer on TikTok. However, the spread of information is not always based on scientific evidence, and individuals may struggle to assess its accuracy. High volumes of discussion don’t necessarily lead to well-informed health decisions; this requires quality control and scientifically valid information. The information quality of popular breast cancer-related videos on TikTok is generally acceptable, yet the content quality is subpar, and there is a presence of misleading information. Furthermore, we noted an imbalance in the topics uploaded by different users and significant quality variations. Verified healthcare professionals play a crucial role in sharing scientific information, but the platform should regulate content from other users and ideally have medical professionals review it. As experts, we must understand patients’ concerns and lead social media discussions, providing accurate and valuable information. This will empower people to take proactive steps in early cancer detection and treatment, ultimately supporting a normal lifestyle.

Further research is needed to evaluate the real impact of social media on early breast cancer detection and postoperative care. We urge healthcare professionals and institutions to understand the benefits and limitations of social media platforms like TikTok, and actively use them to share medical knowledge and provide more scientifically grounded information, thereby reshaping public perception and promoting healthier behaviors.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241277688 - Supplemental material for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241277688 for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study by Yang Qu, Jie Lian, Bo Pan, Jiahui Zhang and Yidong Zhou in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076241277688 - Supplemental material for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study

Supplemental material, sj-pdf-2-dhj-10.1177_20552076241277688 for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study by Yang Qu, Jie Lian, Bo Pan, Jiahui Zhang and Yidong Zhou in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076241277688 - Supplemental material for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study

Supplemental material, sj-docx-3-dhj-10.1177_20552076241277688 for Assessing the quality of breast cancer-related videos on TikTok: A cross-sectional study by Yang Qu, Jie Lian, Bo Pan, Jiahui Zhang and Yidong Zhou in DIGITAL HEALTH

Footnotes

Acknowledgments

Thanks to the authors’ colleagues for their contributions to the statistical and manuscript review of this study.

Author contributions

Thesis concept was accomplished by Yidong Zhou and Yang Qu. Data collection and analysis were done by Yang Qu, Jie Lian, and Jiahui Zhang. Original draft writing was done by Yang Qu, Jie Lian, and Bo Pan. Paper review and editing were done by Yidong Zhou and Bo Pan.

Declaration of conflicting interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Ethical approval

Approval was not required.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of China, CAMS Innovation Fund for Medical Sciences (CIFMS), the National High-Level Hospital Clinical Research Funding (grant numbers 81001183, 2021-I2M-1-014, 2022-PUMCH-B-039, 2022PUMCH-A-165).

Guarantor

Yidong Zhou.

Informed consent

All our information was obtained from the public video platform TikTok, no patient's consent was needed.

ORCID iD

Yidong Zhou

Supplemental material

Supplemental material for this article is available online.

References

Momenimovahed

Salehiniya

. Epidemiological characteristics of and risk factors for breast cancer in the world. Breast Cancer 2019; 11: 151–164.

Ziebland

Chapple

Dumelow

, et al. How the internet affects patients’ experience of cancer: a qualitative study. Br Med J 2004; 328: 564.

Martínez-Miranda

Casuso-Holgado

Jesús Jiménez-Rejano

. Effect of patient education on quality-of-life, pain and fatigue in breast cancer survivors: a systematic review and meta-analysis. Clin Rehabil 2021; 35: 1722–1742.

Villa-Ruiz

Kassamali

Mazori

, et al. Overview of TikTok’s most viewed dermatologic content and assessment of its reliability. J Am Acad Dermatol 2021; 85: 273–274.

Zhu

Zhang

, et al. How health communication via TikTok makes a difference: a content analysis of TikTok accounts run by Chinese provincial health committees. Int J Environ Res Public Health 2019; 17: 192.

Basch

Hillyer

Yalamanchili

, et al. How TikTok is being used to help individuals cope with breast cancer: cross-sectional content analysis. JMIR Cancer 2022; 8: e42245.

Liu

Peng

, et al. Assessment of the reliability and quality of breast cancer related videos on TikTok and Bilibili: cross-sectional study in China. Front Public Health 2023; 11: 1296386.

Yang

. Digital health literacy: bibliometric analysis. J Med Internet Res 2022; 24: e35816.

Juarez

Hurria

Uman

, et al. Impact of a bilingual education intervention on the quality of life of Latina breast cancer survivors. Oncol Nurs Forum 2013; 40: E50–E60.

10.

Kong

Song

Zhao

, et al. TikTok as a health information source: assessment of the quality of information in diabetes-related videos. J Med Internet Res 2021; 23: e30409.

11.

Song

Xue

Zhao

, et al. Short-video apps as a health information source for chronic obstructive pulmonary disease: information quality assessment of TikTok videos. J Med Internet Res 2021; 23: e28318.

12.

Zheng

Ning

Levoska

, et al. Acne and social media: a cross-sectional study of content quality on TikTok. Pediatr Dermatol 2021; 38: 336–338.

13.

Basch

Hillyer

Jaime

. COVID-19 on TikTok: harnessing an emerging social media platform to convey important public health messages. Int J Adolesc Med Health 2022; 34: 367–369.

14.

Basch

Meleo-Erwin

Fera

, et al. A global pandemic in the time of viral memes: COVID-19 vaccine misinformation and disinformation on TikTok. Hum Vaccines Immunother 2021; 17: 2373–2377.

15.

Schmidt

Zollo

Scala

, et al. Polarization of the vaccination debate on Facebook. Vaccine 2018; 36: 3606–3612.

16.

Chou

W-YS

Klein

WMP

. Addressing health-related misinformation on social media. JAMA 2018; 320: 2417–2418.

17.

Stellefson

Shuster

Chaney

, et al. Web-based health information seeking and eHealth literacy among patients living with chronic obstructive pulmonary disease (COPD). Health Commun 2018; 33: 1410–1424.

18.

Goobie

Guler

Johannson

, et al. YouTube videos as a source of misinformation on idiopathic pulmonary fibrosis. Ann Am Thorac Soc 2019; 16: 572–579.

19.

Shoemaker

Wolf

Brach

. Development of the patient education materials assessment tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014; 96: 395–403.

20.

Yeung

Abi-Jaoude

. TikTok and attention-deficit/hyperactivity disorder: a cross-sectional study of social Media content quality. Can J Psychiatry 2022; 67: 899–906.

21.

Yang

Zhan

. Is TikTok a high-quality source of information on thyroid cancer? Endocrine 2023; 81: 270–276.

22.

Singh

. YouTube for information on rheumatoid arthritis—a wakeup call? J Rheumatol 2012; 39: 899–903.

23.

Bernard

Langille

Hughes

, et al. A systematic review of patient inflammatory bowel disease information resources on the world wide web. Am J Gastroenterol 2007; 102: 2070–2077.

24.

Nagpal

SJS

Karimianpour

Mukhija

, et al. YouTube videos as a source of medical information during the Ebola hemorrhagic fever epidemic. SpringerPlus 2015; 4: 457.

25.

Korda

Itani

. Harnessing social media for health promotion and behavior change. Health Promot Pract 2013; 14: 15–23.

26.

Filion

. The signed Kolmogorov–Smirnov test: why it should not be used. Gigascience 2015; 4: 9.

27.

Wei

. The adoption of repeated measurement of variance analysis and Shapiro–Wilk test. Front Med 2022; 16: 659–660.

28.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

29.

Donner

Eliasziw

Shoukri

. Review of inference procedures for the interclass correlation coefficient with emphasis on applications to family studies. Genet Epidemiol 1998; 15: 627–646.

30.

Knapp

Madden

Wang

, et al. Internet use and eHealth literacy of low-income parents whose children have special health care needs. J Med Internet Res 2011; 13: e75.

31.

Berkman

Sheridan

Donahue

, et al. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med 2011; 155: 97–107.

32.

Falisi

Wiseman

Gaysynsky

, et al. Social media for breast cancer survivors: a literature review. J Cancer Surviv 2017; 11: 808–821.

33.

Döbrössy

Girasek

Susánszky

, et al. “Clicks, likes, shares and comments” a systematic review of breast cancer screening discourse in social media. PLoS One 2020; 15: e0231422.

34.

Wolfers

Utz

. Social media use, stress, and coping. Curr Opin Psychol 2022; 45: 101305.

35.

Hani

Marwan

Andre

. The effect of celebrity endorsement on consumer behavior: case of the Lebanese jewelry industry. Arab Econ Bus J 2018; 13: 190–196.

36.

Vraga

Stefanidis

Lamprianidis

, et al. Cancer and social media: a comparison of traffic about breast cancer, prostate cancer, and other reproductive cancers on Twitter and Instagram. J Health Commun 2018; 23: 181–189.

37.

Xiong

Young

Fernandez

, et al. Understanding breast cancer surgery through TikTok: an analysis of video content and viewer engagement. Ann Surg Oncol2024; 31(7): 4182–4184.

38.

Pan

, et al. Xigua video as a source of information on breast cancer: content analysis. J Med Internet Res 2020; 22: e19668.

39.

Morena

Ahisar

Wang

, et al. Content quality of YouTube videos about metastatic breast cancer in young women: systematic assessment. JMIR Cancer 2023; 9: e45101.

40.

Choi

. Older adults' credibility assessment of online health information: an exploratory study using an extended typology of web credibility. Assoc Inf Sci Technol 2020; 71: 1295–1307.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB

0.10 MB