Evaluating Online Information for Young Adult Cancer Symptoms: A Content Analysis of Top Google-Ranked Webpages

Abstract

Background:

Among young adults (20–39), cancer is the fifth leading cause of death. Delayed diagnoses in this population are frequent, contributing to reduced survival and higher morbidity. Delays may be driven by individuals attributing symptoms as nonserious and failing to seek timely medical care. Google search is commonly used for health information seeking, but we do not know the current online symptom content quality that young adults may encounter.

Purpose:

We aimed to answer 1. What is the content quality of top-ranked webpages for common young adult cancer symptom searches? 2. Does quality differ by website type (e.g., academic/health care vs. for profit)?

Methods:

Using 18 young adult cancer symptoms as input into the SEMRush Keyword Magic Tool, we generated a list of the most common keyword searches and the top-ranked webpages (i.e., first three pages listed in Google output). We evaluated 162 pages on 9 quality metrics, including the JAMA benchmark criteria.

Results:

Two-thirds of pages (66.7%, n = 108) were written at less than a 9th-grade reading level, and three-quarters (72.8%, n = 118) provided actionable content about when to seek medical care for symptoms. However, only 13.6% (n = 22) of pages included content framed for young adults. On average, pages met about half (2.33) of four JAMA criteria (authorship, disclosures, currency/up-to-date, and references).

Conclusion:

Academic/health and government organizations should devote resources to improving information about young adult cancer symptoms on their webpages and optimize these pages to appear higher in search result rankings.

Keywords

young adult cancer symptom appraisal online information seeking website optimization

Introduction

Young adult (20–39 years) cancer incidence has increased in the United States (US) in recent decades,^1–3 and cancer is now the fifth leading cause of death for young adults.⁴ Moreover, this age group is more likely than children or older adults to experience delayed cancer diagnoses.^1,2,5–12 Delays often result in later-stage disease at diagnosis, fewer treatment options, poorer survival, greater comorbidities, and worse quality of life.^1,2,13–17 Delayed cancer diagnoses can be driven by individuals attributing abnormal physical symptoms to nonserious causes and thus perceiving low urgency for seeking medical evaluation.^18–26 This process of evaluating whether a bodily change is normal or abnormal, attributing it to a possible cause, and deciding whether it warrants medical attention is often referred to as symptom appraisal.^19–21 Among other delay factors, young adults may delay care seeking based on insufficient cancer symptom knowledge and low perceived susceptibility to serious disease,^8,27–31 both of which can be compounded by online information that is inaccurate, unhelpful, or promotes nonurgency of care seeking.^32–43

To identify opportunities for encouraging young adults to seek timely care for symptoms that could be due to cancer, we need to evaluate the landscape of online cancer symptom-related content to understand the quality of information that online searchers are currently encountering. Symptom appraisal and care-seeking decisions are increasingly shaped by information found online^{32–36,44–46} and in a nationally representative survey, more than 70% of US adults chose the internet as their first source when they need health information.⁴⁷ Young adults use the internet to find health information at higher rates than older adults^48–50 and, before a cancer diagnosis, may use the internet to manage uncertainty about symptoms, including forming attributions about the cause, appraising the severity, and determining the urgency of seeking care.⁵¹ Online health information can serve as a barrier or facilitator to timely medical care seeking for symptoms, potentially based on content quality.^37–43 Some research suggests online health information seekers may be more empowered to visit medical professionals than those who do not seek health information online.⁴³ Conversely, nearly half of newly diagnosed colorectal cancer patients recalled finding information online before diagnosis that persuaded them to believe their symptoms were not serious, and consequently, they delayed seeking medical care.⁵² Evaluating the quality of online content most likely to be encountered by young adults searching about symptoms is therefore an important step toward identifying opportunities to improve the information shaping their appraisal and care-seeking decisions.

Although researchers have analyzed Google (the most common online search tool)⁵³ search result rankings (i.e., the order in which webpages are listed in search results) and webpage content quality for other health conditions and populations,^54–59 there is a lack of knowledge about the quality of webpages most likely to be accessed by young adults searching for information about symptoms that could be caused by cancer. To evaluate online information about symptoms common in young adult cancer, we need to identify the webpages most likely to be returned in Google search results (i.e., top-ranked) and assess the pages for key quality indicators. While there is not a universal online content quality definition or standardized set of criteria, quality analyses of health webpages often focus on dimensions related to readability and actionability for a specific audience or disease,^60–64 credibility,^60,65–69 and efforts to minimize risk of exposure to misinformation.^70,71 We evaluated nine criteria drawn from these established dimensions and based on prior research linking them to whether health information is understood, trusted, and acted upon^{45,46,52,72–75}: readability, actionability, young adult framing, exclusion of personal testimonials, and exclusion of third-party advertising, as well as the four JAMA benchmark criteria, namely authorship, disclosures, currency/up-to-date, and references, which are among the most widely used and validated standards for evaluating the quality and credibility of online health information.^76,77 A webpage meeting more of these criteria is better positioned to provide young adults with information that is readable, trustworthy, and actionable for care-seeking decisions. Readability ensures content is accessible to a lay audience, while actionability includes clear guidance on when to seek care.⁴⁰ Young adult framing, an observable content-level indicator of audience targeting, is important because targeted health information is more likely to be perceived as relevant and acted upon than generic content,⁷⁸ and webpages lacking age-specific framing may be dismissed as inapplicable by young adults. Credibility criteria such as providing peer-reviewed references, displaying a medically credentialed author, maintaining current information, and disclosing ownership increase trustworthiness,^73–75 and excluding personal testimonials and third-party advertising minimizes exposure to nonevidence-based information.^70,71

In this study, we aimed to elucidate opportunities for reputable, evidence-based sources, such as academic/health care institutions and government organizations, to improve their webpage content by answering two research questions: 1. What is the content quality of top-ranked webpages for common young adult cancer symptom Google searches? 2. Does content quality differ by type of website (academic/health care vs. for-profit and government vs. for-profit)? By focusing on priority areas, academic/health care and government organizations can more strategically allocate resources.

Materials and Methods

Identifying webpages for assessment

To generate a list of Google’s top-ranked (i.e., listed first below sponsored content) young adult cancer symptom-related webpages for content quality evaluation, we followed a four-step process. First, we identified 18 symptom words and phrases based on empirical evidence of symptoms most common in young adult cancer⁷⁹ (see Supplementary Table S1), as the cancers most common in young adults produce a distinct aggregate symptom profile that differs from those predominating in older adults.^1,2,79 Second, to determine which terms are most frequently used in searches related to these symptoms, we entered each of the 18 symptom words/phrases into the SEMrush Keyword Magic Tool, often used in Google search ranking studies.⁷⁰ When a word or phrase is entered into SEMrush Keyword Magic, the tool provides the monthly search volume of keywords/phrases related to that word/phrase (i.e., the frequency with which people entered a certain term or phrase into Google search). SEMrush aggregates search data from all users, and results are not based on an individual user’s variation in browser history, location, or login status, and thus provide a generalizable basis for identifying top-ranked webpages. Third, we recorded the top three most used keywords for each of the 18 symptom words/phrases. When recording the top three most used keywords, we excluded those unrelated to young adult cancer symptom health concerns (e.g., “blood in dog poop”) or those that were a verbatim duplicate. During this step, we also recorded the search volume for each keyword to generate an overall context of frequency of the evaluated searches. Fourth, we recorded the top three highest-ranked webpages (i.e., the pages that appear at the top of the Google search results page) for each of the top three keywords. We evaluated the top three webpages as those pages, on average, receive more than two-thirds of all clicks from the Google search result page.⁸⁰ We categorized each specific webpage based on its overall website type (government, academic/health care organization, for-profit, or other) using the “about” page. As an example of our four-step process: 1. “Lump” is a common cancer symptom in young adults⁷⁹ and thus was included in our list of 18 author-identified terms. 2. We entered “lump” into the Keyword Magic Tool. 3. We recorded the top three most frequently used keywords related to lump, one of which was “lump in armpit” (which received an average volume of 18,100 searches per month). 4. We evaluated the content quality of the three highest Google-ranked webpages for “lump in armpit,” one of which was https://www.healthline.com/health/armpit-lump.

Quality assessment procedure

We generated a list of webpages in May 2023 and conducted content evaluation in May and June 2023. The 54 keywords (i.e., the top three most frequently used keywords for each of the 18 symptom word/phrases) yielded 162 webpages (i.e., top three ranked webpages for each of the 54 keywords). To assess content quality of the 162 webpages, the first author scored each webpage on nine quality dimensions (see Table 1). We weighted all quality dimensions equally because empirical evidence on the relative influence of each dimension on symptom appraisal and care-seeking behavior is limited. Each dimension was worth 1 point (i.e., a yes/no rating of whether it met a criterion), and each webpage received a total quality score of 0–9, where 9 means the webpage includes all the quality dimensions. We assessed readability using the Flesch–Kincaid grade-level analysis (Readable.com); 1 point was awarded if the page was below a 9th-grade reading level. We coded actionability as present if the page included any mention of when to see a doctor or when to seek medical care for the symptom, regardless of how prominently this was featured. We coded young adult framing as present if the page included any language directing the content toward young adults, such as references to people in their 20 seconds or 30 seconds or use of terms such as “young adults” or “young people,” whether appearing in the body text, subheadings, or callout boxes and regardless of whether the underlying medical content differed from that provided for other age groups. Our assessment also included the four JAMA benchmark criteria^75–77: authorship (the page provided an author or content reviewer name with medical credentials), disclosures (the website “about” page clearly listed ownership/affiliations and contact information), currency/up-to-date (webpage clearly states that they have reviewed it within one year of May 26, 2023), and references (webpage provides a list of peer-reviewed citations, not just hyperlinks to other pages). We also assigned a point each for excluding personal testimonials and excluding third-party advertising links.

Table 1.

Percentage of Webpages Meeting each Quality Criterion Overall and by Website Type

Criteria	Overall^a (N = 162)	Academic/health care (n = 78)	Government (n = 13)	For-profit (n = 56)
JAMA benchmark
Authorship	43.2 (70)	26.9 (21)***	7.7 (1)***	78.6 (44)
References	63.0 (102)	66.7 (52)	30.8 (4)**	73.2 (41)
Disclosure	84.0 (136)	94.9 (74)***	92.3 (12)	69.6 (39)
Currency/up-to-date	43.2 (70)	43.6 (34)	30.8 (4)	51.8 (29)
Total JAMA Score [0–4], mean (SD)	2.3 (1.0)	2.2 (0.7)**	1.6 (1.0)**	2.7 (1.1)
Other quality criteria
Excluded testimonials	95.7 (155)	96.2 (75)	100 (13)	100 (56)
Excluded advertising	42.6 (69)	46.2 (36)**	100 (13)***	21.4 (12)
When to see doctor	72.8 (118)	85.9 (67)**	61.5 (8)	60.7 (34)
Less than 9th-grade reading level	66.7 (108)	62.8 (49)	53.9 (7)	71.4 (40)
Young adult framing	13.6 (22)	18.0 (14)	0 (0)	8.9 (5)
Total Quality Score [0–9], mean (SD)	5.3 (1.4)	5.4 (1.0)	4.8 (1.6)	5.4 (1.4)

9% (n = 15) of evaluated webpages were not academic/health care, government, or for-profit pages. These included nonprofit organizations, blogs/forums, and professional associations. They are included in the overall scores.

**p < 0.01, ***p < 0.001.

Significant differences compared with for-profit pages with chi-square tests of independence for individual criteria and t tests for JAMA and total quality score.

Data analysis

We summarized each quality dimension, the JAMA benchmark quality score, and the total quality score for the webpages overall and for the three website types of interest (for-profit, academic/health care, government). We used t tests to determine if JAMA score or overall quality score varied by website type (for-profit vs. academic/health care, and for-profit vs. government). We used chi-squared tests to analyze if academic/health care or government webpages differ from for-profit on specific criterion. All analyses were conducted using Stata version 17.0 (Stata Corp., College Station, TX).

Results

Sample characteristics

Average monthly search volume for the 54 symptom keywords ranged from 30 to 201,000, with an overall average of 23,516 searches per month (Supplementary Table S1). Nearly half of the 162 webpages were published by academic/health organizations (48.1%, n = 78), for-profit organizations comprised about one-third of all webpages (34.6%, n = 56), and government webpages made up only 8.0% (n = 13). The remaining 9.3% (n = 15) were published by nonprofit organizations, news and broadcast media companies, professional associations, or were forums/blogs. Across all website types, Mayo Clinic (13.0%, n = 21) and Cleveland Clinic (12.3%, n = 20) were the top publishers of webpages appearing among the top search results. Mount Sinai, Very Well Health, and Medical News Today were tied for the third most frequent publisher (5.6%, n = 9).

What is the content quality of top-ranked webpages for common young adult cancer symptom Google searches?

Table 1 presents the percentage of pages meeting each quality criterion overall and by website type. About two-thirds of webpages were written at less than a 9th-grade reading level or less (66.7%, n = 108). Nearly three-quarters included recommendations on when to see a doctor (72.8%, n = 118), but only 13.6% (n = 22) of webpages included young adult framing. Most disclosed website ownership (84.0%, n = 136) and included a peer-reviewed reference list (63.0%, n = 102), while less than half displayed an author (43.2%, n = 70), maintained up-to-date information (43.2%, n = 70). Nearly all excluded testimonials (95.7%, n = 155), but fewer than half excluded advertising (42.6%, n = 69).

The mean total quality score was 5.3 out of 9 (SD = 1.4), indicating that, on average, webpages met just over half of all of the criteria. More than a quarter of webpages (27.2%, n = 44) scored below the midpoint of 4.5. For the four JAMA benchmark indicators (authorship, references, disclosure, currency/up-to-date), the mean score was 2.3 out of 4 (SD = 1.0), indicating that, on average, webpages were meeting only a little more than half of the JAMA benchmark indicators. Nearly a quarter of webpages (21.6%, n = 35) scored below the midpoint on the JAMA benchmark indicators.

Does content quality differ by type of website?

Academic/health care websites

Less than two-thirds of academic/health care webpages were written at a 9th-grade reading level or less (62.8%, n = 49). Most specified when to see a doctor (85.9%, n = 67), but only 18.0% (n = 14) included young adult framing. Nearly all academic/health webpages disclosed website ownership (94.9%, n = 74), while 66.7% included a peer-reviewed reference list (n = 52), 43.6% maintained current/up-to-date information (n = 34), and only 26.9% displayed a medically credentialed author/reviewer (n = 21). Less than half of academic/health care webpages excluded advertising (46.2%, n = 36), but nearly all excluded testimonials (94.9%, n = 74). Of all website types, academic/health care organization webpages had the highest overall mean quality score, with 5.4 (SD = 1.0) out of 9 quality criteria met on average. The academic/health care mean JAMA benchmark score was 2.3 (SD = 0.7) out of 4.

Compared with for-profit webpages, academic/health care webpages were more likely to disclose website ownership (χ²(1) = 15.7, p < 0.001), exclude advertising (χ²(1) = 8.7, p = 0.003), and recommend when to seek health care (χ²(1) = 11.4, p = 0.001), but had a lower overall JAMA score (t(132) = 2.70, p = 0.008, d = 0.46) and were less likely to include a credentialed author (χ²(1) = 34.8, p < 0.001) than for-profit webpages.

Government websites

A little more than half of government webpages were written at less than a 9th-grade reading level (53.9%, n = 7). About 61.5% (n = 8) specified when to see a doctor, but none included young adult framing. Most disclosed website ownership (92.3%, n = 12), while less than a third included references (30.8%, n = 4) and were kept current/up-to-date (30.8%, n = 4). Only one (7.7%) included a credentialed content author/reviewer. All government websites (n = 13) excluded testimonials and advertising. Government webpages had the lowest mean overall quality score with 4.8 out of 9 (SD = 1.6) and the lowest overall mean JAMA benchmark score with 1.6 out of 4 (SD = 1.0). Compared with for-profit webpages, government sites were more likely to exclude advertising (χ²(1) = 28.2, p < 0.001) but less likely to include a credentialed author (χ²(1) = 23.4, p < 0.001) and peer-reviewed references (χ²(1) = 8.4, p = 0.004) and had a lower JAMA score (t(67) = 3.45, p = 0.001, d = 1.07) than for-profit webpages.

Discussion

The present findings indicate there are several opportunities for academic/health and government organizations to improve the content reached by young adults seeking information online about symptoms that could be caused by cancer. These organizations are uniquely positioned to prioritize evidence-based health information over profit-driven interests, though in the present study, no single website type uniformly outperformed others across all quality dimensions. For-profit webpages performed better on authorship than both academic/health care and government webpages and were more likely to include peer-reviewed references than government webpages, while academic/health care webpages were stronger on disclosure, actionability, and excluding third-party advertising than for-profit webpages. Overall, webpages of all types most commonly fell short on young adult framing, displaying a credentialed author, maintaining current information, and excluding third-party advertising. Academic/health care and government organizations that strategically address these gaps have the potential to produce content that could reach young adults at a critical moment in their symptom appraisal and care-seeking decisions.

Search engine optimization

In the list of top webpages for common young adult symptom searches, academic/health care webpages were the most common, but government pages were rarely among the top three search results. For-profit webpages frequently optimize health content to appear higher in results; therefore, it is also important for academic/health care and government webpages to be seen at the top of symptom searches because they are more likely to prioritize evidence-based, accurate information over commercial interests. Academic and government webpages prioritize educating their readers⁸¹ and are more likely to provide scientifically backed information than for-profit webpages.⁸² These organizations might devote more resources to search engine optimization (SEO) to increase the likelihood of factual information reaching young adults. SEO efforts, such as including certain keywords on webpages and improving a website’s technical aspects, are used to increase the probability that Google’s algorithm lists a page first for specific keyword searches.⁸³ While a comprehensive SEO strategy warrants dedicated personnel and resources, some pages can be improved with minimal resource devotion; for example, a free-to-use website quality assessment tool was developed for medical and health websites and is correlated with better Google search result rankings.⁸⁴

Readability, framing, and actionability

We found several areas where on-page content could be improved to more effectively assist young adult searchers. Only about half of government webpages and two-thirds of academic/health care webpages were written at less than a 9th-grade reading level. On average, United States read at an 8th-grade level.⁸⁵ High-level, hard-to-understand content could cause young adults to get discouraged and fail to engage with and understand critical information. Free online readability checkers (e.g., Readable.com) can assist writers with producing more accessible and understandable content.

Most webpages lacked young adult framing, highlighting a key area for improvement. Young adults encountering generic symptom content with no age-specific framing may not perceive the information as applicable to them,⁷⁸ potentially missing an opportunity to recognize that a symptom warrants evaluation, even when the underlying guidance would be the same regardless of age.

Moreover, while many academic/health care and government pages recommended when to see a doctor, there is still a content improvement opportunity given its importance in the problem of young adult cancer diagnosis delays. Consideration should be given to providing timeframes, for when to contact a medical provider (e.g., symptoms persisting more than two weeks), as well as the prominence of these timeframes on the webpage (e.g., consider moving it toward the top or separating out in a callout box).

Credibility

Organizations should seek to meet the four JAMA benchmark criteria (authorship, disclosures, currency, references) when possible, as these can enhance the trustworthiness and credibility of health websites.^73–75 Consumers use the currency of information, or whether it has been recently updated, to evaluate its reliability,⁷³ but less than half of academic and less than a third of government pages included a within one-year review date on the page. In addition, government and academic/health care webpages usually did not list a medically credentialed author, although this may not be possible in the case of government copyright ownership. However, providing the name of a medically credentialed author (or content reviewer) may result in greater acceptance of information; in a web-based health content experiment, participants were more accepting of physician-authored advice versus patient-authored.⁷² Moreover, in a systematic review of 37 studies involving consumers’ evaluation of online health information, 84% of studies indicated that consumers judged the quality of content based on if it was written by someone with expertise in that specific subject area.⁷³ In the present study, most websites disclosed ownership, which is associated with increased trust and transparency among health information seekers.⁷³ Finally, providing peer-reviewed references was not common for government organizations. In a nationally representative study, only about one-third of young adults had “a lot” of trust in government sources of cancer information;⁴⁸ for those who are less trusting, providing references may increase trustworthiness.

Minimizing risk of misinformation

Organizations should take efforts to ensure that young adults are not directed to other nonevidence-based webpages via advertisements. More than half of academic/health care pages included third-party advertising linking to other websites; monetizing health care websites may be an increasingly common strategy.⁸⁶ While this could be an opportunity to generate resources for improving reputable online health content, more research is needed on the risk (e.g., the risk of being directed to a different, less reputable website) this poses to searchers with lower e-Health literacy and to understand the degree of control organizations have over the types of ads displayed. In addition, research is needed to understand if the presences or pursuit of advertising influences website content. Personal testimonials can also present misinformation via one-off or not evidence-based experiences; however, nearly all websites exclude these.

Implications and future directions

In an era of increasing reliance on online information to make health decisions, reputable organizations have a responsibility to promote information about key public health problems, such as cancer in young adults. Importantly, even if young adults who are searching for information about concerning medical symptoms do not have cancer, they should still be able to find credible, actionable information about when symptoms require medical evaluation. The nine quality criteria were weighted equally for the present analysis but may carry different implications for young adults seeking information; for example, readability may affect whether content is understood, authorship whether it is trusted, and actionability whether it promotes timely care seeking. Although this study did not directly assess how young adults interpret or act on the content they encounter, the quality dimensions evaluated here have established associations with comprehension, trust, and health decision-making, and there is justification for their potential to influence symptom appraisal and care-seeking behavior.^{45,46,52,72–75}

Several directions for future research emerge from this work. Research examining user-centered outcomes and perceptions of quality would help determine the relative influence of each dimension on young adults’ comprehension, trust, and care-seeking decisions. Future assessments could move beyond the presence of young adult framing to evaluate whether the clinical guidance on symptom webpages is appropriate for this age group. Although individual cancer symptoms are similar across age groups, the cancers most common in young adults, such as thyroid, testicular, and cervical cancers, lymphomas, and sarcomas, are not the most common in older adults.^1,2,79 Some symptom content may warrant age-specific adaptation; for example, cognitive changes or sexual dysfunction may reflect expected aging in older adults but necessitate more prompt evaluation in someone in their 20 seconds or 30 seconds.^87,88 Future research could also examine the quality of other types of online health information that young adults might encounter beyond Google results and webpages, such as social media content and YouTube videos, as well as correlations between online information seeking and cancer diagnosis delays. With the increasing use of artificial intelligence (AI) chat bots for online information searches, including health symptoms, improving online content is urgently important because AI uses published content to generate answers.⁸⁹ Google recently integrated its own AI bot, Gemini, into the search result interface, and now AI-generated answers typically appear above website links.⁹⁰ Research on how AI-generated information in search results impacts online symptom information seeking is needed.

Finally, even when online information prompts young adults to seek medical care, clinical evaluation guidelines may also need to adapt. Clinicians may be slow to suspect cancer in young adult patients, given the low rate of cancer in this age group and the historical framing of cancer as a disease of older adults. For example, blood in the stool in an older adult typically prompts evaluation for colorectal cancer, but in a young adult, the same symptom is often attributed to hemorrhoids or diet, contributing to later-stage diagnosis of the rising number of early-onset colorectal cancers.^51,91,92 The United Kingdom’s NICE NG12 guideline organizes cancer recognition recommendations by symptom rather than by cancer site and supports primary care clinicians in recognizing and referring young people (in addition to children and adults) with symptoms that could be caused by cancer.⁹³

Limitations

This study had some limitations. First, Google does not provide the age of users for specific search terms, so we do not know how many young adults were using the identified top volume keywords. Thus, we restricted our initial keyword search terms to symptoms or words that are related to the most common young adult cancer symptoms to generate the best analyses related to this age group. Although individual symptoms overlap across age groups, this approach ensured that the content we evaluated would be the content young adults are most likely to encounter when searching for information about symptoms that could be caused by the cancers they are at greatest risk of developing. Moreover, the absence of young adult framing on most pages, regardless of who is searching, represents a content gap relevant to this population, given that targeted information is more effective than generic content. In addition, we did not assess if websites contained misinformation or harmful disinformation or which sites are most understandable, actionable, and trusted by young adults. However, validated evaluation tools, such as the JAMA benchmark rating system, are a way to gain a broad picture of content quality across many websites. Finally, content evaluation was conducted by a single coder; future studies might involve multiple coders and measuring interrater reliability.

Conclusions

We highlighted multiple ways to improve content reached by young adults using online information that may inform attributions about symptoms that could be due to cancer, though how young adults interpret and act on this content warrants direct examination in future research. We recommend reputable organizations, such as academic/health care institutions and government agencies, dedicate personnel and resources to improving webpages about cancer symptoms. Specifically, pages should be written at less than a 9th-grade reading level, include actionable information, frame content for young adults, and be updated yearly.

Authors’ Contribution

N.C.A.: Conceptualization, methodology, formal analysis, data curation, visualization, and writing—original draft. H.O.: Supervision, and writing—review and editing.

Supplemental Material

sj-docx-1-jay-10.1177_21565333261464961 — Supplemental material for Evaluating Online Information for Young Adult Cancer Symptoms: A Content Analysis of Top Google-Ranked Webpages

Supplemental material, sj-docx-1-jay-10.1177_21565333261464961 for Evaluating Online Information for Young Adult Cancer Symptoms: A Content Analysis of Top Google-Ranked Webpages by Natasha C. Allard, and Heather Orom

Footnotes

Acknowledgment

The authors thank Daniel Cave for his mentorship on all things Google.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

The authors declare that there was no funding for this research.

References

1.American Cancer Society. Special Section: Cancer in Adolescents and Young Adults. American Cancer Society; 2020.

2. Miller

, Fidler-Benaoudia

, Keegan

, et al. Cancer statistics for adolescents and young adults, 2020. CA Cancer J Clin 2020;70(6):443–459; doi: 10.3322/caac.21637

3. Bleyer

. Increasing cancer in adolescents and young adults: Cancer types and causation implications. J Adolesc Young Adult Oncol 2023;12(3):285–296; doi: 10.1089/jayao.2022.0134

4.Centers for Disease Control and Prevention NCfIPaC. WISQARS Leading Causes of Death Visualization Tool. Available from: https://wisqars.cdc.gov/lcd/ [Last accessed: August 2, 2025].

5. Bleyer

, Barr

. Cancer in young adults 20 to 39 years of age: Overview. Semin Oncol 2009;36(3):194–206; doi: 10.1053/j.seminoncol.2009.03.003

6. Koo

, Lyratzopoulos

, Herbert

, et al. Association of self-reported presenting symptoms with timeliness of help-seeking among adolescents and young adults with cancer in the BRIGHTLIGHT Study. JAMA Netw Open 2020;3(9):e2015437; doi: 10.1001/jamanetworkopen.2020.15437

7. Forster

, Herbert

, Koo

, et al. Associations between diagnostic time intervals and health-related quality of life, clinical anxiety and depression in adolescents and young adults with cancer: Cross-sectional analysis of the BRIGHTLIGHT cohort. Br J Cancer 2022;126(12):1725–1734; doi: 10.1038/s41416-022-01698-6

8. Herbert

, Lyratzopoulos

, Whelan

, et al. Diagnostic timeliness in adolescents and young adults with cancer: A cross-sectional analysis of the BRIGHTLIGHT cohort. Lancet Child Adolesc Health 2018;2(3):180–190.

9. Dommett

, Pring

, Cargill

, et al. Achieving a timely diagnosis for tagers and young adults with cancer: The ACE “too young to get cancer?” study. BMC Cancer 2019;19(1):616.

10.

10. Miller

, Stal

, Gallagher

, et al. Time from diagnosis and correlates of health-related quality of life among young adult colorectal cancer survivors. Cancers (Basel) 2021;13(16):4045; doi: 10.3390/cancers13164045

11.

11. Barr

, Ferrari

, Ries

, et al. Cancer in adolescents and young adults: A narrative review of the current status and a view of the future. JAMA Pediatr 2016;170(5):495–501; doi: 10.1001/jamapediatrics.2015.4689

12.

12. Fidler

, Gupta

, Soerjomataram

, et al. Cancer incidence and mortality among young adults aged 20-39 years worldwide in 2012: A population-based study. Lancet Oncol 2017;18(12):1579–1589; doi: 10.1016/s1470-2045(17)30677-0

13.

13. Bellizzi

, Smith

, Schmidt

, Adolescent and Young Adult Health Outcomes and Patient Experience (AYA HOPE) Study Collaborative Group. et al. Positive and negative psychosocial impact of being diagnosed with cancer as an adolescent or young adult. Cancer 2012;118(20):5155–5162; doi: 10.1002/cncr.27512

14.

14. Chao

, Bhatia

, Xu

, et al. Incidence, risk factors, and mortality associated with second malignant neoplasms among survivors of adolescent and young adult cancer. JAMA Netw Open 2019;2(6):e195536; doi: 10.1001/jamanetworkopen.2019.5536

15.

15. Chao

, Xu

, Bhatia

, et al. Cardiovascular Disease Risk Profiles in Survivors of Adolescent and Young Adult (AYA) Cancer: The Kaiser permanente AYA Cancer Survivors Study. J Clin Oncol 2016;34(14):1626–1633; doi: 10.1200/jco.2015.65.5845

16.

16. Lee

, DuBois

, Coccia

, et al. Increased risk of second malignant neoplasms in adolescents and young adults with cancer. Cancer 2016;122(1):116–123; doi: 10.1002/cncr.29685

17.

17. Olsson

, Steineck

, Enskär

, et al. Sexual function in adolescent and young adult cancer survivors—a population-based study. J Cancer Surviv 2018;12(4):450–459; doi: 10.1007/s11764-018-0684-x

18.

18. Bish

, Ramirez

, Burgess

, et al. Understanding why women delay in seeking help for breast cancer symptoms. J Psychosom Res 2005;58(4):321–326.

19.

19. Walter

, Webster

, Scott

, et al. The Andersen Model of Total Patient Delay: A systematic review of its application in cancer diagnosis. J Health Serv Res Policy 2012;17(2):110–118; doi: 10.1258/jhsrp.2011.010113

20.

20. Scott

, Walter

, Webster

, et al. The model of pathways to treatment: Conceptualization and integration with existing theory. Br J Health Psychol 2013;18(1):45–65; doi: 10.1111/j.2044-8287.2012.02077.x

21.

21. Andersen

, Cacioppo

. Delay in seeking a cancer diagnosis: Delay stages and psychophysiological comparison processes. Br J Soc Psychol 1995;34(Pt 1):33–52; doi: 10.1111/j.2044-8309.1995.tb01047.x

22.

22. Moodley

, Cairncross

, Naiker

, et al. From symptom discovery to treatment—Women’s pathways to breast cancer care: A cross-sectional study. BMC Cancer 2018;18(1):312; doi: 10.1186/s12885-018-4219-7

23.

23. Quinn-Scoggins

, Cannings-John

, Moriarty

, et al. Cancer symptom experience and help-seeking behaviour during the COVID-19 pandemic in the UK: A cross-sectional population survey. BMJ Open 2021;11(9):e053095.

24.

24. Hamilton

, Walter

, Rubin

, et al. Improving early diagnosis of symptomatic cancer. Nat Rev Clin Oncol 2016;13(12):740–749; doi: 10.1038/nrclinonc.2016.109

25.

25. Ford

, Farah

. Early detection and diagnosis of oral cancer: Strategies for improvement. J Cancer Policy 2013;1(1–2):e2–e7; doi: 10.1016/j.jcpo.2013.04.002

26.

26. Venchiarutti

, Pho

, Clark

, et al. A qualitative exploration of the facilitators and barriers to early diagnosis and treatment of head and neck cancer: Perceptions of patients and carers. Eur J Cancer Care (Engl) 2022;31(6):e13718; doi: 10.1111/ecc.13718

27.

27. Miedema

, Easley

, Hamilton

. Young adults’ experiences with cancer: Comments from patients and survivors. Can Fam Physician 2006;52(11):1446–1447.

28.

28. Hart

, Cowie

, Jesudason

, et al. Adolescents and young adults’ (AYA) views on their cancer knowledge prior to diagnosis: Findings from a qualitative study involving AYA receiving cancer care. Health Expect 2021;24(2):307–316; doi: 10.1111/hex.13170

29.

29. Gibson

, Pearce

, Eden

, et al. Young people describe their prediagnosis cancer experience. Psychooncology 2013;22(11):2585–2592; doi: 10.1002/pon.3325

30.

30. Kent

, Parry

, Montoya

, et al. “You’re too young for this”: Adolescent and young adults’ perspectives on cancer survivorship. J Psychosoc Oncol 2012;30(2):260–279; doi: 10.1080/07347332.2011.644396

31.

31.US Department of Health and Human Services. Closing the Gap: Research and Care Imperatives for Adolescents and Young Adults with Cancer. Report of the Adolescent and Young Adult Oncology Progress Review Group. US Department of Health and Human Services: 2006.

32.

32. White

, Horvitz

. Cyberchondria: Studies of the escalation of medical concerns in web search. ACM Trans Inf Syst 2009;27(4):1–37.

33.

33. Aoun

, Lakkis

, Antoun

. Prevalence and outcomes of web-based health information seeking for acute symptoms: Cross-sectional study. J Med Internet Res 2020;22(1):e15148.

34.

34. Avery

, Ghandi

, Keating

. The ‘Dr Google’ phenomenon–missed appendicitis. N Z Med J 2012;125(1367):135–137.

35.

35. Bujnowska-Fedak

, Węgierek

. The impact of online health information on patient health behaviours and making decisions concerning health. Int J Environ Res Public Health 2020;17(3):880.

36.

36. Kuehn

. More than one-third of US individuals use the Internet to self-diagnose. JAMA 2013;309(8):756–757.

37.

37. Marcu

, Muller

, Ream

, et al. Online information-seeking about potential breast cancer symptoms: Capturing online behavior with an internet browsing tracking tool. J Med Internet Res 2019;21(2):e12400; doi: 10.2196/12400

38.

38. Hochberg

, Allon

, Yom-Tov

. Assessment of the frequency of online searches for symptoms before diagnosis: Analysis of archival data. J Med Internet Res 2020;22(3):e15065; doi: 10.2196/15065

39.

39. Crocco

, Villasis-Keever

, Jadad

. Analysis of cases of harm associated with use of health information on the internet. JAMA 2002;287(21):2869–2871.

40.

40. Fridman

, Johnson

, Elston Lafata

. Health information and misinformation: A framework to guide research and practice. JMIR Med Educ 2023;9:e38687; doi: 10.2196/38687

41.

41. Wang

, Shi

, Kong

. Online health information seeking: A review and meta-analysis. Health Commun 2021;36(10):1163–1175; doi: 10.1080/10410236.2020.1748829

42.

42. Swire-Thompson

, Lazer

. Public health and online misinformation: Challenges and recommendations. Annu Rev Public Health 2020;41(1):433–451.

43.

43. Suziedelyte

. How does searching for health information on the Internet affect individuals’ demand for health care services? Soc Sci Med 2012;75(10):1828–1835; doi: 10.1016/j.socscimed.2012.07.022

44.

44. Jia

, Pang

, Liu

. Online health information seeking behavior: A systematic review. MDPI 2021.

45.

45. Mueller

, Jay

, Harper

, et al. Web use for symptom appraisal of physical health conditions: A systematic review. J Med Internet Res 2017;19(6):e202; doi: 10.2196/jmir.6755

46.

46. Mueller

, Jay

, Harper

, et al. The role of web-based health information in help-seeking behavior prior to a diagnosis of lung cancer: A mixed-methods study. J Med Internet Res 2017;19(6):e189; doi: 10.2196/jmir.6336

47.

47.(HINTS) HINTS. 2019. Available from: https://hints.cancer.gov/view-questions/question-detail.aspx?PK_Cycle=12&qid=688 [Last accessed: August 10, 2025].

48.

48. Allard

, Orom

. Examining beliefs and information-seeking behaviors of young adults aged 20-39 to help inform cancer prevention communication. Unpublished Manuscript 2022.

49.

49. Amante

, Hogan

, Pagoto

, et al. Access to care and use of the internet to search for health information: Results from the US National Health Interview Survey. J Med Internet Res 2015;17(4):e106; doi: 10.2196/jmir.4126

50.

50. Hunsaker

, Hargittai

, Micheli

. Relationship between internet use and change in health status: Panel study of young adults. J Med Internet Res 2021;23(1):e22051; doi: 10.2196/22051

51.

51. Allard

, Bouchard

, Ford

, et al. Understanding symptom appraisal, help-seeking, and diagnostic delays in young adult cancer: A qualitative study guided by the model of pathways to treatment. Cancer Causes Control.

52.

52. Thomson

, Siminoff

, Longo

. Internet use for prediagnosis symptom appraisal by colorectal cancer patients. Health Educ Behav 2012;39(5):583–588; doi: 10.1177/1090198111423941

53.

53. McTavish

, Harris

, Wathen

. Searching for health: The topography of the first page. Ethics Inf Technol 2011;13(3):227–240; doi: 10.1007/s10676-011-9272-8

54.

54. Nanda

, Hay

, Marchetti

. Analysis of keywords used in internet searches for melanoma information: Observational study. JMIR Dermatol 2021;4(1):e25720; doi: 10.2196/25720

55.

55. Andrzejewski

, Rasberry

, Mustanski

, et al. Sexual and reproductive health web sites: An analysis of content for sexual and gender minority youth. Am J Health Promot 2020;34(4):393–401; doi: 10.1177/0890117119899217

56.

56. Hopkins

, Secrest

. Public health implications of Google searches for sunscreen, sunburn, skin cancer, and melanoma in the United States. Am J Health Promot 2019;33(4):611–615; doi: 10.1177/0890117118811754

57.

57. Ferreira

, Traeger

, Machado

, et al. Credibility, accuracy, and comprehensiveness of internet-based information about low back pain: A systematic review. J Med Internet Res 2019;21(5):e13357; doi: 10.2196/13357

58.

58. Alexander

, Seenan

. Credibility, accuracy, and comprehensiveness of readily available internet-based information on treatment and management of peripheral artery disease and intermittent claudication: Review. J Med Internet Res 2022;24(10):e39555; doi: 10.2196/39555

59.

59. Șulea

, Nădășan

, Ursachi

, et al. What patients find on the internet when looking for information about percutaneous coronary intervention: Multilanguage cross-sectional assessment. J Med Internet Res 2022;24(12):e41219; doi: 10.2196/41219

60.

60. Saleh

, Fisher

, Provencher

, et al. A systematic evaluation of the quality, accuracy, and reliability of internet websites about pulmonary arterial hypertension. Ann Am Thorac Soc 2022;19(8):1404–1413.

61.

61. Jones

, Cassie

, Thompson

, et al. Delivering healthcare information via the internet: Cardiac patients’ access, usage, perceptions of usefulness, and web site content preferences. Telemed J E Health 2014;20(3):223–227; quiz 228.

62.

62. Monton

, Lambert

, Belzile

, et al. An evaluation of the suitability, readability, quality, and usefulness of online resources for family caregivers of patients with cancer. Patient Educ Couns 2019;102(10):1892–1897.

63.

63. Stvilia

, Mon

, Yi

. A model for online consumer health information quality. J Am Soc Inf Sci 2009;60(9):1781–1791.

64.

64. Masoni

, Guelfi

. Evaluating online health information in terms of readability, understandability and actionability. Form@re 2020;20(1):356–367.

65.

65. Kloosterboer

, Yannuzzi

, Patel

, et al. Assessment of the quality, content, and readability of freely available online information for patients regarding diabetic retinopathy. JAMA Ophthalmol 2019;137(11):1240–1245.

66.

66. Guo

W-J

, Wang

W-K

, Xu

, et al. Evaluating the quality, content, and readability of online resources for failed back spinal surgery. Spine (Phila Pa 1976) 2019;44(7):494–502.

67.

67. Riordain

, McCreary

. Head and neck cancer information on the internet: Type, accuracy and content. Oral Oncol 2009;45(8):675–677.

68.

68. Corcelles

, Daigle

, Talamas

, et al. Assessment of the quality of Internet information on sleeve gastrectomy. Surg Obes Relat Dis 2015;11(3):539–544.

69.

69. Cuan-Baltazar

, Muñoz-Perez

, Robledo-Vega

, et al. Misinformation of COVID-19 on the Internet: Infodemiology Study. JMIR Public Health Surveill 2020;6(2):e18444; doi: 10.2196/18444

70.

70. Fu

, Zook

, Spoehr-Labutta

, et al. Search engine ranking, quality, and content of web pages that are critical versus noncritical of human papillomavirus vaccine. J Adolesc Health 2016;58(1):33–39; doi: 10.1016/j.jadohealth.2015.09.016

71.

71. Pennycook

, Epstein

, Mosleh

, et al. Understanding and reducing the spread of misinformation online. ArXiv 2019:1–84.

72.

72. Claggett

, Kitchens

, Paino

, et al. The effects of website traits and medical skepticism on patients’ willingness to follow web-based medical advice: Web-based experiment. J Med Internet Res 2022;24(2):e29275; doi: 10.2196/29275

73.

73. Sun

, Zhang

, Gwizdka

, et al. Consumer evaluation of the quality of online health information: Systematic literature review of relevant criteria and indicators. J Med Internet Res 2019;21(5):e12522; doi: 10.2196/12522

74.

74. Kim

, Eng

, Deering

, et al. Published criteria for evaluating health related web sites: Review. BMJ 1999;318(7184):647–649; doi: 10.1136/bmj.318.7184.647

75.

75. Silberg

, Lundberg

, Musacchio

. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor—Let the reader and viewer beware. JAMA 1997;277(15):1244–1245.

76.

76. Berland

, Elliott

, Morales

, et al. Health information on the Internet: Accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612–2621; doi: 10.1001/jama.285.20.2612

77.

77. Wrigley Kelly

, Murray

, McCarthy

, et al. An objective analysis of quality and readability of online information on COVID-19. Health Technol (Berl) 2021;11(5):1093–1099; doi: 10.1007/s12553-021-00574-2

78.

78. Kreuter

, Wray

. Tailored and targeted health communication: Strategies for enhancing information relevance. Am J Health Behav 2003;27(Suppl 3):S227–S32; doi: 10.5993/ajhb.27.1.s3.6

79.

79. Bleyer

. CAUTION! Consider cancer: Common symptoms and signs for early detection of cancer in young adults. Semin Oncol 2009;36(3):207–212; doi: 10.1053/j.seminoncol.2009.03.004

80.

80. Bailyn

. Google Click-Through Rates (CTRs) by Ranking Position in 2023. SEO Blog; 2023. Available from: https://firstpagesage.com/seo-blog/google-click-through-rates-ctrs-by-ranking-position [Last accessed: August 15, 2025].

81.

81. LaValley

, Kiviniemi

, Gage-Bouchard

. Where people look for online health information. Health Info Libr J 2017;34(2):146–155; doi: 10.1111/hir.12143

82.

82. Merrell

, Cone

, Rafiq

. The authority and utility of Internet information. Stud Health Technol Inform 2008;131:265–272.

83.

83. Berman

, Katona

. The role of search engine optimization in search marketing. Marketing Sci 2013;32(4):644–651.

84.

84. Dunne

, Cummins

, Hannigan

, et al. A method for the design and development of medical or health care information websites to optimize search engine results page rankings on Google. J Med Internet Res 2013;15(8):e183.

85.

85. Cotugna

, Vickery

, Carpenter-Haefele

. Evaluation of literacy level of patient education pages in health-related journals. J Community Health 2005;30(3):213–219; doi: 10.1007/s10900-004-1959-x

86.

86. Bartel

. How Amanda Todorovich and the Cleveland Clinic Monetized Their Blog (And Why the Resulting Revenue is Just Gravy). Top Rank Marketing; 2018. Available from: https://www.toprankmarketing.com/blog/cmworld-amanda-todorovich-cleveland-clinic-blog/ [Last accessed: August 10, 2025].

87.

87. Fulop

, Larbi

, Witkowski

, et al. Aging, frailty and age-related diseases. Biogerontology 2010;11(5)(:547–563.

88.

88. Corona

, Mannucci

, Ricca

, et al. The age‐related decline of testosterone is associated with different specific symptoms and signs in patients with sexual dysfunction. Int J Androl 2009;32(6):720–728.

89.

89. Reardon

. AI Chatbots Can Diagnose Medical Conditions at Home. How Good Are They? 2023. Available from: https://www.scientificamerican.com/article/ai-chatbots-can-diagnose-medical-conditions-at-home-how-good-are-they/ [Last accessed: June 1, 2026].

90.

90. Imran

, Almusharraf

. Google Gemini as a next generation AI educational tool: A review of emerging educational technology. Smart Learn Environ 2024;11(1):22.

91.

91. Lamprell

, Pulido

, Arnolda

, et al. People with early-onset colorectal cancer describe primary care barriers to timely diagnosis: a mixed-methods study of web-based patient reports in the United Kingdom, Australia and New Zealand. BMC Prim Care 2023;24(1):12; doi: 10.1186/s12875-023-01967-0

92.

92. Fritz

CDL

, Otegbeye

, Zong

, et al. Red-flag signs and symptoms for earlier diagnosis of early-onset colorectal cancer. J Natl Cancer Inst 2023;115(8):909–916; doi: 10.1093/jnci/djad068

93.

93.National Institute for Health and Care Excellence (NICE). Suspected cancer: recognition and referral. NICE: London; 2015.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.02 MB