Telling the Whole Story: Bibliometric Network Analysis to Evaluate Impact of Media Attention on Chronic Traumatic Encephalopathy Research

Abstract

There is a national debate regarding the existence of a relationship between contact sport participation and future risk of neurodegenerative disease. We employed bibliometrics and altmetrics to quantify the academic, popular, and social media impact of published scientific articles that report an association between contact sports or military service with chronic traumatic encephalopathy (CTE+), and compare with those scientific articles that report null or no association of contact sports or military service with CTE (CTE-). In this cross-sectional study, we extracted number of citations, total link strength, altmetric score, number of news stories, media outlets, and Twitter interaction from published CTE articles. The top 10 most cited articles were statistically compared on these outcomes using Mann-Whitney U tests. CTE+ publications had an average of 101 citations per article, Altmetric score of 272, 36 news stories in 26 media outlets, and upper-bound of Twitter users of 402,159. CTE- publications had an average of 29 citations per article, Altmetric score of 39, two news stories and media outlets, and upper-bound of Twitter users of 91,070. Top 10 CTE+ publications had, on average, 94% more citations (p < 0.001), 95% higher altmetric scores (p = 0.01), 99% higher number of news stories (p = 0.01), 98% higher number of media outlets (p = 0.01), and reached 95% more Twitter users than top 10 CTE- publications (p = 0.11). The bibliometric analysis indicates a significant inequality in media dissemination and popular consumption of scientific findings that do not support a relationship between contact sports or military service and future neurodegeneration.

Introduction

Bibliometric network analysis provides objective quantification of the impact of scientific work in the peer-reviewed literature. Recently, scientific journals have incorporated additional, alternative metrics of scientific impact to reflect more modern avenues of research dissemination; so-called altmetrics have been developed to measure scholarly work across a broader spectrum of academic and public engagement. This type of analysis is ideally suited to compare the academic and popular media impact of controversial topics in sports medicine. One such topic is chronic traumatic encephalopathy, or CTE, of which scientific and public interest has exploded in the last decade. CTE is a degenerative neurological disease that has captured the public's attention since its association with a deceased National Football League player in 2005.¹ We sought to examine the relationship between media and social media coverage of CTE science and the scientific impact of work in the field.

In 1928 in JAMA, Harrison S. Martland reported the first cases of “punch-drunk syndrome” in boxers,² while MacDonald Critchley coined the term chronic traumatic encephalopathy (CTE) in 1947.³ Scientific interest in CTE has exploded in the last decade. Between 1947-2009, the average number of CTE publications per year was 0.48. The number of annual publications steadily grew to a peak of 113 in 2019; a 96% increase from 2010. Federal funding for CTE research has seen a similar meteoric rise: the National Institute of Health's (NIH's) RePORTER website indicates over $508 million dollars in current CTE research funding from NIH alone.

The scientific literature contains conflicting data on future neurologic risk of contact sports or military service. For example, Mez and colleagues⁴ reported that 99% of former professional football players studied had CTE at autopsy. Savica and colleagues⁵ reported that the incidence of dementia in former high school football players was 1.2%, compared with a 2.2% rate of dementia in glee club members the same age from the same high schools.

Popular and social media attention to CTE has paralleled the growth of scientific research since 2010.⁶ The central question of whether there is a relationship between contact sport participation and future risk of neurodegeneration has spurred a national debate, with many conflicting perspectives. The issue is further charged because traumatic brain injury is a signature injury of modern military conflict, with potential for long-term consequences.

We employed bibliometrics and altmetrics to quantify the academic, popular, and social media impact of published scientific articles that report an association between contact sports or military service with CTE, and compare with those scientific articles that report null or no association of contact sports or military service with CTE.

Methods

Article search

Articles were identified from Web of Science on March 3rd, 2022. Web of Science was utilized as this is the primary database that is compliant with the bibliographic software used in the study. Search terms included “chronic traumatic encephalopathy” OR “CTE” AND (sport OR military) NOT (animal OR mouse OR pig OR rat). The search yielded 785 articles published in 310 academic journals by 3499 authors. Articles were reviewed by three members of the study team. We categorized the findings into two groups based upon criteria by Wolfson and colleagues⁷: reporting an association between contact sports or military exposure with CTE (CTE+) or reporting no definitive conclusions/report no association between contact sports or military exposure with CTE. Because CTE is only diagnosable postmortem, articles also were classified as CTE- if no association was reported between contact sports or military exposure and traumatic encephalopathy syndrome (the in vivo equivalent of CTE) or associated neurodegenerative diseases.

Data extraction

Outcomes extracted from the publications included: 1) year of publication; 2) primary author; 3) primary author institution; 4) population of interest; and 5) publication type. Only data from original research (i.e., not systematic reviews/meta-analyses) were included in the final analyses.

Altmetric scores

Altmetrics is a formula-based continuous variable that is automatically calculated based upon the attention given to a research publication. The formula includes the number of mentions for an article and a weighted count of the number of sources that report on the article.⁷ Altmetrics were retrieved from the publisher's website on March 3, 2022 for number of citations, news stories, media outlets, and Twitter interactions.

Statistical analysis

The top 10 most cited articles with original data were analyzed using Mann-Whitney U tests. The test statistic and p value for each comparison are reported using SPSS version 28. An alpha value of p < 0.05 was considered statistically significant. Citation analyses (i.e., relationship between cited works) were conducted to analyze individual publications and academic organizations who publish CTE research (vosViewer).

Results

Overall, CTE+ publications had an average of 101 citations per article, Altmetric score of 272, 36 news stories in 26 media outlets, and upper bound of Twitter users of 402,159. CTE- publications had an average of 29 citations per article, Altmetric score of 39, two news stories and media outlets, and upper bound of Twitter users of 91,070.

Altmetrics for the top 10 most cited publications for CTE+ and CTE- can be viewed in Table 1.^{1,4,5,8

–24} CTE+ publications had, on average, 94% more citations (Mann-Whitney U = 100.0; p < 0.001), 95% higher altmetric scores (Mann-Whitney U = 52.5; p = 0.01), 99% higher number of news stories (Mann-Whitney U = 70.0; p = 0.01), 98% higher number of media outlets (Mann-Whitney U = 69.5; p = 0.01), and reached 95% more Twitter users than CTE- publications (Mann-Whitney U = 52.0; p = 0.11). Each of the 10 most cited CTE+ publications were published in 2014 or earlier, except for Mez and colleagues,⁴ which was published in 2017.

Table 1.

Altmetric Information for the Top Original Research for CTE+ and CTE− Articles

		Article	Institution	Citations^{^}	Altmetric^{^}	News stories^{^}	Media outlets^{^}	Upper bound of Twitter users	PMID number
CTE+	1	McKee 2013¹¹	Boston University	1180	2276	342	252	615,396	23208308
	2	Omalu 2005¹	University of Pittsburgh	601	427	43	31	691,976	15987548
	3	Goldstein 2012⁸	Boston University	594	104	7	5	2,672,963	22593173
	4	Mez 2017⁴	Boston University	463	4264	628	346	8,125,583	28742910
	5	McKee 2010⁹	Boston University	409	46	3	3	21,781	20720505
	6	Johnson 2012¹⁴	University of Pennsylvania	366	19	1	1	38,218	21714827
	7	Omalu 2006¹³	University of Pittsburgh	319	74	6	5	134	17143242
	8	Omalu 2011¹²	University of California Davis	252	34	1	1	7,073	21358359
	9	McKee 2014¹⁰	Boston University	195	404	19	19	845,025	24924675
	10	Bienek 2015¹⁵	Mayo Clinic	166	325	43	34	1,454,344	26518018
	Average	—	455	797	109	70	1,447,249	—
CTE-	1	Hazrati 2013¹⁶	University of Toronto	95	24	0	0	46,853	23745112
	2	Savica 2012⁵	Mayo Clinic	95	n/a	3	3	255	22469346
	3	Noy 2016¹⁷	University of Manitoba	45	34	1	1	80,285	27815395
	4	Iverson 2019¹⁸	Harvard Medical School	22	152	7	7	312,928	31169877
	5	Laffey 2018²⁰	University of California Los Angeles	8	n/a	n/a	n/a	n/a	30412511
	6	Tripathy 2019²¹	Western Michigan University	8	n/a	n/a	n/a	n/a	30741674
	7	Grashow 2020²²	Harvard Medical School	7	n/a^*	1	1	n/a^**	32281676
	8	Postupna 2021²³	University of Washington	2	1	0	0	6,539	33796061
	9	Iverson 2021¹⁹	Harvard Medical School	1	1	0	0	1,223	33935940
	10	Schaffert 2021²⁴	University of Texas Southwestern	1	12	1	1	30,986	33708171
	Average	—	26	37	2	2	68,724	—

n/a = publisher does not use altmetrics; not included in average calculation.

Publication used PlumX metrics, which does not have a total score.

Publication does not report upper bound of Twitter users; 255 is number of tweets or retweets and was not included in the average calculation.

Statistically significant difference between groups (p < 0.05).

CTE+, positive association between contact sports or military exposure and CTE; CTE−, negative/neutral association between contact sports or military exposure and CTE; PMID, PubMed identifier.

Figure 1 shows the bibliometric network map of individual CTE publications. Boston University had the most publications (122), citations (9000), and total link strength (8570) of all academic institutions (Fig. 2). Other notable institutions were the Veteran Affairs for Boston Healthcare System (publications, 62; citations, 4301; and total link strength, 5368), Harvard Medical School (publications, 47; citations, 728; and total link strength, 2245), Emerson Hospital (publications, 37; citations, 4975; total link strength, 4247) and the Mayo Clinic (publications, 26; citations, 2869; and total link strength, 2014).

FIG. 1.

Bibliometric network map of individual peer-reviewed publications on chronic traumatic encephalopathy. Larger nodes indicate a higher number of citations for the publications. Nodes that are closer together are more interrelated (in terms of citations and common references) than nodes that are further away from each other. To be included in the individual publications network, the article must have been cited 50 times (n = 139). The color of the node is indicative of the year it was published: purple means the article was published in 2012 or earlier, whereas yellow indicates the article was published in 2019 or later.

FIG. 2.

Bibliometric network map of academic institutions who publish chronic traumatic encephalopathy research. Larger nodes indicate a higher number of citations for the publications. Nodes that are closer together are more interrelated (in terms of citations and common references) than nodes that are further away from each other. To be included in the academic organizations network, the institution must have published 10 articles and been cited 100 times (n = 45). The color of the node is indicative of the year it was published: purple means the article was published in 2012 or earlier, whereas yellow indicates the article was published in 2019 or later.

Discussion

Bibliometric analysis reveals a clear distinction between scientific publications reporting associations between contact sports or military service and chronic traumatic encephalopathy and those publications that refute an association. The most cited scientific articles with original data indicating a risk of future traumatic encephalopathy had substantially higher citation rates and popular/social media impact. For example, the most highly cited CTE- publication had 43% fewer citations than the 10th most cited CTE+ publication. The network maps also reveal disparate impact between the organizations and individual articles that report CTE+ research, and the temporal overlay demonstrates the outsized influence of early CTE+ publications from 2005-2013 (Fig. 1). The underlying reason for the difference in media circulation cannot be determined from this data. The bibliometric analysis indicates a significant inequality in media dissemination and popular consumption of scientific findings that do not support a relationship or do not demonstrate a clear positive relationship between contact sports or military service and future neurodegeneration.

Disproportionate media attention to CTE cases may create unintended consequences, changes in physician behavior, and perpetuate public fear due to false perceptions regarding CTE science.^25
-27 Obtaining CTE information from sports news sources increases odds of individuals believing concussions inevitably cause CTE.²⁶ Approximately 10% of collegiate football players believe neurodegenerative disease is a long-term inevitability after concussions.²⁷ Further, prior studies have reported that a meaningful percentage of National Football League (NFL) players were diagnosed with CTE, which can only presently be diagnosed postmortem.²⁸ Previous studies concluded that medical professionals need to increase their knowledge of CTE to improve the likelihood of disseminating balanced and informed information to the public.²⁶ A study by the Centers for Disease Control and Prevention found 70% of parents report healthcare providers as their primary source of CTE information but know little about the disease themselves.²⁹ Medical professionals and researchers must have ownership over the responsibility of communicating relevant findings to their populations and the media.

These data should be interpreted in light of some limitations. The underlying reason for the difference in media circulation cannot be determined from this data. For example, we cannot determine the quality of the evidence between published papers from this information. The difference in number of citations between groups could be related to an older publication year, on average, for CTE+. However, many of the top 10 CTE- articles had comparable publications years to CTE+ articles with far fewer publications. Citations are just one type of bibliographic measure of a scientific work's impact, both for articles themselves and for individual authors. There are no known data regarding the reliability and accuracy of altmetric scores. The journals' impact factor also seemed to be related to CTE+, which could have affected altmetric scores. At the time of analysis, there was too small a sample size for research studies related to head impacts and other neurodegenerative diseases (e.g., Alzheimer's disease, amyotrophic lateral sclerosis) to investigate differences in public interest between these diseases and CTE. Finally, the impact of these findings is limited to the quality of the articles included, which did not often assess specific variables related to head impact exposure.

The state of empirical evidence regarding chronic traumatic encephalopathy is complicated. CTE exists, correlates with a history of head injury and repetitive head impacts, and is currently only diagnosable postmortem.³⁰ Recent attempts to introduce diagnostic criteria for traumatic encephalopathy syndrome (TES; i.e., the precursor of CTE in vivo) are limited by overreliance on self-reported or subjective complaints, clinical impressions, and lack of specificity (by the authors' own admission).^30
-32 These criteria have a high rate of false positives (6.6-45.2%) for TES in populations with anger issues, depression, or chronic pain.^19,33,34 Research efforts are underway to provide objective biological markers for TES in vivo to support subjective reports and clinical presentation, but no valid assessments currently exist.

Conclusion

Bibliometric network analysis revealed that publications that report an association between contact sports or military exposure and chronic traumatic encephalopathy have 94% more scientific citations and 95-99% higher public and social media coverage than publications that do not report this association.

Author Disclosure Statement

No competing financial interests exist.

References

Omalu

, DeKosky

, Minster

, et al. Chronic traumatic encephalopathy in a National Football League player. Neurosurgery, 2005; 57(1):128–134; doi: 10.1227/01.neu.0000163407.92769.ed.

Martland

. Punch drunk. JAMA, 1928; 91(15):1103–1107.

Critchley

Punch-drunk syndromes: the chronic traumatic encephalopathy of boxers. Hommage a Clovis Vincent. 1949:131–141.

Mez

, Daneshvar

, Kiernan

, et al. Clinicopathological evaluation of chronic traumatic encephalopathy in players of American football. JAMA, 2017; 318(4):360–370; doi: 10.1001/jama.2017.8334.

Savica

, Parisi

, Wold

, et al. High school football and risk of neurodegeneration: a community-based study. Mayo Clin Proc, 2012; 87(4):335–40; doi: 10.1016/j.mayocp.2011.12.016.

Smith

, Johnson

, Trojanowski

, et al. Chronic traumatic encephalopathy—confusion and controversies. Nat Rev Neurol, 2019; 15(3):179–183; doi: 10.1038/s41582-018-0114-8.

Wolfson

, Kuhn

, Kerr

, et al. Chronic traumatic encephalopathy research viewed in the public domain: what makes headlines?. Brain Inj, 2020; 34(4):528–534; doi: 10.1080/02699052.2020.1725843.

Goldstein

, Fisher

, Tagge

, et al. Chronic traumatic encephalopathy in blast-exposed military veterans and a blast neurotrauma mouse model. Sci Transl Med, 2012; 4(134):134ra60; doi: 10.1126/scitranslmed.3003716.

McKee

, Gavett

, Stern

, et al. TDP-43 proteinopathy and motor neuron disease in chronic traumatic encephalopathy. J Neuropathol Exp Neurol, 2010; 69(9):918–929; doi: 10.1097/NEN.0b013e3181ee7d85.

10.

McKee

, Robinson

. Military-related traumatic brain injury and neurodegeneration. Alzheimers Dement, 2014; 10(3 Suppl):S242–S253; doi: 10.1016/j.jalz.2014.04.003.

11.

McKee

, Stern

, Nowinski

, et al. The spectrum of disease in chronic traumatic encephalopathy. Brain, 2013; 136(Pt 1):43–64; doi: 10.1093/brain/aws307.

12.

Omalu

, Bailes

, Hamilton

, et al. Emerging histomorphologic phenotypes of chronic traumatic encephalopathy in American athletes. Neurosurgery, 2011; 69(1):173–183; doi: 10.1227/NEU.0b013e318212bc7b.

13.

Omalu

, DeKosky

, Hamilton

, et al. Chronic traumatic encephalopathy in a National Football League player: part II. Neurosurgery, 2006; 59(5):1086–1092; doi: 10.1227/01.Neu.0000245601.69451.27.

14.

Johnson

, Stewart

, Smith

. Widespread τ and amyloid-β pathology many years after a single traumatic brain injury in humans. Brain Pathol, 2012; 22(2):142–149; doi: 10.1111/j.1750-3639.2011.00513.x.

15.

Bieniek

, Ross

, Cormier

, et al. Chronic traumatic encephalopathy pathology in a neurodegenerative disorders brain bank. Acta Neuropathol, 2015; 130(6):877–889; doi: 10.1007/s00401-015-1502-4.

16.

Hazrati

, Tartaglia

, Diamandis

, et al. Absence of chronic traumatic encephalopathy in retired football players with multiple concussions and neurological symptomatology. Front Hum Neurosci, 2013; 7:222; doi: 10.3389/fnhum.2013.00222.

17.

Noy

, Krawitz

, Del Bigio

. Chronic traumatic encephalopathy-like abnormalities in a routine neuropathology service. J Neuropathol Exp Neurol, 2016; 75(12):1145–1154; doi: 10.1093/jnen/nlw092.

18.

Iverson

, Luoto

, Karhunen

, et al. Mild chronic traumatic encephalopathy neuropathology in people with no known participation in contact sports or history of repetitive neurotrauma. J Neuropathol Exp Neurol, 2019; 78(7):615–625; doi: 10.1093/jnen/nlz045.

19.

Iverson

, Merz

, Terry

. Examining the research criteria for traumatic encephalopathy syndrome in middle-aged men from the general population who played contact sports in high school. Front Neurol, 2021; 12:632618; doi: 10.3389/fneur.2021.632618.

20.

Laffey

, Darby

, Cline

, et al. The utility of clinical criteria in patients with chronic traumatic encephalopathy. NeuroRehabilitation, 2018; 43(4):431–441. doi: 10.3233/nre-182452.

21.

Tripathy

, Shade

, Erskine

, et al. No evidence of increased chronic traumatic encephalopathy pathology or neurodegenerative proteinopathy in former military service members: a preliminary study. J Alzheimers Dis, 2019; 67(4):1277–1289; doi: 10.3233/jad-181039.

22.

Grashow

, Weisskopf

, Baggish

, et al. Premortem chronic traumatic encephalopathy diagnoses in professional football. Ann Neurol, 2020; 88(1):106–112; doi: 10.1002/ana.25747.

23.

Postupna

, Rose

, Gibbons

, et al. The delayed neuropathological consequences of traumatic brain injury in a community-based sample. Front Neurol, 2021; 12:624696. doi: 10.3389/fneur.2021.624696.

24.

Schaffert

, Didehbani

, LoBue

, et al. Frequency and predictors of traumatic encephalopathy syndrome in a prospective cohort of retired professional athletes. Front Neurol, 2021; 12:617526; doi: 10.3389/fneur.2021.617526.

25.

Belanger

, Wortzel

, Vanderploeg

, et al. A model for intervening with veterans and service members who are concerned about developing chronic traumatic encephalopathy (CTE). Clin Neuropsychol, 2020; 34(6):1105–1123; doi: 10.1080/13854046.2019.1699166.

26.

Beidler

, Bogar

, Wallace

, et al

The burden of unsubstantiated messaging

: collegiate athletes' chronic traumatic encephalopathy mechanism beliefs. Brain Inj, 2021; 35(10):1259–1266; doi: 10.1080/02699052.2021.1972146.

27.

Baugh

, Kroshus

, Kiernan

, et al. Football players' perceptions of future risk of concussion and concussion-related health outcomes. J Neurotrauma, 2017; 34(4):790-797; doi:10.1089/neu.2016.4585.

28.

Zuckerman

, Yengo-Kahn

, Kuhn

, et al. In reply: recommendation to create new neuropathological guidelines for the postmortem diagnosis of chronic traumatic encephalopathy. Neurosurgery, 2021; 89(6):E336–E337; doi: 10.1093/neuros/nyab347.

29.

Daugherty

, Sarmiento

. Chronic traumatic encephalopathy: what do parents of youth athletes know about it?. Brain Inj, 2018; 32(13-14):1773–1779; doi: 10.1080/02699052.2018.1530801.

30.

Katz

, Bernick

, Dodick

, et al. National Institute of Neurological Disorders and Stroke Consensus Diagnostic Criteria for Traumatic Encephalopathy Syndrome. Neurology, 2021; 96(18):848–863; doi:10.1212/wnl.0000000000011850.

31.

Mez

, Alosco

, Daneshvar

, et al. Validity of the 2014 traumatic encephalopathy syndrome criteria for CTE pathology. Alzheimers Dement, 2021; 17(10):1709–1724; doi:10.1002/alz.12338.

32.

Reams

, Eckner

, Almeida

, et al. A clinical approach to the diagnosis of traumatic encephalopathy syndrome: a review. JAMA Neurol, 2016; 73(6):743–749; doi: 10.1001/jamaneurol.2015.5015.

33.

Iverson

, Gardner

. Risk for misdiagnosing chronic traumatic encephalopathy in men with anger control problems. Front Neurol, 2020; 11:739; doi: 10.3389/fneur.2020.00739.

34.

Iverson

, Gardner

. Risk of misdiagnosing chronic traumatic encephalopathy in men with depression. J Neuropsychiatry Clin Neurosci, 2020; 32(2):139–146; doi: 10.1176/appi.neuropsych.19010021.