Top Altmetric Scores in the Parkinson’s Disease Literature

INTRODUCTION

Social media, such as Facebook and Twitter, have profoundly changed our cultural and societal landscapes, and dramatically altered the way the news is reported and how opinions are conveyed. By 2020, the number of worldwide social media users is expected to reach 2.95 billion, around a third of the Earth’s population [1]. Nearly all aspects of life have been changed by this phenomenon, politics being one example that may come to mind, but the advent of social media has also affected science.

Academics have long relied on citation-related metrics as the main indicators of a publication’s impact [2]. With the advent of social media, a new class of social web-based metrics (“altmetrics”) has surfaced as a fast, format-agnostic complement to traditional indicators, showcasing attention amongst many different stakeholder groups. There is discussion regarding altmetrics’ significance, specifically relating to how altmetrics correlate with traditional, citation-based metrics, as well as concerning its general usefulness [3, 4]. The current view is that even though there may be a correlation between altmetrics and traditional metrics, the two most likely represent different aspects of a publication’s reach [5].

Our aim was to study which recent papers in the field of Parkinson’s disease (PD) had received the highest Altmetric Attention Scores, a weighted score assigned by an altmetrics reporting service, Altmetric, and to compare this to the traditional metrics.

METHODS

Scientific articles indexed in PubMed with the MeSH term “Parkinson’s Disease” or with the keyword “Parkinson” in the title were selected and ranked according to their Altmetric Attention Score using Altmetric Explorer database (Version 2 for Institutions)—updated 29/12/2016. Altmetric Attention Scores are calculated via an automated algorithm created by the company Altmetric, representing a weighted count of the amount of attention a particular paper has received in various online sources. The score is weighted to reflect the relative reach of each type of source. A news story is worth 8 points, a tweet is worth 1 point, and a Facebook post is worth a quarter point. This is based on the assumption that news stories are likely to generate more attention that the single Facebook post. Regarding news sites, each is assigned a tier, which determines the weight of their contribution to the overall attention score meaning that newspapers with worldwide readership contribute more heavily to the score than do local news outlets. Other sources are weighted differently [6]. Articles’ traditional citations were obtained from Scopus, and the two scores were compared qualitatively.

RESULTS

Table 1 shows the 20 publications on PD with the highest Altmetric Scores. Mainstream media mentions and Twitter mentions comprised a majority of the attention that these articles received online. The three highest scoring papers in field of PD have received a considerable amount of attention from news media and Twitter, even though they are fundamentally very different articles. The first (#1) reports a new disease mechanism for PD, establishing a potentially exciting link between intestinal microbiota, motor symptoms and neuro-inflammation in a cleverly designed animal model of PD [7]. The second (#2) reports a significant increase in the incidence of PD over the last decade, which understandably raises concern among policy makers and the general public [8]. In both cases, the scientific merits and public attention went hand in hand. This was somewhat different for the third paper (#3), which describes a unique and remarkably consistent gait pattern (a unilaterally reduced arm swing on the right side, which could potentially signal the presence of early parkinsonism) in a series of five highly ranked Kremlin officials, among them Russian President Vladimir Putin and Prime Minister Medvedev. The worldwide fascination with the Russian President, the excitement of the possible implication of early parkinsonism in elite Russian politicians (as well as the alternative explanation that the absent arm swing was actually a “gunslinger’s gait”, explained by KGB or other forms of weaponry training), the compelling videos that came with the article, and the public attention given to the special edition of the British Medical Journal (where the paper was published) arguably contributed to the paper’s relatively generous amount of online attention [9]. The strict scientific merits were more modest for this paper, although it does draw attention to the presence and importance of the earliest stages of PD.

Fatigue was the subject of two very highly ranked papers (#5, #6). Paper #5 reports on a symposium in which members of the general public participated [10]. This prompted the attention of the Movement Disorders Society with the publication of paper #6, considering fatigue as being one of the most important, albeit neglected aspect of PD [11]. Both were highly reported upon in news outlets, suggesting this is also an important issue for the general public.

The fields most consistently covered in this list are new disease mechanisms (#1, #4, #14, #15) [7, 12–14], treatment options (#8, #9, #11, #12, #16, #17, #19) [15–21] and risk factors for the development of PD (#10, #13, #20) [22–24]. These are topics that rank very highly among the concerns of PD patients and their families, because no curative treatment exists. Other subjects included MRI changes in patients with cognitive impairment associated with PD (#7) [25] and impulse-control disorders associated with dopaminergic agonists (#18) [26].

Virtually all papers were published in journals with high impact factors, indicating a potential correlation between traditional metrics and altmetrics. One exception worth mentioning is the paper in 8th position—dealing with cannabis (medical marijuana), as a possible treatment for motor and non-motor symptoms of PD—with the highest number of raw mentions by far, mainly due to Twitter. The use of medical marijuana is still the subject of lively debate in many parts of the world [27], which likely contributed to the paper’s attention in social media. It is also interesting to note that the study that was published in the New England Journal of Medicine (the journal with the highest Impact Factor in this analysis) ranked only 16th, in the Altmetric-score ranking [19], suggesting that traditional metrics and altmetrics do not measure the same construct, and indeed offer complementary perspectives.

DISCUSSION

Altmetric provides a new perspective on the attention surrounding scholarship. The scores reflect the online attention received by recent scholarly publications. Indeed, the papers included in this analysis on the subject of PD are all newly-published work: all 20 papers were published recently, with the majority having come out in the last two years. Notably, Altmetric started collecting content from publishers only during the second half of 2011 [6], and none of the articles referenced by Ponce and Lozano in their 2011 review of the most influential works in PD [28] made it to the top 20 list of altmetrics. Even though Ponce and Lozano’s paper was published in a time when altmetrics was only starting to collect content from publishers, it would be reasonable to expect that at least some of those very influential publications had received higher Altmetric Attention Scores. This finding emphasizes that altmetrics is particularly sensitive to recent news. This suggests that more recent publications may receive higher Altmetric Attention Scores, and that altmetrics may overlook seminal work previously published in the field.

Contributing to the apples-to-oranges challenges in comparing altmetrics to citations is the fact that altmetrics accrue at a much faster rate than citations: within a year of publication, it is reasonable to expect that any given paper has reached the end of its social-media-buzz cycle [29]. In order to contrast this with the much-longer lifecycle of scholarly impact, we looked at the 1,639 PD papers published in 2006 which have been cited at least ten times and found that 1,362 of them, or 83%, were still being actively cited a decade after going to press.

Additionally, papers that were very highly cited in the scientific literature—including results from important clinical trials published since 2012—have received unexpectedly low Altmetric Attention scores. One example is the paper by Okun and collaborators, which described the results from constant-current subthalamic brain stimulation (STN-DBS) in PD [30]. This paper, published in 2012 in Lancet Neurology, received a low attention score of 24, with 141 citations. Another influential treatment study from Olanow and collaborators—again published in Lancet Neurology, now in 2013—addressed the effects of intrajejunal infusion of levodopa-carbidopa intestinal gel with advanced PD [31]. Despite high citation counts (124), this publication also received a low attention score of 34. These observations highlight that Altmetric Attention Scores may not always showcase the most important articles in terms of scientific merits alone, but rather reflect a different construct. Impact factors reflect the actual use of scientific results, whereas the disposable nature of current press releases may have less value in evaluating the impact of science itself. Perhaps seminal work is not picked up by the altmetrics scores, simply because the authors paid less attention to issuing a press release, or because they were themselves less active on social media. Clearly, high altmetrics scores can be a reflection of good quality research, but as our present paper shows, marked discrepancies also exist, and altmetrics scores must be interpreted accordingly.

It is reasonable to use traditional citation-based metrics to measure the impact of a scholarly article, in the sense that citations document the fact that new evidence was built upon a given paper’s findings. Altmetrics, on the other hand, appear to reflect public interest, rather than the scientific merits of an article. There are cases, however, when word of an interesting new finding is spread online among the social-media-savvy subset of a given scientific community. In these instances, there is evidence that the volume of tweets in the first three days after publication can be predictive of future citations in the scientific literature [32]. While some investigators may be more effective than others in their ability to persuade fellow scientists to cite their work, it is likely that public relations strategies in the world of altmetrics can skew the results in a more dramatic fashion. Internet presence is highly dependent on the number of viewers and followers, and news outlets often report misleading, eye-catching information masquerading as science, which could influence web-based attention scores [33]. As such, the altmetrics really offer a new index of scientific success, complementary to traditional citation-based measures.

The majority of the papers were published in prestigious neurology journals, but we also observed a notable exception (the cannabis paper, where public attention led to high altmetric scores for a paper published in a relatively modest neurology journal). Most papers provided new insights regarding treatment, pathophysiology or risk factors for PD. Most online attention received by these articles was found in mainstream media and on Twitter.

While all of the articles considered in the analysis have been published within the last five years, there is not a level playing field when it comes to the varying amounts of time each paper has had to accrue attention. A paper from 2012 has had significantly more time to have been discovered by a broader audience than has a paper from 2016. On a similar note, a paper appearing in a highly prestigious journal receives an automatic head start in getting noticed by the general public when compared to a paper published on the same day but appearing in a more obscure journal. In order to aid with journal and published-date normalization, we have included a column in Table 1 which lists where the article in question ranks when compared to the Altmetric scores of all the other papers published in that same journal within the same three-month period. If you consider this column in aggregate, you can see that for a PD paper published in Neurology to have made our list, it must have been among the top three of its peers, while for a PD paper appearing in Science or Science Translational Medicine, being in the high teens or even mid-twenties seemed to suffice.

In the current era of “alternative facts” and “post-truths” [34], the shift from traditional metrics to altmetrics must be approached judiciously. Social media represent a uniquely powerful tool for the widespread dissemination of a message, and the work presented here suggests that scholarly publications are no exception. The business of illegitimate science is a profitable one [35], and subpar publications could theoretically end up receiving high Altmetric Attention Scores. In addition, fake accounts in Twitter and Facebook may artificially broaden a publications’ reach [36], and even generate unjustified hype regarding a diagnostic test or treatment. In our list, however, all papers came from respected publishers. However, credible organizations should remain alert for the possibility of “false papers” receiving undeserved social media attention. These considerations underscore the urgent need for the introduction of some informed and unbiased organization to filter all the social media buzz around PD, and to highlight what is credible news, and what is “fake” news or poor science. Thus, traditional metrics and altmetrics can jointly help to weigh both the evidence and the resultant buzz, and shape the ultimate interpretation. Also, regarding news outlets, while Altmetric does not attempt to understand whether or not the same news story is being reprinted by a number of news outlets, an attempt is made to account for news outlet quality and to make sure that multiple stories from the same news outlet do not falsely increase the Altmetric Attention Score for a given paper [37].

Finally, an interesting direction for future investigation would be to compare the typology of PD articles with the highest Altmetric Attention Scores to analogous sets of articles for related diseases in an attempt to determine whether disease characteristics are in any way correlated to the types of scientific findings which garner the most online attention.

DISCLOSURES

RA and BB co-authored one of the papers listed in Table 1. AAS and SK are employed by Digital Science and Altmetric LLC, the two companies behind Altmetric.com.

CONFLICT OF INTEREST

No conflict of interest.