#JunkScience: Investigating pseudoscience disinformation in the Russian Internet Research Agency tweets

1. Method

We accessed the same sample data set as Linvill et al. (2019), whose 3 million tweet set was made accessible on the website RussiaTweets.com. A number of pseudoscience and ‘controversial’ science topics and their related derivatives and keywords were searched in the Russia Tweets database utilising an exploratory mixed-method research design to get a general understanding of the data (Creswell, 2014). To ensure comparative results with Broniatowski et al. (2018), vaccination was selected as the first topic to investigate. Additional topics initially considered included a range of science and pseudoscience topics. These were selected due to being high profile topics for pseudo or anti-science messages based on an informal poll of science communication scholars.

The topics considered were astrology, chemtrails, climate change, Ebola, essential oils, flat Earthism, genetically modified organisms (GMO), homoeopathy, hypnosis, reiki and Zika. We subsequently focused on five topics in addition to vaccination: climate change, GMO, Ebola, flat Earth beliefs (flat Earthism) and Zika, as the five most prevalent topics from initial inspection of the data.

A control sample of 30,000 English-language tweets was collected from Twitter using an application program interface (API). To achieve as comparable a control group as possible, these were accessed in blocks of 2500 tweets from 1 randomly chosen day in each month of 2016. This gave us a reasonable sample to approximate the sorts of online discussions that were occurring on Twitter during the peak time of the Russian troll activities, between 2014 and 2017. This control sample set was free of known organised troll influences from the time as Twitter had identified and removed any troll accounts and their associated tweets prior to our control sample collection.

A number of words and phrases were associated with each topic, and these became the search terms used to ascertain the frequency of each topic. These words and phrases were identified through general browsing of the sample data, and then ‘snowballing’ off the parent terms (e.g. ‘climate change’ to ‘#climatechange’ and ‘climate denial’) as new related terms came up in the resultant tweets or from brainstorming associated concepts. Derivatives of the search terms were also used for data exploration and as a means of assessing sentiment in the tweet content, for example, ‘GMO’ and ‘boycott GMO’. The below lists made up all the unique search terms chosen, that is, none of the above terms would ‘double count’ with another term – unless the tweet contained both words and phrases in its own right. This removed instances of double counting a tweet such as @bigboysneed’s ‘BOYCOTT @KelloggCompany turning OUR food to GMO junk’ (14 December 2015) reporting a positive indication for both ‘GMO’ and ‘boycott GMO’. It should be noted that spaces mattered in the search function, and so the phrase ‘global warming’ was treated as a separate term to ‘globalwarming’, with the former finding tweets containing both words in a sentence and the latter finding tweets using the hashtag #globalwarming. The search terms from each parent term were as follows:

Each word or set of two words was preceded by a plus symbol (‘+’) in the search bar, so that the results were inclusive of both words but not in any exact order. Each term’s frequency in the Russia Tweets was recorded, and then each tweet in the results was checked for repetition in the sample. For terms with a large number of results returned, a 20% sample of tweets was taken by looking at every fifth page of the results. The repetition data from this sample, however, remained reflective of the full results returned. Repetition data were gathered by selecting a key phrase in the tweet, and searching for that exact phrase in the sample set using inverted commas around the phrase. For example, the phrase ‘While Marco Rubio Campaigns, Florida Mayors Plead For Help Against Climate Change’ appeared using hashtags #StopTheGOP #GOPDebate (@cameronjjj11, 29 January 2016) and in a separate tweet using #ABCNews #ABCGOPDebate (@jas_bac, 7 February 2016). Not selecting the entire tweet enabled us to find repetition of these complete sentences that differed only by their hashtags, a likely IRA tactic. This also accounted for instances when tweets contained a different link at the end of otherwise identical tweets. Each term was then searched in the sample set for comparison of frequency.

Generation of an odds ratio was used to determine the likelihood of each topic appearing in the Russia Tweets when compared to the control sample, and each had an associated 95% confidence interval to indicate the significance of the finding (Szumilas, 2010).

To get an indication of sentiment in the Russia Tweets, we looked at the most highly repeated tweets or phrases in each topic. These were then considered qualitatively for intended message or motivation, and general trends observed in the repeated tweets were noted such as location, date or time posted, country of origin (‘United States’, ‘Unknown Region’ or otherwise), ‘retweet’ or account-posted content, or thematic similarities across different high-repeat tweets.

2. Results

Climate change

Of the six topics investigated, climate change content was the most present in the data, with more than 1 in every 1000 Russian tweets related to climate change (0.144%). The content was 8.6 times more likely to appear in the Russia Tweets than the control sample (95% CI [3.59–20.75]), showing a significantly out of proportion presence when compared with normal Twitter use.

The topic of climate change also saw the highest frequency of repeat tweets, with 21 separate tweets being repeated 20 times or more, and an additional 60 tweets repeated between 5 and 19 times. Tweets repeated 20 or more times are presented below in Table 1. A good portion of the tweets in this category contain a link that redirects to the same domain sale website seen in the vaccination results, indicating again the likelihood of a dedicated disinformation website. There is a notable presence of climate change denial content in the repeats, seen in tweets such as ‘ . . . 97% SCIENTISTS #ClimateChange IS AN URGENT PROBLEM IS FICTION . . . ’ (n = 30), ‘Bill Nye Ties Hurricanes to Climate Change, Gets Shut Down by a REAL Scientist [LINK]’ (n = 24) and ‘Liberals are Pushing to Make “Climate Change Denial” a CRIME’ (n = 24). Many additional tweets are ambiguous in meaning without the link provided, which is perhaps a tactic in its own right if the IRA was employing ‘click bait’ article titles.

OPEN IN VIEWER

Table 1.

Climate change related tweets repeated 20 or more times in the Russia Tweets.

Repeated Tweets	Frequency
Disgraced NY Times Forced to Admit They LIED on Climate Change Piece	63
‘The View’ Claims Jesus Cares More for Refugees and Climate Change Than Abortion	35
#climatechange zealots hypothesize about Armageddon scenarios while holding a straight face. #remarkable	32
#Obama Claims #ClimateChange Greatest Threat @POTUS ➰ FLOTUS FLY SEPARATE PLANES #JunkScience #TCOT #PJNET http://t.co/tKDHRRPUE2	32
RT @DrMartyFox: 97% SCIENTISTS #ClimateChange IS AN URGENT PROBLEM IS FICTION. http://t.co/LuH7L6JYtQ #JunkScience #PJNET	30
Hey Liberals! THIS is Why Irma Got So Strong and It’s NOT Global Warming [LINK]	24
SCIENTISTS – Yellowstone Volcano a Bigger Threat than Climate Change! [LINK]	24
Pope Francis Thinks ‘Climate Change’ is the Root of All Evil	24
BREAKING: Mccain Sides with Gore, Dems on ‘Climate Change’ Agenda	24
Pope Declares People Have ‘Moral Responsibility’ to Believe in Climate Change [LINK]	24
Bill Nye Ties Hurricanes to Climate Change, Gets Shut Down by a REAL Scientist [LINK]	24
PATHETIC STUNT – CNN’s Acosta Tries to Use Hurricanes to Push ‘Climate Change’	24
Joy Behar Suggest Hurricanes Should Be Named After Climate Change Deniers [LINK]	24
Hillary Claims California Wildfires are Caused By ‘Climate Change’ [LINK]	24
John Kerry Just Argued ‘Climate Change’ is a Bigger Risk than North Korea [LINK]	24
Liberals on ‘The View’ Believe Al Gore’s Movie is Proof of Climate Change [LINK]	24
Liberals are Pushing to Make ‘Climate Change Denial’ a CRIME!	24
Trump is About to Trash Obama’s BIGGEST Climate Change Regulation	23
CNN Anchor Tries to Link Hurricane Harvey to Climate Change, FAILS MISERABLY	23
Lindsey Graham has an entirely reasonable position on climate change, sometimes http://t.co/kZMZka7Ja7 http://t.co/55juNRjwAT	22
@POTUS- We R Being Lied 2!! #ClimateChange biggest threat as it churns $ for corrupt govt! #JunkScience #PJNET	20

The most repeated phrase was ‘Disgraced NY Times Forced to Admit They LIED on Climate Change Piece’ which was seen 63 times in the data across a number of different hashtags and associated links. All of these tweets were original content tweets posted on 10 August 2017, with a 70%/30% split between Unknown Region and United States account origin, respectively. The links again lead to the same domain-purchasing website seen in the most repeated vaccination tweet. An unusual observation in the 63 repeated tweets was the inclusion of hashtags that seemed to reference the Twitter handle it was posted from at the start of some, but not all, of the tweets. For example, @amberlinetr’s tweet began with #amb, @darcyystr’s tweet with #darccy, @ivaassntr with #iv, @anisilope with #lope, @arabmtr with #arbt and so on. These handle associated hashtags were present at the start of 48 of the 63 repeats. This may have been a mechanism of creating unique hashtags to track the reach of an individual tweet, or provide some personnel performance information within the IRA organisation structures.

In total, we found over 4000 tweets relating to climate change in the Russia Tweets, far surpassing any other topic investigated here. This suggests a considered agenda in the inclusion of climate change discussion in the IRA’s disinformation strategies. Further access to the links associated with each tweet would allow for a better sentiment analysis; however, it is clear from the repeated tweets that the content contains tweets both accepting climate change and, most importantly, when considering impact and motive, denying it.

Summary

Categorical breakdown of the results shows that science emphasis in the Russia Tweets was not seen across all topics. Due to the time frame of the control sample taken and the peak of public awareness of the Ebola virus, conclusions are difficult to draw for this topic. However, discussion of the Zika virus epidemic was somewhat amplified in the Russia Tweets. GMO and flat Earthism content were on par with the control, while discussion of climate change and vaccination was seen in a far greater amount in the Russia Tweets. Each topic’s relative proportion (percentage) in both the control sample and our data set is presented in Figure 1.

OPEN IN VIEWER

Figure 1.

Percentage of topic frequency in the Russia Tweets vs Control Sample for flat Earth, GMO, Ebola virus, zika virus, vaccines and climate change.

This figure also highlights the relative proportion of climate change content to vaccine content: though the boost of vaccine content was higher than that of climate change (odds ratios of 11.7 and 8.6, respectively), the sheer amount of climate change content was far greater than that of vaccines. This suggests a targeted effort within the IRA to increase and perhaps influence the climate change discussion. Although we have not completed a quantitative sentiment analysis to definitively confirm what proportion of these tweets were pro- or anti-climate science, qualitative analysis of the most repeated tweets shows climate change denial was certainly present in the data, which may have impacts on the wider climate change discussion on Twitter and in broader society.

3. Discussion

These results confirm the presence of pseudoscience in the troll tweets of the Russian IRA. Our initial hypothesis – that science was widely being targeted – was not supported with the data. Rather, our results found certain topics were chosen as targets over others. It is likely that the science content seen throughout the tweets is the result of both motivated and camouflage activities, as we have seen discussion of some topics on par with the frequency in our control sample (GMO, flat Earthism) while others (vaccination, climate change, Zika) are significantly more common in the Russia Tweets. We observe and discuss here some possibilities motivating the choice of the two topics seen to have been strongly promoted. Both vaccines and climate change science have a broad societal impact from misinformation and disinformation, and both involve positions strongly aligned with political ideologies (Davis, 2019; Metag et al., 2017; Nurse and Grant, 2019), so these two topics can be utilised as tools for the spread of political discord in America. It very much appears that the attack was not on science per se, but rather that the IRA strategically chose science topics to influence wider America. However, the effects this may have had on science understanding are real and potentially long lasting. Thus, it will be important for the science community and society at large to remain vigilant to these campaigns of cyber influence, as the use of online trolls and bots is certainly not confined to Russia. Here we explore these points in greater detail.

So why these topics? We suggest one motivating similarity between the two science topics we found most emphasised in the Russia Tweets may be the wider reaching societal impact from misinformed citizens. Of the six topics chosen to investigate, climate denial and anti-vaccination beliefs arguably pose the biggest threat to the well-being of a society. This social implication was not shared to the same extent by the two topics we found to be least promoted by the IRA (GMO, flat Earthism). For vaccines and climate change, the issue goes beyond a personal one: an anti-vaccination movement is dangerous because it reduces herd immunity, and disagreement on the need for climate action risks the catastrophic predictions associated with no action (Fine et al., 2011; McAdam, 2017; Oreskes and Conway, 2010). The spread of AIDS disinformation in the 1980s had similar societal impacts: it fuelled misunderstanding of an epidemic that was already misunderstood, and further promoted homophobia (Boghardt, 2009; Greene, 2007; O’Hare et al., 1996). This period saw a great amount of stigma around HIV infection and transmission, and this fear was exploited by the then-KGB (Boghardt, 2009).

The peak activity of the Russian trolls also aligns with the increase of ‘post-truth’ conversations observed by Lewandowsky et al. (2017). This interesting correlation may be unrelated, but it could also suggest that Russia was either encouraging or capitalising on this trend with their disinformation campaign. It is possible that this ‘post-truth era’ has also been influenced somewhat by their various disinformation efforts over the years, though the likelihood of this is difficult to say. There are some empirical observations that may support this, such as the lingering AIDS conspiracies that are descendants of Operation INFEKTION.

The topics may also have been chosen due to their influence or association with political beliefs. This is somewhat similar to the earlier observation; however rather than considering the motivation of the trolls to be targeting society at large, it considers them in their political role.

It has already been shown by Linvill et al. (2019) that there was a broad support for right wing and an attack on left wing politics in the Russia Tweets. In America, politics and science beliefs are seen to be aligned strongly or weakly for certain topics, and the Pew Research Center indicates that there is strong alignment between political party and belief regarding opinions of anthropogenic global warming (Iyengar and Massey, 2019; see also Nurse and Grant, 2019), while weak alignment on the issue of GMO foods, which may explain why this topic was less targeted (Rainie, 2017). Studies into the American understanding of climate change science have noted that ‘the politicization of climate change drives conflict in climate-related discourse and constrains climate policy’ (Ballew et al., 2019: 5). Furthermore, online discussions of climate change and climate politics are found to be dominated by non-experts, which may provide an environment ripe for disinformation activity (Schafer, 2012). This has been seen in Twitter discourse following the release of the Fifth Assessment Report from the Intergovernmental Panel on Climate Change (IPCC), where non-experts accounted for the majority of the responding discussions (Newman, 2016), and more generally, it has been found that climate politics and climate change discussion in online spaces occurs mostly without input from scientists or scientific institutions (Schafer, 2012). The Yale Program on Climate Communication conducted surveys during the 2016 election campaign which also indicated that those sceptical or unsure about climate science were more likely to vote for Donald Trump over Hillary Clinton, but notably global warming was not an issue important to their voting choices (Roser-Renouf et al., 2016). Thus, the focus on climate change discussion by the Russian trolls is unlikely to have been a tactic to influence election choices, but rather to access a subset of Americans with whom to engage. In this case, those more likely to cast a Republican vote.

Similar correlations between ideology and vaccination beliefs have also been observed (Baumgaertner et al., 2018). Furthermore, Davis (2019: 360) suggests that anti-vaccination discourse, in general, is part of a larger movement of ‘anti-public’ discourse, which ‘[intends] to mislead rather than inform’. He argues that anti-public discourse of topics such as anti-vaccination and anti-climate science undermine principles of democratic debate, and notes it has been particularly fuelled by online media, again, pointing to the ripeness of social media as a platform for disinformation campaigns. It is not unreasonable then to suggest that discussion of climate change and vaccine science may have been chosen by the Russian trolls to access specific political spheres and further increase polarisation, and thus conflict, across the political spectrum.

There may be other motivated bodies moving against climate action and vaccination for various reasons, so it would be naive to assign this phenomenon to Russia alone (Oreskes and Conway, 2010). However, it is likely that disinformation campaigns in these topics would have long lasting impacts on the Twitter users exposed, and by extension on the societies in which they operate. Once accepted, false information is difficult to correct in the receiver, and biases remain even in individuals who acknowledge the correction (De keersmaecker and Roets, 2017). Cognitive psychology, it seems, sides with the IRA. If the trends in this research were found to continue throughout the wider social media campaign of the IRA, it is possible that Russian-influenced climate and vaccine science reached tens of millions of people across their social media channels.

We do not think attacking science in general was the motive of the IRA. Reports have indicated time and again the clear political motive behind their activities around the 2016 US presidential election. However, with the main objective of historical KGB Active Measures campaigns being to ‘drive a wedge’ (Andrew and Mitrokhin, 2001) in American society, these topics have almost undoubtedly been capitalised upon to advance this motive. Functional democracies rely on well-informed public, and damaging this understanding of truth is detrimental to modern American democracy (Stebenne, 2018). However, the use of science to further this agenda is a real challenge for the public understanding of science and correcting it is a difficult task. Despite sitting within a bigger political narrative, the impacts on science understanding could be long reaching, as has been seen in the disinformation campaigns of the Cold-War era.

There is good evidence that social media trolling tactics are being used by countries other than Russia. Research at Oxford University has documented the use of ‘cyber troops’ in 28 countries, with the purpose of influencing public opinion through social media (Bradshaw and Howard, 2017). Not all of these troops are human, however, and large influence campaigns are being run by automated bots (Bradshaw and Howard, 2017). It is worth noting that the IRA also employed the use of bots in their activities, alongside that of the human-run troll accounts, and some other countries are also adopting this mixed strategy in their cyber campaigns (Bradshaw and Howard, 2017; The New York Times, 2019). Twitter has also provided the details of 3000 Iranian accounts associated with a political influence campaign, alongside the 4000 Russian ones released (Twitter, 2018). This may indicate a growing cyber trend in public influence and propaganda tactics, and poses a challenge to the social media companies that are the platform for so much public communication today. The US Federal Bureau of Intelligence has boosted their counterintelligence operations in preparation for the 2020 US Presidential election, and it will be important to remain aware and critical of these social media influence campaigns which are likely to rear their heads again in the coming years (Barnes and Goldman, 2019).

4. Limitations and future research

The tweets provided by Linvill and Warren are not a complete set of the known Russian troll tweets, although they make up a considerable portion. Though the raw numbers of tweets and repeats may not be representative of the full set, we are confident that patterns observed in the sample set used (n = 2,973,371) remain significant and point to an intentional pushing of climate science and vaccination content.

It is very likely that we did not capture all science and science-related topics in our initial exploratory process. Furthermore, within each of the six topics chosen it is probable that we missed some related keywords or keyword variations. However, the breadth of key words used and the method for determining those keywords would likely have yielded most, if not all, of each topic’s related tweets, and missed tweets are unlikely to have changed the overall conclusions drawn. It should also be noted that while every effort was made to find a clean control sample, it cannot be ruled out that elements of the control sample were affected by the IRA tweets. Moreover, the relationships between the key themes of the 2016 Presidential election – and Trump’s rhetorical orientation – and the topics chosen by the IRA are difficult to disentangle. Did the IRA follow Trump, or did both mine a vein of discord in American politics? While direct conclusions would be difficult to draw, future research could explore the timing of key developments in Donald Trump and Hillary Clinton’s rhetoric and similar threads in the IRA’s tweets.

We had external limitations in the production of our control sample due to Twitter’s data collection rules, which in part respond to the European Union’s GDPR. Our sample, taken from 2016, may not have been representative of the entire window of activity of the Russian trolls (2014–2017), particularly for the time frames of the virus epidemic topics (Kasereka Masumbuko et al., 2018; Valentine et al., 2016). This may have influenced the comparisons should there have been a particular event that spiked a lot of tweet activity. The years 2015–2017 were the height of the Russian Troll activity however, so we are satisfied that the year chosen (a) is the central year of this range and (b) encompasses the date of the 2016 US election.

There is much research to be done on the Russia Tweets, especially as the 2020 US Presidential election nears. Full sentiment analysis of the topics investigated in this research would give a stronger evidence base from which to infer Russian motives, and expanding the topics investigated to include a wider range of science may help reveal further patterns in the IRA’s targets. More generally, investigation is needed into the day-to-day workings of the IRA through observations of patterns in date, retweet behaviour, account type and so on. We noted some potentially interesting date patterns, such as an increase in tweeting around 11 September 2017 (although this was not investigated beyond researcher observation and is not confirmed). Better understanding of how the IRA operates may help identify future trolling and disinformation efforts when they appear.