#JunkScience: Investigating pseudoscience disinformation in the Russian Internet Research Agency tweets

Abstract

Recent research has identified anti-vaccination propaganda in the so-called Russian Troll Tweets strongly associated with the 2016 US Presidential election. This study builds on this: hypothesising that if vaccination content was found in the sample, the Russia Tweets would be likely to contain other science content, and perhaps, similar pseudo or anti-science messages. As well as vaccination, climate change, genetically modified organisms, Ebola, flat Earth beliefs (flat Earthism) and Zika were found in the Russia tweets. Genetically modified organisms and flat Earthism appear to have been camouflage content – tweeted at similar rates to other Twitter users – while climate change, Ebola, Zika and vaccination appear to have been emphasised beyond the background rate for strategic disinformation purposes.

Keywords

climate change disinformation international politics representations of science science attitudes and perceptions social media

In 1983, a story appeared in a pro-Soviet newspaper in India that claimed the United States government had instigated the development of the AIDS virus to release it as a bioweapon (Boghardt, 2009). In 1985, this story was picked up by Literaturnaya Gazeta (‘Literary Newspaper’), the official newspaper of the Soviet Union, and by the late 1980s, this HIV origin-story had been published in over 80 countries via the Soviet-controlled press. The Soviet Union insisted their story – that the AIDS virus had been created at a biomedical research facility in Fort Detrick – was accurate. This changed on 30 October 1987, when they suddenly denied any support for the claim at all (Schoen and Lamb, 2012). What had happened a few days prior was the release of a report from the US Department of State about investigations into the then Russian Soviet Federative Socialist Republic (Russian SFSR). The report revealed that the Russian SFSR had been conducting a number of so-called ‘Active Measures’ operations through their Committee for State Security, or Komitet gosudarstvennoy bezopasnosti (KGB) – and this included a disinformation campaign called Operation INFEKTION, which had the purpose of spreading false origin stories about the AIDS virus (Boghardt, 2009). These Active Measures had the objective of undermining the ‘Main Adversary’ of the Russian SFSR: the United States (Andrew and Mitrokhin, 2001; Shultz and Godson, 1984).

The Department of State report thoroughly debunked the ‘scientific falsehoods’ of Operation INFEKTION: they had experts assert that the AIDS virus could not have been man-made (United States Department of State, 1987). But the conspiracy that had been spreading throughout the press seemed to stick in the minds of the American population. Remnants of this conspiracy still exist today. Recent studies indicate this belief is still present among some demographics in modern day America¹ (Bogart and Thorburn, 2005; Brooks et al., 2018; Nattrass, 2013). With the scientific corrections framed by the allusion of a continued cover-up, they were largely unheard by the public. The impact of a spark of fake news planted in a small Indian newspaper decades ago exhibits the long term reach the KGB’s Active Measures campaign had on trust in science in Western society. Emerging evidence of recent efforts from modern day Russia – as discussed in this article – may continue to affect trust in science today and into the future.

The Active Measures programme was a hallmark of KGB espionage, and manifested in a number of operations, including assassinations, propaganda and their dezinformatziya – disinformation – campaigns like Operation INFEKTION (Andrew and Mitrokhin, 2001; Boghardt, 2009). Disinformation is similar to misinformation, though its intension differs. Where misinformation is false information accidentally or unknowingly shared, disinformation is false information that is both deliberately misleading and deliberately shared (Shultz and Godson, 1984). A ‘continued-influence effect’ can also be observed in false information (whether disinformation or misinformation), where false information can stick in the receiver’s mind, and corrections are rarely fully successful if a belief is widely or deeply held (Lewandowsky et al., 2012). This makes disinformation campaigns only more pernicious, likely to hang around and continue to bias people’s thinking even if corrected.

In early 2017, the United States Office of the Director of National Intelligence (DNI) released an assessment of Russian influence tactics in the 2016 US Presidential election. They described a multifaceted campaign that used both direct and covert methods to undermine the democratic process in America, denigrate Presidential candidate Hillary Clinton, and support the election of Donald Trump as 45th President of the United States (Office of the Director of National Intelligence, 2017). The most infamous component of this campaign was the leaking of thousands of emails from key members of the US Democratic National Committee, including Hillary Clinton (WikiLeaks, 2016, 22 July). However, Russian operations were also in play at a much more covert level through a ‘troll farm’ known as the Internet Research Agency (IRA). Located in St Petersburg, Russia, the IRA imitated American Twitter users between 2014 and 2017 (Linvill et al., 2019). Their objective was to conduct an ‘information warfare against the United States of America’ (The New York Times, 2018), with IRA operatives generating online content from thousands of fake Twitter accounts. The organisation and 13 associated Russian nationals were indicted by the US government early in 2017 and charged with influence operations targeting the 2016 US Presidential election (The New York Times, 2018).

Early in 2019, special counsel Robert Mueller released his report documenting a 2-year investigation into the 2016 US Presidential campaign influence operations. Though highly redacted, this report gave further detail on the Russian operations, including the structure of the IRA and their wider reaching social media operations on Facebook, YouTube, Twitter, Tumblr and Instagram. The report includes an estimation that the Active Measures on social media reached 126 million people through Facebook, and that ‘approximately 1.4 million people . . . may have been in contact with an IRA-controlled account’ on Twitter (The New York Times, 2019). President Trump has consistently refuted the Russian influence on the 2016 Presidential campaign since allegations were first made, despite the undeniable evidence for the strategic and targeted influence operations by the IRA (Gabbatt, 2019). However, on 30 May 2019, President Trump posted a tweet that said, ‘I had nothing to do with Russia helping me to get elected’, countering his previous statements that Russia had no influence on the election at all (though this may have been inadvertent; Gabbatt, 2019).

In 2018, Linvill and Warren (2018) of Clemson University accessed nearly 3 million of these tweets from a list of Central Intelligence Agency (CIA) identified Twitter users and published them as an open access resource on GitHub. This resource was subsequently picked up by the Defending Democracy Together project and became the associated website RussiaTweets.com. Linvill et al. (2019) analysed a sample of these tweets from the month directly preceding the 2016 US election day – finding that a large proportion of tweets were ‘camouflage’ tweets, those that increase the believability and ‘realness’ of each Twitter account, while having no inherent political purpose on their own. The next highest proportion of tweets was ‘attack left’, then ‘support right’ – tweets that criticised or supported a particular political agenda. These findings give quantitative reflection of the widely accepted IRA motive to promote the election of Donald Trump. The Russian trolls generated a significant amount of political content with a discernible emphasis in their messaging. Without a doubt, the trolls’ overarching goal was to ensure the election of Donald Trump, but in their efforts to undermine truth and politics in the American public, a more science-targeted and science-implicating topic has been revealed.

Broniatowski et al. (2018) identified a surprising topic in the content of the Russian Tweets: vaccination. Their study compared the Russian trolls (who were real people at the IRA writing and promoting content through fake accounts) with bots (entirely computer generated tweets) and a standard Twitter sample (normal Twitter users). It was found that discussion of vaccination content was significantly more common in the troll tweets compared to a standard sample of tweets. Furthermore, while randomised bots posted anti-vaccination sentiment, the Russian troll accounts tweeted about both sides of the debate – using the hashtag #VaccinateUS to promote the topic as a polarising issue. The trolls, it seemed, were deliberately spreading discord. This correlates with the broader patterns seen in Linvill et al.’s (2019) research, which indicated tweet content mimicking accounts across the political spectrum (Broniatowski et al., 2018). If the vaccine content was intentionally weaponised to create a political wedge in society, it may have seen some successes – with the United States seeing the highest rates of measles in 2019 since 1994 (Patel et al., 2019).

This study builds on the following: hypothesising that if vaccination content was found in the sample, the Russia Tweets would be likely to contain content about other science topics – and perhaps, similarly convey pseudo or anti-science messages analogous to anti-vaccination messaging – as done during Operation INFEKTION. In the context of the larger political game Russia was playing, where does science fit in? Has it been weaponised for agenda setting motives, or does it merely help camouflage the trolls as more believable Americans? Thus we wanted to investigate what, if any, science topics were present in the sample of Russia Tweets, and how much they were being discussed in comparison to normal Twitter use at the time.

1. Method

We accessed the same sample data set as Linvill et al. (2019), whose 3 million tweet set was made accessible on the website RussiaTweets.com. A number of pseudoscience and ‘controversial’ science topics and their related derivatives and keywords were searched in the Russia Tweets database utilising an exploratory mixed-method research design to get a general understanding of the data (Creswell, 2014). To ensure comparative results with Broniatowski et al. (2018), vaccination was selected as the first topic to investigate. Additional topics initially considered included a range of science and pseudoscience topics. These were selected due to being high profile topics for pseudo or anti-science messages based on an informal poll of science communication scholars.

The topics considered were astrology, chemtrails, climate change, Ebola, essential oils, flat Earthism, genetically modified organisms (GMO), homoeopathy, hypnosis, reiki and Zika. We subsequently focused on five topics in addition to vaccination: climate change, GMO, Ebola, flat Earth beliefs (flat Earthism) and Zika, as the five most prevalent topics from initial inspection of the data.

A control sample of 30,000 English-language tweets was collected from Twitter using an application program interface (API). To achieve as comparable a control group as possible, these were accessed in blocks of 2500 tweets from 1 randomly chosen day in each month of 2016. This gave us a reasonable sample to approximate the sorts of online discussions that were occurring on Twitter during the peak time of the Russian troll activities, between 2014 and 2017. This control sample set was free of known organised troll influences from the time as Twitter had identified and removed any troll accounts and their associated tweets prior to our control sample collection.

A number of words and phrases were associated with each topic, and these became the search terms used to ascertain the frequency of each topic. These words and phrases were identified through general browsing of the sample data, and then ‘snowballing’ off the parent terms (e.g. ‘climate change’ to ‘#climatechange’ and ‘climate denial’) as new related terms came up in the resultant tweets or from brainstorming associated concepts. Derivatives of the search terms were also used for data exploration and as a means of assessing sentiment in the tweet content, for example, ‘GMO’ and ‘boycott GMO’. The below lists made up all the unique search terms chosen, that is, none of the above terms would ‘double count’ with another term – unless the tweet contained both words and phrases in its own right. This removed instances of double counting a tweet such as @bigboysneed’s ‘BOYCOTT @KelloggCompany turning OUR food to GMO junk’ (14 December 2015) reporting a positive indication for both ‘GMO’ and ‘boycott GMO’. It should be noted that spaces mattered in the search function, and so the phrase ‘global warming’ was treated as a separate term to ‘globalwarming’, with the former finding tweets containing both words in a sentence and the latter finding tweets using the hashtag #globalwarming. The search terms from each parent term were as follows:

Climate change: global warming, globalwarming, climate change, climatechange, climate denial, climatefacts, climate facts, global NOAA, global cooling, junkscience, climatescam;

Flat earthism: flat earth, round earth, ball earth, egg universe, disc earth, disc planet, flat planet, spherical earth, spherical planet, sphere planet, sphere earth;

Genetically modified organisms: GMO, Frankenfoods, genetically modified, genetic modification, GMF;

Vaccination: vaccines; vaccine, vaccination, vaccinate, antivax, MMR, vaxxed, flu shot, mmr, hpv, vaccs, vacs, vaccinateUS, SB277, vaccineswork;

Ebola virus: ebola;

Zika virus: zika.

Each word or set of two words was preceded by a plus symbol (‘+’) in the search bar, so that the results were inclusive of both words but not in any exact order. Each term’s frequency in the Russia Tweets was recorded, and then each tweet in the results was checked for repetition in the sample. For terms with a large number of results returned, a 20% sample of tweets was taken by looking at every fifth page of the results. The repetition data from this sample, however, remained reflective of the full results returned. Repetition data were gathered by selecting a key phrase in the tweet, and searching for that exact phrase in the sample set using inverted commas around the phrase. For example, the phrase ‘While Marco Rubio Campaigns, Florida Mayors Plead For Help Against Climate Change’ appeared using hashtags #StopTheGOP #GOPDebate (@cameronjjj11, 29 January 2016) and in a separate tweet using #ABCNews #ABCGOPDebate (@jas_bac, 7 February 2016). Not selecting the entire tweet enabled us to find repetition of these complete sentences that differed only by their hashtags, a likely IRA tactic. This also accounted for instances when tweets contained a different link at the end of otherwise identical tweets. Each term was then searched in the sample set for comparison of frequency.

Generation of an odds ratio was used to determine the likelihood of each topic appearing in the Russia Tweets when compared to the control sample, and each had an associated 95% confidence interval to indicate the significance of the finding (Szumilas, 2010).

To get an indication of sentiment in the Russia Tweets, we looked at the most highly repeated tweets or phrases in each topic. These were then considered qualitatively for intended message or motivation, and general trends observed in the repeated tweets were noted such as location, date or time posted, country of origin (‘United States’, ‘Unknown Region’ or otherwise), ‘retweet’ or account-posted content, or thematic similarities across different high-repeat tweets.

2. Results

Flat earthism

Flat Earth content made up 0.0038% of the Russia Tweets sample, with the majority of these coming directly from the ‘flat earth’ search term (0.0030%). Related terms such as ‘round earth’, ‘egg universe’ or ‘dome universe’ which are commonly associated with flat Earth theories were not present in the data at all² (Marshall, 2018). Only a handful of tweets in this topic were repeated, with two tweets in a tie for the most repeats. In total, 1 tweet, posted 10 times on the 9 July 2017, was ‘These people are for real?? I always thought #FlatEarth was some Internet meme [LINK]’. This included a link to a 2017 Denver Post article about a group of American flat Earthers titled These Coloradans say Earth is flat. And gravity’s a hoax. Now, they’re being persecuted. All of these were retweets and none came from users identifying as United States residents. Eight tweets were of ‘Unknown Region’, one was from Germany and one was from Spain. The headline of the article linked employs ‘forward referencing’, a technique that drives greater click-through rates when the preview is shown in a twitter feed, and is also sympathetic to the flat Earthers as a victim of persecution (Kuiken et al., 2017).

The other tweet, ‘Wish B.o.B. did the halftime show. Woulda loved a flat earth lecture during half time. #StayWoke @bobatl’ was posted on various days throughout the first-half of 2016 (14 February 2016 to 13 June 2016) and referenced American rapper B.o.B who posted a controversial picture on Twitter in January 2016 questioning where the curve of the Earth was in the scenic background behind him (Platon, 2016). These tweets were all from ‘American’ originating accounts, and they were all posted as account-created or ‘original’ content, that is, none were retweeting a real or other user.

Flat Earth content in the control sample made up 0.0067%, with no repeated content. By the odds ratio, flat Earthism was less likely to appear in the Russia tweets than in the control sample (0.565, 95% CI [0.14–2.29]), however, the confidence interval does not indicate this result to be significant. It is possible that a more complex sentiment analysis may reveal more significant findings, however, from this study, it is likely that flat Earth content, as much as it was there, formed part of the IRA’s means of camouflage operations on Twitter, rather than being a targeted topic in its own right.

GMO

GMO content made up 0.011% of the Russia Tweets. The highest repeated tweet was again seen a frequency of 10 repeats, all posted as retweets on 24 August 2016 from United States originating accounts. The tweet was: ‘GMO’s #MustBeBanned’, an undeniably clear message. At eight repeats each, two tweets followed behind this one: ‘#MakeMeHateYouInOnePhrase Organic, gluten-free, non-GMO veggie burger’ (all retweeted on 17 October 2016) and ‘Non-GMO, locally sourced kale/quinoa blend that is artisanally extracted & liquefied so that it can be vaped. #HipsterSchoolSuppliesList’ (all retweeted on 3 August 2016). These are both examples of the ‘hashtag gamer’ category of Russian troll, who used trending hashtags to engage in conversations outside of their own account network (Linvill et al., 2019; Linvill and Warren, 2019). Both can also be interpreted as mocking the ideology behind the first tweet, with cynicism implied towards the non-GMO food movement.

In the control sample, GMO content made up 0.010% of all tweets. Odds ratio analysis on the frequency of this topic gave a value of 1.06 (95% CI [0.34–0.33]), showing the GMO content in the Russia Tweets was approximately equal in proportion to the GMO content in our control sample. This correlation was deemed significant by the confidence interval, suggesting there was a concerted effort to align the Russia Tweets frequency of GMO content with normal Twitter behaviour (camouflage).

Ebola virus

Content related to the West African Ebola virus epidemic made up 0.020% of tweets in the Russia Tweets database, compared to zero instances of associated terminology in the control sample. This may be due to the control sample’s range (across 2016) and a possible reduction in public attention towards the end of the epidemic’s time frame of 2013–2016 (Kasereka Masumbuko et al., 2018). For analysis purposes, 1 positive result in the control sample was estimated and provided an odds ratio of 6.11 (95% CI [0.86–43.45]). Given the large confidence interval, it is difficult to gauge the significance of this finding, and given the epidemic’s strong associations with a particular time frame, a control sample from 2014 or 2015 might be a more accurate representation of public discussion to which to compare the Russia Tweets. Indeed, within the troll tweets, the most keyword related tweets appeared largely throughout 2015, with diminishing activity being seen into 2017. No repeat data were collected for this topic.

Zika virus

Keywords related to the Zika virus appeared more frequently in the Russia Tweets than the ebola keywords, at 0.077%, and about twice as frequently as the control sample with an odds ratio of 2.11 (95% CI [1.17–3.81]). It is notable that the peak activity on this topic by the Russian associated troll accounts occurred throughout 2016, with almost no mention of the virus in the years prior. If these tweets were as an act of camouflage, it is possible that if public discussion increased in 2016 then the troll discussion actively increased alongside, amplifying the thoughts of real users about the virus. No repeat data were collected for this topic.

Vaccination

Following Broniatowski et al. (2018), we expected a level of significant vaccine content in our data sampling, and results supported this prediction. Vaccination and its related search terms made up 0.039% of the Russia Tweets content, compared with 0.0033% in the control sample. There was a significant increase of content in the Russia Tweets, with the odds ratio indicating a likelihood of presence in the Russia tweets 11.7 times higher than the control sample (95% CI [1.65–85.16]). At 23 tweets, the most repeated tweet in this topic was ‘Diseases Expert Calls for White Genocide Since Most Vaccine Deniers are White [LINK]’ which were all posted as original content tweets on 19 September 2017 from a mix of United States and Unknown Region accounts. Various different links were provided across the tweets that now direct to a domain-purchasing site, possibly suggesting a dedicated disinformation website had been created at the time of publishing with a domain that has now lapsed. The next most repeated tweet (n = 16) ‘1. Get flu shot 2. Avoid being invited to Xmas parties. 3. Get flu 4. Say Happy Holidays 5. Find that elf on shelf #ToDoListBeforeChristmas’ seemed to fit better into a camouflage category, and all instances of the tweet were retweeted on 28 November 2016 from United States accounts.

Climate change

Of the six topics investigated, climate change content was the most present in the data, with more than 1 in every 1000 Russian tweets related to climate change (0.144%). The content was 8.6 times more likely to appear in the Russia Tweets than the control sample (95% CI [3.59–20.75]), showing a significantly out of proportion presence when compared with normal Twitter use.

The topic of climate change also saw the highest frequency of repeat tweets, with 21 separate tweets being repeated 20 times or more, and an additional 60 tweets repeated between 5 and 19 times. Tweets repeated 20 or more times are presented below in Table 1. A good portion of the tweets in this category contain a link that redirects to the same domain sale website seen in the vaccination results, indicating again the likelihood of a dedicated disinformation website. There is a notable presence of climate change denial content in the repeats, seen in tweets such as ‘ . . . 97% SCIENTISTS #ClimateChange IS AN URGENT PROBLEM IS FICTION . . . ’ (n = 30), ‘Bill Nye Ties Hurricanes to Climate Change, Gets Shut Down by a REAL Scientist [LINK]’ (n = 24) and ‘Liberals are Pushing to Make “Climate Change Denial” a CRIME’ (n = 24). Many additional tweets are ambiguous in meaning without the link provided, which is perhaps a tactic in its own right if the IRA was employing ‘click bait’ article titles.

Table 1.

Climate change related tweets repeated 20 or more times in the Russia Tweets.

Repeated Tweets	Frequency
Disgraced NY Times Forced to Admit They LIED on Climate Change Piece	63
‘The View’ Claims Jesus Cares More for Refugees and Climate Change Than Abortion	35
#climatechange zealots hypothesize about Armageddon scenarios while holding a straight face. #remarkable	32
#Obama Claims #ClimateChange Greatest Threat @POTUS ➰ FLOTUS FLY SEPARATE PLANES #JunkScience #TCOT #PJNET http://t.co/tKDHRRPUE2	32
RT @DrMartyFox: 97% SCIENTISTS #ClimateChange IS AN URGENT PROBLEM IS FICTION. http://t.co/LuH7L6JYtQ #JunkScience #PJNET	30
Hey Liberals! THIS is Why Irma Got So Strong and It’s NOT Global Warming [LINK]	24
SCIENTISTS – Yellowstone Volcano a Bigger Threat than Climate Change! [LINK]	24
Pope Francis Thinks ‘Climate Change’ is the Root of All Evil	24
BREAKING: Mccain Sides with Gore, Dems on ‘Climate Change’ Agenda	24
Pope Declares People Have ‘Moral Responsibility’ to Believe in Climate Change [LINK]	24
Bill Nye Ties Hurricanes to Climate Change, Gets Shut Down by a REAL Scientist [LINK]	24
PATHETIC STUNT – CNN’s Acosta Tries to Use Hurricanes to Push ‘Climate Change’	24
Joy Behar Suggest Hurricanes Should Be Named After Climate Change Deniers [LINK]	24
Hillary Claims California Wildfires are Caused By ‘Climate Change’ [LINK]	24
John Kerry Just Argued ‘Climate Change’ is a Bigger Risk than North Korea [LINK]	24
Liberals on ‘The View’ Believe Al Gore’s Movie is Proof of Climate Change [LINK]	24
Liberals are Pushing to Make ‘Climate Change Denial’ a CRIME!	24
Trump is About to Trash Obama’s BIGGEST Climate Change Regulation	23
CNN Anchor Tries to Link Hurricane Harvey to Climate Change, FAILS MISERABLY	23
Lindsey Graham has an entirely reasonable position on climate change, sometimes http://t.co/kZMZka7Ja7 http://t.co/55juNRjwAT	22
@POTUS- We R Being Lied 2!! #ClimateChange biggest threat as it churns $ for corrupt govt! #JunkScience #PJNET	20

The most repeated phrase was ‘Disgraced NY Times Forced to Admit They LIED on Climate Change Piece’ which was seen 63 times in the data across a number of different hashtags and associated links. All of these tweets were original content tweets posted on 10 August 2017, with a 70%/30% split between Unknown Region and United States account origin, respectively. The links again lead to the same domain-purchasing website seen in the most repeated vaccination tweet. An unusual observation in the 63 repeated tweets was the inclusion of hashtags that seemed to reference the Twitter handle it was posted from at the start of some, but not all, of the tweets. For example, @amberlinetr’s tweet began with #amb, @darcyystr’s tweet with #darccy, @ivaassntr with #iv, @anisilope with #lope, @arabmtr with #arbt and so on. These handle associated hashtags were present at the start of 48 of the 63 repeats. This may have been a mechanism of creating unique hashtags to track the reach of an individual tweet, or provide some personnel performance information within the IRA organisation structures.

In total, we found over 4000 tweets relating to climate change in the Russia Tweets, far surpassing any other topic investigated here. This suggests a considered agenda in the inclusion of climate change discussion in the IRA’s disinformation strategies. Further access to the links associated with each tweet would allow for a better sentiment analysis; however, it is clear from the repeated tweets that the content contains tweets both accepting climate change and, most importantly, when considering impact and motive, denying it.

Summary

Categorical breakdown of the results shows that science emphasis in the Russia Tweets was not seen across all topics. Due to the time frame of the control sample taken and the peak of public awareness of the Ebola virus, conclusions are difficult to draw for this topic. However, discussion of the Zika virus epidemic was somewhat amplified in the Russia Tweets. GMO and flat Earthism content were on par with the control, while discussion of climate change and vaccination was seen in a far greater amount in the Russia Tweets. Each topic’s relative proportion (percentage) in both the control sample and our data set is presented in Figure 1.

Figure 1.

Percentage of topic frequency in the Russia Tweets vs Control Sample for flat Earth, GMO, Ebola virus, zika virus, vaccines and climate change.

This figure also highlights the relative proportion of climate change content to vaccine content: though the boost of vaccine content was higher than that of climate change (odds ratios of 11.7 and 8.6, respectively), the sheer amount of climate change content was far greater than that of vaccines. This suggests a targeted effort within the IRA to increase and perhaps influence the climate change discussion. Although we have not completed a quantitative sentiment analysis to definitively confirm what proportion of these tweets were pro- or anti-climate science, qualitative analysis of the most repeated tweets shows climate change denial was certainly present in the data, which may have impacts on the wider climate change discussion on Twitter and in broader society.

3. Discussion

These results confirm the presence of pseudoscience in the troll tweets of the Russian IRA. Our initial hypothesis – that science was widely being targeted – was not supported with the data. Rather, our results found certain topics were chosen as targets over others. It is likely that the science content seen throughout the tweets is the result of both motivated and camouflage activities, as we have seen discussion of some topics on par with the frequency in our control sample (GMO, flat Earthism) while others (vaccination, climate change, Zika) are significantly more common in the Russia Tweets. We observe and discuss here some possibilities motivating the choice of the two topics seen to have been strongly promoted. Both vaccines and climate change science have a broad societal impact from misinformation and disinformation, and both involve positions strongly aligned with political ideologies (Davis, 2019; Metag et al., 2017; Nurse and Grant, 2019), so these two topics can be utilised as tools for the spread of political discord in America. It very much appears that the attack was not on science per se, but rather that the IRA strategically chose science topics to influence wider America. However, the effects this may have had on science understanding are real and potentially long lasting. Thus, it will be important for the science community and society at large to remain vigilant to these campaigns of cyber influence, as the use of online trolls and bots is certainly not confined to Russia. Here we explore these points in greater detail.

So why these topics? We suggest one motivating similarity between the two science topics we found most emphasised in the Russia Tweets may be the wider reaching societal impact from misinformed citizens. Of the six topics chosen to investigate, climate denial and anti-vaccination beliefs arguably pose the biggest threat to the well-being of a society. This social implication was not shared to the same extent by the two topics we found to be least promoted by the IRA (GMO, flat Earthism). For vaccines and climate change, the issue goes beyond a personal one: an anti-vaccination movement is dangerous because it reduces herd immunity, and disagreement on the need for climate action risks the catastrophic predictions associated with no action (Fine et al., 2011; McAdam, 2017; Oreskes and Conway, 2010). The spread of AIDS disinformation in the 1980s had similar societal impacts: it fuelled misunderstanding of an epidemic that was already misunderstood, and further promoted homophobia (Boghardt, 2009; Greene, 2007; O’Hare et al., 1996). This period saw a great amount of stigma around HIV infection and transmission, and this fear was exploited by the then-KGB (Boghardt, 2009).

The peak activity of the Russian trolls also aligns with the increase of ‘post-truth’ conversations observed by Lewandowsky et al. (2017). This interesting correlation may be unrelated, but it could also suggest that Russia was either encouraging or capitalising on this trend with their disinformation campaign. It is possible that this ‘post-truth era’ has also been influenced somewhat by their various disinformation efforts over the years, though the likelihood of this is difficult to say. There are some empirical observations that may support this, such as the lingering AIDS conspiracies that are descendants of Operation INFEKTION.

The topics may also have been chosen due to their influence or association with political beliefs. This is somewhat similar to the earlier observation; however rather than considering the motivation of the trolls to be targeting society at large, it considers them in their political role.

It has already been shown by Linvill et al. (2019) that there was a broad support for right wing and an attack on left wing politics in the Russia Tweets. In America, politics and science beliefs are seen to be aligned strongly or weakly for certain topics, and the Pew Research Center indicates that there is strong alignment between political party and belief regarding opinions of anthropogenic global warming (Iyengar and Massey, 2019; see also Nurse and Grant, 2019), while weak alignment on the issue of GMO foods, which may explain why this topic was less targeted (Rainie, 2017). Studies into the American understanding of climate change science have noted that ‘the politicization of climate change drives conflict in climate-related discourse and constrains climate policy’ (Ballew et al., 2019: 5). Furthermore, online discussions of climate change and climate politics are found to be dominated by non-experts, which may provide an environment ripe for disinformation activity (Schafer, 2012). This has been seen in Twitter discourse following the release of the Fifth Assessment Report from the Intergovernmental Panel on Climate Change (IPCC), where non-experts accounted for the majority of the responding discussions (Newman, 2016), and more generally, it has been found that climate politics and climate change discussion in online spaces occurs mostly without input from scientists or scientific institutions (Schafer, 2012). The Yale Program on Climate Communication conducted surveys during the 2016 election campaign which also indicated that those sceptical or unsure about climate science were more likely to vote for Donald Trump over Hillary Clinton, but notably global warming was not an issue important to their voting choices (Roser-Renouf et al., 2016). Thus, the focus on climate change discussion by the Russian trolls is unlikely to have been a tactic to influence election choices, but rather to access a subset of Americans with whom to engage. In this case, those more likely to cast a Republican vote.

Similar correlations between ideology and vaccination beliefs have also been observed (Baumgaertner et al., 2018). Furthermore, Davis (2019: 360) suggests that anti-vaccination discourse, in general, is part of a larger movement of ‘anti-public’ discourse, which ‘[intends] to mislead rather than inform’. He argues that anti-public discourse of topics such as anti-vaccination and anti-climate science undermine principles of democratic debate, and notes it has been particularly fuelled by online media, again, pointing to the ripeness of social media as a platform for disinformation campaigns. It is not unreasonable then to suggest that discussion of climate change and vaccine science may have been chosen by the Russian trolls to access specific political spheres and further increase polarisation, and thus conflict, across the political spectrum.

There may be other motivated bodies moving against climate action and vaccination for various reasons, so it would be naive to assign this phenomenon to Russia alone (Oreskes and Conway, 2010). However, it is likely that disinformation campaigns in these topics would have long lasting impacts on the Twitter users exposed, and by extension on the societies in which they operate. Once accepted, false information is difficult to correct in the receiver, and biases remain even in individuals who acknowledge the correction (De keersmaecker and Roets, 2017). Cognitive psychology, it seems, sides with the IRA. If the trends in this research were found to continue throughout the wider social media campaign of the IRA, it is possible that Russian-influenced climate and vaccine science reached tens of millions of people across their social media channels.

We do not think attacking science in general was the motive of the IRA. Reports have indicated time and again the clear political motive behind their activities around the 2016 US presidential election. However, with the main objective of historical KGB Active Measures campaigns being to ‘drive a wedge’ (Andrew and Mitrokhin, 2001) in American society, these topics have almost undoubtedly been capitalised upon to advance this motive. Functional democracies rely on well-informed public, and damaging this understanding of truth is detrimental to modern American democracy (Stebenne, 2018). However, the use of science to further this agenda is a real challenge for the public understanding of science and correcting it is a difficult task. Despite sitting within a bigger political narrative, the impacts on science understanding could be long reaching, as has been seen in the disinformation campaigns of the Cold-War era.

There is good evidence that social media trolling tactics are being used by countries other than Russia. Research at Oxford University has documented the use of ‘cyber troops’ in 28 countries, with the purpose of influencing public opinion through social media (Bradshaw and Howard, 2017). Not all of these troops are human, however, and large influence campaigns are being run by automated bots (Bradshaw and Howard, 2017). It is worth noting that the IRA also employed the use of bots in their activities, alongside that of the human-run troll accounts, and some other countries are also adopting this mixed strategy in their cyber campaigns (Bradshaw and Howard, 2017; The New York Times, 2019). Twitter has also provided the details of 3000 Iranian accounts associated with a political influence campaign, alongside the 4000 Russian ones released (Twitter, 2018). This may indicate a growing cyber trend in public influence and propaganda tactics, and poses a challenge to the social media companies that are the platform for so much public communication today. The US Federal Bureau of Intelligence has boosted their counterintelligence operations in preparation for the 2020 US Presidential election, and it will be important to remain aware and critical of these social media influence campaigns which are likely to rear their heads again in the coming years (Barnes and Goldman, 2019).

4. Limitations and future research

The tweets provided by Linvill and Warren are not a complete set of the known Russian troll tweets, although they make up a considerable portion. Though the raw numbers of tweets and repeats may not be representative of the full set, we are confident that patterns observed in the sample set used (n = 2,973,371) remain significant and point to an intentional pushing of climate science and vaccination content.

It is very likely that we did not capture all science and science-related topics in our initial exploratory process. Furthermore, within each of the six topics chosen it is probable that we missed some related keywords or keyword variations. However, the breadth of key words used and the method for determining those keywords would likely have yielded most, if not all, of each topic’s related tweets, and missed tweets are unlikely to have changed the overall conclusions drawn. It should also be noted that while every effort was made to find a clean control sample, it cannot be ruled out that elements of the control sample were affected by the IRA tweets. Moreover, the relationships between the key themes of the 2016 Presidential election – and Trump’s rhetorical orientation – and the topics chosen by the IRA are difficult to disentangle. Did the IRA follow Trump, or did both mine a vein of discord in American politics? While direct conclusions would be difficult to draw, future research could explore the timing of key developments in Donald Trump and Hillary Clinton’s rhetoric and similar threads in the IRA’s tweets.

We had external limitations in the production of our control sample due to Twitter’s data collection rules, which in part respond to the European Union’s GDPR. Our sample, taken from 2016, may not have been representative of the entire window of activity of the Russian trolls (2014–2017), particularly for the time frames of the virus epidemic topics (Kasereka Masumbuko et al., 2018; Valentine et al., 2016). This may have influenced the comparisons should there have been a particular event that spiked a lot of tweet activity. The years 2015–2017 were the height of the Russian Troll activity however, so we are satisfied that the year chosen (a) is the central year of this range and (b) encompasses the date of the 2016 US election.

There is much research to be done on the Russia Tweets, especially as the 2020 US Presidential election nears. Full sentiment analysis of the topics investigated in this research would give a stronger evidence base from which to infer Russian motives, and expanding the topics investigated to include a wider range of science may help reveal further patterns in the IRA’s targets. More generally, investigation is needed into the day-to-day workings of the IRA through observations of patterns in date, retweet behaviour, account type and so on. We noted some potentially interesting date patterns, such as an increase in tweeting around 11 September 2017 (although this was not investigated beyond researcher observation and is not confirmed). Better understanding of how the IRA operates may help identify future trolling and disinformation efforts when they appear.

Footnotes

Acknowledgements

Thanks to Yuanyuan Shang and Dashiell van Wyk. Yuanyuan for her help generating the authors’ control sample and her experience in the social media research domain, and Dashiell for his help with running the search term analysis in the control sample, and for being the first author’s resident coding man.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Indigo J. Strudwicke

Will J. Grant

Notes

Author biographies

Indigo J. Strudwicke is a 4th-year student studying a Bachelor of Philosophy (Science) at the Australian National University. She is interested in the use of science in politics, and the politics of science.

Will J. Grant is Senior Lecturer at the Australian National Centre for the Public Awareness of Science, the Australian National University. His research and teaching focuses on the relationship between science, politics and new technology.

References

Andrew

Mitrokhin

(2001) The Sword and the Shield: The Mitrokhin Archive and the Secret History of the KGB. New York: Basic Books.

Ballew

Leiserowitz

Roser-Renouf

Rosenthal

Kotcher

Marlon

, et al. (2019) Climate change in the American mind: Data, tools, and trends. Environment: Science and Policy for Sustainable Development 61(3): 4–18.

Barnes

Goldman

(2019) F.B.I. warns of Russian interference in 2020 race and boosts counterintelligence operations. The New York Times. Available at: https://www.nytimes.com/2019/04/26/us/politics/fbi-russian-election-interference.html

Baumgaertner

Carlisle

Justwan

(2018) The influence of political ideology and trust on willingness to vaccinate. PLoS ONE 13(1): e0191728.

Bogart

Thorburn

(2005) Are HIV/AIDS conspiracy beliefs a barrier to HIV prevention among African Americans? Journal of Acquired Immune Deficiency Syndromes 38(2): 213–218.

Boghardt

(2009) Soviet bloc intelligence and its AIDS disinformation campaign. Studies in Intelligence 53: 1–24.

Bradshaw

Howard

(2017) Troops, trolls and troublemakers: A global inventory of organized social media manipulation. In: Woolley

Howard

(eds) Project on Computational Propaganda. Oxford: Oxford Internet Institute, pp. 47.

Broniatowski

Jamison

AlKulaib

Chen

Benton

, et al. (2018) Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. American Journal of Public Health 108(10): 1378–1384.

Brooks

Allen

Jr Regan

Mutchler

Cervantes-Tadeo

Lee

S-J

(2018) HIV/AIDS conspiracy beliefs and intention to adopt preexposure prophylaxis among black men who have sex with men in Los Angeles. International Journal of STD & AIDS 29(4): 375–381.

10.

Creswell

(2014) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th edn. Thousand Oaks, CA: SAGE.

11.

Davis

(2019) ‘Globalist war against humanity shifts into high gear’: Online anti-vaccination websites and ‘anti-public’ discourse. Public Understanding of Science 28(3): 357–371.

12.

De keersmaecker

Roets

(2017) ‘Fake news’: Incorrect, but hard to correct. The role of cognitive ability on the impact of false information on social impressions. Intelligence 65: 107–110.

13.

Fine

Eames

Heymann

(2011) ‘Herd immunity’: A rough guide. Clinical Infectious Diseases 52(7): 911–916.

14.

Gabbatt

(2019) Trump says Russia helped elect him – Then quickly backtracks. The Guardian. Available at: https://www.theguardian.com/us-news/2019/may/30/trump-russia-2016-election-helped-elect-president

15.

Greene

(2007) A history of AIDS: Looking back to see ahead. European Journal of Immunology 37(S1): S94–S102.

16.

Iyengar

Massey

(2019) Scientific communication in a post-truth society. Proceedings of the National Academy of Sciences of the United States of America 116(16): 7656–7661.

17.

Kasereka Masumbuko

Underschultz

Hawkes

(2018) Ebola virus epidemic in war-torn eastern DR Congo. The Lancet 392(10156): 1399–1401.

18.

Kuiken

Schuth

Spitters

Marx

(2017) Effective headlines of newspaper articles in a digital environment. Digital Journalism 5(10): 1300–1314.

19.

Lewandowsky

Ecker

UKH

Cook

(2017) Beyond misinformation: Understanding and coping with the “post-truth” era. Journal of Applied Research in Memory and Cognition 6(4): 353–369.

20.

Lewandowsky

Ecker

UKH

Seifert

Schwarz

Cook

(2012) Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest 13(3): 106–131.

21.

Linvill

Warren

(2018) The Russia Tweets [Website]. Available at: http://russiatweets.com, published by FiveThirtyEight and Defending Democracy Together.

22.

Linvill

Warren

(2019) Troll factories: Manufacturing specialized disinformation on Twitter. Available at: http://pwarren.people.clemson.edu/Troll_Factories_v2_Linvill_Warren.pdf

23.

Linvill

Boatwright

Grant

Warren

(2019) ‘THE RUSSIANS ARE HACKING MY BRAIN!’ Investigating Russia’s Internet Research Agency twitter tactics during the 2016 United States presidential campaign. Computers in Human Behavior 99: 292–300.

24.

Marshall

(2018) The universe is an egg and the moon isn’t real: Notes from a Flat Earth conference. The Guardian, 2 May. Available at: https://www.theguardian.com/science/blog/2018/may/02/the-universe-is-an-egg-and-the-moon-isnt-real-notes-from-a-flat-earth-conference

25.

McAdam

(2017) Social movement theory and the prospects for climate change activism in the United States. Annual Review of Political Science 20(1): 189–208.

26.

Metag

Füchslin

Schäfer

(2017) ‘Global warming’s five Germanys: A typology of Germans’ views on climate change and patterns of media use and information. Public Understanding of Science 26(4): 434–451.

27.

Nattrass

(2013) Understanding the origins and prevalence of AIDS conspiracy beliefs in the United States and South Africa. Sociology of Health and Illness 35(1): 113–129.

28.

Newman

(2016) Tracking the release of IPCC AR5 on Twitter: Users, comments, and sources following the release of the Working Group I Summary for Policymakers. Public Understanding of Science 26(7): 815–825.

29.

Nurse

Grant

(2019) I’ll see it when I believe it: Motivated numeracy in perceptions of climate change risk. Environmental Communication. Epub ahead of print 16 June. DOI: 10.1080/17524032.2019.1618364.

30.

O’Hare

Williams

Ezoviski

(1996) Fear of AIDS and homophobia: Implications for direct practice and advocacy. Social Work 41(1): 51–58.

31.

Office of the Director of National Intelligence (2017) Assessing Russian Activities and Intentions in Recent US Elections. Washington, DC: Office of the Director of National Intelligence.

32.

Oreskes

Conway

(2010) Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming, 1st U.S. edn. New York: Bloomsbury Press.

33.

Patel

Lee

Redd

Clemmons

McNall

Cohn

, et al. (2019) Increase in measles cases – United States, January 1–April 26, 2019. Morbidity and Mortality Weekly Report (MMWR) 68: 402–404.

34.

Platon

(2016) B.o.B tries to convince twitter earth is flat, Neil deGrasse Tyson responds. Hip-Hop. Available at: https://www.billboard.com/articles/columns/hip-hop/6851778/bob-earth-flat-twitter-neil-degrasse-tyson

35.

Rainie

(2017) U.S. public trust in science and scientists. Internet & Technology. Available at: https://www.pewinternet.org/2017/06/27/u-s-public-trust-in-science-and-scientists/

36.

Roser-Renouf

Maibach

Leiserowitz

Rosenthal

(2016) Global Warming’s Six Americas and the Election, 2016. Yale University and George Mason University. New Haven, CT: Yale Program on Climate Change Communication.

37.

Schafer

(2012) Online communication on climate change and climate politics: A literature review. WIREs Climate Change 3(6): 527–543.

38.

Schoen

Lamb

(2012) Deception, Disinformation, and Strategic Communications: How One Interagency Group Made a Major Difference. Washington, DC: National Defense University.

39.

Shultz

Godson

(1984) Dezinformatsia: Active Measures in Soviet Strategy. Washington, DC: Pergamon-Brassey’s.

40.

Stebenne

(2018) Is American democracy engandered? Drake Law Review 66(4): 919–950.

41.

Szumilas

(2010) Explaining odds ratios. Journal of the Canadian Academy of Child and Adolescent Psychiatry = Journal de l’Academie canadienne de psychiatrie de l’enfant et de l’adolescent 19(3): 227–229.

42.

The New York Times (Producer) (2018) Read the special counsel’s indictment against the Internet research agency and others. Politics. Available at: https://nyti.ms/2C1yTjt

43.

The New York Times (Producer) (2019) Read the Mueller report: Searchable document and index. Politics. Available at: https://nyti.ms/2ZlVwru

44.

Twitter (2018) We’re focused on serving the public conversation. Elections Integrity. Available at: https://about.twitter.com/en_us/values/elections-integrity.html

45.

United States Department of State (1987) Soviet Influence Activities: A Report on Active Measures and Propaganda, 1986 – 87. Washington, DC: United States Department of State.

46.

Valentine

Marquez

Pammi

(2016) Zika virus epidemic: An update. Expert Review of Anti-Infective Therapy 14(12): 1127–1138.

47.

WikiLeaks (2016) Search the DNC email database. 22 July. Available at: https://wikileaks.org/dnc-emails/