Social Media,Science,and Attack Discourse: How Twitter Discussions of Climate Change Use Sarcasm and Incivility

Abstract

Conflict in online discussions of science has the potential to polarize individuals’ perceptions of science, yet science communication scholarship has paid little attention to systematic study of how verbal attacks play out in online discussions of science. This study analyzes sarcasm and incivility in Twitter discussions of climate change during an extreme weather event (n = 4,094). We found instances of incivility and sarcasm were low overall. Incivility was used in association with political topics, and both incivility and sarcasm were used alongside skeptical perspectives of climate change and by those who mention right-leaning politics in their profiles.

Keywords

incivility sarcasm climate change weather social media Twitter

Weather events are conduits for public perceptions around climate change (e.g., Borick & Rabe, 2014; Capstick & Pidgeon, 2014; Egan & Mullin, 2012). Research shows that traditional media spaces like television weathercasts catalyze the connections people make between everyday temperatures or extreme weather events and climate change (Maibach, Placky, et al., 2016). Alongside these connections between weather and climate change perceptions made via more traditional forms of media, people are also increasingly turning to online media to seek out information about the weather (Lazo, Morss, & Demuth, 2009; Purcell, Rainie, Mitchell, Rosenstiel, & Olmstead, 2010) and to discuss weather via social media (Leiserowitz, Maibach, Roser-Renouf, Feinberg, & Howe, 2013). This trend reflects a shift in which individuals are turning to online sources for seeking out, understanding, and discussing scientific issues more broadly (e.g., Anderson, Brossard, & Scheufele, 2010; Brossard, 2013; Su, Akin, Brossard, Scheufele, & Xenos, 2015; Uldam & Askanius, 2013). Social media are increasingly prominent in science communication across a range of issues.

Yet, despite this shift toward online information consumption about scientific issues, online discussions harbor conflict, negativity, and personal attacks. Scholarship has focused on this phenomenon, analyzing how uncivil language is portrayed in online discussions (e.g., Coe, Kenski, & Rains, 2014), why people use incivility online (e.g., Cicchirillo, Hmielowski, & Hutchens, 2015), and what effects incivility can have on public perceptions (e.g., Anderson, Yeo, Brossard, Scheufele, & Xenos, 2016; Borah, 2014; Hwang, Kim, & Huh, 2014). Research in science communication shows that incivility in online comments can polarize risk perceptions along existing divides (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014). This speaks to the need to better understand how incivility pervades online discussions of scientific issues that shape public perceptions of and engagement with science.

Most of this research is located under the broad construct of incivility, commonly defined as an explicit attack that insults another person’s character and detracts from healthy, heated debate (Brooks & Geer, 2007), but scholars have tested other important language to online comments. Research has shown that people are using more subtle forms of attack discourse, such as sarcasm, in online discussions (Angouri & Tseliga, 2010; Rowe, 2015; Zappavigna, 2012). Sarcasm is a discursive strategy intended to provoke someone through the use of irony, or the use of contradiction to convey meaning, and often relies on humor or exaggerated language (Whalen, Pexman, & Gill, 2009). Early evidence shows that incivility and sarcasm together are harmful to participation in online discussions (Ziegele & Jost, 2016).

Despite the trend toward science information consumption in online environments and the growing attention to incivility in online discussions, little research has examined how social media discussions of science exhibit personal attacks through uncivil or sarcastic language. Our study adds an important systematic approach to assessing the extent of incivility and sarcasm in online discussions of science. We explore the question: How do incivility and sarcasm enter online discussions of climate change?

Climate change is a politicized topic, subject to polarization around ideological and partisan lines (McCright & Dunlap, 2011). As such, online climate change discussion may be particularly prone to the use of incivility and sarcasm. Using a content analysis of climate change tweets during an extreme weather event (n = 4,094), we find that overall instances of sarcasm and incivility were low. However, when discussions were skeptical of climate change, they engaged with both tones of incivility and sarcasm. Our findings also show that while incivility was associated with political topics in the tweet, sarcasm was not. Neither tone was associated with an increased mention of politics in the profiles of the users, though tweets that did use incivility and sarcasm tended to be associated with users who indicated an affinity for right-leaning political orientation in their profile information but not for users who indicated left-leaning political orientations.

Online Incivility

Scholars have defined incivility as an explicit attack that goes beyond impoliteness or the heated debate that is beneficial to fostering democratic discourse (Brooks & Geer, 2007; Papacharissi, 2004). Incivility is detrimental to deliberative debate and is an attack on one’s character. Name-calling, mockery, negative character exaggerations through spin, or the attachment of negative emotion to another person are examples of incivility (Gervais, 2017).

Research shows that incivility is consistently found in various outlets of online discussion, with most systematic analyses focusing on commenting sections. A study of one newspaper’s website found that incivility was present in more than 20% of its comments, with name-calling being the most prevalent form of incivility (Coe et al., 2014). Another study examining mainstream media and blog site comments found that more than 25% of all comments contained insulting language, such as demeaning words (Lee & Pang, 2014). A study of Usenet discussion threads found uncivil posts made up 14% of all posts (Papacharissi, 2004). In short, there is evidence incivility commonly occurs across a range of types of online discussion. Little empirical research, however, examines incivility in science-based online discussions despite attention to the topic in the scientific blogosphere and discussions of it in the broader discourse on climate change (“and Then There’s Physics,” 2015; Kahan, 2012). One study, however, found that in comments on newspaper stories of climate change, “degrading comments”—defined as “lowering character, quality, or rank via ad hominem attacks,” a similar definition of incivility that other scholars have used—made up 12% of comments (Graham & Wright, 2015, p. 11).

Incivility as a rhetorical strategy is valued by commenters who want freedom of expression (R. P. Clark, 2007), to be entertained (Alonzo & Aiken, 2004), and to be humorous in a controversial way while causing another person emotional distress (Phillips, 2011). When individuals encounter uncivil discussion on a message board, they react with anger and aversion (Gervais, 2015). In addition, those who watch more uncivil media and who are more likely to engage in verbal aggression (e.g., insult others) in general are more likely to engage in uncivil tactics online (Cicchirillo et al., 2015). In short, one might use uncivil language not only to entertain oneself but also attack someone else, and these actions feed on each other.

Several empirical studies show that distress, anger, and aversion go beyond just the individuals involved in the direct discussion. Incivility can have deleterious effects on people who are simply exposed to the comments. Research shows that it contributes to perceptions that others’ attitudes are polarized (Hwang et al., 2014), it increases narrow-mindedness (Borah, 2013), it develops perceptions of bias in online posts (Anderson et al., 2016), it limits satisfaction with online discussions (Gervais, 2015), and it fosters anger (Gervais, 2017). Uncivil comments and their effects cross a range of general issues in addition to the science-related perceptions, in particular.

Sarcasm in Online Discourse

The Oxford English Dictionary defines sarcasm as, “a sharp, bitter, or cutting expression or remark; a bitter gibe or taunt.” We define sarcasm as a form of provocation that relies on irony and humor (Whalen et al., 2009) and is a subtler type of attack than other forms of incivility. In discourse, sarcasm is often conveyed through the use of nonliteral language such as irony, which assumes that the audience will understand that the speaker’s intended meaning is the opposite of what he or she is actually saying (Whalen et al., 2009). Sarcasm and irony are closely aligned concepts; verbal irony is often used as an operationalization of sarcasm in discourse analysis (Kunneman, Liebrecht, van Mulken, & van den Bosch, 2014). While most sarcasm uses irony, not all irony is sarcasm, as sarcasm requires an object of critique (Attardo, 2000). In our study, our operationalization stated that sarcasm needed to include irony as part, but not all, of the construct.

Because sarcasm in discourse is coupled with irony, its success as a discursive strategy relies on its correct interpretation by the listener or receiver (Jorgensen, 1996; Whalen et al., 2009); therefore, common ground should exist between participants in the conversation (Averbeck & Hample, 2008). Some have noted that in using sarcasm, the speaker assumes a pretense of irony, of which the listener may or may not be aware (H. H. Clark & Gerrig, 1984). Therefore, sarcasm is best used in contexts where it is unlikely to be misinterpreted. Users of sarcasm in text-based computer-mediated communication rely on multiple characteristics to communicate it. For instance, speakers often use other discursive markers such as heavy punctuation or valence modulations—such as adjectives or adverbs—in conjunction with the ironic statement to more clearly communicate sarcastic intent (Whalen et al., 2009).

While motivations for online sarcasm use are not as well-known, Zappavigna (2012) argued that humor, which can include sarcasm, is used on social media to maintain relationships and express affiliation. Sarcasm has been characterized as a form of wit “intended to insult, mock, or amuse,” (Riloff et al., 2013, p. 704). It has been characterized as a device for joking, teasing, word play, and mockery (Boxer & Cortés-Conde, 1997). Sarcastic attacks do have bite, but can allow both sides to save face and maintain a relationship under the guise of joking. Attacks directed at outsiders can bond the conversation participants in a shared relational identity at the expense of the target (Boxer & Cortés-Conde, 1997). As social media are in many cases at least semipublic forums for communication, sarcasm may be especially useful in online communication contexts to identify and appeal to like-minded others through the critique of outsiders, while maintaining an appearance of civility through plausible deniability.

On Twitter, sarcasm often takes the form of expressing a negative sentiment in positive terms, or occasionally vice versa (Kunneman et al., 2014; Riloff et al., 2013). By using opposite framing, sarcasm functions as a type of language intensifier, which is used in order to both express the attitude of the speaker toward the object and to influence the receiver in some way regarding the object (Bowers, 1964). As sarcasm in text-based media is used with other intensifiers or markers for clarity, sarcasm in text-based, online communication contexts is a rhetorical device for convincing others of the merits or faults of a particular object. Sarcasm plays dual roles in online discourse around potentially contentious issues as both a tool to save face while engaging in provocative or uncivil speech and as a persuasive technique.

Research has found some evidence of sarcasm in various online discussions. One analysis of newspaper comments found that sarcasm was present in 10% of all comments (Rowe, 2015). Papacharissi’s (2004) study of Usenet discussions analyzed sarcasm as one type of impolite message, which included name-calling and vulgarity, and found that impoliteness made up 22% of all posts. Because of the combined nature of Papacharissi’s impolite construct, it is difficult to know how much of those messages were sarcastic versus another type of impoliteness she was coding. Despite this research in other contexts, little is known about how sarcasm plays out in science-based discussions online.

Existing scholarship supports the distinction between the constructs of incivility and sarcasm. Papacharissi’s (2004) early study on incivility in online discussion threads categorized sarcasm as impolite, rather than uncivil, and others have followed this categorization (e.g., Rowe, 2015). While sarcasm fits some characterizations of public incivility, it is more subtle than an explicit attack (Papacharissi, 2004). Recent research has considered incivility and sarcasm as separate constructs. Sarcasm is a response to incivility and may prolong the deleterious effects of incivility (Ziegele & Jost, 2016).

Our review suggests sarcasm may be more than just a response to uncivil online language. While some of the motivations for using the two are the same (e.g., entertainment, humor), they may also be used for different purposes (e.g., tearing someone down for purposes of self-amusement vs. tearing someone down for purposes of being humorous in front of a like-minded other). The effects of sarcasm in the online context are not as well-studied as the effects of incivility, with little research exploring the topic (except see Ziegele & Jost, 2016). Evidence suggests both incivility and sarcasm can undermine the democratic potential of networked technologies to encourage public engagement with science. Scholarship also points to the positive potential of social media to engage individuals in science. As such, research needs to examine these two distinct constructs in different discussions of science to understand what topics and themes draw them out. Here, we explore how sarcasm and incivility occur during climate change discussions on Twitter.

Sarcasm and Incivility in Online Discussions of Climate Change

We chose to study an event that was likely to spur climate change references within a common topic of discussion: an extreme weather event. A growing body of scholarship is investigating how extreme weather events shape climate change perceptions (e.g., Joireman, Barnes Truelove, & Duell, 2010; Shao, 2015; Zhao et al., 2013). General discussion about climate change is low, with the majority of Americans reporting they rarely or never talk about the issue with their friends or family (Maibach, Leiserowitz, Rosenthal, & Cutler, 2016). Yet the majority of people in the United States (89%) have talked with others about an extreme weather event at least once, and 20% of people who have experienced an extreme weather event have discussed it over social media (Leiserowitz, Maibach, et al., 2013). Therefore, it is likely that people who are discussing extreme weather events may discuss them in conjunction with climate change. The extreme weather event analyzed here is a regional flooding event in the state of Colorado that occurred over the course of nearly a week.

In September 2013, communities along the Colorado Front Range were flooded in a disaster that affected 17 counties (Smith & Hennen, 2013). At least one mountain community was cut off by floodwaters and road damage. Many others experienced mandatory evacuations, and the National Guard assisted in rescue efforts. Thousands of homes were damaged or destroyed by the floods. As an extreme weather event, the 2013 Colorado floods received a great deal of attention from news media and individuals alike. In a survey conducted in June 2013, a few months prior to the flooding event, about 4 in 10 Coloradans reported that global warming has a moderate or large influence on flooding of rivers and lakes in Colorado, and approximately half of Coloradans reported they perceive that global warming will influence flooding of major cities in Colorado over the next 50 years (Leiserowitz, Feinberg, Howe, & Rosenthal, 2013). Considering the connection other research has made between flood events and public perceptions about climate change (e.g., Spence, Poortinga, Butler, & Pidgeon, 2011), individuals may have drawn a connection between this flooding event and climate change.

Climate change is a scientific issue punctuated by political opinions that frame how people perceive the topic (McCright & Dunlap, 2011). This reliance on politics to form perceptions about climate change is related to skepticism around questions of existence and causes of climate change (Whitmarsh, 2011). Research on content related to online climate change discussions provides evidence that social media can foster negative or skeptical climate change perceptions (De Kraker, Kuijs, Cörvers, & Offermans, 2014; Kaiser, 2017). Online discussions prevail around events that draw skepticism, such as “Climategate,” the public release of university scientists’ emails that climate skeptics said were evidence that climate scientists are manipulating data to promote climate change as human-caused (Jaspal, Nerlich, & Koteyko, 2012; Koteyko, Jaspal, & Nerlich, 2013; Porter & Hellsten, 2014). Other events, however, such as the release of the Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report elicit discussion around climate change as scientific fact versus uncertain (O’Neill, Williams, Kurz, Wiersma, & Boykoff, 2015). In short, research shows that social media discussions of climate change focus on its causes and fuel skeptical perspectives, depending on the issue context. Other examples point to the possibility of social media to foster skeptical viewpoints on other scientific topics, such as vaccines (e.g., Kata, 2012). Research also shows that Twitter users discuss climate change within like-minded communities (Williams, McMurray, Kurz, & Hugo Lambert, 2015).These are prominent indications that social media can foster inaccurate information about science, bolstering the viewpoints of individuals who are already skeptical of science.

In this study, we analyzed how incivility and sarcasm enter Twitter discussions on the topic of climate change alongside two prominent and conflict-based subtopics of climate change: politics and climate change causes. While plenty of research has examined incivility in online commenting as well as climate change in social media discussions, little research has systematically examined how different attack-based tones such as sarcasm and incivility play out in online discussions of climate change. We pose the following research questions:

Research Question 1: Does the use of incivility relate to the themes of (a) politics and (b) flooding causes and the profile mentions of (c) politics, and (d) political leanings?

Research Question 2: Does the use of sarcasm relate to the themes of (a) politics and (b) flooding causes and the profile mentions of (c) politics, and (d) political leanings?

Method

We conducted a quantitative content analysis of tweets about the 2013 Colorado flood event. Following is a description of the data and content analysis procedures.

Data

A sample of 4,094 public tweets dated between September 1, 2013, and October 31, 2013, were analyzed. These dates were chosen because they start a week and a half before the floods, ensuring we gathered data from the beginning of the conversations through the subsequent recovery period. These tweets represented a subset of a larger tweet sample (N = 210,317) related to the September 2013 Colorado floods, as evidenced by the appearance of certain keywords and hashtags. For instance, the large sample contained tweets that matched #coflood or #boulderflood. The total sample was purchased in March 2014 from API aggregation company Gnip. Gnip provided access to both real-time and historical Twitter feeds of public tweets (Gnip, n.d.). Gnip was acquired by Twitter in April 2014 (Auerbach, 2014), after the data for this study was already collected and purchased. The purchased data included the text of the actual tweet, including information about any links contained within the tweets; the User ID, text of the public profile, and location of the Twitter user; and other Gnip metadata such as the time the tweet was posted. Only the text of the tweets and the user profiles were analyzed in the context of this study. Links to other online material within the tweet text were considered in the context of making coding decisions, but were not specifically analyzed on their own.

The subset of tweets analyzed for this study contained the keywords “Colorado” and “flood” in conjunction with one or more of the keywords “climate change,” “global warming,” or “environment.” While incivility and sarcasm may have been present in the other tweets about the flood not specifically mentioning climate change and the environment, this study was particularly interested in how incivility and sarcasm operated in the context of specific discussions about climate. Therefore, we decided to focus on the 4,094 tweets containing these keywords related to the flood and global warming or the environment, in order to better understand how people used Twitter to specifically talk about climate change concerns in relation to the floods. The entire corpus of tweets regarding anything related to the floods was not analyzed. This allowed us to hand-code the sample, rather than rely on a computer-assisted or algorithmic approach to facilitate to our content analysis of sarcasm, which is a latent variable. The number of tweets analyzed is in line with or significantly more than other studies that have analyzed science-based discussions on Twitter using human-coder approaches (e.g., Dalrymple, Young, & Tully, 2016; Gastrow, 2015). Other analyses that have analyzed samples of tweets in the tens of thousands have relied on computer-assisted coding (Mondragon, Gil de Montes, & Valencia, 2017; Veltri & Atanasova, 2015). There were some instances in which we coded tweets that were retweeted by other users. We did not remove these from the analyses, as they represent further amplification or reach of certain language and ideas (Sutton et al., 2014; Sutton, League, Sellnow, & Sellnow, 2015).

Content Analysis

We developed a codebook using a combination of prior definitions of our concepts and trends and themes that emerged from the data. To gain intercoder reliability using Krippendorff’s Alpha, a number of practice coding sessions took place. After each, intercoder reliability was assessed and categories with unsatisfactory reliability levels were discussed until a satisfactory number was reached. Random sets of tweets from the sample were used for these practice coding sessions. After the final practice coding session using 200 tweets, intercoder reliability for sarcasm was .9773 and incivility was .9830. Intercoder reliability for the political tweets was .9404, for the climate change tweets was .9921, and for the climate change connection tweets was .9411. The complete set of tweets was split between four human coders and analyses were performed on the coded set.

Sarcasm was the most difficult concept for achieving intercoder reliability and occurred after several rounds of initial coding. We began with a conceptualization in the codebook defining sarcasm as a subtle or covert attack and doing so by drawing upon humor through irony, or the idea that the meaning of the language being used is opposite to what is being stated. After the practice rounds, the codebook operationalization was refined to state that the tweet needed to include at least two of the following qualities: contradictory statements to suggest the user has opposing ideas to the remark, heavy punctuation, excessive use of capital letters, quotation marks, laughter expressions in the context of a provocation, presence of extreme adjectives or adverbs together, exaggerated or melodramatic statements, a subtle or covert attack, or humor used within the context of a covert attack or contradictory statement. If the tweet happened to mention the word sarcasm in some form, such as using #sarcastic, it was coded as sarcasm, but instances of this were very low. The tweet also needed to be user-generated content, as opposed to content generated from a media source. Tweets that contained both user-generated content and media-generated content (e.g., a retweet of a headline with additional commentary) could be coded for sarcasm.

Examples of sarcasm include “Colorado though!!! Weather agencies calling it “biblical” flooding . . . oh but there is no global warming??” because of the subtle attack and excessive use of punctuation. The question indicates irony. Another example, “Fires in Colorado, blamed on climate change. Flooding in Colorado, blamed on climate change. Guessing next it will be the snow. #transparency” was coded as sarcastic because of the subtle attack as well as the use of “#transparency” which suggests the user’s beliefs are contradictory to the statement. Key characterizations of sarcasm include taunts or gibes and irony, and verbal irony is often used as an operationalization of sarcasm in discourse analysis (Kunneman et al., 2014). Muresan, Gonzalez-Ibanez, Ghosh, and Wacholder (2016) note the difficulty of identifying sarcasm in text-based mediums, explaining the necessity of relying on lexical features like punctuation and capitalization to identify nonliteral, or ironic, statements. In developing our codebook, we originally also found it difficult to reliably agree on sarcasm when it was operationalized as requiring only one element. Our coding came to rely on the same sorts of lexical markers mentioned by Muresan et al. (2016) to determine the presence of irony. Additionally, requiring that a code for sarcasm contained at least two of the elements listed above helped to ensure that we were capturing irony as well as the dual qualities of sarcasm being a type of taunting combined with irony. Examples that are not sarcasm include, “It’s flooding here in Colorado. Boulder is a mess. I sure wish we’d pay some attention to climate change, sigh.” (while this may contain a subtle attack, it lacks the ironic or opposite-meaning language), and “Colorado floods, proving Global Warming is false. #lies” (this example contains language that means the opposite of what is stated, but does not contain other markers such as a subtle or covert attack or a melodramatic or exaggerated statement).

Other categories proved significantly easier to define, and included incivility, the other key tone in our analyses, and two thematic categories: political mention and climate change’s connection to the Colorado floods. These thematic categories were identified because climate change has developed as a political issue in the minds of the public (McCright & Dunlap, 2011) and is an issue in which its causes have been contested in media discourse (Boykoff & Boykoff, 2004). Past research shows that political issues are popular topics that attract uncivil comments (Coe et al., 2014).

Incivility was operationally defined as an explicit attack against an opposing idea, entity, or person, which often include disparaging remarks or expletives, without the use of irony. An example of incivility from the Twitter dataset would be: “Hello dirt dumb @GOP Climate change is REAL Ask the People of Colorado” because of the disparaging remark against @GOP’s intelligence.

Political mention was operationally defined as a tweet making a specific connection to political actions, figures, parties, activist groups, or calls to action for political participation. Examples include: “Global warming to blame for Colorado flooding??? Is Reid really that stupid?” and “Americans, tell politicians the climate change is real. Colorado Flood. AHEC http://www.ahecEnergy.com has a global clean energy solution. RT.”

A tweet was coded for climate change mention when either climate change or global warming was explicitly stated within the tweet. Coding for this was straightforward and did not require the use of examples. Tweets that were coded 1 for the presence of climate change mention were also coded for connection to the Colorado floods. The tweet could say there was a connection—in which the tweet mentioned climate change being a cause or connected to the floods; that there was no connection—in which the tweet stated that climate change was unrelated or nonexistent; or as uncertain—in which an inquiry into the possible connection or lack thereof was made, but no side was directly stated. Examples of each include, “Thousand-Year Flood’ in Colorado Has Climate Change Written All Over It | Common Dreams https://t.co/hB3aJQTiIQ” (connection). “Were Colorado floods result of global warming? Probably not http://t.co/SlV4aR5RdR” (no connection), and “Colorado Flooding: Did Climate Change Play A Role In Recent Disaster? http://t.co/AtG2dkvNM4” (uncertain).

Results

In total, 138 tweets used uncivil language, and 144 tweets used sarcastic language. These represent 3.5% and 3.4% of the total number of tweets analyzed. The results indicated that these tones were not used at the same time in the discussions of climate change and the floods, with only 15 tweets coded as both sarcastic and uncivil.

First, we analyzed whether tweets that were uncivil and sarcastic were more likely to be associated with political comments than not. We found that among tweets that were uncivil, 87.7% mentioned politics and 12.3% did not mention politics, χ²(1, n = 4,079) = 373.59, p < .001. When tweets were civil,¹ on the other hand, 19.1% mentioned politics and 80.9% did not mention politics. Thus, politics were more likely to be mentioned alongside uncivil tweets. We found that among tweets that were sarcastic, 47.2% of the tweets mentioned politics and 52.8% did not mention politics. When tweets were not sarcastic, 20.4% of them mentioned politics and 79.6% of them did not mention politics, χ²(1, n = 4,079) = 59.32, p < .001. In other words, political comments were most likely to be made without the use of sarcasm. This points to a difference among the uncivil and sarcastic tweets, with incivility associated with more political comments and sarcasm not associated with political comments.

Next, we analyzed how likely tweets were to mention incivility and sarcasm alongside the different connections people made between the flooding and climate change. When tweets were uncivil, 1.4% of them did not mention climate change at all, 17.4% of them claimed the floods were caused by climate change, 76.8% of them claimed the floods were not caused by climate change, and 4.3% of them mentioned climate change in connection to floods but were ambivalent about whether the floods were caused by climate change, χ²(3, n = 4,079) = 590.624, p < .001. Post hoc tests of the adjusted standardized residuals showed that these claims made in the uncivil tweets were significantly different from the claims made in civil tweets. Among civil tweets, 38.5% of them did not mention climate change, 34.7% of them claimed the floods were caused by climate change, 9.4% of them claimed the floods were not caused by climate change, and 17.3% of them made an ambivalent connection. Therefore, tweets that said the floods were not caused by climate change were more likely to be associated with uncivil language.

When tweets were sarcastic, 9.7% did not mention climate change, 16% claimed the floods were caused by climate change, 71.5% of them claimed the floods were not caused by climate change, and 4.6% mentioned an ambivalent connection, χ²(3, n = 4,079) = 518.05, p < .001. Post hoc tests of the adjusted standardized residuals showed that these claims made in the sarcastic tweets were significantly different from the claims made in tweets that were not sarcastic. When the tweets were not sarcastic, 38.2% did not mention climate change, 34.8% claimed the floods caused climate change, 9.5% of them claimed the floods were not caused by climate change, and 2.8% made an ambivalent connection between the floods and climate change. This showed a similar pattern to the uncivil tweets, with tweets claiming that the floods were not caused by climate change more likely to be associated with sarcastic language.

Next, we analyzed whether tweets that are uncivil and sarcastic were more likely to be associated with users who mentioned politics in their profiles than not. When tweets were uncivil, 44.2% of the associated user profiles mentioned politics, and 55.8% of them did not mention politics. This is significantly different from tweets that were civil, in which 16.3% of the associated user profiles mentioned politics in their profiles and 83.7% did not, χ²(1, n = 3,969) = 72.91, p < .001. Thus, tweets that were civil were significantly more likely to be associated with user profiles that did not mention politics. However, tweets that were uncivil were not more likely to be associated with user profiles that mention politics than civil tweets. When tweets were sarcastic, 30.8% of them mentioned politics in the related user profile and 69.2% did not. This is significantly different from tweets that were not sarcastic, with 16.7% of them mentioning politics in the related user profiles and 83.3% of them not mentioning politics, χ²(1, n = 3,969) = 10.06, p < .001. Thus, sarcastic tweeters were less likely to mention politics in their profiles.

Finally, we analyzed the political leaning of user profiles associated with uncivil and sarcastic tweets. We found that among tweets that used uncivil language, 14.8% of the associated user profiles mentioned left-leaning politics, 73.8% of them mentioned right-leaning politics, and 11.5% of them mentioned other political orientations. Post hoc analyses of adjusted standardized residuals comparing uncivil and civil tweets showed that civil tweets were significantly more likely to mention left-leaning politics in the associated user profile (70%) than uncivil tweets were, and significantly less likely to mention right-leaning politics (19.1%) than uncivil tweets were, χ²(2, n = 684) = 95.53, p < .001. That is to say, uncivil tweets were significantly more likely to be associated with user profiles indicating a right-wing political affinity, and civil tweets were significantly more likely to be associated with profiles indicating an affinity with left-wing political views. “Other” political orientations did not differ in civil tweets (10.6%) from uncivil tweets.

Among tweeters who used sarcastic language, 9.1% mentioned left-leaning politics in their profiles, 68.2% mentioned right-leaning politics, and 22.7% mentioned other political orientations. Post hoc analyses of adjusted standardized residuals comparing sarcastic and nonsarcastic tweets showed that nonsarcastic tweets were significantly more likely to mention left-leaning politics (69.2%), less likely to mention right-leaning politics (20.9%), and less likely to mention other political orientations (9.8%), χ²(2, n = 684) = 67.50, p < .001.

Discussion

The goal of this study was to examine how sarcasm and incivility enters social media discussions of scientific issues. Using Twitter-based discussions of climate change during a regional flooding event in Colorado in 2013, we examined how incivility and sarcasm were used alongside the different themes of politics and of whether climate change contributed to the occurrence of the floods. We found that political topics drew out uncivil tones, but not sarcastic tones. Discussions that claimed climate change had nothing to do with the floods drew out both sarcastic and uncivil language. We found that users who claimed to have right-leaning politics in their user profiles also used these tones. As scientific discourse continues to expand in social media spaces, this study adds an important piece to understanding conflict in those discussions. Our findings indicate that controversial aspects of scientific issues, such as politics or skeptical perspectives, draw out more uncivil and sarcastic language. As online media continues to grow as a means for science information consumption, our study has important implications for how social media discussions have the potential to affect how people engage with science.

Before we discuss these findings further, it is important to address limitations of this study. First, analyses of social media typically address the interactive nature of online discussions, and our study analyzed tweet and profile content individually. Analysis of the network structure of Twitter users involved in these discussions of climate change was beyond the scope of this project, although future research should consider undertaking examination of the back-and-forth nature of tweets and the ways in which people are connected relate to their use of different tones. As we will discuss here, it is likely group dynamics are particularly important for understanding the implications of complex tones like sarcasm. Second, our sample was based on tweets that included specific hashtags and keywords that meant content had to specifically mention the word flood in conjunction with other words like Colorado. It is possible that these keywords limited the scope of a larger discussion of the topic using alternative phrases (e.g., weather-related terms). Finally, we found low levels of incivility and sarcasm in our data. This finding is in line with past research that presented evidence that the percentage of newspaper comments about environmental topics that employ uncivil language (17%) is lower than other topics, such as politics, sports, or the economy (all greater than 25%) (Coe et al., 2014). Collins and Nerlich (2014) pointed out that about 3% of user comments they analyzed in their study of newspaper stories on climate change were removed completely by the newspaper’s moderators, indicating that those were comments that might meet criteria of incivility. Therefore, it is not necessarily surprising that incivility and sarcasm are each present in slightly more than 3% of all tweets in the dataset in the present study.

In addition, our choice to use data from a general severe weather event rather than tweets specifically tagged as sarcastic (e.g., the #overlyhonestmethods discussion)² or that were more likely to be uncivil (e.g., in response to a major polarizing event such as politician taking a clear action against climate change) likely also contributed to the low levels of both constructs. However, analyzing climate change discussions among more everyday or generalized weather-based conversation serves an important purpose of understanding how these tones enter general science-related discussions—as many studies analyzing uncivil discourse online have already done—rather than in specialized discussions with the specific purpose of talking sarcastically that might happen among a particular set of individuals highly knowledgeable or interested in science (e.g., graduate students in the #overlyhonestmethods discussion). Nevertheless, interpretation of these results should be approached with caution given their limited reach.

Future research should attempt our approaches to coding sarcasm in other science communication contexts and in those not limited to a particular event to analyze how these conclusions would hold in other data contexts. What themes and subjects draw out uncivil or sarcastic tones in these settings? Using a data collection approach, such as hashtagged content, that already clearly demarcates content as sarcastic would free up researchers to focus content analysis on other latent variables in these discussions. In addition, as more social media analyses move toward a computer sentiment analysis approach that uses machine learning to code large amounts of data based on human instruction (e.g., Su et al., 2016), research should conduct larger scale analyses of complex constructs like sarcasm. This would allow researchers to see how coding of such complex tones holds across large sets of data. Using a human-coded approach among four coders allowed us to manageably code a large set of tweets, but it is possible the coders missed certain instances of either tone in the tweets. As such, our results are on the conservative end of representations of these tones in this particular case.

Future research should also address these questions with newer data, as attention to toxic talk in online spaces has risen dramatically. It is possible that subtle attacks through sarcasm may evolve, given the efforts platforms such as Twitter have taken to reduce bullying.

Our study adds an important systematic approach to assessing the extent of incivility and sarcasm in online discussions of science, an understudied aspect of science communication. Our findings indicate that climate change discussions on Twitter maintain a persistent presence of both tones. We found little evidence, however, that people are using these tones together. It is likely that users find a place for each rhetorical or discursive strategy—sarcasm and incivility—in the social media space. It would be worthwhile to conduct additional analyses in future research that examines if people are using sarcasm in conversations among different group dynamics. For instance, is sarcasm intended to engage like-minded others in a humorous manner, while incivility is intended to provoke negative feelings or shame among non–like-minded others? Or, perhaps sarcasm is used with non–like-minded others so as not to provoke or encourage expressions of the other perspective but instead discourage them from talking more. Ziegele and Jost’s (2016) study shows that exposure to sarcastic responses to incivility can curb participation in online discussions.

Analyzing group dynamics is particularly interesting in light of our findings that sarcastic and uncivil tones were more skeptical about climate change with those tweets more likely to represent the view that the floods were not connected to climate change causes and occurring among politically right-leaning individuals. While our data cannot speak directly to group dynamics, these two groups (climate change skeptics and right-leaning individuals) are among minority opinions on climate change in the United States (Leiserowitz, Maibach, Roser-Renouf, Rosenthal, & Cutler, 2017a, 2017b). The question remains as to whether humorous rhetorical and discursive strategies such as sarcasm are used by these individuals so they feel comfortable speaking out in a hostile opinion environment. Some evidence indicates people are willing to speak out on climate change in online discussion settings even when they hold minority opinions (Porten-Cheé & Eilders, 2015). It is possible they are relying on defensive (incivility) or humorous (sarcastic) tones to easily dismiss non–like-minded others while building rapport among others like them. Further research is needed to understand the motivations for using these types of tones in science-related discussion and how group dynamics play a role in their use. While our study did not code conversational threads, one approach future research might also consider is to code for conversational context in order to better understand how sarcasm is used in conjunction with group dynamics. For instance, previous interactions among a set of commenters or tweeters that include attack-based discussions may inform future discussions—and even future mentions of such attacks—on similar topics.

Our findings also show political mentions draw out uncivil language but not sarcasm. It is possible that mentions during the immediate aftermath of an event like this are primarily centered around responses to the impacts of the event. These responses—if negative—likely draw out simple attacks to a dislike of immediate political responses to the event rather than a more nuanced discussion that would rely on sarcasm. Yet there are certainly sarcastic tweets that touch on politics, such as: “Harry Reid blames global warming for the Colorado floods. I guess there had never been a flood before we started driving car [sic].” Furthermore, weather events are not immune to politically polarizing discourse, and politicians have used current weather conditions as evidence to confirm or disconfirm climate change’s existence (Hardy & Jamieson, 2016). Future research should explore if and how sarcasm is used across political aspects of other events related to climate change and other controversial scientific issues.

Our study identifies the persistent—although limited—use of sarcasm and incivility in social media discussions of climate change, particularly among politically conservative individuals on skeptical aspects of climate change. Given the negative effects of uncivil comments previous research has identified, social media discussions of science have the potential to detract from healthy public engagement with science.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Colorado State University Water Center (Grant/Award Number: 1301072).

Notes

Author Biographies

Ashley A. Anderson (PhD, University of Wisconsin-Madison) is an assistant professor in the Department of Journalism and Media Communication at Colorado State University. Her research interests include public opinion and public engagement around issues of science and emerging technology, particularly in the online communication environment. She completed a postdoctoral research fellowship in the Center for Climate Change Communication at George Mason University.

Heidi E. Huntington (PhD, Colorado State University) is an assistant professor of business communication in the College of Business at West Texas A&M University. Her research interests center on the intersection of social and user-generated media with public opinion and engagement surrounding wedge issues, including political and science communication.

References

Alonzo

Aiken

(2004). Flaming in electronic communication. Decision Support Systems, 36(3), 205-213. doi:10.1016/S0167-9236(02)00190-2

and Then There’s Physics says. (2015, November 2). Tone. Retrieved from https://andthentheresphysics.wordpress.com/2015/11/02/tone/

Anderson

A. A.

Brossard

Scheufele

D. A.

(2010). The changing information environment for nanotechnology: Online audiences and content. Journal of Nanoparticle Research, 12, 1083-1094. doi:10.1007/s11051-010-9860-2

Anderson

A. A.

Brossard

Scheufele

D. A.

Xenos

M. A.

Ladwig

(2014). The “Nasty Effect:” Online incivility and risk perceptions of emerging technologies: Crude comments and concern. Journal of Computer-Mediated Communication, 19, 373-387. doi:10.1111/jcc4.12009

Anderson

A. A.

Yeo

S. K.

Brossard

Scheufele

D. A.

Xenos

M. A.

(2016). Toxic talk: How online incivility can undermine perceptions of media. International Journal of Public Opinion Research, edw022. doi:10.1093/ijpor/edw022

Angouri

Tseliga

(2010). “You have no idea what you are talking about!” From e-disagreement to e-impoliteness in two online fora. Journal of Politeness Research. Language, Behaviour, Culture, 6, 57-82. doi:10.1515/jplr.2010.004

Attardo

(2000). Irony as relevant inappropriateness. Journal of Pragmatics, 32, 793-826. doi:10.1016/S0378-2166(99)00070-3

Auerbach

(2014, April 16). Twitter buys social data media firm Gnip. The Telegraph(U.K.). Retrieved from http://www.telegraph.co.uk/technology/twitter/10770489/Twitter-buys-social-data-firm-Gnip.html

Averbeck

J. M.

Hample

(2008). Ironic message production: How and why we produce ironic messages. Communication Monographs, 75, 396-410. doi:10.1080/03637750802512389

10.

Borah

(2013). Interactions of news frames and incivility in the political blogosphere: Examining perceptual outcomes. Political Communication, 30, 456-473. doi:10.1080/10584609.2012.737426

11.

Borah

(2014). Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, 41, 809-827. doi:10.1177/0093650212449353

12.

Borick

C. P.

Rabe

B. G.

(2014). Weather or not? Examining the impact of meteorological conditions on public opinion regarding global warming. Weather, Climate, and Society, 6, 413-424. doi:10.1175/WCAS-D-13-00042.1

13.

Bowers

J. W.

(1964). Some correlates of language intensity. Quarterly Journal of Speech, 50, 415-420. doi:10.1080/00335636409382688

14.

Boxer

Cortés-Conde

(1997). From bonding to biting: Conversational joking and identity display. Journal of Pragmatics, 27, 275-294. doi:10.1016/S0378-2166(96)00031-8

15.

Boykoff

M. T.

Boykoff

J. M.

(2004). Balance as bias: Global warming and the US prestige press. Global Environmental Change, 14, 125-136. doi:10.1016/j.gloenvcha.2003.10.001

16.

Brooks

D. J.

Geer

J. G.

(2007). Beyond negativity: The effects of incivility on the electorate. American Journal of Political Science, 51, 1-16.

17.

Brossard

(2013). New media landscapes and the science information consumer. Proceedings of the National Academy of Sciences of the USA, 110(Suppl. 3), 14096-14101. doi:10.1073/pnas.1212744110

18.

Capstick

S. B.

Pidgeon

N. F.

(2014). Public perception of cold weather events as evidence for and against climate change. Climatic Change, 122, 695-708. doi:10.1007/s10584-013-1003-1

19.

Cicchirillo

Hmielowski

Hutchens

(2015). The mainstreaming of verbally aggressive online political behaviors. Cyberpsychology, Behavior, and Social Networking, 18, 253-259. doi:10.1089/cyber.2014.0355

20.

Clark

H. H.

Gerrig

R. J.

(1984). On the pretense theory of irony. Journal of Experimental Psychology General, 113, 121-126.

21.

Clark

R. P.

(2007). The frames of incivility. Poynter. Retrieved from https://www.poynter.org/news/frames-incivility

22.

Coe

Kenski

Rains

S. A.

(2014). Online and uncivil? Patterns and determinants of incivility in newspaper website comments: Incivility in newspaper website comments. Journal of Communication, 64, 658-679. doi:10.1111/jcom.12104

23.

Collins

Nerlich

(2014). Examining user comments for deliberative democracy: A corpus-driven analysis of the climate change debate online. Environmental Communication, 9, 189-207. doi:10.1080/17524032.2014.981560

24.

Dalrymple

K. E.

Young

Tully

(2016). “Facts, not fear”: Negotiating uncertainty on social media during the 2014 Ebola crisis. Science Communication, 38, 442-467. doi:10.1177/1075547016655546

25.

De Kraker

Kuijs

Cörvers

Offermans

. (2014). Internet public opinion on climate change: A world views analysis of online reader comments. International Journal of Climate Change Strategies and Management, 6, 19-33. doi:10.1108/IJCCSM-09-2013-0109

26.

Egan

P. J.

Mullin

(2012). Turning personal experience into political attitudes: The effect of local weather on Americans’ perceptions about global warming. Journal of Politics, 74, 796-809. doi:10.1017/S0022381612000448

27.

Gastrow

(2015). Science and the social media in an African context: The case of the square kilometre array telescope. Science Communication, 37, 703-722.

28.

Gervais

B. T.

(2015). Incivility online: Affective and behavioral reactions to uncivil political posts in a web-based experiment. Journal of Information Technology & Politics, 12, 167-185. doi:10.1080/19331681.2014.997416

29.

Gervais

B. T.

(2017). More than mimicry? The role of anger in uncivil reactions to elite political incivility. International Journal of Public Opinion Research, 29, 384-405. doi:10.1093/ijpor/edw010

30.

Gnip. (n.d.). Complete Twitter data access. Retrieved from https://gnip.com/sources/twitter/

31.

Graham

Wright

(2015). A tale of two stories from “Below the Line” comment fields at the Guardian. International Journal of Press/Politics, 20, 317-338. doi:10.1177/1940161215581926

32.

Hardy

B. W.

Jamieson

K. H.

(2016). Overcoming endpoint bias in climate change communication: The case of Arctic sea ice trends. Environmental Communication, 11, 205-217. doi:10.1080/17524032.2016.1241814

33.

Hwang

Kim

Huh

C. U.

(2014). Seeing is believing: Effects of uncivil online debate on political polarization and expectations of deliberation. Journal of Broadcasting & Electronic Media, 58, 621-633. doi:10.1080/08838151.2014.966365

34.

Jaspal

Nerlich

Koteyko

(2012). Contesting science by appealing to its norms: Readers discuss climate science in the Daily Mail. Science Communication, 35, 383-410. doi:10.1177/1075547012459274

35.

Joireman

Barnes Truelove

Duell

(2010). Effect of outdoor temperature, heat primes and anchoring on belief in global warming. Journal of Environmental Psychology, 30, 358-367. doi:10.1016/j.jenvp.2010.03.004

36.

Jorgensen

(1996). The functions of sarcastic irony in speech. Journal of Pragmatics, 26, 613-634. doi:10.1016/0378-2166(95)00067-4

37.

Kahan

(2012). Why we are poles apart on climate change. Nature, 488, 255-255.

38.

Kaiser

(2017). Public spheres of skepticism: Climate skeptics’ online comments in the German networked public sphere. International Journal of Communication, 11, 22.

39.

Kata

(2012). Anti-vaccine activists, Web 2.0, and the postmodern paradigm—An overview of tactics and tropes used online by the anti-vaccination movement. Vaccine, 30, 3778-3789. doi:10.1016/j.vaccine.2011.11.112

40.

Koteyko

Jaspal

Nerlich

(2013). Climate change and “climategate” in online reader comments: A mixed methods study. Geographical Journal, 179, 74-86. doi:10.1111/j.1475-4959.2012.00479.x

41.

Kunneman

Liebrecht

van Mulken

van den Bosch

(2014). Signaling sarcasm: From hyperbole to hashtag. Information Processing & Management, 51, 500-509. doi:10.1016/j.ipm.2014.07.006

42.

Lazo

J. K.

Morss

R. E.

Demuth

J. L.

(2009). 300 billion served: Sources, perceptions, uses, and values of weather forecasts. Bulletin of the American Meteorological Society, 90, 785-798.

43.

Lee

Pang

(2014, September 2-5). Responding to the haze: Information cues and incivility in the online small world. In Proceedings of ISIC, the Information Behaviour Conference. Leeds, England. Retrieved from http://www.informationr.net/ir/19-4/isic/isic04.html#.Vv73kGQrK2w

44.

Leiserowitz

Feinberg

Howe

Rosenthal

(2013). Climate change in the American mind: A focus on California, Colorado, Ohio, and Texas. New Haven, CT: Yale Project on Climate Change Communication.

45.

Leiserowitz

Maibach

Roser-Renouf

Feinberg

Howe

(2013). Extreme weather and climate change in the American mind: April 2013. New Haven, CT: Yale Project on Climate Change Communication.

46.

Leiserowitz

Maibach

Roser-Renouf

Rosenthal

Cutler

(2017a). Climate change in the American mind: May 2017. New Haven, CT.: Yale Program on Climate Change Communication.

47.

Leiserowitz

Maibach

Roser-Renouf

Rosenthal

Cutler

(2017b). Politics & global warming: May 2017. New Haven, CT: Yale Program on Climate Change Communication.

48.

Maibach

Leiserowitz

Rosenthal

Cutler

(2016). Is there a climate “spiral of silence” in America? New Haven, CT: Yale University and George Mason University. Retrieved from http://climatecommunication.yale.edu/publications/climate-spiral-silence-america/

49.

Maibach

Placky

B. W.

Witte

Seitter

Gardiner

Myers

. . .Cullen

(2016). TV meteorologists as local climate change educators. In Oxford research encyclopedia of climate science. New York, NY: Oxford University Press.

50.

McCright

A. M.

Dunlap

R. E.

(2011). The politicization of climate change and polarization in the American public’s view of global warming, 2001-2010. Sociological Quarterly, 52, 155-321. doi:10.1111/j.1533-8525.2011.01198.x

51.

Mondragon

N. I.

Gil

Montes

Valencia

(2017). Ebola in the public sphere. Science Communication, 39(1), 101-124. doi:10.1177/1075547016688908

52.

Muresan

Gonzalez-Ibanez

Ghosh

Wacholder

(2016). Identification of nonliteral language in social media: A case study on sarcasm. Journal of the Association for Information Science and Technology, 67, 2725-2737. doi:10.1002/asi.23624

53.

O’Neill

Williams

H. T. P.

Kurz

Wiersma

Boykoff

(2015). Dominant frames in legacy and social media coverage of the IPCC Fifth Assessment Report. Nature Climate Change, 5, 380-385. doi:10.1038/nclimate2535

54.

Papacharissi

(2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6, 259-283. doi:10.1177/1461444804041444

55.

Phillips

(2011). Meet the trolls. Index on Censorship, 40, 68-76. doi:10.1177/0306422011409641

56.

Porten-Cheé

Eilders

(2015). Spiral of silence online: How online communication affects opinion climate perception and opinion expression regarding the climate change debate. Studies in Communication Sciences, 15, 143-150. doi:10.1016/j.scoms.2015.03.002

57.

Porter

A. J.

Hellsten

(2014). Investigating participatory dynamics through social media using a multideterminant “frame” approach: The case of climategate on YouTube. Journal of Computer-Mediated Communication, 19, 1024-1041. doi:10.1111/jcc4.12065

58.

Purcell

Rainie

Mitchell

Rosenstiel

Olmstead

(2010). Understanding the participatory news consumer. Pew Internet and American Life Project. Retrieved from http://www.pewinternet.org/2010/03/01/understanding-the-participatory-news-consumer/

59.

Riloff

Qadir

Surve

De Silva

Gilbert

Huang

(2013, October 18-21). Sarcasm as contrast between a positive sentiment and negative situation. In EMNLP (pp. 704-714). Seattle, WA. Retrieved from http://www.cs.utah.edu/~alnds/papers/sarcasm_2013.pdf

60.

Rowe

(2015). Civility 2.0: A comparative analysis of incivility in online political discussion. Information, Communication & Society, 18, 121-138. doi:10.1080/1369118X.2014.940365

61.

Shao

(2015). Are actual weather and perceived weather the same? Understanding perceptions of local weather and their effects on risk perceptions of global warming. Journal of Risk Research, 19, 722-742. doi:10.1080/13669877.2014.1003956

62.

Smith

Hennen

(2013, September 20). Record rain, steep canyons fueled Colorado floods. Retrieved from http://www.cnn.com/2013/09/18/us/colorado-flooding/index.html

63.

Spence

Poortinga

Butler

Pidgeon

N. F.

(2011). Perceptions of climate change and willingness to save energy related to flood experience. Nature Climate Change, 1, 46-49. doi:10.1038/nclimate1059

64.

L. Y.-F.

Akin

Brossard

Scheufele

D. A.

Xenos

M. A.

(2015). Science news consumption patterns and their implications for public understanding of science. Journalism & Mass Communication Quarterly, 92, 597-616. doi:10.1177/1077699015586415

65.

L. Y.-F.

Cacciatore

Liang

Brossard

Scheufele

Xenos

M. A.

(2016). Analyzing public sentiments online: Combining human- and computer-based content analysis. Information, Communication & Society, 20, 406-427. doi:10.1080/1369118X.2016.1182197

66.

Sutton

League

Sellnow

T. L.

Sellnow

D. D.

(2015). Terse messaging and public health in the midst of natural disasters: The case of the boulder floods. Health Communication, 30, 135-143. doi:10.1080/10410236.2014.974124

67.

Sutton

Spiro

E. S.

Johnson

Fitzhugh

Gibson

Butts

C. T.

(2014). Warning tweets: Serial transmission of messages during the warning phase of a disaster event. Information, Communication & Society, 17, 765-787. doi:10.1080/1369118X.2013.862561

68.

Uldam

Askanius

(2013). Online civic cultures: Debating climate change activism on YouTube. International Journal of Communication, 7, 1185-1204.

69.

Veltri

G. A.

Atanasova

(2015). Climate change on Twitter: Content, media ecology and information sharing behaviour. Public Understanding of Science, 26, 721-737. doi:10.1177/0963662515613702

70.

Whalen

J. M.

Pexman

P. M.

Gill

A. J.

(2009). “Should be fun—not!” Incidence and marking of nonliteral language in e-mail. Journal of Language and Social Psychology, 28(3), 263-280. doi:10.1177/0261927X09335253

71.

Whitmarsh

(2011). Scepticism and uncertainty about climate change: Dimensions, determinants and change over time. Global Environmental Change, 21, 690-700. doi:10.1016/j.gloenvcha.2011.01.016

72.

Williams

H. T. P.

McMurray

J. R.

Kurz

Hugo Lambert

(2015). Network analysis reveals open forums and echo chambers in social media discussions of climate change. Global Environmental Change, 32, 126-138. doi:10.1016/j.gloenvcha.2015.03.006

73.

Zappavigna

(2012). Discourse of Twitter and social media: How we use language to create affiliation on the Web (1st ed.). London, England: Bloomsbury Academic.

74.

Zhao

Maibach

Gandy

Witte

Cullen

Klinger

. . .Pyle

(2013). Climate change education through TV weathercasts: Results of a field experiment. Bulletin of the American Meteorological Society. 95, 117-130. doi:10.1175/BAMS-D-12-00144.1

75.

Ziegele

Jost

P. B.

(2016). Not funny? The effects of factual versus sarcastic journalistic responses to uncivil user comments. Communication Research. Advance online publication. doi:10.1177/0093650216671854