Not Funny? The Effects of Factual Versus Sarcastic Journalistic Responses to Uncivil User Comments

Abstract

Incivility in user comments on news websites has been discussed as a significant problem of online participation. Previous research suggests that news outlets should tackle this problem by interactively moderating uncivil postings and asking their authors to discuss more civilized. We argue that this kind of interactive comment moderation as well as different response styles to uncivil comments (i.e., factual vs. sarcastic) differently affect observers’ evaluations of the discussion atmosphere, the credibility of the news outlet, the quality of its stories, and ultimately observers’ willingness to participate in the discussions. Results from an online experiment show that factual responses to uncivil comments indirectly increase participation rates by suggesting a deliberative discussion atmosphere. In contrast, sarcastic responses indirectly deteriorate participation rates due to a decrease in the credibility of the news outlet and the quality of its stories. Sarcastic responses however increase the entertainment value of the discussions.

Keywords

news websites interactivity moderation community management participation user comments incivility

In today’s online media landscape, news consumers can use the comment functions on news websites and on the sites of news outlets on social network platforms such as Facebook to learn about the issue-related attitudes of other users and to voice their own opinion toward news topics or other user comments (Rowe, 2015; Springer, Engelmann, & Pfaffinger, 2015). Still, however, many users refrain from participating in public news discussions; in most Western industrialized countries, only between 8% and 19% of the online users write comments at least once a week (Newman, Levy, & Nielsen, 2015). Both from a participatory and an economic point of view, it can be considered as desirable to increase these rates to raise news consumer loyalty and support opinion diversity in the public spheres on news websites (Ruiz et al., 2011; Vujnovic, 2011).

For many users, a significant inhibitor of online participation is the low quality of the comments other users have posted (Springer et al., 2015; Stroud, van Duyn, & Peacock, 2016). Indeed, contrary to the ideal of deliberative discussions (Dahlberg, 2001), user discussions on news websites are often not constructive, results oriented, and polite, but contain a high amount of incivility instead (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014; Coe, Kenski, & Rains, 2014). Although incivility in public discourse is not unanimously considered as something negative, research has reported various deleterious effects of uncivil user comments on readers and media outlets (see the following section).

The low quality of user comments has prompted some news outlets to prohibit comments on sensitive topics or to abandon comment sections altogether (Reich, 2011; Thurman, Cornia, & Kunert, 2016). Other organizations have employed a pragmatic approach toward comment moderation and filter or delete uncivil comments before or after their publication (Domingo et al., 2008; Ruiz et al., 2011). Yet, these strategies have been criticized for inadequately limiting the Internet’s potential for citizen participation and free speech (Janssen & Kies, 2005; Wright, 2006). In addition, an increasing number of news outlets distribute their stories on Facebook where user comments cannot be fully prohibited or filtered before publication (Rowe, 2015; Ziegele, Breiner, & Quiring, 2014).

As an alternative strategy, researchers have recommended news outlets to actively foster the development of civil norms and behaviors in comment sections by publicly interacting with commenters (Marchionni, 2015; Meyer & Carey, 2014; Stroud, Scacco, Muddiman, & Curry, 2015). For example, journalists can answer questions of their users or appreciate thoughtful contributions. Uncivil commenters, in contrast, can be prompted to discuss more respectfully. Such an interactive approach to comment moderation could improve the discussion atmosphere in comment sections without overly limiting free speech.

Although relatively few news outlets have applied such an interactive moderation strategy to date,¹ studies show that many of their users would appreciate an increase of journalists’ involvement in comment sections (Bergström & Wadbring, 2014; Diakopoulos & Naaman, 2011; Stroud et al., 2016). Yet, although some journalists and editors hold a positive attitude toward this kind of interactive engagement, there is also significant concern whether standards of journalistic impartiality could be maintained if the newsroom participated in the discussions (Diakopoulos & Naaman, 2011; Meyer & Carey, 2014).

Past research on interactive comment moderation has focused on the effects of a “helpful” moderator in predominantly civil news discussions and has shown that these moderators can help further improving the quality of subsequent users’ contributions (Stroud et al., 2015). To date, however, we do not know of any research that has tested whether an interactive moderation of uncivil comments can counteract the potentially undesirable effects of incivility. Our study will contribute to this field by theorizing how different journalistic responses to uncivil user comments (i.e., factual vs. sarcastic responses) affect observers’ perceptions of a deliberative discussion atmosphere, the entertainment value of the discussions, the credibility of the news outlet as well as the quality of its stories, and ultimately, observers’ willingness to participate. We will also investigate whether the effects of interactive moderation of uncivil comments depend on the “weight” of the news topic (i.e., high involving vs. low involving). Thereby, our research can contribute to a better understanding of the profits and perils of a new participatory comment moderation strategy, and can shed light on what happens when journalists engage in publicly visible interpersonal communication with their users.

Incivility in Public Discourse

Incivility is a widely spread issue in public debates. Some people have already proclaimed an “incivility crisis” (Carter, 1998, p. 10) because extreme and confrontational positions would dominate public discourse and interpersonal exchanges. Others have argued that civility “is just a genteel way to mask the inevitable tensions and antagonisms of democratic society” (Kennedy, 1998, para. 1). From this point of view, the call for civility is only an excuse for banning arguable and diverse opinions from public discourse. A third opinion is that controversy and conflict alone do not constitute incivility but that a dismissive, disrespectful, aggressive, and hostile tone makes a statement uncivil (Coe et al., 2014; Hwang, Kim, & Kim, 2016). From this angle, incivility means “expression of disagreement by denying and disrespecting the justice of the opposing views” (Hwang et al., 2016, p. 5). It encompasses rhetorical and stylistic elements such as verbal intimidation, ad hominem attacks, overgeneralizations, vulgar or disrespectful speech, and exaggerated claims (Massaro & Stryker, 2012; Sobieraj & Berry, 2011). Using these elements can violate interpersonal conversation norms and undermine the ideal of deliberative discussions, which require a civil and rational exchange of positions (Anderson et al., 2014; Papacharissi, 2004).

Analyzing online media, political talk radio, and cable news, Sobieraj and Berry (2011) found that incivility has increased significantly in the past decades and occurs extensively in professional media. In the context of digitalization, media consumers can be the source of public incivility as well. Coe and colleagues (2014) analyzed the comment section of a newspaper website, and found that more than every fifth comment contained some form of incivility, with name-calling being the most prominent form.

Research has documented various effects of incivility. On the “positive” side, the use of incivility can arouse one’s attention and increase the recall of oppositional opinions (Hwang et al., 2016). Other studies, however, mostly describe “negative” effects of incivility. Mutz and Reeves (2005) found that televised incivility reduced media consumers’ political trust. Various studies report an increase in people’s averse and hostile feelings after being exposed to incivility, such as anger and hatred (Hwang et al., 2016; Phillips & Smith, 2004). Within the online domain, many news site visitors assert that incivility limits their willingness to participate in online discussions (Springer et al., 2015; Stroud et al., 2016). Moreover, Anderson and colleagues (2014) found in an experimental study that uncivil comments led to greater perceived risks of nanotechnology among people who were already skeptical toward this technology. Ultimately, uncivil comments have been found to reinforce individuals’ prejudicial attitudes and behaviors (Hsueh, Yogeeswaran, & Malinen, 2015) and adversely affect quality perceptions of journalistic content (Prochazka, Weber, & Schweiger, 2016).

Based on this general assessment, the aim of our study is not to limit incivility in online discussions by eliminating disagreement or uncomfortable opinions from public debates but rather to foster a discussion atmosphere where opinions can be exchanged in a respectful way. In the following sections, we will analyze whether an interactive moderation of uncivil comments can help to reach this aim.

A Framework for Analyzing the Effects of Interactive Moderation on Participation

Past research has analyzed how incivility affects media consumers’ trust in news media and in politics as well as their overall evaluations of discussions, their attitudes toward social issues, and their participatory behavior. Therefore, we will analyze the effects of interactive moderation of uncivil comments within this framework. Our argument will be guided by the following model that considers observers’ participation willingness as the outcome and their perceptions of a deliberative or entertaining discussion atmosphere as well as their perceptions of the credibility of the news outlet and the quality of its stories as antecedents of participation. This conceptual model and the hypothesized effects are displayed in Figure 1. Before explaining the hypotheses and research questions, we will first provide a general assessment of moderation as a form of discussion management.

Figure 1.

Conceptual model.

Comment Moderation on News Websites

Given the possible detrimental effects of incivility, the strategies that news outlets employ to handle uncivil user comments have increasingly attracted scholarly attention. Essentially, three strategies can be observed in practice: some news outlets have limited the possibility to write comments to “harmless” topics (Reich, 2011), some have shut down comment sections (Thurman et al., 2016), and others moderate user comments (Stroud et al., 2015).

Moderation can be defined as any kind of institutional engagement aimed at the organization or regulation of the processes or contents of online discussions. Numerous research strands argue that moderation is a crucial element of successful online discussion cultures of almost every kind (Kraut, Resnick, & Kiesler, 2011; Wright, 2006). There are at least three basic forms of moderation, namely, collaborative moderation, content moderation, and interactive moderation. Collaborative moderation provides features that allow users themselves to rate the quality of comments by “liking” them or by “flagging” them as inappropriate (Diakopoulos & Naaman, 2011). A comment with enough flags can then be automatically removed from the site or sent to a moderator for review. Content moderation occurs when moderators delete comments, which they perceive as inappropriate without giving reasons; or, they act as gatekeepers and filter inappropriate comments before publication (Wright, 2006).

Interactive moderators, in contrast, participate actively and visibly in the discussions by performing a broad range of interactions with the comments of their users (Kraut et al., 2011; Stroud et al., 2015; Wright, 2006). Broadly spoken, these moderators either act in a “helpful” or a “regulative” role. In a helpful role, interactive moderators provide additional information, clarify questions, or compliment thoughtful contributions (Stroud et al., 2015; Wright, 2006). In a regulative role, they mediate conflicts, rebuke discussants who violate the predefined rules, or keep discussions on topic (Wright, 2006).

Interactive moderation on news websites can be seen as a part of the concept of journalism-as-conversation (Marchionni, 2015) or reciprocal journalism (Holton, Coddington, Lewis, & de Zuniga, 2015). Both concepts emphasize the importance of a “greater inclusion of and interaction with publics across digital media platforms” (Holton et al., 2015, p. 2527). Basically, the activities of interactive comment moderators can also be compared with the duties of moderators in political talk shows who too interactively organize and supervise discussion spaces and act as “promoters of high-quality information exchange” (Vraga et al., 2012, p. 8). This analogy is important because like their “offline” colleagues, interactive comment moderators can respond to comments using different language styles, which can be assumed to affect how observers perceive and evaluate the discussions. Vraga and colleagues (2012) distinguished three hosting styles: a “host as correspondent” moderates discussions in a dispassionate, fair, polite, and neutral manner. A “host as comic” interacts with guests in a humorous and/or satiric way and makes jokes and quips on the guests’ expenses. A “host as combatant” employs an aggressive and combative moderation style to engage the guests’ positions and to elicit more information.

Interactive comment moderators could potentially apply each of the three strategies. For the purpose of this study, we will focus on a “regulative” interactive moderator who responds to uncivil comments using either a factual, correspondent style or a more satiric and sarcastic style. This kind of moderation and the two moderation styles are chosen for two reasons: First, research on interactive moderation has investigated how “helpful” moderators contribute to improving the quality of online discussions (Stroud et al., 2015) but the effects of a “regulative” moderator have remained unexplored.

Second, with regard to moderation styles, it makes sense to contrast styles that aim at keeping users from writing uncivil comments but that differ in the way they address these users. The factual and the sarcastic moderation styles are suitable contrasts in this context. On the one hand, research on community management and online deliberation often recommends moderators to mediate conflicts in a neutral manner and to criticize inappropriate user behavior openly, yet politely (e.g., Kraut et al., 2011; Wright, 2006). This factual moderation style can be compared with the previously described “host as correspondent” approach (Vraga et al., 2012). On the other hand, various studies have reported a rise of satirical information and entertainment media programs in the past decade (Rill & Cardiel, 2013; Vraga et al., 2012). Satire can be defined as an artistic form “in which human or individual vices, follies, abuses, or shortcomings are held up to censure” (Elliott, 2014, para. 1). It uses irony, ridicule, and other methods to reveal social grievances or to point out the faults of a subject (Kreuz & Roberts, 1993). Sarcasm has also been considered as a rhetorical device of satire that, as incongruous communication, is linked to humor (Fine & Martin, 1990). But while satire typically comments on society, sarcasm aims at unmasking a particular subject’s inappropriate argument or behavior (Kreuz & Roberts, 1993). Given the popularity of satiric programs, interactive moderators, thus, might consider using satire, irony, and sarcasm to rebuke the authors of uncivil comments. For example, they could playfully and excessively parody uncivil commenters’ behaviors in order to hold them up a mirror and reveal the inappropriateness of their behavior. Such a moderation strategy might be considered as a blend of the “host as comic” and “host as combatant” approaches (Vraga et al., 2012). In fact, using sarcasm and satire has already been found to be successful in keeping so-called “trolls” away from online forums (Coles & West, 2016).

However, the use of satire and sarcasm aims at exposing target subjects to ridicule. It could, therefore, be argued that satiric and sarcastic responses to uncivil comments are subtle and humorously exaggerated varieties of incivility themselves (Anderson, Huntington, & Kandra, 2015). In fact, individuals sometimes interpret satire and especially sarcasm as funny yet offensive and impolite ways of criticizing subjects (Kuipers, 2015; Toplak & Katz, 2000). Still, there is a (thin) conceptual line between satire/sarcasm and traditional understandings of incivility: Uncivil statements are serious, hostile, and disrespectful statements that threaten social norms and collective traditions of democracy (Papacharissi, 2004), whereas a main purpose of satire and satiric sarcasm is to reveal violations of these norms and collective traditions in a playful and humorous way (Stroud & Muddiman, 2013). Nevertheless, moderators trying to fight incivility with satire and sarcasm might be perceived by their audiences as acting in an uncivil manner themselves. Put differently, uncivil and satirical/sarcastic communicators pursue different goals, but observers may perceive both kinds of statements as more or less uncivil.

In sum, the current study will investigate the effects of two distinct interactive moderation styles that both aim at keeping users from writing uncivil comments. Factual moderation criticizes inappropriate behaviors directly, yet politely, while sarcastic moderation employs satirical devices to unmask inappropriate behaviors in a more humorous and playful way. Both moderation styles can be assumed to demonstrate that a news outlet takes action against uncivil comments, yet beyond that, observers of the discussions might evaluate them differently. The different potential effects of the two moderation styles will be disentangled in the following section.

Interactive Moderation of Uncivil Comments, Perceived Discussion Atmosphere, and Willingness to Participate

Incivility in user comments is regularly portrayed as a threat to a deliberative discussion atmosphere. According to diverse scholars, such an atmosphere is characterized by mutual respect, alertness for others’ perspectives, reflexivity, and justification of validity claims through the use of arguments, for example (Dahlberg, 2001). In fact, research has found that users perceive unmoderated discussions with uncivil comments as hostile and their authors as dominant and unfriendly (Ng & Detenber, 2005). Unmoderated uncivil comments could, hence, suggest observers of the discussions that rudeness and disrespect are acceptable norms within the participatory space of a news website (Stroud et al., 2015). Moderators of public online discussions, in contrast, are often seen as facilitators and preservers of civil and high-quality debates (Ruiz et al., 2011; Stroud et al., 2015; Wright, 2006). Using an online experiment, Wise, Hamman, and Thorson (2006) showed that participants indicated a higher willingness to participate in a “moderated” community than in an “unmoderated” one. The authors explain this finding by suggesting that users assumed moderated communities not to go “feral due to a lack of moderation” (p. 30). Similarly, Stroud and colleagues (2015) have shown that interactive moderators in particular can increase the salience of deliberative norms by performing exemplary deliberative interactions, which users then adopt. Although these assumptions mostly concern a “helpful” moderator, they can be transferred to the “regulative” interactive moderation of uncivil comments as well: By prompting the authors of uncivil comments to behave more respectfully, moderators publicly demonstrate that incivility is a norm violation and that they will ensure civilized discussions. Thereby, compared with unmoderated discussions, an interactive moderation of uncivil comments should increase observers’ perceptions of a deliberative discussion atmosphere.

However, this effect can be assumed to occur only when moderators respond to uncivil commenters in a factual and respectful style. Although satiric and sarcastic responses to uncivil comments can also unmask the inappropriateness of uncivil behavior, these messages are likely to be perceived as offensive, impolite, or even uncivil themselves (Kuipers, 2015; Toplak & Katz, 2000). In a similar vein, viewers of political TV shows perceived moderators who made jokes at their guests’ expenses as aggressive (Vraga et al., 2012). Therefore, compared with unmoderated discussions, satiric or sarcastic responses to uncivil comments could even deteriorate observers’ perceptions of a deliberative discussion atmosphere further. Hence, we hypothesize the following:

Hypothesis 1a (H1a): Factual moderation of uncivil comments will increase observers’ perceptions of a deliberative discussion atmosphere.

Hypothesis 1b (H1b): Sarcastic moderation of uncivil comments will deteriorate observers’ perceptions of a deliberative discussion atmosphere.

Regarding the link between perceptions of a deliberative discussion atmosphere and participation, various survey studies report that perceptions of a hostile, uncivil, and generally uncomfortable discussion atmosphere in the comment sections limit users’ willingness to contribute comments (Diakopoulos & Naaman, 2011; Stroud et al., 2016; Wise et al., 2006). One possible explanation is that these users fear that their postings could be disapproved by the authors of uncivil comments and that their self-worth might be damaged by such disapproval. In terms of politeness research, writing a comment in an uncivil environment can be a “face-threatening” activity (Brown & Levinson, 1987), which users anticipate and which lets them decide not to write a comment. In contrast, when discussions appear sociable and respectful, this fear should be unnecessary. Hence, we hypothesize the following:

Hypothesis 1c (H1c): The more observers perceive a deliberative discussion atmosphere, the higher will be their willingness to participate.

In sum then, it can be assumed that factual responses to uncivil comments should indirectly increase the willingness to participate via the perceived deliberativeness of the discussion atmosphere. In contrast, an inverse effect is assumed for sarcastic responses.

Media consumers also assess the quality of discussions on a second dimension, namely, the perceived entertainment value. In this context, experimental studies found a positive effect of incivility in political TV programs on the entertainment value of the program (Mutz & Reeves, 2005), while political talk shows anchored by a factual “host as correspondent” received a comparatively low entertainment score (Vraga et al., 2012). For user comments, Springer and colleagues (2015) showed that some users enjoy the sense of conflict in the discussions. Other studies confirm that both users who read and write comments seek affective gratifications in addition to cognitive gratifications (Diakopoulos & Naaman, 2011; Stroud et al., 2016). Moderators who respond to uncivil comments in a factual manner and request a more civil behavior from them might, therefore, be perceived as “killjoys” and this perception could impair the perceived entertainment value of the discussions. Satire and sarcasm, in contrast, are inherently linked to humor and entertainment (Rill & Cardiel, 2013). Therefore, observers should perceive discussions in which moderators respond to uncivil comments with satire or sarcasm as even more entertaining than unmoderated discussions. Hence, we hypothesize the following:

Hypothesis 2a (H2a): Factual moderation of uncivil comments will impair observers’ evaluations of the entertainment value of the discussions.

Hypothesis 2b (H2b): Sarcastic moderation of uncivil comments will increase observers’ evaluations of the entertainment value of the discussions.

Regarding the relation between entertainment and active participation, the consumption of satire has been linked to an increase in a measure of political participation that also included comment writing (Hoffman & Young, 2011). Yet, the authors ascribed this effect to a satire-induced rise of individuals’ perceived internal efficacy rather than to the entertainment value of the satire (Hoffman & Young, 2011). Still, in other experiments, the message-induced arousal of amusement significantly increased participants’ willingness to engage in another form of communication, namely, sharing the message with others (Berger, 2011). However, the mere feeling of being entertained has also been described as a state of consumerism that could impede active participation (Besley, 2006). Due to these inconsistent findings, we ask the following research question:

Research Question 1 (RQ1): How does the perceived entertainment value of discussions affect observers’ willingness to participate?

Interactive Moderation of Uncivil Comments, Media Credibility, News Quality, and Willingness to Participate

Journalists and editors have repeatedly expressed concern that uncivil user comments could damage core components of their news brand and negatively influence readers’ attitudes toward their news stories (e.g., Diakopoulos & Naaman, 2011; Reich, 2011). Studies in this domain have however produced mixed results. For example, researchers have provided empirical evidence that uncivil comments strengthen the perceived credibility of a related news story instead of weakening it (Thorson, Vraga, & Ekdale, 2010). The authors explain their findings with social judgment theory, arguing that under the condition of an extreme discrepancy between the tone of the news story and the related comments, the comments might be perceived as considerably less credible than the story. Then, a contrast effect in observers’ judgment behavior can occur, making “the news story look more credible by comparison” (Thorson et al., 2010, p. 304). Other studies, however, have argued that readers of comment-accompanied news stories use uncivil comments as easily accessible cues when judging the quality (informational quality and formal quality) of a related news story (Prochazka et al., 2016). The uncivil tone in user comments could then “spill over” into observers’ news-related quality judgments. Results from an experiment support these assumptions, showing that uncivil and unreasoned comments, in particular, negatively affect participants’ assessment of the quality of the related news story (Prochazka et al., 2016).

Despite a lack of agreement regarding the direction of the effects, uncivil comments seem to affect the perceived quality of associated news stories, as indicated by their adherence to core criteria of journalistic work (e.g., impartiality, relevance, fairness, believability, professionality). However, research has not yet examined whether interactive journalistic moderation of uncivil comments can alter these effects.

In theory, news outlets responding to an uncivil comment can emphasize the norm-violating behavior of the comment author and the low quality of the users’ arguments compared with the professional journalistic work. For observers of the discussions, such an emphasis could increase the salience of the difference between the user comment and the news story, thereby also making a news-quality-boosting contrast effect more likely to occur (Thorson et al., 2010). An alternative explanation builds on the concept of “social presence” according to which individuals evaluate a mediated communication situation in terms of the degree of interpersonal awareness of the communication partner(s) (Marchionni, 2015; Short, Williams, & Christie, 1976). Through interactive moderation, journalists could particularly increase users’ perception of copresence, that is, “the feeling of existing with another person” (Marchionni, 2013, p. 139). Thereby, the somewhat impersonal news organization becomes more “human” in the perception of its users. In a first step, this perception could increase the credibility of the news outlet as a whole: Newhagen and Nass (1989) have suggested that the “inability to establish reporters or editors as distinct individuals . . . works against credibility ratings for newspapers” (p. 279). This suggests that users’ credibility judgments toward a news outlet could be more favorable when it performs “human” interactions with its readers. As media consumers tend to judge the quality of a particular message of a media outlet on the basis of their general brand evaluations (Urban & Schweiger, 2014), interactive comment moderation could also indirectly increase quality perceptions of news stories via an increase of the credibility of the news outlet as a whole.

In an e-Commerce context, various studies have provided empirical evidence that perceptions of social presence can indeed increase the credibility of an organization (e.g., Hassanein & Head, 2007). However, news media research also found negative effects of social presence on the credibility of news media and news stories (Marchionni, 2015). Regarding interactive comment moderation, we again assume that the style of interactive moderation plays a deciding role whether positive or negative effects will occur. Regarding a sarcastic moderation style, researchers have noted that mediated (political) satire is particularly expected to expose powerful individuals or groups (Baym, 1995; Kuipers, 2015). News outlets using satire and sarcasm to unmask the behavior of individuals without authority could violate readers’ normative expectations toward professional journalism that values impartiality and fairness, for example (e.g., Urban & Schweiger, 2014). In other words, users might perceive a sarcastic moderation style as inappropriate and unfair, and this perception could be used as a cue when judging the credibility of the news outlet and the quality of its stories. Supporting these assumptions, Vraga and colleagues (2012) found that a “host as comic” moderation style did not only impair participants’ perceptions of the host credibility but also their perceptions of the credibility of the program as a whole. As we have previously assumed that individuals’ judgments of general media credibility also affect their judgment of the quality of specific news stories, an additional indirect effect of sarcastic interactive moderation on perceived news quality via a loss of brand credibility could be likely to occur.

A factual and polite moderation style, in contrast, should be perceived as less biased and unfair, and therefore, should be more in line with users’ expectations toward professional journalism. The previously described positive effects of social presence and the reinforced contrast effect on users’ perceptions of media credibility and news story quality could, hence, be likely to occur when moderators respond to uncivil comments in a factual manner. Studies showing that factual talk show moderators and “straight” news stories receive higher credibility scores than humorous moderators and opinionated news stories support this assumption (Meyer, Marchionni, & Thorson, 2010; Vraga et al., 2012). Therefore, we hypothesize the following:

Hypothesis 3a (H3a): Factual moderation of uncivil comments will increase the perceived credibility of the news outlet.

Hypothesis 3b (H3b): Sarcastic moderation of uncivil comments will impair the perceived credibility of the news outlet.

Hypothesis 4a (H4a): Factual moderation of uncivil comments will increase the perceived news story quality.

Hypothesis 4b (H4b): Sarcastic moderation of uncivil comments will impair the perceived news story quality.

Few research studies have investigated how media credibility and news story quality affect online participation. Still, these few studies have produced largely consistent results, namely, that high media credibility increases news participation (Chung, 2008; Holton et al., 2015). On the one hand, this finding can be explained with the evident fact that news consumption is a prerequisite of participation via comments on news websites. As media consumers tend to consume the news media they trust in (Wanta & Hu, 1994), media credibility should also increase users’ willingness to participate. Still, Chung (2008) found news credibility to be a significant predictor of the use of human interactive features such as the comment function even among the users of a news website. This can be explained with community research, which found that credibility-related factors such as perceived similarity between members and administrators, as well as trust and likability increase community commitment, which in turn encourages participation (Holton et al., 2015; Kraut et al., 2011). Overall, we can, therefore, expect a positive effect of news media credibility on the willingness to participate.

The picture is less clear, however, regarding the link between perceived news quality and participation. In a nation-wide U.S. survey, among users who comment on news websites, about one third reported that their comments are driven by the desire to correct inaccuracies or misinformation (Stroud et al., 2016). Other studies similarly argue that users write comments particularly when they are discontented with a news story’s quality (Diakopoulos & Naaman, 2011; Springer et al., 2015). Hence, users should be more likely to participate when they perceive that a particular news story has quality deficits. In sum, we hypothesize the following:

Hypothesis 3c (H3c): The more observers perceive a news outlet as credible, the higher will be their willingness to participate.

Hypothesis 4c (H4c): The more positive observers evaluate a news story’s quality, the lower will be their willingness to participate.

The Interplay Between Moderation (Styles) and Discussion Topics

Incivility in user comments occurs across different discussion topics—although not to the same extent. Research has shown that “weightier topics generate a higher percentage of uncivil commentary” (Coe et al., 2014, p. 669). Similarly, numerous journalists, editors, and community managers have reported that news stories describing social or political conflicts—“weightier” topics—regularly attract uncivil comments and that these topics, therefore, require heavier moderation than “lighter” topics (Reich, 2011). Regarding the interactive moderation of comments sections, it can be inferred from these arguments that journalists may particularly want to respond to uncivil comments on “weightier” topics and neglect the comparably few uncivil comments posted to “lighter” topics. However, to date, it has remained unexplored whether users discriminate between the need of journalistic moderation of “light” and “weightier” discussion topics. From a dual processing perspective (Petty & Cacioppo, 1986), it can be assumed that users will put greater effort in reading and judging “weightier” issues that are highly involving (because of the relevance of these issues and their potential for affective arousal). Users then should also elaborate more on whether factual or sarcastic moderation of uncivil user comments is necessary and appropriate, for example. This might in turn influence the individual willingness to participate or the perceptions of the discussion atmosphere. However, because barely any research has been published on this specific question, we will examine the interplay between moderation activities and the discussion topic with the following research question:

Research Question (RQ2): Will the effects of journalistic responses to uncivil comments depend on the weight of the discussion topics?

Method

Research Design and Sample

To test the hypotheses and answer the research questions, we conducted a 3 × 2 between-subjects web-based experiment. Sixty students of an undergraduate course at a large university in Germany were asked to distribute the link to the experiment in their social networks. Participation was voluntary and participants were offered the opportunity to take part in a lottery for gift vouchers. This procedure resulted in a total sample of 811 participants. We excluded participants who had not answered all sociodemographic questions and who did not consume any news online. Of the resulting sample (n = 731), 60% were female and the mean age was 24 years (SD = 7.37). Sixty-one percent of the participants had received a university entrance diploma and another 22% had an academic degree.

Procedure and Stimulus Material

After answering sociodemographic questions and some general measures of media consumption, participants were asked to provide their evaluation of the fictitious Facebook news site compact. For this purpose, we randomly assigned participants to one of six sets of three consecutive screenshots of the site. Participants had to click a “Continue” button to access the next screenshot. The first and the last screenshot varied between the experimental groups, while the second screenshot served as a “buffer” and was the same for all groups. Each screenshot showed a different news story, which the compact had recently published. All news stories were supplemented by the two most recent user comments. While all these comments were civil in the second “buffer” screenshot, one comment in the first screenshot was uncivil, harshly attacking and denigrating the media, the quality and credibility of their coverage, and the topic of the news story. In the third screenshot, there was an uncivil comment as well, which insulted politicians and journalists. These two comments were used in all experimental groups. Yet, depending on the experimental group, the user comments either remained unanswered by the compact or they were answered in a factual or sarcastic manner (independent variable 1). The factual responses answered the uncivil user comments in a constructive and polite way: In the first screenshot, the compact prompted the commenter to stop insulting and denigrating others, and in the third screenshot, the compact asked the uncivil commenter to give reasons for his or her critique and to express it in a more civil way next time. The sarcastic moderation contained essentially the same prompts and requests, yet worded in a more satiric and ironic way: In the first screenshot, the compact asked the commenter whether he or she was proud of running around screaming, insulting, and denigrating others. Then, the compact recommended the commenter to take a deep breath and swallow some tranquilizers before writing a comment next time. In the third screenshot, the compact ironically prompted the commenter to rethink whether his or her comment included a sufficient number of exclamation marks in order to be persuasive and sarcastically pointed out the “countless” arguments in the comment. The uncivil comments as well as the different journalistic responses are translated and documented in Table 1.

Table 1.

Translation of the Stimulus Material.

	Screenshot 1	Screenshot 3
“Weightier” topic	After death of an infant: Recent study on the positive effects of measles vaccinations fuels the debate about compulsory vaccination	Number of child abuses decreases slightly
“Lighter” topic	Recent study shows that Aspirin helps relieve osteoporosis	Illegal employment rates decrease slightly
Uncivil comment	Aren’t you ashamed of spreading such MANIPULATIVE TRASH?! Of course you aren’t, damn LYING PRESS!!!! Just admit that you are owned by those fucking pharmaceutical companies!	I won’t even read such crap! The only statistics you can trust are those you falsified yourself! The real issue here is much bigger but you morons just don’t dare telling it! Poor journalism, just rehashing the sugarcoated figures of our corrupt and disgusting politicians!!!!!!!
Factual moderation	Dear (blurred), please avoid insulting and denigrating others here. Try to express your critique in a more civilized way, as this will help everyone. Best regards from the editorial office.	Dear (blurred), would you please read the article more thoroughly and then try to comment a bit more politely next time. Please also try to give reasons for your claims. We welcome critical contributions but also want discussions to be respectful and constructive. Best regards from the editorial office.
Sarcastic moderation	Hey (blurred), are you proud of running around screaming, insulting, and denigrating others? Better take a deep breath and swallow some tranquilizers before commenting next time. This will help everyone here.	Hey (blurred), of course you don’t need to read the article! The countless arguments in your comment already prove that you have studied many reliable sources. But next time, consider that your comment will be even more persuasive if you use more exclamation marks!!!!!!!!!!!

Furthermore, in order to manipulate the issue involvement of our participants, the topics of the articles were varied (independent variable 2): Half of the participants were exposed to articles about somewhat “lighter” and less obtrusive issues, the other participants were exposed to “weightier” and more obtrusive issues of a higher emotional potential. The “lighter” topics were about the potentially positive effects of aspirin on the prevention of osteoporosis in the first screenshot (a disease that is very uncommon within the average age group of our participants) and about the declining rates of illegal work in the third screenshot. The “weightier” articles dealt with the effects of vaccinations in the first screenshot (a very controversial issue in Germany at the time of the study) and with recent successes in the fight against child maltreatment in the third screenshot.

The dependent variables used in this study were measured right after the participants had finished inspecting all the three screenshots to which they were assigned. Hence, the participants had to provide their overall evaluations of the news stories and the discussions.

In sum, we tried to ensure internal and external validity with our design: regarding external validity, we used three screenshots to give the participants a bigger picture of the activities of the compact. The neutral and civilly discussed “buffer” article between the two “experimental” articles reflected the real-world fact that not all user discussions contain incivility (Coe et al., 2014). Moreover, we used multiple user comments and media responses that were designed in the style of real-world news discussions on established Facebook news sites (i.e., Die Welt and Tagesschau). We also used real news stories that had been published on other news websites. Finally, we conducted a pretest to ensure that the Facebook site of the compact, the news stories, the user comments, and the journalistic responses were perceived as realistic. Regarding internal validity, we varied only the tone but not the length and the core content of the journalistic responses to uncivil comments. For the topic involvement treatment, we varied only the level of emotionality, obtrusiveness, and controversy of the issues, but we tried to maintain the tone and the nature of the news stories between the conditions (e.g., in each condition, there was a scientific study showing the positive effects of aspirin or vaccinations, respectively).

Measures

Unless noted otherwise, all variables were measured on 5-point scales ranging from 1 = I do not agree at all to 5 = I fully agree.

Willingness to participate

Right after exposing the participants to the stimulus material, they were asked to indicate their willingness to participate in one or more of the discussions about the news stories (e.g., “I would like to participate in one of the discussions about the news stories,” four items, α = .77). The scale was constructed using items from previous research on online participation (Borah, 2014; Ng & Detenber, 2005). In addition to the measurement of the (hypothetical) willingness to participate, participants were given the (voluntary) opportunity to write a comment themselves at the end of the questionnaire.

Discussion atmosphere

We adopted items from Vraga and colleagues (2012) and Ng and Detenber (2005) to measure the perceived deliberativeness of the discussions (e.g., “the discussions on this site were respectful,” “it is possible to discuss here without being denigrated by others,” four items, α = .80) and their entertainment value (e.g., “discussions were entertaining,” two items, α = .81).

News outlet credibility and news quality

Based on previous research on news credibility, journalistic quality, and media brand personality (Kim, Baek, & Martin, 2010; McCroskey & Teven, 1999; Thorson et al., 2010; Urban & Schweiger, 2014), we constructed two scales to measure participants’ evaluations of the credibility of the compact (five items on a semantic differential, for example, “incompetent—competent,” “trustworthy—untrustworthy,” α = .90) and the perceived journalistic quality of the news stories associated with the user discussions (e.g., “news articles were fair,” “news articles reported the relevant facts,” four items, α = .79).

Treatment check and control variables

Age, gender, and education of the participants were measured right at the beginning of the questionnaire. For the treatment check, participants were asked in a later part of the survey to indicate the level of irony and humor in the journalistic responses (“the responses were funny,” “the responses were ironic,” two items, α = .75), the level of objectiveness (e.g., “the responses were matter-of-factly,” two items, α = .89), and the level of aggressiveness (“the responses were aggressive”). Furthermore, participants were asked to evaluate the levels of controversy and emotionality of the events that were described in the articles (e.g., “the issues were emotionally involving,” “the issues were controversial,” five items, α = .77). Finally, participants indicated on a 5-point scale the degree to which they had perceived the experimental condition as realistic (“altogether, how close to reality did you find what you saw on the Facebook site of the compact?” scale from 1 = not at all close to reality to 5 = very close to reality).

Results

Preliminary Analyses

Treatment check—Perception of moderation styles

Participants exposed to the factual journalistic response styles reported a lower level of perceived ironic humor (M = 1.81, SD = 0.79) than participants in the sarcastic moderation condition (M = 3.60, SD = 1.01, F(1, 582) = 577.71, p < .001, $η_{p}^{2}$ = .50). In contrast, participants in the factual moderation condition rated the moderation as more matter-of-factly (M = 3.99, SD = 0.81) than participants in the sarcastic moderation condition (M = 2.26, SD = 1.00, F(1, 582) = 536.16, p < .001, $η_{p}^{2}$ = .48). Finally, sarcastic responses were rated as significantly more aggressive (M = 2.65, SD = 1.16) than factual responses (M = 1.56, SD = 0.83, F(1, 582) = 154.07, p < .001, $η_{p}^{2}$ = .23).

Treatment check—Perception of topics

An ANOVA showed that participants in the “lighter” topic condition reported lower levels of issue emotionality and issue controversy (M = 2.26, SD = 0.60) than participants who were exposed to “weightier” topics (M = 3.15, SD = 0.72, F(1, 730) = 332.81, p < .001, $η_{p}^{2}$ = .31).

Control variable—Perceived realism of the stimulus

An ANOVA revealed no significant differences between the perceived realism of the stimulus, neither between the sarcastic (M = 3.47, SD = 0.97) and factual moderation conditions (M = 3.59, SD = 0.98), F(1, 730) = 1.46, p = .23), nor between the weightier topic (M = 3.58, SD = 0.97) and lighter topic conditions (M = 3.50, SD = 0.98, F(1, 730) = 1.35, p = .25).

Randomization checks

The sarcastic moderation group did not differ significantly from the factual moderation group or the unmoderated control groups regarding gender (χ² = 3.74, df = 2, p = .16), education (χ² = 9.2, df = 4, p = .12), and age of the participants (F(1, 730) = 0.78, p = .46). Similarly, participants were distributed equally on the “weightier” and “lighter” issues regarding gender (χ² = .19, df = 1, p = .71), education (χ² = 1.80, df = 2, p = .41), and age (F(1, 730) = 0.75, p = .39).

The Effects of Factual and Sarcastic Moderation

All hypotheses were tested through structural equation modeling using AMOS 23. Two models were calculated to compare unmoderated discussions with factually moderated discussions (Model 1) and unmoderated discussions with sarcastically moderated discussions (Model 2). The structural equation models (SEMs) included the confirmatory factor analyses, the dichotomous independent variables (no moderation vs. factual moderation/no moderation vs. sarcastic moderation), the four mediator variables (“deliberative discussion atmosphere,” “entertainment value,” “credibility of the news outlet,” “news story quality”), the outcome (“willingness to participate”), and finally the hypothesized paths between the variables. The models showed an acceptable fit to the data (Hu & Bentler, 1999): χ²/df = 1.80 (n = 368), p < .001, comparative fit index (CFI) = .95, root mean square error of approximation (RMSEA) = .05, standardized root mean square residual (SRMR) = .04 for model 1, χ²/df = 1.87 (n = 363), p < .001, CFI = .96, RMSEA = .05, SRMR = .05 for Model 2. Figure 2 shows the “factual moderation model” and Figure 3 shows the “sarcastic moderation model.” Indirect effects were calculated using bootstrapping (n = 5,000) with bias-corrected 95% confidence intervals (Hayes, 2013).

Figure 2.

Factual moderation model.

Figure 3.

Sarcastic moderation model.

According to the data, factual responses to uncivil comments increased observers’ perceptions of a deliberative discussion atmosphere (β = .11, p = .04), which supports H1a. Contrary to H1b, however, sarcastic responses did not significantly deteriorate these perceptions (β = −.04, p = .54). Observers’ perceptions of a deliberative discussion atmosphere significantly predicted their willingness to participate both in the “factual moderation model” (β = .17, p = .01) and in the “sarcastic moderation model” (β = .29, p < .001). These results support H1c. Consequently, the mediation effect of comment moderation on observers’ willingness to participate via perceptions of a deliberative discussion climate was significant only when uncivil comments were moderated in a factual manner (β = .03, p = .03, lower level confidence interval (LLCI) = 0.003, upper level confidence interval (ULCI) = 0.01). Apparently, the presence of a factual moderator can indeed encourage participation by suggesting that the discussions on the site are civil, respectful, and reciprocal.

Turning to the entertainment-related effects of journalistic responses to uncivil comments, factual responses did not significantly impair the perceived entertainment value of the discussions (β = .04, p = .51). H2a is not supported. Consistent with H2b, however, sarcastic responses significantly increased observers’ perceptions that the discussions were amusing and entertaining (β = .15, p = .01). Still, observers’ perceptions of an entertaining discussion atmosphere did predict their willingness to participate neither in the “factual moderation model” (β = .04, p = .47) nor in the “sarcastic moderation model” (β = −.05, p = .35). Regarding RQ1, the mere feeling of being entertained apparently does not motivate observers to post a comment themselves.

The final set of hypotheses predicted different effects of interactive moderation styles on the credibility of the news outlet as well as on observers’ evaluations of the quality of its news stories. Contrary to H3a and H4a, factual moderation did not significantly improve the compact’s credibility (β = .06, p = .38) or the perceptions of the quality of its news stories (β = .00, p = .94). However, supporting H3b, sarcastic responses to uncivil comments made observers judge the compact as significantly less credible and trustworthy (β = −.22, p < .001). Additional analyses confirmed that a sarcastic moderation style was perceived as inappropriate and that inappropriateness mediated the effect of moderation style on media credibility.² Still, participants’ perceptions of the quality of the news stories remained unaffected by sarcastic moderation (β = −.02, p = .67), which contradicts H4b.

H3c predicted a positive effect of the credibility of the compact on participants’ willingness to participate. However, the data do not support this hypothesis: Perceived credibility did not affect participation willingness in the “factual moderation model” (β = −.18, p = .09) and in the “sarcastic moderation model” (β = −.01, p = .96). In contrast, perceived news story quality had a significant positive effect on participants’ willingness to join the discussions both in the “factual moderation model” (β = .29, p = .01) and in the “sarcastic moderation model” (β = .23, p = .02). H4c, however, predicted effects in the opposite direction and was, therefore, not supported as well. Interestingly, however, we observed a negative indirect effect of sarcastic responses to uncivil comments on participation willingness via the routes of credibility and news story quality: Sarcastic responses reduced the credibility of the compact and credibility ratings correlated with perceived news story quality (β = .71, p < .001).³ Observers’ quality judgments in turn predicted their willingness to join the discussions. This mediation effect of sarcastic responses via credibility and news story quality on participation willingness was significant (β = −.06, p < .001, LLCI = −.10, ULCI = −.03). Obviously, the negative effects of sarcastic responses on the credibility of the compact “spilled over” on individuals’ assessments of the quality of its news stories and thereby impeded participation rates.

Although we did not predict a direct effect of moderation on participants’ willingness to join the discussions, the respective path was included in the models as a control variable. The effect was significant neither in the “factual moderation model” (β = −.06, p = .71) nor in the “sarcastic moderation model” (β = −.02, p = .74).

The Effects of the Discussion Topic

RQ2 asked for moderation effects of the “weight” of the discussion topic on the variables in our model. To assess these effects, we computed single grouped SEMs for each variable, with the “weightier” versus “lighter” topic conditions as a grouping variable. We then restricted the models by iteratively setting equal the strength of one path of one model with the respective path of the other model. A moderation effect is significant when this restriction significantly impedes the quality of the models (measured by chi-square tests of difference) because then, it cannot be assumed that the path strengths are equal for both groups (Byrne, 2010). Tables 2 and 3 contain the results for the “factual moderation model” (Table 2) and the “sarcastic moderation model” (Table 3).

Table 2.

Moderation Effects of Topic Weight on the Effects of a Factual Moderation Style.

Hypothesis	Path	β_weightier	β_lighter	$χ_{diff}^{2}$	p
H1a	m^a → ddc^b	.10	.13	0.02	.90
H1c	ddc → wtp^c	.34	.02	7.34	.007
H2a	m → ev^d	.13	−.04	2.22	.14
RQ1	ev → wtp	.02	.00	0.06	.81
H3a	m → cno^e	.02	.10	0.37	.54
H3c	cno → wtp	−.06	−.33	0.95	.33
H4a	m→ nsq^f	.03	−.06	0.47	.50
H4c	nsq →wtp	.22	.28	0.14	.71
Direct effect	m → wtp	−.04	−.05	0.69	.41

Moderation.

Deliberative discussion climate.

Willingness to participate.

Entertainment value.

Credibility of news outlet.

News story quality.

Table 3.

Moderation Effects of Topic Weight on the Effects of a Sarcastic Moderation Style.

Hypothesis	Path	β_weightier	β_lighter	$χ_{diff}^{2}$	p
H1a	m^a → ddc^b	−.05	−.01	0.12	.73
H1c	ddc → wtp^c	.39	.22	0.50	.48
H2a	m → ev^d	.20	.10	0.79	.37
RQ1	ev → wtp	−.07	−.03	0.05	.82
H3a	m → cno^e	−.33	−.11	4.51	.03
H3c	cno → wtp	.00	−.06	0.10	.75
H4a	m → nsq^f	.06	−.10	2.54	.11
H4c	nsq →wtp	.12	.33	1.60	.21
Direct effect	m → wtp	−.13	.08	3.22	.07

Moderation.

Deliberative discussion climate.

Willingness to participate.

Entertainment value.

Credibility of news outlet.

News story quality.

Most interestingly, topic weight affected the effects of sarcastic comment moderation on the credibility of the compact (Table 3). The negative effects were significantly stronger when the news outlet employed humor and sarcasm to moderate uncivil comments on emotional and controversial topics (β = −.33) than uncivil comments on “lighter” topics (β = −.11, $χ_{diff}^{2}$ (1, 319) = 4.51, p = .03). These results suggest that individuals perceive a sarcastic moderation as particularly inappropriate when the discussions circle around socially significant issues.⁴ This perception can amplify the damage on a news outlet’s credibility.

In the “factual moderation model” (Table 2), topic weight moderated the effects of the perceptions of a deliberative discussion climate on participants’ willingness to join the discussion. The effect was significant only in the “weightier” topic conditions ( $χ_{diff}^{2}$ (1, 319) = 7.34, p = .007), which also suggests that the indirect effect of factual moderation on the willingness to participate via the perception of a deliberative discussion climate was significant only for “weightier” topics. Obviously, when deciding about whether or not to join news discussions, our participants considered the deliberativeness of the discussion climate—but they did so only when the discussion topics were controversial and highly emotional.

Discussion

In an effort to increase the quality of user discussions on news websites and Facebook pages, researchers have recommended journalists and community managers to engage in public interactions with their commenting users. The results of previous studies have suggested that the quality of user comments increases when a “helpful” moderator responds to or highlights civil comments containing questions or valuable information. But how should news outlets react when users post uncivil comments? Simply deleting these comments could be perceived as an illegitimate limitation of free speech, yet leaving them uncommented could discourage other users from participating in the discussions and possibly damage the credibility of a news outlet and the quality of its stories. Our study investigated the effects of an interactive comment moderation strategy that could potentially tackle these undesired effects. This strategy recommends journalists and community managers to respond publicly to uncivil comments, ask their authors to discuss more civilized, and thereby demonstrate that the news outlet cares for high-quality discussions.

The results of our experiment suggest that regulative interactive moderation of uncivil comments can have some beneficial effects on observers’ evaluations of the discussions—yet, these effects depend heavily on the style of speech news outlets use: Compared with unmoderated discussions, a news outlet responding to uncivil commenters in a factual and polite manner made observers perceive a more deliberative discussion atmosphere, and thereby indirectly stimulated their willingness to post a comment themselves. This effect should encourage news outlets to publicly voice the rules of appropriate discussion behavior on occasion in order to show their endorsement of high-quality debates. Interactive moderation of uncivil comments can hence also be seen as an opportunity to increase the visibility of a news site’s general rules of conduct in general. This is particularly important as many news outlets have not documented these rules explicitly or do not make them easily accessible (Stroud, Scacco, & Curry, 2013).

In contrast, when the news outlet “fought fire with fire” and used satire and sarcasm to expose an uncivil commenter’s behavior as inappropriate, both the news outlet’s credibility and the perceived quality of its stories incurred more damage than when the uncivil comment was left unmoderated. These perceptions also indirectly impaired observers’ willingness to join the discussions and they were amplified when the discussions circled around highly controversial and emotional issues. These findings support our argument that news outlets should be careful with using satire and sarcasm to publicly expose the behavior of “ordinary citizens.” Observers of mediated conflicts tend to interpret these conflicts based on how they would feel and judge the other party if they were involved in the conflicts themselves (Mutz & Reeves, 2005). Moreover, publicly exposing an individual could be considered as a violation of face-to-face social norms and can make a communicator appear dislikable and unfair. Ultimately, the fear of being exposed through a sarcastic journalistic response could further impair observers’ willingness to post comments instead of increasing it.

Yet, although we predominantly reported negative effects of sarcastic moderation, we did not analyze the effects of this moderation style on users who write uncivil comments. In other contexts, “trolling” the comment authors who behave uncivilly has been found to be an effective strategy to change their behavior (Coles & West, 2016). It can still be hoped that sarcastic journalistic responses to uncivil comments will have a similar effect. Moreover, the positive effect of sarcastic moderation on the perceived entertainment value of the discussions should not be neglected. News outlets might see some benefit in hosting entertaining discussions about their “lighter” news stories—particularly on social platforms such as Facebook. Supporting this assumption, we found a significant correlation between the participants’ ratings of the entertainment value of the discussions and their intention to visit the site of the compact regularly (r = .136, p < .001). Hosting entertaining discussions could hence have at least some economic value for news outlets. Still, however, this virtue could possibly also be achieved by injecting more “harmless” forms of humor into the discussions.

Overall, these findings suggest that the mere presence of a newsroom in the comment sections alone does not sufficiently encourage observers to engage in the discussions or to perceive a sociable and constructive discussion atmosphere. Rather, observers seem to judge the interactive moderator with a rather critical eye according to whether he or she adheres to social norms and professional journalistic standards. Consistent with social information processing theory in text-only environments (Walther, 2011), observers rely on a communicator’s style of speech when making these judgments. A sarcastic moderation was perceived as inappropriate and participants used this cue to make general judgments about the credibility of the news outlet as a whole as well as about the quality of its news stories. These effects might have been particularly strong because the news outlet was unfamiliar to the observers and they only had a limited number of other cues to rely on when making these judgments (i.e., the uncivil comment itself and the news stories). Still, however, these negative effects did not occur when the news outlet interacted with uncivil commenters in a factual manner. This finding offers a new perspective on social presence, one that exceeds previous research, which had suggested that the social presence of reporters generally has negative effects when users are unfamiliar with a news organization (Hamman, 2006). Rather, the perceived professionality and likability of the social presence could account for different effects. That previous research did not find positive effects of social presence of a news outlet on its perceived credibility could, hence, also be a result of users disliking the specific manner by which journalists increased their social presence (Marchionni, 2015).

On that note, it has to be said that we cannot exclude that our participants also disliked the interactive moderation we analyzed—possibly, they might have perceived the journalistic interventions as a limitation of free speech. Although the majority of our participants held a positive attitude toward journalistic involvement in the comment sections—56% agreed with the statement that “journalists should interactively participate in user discussions on news websites,” while 20% disagreed with it—future studies should investigate more precisely the attitudes of users regarding regulative interactive moderation in particular. Still, we hold to our argument that this kind of interactive moderation is less restrictive than the currently practiced strategies of filtering and deleting comments (Ruiz et al., 2011).

Finally, regarding the users’ willingness to participate in the discussions, only factual moderation slightly increased this willingness by suggesting a deliberative discussion climate. Interestingly, however, when it comes to users’ actual participation behavior (which we measured by asking our participants to write a comment themselves), it is noteworthy that moderation even had a negative effect: Participants left a comment significantly less often when the discussions were moderated (23%) than when they were unmoderated (36%, χ² = 10.59, df = 2, p < .01). The style of the moderation did not make a difference here. This result lends support to the previously reported finding that a single uncivil contribution can increase users’ willingness to participate instead of decreasing it in the short term (Borah, 2014). In the long term, however, too much incivility is likely to deteriorate participation rates in online discussions (e.g., Stroud et al., 2016). A possible reason for the short-term effect is that some users feel a personal or social responsibility to rebuke uncivil commentators. When these users notice that journalists already supervise the discussions or that they rebuke uncivil commentators, they might feel less responsible to do so themselves. Future studies should, therefore, investigate more closely whether interactive moderation of user comments fosters some kind of diffusion of social responsibility (Latané & Nida, 1981) and whether this has consequences for news outlets trying to build strong user communities.

In sum, together with the findings of previous studies, our research can help develop realistic recommendations regarding the use of interactive journalistic moderation. News outlets could apply a mixed-mode moderation strategy that primarily aims at endorsing civilized and thoughtful comments but that also does not refrain from rebuking the authors of uncivil comments. This strategy might be complemented by the use of “harmless” humor in order to keep discussions sociable and to increase users’ commitment to the news outlet. At the same time, news outlets should try to maintain their professional journalistic norms when participating in the discussions. Responding to a comment with an inappropriate moderation style can make things even worse than they would have been if the comment was left unanswered, especially when the discussions circle around emotional and controversial topics.

Our findings should be interpreted only in light of several methodological limitations. First, our disproportionately young, well-educated, and female sample could limit the generalizability of our findings. Studies have found, for example, that media consumers with higher levels of education tend to be more critical than those with lower levels of education when judging the credibility of online news media (Johnson & Kaye, 1998). In contrast, younger individuals tend to trust news media more than older individuals (Johnson & Kaye, 1998). These biases in our sample, however, should not overly limit the external validity of our findings because social media users—such as those reading news stories on Facebook—tend to be younger and better educated than non-users in reality, too (Perrin, 2015). Still, future research should test the robustness of the effects reported in the present study using a more diverse and representative sample.

A second limitation concerns the experimental setting of the present study. Our participants were forced to read news stories they might not have been interested in under realistic circumstances. The generalizability of our findings is further limited because we only analyzed a restricted set of discussions (two uncivil ones) and moderation styles (two factual and two sarcastic responses). In this case, it cannot be ruled out that the reported effects stem from the specific characteristics of the messages analyzed (Jackson & Jacobs, 1983). Future studies should use multiple-message designs to assess the robustness of the effects reported here. In addition, future studies should investigate whether the effects of interactive moderation are different when discussions are already civil or when the newsroom engagement aims at helping and encouraging users instead of rebuking them.

Third, the effects we reported are predominantly small. On the one hand, this can be seen as a consequence of our stimulus design: Some participants mentioned that it was difficult to provide overall evaluations for the three news stories and discussions that were shown in a row. Maybe the effects would have been stronger if we had confronted participants only with one news story and one discussion. Yet, we decided to expose participants to three consecutive news stories and discussions in order to give them a bigger and more valid picture of the activities of the compact’s news site. On the other hand, the small effect sizes might be a consequence of the use of a fictitious news brand for which the participants were not either able to or motivated to provide detailed judgments. Future studies should analyze more closely whether the effects reported in this study also appear for well-known news brands. Finally, we could only analyze short-term effects of a new phenomenon that users are not yet very familiar with. In the long term, readers might slowly get used to the interactive journalistic moderation and begin to accept it as a part of their personal concept of “journalism-as-a-conversation.”

Despite these limitations, the current study provides valuable insights into the effects of interactive journalistic moderation of uncivil user comments. Dealing with incivility in online discussions and cultivating deliberative discussion norms will prevail as important issues for journalism as well as for professional communicators engaging in social media activities in general. Our results should not mislead news organizations and other professional (mass) communicators to abandon interactive moderation of uncivil comments as a fruitful strategy. Yet, this engagement will have to be planned carefully, introduced slowly, and it will have to be consistent with professional and ethical standards. A hasty engagement in the comment sections—even if it is meant to be humorous—quickly may result in “unfunny” consequences for the reputation of (news) organizations.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Parts of this research were supported by German Research Foundation Grant QU 215/3-1 to Oliver Quiring.

Notes

Author Biographies

Marc Ziegele, PhD, is a postdoctoral researcher in the Department of Communication at the University of Mainz, Germany. His research interests concern online participation research, research on changes in journalism, online privacy behavior, analyses of electronic-word-of-mouth, and theory development within interactive online environments.

Pablo B. Jost, MA, is a research associate in the Department of Communication at the University of Mainz. His main research focus is on political communication and journalism, especially in the context of digital media change, and factors of online persuasion.

References

Anderson

A. A.

Brossard

Scheufele

D. A.

Xenos

M. A.

Ladwig

(2014). The “nasty effect”: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19, 373-387. doi:10.1111/jcc4.12009

Anderson

A. A.

Huntington

H. E.

Kandra

(2015, August 6-9). Incivility or sarcasm? Expanding the concept of attacks in online social media. Paper presented at the AEJMC 2015 Conference, San Francisco, CA.

Baym

N. K.

(1995). The performance of humor in computer-mediated communication. Journal of Computer-Mediated Communication, 1. doi:10.1111/j.1083-6101.1995.tb00327.x

Berger

(2011). Arousal increases social transmission of information. Psychological Science, 22, 891-893. doi:10.1177/0956797611413294

Bergström

Wadbring

(2014). Beneficial yet crappy: Journalists and audiences on obstacles and opportunities in reader comments. European Journal of Communication, 30, 137-151. doi:10.1177/0267323114559378

Besley

J. C.

(2006). The role of entertainment television and its interactions with individual values in explaining political participation. The International Journal of Press/Politics, 11(2), 41-63. doi:10.1177/1081180X06286702

Borah

(2014). Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, 41, 809-827. doi:10.1177/0093650212449353

Brown

Levinson

S. C.

(1987). Studies in Interactional Sociolinguistics: Vol. 4. Politeness: Some universals in language usage. Cambridge, UK: Cambridge University Press.

Byrne

B. M.

(2010). Structural equation modeling with AMOS: Basic concepts, applications, and programming (2nd ed.). New York, NY: Taylor & Francis.

10.

Carter

S. L.

(1998). Civility: Manners, morals, and the etiquette of democracy. New York, NY: HarperCollins.

11.

Chung

D. S.

(2008). Interactive features of online newspapers: Identifying patterns and predicting use of engaged readers. Journal of Computer-Mediated Communication, 13, 658-679. doi:10.1111/j.1083-6101.2008.00414.x

12.

Coe

Kenski

Rains

S. A.

(2014). Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication, 64, 658-679. doi:10.1111/jcom.12104

13.

Coles

B. A.

West

(2016). Trolling the trolls: Online forum users constructions of the nature and properties of trolling. Computers in Human Behavior, 60, 233-244. doi:10.1016/j.chb.2016.02.070

14.

Dahlberg

(2001). Computer-mediated communication and the public sphere: A critical analysis. Journal of Computer-Mediated Communication, 7. doi:10.1111/j.1083-6101.2001.tb00137.x

15.

Diakopoulos

N. A.

Naaman

(2011). Towards quality discourse in online news comments. In Proceedings of ACM CSCW11 Conference on Computer-Supported Cooperative Work (pp. 133-142). New York, NY: Association for Computing Machinery.

16.

Domingo

Quandt

Heinonen

Paulussen

Singer

J. B.

Vujnovic

(2008). Participatory journalism practices in the media and beyond: An international comparative study of initiatives in online newspapers. Journalism Practice, 2, 326-342.

17.

Elliott

R. C.

(2014). Satire: Encyclopaedia Britannica. Available from http://www.britannica.com

18.

Fine

G. A.

Martin

D. D.

(1990). A partisan view: Sarcasm, satire, and irony as voices in Erving Goffman’s Asylums. Journal of Contemporary Ethnography, 19, 89-115. doi:10.1177/089124190019001005

19.

Hamman

(2006). Two voices: Social presence, participation, and credibility in online news (Master’s thesis, Faculty of the Graduate School, University of Missouri-Columbia). Available from https://mospace.umsystem.edu

20.

Hassanein

Head

(2007). Manipulating perceived social presence through the web interface and its impact on attitude towards online shopping. International Journal of Human-Computer Studies, 65, 689-708. doi:10.1016/j.ijhcs.2006.11.018

21.

Hayes

A. F.

(2013). Methodology in the Social Sciences: Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford Press.

22.

Hoffman

L. H.

Young

D. G.

(2011). Satire, punch lines, and the nightly news: Untangling media effects on political participation. Communication Research Reports, 28, 159-168. doi:10.1080/08824096.2011.565278

23.

Holton

A. E.

Coddington

Lewis

S. C.

de Zuniga

H. G.

(2015). Reciprocity and the News: The Role of Personal and Social Media Reciprocity in News Creation and Consumption. International Journal of Communication, 9, 2526-2547.

24.

Hsueh

Yogeeswaran

Malinen

(2015). “Leave your comment below”: Can biased online comments influence our own prejudicial attitudes and behaviors? Human Communication Research, 41, 557-576. doi:10.1111/hcre.12059

25.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.

26.

Hwang

Kim

(2016). Influence of discussion incivility on deliberation: An examination of the mediating role of moral indignation. Communication Research, 45, 213-240. doi:10.1177/0093650215616861

27.

Jackson

Jacobs

(1983). Generalizing about messages: Suggestions for design and analysis of experiments. Human Communication Research, 9, 169-191. doi:10.1111/j.1468-2958.1983.tb00691.x

28.

Janssen

Kies

(2005). Online forums and deliberative democracy. Acta Politica, 40, 317-335. doi:10.1057/palgrave.ap.5500115

29.

Johnson

T. J.

Kaye

B. K.

(1998). Cruising is believing? Comparing Internet and traditional sources on media credibility measures. Journalism & Mass Communication Quarterly, 75, 325-340. doi:10.1177/107769909807500208

30.

Kennedy

(1998). State of the debate: The case against “civility.” Available from http://prospect.org

31.

Kim

Baek

T. H.

Martin

H. J.

(2010). Dimensions of News Media Brand Personality. Journalism & Mass Communication Quarterly, 87(1), 117-134. doi.org/10.1177/107769901008700107

32.

Kraut

R. E.

Resnick

Kiesler

(2011). Building successful online communities: Evidence-based social design. Cambridge, MA: MIT Press.

33.

Kreuz

R. J.

Roberts

R. M.

(1993). On satire and parody: The importance of being ironic. Metaphor and Symbolic Activity, 8, 97-109. doi:10.1207/s15327868ms0802_2

34.

Kuipers

(2015). Satire and dignity. In Meijer Drees

de Leeuw

(Eds.), The power of satire (pp. 19-32). Amsterdam, The Netherlands: John Benjamins.

35.

Latané

Nida

(1981). Ten years of research on group size and helping. Psychological Bulletin, 89, 308-324. doi:10.1037/0033-2909.89.2.308

36.

Marchionni

D. M.

(2013). Journalism-as-a-conversation: A concept explication. Communication Theory, 23, 131-147. doi:10.1111/comt.12007

37.

Marchionni

D. M.

(2015). Journalism-as-a-conversation: An experimental test of socio-psychological/technological dimensions in journalist-citizen collaborations. Journalism, 16, 218-237. doi:10.1177/1464884913509783

38.

Massaro

T. M.

Stryker

(2012). Freedom of speech, liberal democracy, and emerging evidence on civility and effective democratic engagement. Arizona Law Review, 54, 375-441.

39.

McCroskey

J. C.

Teven

J. J.

(1999). Goodwill: A reexamination of the construct and its measurement. Communication Monographs, 66, 90-103. doi.org/10.1080/03637759909376464

40.

Meyer

H. K.

Carey

M. C.

(2014). In moderation: Examining how journalists’ attitudes toward online comments affect the creation of community. Journalism Practice, 8, 213-228. doi:10.1080/17512786.2013.859838

41.

Meyer

H. K.

Marchionni

Thorson

(2010). The journalist behind the news: Credibility of straight, collaborative, opinionated, and blogged “news.” American Behavioral Scientist, 54, 100-119. doi:10.1177/0002764210376313

42.

Mutz

D. C.

Reeves

(2005). The new videomalaise: Effects of televised incivility on political trust. American Political Science Review, 99, 1-15.

43.

Newhagen

Nass

(1989). Differential criteria for evaluating credibility of newspapers and TV news. Journalism & Mass Communication Quarterly, 66, 277-284. doi:10.1177/107769908906600202

44.

Newman

Levy

D. A. L.

Nielsen

R. K.

(2015). Reuters Institute digital news report 2015: Tracking the future of news. Available from https://reutersinstitute.politics.ox.ac.uk

45.

E. W. J.

Detenber

B. H.

(2005). The impact of synchronicity and civility in online political discussions on perceptions and intentions to participate. Journal of Computer-Mediated Communication, 10, 0. doi:10.1111/j.1083-6101.2005.tb00252.x

46.

Papacharissi

(2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6, 259-284. doi:10.1177/1461444804041444

47.

Perrin

(2015). Social media usage: 2005-2015. Available from http://www.pewinternet.org

48.

Petty

R. E.

Cacioppo

J. T.

(1986). Communication and persuasion: Central and peripheral routes to attitude change. Berlin, Germany: Springer.

49.

Phillips

Smith

(2004). Emotional and behavioural responses to everyday incivility: Challenging the fear/avoidance paradigm. Journal of Sociology, 40, 378-399. doi:10.1177/1440783304048382

50.

Prochazka

Weber

Schweiger

(2016). Effects of civility and reasoning in user comments on perceived journalistic quality. Journalism Studies, 19, 62-78. doi:10.1080/1461670X.2016.1161497

51.

Reich

(2011). User comments: The transformation of participatory space. In Singer

J. B.

Hermida

Domingo

Heinonen

Paulussen

Quandt

Vujnovic

(Eds.), Participatory journalism: Guarding open gates at online newspapers (pp. 96-117). Malden, MA: Wiley-Blackwell.

52.

Rill

L. A.

Cardiel

C. L. B.

(2013). Funny, ha-ha: The impact of user-generated political satire on political attitudes. American Behavioral Scientist, 57, 1738-1756. doi:10.1177/0002764213489016

53.

Rowe

(2015). Deliberation 2.0: Comparing the deliberative quality of online news user comments across platforms. Journal of Broadcasting & Electronic Media, 59, 539-555. doi:10.1080/08838151.2015.1093482

54.

Ruiz

Domingo

Micó

J. L.

Díaz-Noci

Meso

Masip

(2011). Public sphere 2.0? The democratic qualities of citizen debates in online newspapers. The International Journal of Press/Politics, 16, 463-487. doi:10.1177/1940161211415849

55.

Short

Williams

Christie

(1976). The social psychology of telecommunications. London, England: John Wiley.

56.

Sobieraj

Berry

J. M.

(2011). From incivility to outrage: Political discourse in blogs, talk radio, and cable news. Political Communication, 28, 19-41. doi:10.1080/10584609.2010.542360

57.

Springer

Engelmann

Pfaffinger

(2015). User comments: Motives and inhibitors to write and read. Information, Communication & Society, 18, 798-815. doi:10.1080/1369118X.2014.997268

58.

Stroud

N. J.

Muddiman

(2013). Selective exposure, tolerance, and satirical news. International Journal of Public Opinion Research, 25, 271-290. doi:10.1093/ijpor/edt013

59.

Stroud

N. J.

Scacco

J. M.

Curry

A. L.

(2013). Interactive features in online news. Available from http://engagingnewsproject.org

60.

Stroud

N. J.

Scacco

J. M.

Muddiman

Curry

A. L.

(2015). Changing deliberative norms on news organizations’ Facebook sites. Journal of Computer-Mediated Communication, 20, 188-203. doi:10.1111/jcc4.12104

61.

Stroud

N. J.

van Duyn

Peacock

(2016). News commenters and news comment readers. Available from http://engagingnewsproject.org

62.

Thorson

Vraga

Ekdale

(2010). Credibility in context: How uncivil online commentary affects news credibility. Mass Communication and Society, 13, 289-313. doi:10.1080/15205430903225571

63.

Thurman

Cornia

Kunert

(2016). Journalists in the UK. Available from https://reutersinstitute.politics.ox.ac.uk

64.

Toplak

Katz

A. N.

(2000). On the uses of sarcastic irony. Journal of Pragmatics, 32, 1467-1488. doi:10.1016/S0378-2166(99)00101-0

65.

Urban

Schweiger

(2014). News quality from the recipients’ perspective. Journalism Studies, 15, 821-840. doi:10.1080/1461670X.2013.856670

66.

Vraga

E. K.

Edgerly

Bode

Carr

D. J.

Bard

Johnson

C. N.

. . . Shah

D. V.

(2012). The correspondent, the comic, and the combatant: The consequences of host style in political talk shows. Journalism & Mass Communication Quarterly, 89, 5-22. doi:10.1177/1077699011428575

67.

Vujnovic

(2011). Participatory journalism in the marketplace: Economic motivations behind the practices. In Singer

J. B.

, et al. (Eds.), Participatory journalism: Guarding open gates at online newspapers (pp. 139-154). Malden, MA: Wiley-Blackwell.

68.

Walther

J. B.

(2011). Theories of computer-mediated communication and interpersonal relations. In Knapp

M. L.

Daly

J. A.

(Eds.), The SAGE handbook of interpersonal communication (4th ed., pp. 443-479). Los Angeles, CA: SAGE.

69.

Wanta

Y.-W.

(1994). The effects of credibility, reliance, and exposure on media agenda-setting: A path analysis model. Journalism & Mass Communication Quarterly, 71, 90-98. doi:10.1177/107769909407100109

70.

Wise

Hamman

Thorson

(2006). Moderation, response rate, and message interactivity: Features of online communities and their effects on intent to participate. Journal of Computer-Mediated Communication, 12, 24-41.

71.

Wright

(2006). Government-run online discussion fora: Moderation, censorship and the shadow of control. The British Journal of Politics and International Relations, 8, 550-568. doi:10.1111/j.1467-856X.2006.00247.x

72.

Ziegele

Breiner

Quiring

(2014). What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on mass media websites. Journal of Communication, 64, 1111-1138. doi:10.1111/jcom.12123