Predicting perceptions of incivility across 20 news comment sections

Abstract

Research on incivility has focused on individual differences or message characteristics that affect perceptions of incivility. Given the potential consequences of incivility in news spaces, it is essential to extend research on incivility perceptions to characteristics of digital news environments. We explore how the perception and availability of comment section features (i.e. comment moderation and anonymity), as well as a feeling of connection with a comment section community, relate to commenters and comment readers’ incivility perceptions. By recruiting survey respondents from a population of individuals who read or post comments on one of 20 diverse news sites, we demonstrate that features of commenting spaces, and especially the feelings of connection formed in those spaces, relate to perceptions of incivility. We find that perceptions of comment section features relate to perceptions of incivility more so than the actual availability of these features. Our findings offer implications for newsrooms looking to foster perceptions of civility in comment sections and protect the value of their journalism.

Keywords

Communication connection digital news reader comments user comments

Journalists and researchers are concerned about incivility in news spaces for good reason (e.g. Meltzer, 2015). Although incivility may encourage political engagement (Chen and Lu, 2017), uncivil comments can incite aggressive replies (Chen and Lu, 2017), polarize reactions to stories (Anderson et al., 2014), and potentially discourage others from participating in discussion (Diakopoulos and Naaman, 2011).

Yet there has been dispute over what counts as incivility. Most scholars describe incivility as a lack of respect or a violation of social norms, though beyond this broad definition there is less agreement (Coe et al., 2014; Mutz, 2015). Researchers have explored the slipperiness of the concept by investigating variables that influence incivility perceptions. Age (Ben-Porath, 2008), conflict avoidance (Mutz, 2015), personality traits (Kenski et al., 2017), and partisan identity (Mutz, 2015) all seem to influence how people view incivility, and these perceptions may have effects in digital spaces (Muddiman et al., 2017).

Previous research has focused on individual differences – characteristics of a person that affect incivility perceptions (e.g. Kenski et al., 2017) – or the message – characteristics of content that prompt incivility perceptions (e.g. Muddiman et al., 2017). It is essential to extend research on perceptions of incivility to characteristics of digital news spaces themselves because environmental characteristics can influence the discursive behavior that is considered appropriate in a given space (e.g. Freelon, 2015). We link perceptions of incivility to three characteristics of a news site – comment moderation (Ksiazek, 2015), anonymity (Rösner and Krämer, 2016), and feelings of connection (Baumeister, 1982) – that have been previously associated with norms and norm violations.

This study advances incivility and news research in several ways. First, others have examined how the features of comment sections influence content and user engagement (Freelon, 2015), but none have examined whether comment section users perceive more incivility in varied contexts. Second, these studies have examined the presence of comment section features, not the extent to which users pay attention to the features nor whether this attention is related to their experiences. Third, we investigate an element of digital spaces that has received little attention in relation to incivility: perceived connectedness with others in the comment section (Anderson, 2006; Coles and West, 2016). Finally, studies of incivility have examined perceptions in controlled experimental settings (Kenski et al., 2017; Mutz, 2015). By recruiting survey respondents from a population of individuals who read or post comments on one of 20 diverse news sites, we explore perceptions of incivility in an externally valid setting. Although we cannot establish a causal direction for these relationships, we draw from previous work to highlight environmental variables that may relate to perceptions of incivility and to suggest ways newsrooms can attenuate these perceptions.

Incivility and news comment sections

Across the range of approaches to incivility, there is one common denominator: incivility is a violation of social norms, ranging from violations of politeness norms (Mutz, 2015) to violations of norms related to democratic practice (Papacharissi, 2004). Norms are those behaviors deemed acceptable in a ‘certain situation or environment’, created and reinforced by interactions with others (Aarts and Dijksterhuis, 2003: 18). To understand what influences incivility perceptions, researchers must investigate the elements of comment section environments that foster norms in those spaces. We explore two such elements – structural features and connectedness – examining whether the comment section environment relates to perceptions of incivility.

Comment section features: Moderation and anonymity

When websites include features that allow for different types of social interactions (known as affordances, Nagy and Neff, 2015), discursive content on those sites varies (e.g. Freelon, 2015). This may be because different features privilege different norms. For example, Sukumaran et al. (2011) find that when features of the comment section cue thoughtfulness (e.g. labels that asked commenters to ‘enrich the discussion’, p. 3407), commenters posted longer, more relevant comments. Although previous research has looked at the content prompted by different site features, there has been little exploration of whether visitors notice differences. We examine how people perceive incivility on a site when news site features – and the norms that likely come with them – vary.

We focus on two structural features of the comment section: comment moderation (when journalists reject comments that violate the site’s policies) and anonymity (when commenters can post without revealing their identity). We choose these features over others because they are studied frequently in the literature, though they tend to be studied separately (Ksiazek, 2015; Rowe, 2015). We extend previous research by examining their comparative effects.

In addition, features of comment sections are designed for use by different stakeholders, which may create conflicting discursive norms (Ziegele et al., 2014). Some features are designed for use by journalists. Newsroom moderation (Ksiazek, 2015), for instance, may promote norms of civility. Comment moderation and journalist interaction have been shown to influence discursive norms and improve overall civility in comments (Ruiz et al., 2011), although the success of the moderation may depend on the incivility definitions used by moderating staff (Park et al., 2016). Moreover, newsrooms may be proactive against incivility because of public criticism about the level of incivility in comment sections (Meltzer, 2015; Santana, 2014), especially because comment incivility negatively affects overall perceptions of the news article and organization (Anderson et al., 2016; Chen, 2017). Even though incivility may be good for business in some ways – for example, by increasing the engagement of commenters in comment sections – journalists do not seem to actively promote uncivil content (Muddiman and Stroud, 2017). Thus, we argue newsrooms have normative reasons to downplay incivility, or at least not to encourage it. It is likely that features of comment sections that are controlled by news staff, such as comment moderation, are related to discursive norms of civility.

Other structural features, such as the ability to post anonymously, are designed for use by news users and may foster different norms. Like moderation, anonymity has been connected to norms. Early research on anonymity suggested that anonymous individuals ignore social norms because anonymity removes fear of personal retribution (Baumeister, 1982). Yet the connection between anonymity and norm violations is complex. Anonymity has, theoretically, been linked to both anti- and prosocial talk (Suler, 2004). Empirical evidence connecting anonymity and incivility is also mixed. On one hand, comments posted on news sites with anonymous platforms had a higher rate of incivility than comments on sites without anonymity (Santana, 2014). In addition, sites that require user registration (Ksiazek, 2015) or a connection to one’s actual identity (Rowe, 2015) are typically associated with less incivility than those that are anonymous. On the other hand, some researchers have found no direct effect of anonymity on uncivil commenting (Chen, 2017). Instead, the social norms of the site may prevail when anonymity exists (Rösner and Krämer, 2016). The social identity model of deindividuation effects (SIDE) suggests that the disinhibiting effect of anonymity, rather than increasing incivility in all contexts, makes contextual norms of the particular site more salient (Postmes et al., 1998; Rains et al., 2017; Spears and Lea, 1994). Thus, we suspect that the presence of anonymity will be related to perceptions of incivility in comment sections, but it is unclear whether news users will perceive more or less incivility when anonymity is available.

Importantly, previous research has focused on the presence or absence of comment section features (e.g. Freelon, 2015; Ziegele et al., 2014). We add an additional layer: a user’s perception that the feature exists on a site. Nagy and Neff (2015) contend that affordances should be approached as part of the technological design of a site but also as ‘imagined’ and perceived by site users (p. 1). Furthermore, research from media psychology offers evidence that perceptions may influence online behavior more so than reality (e.g. Neubaum and Krämer, 2017). It may be the features themselves, a user’s recognition of a feature, or an interaction between the two (i.e. whether the effects are stronger for users who have an accurate perception of a news site’s features than for users who have an inaccurate perception of a news site’s features) that relate to perceptions of incivility. Other research has looked at perceptions of site features (e.g. Rains, 2007), but scholars have yet to compare perceptions of site features to their actual availability. We are able to explore this distinction because we survey comment section visitors recruited from actual news sites, meaning that the newsrooms could provide information about the extent to which these features were actually available on their sites. In sum, we offer the following hypothesis and research questions:

H1. The (a) actual presence of moderation and (b) user perception of moderation in a comment section will relate to fewer perceptions of incivility than the lack of actual and perceived moderation.

RQ1. Are the (a) ability to post anonymously and (b) user perception that a newsroom allows commenters to post anonymously related to perceptions of incivility?

RQ2. Is there an interaction effect between (a) perceived and actual moderation and (b) perceived and actual anonymity in predicting perceptions of incivility?

Comment section connectedness

Online spaces can be valuable tools not only for discussion but also for social connection (Barnes, 2016; Norris, 2002), which is closely related to feelings of belonging (Grieve et al., 2013). Comment section features afford social and discursive norms (Kim et al., 2019) that drive online behavior (Postmes et al., 1998) and build solidarity among users (Spears and Lea, 1994). We suspect that a comment section visitor’s feeling of connection to others in the space may be related to the visitor’s perception that others are violating social norms, that is, whether they perceive that others are behaving uncivilly.

In digital news spaces, the comment section often brings news users together and fosters connections (Coles and West, 2016; Hopp et al., 2018; see Barnes, 2016 for an example of how online communities can also build barriers). People often come to comment sections precisely to find connection (Mitchelstein, 2011), a sense of shared reality (Konnokovs, 2013), and social interaction (Diakopoulos and Naaman, 2011). Research has investigated comment section norms in the context of partisan groups (Rains et al., 2017), but not in relation to building connections with the comment section as a whole, making research on connectedness and incivility an important next step.

Connection with others in the comment section may curb incivility by establishing prosocial norms. Specifically, connection with others online is related to more disclosure (Ma and Chan, 2014), as well as to prosocial (Mikal et al., 2016) and cooperative behaviors (Postmes et al., 1998). Feelings of connection with the group may also make individuals more likely to conform to or maintain norms of civility (Postmes et al., 1998). There is also support for the inverse of this relationship. Because incivility has been linked to negative evaluations of the news site (Anderson et al., 2016) and other commenters (Rains et al., 2017), it may relate negatively to feelings of connection. Perceptions of civility are likely relevant to connectedness because civil discourse is associated with mutual understanding and cohesion (Smith and Bressler, 2013).

Studies of comment section connectedness have commonly measured perceptions of connection rather than social or behavioral measures (e.g. Barnes, 2016; Hopp et al., 2018). We continue this approach by considering connectedness as a perception, and although we do not hypothesize as to whether connectedness causes civility or vice versa, we anticipate that increased connectedness will relate to fewer perceptions of incivility:

H2. Feelings of connectedness to commenters will negatively relate to perceptions of incivility.

Comment sections are comprised of users with different degrees of participation. More specifically, commenters and comment readers are distinct in that the former actively post and visibly engage with other users, while the latter do not (Blanchard and Markus, 2004; Springer et al., 2015). To date, little research has directly compared these groups, with only a few studies focusing on user motives (Diakopoulos and Naaman, 2011) and perceptions of news brands (Krebs and Lischka, 2019). We focus on comment readers’ perceptions of incivility compared to that of commenters.

It is possible that the relationship between connection and perceived social norms differs depending on how much one participates in the comment section (Blanchard and Markus, 2004). Hopp et al. (2018), for instance, found that increased engagement with a news site increased community value for the site. Moreover, it is possible that the act of expressing oneself changes one’s experience by promoting greater engagement, cognitive elaboration, and commitment to the message and conversation (Nekmat, 2012). Commenters and comment readers who differ in their level of engagement may then respond differently to the site’s features and feel different levels of connection with other commenters on the site.

Yet even if a user does not comment, that user can still be affected by the content of the comment section. Studies have found that simply reading news comments can influence perceptions of news brands (Krebs and Lischka, 2019), credibility (Anderson et al., 2016), and perceptions of content (Anderson et al., 2014). Consequently, comment readers may react to the site’s features and build connections with others on the site in ways similar to that of commenters. To compare these two groups and their relationships with the comment section, we pose the following research questions:

RQ3. Will commenters and comment readers perceive different levels of incivility in comment sections?

RQ4. Are the relationships between perceived incivility and (a) perceived moderation, (b) perceived anonymity, and (c) feelings of connectedness with commenters moderated by a person’s status as a commenter or comment reader?

Method

The Center for Media Engagement at The University of Texas at Austin collaborated with the Coral Project and 20 news sites to conduct a survey of commenters and comment readers (see Supplemental Online Appendix A). The newsrooms represented a variety of media, including newspapers, radio, television, magazine, and digital-only sources. We analyze all of the news media because comparing newspapers and non-newspapers produced similar results (see, for robustness tests, Supplemental Online Appendix B). The nature of collecting cross-sectional data from actual site users means our resulting sample is non-probabilistic and non-causal. Yet this approach yielded data about comment sections with greater external validity than representative data from the general population or experimental data collected in a controlled setting (Nah and Chung, 2012). We asked a variety of newsrooms to distribute our survey, inviting newsrooms that varied in geographic location, size, and medium. We reached out to each organization by email and continuously updated our list by inviting new newsrooms to participate as others declined. The study took place from 25 May through 13 December 2016.

Each organization distributed the same survey, the only difference being that we tailored each survey so that it named the participating newsroom. Respondents who accessed the survey shared by a specific newsroom answered questions about the comment section on that newsroom’s site. We let each outlet determine how it wanted to encourage its audience to take part in the survey (using a banner ad on the website, writing an article about the survey, sharing the survey link on social media, or distributing the link via a newsletter). Twenty-two organizations promoted the survey. This project includes results for 20¹ of the newsrooms because one eliminated comments mid-way through the survey and a second received few responses. The sites used a variety of commenting platforms when the survey was fielded: seven used Disqus, four used Livefyre, three used Facebook, three used in-house platforms, two used Civil Comments, and one used Viafoura.

Respondents

Respondents included 9050 comment section visitors (including both commenters and comment readers). They were 85 percent (n = 7606) White, 5 percent (n = 439) Hispanic/Latino, and 60 percent (n = 5421) male. They were on average 53.04 years old (standard deviation (SD) = 14.35, range 18–91 years old) and had a mean of 16.32 years of education (approximately a 4-year college degree; SD = 1.72 years). For demographics across all newsrooms, see Supplemental Online Appendix A.

Measures

Perceived incivility

The predicted variable in this study is perceived incivility. To retain respondents who were not provided incentives and were recruited while browsing the news, we use only one item to measure this variable and others in our study. Although single-item measures are not ideal, they are common in studies that investigate perceptions of news (Anderson et al., 2014, 2016). We also ensured that the single-item measures were similar to items used in prior research. To measure incivility perceptions, we used an item from Mutz and Reeves (2005). Respondents were asked ‘how civil or uncivil do you think the comments are on the (news organization) site?’ Response options ranged from very civil (1) to very uncivil (5). On average, respondents saw their respective news site as neither civil nor uncivil, although the range was considerable (M = 3.19, SD = 1.30 to M = 2.03 to M = 4.18).

Actual anonymity and moderation

To measure whether anonymity and moderation actually existed on a site, we asked the newsrooms to report whether their sites included each (1) or not (0). The newsrooms reported ‘whether people can post anonymously’ and ‘whether comments are moderated before they appear’. Thirteen of the websites (65%) reported allowing anonymity and three (15%) reported using moderation.

Perceived moderation

To determine whether respondents perceived moderation on a site, respondents were asked, ‘to the best of [their] knowledge’ whether they thought ‘comments are moderated before they appear’. Responses were dichotomous, with respondents reporting that moderation did (1) or did not (0) occur. Across all newsrooms, 21 percent thought that their respective news site moderated comments, although this number ranged from 7 to 75 percent.

Perceived anonymity

To determine whether respondents perceived anonymity, respondents were asked to report ‘to the best of [their] knowledge’ whether they thought people in the comment section could ‘post anonymously’. Responses were dichotomous. The respondent could report that the feature was available (1) or not (0). Across all newsrooms, 54 percent thought that their respective news site allowed people to comment anonymously, with considerable range, from 4 to 77 percent across newsrooms.

Connectedness

Following the lead of research studying online social connectedness, we measured connectedness as a feeling or perception among respondents, again limiting our measure to a single item due to constraints on survey length. The item we chose was similar to items used by Grieve et al. (2013), who measured respondents’ feelings of connection and disconnection with others on Facebook. Respondents were asked, ‘how connected or disconnected do you feel to the other commenters on the (news organization) site?’ Response options ranged from very disconnected (1) to very connected (5). On average, respondents were neither connected nor disconnected with commenters on the site (M = 2.73, SD = 1.13 to M = 2.13 to M = 3.28).

Visitor status

To distinguish between commenters (n = 6273) and comment readers (n = 2777), we started with a measure of comment frequency. Those who said they had never commented on the site but had read the comments were identified as comment readers and those who said they commented every day, a few times a week, weekly, a few times a month, or monthly or less frequently were identified as commenters (see Table 1).² The ratio of commenters to comment readers in this sample is generally reflective of the ratio of commenters to comment readers across the United States. (Stroud et al., 2016; Van Duyn et al., 2019).

Table 1.

Descriptive statistics for commenters and comment readers.

Variable		Commenters	Readers
Perceived incivility	M (SD)	3.09 (1.30)	3.44 (1.27)
Perceived incivility	Range mean across newsrooms	M = 1.93 to M = 4.03	M = 2.16 to M = 4.42
Perceived anonymity	% (n)	57% (3560)	47% (1296)
Perceived anonymity	% Range across newsrooms	4% to 78%	0% to 72%
Perceived moderation	% (n)	23% (1453)	16% (442)
Perceived moderation	% Range across newsrooms	8% to 76%	0% to 60%
Feelings of connectedness	M (SD)	2.87 (1.10)	2.41 (1.13)
Feelings of connectedness	Range mean across newsrooms	M = 2.32 to M = 3.40	M = 1.78 to M = 3.12
Total N		6273	2777

SD: standard deviation.

Control variables

Two control variables – sex (male = 1, female = 0) and age (see descriptive statistics above) – were included because males and younger individuals may be less affected by incivility than others (Ben-Porath, 2008; Kenski et al., 2017).³

Analysis

There are a number of ways in which the newsrooms differ (e.g. size, type of newsroom) and in which the variables of interest (e.g. connectedness, perceptions of site features, visitor status) may relate⁴ to one another. Thus, we control for newsroom differences and all of the model variables using multi-level regression modeling to ensure that significant relationships appear above and beyond the other variables in the model. The Supplemental Online Appendix D provides details about differences in these newsroom characteristics and additional robustness tests; these robustness tests should, however, be considered preliminary because many of the characteristics cannot be teased apart from one another (e.g. newspapers in our sample also tended to be the larger outlets). We first ran an unconditional model predicting perceived incivility that included a random intercept accounting for the newsrooms (see Table 2, Model 1). This model, first, provides the intraclass correlation coefficient (ICC = 0.19), which indicates there is indeed between newsroom variance that needs to be taken into account using multi-level models. Second, it provides a Level 1 variance that serves as a baseline for calculating the amount of Level 1 variance explained by the addition of other variables. The next model adds all main effects variables at Level 2 and Level 1 (Table 2, Model 2). The final model adds all interaction effects (Table 2, Model 3).

Table 2.

Multi-level regression models predicting perceived incivility.

	Model 1Unconditional	Model 2Main effects	Model 3Interactions
Fixed effects
	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)
Intercept	3.02^***(0.13)	4.61^***(0.17)	4.74^***(0.18)
Newsroom-level variables (Level 2)
Actual newsroom moderation	–	–0.12(0.29)	–0.13(0.29)
Actual newsroom anonymity	–	0.05(0.21)	0.03 (0.21)
Individual-level variables (Level 1)
Age	–	–0.002^**(0.001)	–0.002^**(0.001)
Male	–	–0.03(0.02)	–0.03(0.02)
Commenter/reader	–	–0.13^***(0.02)	–0.31^***(0.06)
Perceived moderation	–	–0.29^***(0.03)	–0.25^***(0.06)
Perceived anonymity	–	0.19^***(0.02)	0.15^*(0.06)
Connectedness	–	–0.50^***(0.01)	–0.55^***(0.02)
Interaction coefficients
Actual newsroom moderation × perceived moderation	–	–	0.02(0.06)
Actual newsroom anonymity × perceived anonymity	–	–	0.03(0.06)
Commenter/reader × perceived moderation	–	–	–0.06(0.06)
Commenter/reader × perceived anonymity	–	–	0.01(0.05)
Commenter/reader × connectedness	–	–	0.07^***(0.02)
Random effects
	Variance component (SD)	Variance component (SD)	Variance component (SD)
News organization variance (Level 2)	0.32(0.56)	0.18(0.43)	0.18(0.43)
Individual variance (Level 1)	1.35(1.16)	1.01(1.00)	1.00(1.00)
Deviance	28,460.08	25,798	25,784
Level-1 variance accounted for	–	0.25	0.26
N of Level 1 units	9050	9050	9050
N of Level 2 units	20	20	20

SE: standard error; SD: standard deviation.

^*p < 0.05; ^**p < 0.01; ^***p < 0.001.

Results

We, first, focused on perceptions of incivility among all site visitors, both commenters and comment readers. To begin, we tested whether actual moderation on a site predicted a lower likelihood of perceiving incivility compared to no moderation on a site (H1a). The model indicated that actual moderation did not significantly predict site visitors’ perceptions of incivility (B = −0.12, standard error (SE) = 0.29, p = 0.68; Table 2, Model 2). H1a was not supported.

Turning to H1b, we tested whether the individual-level perception of moderation on a site predicted perceived incivility on the site. This hypothesis was supported. Respondents who perceived that a newsroom moderated comments were significantly less likely to perceive incivility in that comment section compared to respondents who did not perceive that a newsroom moderated comments (B = −0.29, SE = 0.03, p < 0.001; Table 2, Model 2).

Next, we tested whether actual allowance of anonymity on a site was related to incivility perceptions (RQ1a). Once again, the presence of the feature alone was not a significant predictor of incivility perceptions (B = 0.05, SE = 0.21, p = 0.83; Table 2, Model 2). However, site visitor perception that a site allowed anonymity was a significant predictor (RQ1b). Perceived anonymity in the comment section was significantly and positively related to perceived incivility (B = 0.19, SE = 0.02, p < 0.001; Table 2, Model 2), suggesting that site visitors who believed the comment section allowed anonymity perceived more incivility than site visitors who believed the comment section did not allow anonymity.

Next, we tested RQ2, which asked whether (a) perceived and actual moderation and (b) perceived and actual anonymity interacted in predicting incivility (see Table 2, Model 3). Each interaction coefficient compared four groups: people who think the feature (moderation or anonymity) exists on a site and the feature actually does (accurate perception), people who think the feature exists on a site but it does not (inaccurate perception), people who think the feature does not exist on a site but it does (inaccurate perception), and people who think the feature does not exist on a site and it does not (accurate perception). Neither the interaction between actual news moderation and perceived moderation (RQ2a; B = 0.02, SE = 0.06, p = 0.70; Table 2, Model 3) nor between actual newsroom anonymity and perceived anonymity (RQ2b; B = 0.03, SE = 0.06, p = 0.59; Table 2, Model 3) was significant. This suggests that site visitors’ perceptions of a site feature are related to perceptions of incivility no matter whether they are accurate or inaccurate in their perception that a newsroom moderates or allows anonymity on the site.

H2 shifted focus to connectedness in comment sections.⁵ In our analysis, connectedness was a significant predictor of perceived incivility in the comment section. As site visitors reported increased connection with others, they reported less incivility in that comment section (B = −0.50, SE = 0.01, p < 0.001; Table 2, Model 2), offering support for H2. Comparing the conditional and unconditional models is instructive. When connectedness alone was added to the model, it explained 24 percent of the individual-level residual variance produced by an unconditional model. The other variables explained much less of the residual variance, ranging from less than 1 percent for belief that the newsroom allowed anonymous comments to 2 percent for belief that the newsroom moderated comments (see, for statistics, Supplemental Online Appendix E).

We then tested whether commenters and comment readers perceived incivility differently. RQ3 asked whether commenters and comment readers perceived different levels of incivility in comment sections. The test answered this question in the affirmative: commenters perceived significantly less incivility in the comment section than comment readers (B = −0.13, SE = 0.02, p < 0.001; Table 2, Model 2).

Finally, RQ4, asked whether there were interaction effects between visitor status and (a) perceptions of moderation, (b) perceptions of anonymity, or (c) connectedness. The interaction effects for visitor status and perceived moderation (B = −0.06, SE = 0.06, p = 0.30; Table 2, Model 3) and visitor status and perceived anonymity (B = 0.01, SE = 0.05, p = 0.88; Table 2, Model 3) were not significant. Perceived moderation and perceived anonymity predicted similar levels of perceived incivility no matter whether a participant was a commenter or comment reader. The interaction between visitor status and connectedness was significant (B = 0.07, SE = 0.02, p < 0.001; Table 2, Model 3). Figure 1 visually displays the interaction effect. For both commenters and comment readers, perceptions of incivility decrease as reported connectedness increases. The difference between commenters and comment readers occurs at lower levels of connectedness, where comment readers report higher levels of incivility compared to commenters. The difference between the groups is small, however. Commenters who rated connectedness at 1 reported perceptions of incivility at 4.23, whereas readers who rated connectedness at 1 reported perceptions of incivility at 4.45, a difference of less than 0.20 out of the range of 5. Furthermore, at higher levels of connectedness, readers and commenters do not differ in their perceptions of incivility on the site. Overall, there is slight evidence that commenters and readers perceive incivility differently based on the connectedness they feel with others on the site, but there is no evidence that commenters and readers perceive incivility differently based on perceptions of moderation and anonymity.

Figure 1.

Plotting differences in perceptions of incivility based on connectedness and visitor status.

Discussion

This study explored the relationships between comment section features, comment section connection, and perceived incivility. When commenters and comment readers perceived that the newsroom moderated comments, they reported lower perceptions of incivility; when they perceived that the newsroom allowed anonymity, they reported higher perceptions of incivility. Most substantially, when commenters and comment readers felt connected to others in the comment section, they perceived less incivility.

Across 20 diverse news comment sections, features of news sites and comment sections – or, more accurately, the belief that certain features and feelings of connection exist in those spaces – were related to incivility perceptions. That we find substantive relationships between perceived incivility and site features and perceived incivility and feelings of connection to other site users supports the value in studying incivility perceptions in addition to researcher-defined uncivil content. We expand previous research, which has focused on individual differences driving perceptions of incivility (Kenski et al., 2017; Mutz, 2015), by pointing to the environmental factors at play in facilitating civil discussion.

We found that a newsroom practice – perception that staff moderated comments – was related to lower levels of incivility perceptions. Alternatively, a user feature – perception of anonymity – was related to an increase in incivility perceptions. These findings support previous studies that link journalistic norms with less incivility (Meltzer, 2015), but with a focus on user perceptions of these norms. In addition, the results add clarity to a comment section conundrum: is anonymity related to more (Santana, 2014) or less incivility (Rains et al., 2017; Spears and Lea, 1994)? While the current study does not uncover trends in actual incivility, the results suggest that people link anonymity and incivility in comment sections they frequent, which could be a reason for the inconclusive findings of previous studies. It may be that online anonymity has become stigmatized as a precedent to incivility online. Issues of cyberbullying and incivility have been attributed to online anonymity (Isaacson, 2016), and the public may pick up on these cues. It is also possible that online anonymity in fact causes incivility, which in turn affects perceptions (Santana, 2014).

Notably, we find that comment section features alone do not influence perceptions of incivility. Even if a newsroom reported moderating comments or allowing anonymity, the mere existence of those features did not relate to incivility ratings; yet thinking these features were available did. In addition, the non-significant interaction effects indicate that people who were accurate in their perceptions about site features perceived incivility in the comment section no differently than people who were inaccurate in their perceptions. If a site moderated comments or allowed anonymous comments, a user was somewhat more likely to perceive that it did so (see, for statistics, Supplemental Online Appendix F). Still, there were a number of individual characteristics, like whether a site visitor was older, male, or a commenter, that better explained whether the visitor noticed features that were actually available. Together, these findings offer evidence for two relationships. One, that incivility perceptions relate most to perceptions of site features, not actual site features. Such a result supports Nagy and Neff’s (2015) concept of an imagined affordance – it is not the technology alone that influences users, but also the way in which users perceive and engage with that technology. This means that while moderation and anonymity may help improve actual incivility in the comment section, it may not affect perceptions of incivility unless the user perceives that these features are available. Two, that site visitors use information from the site to inform their beliefs about the comment section, but perceptions of the site’s features can also vary depending on individuals and, perhaps, on how commenting policies are communicated. In this study, of the three newsrooms that moderated comments, two did not explicitly mention on the site that comments were ‘moderated’. As a result, how newsrooms communicate their commenting policy, and how different individuals interpret the policy, may be just as important as if they communicate it.

Beyond structural features, these results strongly suggest that feelings of connection among commenters and comment readers most strongly relate to fewer incivility perceptions. Even ‘lurkers’ who read but do not post comments (Blanchard and Markus, 2004; Springer et al., 2015) perceive less incivility when they feel connected to commenters on the site. This finding is further supported when we replace the dichotomous commenter/reader variable with a frequency: the overall negative relationship between connectedness and perceived incivility held and the level-one variance explained was nearly identical no matter if the variable was coded as dichotomous (commenter vs comment reader) or as a measure of frequency (see Supplemental Online Appendix C). In sum, all site visitors in our sample, commenter or comment reader, perceived less incivility when they felt more connected to others.

Although comment readers and commenters generally reported similar levels of incivility perceptions, we did find that comment readers who reported being most disconnected perceived more incivility than commenters who reported similar levels of disconnectedness. It is possible that commenters may perceive less incivility because the act of expressing oneself leads to greater cognitive involvement in the conversation (Nekmat, 2012), regardless of how disconnected one feels. Commenters may also gauge the civility of the site before participating and are in turn more likely to engage in comment sections they perceive as more civil. Given that the negative relationship between connectedness and incivility perceptions was also present for comment readers, however, it appears that connectedness also influences comment readers. It is possible that whether the incivility is directed toward one’s ingroup or outgroup produces different levels of connectedness among commenters and comment readers alike (see Hutchens et al., 2019). While this study cannot ascertain how perceptions of incivility and connectedness intersect with group identification, this will be an important avenue for future research.

More generally, these results should encourage researchers to focus on connectedness and perceptions of incivility. Although research has long emphasized the importance of connection in digital spaces (Norris, 2002), sometimes even in news comment sections (Chen et al., 2011; Konnokovs, 2013; Mitchelstein, 2011), little research has examined the links commenters and comment readers build in comment sections and how these links affect discursive norms. This study makes clear that a sense of connectedness has a more robust relationship with perceptions of incivility than the structural features and norms they produce. Although discussion of technology’s role in incivility is an important one, there should also be attention paid to how connected commenters and comment readers feel with one another.

This study has a number of limitations. First, we use cross-sectional survey data that are neither nationally representative nor able to uncover a causal direction. This offers several avenues for future research, particularly for experimental research that can ascertain causal order. For instance, does connectedness prompt more civil discussion (Postmes et al., 1998) or does civil discussion prompt more connectedness (Smith and Bressler, 2013)? Are commenters and comment readers’ perceptions of incivility informed by features of the site or do their perceptions of incivility lead them to make a post hoc assessment of what features are available? We do not verify the temporal order, but we do illuminate the relationships between connection, perceptions of site features, and incivility perceptions. Second, our approach also limits the generalizability of our results to the US population. Nevertheless, surveying actual commenters and comment readers on a site where they engage with news offers an understanding of the thoughts and attitudes of the people likely to be affected by comment sections (Nah and Chung, 2012). As with any survey, many of our variables were related to each other, and a number of the control variables were related to the outcome of perceived incivility. Still, the hypothesized relationships among connectedness, anonymity, and moderation occurred in the regression models above and beyond a variety of control variables, suggesting that the results are robust.

Finally, the survey needed to be as concise as possible because respondents completed the study on a visit to a news site. Therefore, each of our measures involved only one item. Although these items were derived from previous research concerning incivility and connectedness, future research should explore these relationships using multidimensional scales that can assess with greater stability the variance between and within these variables. Furthermore, we did not compare perceptions of incivility to actual incivility present in comment sections because we do not have data tracking what comments participants read on each site. Although we have reason to believe that the perception of incivility may matter just as much and if not more to norms and online behavior (Rains, 2007), we cannot compare perceptions of incivility to actualities in this study.

Conclusion

We find that commenters and comment readers perceive different amounts of incivility in varied digital news spaces and that there is a strong negative relationship between connectedness and perceptions of incivility. This is important given that incivility in the comment section can hurt the quality of public discourse (Chen and Lu, 2017) and perceptions of the news organization and content (Anderson et al., 2014). Newsrooms can take immediate action on these findings by working to build stronger connections among those who visit the comments on their sites, as well as publicizing features the newsroom uses to maintain civil norms in their comment spaces. These results should also encourage scholars to further explore perceptions of incivility, especially how commenter and comment reader perceptions of the site and visitors’ feelings of connection to the comment section can attenuate such perceptions.

Supplemental Material

Supplementary_Appendices – Supplemental material for Predicting perceptions of incivility across 20 news comment sections

Supplemental material, Supplementary_Appendices for Predicting perceptions of incivility across 20 news comment sections by Emily Van Duyn and Ashley Muddiman in Journalism

Footnotes

Acknowledgements

This study was conducted with the Center for Media Engagement. The authors would like to thank Natalie Jomini Stroud for her support.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank The Coral Project for funding this study.

ORCID iD

Emily Van Duyn

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Emily Van Duyn (PhD, University of Texas at Austin) is a Postdoctoral Fellow with the Program on Democracy and the Internet at Stanford University. Her research explores the contexts in which people receive information and talk (or do not talk) about politics.

Ashley Muddiman (PhD, University of Texas at Austin) is an Assistant Professor in the Communication Studies Department at the University of Kansas and a Faculty Research Associate with the Center for Media Engagement. Her research focuses on digital news and political incivility.

References

Aarts

Dijksterhuis

(2003) The silence of the library: Environment, situational norm, and social behavior. Journal of Personality and Social Psychology 84(1): 18–28.

Anderson

Brossard

Scheufele

, et al. (2014) The ‘nasty effect’: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication 19(3): 373–387.

Anderson

Yeo

Brossard

, et al. (2016) Toxic talk: How online incivility can undermine perceptions of media. International Journal of Public Opinion Research 30(1): 156–168.

Anderson

(2006) Imagined Communities: Reflections on the Origin and Spread of Nationalism. London: Verso Books.

Barnes

(2016) The ‘imagined community’ of readers of hyperlocal news: A case study of Baristanet. Journal of Alternative and Community Media 1: 84–96.

Baumeister

(1982) A self-presentational view of social phenomena. Psychological Bulletin 91(1): 3–26.

Ben-Porath

(2008) Codes of professionalism; norms of conversation: How political interviews shape public attitudes toward journalists. PhD Dissertation, University of Pennsylvania, Philadelphia, PA.

Blanchard

Markus

(2004) The experienced sense of a virtual community: Characteristics and processes. ACM SIGMIS Database: The DATABASE for Advances in Information Systems 35(1): 64–79.

Chen

(2017) Online Incivility and Public Debate: Nasty Talk. Berlin: Springer.

10.

Chen

(2017) Online political discourse: Exploring differences in effects of civil and uncivil disagreement in news website comments. Journal of Broadcasting & Electronic Media 61(1): 108–125.

11.

Chen

Chock

Gozigian

, et al. (2011) Personalizing news websites attracts young readers. Newspaper Research Journal 32(4): 22–38.

12.

Coe

Kenski

Rains

(2014) Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication 64(4): 658–679.

13.

Coles

West

(2016) Weaving the internet together: Imagined communities in newspaper comment threads. Computers in Human Behavior 60: 44–53.

14.

Diakopoulos

Naaman

(2011) Towards quality discourse in online news comments. In: Proceedings of the ACM 2011 conference on computer supported cooperative work, Mach, pp. 133–142. Available at: https://www.scholars.northwestern.edu/en/publications/towards-quality-discourse-in-online-news-comments

15.

Freelon

(2015) Discourse architecture, ideology, and democratic norms in online political discussion. New Media & Society 17(5): 772–791.

16.

Grieve

Indian

Witteveen

, et al. (2013) Face-to-face or Facebook: Can social connectedness be derived online? Computers in Human Behavior 29(3): 604–609.

17.

Hopp

Santana

Barker

(2018) Who finds value in news comment communities? An analysis of the influence of individual user, perceived news site quality, and site type factors. Telematics and Informatics 35(5): 1237–1248.

18.

Hutchens

Silva

Hmielowski

, et al. (2019) What’s in a username? Civility, group identification, and norms. Journal of Information Technology & Politics 16(3): 203–218.

19.

Isaacson

(2016) How to fix the internet: Anonymity has poisoned online life. The Atlantic, 15 December. Available at: https://www.theatlantic.com/technology/archive/2016/12/how-to-fix-the-internet/510797/

20.

Kenski

Coe

Rains

(2017) Perceptions of uncivil discourse online: An examination of types and predictors. Communication Research. Epub ahead of print 3 April. DOI: 10.1177/0093650217699933.

21.

Kim

Shin

Cho

, et al. (2019) SNS dependency and community engagement in urban neighborhoods: The moderating role of integrated connectedness to a community storytelling network. Communication Research 46(1): 7–32.

22.

Konnokovs

(2013) The psychology of online comments. The New Yorker, 23 October. Available at: https://www.newyorker.com/tech/annals-of-technology/the-psychology-of-online-comments

23.

Krebs

Lischka

(2019) Is audience engagement worth the buzz? The value of audience engagement, comment reading, and content for online news brands. Journalism 20(6): 714–732.

24.

Ksiazek

(2015) Civil interactivity: How news organizations’ commenting policies explain civility and hostility in user comments. Journal of Broadcasting & Electronic Media 59(4): 556–573.

25.

Chan

(2014) Knowledge sharing and social media: Altruism, perceived online attachment motivation, and perceived online relationship commitment. Computers in Human Behavior 39: 51–58.

26.

Meltzer

(2015) Journalistic concern about uncivil political talk in digital news media: Responsibility, credibility, and academic influence. The International Journal of Press/Politics 20(1): 85–107.

27.

Mikal

Rice

Kent

, et al. (2016) 100 million strong: A case study of group identification and deindividuation on Imgur.com. New Media & Society 18(11): 2485–2506.

28.

Mitchelstein

(2011) Catharsis and community: Divergent motivations for audience participation in online newspapers and blogs. International Journal of Communication 5: 2014–2034.

29.

Muddiman

Stroud

(2017) News values, cognitive biases, and partisan incivility in comment sections. Journal of Communication 67(4): 586–609.

30.

Muddiman

Pond-Cobb

Matson

(2017) Article negativity bias or backlash: Interaction with civil and uncivil online political news content. Communication Research. Epub ahead of print 6 January. DOI: 10.1177/0093650216685625.

31.

Mutz

(2015) In-Your-Face Politics: The Consequences of Uncivil Media. Princeton, NJ: Princeton University Press.

32.

Mutz

Reeves

(2005) The new videomalaise: Effects of televised incivility on political trust. American Political Science Review 99(1): 1–15.

33.

Nagy

Neff

(2015) Imagined affordance: Reconstructing a keyword for communication theory. Social Media + Society 1(2): 1–9.

34.

Nah

Chung

(2012) When citizens meet both professional and citizen journalists: Social trust, media credibility, and perceived journalistic roles among online community news readers. Journalism 13(6): 714–730.

35.

Nekmat

(2012) Message expression effects in online social communication. Journal of Broadcasting & Electronic Media 56(2): 203–224.

36.

Neubaum

Krämer

(2017) Monitoring the opinion of the crowd: Psychological mechanisms underlying public opinion perceptions on social media. Media Psychology 20(3): 502–531.

37.

Norris

(2002) The bridging and bonding role of online communities. The International Journal of Press/Politics 7(3): 3–13.

38.

Papacharissi

(2004) Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society 6(2): 259–283.

39.

Park

Sachar

Diakopoulos

, et al. (2016) Supporting comment moderators in identifying high quality online news comments. In: Proceedings of the 2016 CHI conferences on human factors in computing systems, pp. 1114–1125. New York: ACM. Available at: https://dl.acm.org/doi/10.1145/2858036.2858389

40.

Postmes

Spears

Lea

(1998) Breaching or building social boundaries? SIDE-effects of computer-mediated communication. Communication Research 25(6): 689–715.

41.

Rains

(2007) The impact of anonymity on perceptions of source credibility and influence in computer-mediated group communication: A test of two competing hypotheses. Communication Research 34(1): 100–125.

42.

Rains

Kenski

Coe

, et al. (2017) Incivility and political identity on the internet: Intergroup factors as predictors of incivility in discussions of news online. Journal of Computer-mediated Communication 22(4): 163–178.

43.

Rösner

Krämer

(2016) Verbal venting in the social web: Effects of anonymity and group norms on aggressive language use in online comments. Social Media + Society 2(3): 1–13.

44.

Rowe

(2015) Civility 2.0: A comparative analysis of incivility in online political discussion. Information, Communication & Society 18(2): 121–138.

45.

Ruiz

Domingo

Micó

, et al. (2011) Public sphere 2.0? The democratic qualities of citizen debates in online newspapers. The International Journal of Press/Politics 16(4): 463–487.

46.

Santana

(2014) Virtuous or vitriolic: The effect of anonymity on civility in online newspaper reader comment boards. Journalism Practice 8(1): 18–33.

47.

Smith

Bressler

(2013) Who taught you to talk like that? The university and online political discourse. Journal of Political Science Education 9(4): 453–473.

48.

Spears

Lea

(1994) Panacea or panopticon? The hidden power in computer-mediated communication. Communication Research 21(4): 427–459.

49.

Springer

Engelmann

Pfaffinger

(2015) User comments: Motives and inhibitors to write and read. Information, Communication & Society 18(7): 798–815.

50.

Stroud

Van Duyn

Peacock

(2016) News commenters and news comment readers. Engaging News Project. Available at: https://mediaengagement.org/research/survey-of-commenters-and-comment-readers/

51.

Sukumaran

Vezich

McHugh

, et al. (2011) Normative influences on thoughtful online participation. In: Proceedings of the 2011 annual conference on human factors in computing systems – CHI’: 11, 7–12 May, pp. 3401–3410. New York: ACM. Available at: https://doi.org/10.1145/1978942.1979450

52.

Suler

(2004) The online disinhibition effect. Cyberpsychology & Behavior 7(3): 321–326.

53.

Van Duyn

Peacock

Stroud

(2019) The gender gap in online news comment sections. Social Science Computer Review. Epub ahead of print 26 July. DOI: 10.1177/0894439319864876.

54.

Ziegele

Breiner

Quiring

(2014) What creates interactivity in online news discussions? An exploratory analysis of discussion factors in user comments on news items. Journal of Communication 64(6): 1111–1138.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.71 MB