Abstract
Online phenomena like echo chambers and polarization are believed to be driven by humans’ penchant to selectively expose themselves to attitudinally congenial content. However, if like-minded content were the only predictor of online behavior, heated debate and flaming on the Internet would hardly occur. Research has overlooked how online behavior changes when people are given an opportunity to reply to dissenters. Three experiments (total N = 320; convenience student samples from Germany) and an internal meta-analysis show that in a discussion-forum setting where participants can reply to earlier comments larger cognitive conflict between participant attitude and comment attitude predicts higher likelihood to respond (uncongeniality bias). When the discussion climate was friendly (vs. oppositional) to the views of participants, the uncongeniality bias was more pronounced and was also associated with attitude polarization. These results suggest that belief polarization on social media may not only be driven by congeniality but also by conflict.
Keywords
Introduction
Current Western societies are plagued by a number of toxic phenomena, many of which are fueled via online venues: the emergence of “echo chambers” (Sunstein, 2007), the dissemination of fake news (Pennycook & Rand, 2021), or increasing polarization (Iyengar et al., 2019), all of which might lead to a fragmentation of society. With some exceptions, the current consensus among scholars about the assumptions that might underlie this wide range of phenomena could be dubbed a “congeniality narrative.” According to this narrative, many toxic online phenomena occur because individuals prefer information that aligns with their attitudes, and online environments are designed in ways that maximally cater to this preference for congeniality (Pariser, 2011). The hallmark of the congeniality narrative is the notion of selective exposure (Hart et al., 2009; Knobloch-Westerwick, 2014). Rooted in the theory of cognitive dissonance (Festinger, 1957), selective-exposure studies provide participants with the opportunity to select among different pieces of information and typically show that attitudinally congenial information is preferred over uncongenial information (congeniality bias; Hart et al., 2009).
Although there is little doubt that the congeniality narrative can account for many forms of online behavior, we argue in the present article that some common online phenomena are difficult to explain by an overarching preference for like-mindedness. For instance, evidence suggests that citizens often quite vehemently argue, engage in emotionally charged debates, and express mutual disagreement in online venues—for example, on Twitter or online discussion forums (Hills, 2019; Suler, 2004). Interestingly, the basic structure of most online discussion forums is equivalent to the setup of most selective-exposure experiments: Individuals are confronted with messages varying in congeniality and have to select which messages to attend to. Applied to online forums, the selective-exposure account would suggest that members attend to congenial comments and preferentially express their agreement with like-minded others. However, the level of debate, conflict, and disagreement in many forums, at least anecdotally, suggests an uncongeniality bias in which readers preferentially select dissenting comments to formulate a retort. But given their structural similarity, why do selective-exposure experiments lead to a congeniality bias whereas online forums might lead to an uncongeniality bias?
The present research is built on the simple assumption that the congeniality bias is confined to settings in which participants can choose only among information sources to read from (selective exposure). However, as soon as an opportunity to interact with others arises, people are motivated to change dissenters. As a consequence, we propose a phenomenon of selective response: People will selectively respond to content that is attitudinally uncongenial to express their disagreement—the congeniality bias will turn into an uncongeniality bias. Selective response and the proposed uncongeniality bias have been already foreshadowed in Festinger’s (1957) theory of cognitive dissonance but have rarely been investigated empirically. Nonetheless, converging evidence supports the notion that selective response and an uncongeniality bias may indeed exist. For instance, there is evidence that uncongenial information is scrutinized more carefully than congenial information (Edwards & Smith, 1996). In line with Mercier (2016), we propose that being confronted with uncongenial content elicits an internal cognitive conflict that may trigger cognitive processes (elaboration; Eveland, 2004) as well as affective processes (aversion; Matz & Wood, 2005). Experiencing this cognitive conflict then motivates attempts to change the dissenter, provided that there is an opportunity to interact with the source of uncongenial information. As a consequence, settings that provide an opportunity to change dissenters (e.g., online discussion forums) are replete with expressions of disagreement and negativity (Zollo et al., 2015).
Statement of Relevance
Many online phenomena, such as attitude polarization or the emergence of echo chambers, rest on the psychological assumption that humans prefer like-minded content and people over counterattitudinal content and people. However, this view does not readily explain the prevalence of heated debate and flaming in social-media settings. We conducted three experiments to show that once social-media users were given an opportunity to interact with others, the preference for like-minded content was eliminated. Rather, users preferentially selected counterattitudinal content for their replies to express their disagreement with others. The tendency to attack dissenting views increased when the overall discussion climate was in favor of a user’s view. This has important implications for understanding social-media phenomena and fighting polarization.
Converging evidence from the literature suggests that not only the congeniality bias but also an uncongeniality bias may polarize attitudes. First, anticipating a discussion with uncongenial views has been linked to political knowledge (Eveland, 2004), and political knowledge has been linked to larger polarization (Taber & Lodge, 2006). Second, scrutiny of uncongenial messages leads to elaboration and the generation of counterarguments (Edwards & Smith, 1996), which in turn goes together with increased polarization (Tesser, 1978). And third, publicly expressing one’s opinion may lead to attitude polarization (Brauer et al., 1995). All of this points to the possibility that exhibiting an uncongeniality bias by selectively responding to uncongenial comments makes individuals more polarized than before.
The uncongeniality bias might be moderated by variables such as the emotionality of comments or the personality of users (see the Supplemental Material available online for a detailed discussion). For present purposes, the opinion climate of a discussion is of prior interest as a moderator. The spiral-of-silence account (Noelle-Neumann, 1974) has proposed that individuals might be afraid to voice their opinions when they believe their views to be minority opinions in society. Because of inconclusive findings in studies that experimentally manipulated discussion climate (e.g., Duncan et al., 2020; Yun & Park, 2011), it is not entirely clear whether being in an oppositional (vs. friendly) climate reduces the overall tendency to reply. However, there is evidence that comments are more assertive in a friendly climate (Nekmat & Gonzenbach, 2013), suggesting that a friendly (vs. oppositional) climate will increase the uncongeniality bias.
Experimentally showing the existence of selective response and an uncongeniality bias helps to broaden our understanding of how engagement with controversial content shapes attitudes and behaviors. Moreover, a link between uncongeniality bias and polarization would have implications for policymaking by raising questions about the effectiveness of depolarization interventions, such as increased exposure to cross-cutting content (van der Meer & Hameleers, 2021) or intergroup contact (Wojcieszak & Warner, 2020).
To investigate the uncongeniality bias, we conducted three online studies that were fashioned after selective-exposure experiments: Participants saw discussion comments of varying congeniality, and they made selections among comments. In a departure from standard selective-exposure experiments, we measured whether participants felt inclined to reply to a comment. The key prediction then was that larger (vs. smaller) cognitive conflict between person and comment would increase the willingness to reply to that comment (uncongeniality bias). Study 1 was focused on finding an uncongeniality bias; Study 2 examined whether it is the affordance of responding to others that turns a congeniality bias into an uncongeniality bias. Finally, Study 3 investigated the role of discussion climate as a moderator of the uncongeniality bias. Studies were reviewed and approved by the local ethics committee.
Study 1
Study 1 was conducted to assess whether an uncongeniality bias exists. We used the standard selective-exposure paradigm by confronting participants with discussion comments that varied in the degree of congeniality. We hypothesized that larger cognitive conflict between person and comment would be associated with a larger willingness to reply (Hypothesis 1). In three exploratory analyses, we (a) coded the content of replies written by participants to ensure that replies were more likely to exhibit disagreement than agreement with a comment; (b) investigated whether the uncongeniality bias is independent from the emotionality of comments; and (c) explored whether the uncongeniality bias is related to personality (agency and communion; Bakan, 1966).
Method
Participants and design
Study 1 was conducted on a convenience sample of university student participants (minimum age of 18 years). The sample was composed of 96 participants (67 female, 29 male; mean age 25.79 years). The study was conducted online, and as compensation participants were given the opportunity to take part in a lottery for vouchers to online bookstores. The study was conducted at a time when detailed power analyses were uncommon (2014), so sample size was only determined to clearly exceed the size of standard selective-exposure experiments reported in the literature. (Hart et al., 2009, reported on 8,000 total participants in 300 independent samples, arriving at N = 27 on average.) No participant was excluded from the analysis.
Study 1 used a within-subjects design in which conflict scores were nested within comments and within participants.
Materials
For our materials, we used the highly controversial topic of alternative medicine. Despite lacking strong evidence for its effectiveness from clinical trials and research (Ernst & Smith, 2018), about 46% of respondents from a 2017 German poll indicated having undergone alternative-medicine treatments (e.g., homeopathy) at least once (Suhr, 2017). German governments have created special regulations for the application of some types of alternative medicine and have exempted some treatments from rigorous clinical testing for approval. Because some health insurance providers have partially covered expenses for alternative-medicine treatments, opponents of alternative medicine have argued that prescriptions of corresponding drugs would come at the expense of the community of solidarity. As a consequence, discussions between proponents and opponents of alternative medicine on the comment sections of German newspapers have been particularly lively and emotionally charged, thus making alternative medicine a highly controversial topic.
Inspired by actual comments from online discussions, we developed a set of 24 fictitious discussion comments on the pros and cons of alternative medicine. The comments were similar in length, and 12 comments argued for alternative medicine and 12 comments against. Moreover, within the pro and the con set of comments, half of the comments were seeded with emotional language to cover high and low emotionality. In two material testing studies, comments were subsequently rated for their valence (N = 153; on a scale from 1 = strongly in favor of alternative medicine to 7 = strongly against alternative medicine) and their emotionality (N = 36; on a scale from 1 = totally non-emotional to 5 = totally emotional). Both valence and emotionality of comments were z-standardized for further analyses. Table 1 provides an example of two discussion comments (translated from German). All comments (original and translated) as well as their prerated average valence and emotionality can be found at https://osf.io/rx3e6/.
Example of Two Discussion Comments (Translated From German)
Measures
In Study 1, attitudes were measured with a single item on a 4-point Likert scale (ranging from 1 = in favor of alternative medicine to 4 = against alternative medicine). In a departure from classical selective-exposure studies, we did not use a dichotomous classification of comments as congenial versus uncongenial. Rather, cognitive conflict between participant and comment was measured continuously as the absolute difference between the original attitude of the person (z-standardized) and the prerated valence of the comment (z-standardized). Willingness to respond was captured through simple yes/no decisions for each comment.
It is conceivable that participants might respond to an uncongenial comment to express their sudden agreement. In order to rule out this possibility, participants who indicated that they would respond to a given comment were requested to write a reply. The valence of all 553 replies composed in Study 1 was subsequently rated by two trained, independent, and blinded coders (on a scale from 1 = strongly in favor of alternative medicine to 5 = strongly against alternative medicine). Agreement between the two independent raters was high, as indicated by an intraclass correlation coefficient analysis (Shrout & Fleiss, 1979), r = .89, 95% confidence interval (CI) = [.87, .91], F(552, 553) = 17.26, p < .001. Subsequently, ratings of one coder (the first author) were used as measures of reply valence (the coding manual can be found at https://osf.io/rx3e6/). Knowing the valence of all replies allowed us to compute so-called disagreement scores (absolute distance between z-standardized comment valence and z-standardized reply valence), an indicator of how much a written reply argued against the comment it responded to.
Study 1 also explored whether the uncongeniality bias is moderated by personality variables, a common approach in research on social media (Liu & Campbell, 2017). To check this, we measured the “Big Two” traits of agency and communion with 10 adjective items each, taken from literature on personality (Gebauer et al., 2013). It is conceivable that because of its adversarial nature the uncongeniality bias is associated with high agency (“getting ahead”), but low communion (“getting along”).
Procedure
Study 1 was conducted online on an in-house platform for conducting experiments. After receiving participant consent, we measured demographic variables, self-assessments on agency and communion, and prior attitudes on alternative medicine. This was followed by a presentation of the discussion comments. Participants were first requested to read through all 24 comments, which were presented on a single web page. The design of the comment display was likened to an online discussion forum. Afterwards, the 24 comments were presented separately and in random order. For each comment, participants were asked: Would you want to reply to this comment? (yes/no). If participants clicked on yes, a new page loaded, with the instruction, “You indicated that you would like to reply to this comment. What would you respond?” and showing a text field where participants could enter their reply. After doing this for 24 trials, participants were thanked and debriefed, and they had the opportunity to enter in the lottery by providing an email address.
Analysis plan
We fitted (general) linear mixed-effects models for each dependent variable separately. The fixed-effects structure was specific to each study. All models included both item and participant as random intercepts (Baayen et al., 2008).
We tested the influence of cognitive conflict and emotion on the likelihood to respond to a comment (binomial response). We fitted two general linear mixed-effects models. Model 1 included the main effect of cognitive conflict as a fixed effect, and Model 2 included both main effects of conflict and emotionality as fixed effects. We further tested whether the respondents’ cognitive conflict predicted their disagreement score (absolute difference between comment valence and reply valence). We fitted a linear mixed-effects model with disagreement score as the dependent variable and conflict score as a fixed effect. For the exploratory analyses on the Big Two personality factors, two additional models—Model 3, which included agency as a fixed effect, and Model 4, which included communion as a fixed effect alongside conflict as well as their respective interactions—were compared to baseline Model 1 separately.
Results
Overall, participants replied to 24.35% (ranging from 0% to 66.67%) of the comments they encountered. First, we tested whether cognitive conflict (distance between participant attitude and comment valence) predicted the likelihood to respond to a comment. We confirmed the expected uncongeniality bias (mixed-effects logistic regression: odds ratio, OR = 1.35, SE = 0.09, 95% CI = [1.19, 1.54], z = 4.51, p < .001; Fig. 1a). Consistent with Hypothesis 1, larger cognitive conflict was associated with higher likelihood of a response. An exploratory analysis testing a second model with both cognitive conflict and emotionality of a comment as predictors indicated that conflict was still predictive of the likelihood to respond (OR = 1.34, SE = 0.09, 95% CI = [1.18, 1.53], z = 4.38, p < .001), whereas emotionality did not predict responses (OR = 1.13, SE = 0.09, 95% CI = [0.96, 1.33], z = 1.43, p = .153; see Table S1 in the Supplemental Material). This finding rules out the possibility that response behavior was driven by the emotional tone of a comment rather than by its degree of uncongeniality. Although these results provide first evidence for the existence of an uncongeniality bias, they do not preclude the (unlikely) possibility that participants replied to uncongenial comments in order to express their sudden agreement with them. To test for this, two independent coders rated the valence of all 553 replies, enabling us to obtain a disagreement score (the absolute difference between original comment valence and subsequent reply valence). Our analysis showed that the cognitive conflict score between participant and original comment predicted the disagreement score between original comment and reply (mixed linear model: b = 0.43, SE = 0.04, 95% CI = [0.35, 0.51], t = 10.75, p < .001; see Fig. 1b; see Table S2 in the Supplemental Material). In other words, this analysis showed that participants tended to respond to uncongenial comments in order to express their disagreement with them.
Results of Study 1. With uncongeniality bias (a), the larger the cognitive conflict (distance between participant attitude and comment valence), the greater the likelihood to respond. Evidence that the uncongeniality bias is driven by disagreement is shown in (b): The larger the cognitive conflict between participant attitude and comment valence, the larger the disagreement score (distance between comment valence and reply valence). Shaded areas represent 89% confidence intervals.
Study 1 also explored the question of whether the likelihood to respond (uncongeniality bias) was dependent on the personality of a reader. However, using measures of the Big Two (Gebauer et al., 2013), we could neither observe an influence of agency or communion on the likelihood to respond in general, nor on the likelihood to respond to uncongenial comments (see Tables S3 and S4 in the Supplemental Material).
Study 2
Beyond replicating the uncongeniality bias, Study 2 aimed to show how a congeniality bias can turn into an uncongeniality bias when an opportunity for interaction and selective response arises. To test for this prediction, we set up two experimental conditions: In a read condition, participants indicated for each discussion comment their willingness to read more about it (a classic selective-exposure instruction). In an otherwise identical write condition, participants were requested to indicate how much they are willing to respond to each comment. For the primary analyses, we expected a congeniality bias in the read condition (Hypothesis 1) and an uncongeniality bias in the write condition (Hypothesis 2). In addition, two exploratory analyses were conducted. First, to ensure that the uncongeniality bias in the write condition was due to disagreement with comments, we had participants in both conditions assess perceived quality of each comment. In line with inoculation theory (Compton, 2013), it is conceivable that participants in the write condition preferentially selected those comments for replies that they deem to be of low quality. Second, we tested whether exposure to comments in the read and write conditions were associated with attitude polarization.
Method
Participants and design
Study 2 was conducted on a convenience student sample of the local university. Because the study was conducted at a time when power analyses were quite uncommon (2015), sample sizes were determined by budgetary considerations only. One hundred eighty-seven participants registered for the experiment, but 41 did not finish it. Applying a threshold for a minimum reading time of 8 s per comment (on average), another 23 participants were excluded. Finally, another five participants were excluded because of missing values in measuring the secondary variable of argument quality. The final sample was N = 118 (58 in the read condition, 60 in the write condition; 93 female, 25 male, mean age = 25.22 years). Again, compensation was based on the opportunity to take part in a lottery for vouchers.
The study employed a design in which participants were randomly assigned to either the read condition or the write condition. However, as the dependent variable was slightly different between the two conditions, we took a conservative approach by analyzing both conditions separately, employing a mixed-model approach with cognitive conflict as a fixed factor and participants and comments as random factors, respectively.
Materials
The discussion comments used for Study 2 were a subset (16 comments) from the material used in Study 1. Half of the comments were in favor of alternative medicine, and half of the comments were arguing against.
Measures
Attitudes on alternative medicine were measured both before and after reading discussion comments, using a single item on a 6-point Likert scale (ranging from 1, against alternative medicine, to 6, in favor of alternative medicine). To capture polarization, we multiplied the difference of the attitude measures after and before the experiment by the sign of the difference of the attitude before the experiment and the neutral scale point (3.5): [sign(before − 3.5) × (after − before)]. Thus, positive polarization scores represented changes toward more extreme attitudes, and negative polarization scores represented changes toward less extreme (or more neutral) attitudes.
As in Study 1, cognitive conflict was measured as the absolute difference between the original attitude of the person with regard to alternative medicine (z-standardized) and the prerated valence of the comment (z-standardized).
In Study 2, we did not require participants to write actual replies. Rather, we measured willingness to engage with single, 4-point Likert-scale items for each comment. In the read condition, participants were asked how much they were willing to read more about a given comment (from 1, not under any circumstances, to 4, in any case). In the write condition, participants were asked how much they were willing to reply to a given comment (from 1, not under any circumstances, to 4, in any case).
To address whether the hypothesized uncongeniality bias in the write condition was due to dissent, participants also had to rate the perceived quality of each comment (on 7-point Likert scales ranging from 1, very low, to 7, very high). Quality ratings were z-standardized for further analyses.
Procedure
The experiment was conducted online via Qualtrics. After participants provided demographic variables and indicated their attitude on alternative medicine, they were shown the 16 discussion comments separately and in random order. For each comment, participants indicated their willingness to engage (willingness to read more in the read condition, willingness to respond in the write condition). After these 16 trials, attitudes on alternative medicine were measured again. Subsequently, participants were shown all 16 comments again on a single page and rated the perceived quality of each item. At the end, participants were thanked, debriefed, given an opportunity to withdraw their data, and given an opportunity to take part in the lottery.
Analysis plan
We fitted (general) linear mixed-effects models for each dependent variable separately. All models included both item and participant as random intercepts. Study 2 tested the influence of cognitive conflict on the likelihood of engaging with a comment. The dependent measures of Study 2 were slightly different for the two conditions. Whereas it was “want to read more” in the read condition, it was “want to reply” in the write condition. Thus, we calculated separate analyses for the two dependent variables. For both analyses, two models were tested: Model 1 included cognitive conflict as a continuous fixed effect, and Model 2 included main effects of conflict, of perceived quality, and of their interaction as fixed effects. For the attitude measures, we submitted attitude-polarization scores to a linear model with condition (read vs. write) as a fixed effect.
Results
Results for the read condition yielded a congeniality bias that was consistent with Hypothesis 1 (b = −0.27, SE = 0.04, 95% CI = [−0.34, −0.20], t = −7.47, p < .001; Fig. 2a; see Table S5 in the Supplemental Material). The willingness to engage with a comment (read more) was reduced with increasing cognitive conflict. A second model explored whether the willingness to engage was also affected by the perceived quality of discussion comments. This analysis revealed that the effect of cognitive conflict on the willingness to engage was eliminated (b = −0.01, SE = 0.08, 95% CI = [−0.17, 0.16], t = −0.07, p = .947), whereas the perceived quality of a comment was positively associated with the willingness to engage (b = 0.25, SE = 0.03, 95% CI = [0.19, 0.31], t = 8.23, p < .001). However, this main effect was qualified by a small interaction effect between cognitive conflict and perceived quality (b = −0.04, SE = 0.02, 95% CI = [−0.08, −0.00], t = −2.02, p = .044; see Table S5 in the Supplemental Material) indicating that participants preferred to read congenial (vs. uncongenial) comments when these comments were perceived as high in quality, whereas no such difference occurred for comments of low perceived quality (Fig. 3a). This finding contributes to the extant selective-exposure literature by showing that the congeniality bias might be restricted to content that is subjectively perceived to be of high quality.
Results of Study 2, showing (a) congeniality bias in the read condition and (b) uncongeniality bias in the write condition. Shaded areas represent 89% confidence intervals.
Relation between cognitive conflict, engagement, and perceived quality. The congeniality bias in the read condition (a) is predicated on high perceived quality; the uncongeniality bias in the write condition (b) is predicated on low perceived quality. Shaded areas represent 89% confidence intervals.
Next, we conducted similar analyses with respect to the write condition. Consistent with Hypothesis 2 and the results of Study 1, we again found an uncongeniality bias indicating that participants were more willing to engage with a comment (i.e., respond) the more this comment elicited cognitive conflict (b = 0.20, SE = 0.03, 95% CI = [0.14, 0.26], t = 6.84, p < .001; Fig. 2b; see Table S6 in the Supplemental Material). Next, we fitted a second model that included cognitive conflict, perceived quality, and the interaction between conflict and quality. This analysis yielded a substantial main effect for cognitive conflict (b = 0.50, SE = 0.07, 95% CI = [0.35, 0.64], t = 6.71, p < .001) and a smaller main effect for quality (b = 0.10, SE = 0.03, 95% CI = [0.04, 0.15], t = 3.33, p = .001), which were qualified by an interaction between conflict and quality (b = −0.08, SE = 0.02, 95% CI = [−0.12, −0.05], t = −4.42, p < .001; see Table S6). This interaction indicates that the tendency to respond to uncongenial comments (uncongeniality bias) was very pronounced for comments of low perceived quality. By contrast, the uncongeniality bias was eliminated for comments of high perceived quality (Fig. 3b).
Finally, we used a single item to explore belief polarization by measuring participants’ attitudes on the controversial topic (alternative medicine) before and after exposure to discussion comments. We created a polarization index that indicates a movement away from the neutral scale point from preexposure to postexposure. The analysis of polarization based on this single item did not show signs of polarization (linear model intercept: b = 0.02, SE = 0.06, 95% CI = [−0.10, 0.13], t = 0.29, p = .771). There was also no difference in polarization between the read and write conditions (b = −0.06, SE = 0.08, 95% CI = [−0.22, 0.09], t = −0.80, p = .422; see Table S7 in the Supplemental Material).
Taken together, these results indicate that one can elicit a congeniality bias as well as an uncongeniality bias, a difference that was only contingent on the way that instructions about engaging with a comment were provided and the way results were measured (reading more vs. replying). This supports the notion that classic selective-exposure effects might not be sufficient to explain behavior in social-media settings where users may talk back to others. Moreover, the analyses of conflict and perceived quality in Study 2 showed that a congeniality bias mainly occurs for comments of high perceived quality, whereas the uncongeniality bias mainly occurs for comments of low perceived quality. However, we could not establish a link between biased selection behavior and belief polarization.
Study 3
In Study 3 (preregistered), we attempted to replicate the uncongeniality bias (Hypothesis 1), but also explored a potential moderator. Inspired by the spiral-of-silence account (Noelle-Neumann, 1974), we manipulated discussion climate in three conditions: the majority of comments were either in line with participants’ attitudes (friendly climate) or running counter to participants’ attitudes (oppositional climate), or comments were balanced as in standard selective-exposure experiments (control). Our preregistered hypotheses were built on the assumption that holding a minority view will dampen the tendency to reply to uncongenial comments (reduced uncongeniality bias; Hypothesis 2). Conversely, we expected that participants in the friendly climate condition are even more inclined to respond to uncongenial comments (increased uncongeniality bias; Hypothesis 3). Primary analyses were complemented by two exploratory investigations. First, to ensure that the uncongeniality bias is driven by disagreement rather than agreement, participants had to express their “like” or “dislike” for each comment (akin to the design of many actual discussion forums). This enabled us to test whether the relation between cognitive conflict and reply behavior is mediated by disagreement (hitting the dislike button). Second, we explored whether exposure and response to comments is associated with attitude polarization.
Method
Participants and design
Basis for the power analysis of the predicted interaction of conditions (control, friendly climate, oppositional climate) and conflict in Study 3 was a mixed-model simulation with data from Study 2. In particular, we specified the parameters θ (0.50, 0.10), β (3.48671, 0, 0, 0.053, −0.12, 0.12), and the scale parameter σ (1). Importantly, the third and fourth beta parameters specify the interaction effect with data from Study 2. The results showed that 180 participants are sufficient to achieve a power of 92.25%. Power-analysis simulations are available at https://osf.io/rx3e6/. Thus, Study 3 was preregistered for 180 participants. With oversampling, 198 students from the local university took part in the study, of whom 180 participants finished the study. We preregistered three exclusion criteria: (a) We excluded 12 participants who had a standard deviation of zero in the original semantic differential scales measuring prior attitudes (50% of items were reverse-coded, and a standard deviation of zero would be indicative of “click-through behavior”); (b) in order to be able to measure polarization, we excluded 5 participants whose prior attitude was completely neutral (5.0 as average across six semantic differential scales ranging from 1 to 9); (c) we excluded participants on the basis of one manipulation-check item in which participants had to indicate whether the comments they saw were largely positive, largely balanced, or largely negative toward alternative medicine. As it turned out, the criteria for this manipulation check lacked precision, thus leading to an exclusion of a further 57 participants and also to different attrition rates (Zhou & Fishbach, 2016). We decided to report the preregistered analyses in the main manuscript, but we also conducted additional robustness checks (see Supplemental Material) with only the first two exclusion criteria. For the preregistered analyses, 106 participants remained (47 in the balanced condition, 39 in the friendly climate condition, and 20 in the oppositional climate condition; 83 females, 20 males, 3 nondisclosed; mean age = 25.55 years).
Study 3 employed a mixed design with condition (balanced, friendly climate, oppositional climate) as a between-subjects variable and cognitive conflict as a within-subjects variable that was nested within discussion comments and within participants.
Materials
All participants saw 12 discussion comments from our original material—six comments in favor of alternative medicine and six comments against in the balanced condition, and three comments in favor of alternative medicine and 9 comments against in the two unbalanced conditions. The two unbalanced conditions were designated as friendly climate or oppositional climate depending on the original attitude of a given participant. The 3:9 ratio reflected a compromise between (a) making sure that participants were able to see that there was a minority view; (b) preventing a too-lopsided distribution for statistical reasons; and (c) being similar to studies about minority influence, which typically vary around 20:80 distributions (e.g., Baker & Petty, 1994).
Measures
Attitudes were measured with 9-point semantic differentials in which participants assessed whether they find alternative medicine bad/good, unreasonable/reasonable, negative/positive, useful/harmful, effective/ineffective, and convincing/unconvincing (the latter half of these items were reverse-coded). Correcting for reverse coding, attitude scores were averaged across these ratings (ranging from 1, against alternative medicine, to 9, in favor of alternative medicine). Finally, we measured attitude polarization using the same procedure as in Study 2, accounting for the different scale midpoint: [sign(before − 5.0) × (after − before)].
Again, cognitive conflict between participant and comment was measured as the absolute difference between the original attitude of the person with regard to alternative medicine (z-standardized) and the prerated valence of the comment (z-standardized).
Response tendencies for each comment were measured with single, 6-point Likert items asking participants how much they are inclined to reply (from 1 = not at all inclined to 6 = fully inclined).
We also explored whether the relation between conflict and response tendency was mediated by dislike. Therefore, participants had to click on a like or a dislike button for each comment that they read.
Procedure
The experiment was conducted online via Qualtrics. After measuring prior attitudes, participants were first required to read 12 discussion comments on a single page. In the balanced condition, participants read six comments against and six comments in favor of alternative medicine. In the other two conditions, participants saw nine comments against and three comments in favor of alternative medicine (with allocations of participants to the friendly vs. oppositional climate condition based on prior attitudes being below or above the scale midpoint). Each comment was associated with a fictitious username and a brief headline. After reading through all 12 forum comments, separate comments were again displayed (in random order). Participants had to click a like or dislike button for each comment and were requested to indicate how much they are inclined to respond to the comment. After the 12 trials, participants’ attitudes on alternative medicine was measured again, followed by the manipulation-check item. After debriefing, we provided the option to withdraw data and take part in the lottery.
Analysis plan
We fitted (general) linear mixed-effects models for each dependent variable separately. All models included both item and participant as random intercepts. In an initial analysis of Study 3, we tested whether the willingness to respond differed among the three conditions. Therefore, we fitted a linear mixed-effects model with condition (balanced, friendly, oppositional) as a fixed effect and submitted the resulting model to a Type III analysis of variance. For the main analysis, we fitted and tested a model that included condition (balanced, friendly, oppositional) and cognitive conflict as fixed effects (main effects and interaction). Mediation between cognitive conflict and response tendency via dislikes was measured by submitting unstandardized indirect effects to 1,000 bootstrapped samples and computing 95% confidence intervals. Attitude-polarization scores were submitted to a linear model with condition (balanced vs. friendly climate vs. oppositional climate) as a fixed effect. The balanced condition served as a baseline.
Results
First, consistent with Hypothesis 1, results again indicated an overall uncongeniality bias (b = 0.18, SE = 0.07, 95% CI = [0.03, 0.33], t = 2.40, p = .017; see Table S8 in the Supplemental Material) showing that larger cognitive conflict was associated with a stronger inclination to respond. Second, we exploratorily investigated the prediction of the spiral-of-silence account according to which individuals are generally less likely to speak out when they feel that they are in a minority. There was no difference in general inclination to respond between participants in an oppositional climate (M = 3.25, SD = 1.60), participants in the balanced condition (M = 3.40, SD = 1.65), and participants in the friendly climate condition (M = 3.19, SD = 1.66), F(2, 106.62) = 0.13, p = .876. This runs counter to the standard spiral-of-silence effect. Third, however, it should be noted that our preregistered hypotheses were built on a more nuanced interpretation of the spiral-of-silence effect, and indeed, interactions between discussion climate and cognitive conflict were partially consistent with our preregistered hypotheses (see Fig. 4 and Table S8). Being in an oppositional climate (vs. the balanced condition) only descriptively reduced the willingness to respond to uncongenial comments (b = −0.30, SE = 0.16, 95% CI = [−0.62, 0.02], t = −1.84, p = .065). On this ground, Hypothesis 2 was not supported. Conversely, in full support of Hypothesis 3, being in a friendly climate (vs. the balanced condition) even increased the willingness to respond to uncongenial comments (b = 0.37, SE = 0.12, 95% CI = [0.14, 0.61], t = 3.15, p = .002). These results indicate that the prevailing climate in a discussion affects subsequent reply behavior: When the discussion climate is friendly, participants are even more inclined to argue against dissenters, whereas the tendency to speak up against counterviews tends to be muted in an oppositional climate. In terms of the spiral-of-silence account, these results suggest that being in a minority did not silence participants but made them less confrontational.
Impact of discussion climate on uncongeniality bias. When compared with the uncongeniality bias that includes balanced exposure to congenial and uncongenial comments (blue line), the uncongeniality bias is exacerbated in a friendly climate (majority of congenial comments; green line), whereas the uncongeniality bias is eliminated in an oppositional climate (majority of uncongenial comments; red line). Shaded areas represent 89% confidence intervals.
To further illuminate the role of disagreement for the occurrence of an uncongeniality bias, we also conducted an exploratory analysis to see whether the effect of cognitive conflict on the willingness to respond is mediated by participants’ likes or dislikes of a forum comment. Although the direct effect of cognitive conflict on willingness to respond was sizable, c = 0.27, SE = 0.06, t(1,270) = 4.87, p < .001, this effect dropped markedly when we accounted for the effect of expressing dislike with a comment, c′ = 0.11, SE = 0.07, t(1,269) = 1.72, p = .087. In other words, cognitive conflict with a comment increased the likelihood of pressing the dislike button, which in turn increased the likelihood of responding to the comment, yielding a full mediation effect (Fig. 5).
Mediation of the uncongeniality bias (effect of cognitive conflict on response) via choosing the dislike button.
In a final exploratory analysis of Study 3, we investigated whether being embedded in discussions with different climates has an impact on attitudes. Similar to Study 2, we computed polarization scores that indicate whether attitudes after exposure to the comments were more extreme than initial attitudes. Again, we found no evidence that polarization occurred in the baseline condition (intercept: b = 0.04, 95% CI = [−0.14, 0.21], p = .692). However, polarization scores were somewhat stronger in a friendly climate compared with the baseline (b = 0.26, 95% CI = [0.00, 0.53], p = .049), whereas no difference between baseline and oppositional climate could be found (b = −0.21, 95% CI = [−0.54, 0.11], p = .183; see Table S9 in the Supplemental Material).
As the preregistered analyses had power issues (relatively large dropout due to imprecise exclusion criteria; see the Participants and Design section), we repeated all analyses by applying only two out of three exclusion criteria as a robustness check. Results are in full support of all three preregistered hypotheses (see Table S10 in the Supplemental Material), and yielded effects similar to the preregistered analyses for the mediation analysis (Fig. S2) and polarization data (Table S11).
Taken together, Study 3 showed that an uncongeniality bias is increased when people believe they hold a majority view, but that an uncongeniality bias is decreased—without a decrease in general likelihood to respond—when people hold a minority view. We could further show that the uncongeniality bias is mediated by dislike toward conflicting comments. Finally, there is some evidence that being in a friendly discussion climate can be linked to polarization. These results were also confirmed in our robustness checks of data that included only two of the three exclusion criteria.
Further Checks and Internal Meta-Analysis
To further substantiate our findings, we conducted a number of additional exploratory analyses. First, we conducted the primary analyses of all three studies using models with both random intercepts and random slopes. These analyses yielded similar effects as the preregistered models with random intercepts only, hinting at the robustness of the data (for details, see Tables S12–15 in the Supplemental Material). Second, we conducted an internal meta-analysis on those conditions of the three studies in which we expected a standard uncongeniality bias to occur (i.e., full Study 1; the write condition of Study 2; and the balanced condition of Study 3). Using a fixed-effects structure and weighting over the number of observations, we found a small summary effect over the three studies (d = 0.164, SE = 0.016, 95% CI = [0.132, 0.195], p < .001). A random-effects model yielded similar effect sizes (d = 0.159, SE = 0.028, 95% CI = [0.103, 0.214, p < .001). As the three conditions were very similar in setup and material, we favored the fixed-effects model, and indeed further analyses indicated that the three conditions were homogeneous (Q = 4.71, p = .095). A graphical depiction of the fixed-effects model (with effect sizes and confidence intervals for the individual study conditions as well as the overall effects) is provided in Figure 6.
Results of a fixed-effects internal meta-analysis on the three conditions in which a standard uncongeniality bias was expected to occur (full Study 1; write condition of Study 2; balanced condition of Study 3). CI = confidence interval; FE = fixed effects.
Discussion
Using a variant of the well-known selective-exposure paradigm, we provide empirical evidence that some types of online behavior (particularly online discussions) may be driven by uncongeniality rather than congeniality. Our findings on selective response contribute to a literature indicating that the classical congeniality narrative cannot account well for how individuals deal with conflicting information. In contradiction to the congeniality narrative, people frequently turn to uncongenial information (Garrett & Stroud, 2014) or scrutinize it even more than congenial information (Taber & Lodge, 2006). Moreover, field studies have demonstrated that unintended exposure to cross-cutting content is frequent (Mutz & Mondak, 2006), that many users prefer a balanced news diet (Fletcher & Nielsen, 2017), and that some even prefer uncongeniality (Heltzel & Laurin, 2021).
Our studies provide evidence that expressing one’s views in response to uncongenial information may strengthen one’s attitudes, thus leading to polarization. This adds to literature showing that uncongenial exposure (Bail et al., 2018), repeated expression of attitudes (Brauer et al., 1995), and discussions with dissenters (Lee et al., 2014) increase attitude polarization. If the uncongeniality bias indeed fuels polarization, increasing the amount of cross-cutting dialogue between divergent stakeholders (Wojcieszak & Warner, 2020) may not be an effective remedy for polarization.
It should be emphasized that our reasoning about selective response and an uncongeniality bias is only beginning to emerge. With that being said, future work should address the limitations of our present studies. First, we used nonrepresentative student convenience samples in all three studies (also with a gender imbalance in favor of females). Although this is not uncommon for these types of experiments, it raises the question of whether the effects can be generalized to or replicated in different samples. Second, we always used discussions on the same controversial topic (pros and cons of alternative medicine), so further research should investigate whether the effects can be replicated for different controversial (or for less controversial) topics. Third, some design issues could be improved (potential demand effects in attitude measurement, potential self-effects by jointly rating likes/dislikes and willingness to respond in Study 3, limited effects of polarization via single exposure). The stability and similarity of effects for three studies using different approaches, however, makes us hopeful that these issues are not fundamental. Fourth and most crucially, by modern standards the studies had some power issues. We addressed this shortcoming with an internal meta-analysis. Moreover, an ongoing analysis of uncongeniality biases in a real discussion forum makes us confident that the effect is not only existent, but even much stronger than shown in the present studies.
Our data suggest that the congeniality narrative cannot account for all online phenomena, particularly not those that entail interaction on controversial topics. Although users of social media might exhibit a preference for congenial content, this may quickly turn into a preference for uncongenial content once an opportunity to interact with others arises, suggesting that people are not as conflict-averse as large parts of the literature have proposed.
Supplemental Material
sj-docx-1-pss-10.1177_09567976231194590 – Supplemental material for Online Interaction Turns the Congeniality Bias Into an Uncongeniality Bias
Supplemental material, sj-docx-1-pss-10.1177_09567976231194590 for Online Interaction Turns the Congeniality Bias Into an Uncongeniality Bias by Jürgen Buder, Anja Zimmermann, Brett Buttliere, Lisa Rabl, Moritz Vogel and Markus Huff in Psychological Science
Footnotes
Transparency
Action Editor: Lasana Harris
Editor: Patricia J. Bauer
Author Contributions
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
