Engagement in subversive online activity predicts susceptibility to persuasion by far-right extremist propaganda

Abstract

Despite the widespread assumption that online misbehavior affects outcomes related to political extremism, few studies have provided empirical evidence to this effect. To redress this gap, we performed two studies in which we explored the relationship between subversive online activities and susceptibility to persuasion by far-right extremist propaganda. Study 1 (N = 404) demonstrates that when individuals are exposed to far-right “scientific racism” propaganda, subversive online activity is significantly associated with feelings of gratification, attribution of credibility to and intention to support the propaganda’s source, as well as decreased resistance (in the form of reactance) to the propaganda. To verify these findings across thematic domains, Study 2 (N = 396) focused on far-right extremist propaganda that advocates “male supremacy.” Results in Study 2 replicated those from Study 1. These findings have implications for understanding subversive online activity, vis-à-vis its association with one’s susceptibility to persuasion by far-right extremist propaganda.

Keywords

Internet male supremacy propaganda race science reactance right-wing extremism source credibility subversive online activity (SOA)

As long as online communication channels have been subject to empirical investigation, researchers have sought to identify the supposed link between online activity and negative offline outcomes (see Scrivens et al., 2020), often through the isolation and identification of online indicators that foretell one’s likelihood of adopting viewpoints that support ideological violence (e.g. Grover and Mark, 2019). This pursuit has become central to the study of violent extremism, as analyses of terrorists’ motivations show radicalization processes to be increasingly occurring online (see Gaudette et al., 2020). Despite the importance of understanding how online activity affects far-right extremism, few studies have provided systematic evidence linking them. At best, evidence in support of the association has been anecdotal, leaving a gap in our understanding of the factors that contribute to online radicalization processes. The current study seeks to redress this gap through an investigation of problematic online behaviors and how they affect outcomes related to persuasion by far-right extremist propaganda.

Subversive online activities

For the purposes of the current study, subversive online activities (SOAs) can be classified into one of two categories: behaviors that are meant to abuse and harass others and the use of online subcultural platforms on which problematic activity occurs.

For the first category, subversiveness is defined by the behavior’s intent and outcome. One such behavior is known as doxing. Most definitions describe doxing as the intentional public release of another individual’s personal information by a third party. This release is typically done to humiliate, threaten, intimidate, or punish the target of the doxing attack (Douglas, 2016). As with doxing, definitions of trolling vary widely across the literature (see Ortiz, 2020). For these studies, we follow Craker and March (2016), who conceptualize trolling as “a form of online bullying and harassment . . . [that] includes starting aggressive arguments and posting inflammatory malicious messages . . . to deliberately provoke, disrupt, and upset others” (p. 74).

Whereas doxing and trolling are considered subversive based on their intended outcomes, the second category of online behaviors can be considered subversive because of the nature of the online platforms and applications on which they occur. These platforms and applications feature affordances that are intentionally or incidentally tailored to an extremist user base. This so-called alt-tech is a collection of platforms that mimic popular social media applications to provide an online social space in which racism, misogyny, and violent ideation are tolerated, and sometimes encouraged (Conway, 2020). Users of these platforms—including notable examples like Gab, Parler, and Gettr—often champion them as bastions of free speech while supporting engagement in antisocial behavior, up to and including the planning of violence against perceived enemies (Ebner, 2019).

There is some overlap between the use of alt-tech and the broader ecosystem of encrypted communication applications and/or anonymized applications that can be used to facilitate deviant behavior. End-to-end encryption scrambles messages such that they can be deciphered only by the sender and the intended recipient, avoiding translation by even law enforcement or the platform itself (Perlroth, 2019). These platforms also facilitate the maintenance of anonymity, which can be extremely difficult to disrupt (Gehl, 2018).

Of course, the use of applications that allow for data encryption or anonymization is not a de facto indication of extremist intent. But, applications that facilitate encrypted communication and/or anonymity allow users to circumvent incrimination if their messages are attributed to them, making them popular among those that espouse extremist rhetoric or plan violence (e.g. Walther and McCoy, 2021).

Although we conceive of the categories of behaviors as qualitatively distinct, they are related in their mutual and overlapping connection to far-right extremism (see Hodge and Hallgrimsdottir, 2020). Given the links between doxing, trolling, use of alt-tech, and use of applications that encrypt communications or anonymize users with one’s proclivity for far-right radicalization, this study seeks to empirically explore their collective association with persuasion by far-right extremist propaganda.

Far-right extremist propaganda

We characterize the far right based on its adherence to “antidemocratic practices and ideals, exclusionary beliefs, existential threats and conspiracies, and apocalyptic fantasies”, as well as its “strategies of violence and terrorism . . . intense nationalism, and/or support for criminal action” (Blee and Creasap, 2010: 270), enforced through “emphasis on hierarchical authority” (Mudde, 2007: 21). These core values are reflected in the two narrower discourses examined in our study: scientific racism and male supremacy. Although these two subtypes do not provide an exhaustive account of the far-right extremist propaganda landscape, we chose to focus on them because (a) they exemplify the far right as we define it in the current study and (b) race science and male supremacy propaganda are pervasive in subversive online spaces.

Scientific racism

Scientific racism refers to a pseudoscience characterized by the hierarchical categorization of races based on assumed biological differences. The origins of race science are based in taxonomies produced by Enlightenment-era scholars that sought to reconceptualize theories of human difference to better suit white populations in the age of colonization and scientific revolution (Sussman, 2014). This idea persisted throughout the 18th, 19th, and 20th centuries to justify the brutalization, conquest, and murder of non-white populations (Jenkins and Leroy, 2021). Contemporary scientific racism continues to leverage questionable practices to rationalize the use of damaging stereotypes, hierarchies, and taxonomies (e.g. Jackson and Depew, 2017).

The online spaces where these messages circulate are populated with individuals who engage in the SOAs described above (e.g. Hodge and Hallgrimsdottir, 2020; Ortiz, 2020). To operationalize the degree to which individuals that engage in SOAs are persuadable by far-right extremist propaganda consistent with scientific racism, we seek to measure the degree to which SOA is associated with gratification, attribution of credibility to the source of the propaganda, psychological reactance (a measure of resistance to a persuasive message), and intent to support the source of the propaganda. In general, we predict that SOA would be positively related to the degree to which one is persuadable by race science propaganda.

H1: SOA is positively related to feelings of gratification in response to propaganda promoting scientific racism.

H2: SOA is positively related to attribution of credibility to the source of propaganda promoting scientific racism.

H3: SOA is negatively related to psychological reactance in response to propaganda promoting scientific racism.

H3(a): SOA is negatively related to anger in response to propaganda promoting scientific racism.

H3(b): SOA is negatively related to counter-arguing against propaganda promoting scientific racism.

H4: SOA is positively related to the intention to support the source of propaganda promoting scientific racism.

Male supremacy

Gender refers to the social, cultural, and historical attributes assigned to individuals, traditionally based on the biological sex they express and/or embody. The concepts of masculinity and femininity have historically been conflated with biological sex and sexual orientation, but since the mid-1900s, gender and identity researchers have sought to uncouple these concepts (Barker, 2016). As a result, many experts have come to understand gender as a culturally bound social and psychological construct that can be (but is not necessarily) linked with an individual’s biological sex (Smiler, 2004).

Deriving from the concept of gender is the concept of masculinity, which has been defined as the drive to develop or express traits that are stereotypically characteristic of biological males (Pleck, 1987), including psychological and behavioral tendencies that demonstrate ambition, an aversion to showing emotion, and a willingness to use violence (Brannon, 1976). Pervasive assumptions about masculinity have been thrown into doubt since the advent of second-wave feminism in the 1960s–1970s, which contended that neither women nor men benefit from traditional gender roles (Messner, 2016).

In opposition to some second-wave feminists, emergent “men’s rights activists” (MRAs) contended that both men and women were oppressed by sex roles, but men did not benefit from these roles at the expense of women (Farrell, 1974). As feminist ideas have grown more mainstream in the late 20th and early 21st centuries, MRAs have shifted their concerns to issues related to dating, sex, relationships, and the growing normative acceptance of homosexuality (Mountford, 2018).

The online spaces in which these viewpoints circulate today are informally referred to as the “manosphere.” The manosphere is populated by digital communities stylized as Pick-Up Artists (men who seek short-term sexual relationships with multiple women), involuntary celibates (individuals who lament their inability to attract sexual partners), Men Going Their Own Way (men who seek to break from a feminist-led society), and those who have taken the “Red Pill” (individuals who have “awoken” to the “fact” that women dictate how the world works). These ideas often foster anger toward women and promote violent behavior to maintain masculine dominance (Hoffman et al., 2020).

For this study, male supremacy refers to an ideology that combines the pursuit of the historical masculine archetype with traditional gender hierarchies, exclusionary beliefs, and the threat of violence against women. Given extant data demonstrating the link between SOAs and proclivity for encouraging gender-focused violence (Kavanagh and Brown, 2020), we predict that SOA will positively relate to persuadability by far-right male supremacy propaganda.

H5: SOA is positively related to feelings of gratification in response to propaganda promoting male supremacy.

H6: SOA is positively related to attribution of source credibility to the source of propaganda promoting male supremacy.

H7: SOA is negatively related to psychological reactance in response to propaganda promoting male supremacy.

H7(a): SOA is negatively related to anger in response to propaganda promoting male supremacy.

H7(b): SOA is negatively related to counter-arguing in response to propaganda promoting male supremacy.

H8: SOA is positively related to the intention to support the source of propaganda promoting male supremacy.

Evaluating the system of variables with structural equation modeling

Although H1–H8 predict relationships between SOA and multiple outcomes, they provide no predictions on how those outcomes interrelate. We therefore also pose a research question to better understand the interrelated system of variables concerning SOA and persuasive outcomes resulting from exposure to far-right extremist propaganda.

RQ1: What is the structural nature of the relationships between SOA and persuasion associated with exposure to race science/male supremacy propaganda?

Addressing this research question will not only allow us to gauge the respective relationships between SOA and all persuasive outcomes, but also provide some insight into the question as to whether SOA exerts a causal influence on these outcomes.

Study 1: race science

Methods

Participants

Data were collected from a paid, opt-in online survey panel of American adults in December of 2020. Respondents younger than 18 or unable to understand English were disqualified from participation. We removed all response sets that were disproportionately incomplete, “straight-lined,” or completed in less than 25% of the median completion time. The remaining sample (N = 404) was large enough to achieve sufficient statistical power for detecting small-to-medium sized effects (f = .175) for all analyses, assuming p value of .05 and a minimum statistical power (1 − β) of .80 (Cohen, 1992).

To recruit a sample akin to populations targeted by far-right extremist propaganda (i.e. young, white males; Miller-Idriss, 2018), we implemented a priori quotas that determined the extent to which certain demographic variables were represented in the sample. Specifically, we sought to recruit a sample that was 90% male, 75% white, and 75% aged 18–35. Our institution of these quotas yielded a sample that was 90.1% male, 74.3% white, and 73.5% aged 18–35. The sample’s overall makeup is summarized in Table 1.

Table 1.

Sample characteristics (Study 1).

Attribute	Category	Count	Proportion (%)
Age
	18–35 years old	297	73.5
	36–49 years old	60	14.9
	50–65 years old	19	4.7
	66 years old or older	28	6.9
Sex
	Male	364	90.1
	Female	37	9.2
	Other/Did not disclose	3	0.7
Ethnicity
	Non-Hispanic Caucasian/White	300	74.3
	African American/Black	62	15.3
	Asian	17	4.2
	Hispanic	10	2.5
	Native American	5	1.2
	Pacific Islander	1	0.2
	Other/Did not disclose	9	2.2

Materials

The focus of the original study from which this study originated investigated the moderating effects attitudinal inoculation (i.e. a counter-persuasion strategy), propaganda subtlety, and propaganda format on the persuasiveness of far-right propaganda messages. As such, participants were exposed to one of four kinds of scientific racism propaganda: an unsubtle video, a subtle video, an unsubtle meme, or a subtle meme.

The unsubtle video features an anti-Semitic discussion in which Jews’ inherent intellect and slyness allow them to exploit non-Jews. The subtle video condition features a prominent racist vlogger who argues that IQ and race are correlated and acts saddened by this fact, as if he is revealing an unfortunate truth. The unsubtle meme juxtaposed an image of Koko the Gorilla with an African child, with a caption suggesting the intellectual superiority of Koko. The subtle meme was presented as a comic in which opponents of scientific racism are inherently averse to “facts” that demonstrate different IQs for different races.¹ None of the stimuli were explicitly affiliated with any specific far-right groups or organizations to control for a priori sympathy for or aversion to extant far-right entities with which participants may have been familiar.

Given that (a) persuasive differences in the four kinds of propaganda are not the focus of the current study, (b) all four kinds of propaganda are pervasive in the online spaces under consideration, and (c) regular users of online spaces associated with SOA are likely to encounter all these kinds of propaganda, we collapsed all participants into a single exposure condition.

Measures

The original studies from which the current research emerged included dependent measures designed to gauge the persuasiveness of far-right propaganda. These dependent measures had been previously validated and used by Braddock (2019) in the study of extremist propaganda and were therefore suitable for our purposes here. So, in addition to measures for our primary predictor variable (SOAs), we included measures to gauge these outcomes—gratification, reactance (comprising counter-arguing and anger), source credibility attribution, and support intention—in the current study as well.

SOAs

To measure participants’ SOA, they were asked to indicate how often they troll other users, dox other users, use applications that anonymize their communication, use applications that encrypt their communication, and use alt-tech. These items were measured with Likert-type scales ranging from 1 (never) to 4 (often) and were randomly embedded in a larger scale measuring multiple online behaviors. Overall SOA score was calculated as the mean of these items (ɑ = .91).²

Gratification

Gratification was measured with two items that were randomly embedded in a larger index gauging emotional response. These items asked participants to indicate how much they felt satisfied and reassured by the propaganda on Likert-type-scales ranging from 1 (none at all) to 7 (a great deal). Cronbach’s alpha is an insufficient metric for describing the reliability of the two-item gratification scale as it would underestimate the scale’s true internal consistency. Eisinga et al. (2012) recommend applying a Spearman–Brown correction (ρ = 2r/[1 + r]) on the bivariate correlation between the items in two-item scales. Using this correction, we calculated the reliability estimate of the two-item gratification index (r = .71, ρ = .83).

Perceptions of source credibility

To indicate how credible participants found the source of the propaganda, they responded to six 7-point semantic differentials (adapted from McCroskey, 1966) anchored by the following pairs of descriptors: trustworthy-not trustworthy, sincere-insincere, honest-dishonest, dependable-not dependable, credible-not credible, and reliable-unreliable. The mean of these six items served as the score for perceived source credibility (ɑ = .95).

Psychological reactance

Past work on reactance—an aversive motivation to resist persuasive attempts—has demonstrated the construct to be the intertwined combination of anger and counter-arguing (Dillard and Shen, 2005).³ We therefore utilized two scales to measure these respective outcomes.

Anger

To indicate the degree to which participants were angry after being exposed to the propaganda, they were presented with three items randomly embedded in a larger emotional response index. These items asked participants to indicate their feelings of anger, irritation, and frustration in response to the propaganda on a scale ranging from 1 (none at all) to 7 (a great deal). The mean of these three items served as the overall score for anger (ɑ = .83).

Counter-arguing

Counter-arguing against the propaganda was measured using a single Likert-type scale ranging from 1 (I accepted all the points made in the message) to 7 (I argued against all the points made in the message). Psychometric research on psychological reactance has shown the use of this item to be strongly correlated with validated, open-ended counter-arguing measures (Parker et al., 2016).

Support intention

Participants were presented with four 7-point Likert-type scales asking whether they would support the source of the race science propaganda to which they were exposed. Specifically, participants were asked whether they would support the propaganda’s source ideologically (e.g. post support on social media), financially (e.g. donate money), logistically (e.g. store weapons), or violently (e.g. fight). The mean of the four items served as the overall score for support intention (ɑ = .96).

Control variables and moderators

Demographics

Given that (a) far-right extremist propaganda disproportionately targets young, white males and (b) material related to race science is likely to induce automatic aversion among non-white participants, we created dummy variables to represent participants’ age, gender, and race categories. We then included the dummy-coded variables in our analyses. This allowed us to control for the automatic aversion that non-white participants may have felt and estimate the respective effects of characteristics common to targets of race science propaganda.

Right-wing authoritarianism and social dominance orientation

Given the politically charged nature of far-right propaganda, it was necessary to account for predisposition for right-wing political positions. To this end, we included measures for right-wing authoritarianism (RWA) and social dominance orientation (SDO) in our analyses as controls, both of which have been empirically linked to far-right and extreme conservative beliefs and attitudes (Pratto et al., 1994).

RWA was originally conceptualized to measure the degree to which a person prefers social dynamics that prioritize uniformity and submission and limit diversity (Altemeyer, 1988). Where RWA relates to submissiveness to authority and adherence to social norms, SDO is associated with support of social hierarchies and in-group superiority bias. Both RWA and SDO concern prejudice, but whereas the former concerns prejudice against “threatening” groups, the latter concerns prejudice against minority or disadvantaged groups.

RWA was measured by presenting participants with 14 nine-point Likert-type scales on which they indicated the degree to which they agreed with various statements (e.g. “What our country really needs is a strong, determined leader who will crush evil and take us back to our true path”). SDO was measured using 16 Likert-type scales on which participants indicated the degree to which they agreed with other kinds of statements (e.g. “Some groups are simply inferior to other groups”). Overall scores for both RWA (ɑ = .76) and SDO (ɑ = .87) were calculated as the means of their respective question sets.

Inoculation condition

The data used for the current study were part of a larger project evaluating the effects of attitudinal inoculation on the persuasiveness of various kinds of far-right extremist propaganda. Given that inoculation treatments have counter-persuasive effects by design, we included a dummy-coded variable for inoculation condition in our models to control for inoculation’s inverse effect on propaganda persuasiveness.

Analyses

Three sets of analyses were performed in SPSS (v. 27) to evaluate the respective relationships between SOA and all outcomes. First, we calculated the bivariate correlations between SOA and all outcome variables. Second, we divided participants into low-, medium-, and high-SOA tertiles to perform analyses of covariance (ANCOVAs) evaluating whether differential levels of SOA deviated from one another in how they are related to salient outcomes. All ANCOVA models included SOA (high, medium, low) as the predictor variable and RWA, SDO, the demographic variables, and inoculation condition as covariates.

Third, we performed a series of multiple regressions to estimate the effects of the predictors on all persuasive outcomes. To identify optimal regression models containing only significant predictors, all models initially regressed the dependent variables on SOA (as a continuous variable), RWA, SDO, gender, race, age, and inoculation condition. If any predictors in the model were not significant, they were removed one-by-one based on highest p value until only significant predictors remained.

To provide a comprehensive view of how SOA and the DVs are structurally related, we also evaluated the system of variables with structural equation modeling techniques in AMOS Graphics (Version 27). It is important to note that although these techniques demonstrate the series of relationships between the SOA and the dependent variables, interpretations that SOA cause these outcomes should be carefully considered. On one hand, some researchers have argued that causation cannot be inferred by structural equation models unless the exogenous (i.e. predictor) variables are manipulated (Holland, 1986). Others have countered that social phenomena that are not amenable to manipulation (like SOA in the current study) can reasonably serve as both outcomes and causes of other phenomena (e.g. Bhrolcháin and Dyson, 2007) or that causation can be inferred based on how the researcher constructs the structural equation model and the modeler’s assumptions about the included variables (Bollen and Pearl, 2013).

Given the nature of our hypotheses and research question, we constructed our structural equation models with the assumption that participants’ SOA exerted effects on salient outcomes rather than the other way around. There is a temporal quality to our analyses that allows us to infer the direction of the relationships between the SOA and the outcome variables. Specifically, by its very nature, the independent variable is an indicator of participants’ past engagement in SOA. In contrast, the dependent measures asked about participants’ responses to far-right propaganda to which they had just been exposed. In this way, the structural equation models evaluated the effects of participants’ past behavior on their emergent beliefs, attitudes, and intentions in response to far-right propaganda stimuli. Moreover, fit indices provide us with information regarding the directionality of the path models (see “Results”). Despite these assumptions and evidence, comprehensive and definitive determinations of causation require longitudinal analyses that are beyond the scope of the current study; we elaborate on the possibility of this research in the final section of this article.

Results

SOA and gratification in response to race science propaganda

H1 predicted a positive relationship between SOA and gratification in response to race science propaganda. The bivariate correlation between these variables was positive and significant (r = .48, p < .001) and a significant ANCOVA, F(2, 351) = 11.00, p < .001, revealed that all levels of SOA were significantly different from one another (at least p < .05) in terms of their feelings of gratification in response to the propaganda (M_high = 3.34, SD_high = 0.32; M_med = 2.49, SD_med = 0.24; M_low = 1.42, SD_low = 0.27).

Consistent with these results, the optimal regression model, F(4, 395) = 48.91, p < .001, included four significant predictors of which SOA was the most potent (see Table 2).

Table 2.

Regression weights for the relation of optimal predictors with gratification in response to race science propaganda.

Predictor	B	95% CI	β	t
Constant	–0.35	[–1.06, 0.37]		–0.95
SOA	0.75	[0.56, 0.94]	.36	7.88***
SDO	0.46	[0.33, 0.58]	.32	7.10***
Race	0.34	[–0.03, 0.71]	.08	1.83^☨
Inoculation	–0.77	[–1.28, –0.25]	–.12	–2.94**
Adjusted R²	.32

SOA: subversive online activity; SDO: social dominance orientation.

N = 399. Race (1 = white, 0 = all other); inoculation (1 = inoculated, 0 = not inoculated).

***

p < .001; **p < .01; ^☨p < .10.

These results support H1.

SOA and attribution of credibility to the source of race science propaganda

H2 asserted a positive relationship between SOA and attribution of credibility to the source of race science propaganda. The correlation between these two variables was positive and significant (r = .47, p < .001) and the ANCOVA demonstrated that the three SOA tertiles were significantly different (at least p < .05) from one another, M_high = 3.82, SD_high = 0.33; M_med = 2.99, SD_med = 0.24; M_low = 2.31, SD_low = 0.27; F(2, 351) = 6.42, p < .01.

Regression analyses offered further support, identifying SOA as one of two significant, positive predictors of source credibility attribution, F(3, 396) = 63.86, p < .001; see Table 3.

Table 3.

Regression weights for the relation of optimal predictors with attribution of credibility to the source of race science propaganda.

Predictor	B	95% CI	β	t
Constant	0.45	[–0.22, 1.13]		1.33
SOA	0.71	[0.52, 0.90]	.34	7.44***
SDO	0.48	[0.35, 0.61]	.34	7.43***
Inoculation	–0.77	[–1.31, –0.28]	–.13	–3.03**
Adjusted R²	.32

SOA: subversive online activity; SDO: social dominance orientation.

N = 399. Inoculation (1 = inoculated, 0 = not inoculated).

***

p < .001; **p < .01.

Taken together, these results support H2.

SOA and psychological reactance in response to race science propaganda

H3 posited an inverse relationship between SOA and reactance in response to race science propaganda. Because reactance consists of anger and counter-arguing, we evaluated SOA’s respective relationships on these outcomes with the analyses described above and measured SOA’s influence on the overall reactance construct using structural equation modeling.

SOA and anger in response to race science propaganda

H3(a) predicted a negative relationship between SOA and anger in response to race science propaganda. The correlation between SOA and anger was not significantly different from zero (r = .03, p = .57). Moreover, an ANCOVA failed to identify significant differences between any of the SOA tertiles regarding their reported anger, F(2, 351) = 2.04, ns.

The regression analyses were successful in identifying the optimal model for predicting anger, F(3, 396) = 7.52, p < .001, but SOA emerged as only a marginal positive predictor (see Table 4).

Table 4.

Regression weights for the relation of optimal predictors with anger in response to race science propaganda.

Predictor	B	95% CI	β	t
Constant	4.26	[3.46, 5.07]		10.39
SOA	0.21	[–0.01, 0.43]	.10	1.91^☨
RWA	–0.26	[–0.44, –0.07]	–.16	–2.72**
SDO	–0.16	[–0.33, 0.01]	–.12	–1.88^☨
Adjusted R²	.05

SOA: subversive online activity; RWA: right-wing authoritarianism; SDO: social dominance orientation.

N = 399.

p < .01; ^☨p < .10.

These findings do not support H3(a).

SOA and counter-arguing against race science propaganda

H3(b) predicted an inverse relationship between SOA and counter-arguing against propaganda that advocates race science. The correlation between these variables was negative and significant (r = −.45, p < .001), and the ANCOVA indicated that participants characterized by low SOA (M_low = 3.90, SD_low = 0.29) reported counter-arguing significantly more than their moderate- and high-SOA counterparts (M_med = 3.27, SD_med = 0.26; M_high = 2.93, SD_high = 0.35).

The optimal regression model for predicting counter-arguing included SOA as a significant negative predictor, F(3, 396) = 66.68, p < .001; see Table 5.

Table 5.

Regression weights for the relation of optimal predictors with counter-arguing against race science propaganda.

Predictor	B	95% CI	β	t
Constant	7.22	[6.59, 7.85]		22.39
SOA	–0.64	[–0.84, –0.44]	–.28	–6.24***
SDO	–0.62	[–0.76, –0.49]	–.40	–8.96***
Race	–0.36	[–0.76, 0.04]	–.07	–1.79^☨
Adjusted R²	.33

SOA: subversive online activity; SDO: social dominance orientation.

N = 399. Race (1 = White, 0 = all other races).

***

p < .001; ^☨p < .10.

These findings support H3(b).

SOA and reactance as the combination of anger and counter-arguing (race science)

To better understand the relationship between SOA and reactance, we constructed a series of path models in which reactance was modeled as a latent construct comprising anger and counter-arguing. To identify the model that best fit the data, we altered path models based on output modification indices and the removal of non-significant paths (see the section titled “Structural relationships between SOA and persuasion by race science propaganda”). Every iteration of the model contained a significant, inverse relationship between SOA and reactance, including the optimal model (β = −.42, p < .001).

The sum of the evidence mostly supports H3.

SOA and intention to support the source of race science propaganda

H4 predicted a positive relationship between SOA and intention to support the source of race science propaganda. The correlation between SOA and support intention was positive and significant (r = .53, p < .001), and the three tertiles were significantly different (at least p < .05) from one another, M_high = 4.18, SD_high = 0.32; M_med = 4.05, SD_med = 0.24; M_low = 3.19, SD_low = 0.27; F(2, 351) = 3.81, p < .05.

Moreover, the optimal regression model included SOA as its most potent positive predictor, F(5, 394) = 48.41, p < .001; see Table 6.

Table 6.

Regression weights for the relation of optimal predictors with intention to support the source of race science propaganda.

Predictor	B	95% CI	β	t
Constant	0.62	[–0.16, 1.39]		1.55
SOA	0.86	[0.67, 1.05]	.39	8.80***
SDO	0.49	[0.36, 0.62]	.33	7.47***
Race	0.38	[0.00, 0.76]	.08	1.98*
Age	0.34	[–0.04, 0.72]	.07	1.77^☨
Inoculation	–0.57	[–1.09, –0.05]	–.09	–2.16*
Adjusted R²	.37

SOA: subversive online activity; SDO: social dominance orientation.

N = 399. Race (1 = white, 0 = all other races); age (1 = 18–35 years old, 0 = all other ages); inoculation (1 = inoculated, 0 = not inoculated).

***

p < .001; *p < .05; ^☨p < .10.

These results support H4.

Structural relationships between SOA and persuasion by race science propaganda

To answer RQ1, we constructed a series of path models to identify one that best matched the data. We began with a simple model whereby SOA directly predicted all outcomes of interest with no other paths (Model 1). According to various model fit standards (Hu and Bentler, 1999), this initial model was a poor fit, comparative fit index (CFI) = 0.89, root mean square error of approximation (RMSEA) = 0.11, standardized root mean square residual (SRMR) = 0.14, χ²(df) = 906.19(149).

Modification indices recommended the addition of multiple paths to improve model fit. Retaining all original paths, Model 2 also included paths from reactance to attribution of source credibility and support intention, as well as a path from attribution of source credibility to gratification. This model represented a good fit to the data, CFI = 0.96, RMSEA = 0.07, SRMR = 0.06, χ²(df) = 419.74(146).

Though Model 2 fit the data well, past work on the persuasiveness of extremist propaganda has shown attribution of source credibility to exert a positive effect on support intention (Braddock, 2019). The addition of this path further improved model fit. Given Model 3’s fit to the data and correspondence with past work, it was selected as the optimal representation, CFI = 0.96, RMSEA = 0.07, SRMR = 0.05, χ²(df) = 399.86(144); see Figure 1.

Figure 1.

Structural relationships between SOA and persuasive outcomes (Study 1).

These path coefficients allow for the calculation of the total (direct + indirect) standardized effects of each predictor on all outcomes (see Table 7).

Table 7.

Total standardized effects exerted by predictors on outcomes (Study 1).

Outcome	Predictor
Outcome	SOA	Reactance	SC
Reactance	–0.42
SC	0.50	–0.48
Gratification	0.57	–0.34	0.71
Support intention	0.59	–0.43	0.20

SOA: subversive online activity; SC: attribution of source credibility.

Table 7 demonstrates that SOA exerts positive total effects on gratification, attribution of credibility, and support intention, as well as a sum negative effect on reactance. These sums of total effects provide further support for H1–H4.

Study 1: summary

The goal of Study 1 was to determine whether engagement in SOA predicts susceptibility to persuasion by race science propaganda. The data indicate that it can. All four hypotheses received support and the optimal path model showed that the relationships between SOA and the persuasive outcomes are the dual function of direct and indirect effects.

Still, these findings describe only how SOA relates to one theme in far-right extremist messaging. To provide a fuller account of how SOA may relate to persuasion by far-right extremist propaganda, it is necessary to replicate these analyses in another thematic domain. Study 2 offers this replication.

Study 2: male supremacy

Methods

Participants

The sample (N = 396) was recruited and subjected to the same quotas and exclusion criteria as in Study 1. Power analyses again indicated a sufficiently large sample size, assuming a statistical power of .80 and an alpha level of .05 for each analysis. As in Study 1 (and for the same reasons), participants were again primarily male (90.2%), white (70.7%), and 18–35 years old (78.3%). See Table 8 for a synopsis of all participant demographics.

Table 8.

Sample characteristics (Study 2).

Attribute	Category	Count	Proportion (%)
Age
	18–35 years old	310	78.3
	36–49 years old	48	12.1
	50–65 years old	22	5.6
	66 years old or older	16	4.0
Sex
	Male	357	90.2
	Female	38	9.6
	Other/Did not disclose	1	0.3
Ethnicity
	Non-Hispanic Caucasian/White	280	70.7
	African American/Black	68	17.2
	Asian	23	5.8
	Hispanic	12	3.0
	Native American	3	0.8
	Other/Did not disclose	10	2.5

Materials

Participants were exposed to an unsubtle video, a subtle video, an unsubtle meme, or a subtle meme. The unsubtle video stimulus involved a MRA discussing dominant “alpha” males’ domination of women and feminized “beta” males. The subtle video stimulus depicts a man repeating that “men are tired” of sexism against males. The unsubtle meme stimulus juxtaposed an image of extreme bondage pornography with text overlay stating that women may “threaten to make [their] holes unavailable” to argue with men. The subtle meme stimulus shows a young, sexually active female who is implied to become a disease-ridden spinster.⁴ For the same reason as in Study 1, none of the stimuli were overtly linked to specific far-right organizations. All stimulus conditions were again collapsed into a single exposure condition.

Measures

All measures used in Study 2 were the same as those used in Study 1. Reliability estimates for all measures were good (ɑ_SOA = .89; ρ_{gratification} = .80; ɑ_sourcecred = .96; ɑ_anger = .88; ɑ_intention = .96; ɑ_RWA = .77; ɑ_SDO = .87; AVE_reactance = 0.70; CR_reactance = 0.77).

Analyses

All analyses in Study 2 were replicated from Study 1.

Results

SOA and gratification in response to male supremacy propaganda

H5 predicted a positive link between SOA and gratification in response to the male supremacy propaganda. The correlation was positive and significant (r = .53, p < .001) and a significant ANCOVA, F(2, 342) = 11.20, p < .001, showed that moderate (M_med = 2.83, SD_med = 0.32) and high (M_high = 2.38, SD_high = 0.29) levels of SOA experienced significantly greater gratification than those who reported engaging in low levels of SOA (M_low = 0.95, SD_low = 0.28; p < .001).

The regression analysis also showed SOA to be a robust predictor of gratification. The optimal regression model, F(2, 385) = 91.39, p < .001, included two significant predictors of which SOA was the strongest (see Table 9).

Table 9.

Regression weights for the relation of optimal predictors with gratification in response to male supremacy propaganda.

Predictor	B	95% CI	β	t
Constant	–0.39	[–0.91, 0.14]		–1.45
SOA	0.92	[0.73, 1.11]	.44	9.58***
SDO	0.31	[0.17, 0.44]	.21	4.55***
Adjusted R²	.32

SOA: subversive online activity; SDO: social dominance orientation.

N = 396.

***

p < .001.

These results support H5.

SOA and attribution of credibility to the source of male supremacy propaganda

H6 predicted a positive relationship between SOA and attribution of credibility to the source of male supremacy propaganda. The correlation between these two variables was positive and significant (r = .51, p < .001). An ANCOVA, F(2, 342) = 18.41, p < .01, similarly demonstrated significant differences between all SOA tertiles (M_high = 2.98, SD_high = 0.30; M_med = 2.79, SD_med = 0.34; M_low = 1.57, SD_low = 0.29; all p < .01).

Furthermore, the optimal regression model included SOA as the strongest predictor of perceived source credibility, F(3, 384) = 55.14, p < .001; see Table 10.

Table 10.

Regression weights for the relation of optimal predictors with attribution of credibility to sources of male supremacy propaganda.

Predictor	B	95% CI	β	t
Constant	–0.31	[–1.06, 0.44]		–0.81
SOA	0.85	[0.66, 1.04]	.42	8.83***
SDO	0.25	[0.11, 0.40]	.18	3.40***
RWA	0.14	[−0.02, 0.30	.08	1.68^☨
Adjusted R²	.30

SOA: subversive online activity; SDO: social dominance orientation; RWA: right-wing authoritarianism.

N = 387.

***

p < .001; ^☨p < .10.

These results offer support for H6.

SOA and psychological reactance in response to male supremacy propaganda

H7 predicted an inverse relationship between SOA and psychological reactance in response to male supremacy propaganda. Analyses involving reactance’s constituent elements (anger and counter-arguing) and reactance as a singular latent construct were replicated from Study 1.

SOA and anger in response to male supremacy propaganda

H7(a) predicted an inverse relationship between SOA and anger in response to male supremacy propaganda. The correlation between these variables was negligible (r = .01, p = .79). Neither the ANCOVA, F(2, 342) = 0.65, p = .52, nor the regression analyses, F(3, 384) = 5.48, p < .001; see Table 11, revealed any significant effects of SOA on anger.

Table 11.

Regression weights for the relation of optimal predictors with anger in response to male supremacy propaganda.

Predictor	B	95% CI	β	t
Constant	3.89	[2.77, 5.00]		6.85
Inoculation	0.57	[–0.05, 1.18]	.09	1.82^☨
Race	0.35	[–0.04, 0.73]	.09	1.77^☨
Gender	–0.26	[–1.30, –0.12]	–.12	–2.35*
RWA	–0.71	[–0.42, –0.11]	–.17	–3.30**
Adjusted R²	.04

RWA: right-wing authoritarianism.

N = 388. Inoculation (1 = inoculated, 0 = not inoculated); race (1 = white, 0 = all other); gender (1 = male, 0 = all other).

p < .01; *p < .05; ^☨p < .10.

These results fail to support H7(a).

SOA and counter-arguing against race science propaganda

H7(b) predicted an inverse relationship between SOA and counter-arguing against male supremacy propaganda. The correlation between these variables was significant and negative (r = −.46, p < .001) and an ANCOVA, F(2, 342) = 4.84, p < .01, showed that participants characterized by low SOA (M_low = 4.71, SD_low = 0.34) counter-argued against the propaganda significantly more (both p < .01) than those characterized by moderate SOA (M_med = 3.60, SD_med = 0.40) or high SOA (M_high = 3.23, SD_high = 0.35).

Furthermore, the optimal regression model included SOA as the strongest inverse predictor of reported counter-arguing, F(4, 384) = 35.80, p < .001; see Table 12.

Table 12.

Regression weights for the relation of optimal predictors with counter-arguing against male supremacy propaganda.

Predictor	B	95% CI	β	t
Constant	6.77	[5.81, 7.72]		13.875
SOA	–0.85	[–1.08, –0.62]	–.36	–7.38***
SDO	–0.33	[–0.50, –0.16]	–.20	–3.71***
RWA	–0.17	[–0.37, 0.02]	–.09	–1.73^☨
Race	0.43	[0.01, 0.86]	.09	1.99*
Adjusted R²	.27

SOA: subversive online activity; SDO: social dominance orientation; RWA: right-wing authoritarianism.

N = 388. Race (1 = White, 0 = all other).

***

p < .001; *p < .05; ^☨p < .10.

These results support H7(b).

Reactance as the intertwined combination of anger and counter-arguing (male supremacy)

As in Study 1, we constructed a series of path models in which reactance was modeled as a latent construct that predicted anger and counter-arguing (see the section titled “Structural relationships between SOA and persuasion by male supremacy propaganda”). Once again, the optimal model included a path signifying a significant, inverse relationship between SOA and reactance (β = −.42, p < .001).

As in Study 1, the respective analyses of the relationship between SOA and reactance’s constituent elements produced conflicting results, but a significant negative path coefficient linking SOA to reactance provides clarifying support.

The sum of this evidence provides support for H7.

SOA and intention to support the source of male supremacy propaganda

H8 predicted a positive relationship between SOA and intent to support the source of the male supremacy propaganda. The correlation between SOA and intention was positive and significant (r = .52, p < .001). An ANCOVA further demonstrated that SOA was positively related with support intention, F(2, 342) = 3.84, p < .05. Individuals who engaged in high or moderate levels of SOA (M_high = 3.93, SD_high = 0.31; M_med = 4.01, SD_med = 0.35) reported significantly greater intention to support the source of the male supremacy propaganda than individuals who engaged in low SOA (M_low = 2.91, SD_low = 0.30).

In addition, the optimal regression model included SOA as the strongest predictor, F(3, 384) = 64.18, p < .001; see Table 13.

Table 13.

Regression weights for the relation of optimal predictors with intention to support the source of male supremacy propaganda.

Predictor	B	95% CI	β	t
Constant	0.02	[–0.77, 0.81]		0.04
SOA	0.90	[0.70, 1.10]	.41	8.83***
SDO	0.30	[0.14, 0.45]	.19	3.79***
RWA	0.25	[0.08, 0.43]	.14	2.90**
Adjusted R²	.33

SOA: subversive online activity; SDO: social dominance orientation; RWA: right-wing authoritarianism.

N = 388.

***

p < .001; **p < .01.

H8 was supported.

Structural relationships between SOA and persuasion by male supremacy propaganda

We again used AMOS Graphics (v. 27) to construct a series of path models to identify the optimal variable structural orientation. Beginning with a simple model (Model 4) reflecting H5–H8, SOA directly predicted all outcomes with no other paths. This model fit the data poorly, CFI = 0.90, RMSEA = 0.11, SRMR = 0.12, χ²(df) = 805.01(148).

Modification indices recommended the addition and retention of the same paths used to construct Model 2 in Study 1. This model (Model 5) fit the data well, CFI = 0.97, RMSEA = 0.06, SRMR = 0.05, χ²(df) = 382.01(145).

We again amended the model to reflect past work on the persuasiveness of extremist propaganda and created a path from attribution of source credibility to support intention (Model 6). This further improved model fit and was chosen as the optimal model, CFI = 0.97, RMSEA = 0.06, SRMR = 0.04, χ²(df) = 372.841(144). Figure 2 depicts Model 6.

Figure 2.

Structural relationships between SOA and persuasive outcomes (Study 2).

Table 14 summarizes the total effects of all predictors on all outcomes.

Table 14.

Total standardized effects exerted by predictors on salient outcomes (Study 2).

	Predictor
Outcome	SOA	Reactance	SC
Reactance	–0.42
SC	0.54	–0.49
Gratification	0.61	–0.35	0.73
Support intention	0.57	–0.38	0.30

SOA: subversive online activity; SC: attribution of source credibility.

These total effects replicate those calculated in Study 1 in terms of sign and magnitude, further supporting H5–H8.

Study 2: summary

Study 2 showed that the relationships identified in Study 1 are robust to a second far-right extremist propaganda domain. The effects of SOA on persuasive outcomes related to exposure to male supremacy propaganda are similar in sign and magnitude as those observed in relation to race science propaganda. Moreover, the system of variables relating all variables in Study 2 almost perfectly replicate the system of variables from Study 1.

Given these results, it seems that SOA can affect how one responds to various kinds of far-right extremist propaganda. This finding has implications for understanding how this propaganda persuades intended audiences, and just as importantly, how we might intervene in radicalization processes affected by exposure to such propaganda.

Discussion

The SOAs outlined above effectively predicted persuasion by far-right extremist propaganda. These findings were replicated across two separate studies with near-identical results. The question remains, however, as to how we can use this knowledge to increase our understanding of online radicalization processes and how we might intervene in them.

Interrupting radicalization processes

This study’s findings offer some key takeaways about the risks of SOA and its role in the assimilation of far-right extremist ideologies. First, an individual’s participation in SOA is associated with an increased risk in their being persuaded by far-right extremist propaganda. Prevention efforts focused on identifying those at risk for receptivity to far-right extremist propaganda would benefit from recognizing some of the SOAs outlined above. Moreover, intervention targets might be better differentiated in terms of their participation in SOA, as it may help optimize the use of finite resources intended to prevent persuasion by extremist propaganda. This follows from research that has noted the benefits of audience segmentation for effective counter-radicalization interventions (see Van Eerten et al., 2017).

In addition, far-right extremist propaganda disseminated on alt-tech or in other online spaces that promote SOA are likely to be perceived as credible and enjoyable by heavy users of those platforms. This should sound alarms about the potential additive harms of SOA for mass disinformation campaigns like the “Stop the Steal” effort to overturn the 2020 Presidential Election. Consider, for instance, recent work showing alt-tech to have been a critical tool for organization and mobilization that contributed to the January 6, 2021 attack on the US Capitol (Munn, 2021). In addition to its coordination capabilities, alt-tech also serves as an incubator in which receptivity to far-right extremist propaganda can increase.

Finally, these studies suggest that SOAs should be incorporated into broader models of individual vulnerability to radicalization. Social dislocation and the search for social meaning are commonly—and rightly—understood as risk factors for radicalization to violent extremism (Miller-Idriss, 2020). However, our findings point to another modality of online radicalization risk. Individuals with established social roles in online spaces marked by SOA are more likely to express support for far-right extremist attitudes and behaviors. It follows that social isolation is not the only online factor that relates to increased risk of radicalization; social embeddedness in online communities that advocate SOA can increase this risk as well. This indicates a need to shift focus from the “quantity” of social embeddedness in online media (i.e. isolated or not) toward the “quality” of embeddedness—that is, what social milieus (and activities performed therein) contribute to one’s vulnerability to online radicalization.

Study limitations and future research

As in any empirical exercise, the results of this study are qualified by certain limitations. First, both studies relied on a sample gleaned from an opt-in survey platform. This limits our ability to project findings about how these treatments would perform in live online settings. Relatedly, these studies were limited to two far-right extremist thematic domains. More research is needed to extend these findings to other kinds of extremist propaganda (e.g. anti-government).

Second, although (a) Models 3 and 6 provided, respectively, better fits to the data relative to when the paths were reversed, and (b) our models predict current outcomes based on past behavior, our findings cannot be considered definitive on causality. Indeed, our data suggest that SOA exerts an influence on examined outcomes, but we cannot conclusively assert that SOA causes these persuasive vulnerabilities without longitudinal data. As such, investigations that evaluate these variables longitudinally are essential for confirming our assumptions.

Third, although the optimal structural equation models (a) included both anger and counter-arguing as constituent elements of reactance and (b) showed significant relationships between SOA and the second-order reactance construct, analyses failed to demonstrate a significant link between SOA and one of reactance’s constituent elements—anger. This was surprising, particularly given the significant positive association between anger and counter-arguing with respect to reactance (Rains, 2013). This deviation from past evidence may be explained by the fact that the unique context in which reactance is being measured in the current studies (i.e. ideological extremism) differs substantially from the other research contexts in which the intertwined model of reactance has traditionally been explored (e.g. health communication, see Ivanov, 2017). Future work in this domain would benefit from evaluating the potential moderating effect of persuasion context on both the structural nature of reactance and how different kinds of online activity relate to it.

Finally, our sample was limited to adults aged 18 and over. Although the use of an adult sample is typical in social scientific experimentation, minors likely comprise a substantial proportion of those who are targeted by and exposed to far-right extremist propaganda like that which was used in this study. It is similarly likely that individuals who engage in the SOAs examined in this study begin engaging in those activities before they turn 18. Given these possibilities, future research should evaluate the effects of SOAs on persuasion by far-right extremist messaging among minors. This would allow us to gain a more comprehensive understanding of how online behaviors can influence radicalization processes over time and life stages.

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Google LLC.

ORCID iD

Kurt Braddock

Notes

Author biographies

Dr. Kurt Braddock is an Assistant Professor in the School of Communication at American University. He is also a faculty fellow in American University’s Polarization and Extremism Research and Innovation Lab (PERIL), where he serves as senior methodologist.

Dr. Brian Hughes is the co-founder and Associate Director of PERIL. He is also a Research Assistant Professor in the Justice, Law, and Criminology program, also at American University.

Beth Goldberg is a research program manager in Google’s Jigsaw unit. She performs research on disinformation and extremism, and seeks to prevent online influence that contributes to violence.

Dr. Cynthia Miller-Idriss is the founder and Director of PERIL at American University. She is also a Professor in American University’s School of Public Affairs and School of Education.

References

Altemeyer

(1988) Enemies of Freedom: Understanding Right-Wing Authoritarianism. San Francisco, CA: Jossey-Bass.

Barker

M-J

(2016) Queer: A Graphic History. London: Icon Books.

Bhrolcháin

Dyson

(2007) On causation in demography: issues and illustrations. Population and Development Review 33(1): 1–36.

Blee

Creasap

(2010) Conservative and right-wing movements. Annual Review of Sociology 36: 269–286.

Bollen

Pearl

(2013) Eight myths about causality and structural equation models. In: Morgan

(ed.) Handbook of Causal Analysis for Social Research. New York: Springer, pp. 301–328.

Braddock

(2019) Vaccinating against hate: Using attitudinal inoculation to confer resistance to persuasion by extremist propaganda. Terrorism and Political Violence.

Brannon

(1976) The male sex role: our culture’s blueprint of manhood, and what it’s done for us lately. In David

Brannon

(eds) The Forty-Nine Percent Majority. Reading, MA: Addison-Wesley, pp. 1–49.

Cohen

(1992) A power primer. Quantitative Methods in Psychology 112: 155–159.

Conway

(2020) Routing the extreme right: challenges for social media platforms. The RUSI Journal 165(1): 108–113.

10.

Craker

March

(2016) The dark side of Facebook: the Dark Tetrad, negative social potency, and trolling behaviors. Personality and Individual Differences 102: 79–84.

11.

Dillard

Shen

(2005) On the nature of reactance and its role in persuasive health communication. Communication Monographs 72(2): 144–168.

12.

Douglas

(2016) Doxing: a conceptual analysis. Ethics and Information Technology 18: 199–210.

13.

Ebner

(2019) Replatforming unreality. Journal of Design and Science 6. https://jods.mitpress.mit.edu/pub/k109mlhg/release/1

14.

Eisinga

te Grotenhuis

Pelzer

(2012) The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International Journal of Public Health 58: 637–642.

15.

Farrell

(1974) The Liberated Man: Beyond Masculinity—Freeing Men and Their Relationships with Women. New York: Random House.

16.

Gaudette

Scrivens

Venkatesh

(2020) The role of the internet in facilitating violent extremism: insights from former right-wing extremists. Terrorism and Political Violence. Epub ahead of print 16 July. DOI: 10.1080/09546553.2020.1784147.

17.

Gehl

(2018) Weaving the Dark Web: Legitimacy on Freenet, Tor, and I2P. Cambridge, MA: MIT Press.

18.

Grover

Mark

(2019) Detecting potential warning behaviors of ideological radicalization in an alt-right subreddit. In: Thirteenth international AAAI conference on web and social media, Münich, Germany, 11–14 June, pp. 193–204. Menlo Park, CA: Association for the Advancement of Artificial Intelligence.

19.

Hodge

Hallgrimsdottir

(2020) Networks of hate: the alt-right, “troll culture,” and the cultural geography of social movement spaces online. Journal of Borderlands Studies 35(4): 563–580.

20.

Hoffman

Ware

Shapiro

(2020) Assessing the threat of incel violence. Studies in Conflict & Terrorism 43(7): 565–587.

21.

Holland

(1986) Statistics and causal inference. Journal of the American Statistical Association 81: 945–970.

22.

Bentler

(1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6(1): 1–55.

23.

Ivanov

(2017) Inoculation theory applied in health and risk messaging. In Parrott

(ed.) Encyclopedia of Health and Risk Message Design and Processing. New York: Oxford University Press. Available at: https://dx-doi-org.web.bisu.edu.cn/10.1093/acrefore/9780190228613.013.254

24.

Jackson

Depew

(2017) Darwinism, Democracy, and Race: American Anthropology and Evolutionary Biology in the Twentieth Century. London: Routledge.

25.

Jenkins

Leroy

(2021) Histories of Racial Capitalism. New York: Columbia University Press.

26.

Kavanagh

Brown

(2020) Towards a research agenda for examining online gender-based violence against women academics. Journal of Further and Higher Education 44(10): 1379–1387.

27.

McCroskey

(1966) Scales for the measurement of ethos. Speech Monographs 33: 65–72.

28.

Messner

(2016) Bad men, good men, bystanders: who is the rapist? Gender and Society 30(1): 57–66.

29.

Miller-Idriss

(2018) The Extreme Gone Mainstream: Commercialization and Far Right Youth Culture in Germany. Princeton, NJ: Princeton University Press.

30.

Miller-Idriss

(2020) Hate in the Homeland: The New Global Far Right. Princeton, NJ: Princeton University Press.

31.

Mountford

(2018) Topic modeling the red pill. Social Sciences 7(3): 42.

32.

Mudde

(2007) Populist Radical Right Parties in Europe. Cambridge: Cambridge University Press.

33.

Munn

(2021) More than a mob: Parler as preparatory media for the U.S. Capitol storming. First Monday 26(3). Available at: https://firstmonday.org/ojs/index.php/fm/article/view/11574

34.

Ortiz

(2020) Trolling as a collective form of harassment: an inductive study of how online users understand trolling. Social Media + Society 6(2): 1–9.

35.

Parker

Rains

Ivanov

(2016) Examining the “blanket of protection” conferred by inoculation: the effects of inoculation messages on the cross-protection of related attitudes. Communication Monographs 83: 49–68.

36.

Perlroth

(2019) What is end-to-end encryption? Another bull’s-eye on big tech. New York Times, 19 November. Available at: https://www.nytimes.com/2019/11/19/technology/end-to-end-encryption.html

37.

Pleck

(1987) The theory of male sex-role identity: its rise and fall, 1936 to the present. In: Brod

(ed.) The New Men’s Studies. Winchester, MA: Allen & Unwin, pp. 21–38.

38.

Pratto

Sidanius

Stallworth

, et al. (1994) Social dominance orientation: a personality variable predicting social and political attitudes. Journal of Personality and Social Psychology 67(4): 741–763.

39.

Rains

(2013) The nature of psychological reactance revisited: a meta-analytic review. Human Communication Research 39(1): 47–73.

40.

Scrivens

Gill

Conway

(2020) The role of the internet in facilitating violent extremism and terrorism: suggestions for progressing research. In: Holt

Bossler

(eds) The Palgrave Handbook of International Cybercrime and Cyberdeviance. Cham: Springer, pp. 1417–1435.

41.

Smiler

(2004) Thirty years after the discovery of gender: psychological concepts and measures of masculinity. Sex Roles 50: 15–26.

42.

Sussman

(2014) The Myth of Race. Cambridge, MA: Harvard University Press.

43.

Van Eerten

J-J

Doosje

Konijn

, et al. (2017) Developing a Social Media Response to Radicalization: The Role of Counter-Narratives in Prevention and De-Radicalization. Department of Psychology and Political Science, University of Amsterdam. Available at: https://repository.wodc.nl/handle/20.500.12832/2215

44.

Walther

McCoy

(2021) US extremism on Telegram: fueling disinformation, conspiracy theories, and accelerationism. Perspectives on Terrorism 15(2): 100–124.