Abstract
Online hate speech on social media platforms causes harm to those who are victimized as well as society at large. The prevalence of hateful content has, thus, prompted numerous calls for improved countermeasures and prevention. For such interventions to be effective, it is necessary to gain a nuanced understanding of influences that facilitate the spread of hate speech. This study does so by investigating what are relevant digital determinants for online hate perpetration. Moreover, the study explores possibilities of different technology-driven interventions for prevention. Thereby, the study specifically considers the digital environments in which online hate speech is most often produced and disseminated, namely social media platforms. We apply frameworks related to the concept of digital affordances to focus on the role that technological features of these platforms play in the context of online hate speech. Data were collected using the Delphi method in which a selected sample of experts from both research and practice answered multiple rounds of surveys with the goal of reaching a group consensus. The study encompassed an open-ended collection of initial ideas, followed by a multiple-choice questionnaire to identify, and rate the most relevant determinants. Usefulness of the suggested intervention ideas was assessed through the three lenses of human-centered design. The results of both thematic analysis and non-parametric statistics yield insights on how features of social media platforms can be both determinants that facilitate online hate perpetration as well as crucial mechanisms of preventive interventions. Implications of these findings for future intervention development are discussed.
Introduction
Hate speech is a form of prejudice-motivated communication toward (individual members of) a group defined by shared characteristics such as race, ethnicity, gender, or sexual identity.1,2 It aims to discriminate, disparage, or intimidate the targeted group.3,4 In the online context, speech also includes (audio)visual forms of expression instantaneously communicated through digital channels.5,6
As a context-dependent manifestation of prejudice, hate speech varies with regards to its explicitness and targets.7,8 However, to the concern of scholars and field experts, the number of detected online hate speech incidents of any form on social media from Twitter to LinkedIn has been rising, giving importance to viewing the issue in its entirety.9–11
Digital spaces in which online hate speech is produced and disseminated are not static but subject to change. Social media companies regularly adjust their policies and functions, whereas new platforms with novel features emerge and users adapt their behavior in response. 12 The platforms' functionalities are products of organizational choices, economic interests, and institutional regulations impacting not only users individually, but also society at large.13,14
In this dynamic context, technological functionalities do not determine user behavior, but they are influential factors in a complex interplay between users and their social environments. With this study, we aim to give an overview of technological functionalities related to online hate perpetration and analyze them from an affordance perspective.
Previous studies looked in detail at specific technological functionalities in relation to harmful behavior or content [e.g., interface features and (in)civility in comment sections 15 or algorithmic recommendations promoting extremist content 16 ]. Yet there is no systematic overview of platform-independent functionalities and their relevance for the production and dissemination of online hate speech. Given the multidisciplinary nature of the subject, compiling this overview requires the consideration of perspectives from social sciences, as well as technology- and policy-oriented research. 17
An effective way of achieving such an interdisciplinary overview is a Delphi study. 18 We used this consensus method combining survey-based and qualitative research techniques to address
With increased attention dedicated to online hate speech, numerous interventions are being envisioned not only by researchers and practitioners in civil society organizations and think tanks, but also by social media companies themselves. 19 These fast-paced changes make it difficult for researchers to map recent developments solely through published research. Therefore, practitioners were also invited as participants, ensuring high actuality and applicability of the results.
To identify directions for future solutions in hate speech prevention, RQ2 asks: What do experts with knowledge on the topic of online hate speech agree on make technology-driven interventions useful for preventing online hate perpetration? Technology-driven interventions are understood as tools and strategies using technological features or being implemented into social media environments to mitigate influences that facilitate online hate perpetration. While mostly focused on prevention, these interventions may include detection and countering of hate speech as part of their preventive effort.
Besides being facilitated by technological features, the production and dissemination of online hate speech results from individual user behaviors and business models of social media platforms focused on maximizing engagement, for example, by popularizing emotionally charged content.20,21 To allow for a holistic approach to RQ2, experts provide their evaluations of intervention possibilities by using the three lenses of human-centered design (HCD). This framework proposes that effective solutions to complex problems are found when user needs are met under consideration of the possibilities and limitations of technology and the economic interests of stakeholders.22–24
Theory
Despite contributing to the spread of hate speech, technological features of social media platforms such as comment functions or pop-up notifications have also been used to counter online aggressive behaviors.25,26 Thus, technological features can not only serve as determinants for online hate perpetration, but can also be crucial for its prevention. The concept of digital affordances helps to approach this multiplicity.
Digital affordances encompass both the features of a digital environment and the users. They refer to actions made possible through the relations and interactions between users and objects. 27 In other words, digital affordances are user behaviors enabled or constrained through technological features. 28 These features are not deterministic: They provide possibilities for action, but realizing these actions partly depends on the perceptions and capabilities of the user.27,29 In the following, we describe two affordance frameworks for the analysis of technological features in the context of online hate speech.
The mechanisms and conditions framework by Davis moves beyond binary conceptions of whether a feature serves a particular function or not by examining how it enables user actions, for whom and under which circumstances. 29 While mechanisms of affordances explain the directionality and intensity with which features make actions possible (e.g., by initiating or responding to an action request they push or pull user behavior in a certain direction), conditions of affordances specify the user to whom action possibilities are afforded (i.e., accounting for the user's knowledge, skills, and social and political contexts). 29
The mechanisms and conditions framework considers affordances from the perspective of the individual user. However, posting online hate speech on social media is not always an individual act, but can also result from group dynamics and social norms.30–32 Thus, it is relevant to investigate how technological features on social media afford the formation of groups, how they structure interactions, distribute agency within these groups, and affect social norms. 33
An approach for doing so is provided by the taxonomy of social network platform affordances for group interactions, consisting of interaction and intervention affordances. 33 Group interactions, enabled by technological features, are distinguished by their degree of openness and by how much the involved actors are connected. Intervention affordances consider how features give agency to users. Both types of affordances influence norms for socially acceptable behavior, which bears consequences for the handling of hate speech. 33
Method
Delphi studies allow for a structured compiling of experts' viewpoints through surveys, making it an effective method for collecting insights from various disciplines. 34 The main aim of a Delphi study is to synthesize these insights by striving for a group consensus. 35 Through multiple rounds of anonymous surveys building up on each other's findings, data are systematically narrowed down from a broad overview to a concise voting. 36
Between rounds, participants receive overviews of their own answers and preliminary results. 37 The method has been applied to understand harmful behaviors such as bullying, its antecedents, and possible intervention measures.18,38,39 It has, to our best knowledge, not been applied to online hate speech yet.
In contrast to conventional surveys, Delphi studies include elements of qualitative research, such as open-ended questions, and work with comparatively small sample sizes (<30 participants are common). 40 The sampling process emphasizes purposeful selection based on participants' expertise rather than probability sampling.38,41 It is possible to also include practitioners in the sample, which is important when studying a timely topic such as online hate speech whose recent developments may not be reflected in academic publications yet. 42
This study specifically discusses technology-related determinants and intervention ideas. Beyond that, the surveys included questions on personal and social determinants. The overall aim was to establish a holistic view on determinants for online hate perpetration across different levels of influence. The analysis presented here concentrates on the particular role of digital determinants as digitization permeates every area of daily life, enabling many possible interactions of digital determinants with personal and social influences.
In the first open-ended survey, participants were asked to list and explain digital determinants for online hate perpetration and ways of using technology to prevent users from posting online hate speech. The results were processed through thematic analysis and provided the structure and content of the second survey, 42 in which participants rated each determinant for relevance on a 5-point Likert scale from not at all relevant to extremely relevant (answer option cannot specify was added).
Participants also assessed technology-driven intervention ideas through the three lenses by indicating whether an idea was desirable for users, technologically feasible, and economically viable. To compensate for the lack of nuance of the dichotomous questions, we added comment fields for further elaboration. Both surveys provided definitions for hate speech to avoid misunderstandings. Basic demographics and occupational information were recorded. The surveys were administered through Qualtrics from June 2021 to January 2022 and approved by the institutional ethics review committee.
In a multi-stage process of purposive sampling, we first invited the authors of systematic reviews related to the keywords “hate speech,” “online hate,” and “cyberhate,” as well as the first authors of the studies discussed in the reviews. The same search terms were used on the Web of Science to identify additional researchers. They were invited to our sample if their academic profiles (publication record, CV, description of research interests) emphasized online hate speech as the primary field of expertise.
Websites such as the International Network Against Cyber Hate (INACH) were searched for organizations active in monitoring, awareness raising, or prevention of hate speech to determine practitioners. We used snowball sampling techniques by asking participants and members of our professional networks to nominate other knowledgeable experts.
We contacted 75 researchers, practitioners, and organizations of whom 28 individuals (57 percent researchers, 38 percent female) participated and 13 completed both surveys. Response rates were 26.7 percent in the first and 53.2 percent in the second round. a The majority of the sample has over 14 years of experience in their current position (21 percent) and holds a Master's degree (41 percent). For further information on the fields of expertise, see Table 1. Participants are affiliated with institutions and organizations in 16 countries across Europe, North America, and Australia.
Participant's Field of Study (Researchers) and Occupations (Practitioners)
Note: Multiple mentioning of fields of study or occupation allowed. Unknown refers to survey participant who did not provide demographic data but completed survey otherwise.
Results
Determinants for online hate perpetration
The first survey resulted in 272 comments on determinants and intervention ideas. Thematic analysis was used to evaluate the results, allowing for an inductive discovery of structural patterns in the data. 43 In this process, an initial set of codes was established from the data to iteratively create themes that describe subsets of the data. Each theme was illustrated by examples of concrete determinants.
These served as items in the second survey, which was analyzed by using non-parametric statistics. Considering the small sample size, the median serves as a measure for relevance (median > 4 indicating high relevance).41,44 The interquartile range (IQR) shows how much opinions deviated (IQR ≤ 1 indicating group consensus, see Table 2). 45 For an overview of all determinants, themes, and relevance ratings, see Table 2.
Digital Determinants for Online Hate Perpetration
Italicized determinants represent those upon which consensus was reached and which are considered relevant (Median = 4), while italicized and bold determinants reach consensus and were considered highly relevant (Median > 4).
Theme descriptions:
(1) Functionalities on social media for interacting and building communities.
(2) Design characteristics of social media platforms and the user reactions and norms they give rise to.
(3) Technological, organizational, and economic aspects of how social media platforms are structured and operate.
(4) Describes the impact of users through the creation and dissemination of content.
IQR, interquartile range.
Intervention ideas for online hate perpetration
The analysis of the intervention ideas similarly employed thematic analysis for the data of the first survey. Results from the second survey were compared to assess how useful the intervention ideas were rated based on the three lenses of HCD. Given the participants' diverse backgrounds, we expected their expertise to vary across the dimensions of the framework (user desirability, technological feasibility, and economic viability) and thus asked them to assess their own expertise on a scale from 0 to 100 (no—very high expertise). For each dimension, we excluded the responses of participants who scored <50 to ensure the data were grounded in a sufficient level of expertise (Table 3).
Self-Assessment of Expertise on Three Lenses of Human-Centered Design
Note: Inclusion criterium: rating of expertise >50.0 percent.
It was verified that exclusion of answers did not lead to aggravated gender imbalance in the sample (addressing concerns that female participants may perceive their own expertise on technology and business dimensions as lower due to internalized stereotypes).
We summarized composite scores on desirability, feasibility, and viability of the intervention ideas into a total score that indicates their potential to prevent users from posting hate speech. This allowed us to rate the intervention ideas through the three lenses of HCD, compare them, and identify the ones that are seen as most useful by the participants. A score of at least 75 percent on each dimension was set to indicate high usefulness.
Three intervention ideas had an overall score of over 75 percent but scored lower on economic viability (Table 4). The experts further elaborated on the desirability, feasibility, and viability of the intervention ideas in textual comments. The analysis of these comments complements the numeric data, while also uncovering limitations in the applicability of the framework.
Intervention Ideas and Evaluation
Intervention suggestions in bold received ≥75 percent of votes on all dimensions, italicized intervention suggestions received ≥75 percent of votes in total, but scored <75 percent on at least one individual dimension.
Theme descriptions:
(1) Rapid and impulsive online communication could be disrupted by creating moments of reflection to nudged users toward positive online behaviour.
(2) Identify perpetrators or users at risk of perpetration and apply changes to their digital environments to prevent hate speech posting and radicalization.
(3) Change social media structurally, by changing algorithms or mitigating negative influences of social media technology (e.g., regarding filter bubbles).
(4) Limit visibility of hateful content and prevent engagement with it through automated detection and removal.
(5) Create digital communities with prosocial norms built on shared interests to provide users with a sense of belonging and options for meaningful interaction.
(6) Changes in functionalities and design of social media platforms.
Seven participants (five researchers) were concerned that the desirability of an intervention cannot be generalized due to differing motivations for using social media. While some users might welcome interventions exposing them to different viewpoints, others who seek belonging and connection with like-minded individuals might be pushed toward more polarized and less regulated spaces. A solution to this dilemma may be to aim for a balance between what is desirable for most users and what is acceptable for the user groups targeted by an intervention.
Despite positive evaluations of feasibility, skepticism was voiced by six researchers and three practitioners about how much interventions should rely on automated solutions. These concerns pertain to a lack of nuance in artificial intelligence-driven hate speech detection and a need for more advanced personality profiling algorithms for targeted interventions. One researcher experienced in Natural Language Generation was especially skeptical about AI's abilities to successfully create counter-narratives. However, it was proposed by two practitioners to experiment with such solutions in controlled environments or to use technological solutions under the supervision of trained moderators.
Three researchers and one practitioner found economic viability difficult to assess and conditioned by user desirability. In their view, interventions such as structural changes would be implemented by social media companies only if desired by their users or made mandatory through legal regulation. However, three practitioners also noted that companies could exert some influence over how functionalities are perceived by users through advertising.
Five participants assigned responsibility for the prevalence of online hate speech to the social media companies by highlighting their shortcomings to realize effective countermeasures. Fittingly, many intervention ideas with positive evaluations—for example, changing recommendation systems, making algorithms transparent, or excluding risk content from being promoted-related to changing technological structures. This emphasizes that experts located the agency and responsibility for effective prevention mostly within the companies.
A recurring concern among five participants was that users may feel personally attacked or restricted in their freedom of speech by invasive interventions. Three practitioners, however, advocated for a shift in this debate, with one stating: “There need[s] to be a greater emphasis on the right to freedom from fear as a corollary to the right to free speech and the freedoms of expression.” Strong opposition was voiced to the suggestion of limiting anonymity online. Even though anonymity relates to relevant determinants, all seven comments on this topic mentioned the importance of protecting members of minority groups or activists from targeting and persecution online.
Discussion
To answer RQ1, we collected expert assessments of the relevance of digital determinants for online hate perpetration and interpreted them under consideration of different affordance frameworks. Notably, none of the social media features in Table 2 should be understood as a direct cause for online hate perpetration because data collected through the Delphi method are not suitable to make inferences about causal relations or the directionality of influence.
Besides, digital determinants interact with personal, social, and contextual factors, meaning that technological features may facilitate online hate perpetration under certain circumstances and for certain users, as suggested by the mechanisms and conditions framework. 29 For example, sharing and commenting content is not inherently harmful, but when combined with an algorithmic logic promoting visibility of emotionally agitative content or used by individuals for the purpose of attacking others, these functionalities can contribute to the spread of hate speech. 20 Personalized content recommendations correlate with online hate when repeatedly exposing users to hateful, radicalizing, or strongly polarizing content but may not do so otherwise. 1
Companies have agencies to intervene by changing recommendation systems, eliminating harmful content, or stopping its monetization. Thus, certain platform features and organizational decisions shape the visibility of content. Another influential feature within control of the platforms is the requirement for personal identification to open an account. While profiles affording anonymity serve to protect users, they can also create personal distance and decrease empathy with victims of hate.1,27 If a lack of empathy with others is understood as social norm, account requirements can become determinants for online hate perpetration.
To answer RQ2, we asked participants to describe intervention ideas and evaluate their usefulness through the three lenses of HCD. We use the term “intervention ideas” as we understand the output of this study to reflect current needs for intervention and give directions for future development processes rather than providing concrete tools. Looking at the intervention ideas from an affordance perspective highlights how they can employ features that are determinants of online hate perpetration for the purpose of prevention.
For example, the comment sections can become places of counter-speech and recommendation algorithms can be re-shaped to break filter bubbles and introduce new viewpoints to users. The availability of hateful content can be mitigated by interventions that dismantle or remove it while providing transparent explanations for this procedure.
The responsibility and action possibilities related to many features are located within the social media companies. They have considerable agency over how their platforms afford group interactions. 33 These design choices are intertwined with the group interactions that are fostered and whether these are prosocial or antisocial. However, design choices are not made in a vacuum and depend on how the platforms react to user needs.
As discussed by our expert sample, anonymity, for instance, shall not be decreased to protect vulnerable users, but intervention strategies can leverage technological features to mitigate the negative impacts of anonymity and foster empathy and pro-social interactions.
Not all suggested intervention ideas represent responses to a specific determinant. In turn, several determinants are not addressed by any intervention idea. This disparity between determinants and mitigating factors could represent missed opportunities for creating more effective interventions. Based on the expert viewpoints this study yields, we suggest for future intervention development to bridge this gap by focusing on the specific aspects that give social media features a facilitating or mitigating influence on online hate perpetration.
The discussed affordance frameworks can help to make sense of the multiplicity of technological features in social media environments and to ensure that the development of interventions accounts for differences between users, their motivations and needs, and the differing levels of agency among actors.29,33 Further, these perspectives account for how design choices on social media platforms impact user behavior and how interventions could induce behavior changes. Evaluating feasibility and viability could further improve assessments of an intervention's long-term and large-scale implementation possibilities. 46
To establish concrete guidelines on intervention development, more data on how the suggested intervention ideas are intended to function and be implemented would be helpful. Given that Delphi studies are narrowed down to numerical outcomes in later rounds, this study would have benefited from obtaining detailed information through in-depth interviews.
While this was beyond feasible commitment for our participants, we recommend future research to consider such methodological combinations. With regards to the three lenses of HCD, future research should extend the framework to also address political and ethical considerations as well as differences in user needs and motivations to reflect on the usefulness of interventions. This is especially important in the context of a sensitive topic such as online hate speech. 47
This study aimed to deepen the understanding of digital determinants for online hate perpetration and possible technology-driven interventions for its prevention by giving a comprehensive overview of what experts in the field of online hate speech regard as relevant determinants and useful intervention ideas. By applying an affordance perspective to data generated by the Delphi Method, we have further shown how different technological features can be determinants or components of interventions. A continuation of this exploration may be a crucial part in the development of future strategies for the prevention of online hate speech.
Footnotes
Authors' Contributions
I.W.: Conceptualization, Methodology, Formal Analysis, Writing—Original Draft; H.V.: Methodology, Writing—Review and Editing, Supervision, Funding Acquisition; K.P.: Conceptualization, Methodology, Supervision; S.P.: Methodology, Supervision, Writing—Review and Editing.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
This research was conducted as part of a program that receives funding from the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 861047.
