Leveraging external resources for offensive content detection in social media

Abstract

Hate speech is a burning issue of today’s society that cuts across numerous strategic areas, including human rights protection, refugee protection, and the fight against racism and discrimination. The gravity of the subject is further demonstrated by António Guterres, the United Nations Secretary-General, calling it “a menace to democratic values, social stability, and peace”. One central platform for the spread of hate speech is the Internet and social media in particular. Thus, automatic detection of hateful and offensive content on these platforms is a crucial challenge that would strongly contribute to an equal and sustainable society when overcome. One significant difficulty in meeting this challenge is collecting sufficient labeled data. In our work, we examine how various resources can be leveraged to circumvent this difficulty. We carry out extensive experiments to exploit various data sources using different machine learning models, including state-of-the-art transformers. We have found that using our proposed methods, one can attain state-of-the-art performance detecting hate speech on Twitter (outperforming the winner of both the HASOC 2019 and HASOC 2020 competitions). It is observed that in general, adding more data improves the performance or does not decrease it. Even when using good language models and knowledge transfer mechanisms, the best results were attained using data from one or two additional data sets.

Keywords

Hateful and offensive language deep language processing transfer learning vocabulary augmentation RoBERTa

1. Introduction

While there are some differences among countries in the legal system regarding free speech [94], freedom of expression – the freedom to express ideas and opinions – is essential in international law both for individuals and societies [44]. This fact, along with the harm caused by hate speech [34,71,87] (or in other interpretations, the harm that is hate speech [100]), has sparked and maintained a debate about hate speech [10,24,41,64], where the focus was on arguments for and against laws regulating hate speech [33,39,92]. Moreover, there is still no clear agreement on whether legal measures constitute the best response to hate speech, or it is other methods (e.g., education or counter-speech [14]), or the combination of the two [29]. Regardless of what methods one deploys to handle hate speech, the first step would be detecting it. Manual discovery of hateful and offensive content, however, will place a burden on those who are tasked with this duty [40]. Moreover, the amount of content that is generated in social media, and the online space, in general, make this unfeasible. For these reasons, there is a strong demand for methods to automatically detect hateful and offensive content.

In this paper, we share our approach for the detection and classification of hateful or offensive content (using English language social media posts) with a focus on how a small labeled corpus can be supplemented by leveraging external resources, such as other corpora, word representations [35], and pre-trained models. First, we would discuss the related literature and the limits of this study in this section. Then, we would describe in Section 2 the HASOC competitions [60,62], as well as the additional sources of data we used in our experiments to complement the limited data available in these competitions. This would be followed by the short depiction of methods applied in Section 3. Then, we would present and discuss our experimental results (Section 4), then we analyse some major take-aways (Section 5) and conclude our study, outlining possible directions for future work in Section 6.

1.1. Related literature

The Internet has fundamentally changed our lives and habits [16], from our interactions with friends to our news consumption [4,25,43] strategies. These changes have positive and negative consequences alike. On the one hand, possibilities for cooperation across continents have spiked, and access to information is easier than ever. Moreover, we can share information and our opinion at a larger speed and to a wider circle than before. On the other hand, when this opportunity is combined with our potential to hide our identity [117], these opportunities can also be (and are) used to spread offensive and hateful content. This has urged many researchers to examine the possibility of automatically detecting and classifying offensive content (for overviews of these works, we refer the reader to [59,113]), the first of these efforts dating back to 1997 [95]. Following this first model based on various rules, a considerable part of the research effort was devoted to similar systems, centering around rules [73], templates (for example, identifying patterns such as I *intensity* *user_intent* *hate target*) [70] or key expressions [37,59].

An approach more complex was pairing classical machine learning models with various feature extraction techniques. For example, Kwok and Wang, in their work, applied Naïve Bayes (NB) classifiers in conjunction with Bag-of-Words (BoW) features [54]. Others, like Grevy et al. applied the same feature extraction technique and used the results as input for the Support Vector Machine (SVM) algorithm [36]. The propensity of BoW features for producing high false positive errors, however, motivated researchers of the field to use different feature extraction methods. Warner and Hirschberg [102], for one, used a template-based strategy to extract features for an SVM classifier, and detect anti-semitic hate speech with an accuracy of up to $94 %$ . While Burnap and Williams [15] used n-grams and dependencies in conjunction with Bayesian Logistic Regression (BLR) in the classification of tweets to achieve a significant improvement over their baseline BoW features. Similarly, Waseem and Hovy [104] also applied n-grams (in particular, character n-grams) in their work and combined them with a logistic regression classifier. Zia et al. [116] applied TF-IDF [88] features in combination with SVM, NB, and k-NN (k-Nearest Neighbour) classifiers. While Davidson et al. supplemented the TF-IDF features with more information, such as reading ease scores, and used them in combination with a Linear SVM model [19].

Another fruitful research direction in the detection of hateful and offensive content was that of the application of Deep Learning models. These models have achieved considerable success in the early two thousands [108], advancing the area of NLP [78], after already having succeeded in signal- and speech processing, as well as computer vision. An important milestone in the headway of Deep Learning in Natural Language Processing was the introduction of word representations, such as Word2Vec [68]. When combined with already established machine learning models, these representations, or embeddings showed their effectiveness in the task of offensive language detection [66,109], providing better results (Area Under the Curve – AUC) than those attained using the BoW approach, while requiring less memory and training time [22]. The introduction of word representations such as Word2Vec, FastText [13] and Glove [80] also made the application of other deep learning models easier, namely the use of Convolutional Neural Networks [8,30,79,118] (CNNs), along with the application of Recurrent Neural Networks (RNNs) – such as the Long Short Term Memory [8,20,23,101] (LSTM) unit – as well as the combination of the two [8,20,23,45,51,101]. Another crucial milestone in the use of deep learning for natural language processing in general (and hate speech detection in particular) was the publication of the transformer model [21]. The success of which is exemplified by the fact that in a specific sub-task of the 2019 OffensEval challenge [110], seven of the top ten teams were using transformer models. The above discussed deep learning models, besides providing high classification scores when used in standalone mode, can also be successfully applied in ensembles [72,75]. As a matter of fact, when considering the average performance on every sub-task in a recent challenge with more than fifty competing teams, the team attaining the best performance used the very same approach [93], combining such models as LSTMs, tranformers, and OpenAI’s GPT [82], with classical machine learning models, such as Random Forest, and SVMs.

Beside the research effort dedicated to the detection and classification of various aspects of hateful and offensive speech, another driving force behind the progress in the area was the organization of shared tasks, challenges and competitions, that provided incentives, and labeled data to researchers working on these tasks. Some examples of recent competitions in the area include the detection of insults in social commentary [46], aggression [53], as well as the detection of hateful or offensive content in different languages, such as German [105], Spanish [11], Italian [89], and English [60,61,110].

1.2. Scarcity of reliably and uniformly labeled data

The task of detecting hateful and offensive language comes with many difficult challenges, the reader can gain an overview of by for example reading the works of MacAvaney et al. [59], Fortuna et al. [26,27] or Kovács et al. [51]. In this paper, however, our main focus is how external sources can be leveraged to complement a small labeled dataset. For this reason, in this section we would discuss solely the problem of data scarcity – along with its causes – and will not discuss other issues, such as when users intentionally obfuscate their language that aims to avoid detection by automatic algorithms [74], or the context-dependent nature of hateful language [42,81,83,98] (including the differentiation between in-group and out-group language users [12]) and its effect on bias [18,50,91]. These challenges and difficulties are in more detail discussed in the papers listed above.

An important factor contributing to the scarcity of uniformly labeled data is the lack of universally accepted definition of hate speech [2,3,32,47,76,77,97,98]. As the 2018 report of the Free Word Centre puts it, “‘Hate speech’ is an emotive term which does not have a uniform definition under international human rights law” [29]. There is, however, a commonly applied definition, applied by major information technology companies (including Facebook, YouTube, and Microsoft), that is based on a code of conduct [17], which in turn is based on the 2008 framework decision of the European Union [28], which defines illegal hate speech as “as the public incitement to violence or hatred directed to groups or individuals on the basis of certain characteristics, including race, colour, religion, descent and national or ethnic origin”.1

¹
https://ec.europa.eu/commission/presscorner/detail/en/MEMO_19_806

However, as Guo and Johnson points it out, the final decision is largely dependent on the human content moderators, which has frequently been the cause of controversy among users and reviewers [38], indicating that this definition is not necessarily productive, and allows for too much subjectivity in its application. Another indication of the inadequacy of our definitions is the low inter-annotation rates one can observe [86]. For example, in the 2019 installment of the ‘Hate Speech and Offensive Content Identification in Indo-European Languages’ competition (from hereinafter referred to as ‘HASOC 2019’), there was an inter-rater agreement of at most

77 %

[62], while some suggest the minimum acceptable rate to be

80 %

[65]. Another difficulty observed in the HASOC 2019 competition was that of the understanding of different language registers, as well as cultural context, as exemplified by the example of “youth talk” [62]

All the above discussed problems lead to the scarcity of labeled data, and in the case of the use of multiple datasets, the incompatibility of different sets [27]. Which not only leads to difficulties to to the sometimes unreliable nature of labelling, but also due to the data-hungry nature of leading machine learning (and in particular deep learning) models [1].

1.3. Contribution

In this study, our efforts were concentrated on alleviating the above discussed issues related to hate speech. We did so using two consecutive HASOC challenges (HASOC 2019 [62], and HASOC 2020 [60]) as examples. Given that the number of labeled instances in the training sets of these text classification competitions was limited, we consider them to be a great case study to examine the effect of leveraging external resources, which includes the use of pre-trained classification models, word representations, and additional data. Although the advantage of additional training data in general is already commonly known, there are conflicting cases as well [31,115]. We should note here, that this work is an extension of the work of Alonso et al. [5], and Kovács et al. [51], thus we consider our main contribution to be the extension of their experiments, and showing that their inferences remain true for the HASOC 2020 competition as well. Lastly, we consider it an important result that not only these techniques of leveraging external resources lead to state-of-the art recognition scores on the HASOC 2019 test set, the same was true for the test set of the HASOC 2020 competition as well, where on the sub-task we tackle, we increased the performance from 0.51 Macro $F_{1}$ -score [69] to above 0.91 (close to 0.92).

2. External data sources

Here, we introduce the various additional resources we examine in the leveraging of external data. These resources include the various datasets (including the datasets provided for the HASOC competitions), as well as word representations, and pre-trained classifiers.

2.1. Datasets

To systematically examine the effect of similarly labelled datasets, we examine three twitter datasets (with tweets primarily in English) that are all available online (including the text of the tweet, and not only the tweet ids, a criterion that lead to the exclusion of some often used data sets from consideration [103,104]). Two of these sets were collected by the same authors at different times (with a one-year gap). While the third was collected by different researchers, but was labelled based on similar principles (as can be seen below, at the detailed description of the data sets). We consider both of these differences worth to examine. As we consider leveraging labelled data from different data collections an especially interesting problem, as earlier, even for the similar data sources, a low generalization capability was reported [7] (though recently successful cross-lingual transfer learning experiments have been carried on HASOC2019 and OLID [85]). For one, the importance in the collections carried out in different years is highlighted by the fact that for example, in the training set of HASOC 2019, the name of Trump occurs 5492 times, while in the training set of HASOC 2020, it occurs only 92 times.

Table 1
The token type counts in Train and Test sets of Hasoc 2019 dataset

Text Digits Punctuation e-mail URL #-tag @-mention Emoticon Emoji Mixed tokens

Train 105829 1000 19348 1 3001 18904 6311 17 1974 3241

Test 27471 270 4538 0 140 1321 511 5 3 720

Total 133300 1270 23886 1 3141 20225 6822 22 1977 3961

	Text	Digits	Punctuation	e-mail	URL	#-tag	@-mention	Emoticon	Emoji	Mixed tokens
Train	105829	1000	19348	1	3001	18904	6311	17	1974	3241
Test	27471	270	4538	0	140	1321	511	5	3	720
Total	133300	1270	23886	1	3141	20225	6822	22	1977	3961

2.1.1. Hasoc2019

The Hasoc 2019 dataset [62] was developed for the competition to predict various classes of offensive or hateful speech. It is provided with a predefined train and test split. The total number of tweets in the training set is 5852, out of which 2261 are labeled as hate speech (HOF or OFF), and 3591 are labeled as non-hate speech (NOT). The total number of tweets in the test set is 1153, out of which 288 are labeled as HOF/OFF, and 865 are labeled as NOT. The tweets may contain #-tags, @-mentions, URLs, emojis, emoticons, and other mixed character strings. Table 1 shows the numbers each token type appeared in the Hasoc 2019 train and test sets.

2.1.2. Hasoc 2020

The Hasoc 2020 dataset [60] was developed for the competition to predict various classes of offensive or hateful speech. It is provided with a predefined train and test split. The total number of tweets in the training set is 3708, out of which 1856 are labeled as hate speech (HOF), and 1852 are labeled as non-hate speech (NOT). The total number of tweets in the test set is 1592, out of which 807 are labeled as HOF, and 785 are labeled as NOT. The tweets may contain #-tags, @-mentions, URLs, emojis, emoticons, and other special characters. Table 2 shows some samples tweets from Hasoc 2020 dataset with their ground truth labels. Table 3 shows the numbers each token type appeared in the Hasoc 2020 train and test sets. The mixed tokens are the tokens that are the combination of letters, digits, a hyphen, full stop short forms, and a single quote mark without any space character (e.g., C14).

Table 2
Hasoc 2020 dataset samples of tweets with their ground truth labels

Table 3

The token type counts in Train and Test sets of Hasoc 2020 dataset

	Text	Digits	Punctuation	Email	URL	#-tag	@-mention	Emoticon	Emoji	Mixed tokens
Train	51799	501	11213	0	1137	353	3374	41	1337	2485
Test	22240	211	4662	1	518	181	1457	17	495	1060
Total	74039	712	15875	1	1655	534	4831	58	1832	3545

2.1.3. OLID

The last dataset of hateful and offensive speech we worked with was that of the OLID2

²
https://scholar.harvard.edu/malmasi/olid or https://sites.google.com/site/offensevalsharedtask/olid.

data set (introduced in [111]), developed for the SemEval competition of 2019 [110]. The motivation to use this data set was to experiment with a data set that was similarly labelled to HASOC 2019 and HASOC 2020 (on the first layer of annotation, hateful or offensive, and innocuous tweets were distinguished from each other, then the tweets deemed to be hateful or offensive were further classified into more fine-grained categories), where even the definition of the two high-level categories were very similar (OLID offensive category: “Posts containing any form of non-acceptable language (profanity) or a targeted offense, which can be veiled or direct. This includes insults, threats, and posts containing profane language or swear words” [111]). Another similarity was that both the OLID dataset, and the HASOC data sets were comprised of tweets, which were written mostly in English language (mixed with some expressions in other languages).

OLID was also provided with a predefined train and test split. The total number of tweets in the training set is 13240, out of which 4400 are labeled as hate speech (HOF or OFF), and 8840 are labeled as non-hate speech (NOT). The total number of tweets in the test set is 860, out of which 240 are labeled as HOF/OFF, and 620 are labeled as NOT. The tweets may contain #-tags, @-mentions, URLs, emojis, emoticons, and other mixed character strings. Table 4 shows the numbers each token type appeared in the Olid-v1 train and test sets.

Table 4

The token type counts in Train and Test sets of Olid-v1 dataset

	Text	Digits	Punctuation	URL	#-tag	@-mention	Emoticon	Emoji	Mixed tokens
Train	246501	1525	43994	2059	5349	33412	76	3626	7309
Test	17489	167	3294	549	1943	608	3	281	582
Total	263990	1692	47288	2608	7292	34020	79	3907	7891

3. Methods

Here, we describe the data processing pipeline used in our experiments, including the pre-processing of text (Section 3.1), the cross-validation technique we used for validating our models (and for ensembling of the trained models – Section 3.2), as well as the short description of the algorithms we applied (Section 3.3).

3.1. Text pre-processing

The first step of our classification pipeline was that of text pre-processing. This step included the tokenization of tweets, as well as the removal or replacement of certain tokens. For example, extra space characters were removed, as long spaces were collapsed into one. Moreover, where necessary, we replaced web addresses with the URL token, and @-mentions with the token @-USER. We also removed hash characters (while retaining the text from hashtags), and emoticons. In addition, following Kovács et al. [51], we also removed emojis from the text. All other special characters (including punctuation) were retained. Moreover, we did not attempt any correction of spelling. Lastly, as we were approach the task from a deep learning perspective (largely motivated by the recent success of various deep learning models in different natural language processing tasks3

³
https://paperswithcode.com/task/text-classification

[96]) our focus was on model that learn a representation from largely raw textual data, hence we did not execute any lemmatization, stemming, or POS-taggins steps either. For the pre-processing step, we used the dedicated natural language processing toolbox of Matlab.4

⁴

https://in.mathworks.com/discovery/natural-language-processing.html

3.2. Cross-validation

For the training of each deep learning model, we used a variation of the regular five-fold cross-validation. This was carried out in the following manner. First, the training data in each data set was divided into five non-overlapping partitions, in a way that class-distributions in the resulting partitions were as similar in each other (as well as the original training set) as possible. We used these sets as validation sets, for early-stopping and hyper-parameter optimization purposes. In the case of every partition, the instances left out were considered as the training set corresponding to the given fold. After this partitioning, for each model used, five separate models were trained, each one optimizing its learnable parameters on the training set of various folds. While the validation set was used for early-stopping, and validation purposes. For each method the decision to classify a tweet into the inoffensive, or hateful/offensive category was done by calculating the average predicted probabilities of the five models trained on the five different folds. Here, it should be noted, that unlike in the 2021 paper of Kovács et al. [51], when incorporating data from multiple corpora, we still adhered to this k-fold partitioning, meaning that a combination of for example the HASOC 2019 and HASOC 2020 data sets meant that in the first fold, the training set from first fold of HASOC 2019 was combined with the training set from the first fold of HASOC 2020 data set, and the same was true for all five folds. However, the validation set to be used only depended on the data set we were originally examining. Meaning that if we were examining the effect that adding the HASOC 2020 data to the HASOC 2019 data, then we used only the validation set of the HASOC 2019 fold for early-stopping, and evaluation purposes.

3.3. Deep learning methods

Motivated by the successful application of deep learning for text classification problems [96], as well as for the problem of detecting offensive or hateful language [8,118], including challenges built around this task [62,101,110], we experimented with various deep learning techniques. Similarly as Alonso et al. [5], we applied a deep neural network model that combined convolutional layers with recurrent layers. We also carried out experiments using the FastText classifier. Lastly, we trained a BERT variant, namely RoBERTa, to attain state-of-the-art results on both HASOC data sets. We shortly introduce the aforementioned methods in the sections below.

Fig. 1.

Architecture of the CNN-BiLSTM model used in our experiments.

3.3.1. CNN-LSTM

When participating in the HASOC 2019 competition, a hybrid Convolutional Neural Network (CNN), and Long-Short Term unit network (LSTM) was used by Alonso et al. [5], and later this model was improved by Kovács et al. [51]. In our work we extended their experiments for meta-parameter optimisation. As Fig. 1 shows, the first part of the model is an input layer, where the batch size is defined using the BatchSize parameter. Here, we also define the maximum amount of tokens we would retain in a tweet, as 128. In order to bring tweets to a format that can be used in the embedding layer in a later stage, we first had to apply the $one_hot$ function defined in the Keras framework, to hash tokens (while disregarding punctuation) to a number, without providing a guarantee for unicity (meaning that different words would potentially be hashed to the same number). Here, we controlled the range of potential numbers with the parameter VocabSize. As the $one_hot$ function turned the original textual representation of tweets into a numerical one, it became possible to feed the resulting vectors into the embedding layer. The meta-parameters we were aiming to optimize in the layer were the embedding dimension, EmbedDim (along with the previously mentioned vocabulary size). After the initial stages, the model (similarly to the work of other researchers, such as Badjatiya et al. [8] and Zhang et al. [114]) contained a series of convolutional layers (maximum two), that used the tanh function in their neurons. Each convolutional layer was followed by a layer of maximum pooling. The kernel size in the first layer was 8, and when included, the second layer had a kernel size of 6. And the number of filters used in each convolutional layer was a meta-parameter we aimed at optimising. Following the pooling layer, a series of (maximum 3) bi-directional LSTM layers were included in the model, the size of which was again a meta-parameter that we optimized. Then a series of (maximum 3) dense layers of varying (and to be optimized) sizes were deployed, followed by dropout layers, where the rate of dropout was the same after each layer, and it was limited between 0.2 and 0.5. Lastly, the model was supplied with an output layer, consisting of two neurons (for the two categories of hateful or offensive, and non-offensive tweets), applying the softmax nonlinearity.

To optimize the value of the meta-parameters discussed, we drew parameter combinations randomly, and with each combination, trained independent networks on the different folds at least three separate times. Based on the resulting macro $F_{1}$ -score attained on the development sets, we were selecting the parameter combination that provided the best performance. The values used in the random search are summarized (in the order of being used in the model) in Table 5. One parameter that had not been discussed earlier is that of the learning rate. In our experiments we used the adam optimizer in every epoch, and after unsuccessful epochs (epochs that had not increased the macro $F_{1}$ -score attained on the corresponding validation set), we halved the learning rate. The meta-parameters this process produced, as well as the results attained on both HASOC data sets will be reported in Section 4.

Table 5
Potential values for the meta-parameters of the CNN-LSTM architecture

Description Name Values

Size of batch during training BatchSize 32, 64

Size of the vocabulary VocabSize 5000, 10000, 20000, 30000, 40000, 50000

Dimension for token embedding EmbedDim 100, 150, 200, 250, 300, 350, 400

Number of filters in the 1st CNN layer FilterNo_1 64, 128

Number of filters in the 2nd CNN layer FilterNo_2 0, 32, 64

Size of pooling operation applied Pool size 2, 4

Number of cells in the 1st BiLSTM layer LSTM_size_1 600, 800, 1000

Number of cells in the 2nd BiLSTM layer LSTM_size_2 0, 400, 600, 800

Number of cells in the 3rd BiLSTM layer LSTM_size_3 0, 100, 200, 400

Number of neurons in the 1st dense layer DenseSize_1 200, 400, 600

Number of neurons in the 2nd dense layer DenseSize_2 0, 100, 200, 400

Number of neurons in the 3rd dense layer DenseSize_3 0, 100, 200

Rate of dropout DropOut 0.2, 0.3, 0.4, 0.5

Initial learning rate LR 0.0005, 0.001, 0.002

Description	Name	Values
Size of batch during training	BatchSize	32, 64
Size of the vocabulary	VocabSize	5000, 10000, 20000, 30000, 40000, 50000
Dimension for token embedding	EmbedDim	100, 150, 200, 250, 300, 350, 400
Number of filters in the 1st CNN layer	FilterNo_1	64, 128
Number of filters in the 2nd CNN layer	FilterNo_2	0, 32, 64
Size of pooling operation applied	Pool size	2, 4
Number of cells in the 1st BiLSTM layer	LSTM_size_1	600, 800, 1000
Number of cells in the 2nd BiLSTM layer	LSTM_size_2	0, 400, 600, 800
Number of cells in the 3rd BiLSTM layer	LSTM_size_3	0, 100, 200, 400
Number of neurons in the 1st dense layer	DenseSize_1	200, 400, 600
Number of neurons in the 2nd dense layer	DenseSize_2	0, 100, 200, 400
Number of neurons in the 3rd dense layer	DenseSize_3	0, 100, 200
Rate of dropout	DropOut	0.2, 0.3, 0.4, 0.5
Initial learning rate	LR	0.0005, 0.001, 0.002

3.3.2. FastText

Another text classification method examined in our experiments was that of FastText [49]. For this, we used the implementation available at github5

⁵
https://github.com/facebookresearch/fastText

. During our experiments using the FastText models, we were applying the following scheme for training and valiation purposes. We trained each model using the parameter optimization built in (invoking the autotune-validation option – with the validation set of the fold – and the autotune-duration option – setting it to 600, resulting in a parameter-optimization process limited in ten minutes). The meta-parameter optimization was carried out in a way that the meta-parameters were chosen based on the

F_{1}

-score attained on the supplied validation set.

Beside the parameter controlling the duration of parameter-optimization for each model, we set only one parameter as constant, that is the dimensionality of embeddings. However, we did so only in case we were using the pre-trained WikiNews word embedding6

⁶

A collection of a million word vectors, trained using the Wikipedia 2017 corpus, the statmst news dataset, and the webbase dataset of UMBC, accessible at: https://fasttext.cc/docs/en/english-vectors.html.

[67]. Given that the dimensionality of the word vectors downloaded was 300 the embedding dimensions of our models also had to be set to that value. We will list and discuss results of these experiments on both HASOC corpora in Section 4.

3.3.3. Transformers

One of the methods utilized in our study was the RoBERTa [57] model, an offspring of the BERT [21] family of models, which in turn is part of the Transformers [99] family tree. The transformers’ first appearance was in Vaswani et al. [99]; they were showcased as evidence that the recurrent cells were extraneous for a recurrent encoder-decoder [9]. For this purpose, Vaswani et al. in [99] showcased an improvement in machine translation by utilizing attention mechanisms without relying on recurrent neural networks, concreting the way for more transformers architectures, like BERT [21] as the main outcome. This achievement lay the first stone for the appearance of several variants, including DistilBert [90], AlBERT [55], TinyBERT [48], and RoBERTa [58] (used here). Our experiments hinged on previous research with DistillBERT and RoBERTa for OffensEval 2020 hate speech detection task [112], we were inclined to use RoBERTa [57] (that is part of the SimpleTransformers library [106]) due to its noticed performance on the task.

In our experiments, we have fine-tuned the RoBERTA model using the cross-validation method described in Section 3.2. As training data, we used both HASOC data sets. We did so by using the default meta-parameters presented in [106], except for the number of epochs (that was set to a maximum of 20, but as we also applied early stopping using the development set, we have never reached this limit), and the learning rate, which was equal to 1e-5 and 4e-5 (HASOC 2020). Results of these experiments are also described in Section 4.

3.4. Classifier ensemble

Following the approach of the 2019 SemEval winners [93], as well as the approach of Kovács et al. [51], we have also experimented with classifier ensembles. This was carried out similar to how we combined versions of the same model trained on different folds. That is, by averaging the probabilities predicted by different models.

4. Results and discussion

Here, the results attained on the two HASOC corpora will be listed and discussed. To facilitate the comparison of the performance of various models in different circumstances, the Section will be organized as follows. In Section 4.1 we discuss the results gotten using the hybrid CNN, BiLSTM model. First on the HASOC 2019 dataset, then on its 2020 counterpart. Then, in Section 4.2 we would do the same with results attained using the FastText classifiers. Lastly, in Section 4.3, we would list the results provided by the RoBERTa model.

Each section will be structured in the same manner. First, results (both average macro- $F_{1}$ score, and weighted- $F_{1}$ score – and where applicable deviations) will be listed for HASOC 2019 for the case where no external resources were added beside those that are already inherently part of the model (e.g. in the case of RoBERTa, we use a pre-trained version). Then, for each subsequent result, we add more external data sources. We consider this mode of presentation instructive regarding the effect of additional resources. Then the same results would be reported for the HASOC 2020 dataset.

Table 6
Meta-parameter values chosen for CNN-LSTM architecture on the different HASOC datasets

Parameter description HASOC 2019 HASOC 2020

Size of batch during training 64 32

Size of the vocabulary 30000 30000

Dimension for token embedding 350 250

Number of filters in the 1st CNN layer 128 64

Number of filters in the 2nd CNN layer 64 32

Size of pooling operation applied 4 4

Number of cells in the 1st BiLSTM layer 600 800

Number of cells in the 2nd BiLSTM layer 0 0

Number of cells in the 3rd BiLSTM layer 0 0

Number of neurons in the 1st dense layer 600 400

Number of neurons in the 2nd dense layer 0 200

Number of neurons in the 3rd dense layer 0 0

Rate of dropout 0.2 0.5

Initial learning rate 0.001 0.001

Number of parameters (in millions) 14.8 13.8

Parameter description	HASOC 2019	HASOC 2020
Size of batch during training	64	32
Size of the vocabulary	30000	30000
Dimension for token embedding	350	250
Number of filters in the 1st CNN layer	128	64
Number of filters in the 2nd CNN layer	64	32
Size of pooling operation applied	4	4
Number of cells in the 1st BiLSTM layer	600	800
Number of cells in the 2nd BiLSTM layer	0	0
Number of cells in the 3rd BiLSTM layer	0	0
Number of neurons in the 1st dense layer	600	400
Number of neurons in the 2nd dense layer	0	200
Number of neurons in the 3rd dense layer	0	0
Rate of dropout	0.2	0.5
Initial learning rate	0.001	0.001
Number of parameters (in millions)	14.8	13.8

4.1. CNN-LSTM

In this section we discuss the classification scores attained using a simple hybrid CNN-BiLSTM model on the two HASOC corpora. To emphasize the effect of additional data used, in the case of both data sets, we focus on results attained using the meta-parameters that were optimized using the core data set only. These meta-parameters are listed above in Table 6 for both the HASOC 2019, and the HASOC 2020 data sets. As can be seen in Table 6, for both data sets, we ended up with a model of similar (small – below 15 million trainable parameters) size, sharing some meta-parameter values (including the size of the vocabulary, the max pooling parameter, and the number of BiLSTM layers used).

Results for each data set will be listed in their respective subsection below. Here, for each meta-parameter setting, the five-fold cross-validation training was carried out five times, thus the reports listed in the tables in the subsections below are those attained as an average of the results from five independent runs. Training five independent ensembles has also allowed us to examine the significance of the improvements attained, which we did using a two-tailed heteroscedastic t-test.

Table 7
CNN-LSTM results on the HASOC 2019 test set (scores are the average of five models; the best results, and those not significantly different – $p > 0.05$ – are emphasized in bold)

Databases F-scores and deviations

HASOC 2020 OLID HASOC 2019 Macro- $F_{1}$ STDEV Weighted- $F_{1}$ STDEV

– – ✓ 0.6557 0.0118 0.7217 0.0148

✓ – ✓ 0.6763 0.0145 0.7486 0.0141

– ✓ ✓ 0.7327 0.0078 0.7931 0.0050

✓ ✓ ✓ 0.7253 0.0211 0.7869 0.0183

Databases	F-scores and deviations
–	–	✓	0.6557	0.0118	0.7217	0.0148
✓	–	✓	0.6763	0.0145	0.7486	0.0141
–	✓	✓	0.7327	0.0078	0.7931	0.0050
✓	✓	✓	0.7253	0.0211	0.7869	0.0183

4.1.1. HASOC 2019

Results attained on the test set of HASOC 2019 are listed in Table 7. As can be seen in Table 7, the addition of the smaller (HASOC 2020) data set already significantly increased both the macro, and the weighted $F_{1}$ -score (with $p < 0.05$ ). A larger increase in classification scores can be seen when OLID data is added to the training process. This increase is significant not only when compared to the results gotten using the HASOC 2019 data alone, but also when compared to those results attained using both HASOC data sets (with $p < 0.05$ ). However, when both the HASOC 2020 and OLID training sets are added to the training data, while the resulting scores are not significantly different from those reported in the third row, there is no increase in the performance either. It is possible that this would change if the model meta-parameters were optimized, taking into consideration the largely increased amount of available training data (for models in the last row approximately four times as many training instances are available than for models in the first row).

Table 8
CNN-LSTM results on the HASOC 2020 test set (scores are the average of five models; the best results, and those not significantly different – $p > 0.05$ – are emphasized in bold)

Databases F-scores and deviations

HASOC 2019 OLID HASOC 2020 Macro- $F_{1}$ STDEV Weighted- $F_{1}$ STDEV

– – ✓ 0.8864 0.0030 0.8863 0.0030

✓ – ✓ 0.8901 0.0021 0.8900 0.0021

– ✓ ✓ 0.9013 0.0039 0.9012 0.0039

✓ ✓ ✓ 0.9040 0.0030 0.9040 0.0030

Databases	F-scores and deviations
–	–	✓	0.8864	0.0030	0.8863	0.0030
✓	–	✓	0.8901	0.0021	0.8900	0.0021
–	✓	✓	0.9013	0.0039	0.9012	0.0039
✓	✓	✓	0.9040	0.0030	0.9040	0.0030

4.1.2. HASOC 2020

When looking at the results attained on the HASOC 2020 listed in Table 8, one can see that the resulting classification scores are markedly higher, despite the best reported result in the original competition [60] being only 0.51 [69]. Another observation one can make is that the macro and weighted $F_{1}$ -scores are very similar. The latter phenomenon is due to the test set of the HASOC2020 data set being balanced. The former phenomenon may also be related to the balanced class distribution of the data set. But while some initial experiments seem to support the notion that balancing the class distribution would be beneficial (training our model on the HASOC 2019 data set using probabilistic sampling [52], with the λ parameter set to 1 – thus presenting the network with an equal number of offensive and not offensive tweets in each batch – we attained an average macro $F_{1}$ -score of 0.6655, and an average weighted $F_{1}$ -score of 0.7381 – a slight improvement over our initial results). It would, however, take more extensive experimentation to arrive at conclusive results.

We can also see on Table 8, that with each data set added, the classification scores are improving. The bigger (in terms of the number of tweets contained) the data set added is, the bigger the improvement is in the classification scores. Lastly, we would note, that in the case of HASOC2020, when both (HASOC2019 and OLID) were added to the training data, the classification scores were improved compared to the case where only one of the additional data sets were leveraged. This improvement, however (when the results in the last line are compared to those presented in the preceding line) is not significant. Moreover, the improvement attained by adding the HASOC2019 data to the HASOC2020 data was also not significant. With the addition of OLID data to the training set, however, we attained significant improvements in the resulting classification scores.

Table 9
CNN-LSTM meta-parameter values for the models that performed best on HASOC 2020 (based on the average performance on the validation set), when trained using all three data sets

Parameter description Top-1 Top-2 Top-3

Size of batch during training 64 32 32

Size of the vocabulary 50000 50000 50000

Dimension for token embedding 400 250 150

Number of filters in the 1st CNN layer 128 64 128

Number of filters in the 2nd CNN layer 32 64 64

Size of pooling operation applied 4 4 4

Number of cells in the 1st BiLSTM layer 600 800 800

Number of cells in the 2nd BiLSTM layer 0 0 400

Number of cells in the 3rd BiLSTM layer 0 0 0

Number of neurons in the 1st dense layer 600 600 400

Number of neurons in the 2nd dense layer 200 200 400

Number of neurons in the 3rd dense layer 100 100 0

Rate of dropout 0.4 0.5 0.2

Initial learning rate 0.001 0.0005 0.001

Number of parameters (in millions) 24.9 19.9 20.1

Parameter description	Top-1	Top-2	Top-3
Size of batch during training	64	32	32
Size of the vocabulary	50000	50000	50000
Dimension for token embedding	400	250	150
Number of filters in the 1st CNN layer	128	64	128
Number of filters in the 2nd CNN layer	32	64	64
Size of pooling operation applied	4	4	4
Number of cells in the 1st BiLSTM layer	600	800	800
Number of cells in the 2nd BiLSTM layer	0	0	400
Number of cells in the 3rd BiLSTM layer	0	0	0
Number of neurons in the 1st dense layer	600	600	400
Number of neurons in the 2nd dense layer	200	200	400
Number of neurons in the 3rd dense layer	100	100	0
Rate of dropout	0.4	0.5	0.2
Initial learning rate	0.001	0.0005	0.001
Number of parameters (in millions)	24.9	19.9	20.1

Ensembling: here, for both HASOC data sets, we trained our hybrid models based on the meta-parameters that were found to be optimal for the original data sets, even when additional data sets were included. However, Kovács et al. [51] earlier found that optimizing the meta-parameters for the increased data sets specifically can be beneficial. For this reason, we have sampled a hundred different set of meta-parameters, and trained independent models using them on the combination of the OLID and the two HASOC data sets. We did so with the goal to select the set of meta-parameters, with which the resulting macro $F_{1}$ -scores on the validation sets would be the highest on average. However, even after training fifty networks, the top-3 meta-parameter combinations (see Table 9) did not produce significantly different validation-scores.

Thus we decided to exploit the ensembling already present in the 5-fold cross-validation approach applied here, in the following manner. When combining the probabilities from the five models trained on the five different folds, we made sure that the predicted probabilities for each fold are coming from a model that we trained using the meta-parameter combination which performed best on average for the given fold. Using this method, we were able to avoid entirely dismissing models that did not perform significantly worse than the best performing model. Moreover, we ensured that for the ensembling step, there would be an increased heterogenity in the models combined.

We have evaluated the resulting ensemble on the test set of the HASOC 2020 corpus. The models on average (based on the average of ten independent runs) attained a macro $F_{1}$ -score and weighted $F_{1}$ -score of 0.909. Scores significantly (with $p < 0.05$ ) higher than those produced by models using our earlier meta-parameter settings (optimized for the HASOC 2020 data set alone), regardless of the number of additional data sets included.

4.2. FastText

In this section we report and discuss our results attained using the FastText classifier. Result here will be presented in a similar manner as in the previous section. One difference, however, is that for each data set combination, an additional line will be present in the paper. This line will represent the results attained when in conjunction with the FastText classifier, we also use a pretrained Word2Vec word representation.

Table 10
FastText results on the HASOC 2019 test set (scores are the average of five models; the best results, and those not significantly different – $p > 0.05$ – are emphasized in bold)

Databases Word embedding F-scores and deviations

HASOC 2020 OLID HASOC 2019 WikiNews-300D Macro- $F_{1}$ STDEV Weighted- $F_{1}$ STDEV

– – ✓ – 0.6736 0.0283 0.7462 0.0293

– – ✓ ✓ 0.7107 0.0031 0.7825 0.0053

✓ – ✓ – 0.7191 0.0076 0.7966 0.0038

✓ – ✓ ✓ 0.7358 0.0088 0.8109 0.0054

– ✓ ✓ – 0.7533 0.0037 0.8151 0.0026

– ✓ ✓ ✓ 0.7575 0.0076 0.8187 0.0078

✓ ✓ ✓ – 0.7534 0.0058 0.8182 0.0028

✓ ✓ ✓ ✓ 0.7645 0.0026 0.8256 0.0020

Databases	Word embedding	F-scores and deviations
–	–	✓	–	0.6736	0.0283	0.7462	0.0293
–	–	✓	✓	0.7107	0.0031	0.7825	0.0053
✓	–	✓	–	0.7191	0.0076	0.7966	0.0038
✓	–	✓	✓	0.7358	0.0088	0.8109	0.0054
–	✓	✓	–	0.7533	0.0037	0.8151	0.0026
–	✓	✓	✓	0.7575	0.0076	0.8187	0.0078
✓	✓	✓	–	0.7534	0.0058	0.8182	0.0028
✓	✓	✓	✓	0.7645	0.0026	0.8256	0.0020

4.2.1. HASOC 2019

The average classification scores and deviations resulting from our experiments on the HASOC 2019 data set using FastText are reported in Table 10. Here, our discussion would be two-fold. First, we would discuss the effect of using the WikiNews-300D word embeddings. This will be followed by a short discussion on the effect of additional data sets on our results. As can be seen in Table 10, regardless of the data sets used for training, the resulting classification scores on the HASOC 2019 test set are improved by the inclusion of the WikiNews-300D word vector representations. This improvement is significant (with $p < 0.005$ ) when using the HASOC corpora only for training. When using all corpora, the improvement for the weighted $F_{1}$ -score is significant with $p < 0.005$ , while for the macro $F_{1}$ -score, the significance level is $p < 0.05$ . While in case of the use of only the OLID and HASOC 2019 data sets, the improvement gained from leveraging WikiNews-300D was not significant.

When discussing the effect of additional data sets, we can see in Table 10, that regardless of whether we use WikiNews-300D, or not, it is constant that by introducing a new data set in any setting improves the classification performance of our models. The difference lies in whether this improvement is significant or not. Over the basic HASOC 2019 data set, the introduction of any additional data results in improvements being significant (with $p < 0.05$ ). Similarly, using OLID in addition to the two HASOC sets improves the resulting classification scores significantly (with $p < 0.05$ ) And while using all two additional data sets provides some improvement over adding only OLID, this improvement is not significant.

It should be noted here, that the models trained using the FastText classifier were much bigger (models saved taking up GigaBytes) than those resulting from the CNN-LSTM hybrid classification (models taking up around 100 MegaBytes). It is hence positive that by incorporating some additional training data (with less instances than the original training set), our hybrid model performed on the same level as the FastText classifier using HASOC 2019 data only. Moreover, when including more additional data, our CNN-LSTM models performed markedly better than FastText using HASOC 2019. It should also be noted, however, that the meta-parameter optimization for CNN-LSTM models took significantly longer than the ten minutes used by FastText. To examine how this affects the results, we have ran the experiments whose results are reported in the last line of Table 10 with doubled meta-parameter optimization time. The resulting classification scores (an average macro- $F_{1}$ score of 0.7667 – with a standard deviation of 0.0045 – and an average weighted- $F_{1}$ score of 0.8293 – with a standard deviation of 0.0022), however, were not significantly better than those we got with the shorter optimization time.

Table 11
FastText results on the HASOC 2020 test set (scores are the average of five models; the best result, and results not significantly different – $p > 0.05$ – are emphasized in bold)

Databases Word embedding F-scores and deviations

HASOC 2019 OLID HASOC 2020 WikiNews-300D Macro- $F_{1}$ STDEV Weighted- $F_{1}$ STDEV

– – ✓ – 0.8751 0.0023 0.8750 0.0023

– – ✓ ✓ 0.8786 0.0055 0.8786 0.0055

✓ – ✓ – 0.8780 0.0021 0.8779 0.0022

✓ – ✓ ✓ 0.8894 0.0014 0.8894 0.0014

– ✓ ✓ – 0.8827 0.0026 0.8826 0.0026

– ✓ ✓ ✓ 0.8884 0.0022 0.8884 0.0022

✓ ✓ ✓ – 0.8820 0.0028 0.8819 0.0028

✓ ✓ ✓ ✓ 0.8888 0.0017 0.8888 0.0017

Databases	Word embedding	F-scores and deviations
–	–	✓	–	0.8751	0.0023	0.8750	0.0023
–	–	✓	✓	0.8786	0.0055	0.8786	0.0055
✓	–	✓	–	0.8780	0.0021	0.8779	0.0022
✓	–	✓	✓	0.8894	0.0014	0.8894	0.0014
–	✓	✓	–	0.8827	0.0026	0.8826	0.0026
–	✓	✓	✓	0.8884	0.0022	0.8884	0.0022
✓	✓	✓	–	0.8820	0.0028	0.8819	0.0028
✓	✓	✓	✓	0.8888	0.0017	0.8888	0.0017

4.2.2. HASOC 2020

In this section we would present our results similarly to those presented in the previous section. Classification scores of our experiments are listed in Table 11 Regarding the use of WikiNews-300D, when using any additional data sets, the involvement of these pre-trained word vector representations significantly improved the resulting classification scores. When the additional data set was OLID, this improvement was significant with $p < 0.5$ , while in every other case, the improvement was sigificant with $p < 0.005$ . When using HASOC 2020 only, however, the improvement gained in classification scores by adding the WikiNews-300D representations was not significant.

When discussing the effect of additional data sets, we should have this discussion in two parts. When WikiNews-300D is not employed we only find a significant improvement (with $p < 0.005$ ) when the OLID dataset is added to the training data. When using the two HASOC training sets for training, the improvement over using only the HASOC 2020 data set is not significant. Similarly, when using both the HASOC 2019 and the OLID training sets in addition to the original HASOC 2020 training set, the improvement compared to results gotten when only adding the OLID training set to the original training set are not significant. When WikiNews-300D is also used, we see a different picture. Regardless of whether we add the training set from HASOC 2019, or OLID to the original training set of HASOC 2019, we can observe a significant improvement (with $p < 0.005$ ) in the resulting classification scores. When involving both additional training sets, however, the resulting classification scores are almost exactly the same (or slightly even lower), when compared to the classification scores attained by adding only one individual data sets. What is interesting to note here, is that though the difference is not significant, the best results in these case are obtained when using the two HASOC data sets only.

What is more, is that on the HASOC 2020 data set, the simple hybrid CNN-LSTM method outperformed not only the best performing team in the original competition [69], but the FastText classifier as well.

4.3. RoBERTa

The cross-validation experiments using RoBERTa on the HASOC 2019 data set has already been carried out, by Kovács et al. [51], and by Alonso et al. [6]. For this, we would not repeat these experiments here. These researchers, however found, that applying the same five-fold cross validation method using RoBERTA lead to state-of-the-art results on the HASOC 2019 data set. While Alonso et al. [6] further improved these results with the addition of part of the OffensEval [112] training set. This indicated that not only the use of the pre-trained RoBERTa was beneficial, but even in that case, adding additional labelled data to the process further improved the results.

We have, however carried out the cross-validation experiments on HASOC 2020. Due to the size of the model, however (the model containing more than one hundred million trainable parameters), we only trained the five different folds once. Our results showed that individual RoBERTa models trained on the different folds, on average classification score of 0.8994 (both for the macro and weighted $F_{1}$ -score – note, that this is lower than the $F_{1}$ -scores attained using a comparable number of trainable parameters in Section 4.1.2 with the CNN-LSTM architecture). When averaging the predictions of these five models, the classification performance increases to 0.9152.

4.4. Classifier ensemble

The classifier combination experiments described in Section 3.4 we carried out on the HASOC 2020 data set. We did so using the combination of the RoBERTa model (trained on the HASOC 2020 data set alone), with the optimized hybrid CNN-LSTM model (trained on the combination of all three data sets – as discussed in detail in the last paragraphs of Section 4.1.2). As for the CNN-LSTM hybrid we had ten models trained, we did so ten times, thus the final result reported being the average of ten independent results. When evaluating the resulting classifier ensemble on the HASOC 2020 test set, we attained a macro, and weighted $F_{1}$ -score of 0.9195, higher than those attained with RoBERTA alone.

5. Analysis

In this study, we carried out extensive experiments for the examination of the effect leveraging additional resources has on the detection of offensive speech in social media posts. Our aim, however extends beyond what could be summarized as discussing whether “more data is better”. For this, it is beneficial to further analyse our experiments and results that examined leveraging three sources of external data: namely, pre-trained transformer models, pre-trained word representations, and data sets resulting from collection carried out based on similar guiding principles.

5.1. Utility of pre-trained transformers

Out of the three sources of additional data leveraged, the use of pre-trained transformers is one the analysis of which we would discuss the least. As the use of these models for NLP tasks is well researched [107,108], including the effect of different pre-training objectives [56]. The inclusion of these models in our experiments was mainly motivated by the following considerations: for one, to contribute to our ability to present state-of-the-art results for the given task. Furthermore, to show the comparative performance attained using the CNN-LSTM model (at comparable trainable parameter counts). And lastly, to demonstrate that RoBERTA models can still benefit from the contribution of the aforementioned CNN-LSTM models, when those models leverage the right data (see, our ensemble results in Section 4.4.

5.2. Utility of pre-trained word representations

What we consider an important take-away for pre-trained word representations is that for each combination of data sets used in the FastText experiments, the addition of WikiNews-300D further improved the classification scores, moreover in 6 out of 8 cases, this improvement was significant (with $p < 0.05$ ).

WikiNews-300D is, however, only one embedding provided for FastText. Thus, to strengthen our conclusions, we repeated the HASOC 2020 experiments with word vectors trained on the CommonCrawl7

⁷
http://commoncrawl.org/

corpus, CommonCrawl-300D [67]. Results of these experiments, along with results attained on the same dataset without the use of pre-trained word representations, are listed in Table 12. Results show that similar to WikiNews-300D, the leveraging of CommonCrawl-300D also improved the classification scores attained with each combination of data sets. Moreover, this improvement proved to be significant in each case (with

p < 0.05

). Furthermore, when compared these results to those presented in Table 11, we can also see that the best overall results attained with FastText were those using the HASOC 2019, OLID, and HASOC 2020 data sets in combination with CommonCrawl-300D, a significant (with

p < 0.00005

) improvement over using the same combination of data sets with WikiNews-300D (a word representation containing fewer word vectors, and trained on less data).

Table 12

FastText results on the HASOC 2020 test set (scores are the average of five models; the best result, and results not significantly different – $p > 0.05$ – are emphasized in bold)

Databases			Word embedding	F-scores and deviations

HASOC 2019	OLID	HASOC 2020	CommonCrawl-300D	macro- $F_{1}$	STDEV	weighted- $F_{1}$	STDEV
–	–	✓	–	0.8751	0.0023	0.8750	0.0023
–	–	✓	✓	0.8789	0.0009	0.8789	0.0009
✓	–	✓	–	0.8780	0.0021	0.8779	0.0022
✓	–	✓	✓	0.8835	0.0024	0.8835	0.0024
–	✓	✓	–	0.8827	0.0026	0.8826	0.0026
–	✓	✓	✓	0.8963	0.0018	0.8963	0.0017
✓	✓	✓	–	0.8820	0.0028	0.8819	0.0028
✓	✓	✓	✓	0.9012	0.0025	0.9012	0.0025

Table 13

FastText results on the HASOC 2019 test set (scores are the average of five models; the best result, and results not significantly different – $p > 0.05$ – are emphasized in bold)

Databases			Word embedding	F-scores and deviations

HASOC 2019	OLID	HASOC 2020	CommonCrawl-300D	Macro- $F_{1}$	STDEV	Weighted- $F_{1}$	STDEV
-	–	✓	–	0.6736	0.0283	0.7462	0.0293
-	–	✓	✓	0.6730	0.0057	0.7373	0.0069
✓	–	✓	–	0.7191	0.0076	0.7966	0.0038
✓	–	✓	✓	0.7298	0.0026	0.7988	0.0013
-	✓	✓	–	0.7533	0.0037	0.8151	0.0026
-	✓	✓	✓	0.7641	0.0039	0.8199	0.0044
✓	✓	✓	–	0.7533	0.0058	0.8182	0.0028
✓	✓	✓	✓	0.7702	0.029	0.8274	0.0044

Table 13 presents the results attained when repeating the FastText experiments on HASOC 2019 using CommonCrawl. Here, we once again can see that the addition of CommonCrawl in most cases improves the performance, and that the best performance is attained with the inclusion of the three datasets as well as the embeddings from CommonCrawl [67].

5.3. Utility of additional labeled data sets

When discussing the utility of additional labeled data sets, it is beneficial to discuss FastText and CNN-LSTM separately. The reason is that for FastText, the use of the pre-existing package meant that the meta-parameters for models were fine-tuned for each combination of data sets. While for the CNN-LSTM model, for each combination of data set we used the same meta-parameter settings that we found optimal for the core data set (be that HASOC 2019 or HASOC 2020) alone.

5.3.1. FastText

Upon reviewing the results attained using FastText on the HASOC 2019 data set (see Section 4.2.1), we find that the addition of each data set improved the results, even if not every improvement was significant. We can also see, that the larger the added data set was, the larger the improvement was (e.g. the addition of HASOC 2020 meant an additional $\approx 3000$ tweets for each training fold, while the addition of OLID meant an additional $\approx 10500$ tweets). While in the case of HASOC 2020 data (see Section 4.2.2), we find that while the addition of OLID data in and of itself lead to a significant improvement in classification score (with $p < 0.005$ ), the same is not true for the addition of HASOC 2019 data. This may be related to some flaws in the labeling of the HASOC 2019 data set [51]. What is important, however, is when comparing the best results attained on HASOC 2020 using the various data set combinations, we find that even when the inclusion of additional data did not always lead to significant improvements in performance, it did not once lead to a significant deterioration of results either.

Table 14
Meta-parameter values chosen on the HASOC 2019 data set for CNN-LSTM architecture using the different data set combinations for training

Parameter description HASOC 2019 HASOC 2020 OLID HASOC 2020

HASOC 2019 HASOC 2019 OLID

HASOC 2019

Size of batch during training 64 32 64 64

Size of the vocabulary 30000 40000 40000 40000

Dimension for token embedding 350 300 400 400

Number of filters in the 1st CNN layer 128 64 128 128

Number of filters in the 2nd CNN layer 64 32 64 64

Size of pooling operation applied 4 4 4 4

Number of cells in the 1st BiLSTM layer 600 800 800 800

Number of cells in the 2nd BiLSTM layer 0 0 400 400

Number of cells in the 3rd BiLSTM layer 0 0 0 0

Number of neurons in the 1st dense layer 600 400 200 200

Number of neurons in the 2nd dense layer 0 200 0 0

Number of neurons in the 3rd dense layer 0 100 0 0

Rate of dropout 0.2 0.4 0.5 0.5

Initial learning rate 0.001 0.0005 0.0005 0.0005

Number of parameters (in millions) 14.8 18.5 28.5 28.5

Parameter description	HASOC 2019	HASOC 2020	OLID	HASOC 2020
Size of batch during training	64	32	64	64
Size of the vocabulary	30000	40000	40000	40000
Dimension for token embedding	350	300	400	400
Number of filters in the 1st CNN layer	128	64	128	128
Number of filters in the 2nd CNN layer	64	32	64	64
Size of pooling operation applied	4	4	4	4
Number of cells in the 1st BiLSTM layer	600	800	800	800
Number of cells in the 2nd BiLSTM layer	0	0	400	400
Number of cells in the 3rd BiLSTM layer	0	0	0	0
Number of neurons in the 1st dense layer	600	400	200	200
Number of neurons in the 2nd dense layer	0	200	0	0
Number of neurons in the 3rd dense layer	0	100	0	0
Rate of dropout	0.2	0.4	0.5	0.5
Initial learning rate	0.001	0.0005	0.0005	0.0005
Number of parameters (in millions)	14.8	18.5	28.5	28.5

Table 15

Results attained on CNN-LSTM models with optimized parameters on the HASOC 2019 test set (scores are the average of five models; the best results, and those not significantly different – $p > 0.05$ – are emphasized in bold)

Databases			F-scores and deviations

HASOC 2020	OLID	HASOC 2019	Macro- $F_{1}$	STDEV	Weighted- $F_{1}$	STDEV
–	–	✓	0.6557	0.0118	0.7217	0.0148
✓	–	✓	0.6909	0.0098	0.7573	0.0113
–	✓	✓	0.7365	0.0013	0.7990	0.0110
✓	✓	✓	0.7464	0.0091	0.8091	0.0073

5.3.2. CNN-LSTM

As we saw in Tables 7, and 8, the best results were attained when adding the larger (OLID) data set, and the inclusion of an additional data set on top of that did not significantly improve the results (and in Table 7, it even – albeit not significantly – worsened classification scores). In Section 4.1.1 we hypothesised that this could be due to having used the same meta-parameters for all data set combinations. To test this hypothesis, we repeated the experiments on the different data sets, first optimizing the meta-parameters for each data set combination. The meta-parameters found are listed in Table 14, while the resulting $F_{1}$ -scores are listed in Table 15. Our results show that while the improvements are not significant for all additional data, for each data set added there is a marked improvement achieved. Overall, by optimizing the meta-parameters, we improved the results for all data set combinations. Moreover, this change also meant that while earlier (see Table 15) the best result was attained when the CNN-LSTM model was trained on the combination of only two data sets, here, we get the best results when training on all three data sets (with a difference of 0.001). What can be discouraging in this result is that it suggests that one would have to repeat the meta-parameter optimization for all different settings. It is important to note, however, is that for the two combinations with the most data, we found the same meta-parameter combination to perform the best.

We repeated the same experiments for the HASOC 2020 data set, and found very similar results (see Table 16 for the meta-parameters and Table 17 for the classification scores). Again we find that when the meta-parameters are optimized, the best results are attained when all data sets are combined for the training of the CNN-LSTM networks. Moreover, when using similar sizes of data set combinations, the meta-parameters selected as performing best (during cross-validation) are similar.

Table 16
Meta-parameter values chosen on the HASOC 202020 data set for CNN-LSTM architecture using the different data set combinations for training

Parameter description HASOC 2020 HASOC 2019 OLID HASOC 2019

HASOC 2020 HASOC 2020 OLID

HASOC 2020

Size of batch during training 32 32 64 64

Size of the vocabulary 30000 30000 50000 50000

Dimension for token embedding 250 250 400 400

Number of filters in the 1st CNN layer 64 64 128 128

Number of filters in the 2nd CNN layer 32 32 32 32

Size of pooling operation applied 4 4 4 4

Number of cells in the 1st BiLSTM layer 800 800 600 600

Number of cells in the 2nd BiLSTM layer 0 0 0 0

Number of cells in the 3rd BiLSTM layer 0 0 0 0

Number of neurons in the 1st dense layer 400 400 600 600

Number of neurons in the 2nd dense layer 200 200 200 200

Number of neurons in the 3rd dense layer 0 0 100 100

Rate of dropout 0.5 0.5 0.4 0.4

Initial learning rate 0.001 0.001 0.001 0.001

Number of parameters (in millions) 13.8 13.8 24.9 24.9

Parameter description	HASOC 2020	HASOC 2019	OLID	HASOC 2019
Size of batch during training	32	32	64	64
Size of the vocabulary	30000	30000	50000	50000
Dimension for token embedding	250	250	400	400
Number of filters in the 1st CNN layer	64	64	128	128
Number of filters in the 2nd CNN layer	32	32	32	32
Size of pooling operation applied	4	4	4	4
Number of cells in the 1st BiLSTM layer	800	800	600	600
Number of cells in the 2nd BiLSTM layer	0	0	0	0
Number of cells in the 3rd BiLSTM layer	0	0	0	0
Number of neurons in the 1st dense layer	400	400	600	600
Number of neurons in the 2nd dense layer	200	200	200	200
Number of neurons in the 3rd dense layer	0	0	100	100
Rate of dropout	0.5	0.5	0.4	0.4
Initial learning rate	0.001	0.001	0.001	0.001
Number of parameters (in millions)	13.8	13.8	24.9	24.9

Table 17

Results attained on CNN-LSTM models with optimized parameters on the HASOC 2020 test set (scores are the average of five models; the best results, and those not significantly different – $p > 0.05$ – are emphasized in bold)

Databases			F-scores and deviations

HASOC 2019	OLID	HASOC 2020	Macro- $F_{1}$	STDEV	Weighted- $F_{1}$	STDEV
–	–	✓	0.8864	0.0030	0.8863	0.0030
✓	–	✓	0.8927	0.0056	0.8927	0.0056
–	✓	✓	0.9066	0.0016	0.9066	0.0016
✓	✓	✓	0.9096	0.0018	0.9096	0.0018

5.3.3. Summary

Overall, what we can say regarding the utility of additional labeled data sets for the classification of offensive speech is that with the right selection of data sets (based on their descriptions), for each data set combination, we improved the performance of the CNN-LSTM model with the inclusion of additional data, while attained an improved or significantly not different classification score with FastText. Given the circumstances, we can only hypothesize on how to avoid worsening the performance, and that is the careful selection of data sources, and the use of well-selected validation data. To eliminate the need for careful selection, however, our goal is to automatize the inclusion of additional data (as is also discussed in Section 6).

To provide a visual aid for the summary above, we have created a plot to visualize the utility of each data set (see Fig. 2). Here, for each data set, we compared the results of experiments where the only difference was the presence or absence of the data set in question. We did so for experiments with CNN-LSTM where meta-parameters were optimized for the given data set combination (Tables 15 and 17), as well as for all FastText experiments (Tables 10, 11, 13 and 12). Figure 2 shows for each data set in what percent of the cases the addition of the data set in question led to:

significantly improved classification scores (both for macro and weighted $F_{1}$ -score).

classification scores not significantly different than hose achieved without the additional data set.

As can be seen in Fig. 2, in our experiments, 37.50% of the time (3 out of 8 cases), the addition of HASOC 2019 data set significantly improved the performance while in 62.50% of the time there was no significant difference in performance. Similarly, the addition of the HASOC 2020 data set significantly improved the performance 62.50% of the time (5 out of 8 experiments). The addition of the largest (OLID) data set, however, improved the performance significantly 100% of the time (16 times out of the 16 pairs of experiments examined).

Fig. 2.

The utility of each dataset in several experiments where dataset addition improved the performance significantly/insignificantly.

6. Conclusions and future work

Here, following the work of Alonso et al. [5,6] and Kovács et al. [51], we have examined what effect the use of various additional data sources has on the offensive language classification problem, using the two concluded HASOC competitions. This work, however, goes beyond [5,6,51] by (i) adding new data, (ii) having a more complete search space of datasets, word representations, and machine learning models (while previous work only investigates some of these aspects), and (iii) achieving state-of-the-art results on HASOC 2020. The additional dataset added was HASOC 2020 [60], which was made available after the initial submission of the papers listed above. The addition of this dataset enabled us to examine the effect of using data collected by the same researchers at a different time. It also enabled us to examine a more complete search space by extending the number of possible combinations. Another way for us to extend the search space was by including an additional pre-trained word-vector representation trained on CommonCrawl data [67]. The results of experiments carried out in this search space lead to state-of-the-art results on the HASOC 2020 dataset, as well as demonstrated the benefits of using additional labelled corpora. This was the case both when the data collection was carried out by the same people, as well as in the case when data collection was carried out by different people, but based on similar principles. In fact, in most cases better results were achieved when data sets collected by different researchers were combined. One potential explanation for this may be the difference in the size of the data sets.

In the future, our work can be extended to additional labelled data sets (since the initial submission of our paper, HASOC 2021 [63] has been organised and concluded, and at the time of the submission, the organisation of HASOC 20228

⁸

https://hasocfire.github.io/hasoc/2022/index.html

is already underway). We also plan to examine whether it would be possible to use the models trained on these data sets later for the extension of existing data sets by labelling additional tweets, and including them in the training set. Another future direction can be to combine the embedding layer we train during the process with pre-trained word embeddings, thus extending the dimensionality of the input data for the convolutional layer, which then would be replaced by two-dimensional convolution. We also plan to focus more on more granular classification tasks, as well as training explainable models. Lastly, as novel transformer architectures are being introduced, we also plan to examine the performance of these [84] on the task of detecting hateful and offensive speech.

References

Adadi , A survey on data-efficient algorithms in big data era, Journal of Big Data 8 (2021), 1–54. doi:10.1186/s40537-020-00387-6.

Alkiviadou , The legal regulation of hate speech: The international and European frameworks, Politička misao 55 (2018), 203–229. doi:10.20901/pm.55.4.08.

Alkiviadou , Hate speech on social media networks: Towards a regulatory framework?, Information & Communications Technology Law 28(1) (2019), 19–35. doi:10.1080/13600834.2018.1494417.

Allcott and

Gentzkow , Social media and fake news in the 2016 election, Journal of Economic Perspectives 31(2) (2017), 211–236. doi:10.1257/jep.31.2.211.

Alonso ,

Saini and

Kovács , TheNorth at HASOC 2019 hate speech detection in social media data, in: Proceedings of the 11th Annual Meeting of the Forum for Information Retrieval Evaluation, 2019.

Alonso ,

Saini and

Kovács , Hate speech detection using transformer ensembles on the HASOC dataset, in: Speech and Computer: 22nd International Conference, SPECOM 2020, Proceedings, Vol. 12335, Springer Nature, St. Petersburg, Russia, 2020, pp. 7–9.

Arango ,

Pérez and

Poblete , Hate speech detection is not as easy as you May think: A closer look at model validation, in: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’19, Association for Computing Machinery, New York, NY, USA, 2019, pp. 45–54. ISBN 9781450361729. doi:10.1145/3331184.3331262.

Badjatiya ,

Gupta ,

Gupta and

Varma , Deep learning for hate speech detection in tweets, in: Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion, 2017, pp. 759–760. doi:10.1145/3041021.3054223.

Bahdanau ,

Cho and

Bengio , Neural machine translation by jointly learning to align and translate, in: 3rd International Conference on Learning Representations, ICLR 2015,

Bengio and

LeCun , eds, Conference Track Proceedings, San Diego, CA, USA, May 7–9, 2015, 2015, http://arxiv.org/abs/1409.0473.

10.

Barendt , What is the harm of hate speech?, Ethic theory, moral prac 22 (2019), 539–553. doi:10.1007/s10677-019-10002-0.

11.

Basile ,

Bosco ,

Fersini ,

Nozza ,

Patti ,

F.M.

Rangel Pardo ,

Rosso and

Sanguinetti , SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter, in: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019, pp. 54–63, https://www.aclweb.org/anthology/S19-2007. doi:10.18653/v1/S19-2007.

12.

Bianchi , Slurs and appropriation: An echoic account, Journal of Pragmatics 66 (2014), 35–44. doi:10.1016/j.pragma.2014.02.009.

13.

Bojanowski ,

Grave ,

Joulin and

Mikolov , Enriching Word Vectors with Subword Information, 2016, arXiv preprint arXiv:1607.04606.

14.

Brown , What is so special about online (as compared to offline) hate speech?, Ethnicities 18(3) (2018), 297–326. doi:10.1177/1468796817709846.

15.

Burnap and

M.L.

Williams , Cyber hate speech on Twitter: An application of machine classification and statistical modeling for policy and decision making, Policy & Internet 7(2) (2015), 223–242. doi:10.1002/poi3.85.

16.

W.-Y.S.

Chou ,

Y.M.

Hunt ,

E.B.

Beckjord ,

R.P.

Moser and

B.W.

Hesse , Social media use in the United States: Implications for health communication, J Med Internet Res 11(4) (2009). doi:10.2196/jmir.1249.

17.

Commission , Code of Conduct on Countering Illegal Hate Speech Online, 2016, https://ec.europa.eu/commission/presscorner/detail/en/MEMO_19_806.

18.

Davidson ,

Bhattacharya and

Weber , Racial bias in hate speech and abusive language detection datasets, in: Proceedings of the Third Workshop on Abusive Language Online, Association for Computational Linguistics, Florence, Italy, 2019, pp. 25–35, https://www.aclweb.org/anthology/W19-3504. doi:10.18653/v1/W19-3504.

19.

Davidson ,

Warmsley ,

Macy and

Weber , Automated hate speech detection and the problem of offensive language, in: Proceedings of the 11th International AAAI Conference on Web and Social Media, ICWSM ’17, 2017, pp. 512–515.

20.

Del Vigna ,

Cimino ,

Dell’Orletta ,

Petrocchi and

Tesconi , Hate me, hate me not: Hate speech detection on Facebook, in: ITASEC, 2017.

21.

Devlin ,

M.-W.

Chang ,

Lee and

Toutanova , BERT: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 4171–4186, https://www.aclweb.org/anthology/N19-1423. doi:10.18653/v1/N19-1423.

22.

Djuric ,

Zhou ,

Morris ,

Grbovic ,

Radosavljevic and

Bhamidipati , Hate speech detection with comment embeddings, in: Proceedings of the 24th International Conference on World Wide Web, WWW ’15 Companion, Association for Computing Machinery, New York, NY, USA, 2015, pp. 29–30. ISBN 9781450334730. doi:10.1145/2740908.2742760.

23.

H.T.-T.

Do ,

H.D.

Huynh ,

K.V.

Nguyen ,

N.L.-T.

Nguyen and

A.G.-T.

Nguyen , Hate Speech Detection on Vietnamese Social Media Text Using the Bidirectional-LSTM Model, 2019, http://arxiv.org/abs/1911.03648.

24.

Dworkin , A new map of censorship, Index on Censorship 35(1) (2006), 130–133. doi:10.1080/03064220500532412.

25.

Ferrara , Disinformation and social bot operations in the run up to the 2017 French presidential election, First Monday 22 (2017). doi:10.5210/fm.v22i8.8005.

26.

Fortuna and

Nunes , A survey on automatic detection of hate speech in text, ACM Comput. Surv. 51(4) (2018), 85–18530, ISSN 0360-0300. doi:10.1145/3232676.

27.

Fortuna ,

Soler Company and

Wanner , How well do hate speech, toxicity, abusive and offensive language classification models generalize across datasets?, Inf. Process. Manag. 58(3) (2021), 102524. doi:10.1016/j.ipm.2021.102524.

28.

Framework Decision on combating certain forms and expressions of racism and xenophobia by means of criminal law, 2008.

29.

Free Word Centre, Responding to ‘hate speech’ with positive measures: A case study from six EU countries, Technical Report, 2018. https://www.article19.org/wp-content/uploads/2018/06/Responding-to-%E2%80%98hate-speech%E2%80%99-with-positive-measures-A-case-study-from-six-EU-countries-.pdf.

30.

Gambäck and

U.K.

Sikdar , Using convolutional neural networks to classify hate-speech, in: Proceedings of the First Workshop on Abusive Language Online, Association for Computational Linguistics, Vancouver, BC, Canada, 2017, pp. 85–90, https://www.aclweb.org/anthology/W17-3013. doi:10.18653/v1/W17-3013.

31.

Gascó ,

Rocha ,

Sanchis-Trilles ,

Andrés-Ferrer and

Casacuberta , Does more data always yield better translations? in: EACL, 13th Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France, April 23–27,

Daelemans ,

Lapata and

Màrquez , eds, The Association for Computer Linguistics, 2012, pp. 152–161, https://www.aclweb.org/anthology/E12-1016/.

32.

Gelashvili , Hate Speech on Social Media: Implications of private regulation and governance gaps, Master’s thesis, Lund University, Sweden, 2018.

33.

Gelber

The Content and Context of Hate Speech: Rethinking Regulation and Responses, Cambridge University Press, 2012. doi:10.1017/CBO9781139042871.

34.

Gelber and

McNamara , Evidencing the harms of hate speech, Social Identities 22 (2016), 324–341. doi:10.1080/13504630.2015.1128810.

35.

Grave ,

Bojanowski ,

Gupta ,

Joulin and

Mikolov , Learning word vectors for 157 languages, in: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), 2018.

36.

Greevy and

A.F.

Smeaton , Classifying racist texts using a support vector machine, in: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’04, Association for Computing Machinery, New York, NY, USA, 2004, pp. 468–469. ISBN 1581138814. doi:10.1145/1008992.1009074.

37.

Gröndahl ,

Pajola ,

Juuti ,

Conti and

Asokan , All you need is “love”: Evading hate speech detection, in: Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security, AISec ’18, Association for Computing Machinery, New York, NY, USA, 2018, pp. 2–12. ISBN 9781450360043. doi:10.1145/3270101.3270103.

38.

Guo and

B.G.

Johnson , Third-person effect and hate speech censorship on Facebook, Social Media + Society 6(2) (2020), 2056305120923003. doi:10.1177/2056305120923003.

39.

Hare and

Weinstein , Extreme Speech and Democracy, Oxford University Press, Oxford; New York, 2009. ISBN 9780199548781.

40.

Hern , Revealed: Catastrophic effects of working as a Facebook moderator, The Guardian (2019), Accessed on 2020-04-26, https://www.theguardian.com/technology/2019/sep/17/revealed-catastrophic-effects-working-facebook-moderator.

41.

Heyman , Hate speech, public discourse, and the first amendment, in: Extreme Speech and Democracy,

Hare and

Weinstein , eds, Oxford Scholarship Online, 2009. doi:10.1093/acprof:oso/9780199548781.003.0010.

42.

Hom , A puyyle about pejoratives, Philos Stud 159 (2012), 383–405. doi:10.1007/s11098-011-9749-7.

43.

P.N.

Howard and

Kollanyi , Bots, #StrongerIn, and #Brexit: Computational propaganda during the UK-EU referendum, SSRN Electronic Journal (2016). doi:10.2139/ssrn.2798311.

44.

Howie , Protecting the human right to freedom of expression in international law, International Journal of Speech-Language Pathology 20(1) (2018), 12–15, PMID: 29124945. doi:10.1080/17549507.2018.1392612.

45.

T.V.

Huynh ,

V.D.

Nguyen ,

K.V.

Nguyen ,

N.L.-T.

Nguyen and

A.G.-T.

Nguyen , Hate Speech Detection on Vietnamese Social Media Text Using the Bi-GRU-LSTM-CNN Model, ArXiv abs/1911.03644, 2019.

46.

Impermium, Detecting Insults in Social Commentary, Accessed on 2020-04–2020-27, https://kaggle.com/c/detecting-insults-in-social-commentary.

47.

A.C.

Isasi and

Juanatey , Hate speech in social media: A state-of-the-art review, 2017.

48.

Jiao ,

Yin ,

Shang ,

Jiang ,

Chen ,

Li ,

Wang and

Liu , TinyBERT: Distilling BERT for Natural Language Understanding, 2020, https://openreview.net/forum?id=rJx0Q6EFPB.

49.

Joulin ,

Grave ,

Bojanowski and

Mikolov , Bag of tricks for efficient text classification, in: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Association for Computational Linguistics, Valencia, Spain, 2017, pp. 427–431, https://www.aclweb.org/anthology/E17-2068.

50.

J.Y.

Kim ,

Ortiz ,

Nam ,

Santiago and

Datta , Intersectional Bias in Hate Speech and Abusive Language Datasets, 2020.

51.

Kovács ,

Alonso and

Saini , Challenges of hate speech detection in social media, SN Comput. Sci. 2(2) (2021), 95. doi:10.1007/s42979-021-00457-3.

52.

Kovács ,

Grósz and

Váradi , Topical unit classification using deep neural nets and probabilistic sampling, in: 2016 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), IEEE, 2016, pp. 000199–000204. doi:10.1109/CogInfoCom.2016.7804549.

53.

Kumar ,

A.K.

Ojha ,

Malmasi and

Zampieri , Benchmarking aggression identification in social media, in: Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), 2018, pp. 1–11.

54.

Kwok and

Wang , Locate the hate: Detecting tweets against blacks, in: Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI’13, AAAI Press, 2013, pp. 1621–1622.

55.

Lan ,

Chen ,

Goodman ,

Gimpel ,

Sharma and

Soricut , Albert: A lite BERT for self-supervised learning of language representations, in: International Conference on Learning Representations, 2020, https://openreview.net/forum?id=H1eA7AEtvS.

56.

L.D.

Liello ,

Gabburo and

Moschitti , Efficient pre-training objectives for Transformers, 2021, CoRR, https://arxiv.org/abs/2104.09694 arXiv:2104.09694.

57.

Liu ,

Ott ,

Goyal ,

Du ,

Joshi ,

Chen ,

Levy ,

Lewis ,

Zettlemoyer and

Stoyanov , RoBERTa: A Robustly Optimized BERT Pretraining Approach, 2019.

58.

Liu ,

Ott ,

Goyal ,

Du ,

Joshi ,

Chen ,

Levy ,

Lewis ,

Zettlemoyer and

Stoyanov , RoBERTa: A Robustly Optimized BERT Pretraining Approach, ArXiv (2019), http://arxiv.org/abs/1907.11692.

59.

MacAvaney ,

H.-R.

Yao ,

Yang ,

Russell ,

Goharian and

Frieder , Hate speech detection: Challenges and solutions, PLOS ONE 14(8) (2019), 1–16. doi:10.1371/journal.pone.0221152.

60.

Mandl ,

Modha ,

Kumar M and

B.R.

Chakravarthi , Overview of the HASOC track at FIRE 2020: Hate speech and offensive language identification in tamil, malayalam, Hindi, English and German, in: Forum for Information Retrieval Evaluation, FIRE 2020, Association for Computing Machinery, New York, NY, USA, 2020, pp. 29–32. ISBN 9781450389785. doi:10.1145/3441501.3441517.

61.

Mandl ,

Modha ,

Mandlia ,

Patel ,

Patel and

Dave , HASOC – Hate Speech and Offensive Content Identification in Indo-European Languages, Accessed, pp. 2019–09–2019-20.

62.

Mandl ,

Modha ,

Patel ,

Dave ,

Mandlia and

Patel , Overview of the HASOC track at FIRE 2019: Hate speech and offensive content identification in indo-European languages, in: Proceedings of the 11th Annual Meeting of the Forum for Information Retrieval Evaluation, 2019.

63.

Mandl ,

Modha ,

G.K.

Shahi ,

Madhu ,

Satapara ,

Majumder ,

Schäfer ,

Ranasinghe ,

Zampieri ,

Nandini and

A.K.

Jaiswal , Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages, CoRR Abs/2112.09301, 2021, https://arxiv.org/abs/2112.09301.

64.

M.J.

Matsuda , Public response to racist spech: Considering the victim’s story, in: Words That Wound: Critical Race Theory, Assaultive Speech, and the First Amendment,

R.D.M.J.

Matsuda ,

C.R.

Lawrence III. and

Williams , eds, Routledge, New York, 1993, pp. 17–52.

65.

McHugh , Interrater reliability: The kappa statistic, biochemia medica: časopis Hrvatskoga društva medicinskih biokemičara /, HDMB 22 (2012), 276–282. doi:10.11613/BM.2012.031.

66.

Mehdad and

Tetreault , Do characters abuse more than words? in: Proceedings of the SIGDIAL2016 Conference, 2016, pp. 299–303. doi:10.18653/v1/W16-3638.

67.

Mikolov ,

Grave ,

Bojanowski ,

Puhrsch and

Joulin , Advances in pre-training distributed word representations, in: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), 2018.

68.

Mikolov ,

Sutskever ,

Chen ,

Corrado and

Dean , Distributed representations of words and phrases and their compositionality, in: Proc. NIPS, 2013, pp. 3111–3119.

69.

A.K.

Mishra ,

Saumya and

Kumar , IIIT_DWDHASOC 2020: Identifying offensive content in multitask indo-European languages, in: FIRE (Working Notes), 2020.

70.

Mondal ,

L.A.

Silva and

Benevenuto , A measurement study of hate speech in social media, in: Proceedings of the 28th ACM Conference on Hypertext and Social Media, HT ’17, Association for Computing Machinery, New York, NY, USA, 2017, pp. 85–94. ISBN 9781450347082. doi:10.1145/3078714.3078723.

71.

Müller and

Schwarz , Fanning the flames of hate: Social media and hate crime, SSRN Electronic Journal (2017). doi:10.2139/ssrn.3082972.

72.

Nina-Alcocer , Vito at HASOC 2019: Detecting hate speech and offensive content through ensembles, in: Working Notes of FIRE 2019 – Forum for Information Retrieval Evaluation, Kolkata, India, December 12–15, 2019,

Mehta ,

Rosso ,

Majumder and

Mitra , eds, CEUR Workshop Proceedings, Vol. 2517, CEUR-WS.org, 2019, pp. 214–220, http://ceur-ws.org/Vol-2517/T3-5.pdf.

73.

Njagi ,

Zuping ,

Hanyurwimfura and

Long , A lexicon-based approach for hate speech detection, International Journal of Multimedia and Ubiquitous Engineering 10 (2015), 215–230. doi:10.14257/ijmue.2015.10.4.21.

74.

Nobata ,

Tetreault ,

Thomas ,

Mehdad and

Chang , Abusive language detection in online user content, in: Proceedings of the 25th International Conference on World Wide Web, WWW ’16, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 2016, pp. 145–153. ISBN 9781450341431. doi:10.1145/2872427.2883062.

75.

Nourbakhsh ,

Vermeer ,

Wiltvank and

van der , Goot, sthruggle at SemEval-2019 task 5: An ensemble approach to hate speech detection, in: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019, pp. 484–488, https://www.aclweb.org/anthology/S19-2086. doi:10.18653/v1/S19-2086.

76.

A.O.

Ohieku and

S.S.

Sabo , Journalism practice in an era of unguided utterances: Framing of hate speech in selected nigerian newspapers, Nigeria Interdisciplinary Journal of Communication Studies 24(1) (2020), 129–140.

77.

O’Regan , Hate speech online: An (intractable) contemporary challenge?, Current Legal Problems 71(1) (2018), 403–429, ISSN 0070-1998. doi:10.1093/clp/cuy012.

78.

D.W.

Otter ,

J.R.

Medina and

J.K.

Kalita , A survey of the usages of deep learning for natural language processing, in: IEEE Transactions on Neural Networks and Learning Systems, 2020, pp. 1–21.

79.

Park and

Fung , One-step and two-step classification for abusive language detection on Twitter, in: ALW1: 1st Workshop on Abusive Language Online, 2017.

80.

Pennington ,

Socher and

C.D.

Manning , GloVe: Global vectors for word representation, in: Proc. EMNLP, 2014, pp. 1532–1543, http://www.aclweb.org/anthology/D14-1162.

81.

Popa-Wyatt and

J.L.

Wyatt , Slurs, roles and power, Philos Stud 175 (2018), 2879–2906. doi:10.1007/s11098-017-0986-2.

82.

Radford and

Narasimhan , Improving Language Understanding by Generative Pre-Training, 2018.

83.

Raehtka , Recognizing The Evolution of Racial Slurs, Democrat & Chronicle (2014), Accessed on 2020-04–2020-28. https://eu.democratandchronicle.com/story/editorial/2014/01/29/recognizing-the-evolution-of-racial-slurs/5017955/.

84.

Raffel ,

N.M.

Shazeer ,

Roberts ,

Lee ,

Narang ,

Matena ,

Zhou ,

Li and

P.J.

Liu , Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, ArXiv (2020), arXiv:1910.10683.

85.

Ranasinghe and

Zampieri , Multilingual offensive language identification for low-resource languages, ACM Trans. Asian Low-Resour. Lang. Inf. Process. 21(1) (2021), ISSN 2375-4699. doi:10.1145/3457610.

86.

Ross ,

Rist ,

Carbonell ,

Cabrera ,

Kurowsky and

Wojatzki , Measuring the reliability of hate speech annotations: The case of the European refugee crisis, in: Proceedings of NLP4CMC III,

Beißwenger ,

Wojatzki and

Zesch , eds, 2016, pp. 6–9. doi:10.17185/duepublico/42132.

87.

Saha ,

Chandrasekharan and

De Choudhury , Prevalence and psychological effects of hateful speech in online college communities, in: Proceedings of the 10th ACM Conference on Web Science, WebSci ’19, Association for Computing Machinery, New York, NY, USA, 2019, pp. 255–264. ISBN 9781450362023. doi:10.1145/3292522.3326032.

88.

Salton and

Buckley , Term-weighting approaches in automatic text retrieval, Inf. Process. Manag. 24(5) (1988), 513–523, http://dblp.uni-trier.de/db/journals/ipm/ipm24.html#SaltonB88. doi:10.1016/0306-4573(88)90021-0.

89.

Sanguinetti,

Comandini,

di Nuovo,

Frenda,

Stranisci,

Bosco,

Caselli,

Patti and

Russo, HaSpeeDe 2 @ EVALITA2020: Overview of the EVALITA 2020 Hate Speech Detection Task, in: Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020), 17-12-2020,

Basile,

Croce,

Di Maro and

Passaro, eds, Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, EVALITA 2020, Vol. 2765, CEUR Workshop Proceedings (CEUR-WS.org), 2020.

90.

Sanh ,

Debut ,

Chaumond and

Wolf , 2019, DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter.

91.

Sap ,

Card ,

Gabriel ,

Choi and

N.A.

Smith , The risk of racial bias in hate speech detection, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 1668–1678, https://www.aclweb.org/anthology/P19-1163. doi:10.18653/v1/P19-1163.

92.

Scheffler , The inherent danger of hate speech legislation, 2015, http://library.fes.de/pdf-files/bueros/africa-media/12462.pdf. ISBN 978-99945-77-30-9.

93.

Seganti ,

Sobol ,

Orlova ,

Kim ,

Staniszewski ,

Krumholc and

Koziel , NLPR@SRPOL at SemEval-2019 task 6 and task 5: Linguistically enhanced deep learning offensive sentence classifier, in: SemEval@NAACL-HLT, 2019.

94.

S.J.

Shapiro , Comparing free speech: United States v. United Kingdom, Law Forum 19(2) (1989), 17–29.

95.

Spertus , Smokey: Automatic recognition of hostile messages, in: Proceedings of IAAI-97, the 9th Conference on Innovative Application of Artificial Intelligence, Proceedings of Iaai-97, the 9th Conference on Innovative Application of Artificial Intelligence Edn, 1997, pp. 1058–1065, https://www.microsoft.com/en-us/research/publication/smokey-automatic-recognition-of-hostile-messages/.

96.

Sun ,

Qiu ,

Xu and

Huang , How to Fine-Tune BERT for Text Classification? 2020.

97.

Topidi , Words That Hurt (2): National and International Perspectives on Hate Speech Regulation., ECMI Working Papers, 2019.

98.

Ullmann and

Tomalin , Quarantining online hate speech: Technical and ethical perspectives, Ethics and Information Technology (2019). doi:10.1007/s10676-019-09516-z.

99.

Vaswani ,

Shazeer ,

Parmar ,

Uszkoreit ,

Jones ,

A.N.

Gomez ,

Ł.

Kaiser and

Polosukhin , Attention is all you need, Advances in Neural Information Processing Systems 2017-Decem(Nips) (2017), 5999–6009, ISSN 10495258.

100.

Waldron , The Harm in Hate Speech, Harward University Press, Cambridge, 2012.

101.

Wang ,

Ding ,

Liu and

Zhou , YNU_Wb at HASOC 2019: Ordered neurons LSTM with attention for identifying hate speech and offensive language, in: Working Notes of FIRE 2019 – Forum for Information Retrieval Evaluation, Kolkata, India, December 12–15, 2019,

Mehta ,

Rosso ,

Majumder and

Mitra , eds, CEUR Workshop Proceedings, Vol. 2517, CEUR-WS.org, 2019, pp. 191–198, http://ceur-ws.org/Vol-2517/T3-2.pdf.

102.

Warner and

Hirschberg , Detecting hate speech on the World Wide Web, in: Proceedings of the Second Workshop on Language in Social Media, Association for Computational Linguistics, Montréal, Canada, 2012, pp. 19–26, https://www.aclweb.org/anthology/W12-2103.

103.

Waseem , Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter, in: Proceedings of the First Workshop on NLP and Computational Social Science, Association for Computational Linguistics, Austin, Texas, 2016, pp. 138–142, https://www.aclweb.org/anthology/W16-5618.

104.

Waseem and

Hovy , Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter, in: Proceedings of the NAACL Student Research Workshop, Association for Computational Linguistics, San Diego, California, 2016, pp. 88–93. doi:10.18653/v1/N16-2013.

105.

Wiegand ,

Siegel and

Ruppenhofer , Overview of the germeval 2018 shared task on the identification of offensive language, in: Proceedings of the GermEval 2018 Workshop, 2018, pp. 1–11. doi:10.1553/0x003a105d.

106.

Wolf ,

Debut ,

Sanh ,

Chaumond ,

Delangue ,

Moi ,

Cistac ,

Rault ,

Louf ,

Funtowicz and

Brew , HuggingFace’s Transformers: State-of-the-Art Natural Language Processing, ArXiv arXiv:1910.03771, 2019.

107.

Wolf ,

Debut ,

Sanh ,

Chaumond ,

Delangue ,

Moi ,

Cistac ,

Rault ,

Louf ,

Funtowicz ,

Davison ,

Shleifer ,

von Platen ,

Ma ,

Jernite ,

Plu ,

Xu ,

Le Scao ,

Gugger ,

Drame ,

Lhoest and

Rush , Transformers: State-of-the-Art Natural Language Processing, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Association for Computational Linguistics, Online, 2020, pp. 38–45. doi:10.18653/v1/2020.emnlp-demos.6.

108.

Young ,

Hazarika ,

Poria and

Cambria , Recent Trends in Deep Learning Based Natural Language Processing, 2017, Comment: Added BERT, ELMo, Transformer. http://arxiv.org/abs/1708.02709.

109.

Yuan ,

Wu and

Xiang , A two phase deep learning model for identifying discrimination from tweets, in: Proc. EDBT 2016, Bordeaux, France, March 15–16, 2016,

Pitoura ,

Maabout ,

Koutrika ,

Marian ,

Tanca ,

Manolescu and

Stefanidis , eds, OpenProceedings.org, 2016, pp. 696–697. doi:10.5441/002/edbt.2016.92.

110.

Zampieri ,

Malmasi ,

Nakov ,

Rosenthal ,

Farra and

Kumar , SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval 2019), in: Proceedings of the 13th International Workshop on Semantic Evaluation, 2019, pp. 75–86. doi:10.18653/v1/S19-2010.

111.

Zampieri ,

Malmasi ,

Nakov ,

Rosenthal ,

Farra and

Kumar , Predicting the type and target of offensive posts in social media, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 1415–1420. doi:10.18653/v1/N19-1144.

112.

Zampieri ,

Nakov ,

Rosenthal ,

Atanasova ,

Karadzhov ,

Mubarak ,

Derczynski ,

Pitenis and

Çöltekin , SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020), in: Proceedings of SemEval, 2020.

113.

Zhang and

Luo , Hate speech detection: A solved problem? The challenging case of long tail on Twitter, Semantic Web 10 (2018), 925–945. doi:10.3233/SW-180338.

114.

Zhang ,

Robinson and

Tepper , Detecting hate speech on Twitter using a convolution-GRU based deep neural network, in: The Semantic Web,

Gangemi ,

Navigli ,

M.-E.

Vidal ,

Hitzler ,

Troncy ,

Hollink ,

Tordai and

Alam , eds, Springer International Publishing, Cham, 2018, pp. 745–760. ISBN 978-3-319-93417-4. doi:10.1007/978-3-319-93417-4_48.

115.

Zhu ,

Vondrick ,

Ramanan and

C.C.

Fowlkes , Do we need more training data or better models for object detection? in: British Machine Vision Conference, BMVC 2012, Surrey, UK, September 3–7, 2012,

Bowden ,

J.P.

Collomosse and

Mikolajczyk , eds, BMVA Press, 2012, pp. 1–11. doi:10.5244/C.26.80.

116.

Zia ,

Akram ,

Nawaz ,

Shahzad ,

Abdullatif ,

Mustafa and

Lali , Identification of hatred speeches on Twitter, in: Proceedings of 52nd the IRES International Conference, 2016, pp. 27–32.

117.

P.G.

Zimbardo , The human choice: Individuation, reason, and order versus deindividuation, impulse, and chaos, in: Nebraska Symposium on Motivation 17, 1969, pp. 237–307.

118.

Zimmerman ,

Kruschwitz and

Fox , Improving hate speech detection with deep learning ensembles, in: Proc. LREC, European Language Resources Association (ELRA), Miyazaki, Japan, 2018, https://www.aclweb.org/anthology/L18-1404.

Leveraging external resources for offensive content detection in social media

Abstract

Keywords

1. Introduction

1.1. Related literature

1.2. Scarcity of reliably and uniformly labeled data

1 https://ec.europa.eu/commission/presscorner/detail/en/MEMO_19_806

2. External data sources

2.1. Datasets

2.1.2. Hasoc 2020

Table 2 Hasoc 2020 dataset samples of tweets with their ground truth labels

2 https://scholar.harvard.edu/malmasi/olid or https://sites.google.com/site/offensevalsharedtask/olid.

3.1. Text pre-processing

3 https://paperswithcode.com/task/text-classification

3.3. Deep learning methods

5 https://github.com/facebookresearch/fastText

3.4. Classifier ensemble

4. Results and discussion

4.3. RoBERTa

4.4. Classifier ensemble

5. Analysis

5.1. Utility of pre-trained transformers

5.2. Utility of pre-trained word representations

7 http://commoncrawl.org/

5.3.1. FastText

References

¹
https://ec.europa.eu/commission/presscorner/detail/en/MEMO_19_806

Table 2
Hasoc 2020 dataset samples of tweets with their ground truth labels

²
https://scholar.harvard.edu/malmasi/olid or https://sites.google.com/site/offensevalsharedtask/olid.

³
https://paperswithcode.com/task/text-classification

⁵
https://github.com/facebookresearch/fastText

⁷
http://commoncrawl.org/