Abstract
The rise of digital communication systems provides an almost infinite source of information that can be useful to feed classification algorithms, so it makes use of an already categorized collection of opinions of the social network Twitter for the formation and generation of a model of classification of short texts; which aims to categorize the emotional tone found in an author’s Spanish-language digital text. In addition, linguistic, lexicographic and opinion mining computational tools are used to implement a series of methods that allow to automatically finding coincidences or orientations that allow determining the polarity of sentences and categorize them as positive, negative or neutral considering their lemmas. The results obtained from the analysis of emotions and polarity of this project, on the test phrases allow to observe a direct relationship between the categorized emotional tone and it is positive, negative or neutral classification, which allows to provide additional information to know the intention that the author had when he created the sentence. Determining these characteristics can be useful as a consistent information objective that can be leveraged by sectors where the prevalence of a product or service depends on user opinion, product rating or turns with satisfaction metrics.
Introduction
The rise of digital communication systems in recent years has allowed millions of users to share their opinions on various topics, people, situations, products or services with other users. The number and diversity of these opinions are very useful for companies, universities, entrepreneurs, among other sectors; as they provide the user’s perception as a source. Currently, social media platforms have become digital spaces where users publish and disseminate information spontaneously and voluntarily, which also represents and emerging challenging sector in the context of big data. In this sense, we find one of the social networks with the greatest boom today called Twitter, where opinions are published and that due to this are characteristics of versatility, speed and message dissemination system is being widely used by governmental and non-governmental institutions.
The computational treatment of these opinions is primarily the responsibility of Natural Language Processing through the areas of Emotion Analysis and Classification of Polarity in Texts [25] and has been among the most active processing areas since the early 2000’s [3]. The fundamental objective of sentiment analysis is to define automatic tools capable of extracting subjective information from natural language texts, such as opinions or feelings, in order to create structured and actionable knowledge to be used by a decision-making system. In this context, in this work, it is considered that an opinion or text with authorship can provide positive, negative or neutral evaluations influenced by the mood of the author.
This paper plans to have more detail about these opinions or texts with authorship in Spanish. The system preprocesses the text that divides it into words by eliminating those that do not contain or give value to the result. Through the lemmatization of the text, which is a linguistic process that, given a flexed word 1 finds it is motto. So for example, he wrote or I will write is reduced to his motto: to write. Using dictionaries of terms or lexicons in which each word is marked with a semantic orientation of polarity or opinion, the emotion of the text (disgust, anger, fear, joy, sadness, surprise) is obtained to finally perform the classification of polarity (positive, negative or neutral).
Sentiment analysis
Sentimental Analysis (SA), a task of the text classification within natural language processing, it is defined as the computational study of opinions, feelings and emotions expressed in texts [14]. The main objective is to determinate the attitude of a writer towards certain products, situations, people or organizations (target); identify the aspects that generate an opinion (features); who owns them (holder); and what is the type of emotion (I like it, I love it, I value it, I hate it) or it is semantic orientation (positive, negative, neutral) [13]. There are different processing tasks that can be performed in the SA, a basic one is to classify the polarity of an opinion expressed in a text (document, phrase, trait or characteristic) into a binary classification, positive or negative. In addition, a more advanced classification of three classes (positive, negative or neutral). A slightly more complex task is the multi-classification of a text according to the degree of polarity of the attitude within a scale; and the most advanced task is the identification of the aspects mentioned in a text and their associated feelings [24].
State of the art
This research is framed in the world of opinion mining, so below is a state of the art related to work. Among the studies focused on the classification of opinions, positive or negative, is the one presented in [20], one of the first investigations on sentiment analysis in which criticism data from films found on the Web are used; they are used in three classification algorithms, surpassing the baselines produced manually by a human. One of the pioneering works in introducing the term of Sentiment Analysis is that presented by [18]. In this publication, this task will be determined as finding expressions of feelings for a given subject and determining their polarity. In [10], a proposal to undermine and summarize consumer reviews was presented; for which it was proposed the creation of a small list of adjectives “seed” labeled manually, if a positive or negative feeling is expressed. Subsequently this list is augmented using WordNet [16].
Until 2014, modified systems in the constant monitoring of messages produced on social networks were evaluated within the framework of the RepLab competition [2] for tweets in English and Spanish. Despite including the Spanish language within the competition, most of the works reported in the literature focus on the English language. Therefore, the different methods applied for the classification of student opinions in English have been applied to the Spanish language [8]. These methods have considered since the use of n-grams of words, the reduction of words to their root (stemming) and even their substitution. The SemEval forum (International Workshop on Semantic Evaluation) is undoubtedly a space that has spread the study of polarity in Tweets and where you can find different corpora associated with the sentiment analysis. Since 2013, it has had support for polarity assessment in Tweets. In particular, in 2017, 58 teams were presented with different proposals, which are summarized in [23].
In [12], the authors carry out research in the area of marketing in which they analyze data collected from Twitter using specific hashtags to search for some objects of interest items such as computers, cars, televisions, among others in order to perform an analysis to effectively discover market intelligence (IM) to help support decision-making.
On the other hand the authors [19] present us another area of research in which the analysis of feelings is implemented, the authors show us the classification of opinions that Twitter users have about Syrian refugees, as is well known Syria is a war camp that prevents people from living peacefully, and that is why Syrians migrate from their place to another place in order to save their lives, for the analysis of opinions the authors searched for the opinions on Twitter with unique hashtags in two languages in order to have a considerable amount of comments, normalized the data for their pre-processing and developed a lexicon that punctuated the comments according to the type of feeling to which they belonged every comment.
Within the area of education we can find [1] the authors use sentiment analysis to classify the polarity of student comments created in order to give feedback to teachers as pre-processing first perform an elimination of emoticons and numbers so that the data have the same structure, once the process is finished, they perform the processing stage where they use classifiers such as SVM and Naïve Bayes. In this way the comments are classified with their respective labels according to the sentiment expressed by them, once labeled and classified these comments are delivered to the teachers so that they can observe more closely the comments classified as negative, since the comments contained with this label are those that could grant the areas of improvement in teaching to students, such as topics that need to be repeated due to lack of understanding.
The SemEval 2018 edition has stood out for having a section (with three tasks) dedicated exclusively to the analysis of affective characteristics in Tweets. A full description of the 75 teams that participated in SemEval task 1 called “Affect in Tweets” can be seen at [17]. They describe the way in which the corpora were constructed and labeled manually for the English, Arabic and Spanish languages, as well as the techniques used in the 319 executions submitted by the teams. In the edition of SemEval 2019, different tasks have been presented that cover the sentiment analysis, having a section called “Opinion, emotion and abusive language detection” and that has had four different tasks that cover the detection of contextual emotions in text [6], the multilingual detection of hate speech and women on Twitter [4] and the identification and categorization of offensive language in social networks [30].
In the last edition of SemEval 2020 several tasks related to the SA were presented, section specifically called “Humor, Emphasis, and Sentiment” in which the tasks of evaluating humor in edited news headlines [9], the analysis of memes [27], Sentiment Analysis for Code-Mixed social media Text [21] were presented and emphasis selection for written text in visual media [28].
In Task 9: Sentiment Analysis for Code-Mixed social media Text, the task is to predict the sentiment of a given code-mixed tweet. The sentiment labels are positive, negative, or neutral, and the code-mixed languages will be English-Hindi and English-Spanish. Besides the sentiment labels, we will also provide the language labels at the word level. The word-level language tags are in en (English), spa (Spanish), hi (Hindi), mixed, and univ (e.g., symbols, @ mentions, hashtags). Efficiency will be measured in terms of Precision, Recall, and F-measure [22].
The participation of the different teams in the tasks is reported individually and that is why we refer the reader to the reports that summarize the participation of all the teams.
Methodology proposed
Sentiment analysis and politics classification provide useful information for areas where out comes enables strategies or decision-making. For example, knowing the opinions of a certain product or service, the popularity of a celebrity or politician, the effectiveness of social, political, scientific, educational or cultural programs through the individual opinions of a community; independent of the socio-cultural context and whose outcome will only depend on who issues the opinion, as well as knowing if it is influenced by emotion at the time of writing this work.
The proposed methodology for the identification of the polarity of a Spanish text given the emotion of its authors based on four stages; each module is explained in the following sections. The polarity that is identified consists of three labels: negative, positive and neutral; while the emotions that were considered, in the development of this work, are five: “anger, fear, joy, sadness and love”. In Fig. 1, you can see the sequential process that takes place for the task described above.
Sequence diagram for identifying the polarity given an emotion.
In the first stay, a series of processes and methods were developed that, given the entry of text into a system, allows the lemmatization of said text (processing supported by Natural Language Processing). Subsequently, the emotion is obtained by means of an algorithm of classification of texts and their polarity by using the lexicon ML-SentiCON [7] for the Spanish language. Finally, the results are presented through a graphical interface. The following sections explain the stages, which integrate the proposed methodology, and the functions in each of them.
The Model initially requires a corpus that provides sufficient information to train it and allows a consistent classification. It is important to mention that the more data the corpus contains, the greater the accuracy that the Classifier Model will provide.
For the present work we used the corpus of SemEval 2018 “Task 1: Affect in tweets” [17], which contains 3562 tweets for training and 680 tweets for tests. Specifically, we used the training file (2018-E-c-Es-train.txt) that contains a structure with the following data: ID: Id of the scanned tweet. Tweet: The text of the classified tweet, mostly includes the user who wrote the tweet. Emotion: Corresponds to the column number where the value equal to 1 is located.
In the training corpus, it was observed that it contained tweets and that they were not associated with a feeling, therefore, they were not considered in this work.
Preprocess
The objective of this stage is to preprocess the text entered into the system; that is, it cleans up characters that do not add value to the analysis of the emotion or polarity. Some of the signs that are removed are punctuation marks, special signs, double spaces, emoticons, among others. Subsequently, a conversion to lowercase of the entire sequence of characters is made, in order to standardize the text and finally the tokens are obtained. For each token, POS tagging is done by obtaining its tag and motto. This data is grouped into a list of tokens that will be used in later stages. The three sub steps that are performed on the sequence of input characters, we call this sequence a phrase, are: Data preparation. In which a clean liness is re-reached again to the input phrase. Tokenization. In which the input phrase is used in smaller, more individual elements called tokens. Lemmatization. From the obtained token, it undergoes analysis for find its motto and its POS tag.
In the first sub-platform “data preparation”, a series of string handling operations are included in the input phrase obtained from the user interface. The result of this data preparation is a sub-string without: Double or triple blank spaces. Special signs. Removal of emojis. Extra spaces at the beginning and end of the chain.
In the second sub-set the “tokenization” refers to the division of a sequence of input characters into smaller units called tokens. This process is done by splitting the sentence using the blank as a parameter.
In the third sub-stage the “ lemmatization”, consists of taking the result of the sub-stage previous and using the TreeTagger library [26], it will be searched, considering the value of each token, its POS tag and the respective lemma. With this data, an object of type token is created and added to list of arrays of objects of type token for later use.
Emotion identification
This stage is responsible for identifying the predominant emotion in the preprocessed text and that has been obtained in the previous stage. The identification is made bye a text classification algorithm based on a decision tree, which is created from a collection of previously classified training data, this is the corpus of SemEval 2018 “Task 1: Affect in tweets” [17]. The module creates a Text Classification Model that receives a phrase (instance) and categorizes it as: anger, fear, joy, sadness or love, corresponding to the emotion identified for that instance.
It is possible to identify the emotion because the training corpus is labeled with the emotion, so from that labeling the same data can be used for training and when new instances arrive, in these identify your emotion.
The process consists of receiving as an input parameter, a list of Token objects and consists of severla stages and relationships between their methods. Getting the result of the classification is one more step, and integrating it into the context, so that it can be used in a subsequent application. The recovery of the emotion serves to be able to show the results in the visualization stage and project it graphically.
Polarity identification
The objective of this stage is to obtain a classification: positive, negative or neutral of the text already classified in the stage of emotion identification. The identification is made from the extraction of the POS tags from the previously generated tokens; subsequently, the lemmas of the POS tags that are declared in the lexicon for the Spanish language are analyzed. In other words, not all lemmas are analyzed, only those found in the ML-Senticon lexicon [29]. At this stage a search for the lemma within the lexicon is carried out, in order to obtain it is positive or negative classification. Frequency control is taken of the positive or negative lemmas found. In the end, the polarity will be the classification with the predominant frequency (positive or negative). If both frequencies are equal, it is classified as a text with neutral polarity. At this stage a search for the lemma within the lexicon is carried out, in order to obtain it is positive or negative classification. Frequency control is taken of the positive or negative lemmas found. At the end, the polarity will be the classification with the predominant frequency (positive or negative). If both frequencies are equal, it is classified as a text with neutral polarity.
At this stage the total polarity of the text is decided, which depends on the positive or negative score of the text, to calculate this score equations2 are used.
The result of polpos and polneg considering only those words in the message (x) that have a reference in the word list L of the ML-Senticon lexicon. Once all the words of the full text have been reviewed, in this case of the tweet, it is necessary to obtain their polarity, so a comparison is made. To do this, it uses the decision conditions presented in equation 3.
This polarity will indicate the positive or negative charge of each message belonging to the text in question. The identification of polarity of the text is made with the support of the lexicon ML-Senticon for the Spanish idiom, which contains a collection of lemmatized words together with it is POS labeling and a positive or negative classification. Each token in the token list generated in the preprocessing stage is sent for analysis and comparison considering it is lemma attribute, matching the lemmas declared and classified in the lexicon. The return value of this search is the category, level, or parent node of the mapped lemma and provides identification as a positive or negative lemma.
Each of the experiments that were performed were supported by the Naïve Bayes classification algorithm [15].
The first experimenter consisted of relying on the AnalysisOfMotion.java and it’s generateArffcorpusEnt and classifierTweet methods, create a test class to extract the phrases from the corpus SemEval 2018 Task 1 Affect in tweets [17]. The resulting phrasescorpusTest.txt file contains a total of 755 tweets with a length of between 100 and 240 characters. This range was taken into account because tweets with lengths of between 140 and 280 characters did not exceed the 133 records out of a total of 2854 contained in the file for testing. Only the 755 records from the phrase file were entered into the text put field of the user interface, waiting for the results.
The second experiment used Weka [5] and it is user interface, implemented a classifier based on Naïve Bayes algorithm [11] and the same corpus of Training mentioned in Section 3.1. The Percentage Split criterion [29] to use the same corpus for training and testing with different percentages of training and testing, in the first instance with 66% of training and 33% for tests, in the second instance in 80% of training and 20% for tests and in the third instance with 90% of training and 10% for tests.
The objective of the various experiments is to obtain a classification the emotions of the tweets with an algorithm different from Decision Trees, which was implemented in a first version. The emotions correctly classified by this model will be weighted and compared with the classification made by the system developed with the same source of training data and number of instances (phrases).
Finally, the third experiment consists of classifying phrases form the training corpus. In order to make a comparison between the classification results of the developed system and the classification model of experiment 2, the training corpus of 2956 records was divided to provide instances for training and testing. Since the developed system uses a phrase or set of phrases for be analyzed and match with the test conditions, 3 experimental evaluations were carried out with the following parameters: Classification model generated with 66% of instances of training corpus equivalent to 1951 records and 33% of test sentences extracted from training corpus equivalent to 1005 sentences. Classification model generated with 80% of instances of training corpus equivalent to 2365 records and 20% of test phrases extracted from training corpus equivalent to 591 sentences. Classification model generated with 90% of instances of training corpus equivalent to 2660 records and 10% of test phrases extracted from training corpus equivalent to 296 sentences.
Experimental results
Test 1. Classification of phrases in evidence corpus
The first experiment consisted of using the corpus of evidence provided by the corpus used (corpus SemEval2018 [17]). The results obtained from the classification of the corpus instances for testing, with 755 instances are shown in Table 1, in which the default parameters of the Naïve Bayes algorithm are used.
Emotion and polarity identification results of 755 sentences, test 1
Emotion and polarity identification results of 755 sentences, test 1
Table 2 serves as a reference to obtain the classification results obtained in the following experiments.
Emotion charge in teh input file
Emotion charge in teh input file
In the first run made in this section, of the total of 2956 instances, the option of Percentage split of 66% (1951 phrases) was made to use the corpus of training also for tests. This implies that 33% (1005 sentences) are the classification test and which 42% was classified correctly. The results of the classification are shown in Table 3.
Ranking results Percentage split = 66%
Later, in the next run, the Percentage split option of 80% was selected to use the Training corpus also for testing. This implies that 20% (591 phrases) are for classification tests and of which 45% were classified correctly. The ranking results are shown in Table 4.
Ranking results Percentage split = 80%
Finally, in the third run, the option of Percentage split of 90% was selected to use the corpus of training also for tests. This implies that 10% (296 phrases) are for classification tests and of which 37.5% were classified correctly. The ranking results are shown in Table 5.
Ranking results Percentage split = 90%
As noted in Section 5.2, in any run, the training corpus was segmented for training and testing. For the case of these tests, it was subjected to the same conditions and the results were as follows.
The results for the first run, with respect to the classification model of 66%, the training were made on 1055 input phrases in the system and the model was generated with 66% of instances. The ranking results are shown in Table 6.
Ranking results Percentage split = 66%
Ranking results Percentage split = 66%
Later, in the next run, the Percentage split option of 80% for training and 20% for testing was selected. The ranking results are shown in Table 7.
Ranking results Percentage split = 80%
Finally, in the third run, the Percentage split option of 90% for training and 10% for testing was selected. The ranking results are shown in Table 8.
Ranking results Percentage split = 90%
Table 9 contains a relationship between the number of emotions and polarity is identified by the proposed methodology. These results correspond to Test 1. Classification of phrases from the corpus of Evidence.
Test Result 1
Test Result 1
From Table 9, it is observed that for each emotion there is a majority tendency to the positive polarity of the phrase; however, for the emotions of Joy and Love, this trend exceeds 50% of the total polarities detected. The emotions with the lowest percentage of positive polarity are Anger and Sadness.
For tests developed in Weka with Naïve Bayes algorithm, the overall results can be seen in Table 10. For the run one has 66% for training and 33% for tests; for run two you have 80% training and 20% for tests and in run three you have 90% training and 10% for tests. These results correspond to Test 2. Classification of phrases from the Training corpus.
From Table 10 it can be seen that regardless of the number of test phrases, the emotions mostly identified are Joy and Anger. In addition, depending on the volume of training data and phrases to be detected, the predominant emotion is disputed between these two categories. As the number of training data increases, the results between the identified emotions and their frequencies tend to resemble each other.
Test Result 2
Finally, Table 11 contains the results of Test 3. Classification of phrases from the Training corpus.
Test Result 2
From Table 11, the classification of the system developed using a classification model based on decision trees, shows that the greatest trend of the same set of phrases as the tests include Joy and Anger as predominant emotions of the classified phrases.
The system developed in this project orients the decision of emotion to only 5 categories, so it provides a closed system that differs with Naïve Bayes algorithm; however, for overall results, these are similar in terms of the number of sentences classified and their emotions for each run.
In this article, he managed to design and implement a system that provides a category or emotional tone and polarity of a phrase or text with consistent authorship between the two characteristics. That is, most tests report that there is a direct relationship between phrases, for example, with joy emotion and positive polarity; as well as a lower relationship of positive polarity with phrases of negative emotion.
Although these results do not provide an accuracy since it depends entirely on the intention of the author, the correct use of the words in Spanish, the use of rudeness or sarcasm, abbreviations, idioms, the classification algorithm, the monitoring method, volume of training and testing data; the category made in this article, provides meta information based on real cases and trained with data developed by experts in the field. This categorization can be used by systems that have high volumes of information for feedback or self-learning systems that take into consideration many language variants for more accurate classification.
This article can be expanded to use some other classification algorithm and extend the categories of emotions, use some other lexicon or load other text classification models to obtain greater precision in categorization.
Footnotes
A word is flexed when it is plural (friends), feminine (friend), conjugated (eating), diminutive (little friend) or superlative (biggest friend).