Deep spatio-temporal emotion analysis of geo-tagged tweets for predicting location based communal emotion during COVID-19 Lock-down

2.1 User’s emotion analysis via Twitter

Recently, internet user’s emotions have been considered either to realize the user’s physiologic emotional state or to represent the user’s actions and their custom patterns in online social media [20]. The rate of recurrence at which people’s usage relationship with twitter and the period of usage per session have been calculated. Research studies have investigated how to enhance the act of content sharing schemes by using user activity information. In [4], user activities’ data is used to determine abnormal actions and justify the consistency of the user profiles. Further analysis in [7, 13] has helped forecast the user activities for learning online social media. In twitter, the user actions can also be categorized relative to the succeeding actions: tweeting, re-tweeting, and commenting. It is significant to notice that these reports do not discover the modifications of topics sent by users. Furthermore, some actions can capture more people’s interest, and these are either determined by the total number of messages or message interactions within people attracted to some topics [9]. Therefore, among the total number of messages in online social media, it is feasible to calculate the total number of people correlated to possible actions, and regarding particular areas within a certain interval of time and also respective to a geographical site.

2.2 Multi-class emotion classification

An event detection scheme [25] is presented at the time of early phases of an incident, depending upon alterations in the internet user’s activities in an Online Social Network (OSN). This method can identify an incident of several topics, which could be useful for various purposes. The suggested event detection scheme is comprised of the succeeding key sections: (1) estimation of the internet user’s site, (2) message abstraction via OSN, (3) topic recognition by using natural language processing (NLP) depends upon the Deep Belief Network (DBN), (4) the internet user’s variation of action examiner in OSN, and (5) sentimental examination for emotion classification depends upon a tree convolutional neural network (tree-CNN). In the instance of public healthiness, early event detection is very applicable for the populace and the experts can proceed with remedial actions. Therefore, the new coronavirus disease (COVID-19) is applied as an instance report in this work. For implementation validation, the sections related to the topic classification and emotional analysis were evaluated with various similar results or applied with various machine learning algorithms. In performance calculation, the suggested event detection system attained accuracy higher than 0:90, whereas various similar approaches attained accuracy rates less than 0:74. Moreover, the proposed technique discovers an incident nearly 3 days before the various other approaches.

According to Savigny et al. [18], experiments are conducted for emotion classification with Indonesian YouTube comments. The dataset consisted of 8000 comments, categorized by using 6 emotion categories. 43000 uncategorized comments have been applied as the corpus to train word embedding simulations. A relative study of various word embedding approaches called Average Word Vector, Aver-age Word Vector with TF-IDF, Paragraph Vector, and Convolutional Neural Network (CNN) was completed. Amongst numerous embedding approaches, the CNN technique outperformed the rest by far, attaining an accuracy of 76.2%, which is an enhancement from the model. Nguyen et al. [9] categorize the collections of geo-tagged documentations correlated to a specific point of interest on public modes, perceiving single input info that gets an incomplete image of clusters because of noisy regions. To overcome this, the enhanced version of the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm was introduced to the density-based clustering algorithm using spatial-textual information (DBSTexC). The reason for avoiding the geographic areas is due to a large amount of point of interest-irrelevant geo-tagged tweets in the corresponding clusters by using the DBSTexC. To increase the quality of clustering, the fuzzy DBSTexC (F-DBSTexC) was also introduced which allows an increase in performance gain up to 27.33% higher than DBSTexC.

A deep-learning technique [1] is employed to categorize the internet user’s opinion stated in reviews (named as RNSA). The RNSA utilizes the RNN along with LSTM to yield the benefit of successive processing and overcome numerous faults in existing techniques. The RNSA consists of 2 key sections such as sentiment analysis and pre-processing. They conduct sentence-level emotion classification by large-scale movie review datasets. The outcomes show that the proposed method (RNSA) yields better performance in which the precision rate is 0.8675 when compared to the existing techniques. Li et al. [20] characterized Coronavirus-oriented online social media posts into 7 situational data classes. They also recognized the specific important features in forecasting the reposted amount in every data class. The accuracy rate of classification of various supervised learning approaches was altered. The accuracy rate of the Support Vector Machine (SVM) is 54%. The accuracy rate of Naive Bayes and Random Forest is 45% and 65%, correspondingly.

A technique of topic recognition based on NLP has been used for Coronavirus detection using a hybrid artificial intelligence pattern [21]. The alteration in the infective capability of the disease was scrutinized in a few days next to the infection and also improved susceptible-infected (ISI) model was recommended. The NLP segment and Long Short Term Memory (LSTM) network were inserted within the ISI model to construct a hybrid AI model for coronavirus detection. NLP + LSTM constructed into the hybrid AI model, the mean absolute percentage (%) mistakes of the detection outcomes, considering the succeeding 6 days, were 0.52%, 0.38%, 0.05%, 0.86% in Wuhan, Beijing, Shanghai, and nationwide respectively.

2.3 Topic modeling using LDA

Vaduva et al. [5] represented a model to predict latent semantic classes consisting of pairs of objects characterized by a specific spatial positioning. A hierarchical structure is suggested for the image content. In which invariant spatial signatures are calculated for pairs of objects and several statistics are prepared to succeed in a good understanding of the spatial distributions inside the predicted semantic classes. A hybrid Latent Dirichlet Allocation (LDA) model [6] is projected to use the Genetic Algorithm (GA) in discovering optimal weights for LDA topics to upgrade the topic classification performance. This study uses LDA to scrutinize the topics of online comments of game applications in a supervised manner. Meanwhile, Chinese contents are generally less structured and too complicated to evaluate. The outcome demonstrates that a hybrid LDA classification mechanism with GA improves the accuracy of LDA in highlighting the meaning of topics and the words representing a topic.

A Variational Auto-Encoder Topic Model (VAETM) [30] is proposed by fusing word vector demonstration & entity vector demonstration to extracts the high-quality topics using short texts. Furthermore, an enhanced supervised VAETM (SVAETM) is suggested to improve the VAETM via label data in the training phase to organize the implication of latent illustration of topic distributions & topic generation. Finally, KL-divergence-based inference algorithms are presented to gather approximate posterior distribution. Experiments are conducted over three public short text datasets such as 20 Newsgroups, IMDB, and Standard literature that show better performance by the proposed method (VAETM and SVAETM) in terms of perplexity, Normalized Point-wise Mutual Information (NPMI), and accuracy. The proposed method achieves accuracy, Precision, and Recall rate are 0.84, 0.84, and 0.83 for topic modeling.

A novel probabilistic model called tpLDA [22] presents prior knowledge like topic & its inner popularity of cluster to build the LDA hyper-parameters which are constantly problematic to describe. The constructed distribution directs the innovative reproductive procedure learning that redirects the dynamic variations in the information on continuous-time slices. The proposed method (tpLDA) attains a high Fowlkes-Mallows index (FMI) at every single time step associated with 15% higher FMI compared to the existing better one & 4% greater FMI at least. Moreover, the proposed technique yields 28.06 % of the Jaccard Index (JC) at the 6th time slice and is less than 1.87% related to the existing optimal models. A unique Fuzzy Topic Modeling (FTM) method [16] is represented to deal with the sparsity problem in short content via fuzzy perception. The global and local period frequencies are calculated by using the Bag-Of-Words model (BOW). The Principal Component Analysis (PCA) is used to reduce the high dimension on global period weighting and also fuzzy c-means algorithm is utilized to extract the related area from the files. The experimental analysis is performed on the 3 short text datasets namely, snippets (small datasets), Twitter, and questions (bigger datasets). Fuzzy Topic Modeling classification accuracy rate for snippets and question datasets are 95%, 94%, 91%, 89%, 87%,73%, 74%, 70%, 68%, 78% is associated with 50, 75, 100, 125 and 200 number of topics (CSTM, GLTM, Mix-gram, SATM, BTM, LTM, DREx + LDA) with SVM classifier.

The proposed work differs from the discussed literature in the following ways: Many types of research have been carried out to analyse the tweet’s opinion and emotion. Still, these researches have restrictions concerning spatial-temporal evaluation of emotion analysis. To overcome these issues, this paper introduces a deep spatio-temporal analysis framework of geo-tagged tweets to predict the communal emotion of people from different locations during a lockdown. This obtained framework presents an optimized version of Latent Dirichlet Allocation (LDA) for finding the optimal hyper-parameters using grid search. The classification model is built using Recurrent Neural Network (RNN) to predict emotions for each topic based on locations. An experimental result provides advancement in topic modeling as well as comparison to the classical LDA model.

3.1 Datasets

This work is experimented with two twitterdatasets that are collected from 27th January 2020 to 15th March 2020, which are in english language and gathered across the globe. The first dataset is the primary COVID-19-related dataset, which is downloaded from twitter streaming API that supplies real-time tweets based on specific filters [11]. The second dataset is an emotion dataset consisting of 40000 tweets classified with 11 emotion labels. The extracted dataset that was considered for experimentation contains numerous set of keywords that satisfies the training threshold for Latent Dirichlet Allocation (LDA) based Grid search and multi-class recurrent neural network-long short term memory (RNN-LSTM) classification model. Significant keyword lists are used to gather tweets about COVID-19 as shown in Table 1. From these two datasets, arbitrarily ten percentage is selected as test data and ninety percentage is selected as training data. Significant keyword lists are used to gather tweets about COVID-19 as shown in Table 1. The twitter streaming API contains the Covid-19 related tweets along with the valuable attributes, which are described in Table 2.

The overall system architecture for the proposed work is shown in Fig. 1. The system architecture of the proposed work begins with the collection of COVID-19-related raw data from twitter streaming API, towards performance and result analysis. The proposed framework comprises two significant models namely, the optimal LDA model and multi-class emotion classification model. LDA is a generative modeling process that arises to identify topics in each document [6]. In this work, an optimized LDA model using grid search is used to perform topic modeling. This paper adopts general notations to refer the few basic terminologies like word, document and corpus [8]. A document contains the sequence of T words represented by R ={ r₁, r₂, … . r _T  }, where r _T is the T^th word in the sequence. A corpus is a set of N documents represented by C ={ R₁, R₂, …… . , R_N }. Let ‘Ķ’ represents the total count of topics, ‘T’ signifies the total count of words in documents and ‘N’ refers total count of documents. In multinomial distribution, ′ R nt ′ is the observed word for t^th word in the n^th document and ′ Z nt ′ denotes the word topic assignment for R _nt . Let ′ R uv ′ represents the specific number of words given in the document. This is considered as observed variable. ^′α′ and ^′β′ are the dirichlet parameter that represent the prior on per document topic distribution and prior on per topic word distribution. Let ′ θ n ′ denotes topic distribution of document n, ‘φ’ Ķ’ is the word distribution for topic ‘Ķ’ and ′ Z ´ un ′ is the topic for the ‘v’ word in document ‘u’. A corpus probabilistic model ^′p (C|α, β) ′ of a document (COVID-19-related tweets) is defined in Equation (1),

p ( C | α , β ) = ∏ n = 1 N ∫ p ( θ n | α ) ( ∏ t = 1 T n ∑ Z nt ´ p ( Z nt ′ | θ n ) p ( R nt | Z nt ′ , β ) ) d θ n(1)

OPEN IN VIEWER

Fig. 1

Architectural diagram of the overall system model.

The LDA parameters are described by differential bound maximization as specified in Equation (2), log p ( C ) ⩾ ∏ n = 1 N E q n - E q n [ log q n ( θ , Z ´ ) ](2) where ′ E qn ′ denotes the expectation in the multinomial update for LDA topic modeling. The optimal LDA component consists of the pre-processing methods which are initially applied to the COVID-19-related datasets. The raw text data will be pre-processed as follows: a) eliminate Unicode strings b) transform URL (www, Http...) into token ‘URL’ c) eliminate re-tweet and username d) eliminate punctuation, number and special characters e) remove a duplicated character that exceeds three f) eliminate extra spaces. These are accomplished by regular expressions. The conversion of words into their root word is completed by Lemmatization. This reduces the total count of unique words exist in the corpus to generate the better topics at the end. The document-word matrix is made to feed input to the LDA model. Grid search is employed as a hyper parameter optimization technique to find the optimal number of topics utilized by the model. By these processes, the optimal learning rate is fixed. Hence, the optimal LDA model is constructed to analyze the topic of the COVID-19-related twitter dataset. In emotion classification module, Feature engineering is accomplished initially. The feature vectors are converted from the raw text data. Then, text is converted into a sequence of tokens using the generation of Word embedding which produce the mapping with their respective embedding. Features based on NLP namely, word count, average word density, and speech tag distribution are also developed. Next, the multi-class emotion classification module along with the softmax activation function which is employed for the output layer. It assigns decimal probabilities to each class. Finally, the multi-class model is trained using RNN-LSTM units. Thus, the geo-tagged COVID-19-related twitter data obtained from the Twitter Streaming API. This data feed input to the LDA topic model, which aggregates the tweets within a fixed number of topics. Now, for each topic, the overall dominant emotion is computed by feeding each tweet to the classification model.

The objective of utilizing a grid search is to acquire the optimal LDA model for topic modeling. The algorithm 1 is designed in the expectation-maximization (EM) strategy, which is a cyclic method to calculate the maximum log-likelihood of a specific model. EM process increases a lower bound based on the factors α and β. The derivation generates the succeeding iterative algorithm:

E-step: For every document, discover the optimum values of the dirichlet factors.

The M-step upgrade for the conditional multinomial dirichlet factor α and Dirichlet distribution p (θ|α)is written out methodically in Equation (3) [8], p ( θ | α ) = Γ ( ∑ l = 1 Ķ ; α l ) ∏ l = 1 Ķ ; Γ ( α l ) ∏ l = 1 Ķ ; θ l α l - 1(3)

The parameter ^′α′ is a Ķ-vector with components α _l  > 0; 1 ={ 1, 2, … . ,  Ķ ; } where Ķ denotes total count of topics, ′ θ l ′ denotes topic distribution and Γ (y) is the gamma function. The EM iteration varies back and forth within presenting an expectation (E) step, which generates a function for determining the log-likelihood by recent search parameters. Then, maximization (M) step alters the parameters to determine the maximum log-likelihood for a specific model. The likelihood presents a better chance of a text to apply to a specific topic from the topic’s list which is generated by the model. Hence, a higher likelihood denotes an enhanced topic model.

In the E-step, the θ is fixed which presents the topic distribution of document likewise φ which correlates to the word distribution under a topic. The topic assignment of every word is calculated with the help of these factors. In the M-Step, θ and φ are updated to boost the expected log-likelihood. In specific, calculated the perplexity of a held-out test set to evaluate the models. The perplexity, utilized by tradition in language modeling, is monotonically diminishing in the likelihood of the test data & algebraically equal to the transposed of the geometric mean per-word likelihood. A lesser perplexity score specifies improved generalization performance. Perplexity measures the performance of the LDA topic model [8]. It is explained by Equation (4) using Equation (5), perplexity ( C test ) = exp - ∑ n = 1 N log p ( R n ) ∑ n = 1 N T n(4) p ( R n ) = ∑ Z ´ n p ( Z ´ n ) ∏ t = 1 T n p ( R t | Z ´ )(5) where ′ p ( R t | Z ´ ) ′ denotes conditional multinomial, ′ Z ´ n ′ represents word topic assignment ‘N’ is the number of documents (in the test sample), ′ R n ′ represents the words in document n, ‘T_n’ denotes the number of words in document n.

Hence, by presenting a grid search in this manner on all the feasible models, the model along with the highest log-likelihood / lowest perplexity is preferred as the better model for topic modeling.

4.1 LDA with grid search

The objective of this module is to construct an optimal LDA model for topic modeling using a grid search. The COVID-19-related tweets are initially preprocessed to eliminate extra spaces, remove unicode strings, remove a duplicated character that exceeds three, eliminate retweet & username, and other special characters. Tokenization is completed by converting text to a word’s list. Lemmatization is carried out to reduce the total number of unique words in the dictionary by converting each word to its root word. The document word matrix is then generated using a count-vectorizer which is an important input to the LDA topic model algorithm. The optimal LDA model is constructed using two search parameters namely the number of topics and learning decay. Table 3 shows the parameter settings used in optimizing the LDA model. The optimal LDA model using grid search is designed based on an expectation-maximization strategy. Figure 2 visualizes the optimal values of the topic model, namely learning decay and the number of topics. An optimal combination of learning decay and a number of topics are thus obtained for the proposed optimized LDA model.

OPEN IN VIEWER

Fig. 2

Optimal LDA Model.

The higher log-likelihood, the more accurate model is as a high likelihood shows that the most unseen portions of COVID-19-related text data. These are categorized within one of the topics which are generated by utilizing the model, and only very few are turned out to not be involved in the topics. The graph depicts the changes in the log-likelihood of the LDA model as a function of the total number of topics and learning decay. From the graph, the optimal number of topics is 10 and the optimal learning decay is 0.5 as the log-likelihood value is maximum at this point is gathered. The most frequently arising keywords are depicted in Fig. 3 for one of the 10 topics. These words had the most influence on the creation of the topics. A topic distance map is represented along with the most important words which were part of each topic is shown in Fig. 3. The circular blobs depict one of the 10 topics which are sufficiently spaced from everyone with a lack of overlapping which represents a good topic model. Thus, the topic distance [8] is computed and the Intertopic distance map [15] is shown in Fig. 3. The succeeding list shows the topmost 10 topics discovered in the Covid-19-related dataset (example., “topic 1”) and the 10 most typical words related to every identified topic (example., “public, cases, fresh, death, period, china, fight, lockdown” for “topic 1”), correspondingly.

OPEN IN VIEWER

Fig. 3

LDA model visualization.

topic 1: public, cases, fresh, death, period, china, fight, lockdown

Several common themes is identified about each topic by observing the topic-wise common keywords. To compute the difference between the default LDA model and the optimized LDA model using a grid search, the likelihood, and perplexity of both the models are calculated and compared. The outcomes are shown in Table 3, which shows the log-likelihood and perplexity of both models that are computed and illustrated. A higher likelihood and lower perplexity denote a good topic model. The outcomes illustrate that the LDA model optimized using grid search is an enhancement over the standard LDA model.

A multi-class model is constructed for emotion classification in this module. The labeled Covid-19-related twitter dataset consisting of 40000 tweets and 11 emotion classes is the base dataset. All the emotion labels are transformed into integers with the help of a label encoder. Then, data binning is completed to provide the word count distribution of the Covid-19-related tweets in the dataset. The network trains much more effectively if an arbitrary sequence length was fixed for the model. The layers of the model are designed with an embedding layer, an LSTM layer comprising of 64 LSTM units, two dense layers utilizing the relu activation function, and a softmax output layer for emotion classification which are shown in Table 4.

An embedding layer is created next to the input layer by calculating the vocabulary size for the dataset, the inputs, and the sequence length of each input. This is ensued with the help of the LSTM layer consisting of 64 units. Since the outcome of LSTM is not a softmax, 2 dense layers are organized after the LSTM layer to match the dimensionality of the expected target. Finally, a softmax output layer is used for multi-class emotion classification. The multi-class model is built to run for 100 epochs that include 27000 batch iterations and the model training loss curve of the model is shown in Fig. 4. As seen, the loss value declines rapidly around the 70 epoch after which it stabilizes to a value of approximately 0.2. The accuracy and additional evaluation metrics are calculated based on the trained model for emotion classification. The trained multi-class models are also used in prediction with Covid-19-related raw text data presented as an input for classification. The model classifies the text to one of 11 emotion classes depending on the probability of the text being appropriate to every class. The usage of the softmax layer restricts the generation of multiple emotions as output from a single piece of text. This is encountered because the dependent probability is used by the softmax for classifying each text into a single dominant emotion. Emotion prediction for Covid-19-related raw text data and different pieces of text data are presented as input, which is depicted in Fig. 5. To relate each topic with emotion, the model predicts an emotion for each of the tweets belonging to each topic and the dominant emotion for each topic is found by selecting the emotion with the highest frequency belonging to each topic. The twitter streaming API is used to obtain a sample of geo-tagged tweets from various locations to classify the emotions of tweets based on location. The different locations are clustered followed by plotting the frequency of occurrence of each emotion for each location.

OPEN IN VIEWER

Fig. 4

Model Training Loss.

OPEN IN VIEWER

Fig. 5

Emotion Prediction for (a) Raw Text Data, (b) each Topic.

Tweets from different areas are grouped together country-wise. This will help in giving an overall view of the popular emotions in different countries. Prediction outcomes classifying each text into one of the 11 emotion classes (positive, worry, negative, fear, happiness, anger, anticipation, sadness, love, neutral, boredom) is shown in Fig. 6. The topmost positive and negative hashtag during the lockdown in India is shown in Figs. 7, 8.

OPEN IN VIEWER

Fig. 6

Emotional analysis of twitter users during lockdown in (a) United Kingdom, (b) United States of America, and (c) India.

OPEN IN VIEWER

Fig. 7

Topmost positive hashtag during lockdown in India.

OPEN IN VIEWER

Fig. 8

Topmost negative hashtag during lockdown in India.

As seen from Table 5, the evaluation metrics, namely the Precision, Recall, F1-score, and accuracy of the model are represented. The equations for these four evaluations metrics are: Accuracy ( % ) = TN + TP TP + TN + FP + FN * 100(6) Precision ( % ) = TP TP + FP * 100(7) Recall ( % ) = TP TP + FN * 100(8) F 1 Score ( % ) = 2 * Precision * Recall Precision + Recall * 100(9) where TN is a true negative, and TP is a true positive. Recall computes the percentage of actual positives, which are properly recognized. FP and FN are denoting the false positive and false negative. In this work, comparative analysis of performance metrics with different LDA techniques namely, Standard LDA [23], LDA with word2vector [31], LDA + Maximum Entropy [29], Hybrid HDP + LDA [27], LDA+ Synonym Lexicon [3], LDA + SVM [26], LDA with feature ontology tree [10], and LDA with grid search (proposed method) are accomplished. The LDA alone cannot give better accuracy and the performance of hybrid LDA is quite better, which is clear from Table 5. Thus, LDA with grid search has presented the highest accuracy rate among all techniques.