Adaptive cognitive intelligence in analyzing employee feedback using LSTM

Abstract

Cognitive computing is the mirroring of human brain and this is made possible by using natural language processing, pattern recognition and data mining. By mirroring the human brain (Cognitive computing system), helps to solve some of the complicated problems without much of human supervision. In the fast-changing world, the major challenge every organization facing is difficulty in retaining its employees. Employees may leave an organization due to low salary, overwork, lack of opportunities and recognition, work culture, work-life imbalance etc. Better ways to retain employees is to understand their requirements and fulfill them. The proposed employee feedback sentiment analysis system collects the employee feedback reviews from open forums and perform sentiment analysis using Recurrent Neural Network – Long Short-term Memory (RNN-LSTM) algorithm. On performing Sentiment analysis, employee review comments are classified as Positive or Negative. A report is generated and sent to the HR of the organization as webapp or mobile app. The report has total number of positive and negative comments and positive and negative counts with respect to salary, work pressure etc. With the report, the organization can arrive at identifying social sentiments of their brand and may take corrective actions to retain employees which benefits both organization and employees. This paper also captures the performance of various models in training and predicting the employee feedback dataset and the models evaluated are Logistic Regression, Support Vector Machine, Random Forest Classifier, AdaBoost Classifier, Gradient Boosting Classifier, Decision Tree Classifier and Gaussian Naïve Bayes. The classification report and accuracy of each model is captured. The dataset size was gradually increased from 200 to 1000 and accuracy was predicted for each model. It was identified that the accuracy of machine learning algorithms was ranging between 66% to 85%. On training RNN-LSTM algorithm with dataset of size 30 k, the accuracy was 88%. It was identified that Deep learning algorithm RNN-LSTM performs better with huge dataset. Increasing dataset size still increase the performance of RNN-LSTM algorithm in training and prediction. Thus, the objective function of the proposed model to perform sentiment analysis on employee feedback review comments is achieved successfully.

Keywords

Cognitive deep learning RNN-LSTM LSTM recurrent neural network

1 Introduction

Today, social media is extensively used by people around the world to share their experiences and their learning. Sentiment Analysis is the common classification tool that analyses an incoming message and determines whether the piece of information is positive, negative or neutral. It also helps to asses opinion, attitudes and emotions. Sentiment analysis can be performed at three levels. They are document level analysis, sentence level analysis and aspect level sentiment analysis. Document level analysis identifies the overall opinion of the document. Sentence level analysis is performed on sentences. Sentence level analysis first determines whether the given sentence has opinion or not. Normally objective sentence has factual information while subjective sentence express opinion. If the sentence expresses any opinion, then sentiment of the sentence is identified. In the proposed method sentence level sentiment analysis is used. Aspect level analysis [8] is finer analysis and it is performed on each aspect of entities (Aspect/features represent components of entity - e.g. the screen of cell phone, service in hotel, picture quality of camera etc.) Aspect level analysis uses the Natural Language Processing (NLP) which is a subset of artificial intelligence (AI) that helps systems to understand the human language. Identification of aspects or features can be done through Topic Modelling or Deep Learning methods.

Sentiment analysis is being used in various field like business, politics, public actions, finance etc. It helps in identifying customer opinion towards products or services and helps improving business. Constant rise in employee attrition may result in reduced productivity and profit, difficulty in running the operations, customer attrition, low workplace morale and deteriorating product or service quality. One way to retain employees is through identifying their requirements and fulfil them. Though various engagement surveys are available within the organization, there is no true reflection of employee’s expectations. There are social mediums like Glassdoor.com, Indeed.com websites which would allow the employee to share freely their views or feedback about the organization. On performing Sentiment analysis on employee’s feedback, an automated monthly report is sent to the HR of the organization in web app or mobile app. Report contains classified positive and negative sentiments. With the report, the organization can arrive at identifying social sentiments of their brand and may take corrective actions to retain employees which benefits both organization and employees.

This paper’s contributions are as follows:

Comparative study of performance of various algorithms like Logistic Regression, SVM, Random Forest Classifier, Ada Boost Classifier, Gradient Boosting Classifier, Decision tree classifier and Naïve Bayes in training and predicting employee feedback dataset.

Investigating the performance of deep learning RNN-LSTM algorithm in training employee feedback dataset

Perform sentiment analysis on employee feedback for selected organization using deep learning RNN-LSTM algorithm.

This paper is organized as follows: Section 2 includes articles relevant to the implementation. Section 3 explains techniques used in this study. Section 4 details the experimental setup and proposed system. Section 5 provides results and discussions and conclusion is discussed in Section 6.

2 Articles related to our work

Tweets related to 2016 US election has been classified [3]. Dictionary is constructed with words of polarity depending on the hashtags for the selected topic. Steps involved in implementation are data gathering, data processing, classification and its accuracy evaluation. Two classifiers used are Google cloud prediction API, Naïve Bayes. Result is also compared with other sentiment analysis tools like Meaning cloud, RapidMiner, StreamCrab, IBM Watson text analytics. Opinion polarity is very effective in predicting human attitude when analysing big data. This helped to understand the opinion polarity that helps to identify deep emotions expressed in social media.

Sentimental analysis of Machine Learning techniques, like a. Probabilistic classifiers, b. Naïve Bayes, c. Bayesian Network, d. Maximum entropy classifier, Linear Classifiers like a. SVM, b. Decision Tree, Rule based classifier were discussed in detail [16]. It was identified that lexicon-based approach solves many sentiment analysis tasks despite its high complexity. Using domain specific corpus provides good results than using domain independent corpus. This helped to learn multiple recent sentiment analysis techniques.

The proposal [1] is based on classifying tweets as extremist and non-extremist classes. Deep learning techniques are used in the tweet’s classification. Methodologies involves Data collection, Pre-processing, Training, Validation and testing. The workflow of the network comprised of following steps: i) collection of tweets, (ii) pre-processing (iii) classification based on LSTM + CNN model and other Machine Learning and Deep Learning classifier. It was understood that proposed system performed better with LSTM + CNN.

Ajay Shrestha and Ausif Mahmood [2] explaineddeep neural networks and their optimizations to improve accuracy. Also, the mathematical formulas behind deep neural networks has been briefed [2, 6]. Architectures of various deep learning algorithms like Deep Convolution Network, Deep Residual network, RNN, Reinforcement learning, Variational auto encoders were explained in detail.

Improved word representation method is discussed in detail [17]. Weighted word vectors are generated using traditional TF-IDF algorithm and it is fed as input into BiLSTM (Bidirectional LSTM) to capture the context information effectively. The proposed method is compared with other techniques like NB, LSTM, CNN and RNN. Results showcases that the proposed method has higher accuracy.

Paper [11] provides survey of various deep learning networks like RvNN, RNN, CNN, DGNN and Deep learning techniques along with their operating framework. Various applications of deep learning is also elaborated.

Sepp Hochreiter and Jurgen Schmidhuber [5] proposed long short-term memory (LSTM). Paper explains how the constant error flow in the naïve approach is overcome by LSTM. LSTM is experimented in Embedded Reber Grammar, Noise-Free and Noisy Sequences, Adding Problem, Multiplication problem. Advantages of LSTM is discussed in detail.

Power consumption forecasting system is proposed using deep learning and ensemble deep learning techniques [9]. LSTM-RNN is identified to be more efficient technique for load forecasting when compared with RestrictedBoltzmannmachine, deepbelief networks, Adaptive neuro-fuzzy inference systems and Optimally pruned extreme learning machines.

Malware detection in android is achieved at low cost by implementing Long Short-term Memory (LSTM) [12]. Performance of LSTM network is better when compared with RNN and other machine learning classifiers.

It is understood that sentiment analysis is very efficient and effective tool when used with appropriate algorithms. In RNN [4, 19] output of previous step is given as the input to current step unlike traditional Feed forward Neural Network where connection between nodes do not form cycle. One of the key challenges in training the artificial neural network is vanishing gradient problem [13]. In some scenarios, the gradient would be relatively small and in worst case scenario this might stop the network from further training. Vanishing gradient problem in RNN is solved using LSTM [7, 10]. RNN-LSTM has memory [18] that captures information that has been calculated so far and allows the information to persist. RNN-LSTM are promising solutions to sequence and time series problems. Since our project involves huge dataset and requires deeper understanding of emotions, Deep learning RNN-LSTM algorithm is selected for training and prediction.

3 Techniques used in this research

Since RNN-LSTM algorithm has the ability to remember or store the previous input and identify the next sequential data it is being used in our research.

LSTM ALGORITHM

Long short-term memory (LSTM) is an artificial (RNN) architecture used extensively in the field of deep learning. LSTM process sequence of data and also has feedback connections.

A common LSTM unit is composed of different memory blocks called cells [13]. Refer Fig. 1. The cell state and the hidden state are the two states that are carried to next cell. The memory blocks stores or remembers the valid data. Manipulating data in memory is achieved using three mechanism called gates. A forget gate, an input gate, and output gate. Refer Fig. 2 [20].

Fig. 1

LSTM Architecture.

Fig. 2

LSTM cell with forget gate, input gate and output gate.

The individual gate design is shown in Fig. 3.

Fig. 3

Gate with input flows.

Forget Gate

A forget gate is responsible for removing information that are no longer required from the cell state.

Sigmoid function takes X(t) and h(t-1) as input and decides which values to keep and which values to discard.

Input Gate

The input gate is responsible for the addition of useful information to the cell state through three steps.

Regulating values using Sigmoid functions.

Creating vectors for new inputs using tanh.

Multiplying values of sigmoid gate to created vector and finally add useful information to cell state.

Output Gate

Output Gate selects useful information from the current cell state and displays the output. It is achieved in 3 steps.

Create vector using tanh and scale the values rom – 1 to 1.

Sigmoid layer decides which parts of the cell state goes to output.

Multiply the values of sigmoid gate with vectors.

Mathematical Implementation

In this section, the mathematical model of the LSTM algorithm is presented. For this purpose, variables are defined first. The goal of the algorithm is to predict the sentiment of the given employee feedback sentences.

Parameters

C: Cell

h: Output values

t-1: Previous LSTM block value

t: Current block values

σ: Sigmoid function

tanh: hyperbolic tangent function

f_t: Forget gate vectors

i_t : Input gate vectors

o_t : Output gate vectors

W: Weights assigned for input

w: Weights assigned for recurrent output

b: bias value

Operations

+: summation

×: multiplication

The gate computations [14, 20] are described in equations below $f_{t} = σ (W_{f} x_{t} + w_{f} h_{t - 1} + b_{f})$ (1) $i_{t} = σ (W_{i} x_{t} + w_{i} h_{t - 1} + b_{i})$ (2) $o_{t} = σ (W_{o} x_{t} + w_{o} h_{t - 1} + b_{o})$ (3) $c_{t} = f_{t} * c_{t - 1} + i_{t} * σ_{c} (W_{c} x_{t} + w_{c} h_{t - 1} + bc)$ (4) $h_{t} = o_{t} * σ_{h} (ct)$ (5)

4 Experimental setup and proposed system

4.1 Comparative study of various algorithm performances

This project begins with collecting the employee feedback dataset and classify them as positive and negative comments. Refer Table 1.

Table 1
Sample Dataset

Positive Negative

People are smart and friendly There is lower compensation than expected

Company is good Pressure is there, review system makes no sense

Pretty good pay Easy to get lost, as it is such a big company

Good managers, benefits, atmosphere Long hours, far locations, a lot of work

Positive	Negative
People are smart and friendly	There is lower compensation than expected
Company is good	Pressure is there, review system makes no sense
Pretty good pay	Easy to get lost, as it is such a big company
Good managers, benefits, atmosphere	Long hours, far locations, a lot of work

Dataset is loaded and cleaned dataset by removing the html tags and numbers. All the sentences are converted into lowercase and split into words. Stop words are removed. The Models are defined. Refer Table 2. sklearn is an Open-source library in Python that defines many supervised, unsupervised learning algorithms. The Machine learning algorithms used in our model are Logistic Regression, Support Vector Machine, Random Forest Classifier, AdaBoost Classifier, Gradient Boosting Classifier, Decision Tree Classifier and Gaussian Naïve Bayes. Refer Fig. 4.

Table 2

Model Definitions

Model	Definition
Logistic Regression	LR = LogisticRegression()
Support Vector Machine	SVM = SVC(probability = True)
Random Forest Classifier	RF RandomForestClassifier(n_estimators = 1500)
AdaBoost Classifier	AB = AdaBoostClassifier(n_estimators = 1000)
Gradient Boosting Classifier	GB = GradientBoostingClassifier(n_estimators = 1000)
Decision Tree Classifier	DT = DecisionTreeClassifier(criterion = “entropy”)
Gaussian Naïve Bayes	NB = GaussianNB()

Fig. 4

Performance computation block diagram.

Split the dataset size ‘test_size’ as 0.2, so that 80% of dataset is allocated for training and 20% of dataset for testing purpose. The dataset is trained with each model and accuracy and classification report for each model is displayed. The size of dataset is changed from 200,400,600,800,1000 and accuracy of each algorithm is noted. The achieved performance is compared manually.

4.2 Performance of RNN-LSTM Algorithm in training the dataset

The sentiment analysis Architecture Diagram Fig. 5 explains the input and output that flows through the system and the training and prediction performed by the RNN-LSTM algorithm.

Fig. 5

Architecture diagram of sentiment analysis on employee feedback reviews.

During training RNN-LSTM algorithm, dataset of size 32 k is passed. Dataset is loaded and cleaned by removing the html tags and numbers. All the sentences are converted into lowercase and split into words. The stop words were removed. Word to int and int to word dictionary is created. Matrix is created to tokenize the words [1].

LSTM model is designed with 500 nodes and set the dropout as 20% (where dropout is the prediction technique which randomly shuts down some neuron from overfitting). Density of LSTM model is set to 1. Activation function used is “Sigmoid”.

LSTM model is trained by calling fit function and it returns history attributes which contains training metrics for each epoch. Accuracy is captured while training the dataset.

4.3 Sentiment analysis with LSTM algorithm

Once the training is over, Web application is launched. User is allowed to select an organization from the list of 10 organizations. On clicking the ‘submit’ button sentiment prediction module is called and each feedback comment for the selected organization is analyzed and classified as Positive or Negative and predicted result count is displayed in the web page. Sentiment analysis Implementation has single LSTM layer, with 500 LSTM cells/unit. Dropout is set as 0.2. Each sentence is split into words. Words are converted into vectors and added to the word to int list. Use the weights created during LSTM training. Inputs of different lengths are padded, and it is reshaped into a matrix of single row and multiple columns. Finally, the sentence is predicted as positive or negative based on the value 0 or 1 available in the matrix position [0,0].

The mathematical procedure involved in identifying the sentiment (0/1) is listed from step 1 to step 11 and the calculations are performed using equations Equations (1) to (5).

Step 1: Execution starts with forget gate (equation 1) where weight W_f and w_f gets multiplied with the input value x_t . and previous output h_t - 1 respectively. Bias b_f is added to the input.

Step 2: Activation function (Sigmoid) is applied to values to convert them between 0 and 1. If the value is greater than 0.5, then it is assumed as 1 else it is assumed as 0.

Step 3: Weights and bias are added for input gate (equation 2), sigmoid function is applied.

Step 4: Similarly weights and bias are added for output gate (equation 3), sigmoid function is applied.

Step 5: For candidate gate (c_t)tanh function is applied.

Step 6: Next sell state vector is calculated using candidate gate equation (equation 4).

Step 7: Output h_t is calculated using the equation 5.

Step 8: Calculated candidate gate value (c_t)and Output (h_t) is passed to the next LSTM cell.

Step 9: Step 1–8 is repeated for all cells.

Step 10: Finally, LSTM produces output.

Step 11: If the obtained calculated value is 1 then sentiment is classified as “Positive” else sentiment is classified as “Negative”.

Similarly repeat step 1 to 11 for all the feedback comments for the selected organization. Predicted result is saved in the Result.xlsx file for the respective feedback comments. Total number of positive and Negative comments and number of feedback comments with respect to salary, workload, culture is created as report and sent to the HR of respective organization.

5 Experimental results and discussion

5.1 Performance of algorithms

The accuracy of each model is captured by varying the size of dataset from 200 to 1000. The performance of all models in training the dataset is shown in Table 3. The graphical representation of results is shown in Figs. 6 –10.

Table 3
Result: Performance of each training model

S.No Models/clasifier Size of data set 200 400 600 800 1000

1 Logistic Regression 85.0% 80.0% 83.0% 84.0% 85.0%

2 Support Vector Machine 70.0% 62.0% 66.0% 61.0% 73.0%

3 Random Forest Classifier 78.0% 76.0% 81.0% 82.0% 80.0%

4 Adaboost Classifier Accuracy 83.0% 78.0% 78.0% 81.0% 81.0%

5 Gradient Boosting Classifier 78.0% 81.0% 81.0% 86.0% 83.0%

6 Decision Tree Classifier 75.0% 70.0% 77.0% 76.0% 75.0%

7 Gaussian Naive Boyes 80.0% 78.0% 76.0% 80.0% 80.0%

S.No	Models/clasifier	Size of data set	200	400	600	800	1000
1	Logistic Regression		85.0%	80.0%	83.0%	84.0%	85.0%
2	Support Vector Machine		70.0%	62.0%	66.0%	61.0%	73.0%
3	Random Forest Classifier		78.0%	76.0%	81.0%	82.0%	80.0%
4	Adaboost Classifier	Accuracy	83.0%	78.0%	78.0%	81.0%	81.0%
5	Gradient Boosting Classifier		78.0%	81.0%	81.0%	86.0%	83.0%
6	Decision Tree Classifier		75.0%	70.0%	77.0%	76.0%	75.0%
7	Gaussian Naive Boyes		80.0%	78.0%	76.0%	80.0%	80.0%

Fig. 6

Result: Accuracy of algorithms when the dataset size is 200.

Fig. 7

Result: Accuracy of algorithms when the dataset size is 400.

Fig. 8

Result: Accuracy of algorithms when the dataset size is 600.

Fig. 9

Result: Accuracy of algorithms when the dataset size is 800.

Fig. 10

Result: Accuracy of algorithms when the dataset size is 1000.

When the dataset size was increased from 200 to 400 there was dip in performance of all the models except Gradient Boosting whose performance rose from 78% to 81%. When the dataset size was increased gradually from 400 to 1000 the performance started to increase.

When the dataset size is 200, the performance of Logistic regression is better with 85% accuracy and support vector machine shows lowest performance with 70% accuracy when compared with other models. Refer Fig. 6.

When the dataset size is 400, the performance of all the models dropped when compared to the previous performance except Gradient Boosting classifier whose performance increased from 78% to 81%. Refer Fig. 7.

When the dataset size is 600, the performance of Logistic regression is better with 83% accuracy, followed by Random forest classifier and gradient boosting classifier with 81% accuracy. Refer Fig. 8.

When the dataset size is 800, the performance of Gradient boosting classifier is better with 86% accuracy and Logistic regression with 84% accuracy. Support Vector Machine performs with lowest accuracy of 61% when compared with other models. Refer Fig. 9.

When the dataset size is 1000, the performance of Logistic regression is better with 85% accuracy and support vector machine with lowest accuracy of 73% when compared with other models. Refer Fig. 10.

5.2 Performance of RNN-LSTM in Training the dataset

The dataset of size 32 k is trained using LSTM model and accuracy is captured. The Epoch (Number of times the entire dataset is passed through by the learning algorithm) is changed as 2,3 and performance is captured.

Loss value refers to the errors made during training or validation. When the loss value is lower better model is achieved. The Validation Accuracy achieved for epoch 2 is 89% and it is shown in Fig. 11.

Fig. 11

Result: Performance of LSTM with dataset size 32 k with epoch 2 and accuracy 89%.

The Validation Accuracy achieved for epoch 3 is 88% and it is shown in Fig. 12.

Fig. 12

Result: Performance of LSTM with dataset size around 32 k with epoch 3 and accuracy 88%.

5.3 Sentiment analysis with RNN-LSTM

On launching the web application, organizations list would appear on the screen. On selecting the organization, feedback review comments are analyzed, and result is updated in the “Result” excel file created in the same location. Sample output is shown in Table 4 and report is generated as shown in Figs. 13 and 14.

Table 4
Result: Sentiment Predictions using LSTM Algorithm

S.No Employee reviews for company 1 Result

1 It is a fast-changing company making leaps and bounds in space of data engineering and data science for better customer experience. There is a lot of good work for developers. Positive

2 Enjoyed working there. Very good culture. Everyone I worked with were very disciplined and very organized. I enjoyed coming every day and working with team members. Positive

3 My typical day started at 7am and ended at 7pm. It was high pressure and non-responsive system that drains an individual and causes one to become apathetic about their work. I’ve never worked in a more toxic environment. The Union has too much power and basically does whatever they want. Negative

4 They like to scare people with threat of suspension or firing. They have their own security force whose only job is to create fear amongst the employees that at any point you can be fired or suspended Negative

S.No	Employee reviews for company 1	Result
1	It is a fast-changing company making leaps and bounds in space of data engineering and data science for better customer experience. There is a lot of good work for developers.	Positive
2	Enjoyed working there. Very good culture. Everyone I worked with were very disciplined and very organized. I enjoyed coming every day and working with team members.	Positive
3	My typical day started at 7am and ended at 7pm. It was high pressure and non-responsive system that drains an individual and causes one to become apathetic about their work. I’ve never worked in a more toxic environment. The Union has too much power and basically does whatever they want.	Negative
4	They like to scare people with threat of suspension or firing. They have their own security force whose only job is to create fear amongst the employees that at any point you can be fired or suspended	Negative

The total count of positive and negative comments is displayed in the pie chart as shown in the Fig. 13.

Fig. 13

Result: Sentiment Predictions using LSTM Algorithm.

The Bar chart representing the number of positive and negative comments with respect to salary, work culture and pressure is displayed after performing sentiment analysis. Refer Fig. 14.

Fig. 14

Result: Bar chart representing positive and negative comments with respect to salary, work culture and work pressure.

On comparing our LSTM performance result with the proposed technique mentioned in [1], it is identified that both the results remain in the same range. Refer Table 5.

Table 5

Result of deep learning based sentiment analysis model

Study	Model	Accuracy
Ahmad et al. [1]	1 Layer LSTM	85.07%
Proposed method	1 Layer LSTM	89%

6 Conclusion

The cognitive performance of various models in training and prediction of employee feedback dataset is found. This paper also analyzes the performance of RNN-LSTM algorithm in training dataset which showed up performance of 89%. Also, sentiment analysis is carried out using RNN-LSTM algorithm.

To run more epochs for large dataset high performance system is required.

With 500 dataset RNN-LSTM displayed an accuracy of 72% [15] whereas with higher datasets like 32 k, accuracy increased to 89%. RNN-LSTM algorithm performed better with bigger dataset. Overall, RNN-LSTM algorithm performed well in training and classifying sentiments.

Though, reasonably good performance is achieved using RNN-LSTM network, still there is scope for future enhancement. To improve the performance and accuracy more hidden layers and 50 K dataset will be used. Number of epochs will be increased to improve the performance and accuracy.

The same procedure could be applied in other applications like college review sentiment analysis [15], product review sentiment analysis. The discussed method could act as an experimental baseline for further research in sentiment analysis using RNN-LSTM algorithm.

References

Ahmad

, Asghar

M.Z.

, Alotaibi

F.M.

, et al., Detection and classification of social media-based extremist affiliations using sentiment analysis techniques, Hum Cent Comput Inf Sci 9(24) (2019).

Shrestha

and Mahmood

, Review of Deep Learning Algorithms and Architectures, in, IEEE Access 7 (2019), 53040–53065.

Alaoui

E.l.

, Youssef

, Messoussi

, Chaabi

, Todoskoff

, Kobi

and Abdessamad , A novel adaptable approach for sentiment analysis on big social data, Journal of Big Data (2018), 5. 10.1186/s40537-018-0120-0.

Litjens

, Kooi

, Bejnordi

B.E.

, Setio

A.A.A.

, Ciompi

, Ghafoorian

, van der Laak

J.A.W.M.

, van Ginneken

and Sánchez

C.I.

, A survey on deep learning in medical image analysis, Medical Image Analysis 42 (2017), 60–88. ISSN 1361–8415.

Hochreiter

and Schmidhuber

, Long Short-term Memory, Neural Computation 9 (1997), 1735–1780. 10.1162/neco.1997.9.8.1735.

, Shiyu

, Xiusheng

and Zhigang

, A study on Deep Neural Networks framework, 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Xi’an, (2016), 1519–1522.

Kwon

, Kim

, Suh

, Kim

and Kim

, A survey of deep learning-based network anomaly detection, Cluster Computing 22 (2019). 10.1007/s10586-017-1117-8.

Zvarevashe

and Olugbara

O.O.

, A framework for sentiment analysis with opinion mining of hotel reviews, 2018 Conference on Information Communications Technology and Society (ICTAS), Durban, (2018), 1–4.

Motepe

, Hasan

, Twala

and Stopforth

, Effective load forecasting for large power consuming industrial customers using long short-term memory recurrent neural networks, Journal of Intelligent & Fuzzy Systems 37 (2019), 1–17. 10.3233/JIFS-190658.

10.

Nicholas

, Ooi

S.Y.

, Pang

, Hwang

and Tan

S.-Y.

, Study of long short-term memory in flow-based network intrusion detection system, Journal of Intelligent & Fuzzy Systems 35 (2018), 1–11. 10.3233/JIFS-169836.

11.

Pouyanfar

, Sadiq

, Yan

, Tian

, Tao

, Presa Reyes

, Shyu

M.-L.

, Chen

S.-C.

and Iyengar

, A Survey on Deep Learning: Algorithms, Techniques, and Applications, ACM Computing Surveys 51 (2018), 1–36. 10.1145/3234150.

12.

Vinayakumar

, Kp

, Poornachandran

and Kumar

S.S.

, Detecting Android malware using Long Short-term Memory (LSTM), Journal of Intelligent & Fuzzy Systems 34 (2018), 1277–1288. 10.3233/JIFS-169424.

13.

Singh

, Kumar

and Kp

, Attention based English to Punjabi neural machine translation, Journal of Intelligent & Fuzzy Systems 34 (2018), 1551–1559. 10.3233/JIFS-169450.

14.

Sooraj

, Manjusha

, Kumar

and Kp

, Deep learning based spell checker for Malayalam language, Journal of Intelligent & Fuzzy Systems 34 (2018), 1427–1434. 10.3233/JIFS-169438.

15.

Sri Devi

, Dhanalakshmi

and Sankar

, An Improved Framework for Sentiment Analysis for College Reviews, International Journal of Advanced Trends in Computer Science and Engineering 9(2) (2020), 1959–1963.

16.

Medhat

, Hassan

and Korashy

, Sentiment analysis algorithms and applications: A survey, 2014 Ain Shams Engineering Journal 5 (2014), 1093–1113.

17.

, Meng

, Qiu

, Yu

and Wu

, Sentiment Analysis of Comment Texts Based on BiLSTM, IEEE Access 7 (2019), 51522–51532. 10.1109/ACCESS.2019.2909919.

18.

, Liu

, Zhu

, Zhang

and Zhao

, Attention-based LSTM, GRU and CNN for short text classification, Journal of Intelligent & Fuzzy Systems (2020), 1–8. 10.3233/JIFS-191171.

19.

Zhang

, Yang

L.T.

, Chen

and Li

, A survey on deep learning for big data, Inf Fusion 42 (2018), 146–157.

20.

Zhao

and Hou

, Recognition of motion state by smartphone sensors using Bi-LSTM neural network, Journal of Intelligent & Fuzzy Systems 35 (2018), 1–10. 10.3233/JIFS-169709.