Sentiment analysis model for Airline customers’ feedback using deep learning techniques

Abstract

Sentiment analysis (SA) has recently developed an automated approach for assessing sentiment, emotion, and these reviews or opinions to extract relevant and subjective information from text-based data. Analyzing sentiment on social networks, such as Twitter, has become a powerful means of learning about the users’ opinions and better understanding and satisfaction. However, it consumes time and energy to disperse and collect surveys from clients, often inaccurate and inconsistent, and evaluating and improving the accuracy of the methods in sentiment analysis is being hindered by the challenges encountered in Natural Language Processing (NLP). This paper uses NLP, text analysis, biometrics, and computational linguistics to detect and extract replies, moods, or emotions from Skytrax Airline Customers' Feedback SACF data. This research uses deep learning models to analyze various approaches applied to small SACF to solve sentiment analysis problems. We applied word embedding (Glove embedding models) to improve the sentiment classification performance of a series of datasets extensively utilized for feature extractions. Finally, a comparative study has been conducted on the SACF data analysis utilizing deep learning (DL)for evaluating the performance of the different models and input features, which is Recurrent Neural Networks (RNN), long short-term memory (LSTM), Gated Recurrent Unit (GRU), 1D Convolutional Neural Networks (CONV1D), and Bidirectional Encoder Representations from Transformers (BERT) for application to big datasets in 2019. This approach was assessed using each classification technique; the precision, recall, f1-score, and accuracy metrics for sentiment analysis have been identified. And The results show that LSTM outperforms in classification accuracy; the outcome was 91%.

Keywords

Sentiment analysis skytrax airline customer’s feedback natural language processing word embedding’s glove deep learning

Introduction

The airline industry is a good case study for analyzing customer feedback because it is a highly competitive industry where customer satisfaction is critical to their strengths and weaknesses. By analyzing online passenger reviews, researchers can identify the factors that affect the perceived service quality, performance, and value of different airlines. Online reviews are a rich data source that reflects the opinions, expectations, and emotions of customers who have experienced the service firsthand.

Collecting consumer feedback through surveys and other mechanisms can assist airlines in identifying improvement opportunities. By analyzing customer comments with sentiment analysis, airlines can better understand what matters most to customers. Additionally, the Internet has increased price transparency, reducing airline profit margins and making passenger experience and satisfaction an even greater priority.^1,2

Despite the importance of the SACF industry, it faces challenges and difficulties,^3,4 including dealing with unorganized and heterogeneous data, as online assessments are usually unorganized and need a consistent format. They can contain various types of information, such as text, ratings, images, videos, emoji’s, etc., and vary in length, style, tone, language, and sentiment. These characteristics make it difficult to process, extract, and integrate relevant information from online reviews using traditional methods.

On the other hand, dealing with noise and bias in online reviews. Online reviews may contain noise and bias that can affect the quality and reliability of data. for example, some reviews may be fake, spam, or malicious, which are intended to manipulate the reputation or rating of a product or service. External factors, such as incentives, social pressures, or personal preferences, may influence revisions. Some reviews may need to be completed, consistent, or accurate, which may mislead the analysis.

In addition to understanding customer needs and preferences, online reviews reflect the opinions, expectations and feelings of customers who have experienced the service firsthand. However, customers may have different needs and preferences when evaluating a service. For example, some customers may care more about price, while others may focus more on quality. Some customers may be more satisfied with a service than others, even if they give the same rating. Therefore, it can be challenging to understand customers' needs and preferences from online reviews and provide them with personalized and comprehensive feedback.

NLP and emotion analysis are interrelated disciplines that can bridge the divide between humans and machines. Text SA is utilized to extract and evaluate data from personal data or internet-based product reviews and comments. Due to its commercial and academic applications and the rise of social networks, SA has become an important issue in NLP.^5,6 Deep learning is learning a portion of knowledge in the form of multiple levels of representation and abstraction to form higher-level information than lower-level information (such as sound, image, etc.).⁷

Several ways for assessing user sentiment in social media data have been reported in prior research⁸ by Domenico et al.⁹ For instance, Wang et al.¹⁰ offer a hybrid machine learning system for categorizing negative or positive user sentiment. Bhat et al.,¹¹ human sentiments are crucial to comprehending individual emotions. Pham and Le¹² Utilize NLP to decipher user sentiment from a set of customer evaluations. Aflakparast and others. Sezgen et al.¹³ point out that sentiment analysis of airline ratings is prevalent in social media such as Twitter,¹⁴ Skytrax,¹⁵ and TripAdvisor;¹⁶ therefore, the Airline Customers' Feedback industry is a relevant and interesting case study for both academic and practical purposes, has been undertaken in a few research.

There are some works from previous studies that presents an approach that relies on the human expert and the repetition of the words used, which is similar to the concept of unsupervised machine learning and is represented by demonstrating the development of an emotion-based apparel design system, with a focus on AW-to-ADP mapping, specifically from AWs to ADPs, permitting emotional design. With neural networks, both conventional and higher-order neural networks were employed, with training data acquired and evaluated using the MSE and the R test. To confirm the mapping, a case study was conducted. According to the results, the linear NN with two hidden layers mapping was superior to the quadratic NN with one hidden layer mapping. Drawing is appropriate for the affective design of apparel products. The paper differs from the Kawabata Evaluation System (KES) in two significant ways: the number of training data is restricted, and the quadratic NN model has only one hidden layer.¹⁷

Also, previous studies analyzed flight demand forecasting for airline network planning, flight scheduling, human resource scheduling, and revenue management. It divides flight bookings into peak, average and off bookings based on season and DOW attributes. The study uses a 2D sequence prediction problem and a robust adaptive learning model for nonlinear data with large fluctuations. The SVR model improves the prediction accuracy and RMSE compared to traditional models. The test dataset includes 2 years’ worth of data from tourism, business, and public roads in China¹⁸

The paper contributions are Using customer data from SACF, Models using word embedding (Glove embedding models) to improve the sentiment classification performance have been applied to a series of datasets, extensively utilized for feature extractions and were compassed in the training and validation phase. Finally, a comparative study has been conducted on the SACF data analysis utilizing deep learning for evaluating the performance of the different models and input features, which is RNN, LSTM, GRU, CONV1D, and BERT for application to big datasets in 2019.

The experimental results on datasets demonstrate performed airline customer sentiment analysis using deep learning algorithms RNN,86% LSTM,91% GRU, 90%CONV1D 87%and BERT transfer learning 90%. In this investigation, the study shows that deep learning is better for analyzing the sentiments within the text; LSTM was the best model, with an accuracy of 91%. Consequently, the proposed method could be a promising solution for improved classification that is scalable. The proposed method would be ideally suited for applications with large datasets, such as the sentiment analysis of product and service reviews. It can also be upgraded such that topic recognition and sentiment analysis can be used to construct the suggested model for different languages. Like Arabic reviews, we can work with different types of emoticons for different media.

Background

Numerous public evaluations, in-text remarks, postings, and status updates accompany youth participation in online activities such as Facebook, Twitter, YouTube, and e-commerce websites. NLP is fundamentally dependent upon sentiment analysis. Nonetheless, we categorize these works.

Sentiment analysis

Definition 1: SA is extracting information about an entity and automatically identifying that entity’s subjectivities. The purpose is to determine whether user-generated text expresses positive, negative, or neutral sentiments. Sentiment classification can be accomplished on three levels of extraction: aspect or feature level, sentence level, and document level. Currently, it is in effect.¹⁶

Definition 2: SA is the computational study of people’s sentiments, emotions, evaluations, and attitudes toward various categories of entities, such as products, services, organizations, people, concerns, events, topics, and associated attributes. Review sites, forums, blogs, micro-blogs, Twitter, and other Web-based social media have all emerged. This new industry is expanding due to the unprecedented quantity of user-generated content now available in digital format. Since the early 2000s, sentiment analysis has been one of the most actively studied topics in natural language processing. Data mining, Web mining, text mining, and information retrieval are prominent applications.¹⁹

The user’s mood can be calculated using a tweet's positive and negative terms.²⁰

sentiment score = \frac{(P - N)}{(P + N + 2)}

(1)

P and N represent the number of positive and negative words in a tweet, while S is the sentiment score:

S \in {- 1, 1} .

(2)

Negative and positive sentiment scores cancel each other, resulting in a sentiment score of zero, i.e. (sentiment score = 0).

Zero-emotion score produces erroneous data, so criteria must be used to distinguish between good and negative tweets.

\begin{array}{l} S = 1 (positive tweet) if sentiment score \geq 0.1 \\ S = - 1 (negative tweet) if sentiment score < 0.1 . \end{array}

(3)

The polarity of tweets is calculated to determine if they are positive, negative, or neutral.

Deep learning

Definition 1: DL is an artificial intelligence (AI) function that processes, detects things, recognizes the voice, translates language and makes choices like the human brain. DL is a machine learning algorithm that may be used, among other things, to identify fraud or money laundering. The advent of deep learning has resulted in an explosion of data in all formats and from all regions of the globe. This information is known as big data. Deep is a subtype of machine learning that analyses data using hierarchical neural networks. Throughout these neural networks, many nodes are interconnected, similar to the human brain. The concept of deep learning was initially proposed in the 1980s. There are two primary reasons why it just became helpful recently²¹:

⁃ An enormous amount of labeled data is required for deep learning. For a self-driving vehicle, for example, millions of photographs and hours of video are required.

⁃ Deep learning requires a large computer capacity. The parallel architecture of high-performance GPUs is effective for deep learning.

Definition 2: DL is a subset of machine learning (ML) that offers great flexibility and learning power by representing the world as concepts with nested hierarchy, wherein these concepts are defined in simpler terms and more abstract representations are reflective of less abstract ones.^1–6 In particular, DL’s hidden-layer architecture enables incremental category learning.²²

Literature review

Sentiment analysis analyses emotions from textual data such as Internet articles, customer ratings, and Tweets. People’s perspectives on a variety of problems are often expressed in tweets. Airline travel is now one of the most popular topics. This data is beneficial when studied using deep learning algorithms since it contains factual information. The literature on sentiment analysis is so extensive and varied.

This section summarizes the literature review of the recent research studies performed by researchers from 2018 to 2022. The study shows that DL is better for analyzing the sentiments within the text, as shown in Table 1.

Table 1.

Sentiment analysis on airline industry the use of machine learning and deep learning.

Authors	Year	Feature extraction	Datasets airline	ML/DL classifier model	The best classifier model for accuracy	Limitation
Patel et al²³	2023	Negative comments, a word cloud and a bar graph. NLP tasks: MLM and SP.	Twitter US airline sentiment	LG,KNN,NB, SVM, DT, RF, BERT.	The best classifier model for BERT accuracy 83%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
Rai et al²⁴	2022	Bag of words, TF-IDF	“Twitter US airline sentiment”, (14,640) tweets	LR, DNN	DNN accuracy 89%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
Monalisha Sahoo et al²⁵	2022	Bag of words, TF-IDF	“Twitter US airline sentiment”, (14,640) tweets	SVM, LR, RF, NB	RF accuracy 92.39%	• Small sample of data
						• Not advanced method
						• Applied only for English language
Singh et al²⁶	2022	TF-IDF	“Twitter US airline sentiment”, (14,640) tweets	LR, RF	Logistic regression accuracy 79.1%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
						• Not advanced method
Jain et al²⁷	2021	Vectors of numbers with Kera’s embedding	− Airline quality airline sentiment data (www.airlinequality.com).186 various airlines around the globe	LR, DT, NB, SVM, LSTM, CNN, CNN-LSTM.	CNN-LSTM accuracy 90.2, 91.3%	• Small sample of data
			− Twitter airline sentiment data (www.data.world/crowdflower/).14,640 tweets labeled and U.S. airlines			• Not enough accuracy
						• Applied only for English language
Kumari et al²⁸	2021	Bagging classifiers, non-bagging classifiers	U.S. airlines Twitter	LR, RF, NB, DTC, SVC, SGDC	LR accuracy 77,27%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
						• Not advanced method
Rana Al-Qahtani²⁹	2021	N-gram, Bag-of word, TF-IDF with, Spacy with (Bigram, Trigram)	“Twitter US airline sentiment”, (14,640) tweets	LR, NB, CNN, XLNET BERT, ALBERT	ALBERT accuracy 98,27%	• Small sample of data
						• Applied only for English language
						• Not advanced method
Hasib et al³⁰	2021	Meta data + TFIDF + trainable embedding	“Twitter US airline sentiment”, (14,640) tweets	DNN+CNN	DNN+CNN accuracy 91%	• Small sample of data
Hasib et al³⁰	2021	Meta data + TFIDF + trainable embedding	“Twitter US airline sentiment”, (14,640) tweets	DNN+CNN	DNN+CNN accuracy 91%	• Applied only for English language
Umer et al³³	2020	TF-IDF, Word2vect	Twitter airline sentiment (14640)	LR, RF, SGDC, SV, LR + SGDC, LSTM, CNN-LSTM.	CNN-LSTM accuracy 0.820	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
Manchikant et al⁴⁷	2020	Bigram, information Gain, object-oriented	“Twitter US airline sentiment”, (14,640) tweets	Simple RNN, LSTM, and Stacked LSTM.	Stacked LSTM accuracy 91%	• Small sample of data
Manchikant et al⁴⁷	2020	Bigram, information Gain, object-oriented	“Twitter US airline sentiment”, (14,640) tweets	Simple RNN, LSTM, and Stacked LSTM.	Stacked LSTM accuracy 91%	• Applied only for English language
Prabhakar et al³²	2019	Blogging	13000 tweets from Skytrax and Kaggle for the top 10 US airline carriers	SVM, DT, RF, Bagging, AdaBoost approach	Stacked AdaBoost approach accuracy 68%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
						• Not advanced method
Monika et al⁴⁸	2019	Word2vect Glove	Twitter US airline sentiment	LSTM, RNN.	LSTM accuracy85%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
Rustam et al.⁴⁹	2019	TF-IDF TF Word2vect	Twitter airline sentiment (14640)	AdaBoost, Calibrated, DT, ET, GNB, gradient Boosting machine, LR, RF, Stochastic gradient Descent, S V, Voting classifier (LR + SGDC), LSTM.	Calibrated accuracy 0.789 0.791 0.780	• Small sample of data
						• As not enough accuracy
						• Applied only for English language
Vadivukarassi et al⁵⁰	2018	Unigrams	“Twitter US airline sentiment”, (14,640) tweets	LR, Neighbors, SVC, DT, RF, AdaBoost, G NB	Random forest classifier 81.35%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
						• Not advanced method
Hemakala et al⁵¹	2018	Part-of-Speech (POS) tags, word N-grams, and	“Twitter US airline sentiment”, (14,640) tweets	DT, RF, SVM, NB, LR, KNN, AdaBoost	A three-class dataset is 84.50%	• Small sample of data
						• Not enough accuracy
						• Applied only for English language
						• Not advanced method

Using the preceding table, we may categorize the data in various ways. Researchers,²⁵ [18:20], and [23:31] have used the little tweet data supplied in,³¹ the majority of which was updated in 2015 and had an estimated size of 14,640 tweets. The obtained accuracy values range from 68 to 98%, with research³² achieving the lowest accuracy and research²⁹ achieving the highest accuracy. Researchers²⁷ and³³ have utilized huge data with an estimated size of 29,530 tweets, with²⁷ achieving an accuracy of 91.3% using conventional approaches and just two trending methods (CNN/LSTM) and³³ achieving an accuracy of 82%. So Based on frequently used embedding models for feature extraction, this paper introduces (SACF) data analysis using deep learning algorithms RNN, LSTM, GRU, CONV1D and BERT transfer learning for application in large data. These algorithms have been updated for 2019 and are provided in³⁴; the approximate size of the tweet dataset is 65,944. The proposed study used the application in airlines on a large amount of data and the application using deep learning models to analyze customer opinions.

Methodology

Word embedding

Word Insertion Through word embedding, each word is represented numerically and in vector form. Word embedding refers to documents that contain exact representations of identically meaningful terms. Word embedding, in particular, is unsupervised learning of word representation comparable to semantic similarity. This refers to words in a coordinated scheme where terms with similar meanings are placed closer together based on a set of relationships.³⁵ Word2vec, Glove, and Rapid Text are the most common word embedding models.³⁶ This research will employ Glove embedding.

Glove: “Global Vectors” is an additional technique for word embedding proposed by Pennington et al.³⁷ Word2vec acquires semantic knowledge by executing a local content window over training data to predict words based on their context. Arguing that local content window models suffice for extracting semantic relationships. Glove combines local content windows and count-based matrix factorization to generate an accumulative global co-occurrence statistic of word-word.

Deep learning methods used for sentiment analysis of SACF

DL is a subfield of machine learning based on the structure and composition of ANNs; some of these algorithms have been implemented in airlines using machine learning, but on a small amount of data, and there have been high accuracy and others that are lower. In this paper, we apply some modern algorithms that have yet to be applied in airlines and deep learning in other fields, as shown in Table 1. Finally, I used a sequential search algorithm in my research because it stores data for long periods and effectively solves problems. Time series sequencing works in an iterative process. The suggested study used five deep learning models for customer review sentiment analysis: basic LSTM, RNN, GRU, covn1Dand Bert transform learning model. In this section, we examine several sentiment analysis methodologies. They are listed below:

Recurrent Neural Network (RNN) model

An RNN is an ANN where the nodes are linked in a temporally ordered directed graph. Derived from feedforward neural networks, RNNs can utilize their internal state to process input sequences of varying lengths. The model modifies the existing data using a function by including additional information. Thus, the entirety of the information is transformed, i.e., there is no distinction between 'essential' and 'less significant' information.^38,39

In our experiment, we employ a straightforward 20-layer RNN. At a certain timestamp, the previous input of the previous hidden layer is retrieved and fed back to the current hidden layer’s current input. On the preceding RNN layer, a Spatial Dropout with 0.4 is inserted in. After the RNN layer, Batch Normalization, Dropout with 0.4, and GlobalMaxPool1D are successively added. Lastly, a three-connected layer with a sigmoid activation function is added to the Dense layer.⁴⁰

Long Short-Term Memory (LSTM) model

The LSTM extends the RNN. In the training of vanilla RNN, it was postulated that gradients would dissipate. It has a special memory mechanism that gives it an advantage over the standard RNN⁴¹ and more complex functionality that regulates information flow. Memory cells, input gates, output gates, and neglect gates are components of an LSTM that determine what is to be read, written, or erased. This allocation of responsibilities enables the network model to store data for extended durations⁴²; LSTM has been recognized as an essential function because it effectively resolves time series and sequential problems⁴³; thus, the proposed method could be a promising solution for improved classification with scalability. The proposed method is ideally suited for applications with large datasets, such as detecting sentiment in product and service reviews.

Gated Recurrent Unit (GRU) Model

Gated recurrent unit is a gate mechanism for RNNs. The comparable architecture of LSTM in which GRU consists of only two gates, the update and reset gate, and has a memory of only one concealed state without a distinct cell. The update gate discovers long-term dependencies with successive inputs and determines what data the previous concealed state must transmit. Although the reset gate is supervised to learn short-term dependencies and generate the amount of information to forget, the LSTM network is the same⁴⁰; it is significantly simpler, quicker, and requires fewer gates to calculate.⁴⁴

1D Convolutional Neural Network (Conv1D) model

The CNN method generates a sequential model, indicating that layers are arranged sequentially. The output of one layer will be the input for the following layer. The base layer is the initial stratum. In the subsequent Conv1D layer, a 3-by-64 filter with the ReLU activation function is utilized. The succeeding layer is the Max pooling 1D layer. The dense layer receives input from the MaxPooling1D layer and generates the final output using the SoftMax activation function,²⁵ which makes a 1D CNN quite useful.³⁹ This method applies to the analysis and retrospective review of time sequences of sensor data (such as proximity or barometer data) and the study of any signal data over a predetermined time frame (such as audio signals).

Bidirectional Encoder Representations from Transformers (BERT Transformer learning) model

Transfer learning is a revolutionary paradigm in machine learning that focuses on applying the knowledge acquired for one job to other comparable tasks. Google introduced a new sort of transformer 2018, the BERT sequential language model, 2018. BERT accepts input in the sequence format X = (I0…, In) and generates contextualized vector representations H = (h0,……. hn) for the input sequence elements.

It is a highly generalizable system for language representation that performs tasks through the encoder. Encoders are derived from the transformer’s neural network design to generate encoded text representations. The BERT-Mini in question has four encoder layers.²⁴ Each encoder block consists of two sub-layers: multi-head attention and feed-forward Figure 1.

Figure 1.

Deep learning algorithms (RNN-LSTM-GRU_COVNID_BERT) proposed model architecture for SACF classification.

The proposed model for sentiment analysis classification SACF

Experimental and result analysis

Data description

In the Suggested technique, we will develop deep learning models on the airline sentiment dataset from numerous sources for the top 10 airlines with a total of 65,947 customer evaluations; in the underlying research effort, just one column is analyzed. The column carries a label for the sentiment. This dataset classifies labels as negative or positive, with 33,894 rows representing negative feelings and 30,546 representing good sentiments. This dataset of airline sentiment was collected and available from Kaggle in 2019.³⁴

Pre-processing

In this situation, we must load all necessary libraries, read the data using pandas, and then check for duplicates. If you discover duplicates, leave them there; they are useless. Instead, the data sets’ missing data will be addressed. If our dataset has any missing data, as seen in Figure 2, our model may cause an immense issue. So, the missing values within the dataset must be managed. It may be managed by replacing missing data using the mean or median and the most common term.⁴⁵

Figure 2.

Data preprocessing works on the following data.

Text pre-processing

Pre-processing text is done using the NLP method, a problem-cracking method that helps people access and absorb vast quantities of existing text material. A token is an entity-functioning text of characters consisting of phrases, emotions, hashtags, links, or single characters. Standardization refers to grouping similarly significant terms, and a building data set with every dataset having a "feeling" for training is produced. Text may be categorized by emotional analysis into various emotions, allowing for consistency and availability of the data.⁴⁵ There are different ways of data processing²⁵: -

⁃ Remove and Stop words: - In the article, ‘a' and ‘the’, prepositions ‘in’, ‘near’, ‘beside’ and in pronouns ‘he’, ‘she’, and ‘it’ stops words.

⁃ Lower Casing: - Lowercasing entails turning every uppercase letter to its lowercase counterpart. That will also be quite useful during parsing.

⁃ Tokenization: - Divining a phrase or text into tokens is known as tokenization. Tokens might be words or single characters.

⁃ Lemmatization: - This process reduces a term to its root form. The basic form of the term will make sense after Lemmatization. Many languages include a variety of word forms; hence stemming and Lemmatization varies across languages. Creating effective lemmatization algorithms is an unexplored field of study.

Data splitting

The data set is divided into training and test sets to enhance the performance of a deep-learning model. If the training accuracy of the model is not high, then the performance will decrease. To develop an effective deep-learning model, the training set and test dataset should be used.⁴⁵

Feature extraction

Data extraction is used when the original raw data is vastly dissimilar and cannot be utilized for deep-learning modeling. It converts unstructured data into the required format, and feature extraction is used when the original raw data is so different that no classification can be applied. Examples of raw data formats include Data, Pictures, Date and Time, Web Data, and Sensor Data. Text data is difficult for a computer to comprehend, so feature extraction is used to translate text data into numerical data.²⁵

In this study, 65,947 Internet reviews were retrieved to provide the baseline dataset for sentiment class action. The web evaluations of five major airlines were compiled: ANA All Nippon Airways, Adria Airways, Aegean Airlines Lingus, and Aeroflot Russian Airlines.

Positive and negative categories were established for Internet reviews. The sentiment distribution of Internet reviews is shown in Figure 3 and Table 2 below.

Figure 3.

Distribution of sentiment in the dataset of airline sentiment.

Table 2.

Distribution of online review sentiment for the first dataset.

Airline	Negative	Positive	Total
Air Canada rouge	1768	394	2162
American airlines	2216	560	2776
British airways	1546	1177	2723
Cathay Pacific airways	720	1632	2352
China Southern airlines	233	1456	1689

Deep learning models for sentiment classification

Recurrent Neural Network (RNN) model

To test the efficacy of various embedding, shown in Figure 4 displays the RNN model's specifics, which are detailed in further detail below.

Figure 4.

Recurrent neural networks model for sequential classification.

Long Short-Term Memory (LSTM) model

To test the efficacy of various embedding, shown in Figure 5 displays the RNN model's specifics, which are detailed in further detail below.

Figure 5.

Long short-term memory model for sequential classification.

Gated Recurrent Unit (GRU) model

The GRU model's specifics are described in the following section to evaluate the efficacy of various embedding’s, shown in Figure 6.

Figure 6.

Gated recurrent unit model for sequential classification.

1D Convolutional Neural Network (Conv1D) model

To evaluate the efficacy of various embedding’s, as shown in Figure 7, the Conv1D model’s specifics are described in the following section.

Figure 7.

Conv1D model for sequential classification.

Bidirectional Encoder Representations from Transformers (BERT Transformer Learning) model

All model checkpoint layers were employed for initializing TF Bert for Sequence Classification. The model needs to initialize many layers of TF Bert for Sequence Classification. Figure 8 depicts the “TF Bert for sequence classification” model, which should likely be TRAINED on a downstream job before use.

Figure 8.

Bidirectional encoder representations from transformers model for sequential classification.

Results analysis and discussion

Apply to evaluate user feedback for analysis. We must categorize the review as favorable or negative based on its contents:

1. We preprocessed the information using several natural language processing approaches.

2. We split the dataset into training sets comprising 80% of the total and testing sets comprising 20%.

3. We created word embedding’s, applied five distinct deep learning models, RNN, LSTM, GRU, CONV1D, and BERT, and compared the outcomes.

Python has a variety of deep learning libraries; therefore, it was used for all implementations.

Model training and testing

A labeled dataset that can train the classifier is indispensable for deep learning. The training dataset consists of samples comprising an input item and a label or class organized as a training set. An algorithm initially examines the labeled data. The system then extracts features from the input data that contain the essential information, allowing sentiment analysis to be conducted using a representation of the original data rather than the original data itself. The algorithm generates a function capable of classifying previously unknown data or evaluating the approach. The dataset is then divided into two types: the training set, which consists of 80% of the total, and the testing set, which consists of the remaining 20%. The user can evaluate and test the corresponding model by preprocessing the input data. During the preprocessing phase, the sign and number are removed. Based on the training dataset, it utilizes stored attributes for mapping.

Epochs

Multiple epochs have been utilized to train the full dataset; one will train the entire dataset. Since a single epoch is too large to feed the model simultaneously, it must be broken down into smaller batches. Val loss and Val accuracy reflect the inaccuracy and correctness of the validation data, respectively. Loss and accuracy indicate the error and correctness of training data.

The loss, Val Loss, accuracy, and Val accuracy for each epoch are shown in Tables 3, 4, 5, 6 and 7 of Step 20 for the RNN, LSTM, GRU, CNN, and BERT models. Here, loss and Val Loss decline and converge at a given point on every epoch, indicating that the model is in optimum condition. In contrast, the model's accuracy and Val Accuracy remain constant on every epoch, as shown in Figures 9, 10, 11, 12, and 13.

Table 3.

Epoch of RNN model.

Model RNN
	Step	Loss	Accuracy	val_loss	Val_ accuracy
1/20	25s 52ms	0.5933	0.7044	0.4672	0.8018
2/20	21s 52ms	0.4709	0.8011	0.4480	0.8055
3/20	23s 57ms	0.4477	0.8163	0.3915	0.8494
4/20	21s 51ms	0.4244	0.8255	0.4098	0.8361
5/20	21s 50ms	0.4161	0.8289	0.4731	0.7883
Epoch

Table 4.

Epoch of LSTM model.

Model LSTM
	Step	Loss	Accuracy	val_loss	val_accuracy
1/20	21s 34ms	0.4221	0.8196	0.3357	0.8574
2/20	13s 33ms	0.3230	0.8666	0.3201	0.8556
3/20	14s 33ms	0.2931	0.8807	0.2743	0.8948
4/20	13s 33ms	0.2681	0.8913	0.2550	0.8981
5/20	13s 33ms	0.2525	0.8994	0.2753	0.8931
Epoch

Table 5.

Epoch of GRU model.

Model GRU
	Step	Loss	Accuracy	val_loss	val_accuracy
1/20	9s 18ms	0.4030	0.8231	0.3114	0.8794
2/20	7s 16ms	0.2776	0.8878	0.3664	0.8508
3/20	7s 16ms	0.2406	0.9045	0.2685	0.8982
4/20	7s 17ms	0.2086	0.9171	0.2591	0.8981
5/20	7s 17ms	0.1658	0.9340	0.2700	0.8982
Epoch

Table 6.

Epoch OF COV1D model.

Model COV1D
	Step	Loss	Accuracy	val_loss	val_accuracy
1/20	12s 9ms	0.3766	0.8365	0.3066	0.8736
2/20	2s 5ms	0.2869	0.8822	0.3273	0.8600
3/20	2s 5ms	0.2457	0.9004	0.3098	0.8739
4/20	2s 5ms	0.2076	0.9168	0.5433	0.8126
5/20	2s 5ms	0.1734	0.9313	0.4229	0.8411
Epoch

Table 7.

Epoch of BERT model.

Model BERT
	Step	Loss	Accuracy	val_loss	val_accuracy
1/20	481s 867ms	0.0721	0.9764	0.3479	0.8972
2/20	462s 862ms	0.0448	0.9860	0.4389	0.8911
3/20	462s 862ms	0.0363	0.9881	0.4937	0.8954
4/20	462s 862ms	0.0326	0.9891	0.4945	0.8906
5/20	462s 862ms	0.0221	0.9931	0.5291	0.8918
Epoch

Figure 9.

Recurrent neural networks model accuracy plot.

Figure 10.

Long short-term memory model accuracy plot.

Figure 11.

Gated recurrent unit model accuracy plot.

Figure 12.

COV1D model accuracy plot.

Figure 13.

Bidirectional encoder representations from model accuracy plot.

Evaluation metrics

The accuracy metric is used as a performance matrix. Accuracy is the proportion of properly categorized data points to the total number of data points. With this precision, we can determine whether or not our model will function effectively. If we get 90% accuracy on the test in feature and feed the model any new data point, it will predict with 90% accuracy. The data is balanced after printing the accuracy plot and the confusion matrix. It will provide the number of missing categorized data points per class. The following metrics are utilized in this study⁴⁵:

Using equation (4), accuracy may be measured.

Accuracy = \frac{correctly classified data points (TP + TN)}{total data points (TP + TN + FP + FN)}

(4)

Precision is the ratio of the number of true positives to the total number of true positive forecasts. Equation (5) is used to compute exact numbers.

Precision = \frac{TP}{TP + FP}

(5)

The recall is the proportion of True Positives accurately identified by our model. Equation (6) determines this value.

Recall = \frac{TP}{TP + FN}

(6)

The F1-score represents the natural logarithm of Precision and Recall. The F1 value is defined by equation (7).

F 1 - score = \frac{2 * (Precision * Recall)}{(Precision + Recall)}

(7)

Word cloud

If the item of the word calculates the frequency after the document's preprocessing, it may be used to create a variety of visualizations. This approach of visual representation is generally used and permits a more intuitive portrayal of the document’s content and qualities. The word cloud gives a more understandable representation of the features of the text by displaying the corpus in proportion to its frequency. The word cloud is a strategy for visually representing the most frequently occurring terms, and keywords that are difficult to spot quickly in a table have the benefit of being visually shown instantly in the word cloud. Figures 14 and 15 show the findings of the Word Cloud for positive and negative terms, respectively.⁴⁶

Figure 14.

Words most often in positive comments.

Figure 15.

Words most often in positive comments.

Confusion matrix

Using alternative measures aims to determine how effectively and correctly a deep learning model will function on new data. Figures 16, 17, 18, 19, and 20 depict the basic confusion matrix for a two-class classifier.

Figure 16.

Confusion matrix of the recommended RNN (deep neural network) Method.

Figure 17.

Confusion matrix of the recommended LSTM (deep neural network) Method.

Figure 18.

Confusion matrix of the recommended GRU (deep neural network) Method.

Figure 19.

Confusion matrix of the recommended CONV1D (deep neural network) Method.

Figure 20.

Confusion matrix of the recommended BERT (deep neural network) Method.

In the above equations, TP, TN, FP, and FN stand for true positive, true negative, false positive, and false negative and are defined respectively.

True Positive (TP): TP is the real positive and was correctly predicted; False Positive (FP): FP is the false positive and was correctly predicted; True Negative (TN): TN is the real negative and was correctly predicted; False Negative (FN): FN is the false negative and was correctly predicted.

Deep learning classification models outcomes

This section discusses the categorization results produced (as shown in Table 8 and Figure 21) that may be observed in the graphs given in those tables. Five unique deep-learning algorithms were evaluated throughout our investigation. ' The major purpose was to classify into good and negative groups. We observed that using the RNN, LSTM, GRU, COV1D, and BERT models and extracting the most representative features improved accuracy, precision, recall, and f1-score compared to earlier research. 86% of their predictions were accurate. The study results were 91, 90, 87.7, and 90%, respectively. According to the study's findings, LSTM outperformed other classifiers with an accuracy of 91% based on rendering in big data updated in 2019, accessible at,³⁴ with a volume of around 65,947 tweets.

Table 8.

classification report model.

Classifiers/model		Precision (%)	Recall (%)	F1_score (%)	Support	Accuracy (%)
RNN techniques	0	0.85	0.86	0.86	6048	0.86
	1	0.86	0.85	0.86	6171
	Macro avg	0.86	0.86	0.86	12,219
	Weighted avg	0.86	0.86	0.86	12,219
LSTM techniques	0	0.90	0.91	0.91	6048	0.91
	1	0.91	0.90	0.91	6171
	Macro avg	0.91	0.91	0.91	12,219
	Weighted avg	0.91	0.91	0.91	12,219
GRU techniques	0	0.91	0.89	0.90	6048	0.90
	1	0.90	0.91	0.90	6171
	Macro avg	0.90	0.90	0.90	12,219
	Weighted avg	0.90	0.90	0.90	12,219
COV1D techniques	0	0.88	0.87	0.87	6048	0.87
	1	0.87	0.88	0.88	6171
	Macro avg	0.87	0.87	0.87	12,219
	Weighted avg	0.87	0.87	0.87	12,219
BERT techniques	0	0.88	0.92	0.90	9124	0.90
	1	0.92	0.88	0.90	9204
	Macro avg	0.90	0.90	0.90	18,328
	Weighted avg	0.90	0.90	0.90	18,328

Figure 21.

Comparison of classifier model accuracy of 5 different classifier model using deep learning.

Shown in Figure 21 demonstrates that the F1 scores of the abovementioned approaches are generally constant throughout 10 tests. The sentiment classification effects of RNN and COVN1D are low, and their F1 score is only around 87%. The F1 score may approach 90% when sequence modelling is performed using GRU and BERT. The approach suggested in this study incorporates LSTM structure, which may capture the context’s semantic information more effectively, resulting in superior sentiment analysis. The F1 score has dramatically increased to almost 91%. Table 8 displays the average accuracy, recall, and F1 score of ten repeated trials for various sentiment analysis approaches; Table 8 and Figure 21 demonstrate that the suggested method achieves 90% precision, 91% recall, and 91% F1 score. Other techniques’ F1 scores vary from 86 to 90%, which is lower than the suggested method’s F1 score. In addition, the F1 scores of GRU and BERT are greater than those of RNN and COVN1D for sequential processing tasks.

Conclusions and future work

Due to the recent development of sentiment analysis, it takes an automated approach to assess sentiment and emotion, and these reviews or opinions extract relevant and subjective information from text-based data. Sentiment analysis on social networks like Twitter has become a powerful way to learn about and better understand users' opinions and satisfaction. The tweets are in positive and negative poles, respectively. Specifically, this study aims to provide an analysis of the content of user sentiment text tweets in airlines. However, it is time and effort-consuming to scatter and collect surveys from customers, often inaccurate and inconsistent, and assess and improve the accuracy of sentiment analysis methods hampered by challenges. NLP. Deep learning models have been a promising solution to NLP challenges in recent years. This paper analyzes different approaches applied to Skytrax Small Airline customer feedback to solve sentiment analysis problems. Previous research analyzes different approaches applied to small SACF datasets. Most of them were updated in 2015 and include approximately 14,640 tweets. They have achieved accuracy ranging from 68 to 98%. On the other hand, other algorithms have been used for the large data set of SACF, which consists of about 29,530 tweets, and the resulting accuracy values range from 82% to 91.3.

In this research, word embedding (glove embedding models) was applied to improve sentiment classification performance on a series of datasets. It was widely used for feature extraction and incorporated into the training and validation phase. A comparative study on data analysis (SACF) using deep learning was conducted to evaluate the performance of different models and input features, namely RNN, LSTM, GRU, CONV1D and BERT for application to large datasets in 2019%. The overall results on the datasets showed conducting airline customer sentiment analysis using deep learning algorithms RNN, LSTM 86%, GRU 91%, CONV1D 87%, and BERT transfer learning 90%. Finally, this approach was evaluated using measures of accuracy, recall, score f1, and accuracy for sentiment analysis that were determined for all classification techniques. The results showed that LSTM excels in classification accuracy, and the score was 91%. In future studies, the deep learning augment technique may be created to optimize the features iteratively. It can also be upgraded such that topic recognition and sentiment analysis can be used to construct the suggested model for different languages. Like Arabic reviews, we can work with different types of emoticons for different media. This study will help to develop a sentiment analysis model for airline customer feedback using deep learning techniques, especially RNN, LSTM, GRU, CONV1D, and BERT, as tools to explore sentiment quality rather than quantity. Our evaluation of the different models and also from industrial contributions in this field, including (manufacturing, hospitality, digital assistants (IoT), cars, etc.).

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Heba Allah Samir

References

Ban

Kim

. Understanding customer experience and satisfaction through airline passengers’ online review. Sustainability 2019; 11(15): 4066.

Dike

Davis

Abrahams

, et al. Evaluation of passengers’ expectations and satisfaction in the airline industry: an empirical performance analysis of online reviews. Benchmarking An Int J. 2023.

Kim

. Analyzing and visualizing comprehensive and personalized online product reviews. Cluster Comput 2019; 22: 2115–2128.

Liao

. Learning judgment benchmarks of customers from online reviews. Or Spectrum 2021; 43(4): 1125–1157.

Tompkins

. Disinformation detection: a review of linguistic feature selection and classification models in news veracity assessments. 2019. arXiv preprint arXiv:1910.12073.

Hepburn

(2018, December). Universal language model fine-tuning for patent classification. In Proceedings of the Australasian Language Technology Association Workshop 2018 (pp. 93–96).

Zhang

WJ(Chris)

Yang

Lin

, et al. On Definition of deep learning. Stevenson, WA, USA: World Automation Congress (WAC), 2018, pp. 1–5, DOI: 10.23919/WAC.2018.8430387

Singh

Imam

Wibowo

, et al. A deep learning approach for sentiment analysis of COVID-19 reviews. Appl Sci 2022; 12(8): 3709.

Di Domenico

Sit

Ishizaka

, et al. Fake news, social media and marketing: a systematic review. J Bus Res 2021; 124: 329–341.

10.

Wang

A sentiment-enhanced hybrid recommender system for movie recommendation: a big data analytics framework. Wireless Commun Mobile Comput 2018; 2018: 1–9.

11.

Bhat

Qadri

Kundroo

, et al. Sentiment analysis of social media response on the Covid19 outbreak. Brain Behav Immun 2020; 87: 136.

12.

Pham

. Learning multiple layers of knowledge representation for aspect based sentiment analysis. Data Knowl Eng 2018; 114: 26–39.

13.

Sezgen

Mason

Mayer

. Voice of airline passenger: a text mining approach to understand customer satisfaction. J Air Transport Manag 2019; 77: 65–74.

14.

Dang

Moreno-García

De la Prieta

. Sentiment analysis based on deep learning: a comparative study. Electronics 2020; 9(3): 483.

15.

Liu

Gursoy

. The impacts of service failure and recovery efforts on airline customers’ emotions and satisfaction. J Trav Res 2019; 58(6): 1034–1051.

16.

Korfiatis

Stamolampros

Kourouthanassis

, et al. Measuring service quality from unstructured data: a topic modeling application on airline passengers’ online reviews. Expert Syst Appl 2019; 116: 472–486.

17.

Zhao

Sun

Gupta

, et al. Developing a mapping from affective words to design parameters for affective design of apparel products. Textil Res J 2017; 87(18): 2224–2232.

18.

Fan

Shi

, et al. Support vector regression model for flight demand forecasting. Int J Eng Bus Manag. 2023; 15: 18479790231174318. doi: 10.1177/18479790231174318

19.

Zhang

Shuai

. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2018; 8(4): e1253.

20.

Mahto

Yadav

Lalotra

. Sentiment prediction of textual data using Hybrid ConvBidirectional-LSTM model. Mobile Inf Syst. 2022; 2022: 1–11.

21.

Habib

. Tweet sentiment analysis using deep learning technique. IJITEE (International Journal of Information Technology and Electrical Engineering). 2021; 10(5): 27–36. https://www.researchgate.net/publication/357857129_Tweet_Sentiment_Analysis_using_Deep_Learning_Technique

22.

Alzubaidi

Bai

Al-Sabaawi

, et al. A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications. J Big Data 2023; 10(1): 46.

23.

Patel

Oza

Agrawal

. Sentiment analysis of customer feedback and reviews for airline services using language representation model. Procedia Comput Sci 2023; 218: 2459–2467.

24.

Rai

Pandey

. us airline twitter data on sentiment analysis using deep neural network. Int J Technol Res Eng. 2022; 9(8): 2347–4718. ISSN (Online), https://ijtre.com/wp-content/uploads/2022/05/2022090801.pdf

25.

Rautaray

Sahoo

. Survey on sentiment analysis to predict twitter data using machine learning and deep learning. Int J Eng Res Technol. 2022; 11(7): 506–512.

26.

Singh

Upreti

. HMRFLR: a hybrid model for sentiment analysis of social media surveillance on airlines. Wireless Pers Commun 2023; 132(1): 97–112.

27.

Jain

Saravanan

Pamula

. A hybrid CNN-LSTM: a deep learning approach for consumer sentiment analysis using qualitative user-generated contents. Trans Asian and Low-Resour Lang Inf Process 2021; 20(5): 1–15.

28.

Kumari

. Collaborative classification approach for airline tweets using sentiment analysis. Turk J Comput Math Educ 2021; 12(3): 3597–3603.

29.

Rana

Bint

. Princess Nourah.Predict sentiment of airline tweets using ML models. 2021, (No. 5228), EasyChair.

30.

Hasib

Habib

Towhid

, et al. (2021, February). A novel deep learning based sentiment analysis of twitter data for us airline service. In: 2021 International Conference on Information and Communication Technology for Sustainable Development (ICICT4SD) (pp. 450–455). IEEE.

31.

Small data www.data.world/crowdflower/

32.

Prabhakar

Santhosh

Krishnan

, et al. Sentiment analysis of US airline twitter data using new adaboost approach. Int J Eng Res Technol 2019; 7(1): 1–6.

33.

Umer

Ashraf

Mehmood

, et al. Sentiment analysis of tweets using a unified convolutional neural network-long short-term memory network model. Comput Intell 2021; 37(1): 409–434.

34.

Large data , and Dataset is collected from Kaggle 2019 from Skytrax's Airlinequality.com website. https://www.kaggle.com/datasets/efehandanisman/skytrax-airline-reviews

35.

Iqbal

Amin

Iqbal

, et al. Sentiment analysis of consumer reviews using deep learning. Sustainability 2022; 14(17): 10844.

36.

Kilimci

. Sentiment analysis based direction prediction in bitcoin using deep learning algorithms and word embedding models. Int J Intell Sys App Eng 2020; 8(2): 60–65.

37.

Pennington

Socher

Manning

(2014, October). Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532–1543).

38.

Patel

Tiwari

. Sentiment analysis by using recurrent neural network. In: Proceedings of 2nd International Conference on Advanced Computing and Software Engineering (ICACSE), 2019, February.

39.

Rodrigues

Fernandes

Shetty

, et al. Real-time twitter spam detection and sentiment analysis using machine learning and deep learning techniques. Comput Intell Neurosci 2022; 2022: 5211949.

40.

Bhowmik

Arifuzzaman

Mondal

MRH

. Sentiment analysis on Bangla text using extended lexicon dictionary and deep learning algorithms. Array 2022; 13: 100123.

41.

AlBadani

Shi

Dong

. A novel machine learning approach for sentiment analysis on Twitter incorporating the universal language model fine-tuning and SVM. Appl Sys Innov 2022; 5(1): 13.

42.

Hussein

Özyurt

. A new technique for sentiment analysis system based on deep learning using Chi-Square feature selection methods. Balkan J Electr Comput En 2021; 9(4): 320–326.

43.

Singh

Wibowo

Grandhi

. A deep learning approach for human face sentiment classification. In: 2021 21st ACIS International Winter Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Winter). IEEE, 2021, January, pp. 28–32.

44.

Jasti

Kumar

Sireesha

. Deep sentiment extraction using fuzzy-rule based deep sentiment analysis. International Journal of Advanced Computer Science and Applications. 2022; 13(6): 404–412. DOI: 10.14569/IJACSA.2022.0130650

45.

Farnaaz

Sureshbabu

. Twitter sentiment analysis using deep learning techniques. IJMTST. 2022; 213–214. DOI: 10.46501/IJMTST0802035

46.

Liao

. Learning judgment benchmarks of customers from online reviews. Or Spectrum 2021; 43(4): 1125–1157.

47.

Manchikanti

Madhurika

. Airline tweets sentiment analysis using RNN and LSTM techniques. Int J Adv Trends Comput Sci Eng 2020; 9(5): 8197–8201.

48.

Monika

Deivalakshmi

Janet

(2019, December). Sentiment analysis of US airlines tweets using LSTM/RNN. In: 2019 IEEE 9th International Conference on Advanced computing (IACC) (pp. 92–95). IEEE.

49.

Rustam

Ashraf

Mehmood

, et al. Tweets classification on the base of sentiments for US airline companies. Entropy 2019; 21(11), 1078.

50.

Vadivukarassi

Puviarasan

Aruna

. An exploration of airline sentimental tweets with different classification model. Int J Res Eng App Manag. 2018; 4(02): 2454–9150. DOI: 10.18231/2454-9150.2018.0124

51.

Hemakala

Santhoshkumar

. Advanced classification method of twitter data using sentiment analysis for airline service. Int J Comput Sci Eng 2018; 6: 331–335.