A review of feature selection techniques in sentiment analysis

The following are some examples of FS techniques using the NLP approach:

1.

Parts of speech tagging (POST): Known as grammatical tagging, or word-category disambiguation, POST is the process of marking up a word in a text (corpus) as equivalent to a specific part of speech. In POST, each term in a document or in sentences is allocated a tag or label that represents its position in the grammatical context. For example, “This camera is very good and perfect,” becomes, “This/DT camera/NN is/VBZ very/RB good/JJ and/CC perfect/JJ,” after POST. The nouns and adjectives in this sentence can be identified from this tagging process, and these can be used for selecting features and as sentiment indicators.

2.

Opinion words and target relations: The relationship between the feature and the opinion words can be exploited to extract the feature. This relationship can be shown by the dependency parser, which is used to identify the dependency relation for a selection of features.

3.

Topic modelling: In this method, a generative probability model that uses a distribution of vocabulary was applied to discover topics from a large collection of text documents [40]. Topic modelling is an unsupervised learning method and its function is to identify the mixture of topics in each document [8]. The output from this method is a group of words. For example, in a collection of documents on comments about a camera, the relevant topics would include memory card, battery, megapixels, viewfinder, weight, and price.

4.

Negation word: This is also an important feature to take into consideration because it can change the sentiment’s orientation. For example, “not bad” corresponds to “good”.

5.

Rules of opinions: These are terminologies or language structures that can be used to describe sentiments and opinions.

In [12, 14], the NLP approach was applied to identify the features in comment sentences. POST was used to assess each sentence, and the words in the sentences that were tagged as NN/NNS/NNP/NNPS were identified as nouns or noun phrases, namely, features. The association rule mining [41] was used to determine frequent features, which are sets of commonly used words or phrases. In their studies, [12] assumed that features can be divided into two types: frequent and infrequent. They argued that an infrequent feature could lead to problems in identifying SWs because it might contain nouns or noun phrases that have no connection to the product. They also argued that infrequent features only have a minor impact of 15%–20% on their proposed system. Therefore, only the frequent features were considered in their studies. They used the concept of the “nearby adjective” in identifying SWs. Thus, if a word is tagged with JJ/JJR/JJS, the word is an adjective, thereby, it represents a sentiment word that is the nearest to the frequent feature. In an investigation into FS, [15] used POST to generate the POS tag of each word (to identify nouns, verbs, adverbs, and adjectives), and an n-gram was used to produce shorter segments from long ones.

In a more recent study, [23] used POST to parse each sentence and generate a POS tag for each word in the sentence. After the POS tagging process was completed, they extracted and identified the nouns and noun phrases in the sentences using the concept of “pattern knowledge”. Eight patterns of knowledge were generated in identifying features in sentences. Meanwhile, the extracted SWs had tagging of adjective/JJ, or adverb/RB and were the closest to the extracted feature. According to [24], there are three steps in the process of extracting features of sentences. In the first step, input documents are converted into tagged documents using the POS tagging software. In the second step, evaluative expressions are extracted using the combined pattern-based noun phrases (cBNP) technique. Lastly, this algorithm extracts product features that are usually represented by noun phrases. In their study, [24] used a list of opinion lexicons for identifying sentiment-hood.

Meanwhile, [11] proposed the “Know-It-All” system to extract explicit features from parsed review data. This system is known as OPINE and works recursively in identifying the “parts” and “properties” of product classes. The process stops if there is no “candidate” to be identified. The system searches a concept that has a connection and extracts the said “parts” as well as the “properties”. The system would also extract noun phrases and retains the noun phrase that has a frequency value of greater than the specified threshold value. The process of evaluating the acquired noun phrase is based on a Pointwise Mutual Information (PMI) score between a phrase and a meronymy discriminator that has a connection with the product class. The process of identifying SWs uses explicit features, but is based on syntactic dependencies identified by a MINIPAR parser. Based on the extracted rules, OPINE will search the lexical heads for potential SW phrases. The system identifies the semantic orientation of each lexical head that has a semantic orientation with a positive or negative feature and retains this as the opinion phrase.

In their study, [18] had successfully used term presence compared to term frequency in their sentiment classification of movie reviews. They applied a unigram (BOW) as the feature type to classify movie reviews into two classes, namely, positive and negative. In [42], the authors investigated the differential impact of classifying product reviews using n-grams, bi-grams, and tri-grams. In their research, they obtained good performances for classification using bi-grams and tri-grams.

In [9], “linguistic filtering patterns” and a “general inquirer dictionary” were used for extracting a list of product features. Their process of identifying SWs was based on dependency analysis, which was divided into a dependency tree and a dependency path. Dependency analysis involves identifying an asymmetric binary relationship between words, known as the “head or governor”, or in other words, the “modifier or dependent”. The authors introduced and adopted six types of syntactic relationships to identify the connection between features and SWs.

In [1], it was pointed out that an “unsupervised model” and language-independent model can identify explicit and implicit features from reviews. An unsupervised model does not require a set of pre-labelled training data. Therefore, this was to their advantage because the model can easily be transferred between domains or languages. They used a graph-based approach to identify implicit features in comments. The graph was drawn based on the use of a polarity lexicon and a list of specified features. SWs from the polarity lexicon were used as nodes connected to each feature that had a connection. The authors gave an initial weight to each connection and to add more precision to the identification of implicit features, they defined a function to measure the strength of the connection between the features and SWs. Table 1 provides a summary of the FS and SW identification techniques.

OPEN IN VIEWER

Table 1

FS (NLP approach) and SWs techniques

Author and publication	Feature selection (NLP)	Sentiment word identification
[12, 14]	POS tagging	Nearby adjective
[11]	“Know-It-All” system	Dependency relationship
[9]	Linguistic filtering patterns and general inquirer dictionary	Dependency relationship
[23]	POS tagging and generation of pattern knowledge to identify feature	Concept of nearest adjective with product feature in a sentence
[24]	POS tagging and combined pattern-based noun phrases (cBNP)	Opinion lexicon
[1]	Unsupervised model and language independent model	None

3.4.2 FS techniques using a combination of NLP approach and non-NLP approach

Non-NLP is an approach that is closely related to the machine learning approach. Previous studies have reported using a hybrid FS technique, which is a combination of an NLP approach with a non-NLP approach. Non-NLP approaches include the genetic algorithm (GA), information gain (IG), rough set theory (RST), decision tree, and minimum redundancy maximum relevance (mRMR). FS techniques can be generally categorised as either univariate methods or multivariate methods based on the conditions for evaluating the features. Univariate methods analyse a single variable at a time, while multivariate methods analyse more than one variable at a time. Wrapper and hybrid techniques are included in the multivariate category. There are generally three FS techniques for machine learning tasks, namely, filter, wrapper, and hybrid [34, 43, 44], which are briefly described as follows:

1.

The filter technique is independent of any machine learning algorithm. During the filtering process, the relevant score for each feature is calculated, and the features with low scores are removed. The resulting feature subset becomes the input for the classification algorithm [45].

2.

The wrapper technique needs a machine learning algorithm and uses it as part of the evaluation function.

3.

The hybrid technique, as the name implies, is a combination of the filter and the wrapper techniques.

A. Filtering techniques

A few conclusions can be drawn from the review of previous studies. The advantage of utilising a univariate filtering technique is that it is simple and fast, with a short processing time. However, there are also limitations to this technique; there is no interaction or dependency between features and no interaction with the machine learning algorithm. Feature evaluation is performed individually and usually, the features are listed based on their predictability. As such, the dependency between features would be ignored. Table 2 provides a summary of the advantages and limitations identified in previous studies that used the univariate and multivariate methods.

OPEN IN VIEWER

Table 2

Summary of advantages and limitations of the univariate and multivariate methods

Author and publication	Technique	Advantage (A)/Limitation (L)
Univariate techniques
[10, 17, 37, 46, 47]	Information gain	A: Simple to implement and fast.
		L: No interaction between the attributes, does not consider overlapping features, and threshold value needs to be set in advance.
[48, 49, 50]	Log likelihood ratio	A: Simple to implement and fast.
		L: Complicated when involving more than one feature.
[47, 51]	Chi-squared	A: Simple to implement and fast.
		L: Does not offer much information about the relationship between features.
[47]	Gain ration	A: Simple to implement and fast.
		L: A low feature count could affect the accuracy of the classification and make it unsatisfactory.
[47]	Relief-F	A: Simple to implement and fast.
		L: The accuracy value for sentiment classification is unsatisfactory.
Multivariate methods
[52]	Feature subsumption	A: Improves classification accuracy.
	hierarchy	L: The subsumption process responds to the negative result of adding more complex features.

B. Hybrid techniques

The hybrid technique is a combination of the filter and wrapper techniques that was used by [15, 53]. The hybrid technique can be used to manage a large-sized dataset [44, 53, 54]. This combination of two techniques could create an optimum feature subset by using standard optimising techniques, or a metaheuristic approach, such as the GA, particle swarm optimization (PSO), and ant colony optimization (ACO) [55]. Table 3 provides a summary of the advantages and limitations of using hybrid methods for SA, as identified from the literature review.

OPEN IN VIEWER

Table 3

Summary of the advantages and limitations of hybrid techniques

Author and literature	Technique	Advantage (A)/Limitation (L)
[10, 56]	IG + Entropy Weighted Genetic Algorithm (EWGA)	A: Increases the classification accuracy value and is able to identify the main features in the sentiment class.
		L: Less efficient in determining the ranking of features.

The FS technique and SWs are the main determinants of the accuracy of sentiment classification. SA is based on the machine learning approach and there is normally a large-sized feature space. A number of studies on SA have combined filtering and wrapper techniques to overcome the limitations that exist in each individual technique. For example, [10, 37] used the IG technique to identify important features for sentiment classification. According to [37], IG was used to measure uncertainty reduction when identifying the feature class character once the feature value has been identified. The most important features were selected to reduce the size of the feature vector and come up with a better classification. They also reported that IG is the best filtering technique because it is able to determine the importance of the features in a document. However, the IG has several limitations, such as the threshold value has to be set in advance; the method does not take into consideration the problem of excessive features; and there is no communication among the features [37, 45].

In [10], the IG filtering technique was combined with the Entropy Weighted Genetic Algorithm (EWGA) metaheuristic technique, which had successfully increased sentiment classification accuracy and produced an optimum feature subset. However, the study has its limitation in that the data was in the form of document data. Document data only considers sentiments based on the whole document without refining the content. In [57], the authors also found that SA at the feature level would thoroughly consider the feature and SWs that exist in a document. The initial stage involved conducting the process of extracting the unigram features of the document. The next stage was the process of FS using the IG technique to determine the importance of the features in the document. However, one weakness of the IG technique is that it must determine the threshold value in advance and it does not consider features that overlap [37].

To overcome this problem, [37] combined IG with RST. This technique can reduce the number of noisy and irrelevant features. The advantage of this hybrid approach is that it can consider overlapping features, as well as obtain minimum feature sets that can reduce time complexity during sentiment classification. However, [10, 37] did not clearly state the methods they used to identify SWs. However, the support vector machine (SVM) technique was used for the sentiment classification process by [10], while [37] utilised the SVM and naive bayes (NB) in the sentiment classification process to identify positive and negative sentiments in the documents. In [56], the authors proposed using a GA to extract feature collections from semantic features of emotional collections. In their work, a conditional random field (CRF) was developed for labelling web-page classifications for different types of comments, such as positive comments, negative comments, and objective comments. They did not mention the identification and the relationship between SW and feature.

Table 4 provides an overview of the available techniques for FS and SW identification.

OPEN IN VIEWER

Table 4

Techniques for FS (NLP + non-NLP) and SW identification

Author and publication	Feature selection (NLP + non-NLP)	Sentiment word identification
[10]	NLP approach + IG + GA	Not mentioned
[56]	GA + CRF	Not mentioned
[37]	NLP approach + IG + RST	Not mentioned

C. Other techniques

i. Rough set theory with filtering technique

In [37], the IG technique was combined with the RST technique. The RST was used to reduce the number of irrelevant, excessive, and noisy features. The advantage of using RST is that it can take into account the combined dependency among the features [58]. However, the RST has two limitations, one of which was the difficulty of obtaining an optimum reduction in the feature subset. This was a non-deterministic polynomial-time hard (NP-hard) problem, thus a heuristic algorithm was suggested to resolve it [37]. The second limitation was its long processing time [37, 59, 60]. Thus, only a few studies have explored the metaheuristic approach for FS in SA research. Table 5 provides a summary of the advantages and limitations of using RST combined with a univariate filtering technique.

OPEN IN VIEWER

Table 5

Summary of advantages and limitations of the RST method, combined with a univariate filtering technique

Author and	Technique	Advantage (A)/Limitation (L)
publication
[37]	IG + RST	IG
		A: Determines the importance of a feature in a document.
		L: Requires a threshold value at the initial stage and does not take into account overlapping
		features.
		RST
		A: Could reduce irrelevant features, noise, and takes into account overlapping features.
		L: Difficult to obtain an optimum subset feature and requires more processing time.

ii. Neural network

In [61], the authors compared three types of neural networks, namely, the probabilistic neural network (PNN), the back propagation neural network (BPN), and the homogeneous ensemble of PNN (HEN). The comparison was performed using varying levels of word granularity as a form of feature for feature level sentiment classification based on a dataset of product reviews collected from the Amazon review website. The hybrid combination of sentiment classification methods, which were based on the PNN and the principal component analysis (PCA), acted as a feature reduction analysis to reduce training time and increase the performance of the classification process. These neural networks were developed to effectively incorporate the supervision from sentiment polarity of text.

A research by [62] proposed a method to learn sentiment-specific word embedding (SSWE) by integrating sentiment information into the loss functions of the three neural networks. Additionally, large scale training corpora were acquired by learning SSWE from massive distant-supervised tweets composed by positive and negative emoticons. To evaluate the effectiveness of SSWE, a dataset from SemEval 2013 was used as the benchmark. The verification process was done by computing word similarities in the embedding space for sentiment lexicons. The results indicated that the integration of SSWE with sentiment information of sentences was able to achieve a good performance in the experiments.

Meanwhile, [63] proposed the use of recursive neural tensor networks (RNTN) to calculate the representations of compositional vector for phrases of variable lengths and syntactic types. They had also developed the Stanford Sentiment Treebank corpus, which contains label parse trees. The label parse trees allow a complete analysis on the compositional effects of sentiments in language. Ultimately, their experiment showed that compared to previous models, the RNTN was able to achieve 80.7% accuracy when applied on fine-grained sentiment prediction across all phrases. RNTN can also capture negations of different sentiments.

iii. Metaheuristic

Several metaheuristic approaches were also applied as FS methods in SA. Table 10 displays a summary of metaheuristic approaches that can be used for selecting features in SA based on dataset, and performance evaluation (accuracy, precision (P), recall (R), and F1 Score).

In [64], a two-stage prediction algorithm was presented. In the first stage, the classifier learned the conditional dependencies among words and encoded them into a Markov Blanket Directed Acyclic Graph for the sentiment variable. In the second stage, a metaheuristic strategy was used to fine-tune the algorithm to yield a higher cross-validated accuracy. Two collections of online movie reviews from IMDB and three collections of online news were used, whereby the algorithm in the dataset was then compared with SVM, NB, and maximum entropy (ME). It was illustrated that this method, in comparison to other methods, was able to identify a parsimonious set of predictive features and obtained better prediction results on sentiment orientations. Results from the experiments suggested that sentiments were generally captured by conditional dependencies among words, keywords, or high-frequency words.

In another context, [65] implemented genetic-based machine learning (GBML) for the purpose of subjectivity detection. In their study, GBML was tested with both English and Bengali news, movie reviews, and blog domains. Results displayed the precision values of 90.22% and 93.00%, respectively, for English news and movie review corpora. Meanwhile, the precision values for Bengali news and blog corpora were 87.65% and 90.6%, respectively. These experiments have proven that GBML can automatically identify the best feature set based on the principle of natural selection and survival of the fittest. In a comparative study by [66], information gain and genetic algorithm were proposed as feature reduction analysis. In their experiment, multidomain and movie review datasets were used for opinion mining. Five classification algorithms, which consisted of naive bayes (NB), logistic regression (LR), support vector machine (SVM), and two ensemble methods (bagged SVM (BSVM) and bayesian NB (BNB)) were used to compare the performance with the proposed hybrid method. Additionally, McNemar’s test was used to compare the level of significance of the classifiers. The results showed that the hybrid method had outperformed other approaches.

The authors in [67] were engaged in a comparative study on different FS techniques in SA. They found that metaheuristic techniques have been widely used for selecting features in text classification problems. Based on their review, metaheuristic techniques were able to obtain optimum feature subsets and increase the performance of sentiment classification. Therefore, they believed that metaheuristic can be potentially used as an FS technique in SA. Additionally, [68] proposed a two-step technique for text classification sentiment. During the first step of this technique, co-clustering was used on words and documents to reduce the dimensional space of SWs. During the second step, genetic algorithm was used in text classification to calculate the weights of the SWs in the text. Reviews of movies, books, and cameras were used from the ROMIP_2011 seminar as datasets. Thus, the findings from these two studies serve as proofs that the current proposed method is better than other classifiers, such SVM and lexical.

In [69], a novel multi-swarm particle swarm optimization (MPSO) algorithm was proposed to be used as an FS technique in selecting emotional features in course reviews. The researchers used a sentiment recognition concept to understand the emotions and feelings of learners. The effectiveness of the MPSO algorithm was evaluated using baseline algorithms, such as IG, mutual information (MI), Chi-square statistic, GA, and single swarm-particle swarm optimization (SSPO). From the experimental results, the MPSO algorithm was found to be effective in reducing the redundancy of text features and in recognising discriminative features. The MPSO algorithm obtained over 88% of micro F-measure and was able to reduce the dimension of initial feature space from 10,000 to 3,000 dimensions.

OPEN IN VIEWER

Table 6

FS in SA using the metaheuristic approach

Author	Metaheuristic approach	Datasets	Accuracy	Precision (P), Recall (R), F1 Score
[10]	Entropy weight genetic algorithm	Movie review, U.S. forum, and Middle Eastern forum.	Movie review (91.70%) U.S. Forum (92.80%) Middle Eastern Forum (93.60%)	–
[56]	Genetic algorithm	Product review and People’s Daily of 1998.	95.1%	–
[65]	Genetic-based machine learning	English news, movie review, Bengali news, and Bengali blog domains.	–	English news, P (90.22%), R (96.01%); Movie review P (93%), R (98.55%) Bengali news, P (87.65%), R (89.06%), Bengali blogs P (90.6%), R (92.4%)
[64]	Markov blanket classifier tabu search	Two collections of online movie review from IMDB, and three collections of online news provided by Infonic Ltd.	87.52% (5-fold cross validation) 92.70% (unigrams, bigrams, semantic features)	–
[66]	Information gain and genetic algorithm	Multi domain product and movie review.	SVM (classifier) – (77%)	–
[68]	Word and document co-clustering genetic algorithm	Russian information retrieval evaluation seminar (ROMIP).	–	Movies, P (67.78%), R (65.02%), F1 Score (66.10%), Books, P (67.26%), R (65.54%), F1 Score (66.17%), Cameras, P (70.56%), R (67.91%), F1 Score (68.61%)
[69]	Multi-swarm particle swarm optimization	Massive open online course.

3.5 Text FS based on traditional methods using metaheuristic approach

As previously mentioned, FS is an NP-hard problem, so it requires an efficient algorithm to solve it, such as a metaheuristic algorithm [43, 70, 71, 72]. Researches on SA require a realistic application for effective and accurate analysis as SA is a field that analyses various opinions or feelings about a variety of topics, and these can be derived from different types of information sources. Therefore, an optimum solution would be impossible to find, unless the search process was conducted thoroughly in the solution space. The metaheuristic approach could be used to find an excellent solution without having to explore the whole solution space. As in any approach, the quality of the solution depends on the methods used. Metaheuristic-based methods have already been shown to be able to solve optimization problems. However, in reality, an application would only need to find a good solution within an appropriate time frame rather than finding the optimum solution.

According to [8, 37], sentiment classification is also a text classification problem. Traditional text classification involves classifying documents into various topics, such as politics, science, and sports. It is done according to topics related to features, with words related to important topics. In contrast, classification in SA is conducted based on SWs in the document or sentences. Examples of SWs are “good”, “bad”, “excellent”, and “amazing”. These SWs denote the user’s opinion about certain objects, such as products and services, or other matters and show whether the user has made a positive, or a negative comment. In this review, the use of FS techniques based on metaheuristics in previous SA studies was compared with the use of metaheuristics in normal text classification studies. This comparison was made because both SA and normal text classification deal with the same data format, namely, text data. Thus, the advantages and limitations of each metaheuristic technique were identified in this review, for resolving the FS problem.

The metaheuristic approach is a high-level strategy and an iterative generation process, which guides the exploration of the search space by using different techniques, such as ACO, GA, and PSO [73, 74]. Table 7 provides a summary of several studies that applied metaheuristic-based FS methods to classify traditional text and sentiments. The advantages and disadvantages of these methods are summarised in Table 8.

Metaheuristic techniques, such as ACO and GA have been used as the FS techniques in traditional text classification process [76, 77, 88]. Particle swarm optimization has only been used for traditional text classification in the Chinese [84] and Arabic [85] languages. Research by [77] found that only ACO was capable of obtaining the optimum feature subset. The advantages of ACO are as follows [77]:

1.

Speed of convergence,

However, the ACO’s processing time could be affected by the dimension problem (total number of features) and data size [77]. This problem can be resolved by combining ACO and RST [89, 90]. RST is a technique that can reduce the size of the feature by eliminating all overlapping features [58]. In another SA study by [37], in which the RST was combined with IG, one disadvantage with RST is that it struggled to obtain the optimum feature subset. Therefore, it was suggested that RST should be combined with a metaheuristic algorithm to obtain the optimum feature reduction [59, 60]. As such, the metaheuristic algorithm is seen as a potential FS technique in SA to obtain an optimum feature set. Tables 9 and 10 show the results of the various FS methods described in this review. The summary is based on different datasets and the different performance criteria that were used in the previous studies.

Based on the discussions in Section 3.4 and the comparisons listed in Table 6, GA has been widely used as a FS technique in SA. Conversely, the study by [77] and the comparative analysis in Table 8 showed that ACO has more potential than GA in producing optimal subset of features and can improve the performance of sentiment classification.

Based on this review, SA technology has become a necessity because it provides many benefits to the public. SA has also become an interesting and challenging research field in this era. However, SA technology has some challenges to overcome due to the diverse information that exists on websites. Some of the factors that could be considered challenges in the field of SA are described as follows [6, 40, 57].

Accuracy: Numerous SA studies were conducted in relation to products, politics, and social media. However, SA services that are commercially available for simplifying consumers’ comments are still limited because such services require accurate and correct information to be channelled to the consumers. Failure to provide accurate information could lead to impaired quality of service. Most of the tasks in SA are done manually (i.e., with the help of humans). Companies that provide commercial services related to SA are concerned about having highly accurate information. However, there are difficulties in understanding the language used in the comments. The use of complex sentences and expressions often cause SA to generate inaccurate information [40]. This problem is literally and closely associated with gaining a more detailed and clear understanding of NLP, with regards to context dependency, semantic relatedness, and ambiguity [6]. To overcome the challenges in SA, it is obvious that each step of the process should be refined and improved [40]. A clear understanding of the process may help to improve the accuracy.

Scalability: SA technology is mostly implemented in the form of web applications. The main purpose of SA technology is to develop a fast, accurate, and efficient online information search engine that can provide sentiment summaries of the information provided by the consumers regarding products and services. It should be possible to access the search engine anytime and anywhere. Nevertheless, the huge size of data on websites, the increasing volume of information, the limited speed of the Internet, and the high data dimensionality are all challenges that need to be addressed to improve this technology. These can be overcome by applying a more complex version of NLP. In [40], the authors proposed that algorithm development is done in parallel so that the segregation of duties within SA is balanced in order for text processing to become faster. They also suggested that cloud, or grid computing should be adapted for the SA web service due to their technological advantages in terms of scalability.

Standards for dataset and evaluation criteria: The use of datasets in SA should follow a certain predetermined standard to control its output quality. Some researchers have used their own datasets [12, 13, 14]. However, there are currently no proper specifications to outline the standards of datasets that should be

used in this field of study [40]. The reason for this is because different standards exist in different studies for their respective assessment measurements and datasets. This often results in complicated decision-making to what would be the best methodology to be used. Previous studies [12, 49, 91, 92] used different datasets and measurements, which makes it difficult to judge which is the best technique. Since it is noticeable that each study had used different datasets and evaluation criteria, a standard specification for datasets and evaluation measurement should be introduced to enable a fair comparison between the different methods [40].

Quality of review data: Most of the data required in SA are obtained from various sources, such as the web, social media, forums, and Twitter. Most of these sources can be accessed by anyone. The information contained in these sources may also be unreliable because consumers might deliberately give misleading opinions. Thus, the quality of the comments or opinions about products can be questionable. In addition, there may be spamming problems with the information available on the website. This can affect the output quality generated by SA. Therefore, it is necessary to identify spam in customers’ reviews [93, 94, 95]. Thus, a solution that can evaluate the quality of the reviews that appear in social media must be developed to ensure that the results generated from SA are of quality, reliable, and highly accurate.

Short form words: Comments on websites may also contain words in their short form, such as “pics” for pictures, and “res” for resolution. This can make it difficult to interpret the actual meaning the user wished to convey. It could also have an adverse effect on feature identification and sentiment type, and cause a problem in the classification process. This could be resolved with the help of a linguistic expert, who could advise on the correct and precise language used.

Object identification: Identifying the object in each review is very important in SA. Failure to recognise the intended object in a user’s comment would make the comment useless. Researchers should be able to distinguish between related and unrelated objects in a user’s comment.

Feature extraction and FS: Based on previous studies, various approaches on feature extraction have been introduced in SA. Researchers need to understand the difference between feature extraction in SA and FS in machine learning. Lately, more researches have combined the process of feature extraction and FS to obtain good subset features and to improve the performance of sentiment classification.

Synonym words: Researchers should also consider the functions of synonym words in SA. In [96], it was argued that synonym words need to be identified and grouped together. Additionally, [97] extensively discussed the function of synonym words. They identified that a lot of distinctive features contain synonyms. Consequently, they concluded that the issue of synonym words in SA is a complex one. Thus, it requires a more detailed research. The author in [8] also argued that the process of grouping synonym words in SA is challenging. Meanwhile, [15] claimed that many synonyms are domain dependent. Different synonyms are used in different senses, and thus, not all synonym words can be generalised since this practice will promote more errors [15]. For example, for the domain “movie”, the words “movie” and “picture” are synonymous. However, for the domain “camera”, the word “picture” is synonymous to the word “photo”. Additionally, the word “movie” is more likely to be synonymous to the word “video”. Most feature expressions are multi-word phrases, and thus, they cannot be easily selected from dictionaries [8]. The author also argued that while most aspects of expressions are describing one similar feature, the case is not applicable in the same domain. For example, “expensive” and “cheap” can both indicate the feature “price”, but they are not synonymous (the two are antonyms).

The previously discussed challenges still prevail in the SA h field. However, this study has critically reviewed and discussed findings from previous studies to identify the advantages and limitations of the various FS techniques applied in SA. The advantages and limitations of FS techniques for normal text classification that apply metaheuristic approaches were also reviewed. The potential of using this type of approach as a FS technique in SA was then considered, and the limitations that could be overcome by using a metaheuristic approach were identified.

Classification of the cross-domain sentiment: The problem of cross-domain is equally challenging and need to be taken into consideration in terms of the sentiment classification process. The same words may have different sentiment polarities in different domains. For example, the word “hot” in “The room is very hot” relays a negative sentiment in the domain “house”. However, in the sentence “The shower had great hot water,” “hot” has a positive sentiment in the domain “hotel”. The problem with cross-domain can affect the performance of sentiment classification if it is not rectified. According to [98], the main four problems with cross-domain are:

1.

Sparsity – occurs when words or phrases in the target domain do not exist in the source domain.

In [98], a combination of the deep learning method and word embedding was suggested to overcome the problems with cross-domain. In their study, [99] were able to develop a sentiment-related index that could determine the association between different lexical elements in specific domains. Then, the SentiRelated algorithm was developed based on this index. This algorithm functions by adding features from the target domain to feature vectors that were extracted from the source domain. This algorithm was capable of reducing the difference between the target domain and the source domain. Nonetheless, numerous solutions were proposed by previous studies to solve the problems with cross-domain. Several factors need to be taken into consideration, such as the different languages used, the cultural factor, linguistic variations, and the different contexts and noises in the data.

4.1 Definition of a SW

The most important elements to consider in SA are “sentiment words”, which can act as indicators of sentiment. SWs are also known as “opinion words”. These words can be used to express positive or negative sentiments. For example, words that convey a positive sentiment may include “good”, “great”, and “excellent”, while negative SWs may include “bad”, “annoying”, and “angry”. A SW can also be represented by phrases and idioms, e.g., “It costs me an arm and a leg”. The identification of SWs and phrases is crucial for the success of SA. A list of such words and phrases is called a “sentiment lexicon” or “opinion lexicon” [8].

4.2 The importance of SW in SA

SW is an integral element in sentiment classification because it enables words, sentences, or documents to be categorised into positive or negative sentiments. In [18], it was stated that on a documental level, the entire content of a document is identified with either a positive or a negative sentiment. Meanwhile, on a sentential level, the first step in classifying sentences is to classify them into either a subjective or an objective sentence. An objective sentence expresses factual information, while a subjective sentence expresses subjective views and opinions [8]. To note, both levels of analysis (documental and sentential) do not indicate users’ preferences. The author further argued that a finer-grained analysis can only be achieved through an analysis on a feature level. In a feature level analysis, object identification, identification of features in an object, and identification of expression sentiments from users on features are crucially important. Additionally, a SW is important in determining the expression sentiments from users regarding their level of satisfactions.

SWs can also help users assess an object, or a product for sale per day by skimming through the results of the sentiment classification analysis from social media. Nonetheless, the amount of information in social media is overwhelming, which makes it slow and difficult for users to assess a product. Therefore, the SA technology can assist users to automatically process SWs contained in the customer review datasets. Among the steps in SA are text preprocessing, FS, feature and SW matching process, and sentiment classification. Lastly, the outputs from the sentiment classification process could give an overall view on product features; whether it falls under the positive or negative category.

4.3 Methods to identify the relationship between feature and SW

The three main characteristics that need to be identified from consumers’ comments are entities or objects, characteristics or feature of an object, and SWs that are associated with each object feature. The process of identifying the actual feature in a document or sentence is very important because it plays a major role in determining the actual objects being reviewed or commented on by the consumers. Once the feature is identified in a sentence, the next step is to identify the SW that is associated with the feature of the sentence. However, if the identification of the actual feature and SW fails, this could result in an inaccurate SA output. Inaccurate analysis output could have a poor impact, and the results of the analysis may not help consumers to obtain the actual information regarding matters that concern them, such as products, services, and politics. The relationship between feature and SWs should be accurate so that the SA can generate accurate results. Nonetheless, problems may arise if there is more than one feature and SW in the same sentence. Appropriate methodologies are required to generate a corresponding relationship between feature and SW.

According to [8, 96, 105], the task of identifying features, the SWs they express, and the relationships between features and sentiment expressions from the sentence are very challenging. These challenges must be taken into account when developing a system related to SA, which are discussed in the following subsections.