Abstract
Words are one of the most essential elements of expressing sentiments in context although they are not the only ones. Also, syntactic relationships between words, morphology, punctuation, and linguistic phenomena are influential. Merely considering the concept of words as isolated phenomena causes a lot of mistakes in sentiment analysis systems. So far, a large amount of research has been conducted on generating sentiment dictionaries containing only sentiment words. A number of these dictionaries have addressed the role of combinations of sentiment words, negators, and intensifiers, while almost none of them considered the heterogeneous effect of the occurrence of multiple linguistic phenomena in sentiment compounds. Regarding the weaknesses of the existing sentiment dictionaries, in addressing the heterogeneous effect of the occurrence of multiple intensifiers, this research presents a sentiment dictionary based on the analysis of sentiment compounds including sentiment words, negators, and intensifiers by considering the multiple intensifiers relative to the sentiment word and assigning a location-based coefficient to the intensifier, which increases the covered sentiment phrase in the dictionary, and enhanced efficiency of proposed dictionary-based sentiment analysis methods up to 7% compared to the latest methods.
Introduction
The history of modern sentiment analysis dates back to the mid-2000, with the increase of online resources and social media [57]. The importance of sentiment analysis has become more apparent by the introduction of web 2.0 enabling users to express their views on a variety of topics through methods such as consumer forums, social media, and e-governance.
Regarding the enormous amount of information available in today’s world, which has made human and manual analysis almost impossible, in addition to the effect of human opinions in many decisions [70], the importance of sentiment analysis processes by machine-based approaches is increasing.
The various applications of intelligent sentiment recognition and analysis are observable in a wide variety of areas such as cinema and film criticism [19, 73], assessment teaching quality [6, 94], business and product feedback [7, 59], e-learning [22, 80], and hospitality management and tourism [2, 50]. In the commercial domain, identifying the polarity and orientation of the ideas offered by users helps manufacturers, how to enhance quality of their products or services [61]. Further, it provides highly smart statistical analysis and online advice to both client and business groups [100]. In the political domain, sentiment analysis is used to analyze people’s orientation and bias toward political campaigning [9, 14]. For e-learning providers, sentiment analysis helps to identify the problems that may occur to learners during the course [42].
In general, sentiment recognition in a text is a process related to the subjective sentences and domain [70, 91]. This dependency has been shown in the topic, time, and writing style. Sentiment analysis is mainly done at the three levels of document [77], sentence [28], and aspect level [34]. In the document-level sentiment analysis, the overall orientation of the document is considered while the internal aspects are disregarded. In the sentence level, only the polarity of the subjective sentences is considered and objective sentences are completely discarded. Finally, in the analysis of sentiments at the aspect level the sentiment corresponding to each aspect is extracted. The final feature in the aspect level consists of a set of pairs including aspect vs sentiment [90, 102].
In general, the three methods of dictionary-based [60, 67], machine learning [74, 99], and the combination of the two previous ones [3, 89] are used for the automatic extraction of sentiments.
Machine learning-based methods using statistical tools, artificial intelligence, and processing tagged databases in the field of natural language extract targeted features, and then classify the newly imported sentences or texts through a training phase based on the tagged views of a specific domain. Conventional approaches to machine learning include Naive Bayes [78, 98], maximum entropy [33, 90], and support vector machine [30, 93].
Sentiment words play a central role in all sentiment analysis methods. Thus, the construction of a high-quality sentiment dictionary and appropriate scoring has attracted the attention of lots of researchers [29, 86]. Even in machine learning-based methods, a sentiment dictionary can be used to extract sentiment-related features that are more purposeful and efficient in identifying text sentiments [81]. Methods applying manual sentiment dictionaries usually require less training and resources and no annotated corpus [1, 47]. These dictionaries have some other advantages such as the possibility of transforming to different domains and languages [41, 58], better applying linguistic phenomena such as intensifier [75], adding language-dependent features [4] and human beings can understand and modify these dictionaries easier [41, 96].
Thus, sentiment dictionaries including a list of words and phrases with their scores are the most important component of the sentiment analysis methods [1, 53]. Conventional methods of sentiment recognition using the sentiment compounds and its scores of these dictionaries, efficiently extract statistical features and assign a score to each text [51]. How a dictionary is produced is highly affect the overall accuracy of the polarity detection system [84].
To the best of our knowledge, the attention to the linguistic phenomena has either been too little or not at all in the former dictionaries. In most dictionaries, for example, the effect of the intensifier, such as “very” in combination with sentiment words is not considered or considered to have the effect of adding or subtracting a constant number to sentiment word score or multiplying basic word score to the calculated weight. The major disadvantages of such approaches are the lack of attention to repetition, location of linguistic phenomena, and differences in sentimental words combined with linguistic phenomena. Hence, in polarity calculations, paying attention to linguistic phenomena and their location can greatly improve the efficiency of the method. Among the existing methods, the Senti-N-Gram Dictionary [24] compared to the previous dictionaries considered the heterogeneous effect and the location of multi-linguistic phenomena. However, few covered sentiment expressions and no attention to all possible sentiment combinations are considered as the weaknesses of this dictionary. In addition, because, different negators are considered as invariant phrases in this dictionary, many of Bigram and Trigram combinations, not covered in this dictionary.
Based on previous research’s weakness, the main novelty of this study is to design a public semi-supervised sentiment dictionary using a wide set of sentiment reviews. A hierarchical fuzzy scoring system and addressing the heterogeneous effect of multiple intensifiers in sentiment compound are other novelties of the proposed method. Based on our best knowledge, none of the existing public sentiment dictionaries has these advantages.
In summary, the innovations of this research are first covering all the bigram and trigram combinations including linguistic phenomena combined with a sentiment word by considering the location effect of the intensifier, and second designing a semi-supervised hybrid method based on statistical rules and fuzzy logic for assigning coefficients to each sentiment compound.
The remainder of the present paper is as follows. Section 2 gives an overview of the previous studies and available dictionaries. Section 3 describes the used dictionaries and databases in detail followed by the proposed method. Section 4 compared the proposed dictionary with the existing methods and evaluate the performance of the developed dictionary to the latest and best general dictionaries. Finally, the conclusion is elaborated in Section 5.
Related work
In general, a sentiment dictionary is made of a set of single words [48], and each entity within this dictionary is associated with a sentiment score. In some dictionaries [37, 72], this score only indicates the positive, negative, or neutral orientations. Some others [95] contain a limited number of graded sets such as strongly positive, mildly positive, neutral, mildly negative, and strongly negative, while some other dictionaries [8, 82] use a finite numerical range to express the intensity of sentiment. These methods can be supplemented with other information such as a list of emoticons and semantic rules like how to deal with the negators [65, 84], considering this information when working with short informal texts is important for a higher efficiency [67]. In dictionary-based methods, the in-document sentiment words are searched for classifying based on the existing dictionary [52] and the overall sentiment of the document is determined by the dominant polarity as positive or negative between the indexes [12, 81]. Generating sentiment dictionaries [46] can be performed via the four methods: Manual methods including Crowdsourcing [20, 38] and Gamification [36, 88]. Bootstrapping using a set of seed words and syntactic or dependent relationships [32, 68]. Adapting a dictionary from another field using the learning transfer [26, 97]. Identifying sentiment words by machine learning-based techniques [5, 97].
This section is divided into the three areas of linguistic phenomena, sentimental dictionaries, and fuzzy logic as the basis of the proposed approach.
Linguistic phenomena
Although much researches have been focused on the development of supervised and unsupervised systems in sentiment analysis, little attention has been paid to linguistic phenomena in this field [101]. Linguistic phenomena such as the negators and intensifiers are one of the most effective factors in sentiment analysis. Not paying enough attention to the negator or intensifier effect on sentiment words and compounds reduces the efficiency of sentiment analysis systems.
Negators
Negators are considered as one of the most important linguistic phenomena affecting the polarity of sentiment words. Large research [10, 40] showed that paying attention to linguistic structures such as negators increase the efficiency of sentiment analysis systems.
The effect of negators is considered in four ways
The present paper applied the inverse hypothesis regarding the complexities of negators and their different effect as changing polarity, emphasizing negativity or making weaker claims [13].
Intensifiers
Paying attention to intensifiers can increase the accuracy and efficiency of sentiment analysis systems [18, 63]. Generally, the intensifiers are divided into amplifiers and downtoners. Intensifiers such as “very” and “entirely” which increase the sentiment of a word are called amplifiers, reinforcers, or overstatements, while intensifiers such as “quite” and “slightly” which decrease the sentiment of a word are known as downtoners or attenuators, [35, 83]. Although none of these words imply polarities on their own, they alter the polarity degree of sentiment words [27]. Calculating the effect of intensifiers is more complicated than simple addition and subtraction [17]. Some methods add or subtract a constant value to exert an intensifying effect on a sentiment word [54, 64]. In [41] the intensifiers were divided into downtoners, weak amplifiers, and strong amplifiers, those of which reducing the sentiment value of the words by 50% are called the downtoners, and those increasing the sentiment value of the words by 50% and 100% are known as weak and strong amplifiers, respectively. In some other methods, each intensifier had a constant coefficient [79, 96] and its effect is done by multiplying the word score in intensifier coefficient. Also, [24] calculated a different coefficient for each intensifier when placed before each sentiment word. Regarding the increased accuracy of all sentiment analysis techniques when using intensifiers [43], more attention should be paid to generate public sentiment dictionaries supporting intensifiers.
Considering linguistic phenomena, sentiment dictionaries can be classified into the two categories of those regarding linguistic phenomena and those disregarding them. The basic sentiment dictionaries contain only sentiment words and with developing and updating dictionaries, some linguistic rules and phenomena have been added to these dictionaries. Some public dictionaries including linguistic phenomena are reviewed in the next section.
Sentiment dictionaries regarding one or more linguistic phenomena
SO-CAL dictionary [84] is considered as one of the first sentiment dictionaries including linguistic phenomena. In the new version of this dictionary, the scoring method considers negators, intensifiers and irrealis moods, and lemmatized verbs and nouns, while disregarding the effect of the intensifier location. Instead of changing polarity, shifting has been used in the sentiment words combined with negators such as not, none, nobody, never, etc., reinforcing the negator expressions in the text while reducing the severity of the frequently occurring terms.
The basis of the SentiStrength 1 [87] dictionary consists of 2310 words. This dictionary employs a list of negators, intensifiers, idioms, and emoticons along with the sentiment word list but did not consider pos tagging. All of the words in this dictionary were manually scored and then in the training phase updated.
The SentiFul Dictionary [66] contains several modifiers, intensifiers/inverters, and modal operators. This dictionary was generated automatically using synonymy, antonymy, hyponymy, derivation, and a combination of known words. Further, it was developed using morphology and word manipulations with different prefixes and suffixes.
Lexical TF-IDF Dictionary [23], primarily produced sentiment n-grams with the criteria that one or two intensifiers or negators appear before a sentiment word. The score of sentiment words, intensifiers, and negators are extracted from the VADER 2 [38] and SO-CAL [84] dictionaries, multiplied by their TF-IDF rating to determine a specific weight.
The VADER [38] and Senti-N-Gram [24] dictionaries are two other important public dictionaries that will be explained in detail in the following sections.
Fuzzy logic
In the classic dictionary generating methods, the scores of the sentimental compounds are calculated using classic mathematical operators. In these methods, some samples of a large database are selected to analyze, and the variance of reviewer value-ratings is ignored. In the proposed method instead of classic operators, fuzzy scores are computed to improve the natural language processing systems in sentiment recognition.
In the proposed method the Gaussian distribution is used, which is the closest fuzzy function to complex natural systems [39]. The main parameters of this fuzzy function include mean and variance, which can be identified and assigned to the sentiment compounds.
The proposed method
In this section, two dictionaries and three databases were used in this research are described in detail and the challenges of the former dictionaries are mentioned. In continuation of the discussion, the proposed method in two relatively different parts, has been explained. The flowchart of the generating dictionary part, of the proposed method, is given in Fig. 1.
Proposed method flowchart.
The VADER Dictionary: This dictionary [38] contains 7516 sentiment words rated between +4 and –4. The dictionary was generated using the words of the LIWC [71], ANEW [15], and GI [35] dictionaries, as well as a complete list of the Western-style emoticons 3 , acronyms 4 , initialisms, and sentiment idioms. In the first step of the implementation of this dictionary, the extracted words were scored by the Crowdsourcing method via AMT 5 by ten independent individuals on a scale of+4 to –4. In addition to sentiment words, this dictionary includes several intensifiers and provides the five syntax rules of the effect of punctuation, capitalization, intensifiers, the word “but”, and the inverse effect of the negator in trigrams. The weaknesses of this dictionary are, few numbers (only 65) of amplifiers and attenuators and assigning a similar coefficient to all intensifiers. This dictionary discards the heterogeneous effect of the occurrence of multiple intensifiers in sentiment phrase. (for example, see “very good” and “deeply good” scores in Table 1). The VADER sentiment words and its score were used directly in the proposed dictionary.
Equal intensifier effect on sentiment words in VADER dictionary
Equal intensifier effect on sentiment words in VADER dictionary
Senti-N-Gram Dictionary: This dictioanry is the latest public semi manual sentiment dictionary and showed the best results in polarity detection(based on our best knowledge). The basis for this dictionary [24] is the VADER dictionary. This dictionary contains 7516 unigrams, 2415 bigrams, and 305 trigrams. Senti-N-Gram Dictionary is a semi-autonomous dictionary combining sentiment words score in VADER and intensifiers and negators coefficients extracted from a set of reviewer value-ratings belong to Amazon database. In order to generate this dictionary, all the unigrams, bigrams, and trigrams reviewer value-ratings in the Amazon database comments, which are a combination of sentiment words with the intensifiers and negators were extracted. After calculating the percentage of decrease or increase in bigrams scores concerning unigram scores and trigrams compared to bigrams, this percentage multiplied with the VADER score to gain the score of the new bigrams and trigrams. Using this scheme, this dictionary considers the heterogeneous effect of multiple intensifiers in sentiment phrase.
The lack of coverage all sentiment compounds is the major disadvantage of this dictionary. Also, when the sentiment compound in the text has not been scored in the dictionary, only the weight of the sentiment word is considered and the effect of the negator or intensifier is lost.
The Amazon website database 6 is used to calculate the score of sentiment compounds in the proposed method. There are about 18 million comments and 80 million contents in 24 different areas in this database. This database is one of the largest and most comprehensive available databases and has been the basis for the proposed semi-automatic dictionary in this research.
Brooke in his master thesis extracted the list of general intensifiers [16] that used as the second database in this research. The coefficients provided by [16] have not been used and the effect of intensifiers in the text is conducted in the proposed method again. The negators are selected based on University of Cambridge standard 7 along with those in the SO-CAL and VADER sentiment dictionaries.
Two manually tagged databases have been used to compare the proposed dictionary with the state-of-the-art dictionaries. First the Taboada database including a collection of eight different domains with 50 positive and negative comments for each domain. This database was collected and tagged by Stanford University in 2004 [85]. The second database 8 is the Pang & Lee database of movie reviews [69], including 1000 positive and 1000 negative reviews on movies.
Fuzzy pattern extraction of sentiment compounds
In the first step of the dictionary design, all bigram and trigram compounds using the VADER dictionary and the intensifiers in [16] are generated (A1). The sentiment compounds including unigram, bigram and trigram and its reviewer value-rating were extracted from the Amazon database (A2). The intersect of A1 list and A2 combinations were stored as a new list, along with the reviewer value-rating (A3).
Next, a fuzzy pattern of value-ratings was obtained for each unigram, bigram, and trigram compound (A4). A large number of articles in this field have used statistical operators (such as mean) to allocate coefficients or scores effectively to sentiment compounds, while none of them, assign a fuzzy pattern to sentiment compounds.
The initial coefficients, obtaines based on A4 fuzzy model have been used to generate random samples for balancing the number of unigrams, bigram and trigram including a specified sentiment word. For example, when the existing unigrams in the Amazon database are 1000 and a bigram including the same unigram are 50 via the extracted fuzzy pattern for this bigram in the previous step, 950 new scores (value ratings) for this bigram are produced (Eqs. (1) and (2)) (see Table 2 for more explanation on the parameters). (A4)
Notations for the algorithms
Notations for the algorithms
After equating the number of samples by the reviewer value-rating fuzzy pattern described in the previous section, the bigram score is calculated via simple averaging for the new samples, then the computed bigram score is divided by the unigram population mean to calculate the initial intensifier coefficient of the first position intensifier (Equation (3)) (A5). It is similarly done using the mean of trigrams and bigrams to calculate the second position intensifier coefficient (Equation (4)) (A6).
Then, in order to get the result closer to human data, the coefficient of the intensifier calculated for kth sentiment word in Equation (3) is multiplied by the score of same word from the VADER dictionary (A7). In other words, the bigrams score is updated using the scores in the VADER dictionary and calculated coefficients in the previous step (Equation (5)).
The primary score of trigrams is recalculated using the obtained score in Equation (5) and the obtained coefficient in Equation (4) (Equation (6)) (A8).
Increment–decrement percentage value for n-grams, Score warping, verification and refinement of the score of n-gram was done similar to [24] to enhance accuracy.
The corrected scores were used to recalculate intensifier coefficients. These coefficients are averaged for different sentiment words per intensifier to get the final intensifier coefficient of the first position based on Equation (7) (A9).
After determining the final coefficient of the first position intensifier, the bigrams score is updated again (A10, Equation (8)), the new scores are used to derive the second intensifier coefficients belong to trigram via Eqs. (9) and (10) (A11-A12).
In the proposed method, the final coefficients of the intensifier in the first and second positions are a combination of fuzzy logic, statistical methods, reviewer value-rating of Amazon database and VADER manual dictionary. The combination of the above steps minimizes errors in calculating the intensifier coefficient.
Based on Fig. 1, the generated dictionary consists of 7516 sentiment words (based on VADER Dictionary), 215 intensifiers and 53 negators. The score of sentiment words is chosen based on VADER. The two coefficients assigned to each intensifier based on its position. The 53 negators has an inverse coefficient –1. As a glance, the score of each sentiment can be calculated, based on Table 1 formula, using the following equations, as:
Base on Eqs. (12-13) and the scores of “enjoy” and “bad” in VADER dictionary that are 2.3, –2.5 respectively and using Table 3 values we can calculate the score of “very big bad” and “very deeply enjoyed” as:
The calculated coefficients for some intensifiers considering their position on sentimental phrase
very big bad = 1.45*2.08*(–2.5)=–7.54
very deeply enjoyed = 1.45*2.5*2.3 = 8.34
In the result section, two polarity document classification algorithms are implemented. In the first polarity detection algorithm [45], score of sentiment phrases in each sentence based on each dictionary are extracted and their mean is calculated. Then, the average of the various sentences is added together and a general assessment of the polarity of the document is made. In the second algorithm proposed in [24], similar to the first algorithm, score of sentiment phrases based on each dictionary are extracted, added together and the polarity of each sentence is calculated. Finally, the number of positive and negative sentences are counted and the higher number determines the document polarity. The document polarity is considered to be negative when the number is equal.
The result section is divided into two parts. In the first part, the results of the fuzzy probability density function and fuzzy statistical tests in calculating the initial scores are presented for several n-gram samples. In the second part, based on different validity criteria such as Recall, Precision, and F-measure, the Accuracy of the proposed system against the state-of-the-art methods in two standard databases in different domains is investigated.
Fuzzy pattern
In the first step, the fuzzy bigram and trigram model should be obtained based on the statistical model and the number of samples in unigram, bigram, and trigram, and then the bigram and trigram parameters are calculated again by generating sufficient samples and homogenizing (Equations (1) and (2)). The calculated fuzzy pattern for “good”, “very good”, and “very very good” is shown in Fig. 2.
The Fuzzy diagram for the three terms “Good”, “Very Good” and “Very Very good”.
The pattern output is restricted to the interval of 1 to 5 considering the actual scoring of sentences in the Amazon database. The standard deviation of bigram decreased in comparison with unigram due to the proximity of scores for the sentences in bigram. In addition, due to the proximity of scores for the sentences in trigram, its standard deviation decreased compared with bigram (Fig. 2).
Table 4 shows the initial statistical mean and the corresponding fuzzy mean of the distribution after homogenizing a sample to indicate the effect of homogenizing in the obtained averages. Based on Table 4, the mean varied about 2 to 5% in trigram and bigram versus simple mean.
Statistical and fuzzy results of a sample of Unigram, Bigram, Trigram
This step is equivalent to Equations (1) and (2). The columns 2 and 3 of Table 4 are the mean and standard deviation of the samples, namely
The proposed dictionary is compared with two state of the art dictionaries in this field, namely VADER [38] and Senti-N-Gram [24]. A comparison was made between the two databases of Taboada [85] and Pang & Lee [69] described in Section 3 to demonstrate the efficiency of the proposed dictionary. The common criteria of True Positives (TP) (the positive documents separated truly), True Negatives (TN) (the negative documents separated truly), False Positives (FP) (the documents with a negative polarity which was mistakenly identified as positive), and False Negatives (FN) (the documents with a positive polarity which was mistakenly identified as negative) were applied for the comparison. Using these criteria, the four parameters of Precision, Recall, F-measure, and Accuracy were obtained. The relationships for these four criteria are given in Equations (14–17).
Precision: The precision of a method is the ratio of correctly classified positive documents to the total classified positive documents.
Recall: Recall is the ratio of correctly classified positive documents to all positive documents. This criterion usually shows higher value than precision because it evaluates the method in positive documents in which most methods perform more efficiently.
F-measure: It is the harmonic mean of precision and recall.
Accuracy: It indicates the performance of the system in all documents.
For the evaluation to be done in similar conditions, the Senti-N-Gram dictionary and proposed method were made using similar samples of amazon database so the conditions for comparison were completely the same.
Table 5 presents the accuracy of the three dictionaries in both databases and indicates that the proposed method has at least 2.45% better accuracy in both databases than all previous methods.
Comparison of Accuracy Parameter Based on the Classification of Documents at Sentence Level Using VADER, Senti-N-Gram Dictionaries, and Proposed method
Table 6 shows accuracy increase using first polarity document classification algorithm [45], in all overall results belong to domain-specific comparison. This table shows the proposed method in all cases is more efficient than the previous methods, especially in the negative documents.
Comparison of Accuracy Parameter Based on Classification Method [45] Using VADER, Senti-N-Gram and Suggested dictionary in Different Areas
As shown in Table 6, the greatest improvement in detecting negative expressions is observed in the domains HOTELS, BOOKS and PHONES with values of 80, 55 and 40%, respectively. The highest increase is related to the PHONES domain, with 12% overall improvement.
Table 7 compares the precision, recall, and F-measure in three dictionaries. Only in BOOKS domain, the recall of the proposed method is less than other methods but the higher precision value in the proposed method based on better detection of negative documents, overcomes this weakness. The value 1 in the recall (R) columns of three dictionaries in Table 7 shows all dictionaries have good results in positive document detection and the main challenge is in negative document classification. The main advantage of the proposed dictionary, is its correctness in negative document classification in all domains (see Table 6 column negative).
Comparison of Precision, Recall, and F-measure Parameters Based on Classification Method [45] Using VADER, Senti-N-Gram and Proposed Dictionaries in Different Areas
For showing proposed dictionary accuracy in different conditions, the simulations repeated using second polarity detection algorithm explained in the last paragraph of section 3 [45]. In this analysis, document polarity detection method is done based on counting the number of positive and negative sentences in the document and choosing higher number as selected polarity [45]. The better results of proposed method can be observed versus two Senti-N-Gram and VADER dictionaries (Tables 8 and 9).
Comparison of Accuracy Parameter Based on Classification Method [24] Using VADER, Senti-N-Gram and Suggested Dictionaries in Different Areas
Comparison of Precision, Recall and F-measure Parameters Based on Classification Method [24] Using VADER, Senti-N-Gram Dictionaries and Proposed Dictionaries in Different Areas
As shown in Table 8, the proposed dictionary increases the number of negative document detections in all eight domains. It reduces the error in positive documents in the MOVIES and MUSIC domains to zero. The highest improvement is in PHONES domain, similar to previous method, although some results are different. For example, in VADER dictionary, PHONES domain, the second classification method using proposed dictionary shows 32 increase in classification of negative documents while the first classification method only shows 20 increase. In this table, the proposed method increased HOTELS, MUSICS and BOOKS accuracy in negative documents 80, 40 and 20%, respectively.
According to the Table 9 results, the F-measure and precision results of proposed method are considered as a valuable performance increasement in all domains except COMPUTERS (for example F-measure improved 8% vs Senti-N-Gram in MUSIC domain). In overall result, on average the suggested method shows 4% and 4.6% better results than Senti-n-gram and VADER dictionary in precision criteria respectively, and 3.2% better results than both dictionaries in F-measure. In positive document classification (recall criteria) the suggested dictionary results average is similar to two previous dictionaries.
Due to the wide range of sentiment analysis applications and increasing the need for automated algorithms for document and sentiment analysis, a hybrid scheme for training semi-manual sentiment dictionary was proposed in the present paper.
The proposed method, assigned a fuzzy coefficient to each intensifier in bigrams and trigrams regarding the position of the intensifier towards the sentiment word that enhanced the efficiency of the existing dictionaries in document polarity recognition.
In the proposed scheme, the intensifier coefficients were corrected during several stages using the fuzzy pattern of review value ratings. Also, the intensifier place relative to the sentiment word and the mutual-effect between the polarity of the sentiment word vs the intensifier was considered. Therefore, two different coefficients were obtained for each intensifier according to the first or second position regarding to sentiment word. This scheme leads to a more efficient score assignment based on its fuzzy and both machine-learning and reviewer value-ratings error correction steps, a wider range of combinations, and enhance accuracy, especially in the negative documents than conventional state of the art public sentiment dictionaries. Finally, the proposed method was implemented on several standard databases and its results were compared with two state of the art dictionaries. Comparing results indicated the accuracy of the proposed method was increased about 7% compared to the latest methods in this field. As future work, the irrealis role in polarity detection can be considered and added to proposed dictionar.
