A fuzzy hybrid recommender system

4.1 Modeling user preferences

To justify and explain the use of interval type-2 fuzzy sets for user preferences, we proceed with an example taken from the case study of recommendation for book purchase. Please note that our users for this work belong to academia. They are students, researchers and professors of the university. To illustrate the concept, consider modeling of user preferences (as per their profile) for cost of an item (book). In case of cost defined using type-1 fuzzy sets, let ‘ρ’ (a crisp value) be some threshold value defined for membership of a book in a particular cost category (low [300, 500], medium [400–600], high [550 & above]). Then system will recommend those books to the user whose membership degree in a cost category is greater than the threshold value ‘ρ’. This includes all the cost categories which are lower in range as compared to the category chosen by a user. For example, if a user chooses medium category, then the membership value will be considered for all low and medium range books or if a user go for high category, then the membership value will be considered for all low, medium and high range books Also, type-1 membership function due to fuzzy c-means clustering is defined as in Equation (5), givenbelow:

U ik = ( ∑ j = 1 c ( Dist ( X k - V i ) Dist ( X k - V j ) ) 2 m - 1 ) - 1 ; 1 ≤ i ≤ c , 1 ≤ k ≤ M(5) where, m is level of fuzziness, c is the no. of clusters (for cost categories), M is the total number of preferred books, U_ik is the membership of kth book cost in the ith category, X_k is the cost of the kth book, V_i is the centroid of the ith cluster and the distance function used is the Euclidean distance [11]. Let a book “Introduction to Computer Graphics” authored by “N. Krishnamurthy” of cost Rupees 595 (which is on the higher side of medium range books) be in the affordable range for an earning person (professor, researcher etc.) but may not be so for a non-earning person (student). Now, suppose the user chooses the medium cost category books as his preference. The system estimates type-1 membership degree in low category to be 0.0835191 and in medium category to be 0.351891 (determined using fuzzy c-means clustering algorithm) which is less than threshold membership value (say ‘ρ= 0.4’) in medium cost category. So the book “Introduction to Computer Graphics” will not be recommended either to a student or to a professor/researcher (an earning person). This is because in type-1 fuzzy sets, membership value is crisp and same for all user types (whether a user is a student, professor or researcher). But understanding of low, medium and high cost may vary from one user to another. So, to resolve such uncertainty problems we can make use of type-2 fuzzy sets. Hence, the item/product attribute ‘cost’ can be modeled using type-2 fuzzy sets. Now, instead of a crisp membership value we have defined a type-2 membership interval [μ²,  μ^1/2], where μ is membership value obtained after applying fuzzy c-means clustering. This type-2 membership interval is mapped to a unique value (to determine in which category the book belongs to, with a membership value greater than threshold) by applying a type-2 membership function stated below in Equation (6). This is obtained by applying a combination of type-2 membership interval bounds and the user-profile specific scaling parameters α and β. This provides the system with flexibility of generating diverse recommendation domains as per varying user requirements and profiles. type 2 ik = α * U ik 2 + β * U ik α + β(6)

Here, U_ik is the membership of kth user in the ith cluster. As α and β are the scaling factors that vary from one user to another (depending on three different user profiles), thus modeling ‘cost’ using type-2 fuzzy sets provides more personalized recommendations to the user. Determination of values for parameters α and β is discussed in detail in the following sub-section 4.2.

Now coming back to the example; for such cases, it is probable that a book may found to be suitable for a professor and not for a student even under similar kinds of circumstances and preferences (depending upon user chosen category for cost and profile). This situation can be modeled by applying type-2 fuzzy membership function (as given in Equation (6)), as the membership can deviate by some value ±ɛ (ɛ is some real number between [0, 1]). Also, this deviation in membership value will provide flexibility and diversity to the domain of books to be recommended to the user and thus it gives recommender system more choices in selecting relevant books to be recommended to a user. So the uncertainty related to the user’s perspective of “What is expensive and what is not!” is handled in a better way in case of type-2 fuzzy sets. For the same scenario as described above under type-1 fuzzy sets, the book “Introduction to Computer Graphics” will be recommended to an earning person (as membership value is increased from 0.351 to 0.436) and not to a non-earning person (as the membership value is decreased from 0.351 to 0.28) (refer Table 1). We have summarized this generation of recommendations customized to user profile and preferences using type 2 fuzzy approach in Fig. 3. Similarly, the user’s preferences for another item attribute like: the year of book publication can also be modeled using IT2 fuzzy sets as different users may have different view towards “Oldness of a book due to its publication year!”.

OPEN IN VIEWER

Fig.3

Generating recommendations customized to user profile and preferences using type 2 fuzzy approach.

Moreover our second case study, which is based on the shopping of women apparels, also considers the item attribute price. As already explained, attribute price has an inherent uncertainty associated with it, so we have modeled price using IT2 fuzzy sets over here.

For the sake of completeness, we analyzed the proposed approach on a variety of datasets. We also studied the standard MovieLens dataset (the dataset was elicited from http://grouplens.org/datasets/movielens/) and explored that the user preferences in it can also be modeled using type-2 fuzzy sets. For this we chose to fuzzify user category (over the attribute user’s age) and movie category (over the attribute movie type). Users are categorized on the basis of age, for example: children, teenagers, youngsters, middle aged people and old aged people. Various movie categories are: action, adventure, drama, romance, comedy, horror etc. But a movie may contain various elements like action, comedy and romance each with a certain degree of percentage. So, different movie types can be combined to make a group. For example: romance, comedy, drama can be combined to make a group, similarly another group can be action, adventure, romance. These groups of movie types have each element with a degree of percentage and each user category may have different choices of these movie groups with varying degree of preference for each group element. So, there is a degree of uncertainty associated with movie choice and we can handle this uncertainty using type 2 fuzzy sets.

Therefore, we can put forward the claim that the proposed approach is generic. It can be applied to any suitable database requiring fuzzy modeling of user preferences.

The user’s perspective of features like cost, publication oldness, age, movie type etc. has an inherent uncertainty associated with it. As a result we model such features using type-2 fuzzy sets. Initially for instance, using fuzzy c-means clustering we determine the cluster(s) to which a selected book can belong in the given book dataset. A book may have multiple memberships. Each membership is denoted by U_ik (membership of ith book in kth cost category). Using type-2 fuzzy sets, each book is associated with one or more intervals like (U ik 2 , U ik 1 / 2), such that U_ik∈ (U ik 2 , U ik 1 / 2). For every U_ik and a particular user profile (professor/researcher/student) the fuzzy system assigns values to parameters α and β in [0, 1] where α and β are scaling factors and their value vary for each user category. Therefore we have: (α_p,  β_p) as scaling factors for professor; (α_r,  β_r) as scaling factors for researcher; and (α_s,  β_s) as scaling factors for student. The system determines typ2_ik for each U_ik against a particular book. In case of multiple memberships, corresponding to single user category, denoted by μ_L (representation of membership of a book in low (L) cost category), μ_M (representation of membership of a book in medium (M) cost category), and μ_H (representation of membership of a book in high (H) cost category); the book is said to belong to the category scoring the maximum typ2_ik (obtained from Equation 6). We have considered three user and three cost categories, so we have 9 rules in total. Finally, the membership value for the item is determined by combining all typ2_ik’s obtained as a result of application of multiple fuzzy inference rules based on uncertainty related to user’s view towards various item attributes. We combine these; by taking simple average of all values to defuzzify the output. The obtained value is used further to determine suitability of the item for recommendation. In case the item appears in the recommendation list, it may get a feedback. Depending on this feedback (whether like or dislike) the parameter values for α and β can be updated (or adjusted) in the fuzzy logic system. We used the fuzzy inference rules stated in Fig. 4 to determine the parameters according to the customer’s purchasing power.

OPEN IN VIEWER

Fig.4

Fuzzy Inference rules.

We would now explain how these parameters or factors (α’s and β’s) were tuned in the system. A fuzzy logic system contains many design parameters whose values can be optimized by the designer before the system is operational. There are many ways to do this, and all make use of a set of data, usually called the training set. This set consists of input–output pairs for the system. We identified a feasible set of parameters for the given inference rules. These were all α’s and β’s (see Fig. 4). Given input–output training samples, we wished to update the scaling parameters so that errors were minimized for training samples. A general method for doing this is to initialize all the parameters including the parameters in antecedent and consequent of the rules and then apply suitable optimization methods implemented for an artificial neural network [20]. When neural networks are used to model a set of existing data, so that predictions can be made on new data, the main challenge is to find the set of weight and bias values that generate the outputs that best match the existing data. Optimization methods for doing this have been extensively studied [36].

The most common technique for estimating optimal neural network weights and biases is called back-propagation [31]. Hence, the parameters α and β were also attuned by back-propagation optimization method using MATLAB Neural Network version 8.0. The method used in back propagation is gradient descent. We preferred to apply this algorithm due to its simplicity and speed. Several hundreds of such runs were executed taking variety of training samples such that all the 2×3 = 6 parameter values for all α’s and β’s are selected and executed multiple times. This helped in estimation and adjustment of the parameter values essential in testing the suitability of an item for recommendation.

The back-propagation program started by splitting the data set of 300 instances. It divided data into a training set of 240 instances (80%) and a test set of 60 instances (20%). Then the program created a neural network with four input nodes (one for each numeric input), four hidden nodes and two output nodes (one for each boundary (lower and upper) of the output intervals for α’s and β’s respectively). This neural network is based on the hyperbolic tangent (tanh) function for hidden node activation, and the softmax function for output node activation [33]. The neural network’s weights and bias values were initialized to small random values (between 0.002 and 0.0002). Then the back-propagation algorithm was used to search for weights and bias values that generated neural network outputs that most closely matched the output values in the training data. Training with back-propagation is an iterative process. The training process stops after 1,700 iterations, or when a mean squared error term drops below 0.002 and then training dataset is used to compute α’s and β’s for test data. The output values for α’s and β’s are approximated according to the study and analysis related to the ‘customer purchasing power’ done in the work by Watson [13]. The authors in their work examined the relationship between customer income, desire to purchase and aspects of economic psychology. The research was designed to investigate how people with differing levels of income and materialism vary in their propensity to spend and/or save. The results indicated that highly earning people are more likely to view themselves as spenders and have more favorable attitudes toward purchasing. Therefore, we assumed the output values for α’s and β’s to be: α_P = 1, β_P = 2; α_R = 1, β_R = 1; α_S = 2 and β_S 1. These values were then used in the execution of test runs while training a back propagation neural network to compute intervals for the estimation of α’s and β’s for various user profiles.

On the completion of the training procedure, the program demonstrated the values of the weights and biases (intermediate calculations) in the neural network. These were determined by the training procedure. The process finishes by computing appropriate parameter values for α’s and β’s. The parameter settings obtained by training neural net are as follows: α_P∈ (0.8,  1.2) and β_P∈ (1.6,  2.1); α _R , β_R ∈ (0.9,  1.1); α_S ∈ (1.8,  2.3) and β_S (0.7,  1.0). The correctness of the obtained parameter values for α’s and β’s was investigated on the basis of the surveys and analysis in [13, 17]. It was observed experimentally, that the performance of the proposed RS approach changed with respect to different parameter settings for α and β for each user category. Also, the performance of the proposed recommendation approach improved noticeably when the gap between parameter values for various user categories was increased; as compared to the instances when this gap was allowed to decrease.

Two-part hybrid recommender systems are quite successful, but under certain domain and data characteristics different hybrids might achieve unlike results. For our intended recommendation system, we decided to use the feature combination strategy [30a] making use of a virtual contributing recommender that provides supplementary features for the actual recommender. The feature combination hybrid is not a hybrid in the sense, because there is only one recommendation component. The idea is to inject features of one source (such as collaborative recommendation) into an algorithm designed to process data with a different source (such as content-based recommendation). In our case, user rating data (to determine preferred items amongst masses) were combined with the item features (based on user’s likes, dislikes and past interactions). The system has only one primary content based recommender component, but the content is drawn from a knowledge source associated with collaborative recommendation. To be more specific, collaborative filtering features are utilized to strengthen content based recommendation.

Though many recommendation systems combine collaborative and content-based filtering techniques to improve rating prediction, however, our approach is unique. In this work, we unify user-item ratings and content features in a single model (Feature Combination); and then extend the basic approach, to allow data exploitation by fuzzy c-means clustering and interval type-2 fuzzy techniques to handle uncertainties related to diverse user preferences. The major advantage of using this approach is its simplicity and computational ease. There is one wholesome recommender with independent yet interconnected modules. In a multi-agent set up like ours, we used different agents to handle different modules and the agents connected with each other via communication messages.

5.1.1 Fuzzy feature combination hybridization (FFCH) method

Our proposed recommendation algorithm offers an extension to a well-known existing “feature combination hybridization method” in which output from one filtering technique is used as an input feature to another [30]. The traditional feature combination method is a two stage process. In the first stage, new features (preferred items) are generated by applying collaborative filtering method. In second stage, content-based filtering method is applied on the obtained features (preferred items) in which user’s preferences are considered to recommend the most relevant items to the user. Proposed recommender system uses “fuzzy feature combination hybridization method” in which the output from one filtering technique is used as an input for another method; only after applying soft computing techniques to the initial output. Hence, it is a three stage process (as shown in Fig. 6). Here, in the first stage, new features i.e. preferred items are generated by a virtual contributing recommender where we have applied ‘user-based collaborative filtering method’ as a contributing recommendation model. A logged-in user gives his ratings for some items which are matched with the other users’ ratings to determine the neighbors or users with similar tastes (by applying Pearson correlation similarity; explained in section 6). In the second stage, the fuzzy agent applies the fuzzy c-means clustering and then the interval type-2 fuzzy logic to the data obtained for generating a flexible product domain depending upon various user categories as per their profiles (as done in the previous section). In the final and third stage, the primary recommender agent apply ‘content-based filtering method’ on the obtained features (preferred items) in which user’s preferences are considered to recommend the most relevant items to the user. Finally, a user feedback is obtained on item recommendations to determine user’s satisfaction with the system.

OPEN IN VIEWER

Fig.6

A block diagram depicting the FFCH technique.

Recommendation is viewed as an information retrieval task. Particularly, it is required to retrieve (recommend) all items which are predicted to be good or relevant. Therefore, it is specially a task necessitating high precision as defined below.

Precision: is defined as a measure of exactness or accuracy. It determines the fraction of relevant items retrieved out of all items retrieved. That is, precision represents the proportion of recommended items that are relevant and actually liked by the user (see Equation 8).

Precision ( p ) = | relevant items recommended | | all items retrieved and recommended |(8)

Recall: (also known as sensitivity) is the fraction of relevant instances that are retrieved. Recall determines the fraction of relevant items retrieved out of the number of relevant items that should have been returned. That is, recall represents the proportion of relevant items that are recommended and all relevant items actually available refer Equation 9). Recall ( r ) = | relevant items recommended | | all relevant items |(9)

F-measure: is a measure that combines precision and recall. It is the harmonic mean of precision and recall as given in Equation 10. F - measure = 2 * p * r p + r(10)

Confusion Matrix: a table of confusion (sometimes also called a confusion matrix), is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives.

Root Mean Squared Error: also known as RMSE. It is a frequently used measure of the differences between values (sample and population values) predicted by a model or an estimator and the values actually observed. It is calculated using Equation (11) and places greater emphasis on larger errors by penalizing more when errors are significant. RMSE = ∑ i = 1 n ( fi - yi ) 2 n(11)

The principal reason for using this metric is that these errors can have the greatest impact on the user decision. For example, on a 5-point scale, a 1-point error may not be perceptible by the user (items rated with either 4 or 5 points are good recommendations), while with a 4-point error, the algorithm could be recommending a very bad item.

We executed offline evaluation over two datasets based on (i) book purchase and (ii) shopping women apparels. The book dataset was elicited from http://csl.du.ac.in/, the official website of Delhi University for its Central Science Library (CSL). It is a huge repository of thousands of books for several academic disciplines. Presently, we concentrated only on the computer science books for building a book dataset for the prototype. CSL provides a book issue facility for 6507 titles under the discipline ‘Computer Science’. For the experimental study, 30 different relevant user profiles from various categories were created, who actually rated over 100 books on the five selected book attributes (type, author, publisher, publication year (oldness) and cost) on the Likert rating scale from –2 to +2. For all the recommendation runs performed (10 runs performed per user), the number of top N recommendations was fixed at 10. Therefore in all, the total number of final recommendations generated by using a given RS equals almost 30×10×10 i.e. 3000 ratings. This dataset was then used for evaluation purpose as detailed in this section.

Similarly we had worked over another product domain i.e. shopping for women apparels. The product descriptions were collected by running the web crawler Visual Web Ripper (http://www.visualwebripper.com/) over the shopping website www.Amazon.com. The collected data were preprocessed in order to map text associated with the item to a set of attributes and values. Various item attributes collected were: item name, type, size, color, price, discount and other additional features were listed down in special features column. Various user profiles are children, teenagers, middle aged people and old aged people. Here, the user profile attribute ‘age’ and the item attribute ‘price’ has an inherent uncertainty associated with it, so this could also be modeled using fuzzy logic (here fuzzy c-means clustering and IT2 fuzzy sets were applied to the respective attributes). Training and evaluation data are computed in a manner similar to that of book purchase dataset. These datasets are used for computing recommendations for the user. For the experimental study in case of shopping apparels dataset, 35 different relevant user profiles from various categories were created, who actually rated over 100 items (women apparels) on the above mentioned attributes on the Likert rating scale from –2 to +2. For all the recommendation runs performed, the number of top N recommendations was fixed at 10. Therefore in all, the total number of final recommendations generated by using a given RS equals almost 35×10×10 i.e. 3500 ratings. This dataset was then used for evaluation purpose for measuring precision and errors as explained below.

Offline evaluation is one of the most popular approaches in the recommender system studies. It can be used for measuring precision as well as computing errors by comparison. It is based on a dataset that is divided into two subsets: training and evaluation. The training subset is the data the algorithm knows, that is, the data the algorithm uses to compute the recommendation or rating predictions. These are then compared with the data in the evaluation subset. The experiments were performed by dividing the dataset into two groups, a training subset and an evaluation subset. The first set corresponds to the data used to train the algorithm. With such information, the algorithm computes the recommendation (and generates prediction ratings) that will be later compared with the original data present (actual user ratings) in the evaluation subset for computing errors. We have followed a random approach for selecting the training subset. The training subset is constructed from a percentage of the available ratings, randomly chosen. For our tests we used all the following percentages: 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, and 90%. Initially, the evaluation subset was composed by randomly selecting 10% of the dataset. Obviously, ratings that appear in the evaluation subset were never included in the training subset. With high percentages of ratings in the training set, we can evaluate the behavior of the algorithm under relatively high density conditions. In contrast, a small percentage allows evaluating the algorithm under sparsity conditions, common in the initial phases, or in domains with a large number of users and/or items. Data sparsity is a major challenge in recommender systems applying collaborative filtering [38] Sparsity refers to the fact that some observed ratings in a user-item rating matrix are too few. Undoubtedly, the most challenging situation belongs to the sparsity conditions prevailing when the training dataset is comprised of only 10% data. Therefore it is imperative to perform accuracy evaluation under data sparsity conditions using the most popular metrics in the literature. We have computed precision, recall and F-measure for training dataset formed of various lower percentages of data. Though in this paper, we have included only the results with training dataset formed of 10% data to avoid repetition of results. Evidently, it was seen that performance of various algorithms decreased with decrease in the density of the rating matrix and increased in the opposite situation. Moreover, error computation using RMSE has been depicted graphically for all the different percentages of data used in the training set. Each test was repeated 10 times, so the results presented in the next sub-section refer to their means.

6.3 Results and analysis

We studied the evolution of accuracy of different recommendation algorithms (CF, Type-1 fuzzy logic and type-2 fuzzy logic approaches) for two different datasets: book purchase dataset and shopping women apparels dataset, when the percentage of data used as training set was changed. The results for the book dataset with around 3000 ratings are shown in Tables 3–5 and for the women apparels dataset with around 3500 ratings are shown in Tables 6–8. Also, to determine how many correct items are recommended or not to the user, both recommended items and user’s feedback have been analyzed and the results of the analysis are shown as confusion matrices. Confusion matrices for book purchase and shopping women apparels datasets are depicted in Tables 3–5 and Tables 6–8 respectively. True positives represent the number of items that were correctly recommended by the system, false positives are items that were recommended by the system but user considered them to be irrelevant, false negatives are items that should have been recommended but the system did not recommend them, and true negatives are items which were correctly detected as not relevant (hence not recommended) for the corresponding user. Analyzing the results from all the tables, we have observed that type-2 fuzzy based approach is outperforming both CF and type-1 fuzzy based approach for both the datasets. Also, in all three confusion matrices, higher value of true positives and true negatives depict that the system has been able to successfully determine relevant and irrelevant recommendations for the user.

Analyzing the results from Tables 3–8, the error rate for various recommendation approaches like CF, fuzzy type-1 and fuzzy type-2 on Book purchase dataset is 0.14, 0.11 and 0.08 respectively. For the experimentation performed on same approaches for Shopping Women Apparels dataset, error rate was found to be 0.16, 0.14 and 0.09 respectively. Similarly, the accuracy rate determined using approaches CF, fuzzy type-1 and fuzzy type-2 is 0.85, 0.88 and 0.92 respectively (on Book Purchase Dataset) and 0.84, 0.86 and 0.90 respectively (on Shopping Women Apparels Dataset).

Furthermore, the measures precision, recall and F1 have been calculated during experimentation for all the approaches (CF, fuzzy type1 and fuzzy type2). However, to optimize space utilization, comprehensive calculations have not been included here. In this case, the average of precision, recall and F1 for the recommended items of the Book Purchase dataset are 61%, 65%, 63% (using CF approach), 65%, 69%, 67% (using fuzzy type-1 approach) and 73%, 75%, 74% (using fuzzy type-2 approach) respectively. Similarly, average values of precision, recall and F1 of recommended women apparels are 63%, 67%, 65% (using CF approach), 67%, 71%, 69% (using fuzzy type-1 approach) and 73%, 76%, 74% (using fuzzy type-2 approach) respectively. Above results clearly signifies that fuzzy type-2 approach is outperforming both fuzzy type-1 and Resnick’s CF approach. Also, the system performance can be considered as quite successful.

As observed, in case of precision, recall andF-measure metrics, type-2 FFCH surely outperforms other algorithms (Resnick’s CF and type-1 FFCH) for both datasets; especially in case of data sparsity conditions (the instances of results depicted here are meant for the training set formed out of 10% data only).

Recommender evaluations using RMSE metrics are shown in Figs. 7 and 8 for Book purchase dataset and Shopping women apparels dataset respectively. It could clearly be seen in both the figures that type-2 fuzzy approach is performing better than type-1 fuzzy and CF approaches. RMSE in case of type-2 fuzzy approach for recommendation is the least of all. In fact we observe that, with a training set of 80% data, the variation amongst the three algorithms for RMSE is less. While with the training set of 10% data, the gap in RMSE results between the proposed algorithm and its counterparts is quite significant. Also, all three algorithms improve as the percentage of training data increases. This behavior is just as expected; by increasing the percentage of data in the training set, we are increasing the density of the rating matrix and, therefore, the algorithm has more information to compute the prediction. Similarly, as the density of the information increases, a slight decrease in the differences among algorithms is observed. Most of them present similar results under relatively high density conditions, while their differences are accentuated as density diminishes.

OPEN IN VIEWER

Fig.7

Recommender evaluations using RMSE metrics for Book purchase dataset.

OPEN IN VIEWER

Fig.8

Recommender evaluations using RMSE metrics for Shopping women apparels dataset.

The greatest limitation an algorithm is subjected to is the lack of information. In other words, the accuracy of a prediction is limited by the amount of information that can be obtained from the rating matrix [7]. As observed, the results under high density conditions are pretty good in all the algorithms, although this changes when the density decreases. The tests under sparsity environments are good indicators of the ability of algorithms to extract more information from the rating matrix. As observed in the experimental results the proposed interval type-2 FFCH algorithm performed considerably well.

Summarizing, in this paper, we presented an IT2 based fuzzy technique (called FFCH) for generating recommendations which improved the quality of recommenders by making the system flexible in capturing user preferences thereby ensuring that the algorithm extracts more information from the available dataset. The system dealt with the uncertainty related to user preferences by increasing the domain flexibility and diversity for recommendations. The approach improved prediction accuracy and reduced errors of the RS as the recommendations produced were found to be more accurate with lesser number of errors.