Collaborative filtering recommendation system based on trust-aware and domain experts

Abstract

Collaborative filtering is a popular tool for recommendation systems. However, collaborative filtering technologies often suffer from high time complexity, the cold-start problem, and low coverage. Recent research shows that social networks and trust-aware methods can effectively solve these problems. Therefore, we propose a Trust Domain Expert Collaborative Filtering recommendation system. First, we divide the user item rating matrix into multiple sub-matrices based on the domain attributes of each item. For each sub-matrix, we then use domain experts to construct a user–expert trust matrix. Finally, combined with the target user’s domain of interest, we predict their missing ratings. Experimental results show that this method not only improves the accuracy and recommended coverage of collaborative filtering-based methods, but also reduces the computation time.

Keywords

Recommendation expert trust-aware collaborative filtering

1. Introduction

Rapid developments in science and technology have resulted in increasingly serious data overloading problems [35], whereby users struggle to find the information they need in an ocean of data. Recommendation systems are an important tool for solving this problem, as they provide users with helpful suggestions that aid the decision-making process [24]. Recommendation system algorithms can be roughly divided into demographic-based [26], content-based [9], and collaborative filtering (CF) [39, 40, 41] algorithms. The most widely used scheme is CF, which determines whether to recommend an item to the user by predicting the ratings of unknown items from the user’s previous article scores [29]. CF recommendation systems are generally based on nearest-neighbor CF [16]. There are two main types of nearest-neighbor filtering in CF: filtering based on the user CF recommendation [30] and filtering based on project (item) CF recommendations [31]. The basic idea of user-based CF recommendation systems is that users may like items that are liked by other users who have similar interests. Here, similar users are identified from the target user’s historical ratings. However, traditional CF has the following major problems.

Sparsity problem [13, 15, 10, 3]: Users typically rate only a small fraction of available items. Most users rate fewer than 1% of items, and the density of available ratings in recommendation systems is often less than 1%. This limits the prediction quality of CF.

Cold-start problem [1, 2, 6]: The cold-start problem occurs when users have rated few or no items, or an item has been rated by few or no users. The CF approach requires many user/item ratings for effective recommendations, and does not work for new users who have not rated any items or new items that have not received any ratings [36]. Similarity-based methods thus fail to identify suitable neighbors and return low-quality recommendations.

Scalability: Traditional CF generally calculates the similarity among all users. Thus, when the number of users is large, the time complexity of the algorithm becomes excessively high.

Because of the above limitations, traditional CF only considers the user-item rating matrix for recommendations, and does not provide a quality recommendation. Besides the rating data, many researchers have used additional information to solve the problems of traditional CF methods. For instance, the social recommendation method was developed to solve the cold-start problem. This method builds a social network from the social relationships within user information. As long as a new user has a direct or indirect social relation network with others, user interest models can generate recommendations for the new user. Massa and Bhattacharjee [21] first introduced the idea of trust into recommendation systems, and proposed the trust perception recommendation system framework. The foundation of trust-based recommendations is the hypothesis that people usually like to refer to the preferences of trusted friends rather than those of the mass population when making decisions. Massa and Bhattacharjee [21] used a trust value between users to replace traditional similarity; such trust values are given explicitly. Experimental results showed that this method could solve the cold-start problem and predict the coverage rate. However, this value of trust between users is Boolean (i.e., 0 or 1), which is a weakness of the trust expression. The trust perception recommendation system has been extended to a trust transmission method, and the user trust matrix has been combined with the user similarity matrix to jointly predict ratings [19, 18]. The results from these approaches indicate that the trust perception recommendation system effectively improves the recommendation coverage without affecting the recommendation accuracy. For the scalability problem, we present an expert strategy that reduces the time complexity of the proposed system. Employing this method requires a small number of high-quality users to predict missing ratings in place of the neighbors in traditional systems. This article refers to these high-quality users as experts.

The remainder of this paper is organized as follows. Section 2 discusses some related studies, before Section 3 introduces the proposed algorithm for the trust-aware method and the selection of expert users. Section 4 presents performance and quality results from applying the trust-aware domain expert collaborative filtering (TDE-CF) recommendation system to the MovieLens dataset. Our conclusions are presented in Section 5.

2. Related work

In recent years, the scale of web-based social network services has developed rapidly. A social network service requires users to register a real identity and maintain real interpersonal relationships. The authenticity of the interpersonal relationships is thus important. The trust between users has positive significance.

The authenticity of the interpersonal relationship is thus useful. In particular, the trust relationship among users has positive meaning. However, research on trust relationships among users is still in its infancy [22]. Most trust-aware research is based on explicit trust, which means there is a user–user trust matrix that provides information. Trust represents the relationship between the source user and target user, and expresses the quality of service providers; e.g., the Epinions open dataset is based on a network of trust whereby users indicate which users they trust or distrust. Therefore, the dataset already has an extra attribute for users. FilmTrust is an online social network combined with a movie rating system where users are encouraged to evaluate their friends’ favorite movies on a scale from 1 to 10. An explicit trust network calculates the value of trust between non-adjacent user nodes in the trust network, and uses trust propagation and aggregation rules. Well-known trust calculation models include the TidalTrust model proposed by Golbeck [8] and the Moletrust model proposed by Massa and Avesani [19]. However, the trust between users is difficult to obtain directly, because many film or music criticism websites dilute the creation of social networks. In a previous study on trust, Liu et al. [14] proposed a novel recommendation method that first generates a domain-specific trust network pertaining to each domain, and then builds a unified objective function to improve the recommendation accuracy by incorporating hybrid direct/indirect trust information into a matrix factorization recommendation model. Qiao et al. [27] proposed a method of calculating the trust value based on user context, with the trust principle from social psychology applied to calculate trustworthiness from user context in a social network. This method divides trust in the social network into the trust produced by familiarity and the trust produced by similarity. Moreover, similarity is divided into internal similarity and external similarity according to its level of importance. The notation used to describe conventional approaches and the proposed method is presented in Table 1.

Table 1
Notation and symbols used in this paper

Symbol	Meaning
$r_{u,i}$	The rating about user $u$ about item $i$
sim(u,v)	The similarity of user $u$ and user $v$
$N_{u}$	A set of $k$ neighbours most similar to user $u$
$\overline{r_{u}}$	Average rating about user $u$
predict ${}_{u,i}$	Predict rating about user $u$ in item $i$
$p_{c,u,i}$	Predict rating about user $u$ in item $i$ under domain $c$
str ${}_{u,v}$	The satisfaction rate of user $u$ on user $v$
trust_adj ${}_{u,v}$	Adjusted trust value of user $u$ to user $v$
trust_ind ${}_{a,b,c}$	Propagated trust value from user a to user $c$ by one intermediate node user $b$
trust_del ${}_{a,c}$	Propagated trust value from user a to user $c$ by all intermediate node node user
trust ${}_{u,v}$	When building trust matrix, both adjusted trust and indirect trust are called trust.
com ${}_{a,b}$	The number of common rating items by user $a$ and user $b$
int ${}^{c}_{u}$	The interesting of user $u$ about domain $c$
$w^{c}_{u}$	Normalized int ${}^{c}_{u}$
$E_{c}$	Expert set in domain $c$

Traditional CF predicts missing ratings using a version of the following expression:

$\displaystyle r_{u,i}=\frac{\sum_{v\in N_{u}}{r_{v,i}*\textit{sim}(u,v)}}{\sum% _{v\in N_{u}}\textit{sim}(u,v)},$ (1)

where $r_{u,i}$ is the predicted rating of user $u$ for item $i$ , $N_{u}$ is the set of $k$ neighbors most similar to user $u$ , and $\textit{sim}(u,v)$ is the similarity between users $u$ and $v$ .

However, this method does not consider the rating metrics of different users, i.e., some users treat ratings fairly casually while others treat the item ratings more seriously. To overcome this problem, [30] proposed the classic CF formula given in Eq. (2), which includes the mean ratings of each user.

$\displaystyle r_{i}=\overline{r_{u}}+\frac{\sum_{v\in N_{u}}(r_{v,i}-\overline% {r_{v}})*\textit{sim}(u,v)}{\sum_{v\in N_{u}}\textit{sim}(u,v)},$ (2)

where $r_{u,i}$ is the predicted score of user $u$ for item $i$ , $\overline{r_{u}}$ is the average score of user $u$ for all items, $N_{u}$ is the set of $k$ neighbors most similar to user $u$ , and $\textit{sim}(u,v)$ is the similarity of users $u$ and $v$ .

There are a variety of similarity metrics, with the most commonly used being the cosine similarity, Pearson similarity, and modified cosine similarity.

Cosine similarity[31, 37, 38]:

$\displaystyle\textit{sim}(i,j)=\textit{cos}(\vec{i},\vec{j})=\frac{\vec{i}*% \vec{j}}{(||\vec{i}||_{2})*(||\vec{j}||_{2})}$ (3)

Modified cosine similarity:

$\displaystyle\textit{sim}(i,j)=\frac{\sum_{u\in I}(r_{i,u}-\overline{r}_{u})(r% _{j,u}-\overline{r}_{u})}{\sqrt{\sum_{u\in I}(r_{i,u}-\overline{r_{u}})^{2}}% \sqrt{\sum_{u\in I}(r_{j,u}-\overline{r}_{u})^{2}}}$ (4)

Pearson similarity[43, 42]:

$\displaystyle\textit{sim}(i,j)=\frac{\sum_{u\in I}(r_{i,u}-\overline{r}_{i})(r% _{j,u}-\overline{r}_{j})}{\sqrt{\sum_{u\in I}(r_{i,u}-\overline{r_{i}})^{2}}% \sqrt{\sum_{u\in I}(r_{j,u}-\overline{r}_{j})^{2}}}$ (5)

3. Recommendation based on trust aware and domain expert

3.1 Recommendation based on trust aware and domain expert description

Traditional CF algorithms often have high time complexity. Some user-provided information cannot be used by most other users, e.g., many users have few item ratings in common with other users. Even though the similarity between the source user and target user may be high, the source user cannot be effectively used to predict the missing ratings of the target user if there are few common ratings. Thus, the present study employs an expert user strategy to select a small number of high-quality users. The selection of expert users reduces the number of neighbors required, which may lessen the sparseness and cold-start problems. Here, we propose the TDE-CF method shown in Fig. 1, which combines trust awareness with expert users. The TDE-CF method predicts missing ratings according to

$\displaystyle p_{c,u,i}=\overline{r}_{u,c}+\frac{\sum_{v\in E_{c}}(r_{v,i}-% \overline{r}_{v,c})*trust_{u,v}}{|E_{c}|},$ (6)

where trust ${}_{u,v}$ is the trust value between users $u$ and $v$ , and $E_{c}$ is the set of experts in domain $c$ .

Figure 1.

TDE-CF algorithm flow chart.

Because an item has different degrees of belonging to different domains, we introduce a measure $\textit{attr}_{u,c}$ to indicate the number of items rated by the user in domain $c$ . Finally, the score of user $u$ for item $i$ can be predicted according to

$\displaystyle p_{u,i}=\frac{\sum_{c\in C_{i}}p_{c,i,u}*\textit{attr}_{u,c}}{% \sum_{c\in C_{i}}|\textit{attr}_{u,c}|}.$ (7)

The algorithm presented in this paper proceeds as follows.

Step 1:

Select the domain expert user $u$ sing the expert user selection algorithm.

Step 2:

Calculate the direct trust relationship between all users and expert users.

Step 3:

Calculate the transmitted trust relationship between the remaining users and expert users.

Step 4:

Predict the user’s rating for an the item in the domain.

Step 5:

If all domains have been selected and processed in the previous steps, go to Step 6; else, return to Step 1.

Step 6:

For all users, combine the degree of user–domain interest and predict the user–item rating.

In the social recommendation system, the most popular social relations networks are based on a trust score. The trust relationship is the most widely used social networking relationship in socialized recommendation systems [22]. Trust has subjectivity, domain relevance, asymmetry, and is not fully transitive [44]. In similarity calculations, for example, the similarity between users in the direction A $\to$ B is equal to that in the direction B $\to$ A. However, in the trust perception mechanism, the trust that user A has in user B is not necessarily equal to the trust that user B has in user A. Users may have different trust values in different domains for the same target user, e.g., user $u$ is a computer expert, and so his friends trust him in the computer domain, but they may not trust him as highly for film comments. On the basis of the above analysis, our trust-aware method must meet this constraint. The trust relationship can be divided into explicit trust and implicit trust [11]. Explicit trust is given directly by the user and indicates the degree of trust in other users, whereas implicit trust is often calculated from ancillary information. Explicit trust has been used as a metric of the degree of trust [18]; user–item rating data have been used to calculate implicit trust between users [25]. Jia [7] used the MovieLens dataset, which does not have explicit trust information; the method presented in this paper refers to [25] and proposes an improved model of calculating trust.

3.2 Trust aware description

Assuming that users have interacted in the past, if user A has provided a greater number of reliable recommendations to user B, then user B has greater trust in user A. Consider an example in which users a, b, and c have a past recommendation history. Users a and b have each made 10 recommendations to user c, although, according to feedback, user c was only satisfied with one of user a’s recommendations but was satisfied with nine of user b’s recommendations. Obviously, user c will have much higher confidence in user b than in user a. In future interactive behavior, user c will be more inclined to adopt the recommendations of user b. The construction of a trust computing model is proposed on this basis. The leave-one-out method is used to determine the user recommendation in the recommendation system. First, Eq. (8) is used to predict the rating of item $i$ for target user $u$ from previous rating information given by user $v$ :

$\displaystyle P_{u,i}=\overline{R_{u}}+(R_{v,i}-\overline{R_{v}}),$ (8)

where $P_{u,i}$ denotes the predicted rating of target item $i$ by user $u$ , $R_{v,i}$ denotes the rating of item $i$ by user $v$ , and $\overline{R_{u}}$ and $\overline{R_{v}}$ denote the average ratings of users $u$ and $v$ , respectively. Depending on the deviation between the evaluation rating and actual rating, the target user then provides information on the target item according to

$\displaystyle\textit{sat}_{u,v}^{i}=\begin{cases}1,&\left|P_{u,i}-R_{u,i}% \right|\leqslant\varepsilon\\ 0,&\text{else}\end{cases}$ (9)

where $\textit{sat}_{u,k}^{i}$ is the estimation capacity of target user $v$ for recommended user $u$ with respect to item $i$ , and $R_{u,i}$ is the actual rating of item $i$ assigned by user $u$ . Finally, we calculate the initial trust of user $u$ in user $v$ as follows:

$\displaystyle\textit{str}_{u,v}=\frac{\sum_{{i=1}}^{i\in I_{u,v}}\textit{sat}_% {u,v}^{i}}{|I_{u,v}|},$ (10)

where $\textit{str}_{u,v}$ is the satisfaction rate of the target user $u$ in user $v$ . The above method of calculating trust is not complete. It is assumed that the trust between users $u$ and $v$ is equal to that between user $u$ and user l. However, because there are more rating items that are common to users $u$ and $v$ than are common to users u and l, the trust that user $u$ has in user $v$ should be higher than that of user $u$ in user l. Therefore, we introduce the number of commonly rated items in calculating the trust value, i.e., the adjusted trust value is higher when users have more commonly rated items. We calculate the adjusted trust according to

$\displaystyle\textit{Ptrust\_adj}_{u,v}=\frac{1}{1+e^{-\frac{|\textit{com}_{u,% v}|}{2}}}*\textit{str}_{u,v},$ (11)

where $\textit{trust\_adj}_{u,v}$ is the adjusted trust of user $u$ in user $v$ , and $\textit{com}_{u,v}$ is the number of rating items common to users $u$ and $v$ . The modified trust value approaches 1 as the number of common items increases, i.e., the trustworthiness increases. The commonality rating of an item is low if the modified trust value is close to 0.5, i.e., there is only partial trustworthiness [11]. Through the above trust calculation, we can build a trust aware network in which nodes represent users and directed arcs indicate the trust value between two users. As a consequence of the previously introduced properties of trust, the network is a directed, weighted graph. Owing to the sparseness of the matrix, the trust values of some users cannot be directly calculated. For example, if users $u$ and $v$ have no common ratings, user $v$ will have zero trust in user $u$ . However, we can generate novel recommendations if we can establish trust between users $u$ and $v$ via a propagation of trust [25, 20] from the trust network. As shown in Fig. 2, the trust values of A $\to$ B, A $\to$ C, B $\to$ D, C $\to$ D, and B $\to$ E are known. If we can predict the trust values of A $\to$ E and A $\to$ D, then the performance of the recommendation system will be improved [43]. We propose a trust propagation method for such situations. Assuming that the trust value between users a and c cannot be directly generated, it may be possible to indirectly generate trust through the process of trust propagation via intermediate user b. For example, if the trust value between users a and b is 0.6 and they have 10 commonly rated items, whereas the trust value between users b and c is 0.8 and they have four commonly rated items, then the trust between a and c can be calculated as trust(a, c) $=$ (0.6 * 10 $+$ 0.8 * 4)/(10 $+$ 4) $=$ 0.65. The trust propagation mechanism is described in Eq. (12); for multiple such user nodes, we calculate their average trust value as shown in Eq. (13).

$\displaystyle\textit{trust\_ind}_{a,b,c}=\frac{\textit{str}_{a,b}*\textit{com}% _{a,b}+\textit{str}_{b,c}*\textit{com}_{b,c}}{|\textit{com}_{a,b}+\textit{com}% _{b,c}|}.$ (12) $\displaystyle\textit{trust\_del}_{a,c}=\frac{\sum_{x\in X}\textit{trust\_ind}_% {a,x,c}}{|X|},$ (13)

Figure 2.

Trust network.

where com ${}_{a,b}$ denotes the number of common items rated by users a and b, and X is a set of intermediate users that have a transfer chain between users a and c. Figure 3 shows that there is no direct trust between users $u$ and $v$ , but u has a trust relationship with t1, t2 … tm, and t1, t2 $\ldots$ tm have a trust relationship with $v$ . Through Eqs (12) and (13), we obtain the value of trust between users $u$ and $v$ . In the proposed method, we only use expert users as intermediate user nodes, and we take the adjusted trust value and propagating trust value as the actual trust values.

Figure 3.

Trust propagation.

3.3 Expert users description

From the above analysis, we know that the scale of the trust network grows exponentially with the number of users. As the number of items in the recommendation system is very large, there will be some users who have rated very few items. These users will have few ratings in common with other users, which may lead to them being highly trusted by other users (if these items are popular). However, such users with a high degree of trust but very few ratings may not give good recommendations. Therefore, although their trust value may be lower, users with additional scoring knowledge may give better recommendations. Hence, we consider this situation and propose a novel algorithm based on filtering expert users to improve the performance of recommendation systems. An expert is someone who has professional expertise in an academic, technical, or other domain. Ordinary users are more inclined to trust such experts. For example, in the spheres of microblogging and Twitter, star users have much greater influence than ordinary users. However, most of these users are only effective in certain areas, e.g., the influence of movie stars is limited to the film domain. Based on this, we shrink the selection of expert users to a certain domain. As mentioned above, our expert users produce information for a number of other users. Expert users are those who have given the most rating information on particular items [34]. On this basis, we derive the following expert selection procedure.

First, we obtain the user-item rating matrix $R_{m*n}$ , from which we determine the signal matrix $A_{m*n}$ . The signal matrix is divided into the number of separate domains being considered, with the columns of each sub-matrix corresponding to one user ID and the rows corresponding to the items included in this domain. Assuming there are nine domains, the signal matrix will divided into nine sub-matrices, with the i-th sub-matrix expressed as $A_{m*n(i)}^{i}$ . These sub-matrices are decomposed as

$\displaystyle A_{m*n(i)}^{i}=U_{m*k(i)}^{i}\Sigma_{k(i)*k(i)}^{i}V_{k(i)*n(i)}% ^{i},$ (14)

The first column of matrix $U_{m*k(i)}^{i}$ contains the importance of users with respect to the items. Thus, for sub-matrix A, we choose the corresponding decomposed matrix U. The first row of matrix U indicates the frequency of each user’s comments. We select the user corresponding to the maximum value in this row as the domain expert.

Because an item may belong to multiple domains and user interests vary across each domain, we separately calculate the user interest in each domain as follows. The items rated by user $u$ are collected in a set I. For every i in I, if i belongs to domain c, then user $u$ ’s interest in domain c is incremented by 1. Finally, we obtain the interest scores for each domain and normalize them, and the result is the user’s degree of interest in every domain. For example, if user $u$ has rated two items, with item 1 belonging to domains 1 and 2, and item 2 belonging to domains 2 and 3, then domain 1 scores 1, domain 2 scores 2, and domain 3 scores 1. The normalized user interest is thus 0.25 in domain 1, 0.5 in domain 2, and 0.25 in domain 3. This process can be expressed as follows.

$\displaystyle\textit{vote}_{u,i}^{c}=\begin{cases}1,&(i\in c)\\ 0,&\text{else}\end{cases},$ (15)

where $\textit{vote}_{u,i}^{c}$ gives the score of item $i$ for domain c. If item $i$ belongs to domain c, then $\textit{vote}_{u,i}^{c}=1$ .

$\displaystyle\textit{int}_{u}^{c}=\sum_{i\in I}{\textit{vote}_{u,i}^{c}},$ (16)

where $\textit{int}_{u}^{c}$ is the number of the items for which user $u$ has ratings in domain c.

$\displaystyle w_{u}^{c}=\frac{\textit{int}_{u}^{c}}{\sum_{c\in C}{\textit{int}% _{u}^{c}}},$ (17)

where $w_{u}^{c}$ is user $u$ ’s interest in domain c after normalization.

3.4 Theorectical comparison with some methods

In [4], various memory-based and model-based algorithms are analyzed. Under the same conditions, memory-based algorithms generally perform better than model-based algorithms in terms of accuracy, but high numbers of users or items induce high computation times for CF. Thus, we do not consider model-based algorithms further, and instead compare five CF-based methods: traditional user-based CF recommendations (UBR), CF-DNC [7], a domain-expert-only method (EXP), a combined expert users and trust-aware method (EXP-TST), and the proposed TDE-CF, which adds user-domain degree information. These four methods are summarized in Table 2.

Owing to the similarity calculation, the time complexity of traditional user-based CF recommendation algorithms is $O(m*n^{2})$ (where m is the number of items and n is the number of users). Our algorithm only calculates the trust between experts and users, and thus has a time complexity of $O(m*n)$ . The trust and trust transmission methods introduced in the proposed TDE-CF effectively alleviate the problems of cold starting and shilling attacks faced by recommendation systems [28].

UBR: Traditional user-based collaborative filtering recommendation. It use similarity to find neighbour users, then use these users to predict the rating.

EXP method: Add expert user in trust aware recommendation algorithm to predict rating. Namely, use expert users to replace traditional selected neighbour users in predicting rating.

EXP-TST method: Add trust propagation method in EXP method. The method is ready for expand predicting coverage. CF-DNC method: Firstly, on the basis of the computational result of user similarity, the preferred similar users of target users are chosen dynamically. The trust computing model is then designed to measure the trust relation between users according to the ratings of similar users. The trustworthy-neighbor set of the target user is selected in accordance with the degree of trust between users. Finally, a novel CF recommendation algorithm based on the double-neighbor selection strategy is designed to generate a recommendation for the target user.

TDE-CF: Consider that many items belong to more than one domain, and that there are multiple predicted ratings for the item. Thus, it is necessary to fuse these ratings. User–domain degree is also considered in this method. First, experts are identified using the expert user calculation method. The network of trust values between users and experts is then constructed and the interest score of all users in each domain is calculated. Finally, in each domain, the user’s predicted rating is calculated and combined with the user domain interest to give the target user’s final predicted rating for an item.

Table 2
Comparison of methods

Name	Method	Advantage	Shortage
UBR	User-based recommender, similarity	Simple in calculate, programmability	High time complexity cold start problem
EXP	Expert, trust awareness	Lower time complexity, extensibility	Result not diversity, Ignore user’s domain interest
EXP-TST	Expert, trust awareness trust propagation	Lower time complexity, higher coverage, extensibility	Ignore user’s domain interest
CF-DNC	Double neighbour choosing	More accuracy, resists malicious attack	Higher time complexity
TDE-CF	Expert, trust, trust propagation user-domain degree	More accuracy, higher coverage, lower time complexity, extensibility	Time complexity is greater than EXP, EXP-TST

4. Experimental

4.1 Data set introduction

Considering the auxiliary information used in our algorithm, we conducted experiments on the well-known MovieLens rating datasets provided by the GroupLens research project team at the University of Minnesota: the MovieLens 100K data set (ML-100K) [33] and MovieLens 1M data set (ML-1M)[45, 12]. The ML-100K data set contains 100,000 ratings given by 943 users for 1682 movies, whereas the ML-1M data set contains 1 million ratings from 6040 users for 3952 movies. Each user has rated at least 20 movies. Users and items are numbered consecutively from 1. The data are randomly ordered.

4.2 Evaluation metrics

The present paper uses the mean absolute error (MAE); root-mean-squared error (RMSE), which is widely used in recommendation research[17, 32]; running time and coverage as evaluation indicators. The MAE and RMSE are calculated as

$\displaystyle\textit{MAE}=\frac{\sum_{i=1}^{n}|p_{i}-r_{i}|}{n},$ (18) $\displaystyle\textit{RMSE}=\frac{\sqrt{\sum_{i=1}^{n}(p_{i}-r_{i})^{2}}}{n},$ (19)

where $p_{i}$ is the predicted rating for the target user on target item $i$ , $r_{i}$ is the actual rating of the target user on target item $i$ , and n is the number of predictions.

Some recommendation systems may not be able to predict ratings for all test data. The introduction of trust can enhance the coverage. Thus, we use the coverage metric to measure whether a value can be predicted. For test ratings, if the recommender cannot find a prediction on the rating, it means that the recommender cannot cover this rating. The coverage metric is computed as

$\displaystyle\textit{coverage}={\frac{P}{T}},$ (20)

where $P$ denotes the number of predicted ratings and $T$ denotes the number of test ratings; e.g., the coverage is 0.10 if a recommender system can predict 100 out of 1000 ratings on items to be predicted. Larger coverage values indicate that the recommendation method can predict more ratings for users with missing ratings [5]. The F-measure is computed to combine RMSE and coverage into a single evaluation metric [6]. First, the RMSE is converted into a precision metric in the range [0,1]. The precision is given by Eq. (21)

$\displaystyle\textit{precision}=1-\frac{\textit{RMSE}}{4}.$ (21)

Where the maximum possible error is set to 4 because the ratings are in the range [1, 5]. The F-measure is

$\displaystyle\textit{F-measure}=\frac{2*\textit{precision*coverage}}{\textit{% precision}+\textit{coverage}}.$ (22)

Because the two data sets are very different in size, we add the running time as an evaluation metric. The experiments were executed in MATLAB running on a 3.30-GHz Intel Core-i5 processor with 64-bit Windows 10 and 8-GB RAM.

4.3 Parameter selection

We conducted multiple experiments with different values of $\varepsilon$ . Table 4 presents the resulting MAE and RMSE for the ML-100K dataset. It can be seen that, as the number of experts increases, the optimal MAE increases while the RMSE remains fairly constant. We consider a moderate value of $\varepsilon=$ 1.5 in the remainder of this paper.

4.4 Experimental results and analysis

To avoid effects of the division of the data set, we consider five-fold cross-validation for the ML-100K data set. Results are presented in Table 4. It is seen that in each data set, the MAE and RMSE differences are less than 2%, and the variances in MAE and RMSE have a magnitude of 0.0001. All results subsequently discussed are the averages of five-fold cross-validation. To evaluate the recommendation quality of the proposed algorithm, we compare the performances of EXP, EXP-TST, CF-DNC, TDE-CF, and a traditional user-based CF recommendation algorithm (UBR).

Figure 4a and c show MAE results for different data sets and different numbers of neighbors. Figure 4b and d show RMSE results for different data sets and different numbers of neighbors. In the case of the ML-100K data set, the EXP and EXP-TST methods were more accurate than UBR and CF-DNC methods because the EXP method selects expert users as high-quality neighbors. The trust-aware value is considered to reinforce the expert user’s weight. The TDE-CF method has the highest accuracy. It not only employs the above idea but also introduces the degree of interest that the user has in the domain, which increases the accuracy. The same applies to the ML-1M data set. The five methods have lowest values of MAE and RMSE when there are 70 experts (neighbors); MAEs (ML-100K, ML-1M) are (0.8888, 0.9512), (0.7725, 0.7375), (0.7837, 0.7418), and (0.7548, 0.7322) while RMSEs (ML-100K, ML-1M) are (1.134, 1.216), (0.9919, 0.9454), (1.011, 0.9555), and (0.9637, 0.9382) for UBR, EXP, EXP-TST, and TDE-CF methods respectively.

Figure 5 shows coverage rates of the EXP, EXP-TST, TDE-CF, CF-DNC, and UBR methods versus the number of experts for different data sets. The coverage represents the percentage of unseen movies for which the rating can be predicted with each method, with higher percentages being better [23]. EXP-TST and TDE-CF have higher coverage (99.66%, 99.90%) than EXP (99.32%, 99.74%) on ML-100K

Table 3
MAE and RMSE Performance with respect to $\varepsilon$

	10		20		30		40		50		60		70
	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
1.0	0.76976	0.98446	0.76128	0.97221	0.75908	0.96867	0.75773	0.96590	0.75698	0.96439	0.75578	0.96260	0.75560	0.96237
1.1	0.77098	0.98743	0.76129	0.97298	0.75875	0.96907	0.75694	0.96581	0.75596	0.96396	0.75445	0.96191	0.75410	0.96145
1.2	0.77323	0.99145	0.76163	0.97439	0.75870	0.96980	0.75648	0.96594	0.75518	0.96372	0.75346	0.96133	0.75295	0.96068
1.3	0.77572	0.99542	0.76224	0.97586	0.75891	0.97066	0.75628	0.96633	0.75480	0.96385	0.75280	0.96114	0.75215	0.96037
1.4	0.77814	0.99898	0.76279	0.97745	0.75885	0.97156	0.75606	0.96686	0.75447	0.96423	0.75231	0.96127	0.75166	0.96043
1.5	0.77879	1.00065	0.76287	0.97830	0.75842	0.97178	0.75553	0.96707	0.75372	0.96407	0.75125	0.96074	0.75047	0.95978
1.6	0.78356	1.00709	0.76550	0.98197	0.76019	0.97451	0.75687	0.96913	0.75489	0.96599	0.75217	0.96223	0.75144	0.96129
1.7	0.78678	1.01155	0.76689	0.98428	0.76090	0.97606	0.75733	0.97028	0.75521	0.96693	0.75237	0.96315	0.75161	0.96215
1.8	0.78953	1.01580	0.76811	0.98641	0.76170	0.97768	0.75792	0.97158	0.75557	0.96794	0.75252	0.96391	0.75161	0.96281
1.9	0.79216	1.01993	0.76937	0.98861	0.76275	0.97960	0.75868	0.97310	0.75616	0.96923	0.75295	0.96497	0.75196	0.96377
2.0	0.79415	1.02285	0.77030	0.99022	0.76350	0.98101	0.75924	0.97422	0.75658	0.97018	0.75332	0.96579	0.75221	0.96443

Table 4

Five-fold cross-validation results

Dataset	MAE							RMSE
$\backslash$ k	10	20	30	40	50	60	70	10	20	30	40	50	60	70
data1	0.77879	0.76287	0.75842	0.75553	0.75372	0.75125	0.75047	1.00065	0.97830	0.97178	0.96707	0.96407	0.96074	0.95978
data2	0.78369	0.77436	0.77030	0.76680	0.76487	0.76443	0.76407	1.00393	0.99217	0.98518	0.98081	0.97749	0.97579	0.97518
data3	0.78193	0.76678	0.76012	0.75795	0.75598	0.75432	0.75318	1.00584	0.98534	0.97578	0.97282	0.97008	0.96730	0.96492
data4	0.78133	0.76252	0.75686	0.75373	0.75151	0.75038	0.74964	1.00221	0.97767	0.97123	0.96614	0.96213	0.96011	0.95865
data5	0.78521	0.76799	0.76162	0.75912	0.75842	0.75722	0.75666	1.00075	0.97800	0.96979	0.96557	0.96345	0.96128	0.95974
Average	0.782189	0.766904	0.761465	0.758624	0.756901	0.755518	0.754804	1.002673	0.982297	0.974752	0.970483	0.967445	0.965044	0.963656
Variance	0.000006	0.000023	0.000028	0.000025	0.000026	0.000032	0.000034	0.000005	0.000041	0.000039	0.000042	0.000041	0.000044	0.000047

and ML-1M. UBR has the lowest coverage. This is because highly similar neighboring users may not have much item rating information, leading to many items not being predicted by these so-called neighbors. The same applies to CF-DNC. With an increase in the number of neighbors, the user can acquire more information on items, and the coverage gradually increases. However, this situation rarely occurs for our method that selects the user with the most ratings as the expert user. It therefore remains possible to maintain high coverage even if we have only 10 expert users. It is possible for a small number of domain experts to rate most items in the domain. In the case of UBR, the coverage of the algorithm is low (18.615%) when there are 10 neighbors.

Figure 4.

Coverage of different methods in different dataset.

Table 5

RMSE, coverage, and F-measure with different methods

	ML-100K			ML-1M
Methods	RMSE	Coverage	F-measure	RMSE	Coverage	F-measure
UBR	1.1343	62.97%	0.67024	1.21610	48.44%	0.57119
CF-DNC	1.0633	77.31%	0.75311	1.13120	59.78%	0.65201
EXP	0.9919	99.32%	0.85595	0.94542	99.74%	0.86499
EXP-TST	1.0106	99.66%	0.85415	0.95555	99.90%	0.86400
TDE-CF	0.9607	99.66%	0.86223	0.93816	99.90%	0.86679

Figure 5.

Coverage of different methods in different dataset.

Table 6

MAE with ML-100K dataset

Filler size	Attack size
	UBR				CF-DNC				TDE-CF
	1%	3%	5%	10%	1%	3%	5%	10%	1%	3%	5%	10%
0%	0.8888	0.8888	0.8888	0.8888	0.8440	0.8440	0.8440	0.8440	0.7505	0.7505	0.7505	0.7505
1%	0.8916	0.9025	0.9129	0.9374	0.8453	0.8495	0.8529	0.8594	0.7505	0.7505	0.7505	0.7505
3%	0.8903	0.9018	0.9044	0.9174	0.8450	0.8483	0.8495	0.8530	0.7505	0.7504	0.7504	0.7505
5%	0.8898	0.8940	0.8998	0.9067	0.8454	0.8458	0.8476	0.8496	0.7505	0.7505	0.7505	0.7505
10%	0.8897	0.8903	0.8896	0.8953	0.8440	0.8446	0.8470	0.8469	0.7505	0.7505	0.7504	0.7505
20%	0.8890	0.8883	0.8893	0.8900	0.8440	0.8438	0.8444	0.8453	0.7525	0.7563	0.7610	0.7643

Table 7

MAE with ML-1M dataset

Filler size	Attack size
	UBR				CF-DNC				TDE-CF
	1%	3%	5%	10%	1%	3%	5%	10%	1%	3%	5%	10%
0%	0.9512	0.9512	0.9512	0.9512	0.8747	0.8747	0.8747	0.8747	0.7317	0.7317	0.7317	0.7317
1%	0.9523	0.9555	0.9571	0.9582	0.8747	0.8752	0.8753	0.8756	0.7317	0.7317	0.7317	0.7317
3%	0.9510	0.9510	0.9497	0.9512	0.8747	0.8748	0.8747	0.8750	0.7317	0.7317	0.7317	0.7317
5%	0.9510	0.9511	0.9509	0.9497	0.8747	0.8747	0.8747	0.8747	0.7317	0.7317	0.7317	0.7317
10%	0.9510	0.9511	0.9510	0.9512	0.8747	0.8747	0.8747	0.8747	0.7317	0.7317	0.7317	0.7317
20%	0.9512	0.9511	0.9512	0.9511	0.8747	0.8747	0.8747	0.8747	0.7337	0.7341	0.7346	0.7353

Figure 6.

ML-100K and ML-1M running time.

To give a fair comparison, Table 5 presents the RMSE, coverage, and F-measure achieved by each method with their optimal number of experts for both the 100 k and 1 M datasets. The results show that our TDE-CF method achieves better performance in terms of both precision and coverage.

Figure 6a and b show the running-time results of UBR, CF-DNC, EXP, EXP-TST, and TDE-CF methods for the data sets. Figure 6a presents results for the smaller data set. CF-DNC has the highest time cost because it needs to calculate twice the relationship value between users. EXP and EXP-TST methods have the lowest running times because the EXP method does not need to calculate the transmitted trust or user–domain degree of interest and the EXP-TST method does not need to calculate the user–domain degree of interest. These two methods thus have a running time less than that of TDE-CF. The time complexity increases with the number of users for our EXP, EXP-TST, and TDE-CF methods, but this increase slows with an increasing number of neighbors until the time complexity is almost unchanged. The reason is that the time consumption of the UBR algorithm is mostly in computing the users’ similarity, and selecting the number of neighbors has little effect on the overall time complexity of the algorithm. However, with an increase in the number of expert users, our TDE-CF method requires an additional calculation of trust between users and experts, resulting in a longer running time. Figure 6 presents results for the larger data set. EXP, EXP-TST, and TDE-CF methods perform well in terms of their running time. The two figures reveal that our TDE-CF method performs well on both data sets, but especially on the large data set.

Figure 7.

Comparison of prediction shift with 1% filler size.

Figure 8.

Comparison of prediction shift with 3% filler size.

Figure 9.

Comparison of prediction shift with 5% filler size.

In this section, we have compared our method with traditional CF methods in terms of accuracy, coverage, F-measure, and runtime performance. The TDE-CF method, which combines expert users, trust-awareness, and trust transmission, exhibits a tradeoff between accuracy and execution time, and outperforms traditional methods in terms of coverage.

4.5 Anti-attack capability

In order to evaluate the anti-attack capability of these three recommended algorithms, we injected attack data into the data set (average attack). In order to ensure fairness, we select 70 as neighbour number which have the best performance. Attack size are selected as 1%, 3%, 5%, 10%. Filling size are selected as 1% , 3%, 5%, 10%, 20%. We record MAE value in Tables 6 and 7.

From the Fig. 7a, 8a, 9a, in Movielens 100K dataset, when the filling of scale is constant, the MAE of the CF algorithm and the CF-DNC algorithm show an increasing trend with the increase of the attack scale. It can be seen that in the CF algorithm and CF-DNC algorithm, the performance of the recommendation system will be affected by the attacked users. The more attack users, the worse result. And under the same filling scale and attacking scale, our TDE algorithm is better than CF and DNC algorithm, our TDE-CF algorithm is almost unaffected by this attack. Only when there are a large number filling scale in dataset, the MAE value will become worse. This may be caused by these attack users who replace the expert users. As for Movielens 1M dataset, like Fig. 7b, 8b, 9b show, the situation are similar to 100 K dataset.

5. Conclusion

With the widespread application of CF recommendation algorithms, improving the accuracy, real-time performance, and coverage has become a very important research issue. Due to traditional algorithm in trust calculation will bring the rough particle of trust and the higher computation in global trust-relationship, we proposed an algorithm which combining trust-aware and domain expert recommendation to reduce the impact of these problems. According to our theoretical and experiment analysis. Its show that the algorithm not only improves the accuracy of recommendation, but also reduce time consuming. However, because of narrowed down the trust into range of domain, the algorithm will not be able to provide a novel recommendations, and when the dataset are not explicitly give the relationship between the item and domain, the algorithm will be powerless. Next, we will focus on better expert user selection algorithm and solving the impact of dataset relationship issues.

Footnotes

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant 61572204, the Project of Science and Technology Plan of Fujian Province of China (No. 2017H01010065) and Science and Technology Planning Project of Quanzhou (Grant No. 2017G019).

References

Asabere

N.Y.

Acakpovi

and Michael

, Improving socially-aware recommendation accuracy through personality, IEEE Transactions on Affective Computing, 2017.

Bobadilla

Ortega

Hernando

and Bernal

, A collaborative filtering approach to mitigate the new user cold start problem, Knowledge-Based Systems26 (2012), 225–238.

Bobadilla

Ortega

Hernando

and GutiÃ©rrez

, Recommender systems survey, Knowledge-Based Systems46 (2013), 109–132.

Cacheda

Carneiro

Fernández

and Formoso

, Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems, ACM Trans Web5(1) (Feb. 2011), 2:1–2:33.

Cai

Leung

H.-F.

Min

Tang

and Li

, Typicality-based collaborative filtering recommendation, IEEE Trans. on Knowl. and Data Eng.26(3) (Mar. 2014), 766–779.

Deng

Huang

and Wu

, On deep learning for trust-aware recommendations in social networks, IEEE Transactions on Neural Networks & Learning Systems28 (2017), 1–14.

Dongyan

and Fuzhi

, A collaborative filtering recommendation algorithm based on double neighbor choosing strategy, Journal of Computer Research and Development5 (2013), 020.

Golbeck

J.A.

, Computing and Applying Trust in Web-based Social Networks, PhD thesis, College Park, MD, USA, 2005. AAI3178583.

Gunawardana

and Shani

, A survey of accuracy evaluation metrics of recommendation tasks, Journal of Machine Learning Research10 (Dec 2009), 2935–2962.

10.

Hema

and SowrirajaPillai

M.N.

, Efficient mining and recommendation of sparse data through collaborative filtering technique in medical transcriptions, In ICICES2014 – S.A. Engineering College, Chennai, Tamil Nadu, India, 2014.

11.

Jamali

and Ester

, Trustwalker: a random walk model for combining trust-based and item-based recommendation, In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, 2009, pp. 397–406.

12.

Zhu

and Zhang

, Rating knowledge sharing in cross-domain collaborative filtering,IEEE Transactions on Cybernetics45 (2014).

13.

Liang

and Ma

, A collaborative filtering recommendation algorithm based on domain nearest neighbor, Journal of Computer Research and Development45(9) (2008), 1532.

14.

Liu

Xia

Chen

Asabere

N.Y.

and Huang

, Trucom: Exploiting domain-specific trust networks for multicategory item recommendation, IEEE Systems Journal11 (2017), 295–304.

15.

Liu

Wang

and Ye

, A recommendation system algorithm based on large scale internet environment, In 2016 13th Web Information Systems and Applications Conference, 2016.

16.

Luo

OuYang

Xiong

and Yuan

, The effect of similarity support in k-nearest-neighborhood based collaborative filtering, Chinese Journal of Computer2 (2010), 5.

17.

Luo

Zhou

M.C.

Leung

Xia

Zhu

You

and Li

, An incremental-and-static-combined scheme for matrix-factorization-based collaborative filtering, IEEE Transactions on Automation Science & Engineering13(1) (2016), 333–343.

18.

Massa

and Avesani

, Trust-aware collaborative filtering for recommender systems, In OTM Confederated International Conferences “On the Move to Meaningful Internet Systems”, Springer, 2004, pp 492–508.

19.

Massa

and Avesani

, Trust-aware recommender systems, In Proceedings of the 2007 ACM conference on Recommender systems, ACM, 2007, pp. 17–24.

20.

Massa

and Avesani

, Trust metrics on controversial users, International Journal on Semantic Web & Information Systems3(1) (2009), 39–64.

21.

Massa

and Bhattacharjee

, Using trust in recommender systems: an experimental analysis, In International Conference on Trust Management, Springer, 2004, pp. 221–235.

22.

Meng

Liu

Zhang

and Hu

, Research on social recommender systems, Journal of Software26(6) (2015), 1356–1372.

23.

Middleton

S.E.

Shadbolt

N.R.

and De Roure

D.C.

, Ontological user profiling in recommender systems, ACM Trans. Inf. Syst.22(1) (Jan 2004), 54–88.

24.

Mustafa

Ibrahim

A.O.

Ahmed

and Abdullah

, Collaborative filtering: Techniques and applications, In 2017 International Conference on Communication, Control, Computing and Electronics Engineering (ICCCCEE), Khartoum, Sudan, 2017.

25.

O’Donovan

and Smyth

, Trust in recommender systems, In International Conference on Intelligent User Interfaces, January 10–13, 2005, San Diego, California, Usa, 2005, pp 167–174.

26.

Pazzani

M.J.

, A framework for collaborative, content-based and demographic filtering, Artificial intelligence review13(5-6) (1999), 393–408.

27.

Qiao

Yang

and Chen

, A trust calculating algorithm based on social networking service users’ context, Chinese Journal of Computers12 (2011), 015.

28.

Ray

and Mahanti

, Strategies for effective shilling attacks against recommender systems, In Lecture Notes in Computer Science, volume 5456, 2009, pp 111–125.

29.

Ren

Zhu

and Zhang

, A new collaborative filtering technique to improve recommendation diversity, In 2016 2nd IEEE International Conference on Computer and Communications, 2016.

30.

Resnick

Iacovou

Suchak

Bergstrom

and Riedl

, Grouplens: An open architecture for collaborative filtering of netnews, In Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, CSCW ’94, New York, NY, USA, 1994. ACM, pp. 175–186.

31.

Sarwar

Karypis

Konstan

and Riedl

, Item-based collaborative filtering recommendation algorithms, In Proceedings of the 10th International Conference on World Wide Web, WWW ’01, New York, NY, USA, 2001. ACM, pp 285–295.

32.

Shani

and Gunawardana

, Evaluating recommendation systems, Recommender Systems Handbook, 2011, pp. 257–297.

33.

Song

, A collaborative filtering recommendation algorithm based on multi-dimensional da ta filling, In 2016 2nd IEEE International Conference on Computer and Communications, 2016.

34.

Wang

and Xu

, Trust-aware recommender system based on expert user, Computer Science, 2011.

35.

Wang

and Yin

, Real-time personalized recommendation based on implicit user feedback data stream, Chinese Journal of Computers39 (2016).

36.

Wei

Chen

Zhou

and Tang

, Collaborative filtering and deep learning based hybrid recommendation for cold start problem, In 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress, 2016.

37.

Zhang

Wang

Liu

and Kong

, Exploiting trust and usage context for cross-domain recommendation,IEEE Access4 (2016), 2398–2407.

38.

Yang

Hao

Min

Kim

and Yau

, An efficient approach to generating location-sensitive recommendations in ad-hoc social network environments, IEEE Transactions on Services Computing8(3) (2015), 520–533.

39.

Yang

Guo

Liu

and Steck

, A survey of collaborative filtering based social recommender systems, Computer Communications41 (2014), 1–10.

40.

Yang

Liang

and Zhao

, Collaborative filtering-based recommendation of online social voting, IEEE Transactions on Computational Social Systems (2017).

41.

Yang

and Zheng

, A survey of collaborative filtering-based recommender systems for mobile internet applications, IEEE Access, 2016.

42.

Yao

Sheng

Q.Z.

Ngu

A.H.H.

and Segev

, Unified collaborative and content-based web service recommendation, IEEE Transactions on Services Computing8(3) (2015), 453–466.

43.

Yongping

D.U.

Xiaoyan

D.U.

and Huang

, Improve the collaborative filtering recommender system performance by trust network construction, Chinese Journal of Electronics25(3) (2016), 418–423.

44.

Zhang

and Xu

, Study on the trust evaluation approach based on cloud model, Chinese Journal of Computers36(2) (2013), 422–431.

45.

Zheng

Zhu

and Yao

, Recommendation rating predicition based on attribute boosting with partial sampling, Chinese Journal of Computers (2016).

Collaborative filtering recommendation system based on trust-aware and domain experts

Abstract

Keywords

1. Introduction

2. Related work

Table 1 Notation and symbols used in this paper

3.1 Recommendation based on trust aware and domain expert description

Table 2 Comparison of methods

4.1 Data set introduction

4.2 Evaluation metrics

4.4 Experimental results and analysis

Table 3 MAE and RMSE Performance with respect to ε

5. Conclusion

Footnotes

Acknowledgments

References

Table 1
Notation and symbols used in this paper

Table 2
Comparison of methods

Table 3
MAE and RMSE Performance with respect to $\varepsilon$