Big data and intelligent software systems

Abstract

Web growth, especially in social networks, is continuously increasing every day. Multiplicity of products offered and web pages has made picking up relevant items a tedious job. On the other hand, different tastes and behaviors of users is creating the probability to find a similar user among a large group of users difficult. As a result, automated software systems have difficulty to discover what is interesting to users.

We have proposed a new approach to adapt to this flow. We will exploit domain knowledge of training data set to create a summary matrix. The summary matrix consists of new and few columns according to the attribute values of the selected feature. We fill the summary matrix with the average ratings based on the number of times that the attribute values appear in the user’s profile for rated items.

We use the summary matrix in two hybrid recommender systems. In our approach, we use meta-level technique which is one of the pipelined hybridization techniques.

The proposed approach will reduce the effects of sparsity, cold start, and scalability which are common problems with the collaborative recommender systems. Furthermore, the proposed approach will improve the recommendation accuracy when there is comparison with the Collaborative Filtering Pearson Correlation approach and it will be faster as well.

Keywords

Big data recommender systems feature engineering hybrid recommender systems meta-level Collaborative Filtering Content-Based Filtering sparsity cold start scalability

1. Introduction

Due to the ubiquity of e-commerce, recommender systems have become an exciting area to work on recently. There are many different recommender systems. However, the researchers have yet to create and develop algorithms to reach satisfactory results for users. Often, users do not have a clear idea about what items are good for them because the qualitative boom is accompanied by an increase in high-resolution throughput data. Big data or large-scale data are the output of the qualitative boom in computing, communications, and digital storage technologies. Big data reflects to a data set that is growing rapidly, because of the spread of digital computers, mobiles, and the growth of the Internet.

Digital information storage capacity doubles every 40 months, roughly since the 1980s [45]. The storage capacity reached to 2.5 Exabyte (the sixth power of 1000 bytes) every day in 2015 [46], and it nearly reached to 3 Exabyte in 2016 [46]. The Cisco forecasts indicate a steady increase in the storage capacity of the coming years [46].

Big data faces several challenges such as storage, transfer, visualization, querying, and updating. These challenges require more predictive analytics, user behavior analytics, or other advanced data analytics methods to discover a useful pattern [47].

Big data is characterized through the quantity and quality of generated and stored data [48, 49]. Increase of the number of users is directly proportional to the increase in the amount of data.

Figure 2 shows the increase of Internet users over the past decades.

Figure 1.

Cisco forecasts of data growth [46].

Figure 2.

Internet users per 100 inhabitants [50].

Recommender systems are popular intelligent software systems that are applied in various domains such as in movies, music, books, jokes, restaurants, financial services [8], and Twitter followers [9], and recommends interesting items to users [4, 6, 7, 10, 11, 13]. These personalized suggestions are a useful alternative to searching algorithms.

Recommender systems are personalized information agents that have become interesting in recent years. It applies in the domains of academia and industry increasingly. Recommender systems are a subclass of software information filtering systems, which analyzes user profiles to predict what the user preference is.

Recommender systems that incorporate data mining techniques get its recommendations by using knowledge learned from actions and attribute values of users and items. Recommender systems are based on previous information about interaction of the users with items to get the recommendations [21]. The past user concerns determine the user future choices.

Recommender systems rely on discovering the historical profiles of users. These profiles include information such as rates, item features, tags, and shared files. This profile is compared with other users. It can be distinguished from other information retrieval systems by semantics and systematic analysis to user interactively. Recommendations resulting from recommender systems are interpreted as responding to a user’s query at information retrieval systems, therefore the recommender systems can be seen as an information agent [41, 42, 43, 44].

There are four techniques of recommender systems: collaborative, content-based, knowledge-based, and demographic [16]. Two main categories are most popular: content-based and collaborative recommender systems [1, 10]. Most recommender systems that apply hybrid recommender systems is a combination of content-based and collaborative recommender systems.

Content-Based Filtering approaches are based on a description of item features and user preferences in his/her profile [3, 14, 15, 21]. It recommends items similar to the same type of items that a user already liked. Content-Based Filtering may be defined as an algorithm of searching and comparing therefore it is similar to processes that are used in information retrieval systems, but without needing user queries. Content-Based Filtering obtains the information from two knowledge sources: item features and its rating that is given by users.

Collaborative Filtering generates the recommendations based only on the past users database of ratings that represents full information about users’ past rates. Collaborative Filtering predicts preferable items for users by calculating the similarity scores between users. These scores will be interpreted according to the used algorithms.

Nevertheless, Collaborative Filtering often suffers from three common problems: sparsity, cold start, and scalability

1.1 Scalability

In many of environments, we need much time to find a similar neighbor when we use Collaborative Filtering. Because, data sets contain millions of users and items. Further, the number of users and items are increasing, so it becomes computationally difficult to find similar neighbors. This increasing in the number of users and items is called scalability problem.

1.2 Sparsity

Mostly, users don’t rate items. Even popular items that user liked or bought still unrated. Because of increasing number of users and items with few ratings, most entries of data sets remain zero. This situation is called sparsity problem. The level of sparsity is determined by the ratio of the number of zeros to the total number of matrix.

1.3 Cold start

We can consider the cold start problem as a special case of the sparsity problem [12]. The cold start problem happens because the user doesn’t have enough rating or any rating at all. To avoid this problem, some companies offer to the consumers some of popular items to evaluate it when they login to the company’s accounts at first time. Otherwise, it is difficult for recommender systems to provide an accurate recommendation to users.

In this paper, we propose summary matrix through exploits the domain knowledge of the data set to create new features, which called feature engineering [17, 18, 24, 27, 28]. The feature is a piece of information in the data set. This piece might contain many attribute values which are useful for prediction and will influence recommendations.

Purpose of create summary matrix to reduce the effects of sparsity, cold start, and scalability problems and improving the recommendation accuracy. We fill the summary matrix with the average ratings based on attribute values of the selected feature. Then, we will apply two approaches of hybrid recommender systems to get the recommendation.

2. Related work

In this Section we review some examples of hybrid recommender systems that are applied in various domains. Netflix Inc. [26] for movie recommendations combines collaborative and content-based filtering through similar habits of users and higher rates of shared movies characteristics. Netflix Inc. released a challenge in 2006 and offered a grand prize of one million US dollars to enhance the recommender system of the company [26]. The person or team who could successfully decrease RMSE for data set by 10 percent, would win [1, 2, 5]. Bellkor’s Pragmatic Chaos team succeeded in achieving an RMSE of 0.8554 with a 10.06% improvement over the Netflix Inc. system [2].

Lawrence et al. [20] described a personalized recommender system to shoppers in supermarkets. This recommender system relies on shoppers’ previous behavior towards purchases to suggest new products for them. The IBM researchers developed this recommender system to implement it as a part of SmartPad which was developed as a personal digital assistant for remote shopping.

Vaz et al. [22] presented a hybrid book recommender systems based on Collaborative Filtering and author’s rankings by users. This hybrid recommender system improves book recommendations through sending proposals for book readers to decide which book to read next.

MovieLens data set [31] is the online movie recommendations data set that we used in our approach. MovieLens proposes some of the most popular movies to new users to evaluate it. These ratings are exploited to recommend other movies to the user. In addition, MovieLens uses Collaborative Filtering based on these ratings to create personalized recommendations.

We can apply several techniques in the same recommender systems to get the recommendations. For example, two different Content-Based Filtering could work together in hybrid recommender systems such as News Dude. News Dude uses both Naive Bayes and K-Nearest Neighbor classifiers in its news recommendations [16].

Consequently, hybrid recommender systems become increasingly interesting for researchers. Theoretical work focused on how to hybridize the algorithms and which situations can expect to benefit from hybridization [1]. Hybrid recommender systems represent the door to improving the recommendations, overcome some of the problems, and improve the performance of algorithms.

Our approach tries to reduce these above-mentioned problems. Many researchers over the past several years have come up with different solutions to resolve scalability, cold start, and sparsity problems. These problems are inherent in collaborative recommender systems. Reducing the data set dimensionality is one solution approaches. Sarwar et al. [33] applied singular value decomposition for matrix factorization that provides lowest rank approximations of the original matrix. Singular value decomposition expresses the matrix as the product of three “simple” matrices, which result in the singular values in decreasing order.

Lu et al. [35] proposed a Confidence Weighted Online Collaborative Filtering (CWOCF) approach. The key idea of the CWOCF approach is to follow the low-rank matrix factorization and exploit confidence weighted classification in optimizing the low-rank matrixes. The CWOCF approach will update the distributions of matrix factorization vectors.

Figure 3.

Meta-level technique [16].

Chen et al. [36] proposed to compute the similarity matrix based on relative distance between user ratings to solve the sparsity problem in recommender systems.

Moghaddam and Selamat [38] proposed a clustering method to solve scalability problem. This method is a hybrid recommender system, which comprises of users’ demographic information and Collaborative Filtering.

Cantador et al. [37] proposed a hybrid recommendation model which combines Content-Based and Collaborative Filtering according to relations among users. The proposed approach is based on clusters that are used to find similarities among individuals at multiple semantic layers.

Smith et al. [29] proposed a latent neural network (LNN) with latent input variables as a hybrid collaborative filtering technique. LNN is a hybrid recommendation algorithm that leverages the advantages of collaborative filtering and content based filtering to achieve much lower error when recommending previously unrated items also to addressing the cold-start problem.

3. Methodology

In this Section, we introduce the outline of the research concerning the meta-level technique, data sets, feature learning and computing the summary matrix.

3.1 Meta-level technique

Hybrid recommender systems are defined as a combination of various knowledge sources and different techniques together to obtain the outputs. Knowledge sources consist of user profiles, community data, and item features. Hybrid recommender systems can be divided into three different major categories (monolithic hybridization, parallel hybridization, and pipelined hybridization) with seven hybridization techniques [1].

Meta-level technique is one of the seven hybridization recommendation techniques in the pipelined hybridization design category. Meta-level technique makes the outputs of previous recommender system become inputs of subsequent one and the final system produces recommendations for users. As a result, the contributing recommender completely replaces raw data with the learned models and resulting data is used as input in the calculation of the actual recommender.

Figure 4.

Overview of the summary matrix.

3.2 Data sets used

In this Section, we introduce each data set that we used in our approach. As well as, we describe some basic statistics of the training data sets. The two data sets that used in this study are available to download from the GroupLens Research website [30].

3.2.1 MovieLens 1M data set

GroupLens Research collected rating data sets from the MovieLens website [31]. The data sets were collected over various periods of time. The rating values range between 0.5 and 5. The data set consist of around 6,040 users and 3,883 items.

3.2.2 HetRec 2011 data set

The 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011) [32] released data set from Delicious, Last.fm Web 2.0, MovieLens, IMDb, and Rotten Tomatoes. This data set contains social networking, tagging, and individual information from sets of around 2,113 users. The rating values range between 0.5 and 5. The data set consist of around 2,113 users and 10,197 items.

Table 1
Statistics of training data sets

Statistics	HetRec	MovieLens
Number of users	2113	6040
Number of items	10197	3883
Number of ratings	515000	598209
Average number of ratings by users	243.73	99.04
Average number of ratings for items	50.5	154.06
Density	2.4%	2.55%

In Table 1, the statistics of training data sets: HetRec 2011 and MovieLens 1M are listed. Average number of users who gave ratings for items and average number of items that were rated by users can be seen in Table 1.

3.3 Feature learning and computing the summary matrix

In machine learning, feature learning (or representation learning) is a set of techniques that learns features [19, 23]. The new representation should make machine learning algorithms simpler and more flexible.

The data set in our paper consists of two major categories: users and items (movies). Each one of the data set contains many features which include many attribute values. For example, user’s category contains gender, occupation, age and ZIP code, item’s category contains title, genres, actors, and year of release. Gender feature contains two attribute values: male and female. Genre feature contains many attribute values such as action, comedy, and drama.

Feature creation is a process to generate new features based on existing attribute values. For example, say, we have genre (action, comedy, crime, romance) as an input values in a data set. We can generate new features like action, comedy, crime, and romance that may have a better relationship. This step is used to highlight the hidden relationship in the attribute values.

Feature engineering is the science of extracting more information from existing data [18]. We are not adding any new data here, but we are making the data we already have more useful. There are various techniques to create new features, as is done in [18]. The summary matrix is based on selected feature for movie data set, in our approach the genre feature is good.

An illustration of obtaining the summary matrix and the techniques that we will apply on this matrix to get the recommendations is given in Fig. 4. It can also be described as follows:

•
Extract all attribute values of the selected feature.
•
Extract the attribute values of the selected feature without repetition.
•
Create the summary matrix with new columns based on attribute values of the selected feature.
•
Fill the summary matrix with the average ratings based on attribute values of the selected feature.
•
Compute similarity scores between users in the summary matrix by using Collaborative Filtering, as in Eq. (1).

$\displaystyle\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!S(i,j)=\frac{\sum_{y\in Y}\left% (r_{y,i}\!-\!\overline{r}_{i}\right)\!\times\!\left(r_{y,j}\!-\!\overline{r}_{% j}\right)}{\sqrt{\sum\limits_{y\in Y}\!\left(r_{y,i}\!-\!\overline{r}_{i}% \right)^{2}}\!\times\!\sqrt{\sum\limits_{y\in Y}\!\left(r_{y,j}\!-\!\overline{% r}_{j}\right)^{2}}}$ (1)
•
Get the recommended item by using Eq. (2).

$\displaystyle r_{i,n}=\frac{\sum_{j\in K}S(i,j)\times r_{j,n}}{\sum_{j\in K}S(% i,j)}$ (2)
•
Get top $K$ similar users ( $K=$ 80). These users will be the candidates to Collaborative Filtering approach. The similarity measure used is the Euclidean distance, as in Eq. (3).

$\displaystyle d(i,j)=\sqrt{\sum_{L=1}^{y}(r_{L,i}-r_{L,j})^{2}}$ (3)

Below we explain our feature learning method. In this method we learn a “summary matrix” that has average rating values for a user on attribute values of a selected column. For this work we selected the genre column as an example.

Let $S(i)$ denotes training sample item $i$ , then $S(i)$ can be represented as:

$\displaystyle S(i)=\left\{\overrightarrow{{\bm{I}}}({\bm{i}}),\overrightarrow{% {\bm{V}}}({\bm{i}}),\overrightarrow{{\bm{D}}}({\bm{i}})\right\}$ (4)

Where, $\overrightarrow{{\bm{I}}}({\bm{i}})$ , $\overrightarrow{{\bm{V}}}({\bm{i}})$ and $\overrightarrow{{\bm{D}}}({\bm{i}})$ stand for the input vector and the two output vectors for training sample item $i$ , respectively. $\overrightarrow{{\bm{I}}}({\bm{i}})$ represents all item features ( $i$ ), whose structure can be shown as:

$\displaystyle\overrightarrow{{\bm{I}}}({\bm{i}})=\left\{\begin{array}[]{l}% \textit{Title }(i)\\ \textit{Year }(i)\\ \textit{Genre }(i)\\ \textit{Location }(i)\\ \textit{Director }(i)\\ \textit{Actors }(i)\\ \textit{Country }(i)\end{array}\right.$ (5)

All entries are either textual or integers. Genre ( $i$ ) is represented as the genre feature ( $i$ ) that will be extract from other item features, and it is textual.

Likewise, $\overrightarrow{{\bm{V}}}({\bm{i}})$ represents all the attribute values of the selected feature, which can be shown as:

$\displaystyle\overrightarrow{{\bm{V}}}({\bm{i}})=\left\{\begin{array}[]{l}% \text{Adventure, Children, Fantasy}\\ \text{Comedy, Romance}\\ \text{Comedy}\\ \text{Action, Crime, Thriller}\\ \text{Adventure, Children, Action}\\ \text{Comedy}\\ \text{Adventure, Children, Action}\end{array}\right.$ (6)

Likewise, $\overrightarrow{{\bm{D}}}({\bm{i}})$ represents unrepeated attribute values of the selected feature that will be the new columns of the summary matrix, which can be shown as:

$\displaystyle\overrightarrow{{\bm{D}}}({\bm{i}})=\left\{\begin{array}[]{l}% \text{Adventure, Children, Fantasy}\\ \text{Comedy, Romance}\\ \text{Comedy}\\ \text{Action, Crime, Thriller}\\ \text{Adventure, Children, Action}\end{array}\right.$ (7)

We can define $W(i,j)$ as the average ratings based on Eqs (3) and (4) and TF (explained next paragraph), for each user’s items in the summary matrix. $i$ represents the users, $j$ represents the items in the summary matrix. Then $W(i,j)$ can be obtained as:

$\displaystyle W(i,j)=\frac{\sum_{\overrightarrow{{\bm{V}}}\in\overrightarrow{{% \bm{D}}}}r_{i}}{\textit{TF}}$ (8)

Table 2
Statistics of the summary matrix

Statistics HetRec MovieLens

Number of users 2113 6040

Number of items 788 301

Number of ratings 218293 252394

Average number of ratings by users 103.31 41.79

Average number of ratings for items 277.02 838.52

Density 13.1% 13.9%

Algorithm 1 Building the summary matrix

1: input:

2: $\vec{v}\leftarrow<v_{1},\ldots,v_{L}>$ // $\vec{v}$ is the vector of attribute values.

3: $\vec{d}\leftarrow<d_{1},\ldots,d_{K}>$ // $\vec{d}$ is the vector of unrepeated attribute values.

4: for $i=1:K$

5: for $j=1:K$

6: if $v_{j}\in d_{i}$

7: $u\leftarrow+r$ // $r$ is the rating values of the data set.

8: $\textit{TF}\leftarrow\textit{TF}+1$

9: end

10: end

11: $W=u/\textit{TF}$ // average ratings for each item of users in the summary matrix.

12: end

Term Frequency (TF) denotes the number of times that the attribute values of the selected feature $\overrightarrow{{\bm{D}}}$ appears in user’s profile for rated items.

The summary matrix will be filled with average ratings for items that are rated by a user in the data set. The summary matrix consists of the same number of users (rows) in the data set, but new items (columns).

In Table 2, the statistics of the summary matrix after implementing Algorithm 1 are listed. The number of items in the summary matrix is reduced. Therefore, the rating density is increased, which contributes to solve the problems: scalability, sparsity and cold start.

Figure 5.
Amount of decrease in the items.

Figure 6.
The rating density.

Figure 7.
General schematic of techniques that we used.

The results obtained through creating the summary matrix can be summarized as follows:

•
Decreasing the number of items.
•
Increasing the ratings matrix density.
•
Increasing the ratings of users.
•
Increasing the ratings of items.

Now, we have two important questions will be provable in the next Section:

•
How useful of reducing the items?
•
Can the proposed approach improve the recommendation accuracy?

Table 3
Time-consuming

Data sets HetRec 2011 MovieLens 1M

Methods CFP HRS-1 HRS-2 CFP HRS-1 HRS-2

Preparing matrix 27.31 2.034 2.034 31.858 2.195 2.195

Calculation of the similarity scores 1.95 0.151 0.231 2.333 0.243 0.405

List of recommendations 0.015 0.014 0.012 0.03 0.012 0.01

Average number of time-consuming for testing one sample (s) 0.568 0.074 0.379 0.83 0.151 0.716

4. Experiments

Algorithm 1 Building the summary matrix
1:	input:
2:	$\vec{v}\leftarrow<v_{1},\ldots,v_{L}>$ // $\vec{v}$ is the vector of attribute values.
3:	$\vec{d}\leftarrow<d_{1},\ldots,d_{K}>$ // $\vec{d}$ is the vector of unrepeated attribute values.
4:	for $i=1:K$
5:	for $j=1:K$
6:	if $v_{j}\in d_{i}$
7:	$u\leftarrow+r$ // $r$ is the rating values of the data set.
8:	$\textit{TF}\leftarrow\textit{TF}+1$
9:	end
10:	end
11:	$W=u/\textit{TF}$ // average ratings for each item of users in the summary matrix.
12:	end

Data sets	HetRec 2011	MovieLens 1M
Preparing matrix	27.31	2.034	2.034	31.858	2.195	2.195
Calculation of the similarity scores	1.95	0.151	0.231	2.333	0.243	0.405
List of recommendations	0.015	0.014	0.012	0.03	0.012	0.01
Average number of time-consuming for testing one sample (s)	0.568	0.074	0.379	0.83	0.151	0.716

In this Section, we re-predict the ratings of a testing data set. Following this, we will review the findings of comparing two techniques of hybrid recommender systems based on the summary matrix with the Collaborative Filtering Pearson Correlation approach based on a training data set. Each technique has a different pattern, which makes it vary in the strengths and drawbacks. Therefore, each technique has characteristic results.

4.1 Overview

Recommender systems have been evaluated in many different evaluation metrics over the past several years [1, 25, 34]. Recommender systems evaluation is difficult because the evaluation results are mutable, it is based on algorithms, data sets, and evaluation metrics together. Evaluation metrics are divided into two major categories according to desired recommendations results. The first category is based on numeric value (i.e. error ratio) that represents the difference of original rate and predicted rate, and is called predictive accuracy metrics. The second category is based on relevance (i.e. separating the range of rating into two groups) that represents the relevant or irrelevant relation between original rate and predicted rate, and is called classification accuracy metrics. There is motivation to use both types of evaluation metrics in this thesis because every category follows a certain pattern for evaluation.

4.2 Data sets and preprocessing

The summary matrix is created by implementing Algorithm 1 on two data sets MovieLens 1M and HetRec 2011, as we mentioned in Section 3. The purpose of creating the summary matrix is to improve the performance and get accurate recommendations.

We propose two techniques of hybrid recommender systems according to the summary matrix. Each one has advantages different from the other because the first technique combines two techniques and another consists of three techniques.

HRS-1 denotes combining summary matrix and Collaborative Filtering Pearson Correlation approach.

HRS-2 denotes combining summary matrix, K-Nearest User, and Collaborative Filtering Pearson Correlation approach.

CFP denotes to Collaborative Filtering Pearson Correlation approach.

Table 3 shows the advantage of reducing items, through reducing time-consuming for testing one sample (in second) to predict the rate.

4.3 Evaluation metrics

We applied five evaluation metrics belonging to two categories. It would be better to choose one or more evaluation metrics to compare the accuracy of different recommender systems [25].

4.3.1 Predictive accuracy metrics

Predictive accuracy metrics are based on numerical differences between predicted ratings and true ratings that users give to the movies. The rating is estimated by five-stars in the selected data set: HetRec 2011 and MovieLens 1M.

Recommender systems evaluation relies on how close predicted ratings are to true ratings. The recommender system is considered successful if the difference between the numerical values is small or vice-versa.

There are many evaluation metrics for evaluating the ability of recommender systems to correctly predict a specific item. Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are two of the most important evaluation metrics [1]. These predictive accuracy metrics are used for recommender systems evaluation because it is easy to calculate and understand.

Figure 8.

Evaluations of predictive accuracy metrics.

Figure 9.

MAE for CFP, HRS-1, and HRS-2.

MAE Eq. (9) measures the average absolute deviation between predicted rating and true rating. RMSE Eq. (10) represents the sample standard deviation of the differences between predicted rating and true rating.

$\displaystyle\textit{MAE}=\frac{\sum_{i=1}^{T}|p_{i}-r_{i}|}{T}$ (9) $\displaystyle\textit{RMSE}=\sqrt{\frac{\sum_{i=1}^{T}|p_{i}-r_{i}|^{2}}{T}}$ (10)

Where $p_{i}$ and $r_{i}$ represent the predicted ratings and the real ratings of users, respectively. $T$ denotes to the total number of predictions generated for all active users in the data set.

RMSE metric used as a condition to determine the winner in the competition of Netflix Inc. [26]. The condition was to improve the results of RMSE metrics of a proposed algorithm 10% compared with the Netflix Inc. algorithm which is called Cinematch.

Table 4

MAE and RMSE evaluations

Data sets	HetRec 2011			MovieLens 1M
Methods	CFP	HRS-1	HRS-2	CFP	HRS-1	HRS-2
MAE	0.67	0.63	0.64	0.77	0.733	0.743
RMSE	0.87	0.823	0.825	0.97	0.927	0.93

Table 4 and Fig. 8 show the results of MAE and RMSE evaluation metrics of HRS-1 and HRS-2 in comparison with CFP.

The performance superiority of HRS-1, can be seen in Fig. 8.

In Fig. 9, MAE for CFP, HRS-1, and HRS-2 with 40%, …, 90% of the training data set is illustrated.

When we have 90% of the available ratings, the performance superiority of HRS-2, can be seen in Fig. 9.

In Fig. 10, RMSE for CFP, HRS-1, and HRS-2 with 40%, …, 90% of the training data set is illustrated.

When we have 80% of the available ratings, the performance superiority of HRS-2, can be seen in Fig. 10.

Figure 10.

RMSE for CFP, HRS-1, and HRS-2.

Figure 11.

Evaluations of classification accuracy metrics.

4.3.2 Classification accuracy metrics

Classification accuracy metrics are based on relevance between predicted ratings and true ratings to determine which items are relevant (i.e. good) and which items are irrelevant (i.e. bad). We will separate the data set into two classes depending on a threshold. All ratings of 0.5 to less than 3 “irrelevant” and 3–5 “relevant”.

We can classify each recommendation such as [40]:

•
True positive, an acceptable item recommended to user.
•
True negative, an unacceptable item not recommended to user.
•
False positive, an unacceptable item recommended to user.
•
False negative, an acceptable item not recommended to user.

Precision and recall are the most popular metrics in the information retrieval field and depend on separation between relevant “positive” and irrelevant “negative” items. Precision and recall are used in [33, 39]. F-measure allows for combining precision and recall into a single score.

Precision Eq. (11) is defined as the ratio of relevant items recommended to a number of items recommended. Precision represents the probability that a recommended item is relevant.

$\displaystyle\textit{Precision}=\frac{\textit{TP}}{\textit{TP}+\textit{FP}}$ (11)

Figure 12.
Precision for CFP, HRS-1, and HRS-2.

Figure 13.
Recall for CFP, HRS-1, and HRS-2.

Recall Eq. (12), defines as the ratio of relevant items recommended to total number of relevant items. Recall represents the probability that a relevant item will recommend.

$\displaystyle\textit{Recall}=\frac{\textit{TP}}{\textit{TP}+\textit{FN}}$ (12)

F-measure Eq. (13), defines as average number of the precision and recall. F-measure represents the balance between precision and recall.

$\displaystyle F=\frac{2\cdot\textit{Precision }\cdot\textit{Recall}}{\textit{% Precision}+\textit{Recall}}$ (13)

Table 5
Precision, recall, and F-measure evaluations

Data sets HetRec 2011 MovieLens 1M

Methods CFP HRS-1 HRS-2 CFP HRS-1 HRS-2

Precision 0.865 0.868 0.839 0.893 0.91 0.88

Recall 0.868 0.88 0.89 0.892 0.9 0.91

F-measure 0.866 0.876 0.864 0.892 0.9 0.89

Table 5 and Fig. 11 show the results of precision, recall, and F-measure evaluation metrics of HRS-1 and HRS-2 comparing with CFP.

The performance superiority of HRS-1 in precision, recall, and F-measure and HRS-2 in recall, can be seen in Fig. 11.

In Fig. 12, precision for CFP, HRS-1, and HRS-2 with 40%, …, 90% of the training data set is illustrated.

The performance superiority of HRS-1 with 40%, …, 90% of the available ratings, can be seen in Fig. 12.

In Fig. 13, recall for CFP, HRS-1, and HRS-2 with 40%, …, 90% of the training data set is illustrated.

Table 6
Comparison according to CWOCF approach

Data sets Predictive Techniques

accuracy metrics CWOCF HRS-1 HRS-2

HetRec 2011 MAE 0.6499 0.63 0.64

RMSE 0.8473 0.823 0.825

MovieLens 1M MAE 0.7609 0.733 0.743

RMSE 0.9580 0.927 0.93

Table 7
Comparison according to LNN approach

Alg 106 2070 2470 2509 2676 2678 3430 3462 614 717 Ave

CBCF 1.248 0.841 0.733 0.939 0.875 1.029 0.880 1.124 1.030 0.681 0.938

CBF 1.221 0.868 0.862 1.003 1.028 1.045 0.949 1.062 1.042 0.46 0.954

LNN 1.247 0.715 0.864 0.964 1.004 0.972 0.758 0.935 1.174 0.125 0.876

LNN3PT 1.250 0.729 0.867 0.833 1.197 1.060 0.791 0.952 1.222 0.000 0.890

HRS-1 1.099 0.705 0.592 0.849 0.774 0.792 0.764 0.699 0.754 0.420 0.751

HRS-2 1.099 0.709 0.618 0.849 0.783 0.792 0.769 0.683 0.754 0.420 0.754

Table 8
Percentage improvement of all results obtained

Data sets HetRec 2011 MovieLens 1M

Methods HRS-1 HRS-2 HRS-1 HRS-2

Performance Time-consuming for one sample testing (s) 87% 33.3% 82% 14%

Predictive accuracy metrics MAE 6.3% 4.9% 4.84% 3.68%

RMSE 5.2% 4.9% 4.1% 3.58%

Classification accuracy metrics Precision 0.5% – 1.3% –

Recall 1.6% 2.55% 0.8% 1.6%

F-measure 1.1% – 1.1% –

Figure 14.
F-measure for CFP, HRS-1, and HRS-2.

Figure 15.
Comparison of up and up-q approaches with our approach.

The performance superiority of HRS-1 and HRS-2 with 40%, …, 90% of the available ratings, can be seen in Fig. 13.

In Fig. 14, F-measure for CFP, HRS-1, and HRS-2 with 40%, …, 90% of the training data set is illustrated.

The performance superiority of HRS-1 with 40%, …, 90% of the available ratings, can be seen in Fig. 14.

In Table 6, a comparison of Confidence Weighted Online Collaborative Filtering (CWOCF) approach [35] depends on the same data set with our approach depending on the summary matrix is listed.

Note that as seen in Tables 6 and 7, and Fig. 15, our approach excelled in evaluations of predictive accuracy metrics.

In Fig. 15, a comparison of Up and Up-q approaches [37] depends on MovieLens 1M data set with our approach depends on the summary matrix is illustrated.

In Table 7, The MAE for the top 10 most rated movies based on Latent Neural Network (LNN) approaches [29] and our approaches is listed.
5. Conclusions and future work

Data sets	HetRec 2011	MovieLens 1M
Precision	0.865	0.868	0.839	0.893	0.91	0.88
Recall	0.868	0.88	0.89	0.892	0.9	0.91
F-measure	0.866	0.876	0.864	0.892	0.9	0.89

Data sets	Predictive	Techniques
HetRec 2011	MAE	0.6499	0.63	0.64
	RMSE	0.8473	0.823	0.825
MovieLens 1M	MAE	0.7609	0.733	0.743
	RMSE	0.9580	0.927	0.93

Alg	106	2070	2470	2509	2676	2678	3430	3462	614	717	Ave
CBCF	1.248	0.841	0.733	0.939	0.875	1.029	0.880	1.124	1.030	0.681	0.938
CBF	1.221	0.868	0.862	1.003	1.028	1.045	0.949	1.062	1.042	0.46	0.954
LNN	1.247	0.715	0.864	0.964	1.004	0.972	0.758	0.935	1.174	0.125	0.876
LNN3PT	1.250	0.729	0.867	0.833	1.197	1.060	0.791	0.952	1.222	0.000	0.890
HRS-1	1.099	0.705	0.592	0.849	0.774	0.792	0.764	0.699	0.754	0.420	0.751
HRS-2	1.099	0.709	0.618	0.849	0.783	0.792	0.769	0.683	0.754	0.420	0.754

	Data sets	HetRec 2011	MovieLens 1M
Performance	Time-consuming for one sample testing (s)	87%	33.3%	82%	14%
Predictive accuracy metrics	MAE	6.3%	4.9%	4.84%	3.68%
	RMSE	5.2%	4.9%	4.1%	3.58%
Classification accuracy metrics	Precision	0.5%	–	1.3%	–
	Recall	1.6%	2.55%	0.8%	1.6%
	F-measure	1.1%	–	1.1%	–

In this paper, we proposed to create a summary matrix that incorporates limited items to alleviate the impact of scalability, sparsity and cold start problems in recommender systems.

The proposed approach increases the rating density, which contributes to solving the aforementioned problems. We use the summary matrix in two hybrid recommender systems and evaluate the results. The results show our summary matrix was helpful in speed, increased the rating density, and got better recommendations. This work suggests several interesting directions for future work. We calculated the likeness between users based on user-user similarity. Item-item similarity may also be tried.

Additionally, we aspired to develop this work to apply it on diverse data sets such as music, books, jokes, and Twitter followers. We would like to conduct a study at a larger scale which would involve feature selection and feature creation.

Footnotes

Acknowledgments

The first author thanks the College of Engineering of Al-Iraqia University and gives special thanks to Mona Mohamed Wafy of Al-Iraqia University. Finally, the author thanks everyone that helped and supported him in bringing this research about.

Appendix-A

In Table A1, the percentage of rating that are given by one user to all items in the training data sets versus the summary matrix is listed. The percentage column of Table A1 shows the rating density. As, it is clearly been increase rating density in the summary matrix contribute to improve the recommendation accuracy.

Table A1

Number of users versus number of items

	HetRec 2011				MovieLens 1M
	Training data set		Summary matrix		Training data set		Summary matrix
Ratings	Items	%	Items	%	Items	%	Items	%
1%–10%	9570	93.85	486	61.67	3385	87.17	176	58.47
11%–20%	423	4.148	120	15.22	125	3.21	72	23.92
21%–30%	132	1.29	63	7.99	16	0.41	23	7.64
31%–40%	61	0.59	43	5.45	357	0	11	3.65
41%–50%	11	0.12	28	3.55			9	2.99
51%–60%			22	2.79			6	1.99
61%–70%			10	1.26			1	0.33
71%–80%			9	1.14			1	0.33
81%–90%			5	0.63			2	0.66
91%–100%			2	0.25
Total	10197		788		3883		301

In Table A2, the percentage of rating that are given by all users to one item is listed.

Table A2

Number of items versus number of users

Ratings	Users	%	Users	%	Users	%	Users	%
	HetRec 2011				MovieLens 1M
	Training data set		Summary matrix		Training data set		Summary matrix
1%–10%	2073	98.1	914	43.25	5894	97.58	3279	54.29
11%–20%	39	1.84	719	34.03	141	2.34	1577	26.12
21%–30%	1	0.05	356	16.85	4	0.06	719	11.9
31%–40%			111	5.25	1	0.02	305	5.05
41%–50%			13	0.62			129	2.14
51%–60%							26	0.43
61%–70%							5	0.083
71%–80%
81%–90%
91%–100%
Total	2113		2113		6040		6040

In Fig. A1, the percentage of the ratings that are given by one user to all items in the training data sets versus the summary matrix is illustrated.

Note that as seen in Fig. A1, 94%–96% of users rated less than 10% of all items in the training data sets. Also in Fig. A2, we notice that 98% of the items in the training data sets are rated by less than 10% of users. This percentage is very low and reduce the opportunities for getting accurate recommendations.

In Fig. A2, the percentage of the ratings that are given by all users to one item is illustrated.

Note that as seen in Figs A1 and A2, re-distribution of ratings in the summary matrix for users and items. All percentages of ratings increased over 10%. This means more opportunities for getting accurate recommendations for users.

Figure A1.

Number of users versus number of items.

Figure A2.

Number of items versus number of users.

References

Jannach

Zanker

Felfernig

and Friedrich

, Recommender Systems: An Introduction, Cambridge University Press, New York, NY, USA, 2011.

Feuerverger

and Khatri

, Statistical significance of the netflix challenge, Institute of Mathematical Statistics 27(2) (2012), 202–231.

Sharif

M.A.

and Raghavan

V.V.

, A large-scale, hybrid approach for recommending pages based on previous user click pattern and content, in: Proceedings of the 21st International Symposium (ISMIS 2014), Roskilde, Denmark, 25–27 June 2014, pp. 103–112.

Agarwal

Chen

B.-C.

Elango

and Ramakrishnan

, Content Recommendation on Web Portals, Communications of the ACM 56(6) (2013), 92–101.

Bell

R.M.

and Koren

, Lessons from the netflix prize challenge, ACM SIGKDD Explorations Newsletter 9(2) (2007), 75–79.

Das

Datar

Garg

and Rajaram

, Google news personalization: Scalable online collaborative filtering, in: Proceedings of the 16th International Conference on World Wide Web (WWW2007), Alberta, Canada, 8–12 May 2007, pp. 271–280.

Linden

Smith

and York

, Amazon com recommendations: Item-to-item collaborative filtering, Internet Computing, IEEE 7(1) (2003), 76–80.

Felfernig

Isak

Szabo

and Zachar

, The VITA financial services sales support environment, in: Proceedings of the 19th National Conference on Innovative Applications of Artificial Intelligence (IAAI2007), Vancouver, British Columbia, Canada, 22–26 July 2007, pp. 1692–1699.

Gupta

Goel

Lin

Sharma

Wang

and Zadeh

R.B.

, WTF: The who-to-follow system at Twitter, in: Proceedings of the 22nd International Conference on World Wide Web (WWW2013), Rio de Janeiro, Brazil, 13–17 May 2013, pp. 1596.

10.

Ricci

Rokach

and Shapira

, Recommender Systems Handbook, Artificial Intelligence, Springer-Verlag New York, NY, USA, 2011.

11.

Grossman

, Facebook, pandora lead rise of recommendation engines, http://content.time.com/time/magazine/article/0,9171,1992403,00.html, 27 May 2010.

12.

Huang

Chen

and Zeng

, Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering, ACM Transactions on Information Systems 22(1) (2004), 116–142.

13.

Adomavicius

and Tuzhilin

, Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions, IEEE Transactions on Knowledge and Data Engineering 11(6) (2005), 734–749.

14.

Joaquin

Naohiro

and Tomoki

, Content-based collaborative information filtering: Actively learning to classify and recommend documents, in: Proceedings of the Second International Workshop on Cooperative Information Agents II, Learning, Mobility and Electronic Commerce for Information Discovery on the Internet, Paris, France, 4–7 July 1998, pp. 206–215.

15.

Pazzani

M.J.

and Billsus

, Content-based recommendation systems, the adaptive web methods and strategies of web personalization, Lecture Notes in Computer Science, Springer Berlin Heidelberg, 4321 (2007), 325–341.

16.

Burke

, Hybrid web recommender systems, the adaptive web methods and strategies of web personalization, Lecture Notes in Computer Science, Springer Berlin Heidelberg, 4321 (2007), 377–408.

17.

Brownlee

, Discover feature engineering, how to engineer features and how to get good at it, http://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/, 26 September 2014.

18.

Ray

, Feature engineering: How to transform variables and create new ones?, https://www.analyticsvidhya.com/blog/2016/01/guide-data-exploration/, 12 March 2013.

19.

Zabokrtsky

, Feature engineering in machine learning, https://ufal.mff.cuni.cz/∼zabokrtsky/courses/npfl104/html/feature_engineering.pdf, 25 March 2015.

20.

Lawrence

R.D.

Almási

G.S.

Kotlyar

Viveros

M.S.

and Duri

S.S.

, Personalization of supermarket product recommendations, Data Mining and Knowledge Discovery 5(1) (2001), 11–32.

21.

Aggarwal

C.C.

, Recommender Systems: The Textbook, Database Management and Information Retrieval, Springer International Publishing, 2016.

22.

Vaz

P.C.

de Matos

D.M.

Martins

and Calado

, Improving an hybrid literary book recommendation system through author ranking, in: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL2012), Washington, DC, USA, 10–14 June 2012, pp. 387–388.

23.

Bengio

Courville

and Vincent

, Representation learning: A review and new perspectives, IEEE Trans PAMI, Special Issue Learning Deep Architectures 35 (2013), 1798–1828.

24.

Razmjooy

Mousavi

B.S.

and Soleymani

, A hybrid neural network imperialist competitive algorithm for skin color segmentation, Mathematical and Computer Modelling 57(3–4) (2013), 848–856.

25.

Herlocker

J.L.

Konstan

J.A.

Terveen

L.G.

and Riedl

J.T.

, Evaluating collaborative filtering recommender systems, ACM Transactions on Information Systems (TOIS) 22(1) (2004), 5–53.

26.

Netflix Inc., American multinational entertainment company, http://www.netflix.com, 29 August 1997.

27.

Bottou

, Feature Engineering, http://www.cs.princeton.edu/courses/archive/spring10/cos424/slides/18-feat.pdf, 22 April 2010.

28.

Baker

, Big data: Week 3 video 3 – feature engineering, https://www.youtube.com/watch?v=drUToKxEAUA, 17 March 2014.

29.

Smith

M.R.

Gashler

M.S.

and Martinez

, A hybrid latent variable neural network model for item recommendation, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland 15–17 July 2015.

30.

GroupLens Research, human-computer interaction research lab, the Department of Computer Science and Engineering at the University of Minnesota, http://www.grouplens.org, 1992.

31.

MovieLens, GroupLens Research, http://www.movielens.org, May 1996.

32.

HetRec workshop, in: Proceedings of the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011), http://ir.ii.uam.es/hetrec2011/, Chicago, IL, USA, 23–27 October 2011.

33.

Sarwar

B.M.

Karypis

Konstan

J.A.

and Riedl

J.T.

, Application of dimensionality reduction in recommender system – a case study, in: Proceedings of Web Mining for E-Commerce – Challenges and Opportunities (WebKDD2000), Boston, MA, USA 20 August 2000.

34.

Kantardzic

, Data Mining Concepts, Models, Methods, and Algorithms, John Wiley and Sons, Inc. New York, NY, USA, 2011.

35.

Hoi

Wang

and Zhao

, Second order online collaborative filtering, in: Proceedings of 5th Asian Conference on Machine Learning (ACML2013), Canberra, Australia, 13–15 November 2013, pp. 325–340.

36.

Chen

Xie

and Guo

, Solving the sparsity problem in recommender systems using association retrieval, Journal of Computers 6(9) (2011), 1896–1902.

37.

Cantador

Bellogín

and Castells

, A multilayer ontology-based hybrid recommendation model, AI Communications 21(2–3) (2008), 203–210.

38.

Moghaddam

S.G.

and Selamat

, A scalable collaborative recommender algorithm based on user density-based clustering, in: Proceedings of 3rd International Conference on Data Mining and Intelligent Information Technology Applications (ICMIA), The Westin Resort Coloane, Macao, 24–26 October 2011, pp. 246–249.

39.

Sarwar

B.M.

Karypis

Konstan

J.A.

and Riedl

J.T.

, Analysis of recommendation algorithms for E-commerce, in: Proceedings of the 2nd ACM Conference on Electronic Commerce (EC2000), Minneapolis, MN, USA, 17–20 October 2000, pp. 285–295.

40.

Cremonesi

Turrin

Lentini

and Matteucci

, An evaluation methodology for recommender systems, in: Proceedings of the 4th International Conference on Automated Solutions for Cross Media Content and Multi-channel Distribution (AXMEDIS2008), Florence, Italy, 17–19 November 2008, pp. 224–231.

41.

Xiao

and Benbasat

, E-commerce product recommendation agents: Use, characteristics and impact, MIS Quarterly 31(1) (2007), 137–209.

42.

Jorge Morais

Oliveira

and Jorge

, A multi-agent recommender system, Advances in Intelligent and Soft Computing, Springer Berlin Heidelberg, 151 (2012), 281–288.

43.

Mahmood

M.A.

El-Bendary

Platoš

Hassanien

A.E.

and Hefny

H.A.

, An intelligent multi-agent recommender system, Advances in Intelligent and Soft Computing, Springer International Publishing, 237 (2014), 201–213.

44.

Veloso

Malheıro

and Burguıllo

J.C.

, A multi-agent brokerage platform for media content recommendation, International Journal of Applied Mathematics and Computer Science 25(3) (2015), 513–527.

45.

Hilbert

and López

, The world’s technological capacity to store, communicate, and compute information, Science 332(6025) (2011), 60–65.

46.

Cisco, The zettabyte era: Trends and analysis, http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/vni-hyperconnectivity-wp.html, 2 Jun 2016.

47.

Cavanillas

J.M.

Curry

and Wahlster

, New Horizons for a Data-Driven Economy, Springer International Publishing, 2016, 143–165.

48.

Mayer-Schonberger

and Cukier

, Big Data: A Revolution That Will Transform How We Live, Work, and Think, Houghton Mifflin Harcourt Publishing Company, New York, NY, USA, 2013.

49.

Hilbert

, What is big data? https://www.youtube.com/watch?v=XRVIh1h47sA&index=51&list=PLtjBSCvWCU3rNm46D3R85efM0hrzjuAIg, 12 August 2015.

50.

International Telecommunication Union (ITU), Key ICT Indicators for Developed and Developing Countries and the World (Totals and Penetration Rates), http://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2016/ITU_Key_2005-2016_ICT_data.xls, June 2016.

Data sets	Predictive	Techniques
	accuracy metrics	CWOCF	HRS-1	HRS-2
HetRec 2011	MAE	0.6499	0.63	0.64
	RMSE	0.8473	0.823	0.825
MovieLens 1M	MAE	0.7609	0.733	0.743
	RMSE	0.9580	0.927	0.93

Big data and intelligent software systems

Abstract

Keywords

1. Introduction

1.2 Sparsity

1.3 Cold start

2. Related work

3.1 Meta-level technique

3.2.1 MovieLens 1M data set

3.2.2 HetRec 2011 data set

Table 1 Statistics of training data sets

4.1 Overview

4.2 Data sets and preprocessing

4.3 Evaluation metrics

4.3.1 Predictive accuracy metrics

Footnotes

Acknowledgments

Appendix-A

References

Table 1
Statistics of training data sets