Graph convolutional collaborative filtering recommendation method based on temporal information during node aggregation process

Abstract

Graph Convolutional Networks (GCN) are prevalent techniques in collaborative filtering recommendations. However, current GCN-based approaches for collaborative filtering recommendation have limitations in effectively embedding neighboring nodes during node and neighbor information aggregation. Furthermore, weight allocation for the user (or item) representations after convolution of each layer is too uniform. To resolve these limitations, we propose a new Graph Convolutional Collaborative Filtering recommendation method based on temporal information during the node aggregation process (TA-GCCF). The method aggregates and propagates information using Gated Recurrent Units, while dynamically updating features based on the timing and sequence of interactions between nodes and their neighbors. Concurrently, we have developed a convolution attention coefficient to ascertain the significance of embedding at distinct layers. Experiments on three benchmark datasets show that our method significantly outperforms the comparison methods in the accuracy of prediction.

Keywords

Graph convolutional neural network collaborative filtering recommendation gated recurrent units temporal information

1 Introduction

In the era of the Internet, the amount of online data has grown so rapidly that users suffer from the dilemma of information overload. Personalized recommendations play a very important role in discovering user preferences and helping users make decisions. Collaborative filtering (CF) [1] is a common method for achieving personalized recommendations. It generates effective recommendations based on users’ historical interactions with items, such as conversations, purchases, clicks. Learning how users and items are represented is crucial to improving the effectiveness of collaborative filtering (CF). Therefore, the trend in research is to enhance the effectiveness of CF by learning the information representation of users and items. The development of graph neural networks (GNNs) [2] has recently advanced the effectiveness of CF, which models the interaction data as graphs (e.g., user-item interaction graphs) and then applies GNNs to learn effective node representations for recommendations, known as graph collaborative filtering.

While graph collaborative filtering has shown significant results in the recommendation domain, existing approaches still suffer from two shortcomings: Graph Convolutional Networks (GCN) [3] usually use interaction records of check-in datasets to construct user-item interaction graph (as shown in the left of Fig. 2) when implementing collaborative filtering recommendations. However, in the user-item interaction graph for CF, each node (user or item) is only described by a one-hot ID, which has no concrete semantics besides being an identifier. [4], this causes the weight distribution in aggregating neighbor embedding to be usually fixed in the current methods. The temporal data in the check-in dataset provides important information for a deeper understanding of user behavior. Without involving node semantics, existing methods have difficulty in using temporal features to capture the decay and deviation of user preferences. Meanwhile, current methods often rely on basic splicing or merging techniques to combine layers of convolved embeddings [4 –6], which is ineffective in distinguishing between different layer embeddings.

Fig. 1

An illustration of TA-GCCF model architecture.

Fig. 2

An illustration of the user-item interaction graph and the high-order connectivity incorporating temporal information. The node u1 is the target user to provide recommendations for.

To resolve these issues, we propose a new method of Graph Convolutional Network for collaborative filtering recommendation. We first introduce a temporal dimension in the user-item interaction graph and use the Gated Recurrent Unit (GRU) [7] to model feature changes at different interaction time nodes. The interaction sequence between users and items is considered as auxiliary information. At the same time, convolutional attention coefficients were devised to combine different embeddings within the graph convolution layer by learning the impact of each layer’s vector representations on the final representations.

2 Related work

Collaborative filtering infers the target user’s preference level for a specific product by utilizing behaviors (ratings, click counts) of users similar to the target user, and then makes relevant recommendations based on this inferred preference level [8, 9]. Matrix factorization [10] represents embedded user/item ID in the low-dimensional vector, predicting users’ ratings on items through the inner product of the user/item embedding. Other collaborative filtering methods have previously incorporated personal history as a pre-existing user embedding, along with embedding historical items to enhance its representation [1, 11]. Stai [12] presents a recommendation framework for personalized multimedia recommendation on an online platform that can recommend rich content accompanying videos to users while inferring their preferences based on interaction records.

The rapid progress of machine learning and deep learning technologies has led to their increasing use in the recommendation field to explore hidden associations between users and items. For instance, the neural collaborative filtering model combined with multilayer perceptron [13] (NCF) and the Collaborative Memory Network model [14] (CMN) are used. Nonetheless, these techniques still generate vector representations in the same way as matrix factorization, which considers the user-item pair as an independent individual and ignores the association between users and items.

In recent years, the natural compatibility of user-item interaction data with graph structures, along with advancements in computational capabilities, has directed increased attention toward recommendation models based on graph neural networks [15, 16]. Berg and others [17] proposed the GC-MC method, considering matrix completion of the recommendation system from the point of view of link prediction on graphs. Qu and others [18] proposed the concept of neighborhood interaction, expanding user-item interaction to neighborhood and neighborhood interaction through the graph. Wang and others [5] proposed the Neural Graph Collaborative Filtering model (NGCF), applied the method of graph convolution in recommendation, obtained higher-order collaborative information through the connectivity of graph network nodes, and incorporated it into the embedding of the target user/item. He and others [4] proposed a lightweight graph convolution model (LightGCN) to simplify the design of GCN and make it more concise and appropriate for recommendation. Liu and others [6] introduced the Gated Recurrent Unit in the graph neural network to solve the information loss problem between high-order connected nodes. Yang et al. [19] proposed STAM, which utilizes Scaled Dot-Product Attention to capture the temporal order of one-hop neighbors and joint attention for different latent subspaces using multi-head attention. Pareja et al. [20] proposed Evolved GCN, which focuses on dynamic graph data and models graphs at different time steps through the evolution of graph convolutional neural networks for recommendation. In their work, Wang [21] proposed LightGCAN. This approach captures static user preferences using a lightweight graph neural network with node aggregation only. Dynamic user preferences are captured using a time-aware graph attention network based on recent interaction terms. The two user preferences are then combined and fed into a dual-channel Deep Neural Network to learn feature interactions and predict matching scores.

While previous research has enhanced recommendation outcomes to an extent, it typically uses only rating information and hardly considers the influence of temporal information on node-aggregated neighbor features. Effective use of this information can further increase the accuracy of the recommendation algorithm.

3 TA-GCCF

3.1 Model architecture

Different from current collaborative filtering recommendation methods, the TA-GCCF approach proposed in this paper updates node features based on the temporal information of neighboring nodes and dynamically calculates the weights of the updated representation vectors, which ultimately improves the recommendation performance. Figure 1 illustrates the architecture of TA-GCCF. There are three components in the model:(1) Embedding layer: which constructs the adjacency matrix and initializes the embedding of users and items by using the user-item interaction graph. (2) Embedding propagation layer: which fuses the Gated Recurrent Unit and aggregates the high-order interaction information between users and items using graph convolution technology, capturing the temporal information of node interactions during the aggregation process. (3) The prediction layer is designed to create convolutional attention coefficients that learn the significance of the embedding vector following various convolution layers. It then uses the inner-product interaction function to determine the user’s preference for the recommended target item.

3.2 TA-GCCF detailed design

3.2.1 Definition of the problem

Definition 1. (Set of Users and Items): In the case of M items and N items, the user set U ={ u₁, u₂, u₃ … u_M }, and the item set I ={ i₁, i₂, i₃ … i_N }.

Definition 2. (User-Item Interaction Matrix): The interaction between users and items constitutes an interaction matrix, denoted as A^M×N, where M and N are the numbers of users and items respectively. Each element of A is denoted as a_mn, indicating whether user u interacts with item i; If there is an interaction between u and i, then a_mn is denoted as 1, otherwise a_mn is 0.

Definition 3. (User-Item Check-in Tensor): The check-in time between the user and item constitutes a two-dimensional interaction matrix, denoted as T^M×N. Each element of T is denoted as t_mn, which represents the interaction time of user u and item i. If there is no interaction between u and i, then t_mn= 0.

3.2.2 Embedding layer

The main role of the embedding layer is to construct an embedding representation of users and items, and the embedding vector contains information about users and items. Given user u and item i, their corresponding embedding representations are e_u and e_i, and the embedding vector lookup table initialized by users and items is as follows: ${user embedding : e}_{u}^{0} = [e_{u_{1}}^{0}, e_{u_{2}}^{0} \dots e_{u_{m}}^{0}]$ (1) ${item embedding : e}_{i}^{0} = [e_{i_{1}}^{0}, e_{i_{2}}^{0} \dots e_{i_{n}}^{0}]$ (2)

Where m represents the number of users, n is the number of items, and the superscript 0 indicates that the embedding is the initial zeroth-layer representation. The embedding lookup table serves as the initial state of user embedding and item embedding, it is processed and optimized in an end-to-end manner.

3.2.3 Embedding propagation layer integrated with gated recurrent unit

To capture the embedding of the user (item) and represent associative relationships between nodes, we aggregate nodes with interactive relationships. Normalized summation operations, which assign a static weight to all neighbors, have been used in many traditional models for node information aggregation. However, this has the limitation that equal aggregation does not distinguish the importance of neighbors, and in fact, the most recent interactive neighbors should have a higher weight during aggregation. Therefore, this paper presents a new aggregation method by adopting the Gated Recurrent Unit (GRU) to update the features of nodes in the convolution layer, and also adding a fully connected layer based on the GRU to assign different feature indices to different nodes, dynamically attenuating the weights of features with a larger time span.

The GRU uses the reset gate (r_t) to control the amount of historical information it retains, the update gate (z_t) to decide the amount of information to discard, and adds new state information ( $\tilde{h_{t}}$ ) to obtain the GRU output (h_t), which is mathematically represented as follows: $\begin{matrix} r_{t} = σ (W_{r} [h_{t - 1}, x_{t}]) \end{matrix}$ (3) $\begin{matrix} z_{t} = σ (W_{z} [h_{t - 1}, x_{t}]) \end{matrix}$ (4) $\begin{matrix} \tilde{h_{t}} = tanh (W_{h} [r_{t} ⊙ h_{t - l}, x_{t}]) \end{matrix}$ (5) $\begin{matrix} h_{t} = (1 - z_{t}) ⊙ h_{t - 1} + z_{t} ⊙ \tilde{h_{t}} \end{matrix}$ (6)

W_r, W_z, W_h represent the weight matrixes, σ(.) is the sigmoid activation function, h_t-1 refers to the preceding moment’s output state, and x_t is the current input. The symbols [·,·] denote the concatenation of two vectors, and ⊙ signifies the Hadamard product.

To simplify representation, the preceding function is formalized as: $\begin{matrix} h_{t} = f_{GRU} (x_{t}, h_{t - 1}) \end{matrix}$ (7)

Where f_GRU () symbolizes the GRU network.

For each node u and i, we perform the following computations: ${hidden}_{u}^{k + 1} = f_{GRU} (\frac{1}{\sqrt{N_{i}} \sqrt{N_{u}}} \cdot {(A ⊙ T)}_{Sort} \cdot e_{i}^{k}, e_{u}^{k})$ (8) ${hidden}_{i}^{k + 1} = f_{GRU} (\frac{1}{\sqrt{N_{i}} \sqrt{N_{u}}} \cdot {(A ⊙ T)}_{Sort} \cdot e_{u}^{k}, e_{i}^{k})$ (9)

Where $e_{u}^{k}$ symbolizes the representation of user u at layer k as information is disseminated, and in parallel, $e_{i}^{k}$ depicts the representation of item i at layer k in the course of information propagation. Adopting a standard graph convolutional neural network [13] design, $\frac{1}{\sqrt{N_{i}} \sqrt{N_{u}}}$ signifies the correlation coefficient.

Using the Laplace paradigm often used in graph convolutional networks, the coefficient indicates the measure of the contribution of the primary neighborhood nodes to the target node - the larger the first-order neighborhoods of the neighboring nodes, the lower the degree of contribution of the neighboring nodes to the target node, thereby preventing over-smoothing incidents. hidden^k+1 denotes the hidden state after the aggregation of the neighboring nodes by the GRU.

The Hadamard product of matrices A and T aims to integrate the two forms of information, the interaction relationship and the interaction time, into a novel graph structure matrix. Thus in the feature propagation process, not only the connectivity between nodes is considered, but also the time attributes of the interaction between nodes. The sort function sorts neighboring nodes according to time.

GRU output ‘hidden’ and corresponding timestamp input are processed by the fully connected layer for feature index calculation: $\begin{matrix} α_{ui}^{k + 1} \\ = \frac{exp (Sigmoid (W_{α} [t_{ui}, {hidden}_{ui}] + bias))}{\sum_{k \in N_{u}} exp (Sigmoid (W_{α} [t_{uk}, {hidden}_{uk}] + bias))} \end{matrix}$ (10) $\begin{matrix} α_{iu}^{k + 1} \\ = \frac{exp (Sigmoid (W_{β} [t_{iu}, {hidden}_{iu}] + bias))}{\sum_{k \in N_{i}} exp (Sigmoid (W_{β} [t_{ik}, {hidden}_{ik}] + bias))} \end{matrix}$ (11)

Where W_α and W_β represent trainable matrices, while “bias” stands for the bias term. The notation k ∈ N_u/i signifies all neighboring nodes of either the u or i node.

The final embeddings of the nodes are computed as the weighted sums of the outputs of all neighboring features: $\begin{matrix} e_{i}^{k} = \sum_{u \in N_{i}} (α_{iu}^{k} \cdot {hidden}_{i}^{k}) \end{matrix}$ (12) $\begin{matrix} e_{u}^{k} = \sum_{i \in N_{u}} (α_{ui}^{k} \cdot {hidden}_{u}^{k}) \end{matrix}$ (13)

This model further stacks multiple single-order propagation layers to explore higher-order interaction information and constructs high-order propagation of the model, as illustrated by the high-order connectivity graph in Fig. 2. The user of interest for recommendation is u1, labeled with the double circle in the left subfigure of the user-item interaction graph. The right of Fig. 2 expanded from u1 is illustrated by the subfigure on the right. For example, the path u1-i6/i3-u4 donates that stacking two convolutional layers can effectively capture the similarity in user behavior; through these two paths u1-i3-u3-i1, u1-i3-u4-i7 indicate that stacking three convolutional layers can explore potential recommendation priorities. In Fig. 2, neighboring nodes are sorted from top to bottom in order of interaction. For the two previous paths, following the use of GRU to aggregate the contributions of neighboring nodes, here it is assumed that the neighbor nodes that interacted first are more interesting, so i6, u4, and i7 are assigned greater weight by the update gate, thus predicting that i7 is more likely to arouse the interest of u1. In summary, the model uses the Z-layer stacking of the embedding propagation layer integrated with the Gated Recurrent Unit to capture the temporal information of user-item interaction for feature attenuation.

3.2.4 Prediction layer

After the propagation of Z layers, multiple representations of user and item nodes are obtained. Following the concatenation operation, user/item representation matrices are obtained: $E_{u} = [e_{u}^{0}, e_{u}^{2}, \dots, e_{u}^{z}]$ , $E_{i} = [e_{i}^{0}, e_{i}^{2}, \dots, e_{i}^{z}]$ . The representations obtained from different embedding layers have different contributions to user preference. If only the embedding result of the final layer is considered, it is easy to cause over-smoothing problems. This paper designs convolutional attention coefficients to dynamically allocate different weights to different representations: $\begin{matrix} γ = Softmax [\frac{{EW}_{q} ⊙ {({EW}_{k})}^{T}}{\sqrt{d_{k}}}] \end{matrix}$ (14)

In the formula, W_q and W_k are trainable matrices, E is a representation matrix, and $\sqrt{d_{k}}$ is a parameter set to prevent gradients from being too small by scaling, default to d, where d is the dimension of the embedding vector.

The obtained γ represents the weight matrix for each layer of representation. The last layer of representation contains higher-order interactive information, therefore, the weight vector updated by the last layer Z is utilized: $\begin{matrix} η = γ [Z] = {β_{0}, β_{1} \dots β_{Z}} \end{matrix}$ (15)

Where β is the weight after comparing the representations of each layer with the final layer.

With the integration of the embeddings captured at each layer, the final user vector representation is constructed. The final embeddings for users and items take the following form: $\begin{matrix} e_{u} = \sum_{L = 0}^{Z} η [L] \times e_{u}^{(L)} \end{matrix}$ (16) $\begin{matrix} e_{i} = \sum_{L = 0}^{Z} η [L] \times e_{i}^{(L)} \end{matrix}$ (17)

In the model prediction section, we employ an inner product interaction function to estimate the user’s preference for the target item through an inner product operation. $\begin{matrix} y_{ui} = e_{u} * e_{i}^{T} \end{matrix}$ (18)

3.3 Optimization

To maximize the score difference between positive and negative samples. This paper uses the Bayesian Personalized Ranking (BPR)[22] loss function to train the model to increase the maximum margin probability. The objective function is as follows $\begin{matrix} L_{BPR} = \sum_{u, i, j \in P} - ln σ ({\hat{y}}_{ui} - {\hat{y}}_{uj}) + λ ∥ Θ ∥^{2} \end{matrix}$ (19)

Where P ={ (u, i, j) | (u, i) ∈ R⁺, (u, j) ∈ R^- } represents the user’s interaction data set with the item, R⁺ is the observed user-item interaction, R^- is the unobserved interaction, σ (.) represents the sigmoid function, λ ∥ Θ ∥ ² is the L2 regularization term to prevent overfitting.

4 Experiments

4.1 Dataset description

To evaluate the effectiveness of TA-GCCF, we use four publicly available benchmark datasets: Gowalla, Yelp2018, Amazon-Book, and MovieLens-1M. The Gowalla dataset is a check-in dataset from Gowalla collected from February 2009 to October 2010. Yelp2018 constitutes a part of the data released by Yelp from 2004 to 2018 as part of the Yelp Dataset Challenge. Amazon-Book contains information on 3 million book reviews for 212,404 books and the user information for these reviews. The MovieLens-1M dataset comprises 1 million ratings of 4,000 movies by 6,000 users, released in February 2003. Table 1 presents statistical information on the four datasets.

Table 1
Statistics of the datasets

Dataset #Users #Items #Interations Density

Gowalla 29858 40981 1027370 0.00084

Yelp2018 31668 38048 1561406 0.00130

Amazon-Book 52643 91599 2984108 0.00062

MovieLens-1M 6040 3900 1000209 0.03816

Dataset	#Users	#Items	#Interations	Density
Gowalla	29858	40981	1027370	0.00084
Yelp2018	31668	38048	1561406	0.00130
Amazon-Book	52643	91599	2984108	0.00062
MovieLens-1M	6040	3900	1000209	0.03816

4.2 Evaluation metrics and baselines

In this paper, we evaluate the performance of the recommendation method by using the commonly used evaluation metrics: Recall@k and Normalized Discounted Cumulative Gain (NDCG@k). We have set k = 20 as the default value.

These two evaluation protocols have also been widely applied in previous research [13, 14]. Recall@N is the proportion of the top-N recommended results that hit the items that the user will visit, while NDCG@N is an evaluation indicator based on sorting results, used to measure the quality of sorting.

To test the effectiveness of the TA-GCCF, this paper compares it to six state-of-the-art collaborative filtering algorithms.

GC-MC [17]: A matrix completion method based on graph convolutional neural networks. this method uses the interactive relationship and other auxiliary information between users and items and performs matrix filling by graph convolution operation to achieve recommendations.

NGCF [5]: This advanced recommendation method is based on graph convolutional neural networks, this model propagates the features of nodes in the user-item graph to fully consider the hidden information in high-order connections to improve recommendation performance.

LightGCN [4]: A collaborative filtering recommendation method based on graph structure, which learns embedding representation by linear propagation on interaction graph, and regards the weighted sum of embedding information from different propagation layers as the final embedding, removing feature transformation and non-linear activation and replacing self-connection with layer combination.

DGCF [23]: An advanced collaborative filtering approach that iteratively refines intent-aware interaction graphs and factor representations using graph disentangling module.

SGL [24]: Introduces a recommendation method for self-supervised learning and designs three types of data augmentation from different aspects to construct the auxiliary contrastive task.

SVD-GCN [25]: A simplified GCN that uses only the K-largest singular values and vectors for recommendation. A reformulation trick is employed to adjust the singular value gap, which helps alleviate the over-smoothing problem.

4.3 Implementation details

The models and baselines in this paper are implemented in Python 3.9, based on the RecBole framework. The machine runs on a CPU Xeon Gold 5117 2.50 GHz with a GPU NVIDIA Grid V100D-32Q and is operated under Windows 10. For each dataset, we randomly select 80% of the historical interactions of each user for the training set, 10% for the testing set, and 10% for the validation set.

The hyperparameters were optimized for all experiments using Adam [26] as the optimizer, and the parameters were initialized in the Xavier [27] manner. Each observed user-item interaction was treated as a positive instance, and a negative sampling strategy was performed to pair it with a negative item with which the user had no previous interaction. The data volume size for each processing is 4096, and the embedding size is 64. The DGCF is tuned in {2, 4, 8} and the SGL is instantiated using SGL-ED. The SVD-GCN is instantiated using the basic SVD-GCN-S. The optimal parameters are determined through grid search, with the L2 regularization coefficient λ adjusted in {10^-5, 10^-4 … 10^-2} and the learning rate in {0.0001, 0.0005, 0.001, 0.005}. The number of model layers Z = 3 and η=γ3 are adjusted according to the experimental results. The model converged after 1000 iterations.

4.4 Performance comparison

4.4.1 Overall comparison

Five independent experiments were conducted on the same four datasets for the model presented in this paper and the comparison model. The average experimental results are shown in Table 2.

Table 2
Overall performance comparison

Dataset Gowalla Yelp2018 Amazon-Book MovieLens

Method Recall NDCG Recall NDCG Recall NDCG Recall NDCG

GC-MC 0.1632 0.0919 0.0965 0.0482 0.0908 0.0467 0.2691 0.2603

NGCF 0.1734 0.1032 0.1035 0.0577 0.0981 0.0557 0.2748 0.2619

DGCF 0.1831 0.1074 0.1141 0.0649 0.1132 0.0649 0.2786 0.2624

LightGCN 0.1983 0.1161 0.1168 0.0658 0.1210 0.0692 0.2804 0.2631

SGL 0.2088 0.1231 0.1294 0.0745 0.1337 0.0781 0.2857 0.2655

SVD-GCN 0.2131 0.1286 0.1337 0.0781 0.1398 0.0842 0.2891 0.2671

TA-GCCF 0.2169 0.1305 0.1361 0.0819 0.1432 0.0875 0.2920 0.2688

Dataset	Gowalla	Yelp2018	Amazon-Book	MovieLens
Method	Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG
GC-MC	0.1632	0.0919	0.0965	0.0482	0.0908	0.0467	0.2691	0.2603
NGCF	0.1734	0.1032	0.1035	0.0577	0.0981	0.0557	0.2748	0.2619
DGCF	0.1831	0.1074	0.1141	0.0649	0.1132	0.0649	0.2786	0.2624
LightGCN	0.1983	0.1161	0.1168	0.0658	0.1210	0.0692	0.2804	0.2631
SGL	0.2088	0.1231	0.1294	0.0745	0.1337	0.0781	0.2857	0.2655
SVD-GCN	0.2131	0.1286	0.1337	0.0781	0.1398	0.0842	0.2891	0.2671
TA-GCCF	0.2169	0.1305	0.1361	0.0819	0.1432	0.0875	0.2920	0.2688

TA-GCCF is found to be improved in all comparisons with the baseline methods recommended above for collaborative filtering. It adequately captures the higher-order interactions between users and items, outperforming GC-MC. Due to the lack of node characterization in collaborative filtering scenarios, TA-GCCF achieves a significant improvement compared to NGCF. LightGCN uses only normalized summation to aggregate node information and the average distribution of representation weights. In contrast, TA-GCCF dynamically learns the weights and achieves significant results. DGCF has mediocre performance, and we speculate that the dimension of the disentangled cannot carry enough features in the case of limited overall dimensionality. SGL compares the original graph with the augmented graph for comparison, ignoring other potential relationships (e.g., user similarity) in the recommender system. SVD-GCN benefits from its replaced neighborhood aggregation and performs optimally in all baselines. However, TA-GCCF outperforms SVD-GCN by fully utilizing multidimensional features.

In terms of the overall comparison of recommendation effects among models, our proposed model significantly improves the two evaluations metrics across all four datasets compared to other methods This demonstrates the rationality of the model design and also proves the model’s efficiency and good generalization capacity.

4.4.2 The effect of the number of embedding propagation layers

To investigate the optimal number of embedding propagation layers, we varied the depth of the model. The experiment search L in the range of {1, 2, 3, 4} and summarize the empirical results in Table 3. As the number of layers in the graph convolutional layer increases, the recommendation effect of the model also improves significantly. Nonetheless, when the layer number increases to 4, the model shows overfitting. This is because the embedding propagation layer’s increased depth leads to an over-smoothing of the feature representation between nodes, which causes a lack of distinction in the node representation and reduces the model’s recommendation effectiveness. And, when the stacking of embedding propagation layers is too high, too much noise is introduced into the model training, which causes overfitting. This also indirectly verifies that the stacking of three embedding propagation layers is sufficient to capture effective collaborative filtering signals.

Table 3
Effect of embedding propagation layer numbers

Dataset Gowalla Yelp2018 Amazon-Book MovieLens

Layer Recall NDCG Recall NDCG Recall NDCG Recall NDCG

C1 0.2082 0.1261 0.1222 0.0738 0.1349 0.0801 0.2847 0.2621

C2 0.2116 0.1281 0.1273 0.0769 0.1375 0.0826 0.2872 0.2658

C3 0.2169 0.1305 0.1361 0.0819 0.1432 0.0875 0.2920 0.2688

C4 0.2128 0.1294 0.1331 0.0792 0.1391 0.0848 0.2887 0.2671

Dataset	Gowalla	Yelp2018	Amazon-Book	MovieLens
Layer	Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG
C1	0.2082	0.1261	0.1222	0.0738	0.1349	0.0801	0.2847	0.2621
C2	0.2116	0.1281	0.1273	0.0769	0.1375	0.0826	0.2872	0.2658
C3	0.2169	0.1305	0.1361	0.0819	0.1432	0.0875	0.2920	0.2688
C4	0.2128	0.1294	0.1331	0.0792	0.1391	0.0848	0.2887	0.2671

4.4.3 Ablation analyses

To further investigate the impact of different modules of TA-GCCF on model accuracy, we performed different transformations on TA-GCCF and obtained the following variant methods for compa-rative experiments. TA-GCCF-G: which removes convolutional attention coefficients to learn the weights of each representation vector, the default is the same weight, i.e. γ_L = 1/(Z + 1). TA-GCCF-A: which removes GRU to capture temporal information, only using the convolutional attention coefficient. TA-GCCF-L: which removes both GRU and convolutional attention coefficients. Figure 3, show the performance differences between TA-GCCF and its different variants, using Recall and NDCG as two metrics to evaluate the effectiveness of each module. From the results of the experiment, it is apparent that the indicators for all variants decrease to varying degrees after the removal of the module across the three datasets. The decrease of TA-GCCF-A is larger than that of TA-GCCF-G, which indicates that temporal features are more effective than convolutional attention coefficients in improving model accuracy. The indicators reach the lowest level when two modules are deleted at the same time, which dem-onstrates the effectiveness of the model.

Fig. 3

Ablation analyses.

As a whole, the accuracy of TA-GCCF’s recom-mendations is significantly improved from these variants. It has been demonstrated that the modules proposed in this paper do not conflict with each other and they all contribute to improving the recommendation performance of the model.

4.4.4 Comparison analysis of convolutional attention parameters

The TA-GCCF’s convolutional attention coefficient default calculation is based on the representation of the last layer when measuring similarity. To investigate whether the learning effect of the representation of the last layer is the best, we perform a comparative experiment in this section using the representation of different layers to compute similarity. The layers set for this experiment are γ1, γ2, and γ3, with the experi-mental results as shown in Table 4. When η=γ3, i.e. the representation of the last layer is used to compute the similarity, it can be seen that TA-GCCF has a performance improvement compared to the first two layers. Because, under the premise of reducing the effect of overfitting after the embedding propagation layer experiment, the representation of the last layer has higher-order connectivity and can capture higher-order interaction features between nodes. Therefore, the effect of using the last layer representation for computation is better.

Table 4
Comparison of convolutional attention parameters conclusion

Dataset Gowalla Yelp2018 Amazon-Book MovieLens

Layer Recall NDCG Recall NDCG Recall NDCG Recall NDCG

C1 0.2152 0.1273 0.1338 0.0791 0.1402 0.0841 0.2901 0.2659

C2 0.2157 0.1287 0.1346 0.0803 0.1417 0.0855 0.2907 0.2673

C3 0.2169 0.1305 0.1361 0.0819 0.1432 0.0875 0.292 0.2688

Dataset	Gowalla	Yelp2018	Amazon-Book	MovieLens
Layer	Recall	NDCG	Recall	NDCG	Recall	NDCG	Recall	NDCG
C1	0.2152	0.1273	0.1338	0.0791	0.1402	0.0841	0.2901	0.2659
C2	0.2157	0.1287	0.1346	0.0803	0.1417	0.0855	0.2907	0.2673
C3	0.2169	0.1305	0.1361	0.0819	0.1432	0.0875	0.292	0.2688

5 Conclusion

In this work, we propose a new method named TA-GCCF., which considers the interaction information between the user and item at the embedding layer; the GRU and graph convolutional neural network are introduced in the embedding propagation layer, and a new aggregation method is proposed to capture the temporal features between nodes; the convolutional attention coefficient is used in the prediction layer to assign weights to different representation vectors, finally predicting the association score between the user and the item using the inner product operation. The experimental results show that compared with the existing mainstream collaborative filtering recommendation models, the model in this paper has achieved better recommendation results.

Footnotes

Acknowledgments

This work is supported by the University Collabo-rative Innovation Project (GXXT-2021-093-2) and Anhui Key R&D Programme Project –Top level Tackling Project (202004a07020050).

References

Sarwar

, Karypis

, Konstan

, et al. Item-based collaborative filtering recommendation algorithms, Proceedings of the 10th International Conference on World Wide Web, Hong Kong, [C], ACM, 2001:285–295.

Scarselli

, Gori

, Tsoi

A.C.

, et al. The graph neural network model[J], IEEE Transactions on Neural Networks 20(1) (2008), 61–80.

Kipf

T.N.

, Welling

, Semi-supervised classification with graph convolutional networks[J], arXiv preprint arXiv:1609.02907, 2016.

X.N.

, Deng

, Wang

, et al. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation[C], Proceedings of the 43rd ACM SIGIR International Conference on Research and Development in Information Retrieval, New York, USA, ACM, 2020:639–648.

Wang

, He

X.N.

, Wang

, et al. Neural Graph Collaborative Filtering[C], Proceedings of the 42nd ACM SIGIR International Conference on Research and Development in Information Retrieval, New York, USA, ACM, 2019:165–174.

Liu

G.Z.

and Chen

H.L.

, Convolutional memory graph collaborative filtering[J], Journal of Beijing University of Posts and Telecommunications 44(03) (2021), 21–26 10.13190/j.jbupt.2020-226.

Cho

, Van Merriënboer,

, Gulcehre,

, et al. Learning phrase representations using RNN encoderdecoder for statistical machine translation[J], arXiv preprint arXiv:1406.1078, 2014.

Zhang

, Gu

, Ji

, et al. Personalized scientific and technological literature resources recommendation based on deep learning[J], Journal of Intelligent & Fuzzy Systems 41(2) (2021), 2981–2996.

Sun,

, Liu,

, Ren,

, et al. NCGAN: A neural adversarial collaborative filtering for recommender system[J],, Journal of Intelligent & Fuzzy Systems 42(4) (2022), 2915–2923.

10.

Koren

, Bell

and Volinsky

, Matrix factorization techniques for recommender systems[J], Computer 42(8) (2009), 30–37.

11.

Adomavicius

and Tuzhilin

, Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions[J], IEEE Transactions on Knowledge and Data Engineering 17(16) (2005), 734–749 10.1109/TKDE.2005.99.

12.

Stai

, Kafetzoglou

, Tsiropoulou

, et al. A holistic approach for personalization, relevance feedback & recommendation in enriched multimedia content[J], Multimedia Tools and Applications 77 (2018), 283–326.

13.

X.N.

, Liao

L.Z.

, Zhang

H.W.

, et al. Neural Collaborative Filtering[C], Proceedings of the 26th International Conference on World Wide Web, New York, USA, ACM, 2017:173–182.

14.

Ebesu

, Shen

, Fang

, Collaborative memory network for recommendation systems[C], Proceedings of the 41st ACMSIGIR International Conference on Research and Development in Information Retrieval, New York, USA, ACM, 2018:515–524.

15.

, et al. Graph neural networks in recommender systems: a survey, ACM Computing Surveys 55(5) (2022), 1–37.

16.

Gao

, Zheng

, Li

, et al. A survey of graph neural networks for recommender systems: Challenges, methods, and directions[J], ACM Transactions on Recommender Systems 1(1) (2023), 1–51.

17.

Berg

, Kipf

T.N.

, Welling

, Graph convolutional matrix completion[J], arXiv preprint arXiv:1706.02263, 2017.

18.

Y.R.

, Bai

, Zhang

W.N.

, et al. An end-to-end neighborhood-based interaction model for knowledgeenhanced recommendation[C], Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data, New York, USA, ACM, 2019. doi:10.1145/3326937.3341257.

19.

Zhen

, et al. Stam: A spatiotemporal aggregation method for graph neural network-based recommendation. Proceedings of the ACM Web Conference 2022. 2022.

20.

Pareja

, Domeniconi

, Chen

, et al. EvolveGCN: Evolving graph convolutional networks for dynamic graphs[C], Proceedings of the AAAI Conference on Artificial Intelligence 34(04) (2020), 5363–5370.

21.

Wang

, Lou

and Jiang

, LightGCAN: A lightweight graph convolutional attention network for user preference modeling and personalized recommendation, Expert Systems with Applications (2023), 120741.

22.

Rendle

, et al. BPR: Bayesian personalized ranking from implicit feedback, arXiv preprint arXiv:1205.2618 (2012).

23.

Wang

, **ang, , et al. Disentangled graph collaborative filtering. Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 2020.

24.

, et al. Self-supervised graph learning for recommendation. Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 2021.

25.

Peng

, Sugiyama

, Mine

, SVD-GCN: A simplified graph convolution paradigm for recommendation. Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022.

26.

Kingma

D.P.

, Ba

, Adam: A method for stochastic optimization[J], arXiv preprint arXiv:1412.6980, 2014.

27.

Glorot

and Bengio

, Understanding the difficulty of training deep feedforward neural networks[C],:, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings (2010), 249–256.

Graph convolutional collaborative filtering recommendation method based on temporal information during node aggregation process

Abstract

Keywords

1 Introduction

3 TA-GCCF

3.1 Model architecture

3.2 TA-GCCF detailed design

3.2.1 Definition of the problem

3.2.2 Embedding layer

4.1 Dataset description

Table 1 Statistics of the datasets Dataset #Users #Items #Interations Density Gowalla 29858 40981 1027370 0.00084 Yelp2018 31668 38048 1561406 0.00130 Amazon-Book 52643 91599 2984108 0.00062 MovieLens-1M 6040 3900 1000209 0.03816

4.3 Implementation details

4.4 Performance comparison

4.4.1 Overall comparison

Footnotes

Acknowledgments

References

Table 1
Statistics of the datasets

Dataset #Users #Items #Interations Density

Gowalla 29858 40981 1027370 0.00084

Yelp2018 31668 38048 1561406 0.00130

Amazon-Book 52643 91599 2984108 0.00062

MovieLens-1M 6040 3900 1000209 0.03816