cd-MBRec: Enhancing multi-behavior recommendation by explicitly modeling commonality and diversity

Abstract

Multi-behavior recommendation models excel in extracting abundant information from user-item interactions to enhance performance; however, they encounter challenges in accuracy due to noise disturbance and ambiguous weight allocation. In this paper, we propose cd-MBRec, a novel model designed to amplify commonality among various behaviors, thereby minimizing noise interference while preserving behavior diversity to highlight semantic variations in feedback across distinct scenarios. Specifically, the model begins by constructing behavior matrices that models separate behaviors, along with an interaction matrix offering a broad overview of user behaviors. It employs graph neural networks to extract higher-order semantic and structural information from input data. Concurrently, the model integrates principles of Weber-Fechner Law for the adaptive allocation of initial weights to the multiple behaviors and utilizes matrix factorization techniques for efficient behavior embedding. Extensive experiments on two real-world datasets demonstrate that cd-MBRec surpasses existing state-of-the-art models in recommendation performance, achieving notable average improvements of 4.96% in HR@10 and 7.75% in NDCG@10.

Keywords

Multi-behavior recommendation graph neural networks matrix factorization

1. Introduction

Recommender systems are essential in online services, including e-commerce [1], social media [2], and news platforms [3]. They provide personalized suggestions for products or services using a wide range of information. Multi-behavior recommendation modeling [4, 5, 6, 7], which has become increasingly important in modern recommendation tasks, is particularly efficacious under circumstances with complex user interaction behaviors. It considers users’ explicit actions and incorporates their implicit feedback as well, significantly improving the performance of recommendation services across various domains. Additionally, multi-behavior recommendation models offer new perspectives and methods to address key issues in recommender systems, such as the cold-start problem and data sparsity [8, 9].

While the incorporation of multiple behaviors in recommender systems offers additional information, it also introduces complex contexts that are often not adequately addressed in current research. Firstly, in real-world scenarios, although multiple behaviors reflect different user intentions, some may be disturbed by abnormal user actions or personal biases during data collection. Excessively emphasizing the differences in behaviors while overlooking their similarities can lead to challenges in prediction due to noise interference [10, 11]. Secondly, user behaviors correspond to diverse intentions. Existing work fails to explicitly explain the weight allocation process of multiple behaviors across various contexts, thereby impairing the adaptability and interpretability of recommendation models.

Figure 1.

Examples of different user intentions through multiple behavioral combinations. Although User 1 and User 2 engage differently with the watch, they exhibit similar interests, highlighting commonalities across multiple behaviors. In contrast, User 2 and User 3, both engaging with the iPhone, demonstrate distinct interests. User 3’s previous purchase suggests a lower likelihood of a repeat purchase in the near term, thus exemplifying diversity in multiple behaviors.

Different types of user behaviors reveal both commonality and diversity. For instance, Fig. 1 illustrates the behavior sequences of three users, involving various interactions such as click, favorite, add-to-cart, and purchase. In this example, both User 1 and User 2 interact with an Apple watch. User 1 follows a sequence of {click, favorite, add-to-cart}, while User 2 goes directly from click to add-to-cart. Despite their differing behavioral combinations, which could be attributed to personal habits, User 1 and User 2 exhibit similar interest in the Apple watch, highlighting the commonality within multiple behaviors. Conversely, examining User 2 and User 3 provides insight into behavior diversity. After clicking, User 2 adds an iPhone to the cart, while User 3 immediately makes a purchase. Since User 3 has already bought an iPhone, she is less likely to buy another phone in a short period, whereas User 2, who has not completed a purchase, shows sustained interest in the iPhone. Therefore, in this scenario, clicking behavior should be given more emphasis than purchasing, and it would be more effective to recommend an iPhone to User 2, illustrating the diversity in behaviors. This example underscores the importance of recognizing both commonality and diversity in multi-behavior recommendation tasks. By understanding these nuances in user-item interactions, we can uncover the potential value of these interactions, leading to more accurate and personalized recommendations.

Based on the above-mentioned analysis, we propose a Multi-Behavior Recommendation model explicitly designed to encapsulate both commonality and diversity (cd-MBRec). Specifically, this model features a behavior-commonality embedding aggregation module, adept at capturing higher-order content and structural information pertaining to user-item interactions. Each individual behavior is modeled using graph neural networks (GNNs). A crucial element of our model is the integration of the interaction matrix. This matrix not only records interactions between users and items but also effectively synthesizes the commonality among various behaviors while obtaining the embedding of users and items.

To depict the diversity of behaviors, we have developed a unique behavior-diversity weight allocation module. Grounded in the principles of the Weber-Fechner Law, this module leverages psychological insights to determine the initial weight of each behavior based on the different stimulus intensities they present to users. It also generates an adaptive scoring matrix, thus effectively handling the behavioral variations in different scenarios. By incorporating commonality interaction behaviors and adjusting the weights of various behaviors, our model can better adapt to users’ actual needs in different contexts. Our contributions can be summarized as follows: –

We present cd-MBRec, an innovative recommendation model that focuses on explicitly capturing both the commonality and diversity of multiple behaviors. This method is adept at minimizing the impact of inherent noise and amplifying the variances in multi-behavioral patterns across various recommendation scenarios, ultimately enhancing overall performance.

–

We introduce the concept of an interaction matrix, which we integrate with multiple behavior matrices to develop a behavior-commonality embedding aggregation strategy. This approach effectively extracts commonalities in multi-behavior modeling and can be applicable to other multi-behavior models, demonstrating promising results.

–

Extensive experiments on two real-world datasets establish that our proposed cd-MBRec model surpasses state-of-the-art models in multi-behavior recommendation tasks.

The rest of this paper is structured as follows: Section 2 briefly reviews related work, Section 3 outlines our methodology, Section 4 illustrates the model’s effectiveness through experiments, and Section 5 provides our conclusion.

2. Related work

2.1 Multi-behavior recommendation

The field of multi-behavior recommender systems has garnered significant interest as a comprehensive alternative to single-behavior models such as DeepFM [12] and SASRec [13]. These models stand out due to their utilization of multiple user behaviors as auxiliary information, which enhances the predictive performance of the system [14, 32]. NMTR [15] adopts shared user and item embedding layers across various behavior types, capturing intricate and diverse interactions. MB-GCN [16] introduces a graph-based approach, unifying multiple user-item interaction matrices into a single graph for analysis. MB-GMN [17] incorporates multi-behavior pattern modeling within the meta-learning paradigm, enabling the discovery of behavior representations across different types. GHCF [18] jointly embeds node (user and item) representations and their relationships into a multi-relational prediction framework based on GCN [19], and it performs non-sampling optimization within a multi-task learning framework. S-MBRec [20] applies GCN to learn user and item embedding for each behavior, and it designs a supervised task to discern the importance of different behaviors. Additionally, it introduces a star-shaped contrastive learning task to capture embedding commonalities between target and auxiliary behaviors. These developments consider a variety of user behaviors, thereby offering more personalized and accurate recommendations. This paper also focuses on utilizing multiple behaviors to improve recommendation performance.

2.2 Graph-based recommendation

The burgeoning interest in graph neural networks for recommender systems has led to significant advancements in understanding and leveraging the complex relationships between users and items. Pioneering models such as GCN [19], GraphSage [21], and GAT [22] utilize the aggregation of neighbor node embedding in the spatial domain to refine the embedding of target nodes. The core idea is to enhance node representations by their neighborhoods. SR-GNN [23] and SURGE [24] have applied GNNs to session-based and sequential recommendations, respectively, achieving promising results. Further developments in graph modeling are represented by NGCF [25], which refines node embedding by adopting the bipartite user-item graph structure. LightGCN [26] argues that multi-layer non-linear graph neural networks can make model training more challenging. It removes all redundant parameters and retains only ID embedding, simplifying the model in a manner similar to matrix factorization (MF). GDE [27] addresses the issue of over-smoothing, a common problem in deep GNNs where features become overly homogenized as the number of layers increases. While these models have made strides in capturing both content and structural information at higher orders, they often lack the flexibility to fully grasp the diversity of user behaviors. To address this gap, our paper proposes a novel approach by constructing a user-item interaction matrix and assigning behavior weights based on different application scenarios. This aims to provide a more adaptable and nuanced understanding of user behaviors in recommender systems.

3. Methodology

We represent the set of users and items with $U = {u_{1}, u_{2}, \dots, u_{| U |}}$ and $I = {i_{1}, i_{2}, \dots, i_{| I |}}$ , respectively. To capture the interactions between $K$ behavior categories, we use a three-dimensional matrix $X \in R^{K \times | U | \times | I |}$ to stand for users’ multiple behaviors, where $x_{k, u, i} = 1 (x \in X)$ if there is an interaction of the $k$ -th behavior category between user $u$ and item $i$ , and 0 otherwise. Each graph $G_{k}$ consists of nodes $V_{k}$ for users and items, and edges $E_{k}$ for interactions. User embedding reflecting preferences and behaviors, encapsulates latent interests and traits, while item embedding captures inherent attributes and unique characteristics. For users and items in the dataset, user and item embedding matrices are $E_{u} \in R^{(| U | \times d)}$ and $E_{i} \in R^{(| I | \times d)}$ respectively, where $d$ is the dimension of the embedding.

To initialize the weights of each behavior, we allocate behaviors in the dataset by $p_{1}, p_{2}, \dots, p_{K}$ , where $p_{k}$ represents the initial weight gained by Weber-Fechner Law of the $k$ -th behavior. We formulate the problem as predicting the probability of user $u$ performing the purchase behavior on item $i$ using $X$ as inputs.

3.1 Model architecture

Figure 2.

The architecture of the proposed cd-MBRec model. It consists of three components: the Behavior-Commonality Embedding Aggregation (EA) module, the Behavior-Diversity Weight Allocation (WA) module, and the Fusion module.

We propose the cd-MBRec model, illustrated in Fig. 2, to model the commonality and diversity present in multi-behavior characteristics. cd-MBRec comprises two separately trainable modules: the Behavior-Commonality Embedding Aggregation (EA) module and the Behavior-Diversity Weight Allocation (WA) module, as well as a Fusion module for integrating. The EA module utilizes graph convolution networks to capture high-order content and structured information. It also integrates behavior matrices and interaction matrices for model training, enabling the model to collect commonality of various behaviors. On the other hand, the WA module plays a crucial role in the model by assigning flexible weights to each behavior. It emphasizes the semantic differences inherent in behavioral feedback across different scenarios. This weight allocation process helps in learning behavior diversity on the recommendation outcomes, leading to more precise and effective recommendations. The collaboration between these two modules, EA and WA, combines commonality and diversity of multiple behaviors. Finally, the Fusion module acts as the coordinator, ensuring fine integration of the contributions from the EA and WA modules. It facilitates a harmonious collaboration, ultimately resulting in improved accuracy and effectiveness of the recommendations.

3.2 Behavior-commonality embedding aggregation

In cd-MBRec, graph convolution networks capture multiple interaction types between users and items, track interaction occurrences, and integrate these results into feature representations. For the graph view, we build a global multi-relation user-item graph $G = (V, E)$ , where $V$ is the node set, and $E$ is the edge set. If user $u$ and item $i$ have an interaction under a certain behavior $k$ , there is an edge $(x_{k, u, i} = 1 \in E)$ in graph $G$ . To study user-item interactions across different behaviors, we construct $K + 1$ graphs $G_{1}, G_{2}, \dots, G_{K + 1}$ , where $K$ represents the number of behavior types. The first $K$ graphs detail interactions per behavior, while the $(K + 1)$ -th graph indicates overall interactions. For example, if user $u_{1}$ clicks on item $i_{1}$ but does not purchase it, $x_{1, 1, 1} = 1$ , $x_{2, 1, 1} = 0$ , and $x_{3, 1, 1} = 0 + 1 = 1$ . Here, $x_{1, 1, 1}$ represents the click behavior, $x_{2, 1, 1}$ represents the purchase behavior, and $x_{3, 1, 1}$ represents the combined interaction. In this case, there are two types of behaviors, so $K = 2$ . In the EA module, these interactions are represented using an adjacency matrix, where undirected edges signify interactions between corresponding users and items.

To capture high-order information with neighborhood nodes in the graph structures of each behavior, we adopt the Light Graph Convolution (LGC) approach, as implemented in LightGCN [26], to learn the embedding representations of users and items. For each layer of convolutional operation on the $k$ -th behavior graph, the propagation of user node features and item node features is defined as follows:

\begin{aligned} E_{u, k}^{(l + 1)} & = \sum_{i \in N_{u, k}} \frac{1}{\sqrt{| N_{u, k} |} \sqrt{| N_{i, k} |}} E_{i, k}^{(l)}, \end{aligned}

(1)

\begin{aligned} E_{i, k}^{(l + 1)} & = \sum_{i \in N_{i, k}} \frac{1}{\sqrt{| N_{i, k} |} \sqrt{| N_{u, k} |}} E_{u, k}^{(l)} . \end{aligned}

(2)

Here, $i \in N_{u, k}$ represents the item nodes adjacent to the user node in the $k$ -th behavior graph, which includes all the nodes interacted with user $u$ . The term $\sum_{i \in N_{u, k}} \frac{1}{\sqrt{| N_{u, k} |} \sqrt{| N_{i, k} |}}$ is a normalization matrix that ensures all nodes have the same measurement scale for their features.

Additionally, each behavior graph undergoes $L$ layers of graph convolution for neighborhood information propagation to capture high-order information. In this process, the initial embeddings $E_{u, k}^{(0)}$ and $E_{i, k}^{(0)}$ for users and items, respectively, in the $k$ -th behavior graph are set as the user’s initial embedding $E_{u}$ and the item’s initial embedding $E_{i}$ . For each behavior graph, the outputs after $L$ layers of graph convolution, denoted as $E_{u, k}$ and $E_{i, k}$ , respectively, are obtained by aggregating the weighted average of outputs from each layer.

\begin{aligned} E_{u, k} & = \sum_{l = 0}^{L} α_{l} E_{u, k}^{(l)}, \end{aligned}

(3)

\begin{aligned} E_{i, k} & = \sum_{l = 0}^{L} α_{l} E_{i, k}^{(l)} . \end{aligned}

(4)

The weight $α_{l}$ denotes the importance of the l-th layer embedding in constituting the final embedding. Referring to LightGCN, we uniformly set $α_{l}$ for each layer as $1 / (L + 1)$ to achieve better results.

To associate all behaviors and form the interaction prediction between users and items, EA performs a weighted average operation on the outputs of the $K + 1$ data graphs that have undergone L layers of convolution.

\begin{aligned} {\bar{E}}_{u} & = \sum_{k = 1}^{K + 1} E_{u, k} / (K + 1), \end{aligned}

(5)

\begin{aligned} {\bar{E}}_{i} & = \sum_{k = 1}^{K + 1} E_{i, k} / (K + 1) . \end{aligned}

(6)

After obtaining the final outputs for users and items, the prediction value in this module is calculated as follows:

\begin{aligned} {\hat{y}}_{u, i}^{E A} = Sigmoid ({\bar{E}}_{u} {\bar{E}}_{i}^{T}) . \end{aligned}

(7)

3.3 Behavior-diversity weight allocation

In the WA module, we optimize the initial embedding $E_{u} \in R^{| U | \times d}$ and $E_{i} \in R^{| I | \times d}$ of users and items by obtaining the weighted interaction matrix.

Since different behaviors have different semantic meanings and carry varying degrees of interest information for users, we assign a weight list $W = {w_{1}, w_{2}, \dots, w_{K}}$ , where $w_{k} \in W$ represents the weight for the $k$ -th behavior. These weights can be adjusted based on the specific application scenario and requirements to better capture users’ latent interests in the recommender systems. By fine-tuning these weights, we can more accurately reflect the importance of each behavior type.

To capture the interaction information between users and items in different behavioral contexts, we construct a user-item interaction matrix for each behavior, denoted as $R_{1}, R_{2}, \dots, R_{K}$ . Specifically, $R_{k}$ represents the interaction matrix for the $k$ -th behavior, where $R_{k} (u, i)$ denotes the number of interactions between user $u$ and item $i$ in the $k$ -th behavior.

Weber-Fechner Law [28], a fundamental principle in psychology, states that the perceived difference in stimuli is proportional to the intensity of the stimulus. Mathematically, Weber-Fechner Law indicates a linear relationship between perceived differences and the logarithm of stimulus intensity. Therefore, for the default weight matrix, we calculate the weights for each behavior based on the proportion of the total occurrences of that behavior in the dataset. The default weight for this behavior is calculated as follows:

\begin{aligned} w_{k} = | l n (P_{k} * 100) | . \end{aligned}

(8)

Next, we multiply and sum each interaction matrix with its corresponding weight to generate a weighted score matrix $R \in R^{(| U | \times | I |)}$ .

\begin{aligned} R = \sum_{k = 1}^{K} w_{k} R_{k} . \end{aligned}

(9)

By performing matrix factorization on the weighted score matrix $R$ , the WA module is able to extract users’ interest profiles while preserving the original behavioral information. During this process, the matrix $R$ is decomposed into two lower-dimensional features, $F_{u}$ and $F_{i}$ , which represent the latent features of users and items, respectively. This decomposition reveals the underlying preferences users have towards items and allows the weights to adjust the impact of each type of behavior during the recommendation process. These weights not only enhance the accuracy of the recommendations but also enable the system to flexibly adjust based on the importance of different behaviors.

To avoid the loss of preference diversity due to the large number of items, we amplify the results by $| I |$ times. In this module, the purchase prediction of user $u$ for item $i$ is given by:

\begin{aligned} {\hat{y}}_{u, i}^{W A} = Sigmoid (F_{u} F_{i}^{T}) \times | I | . \end{aligned}

(10)

3.4 Fusion strategy

We assign weights to each module and then multiply and sum the weighted predictions of each module. Specifically, we introduce a participation parameter $α_{w}$ for the WA module. Assuming the predicted values of the EA module and the WA module for the likelihood of user $u$ and item $i$ having a purchase interaction in the next time period are ${\hat{y}}_{u, i}^{E A}$ and ${\hat{y}}_{u, i}^{W A}$ , respectively, the final prediction of the cd-MBRec model is calculated as follows:

\begin{aligned} {\hat{y}}_{u, i} = (1 - α_{w}) {\hat{y}}_{u, i}^{E A} + α_{w} {\hat{y}}_{u, i}^{W A} . \end{aligned}

(11)

3.5 Model training

3.5.1 Training the EA module

When training the model using the Mini-Batch method, the trainable parameters are divided into two parts: the initial user embedding and item embedding $Φ = E^{(0)}$ , which are affected by the Mini-Batch size, and other parameters $Θ$ . We employ the Bayesian Personalized Ranking (BPR) loss, which is a pairwise loss that maximizes the difference between positive samples (items with high user ratings) and negative samples (items with low user ratings) to improve the effectiveness of the recommender systems. During the training phase, we optimize the following objective using the Adam algorithm:

\begin{aligned} L_{BPR} = - \sum_{\bar{u} = 1}^{M} \sum_{\bar{i} \in N_{u}} \sum_{\bar{j} \in N_{u}} l n σ ({\hat{y}}_{\bar{u}, \bar{i}} - {\hat{y}}_{\bar{u}, \bar{j}}) + λ ({∥ E^{(0)} ∥}_{F}^{2} + {∥ Θ ∥}_{F}^{2}), \end{aligned}

(12)

where

{\hat{y}}_{\bar{u}, \bar{i}}

represents the final preference prediction of user

\bar{u}

for item

\bar{i}

, and

λ

is the coefficient for

L 2

regularization.

3.5.2 Training the WA module

By decomposing the weighted score matrix R, the WA module uses the Mean Squared Error (MSE) loss function and the Stochastic Gradient Descent (SGD) optimization algorithm to find the optimal low-dimensional latent vectors, i.e., the user embedding matrix $E_{u} \in R^{(| U | \times d)}$ and the item embedding matrix $E_{i} \in R^{(| I | \times d)}$ .

3.5.3 Complexity analysis

In the cd-MBRec model, the time complexity analysis is segmented into three principal components: 1) The EA module, responsible for capturing commonalities across user behaviors through graph convolutional networks, incurs a computational complexity of $O (K * L * | U | * | I | * d)$ , where $K$ denotes the number of behavior types, $L$ is the number of graph convolution layers, $| U |$ and $| I |$ are the cardinalities of the user and item sets, respectively, and $d$ is the dimensionality of the embeddings. 2) The WA module, which allocates weights to behaviors and performs matrix factorization on the interaction matrix, has a complexity of $O (T * | U | * | I | * d)$ . Here, $T$ represents the number of iterations for the matrix factorization process to converge. 3) The fusion strategy, which integrates the outputs from both modules, operates with a complexity of $O (| U | * | I |)$ , reflecting the computational overhead of combining predictions into a final output.

When integrating these components, the initial total time complexity of the model is expressed as $O (K * L * | U | * | I | * d) + O (T * | U | * | I | * d) + O (| U | * | I |)$ . Given that $K$ and $L$ are constants and assuming $T$ scales with the convergence requirements, the term $O (T * | U | * | I | * d)$ is typically the most significant, particularly for large datasets. Thus, the simplified expression predominantly focuses on this dominant factor $O (T * | U | * | I | * d)$ .

4. Experiments

We conducted experiments on two real-world datasets to evaluate the performance of the cd-MBRec model. Our goal was to answer the following questions:

–
RQ1: How does cd-MBRec perform compared to the state-of-the-art baselines?
–
RQ2: How do the sub-modules in cd-MBRec affect its recommendation performance?
–
RQ3: How do different configurations of key hyperparameters affect the performance of cd-MBRec?
–
RQ4: Is the behavior-commonality embedding aggregation strategy in the EA module generalizable?
–
RQ5: Is the Weber-Fechner law efficient in initializing weights in the WA module?

4.1 Experiment settings

Datasets. We apply two real-world datasets to evaluate the model performance objectively, known as UserBehavior1 and IJCAI.2

UserBehavior is a dataset from Taobao, one of the most popular e-commerce platforms in China. It contains four types of user-item relationships: page-view, favorite, add-to-cart, and purchase, each accompanied by a corresponding interaction timestamp.

IJCAI consists of users’ shopping logs on the Tmall platform for the six months leading up to and including Double Eleven day (November 11th). To avoid the over-large scale of input data, we processed only the data from November 1st to November 11th. This dataset includes four types of user-item relationships: click, add-to-favorite, add-to-cart, and purchase, along with the corresponding event dates.

For consistency and convenience, we unified the four types of behaviors across both datasets. We used pv for page-view and click, fav for favorite and add-to-favorite, cart for add-to-cart, and purchase for buy and purchase. A detailed description of the two datasets is given in Table 1.

Table 1
Statistics of the preprocessed datasets

Datasets Distribution Behaviors

pv fav cart purchase

UserBehavior Quantity 624,332 22,151 60,199 71,756

Proportion 80.2% 2.85% 7.73% 9.22%

IJCAI Quantity 262,915 17,331 558 21,656

Proportion 86.93% 5.73% 0.18% 7.16%

Datasets	Distribution	Behaviors
UserBehavior	Quantity	624,332	22,151	60,199	71,756
	Proportion	80.2%	2.85%	7.73%	9.22%
IJCAI	Quantity	262,915	17,331	558	21,656
	Proportion	86.93%	5.73%	0.18%	7.16%

Note that we did not remove dependencies between behaviors in the datasets, meaning purchase behaviors might appear earlier than pv, fav or cart behaviors. This is because real-world e-commerce data is segmented by time periods, and all behaviors within a given period can provide valuable information for recommendations. We considered the users’ purchase behaviors as targets since they are a good indicator of total sales on online retail websites.

Baselines. To examine the performance of our cd-MBRec model, we compared it with several popular models in recommender systems, including: –

MF [29]: The matrix factorization model is one of the most classic recommendation models.

–

NGCF [25]: This model utilizes a multi-layer graph neural network to stack and propagate embeddings on the user-item bipartite graph, enabling higher-order information aggregation.

–

LightGCN [26]: This model discards weights and non-linear activation functions from NGCF, using a weighted summation approach that demonstrates high efficiency and accuracy.

–

MB-GMN [17]: The model employs graph meta-networks to address the multi-behavior recommendation problem. It uses meta-networks to learn relationships between users and items and utilizes attention mechanisms to capture behavior correlations.

–

MB-NGCF: We enhance NGCF [25] to accommodate multiple behaviors present in datasets, leading to an advanced variant named MB-NGCF.

–

MB-LightGCN: We modify LightGCN [26] to consider multiple behaviors in datasets, resulting in an advanced variant called MB-LightGCN.

Among them, MF, NGCF, and LightGCN are designed for single-behavior recommendations, while MB-NGCF, MB-LightGCN, and MB-GMN are tailored for multi-behavior recommendations.

In multi-behavior recommendation research, numerous publicly available datasets exhibit distinct characteristics, influencing the choice of datasets across various studies employing different methodologies. Recent works such as [30, 31] utilized the IJCAI dataset for their experiments. However, they did not incorporate the UserBehavior dataset, leading to our decision to exclude these studies from our analysis.

Evaluation metrics. We used two metrics widely adopted in top-N recommendation tasks: Hit Rate (HR@N) and Normalized Discounted Cumulative Gain (NDCG@N), to evaluate the performance of our proposed cd-MBRec model and the baseline models.

Implementation details. In the cd-MBRec model, we trained the EA module and the WA module separately, using a fusion strategy for the final prediction. The EA module uses the Bayesian Personalized Ranking (BPR) loss function and the Adam optimizer, while the WA module applies the Mean Squared Error (MSE) loss function and Stochastic Gradient Descent (SGD) for optimization.

We varied the involving weight from 0 to 0.9 and selected the embedding dimension from 4, 16, 32, …, 1024. The parameters are set based on empirical knowledge, and their selection for optimal model performance is guided by experimental results. The cd-MBRec model uses the leave-one-out strategy to split the training and test sets. Specifically, the first item purchased by each user in the subsequent time period was kept as the test dataset, while all previous items were used for training. We randomly selected 99 items that were not purchased to serve as negatives samples.

4.2 Performance validation (RQ1)

We evaluated model performance in predicting users’ next purchasing items on the UserBehavior and IJCAI datasets. Tables 2 and 3 show the results of top-N item recommendation for different methods. The best and second-best results are highlighted in bold and underlined, respectively. Single-behavior, multi-behavior baselines, and our model are separated by horizontal lines, with improvement rates calculated at the bottom of the tables.

Table 2
Performance comparison of different models on the UserBehavior dataset

Models UserBehavior

HR@1 HR@5 HR@10 HR@20 NDCG@1 NDCG@5 NDCG@10 NDCG@20

MF 7.66 15.28 20.00 28.91 7.66 11.75 13.25 15.49

NGCF 12.80 19.75 25.66 35.39 12.80 16.30 18.19 20.63

LightGCN 13.13 19.07 24.53 33.86 13.13 16.13 17.88 20.21

MB-GMN 13.57 25.55 35.06 48.52 13.57 19.65 22.80 26.10

MB-NGCF 13.65 34.19 46.91 61.53 13.65 24.17 28.27 31.97

MB-LightGCN 33.28 46.73 52.56 61.20 33.28 40.56 42.44 44.60

cd-MBRec 41.46 50.78 57.13 65.70 41.46 46.15 48.21 50.33

Impr. 18.29% 4.46% 3.89% 3.06% 18.29% 9.13% 8.73% 8.12%

Models	UserBehavior
MF	7.66	15.28	20.00	28.91	7.66	11.75	13.25	15.49
NGCF	12.80	19.75	25.66	35.39	12.80	16.30	18.19	20.63
LightGCN	13.13	19.07	24.53	33.86	13.13	16.13	17.88	20.21
MB-GMN	13.57	25.55	35.06	48.52	13.57	19.65	22.80	26.10
MB-NGCF	13.65	34.19	46.91	61.53	13.65	24.17	28.27	31.97
MB-LightGCN	33.28	46.73	52.56	61.20	33.28	40.56	42.44	44.60
cd-MBRec	41.46	50.78	57.13	65.70	41.46	46.15	48.21	50.33
Impr.	18.29%	4.46%	3.89%	3.06%	18.29%	9.13%	8.73%	8.12%

Table 3

Performance comparison of different models on the IJCAI dataset

Models	IJCAI
	HR@1	HR@5	HR@10	HR@20	NDCG@1	NDCG@5	NDCG@10	NDCG@20
MF	1.05	6.49	11.51	21.33	1.05	3.72	5.34	7.80
NGCF	3.66	11.48	20.14	32.95	3.66	7.49	10.26	13.47
LightGCN	4.24	10.95	17.75	27.51	4.24	7.49	9.67	12.13
MB-GMN	3.84	15.72	26.66	42.12	3.84	9.82	13.21	17.25
MB-NGCF	19.36	43.26	56.83	70.73	19.36	31.57	35.96	39.48
MB-LightGCN	44.68	55.13	59.75	66.52	44.68	50.43	51.92	53.62
cd-MBRec	48.38	60.78	67.31	75.50	48.38	54.61	56.70	58.72
Impr.	10.43%	5.58%	6.02%	5.79%	10.43%	6.64%	6.76%	6.59%

We observed that most multi-behavior models (cd-MBRec, MB-GMN, MB-NGCF and MB-LightGCN) surpass models that rely on single behavior for recommendations (MF, LightGCN and NGCF), proving the necessity of utilizing different types of behaviors to get more accurate performance.

Our cd-MBRec stably outperforms baselines (average improvements of 4.96% in HR@10 and 7.75% in NDCG@10), demonstrating its superiority in multi-behavior recommendation tasks. By taking interaction matrices into account and assigning independent weights to each behavior, cd-MBRec effectively captures the commonality and diversity between multiple user behaviors. Additionally, the utilization of graph neural networks to optimize user and item embedding enables the identification of potential correlations between behaviors, leading to improved prediction of user interests in recommender systems.

4.3 Ablation study (RQ2)

We conducted experiments of sub-modules in the cd-MBRec framework by manually controlling the involvement parameter to 1 or 0. In cd-MBRec, the involvement parameters of EA and WA are dynamically adjusted based on specific requirements. Table 4 shows the recommendation performance of each module on the two datasets, with the following configurations.

Table 4
Ablation study of the cd-MBRec model

Models UserBehavior IJCAI

HR@10 NDCG@10 HR@10 NDCG@10

w/o EA 42.22 32.78 59.56 45.72

w/o WA 56.96 47.89 66.95 56.70

cd-MBRec 57.13 48.21 67.31 56.70

Models	UserBehavior	IJCAI
w/o EA	42.22	32.78	59.56	45.72
w/o WA	56.96	47.89	66.95	56.70
cd-MBRec	57.13	48.21	67.31	56.70

–

w/o EA: The involvement parameter of the EA module is set to 0.

–

w/o WA: The involvement parameter of the WA module is set to 0.

The results validate the effectiveness of each sub-module within cd-MBRec. By integrating the EA and WA modules, the cd-MBRec framework outperforms each module individually across both datasets. This demonstrates the model’s capability to achieve a fine balance in capturing the commonalities and diversities of user interactions, thereby optimizing recommendation performance.

Furthermore, we also found that the EA module outperforms the WA module since the model works worse without the former. This could be attributed to the ability of the EA module to more efficiently capture the high-order and low-order interaction information between users and items using multi-behavior graph neural networks.

Figure 3.

Performance of cd-MBRec with different weights assigned to the WA module.

4.4 Hyperparameter analysis (RQ3)

We explored various configurations of key parameters in the cd-MBRec model, including the involving weight of the WA and EA modules and the embedding dimension in the WA module. Our objective is to comprehensively evaluate the performance of the proposed cd-MBRec model under different hyperparameter settings and uncover the internal relations between these settings and the results.

The impact of involving weight $α_{w}$ . We varied $α_{w}$ from 0 to 0.9 to examine the collaboration and division between the EA and WA modules, as illustrated in Fig. 3. The y-axis shows the increase in performance ratio relative to the default WA parameter setting. As $α_{w}$ increases, the weight assigned to the WA module also increases. This investigation allows us to optimize the model’s performance according to specific requirements in recommendation scenarios.

Our findings underscore the importance of appropriately adjusting the involving weight in different application scenarios to achieve an optimal balance between the EA and WA modules, thus optimizing recommendation performance.

In the UserBehavior dataset for single-item recommendation ( $N =$ 1), we observed a gradual improvement in the performance of cd-MBRec as the involvement weight increases from 0 to 0.2. This implies that within this weight range, integrating the EA and WA modules enhances recommendation accuracy. However, as $α_{w}$ becomes larger, there is as risk of overlooking the higher and lower order information and structural details provided by the EA module, potentially diminishing recommendation performance. For multi-item recommendation ( $N >$ 1), cd-MBRec shows slight fluctuations in Hit Rate compared to using the EA module alone, but demonstrated superior performance in the NDCG metric when $α_{w}$ is below 0.6. Thus, adjusting the involving weight significantly influences the sequence and quality of recommendation results.

In the IJCAI dataset, we observed a decreasing trend in cd-MBRec’s recommendation performance as the involving weight $α_{w}$ increases. This trend suggests that the recommendation outcomes tend to rely more on the EA module rather than the WA module. One possible explanation is that the user behavior characteristics before the Double Eleven day are more complex and diverse, making it challenging to establish suitable weights in the WA module. Meanwhile, users’ higher and lower order information plays a crucial role in this context. Furthermore, we observed that the decline in NDCG rate surpasses the decline in Hit Rate as the involving weight increases. This indicates that before the shopping festival, the sequence and quality of recommendation heavily depend on the information provided by the EA module.

Figure 4.

Performance of cd-MBRec with different dimensions of the hidden state.

The impact of embedding dimension $d$ . We explored the embedding dimension of our cd-MBRec model across the range \{4, 16, 32, 64, 128, 256, 512, 1024\}. The performance improvement ratio relative to the default parameter setting of $d =$ 256 is presented in Fig. 4.

On both datasets, we observed improved results as the embedding dimension $d$ increases, particularly noticeable when the recommendation list contains fewer items ( $N$ is small). A higher embedding dimension enhances our model’s ability to capture latent features, thereby better analyzing and representing the intricate behavior relationships between users and items across two datasets. In this context, the WA module shows reduced sensitive to noise and outliers in the training data within a high-dimensional space, migrating potential negative impacts associated with a higher embedding dimension.

However, it is important to note that a larger embedding dimension does not necessarily lead to significant improvements in recommendation performance when $N$ is large. Moreover, higher dimensions may increase computational complexity and resource consumption, potentially leading to over-fitting issues if the embedding dimension is excessively high. Therefore, selecting an appropriate embedding dimension is crucial, tailored to the specific problems and scenarios at hand.

4.5 Generality of the commonality extracting strategy (RQ4)

To investigate the generality of the multi-behavior commonality extracting strategy employed in the EA module, we conducted comparative experiments on the UserBehavior and IJCAI datasets. LightGCN and MB-GMN were used to model behaviors between users and items. The experimental data are presented in Table 5.

Table 5
Impact of the commonality extracting strategy on the performance of LightGCN and MB-GMN

Models UserBehavior IJCAI

HR@10 NDCG@10 HR@10 NDCG@10

LightGCN Single-behavior 24.53 17.88 17.75 9.67

Multi-behavior 52.56 42.44 59.75 51.92

Multi-behavior $+$ Integration 54.44 45.83 63.01 55.43

MB-GMN Single-behavior / / / /

Multi-behavior 35.06 22.80 26.66 13.21

Multi-behavior $+$ Integration 49.94 32.30 26.11 13.56

Models	UserBehavior	IJCAI
LightGCN	Single-behavior	24.53	17.88	17.75	9.67
	Multi-behavior	52.56	42.44	59.75	51.92
	Multi-behavior $+$ Integration	54.44	45.83	63.01	55.43
MB-GMN	Single-behavior	/	/	/	/
	Multi-behavior	35.06	22.80	26.66	13.21
	Multi-behavior $+$ Integration	49.94	32.30	26.11	13.56

–

Single-behavior: This approach focuses on a single specific behavior, with purchase as the target behavior. Since MB-GMN is designed for multi-behavior recommendations, its performance metrics are not applicable in the single-behavior context.

–

Multi-behavior: User’s inputs are derived from the combination of multiple behaviors. LightGCN is adapted into MB-LightGCN, as detailed in Tables 2 and 3.

–

Multi-behavior $+$ Integration: Each behavior is treated as a separate input. An additional interaction matrix indicated whether interactions exist between users and items, leveraging the proposed commonality extraction strategy in this work.

Figure 5.

Comparison of different weight allocation strategies.

The results indicate that considering multiple behaviors together yields better recommendation performance compared to focusing on a single behavior alone. Our integration strategy, which includes the interaction matrix, explicitly accounts for behavior commonality and outperforms the multi-behavior fusion approach. This demonstrates its superiority in exploring user interests through various behaviors.

4.6 Effectiveness of the weber-fechner law (RQ5)

Due to the significant disparity in the proportions of different behaviors within the dataset, adjusted the weights in the WA module to account for this diversity, thereby mitigating the data sparsity issue. The initial weights can be set either manually or automatized using the Weber-Fechner law. In Fig. 5, we evaluate the model’s performance using three different weight allocation strategies.

Specifically, the label 1:1:1:1 indicates that all user behaviors are equally weighted. The label 2:1:1:1 means that the weight of pv is doubled (since pv is the most frequently occurring behavior in both datasets), while the weights of other behaviors remain unchanged. The Weber columns display the results of weights calculated using the Weber-Fechner law, which is the default in our model.

Increasing the weight proportion of pv behavior reveals distinct trends in the UserBehavior and IJCAI datasets, suggesting that weight distribution should be adaptively adjusted based on different situations. Weights determined by the Weber-Fechner Law achieve the best performance, demonstrating the superiority of our proposed method.

Moreover, the fixed weight allocation provided by the Weber-Fechner Law enhances cd-MBRec’s interpretability, making the model more comprehensible and manageable. This distinct feature gives cd-MBRec a significant advantage over other models.

5. Conclusion

In this paper, we introduce cd-MBRec, an innovative recommendation model designed to explicitly capture both the commonalities and diversities inherent in multi-behavioral user interactions. Our model employs two key strategies: the behavior-commonality embedding aggregation strategy, which effectively reduces noise from user habits, and the diversity-enhancing strategy, which emphasizes the significance of multiple behaviors in various contextual scenarios. Extensive experiments on two real-world datasets demonstrate that cd-MBRec significantly outperforms state-of-the-art methods. Future work will focus on further exploring the applicability and effectiveness of multi-behavioral modeling and denoising techniques across a broader range of applications.

Footnotes

Acknowledgments

This work is partly supported by the Shanghai Science and Technology Innovation Action Plan Project (No. 22511100700).

Compliance with Ethical Standards

Conflict of interest statement. No conflict of interest exists in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my coauthors that the work described is original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed. And this article does not contain any studies with human participants performed by any of the authors.

Notes

References

Covington

Adams

Sargin

, Deep neural networks for youtube recommendations, in: Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 191–198.

Zhou

Qin

Chen

Zhang

, Online social media recommendation over streams, in: IEEE 35th International Conference on Data Engineering, 2019, pp. 938–949.

Karimi

Jannach

Jugovac

, News recommender systems-survey and roads ahead, Information Processing & Management 54(6) (2018), 1203–1227.

Gao

Gan

Chen

Feng

Chua

T.-S.

Jin

, Neural multi-task recommendation from multi-behavior data, in: IEEE 35th international Conference on Data Engineering, 2019, pp. 1554–1557.

Loni

Pagano

Larson

Hanjalic

, Bayesian personalized ranking with multi-channel user feedback, in: Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 361–364.

Meng

Zhang

Guo

Liu

Zhang

Zheng

Tang

Zhang

, Hierarchical projection enhanced multi-Behavior recommendation, in: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 4649–4660.

Wei

Huang

Xia

Zhao

Yin

, Contrastive meta learning with behavior multiplicity for recommendation, in: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, 2022, pp. 1120–1128.

Zhang

Yao

Sun

Tay

, Deep learning based recommender system: A survey and new perspectives, ACM Computing Surveys 52(1) (2019), 1–38.

Wang

Zhang

Wang

, A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation, IEEE Transactions on Knowledge and Data Engineering 35(5) (2023), 4425–4445.

10.

Said

Jain

B.J.

Narr

Plumbaum

, Users and noise: The magic barrier of recommender systems, in: Proceedings of the 20th International Conference on User Modeling, Adaptation, and Personalization, 2012, pp. 237–248.

11.

Bag

Kumar

Awasthi

Tiwari

M.K.

, A noise correction-based approach to support a recommender system in a highly sparse rating environment, Decision Support Systems 118 (2019), 46–57.

12.

Guo

Tang

, DeepFM: a factorization-machine based neural network for CTR prediction, in: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 1725–1731.

13.

Kang

W.-C.

McAuley

, Self-attentive sequential recommendation, in: IEEE International Conference on Data Mining, 2018, pp. 197–206.

14.

Xie

Zhu

Chen

Zhang

Zhuang

Lin

, Multi-view multi-behavior contrastive learning in recommendation, in: Proceedings of the International Conference on Database Systems for Advanced Applications, 2022, pp. 166–182.

15.

Gao

Gan

Chen

Feng

Chua

T.-S.

Yao

Song

Jin

, Learning to recommend with multiple cascading behaviors, IEEE Transactions on Knowledge and Data Engineering 33(6) (2019), 2588–2601.

16.

Jin

Gao

Jin

, Multi-behavior recommendation with graph convolutional networks, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 659–668.

17.

Xia

Huang

Dai

, Graph meta network for multi-behavior recommendation, in: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 757–766.

18.

Chen

Zhang

Wang

Liu

, Graph heterogeneous multi-relational recommendation, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 3958–3966.

19.

Kipf

T.N.

Welling

, Semi-Supervised Classification with Graph Convolutional Networks, in: International Conference on Learning Representations, 2017.

20.

Wang

Shi

Xiao

, Self-supervised graph neural networks for multi-behavior recommendation, in: Proceedings of the 31st International Joint Conference on Artificial Intelligence, 2022, pp. 2052–2058.

21.

Hamilton

Ying

Leskovec

, Inductive representation learning on large graphs, in: Proceedings of the Advances in Neural Information Processing Systems, 2017, pp. 1025–1035.

22.

Veličković

Cucurull

Casanova

Romero

Liò

Bengio

, Graph attention networks, in: Proceedings of the International Conference on Learning Representations, 2018.

23.

Tang

Zhu

Wang

Xie

Tan

, Session-based recommendation with graph neural networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 346–353.

24.

Chang

Gao

Zheng

Hui

Niu

Song

Jin

, Sequential recommendation with graph neural networks, in: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 378–387.

25.

Wang

Feng

Chua

T.-S.

, Neural graph collaborative filtering, in: Proceedings of the 42nd international ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 165–174.

26.

Deng

Wang

Zhang

Wang

, Lightgcn: simplifying and powering graph convolution network for recommendation, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 639–648.

27.

Peng

Sugiyama

Mine

, Less is more: Reweighting important spectral graph features for recommendation, in: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 1273–1282.

28.

Dehaene

, The neural basis of the Weber-Fechner law: A logarithmic mental number line, Trends in Cognitive Sciences 7(4) (2003), 145–147.

29.

Ran

Wang

Zhang

L.Y.

, A differentially private nonnegative matrix factorization for recommender system, Information Sciences 592 (2022), 21–35.

30.

Xia

Huang

Pei

, Multi-behavior sequential recommendation with temporal graph transformer, IEEE Transactions on Knowledge and Data Engineering 35(6) (2023), 6099–6112.

31.

Zhang

Chen

Zhao

Han

, Denoising and prompt-tuning for multi-behavior recommendation, in: Proceedings of the World Wide Web Conference, 2023, pp. 1355–1363.

32.

Yuan

Guo

Liu

Tang

, Multi-behavior sequential transformer recommender, in: Proceedings of the 45th international ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 1642–1652.

Datasets	Distribution	Behaviors
		pv	fav	cart	purchase
UserBehavior	Quantity	624,332	22,151	60,199	71,756
	Proportion	80.2%	2.85%	7.73%	9.22%
IJCAI	Quantity	262,915	17,331	558	21,656
	Proportion	86.93%	5.73%	0.18%	7.16%

cd-MBRec: Enhancing multi-behavior recommendation by explicitly modeling commonality and diversity

Abstract

Keywords

1. Introduction

2.1 Multi-behavior recommendation

2.2 Graph-based recommendation

3. Methodology

3.1 Model architecture

3.5.1 Training the EA module

3.5.3 Complexity analysis

4. Experiments

Table 1 Statistics of the preprocessed datasets Datasets Distribution Behaviors pv fav cart purchase UserBehavior Quantity 624,332 22,151 60,199 71,756 Proportion 80.2% 2.85% 7.73% 9.22% IJCAI Quantity 262,915 17,331 558 21,656 Proportion 86.93% 5.73% 0.18% 7.16%

Table 4 Ablation study of the cd-MBRec model Models UserBehavior IJCAI HR@10 NDCG@10 HR@10 NDCG@10 w/o EA 42.22 32.78 59.56 45.72 w/o WA 56.96 47.89 66.95 56.70 cd-MBRec 57.13 48.21 67.31 56.70

5. Conclusion

Footnotes

Acknowledgments

Compliance with Ethical Standards

Notes

References

Table 1
Statistics of the preprocessed datasets

Datasets Distribution Behaviors

pv fav cart purchase

UserBehavior Quantity 624,332 22,151 60,199 71,756

Proportion 80.2% 2.85% 7.73% 9.22%

IJCAI Quantity 262,915 17,331 558 21,656

Proportion 86.93% 5.73% 0.18% 7.16%

Table 4
Ablation study of the cd-MBRec model

Models UserBehavior IJCAI

HR@10 NDCG@10 HR@10 NDCG@10

w/o EA 42.22 32.78 59.56 45.72

w/o WA 56.96 47.89 66.95 56.70

cd-MBRec 57.13 48.21 67.31 56.70