A feature-aware long-short interest evolution network for sequential recommendation

Abstract

Recommendation systems are an effective solution to deal with information overload, particularly in the e-commerce sector, in which sequential recommendation is extensively utilized. Sequential recommendations aim to acquire users’ interests and provide accurate recommendations by analyzing users’ historical interaction sequences. To improve recommendation performance, it is vital to take into account the long- and short-term interests of users. Despite significant advancements in this domain, some issues need to be addressed. Conventional sequential recommendation models typically express each item with a uniform embedding, ignoring evolutionary patterns among item attributes, such as category, brand, and price. Moreover, these models often model users’ long- and short-term interests independently, failing to adequately address the issues of interest drift and short-term interest evolution. This study proposes a new model, the Feature-aware Long-Short Interest Evolution Network (FLSIE), to address the above-mentioned issues. Specifically, the model uses explicit feature embedding to represent item attribute information and employs a two-dimensional (2D) attention mechanism to distinguish the significance of individual features in a specific item and the relevance of each item in the interaction sequence. Furthermore, to avoid the issue of interest drift, the model employs a long-term interest guidance mechanism to enhance the representation of short-term interest and adopts a gated recurrent unit with attentional update gate to model the dynamic evolution of users’ short-term interest. Experimental results indicate that our presented model outperforms existing methods on three real-world datasets.

Keywords

Sequential recommendation behavior sequence feature-level preference attention mechanism long- and short-term interests

1. Introduction

With the rapid growth of information, recommendation systems play an important role in e-commerce [1], news [2], social media [3], and other online platforms. A Sequential Recommendation (SR) is defined as recommending the subsequent item based on the user’s sequential historical behavior, aiming to improve user experience and bring colossal business value to the platform [4]. In the real world, users have both persistent long-term interests and dynamic short-term interests. Therefore, it is critical to accurately model the evolution of users’ long- and short-term interests from their complex behavior.

In recent studies, researchers have presented a number of SR models that aim at modeling user’s long-term and short-term interests and capturing the evolution patterns of users’ historical behavior. These models include traditional methods, neural network-based methods, and transformer-based methods. First, traditional methods mainly rely on Markov chains to learn users’ short-term interest [5, 6], but they ignore users’ long-term interest and cannot accurately capture dynamic changes in complex scenarios. Second, neural network-based methods include Recurrent Neural Network (RNN)-based methods [7, 8] and Convolutional Neural Network (CNN)-based approaches [9] that construct complex relationships by learning a large amount of user interaction behavior and capturing the evolution of short-term interests [10]. However, these methods only consider the user’s recent actions, resulting in the model having solely short-term memory and recommending only items related to the user’s recent behavior. Third, transformer-based models [11, 12] adopt a self-attention mechanism and have achieved satisfactory results. However, the self-attention mechanism has not considered the position information of sequences. Most algorithms fail to distinguish between users’ long-term and short-term interests and instead model the overall interaction sequence of the user. However, this distinction is crucial for practical sequential recommendations [13]. Despite the ability of these sequential methods to achieve satisfactory results, two challenges still need to be addressed.

This first challenge is how to capture users’ long- and short-term interests from the feature level. Most existing solutions use the item ID as the unique attribute for each item. However, each item is associated with multiple features, such as category, brand, and price. Essentially, user’s behavior is driven by the evolution of users’ preferences for item features. Therefore, using a uniform embedding to represent each item does not distinguish the importance of different item features. As illustrated in Fig. 1, the user initially chooses to browse the iMac and wants to browse the same brand. Thus, when browsing similar items (e.g., two mobile phones), users are more likely to click on those items with attributes they actually care about (e.g., brand and price). After purchasing a phone, the user’s needs may shift to phone supplements (e.g., headphones and chargers), so the user can shop according to the product type. Finally, while browsing headphone merchandise, the user will purchase AirPods. In this example, the user ends up interacting with earphones because he or she is interested in the Apple brand. AirPods are commonly regarded as an essential iPhone accessory, significantly elevating the demand for AirPods among users. Thus, each user has complex dependencies on different attributes, and the change in item attributes will dramatically affect the user’s decision.

Figure 1.

An example of user sequential behavior and an illustration of the evolution of user preferences on item features.

The second challenge addressed in this research is how to model the evolution of long- and short-term interests. Most approaches typically model the user’s short-term and long-term interests independently and then concatenate them to reflect the their final interests. However, the linear concatenation between long-term and short-term interests lacks a modeling of user habits, which leads to interest drift [14, 15]. GRU4Rec [7] utilizes a gated recurrent unit (GRU) to extract information from user interaction sequences, while Caser [9] embeds a series of recent items into an “image” in the time and latent spaces and then uses convolutional filters to learn the sequence patterns, similar to using a convolution filter to learn local features of the image. Although these methods can successfully extract the main interests of users from the interaction sequence, they cannot model the evolution process of users’ interests and solve the problem of interest drift. For example, short-term interests take precedence when users browse similar items consecutively, whereas when they switch to substantially different items, users’ behavior is primarily influenced by long-term interests. As illustrated in Fig. 1, the user may consistently gravitate toward electronic products (a reflection of their long-term interest) while simultaneously becoming interested in clothes at a given moment (a short-term interest). Therefore, it is crucial to adaptively combine short- and long-term interests to predict future interactions.

In this study, we propose a novel model, namely FLSIE, to address the abovementioned issues. FLSIE can effectively leverage item features to capture users’ feature-level preferences. Moreover, FLSIE designs a long- and short-term interests evolution mechanism that not only captures the long-term and short-term interests of users from their behavioral sequences but also strengthens the influence of users’ habits on the evolution of their preferences. Finally, accurate prediction/recommendation is achieved by using user dynamic preference. The proposed method FLSIE includes two main parts.

For addressing the first challenge, we draw inspiration from the success of attribute-aware neural models [16, 17] and utilize explicit feature embedding to represent item information. This approach provides a solid foundation for accurately representing user interests. Furthermore, we adopt a 2D attention mechanism that includes vertical attention to obtain user feature-level preferences and horizontal attention to distinguish the significance of items in the interaction sequence. Therefore, the model can successfully capture the user’s short- and long-term interests in the attributes of each item within the sequences.

To solve the second challenge, we adopt the approach of user dynamic interest modeling [18, 19]. We analyze users’ long- and short-term interests to simulate the user interest evolution. Specifically, our model utilizes GRU to encode short-term preferences and explicitly extract users’ current interests. Then, we use a user’s habits to guide short-term preferences and reduce the interference of interest drift. Finally, we employ a gated recurrent unit with attentional update gate (AUGRU) [20, 10] to capture the evolution of dynamic preferences.

In summary, this study offers the following main contributions:

•

We introduce explicit feature embedding and utilize an attribute-aware attention mechanism to learn the feature representation of items and then capture users’ feature-level preferences from their interaction sequences.

•

We propose a novel FLSIE that captures user preferences from their historical behavior sequence. Moreover, AUGRU is utilized to capture the evolution of user preferences. We also introduce a joint learning mechanism that combines long-term interests and current interests to learn the evolution of user interests.

•

Extensive experiments are conducted on three public recommendation datasets (Amazon, Gowalla, and Yelp). The experimental results demonstrate that the FLSIE model achieves significant performance improvements in HR and NDCG compared with existing baselines.

2. Related work

This section provides an overview of the existing research on sequential recommendation. Considering that user preferences are constantly changing, SR models mainly rely on Markov chains, neural networks, and attention mechanisms to capture users’ short-term interest, long-term interest, and long- and short-term interests.

2.1 Sequential recommendation

In early studies, methods based on Markov chains were utilized to capture the user’s potential interest in the user interaction sequences. For example, Factorizing Personalized Markov Chains (FPMC) [5] is a notable next basket of recommendation algorithms that combines Matrix Factorization (MF) and Markov Chains (MC) to simulate a users’ general preferences and mine their sequential behavior, respectively. He et al. [21] employed first-order MCs, while He et al. [22] and He and McAuley et al. [6] utilized higher-order MCs to consider more dependencies for determining user preferences. However, those MC-based approaches only considered the relationship between users and items and only utilized the correlation between adjacent items to generate recommendations. Therefore, the methods neither could take into account the dependency relationships in the user behavior sequences nor capture the long-term preferences of users.

On the one hand, Recurrent Neural Networks (RNN) have been extensively utilized in recommendation systems because of their powerful feature extraction and generalization capabilities. For example, Hidasi et al. [7] used GRU to encode the user’s recent session to obtain the user’s current interest. Yuan et al. [8] classified the context information and redefined the GRU unit’s update gate and reset gate to model the dynamic user interests. Quadrana et al. [23] presented the HRNN, and Ma et al. [24] developed the HGN, both employing hierarchical RNN to capture users’ interests. Nevertheless, the RNN-based sequential recommendation model relied on the recurrent structure of the current action, resulting in high training costs, although it can capture the dependencies between items.

On the other hand, Convolutional Neural Networks (CNN) and Graph Neural Networks (GNN) are commonly used in sequential recommendation. Tang et al. [9] proposed a convolutional sequential embedding recommendation model called Caser that employs CNN to extract sequence features. Although Caser can extract higher-level features by using fully connected layers, it only considers the conditional probability distribution of the last item in the sequence. Consequently, capturing complex relationships and long-term dependencies in a sequence can be difficult. Moreover, Wu et al. [25] proposed SR-GNN that utilizes GNN to extract item-embedding vectors and combine long-term and short-term preferences via the attention layer. These sequential recommenders can effectively identify the user’s primary interests by utilizing sequential interactions. However, the aforementioned models fail to capture how user interests evolve over time.

2.2 Attention mechanism for sequential recommendation

Recently, transformer-based methods [26] have successfully captured long-term dependencies in sequences. Specifically, the self-attention mechanism module has achieved state-of-the-art results in sequence recommendation. For instance, Kang et al. [11] developed SASRec that utilizes a self-attention mechanism to consider the relationship between items for adaptively modeling user behavior. Sun et al. [12] developed BERT4Rec that utilizes bidirectional self-attention modules to capture pairwise relationships between user interactions and items. Li et al. [27] extended SASRec by increasing the items’ absolute position embeddings and time interval embeddings. Li et al. [28] and Liu et al. [29] employed an encoder-decoder framework to model user-item interactions and capture short-term preferences. Furthermore, Lin et al. [30] used a dual-interest factorization-heads attention sequential recommendation model that captures shifting patterns of behavior through positive and negative feedback. In their work, skipping and not skipping can be viewed as negative and positive feedback. Compared with RNN and CNN, the attention mechanism has higher parallelism and can better model users’ diverse preferences. However, the abovementioned methods mainly focus on capturing users’ interests from recent behavior, failing to capture users’ general interests and complex conversion patterns.

2.3 Modeling on item attributes and evolution

In the realm of sequential recommendation, modeling user interests involves two critical aspects: Attribute-aware and interaction behavior. On the one hand, additional item attributes are used to enhance the potential representation of vectors, which have been proven to play a critical role in alleviating sparse interaction data. For instance, Zhang et al. [31] adopted feature-based self-attention to learn how features transfer between items. Moreover, some studies [32, 33, 34, 35] adopted different decoupled representation learning methods based on the correlation between items and their attributes to achieve higher expressive power. Furthermore, Zhu et al. [36] utilized a disentangled self-attention mechanism to simulate the interactions of complex features. Although sequential recommendation methods mentioned above have been successful, they ignore the evolution of user interests among item attributes.

On the other hand, the evolution of users’ interests can be captured using rich interaction information between users and items. For instance, Lv et al. [17] and Ying et al. [37] divided user’s interaction sequences into long- and short-term subsequences and then explicitly extracted embeddings for long- and short-term interests. Ma et al. [38] modeled long- and short-term interests of users, utilizing bilinear functions to accurately capture the co-occurrence patterns between relevant items. Some works [39, 40] not only focused on the changes in user sequences but also paid attention to the evolution of user preferences over time. Ma et al. [1] divided user sequences based on attribute values and then modeled the item evolution over attribute dynamics with these users. These works have successfully extracted the main interests of users through sequential interactions, but it ignores the evolution of users’ short-term interest.

Different from the aforementioned works, our method introduces a unified framework for capturing the long- and short-term evolution of user behavior. This framework partitions the user’s behavior sequences into long- and short-term subsequences and employs attention mechanisms and RNN to extract the user’s long-term interest and short-term interest in the item attributes. Moreover, AUGRU is utilized to flexibly combine long- and short-term interests. The experimental results show that this approach can achieve a significant improvement in recommendation accuracy and provide a better recommendation experience.

3. Problem statement

Before delving into the discussion of our proposed model, let us first clearly define the problem and basic concepts. Formally, assume $U=\{u_{1},u_{2},u_{3},\ldots,u_{|u|}\}$ and $V=\{v_{1},v_{2},v_{3},\ldots,v_{|v|}\}$ denote the user set and item set, respectively. $S^{u}=\{i_{1},i_{2},i_{3},\ldots,i_{|S|}\}$ represents a chronological sequence of user-item interactions, and $i_{t}\in V$ records the t-th item interacted with by the user. For user u, his and her continuous interactions (or sessions) are marked as $L^{u}=\{S_{1}^{u},S_{2}^{u},S_{3}^{u},\ldots,S_{T}^{u}\}$ , where T represents the total time step length. $S_{t}^{u}\in S^{u}(t\in[1,T])$ indicates the interaction item set of the user at time step t. Therefore, given a fixed time step t, the item set $S_{t}^{u}=(i_{1},i_{2},i_{3},\ldots,i_{|S_{t}^{u}|})$ can reflect his short-term interest. This factor is crucial in predicting the next purchased item. Conversely, the set of items up to the earlier time step t, denoted as $L_{t-1}^{u}=S_{1}^{u}\cup S_{2}^{u}\cup\ldots\cup S_{t-1}^{u}$ , is significant in providing insights into the long-term interest (i.e., general taste) of the user.

In summary, our goal is to encode the user’s short-term and long-term behavior sequences $S_{t}^{u}$ and $L_{t-1}^{u}$ into corresponding short-term and long-term interest vectors $h_{u}^{S}$ and $h_{u}^{L}$ , respectively. In this manner, we can obtain the user’s final interests based on $h_{u}^{S}$ and $h_{u}^{L}$ and accordingly recommend the next possible interactive items to the user. Table 1 presents the key notations used in this research.

Table 1
List of key notations

Notation	Description
U, V	Set of users and items, respectively
$S^{u}$	Historical interaction sequences of user u
$S^{u}_{t}$	Short-term behavioral sequences of u
$L^{u}_{t-1}$	Long-term behavioral sequences of u
$h^{S}_{u}\in R^{d}$	Short-term interest vector
$h^{L}_{u}\in R^{d}$	Long-term interest vector
$d_{f}$	Attribute’s latent dimension
$f_{vk}\in R^{d_{f}}$	Explicit embedding of feature k on item v
s, l	Short-term and long-term maximum sequence lengths, respectively
$F_{u}\in R^{d}$	User’s final interest state

Figure 2.

Overall structure of FLSIE.

4. Feature-aware long-short interest evolution network (FLSIE)

In this section, we discuss the various components of the proposed FLSIE model. Figure 2 shows the overall structure of the FLSIE model, which comprises the side information embedding module, the long-term interest modeling module, the short-term interest evolution module, and the prediction module. Initially, we split the user’s historical interaction records into two sequences: fixed-length long-term interaction sequences $L^{u}_{t-1}$ and short-term interaction sequences $S^{u}_{t}$ . An attention mechanism is employed to fuse the side information for enhancing the item embedding representation. Subsequently, the user’s long- and short-term interests are extracted using the attention mechanism and GRU, respectively. AUGRU is utilized to model the evolution of users’ short-term interests. Finally, prediction is performed on the basis of the user’s final interest, and the model parameters are trained using the joint BPR loss function. In the subsequent sections, each model component is introduced in detail.

4.1 Input embedding with side information

In this subsection, we initially introduce how to explicitly embed side information from the system, such as item attributes. This process involves transforming each attribute into its corresponding embedding representation, enhancing the model’s ability to convey the significance and impact of these attributes. Subsequently, by employing an attention mechanism, we can more precisely capture users’ varying levels of attention across different attributes, thereby constructing a more accurate representation of user feature-level preferences.

4.1.1 Explicit feature embedding

In e-commerce recommendation scenarios such as Taobao, users pay attention to a certain product and its attributes, including category, price, brand, and so on. Different user attributes, such as gender, age, and city, will generate different interests in the items. Side information, such as user and item attributes, can provide additional information for improving prediction accuracy. Nonetheless, the majority of existing sequential models continue to utilize a unified embedding to represent each item. These models fail to distinguish the importance of different item features. Therefore, we utilize an item-embedding approach based on various feature scales of the item. Formally, let K be the number of features in the system. Then, we denote the explicit embedding of feature $k$ for item $v$ as:

$\displaystyle f_{vk}=W_{k}^{f}x_{v}^{f},$ (1)

where $\forall k\in[1,K]$ , $x_{v}^{f}$ is a one-hot vector for the attribute of item $v$ , $W_{k}^{f}$ is the transformation matrix of feature $k$ , and $f_{uk}\in R^{d_{f}\times 1}$ .

Similarly, we generate the embedding of feature $p$ for user $u$ as:

$\displaystyle f_{u\rho}=W_{\rho}^{f}x_{u}^{f},$ (2)

where $x_{u}^{f}$ is a one-hot vector for the attribute of user $u$ , and $W_{\rho}$ is the transformation matrix of feature $\rho$ .

Then, we concatenate features of different scales of the user to represent the user’s embedding which is defined as:

$\displaystyle e_{u}=\operatorname{concat}(\{{f}_{up}\mid p\in\mathcal{P}\}).$ (3)

4.1.2 Side information fusion

On the basis of explicit feature embedding, we utilize an attention mechanism to acquire user feature-level preferences. In traditional attention sequential recommendation models, users’ attention to items is usually horizontally distributed along the sequence, allowing for distinguishing these users’ attention to different items. However, the previous models often overlook the importance of different item features. By contrast our model associates each item with multiple features. To describe users’ interests in more detail from the perspective of features, we design vertical attention for features and horizontal attention for items. Formally, for the behavior sequences $S_{u}=\{i_{1},i_{2},i_{3},\ldots,i_{|S|}\}$ , the feature embedding sets $F_{v^{m}}=(f_{v^{m}1},f_{v^{m}2},\ldots,f_{v^{m}K})^{T}$ can be obtained from Eq. (1), where $v^{m}(m\in[1,|S|])$ .

Vertical Attention.

The purpose of implementing vertical attention is to enhance item embedding by assigning weights to different item features. Specifically, after fusing the side information, the embedding of item $e_{v^{m}}\in R^{d\times 1}$ is calculated as follows:

$\displaystyle e_{v^{m}}=\sum_{k=1}^{K}\alpha_{k}^{v}f_{v^{m}k},$ (4) $\displaystyle\alpha_{k}^{v}=\frac{\exp(\Pr(e_{u},f_{v^{m}k}))}{\sum_{j=1}^{K}% \exp(\Pr(e_{u},f_{v^{m}j}))},$ (5)

where $\Pr(\cdot)$ is a projecting function and returns a scalar of the user and feature embeddings. By utilizing user embedding $e_{u}$ as the query vector and normalizing the similarity between $f_{v^{m}j}$ and u via the softmax function, we can calculate the attention score $\alpha_{k}^{v}$ to determine the importance of feature j for u. Moreover by using the vertical attention method, we can assign much lower attention weights to unimportant features, thus reducing their impact on the enhanced item embedding.

Horizontal Attention.

The purpose of horizontal attention is to discern the relative significance of various items within a specific sequence. Assume that the enhanced historical item embeddings are $(e_{v^{1}},e_{v^{2}},\ldots,e_{v^{|S|}})$ based on Eq. (4), we can then use them to model users’ long-term interests. Further details will be provided in the next section.

4.2 Long-term interest modeling with horizontal attention

The user’s long-term interest indicates the user’s historical habits. In the online shopping scenario, users’ interaction is characterized by randomness, but in most cases, they will interact with the items they are interested in. Therefore, unrelated actions can affect the expression of a user’s long-term interest. To capture users’ long-term interest, we use horizontal attention for items in historical interactions to reduce the impact of irrelevant behavior.

This model leverages the item set $L_{t-1}^{u}$ , with which the user has previously interacted. The user’s embedding $e_{u}$ is taken as the query vector to assign individual weights to each item. The embeddings of these items are then aggregated to generate the user’s long-term interest vector $h_{u}^{L}$ . Formally, the horizontal attention networks may be defined as follows:

$\displaystyle h_{u}^{L}=\sum_{l=1}^{L}\beta_{l}^{u}e_{v^{l}},$ (6) $\displaystyle\beta_{l}^{u}=\frac{\exp(e_{u}^{T}e_{v^{l}})}{\sum_{i=1}^{L}\exp(% e_{u}^{T}e_{v^{i}})},$ (7)

where $\beta_{l}^{u}$ is the attention weight for merging different values of $e_{v^{l}}$ .

4.3 Short-term interest modeling with interest evolution

A user’s short-term interest is determined by the items they have recently interacted with. Previous studies [9, 41] indicate that users tend to engage with items that are closely associated with the content they have accessed recently. Therefore, in the sequential recommendation, the users’ short-term interest must be simulated effectively, as reflected by recently visited items.

Figure 3.

shows the structures of GRU and AUGRU. Here, $i_{t}$ represents the input at time step t, $h_{t-1}$ is the hidden state at the previous time step, and $h_{t}$ is the hidden state passed to the next time step. In GRU, $i_{t}$ represents the embedding of the t-th item in the user’s short-term interaction sequences, while in AUGRU, $i_{t}$ represents the hidden state of the GRU at time t in the interest extraction layer. $\mathbf{r_{t}}$ and $\mathbf{u_{t}}$ are the reset gate and update gate of GRU, respectively. However, for AUGRU, the original update gate $\mathbf{u_{t}}$ is replaced by an attention-based update gate $\mathbf{u^{\prime}_{t}}$ , while the update rules for other gate units remain unchanged.

A) Interest Extractor Layer.

To extract the user’s short-term interest and global dependencies, we represent the user’s k most recent interactions by using the embedding sequence $E_{S}={\{[e_{t},e_{t+1},\ldots,e_{t+k}]|1<t+k<l\}}$ , where t is the most recent timestamp. Then, we adopt a GRU model with ES as input to perform this task. The framework of GRU is illustrated in Fig. 3, in which the GRU is a variant of Long Short-Term Memory (LSTM) and has a simpler gate structure. The GRU can be described as:

$\displaystyle\mathbf{r}_{t}=\sigma(W_{rt}\mathbf{i}_{t}+W_{rh}\mathbf{h}_{t-1}% +\mathbf{b}_{r}),$ (8) $\displaystyle\mathbf{u}_{t}=\sigma(W_{ut}\mathbf{i}_{t}+W_{uh}\mathbf{h}_{t-1}% +\mathbf{b}_{u}),$ (9) $\displaystyle\tilde{\mathbf{h}}_{t}=\tanh(W_{ht}\mathbf{i}_{t}+W_{uh}(\mathbf{% r}_{t}\odot\mathbf{h}_{t-1})+\mathbf{b}_{h}),$ (10) $\displaystyle\mathbf{h}_{t}=(\mathbf{1}-\mathbf{u}_{t})\odot\mathbf{h}_{t-1}+% \mathbf{u}_{t}\odot\tilde{\mathbf{h}}_{t},$ (11)

where $\mathbf{r}_{t}$ and $\mathbf{u}_{t}$ are reset gate and update gate, respectively; $\tilde{\mathbf{h}}_{t}$ is the memory ability of the GRU unit and $h_{t}$ is the t-th hidden states. $\sigma$ is the sigmoid activation function, and $\odot$ is the element-wise product. For $W_{rt},W_{ut},W_{ht}\in R^{n_{H}\times n_{I}}$ , $W_{rh},W_{uh},W_{uh}\in R^{n_{H}\times n_{H}},n_{H}$ is the hidden size, and $n_{I}$ is the input size. $i_{t}$ is the input of GRU. $i_{t}=e_{t+k}$ represents the t-th behavior that the user taken in the most recent interaction. ${h_{u}^{S}}=[h_{1},h_{2},\ldots,h_{T}]$ is the user’s short-term multi-interest sequence composed of all T hidden states.

However, ${h_{u}^{S}}$ only captures dependencies between behavior. User preferences are not only influenced by their recent behavior but also by long-term behavior.

B) Interest Guidance Layer.

Given the diversity of interests, users’ interests may drift. The influence of interest drifting on user behavior may manifest as users becoming interested in certain electronic products in a specific period. At the same time, users may need clothes in another period. Therefore, our model introduces an interest guidance attention mechanism to enhance users’ short-term interest coding. The function of the attention mechanism is described as follows:

$\displaystyle\gamma_{t}=\frac{\exp(h_{t}Wh_{u}^{L})}{\sum_{j=1}^{T}\exp(h_{j}% Wh_{u}^{L})},$ (12)

where $h_{u}^{L}$ is the long-term preference of u, $W\in R^{n_{H}\times n_{A}}$ , $n_{H}$ is the dimension of the hidden state, and $n_{A}$ is the embedded dimension of long-term user interest, and $h_{t}\in{h_{u}^{S}}^{\prime}$ is the t-th hidden states at the Interest Extractor Layer.

C) Interest Evolution Layer.

In earlier study [37], users’ final preferences were formed by concatenating their long-term and short-term interests, which limited the performance of the model. In the interest evolution layer, we use a GRU with attention mechanism (AUGRU) to model the evolution of user interest. As AUGRU adaptively fuses users’ short-term and long-term interests, the interference of interest drift can be effectively avoided. The neural framework of AUGRU is shown in Fig. 3. The calculation formula of AUGRU is similar to that of GRU in the interest extraction layer:

$\displaystyle\mathbf{u}_{t}^{\prime}=\gamma_{t}\times\mathbf{u}_{t},$ (13) $\displaystyle\mathbf{h}_{t}^{\prime}=(\mathbf{1}-\mathbf{u}_{t}^{\prime})\odot% \mathbf{h}_{t-1}^{\prime}+u_{t}^{\prime}\odot\tilde{\mathbf{h}}_{t}^{\prime},$ (14)

where $u_{t}$ is the original update gate of GRU and $\mathbf{u}_{t}^{\prime}$ is the attentional update gate of AUGRU. The input to AUGRU is denoted by $i_{t}^{\prime}=h_{t}$ , where $h_{t}$ is the interest state corresponding to the interest extraction layer. The final hidden state is denoted by $h_{T}^{\prime}$ , i.e., user final interest state $F_{u}$ .

4.4 Prediction and model optimization

4.4.1 Prediction

The basic idea of the FLSIE model is to acquire the long-term interest $h_{u}^{S}$ and short-term interest $h_{u}^{L}$ of user u on his or her historical sequences $L^{u}$ , and then generate the final interest $F_{u}$ through the interest evolution layer. Therefore, we employ the dot product of the user’s final interest state $F_{u}$ and the item embedding $e_{v}$ as the prediction of the user’s normalized rating score for item v as follows:

$\displaystyle\widehat{y}_{u,v}=F_{u}e_{v}^{T},$ (15)

where $\widehat{y}_{u,v}$ refers to the ranking score of candidate item i in the item sets. Therefore, all prediction scores of the candidate items can be calculated according to Eq. (15). Finally, a set of top-N candidate items with the highest ranking scores is recommended.

[h] : Learning Algorithm of FLSIE $S^{u}_{t}$ : short-term behavior sequences, $L^{u}_{t-1}$ : long-term behavior sequence, $\eta$ : learning rate, $\lambda$ : L2-normalization coefficient, $d_{f}$ : attribute’s latent dimensionalityoptimal model parameters $\theta$ Randomly initialize all parameters $\theta$

convergence shufe the set of observations (u, $L^{u}_{t-1}$ , $S^{u}_{t}$ ) for all users

observation (u, $L^{u}_{t-1}$ , $S^{u}_{t}$ ) compute $h^{L}_{u}$ according to Eqs (1)–(7)

compute $F_{u}$ according to Eqs (8)–(14)

compute loss L according to Eq. (16)

update model parameters $\theta$ according to L and optimizer

return $\theta$

4.4.2 Model optimization

Our model optimization aims to provide a ranked list of items based on the short- and long-term behavior sequences. Therefore, we prioritize the relative ranking of predicted ratings instead of their absolute values. During the training process, let $v_{l}^{+}\in L^{u}$ and $v_{s}^{+}\in S^{u}$ be the positive sample set $V^{+}$ . The unobserved item $v_{l}^{-}\notin L^{u}$ and $v_{s}^{-}\notin S^{u}$ are uniformly sampled to form a negative sample set $V^{-}$ . Our expectation is that the predicted rating for item $v\in V^{+}$ would be greater than that of an item $v^{-}\in V^{-}$ , i.e., $\widehat{y}_{u,v^{+}}>\widehat{y}_{u,v^{-}}$ . Therefore, a pair-wise loss function is introduced to adhere to the principle of Bayesian Personalized Ranking (BPR) [42] as follows:

$\displaystyle L=\sum_{v_{l}^{+}=2}^{m}\sum_{v_{l}^{-}\neq L_{u}}-\ln\sigma(% \widehat{y}_{u,v_{l}^{+}}-\widehat{y}_{u,v_{l}^{-}})+\sum_{v_{s}^{+}=2}^{m}% \sum_{v_{s}^{-}\neq L_{s}}-\ln\sigma(\widehat{y}_{u,v_{s}^{+}}-\widehat{y}_{u,% v_{s}^{-}})+\lambda\rVert\Theta\rVert^{2},$ (16)

where $\Theta$ represents all the learnable parameters, $\sigma(\cdot)$ is the sigmoid function, and $\lambda\rVert\Theta\rVert^{2}$ is the L2 regularization to prevent overfitting. The Adam algorithm [43] is used for optimization. Further details about the learning algorithm are presented in Algorithm 4.4.1.

5. Experiments

5.1 Experimental setting

In this section, we provide a detailed description of the experimental setup, including the datasets used, evaluation metrics, baseline models, and implementation details.

5.1.1 Datasets

We evaluate the performance of our proposed FLSIE model in terms of its ability to recommend the next item on three real-world datasets, namely, Beauty, Yelp, and Gowalla. The details are introduced as follows:

•
Amazon [44] is globally recognized as one of the largest internet retailers and has an extensive collection of behavioral data. We chose the Beauty category dataset because of its reputation for high sparsity and variability. The item attributes of this dataset consist of fine-grained categories and brands, all of which are discrete and classified.
•
Gowalla [37] is a location-based social networking site that collects user check-in times and point-of-interest information. We analyze the item attributes generated within the most recent seven-month period.
•
Yelp is a widely recognized dataset of business recommendations. Following the method of previous work [44], only interaction records after January 1, 2019, can be used in our experiment. Both businesses and users contain continuous and discrete features. Therefore, the categories of position information and businesses are regarded as attributes.

In evaluating the model’s performance, the leave-one-out strategy [45] is employed following the pre-processing steps from prior works [32, 37]. The presence of a review or rating in these datasets indicates implicit feedback, demonstrating user interaction with an item. Timestamps are utilized to establish the order of actions in the sequence. Furthermore, for the above mentioned dataset, all items with less than 20 user interactions are removed. Then, we consider all user interactions in the recent period as a session to represent the user’s short-term behavioral sequences. The rest of the interactions represents the user’s long-term behavioral sequences. Here, users with less than 3 item sessions or less than 3 sessions should be filtered out. Therefore, for the historical sequence $S_{u}=\{i_{1},i_{2},i_{3},\ldots,i_{|S|}\}$ for each user, we select the most recent action $i_{|S|}$ as the test set, the second most recent action $i_{|S-1|}$ as the validation set, and all remaining actions as the training set. The statistics of all datasets after processing are shown in Table 2.

Table 2
Statistics of three datasets

Dataset User Item Train session Test session Avg. session

Amazon 8,024 17,289 30,526 11,542 4.28

Yelp 17,421 40,068 98,423 17,255 3.63

Gowalla 5,522 12,193 15,225 5,635 9.7

5.1.2 Evaluation metrics

Dataset	User	Item	Train session	Test session	Avg. session
Amazon	8,024	17,289	30,526	11,542	4.28
Yelp	17,421	40,068	98,423	17,255	3.63
Gowalla	5,522	12,193	15,225	5,635	9.7

The recommendation model generate a ranked list of items for each user. To evaluate the baseline model and our model, we utilize two commonly used evaluation metrics, namely, Hit Ratio at rank K (HR@K) and Normalized Discounted Cumulative Gain at rank K (NDCG@K) [4, 21], with K chosen from {10, 20}.

•
HR@K. This metric represents the proportion that the ground truth is among the top N items of the ranked list. It can be defined as follows:

$\displaystyle\mathrm{HR}@\mathrm{K}=\frac{1}{|\mathcal{U}|}\sum_{u\in\mathcal{% U}}I(R_{u,g_{u}}\leqslant K).$ (17)
•
NDCG@K. This metric is a position-aware metric that distributes greater weight at higher positions and is related to the ranking position of real items. It not only ensures the accuracy of the recommendation but also considers the exact position of the target item in the list. DNCG@K can be defined as follows:

$\displaystyle\mathrm{NDCG}@\mathrm{K}=\frac{1}{|\mathcal{U}|}\sum_{u\in% \mathcal{U}}\frac{I(R_{u,g_{u}}\leqslant K)}{\log_{2}(R_{u,g_{u}}+1)},$ (18)

where $g_{u}$ is the interaction term of user u in original dataset, and $R_{u,g_{u}}$ denotes the rank of the ground truth item. $I(\cdot)$ is an indicator function and $I(\cdot)=1$ if $R_{u,g_{u}}\leqslant K$ is true, otherwise, $I(\cdot)=0$ .

5.1.3 Baseline methods

To evaluate the proposed model, we compare FLSIE with eight sequential recommendation methods. These methods include traditional factorization-based methods, deep-learning based methods, and long- and short-term sequential recommendation methods.

•
BPR [42]. This method optimizes the MF model with Bayesian Personalized Ranking, directly calculates the commodity ranking, and then completes the recommendation according to the commodity ranking.
•
FPMC [5]. This method integrates matrix factorization and Markov chains to capture the user’s long-term preferences and dependencies on the interaction items in the sequence. Personalized and dependent information is then fused in a linear fashion for the next basket recommendation.
•
GRU4Rec [7]. A classic approach of session-based recommendation, this method utilizes recurrent neural networks to capture sequential patterns.
•
NARM [28]. This method is a hybrid encoder to combine the attention mechanism into the hidden state of the gated loop unit and expresses the user’s short-term interest by concatenating his or her behavioral characteristics and intent characteristics.
•
STAMP [29]. This short-term attention priority model captures the general interest from a sequence of interactions and takes the embedding of the last interactive item as the current interest.
•
SASRec [11]. This method utilizes a self-attention mechanism to combine with location embedding to capture user preferences.
•
ADAR [35]. This model uses attention mechanisms and matrix factorization to learn the embedding of item attributes and then accomplish much more personalized recommendations.
•
SHAN [37]. The first layer of this method captures the long-term preferences of the user, whereas the next layer integrates the long-term and short-term interests to represent the user’s final hybrid representation.

5.1.4 Implementation details

To achieve a fair comparison, we implement all baseline models and our proposed model using the recommendation framework RecBole [46] while ensuring that identical settings are used for all models. We utilize the Adam optimizer [43] to optimize all models while implementing early stopping when no improvement is observed on the validation dataset after 20 epochs. The batch size is 256, while the learning rate and feature embedding size are varied within the ranges of [1e-1, 1e-2, 1e-3, 1e-4] and [32, 64, 128, 256], respectively. The L2 regularization factor $\lambda$ is adjusted from [1e-4, 1e-3, 1e-2, 1e-1, 0]. Moreover, to account for the individual dataset sparsity levels, we individually tailor the dropout rates and set them at [0.2, 0.5, 0.2]. Regarding the other parameters, we perform a grid search to find the best configuration.

5.2 Performance comparison with baselines

Table 3
Comparative performance of all models on three datasets. The bold and underlined values in each line represent the optimal and second-best results, respectively (the larger the value, the better the performance). The last column shows the percentage increase in performance in contrast to the best baseline

Dataset	Metrics	BPR	FPMC	ADAR	GRU4Rec	NARM	STAMP	SHAN	SASRec	FLSIE	Improv.
Amazon	HR@10	0.0512	0.0857	0.1253	0.1587	0.1783	0.1492	0.1723	0.1857	0.2201	18.53%
	HR@20	0.0834	0.1242	0.1527	0.2143	0.2438	0.1957	0.2378	0.2514	0.3107	23.59%
	NDCG@10	0.0334	0.0421	0.0627	0.0813	0.1011	0.0721	0.0957	0.1123	0.1365	21.55%
	NDCG@20	0.0489	0.0784	0.0989	0.1156	0.1227	0.1054	0.1113	0.1367	0.1584	15.87%
Gowalla	HR@10	0.0071	0.0921	0.1041	0.1443	0.1657	0.1583	0.1725	0.1715	0.1817	5.33%
	HR@20	0.0144	0.1125	0.1375	0.1854	0.2046	0.2013	0.1987	0.2102	0.2251	7.09%
	NDCG@10	0.0031	0.0068	0.0175	0.0621	0.0872	0.0735	0.0813	0.0912	0.0982	7.68%
	NDCG@20	0.0046	0.0143	0.023	0.0891	0.1053	0.0902	0.1078	0.1127	0.1228	8.96%
Yelp	HR@10	0.0121	0.0285	0.0395	0.0489	0.0531	0.0612	0.0859	0.0753	0.0934	8.73%
	HR@20	0.0315	0.0421	0.0654	0.0712	0.0854	0.1089	0.1142	0.0912	0.1285	12.52%
	NDCG@10	0.0049	0.0057	0.0124	0.0213	0.0295	0.0492	0.0583	0.0312	0.0647	10.98%
	NDCG@20	0.0091	0.0198	0.0286	0.0425	0.0374	0.0541	0.0792	0.0453	0.0863	8.97%

In this section, we compare the recommended performance of the proposed model FLSIE with the abovementioned baselines on three real public datasets, as shown in Table 3. The observation results are as follows:

•

BPR, FPMC, and ADAR all adopt matrix factorization methods. However, their results are lower than those of other baseline methods because BPR, FPMC, and ADAR do not effectively use sequence information. Nevertheless, FPMC performs better than BPR because the former models the user’s recent interactions with a first-order Markov chain. ADAR considers the attributes of items, so its results are higher than those of the other two baselines. This finding indicates that attribute information can enhance the representation of items.

•

GRU4Rec and NARM perform better than traditional baselines, suggesting that RNN-based models can effectively handle sequence information within sessions. NARM considers the user’s current intention and outperforms GRU4Rec on all three datasets. NARM achieves excellent results, only slightly lower than the strongest baseline.

•

STAMP and SASRec both perform well. On the Amazon and Gowalla datasets, SASRec becomes the strongest baseline, indicating that self-attention mechanisms have relatively good predictive power. STAMP outperforms GRU4Rec on the Gowalla and Yelp datasets because it reduces the impact of interest drift by treating the last click as the current interest.

•

SHAN outperforms others on the sparse Yelp dataset by distinguishing between the long-term and short-term interests of users. Additionally, the number of long-term items exceeds that of short-term items, hence the difficulty for SASRec with a self-attention mechanism to assign appropriate weights to vital items within the short-term sequence.

Finally, our proposed model outperforms all baselines on the Amazon, Gowalla, and Yelp datasets, achieving optimal performance. Specifically, on the Amazon dataset, FLSIE gains 21.06% HR and 18.71% NDCG improvements on average compared with the strongest baseline SASRec. On the Yelp dataset, our model achieves more than 10% improvement in the HR and more than 9% improvement in the NDCG. Furthermore, the model has attained some improvement on the Gowalla dataset. FLSIE first leverages attribute-enhanced item representation and can learn user interests from item attributes in a fine-grained manner. The model incorporates attention mechanisms and RNNs to model the long- and short-term interests of users. In addition, our model employs habit-guiding mechanisms to reduce interest drift and deliver more personalized recommendations based on long- and short-term interest evolution.

5.3 Ablation experiments

Our model consists of several components: an explicit feature embedding module, long-term and short-term interest modeling modules, a habit-guided attention module, and a long- and short-term interest evolution module. To investigate the effectiveness of each component, we conduct an ablation study by removing some components to obtain different variant models. FLSIE-Att is the model in which the item attributes are removed, FLSIE-S is the model with only the user’s long-term interest is modeled, FLSIE-L is the model that only considers the user’s recent preferences, FLSIE-Hb is the model in which the habit-guided attention module is removed, and FLSIE-Agg is the model that directly concatenates the long-term and short-term interest models without considering the adaptive fusion between them. We conduct an ablation study by comparing our model to these five variants.

Figure 4.

Performance comparison of FLSIE and its variants on three different datasets. The horizontal axis represents the various models, while the primary and secondary ordinates show the models’ scores on HR@20 and NDCG@20 evaluation metrics, respectively. The bar chart presents the HR@20 metric, while the line chart depicts the NDCG@20 metric.

Figure 4 shows the ablation results. Our proposed full model (i.e., FLSIE) consistently outperforms all of its variants. We have observed the following: First, FLSIE utilizes explicit feature embedding to learn user interests at a much finer granularity from attributes. Given the different sparsity levels of the datasets, FLSIE-Att performs better on the dense Amazon dataset than on the sparse Gowalla and Yelp datasets, effectively alleviating the cold-start problem. Second, on the Amazon dataset, FLSIE-S performs better than FLSIE-L. By contrast, on the Gowalla and Yelp datasets, FLSIE-L performs better than FLSIE-S, indicating that differentiating between users’ long-term interest and short-term interest can affect their next actions. Meanwhile, on datasets in which FLSIE-S performs well, FLSIE-Hb has a better performance improvement, suggesting that the habit-guiding mechanism can reduce user interest drift. Finally, the results of the FLSIE-Agg model are slightly lower than those of the full model. Thus, by modeling the user’s long-term and short-term interest evolution, we can adaptively integrate the two interests to provide more personalized recommendations to users.

5.4 Influence of hyper parameters

We study the impact of hyper parameters on model performance, including the size of attribute embedding dimension $d_{f}$ and the maximum length of short-term and long-term sequences.

Influence of an Attribute’s Latent Dimensionality $d_{f}$ .

We conduct comparative experiments with five different sizes of attribute-embedding dimensions $d_{f}\in\{16,32,64,128,256\}$ while keeping other optimal parameters constant. The HR@20 results on the Amazon, Gowalla and Yelp datasets are shown in Fig. 5. Then following observations can be drawn:

First, a much small $d_{f}$ cannot fully represent the latent features of attributes. A Large $d_{f}$ can contain more information, but overfitting may occur when $d_{f}$ exceeds 128, leading to a decrease in model performance. However, FLSIE can exhibit the best performance in most latent dimensions. On the dense Amazon dataset, a large $d_{f}$ does not always lead to better model performance. However, each model requires large latent dimensions for sparse Gowalla and Yelp datasets to achieve optimal results. Second, as the embedding size $d_{f}$ increases, the performance of the model continues to improve. When selecting an appropriate $d_{f}$ , the model achieves optimal performance. In the experiment, we set the attribute-embedding size $d_{f}$ to 128 to capture more features and obtain much higher accuracy.

Figure 5.

Influence of the latent dimensionality $d_{f}$ of attributes on the ranking performance (NDCG@20).

Figure 6.

Performance of FLSIE with varying s and l values at NDCG@20 on the Amazon and Yelp datasets.

Impact of Short- and Long-Term Max Sequence Lengths.

We investigate the impact of the short-term and long-term maximum sequence lengths (represented by parameters s and l, respectively) on our model. To validate our findings, we conduct experiments with varying sequence lengths on different datasets and evaluate the model’s effectiveness by using the NDCG@20 metric. As shown in Fig. 6, increasing the sequence length within a certain range can improve the model’s performance. However, setting s and l to very extremely values may result in losing the interest information of users, whereas setting them to large values may cause sparsity issues and increase the computational costs. Two main reasons contribute to this phenomenon. First, as the sequence length increases, much more historical interaction sequences are captured and modeled. Second, overly long sequences may contain too much redundant information and introduce noise, which can limit the model’s performance. Our results indicate that it is advantageous to consider a user’s previous and recent interactions when modeling his/her ultimate interest. To optimize training efficiency, we set s $=$ 50 and l $=$ 200 for the Amazon dataset. For the Gowalla dataset, we set s $=$ 300 and l $=$ 400 for the best performance. Considering the similarity of the results of the Yelp dataset and the Gowalla dataset, we set the s $=$ 200 and l $=$ 400.

6. Conclusion and future work

In this study, we propose a novel sequential recommendation framework called the Feature-aware Long-Short Interest Evolution Network (FLSIE). FLSIE aims to capture users’ preferences for item features and effectively model the evolution of their long-term and short-term interests. Specifically, the model employs explicit feature embeddings to represent item attribute information and utilizes a two-dimensional (2D) attention mechanism to differentiate the importance of individual features within specific items and their relevance in the interaction sequences. Furthermore, to address the issue of interest drift, we introduce a long-term interest guidance mechanism to enhance the representation of short-term interests. Additionally, we adopt AUGRU to dynamically model the evolution of users’ short-term interests. The experimental results demonstrate that the FLSIE model outperforms other baseline methods in terms of HR and NDCG evaluation metrics on three public datasets.

However, we also recognize that the FLSIE model has some limitations, including its relatively high computational complexity and the treatment of various user-item interactions (e.g., click, bookmark, add to the shopping cart, and purchase) as equal. In reality, different behavior types contribute differently to users’ interests, which deserves further consideration and optimization. In the future, to further enhance the model’s performance, we will plan to employ GNN to study the temporal evolution relationship between different user-item interaction types. By modeling the dynamic relationships among various interaction types, we aim to more accurately capture changes in user interests and improve the accuracy of the recommendation model.

Footnotes

Acknowledgments

This work was supported by the Chunhui Plan Cooperative Project of Ministry of Education (Grant No: Z2011018), the Sichuan Science and Technology Program (Grant No: 2023YFQ0044) and the Postgraduate Course Teaching Team Project of Xihua University (Grant No: YTD202303).

References

Liu

Yuan

Yang

and Zhang

, CAEN: A Hierarchically Attentive Evolution Network for Item-Attribute-Change-Aware Recommendation in the Growing E-commerce Environment, in: Proceedings of the 16th ACM Conference on Recommender Systems, 2022, pp. 278–287.

Zhang

Liu

and Xie

, Neural News Recommendation with Long- and Short-term User Representations, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 336–345.

Ren

Liang

Wang

and de Rijke

, Social collaborative viewpoint regression with explainable recommendations, in: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, 2017, pp. 485–494.

Liao

Zhang

Nie

and Chua

T.-S.

, Neural collaborative filtering, in: Proceedings of the 26th International Conference on World Wide Web, 2017, pp. 173–182.

Rendle

Freudenthaler

and Schmidt-Thieme

, Factorizing personalized markov chains for next-basket recommendation, in: Proceedings of the 19th International Conference on World Wide Web, 2010, pp. 811–820.

and McAuley

, Fusing similarity models with markov chains for sparse sequential recommendation, in: 2016 IEEE 16th International Conference on Data Mining (ICDM), IEEE, 2016, pp. 191–200.

Hidasi

Karatzoglou

Baltrunas

and Tikk

, Session-based Recommendations with Recurrent Neural Networks, Computer ence, 2015.

Yuan

Wang

Liu

and Li

, Attention-based context-aware sequential recommendation model, Information Sciences 510 (2020), 122–134.

Tang

and Wang

, Personalized top-n sequential recommendation via convolutional sequence embedding, in: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, 2018, pp. 565–573.

10.

Zhou

Mou

Fan

Bian

Zhou

Zhu

and Gai

, Deep interest evolution network for click-through rate prediction, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 5941–5948.

11.

Kang

W.-C.

and McAuley

, Self-attentive sequential recommendation, in: 2018 IEEE International Conference on Data Mining (ICDM), IEEE, 2018, pp. 197–206.

12.

Sun

Liu

Pei

Lin

and Jiang

, BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 1441–1450.

13.

Lian

Mahmoody

Liu

and Xie

, Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation, in: IJCAI, 2019, pp. 4213–4219.

14.

Wang

Zhang

Liu

and Ma

, Make it a chorus: knowledge-and time-aware item modeling for sequential recommendation, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 109–118.

15.

Qin

Ren

Fang

Zhang

and Yu

, Sequential recommendation with dual side neighbor-based collaborative relation modeling, in: Proceedings of the 13th International Conference on Web Search and Data Mining, 2020, pp. 465–473.

16.

Bai

Nie

J.-Y.

Zhao

W.X.

Zhu

and Wen

J.-R.

, An attribute-aware neural attentive model for next basket recommendation, in: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018, pp. 1201–1204.

17.

Jin

Sun

Lin

Yang

and Ng

, SDM: Sequential deep matching model for online large-scale recommender system, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 2635–2643.

18.

Yang

and Zhang

, MRIF: Multi-resolution Interest Fusion for Recommendation, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 1765–1768.

19.

Tian

Chang

Niu

Song

and Li

, When Multi-Level Meets Multi-Interest: A Multi-Grained Neural Model for Sequential Recommendation, in: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 1632–1641.

20.

Xiong

Merity

and Socher

, Dynamic memory networks for visual and textual question answering, in: International Conference on Machine Learning, PMLR, 2016, pp. 2397–2406.

21.

Kang

W.-C.

and McAuley

, Translation-based recommendation, in: Proceedings of the Eleventh ACM Conference on Recommender Systems, 2017, pp. 161–169.

22.

Fang

Wang

and McAuley

, Vista: A visually, socially, and temporally-aware model for artistic recommendation, in: Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 309–316.

23.

Quadrana

Karatzoglou

Hidasi

and Cremonesi

, Personalizing session-based recommendations with hierarchical recurrent neural networks, in: Proceedings of the Eleventh ACM Conference on Recommender Systems, 2017, pp. 130–137.

24.

Kang

and Liu

, Hierarchical gating networks for sequential recommendation, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 825–833.

25.

Tang

Zhu

Wang

Xie

and Tan

, Session-based recommendation with graph neural networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019, pp. 346–353.

26.

Vaswani

Shazeer

Parmar

Uszkoreit

Jones

Gomez

A.N.

Kaiser

Ł.

and Polosukhin

, Attention is all you need, Advances in Neural Information Processing Systems 30 (2017).

27.

Wang

and McAuley

, Time interval aware self-attention for sequential recommendation, in: Proceedings of the 13th International Conference on Web Search and Data Mining, 2020, pp. 322–330.

28.

Ren

Chen

Ren

Lian

and Ma

, Neural attentive session-based recommendation, in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 1419–1428.

29.

Liu

Zeng

Mokhosi

and Zhang

, STAMP: short-term attention/memory priority model for session-based recommendation, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 1831–1839.

30.

Lin

Gao

Zheng

Chang

Niu

Song

Jin

and Li

, Dual-interest Factorization-heads Attention for Sequential Recommendation, 2023, 917–927.

31.

Zhang

Zhao

Liu

Sheng

V.S.

Wang

Liu

Zhou

et al., Feature-level Deeper Self-Attention Network for Sequential Recommendation, in: IJCAI, 2019, pp. 4320–4326.

32.

Yuan

Duan

Tong

Shi

and Zhang

, ICAI-SR: Item categorical attribute integrated sequential recommendation, in: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 1687–1691.

33.

Liu

Cai

Dong

Zhu

and Shang

, Noninvasive self-attention for side information fusion in sequential recommendation, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 4249–4256.

34.

Xie

Zhou

and Kim

, Decoupled side information fusion for sequential recommendation, in: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 1611–1621.

35.

Sun

Zhang

Wang

Yin

and Zhang

, Attribute-aware deep attentive recommendation, The Journal of Supercomputing, 2021, 5510–5527.

36.

Zhu

Liu

and Wu

, Disentangled self-attentive neural networks for click-through rate prediction, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 3553–3557.

37.

Ying

Zhuang

Zhang

Liu

Xie

Xiong

and Wu

, Sequential recommender system based on hierarchical attention network, in: IJCAI International Joint Conference on Artificial Intelligence, 2018.

38.

Zhang

Sun

Liu

and Coates

, Memory augmented graph neural networks for sequential recommendation, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 5045–5052.

39.

Fang

Zhang

Shu

and Guo

, Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations, ACM Transactions on Information Systems (TOIS) 39(1) (2020), 1–42.

40.

Wang

Zhang

Chen

Liu

and Ma

, Toward dynamic user intention: Temporal evolutionary effects of item relations in sequential recommendation, ACM Transactions on Information Systems (TOIS) 39(2) (2020), 1–33.

41.

Hidasi

and Karatzoglou

, Recurrent neural networks with top-k gains for session-based recommendations, in: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018, pp. 843–852.

42.

Rendle

Freudenthaler

Gantner

and Schmidt-Thieme

, BPR: Bayesian personalized ranking from implicit feedback, in: UAI 2009, Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, June 18–21, 2009, 2009.

43.

Kingma

D.P.

and Ba

, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014.

44.

Zhou

Wang

Zhao

W.X.

Zhu

Wang

Zhang

Wang

and Wen

J.-R.

, S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization, in: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 1893–1902.

45.

Chen

Zhang

Nie

Liu

and Chua

T.-S.

, Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention, in: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, 2017, pp. 335–344.

46.

Zhao

W.X.

Hou

Lin

Chen

Pan

Wang

Tian

et al., Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4653–4664.

A feature-aware long-short interest evolution network for sequential recommendation

Abstract

Keywords

1. Introduction

2.1 Sequential recommendation

2.2 Attention mechanism for sequential recommendation

2.3 Modeling on item attributes and evolution

3. Problem statement

Table 1 List of key notations

4.1 Input embedding with side information

4.1.1 Explicit feature embedding

Vertical Attention.

Horizontal Attention.

A) Interest Extractor Layer.

B) Interest Guidance Layer.

C) Interest Evolution Layer.

4.4.1 Prediction

5.1 Experimental setting

5.1.1 Datasets

5.2 Performance comparison with baselines

Influence of an Attribute’s Latent Dimensionality d f .

Impact of Short- and Long-Term Max Sequence Lengths.

Footnotes

Acknowledgments

References

Table 1
List of key notations

Influence of an Attribute’s Latent Dimensionality $d_{f}$ .