Keywords

1 Introduction

With a dramatic increase in the Web information environment, information overload has received wide social attention. In most of all online services, recommender modules have become essential components to help users relieve the pressure on information overload and obtaining interesting information from huge amounts of data. Recommender systems can be also used to increase personal comfort by suppling items or products orderly with gauge of historical activities or synergetic relationships of items or users. For example, recommender systems could automatically list items of interest, or give a suggestion of new discounts relevant to each other. Many recommender systems are working on the assumption that user identification, item label and past activities are available with the temporal variation. It is more importance to consider a framework with complex scenarios of information scarcity. The core and the most crucial link of recommender systems is to search a way of computing relevance degree of items, users, or user-items with or without temporal information. Temporal information is important as it could supply an evidence of when users take items into account. There is a fact that to compare with the items considered long ago, recent considered items are more likely to be taken. The temporal information with user’s activity are recorded as sessions. A session is a group of interaction occurred in a given time frame. The time frame can be an hour, a day, several days, several weeks, even months or years.

Traditional methods of personalized recommendations usually provide functions by building up a user profile, such as user-user similarity based on past activities computed by collaborative filtering approaches [1] or latent factor vectors built by matrix factorization [2, 3]. There exists a problem of cold-start for recommendation approaches that need user past activities, and it is not rare that new accounts, unlogged users, or users without tracking information use recommender systems. Conventional recommender algorithms [4] have been applied to address the challenging problem of recommendations on session-based recommenders. RNNs (Recurrent Neural Networks) have been majority approaches with promising results in the research of session-based recommender systems [5,6,7,8]. There are some limitations in these aforementioned approaches, such as estimation of user representations without adequate user behavior in a session, complex transition among the distant items. To overcome the above limitations, SR-GNNs (Session-based Recommendation with Graph Neural Networks) [9] have been introduced to generate representations for graphs, which can be constructed directly from historical session sequences. Based on the session graph, it is easy to capture complex item transitions and make item embedding vectors with high quality. However, there is shortcoming that the user behavior in past session is non accessible, which might miss valuable information for the present session, in SR-GNNs.

In this paper, we provide a Hierarchical GNNs model for personalized recommendations, in which there are two layers of GNNs constructed to address the problems of both session-based recommenders and session-aware recommenders. The first layer of GNNs provides user identifiers and propagate information, which might give an improvement of recommendation accuracy, and the second layer of GNNs generates the recommendations coupled with the first layer. Extensive experiments conducted on two industry datasets demonstrate the effectiveness of our proposed approach over the state-of-arts.

2 Related Work

In this section, we review some related works on session-based recommender systems, including RNNs-based methods and GNNs-based methods.

Recurrent Neural Models.

RNNs are often applied for capturing the feature of sequential data [10], which include image and video captioning, time series prediction and much more. Long Short-Term Memory (LSTM) [11] networks and Gated Recurrent Units (GRUs) [12] are two types of RNNs with powerful performance on various abovementioned tasks. In the work of [5], RNNs are first applied to session-based recommendations, which provide recommendations for new sessions by learning a one-hot representation from the clicked item-IDs in the current session. After that, several valuable works are proposed based on RNNs methods, such as proper data augmentation techniques [6], the fusion of sequential patterns and co-occurrence signals [7], and attention mechanism [8].

Graph Neural Networks.

Neural networks have been exploited for capturing representation for graph-structured data, such as social network, knowledge bases and etc. In work of [13, 14], GNNs are proposed to operate on directed graphs by using a form of RNNs. GNNs have been applied for various tasks, such as script even prediction [15], situation recognition [16], and image classification [17]. Recently, SR-GNNs [9] have been employed efficiently for session-based recommendations. Different with [9], both session-based recommenders and session-aware recommenders are considered by applying a Hierarchical GNNs model.

3 The Proposed Method

In this section, we will provide the idea of our proposed Hierarchical GNN model, which consist of two layers of GNNs for personalized recommendation with user-context. Our model is based on SR-GNN model presented in [9], which aims to predict which item a user will click next based on current sequential session data. Different from SR-GNN, our work takes the evolution of the user interests over time into account, showing in Fig. 1. We describe our model by two parts, i.e. the architecture of hierarchical GNNs and learning process of supper parameters.

Fig. 1.
figure 1

Generic structure and workflow of our model.

Full size image

3.1 Architecture

As shown in Fig. 1, our HGNN model takes a user-level GNN (U-GNN) to model the user activity across sessions, and a session-level GNN (S-GNN) to model the global preference of common items from all sessions. Different from SR-GNN, at each time step, S-GNN and U-GNN make a joint decision to the result of computing recommendations. To SR-GNN, it is easier for our HGNN model to learn the user’s preference on selection of item sequences. In [9], authors claimed that sessions could be represented directly by nodes involved in a session, then they focused on developing a method to learn long-term preference and current interests on items from sessions. In their situation, there doesn’t exist the information of user activities across sessions. On the contrary, we have taken this factor into account, as there are a lot of information that user’s based activities could reveal. For example, it is not hard to associate animations for toddlers with children, rather than a video of hot-pot making. However, the baby A may prefer Spider-Man than Superman, just contrary to the baby B. Therefore, if there are videos of Spider-Man and Superman simultaneously, the recommendation ranking of videos should be different for two babies.

In session-based recommendation, \( V = \left\{ {v_{1} ,v_{2} , \ldots ,v_{m} } \right\} \) stands for the set consisting of all unique items involved in all the sessions. The clicked sequence \( S_{u} \) is ordered by timestamps from a session of the user U. In order to predict the next click, top-K values will be the basis of the candidate selection for recommendation. Same to [9], we embed every item into a unified embedding space and the latent vector of each item is learned via GNNs, and based on node vectors, each session can be represented by two embedding vector, which are composed of node vectors used in U-GNNs and S-GNNs.

3.2 Learning

In our S-GNNs, we take the same learning method for item embedding, and it is similar to obtain update functions for U-GNNs. It is different to obtain long-term preference and current interests of the input session after feeding all session graphs into GNNs. As in the workflow of our model, U-GNNs and S-GNNs will generate two independent local or global embedding vectors. To compute the hybrid embedding vector by taking linear transformation over the concatenation of four embedding vectors, including local and global embedding vectors obtained from U-GNNs and S-GNNs.

From our framework, when user is unknown our model would deteriorate to SR-GNN model, as the anonymous user makes U-GNNs same to S-GNNs. And we will test this part for our model. We applied the Back-Propagation through Time (BPTT) algorithm to train our model.

4 Experiments

In this section we describe our experimental setup and provide an in-depth discussion of the achieved results.

4.1 Datasets

Same to [9], we also evaluate the proposed method on two public representative datasets, including Yoochoose [18] and Diginetica [19]. The Yoochoose and Diginetica datasets are obtained from the RecSys Challenge 2015 and CIKM Cup 2016 respectively. The first dataset consists of a six-month stream of user clicks on an e-commerce website, and for the second one only transactional data is used.

As shown in Table 1, the most recent training sequences of Yoochoose are employed in the ratio of 1/128. In order to compare fairly with other methods, we drop sessions and items, which cannot meet session length greater than 1 or item occurrences not less than 5. Then there is a data-partition policy for generating training and test sets from the input sequence, with which the sessions of subsequent days and months (i.e. 30 days) are divided as test sets for Yoochoose and Diginetica respectively. Furthermore, to evaluate the performance of our approach, we filter out all anonymous users on the Diginetica dataset.

Table 1. The datasets used in experiments
Full size table

4.2 Evaluation metrics and parameter setup

There are two types of metrics taken for measuring performance of compared methods, including P@20 and MRR@20. These metrics are designed for the recommendation setting, as is often to make multiple recommendations for each user. P@20 (Precision), which is a popular measure of predictive accuracy, stands for the proportion of correctly recommended items amongst the top-20 items. MRR@20 (Mean Reciprocal Rank), as the average of reciprocal ranks of correctly recommended items, takes the order of ranking into account. The number 20 means to set MRR to 0 once ranking exceeds 20. The larger the MRR value, the higher the ranking of correct recommendations in the top list.

To evaluate the performance of our proposed method, there are five representative baselines involved to comparisons, including POP, User-KNN, GRU4REC [5], NARM [20] and SR-GNN [9]. POP and S-POP recommend the top-N frequent items in the training set and in the current session respectively. GRU4REC employs RNNs to provide session-based recommendations by modeling the session sequences. NARM also applies RNNs to make recommendations, however it take attention mechanism into account for capturing user’s sequential behavior and main purpose. SR-GNN exploits GNNs to model session sequences seamless with the ability of computing complex transitions of items.

Following previous method [9], the dimensionality of latent vectors \( d = 100 \) is established for both datasets. We initialize all parameters with a Gaussian distribution with a mean of 0 and a standard deviation of 0.1. In order to optimize these parameters, a mini-batch Adam optimizer is applied with the initial learning rate of 0.001. And we set the L2 penalty and the batch size to 10−5 and 100 respectively.

4.3 Comparison with other methods

We evaluate the performance of our proposed approach by making comparisons with several other session-based recommendation methods, where the result in terms of P@20 and MRR@20 is listed in Table 2.

Table 2. Comparisons of various methods over three datasets
Full size table

From the result listed in Table 2, it is obvious that our proposed approach is outstanding from other five methods in terms of P@20 and MRR@20 over three datasets. In the architecture of our proposed model, we not only consider the impact of items of common sessions, but also take user’s preference as well. All these session sequences are integrated as graph-structured data seamless. The result also shows that neural network-based approaches outperform the traditional methods by adopting deep learning mechanisms. Furthermore, SR-GNN is so outstanding from GRU4REC and NARM because of capturing more complex and implicit connections between user clicks. To be more advanced, our proposed model takes consider of user session information as well as global session information.

5 Conclusions and Future Work

In this paper, we proposed a mode based on Hierarchical GNNs to address the problem of personalized recommendations with user-session context, which exploits two-layer GNNs by involving both session-based recommenders and session-aware recommenders. This method is able to capture temporal information by analyzing the user activity over sessions and the present session status, which means that our proposed method has an ability to acquire the knowledge hidden in the long-term dynamics of user sessions. Our proposed Hierarchical GNNs model outperform state-of-the-art session-based methods for personalized recommendation by taking experiments on Diginetica.

In future works, we have a plan to search the way of refining user representations by exploiting features of items and users.