Next Item Recommendation with Self-Aention
Shuai Zhang
UNSW and Data61, CSIRO
Sydney, NSW 2052, Australia
shuai.zhang@student.unsw.edu.au
Yi Tay
Nanyang Technological University
Singapore
ytay017@e.ntu.edu.sg
Lina Yao
University of New South Wales
Sydney, NSW 2052, Australia
lina.yao@unsw.edu.au
Aixin Sun
Nanyang Technological University
Singapore
axsun@ntu.edu.sg
ABSTRACT
In this paper, we propose a novel sequence-aware recommendation
model. Our model utilizes self-aention mechanism to infer the
item-item relationship from user’s historical interactions. With
self-aention, it is able to estimate the relative weights of each item
in user interaction trajectories to learn beer representations for
user’s transient interests. e model is nally trained in a metric
learning framework, taking both short-term and long-term inten-
tions into consideration. Experiments on a wide range of datasets
on dierent domains demonstrate that our approach outperforms
the state-of-the-art by a wide margin.
KEYWORDS
Recommender Systems; Sequential Recommendation; Self-Aention
ACM Reference format:
Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. 2018. Next Item Recommen-
dation with Self-Aention. In Proceedings of Conference Submission, , Month
2018/9, 10 pages.
DOI:
1 INTRODUCTION
Anticipating a user’s next interaction lives at the heart of making
personalized recommendations. e importance of such systems
cannot be overstated, especially given the ever growing amount
of data and choices that consumers are faced with each day [
26
].
Across a diverse plethora of domains, a wealth of historical interac-
tion data exists, e.g., click logs, purchase histories, views etc., which
have, across the years, enabled many highly eective recommender
systems.
Exploiting historical data to make future predictions have been
the cornerstone of many machine learning based recommender
systems. Aer all, it is both imperative and intuitive that a user’s
past interactions are generally predictive of their next. To this end,
many works have leveraged upon this structural co-occurrence,
along with the rich sequential paerns, to make informed decisions.
Our work is concerned with building highly eective sequential
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
Conference Submission,
© 2018 Copyright held by the owner/author(s). . .. $15.00
DOI:
recommender systems by leveraging these auto-regressive tenden-
cies.
In the recent years, neural models such as recurrent neural
network (RNN)/convolutional neural network (CNN) are popu-
lar choices for the problem at hand [
9
,
23
]. In recurrent models,
the interactions between consecutive items are captured by a re-
current matrix and long-term dependencies are persisted in the
recurrent memory while reading. On the other hand, convolution
implicitly captures interactions by sliding parameterized transfor-
mations across the input sequence [
7
]. However, when applied to
recommendation, both models suer from a shortcoming. at is,
they fail to
explicitly
capture item-item interactions
1
across the
entire user history. e motivation for modeling item-item relation-
ships within a user’s context history is intuitive, as it is more oen
than not, crucial to understand ne-grained relationships between
individual item pairs instead of simply glossing over them. All in
all, we hypothesize that providing an inductive bias for our models
would lead to improve representation quality, eventually resulting
in a signicant performance improvement within the context of
sequential recommender systems.
To this end, this paper proposes a new neural sequential recom-
mender system where sequential representations are learned via
modeling not only consecutive items but across
all user interac-
tions
in the current window. As such our model can be considered
as a ‘local-global’ approach. Overall, our intuition manifests in
the form of an aention-based neural model that explicitly invokes
item-item interactions across the entire user’s historical transaction
sequence. is not only enables us to learn global/long-range repre-
sentations, but also short-term information between
k
-consecutive
items. Based on this self-matching matrix, we learn to aend over
the interaction sequence to select the most relevant items to form
the nal user representation. Our experiments show that the pro-
posed model outperforms the state-of-the-art sequential recommen-
dation models by a wide margin, demonstrating the eectiveness
of not only modeling local dependencies but also going global.
Our model takes the form of a metric learning framework in
which the distance between the self-aended representation of
a user and the prospective (golden) item is drawn closer during
training. To the best of our knowledge, this is the rst proposed
1
In RNNs, this is captured via memory persistence. While in CNNs, this is only weakly
captured by the sliding-window concatenated transformations. In both cases, there is
no explicit interaction.
arXiv:1808.06414v2 [cs.IR] 25 Aug 2018