Variational Autoencoders for Collaborative Filtering
Dawen Liang
Netix
Los Gatos, CA
dliang@netix.com
Rahul G. Krishnan
MIT
Cambridge, MA
rahulgk@mit.edu
Matthew D. Homan
Google AI
San Francisco, CA
mhoman@google.com
Tony Jebara
Netix
Los Gatos, CA
tjebara@netix.com
ABSTRACT
We extend variational autoencoders (vaes) to collaborative ltering
for implicit feedback. This non-linear probabilistic model enables us
to go beyond the limited modeling capacity of linear factor models
which still largely dominate collaborative ltering research. We
introduce a generative model with multinomial likelihood and use
Bayesian inference for parameter estimation. Despite widespread
use in language modeling and economics, the multinomial likeli-
hood receives less attention in the recommender systems literature.
We introduce a dierent regularization parameter for the learning
objective, which proves to be crucial for achieving competitive per-
formance. Remarkably, there is an ecient way to tune the parame-
ter using annealing. The resulting model and learning algorithm has
information-theoretic connections to maximum entropy discrimi-
nation and the information bottleneck principle. Empirically, we
show that the proposed approach signicantly outperforms several
state-of-the-art baselines, including two recently-proposed neural
network approaches, on several real-world datasets. We also pro-
vide extended experiments comparing the multinomial likelihood
with other commonly used likelihood functions in the latent factor
collaborative ltering literature and show favorable results. Finally,
we identify the pros and cons of employing a principled Bayesian
inference approach and characterize settings where it provides the
most signicant improvements.
KEYWORDS
Recommender systems, collaborative ltering, implicit feedback,
variational autoencoder, Bayesian models
ACM Reference Format:
Dawen Liang, Rahul G. Krishnan, Matthew D. Homan, and Tony Jebara.
2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of
The 2018 Web Conference (WWW 2018). ACM, New York, NY, USA, 10 pages.
https://doi.org/10.1145/3178876.3186150
This paper is published under the Creative Commons Attribution-NonCommercial-
NoDerivs 4.0 International (CC BY-NC-ND 4.0) license. Authors reserve their rights to
disseminate the work on their personal and corporate Web sites with the appropriate
attribution.
WWW 2018, April 23–27, 2018, Lyon, France
©
2018 IW3C2 (International World Wide Web Conference Committee), published
under Creative Commons CC BY-NC-ND 4.0 License.
ACM ISBN 978-1-4503-5639-8/18/04.
https://doi.org/10.1145/3178876.3186150
1 INTRODUCTION
Recommender systems are an integral component of the web. In
a typical recommendation system, we observe how a set of users
interacts with a set of items. Using this data, we seek to show users
a set of previously unseen items they will like. As the web grows
in size, good recommendation systems will play an important part
in helping users interact more eectively with larger amounts of
content.
Collaborative ltering is among the most widely applied ap-
proaches in recommender systems. Collaborative ltering predicts
what items a user will prefer by discovering and exploiting the
similarity patterns across users and items. Latent factor models
[
13
,
19
,
38
] still largely dominate the collaborative ltering research
literature due to their simplicity and eectiveness. However, these
models are inherently linear, which limits their modeling capacity.
Previous work [
27
] has demonstrated that adding carefully crafted
non-linear features into the linear latent factor models can signif-
icantly boost recommendation performance. Recently, a growing
body of work involves applying neural networks to the collabora-
tive ltering setting with promising results [14, 41, 51, 54].
Here, we extend variational autoencoders (vaes) [
24
,
37
] to col-
laborative ltering for implicit feedback. Vaes generalize linear
latent-factor models and enable us to explore non-linear proba-
bilistic latent-variable models, powered by neural networks, on
large-scale recommendation datasets. We propose a neural gen-
erative model with multinomial conditional likelihood. Despite
being widely used in language modeling and economics [
5
,
30
],
multinomial likelihoods appear less studied in the collaborative
ltering literature, particularly within the context of latent-factor
models. Recommender systems are often evaluated using ranking-
based measures, such as mean average precision and normalized
discounted cumulative gain [
21
]. Top-
N
ranking loss is dicult to
optimize directly and previous work on direct ranking loss mini-
mization resorts to relaxations and approximations [
49
,
50
]. Here,
we show that the multinomial likelihoods are well-suited for mod-
eling implicit feedback data, and are a closer proxy to the ranking
loss relative to more popular likelihood functions such as Gaussian
and logistic.
Though recommendation is often considered a big-data problem
(due to the huge numbers of users and items typically present in a
recommender system), we argue that, in contrast, it represents a
uniquely challenging “small-data” problem: most users only inter-
act with a tiny proportion of the items and our goal is to collectively
arXiv:1802.05814v1 [stat.ML] 16 Feb 2018