ICIC Express Letters
Part B: Applications ICIC International
c
2017 ISSN 2185-2766
Volume 8, Number 8(tentative), August 2017 pp. 1–ICICELB-1703-019
LAMBDAXGB: RESEARCH ON LEARNING TO RANK
BASED ON LAMBDAMART
Liyan Xiong
1
, Xiaoxia Chen
1
, Xiaohui Huang
1
, Weichun Huang
2
Maosheng Zhong
1
and Hui Zeng
1
1
Scho ol of Information Engineering
2
School of Software Engineering
East China Jiaotong University
No. 808, Shuanggang East Avenue, Nanchang 330013, P. R. China
{ xly ecjtu; chenxiao970508; hwc1968 }@163.com; hxh016@hotmail.com
zhongmaosheng@sina.com; 331549185@qq.com
Received March 2017; accepted May 2017
Abstract. In this paper, RankNet, LambdaRank, LambdaMART and the XGBoost are
studied and analyzed. The idea of improving the LambdaMART is proposed, that is,
adding the regulation to the loss function of LambdaMART. Two commonly regulation-
s L1 and L2 are added to the loss function of the LambdaMART and three algorithms
are proposed, including LambdaXGB L1, LambdaXGB L2 and LambdaXGB. Through the
MQ2008 dataset, this paper reveals the NDCG evaluation result compared with RankNet
and LambdaMART, and verifies the effectiveness of these algorithms. The results demon-
strate that our approach gives state-of-the-art results on a rank of dataset.
Keywords: RankNet, LambdaMART, LambdaXGB, LambdaXGB L1, LambdaXGB
L2
1. Introduction. With the increasing selection, search engines and recommendation sys-
tems are more and more dependent on the sort. However, single factor is only considered
by the traditional sorting algorithm. With the exponential growth of processed data, mul-
tiple factors need to be combined for sorting, endowed with different weights. So that is
something about Learning to Rank [1]. Learning to Rank is a sort of supervised learning
method, which can get a rank model according to the training data, and then use this
rank model to sort the data.
The pairwise is transformed into binary classification problem in ranking the documents.
For the documents of the same query, the training samples of binary classifier training
are obtained for any two different labels. All the document pairs are sorted to get a
partial order, and the final rank is achieved. The pairwise approach includes RankNet,
Lamb daRank, LambdaMART, Ranking SVM, IR SVM, RankBoost.
This work contributes to the follow aspect: distinguished from the existing method
of LambdaMART, we add the regulation to the loss function of LambdaMART to build
the new models, including LambdaXGB L1, LambdaXGB L2 and LambdaXGB. And the
experiment demonstrates that our approach gives state-of-the-art results on a rank of
problems.
The rest part of the paper is structured as follows. We discuss related work in Section
2. We discuss the LambdaXGB model in Section 3. The experiment and result analysis
are shown in Section 4. Finally, we conclude the paper and discuss the directions of the
future works in Section 5.
2. Related Work. The RankNet is an underlying model, which maps an input feature
vector to a number during training. For a given query, inputting the document of the
query, there is output ranking model f(d, w). The cross entropy cost function penalizes
1