学习排名算法综览：从RankNet到LambdaRank到LambdaMART

需积分: 10 188 浏览量更新于2024-07-18 收藏 194KB PDF 举报

"《从RankNet到LambdaRank再到LambdaMART：概述》是微软研究员Christopher J. C. Burges撰写的一篇论文，发表于2010年1月。该文章详细介绍了学习排序（Learning to Rank）领域中的三个关键算法：RankNet、LambdaRank以及LambdaMART。RankNet是最早的两种方法之一，它通过构造一个神经网络模型来评估查询与文档之间的相关性，从而进行排序。 RankNet首次提出了一种将排序问题转化为二分类问题的方法，通过比较两个样本对来训练模型。然而，LambdaRank是对RankNet的改进，它引入了更加复杂的结构和正则化技术，以提高模型的稳定性和泛化能力。LambdaMART在此基础上进一步发展，采用梯度提升机（Boosted Trees）技术，将弱分类器组合成强大的排序模型，尤其在实际世界中的排名问题中取得了显著的成功。 2010年的雅虎学习排序挑战赛中，LambdaMART的集成版本赢得了Track1的比赛，这证明了其在解决复杂排名任务上的优越性能。这篇综述文章旨在提供一个自包含且详尽的算法描述，以便读者更好地理解和应用这些方法。作者Burges博士在文章上传时还提到，他接收到用户请求对下载文件进行增强，可能意味着文中提供了额外的优化技巧、案例研究或对算法改进的讨论。阅读这篇论文，读者可以深入了解这三个算法的核心思想、实现细节和它们在实际场景中的应用场景，这对于从事搜索引擎优化、信息检索、推荐系统等领域的专业人士来说，是一份极具价值的学习资料。"

From RankNet to LambdaRank to LambdaMART: An Overview 3

C =

(1 − S

i j

)σ(s

− s

) + log(1 +e

−σ(s

−s

)

The cost is comfortingly symmetric (swapping i and j and changing the sign of S

i j

should leave the cost invariant): for S

i j

= 1,

C = log (1 + e

−σ(s

−s

)

while for S

i j

= −1,

C = log (1 + e

−σ(s

−s

)

Note that when s

= s

, the cost is log 2, so the model incorporates a margin (that

is, documents with different labels, but to which the model assigns the same scores,

are still pushed away from each other in the ranking). Also, asymptotically, the

cost becomes linear (if the scores give the wrong ranking), or zero (if they give the

correct ranking). This gives

∂C

∂ s

= σ



(1 − S

i j

) −

1 + e

σ(s

−s

)



= −

∂C

∂ s

(1)

This gradient is used to update the weights w

∈ R (i.e. the model parameters) so

as to reduce the cost via stochastic gradient descent

→ w

− η

∂C

∂ w

= w

− η



∂C

∂ s

∂ w

∂C

∂ s

∂ w



(2)

where η is a positive learning rate (a parameter chosen using a validation set; typi-

cally, in our experiments, 1e-3 to 1e-5). Explicitly:

δC =

∑

∂C

∂ w

δ w

∑

∂C

∂ w



−η

∂C

∂ w



= −η

∑



∂C

∂ w



< 0

The idea of learning via gradient descent is a key idea that appears throughout

this paper (even when the desired cost doesn’t have well-posed gradients, and even

when the model (such as an ensemble of boosted trees) doesn’t have differentiable

parameters): to update the model, we must specify the gradient of the cost with

respect to the model parameters w

, and in order to do that, we need the gradient

of the cost with respect to the model scores s

. The gradient descent formulation of

boosted trees (such as MART [8]) bypasses the need to compute ∂C/∂ w

by directly

modeling ∂C/∂ s

We use the convention that if two quantities appear as a product and they share an index, then

that index is summed over.

剩余19页未读，继续阅读

fyl222

粉丝: 1
资源: 23

学习排名算法综览：从RankNet到LambdaRank到LambdaMART

Learning To Rank之LambdaMART的前世今生

ranknet-tensorflow2.0:TensorFlow 2.0中RankNet到LambdaRank的实现

ranknet 与lambdarank

写一个tensorflow能用的ranknet

Ranknet方法在移动端的应用

Ranknet方法在移动终端的应用

Ranknet方法在移动端的应用的开发工具

问答搜搜匹配排序, 使用 mq2008 数据集和 ranknet 网络 有代码有数据 可直接运行

elasticsearch ltr

elasticsearch使用ltr

最新资源

问答搜搜匹配排序, 使用 mq2008 数据集和 ranknet 网络有代码有数据可直接运行