ML-Tree：树结构驱动的多标签学习算法

59 浏览量更新于2024-08-30 收藏 6.82MB PDF 举报

"ML-TREE：一种基于树结构的多标签学习方法" 在多标签学习领域，目标是通过分析带有已知标签的训练样本，来预测未知实例的多个可能标签。ML-TREE是一种创新的树结构为基础的方法，它专门设计用于解决这类问题。此方法的核心在于构建一个分层树模型，通过在树的每个节点应用一对一的支持向量机（SVM）分类器进行归纳，以递归地将数据分割成子集。 ML-Tree算法的关键特性在于其预测标签向量的定义。每个树节点都有一个这样的向量，它捕捉了从树模型中传递的预测标签信息，这在多标签预测中起着关键作用。此外，这种方法还能够自动发现标签之间的关系。如果两个标签经常在同一个叶子节点上共同出现作为预测标签，那么ML-Tree会认为这两个标签具有相关性。这种共现的频率被用作估计标签关系强度的指标。为了验证ML-Tree的有效性，研究者在11个来自不同领域的实际数据集上进行了实验，并将其性能与六种成熟的多标签学习算法进行了对比。他们使用了16种常见的评价指标对这些方法进行了全面评估。此外，还应用了Friedman和Nemenyi检验来确定性能差异的统计显著性。实验结果有力地支持了ML-Tree方法的优越性，表明了其在多标签学习任务中的有效性。 ML-TREE是一种创新的、基于树结构的多标签学习算法，它通过构建分层树模型并利用预测标签向量来实现多标签预测和自动发现标签关系。这种方法在多种数据集和基准测试上的表现优于其他算法，证明了其在处理多标签预测问题时的高效性和准确性。对于那些需要处理复杂数据集和多个潜在输出标签的机器学习项目，ML-TREE提供了一种强大且有前途的解决方案。

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

WU et al.: TREE-STRUCTURE-BASED APPROACH TO MULTILABEL LEARNING 3

multilabel learning based on SVM algorithm where they use

the average fraction of incorrectly ordered pairs of labels as

cost function.

D. Tree-Based Methods

The methods that adapt the tree structure for the task of

multilabel learning are referred as the tree-based methods.

Clare and King [7] propose the ML-C4.5 algorithm which

adapts the C4.5 algorithm for multilabel data by modifying

the formula of entropy calculation. Comité et al. [2] extend

an alternative decision tree to handle multilabel data, where

the AdaBoost.MH algorithm proposed by Schapire and

Singer [1] is employed to train the multilabel alternating

decision trees. Blockeel et al. [19] propose the concept

of predictive clustering trees (PCTs). Vens et al. [20]

introduce several approaches for the PCTs algorithm to

hierarchy multilabel classiﬁcation where instances may

belong to multiple classes and these classes are organized

in a hierarchy. Kocev et al. [21] consider two ensemble

learning techniques, bagging and random forests, and apply

them to PCTs for multilabel learning. Tsoumakas et al. [23]

propose the HOMER algorithm to handle data sets with a

large number of labels. HOMER partitions the whole label set

into disjointed subsets. The partition process is implemented

by a balance clustering algorithm. Bengio et al. [24]

put forward an algorithm for learning a tree structure of

classiﬁers by optimizing the overall tree loss. This algorithm

is originally proposed for multiclass problem, but its way

of constructing a label tree is also applicable to multilabel

problem. Punera et al. [25] develop a new technique that

extracts a suitable hierarchical structure automatically from

a corpus of label documents. Deng et al. [26] present a

novel approach to learn a label tree for learning a large-scale

classiﬁcation with many classes efﬁciently. Madjarov and

Gjorgjevikj [27] build decision trees for multilabel

classiﬁcation, where the leaves contain SVM-based classiﬁers

to provide multilabel predictions. Fu et al. [28] construct a

tree structure of the labels to describe the label dependency

in multilabel data. Cesa-Bianchi et al. [29] study the

hierarchical classiﬁcation problem and introduce a reﬁned

evaluation scheme to turn the hierarchical SVM classiﬁer

into an approximated Bayes optimal classiﬁer. They

also study the problem of hierarchical classiﬁcation in a

taxonomy that allows classiﬁcations to be associated with

multiple and/or partial paths. They further propose a new

incremental algorithm to learn classiﬁer for each node of the

taxonomy [30]. Bi and Kwok [31] provide a novel hierarchical

multilabel classiﬁcation algorithm which can be used on both

tree and DAG structure hierarchies. Zhang and Zhang [32]

use a Bayesian network structure to encode the conditional

dependencies of the labels and the feature set efﬁciently, with

the feature set as the common parent of all labels. In this

network, multilabel learning is decomposed into a series of

single-label classiﬁcation problems. Zhang and Zhang [32]

describe the multilabel problem from the perspective of the

Bayesian probability and classify the multilabel learning

methods into the ﬁrst-order, the second-order, and high-order

approaches on the basis of the order of label correlations in a

Fig. 1. Example to illustrate the process of building a hierarchical tree

model. h

: the one-against-all SVM classiﬁers learnt at each node. Dash lines:

the decision boundaries of h

. b: the predictive label vector indicating the

predictive labels of a node. p: the class purity vector representing the purity of

the classes of a node. In the example, we set λ = 0.8, meaning that the classes

with purity value p

(l) ≥ 0.8 are predictive labels and their corresponding

value of b

(l) are set to be 1. The predictive labels of a parent node will be

transferred to its child nodes.

system. Our proposed method is different from these previous

works in the way that the predictive labels are formulated

at each node of the hierarchy. The predictive labels are

shared across the hierarchical tree structure and transferred

in a top-down manner. With this assumption, the original

multilabel classiﬁcation is built into the nested single-label

prediction problems in a hierarchical way, which preserves

the correlations between multiple labels.

III. ML-T

REE FOR MULTILABEL LEARNING

A. Hierarchical Tree Model

In this paper, we develop a new hierarchical tree algorithm

for multilabel learning, that is, ML-Tree. A hierarchical tree

is constructed where the root node corresponds to all the

instances in the training data set D, the instances are recur-

sively partitioned into smaller subsets while moving down the

tree. Each internal node v contains the union of the training

instances of its child nodes, D

=∪

i=1

, c

∈ children(v)

and D

∩ D

=∅, i = j for i, j = 1, 2, ...,k. At each

node v of the tree, one-against-all SVM classiﬁers are used to

partition the corresponding data set D

into disjoint subsets.

This process continues until the remaining instances at the

node cannot be further split by the induced classiﬁer.

Fig. 1 shows an example of a hierarchical tree constructed

from a multilabel data set containing three labels in 2-D feature

spaces. Each node v contains a set of one-against-all SVM

classiﬁers h

. Dash lines at the node: the decision boundaries

given by the classiﬁers. The top node contains all training

data. At the top level, the whole data set is partitioned into

threedatasubsets{A, B, C}. The second subset B is further

partitioned into two subsets {B

, B

} at the bottom level.

Each node in the tree contains two important components,

that is, class purity vector and predictive label vector (see

Fig. 1), for handling multilabel learning in training phase

as well as prediction in testing phase. Suppose we have a

set of multilabel training instances D

at node v, x

∈ D

剩余13页未读，继续阅读

weixin_38696877

粉丝: 6
资源: 929

ML-Tree：树结构驱动的多标签学习算法

AttentionXML：“ AttentionXML：用于高性能极端多标签文本分类的基于标签树的Attention-Aware深度模型”的实现

CSS-Tree:纯 CSS 树图

ML-DecisionTree-RandomForest-GridSearch-RandomizedGridSearch:机器学习-决策树，随机森林，网格搜索，随机网格搜索

pyspark-decision-tree：2.4.6的apsendendo o funcionamento daárvorededecisão

chae-tree：在elm中轻松创建多级导航

ml-algorithms:为学习目的实现了一些机器学习算法

ml-library:Python中的ML库

ML-Model-Selection:我评估了三种不同机器学习模型的性能

Crop-recommendation-using-ML-with-ETE-Heroku-Deployment:这项工作将帮助任何人知道哪种类型的农作物适合在其农场中收获，并根据机器学习技术的使用来在其中产生结果

Machine-Learning:机器学习

最新资源