6
multi-source transfer learning, which mainly has the follow-
ing two steps in e ach iteration.
1. Candidate Class ifier Construction: A group of candi-
date weak classifiers are respectively trained on the
weighted instances in the pairs of each source domain
and the target domain, i.e., D
S
i
∪ D
T
(i = 1, · · · , m
S
).
2. Instance Weighting: A classifier which has the minimal
classification error rate
¯
δ on the target domain instances
is selecte d (denoted by j), and then is used for updating
the weights of the instances in D
S
j
and D
T
.
Finally, the selected classifiers from each iteration are com-
bined to form the fi nal classifier. Another parameter-based
algorithm, i.e., TaskTrAdaBoost, is also proposed in that
work [26], which is introduced in Sect ion 5.3.
Some approaches realize instance weighting stra tegy in a
heuristic way. For example, Jiang and Zhai proposed a gen-
eral weighting framework for the adaptation of instances
[27]. According to the paper, three types of instances (i.e., la-
beled source-domain, labeled target-domain, and unlabeled
target-domain insta nces) are used to construct the target
classifier. There are three terms in the obj e ctive function,
which are designed corresponding to the instances’ types
for minimizing the cross-entropy loss.
• Labeled Target-domain Instance: The classifier should mini-
mize the cross-entropy loss on them, which is actually a
standard supervis e d learning task.
• Unlabeled Target-domain Instance: These instances’ true con-
ditional dist ributions P (y|x
T,U
i
) are unknown and should
be estimated. A possib le solution is to train an auxiliary
classifier on the labeled source-domain and target-domain
instances to help est imate the conditional distributions or
assign pseudo labels to these instances.
• Labeled Source-domain Instance: The authors define the
weight of x
S,L
i
as the product of two parts, i.e., α
i
and
β
i
. T he weight β
i
is ideally equal to P
T
(x
i
)/P
S
(x
i
),
which can be estimated by non-parametric methods such
as KMM or can be set uniformly in t he worst ca se. The
weight α
i
is us e d to filter ou t the source-domain instances
that differ greatly from the target domain.
A heuristic method can be used to produce the value of α
i
,
which contains the following three steps.
1. Auxiliary Classifier Construction: An auxiliary classifier
trained on the labeled target-domain instances are used
to classify the unlabeled source-domain ins tances.
2. Instance Ranking: The source-domain instances are
ranked based on the probabilistic prediction results.
3. Heuristic Weighting (β
i
): The weights of the top-k
source-domain instances with wrong predictions are set
to zero, and the weights of others are set to one.
The objective function of this framework consists of four
terms, i.e., the above-mentioned three terms with three
tradeoff parameters controlling t he balance among the types
of instances and a regularizer controlling the complexity of
the model.
4.2 Feature Transformation Strategy
Feature transformation strategy is often adopted in feature-
based approaches. Feature-based approaches transform
each original feature into a new feat ure representat ion for
TABLE 2
Metrics Adopted in Transfer Learning.
Measurement Related Algorithms
Maximum Mean Discrepancy [28] [29] [30 ] [31] [32 ]· · ·
Kullback-Leibler Divergence [33] [34] [35 ] [36] [37 ]· · ·
Jensen-Shannon Divergence [38] [39] [40 ] [41] [42 ]· · ·
Bregman Divergence [43] [44] [45 ] [46] [47 ]· · ·
Hilbert-Schmidt Independ e nce Criterion [48] [29] [49] [50] [51]· · ·
transfer learning. The objectives of constructing a new fea-
ture representation include minimizing the marginal and the
conditional distribution difference, preserving the proper-
ties or the potential structures of the data, and finding the
correspondence between features. The operations of feature
transformation can be divided into three types, i.e., feature
augmentation, feature reduction, and feature alignment. Be-
sides, feature reduction can be further divided into several
types such as feature mapping, feature clustering, feature
selection, and feature encoding. A complete feature trans-
formation process designed in an a lgorithm may consist of
several operations.
4.2.1 Distribution Difference Metric
One primary objective of feature transformation is to reduce
the distribution difference of the source and the target do-
main instances. Therefore, how to measure the distribution
difference or the s imilarity between domains effectively is
an important issue.
The measurement termed Maximum Mean Discrepancy
(MMD) is widely used in the field of transfer learning,
which is formulated as follows [28]:
MMD(X
S
, X
T
) =
1
n
S
n
S
X
i=1
Φ(x
S
i
) −
1
n
T
n
T
X
j=1
Φ(x
T
j
)
2
H
.
MMD can be easily comp uted by using kernel trick. Briefly,
MMD quantifies the distribution difference by calcuating
the distance of the mean valu e s of the instances in a RKHS.
Note that the abov e -mentioned KMM actually produces
the weights of instances b y minimizing the MMD distance
between domains.
Table. 2 lists some commonly used metrics and the
related algorithms. In addition to Table. 2, there a re s ome
other measurement criteria adopted in transfer learning,
including Wasserstein distance [52], [53], Central Moment
Discrepancy [54], etc. Some studies focus on optimizing
and improving the existing me asurements. Take MMD as
an example. Gretton et al. proposed a multi-kernel version
of MMD, i.e., MK-MMD [55], which takes advantage of
multiple kernels. Besides, Yan et al. proposed a weighted
version of MMD [56], which attempts to address the issue
of class weight bias.
4.2.2 Feature Augmentation
Feature au gm e ntation ope rations are widely used in fea-
ture transformation, especially in symmetric feature-based
approaches. To be more specific, there are several ways to
realize feature augmentation such as feature replication and
剩余26页未读，继续阅读
iamcbl_hacker
- 粉丝: 1
- 资源: 4
我的内容管理
展开
最新资源
- Vue实现iOS原生Picker组件：详细解析与实现思路
- Arduino蓝牙小车：参数调试与功能控制
- 百度Java面试精华：200页精选资源涵盖核心知识点
- Swift使用CoreData填坑指南：CoreData在Swift 3.0的变化
- 微距离无线充电器创新设计及其实验探索
- MTK Android平台开发全攻略：44步详解流程
- RecyclerView全面解析：替代ListView的新选择
- Android开发：自动适配中英文键盘解决方案
- Android调用WebService接口教程
- Android开发：BitmapUtil图片处理全解析与实例
- Android多线程断点续传实现详解
- PCA算法在人脸识别会议签到系统中的应用
- EventBus 3.0：Android事件总线详解与实战应用
- Android FileUtil：全面解析文件操作实用技巧与实例
- RecyclerView添加头部和尾部实战教程
- Android实现微博滑动固定顶部栏实战与优化
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
