没有合适的资源?快使用搜索试试~ 我知道了~
首页L1-TWSVM:基于鲁棒L1范数的高效分类算法
L1-TWSVM:基于鲁棒L1范数的高效分类算法
0 下载量 119 浏览量
更新于2024-07-15
1
收藏 923KB PDF 举报
本文主要探讨了在孪生支持向量机(TWSVM)的背景下,如何通过引入最小的L1范数距离标准来提高分类任务的效率和鲁棒性。TWSVM原本基于平方的L2范数距离,这种度量方式在面对异常值(outliers)时易受影响。为了解决这个问题,研究者提出了一个新的目标函数L1-TWSVM,它采用的是更为稳健的L1范数距离作为度量标准。 L1-TWSVM的关键在于优化策略:通过最大化类间距离色散与类内距离色散的比例,从而增强模型的分类能力。相比于传统的L2范数,L1范数以其不敏感于异常值的特性,使得L1-TWSVM能够更好地处理噪声数据,提高模型的鲁棒性。然而,优化L1-TWSVM的目标函数涉及到一个非光滑的L1范数项,这增加了求解的复杂性。 作者设计了一个简单但有效的迭代算法来解决这一优化问题,确保了算法的收敛性,即使在遇到非光滑性的情况下也能找到最优解。这个算法的实用性得到了实证验证,通过在UCI数据集和合成数据集上的广泛实验,L1-TWSVM显示出优越的性能,无论是在准确度还是鲁棒性上,都优于当前相关领域的最新技术。 本文的贡献在于提出了一种改进的TWSVM版本,即L1-TWSVM,它利用了L1范数的距离度量,不仅提高了分类的准确性,还显著增强了模型在面对异常值情况下的鲁棒性。这对于实际应用中的机器学习任务,尤其是在处理噪声数据或异常情况下的分类问题具有重要的意义。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/15665526/bg5.jpg)
Machine Learning (2019) 108:993–1018 997
as we now state. It is easy to derive the solutions of (4)and(5) by solving the generalized
eigenvalue problems
Ez
1
λ
1
Fz
1
, z
1
0(6)
Lz
2
λ
2
Mz
2
, z
2
0(7)
where the minimum of (4) is attained at an eigenvector corresponding to the smallest eigen-
value λ
1
of (6). Consequently, if z
1
denotes the eigenvector corresponding to λ
1
, then the
augmented vector z
1
w
T
1
b
1
)
T
determines the plane x
T
w
1
+ b
1
0, which is close to
data points of Class 1. Similarly, the augmented vector z
2
w
T
2
b
2
)
T
determines the plane
x
T
w
2
+ b
2
0, which is close to data points of Class 2.
2.2 TWSVM
In this section, after a brief review of GEPSVM, we introduce TWSVM, which is an improved
version of GEPSVM. To obtain two planes, TWSVM solves two convex programming prob-
lems rather than solving a system of two linear equations as GEPSVM does (Mangasarian
and Wild 2006). The two objective functions of TWSVM are expressed as follows:
min
w
1
,b
1
1
2
||Aw
1
+ e
1
b
1
||
2
2
+c
1
e
T
2
q
1
s.t. −
(
Bw
1
+ e
2
b
1
)
+ q
1
≥ e
2
, q
1
≥ 0(8)
min
w
2
,b
2
1
2
||Bw
2
+ e
2
b
2
||
2
2
+c
2
e
T
1
q
2
s.t.
(
Aw
2
+ e
1
b
2
)
+ q
2
≥ e
1
, q
2
≥ 0(9)
where ||·||
2
denotes the L2-norm, q
1
and q
2
are slack vectors, and c
1
and c
2
are nonnegative
penalty coefficients, which are the balance factors of the positive and negative samples,
respectively, and can overcome the problem of sample imbalance in TWSVM. It should be
noted that in TWSVM, the distance is measured by the L2-norm, which is likely to exaggerate
the effect of outliers by the square operation. The optimization strategy of TWSVM is that
points of the same class are clustered as compactly as possible and are as far as possible
from data in the other class, which guarantees the minimization of the objective function. By
solving formulas (8)and(9), we can obtain two nonparallel planes:
x
T
w
1
+ b
1
0, x
T
w
2
+ b
2
0(10)
A new data point x is assigned to Class 1 or Class 2 depending on its proximity to each
of the two nonparallel planes. We can obtain the corresponding Wolfe dual problems of
formulas (8)and(9):
max
α
e
T
2
α −
1
2
α
T
G
H
T
H
−1
G
T
α
s.t. 0 ≤ α ≤ c
1
e
2
(11)
max
β
e
T
1
β −
1
2
β
T
H
G
T
G
−1
H
T
β
s.t. 0 ≤ β ≤ c
2
e
1
(12)
123
![](https://csdnimg.cn/release/download_crawler_static/15665526/bg6.jpg)
998 Machine Learning (2019) 108:993–1018
where α ∈ R
m
2
and β ∈ R
m
1
are Lagrange multipliers, we can derive two nonparallel planes
using α and β:
z
1
w
T
1
b
1
T
−
H
T
H
−1
G
T
α
z
2
w
T
2
b
2
T
G
T
G
−1
H
T
β (13)
Note that the inverse matrices
H
T
H
−1
and
G
T
G
−1
in Eq. (13) easily encounter
singularity problems. To prevent matrix singularity, the regularization term εI,whereε is
a positive scalar that is small enough to preserve the structure of the data, is introduced
(Jayadeva and Chandra 2007; Mangasarian and Wild 2006). Because
H
T
H + εI
−1
and
G
T
G + εI
−1
are positive definite, they do not suffer from singularity problems.
3 Efficient and robust TWSVM based on L1-norm distance
TWSVM has become a hotspot in the research of data classification due to its good clas-
sification performance. However, in TWSVM, the distance is measured by the L2-norm. It
is well known that the squared L2-norm distance is sensitive to outliers, which implies that
abnormal observations may affect the solution obtained by TWSVM. In the literature (Ding
et al. 2006;Gao2008;Kwak2008;Lietal.2015a; Nie et al. 2015; Wright et al. 2009), the
L1-norm distance is usually considered as a robust alternative to the L2-norm distance for
improving the generalization ability and flexibility of the model. Motivated by the basic idea
of L1-norm-based modeling, we propose a robust classifier based on the L1-norm distance
metric, which replaces the squared L2-norm distance in the distance metric learning objective
functions in formulas (8)and(9), thereby leading to the following optimization problems:
min
w
1
,b
1
1
2
||Aw
1
+ e
1
b
1
||
1
+c
1
e
T
2
q
1
s.t. −
(
Bw
1
+ e
2
b
1
)
+ q
1
≥ e
2
, q
1
≥ 0(14)
min
w
2
,b
2
1
2
||Bw
2
+ e
2
b
2
||
1
+c
2
e
T
1
q
2
s.t.
(
Aw
2
+ e
1
b
2
)
+ q
2
≥ e
1
, q
2
≥ 0 (15)
where ||·||
1
denotes the L1-norm. In a solution that minimizes the objective functions, each
plane is as close as possible to one of the two classes and as far as possible from the other
class. Because formulas (14)and(15) are convex optimization problems with non-convex
constraints in the form of inequalities, they have the local optimal solutions, and we can
obtain two nonparallel planes by solving them:
x
T
w
1
+ b
1
0, x
T
w
2
+ b
2
0 (16)
The original problems in formulas (14)and(15) can be optimized in the following forms:
min
w
1
,b
1
1
2
m
1
i1
a
T
i
w
1
+ e
i
1
b
1
d
i
2
+ c
1
e
T
2
q
1
s.t. −
(
Bw
1
+ e
2
b
1
)
+ q
1
≥ e
2
, q
1
≥ 0 (17)
123
剩余25页未读,继续阅读
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/default.jpg!1)
weixin_38600460
- 粉丝: 5
- 资源: 955
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 新型矿用本安直流稳压电源设计:双重保护电路
- 煤矿掘进工作面安全因素研究:结构方程模型
- 利用同位素位移探测原子内部新型力
- 钻锚机钻臂动力学仿真分析与优化
- 钻孔成像技术在巷道松动圈检测与支护设计中的应用
- 极化与非极化ep碰撞中J/ψ的Sivers与cos2φ效应:理论分析与COMPASS验证
- 新疆矿区1200m深孔钻探关键技术与实践
- 建筑行业事故预防:综合动态事故致因理论的应用
- 北斗卫星监测系统在电网塔形实时监控中的应用
- 煤层气羽状水平井数值模拟:交替隐式算法的应用
- 开放字符串T对偶与双空间坐标变换
- 煤矿瓦斯抽采半径测定新方法——瓦斯储量法
- 大倾角大采高工作面设备稳定与安全控制关键技术
- 超标违规背景下的热波动影响分析
- 中国煤矿选煤设计进展与挑战:历史、现状与未来发展
- 反演技术与RBF神经网络在移动机器人控制中的应用
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)