L1-TWSVM：基于鲁棒L1范数的高效分类算法

99 浏览量更新于2024-07-15 1 收藏 923KB PDF 举报

本文主要探讨了在孪生支持向量机（TWSVM）的背景下，如何通过引入最小的L1范数距离标准来提高分类任务的效率和鲁棒性。TWSVM原本基于平方的L2范数距离，这种度量方式在面对异常值（outliers）时易受影响。为了解决这个问题，研究者提出了一个新的目标函数L1-TWSVM，它采用的是更为稳健的L1范数距离作为度量标准。 L1-TWSVM的关键在于优化策略：通过最大化类间距离色散与类内距离色散的比例，从而增强模型的分类能力。相比于传统的L2范数，L1范数以其不敏感于异常值的特性，使得L1-TWSVM能够更好地处理噪声数据，提高模型的鲁棒性。然而，优化L1-TWSVM的目标函数涉及到一个非光滑的L1范数项，这增加了求解的复杂性。作者设计了一个简单但有效的迭代算法来解决这一优化问题，确保了算法的收敛性，即使在遇到非光滑性的情况下也能找到最优解。这个算法的实用性得到了实证验证，通过在UCI数据集和合成数据集上的广泛实验，L1-TWSVM显示出优越的性能，无论是在准确度还是鲁棒性上，都优于当前相关领域的最新技术。本文的贡献在于提出了一种改进的TWSVM版本，即L1-TWSVM，它利用了L1范数的距离度量，不仅提高了分类的准确性，还显著增强了模型在面对异常值情况下的鲁棒性。这对于实际应用中的机器学习任务，尤其是在处理噪声数据或异常情况下的分类问题具有重要的意义。

Machine Learning (2019) 108:993–1018 997

as we now state. It is easy to derive the solutions of (4)and(5) by solving the generalized

eigenvalue problems

 λ

, z

 0(6)

 λ

, z

 0(7)

where the minimum of (4) is attained at an eigenvector corresponding to the smallest eigen-

value λ

of (6). Consequently, if z

denotes the eigenvector corresponding to λ

, then the

augmented vector z





)

determines the plane x

+ b

 0, which is close to

data points of Class 1. Similarly, the augmented vector z





)

determines the plane

+ b

 0, which is close to data points of Class 2.

2.2 TWSVM

In this section, after a brief review of GEPSVM, we introduce TWSVM, which is an improved

version of GEPSVM. To obtain two planes, TWSVM solves two convex programming prob-

lems rather than solving a system of two linear equations as GEPSVM does (Mangasarian

and Wild 2006). The two objective functions of TWSVM are expressed as follows:

min

||Aw

+ e

s.t. −

(

+ e

)

+ q

≥ e

, q

≥ 0(8)

min

||Bw

+ e

s.t.

(

+ e

)

+ q

≥ e

, q

≥ 0(9)

where ||·||

denotes the L2-norm, q

and q

are slack vectors, and c

and c

are nonnegative

penalty coefﬁcients, which are the balance factors of the positive and negative samples,

respectively, and can overcome the problem of sample imbalance in TWSVM. It should be

noted that in TWSVM, the distance is measured by the L2-norm, which is likely to exaggerate

the effect of outliers by the square operation. The optimization strategy of TWSVM is that

points of the same class are clustered as compactly as possible and are as far as possible

from data in the other class, which guarantees the minimization of the objective function. By

solving formulas (8)and(9), we can obtain two nonparallel planes:

+ b

 0, x

+ b

 0(10)

A new data point x is assigned to Class 1 or Class 2 depending on its proximity to each

of the two nonparallel planes. We can obtain the corresponding Wolfe dual problems of

formulas (8)and(9):

max

α −





−1

s.t. 0 ≤ α ≤ c

(11)

max

β −





−1

s.t. 0 ≤ β ≤ c

(12)

123

998 Machine Learning (2019) 108:993–1018

where α ∈ R

and β ∈ R

are Lagrange multipliers, we can derive two nonparallel planes

using α and β:







−





−1













−1

β (13)

Note that the inverse matrices





−1

and





−1

in Eq. (13) easily encounter

singularity problems. To prevent matrix singularity, the regularization term εI,whereε is

a positive scalar that is small enough to preserve the structure of the data, is introduced

(Jayadeva and Chandra 2007; Mangasarian and Wild 2006). Because



H + εI



−1

and



G + εI



−1

are positive deﬁnite, they do not suffer from singularity problems.

3 Eﬃcient and robust TWSVM based on L1-norm distance

TWSVM has become a hotspot in the research of data classiﬁcation due to its good clas-

siﬁcation performance. However, in TWSVM, the distance is measured by the L2-norm. It

is well known that the squared L2-norm distance is sensitive to outliers, which implies that

abnormal observations may affect the solution obtained by TWSVM. In the literature (Ding

et al. 2006;Gao2008;Kwak2008;Lietal.2015a; Nie et al. 2015; Wright et al. 2009), the

L1-norm distance is usually considered as a robust alternative to the L2-norm distance for

improving the generalization ability and ﬂexibility of the model. Motivated by the basic idea

of L1-norm-based modeling, we propose a robust classiﬁer based on the L1-norm distance

metric, which replaces the squared L2-norm distance in the distance metric learning objective

functions in formulas (8)and(9), thereby leading to the following optimization problems:

min

||Aw

+ e

s.t. −

(

+ e

)

+ q

≥ e

, q

≥ 0(14)

min

||Bw

+ e

s.t.

(

+ e

)

+ q

≥ e

, q

≥ 0 (15)

where ||·||

denotes the L1-norm. In a solution that minimizes the objective functions, each

plane is as close as possible to one of the two classes and as far as possible from the other

class. Because formulas (14)and(15) are convex optimization problems with non-convex

constraints in the form of inequalities, they have the local optimal solutions, and we can

obtain two nonparallel planes by solving them:

+ b

 0, x

+ b

 0 (16)

The original problems in formulas (14)and(15) can be optimized in the following forms:

min





i1



+ e





+ c

s.t. −

(

+ e

)

+ q

≥ e

, q

≥ 0 (17)

123

剩余25页未读，继续阅读

weixin_38600460

粉丝: 5

L1-TWSVM：基于鲁棒L1范数的高效分类算法

通信无人机无线传感器网络中的节能数据采集Matlab复现.rar

【优化算法】孪生支持向量机（TWSVM）【含Matlab源码 1257期】.zip

llap twsvm

便携式盲人无障碍辅助通行装置（含GY53-L1激光测距，HC-SR04超声波测距，震动马达）

STM32实现激光测距传感器测距WT-VL53L0 L1（标准库与HAL库实现）

STM32F103驱动VL53L1激光测距传感器【支持STM32F1系列单片机】.zip

STM32H750驱动VL53L1激光测距传感器【支持STM32H7系列单片机】.zip

VL53L1X测距API深度解析：强制测距与数据获取流程

光电测距仪检定规范标准解读

光电测距高程导线测量规范技术标准

最新资源