鲁棒灵活判别词典学习：联合子空间恢复与增强局部性
需积分: 9 91 浏览量
更新于2024-07-14
收藏 2.45MB PDF 举报
"本文介绍了一种名为鲁棒灵活判别词典学习（RFDDL）的新方法，该方法结合了联合子空间恢复和增强局部性，旨在提高数据表示和分类的性能。RFDDL针对数据和原子中的噪声及稀疏错误进行鲁棒处理，同时确保标签一致性和准确的重构。通过恢复基础干净数据和原子子空间，RFDDL增强了数据编码的鲁棒性。此外，它通过灵活的重建误差最小化处理非线性流形上的数据，防止过拟合。RFDDL还利用灵活稀疏编码误差来编码标签一致性，以及在恢复的原子上定义拉普拉斯矩阵以编码局部性，结合类内紧凑性和类间分离，形成具有判别性的系数约束和分类器。实验结果证明了RFDDL方法的有效性。"
本文关注的焦点是机器学习领域中的特征表示和分类问题，特别是通过词典学习来提升模型的性能。词典学习（Dictionary Learning, DL）是一种将数据表示为稀疏编码的线性组合的方法，常用于图像处理、信号处理等领域。RFDDL是DL的一个扩展，它在两个关键方面进行了创新：
1. 联合子空间恢复：RFDDL首先考虑数据和原子中的噪声和稀疏错误，通过联合恢复基础干净数据和原子子空间，提高了数据表示的准确性和鲁棒性。这一步骤有助于减少噪声影响，提高编码的质量。
2. 增强局部性：在恢复的子空间中，RFDDL强调了数据的局部特性。通过编码局部性，可以更好地捕捉数据的内在结构，这对于非线性数据尤其重要。这有助于提升分类的准确性。
此外，RFDDL还解决了两个关键问题：
3. 灵活的重建误差最小化：为了解决非线性流形上数据的处理，RFDDL采用灵活的方式最小化重建误差，避免了过拟合，使得模型能够适应复杂的数据分布。
4. 标签一致性编码：RFDDL引入了区别性的灵活稀疏代码错误，促进了系数的平滑，这有助于保持标签的一致性，确保分类结果的可靠性。
最后，RFDDL通过在原子上构建拉普拉斯矩阵，结合了类内紧凑性和类间分离的信息，形成了具有判别性的系数约束，这进一步优化了分类器的性能。
RFDDL是词典学习的一种增强形式，它通过联合子空间恢复、增强局部性、灵活的重建误差最小化和标签一致性编码等策略，提升了数据表示的鲁棒性和分类的准确性。这一方法对于处理噪声数据和非线性结构数据的机器学习任务具有重要的应用价值。
where H is the label set with binary entries (0 or 1), α and β are
positive scalars.
12
, ,...,
KN
N
Q q q q
are “discriminative”
sparse codes for
X
, where
T
0 ,1,1, 0
K
i
q
is an ideal
“discriminative” sparse code for x
i
if the nonzero values of q
i
occur at those indices where x
i
and atom d
i
share the same label
[9]. Suppose that
9
1
i
i
Xx
has 3 classes, where
1 2 3 4
, , ,x x x x
are
from class 1,
56
,xx
are from class 2, and the rest from class 3,
the “discriminative” sparse codes matrix Q is defined as
1 1 1 1 0 0 0 0 0
1 1 1 1 0 0 0 0 0
0 0 0 0 1 1 0 0 0
0 0 0 0 1 1 0 0 0
0 0 0 0 0 0 1 1 1
0 0 0 0 0 0 1 1 1
Q=
. (3)
Thus, term
2
|| ||
F
Q AS
is the discriminative sparse-code error,
where
KK
A
converts the original sparse codes in S to be
more discriminative in a feature subspace
K
.
2
|| ||
F
Q AS
can
encourage the label consistency in the resulting codes, but
A
is
arbitrarily defined, so it cannot preserve local information and
inherit the structure information of samples. Setting all nonzero
entries to ones is also too hard, since the coefficients in S are
essentially soft, i.e., a large value s
i,j
means that the contribution
of each
j
x
to reconstruct
i
x
is large, and small otherwise.
Note that D-KSVD and LC-KSVD can be equivalent with
the uniform atom allocation [36], which is also based on the
identical initialization conditions and optimization methods. By
the equivalence, we can reformulate LC-KSVD as
2
* * *
,,
0
0
, , argmin
. . , 1,2,...,
D S W
F
i
XD
D S W S
HW
s t s T i N
, (4)
which is just the problem of D-KSVD, where
is the number
of dictionary atoms allocated per class. The analysis in [36] also
shows that D-KSVD is preferable because of its simplicity and
efficiency, compared to the LC-KSVD algorithm.
B. Review of LCLE-DL
LCLE-DL is another one related DL method, so we also briefly
revisit it. LCLE-DL calculates a discriminative dictionary D by
combining the label embedding of atoms and locality constraint
of atoms jointly. The problem of LCLE-DL is defined as
22
22
, , ,
22
2
min
, . . 1, 1, ,
TT
D S V L
i
X DS tr S LS X DV tr V UV
S V st d i K
, (5)
where S and V denote the coefficient matrices,
2
2
X DS
and
2
2
X DV
denote the reconstruction error, and
2
2
SV
is the
regularization used to transfer the label constraint
T
Tr V UV
to/from the locality constraint
T
Tr S LS
. U is the scaled label
matrix constructed by the labels of dictionary.
,
and
are
parameters. L is a graph Laplacian matrix defined as
1,
1
, , ,
K
K i i j
j
L G M where G diag g g g M
, (6)
where the nearest neighbor graph is weighted by M as
2
,
exp /
0
i j j i
ij
d d if d kNN d
M
else
, (7)
which is defined by the Gaussian function, where
is the kernel
width,
i
kNN d
is the k-nearest neighbor set of atom
i
d
, G is a
diagonal matrix and
,ij
M
encodes the similarity between atoms
i
d
and
j
d
. By the above definitions,
2
2
T
X DS Tr S LS
can encode the reconstruction error with the locality constraint,
where
T
Tr S LS
inherits the manifold structure of training set.
2
2
T
X DV Tr V UV
encodes the reconstruction error with
label embedding, where
T
Tr V UV
forces the intra-class atoms
in D to have similar profiles. Term
2
2
SV
is a regularization
over the coefficients, which ensures the mutual transformation
between the label embedding and locality constraint.
III. ROBUST FLEXIBLE DISCRIMINATIVE DICTIONARY
LEARNING (RFDDL)
A. Problem Formulation
The presented RFDDL model improves the robust properties of
discriminative DL to noise and sparse errors in twofold. First, it
calculates the robust discriminative dictionary and codes in the
recovered clean data and atom subspaces. Specifically, RFDDL
decomposes the original X and dictionary D in each iteration to
recover underlying clean data
new
X
and clean dictionary
new
D
,
and models the errors
E
and
D
E
at the same time in terms of
new
X X E
and
new D
D D E
, where L
2,1
-norm is used on
E
and
D
E
so that the sparse errors in data and atoms can be
corrected jointly. Then, RFDDL performs the discriminant DL
over clean
new
X
and
new
D
for accurate representations. Second,
our RFDDL encodes the locality and defines a Laplacian matrix
by computing discrimination-promoting reconstruction weights
over recovered clean atoms, which can encourage intra-class
samples to have similar sparse codes and inter-class samples to
have different ones, and will be detailed in next subsection. By
combing the joint subspace recovery and Laplacian regularized
reconstruction error, discriminative sparse-code error and data
classification error, the initial problem of RFDDL is given as
2
T
2,1 2,1
, , , , ,
2 2 2
2,1
min
+
. . ,
D
new new D
F
D S L E E W
e
F F F
new new D
X D LS E E
Q LS SH H WLS W
s t X X E D D E
, (8)
where
T
2,1 2,1
D
EE
is the L
2,1
-norm based error,
,
and
are the parameters. Note that the L
2,1
-norm can ensure the
regularized matrix to be sparse in rows, and the L
2,1
-norm based
metric is robust to noise and outliners in data and atoms [2], [7],
[27]. L
2,1
-norm based classifier
2,1
W
can force the columns of
W to be sparse so that the discriminative soft labels can be
predicted in the latent sparse subspace. Q and H are similarly
defined as LC-KSVD, and
2
e
F
SH
is the Frobenius-norm based
coefficients,
T
/
e
H I ee N
is the “centering matrix”, that is,
T
/
e
SH S See N
can be considered as the normalized coding
coefficients. It is clear that the discriminative Laplacian matrix
is associated with the learning of codes and classifier, which
can potentially obtain the more accurate codes, discriminative
dictionary and powerful discriminative classifier jointly.
Please note that the linear reconstruction
2
new new
F
X D LS
may be overfitted especially when the number of training data
is limited. To enable the data sampled from nonlinear manifold
to be handled potentially by DL and avoid the overfitting, our
RFDDL proposes a flexible reconstruction residue motivated
剩余14页未读，继续阅读
102 浏览量
274 浏览量
2025-01-06 上传
2025-01-06 上传
weixin_38615591
- 粉丝: 5
- 资源: 976
我的内容管理
展开
最新资源
- pip-chill:更轻松的“点冻结”
- 实存帐存对比表DOC
- jquery.page分页控件.zip
- sql-q:JDBC 模板
- 数据结构
- ange-button
- stable-baselines:稳定基线的镜子
- 电子功用-太阳能电池板激光刻划系统及刻划方法
- 材料调拨管理方法DOC
- ut-ussd
- NewRepo:创建一个空白仓库
- galgebra:SymPy的符号几何AlgebraCalculus软件包
- 在 C# 中使用 MATLAB 结构体和 Builder NE：“MATLAB 艺术”帖子的代码 - 展示了如何在 MATLAB 和 C# 之间传递结构体。-matlab开发
- mysql-8.0.18-winx64.zip
- js特效脚本含源码和说明迅雷网七屏flash广告轮换
- 电子功用-带有市电互补功能的太阳能模块化嵌入式控制器
