递归最小二乘在分析字典学习中的新算法

195 浏览量更新于2024-08-26 收藏 580KB PDF 举报

"本文介绍了一种基于递归最小二乘(Recursive Least Squares, RLS)的分析字典学习算法，旨在解决在稀疏表示中分析模型下的字典学习问题，尤其是在存在噪声观测的情况下。文章由来自南昌大学电子与信息工程学院的Ye Zhang、Haolong Wang以及英国萨里大学电子工程系的Wenwu Wang共同撰写。他们提出的新算法通过RLS直接从噪声测量数据中估计字典，以克服传统分析字典学习(Analysis Dictionary Learning, ADL)算法如Analysis K-SVD在迭代更新字典原子时因预估源信号慢速收敛的局限性。为了改进新算法的收敛特性，初始字典是通过使用K平面聚类算法从小规模训练集中估计得出的。" 在稀疏表示领域，字典学习是核心问题之一，它涉及到寻找一组基（字典），使得信号可以通过这些基的线性组合以尽可能稀疏的形式表达。分析模型与合成模型是两种主要的字典学习框架。在分析模型中，信号不是通过原子的直接叠加来表示，而是通过原子的线性组合与一个分析矩阵相乘来表示。这种方法在图像处理、信号处理和机器学习等领域有广泛应用。传统ADL算法如Analysis K-SVD，在处理噪声观测时，需要先预估源信号，再更新字典，这一过程可能导致收敛速度缓慢。 Ye Zhang等人提出的RLS-based ADL算法则直接利用RLS方法从噪声观测中估计字典，RLS是一种在线优化算法，能够快速适应系统变化，具有良好的收敛性和鲁棒性，因此可以提高整个字典学习过程的效率。为了进一步优化算法性能，他们在开始阶段使用K-plane聚类算法对有限的训练样本进行处理，估计初始字典。K-plane聚类是一种有效的数据聚类方法，它可以找到数据的多维结构，为字典提供合适的初始化，从而加速后续的RLS迭代过程。这篇研究论文提出的新方法在处理噪声环境下的字典学习问题时，通过结合RLS算法的快速收敛优势和K-plane聚类的初始化能力，有望实现更快更稳定的字典学习过程，对于提升稀疏表示的性能和效率具有重要意义。

AN ANALYSIS DICTIONARY LEARNING ALGORITHM BASED ON RECURSIVE LEAST

SQUARES

Ye Zhang

, Haolong Wang

, Wenwu Wang

Department of Electronic and Information Engineering, Nanchang University, Nanchang, China

Department of Electronic Engineering, University of Surrey, Guildford, United Kingdom

Emails: zhangye@ncu.edu.cn, hlwangncu@163.com, w.wang@surrey.ac.uk

ABSTRACT

We consider the dictionary learning problem in sparse rep-

resentations based on an analysis model with noisy observa-

tions. A typical limitation associated with several existing

analysis dictionary learning (ADL) algorithms, such as Anal-

ysis K-SVD, is their slow convergence due to the procedure

used to pre-estimate the source signal from the noisy mea-

surements when updating the dictionary atoms in each itera-

tion. In this paper, we propose a new ADL algorithm where

the recursive least squares (RLS) algorithm is used to estimate

the dictionary directly from the noisy measurements. To im-

prove the convergence properties of the proposed algorithm,

the initial dictionary is estimated from a small training set by

using the K-plane clustering algorithm. The proposed algo-

rithm, as shown by experiments, offers advantages over the

Analysis K-SVD, in both the runtime and atom recovery rate.

Index Terms— Recursive least squares; analysis dictio-

nary learning; sparse representation

1. INTRODUCTION

In the past decade spare representations based on the synthe-

sis model have been studied extensively. Consider a signal

x ∈ R

, the sparse representations of x over a dictionary

D can be described as x = Da, where D ∈ R

M×N

is a

possibly overcomplete dictionary (N ≥ M ), and a ∈ R

containing the coding coefﬁcients, is assumed to be sparse,

i.e. ∥a∥

= n ≪ N , where the ℓ

quasi-norm ∥·∥

counts

the number of nonzero components in its argument. In oth-

er words, the signals are represented as a linear combination

of a small number of signal components (i.e. atoms) chosen

from the dictionary. A great deal of effort has been dedicat-

ed to the problem of learning the dictionary D from signal

examples [1–9] based on the synthesis model.

There is a dual viewpoint to sparse representations based

on e.g. the analysis model, where an analysis dictionary or

operator Ω ∈ R

P ×M

(P ≥ M) is sought to transform x ∈

to a high dimensional space, i.e. Ωx = z. The coefﬁcient

vector z ∈ R

is called the analysis representation of x and

assumed to be sparse. In this model, the rows of Ω that are

associated with zero entries in z deﬁne a subspace that the sig-

nal x is orthogonal to, as opposed to the few non-zero entries

of a in the synthesis model. In general, Ω can be learned from

the observed signals Y = [y

... y

] ∈ R

M×K

measured

in the presence of additive noise, i.e. Y = X + V, where

X = [x

... x

] ∈ R

M×K

contains the original signals,

V = [v

... v

] ∈ R

M×K

is noise and K is the number of

signals. Compared to the synthesis counterpart, there are on-

ly a few algorithms proposed recently for analysis dictionary

learning (ADL) [10–13].

In [10], the Analysis K-SVD algorithm was proposed for

dictionary learning. By keeping Ω ﬁxed, the optimal back-

ward greedy (OBG) algorithm has been employed to estimate

the sub-matrix of Ω whose rows are orthogonal to

X, which

is the estimation of X from Y. A data set, i.e., the sub-matrix

of Y, can be obtained, with its columns corresponding to that

X. The smallest singular values of the sub-matrix are then

used to update Ω. The algorithm is effective, however, the

original signal X should be estimated by using the greedy al-

gorithm with heavy computation. In [11], a subset pursuit

algorithm has been proposed for the ADL. The algorithm is

based on the two-phase approach. In the ﬁrst phase, the al-

gorithm exploits the analysis representation of Y to obtain

the subset Y

, rather than using

to determine Y

, where

is the subset of Y that corresponds to the j-th row of Ω,

which is orthogonal to Y

. In the second phase, the j-th row

of the analysis dictionary can be updated by the eigenvector

associated with the smallest eigenvalue of the Gram matrix

of Y

. In [12], a sequential minimal eigenvalue algorithm

for the ADL was proposed by considering the orthogonality

between the rows of Ω and a sub-set of training signals X.

Once the sub-set is found, the corresponding row of the dic-

tionary can be updated with the eigenvector associated with

the smallest eigenvalue of the autocorrelation matrix of these

signals. However, as the number of the rows in Ω increases,

so does the computational cost of the method. In [13], a pro-

jected subgradient algorithm was proposed for analysis oper-

ator learning, where an ℓ

-norm penalty function is applied

to Ωx, and a uniformly normalized tight frame is employed

ICSP2014 Proceedings

831

下载后可阅读完整内容，剩余4页未读，立即下载

weixin_38618819

粉丝: 4
资源: 894

递归最小二乘在分析字典学习中的新算法

块递归最小二乘字典学习算法

核递归最小二乘算法的Julia实现

用于分析字典学习的递归最小二乘算法

基于块的递归最小二乘字典学习算法提升信号表示性能

KernelRecursiveLeastSquares.jl:内核递归最小二乘算法的 Julia 实现

船舶的递归最小二乘辨识建模

基于字典学习的超宽带信号稀疏表示与降噪方法

基于特征化字典的低秩表示人脸识别.pdf

基于稀疏表示的目标跟踪mei xue

基于迭代残差网络的双能CT图像材料分解研究.docx

最新资源