平滑递归神经网络(SRNN)在压缩传感中的非光滑非凸优化应用

需积分: 13 47 浏览量更新于2024-07-15 收藏 940KB PDF 举报

"这篇研究论文深入探讨了一种基于KKT条件的平滑递归神经网络（SRNN）在压缩传感中的应用，用于处理非光滑非凸优化问题。论文提出将非凸非光滑近似模型应用于L0范数最小化模型，以解决由压缩感测产生的问题。通过数学工具分析了网络解的存在性、唯一性和极限行为，并对比了多种非凸逼近函数的效果，发现变换后的L1函数（TL1）、对数函数（Log）和反正切惩罚函数在稀疏信号恢复中表现出色。SRNN-TL1在处理传感器矩阵的相干性问题上表现稳定且具有竞争力，优于一些现有的数值算法和神经网络方法。" 在这篇研究中，作者Dan Wang和Zhuhong Zhang首先引入了KKT条件，这是一种在优化问题中用来确定局部最优解的必要条件。KKT条件结合了拉格朗日乘子法和梯度相等性，是解决约束优化问题的关键工具。在非光滑非凸优化的背景下，KKT条件的等效形式被用于指导SRNN的设计。 SRNN是一种特殊类型的递归神经网络，其特点是能够处理复杂的序列数据和非线性关系。在压缩传感（Compressed Sensing, CS）领域，目标通常是找到一个稀疏信号的最优化表示，即L0范数最小化问题。L0范数衡量的是非零元素的数量，而L1范数是它的平滑近似，更容易求解。论文中，作者通过非凸非光滑近似模型来替代原始的L0范数问题，使得问题可以通过SRNN进行有效的处理。为了评估不同非凸逼近函数的性能，作者进行了多种函数的比较实验，包括TL1、Log和反正切函数。这些函数都是为了在保持稀疏性的前提下，提供更易于优化的损失函数。实验结果显示，TL1函数在稀疏信号恢复方面表现出色，尤其是在处理不同类型的感测矩阵时，其鲁棒性和抗相干性得到了验证。此外，论文还讨论了SRNN的解的存在性、唯一性和极限行为，这些都是通过一系列数学工具如连续性和微分理论进行分析的。这些性质对于理解SRNN如何在非光滑非凸优化问题中收敛至合理解至关重要。这项研究为压缩传感的非光滑非凸优化提供了一个新的解决方案，即基于KKT条件的平滑递归神经网络。通过理论分析和实证比较，SRNN-TL1方法在解决稀疏信号恢复问题上展现出优越的性能，特别是在处理高相干性传感矩阵时。这一方法不仅为压缩传感带来了新的视角，也为神经网络在其他非凸优化问题的应用提供了有价值的参考。

3 Preliminaries

This section formulates some basic deﬁnitions and prop-

erties which will be used in the subsequent sections. It

involves mainly the restricted isometry property (RIP)

[2, 46], coherence [40], sparse approximation functions

[47], nonsmooth analysis [48] and smoothing functions

[23, 26, 37].

For two given vectors x; y 2 R

, let hx; yi represent the

inner product between x and y, and x

stand for the L

norm of x with 0\p\1. Matrix A 2 R

mn

is said to

satisfy the S-restricted isometry property (S-RIP) [2, 46]if

ð1  d

Þ x

 Ax

ð1 þ d

Þ x

for all S-sparse signals x 2 R

, where d

is the smallest

restricted isometry constant (RIC) for which the above

inequality holds with d

2ð0; 1Þ . Candes and Tao [46]

claimed that sensing matrices with small RIC, e.g., random

Gaussian and Bernoulli matrices, were suitable for sparse

signals recovery. However, it is NP-hard to check whether

a ﬁxed matrix satisﬁes the RIP or not [49]. Fortunately, the

version of coherence or mutual coherence [40] is closely

related to one such RIP and easy to examine, in which the

coherence degree of A, i.e., sðAÞ, is employed to measure

the similarity between the matrix columns A

and A

namely

sðAÞ¼max

i6¼j





;

ð2Þ

with A ¼½A

; :::; A

2R

mn

. Note that if sðAÞ¼1, there

exists i and j such that A

and A

are linearly dependent. In

particular, a RIP matrix has a small coherence degree,

while a highly coherent matrix yields a large RIC.

In order to achieve sparse signal recovery for a com-

pressed sensing problem as described by the L

-model,

deﬁne a step function s : R ! R given by

sðhÞ¼

1; if h 6¼ 0

0; if h ¼ 0:



ð3Þ

Thereby, x

i¼1

sðx

Þ. Further, since sðÞ is discon-

tinuous, it is replaced by a given continuous approximation

function F

ðÞ satisfying the following three conditions

[47]:

(1) F

ð0Þ¼0, F

ðÞ is an even function and is not

identically equal to zero;

(2) F

ðÞ is nondecreasing and concave on ½0; þ1Þ;

(3)

ðhÞ

is nonincreasing in h on ð0; þ1Þ.

Here, a is a parameter which controls the closeness of

approximation, and accordingly, x

can be replaced by a

function JðxÞ deﬁned by

JðxÞ¼

i¼1

ðx

Þ:

ð4Þ

We next introduce the versions of Clarke generalized

gradient [48] and smoothing function. To this point, let

f : R

! R be a locally Lipschitz function and D

denote

the set of points at which f is differentiable. Hence, the

Clarke generalized gradient of f at x can be deﬁned below,

of ðxÞ¼co lim

!x;u

rf ðu



ð5Þ

where ‘co’ means the convex hull. Note that if f is a locally

Lipschitz function with rank L near x , of ðxÞ is a nonempty,

convex and compact subset of R

, satisfying n

L with

8n 2 of ðxÞ. For a given minimization problem with

objective function f, we say that x



is a Clarke stationary

point in set X if there exists n



2 of ðx



Þ such that

x  x



; n



i0h for 8x 2 X.IfX is nonempty, closed and

convex in R

, the normal cone to X at x 2 X is deﬁned by

ðxÞ¼cl

[

c 0

cod

ðxÞ



v 2 R

jhy  x; vi0 ; 8y 2 X



;

ð6Þ

where ‘cl’ denotes the closure hull and d

is a globally

Lipschitz function with d

ðxÞ¼minf u  x

; u 2 Xg.

On the other hand, smoothing approximation is a usual way

to handle nonsmooth optimization problems, for which

some parameterized smooth functions can be adopted to

approximate their nonsmooth functions.

Deﬁnition 1 [23, 26, 37] Let h : R

! R be a locally

Lipschitz function, We say that

h : R

½0; þ1Þ ! R is a

smoothing function of h,if

h satisﬁes the following four

conditions:

(1)

hð:; lÞ is continuously differentiable on ½0; þ1Þ;

(2) for any ﬁxed x 2 R

, lim

l!0

hðx; lÞ¼hðxÞ;

(3) there exists a positive constant j

such that

hðx; lÞ



j

with 8l 2½0; 1Þ and x 2 R

;

(4) lim

z!x;l!0

hðz; lÞ



 ohðxÞ.

From the version of smoothing function above, we

observe that any locally Lipschitz function hðxÞ has a

smoothing approximation function

hðx; lÞ satisfying

lim

z!x;l!0

hðz; lÞ¼hðxÞ and

jhðxÞ

hðx; lÞjj

l; 8l 2½0; 1Þ; x 2 R

ð7Þ

2908 Neural Comput & Applic (2019) 31:2905–2920

123

剩余15页未读，继续阅读

weixin_38516040

粉丝: 3
资源: 918

平滑递归神经网络(SRNN)在压缩传感中的非光滑非凸优化应用

凸优化入门：凸集、凸函数与KKT条件

凸优化理论：等式约束与KKT条件

Yalmip实现双层优化求解及KKT条件应用

最优化 kkt条件

最优化与KKT条件

最优化和KKT条件

最优化与kkt条件

svm.rar_KKT SVM_KKT条件_SVM 的KKT条件_kkt_kkt conditions

基于FCM与KKT条件的增量学习方法

KKT_KKT条件_kkt_KKT方法_

最新资源