2.2. ARIP in compressive sensing
The restricted isometric property (RIP) [29,30,40,58,64] has
become a powerful tool for convergence analysis in L0 methods,
i.e., ROMP [42], CoSaMP [43,44], Subspace Pursuit [45], SAMP [56]
and iterative hard thresholding (IHT) [15–17].
Definition 2.1 (RIP). Matrix AA
R
mn
is said to satisfy the
restricted isometry property (RIP) with the restricted isometry
constant
d
s
if
d
s
is the smallest constant satisfying
ð1
d
s
Þ:x:
2
2
r :Ax:
2
2
r ð1 þ
d
s
Þ:x:
2
2
ð2:1Þ
for any xA
R
n
with :x:
0
r s. Notice 2s o m o n and
d
s
A ½0, 1Þ.
Multiplying (2.1) with c4 0 leads to (2.2):
c
2
ð1
d
s
Þ:x:
2
2
r :cAx:
2
2
r c
2
ð1þ
d
s
Þ:x:
2
2
ð2:2Þ
If scaling factor c a 1, i.e., c ¼20, then c
2
ð1
d
s
Þ and c
2
ð1þ
d
s
Þ
are no longer symmetrical to 1, namely RIP in (2.1) does not apply
to scaling matrix cA if ca 1. To tackle this scaling problem, the
Asymmetrical RIP [16] is introduced as following:
Definition 2.2 (ARIP). Matrix AA
R
mn
is said to satisfy the
asymmetrical RIP (ARIP) with s order restricted isometry constant
l
s
and L
s
, which denote the biggest and smallest constants,
respectively, such that
l
s
:x:
2
r :Ax:
2
r L
s
:x:
2
ð2:3Þ
holds for any xA
R
n
with :x:
0
r s.
Note that (2.4) holds
l
2s
r l
s
r L
s
r L
2s
ð2:4Þ
Proof. For x A
R
n
with :x:
0
r s , (2.3) holds. Since :x:
0
r sr 2s,
according to Definition 2.2, there is
l
2s
:x:
2
r :Ax:
2
r L
2s
:x:
2
ð2:5Þ
Since l
s
and L
s
denote the biggest and smallest constants in
(2.3), respectively, there are L
2s
r l
s
and L
s
r L
2s
, together with
l
s
r L
s
finishing the proof of (2.4). &
Note that l
2s
and L
2s
are the biggest and smallest constants,
respectively, satisfying
l
2s
:uv:
2
r :AðuvÞ:
2
r L
2s
:uv:
2
ð2:6Þ
for any u, vA
R
n
with :u:
0
r s and :v:
0
r s. Obviously, Asymme-
trical RIP is the generalization of symmetrical RIP.
2.3. ARIP in matrix rank minimization
Definition 2.3 (RIP). Linear operator A :
R
mn
-
R
p
is said to
satisfy the restricted isometry property (RIP) with the restricted
isometry constant
d
s
if
d
s
is the smallest constant satisfying
ð1
d
s
Þ:X:
2
F
r :AðXÞ:
2
2
r ð1 þ
d
s
Þ:X:
2
F
ð2:7Þ
for all X A
R
mn
with rankðXÞr s.
RIP in (2.7) holds for isometry random map, i.e., AðXÞ¼AVecðXÞ
and AA
R
pmn
is a Gaussian random matrix with entries
Aði, jÞNð0, 1=pÞ. However matrix A is also sensitive to rescaling.
Moreover, linear operator A in low rank matrix completion
does not satisfy symmetrical RIP in (2.7). These two situations
induce introduction of Non-symmetrical RIP for matrix rank
minimization.
Definition 2.4 (ARIP). The linear operator A :
R
mn
-
R
p
is said to
satisfy asymmetrical restricted isometry property(ARIP) with the
restricted isometry constants l
2s
and L
2s
, which are the biggest
and smallest constants, respectively, satisfying
l
2s
:UV:
2
F
r :AðUVÞ:
2
2
r L
2s
:UV:
2
F
ð2:8Þ
for all U, V A
R
mN
with rankðUÞr s, rankðVÞr s. Like :U:
0
, rank
function is also subadditive, and by Definition 2.4, the inequality
l
2s
r l
s
r L
s
r L
2s
in (2.4) also holds.
2.4. Barzilai–Borwein(BB) step size
Newton-iteration is
x
k þ1
¼x
k
ðH
k
Þ
1
g
k
ð2:9Þ
where Hessian matrix H
k
¼
r
2
f ðx
k
Þ and g
k
¼
r
f ðx
k
Þ. Now approx-
imate Hessian matrix H
k
by H
k
a
k
I with
a
k
4 0 and substitute
into (2.9), there is
x
k þ1
¼x
k
ð1=
a
k
Þg
k
ð2:10Þ
Substitute H
k
a
k
I into secant equation
D
g
k
¼H
k
D
x
k
where
D
g
k
¼g
k
g
k1
and
D
x
k
¼x
k
x
k1
; there is
D
g
k
¼
a
k
D
x
k
. Denote
inner product with /S and let
a
k
¼
a
bb1
ðkÞ¼argmin
a
:
D
g
k
aD
x
k
:
2
2
¼
/
D
x
k
,
D
g
k
S
/
D
x
k
,
D
x
k
S
ð2:11Þ
For f ðx
k
Þ¼0:5:Ax
k
y:
2
, g
k
¼
r
f ðx
k
Þ¼A
T
ðAx
k
yÞ, there is
a
k
¼:Aðx
k
x
k1
Þ:
2
=:x
k
x
k1
:
2
ð2:12Þ
which serves as initial value for
a
k
, and 1=
a
k
is the initial step size
for k iteration. In order to guarantee the monotone decrease in
f ðx
k
Þ, Back tracking line search is used to reduce the step size 1=
a
k
by increasing
a
k
with
a
k
¼1:1
a
k
, which is the statement (3) of
Algorithm 1.
Substitute k with kþ1, Eq. (2.12) becomes
a
k þ1
¼
a
bb1
¼:Aðx
k þ1
x
k
Þ:
2
=:x
k þ1
x
k
:
2
ð2:13Þ
which serves as initial value for
a
k þ1
, and 1=
a
k þ1
is the initial
step size in kþ1 iteration.
Note that 1=
a
bb1
denotes the first type BB step size, which is
larger than the second type BB step size 1=
a
bb2
as will be proved
by Proposition 9.
3. Monotone SCIHTBB
3.1. Sparsity constrianed QP(SCQP)
Finding x
s
A
R
n
such that Ax
s
¼yA
R
m
and :x
s
:
0
r s are
equivalent to solving
x
s
¼argmin
x
f ðxÞ¼0:5:Axy:
2
, s:t::x:
0
r s ð3:1Þ
Formulation (3.1) is named as sparsity constrained quadratic
program (SCQP) in this paper.
Note that we confine the consideration to the case in which
AA
R
mn
is a random matrix, y A
R
m
and 2somon. In the above
cases, sparse solution x
s
is unique. Its proof is given by Lemma
3.1 in [65].
Gradient of f ðxÞ at iteration x
k
has the form of
g
k
¼
r
f ðx
k
Þ¼A
T
ðAx
k
yÞð3:2Þ
Z. Xie, S. Chen / Neurocomputing 74 (2011) 3663–3676 3665