求解广义瑞利商极值问题：数学与实践应用
需积分: 28 19 浏览量
更新于2024-07-16
收藏 2.91MB PDF 举报
"这篇论文探讨了在单位球面上最大化广义瑞利商的问题，即针对对称矩阵B、D和正定对称矩阵W，如何找到局部最优和全局最优解。这个问题不仅在数学理论上有其独立的研究价值，而且在实际应用中也有重要体现，如稀疏 Fisher 分割分析。作者首先通过研究优化条件来刻画局部最大值和全局最大值，揭示了解决全局最优解的关键在于一个特殊的非线性特征值问题。在特殊情况下，如果D=μW（μ>0），全局最优解集合实质上是特定定义矩阵的最大特征值对应的特征空间。这种对全局解的特性描述为寻找最大值问题提供了启示，并且提出了一个用于任何单调收敛迭代的全局最大值搜索策略。第二部分，文章实现了一种基于Riemannian信任域方法的实际算法，该算法具有良好的收敛性质：全局收敛性和局部超线性收敛性。初步的数值测试展示了算法的性能并给出了经验评估。"
这篇论文的核心知识点包括：
1. **广义瑞利商**：是线性代数中的一个重要概念，它扩展了标准的瑞利商，通常用于描述向量在不同线性变换下的相对大小。在本研究中，涉及到两个广义瑞利商的和，即x^T D x 和 x^T B x / x^T W x。
2. **单位球面**：优化问题的约束条件，所有解的范数（欧几里得长度）必须等于1。
3. **优化条件**：通过分析梯度和Hessian矩阵，确定局部极值点的性质，这是解决优化问题的关键步骤。
4. **非线性特征值问题**：在寻找全局最优解时，与之密切相关的特殊问题，对于特定矩阵的特征值分析至关重要。
5. **特殊案例分析**：当D=μW（μ>0）时，全局最优解可以通过最大特征值的计算得到，这简化了问题的求解。
6. **Riemannian信任域方法**：这是一种在流形上进行优化的算法，具有全局收敛性和局部超线性收敛性的优点，可以有效地解决广义瑞利商的极值问题。
7. **算法实现与数值测试**：将理论方法转化为实际算法，并通过数值实验验证了算法的性能和收敛性，提供了实证结果支持。
8. **应用背景**：问题来源于稀疏 Fisher 分割分析，这是一种机器学习技术，用于数据分类和降维，表明该优化问题有实际应用价值。
116 L.-H. Zhang
where p
i
(·) for i =1, 2,...,n, are given nonnegative penalty functions and λ
i
> 0
are the tuning parameters controlling the strength of the incentive for classification on
less dimensions. The penalty functions p
i
(·) and the parameters λ
i
are not necessarily
the same for all i. For ease of presentation, however, we can assume that they are
identical for all i, and then we denote p
i
(·)λ
i
=p
λ
(·). A favorable choice of p
λ
(·) is
the L
1
penalty: p
λ
(|x
i
|) = λ|x
i
| (see e.g., [29, 37]). Another popular one is the so-
called smoothly clipped absolute deviation penalty (SCAD) proposed in [14], which
is proved to be very effective in parametric models such as generalized linear models
and robust regression models [14, 25]. To solve (2.3), Fan and Li [14] propose a local
quadratic approximation for the penalty function p
λ
(·): Suppose x
(0)
is an initial
point. If x
(0)
i
is very close to zero, then set x
(1)
i
= 0; otherwise, the penalty function
p
λ
(|x
i
|) is locally approximated by a quadratic function, i.e.,
p
λ
(|x
i
|) ≈p
λ
|x
(0)
i
|
+
x
2
i
−(x
(0)
i
)
2
2|x
(0)
i
|
p
λ
|x
(0)
i
|
+
,
where p
λ
(|x
(0)
i
|
+
) denotes the derivative of p
λ
(·) at |x
(0)
i
| from above. As a result, a
local approximation for (2.3)is
max
x
2
=1
x
S
b
x
x
S
w
x
−x
Σ
0
x +c
0
, (2.4)
where Σ
0
:=diag{
p
λ
(|x
(0)
1
|
+
)
2|x
(0)
1
|
,...,
p
λ
(|x
(0)
n
|
+
)
2|x
(0)
n
|
}, and c
0
is a constant. The resulting prob-
lem (2.4) consequently is an application of (1.1). For the discussion on the conver-
gence of such approximation as well as more variable selection techniques, please
refer to [12, 14, 19, 25, 29, 37].
3 Characterization of the solution
Returning to the problem (1.1), in this section, we attempt to shed some lights on the
solution of (1.1) by describing the optimality conditions. Let f
|
M
(x) : M → R be
the restriction of the cost function f(x), and we will provide the local optimality con-
ditions in Sect. 3.1 and present some global optimality conditions in Sect. 3.2. These
optimality conditions not only characterize the solutions of (1.1) to some extent, but
also motivate a starting point strategy in Sect. 3.3 for certain iterative algorithm.
3.1 Optimality conditions for the local solution
Viewing the unit sphere M as a Riemannian submanifold of the Euclidean space R
n
endowed with the natural inner product, the tangent space T
x
M at any point x ∈M
is given by [5]
T
x
M =
z|z =P
x
y, ∀y ∈R
n
,
剩余28页未读，继续阅读
点击了解资源详情
点击了解资源详情
点击了解资源详情
2020-04-25 上传
2021-06-01 上传
2022-08-03 上传
2023-06-08 上传
2023-06-09 上传
2023-06-09 上传
