联合矩阵图形Lasso：多稀疏矩阵高斯图形模型学习
167 浏览量
更新于2024-07-15
收藏 2.39MB PDF 举报
"本文主要探讨了联合学习多个稀疏矩阵高斯图形模型的方法，并提出了一种称为联合矩阵图形套索（Joint Matrix Graphical Lasso）的新型算法。该方法旨在发现不同条件下矩阵变量中行与列之间的条件独立性结构。通过借用不同图形模型之间的强度，该方法基于最大似然性原则并施加了惩罚行和列精度矩阵的约束。与传统的联合矢量图形模型相比，此模型具有更高的简洁性和灵活性，并且在一致性和稀疏性方面有坚实的渐近性质。模拟实验和真实数据集分析证实了该方法在识别图形结构和估计精度矩阵方面的优越性能。"
在高斯图形模型中，矩阵正常分布被广泛用于处理具有结构信息的多维数据。在这种模型中，每个节点代表矩阵的一行或一列，边的存在与否表示对应行或列之间的条件独立性。当矩阵稀疏时，这些模型可以帮助我们理解和提取数据的主要结构，特别是在大型高维数据集中。
联合学习多个稀疏矩阵高斯图形模型是一种有效的方法，可以捕捉不同条件或环境下的复杂依赖关系。通过联合学习，可以发现共有的模式和特定条件下的差异，这对于数据分析和预测任务非常有用。联合矩阵图形套索引入了新的优化策略，它同时考虑了行和列的精度矩阵，这不仅增强了模型的表达能力，也提高了学习的效率。
文章中提到的“一致性”（Consistency）是指在样本数量趋于无穷大时，模型参数的估计值能收敛到真值的特性。而“稀疏性”（Sparsistency）则是指在保证模型性能的同时，能够得到尽可能稀疏的解，即尽可能少的非零元素。这种特性对于理解和解释模型至关重要，因为过于复杂的模型可能难以解释和应用。
在模型的渐近分析中，作者指出联合矩阵图形套索相比于联合矢量图形模型拥有更快的收敛速度。这意味着在相同数据规模下，新方法能够更快地达到稳定状态，从而提高学习效率和准确性。
通过大量的仿真实验，作者验证了所提方法在识别图形结构上的优势，能够准确地发现条件独立性关系。同时，精度矩阵的估计也表现出优秀的性能，这对于进一步的统计推断和预测任务非常重要。此外，实际数据集的分析进一步证明了联合矩阵图形套索在处理真实世界问题时的有效性。
这篇文章提出了一种新颖的联合学习框架，适用于稀疏矩阵高斯图形模型，其在理论和实践上都显示出了强大的性能。这种方法不仅有助于深入理解复杂数据结构，还为数据分析和机器学习领域提供了新的工具。
2608 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL. 26, NO. 11, NOVEMBER 2015
Danaher et al. [25] proposed the fused graphical
Lasso (FGL) by applying the fused Lasso penalty [30]
P(
1
,...,
K
) =
k<k
i=j
k
ij
−
k
ij
to (1), which encourages K precision matrices to have identical
element values. In addition, they also propose the group graph-
ical Lasso (GGL) by applying the group Lasso penalty [31]
P(
1
,...,
K
) =
i=j
K
k=1
k
ij
2
to (1), which encourages K precision matrices to have a
common pattern of sparsity.
B. Node-Based Joint Graphical Lasso
First, we review the RCON given in [26].
Definition 1: The RCON induced by a matrix norm ·is
defined as
(
1
,
2
,...,
N
) = min
V
1
,...,V
N
⎡
⎢
⎢
⎢
⎣
V
1
V
2
.
.
.
V
N
⎤
⎥
⎥
⎥
⎦
s.t.
n
= V
n
+ (V
n
)
T
for n = 1, 2,...,N.
Indeed, (·) is a norm for all matrix norms ·. Thus,
it is convex. In this paper, we consider only a particular class
of RCON, where · is an
1
/
r
norm, given by
V =
p
j=1
V
j
r
, with V =[V
1
, V
2
,...,V
p
] and
1 ≤ r ≤∞. In the following, (·) is denoted by
r
(·).
Based on RCON, the perturbed-node joint graphical
Lasso (PNJGL) [26] is proposed to detect the perturbed nodes
2
among multiple networks using the structure penalty
P(
1
,...,
K
) =
k<k
r
(
k
−
k
).
When r = 2orr =∞, PNJGL encourages the precision
matrices {
k
}
K
k=1
to be a union of a few rows and the
corresponding columns, which can be interpreted as a set of
perturbed nodes across K networks.
Moreover, the common hub (cohub) node joint graphical
Lasso (CNJGL) [26] is proposed to detect the cohub nodes
3
among multiple networks using the structure penalty
P(
1
,...,
K
) =
r
(
1
− diag(
1
),...,
K
− diag(
K
)).
CNJGL encourages the supports of {
k
}
K
k=1
to be the same,
and the union of a set of rows and columns among K precision
matrices, which can be interpreted as a set of common hub
nodes among K networks.
2
The perturbed nodes have a completely different connectivity pattern to
other nodes in the multiple networks [26].
3
The cohub nodes serve as hubs in each of the multiple networks [26].
III. JOINT MATRIX GRAPHICAL MODELS
In this section, we propose the joint matrix graphical
Lasso for learning multiple MGGMs sharing the same matrix
variable under distinct conditions. According to different appli-
cations, we propose the edge-based and the node-based joint
matrix graphical Lasso, respectively.
A. Problem Formulation
Suppose that a random matrix Y ∈ R
p×q
follows the matrix
normal distribution MN
p,q
(M,,)whose density function
is defined in terms of
P(Y |M,,) =
1
(2π)
pq/2
||
q/2
||
p/2
·exp
−
1
2
tr((Y − M)
T
−1
(Y − M)
−1
)
where M ∈ R
p×q
is mean matrix; ∈ R
p×p
and ∈ R
q×q
are row and column covariance matrices, respectively. The row
precision matrix
−1
encodes the conditional independence
among rows in the matrix variable, while the column precision
matrix
−1
encodes that among columns.
Further suppose that Y
k
i
∈ R
p×q
(i = 1, 2,...,n
k
) are sam-
pled i.i.d. from MN
p,q
(O
p×q
,
k
,
k
) for k = 1, 2,...,K
with K ≥ 2. Here n
k
is the number of samples in the kth class,
and the features are shared among K classes. For convenience,
let A
k
= (
k
)
−1
and B
k
= (
k
)
−1
(k = 1, 2,...,K ),and
further let {A}={A
1
,...,A
K
} and {B}={B
1
,...,B
K
}.
Then the negative log likelihood for the data takes the
form of
L({A}, {B}) =
K
k=1
1
n
k
pq
n
k
l=1
tr
A
k
Y
k
l
B
k
Y
k
l
T
−
1
p
log|A
k
|−
1
q
log|B
k
|
.
Meanwhile, we can also consider the weighted negative
log likelihood. Clearly, minimizing L({A}, {B}) leads to the
maximum likelihood estimates (MLEs). However, the MLEs
are usually dense. The
1
-regularization has been employed to
induce sparsity, resulting in sparse precision estimation.
In this paper, we propose the joint matrix graphical Lasso
for estimating multiple MGGMs by minimizing the penalized
negative log likelihood
min
{A
k
∈S
p
++
,B
k
∈S
q
++
}
K
k=1
L({A}, {B}) + λ
1
K
k=1
A
k
1
+ρ
1
K
k=1
B
k
1
+λ
2
P
1
({A}) + ρ
2
P
2
({B})
(2)
where λ
1
, λ
2
, ρ
1
,andρ
2
are nonnegative tuning parameters.
Here, P
1
({A}) and P
2
({B}) are convex structure penalty
functions, which aim at preserving the common structures in
the row and the column precision matrices, respectively. When
A
k
= I
p
or B
k
= I
q
(k = 1, 2,...,K ), our model reduces to
剩余14页未读，继续阅读
2018-04-14 上传
2012-03-10 上传
2021-03-13 上传
2018-11-13 上传
2021-02-11 上传
2012-04-27 上传
233 浏览量
2017-05-19 上传
2021-10-03 上传
weixin_38621897
- 粉丝: 6
- 资源: 956
我的内容管理
展开
最新资源
- BottleJS快速入门：演示JavaScript依赖注入优势
- vConsole插件使用教程：输出与复制日志文件
- Node.js v12.7.0版本发布 - 适合高性能Web服务器与网络应用
- Android中实现图片的双指和双击缩放功能
- Anum Pinki英语至乌尔都语开源词典：23000词汇会话
- 三菱电机SLIMDIP智能功率模块在变频洗衣机的应用分析
- 用JavaScript实现的剪刀石头布游戏指南
- Node.js v12.22.1版发布 - 跨平台JavaScript环境新选择
- Infix修复发布：探索新的中缀处理方式
- 罕见疾病酶替代疗法药物非临床研究指导原则报告
- Node.js v10.20.0 版本发布，性能卓越的服务器端JavaScript
- hap-java-client：Java实现的HAP客户端库解析
- Shreyas Satish的GitHub博客自动化静态站点技术解析
- vtomole个人博客网站建设与维护经验分享
- MEAN.JS全栈解决方案：打造MongoDB、Express、AngularJS和Node.js应用
- 东南大学网络空间安全学院复试代码解析
