没有合适的资源?快使用搜索试试~ 我知道了~
首页张量网络入门:低秩分解与大规模优化
本篇文章《张量网络 part 1》聚焦于现代工程与数据科学中的关键问题——处理高维度、大规模且结构复杂的多模态数据。随着大数据时代的到来,标准机器学习算法常常受到所谓的“维度灾难”(curse of dimensionality)的困扰,即数据规模和跨模态关联的复杂性呈指数级增长,这使得分析大规模、多模态和多关系的数据集变得极其困难。张量网络作为一种新兴工具,因其低秩分解的能力,正逐渐成为解决这一问题的有效途径。 文章首先介绍了背景和动机,强调了在大数据处理中面临的挑战,包括如何高效处理海量信息和复杂关系。张量网络通过将多维数据表示为多方式数组或张量,能够提供一种有效的降维方法,从而缓解维度灾难带来的问题。通过直观的图形表示,张量网络的数学操作得以清晰呈现,使得从矩阵运算到张量网络操作的迁移更为顺畅,同时为后续的理论发展、实际应用和非欧几里得扩展提供了平台。 文章的核心内容集中在Tucker和tensor train (TT)分解,以及它们的扩展上。这些低秩分解方法具有线性甚至超线性的(如对数级)可扩展性,这是在后续章节详细讨论的。张量网络的这种特性使得在处理大型优化问题时,算法的时间复杂度显著降低,这对于许多依赖于多线性代数但又不直接使用其数学表达式的研究领域来说尤其有价值。 《张量网络 part 1》旨在为多学科的机器学习和数据分析社区提供一个深入理解张量网络降维原理和操作框架的平台,以应对现代工程和数据科学中的挑战,推动在大规模优化问题上的实际应用。通过阅读本文,读者将能够掌握如何利用张量网络来实现高效的数据分析和模型构建,尤其是在面临复杂数据结构时。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/10515544/bg10.jpg)
1.3. Curse of Dimensionality 261
(a)
X
X
R
11
I
R
22
I
I
1
I
2
R
1
R
2
12
()´II
Û
I
1
I
2
R
1
R
2
=
(b)
...
Vector (each entry is a block matrix)
Block matrix
Matrix
=
(c)
Matrix
Û
=
Figure 1.7: Hierarchical matrix structures and their symbolic representation as
tensors. (a) A 4th-order tensor representation for a block matrix X
P
R
R
1
I
1
R
2
I
2
(a matrix of matrices), which comprises block matrices X
r
1
,r
2
P
R
I
1
I
2
. (b) A 5th-
order tensor. (c) A 6th-order tensor.
number of variables, that is, with the dimensionality of the function.
In a general context of machine learning and the underlying optimiza-
tion problems, the “curse of dimensionality” may also refer to an ex-
ponentially increasing number of parameters required to describe the
data/system or an extremely large number of degrees of freedom. The
term “curse of dimensionality”, in the context of tensors, refers to the
![](https://csdnimg.cn/release/download_crawler_static/10515544/bg11.jpg)
262 Introduction and Motivation
phenomenon whereby the number of elements, I
N
, of an N th-order ten-
sor of size
p
I
I
I
q
grows exponentially with the tensor order, N .
Tensor volume can therefore easily become prohibitively big for multi-
way arrays for which the number of dimensions (“ways” or “modes”)
is very high, thus requiring enormous computational and memory re-
sources to process such data. The understanding and handling of the
inherent dependencies among the excessive degrees of freedom create
both difficult to solve problems and fascinating new opportunities, but
comes at a price of reduced accuracy, owing to the necessity to involve
various approximations.
We show that the curse of dimensionality can be alleviated or even
fully dealt with through tensor network representations; these natu-
rally cater for the excessive volume, veracity and variety of data (see
Figure 1.1) and are supported by efficient tensor decomposition algo-
rithms which involve relatively simple mathematical operations. An-
other desirable aspect of tensor networks is their relatively small-scale
and low-order core tensors, which act as “building blocks” of tensor
networks. These core tensors are relatively easy to handle and visual-
ize, and enable super-compression of the raw, incomplete, and noisy
huge-scale datasets. This also suggests a solution to a more general
quest for new technologies for processing of exceedingly large datasets
within affordable computation times.
To address the curse of dimensionality, this work mostly focuses on
approximative low-rank representations of tensors, the so-called low-
rank tensor approximations (LRTA) or low-rank tensor network de-
compositions.
1.3.2 Separation of Variables and Tensor Formats
A tensor is said to be in a full format when it is represented as an orig-
inal (raw) multidimensional array (Klus and Schütte, 2015), however,
distributed storage and processing of high-order tensors in their full
format is infeasible due to the curse of dimensionality. The sparse for-
mat is a variant of the full tensor format which stores only the nonzero
entries of a tensor, and is used extensively in software tools such as the
![](https://csdnimg.cn/release/download_crawler_static/10515544/bg12.jpg)
1.3. Curse of Dimensionality 263
Tensor Toolbox (Bader and Kolda, 2015) and in the sparse grid ap-
proach (Garcke et al., 2001; Bungartz and Griebel, 2004; Hackbusch,
2012).
As already mentioned, the problem of huge dimensionality can be
alleviated through various distributed and compressed tensor network
formats, achieved by low-rank tensor network approximations. The un-
derpinning idea is that by employing tensor networks formats, both
computational costs and storage requirements may be dramatically re-
duced through distributed storage and computing resources. It is im-
portant to note that, except for very special data structures, a tensor
cannot be compressed without incurring some compression error, since
a low-rank tensor representation is only an approximation of the orig-
inal tensor.
The concept of compression of multidimensional large-scale data
by tensor network decompositions can be intuitively explained as fol-
lows. Consider the approximation of an N-variate function f
p
x
q
f
p
x
1
, x
2
, . . . , x
N
q
by a finite sum of products of individual functions,
each depending on only one or a very few variables (Bebendorf, 2011;
Dolgov, 2014; Cho et al., 2016; Trefethen, 2017). In the simplest sce-
nario, the function f
p
x
q
can be (approximately) represented in the
following separable form
f
p
x
1
, x
2
, . . . , x
N
q
f
p
1
q
p
x
1
q
f
p
2
q
p
x
2
q
f
p
N
q
p
x
N
q
. (1.1)
In practice, when an N-variate function f
p
x
q
is discretized into an N th-
order array, or a tensor, the approximation in (1.1) then corresponds to
the representation by rank-1 tensors, also called elementary tensors (see
Section 2). Observe that with I
n
, n
1, 2, . . . , N denoting the size of
each mode and I
max
n
t
I
n
u
, the memory requirement to store such a
full tensor is
±
N
n
1
I
n
¤
I
N
, which grows exponentially with N . On the
other hand, the separable representation in (1.1) is completely defined
by its factors, f
p
n
q
p
x
n
q
,
p
n
1, 2, . . . , N ), and requires only
°
N
n
1
I
n
!
I
N
storage units. If x
1
, x
2
, . . . , x
N
are statistically independent random
variables, their joint probability density function is equal to the product
of marginal probabilities, f
p
x
q
f
p
1
q
p
x
1
q
f
p
2
q
p
x
2
q
f
p
N
q
p
x
N
q
, in an
exact analogy to outer products of elementary tensors. Unfortunately,
the form of separability in (1.1) is rather rare in practice.
![](https://csdnimg.cn/release/download_crawler_static/10515544/bg13.jpg)
264 Introduction and Motivation
The concept of tensor networks rests upon generalized (full or par-
tial) separability of the variables of a high dimensional function. This
can be achieved in different tensor formats, including:
• The Canonical Polyadic (CP) format (see Section 3.2), where
f
p
x
1
, x
2
, . . . , x
N
q
R
¸
r
1
f
p
1
q
r
p
x
1
q
f
p
2
q
r
p
x
2
q
f
p
N
q
r
p
x
N
q
, (1.2)
in an exact analogy to (1.1). In a discretized form, the above CP
format can be written as an N th-order tensor
F
R
¸
r
1
f
p
1
q
r
f
p
2
q
r
f
p
N
q
r
P
R
I
1
I
2
I
N
, (1.3)
where f
p
n
q
r
P
R
I
n
denotes a discretized version of the univariate
function f
p
n
q
r
p
x
n
q
, symbol
denotes the outer product, and R is
the tensor rank.
• The Tucker format, given by
f
p
x
1
, . . . , x
N
q
R
1
¸
r
1
1
R
N
¸
r
N
1
g
r
1
,...,r
N
f
p
1
q
r
1
p
x
1
q
f
p
N
q
r
N
p
x
N
q
,
(1.4)
and its distributed tensor network variants (see Section 3.3),
• The Tensor Train (TT) format (see Section 4.1), in the form
f
p
x
1
, x
2
, . . . , x
N
q
R
1
¸
r
1
1
R
2
¸
r
2
1
R
N
1
¸
r
N
1
1
f
p
1
q
r
1
p
x
1
q
f
p
2
q
r
1
r
2
p
x
2
q
f
p
N
2
q
r
N
2
r
N
1
p
x
N
1
q
f
p
N
q
r
N
1
p
x
N
q
, (1.5)
with the equivalent compact matrix representation
f
p
x
1
, x
2
, . . . , x
N
q
F
p
1
q
p
x
1
q
F
p
2
q
p
x
2
q
F
p
N
q
p
x
N
q
, (1.6)
where F
p
n
q
p
x
n
q P
R
R
n
1
R
n
, with R
0
R
N
1.
• The Hierarchical Tucker (HT) format (also known as the Hierar-
chical Tensor format) can be expressed via a hierarchy of nested
![](https://csdnimg.cn/release/download_crawler_static/10515544/bg14.jpg)
1.3. Curse of Dimensionality 265
separations in the following way. Consider nested nonempty dis-
joint subsets u, v, and t
u
Y
v
t
1, 2, . . . , N
u
, then for some
1
¤
N
0
N, with u
0
t
1, . . . , N
0
u
and v
0
t
N
0
1, . . . , N
u
,
the HT format can be expressed as
f
p
x
1
, . . . , x
N
q
R
u
0
¸
r
u
0
1
R
v
0
¸
r
v
0
1
g
p
12
N
q
r
u
0
,r
v
0
f
p
u
0
q
r
u
0
p
x
u
0
q
f
p
v
0
q
r
v
0
p
x
v
0
q
,
f
p
t
q
r
t
p
x
t
q
R
u
¸
r
u
1
R
v
¸
r
v
1
g
p
t
q
r
u
,r
v
,r
t
f
p
u
q
r
u
p
x
u
q
f
p
v
q
r
v
p
x
v
q
,
where x
t
t
x
i
: i
P
t
u
. See Section 2.2.1 for more detail.
Example. In a particular case for N =4, the HT format can be
expressed by
f
p
x
1
, x
2
, x
3
, x
4
q
R
12
¸
r
12
1
R
34
¸
r
34
1
g
p
1234
q
r
12
,r
34
f
p
12
q
r
12
p
x
1
, x
2
q
f
p
34
q
r
34
p
x
3
, x
4
q
,
f
p
12
q
r
12
p
x
1
, x
2
q
R
1
¸
r
1
1
R
2
¸
r
2
1
g
p
12
q
r
1
,r
2
,r
12
f
p
1
q
r
1
p
x
1
q
f
p
2
q
r
2
p
x
2
q
,
f
p
34
q
r
34
p
x
3
, x
4
q
R
3
¸
r
3
1
R
4
¸
r
4
1
g
p
34
q
r
3
,r
4
,r
34
f
p
3
q
r
3
p
x
3
q
f
p
4
q
r
4
p
x
4
q
.
The Tree Tensor Network States (TTNS) format, which is an ex-
tension of the HT format, can be obtained by generalizing the two
subsets, u, v, into a larger number of disjoint subsets u
1
, . . . , u
m
,
m
¥
2. In other words, the TTNS can be obtained by more flexi-
ble separations of variables through products of larger numbers of
functions at each hierarchical level (see Section 2.2.1 for graphical
illustrations and more detail).
All the above approximations adopt the form of “sum-of-products” of
single-dimensional functions, a procedure which plays a key role in all
tensor factorizations and decompositions.
Indeed, in many applications based on multivariate functions, very
good approximations are obtained with a surprisingly small number
of factors; this number corresponds to the tensor rank, R, or tensor
剩余183页未读,继续阅读
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/default.jpg!1)
qq_21583515
- 粉丝: 0
- 资源: 16
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 多模态联合稀疏表示在视频目标跟踪中的应用
- Kubernetes资源管控与Gardener开源软件实践解析
- MPI集群监控与负载平衡策略
- 自动化PHP安全漏洞检测:静态代码分析与数据流方法
- 青苔数据CEO程永:技术生态与阿里云开放创新
- 制造业转型: HyperX引领企业上云策略
- 赵维五分享:航空工业电子采购上云实战与运维策略
- 单片机控制的LED点阵显示屏设计及其实现
- 驻云科技李俊涛:AI驱动的云上服务新趋势与挑战
- 6LoWPAN物联网边界路由器:设计与实现
- 猩便利工程师仲小玉:Terraform云资源管理最佳实践与团队协作
- 类差分度改进的互信息特征选择提升文本分类性能
- VERITAS与阿里云合作的混合云转型与数据保护方案
- 云制造中的生产线仿真模型设计与虚拟化研究
- 汪洋在PostgresChina2018分享:高可用 PostgreSQL 工具与架构设计
- 2018 PostgresChina大会:阿里云时空引擎Ganos在PostgreSQL中的创新应用与多模型存储
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)