张量网络入门：低秩分解与大规模优化

需积分: 10 178 浏览量更新于2024-07-18 1 收藏 9.14MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

1.3. Curse of Dimensionality 261

(a)

()´II

(b)

...

Vector (each entry is a block matrix)

Block matrix

Matrix

(c)

Matrix

Figure 1.7: Hierarchical matrix structures and their symbolic representation as

tensors. (a) A 4th-order tensor representation for a block matrix X



(a matrix of matrices), which comprises block matrices X



. (b) A 5th-

order tensor. (c) A 6th-order tensor.

number of variables, that is, with the dimensionality of the function.

In a general context of machine learning and the underlying optimiza-

tion problems, the “curse of dimensionality” may also refer to an ex-

ponentially increasing number of parameters required to describe the

data/system or an extremely large number of degrees of freedom. The

term “curse of dimensionality”, in the context of tensors, refers to the

262 Introduction and Motivation

phenomenon whereby the number of elements, I

, of an N th-order ten-

sor of size



  

grows exponentially with the tensor order, N .

Tensor volume can therefore easily become prohibitively big for multi-

way arrays for which the number of dimensions (“ways” or “modes”)

is very high, thus requiring enormous computational and memory re-

sources to process such data. The understanding and handling of the

inherent dependencies among the excessive degrees of freedom create

both diﬃcult to solve problems and fascinating new opportunities, but

comes at a price of reduced accuracy, owing to the necessity to involve

various approximations.

We show that the curse of dimensionality can be alleviated or even

fully dealt with through tensor network representations; these natu-

rally cater for the excessive volume, veracity and variety of data (see

Figure 1.1) and are supported by eﬃcient tensor decomposition algo-

rithms which involve relatively simple mathematical operations. An-

other desirable aspect of tensor networks is their relatively small-scale

and low-order core tensors, which act as “building blocks” of tensor

networks. These core tensors are relatively easy to handle and visual-

ize, and enable super-compression of the raw, incomplete, and noisy

huge-scale datasets. This also suggests a solution to a more general

quest for new technologies for processing of exceedingly large datasets

within aﬀordable computation times.

To address the curse of dimensionality, this work mostly focuses on

approximative low-rank representations of tensors, the so-called low-

rank tensor approximations (LRTA) or low-rank tensor network de-

compositions.

1.3.2 Separation of Variables and Tensor Formats

A tensor is said to be in a full format when it is represented as an orig-

inal (raw) multidimensional array (Klus and Schütte, 2015), however,

distributed storage and processing of high-order tensors in their full

format is infeasible due to the curse of dimensionality. The sparse for-

mat is a variant of the full tensor format which stores only the nonzero

entries of a tensor, and is used extensively in software tools such as the

1.3. Curse of Dimensionality 263

Tensor Toolbox (Bader and Kolda, 2015) and in the sparse grid ap-

proach (Garcke et al., 2001; Bungartz and Griebel, 2004; Hackbusch,

2012).

As already mentioned, the problem of huge dimensionality can be

alleviated through various distributed and compressed tensor network

formats, achieved by low-rank tensor network approximations. The un-

derpinning idea is that by employing tensor networks formats, both

computational costs and storage requirements may be dramatically re-

duced through distributed storage and computing resources. It is im-

portant to note that, except for very special data structures, a tensor

cannot be compressed without incurring some compression error, since

a low-rank tensor representation is only an approximation of the orig-

inal tensor.

The concept of compression of multidimensional large-scale data

by tensor network decompositions can be intuitively explained as fol-

lows. Consider the approximation of an N-variate function f

q 

, x

, . . . , x

by a ﬁnite sum of products of individual functions,

each depending on only one or a very few variables (Bebendorf, 2011;

Dolgov, 2014; Cho et al., 2016; Trefethen, 2017). In the simplest sce-

nario, the function f

can be (approximately) represented in the

following separable form

, x

, . . . , x

q 

q   

. (1.1)

In practice, when an N-variate function f

is discretized into an N th-

order array, or a tensor, the approximation in (1.1) then corresponds to

the representation by rank-1 tensors, also called elementary tensors (see

Section 2). Observe that with I

, n



1, 2, . . . , N denoting the size of

each mode and I



max

, the memory requirement to store such a

full tensor is



, which grows exponentially with N . On the

other hand, the separable representation in (1.1) is completely deﬁned

by its factors, f



1, 2, . . . , N ), and requires only



storage units. If x

, x

, . . . , x

are statistically independent random

variables, their joint probability density function is equal to the product

of marginal probabilities, f

q 

q   

, in an

exact analogy to outer products of elementary tensors. Unfortunately,

the form of separability in (1.1) is rather rare in practice.

剩余183页未读，继续阅读

qq_21583515

粉丝: 0
资源: 16

张量网络入门：低秩分解与大规模优化

fastai/pytorch课程讲义ipynb文件course-v3(2019)part1&part2;

logictensornetworks:数据和知识的深度学习和逻辑推理

deeplearning part1

请详细介绍有限尺寸闭合张量网络收缩算法的步骤

使用张量网络进行语音分类

基于轻量级深度张量网络的高光谱图像分类研究

解释张量网络的微分原理

pytorch张量神经网络

python pytorch 输入为一个[4,3,128,128]的张量和一个维度为[4,1,1,1]的张量，代码将两个张量拼接平整后输入三层全连接层神经网络，输出为一个维度为[4,1,1,1]的张量

神经网络模型张量是什么

怎样用神经网络将几个张量合并成一个张量

输入张量和输出张量具体区别在哪儿

网络流量怎么转化成张量

神经网络张量切片操作

神经网络中如何转换张量维度

0维张量转换为1维度

张量的差分运算是n-1张量

卷积神经网络的输出只能固定维度的张量对吗

[10000,2]的张量与[1]的张量相乘

pytorch 输入为一个[4,3,128,128]的张量和一个维度为[4,1,1,1]的张量，代码将两个张量拼接平整后输入三层全连接层神经网络，输出为一个维度为[4,1,1,1]的张量。写一个模型的类

最新资源