深度学习图像建模：像素递归神经网络

需积分: 0 164 浏览量更新于2024-08-05 收藏 2.83MB PDF 举报

"2016年，DeepMind发表的研究论文《像素的递归神经网络-Pixel Recurrent Neural Networks》" 这篇论文介绍了像素递归神经网络（Pixel Recurrent Neural Networks，PixelRNN）这一深度学习模型，它在无监督学习领域具有重要意义。该模型的核心是通过递归神经网络（RNN）序列预测图像中的每个像素值，同时考虑了两个空间维度的依赖关系，以建模自然图像的概率分布。传统的图像建模方法往往难以同时满足表达性、计算效率和可扩展性的要求。PixelRNN通过引入新的架构创新，如快速二维的递归层和残差连接在深层递归网络中的有效应用，解决了这些问题。这些创新使得模型能够更准确地捕捉图像中的复杂结构和全局一致性。在自然图像的对数似然得分上，PixelRNN显著优于先前的最佳结果，展示了其在表示和理解图像数据的强大能力。此外，PixelRNN生成的样本具有清晰度、多样性和整体连贯性，这意味着它可以生成逼真的图像，这对于图像合成和增强等应用非常有用。论文还提供了在广泛使用的ImageNet数据集上的基准测试，这是对模型性能的一种严格检验。ImageNet包含了大量类别丰富的图像，因此在该数据集上的成功表明，PixelRNN在处理多样化视觉信息时具有广泛的适应性。递归神经网络（RNN）是一种能够处理序列数据的深度学习模型，因为它们具有记忆单元，可以捕捉序列中的长期依赖关系。在PixelRNN中，RNN被用于逐行和逐列预测图像像素，这种顺序处理方式允许模型逐渐构建出整个图像的表示。残差连接则帮助解决了深度学习网络中常见的梯度消失问题，使得信息能更有效地传递到网络的深层。 PixelRNN是深度学习在无监督图像建模领域的一个里程碑，它的贡献在于提出了一种能够高效且准确地捕获自然图像复杂性的新方法。这项工作对于后续的生成对抗网络（GANs）、变分自编码器（VAEs）和其他基于深度学习的图像生成模型的发展有着深远的影响。

Pixel Recurrent Neural Networks

The value p(x

, ..., x

i−1

) is the probability of the i-th

pixel x

given all the previous pixels x

, ..., x

i−1

. The gen-

eration proceeds row by row and pixel by pixel. Figure 2

(Left) illustrates the conditioning scheme.

Each pixel x

is in turn jointly determined by three values,

one for each of the color channels Red, Green and Blue

(RGB). We rewrite the distribution p(x

) as the fol-

lowing product:

p(x

i,R

)p(x

i,G

, x

i,R

)p(x

i,B

, x

i,R

, x

i,G

) (2)

Each of the colors is thus conditioned on the other channels

as well as on all the previously generated pixels.

Note that during training and evaluation the distributions

over the pixel values are computed in parallel, while the

generation of an image is sequential.

2.2. Pixels as Discrete Variables

Previous approaches use a continuous distribution for the

values of the pixels in the image (e.g. Theis & Bethge

(2015); Uria et al. (2014)). By contrast we model p(x) as

a discrete distribution, with every conditional distribution

in Equation 2 being a multinomial that is modeled with a

softmax layer. Each channel variable x

i,∗

simply takes one

of 256 distinct values. The discrete distribution is represen-

tationally simple and has the advantage of being arbitrarily

multimodal without prior on the shape (see Fig. 6). Exper-

imentally we also ﬁnd the discrete distribution to be easy

to learn and to produce better performance compared to a

continuous distribution (Section 5).

3. Pixel Recurrent Neural Networks

In this section we describe the architectural components

that compose the PixelRNN. In Sections 3.1 and 3.2, we

describe the two types of LSTM layers that use convolu-

tions to compute at once the states along one of the spatial

dimensions. In Section 3.3 we describe how to incorporate

residual connections to improve the training of a PixelRNN

with many LSTM layers. In Section 3.4 we describe the

softmax layer that computes the discrete joint distribution

of the colors and the masking technique that ensures the

proper conditioning scheme. In Section 3.5 we describe the

PixelCNN architecture. Finally in Section 3.6 we describe

the multi-scale architecture.

3.1. Row LSTM

The Row LSTM is a unidirectional layer that processes

the image row by row from top to bottom computing fea-

tures for a whole row at once; the computation is per-

formed with a one-dimensional convolution. For a pixel

the layer captures a roughly triangular context above the

pixel as shown in Figure 4 (center). The kernel of the one-

Figure 3. In the Diagonal BiLSTM, to allow for parallelization

along the diagonals, the input map is skewed by offseting each

row by one position with respect to the previous row. When the

spatial layer is computed left to right and column by column, the

output map is shifted back into the original size. The convolution

uses a kernel of size 2 × 1.

dimensional convolution has size k × 1 where k ≥ 3; the

larger the value of k the broader the context that is captured.

The weight sharing in the convolution ensures translation

invariance of the computed features along each row.

The computation proceeds as follows. An LSTM layer has

an input-to-state component and a recurrent state-to-state

component that together determine the four gates inside the

LSTM core. To enhance parallelization in the Row LSTM

the input-to-state component is ﬁrst computed for the entire

two-dimensional input map; for this a k × 1 convolution is

used to follow the row-wise orientation of the LSTM itself.

The convolution is masked to include only the valid context

(see Section 3.4) and produces a tensor of size 4h × n × n,

representing the four gate vectors for each position in the

input map, where h is the number of output feature maps.

To compute one step of the state-to-state component of

the LSTM layer, one is given the previous hidden and cell

states h

i−1

and c

i−1

, each of size h × n × 1. The new

hidden and cell states h

, c

are obtained as follows:

, f

, i

, g

] = σ(K

~ h

i−1

+ K

~ x

)

= f

 c

i−1

+ i

 g

= o

 tanh(c

)

(3)

where x

of size h × n × 1 is row i of the input map, and

~ represents the convolution operation and  the element-

wise multiplication. The weights K

and K

are the

kernel weights for the state-to-state and the input-to-state

components, where the latter is precomputed as described

above. In the case of the output, forget and input gates o

and i

, the activation σ is the logistic sigmoid function,

whereas for the content gate g

, σ is the tanh function.

Each step computes at once the new state for an entire row

of the input map. Because the Row LSTM has a triangular

receptive ﬁeld (Figure 4), it is unable to capture the entire

available context.

剩余10页未读，继续阅读

Crazyanti

粉丝: 26
资源: 302

深度学习图像建模：像素递归神经网络

【10】Towards End-to-End Speech Recognitionwith Recurrent Neural Networks.pdf

2016-Session based Recommendations with Recurrent Neural Networks.pdf

基于Pixel Recurrent Neural Networks的图像生成网络

A Comprehensive Review of Deep Learning-based Single Image S

深度学习中的Pixel-RNN模型在TensorFlow的实现

【Advanced】Using MATLAB to Implement Long Short-Term Memory (LSTM) Networks for Classification and ...

Challenges and Solutions for Multi-Label Classification Problems: 5 Strategies to Help You Overcome ...

IncompatibleClassChangeError(解决方案).md

中国智慧工地行业市场研究（2023）Word(63页).docx

java大题啊实打实的

最新资源