对抗自编码器：生成模型与应用探索

需积分: 42 103 浏览量更新于2024-07-18 1 收藏 7.78MB PDF 举报

"Adversarial Autoencoders" 是一篇由 Alireza Makhzani、Jonathon Shlens、Navdeep Jaitly 和 Ian Goodfellow（来自 University of Toronto 和 Google Brain）以及 Brendan Frey（同样来自 University of Toronto）合作撰写的论文，着重于深度学习领域中的创新方法。该研究结合了对抗神经网络 (GAN) 和自动编码器的概念，提出了一种名为 Adversarial Autoencoder (AAE) 的新型模型。在传统自动编码器的基础上，AAE引入了生成对抗网络的原理，将概率建模与生成模型相结合。自动编码器通常用于数据压缩和特征学习，而 GAN 则是通过训练两个对抗性的模型——一个生成器和一个判别器，来生成与训练数据分布相近的新样本。AAE的核心思想是让隐藏层编码向量的后验分布与给定的任意先验分布匹配。这样做的好处是，无论从哪个部分采样，都能得到有意义的数据表示，从而训练出一个强大的生成模型，它能将预设的先验分布映射到真实数据分布。论文展示了 AAEs在多种应用场景中的实用性，例如： 1. 半监督分类：通过利用未标记数据，AAE能够提升分类任务的性能，尤其是在数据标注不足的情况下。 2. 图像内容和风格分离：AAE可以帮助分析图像中的内容和样式特征，这对于图像编辑和理解具有重要意义。 3. 无监督聚类：由于其生成能力，AAE能够对数据进行无监督的分组，揭示潜在的数据结构。 4. 维度降维：AAE的潜在空间表示可以用来有效地减少原始数据的维度，同时保持关键信息。 5. 数据可视化：通过生成的中间表示，AAE能够生成数据的直观表示，有助于理解复杂数据集的内在模式。实验部分详细探讨了这些应用，并展示了 AAEs在实际问题上的性能优势，证明了它作为深度学习工具的强大潜力。这篇论文不仅推动了深度学习技术的发展，也为理解和利用对抗性训练在其他机器学习任务中的作用提供了新的视角。

Figure 2: Comparison of adversarial and variational autoencoder on MNIST. The hidden code

the hold-out images for an adversarial autoencoder ﬁt to (a) a 2-D Gaussian and (b) a mixture of 10

2-D Gaussians. Each color represents the associated label. Same for variational autoencoder with (c)

a 2-D gaussian and (d) a mixture of 10 2-D Gaussians. (e) Images generated by uniformly sampling

the Gaussian percentiles along each hidden code dimension

in the 2-D Gaussian adversarial

autoencoder.

where the aggregated posterior

q(z)

is deﬁned in Eq. (1) and we have assumed

q(z|x)

is Gaussian

and

p(z)

is an arbitrary distribution. The variational bound contains three terms. The ﬁrst term can

be viewed as the reconstruction term of an autoencoder and the second and third terms can be viewed

as regularization terms. Without the regularization terms, the model is simply a standard autoencoder

that reconstructs the input. However, in the presence of the regularization terms, the VAE learns a

latent representation that is compatible with

p(z)

. The second term of the cost function encourages

large variances for the posterior distribution while the third term minimizes the cross-entropy between

the aggregated posterior

q(z)

and the prior

p(z)

. KL divergence or the cross-entropy term in Eq. (2),

encourages

q(z)

to pick the modes of

p(z)

. In adversarial autoencoders, we replace the second two

terms with an adversarial training procedure that encourages

q(z)

to match to the whole distribution

of p(z).

In this section, we compare the ability of the adversarial autoencoder to the VAE to impose a speciﬁed

prior distribution

p(z)

on the coding distribution. Figure 2a shows the coding space

of the test

data resulting from an adversarial autoencoder trained on MNIST digits in which a spherical 2-D

Gaussian prior distribution is imposed on the hidden codes

. The learned manifold in Figure 2a

exhibits sharp transitions indicating that the coding space is ﬁlled and exhibits no “holes”. In practice,

sharp transitions in the coding space indicate that images generated by interpolating within

lie on

the data manifold (Figure 2e). By contrast, Figure 2c shows the coding space of a VAE with the same

architecture used in the adversarial autoencoder experiments. We can see that in this case the VAE

roughly matches the shape of a 2-D Gaussian distribution. However, no data points map to several

local regions of the coding space indicating that the VAE may not have captured the data manifold as

well as the adversarial autoencoder.

Figures 2b and 2d show the code space of an adversarial autoencoder and of a VAE where the imposed

distribution is a mixture of 10 2-D Gaussians. The adversarial autoencoder successfully matched the

aggregated posterior with the prior distribution (Figure 2b). In contrast, the VAE exhibit systematic

differences from the mixture 10 Gaussians indicating that the VAE emphasizes matching the modes

of the distribution as discussed above (Figure 2d).

剩余15页未读，继续阅读

闲秋

粉丝: 11
资源: 9

对抗自编码器：生成模型与应用探索

adversarial-autoencoder, 敌对 autoencoder ( AAE )的Chainer实现.zip

Python-AdversarialAutoencoders的Tensorflow实现

adversarial autoencoders

Adversarial Latent Autoencoders.pdf

Adversarial_Autoencoder：Adversarial自动编码器向导指南

Autoencoders-开源

autoencoders_pytorch

Adversarial Variational Bayes.pdf

Ian Goodfellow introduction to GANs ( Generative Adversarial Networks)

Autoencoders-and-GANs-with-Python

最新资源