Under review as a conference paper at ICLR 2017
3.3 B-GAN
Our objective is to minimize the f-divergence between the distribution of the training data
p(x)
and
the generated distribution
q(x)
. We introduce two functions constructed using neural networks:
r
θ
D
(x) : X → R
parameterized by
θ
D
, and
G
θ
G
(z) : Z → X
parameterized by
θ
G
. Measure
q(x; θ
G
)dx
is a probability measure induced from the uniform distribution by
G
θ
G
(z)
. In this case,
r
θ
D
(x)
is regarded as a density ratio estimation network and
G
θ
G
(z)
is regarded as a generator
network for minimizing the f-divergence between p(x) and q(x).
Motivated by Section 3.2, we construct a b-GAN using the following two steps.
1.
Update
θ
D
to estimate the density ratio between
p(x)
and
q(x; θ
G
)
. To achieve this, we
minimize Eq. 5 with respect to
r
θ
(x)
. In this step, the density ratio model
r
θ
(x)
in Eq. 5
can be considered as r
θ
D
(x) in this step.
2.
Update
θ
G
to minimize the f-divergence
D
f
(p||q)
between
p(x)
and
q(x; θ
G
)
using the
obtained density-ratio. We are able to suppose that
q(x; θ
G
)r
θ
(x)
is close to
p(x)
. Instead
of D
f
(p||q), we update θ
G
to minimize the empirical approximation of D
f
(qr
θ
||q).
The b-GAN algorithm is summarized in Algorithm 1, where B is the batch size. In this study, a
single-step gradient method (Goodfellow et al., 2014; Nowozin et al., 2016) is adopted.
Algorithm 1: b-GAN
for number of training iterations do
sample
ˆ
X = {x
1
, ..., x
B
} from p(x) and
ˆ
Z = {z
1
, ..., z
B
} from an uniform distribution.
D-step: Update θ
D
:
θ
t+1
D
= θ
t
D
− ∇
θ
D
1
B
B
X
i=1
f
0
r
θ
D
(G(z
i
))
r
θ
D
(G(z
i
)) − f
r
θ
D
(G(z
i
))
− f
0
r
θ
D
(x
i
)
!
.
G-step: Update θ
G
:
θ
t+1
G
= θ
t
G
− ∇
θ
G
1
B
B
X
i=1
f
r(G
θ
G
(z
i
))
!
.
end for
In the D-step, the proposed algorithm estimates
p
q
toward any divergence; thus, it differs slightly from
the D-step of f-GAN because the estimated values, i.e.,
f
0
(
p
q
)
, are dependent on the divergences. We
also introduce an f-GAN-like update as follows. As mentioned in Section 2, we have two options in
the G step.
1.
D-step: minimize
E
x∼p(x)
[−f
0
(r
θ
D
(x))] + E
x∼q(x)
[f
0
(r
θ
D
(x))r
θ
D
(x) − f(r
θ
D
(x))]
w.r.t
θ
D
.
2.
G-step: minimize
E
x∼q(x;θ
G
)
[−f
0
(r(x))]
or
E
x∼q(x;θ
G
)
[−f
0
(r(x))r(x) + f(r(x))]
w.r.t
θ
G
.
3.4 ANALYSIS
Following Goodfellow et al. [2014], we explain the validity of the G-step and D-step. We then
explain the meaning of b-GAN. Finally, we analyze differences between b-GAN and f-GAN.
The density ratio is estimated in the D-step. The estimator of
r(x)
is an M-estimator and is asymptot-
ically consistent under the proper normal conditions (Appendix E.1).
In the G-step, we update the generator as minimizing
D
f
(p||q)
by replacing
p(x)
with
r
θ
(x)q(x)
.
We assume that
q(x; θ
G
)
is equivalent to
p(x)
when
θ
G
= θ
∗
,
q(x; θ
G
)
is identifiable, and the
optimal
r(x)
is obtained in the D-step. By our assumption, the acquired value in the G-step is
ˆ
θ
,
which minimizes the empirical approximation of
D
f
(r(x)q(x; θ
G
)kq(x; θ
G
)) = E
x∼q(x;θ
G
)
[r(x)]
.
4
剩余16页未读，继续阅读
遂言
- 粉丝: 6
- 资源: 19
我的内容管理
展开
最新资源
- Lombok 快速入门与注解详解
- SpringSecurity实战：声明式安全控制框架解析
- XML基础教程：从数据传输到存储解析
- Matlab实现图像空间平移与镜像变换示例
- Python流程控制与运算符详解
- Python基础：类型转换与循环语句
- 辰科CD-6024-4控制器说明书：LED亮度调节与触发功能解析
- AE particular插件全面解析：英汉对照与关键参数
- Shell脚本实践：创建tar包、字符串累加与简易运算器
- TMS320F28335：浮点处理器与ADC详解
- 互联网基础与结构解析：从ARPANET到多层次ISP
- Redhat系统中构建与Windows共享的Samba服务器实战
- microPython编程指南：从入门到实践
- 数据结构实验：顺序构建并遍历链表
- NVIDIA TX2系统安装与恢复指南
- C语言实现贪吃蛇游戏基础代码
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
