Figure 3: Samples from an unconditional diffusion model with classifier guidance to condition
on the class "Pembroke Welsh corgi". Using classifier scale 1.0 (left; FID: 33.0) does not produce
convincing samples in this class, whereas classifier scale 10.0 (right; FID: 12.0) produces much more
class-consistent images.
We can now substitute this into the score function for p(x
t
)p(y|x
t
):
∇
x
t
log(p
θ
(x
t
)p
φ
(y|x
t
)) = ∇
x
t
log p
θ
(x
t
) + ∇
x
t
log p
φ
(y|x
t
) (12)
= −
1
√
1 − ¯α
t
θ
(x
t
) + ∇
x
t
log p
φ
(y|x
t
) (13)
Finally, we can define a new epsilon prediction
ˆ(x
t
)
which corresponds to the score of the joint
distribution:
ˆ(x
t
)
:
=
θ
(x
t
) −
√
1 − ¯α
t
∇
x
t
log p
φ
(y|x
t
) (14)
We can then use the exact same sampling procedure as used for regular DDIM, but with the modified
noise predictions
ˆ
θ
(x
t
)
instead of
θ
(x
t
)
. Algorithm 2 summaries the corresponding sampling
algorithm.
4.3 Scaling Classifier Gradients
To apply classifier guidance to a large scale generative task, we train classification models on
ImageNet. Our classifier architecture is simply the downsampling trunk of the UNet model with
an attention pool [
49
] at the 8x8 layer to produce the final output. We train these classifiers on the
same noising distribution as the corresponding diffusion model, and also add random crops to reduce
overfitting. After training, we incorporate the classifier into the sampling process of the diffusion
model using Equation 10, as outlined by Algorithm 1.
In initial experiments with unconditional ImageNet models, we found it necessary to scale the
classifier gradients by a constant factor larger than 1. When using a scale of 1, we observed that the
classifier assigned reasonable probabilities (around 50%) to the desired classes for the final samples,
but these samples did not match the intended classes upon visual inspection. Scaling up the classifier
gradients remedied this problem, and the class probabilities from the classifier increased to nearly
100%. Figure 3 shows an example of this effect.
To understand the effect of scaling classifier gradients, note that
s ·∇
x
log p(y|x) = ∇
x
log
1
Z
p(y|x)
s
,
where
Z
is an arbitrary constant. As a result, the conditioning process is still theoretically grounded
in a re-normalized classifier distribution proportional to
p(y|x)
s
. When
s > 1
, this distribution
becomes sharper than
p(y|x)
, since larger values are amplified by the exponent. In other words, using
a larger gradient scale focuses more on the modes of the classifier, which is potentially desirable for
producing higher fidelity (but less diverse) samples.
In the above derivations, we assumed that the underlying diffusion model was unconditional, modeling
p(x)
. It is also possible to train conditional diffusion models,
p(x|y)
, and use classifier guidance in
the exact same way. Table 4 shows that the sample quality of both unconditional and conditional
models can be greatly improved by classifier guidance. We see that, with a high enough scale, the
guided unconditional model can get quite close to the FID of an unguided conditional model, although
training directly with the class labels still helps. Guiding a conditional model further improves FID.
Table 4 also shows that classifier guidance improves precision at the cost of recall, thus introducing
a trade-off in sample fidelity versus diversity. We explicitly evaluate how this trade-off varies with
8
剩余43页未读，继续阅读
IT徐师兄
- 粉丝: 1974
- 资源: 2689
我的内容管理
收起
- 我的资源
快来上传第一个资源
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助
会员权益专享
最新资源
- 电力电子系统建模与控制入门
- SQL数据库基础入门：发展历程与关键概念
- DC/DC变换器动态建模与控制方法解析
- 市***专有云IaaS服务：云主机与数据库解决方案
- 紫鸟数据魔方：跨境电商选品神器，助力爆款打造
- 电力电子技术：DC-DC变换器动态模型与控制
- 视觉与实用并重：跨境电商产品开发的六重价值策略
- VB.NET三层架构下的数据库应用程序开发
- 跨境电商产品开发：关键词策略与用户痛点挖掘
- VC-MFC数据库编程技巧与实现
- 亚马逊新品开发策略：选品与市场研究
- 数据库基础知识：从数据到Visual FoxPro应用
- 计算机专业实习经验与项目总结
- Sparkle家族轻量级加密与哈希：提升IoT设备数据安全性
- SQL数据库期末考试精选题与答案解析
- H3C规模数据融合：技术探讨与应用案例解析
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
