Hands-on Bayesian Neural Networks - a Tutorial for Deep Learning Users 7
If the cost of a false positive varies across dierent classes, it should be used to compute the risk
and choose the minimal risk prediction.
3 MOTIVATION FOR BAYESIAN METHODS IN DEEP LEARNING
Dening a prior belief
p(θ)
on the model parametrization (Section 2) is regarded by some users to
be hard if not impossible. Dening a prior for a simple functional model is considered intuitive, e.g.,
explicitly adding a regularization term to favor a lower degree polynomial function or a smoother
function [
54
]. However, dening priors is harder for the multi-layer models used in deep learning.
So, why do we bother to use Bayesian methods for deep learning given that it is hard to clearly
comprehend deep neural networks behavior when dening the priors? The functional relationship
encoded by an articial neural network implicitly represents the conditional probability
p(y|x, θ)
,
and Bayes formula is an appropriate tool to use to invert conditional probabilities, even if one has a
priori little insight about
p(θ)
. While there are very strong theoretical principles and schema on
which this Bayes formula can be based [
76
], we focus in this section on some practical benets of
using Bayesian Deep networks.
First, Bayesian methods provide a natural approach to
quantify uncertainty
in deep learning.
Bayesian neural networks often have better calibration than classical neural networks [
46
,
58
,
66
],
i.e., their predicted uncertainty is more consistent with the observed errors. In other words, they
are neither overcondent nor undercondent compared to their non-Bayesian counterpart.
Working with a Bayesian neural network allows to distinguish between
epistemic uncertainty
,
i.e., the uncertainty due to a lack of knowledge, measured by
p(θ |D)
, which can be reduced with
more data, and
aleatoric uncertainty
, i.e., the uncertainty due to the (partially) aleatoric nature of
the data and measured by
p(y|x, θ)
[
14
,
44
]. This makes BNNs very data ecient, as they can learn
from a small dataset without overtting. At prediction time, out-of-training distribution points will
just lead to high epistemic uncertainty. It also makes BNNs an interesting tool for active learning
[
19
,
88
], as one can interpret the model predictions and see if, for a given input, dierent probable
parametrizations lead to dierent predictions. In this latter case, labelling this specic input will
eectively reduce the epistemic uncertainty.
Furthermore, the No-free-lunch theorem for machine learning [
94
] can be interpreted as saying
that any supervised learning algorithm includes some kind of implicit prior (while this interpretation
is more philosophical than mathematical, and thus subject to discussion). Bayesian methods, when
used correctly, will at least make the prior explicit. Now, if
integrating prior knowledge
seems
hard with tools that are basically black boxes, it is not impossible. In Bayesian deep learning, priors
are often considered as soft constraints, like regularization. Most regularization methods already
used for point estimate neural networks can be understood from a Bayesian perspective as setting
a prior, as demonstrated in Section 5.3. Moreover, previously learned posterior can be recycled as
prior when new data becomes available. This makes Bayesian neural networks a valuable tool for
online learning [64].
Last but not least, the Bayesian paradigm enables
the analysis of learning methods
and
draws links between them. Some methods initially not presented as Bayesian can be
implicitly
understood
as being approximate Bayesian, like regularization (Sec.5.3) or ensembling (Sec.8.2.2).
This, in turn, supports the understanding of why certain methods that are easier to use than a
strict application of the Bayesian algorithms can still give meaningful results from a Bayesian
perspective. In fact, most Bayesian neural network architectures used in practice rely on methods
that are approximately or implicitly Bayesian (Sec.8), because the exact algorithms are often too
expensive. The Bayesian paradigm also provides a systematic framework to design new learning
and regularization strategies, even for point-estimate models.
ACM Comput. Surv., Vol. 1, No. 1, Article . Publication date: July 2020.
剩余34页未读，继续阅读
syp_net
- 粉丝: 158
- 资源: 1187
我的内容管理
展开
最新资源
- 构建Cadence PSpice仿真模型库教程
- VMware 10.0安装指南：步骤详解与网络、文件共享解决方案
- 中国互联网20周年必读：影响行业的100本经典书籍
- SQL Server 2000 Analysis Services的经典MDX查询示例
- VC6.0 MFC操作Excel教程：亲测Win7下的应用与保存技巧
- 使用Python NetworkX处理网络图
- 科技驱动：计算机控制技术的革新与应用
- MF-1型机器人硬件与robobasic编程详解
- ADC性能指标解析：超越位数、SNR和谐波
- 通用示波器改造为逻辑分析仪：0-1字符显示与电路设计
- C++实现TCP控制台客户端
- SOA架构下ESB在卷烟厂的信息整合与决策支持
- 三维人脸识别：技术进展与应用解析
- 单张人脸图像的眼镜边框自动去除方法
- C语言绘制图形：余弦曲线与正弦函数示例
- Matlab 文件操作入门：fopen、fclose、fprintf、fscanf 等函数使用详解
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
