although more complex loss functions are usually used. This squared `
2
loss can
be used in a regression problem. In a classification problem, the cross entropy
loss is often used. The ground-truth in a classification problem is a categorical
variable t. We first convert the categorical variable t to a C dimensional vector
t (cf. the distance metric and data transformation note). Now both t and x
L
are probability mass functions, and the cross entropy loss measures the distance
between them. Hence, we can minimize the cross entropy (cf. the information
theory note.) Equation 5 explicitly models the loss function as a loss layer,
whose processing is modeled as a box with parameters w
L
.
Note that some layers may not have any parameters, that is, w
i
may be
empty for some i. The softmax layer is one such example.
3.2 The forward run
Suppose all the parameters of a CNN model w
1
, . . . , w
L−1
have been learned,
then we are ready to use this model for prediction. Prediction only involves run-
ning the CNN model forward, i.e., in the direction of the arrows in Equation 5.
Let’s take the image classification problem as an example. Starting from
the input x
1
, we make it pass the processing of the first layer (the box with
parameters w
1
), and get x
2
. In turn, x
2
is passed into the second layer, etc.
Finally, we achieve x
L
∈ R
C
, which estimates the posterior probabilities of x
1
belonging to the C categories. We can output the CNN prediction as
arg max
i
x
L
i
. (7)
Note that the loss layer is not needed in prediction. It is only useful when
we try to learn CNN parameters using a set of training examples. Now, the
problem is: how do we learn the model parameters?
3.3 Stochastic gradient descent (SGD)
As in many other learning systems, the parameters of a CNN model are opti-
mized to minimize the loss z, i.e., we want the prediction of a CNN model to
match the ground-truth labels.
Let’s suppose one training example x
1
is given for training such parameters.
The training process involves running the CNN network in both directions. We
first run the network in the forward pass to get x
L
to achieve a prediction using
the current CNN parameters. Instead of outputting a prediction, we need to
compare the prediction with the target t corresponding to x
1
, that is, continue
running the forward pass till the last loss layer. Finally, we achieve a loss z.
The loss z is then a supervision signal, guiding how the parameters of the
model should be modified (updated). And the SGD way of modifying the pa-
rameters is
w
i
←− w
i
− η
∂z
∂w
i
. (8)
6
剩余30页未读，继续阅读
追求一个梦想s
- 粉丝: 6
- 资源: 10
我的内容管理
展开
最新资源
- zlib-1.2.12压缩包解析与技术要点
- 微信小程序滑动选项卡源码模版发布
- Unity虚拟人物唇同步插件Oculus Lipsync介绍
- Nginx 1.18.0版本WinSW自动安装与管理指南
- Java Swing和JDBC实现的ATM系统源码解析
- 掌握Spark Streaming与Maven集成的分布式大数据处理
- 深入学习推荐系统：教程、案例与项目实践
- Web开发者必备的取色工具软件介绍
- C语言实现李春葆数据结构实验程序
- 超市管理系统开发：asp+SQL Server 2005实战
- Redis伪集群搭建教程与实践
- 掌握网络活动细节：Wireshark v3.6.3网络嗅探工具详解
- 全面掌握美赛：建模、分析与编程实现教程
- Java图书馆系统完整项目源码及SQL文件解析
- PCtoLCD2002软件：高效图片和字符取模转换
- Java开发的体育赛事在线购票系统源码分析
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
