It is perhaps clearer to write deeper networks like this using intermediary variables:
NN
MLP 2
(x) =y
h
1
=g
1
(xW
1
+ b
1
)
h
2
=g
2
(h
1
W
2
+ b
2
)
y =h
2
W
3
The vector resulting from each linear transform is referred to as a layer. The outer-most
linear transform results in the output layer and the other linear transforms result in hidden
layers. Each hidden layer is followed by a non-linear activation. In some cases, such as in
the last layer of our example, the bias vectors are forced to 0 (“dropped”).
Layers resulting from linear transformations are often referred to as fully connected, or
affine. Other types of architectures exist. In particular, image recognition problems benefit
from convolutional and pooling layers. Such layers have uses also in language processing,
and will be discussed in Section 9. Networks with more than one hidden layer are said to
be deep networks, hence the name deep learning.
When describing a neural network, one should specify the dimensions of the layers and
the input. A layer will expect a d
in
dimensional vector as its input, and transform it into a
d
out
dimensional vector. The dimensionality of the layer is taken to be the dimensionality
of its output. For a fully connected layer l(x) = xW + b with input dimensionality d
in
and
output dimensionality d
out
, the dimensions of x is 1 × d
in
, of W is d
in
× d
out
and of b is
1 × d
out
.
The output of the network is a d
out
dimensional vector. In case d
out
= 1, the network’s
output is a scalar. Such networks can be used for regression (or scoring) by considering
the value of the output, or for binary classification by consulting the sign of the output.
Networks with d
out
= k > 1 can be used for k-class classification, by associating each
dimension with a class, and looking for the dimension with maximal value. Similarly, if
the output vector entries are positive and sum to one, the output can be interpreted as
a distribution over class assignments (such output normalization is typically achieved by
applying a softmax transformation on the output layer, see Section 4.3).
The matrices and the bias terms that define the linear transformations are the parame-
ters of the network. It is common to refer to the collection of all parameters as θ. Together
with the input, the parameters determine the network’s output. The training algorithm is
responsible for setting their values such that the network’s predictions are correct. Training
is discussed in Section 6.
4.1 Representation Power
In terms of representation power, it was shown by (Hornik, Stinchcombe, & White, 1989;
Cybenko, 1989) that MLP1 is a universal approximator – it can approximate with any
desired non-zero amount of error a family of functions
8
that include all continuous functions
8. Specifically, a feed-forward network with linear output layer and at least one hidden layer with a “squash-
ing” activation function can approximate any Borel measurable function from one finite dimensional space
to another.
13
剩余74页未读，继续阅读
iThinkCRL
- 粉丝: 0
- 资源: 1
我的内容管理
展开
最新资源
- 多功能HTML网站模板：手机电脑适配与前端源码
- echarts实战：构建多组与堆叠条形图可视化模板
- openEuler 22.03 LTS专用openssh rpm包安装指南
- H992响应式前端网页模板源码包
- Golang标准库深度解析与实践方案
- C语言版本gRPC框架支持多语言开发教程
- H397响应式前端网站模板源码下载
- 资产配置方案：优化资源与风险管理的关键计划
- PHP宾馆管理系统（毕设）完整项目源码下载
- 中小企业电子发票应用与管理解决方案
- 多设备自适应网页源码模板下载
- 移动端H5模板源码，自适应响应式网页设计
- 探索轻量级可定制软件框架及其Http服务器特性
- Python网站爬虫代码资源压缩包
- iOS App唯一标识符获取方案的策略与实施
- 百度地图SDK2.7开发的找厕所应用源代码分享
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
