胶囊网络中的动态路由机制：实现MNIST数据集上的顶尖性能
1星 需积分: 9 116 浏览量
更新于2024-09-09
收藏 1.82MB PDF 举报 Dynamic Routing Between Capsules
Capsule 网络是一种基于神经网络的结构，它由多个 capsule 组成，每个 capsule 都是一组神经元的集合。这些神经元的活动向量（activity vector）表示特定实体类型的实例化参数，如对象或对象部分。 Capsule 网络的关键思想是使用活动向量的长度来表示实体存在的概率，向量方向表示实例化参数。
_capsule_之间的动态路由（Dynamic Routing）机制是指较低级别的 _capsule_ 通过变换矩阵对更高级别的 _capsule_ 的实例化参数进行预测。当多个预测一致时，更高级别的 _capsule_ 变得活跃。这种机制使得 Capsule 网络能够学习到更 abstract 的实体表示，从而提高了网络的表达能力和泛化能力。
在 MNIST 数据集上的实验结果表明，使用 Dynamic Routing 机制的多层 Capsule 系统达到了顶尖的性能，远远优越于卷积网络在识别高度重叠数字时的性能。这种结果证明了 Capsule 网络的强大表达能力和泛化能力。
Dynamic Routing 机制的关键是 routing-by-agreement 机制，即较低级别的 _capsule_ 偏向于将输出发送至高级别的 _capsule_，有了来自低级别 _capsule_ 的预测，高级别 _capsule_ 的活动向量具备较大的标量积。这使得高级别 _capsule_ 能够根据低级别 _capsule_ 的预测选择合适的实例化参数，从而提高了网络的表达能力和泛化能力。
在实际应用中，Dynamic Routing 机制可以用于图像识别、自然语言处理、语音识别等领域，提高网络的表达能力和泛化能力，从而提高了模型的性能和鲁棒性。
Dynamic Routing Between Capsules 是一种强大且灵活的神经网络结构，它能够学习到更 abstract 的实体表示，提高了网络的表达能力和泛化能力。这种结构可以应用于多种领域，提高模型的性能和鲁棒性。
should be coupled to capsule j.
c
ij
=
exp(b
ij
)
P
k
exp(b
ik
)
(3)
The log priors can be learned discriminatively at the same time as all the other weights. They depend
on the location and type of the two capsules but not on the current input image
2
. The initial coupling
coefficients are then iteratively refined by measuring the agreement between the current output
v
j
of
each capsule, j, in the layer above and the prediction ˆu
j|i
made by capsule i.
The agreement is simply the scalar product
a
ij
= v
j
.ˆu
j|i
. This agreement is treated as if it were a log
likelihood and is added to the initial logit,
b
ij
before computing the new values for all the coupling
coefficients linking capsule i to higher level capsules.
In convolutional capsule layers each unit in a capsule is a convolutional unit. Therefore, each capsule
will output a grid of vectors rather than a single vector output.
Procedure 1 Routing algorithm.
1: procedure ROUTING(ˆu
j|i
, r, l)
2: for all capsule i in layer l and capsule j in layer (l + 1): b
ij
0.
3: for r iterations do
4: for all capsule i in layer l: c
i
softmax(b
i
) . softmax computes Eq. 3
5: for all capsule j in layer ( l + 1): s
j
P
i
c
ij
ˆu
j|i
6: for all capsule j in layer ( l + 1): v
j
squash(s
j
) . squash computes Eq. 1
7: for all capsule i in layer l and capsule j in layer (l + 1): b
ij
b
ij
+ ˆu
j|i
.v
j
return v
j
3 Margin loss for digit existence
We are using the length of the instantiation vector to represent the probability that a capsule’s entity
exists, so we would like the top-level capsule for digit class
k
to have a long instantiation vector if
and only if that digit is present in the image. To allow for multiple digits, we use a separate margin
loss, L
k
for each digit capsule, k:
L
c
= T
c
max(0,m
+
||v
c
||)
2
+ (1 T
c
) max(0, ||v
c
|| m
)
2
(4)
where
T
c
=1
iff a digit of class
c
is present
3
and
m
+
=0.9
and
m
=0.1
. The
down-weighting
of the loss for absent digit classes stops the initial learning from shrinking the lengths of the activity
vectors of all the digit capsules. We suggest
=0.5
. The total loss is simply the sum of the losses of
all digit capsules.
4 CapsNet architecture
A simple CapsNet architecture is shown in Fig. 1. The architecture is shallow with only two
convolutional layers and one fully connected layer. Conv
1
has
256
,
9 ⇥ 9
convolution kernels with a
stride of 1 and ReLU activation. This layer converts pixel intensities to the activities of local feature
detectors that are then used as inputs to the primary capsules.
The primary capsules are the lowest level of multi-dimensional entities and, from an inverse graphics
perspective, activating the primary capsules corresponds to inverting the rendering process. This is a
very different type of computation than piecing instantiated parts together to make familiar wholes,
which is what capsules are designed to be good at.
The second layer (PrimaryCapsules) is a convolutional capsule layer with
32
channels of convolutional
8
D capsules (i.e. each primary capsule contains 8 convolutional units with a
9 ⇥ 9
kernel and a stride
of 2). Each primary capsule output sees the outputs of all
256⇥ 81
Conv
1
units whose receptive fields
overlap with the location of the center of the capsule. In total PrimaryCapsules has
[32, 6, 6]
capsule
2
For MNIST we found that it was sufficient to set all of these priors to be equal.
3
We do not allow an image to contain two instances of the same digit class. We address this weakness of
capsules in the discussion section.
3
剩余10页未读，继续阅读
2018-03-17 上传
2018-10-17 上传
2017-10-28 上传
2023-03-16 上传
2024-10-31 上传
2024-10-31 上传
攻城无数
- 粉丝: 178
- 资源: 44
我的内容管理
展开
最新资源
- SSM动力电池数据管理系统源码及数据库详解
- R语言桑基图绘制与SCI图输入文件代码分析
- Linux下Sakagari Hurricane翻译工作：cpktools的使用教程
- prettybench: 让 Go 基准测试结果更易读
- Python官方文档查询库，提升开发效率与时间节约
- 基于Django的Python就业系统毕设源码
- 高并发下的SpringBoot与Nginx+Redis会话共享解决方案
- 构建问答游戏：Node.js与Express.js实战教程
- MATLAB在旅行商问题中的应用与优化方法研究
- OMAPL138 DSP平台UPP接口编程实践
- 杰克逊维尔非营利地基工程的VMS项目介绍
- 宠物猫企业网站模板PHP源码下载
- 52简易计算器源码解析与下载指南
- 探索Node.js v6.2.1 - 事件驱动的高性能Web服务器环境
- 找回WinSCP密码的神器：winscppasswd工具介绍
- xctools：解析Xcode命令行工具输出的Ruby库
