![](https://csdnimg.cn/release/download_crawler_static/18524937/bg1.jpg)
Aggregated Residual Transformations for Deep Neural Networks
Saining Xie
1
Ross Girshick
2
Piotr Doll
´
ar
2
Zhuowen Tu
1
Kaiming He
2
1
UC San Diego
2
Facebook AI Research
{s9xie,ztu}@ucsd.edu {rbg,pdollar,kaiminghe}@fb.com
Abstract
We present a simple, highly modularized network archi-
tecture for image classification. Our network is constructed
by repeating a building block that aggregates a set of trans-
formations with the same topology. Our simple design re-
sults in a homogeneous, multi-branch architecture that has
only a few hyper-parameters to set. This strategy exposes a
new dimension, which we call “cardinality” (the size of the
set of transformations), as an essential factor in addition to
the dimensions of depth and width. On the ImageNet-1K
dataset, we empirically show that even under the restricted
condition of maintaining complexity, increasing cardinality
is able to improve classification accuracy. Moreover, in-
creasing cardinality is more effective than going deeper or
wider when we increase the capacity. Our models, named
ResNeXt, are the foundations of our entry to the ILSVRC
2016 classification task in which we secured 2nd place.
We further investigate ResNeXt on an ImageNet-5K set and
the COCO detection set, also showing better results than
its ResNet counterpart. The code and models are publicly
available online
1
.
1. Introduction
Research on visual recognition is undergoing a transi-
tion from “feature engineering” to “network engineering”
[25, 24, 44, 34, 36, 38, 14]. In contrast to traditional hand-
designed features (e.g., SIFT [29] and HOG [5]), features
learned by neural networks from large-scale data [33] re-
quire minimal human involvement during training, and can
be transferred to a variety of recognition tasks [7, 10, 28].
Nevertheless, human effort has been shifted to designing
better network architectures for learning representations.
Designing architectures becomes increasingly difficult
with the growing number of hyper-parameters (width
2
, fil-
ter sizes, strides, etc.), especially when there are many lay-
ers. The VGG-nets [36] exhibit a simple yet effective strat-
egy of constructing very deep networks: stacking build-
1
https://github.com/facebookresearch/ResNeXt
2
Width refers to the number of channels in a layer.
256, 1x1, 4
4, 3x3, 4
4, 1x1, 256
+
256, 1x1, 4
4, 3x3, 4
4, 1x1, 256
256, 1x1, 4
4, 3x3, 4
4, 1x1, 256
....
total 32
paths
256-d in
+
256, 1x1, 64
64, 3x3, 64
64, 1x1, 256
+
256-d in
256-d out
256-d out
Figure 1. Left: A block of ResNet [14]. Right: A block of
ResNeXt with cardinality = 32, with roughly the same complex-
ity. A layer is shown as (# in channels, filter size, # out channels).
ing blocks of the same shape. This strategy is inherited
by ResNets [14] which stack modules of the same topol-
ogy. This simple rule reduces the free choices of hyper-
parameters, and depth is exposed as an essential dimension
in neural networks. Moreover, we argue that the simplicity
of this rule may reduce the risk of over-adapting the hyper-
parameters to a specific dataset. The robustness of VGG-
nets and ResNets has been proven by various visual recog-
nition tasks [7, 10, 9, 28, 31, 14] and by non-visual tasks
involving speech [42, 30] and language [4, 41, 20].
Unlike VGG-nets, the family of Inception models [38,
17, 39, 37] have demonstrated that carefully designed
topologies are able to achieve compelling accuracy with low
theoretical complexity. The Inception models have evolved
over time [38, 39], but an important common property is
a split-transform-merge strategy. In an Inception module,
the input is split into a few lower-dimensional embeddings
(by 1×1 convolutions), transformed by a set of specialized
filters (3×3, 5×5, etc.), and merged by concatenation. It
can be shown that the solution space of this architecture is a
strict subspace of the solution space of a single large layer
(e.g., 5×5) operating on a high-dimensional embedding.
The split-transform-merge behavior of Inception modules
is expected to approach the representational power of large
and dense layers, but at a considerably lower computational
complexity.
Despite good accuracy, the realization of Inception mod-
els has been accompanied with a series of complicating fac-
1
arXiv:1611.05431v2 [cs.CV] 11 Apr 2017