高效网络设计：深度、宽度与分辨率的均衡策略

5星 · 超过95%的资源需积分: 19 140 浏览量更新于2024-09-01 收藏 908KB PDF 举报

Efficient Net.pdf 是一篇重要的论文，由 Mingxing Tan 和 Quoc V. Le 联合发表，深入探讨了卷积神经网络（Convolutional Neural Networks, ConvNets）的模型扩展策略。传统上，研究人员会根据固定的资源预算开发网络，然后随着可用资源的增加来提升准确率。然而，该研究揭示了一个关键发现：网络的深度（depth）、宽度（width）和输入图片分辨率（resolution）的平衡对于性能提升至关重要。论文的核心观点在于提出了一种新的模型缩放方法，即通过一个简单但高效的一体化复合系数，对深度、宽度和分辨率进行统一的尺度调整。这种方法旨在优化网络的性能，既能保持较高的识别精度，又能控制模型的大小和计算效率。作者将这一方法应用到 MobileNets 和 ResNet 上，显著提升了它们的性能。为了进一步优化，作者利用神经架构搜索技术设计了一种新的基础网络架构，进而构建出名为 EfficientNets 的系列模型。这个系列在 ImageNet 数据集上取得了卓越的成绩，如 EfficientNet-B7，达到了84.4% 的 top-1 准确率和97.1% 的 top-5 准确率，相比现有的最佳 ConvNet，模型尺寸小8.4倍，推理速度快6.1倍。这表明EfficientNets在保持高精度的同时，实现了更高的效率和更优的资源利用。该论文不仅展示了模型缩放方法的创新性，还为设计更加轻量级且性能优越的神经网络提供了实用的指导原则。它强调了在模型设计过程中，对网络结构各个维度的均衡考虑对于达到最优性能的重要性，这对于未来深度学习模型的发展具有深远的影响。研究者和工程师们可以参考这篇论文，了解如何在有限的资源下构建出既高效又准确的模型，以满足不断增长的实时计算和能源效率的需求。

展开

EfﬁcientNet: Rethinking Model Scaling for Convolutional Neural Networks

Mingxing Tan

Quoc V. Le

Abstract

Convolutional Neural Networks (ConvNets) are

commonly developed at a ﬁxed resource budget,

and then scaled up for better accuracy if more

resources are available. In this paper, we sys-

tematically study model scaling and identify that

carefully balancing network depth, width, and res-

olution can lead to better performance. Based

on this observation, we propose a new scaling

method that uniformly scales all dimensions of

depth/width/resolution using a simple yet highly

effective compound coefﬁcient. We demonstrate

the effectiveness of this method on scaling up

MobileNets and ResNet.

To go even further, we use neural architecture

search to design a new baseline network and

scale it up to obtain a family of models, called

EfﬁcientNets, which achieve much better accu-

racy and efﬁciency than previous ConvNets. In

particular, our EfﬁcientNet-B7 achieves state-

of-the-art 84.4% top-1 / 97.1% top-5 accuracy

on ImageNet, while being

8.4x smaller

and

6.1x faster

on inference than the best existing

ConvNet. Our EfﬁcientNets also transfer well and

achieve state-of-the-art accuracy on CIFAR-100

(91.7%), Flowers (98.8%), and 3 other transfer

learning datasets, with an order of magnitude

fewer parameters. Source code is at

https:

//github.com/tensorflow/tpu/tree/

master/models/official/efficientnet.

1. Introduction

Scaling up ConvNets is widely used to achieve better accu-

racy. For example, ResNet (He et al., 2016) can be scaled

up from ResNet-18 to ResNet-200 by using more layers;

Recently, GPipe (Huang et al., 2018) achieved 84.3% Ima-

geNet top-1 accuracy by scaling up a baseline model four

Google Research, Brain Team, Mountain View, CA. Corre-

spondence to: Mingxing Tan <tanmingxing@google.com>.

Proceedings of the

International Conference on Machine

Learning, Long Beach, California, PMLR 97, 2019.

0 20 40 60 80 100 120 140 160 180

Number of Parameters (Millions)

Imagenet Top-1 Accuracy (%)

ResNet-34

ResNet-50

ResNet-152

DenseNet-201

Inception-v2

Inception-ResNet-v2

NASNet-A

ResNeXt-101

Xception

AmoebaNet-A

AmoebaNet-C

SENet

EfﬁcientNet-B7

Top1 Acc. #Params

ResNet-152 (He et al., 2016) 77.8% 60M

EfﬁcientNet-B1 79.2% 7.8M

ResNeXt-101 (Xie et al., 2017) 80.9% 84M

EfﬁcientNet-B3 81.7% 12M

SENet (Hu et al., 2018) 82.7% 146M

NASNet-A (Zoph et al., 2018) 82.7% 89M

EfﬁcientNet-B4 83.0% 19M

GPipe (Huang et al., 2018)

†

84.3% 556M

EfﬁcientNet-B7 84.4% 66M

†

Not plotted

Figure 1. Model Size vs. ImageNet Accuracy.

All numbers are

for single-crop, single-model. Our EfﬁcientNets signiﬁcantly out-

perform other ConvNets. In particular, EfﬁcientNet-B7 achieves

new state-of-the-art 84.4% top-1 accuracy but being 8.4x smaller

and 6.1x faster than GPipe. EfﬁcientNet-B1 is 7.6x smaller and

5.7x faster than ResNet-152. Details are in Table 2 and 4.

time larger. However, the process of scaling up ConvNets

has never been well understood and there are currently many

ways to do it. The most common way is to scale up Con-

vNets by their depth (He et al., 2016) or width (Zagoruyko &

Komodakis, 2016). Another less common, but increasingly

popular, method is to scale up models by image resolution

(Huang et al., 2018). In previous work, it is common to scale

only one of the three dimensions – depth, width, and image

size. Though it is possible to scale two or three dimensions

arbitrarily, arbitrary scaling requires tedious manual tuning

and still often yields sub-optimal accuracy and efﬁciency.

In this paper, we want to study and rethink the process

of scaling up ConvNets. In particular, we investigate the

central question: is there a principled method to scale up

ConvNets that can achieve better accuracy and efﬁciency?

Our empirical study shows that it is critical to balance all

dimensions of network width/depth/resolution, and surpris-

ingly such balance can be achieved by simply scaling each

of them with constant ratio. Based on this observation, we

propose a simple yet effective compound scaling method.

Unlike conventional practice that arbitrary scales these fac-

tors, our method uniformly scales network width, depth,

arXiv:1905.11946v3 [cs.LG] 23 Nov 2019

下载后可阅读完整内容，剩余9页未读，立即下载

身份认证购VIP最低享 7 折!

30元优惠券

凌霜雪

粉丝: 35

高效网络设计：深度、宽度与分辨率的均衡策略

efficientnet-b5_notop.h5

EfficientNet

efficientnet-b0-355c32eb.pth

EfficientNet.pdf

EfficientDet.pdf

Adaptive Focus for Efficient Video Recognition.pdf

ResNet和EfficientNet遥感图像场景分类研究.pdf

EfficientNet_Rethinking Model Scaling for Convolutional Neural Networks.pdf

基于CBAM-EfficientNet的垃圾图像分类算法研究.pdf

Self-supervised_Attention_Mechanism_for_Pediatric_Bone_Age_Assessment_with_Efficient_Weak_Annotation.pdf

最新资源