深度卷积网络架构研究：创新与分类

需积分: 9 129 浏览量更新于2024-07-17 收藏 944KB PDF 举报

"该文档是一份深度卷积神经网络（CNN）最新架构的调查报告，由Asifullah Khan等人撰写。报告中指出，CNN在各种竞争性基准测试中展现出最先进的性能，这主要得益于其多层特征提取能力，能自动学习数据的层次表示。大量数据的可用性和硬件处理单元的进步推动了CNN研究的发展。近期，很多创新的CNN架构被提出，通过改进架构设计、优化参数等手段提升在视觉任务中的表现。报告将近年来的CNN架构创新分为七类：空间利用、深度、多路径、宽度、特征图利用、通道增强和注意力机制。同时，该文还介绍了CNN的基本组成部分，讨论了当前面临的挑战和CNN的应用情况。" 深度卷积神经网络（CNN）是神经网络的一个特例，它们在图像识别、图像分类、目标检测等计算机视觉任务中取得了显著成果。CNN的强大学习能力来源于其多级特征提取层，这些层能自下而上地学习数据的层次结构。随着大数据集的出现和硬件计算能力的增强，CNN的研究进入了一个快速发展的阶段。近年来，研究者们不断探索新的CNN架构以提高性能。例如，通过调整网络的深度、宽度或采用多路径结构来增加模型复杂性和表达能力。深度是指网络中的层数，更深层次的网络可以学习到更复杂的特征；宽度则涉及每一层中的神经元数量，更宽的网络可以捕捉更多细节信息；多路径结构允许信息通过不同的路径进行处理，增加了模型的多样性。此外，特征图的利用是另一个关键创新点，如残差学习和空洞卷积，它们提高了网络对空间信息的捕获能力。通道增强关注不同通道间的信息交互，比如SE模块（Squeeze-and-Excitation Networks），它可以动态调整通道权重，强调重要特征。注意力机制则借鉴人类视觉系统，使模型能够根据输入数据动态聚焦于重要部分，如自注意力和位置注意力机制。 CNN的组件包括卷积层、池化层、全连接层等，它们协同工作以构建复杂的特征表示。优化参数（如学习率、权重初始化和正则化）也是提升性能的关键。然而，CNN还面临着过拟合、计算效率低和内存需求大等问题，因此研究人员也在探索如权值共享、模型压缩和量化等技术来解决这些问题。最后，CNN的应用广泛，包括但不限于图像识别、自然语言处理、自动驾驶和医学影像分析等领域。未来，CNN将继续发展，结合新的理论和技术，以适应更复杂的任务和更高的效率要求。

This performance improvement expedited the research in CNN by extending its application in

optical character recognition (OCR) to other script’s character recognition

63–65

, deployment in

image sensors for face detection in video conferencing, and regulation of street crimes, etc.

Likewise, CNN based systems were industrialized in markets for tracking and detection of

customers

. Moreover, CNN’s potential in other applications such as medical image

segmentation, anomaly detection, and robot vision was also evaluated

67–69

3.3. 2006-2011: Revival of CNN

Deep NNs have a very complex structure and time intensive training phase that sometimes

spanned over weeks and even months. In early 2000, there was no appropriate approach for the

training of deep networks. Because of these limitations, CNN was not scaled well enough to be

applied to complex problems. These challenges halted the use of CNN in ML related tasks.

To address these problems, in 2006 many methods were developed to overcome the difficulties

encountered in the training of deep CNNs and learning of invariant features. Hinton proposed

greedy layer-wise training approach in 2006

, for deep architectures. The renaissance of a deep

learning

71,72

was one of the factor, which brought deep CNNs into the limelight. Huang et al.

used max pooling

instead of subsampling, which showed good results by learning of invariant

features

. In late 2006, researchers started using graphics processing units (GPUs)

accelerate training of deep NN and CNN architectures

75,76

. In 2007, NVIDIA launched the

CUDA programming platform

77,78

, which allows exploitation of parallel processing capabilities

of GPU with a much greater degree

. The use of GPUs for NN training

75,79

and hardware

improvements were the main factor, which revived the research in CNN. In 2010, Fei-Fei Li’s

group at Stanford, established a large database of images known as ImageNet, containing

millions of labeled images

. This database was coupled with the annual ILSVRC competition,

where submitted models performances were evaluated and scored

. This was a turning point in

improving the performance and increasing the use of CNN.

3.4. 2012-2014: Rise of CNN

Availability of big training data, hardware advancements, and computational resources

contributed to advancement in CNN algorithms and renaissance of CNN in object detection,

image classification, and segmentation related tasks

11,82

. However, the success of CNN in image

classification tasks was not just the result of aforementioned factors but mainly contributed by

architectural modifications, parameter optimization, incorporation of regulatory units, and

reformulation and readjustment of connections within the network

34,35,83

The main breakthrough in CNN performance was brought by AlexNet

. AlexNet won the 2012-

ILSVRC competition, which was one of the most difficult challenges for image detection and

classification. AlexNet improved performance by exploiting depth (incorporate multiple levels of

transformation) and introduced regularization term in CNN. The exemplary performance of

AlexNet

as compared to conventional ML techniques in 2012-ILSVRC (AlexNet reduced

error rate from 25.8 to 16.4) suggested that the main reason of the saturation in CNN

performance before 2006 was largely due to the unavailability of enough training data and

computational resources. These resource deficiencies made it hard to train a high-capacity CNN

without deterioration of performance

With CNN becoming more of a commodity in the computer vision (CV) ﬁeld, a number of

attempts have been made to improve the original architecture of AlexNet

. Similarly, in 2013

and 2014 researchers worked on parameter optimization to accelerate CNN performance in

diverse applications with a small increase in computational complexity. In 2013, Zeiler and

Fergus

defined a mechanism to visualize learned filters of each CNN layer. Visualization

approach was used to improve the feature extraction approach by reducing the size of filters.

Similarly, VGG architecture

proposed by the Oxford group, which was runner-up at the 2014-

ILSVRC competition, made the receptive field much small in comparison to that of AlexNet but

with increased volume. In VGG, depth is increased from 9 layers to 16, by making the volume of

features maps double at each layer. In the same year, GoogleNet

that won 2014-ILSVRC

competition, not only exerted its efforts to reduce computational cost by changing layer design,

but also widen the width in compliance with depth to improve CNN performance. GoogleNet

introduced the concept of split, transform and merge blocks, within which multiscale and

multilevel transformation is incorporated to capture both local and global information

27,85,86

. The

use of multilevel transformations help CNN in tackling details of images at various levels. In the

year 2012-14, the main improvement in the learning capacity of CNN was achieved by

increasing its depth and parameter optimization strategies. This suggests that the representation

depth is beneficial in improving the generalization of classifier.

3.5. 2015-Present: Rapid increase in Architectural Innovations and Applications

of CNN

The major improvements in CNN performance were occurred from 2015-2018. The research in

CNN is still on going and it has a significant potential of improvement. Representational depth

improves generalization by defining diverse level of features ranging from simple to complex.

Multiple levels of transformation make learning easy by chopping complex problems into

smaller modules. However, the main challenge faced by deep architectures is the problem of

negative learning, which occurs due to diminishing gradient at lower layers of the network. To

handle this problem, different research groups worked on readjustment of layers connections and

designing of new modules. In earlier 2015, Srivastava et al.

used the concept of cross-channel

connectivity and information gating mechanism

88,89

to solve the vanishing gradient problem and

to improve the network representational capacity. This idea got famous in late 2015 and a similar

concept of residual blocks or skip connections was coined

. Residual blocks are a variant of

cross-channel connectivity, which smoothen learning by regularizing the flow of information

across blocks

90–92

. This idea was used by ResNet architecture (proposed by Microsoft)

for the

training of 150 layers deep network. The idea of cross-channel connectivity is further extended

to multilayer connectivity by different researchers to improve representation

93,94

(resulting in

Deluge, DenseNet etc).

In the year 2016, the width of the network was also explored in connection with depth to

improve feature learning stage

28,30

. Apart from this, no new architectural modification became

prominent but instead, different researchers used hybrid of the already proposed architectures to

improve deep CNN performance

27,90–92,95

. This fact gave the intuition that there might be other

factors more important as compared to the appropriate assembly of the network units that can

effectively regulate CNN performance. In this regard, Hu et al. (2017)

identified that the

network representation has a role in learning of deep CNNs. Hu et al. introduced the idea of

feature map exploitation and pinpointed that less informative and domain extraneous features

affect the performance of the network to a larger extent. Hu et al.

exploited the aforementioned

剩余61页未读，继续阅读

zhang_zhen_yu

粉丝: 0
资源: 2

深度卷积网络架构研究：创新与分类

Convolutional Neural Networks in Visual Computing A Concise Guide 无水印原版pdf

CNN综述 A Survey of the Recent Architectures of Deep Convolutional Neural Networks

《A Survey of the Recent Architectures of Deep Convolutional Neural Networks》

Hands-On Convolutional Neural Networks with TensorFlow

A Semi-Supervised Assessor of Neural Architectures.pdf

An Introduction to YOLOv8: The Evolutionary Journey of Convolutional Neural Networks

Practical Convolutional Neural Networks Implement advanced d l models using Py

Fast Object Detection in 3D Point Clouds Using Convolutional Neural Networks

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Deep Neural Networks in a Mathematical Framework-Springer(2018).pdf

最新资源