深度卷积神经网络最新架构调研

paper

需积分: 27 21 浏览量更新于2024-07-15 收藏 1.02MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"《A Survey of the Recent Architectures of Deep Convolutional Neural Networks》是一篇2019年的计算机视觉领域的研究论文，作者包括Asifullah Khan、Anabia Sohail、Umme Zahoora和Aqsa Saeed Qureshi。该文深入探讨了深度卷积神经网络（CNNs）的最新架构，分析了这些创新设计如何提升模型性能，并且关注了参数优化在CNN设计中的应用。" 本文主要研究了深度卷积神经网络的最新发展，特别是在计算机视觉任务中取得的先进成果。CNNs因其在特征提取上的强大能力而备受关注，它们通过多层隐藏层自动学习数据的表示。随着大量数据的可用性和硬件处理单元的提升，CNN的研究进展迅速，催生了一系列新颖的深度CNN架构。论文中讨论了多种提升CNN性能的设计理念，例如采用不同的激活函数。激活函数是神经网络中至关重要的组成部分，它们为神经元引入非线性，如ReLU（Rectified Linear Unit）、Leaky ReLU和ELU（Exponential Linear Unit），这些不同的激活函数可以增强模型的学习能力和适应性。此外，论文还提到了参数优化对CNN性能的影响。优化技术，如梯度下降、动量优化、Adam（Adaptive Moment Estimation）等，有助于更快地收敛并找到更好的局部极小值。同时，结构优化如残差连接（Residual Connections）和密集连接（Dense Connections）也被证明能有效解决梯度消失和爆炸问题，提升网络的训练效果。论文还可能涵盖了其他创新结构，如Inception模块、ResNeXt、 SENet（Squeeze-and-Excitation Network）等，这些网络设计在提高模型效率的同时，保持甚至提升了准确率。它们通过减少计算复杂度、增加模型深度或改进信息流的方式，增强了模型的表达能力。《A Survey of the Recent Architectures of Deep Convolutional Neural Networks》是对近年来CNN架构的全面综述，对于理解深度学习在计算机视觉领域的发展和进步具有很高的参考价值。通过这篇论文，读者能够了解最新的研究趋势和技术，以及如何通过设计创新和参数优化来提升CNN的性能。

资源详情

资源推荐

3.2 Early 2000: Stagnation of CNN

In the late 1990s and early 2000s, interest in NNs reduced and less attention was given to explore

the role of CNNs in different applications such as object detection, video surveillance, etc. Use

of CNN in ML related tasks became dormant due to the insignificant improvement in

performance at the cost of high computational time. At that time, other statistical methods and, in

particular, SVM became more popular than CNN due to its relatively high performance [66]–

[68]. It was widely presumed in early 2000 that the backpropagation algorithm used for training

of CNN was not effective in converging to optimal points and therefore unable to learn useful

features in supervised fashion as compared to handcrafted features [69]. Meanwhile, different

researchers kept working on CNN and tried to optimize its performance. In 2003, Simard et al.

improved CNN architecture and showed good results as compared to SVM on a hand digit

benchmark dataset; MNIST [64], [68], [70]–[72]. This performance improvement expedited the

research in CNN by extending its application in optical character recognition (OCR) to other

script’s character recognition [72]–[74], deployment in image sensors for face detection in video

conferencing, and regulation of street crimes, etc. Likewise, CNN based systems were

industrialized in markets for tracking customers [75]–[77]. Moreover, CNN’s potential in other

applications such as medical image segmentation, anomaly detection, and robot vision was also

explored [78]–[80].

3.3 2006-2011: Revival of CNN

Deep NNs have generally complex architecture and time intensive training phase that sometimes

spanned over weeks and even months. In early 2000, there were only a few techniques for the

training of deep Networks. Additionally, it was considered that CNN is not able to scale for

complex problems. These challenges halted the use of CNN in ML related tasks.

To address these problems, in 2006 many interesting methods were reported to overcome the

difficulties encountered in the training of deep CNNs and learning of invariant features. Hinton

proposed greedy layer-wise pre-training approach in 2006, for deep architectures, which revived

and reinstated the importance of deep learning [81], [82]. The revival of a deep learning [83],

[84] was one of the factors, which brought deep CNNs into the limelight. Huang et al. (2006)

used max pooling instead of subsampling, which showed good results by learning of invariant

features [46], [85].

In late 2006, researchers started using graphics processing units (GPUs) [86], [87] to accelerate

training of deep NN and CNN architectures [88], [89]. In 2007, NVIDIA launched the CUDA

programming platform [90], [91], which allows exploitation of parallel processing capabilities of

GPU with a much greater degree [92]. In essence, the use of GPUs for NN training [88], [93] and

other hardware improvements were the main factor, which revived the research in CNN. In 2010,

Fei-Fei Li’s group at Stanford, established a large database of images known as ImageNet,

containing millions of labeled images [94]. This database was coupled with the annual ImageNet

Large Scale Visual Recognition Challenge (ILSVRC) competitions, where the performances of

various models have been evaluated and scored [95]. Consequently, ILSVRC and NIPS have

been very active in strengthening research and increasing the use of CNN and thus making it

popular. This was a turning point in improving the performance and increasing the use of CNN.

3.4 2012-2014: Rise of CNN

Availability of big training data, hardware advancements, and computational resources

contributed to advancement in CNN algorithms. Renaissance of CNN in object detection, image

classification, and segmentation related tasks had been observed in this period [9], [96].

However, the success of CNN in image classification tasks was not only due to the result of

aforementioned factors but largely contributed by the architectural modifications, parameter

optimization, incorporation of regulatory units, and reformulation and readjustment of

connections within the network [39], [42], [97].

The main breakthrough in CNN performance was brought by AlexNet [21]. AlexNet won the

2012-ILSVRC competition, which has been one of the most difficult challenges in image

detection and classification. AlexNet improved performance by exploiting depth (incorporating

multiple levels of transformation) and introduced regularization term in CNN. The exemplary

performance of AlexNet [21] compared to conventional ML techniques in 2012-ILSVRC

(AlexNet reduced error rate from 25.8 to 16.4) suggested that the main reason of the saturation in

CNN performance before 2006 was largely due to the unavailability of enough training data and

computational resources. In summary, before 2006, these resource deficiencies made it hard to

train a high-capacity CNN without deterioration of performance [98].

With CNN becoming more of a commodity in the computer vision (CV) ﬁeld, a number of

attempts have been made to improve the performance of CNN with reduced computational cost.

Therefore, each new architecture try to overcome the shortcomings of previously proposed

architecture in combination with new structural reformulations. In year 2013 and 2014,

researchers mainly focused on parameter optimization to accelerate CNN performance in a range

of applications with a small increase in computational complexity. In 2013, Zeiler and Fergus

[28] defined a mechanism to visualize learned filters of each CNN layer. Visualization approach

was used to improve the feature extraction stage by reducing the size of the filters. Similarly,

VGG architecture [29] proposed by the Oxford group, which was runner-up at the 2014-ILSVRC

competition, made the receptive field much smaller in comparison to that of AlexNet but, with

increased volume. In VGG, depth was increased from 9 layers to 16, by making the volume of

features maps double at each layer. In the same year, GoogleNet [99] that won 2014-ILSVRC

competition, not only exerted its efforts to reduce computational cost by changing layer design,

but also widened the width in compliance with depth to improve CNN performance. GoogleNet

introduced the concept of split, transform, and merge based blocks, within which multiscale and

multilevel transformation is incorporated to capture both local and global information [33], [99],

[100]. The use of multilevel transformations helps CNN in tackling details of images at various

levels. In the year 2012-14, the main improvement in the learning capacity of CNN was achieved

by increasing its depth and parameter optimization strategies. This suggested that the depth of a

CNN helps in improving the performance of a classifier.

3.5 2015-Present: Rapid increase in Architectural Innovations and Applications

of CNN

It is generally observed the major improvements in CNN performance occurred from 2015-2019.

The research in CNN is still on going and has a significant potential of improvement.

Representational capacity of CNN depends on its depth and in a sense can help in learning

complex problems by defining diverse level of features ranging from simple to complex.

Multiple levels of transformation make learning easy by chopping complex problems into

剩余66页未读，继续阅读

一个处女座的程序猿

粉丝: 122w+
资源: 59

深度卷积神经网络最新架构调研

A Survey of the Recent Architectures of Deep Convolutional Neural Networks.pdf

Convolutional Neural Networks in Visual Computing A Concise Guide 无水印原版pdf

CNN综述 A Survey of the Recent Architectures of Deep Convolutional Neural Networks

An Introduction to YOLOv8: The Evolutionary Journey of Convolutional Neural Networks

Hands-On Convolutional Neural Networks with TensorFlow

Practical Convolutional Neural Networks Implement advanced d l models using Py

Fast Object Detection in 3D Point Clouds Using Convolutional Neural Networks

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

CRC Networks of the Future Architectures Technologies and Implementations

Networks of the Future Architectures, Technologies, and Implementations azw3

Artificial Neural Networks Architectures and Applications

Introduction to the Math of Neural Networks

Deep Neural Networks in a Mathematical Framework-Springer(2018).pdf

Hyperbolic Neural Networks：Theory, Architectures and Application

A Semi-Supervised Assessor of Neural Architectures.pdf

Deep Learning: Recurrent Neural Networks in Python

Very Deep Convolutional Networks for Large-Scale Image Recognition" by Karen Simonyan and Andrew Zisserman (2014)

最新资源