X.W. Ye, T. Jin and C.B. Yun
The milestone success of AlexNet shocked scholars and
engineers all over the world and attracted more attention to
the research on deep learning. Up to now, a lot of DNNs
have been proposed for many kinds of application purposes.
CapsuleNet is able to recognize and reconstruct target
objects in images (Hinton et al. 2011, Sabour et al. 2017).
VGG-Net (Simonyan and Zisserman 2014), ZF-Net (Zeiler
and Fergus 2014), GoogLeNet (Szegedy et al. 2014) and
ResNet (He et al. 2016) are good at classification. U-Net
(Ronneberger et al. 2015), DeconvNet (Noh et al. 2015),
CRF-RNN (Zheng et al. 2015), ENet (Paszke et al. 2016),
PSPNet (Zhao et al. 2017), RefineNet (Lin et al. 2017),
fully convolutional network (FCN) (Shelhamer et al. 2017),
DenseNet (Huang et al. 2017) and Deeplab (Chen et al.
2018) are suitable for segmentation tasks. R-CNN (Girshick
et al. 2014, Ren et al. 2015), MobileNet (Howard et al.
2017), SegNet (Badrinarayanan et al. 2017) and ShuffleNet
(Zhang et al. 2018) are fit for target detection tasks. GAN
(Goodfellow et al. 2014), f-GAN (Nowozin et al. 2016),
EBGAN (Zhao et al. 2016) and InfoGAN (Chen et al.
2016) could be utilized for imaginary processing of images,
videos, etc. More studies can be found in LeCun et al.
(2015). The historical development of deep learning is
illustrated in Fig. 4.
2.2 Frameworks and datasets for deep learning
Deep learning frameworks are crucial tools for the
application of deep learning-based approaches and have
been developed by many companies and research institutes.
Caffe was proposed by the University of California,
Berkeley in 2013, and it supports CNN well. The
explanations, demos and related papers can be found at
http://caffe.berkeleyvision.org/. Tensorflow is an open
source software developed by Google in 2015, which can
connect well with python and C++. The detailed resource
can be found at https://tensorflow.google.cn/. PyTorch was
developed by Facebook in 2016, and it supports a dynamic
computation graph. Examples and tutorials are available at
https://github.com/pytorch. Besides the above-mentioned
popular frameworks, there are other frameworks. MXNet
was developed by Amazon in 2015, and is available at
http://mxnet.incubator.apache.org/. CNTK was developed
b y M i c r o s o f t i n 2 0 1 6 , a v a i l a b l e a t
https://archive.codeplex.com/?p=cntk. PaddlePaddle was
developed by Baidu in 2016, available at
https://www.paddlepaddle.org.cn/.
The demand for tremendous training data is a big
challenge in the training process. To sufficiently train DNNs
for different tasks, the number of training samples is
counted by tens of thousands. Thus, a variety of datasets
were established to support the training demand. MNIST is
a dataset of handwritten digits containing 60000 training
images and 10000 testing images, available at
https://datahack.analyticsvidhya.com/contest/practice-
problem-identify-the-digits/#data_dictionary. MS-COCO is
a dataset for object detection and segmentation, available at
http://cocodataset.org/#people. WordNet is a large lexical
dataset of English, containing words of nouns, verbs,
adjectives and adverbs, available at
https://wordnet.princeton.edu/. ImageNet is a dataset of
images built based on WordNet to provide the graphical
explanation of each word in the form of synonym sets,
available at http://www.image-net.org/. Open images
dataset contains millions of images covering thousands of
classifications with labeled bounding boxes, available at
https://github.com/openimages/dataset. Wikipedia Corpus
contains words from over 4 million articles and is a
powerful natural language processing dataset, available at
https://nlp.cs.nyu.edu/wikipedia-data/. More datasets of
different categories can be found at
https://www.analyticsvidhya.com/blog/2018/03/comprehens
ive-collection-deep-learning-datasets/. The industrial chain
of deep learning is illustrated in Fig. 5.
3. Applications of deep learning in the SHM of civil
infrastructures
Researchers and engineers in the field of civil
engineering have already noticed the fantastic prospects and
innovative technological strength brought about by deep
learning-based approaches (DeVries et al. 2018, Spencer et
al. 2019). Many kinds of attempts have been made to apply
deep learning-based approaches to the SHM of civil
infrastructures (Vodrahalli and Bhowmik 2017). In this
section, the research work has been collected and mainly
classified into two categories: structural damage detection
and structural condition assessment.