IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 14, NO. 10, OCTOBER 2018 4665
Deep Learning for Smart Industry: Efficient
Manufacture Inspection System
With Fog Computing
Liangzhi Li , Student Member, IEEE, Kaoru Ota , Member, IEEE,
and Mianxiong Dong
, Member, IEEE
Abstract—With the rapid development of Internet of
things devices and network infrastructure, there have been
a lot of sensors adopted in the industrial productions, re-
sulting in a large size of data. One of the most popular ex-
amples is the manufacture inspection, which is to detect
the defects of the products. In order to implement a robust
inspection system with higher accuracy, we propose a deep
learning based classification model in this paper, which can
find the possible defective products. As there may be many
assembly lines in one factory, one huge problem in this sce-
nario is how to process such big data in real time. Therefore,
we design our system with the concept of fog computing. By
offloading the computation burden from the central server
to the fog nodes, the system obtains the ability to deal with
extremely large data. There are two obvious advantages in
our system. The first one is that we adapt the convolutional
neural network model to the fog computing environment,
which significantly improves its computing efficiency. The
other one is that we work out an inspection model, which
can simultaneously indicate the defect type and its degree.
The experiments well prove that the proposed method is
robust and efficient.
Index Terms—Deep learning, fog computing, manufac-
ture inspection, smart industry.
I. INTRODUCTION
I
NDUSTRY 4.0, or in other words, the smart industry, is a
word representing the current trend of manufacturing revo-
lution [1]. Two most important concepts in this smart industry
era are automation and data. The former one is one of our main
objectives and the latter one is one of our most useful tools.
Data can be analyzed and learned by some artificial intelligence
(AI) methods, deep learning as an example, and empower the
computers and manipulators with humanlike abilities. In order
to collect more data, which is essential for the AI approaches,
more and more Internet of things (IoT) [2] enabled devices are
Manuscript received April 16, 2018; accepted May 9, 2018. Date
of publication June 1, 2018; date of current version October 3, 2018.
This work was supported in part by the JSPS KAKENHI under Grant
JP16K00117 and in part by the KDDI Foundation. Paper no. TII-18-0932.
(Corresponding author: Mianxiong Dong.)
The authors are with the Department of Information and Electronic
Engineering, Muroran Institute of Technology, Muroran 050-8585,
Japan (e-mail: 16096502@mmm.muroran-it.ac.jp; ota@mmm.muroran-
it.ac.jp; mxdong@mmm.muroran-it.ac.jp).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TII.2018.2842821
deployed in smart factories [3]. Using these data, people have
come up with lots of new ways to make the manufacturing pro-
cess more automated and efficient. In t his paper, we focus on the
autonomous manufacturing inspection, which is an area with a
long history but still has many problems currently, especially in
the big data era.
One serious problem existing in this task is that current in-
spection system cannot guarantee good performances while
keeping the processing efficiency. The traditional methods, such
as filters or some feature-based classification approaches, are
simple but not so effective in all the scenarios. Then, the deep
learning based methods [4] turned up, bringing greatly improved
analysis and recognition abilities, however, at the same time,
slowing down the running speed, as these methods usually re-
quire a lot of computation [5], [6].
It may not be a serious problem when the data size is not so
large. But with the rapid development of smart industry, people
invest their maximum effort on the manpower reduction by de-
ploying vision sensors [7], [8] in each manufacturing line and
empowering them with the ability of autonomous defect detec-
tion, which results in a hugely increased data size. Considering
the production capacity, the computing efficiency has become
the bottleneck to implement a real-time inspection system f or
smart industries.
Computation offloading is important to improve the comput-
ing efficiency and build a r eal-time system [9], and the newly
developed fog computing [10] can serve as a good solution in
this scenario. In this paper, we design our manufacture inspec-
tion system, named DeepIns, as three modules, i.e., the fog-side
computing module, the backend communication module, and
the server side computing module. The two computing modules
are to calculate the deployed deep models, and the communica-
tion module is responsible for the data exchanges and command
transfer.
We focus on the combination of these three modules. In order
to further decrease the response latency and network traffic,
we design the fog-side computing module with an early-exit
feature, which can stop the inference process and obtain the
classification results in advance.
The application scenario is shown in Fig. 1; in some places
of the production line, which usually move along one side,
several sensors or cameras are deployed to capture the visual
information regarding the products. Then, the information will
1551-3203 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications
standards/publications/rights/index.html for more information.