0278-0046 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2019.2962437, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
An Efficient CNN Model Based on Object-level
Attention Mechanism for Casting Defects
Detection on Radiography Images
Chuanfei Hu and Yongxiong Wang
Abstract—Automatic detection of casting defects on ra-
diography images is an important technology to automatize
digital radiography (DR) defect inspection. Traditionally, in
an industrial application, conventional methods are ineffi-
cient when the detection targets are small, local and subtle
in the complex scenario. Meanwhile, the outperformance of
deep learning models, such as the convolution neural net-
work (CNN), is limited by a huge volume of data with precise
annotations. To overcome these challenges, an efficient
CNN model, only trained with image-level labels, is first pro-
posed for detection of tiny casting defects in a complicated
industrial scene. Then, we present a novel training strategy
which can form a new object-level attention mechanism for
the model during the training phase, and bilinear pooling
is utilized to improve the model capability of detecting
local contrast casting defects. Moreover, to enhance the
interpretability, we extend class activation maps (CAM) to
bilinear CAM (Bi-CAM) which is adapted to bilinear archi-
tectures as a visualization technique to describe the reason
about the model output. Experimental results show that the
proposed model achieves superior performance in terms
of each quantitative metric and is suitable for most actual
applications. The real-time defect detection of castings is
efficiently implemented in the complex scenario.
Index Terms—Bilinear CAM, bilinear pooling, convolu-
tional neural network, digital radiography defect inspection,
object-level attention mechanism.
I. INTRODUCTION
A
LUMINIUM castings have been extensively applied to
various parts of automobile [1], aerospace [2], aircraft
[3] and electric devices [4] whose qualities affect the fatigue
behavior of overall product [5]. Due to the complexity and
diversity of casting process [6], defects are inevitable during
the process, such as gas holes, sand holes and flaws. And these
defects cannot be recognized by surface detection technology
when the defects have existed in the interior of castings.
In order to obtain the internal information and guarantee
the completeness of castings, radiography is often used for
nondestructive testing [7] which has been widely used in
quality controlling [8] and security inspection system [9].
Manuscript received August 06, 2019; revised November 01, 2019;
accepted December 12, 2019. This work was supported in part by the
National Natural Science Foundation of China under Grant 61673276.
(Corresponding author: Yongxiong Wang.)
C. Hu and Y. Wang are with Department of Control Science and Engi-
neering, University of Shanghai for Science and Technology, Shanghai
200093, China (w64228013@126.com; wyxiong@usst.edu.cn). They
contributed equally to this work.
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
( )
=
riu2
RP
LBP
,
195
25
25
Unrelated pallet
Unrelated background
Unrelated label Related casting Selected patch
Fig. 1. On the left, the various regions of the radiography images are
masked by different colors. The result reveals the complicacy of the
scenario with various structures and positions of castings. Besides, we
extract uniform LBP features of the image patches which are selected
from related or unrelated areas. On the right, the Chi-square distances
of the LBP histograms are computed for the defect patch (in the first row
of the table) to the others. The comparison shows that the features of
defects are appeared in the other local regions. The locality attribute
of casting defects is revealed. Uniform LBP algorithm is denoted as
LBP
riu2
P,R
(·) where P represents the number of local neighbors (P = 8)
on a circle of radius R (R = 2).
In radiographic testing processing, most manufacturers rely
mainly on manual detection according to the experiences of
operators about the shape, brightness and contrast of castings.
Such manual method is not only mechanical and inefficient,
but also may cause ophthalmic diseases due to the high-
frequency illumination of the display. Consequently, the digital
radiography (DR) based automatic inspection system has been
one of the research focuses.
Over the past decades, conventional computer vision tech-
nology has extensive applications in many automatic inspec-
tion systems [10]–[13]. Handcrafted features and statistic
based machine learning models are often the main tools,
which use image algorithms to generate the feature vectors
of texture, color, shape and spectral cues, and then adopt sta-
tistical machine learning model to realize inspection systems.
With the advantage of simply and effective models, machine
learning methods achieve satisfactory performances in most
applications. However, the general drawback of conventional
methods is that their performances depend on the effective
representation extracted by handcrafted feature algorithms.
Designing a reliable and robust representation may be not