790 IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 14, NO. 2, FEBRUARY 2018
Deep Convolutional Computation Model for
Feature Learning on Big Data in
Internet of Things
Peng Li, Zhikui Chen , Laurence Tianruo Yang , Qingchen Zhang, and M. Jamal Deen
Abstract—Currently, a large number of industrial data,
usually referred to big data, are collected from Internet of
Things (IoT). Big data are typically heterogeneous, i.e., each
object in big datasets is multimodal, posing a challenging
issue on the convolutional neural network (CNN) that is one
of the most representative deep learning models. In this
paper, a deep convolutional computation model (DCCM) is
proposed to learn hierarchical features of big data by using
the tensor representation model to extend the CNN from
the vector space to the tensor space. To make full use of
the local features and topologies contained in the big data,
a tensor convolution operation is defined to prevent over-
fitting and improve the training efficiency. Furthermore, a
high-order backpropagation algorithm is proposed to train
the parameters of the deep convolutional computational
model in the high-order space. Finally, experiments on three
datasets, i.e., CUAVE, SNAE2, and STL-10 are carried out to
verify the performance of the DCCM. Experimental results
show that the deep convolutional computation model can
give higher classification accuracy than the deep compu-
tation model or the multimodal model for big data in IoT.
Index Terms—Big data, convolutional neural network
(CNN), deep convolutional computation model (DCCM),
high-order backpropagation (HBP) algorithm, Internet of
Things (IoT), tensor computation.
I. INTRODUCTION
I
N recent years, the Internet of Things (IoT) has attracted
much attention from researchers and engineers. IoT in-
tegrates the cyber world and physical world based on their
Manuscript received June 24, 2017; accepted July 29, 2017. Date
of publication August 14, 2017; date of current version February 1,
2018. This work was supported in part by the National Natural Sci-
ence Foundation of China under Grant U1301253, Grant 61672123,
and Grant 61602083, in part by the Fundamental Research Funds for
the Central Universities under Grant DUT2017TB02, and in part by the
Dalian University of Technology Fundamental Research Fund under
Grant DUT15RC(3)100. Paper no. TII-17-1345. (Corresponding author:
Laurence Tianruo Yang.)
P. Li and Z. Chen are with the Key Laboratory for Ubiquitous Net-
work and Service Software of Liaoning Province and the School of Soft-
ware Technology, Dalian University of Technology, Dalian 116023, China
(e-mail: lipeng2015@mail.dlut.edu.cn; zkchen@dlut.edu.cn).
L. T. Yang and Q. Zhang are with the Department of Computer Science,
St. Francis Xavier University, Antigonish, NS B2G 2W5, Canada (e-mail:
ltyang@gmail.com; qzhang@stfx.ca).
M. J. Deen is with the Department of Electrical and Computer Engi-
neering, McMaster University, Hamilton, ON L8S 4L8, Canada (e-mail:
jamal@mcmaster.ca).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TII.2017.2739340
interactions via sensor devices, mobile personal computing, and
advanced communication technologies to provide transparent,
smart, ubiquitous, green, and safe services for humans [1]–[3].
IoT can be viewed as the extension of cyberspace for interven-
tion and operation in dangerous or inaccessible environments
and their coordination [4]. IoT has been widely used in many ap-
plications such as industrial control, intelligent transportation,
and smart medical imaging with a large number of industrial
data generating. Industrial big data are usually heterogeneous;
specifically, each object in industrial big datasets possesses the
attribute of multimodality. For example, a segment of daily life
surveillance video with additional information adopts a large
number of images to display the content and uses temperature
data and spatial data to indicate weather and location. Industrial
big data with complex multimodal characteristics pose an im-
portant challenge for data analyses. Therefore, the data analysis
of big data in IoT requires novel models and technologies.
Deep learning, as a novel machine learning model, utilizes the
supervised/unsupervised method to learn hierarchical features
for the tasks of classification and pattern recognition [5]. The
most well-known deep learning model is the convolutional neu-
ral network (CNN) that is constituted by convolutional layers,
sampling layers, and fully-connected layers [6]. CNN achieves a
state-of-the-art performance in image classification and speech
recognition [7]. However, it is difficult for the CNN to learn
heterogeneous features for industrial big data since those data
are of complex multimodality and high heterogeneity. Specifi-
cally, CNN works in the vector space so it could not reveal the
inherent high-dimensional features of big data.
Over past few years, some multimodal deep learning models
(MDLs) have been developed for big data feature learning such
as the bimodal deep autoencoder model and the multimodal
deep restricted Boltzmann machine [8]–[10]. They typically
learn features from each modality and then connect the learned
features into a single vector as the joint representation of ev-
ery heterogeneous sample. MDLs have made some progress in
heterogeneous data feature learning. However, it is difficult for
them to produce a desirable result s ince they cannot reveal the
hidden correlations over different modalities by connecting the
learned features linearly.
Motivation Tensor: A tensor is a compact way for represent-
ing the heterogeneous data structures and preserving the raw
structures of the data, which allows us to consider the mutuality
and the complementarity between the different modalities [11].
1551-3203 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications
standards/publications/rights/index.html for more information.