Signal Processing 149 (2018) 118–123
Contents lists available at ScienceDirect
Signal Processing
journal homepage: www.elsevier.com/locate/sigpro
Semi-supervised vehicle classification via fusing affinity matrices
Maojin Sun
a
, Shijie Hao
b
,
∗
, Guangcan Liu
c
a
Transportation Management College, Dalian Maritime University, China
b
School of Computer and Information, Hefei University of Technology, China
c
School of Information and Control, Nanjing University of Information Science and Technology, China
a r t i c l e i n f o
Article history:
Received 29 July 2017
Revised 7 February 2018
Accepted 12 March 2018
Available online 16 March 2018
Keywords:
Vehicle type classification
Semi-supervised learning
Graph fusion
a b s t r a c t
Vehicle classification plays a fundamental role in various intelligent transportation systems. With the
rapid development of traffic surveillance, the amount of visual vehicle data has been increasing tremen-
dously, and can be easily collected. However, it is labor-intensive to manually annotate the semantic
labels for these data, posing the challenge of label insufficiency to the vehicle classification tasks. In this
context, we use a semi-supervised learning model to classify vehicle types, which only needs a small
number of pre-labeled data and propagates these labels to the rest data at hand. In our model, we com-
bine multiple features via fusing their affinity matrices to enhance the classification accuracy. We conduct
several experiments to validate our method on a public vehicle dataset. Experimental results support the
effectiveness of our method.
©2018 Elsevier B.V. All rights reserved.
1. Introduction
In nowadays society, the number of vehicles on the road has
been increasing tremendously. The prevalence of visual surveil-
lance network makes it convenient to collect massive vehicle-
related information, which is of great value for fields such as traffic
management, social security, and smart city [1–3] . One of the fun-
damental techniques in these applications is vehicle classification
from visual surveillance data or web images [3–5] .
As a typical visual classification task, a vehicle classification
framework usually has two stages, i.e. feature extraction and classi-
fier construction. Various well-known models for these two stages
can be adopted in this task, such as bag-of-visual-words (BOVW)
and support vector machine (SVM), respectively. However, the mis-
placement of intra- and inter- vehicle appearance variations brings
a challenge to this classification task, which raises a high demand
to the feature representation. For example, as illustrated in Fig. 1 ,
vehicles from a same type often have large visual differences, while
two vehicles from different types can be visually similar. With the
recent success of convolutional neural network (CNN), this prob-
lem seems to be largely relieved as the network builds a good
mapping over the semantic gap [6] . Many CNN-based methods
have been proposed to fulfill various vehicle classification tasks [7–
10]
. In these methods, it is assumed that sufficient labeled training
data are at hand. However, for real-world application, the man-
ual annotation with a large-scale vehicle image dataset is labor-
∗
Corresponding author.
intensive. Without sufficient labeled data, CNN based models tend
to be ineffective as they can suffer a serious over-fitting problem.
Fortunately, the unlabeled vehicle images can be much eas-
ier to obtain from surveillance cameras or the Internet nowadays.
Despite of the lack of labels, sufficient unlabeled data still con-
tains important information, such as data distribution and pair-
wise affinity, which can be beneficial to the learning process. In
this context, the issue of insufficient labeled data can be addressed
by the strategy of semi-supervised learning (SSL). Generally, SSL
model is highlighted in fully utilizing both labeled and unlabeled
data, and thus builds a better classifier than the one only uses lim-
ited labeled data [11] . In this paper, we focus on the task of vehi-
cle type classification. We use Graph-based SSL (GSSL) [12] , one of
the state-of-the-art SSL based classification models, to infer the ve-
hicle type. By taking vehicle images (both labeled and unlabeled)
as nodes, and their visual affinities as edge weights, a graph struc-
ture is constructed to represent the whole dataset. In particular, we
fuse multiple vehicle feature representations at a mid-level dur-
ing the graph construction process. We then use the label propa-
gation strategy to spread the vehicle type labels from the labeled
instances to the unlabeled ones.
The contribution of this research is two-fold. First, we introduce
a feasible technical roadmap, i.e. GSSL, to solve the label insuffi-
ciency issue in the task of vehicle type classification. Second, the
graph fusion process is able to leverage multiple feature represen-
tations under different distance measures. From the experimental
validation, our model can achieve acceptable accuracy ( ∼70%) by
only using 5% of the labels. In the rest of this paper, we briefly in-
troduce the related research in Section 2 . In Section 3 , we describe
https://doi.org/10.1016/j.sigpro.2018.03.006
0165-1684/© 2018 Elsevier B.V. All rights reserved.