Super-resolution Reconstruction for License Plate Image in
Video Surveillance System
Xiaole Yan, Qiu Shen, Xin Liu
Nanjing University of Aeronautics and Astronautics
College of Astronautics
Nanjing, P. R. China
yanxiaole0401@foxmail.com
shenqiu@nuaa.edu.cn
liuxinstar1984@gmail.com
Abstract
—
In order to improve the recognition of license plate
texts in the real traffic surveillance video, super-resolution
reconstruction (SR) method is applied to reconstruct a
high-resolution (HR) image from consecutive frames in the video
sequence. Current SR methods almost focus on small local
translation and rotation, which limits the reconstruction of fast
moving or significantly zooming objects, such as license plate
images in surveillance video. In this paper, we combine
Fourier-Mellin transform (FMT) and Vandewalle’s algorithm into
a new technique to improve the registration accuracy of license
plate image. FMT is introduced for scaling estimation, while
Vandewalle’s algorithm is utilized for rotation and translation
estimation. Additionally, the image reconstruction is carried out
by the projection onto convex sets (POCS) method. The
experiments on simulated and real image sequences are carried
out respectively, and the results demonstrate that our approach
can achieve better performance on reconstructing a HR license
plate image.
Keywords
—
super-resolution reconstruction; image registration;
license plate image; surveillance video
I. INTRODUCTION
As a crucial part of urban Intelligent Transport System
(ITS), vehicle license plate recognition system is widely
applied in traffic surveillance, vehicle location and parking
lots management. However, due to the limitations in imaging
hardware equipment and bad weather conditions, the obtained
video images are typically of relatively low quality, causing it
difficult to satisfy the requirements of direct license plate
recognition [1]. Super-resolution reconstruction (SR) is a
process to reconstruct a high-resolution (HR) image from
single or multiple low-resolution (LR) observations. Thus, SR
method can increase the spatial resolution by combining
information from multiple frames in the video, so as to
improve the recognition accuracy of license plate image [2].
SR methods usually consist of two main steps: image
registration and image reconstruction. On one hand,
registration precision is critical for reconstructing a HR image,
so subpixel-level registration accuracy is a basic requirement
for a good reconstruction. Current SR registration algorithms
can be divided into two classes, spatial domain methods and
frequency domain methods, the overview of these methods is
presented in [3]. On the other hand, temporal image
reconstruction models include non-uniform interpolation
method, maximum posterior probability (MAP) method,
iterative back-projection (IBP) method, projection onto
convex sets (POCS) method, etc. [4-5]. Since the effect of
different registration and reconstruction algorithms varies in
different situations, it is of vital importance to choose a best
suitable SR framework for the particular real application.
Compared with common image SR problems, license plate
images in surveillance video have the following
characteristics [6-7]. Firstly, translation, rotation and scaling
motion may occur simultaneously between video frames, and
the movements may be more than one pixel. Secondly, the
license plate region has sharp edges, and the license plate
characters have distinct structural features. Therefore, a
proper SR method for license plate image is imperative.
According to the above characteristics, Vandewalle’s
algorithm [8] is selected for image registration due to its
validity and high precision. Unfortunately, the assumption of
planar translation and rotation motion model makes it
unsuitable for the SR of license plate images, which always
have significantly scaling. To solve this problem,
Fourier-Mellin transform (FMT) [9] is introduced for scaling
estimation. The new proposed method combines the
advantages of FMT and Vandewalle’s algorithm, so that it can
accommodate translation, rotation and scaling estimation
simultaneously while guarantee the subpixel registration
accuracy. For image reconstruction, POCS is employed to
meet the requirements of flexibility and low complexity in
real time license plate image recognition.
This paper is organized as follows. In Sec. II, we introduce
the framework of SR for license plate image from surveillance
video. In Sec. III, we present our proposed algorithm of
frequency domain registration. Sec. IV shows the results on
simulated and real images and the comparison to other
algorithms. A brief conclusion is given in Section. V.
II. THE FRAMEWORK OF SUPER-RESOLUTION
RECONSTRUCTION FOR LICENSE PLATE IMAGE
License plate recognition is a significant requirement in
ITS, but image resolution seriously influences the recognition
accuracy. However, a license always appears in several
sequential frames in surveillance video, which may supply
more information to increase the resolution of license plate
image. As shown in Fig. 1, four sequential frames taken from
surveillance video have strong correlation with each other, but
they display the license in different time with different
position and pose. Consequently, a higher resolution image
can be reconstructed from the four frames by SR method.