NON-RIGID IMAGE DEFORMATION ALGORITHM BASED ON MRLS-TPS
Huabing Zhou
1
, Yuyu Kuang
1
, Zhenghong Yu
2
, Shiqiang Ren
1
, Anna Dai
1
, Yanduo Zhang
1
,Tao Lu
1
,Jiayi Ma
3
1
Hubei Provincial Key Laboratory of Intelligent Robot, Computer Science and Engineering School,
Wuhan Institute of Technology, Wuhan 430205, China
2
Guangdong Polytechnic of Science and Technology, Zhuhai 519090, China
3
Electronic Information School, Wuhan University, Wuhan 430072, China
ABSTRACT
In this paper, we propose a novel closed-form transforma-
tion estimation method based on moving regularized least
squares optimization with thin-plate spline (MRLS-TPS) for
non-rigid image deformation. The method takes the user-
controlled point-offset-vectors as the input data, and estimates
the spatial transformation about the two control point sets for
each pixel. To achieve a realistic deformation, we formulates
the transformation estimation as a vector-field interpolation
problem by a moving regularized least squares method. Un-
like MLS, the mapping function is modeled by a non-rigid
function thin-plate spline with regularization technique, such
that the deformation can satisfy both global linear affine mo-
tion and local non-rigid warping. We derive a closed-form
solution of the transformation and achieve a fast implemen-
tation. In addition, the proposed method can give a wonder-
ful user experience, fast and convenient manipulating. Exten-
sive experiments on real images demonstrated the proposed
method outperforms other state-of-the-art methods and the
commercial software Adobe PhotoShop CS 6, especially in
case of flexible object motion.
Index Terms— Deformation; moving regularized least-
squares method; thin-plate spline function
1. INTRODUCTION
Image deformation, which aims to render new visual effects
via geometric transformation and pixel interpolation, is de-
veloped upon computer graphics and image processing. It
has a number of useful applications ranging from medical
imaging, movie effects, virtual reality, criminal detection to
animation, structure from motion, face synthesis and analy-
sis [1, 2, 3, 4, 5, 6, 7, ?, ?, 10]. The deformation is typically
controlled by the user-selected handles. As the user modifies
The authors gratefully acknowledge the financial supports from the
National Natural Science Foundation of China under Grants 41501505,
61503288 and 61502354, the PhD Start-up Fund of Guangdong Natural Sci-
ence Foundation under Grant 2016A030310306, the Natural Science Fund of
Hubei province under Grant 2014CFA130 and the Hubei province scientific
and technological research project under Grant Q20151504.
the position of these handles, the image should deform in an
intuitive fashion.
Some work has focused on specifying deformations us-
ing different types of handles. These handles take the forms
of points, lines, and even polygon grids [11]. Alternatively,
some methods investigate the type of transformations to per-
form the desirable deformation. These methods view the
problem as scattered data interpolation, and the goal is to gen-
erate a dense correspondence based on a set of user-controlled
handles. Bookstein et al. [12] used thin-plate splines to
compute deformation fields corresponding to scattered data
samples (manually controlled by the user). Lee et al. [13]
computed the B-spline basis functions and weights for scat-
tered data interpolation, and proposed a method for adaptively
varying the resolution of the control point lattice to avoid a
very large number of basis functions. Schaefer [1] proposed
to deform images based on Moving Least Squares (MLS) [14]
using linear functions such as rigid transformation. The use
of MLS and rigid formations makes the deformation ‘as-rigid-
as-possible’ [15].
The methods mentioned above model the deformation
with rigid transformation. However, some deformation does
not have the property of piecewise rigid, such as the waving
flag and the change of expression. We need to use non-rigid
model to solve this problem. The spline tool TPS [16] has
a close-form solution which can be decomposed into global
linear affine motion and local non-rigid warping component
controlled by coefficients. It can produces a smooth func-
tional mapping for supervised learning and has no free pa-
rameters that need manual tuning. To generate a smooth
mapping fitting for the control points, we choose the TPS for
parametrization.
Therefore, we formulate the transformation estimation as
a vector-field interpolation problem by a moving regular least
squares method with thin-plate spline. We name it a mov-
ing regularized least squares method with the thin-plate spline
(MRLS-TPS). The contribution in this paper includes the fol-
lowing two aspects. Firstly, we introduce the TPS function
and regularization technology to the deformation problem,
which can help to get more realistic deformation. Secondly,
,((( ,&,3