53
A Color-based Particle Filter
Katja Nummiaro
1
, Esther Koller-Meier
2
and Luc Van Gool
1,2
1
Katholieke Universiteit Leuven, ESAT/PSI Visics, Belgium
2
Swiss Federal Institute of Technology (ETH), D-ITET/BIWI, Switzerland
Abstract — Robust real-time tracking of non-rigid objects
is a challenging task. Particle filtering has been proven very
successful for non-linear and non-Gaussian estimation prob-
lems. However, for the tracking of non-rigid objects, the
selection of reliable image features is also essential.
This paper presents the integration of color distributions
into particle filtering, which has typically used edge-based
image features. Color distributions are applied as they are
robust to partial occlusion, are rotation and scale invariant
and computationally efficient. Thus, the target model of
the particle filter is defined by the color information of the
tracked object. As the tracker should find the most probable
sample distribution, the model is compared with the current
hypotheses of the particle filter using the Bhattacharyya
coefficient, which is a popular similarity measure between
two distributions. The proposed tracking method directly
incorporates the scale and motion changes of the objects.
Comparisons with the well known mean shift tracker show
the advantages and limitations of the new approach.
Keywords— Object tracking, Condensation algorithm,
Color filtering, Bhattacharyya coefficient, Mean shift track-
ing
I. Introduction
Object tracking is required by many vision applications
such as human-computer interfaces [1], video communica-
tion/compression [2] or surveillance [3], [4], [5]. In this con-
text, particle filters provide a robust tracking framework
as they are neither limited to linearsystemsnorrequire
the noise to be Gaussian. They have typically been used
with edge-based image features [6], [7], [8]. The idea of a
particle filter – to apply a recursive Bayesian filter based
on sample sets – was independently proposed by several
research groups [7], [9], [10]. Our work has evolved from
the Condensation algorithm [7] which was developed in the
computer vision community. At the same time this filtering
method was studied both for statistics and signal process-
ing known as Bayesian bootstrap filter [9] or Monte Carlo
Filter [10].
We propose to use such a particle filter with color-based
image features. Color histograms in particular have many
advantages for tracking non-rigid objects as they are robust
to partial occlusion, are rotation and scale invariant and are
calculated efficiently. A target model is tracked with a par-
ticle filter by comparing its histogram with the histograms
of the sample positions using the Bhattacharyya distance.
A complete segmentation of the image is not required as
the image content only needs to be evaluated at the sample
positions. Figure 1 shows an application of the color-based
particle filter for tracking the face of a soccer player.
A related approach is the mean shift tracker [11] which
also uses color distributions. By employing multiple hy-
potheses and a model of the system dynamics our proposed
E-mail: knummiar@esat.kuleuven.ac.be .
Fig. 1. A color-based target model and the different hypotheses
(black ellipses) that are calculated with the particle filter. The white
ellipse represents the expected object location.
method can track objects more reliably in cases of clutter
and occlusions. In comparison to edge-based particle fil-
ters [6], [7], [8], our target model is more robust against
out-of-plane rotations. In [12] color information has been
used in particle filtering for initialization and importance
sampling. Furthermore, in a recent paper [13], color cues
have been employed in a foreground and background model
using Gaussian mixtures. Our target model has the ad-
vantage of matching only objects that have a similar his-
togram, whereas for Gaussian mixtures objects that con-
tain one of the colors of the mixture will already match.
The outline of this paper is as follows. In Section II we
briefly describe particle filtering and in Section III we indi-
cate how color distributions are used as object models. The
integration of the color information into the particle filter is
explainedinSectionIV. Astracked objects may disappear
and reappear an initialization based on an appearance rule
is introduced in Section V. Section VI compares the mean
shift tracker [11] with our proposed tracking framework as
both methods use related color features as object models.
In Section VII we then present some experimental results
for surveillance and soccer scenes.
II. Particle Filtering
Particle filtering [7] was developed to track objects in
clutter, in which the posterior density p(X
t
|Z
t
)andthe
observation density p(Z
t
|X
t
) are often non-Gaussian. The
quantities of a tracked object are described in the state
vector X
t
while the vector Z
t
denotes all the observations
{z
1
,...,z
t
} up to time t.