1
Abstract
We propose an adaptive figure-ground classification
algorithm to automatically extract a foreground region
using a user-provided bounding-box. The image is first
over-segmented with an adaptive mean-shift algorithm,
from which background and foreground priors are esti-
mated. The remaining patches are iteratively assigned
based on their distances to the priors, with the foreground
prior being updated online. A large set of candidate seg-
mentations are obtained by changing the initial foreground
prior. The best candidate is determined by a score function
that evaluates the segmentation quality. Rather than using
a single distance function or score function, we generate
multiple hypothesis segmentations from different combi-
nations of distance measures and score functions. The final
segmentation is then automatically obtained with a voting
or weighted combination scheme from the multiple hy-
potheses. Experiments indicate that our method performs
at or above the current state-of-the-art on several datasets,
with particular success on challenging scenes that contain
irregular or multiple-connected foregrounds. In addition,
this improvement in accuracy is achieved with low com-
putational cost.
1. Introduction
Foreground extraction in still images plays a key role in
vision applications [1]. Popular approaches include inter-
active graph cut [2], random walk [3], geodesic [4], in-
formation theory [5], and variational solutions [6]. On the
one hand, we are looking for better interactive approaches
that provide a priori knowledge to guide segmentation.
Bounding box assignment and seed positioning are two
representative methods [7,8]. On the other hand, we desire
simple models that free users from troublesome algorithm
design. The uncertainty of model selection and goodness
evaluation makes robust segmentation difficult [9,10].
Different models lead to different results and there exists no
dominant winner [11]. Recent attempts report encouraging
results through the aid of reference distributions or multiple
hypotheses [12,13], although widely applicable solution in
the absence of a priori knowledge remains a big challenge.
(a)original & mask (b)initial patches (c)adaptive patches
...... ......
(d) D
K
candidates (e) D
M
candidates
(f) D
K
multiple hypotheses (g) D
M
multiple hypotheses
(h) similarity voting result (i) probability map result
Figure 1. Adaptive figure-ground classification pipeline
1st row block: Adaptive mean-shift patches generation;
2nd row block: Multiple candidates from multiple initializations;
3rd row block: Multiple hypotheses by eight evaluation scores;
4th row block: Two Automatic selection results.
The possibility of foreground extraction from a box in-
put was explored in recent work [7,12,14,15]. However,
these methods suffer from restrictive assumption about
latent distributions [12], inability to treat complicated scene
topologies [14], or inefficient similarity measure [15]. In
this paper, we propose a box-based foreground extraction
method that gives promising solutions in a broadly appli-
Adaptive Figure-Ground Classification
Yisong Chen
1
Antoni B. Chan
2
1
Graphics Laboratory, EECS Department
Key Laboratory of Machine Perception
(MOE), Peking University
chenys@graphics.pku.edu.cn
2
Video, Image, and Sound Analysis Lab (VISAL)
Department of Computer Science
City University of Hong Kong
abchan@cityu.edu.hk