“GrabCut” — Interactive Foreground Extraction using Iterated Graph Cuts
Carsten Rother
∗
Vladimir Kolmogorov
†
Microsoft Research Cambridge, UK
Andrew Blake
‡
Figure 1: Three examples of GrabCut. The user drags a rectangle loosely around an object. The object is then extracted automatically.
Abstract
The problem of efficient, interactive foreground/background seg-
mentation in still images is of great practical importance in im-
age editing. Classical image segmentation tools use either texture
(colour) information, e.g. Magic Wand, or edge (contrast) infor-
mation, e.g. Intelligent Scissors. Recently, an approach based on
optimization by graph-cut has been developed which successfully
combines both types of information. In this paper we extend the
graph-cut approach in three respects. First, we have developed a
more powerful, iterative version of the optimisation. Secondly, the
power of the iterative algorithm is used to simplify substantially the
user interaction needed for a given quality of result. Thirdly, a ro-
bust algorithm for “border matting” has been developed to estimate
simultaneously the alpha-matte around an object boundary and the
colours of foreground pixels. We show that for moderately difficult
examples the proposed method outperforms competitive tools.
CR Categories: I.3.3 [Computer Graphics]: Picture/Image
Generation—Display algorithms; I.3.6 [Computer Graphics]:
Methodology and Techniques—Interaction techniques; I.4.6 [Im-
age Processing and Computer Vision]: Segmentation—Pixel clas-
sification; partitioning
Keywords: Interactive Image Segmentation, Graph Cuts, Image
Editing, Foreground extraction, Alpha Matting
1 Introduction
This paper addresses the problem of efficient, interactive extrac-
tion of a foreground object in a complex environment whose back-
ground cannot be trivially subtracted. The resulting foreground ob-
ject is an alpha-matte which reflects the proportion of foreground
and background. The aim is to achieve high performance at the
cost of only modest interactive effort on the part of the user. High
performance in this task includes: accurate segmentation of object
from background; subjectively convincing alpha values, in response
to blur, mixed pixels and transparency; clean foreground colour,
∗
e-mail: carrot@microsoft.com
†
e-mail: vnk@microsoft.com
‡
e-mail: ablake@microsoft.com
free of colour bleeding from the source background. In general,
degrees of interactive effort range from editing individual pixels, at
the labour-intensive extreme, to merely touching foreground and/or
background in a few locations.
1.1 Previous approaches to interactive matting
In the following we describe briefly and compare several state of
the art interactive tools for segmentation: Magic Wand, Intelligent
Scissors, Graph Cut and Level Sets and for matting: Bayes Matting
and Knockout. Fig. 2 shows their results on a matting task, together
with degree of user interaction required to achieve those results.
Magic Wand starts with a user-specified point or region to com-
pute a region of connected pixels such that all the selected pixels
fall within some adjustable tolerance of the colour statistics of the
specified region. While the user interface is straightforward, finding
the correct tolerance level is often cumbersome and sometimes im-
possible. Fig. 2a shows the result using Magic Wand from Adobe
Photoshop 7 [Adobe Systems Incorp. 2002]. Because the distri-
bution in colour space of foreground and background pixels have a
considerable overlap, a satisfactory segmentation is not achieved.
Intelligent Scissors (a.k.a. Live Wire or Magnetic Lasso)
[Mortensen and Barrett 1995] allows a user to choose a “minimum
cost contour” by roughly tracing the object’s boundary with the
mouse. As the mouse moves, the minimum cost path from the cur-
sor position back to the last “seed” point is shown. If the computed
path deviates from the desired one, additional user-specified “seed”
points are necessary. In fig. 2b the Magnetic Lasso of Photoshop 7
was used. The main limitation of this tool is apparent: for highly
texture (or un-textured) regions many alternative “minimal” paths
exist. Therefore many user interactions (here 19) were necessary to
obtain a satisfactory result. Snakes or Active Contours are a related
approach for automatic refinement of a lasso [Kass et al. 1987].
Bayes matting models colour distributions probabilistically to
achieve full alpha mattes [Chuang et al. 2001] which is based on
[Ruzon and Tomasi 2000]. The user specifies a “trimap” T =
{T
B
,T
U
,T
F
} in which background and foreground regions T
B
and
T
F
are marked, and alpha values are computed over the remain-
ing region T
U
. High quality mattes can often be obtained (fig.
2c), but only when the T
U
region is not too large and the back-
ground/foreground colour distributions are sufficiently well sepa-
rated. A considerable degree of user interaction is required to con-
struct an internal and an external path.
Knockout 2 [Corel Corporation 2002] is a proprietary plug-in for
Photoshop which is driven from a user-defined trimap, like Bayes
matting, and its results are sometimes similar (fig. 2d), sometimes
of less quality according to [Chuang et al. 2001].