Probabilistic Graph and Hypergraph Matching
Ron Zass and Amnon Shashua
School of Computer Science and Engineering
The Hebrew University of Jerusalem
zass,shashua@cs.huji.ac.il
Abstract
We consider the problem of finding a matching between
two sets of features, given complex relations among them,
going beyond pairwise. Each feature set is modeled by
a hypergraph where the complex relations are represented
by hyper-edges. A match between the feature sets is then
modeled as a hypergraph matching problem. We derive
the hyper-graph matching problem in a probabilistic setting
represented by a convex optimization. First, we formalize a
soft matching criterion that emerges from a probabilistic in-
terpretation of the problem input and output, as opposed to
previous methods that treat soft matching as a mere relax-
ation of the hard matching problem. Second, the model in-
duces an algebraic relation between the hyper-edge weight
matrix and the desired vertex-to-vertex probabilistic match-
ing. Third, the model explains some of the graph match-
ing normalization proposed in the past on a heuristic ba-
sis such as doubly stochastic normalizations of the edge
weights. A key benefit of the model is that the global op-
timum of the matching criteria can be found via an iterative
successive projection algorithm. The algorithm reduces to
the well known Sinkhorn [15] row/column matrix normal-
ization procedure in the special case when the two graphs
have the same number of vertices and a complete matching
is desired. Another benefit of our model is the straightfor-
ward scalability from graphs to hyper-graphs.
1. Introduction
Object representations through graph and hypergraph is
very useful and popular for applications of tracking, query
to image database matching and visual recognition in gen-
eral. For an overview of recent literature, we refer the reader
to a number of special issues that appeared on the subject
[1, 2, 3] and the survey article [7].
A hypergraph representation allows us to exploit model
relationships between different objects, or between object
features under consideration. Normally nodes (vertices)
represent features, object parts or elementary units of con-
sideration for the task at hand, and hyper-edges represent
a relationship among a tuple of nodes. In a graph, the re-
lationship is pairwise in nature (as edges are defined by
pairs of nodes), whereas in a hypergraph the relationships
are multi-faceted involving multiple features at a time.
The hypergraph matching problem seeks a vertex-to-
vertex mapping between two hypergraphs such that the
overall discrepancy between the corresponding matching
hyper-edges is minimized. Since the number of nodes in
the two hypergraphs are not necessarily equal, the matching
problem includes finding an optimal matching sub-graph
under the considerations above. For example, in an image
tracking problem, we would like to find the largest subset
of pixels in the tracked area of consideration which pre-
serves as much as possible the inter-pixel relations. Like-
wise, in an object recognition task, objects can undergo
certain global transformations (such as affine) followed by
some non-rigid deformations, the valence d = n + 1 (num-
ber of vertices in a hyper-edge) of the hyper-edge commen-
surate with the minimal number n of points required for re-
solving the global transformation. A hyper-graph matching
then seeks an alignment between the query and an image in
which a large proportion of hyper-edges agree on the same
or similar global transformation.
On the other hand, there exist a number of problems with
the use of graph matching. First, we notice the high compu-
tational complexity of many operations on graphs. For ex-
ample, computing the similarity of two graphs is typically
exponential in the number of nodes of the two graphs in-
volved. Secondly, the repository of algorithmic procedures
in the graph domain is quite limited when compared to the
tools available for alternative representations such as vector,
bag-of-features, and so forth.
In this paper we derive the hyper-graph matching prob-
lem in a probabilistic setting represented by a convex op-
timization. First, we formalize a soft matching criterion
that emerges from a probabilistic interpretation of the prob-
lem input and output, as opposed to previous methods that
treat soft matching as a mere relaxation of the hard match-
ing problem. Second, the model induces an algebraic rela-
1