![](https://csdnimg.cn/release/download_crawler_static/18442213/bg4.jpg)
4
common visual atoms should be trained jointly to en-
hance the discrimination. In this section, a visual tree
method is proposed to generate such groups of visually
correlated categories by clustering a large number of
categories into disjoint groups according to their inter-
category visual correlations.
3.1 Visual Category Representation
To characterize the inter-category visual correlations,
we first estimate the visual representation of a n im-
age category b ased on its relevant images. Let I
i
be a
collection of relevant images for the ith category. We
compute the ave rage visual feature as the overall visual
representation for it. First, the content of image I
j
∈ I
i
is represented using the BoW model. Specifically, we
encode the local SIFT features extracted from it over
a dictionary using sparse coding, and then pool the
sparse codes with max-pooling to form an image-level
representation h
j
for it. In implementation, a dictionary
of size 4, 096 was used, and a two-level spatial pyramid
partition (1× 1, 2 ×2) was employed to incorporate weak
spatial information. Second, the visual representation H
i
of the ith category is defined as the av erage feature based
on all the relevant images:
H
i
=
1
|I
i
|
X
I
j
∈I
i
h
j
, (1)
where |I
i
| is the number of images in I
i
. Finally, we
normalize the l
2
norm of H
i
to be 1.
3.2 Image Category Clustering
After the visual representations {H
i
}
M
i=1
for M categories
are computed, theoretically any clustering methods can
be used to cluster them into disjoint groups based on
the representations. For instance, one could simply use
the k-means algorithm to group the categories b y taking
{H
i
}
M
i=1
as input. Table 1 presents a number of category
groups which are identified by the method following this
idea based on the ILSVRC2010 data set.
Another approach is to first compute the visual sim-
ilarities between the categories, and then cluster them
into groups according to the similarity values. Specifi-
cally, we define the visual affinity value s(i, j) between
the ith and jth classes a s:
s(i, j) = exp
−
d(H
i
, H
j
)
σ
, (2)
where σ is the bandwidth which is automatically de-
termined b y the self-tuning technique proposed in [54],
and d(·, ·) is the E uc lidean distance oper ator. The inter-
category affinity values of M categories can be repre-
sented as an M -by-M matrix S, where S
i,j
= s(i, j).
The affinity propagation (AP) [21] clustering algorithm is
employed to partition the categories into a set of disjoint
groups by taking S as input because of the e ffec tiveness
of AP clustering in many applications. Assuming that all
the categories have equal chances to be the exemplars,
their preferences are set to be a common value which is
chosen as the median of all the similarity values. In Table
2, we show several category groups which are generated
by using AP clustering with affinity matrix S as input
for the ILSVRC2010 set.
It is see n in Table 1 and 2 that category clustering aims
to assign a small number of visually similar c ategories
into the same group, so that their inter-related dictionar-
ies should share some common visual atoms. Given such
a group of visually correlated ca tegories, the proposed
joint dictionary learning (JDL) algorithm (Section 4) can
be utilized to learn more d iscriminative dictionaries by
explicitly sepa rating the common visual atoms from the
category-specific ones.
3.3 Relation to Label Tree [4]
It is worth noting that the label tree method [4] can also
be used as an effective ap p roach to category clustering.
Specifically, to obtain the confusion matrix C ∈ R
M×M
(M is the number of ca tegories), which is needed in
the label tree method, we train M one-vs-rest (OVR)
binary S V Ms, and then evaluate them on a validation
set. The categories are partitioned into a number of
disjoint groups using spectral clustering [ 38] with B as
the affinity matrix, where B =
C+C
T
2
.
One can observe that the label tree method tends to
assign the classes which are easily confused by the OVR
SVMs into the same group. However, the visual tree
method is designed to put the categories which have
strong inter-category visual correlations into the same
group, so that the JDL algorithm can be applied to learn
more discriminative dictionaries. In Sec tion 6.3.1, we not
only compare the performance of the visual tree with
the label tree in the task of category clustering, but
also quantitatively evaluate how they contribute to the
performance of the proposed JDL algorithm on image
classification.
4 JOINT DICTIONARY LEARNING
After a large number of visual categories are parti-
tioned into a set of disjoint groups, we present a joint
dictionary learning (JDL) algorithm in this section. It
can simultaneously learn one common dictionary and
multiple category-specific dictionaries for the visually
correlated categories in the same group. Obviously, the
discriminative dictionaries of different groups can be
learned indep endently by performing the JDL algorithm
sequentially or in parallel.
4.1 Formulation of JDL
Given a group of C visually correlated categories, let
X
i
∈ R
d×N
i
, i = 1, . . . , C, be a collection of training
points for the ith class, and D
i
∈ R
d×K
i
is its visual dic-
tionary, where d is the dimension of a training sample, N
i
is the number of training samples for the ith class, and