J Sign Process Syst (2014) 74:5–18 7
view of image retrieval, it is difficult to search these images
with abstract tags. So the problem of abstract tags is not
trivial and we address the challenge of identifying and
refining abstract tags in this paper.
Besides refining the noisy tags, some schemes are pre-
sented to enrich the tags of social images [12, 14–17].
The authors in [14] utilize the tag co-occurrence to rec-
ommend tags for social images, but these tags would be
unrelated with the content of images. In [12], the authors
enrich the tags of images through adding synonyms and
hypernyms after refining the irrelevant-content tags. Bucak
et al. [15] utilize a multi-label learning algorithm to train
image classifiers with some missing tags and enrich missing
tags using these classifiers. Yang et al. [16] propose a new
scheme to detect the tags of regions in social images and
add six more tags corresponded with six different proper-
ties of regions to enrich the tags of social images, where the
detected tags need to be specific tags and corresponded with
regions of images. In [17], authors exploit the tags asso-
ciated with social videos and images, visual similarity and
the Wikipedia to suggest new tags to key frames of videos.
Compared to these schemes, we focus on enriching the tags
with different semantic levels.
Some pioneering works have been done to utilize the
specific domain knowledge to build up the concept ontol-
ogy. A concept ontology for multimedia domain has been
built up for bridging the semantic gap, applied to the high-
level feature detection task of TRECVID [18]. Lu et al.
[19] propose a framework to automatically develop a lex-
icon of high-level concepts with the small semantic gap,
but this lexicon could not represent the relationship between
concepts because of its flat structure. Some more gen-
eral ontologies have already been developed. For example,
WordNet [20] is the most popular one which ImageNet
[21] use to organize images, and the basic unit of Word-
Net is the synsets (synonym sets) other than single keyword
which users like to use. However, these efforts focus on the
relationship between tags and cannot tell which concepts
are abstract intuitively in the ontology, where we solve the
problem in our constructed ontology.
3 Abstract Tag Refinement
Figure 2 provides an overview of our proposed scheme
for abstract tag refinement and enrichment.For a given
image with the abstract tag, it is difficult to iden-
tify the abstract tag without any prior knowledge. For
solving this problem, we utilize the expertise (i.e. ontol-
ogy) to identify the candidate of abstract tags for social
images. It is discussed detailedly in Section 3.1. After that,
the specific tags for abstract tags would be detected through
the image context and tag context of abstract candidates.
To speed up the search, we utilize the ontology and tag
context of abstract candidates to narrow the search range.
Then k-NN approach is used to find the image context (i.e.
neighbors of the given image) of abstract candidates. Using
the image context, the specific tag is determined by the
voting of neighbors with Gaussian weights (discussed in
Section 3.2 and 3.3). At last, we introduce the details about
our scheme of tag refinement and enrichment based on the
specific tag and abstract tag in Section 3.4.
3.1 Ontology Construction
To utilize the expertise, we construct an ontology with pop-
ular tags which are related to social images. Although using
synsets defined in WordNet is more precise than using a
single word or phrase for tag interpretation, users in the col-
laborative image tagging systems may not use such complex
synsets to label social images. Motivated by this observa-
tion, an ontology with tags as its nodes is constructed for
Figure 2 The framework of our algorithm for abstract tag refinement and enrichment.