Principal Components Analysis-Based Visual
Saliency Detection
Bing Yang, Xiaoyun Zhang, Jing Liu, Li Chen and Zhiyong Gao
Institute of Image Communication and Network Engineering
Shanghai Key Laboratory of Digital Media Processing and Transmissions
Shanghai Jiao Tong University, Shanghai 200240, China
{yangbing,xiaoyun.zhang,hilary,hilichen,zhiyong.gao}@sjtu.edu.cn
Abstract—In this paper, a novel patch-wise saliency detection
algorithm is proposed based on Principal Component Analysis
(PCA). As a powerful statistical procedure in data analysis, PCA
are fully exploited to convert color space and produce compact
patch representation. Specifically, images are first converted to
linearly uncorrelated channels and divided into non-overlapped
patches. Then the patches are represented by the coefficients of
principal components using PCA analysis. Based on the compact
representation of patches, two types of distinctiveness are intro-
duced: center-surround contrast and global rarity. Experimental
results demonstrate that the PCA-based color space conversion
and patch representation can improve the accuracy of human
fixations prediction, and the proposed algorithm outperforms the
mainstream algorithms on predicting human fixations.
Index Terms—Saliency detection, Principal Component Anal-
ysis (PCA), Center-surround, Rarity.
I. INTRODUCTION
Visual attention has been extensively studied recently due to
its wide applications in computer vision, contrast enhancement
and video compression [1]–[3]. Numerous saliency models
have been proposed by psychologists, neurophysiologists and
computer scientists to imitate human visual attention. Accord-
ing to the motivation of inferring visual attention, the saliency
detection models can be categorized into bottom-up models
and top-down models. Top-down models are task-driven while
bottom-up models are data-driven and more related to the
nature of human visual system. So we focus on bottom-up
model in this paper.
Although there is no consensus on the mechanism of human
visual system, it is widely accepted that human tends to focus
on distinct regions that stand out from the entire image. Based
on this motivation, many researchers have proposed various
saliency models to estimate visual saliency [4]–[19]. Despite
the variety in existing saliency detection models, they all deal
with the following three key issues: color space, visual units
(pixel-wise or patch-wise) and features, and distinctiveness
definition.
Color space that comes along with color images has been
widely researched in the past years. Many image processing
algorithms operate on separate color channel and fuse the out-
put of each channel either uniformly or non-uniformly. When
these methods perform on redundant color space (e.g., RGB),
over-emphasis on redundant information may be caused. Even
though LAB color space reduces the correlation from a statis-
tical point of view, it does not decorrelate specific images.
Patch-based methods, which have been widely used in
literatures, have intrinsic advantages over pixel-based methods
for saliency detection problem. First, saliency is a region-based
concept since single pixel with high distinctiveness can not be
captured. Second, patch-based methods process the patch as
a whole. They always have lower computational complexity,
which is desired for the pre-processing procedure of high-level
tasks such as image understanding. Therefore, patch-based
method is used in the proposed algorithm and patch representa-
tion comes as an intrinsic problem. Direct representation using
pixel values introduces noises and neglects the intrinsic spatial
correlations among patches. In [15], dimensionality reduction
is adopted for co-similarity matrix, which is equivalent to
a PCA representation of image patches. In our paper, we
formulate a compact and informative representation of patches
directly from PCA point of view. Principal components are
extracted to represent each patch and the dimensions corre-
sponding to noises are thrown away to make the representation
more effective.
The last key issue associated with a saliency detection
algorithm is how to define distinctiveness. Different definition
of distinctiveness results in different algorithm. A commonly
used definition is center-surround contrast [7]. Large contrast
between center and surrounding regions indicates highly in-
formative regions, hence, attracting more attentions. Another
type of distinctiveness is defined as global rarity [4], [12]. A
patch with features that rarely appear over the entire image
is believed distinct and draws more attentions. As pointed out
by [5], [9], center-surround contrast and global rarity work
complementarily for saliency detection of images.
In this paper, we propose a patch-wise image saliency de-
tection algorithm using PCA analysis. As a powerful statistical
procedure, PCA plays an important role throughout the pro-
posed scheme. More specifically, the RGB color space is first
transformed into uncorrelated color space where correlations
among different channels are discarded through PCA analysis.
Then, the image of each channel is divided into patches and
the principal components of each patch are extracted as patch
features based on PCA. With such a compact patch representa-
tion, visual saliency is measured by patch distinctiveness both
locally and globally. Local distinctiveness focuses on center-