4
data [77]. Let us assume that we have a mapping function
Φ : R
n
→ R
d
that can map data x
i
into a feature space
Φ(x
i
). Depending on the mapping function, the feature space
dimension d can be arbitrarily large. For simplicity let us
assume the data mapped into the feature space is centered,
i.e.,
1
N
P
N
i=1
Φ(x
i
) = 0. As a result, the covariance matrix in
the feature space can be computed as follows:
¯
C =
1
N
N
X
i=1
Φ(x
i
)Φ(x
i
)
T
, (2)
To perform PCA in the feature space, we need to find a set of
eigenvectors and non-zero eigenvalues in the feature space. Let
λ
j
and v
j
denote the eigen values and the corresponding eigen
vector of
¯
C, respectively. One can represent eigen vectors
as v
j
=
P
N
i=1
α
j
i
Φ(x
i
), where α
j
1
, α
j
2
, . . . , α
j
N
are a set of
coefficients. Since the eigenvalue and eigenvectors must satisfy
λ
j
v
j
=
¯
Cv
j
, for performing PCA in the feature space, we
consider an equivalent system as:
λ
j
hΦ(x
i
), v
j
i = hΦ(x
i
),
¯
Cv
j
i, for i = 1, . . . , N, (3)
λ
j
hΦ(x
i
),
N
X
l=1
α
j
i
Φ(x
l
)i = hΦ(x
i
),
¯
C
N
X
l=1
α
j
i
Φ(x
l
)i, (4)
where, h·, ·i indicates inner product between two vectors. Let
us define a column vector as α
j
= [α
j
1
, α
j
2
, . . . , α
j
N
]
T
and a
non-singular N ×N matrix K, where each entry of the matrix
is defined as, K
rt
:= hΦ(x
r
), Φ(x
t
)i. Using these notations,
we can simplify the above equation as:
Nλ
j
α
j
= Kα
j
. (5)
Then the solution α
j
belonging to non-zero eigenvalues
is found by requiring that corresponding vectors in R
d
be
normalized, i.e., hv
j
, v
j
i = 1 [95]. Consequently, we can
find the corresponding eigenvectors v
j
. For a new vectorized
test image x
test
, we can compute its projection onto the
eigenvectors. We can calculate the projection with respect to
the j
th
eigenvector in the feature space as:
hv
j
, Φ(x
test
)i =
N
X
i=1
α
j
i
hΦ(x
i
), Φ(x
test
)i. (6)
Note that in Eq. 6, we do not require the definition of mapping
function Φ in its explicit form and only need it in the dot
product form. Hence, we can define functions to perform dot
products without actually having to explicitly map data points
using the mapping function Φ. These functions are commonly
referred to as kernel functions [30]. An example of a kernel
function is the the RBF kernel [30], defined as K(x
i
, x
j
) =
e
−kx
i
−x
j
k
2
/2σ
2
, where σ is a parameter of the kernel. Similar
to PCA, the projection values computed using Eq. 6 are used
as a feature representation for any test data. Note that, kernel
PCA can be derived for data without the assumption that data
is centered in feature space with minor modification in the
derivation process [76].
Both PCA and kernel PCA produce features that are de-
scriptive as the lower dimensional subspace learned has the
capacity to represent the image space of training data [31].
However, the learned embeddings are not necessarily compact.
Fig. 4. Learning features with the help of a stacked auto-encoder network
trained to reconstruct the input images using mean-squared error.
B. Deep learning-based Features
Deep Auto-encoders
An auto-encoder is a (convolutional) neural network consisting
of an encoder (En) and decoder (De) networks as shown in
Figure 4. The encoder comprises of a set of convolution layers
followed by activation functions. The decoder consists of
transposed convolutional layers and commonly has a structure
similar to that of an inverted encoder. An auto-encoder is
trained with the objective of minimizing the distance between
the input and the output of the network. In theory, any distance
measure, such as:
L
mse
= kx − De(En(x))k
2
2
, (7)
can be considered to learn the parameters of the auto-encoder,
where x is the input image.
It is the usual practice to have a bottleneck latent-space in
between with a dimension smaller than the input. Due to this
bottleneck, auto-encoder retains only the essential information
in the latent space which is required for reconstruction. These
encoder features are informative and can be expected to
exhibit descriptiveness. It has been shown in the literature
that adding noise to the input can further improve the quality
of the learned representation by reducing the problem of
over-fitting, making it more generalizable [92]. When noise is
added to the input, the network is referred to as a de-noising
auto-encoder [92]. In a de-noising auto-encoder, given a
noisy image, the network is expected to reconstruct the clean
version of the image.
Geometric Transformation based Self-supervision
Self supervision is a machine learning technique that can
be used to learn informative representations from unlabeled
data. Golan et al. showed that self supervision can yield
representations that work in favor of one class classification
[27], [28]. In their work, a random geometric transform chosen
from a pre-determined set of transforms is first applied to
every input image during training. The network is trained to
correctly predict the applied transformation. Let us denote a
set of k geometric transformations as {T
1
, T
1
, . . . , T
k
}, where
for any given image x both x, T
i
(x) ∈ R
H×W
and T
1
denotes
identity transformation. During training, for a given input, a
number 0 ≥ r ≥ k is randomly chosen. The input image x is
transformed using the r
th
transformation to arrive at T
r
(x).
The transformed image is passed through a convolutional
剩余18页未读，继续阅读
syp_net
- 粉丝: 158
- 资源: 1187
我的内容管理
展开
最新资源
- 计算机人脸表情动画技术发展综述
- 关系数据库的关键字搜索技术综述：模型、架构与未来趋势
- 迭代自适应逆滤波在语音情感识别中的应用
- 概念知识树在旅游领域智能分析中的应用
- 构建is-a层次与OWL本体集成：理论与算法
- 基于语义元的相似度计算方法研究：改进与有效性验证
- 网格梯度多密度聚类算法：去噪与高效聚类
- 网格服务工作流动态调度算法PGSWA研究
- 突发事件连锁反应网络模型与应急预警分析
- BA网络上的病毒营销与网站推广仿真研究
- 离散HSMM故障预测模型：有效提升系统状态预测
- 煤矿安全评价：信息融合与可拓理论的应用
- 多维度Petri网工作流模型MD_WFN：统一建模与应用研究
- 面向过程追踪的知识安全描述方法
- 基于收益的软件过程资源调度优化策略
- 多核环境下基于数据流Java的Web服务器优化实现提升性能
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
