dimensionality reduction process into a unified framework,
which results in an optimized graph rather than a prede-
fined one.
In this paper, we first propose a semi-supervised di-
mensionality reduction method called weighted pairwise
constraints based semi-supervised dimensionality reduction
(WPCSSDR). Then, a novel semi-supervised dimension-
ality reduction method called adaptive semi-supervised
dimensionality reduction (ASSDR) is proposed which uses
WPCSSDR as a subprogram. ASSDR first initialize all the
pairwise constraints with equal weights and construct a
neighborhood graph with initial adjacency weight matrix,
and then the following procedure is repeated until the stop
condition is satisfied: (1) reducing the dimensionality of the
original space with the current weighted pairwise con-
straints and the current adjacenc y weight matrix using
WPCSSDR; (2) clustering in the reduced subspace; (3)
updating the weights of the pairwise constraints according
to the clustering result; (4) updating the adjacency weight
matrix. As a result, we can get the optimized weights of the
pairwise constraints and the optimized adjacency weight
matrix of the neighborhood graph, as well as the projection
matrix.
2 Adaptive semi-supervised dimensionality
reduction algorithm (ASSDR)
2.1 The problem
Here we define the weighted pairwise constraints based
semi-supervised dimensionality reduction problem as fol-
lows: Suppose we have a set of D-dimensional data sam-
ples X ¼fx
1
; x
2
; :::; x
n
gR
D
together with some pairwise
must-link constraints (M) and cannot-link constraints (C)as
domain knowledge: ðx
i
; x
j
Þ2M,ifx
i
and x
j
belong to the
same class; ðx
i
; x
j
Þ2C,ifx
i
and x
j
belong to the different
classes. In addition, each pairwise constraint ðx
i
; x
j
Þ has a
weight S
ij
to indicate the importance of information owned
by itself, which mea ns one should be paid more attention to
the pairwise constraint ðx
i
; x
j
Þ if S
ij
is large. In this case,
what we want to do is to find a set of linear projection
vectors W ¼½w
1
; w
2
; :::; w
d
2R
Dd
, where d\\D, such
that the transformed low dimensional projections
Y ¼fy
1
; y
2
; :::; y
n
gR
d
, where y
i
¼ W
T
x
i
, can preserve
some properties of the original dataset as well as the
pairwise constraints in M and C. For the convenience of
discussion, one dimensional case is discussed below,
namely y
i
¼ w
T
x
i
, which is easy to be extended to the high
dimensional case.
2.2 Weighted pairwise constraints based semi-
supervised dimensionality reduction
(WPCSSDR)
To make use of the pairwise constraints, the pairwise points
in M should end up close to each other whi le the pairwise
points in C should end up far from each other. This means
the instances belonging to the same class in the original
space should be close to each other in the reduced sub-
space, and the instances belonging to different classes in
the original space should be far from each other in the
reduced subspace. In addition, if ðx
i
; x
j
Þ2M and S
ij
is
large, it means the Euclidean distance of x
i
and x
j
in the
low dimension should be smaller to each other than with
small weight; if ðx
i
; x
j
Þ2C and S
ij
is large, it means the
Euclidean distance of x
i
and x
j
in the low dimension should
be larger from each other than with small weight.
As for the weighted must-link constraints M, the in-
traclass compactness is characterized by the term as follows:
Q
M
ðwÞ¼
X
ðx
i
;x
j
Þ2Morðx
j
;x
i
Þ2M
w
T
x
i
w
T
x
j
2
S
ij
¼ 2
X
i
w
T
x
i
D
M
ii
x
T
i
w
2
X
i;j
w
T
x
i
S
M
ij
x
T
j
w
¼ 2w
T
XðD
M
S
M
ÞX
T
w
¼ 2w
T
XL
M
X
T
w
ð1Þ
S
M
ij
¼
S
ij
ðx
i
; x
j
Þ2Morðx
j
; x
i
Þ2M
0 else
ð2Þ
where D
M
is a diagonal matrix whose entries are column
sums of S
M
(or row sums, since S
M
is symmetric),
D
M
ii
¼
P
j
S
M
ij
, L
M
¼ D
M
S
M
is the Laplacian matrix [26].
Q
M
ðwÞ should be as small as possible, which means the
weighted distance sum in the transformed low dimensional
subspace between instances involved in the must-link
constraints M should be small.
On the other hand, the interclass separability of the
weighted cannot-link constraints C can be characterized by
the term:
Q
C
ðwÞ¼
X
ðx
i
;x
j
Þ2Corðx
j
;x
i
Þ2C
w
T
x
i
w
T
x
j
2
S
ij
¼ 2
X
i
w
T
x
i
D
C
ii
x
T
i
w
2
X
i;j
w
T
x
i
S
C
ij
x
T
j
w
¼ 2w
T
XðD
C
S
C
ÞX
T
w
¼ 2w
T
XL
C
X
T
w
ð3Þ
Int. J. Mach. Learn. & Cyber. (2017) 8:793–805 795
123