![](https://csdnimg.cn/release/download_crawler_static/17053197/bg4.jpg)
Zhang et al. [40] empirically validate the feasibility and the effec-
tiveness of using the Universum learning in semi-supervised learn-
ing. Zhang et al. design a graph-based method to make full use of
the Universum examples, which helps depict the prior information
for possible classifiers. While other researches focus on combining
the Universum with some traditional learning methods so as to
create a new effective one, e.g., the novel U-SVM [36] solving the
non-standard learning problems [10,14,29] through coping with
sparse high-dimensional data [9], the document clustering with
the Universum utilizing the Universum examples to improve the
clustering performance [39], and the boosting with the Universum
named U Boost [26]. Moreover, some researches introduce the
Universum idea into the supervised learning, especially SVM-based
methods. The twin SVM with Universum [22] and the non-parallel
SVM with Universum [23] are both proposed. The former locates
Universum examples in a non-parallel insensitive loss tube by
using two hinge loss functions to acquire the prior knowledge
[22]. The latter seeks the classifier by maximizing the two margins
associated with the two closest neighboring classes which are
combined by two nonparallel hyper-planes [23].
Based on the existing Universum methods, the advantages of
the Universum learning include two aspects. First, it helps boost
the classifier through acquiring more prior information. Second,
it is suitable for both supervised and semi-supervised learning.
However, not all Universum examples are informative. Even worse,
some Universum examples would play the role as outliers, which
might seriously disturb the training process. Some researches have
found that the Universum examples scattering around the classifi-
cation boundary are most useful since they provide more detailed
knowledge about the discriminant hyperplane [7,28]. But the clas-
sification boundaries are variable. To solve this problem, a special
kind of Universum examples named in-between Universum exam-
ples (U
Mean
) are selected as the informative ones. One way to obtain
the U
Mean
is to calculate the means of each pair between two pat-
terns respectively from two classes of interest. In the proposed
method, the U
Mean
is adopted. Doing so can make up for the loss
of prior distribution information caused by the conventional
matrix-oriented framework and make the classification boundary
more flexible. Here the matrix-oriented classifier design with the
Universum is introduced into the MVL framework, which does
not break the MVL framework. In doing so, we can not only inherit
the advantage of the original methods, but also make the proposed
method more regularized.
3. Architecture of proposed UMultiV-MHKS
In this section, the detailed description about the architecture of
UMultiV-MHKS is provided. Firstly, supposing that there are N
labeled samples fðx
i
;
u
i
Þg
N
i¼1
and some Universum examples
fðx
i
;
u
i
Þg
U
i¼1
in a single view sense, where x
i
2 R
d
and the corre-
sponding class label
u
i
2fþ1; 1g. The optimization problem,
i.e., the MVL framework (1) is characterized as follows:
min L ¼
X
M
p¼1
L
p
w
þ L
b
:
As stated above, L
p
w
is guaranteeing that each view can correctly
classify samples. L
b
is minimizing the disagreement among different
views by making the output of each view be maximally close to the
average output of all views. Further, L
p
w
can be described as follows:
L
p
w
¼ R
p
emp
þ c
p
R
p
reg
þ f
p
R
uni
:
According to the framework (3), for the pth view, both c
p
and f
p
are
the regularization parameters. R
p
emp
is the empirical risk term. R
p
reg
is
the generalization term. R
p
uni
is the Universum regularization term.
The detailed definition of the three terms are given below:
R
p
emp
¼
X
N
i¼1
u
i
g
p
A
p
i
1 b
p
i
2
; ð5Þ
R
p
reg
¼ u
p
T
S
1
u
p
þ
~
v
p
T
S
2
~
v
p
; ð6Þ
R
p
uni
¼
X
U
i¼1
g
p
A
p
i
2
; ð7Þ
where both A
p
i
and A
p
i
are the pth matrix size of the x
i
and x
i
,
respectively. b
p
i
is an arbitrary scalar quantity. Both S
1
¼ mI
mm
and S
2
¼ nI
nn
are two pre-defined matrices. The reshaping way
turning the original sample into one matrix form is the same as
the pre-defined M matrixization ways [33]. Based on Eqs. (4), (5)
and (7), the detailed description of both R
p
emp
and R
p
uni
are given as
follows:
R
p
emp
¼
X
N
i¼1
u
i
u
p
T
A
p
i
~
v
p
þ
v
p
0
1 b
p
i
2
; ð8Þ
R
p
uni
¼
X
U
i¼1
u
p
T
A
p
i
~
v
p
þ
v
p
0
2
: ð9Þ
Both u
p
and
~
v
p
are two weight vectors, and
v
0
is the bias. The L
b
is
defined as:
L
b
¼
c
X
M
p¼1
X
N
i¼1
u
i
u
p
T
A
p
i
~
v
p
þ
v
p
0
X
M
q¼1
r
q
u
i
u
q
T
A
q
i
~
v
q
þ
v
q
0
2
: ð10Þ
c
is the coupling parameter. r
q
denotes the importance of the corre-
sponding view, where r
q
P 0;
P
M
q¼1
r
q
¼ 1. It is clear that the bigger
the r
q
is, the more important the corresponding view is. In this
paper, r
q
¼
1
M
for simplicity.
Combining Eqs. (6), (8)–(10), the optimization function of the
proposed MVL with the Universum learning can be given as
follows:
min
u
p
2R
m
v
p
2 R
nþ1
p ¼ 1; ...; M
L ¼ L
1
þ L
2
: ð11Þ
In detail,
L
1
¼
X
M
p¼1
X
N
i¼1
u
i
u
p
T
A
p
i
~
v
p
þ
v
p
0
1 b
p
i
2
þ c
p
u
p
T
S
1
u
p
þ
~
v
p
T
S
2
~
v
p
þ f
p
X
U
i¼1
ðu
p
T
A
p
i
~
v
p
þ
v
p
0
Þ
2
!
; ð12Þ
L
2
¼
c
X
M
p¼1
X
N
i¼1
u
i
u
p
T
A
p
i
~
v
p
þ
v
p
0
X
M
q¼1
r
q
u
i
u
q
T
A
q
i
~
v
q
þ
v
q
0
2
: ð13Þ
Here we set Y
p
¼ y
p
1
; ...; y
p
N
T
, Y
p
¼ y
1
p
; ...; y
U
p
T
, y
p
i
¼ u
i
u
pT
A
p
i
;
1
T
; y
p
i
¼ u
pT
A
p
i
; 1
T
; i ¼ 1; ...; N; b
p
¼ b
p
1
; ...b
p
N
T
,
v
p
¼
~
v
pT
;
v
p
0
T
.
Thus Eq. (11) can be simplified in matrix form as follows:
min
u
p
2R
m
;
v
p
2R
nþ1
p¼1;...;M
L
0
¼
X
M
p¼1
Y
p
v
p
1
N1
b
p
T
Y
p
v
p
1
N1
b
p
þ c
p
u
p
T
S
1
u
p
þ
v
p
T
~
S
2
v
p
þ f
p
Y
p
v
p
2
þ
c
X
M
p¼1
Y
p
v
p
X
M
q¼1
r
q
ðY
q
v
q
Þ
!
2
; ð14Þ
where S
2
is defined as a matrix with the dimensionality of n n, and
f
S
2
¼
S
2
0
00
.
Z. Wang et al. / Knowledge-Based Systems 70 (2014) 376–391
379