Vol. XX, No. X ACTA AUTOMATICA SINICA Month, 201X
Flexible support vector regression and its application to
fault detection
Yi Hui
1
Song Xiao-Feng
1
Jiang Bin
1
Liu Yu-Fang
1, 2
Zhou Zhi-Hua
2
Abstract Hyper-parameters, which determine the ability of learning and generalization for support vector regression (SVR), are
usually fixed while training. Thus when SVR is applied to complex system modeling, this parameters-fixed strategy leaves the SVR
in a dilemma of selecting rigorous or slack parameters due to complicated distributions of sample dataset. Therefore in this paper
we proposed a flexible support vector regression (F-SVR) in which parameters are adaptive to sample dataset distributions during
training. The method F-SVR divides the training sample dataset into several domains according to the distribution complexity, and
generates different parameter set for each domain. The efficacy of the proposed method is validated on an artificial dataset, where
F-SVR yields better generalization ability than conventional SVR metho ds while maintaining good learning ability. Finally, we also
apply F-SVR successfully to practical fault detection of a high frequency power supply.
Key words Support Vector Regression, Flexible, Fault Detection, Power Supply
Citation Shang Shu-Lin, Zuo Nian-Ming, Zhang Zhe. Preparation of Papers for Acta Automatica Sinica. Acta Automatica Sinica,
2013, 39(1): 1−3
DOI 10.3724/SP.J.1004.2012.xxxxx
Supp ort vector regression, which is known as one of the
most powerful tools for function approximation, was pro-
p osed by Vapnik in 1995 with the principle of Structure
Risk Minimization(SRM)
[1]
. Compared with tools based
on Experience Risk Minimization(ERM), such as Neural
networks and least square methods, it requires less samples
and yields better generalization ability, and hence has been
widely used in fields like time series prediction
[2][3]
, process
control
[4]
and fault diagnosis
[5]
. Although SVR has been
well studied and many remarkable achievements have been
obtained, the theoretical estimation of regression parame-
ter remains unsolved in the last decade. There are some
practical recommendations on this issue: Cherkassky et
al.(1999) applied VC generalization bounds to control the
mo del complexity; Chapelle & Vapnik(2000, 2002) penal-
ized the regression based on the ratio of expected training
error and generalization error; Scholkopf et al.(1999, 2002)
prop osed the famous v-SVR that can automatically choose
an appropriate epsilon with given prior knowledge, and
tackling their work, Kwok and Tsang(2003) refined it in
case of Gaussian noise.
As there is no general consensus on the setting of proper
parameters, re-sampling approaches have wildly been em-
ployed in practical applications, such as cross valida-
tion(Muller et al., 1997; M. Chang & C. Lin, 2005), gradi-
ent descent(Chapelle, 2002) and swarm intelligence(Ustun
et al., 2005; W.Hong, 2006; Tian et al., 2009). These meth-
o ds can provide practical parameters, but are expensive in
computational cost. In this case, Cherkassky & Ma (2004)
prop osed their empirical selection for the SVR parame-
ters which gives a simple yet practical setting of regression
parameters without re-sampling. However, all approaches
mentioned above consider only in a global fashion while re-
gressing, and there can hardly be a set of parameters that
fits the training samples well in many cases.
Manuscript received Month Date, Year; accepted Month Date, Year
The work is supported by grants from the National Science Founda-
tion of China(61034005,61171191,61203072), the Science Foundation
of Jiangsu Province(BK2010500 ), the Doctoral Fund of Ministry of
Education of China(20113218110011), the Priority Academic Pro-
gram Development (PAPD) of Jiangsu Higher Education Institu-
tions, and the Open Project of State Key Laboratory of Industrial
Control Technplogy(ICT1234).
Recommended by Associate Editor BIAN Wei
1. College of Automation Engineering, Nanjing University of Aero-
nautics and Astronautics, Nanjing, 210016, P. R. China 2.Guodian
Science and Technology Research Institute , Nanjing, 210031, P. R.
China
Recently, some adaptive methods have been proposed
for setting regression parameters
[18−20]
. These methods
are capable to adjust the parameters according to the sam-
ple distribution, hence perform better than parameter-fixed
metho ds. For example, Hao (2010) has changed the fixed
parameter ε into a variable in the regression, which effi-
ciently sets the parameter ’epsilon’ without re-sampling.
However, it optimizes only one of the three parameters and
is hard to give a better trade-off between over-fitting and
under-fitting in some complicated cases.
Inspired by Cherkassky
[17]
and Hao’s
[21]
work, we pro-
p osed a new approach to optimize the process of regression.
This approach flexibly divides the samples into several re-
gions, and for each region, it selects proper parameters ac-
cording to its distribution.
The paper is organized as follows. Section 1 presents the
problem of existing approaches. Section 2 describes the
prop osed flexible support vector regression method; Sec-
tion 3 presents the performance of the proposed method
by comparisons with existing approaches, and applies it to
the fault detection of a new kind of power supplies. Finally,
conclusions and discussion are given in section 4.
1 Problem formulation
Given training data {(x
1
, y
1
), . . . , (x
l
, y
l
)} with unknown
distribution function, the core technique for SVR is to find
a proper parameter vector p
0
and the optimal hyper-plane
(the vector w
0
):
(p
0
, w
0
) = arg min :
p,w
R
SRM
(w, p) = R
ERM
+ ϕ(w)
= arg min
p,w
:
1
l
l
i=1
Loss(y
i
, f(x
i
, w, p)) +
1
2
(w · w)
(1)
where p = {C, ε, K(·)}, C is the penalty parameter, ε is
the insensitive parameter, K(.) is the kernel function, and
Loss(y
i
, f(x
i
, w, p)) = C
i
·
y
i
− (
l
i=1
β
i
K(x, x
i
) + b)
ε
. The
intrinsic difference between support vector regression and
conventional regressions is that SVR considers not only
learning ability but also VC confidence (generalization abil-