基于正弦余弦算法的支持向量回归参数优化
需积分: 9 74 浏览量
更新于2024-07-18
收藏 984KB PDF 举报
"本文主要探讨了支持向量机（Support Vector Machine, SVM）在回归问题中的应用，特别是支持向量回归（Support Vector Regression, SVR）的参数优化问题。研究中，作者使用了正弦余弦算法（Sine Cosine Algorithm, SCA）来优化SVR的参数，以提升时间序列预测的准确性。时间序列预测是基于数据驱动的预测方法的重要组成部分，尤其在处理大量传感器数据而无需深入理解系统内在故障机制的情况下。"
支持向量机（SVM）是一种广泛应用的监督学习模型，最初用于分类任务，后来发展出支持向量回归（SVR）用于连续值预测，如时间序列分析。SVR通过构造一个最大边距超平面，将数据点尽可能地分开，并且可以处理非线性关系。然而，SVM的性能高度依赖于其关键参数的选择，包括核函数类型、惩罚参数C和核参数γ。
正弦余弦算法（SCA）是一种新型的全局优化算法，源于自然界的振动模式，因其快速收敛性和全局搜索能力而在参数优化问题中受到关注。在本研究中，SCA被应用于SVR的参数C和γ的优化过程，以找到最佳的参数组合，从而提高预测模型的精度和稳定性。
时间序列预测是预测未来事件的一种统计方法，广泛应用于金融、气象、工业设备故障预测等多个领域。传统的预测模型往往需要深入理解系统的动态行为，但数据驱动的方法如SVR则依赖于历史数据，减少了对先验知识的依赖。
在文章中，作者可能详细介绍了实验设计、数据集选择、SCA优化过程以及性能评估指标。他们可能比较了SCA优化的SVR与未优化的SVR，以及其他常见时间序列预测方法（如ARIMA、随机森林等）的预测效果，展示了参数优化对于提升SVR预测性能的重要性。
这篇研究通过结合支持向量回归和支持向量机的参数优化，尤其是利用正弦余弦算法进行优化，为时间序列预测提供了一种可能的改进策略。这一工作对于那些依赖准确预测结果的领域，如故障预防和资源管理，具有重要的实践价值。
S. Li et al. / Expert Systems With Applications 91 (2018) 63–77 65
the convex optimization problem shown in Eq. (2) .
minimize
1
2
w
2
+ C
l
i =1
ξ
i
+ ξ
∗
i
(2)
subject to
⎧
⎨
⎩
w,
(
x
)
+ b − y
i
≤ ε + ξ
i
y
i
− w,
(
x
)
− b ≤ ε + ξ
∗
i
ξ
i
, ξ
∗
i
≥ 0
(2)
where C is a constant known as the penalty factor, and the ξ
i
, ξ
∗
i
term show the amount of difference between the estimated value
and the target value.
It turns out that the optimization problem can be solved more
easily in its dual formulation. Substituting the saddle point con-
dition and using K( x
i
, x
j
) =
T
( x
i
)( x
j
) instead of ( · ) explicitly,
the kernel version of dual optimization problem is yielded by elim-
inating the dual variables η
i
, η
∗
i
in Eq. (3) . The K ( x, x
) is called the
kernel function satisfying the Mercer condition ( Mercer, 1909 ).
minimize
1
2
l
i, j=1
α
∗
i
− α
i
α
∗
j
− α
j
K
x
i
· x
j
+ ε
l
i =1
α
∗
i
+ α
i
−
l
i =1
y
i
α
∗
i
− α
i
subject to
⎧
⎨
⎩
l
i =1
α
∗
i
− α
i
= 0 ,
0 ≤ α
i
, α
∗
i
≤ C
(3)
After solving the dual optimization problem above, w can be
solved explicitly in Eq. (4) where α
i
and α
∗
i
are Lagrange mul-
tipliers and the samples with positive and non-zero α
i
and α
∗
i
are called the support vectors (SVs). By exploiting the so called
Karush–Kuhn–Tucker (KKT) conditions which describe determine
necessary and sufficient conditions for a global optimum, the prod-
uct between dual variables and constraints has to vanish at the
optimal solution and the parameter b can be computed in Eq. (5) .
Then the function f ( x ) can be rewritten in the so-called support
vector expansion shown in Eq. (6 ). In a sense, the complexity of a
function is independent of the dimensionality of the input space,
and depends only on the number of SVs.
w =
l
i =1
(α
∗
i
− α
i
) x
i
(4)
b = y
j
−
w, (x )
− ε for 0 ≤ α
i
≤ C
b = y
j
−
w, (x )
+ ε for 0 ≤ α
∗
i
≤ C (5)
f (x ) =
l
i =1
( α
i
− α
∗
i
) K( x
i
, x ) + b (6)
The choice of kernel function for specific data patterns, which
is another attractive question in the application of SVR, appeares
somewhat arbitrary till now. Some effort s ( Mohammadi, Shamshir-
band, Anisi, Alam, & Petkovi
´
c,
2015; Shamshirband et al., 2014 )
empirically indicate that the use of the gaussian RBF kernel is su-
perior to other kernel functions because of its accessibility to im-
plement and powerful mapping capability. Therefore, the gaussian
RBF kernel function K(x, x
) = exp (−
x −x
2
2 σ
2
) was specified in this
study. There are two parameters involved in SVR algorithm. The
parameter C controls the trade-off between the complexity of the
function and the frequency in which errors are allowed. The pa-
rameter σ affects the mapping transformation of the input data to
the feature space and controls the complexity of the model. Thus,
it is important to select suitable parameters, and the value of pa-
rameter σ should be selected more carefully than C as reported in
Schölkopf and Smola (2002) .
3.2. Sine cosine algorithm
Sine cosine algorithm can be able to explore different regions
of a search space, avoid local optima, converge towards the global
optimum, and exploit promising regions of a search space during
optimization effectively as stated in Mirjalili (2016) . Based on sine
and cosine functions, SCA starts to search the best global solution
with a set of random candidate solutions and updates their posi-
tions outwards or towards the best solution. Different regions of
the search space are explored when the sine and cosine functions
return a value greater than 1 or less than −1 . Promising regions
of the search space is exploited when sine and cosine return value
between −1 and 1.
In SCA, the dimension of the search space is determined by the
number of parameters that are needed to optimize. The number
of search agents is determined by the user. The positions ( X
i
) of
the current solution are randomly initialized in the search space
and then adjusted to the old positions as in Eq. (7) to make sure
that the solutions always update their positions around the best
solution obtained.
X
t+1
i
=
X
t
i
+ r
1
sin ( r
2
)
r
3
P
t
i
− X
t
i
, r
4
< 0 . 5
X
t
i
+ r
1
cos ( r
2
)
r
3
P
t
i
− X
t
i
, r
4
≥ 0 . 5
(7)
where X
t
i
is the position of the current solution in i th dimension
at t th iteration, and P
i
is position of the destination point in i th
dimension.
Four random and adaptive variables also are integrated to fa-
cilitate divergence and convergence of the search agents. In order
to balance exploration and exploitation to find the promising re-
gions of the search space and eventually converge to the global
optimum, the range of sine and cosine is changed adaptively by
defining the parameter r
1
in Eq. (8) . So, the parameter r
1
dictates
the next positions region (or movement direction) which could be
either in the space between the solution and destination or out-
side it. The cyclic pattern of sine and cosine function allows a so-
lution to be repositioned around another solution. This can guar-
antee exploitation of the space defined between two solutions. By
changing the range of the sine and cosine functions, the solutions
should be able to explore the outside search space between their
corresponding destinations. The random location either inside or
outside is achieved by defining a random number for r
2
in [0,
2 π] in Eq. (7) . So, the random parameter r
2
defines how far the
movement should be towards or outwards the destination. There-
fore, this mechanism guarantees exploration and exploitation of
the search space respectively. The random parameter r
3
brings a
random weight for the destination in order to stochastically em-
phasize ( r
3
> 1) or deemphasize ( r
3
< 1) the effect of destination
in defining the distance. Finally, the random parameter r
4
ranging
from 0 to 1 equally switches between the sine and cosine compo-
nents.
r
1
= a − t
a
T
(8)
where t is the current iteration, T is the maximum number of iter-
ations, and a is a constant.
In addition, the pseudo code of the SCA algorithm ( Algorithm 1 )
is presented below. The SCA algorithm starts the optimization pro-
cess by a set of random solutions. The algorithm then saves the
best solutions obtained so far, assigns it as the destination point,
and updates other solutions with respect to it. Meanwhile, the
ranges of sine and cosine functions are updated to emphasize ex-
ploitation of the search space as the iteration counter increases.
The SCA algorithm terminates the optimization process when the
iteration counter goes higher than the maximum number of itera-
tions by default.
As you can see, SCA is more efficient than many other
population-based optimization algorithm to determine the global
剩余14页未读，继续阅读
2022-05-26 上传
2012-04-08 上传
2023-05-11 上传
2023-07-15 上传
2023-07-13 上传
2023-06-23 上传
2023-09-23 上传
2023-05-12 上传
2024-01-31 上传
ucnncu
- 粉丝: 0
- 资源: 1
我的内容管理
展开
最新资源
- 天池大数据比赛：伪造人脸图像检测技术
- ADS1118数据手册中英文版合集
- Laravel 4/5包增强Eloquent模型本地化功能
- UCOSII 2.91版成功移植至STM8L平台
- 蓝色细线风格的PPT鱼骨图设计
- 基于Python的抖音舆情数据可视化分析系统
- C语言双人版游戏设计：别踩白块儿
- 创新色彩搭配的PPT鱼骨图设计展示
- SPICE公共代码库：综合资源管理
- 大气蓝灰配色PPT鱼骨图设计技巧
- 绿色风格四原因分析PPT鱼骨图设计
- 恺撒密码：古老而经典的替换加密技术解析
- C语言超市管理系统课程设计详细解析
- 深入分析：黑色因素的PPT鱼骨图应用
- 创新彩色圆点PPT鱼骨图制作与分析
- C语言课程设计：吃逗游戏源码分享
