没有合适的资源?快使用搜索试试~ 我知道了~
首页sigmoid函数的实现.PDF
资源详情
资源评论
资源推荐
Efficient digital implementation of the sigmoid
function for reprogrammable logic
M.T. Tommiska
Abstract: Special attention must be paid to an efficient approximation of the sigmoid function in
implementing FPGA-based reprogrammable hardware-based artificial neural networks. Four
previously published piecewise linear and one piecewise second-order approximation of the
sigmoid function are compared with SIG-sigmoid, a purely combinational approximation. The
approximations are compared in terms of speed, required area resources and accuracy measured by
average and maximum error. It is concluded that the best performance is achieved by SIG-sigmoid.
1 Introduction
Artificial neural networks (ANNs) have been mostly
implemented in software. This has benefits, since the
designer does not need to know the inner workings of
neural network elements, but can concentrate on the
application of the neural network. However, a disadvantage
in real-time applications of software-based ANNs is slower
execution compared with hardware-based ANNs.
Hardware-based ANNs have been implemented as both
analogue and digital circuits. The analogue implementations
exploit the nonlinear characteristics of CMOS (complemen-
tary metal-oxide semiconductor) devices, but they suffer
from thermal drift, inexact computation results and lack of
reprogrammability.
Digital hardware-based implementations of ANNs have
been relatively scarce, representive examples of recent
research can be found in [1–3]. Recent advances in
reprogrammable logic enable implementing large ANNs
on a single field-programmable gate array (FPGA) device.
The main reason for this is the miniaturisation of component
manufacturing technology, where the data density of
electronic components doubles every 18 months [4].
Special attention must be paid to an area-efficient
implementation of every computational element when
implementing large ANNs on digital hardware. This holds
true for the nonlinear activation function used at the output
of neurons [5].
A common activation function is the sigmoid function
(Fig. 1)
y ¼
1
1 þ e
x
ð1Þ
An advantage of the sigmoid function is its derivative (see
Fig. 1)
dy
dx
¼ yð1 yÞð2Þ
whose existence is essential in neural network training
algorithms. Since the sigmoid function has a symmetry
point at (0, 0.5), only half of the x – y pairs have to be
computed
y
x>0
¼ 1 y
x0
or ð3Þ
y
x<0
¼ 1 y
x0
ð4Þ
A straightforward sigmoid implementation requires a lot of
area, and an approximation is the only practical solution in
digital ANNs.
2 Implementations of sigmoid function
Digital hardware implementations of the sigmoid function
are divided into three main categories: piecewise linear
(PWL) approximations, piecewise second-order approxi-
mations and combinational approximations. The efficiency
criteria for a successful approximation are the achieved
accuracy, speed and area resources.
The maximum and average error are used to evaluate the
accuracy of an approximation. Following the methodology
in [6], if a function f (x) is approximated by a function
^
ff ðxÞ
in the interval x 2ða
0
; a
1
Þ; the average E
ave
and maximum
E
max
errors are obtained by uniformly sampling x on 10
6
equally spaced points in the domain of ða
0
; a
1
Þ
E
ave
¼
P
10
6
1
i¼0
j
^
ff ðx
i
Þf ðx
i
Þj
10
6
E
max
¼ max j
^
ff ðx
i
Þf ðx
i
Þj
8
>
>
<
>
>
:
ð5Þ
Evaluating speed is straightforward, provided that all the
implementations under comparison compute the sigmoid
function in a single clock cycle. In this case the speed metric
is the maximum clock rate, typically denoted in megahertz
(MHz).
When evaluating the area resources of FPGA-based
implementations, the basic unit is a logic element (LE)
discussed in Section 3.
A straightforward implementation of the sigmoid func-
tion is not feasible, since both division and exponentiation
are very demanding operations, as they require a lot of area
resources and converge slowly.
In this paper, only fixed-point notation is used. Floating-
point arithmetic does not suit a digital approximation for
two main reasons: the area requirements of floating-point
q IEE, 2003
IEE Proceedings online no. 20030965
doi: 10.1049/ip-cdt:20030965
The author is with Signal Processing Laboratory, Helsinki University of
Technology, Otakaari 5 A, FIN-02150, Finland
Paper first received 12th June 2003 and in revised form 20th August 2003
IEE Proc.-Comput. Digit. Tech., Vol. 150, No. 6, November 2003 403
© 2003 IEE. Reprinted with permission from IEE Proceedings – Computers and Digital Techniques 150, number 6, pages 403-411.
chw2002
- 粉丝: 1
- 资源: 5
上传资源 快速赚钱
- 我的内容管理 收起
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
会员权益专享
最新资源
- c++校园超市商品信息管理系统课程设计说明书(含源代码) (2).pdf
- 建筑供配电系统相关课件.pptx
- 企业管理规章制度及管理模式.doc
- vb打开摄像头.doc
- 云计算-可信计算中认证协议改进方案.pdf
- [详细完整版]单片机编程4.ppt
- c语言常用算法.pdf
- c++经典程序代码大全.pdf
- 单片机数字时钟资料.doc
- 11项目管理前沿1.0.pptx
- 基于ssm的“魅力”繁峙宣传网站的设计与实现论文.doc
- 智慧交通综合解决方案.pptx
- 建筑防潮设计-PowerPointPresentati.pptx
- SPC统计过程控制程序.pptx
- SPC统计方法基础知识.pptx
- MW全能培训汽轮机调节保安系统PPT教学课件.pptx
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论2