Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing
Sigmoid Function Approximation for ANN
Implementation in FPGA Devices
Djalal Eddine KHODJA1 , Aissa KHELDOUN2, and Larbi REFOUFI2
(1)
Faculty of Engineering Sciences, University Muhamed Boudiaf of Msila
B.P N 116 Ichebilia (28000), Algeria, Tel/Fax: +213 35 55 18 36,
E-mail:
[email protected](2)
Signals & Systems Laboratory
Institute of Electronics and Electrical Engineering
Boumerdes University
[email protected]Abstract - The objective of this work is the implementation of Artificial Neural Network on a
FPGA board. This implementation aim is to contribute in the hardware integration solutions
in the areas such as monitoring, diagnosis, maintenance and control of power system as well
as industrial processes. Since the Simulink library provided by Xilinx, has all the blocks that
are necessary for the design of Artificial Neural Networks except a few functions such as
sigmoid function. In this work, an approximation of the sigmoid function in polynomial form
has been proposed. Then, the sigmoid function approximation has been implemented on
FPGA using the Xilinx library. Tests results are satisfactory.
Keywords - ANN, FPGA, Xilinx, Sigmoid Function, power system..
control
of
power
systems.
In this study, we begin by adapting ANN to allow
optimal implementation. This implementation must
ensure efficiency, timeless and a minimum possible
space on the FPGA. Then, we schedule the ANN on
the System Generator. The System Generator to
generate VHDL code. This code is verified and
implemented on a FPGA Spartan-like by the ISE
software from Xilinx Fondation.
1 Introduction
Monitoring, control and maintenance of any element of
wide area power system become an important issue for
ensuring the continuity of electric supply and avoiding
the black out. It is important to early detect the defects
that can occur in these elements by monitoring their
operations and then developing methods for applying
adaptive control and preventive maintenance [1, 2, 3].
The fast implementation of the developed
methods is necessary needs for the industry and the
complex (smart) power systems. These methods may
be investigated using several techniques that have
different characteristics to solve the encountering
problems [4, 5]
The most commonly used techniques are the
artificial intelligence-based techniques such as
Artificial Neural Networks (ANN) [5, 6, 7] that they
are easier to implement on electronic circuit board such
as: Digital Signal Processing (DSP) chips, Application
Specific Integrated Circuits (ASICs) or Field
programmable gate array (FPGAs) [8, 9, 10].
The objective of this work is the implementation of
artificial neural network on a FPGA. This
implementation aim to contribute in hardware
integration solutions using FPGA applied to different
areas such as monitoring, diagnosis, maintenance and
ISBN: 978-960-474-262-2
2 ANN and simulation
To ensure efficiency, timeless and a minimum possible
space on the FPGA, we propose a network that
contains two hidden layers the first layer has three
neurons and the second has two neurons, and output
layer has two neurons.
We conducted a machine learning using the
MATLAB software to where we obtain a smallest
squared error (2.7401 e-015) for 202 inputs, (see Fig .1).
The design of ANN through the use Simulink is
shown in the figure 2.
112
Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing
From the obtained results in the testing phase, it
can be found that almost ANN in accordance (with
error E-008) with the desired predetermined outputs.
3 ANN Review and Simulation of the
System Generator
The Simulink library provided by Xilinx, works in the
same principle as the other elements of Simulink. It
contains blocks representing different functions that are
subjected and interconnected to form algorithms [8].
These blocks do not only serve to simulation, but can
also
generate
VHDL
or
Verilog
code.
In addition, the library provided by Xilinx to
Simulink, has all blocks that are necessary for the
design of ANN except a few functions such as sigmoid
function..
For this reason, we suggested an implementation of
the sigmoid function through the use of Taylor series
[9], the resulting function (of order 5) causes an error
of 0.51% with respect to the continuous model.
In our work, the sigmoid function has been
approximated in polynomial form as given in the
following :function,
2.1 Design of ANN in Simulink
Figure 1 Evolution of average quadratic error of ANN
In1
In2Out1
In1
In3
In2Out1
In1
E1
Input 1
In1
Out1
In3
neuron1
Input 1
In2
neuron1
neuron1
In2Out1
E2
In1
In3
Input 1
In1
In2Out1
Neuron2
E3
In1
Out1
In2Out1
f ( x)
In2
In3
neuron2
y{1}
neurone2
1
1 e cx
(1)
In3
neuron3
Inputs
1
In1
2
In2
3
In3
Layer 2
Layer 1
Outputs layer
Equation (1) can be rewritten as follows :
outputs
f ( x) c bx ax 2
-K-
w11
MATLAB
-K-
Function
w21
w11
Out1
logsig
-K-
In2
-Kw31
(2)
-K-
In1
Input 1
Input 1
B1
B1
w21
MATLAB
Function
1
Out1
Where the coefficients c, b, a are given by:
logsig1
1 e a1
b
1 e a 2
a 1 e a 3
Figure 2 ANN using Simulink library
first:
2.2 Test of ANN
Once the ANN that is implemented and learning
performance has reached. Satisfactory tests are
investigated, their results are presented in table1.
with:
Table.1: Simulation (test) Results of ANN for different
cases.
a1
a a2
a3
Output
of ANN
Safety case
Fault1
Fault2
S1
1.2222 e-018
1.0000 e+000
6.2836 e-016
S2
1.0000 e+000
1.6661 e-007
1.0000 e+000
ISBN: 978-960-474-262-2
113
where a V/ F
Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing
a1
1
and :
0
F 2
4
02
22
4 2 ,
01
21
41
Figure.4 shows that the sigmoid function curve
obtained by Simulink is in accordance with the curve
of sigmoid function obtained by Xilinx. The design of
ANN using Simulink is shown in Figure.5
According to the construction of ANN in the
Simulink/Xilinx the system generator can also generate
the VHDL code.
4 Implementation of ANN in FPGA
We place the entire contents of the block of ANN in
the same subsystem, with a single generator system,
then, it generates the VHDL code (see figure.6).
This part is devoted to description of the
implementation of ANN on FPGA. In order, to do this,
we use the Xilinx System Genrator.
Synthesizer: you click on Synthesizer to find space
resources occupied by the VHDL (see figure.7)
The result is displayed in the Synthesizer as given in
table.2.
From the obtained results shown in the table, we notice
that the VHDL code occupies an area of 208% on
FPGA,
means
that
the
FPGA
type
Spartan2 xc2s200e-6tq does not support this VHDL
code.
However, the implementation of the code obtained in a
system Spartan3 gives the results shown in table 3.
From this table, we found that the code implemented
in Spartan3 takes only 63% of storage space of FPGA.
Modeling the resulting function is given by Xilinx as
shown in Fig.3:
x sysgen
0.2549
CMult1
0.5
a
sysgen
b a+b
AddSub1
Constant
1
In1
In
Gateway In
x sysgen
0.0332
Mult
CMult
sysgen
[a:b]
sysgen
not
Slice
Inverter
a
sysgena - b
b
-4
AddSub2
Constant1
a
sysgen
a-b
b
AddSub3
Constant2
-4
sysgen
b z (ab)
b sysgen
ab
sub
AddSub
Sy stem
Generator
sysgen
[a:b]
hi
sysgen
lo
Slice1
Concat
sysgen
[a:b]
Slice2
sysgen
not
Inverter1
1
Constant3
sel
d0
sysgen
Out
d1
d2
Gateway Out
1
Out1
Mux
5 Conclusion
0
Constant4
Figure 3 modelling of sigmoid function .
In this work, we proposed a simple algorithm for the
implementation of the ANN. The proposed hardware
synthesis algorithm is performed by the System
Generator. Routing and implementation in FPGA type
Spartan2E was made by ISE Foundation.
The use of high-level design tool, System Generator
is very beneficial for the verification of the behaviour
of the algorithm in Simulink. The simulations for the
sigmoid function show that the obtained results using
Xilinx give the same performance as the sigmoid
function
obtained
by
Simulink.
Furthermore, the implementation of the obtained
code using two different systems Spartan2 and
Spartan3 leads to the conclusion that the FPGA type
Spartan3 supported the generated VHDL code gives
adequate results.
Finally, one can conclude that the implementation
of functions in FPGAs is easier because now VHDL
code of any circuit can be generated not only by an
advanced language but also by the software MATLAB.
This gives us the ability to simulate any function in
simulink, using the library of Xilinx after that the latter
is directly converted into VHDL by System Generator.
1
0.9
simulink
xilinx
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
200
400
600
800
1000
1200
Figure 4 Comparaison of Sigmoid functions of Simulink
and Xilinx
ISBN: 978-960-474-262-2
114
Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing
In1
Input 1
In1
In3
In2 Out1
In3
neuron1
In1
E1
Input 1
neuron1
In3
In1
Out1
In2
In1
In2 Out1
neuron2
E3
In1
neuron1
In2 Out1
E2
Input 1
In2 Out1
In1
In3
In2 Out1
neuron2
Out1
In2
neuron2
y{1}
In3
neuron3
Inputs
Layer2
Layer1
Table.2 : Space used on FPGA
Layer of outputs
Output
Fig.5 programming ANN using SIMULINK
Table.3: Summary of used space on the FPGA.
Fig.7 Space occupied by the implementation.
References
Figure 6 System Generator interface.
ISBN: 978-960-474-262-2
[1]
G.Zwinngelsten, Diagnostic des dfaillances:
thorie et pratique pour les systmes
industriels , Ed. Herms Paris. 1995.
[2]
B.Dubuisson ; Dtection et diagnostic des
pannes sur processus , Technique de
lingnieur. R7597,1992.
B.Chetate, DJ.Khodja, Diagnostic en temps
rel des dfaillances dun ensemble Moteur
asynchrone-convertisseur
lectronique
en
utilisant les rseaux de neurones artificiels,
Journal Electrotekhnika, Moscou 12/2003,
pp:16-20
[3]
115
Recent Researches in Circuits, Systems, Electronics, Control & Signal Processing
Table 3 Summary of used space on the FPGA
[4]
DJ.Khodja, B.Chetate, ANN system for
identification and localisation of faillures of
anatomawed electric asynchronous drive, 2nd
International
Symposium
on
Electrical,
Electronic and Computer engineering en
Exhibition, March, 11-13, NEU-CEE 2004,
NICOSIA, TRNC pp : 156-161.
[5] DJ.Khodja, B.Chetate, Development of Neural
Network module for fault identification in
Asynchronous machine using various types of
reference signals, 2nd International Conference
PHYSICS and CONTROL, August,24-26,
Physcon 2005, St Ptersburg, Russia, 2005,. pp :
537-542.
[6] V.A.Tolovka. Rseaux
de
neurones :
Apprentissage, Organisation et Utilisation , Ed,
Entreprise
de
rdaction
de
journal
Radiotekhnika Moscou, 2001.
[7]
N. Moubayed, "Detection et Localisation Des
Dfauts Dans Les Convertisseurs Statiques",
6TH
International
Conference
on
Electromechanical
and
Power
Systems,
Chiinu, Rep. Moldova, October, 2007.
[8] K. Roy-Neogi and C. Sechen, "Multiple FPGA
Partitioning with Performance Optimization",
IEEE Computer Society, Proceedings of the
Third International ACM Symposium on FieldProgrammable Gate Arrays (FPGA95), 1995.
[9] K. Renovell, P. Faure, J.M. Portal, J. Figueras
and Y. Zorian, "IS-FPGA: A New Symmetric
FPGA Architecture With Implicit SCAN", IEEE
ITC International Test Conference, Paper 33.1,
0-7803-7169-0/2001.
[10] Areibi, G. Grewal, D. Banerji and P. Du
"Hierarchical FPGA Placement", CAN. J.
ELECT. COMPUT. ENG., VOL. 32, NO. 1,
Winter 2007.
ISBN: 978-960-474-262-2
116
