FANNTool 1.

0 User's Guide
FANNTool 1.0
User's Guide
Birol
[email protected]
Michael Schaale
Michael.Schaale@u!berli".de
#age 1 o $1
FANNTool 1.0 User's Guide
Introduction
A" ar%iicial "eural "e%&ork 'ANN() usuall* called +"eural "e%&ork+ 'NN() is a ma%hema%ical
model or com,u%a%io"al model %ha% %ries %o simula%e %he s%ruc%ure a"d-or u"c%io"al as,ec%s
o biological "eural "e%&orks. A" ANN ,erorms a "o"!,arame%ric "o"!li"ear mul%i.aria%e
mul%i,le regressio".
The Fas% Ar%iicial Neural Ne%&ork librar* 'FANN( is a ree o,e" source "eural "e%&ork
librar*) &hich im,leme"%s mul%ila*er ar%iicial "eural "e%&orks i" / a"d su,,or%s bo%h ull*
a"d s,arsel* co""ec%ed "e%&orks. /ross!,la%orm e0ecu%io" i" bo%h i0ed a"d loa%i"g ,oi"%
is su,,or%ed. 1% i"cludes a rame&ork or eas* ha"dli"g o %rai"i"g da%a se%s. 1% is eas* %o
use) .ersa%ile) &ell docume"%ed) a"d as%. #2#) /33) .N4T) Ada) #*%ho") 5el,hi) 6c%a.e)
7ub*) #rolog #ure 5a%a a"d Ma%hema%ica bi"di"gs are a.ailable.
FANNTool is a GU1 %o %he FANN librar* &hich allo&s i%s eas* usage &i%hou% %he "eed o
,rogrammi"g. This %ool e"ables 8ou %o9
... ,re,are %he da%a i" FANN librar* s%a"dard
... desig" a" ANN
... %rai" %he desig"ed ANN
... %es% %he %rai"ed ANN
... ru" %he %rai"ed ANN
#age $ o $1
FANNTool 1.0 User's Guide
Data example #1 : Time series of sun spots
:e%'s lear" more abou% FANNTool b* &orki"g %hrough a" e0am,le9 &e &a"% %o ,redic% %he
mo"%hl* mea" o su"s,o%s b* usi"g 'e0%ra,ola%i"g rom( ,as% .alues. The da%a are gi.e" i"
The AS/11 ile 'SunSpotsRaw-1980-2006.txt' lis%s li"e!b*!li"e %he mo"%hl* mea"s o
su"s,o%s rom ;a"uar* 1<=0 %o 6c%ober $00> '? @$$ .alues(.
No& lau"ch FANNTool a"d selec% Data Processn! %o load %he da%a ile
'SunSpotsRaw-1980-2006.txt') %he T"e Seres Data Processn! dialog o,e"s.
/heck %he radio bu%%o" T"e seres a"d look a% a ,lo% mea" o su" s,o%s .s. %ime.
Data Processn! reads AS/11 ra& da%a a"d ...
... 'o,%io"all*( scales %hem 9 ANN ge"erall* ,reer da%a &i%hi" a ra"ge o '0) 1( or '!1) 1(
%hus %he real!&orld .alues usuall* "eed %o be scaled
... ,roduces a FANN com,a%ible da%a orma%.
The FANN librar* uses a sim,le bu% i0ed AS/11 da%a orma%(
The ile mus% be orma%%ed like
"umA%rai"Ada%a "umAi",u%A"odes "umAou%,u%A"odes
"umAi",u%A"odes i",u% da%a 'se,ara%ed b* s,ace( se% B1
#age @ o $1
FANNTool 1.0 User's Guide
"umAou%,u%A"odes ou%,u% da%a 'se,ara%ed b* s,ace( se% B1
.
.
"umAi",u%A"odes i",u% da%a 'se,ara%ed b* s,ace( se% B"umA%rai"Ada%a
"umAou%,u%A"odes ou%,u% da%a 'se,ara%ed b* s,ace( se% B"umA%rai"Ada%a
For exa"ple
# $nput 1 %utput wt& 2 'ata sets(
2 3 1
1 1 1
0
1 0 1
1
... s,li%s %he da%a i"%o a %es% a"d a %rai"i"g da%a se% 'o,%io"(
... shules %he da%a se%s 'o,%io"(
For %he da%a se% 'SunSpotsRaw-1980-2006.txt' &e make %he ollo&i"g se%%i"gs9
Number of Input dim. 9 or e0am,le '>' mea"s %ha% > mo"%hl* mea" .alues are i",u% &hile
%he se.e"%h ser.es as ou%,u%
Minimum Output Value 9 desired Mi"imum 6u%,u% .alue a%er Scali"g 'here9 0.0(
Maximum Output Value 9 desired Ma0imum 6u%,u% .alue a%er Scali"g 'here9 1.0(
Fraction of data for training 9 Frac%io" o %he da%a se% %ha% is used or %he "e%&ork %rai"i"g
'here9 0.7) i.e. 70 C o %he da%a are alloca%ed or %rai"i"g(
Fraction of data for testing 9 Frac%io" o %he da%a se% %ha% is used or %he "e%&ork %es%i"g
'here9 0.@) i.e. @0 C o %he da%a are alloca%ed or %rai"i"g(
Suffle 9 shule order o %he da%a) i.e. scramble %he da%a seDue"ce
Scale 9
For %his e0am,le
mi" ? 0.0) ma0 ? $00.@ 'es%ima%ed rom %he da%a(
'5esired( Mi"imum 6u%,u% Ealue ? 0
'5esired( Ma0imum 6u%,u% Ealue ? 1
Scaled Ealue ? '1",u% Ealue ! Mi"imum( - 'Ma0imum ! Mi"imum (
For %his e0am,le9
#age F o $1
FANNTool 1.0 User's Guide
Scaled Ealue ? '1",u% Ealue G 0)0( - '$00.@ ! 0.0(
A"d %he re.erse &a* 'descali"g(9
1",u% Ealue ? 'Scaled Ealue H'$00.@ ! 0.0(( 3 0.0
1" %he T"e seres Data Processn! dialog *ou ca" s&i%ch be%&ee" %&o %*,es o gra,hs9
!1" #istogram !rater crude s$etc"
!%" &'( plot
Data File Name9 4"%er ,a%h a"d ile "ame s%em or %he s%orage o %he FANN orma%%ed
%rai"i"g a"d %es% da%a. For e0am,le 9 '%ut' ,roduces
'%ut-tran.'at' ( Trann! Data Fle
'%ut-test.'at' ( Test Data Fle
'%ut-scale.txt' ( Text Fle contan Scaln! Para"eters
#age I o $1
FANNTool 1.0 User's Guide
Data example #% : multiple regression
There is a a"o%her 7a& da%a orma% &hich is ma%ri0!like. 4ach li"e i"cludes bo%h i",u% a"d
ou%,u% da%a. 2ere %he ou%,u% da%a are co"%ai"ed i" %he las% colum"s.
1" %he e0am,le da%a ile 'Par)nsonRaw.txt' ile %here are $@ colum"s) &here %he irs% $$
colum"s are %he i",u% dime"sio"s a"d %he $@
rd
colum" co"%ai"s %he ou%,u% da%a.
The da%ase% &as crea%ed b* Ma0 :i%%le o %he U"i.ersi%* o 60ord) i" collabora%io" &i%h %he
Na%io"al /e"%re or Eoice a"d S,eech) 5e".er) /olorado) &ho recorded %he s,eech
sig"als. The origi"al s%ud* ,ublished %he ea%ure e0%rac%io" me%hods or ge"eral .oice
disorders.
The ile co"sis%s o $@ eleme"% AS/11 records &hose ields ha.e %he ollo&i"g ,h*sical
mea"i"g9
)ec Variable Meaning
1 M5E#9Fo'2J( A.erage .ocal u"dame"%al reDue"c*
$ M5E#9Fhi'2J( Ma0imum .ocal u"dame"%al reDue"c*
@ M5E#9Flo'2J( Mi"imum .ocal u"dame"%al reDue"c*
F M5E#9;i%%er'C( K
I M5E#9;i%%er'Abs(
> M5E#97A# Se.eral measures o .aria%io" i" u"dame"%al reDue"c*
7 M5E#9##L
= ;i%%er955# -
< M5E#9Shimmer K
10 M5E#9Shimmer'dB(
11 Shimmer9A#L@
1$ Shimmer9A#LI Se.eral measures o .aria%io" i" am,li%ude
1@ M5E#9A#L
1F Shimmer955A -
1I N27 Measure B1 o ra%io o "oise %o %o"al com,o"e"%s i" %he .oice
1> 2N7 Measure B$ o ra%io o "oise %o %o"al com,o"e"%s i" %he .oice
17 7#54 No"li"ear d*"amical com,le0i%* measure B1
1= 5$ No"li"ear d*"amical com,le0i%* measure B$
1< 5FA Sig"al rac%al scali"g e0,o"e"%
$0 S,read1 K
$1 S,read$ No"li"ear measures o u"dame"%al reDue"c* .aria%io"
$$ ##4 -
$@ S%a%us 2eal%h s%a%us o %he subMec% 'o"e( ! #arki"so"'s) 'Jero( G heal%h*
#age > o $1
FANNTool 1.0 User's Guide
:au"ch FANNTool* selec% Data Processn! a"d load %he 'Par)nsonRaw.txt' da%a ile.
1" %his &i"do& '&hich "o& looks a bi% diere"% rom %he da%a e0am,le B1( &e mus% s,eci*
%he "umber o ou%,u% dime"sio"s '? B o ou%,u% "odes(.
Number of Output dim* 9 or %his e0am,le '1' . 1% ma* be grea%er) i.e. %he las% " colum"s
are used as ou%,u% .alues &hile %he da%a i" %he remai"i"g colum"s ser.e as i",u% .alues.
Item No 9 4ach colum" o %he da%a ma%ri0 '? i",u% dime"sio"( ca" be scaled se,ara%el*.
The remai"i"g bu%%o"s-i",u% ields are %he same as i" %he ,recedi"g da%a e0am,le B1.
#age 7 o $1
FANNTool 1.0 User's Guide
Setting up a net+or$ for te sun spot time series data
A%er ha.i"g co".er%ed %he ra& da%a i"%o a FANN sui%able da%a orma% o"e ma* ,roceed
&i%h %he dei"i%io" o %he "e%&ork.
Firs% load %he %rai"i"g a"d %es% da%a 9
File , Open Train Data File 9 :oads %rai"i"g da%a
File , Open Test Data File 9 :oads %es% da%a
A%er %he readi"g o %he iles %he "umber o i",u% "odes '?dime"sio"s( a"d ou%,u% "odes is
sho&" 'a"d remai"s i0ed(.
#age = o $1
FANNTool 1.0 User's Guide
Ne0% %he hidde" la*er-s has-ha.e %o be s,eciied.
This "umber ca" o"l* be al%ered be%&ee" @ a"d I) %hus %he "umber o hidde" la*ers is
co"s%rai"ed %o 1) $ or @.
# of -a.er 9 Number o la*ers ? 1",u% '1( 3 2idde" '1 %o @( 3 6u%,u% '1(
so %here is a mi"imum o @ la*ers &i%h 1 hidde" la*er a"d a ma0imum o I la*ers &i%h @
hidde" la*ers. '7em.9 a""lib su,,or%s e.e" more hidde" la*ers bu% %here is "o "eed or i%(.
#id* la.er 1 9 B o "euro"s i" %he 1
s%
hidde" la*er
#id* la.er % 9 B o "euro"s i" %he $
"d
hidde" la*er ' B o :a*er N F (
#id* la.er / 9 B o "euro"s i" %he @
rd
hidde" la*er ' B o :a*er ? I (
Training Metod 9 8ou ca" choose o"e o %he ollo&i"g %rai"i"g algori%hms9
FANN+TRA$N+$N,R-.-NTA/( S%a"dard back,ro,aga%io" algori%hm i" ,a%%er"
mode9 %he &eigh%s are u,da%ed a%er each %rai"i"g ,a%%er". This mea"s %ha% %he
&eigh%s are u,da%ed ma"* %imes duri"g a si"gle e,och. For %his reaso" some
,roblems) &ill %rai" .er* as% &i%h %his algori%hm) &hile o%her more ad.a"ced ,roblems
&ill "o% %rai" .er* &ell.
FANN+TRA$N+0AT,1( S%a"dard back,ro,aga%io" algori%hm i" ba%ch mode9 %he
&eigh%s are u,da%ed a%er calcula%i"g %he mea" sDuare error or %he &hole %rai"i"g
se%. This mea"s %ha% %he &eigh%s are o"l* u,da%ed o"ce duri"g a" e,och. This raises
%he ,roblem %he algori%hm &ill co".erge slo&er. Bu% si"ce %he mea" sDuare error is
calcula%ed more correc%l* %ha" i" i"creme"%al %rai"i"g) some ,roblems &ill reach a
be%%er solu%io"s &i%h %his algori%hm.
FANN+TRA$N+RPR%P( A more ad.a"ced ba%ch %rai"i"g algori%hm &hich achie.es
good resul%s or mos% ,roblems. The 7#76# %rai"i"g algori%hm is ada,%i.e) a"d does
%hereore "o% use a i0ed lear"i"gAra%e. Some o%her ,arame%ers ca" ho&e.er be se%
%o cha"ge %he &a* %he 7#76# algori%hm &orks) bu% i% is o"l* recomme"ded or users
&i%h i"sigh% i" ho& %he 7#76# %rai"i"g algori%hm &orks. The 7#76# %rai"i"g
algori%hm is described b* O7iedmiller a"d Brau") 1<<@P) bu% %he ac%ual lear"i"g
algori%hm used here is %he i7#76#! %rai"i"g algori%hm &hich is described b* O1gel a"d
2uske") $000P &hich is a .arie"% o %he s%a"dard 7#76# %rai"i"g algori%hm.
FANN+TRA$N+23$,4PR%P( A more ad.a"ced ba%ch %rai"i"g algori%hm &hich
achie.es good resul%s or ma"* ,roblems. The Duick ,ro,aga%io" 'Duick,ro,( %rai"i"g
algori%hm uses %he lear"i"gAra%e ,arame%er alo"g &i%h o%her more ad.a"ced
,arame%ers) bu% i% is recomme"ded %o cha"ge %hese ad.a"ced ,arame%ers b* users
&i%h i"sigh% i" ho& %he Duick,ro, %rai"i"g algori%hm &orks o"l*. The Duick,ro,
%rai"i"g algori%hm is described b* OFahlma") 1<==P.
#age < o $1
FANNTool 1.0 User's Guide
0cti1ation Function #idden 9 2idde" :a*er Ac%i.a%io" Fu"c%io"
0cti1ation Function Output 9 6u%,u% :a*er Ac%i.a%io" Fu"c%io"
The gra,hic belo& sho&s a ske%ch o %he sigmoid u"c%io" &i%h diere"% '0.$I) 0.I) 1(
s%ee,"ess .alues.
1" %he Fne Tunn! sec%io" *ou &ill see %he Steepness o,%io". The igure abo.e illus%ra%es
%he eec% o .ar*i"g %his .alue o" %he s%ee,"ess o %he %ra"si%io" rom %he u"c%io"'s 'here9
FANNAS1GM615( mi"imum %o i%s ma0imum .alue. The smaller %he s%ee,"ess becomes
%he more li"ear %he %ra"ser u"c%io" beha.es. A"d re.ersel* %he larger %he s%ee,"ess is
chose" %he more %he smoo%h sigmoid u"c%io" a,,roaches a s%e, u"c%io". For more
de%ails see here. FANNTool allo&s %o choose diere"% s%e, u"c%io"s rom a huge .arie%*
'see FANN docume"%a%io" h%%,9--lee"isse".dk-a""-h%ml-iles-a""Ada%a!
h.h%mlBa""Aac%i.a%io"u"cAe"um(.
#age 10 o $1
FANNTool 1.0 User's Guide
Net+or$ training
Neural Net+or$ , Detect , Optimum Training 0lgoritm: 4ach ,ossible
Trai"i"g Algori%hm is used or se.eral e,ochs. All o%her ,arame%ers are i0ed a"d %he
&eigh% i"i%ialiJa%io" is ide"%ical. The Trai"i"g Algori%hm sho&i"g %he lo&es% MS4 is ,icked.
Neural Net+or$ , Detect , Optimum 0cti1ation Functions9 4ach ,ossible
Ac%i.a%io" Fu"c%io" is used or se.eral e,ochs. All o%her ,arame%ers are i0ed a"d
%he &eigh% i"i%ialiJa%io" is ide"%ical. The Ac%i.a%io" Fu"c%io" *ieldi"g %he lo&es% MS4 is
,icked.
Neural Net+or$ , Train , Normal9 Fi0ed %o,olog* %rai"i"g. The siJe a"d %o,olog*
o %he ANN is de%ermi"ed i" ad.a"ce a"d %he %rai"i"g al%ers %he &eigh%s i" order %o
mi"imiJe %he diere"ce be%&ee" %he desired ou%,u% .alues a"d %he ac%ual ou%,u% .alues.
This ki"d o %rai"i"g is su,,or%ed b* a""A%rai"Ao"Ada%a.
Neural Net+or$ , Train , 2ascade9 4.ol.i"g %o,olog* %rai"i"g. The %rai"i"g
s%ar%s &i%h a" em,%* ANN) o"l* co"sis%i"g o i",u% a"d ou%,u% "euro"s. 2idde"
"euro"s a"d co""ec%io"s are added duri"g %rai"i"g) i" order %o reach %he same goal as or
i0ed %o,olog* %rai"i"g. This ki"d o %rai"i"g is su,,or%ed b* FANN /ascade Trai"i"g.
Stop Function 9 S%o, cri%erio" used duri"g %rai"i"g.
FANNAST6#FUN/AMS49 S%o, cri%erio" is Mea" SDuare 4rror 'MS4( .alue.
FANNAST6#FUN/AB1T9 S%o, cri%erio" is %he "umber o bi%s %ha% ail. 'Number o bi%s'
mea"s %he "umber o ou%,u% "euro"s &hich dier more %ha" %he bi% ail limi%.
3poc 4et+een )eports 9 The "umber o e,ochs be%&ee" ,ri"%i"g %he s%a%us %o %he :og
'ile( a"d Gra,hic '&i"do&(
Max 3poc 9 The ma0imum "umber o %rai"i"g e,ochs
#age 11 o $1
FANNTool 1.0 User's Guide
The Mai" Qi"do& a%er %he %rai"i"g &i%h %he Gra,h %ab ....
.... a"d &i%h %he :og %ab o,e"ed.
#age 1$ o $1
FANNTool 1.0 User's Guide
Tabs in te main +indo+
-og 'see abo.e(
All %he %e0% ou%,u% &ri%%e" %his sec%io".
,lear /o! clears %he log scree"
Sa5e /o! sa.es %he co"%e"% o %he :og &i"do& %o a %e0% ile.
%pen /o! o,%io"all* loads %he log rom a 'sa.ed( %e0% ile
5rapic 'see abo.e(
#lo%s a MS4 .s. 4,och gra,h duri"g %he %rai"i"g. 1 6/S is "o% ac%i.a%ed %he ,lo%%ed MS4
is %he MS4 or %he %rai"i"g da%a o"l*. R 6/S is ac%i.a%ed %he mea" o %he MS4 or %he
%rai"i"g da%a a"d %he MS4 or %es%i"g is used. 8ou ca" also see %he "umeric MS4 .alues a%
%he bo%%om o %he gra,hic &i"do&.
2ascade Tuning
/ascade %rai"i"g diers rom ordi"ar* %rai"i"g i" %he se"se %ha% i% s%ar%s &i%h a" em,%*
"eural "e%&ork a"d %he" adds "euro"s o"e b* o"e) &hile i% %rai"s %he "eural "e%&ork. The
mai" be"ei% o %his a,,roach) is %ha% *ou do "o% ha.e %o guess %he "umber o hidde"
la*ers a"d "euro"s ,rior %o %rai"i"g) bu% cascade %rai"i"g ha.e also ,ro.ed be%%er a%
sol.i"g some ,roblems.
The basic idea o cascade %rai"i"g is %ha% a "umber o ca"dida%e "euro"s are %rai"ed
se,ara%e rom %he real "e%&ork) %he" %he mos% ,romisi"g o %hese ca"dida%e "euro"s is
i"ser%ed i"%o %he "eural "e%&ork. The" %he ou%,u% co""ec%io"s are %rai"ed a"d "e&
ca"dida%e "euro"s is ,re,ared. The ca"dida%e "euro"s are crea%ed as shor%cu% co""ec%ed
"euro"s i" a "e& hidde" la*er) &hich mea"s %ha% %he i"al "eural "e%&ork &ill co"sis% o a
"umber o hidde" la*ers &i%h o"e shor%cu% co""ec%ed "euro" i" each. For de%ailed
i"orma%io" abou% %he ,arame%ers co"sul% %he FANN docume"%a%io"
'h%%,9--lee"isse".dk-a""-h%ml-iles-a""Acascade!h.h%mlB#arame%ers(.
#age 1@ o $1
FANNTool 1.0 User's Guide
FineTuning
1" %his %ab se.eral ,arame%ers &hich are li"ked %o %he lear"i"g s%age ca" be 'i "ecessar*(
ada,%ed a"d i"e!%u"ed.
#age 1F o $1
FANNTool 1.0 User's Guide
The mea"i"g o %he ,arame%ers is briel* summariJed here9
2uc)prop Deca6 Factor9 The deca* is a small "ega%i.e .alued "umber &hich is %he ac%or
%ha% %he &eigh%s should become smaller i" each i%era%io" duri"g Duick ,ro,aga%io" %rai"i"g.
This is used %o make sure %ha% %he &eigh%s do "o% become %oo high duri"g %rai"i"g.
The deaul% deca* is !0.0001.
2uc)prop .u Factor9 The mu ac%or is used %o i"crease a"d decrease %he s%e,!siJe duri"g
Duick,ro, %rai"i"g. The mu ac%or should al&a*s be abo.e 1) si"ce i% &ould o%her&ise
decrease %he s%e,!siJe &he" i% &as su,,ose %o i"crease i%.
The deaul% mu ac%or is 1.7I.
RPR%P $ncrease Factor9 The i"crease ac%or is a .alue larger %ha" 1) &hich is used %o
i"crease %he s%e,!siJe duri"g 7#76# %rai"i"g.
The deaul% i"crease ac%or is 1.$.
RPR%P Decrease Factor9 The decrease ac%or is a .alue smaller %ha" 1) &hich is used %o
decrease %he s%e,!siJe duri"g 7#76# %rai"i"g.
The deaul% decrease ac%or is 0.I.
RPR%P .n"u" Step-S7e9 The mi"imum s%e,!siJe is a small ,osi%i.e "umber
de%ermi"i"g ho& small %he mi"imum s%e,!siJe ma* be.
The deaul% .alue del%a mi" is 0.0.
RPR%P .ax"u" Step-S7e9 The ma0imum s%e,!siJe is a ,osi%i.e "umber de%ermi"i"g
ho& large %he ma0imum s%e,!siJe ma* be.
The deaul% del%a ma0 is I0.0.
Desre' -rror 8.S-99 desired mea" sDuared error o %he "e%&ork.
0t Fal /"t9 The "umber o ail bi%s) i.e. %he "umber o ou%,u% "euro"s &hich dier more
%ha" %he bi% ail limi%.
1''en Act5aton Steepness9
%utput Act5aton Steepness9
The s%ee,"ess o a" ac%i.a%io" u"c%io" describes ho& as% %he ac%i.a%io" u"c%io" goes
rom %he mi"imum %o %he ma0imum. A large .alue *ields a s%ee, ac%i.a%io" u"c%io".
Qhe" %rai"i"g "eural "e%&orks &here %he ou%,u% .alues should be a% %he e0%remes 'usuall*
0 a"d 1) de,e"di"g o" %he ac%i.a%io" u"c%io"() a s%ee, ac%i.a%io" u"c%io" ca" be used
'e.g. 1.0(.
The deaul% ac%i.a%io" s%ee,"ess is 0.I.
#age 1I o $1
FANNTool 1.0 User's Guide
/earnn! Rate9 The lear"i"g ra%e is used %o de%ermi"e ho& aggressi.e %rai"i"g should be
or some o %he %rai"i"g algori%hms 'FANNAT7A1NA1N/74M4NTA:)
FANNAT7A1NABAT/2) FANNAT7A1NALU1/S#76#(. 5o ho&e.er "o%e %ha% i% is "o% used
i" FANNAT7A1NA7#76#.
The deaul% lear"i"g ra%e is 0.7
.o"entu"9 The lear"i"g mome"%um ca" be used %o s,eed u,
FANNAT7A1NA1N/74M4NTA: %rai"i"g. A %oo high mome"%um &ill ho&e.er "o% be"ei%
%rai"i"g. Se%%i"g mome"%um %o 0 &ill be %he same as "o% usi"g %he mome"%um ,arame%er.
The recomme"ded .alue o %his ,arame%er is be%&ee" 0.0 a"d 1.0.
The deaul% mome"%um is 0.
,onnecton Rate9 The co""ec%io" ra%e co"%rols ho& ma"* co""ec%io"s %here &ill be i" %he
"e%&ork. 1 %he co""ec%io" ra%e is se% %o 1) %he "e%&ork &ill be ull* co""ec%ed) bu% i i% is se%
%o 0.I o"l* hal o %he co""ec%io"s &ill be se%.
S&u::le Tran Data9 Shules %rai"i"g da%a) ra"domiJi"g %he order. This is recomme"ded or
i"creme"%al %rai"i"g) &hile i% ha.e "o i"lue"ce duri"g ba%ch %rai"i"g.
$ntal7e t&e we!&ts 8;'row<N!u6en Al!.99 1"i%ialiJe %he &eigh%s usi"g %he Qidro& 3
Ngu*e"Ts algori%hm. This u"c%io" beha.es similar %o :ann+ran'o"7e+we!&ts89. 1% &ill use
%he algori%hm de.elo,ed b* 5errick Ngu*e" a"d Ber"ard Qidro& %o se% %he &eigh%s i" such
a &a* as %o s,eed u, %rai"i"g. This %ech"iDue is "o% al&a*s successul) a"d i" some cases
i% ca" be less eicie"% %ha" a ,urel* ra"dom i"i%ialiJa%io".
-rror Functon9 4rror u"c%io" %o be mi"imiJed duri"g %rai"i"g.
FANNA47767FUN/A:1N4A79 S%a"dard li"ear error u"c%io".
FANNA47767FUN/ATAN29 Ta"h error u"c%io") usuall* be%%er bu% ca" reDuire a lo&er
lear"i"g ra%e. This error u"c%io" aggressi.el* %arge%s ou%,u%s %ha% dier much rom %he
desired .alue) &hile "o% %arge%i"g ou%,u%s %ha% dier sligh%l* o"l*. This error u"c%io" is "o%
recomme"ded or cascade %rai"i"g a"d i"creme"%al %rai"i"g.
%5ertrann! ,auton S6ste" 8%,S99 Qhe" selec%ed a %es% da%a '?da%a bei"g i"de,e"de"%
o %he ANN %rai"i"g( ile has %o be loaded %oo. 5uri"g %he %rai"i"g *ou ca" &a%ch %he %es%
da%a's a"d %he %rai"i"g da%a's MS4 .alues. The %rai"i"g &ill s%o, i %he o.erall MS4 '?MS4
6/S() i.e. %he ari%hme%ic mea" o %he %es% a"d %rai"i"g da%a MS4)
MS4 6/S ? 'MS4'Trai"i"g ( 3 MS4 'Tes%(( - $
&ill sho& a" i"crease. A" i"creasi"g MS4 6/S i"dica%es a decreasi"g ge"eraliJa%io"
ca,abili%* o %he "e%&ork. 5uri"g %he %rai"i"g %he gra,h o %he MS4 6/S is ,lo%%ed i" %he
=rap&c &i"do&.
#age 1> o $1
FANNTool 1.0 User's Guide
Sa1ing trained Neural Net+or$s
A%er %he regular e"d o a %rai"i"g or &he" i% &as s%o,,ed b* %he user) %he ,rogram
asks %he user &e%her %o sa.e %he "e%&ork9
Qhe" a"s&eri"g &i%h '*es' %he Sa5e ANN dialog o,e"s. Sim,l* choose %he ANN 8ou &a"%
%o sa.e %he" click Sa5e ANN(
/atest ANN9 ANN &he" %rai"i"g s%o,,ed
.n"u" Trann! .S-9 mi"imum MS4 .alue &i%h Trai"i"g 5a%a 5uri"g %rai"i"g
,rocess
.n"u" Testn! .S-9 mi"imum MS4 .alue &i%h Tes%i"g 5a%a 5uri"g %rai"i"g
,rocess 'i 6/S &as ac%i.a%ed(
.n"u" %,S .S-9 mi"imum mea" MS4 .alue &i%h Tes%i"g a"d Trai"i"g 5a%a
5uri"g %rai"i"g ,rocess 'i 6/S &as ac%i.a%ed(
A %rai"ed "e%&ork ca" be Run i"side FANNTool i" %&o &a*s
)un 9 6"e ca" Run a %rai"ed ANN &i%hi" %he FANNTool b* usi"g $ me%hods9
Firs% me%hod9 Neural Net+or$ , )un , Normal 9 Sim,l* selec% sa.ed ANN ile. The"
%he Run ANN dialog o,e"s. Add i",u% .alues a"d %he" click %he Run bu%%o" a"d see %he
a"s&er o %he ANN i" %he ou%,u% sec%io". All i",u%s mus% be scaled. 6u%,u% recei.ed %here
are descaled %o real .alues.
#age 17 o $1
FANNTool 1.0 User's Guide
Seco"d me%hod9 Qri%e scaled da%a i" a FANN com,a%ible da%a orma% ile 'see abo.e(.
No& load %his ile %o %he %es% da%a sec%io"'File , Open Test Data File(. No& use %he
s%ored ANN9 Neural Net+or$ , )un , +it File. The resul% o %he recall is &ri%%e" %o %he
/o! scree".
#age 1= o $1
FANNTool 1.0 User's Guide
Neural Net+or$ Information
6b%ai" i"orma%io" abou% %he sa.ed ANN iles9 sim,l* selec% a" ANN ile a"d %he
Neural Networ) n:or"aton dialog o,e"s u, a"d 8ou ca" see %he "e%&ork's s%ruc%ure)
some ,arame%ers a"d selec%ed &eigh% o %he ANN .
#age 1< o $1
FANNTool 1.0 User's Guide
)ecall result for te sun spot time series example
A%er %he %rai"i"g o %he su" s,o% da%a %he "e%&ork &as sa.ed %o %he ile 'SunSpots.net'.
Ne0% %he &hole'U( ra& da%a &as im,or%ed agai" a"d s%ored i" a scaled .ersio" &i%hou%'U(
shuli"g %he da%a. The Fracton o: Data :or Trann! ield &as se% %o 1.0 ,rior %o e0,or%.
Fi"all* %he '>-tran.'at' da%a se% &as loaded i"%o %he Test Data Fle ield. Ne0% Neural
Net+or$ , )un , +it File &as selec%ed a"d %he %rai"ed "e%&ork 'SunSpot.net' &as
selec%ed. The "e%&ork's res,o"ses %oge%her &i%h %he desired res,o"ses a,,eared i" %he
:og &i"do& a"d &ere s%ored i" a" AS/11 da%a ile. Those da%a ma*be im,or%ed i"%o .S
-xcel or %pen%::ce ,alc or a rescali"g or %he* ma* be ,lo%%ed direc%l* &i%h !nuplot 'see
belo&(.
#age $0 o $1
FANNTool 1.0 User's Guide
2oncluding remar$s
The ra"ge o ,ossible a,,lica%io"s o Ar%iicial Neural Ne%&orks is huge a"d "o&ada*s %he*
are used Dui%e reDue"%l*. The so%&are FANNTool &ill ho,eull* e"courage more scie"%is%s
a"d "o"!scie"%is%s %o look i"%o %he subMec%. This guide briel* describes ho& %o use %his
,rogram a"d i% does "o% claim %o be com,le%e or absolu%el* clear "or &ill %he so%&are be
bug!ree. 1 8ou e"cou"%er a"* ,roblems or i 8ou ha.e a"* sugges%io"s ,lease eel ree %o
co"%ac% us b* email) &e &ill be ha,,* i &e ca" hel, 8ou.
All da%a a"d "e% iles me"%io"ed abo.e are dis%ribu%ed &i%h %his docume"%a%io". Qe
recomme"d %o use %hese e0am,les %o lear" more abou% FANNTool a"d %o ,rac%ice i%
i"%e"si.el*.
More e0am,les ca" be ,ulled rom m* Blog &hich is &ri%%e" i" %urkish'U( la"guage
:e%%er recog"i%io"
Nodule cou"% de%ermi"a%io" i" s,herical cas% iro"s
/lassiica%io" o hear% disease
4*e i"di"g b* usi"g a" ANN
Blood do"a%io" ,redic%io" b* usi"g ANN
ANN ,la*i"g Tic Tac Toe
FANNTool ,ublica%io"s
BlueSid
#age $1 o $1