0% found this document useful (0 votes)
25 viewsML 4
ML UNIT 4 HANDWRITING NOTES
Uploaded by
SrinithiCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
25 viewsML 4
ML UNIT 4 HANDWRITING NOTES
Uploaded by
SrinithiCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 44
UNIT= 1V
NUEPRAL NETWORKS
PRECEPTRON!
The Perception 46 o Pec) fonoasd robwork
Woith One evtpot nebroy ‘ak ‘learn a Separating
“Perpone tm a poHesn Space
The ‘pn’
7 ‘¥ 0” Wmear .p,, Neoreng fol fonward
ona threshold output P
Separates Lemeai 7 heoron . The percapbon.
ty Seporabe tof rotor
Ss
inna dayer Dy )
© The * Precephon is Sd as forvoond
hebwork with ‘ono oulpub havo thot barns a
Separating hajpor perc. tr a pattern {pec
The ‘y” Iemeor Fr nevoons feoal
forward > ome bhrahold ootpot Fy revoon
«The pre ophor, Separates Leenaanhy separable
Se of patterns
4 SLP ts Dhe simplost hype of artical
nevrol —malsworky And dan ory oloasify le’
Seporabte codes with a binary target(i,o) x Wwe Cow Conneck ong Sere of MeCulloh,
pitts newrons bogthost in any ES Lhe «Pe ovrranae
of eue input Aoxgest of Me Culoch.-p its neovaons foods:
Forward ko one Ubput dower of MeGtloch- pitts
hours rocking 48 Pynown A perceptron .
ve Ary Siovgla dager! ood - forvoarrd pabwork
Consists of me os mooo ovtpvt noveons each of
Lghech Connected — with a facts lui 4 all
of ke ympots Xi ,
% The percaphon tb bem of Sionglo Loge
ashficcal nebwork wtbh only Ohe Aavon .
x The pertap bon te & nebwork tm Which
the navoor unlb Colealales phe Lernaay (pmbinehior
of ids peel Valed or boolaan expuls and foot
it Dhrogh a bireasholel | op The — penephen Somabime, repevned ne
o Thoehold logic Unt} CTLU) Since 14 disomeéneles
the dake : im. vohabhow bha Sam is
Greaker — thon Threeshald Vole .
seTn the Sirmplat Ge Bho cpolnoork. has
Only sre dnputs and sing solpot - The wtpot of
Nw wer, dg .
= L :
Y FZ worst
To Sienoly .
Vohe ane es dager preceptor tmHial (ath
cuca tecause jt does rot
hawe. poovious lage
3 TE som oll bho weighted Trypuks + Tf
the gum iis. kan Dha TProshold
Awenxdrx
x= impub dal d= produced output and desired votpot « 4)
x af bre ookpub — af tho Percy, ds
hen ue do’ mol taka ang athion .
¥TP phe ovtpot 18 tworretk ten dhe
Cae rg hus vechr jg Ww > W4AW
learmemg - ,
perce phon Leurenshed 4 a
) Soteck random Sample from -bretouing
DTP» dhe . Sead xg Coeds do -
notte —
STH. 2 eiticabien ig. Incomveck moeliky bea
woratt — Vector W she
. » Wie Wit 9d,cn> xicn)
4) Repeak che — protadyna unbil Pht
"entire. rrowentong sols is classified
omeeitd Motkilager percepien : : ©
ah Maki Powter Percephon (mip) bas
He sume evohne of a Single lavtey percaphors
Uotth te oy more dudolon Loupers
KAD mip is a nebook Of Simple roomy,
Called — pe graptions - ;
# A typicalty molt layes’ nebook. anda
of A ek of Sevsco nodes forming blo thpub
Ore. or mare whuictdin of Computaktoneal roday
and an ovtpot Jager nods .
# Lt ig mot possible bo find tuatlghts
Which erables Single Lasgo pescepiron to deal.
tort hon Lonearly Separobe problems Juha XOR
| Hiratns of darng in peripten # Linea
Sepawobility » ros
y Consider dro imput patteens (i, 9)
thovena Uassifie imby sos classes -
© yy Bach point with — enter Syralel of
X or 0D Meprasenbs a palterr With. Solr
of Value Cxi,¥o) » ©
Each pedfeon a classified imlo roo clasess
Noti@ — tPals plrwsec cloaca Can de Sparroded
Wibh a S fame L . “They ama know as
kmenry separable Petters .
4 hemor Sepawabilihy ropers do the. fact
thab claws of — pactteams with f)-démansi'mmal
Vector ¥= Cee ...p) Gan de Sepanated with
A Single — doetsion’ Qaface .’
¥ Tf ‘hoo classes of Petierns Can tbo
S2parrated bg Q chusim bomdaw, Tepresenfed by
the Limearr equation Akon Play ane Satd tp
be u Soparerble_: The Simpl mekwork
Con comectty olosslhy any posters
x Decesiun bomdony of Aoreacty Separohe
Classes (Can be dotermioad ertkor by Some acer
- pprocaduney OF by Soboiong Limaare eqbabi om Sys ler
| based om ‘yapresentatin, pattern of Gach Closer.
# TP Such dleu'sion bom
_ apes nop
exist then the bwO chssas are cad fo be
| Urea 'nsepawa.bla « @
eae love parable Prollams cannot *
Solued Dhe sinmplo nehwork,, mone Sophes Hi cofod
Archoleckmo SS maaded .
Activation Fictions :
Adiivation Andtsion oho known a
troms fer Anebions ts vsed ty mop input nodes
re OFF odes in corteum fahip,
The activation Prcbimn is tha most
Important Pecboy ‘nw poured nohoork, Ughich
aloceclod lohethok or rot a how will be ackvatiey
ST net end fremefemed to tha rex Layer
AcKVabiun tnebian helps in Normalire'
dhe Olpot — Hahracen, Otol OY -l4oy py halps
tn Bho prey, of bea. ck pro ebin clue hy thacy
Affe ren tic hl Property
Dow back: i Hi
Prpagation Loss Anctien
updased and Korean? (ee balps bho ms
Frodient- An essent Cunes bo Qchatio
Morera ,
thay Look
Dt basi cally awd Un cunry ravral
rebwot Dhak given inpot ov vevecvieng insmod an te yokbvant oy i. is hrrelevant .
Thae Qchvabion Rncbion rake bho mally
hebwork, Uo hue Gpeantsor soprosentakirod "
power bran a simgl Layer network only °
em ~ Merely 1s Indrooloced
Gpocuastic GRADIENT DESCENT |
| a Sito ot PP
Qik): Tw stochastic
real Selaceal
doa sek for
. Li
TE woeons cas Sinbad
with @ wraem probobi
Fra dienk Aostonis Pwo Supe
romdomly oe Whole
each iteration «
In qredtent Descont Shere is atom
Cole) “poten” Uhich clanstas to —_ foteL yvom ber
of Sampiss frre a dateset thab Is used for
vsebol for qty to The MEME in
a Wes Koley amd less Tomolous wamnor , Tha prblers
_ artsez When Your blob 15 big rf you shave 0 willinw Samples wh qo FD)
daboset so if you vse & typical Gradient
Descenk opbimizobion techratye 1 you wotl? shave te
Use abl of dhe ore million Samples doy Gmpohg
the ikerabien , Whib performing phe qraolent
Descenband cf has Jo be chro for awry
‘Nerabten unk dha mimiona are nached ,
Tb beournat Comp obcubtoratl Very expene) ve.
Uno pertorre This pichiem iS Shovel: ee
Cradionr Pesanb. Te van only a img’ Sa7mPO-
The Sampb is randomly Sofflad and Selected for
Pexformeng Ha tevabion.
Sad is a Varant of gradienk Descent
algorithn used fx optientzimg wading laa
els “4 a
wo.
AAVom
en
T+ is
© one example
Vr USS
Sinte sqp updates it hes sbio abi aly
0 escape from Lowel mimtana awd gp. Converoge
Ao global murtiowm « in Sorp it haw 2
a and
ability tp escape fom — Loaal mermnnrnn
ima +
Urverge 40 0 geo! mine
Due tv noi up dotes
Disadvantages :
The Updates in SbID ome neisy and
Jose luis Yariouee , tohicl tam wake bhe
Optimizabton procs ess Stable oud lod be
ERPOR Back PROPAStaTICN
Bo.cl,propagasbiien is o eat mabhod
Used for c mii Jaxer veoral vlrcerb :
elt . Dire Aolba ala
De ts cabled pha © gevawa.
TH is O Gradient daaconté wobtud Which
biwewnt she bobal Lqpoved emer of tha
Cofpur cempoied by tho he: -
The Back :
he Prope sabion Aleoytt Looby
| bhe Wiviwvow Vabe
| On wee : Of the emer freien
; Wetodnt Space, using a techn jue, colled | the. The yeignts Mak rmimioninn” ihte emg ©
Rundion tha te tonsidesed +o be a Solution to
the Aaowing prcbiann «
Beck poo parti or LK Q Systemic Wobtiod fry
training — vunbbiple pun. Pt is a olt2aHor
"4 gener
of Widyso-Heff — enset Connection aula. B07. of ANN
Applicabions uses beck proparcplrion -
Coreider a Sienpla navel network,
a> Nuveen das Suromiong Jenchion and
achivadcien jpnetino «
> Any mon dcnear froebl on Lohich ‘deffesenbioti,
Leer juhame and Umnoreases euersohena with
Sem an be vsed ag, achvalim Pmction . Brampla: hogitic froction, Are tangent. fciim, &)
“Huperbolléc Fangent activatton fnciten.
4 These activabion frockten mates ha molbi Large
nebwork bo howe greabed Topsasenkobional poves
Bimgle Aospes ebook only (when pun beinooe ty
us jnprodlucod «
Need of sudden agers:
#A nebwork with only hwo Joao Cpu
aed ompot) ' Com Yepmionk Dhe onpk wish
Whatewr raptacniabion almaacly exists (ro ha
Cnpub dako
# Ff the dalra os lis Contiryes oven
Admacerdy Separaba the dmnabe represenbaltion
ts lmumsistent and the mappd cannot bo fe
Joamned vsiog wo Jaspers C Laps ard cape )
Therefoxe dudolen Hoge) ome vied belraaon
Erput od svtpot ospy
werghls Connects vndk Duy oe BA
f the ocotpub of tha Unt Ud scaled by the
Vole of, Dhe, tonneching » weryhlr ond ot i foal
formoed. bo proovtde & portion Y the ack vation for
the milk im tra rey bigher loyor | Backpropegabion con be applind -b» an antien
-mevral mebnork with any rambo of Iuidden Jagers.
| The fraderina objectiio 1S bo odjut = Aho
fwrighb 25 sphab pha appliolion of a sek of
Fnpulk fprocacdnad bho asiinar) ovputs
eaasig protedime:
Tle mehooske Js usally Qrattmad with a
haber of tmpub- oulpub pairs
# Clenrate ucgfls randomly bo Srrabt
Tavdom Valve ( betk posi tive and
dob the mobwoorb is
of Weigh ls
¥ chose aq few
Seb -
# Apply the Impuk veckr Jo crsboork impol
CAletate . the crobwovh. Qutpuk *
7p Clulate phe emer tha. deffesons.
bre qrebivork dtp ard the desired ootpot
AAs Aho - esecighte of dee neboork jn a
Long Bisle mirininas PHS Ome
& Ropecds Steps 9 —b& Rr each pur of topat - RE
~ +) the eaevr for tho entrt
ahe kaw sels ual
ashen re aceoptably ow -
crease) Ao entre.
me stetrobed by forge vol
pols from the froumang.
bebrooer Feward pas anf backward pass: @
Back propagation hooral crobworks aroun ong Convo Deeg
i Signals mows drsard,
say one end fe cast OTT
Tn ate finned poss, the aabobbler 4
1s come} ovt, Jower by dose?
The bot of bho one
fo he pack layer
loyer Is bio inpot
Tn reswrse. pas
The owes of the —oulpu rewosn,
3 cae adjusted firs.b ime the Ayorrgeb Valin
of ench output: mewarr le auotlelle bo guide
the adijpsment of the axocdted a
dolla we - aa
Next be adjust ‘the sel of the
wriddle S~ ASB We ~~ arrdlolle Jager an
Te te tergelr Values , i} wrakas Bho problon, i)
Solechion of owomber . of Anddden ons :
The nemtat of dudden wit deperds on Lhe
Mmerber of Unpuk vniks -
gNever choow h bo be ome bhai noice
phe umbes of —impuf ons -
You cam Lead p potters of T elements 1%
dog, p deden wl
eo Enane Bak we
st wom frenamelrg excwmples
yp \Peadne, exhachim veybnes fewer hidden
unite than Imps *
; of dajrimted
Lear mond a "ee input +
Jeske hove ae Least Yo tras
inpub Seq vines more duiddan
The rmeber of) eden units tepinad for
a classifiot'on soak —inueaases with cbhe rumbor ¥
Clouser im tha faak-
' tre frauen’
¥ ovk oq domape >
Hames »
Eecroes INFLUENCING Breck PROPR GIATICN TRAINING
Bias: Nenvorks with biases can repoasent
Telabtonedup aobveer impus ad ovlpols mere easily
Phom nelworks witht bieses Adds Qa bias dso each pewoon os vsvally
Chosjvablo wv offer tha ovigum of the achvahiow
Ruchion °
The wocghir of the bias 13 tratmable
Sinmvilor po ueighh expect tak Input is always
+l,
Momentom !
The Use af pwamenbmn ou homens She Stebibiy
Of dhe few poouss. Momentwn used or
the reworulrg Process going Im the Same gene
dvrectton analogos to dhe way thod Momantw
el mving — object behaves
In propaga hen— with momentm, the wera
Chowge fs a Cembirabon fg the Omnent gradient
ond — the PRIOR Grooluent -
cd Voon i ‘
Ct is Siwmnple , feast curd cosy fo program »
ca is Pbxibe- ,
A Standaxd ap poach 4 wosks eficientty
No Need bo, dhe prior hucvlecly. abut the
Disadvan '
Ltr possibly be Sensitive fo haley dake te
i tregubamhy
thls da dal srellamb- on Pha
Tha perfecrnans of this Ia duighly Need — evccatsive fire fox Arcata
The eed for a mabix~ bated mudtod fr
of mene batch -
back frepag ation Imlead
Shallro Nelroosks +
awmbes of Aovsos, jm he hovral nohvork phate
UUme Swale nombert of daugess usatly veqrodeal
cH daring a dingl cu'dolen daxper cin cloap
nevral aebworks refer bo meoval malworhes Dhote
Arowe ww Lbiple hu'ddow Jaxer.
Both types of helworke pesform tartatn
Foss potter bRain ‘The other and
