s l o n and DreP Leort4

odule-3: Ntugal Nttwosh, (omputs

(haptet kasë No Derp hraning: Long Shost trapmmóriy. *i***5 ** * i**''*'*"''****J

MLP bra_Nrota CaS ainputand qiues

us Output Simiiaaly CNN takc imaqe
0g cinput_and uoith t h hup 60-
Lunvolutloo lautand ma poolinq nate
output but oha my inpuBis
Squtmcr SA Lohati¢ my input A
Srquemce O oomds AmaLoA duod
Aeiro) ia moSt Lanquaqr thr
rquemcr mettruseg Inis phont-
is dast iafhis- xampi1 i
iSayeast mtan1s- am_adsning
to hon SotniStquemcr
impoatant_hae En many ttat
mocessinq allated task Srqurmcyis
aY impoxtant m+ne tchniquLP_
bag o nmd IeIOR osd av rc_ur
Lomplitlu- oliscandinq he Sequemt in¢onmatioi
So RNNis
isNeu Popula techniQuR,
uoherY u atusingSrqutmue in¢oimation
ushar NNCun- brusrd
In. maaylatunal tanquage astesin
raskiAhene Sequemce is imppatant_w
AM

i m e time gmirs daka windoord the

dutaand USE ask time St0 datq.
to mrolict thi autput_ So it ishasi.ccluy
Bhagwan Book
olet
 3r ingoTmatl on
windourd thr claq N
| Page No. Mw
*****i* *************

do tq» domaln
IH Tim
indotgco

(8 Machinr taunglaion SupposrLOY Ored

to xanslatr om nrCanQuag to
anothNa Soin hisCas Srqucnce
i S impaxtant

Atxa NOO)
( Sptich Siri aqn tcognition Seq io Qudio tagl
Sont
(S Imaqe_Captioning Imaqe ohat is
1mtht imug
oogle image 8roasch

So Imaq Captloninqisimpomtant in
AbmgLe imaqti^LoaLhha ohith
t n r edto ind ohat is ínsid e_
hrimaqt tohithSxquner fs
impostant and hasrd onoutput_
Cnglr

Lorr ideq outDup de0tmds on tnr_

SrqutmLeDinput thrn-

netwon% Callrd iuma NeuNau

NtODAK-
Bhagwan Book
8 Ama7on int ood tvieeo

Page No.
Smtema oL Date.... ************

Th Phone is Vty ast -

Mis
oOvdL oOTd Uond Loovdy
l E in tneNocabulary thrciS Lok Laoids
thrm e orpxesrmd ocad3 3 a bnr
Hot EmLodrd Ctod

i S

alox allon
ij AOK
t build MP oath top o4 thrsr yrctoS
Or hce Srcto a Si lok
aumimE Ccn huiid MLP likr thiS
CMd Iim_StiU_ paeScmur th
Sequtmce indoamafio0 i n MLP Madtl
but theis one rtnbltn uitn it
L t i n clagtim hiatis maximum
t c c in a Srmttmceis s So thiht is
SoK climemsion anoSamr- nothea Sembemx

imput Siur is thanging

V han itul
crcute obirm r bryo qiitnis finbLYn
oy laidinqamadl ay Langldning tnt
ma unond smtrmce bub thtxR iS ándthn
OBhagwan Book
toblvmLt You queny Suntonce is la je
brcaust gutny Sntoncr can bt of an 1tngtn
 Ystgtinq scguetner ingotmaHon by putln
0oid Vtetors in srquenee
Page No. Date.... ***************
.

to

OL
fC fC
Sok
O
TSF

is
L

anothei aoblem is that infut Silt is

masslue Causs unrcessoaily amqe Mttunork
tandtaainingSuch 1a locut prtua01x
suNrLr3Say

So toawaid o t develap NLn Nrtusk

Scotatch CaledRecuUnt neuna
Aletuoodk (RAN_

Arcarmt Neura netuosk (RNN Rtumnt

mtansRrprakia undwr drglgning
Ntu ypr od neure nrtamonk. uith
lot epeating- Skxuctua

Amazon gond tKENitlnà binany cla.Rs/o,17

Bhagwan Book
Semfune
bin0ry
CSeg o Coords )
 Page No.
K12 13 y S uovds: ont hot
Date..
eocli
* ***

vector
iLMiL4Miz:: Mim

Nocabulay Cantain duoosds

Siz o Nambd ulay d
So-
Nlanam ddim NIcto

Tash classidy Sqod uaotds L&tottnLr) 0O1L

Tme t
t t oontunonS Sam

TO T00%10D D,
OeRD2
ioea
tl'Sunl

13

Cdb0)|ata+ bimx StrmaPL urCLXR-

edingq odaß a nitoa o Stmttmee
Cind by MulHiplyinqitLyith uriant
matai dto uis att:0utPut-
O ultH ahich isit`rld L o o
oim Vrctoi and cat timt Sten 2
Lor Ci gindinq - O 0 uslog 2nd
ond od Smtrner- and S Odsd
Bhagwan Book

utput OL +o tt 0,2 oith tne hrl o

wtigt Veet0 w
 Oq d (o,tol t uoMn)
PageNo. Date. * *****'**°

and Oe a hrepAtprating this o

_C using Yuiou8 o d OutputE0
o qetN Curat dOutput Oq
and i0 thisc y r cr
using
QrOLLmceingdoamatlon and Softma
A
Classi4LntaE he md t o At
them is 3 6ingo¢ ariqht VrctoA
u r ustd in BLN
infutomd to- hiddtmlaytu'
Connrct gut0ut od tnf iOuS Layti
fo th Dat, laa rig ht
Lonnectdinal toondoLtput to
th So^tmaelassisi

TO O

timt
Repratina Shructur Rrurt ||
a ntialtze_aLo%dinq to-
cLtiz4ution Unit CRtLo-HeSigmaid-hlorat
Ane ieht ( ¢ ctoi

22K 28

O
Bhagwan Book -
 Time-t
Page No.

i t is last paint

anothts way o epresontatton o RNN

DSH logistie_

os5 dunu
Läaining KNNsi Backpiap ONK i n 99)

ZrsasD-
andM

T WT

- fw
*Hme
doruo.ardfso?
diarckiaQ
tn
droo

O10 wxiot w'og 3

BhAgwan Book
aehoord rop OPPosite diretion C oO
Page No d .
d d d Date. ***** **''**'*'***''***

d di
vsaxudoandtioonas

Ihis is Cailird bech ropogaklo0 0utq

Haiadtexbckpxolo4aelon or have
rieun+ ouo 0 this
th ont dataPoint Lrl-(491) and
lata loint
and kerp updatiaq orighesbut thure
iS onr xoblecmuitn this is Q.acdient
S aord is muti
p1icabioon O m.anu taandsand mast
on allo them iaLess than Lohich
CeLSrS anishing adient rabltm
e i r D.ayi SMtm 00 nnds.
Fhrm fodinding graoliimt 0 riqht
nerd to mutiply
Loo aluyg ohich 0t anginq nm
_0- ¥hcm it i s Vanishinq 1aditmt

phan hanexpladin 4adien4

pobleg O CAn ucinqVanishin4

gsdient oblem and eaplodinq q1odltnt
robiem brausr tor an1 beekkopagatin4
Bhagwan Book
0V timt
 Simpie MN he
an PapNpdidicakion to Simpit NN (ALrd STM& GRU

thr 0xr aualding- eMPla.dinggradirmt_

oblem andNanishinq aolitnt pmoblem

4 S A KAN Oachiiectu

OMaau to on RMDIthis isCalled Mao4

t o onrRNA becaus input
iSStqEmLPO alue3 and the4r
sen 0utput

Om

i MiM

Muny toanr RMN is ustdul inStmtimzat

nalysis g Amcanndond uiruo oe
Nrrd to classity Rrsitu ntotv o1-Ve
Gnd Moyir vito e nrd to rdict
atiaq LLAySs1 muticleaS Classidiroti
qssinqyiCA

o0r to mans RNN in barto many

RMN lhtoiSno SequimL
in infut Sinqle inputbu y
Cold
Bhagwan Book
a&rquknLe.
 4 CMany3quone
Page No. Date..
****** *********°**

Sim
RAN

image +im
tont)
hor to manyRNN aar uStulin imagt
Laptioninq imaqLationing aKr giurn
aimay- 93 inpu4 Ond utput
SrmBrm hichfnpLaio uhat
Lis in thiimaqp

(bntain 2 human ga
output lman)
nut
(on)

( Huny to Man4 MN.

19 1 13 Many

i
autpu gad nut
in many RAINo an
Manyton Mny- RNN s usteulin
o Sprech dttrction(liut 3 uond
Bhagwan Book
t dttrcf Subiret,verb
Stntunut as inpy+
and objeet
 crcocN outPut -* i 112.
Page No. Ipate.

i in <-input
fntod
tim
nput and 0utput is nut SarmR
i t is_aláo coled i tmtodd drcodtq
Netmoih equtmerto `rqutmce model1
his typaRN0ruszu in Machint
Anslotion Pa. tnglish Sp.anishin
inputdon tnanapir 2 aads
and utput LoLooads

onrOnr to onr fhr is anly

ioput_and 6nly dnt outp4t
Srqutm of input
Cind 0ut0u+

Need eod LatM GRu

iy

Bhagwan Book

Mony to Moy O Sumt lmgh

> Hime
 drptnds Vry Mh on i brrougr it
Page losr to Ond 3 drptn hDta3
i Ias and 133 dc ptmds cn i 3 ,
Cand i but in th 'gtal c r d ue
may htur apnuicalion ubrtat i u i s
much dtptmdsoa Ai and ILSS
cepnds on So in this CasE
Smallehanq in iy Should(tate
SimitarChanq on i but i t
is no becaLusr 4uand iy
aula and backxogarien
10anishing thisdxaditmt becaSr hr
depth is Lenq Sia bothin
in_oaua
Dass and backooad passi¢Lata usod
0 Lofd Gu+put Oir drrmds_on tariit
uacad0 inputitsyY Ney hmd
Luhith meang Simplr RNN SExucfar
LCn't tokr Coat olanq team
dtpmdenuss So togrsolvethis hablem
proplr_Com uitn and idtq
LalldLSIM|hRu LsIMStands do1
ong o s t t u e m mumo4-
STM lonq ShoattuamHmyYRMN
u takr Lat Lonqttam
depadtntsya3Lr 0S Shaa
tepmdtp:tndtmu-
Bhagwan Book
 it
Page No. Date..

w
o ot
Mit
dofPiodust
Matain'h ert2

O4uDhi tw'o-

hoth hr inuctaaro4Snple RnN

Som th Dnly oitertnLris that
instrad D¢ multipiy-
oianE with and Ot-1 and Sam_
EaninqSumo thtam Lar using
Lontatination _o4 thera and Apalird-
dunttian a dot aodus a4
Heni thr ond goim_ f_ertkcantin4
Simpli RNNiS idelyusro nam adays
dastbths undsaStandlinq hou LSIMS mosk
Please tdteahttp:_tolahithubioPoS+S|
20n-undtnitanding1sTnS ish is
Lolah'S blog
ht
Hga

Bhawan Book
 C C1 3tate
Page No. Dalc... **'°'*******

waneti

GRU Gr.attd Rr.cusaonkunit) 1SIM

aS Csx.otec q 99 nRU
ModeAO Slcopli4r.o VmSL00
Cn
-OLSIM GRu uDas. Carqt in
CaRuis NAy Similc to
ISTMS S I M S has 3 atrinput
Aete Dutputfateand _go1ot8at
CaRD SS much Simi{ied VessioA
and- inspiord- ATom SIMS Cnd
CaR is asa to nain- nd as
LsIM
LotO¢Teal
tmond obirn- Propie had uird
botn_ LSIM ond Some Las
CaRu and- in- SomrLSIM
nodh httte
heat is no rad- ninnea Loho is htfu
lgut u04 ant_ morOtimze and
SimplaA AO
ht ht
LSTMS

CE- C mans Subsimct

drom eaen od
tani_
Fhe elmnvnt of
~hE ioput Vecto.

AE
wh41)
(d-[he-1,2tbi)_ f W Che-K)
Cetanhwe:he-M1tbe)_h aah(w.[*haA
Cet Ce- + C h -2) *hetzxhe
Dean Be Wo Che -1 t bo) hov d raoor equafion
ht O tonh CC4) ostur bathpropoJaHo0
-custts (onv miR
 Vp ANNS while Itarning MLP dir3t or Iron
Page No. Percptron and by etmdbate.i.t... . .
Catatr Derp MLP andin_eNN u r
leaanConvalubional ytA Mo poal
And b nttmdiaqConvoltional laura
one adtcaotht L_CatateDeep CMN-
dimilaaly uingRALsIM4 GRu
unit ai Can Layeaonton top O4
thrc0thea o (tatt Deep RMN

MLP ont layn DePMLP

CNN conv.layaa MaXpool Drrp cnN
RNN|kSTM|ARu Derp BnN
donunaxq DaalogaHon Diarckloan o thr cTio
bachoano paapegakion=wease_ dinsctlonCTToftn
oLoo

sTHGRuRNN

leytn-

input in
Timc
le hnt machinaay tnak uutloaneddor Drrp NN
Bhagwan Book
eg. Dropout BN Flakton a t appli cabkburt uS aoy
 Didiscetional RNN Simple RNN s a uniditehHora
Page No. RNAio which Dat.utRut
ed i sdrpndem0 RgA44Gnd 4
Lout ohaki i¢my ut deptmdent on
S So to deal toith Sqch mobtan
diaitionRALN Lomc8into h
ictu N idiatthional RMNy0s hav
Conntckion daa bothdiattia dorwod
to oacktoand andback0o0adto-gouy.ad

Outat

inut L 13 IH

Bidirretiana t rtd aBidiaretiaawhen

isdeptmdmt n_at +K
Jis J

Concafena

2eras

Timt
N o 1S 1Sdrprmdson i s and ni
thsauqh brk dmouoandlonnrtion and
G dprmds onMiu maughbackwand
(onmr tion Bidistukiop RNNuSeulin NIP tasr
Bhaqy
Bhagwan Book
audio
SPeech
untmuR
&ntunR