| ESTER CCE SEEEEEP EES | unit: 1 , Recarent Heura) nekworks aERMM A its Types: (pag) [ at ee g forwareh leurs) ug Recuen} Neure| (Ag) - Takwor ¢ Networks a LST) Arhiteare (Ppa). y Entoder A Decoder Arete chupe -"7 AE Rewurrent Neural Hetwork.” i) RM are type of neural nekvork that can be “std fo model the sequential date DRA wbicy aye Bred from Peed forward nelworrs , are similar te human brains In heir behaviour. It Hey can anh aipake sequential data (9 away that other algontns cag't- 3) Brit uses the game weight for th@ each clement ok the sequente, Aeueasing the 0. ob parcim Obes A allowing the model & generalize te sequenceg af varying length of = Meurtl Hekwor, where output 4) RNA ig a type ave Ped as input fo cunent froma prevrous step slep- 5) a of the inputs and oxtptts in skandard neural are independert ot on another, a nekoos 6) However in some c{ttumsfdne@s , suth ag when predicting, the — next word ot @ phrases the anor words ar p@ceesaty A so hE previers words mast b@ remambered . | ell| NAS q result, RAN cows cfeaked | whith used of hidden layer to overoome this proble nny 8) Ime cornponent of Brin 5 ‘Hadden stake!) whith Kemernber, — specific information abort & sequen, @ RAN have Jinemory! that stoves all iaPornna tion about ye" caleulahons " soting for Gacy inet sine ovhorne by porforen ing inosh lay ets 0) Tr employs sarne \e re dees he same wre same sk of a) | ff orcnikecre of RNA” oO _ _ Le it} t 19 anfaid @ y ) ( ; V th) some. Te | ES aS» Ww) oS Chet - 4 aD e O20-® ) Taput layer": 9» x(t) te baMen a5 ak ene step ‘t’. LA , Cound berg one -hot 4 wa word of sentenie the input to whe ne foo Y) fey eracno? “ vedios comespanaihb ) Haden stare ~ 5 state at Henge | a) hilt) represents 9 piade and at %& memory > the nehwor' ») hk) is calculated paced 9 tne carr eor pidden sta® input A previows pey= P(u x(t) + w rlE-D) taken as non-linear c) The Pancho ‘pi is qs (tanh, ROLY: bans formakon s4 th '3) dutpul -S 2) ols) iustrakes the oalpak of the nehworks. [J Tapes of RNIN wu) There are Pour bynes of RAN based on tne nuraber of inputs A. oovfpuls 1D nekoork.. @ one to one | b) one to many. c) Many to One, d) Many te Many - 's) one to Many-t TA the type of RIN , there ¥% ‘fe : bA many outputs assouated with jk - on one of the most ased exarnnles of this nefwork‘ i we "Tage Captioning’ where given 2% irnnage I : as OredAlk a sembenies haveAg mune dV ' ko Many Nagracn: / Magnan” aatiae oe , single 4 Inet \8) Many bo One-s ‘ a) In this ype ok nefwork, many inputs are fe bo the nerwerls al several crates generahag only on outpat 4) Inis type ok nekworh is weed im the prob leans lhe senkimontal analysis. malhele wortls as inet a J) where we give " seamen of tre sentence, as predic only the output many f one 3) singe ottpat th bon Muligle Taguls 11) one to One -t a) This ype of RNY behaves he samme as aearel networks ex ‘ ‘ont b) Known gg Vanilla Neural Nehworls . al: ) OM@ inpyk A one o4kok A+ sing Toot) Many to Many * a) Tn this tyne mylinle — fnpals A multiple ou tpuls d) Example fan be language pansla Hon we provide mulhle words — fron as {apart A predict maullinle wor AS second languag? ag ontpuk 6) nlagrees mating i wicing Le Mony . outpy's oO oad paige Topas q q q ak Gmpargen between Feed Forward Heard! ane RNY - Compas? Teed, Forwarth Rocanent Neural Prrioere plefw orks S slebwor'S signal Row Forward Bidireck CI airecHoo ant Kona] Delay” ‘ . Ho e intrody ed Ne Compex iy low tigi Heaton Kndenenden® Ys - 0 WW same \ayer af poural nebwork , there one langage1 High Slow Cormmnonty Pattern recoguition, longuage tons laths, used for spee chy Hcogartion Roholit faith}: A charger +€tognition _ a Lew ) Tis model is a subtype of Recanénk Heard] gerwor (RH) 2) Bit eB aition 3) Tr Ts wed be recogoize patterns sequence such as those that appear in s@psor dakar | glory pyines or = Natdral langyage- In date 4) Lstm are ale bo prowss A analyze seqxntia| data such ag time sees , text A speech. SS LstM ace memory Gl A galec to contro) the flow of information | allo win 4 teen to selechvely teas oy dis ard inPrmation, J] Pobierm of RAN 6) The pro bern waits "enh {stat they, sino Oly store the preyous data in, “short ters memory .Jone tne menor) in ik runs o4F, iF ciemply dey 4 the retained information 4 replace ib eoith ned Aara . [I Se\o ce\| Srate . (Memory) | Tiel fy het) outpud crate [Bee forge o yrade care Topuk ! rade b chate Gate : a) he 461M mode) attempts te escape this problem 4 . d it by reYAinin g selected informaher in "Mong - Herm " 4 men or} iy a) this long - teen memory 15 stored in the qiso ‘a hidden, state , fom nedral nehorhs is presents ‘cety state, there tS whith we already new sed a which short - Ferm memory (0) Jo each computation a) seo ,. the carrent inpat ye (4) Vs used, the previous of short keen merno ry “¢ (He) "A the arewbdys Widder state" h(e-!)"t rae AOE a ) hese 3 values pass the fillowing gakes a) Forgel Gate -' - Tn th sale , it & decided whith current anc previous information Ken A which ty Hnrwn ott ) Topat Gate - ~ TE aecid®s ho valuable fhe cunent input is b& golve he task: ©) Oubpuk Gaye”: — The oatpt of the LsTmt model 1S then caliulabed in Hidden state Tk is tesponsin\e + deciding whith \nPormah™ ko user otkpak of Lstm. 12) Application of ssTM a) Languaye — trans\ation c) Time- Series Predichiy b) Wolke Recognition d) Video Analysis.A Eocoder and Yeoder — frchitectut \ Diagrany ~ [yo] aly . : Yecoder f_y ver Euoder f__, | tiddeo]_y ] Yekoork ’ yd Day efor oie ? Sal "Dal xa\> widely used Framework for developing neural] nokworks —thak can perores ‘natural fan gia ge processing | (nip) tush sath aS language translaho: ek. whith — requeres sequence — fe ~ sequence No modeling - 2 Consist 6h 2 main components | ad Entoder — bd) Dewder . takes the input sequence anol 3) The ene der . representation proauces o Bxede length vector ch ik. often tePherek as " Jatent im plemnentafivo'! I L) This representation 5 designed fo cagure the ingortint — information LK inpub an condensed form. 5) The decoder then fens the alent rentesen ~tahon aac generals an ottdt sequenc?, 6) Most fandarnental bailing blots oF components : : to pull entoder - decot@r aren fhecture 1Seek Kind ob neural nehwortes tine Ry mal CON hE od based encod ie ee on PNloder — Alco five CAN 4 Ender BN ! S Decoclep : This arcnihe chung can be used for tasks like image cote where the inpxt fs image f outnul Ws sequene ot words desithing the image. 2) Tn above exarmpl@, (NN fs used Br exheacting Routares fora image while RYN will generate corresponding — lexh sequence. a) There are few Vins Hi Hons while using diPbreng neural nekoork, along eniode-clecoder atchibedete a) MW OE COmputahonally expensive 4 may require a lok ob taining Aate. YY ay [ist can cater form vanishA4 ) exploding gradient problems \e) Advantages a) Sequence -to sequence Modelling: ») Tretn s PRE \earnin4d- Ny) pisadvante g7S a) lak ot Exolie b) viPFicalty wit Long seqsence ik Plignenent » ¢) Training comolexiky ©