Assqnment no02,

Enist ond explain ditE perfoYmane mem cs 0 CAZ

perfmetmc of CA

confusien mom
classxhcurocyl laaseignreport

Precision Recal

isan NXN maix

Sn0 0} target
predicted
Fer evalunnq perf0f
model where
Jhe mamA cam pate acual
class
by m Lmodel. J} i a tabutar summany or
wth thdse
Suppont

class

v
)
AUC

noof
cOnect and income predic" m adeby ctassÍHer
is used
ct

to meas Ure perf of classmode.

TYpe lenor- Predictd valueis -Ye Achual value

but its ue POsiiveNegahve

Type TPor- predichcd vatue is volue
TP Fp
tve U1 i alse
b
Resihue

The predictrd value fats tues Predich

FN TN
and i1s tve |Negahu

The pre diched value is -ve and its -ve.

The 4 basic termino loqy used to detcmine cassiF" momcs

TP when aChualvol ue s
+Ve an prediched also tYed

TN- when attual vatue ve and predictd is also Ve

FP when cctual value is ve bu!pediced t e
a lso caled Typel emor
when actualvatue is tve but predictis -Ve
dlso catled type2 enOT
Wh hes of hese4 values we can calcutate TPR ,FNR, FPR, TN
TPR TP ENR FN FN
AC+ tve TPEN Act tUe TP+ FN

TNR= TN TN F PR
Act e
TNFP Act -ve TN† FP

8 ClassiT ACcuracy - Accura cy simple measure how oFlen ctossifter

mare he comec predn ts raho bw no oF Coect pred"and total no
OF predn heacurgcy mehic is notSuited for imbalanced cla sses
J+ hasits own di'sad, for imbalanced data tohen mode prdic That
eah p beleng to majonty cass \abel, acur he y wil be high.
Sut mode!is accuratt nt
T is measure oF corness that Ts achleved in True Prrdn

and
els hoco
Aer isvalid
not stewed
many
hoice

Or
prdict
f e

hee
va |s
qc Cualy tve ot of
fn clanf Pob which
no cla s imlb alance
entire

ate
+P prdite
well balan ced

Aeeuay TPtTN (oTrCt ptdic?

Tp+ TNtFP+fN 701a prrdn

c) ciassiicahon RepOrt:

a precision J iS measure oF COre cneSS hat is achieve is tne

pred Jnsimp)e oords i tenls us hod many pred are actually tUe
out of Enhre Tne tue piediChed.
J+ iS defined as Taho Df fotlno of COmecty class) 5ed tUe classes by
totoln0 of predicd tve casses O ot oF predichve tue classes.
hoo much we predtchd come cly The preisior, value lie b oe1.
preis" shid e high Cid eatly)

His usefw mecin eo cases where False tue is hiqher concem

than False -ve

precsi on = TP
TPtFP.
2)
Kecall s ensihit- Itis measure of acualobsevahon s which are
Predicedcanety lehow many obse oE +Ve ctass are actialy
s
sensitiity Recal
predicted as tve. +js also known as as
metic wben we want tocaptur e
Yalid chojce oevaluOhan
man 4 tUe as pOssible.
total no 0£ comecHy classifyed
Recall is defned as ratio 0f
cerect+4 te tUe ca sses,
tue casses diide by total bo oF J

we have predicted coTrefy

Or outOF the tVe olasses how much
Recoll should be high Cid eally).

Rea)is seFul mehc in ases where False Neqathue

umps False tve"
Reall TP
TPt FN

s no of actual ccCUHn(es oF Class in specfied

Suppot
datoset Fmbalanced support in taining data nnay indicare
Shuc ral weakneSS in feported scoesaf classiBer and
CouLd indicak need for shahied sampoq or Tebalancing
Suppo dosent dhanqe btw models but instcad dlqnoses
the eval^ process
E Score - f s no bhwo*| s amonic mean 0fpreasan
b and
recall: We use harmonie mean bt it is notsensihve to
enheme
1arqewalue, untke simpie aveages.
ELSCoTe sas 0E maintoins balance bnw pre'sn andrall for

is(oN Qqain Ur FL
ur F scor slon

Escore = Q(Prcus?x Reot)

(Prusn+Rea)
fecay
 FIScore give the same wta ae to re call ond preoslh
There isa 0eghtd we can aive diF otage fo re
congve
co

sc ore in cohich
0taa
di if otage
9tve dièf
predsin AS discussed in pNious ious sec iFE pob
to iai ond prro'sion
(Preusion eea)
(o'Prcusion) t fecall

e) AVC (Area under RC Cune)- AROC (Receiveop erahn9 chtshel cheS

graphical prst f prrFm o bing anmdel To cHatt Roccun
0e needto eualuatt ourb now oo s
on set of tstaae
tor eaCh data p, the mode
lowpr apObo bìi sco tinat h e da
point belong to the posihve cla ss,
An RO CuNe plot TPR VSePR Ot dirf lassihcat' hresho]d
Loweng he dassfcahon hsiald ca ssifesmoe Hemsas+ve
thus ineosig both folse tue Tue p0sihve
The toll g
Shows a typica) RoC cuNe.
AvC stands for "Areo under theROC CuNe."That is AUO meaSure the
en ire 9D arta underneath en hre ROC

tol,)
from (o,0)
iwdrse

Wecan caltulatt two qulan ities. the

TPR and fPR.The TPR is the prop
0f tve data point that are comc+ly 027
Classited qs +ue ond FPR is the
are
Propt of -ve dato point that
Incorectly cassifed as tve

TPR TP IP+FP)

TH (TNtfN) TN = 1 FN=O
FpR FPlLEP+rN) FP =0 FP=o

FNR FN(FNTP) FN=0 FN =0

 (3)

Ohot i5
confusion Mornr ? Erpiain oith
h Conusion m aik isan NxN
mató% u5ed for evo tuahn9 1he
Petomonce oF ctassirn moder, whereN iS the no. oT
+arg
CIosseS Themamx comp ares the actual torget votues wHD
those predicted by the mlc leoTning modej
A Confuston summay oF the no oF
matix s a tabular
Comect and incorrect predic made by oclassifer.
J4S used to measure the per ot a assítHcat?model
Predictrd value is Actu al value
+ve Rits tve POsitive Negative
Type leror.
Tne prdi cted va lU e is tVe
P FP
but it false
posithue
TN
py is-ye arnd -Ve
Negahve
ifs

7ypepe mor
Pv is -vebut its tUe,
& pu is als0 tue
when actua) vaiue Used to detMine classfn aehcs
N Lo hen a chua) vaue is -0e and prcdiced value is als0 -Ve

when a chu al vatue is -Ve but predict is tUe Also kno wo

as Typelemor.
FN hen actual is tve but pd" is -Ve also caled Type l
emor
with he help of four alues we can aleula e TPRENR
 TPR TP TP PNR FN
A chual f ve TP+FN Actud) tUe TP+ FN

TNR= FP
FPR
Actual -0e TNt FP Atual-Ve TNtFP.

eq 0e have total 90 cats anddoqs : ou mode) predict whether

A chualvalue :doq 'cat)'doq'' ca!'dog. doq. 'cat', doq.'cat,'dog.
'dog.'cat.dog, 'doq: 'cat',ldog'doq'cat7
dog'.'dog.

prechchdvalue 'doq'dog,'dog''ca!'dog','dog, 'cat'cat.

Cat,ee ca,
dog doq'.'chog, 'cat,'dog!'doq)\cat'clog
'dogcat)
PV
posihve
P
Neggivd ) TP=6
0upcdicteol
0u prcd'ctd +ve
- its e

that an aninal is Cat and

Cs) C) is actuatty it
lpoSiHe

AV (PP) TN Yourpditd-Ue and its tue ou

Negarue
(2) (1) that animal i5 no+ cat eit
Predicted

Apei achuatly it adoq

FP(2) -toupredictrd tue and its false ou prediced
hatanimal la is a ca 4 but it actu anly its a
dog
fN U) TypelT 0upredictr d -ve and its talse oupredicted
an imolisnot cat but it Ga chually cat

Q3 EXplain in bnef oith formula .

) Contusion math x: A confusion mamt i5 an NN nat used

for evaluqtin g the perF mahix used for evaluahng foTperforma
04 cass)fn model, hen N is no oF taTget cld sses Themax
compoares the
acua| targ et value wih those predichdby
MLmodel CM is tabular Um mo
A
of the no: OF co re ctS y

an in omec
d predic+" made bu ciass| er: Atis us ed oneqsure
the Perf OFa cassif model.
 (4)

A Ctual values
TP wh en actualvaiue tue predi istve
hepcdict poitve NegaHve
tvee
als0 tve
TN- when actual value predic Laue Fp TV iS+e
PosiH

|Sals0-Ve false.
When achial value
FP (Typerenor):

piedichd
FN
-Ve and prd (5tVe
actuat 1S
tUe but

NegaHve
When
FN (T4 pett)
prdis -Ve ThepVi's -Ve
TPR P TYpeDeror
The predictrd Vaiue
budits -e
Achualposi te TPtFN
iS -ve butits tUe

ENR EN
Act PosiHve TP+ FN

TN TN
TNR =
TN Fp
Act Ve

PP FP
FPR= TNtFP.
Acte

Classiicat Peport
conectness Hh at is achieved nte
Preislon: TiS measure of
tVe o ctt
ednare actuall
pred In simple words- how many pr

totaltUe prediekd
of entre
deined as atho 0F total no- OF coecty lasS
ified
Preision
tve dassed diided by totalno OF prediced
classes. ue
tue class howmucb weprediched aot
Orout0F predichue.
les bthw o p it shtd be hiq h

The preusion ualu e where Fe is highb Concem

Preusi0n is Useful memo in
caSes

than FN

P TP
TP tFP
b) Recan or sensithiity

of a ctual obsewat which are predlchrd oTrechay ehoy

1 meas4re tve Tt is also
as
bbsovaton and qcua|ly predichrd
of ve dass
evdluaHon mehc when Wewont
as sensitiuty Recau isvaltd choice OF

to capture qs many tve qs posible.

Recall i5 defned as ratHo o£ totat no 0f corecty dassified tveclasses

diide by total tve classes. or out of posiive classes, how much
no of
We have predictrd core cHly- Re call shouldbe high )
Re all s Usefu| memc in ctas cases where FN tumps FP
Re call TP
TPtFN
) Suppor supportisthe n0 0F actu al ocCUren ces 0f 1
thecda ss in the
in dicat e
speaied dataset Tmbalanced supp in taining data moy ot
Suchural weqleness in repotrd srores n heclass)er and could jndicate
the need for s
ratifed sampling OT
ebalaning
Support dosent Change brw model but instead diaqnose eua'potess -

A) s(ore - F sore is no btw o4 and is harmonic mean of preusion

we ase meqn becoz it is not sen sitiue fo

hamonic
and recQll
avq
vaues, Unuke Simple
exhemely large sion recail for your)
P SOre of mantun ab
alance btw the preu
is 10w agaio do
ur
¢f
Stere t all
SO t
is l0w,fis 10w
classifer. 4f Prusion
Pistorelsl0w
2t (P * R)
R (p +R)

P scoreqive he same otge to re call and predsion

Tnere is a otyd H scoTein ohich oe can qiue di ffe rent wto to
Hcaland preu'sion- 4s di scussed dief prob give ditf otqe torray
and prision

Fp (i+B) (PR)
(B' P) +B
e) AyC (Area under ROC cunve)
ROCis (Reteivineer operahng Chorackn'shc) cune (s graphica
pTesen to o the pefemmance f
bina lanfnmodel
dass
TO Cre ate a RC curve,we need to evau ate our binany
modelon set of testdata
ojp a probabiity score hat
or eqch datapt, themodel oill

the data polnt belonq to tvedass

An ROG js TPR Vs FPR dt diff classi?
th ryhotd
fhresho ld. Low eina ctamif
clasites more as tue,ths Tinq
iHm
both
stand for RoG cUeve. Avcm
eawe
Alb C

ROC wve
Q ou ea undeheah entre

tom to,o) to lt,)

Wecuucaluatc tuqueii es TPR RPR The TPR is

te dota chct aue aCtly. Laniti ed as

the pnopt
ud pPR Is tepnoptA
pts
ee data pt thcuaue cont

lanfi ed

PPRIs false posiue rate

£p =O, ohen PzO
The close) tD O, the betu att

PPR =CP
CP+TN)
rac
PNR:
-the
T} is the

Ctoser to o,
false -ve
the beHs FN Datt =0 hen NO
FN
(EN+p)
G4)
7 Exp1ain in brel

he
coet
One
anl totul
no-
prdi chon.
TA's
measue hoo
th gato
on th
Clammfie ma
betw no.
prdn. thW comut
auway prdn
cames-Awsay auaey mehsc is not s
tbalawed hes its uited fur
data, owe
wwn disaduortuges, fu
lcUan
lael.the
ls we auay
awe o
be (igh belong
But nodal nof te
Ampie wy d conee. tuet is
te aulleued
ot
eiye totul ws
ow is
meprd' n
e Mauy pmd^
pmdod aue atuauy
atualuy ttuecut
Arwrcy is vaud
an wu balaee ehojce of
swculct
\mbo
ae slreus d
fu clanta
d
thee is not no
prelolcws

dao
ulic

Auway TP+N (Uuet Pred

TPtTNtFP+FN Total pnd
Ssuppot:
suppot s th no: ot
AReitied cutual
datet tus lawd osaue o

canife
inolirate
otal wcalnen
uppet n th
treiing
clanin
d
te
and ate
Couldindicate tu ned fer
suppot duosit
Shtfed
ajasosea tu ewa? bwmode sauping
bt
proeen

AuC
Roc
(Rawd Rocowwe
(Peis opuahngdhauatishe) Ben
4\stthe
qraphical deprant
Patn a
ot no d
on set R
o
taud

to tue
the Perf OFa clasSIT 02
 pict tPR VS EPR at dlft douih catheshold
nq fe
t clamf tre hold clomhes nor lten ao te
Te AuCeeue t Rutie20arla tuudentath teetire
e caen calelat tee qaitteA TPR FPR TPRIs Hee

ERR is pnepotn ot e date pt ttiat one comty can t

te
Goratr t contvia matix nel He cway, Receyy
Pisice scere tuda DAetat erechiekd vatee

Deiheeqch
dichd ueiis
Ueyau Dxdicted cooottt Posh
TP
(s) ()
yeupdietrd
Attto'val
FP
(2) ()
(ocAelsnot coakie it ackualuy
3)le po siti e (ype eny
fee
CP-&
bu pndictd te audis goupxauctrd cettna cokie

is

4) NCTypet érey=) you prdictd -ie ib false

Veu pndictd Coo lae is not cookie btaualy i/s codleis

Prrd Ss).
TP +TY Co
Away TP TN+ Fp+N TOtal prdn
 2) Preus ion TP
TPtEP
Prdict°atually

Totw prditd Fue

tue G07
3 Rer al e
TN
Pa did Autuouy
TP+FN tve
TotlAtu altue 64|

(RtP)
(o8st075)
ASE TOOLS
Ge) ist aud
eplain
Petomane meics
and for probleain dtil. fun

nce of MAE (MeanAbsoluero)

5et oF
*15 Vey simpe metie which caltulatt thu absolute diff btw
actual and
Ptdictd val
-1op
1re no MAE i Atual value
deve
n togls Aduatges
tools
qetin t
you Same uit as op va abls
F CT a 4- 1is most obst to utHiess
eipto
Bisodvutages B MAE
)
ponen: I: Tu aph MAE is not of diffyc able so we hau to pply
Volous optimizeo like radleut descet w ch can be
ect
di lfentiable.
+
the
erco
-Tuimpoeue o4 MAE -

erco The M AE S Cnuial pertmawe stutie tu cqnemon o del ínce

it s to wneestund.
=eqro
quir -tntupeytable ame uiable tool fn cnenng te aeu aly
prdich. mong mamy rao ws thse oe most siqnifiai
pes
iagt
 Reili e o o uttiens: The MAE s not asimpatrd by extrume
Tsws as otthens mettcs, Suh a .
MBE Thys nnasce t om
ppmonce e as ue u datu et has inuude outtien s dY
chevaln
neaz sconAindiudual dilPu eny es o,e giuen equal In

Cwq Tus make it 4sE simple to coupaG the pertemahic

f sew D1al modelet vouat o4 same model.

saugut fumward-Tw MIA intuptut i bancand obious

Stutic that psut the auq maquitude oj fmast ui stace
* is Äple fn non tuiiecedStuceluotd eny to couphand
4>|bame uit as spo nye veihlei Th MAEsepryed in M

tusu uit as pose vouiabie, malcinq itThisis simple to

ia
(omprhend the size cq the pnditn
emo iyp
trying t wmdeytand th pertonanu o4 thmdet
o theiste you tuping to sor
Uscd inseusal d'sipes aAE iS wsed to fimanee
engg cund metolo y itis wen gaaded asstd
gauded

6) of fur iwto alcut jeyen MAE proide into cbout

magritude 4 em podued by the medel
copaision amd selct" th best one
quow
Clo fu model

tpemmet e o nedel by deenining

te pdictd absoltc mean per cutage emsr.

Mean squaed elor (MSE)

His t most uped ud uery siwple mehic ith iHe bìt oz

in naean clb south eor.

M eay squar enw stut that fiuding squoed d B_QUua
amd pdichd valee.
 Squaring
ud
the
quaates
dI hu

that
enes
te
moves -ve mcansquae emor di.
a model itto auy emeys
MSE u Tuis dow ot Ocw n
wll
alean
auality
Moatou, squai ug magmh es
bpututn
e elkut cayinauaye
Puuish qnetn
eatn wste e
wistue
cis propc tieuaty
Smalle ons uis ltibwte is necnay f you wct oethan
istuees to be tewer. yw mod els

Root wcau sq weem is measne

a
4 prusion
fhat does male
t»e tu natual o
d ata wits itis cal ard by s quoing the MSE
To tuthe afy,
cl
MsE
Copoble tote voiane
is
tostud R MIE
el
delation p

(yi-g)*
A duoutages 4 MsE
gaph o MSE is difucnhable, go
(0 tou caneaily
tuwnti on. us e iteu a

The vae you qei

tty
tu eg the o|p colatug
vajalele \s in MEs a
squored it
ut isin m. then
mt squaued.
otu callang ot o jp
MsE heolp
you hawe outi
owttie
ey in the
most
cala
datet
atd MSE is
tn
peralizer the
it
So in swet,HiS biq5a.
ot Robwt to outlies
aduatue in wic
MAE
Te importance 4
MSE:
bhw MSE
quantiHes Hhe
pMdictd oud
atal vales
ou9 squQred
putmanee o Tis help in diA
model, cnenìng the
Well maleng
maleinaq it
nod s
eamea to
wdwtand how
ihing inn data.
the Pert OfacassiT
yA sihuty tD outties:
to outies ine MSE 6quaye the emor, t

weiqth to laugen emor Tis prop

prop enty is
bemi a

ucas wye te
when outiers oe sigushcant, asit iqdigt
me del may be wdepefong
3) Contiwo us Value: MSE pmide
proMde a contiwews op,which
ow f4 Suooth optizith in auguit Tis
thisI3
is udy
pouti

4)
47
wsct
Cue w

cowpaisíon
in
ed
qadiet
tb
desl
iie the e.
et mtipd,

Model: MSE s aa studod netc tn compai hy

of
wh,e devative

di(fyeut o des, A (ous MSE (ndicats a beltn fit,

shoigu fuwod eval c noded pefoname
allowing
s) Bunis fu ot meics: MSE is the foudat'anfes oth
imp mehics ike Root meqn sq. mor (msE) Ý R squoed.
wich cau roude ad jnngsinto mad el pefumanse

Koot Mean Squae EmY (RmsE)

RmsE s tlea by the nameis elt that is asiwp le s4uae root

4 meau sq uOe enoT

RmsE

A duautaqes of RMsE:
Tue olp vals you qet isjn tu Same it as the rq uiid
o|p vaable ch makes jnhupntat o4 l0ss e
eany
Disad oataq es o RmSE
+ is no that mbust to outies Conpuel to mA
 The inUrtunce
RmsE:
pobiity RmsE js in tu same s HHe
wit as th tug vouable
tuqetet ma
I eones to inupt conpoed t msE Jch isin squoedu
TAS inyPrat shkok aldtn8. wwdu tand tu
uy din

in poutical tunus
moder's

Sniwtyto hege oos

ens duto the squaina
: lce MsE, RMSE, mej e
wight to lag
4 ditMeme Tws mace it pot ouday
wwetu ln steuayios
mod l wealenes
wetuge (auge deivat aue cnheal
n thse uea

PztmaIe to nupaisi on : R SnSE Sewe as a Stuudod mehie fs

aluaig aud companq diCtieut odels A (os RmSE
ind'caw a beltu it toth date, auouing fur shuightPonad
anement ot me del petMane.

malcimq it nutub le fh optwzat? auguitnad On divat?

ney
8uh as adiut ele
d eu ent.

tous
otpndichions
on Vaiahbiiy
IF
:
RmsE :Ca
RmsE is(uqe
ie
giveiniqt
ingigt
ative to mean
to
nto vaui abitty
of teuger
vauiable, tndicat gg tuat model wCy not captue t
wdlging patuns tiey
luui on
RmSE we hae toNuPy Nuwpy 4Uaemot
o msE.

D
R sq uey ed CR2):

o cotiaent ot detunn yis asust cal mcasue in

R squaued

agren modelthat detamine tepopot ot vaan ee

jn he dyndt vaulab le that can be 20planed bythe
|ndependent vauable n other wond, rgg uacd shows how
well te dat A e neqrsjon nodel
 (9)

yueed can tulke eeqny values be? otol AlHowg t

stusheal mease pronde gome usetu insignt hcgoaig
the g | model, the user ho ud not ony on my
meabe in t enemeut of stutishcal mode.
squaed is howne?
the cqo mede erplain o bsewed dote. fu eqi an
shtTee squaed o Goy. we al 601. ° tat
vatiahi-y

cbseed inthtuqer vaable isep1aineg

catrion mode. Gu aay a bigh' rsqwacd indcato
mosevaabi biy is enplained'by tte mode.

R-l- yi-y)?

R=l- RSS 2-coefof det, m

TSS Rss =sum osq s jduals

TSS totul sumos9
Rss =i(yi- f(zi)
RSS=sjdudSum os

TSS = Pcdihed
yi-) fi)-pdictd valu ofy-i
TSS= totlsumef sq uoe n uppe (iniysumat
n= no osbsem
Impotonte o4 R

poion f oian ce xplained: R°auan tiies tepopo valu an ce

atpeud eut vaiahle that can beexplaned by indep end etvan in mod

k4 |indrcete tat mode elaun o vauance wle an e

0 indicate
o
Ata iteain meY

Modu Conpusion
Conpaing diff model s A viqhez-
RValue S
ugg est a betu to dota maleing it eaner to
ideify wich model May be moe eltutive
qooduesof At
4
:p
e'pwude shrdgutfmord mi aMue of guodnn
detnine how
mea
nell te model captuue Hhe
undeying tunds in datu.

Rage ofval: R vaerange fom o to|. wt Uay dosen to 7

indicateing abeltu nodee A. Tn Sometxt Rcan be -ve
indicohvgthat model is
based prdi petomng onse then imle mea

hejwsthng fu comple*ity-
aeots fu tu Acjustcol Risa nmoditie d veslon thot
no. ct pdietns in th nmode
mone aerat meeue proiding
wen cowpaung modelith dieA no
ind ep mde aiable.
tom skleqm meicsinport

pint(a)
Cesche Supeised ML techniques wth s ta ble eg

D2hnein eor Rearession ? Ep1ain mtR.

vatueoF va based an value
Unalusis is used to predí

The v0ruou wanttD prdtet dep endent Var

s
nother var-
Thevar uou are usina to prdict Oth er vars value rs indep va r.

1R1s Suoeised maeel majomy

m used in foxca stng
dot to bus' f the
S L Ore tnose wnere weuse tne toining
modal then tst o cusa y U4 model using
and
loss fn
forT Ustin 9 tchniqu
+isone af most wide Hnown ime senes
Onichis used f prdtchve madelting
0dep var and dep var
Ttas5ume linearraship bn setof

VOT x indep va, meslope bey intep

jndep
This invo]e more than one uar and Onc dep var:

meegnof 1Ris
Bnn Bot B1Xn BOn

,Y2-nindepvar.
Bo =ints (ept

best SEline eqn that can

prdiA the valu¢ bard jndep Var
noares setoF reca rds are ntwtb x and Valuesand
these values Orc used to lcam fuso f ou coant to
 edictY hrom untcnouon x hls lrommed fun (anbe Used Jn

Tegress
toe have to fn d the value o N, sofun is tg uIrca tha
Predict tontinuows in the cas e o4 grem given x as
independent Aaturei

9- bo tbX te
Obserd value of

S ope B
Predictdvalue
0f Y FoY Xi

Intotept Bo

ophmal (oe iient:

n thelinear qrs", ópial coet ae tthose that nininize theum
ot squoed di bw the pdieted and otal eu.Theottt
ponwt the slop e intuept liner eqn

G4) Explain in bief vondai onsof Gradient pescent

Gradient descent is an optimizen alqo wsed to find min ot fu. n ML

iS comm sed to fnd opimad value o4 param in mod hat
ey
noniy
minimizes the cost tuncton.

|: Barhqradient desten+:

A1so caled as vanjua qd,isthe sinmplestamant 0f qd

Jn bathad, en tre ining datuset is used to compte q rad ient

of cosfum ort model Param et rs in each

tat
Thiscan be computahion qly expensive For (arqe datset but
CF
Hguarankes conuergence to iocal mini
where 5
6 paramekrvechor
dis leamngate

Stochashc GD freach tntng

d that date model parame
up
s Vaiant of q entie datasel to
bgd wh ich use
e4 In datase Unike baed on vandomny
Sqp updatr the pgvames
computt 9
sletd heling g the updahs
to fastuconueqence beon oritf atnin
e
Tls an lcad 5Uut In mo
bw it can cla6
hequut and noisy bwit
o daty
du to ~andomnen o

Oisparam veu, aslegmin

g re etrd haining egi
sel
Single nandonly
compuurdlon
on

Mim bateh ! and sgd in minibgd the qradlen t

bho b4 dataset)
4 4 conpromise
cy boining
omputd on smal random Subset
ini batch.
caled
*ypicaly bhw lo to lo0 eg qd whle
f etqo tompaed to
Tis duee Cony yh! Cost Ho sG
also duung the Uamance of wpdats compaca
bccoz itstice
dep lcqi n o
balance bho cone qen ce peed p stabilny
YI)()
whee o is pv
mis mb siz e

On nb sze m
A) momentum o:
add momentym s update nle
31s vodont cq qd that m to

the
Ine m mentum temauMALL Ate oad ale time and dampe
osullarnin thecost fun, leadi ng to fustu tomgene
TMsIs pett aloy w ful in caen whee 4 nau lot of noise uvate
wuich can cause toadnt ad to gel stulein local ntnina

where
t Puatitrat+ LR, yJ(O+ d = ) oc
a Jun9 euaattol tt-\
Ut i5 louty vuts at ilHratt

yis momuntun Cott w dh conol contib^ o puious vu. veum

to the wyen t louy veut

S> Nestov Aceleratd Gradlet

tan extyn momentum thattalee into aethe
o
d
futye qrad
vo when comp uing tum. Tishp todue
momentum Oceshedttng
and lcad
can to fus tu contqenee tnan momem
tu g.

wher e 6t = Pv at titerat

VI(0{-9uy) isy ot Twto ecleted at y

is eloutyutr at titua
O4

is momentu CoUtohioh CoMhos(enbsb po us vet to

Cueu vouty eltor

6 Adagra di Hisvaiant ga that adapt earing oate of ea

Parameti bascd on ib istoi ca a valws.

gedued ile
hae their leauig rate
lauqe q
Paameths with
wth Sma hoe their tis to omalize R 4. hp
hye cfhas
te d ats and (on be wetw \n eases o e

diG scalls qudieut ot

 (12)

St st-l49t

Jst te

RMSProp r tn eokp,
qd hat also adapt seaning
oj
Rmspaop ls Nam ant v9
bw instad wnnq istmcal q valws, iH vsemovng tht
hep to mdwe 1e fer pararn t
is
Squacd q valus
hae g
lage squacd
dlnge

fr Adaphve moment tim^(5 van at

Adam: stund
qd that Cabine tdea o aoagmad ems pep
rat fn each panim berd on histencal
H adopt leaming
ales aul
arpute
wdey uged opHm asgo in 02
Adau 'swe o wost

bewr ts eHu't Stub le R r0b ww to di ff type

c4 fu n aud datset

Thye ae jws most fw

popor vawant y
coie
d. ga

vaiaut has ib own shuq eaenm

wwi Uant to wse depud m sphc pwb
dataset ct hand