ONeceSsaT Condition eist or

The azadient ol the PuncHon at the q ven point

Should he egual to 2e0 i'e: Vlat)=ß
the aradient is
222
set the, qracient equa) to 2e2o
222=0 22

This implieg that 2 o A320.

There lox the qralient is Oat the poin (0,o=(o,o)
( Hessian mataix (Seco nd oder)!
THs te asquare matithat contains al o
|the geond- ordrparttat deivohves o the functon
on the diaqonal

oolao The dekamingt ol the Hess iGn matix is

Positive de inite cühichmeans that the Hesgian
matik is posive oe ate
 shiet ylobal minimia
A point xis agbict qdobal minimizr
To the case ol funoti on ) t 222
COe can see thoat P ( t 0 and (ae)>o
ie to)>lo,o)
Thexelorthe origin is asmataloba)
P(21,2a) 2 + 2 2 2
Eind in which irecton does the Funchon
coeases most oapidl a t the point 2 2,
>let's Cind the qr9dient oFF

The arai ent at the point 2 21]2

o. ()= \2 (2)) +(0)3, (2)243c0?C2)

So at the point a[2,1the qndient

is (SD1 which points in the dire cHon ofvechor
the
stepest incge ase
esTo Find thie ddnecon in cohich the kunchon
deeeas es moct ora pidlweneedto take
the negatve of this qoraaient veoted
The Aunchon f cecease most aapid ly at the
Point =la]ln the direchon of vector C-s-1o
a] Consider the follooing Pro blem
minimize
to Jo| s
Subject

aDoe S the point o, e ] o Scisfy the

condition for mioimiz er?
|st orde neceSSay
local minimzer
b) Is he point ( 2 ) o a locct minizer
c strct Jocal mìnimizer
local mcim zer Doneof the borc
sto'ct

) Necessary Condition (st order)

Tbe Gradient of the fxn at given point
Showd be equal to
ie. f (e*) =o
Io caSe of fCo)c2
The GVdient is
Vf ()= 2
the Gredient equl tto
o Ze r .

This inplies
There fot, it atishes |st order
condition
 Hessian mato'x (end order)
H(E) = vf(a)=

Note '- Th

loccl miginizer
-e’Stoct lo caL maxi mizer
wco y +re occl minimze

It'S local mxinizer

Neotonls method ind minimizer of

Q4JUsing -Sinz
f(e) =
inticL Jalue ls o)
Suppose that
e =16S
und their require d accudy

’ c0e stop cohen

Lets conpute p" (e) =|+si
o)= o . 5 i n radin )
forlc'
(k-)p(ee )
-05-05 -cO S(0.5)
I+ sin(o-5)
-O5 - 03775
|-479
=0.7552
Sinilgr mamner oe obtaio.
Proceeding in
 (3)() f')=o-7890
C2) |

=O-7 39o
f"(aa)
e =lo5
Sn ce la (a) roinimizer
Noo, stoct
Sinne is CL
of
minimize
&5) find the
tange of o3
to oithin a
interveL fo, 2]
in the Bise chon method.

[o,b] = [o, 2]
C= 9tb
2
2
(2 -70.|= -23
|-14 . 5+ 6o
-f(c) = fa) = set a :
Since
-fC) <o, coe

TnterNal 9, b) = ,2]
2)
2|5
c=lt
2
60. (15)- 70. (5)
(r5)-14-(s)' +
f(c) = fCu5)=
.f() = -j2-s 75
<o, 0e hare a-1-5
Since f(r5)
Interval a,b] = :5,2]
a =|:75
2
C= | 5 + (r7s)
-ta.(:75)5 +Go(r75)- 7o:
f(1:75) =( 75)
= -4-4o 23.
Get a=75
f(75) <o
 [o,b) -( 75, approxinately less than
b- 0-25 cohich is O-3
b L <= toleran ce)
minimizer= tb

=1-875 + AO6As 2
2
|937S

constrint s into
&6) onvert follooing
ua d tic loss fn

)x)2
’)L, C) =max (, x- 2)
to
Dant
2)for x20o, oc
x
)22+7l6 pehdUze volues of less than
Such that
d) e2+1 >9 So loss. fun ction is
0,2 o 0 - )
L2 (e) =max

ptying give s x s d, e coast

3) for 2e t7 sl6, Sim goeate than 2 ,
tbct qne
pencliz e vaUues of c

So the JasS La lox)= max ((o, -2)

 far etg,sinpliying gives
velues of c that cHe
to penalize
loss function is.
iecs +hn tn(9) So the

Ly Ca) max(o, An (9)-I 2

equ ality congtrcçnt

66r d 12, thie 1s
$. he loss fn s

&.7 Consider the problen

minimize

Subject to | l=|
ohere Q= a > o
fan Plx) = (ilx|-)* and
a.Using the pendlty Unconsheu'n t
penclty paratneter y.c0mte
obose Solution
pro blem
optimizatien
ppximates the
problem bsed
’ The nconStucuied
mlnimize
penclh fxn is
simple optinizetion problem
8)Consider tthe
minimize
Subject to of the penu
find the smllest Value obtioed Using
Such thCt so |
y
pcramefe no futhur than
method is
this
So
tou e
Qqminimize z
Ye)- (mar Sa -,03)
Cindhe 9malbstvalu othe penaft ararmat
sueh Atat the solnobtojnd sing Hapenal t
otthodds no furto thane boom t hru solh

the unconehraind oob)em s

minimize e (max (Q-x0))
A

az 2/ a 24a
b F')= d (Ket 4a (mar la-x,0))

sinc the bu solwon s atathe i bpth

the us " ond the s olutonobtained using tu
Penalt mothods
( t a = |(a- |(2))-a] = e)

To enSure that the ciescnre s F

y 2e
Por he smallst valw ot
tence the smal st value o the pnal ty
obtauNd
parameter Pthat gugnees the solun
using the penalty methd As wthin Zof the rusn
 Date

Pape

22li5

a) Eind the maqinal 'ditributian c andam

\aMiable X and

Aanginal frohability mas guntian c K

PK=-) = PK -) yeO5)+ PCK-1,y = ) t
PCK=-, Y = )

2lIs + is + '5
o. 333

PCK=D pk=l y= o) +PK=1, y=)PCs=LY2)

O.26666

Ilangiaa Probabitity marsontioa af

pcy= D= PK=1y=) +P(Ks0,y=+Px=1Y=)

 Date.
Page

PY=2)= PCx=-L, Y=2)+Px= o, Y=2) +PE1 Ye2)

bconditional distnib,otign
ebt X giues E2

PCY=

P2)

o.233

AO, 2
P(Y=2) o.333

o-332

CoaditioAgldistnib0 tion c) aiuen K=l

 Date
Pape.

PColx=)= P(K=1, Y2o) o.266G

o25
pCK=)

pC*=)o.2666
5 = o25
P=2x=)Px=L 2|LS
2)= o.2666 =O.5

d) chedk ie x aad Y ane iOdeperdet

p =|y=2) Paz-) py=2)=o4X o.233= G.1

XPCx=o. y=2) =fk=)X P=2) 0.332X O.233=0.ll08R4

X P = l y ) pX=) pCy) = O.2666 X G.4= 0.l0664

X| PCxe l, y=):PKe0)x POY=)*a.3566 X 0.333=O.0887

R Jhenelane x and y aneenot t fodependent

 Date.
Page.

5
I32 32 l22 3la2

l32 32l4(l6u O Plsh

) iangina\ distaibutian c2 HaAdam uatiable

PCK=o)=p (x0y=)+ ex-G,Y=) Pxo,Y= 3)

+ px=,y= 4)+ PCXs o,Y= 5) + Piz0,Y2)
t2 + 3
32 32 32

O.25

tttL O.625

32 32

GonN

O.oq315
 Date,
Page

P(Y=2)E°1lst l3 G.o 9375

PY=3E l32+ /t'/64 o.I71
0.20 3|
p(y=5)= l32 + le+o == o.|&75
ley=6) 2l32 + l + 2l64 R o.25

b) conditiongl distibuhoo ch K giues Y=2

pCx=x|Y= 2) pCxaxYE2)
py=2)

o. 09375

TI6
PY=2) O.0937s

PK=a|Y=l32 O.333
O.oq3S

PCKe2)

PCY=1]x=) pCK=o ya)=0.2s

PCK=)

PY2x=2)E -O.25
 Date.
Page

PY=|X=2)l64 O.125
O.25

o.l25
P Y l a ) = lG4
o.25

Ql25

PCy=e|k=2) 2 l a 0.2S
O.12S

) PCxK1 Y=)
PCKsI Y=2) =p x-o.y=)+ pC=1 y=2)
0.0652

C) PCX=)E PCx=o) + PCx=)

O.25 t o. 62F-= 0.875

Gi) PY3) 0

Cuà PY4=3)= PCy2e) + P(YE2) + P(Y=3)

O.375to.oQ375t O.|7 0.35

+2*=o, y=3) +PKo,Y= ) +P«=l, Y= )

PCKelys3) +p(x=I,Y=4) +PCK=2,y=) P(K2,y
 Date
Page

OtO+J++
32 32
+L +
8 32 32

I6 64
O. 578125

) check X and Y ane iodepdent

PCXEO,Y=)= o.2s xo.oq375 = o.a2 34375

|P(x2O2)= o.25 *ooq375 o.O234
P(x=OY=3): o.25xO. I8 O029
P(X=Y= 4)E o.25 7 O.203) O.0So 7
PCx =o,Y=5)=o2 O875- O.
O.o4 6
Px=Y=G)=o.25x .29 O.OG25
PKELY )= o.6256.a9379 o.OSes

lo 73
pK=I Y= 3) O.625XO.71e= O.e2t
PCx= Y= 4) o.625 X o.203)
P&= Y 5) o.625 XO87s =

PX=2Y=DE Q, 12S X o.o9375 = O.0|71

POxeY=2)= 012s Xo.o9325 0.o|1
PX=2 Y=3) o.)S X o. 1712 o.0214
PCx= = 4)= o.l25X O.2o31
Y4)o.l25
P (x22 5) =o.l25 K o.17S
|Pcxe2 Y=6)-0.12s X o.95

Jheneane xGod y ae nct Sadependent

 Date,
Page

3)
274xty)

uhene

Fon
Cundigined)
IO+0)

27(0+)

270+2)

270+o) S4

27(24)
222) 270t2)

Fon x 2

274+o)
 Date
Page.

27) 135

yE2 e2) =
27(4t2) G2

|27 154
'hash62

khud Kanlo AAge ka

 Date
Page

woRain dices ae tossed

CaD a, 2) C3) 3,)(35) C2,6)

Ca,)C4.2))C4)(45) (H6)
(5.)Cs) Cs5.), 5.5),(5,)
6)6,2), (6,3)(s.)G)as)

PCX-y=2)
C)eCX+Y= )
Jhene ane S combinations C2,0, Cas), 4 )
(5,) C6,2)

36 36 36 36 36 36

36
*15 = 5
36 12
 Date
Page.

CL) (2,2) 3,3) ) (S,9 Ce,6)

36 36 6

Emom theSamptec=
Fon Y=4 possible Outcom a e

Thene ane 5 G0mbiagtiaos Aoly o9e

COmbaaa0

d) checl2 uhethen X aod Y ane fadependent

So this cOse goin pnobability gmas qunciaa

36

Gon alpoc ble \alug c Xaad y cwh ene

Y are ioteaer betwerO L66

Lets chece e X oad Y are independrt

letstake case
eCxelY=2)= 36
 Date
Page

fnoduct c manginal pra babilityT

36

Since p(x=, y= 2) = p(xa) XPY=2) variabe

ane both iadependent

B similan aktetion calculatiaa pon

-parrible eteees combinations Xand y
can be veniged X and Y ane indepeaden
 Date
Page.

2 3
0. O.2

Io3 O.6
o.3

Condiianal x giuen Y=l

pCx=lY=) = PCxs xY=)=

PK= y=) 0.25

o.4

-O.25

PK=3 Y)= O.2 O.S

O.4
 Date
Page.

2 3

2
O.232o 0.52S7

- white
2-Red

X- uhite balls_ cHOuUn

Bindgoit prababìlity distnibtiaa

Sum GccHOssMOuS
Fon X=0:

Fan K=: I+ 2
77

FonxE 2
P&=o) +pa=DtrK= 2)= 1

Sum aciOss calumos

t 3 t
-O.2390

7 28
 Date

Pape.

Fon Y=2 t |+O 0.242

|4

Fon Y=3:

Jhenegone # is gaiat pnohability distributign

 Date
Page.

Sntentl

3k

\ok

Aad R

3R+ 6R+ 9R+ 5R +8R + Rt 7Rt OR +I3R =|

12k=L
-. k=L
72

2 3
Sl72 al2 1872
|12 I| 12 24|72
2 Tl72 tol72 1312 3ol2
I5|72 24 32]72
 Date.
Page.

3
oR 25
2R5R7R
2R 4R5R s R R
2. R 3R 5R5R5R
3 RGRl6R5R

ORt Rt 2Rt SR7R QR+ R+2+ uR 5R+6R

&R+ R+3R+SR +SR+ 5R +6R+ R+2R+ 4R +

R=L

2 3

ooloooSco loolioo 25]co

2[1coloo5lioo5lioo6luco5loo
2 2voo4oo6oo6 ioolae
iooSo 24lco
Blioo 6oo Po0 P4lco29lios

pCK>N)and P.CK)=3)e

1)E PCK>0pper tniangula matix

PCKaY=D+eCx2o,Y=2) +
PGoYe3)+PKeoYs) +PKO,Y= S)+PRE), Y)
 Date
Page

+PCK, y=3) +PCx= 1,Y=4)+PCX=1,Y=5)+

75

Ci) PHax Cx, )=3)

ptaxCx, )=3)= PCxeoYe2)+ pCxl,Y=3)+

2ePCx=2, Y=3)+ ex33y=3)
S+S tG=21
(o0

PCzeo) e pCxeo, Y=o)+ PCxoye)+PCK=oY=2)

4PCx=o. y=3) ph=0,Y= 4)+ PCx=o,Y: 5)+

p(2=1)s Pxe1,Y=) +P«sl,Y= 2) rPCx=1, Y2 3)

PCK=1, =4)+PxelY=S:PCK=2 Yz)*
=2RthktSRtGR 8Rt3R+2R 3o R
(00
 Date.
Page.

P2=2)= PCx=2,=2) +t PCK=2 y=)+P C=2,Ya4)

+PCX=2, Y=S)+ PCx =2Ys 2)+4
35R+ SR SRt6R 4R=25R

PCZ=3)= P(x=3 Y=4) +pCx=3, y=5) +P(K*3,Y=3)

6R+ 6RSR 1R