UNIT 3

LECTURE 87 : Distance measures between distributions

Hi there,

This is your pal Scooby

Following are my notes for the lecture. These are my handwritten detailed notes. I felt that Prof. R
taught a lot more in the videos than he wrote in the slides. I tried to make them without any
mistakes and cover most of the video material.

But as Beyonce has rightly said : 'To err is human'

I however DO NOT TAKE any responsibilty if you fail your exams following my notes. So please
check the actual lecture slides and videos if you suspect something is wrong.

Also tell me about it at : [email protected]

The notes DO NOT CONTAIN solutions to any homework problems.Only the problems and
exercises solved in the videos are included in these notes.

Here's a motivational quote

'I dont give a f about what you think of me, I am not hereto impress you'
~ Posted on a chick's Insta, Had a lot of likes

Here's a cute photo I googled

I hope these notes will help you too

Let's get started

Lec + B .. Disbance bett^leen rneoAulca B.l
D' QoaL d vtext 5 lectureg
Ne upunttl horal vraosttg qsed sornpte rtteot, Xn as ouc( e,Sti-moLor. Becouse [t
Vrad stme greob pr"gav+tsA . LLN o.EAured -th"ak X" conwre.flad *o tne reol" Etxl
ir,t probobitity /otmost surety and r*Le C' L 'T ossure.d -lt".of 11^e asyrnptailr e
db{ributtsrn d x" 1n;as gar.rssion .

S* .A , '(vre, Rss exomple . rnrc had p ar taa garamel. 4 tnteyv-sk

A
ut-xt gard ['/a.s
D
fn- = X,n Xn r--+
h-+ 00 P
cLT satd
fi=- ,,-,0.)
h-+@
ri(o,r)
-lP(t -P) "
We, alrso sa\^l Andiu\ot eXarraple .. avtz *ratut,
)terre rnle hcrd Xr, X:., -, ..X^ ^, Exp(x.)
Ao rarc. t<yrev{ g [xrl = t/x
U"i^g LtN cornbined hJ{tt^" C€rnt I'f^pp^ft (L"-e.ore*n t \n t^Ja O. qoo)
Xn =
c-aryrd,tdate {otc -the eshyrrtatoy
"{ A.
r,n}a arppl,ted Aeltr' rrtetho? fo .[,vt.the asynptatic d,tstributton,
trlhat r.€ bAAi.,atg dtd tnlas corarpuuecl E[x] a.nd. h]e kvtern E^ E4t
vl4@ E txl
Bou.rilg a p*,.Yt of, v*e&locls LLke C"ont. M"V
Tnavem & C.L.Tr,g4
fo"nd avt eshn'tatort Srt g . -0ve solved f..t g ltn )he ax pvessicrru
E I*l
Tt ig rs one o{ thc sirva,lplc, ea,st1 & method ayr.d Ls on I "{
Yvte{^od od eshntatioyt catl.ed
tefiev^tc )
vy,to-thoc,l o.[ -o mar11/,b
4 .
o ur-L principal"

vul4rk t+ g e gtrd ,like_ a r,o|rueru. g (1, o-,1 .If heve,

2-aaartaro
the e:tpzctab,cru lE txl urx- wttt on ry cotculqLe
pt I.
porrcvJu\ rrnetiaod cnil ed Jy.axiyaau t14, lJkel,rhood
^)e
so hle halre. o\ m.r,tcl,r, ?nqte,
rnathocl tnafcan- tuetp .lr/A
Iikelih ooE M-ethocl rs o- rnuch vnoy.r.,
e:drnate dtf{ere* gavu,rnzVerl T,rra Finri p J"d APP roqcln {o
eskynab,vs & und r^'ttt h ari: reqtl
r\tu, pro\aer#cr , -Like hrc hlttl. be- able {o w
PY o\€ a;qrrnph+tc normal.,+1 8,
4re varu artcz witt be +t re best posstble varuAn,r.€, hle
rt,?av\L th..r"e a
v\r'H,6n"
.t"t
rb
+Hrrtqlitq qr,Aodated . As the wam-
"l soL"{+ri,r,jt
w€, rnrltl how to rynaxinnise' -lh4b carr- lae a- toak irn and
I4-axivrtuYrt Likelihood +houttd f"
t^s*,tr- of 1o,1, qo-Lo mzBnod hn+h o, rueh)
grob\ewv {nana .

/rrratty rnre govreratrsz

)ha MaxlI,nuv* tit elrhood €S\.w\ahus +D M- esttrnatioa,.
we try-tn t f^xiw.tise pvecision . {r,-is is moya close -{o vnacJrutn-a
{ere-
Leovru^6 iz {he vreotr:ru $, precttc+toru
In ard+t to "2t4,tt Yoe Yreed to
 8.2
> Totat Vartatiot Dtstqnce
Letr recoll yuhat vue did in th e pre,vious {ero LertureA . h)e ho.d orrr data, { n o,bservah^{
Xr1 X1, X3---.... xn a(( iid - rue c{eJthed q sta+is*.co'r rnoder (e, tpeioeo )
tde actid'there hro.s o, dtstribukort ivt*trefannityo$ dtstribultorrs llpr?eeo
{r"mrarfor'cl,t orrr obgeyVqk6}as h}ere generated . l3{S denoke +h,af po*U,"qt*,,
povarnel-ev bg e+.
tdell*t$hbiU+V
d g ossur€s {hq* o& soovt ar {tra.a g+
J- hole cu part*tLulax P"" (+trot is uru\ue) .-oryespoyrdt^s *o g*. -r

)tence t haw
X11 Xa rXg Xn K1 Fe* n:here e* ,tb the true pavarnet6c
Now wry statsttco{ goqt hros eirnpte a. wartted {o
}.^d {-lus Fn* tsut 1 on\q
kuq\e nag data /observa*tsna^ Xr, Xr,- --
-X6 and *il+6 *,e hetp {h; i "'""(
wttl rnake an ealtvlratbt 6 6 (Xr1Xr,xg-
---Xn) {
And hx+L *ae varrte
=
t' attoivt q{t" p'"fif^l <n {ne d.tq. , 4 con-t &r- Ty c\tstri.lou\<on p5
tdeattg t want -F5 *o br worrk hopser'
heirt oloi^ dtfla,enr iou,*, "* 4
\^)fid* I cayr expect {s Sr
i"i?Y
+}re
?,]r1*-t';,:j:^that
*r,uo d<-stribldons Fr* ana
,rs ,.eqnE thot nrhen teotculate rhe f5 -{o be ctose.
r-**u3 "{;; orent A, ,ri*t bott&
{he distrlbukoas . tt should ss fr"pp;; *hot *he
each other . 1e -Hnerc, dtlf<rcn6s values qv€ ctose tu
*t,o,rii be smatt . Te ,

We ang cotceyned with*he lrylaa-yri{,,rto

r il*?H#'llffi^
r

s* rar -p,(A)
r is"*^fHil1',ff-
P5 is ctose to Fr" if
I (a) -
F6
& (A)
I r.g swtarl
f,c o{r A c E
3o ilt ovder -|o study lhis probterarr , bv aee )rheonrcefl ,4
fofau VaRrartoN DtsTANcE
De{.n : We hqre *too proba}cltitg rrvteoruy\x P, and FU, ^ -tne +oto.t vaxjo}ion
clrstancg betwegr,r, the trrrco yv\ea.suvta ,ts cle(tnect aa
TV(F,,Fu,) = lq,tal - q,(A)l
^ryI
Ac E

" Note that -ttUs is a gznevatL deJ-tn{too *here axe -trroo pararrretevs g qnd e/
wa doit +att< aborrt 1'rrst
re.ol pov'ametp-y or e>*trnl+orc
, Noka t.rat rare aY€ Th"t, -the rn taximurn 4 I Pn (nl - F; (a) I r^vhelrg h
is arq gubset + E , -d{ +he fiaxiwrurna. r^}ttt be sh^atL ttten lPnrel - $,{all
hlrtf be EmdJ fl" ry errcn*, AcE au{oynat*ly.
 8.3
> A littte probtenn"
fne def,n g| {ro'totvaxjattort is i't atvrec{Q la evevy dedLot4e tas\c
a\l good burt usivtg
beconAe hle horc {o cr.tautate lp"tn) - F.,(411 t*, all *ubsehs A q e L +ten,
*aKe *tre ntaxinnuvvl' ,ttt
4 it . Ewt.t tf auppose hle had an E te sorraple space
wt.{'14 sqU Io eternenta we h}tll rn-o,e +o
$,n,{ avdcovapqre llutal -ryAt\
t*{ do subseG ie 4oz4 s,ubs€k €vew *,r,* 4* +. con*tn,ou.g ca,s€. r ralc
wtt horre in$r,tite gubse[s A eonfiruroul space E..
"t ^
f,ovturnatol,l r,,,tathenztattclans ho.ne <irmpti-fltedihir. ptolaterrr b1 dertivinl{heoremr
r,nte wttt gLe those -t*t, dtscreke aa a^s ccrnkhuouA dtsbriLuhort
"ieU'

>{otaU Voxr'a,tton distanLe betuceen dtscr€Le rn eoauy-eA

Ao -ltrst rue utrill do t?re cnlcutarttorr d totat varu'altorzr {6, disq,ste d1gtributoa.s
lsb E be dtsc,hete ie .tk, .{rruite oT couwtobts
Uke : Bevu,rotrttt PrissorL Btnonn"ial
E: to, tt to,t,L,?.-"') torl ,2fr'--"rul
o.nd \^re \c.now hor^r *he p"nt P"(*) is de{+ned
P"t*) = Pe ( X= 2c) +olc oll x e E,

pr^f prolodcti,ty +hat

funclton rv X hkLs x
{vte propertar p r^{ atle also welt defrn ect
"+
?robat>lttu arc. alhgays grcater< than e4uA[
fv'e p,L$ sh aald s,^,*i +0 1 a. +otol + o . ps.(x\ 7t o vx e E
f r* abit!
E P.(z) =1
xeE
(v^e tobal valdAhor dislqnLe lte-b,oeqvv
B -Fy is a
pmf ,s ?* and p", :
an_d ei"r,rpte Jur,rcttorr,
"|+hu
TV(q,Fe,)=
*f xeE Iprr*l p",(x)|
Nsfe honr *lajs l,laeorem hos sQrar$cartty reduced otrv probtem o{ catc,ura*i^g ov= c
each subsel A q E . Now hre just need -lo g, fo o. parrJruc-,r.r\anr' e6 e E, *tnen
colcqlate p"txi) - Pe,(xi)\ . r,oe {lnen krorne-'{o do tta.is c.a\cutatt6yr 6\rer alt xi/s
$t
ivr E avrd odd -t-l^a r€,su\13 -to +vd TV.
Next hle hxtl see- a i.oose proo} + hrt^y ihis \dorks
 8.4
F A{oose ?vq
Ao basicot\y w{nat rarho,.ue {o pvore ts -f,/(l[.,Ee,) = X ht (z) B(* )\
* LE E
?4 9,. shohes +h^t .\her€, e,xrsts an, A guchthot
l F*ta) - q (A) l = ** f, l p.rxl - fr,rrl l
x€t-
o contptete {lne
PYO* t^)e nritt hor6 +o qygue th^t -Hne aborre a, ts tndeed the
subset o* E oY,eT nyhic h" ntaxivnum
witl not be dcrne tn {h*g 4 lB(A)- Q,ta:\ .ts achlarned. 4t^ef
ptq s{avt
- Le.G
P.r.lR. sugges{s , Lets ta.ke A such flnot A tsa. cotfecton
oU *hose r(s tn E _6d
r^lhere Pstec) ), prr(:c) 4e
A = tx€ E : pe(e() E
?e,1rc) i
we h.,tust onsh!e}( f\sv6 wlaat 4c h{u be.
G+ h}ilI be 1}re 6\\ecfi.on drallt?re*e
x's tn E rnshere
fe(xl < pe,(x) -ie Ac treE , pe(z) ( = ?e,(2t)3
l"e bosico,lly ".re pavddoaed E iutto A and Ac
> tc*s colcu\ate Pofat Pu, (A) &" ourr ?yo?osed A
Pnta) = x tcA?sloLl = X pstrc)
et:
[str0], pr,(rt)

$,tal
leA
i l?'(o) I, pr,(rc)
x:1r(x))zfnrtx)
Nor,,ono,6 *lnat eacln,'telr,rt aJ-theRns
Stal - lP6(n1 U (p*ttl - pe,(*)) Sumvraattor.u nttU be posl*trre, berortseNe

2t: fntx))6,t.t ?T ..?"14"9 o\€r x e A and tt lay

e\Ei)^ittdn taker *hoEe x/s €orurlnjcl,u
Pe(z) - ps'(x) >o .-tlence nre caru grt
tBtal-P",(a)= a rnodu\us o\re.y eo,ch *ztm orr RttS
f
xeA
Iprtxr -?e,(%)l Atso nofe 4hat L.HS. is a gum ot-
postdw larms eo it ttseg wr\l 8e
Pos\+tMe . *lencz nre F,ut o_ modut uS
OvEr U.HS {oo
) Nouo hle do ilte same calcutationt ouer A"
Nour rtoG -that eadn *exvv.t inslde

Qtr"l
2t: < Pe,8) x/s {o{ wh(ch p6rxl < pe, (x)
Fot'2()

r-Po(A) - 1+(,(A) = -rfl: ?*v) _?e,oL)l

So PnW - Fe,(r() - - l[e (x) - pgtxll
{lso t"sa t<v,row {-t. *,y rzvent A
.F (A') = 1-P{A) .rue use.+ltitr,
llutttlLy b*h g^dea
ry -1 -b eubs{t*u6. ir,r LHs
ftrrrvurta
fi,tnl - fn, {al =
E^,1
pu txl - ps,(r) I Lvlg ls n|Lral {o
RHS. snd KHs
iS^ pos*irne u So .{oke rrrodulu.
lPu cul - $,to) I = L
2G
s
AC
lgutx) - pr,cx) l 4 LHs
^re
 8,5
,Eo Occ-orrdtlg -lo cttry C.otcutakortS hrc $ound

lB (A) - Fr,(a) I =. Z
x€A
I pucx: - flr,(xl I

lFetal Pr,(a) \ Z I g"txl - ps,(x)l

:ce Ac
,{aattrg boltr unx geb

a lt2rtar S,te) I
I p" (*) - fe, (x)l
Z (nUAc)
xe
No1n5 AU As : E i{se$

4ey(e {* oux gatteutart subset A pvoposecl er,,vLier+

I B tal - q,({) I = f._ \ pnrr.r - pu,(x)\
e
.,1
xeE
We .!.tat^e frund A ,tubget A -ftH rro?ucl,t *i.r.e aborre elua*ton0r"fat. Tr,urnr out
+tai3 pox+tcrrtax A Ls -th,e gubr{,t .frrc rnrtruicku vnaxirnlnt {,. LHg {a olota(neD .
WR"
,r\4WA th&t Wd & Eo urrttl tue.

>TotaL Va{aticryu dtstonee botroeen contrnuor,ts rne.qsureA

L€t E be clracttnuor,g
tjt<e : Qo.usstavt , Ex\correuttiaL
E : E, fo, oc)
V)e t/Mot") +l^ct o rrovrdoym varctalote X 4^Jt{4A Ar\^e}4" dtstribw+tgn IPo har o pd{

rPrtx€A) = {' J",*' a*

Irxldx {o,. otl' A
{o,.ou, ct
Ac,F
Jo
'The p cl{ },tqs ttor"stnq proper*te,r
{r:
Posih,ua , {n(*) }o + e( € E
Tofatsurn=1 t
f+rtx)dx=1
'E
{r^r* .tike fu dtscr€,[:e c-ose Tv is deJ<nad ,replacij Z by J
TY (8, -F,,) =i -[,t*rl dx
Jrl_*,
E

h)e could dexi,,re, * f*+ iurt l-ika eanr-l-ie.v h,r+l,L

fie not encouraged {o
= t*e E : f.txl)j.,t*)1
,eode.r t^g
4,yrpor.ot do this. as pre\ bot11&
re?ettttw.
 b,6
> {questdor,r,
9q g waraffs you-to ask.
*je horo kvroto {louolo colcutqfe TV (Pe, Br) f{" q, conkvuls,trl jar,r,tity d
dtstrtbuk'orrE aE nre.l a.s 0- discvebel fa"nitr d. meosures

{ort discreYc catculaLe +AlP"txl-Pr,tx)l

o(J 7Gg

* J lf*txt-{r,t*: la*
{* cor,rltu,urerr,r.s c.qse

$ut t*hat if I-wont lo $^d *he T.V distorLss lee*rareen a bernoutti & a,
ohe rs dtscrek rnrhite the other rye ts c,oyt*thrratr^g .
6au*iavu . le
One *rght ry , ,rrrhy rntould tare wo.ytt, lo do taqt , but nol:e rnshat r".re hanre
been d"inq {or( most ol +Lu pavK iu, *lrte p\"evttruts tectures. Letg -[or exarnpte
Say 'that vue toot< "yt o,bsevvattorrs Xr, Xlr Xg --.-.Xn ru Bey(p) . \ohita
ust19 the ce:rflraL lin^it *reovewv \rue soid th.at at+hour€lu Xi /s arte
bernouttl , savrnjte vweo,t fn hrill a\so lce dtscreG as il is a sun4 o$ bcnnoutti
)tor*er€r :["v tortge ru (o.sywrptofcc lJryfid) ik dtstributtoa cor- bD sr.lw
Q9a' flor,rl.sior,r" dtstrihtticrn,ie acsrnttvtuousdistr0ou+(6y1 ilfp,p(,-B))
fio do+evt't it wta\<e Jewsg 5 as[< +he ques{i,.rn aborat *{.re
{otaL vartialcot dtstawcr, benletn Xn : a dtscrc,['e vv . {"" ftnitz rn,
Y : 2,a.ur,rtan 'tv ie csntfuuror,s
hXn $ttot^s a. Bivtonttat d.Gtrrbutto,t vthevt" Xi n, Bern(p)
r^r€ dyaW ifs pdJ & coT|4 pare it +o Z *, (o,
N 1) tnse tatr*t Y
Ae-e a .!oV 4
ob^v
sinatio.,.{.ikel btc {.nzm-

elousslara- pdJ

vg Birrorn(8, p)
o

o12_ 71.se 1g

{ouratnet 1o colc-l,rlali guc}r, cas€A h5e- h}tt\ {^or"e {o d,o *he catc-r.rtatiorr,

t*. 4rnr- basic d$,n * TV (q, q,,)

h)a witl AP-e- orv exornp{ e )olf:+
 B "1

> ?ropex+i* o[ T5tal Vaniatroa

Le6 see some ' propert{tes , fines e ?yopexk'el tsitt dqi^ilA hotd +rue $t discvet0 ar nxtt
ag coyt*igoqs distribu$arus, le( prorre {herrt mogt Senevallg- , So lfli, vecqll
TV (lPs, g,) = m^x I P.(at - Q(nl\
AaE
ll) syltrETRrc TV ( Fs, ps, ) = TV ( IPe,, IPs)

? d*,n Tv (8,, Pr) : fgE lF",(A) _ ps(A)\ = AeE

^nllf
\Prte.t - P*,fel\ =T/(B,Pg,)

?ruiris a wtod
3o rte c-on $.ip 'r,,?"
wdr,at,s inside

tz) ?osrTrvE T/( F" ,8,) >o

Du hlt rv( 8,E,1 = -&roll No yrrtallerr uyhor:t I ua.axivrzrig,s
oo
ffr 16ta1
*lne l,nodutr-r , i+ Ntil stil be
& rnodulus ao o > .-
Y|ry"
lZ) Dafrrn"ite: TV(8,8,) =g intprkr B=8,
fo d4'n Tv(B, B') =
trr llPo(A)- &.(e)l : e (ue++rni. be zero)
Lo t {te na-ayirnunrt value I
d Btr,) _ lpr,(a) | ts elucL *t znro
{* eon'te subsek t^y A*^, .4tris impLter lPnlar**
) -Ft(A,nax) I = o
{s vrnodulus edsowte{hinl 1o + Bre*r*t=rpe,(Amax)
& {he maxivwuvw .v ntot, {,
,LeVo .t?rr's ylLe.onf
alL lhesub,rets A

4 € r^rt[ atso ftr,.re {he vahe of I p"trt _ Ey(e) I eluol lo t*vo

\:nce lBh) - B(qll= o :> Ipe(a) = -Fs,(A) tor( arul subsek
A cE
fn r is +ho zxucL det,vt -trpo dtshibu*tor^s bei y.g
+ tnot
i"tr, P,, r,uhen .Ba) =
\e-:n^cz

{v @n, Py) = o i,rnplia,r ft 8,,

oc"
 g.B

l+> trinr'rque I TV(Fs,Fe,) 4 TV(Fs,g"r; + TV(&r,lPo')

true-1ual,t*t6-

dirst leK recn{X

-lrtle
t-.r*[ -lriangle inqu,oUl,Xr. Lef a, b ba.tro rral rngmbarr
Lett t"sti6 tt a6arn,
Tv(ts*.,80,) = ru1 llPetnl-B,tall =. trt \p"Al -P*,(A) +Po,(A) - F.,ta)\
IrY - ?ntal - Pea (a)
o.
b= JP*,,(R1 - Be,(A)
8o lBtal - q'(ar +pn,(n1 -Brer1 a rpr(at-&,(a)r
+ &, (a) - pe,(A)l r
,(evtce

yvr&?r I Fe (a) - 6,(t) I

Aca AcE AcE
(

luo T/ (pr, B,)

lA> UpgN $orrn"d : Tv (Ee, pn,) z 1

Nor. -{hoe I tr, (e) - &.,(a)l is +^z rraocluhs * dt$.[ererrrce

4 fuo nurr.bet*
+t^at ore bzlwzew O ond I .

g& rxtoyrtr r& pneob,i-t- Pttrl + t

'( A cE
1%'(a) +1

t]etwe .lhe- di*)\uence PnAl - Fe,(A) ig o ,t^n rba< bdween -1 '{o t

<L
uuig
ryrodul'*s +h"' a 'wtrnbex Le\ween o -;
Y*Y,
Au {re Vyo?tllit rynrnzlty, yLow-negattue , d{irnher\l*s 3, aals$pn6 -+ha
a,|.l.

friangte LY\eydr+t ariobtex rr* +D call fv ,d.isto.Irrcf , bw *l.tzclt'shib"ulrto,u

 6.q

> E2teYctaej
G'prI"
o) TV ( gev (.5) , Ber 6 -1))
b) TV (gev (,s), Bey (.q))
{o sotve *he oborre *raoo -tet 4^d gueroily TV (Ber(Pr) , Bev(pz))
Ue dtstrtbutovtS
{s ,^re o.)^e deo.t^ug w+th dtscre 3o
TV ( B, Fn') =
t E= I Fot{) - he'tx)l
E = to, 13 +*( X a., Ber(p) h(t)=p
Ffo; =l-p
Tv( Bev(pr), Bey(p") ): * I \ _ prJrl\
<- XEE [.txl

=* [lFri'l-Fr,r,ll * lFrl,l -hgll rl= LI

r i"l
{I lPr- Pz\ + lP,- Pl\J = \?"-Prl
= 2"1

+* (a) pz= "I TV(aer(.s), eer(.i)) : 1.S -.11 =, 4

(b) pr=t$ pz = '1 rv(eer(-s),Bev(.Q)) =\.q-.51 = .1

c) {v (r4tt) , u^if ;o,r1)

{1,te *,^,o dtstribrtknns Gne, coaHntrous eo nre use 'T\,t = * J 1{, t*l - {r' cx) I
Xrr E
eolr Exp (r ) f^=r(*) = Ae d* {ax ><vo
o o fort ><-< o

{"r( tl}^i+ [0,11 f

(z \-
(Lz I
i- t fot xe [o,tJ
O"
o fo'. x 4lortT
@ ?dt ,rto)\
Uv.rt
,

Tv =l_Z frI exptrl - unif to,tl\a* i- r

-@
1 o
=
*lin, -of "f,r -6x)d:c+ Jtal ",r az
o
J
\
ailp$)
I
=l2- I o + n+ {JL - a.li ]
l, o
o ) x
)p
t* xe (0, r) &^if [o,l] ) ErF(r)
=
'^ Ll* e
L
I +cl e
L
t1
e
 8"1o
(d) Ty 1X, X+a) &,.( ang ae (o,1), r":here
X* Ber(o.S;
ProJ -R.. assumes *ir-ot nrc knovo Wha+the ctistributtcrn .l X+a, wttl be once voe knornc
; n, Ber (p) And he is absotrrtety ri6la*, . t.e,Ts see hor,'r t{s dcvre
Gtiuen X t Bev (p) urh-rch n4eon( ]P( X= O) = l-P
E(x= l) = P
and o tol( qtl oLlaevval'r'r^ { X
Nonr tek co.ll +Prevrenl vavtalo [e Y = X+ a"
$ivst r"rc ftncl tha Support + Y , As vte see Y {.saJunckorr { X so V.to.ke,s
Y=l+a tngt^s& X=1 L4 Ygta'l+al
Y= o+0, rnvhevr,x=o
Noro rnre co-tc-ulalc {lzte- d'r.stAbukon e+ Y pcir,ttr,n{se
P(Y=a-) =F(x+o"=&) =P(X:0) =l-P
f (V = l+a) = f)(y-ua = 1+a) = T)(x = 1) : p
1P [Y = U
) r"ritt be O r,nrhen t1 * to,tl
)fr J+ltatty I^,{, lnave accordlr,tg to otrt protiu*
+"rt, X f*r X+a
f(ol = 1 11
vt
.l^
p( a) 1-'5 = 'g
.F.J
p(t+a) = '5 = '5
,
5
P(r) =

l*.G r,rot c rn{,cdaLe

TV(X,X.+a) = 1
a, xeE
X lPnt"r-P(x)\
trle ta.ntt uge
Px
('l
{* dts+rlbutor^ ,lX
(.) {* dishibution 6{Y
ft
E hevu is f o,1 , L,1+a] 4I otin*poirfu Ba€ bo'th @f = o
I p*tol - pyto)l + I Px(t) - PY(t)
Tv (X,X+a) =3d +
\

lf*t"l - P-tal1 lp*t,-o1 -Py(t-a)[

-t t.5 - ol + l-5 0l
I
{
lo - .Sl + lo .51 l=
(W, was -the afi,t'"iar" p'ro cectuve i* csgatoli'^t {v
Ne wi[l rnoh) Aee \^rr^dt toe ooutJ llatne dota nx,setl , Al.so tJ"af it
nteor.,a 'f* TV +o bz I
hrt thrg ig o- pvo,btervr {oa T,y. l-e].* see ho u.:
 B"u
We \^xtl c\i*ed tlris problenr- ensn- frr*h"r -to
6arn insi8ln1s {vttolPre rooture "l
-r.V
But $vst undevs*and sorne de{rtrdo'rx
) 9upport { Random Vanlabte X : Tt is a set ol vaJare^r jl^-ct X ta.ke.a
Erreg X+Beyn(p) Support -to,tt
Fo/ eg X ,r, Poss (A) Eupport
= t o,1 tz ,? -_ _.I
F. u{ X * Nttror) Supptrrt = R
> Dtsl'aUtt support : Nor,o suPpose rnre horre taro Yondo?), vorciobtes X o.rrd y
If their suppov-[s dorr'* shqye on-y cormmovr element . Jhen rrre sa,y X
and Y harre disjotnt supgart
f"" E X..,Bern(p) suppcrr+ofX = to,tt
X+a tuoa{heSupport =tqr!+a}
pntcrA e = o _tlne _fott owtvlq-/;ral ho.rne ho comrncrn
<,tem e,r* Vettcz
X ond x+ o. J.uo.w disjotr^* "^uUp-t
Il'ow t'ue invoked the corcept o{ dtsl'obtt ..prrr+ bec,o*se by de{,n ol rv
TV (pr,
Fr,) = I g.tar _ Fs(ntl
ffit
lenern: tt''us fv t's bound.ed
'^le a' set !v I (a-l pr,ovtd ec,.rtli<t) ottd { I co^
$nd A such lfrat rv blcornes I , *hat nri,tt be *lner,nc,.xtntuvta
rrolAs cyrd henet that vrrtlt be T-V(pe,
fir)' _ L
FL1h,"ru X rs a vastr'abte **rqt Sttoxss F"
Avppose Y ir o" vqnrabte {hat $"Uo*,
B/
8, A Sorree hov^c L.nor,.) Jhat x ond Y halre cttslov* eupgcrrt
Thevr, a cortu choose A = Sx ie 4 X ((.cotrtd hcrw lo.ken Sy )
suppcrr{
Norar Pr(al = P"(XeSx)= t byd"{,n{sutepartol x
Aitso B, (ol = ts, (Y e sx) = o becauge Y r.v. nev,er( {ukeavo-Lue
f*'* -'t?re se* Sx
*lence 11 a4 "* TV dtslayrce
-TV ( P*, tf
G,) = I x, Y har,e drs.lbtnt s"pprrt

{rrr,ted r^il{n thrs knot.otedle hse cav\ aytue $r. {Jtn pv€viotts 1x'ob\enn,
-U^akX toa su7por+ to,t} avtd X+ a" t"o..a suppm{ to, l+al
Qinevr c\ e (0, 1) .anz av(lPe,.Fe ) - 1 Equrvalevttty
abok A={o,1} tuppor+d X
Suppose

F(xeto,t3)=l I F(x+aefo,l!) = o
lP*(at-8,(A)l =l =TV
 8.1L

o a t t+a oA tt+a. oo" ll+q,

Az.5 ?='LE 8='OS
TV=l TV=l TV:1
Nd6 qbo\€ dvqhrn pmf f.^ Xe t o, t ]
X+a e tq, t+o.1
l$e see '{44ok qg N corna atose *o :zlyo *he dt's*ribukons v^e.\e tntc one
ono+her hrt the dtgtance cxterterrt that urre cleJ,tne el ie TV ctoes t"ot
chonge o.t o.[L , il attqtns tfs h^aximuy]r vaLte .L' t"rr all q e (o, t)
te ouy dlstonce crllevtort is 'no'F oble *o €J^copsulala 'lhe closenesg
the ttoo ?*L's i$ he suppor.t ,ts drsjotrrt. ;els see qnolher axarrrpte
+
te) Qlwn" Xi rg Bey (.5) 6nd \ ,"N(o,r-) Compu[e rv (zm(X"-l) , Z)
fl^rs* t&(s see w i+ XF NBer (p ) {hen by cLT
whal ir,*u*ton soys . LOe krro

J7
X^-r ., (d) )i (o,t) 3o h.ne Jfr(X^ -'5) (d)
n->0c , > ico,t)
h-)o
P(r-P) ?= "5
'5(t 5)
ho ltu'*s ,ts es+aHtsh.d q CLf thot os YL increasaa -thu elishibwh'o,t
o{ .
+ 2fi(F" -.f) -took !.ike +iro+ ,Nto,r1
3ru^*siqrrt
Xo ure would z+d titra* oufi d{stq!'\ce c,,.itevio-rt / t vtcasurt be omea srnaltert
and srna[ts]L 0g n tvrcreastu
not-c i^cf 06 2m ([ -.f') {s a dtscvre+e Y,V &
4t*o o Y'v h}e nrilt rue1he cQ'n C TV
Z - N(o,t) [S etvvt{introu*

tz{s t"t Jtvrd +tte t"Ppd + Am ( xn - '5 )

vr- Bevnou\lrs
h)e leno,,o X, Xt+Xr+X3+-""Xn=Y.rte'Arl d
h
Vo\ueg -\-t&ak )0r $o ) 1 2.b nl
take.s Lh n nn -l nJ
{orr a.ftxe.d vr
rrn (f -'5) , {fi (* - '5) , ofi(* - .5)
J
ostl Uiust cornstarlG

faZn(&-,5) =
Sn
{akes
 8.1ts

+t^at howerner toxge rL ts , crs 1*g os iK

$ni+e , {hu suppo-r..b
Nor,o note

% a ]h ( Xn -,5 ) has n+ 1 eterrrevrts 'iyt tn"e seTs .

-G'"*" are potnls on tlne
.trrlee qnd vue knora.r that qD/ cerrrlu,uroi-us distribxttora,
TOt ff*.
-the probatar.u+y Zero of a stn$e "p.w on the real \tne
jtnc{ TV ( Ztl ( Xrr - ttz) , Y)
To

TV = rna)L lB*tal-q,(A)l
AcE
LUc choose A cts euppor+ o{ lm ( & - !/z\ dt"at ^* t*nd eotLiex
Btor P ( 2lh (R-'5) e sn) = I by defh 4 *.vPt{ "{ x
B,(o) = P ( a Es,,) = F( E= r(*-T)) +F[z= r(*-*))
+ + P(z ,(, -*))
=
X -,ilt0, f ) tS covtHvtuourr so er^ch tevwt ts zero (o)
Btol = 0
Ao'^re frund 0, subset A-sx c E tho* tr**
-fV (16,8,) I
=
{ence thts {he value d Tv as tK upper loound rs 1
jg

Ao ]-V ( rm (Er-,a), Nto,r)) E I

Jtrrs is *lne eXac* opposite 4 we expect,ed. No rna#ry hornl
"rt^"+
-hn6e n -tB *he cltstancg r€Matns vnaxirnurn (te 1) as ,lon3 aA n ls $ntfe, .

TV a+ter( aLt ttis c,aunier evtctence . We sqy

I'tr[r{:ore h}e S*itl harre {ol1tr, tn,
,*ft', \n e hlrtt sktpthE cases d dtsjotnt supyort ond nre one goob to Bo
must lrob ,[ovg eb \^]hfrL ouY goal rnlqA
IaJe
we hod cr. true pora.rn eiev d.en-oled bv g+ . Atthough \^!e clid noL
Knovu "L+, N€ kner,^l i+ r"ltll be a fixe? " vrurrtbuc +ry pavrarrreker set O
Now nrc knoul ony genemL O € wilt nore distrile;l{lnw Pu
e n
' 4i" true parc.metelt g* wtrl harne distrnbukon
hle a\so kno^ JPn.,r.

\nre Coyt Cc.Lurtate the {ottowc\^3 f-trutc*tort

g '---+ TV (-8s, Pu*) re, Tv (Ps, 8n* ) Loy. be
cCI^stclered a, $unatton o) Ue O
Tne g+ *rflr'*gh unknonm, ,\a
a ftrytstayt heve
 g,11

Lefg see horc tht3 func'tton -toroks fit ce,r.,*qin cCI,.aea

Aoy 0=p P€(o,t) q^,Berr(p)

fr $xed $* n Ber(P*1
TV ( Bertp: , Ber(f )l as coM prrted {r,r oxexcise tal 8'
(u) : lP - P* t

fv( aer<pl, B,et(p*l )

c
?^x ts
P
? o
d I
,
E*U
q)
-tA {t-"-..."d I t,
q ,\>!
ay 0=X .\e{o,oc; - Exp(x)
X* {txed, 80* ^, Exp(t' ) l=r I
-fi. *t*g..t+y reb .\t = t \<t u if ,\>l
TV ( Exht^) , Exp(t) ) e
\
tI x" if \<1
= {. .I
o

\,'le Seg Xd Ax ; x has sotu{tctvr ?ro = .lmtA)

"C

A-t
{"" .\ < I h}€ uuitl do
)to @
,h. t
*f j dx- o
AEx dx +
J
)to
,\E TL 6'd, (x ) '-^

t" ,\ > t ,"le t^tilt do

t I
= ltr
do
xerx- dx dx + \a^x*
z-LJ o Ji' ] =
8o thc {urnakm,r -fV (Exptr), €xp(l)) Lsrcr -l.ike
fr(ryv,1, t' )) u"i\
TVto+el+ttl*
WoLfrarrrn oL
l_

l" axir
,tLt
 8.15
Ao the poCnt g[atl 'the prevtous dtsorsslorr rras,*he func+toa rV (l?s,tPe* ) wttt attain,
thevalue O rnrhev,r, 9= 0* . As t^)e aaAume.tyt tlrtis ca'urrse .Llaot O is identt$tobte
On(e hle know g rnre r,rritl haW uhrqu e JPg
As v.)e scw lhe $rnc*tora TV (F9, Fg+ ) o{taivrs -lhe vaL.re o at g
tde hltll iust trvrlt'tivnise- Tv..(&,
=g*
Pnn) onj. -{i^a rsn"r* it's 5e.o.fnat h}[l h{, uu{u-
0 ts s* 1 TraereJore hle r^ltrt'ilfnJ..i'
Le+ wE repeott *he orgumznt vvrrrl_ cteavty
;;ffi.:tT
tumfzstrare,1.

-t t hael g*
: a J-txed 'true pararnetex *hat f wanted lo knor^>
-+ f rnqde a' fucdoyr co[ed-Tv (Pe,B,r) +hot tsa vrneo,s"t= ?t drshance bw ihe
ttoo dtstributtsrts
& and Jpe"+
+ 1- hettt urrinimlse *hrs dris-tan ce , becouse os the d*hvtce wt\t xo *o -*ro *he.
g hlttl come ctose -b g{e

*'l 3o {lne p6ht at rnv[*ch Jur,rctcn fV(trn,$*) be h,tirtirvuse) roitt be g=#

A haue vrnl -lrue, paxamclfu g* ";itl
-+ Yot/a l!,1
Buf THERE's oNLY A L\TTLE PRoBI-E}t , T DoNt KsroW WHar 9+
\s 30 How wouLD T Frrlp rv (lPo, Ps*). AND TF t KNEW
B* wHY wouLD r NEED To MAKE {v" ir,,r rHE Frest PLACE 7.

-* 9o trnsteort cr! th^e exact TV (B,Pe,*) , T tnrltl vnake on e's+th atou t"*
TV ttsetJ iJ, f;/ (-q,n,"*; . Caral,hic bflaig hroutd took-Like
__t
TV(8, B")

TV (q, Pe*)

g* o

-+ go tnstead d r,ninrvudsing Tv { h,itt vwinirvttse -the es*trnator., rvie*

+ So ra,tirn,tinturn qf t'V v.yitl be ctose to g* d
+ l3ul nov0 see how cortplex anc{ usetessllneproceae has hec,crne
a. ulttl
{."t azrake an++he {trnctiorr TV itse$ colted fv
es+imoto.r.,
-tt'ren
4- wiu mtnimise {ltis estda*or*
functtcrr f" +o J.^.r d
Tnen a hlilt lrope tln-is 6 ts ctose {o g+' I
+ f Ptonned lrntikally +hob A r"rttt be obte 10 es+lmate D ulrtf^ -the laeip
,{r,r,"t dts+a.nce r,,ueasuhe -Tv but trvrrr our.t t hr-(Lt harc +o e,stimatZ
rV ttsetf & fhd! j*t r,nr9iyd 'rnoLn And iEnot pt ry delut hon:tomahe fv
rde nitl ae€ ,o. mrrclr, Yat+et< me'thod .f ea-ltmah yn- { . ls{i gen
 B.16
tr Kutt back - Leib ler ( rcU) dtrnergerrce
{ro known as vetattve €"ntropJ tkre KL clircrgence be'tv,leentoo pvobabitilXr-
rneosures is cle$ned as

E Fnr*,&(tfi)
2(e E
if E is drso^etd

KL(trs, tro )
(rt) ,Z- ( fet*l .f E t& conkruror,r,l
I tn
f g'(x)
E

If> J\trot syrnrnelric Kt- (P+,Pe,) # KL(pe,, [Pu)

Dulal" L

# X P;r'r to*l
ht*"tI Fe(xl 1I + xeg
PE t*s
\r bdcxl
Pe (x)
1 L

[Z) ]rTon negattrne KL (Pe, Fer) 2, o

Presenttnq +he prrd betonl j Noneedlo go{*rvotghrt if nottntere^c'Led"
lq(:c) a says
-logbt) ts concovle$rnc*ion ^bo Tevrsen
rt E I uXtx) ] -< 1",6 r [x1

KL = -i + l"l [t) o* rayhere f = {rt"r

E : +,={6,(x)
-KL = -j $ ,"e
[t) = n*ttJ
J+
Nour
J{'"€(iJ =,te,a ff)l
E
= ( r% (B(#))
=r%
({r{" t'')=r*l
J'= o

So -KL < o +> YL >/o p*9,

Z) DeJtruite KL (Po , !Pe,) = O 4hpn Pr = Pd
Nok provtnX it nnothervrattcatty aL n"4e', dtd not.
B=Ee' and in {his co6rse h}e
a
ihrs ts 3ood. 4hlr.^q-as -*n.u
KL(Ps,Pd) =o rneans
n-6x r"rilt hdl t*
sssurne iaentfl"nnaryt + U {tris t^rttt irnp\y E=0t
f,na e'f i$ ,* veptace O' 4 9*
<n abo're O+h
lrr) Does moV niltslry4heirianXle ineluali$
kL ,Po,) KL( Pu,&,) + KLlyi,,
Wp
{ pu,)
 8"'(7

Tn, KL dive rgence. ie v,at a drstance nreosure becouse it does not

cloes rtot sa+ts{.7 -triangte inelur.ati\ blrt h}e don't, want '844} {* ourL
ezttrvra*ton €iva|e^rfftr . Ati nre toavtt ts ' KL (Bg+,lPg ) -louches 'tl,re g axis
of 9= 0* Tu's hdp"pens hvhen KL =o hle corr. frnd {ha pammekr# +l.-',ot
hainfnztjses i+ & volta rnre hau€ ouy -Frue poxarneflr:

4,t^^tae quzs{ton wtlt a.tt8e orr horc'lo make KU(Pd,Ee ) wheru we clo.rt'
k;o^D g* beforrehond & +re ons^)crt ls I hlitl a6aln es+irna6, KL a.s ;)L
Eut{#uis 'ttvrae nre will hame a cr,ew.r strakcrXl or,1 our rpi716[6 .

F Esttmo,hnq ++Le l<L

Mote the ^noO,,frr*, ,r*{.rtltr
H g ts oux voriable , tt wttl vavh ,horuernerc 9* ,t -fiixed . yr]e dont knorc g#
dnd plavt *o cs+irylat it but it " is {rxed corrs+ant.
we will rnoke o jurtchoa ol g ustn6 KL (e1e) , Not-e here *hat KL -{.a
osgJmetric . t^ce hq\€ choSerr KL (e1 el & 1ot tll..oit'testway as it rullhelp ur
Tust +o n t*_rh.the rnemorg .i X is & %v {ottowi.n6. P.-.{ }aCxl
*he
-[-r,rerr
de{tvtidon % E[x] = X xpfxl
fnis pcxt co,^, be used +o f*^d ;".i*reetahora o{ .rrf junettcn ,{ ,c

e [ >e+al =I.(xz+3)ptx) ; E[sintxll=)tef E Sant")P("t)

xeE
E I trSrxl] = E {o6(>t) pcx) e l1(xtl = X gt"ct Ptr:
)
,teE x€E

LeG rnrrtte KL (B*, Er)

-Y
Ku (Frn, B)

Now ,roh u.yhat the lt{teyyrt iS

Ee* ; l2(fo (,())
I x
xeE ie+
txt logu* tzc) (wr ts a texrn+hd u.ntt
O* b-* r^iLl hare
depend orr
deyenQelce sn g
4ru's & a cans+av*., rde c{orr/t knon: i* brtt itr a
4** crr.rstanL Sure
 8.1&
+ No{d see if suppose X is o vandsrvr voxioble r qrty' {unc+ton h( ) trxll olso be
q Yondarn vou(toble .Fay ea-
o*, i^ Ji so is Sivt(X), 3o is I'u(x)
X rs Yov Eo is X2+4 , ,
Let3 coII h(x) = Y
h(x) U , L.L.NtettS tr,r
Now no rraatte't *hat *he d*tribxrLtovr
,n
yr rl ,p/a.s, tVl ie. *n thrxrl
P /a.s
E [r^tx[
-;,@ =
n r'=i
I\4oTavafed by *his hrc rePlace
rl
Es* h.a(lrn(x))l by 1
rL
ZL=1
{r(Ps(xi)) ivt {lne pwvio,u,
expre*iarr_
-+ *ttaottg ,re .!n ar€ avt expre;,rtovt .fon,
n
KL (-Fr",R) constar*, -* t2 Pe(xi)
E
,\r,rA +nt is becanse oi*houql/t the 9{ tA ey\mPsulalld tn the cc'vrsta.nr
gr"eat
u+D
teyrvr f- ptan to dt{Jo er,ttiak it fl"a 'ihe wtirutrnun4, ao -d(coyrstanL) = o
p J{-ax twtuwv llkelilr-ood
rL
O tPe*, Fo) en,ns*,ay* -1 tog fu
(xt)
rL
E,
ts+ t a.tnt +o wui.ni rnise K L $o it3 e.1ui vate *Y 4o 'r,ninr misi ng .tfi e H s
r,^:

wt"thowt {ha crvtstarrterm . AHftor,r6ln, *he h^14i vrLum vaLue

)lwe A 6i fa:lmeh yvttrruimtryn 'nay change
occr.rys tn:orr/4.

[:b -* ], ** Pexil
-+ -larnintiynX ^- $.rtatto-n
,ts e1,rivate"f,A rnaxintoin6- ihs vue6o.ttrre
$o aborre, otpvr:r*torr t's elutvatent ft Mr.x L *rS prCXi)
t=r u rs' '
960 n 1, .
17ro is jrrst a corts+awt so it wo*i, 4r.+ +r* g ct uytricl,u $cda,n ,,a.",axtr,.rs2
go -rrzte rurinn .s eluivanenr *o
#t Ti ,t
abov^e exp
F, (x;)

-+ NE khorll la@") + tofttb) + Lali.,) = Log (aa c) tA u onfurno.e ihar B

n l
hilo+
l=l Fslx;) - IrA
u
f!"txti
 B,1t
il.te expre,u'io n nol,o bdomu

4( rt p"tx;;
n
h/t0)t )
9e [=l

-r, Now
hle trarne *he protcle rn- maxtnnisa -|."6 ( {tel) ^rtt V ntheste

fel = ,1 Pr(xi1 .
.ttot
Narrl urraxi rta.rstlp
(-1, -tA (Jrol) {s elutvalenb -lo tnnoxi'raisi ?
vr,axtrv'rr'lel bz"o"s' load (') ia
if nrc j'ust
^or.t
Sda fi-e I -tln-ab

tu conbir,trt *.V increasinX (tn'c*tozr-' log(t)

3o *lne prrobtervr abor,.e i,a erluivalenLto

n E

#61lYerxt)

fu tc'the %to'x)vnum 'l"tketlhc'od Trtncipla

n
---+ lt..is Jtrrc+to F" t*;t is co0+t," -a.iketj.ho-D
,{ Junolt"n-

avtd tnre ^anaxir'rrr'ise it raYr* d

V {nhifr* ,.[ Ekeli hood

No.t6 r,,yhat r.re collad the.Ltketiho0 Junctcon l( putxil
t=1

,hrlfose {* ( allscne6 ol.ds{rtbukm.r iff$ndfott",^r{^g 5 obsevva*ton

t [,
2-, 7,1 , o]
7- ask urhnt is Mr probariti\ +hot S qenerate8 the- 2ata abone

fi E (Xr= r) [?tx"=2) B(x, --b) ts(lq=1) fr(Xs: o)

)6^ 'lilis wtll be a {uvtcdoa{ v . 1^fL h}tLL maxirntae ' -ttru's
l.rctma
U c.aU. +t^et vqhe
4
I the +vtnxiwutyw Jtke lihooD estmalol
*tu's ts ttne vatrre d v tt, which lhe sarnple
t t, L,l ,r,o3 ,,trtlt loe reolise)
^)t+h
morxfrnum Y.r/nci;!t}
olvnon1-- dt a'l'strit2'r-rto'rr fra'n V
 B.20
F Liketihoo? , Discrel-e case (t)
we t*itt norar delle deep 1a16 th-rs -[ike,tihooE functtarr- , J*v.{ Jor dtscrete c"otre
L+ (f , (B)eeO ) be a s{attskcal rnodel t/oe how somple n ttd, {
obsevvakaru ,wc o.ssum" +?rcy @yne lrnn a. distributton 'tflrat ls discretf
te E r's $ra"it6 or dountable
D4'n
---.
rnaP Lr, (ov;usu L)
4 ft" rnolel is a
-Ihe ttketihooD c{e}r'ned aa

Lrr, '. En x O --* -R

(&rxr,xa"')hro)
'+ F, tXr =)Lr1 Xz=x1rXs= x3. '*- ;16 =xnl
\dl't^t hle avp dorlnq- hew U lnare tlre questco.t e{ nyhat DistyibuL,ri"
o* ? tf e i= $rnp!e. .l^oe
ftenerate] cruY taenttftabte thrg ts eluiva.tent 1o *k r^rhcrt s
ryh
gnruJtc&-? o,,^r..da+a! so hle nnake o" {urtcttm Ln' te a ;funakor.r
An ( z(t rHt,:(r ---xn,9) +h6t -tetls ,rs {hZ pr.oha}clU! *hab -Fe
+ne pathcu[art dc^-tt r€.cttizatton Czt, fl2..,-etn> . tOe, p{"^ -to reNa:<afr-d,
g +het wlaxinruses thts probabill? f-^ **ru
p like.lihooD + y Bernoutlf rnodeV
q Xr , X.r. ,Ys- -. . Xn rc Ber(V1 frr sorne p€ (o,1)
8o E = +
Sufa[cor-tXi =to,t}
O= Paxameke,r, spoce
{et ? = (o,t;
t' (v.,xzrxl .xn) e {to,t3n
I Can c.dculate ,ttkehhood qs de$ned
eavLkL
a,i t\tt\5...-.xn,p) =
$(X=*, ,Xr=,&,Xg =\! . ..Xn=rn)
Xr, Yt ,Xy Xn c.Re trrrdepenEent
=
h
Pt (x, = )t) Ft (*r=xr) -
F(x^ = xr)
I-xr
=I[ Po'(' -P)
t=l
n n
xl:l ,(( Et=t (r-xil
P
(r-p)
J_funchtvw f ""t (r - p)n
- --n

'i;'i
 8. 21

> Ikelihod .folt 9ocsson )todeL

T-! Xt, Xr, Xs - - . ' xn 'rtid Patssh)
for sonte Xzo
Ao Xt totce qlL utsn nexatuue tnteqers hence [ = JNf (incl.*at"',qzert)
-T6e
porornetert g {s A here q'nd hx kvrow I>o so =
(oroo)

V(xr, >tr.---.XnI GNI[n, V A>o

J- (xr, Xrrxs----.x",, tr) = q,(X1=X1rXr=xzr " 'X,=xn)

We know X,, X". --..Xn ove ttrlep"r)unk
= F^tx, =xr) ffi(Xr=x") - - ... P^(xn xn) - =

r*'tl .ft" I f'n x f" e I

)qlo xr ! xr\ Xn t
6

(a)^ ),'' r' .fr f^

x 1.
x z\. x :\" x n!

L WL
-An ^ f,nzi
Juwc-+t e A'='
x, LyrL - x, l