Each experiment

Probability space R F P
Power set

I sample space
I collection of subsets of R E 2h
P F to o B
mapping from
is a

r
what properties must I satisfy F E 2
i
0 E F A
g
ii r EE NT
A EI A EE
N A B E F AUB EF

what properties must P satisfy

P F on
i P O 0
i p r 1
iii p AUB PCA PCB if A B are disjoint
Example 1

HA HT TH TT
I
5 0
Hit HT THZETTZ
AH HT HH THT HH.TT HT TH3 HT.TT ETHTT3
HH HT TH HH.HT.TT HA TH.TT HT.TH.TT
HH HT TH TT

PCA flat
Example 2

r and F same as above

p A a 0.81 ICH HEA t 0.011 TT C A t

0.09 ICHTEA t 0.09 f THEA

Example 3

P A 0.5 ICH HEA t 0.5 ICTTEA

Exampleofaninprobability Fn

P 0 O 025
pf HH3 0.25 P ET131
P HH 1137 0 4
Events
A Er is an event Technically AEF
Two events A B are independent if

Plan B Pla PCB

Conditioning
has happened is
Probability Of event A given event B

PCANBI
PLA 1B
PCB

Bandom Yariables

D HH HT TH TT 5 F I P A i
Ly I Al

X i r R
Let X be the number of heads Then X expressed as
a function is given by

X HH 2 X TH L
X HT I y X TT O
 Probability Mars function example

f x PIX x

p wer x w x

x n
Ex PC were XCw 23 Pcr I

Let X be the Of heads for two coin toss expt

p L
f 2

pf were X Cut 23 p it 3
Y4
f l P Xii

p were X Cwhl 3 P HT TH3

1
2
Expectation
Ex f cx captures the average value of
EX
var E LX Ext EEK EXT capture variation

If X Of heads in the 2 coin toss expf

F X O pf o t l P X l t 2 PCx 2

Iz t 2
tf
Linearity of expectation
F axt by a a EX BEY
 Multipletariade's
X S IR
Y c IR

Jointdistribution

f G4y P X X y y
P Ew X Cw sign wer XCw y3

Conditional distribution

f yldly pcx.sc y y
PCX y y
PCYn y

independence

X and Y are independent RVs it

X b H fns g h
a y
are ind events f yCX y f fyly EfgCx hCyD
for all a b ECgcxDEChCyD
y
X and Y are
Coulx D 0
E yg E x Ely
uncorrelated
F XY EX Ey
Conditioniample D AH HT TH TTY F 2
P C 14 Axiom

P H 314 A HH T TH Event that ahead

was seen
p B 31g B HT TH TT Event that a tail
was seen

PCAIB PCAAB P HT 1 3
PCB 314
n 2
3

Independence example 1

r EHH.HT.TH.TT F T P c Icl Axiom

A HH HT P A t
Ya
B HH TH PCB L

NAIB PCfHH Ya PCA

Ya

H B are independent events in this

probability space
Independenceexampies
Axiom Note the

PCC O 5 I HH EC 0.5 I Trec strange den

A HH HT P os
B HH 1
3 PCB 05

P AIB PCE't o
IF PIA
Is

A and B are NOT independent events in this probability

space

Independentample
P c Ict Axiom

X w ICW H

Y w I Iwa T

X and Y are independent RVs because

X l Y 07 are Ind events

X 1 yet are Ind events

Keo o are Ind events

Y
are Ind events
X o yet
 X a CY b are independent events
for a and b
any

Conditionpectationexample
D HAH HHT HTH HTT THA THT TTH TTT

I 21
Axioms
Pcc GL 14

X Of Heads

Y of Tails
w of Heads in the first two tosses
2 1 Toss 3 Head i.e Z W ICW H

F Exly 3 Y
E E Ext'D 3 EY
I 5 EX

Prove
W 2 are independent Random Variables

Prove
Cov x y F XY HX Ely co
 More Conditional Expectation Examples
Experiment You keep tossing
a biased coin independently
till a Head let 0 be the bias
you get for heads

Question What is the expected number of

coin tosses
D EH T3
X coin tosses to get heads

X Any Infinite string of H T Position of first H

Let A be the event that the ith toss is Heads

F X F XIA PCA t f XIA PCA

l O t it F Ex Ct O

e 0 EX I
EX YO
 Experiment You keep tossing a
fair coin
infinitely
many times

X of tosses to get 2 consecutive heads

A Event that ith toss is heads

F Ex f XIA PCA t F XIA PCA

F XIA It EX

F XIA e
E XIA Aa PCAA F XIA AE MAE

F XIA Aa 2

F XIA AE 2T EX

F Ex G5 0.512 to 5C2tEX to 5 Cit EX

EX 1.5 O 75 EX

i EX 6
 Bayes Rule

PLAN B PCA 1B PCB PCBIA PCA

Bayes Ruleexample

prevalence Of COVID O 01

Prob of PCR test being re

when COVD is true 0.9

Prob of PCR test being ve
when COVID is false 0.01

A Random person takes a PCR test and the

test result says positive What is the
probability
Of COVID for this person

P COV'D True O 01

PCPCR True I could True O 9 False negative rate O

P PCR True I could False O Ol False Positive rate O Ol

P could True 1 PCR True

p PCR True I covid True p could _True 0 009

PC PCR True p PCB True

P PCR True P PCR True I could True PCcoviD True

p PCR True I could False PCCOVID False

O 9 0.01 t o ol 0.99
0.009 t 0.0099
O O 19

P could True PCR True 0.0092 0.5

O 019
 DiscreteDistributiamples
i Bernoulli O XE 0,13
O
PIX o i

P X 11 0

ii Binomial no Xt 0,1 n
m
05
PCx m
nom om Ci

iii Geometric 10 Xt 1,2 3

PIX K Ct 09 1.0

Properties
X Xn n Bernoulli O independent
X t Xn Y n Binomial n O
 Continuously bles

X S 113 and Range X is uncountable

F x P X EX CDF or cumulative Distribution

Function
Iroputies
l
Fx a
F C
o o

F is an
increasing function

xtdx3 Note the difference

f x x p XE x
to PMF
doc

Properties
f lx O

I
J f X da

cx dx 5 u
f is the derivative off
f

F x f x Fx is the anti derivative of f

Example 1
Xn Unit Ca b
I
0 see a
F Cx if
if Xt Laib o
b a
1 if I b a b

0 if 11lb
f x X E a a

if Xt aib
Ia
0 if X Z b
a 0 b

Expectation Variance of a continuous RY

N

EX f x f cx dx
Xi N

Yar Effx Ext EK Ext

Example
Xn Unit a b

PT EX Atb
2

YarEx CBI
12
Joint distribution

f x
y a P XE x xtdx YE Cy ytdy
y

Properties

f x y dy f cx
y

of yH g doc fy ly
f f yG4Y fy lybDf bD
yldly
fycy fyly
Example X y f y fx
f Xt 1,2 1,4
xylol g t if yt
if X E 2,3 Y C 2,3
Y Yg
n

0 Otherwise
3
2

f X 34 if Xt 1,2
Yc if It 2,3
O i 2 3 0 Otherwise
 The Normal Distribution

X n N Mah
4
f
ja expf ta.la lui

F EX µ EX MI oh

z n N 10,1 standard Normal

Feely tar expl izz

F 23 0 EG I

T
Let X 02 t M EX µ Effi Mi
H
X n N guest
 The Multivariate Normal

Let Z 2 22
Z n N 0,1
Zz n N 0,1

Then
fz z at expf tzcz.tk
z n N 83 too

EG Oo E 22T o

Let
X W Z t
µ

F Ex M

E X M x MJ E wz wz T
EWzzTw
W E 22
1
WT
W WT
E
X n NCH E
What about density

Let Zn N Od Id

Let X WZtµ

Then X n N E where E WW
µ

f x
L CtaCx x5Ecx xD
ftp.yaae
 Xu NCM E gut Rd Et Rd'd

f x I exptzlx xjsicx.su
a dkfE
EX M E x A Cx MY E

Let d 2 E E
fee I't f
Recall earlier Let µ Oo

X WZ and E WWT

What all e are legal

what about b a bind
E What all are legal
Jac d
d
In general for E E lRd when is it legal
Qn why are other values illegal

464 ftp.eae tiEp.a.a3ffe fIM

2tffpedPftz talxitxI 2ex xD

ftp.eaemftz feaffxi exaitci ey.se

f exa5 exp C taxi
xcx ftp.expf z ylx

x f Kilda f 2
z

X IX X n N loca l et

X h N o l

111
rly
Fx
ftp.expf tz ffcxa eoe 5tci ebx
J

z.EeaCxa e45
fzexPf expf tax

f Kalai Fx Cx
µ

XLIX X N ex I K

x N O l
 pls

e74
sci

Te

vn t

y
 x
it
42

2 X NN 1,075
22 em
712

24 2 Xin NG o B
 In
a

0 25,1 25

2 i
C
lX 74
712
05

Exercise
ab
sT legal Values of a b c dare
Xu N Oo
Cd a O d O b C E trad trad

What is the distribution of XLIX X

X 1 2 Xa
 General rules for conditioning of Normals
Mda
db Eaa daxda
E
f film Egg Egg Earsdaxds
Eba dbXda
CRd

b dgXd

Precision matrix
e
III i I
Let Xu N o E Nco a

we have that
Then
Xa IX Mais Aaj
n
se N

Xb
n N 0 E
Bb

where Naib Aaj hab Kb

Mrly
N Abia 1155
Xblxa da n

Xa n N 0 Eaa
Where Mya Aj's Nba Xa
 KL Divergence and Entropy

Consider distributions P and q of a discrete

random Variable values in 42 in
taking

KL Png FE P log 9
KL divergence

i KL P 1197 0 P q
ii KLCP1197 30

HCP FE Pilog P Entropy

Let p Ber O

Then HCP Oleg Lo t Ci 03 boy't

O
 Law of Large numbers

X X2 1 Xn Ind RVs

f f f Expectation and
n Variance exist

I n X t 1An

I b N Ex varenxi CLT

PHI EX I E 0 as n sa

LLN

Gamma distribution
If X in GammaCa p

Icsc
BI xd e
Bx

Exercise Zn NCO 1

P T E 221 n
I
 Random variable

Moments EX EX Ex
Centred moments 0 F x Ext EG Ex

Moment
Generating function

f Ct E etX
E it txt
tI tt

It f EX t
taExt t
f lo Ex
f lo Exa
 Maximum likelihood estimator
Parameter space
P Po O to NIM.D MERT
e.g
E W
set of distribution parameterised
distributions
by 0
Data k da kn drawn i i d from some Po

Task Estimate or
find 0 from the data

L O P x x ix 10

II Polki

2h10 n
EE log Polxi
E
LL 10
E log Polk l 1 log 7 1 10

ML estimate of 0 argmin 2210

Ot 0
 Ps NIM D gut IR

24,22 xn with Xi EIR

LLVM Eh log ftp.exptzcx xi

ta x µ t const

L2 M e
2 x gift const

I l LL
µ FE x
x O

Il 241µmol o
Min s Eye

Exercise

P N µ oh MEIR ftp 3

P Unit a b at R b E IR

X Iz Xp
i

am M.in X

bn m Ki
 P Bern O Otto D

X X x t X t 0,13

Om In EX