Scribd
Upload
45 views6 pages

Lecture 15

The document defines conditional distributions and densities. It provides examples of how to calculate the conditional distribution F_x|M_ and conditional density f_x|M_ when the event M can be expressed in terms of a random variable X. It then discusses total probability theorems for continuous random variables, including how to calculate P(A|X=x) and derive Bayes' rule for continuous distributions. Bayes' rule states that the conditional density f(x|Y=y) is equal to the conditional density of Y given X multiplied by the marginal density of X, divided by the marginal density of Y.

Uploaded by

amanmatharu22
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
45 views6 pages

Lecture 15

The document defines conditional distributions and densities. It provides examples of how to calculate the conditional distribution F_x|M_ and conditional density f_x|M_ when the event M can be expressed in terms of a random variable X. It then discusses total probability theorems for continuous random variables, including how to calculate P(A|X=x) and derive Bayes' rule for continuous distributions. Bayes' rule states that the conditional density f(x|Y=y) is equal to the conditional density of Y given X multiplied by the marginal density of X, divided by the marginal density of Y.

Uploaded by

amanmatharu22
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Lecture - 6

February 10, 2012


Denition 0.1. Conditional Distribution: Let M denote an event for which P(M) =
0. Assuming the occurrence of M, the conditional distribution F
_
x|M
_
of a random variable
X is dened as
F
_
x|M
_
= P
_
X x|M

=
P
_
X x, M

P[M]
Note that F
_
|M
_
= 1 and F
_
|M
_
= 0
Furthermore, the conditional distribution has all of the properties of an ordinary distri-
bution function. For example,
P
_
x
1
< X x
2
|M

= F
_
x
2
|M
_
F
_
x
1
|M
_
=
P
_
x
1
< X x
2
|M

P[M]
Denition 0.2. Conditional Density: The conditional density f
_
x|M
_
is dened as
f
_
x|M
_
=
d
dx
F
_
x|M
_
= lim
x0
P
_
x < X x + x|M

x
The conditional density has all of the properties of an ordinary density function.
Conditional distribution when X is dened in terms of X
If M is an event that can be expressed in terms of r.v. X, then F
_
x|M
_
can be determined
from ordinary distribution F(x). Below we give several examples.
1
As a rst example, consider M = [X a] and nd both
F
_
x|M
_
= F
_
x|X a

= P
_
X x, X a

F
_
x|X a

. Note that
F
_
x|X a

= P
_
X x, X a

=
P
_
X x, X a

P
_
X a

If x a, we have
_
X x, X a

=
_
X a

and
F
_
x|X a

=
P
_
X a

P
_
X a
= 1 , x a
If x < a, then
_
X x, X a

=
_
X x

and
F
_
x|X a

=
P
_
X x

P
_
X a
=
F
X
(x)
F
X
(a)
, x < a
At x = a , F
_
x|X a

would jump 1
F
X
(a)
F
X
(a)
if F
X
(a) = F
X
(a).
The conditional density is
f
X
_
x|X a
_
=
d
dx
F
_
x|X a

=
d
dx
_
F
X
(x)
F
X
(a)
_
=
_
F
X
(x)
F
X
(a)
, x < a
0 o.w.
As a second example, consider M = [b < X a] so that
F
_
x|b < X a

=
P
_
X x, b < X a

P
_
b < X a

Since,
_
X x, b < X a

=
_
b < X a

a x
=
_
b < X x

b < x < a
2
F
_
x|b < X a

=
_
_
_
1 a x
F
X
(x)F
X
(b)
F
X
(a)F
X
(b)
, b < x < a
0 x b
F
_
x|b < X a

is continuous at x = b.
At X = a , F
_
x|b < X a

jumps 1
F
X
(a)F
X
(b)
F
X
(a)F
X
(b)
, a value is zero if F
X
(a) = F
X
(a).
The corresponding conditional density is
f
_
x|b < X a
_
=
d
dx
F
_
x|b < X a
_
Example :
Find f
_
x||X | k
_
, Where X is N(,
2
).
First note that (X )|X k
_
k < X + k
_
. so that,
P
_
|X | k

= P
_
k < X + k

=
1

2
_
+k
k
e
(1/2)(
x

)
2
dx
By change of variable, this last result becomes,
P
_
|X | k

= 2(k) 1
Hence, by previous example we have
f
_
x||X | k
_
=
_
1

2
e
(1/2)(
x

)
2
2(k)1
, k < X + k
0 o.w.
3
Total Probability - Continuous form
Let X be any random variable and dene B =
_
X x

. Let A
1
, , A
n
be a partition of
sample space S. That is,
n
_
i=1
A
i
= S and A
i
A
j
= i = j
From the discrete form of the Theorem of Total Probability discussed in chapter 1,
we have
P(B) = P
_
B|A
1
_
P(A
1
) + P
_
B|A
2
_
P(A
2
) + + P
_
B|A
n
_
P(A
n
)
Now B =
_
X x

, This becomes,
P
_
X x

= P
_
X x|A
1
_
P(A
1
) + P
_
X x|A
2
_
P(A
2
) + + P
_
X x|A
n
_
P(A
n
)
Hence,
F
X
(x) = F
_
x|A
1
_
P(A
1
) + F
_
x|A
2
_
P(A
2
) + + F
_
x|A
n
_
P(A
n
)
f
X
(x) = f
_
x|A
1
_
P(A
1
) + f
_
x|A
2
_
P(A
2
) + + f
_
x|A
n
_
P(A
n
)
Several useful formulas can be derived from this result. For example, let A be any event,
and let X be any random variable. Then we can write the version of Bayes rule as
P
_
A|X x
_
=
P
_
A, X x

P[X x]
=
P
_
X x|A

P(A)
P[X x]
=
F(x|A)P(A)
F(x)
=
F(x|A)P(A)
F
_
x|A
1
_
P(A
1
) + F
_
x|A
2
_
P(A
2
) + + F
_
x|A
n
_
P(A
n
)
As a second example, we derive a formula for P
_
A|x
1
< X x
2
_
. Now, the conditional
distribution F(x|A) has the same properties as an Ordinary distribution. That is, we can
write P
_
x
1
< X x
2
|A
_
= F(x
2
|A) F(x
1
|A) so that
4
P
_
A|x
1
< X x
2
_
=
P
_
x
1
< X x
2
|A

P
_
x
1
< X x
2
P(A)
=
F
X
(x
2
|A) F
X
(x
1
|A)
F
X
(x
2
) F
X
(x
1
)
P(A).
In general, we can not write,
P
_
A|X = x
_
=
P
_
A, X = x

P[X = x]
. Since, this may result in an indeterminant (
0
0
) form Instead, we must write,
P
_
A|X = x
_
= lim
x0
+
P
_
A|x < X x + x

= lim
x0
+
F
X
(x + x|A) F
X
(x|A)
F
X
(x + x) F
X
(x)
P(A).
= lim
x0
+
F
X
(x+x|A)F
X
(x|A)
x
F
X
(x+x)F
X
(x)
x
P(A)
Which yields,
P
_
A|X = x
_
=
f(x|A)
f(x)
P(A) (1)
Now, multiply both sides of this last result by f
X
and integrate to obtain
_

P
_
A|X = x
_
f
X
(x)dx = P(A)
_

f(x|A)dx
But, the area under f(x|A) is unity. Hence, we obtain
P(A) =
_

P
_
A|X = x
_
f
X
(x)dx
The continuous version of the Total Probability Theorem .
Bayes Theorem - Continuous form
Target : to show
f(x|Y = y) =
f
Y
_
y|X = x
_
f
Y
(y)
f
X
(x).
5
We have observed from (1),
f(x|A) =
P
_
A|X = x

P(A)
f
X
(x) (2)
Suppose,
A =
_
y < Y y + y
_
Then
f(x|Y = y) = lim
y0
f
_
x|y < Y y + y
_
=
lim
y0
P
_
Y y+y|x

P(Y y|x)
y
lim
y0
P
_
y<Y y+y

P(Y y)
y
f
X
(x) from (2)
=
f
Y
_
y|X = x
_
f
Y
(y)
f
X
(x).
Alternative Way (as done in the class):
f(x|Y = y) = lim
y0
f
_
x|y < Y y + y
_
= lim
y0
lim
x0
P
_
x < X < x + x|y < Y y + y

x
= lim
y0
lim
x0
P
_
y < Y y + y, x < X < x + x

xP
_
y < Y y + y

=
lim
x0
lim
y0
P
_
Y y+y|x<X<x+x

P
_
Y y|x<X<x+x

y
lim
x0
P
_
x<X<x+x

x
lim
y0
P
_
Y y+y

P(Y y)
y
=
lim
x0
f(y|x < X < x + x)
f
Y
(y)
f
X
(x)
=
f
Y |X
(y|x)f
X
(x)
f
Y
(y)
(3)
Important Remark: We assume all limits exist.
6

You might also like