Spring 2024 / 430.
314
Pairs of random variables
Part III
Jonghyun Choi
Associate Professor, SNU ECE
Slides referred from Youngmin Kim and Sunghyun Choi with partial permission
By last lectures…
• Functions of two random variables
• Expected values
Today:
• Conditioning by an event
• Conditioning by a random variable
• Independent random variables
• Bivariate Gaussian random variables
Conditioning by an event
Conditional joint PMF
• Definition 4.9 For discrete random variables 𝑋 and 𝑌 and an event, 𝐵
with 𝑃[𝐵] > 0, the conditional joint PMF of 𝑋 and 𝑌 given 𝐵 is
𝑃!,#|% 𝑥, 𝑦 = 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝐵].
• Theorem 4.19 For any event 𝐵, a region of the 𝑋, 𝑌 plane with 𝑃[𝐵] > 0,
𝑃!,# (𝑥, 𝑦)
𝑥, 𝑦 ∈ 𝐵,
𝑃!,#|% 𝑥, 𝑦 = / 𝑃[𝐵]
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
Example 4.13
Conditional joint PDF
• Definition 4.10 Given an event 𝐵 with 𝑃[𝐵] > 0, the conditional joint
probability density function of 𝑋 and 𝑌 is
𝑓!,# (𝑥, 𝑦)
𝑥, 𝑦 ∈ 𝐵,
𝑓!,#|% 𝑥, 𝑦 = / 𝑃[𝐵]
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.
Conditional expected value
• Theorem 4.20 For random variables 𝑋 and 𝑌 and an event 𝐵 of nonzero
probability, the conditional expected value of 𝑊 = 𝑔(𝑋, 𝑌) given 𝐵
(notated as 𝜇&|% ) is
Discrete: 𝐸 𝑊|𝐵 = ∑'∈)! ∑*∈)" 𝑔 𝑥, 𝑦 𝑃!,#|% (𝑥, 𝑦) ,
, ,
Continuous: 𝐸 𝑊|𝐵 = ∫+, ∫+, 𝑔 𝑥, 𝑦 𝑓!,#|% (𝑥, 𝑦) 𝑑𝑥 𝑑𝑦.
Conditional variance
• Definition 4.11 The conditional variance of the random variable 𝑊 =
-
𝑔(𝑋, 𝑌) (notated as 𝜎&|% ) is
-
𝑉𝑎𝑟 𝑊 𝐵 = 𝐸 𝑊 − 𝜇&|% 𝐵 .
• Theorem 4.21
- -
𝑉𝑎𝑟 𝑊 𝐵 = 𝐸 𝑊 𝐵 − 𝜇&|% .
Example 4.16
Quiz 4.8
Conditioning by a random variable
Conditional PMF
• Definition 4.12 For any event 𝑌 = 𝑦 such that 𝑃# (𝑦) > 0, the
conditional PMF of 𝑋 given 𝑌 = 𝑦 is
𝑃!|# 𝑥 𝑦 = 𝑃 𝑋 = 𝑥 𝑌 = 𝑦 .
• Theorem 4.22 For random variables 𝑋 and 𝑌 with joint PMF 𝑃!,# (𝑥, 𝑦),
and 𝑥 and 𝑦 such that 𝑃! (𝑥) > 0 and 𝑃# (𝑦) > 0,
𝑃!,# 𝑥, 𝑦 = 𝑃!|# 𝑥 𝑦 𝑃# 𝑦 = 𝑃#|! 𝑦 𝑥 𝑃! 𝑥 .
Example 4.17
Conditional expected value of a function
• Theorem 4.23 X and Y are discrete random variables. For any y ∈S_Y,
the conditional expected value of g(X,Y) given Y =y is
𝐸 𝑔 𝑋, 𝑌 𝑌 = 𝑦 = H 𝑔 𝑥, 𝑦 𝑃!|# (𝑥|𝑦) .
'∈)!
• A special case)
𝐸 𝑋 𝑌 = 𝑦 = H 𝑥 𝑃!|# (𝑥|𝑦) .
'∈)!
Example 4.17
Example 4.18
• In example 4.17, find 𝐸[𝑌|𝑋 = 𝑥] for 𝑥 = 1,2,3,4.
Conditional PDF
• Definition 4.13 For 𝑦 such that 𝑓# (𝑦) > 0, the conditional PDF of 𝑋
given {𝑌 = 𝑦} is
𝑓!,# (𝑥, 𝑦)
𝑓!|# 𝑥 𝑦 = .
𝑓# (𝑦)
This implies
𝑓!,# (𝑥, 𝑦)
𝑓#|! 𝑦𝑥 = .
𝑓! (𝑥)
Example 4.19
Theorem 4.24
𝑓!,# 𝑥, 𝑦 = 𝑓#|! 𝑦 𝑥 𝑓! 𝑥 = 𝑓!|# 𝑥 𝑦 𝑓# 𝑦 .
Proof goes with the following theorem:
𝑃 𝐴𝐵 = 𝑃 𝐴 𝐵 𝑃 𝐵 = 𝑃 𝐵 𝐴 𝑃 𝐴 .
Conditional expected value of a function
• Definition 4.14 For continuous random variables 𝑋 and 𝑌 and any 𝑦
such that 𝑓# (𝑦) > 0, the conditional expected value of 𝑔(𝑋, 𝑌) given
𝑌 = 𝑦 is
,
𝐸 𝑔 𝑋, 𝑌 𝑌 = 𝑦 = P 𝑔 𝑥, 𝑦 𝑓!|# 𝑥 𝑦 𝑑𝑥 .
+,
• Definition 4.15 (Conditional expected value) The conditional expected
value 𝐸[𝑋|𝑌] is a function of random variable 𝑌 such that if 𝑌 = 𝑦 then
𝐸[𝑋|𝑌] = 𝐸[𝑋|𝑌 = 𝑦].
Example 4.20
Iterated expectation 1
• Theorem 4.25
𝐸𝐸𝑋𝑌 =𝐸 𝑋 .
Iterated expectation 2
• Theorem 4.26
𝐸 𝐸 𝑔(𝑋) 𝑌 = 𝐸 𝑔(𝑋) .
Example 4.21
Problem 4.10.15
• Following the example 4.21, what is the PDF of 𝑊 = 𝑌 − 𝑋?
Quiz 4.9 (A)
Quiz 4.9 (B)
Summary
• Conditioning by an event
• Conditioning by a random variable
Next lectures:
• Independent random variables
• Bivariate Gaussian random variables
