AE 248: AI and Data science

Random Variables

Prabhu Ramachandran

2024-01-01

Random Variable

• 𝑋: Quantity of interest determined by the result of an experiment

Examples

1. Consider the game of tossing a coin and heads you pay the opponent Rs.10 and tails you
get paid Rs.10. Let 𝑋 be the total earnings of a player in 10 coin tosses.
2. Let 𝑋 denote the sum of two fair dice.
3. Let 𝑋 denote the lifetime of a battery.

Random Variables

• 𝑋: Quantity of interest determined by the result of an experiment

• One may assign probabilities to the values of 𝑋

Examples

1. Let 𝑋 denote the sum of two fair dice.

𝑃 {𝑋 = 2} = 𝑃 {(1, 1)} = 1/36

𝑃 {𝑋 = 3} = 𝑃 {(1, 2), (2, 1)} = 2/36

1
Random Variables

• Discrete RV: countable sequence of values (finite/infinite).

• Continous random variable: continuum of values.
• Indicator RV: 1 when event occurs; 0 o/w

CDF

• Cumulative distribution function or distribution function

• 𝐹 (𝑥) = 𝑃 {𝑋 ≤ 𝑥}
• 𝑃 {𝑎 < 𝑋 ≤ 𝑏} = 𝐹 (𝑏) − 𝐹 (𝑎)

Discrete Random Variables

• Probability mass function

• 𝑝(𝑎) = 𝑃 {𝑋 = 𝑎}
• 𝑝(𝑥𝑖 ) > 0 for some 𝑥𝑖 that occur
∞
• ∑𝑖 𝑝(𝑥𝑖 ) = 1
• 𝐹 (𝑎) = ∑∀𝑥≤𝑎 𝑝(𝑥)

Example

• Let 𝑋 have values 1, 2, 3

• Let 𝑝(1) = 1/2, 𝑝(2) = 1/3, 𝑝(3) = ?
• Plot PMF and CDF.
• Write out the CDF in words.

Example

• Let 𝑋 denote the sum of two fair dice

• Plot PMF and CDF.

𝑃 {𝑋 = 2} = 𝑃 {(1, 1)} = 1/36

𝑃 {𝑋 = 3} = 𝑃 {(1, 2), (2, 1)} = 2/36

2
Continuous RV

• 𝑓(𝑥): probability density function.

𝑃 {𝑋 ∈ 𝐵} = ∫ 𝑓(𝑥)𝑑𝑥
𝐵
𝑏
𝑃 {𝑎 < 𝑋 ≤ 𝑏} = ∫ 𝑓(𝑥)𝑑𝑥
𝑎

• Note that 𝑃 {𝑋 = 𝑎} = 0
• and 𝑓(𝑎) ≠ 0

Continous RV
𝑎
𝐹 (𝑎) = ∫ 𝑓(𝑥)𝑑𝑥
−∞
𝑑
𝐹 (𝑎) = 𝑓(𝑎)
𝑑𝑎

Question

Can 𝑓(𝑎) be negative for any 𝑎?

A) Yes
B) No
C) Don’t know

Question

Can 𝑓(𝑎) ≥ 1 for any 𝑎?

A) Yes
B) No
C) Don’t know

3
Question

How many mistakes are in the plot drawn on the board?

A) 0
B) 1
C) 2
D) 3

Recap of terms/notation

• Σ: Space of all events

• 𝑋 ∶ Σ → ℝ: Random variable
• 𝐹𝑋 ∶ ℝ → [0, 1] : CDF
• 𝑝𝑋 ∶ ℝ → [0, 1]: PMF
• 𝑓𝑋 ∶ ℝ → ℝ+ : PDF

Example

Let the pdf be given as:

𝑒−𝑥 𝑥 ≥ 0
𝑓(𝑥) = {
0 𝑥<0

• Find and plot the CDF, 𝐹 (𝑥)

• What is 𝑃 {𝑎 − 𝜖/2 ≤ 𝑋 ≤ 𝑎 + 𝜖/2}?
• What is 𝑃 {𝑋 > 1}?

Relationship to histograms

• Similarity of PDFs and histograms

• Emperical PDF through a histogram

Jointly distributed RVs

• Multiple RVs
• 𝐹 (𝑥, 𝑦) = 𝑃 {𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦}
• 𝐹𝑋 (𝑥) = 𝑃 {𝑋 ≤ 𝑥} = 𝑃 {𝑋 ≤ 𝑥, 𝑌 ≤ ∞} = 𝐹 (𝑥, ∞)

4
Joint PMF

• 𝑝(𝑥𝑖 , 𝑦𝑗 ) = 𝑃 {𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 }
• Can compute 𝑝𝑋 and 𝑝𝑌 from above.

Example

From textbook:
Suppose that 15 percent of the families in a certain community have no children, 20 percent
have 1, 35 percent have 2, and 30 percent have 3 children; suppose further that each child is
equally likely (and independently) to be a boy or a girl. If a family is chosen at random from
this community, if B is the number of boys, and G, the number of girls, in this family, find the
joint probability mass function.

Joint CDF/PDF
𝑎 𝑏
• 𝐹 (𝑎, 𝑏) = ∫−∞ ∫−∞ 𝑓(𝑥, 𝑦)𝑑𝑥𝑑𝑦
• 𝑓(𝑎, 𝑏) = 𝜕𝑎𝜕𝑏
2
𝜕
𝐹 (𝑎, 𝑏)
∞
• 𝑓𝑋 (𝑥) = ∫−∞ 𝑓(𝑥, 𝑦)𝑑𝑦

Question

2𝑒−𝑥 𝑒−2𝑦 0 < 𝑥 < ∞, 0 < 𝑦 < ∞

𝑓(𝑥, 𝑦) = {
0 otherwise

Find 𝑃 {𝑋 > 1, 𝑌 < 1},

A) 1.0
B) 𝑒−1 (1 − 𝑒−2 )
C) 𝑒−1 (1 − 𝑒−3 )
D) 𝑒−2 (1 − 𝑒−1 )

Question

2𝑒−𝑥 𝑒−2𝑦 0 < 𝑥 < ∞, 0 < 𝑦 < ∞

𝑓(𝑥, 𝑦) = {
0 otherwise

Find 𝑃 {𝑋 < 𝑎}

A) 1/2

5
 B) 1 − 𝑒−𝑎
C) 1 − 𝑒−2𝑎
D) 1 − 𝑒−𝑎
2

Question

2𝑒−𝑥 𝑒−2𝑦 0 < 𝑥 < ∞, 0 < 𝑦 < ∞

𝑓(𝑥, 𝑦) = {
0 otherwise

Find 𝑃 {𝑋 < 𝑌 },

A) 1.0
B) 1/2
C) 1/3
D) 1/4

Independent RVs

• 𝑃 {𝑋 ∈ 𝐴, 𝑌 ∈ 𝐵} = 𝑃 {𝑋 ∈ 𝐴}𝑃 {𝑌 ∈ 𝐵} for any 𝐴, 𝐵

• iff 𝑃 {𝑋 ≤ 𝑎, 𝑌 ≤ 𝑏} = 𝑃 {𝑋 ≤ 𝑎}𝑃 {𝑌 ≤ 𝑏} ∀𝑎, 𝑏

• 𝐹 (𝑎, 𝑏) = 𝐹𝑋 (𝑎)𝐹𝑌 (𝑏) ∀𝑎, 𝑏

• 𝑝(𝑥, 𝑦) = 𝑝𝑋 (𝑥)𝑝𝑌 (𝑦) ∀𝑥, 𝑦
• 𝑓(𝑥, 𝑦) = 𝑓𝑋 (𝑥)𝑓𝑌 (𝑦) ∀𝑥, 𝑦

Example

𝑒−𝑥 𝑥 ≥ 0
𝑓(𝑥) = {
0 𝑥<0

Find 𝑓𝑋/𝑌 (𝑎) of the random variable 𝑋/𝑌

Hint: find 𝐹𝑋/𝑌 (𝑎) = 𝑃 {𝑋/𝑌 ≤ 𝑎}

Answer

𝐹𝑋/𝑌 (𝑎) = 𝑃 {𝑋/𝑌 < 𝑎}

∞ 𝑎𝑦
=∫ ∫ 𝑒−𝑥 𝑒−𝑦 𝑑𝑥𝑑𝑦
0 0
1
=1−
𝑎+1

6
Multiple independent RV's

• Same idea, just more random variables

Conditional Distributions

𝑝𝑋|𝑌 (𝑥|𝑦) = 𝑃 {𝑋 = 𝑥|𝑌 = 𝑦}

𝑃 {𝑋 = 𝑥, 𝑌 = 𝑦}
=
𝑃 {𝑌 = 𝑦}
𝑝(𝑥, 𝑦)
=
𝑝𝑌 (𝑦)

𝑓(𝑥, 𝑦)
𝑓𝑋|𝑌 (𝑥|𝑦) =
𝑓𝑌 (𝑦)

Expectation

• Weighted average of possible values of 𝑋 with 𝑃 {𝑋 = 𝑥}

• First moment of the RV, 𝑋
• 𝐸[𝑋] = ∑𝑖 𝑥𝑖 𝑃 {𝑋 = 𝑥𝑖 }
∞
• 𝐸[𝑋] = ∫−∞ 𝑥𝑓(𝑥)𝑑𝑥

Clarifications on notation

• 𝑋 vs 𝑥
• 𝑃 {𝑋 < 𝑥}
• 𝐹 (𝑥) vs 𝐹𝑋 (𝑥)?
• 𝐸[𝑋] why not 𝐸(𝑥)?

Question

Consider a fair die. Let 𝑋 be the random variable corresponding to the number facing up.
What is 𝐸[𝑋]?

A) 3
B) 3.5
C) 2.5
D) None of the above

7
 AE 248: AI and Data science
Random Variables

Prabhu Ramachandran

2024-01-01

Random Variable

• 𝑋: Quantity of interest determined by the result of an experiment

Examples

1. Consider the game of tossing a coin and heads you pay the opponent Rs.10 and tails you
get paid Rs.10. Let 𝑋 be the total earnings of a player in 10 coin tosses.
2. Let 𝑋 denote the sum of two fair dice.
3. Let 𝑋 denote the lifetime of a battery.

Random Variables

• 𝑋: Quantity of interest determined by the result of an experiment

• One may assign probabilities to the values of 𝑋

Examples

1. Let 𝑋 denote the sum of two fair dice.

𝑃 {𝑋 = 2} = 𝑃 {(1, 1)} = 1/36

𝑃 {𝑋 = 3} = 𝑃 {(1, 2), (2, 1)} = 2/36

1
Random Variables

• Discrete RV: countable sequence of values (finite/infinite).

• Continous random variable: continuum of values.
• Indicator RV: 1 when event occurs; 0 o/w

CDF F(x)

• Cumulative distribution function or distribution function

• 𝐹 (𝑥) = 𝑃 {𝑋 ≤ 𝑥}
• 𝑃 {𝑎 < 𝑋 ≤ 𝑏} = 𝐹 (𝑏) − 𝐹 (𝑎)

Discrete Random Variables

• Probability mass function

• 𝑝(𝑎) = 𝑃 {𝑋 = 𝑎}
• 𝑝(𝑥𝑖 ) > 0 for some 𝑥𝑖 that occur
∞
• ∑𝑖 𝑝(𝑥𝑖 ) = 1
• 𝐹 (𝑎) = ∑∀𝑥≤𝑎 𝑝(𝑥)

Example

• Let 𝑋 have values 1, 2, 3

• Let 𝑝(1) = 1/2, 𝑝(2) = 1/3, 𝑝(3) = ?
• Plot PMF and CDF.
• Write out the CDF in words.

Example

• Let 𝑋 denote the sum of two fair dice

• Plot PMF and CDF.

𝑃 {𝑋 = 2} = 𝑃 {(1, 1)} = 1/36

𝑃 {𝑋 = 3} = 𝑃 {(1, 2), (2, 1)} = 2/36

2
Continuous RV

• 𝑓(𝑥): probability density function.

𝑃 {𝑋 ∈ 𝐵} = ∫ 𝑓(𝑥)𝑑𝑥
𝐵
𝑏
𝑃 {𝑎 < 𝑋 ≤ 𝑏} = ∫ 𝑓(𝑥)𝑑𝑥
𝑎

• Note that 𝑃 {𝑋 = 𝑎} = 0
• and 𝑓(𝑎) ≠ 0

Continous RV
𝑎
𝐹 (𝑎) = ∫ 𝑓(𝑥)𝑑𝑥
−∞
𝑑
𝐹 (𝑎) = 𝑓(𝑎)
𝑑𝑎

Question

Can 𝑓(𝑎) be negative for any 𝑎?

A) Yes
B) No
C) Don’t know

Question

Can 𝑓(𝑎) ≥ 1 for any 𝑎?

A) Yes
B) No
C) Don’t know

3
Question

How many mistakes are in the plot drawn on the board?

A) 0
B) 1
C) 2
D) 3

Recap of terms/notation

• Σ: Space of all events

• 𝑋 ∶ Σ → ℝ: Random variable
• 𝐹𝑋 ∶ ℝ → [0, 1] : CDF Gradient of CDF = PDF
Area to the left of a given point on PDF = CDF
• 𝑝𝑋 ∶ ℝ → [0, 1]: PMF
• 𝑓𝑋 ∶ ℝ → ℝ+ : PDF

Example

Let the pdf be given as:

𝑒−𝑥 𝑥 ≥ 0
𝑓(𝑥) = {
0 𝑥<0

• Find and plot the CDF, 𝐹 (𝑥)

• What is 𝑃 {𝑎 − 𝜖/2 ≤ 𝑋 ≤ 𝑎 + 𝜖/2}?
• What is 𝑃 {𝑋 > 1}?

Relationship to histograms

• Similarity of PDFs and histograms

• Emperical PDF through a histogram

Jointly distributed RVs

• Multiple RVs
• 𝐹 (𝑥, 𝑦) = 𝑃 {𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦}
• 𝐹𝑋 (𝑥) = 𝑃 {𝑋 ≤ 𝑥} = 𝑃 {𝑋 ≤ 𝑥, 𝑌 ≤ ∞} = 𝐹 (𝑥, ∞)

4
Joint PMF

• 𝑝(𝑥𝑖 , 𝑦𝑗 ) = 𝑃 {𝑋 = 𝑥𝑖 , 𝑌 = 𝑦𝑗 }
• Can compute 𝑝𝑋 and 𝑝𝑌 from above.

Example

From textbook:
Suppose that 15 percent of the families in a certain community have no children, 20 percent
have 1, 35 percent have 2, and 30 percent have 3 children; suppose further that each child is
equally likely (and independently) to be a boy or a girl. If a family is chosen at random from
this community, if B is the number of boys, and G, the number of girls, in this family, find the
joint probability mass function.

Joint CDF/PDF
𝑎 𝑏
• 𝐹 (𝑎, 𝑏) = ∫−∞ ∫−∞ 𝑓(𝑥, 𝑦)𝑑𝑥𝑑𝑦
• 𝑓(𝑎, 𝑏) = 𝜕𝑎𝜕𝑏
2
𝜕
𝐹 (𝑎, 𝑏)
∞
• 𝑓𝑋 (𝑥) = ∫−∞ 𝑓(𝑥, 𝑦)𝑑𝑦

Question

2𝑒−𝑥 𝑒−2𝑦 0 < 𝑥 < ∞, 0 < 𝑦 < ∞

𝑓(𝑥, 𝑦) = {
0 otherwise

Find 𝑃 {𝑋 < 1, 𝑌 < 1},

A) 1.0
B) 𝑒−1 (1 − 𝑒−2 )
C) 𝑒−1 (1 − 𝑒−3 )
D) 𝑒−2 (1 − 𝑒−1 )

Question

2𝑒−𝑥 𝑒−2𝑦 0 < 𝑥 < ∞, 0 < 𝑦 < ∞

𝑓(𝑥, 𝑦) = {
0 otherwise

Find 𝑃 {𝑋 < 𝑎}

A) 1/2