Lecture 23: Random Variables, Expectation, and

Variance
Introduction to Mathematical Modeling, Spring 2025

Lecturer: Yijun Dong

Helpful reference: All of Statistics by Larry Wasserman §2

Random variables

Definition (Random variable)

Given a probability space (!, A, P), a random variable is a mapping

X :!→R (1)

that assigns a real number X (ω) to each outcome ω ↑ !.

• Example: Flip a fair coin twice. Each ω ↑ ! = {HH, HT , TH, TT } can

be represented as a two-bit binary number X = X1 + 2X2 where Xi = 0
if tails and Xi = 1 if heads:

X (HH) = 3, X (HT ) = 2, X (TH) = 1, X (TT ) = 0.

• Cumulative distribution function (CDF) is a function FX : R → [0, 1]:

FX (x) = Pr[X ↭ x],

1
Random variables

Definition (Random variable)

Given a probability space (!, A, P), a random variable is a mapping

X :!→R (1)

that assigns a real number X (ω) to each outcome ω ↑ !.

• Cumulative distribution function (CDF) is a function FX : R → [0, 1]:

FX (x) = Pr[X ↭ x],

• Probability density function (PDF) is a function fX : R → [0, 1] s.t.

! b ! x
Pr(a < X < b) = fX (x)dx ↓ a, b ↑ R, and FX (x) = fX (t)dt.
a →↑

d
In particular, fX (x) = dx FX (x) for all x where FX is di”erentiable .

1
Examples: discrete random variables

Discrete random variable: X : ! → Z with PDF fX : Z → [0, 1]

• Bernoulli random variable with parameter p ↑ [0, 1]:



p
 if x = 1,
fX (x) = 1↔p if x = 0,


0 otherwise.

• Binomial random variable with parameters n ↑ N and p ↑ [0, 1]:

& '
n x
fX (x) = p (1 ↔ p)n→x .
x

• Poisson random variable with parameter ε > 0:

εx e →ω
fX (x) = .
x!

2
Examples: continuous random variables

Continuous random variable: X : ! → R with PDF fX : R → [0, 1]

• Uniform random variable supported over a closed interval [a, b]:

(
1
b→a if x ↑ [a, b],
fX (x) =
0 otherwise.

• Gaussian random variable with mean µ ↑ R and variance ϑ 2 (ϑ > 0):

1 →
(x→µ)2
fX (x) = ↗ e 2ω 2 .
2ϖϑ 2

• Exponential random variable with parameter ε:

(
εe →ωx if x ↘ 0,
fX (x) =
0 otherwise.

3
Independent random variables

Definition (Independent random variables)

Two random variables X and Y are independent if for all x, y ↑ R,

Pr(X = x, Y = y ) = Pr(X = x) Pr(Y = y ). (2)

Example: Let X1 , · · · , Xn be independent Bernoulli random variables with

)n
parameter p ↑ [0, 1]. Then, X = i=1 Xi is a binomial random variable with
parameters n and p.
Proof: For any x ↑ N, let C(n, x) = {S ≃ [n] : |S| = x} be the set of all
subsets of [n] with size x. Then,
fX (x) = Pr(X1 + · · · + Xn = x)
*
= Pr(Xi = 1 ↓ i ↑ S) Pr(Xi = 0 ↓ i ↑
/ S)
S↓C(n,x)
* & '
n x
= p x (1 ↔ p)n→x = p (1 ↔ p)n→x .
x
S↓C(n,x)
4
Multivariate distributions

• Random vector: X = (X1 , ..., Xn ) where X1 , ..., Xn are random variables.

• The joint distribution of (X1 , ..., Xn ) gives the PDF of X,
fX : Rn → [0, 1]:
fX (x) = f(X1 ,...,Xn ) (x1 , ..., xn ) = Pr(X1 = x1 , ..., Xn = xn ).

• If X1 , ..., Xn are independent, then

n
+
fX (x) = fXi (xi ).
i=1

• The marginal distribution of Xi is the PDF fXi : R → [0, 1] such that

!
fXi (x) = fX (x1 , · · · , xi→1 , x, xi+1 , . . . , xn ) dx1 · · · dxi→1 dxi+1 · · · dxn .
Rn→1

5
Multivariate distributions

• Random vector: X = (X1 , ..., Xn ) where X1 , ..., Xn are random variables.

• The joint distribution of (X1 , ..., Xn ) gives the PDF of X,
fX : Rn → [0, 1]:
fX (x) = f(X1 ,...,Xn ) (x1 , ..., xn ) = Pr(X1 = x1 , ..., Xn = xn ).

• If X1 , ..., Xn are independent, then

n
+
fX (x) = fXi (xi ).
i=1

• The marginal distribution of Xi is the PDF fXi : R → [0, 1] such that

!
fXi (x) = fX (x1 , · · · , xi→1 , x, xi+1 , . . . , xn ) dx1 · · · dxi→1 dxi+1 · · · dxn .
Rn→1
• Example: a Gaussian random vector X ↑ Rn with mean µ ↑ Rn and
covariance ! ↑ Rn↔n with ! ⇐ 0 is given by
& '
1 1
fX (x) = , exp ↔ (x ↔ µ)T !→1 (x ↔ µ) .
(2ϖ)n det(!) 2
5
Expectation, Variance, and Moments

• For a continuous random variable X with PDF fX , the expectation of

X is
!
E[X ] = x fX (x) dx.
R
• For a discrete random variable X with PDF fX , the expectation of X
is
*
E[X ] = x fX (x).
x↓Z

• For a continuous random variable X with PDF fX and µ = E[X ]:

• The variance of X is
!
V[X ] = E[(X → µ)2 ] = (x → µ)2 fX (x) dx.
R
• For k = 1, 2, · · · , the k-th moment of X is
!
E[X k ] = x k fX (x) dx.
R

6
Properties of expectation

• (Linearity) For any random variables X1 , ..., Xn and constants

a1 , ..., an ↑ R, - n .
* * n
E ai Xi = ai E [Xi ] (3)
i=1 i=1

• If X1 , ..., Xn are independent random variables, then

- n . n
+ +
E Xi = E [Xi ] (4)
i=1 i=1

7
Properties of variance

• V(X ) = E[X 2 ] ↔ E[X ]2

• V(aX + b) = a2 V(X )

• if X1 , ..., Xn are independent random variables

- n . n
* *
V ai Xi = ai2 V [Xi ] (5)
i=1 i=1

8
Supplementary material: moment generating function

Definition (Moment generating function (MGF))

The moment generating function of a random variable X is defined as
!
MX (t) = E[e tX ] = e tx fX (x) dx, t ↑ R.
R

Properties:
• By Taylor expansion,
* tk↑
t2
MX (t) = 1 + tE[X ] + E[X 2 ] + · · · = E[X k ].
2 k!
k=0
Assuming all moments are well-defined, for any k ↑ N,
dk
dt k M X (t) = E[X k
].
• If X1 , ..., Xn are independent random variables with MGFs MX1 , ..., MXn ,
)n
then the MGF of Y = i=1 Xi is
MY (t) = MX1 (t)MX2 (t) · · · MXn (t).
9