Basic Quantitative Method

Chapter 5 Moments (動差)

Department of Economics
National Taiwan University

2023.7.21.

(NTU Econ) Basic Quantitative Method 2023.7.21. 1 / 51

 Moments

Section 1

Moments

(NTU Econ) Basic Quantitative Method 2023.7.21. 2 / 51

 Moments

Moments

Moments can help us to summarize the distribution of a random

variable.
Analogy: the height, weight, hair color...etc. of a person
Two important moments:
Expectation
Variance

(NTU Econ) Basic Quantitative Method 2023.7.21. 3 / 51

 Moments

Expectation

Definition (Expectation)
Let S = supp(X). The expectation of a random variable X is defined
as
⎧
⎪∑x∈S x f (x)
⎪
⎪ discrete
E(X) = ⎨
⎪
⎪
⎪ x f (x)dx continuous
⎩∫x∈S

Expected value; Mean (value)

A probability-weighted sum of the possible values.
Expectation is a constant.
A conventional notation: E(X) = µ

(NTU Econ) Basic Quantitative Method 2023.7.21. 4 / 51

 Moments

Example: Fair Price for a Stock

An investor is considering whether or not to invest in a stock for

one year.
Let Y represent the amount by which the price changes over the
year with the following distribution
y −2 0 1 4
f (y) 0.1 0.4 0.3 0.2
Then the expected earning is

E(Y) = 0.9

That is, “on average, the investor expects to earn 0.9.” ⇐ What
does this mean?
(NTU Econ) Basic Quantitative Method 2023.7.21. 5 / 51
 Moments

Simulation Results

If you invest in this stock N years. Let Yi denote the price change
for year i, and the average earning is thus ∑ i=1
N
Yi
N
∑N
i=1 Yi
We can see that N is very close to E(Y) = 0.9 when N is
large.
1.5
1.0
Average Earning

0.5
0.0
−0.5

0 200 400 600 800 1000

(NTU Econ) Basic Quantitative Method 2023.7.21. 6 / 51

 Moments

Expectation

Theorem (The Rule of Lazy Statistician)

Let X be a random variable, and let g(⋅) be a real-value function.
Then

E(g(X)) = ∑ g(x) f (x), E(g(X)) = ∫ g(x) f (x)dx

x x

Example:
⎧
⎪
⎪ 2 with P(X = 2) = 1/3
⎪
⎪
X = ⎨ 1 with P(X = 1) = 1/3
⎪
⎪
⎩ −1 with P(X = −1) = 1/3
⎪
⎪

Consider g(X) = X 2 , find E(g(X))

(NTU Econ) Basic Quantitative Method 2023.7.21. 7 / 51
 Moments

Variance and Standard Deviation

Definition (Variance/SD)
The variance of a discrete random variable X is defined by
⎧
⎪∑x (x − E(X))2 f (x)
⎪
⎪ 2
Var(X) = E [(X−E(X)) ] = ⎨
⎪
⎪
⎪ (x − E(X))2 f (x)dx
⎩∫ x

It describes how far values lie from the mean.

A conventional notation: Var(X) = σ 2
√
The standard deviation is SD(X) = Var(X), and denoted by
SD(X) = σ.

(NTU Econ) Basic Quantitative Method 2023.7.21. 8 / 51

 Moments

Constant as a Random Variable

Given a constant c, then

E(c) = c,

and
Var(c) = 0.
Therefore,
E(E(X)) = E(X)

(NTU Econ) Basic Quantitative Method 2023.7.21. 9 / 51

 Moments

Some Important Properties

Given constants a and b:

E(aX + b) = aE(X) + b
Var(aX + b) = a 2 Var(X)
E(X − E(X)) = 0
Var(X) = E(X 2 ) − [E(X)]2

(NTU Econ) Basic Quantitative Method 2023.7.21. 10 / 51

 Moments

Example: Fair Price for a Stock

Definition
The fair price of a stock is defined by a price such that the expected
return equals the risk free rate.

Suppose that the stock price is p t .

The return is
(p t + Y) − p t Y
=
pt pt
As an alternative, the investor can put the money in the bank
with a 5% interest rate (risk-free).
Hence, E ( pYt ) = 0.05 shows that p t = 18 is the fair price of the
stock.
(NTU Econ) Basic Quantitative Method 2023.7.21. 11 / 51
 Moments

Expectation as the Best Constant Predictor

Consider a constant predictor of X, say c.

Mean Square Prediction Error

MSPE = E [(X − c)2 ]

It can be shown that

E(X) = arg min E [(X − c)2 ]

c

(NTU Econ) Basic Quantitative Method 2023.7.21. 12 / 51

 Moments

More on Expectation

In general, unless g(⋅) is linear,

E(g(X)) ≠ g(E(X))

For instance, in the previous example, g(X) = X 2 ,

4
E(X 2 ) = 2 ≠ = [E(X)]2
9

One more example,

1 1
E( )≠
X E(X)
Proof: by Jensen’s Inequality
(NTU Econ) Basic Quantitative Method 2023.7.21. 13 / 51
 Moments

Jensen’s Inequality

Theorem
If X is a random variable and g(⋅) is a convex function, then

E(g(X)) ≥ g(E(X))

Proof.

(NTU Econ) Basic Quantitative Method 2023.7.21. 14 / 51

 Moments

Standardized Random Variables

As expectation and variance are the two most important

moments, sometimes we will denote the random variable as

X ∼ (E(X), Var(X)) or X ∼ (µ, σ 2 )

Definition (Standardized Random Variables)

Given X ∼ (µ, σ 2 ), and let

X−µ
Z=
σ
Then Z ∼ (0, 1) is called a standardized random variable.

(NTU Econ) Basic Quantitative Method 2023.7.21. 15 / 51

 Moments

Moments

k-th Moments
E(X k )
k-th Central Moments

E[(X − E(X))k ]

k-th Standardized Moments

k
⎛⎡⎢ X − E(X) ⎤⎥ ⎞
γ k = E ⎜⎢⎢ √ ⎥ ⎟
⎥
⎝⎣⎢ Var(X) ⎥ ⎠
⎦

(NTU Econ) Basic Quantitative Method 2023.7.21. 16 / 51

 Moments

Skewness

3
3rd standardized moment (Skewness): γ3 = E [( √X−E(X)
Var(X)
)]

(NTU Econ) Basic Quantitative Method 2023.7.21. 17 / 51

 Moments

Kurtosis

4
4th standardized moment (Kurtosis): γ4 = E [( √X−E(X)
Var(X)
)]

(NTU Econ) Basic Quantitative Method 2023.7.21. 18 / 51

 Moment Generating Functions

Section 2

Moment Generating Functions

(NTU Econ) Basic Quantitative Method 2023.7.21. 19 / 51

 Moment Generating Functions

Moment Generating Functions

Definition (MGF)
Let X be a discrete (continuous) random variable, and the pmf (pdf)
is f (x). Given h > 0 and for all −h < t < h, if the following function

M X (t) = E(e tX )

exists and is finite, it is called the moment generating function

(MGF) of the random variable X.

One use of the MGFs is that, in fact, it can generate moments of

a random variable.

(NTU Econ) Basic Quantitative Method 2023.7.21. 20 / 51

 Moment Generating Functions

Properties

Theorem (Moment Generating)

(k) (k)
E(X k ) = M X (0) = M X (t)∣t=0 ,
(k)
where M X (t) denotes the k-th derivative of M X (t).

Proof: expand e tX as

(tX)2 (tX)3 (tX)4

e tX = 1 + tX + + + +⋯
2! 3! 4!

(NTU Econ) Basic Quantitative Method 2023.7.21. 21 / 51

 Moment Generating Functions

Properties

Theorem (Uniqueness)
For all t ∈ (−h, h), if M X (t) = MY (t), then X and Y has exactly the
same distribution, FX (c) = FY (c) for all c ∈ R.

Proof: Beyond the scope of this course (via so-called the inverse
Fourier transform).

(NTU Econ) Basic Quantitative Method 2023.7.21. 22 / 51

 Moment Generating Functions

Properties

Theorem (MGF of Linear Transformations)

Given the MGF of X is M X (t). Let Y = aX + b, then

MY (t) = e bt M X (at).

Proof: By definition.

(NTU Econ) Basic Quantitative Method 2023.7.21. 23 / 51

 Covariance and Correlation

Section 3

Covariance and Correlation

(NTU Econ) Basic Quantitative Method 2023.7.21. 24 / 51

 Covariance and Correlation

Expected Values of Functions of Bivariate Random Variables

Definition
Let X and Y be discrete (continuous) random variables with joint
pmf (pdf) f XY (x, y). Let g(X, Y) be a function of these two random
variables, then:

E[g(X, Y)] = ∑ ∑ g(x, y) f XY (x, y)

x y

E[g(X, Y)] = ∫ ∫ g(x, y) f XY (x, y)d ydx

x y

(NTU Econ) Basic Quantitative Method 2023.7.21. 25 / 51

 Covariance and Correlation

Covariance

Definition (Covariance)

Cov(X, Y) = E([X − E(X)][Y − E(Y)])

It is typically denoted by σXY .

A measurement of comovement among two random variables.
x − E(X) y − E(Y) Cov(X, Y)
+ + +
− − +
+ − −
− + −
It can be shown that

Cov(X, Y) = E(XY) − E(X)E(Y)

(NTU Econ) Basic Quantitative Method 2023.7.21. 26 / 51
 Covariance and Correlation

Properties

Theorem
Given constants a, b, c, and d
E(aX + bY) = aE(X) + bE(Y)

Cov(X, X) = Var(X)

Cov(X, c) = 0

Var(aX + bY) = a2 Var(X) + b 2 Var(Y) + 2abCov(X, Y)

(NTU Econ) Basic Quantitative Method 2023.7.21. 27 / 51

 Covariance and Correlation

Correlation Coefficient

Definition (Correlation Coefficient)

The correlation coefficient is defined by

Cov(X, Y)
ρ XY = Corr(X, Y) = √ √
Var(X) Var(Y)

A unit-free measure of comovement.

(NTU Econ) Basic Quantitative Method 2023.7.21. 28 / 51

 Covariance and Correlation

Correlation Coefficient

Theorem
The correlation coefficient lies between 1 and -1:

−1 ≤ ρ XY ≤ 1

Proof: by Cauchy-Schwarz inequality,

[E(UV )]2 ≤ E(U 2 )E(V 2 )

(NTU Econ) Basic Quantitative Method 2023.7.21. 29 / 51

 Covariance and Correlation

Correlation Coefficient

Note:
ρ XY = 1 (perfect correlation)
ρ XY = −1 (perfect negative correlation)
ρ XY = 0 (zero correlation, no correlation, uncorrelated)
However, no correlation does not mean that there is no
relationship between X and Y
It just suggests that there is no linear relationship between X and
Y

(NTU Econ) Basic Quantitative Method 2023.7.21. 30 / 51

 Independent Bivariate Random Variables

Section 4

Independent Bivariate Random Variables

(NTU Econ) Basic Quantitative Method 2023.7.21. 31 / 51

 Independent Bivariate Random Variables

Expectation of Functions of Independent Bivariate Random Variables

Theorem
Let X and Y are independent variables. Then

E[g(X)h(Y)] = E[g(X)]E[h(Y)]

(NTU Econ) Basic Quantitative Method 2023.7.21. 32 / 51

 Independent Bivariate Random Variables

Independent Bivariate Random Variables and MGF

Theorem
X and Y are independent random variables. Their MGFs are M X (t)
and MY (t), respectively. Let Z = X + Y, then

M Z (t) = M X (t)MY (t)

Proof. By definition and the previous theorem.

(NTU Econ) Basic Quantitative Method 2023.7.21. 33 / 51

 Independent Bivariate Random Variables

Independent Bivariate Random Variables

Theorem
Given that X and Y are independent:
E(XY) = E(X)E(Y)

Cov(X, Y) = 0

Var(X + Y) = Var(X) + Var(Y)

(NTU Econ) Basic Quantitative Method 2023.7.21. 34 / 51

 Independent Bivariate Random Variables

Independent vs. Uncorrelated

X, Y independent implies X, Y uncorrelated, however, the reverse

is not true.
Independence require all possible realizations x and y to satisfy

P(X = x, Y = y) = P(X = x)P(Y = y).

To check X, Y uncorrelated, only one equation needs to hold:

∑ ∑(x − E(X))(y − E(Y))P(X = x, Y = y) = 0.

x y

(NTU Econ) Basic Quantitative Method 2023.7.21. 35 / 51

 Independent Bivariate Random Variables

Example

Consider random variable X has the following distribution:

x P(X = x)
-1 1/3
0 1/3
1 1/3
Now let Y = X 2
It can be shown that Cov(X, Y) = 0 but clearly they are not
independent.

(NTU Econ) Basic Quantitative Method 2023.7.21. 36 / 51

 Conditional Expectation and Variance

Section 5

Conditional Expectation and Variance

(NTU Econ) Basic Quantitative Method 2023.7.21. 37 / 51

 Conditional Expectation and Variance

Conditional Expectation

Definition
The conditional expectation of Y given X = x is

E(Y∣X = x) = ∑ y fY∣X=x (y)

y

E(Y∣X = x) = ∫ y fY∣X=x (y)d y

y

Hence, E(Y∣X = x) = g(x)

Since E(Y∣X = x) is a function of x, it follows that

E(Y∣X) = g(X)

(NTU Econ) Basic Quantitative Method 2023.7.21. 38 / 51

 Conditional Expectation and Variance

Conditional Variance

Definition
The conditional variance of Y given X = x is

Var(Y∣X = x) = E([Y − E(Y∣X = x)]2 ∣X = x)

Hence, Var(Y∣X = x) = h(x)

Since Var(Y∣X = x) is also a function of x, it follows that

Var(Y∣X) = h(X)

It can be shown that

Var(Y∣X = x) = E(Y 2 ∣X = x) − [E(Y∣X = x)]2

(NTU Econ) Basic Quantitative Method 2023.7.21. 39 / 51

 Conditional Expectation and Variance

Important Theorems

Theorem
Useful Rule
E[h(X)Y∣X] = h(X)E[Y∣X]
Simple Law of Iterated Expectation

E(E[Y∣X]) = E(Y)

E(E[XY∣X]) = E(XY)

Application:

Var(Y∣X) = E(Y 2 ∣X) − [E(Y∣X)]2

(NTU Econ) Basic Quantitative Method 2023.7.21. 40 / 51

 Conditional Expectation and Variance

Important Theorems

Theorem (Best Conditional Predictor)

Conditional expectation E(Y∣X) is the best conditional predictor of Y
in the sense of minimizing the conditional mean squared error:

E(Y∣X) = arg min E[(Y − g(X))2 ]

g(X)

(NTU Econ) Basic Quantitative Method 2023.7.21. 41 / 51

 Conditional Expectation and Variance

Example: GPA vs. Study Hours

Let Y = GPA, X = Study Hours

We would like to know E(Y∣X) (to forecast Y)
We further assume that E(Y∣X) is a linear function:

E(Y∣X) = α + βX

It can be shown that

E(XY) − E(X)E(Y) Cov(X, Y)
β= =
E(X 2 ) − E(X)2 Var(X)

α = E(Y) − βE(X)

(NTU Econ) Basic Quantitative Method 2023.7.21. 42 / 51

 Conditional Expectation and Variance

Example: GPA vs. Study Hours

We can define the forecast error as

є ≡ Y − E(Y∣X) = Y − (α + βX)

Hence,
Y = α + βX + є

Interpretation: your GPA is determined by

(a) Systematic Part: α + βX, which can be explained by study hours
(b) Irregular Part: є, which captures other factors other than study
hours. For instance, good/bad luck, mood, illness, etc.

(NTU Econ) Basic Quantitative Method 2023.7.21. 43 / 51

 Multivariate Random Variables

Section 6

Multivariate Random Variables

(NTU Econ) Basic Quantitative Method 2023.7.21. 44 / 51

 Multivariate Random Variables

Expected Values of Functions of Random Variables

For discrete random variables, the expected values of

g(X1 , X2 , . . . , X n ) is given by

E[g(X1 , X2 , . . . , X n )]
= ∑ ⋯ ∑ g(x1 , x2 , . . . , x n ) f X (x1 , x2 , . . . , x n )
x1 xn

For continuous random variables,

E[g(X1 , X2 , . . . , X n )]
= ∫ ⋯ ∫ g(x1 , . . . , x n ) f X (x1 , . . . , x n )dx n ⋯dx1
x1 xn

(NTU Econ) Basic Quantitative Method 2023.7.21. 45 / 51

 Multivariate Random Variables

Properties

n n
E (∑ X i ) = ∑ E(X i )
i=1 i=1
n n n i−1
Var (∑ X i ) = ∑ Var(X i ) + 2 ∑ ∑ Cov(X i , X j )
i=1 i=1 i=1 j=1

(NTU Econ) Basic Quantitative Method 2023.7.21. 46 / 51

 Multivariate Random Variables

Expectation of Functions of Independent Random Variables

Theorem
Let X1 , X2 , . . . , X n are independent variables. Then

E[h(X1 )h(X2 )⋯h(X n )] = E[h(X1 )]E[h(X2 )]⋯E[h(X n )]

(NTU Econ) Basic Quantitative Method 2023.7.21. 47 / 51

 Multivariate Random Variables

Independent Random Variables and MGF

Theorem
X1 , X2 ,. . . , X n are independent with MGF: M X1 (t), M X2 (t),. . .,
M X n (t). Let Y = ∑ni=1 X i , then
n
MY (t) = M X1 (t)M X2 (t)⋯M X n (t) = ∏ M X i (t)
i=1

(NTU Econ) Basic Quantitative Method 2023.7.21. 48 / 51

 Multivariate Random Variables

IID Random Variables

Given that {X i }ni=1 are i.i.d. random variables.

Clearly,
E(X1 ) = E(X2 ) = ⋯ = E(X n )
Var(X1 ) = Var(X2 ) = ⋯ = Var(X n )
Cov(X i , X j ) = 0 for any i ≠ j
I.I.D. random variables with mean µ and variance σ 2 are denoted
by
{X i }ni=1 ∼i.i.d. (µ, σ 2 )

(NTU Econ) Basic Quantitative Method 2023.7.21. 49 / 51

 Multivariate Random Variables

Properties of i.i.d. Random Variables

Theorem
Let {X i }ni=1 are i.i.d. random variables with E(X i ) = µ and
Var(X i ) = σ 2 , and let
n
Y = ∑ Xi .
i=1

Then
E(Y) = nµ,
Var(Y) = nσ 2 .

(NTU Econ) Basic Quantitative Method 2023.7.21. 50 / 51

 Multivariate Random Variables

Example

Let
{X i }ni=1 ∼i.i.d. Bernoulli(p)

That is,
E(X i ) = p and Var(X i ) = p(1 − p)

Let n
Y = ∑ Xi
i=1

What is the distribution of Y?

Find E(Y) and Var(Y)

(NTU Econ) Basic Quantitative Method 2023.7.21. 51 / 51