Advanced Econometrics

Masters Class

Chrispin Mphuka

UNZA

June 2010

CM (Institute) Econometrics Lecture 4 06/2013 1 / 32

Assumptions of the CLRM (Recap)

Linearity: yi = xi 1 β1 + xi 2 β2 + ...xik βk + εi (i = 1, 2, 3, ..., n)

Full rank: The n K data matrix X has full column rank
Exogeneigty of regressors: E (εi jX1 , ..., Xn ) = 0 (i = 1, 2, ..., n)
Homoscedasticity and nonautocorrelation: Each disturbance εi has
the same …nite variance σ2 and is uncorrelated with every other
disturbance εj .
Exogenously generated data
Normal distribution of disturbances

CM (Institute) Econometrics Lecture 4 06/2013 2 / 32

Consistency
1
Recall :OLS estimator is b = X0 X X0 y
DEFINITION: Convergence in Probability- The random variable xn
converges in probability to a constant c if
limn !∞ Pr ob (jxn c j > ε) = 0 for any positive c.

CM (Institute) Econometrics Lecture 4 06/2013 3 / 32

Consistency
1
Recall :OLS estimator is b = X0 X X0 y
DEFINITION: Convergence in Probability- The random variable xn
converges in probability to a constant c if
limn !∞ Pr ob (jxn c j > ε) = 0 for any positive c.
De…nitin: Consitent Estimator- An estimatorθbn of a parameter θ is a
consistent estimator of θ if and only if p lim b
θ = θ.

CM (Institute) Econometrics Lecture 4 06/2013 3 / 32

Consistency
1
Recall :OLS estimator is b = X0 X X0 y
DEFINITION: Convergence in Probability- The random variable xn
converges in probability to a constant c if
limn !∞ Pr ob (jxn c j > ε) = 0 for any positive c.
De…nitin: Consitent Estimator- An estimatorθbn of a parameter θ is a
consistent estimator of θ if and only if p lim b
θ = θ.
We can now rewrite tyhe ols estimators as

1
b = β + X0 X X0 ε (1)
1
X0 X X0 ε
= β+
n n

CM (Institute) Econometrics Lecture 4 06/2013 3 / 32

Consistency

1
X0 X X0 ε
p lim b = p lim β+p lim p lim (2)
n n
1
X0 X X0 ε
= β+p lim p lim (3)
n n

X0 X
Assume p lim n = Q a positive de…nite matrix

CM (Institute) Econometrics Lecture 4 06/2013 4 / 32

hence

1 X0 ε
p lim b = β+Q p lim (4)
n
1
= β +Q 0 (5)
0
Xε
= β since p lim =0 (6)
n

CM (Institute) Econometrics Lecture 4 06/2013 5 / 32

Consistency

X0 ε
since p lim n = 0 as follows;

X 0ε 1 n 1 n
= ∑ xi εi = ∑ wi = w (7)
n n i =1 n i =1
from the exogeneighty assussumption we have:
E (wi ) = EX [E [wi jxi ]] = EX [xi E [εi jxi ]] = 0 and

var [w ] = E [var [w jX ]] + var [E [w jX ]] (8)

= E [var [w jX ]] since E [εi jxi ] = 0

σ2 X 0X
but var [w jX] = E [w w 0 jX] = n1 X0 E [εε0 jX] X n1 = n n
σ2 X0 X
) var [w ] = n E n =) lim var [w] = 0.Q = 0

CM (Institute) Econometrics Lecture 4 06/2013 6 / 32

Asymptotic normality of the OLS estimator

Recall:

b = β+ X0 X X0 ε (9)
1
=) b β = X0 X X0 ε
1
p X0 X X0 ε
=) n (b β) = p
n n
p
If the limiting distribution of the random vector n (b β) exists
then that limiting distribution is the same as that of:
" 1
#
X0 X X0 ε X0 ε
p lim p =Q 1 p (10)
n n n

CM (Institute) Econometrics Lecture 4 06/2013 7 / 32

Asymptotic normality of the OLS estimator
We need to establish the limiting distribution of

1 p
p X0 ε = n (w E (w )) (11)
n

CM (Institute) Econometrics Lecture 4 06/2013 8 / 32

Asymptotic normality of the OLS estimator
We need to establish the limiting distribution of

1 p
p X0 ε = n (w E (w )) (11)
n
We use the Lindberg-Feller CLT (D.19.A) page 913 Green 5th edition.
using the formulation, w is the average of n independent random
vectors wi = xi εi with means 0 and variances p
var [xi εi ] = σ2 E [xi xi0 ] = σ2 Qi . the variance of nw is :

1
σ2 Q n = σ2 [Q1 + Q2 + ... + Qn ] (12)
n

CM (Institute) Econometrics Lecture 4 06/2013 8 / 32

Asymptotic normality of the OLS estimator
We need to establish the limiting distribution of

1 p
p X0 ε = n (w E (w )) (11)
n
We use the Lindberg-Feller CLT (D.19.A) page 913 Green 5th edition.
using the formulation, w is the average of n independent random
vectors wi = xi εi with means 0 and variances p
var [xi εi ] = σ2 E [xi xi0 ] = σ2 Qi . the variance of nw is :

1
σ2 Q n = σ2 [Q1 + Q2 + ... + Qn ] (12)
n
As long as the regressors are well behaved lim σ2 Q n = σ2 Q
n !∞

CM (Institute) Econometrics Lecture 4 06/2013 8 / 32

Asymptotic normality of the OLS estimator
We need to establish the limiting distribution of

1 p
p X0 ε = n (w E (w )) (11)
n
We use the Lindberg-Feller CLT (D.19.A) page 913 Green 5th edition.
using the formulation, w is the average of n independent random
vectors wi = xi εi with means 0 and variances p
var [xi εi ] = σ2 E [xi xi0 ] = σ2 Qi . the variance of nw is :

1
σ2 Q n = σ2 [Q1 + Q2 + ... + Qn ] (12)
n
As long as the regressors are well behaved lim σ2 Q n = σ2 Q
n !∞
If [xi εi ] , i = 1, 2, ..., n are independent vectors distributed with mean
X0 ε
0 and variance σ2 Qi < ∞, and p lim n = 0, then

1 d
p X0 ε ! N 0, σ2 Q (13)
CM (Institute) nEconometrics Lecture 4 06/2013 8 / 32
Asymptotic normality of the OLS estimator

Therefore:
p d
n (b β) ! N 0, σ2 Q 1
(14)
Theorem: Asymptotic Distribution of b with independent
observations- If fεi g are independently distributed with mean zero
and …nite variance σ2 and the xik is well behaved, then
a σ2 1
b~N β, Q (15)
n
1 1
In practice we estimate n1 Q with (X0 X) and σ2 with
e0 e/ (n K )

CM (Institute) Econometrics Lecture 4 06/2013 9 / 32

and the estimator of the Asy. Var[b]
s2
1 0
Expanding: s 2 = n K ε Mε

1 h i
1
s2 = ε0 ε ε 0 X X0 X X0 ε
n K " #
1
n ε0 ε ε0 X X0 X X0 ε
=
n K n n n n

hence
( " 1
#)
n ε0 ε ε0 X X0 X X0 ε
p lim s 2 = p lim (16)
n K n n n n
ε0 ε
= p lim
n
= σ2
CM (Institute) Econometrics Lecture 4 06/2013 10 / 32
and the estimator of the Asy. Var[b]

s2
By product rule:
1
X0 X
p lim s 2 = σ2 Q 1
(17)
n

The appropriate estimator of the asymptotic covariance matrix of b is

1
Est.Asy .Var [b] = s 2 X0 X (18)

CM (Institute) Econometrics Lecture 4 06/2013 11 / 32

Asymptotic Distribution of a function of b: The Delta
Method

Letf (b) be a set of J continous linear or nonlinear and continously

di¤erentiable functions of the least squares estimator, and let

∂f (b)
C (b ) = (19)
∂b0
By the Slutsky Thoerem

p lim f (b ) = f ( β) (20)

∂f (b)
and p lim C (b ) = =Γ (21)
∂b0
Using the linear Taylor expansion:

f (b ) = f ( β ) = Γ (b β) + higher order terms (22)

CM (Institute) Econometrics Lecture 4 06/2013 12 / 32
Asymptotic Distribution of a function of b: The Delta
Method

Theorem: Asymptotic Distribution of a Function of b

- If f (b) is a set of continous and continously
di¤erentiable functions of b such that Γ = ∂f (b) /∂b0
then:
a σ2
f (b) ~N f ( β) , Γ Q 1
Γ0 (23)
n
In practice the estimator of the asymptotic
h covariancei
matrix would be: Est.Asy. Var[f(b)] = C s 2 X0 X 1 C0

CM (Institute) Econometrics Lecture 4 06/2013 13 / 32

Instrumenal variable and two stage least squares

Motivation - Woodrige Ch4 and CH5 -

Consider:
y = β0 + β1 x1 +... + βK xK +u (24)

E (u ) = 0, Cov (xj , u ) = 0, j = 1, 2, ..., K 1 (25)

Thus xK is potentially endogenous i.e it is

correlated with the error term Cov (xK , u ) 6= 0

CM (Institute) Econometrics Lecture 4 06/2013 14 / 32

Instrumenal variable and two stage least squares

To put thing clearer, we assume that u = γq + ε .

ε has 0 mean and is uncorrelated with x1 , x2 , ..., xK
and q . We also assume that E (q ) = 0
=) E [u ] = γE [q ] + E [ε] = 0.
regressing q on the regressors we have:
q = δ0 +δ1 x1 +... + δK xK +r (26)

CM (Institute) Econometrics Lecture 4 06/2013 15 / 32

Instrumenal variable and two stage least squares

E (r ) = 0, Cov (xj , r ) = 0, j = 1, 2, ..., K .

Now replacing 26 into 24 and simplifying we have

y = ( β0 + γδ0 ) + ( β1 + γδ1 ) x1 +...+ ( βK + γδK ) xK +u (27)

We can assume that δ0 , δ1 , ...δK 1 are all equal to zero

So
cov [xK , q ]
p lim b
βK = βK + γ (28)
var [xK ]
This shows that if the regressors are endogenous the
coe¢ cients are inconsistent

CM (Institute) Econometrics Lecture 4 06/2013 16 / 32

IV Estimation... I

Assumptions
-we have another n L matrix of instruments, Z . where L=K
- E [xi ,εi ] = γ 6= 0, hence p lim (1/n) X0 ε = γ
-[xi , zi ,εi ], i = 1, 2, ...n, are an i.i.d. sequence of random variables
-E xik2 = Qxx,kk < ∞, a …nite constant, k=1, 2,...,K
-E z2il = Qzz,ll < ∞, a …nite constant, l=1, 2,...,L
-E [zil xik ] = Qzx,lk < ∞, a …nite constant, l=1, 2,...,L, k=1, 2,...,K
-E [εi jzi ] = 0
P lim (1/n) Z0 Z = Qzz , a …nite positive de…nite (assumed) matrix
p lim (1/n) Z0 X = QZX , a …nite, L K matrix with rank K
0
p lim (1/n) Z ε = 0

CM (Institute) Econometrics Lecture 4 06/2013 17 / 32

IV Estimation... II
1
Recall: b = β + (X 0 X ) 1
X 0 ε=β + X 0X
n
X 0ε
n

1
X0 X X0 ε
Now, p lim b = β + p lim p lim (29)
n n
= β + Qxx1 γ 6= 0

CM (Institute) Econometrics Lecture 4 06/2013 18 / 32

IV Estimation... III

Hence ols is now inconsistent

Z 0y Z 0X Z 0ε
but consider: p lim = p lim β + p lim (30)
n n n
Z 0X
= p lim β (31)
n

CM (Institute) Econometrics Lecture 4 06/2013 19 / 32

IV Estimation... IV
Since Z has the same number of columns as X,
we can have:
1
Z 0X Z 0y
p lim p lim =β (32)
n n

CM (Institute) Econometrics Lecture 4 06/2013 20 / 32

IV Estimation... IV
Since Z has the same number of columns as X,
we can have:
1
Z 0X Z 0y
p lim p lim =β (32)
n n

This leads us to the Instrumental Variable

Estimator
1
bIV = Z0 X Z 0y (33)

CM (Institute) Econometrics Lecture 4 06/2013 20 / 32

IV Estimation... V

To get the asymptotic distribution of the IV

estimation we have:
1
p Z 0X 1
n (bIV β) = p Z 0ε (34)
n n
1 d
but, p Z 0ε ! N 0, σ2 Qzz (35)
n

CM (Institute) Econometrics Lecture 4 06/2013 21 / 32

 Which implies that:
1
Z 0X 1 d σ2
p Z 0ε ! N 0, Q 1 Qzz Qxz1 (36)
n n n zx

a h 2
i
) bIV ~N β, σn Qzx1 Qzz Qxz1

CM (Institute) Econometrics Lecture 4 06/2013 22 / 32

IV Estimation... VI

Estimated Residuals:
h i
1 1
bε= y Xb IV = y X Z 0X Z 0y = I X Z 0X Z0 ε (37)

Thus the estimat of the variace of disturbances

is:

bε0bε
b2 =
σ (38)
n
1 h 1
i 0 h
1
i
= I X Z 0X Z0 ε I X Z 0X Z0 ε
n

CM (Institute) Econometrics Lecture 4 06/2013 23 / 32

IV Estimation... VII

1 1
ε0 ε ε0 Z X 0Z X 0X Z 0X Z 0ε
b2 =
=) σ + (39)
n n n n n n
1
ε0 X Z 0X Z 0ε
2 (40)
n n n

ε0 ε
b2 = p lim
hence : p lim σ = σ2 (41)
n

CM (Institute) Econometrics Lecture 4 06/2013 24 / 32

IV Estimation... VIII

Deriving Asymptotic variance of bIV :

1
1 Z 0X Z 0ε
bIV = Z 0 X Z 0 y = β+ (42)
n n
1
p Z 0X 1
n (bIV β) = p Z 0ε (43)
n n

CM (Institute) Econometrics Lecture 4 06/2013 25 / 32

 1 d
but, p Z 0ε ! N 0, σ2 Qzz (44)
n

Which implies that:

1
Z 0X 1 d σ2
p Z 0ε ! N 0, Q 1 Qzz Qxz1 (45)
n n n zx

a h 2
i
) bIV ~N β, σn Qzx1 Qzz Qxz1

CM (Institute) Econometrics Lecture 4 06/2013 26 / 32

IV Estimation... XI
Understanding IV further We still use the
Woodrige motivation for IV.
Now suppose we have an observable variable
z1 which is uncorrelated with u : i.e., Cov [z1 , u ] = 0
but z1 is at the same time partially correlated
with xK
Recall: y = β0 + β1 x1 + ... + βK xK + u, E (u ) = 0,
Cov (xj , u ) = 0, j = 1, 2, ..., K 1 and Cov (xK , u ) 6= 0
The requirement that z1 and xK are uncorrelated
means that θ 6= 0 in the ¤ regression: i.e
Cov [z1 , xK ] 6= 0

xK = δ0 +δ1 x1 +... + δK 1 xK 1 + θ 1 z1 +r K (46)

The two conditions mean that z1 quali…es as an
instrument for xK
CM (Institute) Econometrics Lecture 4 06/2013 27 / 32
IV Estimation... X

Pluging equation 46 into our main equation:

We have:

y = β0 + β1 x1 + ... + (47)
βK (δ0 + δ1 x1 + ... + δK 1 xK 1 + θ 1 z1 + rK ) + u (48)
= α0 + α1 x1 + ... + αK 1 xK 1 + λ1 z1 + ν

where ν = u + βK rK , αj = βj + βK δj , λ1 = βK θ 1

CM (Institute) Econometrics Lecture 4 06/2013 28 / 32

IV Estimation... XI

Example: Woodrige Page 87: Consider

log (wage ) = β0 + β1 exp er + β2 exp er 2 + β3 educ + u (49)

Education is likely correlated with u because

ability is omitted in our model.
What instrument can we choose?

CM (Institute) Econometrics Lecture 4 06/2013 29 / 32

Two Stage Least Squares Estimation

If Z has more variables than X , then there is

need for a two-stage least squares.
The problem is Z 0 X will be L K with rank K < L
hence it will be singular
Solution: Project columns of X on Z . i.e:
b = Z Z0 Z 1
X Z0 X = PZ X (50)

CM (Institute) Econometrics Lecture 4 06/2013 30 / 32

 bIV = b 0X X
X b 0y (51)
h i 1
1 1
= X0 Z Z0 Z Z0 X X0 Z Z0 Z Z0 y
1
= X0 PZ X X0 PZ y
1
= (PZ X )0 (PZ X ) (PZ X )0 y (52)
= b 0X
X b X
b 0y (53)

CM (Institute) Econometrics Lecture 4 06/2013 31 / 32

Two Stage Least Squares Estimation

The Estimator of the variance is:

2 (y X bIV ) (y X bIV )
sIV = (54)
n
Caution : do not base the calculation of the
disturbances on Xb

CM (Institute) Econometrics Lecture 4 06/2013 32 / 32