Advanced Econometrics

Masters Class

Chrispin Mphuka

UNZA

January 2010

CM (Institute) Econometrics Lecture 3 01/2010 1 / 17

Assumptions of the CLRM (Recap)

Linearity: yi = xi 1 β1 + xi 2 β2 + ...xik βk + εi (i = 1, 2, 3, ..., n)

Full rank: The n K data matrix X has full column rank
Exogeneigty of regressors: E (εi jX1 , ..., Xn ) = 0 (i = 1, 2, ..., n)
Homoscedasticity and nonautocorrelation: Each disturbance εi has
the same …nite variance σ2 and is uncorrelated with every other
disturbance εj .
Exogenously generated data
Normal distribution of disturbances

CM (Institute) Econometrics Lecture 3 01/2010 2 / 17

Linearity

1
Recall :OLS estimator is b = (X0 X) X 0y
1
Let C = (X0 X) X0 =) b = Cy

CM (Institute) Econometrics Lecture 3 01/2010 3 / 17

Linearity

1
Recall :OLS estimator is b = (X0 X) X 0y
1
Let C = (X0 X) X0 =) b = Cy
This shows that the OLS estimators are a linear combination of the
regressand.

CM (Institute) Econometrics Lecture 3 01/2010 3 / 17

Unbiasedness

1 1 1
b = X0 X X0 y = X0 X X0 (Xβ + ε) = β+ X0 X X0 ε = β + Cε
(1)
Now taking expectations of the OLS estimator conditioning of the X
matrix:

h i
1
E (bjX) = β + E X0 X X0 ε jX (2)
1
= β + X0 X X0 E [ ε jX ] (3)

By Assumption 1, the last term is 0

Hence:
E (bjX) = β (4)
This shows that under the CLRM assumptions the OLS estimators are
unbiased
CM (Institute) Econometrics Lecture 3 01/2010 4 / 17
The Variance of the OLS estimator

Var (bjX) = E (b β ) (b β ) 0 jX

1 1
= E X0 X X0 εε0 X X0 X jX
1 1
= X0 X X0 E εε0 jX X X0 X
1 1
= X0 X X0 σ 2 I X X0 X
1
= σ 2 X0 X

CM (Institute) Econometrics Lecture 3 01/2010 5 / 17

Gauss Markov Thoerem

In the classical linear regression model with regressor matrix X, the

least squares estimator b is the linear unbiased estimator of β. for any
vector W, the minimum variance estimator of W0 β, in the classical
regression model is W0 b where b is the least squares estimator.

CM (Institute) Econometrics Lecture 3 01/2010 6 / 17

Proving Minimum Variance

Let b0 = Ay be another linear unbiased estimator, where A is a

K n matrix. if b0 is unbiased then :

E [AyjX] = E [(AXβ + Aε) jX] = β (5)

CM (Institute) Econometrics Lecture 3 01/2010 7 / 17

Proving Minimum Variance

Let b0 = Ay be another linear unbiased estimator, where A is a

K n matrix. if b0 is unbiased then :

E [AyjX] = E [(AXβ + Aε) jX] = β (5)

This implies that AX = I

CM (Institute) Econometrics Lecture 3 01/2010 7 / 17

Proving Minimum Variance

Let b0 = Ay be another linear unbiased estimator, where A is a

K n matrix. if b0 is unbiased then :

E [AyjX] = E [(AXβ + Aε) jX] = β (5)

This implies that AX = I

The variance-covariance matrix of C is

Var [b0 jX] = σ2 AA0

CM (Institute) Econometrics Lecture 3 01/2010 7 / 17

Proving Minimum Variance

Let b0 = Ay be another linear unbiased estimator, where A is a

K n matrix. if b0 is unbiased then :

E [AyjX] = E [(AXβ + Aε) jX] = β (5)

This implies that AX = I

The variance-covariance matrix of C is

Var [b0 jX] = σ2 AA0

1
Now let D = A (X0 X ) X0 () Dy = b0 b

CM (Institute) Econometrics Lecture 3 01/2010 7 / 17

Proving Minimum Variance

Let b0 = Ay be another linear unbiased estimator, where A is a

K n matrix. if b0 is unbiased then :

E [AyjX] = E [(AXβ + Aε) jX] = β (5)

This implies that AX = I

The variance-covariance matrix of C is

Var [b0 jX] = σ2 AA0

1
Now let D = A (X0 X ) X0 () Dy = b0 b
Then

1 1 0
Var [b0 jX] = σ2 D + X0 X X0 D + X0 X X0

CM (Institute) Econometrics Lecture 3 01/2010 7 / 17

Proving Minimum Variance

1
But AX = I = DX + (X0 X) X0 X. ) DX = 0.
Therefore:

Var [b0 jX] = σ2 X0 X + σ2 DD0 = Var [bjX] + σ2 DD0

The conditional variance covariance matrix of b0 equals that of b plus
a nonnegative de…nite matrix.
Thus
Var [bjX] 6 Var [b0 jX]

CM (Institute) Econometrics Lecture 3 01/2010 8 / 17

Estimator of the variance
The Least squares residual is: e = My = M (Xβ + ε) = Mε, Since
MX = 0 by construction
The estimate of the variance os disturbances is based on residual
sum of squares.

e0 e = ε0 Mε

E e0 ejX = E ε0 MεjX
since ε0 Mε is a scalar matrix so it is equal to its trace. Thus
E [tr (ε0 Mε) jX] = E tr Mεε0 jX
Since X is assumed nonstochastic we have ;

E tr Mεε0 jX = tr ME εε0 jX = tr Mσ 2 I
= σ2 tr (M)

CM (Institute) Econometrics Lecture 3 01/2010 9 / 17

Estimator of the variance

The
h trace of M is : i
1 1
tr In X (X0 X) X0 = tr (In ) tr X (X0 X) X0 =
1
n tr (X0 X) X0 X = n tr (IK ) = n K
Thus E [e0 ejX] = σ2 (n K)
Therefore the variance estimator is

e0 e
s2 =
n K

CM (Institute) Econometrics Lecture 3 01/2010 10 / 17

Statistical Inference

Since b is a linear function of ε which is mutivarite normal it follows

that b will also be multivariate normal.
Therefore:

bjX~N β, σ2 (X0 X) 1

So each element of b jX is also distributed as;

bk jX~N βk , σ2 (X0 X)kk1

CM (Institute) Econometrics Lecture 3 01/2010 11 / 17

Hypothesis Testing

Let skk be the kth diagonal element of (X0 X) 1 then

bk βk
zk = p ~N (0, 1)
σ2 s kk

If σ2 was known then we can use the standard normal distribution for
statistical inference.
But normally this is not the case so: we need a new statistic . We
start with

(n k) s2 e0 e ε 0 ε
= = M
σ2 σ2 σ σ
This is a quatratic form of a standard normal vector σε . Therefore it
has a chisquared distribution with degrees of freedom tr (M) = n K

CM (Institute) Econometrics Lecture 3 01/2010 12 / 17

Hypothesis Testing

CM (Institute) Econometrics Lecture 3 01/2010 13 / 17

Hypothesis Testing

Lets de…ne
b β 1 0 ε
= X0 X X
σ σ
Theorem: A linear function Lx and a symmetric idempotent form
x0 Ax in a standard normal vector are statistically independent if
LA = 0 (Proof see Theorem B12 )
let σε be x then the requirement of the theorem is that:
(X0 X) 1 X0 M = 0 which holds since MX = 0
Theorem: If ε is normally disributed, then the least squares coe¢ cient
estimator b is statistically independent of the residual vector e and
therefore, all functions of e .

CM (Institute) Econometrics Lecture 3 01/2010 13 / 17

Hypothesis Testing

Recall: a standard normal divided by an independent chisquare which

is divided by its repective degrees of freedom if a t distribution.
Therefore
bk βk
p
σ2 skk bk βk
tk = = p
(n k )s 2 s 2 skk
σ2
/ (n K)

This is now distributed as a t-distribution with n K degrees of

freedom

CM (Institute) Econometrics Lecture 3 01/2010 14 / 17

Con…dence Intervals

Pr ob (bk tα/2 sbk βk bk + tα/2 sbk ) = 1 α

Look at Example 4.4 Page 52 Green

CM (Institute) Econometrics Lecture 3 01/2010 15 / 17

Data Problems

Multicollinearity
- e¤ects of high correlation among variables include: small changes in
the data produces wide swings in the parameter estimates;
coe¢ cients may have very high standard errors and low sigini…cance
levels, coe¢ cients may have wrong signs or implausible maginitudes
- Detection include using the variance in‡ation factor, 1 1R 2 .Values in
k
excess of 20 are assumed indicate serious multicollinearity.
-Correction for collinearity includes droping the problem data, using
additional data, using principal components

Look at Example 4.4 Page 52 Green

CM (Institute) Econometrics Lecture 3 01/2010 16 / 17

Data Problems

Multicollinearity
- e¤ects of high correlation among variables include: small changes in
the data produces wide swings in the parameter estimates;
coe¢ cients may have very high standard errors and low sigini…cance
levels, coe¢ cients may have wrong signs or implausible maginitudes
- Detection include using the variance in‡ation factor, 1 1R 2 .Values in
k
excess of 20 are assumed indicate serious multicollinearity.
-Correction for collinearity includes droping the problem data, using
additional data, using principal components
Missing Observations
-if missing at random then you can ignore the case
- if missing in a systematic manner then some people have resorted to
imputing the missing data using available information.

Look at Example 4.4 Page 52 Green

CM (Institute) Econometrics Lecture 3 01/2010 16 / 17

Data Problems

In‡uential Data Points

-we identify which residuals are too large to identify outliers
e = 2 ei 1/2
-we use b
[s (1 p ii )]
- if the value is above 2.0 then it suggests problem value.
Solution: In cross section data we can drop the data but this is not
advisable in time series data

CM (Institute) Econometrics Lecture 3 01/2010 17 / 17