Lecture 3

Objectives
• By the end of this topic students will
– be familiar with the classical assumptions
– know the statistical properties of the estimators
under the classical assumptions
• References
– Gujarati p65 – 76 (Classical Assumptions)
– Gujarati, p107 – 112 (Normality Assumption)
 The Model
– True Model (Population Regression Function)
Yi    X i  ui
• Yi is the ith observation of the dependent variable
• Xi is the ith observation of the independent or explanatory
variable
• ui is the ith observation of the error or disturbance term
  and  are unknown parameters (coefficients)
• i = 1,..., n observations
 The Model
– Estimated model (Sample Regression Function)
Yi  ˆ  ˆX i  uˆi
– where
– ˆ and ˆ are the estimators of the coefficients
(parameters)
– ûi are the i estimated residuals
• Objective: estimate the PRF on basis of
SRF
 The Classical Assumptions
– Assumption 1: Yi  ˆ  ˆX i  uˆi
• The model is linear in the parameters
– Assumption 2:
• The X values are fixed in repeated sampling. X is
nonstochastic
– Assumption 3: E  ui | X i   0
• Zero mean of the disturbance ui
• Given the value of X, the mean, or expected value
of the disturbance term, ui , is zero.
Y Conditional Distribution of the Disturbances

. Mean E(ui|Xi) Yi=α+βXi+ui

-ui

+ui

X
X1 X2 X3
 The Classical Assumptions
– Assumption 4:
• Homoscedasticity or equal variance of ui
• Given the value of X, the variance of, ui, the
disturbance term is the same for all observations.

var  ui | X i   E ui  E  ui | X i  
2

 E (u | X i ) uses assumption 3
2
i

 2
 The Classical Assumptions
– Assumption 5:
• No autocorrelation between the disturbances
• Given any two X values, Xi and Xj, (i  j ) , the
correlation between any two ui and uj, (i  j,)is zero.

cov  ui u j   E ui  E  ui | X i   u j  E  u j | X j  

 E  ui | X i   u j | X j  uses Assumption 3
0
 The Classical Assumptions
– Assumption 6
• Zero covariance between Xi and ui or E(Xi ui )=0

cov  ui X i   E ui  E  ui | X i    X i  E  X i  
 E  ui   X i  E  X i   uses Assumption 3
 E  ui X i   E  ui  E  X i  since E  X i  is nonstochastic
 E  ui X i  since E  ui   0
 0 by assumption
 The Classical Assumptions
– Assumption 7:
• The number of observations (n) must be greater than the
number of parameters to be estimated (k)
– Assumption 8:
• Variability in X values
• Technically var(X) must be a finite positive number
– Assumption 9:
• The regression model is correctly specified. There is no
specification error or bias in the model used for
empirical analysis
 The Classical Assumptions
– Assumption 10:
• There is no perfect multicollinearity. There are no perfect
linear relationships among the explanatory variables
– Please remember the following shorthand:
• Model: Yi=α+βXi+ui
• zero mean: E(ui) = 0
• Var (ui) = 2<
• uncorrelated: E(uiuj) = 0, i j
• Cov(Xiui) = 0
 Statistical Properties of OLS
• Normality Assumption: ui ~ N(0,2)
– (1.) It can be proven that if the errors are
classical then OLS estimators are the Best
Linear Unbiased Estimators (BLUE).
– Linear means linear in the dependent variable.
N

– (X i  X )(Yi  Y )
May be rewritten as
ˆ  i 1
N

 i
( X
i 1
 X ) 2
 Statistical Properties of OLS
N

(X i  X )(Yi ) N
̂  i 1
N
 wY i i

 i
( X
i 1
 X ) 2 i 1

(Xi  X )
– where wi  N

 i
( X
i 1
 X ) 2

• proof p62 n1,n2 and 3A.2 (p100)

– can similarly be written as a linear function of Yi
̂
 Statistical Properties of OLS
• (2.) Unbiased
– The expected value of the estimator is the true
underlying parameter

E ( ˆ )  
• (3.) Minimum Variance

ˆ ~
– Of all the linear,Var (  estimators,
unbiased
OLS
)  Var (,OLS
) has the
~
smallest variance 
 Statistical Properties of OLS
• The variance of the OLS estimator:
 2
Var ( ˆ ) 
(Xi  X ) 2

Var (ˆ ) 
 2
X i
2


n an
– Note:it is necessary to obtain i  X )of the variance
( Xestimate 2

of the coefficients in order to calculate confidence

intervals and to carry out statistical inference
 Statistical Properties of OLS
• Estimates of variance of error term
2
ˆ
ui
ˆ  s  
2 2

i ( n  2)

• (4.) Consistency
Pr{| ˆ   |  }  1 as n goes to infinity for small 
• This implies
Var ( ˆ )  0 and E( ˆ )   as n goes to infinity
 Statistical Properties of OLS
• (5.) Normality
 
 
ˆ   2

 ~ N , n
 
 (Xi  X ) 
2

 i 1 
 n

   Xi 
2 2

ˆ ~ N  , n i 1 
 2 
 n ( X i  X ) 
 i 1 
 The Model
• Properties of the OLS Estimator under the
classical assumptions and the normality
assumption
– B.L.U.E.
– Unbiased
– Minimum Variance
– Consistent
– Normally distributed