See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/319629465

Econometrics Lecture Notes

Research · September 2017

DOI: 10.13140/RG.2.2.15188.09608

CITATIONS READS

0 10,134

1 author:

Vijayamohanan Pillai N

138 PUBLICATIONS   496 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Electricity Economics View project

Art and Literature View project

All content following this page was uploaded by Vijayamohanan Pillai N on 11 September 2017.

The user has requested enhancement of the downloaded file.

 Monday, March 03, 2014

Joint Distribution of X and Y

Econometrics:
f ( x , y ) = P [( X = x ) ∩ (Y = y )]
Basic statistics: 2D RVs
Discrete :
0 ≤ f ( x, y) ≤ 1
∑∑x y
f ( x, y) = 1

Continuous :
f ( x, y) ≥ 0
∞ ∞

∫∫
−∞−∞
f ( x , y ) dydx = 1
Monday, March 03, CDS MPhil Econometrics 1 Monday, March 03, CDS MPhil Econometrics 2
2014 Vijayamohan 2014 Vijayamohan

CDF of a Joint Distribution Marginal Distributions

F ( x , y ) = P [( X ≤ x ) ∩ (Y ≤ y )]
f X ( x ) = ∑ f ( x, y )
y

Continuous
fY ( y ) = ∑ f ( x , y )
x y

F ( x, y) = ∫∫
− ∞− ∞
f ( s , t )dtds x

2
d
[ F ( x , y )] = f ( x , y ) ∞
dxdy f X ( x) = ∫ f ( x, y )dy
−∞

Discrete ∞

x y f y ( y) = ∫ f ( x , y )dx
F ( x, y) = ∑∑
−∞ −∞
f ( s, t ) −∞

Monday, March 03, CDS MPhil Econometrics 3 Monday, March 03, CDS MPhil Econometrics 4
2014 Vijayamohan 2014 Vijayamohan

Joint cumulative distribution function

Joint Distribution of X and Y

0 ≤ f ( x, y) ≤ 1 F ( x , y ) = P [( X ≤ x ) ∩ (Y ≤ y )]
X f(y)
Y X
1 2 3 4 f(y)
∑∑
x y
f ( x, y) = 1 Y
1 2 3 4
F ( x, y)
1 0.12 0.08 0.07 0.05 0.32 x y

1 0.12 0.08 0.07 0.05 0.32 = ∑∑ f ( s, t )

2 0.08 0.15 0.21 0.13 0.57 −∞ −∞
f X ( x ) = ∑ f ( x, y )
2 0.08 0.15 0.21 0.13 0.57
3 0.01 0.01 0.02 0.07 0.11 y

3 0.01 0.01 0.02 0.07 0.11

f(x) 0.21 0.24 0.3 0.25 1
fY ( y ) = ∑ f ( x , y ) f(x) 0.21 0.24 0.3 0.25 1
x

F(2, 2) = f(1,1) + f(1,2) + f(2,1) + f(2,2) =

0.12 + 0.08 + 0.08 + 0.15 = 0.43
Monday, March 03, CDS MPhil Econometrics 5 Monday, March 03, CDS MPhil Econometrics 6
2014 Vijayamohan 2014 Vijayamohan

1
 Monday, March 03, 2014

Expectation of a function of Jointly Conditional Distribution of Y given X = x

Distributed RVs
Note :
Discrete P[ A ∩ B ]
P[ A | B] =
Discrete E [u ] = ∑ ∑ uf ( x , y ) P[B]
∑ xf ( x )
x y
E[ X ] =
E [ h ( x , y )] = ∑ ∑ h( x , y ) f ( x , y ) P [( X = 1) ∩ (Y = 1)] f (1,1)
x y
P [ X = 1 | Y = 1] = =
P [Y = 1]
E [ h ( x )] = ∑ h( x ) f ( x ) E [ X 2 + 5Y ] = ∑ x
∑ ( x 2 + 5 y) f ( x, y)
y
f Y (1)

f ( x, y) X f(y)
Continuous f X |Y ( x | Y = y ) = Y
Continuous fY ( y ) 1 2 3 4
E [ h ( x )] = ∫ h( x ) f ( x )dx E [ h ( x , y )] =
∞ ∞

∫ ∫ h( x , y ) f ( x , y )dxdy
1 0.12
2 0.08
0.08
0.15
0.07
0.21
0.05 0.32
0.13 0.57
−∞−∞ f X |Y ( x | Y = y ) f Y ( y ) = f ( x , y )
3 0.01 0.01 0.02 0.07 0.11
f(x) 0.21 0.24 0.3 0.25 1

Monday, March 03, CDS MPhil Econometrics 7 Monday, March 03, CDS MPhil Econometrics 8
2014 Vijayamohan 2014 Vijayamohan

Independent RVs: X and Y Conditional Expectation

Find the conditional PDF from:
f X |Y ( x | Y = y ) = f x ( x )
f ( x, y) f ( x, y)
f X |Y ( x | Y = y ) = fY | X ( y | X = x ) =
fY | X ( y | X = x) = f y ( y) fY ( y ) fX (x)

f ( x , y ) = f X ( x ) f Y ( y ) ⇔ Independent Apply the expectation formula as usual E [ X ] = ∫ X . f ( x )dx

F ( x , y ) = FX ( x )FY ( y ) ⇔ Independen t E[ X | Y = y ] = ∫ X . f X |Y ( x | Y = y )dx
X f(y)
Y
1 2 3 4 E [Y | X = x ] = ∫ Y . fY | X ( y | X = x )dy
1 0.12 0.08 0.07 0.05 0.32
X f(y)
2 0.08 0.15 0.21 0.13 0.57 Y
1 2 3 4
3 0.01 0.01 0.02 0.07 0.11 1 0.12 0.08 0.07 0.05 0.32
f(x) 0.21 0.24 0.3 0.25 1 2 0.08 0.15 0.21 0.13 0.57
Monday, March 03, CDS MPhil Econometrics 9 Monday, March 03, CDS MPhil Econometrics 3 0.01 0.01 0.02 0.07
10 0.11
2014 Vijayamohan 2014 Vijayamohan f(x) 0.21 0.24 0.3 0.25 1

Conditional Variance Properties of Expectation

Find the conditional PDF from previous formula

Apply the expectation formula as usual E(a) = 0, a : constant

Find conditional mean: E[X | Y = y)

Find conditional second moment: E[X 2 | Y = y]

E(aX + b) = aE(X) + b, a, b : constant
Use the variance formula as usual, using these components

Var( X ) = E[ X 2 ] − ( E[ X ])2
E(XY) = E(X)E(Y), X, Y independent RVs
Var( X | Y = y ) = E[ X 2 | Y = y] − ( E[ X | Y = y])2

E[ X 2 |Y = y] = ∫X f X |Y ( x | Y = y ) dx
2

E[ X | Y = y] = ∫ X . f X |Y ( x | Y = y ) dx

Monday, March 03, CDS MPhil Econometrics 11 Monday, March 03, CDS MPhil Econometrics 12
2014 Vijayamohan 2014 Vijayamohan

2
 Monday, March 03, 2014

Properties of Variance Covariance between X and Y

Var(X) = E[X – E(X)]2 = E(X2) – [E(X)]2 • Covariance = 0 for independent X, Y

• Positive for large X with large Y
Var(a) = 0, a : constant • Negative for large X with small Y (vice versa)
Var(aX + b) = a2 Var(X), a, b : constant • Formula is similar to our familiar variance formula

Cov[ X , Y ] = E{[ X − E ( X )][Y − E (Y )]}

If X, Y independent RVs,
= E ( XY ) − E ( X ) E (Y )
Var(X ∓ Y) = Var(X) + Var(Y);
Var(aX ∓ bY) = a2 Var(X) + b2 Var(Y)

Monday, March 03, CDS MPhil Econometrics 13 Monday, March 03, CDS MPhil Econometrics 14
2014 Vijayamohan 2014 Vijayamohan

Covariance between X and Y Properties of Variance and Covariance

Cov(X, Y) = Cov(Y,X)
Cov(X, X) = Var(X)
Cov(Y, Y) = Var(Y)

If X and Y are NOT independent:

Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y)
Var(X – Y) = Var(X) + Var(Y) – 2Cov(X,Y)
Var(aX ∓ bY ∓ c) = a2Var(X) + b2Var(Y) ∓
2abCov(X,Y)
Monday, March 03, CDS MPhil Econometrics 15 Monday, March 03, CDS MPhil Econometrics 16
2014 Vijayamohan 2014 Vijayamohan

Properties of Covariance Properties of Covariance

If X and Y are independent: Assume that (X, Y ) is equally likely to take

1. Cov(X,Y) = E(XY) – E(X)E(Y) = 0 the four values {(– 1, 0); (0, – 1); (1, 0); (0, 1)}.
{(– )}.

→ Var(X ∓ Y) = Var(X) + Var(Y)

Is E(XY ) = 0 = E(X)E(Y )?

2. Cov(a + bX, c + dY) = bd Cov(X,Y)

Are these random variables independent or not?
Independent random variables have a
covariance of zero
What if the covariance is zero?
Monday, March 03, CDS MPhil Econometrics 17 Monday, March 03, CDS MPhil Econometrics 18
2014 Vijayamohan 2014 Vijayamohan

3
 Monday, March 03, 2014

Correlation What if random variable X and Y have linear

Cov( X , Y )
relationship, that is,
Corr( X , Y ) =
Var( X )Var(Y ) Y = aX + b
where a≠0
Cov[ X , Y ] = E ( XY ) − E ( X ) E (Y )

Cov ( X , Y )
ρ ( X , Y ) = ρ XY = Cov( X , Y ) aVar ( X )
σ XσY Corr ( X , Y ) = =
Var ( X )Var (Y ) Var ( X ) a 2Var ( X )
Cov ( X , Y ) = ρ XY σ X σ Y

Values between -1 and 1, and What if a > 0?

independent random variables have a if a < 0?
correlation of zero
Monday, March 03, CDS MPhil Econometrics 19 Monday, March 03, CDS MPhil Econometrics 20
2014 Vijayamohan 2014 Vijayamohan

Monday, March 03, 2014 CDS MPhil Econometrics 21

Vijayamohan

4
 3/3/2014

CDS M Phil Econometrics

Old Least Squares (OLS)

Vijayamohanan Pillai N
3 March 2014 Vijayamohan: CDS MPhil: 1 3 March 2014 Vijayamohan: CDS MPhil: 2
CDS M Phil Econometrics Vijayamohan
Econometrics Econometrics
CDS M Phil Econometrics Vijayamohan

Simple Linear Regression Model Types of Relationships

Linear relationships Curvilinear relationships

• Only one independent variable, X
Y Y

• Relationship between X and Y is

described by a linear function
X X
• Changes in Y are assumed to be
Y Y
caused by changes in X

X X
3 March 2014 Vijayamohan: CDS MPhil: 3 3 March 2014 Vijayamohan: CDS MPhil: 4
Econometrics Econometrics

Types of Relationships Types of Relationships

Strong relationships Weak relationships No relationship

Y
Y Y

X
X X
Y
Y Y

X
X X
3 March 2014 Vijayamohan: CDS MPhil: 5 3 March 2014 Vijayamohan: CDS MPhil: 6
Econometrics Econometrics

1
 3/3/2014

Simple Linear Regression Model

Simple Linear Regression Model
Parameters of the conditional mean function:
Y
Consists of and
E(Y
E(Y | X = x)
the conditional mean function = + x
Slope:
E(Y
E(Y | X = x) = + x and β the rate of change in
E(Y | X = x)
the conditional variance (scedastic)
scedastic) 1
for a unit change in X
function Intercept:
2> value of E(Y | X = x) when x = 0
Var(
Var(Y | X = x) = 0, constant
X

3 March 2014 Vijayamohan: CDS MPhil: 7 3 March 2014 Vijayamohan: CDS MPhil: 8
Econometrics Econometrics

Simple Linear Regression Model Simple Linear Regression Model

Yi = E(Y |X = Xi)+ui
As the scedastic function 2> 0, ui : random variables E(Y|X) = + X
the observed value of the i th response y4 {
.
u4
Yi ≠ E(Y
E(Y | X = Xi) ui : the
vertical
y3 .}u3 distance

This difference → statistical error ui

y2 u2{. between
the point yi
and the
Yi = E(Y |X = Xi)+ui mean
ui =Yi – E(Y |X = Xi) y1 .} u1 function

x1 x2 x3 x4
3 March 2014 Vijayamohan: CDS MPhil: 9 3 March 2014 Vijayamohan: CDS MPhil: 10
Econometrics Econometrics

Simple Linear Regression Model Simple Linear Regression Model

The parameters of the conditional mean

function and : unknown
unknown;;

must be estimated using

observed data on Yi and Xi

Estimates of parameters :
computable functions of data and are
therefore statistics
statistics..

3 March 2014
3 March 2014 Vijayamohan: CDS MPhil:
Vijayamohan: CDS MPhil: 11 11 3 March 2014 Vijayamohan: CDS MPhil: 12
Econometrics
Econometrics Econometrics

2
 3/3/2014

Simple Linear Regression Model The Classical / Old / Ordinary

Least squares (OLS)
To estimate the unknown and Intuitively, OLS is fitting a line through the
sample points such that the sum of squared
using observed data on Yi and Xi residuals is as small as possible

Ordinary Least Squares,

Squares, or OLS
OLS:: hence the term least squares

Parameter estimates are chosen to minimize The residual, û , is an estimate of the error
term, u
the residual sum of squares.
and is the difference between the fitted line
(sample regression function) and the sample
point
3 March 2014
3 March 2014 Vijayamohan: CDS MPhil:
Vijayamohan: CDS MPhil: 13 13 3 March 2014 Vijayamohan: CDS MPhil: 14
Econometrics
Econometrics Econometrics

Algebraic Properties of OLS Gauss-Markov Theorem

• The sum of the OLS residuals is zero • Of the class of linear unbiased estimators,
the OLS estimators have the smallest
variance
• Thus, the sample average of the OLS
residuals is zero as well • Least squares estimators are BLUE
• The sample covariance between the • Best
regressors and the OLS residuals is zero • Linear
• Unbiased
• Estimators
• The OLS regression line always goes
s. t. certain assumptions
through the mean of the sample
3 March 2014 Vijayamohan: CDS MPhil: 15 3 March 2014 Vijayamohan: CDS MPhil: 16
Econometrics Econometrics

OLS Assumptions OLS Estimators

y4 .
{

y3 .}
y2 {
.

y1 .}
x1 x2 x3 x4
3 March 2014 Vijayamohan: CDS MPhil: 17 3 March 2014 Vijayamohan: CDS MPhil: 18
Econometrics Econometrics

3
 3/3/2014

OLS Estimators OLS Estimators

∑ (x i − x )(yi − y ) n

∑ (x − x) > 0
2
ˆ
β= i=1
; i
n

∑ (xi − x )
2 i=1

i=1

3 March 2014 Vijayamohan: CDS MPhil: 19 3 March 2014 Vijayamohan: CDS MPhil: 20
Econometrics Econometrics

OLS Estimators Mean and Variance of OLS Estimators

yi = xi + ui

∴
E(ui) = 0, ∀i

3 March 2014 Vijayamohan: CDS MPhil: 21 3 March 2014 Vijayamohan: CDS MPhil: 22
Econometrics Econometrics

Simple Linear Regression Example Sample Data for House Price Model
House Price in Square Feet
• A real estate agent wishes to examine the Rs1000s (Y) (X)
relationship between the selling price of a 245 1400
312 1600
home and its size (measured in square feet)
279 1700
308 1875
• A random sample of 10 houses is selected 199 1100
– Dependent variable (Y) = house price in 219 1550
405 2350
Rs1000s
324 2450
– Independent variable (X) = square feet 319 1425
255 1700
3 March 2014 Vijayamohan: CDS MPhil: 23 3 March 2014 Vijayamohan: CDS MPhil: 24
Econometrics Econometrics

4
 3/3/2014

400
Graphical Presentation Summary Statistics
House Price in Rs 1000s
350
300
250
200

1000 1500 2000 2500

Square Feet

House price model: scatter plot (Stata)

twoway (scatter houseprice squarefeet)
3 March 2014 Vijayamohan: CDS MPhil: 25 3 March 2014 Vijayamohan: CDS MPhil: 26
Econometrics Econometrics

Stata Output Prediction using Regression Analysis

Predict the price for a house

with 2000 square feet:

house price = 98.25 + 0.1098 (sq.ft.)

= 98.25 + 0.1098(2000)
The regression equation is:
= 317.85
The predicted price for a house with
house price = 98.24833 + 0.10977
The image cannot be display ed. Your computer may not hav e enough memory to open the image, or the image may hav e been corrupted. Restart y our computer, and then open the file again. If the red x still appears, y ou may hav e to delete the image and then insert it again.

(square feet)
2000 square feet is 317.85(Rs1,000s)
= Rs317,850
3 March 2014 Vijayamohan: CDS MPhil: 27 3 March 2014 Vijayamohan: CDS MPhil: 28
Econometrics Econometrics

Interpolation vs. Extrapolation Measures of Variation

When using a regression model for prediction, Total variation is made up of two parts:
predict only within the relevant range of data
Relevant range TSS = RSS + ESS
for interpolation

450 Total Sum of Regression Sum Error Sum of

400 Squares of Squares Squares
House Price (Rs1000s)

350
300
250
200
150 Do not try to
TSS = ∑ (Yi − Y )2 RSS = ∑ (Yˆi − Y )2 ESS = ∑ (Yi ˆi )2
−Y
100
50 extrapolate
0
0 1000 2000 3000
beyond the range
of observed X’s
= +
Square Feet

3 March 2014 Vijayamohan: CDS MPhil: 29 3 March 2014 Vijayamohan: CDS MPhil: 30
Econometrics Econometrics

5
 3/3/2014

Measures of Variation Measures of Variation

TSS = total sum of squares Y

Yi ∧ ∧
Measures the variation of the Yi values ESS = ∑(Yi - Yi )2 Y
around their mean Y _
RSS = regression sum of squares TSS = ∑(Yi - Y)2
∧
Explained variation attributable to the Yi ∧ _
relationship between X and Y _ RSS = ∑(Yi - Y)2 _
ESS = error sum of squares
Y Y

Variation attributable to factors other

than the relationship between X and Y
Xi X
3 March 2014 Vijayamohan: CDS MPhil: 31 3 March 2014 Vijayamohan: CDS MPhil: 32
Econometrics Econometrics

Coefficient of Determination, R2 Stata Output

Portion of the total variation in the

dependent variable
explained by
variation in the independent variable

RSS regression sum of squares

R2 = =
TSS total sum of squares
RSS 18934.9348
R2 = = = 0.58082
TSS 32600.5

58.08% of the variation in house prices

is explained by variation in square feet
0 ≤ R2 ≤ 1
3 March 2014 Vijayamohan: CDS MPhil: 33 3 March 2014 Vijayamohan: CDS MPhil: 34
Econometrics Econometrics

Examples of Approximate R2 Values Examples of Approximate R2 Values

R2 = 1 Y
Y
0 < R2 < 1
Perfect linear relationship
Weaker linear
between X and Y:
relationships between
X X and Y:
X
100% of the variation in Y Y
Y Some but not all of the
is explained by variation in Y is
explained by
variation in X
variation in X
X
X

3 March 2014 Vijayamohan: CDS MPhil: 35 3 March 2014 Vijayamohan: CDS MPhil: 36
Econometrics Econometrics

6
 3/3/2014

Examples of Approximate R2 Values Standard Error of Estimate (Root

(Root MSE)

R2 = 0 The standard deviation of the

Y
variation of observations
No linear relationship
between X and Y: around the regression line

The value of Y does n

X not depend on X.
∑ (Y i − Yˆi )2
ESS
(None of the variation
σ̂ u = = i =1
n −2 n −2
in Y is explained by
variation in X)

3 March 2014 Vijayamohan: CDS MPhil: 37 3 March 2014 Vijayamohan: CDS MPhil: 38
Econometrics Econometrics

Stata Output Comparing Standard Errors

Y Y

X X
13665 .5652
= 1708 .19565 = σu = 41.33032
ˆ
8

3 March 2014 Vijayamohan: CDS MPhil: 39 3 March 2014 Vijayamohan: CDS MPhil: 40
Econometrics Econometrics

Inference About the Slope Inference about the Slope:

Slope: t-
t-test
t test for a population slope:

Is there a linear relationship between X and Y?

Null and alternative hypotheses

MSE σˆ u
SE ( βˆ ) = =
Var ( X ) Var ( X )
H0: β = 0 (no linear relationship)
n
2
H1: β ≠ 0 (linear relationship does exist)
ESS ∑
uˆi
σ̂ u2 = = i =1
n −2 n −2
βˆ − β
The image cannot be display ed. Your computer may
not hav e enough memory to open the image, or the
image may hav e been corrupted. Restart y our
computer, and then open the file again. If the red x

Test statistic t=
still appears, y ou may hav e to delete the image and
then insert it again.

SE( βˆ ) d.f. = n − 2
3 March 2014 Vijayamohan: CDS MPhil: 41 3 March 2014 Vijayamohan: CDS MPhil: 42
Econometrics Econometrics

7
 3/3/2014

Stata Output Inferences about the Slope:

Slope: t-
t-test
Does square footage of the house
Test Statistic: t = 3.329 P-value = 0.01039 < α
affect its sales price?
H0: β1 = 0 From Stata output:
H 1: β 1 ≠ 0 Coefficients Standard Error t Stat P-value
Intercept 98.24833 58.03348 1.69296 0.12892
Square Feet 0.10977 0.03297 3.32938 0.01039

d.f. = 10-2 = 8
Decision: ? Reject H0
/2=.025 /2=.025
Do not reject H0
Conclusion: ?
There is sufficient
Reject H0 -tα/2 0 tα/2 Reject H0 evidence that square
-2.3060 2.3060 3.329 footage affects house
price
3 March 2014 Vijayamohan: CDS MPhil: 43 3 March 2014 Vijayamohan: CDS MPhil: 44
Econometrics Econometrics

F-test for Significance Stata Output

The image cannot be display ed. Your computer may not

RMS
hav e enough memory to open the image, or the image

F-test statistic:
may hav e been corrupted. Restart y our computer, and

F=
then open the file again. If the red x still appears, y ou
may hav e to delete the image and then insert it again.

EMS The image cannot be display ed. Your computer may not hav e enough memory to open the

RSS RMS 18934.9348

F= = = 11.0848
image, or the image may hav e been corrupted. Restart y our computer, and then open the file
again. If the red x still appears, y ou may hav e to delete the image and then insert it again.

RMS = EMS 1708.1957

k With 1 and 8
ESS degrees of freedom P-value for
EMS = the F Test
n − k −1

F follows an F-distribution with k numerator and

(n – k – 1) denominator degrees of freedom

(k = the number of independent variables in the

regression model)

3 March 2014 Vijayamohan: CDS MPhil: 45 3 March 2014 Vijayamohan: CDS MPhil: 46
Econometrics Econometrics

F-test for Significance t-test for Correlation Coefficient

H0: β1 = 0 Test Statistic: Hypotheses

H1: β1 ≠ 0 RMS
F= = 11.08
EMS
α = .05 H0: ρ = 0 (no correlation between X and Y)
df1= 1 df2 = 8
Decision: HA : ρ ≠ 0 (correlation exists)
Critical Reject H0 at α = 0.05
Value:
Test statistic
Fα = 5.32
Conclusion: r -ρ (with n – 2 df)
α = .05 t= where
There is sufficient evidence
1 − r2
that house size affects
0 F r = + r 2 if βˆ > 0
Do not
reject H0
Reject H0 selling price n −2
F.05 = 5.32 r = − r 2 if βˆ < 0
3 March 2014 Vijayamohan: CDS MPhil: 47 3 March 2014 Vijayamohan: CDS MPhil: 48
Econometrics Econometrics

8
 3/3/2014

Example: House Prices Example: House Prices – Test Solution

Is there evidence of a linear relationship
between square feet and house price r −ρ .762 − 0 Decision:
t= = = 3.329
at 0.05 level of significance? 1 − r2 1 − .7622 Reject H0
n −2 10 − 2
Conclusion:
H0: ρ = 0 (No correlation)
There is
H1: ρ ≠ 0 (correlation exists) d.f. = 10-2 = 8 evidence of a
α =.05 , df = 10 - 2 = 8 α/2 = 0.025 α/2 = 0.025 linear
association at
r −ρ .762 − 0 5% level of
t= = = 3.329 significance
1 − r2 1 − .7622 Reject H0
-tα/2
Do not reject H0 Reject H0
tα/2
0
n −2 10 − 2 -2.3060 2.3060
3.329
3 March 2014 Vijayamohan: CDS MPhil: 49 3 March 2014 Vijayamohan: CDS MPhil: 50
Econometrics Econometrics

Multiple Regression
Analysis

• y = β0 + β1x1 + β2x2 + . . . βkxk + u

3 March 2014 Vijayamohan: CDS MPhil: Econometrics 51

9
 3/3/2014

CDS M Phil Econometrics

Old Least Squares (OLS)

Vijayamohanan Pillai N
3 March 2014 Vijayamohan: CDS MPhil: 1 3 March 2014 Vijayamohan: CDS MPhil: 2
CDS M Phil Econometrics Vijayamohan
Econometrics Econometrics
CDS M Phil Econometrics Vijayamohan

Multiple Regression Analysis

Multiple Regression General form of the multiple linear regression model:

Analysis Define column vectors of n observations on y

and K – 1 variables.

• y = β0 + β1x1 + β2x2 + . . . βkxk + u y i = f ( x i 2 , x i 3 ,..., x ik ) + u i

y i = β1 + β 2 x i 2 + β 3 x i 3 + ... + β k x ik + ui

This can be expressed as

k
y i = ∑ β j x ij + ui
j =1

3 March 2014 Vijayamohan: CDS MPhil: Econometrics 3 3 March 2014 Vijayamohan: CDS MPhil: 4
Econometrics

Multiple Regression Analysis Multiple Regression Analysis

yi = β1 + β2 xi2 + β3xi3 + ... + βk xik + ui  y1   x11 x12 ... x1k  β1  u1 
y  x x22 ... x2k  β2  u2 
i = 1, …, n y =  2  =  21 +
 ...   ... ... ... ...   ...   ... 
y1 = β1x11 + β2 x12 + β3 x13 + ... + βk x1k + u1       
yn   xn1 xn2 ... xnk  βk  un 

y2 = β1x21 + β2 x22 + β3x23 + ... + βk x2k + u2

= Xβ + u
y3 = β1x31 + β2 x32 + β3x33 + ... + βk x3k + u3 x1 is a column of ones:
……………………….
[x11 L xn1 ] = [1 L 1]
T T

yn = β1xn1 + β2 xn2 + β3 xn3 + ... + βk xnk + un K – 1 variables and 1 constant

3 March 2014 Vijayamohan: CDS MPhil: 5 3 March 2014 Vijayamohan: CDS MPhil: 6
Econometrics Econometrics

1
 3/3/2014

The Classical Assumptions

The Classical Assumptions
A1. Linearity
A2. E(u) = 0
Linearity of the parameters (and disturbance)
The disturbance term has a zero expectation
Sometimes models which appear to be non-
non-linear
u1  0 
can be estimated using the least-
least-squares procedure.
u   
0
E(u) = E  2  =  
n  M  M
y = exp β1 ∏ xkβ k
exp u    
un  0 
k =2 n
ln(y) = β1 + ∑ βk ln(xk ) +u
k =2
3 March 2014 3 March 2014 Vijayamohan: CDS MPhil: 8
Vijayamohan: CDS MPhil: Econometrics 7
Econometrics

The Classical Assumptions The Classical Assumptions

What does ‘Nonstochastic’ Mean?
A3. Nonstochastic Regressors
Assumption A3 comes in two forms:
X is a non-stochastic n x k matrix. a weak version and a strong version.
That is, it consists of a set of fixed numbers. • Strong Version: The explanatory variables
should be non - stochastic.

• Weak Version: The explanatory variables

are random but distributed independently of
the error term.
3 March 2014 Vijayamohan: CDS MPhil: 9 3 March 2014 Vijayamohan: CDS MPhil: 10
Econometrics Econometrics

The Classical Assumptions:

The Classical Assumptions Homoscedasticity
What does ‘Nonstochastic’ Mean? Variance
The term ‘nonstochastic’ essentially means
that the variables are determined “outside”
the context of the regression

(this is another reason why we use the term

independent when referring to explanatory
variables)

3 March 2014 Vijayamohan: CDS MPhil: 11 3 March 2014 Vijayamohan: CDS MPhil: 12
Econometrics Econometrics

2
 3/3/2014

The Classical Assumptions:

The Classical Assumptions Spherical disturbances
Var(u) = E(uuT ) = σ2In
A4. Spherical Disturbances  E(u1 2
) E(u1u2 ) L E(u1un )  σ 2
0 L 0
   
E(u2u1 ) E(u2 2) L E(u2un) =  0 σ2 L 0  = σ2In
T
Var(u) = E(uu ) = σ In 2 E(uuT ) =   M
 M M O M  M O M 
   
E(unu1) E(unu2 ) L E(u2 n)   0 0 L σ2 
 E(u12 ) E(u1u2 ) L E(u1un ) σ2
  0 L 0
  Spherical disturbances ⇒
E(u2u1 ) E(u22) L E(u2un )
E(uu ) = 
T 0 σ2 L 0 
 M M O M  =
   M M O M  (i) Var(ui) ≡ E(u2
i ) = σ
2 ∀ i = 1,..., n homoskedasticity
E(unu1) E(unu2 ) L E(u2 
n) 
 
 0 0 L σ2 
and

(ii) Cov(ui , u j ) ≡ E(ui , u j ) = 0 ∀i≠ j

= σ2In No autocorrelation

3 March 2014 Vijayamohan: CDS MPhil: 13 3 March 2014 Vijayamohan: CDS MPhil: 14
Econometrics Econometrics

The Classical Assumptions: The Classical Assumptions:

Homoscedasticity Homoscedasticity

y
f(y|x) f(y|x)

. E(y|x) = β0 + β1x . E(y|x) = β0 + β1x

. .
Homoscedasticity
.
Heteroscedasticity

x1 x2 x x1 x2 x3 x
3 March 2014 Vijayamohan: CDS MPhil: 15 3 March 2014 Vijayamohan: CDS MPhil: 16
Econometrics Econometrics

The Classical Assumptions The Classical Assumptions

A5. Identifiability A5. Identifiability

No exact linear relationships among the variables No exact linear relationships among the variables
(no perfect multicollinearity). (no perfect multicollinearity).

Specifically, Implicit within this assumption are

the requirements of more observations than
X is n x K with rank K. (X has full (column) rank) variables (micronumerosity)
and sufficient variability in the values of the
⇒ the columns of X are linearly independent. regressors.

Non--singular X
Non
3 March 2014 Vijayamohan: CDS MPhil: 17 3 March 2014 Vijayamohan: CDS MPhil: 18
Econometrics Econometrics

3
 3/3/2014

The Classical / Old / Ordinary

The Classical Assumptions Least squares (OLS)
Sample counterpart of the k-variable regression model:
A6. Normality
y = Xˆ
β + û
Final assumption:
the disturbances are normally distributed.
Where β̂ and û are the sample counterparts of β and u.

u ~ N (0, σ 2 I n )
OLS aims to minimise the difference between
useful for the purposes of statistical inference an observed value of yi and its predicted value.

but not necessary for analysing the properties of ⇒ the error û = y − ŷ be the least.
the estimators. ŷ = Xˆ
β

3 March 2014 Vijayamohan: CDS MPhil: 19 3 March 2014 Vijayamohan: CDS MPhil: 20
Econometrics Econometrics

The Classical / Old / Ordinary The Classical / Old / Ordinary

Least squares (OLS) Least squares (OLS)

Specifically, the problem is to find an A necessary condition for a minimum is that the first-
estimator that minimises the error sum of order conditions equal zero.
squares:
∂û′ û
= 2X′Xˆ
β − 2X′y = 0
û′ û = (y − ŷ)′ (y − ŷ) ∂ˆβ

= y ′y + ˆ
β′X ′Xˆ β′X ′y
β − 2ˆ Therefore, rearrange to give the normal equation

(X′X)ˆ
β = X′y

β = (X ′X)−1 X ′y
ˆ

3 March 2014 Vijayamohan: CDS MPhil: 21 3 March 2014 Vijayamohan: CDS MPhil: 22
Econometrics Econometrics

The Classical / Old / Ordinary The Classical / Old / Ordinary

Least squares (OLS) Least squares (OLS)

XTX is invertible provided X has full rank.

β = (X′X)−1 X ′y
ˆ
(Why?)
For a 2-variable model,
∂2û′û
Cov(X , y) = 2X′X
ˆ
β= ; Var(X) > 0 ∂ˆ β′
β ∂ˆ
Var(X)
XTX is positive definite for a minimum.

3 March 2014 Vijayamohan: CDS MPhil: 23 3 March 2014 Vijayamohan: CDS MPhil: 24
Econometrics Econometrics

4
 3/3/2014

Statistical Properties of Least Squares Estimator Statistical Properties of Least Squares Estimator

Mean of β̂ Variance of β̂ Var(ˆ

β) = E (ˆ [β − β)′
β − β)(ˆ ]
β = (X ′X)−1 X ′y
ˆ
Noting that β − β = (X ′X)−1 X ′u
ˆ

= β + (X ′X)−1 X ′u
then
β) = E[(X ′X) −1 X ′uu′X(X ′X) −1 ]
Var(ˆ
E(ˆ
β) = β Property of unbiasedness β) = σ2 (X ′X)−1
Var(ˆ

[key assumptions: E(u) = 0

and non-stochastic regressors] [key assumptions: Var(ui) ≡ E(u2
i )= σ
2
for all i, i≠j.
and Cov(ui , u j ) ≡ E(ui , u j ) = 0
3 March 2014 Vijayamohan: CDS MPhil: 25 3 March 2014 Vijayamohan: CDS MPhil: 26
Econometrics Econometrics

Statistical Properties of Least Squares Estimator Statistical Properties of Least Squares Estimator

Unbiased Estimator of σ2
Variance of β̂
β) = σ2 (X ′X)−1
Var(ˆ
β) = σ2 (X ′X)−1
Var(ˆ
For statistical inference we require an estimate of

In the scalar case σ2 Variance of β̂

Var(ˆ
β2 ) =
∑ x2i and therefore, σ2. n
∑ û2i
An unbiased estimator of σ2 is: ˆ i =1
Minimum σ2 = s2 =
n−k
Variance (best)
property Its square root = the standard error of the regression

3 March 2014 Vijayamohan: CDS MPhil: 27 3 March 2014 Vijayamohan: CDS MPhil: 28
Econometrics Econometrics

OLS: Goodness of Fit OLS: Measure of Goodness of Fit

What proportion of the total variation of y is RSS
Coefficient of determination: R2 =
accounted for by the variation in X? TSS
or 2 ESS
In terms of sums of squares R =1−
TSS
TSS = RSS + ESS

∑ (yi − y )2 = ∑ (ŷi − y ) • R2 never decreases when a new X variable

2
+ ∑ û2i is added to the model
∑ (y − y ) is the total sum of squares (TSS)
2
i – This can be a disadvantage when comparing
models
∑ (ŷ − y ) is the (explained ) regression sum of squares (RSS)
2
i • What is the net effect of adding a new
variable?
∑ û2
i is the error sum of squares (ESS)
3 March 2014 Vijayamohan: CDS MPhil: 29 3 March 2014 Vijayamohan: CDS MPhil: 30
Econometrics Econometrics

5
 3/3/2014

OLS: Measure of Goodness of Fit OLS: Measure of Goodness of Fit

• What is the net effect of adding a new Adjusted R2

variable? • Shows the proportion of variation in Y
explained by all X variables adjusted for the
– We lose a degree of freedom when a new X number of X variables used
variable is added ESS /(n − k)
R 2adj = 1 −
– Did the new X variable add enough TSS /(n − 1)
explanatory power to offset the loss of one   n − 1 
= 1 − (1 − R 2 ) 
degree of freedom?   n − k 
(where n = sample size, k = number of parameters)

3 March 2014 Vijayamohan: CDS MPhil: 31 3 March 2014 Vijayamohan: CDS MPhil: 32
Econometrics Econometrics

OLS: Measure of Goodness of Fit OLS: Measure of Goodness of Fit

Adjusted R2 The relationship between R2 and R 2

ESS /(n − k)   n − 1 
R 2adj = 1 − = 1 − (1 − R 2 )  R2 = 1 −
n −1
(1 − R 2 ) =
1 −k n −1 2
+ R
TSS /(n − 1)   n − k  n−k n−k n−k

(where n = sample size, k = number of parameters)

Two other measures of fit are the Schwartz criterion
– Penalize excessive use of unimportant (SC) ESS k
independent variables SC = ln + ln n
n n
– Smaller than r2
and the Akaike information criterion (AIC)
– Useful in comparing among models
ESS 2k
AIC = ln +
n n
3 March 2014 Vijayamohan: CDS MPhil: 33 3 March 2014 Vijayamohan: CDS MPhil: 34
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

we know that the expected value is In order to make meaningful inference the
variable must be normally distributed.
E(ˆ
β) = β
One of the assumptions introduced above was:
u ~ N(0, σ2In )
And we know that a suitable β) = σ2(X′X)−1
Var(ˆ
estimate of the variance of This means that u has a multivariate normal
the slope parameter is β) = s2(X′X)−1
Var(ˆ distribution.
n
(If it is not, then n must be large – i.e. becomes
∑ û2i normally distributed by virtue of the central limit
2 2 i =1
σ =s =
ˆ theorem.
n−k
3 March 2014 Vijayamohan: CDS MPhil: Econometrics 35 3 March 2014 Vijayamohan: CDS MPhil: 36
Econometrics

6
 3/3/2014

OLS: Statistical Inference OLS: Statistical Inference

This implies that the sampling error ( β̂ - β) is
related to u as follows: In hypothesis testing we are concerned about
testing whether a particular finding is compatible
β − β = (X ′X)−1 X ′y − β
ˆ with some stated hypothesis or not.
⇒ (X ′X)−1 X ′(Xβ + u) − β
⇒ (X ′X)−1 X ′u A common hypothesis test is formulated as

which implies that the sampling error is also H0 : βk = βk

multivariate normally distributed:
H1 : βk ≠ βk
β − β) ~ N(0, σ2 (X′X)−1)
(ˆ

3 March 2014 Vijayamohan: CDS MPhil: 37 3 March 2014 Vijayamohan: CDS MPhil: 38
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

Using our earlier result, if we were looking at the kth diagonal
element of (XTX)-1 we obtain If we do not know σ2, it is natural to replace it
with the estimate s2.
βk − βk ) ~ N(0, σ2 ((X ′X)−1)kk )
(ˆ

where ((X ′X)−1 )kk is the (k,k) element of (XTX)-1 This generates the t-ratio

If we define the ratio zk by dividing ˆ

β − βk ˆ
βk − βk ˆ
βk − βk
tk = =
s2 ((X T X)−1 )kk SE(ˆβk )
by its standard deviation
ˆ
βk − βk
zk =
σ2 ((X ′X)−1 )kk where SE( β̂k) denotes the standard error of β̂ k.
This is distributed as tcα/2;(n-k).
where zk ~ N(0,1)

3 March 2014 Vijayamohan: CDS MPhil: 39 3 March 2014 Vijayamohan: CDS MPhil: 40
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

An Example: Investment Function An Example: Investment Function

Theory of the behaviour of investors:
Hypothesis:
They care only about real interest rates.

Real I = f(real gdp,

Equal increases in interest rates and rate of
an interest rate (the 90-day T-bill rate),
inflation (change in the log of CPI), inflation would have no independent effect on
real disposable personal income,
investment.
trend)
The test statistic is:
So the Ho : β2 + β3 = 0.
ˆ
β +ˆ β3 − 0
t= 2
SE(ˆβ2 + ˆ
β3 )

3 March 2014 Vijayamohan: CDS MPhil: 41 3 March 2014 Vijayamohan: CDS MPhil: 42
Econometrics Econometrics

7
 3/3/2014

OLS: Statistical Inference OLS: Statistical Inference

An Example: Investment Function Testing Linear Restrictions

As well as testing restrictions on individual
regression coefficients we may wish to test linear
combinations of them. For example

(a) H0 : β2 + β3 = 1
(b) H0 : β3 = β 4
SE(.) = [0.003192 + 0.002342 +2(–3.718 x 10–6 )]1/2
= 0.002866 (c) H0 : β2 = β3 = β4 = ... = βk = 0
95% critical value:
− 0.0086 + 0.00331 t(203 – 5) = 1.96
t= = −1.845
0.002866
3 March 2014 Vijayamohan: CDS MPhil: 43 3 March 2014 Vijayamohan: CDS MPhil: 44
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

Testing Linear Restrictions: The F Test Testing Linear Restrictions: The F Test

The t-distribution can be used to test a single null F-Test of Entire Equation (“Testing the Joint
hypothesis Significance of the Explanatory Variables”)

If we want to conduct a “joint” test then we can no H0: β2 = β3=…= βk= 0 Equivalently: H0: R2 = 0
longer use the t distribution
H1: H0 not true (at least one of the β is nonzero)
For example, suppose we want to test whether all the
explanatory variables in the model are significantly
Cannot say: “if the coefficients are individually
different from zero
insignificant this means they must be jointly
insignificant”
3 March 2014 Vijayamohan: CDS MPhil: 45 3 March 2014 Vijayamohan: CDS MPhil: 46
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

Testing Linear Restrictions: The F Test Testing Linear Restrictions:

Multiple Restriction F Test
F-Test of Entire Equation (“Testing the Joint
This time suppose we wish to test whether
Significance of the Explanatory Variables”)
a subset of regression coefficients are
zero. In this case
H0: β2 = β3=…= βk= 0 Equivalently: H0: R2 = 0
ˆβ 
H1: H0 not true (at least one of the β is nonzero) y = [X 2 X 3 ] 2  + û H0: β3= 0
ˆ
β 3 
RSS R 2 ⇒ X 2ˆ
β 2 + X 3ˆ
β 3 + û H1: H0 not true
F = k −1 or F = k −1
ESS 1 − R2
n−k n−k
3 March 2014 Vijayamohan: CDS MPhil: 47 3 March 2014 Vijayamohan: CDS MPhil: 48
Econometrics Econometrics

8
 3/3/2014

OLS: Statistical Inference OLS: Statistical Inference

Testing Linear Restrictions: Testing Linear Restrictions:
Multiple Restriction F Test Multiple Restriction F Test

y = [X 2
ˆβ 
X 3 ] 2  + û
Thus the unrestricted H0: β3= 0
H0: β3= 0
ˆ
β 3  regression is: H1: H0 not true
⇒ X 2ˆ
β 2 + X 3ˆ
β 3 + û H1: H0 not true ˆβ 
y = [X 2 X 3 ] 2  + û
ˆ
β 3 
To find the change in the fit of a multiple regression y = X 2ˆ
β 2 + X 3ˆ
β 3 + û
when an additional variable x3 is added to a model
that already contains K – 1 variables: And the restricted y = [X ˆ
β 
2 X 3 ] 2  + û *
regression: 0 
J = 1 linear restriction
y = X 2ˆ
β 2 + û *
3 March 2014 Vijayamohan: CDS MPhil: 49 3 March 2014 Vijayamohan: CDS MPhil: 50
Econometrics Econometrics

OLS: Statistical Inference OLS: Statistical Inference

Testing Linear Restrictions: Testing Linear Restrictions:
Multiple Restriction F Test Multiple Restriction F Test
The residuals from the restricted regression: Residuals from the restricted regression: û* = y − X2ˆ
β2
û* = y − X2ˆ
β2 Residuals from the unrestricted regression:
û = y − X2ˆ
β2 − X 3ˆ
β3
whereas the residuals from the unrestricted regression:
The test statistic is
û = y − X2ˆ
β2 − X 3ˆ
β3 (û*Tû* − ûTû) / J
F= J = 1; K= 2
ûTû /(n − k)
Essentially we wish to see whether the reduction in
the ESS is 'large enough' to suggest that X3 is or (R2 − R2
*)/ J
F=
significant. (1 − R 2 ) /(n − k)
3 March 2014 Vijayamohan: CDS MPhil: 51 3 March 2014 Vijayamohan: CDS MPhil: 52
Econometrics Econometrics

OLS: Statistical Inference An Example: Production Function

Testing Linear Restrictions:
• Cobb-Douglas: ln y = α + β1 ln L + β2 ln K + u
Multiple Restriction F Test
• Translog: Unrestricted model: K = 6

1 1
ln y = α + β1 ln L + β2 ln K + β3 (ln L)2 + β4 (ln K)2 + β5 ln L ln K + u
An Example: Production Function 2 2

• Translog function relaxes the CD assumption of a unitary

elasticity of substitution
• CD obtained by the restriction: β3 = β4 = β5 = 0
J=3

3 March 2014 Vijayamohan: CDS MPhil: 53 3 March 2014 Vijayamohan: CDS MPhil: 54
Econometrics Econometrics

9
 3/3/2014

Estimated production Functions An Example: Production Function

CD or Translog?
The F-statistic for the hypothesis of CD model:

Ho: CD model is appropriate

(û*Tû* − ûTû) / J
F=
ûTû /(n − k)
(0.85163 − 0.67993) / 3
F= = 1.768
0.67993 /21

Critical value: F(3,21) = 3.07

Conclusion?

3 March 2014 Vijayamohan: CDS MPhil: 56

55 Econometrics
3 March 2014 Vijayamohan: CDS MPhil: Econometrics

An Example: Production Function An Example: Production Function

CD function: Constant Returns to Scale? CD function: Constant Returns to Scale?
• Hypothesis of constant returns to scale: • Hypothesis of constant returns to scale:
• Equivalent to a restriction that the two Ho : β2 + β3 = 1
coefficients of CD function sum to unity ˆ
β2 + ˆ
β3 − 1
t=
SE(ˆβ2 + ˆ
β3 − 1)
Ho : β2 + β3 = 1 • Can have t-test:

(0.603 + 0.3757 − 1)
•F-test with J = 1 and K = 3 t= = −0.3402
[0.01586 + 0.00728 − 2(0.00961)]1 / 2
(0.603 + 0.3757 − 1)2 = – (F1/2)
F= = 0.1157 = t2
0.01586 + 0.00728 − 2(0.00961)
• Critical value: ‘rule of thumb’
• Critical value: F(1,24) = 4.26 Conclusion? Conclusion?
3 March 2014 Vijayamohan: CDS MPhil: 58
57 Econometrics
3 March 2014 Vijayamohan: CDS MPhil: Econometrics

Test of Structural Break: The Chow test Test of Structural Break: The Chow test
300
250

• To test structural change in time series data

200
Savings
150 100
50

1000 2000 3000 4000 5000

Income

Savings and personal disposable income

(billions of dollars) US, 1970 – 1995.
3 March 2014 Vijayamohan: CDS MPhil: 59 3 March 2014 Vijayamohan: CDS MPhil: 60
Econometrics Econometrics

10
 3/3/2014

Test of Structural Break: The Chow test Test of Structural Break: The Chow test
200

300
Steps
1. Run the two sub-period regressions and the full-
150

250
period regression
Savings

Savings
Full-period: St = a0 + b0Yt n = 26 (n1 + n2)
100

200
Sub-period 1: St = a1 + b1Yt n1 = 12
Sub-period 2: St = a2 + b2Yt n2 = 14
150
50

500 1000 1500 2000 2500 2000 3000 4000 5000 6000
Income Income
2. The pooled regression = restricted regression,
1982 – 1995 obtained under the restrictions that a1 = a2 (= a0)
1970 – 1981
and b1 = b2 (= b0).
300
250

Point of structural break: 1982 ′ û*)

3. Get the restricted ESS from it (ESSR): (û*
200
Savings

(assumed)
150 100

3 March 2014 Vijayamohan: CDS MPhil: 61 3 March 2014 Vijayamohan: CDS MPhil: 62
50

Econometrics 1000 2000 3000

Income
4000 5000 Econometrics

Test of Structural Break: The Chow test Test of Structural Break: The Chow test
Steps Steps
1. Run the two sub-period regressions and the full- Sub-period 1: St = a1 + b1Yt n1 = 12
period regression Sub-period 2: St = a2 + b2Yt n2 = 14
Full-period: St = a0 + b0Yt n = 26 (n1 + n2)
Sub-period 1: St = a1 + b1Yt n1 = 12 4. The sub-period regressions = unrestricted
Sub-period 2: St = a2 + b2Yt n2 = 14 regressions.

4. The sub-period regressions = unrestricted 5. Get the unrestricted ESS from them: (û′û)
regressions. : ESSUR = ESS1 + ESS2.

5. Get the unrestricted ESS from them: (û′û) (û*Tû* − ûTû) / k

: ESSUR = ESS1 + ESS2. F= k = j; n – 2k = (n1 – k) + (n2 – k)
ûTû /(n − 2k)

3 March 2014 Vijayamohan: CDS MPhil: 63 3 March 2014 Vijayamohan: CDS MPhil: 64
Econometrics Econometrics

Test of Structural Break: The Chow test Test of Structural Break: The Chow test
Pooled Regression Result: 1970 – 1995 Sub-period 1: 1970 – 1981

Restricted ESS: ′ û*)

(û* = 23248.3 Unrestricted ESS1 = 1785.033

3 March 2014 Vijayamohan: CDS MPhil: Econometrics 65 3 March 2014 Vijayamohan: CDS MPhil: Econometrics 66

11
 3/3/2014

Test of Structural Break: The Chow test Test of Structural Break: The Chow test
Sub-period 2: 1982 – 1995
(û*Tû* − ûTû) / k k = j; n – 2k = (n1 – k) + (n2 – k)
F=
ûTû /(n − 2k)

(23248.3 − 11790.254) /2
F= = 10.69
11790.254 /22

Critical value: F(2,22) at 1% α = 5.72

Unrestricted ESS2 = 10005.221 H0 = a1 = a2 = a0 and b1 = b2 = b0:

Parameter stability
Unrestricted ESS = ESS1 + ESS2 = (û′û)
1785.033 + 10005.221 = 11790.254 Conclusion?
3 March 2014 Vijayamohan: CDS MPhil: Econometrics 67 3 March 2014 Vijayamohan: CDS MPhil: 68
Econometrics

12
 Monday, March 03, 2014

CDS M Phil Econometrics

Functional Forms
and Growth Rates

Vijayamohanan Pillai N
Monday, March 03, 2014 CDS MPhil Econometrics 1 Monday, March 03, CDS MPhil Econometrics 2
Vijayamohan 2014 Vijayamohan

Beta Coefficients Functional Form

• “standardized coefficient” or “beta • OLS can be used for relationships

coefficient” that are not strictly linear in x and y
by using nonlinear functions of x
• Idea is to replace y and each x variable with and y – will still be linear in the
a standardized version – i.e. subtract mean parameters
and divide by standard deviation

• Coefficient reflects standard deviation of y • the natural log of x, y or both

for a one standard deviation change in x
• quadratic forms of x
• interactions of x variables
Monday, March 03, CDS MPhil Econometrics 3 Monday, March 03, CDS MPhil Econometrics 4
2014 Vijayamohan 3 2014 Vijayamohan

Interpretation of Log Models Interpretation of Log Models

Semi-log models:
• Double-log model:
(1)If the model is ln(y) = b0 + b1x + u
• If the model is ln(y) = b0 + b1ln(x) + u
b1 is approximately the percentage change
in y given a 1 unit change in x
• b1 is the elasticity of y with respect to x
If x = time, b1 = growth rate

Monday, March 03, CDS MPhil Econometrics 5 Monday, March 03, CDS MPhil Econometrics 6
2014 Vijayamohan 2014 Vijayamohan

1
 Monday, March 03, 2014

Interpretation of Log Models Why use log models?

• Semi-log models:
• They give a direct estimate of elasticity
(2) If the model is y = b0 + b1ln(x) + u
• For models with y > 0, the conditional
• b1 is approximately the change in y for a distribution is often heteroskedastic or
100 percent change in x skewed, while ln(y) is much less so
• Used in Engel expenditure models:
• German statistician, Ernst Engel (1821- • The distribution of ln(y) is more narrow,
1896): limiting the effect of outliers
• “Total expenditure on food tends to increase
in AP as total expenditure increases in GP”

Monday, March 03, CDS MPhil Econometrics 7 Monday, March 03, CDS MPhil Econometrics 8
2014 Vijayamohan 2014 Vijayamohan

Some Rules of Thumb Reciprocal model

• What types of variables are often used in 1 β
yi = α + β + ui dy
=− 2
log form? xi dx x

Slope everywhere negative and decreases in

• Dollar amounts that must be positive absolute value as x increases.
• Very large variables, such as population As x → 0, y → ∞, and as x → ∞, y → α.
y y
y
• What types of variables are often used in β<0
level form? β>0 β>0 α>0
α>0 α<0
α
• Variables measured in years x
α
• Variables that are a proportion or percent -α x
x β/α
Monday, March 03, CDS MPhil Econometrics 9 Monday, March 03, CDS MPhil Econometrics 10
2014 Vijayamohan 2014 Vijayamohan

Reciprocal model: Phillips Curve Logarithmic reciprocal model

Percentage rate of change of money wages (y) 1 β
ln yi = α − β + ui α− +u
xi i
and unemployment rate of the UK for 1861-1957
xi yi = e

dy yβ
=
dx x2
AW Phillips (1958) Economica Vol 15: 283-299
• Slope is positive for positive x
of money wages
Rate of change

d2 y  β2 2 β 
The natural rate of = y 4 − 3  eα
dx2 x x 

%

unemployment
0
Unemployment rate % Point of inflexion at x = β/2.
-α
β/2
Monday, March 03, CDS MPhil Econometrics 11 Monday, March 03, CDS MPhil Econometrics 12
2014 Vijayamohan 2014 Vijayamohan

2
 Monday, March 03, 2014

Logarithmic reciprocal model Quadratic Models

Point of inflexion at x = β/2.

To the left of this point, slope ↑ with x; • y = β 0 + β 1x + β 2 x 2 + u

to the right, slope ↓.
y • What’s the slope here?
β
eα α−
xi
+ ui
yi = e
• Not β1 alone
As x → ∞,
y → eα dy
= β1 + 2β2 x
dx
β/2
Monday, March 03, Designed
CDSbyMPhil
Skip Tyler,
Econometrics
Varina High School 13 Monday, March 03, CDS MPhil Econometrics 14
2014 Vijayamohan 2014 Vijayamohan

More on Quadratic Models Interaction Terms

For β1 < 0 and β2 > 0, dy
= β1 + 2β2 x • y= 0 + 1x 1 + 2x 2 + 3x 1x 2 +u
dx
the turning point is at
• What’s the slope wrt x1 ?
x* = β1 (2β2 ) , Y will be a min or max • Not 1 alone
at this level of x
dy
= β1 + β3 x2 ,
which is the same as when dx1

so to summarize the effect of x1 on y

β1 > 0 and β2 < 0
we typically evaluate the above at x2

Monday, March 03, CDS MPhil Econometrics 15 Monday, March 03, CDS MPhil Econometrics 16
2014 Vijayamohan 2014 Vijayamohan

Growth rates Applications of DE

Simple Compound Interest:
Compound year: At – 1.
Amount deposited last year:
Exponential interest: r
Rate of interest:

Kinked exponential year: At = (1+r) At–1, or

Amount obtainable this year:
At — (1+r) At – 1 = 0,
0, a DE(1).

Let us start with an application of

What is its time path?
path?
Difference equations formula: At = A0(1+r)t
The compound interest formula:
What is the nature of the time path?
path?

Monday, March 03, CDS MPhil Econometrics 17 Monday, March 03, CDS MPhil Econometrics 18
2014 Vijayamohan 2014 Vijayamohan

3
 Monday, March 03, 2014

Applications of DE Consider the compound interest DE(1):

At — (1+r) At–1 = 0.
The compound interest formula:
formula
Rewriting,
(1+r)t:
At = A0(1+r
At — At–1 = rAt–1; or
An initial amount A0 grows into a final
∆At = rAt–1.
amount At in t years.
Change in A is proportional to its previous

What is the growth rate?

rate value,

r being the constant of proportionality. Then

r : Average annual compound growth rate!
∆At/At–1 = (At — At–1)/At–1 = r:
At is compound growth function.
the growth rate!
Monday, March 03, CDS MPhil Econometrics 19 Monday, March 03, CDS MPhil Econometrics 20
2014 Vijayamohan 2014 Vijayamohan

The compound growth function Simple (annual) growth rate

(1+r)t
At = A0(1+r
Annual rate of change:
given two consecutive years, t = 1 and 0:
Given At, A0 and t,
Change: ∆A = A1 – A0
How to obtain r, Rate of change: ∆A/A0 = (A1 – A0)/A0
average annual compound growth rate?
A1
= −1
A0
r = (At/A0)(1/t) – 1; or Given t years, average annual growth rate
At
 ln( At / A0 )  −1
r = exp  ( At − A0 ) / A0 A0
 −1 ==
 t  t t
Monday, March 03, CDS MPhil Econometrics 21 Monday, March 03, CDS MPhil Econometrics 22
2014 Vijayamohan 2014 Vijayamohan

Example Example
Year Index Annual Year Index Annual
Average annual growth rates –
For 2001:
growth growth Simple and Compound:
rate r = (110 – 100)/100 = 0.1 or 10%; rate
2000 100 2000 100
No. of years, t = 2004 – 2000 = 4.
For 2002:
(180 − 100 ) / 100 0.8
2001 110 0.1 r = (125 – 110)/110 = 0.136 or 2001 110 0.1
Simple r = = = 0 .2
4 4
13.6%; or 20%
2002 125 0.136 2002 125 0.136
For 2003:
1
r = (150 – 125)/125 = 0.2 or 20%;  180  4
2003 150 0.2 2003 150 0.2 Compound r =   −1
For 2004:
 100 
2004 180 0.2 r = (180 – 150)/150 = 0.2 or 20% 2004 180 0.2 = 1.158 – 1 = 0.158 or 15.8%

Monday, March 03, CDS MPhil Econometrics 23 Monday, March 03, CDS MPhil Econometrics 24
2014 Vijayamohan 2014 Vijayamohan

4
 Monday, March 03, 2014

The compound and average annual growth rate The compound and average annual growth rate

Note the difference:

Simple growth rate: 20%;
Note that the growth rate in both
Compound growth rate: 15.8%.
the cases is based on two points in Which one to take?

time..
time
Economic variables in their growth almost
invariably obey the law of

compound interest, not simple interest.

It does not take into account the
Why?
intermediate values of the series. Take for example, the growth of an index
(inflation, production, etc.)
Monday, March 03, CDS MPhil Econometrics 25 Monday, March 03, CDS MPhil Econometrics 26
2014 Vijayamohan 2014 Vijayamohan

The Law of Compound Interest Discrete time vs. continuous time

Growth of a Price Index

180
Say, inflation is 10% a year.
So far time taken as discrete:
discrete
160

140

Base year (0) index: 100 120

100
growth: At = A0(1+r)t
Compound growth
In d e x

Year 1 index: 110 (10% ↑ over 100)

80

60

40

Year 2 index: 121 (10% ↑ over 110) 20

…………….
0
0 1 2 3 4 5 With continuous time:
time
Index 100 110 121 133.1 146.41 161.051
Year

A compound growth! Exponential growth or Geometric growth:

Growth is compounded because each year’s A(t) = A(0)ert

10% rise is applied to a level that is where e = 2.71828….
increasing every year.
year.
is the base of natural exponential function.
function

Monday, March 03, CDS MPhil Econometrics 27 Monday, March 03, CDS MPhil Econometrics 28
2014 Vijayamohan 2014 Vijayamohan

Continuous time: Exponential Growth exponential growth function

Note dA/dt = rA can be rewritten as
As in the discrete time case
At — At–1 = rAt–1; or dA/dt — rA = 0,
a homogeneous
∆At = rAt–1.
first order
Change in A is proportional to its value,
value
differential equation.
r being the constant of proportionality. Then

the rate of change dA/dt = rA.

Its solution is given by
Rewriting, we get r as the growth rate:
rate
A(t) = A(0)ert,
1 dA the exponential growth function.
=r
A dt
Monday, March 03, CDS MPhil Econometrics 29 Monday, March 03, CDS MPhil Econometrics 30
2014 Vijayamohan 2014 Vijayamohan

5
 Monday, March 03, 2014

The general principle behind exponential growth The classic riddle

The larger a number gets

gets,, the faster it grows.
grows A child is offered two choices for an
increasing weekly allowance:
Any exponentially growing number will
eventually grow larger than any other number the first option begins at Re.1 and
which grows at only a constant rate for the increases by Re. 1 each week,
same amount of time.
time
while the second option begins at 1
paise and doubles each week.
This is demonstrated by a classic riddle:
riddle

Monday, March 03, CDS MPhil Econometrics 31 Monday, March 03, CDS MPhil Econometrics 32
2014 Vijayamohan 2014 Vijayamohan

The classic riddle The classic riddle

Option 1 Option 2 Option 1 Option 2 Option 1 Option 2
Option 1 Option 2
Week : Rs : Ps Week : Rs : Ps Week : Rs : Ps Week : Rs : Ps
0 1 1 0 1 1 11 12 2048
11 12 2048
1 2 2 1 2 2 12 13 4096
12 13 4096
2 3 4 2 3 4 13 14 8192
13 14 8192
3 4 8 3 4 8 14 15 16384
14 15 16384
4 5 16 15 16 32768
4 5 16 15 16 32768
5 6 32 16 17 65536
5 6 32 16 17 65536
6 7 64 17 18 131072
17 18 131072
6 7 64 7 8 128 18 19 262144
18 19 262144
7 8 128 8 9 256 19 20 524288
19 20 524288 the second option eventually
8 9 256 9 10 512
9 10 512 10 11 1024 grows much larger.
10 11 1024 The first option, growing at a constant rate of
Re. 1/week, pays more in the short run.
Monday, March 03, CDS MPhil Econometrics 33 Monday, March 03, CDS MPhil Econometrics 34
2014 Vijayamohan 2014 Vijayamohan

The classic riddle The classic riddle

Mathematically,
Mathematically,

In the second case, the allowance in

In the first case, the payout in week t is week t is 2t paise;
simply t + 1 Rupees
Rupees:: a linear growth.
growth thus, in week 15 the payout is 215 =
32768 Paise = Rs. 327.68.
The payout grows at a constant rate of Re. 1
All formulas of the form kt, where k is an
per week;
unchanging number greater than 1 (e.g.,
in week 15, the payout is 15 + 1 = Rs
Rs.. 16. 2), and t is the amount of time elapsed,
grow exponentially.

Monday, March 03, CDS MPhil Econometrics 35 Monday, March 03, CDS MPhil Econometrics 36
2014 Vijayamohan 2014 Vijayamohan

6
 Monday, March 03, 2014

Compound / Exponential growth function Incomparable Exponential Growth

So kt , where k is constant and t is variable is an The graph illustrates
exponential function. how an exponential
growth surpasses both
Now consider the
he compound growth function: linear and cubic
(1+r)t;
At = A0(1+r growths:
(1+rr)t can be written as kt,
The part (1+ Red line:
line Linear growth
where k = (1+r
(1+r): (50t): 500, at t = 10

⇒ the compound growth function itself is Blue line:

line Polynomial
exponential;;
exponential (t3): 1000, at t = 10
The two functions: two sides of the same coin – Green line:
line Exponential
one discrete; one continuous. (2t): 1024, at t = 10
Monday, March 03, CDS MPhil Econometrics 37 Monday, March 03, CDS MPhil Econometrics 38
2014 Vijayamohan 2014 Vijayamohan

An Exponential story:
story: Rice on a Chessboard An Exponential story:
story: Rice on a Chessboard

An ascetic presented an arrogant king with a All went well at first,

beautiful,, hand-
beautiful hand-made chessboard.
chessboard. but the requirement for 2n − 1 grains on the nth
The king asked what he would like in return square demanded
for his gift.
over a million (1,048,576
(1,048,576)) grains on the 21st
The ascetic surprised the king by asking for square,
one grain of rice on the first square
square,
more than a trillion (1.09951E+12) on the 41st,
two grains on the second
second, and 9.22337E+18 on the 64th.
four grains on the third
third, etc.
and there simply was not enough rice in the
The king readily agreed and asked for the
whole world for the final squares.
rice to be brought.
The king was thus humbled.
Monday, March 03, CDS MPhil Econometrics 39 Monday, March 03, CDS MPhil Econometrics 40
2014 Vijayamohan 2014 Vijayamohan

The number of grains of rice on the first

half of the chessboard is
kilo- K 103
meg
M 106
1 + 2 + 4 + 8... + 2,147,483,648 =
a
giga G 109 4,294,967,295
tera T 1012
peta P 1015 Or (232 − 1) grains of rice,
exa E 1018 *
or about 100,000 kg of rice (assuming
25 mg as the mass of one grain of rice)

Monday, March 03, CDS MPhil Econometrics 41 Monday, March 03, CDS MPhil Econometrics 42
2014 Vijayamohan 2014 Vijayamohan

7
 Monday, March 03, 2014

On the entire chessboard there would be

On the 64th square of the chessboard
264 − 1 = 18,446,744,073,709,551,615
alone there would be 263 =
grains of rice, weighing 461,168,602,000
9,223,372,036,854,775,808 grains of rice,
metric tons, which would be a heap of
or more than two billion times as much
rice larger than Mount Everest.
as on the first half of the chessboard.
This is around 1,000 times the global
production of rice in 2010 (464,000,000
metric tons)
Monday, March 03, CDS MPhil Econometrics 43 Monday, March 03, CDS MPhil Econometrics 44
2014 Vijayamohan 2014 Vijayamohan

Limitations Limitations
For instance, a population growth
Even when exponential growth
rate of 2% per year may seem small,
seems slow in the short run,

it becomes impressively fast in but it actually implies

the long run,

run doubling after 35 years,

with the initial quantity doubling again after another 35 years

doubling at the doubling time,

time
then doubling again and again. (i.e. becoming 4 times the initial
population), etc.

Monday, March 03, CDS MPhil Econometrics 45 Monday, March 03, CDS MPhil Econometrics 46
2014 Vijayamohan 2014 Vijayamohan

Doubling time Doubling time

The doubling time (or generation time)
time is the Consider the compound growth function:
period of time required for a quantity to (1+r)t
At = A0(1+r
double in size or value.
At becoming double A0 ⇒ At/A0 = 2; that is,
When the relative growth rate is constant,
constant
(1+r)t;
2 = (1+r
the quantity undergoes exponential growth
Taking (natural) log and rewriting,
and has a constant doubling time
t = ln(2)/ln(1 + r)
which can be calculated directly from the
growth rate.
rate = 0.693147/ln(1 + r).

Monday, March 03, CDS MPhil Econometrics 47 Monday, March 03, CDS MPhil Econometrics 48
2014 Vijayamohan 2014 Vijayamohan

8
 Monday, March 03, 2014

Doubling time Doubling time

Consider the exponential growth function: Annual growth rate of population
in India in 2001:
At = A0 ert
Male: 1.71%; Female: 1.92%; Total: 1.81%
At becoming double A0 ⇒ At/A0 = 2; that is, Doubling time for male population:

2 = ert; = 0.693147/ln(1.0171)
= 0.693147/0.016955 = 40.88 years.
Taking natural log and rewriting,
Doubling time for female population:
t = ln(2)/r = 0.693147/r. = 0.693147/ln(1.0192) = 36.45 years.

Note: If r is small, ln(1 + r) ≈ r. Doubling time for total population:

= 0.693147/ln(1.0181) = 38.64 years.
Monday, March 03, CDS MPhil Econometrics 49 Monday, March 03, CDS MPhil Econometrics 50
2014 Vijayamohan 2014 Vijayamohan

Doubling time and Growth rate Limitations (Continued….)

Growth Doubling Growth Doubling So some people challenge the

rate (%) time rate time
(years) (%) (years) exponential growth model
1 69.66 6 11.9 on the ground that
2 35 7 10.24
it is valid for the short term only,
only
3 23.45 8 9.01
i.e. nothing can grow indefinitely.
indefinitely
4 17.67 9 8.04
5 14.21 10 7.27

Monday, March 03, CDS MPhil Econometrics 51 Monday, March 03, CDS MPhil Econometrics 52
2014 Vijayamohan 2014 Vijayamohan

Exponential Growth Rate Exponential Growth rate

Given Exponential growth function: ln[ A( t ) / A( 0)]

r=
A(t) = A(0)ert t
Then Compare with compound growth rate:

A(t)/A(0) = ert, or  ln( At / A0 ) 

r = exp   −1
 t 
rt = ln[A(t)/A(0)], or

ln[ A( t ) / A(0)] Compound growth rate =

r= Exp(exponential
Exp(exponential growth rate) – 1.
t

Monday, March 03, CDS MPhil Econometrics 53 Monday, March 03, CDS MPhil Econometrics 54
2014 Vijayamohan 2014 Vijayamohan

9
 Monday, March 03, 2014

Exponential / Compound Growth rate Least--squares growth rate

Least
Least-squares growth rates are used wherever
Note that the growth rate in both the
there is a sufficiently long time series to
cases is based on two points in time
permit a reliable calculation.
only
only..
The least-squares (annual) growth rate, r, is
estimated by fitting a linear regression trend
It does not take into account the
line to the logarithmic (annual) values of the
intermediate values of the series. variable in the relevant period.

The regression equation takes the form

Hence Regression (Least-
(Least-squares)
growth rate.
rate. ln yt = a + bt.

Monday, March 03, CDS MPhil Econometrics 55 Monday, March 03, CDS MPhil Econometrics 56
2014 Vijayamohan 2014 Vijayamohan

Least--squares growth rate

Least Least--squares growth rate
Least
The regression equation, ln yt = a + bt The regression equation, ln yt = a + bt
may be taken as may also be taken as

(i) the logarithmic transformation of the (ii) the logarithmic transformation of the
equation y(t) = y(0)ert ,
exponential growth equation, equation yt = yo (1 + r)t ,
compound growth equation,
with a = ln yo and b = ln(1 + r).
with a = ln y(0) and b = r, the parameters to be
estimated.
If b* is the least-squares estimate of b, the
If b* is the least-squares estimate of b, the same average annual growth rate, r, is obtained as
gives the average annual exponential growth [exp(b*) – 1].
rate, r,
Remember Compound growth rate =
and is multiplied by 100 for expression as a
percentage. Exp(exponential
Exp(exponential growth rate) – 1.
Monday, March 03, CDS MPhil Econometrics 57 Monday, March 03, CDS MPhil Econometrics 58
2014 Vijayamohan 2014 Vijayamohan

Growth rate of real state domestic product (SDP) of

Kerala for 1960-
1960-61 – 1999-
1999-2000

Estimating
Growth rate of
real state domestic product (SDP) of
Kerala
for 1960-
1960-61 – 1999-
1999-2000:

Growth rate : Coefficient of t = 0.035215

= 3.52%

Monday, March 03, CDS MPhil Econometrics 59 Monday, March 03, CDS MPhil Econometrics 60
2014 Vijayamohan 2014 Vijayamohan

10
 Monday, March 03, 2014

Growth rate of real state domestic product (SDP) of Growth rate of real state domestic product (SDP) of
Kerala for 1960-
1960-61 – 1999-
1999-2000 Kerala for 1960-
1960-61 – 1999-
1999-2000
NSDP Kerala

NSDP Kerala
3e6
3e6
ln SDP = 13.432 + 0.0352155 time 2.5e6
Note the graph: 2.5e6

2e6
(476) (29.4) 2e6
A break, kink, around 1987-88.
1.5e6
R2 = 0.9578 F(1, 38) = 862.4; n = 40.1.5e6 1e6
1e6
1960 1965 1970 1975 1980 1985 1990 1995 2000
1960 1965 1970 1975 1980 1985 1990 1995 2000
Post-1987-88: higher growth rate.
15.00 LNSDP Kerala

14.75 Exponential growth rate: 0.0352 Two sub-period growth rates possible
14.50
or 3.52% per year.
year.
14.25 by estimating them separately,
14.00
Compound growth rate:
13.75
exp
exp(0.0352
(0.0352)) – 1 = 1.03584 – 1 = ln Yt = a1 + b1t + u1t;
13.50
1960 1965 1970 1975 1980 1985 1990 1995 2000
0.03584 or 3.58% per year.
year.
ln Yt = a2 + b2t + u2t; or,
t=k
Monday, March 03, CDS MPhil Econometrics 61 Monday, March 03, CDS MPhil Econometrics 62
2014 Vijayamohan 2014 Vijayamohan

Growth rate of real state domestic product (SDP) of

Kerala for 1960-
1960-61 – 1999-
1999-2000
NSDP Kerala

3e6

Note the graph: 2.5e6

2e6

A break, kink, around 1987-88.

1.5e6

1e6

1960 1965 1970 1975 1980 1985 1990 1995 2000

t=k
Estimating
equivalently, by fitting the single equation:
Discontinuous sub-
sub-period
ln Yt = a1D1 + a2D2 + b1D1t + b2D2t + ut; Growth rates of
real state domestic product (SDP) of
where Dj = 1 in sub-period j; Kerala
= 0, otherwise. for 1960-
1960-61 – 1999-
1999-2000:

t=k
Monday, March 03, CDS MPhil Econometrics 63 Monday, March 03, CDS MPhil Econometrics 64
2014 Vijayamohan 2014 Vijayamohan

Discontinuous sub-
sub-period growth rates for Kerala: Discontinuous sub-
sub-period growth rates for Kerala:
1960--61 – 1999-
1960 1999-2000 with kink in 1987-
1987-88 1960--61 – 1999-
1960 1999-2000 with kink in 1987-
1987-88
NSDP Kerala

3e6

2.5e6

2e6

1.5e6

1e6

1960 1965 1970 1975 1980 1985 1990 1995 2000

ln SDPt = 13.505D
13.505D1 + 12.54D
12.54D2 + 0.02939D
0.02939D1t

(651) (94.4) (22.7)

+ 0.061995D
0.061995D2t
(16)
R2 = 0.9857; F(3,36) = 825.4;
825.4; n = 40.
Monday, March 03, CDS MPhil Econometrics 65 Monday, March 03, CDS MPhil Econometrics 66
2014 Vijayamohan 2014 Vijayamohan

11
 Monday, March 03, 2014

Kinked Exponential Model Kinked Exponential Model

The two trend lines:
discontinuous. a1+b1k = a2 + b2k
Kinked Exponential Model for growth rate
This discontinuity can be estimation
eliminated via a linear
Substituting for a2 in
restriction:
t=k ln Yt = a1D1 + a2D2 + b1D1t + b2D2t + ut,
At the kink point, t = k, the two trend line
values are equal: We get the restricted form:

a1+b1k = a2 + b2k, or ln Yt = a1 + b1 (D1t + D2k) + b2 (D2t – D2k) + ut;

a2 = a1+b1k – b2k.
This gives the Kinked Exponential Model for Note: D1 + D2 = 1.
growth rate estimation.
estimation
Monday, March 03, CDS MPhil Econometrics 67 Monday, March 03, CDS MPhil Econometrics 68
2014 Vijayamohan 2014 Vijayamohan

Kinked Exponential Model Kinked Exponential Model

a1+b1k = a2 + b2k
ln Yt = a1 + b1 (D1t + D2k) + b2 (D2t – D2k) + ut;

Ref:
The OLS estimates of b1 and b2 give the
James K Boyce t=k
exponential growth rates for the two sub-
Kinked Exponential Models for Growth Rate
periods.
Estimation
There is a kink between the two trend lines, Oxford Bulletin of Economics and Statistics, 1986,
whenever b1 estimate ≠ b2 estimate. vol. 48, issue 4, pages 385-91

Monday, March 03, CDS MPhil Econometrics 69 Monday, March 03, CDS MPhil Econometrics 70
2014 Vijayamohan 2014 Vijayamohan

Kinked Exponential growth rates for Kerala:

1960-61 – 1999-2000 with kink in 1987-88

Estimating
Kinked Exponential
Growth rates of
real state domestic product (SDP) of
Kerala
for 1960-
1960-61 – 1999-
1999-2000:

Test whether there is a kink between the two

trend lines
Monday, March 03, CDS MPhil Econometrics 71 Monday, March 03, CDS MPhil Econometrics 72
2014 Vijayamohan 2014 Vijayamohan

12
 Monday, March 03, 2014

Kinked Exponential growth rates for Kerala:

1960-61 – 1999-2000 with kink in 1987-88 NSDP Kerala

3e6

2.5e6

2e6

1.5e6

1e6

1960 1965 1970 1975 1980 1985 1990 1995 2000

ln SDPt = 13.515 + 0.02835 (D1t + D2k)

(675) (25.3)

+ 0.05798 (D2t – D2k)

(19.9)

R2 = 0.984744; F(2,37) = 1194; n = 40.

Monday, March 03, CDS MPhil Econometrics 73 Monday, March 03, 2014 CDS MPhil Econometrics 74
2014 Vijayamohan Vijayamohan

13
CDS M Phil Econometrics
Non--Normality,
Non
Multicollinearity,
Multicollinearity,
OLS Specification Error

Violation of
Assumptions
Vijayamohanan Pillai N

CDS M Phil Econometrics Vijayamohan 1 CDS M Phil Econometrics Vijayamohan 2

Assumption Violations: Assumption Violations

How we will approach the question.
•Problems with u:

• Definition •The disturbances are not normally

distributed
• Implications
• Causes •The variance parameters in the
• Tests covariance-variance matrix are
different
• Remedies
•The disturbance terms are correlated
3/3/2014 CDS M Phil Econometrics 3 3/3/2014 4
Vijayamohan CDS M Phil Econometrics Vijayamohan

Assumption Violations:
Violations: Residual Analysis
•Problems with X:
•The explanatory variables and the • The residual for observation i, ei, = the
disturbance term are correlated difference between its observed and
predicted value
ei = Yi − Ŷi
•There is high linear dependence between
two or more explanatory variables • Check the assumptions of regression
by examining the residuals
•Incorrect model – e.g. exclusion of
relevant variables; inclusion of irrelevant
variables; incorrect functional form • Graphical Analysis of Residuals
3/3/2014 5
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan 6

1
 Residual Analysis or Residual Analysis for
Model Adequacy Tests e i = Yi − Ŷi
Linearity
Model adequacy diagnosis: An important stage,
Y Y
before hypothesis testing in forecast modelling

The fitted model is said to be adequate if it

explains the data set adequately, i.e., if the
x x
residual does not contain (or conceal) any
‘explainable non-randomness’ left from the

residuals

residuals
(‘explained’) model. x x

i.e. if the residual is purely random/white noise

Not Linear Linear
If all the OLS assumptions
CDS M Phil Econometrics
are satisfied.
Vijayamohan CDS M Phil Econometrics Vijayamohan
7 8

Residual Analysis for (1) Non-zero Mean for the

Independence
residuals (Definition)
o The residuals have a mean other than 0.
Not Independent
Independent o E ( u i ) ≠ 0 ( i = 1 ,2 ,... n )
residuals

X
o Note that this refers to the true residuals.
residuals

X Hence the estimated residuals have a

mean of 0, while the true residuals are
non-zero.
residuals

X
o E(u) > or < 0.

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

Chap 10

Non-zero Mean for the Non-zero Mean for the residuals

residuals (Implications)
• The true regression • Causes:
line is
• some form of specification error:
Yi = a + bX i + µe omitted variables.

• Therefore the ^
a
intercept is biased. a + μe
• The slope, b, is
unbiased.

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

11 12

2
 (2) Non-normally distributed Non-normally distributed errors
errors : Definition : Implications
• The residuals are not NID(0, σ) o The model is to some degree misspecified.
misspecified.
Normality Tests
Assumption Value Probability Decision(5%) o A collection of truly stochastic
Skewness 5.1766 0.000000 Rejected disturbances should have a normal
Kurtosis 4.6390 0.000004 Rejected
Histogram of Residuals of rate90 distribution:
35.0

The central limit theorem states that as

26.3
the number of random variables increases,
Count

17.5
the sum of their distributions tends to be a
8.8 normal distribution.
0.0
-1000.0 -250.0 500.0 1250.0 2000.0
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan
Residuals of rate90 13 14

Non-normally distributed errors : Non-normally distributed errors :

Implications (cont.) Implications (cont.)
• If the residuals are not normally distributed, then • If the residuals are normally distributed, then the
the estimators of a and b are also not normally LS estimator is also the ML estimator.
distributed. • MLEs are asymptotically efficient among
• Estimates are, however, still BLUE. consistent and asymptotically normally distributed
• Estimates are unbiased and have minimum estimators.
variance. If residuals are non-Normal,
• BUT, no longer asymptotically efficient, even • It is only our hypothesis tests which are affected.
though they are asymptotically unbiased
• (consistent).

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

15 16

Non-normally distributed Non-Normality Tests:

Residual Analysis for Normality
errors: Causes
• Generally caused by a specification error. A normal probability plot of the residuals
• Usually an omitted variable. can be used to check for normality:
• Can also result from Percent
100
o Outliers in data. The plotted
points
o Wrong functional form. reasonably
linear
0
-3 -2 -1 0 1 2 3
Residual

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

17 18

3
 Non-Normality Tests: Non-Normality Tests:
Residual Analysis for Normality Residual Analysis for Normality

Residual distribution Positively skewed Residual distribution heavy-tailed

The plotted
Percent Percent
100 100 points in the
The plotted
upper tail lie
points lie
above the
above the
comparison
comparison
line and
line on both
0 0 those in the
tails of the
lower tail
-3 -2 -1 0 1 2 3 distribution -3 -2 -1 0 1 2 3
Residual Residual
below the
line
CDS M Phil Econometrics Vijayamohan 19 CDS M Phil Econometrics Vijayamohan 20

Non-normally distributed errors : Tests for

non-normality (cont.)
• Jarque-Bera test
o This test examines both the skewness and
kurtosis of a distribution to test for normality.

o Where S is the skewness and K is the kurtosis of

the residuals.
o JB has a χ2 distribution with 2 df.
o Ho: S = 0; K = 3 (residuals normal) A portmanteau test, since the four lowest moments
about the origin are used jointly for its calculation.
o If Estimated JB near zero, p-value > 0.05,
» do not reject Ho.
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan
21 22

Non-normally distributed errors : Tests for Non

Non--normality Tests :
non-normality (cont.) An Example

Jarque, Carlos M.;

Jarque, M.; Anil K. Bera (1980). "Efficient
tests for normality, homoscedasticity and serial
independence of regression residuals". Economics
Letters 6 (3): 255–
255–259.
o Jarque,
Jarque, Carlos M.;
M.; Anil K. Bera (1981). "Efficient
tests for normality, homoscedasticity and serial
independence of regression residuals: Monte Carlo
evidence". Economics Letters 7 (4): 313–
313–318.
Variable name: resid; label: residuals

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

23 24

4
 Residual vs. Predictor Plot Residual vs. Fit Plot
100

100
Non-normality Tests :

Non-normality Tests :
An Example

An Example
50

50
Residuals

Residuals
0

0
Non-

Non-
-50

-50
1000 1500 2000 2500
Square feet 200 250 300 350 400
Fitted values

CDS M Phil Econometrics Vijayamohan 25 CDS M Phil Econometrics Vijayamohan 26

Histogram Stata Normality tests

.02

StatIsitics:
Non-normality Tests :

.015

Summaries, Tables and Tests:

An Example

Distributional plots and tests:

Density
.01

Skewness and kurtosis

normality tests,
.005

Shapiro-Wilk normality tests

0

-50 0
Residuals
50 100
Shapiro-Francia normality
Non-

tests

CDS M Phil Econometrics Vijayamohan 27 CDS M Phil Econometrics Vijayamohan 28

Normal Probability Plot Non-

Non-normality Tests :
An Example
1.00
Non-normality Tests :

0.75
Normal F[(resid-m)/s]
An Example

0.50
0.25
0.00
Non-

0.00 0.25 0.50 0.75 1.00

Empirical P[i] = i/(N+1)

Graphics :
Distributional graphs : Normal probability plot
CDS M Phil Econometrics Vijayamohan 29 CDS M Phil Econometrics Vijayamohan 30

5
 Non-
Non-normally distributed errors: Multicollinearity:: Definition
Multicollinearity
Remedies
• Try to modify your theory. • Multicollinearity : the condition where the
independent variables are related to each other.
• Omitted variable? Causation is not implied by multicollinearity.

• Outlier needing specification? • As any two (or more) variables become more and
more closely correlated, the condition worsens, and
• Modify your functional form by taking ‘approaches singularity’.
some variance transforming step such • Since the X's are supposed to be fixed, this a sample
problem.
as
• Since multicollinearity is almost always present, it is
• square root, exponentiation, logs, etc. a problem of degree, not merely existence.

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

31 32

Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

• Consider 2 explanatory variable model: • Y = Xβ

Xβ + U
• Yi = βo + β1X1i + β2X2i + Ui • In mean-
mean-deviations, we have

(x ′x) =  ∑ 1 ∑ x1x2 
 x2
• In matrix format,
• Y = Xβ
Xβ + U  ∑ x2 x1 ∑ x22 

β = (x ′x)−1(x′y)
ˆ
1  ∑ x2
2
− ∑ x1x2   ∑ x1y 
(x ′x)−1 =   (x′y) =  
−1 D  − ∑ x1x2 ∑ x12   ∑ x2 y 
Var(ˆ
β) = u (x ′x)
σ2

• Where D = |(x
|(xTx)|
CDS M Phil Econometrics Vijayamohan 33 CDS M Phil Econometrics Vijayamohan 34

Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

• Yi = βo + β1X1i + β2X2i + Ui • Yi = βo + β1X1i + β2X2i + Ui

1  ∑ x2 − ∑ x1x2 
2
β = (x ′x)−1(x ′y)
ˆ
u (x ′x)
β) = σ2
Var(ˆ −1 (x′x)−1 = 
D − ∑ x1x2

∑ x12 
1  ∑ x2 ∑ x1x2   ∑ x1y 
2
−
(x ′x)−1 =  (x′y) =   ∑ x22
D  − ∑ x1x2 ∑ x12   ∑ x2 y  β1 ) = σ2
Var(ˆ u
∑ x12 ∑ x22 − (∑ x1x2 )2

β1 =
ˆ ∑ x1y ∑ x22 − ∑ x2 y ∑ x1x2 var( ˆ
β1 ) =
1 σ2
u

∑ x12 ∑ x22 − (∑ x1x2 )2

(1 − 2
R1 .2 ) ∑ x 12
R1.22 = R2 in the regression of x1 on all the other variables
CDS M Phil Econometrics Vijayamohan 35 CDS M Phil Econometrics Vijayamohan 36

6
 Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity
o B) Perfect Multicollinearity
Consider the following cases
Given X = (x1, x2,…,
,…,x
xk)
A) No multicollinearity
xi : the i-th column of X with n observations.
X1 ⊥ X2 : Orthogonal ⇒ Cov(X1, X2) = R1.22 = 0.
The regression would appear to be identical If Xi is a perfect linear combination of one or
to separate bivariate regressions: both more other variables Xj,
coefficients and variances
x1c1 + x2c2 + ...x k c k = 0
ˆ
β1 =
∑ x1y ∑ x22 − ∑ x2 y ∑ x1x2 ˆ
β1 =
∑ x1 y
∑ x12 ∑ x22 − (∑ x1x2 )2 ∑ x12 where the constants (c
(ci) are not all zero,
then X'X is singular : |X'X| = 0.
1 σ2
u σ2
var( ˆ
β1 ) = var( ˆ
β1 ) = u The matrix does not have full rank.
2
(1 − R 1 .2 ) ∑ x 12 ∑ x12
CDS M Phil Econometrics Vijayamohan 37 CDS M Phil Econometrics Vijayamohan 38

Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

For example if x1 = kx2 then the variables x1

and x2, are exactly linearly related. The more common problem:
Imperfect or near multicollinearity
⇒ the matrix XTX is singular
⇒ the inverse does not exist. two or more of the explanatory variables are
approximately linearly related
Substitute x1 = kx2 in the following and see:

ˆ
β1 =
∑ x1y ∑ x22 − ∑ x2 y ∑ x1x2 x1c1 + x 2 c 2 + ...x k c k =& 0
∑ x12 ∑ x22 − (∑ x1x2 )2
CDS M Phil Econometrics Vijayamohan 39 CDS M Phil Econometrics Vijayamohan 40

Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

The impact of multicollinearity on the standard

For example: x1 = kx2 + v (where v is a errors of regression estimates :
random element:
In the context of a simple (bivariate) regression:

higher the r, the greater the prevalence Yi = β1 + β2 X i + ui

of multicollinearity.
σ2
var(ˆ
β2 ) = u

(Naturally, without v, r = 1 by definition.) ∑ xi2 where

x2
i = ∑ (Xi − X)2
CDS M Phil Econometrics Vijayamohan 41 CDS M Phil Econometrics Vijayamohan 42

7
 Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

With several explanatory variables : 1 σ2

u
var( ˆ
βk ) =
1 σ2
u
(1 − R2
k. ) ∑ x 2k
var( ˆ
βk ) =
(1 − R 2
k. ) ∑ x 2k ⇒ the (variance and standard error) of
the coefficients depend not only on the
where R2k- is the coefficient of standard error of the regression and the
determination when Xk is regressed on variation in the explanatory variable in
all the other explanatory variables. question
but also on R2k.

CDS M Phil Econometrics Vijayamohan 43 CDS M Phil Econometrics Vijayamohan 44

Multicollinearity:: Implications
Multicollinearity Multicollinearity:: Implications
Multicollinearity

1 σ2
var( ˆ
βk ) = u
• If the independent variables are highly
(1 − R2
k.) ∑ x 2k correlated, var( ˆ
β k ) inflated
Ceteris paribus,
• ⇒ t ratio's are lower
The correlation of Xk with the other variables ↑, • ⇒ insignificant β̂ k

var ↑. • and R2 tends to be high as well.

• ⇒ Significant F
Variation in Xk ↑, var ↓.
• Sign changes occur with the introduction of
Overall fit of the regression ↑, var ↓. a new variable
• The β̂ k still BLUE (but useless)
CDS M Phil Econometrics Vijayamohan 45 CDS M Phil Econometrics Vijayamohan 46

Multicollinearity: Multicollinearity:
Tests/Indicators Tests/Indicators
Some diagnostic statistics:
Nonexperimental data
never orthogonal (R2k. = 0) 1. Klein’s Rule of Thumb (Lawrence Klein: 1980)

So to some extent, multicollinearity always Multicollinearity is problem if

present. R2k. from auxiliary regression > the overall R2 .
1
When is multicollinearity a problem? 2. Variance inflation factor (VIF):
(1 − R 2
k. )

Some diagnostic statistics:

“The most useful single diagnostic guide”
J Johnston (1984)
CDS M Phil Econometrics Vijayamohan 47 CDS M Phil Econometrics Vijayamohan 48

8
 Interpreting VIFs Interpreting VIFs

• No multicollinearity ⇒ VIF = 1.0 • If the VIF is greater than 10.0, then

multicollinearity is probably severe.
• If the VIF for a variable were 9,
its standard error would be three times as • ⇒ 90% of the variance of Xj is explained by
large as it would be if its VIF was 1. the other Xs.
• In such a case, the coefficient would have to
be 3 times as large to be statistically • In small samples, a VIF of about 5.0 may
significant. indicate problems

CDS M Phil Econometrics Vijayamohan 49 CDS M Phil Econometrics Vijayamohan 50

Multicollinearity: Multicollinearity:
Tests/Indicators
Tests/Indicators (cont.)
• Also Tolerance: How large a VIF value has to be
1 2 to be “large enough”?
TOLk = = (1 − Rk.)
VIF
Belsley (1991) suggests :
• If the tolerance equals 1, the variables
are unrelated.
1. Eigen values of X’X
• If TOLj = 0, then they are perfectly 2. Condition index (CI) and
correlated.
3. Condition number (CN)

CDS M Phil Econometrics Vijayamohan 51 CDS M Phil Econometrics Vijayamohan 52

Multicollinearity: Multicollinearity:
Tests/Indicators Tests/Indicators
Given the eigenvalues λ1 > λ2 > λ3 > …. Largest CI = 5 – 10 ⇒ no problem
Largest CI = 30 – 100 ⇒ problematic
CIj = (λ1/ λj); j = 1, 2, 3, ….. Largest CI = 1000 – 3000 ⇒ severe problem

See
Stata/SPSS reports sqrt of CIj 1. DA Belsley, 1991 Conditioning Diagnostics:
Collinearity and weak data in Regression, Wiley
2. DA Belsley, E Kuh and RE Welsch 1980 Regression
CN = sqrt(Max eigenvalue/ Min eigenvalue) Diagnostics: Identifying Influential Data and
Sources of Collinearity, Wiley
Or CN = sqrt(Max CIj). 3. Norman R Draper and Harry Smith 2003 Applied
Regression Analysis 3rd Ed. Wiley
CDS M Phil Econometrics Vijayamohan 53 CDS M Phil Econometrics Vijayamohan 54

9
 Multicollinearity: Tests/Indicators
Multicollinearity: Multicollinearity: Tests/Indicators
Multicollinearity:
An Example An Example

CDS M Phil Econometrics Vijayamohan 55 CDS M Phil Econometrics Vijayamohan 56

Multicollinearity: Tests/Indicators
Multicollinearity:
For Condition Index and Number
An Example
Download the collin command

Postestimation statistics for regress:

In Reports and statistics:
Stata Variance inflation factors
CDS M Phil Econometrics Vijayamohan 57 CDS M Phil Econometrics Vijayamohan 58

Multicollinearity:: Tests/Indicators
An Example
Multicollinearity

Sqrt(λ1/ λ1)
Sqrt(λ1/ λ2)
Sqrt(λ1/ λ3)

CDS M Phil Econometrics Vijayamohan 59 CDS M Phil Econometrics Vijayamohan 60

10
 Multicollinearity: Causes Multicollinearity:: Remedies
Multicollinearity

• Sampling mechanism. Poorly constructed • Increase sample size

design & measurement scheme or limited • Omit Variables
range.
• Scale Construction/Transformation
• Statistical model specification: adding • Factor Analysis
polynomial terms or trend indicators.
• Too many variables in the model - the • Constrain the estimation. Such as the case
model is overdetermined. where you can set the value of one
coefficient relative to another.
• Theoretical specification is wrong. • Ignore it - report adjusted r2 and claim it
Inappropriate construction of theory or
even measurement warrants retention in the model.
CDS M Phil Econometrics Vijayamohan 61 CDS M Phil Econometrics Vijayamohan 62

Model Specification: Definition Model Specification: Definition

(cont.)
Specification error • There are basically 4 types of
misspecification we need to examine:
• covers any mistake in the set of
assumptions of a model and the o exclusion of a relevant variable
associated inference procedures o inclusion of an irrelevant variable
o functional form
• But it has come to be used for errors in o measurement error and misspecified
specifying the data matrix X. error term

CDS M Phil Econometrics Vijayamohan 63 CDS M Phil Econometrics Vijayamohan 64

Too Many or Too Few Variables 1. Omission of relevant variable:

Exclusion/ Underfitting Bias

• What happens if we include variables in Suppose the true model is:

our specification that don’t belong? y = β 0 + β1x1 + β2 x2 + u,
• There is no effect on our parameter But we estimate
estimate, and OLS remains unbiased ~ ~ ~
y = β0 + β1x1 + u,

• What if we exclude a variable from our Then

∑ (xi1 − x1 )yi
specification that does belong?
~
• OLS will usually be biased β1 =
∑ (xi1 − x1 )2
65 66

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

11
 Omitted Variable Bias Omitted Variable Bias
(cont) ~
β =
∑ (x i1 − x1 )y i
∑ (xi1 − x1 )2
1

~
β1 =
∑ (xi1 − x1 )yi
But y = β0 + β1x1 + β2 x2 + u,
∑ (xi1 − x1 )2 ~ ∑ (xi1 − x1 )xi2 + ∑ (xi1 − x1 )ui
β = β1 + β2
So the numerator becomes ∑ (xi1 − x1)2 ∑ (xi1 − x1)2
since E(ui) = 0, taking expectatio ns we have
∑ (x i1 − x1 )(β0 + β1x i1 + β2 xi2 + ui ) =

β1 ∑ (x i1 − x1 ) + β2 ∑ (x i1 − x1 )x i2 + ∑ (x i1 − x1 )ui
2 ~
E( β1 ) = β1 + β2
∑ (xi1 − x1 )xi2
∑ (xi1 − x1)2
67 68

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

Omitted Variable Bias (cont) Mis--specified Models

Mis
~ ~ ~
For the mis-specified model: y = β0 + β1x1 ,
Consider the regression of x2 on x1
~ σ2 1 σ2
Var( β1 ) = var( ˆ
β1 ) = u

~ ~ ~ ~ ∑ (xi1 − x1 )xi2 ∑ 2
x1 (1 − 2
R1 .2 ) ∑ x 12
x2 = δ0 + δ1x1 then δ1 =
∑ (xi1 − x1)2 Thus ~
( )
Var β1 < Var ˆ
β1 ( )
~ ~ unless x1 and x2 are uncorrelated
so E( β1) = β1 + β2 δ1
70
CDS M Phil Econometrics Vijayamohan 69 CDS M Phil Econometrics Vijayamohan

Misspecified Models (cont) 2. Inclusion of Irrelevant Variable:

Inclusion/overfitting
Inclusion/overfitting bias
Suppose this time the “true” model is: y = X1β1 +u
• While the variance of the estimator is but we estimate:
smaller for the misspecified model, unless y = X1b1 + X2b2 + e
β2 = 0 the misspecified model is biased
This equation is over-specified and tends to
occur when researchers adopt a “kitchen
• As the sample size grows, the variance of sink” approach to model building.
each estimator shrinks to zero, making the
variance difference less important This specification error does not lead to bias:
both the parameters and the error variance
are unbiasedly estimated.
71

CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan 72

12
 Inclusion of Irrelevant Variable: 3. Functional Form Mis-
Mis-specification
Inclusion/overfitting
Inclusion/ overfitting bias
β 
E(b1) = β =  1  A third type of mis-specification
0 occurs when we adopt an incorrect
functional form.
However, the estimates are inefficient: that
is to say, including irrelevant variables For example, we estimate a linear
raises the standard errors of our regression model whereas the "true"
coefficient estimates.
regression model is log-linear.

CDS M Phil Econometrics Vijayamohan 73 CDS M Phil Econometrics Vijayamohan Chap

Functional Form Mis

Mis--specification:
specification: Functional Form Mis
Mis--specification:
specification:
Implications Causes

• Incorrect functional form can result in • Theoretical design.

autocorrelation or heteroskedasticity.
o something is omitted,
o irrelevantly included,
• Next class
o mismeasured or
o non-linear.

CDS M Phil Econometrics Vijayamohan 75 CDS M Phil Econometrics Vijayamohan 76

Functional Form Mis

Mis--specification:
specification: Specification Error Tests
Tests
• Actual Specification Tests A common test for mis-specification is Ramsey’s
o No test can reveal poor theoretical regression specification error test (RESET) –
Ramsey (1969).
construction per se. y = Xβ + Zα + u
o The best indicator : the model has some The test is H0: α = 0.
undesirable statistical property;
o e.g a misspecified functional form will often Z contains powers of the predicted values of
[ ]
be indicated by a significant test for the dependent variable
autocorrelation. Z = ŷ 2 ŷ 3 ŷ 4
o Sometimes time-series models will have Essentially the variables are 'proxying' for
negative autocorrelation as a result of poor
other possible variables (or nonlinearities).
design.
CDS M Phil Econometrics Vijayamohan 77 CDS M Phil Econometrics Vijayamohan 78

13
 Ramsey’s regression specification
Specification Error Tests error test (RESET) – An Example

We then perform a standard F-test on the significance

of the additional variables.
′ e* − e′e) / J
(e* (R2 − R 2
*) / J
F= or F=
e′e /(n − k) (1 − R 2 ) /(n − k)

R2: from the model with Z variables

R*2: from the model without Z variables Statistics:

In Linear models and related
If the estimated F value is significant, accept the Regression diagnostics
hypothesis that model without Z variables is mis- Stata
specified Specification tests, etc.
CDS M Phil Econometrics Vijayamohan 79 CDS M Phil Econometrics Vijayamohan 80

Model Specification: Tests Model Specification: Remedies

• Specification Criteria for lagged designs • Model Building

o Most useful for comparing time series o Hendry and the LSE school of “top-
models with same set of variables, but down” modeling.
differing number of parameters o Nested Models
AIC (Akaike Information Criterion) o Stepwise Regression.
Schwartz Criterion Stepwise regression is a process of including
the variables in the model “one step at a
time.”

CDS M Phil Econometrics Vijayamohan 81 CDS M Phil Econometrics Vijayamohan 82

14
 3/3/2014

Assumption of Spherical Disturbances

CDS M Phil Econometrics
Var (u) = E(uu T ) = σ2 In

OLS  E(u12 ) E(u1u2 )


E(u2u1) E(u2
L E(u1un )

σ2

0 L 0

E(uu ) = 
T 2) L E(u2un ) 0 σ2 L 0 = σ2In
  =

M M O M
 M M O M 
 
E(unu1 ) E(unu2 ) L E(u2 
n)  L σ2 
Violation of  0

Therefore the requirement for spherical disturbances is

0

Assumptions (i) Var(ui) ≡ E(u2

i )= σ
2 ∀ i = 1,..., n homoskedasticity

and

(ii) Cov (ui , u j ) ≡ E(ui , u j ) = 0 ∀i ≠ j No autocorrelation

Vijayamohanan Pillai N
3-Mar-14 1 3-Mar-14 2
CDS M Phil Econometrics Vijayamohan
CDS M Phil Econometrics
Vijayamohan

Heteroskedasticity:: Definition
Heteroskedasticity

• Heteroskedasticity is a problem where

Heteroscedasticity the error terms do not have a constant
variance.

E (ui2 ) = σ i2

• That is, they may have a larger

variance when values of some Xi (or
the Yi’s themselves) are large (or
3-Mar-14 small).
CDS M Phil
Econometrics
Vijayamohan 3 3-Mar-14 CDS M Phil Econometrics 4
Vijayamohan

Heteroskedasticity:: Definition
Heteroskedasticity
Example of Heteroskedasticity
• This often gives the plots of the
f(y|x) residuals by the dependent variable
or appropriate independent variables
a characteristic fan or funnel shape.
. 180

. E(y|x) = β0 + β1x 160

140

. 120
100
80
60
Series1

40
20
0
0 50 100 150
x1 x2 x3 x
3-Mar-14 3-Mar-14 CDS M Phil Econometrics 6
5 Vijayamohan
CDS M Phil Econometrics Vijayamohan

1
 3/3/2014

Residual Analysis for Heteroskedasticity:: Definition

Heteroskedasticity
Equal Variance

Y
Y

x x
residuals

residuals

x x

Non-constant variance Constant variance

3-Mar-14
CDS M Phil 7 3-Mar-14 CDS M Phil Econometrics 8
Econometrics Vijayamohan
Vijayamohan

Heteroskedasticity
Heteroskedasticity
 E (u12 ) E (u1u2 ) L E (u1un )  σ 12 0 L 0
   
With nonspherical errors (e.g. E (uu T ) = 
E (u2 u1 ) E (u22 ) L E (u2un )  0 σ 22 L 0
  =
M M O M  M M O M 
heteroskedasticity and/or autocorrelation)  
L E (un2 )   
E (unu1 ) E (un u2 )  0 0 L σ n2 
Var (u ) = E (uu′) = σ 2 I
ω11 0 L 0
 0 ω 0
no longer applies. L
=σ2 
= σ 2Ω = Σ
22

 M M O M
 
E ( uu ′ ) = σ 2 Ω  0 0 L ω nn 

3-Mar-14 CDS M Phil Econometrics 9 3-Mar-14 CDS M Phil Econometrics 10

Vijayamohan Vijayamohan

Autocorrelation Heteroskedasticity
 E (u12 ) E (u1u2 ) L E (u1un )  Given our model, y = Xβ + u
  where X is a non-stochastic matrix with full
E (u2u1 ) E (u22 ) L E (u2un )
E (uuT ) =  column rank
 M M O M 
  E(u) = 0 and E(uu′) = σ2 Ω = Σ
E (unu1 ) E (unu2 ) L E (un2 ) 

ρ1 L ρ n −1 
The OLS estimator of β is
 1
 L ρ n −2 
2  ρ1 β = β + (X ′X ) − 1 X ′u
ˆ
1
=σ
 M M O M  = σ 2Ω = Σ
  E(ˆ
β) = β
 ρ n −1 ρ n −2 L 1 
So OLSE is still unbiased
3-Mar-14 CDS M Phil Econometrics 11 3-Mar-14 CDS M Phil Econometrics 12
Vijayamohan Vijayamohan

2
 3/3/2014

Heteroskedasticity Heteroskedasticity:: Causes

Heteroskedasticity
•The variance matrix is • It may be caused by:
Var (ˆ
β) = E{(ˆ β − β)′ }
β − β)(ˆ – Model misspecification - omitted

[ ]
variable or improper functional form.
⇒ E (X ′X)−1 X ′uu′ X(X ′X)−1
– Learning behaviors across time
⇒ (X ′X)−1 X ′E(uu′)X(X ′X)−1 – Changes in data collection or
⇒ (X ′X)−1 X ′σ2 ΩX(X ′X)−1 definitions.
⇒ σ2 (X ′X)−1(X ′ΩX)(X ′X)−1 – Outliers or breakdown in model.
−1 • Frequently observed in cross sectional
•Therefore any inference based on s (X ′X)
2
data sets where demographics are
will be incorrect. involved (population, GNP, etc).
3-Mar-14
CDS M Phil Econometrics s2 may be a biased estimator of σ
Vijayamohan
132 3-Mar-14
CDS M Phil Econometrics Vijayamohan
14

Heteroskedasticity:
Heteroskedasticity:
Heteroskedasticity:: Implications
Heteroskedasticity
Implications (cont.)
• The regression βs are unbiased • Types of Heteroskedasticity
/consistent.
– There are a number of types of
• But they are no longer the best heteroskedasticity.
estimator. • Additive
• They are not BLUE (not minimum • Multiplicative
variance - hence not efficient). • ARCH (Autoregressive conditional
– So confidence intervals are invalid. heteroskedastic) - a time series
– Wrong inference problem.
•
3-Mar-14 15 3-Mar-14 16
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

Testing for Heteroskedasticity

Testing for Heteroskedasticity
A number of formal tests : • Essentially want to test
• H0: Var(u|x1, x2,…, xk) = s2,
•Ramsey RESET test equivalent to
•Park test • H0: E(u2|x1, x2,…, xk) = E(u2) = s2
•Glejser test • If assume the relationship between u2
•Goldfeld-Quandt test and xj is linear, can test H0 as a linear
•Breusch-Pagan test restriction
•White test • So, for u2 = d0 + d1x1 +…+ dk xk + v
• this means testing
»H0: d1 = d2 = … = dk = 0
3-Mar-14 CDS M Phil Econometrics 17 3-Mar-14 18
Vijayamohan CDS M Phil Econometrics Vijayamohan

3
 3/3/2014

The Breusch-
Breusch-Pagan Test The Breusch
Breusch--Pagan Test
• Estimate the residuals from the OLS • can have 3 tests:
regression
1. Θ = ½ RSS,
• Get zi = û2
i /σ
2
that is
where RSS = regression sum of squares
• the residuals squared divided by σ2 = ∑ û2i / n from regressing zi on all of the xs ;

Θ ∼ χ2(k – 1) df.
• Regress zi on all of the xs.

• can have 3 tests:

3-Mar-14 19 3-Mar-14 20
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

The Breusch-
Breusch-Pagan Test The Breusch
Breusch--Pagan Test
2. The F statistic is just the reported F
statistic for overall significance of the 3. The (Breusch-Pagan-Godfrey) LM
regression, statistic is LM = nR2,

• F = [R2/k] / [(1 – R2)/(n – k – 1)],

• which is distributed Fk, n – k – 1 • which is distributed as χ2k-1

3-Mar-14 21 3-Mar-14 22
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

The Breusch
Breusch--Pagan Test : The Breusch
Breusch--Pagan Test :
An Example An Example
Consumption $ Income $
55 80
65 100
70 85
80 110
79 120
84
98
115
130
χ2 (1) = 3.84, α = 5%
95 140 χ2 (1) = 6.63, α = 1%
90 125
75 90
74 105
110 160
113 150
125 165
108 145
115 180
140 225
120 200
145
130
240
185 In Statistics:
152 220
Linear models and related
144
175
180
210
245
260
Stata Regression diagnostics
135 190
140 205 Specification tests, etc. F (1, 28) = 4.20, α = 5%
178
191
265
270 F (1, 28) = 7.56, α = 1%
137 230
3-Mar-14
189 250 CDS M Phil Econometrics 23 3-Mar-14 CDS M Phil Econometrics 24
Vijayamohan Vijayamohan

4
 3/3/2014

The White Test: The White Test:

White’s Generalized Heteroskedasticity test White’s Generalized Heteroskedasticity test
• The Breusch-Pagan test will detect any •The test proceeds as follows:
linear forms of heteroskedasticity
•Step 1: Estimate the original equation by least squares
• The White test allows for nonlinearities and obtain the residuals
by using squares and crossproducts of
•Step 2: Regress the squared residuals on a constant,
all the xs all the regressors, the regressors squared and their
• using an F or LM to test whether all the cross-products (interactions). For example, with two
explanatory variables
xj, xj2, and xjxk are jointly significant
[x 1 x2 x3 x22 x23 x2 x 3 ]
• can get to be unwieldy
•where x1 represents the constant term
3-Mar-14 CDS M Phil Econometrics 25 3-Mar-14 CDS M Phil Econometrics 26
Vijayamohan Vijayamohan

The White Test: The White Test:

White’s Generalized Heteroskedasticity test White’s Generalized Heteroskedasticity test:
An Example
•Step 3: The test statistic is nR 2 ~ χ(2k −1)

H0: Constant variance

•If nR2 > χ2

•then we have an issue with

heteroskedasticity.

3-Mar-14 CDS M Phil Econometrics 27 3-Mar-14 CDS M Phil Econometrics 28

Vijayamohan Vijayamohan

The White Test: The White Test: An Example

White’s Generalized Heteroskedasticity test:
An Example

Generate variables in Stata

NR2 = 15 x 0.3571 = 5.3565.

χ2 distribution with 5 df = 11.0705, α = 5%

• nR2 < χ2
3-Mar-14 CDS M Phil Econometrics 29 Conclusion ? CDS M Phil Econometrics 30
Vijayamohan
3-Mar-14
Vijayamohan •⇒homoskedasticity

5
 3/3/2014

Alternative form of the White test Remedies for Heteroskedasticity

• Consider that the fitted values from OLS, ŷ,

are a function of all the xs
•This depends on the form
heteroskedasticity takes.
• Thus, ŷ2 will be a function of the squares
and crossproducts and ŷ and ŷ2 can proxy •Indirect: Re-specify the model;
for all of the xj, xj2, and xjxk; so •Use heteroscedastic-consistent SEs

• Regress the residuals squared on ŷ and ŷ2 •Direct: GLS (WLS)

and use the R2 to form an F or LM statistic
adjust the variance-covariance
matrix
3-Mar-14 CDS M Phil Econometrics 31 3-Mar-14 CDS M Phil Econometrics 32
Vijayamohan Vijayamohan

Heteroskedastic Consistent SEs Heteroskedastic Consistent SEs

OLS estimate: unbiased and consistent. But White (1980) argues that all we really need is
−1 −1 an estimate of
β) = σ (X′X) (X′ΩX)(X′X)
Var(ˆ 2 2
X ′σ Ω X

•This can be re-written as Under very general conditions, it can be

−1 −1
shown that
β) = (X ′X)
Var(ˆ X ′Diag(σ2
i )X(X ′X)
n n
X′σ2 ΩX = ∑ σ2i xixi′ = ∑ e2i xixi′
i =1 i =1
•where Diag(σ2 2 2 2
i ) = Diag(σ1 , σ2 ,..., σ n )
Therefore the adjusted variance is
i.e., we need to estimate σ 12 0 n
L 0
all the σ2i ' s 
0 σ 22 L 0
 β) = (X ′X)−1 ∑ e2
est.asym.Var(ˆ i x ix i′(X ′X)
−1

- which is impossible. σ Ω=

2
i =1
 M M O M 
3-Mar-14 CDS M Phil Econometrics  33  3-Mar-14 CDS M Phil Econometrics 34
Vijayamohan
 0 0 L σ n2  Vijayamohan

Heteroskedastic Consistent SEs Heteroskedastic Consistent SEs

SEs::
Robust SEs
• A consistent estimate of the variance,
• the square root can be used as a standard Important to remember:
error for inference Robust standard errors only have
asymptotic justification –
• Typically known as robust standard errors. with small sample sizes t statistics formed
with robust standard errors will not have a
• Sometimes the estimated variance is distribution close to the t, and inferences
corrected for degrees of freedom by will not be correct
multiplying by n/(n – k – 1). In Stata, Linear regression:
SE/Robust:
• As n → ∞ it’s all the same, though.
(select robust – default)
3-Mar-14 CDS M Phil Econometrics 35 3-Mar-14 CDS M Phil Econometrics 36
Vijayamohan Vijayamohan

6
 3/3/2014

Generalized Least Squares

• It’s always possible to estimate robust

standard errors for OLS estimates,
• But if we know something of the specific
form of the heteroskedasticity, we can
obtain more efficient estimates than OLS

• The basic idea is to transform the model

into one that has homoskedastic errors –
• called generalized least squares

3-Mar-14 CDS M Phil Econometrics 38

3-Mar-14 CDS M Phil Econometrics 37 Vijayamohan
Vijayamohan

Generalized Least Squares Generalized Least Squares

• Given E(uu′) = σ2 Ω a positive definite

• Now premultiply the model
matrix.
• Any positive definite matrix can be • y = Xβ + u by P–1 to get
expressed in the form: PP’, where P is • y* = X*β + u*
nonsingular:
• Where y* = P–1y ;
• Ω = PP’, so that
• X* = P–1X ;
• P–1Ω P–1’ = I and
• P–1’P–1 = Ω–1 • u* = P–1u

3-Mar-14 CDS M Phil Econometrics 39 3-Mar-14 CDS M Phil Econometrics 40

Vijayamohan Vijayamohan

Generalized Least Squares Generalized Least Squares

• y * = X* β + u *
Given P–1Ω P–1’ = I and E(uu′) = σ2 Ω • OLS estimate of β is: b is the Generalized
• b = (X*’ X*)–1X*’y* Least Squares
(GLS) or Aitken
y* = X*β + u* • = (X’Ω–1 X)–1X’ Ω–1 y
estimator of β
Where y* = P–1y ; X* = P–1X ; u* = P–1u
• A BLUE of β with
Now E(u*u*’) = E(P–1uu’P–1’) = (P–1σ2ΩP–1’) = • Var(b) = σ2 (X*’ X*)–1
• = σ2 (X’Ω–1 X)–1
(σ2P–1ΩP–1’) = σ2I : Homoscedastic 1
• An unbiased estimate of σ2 is: ˆ
σ2 = e′Ω −1e
• Where e = (y – Xb) n −k
OLS assumptions satisfied
3-Mar-14 CDS M Phil Econometrics 41 3-Mar-14 CDS M Phil Econometrics 42
Vijayamohan Vijayamohan

7
 3/3/2014

Generalized Least Squares Generalized Least Squares:

Weighted Least squares

• If u is normally distributed, so is u*
• Thus b is a ML estimator • GLS is a weighted least squares
• So has min var in the class of all (WLS) procedure where each
unbiased estimators. squared residual is weighted by
the inverse of Var(ui|xi)

3-Mar-14 CDS M Phil Econometrics 43 3-Mar-14 CDS M Phil Econometrics 44

Vijayamohan Vijayamohan

Weighted Least Squares Weighted Least Squares

• Let heteroskedasticity be modeled as For example,
Var(u|x) = s2h(x), A common specification: var(u) ∝ to one of the
regressors or its square:
• where h(x) ≡ hi to be specified.
σ2 2 2
i = σ x ki
⇒ h(x) = x2
ki ; hi = x ki
• Now E(ui/√hi|x) = 0, because
• hi is only a function of x, and The weighted (transformed) LS regression model:
• Var(ui/√hi|x) = s2, because we know y x x u
= βk + β1 1 + β2 2 + ... +
• Var(u|x) = s2hi xk xk xk xk

E(ui/√xki) = 0, and WLS minimizes the

• So divide the whole equation by √hi and we Var(ui/√xki) = s2 Homoscedastic weighted sum of
have a model with homoskedastic error squares
(weighted by 1/hi)
3-Mar-14 CDS M Phil Econometrics 45 3-Mar-14 CDS M Phil Econometrics 46
Vijayamohan Vijayamohan

Feasible GLS Feasible GLS

• Run the original OLS model,
• More typical is the case where we don’t • save the residuals, û,
know the form of the heteroskedasticity • square them
• In this case, need to estimate h(xi) • Regress û2 on all of the independent
variables and
• This is the case of FGLS
• get the fitted values, ê
• Do WLS using 1/ê as the weight

3-Mar-14 CDS M Phil Econometrics 47 3-Mar-14 CDS M Phil Econometrics 48

Vijayamohan Vijayamohan

8
 3/3/2014

FGLS: Stata FGLS: Stata

3-Mar-14 CDS M Phil Econometrics 49 3-Mar-14 CDS M Phil Econometrics 50

Vijayamohan Vijayamohan

FGLS: Stata FGLS: Stata

Also Download wls0, using

3-Mar-14 CDS M Phil Econometrics 51 3-Mar-14 CDS M Phil Econometrics 52

Vijayamohan Vijayamohan
100

FGLS: Stata
50
Residuals
0
-50
-100

100

4 5 6 7 8
Price (Rs.)
50
Residuals
0
-50

Other weight types abse and loge2 and

squared fitted values (xb2
xb2).
-100

3-Mar-14 CDS M Phil Econometrics

2.5 3 3.5 4 53 4.5 3-Mar-14 CDS M Phil Econometrics 54
Vijayamohan Advertising (Rs 1000s) Vijayamohan

9
 3/3/2014

FGLS: Stata FGLS: Stata

Other weight types abse and loge2 and

squared fitted values (xb2
xb2). Compare with the FGLS done by steps
3-Mar-14 CDS M Phil Econometrics 55 3-Mar-14 CDS M Phil Econometrics 56
Vijayamohan Vijayamohan

Autocorrelation: Definition

Autocorrelation • Autocorrelation is correlation of the

errors (residuals) over time
Time (t) Residual Plot
Here, residuals show a 15
cyclic pattern, not 10

random. Cyclical 5
Residuals

0
patterns are a sign of -5 0 2 4 6 8

positive autocorrelation -10

-15
Time (t)

3-Mar-14 Violates the regression assumption that

CDS M Phil
Econometrics residuals are random and independent
Vijayamohan 57 3-Mar-14 58
CDS M Phil Econometrics Vijayamohan

Autocorrelation: Definition Autocorrelation: Definition

• The assumption violated is • Types of Autocorrelation

E (ui u j ) = 0

– Autoregressive (AR) processes

• Thus the Pearson’s r between the
– Moving Average (MA) processes
residuals from OLS and the same
residuals lagged on period is non-zero.
E ( ut ut −1 ) = 0

3-Mar-14 CDS M Phil Econometrics 59 3-Mar-14 CDS M Phil Econometrics 60

Vijayamohan Vijayamohan

10
 3/3/2014

Autocorrelation: Definition Autocorrelation: Definition

• Autoregressive processes (cont.)

– In 2nd order autocorrelation the residuals are
• Autoregressive processes AR(p) related to their t-2 values as well – AR(2):
– The residuals are related to their preceding
values. u t = ρ 1 u t −1 + ρ 2 u t − 2 + ε t
u t = ρ u t −1 + ε t – Larger order processes may occur as well:
AR(p)
– This is classic 1st order autocorrelation: AR(1)
process u t = ρ 1 u t −1 + ρ 2 u t − 2 + ... + ρ p u t − p + ε t

3-Mar-14 CDS M Phil Econometrics 61 3-Mar-14 CDS M Phil Econometrics 62

Vijayamohan Vijayamohan

Autocorrelation: Definition Autocorrelation: Definition

• Moving Average Processes MA(q)

Higher order processes for MA(q) also exist.
ut = ε t − θε t −1 ut = ε t − θ1ε t −1 − θ2ε t −2 − ... − θ qε t −q
• The error term is a function of some
random error and a portion of the previous The error term is a function of some random
random error. error and some portions of the previous
• MA(1) process random errors.

3-Mar-14 CDS M Phil Econometrics 63 3-Mar-14 CDS M Phil Econometrics 64

Vijayamohan Vijayamohan

Autocorrelation: Definition Autocorrelation: Definition

– AR processes represent shocks to

• Mixed processes ARMA(p,q) systems that have long-term
memory.
u t = ρ 1 u t − 1 + ρ 2 u t − 2 + ... + ρ p u t − p + ε t
− θ 1 ε t − 1 − θ 2 ε t − 2 − ... − θ q ε t − q – MA processes are quick shocks to
systems, but have only short
term memory.
• The error term is a complex function of
both autoregressive {AR(p)} and moving
average {MA(q)} processes.

3-Mar-14 CDS M Phil Econometrics 65 3-Mar-14 CDS M Phil Econometrics 66

Vijayamohan Vijayamohan

11
 3/3/2014

Autocorrelation: Implications Autocorrelation: Causes

• Coefficient estimates are unbiased, • Specification error

but the estimates are not BLUE – Omitted variable
• The variances are often greatly • Wrong functional form
underestimated (biased small) • Lagged effects
• Hence hypothesis tests are • Data Transformations
exceptionally suspect.
– Interpolation of missing data
– differencing

3-Mar-14 CDS M Phil Econometrics 67 3-Mar-14 CDS M Phil Econometrics 68

Vijayamohan Vijayamohan

Autocorrelation: Tests Testing for +ve

+ve Autocorrelation
• The Durbin-Watson statistic is used to test for H0: positive autocorrelation does not exist
autocorrelation H1: positive autocorrelation is present
H0: residuals are not correlated
Calculate the Durbin-Watson test statistic = d
H1: positive autocorrelation is present (Using Stata or SPSS)

The possible range is 0 ≤ d ≤ 4 Find the values dL and dU from the D-W table (for sample
n size, n and number of independent variables, k)
∑ (uˆ t − uˆ t −1 )2 d should be close to 2 if H0 is true
Decision rule: reject H0 if d < dL
d= t =2
n d < 2 ⇒ positive autocorrelation,
∑ uˆ
t =1
2
t d > 2 ⇒ negative autocorrelation Reject H0 Inconclusive Do not reject H0
2
0 dL dU
3-Mar-14 69 3-Mar-14 70
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

Testing for +ve

+ve Autocorrelation Durbin-Watson d Test:
Decision Rules
H0: positive autocorrelation does not exist
H1: positive autocorrelation is present Null Hypothesis Decision If
No + autocorrelation Reject 0 < d < dL
No + autocorrelation No Decision dL ≤ d ≤ dU
Decision rule: reject H0 if d < dL or 4 – dL < d < 4
No - autocorrelation Reject 4 – dL < d < 4
No - autocorrelation No Decision 4 – dU ≤ d ≤ 4 – dL
No +/- autocorrelation Do not reject dU < d < 4 – dL

Do not reject H0
d Testing for +ve
+ve Autocorrelation
0 dL dU 2 4 – dU 4 – dL 4
3-Mar-14 71 3-Mar-14 72
CDS M Phil Econometrics Vijayamohan CDS M Phil Econometrics Vijayamohan

12
 3/3/2014

Testing for +ve

+ve Autocorrelation Testing for +ve
+ve Autocorrelation
160
(continued)
• Suppose we have the following time 140

Example with n = 25:

120
series data: 100

Sales
80 y = 30.65 + 4.7038x
160
140
60 R2 = 0.8976
Durbin-Watson Calculations 40
120
100
Sum of Squared 20
Difference of Residuals 3296.18 0
Sales

80 y = 30.65 + 4.7038x
Sum of Squared 0 5 10 15 20 25 30
60 R2 = 0.8976 3279.98 Time
Residuals
40
20 Durbin-Watson Statistic 1.00494
0 T
0 5 10 15
Time
20 25 30
∑ (uˆ t −u
ˆ t −1 )2
3296.18
d= t =2
= = 1.00494
• Is there autocorrelation? T
3279.98
∑ uˆ
2
t
t =1
3-Mar-14 73 3-Mar-14 CDS M Phil Econometrics 74
CDS M Phil Econometrics Vijayamohan Vijayamohan

Testing for +ve

+ve Autocorrelation Autocorrelation: Tests (cont.)

• Here, n = 25 and k = 1 : one independent variable

• Using the Durbin-Watson table, • Durbin-Watson d (cont.)
– dL = 1.29 and dU = 1.45 – Note that the d is symmetric about 2.0,
so that negative autocorrelation will be
• d = 1.00494 < dL = 1.29,
indicated by a d > 2.0.
• Therefore the given linear model is not the appropriate
– Use the same distances above 2.0 as
model to forecast sales upper and lower bounds.

Decision: reject H0 since significant +ve

autocorrelation exists
d = 1.00494 < dL

Reject H0 Inconclusive Do not reject H0

0 dL=1.29 dU=1.45 2
CDS M Phil 75 3-Mar-14 CDS M Phil Econometrics 76
Econometrics 3-Mar-14 Vijayamohan
Vijayamohan

Autocorrelation: Tests (cont.) Autocorrelation: Remedies

• Durbin’s h • Generalized Least Squares
– Cannot use DW d if there is a lagged
endogenous variable in the model
• First difference method
– Take 1st differences of your Xs
 d T and Y
h = 1 − 
 2  1 − TS 2
y t −1 – Regress ∆Y on ∆X
– Assumes that ρ= +1
– Syt-12 is the estimated variance of the Yt-1
term • Generalized differences
– h has a standard normal distribution – Requires that ρ be known.

3-Mar-14 CDS M Phil Econometrics 77 3-Mar-14 CDS M Phil Econometrics 78

Vijayamohan Vijayamohan

13
 3/3/2014

Autocorrelation: Remedies Autocorrelation: Remedies

• Cochran-Orcutt method (cont.)
Cochran-Orcutt method
– (3) using the ρ obtained, perform the
(1) Estimate model using OLS and obtain
regression on the generalized differences
the residuals, ut.
(2) Using the residuals run the following (Yt − ˆ
ρYt −1) = B1(1 − ˆ
ρ) + B2(Xt − ˆ
ρXt −1) + (ut − ˆ
ρut −1)
regression.
– (4) Substitute the values of B1 and B2 into
û t = ˆ
ρû t −1 + v t the original regression to obtain new
estimates of the residuals.
– (5) Return to step 2 and repeat – until ρ no
longer changes.
3-Mar-14 CDS M Phil Econometrics 79 3-Mar-14 CDS M Phil Econometrics 80
Vijayamohan Vijayamohan

14
View publication stats