• Perfect and imperfect multicollinearity

• Effects of multicollinearity
• Detecting multicollinearity
• Remedies for multicollinearity
The nature of Multicollinearity
Perfect multicollinearity:
When there are some functional relationships existing
among independent variables, that is  iXi = 0

or 1X1+ 2X2 + 3X3 +…+ iXi = 0

Such as 1X1+ 2X2 = 0  X1= -2X2
If multicollinearity is perfect,
perfect the regression coefficients of
the Xi variables, is, are indeterminate and their standard
errors, Se(i)s, are infinite.
Example:
Y =^0 + ^1X1 + ^
2X2 + ^

3-variable Case:
(yx1)(x22) - (yx2)(x1x2)
^
1 =
(x12)(x22) - (x1x2)2
If x 2 =  x 1,
(yx1)(2x12) - (yx1)(x1x1) 0
^
1 = =
(x1 )( x1 ) -  (x1x1)
2 2 2 2 2
0
Indeterminate
(yx2)(x12) - (yx1)(x1x2)
^
2 =
(x12)(x22) - (x1x2)2
Similarly
If x2 = x1 ^
(yx1)(x12) - (yx1)(x1x1) 0
2 = =
(x1 )( x1 ) -  (x1x1)
2 2 2 2 2
0
Indeterminate
If multicollinearity is imperfect,

x 2 = 1 x 1 +  where  is a stochastic error

(or x 2 = 0 + 1 x 1 +  )

Then the regression coefficients, although determinate,

possess large standard errors, which means the
coefficients can be estimated but less accurate.
^ (yx1)(2x12 +  2 ) - ( yx1 + y )( x1x1+ x1 )
1 =
(x12)(2 x12 +  2 ) - ( x1x1 + x1 )2

0 =0
(Why?) –
uncorrelated
Example: Production function
(log form)
Yi = 0 + 1X1i + 2X2i + 3X3i + i
Y X1 X2 X3 Y: Output

122 10 50 52 X1: Capital

170 15 75 75 X2: Labor
202 18 90 97 X3: Land
270 24 120 129
X1 = 5X2
330 30 150 152
Example: Perfect multicollinearity
a. Suppose D1, D2, D3 and D4 = 1 for spring,
summer, autumn and winter, respectively.
Yi = 0 + 1D1i + 2D2i + 3D3i + 4D4i + 1X1i + i.
b. Yi = 0 + 1X1i + 2X2i + 3X3i + i
X1: Nominal interest rate;
X2: Real interest rate;
X3: CPI
c. Yt = 0 + 1Xt + 2Xt + 3Xt-1 + t
Where Xt = (Xt – Xt-1) is called “first different”
 Imperfect Multicollinearity

Yi = 0 + 1X1i + 2X2i + … + KXKi + i

When some independent variables are linearly
correlated but the relation is not exact, there is
imperfect multicollinearity.
0 + 1X1i + 2X2i +  + KXKi + ui = 0
where u is a random error term and k  0 for
some k.
When will it be a
 Consequences of imperfect multicollinearity

1. The estimated coefficients are still BLUE,

however, OLS estimators have large variances and covariances,

thus making the estimation with less accuracy.

2. The estimation confidence intervals tend to be
much wider, leading to accept the “zero null
hypothesis” more readily.
3. The t-statistics of coefficients tend to be
Can be
statistically insignificant.
detected from
4. The R2 can be very high. regression
5. The OLS estimators and their standard errors results
can be sensitive to small change in the data.
 OLS estimators are still BLUE
under imperfect multicollinearity
Why???
Remarks:
•Unbiasedness is a repeated sampling property, not
about the properties of estimators in any given
sample
•Minimum variance does not mean small variance
•Imperfect multicollinearity is just a sample
phenomenon
 Effects of Imperfect
Multicollinearity

Unaffected:
a. OLS estimators are still BLUE.
b. The overall fit of the equation
c. The estimation of the coefficients of
non-multicollinear variables
The variances of OLS estimators
increase with the degree of
multicollinearity
Regression model:
Yi = 0 + 1X1i + 2X2i + i

High correlation between X1 and X2

Difficult to isolate effects of X1 and X2 from
each other, as should in OLS partial regression

coefficients
Closer relation between X1 and X2

 larger r223 toward 1

 larger VIF; VIF=1/(1 - r223 )
approaches infinity
 larger variances
 Larger var   
ˆ 
k

a. More likely to get unexpected signs.

signs

  
b. se ˆ k
2
tends to be large

Larger variances tend to increase the

standard errors of estimated coefficients.
coefficients
c. Larger standard errors  Lower t-values
ˆ  *
k k
tk 
 
se ˆ k
d. Larger standard errors
 Wider confidence intervals

 
ˆ k  t df , / 2  se ˆ k

Less precise interval estimates.

 Detecting Multicollinearity

IN FACT, No formal “tests” for multicollinearity

Careful considerations should be taken care of:
1. It is a question of degree and not a kind of – its presence
is not the main issue
2. Since the explanatory variables are assumed to
nonstochastic, it is a feature of the sample not the
population – issue of sample selection
3. Therefore, we test the degree of multicollinearity and not
its presence
 Detecting Multicollinearity

IN FACT, No formal “tests” for multicollinearity

1.Few significant t-ratios but a high R 2 and a
collective significance of the variables (F test)
2. High pairwise correlation between the explanatory
variables – does not always hold
3. Examination of partial correlations – sometimes
4. Estimate auxiliary regressions
5. Calculate condition index from eigenvalues
6. Estimate variance inflation factor (VIF)
Dependent Variable: Y
Variable Coefficient Std. Error t-Statistic Prob.
C 2933.906 8172.257 0.359008 0.7271
X1 50.53776 69.70081 0.725067 0.4850
X2 -103.5042 51.14744 -2.023644 0.0705
X3 6.115795 3.713679 1.646829 0.1306
X4 -105.9787 151.9428 -0.697491 0.5014
X5 0.123797 0.122672 1.009174 0.3367
R-squared 0.754505 Akaike info criterion 16.23749
Adjusted R- 0.631757 Schwarz criterion 16.52721
squared
S.E. of regression 706.1345 F-statistic 6.146800

Sum squared 4986260. Prob(F-statistic) 0.007426

resid
 High pairwise correlation between
the explanatory variables

CORRELATION MATRIX

X1 X2 X3 X4 X5
X1 1.00000 0.99686 0.99135 0.52583 0.97214
X2 0.99686 1.00000 0.99127 0.54331 0.96524
X3 0.99135 0.99127 1.00000 0.46144 0.97262
X4 0.52583 0.54331 0.46144 1.00000 0.53618
X5 0.97214 0.96524 0.97262 0.53618 1.00000
 Auxiliary Regressions

Auxiliary Regressions - regress each explanatory

variable on the remaining explanatory variables
X 2i  a1  a2 X 3i  a3 X 4i
X 3i  b1  b2 X 2i  b3 X 4i
X 4i  c1  c2 X 2i  c3 X 3i

The R2 will show how strongly Xji is collinear with

the other explanatory variables
 Dependent
Variable R2 VIF Prob (F-stat)
X1 0.995877 242.5418 0.0000
X2 0.99766 427.3504 0.0000
X3 0.995625 228.5714 0.0000
X4 0.793711 4.8476 0.00091
X5 0.974219 38.7883 0.0000
Fi=(R2xi.x2x3…xk)/(k – 2)/(1 - R2xi.x2x3…xk)/(n – k + 1)
If F>F stat. Xi is collinear with particular X’s
Or if auxillary R2 > overall R2 – troubled multi..
Following Klien’s rule of thumb
Detection of Multicollinearity
Example: Data set: Table#M1
COi = 0 + 1Ydi + 2LAi + i
CO: Annual consumption expenditure
Yd: Annual disposable income
LA: Liquid assets
Since LA is highly related to YD

Results:
High R2 and adjust R2
Less significant t-values
 OLS estimates and SE’s can be
sensitive to specification and small
changes in data

Specification changes:
Add or drop variables
Small changes:
Add or drop some observations
Change some data values
 High Simple
Correlation
Coefficients

rij 
 X  X X  X 
i i j j

 X  X    X  X 
i i
2
j j
2

Remark: High rij for any i and j is a sufficient indicator for

the existence of multicollinearity but not necessary.
 Variance Inflation Factors (VIF)
method
Procedures:
(1) Y   0  1 X 1   2 X 2  . . .   k X k  
(2) X 1  1   2 X 2   3 X 3  ...   k X k 
Obtain Rk2

(3)  
VIF ˆk 
1
1  Rk
2

Rule of thumb: VIF > 5  multicollinearity

Notes: (a.) Using VIF is not a statistical test.
(b.) The cutting point is arbitrary, some use VIF>10
Remedial Measures
1. Drop the Redundant Variable

Using theories to pick the variables to drop.

Do not drop a variable strongly supported by theory.
(Danger of specification error)
 Insignificant

Insignificant

Since M1 and M2
are highly related

Other examples: CPI <=> WPI;

CD rate <=> TB rate
GDP  GNP  GNI
Check after dropping variables:
• The estimation of the coefficients of other variables are not
affected. (necessary)
• R2 does not fall much when some collinear variables are dropped.
(necessary)
• More significant t-values vs. smaller standard errors (likely)
2. Redesigning the Regression
Model
There is no definite rule for this method.
    
Ft  f ( PFt , PBt Yd t , N t , Pt )  t
Ft = average pounds of fish consumed per capita
PFt = price index for fish
PBt = price index for beef
Ydt = real per capita disposable income
N = the # of Catholic
P = dummy = 1 after the Pop’s 1966 decision, = 0 otherwise

Ft   0  1 PFt   2 PBt   3 LnYdt   4 N t   5 Pt   t

 High correlations

VIFPF = 43.4
Signs are unexpected VIFlnYd =23.3
Most t-values are insignificant
VIFPB = 18.9
VIFN =18.5
VIFP =4.4
 Drop N, but not improved

Improved

Use the Relative Prices

(RPt = PFt/PBt)
  
Ft  f ( RPt , Yd t , P t )   t

Ft = 0 + 1RPt + 2lnYdt + 3Pt + t

 Improved much

Using the lagged term of RP to allow the lag effect

in the regression
Ft = 0 + 1RPt-1 + 2lnYdt + 3Pt + t
 3. Using A Priori Information
From previous empirical work, e.g.
Consi = 0 + 1Incomei + 2Wealthi + i
and a priori information: 2 = 0.1.
0.1
Then construct a new variable or proxy,
(Cons*i = Consi – 0.1Wealthi)
Run OLS: Cons*i = 0 + 1Incomei + i
4. Transformation of the Model
Taking first differences of time series data.
Origin regression model:
Yt = 0 + 1X1t + 2X2t + t
Transforming model: First differencing
Yt = 1X1t + 2X2t + ut
Where Yt = Yt- Yt-1, (Yt-1 is called a lagged
lag term)
X1t = X1t- X1,t-1, X2t = X2t- X2,t-1,
5. Collect More Data (expand sample size)

Larger sample size means smaller variance of

estimators.

6. Doing Nothing

Point out the problem of multicollinearity.