Section 11 Endogenous Regressors and

Instrumental Variables
If x is correlated with u
 If cov(x, u)  0, then OLS is biased and inconsistent
o Coefficient on x will pick up the effects of the parts of u that are correlated with it
in addition to the direct effects of x
o Direction of bias depends on sign of correlation between x and u
o This is exactly analogous to omitted-variables bias
 Measurement error (discussed above under internal validity)
o Suppose that the dependent variable is measured accurately but that we measure
x with error: xi  xi  i .
o The estimated model is yi  0  1 xi   ui  1i  .
o Because  is part of x and therefore correlated with it, the composite error term
is now correlated with the actual regressor, meaning that the OLS slope
estimator is biased and inconsistent.
2x
 If u and  are independent and normal, then plim ˆ 1  1.
2x  2
 The estimator is biased toward zero.
 If most of the variation in x comes from x, then the bias will be small.
 As the variance of the measurement error grows in relation to the
variation in the true variable, the magnitude of the bias increases.
 As a worst-case limit, if the true x doesn’t vary across our sample of
observations and all of the variation in our measure x is random noise,
then the expected value of our coefficient is zero.
o Best solution is getting a better measure.
o Alternatives are instrumental variables or direct measurement of degree of
measurement error.
 For example, if an alternative, precise measure is available for some
arguably random sub-sample of observations, then we can calculate the
variance of the true variable and the variance of the measurement error
and correct the estimate.
 Omitted-variables bias
o We derived this result at the beginning of the multiple regression analysis
o Omitted variable is included in error. If omitted variable is correlated with
included variable, then OLS estimator of coefficient on included variable is
biased and inconsistent.

~ 133 ~
 Simultaneous-equations bias (simultaneity bias)
o Suppose that y and x are part of a larger theoretical system of equations:
y  0  1 x    u
x   0  1 y    v
o The two variables are “jointly determined” and both are endogenous.
 There is “feedback” from y to x, or “reverse causality” (actually,
bidirectional)
o u  y  x , so u and x are correlated
o Supply and demand curves are difficult to estimate because both q and p are
endogenous
o We will study a simple supply-demand model in great detail when we talk about
simultaneous-equation models shortly

Instrumental variables
 Recall the method of moments analysis by which we derived the OLS estimators
o We used the assumed population moment conditions
E  u   0, cov  x , u   0 to derive the OLS normal equations as sample moment
1 n 2 1 n
conditions: 
n i 1
uˆi  0,  x i uˆi  0
n i 1
o If cov  x , u   0 , then the population moment conditions are invalid and we will
get biased and inconsistent estimators from the OLS sample moment conditions.
 The instrumental-variables estimator can be derived from the method of moments.
 As usual, suppose that y  0  1 x  u, but suppose that cov  x , u   0.
 Let z be a variable with the following properties:
o z does not have a direct effect on y. It does not belong in the equation alongside
x. (z affects y only through x, not independently.)
o z is exogenous. It is not correlated with u.
o z is strongly correlated with x, the endogenous regressor.
 This makes z a valid instrumental variable.
 We can exploit cov  z , u   0 as our second moment condition in place of cov  x , u   0 ,
which is not true for this model.
 The sample moment conditions are

 
n n

 uˆi   yi  ˆ 0IV  ˆ 1IV xi  0

i 1 i 1

 uˆ z   z  y 
n n

i i i i  ˆ 0IV  ˆ 1IV x i  0.
i 1 i 1

~ 134 ~
 n

z i  z  yi  y 
 Solving the normal equations yields ˆ 1IV  i 1
n
.
z
i 1
i  z  x i  x 

x i  x  yi  y 
 Compare this to the standard OLS slope estimator ˆ 1OLS  i 1
n
.
x
i 1
i  x  x i  x 

In matrix terms, ˆ   ZX Zy vs. ˆ   XX Xy

IV 1
OLS 1

 Properties of IV estimator:
o Consistent as long as z is exogenous
o Asymptotically normal
 
 2 
o ˆ 1IV ~ N  1 , n   x, z 
 , rxz  corr
 r2 
xz   x i  x 
2
 
 i 1 

 y 
n 2
i  ˆ 0IV  ˆ 1IV x i
o As usual, we estimate 2 by 2IV  i 1

N 2
 Weak instruments: If rxz is near zero, then the variance of ˆ 1IV is large and the IV
estimator is unreliable.

Two-stage least squares

 What if we have multiple strong instruments and/or multiple endogenous regressors in a
multiple regression?
 With more instruments than endogenous regressors, we have an “overidentified” system
with alternative choices of instruments.
o Suppose that xk is endogenous but the first k – 1 regressors are exogenous
o Suppose that z1 through zL are L valid instruments
o Any linear combination of the instruments is admissible
o Let’s choose the one that is as highly correlated with xK as possible:
 To get that, we regress x k   0  1 x1  ...   k 1 x k 1  1 z1  ...  L z L  vk
and use the fitted values xˆ k as the instrument for xk
 This amounts to doing two separate regressions, the first-stage regression of xk on the
exogenous x variables and the instruments z, then a second-stage regression of
y  0  1 x1  ...  k 1 x k 1  k xˆk  u *
 The estimators of  from the second-stage regression are called 2SLS estimators.

~ 135 ~
 But it’s not exactly like doing two separate regressions because our estimator of the error
variance uses the actual values of xK rather than the fitted values:

 y 
n 2
i  ˆ 0  ˆ 1 x i ,1  ...  ˆ k x i ,k
ˆ 2IV  i 1

nk
o (If you do the second regression manually substituting in the fitted values, Stata
will use the fitted values to calculate the residuals rather than the actual.)
 2SLS easily extends to multiple endogenous regressors, as long as there are more
independent instruments than endogenous regressors.
o Suppose there are G “good” exogenous regressors, B = k – G “bad” endogenous
regressors, and L “lucky” instrumental variables.
o L > B means overidentified, L = B is just identified, L < B means underidentified
(and can’t be estimated by IV)
o y  0  1 x1  ...  G xG  G 1 xG 1  ...  k x k  u
o First-stage regressions:
x G  j   0 j  1 j x1  ...   Gj x G  1 j z1  ...   Lj z L  v j , j  1,..., B
o Get fitted values:
xˆG  j  ˆ 0 j  ˆ 1 j x1  ...  ˆ Gj xG  ˆ 1 j z1  ...  ˆ Lj z L , j  1,..., B
o Regress original equation replacing endogenous regressors with fitted values
y  0  1 x1  ...  G xG  G 1 xˆG 1  ...  K xˆ K  u *
 To implement 2SLS in Stata, use ivregress 2sls depvar exvars (endvars = instvars) ,
options

Overidentification and generalized method of moments

 If we have additional instruments beyond the minimum (i.e., an overidentified system),
then we have more information than we need to estimate the model.
 Suppose that z1 and z2 are both valid instruments for endogenous x
 All three moment conditions:

 y 
n

i  ˆ 0  ˆ 1 x i  mˆ 1  0
i 1

z y 
n

i ,1 i  ˆ 0  ˆ 1 x i  mˆ 2  0
i 1

z y 
n

i ,2 i  ˆ 0  ˆ 1 x i  mˆ 3  0
i 1

are theoretically true.

 This opens the door for two possibilities:
o We can determine the degree to which we cannot satisfy all three of these
conditions simultaneously and use that as evidence of whether the model’s

~ 136 ~
 assumptions are valid. (If the model is perfect, then all three should be zero
except for sampling error.)
 These are specification tests discussed below
o We can think about alternative estimators (called GMM estimators) that would
minimize a weighted average of the squares of the m moments.
o 2SLS is a GMM estimator with a particular weighting of the moment conditions.

Instrument strength
 A strong instrument must provide correlation with part of the endogenous regressor that
is not explained by the other (exogenous) regressors.
 Regression of x K   0  1 x1  ...   k 1 x k 1  1 z1  vk allows us to test 1 = 0 with a
standard F = t2 test.
o However, conventional wisdom says that the instrument is weak unless F > 10
rather than the standard critical values for testing this hypothesis.
o This test can be applied with multiple instruments and one endogenous regressor,
with 10 still being the traditional threshold for weak instruments.
o (There are appropriate criteria for more complicated cases.)

Specification tests
 If the model is overidentified, then we can do two kinds of tests:
o A Hausman test of whether the x variables that we are treating as endogenous
truly are endogenous
o A test of the overidentifying restrictions, which can be interpreted as a test of
instrument validity
 Hausman test
o H 0 : cov  x , u   0, H 1 : cov  x , u   0
o Under null hypothesis, OLS is consistent and efficient, IV is consistent but
inefficient. Since both are consistent, q  ˆ OLS  ˆ IV  0 in large samples
o Under alternative hypothesis, OLS is inconsistent but IV is consistent, so
q  ˆ OLS  ˆ IV  c  0 in large samples.
o Stata command hausman implements the procedure
o Wooldridge gives alternative implementation adding residuals from first-stage
regression to OLS of original equation and testing whether they are significant
with a standard t test
 If they are significant, then the v̂ residual is correlated with y. This is the
part of xk that is not exogenous, so if it is correlated with y then there is
endogeneity and we need 2SLS because OLS is biased and inconsistent.
 Tests for instrument validity

~ 137 ~
 o Is z correlated with u?
 Can’t do direct test because we can’t get consistent estimators for u
without valid instruments, and we can’t know whether instruments are
valid without consistent estimator of u.
 With extra instruments (overidentified model), we can use some to test
the others.
o LM test: Do 2SLS/IV, get residuals, regress û on all z instruments and
exogenous regressors. Under null hypothesis that all instruments (and the overall
model specification) are valid, nR2 from this regression ~ 2 with L – B degrees of
freedom.
o The J statistic is another common test of overidentifying restrictions:
 As above, regress the 2SLS/IV residuals on the exogenous variables in
the equation and all the instruments.
 Compute the F statistic for the null hypothesis that the coefficients on the
instruments are zero.
 The test statistic LF (where L is the number of instruments) is
asymptotically distributed as a 2 with L – B degrees of freedom (number
of instruments – number of endogenous regressors = number of
overidentifying restrictions to be tested).
 Why does the J test or the LM test work?
 If the instruments are exogenous, then they should not be
correlated with y except through their effects on x.
 The 2SLS residuals are the part of y that is orthogonal to the part
of z that works through x.
 If that is the only correlation that z has with y (there is no direct
effect either direction), then the residuals should be uncorrelated
with z, conditional on the other x variables, the included
exogenous variables.
o Rejection of the null hypothesis tells us that at least one of the overidentifying
restrictions does not hold, which may mean that one or more of the instruments
is invalid.

Examples of IV regression
 Article about Waldman’s autism analysis in WSJ
o Waldman
 Unit of observation = state
 Dependent variable = autism prevalence
 Endogenous regressor = TV watching
 Instrument = rainfall

~ 138 ~
o Angrist
 Unit = individual male
 Dependent = future earnings
 Endogenous regressor = military service during Vietnam War
 Instrument = draft lottery
o Angrist & Krueger
 Unit = individual student
 Dependent = future earnings
 Endogenous regressor = age of HS graduation
 Instrument = birthdate & restrictions on entry
o Levitt
 Unit = city?
 Dependent = violent crime
 Endogenous regressor = number of police
 Instrument = elections
o Hoxby
 Unit = school
 Dependent = test scores
 Endogenous regressor = competition between school districts
 Instrument = streams
o Iyer
 Unit = Indian region
 Dependent = quality of public goods
 Endogenous regressor = British rule
 Instrument = death of heirless indigenous ruler
o Feyrer & Sacerdote
 Unit = island country
 Dependent = wealth
 Endogenous regressor = date of colonization
 Instrument = wind patterns
o Oster
 Unit = ?
 Dependent = risky sex in presence of HIV
 Endogenous regressor = degree of confrontation of HIV
 Instrument = distance from origin of HIV
o Bennedsen et al.
 Unit = Danish family firm
 Dependent = firm profitability
 Endogenous regressor = nepotism (did firm stay in family)
 Instrument = first-born son in family
o Olken
~ 139 ~
  Unit = region of Indonesia
 Dependent = social capital
 Endogenous regressor = amount of TV/radio consumed
 Instrument = signal strength of reception in region
 HGL Problem 10.1
o Now done as daily problem earlier.

o
o Description
 State data
 RENT paid on houses is DV
 MDHOUSE is median house price (in equation)
 PCTURBAN is % of population in urban areas (in equation)
 FAMINC is median family income (instrument)
 REG1–REG3 are regional dummies (instruments)
o (1) is OLS, (2) is first-stage regression, (3) is regression on exogenous x
without instruments
o (a) Why might MDHOUSE be endogenous?
o (b) Test if instruments are weak
 Instruments should have F > 10

~ 140 ~
 
o (c) Hausman test using (4)
 VHAT is the residual from the first-stage regression (2)
 (4) adds VHAT to second-stage regression
 Can use either fitted or actual MDHOUSE values here
because actual = fitted + VHAT
 H0 is that MDHOUSE is exogenous, in which case VHAT should not
add anything to the OLS structural regression
 t = –1.589/0.389 = –3.99. Critical value = 2.01, so reject exogeneity
of MDHOUSE
 We can’t use OLS and need to do 2SLS
o (d) Note that (4) and (5) have identical coefficients because VHAT is
orthogonal to regressors (PCTURBAN) by construction, so no omitted-
variable bias
 Comparing 2SLS (5) to OLS (1)
 PCTURBAN coefficient is smaller and se is larger
 MDHOUSE coefficient is larger and very significant
o (e) Testing overidentifying restrictions:
 Regress EHAT (2SLS residuals) on instruments and exogenous
regressors
 If only effect of instruments on RENT is through MDHOUSE, then
this regression should have no correlation:
 Effects of PCTURBAN have already been taken out in 2SLS
regression and can’t be in EHAT
 Instruments should have no independent effect if they are
valid
 NR2 = 50  0.226 = 11.3. Critical value for 2(3) = 7.815, so we reject
the null hypothesis of instrument validity.
 This test does not indicate which of the instruments might be at fault,
or indeed where in the collective hypotheses of the model
(exogeneity, absences from equation, correct specification) the
problem lies

~ 141 ~