Introductory Econometrics: A Modern Approach (7e)

Chapter 6
Multiple Regression Analysis:
Estimation

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 1
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (18 of 37)

• Including irrelevant variables in a regression model

• Omitting relevant variables: the simple case

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 2
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (19 of 37)

• Omitted variable bias

• Conclusion: All estimated coefficients will be biased

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 3
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (20 of 37)

• Example: Omitting ability in a wage equation

• When is there no omitted variable bias?

• If the omitted variable is irrelevant or uncorrelated

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 4
 Introductory Econometrics: A Modern Approach (7e)

Summary of Direction of Bias

Corr(x1, x2) > 0 Corr(x1, x2) < 0

b2 > 0 Positive bias Negative bias

b2 < 0 Negative bias Positive bias

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 5
 Introductory Econometrics: A Modern Approach (7e)

Omitted Variable Bias Summary

• Two cases where bias is equal to zero
• b2 = 0, that is x2 doesn’t really belong in model
• x1 and x2 are uncorrelated in the sample

• If correlation between x2 , x1 and x2 , y is the same direction, bias will

be positive
• If correlation between x2 , x1 and x2 , y is the opposite direction, bias
will be negative

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use.
6
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (21 of 37)

• Omitted variable bias: more general cases

• No general statements possible about direction of bias

• Analysis as in simple case if one regressor uncoreelated with others

• Example: Omitting ability in a wage equation

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 7
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (22 of 37)

• Standard assumptions for the multiple regression model (cont.)
• Assumption MLR.5 (Homoskedasticity)

• Example: Wage equation

• Short hand notation

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 8
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (23 of 37)

• Theorem 3.2 (Sampling variances of the OLS slope estimators)
• Under assumptions MLR.1 – MLR.5:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 9
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (24 of 37)

• Components of OLS Variances:

• 1) The error variance

• A high error variance increases the sampling variance because there is more
“noise” in the equation.
• A large error variance doesn‘t necessarily make estimates imprecise.
• The error variance does not decrease with sample size.

• 2) The total sample variation in the explanatory variable

• More sample variation leads to more precise estimates.
• Total sample variation automatically increases with the sample size.
• Increasing the sample size is thus a way to get more precise estimates.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 10
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (25 of 37)

• Components of OLS Variances (contd.)

• 3) Linear relationships among the independent variables

• Regress xj on all other independent variables (including constant)

• The R-squared of this regression will be the higher when xj can be better
explained by the other independent variables.

• The sampling variance of the slope estimator for xj will be higher when xj can be
better explained by the other independent variables.

• Under perfect multicollinearity, the variance of the slope estimator will

approach infinity.
© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 11
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (26 of 37)

• An example for multicollinearity

• The different expenditure categories will be strongly correlated because if a school has a lot of
resources it will spend a lot on everything.

• It will be hard to estimate the differential effects of different expenditure categories because all
expenditures are either high or low. For precise estimates of the differential effects, one would
need information about situations where expenditure categories change differentially.

• As a consequence, sampling variance of the estimated effects will be large.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 12
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (27 of 37)

• Discussion of the multicollinearity problem

• In the above example, it would probably be better to lump all expen-

diture categories together because effects cannot be disentangled.
• In other cases, dropping some independent variables may reduce
multicollinearity (but this may lead to omitted variable bias).

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 13
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (28 of 37)

• Only the sampling variance of the variables involved in
multicollinearity will be inflated; the estimates of other effects may be
very precise.
• Note that multicollinearity is not a violation of MLR.3 in the strict
sense.
• Multicollinearity may be detected through “variance inflation
factors.”

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 14
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (29 of 37)

• Variances in misspecified models
• The choice of whether to include a particular variable in a regression can be
made by analyzing the tradeoff between bias and variance.

• It might be the case that the likely omitted variable bias in the
misspecified model 2 is overcompensated by a smaller variance.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 15
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (30 of 37)

• Variances in misspecified models (cont.)

• Case 1:

• Case 2:

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 16
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (31 of 37)

• Estimating the error variance

• An unbiased estimate of the error variance can be obtained by substracting the number of
estimated regression coefficients from the number of observations. The number of observations
minus the number of estimated parameters is also called the degrees of freedom. The n
estimated squared residuals in the sum are not completely independent but related through the
k+1 equations that define the first order conditions of the minimization problem.

• Theorem 3.3 (Unbiased estimator of the error variance)

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 17
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (32 of 37)

• Estimation of the sampling variances of the OLS estimators

• Note that these formulas are only valid under assumptions MLR.1-
MLR.5 (in particular, there has to be homoskedasticity)

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 18
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (33 of 37)

• Efficiency of OLS: The Gauss-Markov Theorem
• Under assumptions MLR.1 - MLR.5, OLS is unbiased
• However, under these assumptions there may be many other estimators that
are unbiased.
• Which one is the unbiased estimator with the smallest variance?
• In order to answer this question one usually limits oneself to linear estimators,
i.e. estimators linear in the dependent variable.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 19
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (34 of 37)

• Theorem 3.4 (Gauss-Markov Theorem)
• Under assumptions MLR.1 - MLR.5, the OLS estimators are the best
linear unbiased estimators (BLUEs) of the regression coefficients, i.e.

• OLS is only the best estimator if MLR.1 – MLR.5 hold; if there is

heteroskedasticity for example, there are better estimators.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 20
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (35 of 37)

• Several Scenarios for Applying Multiple Regression
• Prediction
• The best prediction of y will be its conditional expectation

• Efficient markets
• Efficient markets theory states that a single variable acts as a sufficient statistic
for predicting y . Once we know this sufficient statistic, then additional
information is not useful in predicting y.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 21
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (36 of 37)

• Several Scenarios for Applying Multiple Regression (contd.)
• Measuring the tradeoff between two variables
• Consider regressing salary on pension compensation and other controls

• Testing for ceteris paribus group differences

• Differences in outcomes between groups can be evaluated with dummy variables

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 22
 Introductory Econometrics: A Modern Approach (7e)

Multiple Regression Analysis: Estimation (37 of 37)

• Several Scenarios for Applying Multiple Regression (contd.)
• Potential outcomes, treatment effects, and policy analysis
• With multiple regression, we can get closer to random assignment by
conditioning on observables.

• Inclusion of the x variables allows us to control for any reasons why

there may not be random assignment.
• For example, if y is earnings and w is participation in a job training program,
then the variables in x would include all of those variables that are likely to be
related to both earnings and participation in job training.

© 2020 Cengage. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a
password-protected website or school-approved learning management system for classroom use. 23