Scribd
Upload
16 views

Chapter 3 Econometric

The document discusses multiple regression analysis and ordinary least squares estimation. It covers the motivation for multiple regression, the mechanics and interpretation of OLS, the expected value and variance of OLS estimators, and the efficiency of OLS as proven by the Gauss-Markov theorem. Key topics include unbiasedness of OLS, omitted variable bias, heteroskedasticity, and the best linear unbiased estimator property of OLS.

Uploaded by

mukesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
16 views

Chapter 3 Econometric

The document discusses multiple regression analysis and ordinary least squares estimation. It covers the motivation for multiple regression, the mechanics and interpretation of OLS, the expected value and variance of OLS estimators, and the efficiency of OLS as proven by the Gauss-Markov theorem. Key topics include unbiasedness of OLS, omitted variable bias, heteroskedasticity, and the best linear unbiased estimator property of OLS.

Uploaded by

mukesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 23

Chapter 3

Multiple Regression Analysis:


Estimation
Overview
• Motivation for multiple regression
• Mechanics and interpretation of ordinary least squares
• The expected value of the OLS estimators
• The variance of the OLS estimators
• Efficiency of OLS: the Gauss-Markov theorem
Motivation for multiple regression
• Better suited for ceteris paribus analysis
• In a simple linear regression (SLR) model like

the zero conditional mean assumption can be controversial


• 𝛽መ1 is a biased/inconsistent estimator of 𝛽1 if not
• Multiple regression analysis allows to explicitly control for factors that
simultaneously affect the dependent variable, as in

• Other motivation
• More flexible functional forms; e.g., Mincer’s (1974) regression
Motivation for multiple regression
• Multiple (linear) regression model

• Analogous to SLR:
• Terminology for variables and parameters
• Linear in parameters
• Possibly nonlinear in variables; e.g.

(How does this affect interpretation of “slope” parameters 𝛽2 , 𝛽3 ?)


• Key assumption establishes ceteris paribus
interpretation of slope parameters
Mechanics and interpretation of ordinary
least squares
• Obtaining the OLS estimates
• Suppose we have a random sample { 𝑥𝑖1 , 𝑥𝑖2 , … , 𝑥𝑖𝑘 , 𝑦𝑖 : 𝑖 = 1, … , 𝑛} from
the population, with

• We want to estimate 𝛽0 , 𝛽1 , … , 𝛽𝑘 and obtain the sample regression function


(SRF)

• Ordinary least squares (OLS) estimates 𝛽መ0 , 𝛽መ1 , … , 𝛽መ𝑘 minimize the sum of
squared residuals (SSR)

• Where 𝑢ො 𝑖 is the residual and 𝑦ො𝑖 the fitted value for observation i
Mechanics and interpretation of ordinary
least squares
• OLS estimates satisfy the OLS first order conditions

• These are sample analogs of


Mechanics and interpretation of ordinary
least squares
• Interpreting the OLS regression equation
• Sample regression function
• Intercept 𝛽መ0 equals predicted value y when 𝑥1 = 𝑥2 = ⋯ = 𝑥𝑘 = 0
• Slopes 𝛽መ1 , … , 𝛽መ𝑘 have partial effect (ceteris paribus) interpretations:

So that

• Multiple regression mimics controlled laboratory setting with


nonexperimental data, by keeping other factors fixed
Mechanics and interpretation of ordinary
least squares
• Examples
• Determinants of college GPA (GPA1)

• Hourly wage equation (WAGE1)


Mechanics and interpretation of ordinary
least squares
• OLS fitted values and residuals
• Recall that the fitted or predicted value of y for observation i is

and that the corresponding residual for observation i equals

• The sample moment conditions (again) imply


• Sample average OLS residuals zero: so that 𝑦ത = 𝑦തො
• Sample covariance regressors and residuals zero
• (𝑥ҧ1 , … , 𝑥ҧ𝑘 , 𝑦)
ത on regression line
Mechanics and interpretation of ordinary
least squares
• Recall from SLR
Mechanics and interpretation of ordinary
least squares
• Goodness-of-fit
• Recall that SST = SSE + SSR, with
• SST the total sum of squares
• SSE the explained sum of squares
• SSR the residual sum of squares (or sum of sq. res.)
• The R-squared (or coefficient of determination)

measures the fraction in the sample variation in y explained by 𝑦ො


• 0 ≤ R2 ≤ 1
• R2 never decreases when variable is added
• Regression with low R2 common in the social sciences, but may estimate ceteris paribus
• relation well
The expected value of the OLS estimators
• Statistical properties of OLS
• Distribution OLS estimator(s) over random sample from
population
• Assumptions
• MLR1 (Linear in parameters):
• MLR2 (Random sampling): We have a random sample
{ 𝑥𝑖1 , 𝑥𝑖2 , … , 𝑥𝑖𝑘 , 𝑦𝑖 : 𝑖 = 1, … , 𝑛} following MLR1’s population model
• MLR3 (No perfect collinearity): In the sample (and therefore in the
population), none of the independent variables is constant, and there are
no exact linear relationships among the independent variables
• MLR4 (Zero conditional mean): (Explanatory variables
are exogenous, as opposed to endogenous)
The expected value of the OLS estimators
• Theorem 3.1: Unbiasedness of OLS
Under Assumptions MLR1–MLR4,
• Including irrelevant variables
Suppose we over-specify the model by including an irrelevant
variable 𝑥3 :
• Assumptions MLR1–MLR4 hold, but 𝛽3 = 0
• Theorem 3.1 implies that OLS estimators are unbiased
• In particular,
• May have undesirable effects on the variances of the OLS estimators
The expected value of the OLS estimators
• Omitted variable bias
• Suppose we underspecify the model by excluding a relevant
variable
• True population model satisfies MLR1–MLR4, e.g.

• Estimate 𝛽1 with the SLR estimator 𝛽෨1 of 𝛽1 in

where
The expected value of the OLS estimators
• It can be shown that

where
• 𝛽መ1 , 𝛽መ2 are the slope estimators from the MLR of wage on educ,abil
• 𝛿ሚ1 is the slope estimator from the SLR of abil on educ
• so that (implicitly conditional on the independent variables)
• because MLR1–MLR4 hold
• the omitted variable bias equals 𝛽2 𝛿ሚ1
The expected value of the OLS estimators
• Bias equals 𝛽2 𝛿ሚ1 , so no bias if
• ability does not affect wages 𝛽2 = 0
• ability and education are not correlated in the sample 𝛿ሚ1 = 0
• Bias is positive if ability has a positive effect on wages and more
able people take more education in the sample
The expected value of the OLS estimators
• Omitted variable bias: some terminology
• Upward bias:
• Downward bias:
• Bias toward zero: closer to 0 than 𝛽1
• Omitted variable bias: general case
• With two or more explanatory variables in the estimated (underspecified)
model, typically all estimators will be biased, even if only one explanatory
variable is correlated with the omitted variable
The variance of the OLS estimators
• Additional assumption for establishing variances and efficiency OLS
• MLR5 (Homoskedasticity):
• MLR1–MLR5 are the Gauss-Markov assumptions (for cross sections)
• Theorem 3.2: Sampling variances of the OLS slope estimators
Under Assumptions MLR1–MLR5 (and, implicitly, conditional on the
independent variables),

where SST is the total sample variation in 𝑥𝑗 and 𝑅𝑗2 is the R-


squared from regressing 𝑥𝑗 on all other independent variables
The variance of the OLS estimators
• Multicollinearity: a value of 𝑅𝑗2 close to, but not equal to, one
The variance of the OLS estimators
• Estimating the error variance
• The error variance 𝜎 2 can be estimated with

• Here, n − k − 1 are the degrees of freedom (df) for multiple regression,


the number of observations (n) minus the number of parameters (k + 1)
• Theorem 3.3: Unbiased estimation of 𝜎 2
Under Assumptions MLR1–MLR5,
• When n is large relative to k, multiplying by 1/n leads to virtually the
same answer
The variance of the OLS estimators
• Standard errors
• Substituting 𝜎ො 2 for 𝜎 2 in the appropriate expressions gives
• the standard error of the regression, 𝜎ො = 𝜎ො 2
• an estimate of the standard deviation of 𝛽መ𝑗 ：
that we called the standard error of 𝛽መ𝑗

• Example: hourly wage equation (WAGE1.DTA)


The variance of the OLS estimators
• Heteroskedasticity
• If MLR1–MLR4 hold, but MLR5 is violated, then
• 𝛽መ𝑗 remains unbiased, but
• the standard errors above are incorrect (see Chapter 8)
• Interpretation standard errors
• Standard error of 𝛽መ𝑗 gives an idea about the possible variation in the
estimate
• recall that we are actually interested in (the population quantity) 𝛽𝑗
• 𝛽መ𝑗 is a random variable with mean 𝛽𝑗 (unbiased) and some standard
deviation/standard error
• once we know that this random variable is (approximately) normally distributed
(see Chapter 4 and 5), we can calculate a confidence interval
Efficiency of OLS: the Gauss-Markov
theorem
• BLUE
• Many unbiased estimators of 𝛽𝑗 exist, but the OLS estimator is the best
linear unbiased estimator
• Estimator: rule that can be applied to any sample of data to produce estimate
• Unbiased: expectation estimator equals population parameter
• Linear: estimator is linear function of the data on the dependent variable:

where each 𝑤𝑖𝑗 can depend on the sample values of all independent variables
• Best: here, an estimator is best if it has the smallest variance
• Theorem 3.4: Gauss-Markov theorem
Under Assumptions MLR1–MLR5, 𝛽መ𝑗 s are the BLUEs of 𝛽𝑗

You might also like