10/22/2021

L/O/G/O

Chapter 2
THE SIMPLE REGRESSION MODEL
Wooldridge: Introductory Econometrics:
A Modern Approach, 5e

Lecturer: Hoàng Thị Diễm Hương

Content
1. Definition of the simple linear regression model
2. The ordinary least squares estimates (OLS)
3. Properties of OLS on any sample
4. Goodness-of-Fit
5. Assumptions for the simple linear regression model
6. Units of measurement and functional form

www.themegallery.com

1
 10/22/2021

1. The simple linear regression model

• Definition of the simple linear regression model
„Explains variable y in terms of variable x“
Intercept Slope parameter

Dependent variable, Independent variable, Error term,

explained variable, explanatory variable, disturbance,
response variable,… regressor,… unobservables,…

• This model consists of 1 dependent variable and 1 independent variable.

www.themegallery.com

1. The simple linear regression model

• Interpretation of the simple linear regression model
„Studies how y varies with changes in x:“

as long as

1 shows how much the dependent The interpretation is only correct if all
variable (y) changes if the independent other things remain unchanged when the
variable (x) is increased by 1 unit independent variable is increased by 1 unit
• The simple linear regression model is rarely applicable in practice but its
discussion is useful for pedagogical reasons.

www.themegallery.com

2
 10/22/2021

1. The simple linear regression model

• Example: Soybean yield and fertilizer
Rainfall, land quality,
presence of parasites, …
Measures the effect of fertilizer on yield,
holding all other factors fixed

• Example: A simple wage equation

Labor force experience,
tenure with current employer,
work ethic, intelligence, …
Measures the change in hourly wage
given another year of education,
holding all other factors fixed

www.themegallery.com

1. The simple linear regression model

• Population regression function (PRF)
( PRF ) : y   0  1 x  u
 E ( y | x)  E (  0  1 x  u | x)   0  1 x  E (u | x)   0  1 x
 This means that the
average value of the
Population regression
dependent variable function
can be expressed as a
linear function of the
explanatory variable
For individuals with x = x2,
the average value of y is
(0 + 1x2)
www.themegallery.com

3
 10/22/2021

2. The ordinary least squares estimates

• In order to estimate the regression model, one needs data
• A random sample of n observations
First observation For example, the i-th
Second observation data point
Third observation
n-th observation

Value of the Value of the Fitted regression

explanatory dependent line
variable of the variable of the
i-th observation i-th observation
www.themegallery.com

2. The ordinary least squares estimates

• The population regression function:
( PRF ) : y   0  1 x  u
• The sample regression function:
( SRF ) : y  ˆ0  ˆ1 x  uˆ
For i-th observation, the residual of the sample function is:
uˆi  yi  ˆ0  ˆ1 xi , i  1, n
• The Ordinary Least Squares Estimates: We choose 𝛽 và 𝛽 to make the sum of
squared residuals as small as possible:
n n
L   uˆ   ( yi  ˆ0  ˆ1 xi ) 2  min
2
i
i 1 i 1
How can we find 𝛽 và 𝛽 to minimize L?

www.themegallery.com

4
 10/22/2021

2. The ordinary least squares estimates

n
• First order conditions for OLS estimates:
L n (y i  y )( xi  x )
 2. ( yi  ˆ0  ˆ1 xi )  0  ˆ1  i 1
ˆ
 0 n
i 1
 (x  x ) i
2

L n i 1
 2. xi ( yi  ˆ0  ˆ1 xi )  0
ˆ
1 i 1 ˆ0  y  ˆ1 x
• Example: Find the sample regression function: (million dong/year)
x (Income) 100 80 98 95 75 79 78 69 81 88
90 75 78 88 62 69 65 55 60 70
1076 4309
 ˆ1   0.9881; ˆ0   12.0937
1089 363
www.themegallery.com
 ( SRF ) : y  12.0937  0.9881x  uˆ
9

2. The ordinary least squares estimates

• Example: CEO Salary and return on equity
salary   0  1 roe  u
Salary in thousands of dollars Return on equity of the CEO‘s firm

• Fitted regression:
  963.191  18.501 roe
salary
Fitted value (predicted value) When the return on equity increases by 1 percent,
of salary then salary is predicted to change by 18.501$, if all
other things remain unchanged.
• Causal interpretation of 𝜷𝟏 ?

www.themegallery.com

10

5
 10/22/2021

2. The ordinary least squares estimates

Fitted regression line

(depends on sample)

Unknown population
regression line

www.themegallery.com

11

3. Properties of OLS on any sample

n
• Deviations from regression line sum up to 0:  uˆ
i 1
i 0
n
• Correlation between deviations and regressors is zero:  x uˆ
i 1
i i 0
L n n
Because:  2. ( yi   0  1 xi )   uˆi  0
ˆ ˆ
ˆ
 0 i 1 i 1

L n n
 2. xi ( yi   0  1 xi )   xi uˆi  0
ˆ ˆ
ˆ
1 i 1 i 1

• Sample averages of y and x (𝒙, 𝒚) lie on regression line: y  ˆ0  ˆ1 x

www.themegallery.com

12

6
 10/22/2021

4. Goodness-of-Fit
• Aim: How well does the explanatory variable explain the dependent variable?
• Total sum of squares: represents total variation in dependent variable.
n
SST   ( yi  y ) 2
i 1
• Explained sum of squares: represents variation explained by regression.
n
SSE   ( yˆ i  y ) 2
i 1
• Residual sum of squares: represents variation that is not explained by regression.
n n
SSR   ( yi  yˆ i )   uˆi2
2

i 1 i 1
• Decomposition of total variation: SST = SSE + SSR
• SSR should be as small as possible.
www.themegallery.com

13

4. Goodness-of-Fit
 Coefficient of determination:
SSR SSE
R2  1  
SST SST
 Properties:
• 0  R2  1
• R2  0: OLS provides a poor fit to the data.
• R2  1: OLS provides a perfect fit to the data.
• R2 is interpreted as the fraction of the sample variation in y that is explained by
the regression.
• Caution: A high R2 does not necessarily mean that the regression has a causal
interpretation!

www.themegallery.com

14

7
 10/22/2021

4. Goodness-of-Fit
 Example: CEO Salary and return on equity
  963.191  18.501 roe
salary
The regression explains only 1.3%
n  209; R 2  0.0132 of the total variation in salaries

 Another expression of R2:

2
 n 
  ( yi  y )( yˆ i  yˆ ) 
R 2  n  i 1   ry2, yˆ
 2 
n
2
  ( yi  y )   ( yˆ i  yˆ ) 
 i 1  i 1 
www.themegallery.com

15

5. Assumptions for the simple linear regression model

 Assumption SLR .1: Linear in parameters
y   0  1 x  u
 Assumption SLR.2: Random sampling
 Assumption SLR.3: Sample variation in explanatory variable
The values of the explanatory variable (x) are not all the same (otherwise it would be
impossible to study how different values of the explanatory variable lead to different
values of the dependent variable).
 Assumption SLR.4: Zero conditional mean: E (ui | xi )  0
The value of the explanatory variable must contain no information about the mean of
the unobserved factors.
 Assumption SLR.5: Homoskedasticity: Var(ui | xi )  const  
2

The value of the explanatory variable must contain no information about the
variability
www.themegallery.com
of the unobserved factors.

16

8
 10/22/2021

5. Assumptions for the simple linear regression model

• Graphical illustration of homoskedasticity

The variability of the unobserved

influences does not dependent
on the value of the explanatory
variable

www.themegallery.com

17

5. Assumptions for the simple linear regression model

• An example for heteroskedasticity: Wage and education

The variance of the unobserved

determinants of wages increases
with the level of education

www.themegallery.com

18

9
 10/22/2021

5. Assumptions for the simple linear regression model

 Theorem 2.1: Unbiasedness of OLS
Using Assumptions SLR.1 through SLR.4:
E ( ˆ0 )   0 ; E ( ˆ1 )  1
• 𝛽 is unbiased for 𝛽 , and 𝛽 is unbiased for 𝛽 .
• In a given sample, the estimated coefficients (𝛽 and 𝛽 ) may be smaller or larger
than the true values 𝛽 and 𝛽 . However, on average, they will be equal to the
values that characterize the true relationship between y and x in the population.
 Theorem 2.2: Sampling variances of the OLS estimators
n n
Under Assumptions SLR.1 through SLR.5: 2 2
 . xi x 2
 2
 2
 2 i
Var( ˆ1 )  n
 Var( ˆ0 )  n
i 1
 . i 1
SSTx n SSTx
(x  x)
i 1
i
2
n. ( xi  x ) 2
i 1
www.themegallery.com

19

5. Assumptions for the simple linear regression model

 Estimating the variance of the error:
Since 𝜎 = 𝑉𝑎𝑟(𝑢 |𝑥 ) is unknown, an estimator of 2 is:
n

 uˆ 2
i
SSR
ˆ 2  i 1

n2 n2
 Theorem 2.3: Unbiased estimator of 2
Under Assumptions SLR.1 through SLR.5, 𝜎 is an unbiased estimator of 2:
E (ˆ 2 )   2
𝜎 = 𝜎 is called the standard error of the regression (SER) or the standard
error of the estimate.

www.themegallery.com

20

10
 10/22/2021

5. Assumptions for the simple linear regression model

Referring to Theorem 2.2, the variances of OLS estimators are:
n n
 . x
2 2
x 2
 2
 2 i
 2 i
Var( ˆ1 )  n
 Var( ˆ0 )  n
i 1
 . i 1
SSTx n SSTx
(x  x)
i 1
i
2
n. ( xi  x ) 2
i 1

If 𝜎 is plugged in for the unknown 2, then we have the estimated variances:
n n
ˆ . x
2 2
x 2
ˆ 2
ˆ 2 i
ˆ 2 i
V
ar( ˆ1 )  n
 V
ar( ˆ0 )  n
i 1
 . i 1
SSTx n SSTx
(x  x)
i 1
i
2
n. ( xi  x ) 2
i 1
www.themegallery.com

21

5. Assumptions for the simple linear regression model

The standard deviations of OLS estimators are: n

  x 2
i
sd ( ˆ1 )  Var( ˆ1 )  sd ( ˆ0 )  Var( ˆ0 )  . i 1
SSTx n SSTx
If 𝜎 is plugged in for the unknown , then we have the estimated standard deviations
of OLS estimators:
n

ˆ ˆ x 2

se( ˆ1 )  V
i
ar( ˆ1 )  se( ˆ0 )  V
ar( ˆ0 )  . i 1
SSTx n SSTx
The estimated standard deviations of the regression coefficients are called “standard
errors”. They measure how precisely the regression coefficients are estimated.
www.themegallery.com

22

11
 10/22/2021

6. Units of measurement and functional form

 The effects of changing units of measurement on OLS statistics:
Example: CEO Salary and return on equity
salary   0  1 roe  u
Salary in thousands of dollars Return on equity of the CEO‘s firm
The estimated equation is:
  963.191  18.501 roe
salary
Let salardol be salary in dollars, then we do not need to run the regression of salardol
on roe to know that the estimated equation is:
  963191  18501 roe
salardol
The intercept and the slope are multiplied by 1000
www.themegallery.com

23

6. Units of measurement and functional form

 Nonlinearities in simple regression:
What is the meaning of 1?
 Log – log model:
ln y   0  1 ln x  u
Meaning of 1: If x changes 1%, then y changes 1%, holding all other things fixed.
 Log – lin model:
ln y   0  1 x  u
Meaning of 1: If x changes 1 unit, then y changes (1100)%, holding all other
things fixed.
 Lin – log model:
y   0  1 ln x  u
Meaning of 1: If x changes 1%, then y changes (1/100) units, holding all other
things fixed.
www.themegallery.com

24

12