Econometrics

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Simple Regression Model

• The simple regression model can be used to study the relationship between two variables.
• The simple regression model has limitations as a general tool for empirical analysis.
• it is sometimes appropriate as an empirical tool.
• Learning how to interpret the simple regression model is good practice for studying multiple
regression, which we will do in subsequent chapters.

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Simple Regression Model

•Much of applied econometric analysis begins with the following premise: 𝑦 and 𝑥
are two variables, representing some population, and we are interested in “explaining
𝑦 in terms of 𝑥,” or in “studying how 𝑦 varies with changes in 𝑥.”

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Simple Regression Model
• In writing down a model that will “explain 𝑦 in terms of 𝑥,” we must confront three
issues.
• First, since there is never an exact relationship between two variables, how do we allow for other factors to affect 𝑦?
• Second, what is the functional relationship between
𝑦 and 𝑥?
• And third, how can we be sure we are capturing a ceteris paribus relationship between 𝑦 and 𝑥 (if that is a desired
goal)?

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Definition of
the Simple Regression Model
• We can resolve these ambiguities by writing down an equation relating 𝑦 to 𝑥 . A simple
equation is
𝑦 = 𝛽! + 𝛽" 𝑥 + 𝑢 (2.1)
• Equation (2.1), which is assumed to hold in the population of interest, defines the simple linear regression
model.
• It is also called the two-variable linear regression model or bivariate linear regression model because it relates the
two variables 𝑥 and 𝑦.

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Definition of
the Simple Regression Model
• When related by (2.1), the variables 𝑥 and 𝑦 have several different names used
interchangeably, as follows:

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

 • The most difficult issue to address is whether model (2.1) really allows us to draw
ceteris paribus conclusions about how 𝑥 affects 𝑦.
• We just saw in equation (2.2) that 𝛽! does measure the effect of 𝑥 on 𝑦,
holding all other factors (in 𝑢) fixed.
• Is this the end of the causality issue? Unfortunately, no.
• How can we hope to learn in general about the ceteris paribus effect of 𝑥 on 𝑦,
holding other factors fixed, when we are ignoring all those other factors?
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
•we are only able to get reliable estimators of 𝛽' and 𝛽( from a random
sample of data when we make an assumption restricting how the
unobservable 𝑢 is related to the explanatory variable 𝑥.
•Without such a restriction, we will not be able
to estimate the ceteris paribus effect, 𝛽! .
•Because 𝑢 and 𝑥 are random variables, we need a concept grounded in
probability.

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

•Before we state the key assumption about how 𝑥 and 𝑢 are related, we can
always make one assumption about 𝑢.
•As long as the intercept 𝛽' is included in the equation, nothing is lost by
assuming that the average value of 𝑢 in the population is zero. Mathematically,
𝐸(𝑢) = 0 (2.5)

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Zero Conditional Mean Assumption
• Suppose Econometric Model :
𝑦 = 𝛽! + 𝛽" 𝑥 + 𝑢

• Taking expected value (conditional on x) and using E(u|x)=0, we have

𝑬(𝒚|𝒙) = 𝜷𝟎 + 𝜷𝟏 𝒙

• We call this Population Regression Function , and is shown as a linear function (of x)
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 Population Regression Function

Population Regression Function

Exists, but the parameters are
UNKNOWN

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

 Population Regression Function

error
error

Population Regression Function

Exists, but the parameters are
UNKNOWN

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

PRF vs SRF

•We generally don’t have population data; using sample data to obtain the parameter
• Statistical inference
•Note:
• We denote population parameter WITHOUT HAT (without ^)
• We denote sample parameter with HAT (^)
• Missing hat may lead to confusion (and also reduce marks)
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
Important: Symbols
• Econometric Model
𝑦 = 𝛽! + 𝛽" 𝑥 + 𝑢
• PRF (𝐸(𝑢|𝑥) = 0)
𝐸(𝑦|𝑥) = 𝛽! + 𝛽" 𝑥
• SRF (by “OLS”)
𝑦0 = 𝛽1! + 𝛽1" 𝑥

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Mechanics of OLS

• We want to minimize the residual (the distance between actual data and estimated value – of
our independent variable)
• Minimize 𝑢%" è minimize (𝑦" − 𝑦)
(
• Because we have 𝑛 observation, we will have as many as 𝑛 of 𝑢%" , i.e {,
𝑢! … 𝑢,# }

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

Mechanics of OLS

• A better idea is to take the absolute value of 𝑢%" , i.e |𝑢%" | , so the representative number
#
is ∑"$! |𝑢%" |
• ..but this is also a problem if we have more than one SRF that has the same ∑#"$! |𝑢 1" |
• To overcome the problem we need to square 𝑢%" , so that the representative number will be
# % %
∑"$! 𝑢%" , or simply ∑ 𝑢%" , named “sum of squares of residual” or “residual sum of
squares”
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 Least Squared Method ^
SRF2:Y2= a1+a2X
Y
^1
SRF1:Y = b1+b2X
1 -2
1

1 -1/2
2 -11/2
0
-1
-1
X
SRF1: S|u| = |1| + |-1| + |-1| + |1| + |-1.5| = 5.5
Su2 =12 + 12 + 12 + 12 + 1.52 = 6.25 smaller
SRF2: S|u| = |2| + |0| + |-1/2| + |1| + |-2| = 5.5
Su2 = 22 + 02 + (-1/2)2 + 12+ (-2)2 = 9.25
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 Ordinary Least Squares (OLS) Method

Yi = b1 + b2Xi + ui
u i = Y i - b1 - b2X i

Minimize error sum of squared deviations:

n n
å ui2 = S(Y i - b1 - b2X i )2 = f(b1,b2)
i=1 i=1

23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge

 Minimize w.r.t. b1 and b2:
n
f(b1,b2) = S(Y i - b1 - b2x i 2
) = f(.)
i =1

¶f(.)
= - 2 S (Y i - b1 - b2Xi )
¶ b1

¶f(.)
= - 2 S Xi (Yi - b1 - b2Xi )
¶ b2
Set each of these two derivatives equal to zero and
solve these two equations for the two unknowns: b1 b2
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 To minimize f(.), you set the two
derivatives equal to zero to get:
¶f(.)
= - 2 S (Y i – b1 – b2Xi ) = 0
¶ b1

¶f(.)
= - 2 S xi (Yi - b1 – b2Xi ) = 0
¶ b2
When these two terms are set to zero,
b1 and b2 become b1 and b2 because they no longer
represent just any value of b1 and b2 but the special
values that correspond to the minimum of f(.) .
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 - 2 S (Y i - b1 – b2Xi ) = 0

-2S Xi (Y i – b1 – b2Xi ) = 0

S Yi - nb1 – b2 SXi = 0
2
S Xi Yi - b1 S X i - b2 S Xi = 0

nb1 + b2 S Xi = S Yi
2
b1 S Xi + b2S Xi = S Xi Yi
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 n SXi b1
= S Yi
S i S i
X X 2
b2 = S Xi Yi
Solve the two unknowns
n SXi Yi - S Xi SYi
b2 =
n SX i - (SXi )
2 2

S(Xi - X )(Yi -Y) Sxy

= =
S( i
X - X ) 2
Sx 2

b1 = Y - b2 x
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge
 Assumptions of Simple Regression
1. The linear regression Model:linear in
parameters
Y = b1+ b2X+ u
2. X values are fixed in repeated sampling, so that X is
not constant (X is nonstochastic).
3. Zero mean value of error terms (disturbance, ui),
E( ui | xi) = 0
4. Homoscedasticity or equal variance of ui, the
conditional variances of u i are identical, i.e.,
var(ui|xi) = s2
23/09/20 Source: Ekki Syamsulhakim, Gujarati & Wooldridge