Chapter 2

Simple linear regression model

 Definition of the simple regression model
• Much of the applied econometrics analysis begins with the
following premises: y and x are two variables, representing
some population, and we are interested in explaining y in
terms of x or in studying how y varies with the changes in x.
• In writing down a model that will explain y in terms of x, we
must confront three issues.
1. Since there is never an exact relationship between two
variables, how do we allow other factors to affect y?
2. What is the functional relationship between y and x?
3. How can we be sure we are capturing a ceteris paribus
relationship between y and x?
 Definition of the simple regression model

•   can resolve these ambiguities by writing down an

We
equation relating y to x.

The above equation defines the simple linear regression

model. It is also called the two variable linear regression
model or bivariate linear regression model because it
relates the two variables x and y.
 THE CONCEPT OF POPULATION REGRESSION
FUNІCTION (PRF)
E(Y | Xi) = f (Xi) is known as conditional expectation
function(CEF) or population regression function (PRF)
or population regression (PR) for short.
In simple terms, it tells how the mean or average of
response of Y varies with X.
E(Y)= f(Xi) is known as unconditional mean or
unconditional expected value
WHICH ONE WILL YOU PREFER?
 80 100 120 140 160 180 200 220 240 260

55 65 79 80 102 110 120 135 137 150

Weekly family
consumption
expenditure Y, $ 60 70 84 93 107 115 136 137 145 152
 
65 74 90 95 110 120 140 140 155 175
 
70 80 94 103 116 130 144 152 165 178
 
 
75 85 98 108 118 135 145 157 175 180

– 88 – 113 125 140 – 160 189 185

– – – 115 – – – 162 – 191

Total 325 462 445 707 678 750 685 1043 966 1211

Conditional means 65 77 89 101 113 125 137 149 161 173

of Y E (Y | X )
The dark circled points in the figure below show the conditional
mean values of Y against the various X values. If we join these
conditional mean values, we obtain what is known as population
regression line or population regression curve

200
$

E(Y | X )
e x p e n d itu re ,

150
c o n s u m p tio n

100
Weekly

50
80 100 120 140 160 180 200 220 240 260
Weekly i n c o m e , $
 THE CONCEPT OF POPULATION REGRESSION
FUNІCTION (PRF)

•  
where and are unknown but fixed parameters known as the
regression coefficients; and are also known as intercept and
slope coefficients, respectively. The equation above itself is
known as the linear population regression function. Some
alternative expressions used in the literature are linear
population regression model or simply linear population
regression. In the sequel, the terms regression, regression
equation, and regression model will be used synonymously.In
regression analysis our interest is in estimating the PRFs like
t he one above, that is, estimating the values of the
unknowns β and β on the basis of observations on Y and X.
1 2
 The meaning of the term linear

•  
Linearity in the variables
The conditional expectation of Y is a linear of

The above equation is linear in variable because is of power

one. But the following regression function is not linear in the
variable because the power of of the power two.

Linearity in the parameters

 The meaning of the term linear

•  
Linearity in the parameters
The second interpretation of linearity is that the conditional
expectation of Y, E(Y | X ), is a linear function of the
i

parameters, the β ’s; it may or may not be linear in the

variable X. In this interpretation E(Y | X ) = β + β X is a linear
7 i 1 2 2

(in the parameter) regression model. Now consider the model

E(Y | X ) = β + β X . Now suppose X = 3; then we obtain E(Y |
i 1 2 i

X ) = , which is nonlinear in the parameter β . The preceding

i 2

model is an example of a nonlinear (in the parameter)

regression model.
 The meaning of the term linear

Linearity in the parameters

Therefore, from now on the term “linear” regression will
always mean a regression that is linear in the parameters; the
β ’s (that is, the parameters are raised to the first power
only). It may or may not be linear in the explanatory
variables.
 The meaning of the term linear

•  
Linearity in the parameters
How to transform a non linear regression model into linear
regression model.

A model that can be made linear in parameter is called an

intrinsically linear regression model.
 Stochastic Specification for PRF
•  
E(Y | X ) i

E(Y | X )+i

If we take the expected value of both side

E[ E(Y | X )+( |)]
i

E(Y | X )+E( |)] because the expected of constant is that

i

constant itself
 Stochastic Specification for PRF
The significance of stochastic disturbance term
1. Vagueness of theory
2. Unavailability of data
3. Core variable versus peripheral variables
4. Intrinsic randomness in human behavior
5. Poor proxy variables
6. Principle of parsimony
7. Wrong functional form
 The Sample Regression Function (SRF)
• By confining our discussion so far to the population
of Y values corresponding to the fixed X’s, we have
deliberately avoided sampling considerations.
• It is about time to face up to the sampling problems,
for in most practical situations what we have is but a
sample of Y values corresponding to some fixed X’s.
• Therefore, our task now is to estimate the PRF on
the basis of the sample information.
 The Sample Regression Function (SRF)

As an illustration, pretend that the population in

the table in the slide above was not known to us
and the only information we had was a randomly
selected sample of Y values for the fixed X’s as
given in the table in the next slide.
The Sample Regression Function (SRF)
Y X Y X
 
70 80
65 100
90 120
95 140
110 160
115 180
120 200
140 220
155 240
150 260
The Sample Regression Function (SRF)
200
SRF2
× First sample (Table 2.4) Regression based on
Second sample (Table 2.5) the second sample SRF1
Weekly consumption expenditure, $
×
150

× Regressio n based on
× the first sample
×
100 × ×

× ×
50

80 100 120 140 160 180 200 220 240 260

Weekly income,
 The Sample Regression Function (SRF)

•   us develop the concept of the sample regression function

Let
(SRF) to represent the sample regression line.

Where is read as Y-hat

= estimator of
= estimator of
= estimator of
Note that an estimator, also known as a (sample) statistic, is
simply a rule or formula or method that tells how to estimate the
population parameter from the information provided by the
sample at hand
 Y
SRF: Yi = β1 + β 2 X i
Y i
Y i
$

ui
PRF: E(Y | Xi)= β1 + β 2 X
e x p e n d itu re ,

i
u i
Y i
Yi
c o n s u m p tio n

E(Y | X i )
E(Y | X i )
Weekly

X
X i
Weekly i n c o m e , $
 The Sample Regression Function (SRF)

•Formula
  for calculating disturbance or error
term and residual.
Disturbance or error term
Residual
 The Sample Regression Function (SRF)

•To  sum up, then, we find our primary objective

in regression analysis is to estimate the PRF

on the basis of the SRF

because more often than not our analysis is

based upon a single sample from some
population.