Topic 3b Bivariate Regression Model

Dr. Olalekan Aworinde

Department of Economics
Babcock University
Ilishan-Remo

These presentation notes are based on Theory of Econometrics Second Edition by

A. Koutsoyiannis
Simple Linear Regression Model- Ordinary Least Squares
Method (OLS)

The OLS is also referred to as the classical least squares (CLS) and it
has the following advantages;
Firstly, the parameter estimates obtained by OLS have some optimal
properties.
Secondly, the computational procedure of OLS is fairly simple.
Thirdly, OLS has been applied in a wide range of economic
relationship with fairly satisfactory results.
Fourthly, the mechanics of OLS are simple to understand.
Fifthly, OLS is an essential component of most other econometrics
techniques.
The Simple Linear Regession Model

The …rst stage of any econometric research is economic theory. For

example, we will use the theory of supply. The theory states that "the
higher the price the quantity supplied and vice versa.
The next stage is the speci…cation of the model. In specifying the
model we have to know the number of equations of the model and
their precise mathematical form. In this class, we will concentrate on
single equation for now.
The next thing is the a-priori expectation of the model and this deals
with the signs and magnitude of the parameter estimates.
The dependent variable is called the regressand and the
independent/explanatory variable is called the regressors.
Model Speci…cation

Y = f (X )

where Y = quantity supplied and X = Price of the commodity.

Yi = b0 + b1 Xi

The above speci…cation shows that price is the only factor in‡uencing
the level of quantity supplied and this implies that there is a one way
causation between variables Y and X.
Our aim is to estimate the numerical values of b̂0 and b̂1 .
b̂0 = 0 it implies that quantity supplied is zero when price is zero.
If b̂0 > 0 it means that some quantity were supplied when the price
drops or falls.
If b̂0 < 0 ignore this because a negative quantity does not make sense
in economics.
Model Speci…cation (Cont’d)
The sign of b̂0 is essential in determining the price elasticity of supply.
b̂1 > 0 because it is expected to be expected to be positive since a
supply curve is upward sloping.
dQ P dY X
Recall that ηp = . = .
dP Q dX Y
dY
From the supply function = b1
dX
To compute the elasticity from a regression line, we use the estimate
of b̂1 and the mean values of price (X̄) and quantity (Ȳ)
X̄
) ηp = b̂1 .
Ȳ
Ȳ = b̂0 + b̂1 X̄
b̂1 X̄
) ηp =
b̂0 + b̂1 X̄
Model Speci…cation (Cont’d)

Given that b̂1 > 0 it follows that

Supply will be elastic (ηp > 1) if b̂0 is negative (b̂0 < 0)
Supply will be inelastic (ηp < 1) if b̂0 is positive (b̂0 > 0)
Supply is unitary (ηp = 1) if b̂0 = 0
Econometric Model
The above model shows that the relationship between variables Y and
X are exact. However, there are other factors that a¤ect Y asides
from X, thus the relationship are inexact and of the form;

Yi = b0 + b1 Xi + µ

Where µ is called the error term or random disturbance term or

stochastic error term and it represents deviations from the line which
may be attributed to some factors and they are;
Ommision of varaibles from the function
Random behaviour of human beings
Imperfect speci…cation of the mathematical form of the model
Error of aggregation
error of measurement (Undercasting/Overcasting error)
The …rst four errors is called error of equation/error of ommission and
the last error is called error of observation.
Explained and Unexplained Variation

Y
Y we're trying to
predict ^β 11X ii
^
^ = β 00 +
ε Expected
(Mean) Y
Y ii

E(Y) = β 0 + β 1X

Prediction, ^
Y
X
XP
Explained and Unexplained Variation (Cont’d)

Yi = b0 + b1 Xi +µ
| {z } | {z } |{z}
[Variations [Systematic/ [Random
in Yi ] Explained Unexplained
Variation] Variation]
The second component in the bracket is the variations in Y explained
by changes in X and the third bracket is the variation not explained
by any speci…c factor.
Assumptions of Error Term

µi is a random real variable which may be positive, negative or zero.

The mean value of µi in any particular period is zero
The variance of µi is constant in each period
The variable µi has a normal distribution
The random term of di¤erent observations (µi , µj ) are independent
µi is independent of the explanatory variables
The explanatory variables are measured without error
The explanatory variables are not perfectly linearly correlated
The macrovariables should be correctly aggregated
The relationship being estimated is speci…ed
The Normal Equations of OLS

The true relationship is Yi = b0 + b1 Xi

The estimated relationship Yi = b̂0 + b̂1 Xi + ei
The estimated regression line Ŷi = b̂0 + b̂1 Xi
ei = Yi Ŷi
Substituting Ŷi we have ei = Yi b̂0 b̂1 Xi

Squaring both sides and taking the sum we have;

n n
Σ ei2 = Σ (Yi b̂0 b̂1 Xi )2
i =1 i =1

The sum of squared residual deviations is to be minimized with

respect to b̂0 and b̂1 so as to have the normal equations.
The Normal Equations of OLS (Cont’d)

The necessary condition is that the …rst order derivatives be equal to

zero that is;

∂Σe 2 ∂Σe 2
= 0 and =0
∂b̂0 ∂b̂1
Using Chain rule; if y = f (w ) and w = f (x )
dy dy dw
) = .
dx dw dx
Where w = Σ(Yi b̂0 b̂1 Xi )
∂Σe 2 ∂w
Σe 2 = w 2, ) = 2w , = 1
∂w ∂b̂0
∂Σe 2
= 2Σ(Yi b̂0 b̂1 Xi ). ( 1) = 0
∂b̂0
ΣYi = nb̂0 + b̂1 ΣXi
The Normal Equations of OLS (Cont’d)

∂Σe 2 ∂w
= 2w , = Xi
∂w ∂b̂1
∂Σe 2
= 2Σ(Yi b̂0 b̂1 Xi ). ( Xi ) = 0
∂b̂1
Σ(Xi Yi b̂0 Xi b̂1 Xi2 ) = 0
ΣXi Yi = b̂0 ΣXi b̂1 ΣXi2
The normal equation gives; ΣYi = nb̂0 + b̂1 ΣXi (1)
ΣXi Yi = b̂0 ΣXi b̂1 ΣXi2
Expressing in matrix of the form AX = B
n ΣXi b̂0 ΣYi
=
ΣXi ΣXi2 b̂1 ΣXi Yi
The Normal Equations of OLS (Cont’d)

Using Crammer’s Rule we can solve for b̂0 and b̂1

ΣX 2 ΣY ΣX ΣXY
b̂0 =
nΣX 2 (ΣX )2
nΣXY ΣX ΣY
b̂1 =
nΣX 2 (ΣX )2

The above formulae are expressed in terms of the original sample

observation on X and Y. In deviations form;

b̂0 = Ȳ b̂1 X̄
Σxi yi
b̂1 =
Σxi2
Recall that Σxi yi = fnΣXY (ΣX )(ΣY )g/n
Σxi2 = fnΣX 2 (ΣX )2 g/n
Numerical Values of the Correlation Coe¢ cient

Substituting the expression for b̂1 we have;

Σxi yi fnΣXY (ΣX )(ΣY )g/n nΣXY (ΣX )(ΣY )

b̂1 = = =
Σxi 2 f nΣX 2 ( ΣX ) 2 g /n nΣX 2 (ΣX )2
To get b̂0 dividing the …rst normal equation by n we have
ΣYi = nb̂0 + b̂1 ΣXi
ΣYi nb̂0 b̂1 ΣXi
= +
n n n
Ȳ = b̂0 + b̂1 X̄ =) b̂0 = Ȳ b̂1 X̄
Example

Using the data in our previous example on the relation between price
and quantity supplied estimate the regression line.
n = 10, ΣX = 110, ΣY = 610, Σx 2 = 330, Σxy = 1810, ΣX 2 =
1540, ΣXY = 8520

ΣX 2 ΣY ΣX ΣXY 1540(610) 110(8520)

b̂0 = = = 0.666
nΣX 2 (ΣX )2 10(1540) (110)2
nΣXY ΣX ΣY 10(8520) 610(110)
b̂1 = = = 5.484
nΣX 2 (ΣX ) 2 10(1540) (110)2
Σxi yi 1810
b̂1 = = = 5.484
Σxi2 330
b̂0 = Ȳ b̂1 X̄ = 61 5.484(11) = 0.666
The estimated supply function gives;
Ŷ = 0.666 + 5.484Xi
Estimation of Supply Elasticity from the Regression Line

X̄
) ηp = b̂1 .
Ȳ
11
5.484 = 0.989
61
ηp is inelastic.