Introduction to

Simple
Regression
Unit 3
Prof TJ Mosikari
Learning outcomes

• Interpret results of a regression model

• Use the OLS method to estimate a regression model in Eviews and to

interpret the results.

• State the classical linear regression model assumptions

Introduction
• The correlation measures the degree to which a set of data points
form a straight line relationship.
• Regression is a statistical procedure that determines the equation for
the straight line that best fits a specific set of data.
• Most problems in economics are of a similar level of complexity.
Unfortunately, the basic tool you have encountered so far – simple
correlation analysis – cannot handle such complexity. For more
complex cases – those involving more than two variables – regression
is the tool to use.
Regression model
• Relation between variables where changes in some
variables may “explain” or possibly “cause” changes in
other variables.
• Explanatory variables are termed the independent
variables and the variables to be explained are termed the
dependent variables.
• Regression model estimates the nature of the relationship
between the independent and dependent variables.
• Change in dependent variables that results from changes in
independent variables, ie. size of the relationship.
• Strength of the relationship.
• Statistical significance of the relationship.
Examples
• Dependent variable is retail price of gasoline in RSA – independent
variable is the price of crude oil.
• Dependent variable is employment income – independent variables
might be hours of work, education, occupation, sex, age, region, years
of experience, unionization status, etc.

• Price of a product and quantity produced or sold:

• Quantity sold affected by price. Dependent variable is quantity of
product sold – independent variable is price.
• Price affected by quantity offered for sale. Dependent variable is
price – independent variable is quantity sold.
 Bivariate or simple linear
regression
• x is the independent variable
• y is the dependent variable
• The regression model is
y   0  1 x  
• The model has two variables, the independent or explanatory variable,
x, and the dependent variable y, the variable whose variation is to be
explained.
• The relationship between x and y is a linear or straight line relationship.
• Two parameters to estimate – the slope of the line β1 and the y-
intercept β0 (where the line crosses the vertical axis).
• ε is the unexplained, random, or error component.
Relationships
• Economic theory specifies the type and structure of relationships that
are to be expected.
• Historical studies.
• Studies conducted by other researchers – different samples and
related issues.
• Speculation about possible relationships.
• Correlation and causation.
• Theoretical reasons for estimation of regression relationships;
empirical relationships need to have theoretical explanation.
Uses of regression
• Amount of change in a dependent variable that results from changes
in the independent variable(s) – can be used to estimate elasticities,
returns on investment in human capital, etc.
• Attempt to determine causes of phenomena.
• Prediction and forecasting of sales, economic growth, etc.
• Support or negate theoretical model.
• Modify and improve theoretical models and explanations of
phenomena.
 Summer Income as a Function of Hours Worked

30000

25000

20000
Income

15000

10000

5000

0
0 10 20 30 40 50 60
Hours per Week
yˆ  2461  297 x

R2 = 0.311
Significance = 0.0031
 ORDINARY LEAST SQUARES (OLS)

• How do we estimate the PRF?

• There are many methods used to estimate the parameters of the
population e.g.
– Maximum likelihood
– The method of moment
– OLS (most popular method used in literature )

ECON321
 OLS
• OLS relies on the idea to
select a line that
represents an average
relationship of the
observed data
• OLS regression line is
placed in such a way that
sum of squared distances
between the dots and
regression line become
small as possible.
ECON321
 OLS
• Purpose of regression analysis is to take a purely
theoretical equation such as

• and use a set of data to create an estimated equation

like:
–i
– hat indicates a sample estimate of the true
population value.
• The purpose of estimation technique is to obtain values
for the coefficients of an otherwise completely
theoretical regression equation.
ECON321
 OLS
• OLS is a technique that calculates the so as to minimize the Sum of
the Squared Residuals, i.e.
– OLS minimizes (I = 1,2,3……..,N).
• The summation symbol, means that all terms to its right should be
added (summed) over the range of the i values attached to the
bottom and top of the symbol
• From the above equation it means
– =
ECON321
 ESTIMATING THE PARAMETERS – OLS
• If the Simple Regression Function (SRF) is
• Yˆi b1  b2 X i  ei
• You can re-write it as ei Yi  Yˆi Yi  b1  b2 X i
• OLS chooses b1 and b2 in such a way that the residual
sum of squares is as small as possible.
Minimise  ei  Yi  b1  b2 X i 
2 2

• See the derivation of the estimators of b1 and b2.

• Characteristics of the OLS estimators (see unit 5)
ECON321
 PUTTING IT ALL TOGETHER
• OLS estimation gives the following sample Lotto
expenditure regression: Ŷi = 7.6182 + 0.0814Xi

o The slope coefficient means that if PDI per week goes up by a

Rand, the mean weekly expenditure on Lotto goes up by about
8 cents.
o The intercept means that if PDI were zero, the mean
expenditure on Lotto will be about R7,62 per week.

ECON321
Class exercise

The unit3data.XLS data set contains data on GDP (Y), GNS

(X), IMP (W) and EPT (Z).
(a) Run a regression of Y on X and interpret the results.
(b) Run a regression of Y on W and one of Y on Z and interpret the
results.
(c) Create a new variable, GDP per capita
(d) Run a regression of W on GDP per capita
(e) Create a new variable, trade balance and name TB
(f) Run a regression of Y on trade balance
(g) Create a new variable, Budget balance
(h) Run a regression of Y on budget balance

GNS= gross national savings POP= population GR= government

revenue
IMP= imports of goods GE= government expenditure
EPT= exports of goods