Applied econometrics for

ManageMent (MgMt MSC 5411)

Chapter two:
RegRession AnAlysis
PART-I:
Simple linear regreSSion
AnAlysis
Teklebirhan Alemnew (Assistant Professor)
[email protected]
AAU, 2024
1
 2.1. Introduction
 As you know, economic theories are mainly concerned with the
relationships between variables.

 These relationships can be stated in mathematical terms which

show the functional relationship of variables.

 The functional relationships of these variables define the

dependence of one variable upon the other variable(s) in the
specific functional form.

 The specific functional forms may be linear, quadratic,

logarithmic, exponential, or any other form.
By: Teklebirhan A. 2
 Cont…
 The two types of regression analysis are,

 Simple Linear regression analysis known as two variables

regression in which the dependent variable is linearly related to a
single explanatory variable.

 Multiple Linear regression analysis in which the regressand is

related to two or more regressors.

By: Teklebirhan A. 3
 2.2. The Concept of Regression Analysis
 The main goal of any econometric analysis is to establish an
acceptable empirical causal relationship between variables.

 Regression analysis is concerned with the study of the dependence

of one variable (the dependent variable) on one or more other
variables (the explanatory variable(s)).

 In other words, Regression analysis is concerned with describing

and evaluating the relationship between a given variable.

 The objective of regression analysis is to estimate and/or predict

the unknown (population) mean value of the dependent variable in
terms of the known values of the explanatory variables. 4
By: Teklebirhan A.
 Cont…
 For instance, a researcher may be interested in studying the
dependence of household’s saving behavior on household annual
disposable income.

 That is, our concern might be with predicting the average

Household’s amount of saving knowing household annual
disposable income.

 Such an analysis is helpful in estimating the marginal propensity to

save (MPS), that is, average change in Household’s amount of
saving for, say, a unit change in disposable income.

By: Teklebirhan A. 5
 Cont…
.
3000
2000
1000

Regression Line
0

0 5000 10000 15000

Household's Monthly Income (in ETB)

Household's Monthly Saving (in ETB) Fitted values

 The line that passes through the average level of saving for each
level of household income is known as the regression line. It shows
how the average saving increases with the household’s income.
6
By: Teklebirhan A.
 Cont…

What is the difference between

Regression and Correlation
Analysis?

By: Teklebirhan A. 7
 Cont…
 In addition, regression analysis is closely related to correlation
analysis but conceptually there is huge difference

 Statistical relationships (Regression analysis) by themselves

cannot logically imply causation.
 To ascribe causality, one must appeal to ‘a priori’ or theoretical
considerations.
 The primary objective of correlation analysis is to measure the
strength or degree of linear association between two variables.

 However, in regression analysis, we try to predict the average

value of the dependent variable on the basis of fixed values of the
explanatory variables. By: Teklebirhan A. 8
 Cont…
 Terminologies in Regression Theory
 The variables in a regression relation consist of dependent and
explanatory variables.

 The dependent variable is the variable whose variation is being

explained by the other variable(s).

 The explanatory variable is the variable whose variation is used to

explain the variation in the dependent variable.

 The following is a representative list of the various terminologies

used in regression analysis:

By: Teklebirhan A. 9
 Cont…
Dependent Variable Explanatory Variable
Explained variable Independent variable
Predictand Predictor
Regressand Regressor
Response Stimulus
Endogenous Exogenous
Outcome Covariate
Controlled variable Control variable

By: Teklebirhan A. 10
 Cont…
 Note that Regression analysis can be simple or multiple depending
on the number of variables included in the analysis.

 That is, if we are studying the dependence of one variable on only a

single explanatory variable, such as the dependence of household’s
saving on the level of real income, such a study is known as simple,
or two-variable, regression analysis.

 However, if we are studying the dependence of one variable on

more than one explanatory variable, such as the dependence of
money demand on disposable rate, inflation rate, liquidity ratio,
and etc, it is known as Multiple Regression Analysis. By: Teklebirhan A. 11
Types of Regression Models

By: Teklebirhan A. 12
 2.3. Population Regression Function Versus
Sample Regression Function
 Population Regression Function (PRF)
 The economic theory of saving (in its simplest form) can be
modeled as stochastic of the following form:

 The econometrics model given in the above is called population

regression model or, simply, the population model.

 This population regression model is called the true relationship

because Y, X and U represent their respective population values,
and α and β are called the true parameters.
By: Teklebirhan A. 13
 Cont…

(Household’s Saving)

By: Teklebirhan A. 14
Cont…

By: Teklebirhan A. 15
Cont…

By: Teklebirhan A. 16
 Cont…

 Rearranging the above equation, we obtain the Population

regression function (PRF) in its stochastic form given by:

By: Teklebirhan A. 17
2.4. Methods of Estimation: The Classical Simple Linear
Regression Analysis

Saving
( Saving )

By: Teklebirhan A. 18
 Cont…
 Specifying the model is the first stage of any econometric
application. The next step is the estimation of the numerical values
of the parameters of economic relationships.

 The parameters of the simple linear regression model can be

estimated by the three most commonly used estimation methods:
1. Ordinary Least Square Method (OLS)
2. Method of Moments (MM)
3. Maximum Likelihood Method (MLM)
 But, having some desirable properties (property of linearity,
unbiasedness, and minimum variance), OLS method is the most
popular method to estimate regression parameters.
By: Teklebirhan A. 19
Cont…

By: Teklebirhan A. 20
 2.4.1. The Basic Assumptions of the Classical Linear
Regression Analysis (OLS) to estimate SLRM & MLRM
 The method of OLS is attributed to Carl Friedrich Gauss, a
German Mathematician.

 OLS is an econometric method used to derive estimates of the

parameters of economic relationships from statistical observations.

 However, it works under some restrictive assumptions.

 The most important of these assumptions are discussed below.

By: Teklebirhan A. 21
 Cont…

A model is termed as linear if it is linear in parameters

By: Teklebirhan A. 22
 Cont…
 This assumption implies that the values of Y corresponding to
various values of X have constant variance.

By: Teklebirhan A. 23
 Cont…

 This assumption is required mainly for hypothesis testing (inference).

By: Teklebirhan A. 24
Cont…

By: Teklebirhan A. 25
u

Cont…
i

By: Teklebirhan A. 26
Cont…

By: Teklebirhan A. 27
 Cont…
9) No model specification error :The econometric model is correctly
specified
 No omission of relevant variable(s),
 No inclusion of unnecessary variable(s),
 Absence of adoption of wrong functional form.
 If not, OLS estimators will be biased & inconsistent
10) Variability in the values of X
 The ‘X’ values in a given sample must not all be the same.
11) Absence of high multi-collinearity among explanatory variables
(specific to Multiple regression models – Chapter 3)
 There is no perfect linear relationship among the explanatory
variables - not perfectly correlated with each other
By: Teklebirhan A. 28
 Cont…

 NB:

 Without the realization of these assumptions, the

application of OLS results would be misleading.

By: Teklebirhan A. 29
2.4.2. Estimation of SLRM by Ordinary Least
Square (OLS) Method

By: Teklebirhan A. 30
 Cont…

Given the Saving function

By: Teklebirhan A. 31
 Cont…
Obs

1. 135 700 94500 490000 7 -124 -868 15376 49

2. 120 800 96000 640000 -8 -24 192 576 64

3. 130 890 115700 792100 2 66 132 4356 4

4. 130 654 85020 427716 2 -170 -340 28900 4

5. 95 500 47500 250000 -33 -324 10692 104976 1089

6. 158 1400 221200 1960000 30 576 17280 331776 900

494
Sum 768 659920 4559816 0 0 27088 485960 2110
4

By: Teklebirhan A. 32
Cont…

By: Teklebirhan A. 33
 2.5. Alternative Functional Forms and
Interpretation of OLS Estimates for SLRM

By: Teklebirhan A. 34
Cont…

By: Teklebirhan A. 35
 Cont…

0.056
0.056

By: Teklebirhan A. 36
Cont…

By: Teklebirhan A. 37
Cont…

By: Teklebirhan A. 38
Cont…

By: Teklebirhan A. 39
Cont…

By: Teklebirhan A. 40
Cont…

By: Teklebirhan A. 41
Cont…

By: Teklebirhan A. 42
Cont…

By: Teklebirhan A. 43
Cont…

By: Teklebirhan A. 44
Cont…

By: Teklebirhan A. 45
 Cont…

Model If X increases by Then Y will change by

Linear 1 unit
Linear-Log 1%
Log-Linear 1 unit
Log-Log 1%

By: Teklebirhan A. 46
2.7. Decomposition of the Variation of Y and
“Goodness of Fit” of an Estimated Model

By: Teklebirhan A. 47
 Cont…

By: Teklebirhan A. 48
 Cont…

By: Teklebirhan A. 49
 Cont…

By: Teklebirhan A. 50
 Cont..

By: Teklebirhan A. 51
 Cont..

By: Teklebirhan A. 52
 Cont..

By: Teklebirhan A. 53
 2.8. Evaluation of an Estimated Model for SLRM
& MLRM
 After estimation of a model, the next stage is to evaluate the
estimated model.

 By evaluation of the model means examining the ‘goodness’ of an

estimated model.

 To judge on the ‘goodness’ of an estimated econometrics model,

there are three criteria. These are

 Economic criterion,

 Statistical criterion (First order test) and

 Econometric criterion (Second Order Tests).

By: Teklebirhan A. 54
 2.8.1.Econometric Criterion: Statistical Desirable
Properties of OLS Estimators and the Gauss-Markov
Theorem
 There are traditional criteria based on which the closeness of an
estimate to the true population parameter can be determined.

 These are called desirable properties of Estimators (or estimates).

 Desirable properties of estimators are two categories:

1) Finite (small sample) desirable properties of estimators and

2) Infinite (large sample) or asymptotic properties of estimators.

By: Teklebirhan A. 55
 Cont…
1. Finite (Small Sample) Properties of Estimators.

The desirable attributes of estimators under smaller sample sizes

are: = a) + b)
a)Unbiasedness
b)Minimum variance
c)Efficiency Estimator
d)Minimum mean square error (MMSE)
e)Linearity Estimator
f)Best, linear, unbiased Estimator (BLUE) - Gauss-Markov Theorem
 An estimator is called BLUE if: linear, unbiased & Minimum
variance
By: Teklebirhan A. 56
Cont…

By: Teklebirhan A. 57
 Cont…
2) Large-Sample (Asymptotic) Properties of Estimators

 It often happens that an estimator does not satisfy one or more of

the desirable statistical properties in small samples.

 But as the sample size increases indefinitely, the estimator

possesses several desirable statistical properties.

 These properties are known as the large-sample, or asymptotic,

properties.

By: Teklebirhan A. 58
 Cont…
 Asymptotic (large sample) desirable properties of estimators are:

 Asymptotic unbiasedness

 Consistency (unbiased + Variance tends to zero as ‘n’ increase)

 Asymptotic efficiency (consistent + min variance)

By: Teklebirhan A. 59
 2.8.2. Statistical Inference: Statistical Test of
Significance of OLS Estimators (First Order Tests)
 In this section, we shall develop statistical criteria for the
evaluation of an estimated model.

 Statistical criteria are developed based on statistical and

probability theories.

 The application of statistical criteria to judge on the goodness of a

model is known as tests of the statistical significance (TSS) or first
order tests of a model.

By: Teklebirhan A. 60
Cont…

By: Teklebirhan A. 61
Cont…

By: Teklebirhan A. 62
Cont…

By: Teklebirhan A. 63
Cont…

By: Teklebirhan A. 64
Cont…

By: Teklebirhan A. 65
Cont…

By: Teklebirhan A. 66
 Cont…
 Thus, with these critical values the rejection and acceptance
regions for the null-hypothesis will be:

By: Teklebirhan A. 67
Cont…

By: Teklebirhan A. 68
Cont…

By: Teklebirhan A. 69
Cont…

By: Teklebirhan A. 70
Cont…

By: Teklebirhan A. 71
Cont…

By: Teklebirhan A. 72
 Cont…
 In statistics, the process of estimating an interval of values
between which the true values of the population parameters are
expected to lie based on the sampling distribution of the sample
estimates is called interval estimation.

 It can be done depending on the sample size;

1) Confidence interval from the Standard Normal Distribution (Z-

Distribution)

2) Confidence interval from the Student’s t-distribution.

By: Teklebirhan A. 73
 Cont…
 Confidence interval from the Standard Normal Distribution (Z-
Distribution)

 The meaning of this confidence interval is that there is 95%

chance for this interval to contain the true value of the unknown
parameter β within its range. 74
By: Teklebirhan A.
2.9. Prediction using Simple Linear
Regression Model

By: Teklebirhan A. 75
Cont…

1200 ETB?

By: Teklebirhan A.
76
 Cont…
 Reporting the Results of Regression Analysis

 The results of the regression analysis derived are reported in

conventional formats.

 It is not sufficient merely to report the estimates of β’s.

 There are two conventional ways to report a regression result:

a) Equation form, i.e., by fitting the estimated coefficients in to the

regression model and

b) Table form

By: Teklebirhan A. 77
Cont…

By: Teklebirhan A. 78
 Cont…
b) Table Form
 In this case, the estimated coefficients, the corresponding t-
statistics, and some other indicators are presented in tabular form.
 Example: The estimated regression result of our saving function
can be presented using table as follows:
Household's Monthly Saving (in ETB)
Household's Monthly Income (in ETB) 0.0557*
(3.17)

Constant 82.07**
(5.36)
Observations 6
R2 0.716

t statistics in parentheses
* p < 0.05, ** p < 0.01, *** p < 0.1

By: Teklebirhan A. 79