Econometrics

Javed Anwar
Basic Econometrics

Introduction:
What is
Econometrics?

2
 Introduction
What is Econometrics?
 Definition 1: Economic
Measurement

 Definition 2: Application of the

mathematical statistics to economic data in order
to lend empirical support to the economic
mathematical models and obtain numerical results

(Gerhard Tintner, 1968)

3
 Introduction
What is Econometrics?

 Definition 3: The quantitative

analysis of actual economic phenomena based on
concurrent development of theory and
observation, related by appropriate methods of
inference

(P.A.Samuelson, T.C.Koopmans and

J.R.N.Stone, 1954)

4
 Introduction
What is Econometrics?
 Definition 4: The social science
which applies economics, mathematics and
statistical inference to the analysis of economic
phenomena (By Arthur S. Goldberger,
1964)
 Definition 5: The empirical
determination of economic laws (By H.

Theil, 1971)
5
 Introduction
What is Econometrics?
 Definition 6: A conjunction of economic theory
and actual measurements, using the theory and
technique of statistical inference as a bridge pier
(By T.Haavelmo, 1944)

 And the others

6
Economic Mathematical
Theory Economics

Econometrics

Economic Mathematic
Statistics Statistics
7
 Introduction
Why a separate discipline?
 Economic theory makes
statements that are mostly qualitative in
nature, while econometrics gives empirical
content to most economic theory

 Mathematical
economics is to express economic
theory in mathematical form without empirical
verification of the theory, while econometrics is
mainly interested in the later

8
 Introduction
Why a separate discipline?
 Economic Statistics is mainly
concerned with collecting, processing and
presenting economic data. It does not being
concerned with using the collected data to test
economic theories

 Mathematical statistics
provides many of tools for economic studies, but
econometrics supplies the later with many
special methods of quantitative analysis based
on economic data

9
Economic Mathematical
Theory Economics

Econometrics

Economic Mathematic
Statistics Statistics
10
 Introduction
Methodology of
Econometrics
(1)Statement of theory
or hypothesis:

Keynes stated: ”Consumption increases

as income increases, but not as much as
the increase in income”. It means that
“The marginal propensity to consume
(MPC) for a unit change in income is
grater than zero but less than unit”

11
 Introduction
Methodology of
Econometrics
(2) Specification of the
mathematical model of
the theory
Y = ß 1+ ß2X ; 0 < ß 2< 1
Y= consumption
expenditure
X= income
ß1 and ß2 are parameters; ß1 is
intercept, and ß2 is slope coefficients
12
 Introduction
Methodology of
Econometrics
(3) Specification of the
econometric model of the
theory
Y = ß1+ ß2X + u ; 0 < ß2< 1;
Y = consumption
expenditure;
X = income;
ß1 and ß2 are parameters; ß1is intercept and ß2 is
slope coefficients; u is disturbance term or error
term. It is a random or stochastic variable

13
 Introduction
Methodology of
Econometrics

(4) Obtaining Data

(See Table 1.1, page 6)

Y= Personal consumption
expenditure
X= Gross Domestic Product
all in Billion US Dollars

14
 Introduction
Methodology of
Econometrics
(4) Obtaining Data
Year X Y

1980 2447.1 3776.3

1981 2476.9 3843.1
1982 2503.7 3760.3
1983 2619.4 3906.6
1984 2746.1 4148.5
1985 2865.8 4279.8
1986 2969.1 4404.5
1987 3052.2 4539.9
1988 3162.4 4718.6
1989 3223.3 4838.0
1990 3260.4
15 4877.5
1991 3240.8 4821.0
 Introduction
Methodology of
Econometrics
(5) Estimating the
Econometric Model
Y^ = - 231.8 + 0.7194 X (1.3.3)
MPC was about 0.72 and it means that for the
sample period when real income increases 1
USD, led (on average) real consumption
expenditure increases of about 72 cents

Note: A hat symbol (^) above one variable will

signify an estimator of the relevant population
value
16
 Introduction
Methodology of Econometrics
(6) Hypothesis Testing
Are the estimates accord with
the
expectations of the theory that
is being
tested? Is MPC < 1 statistically?
If so,
it may support Keynes’ theory.
Confirmation or refutation of
economic theories
17 based on
 Introduction
Methodology of
Econometrics
(7) Forecasting or
Prediction
 With given future value(s) of X, what is the
future value(s) of Y?
 GDP=$6000Bill in 1994, what is the forecast
consumption expenditure?
 Y^= - 231.8+0.7196(6000) = 4084.6
 Income Multiplier M = 1/(1 – MPC) (=3.57).
decrease (increase) of $1 in investment will
eventually lead to $3.57 decrease (increase)
in income

18
 Introduction
Methodology of
Econometrics
(8) Using model for
control or
policy purposes
Y=4000= -231.8+0.7194 X  X  5882
MPC = 0.72, an income of $5882 Bill
will produce an expenditure of $4000
Bill. By fiscal and monetary policy,
Government can manipulate the
control variable X to get the desired
level of target variable Y

19
 Introduction
Methodology of Econometrics
Figure 1.4: Anatomy of
economic modelling
• 1) Economic Theory
• 2) Mathematical Model of Theory
• 3) Econometric Model of Theory
• 4) Data
• 5) Estimation of Econometric Model
• 6) Hypothesis Testing
• 7) Forecasting or Prediction
• 8) Using the Model for control or policy purposes

20
Economic Theory

Mathematic Model Econometric Model Data Collection

Estimation

Hypothesis Testing
Application
in control or
Forecasting policy
studies
21
Basic Econometrics
Chapter 1:
THE NATURE OF
REGRESSION ANALYSIS

22
1-1. Historical origin of the term
“Regression”

 The term REGRESSION was introduced by

Francis Galton
 Tendency for tall parents to have tall
children and for short parents to have
short children, but the average height of
children born from parents of a given
height tended to move (or regress)
toward the average height in the
population as a whole (F. Galton, “Family
Likeness in Stature”)

23
 1-1. Historical origin of the term
“Regression”

 Galton’s Law was confirmed by Karl

Pearson: The average height of sons of a
group of tall fathers < their fathers’
height. And the average height of sons
of a group of short fathers > their
fathers’ height. Thus “regressing” tall
and short sons alike toward the average
height of all men. (K. Pearson and A.
Lee, “On the law of Inheritance”)
 By the words of Galton, this was
“Regression to mediocrity”

24
 1-2. Modern Interpretation of
Regression Analysis
 The modern way in interpretation of
Regression: Regression Analysis is
concerned with the study of the
dependence of one variable (The
Dependent Variable), on one or more
other variable(s) (The Explanatory
Variable), with a view to estimating
and/or predicting the (population) mean
or average value of the former in term
of the known or fixed (in repeated
sampling) values of the latter.
 Examples: (pages 16-19)

29
 Dependent Variable Y; Explanatory Variable
Xs
1. Y = Son’s Height; X = Father’s Height
2. Y = Height of boys; X = Age of boys
3. Y = Personal Consumption Expenditure
X = Personal Disposable Income
4. Y = Demand; X = Price
5. Y = Rate of Change of Wages
X = Unemployment Rate
6. Y = Money/Income; X = Inflation Rate
7. Y = % Change in Demand; X = % Change in
the
advertising budget
8. Y = Crop yield; Xs = temperature, rainfall,
sunshine,
fertilizer
30
 1-3. Statistical vs.
Deterministic
Relationships

 In regression analysis we are

concerned with STATISTICAL
DEPENDENCE among variables (not
Functional or Deterministic), we
essentially deal with RANDOM or
STOCHASTIC variables (with the
probability distributions)

31
 1-4. Regression vs.
Causation:
Regression does not necessarily imply
causation. A statistical relationship
cannot logically imply causation. “A
statistical relationship, however strong
and however suggestive, can never
establish causal connection: our ideas of
causation must come from outside
statistics, ultimately from some theory
or other” (M.G. Kendal and A. Stuart,
“The Advanced Theory of Statistics”)

32
 1-5. Regression vs.
Correlation
 Correlation Analysis: the primary
objective is to measure the strength or
degree of linear association between
two variables (both are assumed to be
random)
 Regression Analysis: we try to estimate
or predict the average value of one
variable (dependent, and assumed to be
stochastic) on the basis of the fixed
values of other variables (independent,
and non-stochastic)
33
1-6. Terminology and Notation
Dependent Explanatory
Variable Variable(s)
 
Explained Variable Independent
Variable(s)
 
Predictand Predictor(s)
 
Regressand Regressor(s)
 
Response Stimulus or control
variable(s)
 
Endogenous Exogenous(es)

34
1-7. The Nature and Sources
of Data for Econometric
Analysis

1) Types of Data :
 Time series data;
 Cross-sectional data;
 Pooled data
2) The Sources of Data
3) The Accuracy of Data

35
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Time Series Data
 A time series is a set of observations on the values that a variable
takes at different times. Such data may be collected at regular time
intervals, such as
 daily (e.g., stock prices, weather reports),
 weekly (e.g., money supply figures),
 monthly [e.g., the unemployment rate, the Consumer Price Index
(CPI)],
 quarterly (e.g., GDP),
 annually (e.g., government budgets),
 quinquennially, that is, every 5 years (e.g., the census of
manufactures),
 decennially (e.g., the census of population).
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Cross-Section Data Cross-section data are data on one or more
variables collected at the same point in time, such as the census of
population conducted by the Census Bureau every 10 years (the
latest being in year 2000), the surveys of consumer expenditures
conducted by the University of Michigan, and, of course, the
opinion polls by Gallup and umpteen other organizations.
 A concrete example of cross-sectional data is given in Table 1.1
 This table gives data on egg production and egg prices for the 50
states in the union for 1990 and 1991. For each year the data on the
50 states are cross-sectional data. Thus, in Table 1.1 we have two
cross-sectional samples.
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Pooled Data
 In pooled, or combined, data are elements of both time series and
cross-section data. The data in Table 1.1 are an example of pooled
data.
 For each year we have 50 cross-sectional observations and for each
state we have two time series observations on prices and output of
eggs, a total of 100 pooled (or combined) observations.
 Likewise, the data given in exercise 1.1 are pooled data in that the
Consumer Price Index (CPI) for each country for 1973–1997 is
time series data, whereas the data on the CPI for the seven
countries for a single year are cross-sectional data. In the pooled
data we have 175 observations—25 annual observations for each of
the seven countries.
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Panel, Longitudinal, or Micro panel Data
 This is a special type of pooled data in which the same cross-
sectional unit (say, a family or a firm) is surveyed over time.
 For example, the U.S. Department of Commerce carries out a
census of housing at periodic intervals.
 At each periodic survey the same household (or the people living at
the same address) is interviewed to find out if there has been any
change in the housing and financial conditions of that household
since the last survey.
 By interviewing the same household periodically, the panel data
provides very useful information on the dynamics of household
behavior
 1-7. The Nature and Sources of Data
for Econometric Analysis
 The Sources of Data
 The data used in empirical analysis may be collected by
 a governmental agency (e.g., the Department of Commerce),
 an international agency (e.g., the International Monetary Fund (IMF) or
the World Bank),
 a private organization (e.g., the Standard & Poor’s Corporation),
 or an individual.
 Literally, there are thousands of such agencies collecting data for one
purpose or another.
 Pakistan – Data Sources
 Pakistan Economic Survey
 SBP Bulletins
 PBS Published Data
 Etc.
 1-7. The Nature and Sources of Data
for Econometric Analysis
 A Note on the Measurement Scales of Variables15
 The variables that we will generally encounter fall into four broad
categories: ratio scale, interval scale, ordinal scale, and nominal
scale. It is important that we understand each.
 Ratio Scale
 For a variable X, taking two values, X1 and X2, the ratio X1/X2 and the
distance (X2 − X1) are meaningful quantities.
 Also, there is a natural ordering (ascending or descending) of the
values along the scale.
 Therefore, comparisons such as X2 ≤ X1 or X2 ≥ X1 are meaningful.
 Most economic variables belong to this category. Thus, it is
meaningful to ask how big is this year’s GDP compared with the
previous year’s GDP.
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Interval Scale
 An interval scale variable satisfies the last
two properties of the ratio scale variable but
not the first.
 Thus, the distance between two time periods,
say (2000–1995) is meaningful, but not the
ratio of two time periods (2000/1995).
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Ordinal Scale
 A variable belongs to this category only if it satisfies the third
property of the ratio scale (i.e., natural ordering).
 Examples are grading systems (A, B, C grades) or income class
(upper, middle, lower).
 For these variables the ordering exists but the distances between the
categories cannot be quantified.
 Students of economics will recall the indifference curves between
two goods, each higher indifference curve indicating higher level of
utility, but one cannot quantify by how much one indifference
curve is higher than the others.
 1-7. The Nature and Sources of Data
for Econometric Analysis
 Nominal Scale
 Variables in this category have none of the
features of the ratio scale variables.
 Variables such as gender (male, female) and
marital status (married, unmarried, divorced,
separated) simply denote categories.
 1-8. Summary and
Conclusions
1) The key idea behind regression analysis
is the statistic dependence of one
variable on one or more other
variable(s)
2) The objective of regression analysis is
to estimate and/or predict the mean or
average value of the dependent
variable on basis of known (or fixed)
values of explanatory variable(s)

46
 1-8. Summary and
Conclusions
3) The success of regression depends on
the available and appropriate data
4) The researcher should clearly state the
sources of the data used in the
analysis, their definitions, their
methods of collection, any gaps or
omissions and any revisions in the data

47
 Basic Econometrics

Chapter 2:
TWO-VARIABLE
REGRESSION
ANALYSIS: Some basic
Ideas

48
 In Chapter 1 we discussed the concept of regression in broad
terms. In this chapter we approach the subject somewhat
formally. Specifically, this and the following two chapters
introduce the reader to the theory underlying the simplest
possible regression analysis, namely, the bivariate, or two
variable, regression in which the dependent variable (the
regressand) is related to a single explanatory variable (the
regressor).
 This case is considered first, not because of its practical
adequacy, but because it presents the fundamental ideas of
regression analysis as simply as possible and some of these
ideas can be illustrated with the aid of two-dimensional
graphs. Moreover, as we shall see, the more general multiple
regression analysis in which the regressand is related to one or
more regressors is in many ways a logical extension of the
two-variable case.
A HYPOTHETICAL EXAMPLE1
As noted in Section 1.2, regression analysis is largely
concerned with estimating and/or predicting the
(population) mean value of the dependent variable on the
basis of the known or fixed values of the explanatory
variable(s). To understand this, consider the data given in
Table 2.1. The data in the table refer to a total population
of 60 families in a hypothetical community and their
weekly income (X) and weekly consumption expenditure
(Y), both in dollars. The 60 families are divided into 10
income groups (from $80 to $260) and the weekly
expenditures of each family in the various groups are as
shown in the table. Therefore, we have 10 fixed values of X
and the corresponding Y values against each of the X
values; so to speak, there are 10 Y subpopulations.
 2-1. A Hypothetical Example

 Total population: 60 families

 Y=Weekly family consumption expenditure
 X=Weekly disposable family income
 60 families were divided into 10 groups of
approximately the same income level
(80, 100, 120, 140, 160, 180, 200, 220,
240, 260)

51
2-1. A Hypothetical Example

 Table 2-1 gives the conditional distribution

of Y on the given values of X
 Table 2-2 gives the conditional probabilities
of Y: p(YX)
 Conditional Mean
(or Expectation): E(YX=Xi )

52
 Table 2-2: Weekly family income X ($), and consumption Y ($)

X 80 100 120 140 160 180 200 220 240 260

Y
Weekly 55 65 79 80 102 110 120 135 137 150
family
consumption 60 70 84 93 107 115 136 137 145 152
expenditure 65 74 90 95 110 120 140 140 155 175
Y ($)
70 80 94 103 116 130 144 152 165 178
75 85 98 108 118 135 145 157 175 180
-- 88 -- 113 125 140 -- 160 189 185
-- -- -- 115 -- -- -- 162 -- 191

Total 325 462 445 707 678 750 685 1043 966 1211

Mean 65 77 89 101 113 125 137 149 161 173

53
2-1. A Hypothetical Example

 Figure 2-1 shows the population regression

line (curve). It is the regression of Y on X
 Population regression curve is the locus of
the conditional means or expectations of
the dependent variable for the fixed values
of the explanatory variable X (Fig.2-2)

54
2-2. The concepts of population
regression function (PRF)

 E(YX=Xi ) = f(Xi) is Population Regression

Function (PRF) or
Population Regression (PR)
 In the case of linear function we have linear
population regression function (or equation
or model)
E(YX=Xi ) = f(Xi) = ß1 + ß2Xi

57
2-2. The concepts of population
regression function (PRF)

 The dark circled points in Figure 2.1 show the conditional

mean values of Y against the various X values. If we join these
conditional mean values, we obtain what is known as the
population regression line (PRL), or more generally, the
population regression curve.
 More simply, it is the regression of Y on X. The adjective
“population” comes from the fact that we are dealing in this
example with the entire population of 60 families. Of course, in
reality a population may have many families.

58
2-2. The concepts of population
regression function (PRF)

 Geometrically, then, a population regression curve is simply

the locus of the conditional means of the dependent variable
for the fixed values of the explanatory variable(s).
 More simply, it is the curve connecting the means of the
subpopulations of Y corresponding to the given values of the
regressor X. It can be depicted as in Figure 2.2.

59
2-2. The concepts of population
regression function (PRF)

E(YX=Xi ) = f(Xi) = ß1 + ß2Xi

 ß1 and ß2 are regression coefficients, ß1is
intercept and ß2 is slope coefficient
 Linearity in the Variables
 Linearity in the Parameters

61
 THE MEANING OF THE
TERM LINEAR
 Since this text is concerned primarily with linear models like
(2.2.2), it is essential to know what the term linear really means,
for it can be interpreted in two different ways.

 E(YX=Xi ) = f(Xi) = ß1 + ß2Xi …..(2.2.2)

62
 THE MEANING OF THE
TERM LINEAR
 Linearity in the Variables
 The first and perhaps more “natural” meaning of linearity is that
the conditional expectation of Y is a linear function of Xi, such
as, for example, (2.2.2).
 Geometrically, the regression curve in this case is a straight line.
 In this interpretation, a regression function such as
E(Y | Xi) = β1 + β2Xi2
is not a linear function because the variable X appears with a power
or index of 2.

63
 THE MEANING OF THE
TERM LINEAR
 Linearity in the Parameters
 The second interpretation of linearity is that the conditional
expectation of Y, E(Y | Xi), is a linear function of the parameters,
the β’s; it may or may not be linear in the variable X.
 In this interpretation E(Y | Xi) = β1 + β2X2 i is a linear (in the
parameter) regression model. To see this, let us suppose X takes
the value 3. Therefore, E(Y | X = 3) = β1 + 9β2, which is
obviously linear in β1 and β2. All the models shown in Figure
2.3 are thus linear regression models, that is, models linear in the
parameters.

64
 2-4. Stochastic Specification of
PRF
 It is clear from Figure 2.1 that, as family income
increases, family consumption expenditure on the
average increases, too. But what about the
consumption expenditure of an individual family
in relation to its (fixed) level of income? It is
obvious from Table 2.1 and Figure 2.1 that an
individual family’s consumption expenditure does
not necessarily increase as the income level
increases

66
 2-4. Stochastic Specification of
PRF
 For example, from Table 2.1 we observe that
corresponding to the income level of $100 there is
one family whose consumption expenditure of
$65 is less than the consumption expenditures of
two families whose weekly income is only $80.
But notice that the average consumption
expenditure of families with a weekly income of
$100 is greater than the average consumption
expenditure of families with a weekly income of
$80 ($77 versus $65).
67
 2-4. Stochastic Specification of
PRF
 What, then, can we say about the relationship between an
individual family’s consumption expenditure and a given level of
income? We see from Figure 2.1 that, given the income level of Xi,
an individual family’s consumption expenditure is clustered around
the average consumption of all families at that Xi, that is, around its
conditional expectation. Therefore, we can express the deviation of
an individual Yi around its expected value as follows:
 2-4. Stochastic Specification of
PRF
 Ui = Y - E(YX=Xi ) or Yi = E(YX=Xi ) + Ui
 Ui = Stochastic disturbance or stochastic
error term. It is nonsystematic
component.
 It can take positive or negative values.
 Component E(YX=Xi ) is systematic or
deterministic. It is the mean
consumption expenditure of all the
families with the same level of income
 The assumption that the regression line
passes through the conditional means
of Y implies that E(U 69
iXi ) = 0
 2-5. The Significance of the
Stochastic
Disturbance Term
 Ui = Stochastic Disturbance
Term is a surrogate for all
variables that are omitted
from the model but they
collectively affect Y
 Many reasons why not include
such variables into the model
as follows: 70
 2-5. The Significance of the
Stochastic
Disturbance Term
 If E(Y | Xi) is assumed to be linear in Xi, as in (2.2.2), Eq. (2.4.1)
may be written as:
2-5. The Significance of the
Stochastic
Disturbance Term
 2-5. The Significance of the
Stochastic
Disturbance Term
Why not include as many as variable into the
model (or the reasons for using ui)
+ Vagueness of theory
+ Unavailability of Data
+ Core Variables vs. Peripheral Variables
+ Intrinsic randomness in human behavior
+ Poor proxy variables
+ Principle of parsimony
+ Wrong functional form

73
 2-6. The Sample Regression
Function (SRF)

 By confining our discussion so far to the population of Y values

corresponding to the fixed X’s, we have deliberately avoided
sampling considerations (note that the data of Table 2.1 represent
the population, not a sample).

 But it is about time to face up to the sampling problems, for in

most practical situations what we have is but a sample of Y values
corresponding to some fixed X’s. Therefore, our task now is to
estimate the PRF on the basis of the sample information.
 2-6. The Sample Regression
Function (SRF)

 Plotting the data of Tables 2.4 and 2.5, we obtain the scattergram
given in Figure 2.4. In the scattergram two sample regression
lines are drawn so as to “fit” the scatters reasonably well: SRF1 is
based on the first sample, and SRF2 is based on the second
sample. Which of the two regression lines represents the “true”
population regression line? If we avoid the temptation of looking
at Figure 2.1, which purportedly represents the PR, there is no
way we can be absolutely sure that either of the regression lines
shown in Figure 2.4 represents the true population regression line
(or curve). The regression lines in Figure 2.4 are known as the
sample regression lines.
2-6. The Sample Regression
Function (SRF)
 2-6. The Sample Regression
Function (SRF)

 Fig.2-4: SRF1 and SRF 2

 Y^i = ^1 + ^2Xi (2.6.1)
 Y^i = estimator of E(YXi)
 ^1 = estimator of 1
 ^2 = estimator of 2
 Estimate = A particular numerical value
obtained by the estimator in an application
 SRF in stochastic form: Yi= ^1 + ^2Xi +
u^i
or Yi= Y^i + u^i (2.6.3)
78
 2-6. The Sample Regression
Function (SRF)

 Primary objective in regression analysis is

to estimate the PRF Yi= 1 + 2Xi + ui on the
basis of the SRF Yi= ^1 + ^2Xi + ei and how
to construct SRF so that ^1 close to 1 and
^2 close to 2 as much as possible

79
2-6. The Sample Regression
Function (SRF)

 Population Regression Function PRF

 Linearity in the parameters
 Stochastic PRF
 Stochastic Disturbance Term ui plays a
critical role in estimating the PRF
 Sample of observations from population
 Stochastic Sample Regression Function
SRF used to estimate the PRF

82
 2-7. Summary and
Conclusions
 The key concept underlying regression
analysis is the concept of the population
regression function (PRF).
 This book deals with linear PRFs: linear
in the unknown parameters. They may
or may not linear in the variables.

83
 2-7. Summary and
Conclusions
 For empirical purposes, it is the stochastic
PRF that matters. The stochastic disturbance
term ui plays a critical role in estimating the
PRF.
 The PRF is an idealized concept, since in
practice one rarely has access to the entire
population of interest. Generally, one has a
sample of observations from population and
use the stochastic sample regression (SRF)
to estimate the PRF.

84
Basic Econometrics

Chapter 3:
TWO-VARIABLE
REGRESSION
MODEL:
The problem of
Estimation
85
3-1. The method of ordinary
least square (OLS)
 Least-square criterion:
 Minimizing U^2i = (Yi
– Y^i) 2
= (Yi- ^1 - ^2X)2
(3.1.2)
 Normal Equation and
solving it for ^1 and ^2 =
Least-square estimators
[See (3.1.6)(3.1.7)]
 Numerical and statistical
86
 3-1. The method of ordinary least
square (OLS)
 OLS estimators are expressed
solely in terms of observable
quantities. They are point estimators
 The sample regression line passes
through sample means of X and Y
 The mean value of the estimated
Y^ is equal to the mean value of the
actual Y: E(Y) = E(Y^)
 The mean value of the residuals
U^i is zero: E(u^i )=0
 u^i are uncorrelated
87 with the
 3-2. The assumptions
underlying the method of least
squares

 Ass 1: Linear regression model

(in parameters)
 Ass 2: X values are fixed in
repeated
sampling
 Ass 3: Zero mean value of ui :
E(uiXi)=0
 Ass 4: Homoscedasticity or equal
variance of ui : Var (uiXi) =
2
[VS. Heteroscedasticity]
 Ass 5: No autocorrelation
88 between
the
 3-2. The assumptions
underlying the method of least
squares

 Ass 6: Zero covariance between ui

and Xi
Cov(ui, Xi) = E(ui, Xi) = 0
 Ass 7: The number of observations
n must be greater than the
number of parameters to be
estimated
 Ass 8: Variability in X values. They
must not all be the
same
 Ass 9: The regression model is
correctly 89
specified
 Ass 10: There is no perfect
3-2. The Assumptions Underlying the Method of
Least Squares

90
3-2. The Assumptions Underlying the Method of
Least Squares

92
3-2. The Assumptions Underlying the Method of
Least Squares

93
3-2. The Assumptions Underlying the Method of
Least Squares

94
3-2. The Assumptions Underlying the Method of
Least Squares

95
3-2. The Assumptions Underlying the Method of
Least Squares

96
3-2. The Assumptions Underlying the Method of
Least Squares

97
3-3. Precision or standard errors of least-
squares estimates
 In statistics the precision of an
estimate is measured by its standard
error (SE)
 var( ^2) = 2 / x2i (3.3.1)
 se(^2) =  Var(^2) (3.3.2)
 var( ^1) = 2 X2i / n x2i (3.3.3)
 se(^1) =  Var(^1) (3.3.4)
 ^ 2
= u^2i / (n - 2) (3.3.5)
 ^ =  ^ 2
is standard error of the
estimate

98
3-3. Precision or standard errors of
least-squares estimates
 Features of the variance:
+ var( ^2) is proportional to 2 and inversely
proportional to x2i
+ var( ^1) is proportional to 2 and X2i but
inversely proportional to x2i and the sample
size n.
+ cov ( ^1 , ^2) = - var( ^2) shows the
independence between ^1 and ^2

99
 3-4. Properties of least-squares estimators:
The Gauss-Markov Theorem
 An OLS estimator is said to be
BLUE if :
+ It is linear, that is, a linear
function of a random variable, such
as the dependent variable Y in the
regression model
+ It is unbiased , that is, its average
or expected value, E(^2), is equal
to the true value 2
+ It has minimum variance in the
class of all such100
linear unbiased
estimators
 3-4. Properties of least-squares
estimators: The Gauss-Markov Theorem

 Gauss- Markov
Theorem:
Given the assumptions of
the classical linear
regression model, the
least-squares estimators,
in class of unbiased linear
101
β̂
2

3-5. The coefficient of determination r 2: A

measure of “Goodness of fit”
Ŷ YÛi = i + i or
Yi - Ŷ = Ŷ
 Y i
Û
- i + i or
 yi ŷ= Û+
i i
Y Ŷ
(Note: = )

Squaring on both side and summing =>

2
β̂
  y i2 = 2
x2i Û
+  2
i ; or

 TSS = ESS + RSS

102
3-5. The coefficient of determination r 2: A
measure of “Goodness of fit”

 TSS =  yi2 = Total Sum of Squares

 ESS =  Y^ i2 = ^22 x2i =
Explained Sum of Squares
 RSS =  u^2I = Residual Sum of
Squares
ESS RSS
1= -------- + -------- ; or
TSS TSS
RSS
RSS
1= r2 + ------- ; or r2 = 1 -
-------
TSS
TSS
103
3-5. The coefficient of determination r2:
A measure of “Goodness of fit”
 r2 = ESS/TSS
is coefficient of determination, it
measures the proportion or
percentage of the total
variation in Y explained by the
regression
Model
 0  r2  1;
 r =  r2 is sample correlation
104
3-5. The coefficient of determination
r2: A measure of “Goodness of fit”

3-6. A numerical Example

(pages 80-83)
3-7. Illustrative Examples
(pages 83-85)
3-8. Coffee demand Function
3-9. Monte Carlo Experiments
(page 85)
3-10. Summary and
conclusions (pages 86-87)
105
Basic Econometrics

Chapter 4:
THE NORMALITY
ASSUMPTION:
Classical Normal
Linear
Regression Model
(CNLRM)
106
 4-2.The normality assumption
 CNLR assumes that each u i is distributed
normally u i  N(0, 2) with:
Mean = E(u i) = 0 Ass 3
Variance = E(u2i) = 2 Ass 4
Cov(u i , u j ) = E(u i , u j) = 0 (i#j) Ass 5

 Note: For two normally distributed

variables, the zero covariance or correlation
means independence of them, so u i and u j
are not only uncorrelated but also
independently distributed. Therefore u i 
NID(0, 2) is Normally and
Independently Distributed
107
 In probability theory and statistics, a
collection of random variables
is independent and identically
distributed if each random variable has the
same probability distribution as the others
and all are mutually independent. This
property is usually abbreviated as i.i.d. or iid
or IID.
4-2.The normality assumption

 Why the normality assumption?

(1) With a few exceptions, the distribution
of sum of a large number of
independent and identically
distributed random variables tends to
a normal distribution as the number of
such variables increases indefinitely
(2) If the number of variables is not very
large or they are not strictly
independent, their sum may still be
normally distributed
109
4-2.The normality assumption

 Why the normality assumption?

(3) Under the normality assumption
for ui , the OLS estimators ^1 and
^2 are also normally distributed
(4) The normal distribution is a
comparatively simple distribution
involving only two parameters
(mean and variance)

110
 4-3. Properties of OLS
estimators under the
normality assumption
 With the normality assumption
the OLS estimators ^1 , ^2 and
^2 have the following
properties:
1. They are unbiased
2. They have minimum variance.
Combined 1 and 2, they are
efficient estimators
3. Consistency, that is, as the
sample size increases
indefinitely, the estimators
converge to their true
population values
111
 4-3. Properties of OLS estimators
under the normality assumption
4. ^1 is normally distributed 
N(1, ^12)
And Z = (^1- 1)/ ^1 is  N(0,1)
5. ^2 is normally distributed N(2 ,^22)
And Z = (^2- 2)/ ^2 is  N(0,1)
6. (n-2) ^2/ 2 is distributed as the
2(n-2)

112
 4-3. Properties of OLS estimators
under the normality assumption
7. ^1 and ^2 are distributed
independently of ^2. They have
minimum variance in the entire class of
unbiased estimators, whether linear or
not. They are best unbiased estimators
(BUE)
8. Let ui is  N(0, 2 ) then Yi is 
N[E(Yi); Var(Yi)] = N[1+ 2X i ; 2]

113
 Some last points of chapter
4
4-4. The method of Maximum likelihood
(ML)
 ML is point estimation method with some
stronger theoretical properties than OLS
(Appendix 4.A on pages 110-114)
The estimators of coefficients ’s by OLS and ML
are
 identical. They are true estimators of the ’s
 (ML estimator of 2) = u^i2/n (is biased
estimator)
 (OLS estimator of 2) = u^i2/n-2 (is unbiased
estimator)
 When sample size (n) gets larger the two
estimators tend to be equal
114
Some last points of chapter
4
4-5. Probability distributions related
to the Normal Distribution: The t, 2,
and F distributions
See section (4.5) on pages 107-108
with 8 theorems and Appendix A, on
pages 755-776
4-6. Summary and Conclusions
See 10 conclusions on pages 109-110

115
Basic Econometrics

Chapter 5:
TWO-VARIABLE
REGRESSION:
Interval Estimation
and Hypothesis Testing

116
 Chapter 5
TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-1. Statistical
Prerequisites
 See Appendix A with key concepts such
as probability, probability
distributions, Type I Error, Type II
Error, level of significance, power of a
statistic test, and confidence interval

117
 A type I error (false-positive) occurs if an investigator rejects a null
hypothesis that is actually true in the population;
 A type II error (false-negative) occurs if the investigator fails to reject a
null hypothesis that is actually false in the population.
 The significance level, also known as alpha or α, is a
measure of the strength of the evidence that must
be present in your sample before you will reject
the null hypothesis and conclude that the effect is
statistically significant. The researcher determines
the significance level before conducting the experiment.

11
 Statistical power, or the power of a hypothesis test
is the probability that the test correctly
rejects the null hypothesis. That is, the
probability of a true positive result. It is only useful
when the null hypothesis is rejected.

11
12
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-2. Interval estimation: Some basic

Ideas
 How “close” is, say, ^2 to 2 ?
Pr (^2 -   2  ^2 + ) = 1 -  (5.2.1)
 Random interval ^2 -   2  ^2 + 
if exits, it known as confidence interval
 ^2 -  is lower confidence limit
 ^2 +  is upper confidence limit

121
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-2. Interval estimation: Some basic Ideas

 (1 - ) is confidence coefficient,
 0 <  < 1 is significance level
 Equation (5.2.1) does not mean that
the Pr of 2 lying between the given
limits is (1 - ), but the Pr of
constructing an interval that contains
2 is (1 - )
 (^2 -  , ^2 + ) 122
is random interval
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-2. Interval estimation: Some basic Ideas

 In repeated sampling, the intervals will
enclose, in (1 - )*100 of the cases, the
true value of the parameters
 For a specific sample, can not say that
the probability is (1 - ) that a given
fixed interval includes the true 2
 If the sampling or probability
distributions of the estimators are
known, one can make confidence interval
statement like (5.2.1)
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-3. Confidence Intervals for Regression

Coefficients
 Z= (^ -  )/se(^ ) = (^ -  ) x2 /
2 2 2 2 2 i
~N(0,1)
(5.3.1)
We did not know  and have to use ^ instead, so:
 t= (^ -  )/se(^ ) = (^ -  ) x2 /^ ~
2 2 2 2 2 i
t(n-2)
(5.3.2)
 => Interval for 
2

Pr [ -t /2  tt /2 ] = 1-  (5.3.3)

124
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-3. Confidence Intervals for Regression

Coefficients
 Or confidence interval for  is
2

Pr [^2-t se(^2)  2  ^2+t

/2 se(^2)] =
/2
1- 
(5.3.5)
 Confidence Interval for 
1

Pr [^1-t se(^1)  1  ^1+t

/2 se(^1)] =
/2
1- 
(5.3.7)
125
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-4. Confidence Intervals for 2

Pr [(n-2)^2/ 2/2  2 (n-2)^2/ 21- /2]

= 1- 
(5.4.3)
 The interpretation of this interval is: If we
establish (1- ) confidence limits on 2 and if
we maintain a priori that these limits will
include true 2, we shall be right in the long
run (1- ) percent of the time

126
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-5. Hypothesis Testing: General Comments
 The stated hypothesis is known as the
null hypothesis: Ho
The Ho is tested against and alternative
hypothesis: H1

5-6. Hypothesis Testing: The confidence

interval approach
One-sided or one-tail Test
H0: 2  * versus H1: 2 > *

127
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
Two-sided or two-tail Test
H0: 2 = * versus H1: 2 # *
^2 - t se(^2)  2  ^2 + t
/2 se(^2)
/2

values of 2 lying in this interval are

plausible under Ho with 100*(1- )%
confidence.
 If 2 lies in this region we do not reject Ho
(the finding is statistically insignificant)
 If 2 falls outside this interval, we reject
Ho (the finding is statistically significant)
128
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-7. Hypothesis Testing:
The test of significance
approach
A test of significance is a procedure by which
sample results are used to verify the truth or
falsity of a null hypothesis
 Testing the significance of regression
coefficient: The t-test
Pr [^2-t /2se(^2)  2  ^2+t /2se(^2)]= 1- 
(5.7.2)
129
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
 5-7. Hypothesis Testing: The test of significance approach
 Table 5-1: Decision Rule for t-test of significance

Type of H0 H1 Reject H0
Hypothesis if
Two-tail 2 = 2* 2 # 2* |t| > t/2,df
Right-tail 2  2* 2 > 2* t > t,df
Left-tail 2 2* 2 < 2* t < - t,df
130
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
 5-7. Hypothesis Testing: The test of significance approach
Testing the significance of 2 : The 2 Test
Under the Normality assumption we have:
^2
2 = (n-2) ------- ~ 2 (n-2)
(5.4.1)
2
From (5.4.2) and (5.4.3) on page
520 =>

131
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
 5-7. Hypothesis Testing: The test of significance approach
 Table 5-2: A summary of the 2 Test

H0 H1 Reject H0 if
2 = 20 2 > 20 Df.(^2)/ 20 > 2 ,df
2 = 20 2 < 20 Df.(^2)/ 20 < 2(1-),df
2 = 20 2 # 20 Df.(^2)/ 20 > 2/2,df
or < 2 (1-/2), df
132
Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-8. Hypothesis Testing:
Some practical aspects
1) The meaning of “Accepting” or
“Rejecting” a Hypothesis
2) The Null Hypothesis and the Rule of
Thumb
3) Forming the Null and Alternative
Hypotheses
4) Choosing , the Level of Significance

133
Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-8. Hypothesis Testing:
Some practical
aspects
5) The Exact Level of Significance:
The p-Value [See page 132]
132
6) Statistical Significance versus
Practical Significance
7) The Choice between Confidence-
Interval and Test-of-Significance
Approaches to Hypothesis Testing
[Warning: Read carefully pages
117-134 ] 134
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-9. Regression Analysis and
Analysis
of Variance
 TSS = ESS + RSS
 F=[MSS of ESS]/[MSS of RSS] =
= 2^2 xi2/ ^2 (5.9.1)
 If ui are normally distributed; H0: 2 = 0 then
F follows the F distribution with 1 and n-2
degree of freedom
135
Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
 5-9. Regression Analysis
and Analysis of Variance
 F provides a test statistic to test the null
hypothesis that true 2 is zero by
compare this F ratio with the F-critical
obtained from F tables at the chosen
level of significance, or obtain the p-
value of the computed F statistic to
make decision

136
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
 5-9. Regression Analysis and Analysis of
Variance
 Table 5-3. ANOVA for two-variable regression
model

Source of Sum of square ( SS) Degree of Mean sum of

Variation Freedom - square ( MSS)
(Df)

ESS (due to y^i2 = 2^2 xi2 1 2^2 xi2

regression)
RSS (due to u^i2 n-2 u^i2 /(n-2)=^2
residuals)
TSS y i2 n-1

137
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-10. Application of
Regression
Analysis: Problem of
Prediction
 By the data of Table 3-2, we obtained the
sample regression (3.6.2) :
Y^i = 24.4545 + 0.5091Xi , where
Y^i is the estimator of true E(Yi)
 There are two kinds of prediction as
follows:
138
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-10. Application of Regression

Analysis: Problem of
Prediction
 Mean prediction: Prediction of the
conditional mean value of Y corresponding
to a chosen X, say X0, that is the point on
the population regression line itself (see
pages 137-138 for details)
 Individual prediction: Prediction of an
individual Y value corresponding
139 to X0 (see
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-11. Reporting the results of

regression analysis
 An illustration:

Y^I= 24.4545 + 0.5091Xi

(5.1.1)
Se = (6.4138) (0.0357) r2= 0.9621
t = (3.8128) (14.2405) df= 8
P = (0.002517) (0.000000289) F1,2=2202.87
140
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-12. Evaluating the results of regression
analysis:
 Normality Test: The Chi-Square (2)
Goodness of fit Test
 2
N-1-k =  (Oi – Ei)2/Ei (5.12.1)
Oi is observed residuals (u^i) in interval i
Ei is expected residuals in interval i
N is number of classes or groups; k is
number of
parameters to be estimated. If p-value
of
obtaining 2N-1-k is high (or 2N-1-k is small)
=>
141
The Normality Hypothesis can not be
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-12. Evaluating the results of regression
analysis:
 Normality Test: The Chi-Square (2)
Goodness of fit Test
H0: ui is normally distributed
H1: ui is un-normally distributed
Calculated-2N-1-k =  (Oi – Ei)2/Ei
(5.12.1)
Decision rule:
Calculated-2N-1-k > Critical-2N-1-k then
H0 can
be rejected
142
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-12. Evaluating the results of regression
analysis:
The Jarque-Bera (JB) test of normality
This test first computes the Skewness (S)
and Kurtosis (K) and uses the following
statistic:
JB = n [S2/6 + (K-3)2/24] (5.12.2)
Mean= xbar = xi/n ; SD2 = (xi-xbar)2/(n-1)
S=m3/m2 3/2 ; K=m4/m22 ; mk= (xi-xbar)k/n

143
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-12. (Continued)
Under the null hypothesis H0 that the
residuals are normally distributed Jarque and
Bera show that in large sample
(asymptotically) the JB statistic given in
(5.12.12) follows the Chi-Square distribution
with 2 df. If the p-value of the computed Chi-
Square statistic in an application is
sufficiently low, one can reject the hypothesis
that the residuals are normally distributed.
But if p-value is reasonable high, one does
not reject the normality assumption.
144
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-13. Summary and Conclusions
1. Estimation and Hypothesis testing
constitute the two main branches of
classical statistics
2. Hypothesis testing answers this
question: Is a given finding compatible
with a stated hypothesis or not?
3. There are two mutually complementary
approaches to answering the preceding
question: Confidence interval and test
of significance. 145
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-13. Summary and
Conclusions

4. Confidence-interval approach has a specified

probability of including within its limits the
true value of the unknown parameter. If the
null-hypothesized value lies in the confidence
interval, H0 is not rejected, whereas if it lies
outside this interval, H0 can be rejected

146
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-13. Summary and Conclusions
5. Significance test procedure develops a
test statistic which follows a well-defined
probability distribution (like normal, t, F,
or Chi-square). Once a test statistic is
computed, its p-value can be easily
obtained.
The p-value The p-value of a test is
the lowest significance level, at which we
would reject H0. It gives exact probability
of obtaining the estimated test statistic
under H0. If p-value is small, one can
reject H0, but if 147it is large one may not
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing
5-13. Summary and Conclusions
6. Type I error is the error of rejecting a true
hypothesis. Type II error is the error of
accepting a false hypothesis. In practice, one
should be careful in fixing the level of
significance , the probability of committing a
type I error (at arbitrary values such as 1%,
5%, 10%). It is better to quote the p-value of
the test statistic.

148
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-13. Summary and Conclusions

7. This chapter introduced the normality test to
find out whether ui follows the normal
distribution. Since in small samples, the t,
F,and Chi-square tests require the normality
assumption, it is important that this
assumption be checked formally

149
 Chapter 5 TWO-VARIABLE REGRESSION:
Interval Estimation and Hypothesis
Testing

5-13. Summary and Conclusions

(ended)

8. If the model is deemed practically adequate,

it may be used for forecasting purposes. But
should not go too far out of the sample range
of the regressor values. Otherwise,
forecasting errors can increase dramatically.

150
Basic Econometrics

Chapter 6

EXTENSIONS OF THE
TWO-VARIABLE LINEAR
REGRESSION MODEL

151
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS

6-1. Regression through the

origin

 The SRF form of regression:

 Yi = ^2X i + u^ i (6.1.5)

 Comparison two types of
regressions:
* Regression through-origin
152
model and
* Regression with intercept
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS

6-1. Regression through the

origin
Comparison two types of
regressions:

^2 = XiYi/X2i (6.1.6) O

^2 = xiyi/x2i (3.1.6) I
var(^2) = 2/X2i (6.1.7) O
var(^2) = 2/x2i (3.3.1) I
^2 = u^i)153
2
/(n-1) (6.1.8) O
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS
6-1. Regression through the
origin
 r2 for regression through-origin model

Raw r2 = (XiYi)2 /X2iY2i

(6.1.9)
 Note: Without very strong a priory expectation, well
advise is sticking to the conventional, intercept-
present model. If intercept equals to zero
statistically, for practical purposes we have a
regression through the origin. If in fact there is an
intercept in the model 154 but we insist on fitting a
regression through the origin, we would be
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS

6-1. Regression through the

origin
 Illustrative Examples:
1) Capital Asset Pricing Model - CAPM (page 156)
2) Market Model (page 157)
3) The Characteristic Line of Portfolio Theory
(page 159)

155
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS

6-2. Scaling and units of measurement

 Let Y = ^ + ^ X + u^
i 1 2 i i
(6.2.1)
 Define Y* =w Y and X* =w X
i 1 i i 2 i
then:
 ^ = (w /w )^
2 1 2 2
(6.2.15)
 ^ = w ^
1 1 1
(6.2.16)
 *^2 w 2^2
= 1
(6.2.17)
157
 Chapter 6
EXTENSIONS OF THE TWO-VARIABLE LINEAR
REGRESSION MODELS
6-2. Scaling and units of measurement
 From one scale of measurement, one can derive the
results
based on another scale of measurement. If w1= w2
the
intercept and standard error are both multiplied by
w1. If
w2=1 and scale of Y changed by w1, then all
coefficients and
standard errors are all multiplied by w1. If w1=1 and
scale of
X changed by w2, then only slope coefficient and its
standard
159
error are multiplied by 1/w2. Transformation from
6-3. Functional form of regression
model

The log-linear
model
Semi-log model
Reciprocal model

161
 6-4. How to measure elasticity
The log-linear model
 Exponential regression model:
 Y =  X e u
i 1 i i
(6.4.1)
By taking log to the base e of both
side:
 lnY = ln + lnX + u , by setting
i 1 2 i i
ln1 = 
 lnY =  + lnX + u
i 2 i i
(6.4.3)
(log-log, or double-log, or log-
linear model)
This can be estimated
162 by OLS by
letting
 6-4. How to measure elasticity

The log-linear model

The elasticity E of a variable Y with
respect to variable X is defined as:
E=dY/dX=(% change in Y)/(% change
in X)
~ [(Y/Y) x 100] / [(X/X) x100]=
= (Y/X)x (X/Y) = slope x (X/Y)

 An illustrative example: The coffee

demand function (pages 167-168)
163
 6-5. Semi-log model:
Log-lin and Lin-log Models

 How to measure the growth rate: The

log-lin model
 Y = Y (1+r) t
t 0
(6.5.1)
 lnY = lnY + t ln(1+r)
t 0
(6.5.2)
 lnY =  +  t , called constant growth
t  2
model (6.5.5)
where 1 = lnY0 ; 2 = ln(1+r)
 lnY =  +  t + u
t  2 i
(6.5.6)
 It is Semi-log model, or log-lin model.
164
The slope coefficient measures the
 6-5. Semi-log model:
Log-lin and Lin-log Models
 Instantaneous Vs. compound rate of
growth
 2 is instantaneous rate of growth
 antilog(2) – 1 is compound rate of
growth
The linear trend model
 Yt = + 2t + ut (6.5.9)
 If 2 > there is an upward trend in
Y
 If 2 < there is an downward trend
165 in Y
 6-5. Semi-log model:
Log-lin and Lin-log Models
 The lin-log model:
 Yi = 1 +2lnXi + ui (6.5.11)
 2 = (Change in Y) / Change in lnX = (Change in
Y)/(Relative change in X) ~ (Y)/(X/X)
(6.5.12)
 or Y = 2 (X/X) (6.5.13)
 That is, the absolute change in Y equal to 2
times the relative change in X.

166
 6-6. Reciprocal Models:
Log-lin and Lin-log Models

The reciprocal model:

 Yi = 1 + 2( 1/Xi ) + ui (6.5.14)
 As X increases definitely, the
term
2( 1/Xi ) approaches to zero and Yi
approaches the limiting or
asymptotic value 1 (See figure
6.5 in page 174)
 An Illustrative example: The
Phillips Curve for the United
Kingdom 167
1950-1966
 6-7. Summary of Functional Forms
Table 6.5 (page 178)
Model Equation Slope = Elasticity =
dY/dX (dY/dX).(X/Y)
Linear Y = X  (X/Y) */
Log-linear lnY = (YX) 
(log-log) lnX

Log-lin lnY = X Y X */

Lin-log Y = lnX 2(1/X) Y) */
Reciprocal Y= - 2(1/X2) - XY) */
X)169
 6-7. Summary of Functional Forms
 Note: */ indicates that the elasticity coefficient is
variable, depending on the value taken by X or Y
or both. when no X and Y values are specified, in
practice, very often these elasticities are
measured at the mean values E(X) and E(Y).
-----------------------------------------------
6-8. A note on the stochastic error term
6-9. Summary and conclusions
(pages 179-180)

170
Basic Econometrics

Chapter 7
MULTIPLE REGRESSION ANALYSIS:

The Problem of
Estimation

171
 7-1. The three-Variable Model:
Notation and Assumptions
 Yi = ß1+ ß2X2i + ß3X3i + u i (7.1.1)
 ß2 , ß3 are partial regression coefficients
 With the following assumptions:
+ Zero mean value of U i:: E(u i|X2i,X3i) = 0. i (7.1.2)
+ No serial correlation: Cov(ui,uj) = 0, i # j (7.1.3)
+ Homoscedasticity: Var(u i) = 2
(7.1.4)
+ Cov(ui,X2i) = Cov(ui,X3i) = 0 (7.1.5)
+ No specification bias or model correct specified
(7.1.6)
+ No exact collinearity between X variables
(7.1.7)
(no multicollinearity in the cases of more explanatory
vars. If there is linear relationship exits, X vars. Are
said
to be linearly dependent)
+ Model is linear in parameters
172
 7-2. Interpretation of
Multiple Regression

 E(Yi| X2i ,X3i) = ß1+ ß2X2i + ß3X3i (7.2.1)

 (7.2.1) gives conditional mean or

expected value of Y conditional upon
the given or fixed value of the X2 and X3

173
 7-3. The meaning of partial
regression coefficients
 Yi= ß1+ ß2X2i + ß3X3 +….+ ßsXs+ ui
 ßk measures the change in the
mean value of Y per unit change
in Xk, holding the rest
explanatory variables constant. It
gives the “direct” effect of unit
change in Xk on the E(Yi), net of
Xj (j # k)
 How to control the “true” effect
of a unit change
174 in Xk on Y? (read
 7-4. OLS and ML estimation of the
partial regression coefficients
 This section (pages 197-201)
provides:
1. The OLS estimators in the case of
three-variable regression
Yi= ß1+ ß2X2i + ß3X3+ ui
2. Variances and standard errors of
OLS estimators
3. 8 properties of OLS estimators (pp
199-201)
175
4. Understanding on ML estimators
 7-5. The multiple coefficient of
determination R2 and the multiple
coefficient of correlation R
 This section provides:
1. Definition of R2 in the context of
multiple regression like r2 in the case
of two-variable regression
2. R = R2 is the coefficient of multiple
regression, it measures the degree of
association between Y and all the
explanatory variables jointly
3. Variance of a partial regression
coefficient
Var(ß^k) = 2/ x2k (1/(1-R2k)) (7.5.6)
Where ß^k is the partial regression
coefficient of regressor X k and R2k is the
R2 in the regression
176 of Xk on the rest
regressors
 7-6. Example 7.1: The
expectations-augmented Philips
Curve for the US (1970-1982)

 This section provides an illustration

for the ideas introduced in the
chapter
 Regression Model (7.6.1)
 Data set is in Table 7.1

177
 7-7. Simple regression in the
context of multiple regression:
Introduction to specification bias

 This section provides an

understanding on “ Simple
regression in the context of
multiple regression”. It will cause
the specification bias which will be
discussed in Chapter 13

178
 7-8. R2 and the Adjusted-R2
 R2 is a non-decreasing function of the number
of explanatory variables. An additional X
variable will not decrease R2
R2= ESS/TSS = 1- RSS/TSS = 1-u^2I / y^2i
(7.8.1)
 This will make the wrong direction by adding
more irrelevant variables into the regression
and give an idea for an adjusted-R2 (R bar) by
taking account of degree of freedom
 R2bar= 1- [ u^2I /(n-k)] / [y^2i /(n-1) ] , or
(7.8.2)
R2bar= 1- ^2 / S2Y (S2Y is sample variance of Y)
K= number of parameters including intercept term
 By substituting (7.8.1) into (7.8.2) we get
R2bar = 1- (1-R2) (n-1)/(n- k)
(7.8.4)
 For k > 1, R2bar179
< R2 thus when number of X
variables increases R2bar increases less than R2 and
 7-8. R2 and the Adjusted-R2

 R2 is a non-decreasing function of the number

of explanatory variables. An additional X
variable will not decrease R2
R2= ESS/TSS = 1- RSS/TSS = 1-u^2I / y^2i
(7.8.1)
 This will make the wrong direction by adding
more irrelevant variables into the regression
and give an idea for an adjusted-R2 (R bar) by
taking account of degree of freedom
 R2bar= 1- [ u^2I /(n-k)] / [y^2i /(n-1) ] , or
(7.8.2)
R2bar= 1- ^2 / S2Y (S2Y is sample variance of Y)
K= number of parameters including intercept term
 By substituting (7.8.1) into (7.8.2) we get
R2bar = 1- (1-R2) (n-1)/(n- k)
(7.8.4)
 For k > 1, R2bar180
< R2 thus when number of X
variables increases R2bar increases less than R2 and
 7-8. R2 and the Adjusted-R2

 Comparing Two R2 Values:

To compare, the size n and the dependent
variable must be the same
 Example 7-2: Coffee Demand Function
Revisited (page 210)
 The “game” of maximizing
adjusted-R2: Choosing the model that
gives the highest R2bar may be dangerous,
for in regression our objective is not for
that but for obtaining the dependable
estimates of the true population regression
coefficients and draw statistical inferences
about them
 Should be more concerned about the
logical or theoretical relevance of the
181
explanatory variables to the dependent
variable and their statistical significance
 7-9. Partial Correlation
Coefficients
 This section provides:

1. Explanation of simple and

partial correlation coefficients
2. Interpretation of simple and
partial correlation coefficients
(pages 211-214)

182
 7-10. Example 7.3: The Cobb-
Douglas Production function
More on functional form

 Yi = 1X22i X33ieUi (7.10.1)

By log-transform of this model:
 lnYi = ln1 + 2ln X2i + 3ln X3i + Ui
= 0 + 2ln X2i + 3ln X3i + Ui

(7.10.2)
Data set is in Table 7.3
Report of results is in page
216
183
 7-11 Polynomial Regression
Models

 Yi = 0 + 1 Xi + 2 X2i +…+ k Xki +

Ui
(7.11.3)
 Example 7.4: Estimating the Total
Cost Function
 Data set is in Table 7.4
 Empirical results is in page 221

--------------------------------------------------------
------
 7-12. Summary
184 and Conclusions
(page 221)
Basic Econometrics

Chapter 8
MULTIPLE REGRESSION ANALYSIS:

The Problem of
Inference

185
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-3. Hypothesis testing in multiple regression:
Testing hypotheses about an individual
partial regression coefficient
Testing the overall significance of the
estimated multiple regression model, that
is, finding out if all the partial slope
coefficients are simultaneously equal to zero
Testing that two or more coefficients are
equal to one another
Testing that the partial regression
coefficients satisfy certain restrictions
Testing the stability of the estimated
regression model over time or in different
cross-sectional units
Testing the functional form of regression
models
186
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-4. Hypothesis testing about
individual partial regression
coefficients
With the assumption that u i ~
N(0,2) we can use t-test to test a
hypothesis about any individual
partial regression coefficient.
H0 :  2 = 0
H1 :  2  0
If the computed t value > critical t
value at the chosen level of
significance, we may reject the null
hypothesis; otherwise,
187
we may not
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance of a
multiple
regression: The F-Test
For Yi = 1 + 2X2i + 3X3i + ........+ kXki
+ ui
 To test the hypothesis H :  =
0 2 3
=....= k= 0 (all slope coefficients are
simultaneously zero) versus H1: Not at all
slope coefficients are simultaneously
zero, compute
F=(ESS/df)/(RSS/df)=(ESS/(k-1))/(RSS/
(n-k)) (8.5.7) (k = total number of
parameters to be estimated
188
including
intercept)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance
of a multiple regression
 Alternatively, if the p-value of F
obtained from (8.5.7) is sufficiently
low, one can reject H0
 An important relationship between
R2 and F:
F=(ESS/(k-1))/(RSS/(n-k)) or
R2/(k-1)
F = ----------------
(8.5.1)
(1-R2)189/ (n-k)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance
of a multiple regression in terms of
R2
For Yi = 1 + 2X2i + 3X3i + ........+ kXki + ui
 To test the hypothesis H :  =  = .....=
0 2 3
k = 0 (all slope coefficients are
simultaneously zero) versus H1: Not
at all slope coefficients are
simultaneously zero, compute
 F = [R2/(k-1)] / [(1-R2) / (n-k)]
(8.5.13) (k = total number of
parameters to be190estimated including
intercept)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance
of a multiple regression
Alternatively, if the p-value of F
obtained from (8.5.13) is sufficiently
low, one can reject H0

The “Incremental” or “Marginal”

contribution of an explanatory
variable:
Let X is the new (additional) term
in the right hand of a regression.
Under the usual assumption of the
normality of ui and the HO:  = 0, it
can be shown that191 the following F
ratio will follow the F distribution
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance
of a multiple regression
[R2new - R2old] / Df1
F com = ----------------------
(8.5.18)
[1 - R2new] / Df2
Where Df1 = number of new
regressors
Df2 = n – number of parameters
in the new model
R2new is standing for coefficient of
determination of the new regression
(by adding
192
X);
R2 is standing for coefficient of
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-5. Testing the overall significance of
a multiple regression
Decision Rule:
If F com > F  Df1 , Df2 one can reject the
Ho that  = 0 and conclude that the
addition of X to the model
significantly increases ESS and hence
the R2 value
 When to Add a New Variable? If |t|
of coefficient of X > 1 (or F= t 2 of that
variable exceeds 1)
 When to Add a 193Group of Variables?
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-6. Testing the equality of two regression
coefficients

Yi = 1 + 2X2i + 3X3i + 4X4i + ui (8.6.1)

Test the hypotheses:
H0: 3 = 4 or 3 - 4 = 0 (8.6.2)
H1: 3  4 or 3 - 4  0
Under the classical assumption it can be shown:
t = [(^3 - ^4) – (3 - 4)] / se(^3 - ^4)
follows the t distribution with (n-4) df because
(8.6.1) is a four-variable model or, more generally,
with (n-k) df. where k is the total number of
parameters estimated, including intercept term.
se(^3 - ^4) =  [var((^3) + var( ^4) – 2cov(^3, ^4)]
(8.6.4)
194
(see appendix)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference

t = (^3 - ^4) /  [var((^3) + var( ^4) – 2cov(^3,

^4)] (8.6.5)
Steps for testing:
1. Estimate ^3 and ^4
2. Compute se(^3 - ^4) through (8.6.4)
3. Obtain t- ratio from (8.6.5) with H0: 3 =
4
4. If t-computed > t-critical at designated
level of significance for given df, then
reject H0. Otherwise do not reject it.
Alternatively, if the p-value of t statistic
from (8.6.5) is reasonable
195
low, one can
reject H0.
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-7. Restricted least square:
Testing linear equality restrictions
Yi = 1X22i X33i eui (7.10.1) and
(8.7.1)
Y = output
X2 = labor input
X3 = capital input
In the log-form:
lnYi = 0 + 2lnX2i + 3lnX3i + ui
(8.7.2)
with the constant return to scale:
2 + 3 = 1 196
(8.7.3)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-7. Restricted least square: Testing
linear equality restrictions
How to test (8.7.3)
 The t Test approach (unrestricted): test
of the hypothesis H0:2 + 3 = 1 can be
conducted by t- test:
t = [(^2 + ^3) – (2 + 3)] / se(^2 - ^3)
(8.7.4)

 The F Test approach (restricted least

square -RLS): Using, say, 2 = 1-3 and
substitute it into (8.7.2)
197
we get: ln(Yi /X2i) =
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-7. Restricted least square: Testing linear
equality restrictions
u^2UR=RSSUR of unrestricted regression
(8.7.2)
and  u^2R = RSSR of restricted regression
(8.7.7),
m = number of linear restrictions,
k = number of parameters in the
unrestricted regression,
n = number of observations.
R2UR and R2R are R2 values obtained from
unrestricted and restricted regressions
respectively. Then
F=[(RSSR – RSSUR)/m]/[RSSUR/(n-k)] =
= [(R2UR – R2R) / m] /198[1 – R2UR / (n-k)]
(8.7.10)
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-7. Restricted least square: Testing
linear equality restrictions
 Note: R2UR  R2R
(8.7.11)
 and  u^2UR   u^2R
(8.7.12)
 Example 8.3: The Cobb-Douglas
Production
function for Taiwanese Agricultural
Sector,
1958-1972. (pages 259-260). Data in
Table 7.3
(page 216)
 General F 199
Testing (page 260)
 Example 8.4: The demand for chicken
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-8. Comparing two regressions: Testing
for structural stability of regression
models
Table 8.8: Personal savings and income
data, UK, 1946-1963 (millions of pounds)
Savings function:
 Reconstruction period:
Y t = 1+ 2X t + U1t (t = 1,2,...,n1)
 Post-Reconstruction period:
Y t = 1 + 2X t + U2t (t = 1,2,...,n2)
Where Y is personal savings, X is
personal income, the us are disturbance
terms in the two equations and n1, n2 are
the number of observations
200 in the two
period
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-8. Comparing two regressions: Testing for
structural stability of regression models
+ The structural change may mean that the
two intercept are different, or the two slopes
are different, or both are different, or any
other suitable combination of the
parameters. If there is no structural change
we can combine all the n1, n2 and just
estimate one savings function as:
Y t = 1 + 2X t + Ut (t = 1,2,...,n1, 1,....n2).
(8.8.3)
How do we find out whether there is a
structural change in the savings-income
relationship between the two period? A
popular test is Chow-Test, it is simply the F
Test discussed earlier
HO: i = i i Vs H1: i that i  i
201
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-8. Comparing two regressions: Testing for
structural stability of regression models
+ The assumptions underlying the Chow test
u1t and u2t ~ N(0,s2), two error terms are
normally distributed with the same variance
u1t and u2t are independently distributed
Step 1: Estimate (8.8.3), get RSS, say, S1 with
df = (n1+n2 – k); k is number of parameters
estimated )
Step 2: Estimate (8.8.1) and (8.8.2)
individually and get their RSS, say, S2 and S3 ,
with df = (n1 – k) and (n2-k) respectively. Call
S4 = S2+S3; with df = (n1+n2 – 2k)
Step 3: S5 = S1 – S4;

202
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-8. Comparing two regressions: Testing for
structural stability of regression models
Step 4: Given the assumptions of the Chow
Test, it can be show that
F = [S5 / k] / [S4 / (n1+n2 – 2k)]
(8.8.4)
follows the F distribution with Df = (k, n1+n2
– 2k)
Decision Rule: If F computed by (8.8.4) > F-
critical at the chosen level of significance a
=> reject the hypothesis that the regression
(8.8.1) and (8.8.2) are the same, or reject the
hypothesis of structural stability; One can
use p-value of the F obtained from (8.8.4) to
reject H0 if p-value low reasonably.
+ Apply for the 203 data in Table 8.8
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-9. Testing the functional form of regression:

Choosing between linear and log-linear regression

models: MWD Test (MacKinnon, White and Davidson)
H0: Linear Model Y is a linear function of regressors,
the Xs;
H1: Log-linear Model Y is a linear function of logs of
regressors, the lnXs;

204
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-9. Testing the functional form of regression:
Step 1: Estimate the linear model and obtain
the estimated Y values. Call them Yf
(i.e.,Y^). Take lnYf.
Step 2: Estimate the log-linear model and
obtain the estimated lnY values, call them lnf
(i.e., ln^Y )
Step 3: Obtain Z1 = (lnYf – lnf)
Step 4: Regress Y on Xs and Z1. Reject H0 if
the coefficient of Z1 is statistically
significant, by the usual t - test
Step 5: Obtain Z2 = antilog of (lnf – Yf)
Step 6: Regress lnY on lnXs and Z2. Reject H1
if the coefficient of Z2 is statistically
significant, 205
by the usual t-test
 Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference

Example 8.5: The demand for Roses

(page 266-267). Data in exercise 7.20
(page 225)

8-10. Prediction with multiple regression

Follow the section 5-10 and the
illustration in pages 267-268 by using
data set in the Table 8.1 (page 241)

8-11. The troika of hypothesis tests: The

likelihood ratio (LR), Wald (W) and
Lagarange Multiplier (LM) Tests

8-12. Summary
206 and Conclusions