Econometrics I Lecture Note DBU

3. MULTIPLE LINEAR REGRESSION

3.1. Introduction
We have studied the two-variable model extensively in the previous unit. But in
economics you hardly found that one variable is affected by only one explanatory
variable. For example, the demand for a commodity is a function of price of the
product, price of other goods (substitute and complementary goods), income of the
consumer, wealth, previous year income consumption behavior etc. Hence the two
variable model is often inadequate in practical works. Therefore, we need to
discuss multiple regression models. The multiple linear regression is entirely
concerned with the relationship between a dependent variable (Y) and two or more
explanatory variables (X1, X2… Xn). For the sake of simplicity let’s consider a three
variable case (one dependent and two explanatory variables).

Y = f(X1, X2)

Yi   0  1 X 1   2 X 2  Ui              3.1

Where Yi is dependent and X1 & X2 are independent (explanatory variables) & Ui

is the stochastic disturbance term. The disturbance term is of similar nature to
that in simple regression, reflecting:

 The basic random nature of human responses

 Errors of aggregation
 Errors of measurement
 Errors in specification of the mathematical form of the model and
 Any other (minor) factors, other than X i that might influence Y.

Equation (3.1) is a multiple regression with two explanatory variables. In general

for K-explanatory variable we can write the model as follows:
Yi   0  1 X 1i   2. X 2i   3. X 3i   k . X ki  Ui            3.2

Where 𝑋𝑘𝑖 = (𝑖 = 1, 2,3, . . . . . . . , 𝐾) are explanatory variables, 𝑌𝑖 is the dependent

variable and 𝛽𝑗 ( 𝑗 = 0,1,2, . . . . (𝑘 + 1)) are unknown parameters and 𝑢𝑖 is the
disturbance term.
1|Page Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

  0 Measures the constant term and measuring the average value of Y when
X1 and X2 are zero.
  1 Measures the change in Y for a unit change in X1 alone (the effects of X1
on Y given that X2 is constant.)
  2 Measures the change in Y for a unit change in X2 alone (the effects of X2
on Y given that X1 is constant).
The coefficients  1 and  2 are called the partial regression coefficients.
Like simple linear regression, the model is assumed to be linear in parameters but
not necessarily in variables. Hence, the linear and nonlinear model parameter
interpretations are not the same. Let’s discuss some forms of nonlinear and linear
models below.

3.2. Model Specification and Estimator Interpretations

As mentioned above, linear regression does not mean that the involved
variables have to be in the linear form. It needs the regression has to be linear
in parameters (in B’s). Hence, we might include different forms of explanatory
(and explained) variables

o squares and higher powers (e.g. 𝑋 2 )

o logarithms (e.g. ln (x), ln (y ))
Explanatory (and explained) variables do not have to be continuous, it can be

o discrete variables (e.g. number of children)

o categorical variables (e.g. region, gender )
o ordinal variables (e.g. level of education)
Therefore, we have to choose carefully the functional form of the relationship
between the dependent variable and each explanatory variable.

o Do we expect a curve instead of a straight line?

o Does the effect of a variable peak at some point and then start to
decline?
2|Page Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

If inappropriate specification is used, coefficients will be biased and inference

about them will be incorrect. We will present some of the most used model
specifications and their interpretation below.

3.2.1. LINEAR FORM

The model is specified as: 𝑦 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 + 𝑢

Assumes that the relationship between the explanatory variable and the
dependent variable is constant:

𝜕𝑦
= 𝛽𝑘 , k = 1,2
𝜕𝑥𝑘

 Interpretation: if x increases by 1 unit, then y will change by 𝛽𝑘 units

 Linear form is used as default functional form until strong evidence that
it is inappropriate is found.

3.2.2. DOUBLE-LOG FORM [Log-Log Form]

𝑙𝑛𝑦 = 𝛽0 + 𝛽1 𝑙𝑛𝑋1 + 𝛽2 𝑙𝑛𝑋2 + 𝑢

• Assumes that the elasticities of the model are constant:

𝜕𝑦⁄
𝜕𝑙𝑛𝑦 𝑦
= = 𝛽𝑘 , k = 1,2
𝜕𝑙𝑛𝑥𝑘 𝜕𝑥𝑘⁄
𝑥𝑘

 Interpretation: if x increases by 1 percent, then y will change by 𝛽𝑘

percent.

 Before using a double-log model, make sure that there are no negative or
zero observations in the data set.

Example: Take the following Cob-Douglas production function

Yi  0 Li 1 ki 2 eu
B B

3|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

Let Y= output, L is labor & K is capital and  0 , 1 &  2 are parameters then to

estimate this equation first transform in to linearity using logarithmic.

Log Yi= log  0  1 log Li   2 log K i  U i

 Then here 1 &  2 measures elasticties not average values.

3.2.3. SEMILOG FORMS

If the specification involves linear and logarithmic as follows:

𝑦 = 𝛽0 + 𝛽1 𝑙𝑛𝑋1 + 𝛽2 𝑙𝑛𝑋2 + 𝑢

𝜕𝑦 𝜕𝑦
• Then, 𝜕𝑙𝑛𝑥𝑘
= 𝜕𝑥𝑘 = 𝛽𝑘 , k = 1,2
⁄𝑥
𝑘

𝛽𝑘
 Interpretation: if x increases by 1 percent, then y will change by units.
100

However, if the model specification is as follows:

𝑙𝑛𝑦 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 + 𝑢
𝜕𝑦⁄
𝜕𝑙𝑛𝑦 𝑦
• Then, = = 𝛽𝑘 , k = 1,2
𝜕𝑥𝑘 𝜕𝑥𝑘

 Interpretation: if x increases by 1 unit, then y will change by (100𝛽𝑘 ) percent.

3.2.4. OTHER FORMS

I. Polynomial Form: 𝑦 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋22 + 𝑢

1
II. Inverse Form: 𝑦 = 𝛽0 + 𝛽1 𝑋 + 𝛽2 𝑋22 + 𝑢
1

4|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

3.2.5. PROBLEMS WITH INCORRECT FUNCTIONAL FORMS

• The functional form has to be correctly specified in order to avoid inconsistent

estimates and wrong inference

• Ideally: the specification is given by underlying theory of the equation

• In reality: underlying theory does not give precise functional form

• In most cases, either linear form is adequate, or common sense will point out
an easy choice from among the alternatives

• Nonlinearity of explanatory variables

• often approximated by polynomial form

• potential higher powers can be detected as omitted variables

• Nonlinearity of dependent variable

• harder to detect based on statistical fit of the regression

• 𝑅 2 is incomparable across models where the y is transformed

• If the nonlinear specification is approximative, it will perform badly out of the

sample range.

3.3. Assumptions

To complete the specification of our simple model we need some assumptions

about the random variable U. These assumptions are the same as those
assumptions already explained in the two-variable model in chapter 2.

1. Disturbances are random variables drawn from a normal distribution

2. Zero mean value of Ui

The random variable U has a zero mean value for each Xi

E (Ui/X1i, X2i) = 0 for each i.

3. Homoscedasticity
The variance of each Ui is the same for all the Xi values var (Ui) = E(Ui2) =  u2
5|Page Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

4. Normality
The values of each Ui are normally distributed Ui ~ N(0,  u2 )

5. No serial correlation (serial independence of the U’s)

The values of Ui (corresponding to Xi) are independent from the values of any
other Uj (corresponding to Xj). Cov (Ui, Uj) = 0 for i  j

6. Independence of Ui and Xi (consistency assumption)

Every disturbance term Ui is independent of the explanatory variables. That is
there is zero covariance between Ui and each X variables.

Cov(Ui, X1i) = Cov (Ui, X2i) = 0 or E(Ui ,X1i) = E(Ui X2i) = 0

Here the values of the X’s are a set of fixed numbers in all hypothetical samples

7. No collinearity between the X variables (No multicollinearity)

The explanatory variables are not perfectly linearly correlated. There is no
exact linear relationships between X1 and X2.
8. Correct specification of the model
The model has no specification error in that all the important explanatory
variables appear explicitly in the function and the mathematical form is
correctly defined.

We can’t exclusively list all the assumptions but the above assumptions are
some of the basic assumptions that enable us to proceed our analysis.

3.4. Estimation with Two Explanatory Variables

In order to understand the nature of multiple regression model easily, we start

our analysis with the case of two explanatory variables, then extend this to the
case of k-explanatory variables.
The model: 𝑌 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 + 𝑢 --------------------------- 3.3
is multiple regression with two explanatory variables. The expected value of the above
model is called population regression equation i.e.
𝐸 (𝑌) = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 − − − − − − − −3.4 , 𝑠𝑖𝑛𝑐𝑒 𝐸( 𝑢) = 0

6|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

Since the population regression equation is unknown to any investigator, it has

to be estimated from sample data. Let us suppose that the sample data has
been used to estimate the population regression equation.
Now let us have sample observations on Y, 𝑋1𝑖 , and 𝑋2𝑖 and obtain estimates of
the true parameters 𝛽0 , 𝛽1 , 𝑎𝑛𝑑 𝛽2

𝑌𝑖 𝑋1𝑖 𝑋2𝑖

𝑌1 𝑋11 𝑋21

𝑌2 𝑋12 𝑋22

𝑌3 𝑋13 𝑋23

  

𝑌𝑛 𝑋1𝑛 𝑋2𝑛

The sample regression function (SRF) can be written as

  
Yi =  o   1 X 1i   2 X 2i  ei

  
Where  o ,  1 and  2 are estimates of the true parameters  o , 1 and 𝛽2

ei is the residual term.

But since U i is unobservable the above equation becomes

  
Yˆi =  o   1 X 1i   2 X 2i is the estimated Regression line.

As discussed in chapter 2, the estimates will be obtained by choosing the values

of the unknown parameters that will minimize the sum of squares of the
residuals. (OLS requires the ei2 be as small as possible). Symbolically,

n   
Min ei2 = (Yi – Yˆi )2 = 
i
(Yi -  o   1 X 1i   2 X 2i )2------------ 3.5

7|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

A necessary condition for a minimum value is that the partial derivatives of the
  
above expression with respect to the unknowns (i.e.  o ,  1 and  2 ) should be
set to zero.

2
   

   Yi   0   1 X 1   2 X 2 
  ei 
2
   2 Yi  ˆ   X  ˆ X ) 0 --------3.6

   0 1 1 2 2
 0  0
2
   

   Yi   0   1 X 1   2 X 2 
  ei 
2
   2 X [Yi  ˆ  ˆ X  ˆ X ]  0 ---- 3.7

   1 0 1 1 2 2
 1  1
2
   

   Yi   0   1 X 1   2 X 2 
  ei 
2
   2 X [Yi  ˆ  ˆ X  ˆ X ] 0 -----3.8

   2 0 1 1 2 2
 2  2

After differentiating, we get the following normal equations (for the sake of
simplicity let’s ignore the i’s sub-scripts):

  
𝑌𝑖 = 𝑛  o +  1 𝑋1 +  2 𝑋2 − − − − − − − − − − − − − − − − − − − − 3.9
  
𝑋1 𝑌𝑖 =  o 𝑋1 +  1 𝑋12 +  2 𝑋1 𝑋2 − − − − − − − − − − − − − − − − − 3.10
  
𝑋2𝑖𝑌𝑖 =  o 𝑋2 +  1  𝑋1 𝑋2 +  2 𝑋22 − − − − − − − − − − − − − − − −3.11

From equation number 3.9 we have


 o = Y  ˆ1 x1  ˆ2 x2                          3.12

By substituting these values in  o in to equation 3.10 & 3.11 yields

YX i X 2 1 (Y  1 x1 ˆ 2x2 )  ˆ1̂  X 1  ˆ2  X 2  X 1            3.13

From equation number 3.12 & 3.13 by rearranging we obtain

(X 1  x1 )(Yi  Y )  ˆ1  ( X 1  x1 ) 2  ˆ 2  ( X 1  x1 )( X 2  x2 )

(X  x2 )(Yi  Y )  ˆ1  ( X 1  x1 ) ( X 2  x2 )  ˆ 2  ( X 2  x2 )

2
2

8|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

If you write the above equation in lower cases letters for deviations from means we
can write the above equations in the following ways.

 x1 y  ˆ1  x1  ˆ2  x1 x2
2

 x2 y  ˆ2  x1 x2  ˆ2  x22

 
 1 and  2 can easily be solved using matrix. We can rewrite the above two equations
in matrix form as follows.

 x1
2
 x1 x2 𝛽̂1  x1 y
[ ][ ] = [ ]-------------------------------------3.14
 x1 x2  x 22 𝛽̂2  x2 y

If we use Cramer’s rule to solve the above matrix we obtain:

  
 o = Y   1 X1   2 X 2


1=  x y  x    x y  x x 
1
2
2 2 1 2

 x  x    x x 
2
1
2
2 1 2
2


 2=  x y  x    x y  x x 
2
2
1 1 1 2

 x  x    x x 
2
1
2
2 1 2
2

Where the variables x and y are in deviation forms

i.e., y = Yi - Y , x1 = X1 - X 1 , x2= X2 - X 2

  
Note: The values for the parameter estimates (  o ,  1 and  2 ) can also be
obtained by using other methods.

 
We can also express  1 and  2 in terms of covariance and variances of 𝑋1 , 𝑋2 𝑎𝑛𝑑 𝑌1
𝐶𝑜𝑣 (𝑋1 , 𝑌)𝑉𝑎𝑟(𝑋2 ) − 𝐶𝑜𝑣 (𝑋2 , 𝑌)𝐶𝑜𝑣(𝑋1 , 𝑋2 )
𝛽̂1 = − − − − − − − −3.15
𝑉𝑎𝑟(𝑋1 )𝑉𝑎𝑟(𝑋2 ) − [𝐶𝑜𝑣(𝑋1 , 𝑋2 )]2

𝐶𝑜𝑣 (𝑋2 , 𝑌)𝑉𝑎𝑟(𝑋1 ) − 𝐶𝑜𝑣 (𝑋1 , 𝑌)𝐶𝑜𝑣(𝑋1 , 𝑋2 )

𝛽̂2 = − − − − − − − 3.16
𝑉𝑎𝑟(𝑋1 )𝑉𝑎𝑟(𝑋2 ) − [𝐶𝑜𝑣(𝑋1 , 𝑋2 )]2

9|Page Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

3.5. THE MULTIPLE COEFFICIENT OF DETERMINATION (𝑹𝟐 )

In the simple regression model, we introduced 𝒓𝟐 as a measure of the proportion
of variation in the dependent variable that is explained by variation in the
explanatory variable. In multiple regression model the same measure is
relevant, and the same formulas are valid the only difference here is that the
inclusion of additional variables. Then 𝑹𝟐 explain the proportion of the variation
in Y explained by the variables X1 & X2 jointly. Thus, 𝑹𝟐 is the coefficient of
determination or called the unadjusted 𝑹𝟐 and it can be derived following the
same procedures as 𝒓𝟐 .
 R 2 y. X X - shows the percentage of the total variation of Y explained by the
1 2

regression plane, that is, by changes in X1 and X2.

 yˆ  Y Y  e
2 2 2
RSS
=  =1–  1
2 i i i
R
y  Y Y  y
y. X1 X 2 2 2 2
i i i
TSS

Where: RSS – residual sum of squares

TSS – total sum of squares

Recall that
 
ŷ i =  1 x1 +  2 x 2i (the variables are in deviation forms)

yi = ŷ i + ei
 
ei2 = (yi - ŷ i )2 = (yi -  1 x1i -  2 x 2i )2
   
or ei2 = ei .ei = ei(yi -  1 x1i -  2 x 2i )= ei .yi -  1 ei .x1 -  2 ei .x2

but ei x1i = ei x2i = 0 Hence ei 2 = ei yi

=  (yi - ŷ i )yi since ei = yi - ŷ i

   
yi (yi -  1 x1i -  2 x 2i ) = yi2 -  1 x1i yi -  2 x2i yi

By substituting the value of ei 2 in the formula of R2, we get

10 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

   
 y i 2   1  x1i y i   2 x yi 
 
2i

e
2
R 2 y. X1 X 2 = 1 – i
 1  
y  yi
2 2
i

2
R =
ESS

ˆ 1
 y x1  ˆ  x2 y
2
, where x1i, x2i and yi are in their deviation forms.
TSS y 2

The value of R2 lies between 0 and 1. The higher R2 the greater the percentage of

the variation of Y explained by the regression plane, that is, the better the

goodness of fit of the regression plane to the sample observations. The closer R 2

to zero, the worse the fit.

2
If you compare R equation with the previous r2

 x1 y
r2 =
ˆ1
 y2

the only difference is we added ˆ  x

2
2
y because we incorporate the 2nd

explanatory variable X2i. Then as you increase your explanatory variables the

value of R2 will increase because as there numerator is increasing the denominator

remains the same. In other words, as the number of regressors (explanatory

variables) increases the coefficient of multiple determination will usually increase.

Due to this reason we call it unadjusted R2. Therefore, if we have two models with

different explanatory variables, we cannot compare them on the basis of R2.

Therefore, to correct for this defect we adjust R2 by taking into account the degrees

of freedom, which clearly decrease as new regressors are introduced in the

11 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

function. Thus, we should have to use Adjusted (corrected) R 2 to compare two

equation with different number of explanatory variables.

Adjusted R 2

e nk
2
i
n 1
R 2 =1 - =1-(1- R 2 )
nk
 yi 2
n 1

Where n-sample size

K-degree of freedom
As the number of explanatory variables increases, the adjusted R 2 is
2
increasingly less than the unadjusted R2. The adjusted R2 ( R ) can be negative,
although R2 is necessarily non-negative. In this case its value is taken as zero.

If the sample size is small R 2 < R 2 but if the sample size is very large R 2 and R 2

will be very close to each other’s.

Note that:

 If R 2 =1, R 2 =1,

(1−𝑘)
 When R 2 =0, R 2 = in this case R 2 will be negative if k>1.
(𝑛−𝑘)

3.6. General Linear Regression Model and Matrix Approach

So far we have discussed the regression models containing one or two

explanatory variables. Let us now generalize the model assuming that it

contains k variables. It will be of the form:

𝑌 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 + 𝛽3 𝑋3 + ⋯ + 𝛽𝑘 𝑋𝑘 + 𝑢

12 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

13 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

Solving the above normal equations will result in algebraic complexity. But we
can solve this easily using matrix. Hence in the next section we will discuss the
matrix approach to linear regression model.
3.6.1 Matrix Approach to Linear Regression Model

14 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

15 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

3.6.2. Statistical Properties of the Parameters (Matrix) Approach

We have seen, in simple linear regression that the OLS estimators (𝛽̂0 & 𝛽̂1 ) satisfy
the small sample property of an estimator i.e. BLUE property. In multiple
regression, the OLS estimators also satisfy the BLUE property. Now we proceed to
examine the desired properties of the estimators in matrix notations:

16 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

17 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

18 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

19 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

This is all about the variance covariance of the parameters.

3.7. Hypothesis Testing in Multiple Regression Model

The principle involved in testing multiple regression is identical with that of simple
regression. However, in multiple regression models we will undertake two tests
of significance. One is significance of individual parameters of the model.This
test of significance is the same as the tests discussed in simple regression
model. The second test is overall significance of the model.

3.7.1. Tests of individual significance

If we invoke the assumption that 𝑈𝑖 ~𝑁(0, 𝜎𝑢2 ), then we can use either the t-test
or standard error test to test a hypothesis about any individual partial
regression coefficient. As we have derived above, we have the following variance
and standard errors of the OLS estimators:
1
x  x  x  x  2 x x  x x  ˆ
2 2 2 2

Var ( ( ˆ0 )    1 2 2 1 1 2 1 2 2
u
 n  x  x  ( x x ) 
2 2 2
1 2 1 2

  x2
2
   x1
2

Var ( ( ˆ1 )  ˆ u   Var( ˆ 2 ) = ˆ u  
2 2

  x1  x2  ( x1 x2) 2    x1  x2  ( x1 x2) 2 
2 2 2 2

20 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

Where ˆ u 2 =
 ei2 , n is number of sample, k is number of parameters which are
nk
estimated . Thus, the Standard errors of ̂ i ' s can be found as follows

se( ˆ0 )  Var ( ˆ0 ) se(ˆ1 )  Var (ˆ1 ) se(ˆ2 )  Var (ˆ2 )

To illustrate consider the following example.

Let 𝑌 = 𝛽̂0 + 𝛽̂1 𝑋1 + 𝛽̂2 𝑋2 + 𝑒𝑖
Hypothesis formulation

1) A) H 0 : 1  0

H 1 : 1  0

B) H0 : 2  0

H1 :  2  0

The null hypothesis (A) states that, holding 𝑋2 constant 𝑋1 has no (linear) influence
on Y. Similarly hypothesis (B) states that holding 𝑋1 constant, 𝑋2 has no influence
on the dependent variable Yi. To test these null hypothesis we will use the
following tests:
I) Standard error test: under this and the following testing methods we
test only for 𝛽̂1 and 𝛽̂2 . 𝑇he test for 𝛽̂0 will be done in the same way.

  x2
2

se(ˆ1 )  ˆ
Var (1 ) = ˆ u  
2

  x1  x2  ( x1 x2) 2 
2 2

  x1
2
 ∑ 𝑒𝑖2
se(ˆ2 )  ˆ
Var ( 2 ) = ˆ u   , where 𝜎
̂ 2
=
2
𝑢 𝑛−3
  x1  x2  ( x1 x2) 2 
2 2

ˆ ˆ
a) If we find that S.E (𝛽̂1 ) > 1 and S.E ( ˆ 2 )'s > 2 (if the estimated S.E is
2 2
greater than half of the estimators).
We don’t reject the null hypothesis and interpret as stated above or

 There is no relationship between Y & X1, X2

21 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

 ˆ1 and ˆ2 are insignificant ( X1 & X2 are not affected Yi )

 ˆ1 , and ˆ2 are equal to zero

 Reject the alternative hypothesis which says the ˆ1 , and ˆ2 are
different from zero
ˆ ˆ
b) If we find that S.E (𝛽̂1 ) < 1 and S.E ( ˆ 2 )'s < 2
2 2
It means if the S.E are less than half of the estimators then we can
interpreter it as follows.
 Reject the null hypothesis which means the value of ˆ1 and ˆ2 is
equal to zero. In other words, accept the alternative means the value
of ˆ1 and ˆ2 are different from zero.
 There is a relationship between the dependent variable Yi and the
independent variables X1 & X2 or X1 & X2 explains Yi
 ˆ1 and ˆ2 are significant
Note: The smaller the standard errors, the stronger the evidence that the
estimates are statistically reliable.
II) The student’s t-test: We compute the t-ratio for each ̂ i ' s

 i  i
t= 
~ 𝑡𝛼(𝑛−𝑘) (i = 0, 1, 2, …., k)
2
S ( i )

This is the observed (or sample) value of the t ratio, which we compare with the
theoretical value of t obtainable from the t-table with n – k degrees of freedom. The
theoretical values of t (at the chosen level of significance) are the critical values
that define the critical region in a two-tail test, with n – k degrees of freedom.

Now let us postulate that

H0:  i = 0

H1:  i  0 or one sided (  i > 0,  i < 0)

The null hypothesis states that, holding X2 constant, X1 has no (linear) influence
on y.
22 | P a g e Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

If the computed t value exceeds the critical t value at the chosen level of

significance, we may reject the null hypothesis; otherwise, we may accept it (  1 is
not significant at the chosen level of significance and hence the corresponding
regression does not appear to contribute to the explanation of the variations in Y).

Look at the following figure

Assume  = 0.05, t = 2.179 for 12 df

2

Acceptance
region

Critical region
95%
Critical region (2.5%)
2.5%

0
Fig 3.7.1. 95% confidence interval for t

Note that the greater the value of t calculated the stronger is the evidence that  i

is significant. For a number of degrees of freedom higher than 8 the critical value
of t (at the 5% level of significance) for the rejection of the null hypothesis is
approximately 2.

3.7.2. Testing the Overall Significance of a Regression

Throughout the previous section we were concerned with testing the significance
of the estimated partial regression coefficients individually, i.e. under the separate
hypothesis that each of the true population partial regression coefficient was zero.

In this section we extend this idea to joint test of the relevance of all the included
explanatory variables. Now consider the following:
𝑌 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 + 𝛽3 𝑋3 + ⋯ + 𝛽𝑘 𝑋𝑘 + 𝑢

23 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

The test of the overall significance of the regression implies testing the null
hypothesis

H0:  1 =  2 = … =  k = 0 Against the alternative hypothesis

H1: not all  i ’s are zero (at least one of the  i ’s is non-zero)

If the null hypothesis is true, then there is no linear relationship between y and
the regressors.
The above joint hypothesis can be tested by the analysis of variance (AOV)
technique. The following table summarizes the idea.

Source of variation Sum of squares Degrees of Mean square

(SS) freedom (Df) (MSS)

Due to regression (ESS)  yˆ 2

i
K–1  yˆ 2

k 1

Due to Residual (RSS) U i

2
N–k U i
2

 k

Total variation (TSS) y 2

i
N–1

The test procedure for any set of hypothesis can be based on a comparison of the
sum of squared errors from the original, the unrestricted multiple regression
model to the sum of squared errors from a regression model in which the null
hypothesis is assumed to be true. When a null hypothesis is assumed to be true,
we in effect place conditions or constraints, on the values that the parameters can
take, and the sum of squared errors increases. The idea of the test is that if these
sum of squared errors are substantially different, then the assumption that the
joint null hypothesis is true has significantly reduced the ability of the model to
fit the data, and the data do not support the null hypothesis.
If the null hypothesis is true, we expect that the data are compliable with the
conditions placed on the parameters. Thus, there would be little change in the
sum of squared errors when the null hypothesis is assumed to be true.
Let the Restricted Residual Sum of Square (RRSS) be the sum of squared errors
in the model obtained by assuming that the null hypothesis is true and URSS be
the sum of the squared error of the original unrestricted model i.e. unrestricted
residual sum of square (URSS). It is always true that RRSS - URSS > 0.

24 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

25 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

(TSS  RSS )
 yˆ k  1  k  1  ESS k  1
2

F= i

U n  k 
i
2
RSS n  k RSS N  k 

Note that there is a relationship between the coefficient of determination R 2 and

the F test used in the analysis of variance. Hence, if we divide the above numerator
and denominator by TSS then:
ESS k  1
R2
k  1  R (n  k )
2
Fcal  TSS 
RSS N  k  (1  R 2 ) n  k (1  R 2 )(k  1)
TSS

where k – 1 refers to degrees of freedom of the numerator

n – k refers to degrees of freedom of the denominator
k – number of parameters estimated
This implies the computed value of F can be calculated either as a ratio of ESS &
TSS or R 2 and 1- R 2 . If the null hypothesis is not true, then the difference between
RRSS and URSS (TSS & RSS) becomes large, implying that the constraints placed
on the model by the null hypothesis have large effect on the ability of the model to
fit the data, and the value of F tends to be large. Thus, we reject the null hypothesis
if the F test static becomes too large. This value is compared with the critical value
of F which leaves the probability of 𝛼 in the upper tail of the F- distribution with
k-1 and n-k degree of freedom.
Decision Rule: If Fcalculated > Ftabulated (F(k – 1, N – k)), reject H0: otherwise you may
accept it, where F(k – 1, N – k) is the critical F value at the  level of significance
and (k – 1) numerator df and (N – k) denominator df.

f(F)

5% of area
F

0 1 2 3 4 5

When R2 = 0, F is zero. The larger the R2, the greater the F value. In the limit, when
R2 = 1, F is infinite. Thus the F test, which is a measure of the overall significance
of the estimated regression, is also a test of significance of R2. Testing the null
26 | P a g e Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

hypothesis is equivalent to testing the null hypothesis that (the population) R 2 is

zero. The F test expressed in terms of R2 is easy for computation.

Importance of F-test

This test is undertaken in multiple regression analysis for the following reasons.

a) To test the overall significance of the explanatory variables i.e. whether the
explanatory variables X1,X2 actually do have influence on the explained
variable Yi or not
b) Test of improvement, means by introducing one additional variable in the
model to test that whether the additional variable will improve the influence
of the explanatory variable on the explained variable or not
Ex. Yi   0  1 X 1   2 X 2  U i

Yi   0  1 X 1   2 X 2   3 X 3  U i

 Now if you take the first equation it has only two explanatory variables X 1
&X2 but in the second we included X3. Then the addition of X3 may affect
positively or negatively the relationship between Y & X1, X2.

C) To test the equality of coefficients obtained from different sample size (chaw
test). Suppose you may have a sample data of agricultural output of North
Shoa Zone from 1974 E.C. up to 1994 E.C. Now if you want to know the
change in agricultural output before & after the fall down of Derge (1983).
By splitting these data in to two you can compare the coefficients & by doing
so you can undertake F-test and see whether there is a change in
agricultural output or not.
D) Testing the stability of the coefficients of the variables as the number of
sample size increase
Ex. You may take first a 10 year sample for your study & estimate your
estimators. But if you increase the number of sample size in to 15 years, will
the coefficients of the variables are stable or not will be tested using F-tests.

27 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

3.7.3. The Confidence Interval for  i

This has been discussed in chapter 2. The (1 - ) 100% confidence interval for  i is

given by

 
 i  t/2 S(  i ), (i = 0, 1, 2, 3, … k)

Example: Suppose we have data on wheat yield (y), amount of rainfall (x2), and
amount of fertilizer applied (X1). It is assumed that the fluctuations in yield can be
explained by varying levels of rainfall and fertilizer.

Table 3.7.1
Yield Fertilizer Rain yi x1i x2i x1i yi x2i yi x1x2
(Y) (X1) fall (X2)

40 100 10

50 200 20

50 300 10

70 400 30

65 500 20

65 600 20

80 700 30

 420 2800 140

Y = 60 X 1 = 400 X 2 = 20 (means)

  
1. Find the OLS estimators (i.e.,  o ,  1 and  2 )
  
Solutions: The formula for  o ,  1 and  2 are

  
 o = Y   1 X1   2 X 2

28 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

  x y  x    x y  x
2
x 2i  Where x’s and y’s are in deviation
1=
1i i 2i 2i i 1i

 x  x    x x 
2
1i
2
2i 1i 2i
2 forms

  x y  x    x y  x
2
x 2i 
 2=
2i i 1i 1i i 1i

 x  x    x x 
2
1i
2
2i 1i 2i
2

Now find the deviations of the observations from their mean values. (Column 4 to
11 in the above table). The next step will be to insert the following values (in
deviation) in to the above formula

x1i yi = 16500, x2i2 = 400, x2iyi = 600, x1i x2i = 7000, x1i2 = 280,000,

 (16500)(400)  (600)(7000) 6600000  4200000 2400000

1 =  =  0.0381
(280,000)(400)  (7000) 2
112000000  49000000 63000000

 (600)(280,000)  (16500)(7000) 168,000,000  115,500,000

 2=  = 0.833
(280,000)(400)  (7000) 2 63,000,000


Now  o = 60 – (0.0381) (400) – (0.833) (20) = 28.1

Hence the estimated function is written as follows

Yˆi = 28.1 + 0.0381X1 + 0.833X2

 
2. Find the variance of  1 ,  2

Solution
 ˆ u2 . x 22  ˆ 2  x22
Var(  1 ) = , Var (  2 ) =
 x  x   x x   x  x   x x 
2 2 2 2 2 2
1 2 1 2 1 2 1 2

In order to use the above formula we need to find ˆ u2

ˆ = 2 e 2
i
but ei = Yi - Yˆ
nk
u

Example. e1 = 40 – 40.24  U12 = (0.24)2 = 0.0576

e2 = y2 - ŷ 2

29 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

e3 = y3 - ŷ 3

: :


Therefore, ei2 = (Yi - Y )2

Y  Yˆ 
i
2

0.0576
5.6644

21.4286

21.4286
Hence u2 = = 5.3532
73

 (5.3572)(400)
Var(  1 ) = = 0.000034
(280,000)(400)  (7000) 2


S(  1 ) = 0.000034 = 0.0058

 (280,000)
Var(  2 ) = (5.3572) = 0.02381
63,000,000


S(  2 ) = 0.02381 = 0.1543

3. Find R2 (the coefficient of determination)

 
 1  x1i yi   2  x2i yi (0.0381)(16,500)  (0.833)(600)
R2 = = = 0.98
y
2
i
1150

Interpretation: 98% of the variation in yield is due to the regression plane (i.e.,
because of variation in the amount of fertilizer and rainfall). The model is a good fit.

4. Test (a) the null hypothesis H0:  1 = 0 against the alternative hypothesis

H1:  1  0

With a significance level  = 0.05

30 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU


 1  1 0.0381
t= 
 = 6.5689 – calculated value
S ( 1 ) 0.0058

ttabulated = t 0.05 (7  3) = 2.78- can be found from the statistical table (t-distribution)
2

Decision: Since tcalculated > ttabulated , we reject H0.

That is  1 is statistically significant. The variable X1, fertilizer significantly affects

yield.

(b) H0:  2 =0

H1:  2 0 with a significance level  = 0.05


 2  2 0.833
tcalculated = 
= = 5.3986
S ( 2 ) 0.1543

ttabulated = t 0.05 (7  3) = 2.78

2


Decision: Since tcalculated > ttabulated , we reject H0.  2 is statistically significant

5. Construct the 95% confidence interval for  1

   
 1 - t/2(n-k) S(  1 ) <  1 <  1 + t/2(n-k) S(  1 ) ,

0.0381- t0.025(4) (0.0058) <  1 < 0.0381+ t0.025(4) (0.0058)

0.0381- 2.78 (0.0058) <  1 < 0.0381+ 2.78 (0.0058)

0.0219<  1 <0.0542

Interpretation: The value of the true population parameter  1 will lie between 0.0219
and 0.0542 in 95 out of 100 cases.

 
Note: The coefficient of X1 and X2 (  1 and  2 ) measures the partial effect. For example

 1 measures the rate of change of Y with respect to X1 while X2 is held constant

6. Test the joint significance of the explanatory variables

31 | P a g e Compiled By: Muhammed Siraj
Econometrics I Lecture Note DBU

H0:  1 = 2 = … = k = 0

H1: at least one 𝛽 ′ 𝑠 is different from zero.

The test statistic is F-statistic

R 2 (k  1) 0.98 (3  1) 0.49
Fcal =    98
(1  R ) N  k  (1  0.98) (7  3) 0.005
2

Assuming that  = 0.05, F0.95, (2, 4) = 6.94

Decision: we reject H0 since Fcal > Ftab. We accept that the regression is significant:
not all  i ’s are zero.

3.8. Partial Correlation Coefficients

In the two variable model we have used the simple correlation coefficient, r, as
measure of the degree of linear association between two variables. For three
variable case we can compute three correlation coefficients: r12 (correlation
between y and x2), r13, and r23 – these are called gross or simple correlation
coefficients, or correlation coefficients of zero order. But, for example, r 12 does not
likely to reflect the true degree of association between Y and X2 in the presence of
X3. Therefore, what we need is a correlation coefficient that is independent of the
influence, if any, of X3 on X2 and Y. Such a correlation coefficient can be obtained
and is known appropriately as the partial correlation coefficient. Conceptually, it
is similar to the partial regression coefficient.

r12.3 = partial correlation coefficient between Y and X2, holding X3 constant

r13.2 = partial correlation coefficient between Y and X3, holding X2 constant

r23.1 = partial correlation coefficient between X2 and X3, holding Y constant

Now we can compute the partial correlations from the simple or zero order,
correlation coefficients as follows.

r12  r13 r23

r12.3 =
(1  r132 )(1  r232 )

32 | P a g e Compiled By: Muhammed Siraj

Econometrics I Lecture Note DBU

r12  r12 r23

r13.2 =
(1  r122 )(1  r232 )

r23  r12 r13

r23.1 =
(1  r122 )(1  r132 )

Note: By order we mean the number of secondary subscripts

Interpretation: Example r12.3 – holding X3 constant, there is a positive or negative

association between Y and X2.

33 | P a g e Compiled By: Muhammed Siraj