Econometrics

AEA 309
Mutabazi, K.D.S. (PhD)

Lectures 4

1
 Testing Hypothesis related with regression
coefficient
• Hypothesis testing forms a major part of the foundation of
econometrics and it is essential to have a clear
understanding of the theory
• We have a random variable X with unknown population
mean µ and variance σ2. Given a sample of data, we use the
sample mean as an estimator of µ
• In the context of the regression model, we have unknown
parameters b1 and b2 and we have derived estimators ̂ and
1 ̂ 2
for them. In what follows, we shall focus on b2 and its
estimator ̂ .
2

2
 Testing Hypothesis in regressions

• In the case of the random variable X, our standard null

hypothesis was that µ was equal to some specific value µ0. In
the case of the regression model, our null hypothesis is that
b2 is equal to some specific value .  0
2

• For both the population mean m of the random variable X and

the regression coefficient b2, the test statistic is a t statistic
X  0 ˆ2   20
t t
s.e.  X  s.e.  ˆ2 
• In both cases, the t statistic it is defined as the difference
between the estimated coefficient and its hypothesized value,
divided by the standard error of the coefficient
• We reject the null hypothesis if the absolute value is greater
than the critical value of t ( t  t crit ) given the chosen 3
significance level
 Testing Hypothesis in regressions
• There is one important difference. When locating
the critical value of t, one must take account of the
number of degrees of freedom.
– In the case of the random variable X, this is n – 1, where
n is the number of observations in the sample
– In the case of the regression model, the number of
degrees of freedom is n – k, where n is the number of
observations in the sample and k is the number of
parameters (b coefficients). For the simple regression
model with 2 parameters, it is n – 2

4
 Testing Hypothesis in regressions

• As an illustration, we will consider a model relating price

inflation to wage inflation. p is the percentage annual rate of
growth of prices and w is the percentage annual rate of
growth of wages.
p = b1 + b2w + u
H0: b2 = 1.0
H1: b2 ≠ 1.0
• We will test the hypothesis that the rate of price inflation is
equal to the rate of wage inflation. The null hypothesis is
therefore H0: b2 = 1.0. (We should also test b1 = 0.)
5
 Testing Hypothesis in regressions
pˆ 1.21  0.82w
(0.05) (0.10)
Suppose that the regression result is as shown (standard errors
in parentheses).
Our actual estimate of the slope coefficient is only 0.82.
We will check whether we should reject the null hypothesis.
ˆ2   20 0.82  1.00
t   1.80.
s.e.  ˆ2  0.10
We compute the t statistic by subtracting the hypothetical true
value from the sample estimate and dividing by the standard
error. It comes to –1.80. 6
 Testing Hypothesis in regressions

n 20 degrees of freedom 18 t crit, 5% 2.101

There are 20 observations in the sample. We have estimated

2 parameters, so there are 18 degrees of freedom
The critical value of t with 18 degrees of freedom is 2.101 at
the 5% level. The absolute value of the t statistic is less
than this, so we do not reject the null hypothesis
In this case it is usual to define b2 = 0 as the null hypothesis.
In words, the null hypothesis is that X does not influence Y.
We then try to demonstrate that the null hypothesis is false.
7
 Testing Hypothesis in regressions
• For the null hypothesis b2 = 0, the t statistic reduces to the
estimate of the coefficient divided by its standard error
ˆ
2  2 0
ˆ2
t 
 
s.e. ˆ2  
s.e. ˆ2
• This ratio is commonly called the t statistic for the
coefficient and it is automatically printed out as part of the
regression results. To perform the test for a given
significance level, we compare the t statistic directly with
the critical value of t for that significance level.

8
. reg EARNINGS S
----------------------------------------------------------------------------
Source | SS df MS Number of obs = 500
-----------+------------------------------ F( 1, 498) = 46.57
Model | 6014.04474 1 6014.04474 Prob > F = 0.0000
Residual | 64314.9215 498 129.146429 R-squared = 0.0855
-----------+------------------------------ Adj R-squared = 0.0837
Total | 70328.9662 499 140.939812 Root MSE = 11.364
----------------------------------------------------------------------------
EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
S | 1.265712 .1854782 6.82 0.000 .9012959 1.630128
_cons | .7646844 2.803765 0.27 0.785 -4.743982 6.273351
----------------------------------------------------------------------------

You can see that the t statistic for the coefficient of S is enormous.
We would reject the null hypothesis that schooling does not affect
earnings at the 1% significance level (critical value about 2.59) 9
 Testing Hypothesis in regressions
• In this case we could go further and reject the null hypothesis
that schooling does not affect earnings at the 0.1% significance
level
• The advantage of reporting rejection at the 0.1% level, instead
of the 1% level, is that the risk of mistakenly rejecting the null
hypothesis of no effect is now only 0.1% instead of 1%. The
result is therefore even more convincing
• The next column in the output gives what are known as the p
values for each coefficient. This is the probability of obtaining
the corresponding t statistic as a matter of chance, if the null
hypothesis H0: b = 0 is true
• If you reject the null hypothesis H0: b = 0, this is the probability
that you are making a mistake and making a Type I error. It
therefore gives the significance level at which the null 10
hypothesis would just be rejected
 Testing Hypothesis in regressions

• If p = 0.05, the null hypothesis could just be rejected at the 5%

level. If it were 0.01, it could just be rejected at the 1% level. If it
were 0.001, it could just be rejected at the 0.1% level. This is
assuming that you are using two-sided tests
• In the present case p = 0 to three decimal places for the coefficient
of S. This means that we can reject the null hypothesis H0: b2 = 0
at the 0.1% level, without having to refer to the table of critical
values of t. (Testing the intercept does not make sense in this
regression
• The use of p values is a more informative approach to reporting
the results of tests. It is widely used in the medical literature
• However, in economics, standard practice is to report results
referring to 5% and 1% significance levels, and sometimes to the
0.1% level (when one can reject at that level).
11
 Testing Hypothesis in regressions

• Returning to the price inflation/wage inflation model, we saw that we

could not reject the null hypothesis b2 = 1, even at the 5%
significance level. That was using a two-sided test

n 20 degrees of freedom 18 t crit, 5% 2.101 two - sided test 

• However, in practice, improvements in productivity may cause the
rate of cost inflation, and hence that of price inflation, to be lower
than that of wage inflation

12
 Testing Hypothesis in regressions

• Certainly, improvements in productivity will not cause price

inflation to be greater than wage inflation and so in this case
we are justified in ruling out b2 > 1. We are left with H0: b2 = 1
and H1: b2 < 1.
• Thus we can perform a one-sided test, for which the critical
value of t with 18 degrees of freedom at the 5% significance
level is 1.73. Now we can reject the null hypothesis and
conclude that price inflation is significantly lower than wage
inflation, at the 5% significance level

n 20 degrees of freedom 18 t crit, 5% 1.734 one - sided test 

13
 Testing Hypothesis in regressions

• Now we will consider the special, but very common, case H0:
b2 = 0.
• It occurs when you wish to demonstrate that a variable X
influences another variable Y. You set up the null
hypothesis that X has no effect (b2 = 0) and try to reject H0.
Null hypothesis: H0: b2 = 0
Alternative hypothesis: H1: b2 ≠ 0

14
 Two sided t test
probability density
function of ̂ 2

reject H0 do not reject H0 reject H0

2.5% 2.5%

–1.96 0 1.96 sd ̂ 2
sd

The figure shows the distribution of ̂ , conditional on H0: b2 = 0 being

2

true. For simplicity, we initially assume that we know the standard

15
deviation
Testing Hypothesis in regressions
• If you use a two-sided 5% significance test, your estimate
must be 1.96 standard deviations above or below 0 if you are
to reject H0.
• However, if you can justify the use of a one-sided test, for
example with H0: b2 > 0, your estimate has to be only 1.65
standard deviations above 0.
• This makes it easier to reject H0 and thereby demonstrate
that Y really is influenced by X (assuming that your model is
correctly specified).

16
 Testing Hypothesis in regressions
Null hypothesis: H0: b2 = 0
Alternative hypothesis: H1: b2 > 0
probability density
function of ̂ 2

do not reject H0 reject H0

5%

0 1.65 sd ̂ 2
17
 Confidence Interval
• We can then obtain the confidence interval for b2, being the
set of all values that would not be rejected, given the sample
estimate ̂ 2 . To make it operational, we need to select a
significance level and determine the corresponding critical
value of t.
ˆ2   20 ˆ2   20
Reject H0 if  tcrit or   tcrit
s.e.   2 
ˆ s.e.   2 
ˆ

Reject H0 if ˆ2   20  s.e.  ˆ2  tcrit or ˆ2   20   s.e.  ˆ2  tcrit

Reject H0 if ˆ2  s.e.  ˆ2  tcrit   20 or ˆ2  s.e.  ˆ2  tcrit   20

Do not reject H0 if ˆ2  s.e.  ˆ2  tcrit   2  ˆ2  s.e.  ˆ2  tcrit
 Confidence Interval
. reg EARNINGS S
----------------------------------------------------------------------------
Source | SS df MS Number of obs = 500
-----------+------------------------------ F( 1, 498) = 46.57
Model | 6014.04474 1 6014.04474 Prob > F = 0.0000
Residual | 64314.9215 498 129.146429 R-squared = 0.0855
-----------+------------------------------ Adj R-squared = 0.0837
Total | 70328.9662 499 140.939812 Root MSE = 11.364
----------------------------------------------------------------------------
EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------+----------------------------------------------------------------
S | 1.265712 .1854782 6.82 0.000 .9012959 1.630128
_cons | .7646844 2.803765 0.27 0.785 -4.743982 6.273351
----------------------------------------------------------------------------

ˆ2  s.e.  ˆ2  tcrit   2  ˆ2  s.e.  ˆ2  tcrit

1.266 – 0.185 x 1.965 ≤ b2 ≤ 1.266 + 0.185 x 1.965
0.902 ≤ b2 ≤ 1.630
Hence we establish that the confidence interval is from 0.902 to 1.630. Stata actually
computes the 95% confidence interval as part of its default output, 0.901 to 1.630. The
discrepancy in the lower limit is due to rounding error in the calculations we have
made. 19
 Analysis of Variance

• The analysis of variance (ANOVA) is a method

to test for the significance of regression.
• ANOVA uses the variance of the observed data
to determine if a regression model can be
applied to the observed data
• The observed variance is partitioned into
components that are then used in the test for
significance of regression
20
 Analysis of Variance

Sum of Squares
• The total variance (i.e. the variance of all of the
observed data) is estimated using the observed
data.
• The variance of a population can be estimated
using the sample variance, which is estimated
using the following relationship:

Equation 1

21
 Analysis of Variance

• The quantity in the numerator of this equation is called the sum of

squares (Eq. 1)
• It is the sum of the square of deviations of all the observations,
from their mean
• This quantity is called the total sum of squares (abbreviated SST)
because it relates to the total variance of the observations
• The denominator is the number of degrees of freedom
associated with the sample variance. Therefore, the number of
degrees of freedom associated with SST is n-1. The sample
variance is also referred to as a mean square because it is
obtained by dividing the sum of squares by the respective
degrees of freedom 22
 Analysis of Variance
• If the regression model is such that the resulting fitted regression
line passes through all of the observations, then we have a
"perfect" model. In this case the model would explain all of the
variability of the observations.
• Therefore, the model sum of squares (also referred to as the
regression sum of squares equals the total sum of squares; i.e.
the model explains all of the observed variance (see figure
below)

23
Analysis of Variance
• A perfect regression model will pass
through all observed data points as
shown in (a). Most models are
imperfect and do not fit perfectly to
all data points as shown in (b)

24
 Analysis of Variance
• For the perfect model, the regression sum of squares (SSR),
equals the total sum of squares, because all estimated values will
equal the corresponding observations.
• SSR can be calculated using a relationship similar to the one for
obtaining SST by replacing yi with ŷ in the relationship of SST.
Therefore:

25
 Analysis of Variance
• The number of degrees of freedom associated with SSR is K
minus one. Based on the preceding discussion, a perfect
regression model exists when the fitted regression line passes
through all observed points
• However, this is not usually the case. Part of the total variability
of the observed data still remains unexplained. This portion of the
total variability or the total sum of squares, which is not explained
by the model, is called the residual sum of squares or the error
sum of squares (abbreviated SSE).

26
• The deviation for this sum of squares is obtained at
each observation in the form of the residuals, ei . The
error sum of squares can be obtained as the sum of
squares of these deviations:

27
 Analysis of Variance
• The number of degrees of freedom associated
with SSE, is n minus k. The total variability of the
observed data (i.e. total sum of squares, SST)
can be written using the portion of the variability
explained by the model, SSR, and the portion
unexplained by the model, SSE, as:

28
 Analysis of Variance
• If given the regression sum of squares equal to
25,604.8 and the error sum of squares is 319.5. The R2
is calculated as:

29
 F-Test of Goodness of fit
Model Y = b1 + b2X + u

 Yˆ  Y    uˆ 2
2
 Y  Y 
2
TSS  ESS  RSS

In an earlier sequence it was demonstrated that the sum of the

squares of the actual values of Y (TSS: total sum of squares)
could be decomposed into the sum of the squares of the fitted
values (ESS: explained sum of squares) and the sum of the
squares of the residuals

30
 F-Test of Goodness of fit
 Yˆi  Y 
2
ESS
R2  
TSS  Yi  Y 2

R2, the usual measure of goodness of fit, was then defined to

be the ratio of the explained sum of squares to the total sum
of squares

Null hypothesis: H0: b2 = 0

Alternative hypothesis: H1: b2 ≠ 0
Since X is the only explanatory variable at the moment, the null
hypothesis is that Y is not determined by X. Mathematically, we
have H0: b2 = 0 31
 F-Test of Goodness of fit
ESS
ESS k  1 TSS k  1 R 2 k  1 
F k  1, n  k    
RSS n  k  RSS
n  k    n  k 
2
1  R
TSS

• Hypotheses concerning goodness of fit are tested via the F statistic,

defined as shown. k is the number of parameters in the regression
equation, which at present is just 2
• n – k is, as with the t statistic, the number of degrees of freedom
(number of observations less the number of parameters
estimated). For simple regression analysis, it is n – 2.
32
 F-Test of Goodness of fit
ESS
ESS k  1 TSS k  1 R 2 k  1 
F k  1, n  k    
RSS n  k  RSS
n  k    n  k 
2
1  R
TSS

The F statistic may alternatively be written in terms of R2. First

divide the numerator and denominator by TSS

33
 F-Test of Goodness of fit
ESS
ESS k  1 TSS k  1 R 2 k  1 
F k  1, n  k    
RSS n  k  RSS
n  k    n  k 
2
1  R
TSS
• We can now rewrite the F statistic as shown. The R2 in the
numerator comes straight from the definition of R2.
• It is easily demonstrated that RSS/TSS is equal to 1 – R2.

R 2 k  1
F k  1, n  k  
1  R 2  n  k 
R2 1
F 1,18  
1  R 2  18
Fcrit ,5% 1,18  4.41 34
 Analysis of Variance
• Moreover, if we are to test

• The statistic used is based on the F - distribution. The null

hypothesis is rejected when the calculated statistic is greater than
the F critical (from the tables)
• Assuming that the desired significance is 0.01, since the p-value is
less than 0.01, the null hypothesis is rejected, implying that a
relation does exist between the explanatory variables and
investment 35
 Analysis of Variance
• We then repeated the regression this time round replacing real
interest with nominal interest rates. The results are shown in
table below:

Regression Results for Investment, GNP, Money Supply and Interest

36
 Analysis of Variance
• In this case, the regression equation can be
written as:

* Indicates significance at a 5% level

37