11/14/2021

Chapter 4

MULTIPLE REGRESSION –
INFERENCE
Wooldridge: Introductory Econometrics:
A Modern Approach, 5e

Lecturer: Hoàng Thị Diễm Hương

Content

1. Sampling distributions of the OLS estimators

2. Testing hypotheses about a single population parameter
3. Confidence intervals
4. Testing hypotheses about a single linear combination of
the parameters
5. Testing multiple linear restrictions

1
 11/14/2021

A review of probability distributions

 Normal distribution:
 Definition: A continuous random variable X whose values lie in (-; +) is said to
be normally distributed if its probability density function is of the form:
(x - μ) 2 0.45

1 -
2σ 2
0.4
f(x) = .e 0.35

σ 2π 0.3

Notation: X ~ N(; 2). 0.25

0.2
 Properties: 0.15

• E(X) = ; Mod(X) =  0.1

0.05

Var(X) = 2 -2 -1
0
0 1 2 3 4 5 6

• If random variables X1,…, Xn are independent (for any n) and X1 ~ N(1; 12);
X2 ~ N(2; 22);…; Xn ~ N(n; n2), then:
n n n

X
i=1
i = X1 +X 2 +...+X n ~ N(  μ , σ
i=1
i
i=1
2
i )

A review of probability distributions

 Normal distribution :
 Properties:
• Central limit theorem: If X1,…, Xn are independent, identically distributed and
E(Xi) = ; Var(Xi) = 2, then for large n (n > 30), we have the approximation:
n

X
i=1
i ~ N( nμ; nσ 2 )
• If X ~ N(; 2) and k is a non-zero constant, then: kX ~ N(k; k22)
 Other distributions:
n
 Chi-squared distribution: If Zi ~ N(0;1) (i = 1, n), then: Z i=1
2
i ~ χ 2 (n)

Z
 Student’s t-distribution: If Z ~ N(0;1) and X ~ 2(m), then: ~ tm
X/m
X1 /m
 Fisher distribution: If X1 ~ 2(m) and X2 ~ 2(n), then: ~ F(m, n)
X 2 /n
4

2
 11/14/2021

1. Sampling distributions of the OLS estimators

 Assumption MLR.6: Normality
The population error u is independent of the explanatory variables x1, x2,..., xk and
normally distributed with zero mean and variance 2:
ui  N (0; 2 ); i  1, n
 yi | xi1 , xi 2 ,..., xik  N (  0  1 x1  ...   k xk ; 2 ); i  1, n

It is assumed that the unobserved factors are

normally distributed around the population
regression function.
The form and the variance of the distribution
does not depend on any of the explanatory
variables.

1. Sampling distributions of the OLS estimators

 Discussion of the normality assumption:
 Why can normality be assumed? Because the error term is the sum of „many“
different unobserved factors. Following the central limit theorem (CLT), sums of
independent factors are normally distributed.
 Problems:
• How many different factors? Number large enough?
• Possibly very heterogenuous distributions of individual factors.
• How independent are the different factors?
 The normality of the error term is an empirical question.
 In many cases, normality is questionable or impossible by definition.
 Examples where normality cannot hold:
• Wages (nonnegative; also: minimum wage)
• Number of arrests (takes on a small number of integer values)
• Unemployment (indicator variable, takes on only 1 or 0)

3
 11/14/2021

1. Sampling distributions of the OLS estimators

 Discussion of the normality assumption:
 In some cases, normality can be achieved through transformations of the
dependent variable (e.g. use log(wage) instead of wage).
 Under normality, OLS is the best (even nonlinear) unbiased estimator.
 Important: For the purposes of statistical inference, the assumption of normality
can be replaced by a large sample size.
 Theorem 4.1: Normal sampling distributions
Under Assumptions MLR.1 through MLR.6:
 
ˆ j  N  j ;Var( ˆ j ) ; j  1, k The estimators are normally distributed around
the true parameters, with the variance that was
Therefore: derived earlier
ˆ j   j
 N  0;1 ; j  1, k The standardized estimators follow a standard
sd ( ˆ j ) normal distribution

2. Testing hypotheses about a single population parameter

• Testing hypotheses about a single population parameter:
– Consider the multiple regression model: y   0  1 x1  ...   k xk  u
– If j = 0 then xj has no effect on y  To know whether xj has an effect on y, we
need to use statistical inference to test the hypothesis H0: j = 0
– In order to construct hypotheses tests, we need the following result:
Theorem 4.2: t-distribution for standardized estimators
Under Assumptions MLR.1 through MLR.6:
ˆ j   j If the standardization is done using the estimated
~ tn( k 1) ; j  1, k standard deviation (= standard error), the normal
se( ˆ j ) distribution is replaced by a t-distribution
where (k+1) is the number of parameters in the population model and df = [n-(k+1)]
is the degrees of freedom.
– Note: The t-distribution tn-(k+1) is close to the standard normal distribution N(0;1) if
n-(k+1) is large.

4
 11/14/2021

2. Testing hypotheses about a single population parameter

• Testing hypotheses about a single population parameter:
Null hypotheses: H0:  j = 0
ˆ j
– Step 1: Calculate t statistic: t ˆ 
j
se( ˆ j )
– Step 2:
If H1:  j ≠ 0 If H1:  j < 0 If H1:  j > 0
 For a given significant  For a given significant  For a given significant
level , find the critical level , find the critical level , find the critical
value t/2(n – k – 1). value t(n – k – 1). value t(n – k – 1).
 Reject H0 if |t| > t/2.  Reject H0 if t < -t.  Reject H0 if t > t.

2. Testing hypotheses about a single population parameter

• Example: Wage equation
Test whether, after controlling for education and tenure, higher work experience
leads to higher hourly wages?

Standard errors
Test H0: exper = 0 One would either expect a positive effect of experience on
Against H1: exper > 0 hourly wage or no effect at all.
t statistic
Degrees of freedom
Critical values for the 5% and the 1% significance level (these
are conventional significance levels). The null hypothesis is
rejected because the t-statistic exceeds the critical value.
„The effect of experience on hourly wage is statistically greater than zero at the 5%
(and even at the 1%) significance level.“
10

5
 11/14/2021

2. Testing hypotheses about a single population parameter

• Guidelines for discussing economic and statistical significance:
– If a regression coefficient is different from zero in a two-sided test, the
corresponding variable is said to be „statistically significant“.
– If a variable is statistically significant, discuss the magnitude of the coefficient to
get an idea of its economic or practical importance.
– The fact that a coefficient is statistically significant does not necessarily mean it is
economically or practically significant!
– If a variable is statistically and economically important but has the „wrong“ sign,
the regression model might be misspecified.
– If a variable is statistically insignificant at the usual levels (10%, 5%, 1%), one
may think of dropping it from the regression.
– If the sample size is small, effects might be imprecisely estimated so that the case
for dropping insignificant variables is less strong.

11

2. Testing hypotheses about a single population parameter

• Testing more general hypotheses about a regression coefficient:
Null hypotheses: H0:  j = aj

ˆ j  a j
– Step 1: Calculate t statistic: t
se( ˆ j )
– Step 2 :

If H1:  j ≠ aj If H1:  j < aj If H1:  j > aj

 For a given significant  For a given significant  For a given significant
level , find the critical level , find the critical level , find the critical
value t/2(n – k – 1). value t(n – k – 1). value t(n – k – 1).
 Reject H0 if |t| > t/2.  Reject H0 if khi t < -t.  Reject H0 if t > t.

12

6
 11/14/2021

2. Testing hypotheses about a single population parameter

• Example: Campus crime and enrollment
– An interesting hypothesis is whether crime increases by 1% if enrollment is
increased by 1%.

Estimate is different from 1 but is this

difference statistically significant?

The hypothesis is rejected

at the 5% level

13

2. Testing hypotheses about a single population parameter

• Computing p-values for t-tests
– If the significance level is made smaller and smaller, there will be a point where
the null hypothesis cannot be rejected anymore. The reason is that, by lowering
the significance level, one wants to avoid more and more to make the error of
rejecting a correct H0.
– The smallest significance level at which the null hypothesis is still rejected, is
called the p-value of the hypothesis test.
– A small p-value is evidence against the null hypothesis because one would reject
the null hypothesis even at small significance levels.
– A large p-value is evidence in favor of the null hypothesis.
– p-values are more informative than tests at fixed significance levels.

14

7
 11/14/2021

2. Testing hypotheses about a single population parameter

• How the p-value is computed (here: two-sided test)
The p-value is the significance level at
which one is indifferent between rejecting
These would be
and not rejecting the null hypothesis.
the critical values
In the two-sided case, the p-value is thus
for a 5%
the probability that the t-distributed
significance level variable takes on a larger absolute value
than the realized value of the test
statistic, e.g.:
P (| t  ratio | 1.85)  2  0.0359  0.0718
From this, it is clear that a null hypothesis
is rejected if and only if the corresponding
p-value is smaller than the significance
value of test level.
statistic For example, for a significance level of
5% the t-statistic would not lie in the
rejection region.
15

3. Confidence intervals
• The confidence interval (CI) for a population parameter:
The 100(1-)% confidence interval (CI) for a population parameter j is:


 j  ˆ j  t / 2 .se( ˆ j ); ˆ j  t /2 .se( ˆ j ) 
• Interpretation of the confidence interval:
– The bounds of the interval are random.
– 100(1-)% is called the confidence level.
– In repeated samples, the interval that is constructed in the above way will cover
the population regression coefficient in 100(1-)% of the cases
• Relationship between confidence intervals and hypotheses tests
If aj  Confidence interval then we reject H0: j = aj in favor of H1: j ≠ aj

16

8
 11/14/2021

3. Confidence intervals
• Example: Model of firms‘ R&D expenditures
Spending on R&D Annual sales Profits as percentage of sales

log( rd )  4.38  1.084 log( sales)  0.0217 profmarg
(0.47) (0.060) (0.0128)
n  32; R 2  0.918; df  32  2  1  29  t0.025 (29)  2.045
1.084  2.045  0.060 0.0217  2.045  0.0128
 (0.961; 1.21)  (0.0045; 0.0479)

The effect of sales on R&D is relatively This effect is imprecisely estimated

precisely estimated as the interval is narrow. as the interval is very wide. It is not
Moreover, the effect is significantly different even statistically significant because
from zero because zero is outside the interval. zero lies in the interval.

17

4. Testing hypotheses about a single linear combination of

the parameters
• Example: Return to education at 2 year vesus at 4 year colleges
Years of education Years of education
at 2 year colleges at 4 year colleges

Test H0: 1 - 2 = 0 against H1: 1 - 2 < 0.

A possible test statistic would be:

Usually not available in regression output

18

9
 11/14/2021

4. Testing hypotheses about a single linear combination of

the parameters
• Example: Return to education at 2 year vesus at 4 year colleges
Years of education Years of education
at 2 year colleges at 4 year colleges

Test H0: 1 - 2 = 0 against H1: 1 - 2 < 0.

Alternative method: Define 1 = 1 - 2 and test H0: 1 = 0 against H1: 1 < 0

A new regressor (= total years of college)

19

5. Testing multiple linear restrictions

• Testing exclusion restrictions:
Suppose that we have the population model (which is called unrestricted model):
y   0  1 x1  ...   q xq   q 1 xq 1  ...   k xk  u
We want to test:
H0: 1 = 2 = ... = q = 0  Restricted model:
H1: There exists j ≠ 0 (j = 1, 2,..., q) y   0   q 1 xq 1  ...   k xk u
 Procedure for testing:
– Step 1: Find the unrestricted model and calculate SSRUR.
– Step 2: Find the restricted model and calculate SSRR.
– Step 3: Calculate:
( SSRR  SSRUR ) / q 2
( RUR  RR2 ) / q
F  ~ F (q,(n  (k  1)))
SSRUR / ( n  (k  1)) (1  RUR
2
) / ( n  (k  1))
– Step 4: For a given significant level , find F (q, (n-(k+1))). Reject H0 if F > F.

20

10
 11/14/2021

5. Testing multiple linear restrictions

• Testing exclusion restrictions:
Example: Major league baseball players‘ salaries
Salary of major league Years in the Average number of
baseball player league games per year

Test whether performance

measures have no effect/can
be exluded from regression Batting average Home runs per year Runs batted
in per year
against
– Estimation of the unrestricted model

None of these variables

is statistically significant
when tested individually

21

5. Testing multiple linear restrictions

• Testing exclusion restrictions:
Example: Major league baseball players‘ salaries
– Estimation of the restricted model

( SSRR  SSRUR ) / q
F ~ F (q,( n  (k  1)))
SSRUR / (n  ( k  1))
A F-distributed variable only takes on positive values.
This corresponds to the fact that the sum of squared
residuals can only increase if one moves from H1 to
H0.
Choose the critical value so that the null hypothesis
is rejected in, for example, 5% of the cases, although
it is true.
22

11
 11/14/2021

5. Testing multiple linear restrictions

• Testing exclusion restrictions:
Example: Major league baseball players‘ salaries
Number of restrictions to be tested

Degrees of freedom in the unrestricted model

The null hypothesis is overwhelmingly
rejected (even at very small significance
levels).
• Discussion
– The three variables are „jointly significant“
– They were not significant when tested individually
– The likely reason is multicollinearity between them

23

5. Testing multiple linear restrictions

• Test of overall significance of a regression:
Suppose that we have the population model:
y   0  1 x1  ...   k xk  u
Null hypothesis: H0: 1 = 2 = ... = k = 0
H1: There exists j ≠ 0 (j = 1, 2,..., k)
2
Or: H0: R = 0
H1: R2 > 0
 Procedure for testing:
– Step 1: Calculate:
R2 / k
F ~ F (k ,(n  (k  1)))
(1  R 2 ) / (n  (k  1))
– Step 2: For a given significant level , find F(k, (n-(k+1))). Reject H0 if F>F.
• The test of overall significance is reported in most regression packages; the
null hypothesis is usually overwhelmingly rejected.

24

12
 11/14/2021

5. Testing multiple linear restrictions

• Testing general linear restrictions with the F-test:
Example: Test whether house price assessments are rational
The assessed housing value
Actual house price (before the house was sold) Size of lot (in feet)

Square footage Number of bedrooms

In addition, other known factors
should not influence the price once
If house price assessments are rational, a 1% the assessed value has been
change in the assessment should be associated controlled for.
with a 1% change in price.

25

5. Testing multiple linear restrictions

• Testing general linear restrictions with the F-test:
– Unrestricted regression

– Restricted regression
The restricted model is actually a regression of
[y-x1] on a constant
– Test statistic

 H0 cannot be rejected.

26

13