Econometrı́a I

Multiple regression analysis: Inference

UPB

Semestre II - 2022

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 1 / 15

 Multiple regression

Classical Linear Model (CLM)

OLS, BLUE
Add another assumption, Normality
u ∼ Normal(0, σ 2 )
The population assumption

y |x ∼ Normal(β0 + β1 x1 + ... + βk xk , σ 2 )

Homoskedastic
Sometimes normality is not the case, but large samples will let us drop
Normality:
βj ∼ Normal[βj , Var (β̂j )]
β̂j − βj
sd(β̂j )

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 2 / 15

 Multiple regression

Testing Hypotheses Single Population Parameter: t test

β̂j − βj
= tdf
se(β̂j )n−k−1

Assume t distribution
σ → σ̂
Testing the null hypothesis
Ho : β j = 0

If accepting null, then accept that xj has no effect on y , controlling

for other x’s

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 3 / 15

 Multiple regression

Testing Hypotheses Single Population Parameter: t test

β̂j
tβ̂j ≡
se(β̂j )

Rejection rule
It has the same sign as the estimator
The larger the estimator, the larger the t
Measures how many estimated standard deviations β̂j is away from
zero.

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 4 / 15

 Multiple regression

t Test: Alternatives

Besides our null, H0 , we need an alternative hypothesis, H1 , and a

significance level
H1 may be one-sided, or two-sided
One sided: H1 : βj > 0 and H1 : βj < 0
Two sided: H1 : βj ̸= 0
If we want to have only a 5% probability of rejecting H0 if it is really
true, then we say our significance level is 5

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 5 / 15

 Multiple regression

Testing one-sided alternative

yi = β0 + β1 xi1 + .... + βk xik + ui

Having picked a significance level α, we look up the (1–α)t h

percentile in a t distribution with n–k–1 df and call this c, the critical
value
H0 : β j = 0 H0 : β j = 0
H1 : β j < 0 H1 : β j > 0
tβ̂j < −c tβ̂j > c

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 6 / 15

 Multiple regression

Testing two-sided alternative

yi = β0 + β1 xi1 + .... + βk xik + ui

Having picked a significance level α, we look up for both of the

(1–α/2)th percentile in a t distribution with n–k–1 df and call this c,
the critical value
H0 : βj = 0
H1 : βj ̸= 0
|tβ̂j | > c

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 7 / 15

 Multiple regression

Testing other hypotheses

yi = β0 + β1 xi1 + .... + βk xik + ui

Having picked a significance level α, we look up for both of the

(1–α/2)th percentile in a t distribution with n–k–1 df and call this c,
the critical value
H0 : βj = aj
aj , Hypothesized value
β̂j −aj
t=
se β̂j
(Estimate−Hypothesized value)
t= Standard error

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 8 / 15

 Multiple regression

Computing p-values for t tests

What is the smallest significance level at which the null hypothesis would
be rejected?
P − value = (|T | > |t|)

T , t distributed random variable with n − k − 1 degrees of freedom

t, numerical value of the test statistic
The p-value is the probability of observing a t statistic as extreme as we
did if the null hypothesis is true.

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 9 / 15

 Multiple regression

Confidence Intervals

Under CLM assumptions, the 95% confidence interval for the population
parameter βj is given by:

[β̂j − tdf ,2.5% ∗ se(β̂j ), β̂j + tdf ,2.5% ∗ se(β̂j )]

Under repeated sampling from the same population, in 95% of the

samples, the interval we compute will contain the true value βj .

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 10 / 15

 Multiple regression

Testing a single linear combination of the parameters

log (wage) = β0 + β1 jc + β2 univ + β3 exper + u

jc, number of years attending a two-year college.

univ , number of years at a four-year college.
exper , months in the workforce.

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 11 / 15

 Multiple regression

Testing a single linear combination of the parameters

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 12 / 15

 Multiple regression

Testing multiple linear restrictions: F test

H0 : βk−q+1 = 0, ...., βk = 0
H1 : Not true
Can’t just check each t statistic separately
We want to know if the q parameters are jointly significant
Estimate
1 Restricted model
2 Unrestricted model
(SSRr − SSRu )/q
F ≡
SSRu /(n − k − 1)

We want to know if the change in SSR is big enough to warrant

inclusion of xk−q , ..., xk

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 13 / 15

 Multiple regression

F statistic

SSRr > SSRu

F statistic, measures the relative increase in SSR when moving from
the unrestricted to a restricted model
q = dfr − dfu , number of restrictons
Sampling distribution ,F
F ∼ Fq,n−k−1
q, numerator degreees
n − k − 1, denominator degrees
R 2 form of the F statistic:
(Rr− Ru2 )/q
F ≡
1 − Ru2 /(n − k − 1)

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 14 / 15

 Multiple regression

F statistic, particular cases

Overall significance
R 2 /k
F ≡
1 − R 2 /(n − k − 1)

H0 : β1 = ... = βk = 0
Close relation to the goodness of fit of the original model
R 2 large, our independent variables do a good job of explaining the
variation in the dependent variable and hence we reject the null
Single linear restriction
F statistic equals the square of the t
p-values will be the same

Multiple regression analysis: Inference (UPB) Econometrı́a I Semestre II - 2022 15 / 15