Outline

1. Some basic ideas

2. The problem of estimation: OLS method
3. Classical Normal Linear Regression Model
(CNLRM)
4. Interval estimation and hypothesis testing
5. Extensions of the two variable linear regression
model

10/24/2017 Mai VU-FIE-FTU 2

4. Interval estimation and hypothesis
testing
4.1. Interval estimation: some basic ideas
4.2. Confidence intervals for regression coefficients
4.3. Confidence interval for σ2
4.4. Hypothesis testing

10/24/2017 Mai VU-FIE-FTU 3

4.1. Interval estimation: some basic ideas

 Because of sampling fluctuations, a single estimate is
likely to differ from the true value, although in
repeated sampling its mean value is expected to be
equal to the true value (E(𝛽መ2 ) = β2). Then in statistics
the reliability of a point estimator is measured by its
standard error.
 Therefore, instead of relying on the point estimate
alone, we may construct an interval around the point
estimator, such that this interval has the probability of
including the true parameter value.
 This is roughly the idea behind interval estimation.

10/24/2017 Mai VU-FIE-FTU 4

4.1. Interval estimation: some basic ideas
 To be more specific, assume that we want to find out
how “close” is, 𝛽መ2 to β2.
 For this purpose we try to find out two positive
numbers δ and α, the latter lying between 0 and 1,
such that the probability that the random interval
(𝛽መ2 −δ, 𝛽መ2 +δ) contains the true β2 is 1 − α. Symbolically:
P (𝛽መ2 −δ≤ β2 ≤ 𝛽መ2 +δ) = 1- α (4.1.1)

10/24/2017 Mai VU-FIE-FTU 5

4.1. Interval estimation: some basic ideas

 Such an interval, if it exists, is known as a confidence
interval;
 1 − α is known as the confidence coefficient;
 α (0 < α < 1) is known as the level of significance
 The endpoints of the confidence interval are known as
the confidence limits (also known as critical values)
 𝛽መ2 −δ being the lower confidence limit
 𝛽መ2 +δ the upper confidence limit
 In practice α and 1 − α are often expressed in
percentage forms as 100α and 100(1 − α) percent.

10/24/2017 Mai VU-FIE-FTU 6

4.1. Interval estimation: some basic ideas
 Equation (4.1.1) shows that an interval estimator, in
contrast to a point estimator, is an interval constructed
in such a manner that it has a specified probability 1 −
α of including within its limits the true value of the
parameter.
 For example, if α = 0.05, or 5 percent, (5.2.1) would
read: The probability that the (random) interval
shown there includes the true β2 is 0.95, or 95 percent.
 The interval estimator thus gives a range of values
within which the true β2 may lie.

10/24/2017 Mai VU-FIE-FTU 7

4.2. Confidence intervals for regression

coefficients β1 and β2
Confidence Interval for β2:
 It was shown in Section 3, that, with the normality
assumption for ui, the OLS estimators 𝛽መ1 and 𝛽መ2 are
themselves normally distributed with means and
variances given therein.
 Therefore, the variable:

Z=
ˆ2 −  2 ( ˆ2 −  2 )
=
x
2
i
(4.2.1)
ˆ
se(  2 ) 

10/24/2017 Mai VU-FIE-FTU 8

4.2. Confidence intervals for regression
coefficients β1 and β2
is a standardized normal variable.
 It therefore seems that we can use the normal
distribution to make probabilistic statements about β2
provided the true population variance σ2 is known.
 But σ2 is rarely known, and in practice it is determined
by the unbiased estimator 𝜎ො 2 . If we replace σ by 𝜎,
ො
(5.3.1) may be written as:

 t=
ˆ2 −  2 ( ˆ2 −  2 )
=
i
x 2

(4.2.2)
ˆ
se(  2 ) ˆ
10/24/2017 Mai VU-FIE-FTU 9

4.2. Confidence intervals for regression

coefficients β1 and β2
 Therefore, instead of using the normal distribution,
we can use the t distribution to establish a confidence
interval for β2 as follows:
P (− t( n − 2 ), ≤ t ≤ t( n − 2 ), 2 ) = 1- α
2

ˆ
↔ P ( − t( n −2), ≤  2 −  2 ≤ t( n − 2 ), 2 ) = 1- α
2
se( ˆ2 )
↔ P[ ˆ2 − t( n − 2), se( ˆ2 )   2  ˆ2 + t( n − 2), se( ˆ2 )] = 1 −  (4.2.3)
2 2

10/24/2017 Mai VU-FIE-FTU 10

4.2. Confidence intervals for regression
coefficients β1 and β2
 Equation (4.2.3) provides a 100(1 − α) percent confidence
interval for β2, which can be written more compactly as:
100(1 − α)% confidence interval for β2:
ˆ2  t( n− 2 ), se( ˆ2 )
2
 The width of the confidence interval is proportional to the
standard error of the estimator: the larger the standard
error, the larger is the width of the confidence interval
 Thus, the standard error of an estimator is often described
as a measure of the precision of the estimator, i.e., how
precisely the estimator measures the true population value.

10/24/2017 Mai VU-FIE-FTU 11

4.2. Confidence intervals for regression

coefficients β1 and β2
Confidence Interval for β1:
P[ ˆ1 − t( n − 2), se( ˆ1 )  1  ˆ1 + t( n − 2 ), se( ˆ1 )] = 1 − 
2 2

100(1 − α)% confidence interval for β2:

ˆ1  t( n− 2), se( ˆ1 )

2

10/24/2017 Mai VU-FIE-FTU 12

4.2. Confidence intervals for regression
coefficients β1 and β2
 Left-sided confidence interval

(
 j  −,  j + t  ,n−kse(  j ) )
 Right-sided confidence interval:

(
 j   j − t  ,n−kse(  j ), + )
10/24/2017 Mai VU-FIE-FTU 13

4.3. Confidence interval for σ2

 Under the normality assumption, the variable:

ˆ 2
 = (n − 2) 2
2

follows the χ2 distribution with n − 2 df.

 Therefore, we can use the χ2 distribution to establish a
confidence interval for σ2 :
P (  ( n − 2),1− ≤  ≤  ( n − 2 ), ) = 1- α
2 2 2

2 2

10/24/2017 Mai VU-FIE-FTU 14

4.3. Confidence interval for σ2
↔ P (  ( n − 2),1− ≤ (n − 2)ˆ 2≤  (2n − 2 ),) = 1- α
2
2 2
2
↔ P [(n-2) ˆ 2 ≤ σ2 ≤ (n-2) ˆ 2 ] = 1- α
 (2n − 2),  (2n − 2),1−
2 2

 where the χ2 value in the middle of this double inequality is

as given by and where χ2 1−α/2 and χ2 α/2 are two values of χ2
(the critical χ2 values) obtained from the chi-square table
for n − 2 df in such a manner that they cut off 100(α/2)
percent tail areas of the χ2 distribution.

10/24/2017 Mai VU-FIE-FTU 15

4.3. Confidence interval for σ2

 Then, given the 100(1 − α)% confidence interval for σ2 :
ˆ 2 ˆ 2
[(n-2) , (n-2) ]
 (2n − 2),  (2n − 2),1−
2 2

10/24/2017 Mai VU-FIE-FTU 16

Example
Consider the following estimated model:
CT = 56,60 + 0,794TN + 0.015TS
(se) (9,65) (0,016) (0,004)
Which is estimated using a sample of 30
observations.
Establish the 95% symmetric confidence interval
for the coefficient on TN.

10/24/2017 Mai VU-FIE-FTU 17

4.4. Hypothesis testing

4.4.1. General comments
4.4.2. Testing an individual regression coefficient β𝑗
4.4.3. Testing the significance of σ2

10/24/2017 Mai VU-FIE-FTU 18

4.4.1. General comments
 The problem of statistical hypothesis testing may be
stated simply as follows: Is a given observation or
finding compatible with some stated hypothesis
or not?
 The word “compatible,” as used here, means
“sufficiently” close to the hypothesized value so that
we do not reject the stated hypothesis.

10/24/2017 Mai VU-FIE-FTU 19

4.4.1. General comments

 In statistics, the stated hypothesis is known as the null
hypothesis and is denoted by the symbol H0.
 The null hypothesis is usually tested against an alternative
hypothesis (also known as maintained hypothesis)
denoted by H1, which may state, for example, that true β2 is
different from unity.
 The theory of hypothesis testing is concerned with
developing rules or procedures for deciding whether to
reject or not reject the null hypothesis.
 There are two mutually complementary approaches for
devising such rules, namely, confidence interval and test
of significance.

10/24/2017 Mai VU-FIE-FTU 20

H0 and H1
 Null hypothesis H0:
✓reflects that there will be no observed effect for your
experiment.
✓is assumed to be true until evidence indicates otherwise.
✓is what we want to reject!
 Alternative hypothesis H1:
✓is an alternative to the null hypothesis.
✓reflects that there will be an observed effect for our
experiment.
✓is what we might believe to be true or hope to prove true!
10/24/2017 Mai VU-FIE-FTU 21

4.4.1. General comments

 Then, hypothesis testing is a given observation or
finding with some stated hypothesis or not.
 Test an individual regression coefficient
✓ Confidence interval
✓ t-distribution of the estimator
✓ p-value
 Test a group of regression coefficients: the joint effect!
✓ Test the overall significance of a multiple regression.
✓ Test the joint significance of a group of variables.

10/24/2017 Mai VU-FIE-FTU 22

4.4.2. Testing an individual regression coefficient
β𝑗
 Two-Sided or Two-Tail Test: We revert to the
consumption-income example. As we know, the estimated
marginal propensity to consume (MPC), 𝛽መ2 , is 0.5091.
Suppose we postulate that:
𝐻 : 𝛽 = 0.3
ቊ 0 2
𝐻1 : 𝛽2 ≠ 0.3
→the true MPC is 0.3 under the null hypothesis but it is less
than or greater than 0.3 under the alternative hypothesis.
→The null hypothesis is a simple hypothesis, whereas the
alternative hypothesis is composite; actually it is what is
known as a two-sided hypothesis.
10/24/2017 Mai VU-FIE-FTU 23

4.4.2. Testing an individual regression coefficient

β𝑗
 Two-sided testing (two-tail):

H 0 :  j =  *j
H1 :  j   *j
 One-sided testing (one-tail):

H 0 :  j   *j H 0 :  j   *j
H1 :  j   *j H1 :  j   *j
10/24/2017 Mai VU-FIE-FTU 24
4.4.2. Testing an individual regression coefficient
β𝑗
 Three methods to test hypothesis:
 Confidence interval approach
 Test of significance approach
 P-value approach

10/24/2017 Mai VU-FIE-FTU 25

The confidence interval approach

2 steps:
• Construct a 100(1-α)% confidence interval for β𝑗
• Decision rule:
✓If β𝑗 under H0 falls within this confidence interval, do
not reject H0.
• 𝛽𝑗 * ∈ confidence interval, do not reject H0
✓If 𝛽𝑗 under H0 falls outside the interval, reject H0.
• 𝛽𝑗 * ∉ confidence interval, reject H0
10/24/2017 Mai VU-FIE-FTU 26
The confidence interval approach

10/24/2017 Mai VU-FIE-FTU 27

Test of significance approach

• Given that H0: β𝑗 = β𝑗 ∗ is true:
j −  *
j
P(−t  /2   t  /2) = (1 −  )
se(  j )

P   *j − t  /2se(  j )   j   *j + t  /2se( j )  = (1 −  )
 
which gives the interval in which β෠ 𝑗 will fall with 1−α
probability, given β𝑗 = β𝑗 ∗ .
=> Region of acceptance of the null hypothesis.

10/24/2017 Mai VU-FIE-FTU 28

Region(s) of acceptance/ Region(s) of rejection
• The 100(1−α)% confidence interval established is known as the
region of acceptance (of H0).
• The region(s)outside the confidence interval is (are) called the
region(s) of rejection (of H0) or the critical region(s).
• A statistic is said to be statistically significant if the value of the test
statistic lies in the critical region.

10/24/2017 Mai VU-FIE-FTU 29

Region(s) of acceptance/ Region(s) of rejection

10/24/2017 Mai VU-FIE-FTU 30

Steps to do a t-test
• Step 1: Establish the hypotheses.
• Step 2: Find the test statistic (observed t value) ts
ˆ j −  *j
ts =
SE ( ˆ j )

• Step 3: Find the critical t value tc

✓Degree of freedom: n – k
✓Level of significance: one-tail testing: α
two-tail testing: α/2
• Step 4: Compare ts to tc. Draw the conclusion.

10/24/2017 Mai VU-FIE-FTU 31

Decision rule for t-test

Type of hypothesis H0 and H1 Rejection zone

H 0 :  j =  *j
Two-tail H1 :  j   *j
|t| > tn-k; α/2
H 0 :  j =  *j (  j   *j )
t > tn-k; α
Right-tail H1 :  j   *j
H 0 :  j =  *j (  j   *j )
Left-tail H1 :  j   *j t < -tn-k; α

10/24/2017 Mai VU-FIE-FTU 32

P-value approach
P-value: the practical level of significance.
• Given that H0 is true, p-value is the probability of
getting a value of the sample test statistic that is at
least as extreme as the one found from the sample
data.
• The lowest significance level at which a null
hypothesis can be rejected.

10/24/2017 Mai VU-FIE-FTU 33

P-value approach
Steps to test an individual regression coefficient using p-
value approach:
• Step 1: Establish the hypotheses.
• Step 2: Find the test statistic (observed t value) ts
• Step 3: Find the p-value associated with the observed t-
value
✓Two-tail: T.DIST.2T(ts, d.f.)
✓Right-tail: T.DIST.RT(ts, d.f.)
✓Left-tail: T.DIST(ts, d.f.)
• Step 4: Compare p-value to α. Draw the conclusion.
✓Reject H0 if p-value < α
10/24/2017 Mai VU-FIE-FTU 34
4.4.3. Testing the significance of σ2 : the χ2 test
 Consider the following variable:
𝜎
ො 2
𝜒 2 = (𝑛 − 2) 2
𝜎
which, as noted previously, follows the χ2 distribution
with (n− 2) df.

𝐻0 : 𝜎 2 = 𝜎02 𝐻0 : 𝜎 2 = 𝜎02 𝐻0 : 𝜎 2 = 𝜎02

൝ ൝ ൝
𝐻1 : 𝜎 2 ≠ 𝜎02 𝐻1 : 𝜎 2 > 𝜎02 𝐻1 : 𝜎 2 < 𝜎02
 This test procedure is called the chi-square test of
significance.

10/24/2017 Mai VU-FIE-FTU 35

4.4.3. Testing the significance of σ2 : the χ2 test

Type of H0: the null H1 : the alt. Method Decision rule: reject H0
hypothesis hypothesis hypothesis
Confidence ˆ 2 ˆ 2
 02  [(n-2) , (n-2) ]
interval  (2n − 2 ),  (2n − 2 ),1−
2 2

Two-tail σ2 =  02 σ2 ≠  02
t- test  02 >  (2n − 2 ), or  02 <  (2n − 2 ),1−
2 2

P-value p-value < α/2 or p-value > 1- α/2

Confidence ˆ 2
 02  [(n-2) , + ]
interval  (2n − 2),
Right tail 2
σ =  2
0
2
σ >  2
0
t-test  02 >  (2n − 2 ),
P-value p-value < α
Confidence ˆ 2
 02  [-  , (n-2) ]
interval  2
( n − 2 ),1−
Left-tail σ2 =  02 σ2 <  02
t-test  02 <  (2n − 2 ),1−
P-value p-value > 1- α

10/24/2017 Mai VU-FIE-FTU 36

4.5. Testing the overall significance
 Remind that: TSS= ESS+ RSS, which decomposed the
total sum of squares (TSS) into two components:
explained sum of squares (ESS) and residual sum of
squares (RSS).
 A study of these components of TSS is known as the
analysis of variance (ANOVA) from the regression
viewpoint.

10/24/2017 Mai VU-FIE-FTU 37

Analysis of variance (ANOVA)

Source of variation SS* df MSS**

(1) (2) (3) (4) = (2)/(3)
Due to regression (ESS)  yˆ
2
i = ˆ22  xi2 1 ˆ22  xi2

Due to residuals (RSS)  uˆ 2

i (n-2)  uˆ2
i
= ˆ 2
n−2
Total sum of squares (TSS) y 2
i (n-1)

10/24/2017 Mai VU-FIE-FTU 38

Testing the overall significance
 We now consider the following variable:
𝑀𝑆𝑆 𝑜𝑓 𝐸𝑆𝑆 𝐸𝑆𝑆/1 ෡22 σ 𝑥 2 /1
𝛽 ෡22 σ 𝑥 2
𝛽
𝑖 𝑖
𝐹= = = 2 =
𝑀𝑆𝑆 𝑜𝑓 𝑅𝑆𝑆 𝑅𝑆𝑆/(𝑛−2) σ𝑢
ෝ𝑖 /(𝑛−2) ෝ2
𝜎

𝑇𝑆𝑆.𝑟 2 /1 𝑟2 (𝑛−2)
𝐹 = = .
1−𝑟 2 𝑇𝑆𝑆/(𝑛−2) (1−𝑟 2 ) 1

→Follows F-distribution with k-1 and n-k degree of

freedom.
→Provide a test of the null hypothesis that the true slope
coefficients is zero.

10/24/2017 Mai VU-FIE-FTU 39

Testing the overall significance

 Hypotheses:

𝐻0 : 𝑟 2 = 0
൝
𝐻1 : 𝑟 2 ≠ 0

 Equivalent hypotheses:
𝐻 :𝛽 = 0
↔ ቊ 0 2
𝐻1 : 𝛽2 ≠ 0

10/24/2017 Mai VU-FIE-FTU 40

Testing the overall significance
Two methods to test the overall significance:

 Test of significance approach

 P-value approach

10/24/2017 Mai VU-FIE-FTU 41

Test of significance approach (F-test)

Steps to test the overall significance of a model:
• Step 1: Establish the hypotheses.

r 2 (n − 2)
• Step 2: Calculate the test statistic: F0 =
(1 − r 2 )

• Step 3: Find the critical F value F(1, n-2)

• Step 4: Conclusion
✓If Fs > F(1, n-2), reject H0
✓If Fs < F(1, n-2), do not reject H0
10/24/2017 Mai VU-FIE-FTU 42
P-value aproach
r 2 (n − 2)
 Step 1: Calculate F0 =
(1 − r 2 )

 Step 2: Calculate p-value= P(F> F0)

 Step 3: Compare p-value to the significance level α

 If p-value < α: reject Ho
 If p-value > α: do not reject H0

10/24/2017 Mai VU-FIE-FTU 43