ECONOMICS 351* -- NOTE 7 M.G.

Abbott

ECON 351* -- NOTE 7

Interval Estimation in the Classical Normal Linear Regression Model

This note outlines the basic elements of interval estimation in the Classical
Normal Linear Regression Model (the CNLRM). Interval estimation – i.e., the
construction of confidence intervals for unknown population parameters – is one of
the two alternative approaches to statistical inference; the other is hypothesis
testing.

1. Introduction

‰ We have previously derived point estimators of all the unknown population

parameters in the Classical Normal Linear Regression Model (CNLRM) for
which the population regression equation, or PRE, is

Yi = β0 + β1X i + u i where ui is iid as N(0, σ 2 ) (i = 1, …, N) (1)

♦ The unknown parameters of the PRE are

(1) the regression coefficients β0 and β1

and
(2) the error variance σ2.

♦ The point estimators of these unknown population parameters are

(1) the unbiased OLS regression coefficient estimators β̂ 0 and β̂1

and
(2) the unbiased error variance estimator σ̂ 2 .

♦ Assume that we have computed the point estimates β̂0 , β$ 1 and σ$ 2 of the
unknown parameters for a given set of sample data (Yi, Xi), i = 1, ..., N.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 1 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

‰ We therefore begin with the following OLS sample regression equation (or
OLS-SRE):

Yi = βˆ 0 + βˆ 1X i + û i = Ŷi + û i (i = 1, …, N) (2)

where
N

∑ xy ∑ (X i − X )(Yi − Y )
βˆ 1 = i i 2 i = i =1
= OLS estimate of β1 ;
∑i x i N
∑ (X i − X ) 2

i =1

βˆ 0 = Y − βˆ 1X = OLS estimate of β0 ;
∑ i u$ i2 RSS
σ$ =
2
= = unbiased OLS estimate of σ 2 ;
(N − 2) (N − 2)
σˆ 2 σˆ 2
Vâr (βˆ 1 ) = = ;
∑ i x i2 ∑ i ( X i −X ) 2
1

ˆ ˆ ⎛ σˆ 2 ⎞ 2 σˆ
sê(β1 ) = Vâr (β1 ) = ⎜⎜ ⎟
2 ⎟
= ;
⎝ ∑i x i ⎠ ∑ i x i2
ˆ σˆ 2 ∑ i X i2 σˆ 2 ∑ i X i2
Vâr (β0 ) = = ;
N ∑ i x i2 N ∑ i ( X i −X ) 2
1
⎛ σˆ ∑ i X
2 2
⎞2
sê(βˆ 0 ) = Vâr (βˆ 0 ) = ⎜⎜ i
⎟⎟ .
⎝ N ∑i x
2
i ⎠

‰ Under the assumptions of the Classical Normal Linear Regression Model

(CNLRM) – including in particular the normality assumption A9 – the sample
t-statistics for β̂1 and β̂ 0 each have the t-distribution with (N − 2) degrees of
freedom: i.e.,

βˆ 1 − β1 βˆ − β
t (βˆ 1 ) = = 1 1 ~ t[N − 2] ;
Vâr (βˆ 1 ) sê(βˆ 1 )
βˆ 0 − β0 βˆ − β0
t (βˆ 0 ) = = 0 ~ t[ N − 2] .
ˆ
Vâr (β0 ) s ê (βˆ )
0

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 2 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

2. Interval Estimation: Some Basic Ideas

2.1 General Form of a Confidence Interval

A confidence interval for the slope coefficient β1 takes the general form

Pr( β̂1L ≤ β1 ≤ β̂1U ) = Pr( β̂1 − δ$ ≤ β1 ≤ β̂1 + δ$ ) = 1 − α (3)

where
α = the significance level (0 < α < 1),
1 − α = the confidence level (or confidence coefficient),
δ$ = a positively-valued sample statistic,
β̂1L = β̂1 − δ$ = the lower confidence limit,
β̂ = β̂ + δ$ = the upper confidence limit.
1U 1

The interval [ β̂1L , β̂1U ] = [ β̂1 − δ$ , β̂1 + δ$ ] is called the two-sided (1 − α)-level
confidence interval, or two-sided 100(1 − α) percent confidence interval, for the
slope coefficient β1.

2.2 Interpretation of Confidence Intervals

1. The confidence interval [ β̂1L , β̂1U ] is a random interval.

• The confidence limits β̂1L = β̂1 − δ$ and β̂1U = β̂1 + δ$ are random
variables (or sample statistics) that vary in value from one sample to
another because the values of β̂1 and δ$ vary from sample to sample.

• But for any one sample of data of size N and the corresponding estimates of
β̂1 and δ$ , the confidence limits β̂1L = β̂1 − δ$ and β̂1U = β̂1 + δ$ are simply
fixed numbers, i.e., they take fixed values. Therefore, any one confidence
interval calculated for a particular sample of data is a fixed, meaning
nonrandom, interval.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 3 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

2. The correct interpretation of the confidence interval [ β̂ 1L , β̂1U ] is based on

the concept of repeated sampling.

• Suppose a very large number of random samples of the same size N (e.g., N
= 50 observations) are independently selected from a given population.

• For each of these random samples of N observations, the values of the

confidence limits β̂1L = β̂1 − δ$ and β̂1U = β̂1 + δ$ are calculated for some
fixed value of the confidence level 1 − α (such as 1 − α = 0.99 or 0.95 or
0.90).

• The probability statement in (3) means that 100(1 − α) percent of all the
confidence intervals so constructed will contain the true (but unknown)
population value of β1.

• But note that any one confidence interval [ β̂1L , β̂1U ] based on one sample of
N observations may or may not contain the true value of β1.

◊ For one sample of N observations, the confidence limits β̂1L = β̂1 − δ$

and β̂1U = β̂1 + δ$ take fixed values because the values of β̂1 and δ$
calculated for a single sample of N observations are fixed numbers.

◊ Because β1 is some fixed but unknown number, β1 either lies inside or

outside the fixed confidence interval calculated for any one sample of N
observations. That is, a single confidence interval computed for one
specific sample of N observations either does or does not contain the true
population value of β1.

◊ Result: A single confidence interval [ β̂ 1L , β̂1U ] based on one sample of

N observations is a fixed, or nonrandom, interval.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 4 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

3. Summary: Interpretation of Confidence Intervals

The two-sided 100(1−α) percent confidence interval for the slope coefficient
β1 is defined by probability statement (3):

Pr( β̂1L ≤ β1 ≤ β̂1U ) = Pr( β̂1 − δ$ ≤ β1 ≤ β̂1 + δ$ ) = 1 − α (3)

(1) Any one confidence interval for β1, based on one sample of data, may or
may not contain the true value of β1. Since the true value of β1 is unknown,
we do not know whether that value does or does not lie inside any one
confidence interval.

(2) The probability statement (3) is therefore a statement about the procedure
used to construct the confidence interval, not about any one confidence
interval estimate calculated for a particular sample of data.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 5 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

3. Confidence Intervals for the Regression Coefficients β0 and β1

3.1 Confidence Interval for β1: Derivation

A two-step derivation:

Step 1: Start with a probability statement formulated in terms of t( β̂1 ), the t-

statistic for β̂1 . This probability statement implicitly defines the two-sided
(1−α)-level confidence interval for β1.

Step 2: Re-arrange this probability statement to obtain an equivalent probability

statement formulated in terms of β1 rather than t( β̂1 ). The resultant
probability statement explicitly defines the two-sided (1−α)-level
confidence interval for β1.

STEP 1: The two-sided (1 − α)-level confidence interval for β1 is implicitly

defined by the probability statement

( )
Pr − t α / 2 [ N − 2] ≤ t (βˆ 1 ) ≤ t α / 2 [ N − 2] = 1 − α (4)

where

1 − α = the confidence level attached to the confidence interval;

α = the significance level, where 0 < α < 1;
tα/2[N−2] = the critical value of the t-distribution with (N−2) degrees of
freedom at the α/2 (or 100α/2 percent) significance level;

and t( β̂1 ) is the t-statistic for β̂1 given by

βˆ 1 − β1 βˆ − β
t (βˆ 1 ) = = 1 1. (5)
Vâr (βˆ 1 ) sê(βˆ 1 )

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 6 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

Upper and Lower α/2 Critical Values of t[N-2] Distribution

left-tail area = α/2 ↓ confidence region ↓ right-tail area = α/2

area = 1 − α
0.45

0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0
-5 -4 -3 -2 -1 0 1 2 3 4 5

− tα/2 tα/2
↑ ↑
Pr(t < − tα/2) = α/2 ↑ Pr(− tα/2 ≤ t ≤ tα/2) = 1−α ↑ Pr(t > tα/2) = α/2

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 7 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

STEP 2: Express the double inequality inside the brackets in probability statement
(4) in terms of β1 rather than t( β̂1 ).

(
Pr − t α / 2 [ N − 2] ≤ t (βˆ 1 ) ≤ t α / 2 [ N − 2] = 1 − α) (4)

(1) Substitute in the double inequality

− t α / 2 [ N − 2] ≤ t (βˆ 1 ) ≤ t α / 2 [ N − 2]

the expression for t( β̂1 ) given in (5) above:

βˆ 1 − β1
− t α / 2 [ N − 2] ≤ ≤ t α / 2 [ N − 2] . (6.1)
sê(βˆ 1 )

(2) Multiply the double inequality (6.1) by the positive number sê(βˆ 1 ) > 0:

− t α / 2 sê(βˆ 1 ) ≤ βˆ 1 − β1 ≤ t α / 2 sê(βˆ 1 ) . (6.2)

(3) Subtract β̂1 from both sides of inequality (6.2):

− βˆ 1 − t α / 2 sê(βˆ 1 ) ≤ − β1 ≤ − βˆ 1 + t α / 2 sê(βˆ 1 ) . (6.3)

(4) Multiply all terms in inequality (6.3) by −1, remembering to reverse the
direction of the inequalities:

βˆ 1 − t α / 2 sê(βˆ 1 ) ≤ β1 ≤ βˆ 1 + t α / 2 sê(βˆ 1 ) . (6.4)

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 8 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

RESULT: The probability statement (4) can be written as

( )
Pr βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) ≤ β1 ≤ βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 ) = 1 − α . (7)

The two-sided (1 − α)-level confidence interval for β1 can therefore be written as

βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) ≤ β1 ≤ βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 )

or more compactly as

βˆ 1 ± t α 2 [ N − 2] sê(βˆ 1 ) or [ βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) , βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 ) ]

where at the (1 − α) confidence level, or 100(1 − α) percent confidence level,

βˆ 1L = βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) = the lower 100(1 − α) percent confidence limit

for β1

and

βˆ 1U = βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 ) = the upper 100(1 − α) percent confidence limit

for β1

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 9 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

Two-Sided (1 − α)-level Confidence Interval for β1

left-tail area = α/2 ↓ confidence area = 1 − α ↓ right-tail area = α/2

0.45

0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0
-5 -4 -3 -2 -1 0 1 2 3 4 5

↑ ↑ ↑
βˆ 1L = βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) β̂1 βˆ 1U = βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 )

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 10 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

3.2 Confidence Interval for β0: Derivation

The confidence interval (or interval estimator) for the intercept coefficient β0 is
derived, interpreted, and constructed in exactly the same way as the confidence
interval for the slope coefficient β1.

1. The two-sided (1 − α)-level confidence interval for β0 is implicitly defined by

the probability statement

( )
Pr − t α / 2 [ N − 2] ≤ t (βˆ 0 ) ≤ t α / 2 [ N − 2] = 1 − α (8)

where

1−α = the confidence level attached to the confidence interval;

α = the significance level, where 0 < α < 1;
tα/2[N−2] = the critical value of the t-distribution with (N−2) degrees of
freedom at the α/2 (or 100(α/2) percent) significance level;

and t( β̂ 0 ) is the t-statistic for β̂ 0 given by

βˆ 0 − β0 βˆ − β0
t (βˆ 0 ) = = 0 . (9)
Vâr (βˆ 0 ) sê(βˆ 0 )

2. The double inequality inside the brackets in probability statement (8) can be
expressed in terms of β0 rather than t( β̂ 0 ), using a derivation analogous to that
used in deriving the confidence interval for β1.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 11 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

RESULT: The probability statement (8) can be written as

( )
Pr βˆ 0 − t α 2 [ N − 2] sê(βˆ 0 ) ≤ β0 ≤ βˆ 0 + t α 2 [ N − 2] sê(βˆ 0 ) = 1 − α . (10)

The two-sided (1 − α)-level confidence interval for β0 can therefore be written as

βˆ 0 − t α 2 [ N − 2] sê(βˆ 0 ) ≤ β0 ≤ βˆ 0 + t α 2 [ N − 2] sê(βˆ 0 )

or more compactly as

βˆ 0 ± t α 2 [ N − 2] sê(βˆ 0 ) or [ βˆ 0 − t α 2 [ N − 2] sê(βˆ 0 ) , βˆ 0 + t α 2 [ N − 2] sê(βˆ 0 ) ]

where at the (1 − α) confidence level, or 100(1 − α) percent confidence level,

βˆ 0 L = βˆ 0 − t α 2 [ N − 2] sê(βˆ 0 ) = the lower 100(1 − α) percent confidence limit

for β0

and

βˆ 0 U = βˆ 0 + t α 2 [ N − 2] sê(βˆ 0 ) = the upper 100(1 − α) percent confidence limit

for β0

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 12 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

3.3 Procedure for Computing Confidence Intervals

Consider the problem of computing a confidence interval for the slope coefficient
β1. Recall that the two-sided (1 − α)-level confidence interval for β1 is given by
the double inequality

βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) ≤ β1 ≤ βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 ) .

Step 1: After estimating the PRE (1) by OLS, retrieve from the estimation results
the OLS estimate β̂1 of β1 and the estimated standard error sê(βˆ 1 ) .

Step 2: Select the value of the confidence level (1 − α), which amounts to
selecting the value of α. Although the choice of confidence level is
essentially arbitrary, the values most commonly used in practice are:

α = 0.01 ⇒ (1 − α) = 0.99, i.e., the 100(1 − α) = 100(0.99) = 99 percent

confidence level;

α = 0.05 ⇒ (1 − α) = 0.95, i.e., the 100(1 − α) = 100(0.95) = 95 percent

confidence level;

α = 0.10 ⇒ (1 − α) = 0.90, i.e., the 100(1 − α) = 100(0.90) = 90 percent

confidence level.

Step 3: Obtain the value of t α 2 [ N − 2] , the α/2 critical value of the t-distribution
with N−2 degrees of freedom, either from statistical tables of the t-
distribution or from a computer software program.

Step 4: Use the values of β̂1 , sê(βˆ 1 ) , and t α 2 [ N − 2] to compute the upper and
lower 100(1 − α) percent confidence limits for β1:

βˆ 1U = βˆ 1 + t α 2 [ N − 2] sê(βˆ 1 ) = the upper 100(1 − α)% confidence limit for β1;

βˆ 1L = βˆ 1 − t α 2 [ N − 2] sê(βˆ 1 ) = the lower 100(1 − α)% confidence limit for β1.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 13 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

4. Determinants of the Confidence Interval for βj

Consider the two-sided 100(1 − α)% confidence interval for βj (j = 0, 1):

βˆ j − t α 2 [ N − 2] sê(βˆ j ) ≤ β j ≤ βˆ j + t α 2 [ N − 2] sê(βˆ j )
or
[βˆ − t
j α2 [ N − 2] sê(βˆ j ), βˆ j + t α 2 [ N − 2] sê(βˆ j ) ]
By inspection, it is apparent that the two-sided confidence interval for βj is wider

(1) the greater the value of sê(βˆ j ) , the estimated standard error of β̂ j , i.e., the less
precise is the estimate of β̂ j ;

(2) the greater the critical value t α 2 [N − 2] , i.e., the greater the chosen value of
the confidence level (1 − α) for the given sample size N.

Explanation: Given sample size N, the value of t α 2 [N − 2] is negatively

related to the value of α, and so is positively related to the value of (1 − α).

Example: Suppose sample size N = 30, so that the degrees-of-freedom N-2 =

28. Then from a table of percentage points for the t-distribution, we obtain the
following values of t α 2 [ N − 2] = t α 2 [28] for different values of α:

α = 0.01 ⇒ (1 − α) = 0.99: α/2 = 0.005 and t0.005[28] = 2.763;

α = 0.02 ⇒ (1 − α) = 0.98: α/2 = 0.01 and t0.01[28] = 2.467;

α = 0.05 ⇒ (1 − α) = 0.95: α/2 = 0.025 and t0.025[28] = 2.048;

α = 0.10 ⇒ (1 − α) = 0.90: α/2 = 0.05 and t0.05[28] = 1.701.

Note that higher values of (1 − α) -- i.e., higher confidence levels -- correspond

to higher critical values of t α 2 [28].

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 14 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

5. Numerical Example: Computing a Two-Sided 95 Percent Confidence

Interval for β1

• Estimate by OLS on the auto1.dta sample of N = 74 observations the simple

linear regression model given by the population regression equation (Stata 10
Tutorial 3)

pricei = β0 + β1weight i + u i

. regress price weight

Source | SS df MS Number of obs = 74

-------------+------------------------------ F( 1, 72) = 29.42
Model | 184233937 1 184233937 Prob > F = 0.0000
Residual | 450831459 72 6261548.04 R-squared = 0.2901
-------------+------------------------------ Adj R-squared = 0.2802
Total | 635065396 73 8699525.97 Root MSE = 2502.3

------------------------------------------------------------------------------
price | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
weight | 2.044063 .3768341 5.42 0.000 1.292857 2.795268
_cons | -6.707353 1174.43 -0.01 0.995 -2347.89 2334.475
------------------------------------------------------------------------------

. display _b[weight]
2.0440626

. display _se[weight]
.37683413

. display invttail(72, 0.025)

1.9934636

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 15 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

• Selected results from OLS estimation of the above linear regression model:

β̂1 = 2.0440626

sê(βˆ 1 ) = 0.37683413

(1 − α) = 0.95 ⇒ α = 1 – 0.95 = 0.05 ⇒ α/2 = 0.05/2 = 0.025

t α 2 [ N − 2] = t 0.025 [74 − 2] = t 0.025 [72] = 1.9934636

• Compute upper 95% confidence limit for β1

βˆ 1U = βˆ 1 + t 0.025 [ N − 2] sê(βˆ 1 ) = 2.0440626 + 1.9934636(0.37683413)

= 2.0440626 + 0.7512051
= 2.7952677 = 2.795

• Compute lower 95% confidence limit for β1

βˆ 1L = βˆ 1 − t 0.025 [ N − 2] sê(βˆ 1 ) = 2.0440626 − 1.9934636(0.37683413)

= 2.0440626 − 0.7512051
= 1.2928575 = 1.293

• Result: The two-sided 95% confidence interval for β1 is: [1.293, 2.795]

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 16 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

6. Simulations for Two-Sided 95 Percent Confidence Interval for β̂1

The True Model: is given by the population regression equation (PRE)

Yi = β0 + β1X i + u i = 70.0 + 0.90 X i + u i

where

β0 = 70.0 and β1 = 0.90;

Yi = weekly consumption expenditures of the i-th household;

Xi = weekly disposable income of the i-th household;

ui = an iid random error term that is assumed to be N(0, σ 2 ) .

Model 3: sets σ 2 = Var ( u i X i ) = 25,600, σ = Var ( u i X i ) = se( u i X i ) = 160.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 17 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

The Monte Carlo Simulations

• Two different sample sizes: N = 60, N = 120.

• Set population values of X, β0 and β1, and σ 2 = Var ( u i X i ) .

• Generate 1,000 independent random samples of Yi and ui values.

• For each of these 1,000 independent random samples, compute the values of the
OLS slope coefficient estimator

∑ xy
βˆ 1 = i i 2 i
∑i x i

and its estimated standard error

1
⎛ σˆ ⎞
2 2 σˆ
sê(βˆ 1 ) = Vâr (βˆ 1 ) = ⎜⎜ ⎟ =
2 ⎟
∑ x
⎝ i i ⎠ ∑ i x i2

where x i ≡ X i − X , y i ≡ Yi − Y , X = ∑ i X i N , and Y = ∑ i Yi N .

• Save the 1,000 values of β̂1 and the 1,000 values of sê(β̂1 ) .

• Use each of the 1,000 values of β̂1 and sê(β̂1 ) to compute the two-sided 95
percent confidence interval for the slope coefficient β1 , and then count the
number and percentage of these 1,000 confidence intervals that contain the
true population value of β1, which is 0.90.

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 18 of 19 pages

ECONOMICS 351* -- NOTE 7 M.G. Abbott

Simulation Results for Model 3 for Sample Sizes N = 60 and N = 120

Observations (1,000 Replications)

For sample size N = 60:

• Number of two-sided 95% confidence intervals that contained true value of

β1 = 940/1000
• Percentage of two-sided 95% confidence intervals that contained true value
of β1 = 94.0%

For sample size N = 120:

• Number of two-sided 95% confidence intervals that contained true value of

β1 = 952/1000
• Percentage of two-sided 95% confidence intervals that contained true value
of β1 = 95.2%

ECON 351* -- Note 7: Interval Estimation in the CNLRM … Page 19 of 19 pages