lOMoARcPSD|39678933

mathemathical economics

MATHEMATICAL ECONOMICS (Jimma University)

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

CHAPTER THREE
THE MULTIPLE LINEAR REGRESSION
3.1 Introduction: The Multiple Linear Regression
3.2 Assumptions of the Multiple Linear Regression
3.3 Estimation: The Method of OLS
3.4 Properties of OLS Estimators
3.5 Partial Correlations and Coefficients of
Multiple Determination
3.6 Statistical Inferences in Multiple Linear
Regression
3.7 Prediction with Multiple Linear Regression
1
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.1 Introduction: The Multiple Linear Regression

Relationship between a dependent & two or more
independent variables is a linear function

Population Random
Population slopes
Y-intercept Error

Yi = β 0 + β1 X 1i + β 2 X 2i + • • • + β K X Ki + ε i
Y = βˆ + βˆ X + βˆ X + • • • + βˆ X + e
i 0 1 1i 2 2i K Ki i

Residual
Dependent (Response) Independent (Explanatory)
variable (for sample) variables (for sample) 2
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.1
3.1 Introduction:
Introduction: The
The Multiple
Multiple Linear
Linear Regression
Regression
) What changes as we move from simple to
multiple regression?
1. Potentially more explanatory power with more
variables;
2. The ability to control for other variables; (and
the interaction of the various explanatory
variables: correlations and multicollinearity);
3. Harder to visualize drawing a line through
three or more (n)-dimensional space.
4. The R2 is no longer simply the square of the
correlation coefficient between Y and X.
3
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.1 Introduction: The Multiple Linear Regression

)Slope ( βj ):
Ceteris paribus, Y changes by βj for every 1 unit
change in X , on average.
)Y-Intercept (β0 ):
j

The average value of Y when all Xj s are zero.

(may not be meaningful all the time)
)A multiple linear regression model is defined to
be linear in the regression parameters rather
than in the explanatory variables.
)Thus, the definition of multiple linear regression
includes polynomial regression.
e.g. Y = β + β X + β X + β X 2 + β X X + ε
i 0 1 1i 2 2i 3 1i 4 1i 2i i
4
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.2
3.2 Assumptions
Assumptions of
of the
the Multiple
Multiple Linear
Linear Regression
Regression
) Assumptions 1 – 7 in Chapter Two.
1. E(ɛi|Xji) = 0. (for all i = 1, 2, …, n; j = 1, …, K)
2. var(ɛi|Xji) = σ2. (i ≠ s) (Homoscedastic errors)
3. cov(ɛi,ɛs|Xji,Xjs) = 0. (i ≠ s) (No autocorrelation)
4. cov(ɛi,Xji) = 0. Errors are orthogonal to the Xs.
5. Xj is non-stochastic, and must assume different
values.
6. n > K+1. (Number of observations > number of
parameters to be estimated). Number of
parameters is K+1 in this case ( β0, β1, …, βK )
7. ɛi ~N(0, σ2). Normally distributed errors.
Downloaded by Solomon Getu ([email protected])
5
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.2
3.2 Assumptions
Assumptions of
of the
the Multiple
Multiple Linear
Linear Regression
Regression
) Additional Assumption:
8. No perfect multicollinearity: That is, no exact
linear relation exists between any subset of
explanatory variables.
) In the presence of perfect (deterministic) linear
relationship between/among any set of the Xjs,
the impact of a single variable ( β j ) cannot be
identified.
) More on multicollinearity in a later chapter!

6
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
The Case of Two Regressors (X1 and X2)
Yˆi = βˆ0 + βˆ1 X1i + βˆ2 X 2i Yˆi = βˆ0 + βˆ1 X1i + βˆ2 X 2i
yˆ i = βˆ1 x1i + βˆ2 x2i ei = Yi −Yˆi = Yi −Y +Y −Yˆi = Yi −Y − (Yˆi −Y )

⇒ ei = yi − yˆ i RSS = ∑e = ∑( yi − β1 x1i − β2 x2i )

ˆ ˆ 2 2
i

) Minimize the RSS with respect to βˆ1 & βˆ2 .

∂(RSS)
= 2∑( yi − βˆ1x1i − βˆ2 x2i )(−x ji ) = 0; j = 1,2 ⇒ −2∑ ei x ji = 0
∂β j
ˆ
1. ∑ ( y i − βˆ1 x1i − βˆ 2 x 2i )( x1i ) = 0
⇒ ∑ y i x 1i = βˆ1 ∑ x 12i + βˆ 2 ∑ x 1i x 2 i
2. ∑ ( y i − βˆ1 x1i − βˆ 2 x 2i )( x 2i ) = 0
⇒ ∑ yi x2i = β1 ∑ x1i x2i + β 2 ∑ x2i
ˆ ˆ 2
7
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
⎛ ∑ y i x1i ⎞ ⎡ ∑ x 1i2 ∑x x ⎤ ⎛ βˆ 1 ⎞
⇒ ⎜⎜ ⎟=⎢ 1i 2i
⎥ ⎜⎜ ˆ ⎟⎟
⎝ ∑ i 2i ⎠ ⎢⎣∑ x 2i x 1i
⎟
y x ∑x 2
2i ⎥⎦ ⎝ β 2 ⎠
F = A • β̂
Solve for the coefficients:
Determinant: A = ∑ 1i ∑ x1i x2i
2
x
= ∑ x12i ∑ x22i − (∑ x1i x 2i ) 2
∑ x1i x2i ∑ 2i
x 2

To find β̂ 1, substitute the first column of A by

A1
elements of F, then find |A1|, and finally find .
A1 =
∑ yi x1i ∑x1i x2i = (∑ yi x1i )(∑x ) − (∑x1i x2i )(∑ yi x2i ) 2
A

∑y xi 2i ∑x 2
2i
2i

A1 (∑ y i x1i )(∑ x 2i ) − (∑ x1i x 2i )(∑ y i x 2i )

2

β1 =
ˆ =
A (∑ x1i )(∑ x 2i ) − (∑ x1i x 2i )
2 2 2
8
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
Similarly, to find β̂2, substitute the second column
of A by elements of F, then find |A2|, and finally
A2
find .
A

A2 =
∑ x ∑ 2
yx
1i i 1i
= (∑ yi x2i )(∑x12i ) − (∑x1i x2i )(∑ yi x1i )
∑x x ∑y x
1i 2i i 2i

A2 (∑ y i x 2i )(∑ x ) − (∑ x1i x 2i )(∑ y i x1i ) 2

β2 =
ˆ =
1i

A (∑ x1i )(∑ x 2i ) − (∑ x1i x 2i )

2 2 2

βˆ 0 = Y − βˆ1 X 1 − βˆ 2 X 2
9
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
The Case of K Explanatory Variables
) The number of parameters to be estimated:
K+1 ( β0 , β1, β2 ,…, β K).

Y1 = βˆ0 + βˆ1 X11 + βˆ2 X 21 + …+ βˆK X K1 + e1

Y = βˆ + βˆ X + βˆ X + …+ βˆ X + e
2 0 1 12 2 22 K K2 2

Y3 = βˆ0 + βˆ1 X13 + βˆ2 X 23 + …+ βˆK X K 3 + e3

Yn = βˆ0 + βˆ1 X1n + βˆ2 X 2n + …+ βˆK X Kn + en 10

Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
.
⎡ Y1 ⎤ ⎡1 X 11 X 21 X 31 … X K1 ⎤ ⎡ βˆ 0 ⎤ ⎡e 1 ⎤
⎢Y ⎥ ⎢1 ⎥ ⎢ˆ ⎥ ⎢ ⎥
⎢ 2⎥ ⎢ X12 X 22 X 32 … X K2 ⎥ ⎢ β 1 ⎥ ⎢e 2 ⎥
⎢Y3 ⎥ = ⎢1 X13 X 23 X 33 … X K3 ⎥ • ⎢ βˆ 2 ⎥ + ⎢e 3 ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢⎣Yn ⎥⎦ ⎢⎣1 X1n X 2n X 3n … X Kn ⎥⎦ ⎢⎣βˆ K ⎥⎦ ⎢⎣e n ⎥⎦
n ×1 n × ( K + 1) (K +1) ×1 n ×1

Y = Xβ + e
ˆ
Downloaded by Solomon Getu ([email protected])
11
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
.
⎡e 1 ⎤ ⎡ Y1 ⎤ ⎡1 X 11 X 21 X 31 … X K1 ⎤ ⎡ βˆ 0 ⎤
⎢e ⎥ ⎢Y ⎥ ⎢1 ⎥ ⎢ˆ ⎥
⎢ 2⎥ ⎢ 2⎥ ⎢ X 12 X 22 X 32 … X K2 ⎥ ⎢ β 1 ⎥
⎢e 3 ⎥ = ⎢Y3 ⎥ − ⎢1 X 13 X 23 X 33 … X K3 ⎥ * ⎢ βˆ 2 ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢⎣e n ⎥⎦ ⎢⎣Yn ⎥⎦ ⎢⎣1 X 1n X 2n X 3n … X Kn ⎥⎦ ⎢⎣βˆ K ⎥⎦

e = Y − Xβ̂
12
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
⎛ e1 ⎞
⎜ ⎟
⎜ e2 ⎟
RSS = ∑ei2 = e12 + e22 + ...+ en2 = (e1 e2 … en ).⎜ ⎟ ⇒RSS= e'e
⎜ ⎟
⎜e ⎟
⎝ n⎠

RSS = (Y − Xβˆ )'(Y − Xβˆ ) = Y'Y − Y'Xβˆ − βˆ ' X'Y + βˆ ' X'Xβˆ
Since Y' Xβˆ is a costant, Y' Xβˆ = (Y' Xβˆ )' = βˆ ' X' Y
⇒ RSS = Y'Y − 2βˆ ' X'Y + βˆ ' (X'X)βˆ
∂( RSS) ∂(RSS )
F.O.C. : =0 ⇒ = −2X' Y + 2X' Xβˆ = 0
∂(βˆ ) ∂(βˆ )
⇒ −2X' (Y − Xβˆ ) = 0
Downloaded by Solomon Getu ([email protected])
13
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
⎡ 1 1 … 1 ⎤ ⎛ e ⎞ ⎛ 0⎞
⎢ X11 X12 … X1n ⎥ ⎜ 1 ⎟ ⎜ ⎟
⎢X ⎥ ⎜ e2 ⎟ ⎜ 0⎟
… X 2n ⎥.⎜ ⎟ = ⎜ ⎟
⇒
. X'e = 0 ⇒ ⎢ 21 X 22
⎢ ⎥⎜ ⎟ ⎜ ⎟
⎢X ⎜ ⎟ ⎜ ⎟
⎣ K1 X K 2 … X Kn ⎥⎦ ⎝ n ⎠ ⎝ 0⎠
e

1. ∑ ei = 0 2. ∑ ei X ji = 0. ( j = 1,2,..., K )

X'e = X'(Y − Xβˆ ) = 0 ⇒ X' Xβˆ = X' Y

ˆβ = ( X' X ) − 1 X' Y
14
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
⎛ βˆ0 ⎞ ⎡ 1 ⎤ ⎡1 X11 … X K1 ⎤
⎜ ⎟ 1 … 1 ⎢ ⎥
⎜ β
ˆ ⎟ ⎢ X X … X 1n ⎢
⎥ 1 X … X K2 ⎥
βˆ = ⎜ 1 ⎟ X/ X = ⎢ 11 12
⎥.
12

⎢ ⎥⎢ ⎥
⎜ ⎟
⎜ βˆ ⎟ ⎢⎣ X K1 X K 2 … X Kn ⎥⎦ ⎢1 X … X ⎥
⎝ K⎠ ⎣ 1n 2n ⎦

⎡ n ∑X … ∑XK ⎤
⎢ ⎥
1

⇒X X=
/ ⎢ ∑ X1 ∑X 2
1 …∑ X 1 X K ⎥
⎢ ⎥
⎢ ⎥
⎢⎣∑ X K ∑X K X1 … ∑ X K2 ⎥⎦

⎡ 1 1 … 1 ⎢ ⎥⎤ ⎡ Y1 ⎤ ⎡ ∑Y ⎤
⎢ YX ⎥
⎢ X 11
X Y=⎢
/
X 12 … X 1 n ⎥ ⎢Y 2 ⎥
⎥ ⇒ X/Y = ⎢
∑ 1⎥
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢⎣ X K 1 … X Kn ⎥⎦ ⎢Y ⎥ ⎢ YX ⎥
⎣∑
X K2
⎣ n⎦ K ⎦
15
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.3
3.3 Estimation:
Estimation: The
The Method
Method of
of OLS
OLS
. ˆβ = (X' X)−1 (X' Y)

-1
⎛ βˆ 0 ⎞ ⎡ n ∑X ∑X … ∑X ⎤ ⎛ ∑Y ⎞
⎜ ⎟ ⎢ 1 2 K
⎥ ⎜ ⎟
⎜ βˆ1 ⎟ ⎢ ∑ X 1 ∑X ∑X X … ∑X X ⎜ ∑ YX 1 ⎟
2
1 1 2 1 K ⎥
⎜ ˆ ⎟=⎢ X ⎜ YX ⎟
⎜ β2 ⎟ ⎢ ∑ 2 ∑X X ∑X ∑X X ⎜∑ 2 ⎟
2
… ⎥
⎥
2 1 2 2 K

⎜ ⎟ ⎢ ⎥ ⎜ ⎟
⎜⎜ ˆ ⎟⎟ ⎢ 2 ⎥ ⎜⎜ ⎟⎟
⎝ β K ⎠ ⎣∑ X K ∑X K X1 ∑X K X2 … ∑ K ⎦
X ⎝ ∑ YX K ⎠
(K+1)×1 (K +1)×(K +1) (K+1)×1

16
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.4 Properties of OLS Estimators

)Given the assumptions of the classical linear
regression model (in Section 3.2), the OLS
estimators of the partial regression coefficients
are BLUE: linear, unbiased and have minimum
variance in the class of all linear unbiased
estimators – the Gauss-Markov Theorem.
)In cases where the small-sample desirable
properties (BLUE) may not be found, we look for
asymptotic (or large-sample) properties, like
consistency and asymptotic normality (CLT).
)The OLS estimators are consistent:
p lim (βˆ − β) = 0 & p lim var(βˆ ) = 0
n →∞ n →∞ 17
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

)In the multiple regression equation with 2
regressors (X1 and X2), Yi = βˆ0 + βˆ1 X1i + βˆ2 X2i + ei , we
can talk of:
¾the joint effect of X1 and X2 on Y, and
¾the partial effect of X1 or X2 on Y.
)The partial effect of X1 is measured by β̂1 and the
partial effect of X2 is measured by β̂ 2 .
)Partial effect: holding the other variable constant
or after eliminating the effect of the other variable.
)Thus, β̂ 1 is interpreted as measuring the effect of
X1 on Y after eliminating the effect of X2 on X1.
18
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

)Similarly, β̂ 2 measures the effect of X2 on Y after
eliminating the effect of X1 on X2.
)Thus, we can derive the estimatorβ̂1 of β1 in two
steps (by estimating two separate regressions):
)Step 1: Regress X1 on X2 (an auxiliary regression
to eliminate the effect of X2 from X1). Let the
regression equation be: X 1 = a + b12 X 2 + e12
Or, in deviation form: x1 = b12x2 + e12
Then, b = ∑x1x2
∑x2
12 2

)e12 is part of X1 which is free from the influence

of X2. Downloaded by Solomon Getu ([email protected])
19
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

)Step 2: Regress Y on e12 (residualized X1). Let the
regression equation be: y = b ye e12 + v in
deviation form.
∑ ye12
Then, b ye = e 2
∑ 12
) b ye is the same as β̂1in the multiple regression,
y = βˆ1x1 + βˆ2 x2 + e. i.e., b ye = βˆ1
)Proof: (You may skip the proof!)
b ye =
∑ ye
12
=
∑ y( x − b x ) 1 12 2

∑e 2
∑ (x − b x ) 2

∑
12 1 12 2

⇒ bye =
∑ yx − b ∑ yx
1 12 2 But, b12 =
xx 1 2

∑ x + b ∑ x − 2b ∑ x x
2
1
2 2
12 Downloaded by2Solomon Getu ([email protected])
12 1 2
∑x 2
20 2
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

∑ xx
∑ yx − ( x )∑ yx
1 2
.
∑
1 2 2

⇒b =
2

∑ ∑
ye
xx xx
∑ x + ( x ) ∑ x − 2( x )∑ x x
2 1 2 2 2 1 2

∑ ∑
1 2 2 2 1 2
2 2

[∑ x22 ∑ yx1 − ∑ x1 x2 ∑ yx2 ]

⇒ bye =
∑2 x 2

(∑ x x ) 2
(∑ x x ) 2

∑ x1 + −2
2 1 2 1 2

∑2 x 2
∑2 x 2

b =
∑x ∑ yx − ∑x x ∑ yx 2

[∑ x22 ∑ yx1 − ∑ x1 x2 ∑ yx2 ]

2 1 1 2 2

∑x ∑x − (∑x x )
ye 2 2 2

∑x
1 2 1 2
2

bye =
2

[∑ x12 ∑ x22 − (∑x1 x2 ) 2 ] ⇒ bye = β̂1

∑2
x 2

Downloaded by Solomon Getu ([email protected])

21
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

)Alternatively, we can derive the estimator βˆ1 of β 1
as follows:
)Step 1: regress Y on X2, & save the residuals, ey2.
1. y = by2 x2 + ey2 …... [ey2 = residualized Y]
)Step 2: regress X1 on X2, & save the residuals, e12.
2. x1 = b12x2 + e12 …… [e12 = residualized X1]
)Step 3: regress ey2 (that part of Y cleared of the
influence of X2) on e12 (part of X1 cleared of the
influence of X2). 3. e = α e + u
y2 12 12

Then, α 12 in regression (3) = βˆ 1 in y = βˆ1 x1 + βˆ 2 x 2 + e!

22
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

) Suppose we have a dependent variable, Y, and
two regressors, X1 and X2.
) Suppose also: y1 and y 2 are the squares of the
r 2
r 2

simple correlation coefficients between Y & X1

and Y & X2, respectively.
) Then,
r y12 = the proportion of TSS that X1 alone explains.
r y22 = the proportion of TSS that X2 alone explains.
) On the other hand, R y2•12 is the proportion of the
variation in Y that X1 & X2 jointly explain.
) We would also like to measure something else. 23
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

For instance:
a) How much does X2 explain after X1 is already
included in the regression equation? Or,
b) How much does X1 explain after X2 is included?
) These are measured by the coefficients of partial
2
determination: ry 2•1 and r y21• 2 , respectively.
) Partial correlation coefficients of the first order:
ry1•2 & ry 2•1.
) Order = number of X's already in the model.
ry1 − ry2r12 ry2 − ry1r12
ry1•2 = ry2•1 =
(1− r )(1− r )
2
y2
2
12
Downloaded by Solomon Getu ([email protected])
(1− r )(1− r )
2
y1
2
1224
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

On Simple and Partial Correlation Coefficients
1. Even if ry1 = 0, ry1.2 will not be zero unless ry2 or
r12 or both are zero.
2. If ry1 = 0; and ry2 ≠ 0, r12 ≠ 0 and are of the same
sign, then ry1.2 < 0, whereas if they are of
opposite signs, ry1.2 > 0.
Example: Let Y = crop yield, X1 = rainfall, X2 =
temperature. Assume: ry1 = 0 (no association
between crop yield and rainfall); ry2 > 0 & r12 <
0. Then, ry1.2 > 0, i.e., holding temperature
constant, there is a positive association between
yield and rainfall.
Downloaded by Solomon Getu ([email protected])
25
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

3. Since temperature affects both yield & rainfall,
in order to find out the net relationship between
crop yield and rainfall, we need to remove the
influence of temperature. Thus, the simple
coefficient of correlation (CC) is misleading.
4. ry1.2 & ry1 need not have the same sign.
5. Interrelationship among the 3 zero-order CCs:
0 ≤ r + r + r − 2ry1ry 2 r12 ≤ 1
2
y1
2
y2
2
12
6. ry2 = r12 = 0 does not mean that ry1 = 0.
Y & X1 and X1 & X2 are uncorrelated does not
mean that Y and X1 are uncorrelated.
26
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

2
) The partial r2, r , measures the (square of the)
y 2•1
mutual relationship between Y and X2 after the
influence of X1 is eliminated from both Y and X2.
) Partial correlations are important in deciding
whether or not to include more regressors.
e.g. Suppose we have: two regressors (X1 & X2);
ry2 = 0.95; and ry22•1 = 0.01.
2

) To explain Y, X2 alone can do a good job (high

simple correlation coefficient between Y & X2).
) But after X1 is already included, X2 does not add
much – X1 has done the job of X2 (very low
partial correlation coefficient between Y & X2).
27
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

) If we regress Y on X1 alone, then we would
have: RSS SIMP = (1 − R y2•1 )∑ y 2
i.e., of the total variation in Y, an amount =
2
∑
(1 − Ry•1 ) yi remains unexplained (by X1 alone).
2

) If we regress Y on X1 and X2, the variation in Y

(TSS) that would be left unexplained is:
RSSMULT = (1 − R 2
y •12 )∑ y 2

) Adding X2 to the model reduces the RSS by:

RSSSIMP − RSSMULT = − y•1 ∑ − − y•12 ∑
2 2 2 2
(1 R ) y (1 R ) y

= (R 2
y•12 − R )∑ y
2
y•1
2
28
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

) If we now regress that part of Y freed from the
effect of X1 (residualized Y) on the part of X2
freed from the effect of X1 (residualized X2), we
will be able to explain the following proportion
of the RSSSIMP:
(R 2
y •12 − R )∑ y 2
y •1
2
R 2
y •12 −R 2
y •1
= =
2 i
r
(1 − R )∑ y
y 2•1 2
y •1
2
i 1 − R y2•1
) This is the Coefficient of Partial Determination
(square of the coefficient of partial correlation).
) We include X2 if the reduction in RSS (or the
increase in ESS) is significant.
) But, when exactly? We will see later!
Downloaded by Solomon Getu ([email protected])
29
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

) The amount (Ry2•12 − Ry2•1 )∑ yi2 represents the
incremental contribution of X2 in explaining the
TSS. (R2 − R2 ) = (1 − R2 )r 2
y•12 y•1 y•1 y 2•1

4. the proportion of the

1. proportion of
incremental contribution
∑y 2
i explained by
of X 2 in explaining the
X 1 & X 2 jointly
unexplained part of ∑ i
y 2

2. proportion of ∑y 2
i 3. proportion of ∑ i that
y 2

explained by X 1 alone X leaves unexplained

1 30
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

) Coefficient of Determination (in Simple Linear
Regression): 2 βˆ ∑ xy
R = Or , R =
2
β ∑
ˆ 2 x2
∑ y 2
∑y 2

) Coefficient of Multiple Determination:

K n

βˆ1 ∑x1 y + βˆ 2 ∑x2 y ∑ j ∑ x ji yi }

{βˆ
Ry2•12 = R 2 = Ry2•12...K =
j =1 i =1

∑y 2 n

∑ i
y 2

i =1
) Coefficients of Partial Determination:
R 2
y•12 −R 2
y•1
R 2
y •12 −R 2
y •2
r 2
y 2•1 = r 2
y1•2 =
1− R 2
y•1
1− R 2
y •2 31
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

)The coefficient of multiple determination (R2)
measures the proportion of the variation in the
dependent variable explained by (the set of all the
regressors in) the model.
)However, the R2 can be used to compare the
goodness-of-fit of alternative regression
equations only if the regression models satisfy
two conditions.
1) The models must have the same dependent
variable.
Reason: TSS, ESS, and RSS depend on the units
in which the regressand Yi is measured.
For instance, the TSS for Y is not the same as the
TSS for log(Y). Downloaded by Solomon Getu ([email protected])
32
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

2) The models must have the same number of
regressors and parameters (the same value of K).
Reason: Adding a variable to a model will never
raise the RSS (or, will never lower ESS or R2)
even if the new variable is not very relevant.
)The adjusted R-squared, R 2 , attaches a penalty to
adding more variables.
)It is modified to account for changes/differences
in degrees of freedom (df): due to differences in
number of regressors (K) and/or sample size (n).
)If adding a variable raises R 2 for a regression,
then this is a better indication that it has
improved the model than if it merely raises R 2.
Downloaded by Solomon Getu ([email protected])
33
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

∑ yˆ 2
∑ e 2
[ ∑ e 2
]
R 2
= = 1− n − (K + 1)
= 1−
∑y ∑y
2
R
[∑
2 2 2
y
]
n−1
(Dividing TSS and RSS by their df).
)K + 1 represents the number of parameters to be
estimated. e2
R 2
= 1−[
∑ •
n −1
]
∑y 2
n − K −1
n −1 As long as K ≥ 1,
R = 1− (1− R ) • (
2 2
)
n − K −1 1− R 2 > 1− R2 ⇒ R 2 < R2

n −1 In general, R 2 ≤ R 2
1− R = (1− R ) • (
2 2
) As n grows larger (relative
n − K −1
to K ), R 2 → R 2 . 34
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.5 Partial Correlations and Coefficients of Determination

1. While R 2 is always non-negative, R 2can be
positive or negative.
2. R. 2 can be used to compare the goodness-of-fit of
two regression models only if the models have
the same regressand.
3. Including more regressors reduces both the RSS
and df; and R 2 raises only if the former effect
dominates.
4. R. 2 should never be the sole criterion for choosing
between/among models:
) Consider expected signs & values of coefficients,
) Look for results consistent with economic theory
or reasoning (possible explanations), ...
Downloaded by Solomon Getu ([email protected])
35
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

Numerical Example:
Y (Salary in X1 (Years of post X2 (Years of
'000 Dollars) high school Experience)
Education)
30 4 10
20 3 8
36 6 11
24 4 9
40 8 12
ƩY = 150 ƩX1 = 25
Downloaded by Solomon Getu ([email protected])
ƩX2 = 50 36
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

Numerical Example:

X1Y X2Y X12 X22 X1X2 Y2 n=5

ƩX1 = 25
120 300 16 100 40 900
ƩX2 = 50
60 160 9 64 24 400 ƩY = 150
216 396 36 121 66 1296 ƩYX1=812
ƩYX2=1567
96 216 16 81 36 576
ƩX1X2=262
320 480 64 144 96 1600 ƩX 2 = 141
1
ƩX1Y ƩX2Y ƩX12 ƩX22 ƩX1X2 ƩY2 = ƩX22 = 510
= 812 = 1552 = 141 = 510 = 262 4772
ƩY2 = 4772
37
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

β = ( X' X ) X' Y
ˆ −1
−1
⎛ βˆ ⎞ ⎡ n
⎜ ⎟ ⎢
0 ∑ X ∑ X 1 ⎤ ⎛ ∑Y ⎞
⎥ ⎜ ⎟ 2

⎜ βˆ ⎟ = ⎢∑ X
1 ∑1 X ∑ X X2
1 1 2 ⎥ • ⎜ ∑YX1 ⎟
⎜⎜ ˆ ⎟⎟ ⎢ ⎜ YX ⎟
β
⎝ ⎠
2 ⎣∑ X ∑ X
2 X ∑1 X 2
2 ⎥
2 ⎦ ⎝∑ 2 ⎠
⎛ βˆ 0 ⎞ ⎡ 5 25 50 ⎤ −1 ⎛ 150 ⎞
⎜ ⎟ ⎜ ⎟
⎢ ⎥
⎜ βˆ 1 ⎟ = 25 141 262 • ⎜ 812 ⎟
⎜⎜ ˆ ⎟⎟ ⎢ ⎥
⎜ 1552 ⎟
β ⎢ ⎥
⎝ 2⎠ ⎣ 50 262 510 ⎦ ⎝ ⎠
⎛ βˆ 0 ⎞ ⎛ - 23.75
⎜ ⎟ ⎜ ⎞
⎛βˆ 0 ⎞ ⎡40.825 4.375 - 6.25⎤ ⎛ 150⎞ ⎟
⎜ ⎟ ⎜ ⎟ ⇒ ⎜ βˆ 1 ⎟ = ⎜ - 0.25
⎢ ⎥
⎜βˆ 1 ⎟ = 4.375 0.625 - 0.75 •⎜ 812⎟ ⎟
⎜⎜ ˆ ⎟⎟ ⎢ ⎥ ⎜⎜ ˆ ⎟⎟ ⎜ ⎟
β ⎢ ⎥ ⎜1552⎟ β
⎝ ⎠ ⎝ 5.5 ⎠
⎝ 2⎠ ⎣ - 6.25 - 0.75 1 ⎦ ⎝ ⎠ 2

Ŷ = −23.75 − 0.25 X1 + 5.5 X 2

Downloaded by Solomon Getu ([email protected])
38
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

)One more year of experience, after controlling

for years of education, results in $5500 rise in
salary, on average.
)Or, if we consider two persons with the same
level of education, the one with one more year of
experience is expected to have a higher salary of
$5500.
)Similarly, for two people with the same level of
experience, the one with an education of one
more year is expected to have a lower annual
salary of $250.
)Experience looks far more important than
education (which has a negative sign).
Downloaded by Solomon Getu ([email protected])
39
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

) The constant term - 23.75 is the salary one

would get with no experience and no education.
) But, a negative salary is impossible.
) Then, what is wrong?
1. The sample must have been drawn from a
subgroup. We have persons with experience
ranging from 8 to 12 years (and post high
school education ranging from 3 to 8 years). So
we cannot extrapolate the results too far out of
this sample range.
2. Model specification: is our model correctly
specified (variables, functional form); does our
data set meet the underlying assumptions?
Downloaded by Solomon Getu ([email protected])
40
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

1. TSS = ∑ y 2 = ∑ Y 2 − nY 2
TSS = 4772 − 5(30) 2
⇒ TSS = 272
2. ESS = ∑ yˆ = ∑ 2
(β x + β x )
ˆ ˆ 2
1 1 2 2

ESS = β1 ∑ x1 + β2 ∑ x2 + 2 βˆ1 βˆ 2 ∑ x1 x2
ˆ 2 2 ˆ 2 2

ESS = βˆ12( ∑X12 − nX12 ) + βˆ22( ∑X 22 − nX 22 )

+ 2βˆ1 βˆ2( ∑X1 X 2 − nX1 X 2 )
ESS = ( −0.25) 2 [141 − 5(5) 2 ] + (5.5) 2 [510 − 5(10) 2 ]
+ 2( −0.25)(5.5) [262 − 5(5)(10)]
⇒ ESS = 270.5Downloaded by Solomon Getu ([email protected])
41
OR : ESS = βˆ1 ∑ yx1 + βˆ2 ∑ yx2
lOMoARcPSD|39678933

2009, JU HASSEN ABDA

ESS = βˆ1 (∑YX1 − nX1Y) + βˆ2( ∑YX2 − nX 2Y)

⇒ ESS = − 0 .25 ( 62 ) + 5 .5( 52 ) = 270 .5
3. RSS = TSS − ESS ⇒ RSS = 272 − 270.5
⇒ RSS = 1.5
ESS 270.5
4. R = =2
⇒ R = 0.9945 2
TSS 272
Our model (education and experience together)
explains about 99.45% of the wage differential.
RSS (n − K − 1) 1.5 2
5. R = 1 −
2
= 1− ⇒ R 2
= 0.9890
TSS (n − 1) 272 4 42
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

Regressing Y on X 1 :

βˆ y1 =
∑ yx 1
=
∑ YX 1 − nX 1Y
=
62
= 3.875
∑x 2
1 ∑X 1
2
− nX 1
2
16
ESS SIMP βˆ y •1 ∑ yx1 3.875 × 62
6. R 2
= = = = 0.8833
∑y
y •1 2
TSS 272
RSSSIMP = (1 − 0.8833)(272) = 0.1167(272) = 31.75
X 1 (education ) alone explanis about 88.33% of the difference s
in wages, and leaves about 11.67% ( = 31.75) unexplaine d.

7. R 2
y •12 −R 2
y •1 = 0.9945 − 0.8833 = 0.1112
(R 2
y •12 − R )∑ y = 0.1112(272) = 30.25
2
y •1
2
43
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

X 2 (experience) enters the wage equation with an

extra (marginal) contribution of explaining about
11.12% ( = 30.25) of the total variation in wages.
Note that this is the contribution of the part of X 2 which
is not related to (free from the influence of) X 1 .

R 2
y •12 −R 2
y •1 0.9945 − 0.8833
8. r 2
y 2•1 = = = 0.9528
1− R 2
y •1 1 − 0.8833
Or, X 2 (experienc e) explains about 95.28%
( = 30.25) of the wage differenti al that X 1 has
left unexplaine d ( = 31.75). 44
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

) The case of two regressors (X1 & X2):

ε i ~ N(0,σ ) 2

σ 2

β1 ~ N( β1 , var(β1 ));
ˆ ˆ var(βˆ )=
∑x (1 − r )
1 2 2
1i 12

σ 2
βˆ2 ~ N (β 2 , var(βˆ2 )); var(βˆ2 ) =
∑x 2
2i (1 − r )
2
12
βˆ0 ~ N(β0 , var(βˆ0 ));
σ 2
var(β0 ) =
ˆ + X 22 var(βˆ1 ) + X12 var(βˆ2 ) + 2X1 X 2 cov(βˆ1 , βˆ2 )
n Downloaded by Solomon Getu ([email protected])
45
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

− σ r 2 2
cov(βˆ1 , βˆ2 ) = 12

∑ x1i x2i (1 − r12 )

2

( ∑ x 1i x 2 i ) 2

r 2
=
∑x ∑x
12 2 2
1i 2i

∑ 1i 12 ) is the RSS from regressing X 1 on X 2 .

x 2
(1 − r 2

∑x 2
2i (1 − r ) is the RSS from regressing X 2 on X 1 .
2
12

RSS
σˆ = 2
is an unbiased estimator of σ .
2

n−3 Downloaded by Solomon Getu ([email protected])

46
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6
3.6 Statistical
Statistical Inferences
Inferences in in Multiple
Multiple Linear
Linear Regression
Regression
−1
⎡ n ∑ X1 … ∑ XK ⎤
⎢ ⎥
2 ⎢∑ 1
X ∑ X1 2
…∑ X1 X K ⎥
var− cov(β) = σ (X X) = σ
ˆ 2 / -1
⎢ ⎥
⎢ 2 ⎥
⎢⎣∑ X K ∑ X K X1 … ∑ X K ⎥⎦
−1
⎡ n ∑ X1 … ∑ XK ⎤
⎢ ⎥
2⎢∑ ∑ …∑ X1 X K ⎥
∧ 2
X1 X1
var − cοο(β) = σˆ
ˆ
⎢ ⎥
⎢ ⎥
)Note that: ⎢
⎣ ∑ X K ∑ X K X 1 … ∑ X K ⎥⎦ 2

(a) (X'X)-1 is the same matrix we use to derive the

OLS estimates, and
(b) σˆ 2 = RSS in the case of two regressors.
n−3 Downloaded by Solomon Getu ([email protected])
47
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

) In the general case of K explanatory variables,
RSS is an unbiased estimator of σ 2.
σˆ =
2

n − K −1
Note:
) Ceteris paribus, the higher the correlation
coefficient between X1 & X2 ( r12 ), the less
precise will the estimates βˆ1 & βˆ2 be, i.e., the CIs
for the parameters β1 & β 2 will be wider.
) Ceteris paribus, the higher the degree of
variation of the Xjs (the more Xjs vary in our
sample), the more precise will the estimates be –
narrow CIs for population parameters.
Downloaded by Solomon Getu ([email protected])
48
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

) The above two points are contained in:
σ2
βˆ j ~ N (β j , ); ∀ j = 1,2,..., K .
RSS j
where RSSj is the RSS from an auxiliary regres-
sion of Xj on all other (K–1) X's and a constant.
) We use t test to test about single parameters and
single linear functions of parameters.
) To test hypotheses about & construct intervals
for individual β j use: ˆ
βj −βj *

~ tn−K−1;∀j = 0,1,...,K.
seˆ(βˆ j )
49
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

) Tests about and interval estimation of the error
variance σ are based on:
2

RSS (n − K − 1)σ̂ 2
= ~ χ 2
n−K −1
σ 2
σ 2

) Tests of several parameters and several linear

functions of parameters are F-tests.
Procedures for Conducting F-tests:
1. Compute the RSS from regressing Y on all Xjs
(URSS=Unrestricted Residual Sum of Squares).
2. Compute the RSS from the regression with the
hypothesized/specified values of parameters (β s)
(RRSS = Restricted RSS).
Downloaded by Solomon Getu ([email protected])
50
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

3. Under H0 (if the restriction is correct)
(RRSS − URSS) / J (RU2 − RR2 )/J
~ FJ ,n−K −1 ~ FJ,n−K −1
URSS/(n − K −1) (1 − RU )/(n− K − 1)
2

where J is the number of restrictions imposed.

If F-calculated is greater than the F-tabulated,
then the RRSS (is significantly) greater than the
URSS, and thus we reject the null.
) A special F-test of common interest is to test the
null that none of the Xs influence Y (i.e., that
our regression is useless!):
Test H0: β1 = β 2 = ... = β K = 0 vs. H1: H0 is not true. 51
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

K n
URSS = (1 − R )∑ y = ∑ y − ∑{βˆ j ∑ x ji yi }.
2 2
i
2
i
j =1 i =1
RRSS = ∑ y . 2
i

( RRSS − URSS) / K R2 / K
⇒ = ~ FK ,n− K −1
URSS /(n − K − 1) (1 − R ) /(n − K − 1)
2

) With reference to our example on wages, test

the following at the 5% level of significance.
a) β1 = 0 ; b) β 2 = 0 ; c) β 0 = 0;
d) the overall significance of the model; and
e) β1 = β 2 .
52
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

var− cov(β ) = σ ( X' X )

ˆ 2 −1

−1
⎡ 5 25 50 ⎤ ⎡40.825 4.375 - 6.25 ⎤
(X' X) −1 ⎢ ⎥
= ⎢25 141 262⎥ ⎢ = ⎢ 4.375 0.625 - 0.75 ⎥⎥
⎢⎣50 262 510⎥⎦ ⎢⎣ - 6.25 - 0.75 1 ⎥⎦
RSS 1.5
σ is estimated by : σˆ =
2 σˆ =
2 2 = 0.75
n − K −1 2
∧
⎡ 40.825 4.375 - 6.25 ⎤
var − cov ( β̂ ) = 0.75 ⎢⎢ 4.375 0.625 - 0.75 ⎥⎥
⎢⎣ - 6.25 - 0.75 1 ⎥⎦
⎡30.61875 3.28125 - 4.6875 ⎤
= ⎢⎢ 3.28125 0.46875 - 0.5625 ⎥⎥
⎢⎣ - 4.6875 - 0.5625 0.75 ⎥⎦ 53
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

⎡ var( βˆ 0 ) cov( βˆ 0 , βˆ 1 ) cov( βˆ 0 , βˆ 2 ) ⎤ ⎡30.61875 3.28125 - 4.6875 ⎤

= ⎢⎢ - 0.5625 ⎥⎥
⎢ ⎥
⎢ var( βˆ 1 ) cov( βˆ 1 , βˆ 2 ) ⎥ 0.46876
⎢ var( βˆ ) ⎥
⎣ 2 ⎦ ⎢⎣ 0.75 ⎥⎦
βˆ1 − 0 − 0.25
a) t c =
seˆ( βˆ1 )
=
0.46875
≈ −0.37 ttab = t 2
0.025 ≈ 4.30
t cal ≤ t tab , ⇒ we do not reject the null.
β −0
ˆ 5.5
t cal > t tab
b) t c = 2
= ≈ 6.35
seˆ( βˆ 2 ) 0.75 ⇒ reject the null.
βˆ0 − 0 − 23.75
c) t c = = ≈ −4.29
seˆ( βˆ0 ) 30.61875
t cal ≤ t tab , ⇒ we do not reject the null! ! !
54
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

2
R /K 0.9945 / 2
d) Fc = = ≈ 180.82
(1 − R ) /(n − K − 1) 0.0055 / 2
2

Ft = F20, .205 ≈ 19 Fcal > Ftab , ⇒ reject the null.

e) From Yˆi = βˆ0 + βˆ1 X 1i + βˆ 2 X 2i , URSS = 1.5
Now run Yˆ = βˆ + βˆ X + βˆ X
i 0 1i 2i

⇒ Yˆi = βˆ 0 + βˆ ( X 1i + X 2 i ). ⇒ RRSS = 12.08

( RRSS − URSS) / J (12.08 − 1.5) / 1
Fc = = ≈ 14.11
(URSS) /(n − K − 1) 1.5 / 2
Ft = F 0.05
1, 2 ≈ 18.51
Fcal ≤ Ftab , ⇒ we do not reject the null.
Downloaded by Solomon Getu ([email protected])
55
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

) Note that we can also use t-test to test the single
restriction that β1 = β2 (equivalently, β1 - β2 = 0).
β̂ 1 − β̂ 2 − 0 β̂ 1 − β̂ 2
= ~ t1
sê(β̂ 1 − β̂ 2 ) vâr(β̂ 1 ) + vâr(β̂ 2 ) − 2côv(β̂ 1 , β̂ 2 )
− 5.75
tc = ≈ −3.76
0.6846532+ 0.8660254− 2( −0.5625)
tt = t 1
= 12.706
0.025
t cal < t tab ⇒ do not reject the null.
) The same result as the F-test, but the F-test is
easier to handle. Downloaded by Solomon Getu ([email protected])
56
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.6 Statistical Inferences in Multiple Linear Regression

To sum up:
Assuming that our model is correctly specified
and all the assumptions are satisfied,
) Education (after controlling for experience)
doesn’t have a significant influence on wages.
) In contrast, experience (after controlling for
education) is a significant determinant of wages.
) The intercept parameter is also insignificant
(though at the margin). Less Important!
) Overall, the model explains a significant portion
of the observed wage pattern.
) We cannot reject the claim that the coefficients
of the two regressors are equal.
Downloaded by Solomon Getu ([email protected])
57
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.7 Prediction with Multiple Linear Regression

) In Chapter 2, we used the estimated simple
linear regression model for prediction: (i) mean
prediction (i.e., predicting the point on the
population regression function (PRF)), and (ii)
individual prediction (i.e., predicting an
individual value of Y), given the value of the
regressor X (say, X = X0).
) The formulas for prediction are also similar to
those in the case of simple regression except
that, to compute the standard error of the
predicted value, we need the variances and
covariances of all the regression coefficients. 58
Downloaded by Solomon Getu ([email protected])
 lOMoARcPSD|39678933

2009, JU HASSEN ABDA

3.7 Prediction with Multiple Linear Regression

Note:
) Even if the R2 for the SRF is very high, it does
not necessarily mean that our forecasts are
good.
) The accuracy of our prediction depends on the
stability of the coefficients between the period
used for estimation and the period used for
prediction.
) More care must be taken when the values of the
regressors (X's) themselves are forecasts.

59
Downloaded by Solomon Getu ([email protected])