Unit - IV
Unit - IV
6]
shortest length. This is the interval given in (91) for which, for the example,
( 9 0 ) 1S
Pr{* > 4.30} = 0.025
for t ~ t2. Hence the symmetric interval on bx is, from (91),
F{H)
= SSE/[N - r(X)] = QIS02
V
F(H) = 4 2 = , — (101)
so so
with s and N — r degrees of freedom, s being the number of rows in K',
it being of full row rank.
The generality of this result merits emphasis: it applies for any linear
hypothesis K'b = m, the only limitation being that K' have full row rank.
Other than this, F(H) can be used to test any linear hypothesis whatever. No
matter what the hypothesis is, it has only to be written in the form K'b = m
and F{H) of (101) provides the test. Having once solved the normal equations
for the model y = Xb + e and so obtained (X'X)"1, h = (X'X^X'y and
σ2, the testing of H: K'b = m can be achieved by immediate application of
F(H). The appeal of this result is illustrated below in subsection c, for the
four hypotheses listed at the beginning of this section. Note that a2 is uni-
versal to every application of F(H). Thus, in considering different hypotheses
the only term in F(H) that alters is Q/s.
b. Estimation under the null hypothesis
When considering the hypothesis H: K'b = m it is natural to ask, "What is
the estimator of b under the null hypothesis?" This might be especially per-
tinent following non-rejection of the hypothesis by the preceding igtest.
The desired estimator, b say, is readily obtainable using constrained least
squares. Thus b is derived so as to minimize (y — Xb)'(y — Xb) subject to
the constraint K'b = m.
[3.6] THE GENERAL LINEAR HYPOTHESIS 113
λχΧ'Χ^λσ2
Under the null hypothesis F(H) has the i^ ^.^-distribution. Hence
h—m
1 ~ tN_r.
σνλϊΧ'Χ)-
.'(X'X)-^?
[3.6] THE GENERAL LINEAR HYPOTHESIS 115
This is as one would expect, since X'fe is normally distributed with variance
χχχ'χ)-^2.
For this hypothesis the value of b is
b = b - (x'xy-^x'cx'xr^rxx'fc m)
X'b- m
= 6- (Χ'ΧΓ'λ.
χχχ'ΧΓ^.
At this point it is appropriate to comment on the lack of emphasis being
given to the /-test in hypothesis testing. This is because the equivalence of
/-statistics with F-statistics that have 1 degree of freedom in the numerator
term makes it unnecessary to consider /-tests. Whenever a /-test might be
proposed, the hypothesis to be tested can be put in the form H: X'b = m
and the F-statistic F(H) derived as here. If the /-statistic is insisted upon it is
then obtained as \/F(H). No further discussion of using the /-test is therefore
necessary.
b = h = and (X'X)- 1 =
K b = b„
and
[K'CX'X^K]- 1 = T - \
giving
(107)
In the numerator we recognize the result [e.g., Searle (1966), Sec. 9.11]
of "invert part of the inverse"; i.e., take the inverse of X'X and invert that
part of it which corresponds to bQ of the hypothesis H: bq = 0. Although
demonstrated here for a bQ that consists of the first q b's in b, it clearly applies
for any subset of q Vs. In particular, for just one b, it leads to the usual F-test
on 1 degree of freedom, equivalent to a /-test (see Exercise 15).
116 REGRESSION [3.6]
The estimator of b under this hypothesis is
= b - (X'X)- 1
V
_o_ "C(&,
-0)
T "V b*
= b - T^b = —
T *qq q u
A- u
T T_1b
2><Z <Z<Z Q-1
0
—
-K -- T T" V
*-pq *-QQ
Comparison of (110) with (109) tempts one to conclude that y'y - (SSE + Q)
is reduction(reduced), the reduction in sum of squares due to fitting the re-
duced model. The temptation to do this is heightened by the fact that
SSE + Q is residual(reduced) as in (108). However, we shall show that only
in special cases is y'y — (SSE + Q) the reduction in sum of squares due to
fitting the reduced model. It is not always so. The circumstances of these
special cases are quite wide, as well as useful, but they are not universal.
First we show that y'y — (SSE + Q) is not generally a sum of squares, since
it can be negative: for, in
y. = b0 + bxxa + b2xi2 + et
The total sum of squares for this reduced model is (y — 4xx)'(y — 4xx) and
not y'y, and so y'y — (SSE + Q) is not the reduction in sum of squares.
Furthermore, (112) is not the only reduced model, because the hypothesis
b1 = b2 + 4 could just as well be used to amend the model to be
Degrees of
Source of Variation Freedom Sum of Squares
Total N SST
Degrees of
Source of Variation Freedom Sum of Squares
Shown in Table 3.7 is the most direct way of computing its parts:
SSR = b'X'y, Q = b^T" 1 ^, SSR - Q by diiferencing, SST = y'y and
SSE by differencing. Although SSR — Q is obtained most readily by differ-
encing it can also be expressed as b'pXvXjbv (see Exercise 12). The estimator
bp is derived from (103) as
bp = bp - ΎραΎ-χ (117)
using K'iX'X)"« = T M as in (107).
[3.6] THE GENERAL LINEAR HYPOTHESIS 121
Example. For the following data
8 2 1 4
10 -1 2 1
9 1 -3 4
6 2 1 2
12 1 4 6
"11 3 21" - 1 .2145 0231 -.0680"
(X'X)-1 = 3 31 20 = .0231 0417 -.0181
21 20 73_ -.0680 -.0181 .0382
39
y'y = 425 and x'y = 55
162
We consider no-intercept models only. Then
fe7 = [—1.39 0.27 2.54]
and the analysis of variance is
Degrees of Sum of
Source Freedom Squares
1
Ö = (*i - «. - 4) [1 -1 0](X'X) -1 {k - b2 - 4)
OJ
(-1.39-0.27-4.0)2 (-5.66) 2
= 152.55.
.2145 + 0.417 - 2(.0231) .21
Hence the F-statistic for testing the hypothesis is 152.2/(52.1/2) = 5.8.
Were a reduced model to be derived by replacing bx by b2 + 4 it would be
y — 4%i = b2(x1 + x2) + b3x3 + e (118)
REGRESSION [3.6]
which the data are
y - 4a?i xx + #2 x3
0 3 4
14 1 1
5 -2 4
-2 3 2
8 5 6
The total sum of squares is now 02 + 142 + 52 + 22 + 82 = 289; and the
residual sum of squares, using SSE from the analysis of variance and Q
from the jF-statistic, is
SSE + Q = 52.1 + 152.2 = 204.3.
Therefore the analysis of variance for the reduced model is
Degrees of Sum of
Source Freedom Squares
Total 5 289.0
The value of 84.7 for the reduction in sum of squares for the reduced model
can be verified by deriving the normal equations for the model (118) directly.
From the data they are
"48 4 l l p; Γ38"
41 73j U3_ L78-
73 —41Ί [38] "-0.23
and hence = (1/1823) =
-41 48J 1.20_
Then the reduction in the sum of squares is
"38"
[-0.23 1.20] = 93.6 - 8.9 = 84.7
78
as in the analysis of variance.
These calculations are, of course, shown here purely to illustrate the sum
of squares in the analysis of variance. They are not needed specifically
because for the reduced model the residual is always SSE + Q. And the
[3.6] THE GENERAL LINEAR HYPOTHESIS 123
estimator of b can be found from (103) as
1.39 1
1
b = 0.27 - (X'X)- - 1 — (-5.66)
.21
2.54. . 0.
1.39" .2145 - .0231" 3.77]
0.27 + .0231 - .0417 (26.95) = -0.23
2.54. .-.0680 + .0181. . 1.20.
wherein bx — b2 = 4, of course, and b2 and bz are as before.
For testing the hypothesis b± = 0, Q = (—1.39)2/(.2145) = 8.9 and the
analysis of variance of Table 3.6 is
Degrees of
Source Freedom Sum of Squares
Total 5 425.0
with
~—1.39" " .2145" ' 0"
b = .27 — .0231 (—1.39)/.2145 = 0.42
2.54_ _-.0680j 2.10
Again these results can be verified from the normal equations of the reduced
model, in this case
31 20 55
.20
P
73j IAJ .I62J
ley give
73 -20] 55 0.42
= (1/1863) =
-20 3lJ |_162 2.10
above; and the reduction in sum of squares is
r
31 20" Γ0.4 ΐ
[0.42 2.10] 364.0.
20 73 2.1 0