ICAL STATIS

PLE AND PARTIAL CORRELATION AND REGRESSION ANALYSIS 12-5

A Then the Sere, we make N observations on each of the variables X, X, and Xy

qu
nomal equations for estimating hyy And are
have been mes -0-2X,(X, -bysX, hyy Xa)
(123)
-0--2x,(x, h1X, hy X)
i by (12.3a)
change of arn Sx, X, -0 and
Ex,x, h21}x}'-byx,x, -0
Ex, X,, -0
. (12-3b)
Ex,x, -b,1}x,X, -b2x,' =0
ents of X, on X, Sine X,'s are measured from their respective means, we have
Cov (X,, X) Ex, x, ... 12.3c)
x. Cov (X,, X,)-2x, x, and

hd the Hence, from (12-36), we get

quantity ... (12-3d)

ne of Solving the equations (12-3d) simultaneously for b,23 and b,a We get
regression at 12 O1 12

|13 O1 |13
...(12-4)
T23 O3
1 I23 1- 3
23 1
are known as pra 23 O2
a regression coeffic 1
are the regre O, 23 13
PSsion coefficient
ay be pointed out SEmilarly, we will get b,a2 , |1 T23 O3 1-r (12-4a)
e order of the prima
T23 1
endent variable bu plane of regression
o primary subscra Substituting these values in (12-2), we obtain the required equation of the
L on (X X4). we have
By symmetry of the results in (12-4) and (12-4a),
y subscripts in it e
ively. (i #j#k= 1, 2, 3) . .(12-4b)
pnsideration have be o; (1-r)
1 h2 13

) = 1 1
... (12-5)
if we write
... (12 1
31 3
es, i.e., by minimisg 1
31 T32
= (1-r)-;2(i2 -h3a)+ h3(72 - h3)
variables.
= 1 - - - +2r2 ha Kas |:;=1 and r i j= 1, 2, 3]

h mm
 FUNDAMENTALS OF MATHEMATICAL STATISTI MUATIPLE A!
12-6

and ). = (Cofactor of elemernt in the ith row and jth column of o) (124 Using the
= (-1)*l x[Determinant obtained on deleting the ith row and jth column
Then 0,, = (Cofactor of element in lst row and 1st column of o)
1 T23

| 732 1

=-(h2-h3 z3)=(h3 23 -h2)

T31
1
O13 = g1 = (-1) 3 = h2 23 - h3
731 T32
1 T13

31
1
-1
1
Substituting from (12-6a) to (12-6e) in (12-4) and (12-4a), the equation of the plane of regmes
of X, on (X, and X) becomes:
X, = -01 O12 -X O1.13
O3 11
X, W12 O13 = 0

Similarly, the equation of the plane of regression of X, on (Xy, X,) willbe given by
X W31 + O32 X3 O33 = 0

In general, the equation of the plane of regression of X, on (x, X) is given by:

X = b,. X, + ba., X : (i #j*k= 1,2, 3),
g
1 - .and bk-j
where b, 1
12-3-1. Generalisation of the Plane of Regression to n Variables
Consider 1-variate distribution with variables X,, X, ., X Assuming that X(s; i = 1Z
regression
are measured from their respective means, the equation of the plane of
(X, Xy ., X) is given by:
X= b34 X t by24X, t . + bln23 . (n - i) n
X, on X,
where b, 34 nb1324 n , by:23 (n-) are partial regression coefficients of
respectively.
The sum of the squares of residuals is given by

S = 2X,23= 2x, - b,234 X, - bis24 X, - .. -bin23 u-)

 12-8
FUNDAMENTALS OF MATHEMATICAL STATIST PLE AND

-O12 0,, =0
as the required equation of the plane of
regression of X, and X, Xy , X
Equation (12-12) can be re-written as:
X, = X,- e popa
O, O1 3 T
Comparing (12-12a) with (12-8), we get

b12.34...n
b13.24..n Q13

1n.23..(n -1) =
Remarks:
1. From the symmetry of the result obtained in (12-13) the equation of the plane of regression of I,
on the remaining variables X, (j #i=1, 2, .., n), is given by:
Al 0,+2 0, +.. +^o, +. + , = 0 ; i=1, 2, ., n
g,
2. We have

and

Since each of o,, o,, O,, and o,, is non-negative and o,, =0,, lcf. Remarks 3and 4 to §12.5), S
of each regression coefficient b234.. n and b134 depends on @
12-4. PROPERTIES OF RESIDUALS

Property 1. The sum of the product of any residual of order zero with any other residual of higher
is zero, provided the subscript of the former occurs anong the secondary subscripts of the latter.
The normal equations for estimating b's in trivariate and n-variate distributions, as obtaime
equations (12.3a) and (12-9a), are:
LX, X3 = 0, 2X, X12 = 0 and ZX, X;3.in =0; i=2, 3, .., n
respectively. Here X,, (i = 1, 2, 3, ..., n) can be regarded as a residual of order zero. Hence the
Property 2. The sum of the product of any two residuals in uhich all the secondary subscripts af e
ocCur anmong the secondary subscripts of the second is unaltered if we onit any or all of the s
subscripts of the first. Conversely, the product sum of any residual of order 'p' with a residuat
p+q, the 'p' subscripts being the same in each case is unaltered by adding to the secondary
of the former any or all the 'q' additional subscripts of the latter.
Let us consider:
Ex,;X,3 =E(x,- b,, X)X;3=2X, X1 - b 2x, X,s
- Zx, X;23 (Using P
Also

-Ex, Xy3 (Lisina

 (Using
Property
1) as (Using
Property
1)
well 1) 12-4) (1215) S
as
Property §2 ,o, .,
(primary Property . Goy
J ) oy
(Using - n,0,
X .-in23..(1 by
X)
X;23. (Using(n-)
ANALYSIS subscripts C
An have )
e.g.,
X3.
n-1) =0
bin23 -nbn:23.. ,
0= ..(n-1) Cy 2n
the other,X1, X34.-b435.2X,
viz., .. we )E(X23. -...- - C, 12 1
REGRESSION T130,0,
+
byn23means,
...
X, all X, =0 also
the Ex, -b2n34 =0 X, ... get
we(12-96), +0
h2
if y + t and :
zeroof -b,,
subscripts. Xy .. respective bi324. nbi324..
80,0,
+ respectively
X is X)-
residuals by435.X X,, X by: - b324.1
2X, b324.1 X,
bi2s4.n-I20,o, and
AND on
2X, Xj.-4.,2X, measured given (12-15)
secondary X, from
their
CORRELATION -byn34..
-1) n +
X,n Lx, tw0 = of .., O, Yon=0
X;u -
X,)X,, +
regression
X, 2,
is
- n o23..,=b234..g,0,
X, by234.. y
-Lx,, of X i=1,residual Goy 11 :
the
product bz4. Residualb234..
the =EX, equations
among
b,, o- oy **
0; the by 12 :
the -2(x, 2|(X,
- 1
PARTIALxin
of 2
occur 2X, plane= are = of
X,
= in R, o»123..
of )E(X,variance
2. sum 3. b's
propertyone- = =
in
of the X's
property
41.
Variance the Ryy get
we
pectively,
AND Xy12 X123..1
The the considerthe the o,-Eliminating R,,
ATIPLE 3. of 2x,, the
all Dividing
the perty
eniary) Sence Hence,
tne, e, s
EMATICAL
STATIS (12-7
.. s resukfrs secondar subscripa
secondary Property 1)Property
1(Using Using
the an
X,
higher
ions,
as
obtained the or
of 12.5], latter. the of of
regression n residual
§ of the ..., Hencesubscripts
the
to Tesidual 3, of
4 of 2, ero. all a
of and sts
= r h
 peral,
STATISTC 2ha
(12
... §2.
xis-Ex,X,s 1-i2
3
§
=
observations [Frcm.
(1. correcorre
on X2 property 2,
property
simple
theX,
MATHEMATICAL -
is ofregressionX
N theR,.23 =
(Using (Using
e123
has is words
X, R,2y
3]
2, =2X-2Xin
andby otherofplane
1,
i=
e;a3) -Xy3)+2X,X123)=xË
have;
OF 011 X, denoted
FUNDAMENTALS of X,, In the e123 0we =
E(X; -E(X, E--Ex+Exi
xi
=
X,. by
variables
usually
on
E(X,)
- means, Ele:as)}]=X-1123)
12
X, given X
1 :
W11
CORRELATION
X,
and = [:
Xy respective
-eia - o-Joi(o
i23)
0
the as
oi23 (12.6). of andX, value Xg
b32 -{e:a3
0z)= V(e,2s)
JV(X,) -Ex;-x,xË,
X -o;
oí23
eachX, of by32
- (X1
e123)
Cov
=E(xjo23
and
estimated
effect X E(e, E(X)} = E(ei3)+
o-o3
on
which + by3their get
0 (12-5)MULTIPLEX, joint X, and -=E[{X, -o,' we
: =
distribution, of b23- from
2n 011 in coefficient
theits X, 0 =
Vie,g3) = (12-17),=Ry:23
1 in distribution
and and = measure
= =
E(X123)
12 defined e23X23 R23 e,3)
1 : oi23 OF X, X,
tri-variate COEFFICIENT
correlation
between in
are between are (X, Substituting
tri-variate
o,, We X{s By Cov
Since def.,
have
a and coefficient
coefficient Also
In
Remark.o
multiple viz.,
X,
where12-5.a
12-10 In
 (12
1
(On
simplification) ..
(12-18a)
correlation
...
(12-18b)
X, ..()
.,
Xy X2.
.1.
total X,,
20 on X-
ANALYSIS the
r,,) X, e,an=
of
of
terms -
(1 coeficiernt ) -2x+
EX;-2x,X3.
(-1)3 Xy3. \1/2
REGRESSION in and -2(X
+Xi3.-2X,X123.) o-o123.
(From
(12.160)) -
coefficient correlation
X,
=
Wgg
,2X,(X
h3 between -Ex-Ex,
X123.
3 ("
Irgls
1)
r2 correlation multiple
AND -2 Cov
(X1,Jv(x,)(e123..
) e123.n
e1-23..n)
coefficient Jo(o-o23..)
and
the o23.1
CORRELATION -- variables. distribution, -Ex;
e,3)=2X,
multiple -20 -r3)20 correlation -
) O11 o
1-
R3
= =
get: theof O,
=1-
>0020 (1 3 n-variate
the ...1
1 pairs *2.1,2, R23 Ve3.
PARIIAL we expresses is
(0=
(12-18),
(-1?
=
20;i (X,,
the of
= caseR,ga Cov
between
AND from R,,s1 a,, 0,, In by
fornmula emark.
From
(*), eneralisation.
denoted
ULTIPLE Hence, efficients
This general,
Sually
ere
12-13 (12-19a)
... variableof are multipie
plane
values that:
=0.4912
X,).
dependent
the the given
data
and of all 0-4
fit given findG1) x 1-0-3721 =0-3735
(X, of 0-61 0-3084
ANALYSIS the 04,and
theon closeness x
that = 11
0:7 (0-4)?
1-
in X, r,,¡, 2x 0-3416
variation
of indicate y 0-3416
formulae: =1-5336
=1-1472 2:7386
the and -
REGRESSION regressionfit.
of better
o, 3-46. 61)
0-6196
= (0-61)?
1- - 1-0.49
061 estimates 0-3721 3
measure
1 (0 0-16 2, 1-0-6196 =
total to =
¡, +
/1-0-3735
/1-0-4912
error a = 0.5205 the 1, that 1-,2
and (0-7)'
nearer
is r, 0-84 + + k=
squares of
the 0-49 0-16
it of using)ii*j# prove
to planeaas hence07,errors 2-15
due of Rí23
AND
squares the
interpreted=
of proportion and r,, = ri3
3 0.3416
by =1-862-15 3-46 then (Given)
23 221+r
Sum
CORRELATION of
if standard
0-7871 0-7009 =
=0-6111
y0-3735 = =
R12 h3
with values
regression
o, ri2
distribution,
1·86,
1-0.16 1-å
obtained Ri23) R13)=
Rin) 1),
r2
Total ro-2 0-3721 R;. r n+r2
= (1-r)
1-r21²(1-)
= =
of
Sum
be
theassociated
The
also
1
the = V0-6196 1 V0-4912 - (1- (1- N(1- 1,(r =
R123
R231 3
o, t + be (1 = r)
as values.
of obtain io 0.49
can ¡; Ë Oz =
Oa21
G3 a = (1+
may plane
interpreted 13
trivariate = o;.*= =
PARTIAL 1
Also = Estimate= Ta =
= is Rí2adata R123 Ra.13 Rg21 O:23 O13 T12
or the
=
abouta coefficients.
by of actual r,,
AND be explained
value In of If
may 12.1.
thescattered Error 122.
LTPLE Theto Sution.
(i)
n mpleetion dard Enanple tion.
(12
-. s determint
STA
ATICAL becaus conca association
betwee expected
values,
wha
multiple ariables
determined
b calcu + as regre Example
kac correl
conclude
that
and0SR, m
and are
we
are linear the multiple
negative, -1 zero, formula. X,
and multiple
X,)
(X2,
on
) the valuesbetween [See and
... multiple
e23 from are case 23 13
be
zero. X, to
, ected duals cion of ing
(22.
er thishen
ed lie re t
e
12:14
FUNDAMENTALS OF MATHEMATICAL STATIST PLE A
Aer, Wie
(: 0s R2s1)
Similarly, we get

Ri = 2
1+r
1- r
2
Cthis
and
R12 1-r2 1+r Smariy,
Ry3 =Ra31 = Rg12 =
12-6. COEFFICIENT OF PARTIAL CORRELATION
Sometimes the correlation betweern two variables X, and X, may be partly due to the correlat
of a third variable, X, with both X, and X,. In such a situation, one may want to
know what n
correlation between X, and X, would be if the effect of X, on each of X, and X, were eliminztn
This correlation is called the partial correlation and the correlation coefficient
between X, an2
after the linear effect of X, on each of them has been eliminated is called the partial
correia
coefficient.
The residual X,, = X- b,a X, may be regarded as that part of the variable X, which rera
after the linear effect of X, has beern eliminated. Similarly, the residual Xa may be ecative sig
interpaee
as the part of the variable X, obtained after eliminating the linear effect of X. Thus the pata
correlation coefficient between X, and X,, usually denoted by r is given by:
12.3 =
Cov (X13, X23)
(12
VVar (X13) Var (X23)
We have, assuming that X,s, (i =1, 2, .., n) are measured from their respective mears.

Cov (X;3 Xy,)= 2X,s X3 X,Xs 2X,(X, -bzsXs)

O2.(h3 G03) Thes
=o,0, (n - I3 Tzg) pper

V(X;g)=
Ex;X1 X, (X,-thy X,)
=
Ex - bhg Ex, X, =o-hs(hs G10,)
le can
= oj(l-) pedicc.
ethe
Similarly, we shall get : V(X,,) = G,(1 - r)
Substituting in (12-20), we get :
alsatio.
"123 = G,0; (h2 - I13 I23) h2 - h3 23 ... (12-2
Voi (1-) o; (1 -r) J1-)(1-')