Conditional expectation and conditional variance of

multivariate normal variable

Conditional distribution of bivariate normal distribution:

The conditional density of X1 , given that X2 = x2 for any bivariate distribution, is defined by

f (X1 |X2 = x2 ) = {conditional density of X1 given that X2 = x2 }

f (x1 , x2 )
= .
f (x2 )
where, f (x2 ) is the marginal distribution of X2 .
Note: If f (x1 , x2 ) is the bivariate normal density, then f (x1 , x2 ) ∼ N (µ1 + σσ22 12
(x2 − µ2 )). Here
 2 
σ1 σ12
mean µ = (µ1 , µ2 ) and covariance Σ = .
σ12 σ22
2
Marginal distribution of X2 = √2πσ 1
e−(x2 −µ2 ) /2σ2 . Therefore,
2 √
f (x1 , x2 ) σ2 −σ2 [x1 −µ1 −(σ1 /σ2 )(x2 −µ2 )2 ]/2(σ1 σ2 −σ12
2 )
f (X1 |X2 = x2 ) = =p e .
f (x2 ) 2πσ1 (σ1 σ2 − σ12 2
)

Multivariate Case:
 
X1
Let X ∼ Np (µ, Σ) be a p variate normal variable. Define the partition of X as X =  · · ·  ,
  X2
X1
X1 = [x1 , x2 , .., xk ]0 , X2 = [xk+1 , xk+2 , ..., xp ]. Such that mean µ =  · · ·  and covariance Σ =
X2
..
 
Σ . Σ12
 11 . 
 · · · .. · · ·  , and Σ22 > 0. Then the condition distribution of X1 given X2 = x2 , is normal
 
..
Σ21 . Σ22
and has mean (expectation) E(X1 |X2 = x2 ) = µ1 + Σ12 Σ−1 22 (x2 − µ2 ) and covariance cov(X1 |X2 =
−1
x2 ) = Σ11 − Σ12 Σ22 Σ21 . (Note: covariance does not depends on the value of x2 of the conditioning
variable X2 .
 
4 0 −1
Problem: Let X be distributed as N3 (µ, Σ), where µT = (µ1 , µ2 , µ3 ),and Σ =  0 5 0  .
−1 0 2
Which of the following random variables are independent? Explain.

1
Solution:

Problem: In the above example specify each of the following

1. The conditional distribution of X1 , given that X3 = x3 .

2. Find the conditional distribution of X1 , given that X2 = x2 ,X3 = x3 .

Solution:

2
3
Genralized inverse of a matrix: For a general matrix A of order m × n, its generalized inverse
is a matrix G of order n × m such that AGA = A.
 
A11 A12
If A = of order m × n has rank r, and A11 is matrix of order r × r. Then if inverse
A21 A22  −1 
A11 0
of A11 exist then generalized inverse of A can be given as G = .
  0 0
1 2 3
Example: A = 4 5 6  has rank 2. Therefore, therefore its generalized inverse G can be given

7 8 9
as  5 2 
−3 3 0
G =  43 − 13 0  .
0 0 0

Example: Let X ∼ N3 (µ, Σ), be normalvariate such that X T = (x1 , x2 , x3 ),µT = (1, −1, −1)
1 0 0
and covariance Σ = 0 4 2  the find the normal pdf of X.

0 2 1

Solution: Here, det(Σ) = 0, i.e. the covariance matrix is singular. Therefore the corresponding
probability density function can be given as

4
 1 T Σ− (X−µ)
f (X) = e−(X−µ)
(2π)k/2 (λ 1 .λ2 ...λk )

where λ1 .λ2 ...λk are the nonzero eigenvalues of covariance matrix Σ, and Σ− is the generalized
inverse of Σ.
Now, eigenvalues of Σ are λ = 0, 1, 5 (Rank 2). So, the nonzero eigenvalues are λ1 = 1, λ2 = 5,
and k = 2.  
1 0 0
Also the generalized inverse of Σ is Σ− =  0 1/4 0  . Therefore the joint probability distribu-
0 0 0
tion of singular normal variable X can be given as h
1 2
i
− (x1 −1)2 +(x2 + 41 ) /2
f (X) = e .
(2π)2/2 (1 × 5)
Since,
 
1 0 0 x1 − 1
(X − µ)T Σ− (X − µ) = [x1 − 1 x2 + 1 x3 + 1]  0 1/4 0   x2 + 1 
0 0 0 x3 + 1
 2
1
= (x1 − 1)2 + x2 + .
4