Scribd
Upload
140 views4 pages

Roweis Gaussianidentities

This document summarizes various identities and properties of Gaussian distributions. It discusses multidimensional Gaussian distributions, linear functions of Gaussian vectors, marginal and conditional distributions, multiplication of Gaussians, quadratic forms, convolution of Gaussians, the Fourier transform of Gaussians, and constrained maximization of quadratic forms.

Uploaded by

knithx2
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
140 views4 pages

Roweis Gaussianidentities

This document summarizes various identities and properties of Gaussian distributions. It discusses multidimensional Gaussian distributions, linear functions of Gaussian vectors, marginal and conditional distributions, multiplication of Gaussians, quadratic forms, convolution of Gaussians, the Fourier transform of Gaussians, and constrained maximization of quadratic forms.

Uploaded by

knithx2
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

gaussian identities

sam roweis
(revised July 1999)
0.1 multidimensional gaussian
a d-dimensional multidimensional gaussian (normal) density for x is:
N (, ) = (2)
d/2
||
1/2
exp
_

1
2
(x )
T

1
(x )
_
(1)
it has entropy:
S =
1
2
log
2
_
(2e)
d
||
_
const bits (2)
where is a symmetric postive semi-denite covariance matrix and the
(unfortunate) constant is the log of the units in which x is measured over
the natural units
0.2 linear functions of a normal vector
no matter how x is distributed,
E[Ax +y] = A(E[x]) +y (3a)
Covar[Ax +y] = A(Covar[x])A
T
(3b)
in particular this means that for normal distributed quantities:
x N (, ) (Ax +y) N
_
A +y, AA
T
_
(4a)
x N (, )
1/2
(x ) N (0, I) (4b)
x N (, ) (x )
T

1
(x )
2
n
(4c)
1
0.3 marginal and conditional distributions
let the vector z = [x
T
y
T
]
T
be normally distributed according to:
z =
_
x
y
_
N
__
a
b
_
,
_
A C
C
T
B
__
(5a)
where C is the (non-symmetric) cross-covariance matrix between x and y
which has as many rows as the size of x and as many columns as the size of
y. then the marginal distributions are:
x N (a, A) (5b)
y N (b, B) (5c)
and the conditional distributions are:
x|y N
_
a +CB
1
(y b), ACB
1
C
T
_
(5d)
y|x N
_
b +C
T
A
1
(x a), BC
T
A
1
C
_
(5e)
0.4 multiplication
the multiplication of two gaussian functions is another gaussian function
(although no longer normalized). in particular,
N (a, A) N (b, B) N (c, C) (6a)
where
C =
_
A
1
+B
1
_
1
(6b)
c = CA
1
a +CB
1
b (6c)
amazingly, the normalization constant z
c
is Gaussian in either a or b:
z
c
= (2)
d/2
|C|
+1/2
|A|
1/2
|B|
1/2
exp
_

1
2
(a
T
A
1
a +b
T
B
1
b c
T
C
1
c)
_
(6d)
z
c
(a) N
_
(A
1
CA
1
)
1
(A
1
CB
1
)b, (A
1
CA
1
)
1
_
(6e)
z
c
(b) N
_
(B
1
CB
1
)
1
(B
1
CA
1
)a, (B
1
CB
1
)
1
_
(6f)
2
0.5 quadratic forms
the expectation of a quadratic form under a gaussian is another quadratic
form (plus an annoying constant). in particular, if x is gaussian distributed
with mean m and variance S then,
_
x
(x )
T

1
(x )N (m, S) dx
= ( m)
T

1
( m) + Tr
_

1
S

(7a)
if the original quadratic form has a linear function of x the result is still
simple:
_
x
(Wx )
T

1
(Wx )N (m, S) dx
= ( Wm)
T

1
( Wm) + Tr
_
W
T

1
WS

(7b)
0.6 convolution
the convolution of two gaussian functions is another gaussian function (al-
though no longer normalized). in particular,
N (a, A) N (b, B) N (a +b, A+B) (8)
this is a direct consequence of the fact that the Fourier transform of a gaus-
sian is another gaussian and that the multiplication of two gaussians is still
gaussian.
0.7 Fourier transform
the (inverse)Fourier transform of a gaussian function is another gaussian
function (although no longer normalized). in particular,
F [N (a, A)] N
_
jA
1
a, A
1
_
(9a)
F
1
[N (b, B)] N
_
jB
1
b, B
1
_
(9b)
where j =

1
3
0.8 constrained maximization
the maximum over x of the quadratic form:

T
x
1
2
x
T
A
1
x (10a)
subject to the J conditions c
j
(x) = 0 is given by:
A +AC, = 4(C
T
AC)C
T
A (10b)
where the jth column of C is c
j
(x)/x
4

You might also like