Solutions to ECE 434 Final Exam, Spring 2003

Problem 1 (21 points) Indicate true or false for each statement below and justify your answers.
(One third credit is assigned for correct true/false answer without correct justification.)
x
x
(a) If H(z) is a positive type z-transform, then so is cosh(H(z)). (Recall that cosh(x) = e +e
.)
2
(b) If X is a m.s. differentiable stationary Gaussian random process, then for t fixed, Xt is
independent of the derivative at time t: Xt0 .
(c) If X = (Xt : t IR) is a WSS, m.s. differentiable, mean zero random process, then X is mean
ergodic in the mean square sense.
(d) If M = (Mt : t 0) is a martingale with E[Mt2 ] < for each t, then E[Mt2 ] is increasing in t
(i.e. E[Ms2 ] E[Mt2 ] whenever s t.)
(e) If X1 , X2 , . . . , is a sequence of independent, exponentially distributed random variables with
mean one, then there is a finite constant K such that P [X1 + + Xn 2n] K exp(n2 ) for all
n.
(f) If X and Y are random variables such that E[X 2 ] < and E[X|Y ] = Y , then
E[X 2 |Y 2 ] = E[X|Y ]2 .
(g) If N = (Nt : t 0) is a random process with E[Nt ] = t and E[Ns Nt ] = min{s, t} for s, t 0,
then N is a Poisson process.
(a) TRUE, The power series expansion of cosh(z) about zero is absolutely convergent yielding
cosh(H(z)) = 1 +

(H(z))2
2!

(H(z))4
+ , and sums and products of positive type functions
4!
0
= CX
(0) = 0, and uncorrelated Gaussians are independent.

(b) TRUE, since CX 0 X (0)

are positive type.

(c) FALSE, for a counter example let Xt = U for all t, where U is a mean zero random variable with 0 < Var(U ) < .
(d) TRUE, since for s < t, (Mt Ms ) Ms , so E[Mt2 ] = E[Ms2 ] + E[(Mt Ms )2 ] E[Ms2 ].
(e) FALSE, since Cram`ers theorem implies that for any  > 0, P [X1 + + Xn 2n] exp((l(2) + )n) for all
sufficiently large n.
(f) FALSE. For example let X have mean zero and positive variance, and let Y 0.
(g) FALSE. For example it could be that Nt = Wt + t, where W is a Wiener process with parameter 2 = .

Problem 2 (12 points) Let N be a Poisson random process with rate > 0 and let Yt =
(a) Sketch a typical sample path of Y and find E[Yt ].
(b) Is Y m.s. differentiable? Justify your answer.
(c) Is Y Markov ? Justify your answer.
(d) Is Y a martingale? Justify your answer.

Rt
0

Ns ds.

(a) Your sketch should show that Y is continuous and piecewise linear. The slope of Y is 0 on the first interval, 1 on
Rt
Rt
2
the second interval, 2 on the third interval, etc. E[Yt ] = 0 E[Ns ]ds = 0 sds = t2 .
(b) YES, since the integral of a m.s. continuous process is m.s. differentiable. Y 0 = N .
(c) N0, because knowing both Yt and Yt for a small  > 0 determines the slope Nt with high probability, allowing
a better prediction of Yt+ than knowing Yt alone.
(d) NO, for example because a requirement for Y to be a martingale is E[Yt ] = E[Y0 ] for all t.

Problem 3 (12 points) Let Xt = U 2 cos(2t) + V 2 sin(2t) for 0 t 1, where U and V are
independent, N (0, 1) random variables, and let N = (N : 0 1) denote a real-valued Gaussian
white noise process with RN ( ) = 2 ( ) for some 2 0. Suppose X and N are independent.
Let Y = (Yt = Xt + Nt : 0 t 1). Think of X as a signal, N as noise, and Y as an observation.
(a) Describe the Karhunen-Lo`eve expansion of X. In particular, identify the nonzero eigenvalue(s)
and the corresponding eigenfunctions.
There is a complete orthonormal basis of functions (n : n 1) which includes the eigenfunctions
found in part (a) (the particular choice is not important here), and the
R Karhunen-Lo`eve expani = (N, i ) = 1 Nt i (t)dt denote the ith
sions of N and Y can be given using such basis. Let N
0
1

1 , N
2 , . . .) are N (0, 2 ) random variables and U, V, N
1 , N
2 , . . .
coordinate of N . The coordinates (N
are independent. Consider the Karhunen-Lo`eve expansion of Y , using the same orthonormal basis.
1 , N
2 , . . . and identify the corresponding eigen(b) Express the coordinates of Y in terms of U, V, N
values (i.e. the eigenvalues of (RY (s, t) : 0 s, t 1)).
b of U given Y = (Yt : 0 t 1), and
(c) Describe the minimum mean square error estimator U
find the minimum mean square error. Use the fact that observing Y is equivalent to observing the
random coordinates appearing in the KL expansion of Y .
(a)Let 1 (t) =

2 cos(2t) and 2 (t) = 2 sin(2t). Then 1 and 2 are orthonormal and the K-L expansion of X is

simply X = U 1 (t)+V 2 (t). The nonzero eigenvalues are 1 = 2 = 1. We could write X (U, V, 0, 0, . . .). Remark:
Since 1 = 2 , any linear combination of these two eigenfunctions is also an eigenfunction. Any two orthonormal
functions with the same linear span as 1 and 2 could be used in place of the two functions given. For example,
another correct choice of eigenfunctions is n () = exp(2jnt) for integers n. We also know this choice works because X is a WSS periodic random process. For this choice, 1 and 1 are the eigenfunctions with corresponding
eigenvalues 1 = 1 = 1, and {1 , 1 } and {1 , 2 } have the same linear span.
1 , V + N
2 , N
3 , N
4 , . . .) and the eigenvalues are 2, 2, 1, 1, . . .. (c) Observing Y = (Yt : 0 t 1) is
(b) Y (U + N
equivalent to observing the coordinates of Y . Only the first coordinate of Y is relevant the other coordinates of Y are
1 .
independent of U and the first coordinate of Y . Thus, we need to estimate U given Y1 , where Y1 = (Y, 1 ) = U + N
1 )
1 )
Var
(U )Var(N
Cov
(U,Y
1
2

The estimate is given by Var(Y ) Y1 = 1+2 Y1 and the covariance of error is Var(U )+Var(N ) = 1+2 .
1
1

Problem 4 (7 points) Let (Xk : k ZZ) be a stationary discrete-time

Markov
process with state


3/4 1/4
space {0, 1} and one-step transition probability matrix P =
. Let Y = (Yt : t IR)
1/4 3/4
be defined by Yt = X0 + (t X1 ).
(a) Find the mean and covariance functions of Y .
(b) Find P [Y5 3].
(a) The distribution of Xk for any k is the probability vector solving = P , or = ( 21 , 12 ). Thus P [(X0 , X1 ) =
(0, 0)] = P [(X0 , X1 ) = (1, 1)] =
E[Xi ] =

E[Xi2 ]

1
2

1 3
2 4

and E[X0 X1 ] =

=
3
8

3
,
8

1 1
= 18 . Thus,
2 4
1
and
. Thus, E[Yt ] = 1+t
8
2
1
s+t
st
+
+
.
4
8
4

and P [(X0 , X1 ) = (0, 1)] = P [(X0 , X1 ) = (1, 0)] =

so Var(Xi ) =

1
4

and Cov(X0 , X1 ) =

3
8

( 12 )2

Cov(Ys , Yt ) = Cov(X0 + sX1 , X0 + tX1 ) = Var(X0 ) + (s + t)Cov(X0 , X1 ) + stVar(X1 ) =

(b) Since Y5 = X0 + 5X1 , the event {Y5 3} is equal to the event {X1 = 1}, which has probability 0.5.

Problem 5 (6 points) Let Z be a Gauss-Markov process with mean zero and autocorrelation
function RZ ( ) = e| | . Find P [Z2 1 + Z1 |Z1 = 2, Z0 = 0].
Since Z is a Markov process, the conditional distribution of Z2 given Z0 and Z1 depends only on Z1 . Note the if Z2
(Z2 ,Z1 )Z1
is estimated by Z1 , then the minimum mean square error estimator is E[Z2 |Z1 ] = Cov
= e1 Z1 , and the
Var(Z1 )
(Z2 ,Z1 )2
2
estimation error is independent of Z1 and is Gaussian with mean zero and variance Var(Z2 ) Cov
Var(Z ) = 1 e .
1

Thus, given Z1 = 2, the conditional distribution of Z2 is Gaussian with mean 2e1 and variance 1 e2 . Thus, the
desired conditional probability isthe sameas the probabilty a N (2e1 , 1 e2 ) random variable is greater than or
32e1
equal to 3. This probability is Q
.
2
1e

Problem 6 (10 points) Let X be a real-valued, mean zero stationary Gaussian process with
b0 = c1 Xa + c2 Xa where the constants c1
RX ( ) = e| | . Let a > 0. Suppose X0 is estimated by X
and c2 are chosen to minimize the mean square error (MSE).
b0 )2 ].
(a) Use the orthogonality principle to find c1 , c2 , and the resulting minimum MSE, E[(X0 X
(Your answers should depend only on a.)
b0 as defined above is the minimum MSE
(b) Use the orthogonality principle again to show that X
estimator of X0 given (Xs : |s| a). (This implies that X has a two-sided Markov property.) (a)The
b0 Xa and X0 X
b0 Xa ., or equivalently ea [c1 e2a + c2 ] = 0 and
constants must be selected so that X0 X

ea [c1 + c2 e2a ] = 0. Solving for c1 and c2 (one could begin by subtracting the two equations) yields c1 = c2 = c
where c =

ea
1+e2a

1
ea +ea

1
.
2cosh(a)

b02 ] = 1 c2 E[(Xa + Xa )2 ] = 1 c2 (2 + 2e2a ) =

The corresponding minimum MSE is given by E[X02 ] E[X

1e2a
1+e2a

Tanh(a).
b0 ) Xu whenever |u| a.
(b) The claim is true if (X0 X
1
(eau + eau )
ea +ea
a+u
u
1
Xa ))Xu ] = e ea +e
a (e

If u a then E[(X0 c(Xa + Xa ))Xu ] = eu

= 0.

Similarly if u a then E[(X0 c(Xa +

+ ea+u ) = 0.

The orthogonality condition is thus true whenever |u| a, as required.

Problem 7 (12 points)

Suppose X and N are jointly WSS, mean zero, continuous time random processes with RXN 0.
The processes are the inputs to a system with the block diagram shown, for some transfer functions
K1 () and K2 ():

K1

Y=X out+Nout

K2

Suppose that for every value of , Ki () 6= 0 for i = 1 and i = 2. Because the two subsystems are
linear, we can view the output process Y as the sum of two processes, Xout , due to the input X,
plus Nout , due to the input N . Your answers to the first four parts should be expressed in terms
of K1 , K2 , and the power spectral densities SX and SN .
(a) What is the power spectral density SY ?
(b) What is the signal-to-noise ratio at the output (equal to the power of Xout divided by the power
of Nout )?
(c) Suppose Y is passed into a linear system with transfer function H, designed so that the output
bt , the best (not necessarily causal) linear estimator of Xt given (Ys : s IR). Find
at time t is X
H.
(d) Find the resulting minimum mean square error.
(e) The correct answer to part (d) (the minimum MSE) does not depend on the filter K2 . Why?
2
(a) SY = |K1 K2 |2 S
X + |K2 | SN , where for notational brevity we suppress the argument () for each function.
R

(b) SN Routput =
(c) H =
(d)

SXY
SY

2
d
|K1 K2 | SX 2
R
d
2S
|K
|
2
N

K1 K2 SX
|K1 K2 |2 SX +|K2 |2 SN

.
.
Z

M M SE

SX SY |H|2

Z
=

|K1

d
2

|K2 |2 SX SN
d
2
2
2 | SX + |K2 | SN 2

|2 |K

SX SN
d
|K1 |2 SX + SN 2

(e) Since K2 is invertible, for the purposes of linear estimation of X, using the process Y is the same as using the
process Y filtered using a system with transfer function

1
.
K2 ()

the filter K2 were dropped from the original system.

Equivalently, the estimate of X would be the same if