Chapter 7: Analysis and Processing of Random

Signals1
Yunghsiang S. Han

Graduate Institute of Communication Engineering,

National Taipei University
Taiwan
E-mail: [email protected]

1 Modified

from the lecture notes by Prof. Mao-Ching Chiu

Y. S. Han

Analysis and Processing of Random Signals

7.1 Power Spectral Density

Fourier series and Fourier transform Analysis of

nonrandom time function in the frequency domain.
For WSS processes X(t), the autocorrelation function
RX ( ) is an measure for the average rate of change of
X(t).
Einstein-Wiener-Khinchin Theorem: Power
spectral density of a WSS random process is given by
the Fourier transform of the autocorrelation function.

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Continuous-Time Random Process

X(t) is a continuous-time WSS random process with
mean mX and autocorrelation function RX ( ).
The power-spectral density of X(t) is given by the
Fourier transform of the autocorrelation function.
SX (f ) = F{RX ( )}
Z +
RX ( )ej2f d.
=

If X(t) is real value, then

RX ( ) = RX ( ).
Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

We have
SX (f ) =

RX ( )[cos(2f ) + j sin(2f )]d

RX ( ) cos(2f )d.

Inverse Fourier transform is given by

RX ( ) = F 1 {SX (f )}
Z +
SX (f )ej2f df.
=

Average power of X(t) is

E[X 2 (t)] = RX (0) =

SX (f )df.

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

SX (f ) is the density of power of X(t) at the frequency

f.
Since RX ( ) = CX ( ) + m2X , the power spectral density
is also given by
SX (f ) = F{CX ( ) + m2X }
= F{CX ( )} + m2X (f ).
Note that m2X is the dc component of X(t).
Cross-power spectral density SX,Y (f ) is defined by
SX,Y (f ) = F{RX,Y ( )},
where
RX,Y ( ) = E[X(t + )Y (t)].
Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

In general, SX,Y (f ) is a complex function of f .

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Example: The autocorrelation function of the random

telegraph process is given by
RX ( ) = e2| | .
The power spectral density is
Z
Z 0
e2 ej2f d +
SX (f ) =

e2 ej2f

1
1
+
=
2 j2f
2 + j2f
4
=
.
2
2
2
4 + 4 f

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Example: The power spectral density of a WSS white

noise whose frequency components are limited to
W f W is shown in the following figure:

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

The average power is

2

E[X (t)] =

N0
df = N0 W.
2

The autocorrelation function for this process is

Z W
1
ej2f df
N0
RX ( ) =
2
W
1 ej2W ej2W
=
N0
2
j2
N0 sin(2W )
=
.
2

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

White noise usually refers to a random process W (t)

Graduate Institute of Communication Engineering, National Taipei University

10

Y. S. Han

Analysis and Processing of Random Signals

whose power spectral density is N0 /2 for all frequencies:

N0
SW (f ) =
2

for all f .

White noise has infinity average power.

Autocorrelation function of W (t) is
N0
( ).
RW ( ) =
2
If W (t) is a Gaussian random process, then W (t) is the
white Gaussian noise process.

Graduate Institute of Communication Engineering, National Taipei University

11

Y. S. Han

Analysis and Processing of Random Signals

Example: Find the power spectral density of

Z(t) = X(t) + Y (t), where X(t) and Y (t) are jointly WSS
process. The autocorrelation function of Z(t) is
RZ ( ) = E[Z(t + )Z(t)]
= E[(X(t + ) + Y (t + ))(X(t) + Y (t))]
= RX ( ) + RY X ( ) + RXY ( ) + RY ( ).
The power spectral density is
SZ (f ) = F{RX ( ) + RY X ( ) + RXY ( ) + RY ( )}
= SX (f ) + SY X (f ) + SXY (f ) + SY (f ).

Graduate Institute of Communication Engineering, National Taipei University

12

Y. S. Han

Analysis and Processing of Random Signals

Discrete-Time Random Process

Let Xn be a discrete-time WSS random process with
mean mX and autocorrelation function RX (k).
The power spectral density of Xn is defined as the
Fourier transform
SX (f ) = F{RX (k)}

X
RX (k)ej2f k .
=
k=

We only need to consider frequencies in the range

1/2 < f 1/2, since SX (f ) is periodic in f with
period 1.
Graduate Institute of Communication Engineering, National Taipei University

13

Y. S. Han

Analysis and Processing of Random Signals

Inverse Fourier transform is given by

Z 1/2
SX (f )ej2f k df.
RX (k) =
1/2

The cross-power spectral density SXY (f ) of two

joint WSS discrete-time processes Xn and Yn is defined
by
SX,Y (f ) = F{RX,Y (k)}
and
RX,Y (k) = E[Xn+k Yn ].

Graduate Institute of Communication Engineering, National Taipei University

14

Y. S. Han

Analysis and Processing of Random Signals

Example: Let the process Xn be a sequence of

uncorrelated random variables with zero mean and
2
variance X
. Find SX (f ).
(
2
X
k=0
RX (k) =
.
0
k 6= 0
The power spectral density of the process can be found to
be
1
1
2
<f < .
SX (f ) = X
2
2

Graduate Institute of Communication Engineering, National Taipei University

15

Y. S. Han

Analysis and Processing of Random Signals

16

Example: Let Yn = Xn + Xn1 , where Xn is the white

noise process given in the previous example. Find SY (f ).
Sol: The mean and autocorrelation function of Yn are
given by
E[Yn ] = 0
and
E[Yn Yn+k ] =

2
2

(1
+

X k = 0

2
X

k = 1

otherwise

The power spectral density is then

2
2
(ej2f + ej2f )
+ X
SY (f ) = (1 + 2 )X
2
{(1 + 2 ) + 2 cos(2f )}.
= X
Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

=1

Graduate Institute of Communication Engineering, National Taipei University

17

Y. S. Han

Analysis and Processing of Random Signals

Example: Let the observation Zn is given by

Zn = Xn + Yn , where Xn is the signal we wish to observe,
Yn is a white noise process with power Y2 , and Xn and Yn
are independent. Suppose that Xn = A for all n, where A
is a random variable with zero mean and variance A2 . Find
the power spectral density of Zn .
Sol: The mean and autocorrelation of Zn are
E[Zn ] = E[A] + E[Yn ] = 0
and
E[Zn Zn+k ] = E[(Xn + Yn )(Xn+k + Yn+k )]
= E[Xn Xn+k ] + E[Xn ]E[Yn+k ]
Graduate Institute of Communication Engineering, National Taipei University

18

Y. S. Han

Analysis and Processing of Random Signals

+E[Xn+k ]E[Yn ] + E[Yn Yn+k ]

= E[A2 ] + RY (k).
Thus Zn is also a WSS process. The power spectral density
of Zn is then
SZ (f ) = E[A2 ](f ) + SY (f ).

Graduate Institute of Communication Engineering, National Taipei University

19

Y. S. Han

Analysis and Processing of Random Signals

Power Spectral Density as a Time Average

Let X0 , . . . , Xk1 be k observations from the
discrete-time, WSS process Xn . The Fourier transform
of this sequence is
xk (f ) =

k1
X

Xm ej2f m

m=0

|
xk (f )|2 is a measure of the energy at frequency f .
Divide this energy by total time k, we obtain an
estimate for the power at frequency f :
1
xk (f )|2 .
pk (f ) = |
k
Graduate Institute of Communication Engineering, National Taipei University

20

Y. S. Han

Analysis and Processing of Random Signals

pk (f ) is called the periodogram estimate.

Consider the expected value of the periodogram
estimate:
1
E[
xk (f )
xk (f )]
E[
pk (f )] =
k "
#
k1
k1
X
X
1
j2f m
Xi ej2f i
Xm e
=
E
k
m=0
i=0
k1
k1 X
X
1
=
E[Xm Xi ]ej2f (mi)
k m=0 i=0

k1 k1
1 XX
=
RX (m i)ej2f (mi) .
k m=0 i=0
Graduate Institute of Communication Engineering, National Taipei University

21

Y. S. Han

Analysis and Processing of Random Signals

Graduate Institute of Communication Engineering, National Taipei University

22

Y. S. Han

Analysis and Processing of Random Signals

23

By the above figure, we have

1
E[
pk (f )] =
k
=

k1
X

j2f m

{k |m |}RX (m )e

m =(k1)
k1
X

m =(k1)




|m |
j2f m
.
RX (m )e
1
k

As k , we have
E[
pk (f )] SX (f ).
The above result shows that SX (f ) is nonnegative for all f
since pk (f ) is nonnegative for all f .
Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

For continuous-time WSS random process X(t), based on

the observation in the interval (0, T ), we have
1
xT (f )|2 .
pT (f ) = |
T
The result shows
lim E[
pT (f )] = SX (f ).

Graduate Institute of Communication Engineering, National Taipei University

24

Y. S. Han

Analysis and Processing of Random Signals

7.2 Response of Linear Systems to Random Signals

Prediction based on previous data

Filtering and Smoothing
Modulation

Graduate Institute of Communication Engineering, National Taipei University

25

Y. S. Han

Analysis and Processing of Random Signals

Continuous-Time Systems
Consider a system in which an input signal x(t) is
mapped into the output signal y(t) by the
transformation:
y(t) = T [x(t)].
The system is linear if
T [x1 (t) + x2 (t)] = T [x1 (t)] + T [x2 (t)].
Time-invariant system is given by
Input x(t) Output y(t);
Input x(t ) Output y(t ).
Graduate Institute of Communication Engineering, National Taipei University

26

Y. S. Han

Analysis and Processing of Random Signals

Impulse response of an LTI system is given by

h(t) = T [(t)].
The response of an LTI system to an input x(t) is
Z

h(s)x(t s)ds =

y(t) = h(t) x(t) =

h(t s)x(s)ds.

The transfer function of the system is given by

Z +
h(t)ej2f t dt.
H(f ) = F{h(t)} =

A system is Causal if the response at time t depends

only on past values of the input, that is, if h(t) = 0 for
t < 0.
Graduate Institute of Communication Engineering, National Taipei University

27

Y. S. Han

Analysis and Processing of Random Signals

If a random process X(t) is the input of an LTI system,

then
Z +
Z +
h(t s)X(s)ds.
h(s)X(t s)ds =
Y (t) =

If X(t) is WSS, then Y (t) is also WSS.

Graduate Institute of Communication Engineering, National Taipei University

28

Y. S. Han

Analysis and Processing of Random Signals

29

Proof: The mean of Y (t) is given by

Z

E[Y (t)]

h(s)X(t s)ds =

Z
=

h(s)E[X(t s)]ds

h( )d = mX H(0).

mX

The auto correlation function is given by

Z

E[Y (t)Y (t + )]

+ Z

h(r)X(t + r)dr

h(s)h(r)E[X(t s)X(t + r)]dsdr

+ Z

h(s)h(r)RX ( + s r)dsdr

h(s)X(t s)ds

depends only on .

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

30

Power Spectral Density of the Output

Taking the transform of RY ( ) we have
Z

SY (f )

RY ( )ej2f d

h(s)h(r)RX ( + s r)ej2f dsdrd.

Changing variables and letting u = + s r, we have

Z

SY (f )

Z
=

h(s)h(r)RX (u)ej2f (us+r) dsdrdu

j2f s

h(s)e

ds

H (f )H(f )SX (f )

|H(f )|2 SX (f ).

Z
j2f r

h(r)e

dr

RX (u)ej2f u du

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Mean and autocorrelation function of Y (t) are not

sufficient to determine probabilities of events involving
Y (t).
If the input is a Gaussian WSS process, the output is
also a Gaussian WSS process which is completely
specified by the mean and autocorrelation function of
Y (t).
It can be shown that
RY,X ( ) = RX ( ) h( );
SY,X ( ) = H(f )SX (f );

(f ) = H (f )SX (f ).
SX,Y (f ) = SY,X

Graduate Institute of Communication Engineering, National Taipei University

31

Y. S. Han

Analysis and Processing of Random Signals

Example: Find the power spectral density of the output

of a linear, time-invariant system whose input is a white
noise process.
Sol: Let X(t) be the input process with
N0
for all f .
SX (f ) =
2
The power spectral density of the output Y (t) is then
2 N0

SY (f ) = |H(f )|

One can generate WSS processes with arbitrary power

spectral density SY (f ) by passing a whitepnoise through
a system with transfer function H(f ) = SY (f ).
Graduate Institute of Communication Engineering, National Taipei University

32

Y. S. Han

Analysis and Processing of Random Signals

Example: Let Z(t) = X(t) + Y (t)

Find the output W (t) if Z(t) is input into an ideal lowpass

filter shown below:

Graduate Institute of Communication Engineering, National Taipei University

33

Y. S. Han

Analysis and Processing of Random Signals

Graduate Institute of Communication Engineering, National Taipei University

34

Y. S. Han

Analysis and Processing of Random Signals

Sol: The power spectral density of the output W (t) is

SW (f ) = |HLP (f )|2 SX (f ) + |HLP (f )|2 SY (f ) = SX (f ).
Thus, W (t) has the same power spectral density as X(t). This does
not imply that W (t) = X(t). To show that W (t) = X(t), in the mean
square sense, consider D(t) = W (t) X(t). Then
RD ( ) = RW ( ) RW X ( ) RXW ( ) + RX ( ).
The corresponding power spectral density is
SD (f )

SW (f ) SW X (f ) SXW (f ) + SX (f )

(f )SX (f ) + SX (f )
|HLP (f )|2 SX (f ) HLP (f )SX (f ) HLP

0.

Therefore RD ( ) = 0 for all , and W (t) = X(t) in the mean square

Graduate Institute of Communication Engineering, National Taipei University

35

Y. S. Han

Analysis and Processing of Random Signals

sense since
E[(W (t) X(t))2 ] = E[D2 (t)] = RD (0) = 0.

Graduate Institute of Communication Engineering, National Taipei University

36

Y. S. Han

Analysis and Processing of Random Signals

Discrete-Time Systems
Unit-sample response hn is the response of a
discrete-time LTI system to the input
(
1 n=0
n =
.
0 n 6= 0
The response of the system to Xn is given by
Yn = hn Xn =

j=

hj Xnj =

hnj Xj .

j=

Graduate Institute of Communication Engineering, National Taipei University

37

Y. S. Han

Analysis and Processing of Random Signals

Transfer function of such system is defined by

H(f ) =

hi ej2f i .

i=

If Xn is a WSS process, then Yn is also a WSS process.

The mean of Yn is given by
mY = mX

hj = mX H(0).

j=

The autocorrelation of Yn is given by

RY (k) =

X
X

hj hi RX (k + j i)

j= i=

Graduate Institute of Communication Engineering, National Taipei University

38

Y. S. Han

Analysis and Processing of Random Signals

The power spectral density of Yn is given by

SY (f ) = |H(f )|2 SX (f ).

Graduate Institute of Communication Engineering, National Taipei University

39

Y. S. Han

Analysis and Processing of Random Signals

Example: An autoregressive moving average (ARMA)

process is defined by
Yn =

q
X
i=1

i Yni +

p
X

i Wni ,

i =0

where Wn is a WSS, white noise input process. The

transfer function can be shown to be
Pp

j2f
i
=0 i e
iP
.
H(f ) =
q
j2f
i
1 + i=1 i e
The power spectral density of the ARMA process is
2
.
SY (f ) = |H(f )|2 W

Graduate Institute of Communication Engineering, National Taipei University

40

Y. S. Han

Analysis and Processing of Random Signals

7.3 Amplitude Modulation By Random Signals

The purpose of a modulator is to map the information

signal A(t) into a transmission signal X(t).
A(t) is a WSS random Process.
A(t) is a low-pass signal.

Graduate Institute of Communication Engineering, National Taipei University

41

Y. S. Han

Analysis and Processing of Random Signals

Graduate Institute of Communication Engineering, National Taipei University

42

Y. S. Han

Analysis and Processing of Random Signals

Amplitude modulation (AM) is given by

X(t) = A(t) cos(2fc t + ),
where is a random variable that is uniformly
distributed in the interval (0, 2) and and A(t) are
independent.
The autocorrelation of X(t) is given by
E[X(t + )X(t)]
= E[A(t + ) cos(2fc (t + ) + )A(t) cos(2fc t + )]


1
1
cos(2fc t) + cos(2fc (2t + ) + 2)
= RA ( )E
2
2
1
= RA ( ) cos(2fc ).
2
Graduate Institute of Communication Engineering, National Taipei University

43

Y. S. Han

Analysis and Processing of Random Signals

X(t) is also a WSS random process.

The power spectral density of X(t) is


1
RA ( ) cos(2fc )
SX (f ) = F
2
1
1
SA (f + fc ) + SA (f fc ).
=
4
4

Graduate Institute of Communication Engineering, National Taipei University

44

Y. S. Han

Analysis and Processing of Random Signals

Graduate Institute of Communication Engineering, National Taipei University

45

Y. S. Han

Analysis and Processing of Random Signals

Demodulation is performed as

Y (t) = X(t)2 cos(2fc t).

We have
SY (f )

1
1
SX (f + fc ) + SX (f fc )
2
2

Graduate Institute of Communication Engineering, National Taipei University

46

Y. S. Han

Analysis and Processing of Random Signals

1
1
{SA (f + 2fc ) + SA (f )} + {SA (f ) + SA (f 2fc )} .
2
2

The ideal lowpass filter passes SA (f ) and blocks

SA (f 2fc ).
The output of the lowpass filter has power spectral
density
SY (f ) = SA (f ).
It can be shown that Y (t) = X(t).

Graduate Institute of Communication Engineering, National Taipei University

47

Y. S. Han

Analysis and Processing of Random Signals

Quadrature Amplitude Modulation (QAM)

QAM signal is given by
X(t) = A(t) cos(2fc t + ) + B(t) sin(2fc t + ),
where A(t) and B(t) are real-valued.
We require that
RA ( ) = RB ( );
RB,A ( ) = RA,B ( ).
SA (f ) = SB (f ) is a real-valued, even function of f
It cab be shown that SB,A (f ) is a purely imaginary odd function
of f .
Graduate Institute of Communication Engineering, National Taipei University

48

Y. S. Han

Analysis and Processing of Random Signals

The autocorrelation function is given by

RX ( ) = RA ( ) cos(2fc ) + RB,A ( ) sin(2fc ).
The power spectral density is
SX (f ) =

1
1
SA (f fc ) + SA (f + fc )+ {SBA (f fc )SBA (f +fc )}.
2
2j

Bandpass random signals arise in communication systems when

wide-sense stationary white noise is filtered by bandpass filters.
Let N (t) be such process with power spectral density SN (f ).
Then we have
N (t) = Nc (t) cos(2fc t + ) Ns (t) sin(2fc t + ),

Graduate Institute of Communication Engineering, National Taipei University

49

Y. S. Han

Analysis and Processing of Random Signals

where Nc (t) and Ns (t) are jointly wide-sense stationary processes

with
SNc (f ) = SNs (f ) = {SN (f fc ) + SN (f + fc )}L
and
SNc ,Ns (f ) = j{SN (f fc ) SN (f + fc )}L ,
where the subscript L denotes the lowpass portion of the
expression in brackets.
Example: The received signal in an AM system is
Y (t) = A(t) cos(2fc t + ) + N (t),
where N (t) is a bandlimited white noise process with spectral density

N0 |f f | < W
c
2
SN (f ) =
0
elsewhere.
Graduate Institute of Communication Engineering, National Taipei University

50

Y. S. Han

Analysis and Processing of Random Signals

Find the signal to noise ration of the received signal.

Sol: We can represent the received signal by
Y (t) = {A(t) + Nc (t)} cos(2fc t + ) Ns (t) sin(2fc t + ).
The AM demodulator is used to recover A(t). After multiplication by
2 cos(2fc t + ), we have
2Y (t) cos(2fc t + )

= {A(t) + Nc (t)}{1 + cos(4fc t + 2)}

Ns (t) sin(4fc t + 2).

After lowpass filtering, the recovered signal is A(t) + Nc (t). The

power in the signal and noise components, respectively, are
2
A

SA (f ) df

Graduate Institute of Communication Engineering, National Taipei University

51

Y. S. Han

Analysis and Processing of Random Signals

2
N
c

SNc (f ) df =

= 2W N0 .

N0
N0
+
2
2

52

df

The output signal-to-noise ratio is then

2
A
SNR =
.
2W N0

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

7.4 Optimal Linear Systems

By observing {Xta , . . . , Xt , . . . , Xt+b } to obtain an

estimate Yt of the desire process Zt .
The estimate Yt is required to be linear:
Yt =

t+b
X

ht X =

=ta

a
X

h Xt .

=b

Mean square error is given by

E[e2t ] = E[(Zt Yt )2 ].
Graduate Institute of Communication Engineering, National Taipei University

53

Y. S. Han

Analysis and Processing of Random Signals

We seek to find the optimal filter, which is

characterized by the impulse response h that
minimizes the mean square error.

Graduate Institute of Communication Engineering, National Taipei University

54

Y. S. Han

Analysis and Processing of Random Signals

Example: Assume that the desired signal is corrupted by

noise:
X = Z + N .
We are interested in estimating Zt . The observation
interval is I.
1. If I = (, t) or I = (t a, t), we have a filtering
problem.
2. If I = (, ), we have a smoothing problem.
3. If I = (t a, t 1), we have a prediction problem.

Graduate Institute of Communication Engineering, National Taipei University

55

Y. S. Han

Analysis and Processing of Random Signals

The Orthogonality Condition

Optimal filter must satisfy the orthogonality
condition:
E[et X ] = E[(Zt Yt )X ] = 0

for all I

or
E[Zt X ] = E[Yt X ]

for all I.

We can find that

E[Zt X ] = E

"

a
X

=b

h Xt X

for all I

Graduate Institute of Communication Engineering, National Taipei University

56

Y. S. Han

Analysis and Processing of Random Signals

a
X

h E[Xt X ]

=b

a
X

h RX (t )

for all I.

=b

X and Zt are jointly wide-sense stationary. Therefore,

we have
RZ,X (t ) =

a
X

h RX (t ).

=b

Graduate Institute of Communication Engineering, National Taipei University

57

Y. S. Han

Analysis and Processing of Random Signals

Letting m = t , we obtain the following key equation

RZ,X (m) =

a
X

h RX (m )

b m a.

=b

We have a + b + 1 linear equations.

Graduate Institute of Communication Engineering, National Taipei University

58

Y. S. Han

Analysis and Processing of Random Signals

Continuous-time Estimation
Use Y (t) to estimate the desired signal Z(t):
Z a
Z t+b
h()X(t )d.
h(t )X()d =
Y (t) =
b

ta

It can be shown that the filter h() that minimizes the

mean square error is specified by
Z a
h()RX ( )d
b a.
RZ,X ( ) =
b

The equation can be solved numerically.

Graduate Institute of Communication Engineering, National Taipei University

59

Y. S. Han

Analysis and Processing of Random Signals

Determine the mean square error of the optimum filter

as follows. The error et and estimate Yt are orthogonal:
i X
h X
ht E[et X ] = 0.
ht X =
E[et Yt ] = E et

The mean square error is then

E[e2t ] = E[et (Zt Yt )] = E[et Zt ];

E[e2t ] = E[(Zt Yt )Zt ] = E[Zt Zt ] E[Yt Zt ]
= RZ (0) E[Zt Yt ]
a
X
h Xt ]
= RZ (0) E[Zt
=b

Graduate Institute of Communication Engineering, National Taipei University

60

Y. S. Han

Analysis and Processing of Random Signals

= RZ (0)

a
X

h RZ,X ().

=b

For continuous case we have

Z
E[e2 (t)] = RZ (0)

h()RZ,X ()d.

Graduate Institute of Communication Engineering, National Taipei University

61

Y. S. Han

Analysis and Processing of Random Signals

Theorem: Let Xt and Zt be discrete-time, zero-mean,

jointly wide-sense stationary processes, and let Yt be an
estimate for Zt of the form
a
X
h Xt .
Yt =
=b

The filter that minimize E[(Zt Yt )2 ] satisfies the equation

a
X
h RX (m )
bma
RZ,X (m) =
=b

and has mean square error given by

a
X
h RZ,X ().
E[(Zt Yt )2 ] = RZ (0)
=b

Graduate Institute of Communication Engineering, National Taipei University

62

Y. S. Han

Analysis and Processing of Random Signals

Example: Observing
X = Z + N

I = {n p, . . . , n 1, n}.

Find the set of linear equations for the optimal filter if Z

and N are independent linear processes.
Sol: We have
RZ,X (m) =

p
X

h RX (m )

m {0, 1, . . . , p}.

=0

The cross-correlation terms are

RZ,X (m) = E[Zn Xnm ] = E[Zn (Znm + Nnm )] = RZ (m).
The autocorrelation terms are given by
RX (m )

E[Xn Xnm ] = E[(Zn + Nn )(Znm + Nnm )]

Graduate Institute of Communication Engineering, National Taipei University

63

Y. S. Han

Analysis and Processing of Random Signals

RZ (m ) + RZ,N (m )
+RN,Z (m ) + RN (m )

RZ (m ) + RN (m ).

The p + 1 linear equations are then

RZ (m) =

p
X

h {RZ (m)+RN (m)} m {0, 1, . . . , p}.

=0

Example: Let Z be a first-order autoregressive process

with average power Z2 and parameter r with |r| < 1 and
2
N is a white noise with average power N
. Find the set of
equations for the optimal filter.
Sol: The autocorrelation for a first-order autoregressive
Graduate Institute of Communication Engineering, National Taipei University

64

Y. S. Han

Analysis and Processing of Random Signals

process is given by
RZ (m) = Z2 r|m|

m = 0, 1, 2, . . . .

The autocorrelation for the white noise is

2
(m).
RN (m) = N

We have the p + 1 linear equations as

Z2 r|m| =

p
X

2
h {Z2 r|m| + N
(m )}

m {0, . . . , p}.

=0

Graduate Institute of Communication Engineering, National Taipei University

65

Y. S. Han

Analysis and Processing of Random Signals

66

2
Divide both sides by Z2 and Let = N
/Z2 , we have

1
h0
1+
r
r2

rp

r
p1
h
1
+

r
1

2
p2
h2 = r2
r
r
1
+

. .
.
.
.
.
.
.
.
.
.. ..
.
.
.

rp

rp1

rp2

1 +

hp

rp

Graduate Institute of Communication Engineering, National Taipei University

Y. S. Han

Analysis and Processing of Random Signals

Prediction
We want to predict Zn in terms of Zn1 , Zn2 , . . . , Znp :
Yn =

p
X

h Zn .

=1

For this problem X = Z so we have

RZ (m) =

p
X

h RZ (m )

m {1, . . . , p}.

=1

In matrix form (Yule-Walker equations) the

Graduate Institute of Communication Engineering, National Taipei University

67

Y. S. Han

Analysis and Processing of Random Signals

equations become
2

RZ (1)

6
6 R (2)
6 Z
6
..
6
6
.
4

7
7
7
7
7
7
5

6
6
6
6
6
6
4

RZ (p)
=

RZ (0)

RZ (1)

RZ (1)
..
.

RZ (0)
..
.

..
.

RZ (p 1)

RZ (p 2)

R h.

RZ (p 1)

68

32

76
6
RZ (p 2) 7
76
76
..
76
76
.
54
RZ (0)

The mean square error becomes

p
X
h RZ ().
E[e2n ] = RZ (0)
=1

We can solve h by inverting the p p matrix RZ .

It can also be solved by Levinson algorithm.

Graduate Institute of Communication Engineering, National Taipei University

h1
h2
..
.
hp

3
7
7
7
7
7
7
5

Y. S. Han

Analysis and Processing of Random Signals

Estimation Using the Entire Realization of the

Observed Process
We want to estimate Zt by Yt :

X
h Xt .
Yt =
=

For continuous-time random process, we have

Z +
h()X(t )d.
Y (t) =

The optimum filters are then

X
h RX (m )
RZ,X (m) =

for all m;

Graduate Institute of Communication Engineering, National Taipei University

69

Y. S. Han

Analysis and Processing of Random Signals

RZ,X ( ) =

70

h()RX ( )d

for all .

Taking Fourier transform of both sides we get

SZ,X (f ) = H(f )SX (f ).
The transfer function of the optimal filter is then
SZ,X (f )
;
H(f ) =
SX (f )
h(t) = F 1 {H(f )}.
h(t) may be noncausal.
Graduate Institute of Communication Engineering, National Taipei University