Signals and systems class, HSE, Spring 2015, A. Ossadtchi, Ph.D.

Lecture 8

Properties of the power spectral density

Introduction

As we could see from the derivation of Wiener-Khinthine theorem the Power Spectral Density (PSD) is
just another way of looking at the second order statistics of a random process. The fact that the
autocorrelation sequence has some very specific properties translates into the properties of the PSD that
we will study here.

Properties of the PSD

1. PSD is a real function of frequency

( ) = lim { ( ) ∗
To prove this consider the definition of the PSD ( )}. In brackets of the
→
expectation operator we are seeing magnitude squared of a complex valued Fourier coefficient. The
magnitude is real-valued by definition and the expected value is therefore also real-valued.

2. The integral over the − : frequency range is proportional to the variance of a zero-mean random
process and 2 is the proportionality coefficient.

Consider the inverse Fourier transform connecting the PSD and the autocorrelation sequence [ ] =
∫ ( ) . If you recall the definition of the autocorrelation sequence you will see that
[0] = + . Therefore, for a zero mean random process [0] = = ∫ ( ) =
∫ ( ) .

3. PSD of a real-valued random process is an even function of frequency, i.e. ( )= (− )

( ) = lim ∗
{ ( ) ( )} and consider its
We can again start from the definition of the PSD
→
(− ) = lim { (− ) ∗ ∗
value at – , i.e. (− )} = lim { ( ) ( )} = ( ). We used
→ →
the fact that Fourier transform coefficient of a real valued random process at frequency − is a
complex conjugate of that at . You can see it by writing out the expression for the DTFT.

Task for you: prove property 3 using the Wiener-Khintchine theorem

4. Integral of the PSD of a real-valued process x in the : frequency range is proportional to the
variance of the random process y obtained from the original random process by ideal filtering in the
same frequency range.
If you define the transfer function of the ideal filter as

1, ∀ ∋[ , ], ∋ [− ,− ]
( )=
0, otherwise
Then the PSD of the random process passed through this filter will be

( ), ∀ ∋ [ , ], ∋ [− ,− ]
( ) = | ( )| ( )=
0, otherwise

According to property 2 the variance of the random process y is

= ∫ ( ) = ∫ ( ) = ∫ ( ) , Q.E.D.

Example 1: A random process x with mean = 2 is passed through the LTI system with transfer
function ( ) = (1 − sin( ))). What will be the mean of the output random process.
According to the equation describing the transformation of the PSD by an LTI system ( )=
| ( )| ( ) . The squared periodogram of a realization of a random process is ( ) =
∗
∑ [ ] ∑ [ ] and at = 0 evaluates to (0) =
√ √
∗
∑ [ ] ∑ [ ] = . Taking the expected value of the estimate and evaluating the
√ √
limit we get that the true values of the mean of the random process , i.e. lim { (0)} = lim { }=
→ →
. Since (0) = 1.
Example 2: What is the variance of the random process whose Power Spectral Density is ( ) =
1 | | < and zero otherwise. What is the correlation coefficient of samples of the random process
2
that are 4 time samples apart?

Discrete linear systems as given by difference equation

Remember the sequence ℎ[ ] = [ ] and let’s presume that this sequence is actually a pulse
response of a system. We know that in order to compute the output of a linear system with known pulse
response we should employ the convolution operator ∗ so that [ ] = ℎ[ ] ∗ [ ] = ∑ ℎ[ ] [ − ] .
The problem is already evident the pulse response is an infinitely long sequence and therefore in order
to compute the output of such a system we would have to compute the infinite summation. Is there any
way around it? How can we actually compute the output of such a system?

Let’s explicitly write several terms of the output of such a system, starting from n=0 and assuming that
the input signal x[n] is zero for negative n.

[0] = [0],
[1] = [1] + [0],
[2] = [2] + [1] + [0] = [2] + [1]
[3] = [3] + [2] + [1] + [0] = [3] + [2]
…….
[ ]= [ ]+ [ − 1] + [ − 2] + ⋯ + [0] = [ ] + [ − 1]

So, as we can see the output for new time moment can be computed using the recursive scheme
employing the previously computed value of the output, scaling it with a and adding the current value
of the input x[n].
So, in this case, instead of specifying our system with pulse response ℎ[ ] = [ ] we can use the
following difference equation
[ ] = [ ] + [ − 1]
So, we can draw the following diagram

Let us now take the Fourier transform of both sides of this difference equation:

( )= ( )+ ( )

and then let’s express ( ) as a function of ( ) as ( ) 1 − = ( ) and thus

( )= ( ) . If you recall the relation between the FT coefficients of the input and output of
an LTI system is ( ) = ( ) ( ) then we can state that the transfer function of the LTI system
given by the difference equation is ( ) = which is exactly the result we could have gotten if
we were to apply the DTFT to the infinitely long pulse response ℎ[ ] = [ ].

Exercise: Refer to one of the previous lectures and compute the DTFT of ℎ[ ] = [ ] using the
formula for summation of the infinite geometric series.

Now, assume that the pulse response of a system is given as a sum of two exponential sequences
ℎ[ ] = [ ]+ [ ].

Then, according to the linearity of the DTFT and based on our previous result we can write the
following expression for the transfer function of such a system

1 1 1− + (1 − )
( )= + = =
1− 1− (1 − )(1 − )
2−( + ) 2−( + ) 2 −( + )
= = =
1−( + ) + 1−( + ) + −( + ) +

To obtain one before the last expression we have simply substituted = and for the last expression
we have multiplied the numerator and denominator by to operate with positive powers of z. Based
on this expression and following the pattern of the previous example we can write out the following
expression for recursive calculation of the output

[ ]=2 [ ]−( + ) [ − 1] − ( + ) [ − 1] + [ − 2],

where the numerator of the transfer function is responsible for the direct manipulation with the input
sequence and its past values and the denominator corresponds to the manipulation with the past
samples of the output sequence. Note that the coefficient corresponding to the current element of the
output sequence y[n] is identically one.

Exercise: Verify this by formally taking the DTFT of the left and right sides of this difference equation
and expressing ( ) = ( )/ ( ) .
The diagram corresponding to the second difference equation is given in the figure below.

Figure 1Diagram of a linear system with rational transfer function

Interestingly, by putting things together you should be able now to see that the negative powers of
correspond to the delay of the corresponding signal by k samples. This becomes absolutely clear if you
look back and remember that the DTFT of a sequence delayed by k samples is obtained by the DTFT of
the original sequence by multiplying it by = .

The system that uses the previous output values in computing its output is called a system with
feedback. Remember when at a party you took a mic, started talking or singing and all of sudden
everybody heard a very loud and growing in intensity (usually harmonic) sound coming out of the
loudspeakers. That happened because you were too close to the speakers and your voice amplified and
output through the loudspeakers entered the mic again and system appeared to have so called positive
feedback loop that caused the system to become unstable!

What about the stability of our last system? Remember this system in fact comprises two systems.
Clearly, if both of the subsystems are stable the entire system is stable. This corresponds to the
requirement that | | < 1 and | | < 1. We can formalize this and say that the system is stable if the
roots of the polynomial − ( + ) + lie within the unit circle. Why we are saying so? As
evident from the equation above each of the roots of the polynomial in the denominator corresponds to
an exponential sequence ℎ [ ] = [ ]. And we know that such sequence converges only when
| | < 1.
 Impulse Response

Amplitude 1.5

0.5

0
0 2 4 6 8 10 12
Samples
Pole/Zero Plot

1
0.8
0.6
0.4
Imaginary Part

0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1 -0.5 0 0.5 1
Real Part
Figure 2: Zero-pole diagram of a system with a1 =0.5 and a2 = 0.2 (bottom panel) and the corresponding pulse response (top panel)

Let us consider another example. The pulse response sequence of a system is ℎ[ ] = 0.5 cos( ).
We can then represent it as a sum of two exponential sequences using the following identity
+
cos( ) =
2
/ /
Thus, ℎ[ ] = 0.5 cos = 0.5 = (0.5 / ) + (0.5 /
) = +
We can then
. ( / / )
( )= / + / = / / =
. . ( . )( . )
. ( / / ) ( / )
. ( / / ) .
= ( / ) .

Note that the denominator has two complex conjugate roots. This property of complex conjugation of
the roots comes from the fact that the original pulse response sequence is real valued.
Zero-pole diagram, impulse response, and the magnitude of the system’s response function are shown
in the following figure.

Impulse Response
Pole/Zero Plot
2
1

1.5
0.5
Imaginary Part

Amplitude
1
0

0.5
-0.5

0
-1
-1 -0.5 0 0.5 1
Real Part 0 2 4 6 8 10 12
Samples
Magnitude Response (dB)

10

8
Magnitude (dB)

-2
0 0.2 0.4 0.6 0.8
Normalized Frequency (  rad/sample)

According to the fundamental theorem of “every non-zero, single-variable, degree n polynomial with
complex coefficients has, counted with multiplicity, exactly n roots.” Therefore, any fraction of the
∑
form ( )= can be represented as the following sum
∑

∑ ∑
( )= = =
∑ ∏ ( − ) −
( )/

= + ∗
=
− ( − )( − )
Note, that we separated the two types of components. The first sums the components corresponding to
single real poles. These poles correspond in time domain to the exponents of the following form
ℎ[ ] = [ ] . The poles that occur in conjugate pairs correspond to the exponentially damped
oscillating sequences of the form ℎ[ ] = | |cos(arg( )) [ ].