Random Signal Analysis & Power Spectral Density: Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Random Signal Analysis & Power Spectral Density: Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Random Signal
Analysis &
Power Spectral
Density
Location 1
Location 2
Location 3
Location 4
0 1 2 3 4 5 6 7 8 9 10 11 12
Time (days)
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Statistical Properties of
Random Processes
 Random signals are described by their statistical properties. For
example, the mean or expected value of {X[n]} is
∞
μ X [ n] = E ( X [ n]) = ∫ x ⋅ p ( x )dx
−∞
ª Note that the mean value itself is a sequence: μ[n], μ[n-1], μ[n-2], …
 The mean square value of {X[n]} at time n is
∞
(
E X [ n] 2
) = ∫ x2 ⋅ p ( x )dx
−∞
-500
-1000 -800 -600 -400 -200 0 200 400 600 800 1000
Lag
(
γ XX [ m, n ] = E ( X [ m ] − μ X [ m ]) ( X [ n ] − μ X [ n ])
∗
) or
= φ XX [ m, n ] − μ X [ m ] μ X [ n ]
(
γ XX [ n, n − k ] = E ( X [ n ] − μ X [ n ]) ( X [ n − k ] − μ X [ n − k ])
∗
)
and
(
γ XY [ m, n ] = E ( X [ m] − μ X [ m]) (Y [ n ] − μY [ n ])
∗
) or
= φ XY [ m, n ] − μ X [ m ] μY [ n ]
(
γ XY [ n, n − k ] = E ( X [ n ] − μ X [ n ]) (Y [ n − k ] − μY [ n − k ])
∗
)
We will soon see that these functions become useful in analyzing the random
signals in the frequency domain
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Stationarity
 In general, all of the previously calculated sequences are functions of time, i.e., the
quantities computed (mean, variance, auto-covariance, etc.) vary in time.
 If the PDF of a random process does not change in time, such a process is called
stationary, however, strict stationarity is practically impossible for all real world
signals.
 The class of signals often encountered in DSP are so-called wide-sense-stationary
(WSS) processes, for which these quantities are either independent of time, or at
least, the time origin.
 For a WSS process:
ª X[n], the mean μX[n] is independent of time, and hence it is constant, μX
ª The autocorrelation and autocovariance functions are also independent of time, m and n,
but only dependent on the difference of time-indices, m-n. Hence they are 1-D functions of
the lag between time m and time n.
φ XX [l ] = φ XX [ n ± l , l ] = E ( X [ n + l ] X ∗ [ n ])
(
γ XX [l ] = γ XX [ n ± l , l ] = E ( X [ n + l ] − μ X ) ( X [ n ] − μ X )
∗
)
= φ XX [l ] − μ X
2
( )
• The mean-square value of a WSS process is its
φ XX [ 0] = E ( X [ n ] X [ n ]) = E X [ n ]
∗ 2
autocorrelation function evaluated at zero lag!
σ X2 = γ XX [0] = φ XX [ 0] − μ X
2 • The variance of a WSS process is its
auto-covariance evaluated at zero lag!
 We can show that the following symmetry properties are also valid
φ XX [ −l ] = φ XX
∗
[l ] These functions, the autocorrelation, auto-covariance,
cross correlation and cross covariance, are all two-sided sequences
γ XX [ −l ] = γ XX
∗
[l ] with conjugate symmetry.
φ XY [ −l ] = φ XY
∗
[l ]
If the original process is purely real, then the conjugation can also
γ XY [ −l ] = γ XY
∗
[l ] be dropped.
φ XX [ 0]φYY [ 0] ≥ φ XY [l ]
2
γ XX [ 0] γ YY [ 0] ≥ γ XY [l ]
2
φ XX [ 0] ≥ φ XX [l ]
Both the autocorrelation and auto covariance functions assume
their maximum values at zero lag. This should make sense, why?
γ XX [ 0] ≥ γ XX [l ]
Digital Signal Processing, © 2006 Robi Polikar, Rowan University
Ergodicity
 First recall: A random signal x[n] is really one of an ensemble of signals that are generated by
a random process. Every time we generate x[n], we in fact get a different realization.
 If the statistical properties of these different realizations are not different, then such a random
processes is said to be stationary. In such a case, the probability distribution function of the
random process is independent of time.
ª This is a very restrictive property, not generally satisfied.
ª Leaving the mean independent of time, if we require that the correlation and covariance functions be
dependent only on the time-difference (lag), then we have wide-sense stationarity.
 Now, notice that we normally do not have access to all realizations, but rather just one. If
each realization behaves exactly as the ensemble, in other words, if we can obtain the
statistical behaviors of the ensemble from just one single realization, such processes are
called ergodic processes. Notice that only (wide sense) stationary signals can be ergodic.
 More formally, a (WS) stationary random signal is said to be ergodic, if all of its statistical
properties can be estimated from a single realization of a sufficiently large finite length.
1 M
σ X [ n ] = σ X = lim ∑ ( [ ] )
2
2 2
x n − μ
M →∞ 2 M + 1
X
n =− M
1 M
φ XX [l ] = lim
M →∞ 2 M + 1
∑ ( x [ n ] x [ n − l ])
n =− M
1 M
γ XX [l ] = lim
M →∞ 2 M + 1
∑ ( x [ n] − μ X ) ( x [ n − l ] − μ X )
n =− M
 Of course, since we also have access to a finite observation only, we would have to get rid of
the limits, and calculate the above quantities for the duration of the signal available. In such a
case, we only calculate their unbiased estimates.
1 M 1 M
μˆ X [ n ] = μˆ X = ∑ x [ n] φXX [l ] = M + 1 ∑
M + 1 n =0
ˆ ( x [ n ] x [ n − l ])
n=0
1 M 1 M
σ X [ n] = σ X = ∑ ( x [ n] − μ X ) γ XX [l ] = M + 1 ∑ ( x [ n] − μ X ) ( x [ n − l ] − μ X )
2 2 2
ˆ ˆ ˆ
M + 1 n =0 n=0
which turns out to be the power spectrum of the original random process, which
is known as the Wiener – Khintchine theorem. If SXX(ω) is normalized by 2π, the
quantity obtained is known as the power density spectrum or power spectral
density (PSD), PXX(ω), although SXX(ω) is also routinely referred to as the PSD as
well. Both functions – due to symmetry of φXX – are guaranteed to be real.
ª Furthermore, the average power of a WSS random signal is computed as
(
PX = E X [ n ]
2
) =φ XX [ 0] =
2
μ X + σ X2
ª Then,
( )
π π
1 1
PX = E X [ n ] = φ XX [ 0] = ∫π S (ω ) e ∫π S (ω ) dω
2 jω 0
dω =
2π 2π
XX XX
− −
That is, the average power is the PSD, integrated (averaged) over all frequencies.
If we wanted to find the average power in a particular frequency band, say [ω1 ω2], then
ω2 −ω1
P[ω1 ω2 ] = ∫ω P (ω ) dω + ∫ω
XX
−
PXX (ω ) d ω
1 2
Hence, PXX(ω) is not the power at frequency ω, but rather the power density, in an
infinitesimal frequency band around ω, or in other words, it is the power per unit frequency
band. To obtain the actual average power in any frequency band, we simply integrate the
power density in that interval.
Pxx = PERIODOGRAM(X) returns the PSD estimate of the signal specified by vector X in the vector Pxx. By default, the
signal X is windowed with a BOXCAR window of the same length as X. The PSD estimate is computed using an FFT
of length given by the larger of 256 and the next power of 2 greater than the length of X. Pxx is the distribution of
power per unit frequency. For real signals, PERIODOGRAM returns the one sided PSD by default; for complex
signals, it returns the two-sided PSD. Note that a one-sided PSD contains the total power of the input signal.
Pxx = PERIODOGRAM(X,WINDOW) specifies a window to be applied to X.WINDOW must be a vector of the same
length as X. If WINDOW is a window other than a boxcar (rectangular), the resulting estimate is a modified
periodogram. If WINDOW is specified as empty, the default window is used.
[Pxx,W] = PERIODOGRAM(X,WINDOW,NFFT) specifies the number of FFT points used to calculate the PSD estimate.
For real X, Pxx has length (NFFT/2+1) if NFFT is even, and (NFFT+1)/2 if NFFT is odd. Note that if NFFT is greater
than the segment the data is zero-padded. If NFFT is less than the segment, the segment is "wrapped" (using
DATAWRAP) to make the length equal to NFFT. This produces the correct FFT when NFFT < L, L being signal or
segment length. W is the vector of normalized frequencies at which the PSD is estimated. W has units of rad/sample.
[...] = PERIODOGRAM(...,'twosided') returns a two-sided PSD of a real signal X. In this case, Pxx will have length NFFT
and will be computed over the interval [0,2*Pi) if Fs is not specified and over the interval [0,Fs) if Fs is specified.
PERIODOGRAM(...) with no output arguments by default plots the PSD estimate in dB per unit frequency in the current
figure window.
ff=[150];
x=sin(2*pi*ff*t); Spectral leaks due to windowing effect of the
Power/frequency (dB/Hz)
0 truncation. Side lobes are created by the
rectangular window.
[Pxx f]=periodogram(x, [], 'twosided', 1024, fs);
-100
0 100 200 300 400 500 600 700 800 900 1000
Frequency, Hz
Power/frequency (dB/Hz)
ff=[150; 200];
xn=A*sin(2*pi*ff*t)+0.1*randn(size(t)); -30
%Equivalent to -40
% xn=sin(2*pi*150*t)+2*sin(2*pi*140*t)
% + 0.1*randn(size(t));
-50
[Pxx f]=periodogram(xn, [], 'twosided', 1024, fs);
plot(f, 10*log10(Pxx)); -60
%Gain in power is calculated with 10*log10(P)
grid
-70
title('Periodogram estimate of the PSD')
xlabel('Frequency, Hz')
ylabel('Power/frequency (dB/Hz)') -80
0 100 200 300 400 500 600 700 800 900 1000
Frequency, Hz
-10
fs=1000; L=100 -20
%t=(0:fs)/fs; -30
Power/frequency (dB/Hz)
L=input('Enter the signal length:');
-40
t=linspace(0, L/fs, L);
-50
A=[1 2];
-60
ff=[150; 160];
-70
xn=A*sin(2*pi*ff*t)+0.1*randn(size(t));
-80
%Equivalent to -90
% xn=sin(2*pi*150*t)+2*sin(2*pi*140*t) + -100
0 50 100 150 200 250 300 350 400 450 500
.1*randn(size(t)); Frequency, Hz
L=80 -10
Power/frequency (dB/Hz)
axis([0 500 -100 0]) -40
grid
-50
title('Periodogram estimate of the PSD')
-60
xlabel('Frequency, Hz')
-70
ylabel('Power/frequency (dB/Hz)')
-80
Ave_pow=fs/length(Pxx)*sum(Pxx) -90
-100
See what happens if you increase noise! 0 50 100 150 200 250 300
Frequency, Hz
350 400 450 500
Pxx = PWELCH(X) returns the Power Spectral Density (PSD) estimate, Pxx, of a discrete-time signal vector X using Welch's averaged, modified
periodogram method. By default, X is divided into eight sections with 50% overlap, each section is windowed with a Hamming window
and eight modified periodograms are computed and averaged. If the length of X is such that it cannot be divided exactly into eight sections
with 50% overlap, X will be truncated accordingly.
Pxx = PWELCH(X,WINDOW), when WINDOW is a vector, divides X into overlapping sections of length equal to the length of WINDOW, and
then windows each section with the vector specified in WINDOW. If WINDOW is an integer, X is divided into sections of length equal to
that integer value, and a Hamming window of equal length is used. If WINDOW is omitted or specified as empty, a default window is used
to obtain eight sections of X.
Pxx = PWELCH(X,WINDOW,NOVERLAP) uses NOVERLAP samples of overlap from section to section. NOVERLAP must be an integer
smaller than the WINDOW if WINDOW is an integer. NOVERLAP must be an integer smaller than the length of WINDOW if WINDOW
is a vector. If NOVERLAP is omitted or specified as empty, the default value is used to obtain a 50% overlap.
[Pxx,W] = PWELCH(X,WINDOW,NOVERLAP,NFFT) specifies the number of FFT points used to calculate the PSD estimate. If NFFT is
specified as empty, the default NFFT -the maximum of 256 or the next power of two greater than the length of each section of X- is used.
Note that if NFFT is greater than the segment the data is zero-padded. If NFFT is less than the segment, the segment is "wrapped" (using
DATAWRAP) to make the length equal to NFFT. This produces the correct FFT when NFFT < L, L being signal or segment length. W is
the vector of normalized frequencies at which the PSD is estimated. W has units of rad/sample.
[Pxx,F] = PWELCH(X,WINDOW,NOVERLAP,NFFT,Fs) returns a PSD computed as a function of physical frequency (Hz). Fs is the sampling
frequency specified in Hz. If Fs is empty, it defaults to 1 Hz. F is the vector of frequencies at which the PSD is estimated and has units of
Hz. For real signals, F spans the interval [0,Fs/2] when NFFT is even and [0,Fs/2) when NFFT is odd. For complex signals, F always spans
the interval [0,Fs).
[...] = PWELCH(...,'twosided') returns a two-sided PSD of a real signal X. In this case, Pxx will have length NFFT and will be computed over the
interval [0,2*Pi) if Fs is not specified and over the interval [0,Fs) if Fs is specified. Alternatively, the string 'twosided' can be replaced with
the string 'onesided' for a real signal X. This would result in the default behavior. The string 'twosided' or 'onesided‘ may be placed in
any position in the input argument list after NOVERLAP.
Power/frequency (dB/Hz)
A=[2 8];
ff=[150; 160]; 0
xn=A*sin(2*pi*ff*t)+5*randn(size(t));
-20
subplot(212) -30
plot(f, 10*log10(Pxx2)); grid 0 50 100 150 200 250 300 350 400 450 500
Frequency, Hz
title('Welch estimate of the PSD')
xlabel('Frequency, Hz')
ylabel('Power/frequency (dB/Hz)')