Statistical Digital Signal Processing

Chapter 3: Nonparametric Power

Spectrum Estimation
 Introduction
• Frequency analysis is the representation of a
signal as a superposition of sinusoidal
components.
• In practical applications, where only a finite
segment of a signal is available, we can only
compute an approximation (estimate) of the
spectrum of the adopted signal model.
• The power spectrum is the Fourier
Transform of the autocorrelation sequence.
• The quality of the estimated spectrum depends on
• How well the assumed signal model represents the data
• What values we assign to the unavailable signal
samples.
• Which spectrum estimation method we use.
• Spectrum estimation in practical problems requires
– sufficient a priori information,
– understanding of the signal generation process,
– knowledge of theoretical concepts, and
– experience.
• In this chapter we discuss the most widely used
correlation and spectrum estimation methods.
• We discuss only nonparametric techniques that
do not assume a particular functional form, but
allow the form of the estimator to be
determined entirely by the data.
• These methods are based on the discrete
Fourier transform of either the signal segment
or its autocorrelation sequence.
 Spectral Analysis of Deterministic
Signals
• Fourier series and Fourier transforms are used
for deterministic signals.
– These require knowledge of signal for -∞ to ∞.
• Rationale
– Every realization of a stochastic process is a
deterministic function.
– Deterministic functions and sequences are used in
many aspects of the study of stationary processes
– Various spectra that can be defined for deterministic
signals can be used to summarize important
features of stationary processes.
DFT based Spectrum analysis
 Effect of Sampling
• From sampling theorem

• In order to avoid distortion due to

overlapping replicas, Fs should be greater
than the maximum frequency in x.
 Effect of Windowing
• Spectrum is estimated from finite duration
segment.

• Approaches to deal with unknown data:

– Periodic extension
– Windowing
– Extrapolation
• As long as the DFT length is greater than or
equal to the data length, the DFT is periodic
sample of the DTFT.
• It is possible to obtain the DTFT from the
DFT by using the sinc function.
– Impractical
• Error made by using linear interpolation can
be made smaller by zero padding.
– Does not increase resolution.
• Windowing in the time domain is convolving
the spectrum with the DFT of the window.
• Ideal window = impulse spectrum.
• Practical windows have to problems
– Loss of resolution due to main lobe width
– Leakage due to side lobe amplitude
 Commonly used windows
• Reading assignment.
 Autocorrelation estimation of
stationary random signals
• Most widely used estimator is

N>50

|l|≤N/4

• Biased:
– For Bartlett window

• Consistency:
 Power Spectrum Estimation of
Stationary Random Signals
• The power spectral density of a zero-mean
stationary stochastic process is

• Goal: an estimate that faithfully characterizes

the power density using a finite segment
realization.
• Challenges:
– Availability of only finite amount of data
– Complexity of problem for infinite data.
 Periodogram
• The periodogram is defined as

• The window may or may not be rectangular.

• It can be expressed in terms of autocorrelation
estimate.

• The periodogram is a natural estimate of the

power spectrum.
 Performance of Periodogram
• Mean of Periodogram

– The periodogram is a biased estimator.

– In frequency domain

– For rectangular window

– Modified periodogram: N changed to N-l.

• The expected value of the periodogram is
the convolution of the power spectrum with
the Fourier transform of the window.
– The main lobe width and the sidelobe amplitude of the
window affect the resolution and variance.

If there is another sinusoidal

peak near the main lobe
width, the two will overlap
and appear as 1.
Example: AR(2) and AR(4)
Example: Two sinusoidal at 0.35π and 0.4 π
• If we use a rectangular window,
 Variance of Periodogram
• For white Gaussian noise, the variance can
be approximated by

• This shows that the variance is independent

of the record length.
• As it does not approach zero as N increases,
the periodogram is not a consistent
estimator.
Example: Periodogram of a white noise
 Covariance of Periodogram
• The covariance of the periodogram is given as

• For ω1=2k1/N and ω2=2k2/N

• Thus, values of the periodogram spaced in

frequency by integer multiples of 2π/N are
approximately uncorrelated.
• In summary
– The periodogram is a very poor estimator of the
power spectrum.
– It is a biased estimator.
– It is an inconsistent estimator.
• The main problem is its variance and its
erratic behavior.
– The variance does not decrease with increasing
N.
 Modified Periodogram
• Instead of using a rectangular window, it
uses other windows
– Hanning, Hamming, Bartlett, …

• Its mean is

– Window property affects resolution.

 Example: two sinusoids

Average Periodogram Average Modified Periodogram

• Variance

– It is not a consistent estimator.

 Blackman-Tukey Method
• Smoothing the periodogram using a moving average
filter to reduce the variance.

• Since samples of the periodogram are uncorrelated

• This can be done in the time domain via

• To avoid ripple
• The Blackman-Tukey approach has three
steps
1. Estimate the autocorrelation sequence from
the unwindowed data.
2. Window the obtained autocorrelation samples.
3. Compute the DTFT of the windowed
autocorrelation.
• The resolution is determined by the
duration of the correlation window
• For large L, its mean can be approximated as

– Asymptotically unbiased if

• Variance,
• Tradeoff between resolution and variance
– To decrease variance Ew must be small
• L<<N
– In terms of resolution, the correlation window
wa should have narrow main lobe width
• Large L
• Both can only be achieved for sufficiently
large N.
• Confidence interval
– Since they are not unbiased and consistent,
need to know whether the spectral details are
real or due to fluctuations of the PSD estimate.
– (1-α)x100 percent confidence interval
Example: three sinusoidal estimation
 Welch-Bartlett Method
• Subdivide the existing record of length N
into K smaller segments.

• The spectrum estimate is obtained by

averaging the K periodograms.

• Assuming that rx(l) is very small for |l|>L, the

signal segments are uncorrelated.
• Mean of estimation

– Asymptotically unbiased if Rx(ejw) is unbiased.

• Variance

– Assuming segments are independent

– Asymptotically consistent estimator.

• If N is fixed and N = KL,
– Increasing K to reduce the variance (or
equivalently obtain a smoother estimate) results
in a decrease in L, that is, a reduction in
resolution (or equivalently an increase in bias).
• The segments can be chosen to overlap.
• Welch showed that overlapping the segment
by 50% reduces the variance by a factor of 2.
• More overlap does not result in additional
reduction.
• Confidence interval
– (1-α)x100 percent confidence interval

– 𝓍2k2 is a chi-squared distribution with 2k

degrees of freedom.
Example: Periodogram of a white
noise
Example: three sinusoidal estimation
 Reading Assignment
• Practical computation of Blackman-Tukey
and Welch-Bartlett.
– Manolakis: pp. 226-227 and 231-232
Analysis of ocean wave data
 Reading Assignment
• Comparison between these four methods
– Hayes: pp. 424-426
 Joint Signal Analysis
• Concerned with the analysis and computation
of the correlation and associated spectral
estimation of jointly stationary RP.
• Let x(n) and y(n) be two zero-mean, jointly
stationary random process with PSD of Rx(ejw)
and Ry(ejw).
• The joint cross-power spectral density is

• rxy(l) is the cross-correlation sequence.

Estimation of Cross Power Spectrum
• Let {x(n),y(n)}0N-1 be the available data for
estimation.
• By using the periodogram, the estimator is
• It can also be written as

• Alternatively,

• This implies a magnitude-squared coherence

function is always 1.
– This is due to the fact that the frequency-domain
correlation coefficient at each frequency is estimated by
using only a single pair of observations from the two
signals.
• Welch approach
– Subdivide the existing data records into K
overlapping smaller segments of length L

• where the 50% overlap is chosen: D=L/2.

– Obtain the cross-periodogram of each segment

– Smooth the cross spectrum

• Estimation of cospectra and quadrature
spectra
– The cospectra is the real part of the cross-power
spectrum

– The quadrature spectra is the imaginary part

• Estimation of cross-amplitude and phase
spectra
– Cross-amplitude spectra is the magnitude of the
cross-power spectrum

– Cross-amplitude spectra is the magnitude of the

cross-power spectrum

– Coherency spectrum
 Estimation of Frequency Response
Functions
• When random processes x(n) and y(n) are
the input and output of some physical
system, the bivariate spectral estimation
techniques can be used to estimate the
system frequency response.
• Application
– Characterization of a channel over which signals
are transmitted.
• Since the cross-correlation of the output of
an LTI system to WSS random process is
given as

• Reading Assignment:
– Estimation of frequency response examples
• Manolakis: pp. 241-246
 Multitaper power spectrum
Estimation
• Rather than use a single rectangular data taper as in
the periodogram estimate, several data tapers are
used on the same data record to compute several
modified periodograms.
• These modified periodograms are then averaged to
produce the multitaper spectral estimate.
• The central premise of this multitaper approach is
that if the data tapers are properly designed
orthogonal functions, then the spectral estimates
would be independent of each other at every
frequency.
– Averaging would reduce variance.
• The data tapers used are selected to be
orthonormal to each other.

• The periodogram estimator based on the kth

taper window is

• The simple averaged multitaper estimator is

• The selection of the K orthonormal tapers
determines the accuracy of the estimation.
– Mean of estimation

– For good estimation

• Each taper must have low sidelobe levels.
• Each spectral estimate is uncorrelated.
• One set is obtained by using the discrete
prolate spheroidal sequences DPSS
• Harmonically related sinusoidal tapers
Example: three sinusoidal estimation
• Reading Assignment
– Cross power spectrum estimation using the
multitapering approach.
• Manolakis: pp:252-254
 Assignment 3
• Record 15 seconds of your speech in a noisy
environment (use sampling frequency of 12
kHz).
• Take a 32.5 msec frame in the middle of the
recorded speech (make sue there is a speech in
the selected frame).
• Obtain the power spectrum by using
– Periodogram,
– Blackman-Tukey approach (L=32, 64, 128) and
– Welch-Barttlet approach (K=1, 4, 8).
• Plot these and discuss what you observe.
• Submission format
– Via Email: [email protected]
– Email title: SDSP_[ID]_Assignment3
– File format: SDSP_[ID]_Assignment3.pdf
• pdf only May 23, 2022
– Submission date: 5PM, May 13 2020
– Individual submission

61