Scribd
Upload
213 views

ELE 774 Adaptive Signal Processing: Dr. Cenk Toker Block F3, Room: 3304/A

This document outlines the course content for ELE 774 - Adaptive Signal Processing. The course will cover topics including introduction to adaptive filtering, Wiener filters, linear prediction, stochastic gradient descent algorithms, LMS algorithms, recursive least squares algorithms, applications of adaptive filtering in areas like system identification, equalization, spectrum estimation, and interference cancellation. The course textbook is Adaptive Filter Theory by Simon Haykin. The course will analyze adaptive filters and filtering algorithms from the perspectives of convergence rate, misadjustment, tracking ability, robustness, and computational requirements.

Uploaded by

prakashrout
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
213 views

ELE 774 Adaptive Signal Processing: Dr. Cenk Toker Block F3, Room: 3304/A

This document outlines the course content for ELE 774 - Adaptive Signal Processing. The course will cover topics including introduction to adaptive filtering, Wiener filters, linear prediction, stochastic gradient descent algorithms, LMS algorithms, recursive least squares algorithms, applications of adaptive filtering in areas like system identification, equalization, spectrum estimation, and interference cancellation. The course textbook is Adaptive Filter Theory by Simon Haykin. The course will analyze adaptive filters and filtering algorithms from the perspectives of convergence rate, misadjustment, tracking ability, robustness, and computational requirements.

Uploaded by

prakashrout
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 32

ELE 774 - Adaptive Signal Processing 1

ELE 774 Adaptive


Signal Processing
Dr. Cenk Toker
Block F3, Room: 3304/A
www.ee.hacettepe.edu.tr/~toker/ELE774-Homepage.html
ELE 774 - Adaptive Signal Processing 2
Course Content (Tentative)
Textbook:
S.Haykin, Adaptive Filter Theory, 4th Ed., Prentice Hall, 2002

1. Introduction
BACKGROUND REVIEW
2. Discrete-time and random signals
3. Mathematical Tools
OPTIMAL LINEAR FILTERS
4. Wiener Filters
5. Linear Prediction
ADAPTIVE FILTERING
6. Stochastic Gradient Descent Algorithms
7. Family of LMS Algorithms
8. Method of Least Squares
9. Recursive Least Squares (RLS) Algorithm
10. Square-Root Algorithms
11. LMS and RLS Algorithms: Practical Issues
12. Kalman Filtering (?)
APPLICATIONS
13. Spectrum Estimation
14. Array Processing
ELE 774 - Adaptive Signal Processing 3
Definition of filtering
A filter
is commonly used to refer to a system that is designed to
extract information
about a
prescribed quantity of interest
from
noisy data.
ELE 774 - Adaptive Signal Processing 4
Applications
Communications; radar, sonar,
Control Systems; navigation,
Speech/Image Processing; echo and noise cancellation,
biomedical engineering
Others; seismology, financial engineering, etc.
ELE 774 - Adaptive Signal Processing 5
!!! Noise and errors are statistical in nature !!!
We will use statistical tools.
Applications
ELE 774 - Adaptive Signal Processing 6
Basic Kinds of Estimation

Filtering
(real-time operation)


Smoothing
(off-line operation)


Prediction
(real-time operation)
ELE 774 - Adaptive Signal Processing 7
Filter

Linear Non-Linear

A filter is linear if the filtered,
smoothed or predicted quantity
at the output of the filter is a
linear function of the observations
applied to the filter input.
Otherwise, it is non-linear.
Filter
u(t)

u(n)
y(t)

y(n)
Linear




Non-linear
!!! Non-linear filters may be hard to
Analyse, if not impossible !!!
or
or
ELE 774 - Adaptive Signal Processing 8
Optimum Filter
Definition: Solution of an optimization problem wrt.
a certain criterion with statistical parameters.
Nonlinear:
Maximum Likelihood (ML) sense (very difficult to implement)
Linear:
Minimum Mean Square Error (MMSE) sense
Wiener filters, (Stationary environment)
Kalman filters, (Non-stationary environment)
Etc. (Any other criterion, e.g. ZF)
ELE 774 - Adaptive Signal Processing 9
Adaptive Filters
Wiener Filter requires



Adaptive filtering can overcome these disadvantages!
Recursive algorithm
No complete a priori information required
Algorithm develops this information with increasing # of iterations.
If the environment is stationary converges to the Wiener soln.
non-stationary tracks the changes.
-a priori information of several statistics
-estimation (knowledge of the system)
is needed before filtering
-inversion of a huge matrix
-computationally inefficient!
ELE 774 - Adaptive Signal Processing 10
Analysis of Adaptive Filters
Rate of convergence (to the optimum Wiener soln.)
Misadjustment (deviation from the optimum Wiener
soln.)
Tracking (the variations in a non-stationary environment)
Robustness(to disturbances of small energy)
Computational Requirements/Cost
Numerical Properties (Numerical stability & accuracy)
ELE 774 - Adaptive Signal Processing 11
Linear Filter Structure
Structure:
FIR (Finite-duration Impulse Response)
Transversal Filter (Tapped-delay line)
Lattice
Systolic array
IIR (Infinite-duration Impulse Response)

ELE 774 - Adaptive Signal Processing 12
Transversal Filter
multiplier
adder
unit-delay
element
Convolution
Sum
ELE 774 - Adaptive Signal Processing 13
Transversal Filter
x
H
: Hermitian transpose
x
H
=(x
T
)
*
ELE 774 - Adaptive Signal Processing 14
Lattice Predictor
ELE 774 - Adaptive Signal Processing 15
Lattice Predictor
Predictor order (# of stages): M
Forward prediction error


Backward prediction error



m
: the mth reflection coefficient
Input seq. u(n) is correlated, backward prediction error
b(n) is uncorrelated
Together with
m
, b(n) approximates d(n)

(innovations
process).
ELE 774 - Adaptive Signal Processing 16
y
3
Systolic Array
Boundary cell
Internal cell
s
ELE 774 - Adaptive Signal Processing 17
Systolic Array
Represents a parallel computing network
Used for efficient pipelined operation
Matrix multiplication
Triangularisation
Back substitution (Matrix eqn. solving)
Example 3x3 matrix/3x1 vector
ELE 774 - Adaptive Signal Processing 18
IIR Filter
May have stability problems,
We will prefer FIR filters for
Adaptive filtering.
ELE 774 - Adaptive Signal Processing 19
Adaptive Filtering Algorithms
Error
Difference between
the filter output
a desired response
Mean Square Error

Weighted Error Squares
+
-
(n) : Error
y(n)
d(n)
Filter
ELE 774 - Adaptive Signal Processing 20
Adaptive Filtering Algorithms
Stochastic Gradient Approach
Cost Function depends on Mean-Square Error criterion
(!!! Stochastic, depends on second order statistics !!!)
Solution of Wiener-Hopf Equations
Results in Wiener soln. but with an iterative approach
Based on Method of Steepest Descent


Use instantaneous values instead of expectations (LMS)

function cost
of
gradient
size) (step
parameter
rate learning
vector
weight - tap of
value old
vector
weight - tap of
value updated Requires
Expectations
E{.}

signal
error
vector
input
- tap
size) (step
parameter
rate learning
vector
weight - tap of
value old
vector
weight - tap of
value updated
ELE 774 - Adaptive Signal Processing 21
Adaptive Filtering Algorithms
Least-Squares Estimation
Cost Function depends on sum of weighted error squares
Low computational complexity due to recursive operation
Three categories
Standard RLS
Relies on Matrix Inversion Lemma
Numerically unstable, high computational complexity
Square-root RLS algorithm
Based on QR-decomposition
Numerically stable
Fast RLS algorithm
Exploits certain matrix structures to reduce complexity.
ELE 774 - Adaptive Signal Processing 22
Applications
Four Classes
Identification
system identification
layered earth modeling
Inverse modeling
deconvolution
adaptive and blind
equalisation
Prediction
linear predictive coding
adaptive differential PCM
spectrum analysis
signal detection
Interference cancellation
noise canceling
echo cancellation
adaptive beamforming
ELE 774 - Adaptive Signal Processing 23
System Identification
Observing the output of a
plant(system), given the input
signal, tries to estimate the
IR of the plant.
Filter coefficient are found by
an adaptive algorithm.


ELE 774 - Adaptive Signal Processing 24
Adaptive Equalization
Removes intersymbol
interference (ISI).
Filter coefficient are found
by
an adaptive algorithm.


ELE 774 - Adaptive Signal Processing 25
Adaptive Spectrum Estimation
Parametric (AR)
model
Linear AR filter
input: white noise
output: observed
signal
aim: find the model
parameters by an
adaptive algorithm.
ELE 774 - Adaptive Signal Processing 26
Adaptive Noise Cancellation
Electrocardiography (ECG)
Acoustic noise in speech
Active noise cancellation
(headphones)
ELE 774 - Adaptive Signal Processing 27
Echo Cancellation
Coupling due to imperfect
balancing in hybrid
transformer creates an echo
in analog telephone lines.
Echo signal can be estimated
by an adaptive filter and the
subtracted out.
ELE 774 - Adaptive Signal Processing 28
Adaptive Beamforming
Multiple sensors
(antenna, microphone,
etc) used to steer the
beam to a specific
position.
Radar, sonar
Commun. systems,
Astrophysical
exploration,
Biomedical signal
processing, etc.
ELE 774 - Adaptive Signal Processing 29
Historical Notes
To understand a science it is necessary to know its history.
Auguste Comte (1798-1857)
Linear Estimation Theory
Method of least squares, Gauss, 1795
Minimum mean square error estimation, late 1930s
Discrete-time Wiener-Hopf equation, Levinson, 1947
Kalman filter, Swerling, 1958 and Kalman, 1960
Stochastic gradient algorithms, late 1950s
Stochastic approximation, Robins and Monro, 1951
LMS algorithm, Widrow and Hoff, 1959
Gradient adaptive lattice (GAL) algorithm, Griffiths, 1977-8

ELE 774 - Adaptive Signal Processing 30
Historical Notes
Recursive Least Squares Algorithm
Kalman filter, Godard Algorithm, Godard, 1974
Relationship between RLS and Kalman, Sayed and Kailath, 1994
QR decomposition based systolic array, Gentleman & Kung, 1981
Fast RLS algorithm, 1970s, Morf
Neural Networks
Logical calculus for neural networks, McCulloch and Pitts, 1943
Perceptron, Rosenblatt, 1958
Back-propagation algorithm, Rumelhart, et al., 1986
Radial basis function network, Broomhead and Lowe, 1988

ELE 774 - Adaptive Signal Processing 31
Applications
Adaptive Equalisation, 1960s
Zero-forcing equaliser, Lucky, 1965
MMSE equaliser, Gersho, 1969, Proakis&Miller, 1969
Godard Algorithm, Godard, 1974
Fractionally Spaced Equaliser (FSE), Brady, 1970
Decision Feedback Equaliser (DFE), Austin 1967, MMSE,
Monsen, 1971.
Speech Coding
Maximum Likelihood speech prediction, Saito and Itakura, 1966
Linear Predictive Coding (LPC), Atal and Hanauer 1970-1
Adaptive Lattice Predictor, Nakhoul and Cossell, 1981
ELE 774 - Adaptive Signal Processing 32
Applications
Spectrum Analysis, early 1900s
Maximum entropy method, Burg, 1967
Method of multiple windows, Thomson, 1982
Adaptive noise cancellation, started at 1965
Adaptive Beamforming
Intermediate Frequency (IF) sidelobe canceller, Howells, 1950
Control law for adaptive array antenna, Applebaum, 1966
Application of LMS, Widrow et al., 1967
Minimum Variance Distortionless Response (MVDR)
beamformer, Capon, 1969

You might also like