Workshop 118 on Wavelet Application in Transportation

Engineering, Sunday, January 09, 2005

Introduction to Wavelet
 A Tutorial
S
S

A1 D
1

A2 D2

A3 D3

Fengxiang Qiao, Ph.D.

Texas Southern University
 TABLE OF CONTENT

Overview
Historical Development
Time vs Frequency Domain Analysis
Fourier Analysis
Fourier vs Wavelet Transforms
Wavelet Analysis
Tools and Software
Typical Applications
Summary
References
 OVERVIEW

Wavelet
♥ A small wave
Wavelet Transforms
♥ Convert a signal into a series of wavelets
♥ Provide a way for analyzing waveforms, bounded in both
frequency and duration
♥ Allow signals to be stored more efficiently than by Fourier
transform
♥ Be able to better approximate real-world signals
♥ Well-suited for approximating data with sharp
discontinuities
“The Forest & the Trees”
♥ Notice gross features with a large "window“
♥ Notice small features with a small "window”
 DEVELOPMENT IN
HISTORY

Pre-1930
♥ Joseph Fourier (1807) with his theories of frequency
analysis
The 1930s
♥ Using scale-varying basis functions; computing the
energy of a function
1960-1980
♥ Guido Weiss and Ronald R. Coifman; Grossman and
Morlet
Post-1980
♥ Stephane Mallat; Y. Meyer; Ingrid Daubechies; wavelet
applications today
 PRE-1930
Fourier Synthesis
♥ Main branch leading to wavelets
♥ By Joseph Fourier (born in France, For any 2π periodical function f ( x ) :
1768-1830) with frequency analysis ∞
theories (1807)
f ( x ) = a0 + ∑ ( ak cos kx + bk sin kx )
From the Notion of Frequency
k =1
Analysis to Scale Analysis
♥ Analyzing f(x) by creating 1 2π
∫
a0 = f ( x )dx
mathematical structures that vary 2π 0
in scale 1 2π
 Construct a function, shift it by some ak = ∫ f ( x ) cos( kx ) dx
amount, change its scale, apply that π 0
structure in approximating a signal 1 2π
 Repeat the procedure. Take that bk = ∫ f ( x ) sin ( kx ) dx
basic structure, shift it, and scale it π 0
again. Apply it to the same signal to
get a new approximation
Haar Wavelet
♥ The first mention of wavelets
appeared in an appendix to the
thesis of A. Haar (1909)
♥ With compact support, vanishes
outside of a finite interval
♥ Not continuously differentiable
 THE 1930s

Finding by the 1930s Physicist Paul Levy

♥ Haar basis function is superior to the Fourier
basis functions for studying small complicated
details in the Brownian motion
Energy of a Function by Littlewood, Paley,
and Stein
♥ Different results were produced if the energy
was concentrated around a few points or
distributed over a larger interval

1 2π 2
Energy = ∫ f ( x ) dx
2 0
 1960-1980

Created a Simplest Elements of a Function Space,

Called Atoms
♥ By the mathematicians Guido Weiss and Ronald R. Coifman
♥ With the goal of finding the atoms for a common function
Using Wavelets for Numerical Image Processing
♥ David Marr developed an effective algorithm using a
function varying in scale in the early 1980s
Defined Wavelets in the Context of Quantum Physics
♥ By Grossman and Morlet in 1980
 POST-1980
An Additional Jump-start By Mallat
♥ In 1985, Stephane Mallat discovered some
relationships between quadrature mirror
filters, pyramid algorithms, and orthonormal
wavelet bases
Y. Meyer’s First Non-trivial Wavelets
♥ Be continuously differentiable
♥ Do not have compact support
Ingrid Daubechies’ Orthonormal Basis
Functions
♥ Based on Mallat's work
♥ Perhaps the most elegant, and the
cornerstone of wavelet applications today
 MATHEMATICAL
TRANSFORMATION
Why
♥ To obtain a further information from the signal
that is not readily available in the raw signal.
Raw Signal
♥ Normally the time-domain signal
Processed Signal
♥ A signal that has been "transformed" by any of
the available mathematical transformations
Fourier Transformation
♥ The most popular transformation
 TIME-DOMAIN SIGNAL
The Independent Variable is Time
The Dependent Variable is the Amplitude
Most of the Information is Hidden in the
Frequency Content
1 1

0.5 0.5

2 Hz

Magnitude
Magnitude

0 0
10 Hz
-0.5 -0.5

-1 -1
0 0.5 1 0 0.5 1
Time Time
1 4

0.5 2 2 Hz +
20 Hz 10 Hz +
Magnitude

Magnitude

0 0
20Hz
-0.5 -2

-1 -4
0 0.5 1 0 0.5 1
Time Time
FREQUENCY TRANSFORMS

Why Frequency Information is Needed

♥ Be able to see any information that is not
obvious in time-domain
Types of Frequency Transformation
♥ Fourier Transform, Hilbert Transform,
Short-time Fourier Transform, Wigner
Distributions, the Radon Transform, the
Wavelet Transform …
 FREQUENCY ANALYSIS

Frequency Spectrum
♥ Be basically the frequency components (spectral
components) of that signal
♥ Show what frequencies exists in the signal
Fourier Transform (FT)
♥ One way to find the frequency content
♥ Tells how much of each frequency exists in a signal

N −1 ∞
X ( k +1) = ∑x( n +1) ⋅W kn
N X ( f ) = ∫ x( t ) ⋅ e −2 jπft dt
n =0 −∞
1 N −1 ∞
x( n + 1) = ∑ X ( k + 1) ⋅WN−kn
N k =0 x( t ) = ∫ X ( f ) ⋅ e 2 jπft
df
2π 
−j 
 
−∞
wN =e N 
STATIONARITY OF SIGNAL (1)

Stationary Signal
♥ Signals with frequency content unchanged in
time
♥ All frequency components exist at all times

Non-stationary Signal
♥ Frequency changes in time
♥ One example: the “Chirp Signal”
STATIONARITY OF SIGNAL (2)
2 Hz + 10 Hz + 20Hz 3 6
0
0 Occur at all times
2 5
0
0

Magnitude

Magnitude
1 4
0
0

Stationary 0 3
0
0

-
1 2
0
0

-
2 1
0
0

-
3 0
0 0
.
2 0
.
4 0
.
6 0
.
8 1 0 5 1
0 1
5 2
0 2
5
Time Frequency (Hz)

Do not appear at all times

0.0-0.4: 2 Hz + 1 2
5
0

0.4-0.7: 10 Hz +
0
.
8

0
.
6 2
0
0
Magnitude

0.7-1.0: 20Hz

Magnitude
0
.
4

0
.
2 1
5
0

Non- 0

Stationary -
0
.
2

-
0
.
4
1
0
0

-
0
.
6 5
0

-
0
.
8

-
1 0
0 0
.
5 1 0 5 1
0 1
5 2
0 2
5
Time Frequency (Hz)
 CHIRP SIGNALS
Frequency: 2 Hz to 20 Hz
Frequency: 20 Hz to 2
Different in Time Domain Hz
1 150 1 150

0.8 0.8

0.6 0.6

0.4 0.4
100
Magnitude

100

Magnitude
Magnitude

Magnitude
0.2 0.2

0 0

-0.2 -0.2
50 50
-0.4 -0.4

-0.6 -0.6

-0.8 -0.8

-1 0 -1 0
0 0.5 1 0 5 10 15 20 25 0 0.5 1 0 5 10 15 20 25
Time Frequency (Hz) Time Frequency (Hz)

Same in Frequency Domain

At what time the frequency components occur? FT can not tell!

 NOTHING MORE, NOTHING
LESS
FT Only Gives what Frequency
Components Exist in the Signal
The Time and Frequency Information
can not be Seen at the Same Time
Time-frequency Representation of the
Signal is Needed

Most of Transportation Signals are Non-stationary.

(We need to know whether and also when an incident was happened.)

ONE EARLIER SOLUTION: SHORT-TIME FOURIER TRANSFORM (STFT)

 ShORT TIME FOURIER
TRANSFORM (STFT)
Dennis Gabor (1946) Used STFT
♥ To analyze only a small section of the signal at a time -- a
technique called Windowing the Signal.
The Segment of Signal is Assumed Stationary
A 3D transform

STFTX( ω) ( t ′, f ) = ∫ [ x( t ) • ω* ( t − t ′) ] • e − j 2 πft dt
t
ω( t ) : the window function

A function of time
and frequency
 DRAWBACKS OF STFT
Unchanged Window
Dilemma of Resolution
♥ Narrow window -> poor frequency resolution
♥ Wide window -> poor time resolution
Heisenberg Uncertainty Principle
♥ Cannot know what frequency exists at what time intervals

Via Narrow Window Via Wide Window

The two figures were from Robi Poliker, 1994

MULTIRESOLUTION ANALYSIS
(MRA)

Wavelet Transform
♥ An alternative approach to the short time Fourier
transform to overcome the resolution problem
♥ Similar to STFT: signal is multiplied with a function
Multiresolution Analysis
♥ Analyze the signal at different frequencies with
different resolutions
♥ Good time resolution and poor frequency resolution at
high frequencies
♥ Good frequency resolution and poor time resolution at
low frequencies
♥ More suitable for short duration of higher frequency;
and longer duration of lower frequency components
ADVANTAGES OF WT OVER
STFT

Width of the Window is Changed

as the Transform is Computed
for Every Spectral Components
Altered Resolutions are Placed
 PRINCIPLES OF WAELET
TRANSFORM

Split Up the Signal into a Bunch of Signals

Representing the Same Signal, but all Corresponding to
Different Frequency Bands
Only Providing What Frequency Bands Exists at What
Time Intervals
DEFINITION OF CONTINUOUS
WAVELET TRANSFORM
1 * t − τ 
CWT ( τ, s ) = Ψ ( τ, s ) =
ψ
x
ψ
x ∫ x( t ) • ψ  dt
s  s 
Translation
(The location of Scale
Wavelet the window)

Mother Wavelet
♥ Small wave
♥ Means the window function is of finite length
Mother Wavelet
♥ A prototype for generating the other window
functions
♥ All the used windows are its dilated or
compressed and shifted versions
 SCALE

Scale
♥ S>1: dilate the signal
♥ S<1: compress the signal
Low Frequency -> High Scale -> Non-
detailed Global View of Signal -> Span Entire
Signal
High Frequency -> Low Scale -> Detailed
View Last in Short Time
Only Limited Interval of Scales is Necessary
 COMPUTATION OF CWT
1 * t − τ 
CWT xψ ( τ, s ) = Ψxψ ( τ, s ) = ∫ x ( t ) • ψ  dt
s  s 
Step 1: The wavelet is placed at the beginning of the
signal, and set s=1 (the most compressed wavelet);
Step 2: The wavelet function at scale “1” is multiplied
by the signal, and integrated over all times; then
multiplied by ;
1 s
Step 3: Shift the wavelet to t= , and get the
transform value at t= and s=1;τ
τ
Step 4: Repeat the procedure until the wavelet
reaches the end of the signal;
Step 5: Scale s is increased by a sufficiently small
value, the above procedure is repeated for all s;
Step 6: Each computation for a given s fills the single
row of the time-scale plane;
Step 7: CWT is obtained if all s are calculated.
 RESOLUTION OF TIME &
FREQUENCY
Better time
resolution;
Poor
frequency
resolution
Frequency

Better
frequency
resolution;
Poor time
resolution Time
• Each box represents a equal portion
• Resolution in STFT is selected once for entire
 COMPARSION OF
TRANSFORMATIONS

From http://www.cerm.unifi.it/EUcourse2001/Gunther_lecturenotes.pdf, p.10

 MATHEMATICAL
EXPLAINATION

1 * t − τ 
CWT ( τ, s ) = Ψ ( τ, s ) =
ψ
x
ψ
x ∫ x( t ) • ψ  dt
s  s 
= ∫ X ( T ) • ψ ∗τ,s ( t ) dt

1 t − τ
ψ ∗τ,s ( t ) = ψ 
s  s 

CWT can be regarded as the inner

product of the signal with a basis
ψ (t) ∗
τ,s

function
DISCRETIZATION OF CWT

It is Necessary to Sample the Time-

Frequency (scale) Plane.
At High Scale s (Lower Frequency f ),
the Sampling Rate N can be Decreased.
The Scale Parameter s is Normally
Discretized on a Logarithmic Grid.
The most Common Value is 2.

S 2 4 8 …
N 2 = s1 s2 ⋅ N1 = f1 f 2 ⋅ N1
N 32 16 8 …
 EFFECTIVE & FAST DWT

The Discretized CWT is not a True

Discrete Transform
Discrete Wavelet Transform (DWT)
♥ Provides sufficient information both for
analysis and synthesis
♥ Reduce the computation time sufficiently
♥ Easier to implement
♥ Analyze the signal at different frequency
S
S
bands with different resolutions
♥ Decompose the signal into a coarse A 1
D1
approximation and detail information
A2 D2

A3 D3
SUBBABD CODING ALGORITHM
Halves the Time Resolution
♥ Only half number of samples resulted
Doubles the Frequency Resolution
♥ The spanned frequency band halved

0-1000 Hz
256
X[n]5 Filter 1 D1: 500-1000 Hz
12
256
S
S Filter 2 D2: 250-500 Hz
A1 128

A1 D1 128
Filter 3 D3: 125-250 Hz
A2 64
A2 D2

A3 D3 A3: 0-125 Hz
64
 DECOMPOSING NON-
STATIONARY SIGNALS (1)

fL

Signal:
0.0-0.4: 20 Hz
0.4-0.7: 10 Hz
0.7-1.0: 2 Hz
Wavelet: db4
fH
Level: 6
 DECOMPOSING NON-
STATIONARY SIGNALS (2)

fL

Signal:
0.0-0.4: 2 Hz
0.4-0.7: 10 Hz
0.7-1.0: 20Hz
Wavelet: db4
fH
Level: 6
 RECONSTRUCTION (1)

What
♥ How those components can be assembled back
into the original signal without loss of information?
♥ A Process After decomposition or analysis.
♥ Also called synthesis
How
♥ Reconstruct the signal from the wavelet
coefficients
♥ Where wavelet analysis involves filtering and
downsampling, the wavelet reconstruction process
consists of upsampling and filtering
 RECONSTRUCTION (2)

Lengthening a signal component by

inserting zeros between samples
(upsampling)
MATLAB Commands: idwt and
waverec; idwt2 and waverec2.
 WAVELET BASES

Time
domain Frequency
domain

-1 2
jω η − η
Wavelet Basis Functions: Morlet (ω0 = frequency) : π 4 e 0 e
2

2 m i m m!
Paul ( m = order ) : DOG (1 − iη) −( m+1)
π( 2m )!

DOG ( m = devivative) :
( - 1) m+1d m −η2
e ( 2
)
1 dη m
Derivative Of a Gaussian Γ m + 
M=2 is the Marr or Mexican hat wavelet  2
WAVELET FAMILY PROPERTIES

P ro p e
 WAVELET SOFTWARE
A Lot of Toolboxes and Software
have been Developed
One of the Most Popular Ones is the
MATLAB Wavelet Toolbox
http://www.mathworks.com/access/helpdesk/help/toolbox/wavelet/wavelet.html
 GUI VERSION IN MATLAB

Graphical User
Interfaces
From the
MATLAB prompt,
type wavemenu,
the Wavelet
Toolbox Main
Menu appears
OTHER SOFTWARE SOURCES

WaveLib [http://www-sim.int-evry.fr/~bourges/WaveLib.html]
EPIC [http://www.cis.upenn.edu/~eero/epic.html]
Imager Wavelet Library
[http://www.cs.ubc.ca/nest/imager/contributions/bobl/wvlt/download/]
Mathematica wavelet programs [http://timna.Mines.EDU/wavelets/]
Morletpackage [ftp://ftp.nosc.mil/pub/Shensa/]
p-wavelets [ftp://pandemonium.physics.missouri.edu/pub/wavelets/]
WaveLab [http://playfair.Stanford.EDU/~wavelab/]
Rice Wavelet Tools [http://jazz.rice.edu/RWT/]
Uvi_Wave Software [http://www.tsc.uvigo.es/~wavelets/uvi_wave.html]
WAVBOX [ftp://simplicity.stanford.edu/pub/taswell/]
Wavecompress [ftp://ftp.nosc.mil/pub/Shensa/]
WaveThresh[http://www.stats.bris.ac.uk/pub/software/wavethresh/WaveThresh
.html]
WPLIB [ftp://pascal.math.yale.edu/pub/wavelets/software/wplib/]
W-Transform Matlab Toolbox [ftp://info.mcs.anl.gov/pub/W-transform/]
XWPL [ftp://pascal.math.yale.edu/pub/wavelets/software/xwpl/]
…
 WAVELET APPLICATIONS

Typical Application Fields

♥ Astronomy, acoustics, nuclear engineering, sub-
band coding, signal and image processing,
neurophysiology, music, magnetic resonance
imaging, speech discrimination, optics, fractals,
turbulence, earthquake-prediction, radar, human
vision, and pure mathematics applications
Sample Applications
♥ Identifying pure frequencies
♥ De-noising signals
♥ Detecting discontinuities and breakdown points
♥ Detecting self-similarity
♥ Compressing images
 DE-NOISING SIGNALS

Highest
Frequencies Appear
at the Start of The
Original Signal
Approximations
Appear Less and
Less Noisy
Also Lose
Progressively More
High-frequency
Information.
In A5, About the
First 20% of the
Signal is Truncated
ANOTHER DE-NOISING
DETECTING DISCONTINUITIES
AND BREAKDOWN POINTS
The Discontinuous
Signal Consists of a
Slow Sine Wave
Abruptly Followed
by a Medium Sine
Wave.
The 1st and 2nd Level
Details (D1 and D2)
Show the
Discontinuity Most
Clearly
Things to be
Detected
♥ The site of the
change
♥ The type of change
(a rupture of the
signal, or an Discontinuity
abrupt change in Points
its first or second
derivative)
♥ The amplitude of
the change
 DETECTING SELF-SIMILARITY
Purpose
♥ How analysis by wavelets
can detect a self-similar, or
fractal, signal.
♥ The signal here is the Koch
curve -- a synthetic signal
that is built recursively
Analysis
♥ If a signal is similar to itself
at different scales, then
the "resemblance index" or
wavelet coefficients also
will be similar at different
scales.
♥ In the coefficients plot,
which shows scale on the
vertical axis, this self-
similarity generates a
characteristic pattern.
 COMPRESSING IMAGES
Fingerprints
♥ FBI maintains a large database
of fingerprints — about 30
million sets of them.
♥ The cost of storing all this data
runs to hundreds of millions of
dollars.
Results
♥ Values under the threshold are
forced to zero, achieving about
42% zeros while retaining
almost all (99.96%) the energy
of the original image.
♥ By turning to wavelets, the FBI
has achieved a 15:1
compression ratio
♥ better than the more traditional
JPEG compression
IDENTIFYING PURE
FREQUENCIES
Purpose
♥ Resolving a signal into constituent
sinusoids of different frequencies
♥ The signal is a sum of three pure sine
waves
Analysis
♥ D1 contains signal components whose
period is between 1 and 2.
♥ Zooming in on detail D1 reveals that
each "belly" is composed of 10
oscillations.
♥ D3 and D4 contain the medium sine
frequencies.
♥ There is a breakdown between
approximations A3 and A4 -> The
medium frequency been subtracted.
♥ Approximations A1 to A3 be used to
estimate the medium sine.
♥ Zooming in on A1 reveals a period of
around 20.
 SUMMARY
Historical Background Introduced
Frequency Domain Analysis Help to See any Information that is
not Obvious in Time-domain
Traditional Fourier Transform (FT) cannot Tell where a Frequency
Starts and Ends
Short-Term Fourier Transform (STFT) Uses Unchanged Windows,
cannot Solve the Resolution Problem
Continuous Wavelet Transform (CWT), Uses Wavelets as Windows
with Altered Frequency and Time Resolutions
Discrete Wavelet Transform (DWT) is more Effective and Faster
Many Wavelet Families have been Developed with Different
Properties
A lot of Software are available, which Enable more Developments
and Applications of Wavelet
Wavelet Transform can be used in many Fields including
Mathematics, Science, Engineering, Astronomy, …
This Tutorial does not Cover all the Areas of Wavelet
The theories and applications of wavelet is still in developing
 REFERENCES
Mathworks, Inc. Matlab Toolbox
http://www.mathworks.com/access/helpdesk/help/toolbox/wavelet/wavelet.html
Robi Polikar, The Wavelet Tutorial, http://users.rowan.edu/~polikar/WAVELETS/WTpart1.html
Robi Polikar, Multiresolution Wavelet Analysis of Event Related Potentials for the Detection of
Alzheimer's Disease, Iowa State University, 06/06/1995
Amara Graps, An Introduction to Wavelets, IEEE Computational Sciences and Engineering, Vol. 2, No 2,
Summer 1995, pp 50-61.
Resonance Publications, Inc. Wavelets. http://www.resonancepub.com/wavelets.htm
R. Crandall, Projects in Scientific Computation, Springer-Verlag, New York, 1994, pp. 197-198, 211-212.
Y. Meyer, Wavelets: Algorithms and Applications, Society for Industrial and Applied Mathematics,
Philadelphia, 1993, pp. 13-31, 101-105.
G. Kaiser, A Friendly Guide to Wavelets, Birkhauser, Boston, 1994, pp. 44-45.
W. Press et al., Numerical Recipes in Fortran, Cambridge University Press, New York, 1992, pp. 498-499,
584-602.
M. Vetterli and C. Herley, "Wavelets and Filter Banks: Theory and Design," IEEE Transactions on Signal
Processing, Vol. 40, 1992, pp. 2207-2232.
I. Daubechies, "Orthonormal Bases of Compactly Supported Wavelets," Comm. Pure Appl. Math., Vol 41,
1988, pp. 906-966.
V. Wickerhauser, Adapted Wavelet Analysis from Theory to Software, AK Peters, Boston, 1994, pp. 213-
214, 237, 273-274, 387.
M.A. Cody, "The Wavelet Packet Transform," Dr. Dobb's Journal, Vol 19, Apr. 1994, pp. 44-46, 50-54.
J. Bradley, C. Brislawn, and T. Hopper, "The FBI Wavelet/Scalar Quantization Standard for Gray-scale
Fingerprint Image Compression," Tech. Report LA-UR-93-1659, Los Alamos Nat'l Lab, Los Alamos, N.M.
1993.
D. Donoho, "Nonlinear Wavelet Methods for Recovery of Signals, Densities, and Spectra from Indirect and
Noisy Data," Different Perspectives on Wavelets, Proceeding of Symposia in Applied Mathematics, Vol 47,
I. Daubechies ed. Amer. Math. Soc., Providence, R.I., 1993, pp. 173-205.
B. Vidakovic and P. Muller, "Wavelets for Kids," 1994, unpublished. Part One, and Part Two.
J. Scargle et al., "The Quasi-Periodic Oscillations and Very Low Frequency Noise of Scorpius X-1 as
Transient Chaos: A Dripping Handrail?," Astrophysical Journal, Vol. 411, 1993, L91-L94.
M.V. Wickerhauser, "Acoustic Signal Compression with Wave Packets," 1989. Available by TeXing this TeX
Paper.
Questions ?