The Story of Wavelets

Historical Overview

1807 ~ 1940s: The reign of the Fourier

Transform

1940s ~ 1970s: STFT and Subband

Coding

1980s & 1990s: The Wavelet Transform

and MRA
 What is a Transform?

Transform: A mathematical operation

p
that takes a function or sequence and
maps it into another one

(e.g) Fourier transform

Frequency domain representation of the function

− jωt
f (t) = ∫ F(ω)e jωtdω
1
F(ω) = ∫ f (t)e dt
2π
 How Does FT Work ?
Recall that FT uses complex exponentials (sinusoids)as
building
g blocks. jω t
e = cos (ω t ) + j sin
i (ω t )

For each frequency of complex exponential, the

sinusoid at that frequency is compared to the signal.
signal

g
If the signal consists of that frequency,
q y the correlation is
high Æ large FT coefficients.

F (ω ) = ∫ f (t )e − jωt dt j ωt
1
f (t ) =
2π ∫ F (ω ) e dω

If the signal does not have any spectral component at a

frequency, the correlation at that frequency is low /
zero, Æ small / zero FT coefficient.
 FT At Work
W k

x1 (t ) = cos(2π ⋅ 5 ⋅ t )

x2 (t ) = cos(2π ⋅ 25 ⋅ t )

x3 (t ) = cos(2π ⋅ 50 ⋅ t )
 FT At Work
W k

x1 (t ) F X 1 (ω )

x2 (t ) F X 2 (ω )

x3 (t ) F X 3 (ω )
 FT At Work
W k

x4 (t ) = cos(2π ⋅ 5 ⋅ t )
+ cos(2π ⋅ 25 ⋅ t )
+ cos(2π ⋅ 50 ⋅ t )

x4 (t ) F X 4 (ω )
 Stationary and Non-stationary
Si
Signals
l
FT identifies all spectral components present in the signal,
however it does not provide any information regarding the
temporal (time) localization of these components. Why?
Stationary signals consist of spectral components that do not
change
g in time all spectral
p components
p exist at all times no
need to know any time information
FT works well for stationary signals
However, non-stationary signals consists of time varying
spectral components
How do we find out which spectral component appears
when?
FT only
l provides
id what h t spectral
t l components t existi t , nott
where in time they are located.
Need some other ways to determine time localization of
spectral components
 Stationary and Non-stationary Signals
Stationary signals’ spectral characteristics do not change
with time

x4 (t ) = cos(2π ⋅ 5 ⋅ t )
+ cos(2π ⋅ 25 ⋅ t )
+ cos(2π ⋅ 50 ⋅ t )

Non-stationary signals have time varying spectra

x5 (t ) = [ x1 ⊕ x2 ⊕ x3 ]
⊕ Concatenation
 St ti
Stationary vs. Non-Stationary
N St ti
X4(ω)

Perfect knowledge of what

frequencies
q exist, but no
information about where
these frequencies are
located in time
X5(ω)
 N
Non-stationary
t ti Signals
Si l
5 Hz 20 Hz 50 Hz

Perfect knowledge of what

frequencies exist, but no
information about where
these frequencies are
located in time

11
FT Sh
Shortcomings
t i

Complexl exponentials
l stretch
h out to
infinity in time
They analyze the signal globally,
globally not
locally
Hence,, FT can onlyy tell what frequencies
q
exist in the entire signal, but cannot tell,
at what time instances these frequencies
occur
In order to obtain time localization of the
spectral components, the signal need to
be analyzed locally
HOW ? 12
 Shortcomings of the FT
Sinusoids and exponentials
p
Stretch into infinity in time, no time localization
Instantaneous in frequency, perfect spectral localization
Global analysis does not allow analysis of non-stationary signals
Need a local analysis scheme for a time-frequency representation
(TFR) of nonstationary signals
Windowed F.T. or Short Time F.T. (STFT) : Segmenting the
signal into narrow time intervals, narrow enough to be considered
stationary, and then take the Fourier transform of each segment,
Gabor 1946.
Followed by other TFRs, which differed from each other by the
selection of the windowing function
 STFT
Time
Ti Frequency Signal to
F
FT Kernel
parameter parameter be analyzed
(basis function)

∫ [x(t ) ⋅W (t − t ′)]⋅ e
ω ′ − j ωt
STFTx (t , ω ) = dt
t

STFT of signal x(t): Windowing Windowing

Computed for each function function
window centered at centered at t=t’
t t’
t=t’
STFT
t’=--8
t’= t’=-
t’=-2

t’=4
t’= 4 t’=8
t’=8
 STFT att Work
W k
1 1
Windowed
sinusoid
d allows
ll
0.5 0.5
FT to be
0 0 computed only
-0.5 -0.5
through the
support of the
-1 -1
windowing
-1.5 -1.5 function
0 100 200 300 0 100 200 300

1 1

0.5 0.5

0 0

-0.5 -0.5

-1 -1

-1.5 -1.5
0 100 200 300 0 100 200 300
16
 STFTxω (t ′, ω ) = ∫ [x(t ) ⋅ W (t − t ′)]⋅ e − jωt dt
S l ti off STFT
Selection t Wi
Window
d
Two extreme cases:
W(t) infinitely long: W(t(t)=1 Î STFT turns into FT,
FT
providing excellent frequency information (good frequency
resolution), but no time information

W(t) infinitely short:W (t ) = δ (t ) Î STFT then gives the time

signal back, with a phase factor. Excellent time information
(good time resolution), but no frequency information

STFTxω (t ′, ω ) = ∫ [x(t ) ⋅ δ (t − t ′)]⋅ e − jωt dt = x(t ′) ⋅ e − jωt ′

t
Wide analysis windowÆ poor time resolution, good frequency
resolution
l
Narrow analysis windowÆgood time resolution, poor frequency
resolution
Once the window is chosen,
chosen the resolution is set for both time and
frequency.
17
 H i
Heisenberg
b P
Principle
i i l
1
Δt ⋅ Δf ≥
4π
Time resolution: How well Frequency resolution: How
two spikes in time can be well two spectral components
separated from each other in can be separated from each
the transform
t ansfo m domain othe in the transform
other t ansfo m
domain

Both time and frequency resolutions cannot be arbitrarily high!!!

Î ÎWe cannot precisely know at what time instance a frequency
component is located. We can only know what interval of
frequencies
q are p
present in which time intervals

18
STFT
ude
Amplitu

….. …..

t0 t1 tk tk+1 tn time

…..
…..
ncy
Frequen

19
 The Short Time Fourier Transform

Take FT of segmented consecutive pieces of a signal.

Each FT then provides the spectral content of that time
segment only
Spectral content for different time intervals
frequency representation
ÎTime-frequency
ÎTime

Time Signal to
Frequency FT Kernel
parameter b analyzed
be l d
parameter (basis function)

∫ [x (t ) ⋅ W (t − τ ) ]⋅ e
ω − jω t
STFT x (τ , ω ) = dt
t
STFT of signal x(t): Windowing
Computed for each function Windowing function
window centered at t=τ (Analysis window) centered at t=τ
20
(localized spectrum)
Fourier Transform
„ Fourier Transform (FT) : Convert the time-domain signals into
frequency-domain spectrum.
∞
X ( f ) = ∫ x ( t ) e − j 2π f t dt
−∞

„ Example : Linear chirp signal

Time Signal Frequency Spectrum

pp. 21
 Examples : Non
Non-stationary
stationary Signals

„ In real world, most interesting signals contain numerous non-

stationary or transitory characteristics.

pp. 22
 Joint Time-Frequency Analysis
„ Joint Time-Frequency Analysis (JTFA) : Give a good time-
frequency representation of the non-stationary signal.
Different Analysis Tools : JTFA v.s FT

pp. 24
STFT / Spectrogram : Example
WT / Scalogram : Example

pp. 26
Spectrogram v.s Scalogram

pp. 27
Comparison of Different Analysis Tools

pp. 28
„ Fourier Transform : (stationary signals)
„ Analyzes the frequency components in the time-domain signals.
„ Joint Time-Frequency Analysis : (non-stationary signals)
„ Short-Time Fourier Transform (STFT) :
„ Maps a signal into a two-dimensional function of time and frequency.
„ Precision is determined by the size of the window
window.
„ Window is always the same for all frequencies.
„ Wavelet Transform (WT) :
„ Uses a windowing technique with variable-sized regions.
„ Does not use a time-frequency region, but rather a time-scale region.
„ Higher computation complexity
Family of Wavelets
Haar
Daubechies
Biorthogonal
h l
Coiflets
Symlets
Morlet
Mexican Hat
Meyer
y
Haar

Any discussion off wavelets begins

with Haar wavelet, the first and
simplest Haar wavelet is
simplest.
discontinuous, and resembles a step
function It represents the same
function.
wavelet as Daubechies db1.
 D b hi
Daubechies

You can obtain a survey of the main properties of this family by typing
( ) from the MATLAB command line. See Daubechies Wavelets: dbN for more detail.
waveinfo('db')
Bi th
Biorthogonal
l

This family
f off wavelets exhibits the
property of linear phase, which is
needed for signal and image
reconstruction. By using two
wavelets one for decomposition (on
wavelets,
the left side) and the other for
reconstruction
o u o (on (o the right
g side)
d )
instead of the same single one,
interesting properties are derived.
You can obtain a survey of the main properties of this family by typing
waveinfo('bior') from the MATLAB command line. See Biorthogonal Wavelet Pairs: biorNr.Nd for more detail.
C ifl t
Coiflets
Built
B il b
by I.
I Daubechies
D b hi at the h request off R.
R
Coifman. You can obtain a survey of the main
properties
p p of this family
y by
y typing
yp g
waveinfo('coif') from the MATLAB command
line. See Coiflet Wavelets: coifN for more
detail.
detail
S l t
Symlets

The symlets are nearly symmetrical

wavelets proposed by Daubechies as
modifications to the db family.
family The
properties of the two wavelet families
are similar.
similar Here are the wavelet
functions psi.
You can obtain a survey of the main properties of this family by typing
waveinfo('sym') from the MATLAB command line. See Symlet Wavelets: symN for more detail.
M l t
Morlet

This wavelet has no scaling function,

f
but is explicit.
M i
Mexican Hat
H t

This wavelet has no scaling function

f
and is derived from a function that is
proportional to the second derivative
function of the Gaussian probability
density function.
function
You can obtain a survey of the main properties of this family by typing
waveinfo('mexh') from the MATLAB command line. See Mexican Hat Wavelet: mexh for more information.
Meyer

The Meyer wavelet and scaling

function are defined in the frequency
domain.
domain
You can obtain a survey of the main properties of this family by typing
waveinfo('meyer') from the MATLAB command line. See Meyer Wavelet: meyr for more detail.
 The Mother of All Oscillatory Little
Basis Functions
The kernel functions used in Wavelet transform are all {
obtained from one prototype function, by scaling and
translating the prototype function.
This prototype is called the mother wavelet {

Translation
1 t −b parameter
ψ a,b (t ) = ψ( )
a a
Scale parameter

1
Normalization factor to ensure that all
a wavelets have the same energy

∞ ∞ ∞
ψ 1,0 (t ) = ψ (t )
∫ ψ ( a ,b) (t ) dt = ∫ ψ (1,0) (t ) dt = ∫ ψ (t ) dt
2 2 2

−∞ −∞ −∞
50
 C ti
Continuous W
Wavelet
l tTTransform
f

Mother wavelet translation

∞
∗⎛ t − b ⎞
CWTx(ψ ) (a, b) = W (a, b) =
1
∫
a −∞
x(t ) ⋅ψ ⎜
⎝ a ⎠
⎟dt

Normalization factor Scaling:

CWT of x(t) at scale
Ch
Changes the
h
a and translation b
support of the
Note: low scale Î high frequency
wavelet based on
the scale
(frequency)

51
 C
Computation
t ti off CWT
mplitude

mplitude
W (1, b0 ) W (10, b0 ) W (10, bN )
Am

Am
b0 bN time b0 bN time
ude

ude
Amplitu

Amplitu
W (1, b N )
W (5, b0 ) W ( 25, b0 ) W (25, bN )

W ( 5, b N )

b0 bN time b0 bN time
∞
∗⎛ t − b ⎞
CWTx(ψ ) (a, b) = W (a, b) =
1
∫
a −∞
x (t ) ⋅ψ ⎜
⎝ a
⎟dt
⎠ 52
 WT at Work
High frequency (small scale)

Low frequency (large scale)

53
Wh Wavelet?
Why W l t?
We require
W i that
h theh waveletl functions,
f i at a {
minimum, satisfy the following:

∞
Wave…
∫ψ (t )dt = 0
−∞
∞
∫ ψ (t ) dt < ∞
2
…let

−∞

54
Th CWT as a Correlation
The C l ti
Recall that in the L2 space an inner product {
is defined as
< f (t ), g (t ) >= ∫ f (t ) g ∗ (t )dt

then W (a, b) =< x(t ),ψ a,b (t ) >

Cross correlation: Rxy (τ ) = x(t ) ⋅ y ∗ (t − τ )dt

∫
=< x(t ), y (t − τ ) >
then
W (a, b) =< x(t ),ψ a,0 (t − b) >
= Rx,ψ a ,o (b)

55
 Filtering Interpretation of Wavelet Transform

R
Recall
ll that
th t ffor a given
i system
t h[ ] y[n]=x[n]*h[n]
h[n], [ ] [ ]*h[ ]

y (t ) = x(t ) * h(t )
= ∫ x(τ )h(t − τ )dτ

Observe that W (a, b) = x(b) *ψ ∗a,0 (−b)

Interpretation:For any given scale a (frequency ~ 1/a),
the CWT W(a,b) is the output of the filter with the
∗
impulse response ψ a,0 ( −b) to the input x(b), i.e., we
have a continuum of filters, parameterized by the scale
factor a.

56
 Constant Q Filtering

A special property of the filters defined by the

mother wavelet is that they are –so so called –
constant Q filters.

Q Factor:
center frequency
b d id h
bandwidth

w (rad/s)

We observe that the filters defined by the mother wavelet

increase their bandwidth, as the scale is reduced (center
frequency is increased)

57
C
Constant
t tQ
B B B B B B
STFT

f0 2f0 3f0 4f0 5f0 6f0 f

Q=
B 2B 4B 8B B
WT
CW

f0 2f0 4f0 8f0

58
Spectrogram & Scalogram
Spectrogram is the square magnitude of the STFT,
2
PS = SPEC x( γ ) ( t , f ) = STFT x( γ ) ( t , f )
2
− j 2 π ft
∫
∗
= x ( t ) ⋅ γ ( t − t 0 ) ⋅ e dt
t

which provides the distribution of the energy of the signal in

the
h time-frequency
f plane.
l
Similarly, scalogram is the square magnitude of the CWT, and
provides the energy distribution of the signal in the time-
scale
l plane:
l
2
PW = SCAL(xψ ) ( a, b) = CWTx(ψ ) ( a, b)
2
1 ∗ ( t − b)
= ∫
a t
x ( t ) ⋅ψ
a
dt
59
 HAAR Wavelet function and
S li ffunction
Scaling ti

φ ( t ) is
1. Time limited
2.Finite Energy
3. Orthogonal
∞

∫ φ ( t )φ ( t − 2 ) dt
−∞
= 0

∞
φ (t ) is orthonormal
∫ φ ( t )φ ( t ) dt = 1
−∞
Hence
 Let V0 be the space spanned by a set of bases
{... φ (t + 1),
1) φ (t), 1) }
(t) φ (t − 1)....
⎧ −−−−−− ⎫
p ⎨φ (t − k )⎬
We denote V0 = Span
K ⎩ ⎭

Let us now sacle Haar basis function and form

new basis set.
We denote, V1 = Span{φ (2t − k )}
K

Similarly
{
V j = Span φ (2 j t − k ) }
k
Nested Spaces
φ (t ) = φ (2t ) + φ (2t − 1)
Bases of V0 can be expressed interms of bases of V1
IIn General,
G l we can write
i
.....V-1 ⊂ V0 ⊂ V1 ⊂ V2 .... ⊂ V∞
 1 1
φ (t ) = φ1,0 (t ) + φ1,1 (t ) − − − (1)
2 2
 Discrete Wavelet Transform
Dyadic sampling of the time –frequency plane {
results in a very efficient algorithm for
computing DWT:
Subband coding using multiresolution analysis z
D di sampling
Dyadic li andd multiresolution
li l i i achieved
is hi d z
through a series of filtering and up/down sampling
operations
x[n] H y[n]

y[n ] = x[n ] * h[n ] = h[n ] * x[n ]

N
= ∑ x[k ] ⋅ h[n − k ]
k =1
N
= ∑ h[k ] ⋅ x[n − k ]
k =1
 Discrete Wavelet Transform
Implementation
x[n]
~

x[n] g [ k ] = ∑ x[ n] g[ − n + 2k ]
yhigh ∑ yhigh
g [ k ] ⋅ g[ − n + 2k ]
n k

~
G G +
2 2

~ 2 ~ 2
H G 2 2 G + H

~ 2 2
H H
∑ yhigh [k ] ⋅ g[−n + 2k ]
~
ylow [k ] = ∑ x[n] h[− n + 2k ]
n k

Decomposition Reconstruction

G Half band high pass 2 Down-sampling

H filter 2 Up-sampling
Half band low pass
2-level DWTfilter
decomposition. The decomposition can be continues as long
as there are enough samples for down-
down-sampling.
 DWT
Length: 512 |H(jw)|
B: 0 ~ π

g[n] h[n]
w
Length: 256 -π/2 π/2
Length: 256 2 2 B: 0 ~ π/2
/ Hz
B: π/2 ~ π Hz
a1 |G(jw)|
d1: Level
1 DWT
Coeff. g[n] h[n]
Length: 128
Length: 128 2 2 w
B: 0 ~ π /4 Hz-π -π/2 π/2 π
B: π/4 ~ π/2 Hz a2
d2: Level
2 DWT g[n] h[n]
Coeff.
2 2 Length: 64
Length: 64
B: 0 ~ π/8 Hz
B: π/8 ~ π/4 Hz
d3: Level
L l …a
a3….Levell 3 approximation
i i
3 DWT
Coefficients
Coeff.
Multistep Decomposition and
R
Reconstruction
t ti
 W
Wavelet
l t Packet
P k t Analysis
A l i
The wavelet
Th l t packet
k t method
th d is
i a generalization
li ti off {
wavelet decomposition that offers a richer range of
possibilities for signal analysis. {

In wavelet analysis, a signal is split into an

approximation and a detail. The approximation is then
itself split into a second-level approximation and detail,
and the process is repeated. For an n-level
decomposition, there are n+1 possible ways to
d
decompose or encode
d the
th signal.
i l
In wavelet packet analysis, the details
as well as the approximations can be
split.
split