Wavelet Transform Theory

Prof. Mark Fowler Department of Electrical Engineering State University of New York at Binghamton

What is a Wavelet Transform?

Decomposition of a signal into constituent parts Note that there are many ways to do this. Some are:
Fourier series: harmonic sinusoids; single integer index Fourier transform (FT): nonharmonic sinusoids; single real index Walsh decomposition: harmonic square waves; single integer index Karhunen-Loeve decomp: eigenfunctions of covariance; single real index Short-Time FT (STFT): windowed, nonharmonic sinusoids; double index provides time-frequency viewpoint Wavelet Transform: time-compacted waves; double index

Wavelet transform also provides time-frequency view

Decomposes signal in terms of duration-limited, band-pass components high-frequency components are short-duration, wide-band low-frequency components are longer-duration, narrow-band Can provide combo of good time-frequency localization and orthogonality the STFT cant do this More precisely, wavelets give time-scale viewpoint this is connected to the multi-resolution viewpoint of wavelets
2

General Characteristics of Wavelet Systems

Signal decomposition: build signals from building blocks, where the building blocks (i.e. basis functions) are doubly indexed. The components of the decomposition (i.e. the basis functions) are localized in time-frequency
ON can be achieved w/o sacrificing t-f localization

The coefficients of the decomposition can be computed efficiently (e.g., using O(N) operations).

Specific Characteristics of Wavelet Systems

Basis functions are generated from a single wavelet or scaling function by scaling and translation Exhibit multiresolution characteristics: dilating the scaling functions provides a higher resolution space that includes the original Lower resolution coefficients can be computed from higher resolution coefficients through a filter bank structure
3

Fourier Development vs. Wavelet Development

Fourier and others:
expansion functions are chosen, then properties of transform are found

Wavelets
desired properties are mathematical imposed the needed expansion functions are then derived

Why are there so many different wavelets

the basic desired property constraints dont use all the degrees of freedom remaining degrees of freedom are used to achieve secondary properties these secondary properties are usually application-specific the primary properties are generally application-nonspecific

What kinds of signals are wavelets and Fourier good for?

Wavelets are good for transients localization property allows wavelets to give efficient rep. of transients Fourier is good for periodic or stationary signals

Why are Wavelets Effective?

Provide unconditional basis for large signal class
wavelet coefficients drop-off rapidly thus, good for compression, denoising, detection/recognition goal of any expansion is have the coefficients provide more info about signal than time-domain have most of the coefficients be very small (sparse representation) FT is not sparse for transients

Accurate local description and separation of signal characteristics

Fourier puts localization info in the phase in a complicated way STFT cant give localization and orthogonality

Wavelets can be adjusted or adapted to application

remaining degrees of freedom are used to achieve goals

Computation of wavelet coefficient is well-suited to computer

no derivatives of integrals needed turns out to be a digital filter bank

Multiresolution Viewpoint

Multiresolution Approach
Stems from image processing field
consider finer and finer approximations to an image

Define a nested set of signal spaces

! V 2 V1 V0 V1 V2 ! L2

We build these spaces as follows: Let V0 be the space spanned by the integer translations of a fundamental signal (t), called the scaling function:
that is, if f(t) is in V0 then it can be represented by:

f (t ) =

a (t k )
k k

So far we can use just about any function (t), but well see that to get the nesting only certain scaling functions can be used.
7

Multiresolution Analysis (MRA) Equation

Now that we have V0 how do we make the others and ensure that they are nested? If we let V1 be the space spanned by integer translates of (2t) we get the desired property that V1 is indeed a space of functions having higher resolution. Now how do we get the nesting? We need that any function in V0 also be in V1 ; in particular we need that the scaling function (which is in V0 ) be in V1 , which the requires that

(t ) =

h ( n)
n

2 (2t n)

where the expansion coefficient is h(n) 2 This is the requirement on the scaling function to ensure nesting: it must satisfy this equation
called the multiresolution analysis (MRA) equation this is like a differential equation that the scaling function is the solution to
8

The h(n) Specify the Scaling Function

Thus, the coefficients h(n) determine the scaling function
for a given set of h(n), (t) may or may not exist may or may not be unique

Want to find conditions on h(n) for (t) to exist and be unique, and also:
to be orthogonal (because that leads to an ON wavelet expansion) to give wavelets that have desirable properties

h(n)

(t)

MRA Equation

h(n) must h(n) must satisfy satisfy conditions conditions

Whence the Wavelets?

The spaces Vj represent increasingly higher resolution spaces To go from Vj to higher resolution Vj+1 requires the addition of details
These details are the part of Vj+1 not able to be represented in Vj This can be captured through the orthogonal complement of Vj w.r.t Vj+1

Call this orthogonal complement space Wj

all functions in Wj are orthogonal to all functions in Vj That is:

< j , k (t ), j ,l (t ) >= j , k (t ) j ,l (t ) dt = 0

j , k , l Z

Consider that V0 is the lowest resolution of interest How do we characterize the space W0 ?

we need to find an ON basis for W0, say { 0,k (t )} where the basis functions arise from translating a single function (well worry about the scaling part later):

0, k (t ) = (t k )
10

Finding the Wavelets

The wavelets are the basis functions for the Wj spaces
thus, they lie in Vj+1

In particular, the function (t ) lies in the space V1 so it can be expanded as

(t ) =

h ( n)
1 n

2 (2t n), n Z

This is a fundamental result linking the scaling function and the wavelet
the h1(n) specify the wavelet, via the specified scaling function h1(n) (t)

Wavelet Equation (WE)

hh(n) must 1 (n) must 1 satisfy satisfy conditions conditions

11

Wavelet-Scaling Function Connection

There is a fundamental connection between the scaling function and its coefficients h(n) , the wavelet function and its coefficients h1(n):
h(n) (t)

MR Equation (MRE)

How are h1(n) and h(n) related? h1(n) (t)

Wavelet Equation (WE)

12

Relationship Between h1(n) and h(n)

We state here the conditions for the important special case of
finite number N of nonzero h(n) (t ) (t k )dt = (k ) ON within V0: (t ) (t k ) dt = (k ) ON between V0 and W0 :

Given the h(n) that define the desired scaling function, then the h1(n) that define the wavelet function are given by

h1 ( n) = (1) n h( N 1 n)
Much of wavelet theory addresses the origin, characteristics, and ramifications of this relationship between h1(n) and h(n)
requirements on h(n) and h1(n) to achieve ON expansions how the MRE and WE lead to a filter bank structure requirements on h(n) and h1(n) to achieve other desired properties extensions beyond the ON case

13

The Resulting Expansions

Let f(t) be in L2(R) There are three ways of interest that we can expand f(t)

We can give an limited resolution approximation to f(t) via

f j (t ) =

a 2
k k

j/2

(2 j t k )

increasing j gives a better (i.e., higher resolution) approximation

! V 2 V1 V0 V1 V2 ! L2
this is in general not the most useful expansion

14

The Resulting Expansions (cont.)

A low-resolution approximation plus its wavelet details

f (t ) =

c
k

j0 ( k ) 2

j0 / 2

(2 t k ) +
j0

d (k )2
j k j = j0

j/2

(2 j t k )

Low-Resolution Approximation

Wavelet Details

Choosing j0 sets the level of the coarse approximation

L2 = V j0 W j0 W j0 +1 W j0 + 2 !
This is most useful in practice: j0 is usually chosen according to application Also in practice, the upper value of j is chosen to be finite

15

The Resulting Expansions (cont.)

Only the wavelet details

f (t ) =

d (k )2
j k j =

j/2

(2 j t k )

Choosing j0=- eliminates the coarse approximation leaving only details

L2 = ! W 2 W1 W0 W1 W2 !
This is most similar to the true wavelet decomposition as it was originally developed This is not that useful in practice: j0 is usually chosen to be finite according to application

16

The Expansion Coefficients cj (k) and dj(k)

0

We consider here only the simple, but important, case of ON expansion

i.e., the s are ON, the s are ON, and the s are ON to the s

Then we can use standard ON expansion theory:

c j0 (k ) = f (t ), j0 , k (t ) = d j (k ) = f (t ), j , k (t ) =

f (t ) f (t )

j0 , k (t ) dt

j , k (t ) dt

We will see how to compute these without resorting to computing inner products
we will use the coefficients h1(n) and h(n) instead of the wavelet and scaling function, respectively we look at a relationship between the expansion coefficients at one level and those at the next level of resolution

17

Summary of Multiresolution View

Nested Resolution spaces:

! V 2 V1 V0 V1 V2 ! L2

Wavelet Spaces provide orthogonal complement between resolutions

L2 = V j0 W j0 W j0 +1 W j0 + 2 !
Wavelet Series Expansion of a continuous-time signal f(t):
f (t ) =

c
k

j0 ( k ) 2

j0 / 2

(2 t k ) +
j0

d (k )2
j k j = j0

j/2

(2 j t k )

MR equation (MRE) provides link between the scaling functions at successive levels of resolution:

(t ) =

h( n)
n
1 n

2 ( 2t n), n Z

Wavelet equation (WE) provides link between a resolution level and its complement

(t ) =

h (n)

2 ( 2t n), n Z
18

Summary of Multiresolution View (cont.)

There is a fundamental connection between the scaling function and its coefficients h(n) , the wavelet function and its coefficients h1(n):
h(n) (t)

MR Equation (MRE)

How are h1(n) and h(n) related? h1(n) (t)

Wavelet Equation (WE)

19

Filter Banks and DWT

20

Generalizing the MRE and WE

Here again are the MRE and the WE:
(t ) =

h( n )
n

2 ( 2t n)

(t ) =

h (n)
1 n

2 ( 2t n)

scale & translate: replace

t 2 jt k

We get:
MRE WE

(2 j t k ) =

h( m 2k ) 2 ( 2 j +1t m)

(2 j t k ) =

h1 ( m 2k ) 2 ( 2 j +1t m)

Connects Vj to Vj+1

Connects Wj to Vj+1

21

Linking Expansion Coefficients Between Scales

Start with the Generalized MRA and WE:
(2 j t k ) =

h( m 2 k )
m

2 ( 2 j +1t m)

(2 j t k ) =

h (m 2k )
1 m

2 ( 2 j +1t m)

c j ( k ) = f (t ), j , k (t )

d j (k ) = f (t ), j , k (t )

c j (k ) =

h( m 2k )
m

f (t ),2( j +1) / 2 (2 j +1t m)

d j (k ) =

h ( m 2k )
1 m

f (t ),2( j +1) / 2 (2 j +1t m)

c j +1 (m)

c j (k ) =

h(m 2k )c
m

j +1 ( m )

d j (k ) =

h (m 2k )c
1 m

j +1 ( m)

22

Convolution-Decimation Structure
New Notation For Convenience: h(n) h0(n)
c j (k ) =

h (m 2k )c
0 m

j +1 ( m)

d j (k ) =

h (m 2k )c
1 m

j +1 ( m)

y0 ( n) = c j +1 (n) h0 (n) =

Convolution

h (m n)c
0 m

y1 ( n) = c j +1 (n) h1 (n) =

j +1 ( m)

h (m n)c
1 m

j +1 ( m)

Decimation n=0 1 2 3 4 5 6 7 8 9

k=0 n = 2k =0

1 2

2 4

3 6

4 8
23

Summary of Progression to ConvolutionDecimation Structure

MRE WE

(2 j t k ) =

h( m 2k )
m

2 ( 2 j +1t m)

(2 j t k ) =

h ( m 2k )
1 m

2 (2 j +1t m)

c j (k ) =

h(m 2k )c
m

j +1 ( m )

d j (k ) =

h (m 2k )c
1 m

j +1 ( m)

cj+1(k)

HPF h1(-n)

dj(k)

LPF h0(-n)

cj(k)

24

Computing The Expansion Coefficients

The above structure can be cascaded:
given the scaling function coefficients at a specified level all the lower resolution cs and ds can be computed using the filter structure HPF h1(-n) 2 dj(k) Wj cj(k) Vj HPF h1(-n) dj-1(k) Wj-1 cj-1(k) Vj-1 HPF h1(-n) dj-2(k) Wj-2 cj-2(k) Vj-2
25

cj+1(k) Vj+1

LPF h0(-n)

LPF h0(-n)

LPF h0(-n)

Filter Bank Generation of the Spaces

Vj-1 Vj Wj-2 Wj-1 Vj+1 Wj

Vj-2

/8 HPF h1(-n)

/4 2

/2

Wj

Vj+1

LPF h0(-n)

HPF h1(-n)

Wj-1

LPF h0(-n)

HPF h1(-n)

Wj-2

LPF h0(-n)

Vj-2
26

DISCRETE FOURIER TRANSFORM

Freq

Time

27

WAVELET TRANSFORM

Freq

Time

28

WT-BASED COMPRESSION EXAMPLE

Side Info h1(1) to h1(8) s(t) h1(9) to h1(16) h1q(1) to h1q(8) h1q(9) to h1q(16)

Q1 Q2

8-Level Wavelet Transform

l8(1) to l8(8)

Q256

l8q(1) to l8q(8)

Frequency

Bits allocated to quantizers to minimize MSE Then allocations less than Bmin are set to zero Eliminates negligible cells Side info sent to describe allocations
Time
29