Exercises from Linear algebra, signal

processing, and wavelets. A unified

approach.
MATLAB version

Øyvind Ryan

Sep 26, 2018

Contents

1 Sound and Fourier series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Digital sound and Discrete Fourier analysis . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Discrete time filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4 Motivation for wavelets and some simple examples . . . . . . . . . . . . . . . . 95

5 The filter representation of wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

6 Constructing interesting wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7 The polyphase representation of filter bank transforms . . . . . . . . . . . . 165

8 Digital images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

9 Using tensor products to apply wavelets to images . . . . . . . . . . . . . . . . . 203

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

3
Chapter 1
Sound and Fourier series

Exercise 1.6: The Krakatoa explosion

Compute the loudness of the Krakatoa explosion on the decibel scale, assuming that
the variation in air pressure peaked at 100 000 Pa.

Solution.

Setting pref =0.00002 Pa and p=100 000 Pa in the decibel expression we get

105
     
p 100000
20 log10 = 20 log10 = 20 log10
pref 0.00002 2 × 10−5
 10 
10
= 20 log10 = 20 (10 − log10 2) ≈ 194db.
2

Exercise 1.7: The sum of two pure tones

a) Consider a sum of two pure tones, f (t) = a sin(2πν1 t) + b sin(2πν2 t). For which
values of a, b, ν1 , ν2 is f periodic? What is the period of f when it is periodic?

Solution.

sin(2πν1 t) has period 1/ν1 , while sin(2πν2 t) has period 1/ν2 . The period is not unique,
however. The first one also has period n/ν1 , and the second also n/ν2 , for any n. The
sum is periodic if there exist n1 , n2 so that n1 /ν1 = n2 /ν2 , i.e. so that there exists a
common period between the two. This common period will also be a period of f . This
amounts to that ν1 /ν2 = n1 /n2 , i.e. that ν1 /ν2 is a rational number.
b) Find two constant a and b so that the function f (t) = a sin(2π440t) + b sin(2π4400t)
resembles the right plot of Figure 1.1 in the book as closely as possible. Generate the
samples of this sound, and listen to it.

5
6 1 Sound and Fourier series

Solution.

The important thing to note here is that there are two oscillations present in the right
part of Figure 1.1 in the book: One slow oscillation with a higher amplitude, and one
faster oscillation, with a lower amplitude. We see that there are 10 periods of the
smaller oscillation within one period of the larger oscillation, so that we should be
able to reconstruct the figure by using frequencies where one is 10 times the other,
such as 440Hz and 4400Hz. Also, we see from the figure that the amplitude of the
larger oscillation is close to 1, and close to 0.3 for the smaller oscillation. A good choice
therefore seems to be a = 1, b = 0.3. The code can look this:

t = 0:(1/f_s):3;
x = sin(2*pi*440*t) + 0.3*sin(2*pi*4400*t);
x = x/max(abs(x));
playerobj = audioplayer(x, f_s);
playblocking(playerobj);

Exercise 1.8: Playing with different sample rates

If we provide another sample rate to the play functions, the sound card will assume
a different time distance between neighboring samples. Play and listen to the audio
sample file again, but with three different sample rates: 2fs , fs , and fs /2, where fs is
the sample rate returned by audioread.

Solution.

The following code can be used

playerobj = audioplayer(x, f_s);

playblocking(playerobj);

playerobj = audioplayer(x, 2*f_s);

playblocking(playerobj);

playerobj = audioplayer(x, f_s/2);

playblocking(playerobj);

The sample file castanets.wav played at double sampling rate can be found in the file
castanetsdouble.wav, and in castanetshalf.wav for half the sampling rate.

Exercise 1.9: Playing sound with added noise

Removing noise from recorded sound can be very challenging, but adding noise is simple.
There are many kinds of noise, but one kind is easily obtained by adding random
numbers to the samples of a sound. For this we can use the function rand as follows.
1 Sound and Fourier series 7

z = x + c*(2*rand(size(x))-1);

This adds noise to all channels. The function for returning random numbers returns
numbers between 0 and 1, and above we have adjusted these so that they are between −1
and 1 instead, as for other sound which can be played by the computer. c is a constant
(usually smaller than 1) that dampens the noise.
Write code which adds noise to the audio sample file, and listen to the result for
damping constants c=0.4 and c=0.1. Remember to scale the sound values after you
have added noise, since they may be outside [−1, 1].

Solution.

The following code can be used.

z = x + c*(2*rand(size(x))-1);
z = z/max(abs(z));
playerobj = audioplayer(z, f_s);
playblocking(playerobj);

With c = 0.4 the resulting sound can be found in the file castanetsnoisehigh.wav, while
with c = 0.1 it can be found in the file castanetsnoiselow.wav.

Exercise 1.10: Playing the triangle wave

Repeat what you did in Example 1.4 in the book, but now for the triangle wave of
Example 1.5 in the book. Start by generating the samples for one period, then plot five
periods, before you generate the sound over a period of three seconds and play it. Verify
that you generate the same sound as in Example 1.5 in the book.

Solution.

The triangle wave can be plotted as follows:

totransl=1-4*abs(t-T/2)/T;
figure()
plot([t t+T t+2*T t+3*T t+4*T],repmat(totransl,1,5), ’k-’)

The samples for one period are created as follows.

oneperiod=[linspace(-1,1,round(samplesperperiod/2)) ...
linspace(1,-1,round(samplesperperiod/2))];
x = zeros(1, antsec*f*length(oneperiod));

Then we repeat one period to obtain a sound with the desired length, and play it as
follows.

x=repmat(oneperiod,1,antsec*f);
playerobj=audioplayer(x, f_s);
playblocking(playerobj);
8 1 Sound and Fourier series

Exercise 1.11: Playing the notes in an octave

In music theory, an octave is a set of pure tones at frequencies f0 , ..., f11 , f12 so that the
ratio of neighboring tones are the same, and so that f12 is double the frequency of f0 ,
i.e. so that
f1 f2 f12
= = ··· = = 21/12 .
f0 f1 f11
Make a program which plays all the pure tones in an octave, and listen to it with
f0 = 440Hz.

Solution.

The following code can be used.

f_s = 44100;
num_sec = 1;
k = 2^(1/12);
f = 440;
t = linspace(0, num_sec, f_s*num_sec);
for s=0:12
playerobj = audioplayer(sin(2*pi*f*t), f_s);
playblocking(playerobj)
f = f*k;
end

Exercise 1.12: The Karplus-Strong algorithm for making

guitar-like sounds

Given initial values x0 , ..., xp , the difference equation

1
xn+p+1 − (xn+1 + xn ) = 0
2
of order p + 1 is known to create guitar like sounds. Show that all xn lie in [−1, 1] when
the initial values do, and write a function

karplus_strong(x_init, f_s)

which takes the initial values (x0 , x1 , ..., xp ) as input, and plays the resulting sound for
ten seconds with sample rate fs . Experiment with randomly generated initial values
between −1 and 1, as well as different sample rates. What can you say about the
frequencies in the resulting sound?

Solution.

The following function can be used

1 Sound and Fourier series 9

function karplus_strong(x_init, f_s)

p=size(x_init,1)-1;
num_sec = 10;
num_samples = f_s*num_sec;
z=zeros(num_samples,1);
z(1:(p+1)) = x_init;
for k=(p+2):num_samples
z(k) = 0.5*(z(k-p)+z(k-p-1));
end
playerobj = audioplayer(z, f_s);
playblocking(playerobj)
end

This function can be tested as follows

p = 100;
f_s = 44100;
x_init = 2*rand(p+1,1)-1;
karplus_strong(x_init, f_s)

Exercise 1.16: Shifting the Fourier basis vectors

Show that sin(2πnt/T + a) ∈ VN,T when |n| ≤ N , regardless of the value of a.

Solution.

Write sin(2πnt/T + a) = cos a sin(2πnt/T ) + sin a cos(2πnt/T ).

Exercise 1.17: Listening to the Fourier series of the triangle

wave

a) Plot the Fourier series of the triangle wave.

Solution.

The following code can be used.

t = linspace(0, T, 100);
x = zeros(1,length(t));
for k = 1:2:19
x = x - (8/(k*pi)^2)*cos(2*pi*k*t/T);
end
figure()
plot(t, x, ’k-’)

b) Write code so that you can listen to the Fourier series of the triangle wave. How
high must you choose N for the Fourier series to be indistinguishable from the triangle
wave itself?
10 1 Sound and Fourier series

Solution.

The following code can be used.

x = repmat(x, 1, antsec/T);
playerobj=audioplayer(x, f_s);
playblocking(playerobj);

Exercise 1.18: Riemann-integrable functions which are not

square integrable
RT
Find a function f which is Riemann-integrable on [0, T ], and so that 0
f (t)2 dt is
infinite.

Solution.

The function f (t) = 1

√
t
= t−1/2 can be used since it has the properties

Z T Z T h iT
f (t)dt = lim t−1/2 dt = lim 2t1/2
0 x→0+ x x→0+ x
1/2 1/2 1/2
= lim (2T − 2x ) = 2T
x→0+
Z T Z T
T
f (t)2 dt = lim t−1 dt = lim [ln t]x
0 x→0+ x x→0+

= ln T − lim ln x = ∞.
x→0+

Exercise 1.19: When are Fourier spaces included in each

other?

Given the two Fourier spaces VN1 ,T1 , VN2 ,T2 . Find necessary and sufficient conditions in
order for VN1 ,T1 ⊂ VN2 ,T2 .

Solution.

The space VN1 ,T1 is spanned by pure tones with frequencies 1/T1 , . . . , N1 /T1 , while
VN2 ,T2 is spanned by pure tones with frequencies 1/T2 , . . . , N2 /T2 . We must have that
the first set of frequencies is contained in the second. This is achieved if and only if
1/T1 = k/T2 for some integer k, and also N1 /T1 ≤ N2 /T2 . In other words, T2 /T1 must
be an integer, and T2 /T1 ≤ N2 /N1 .
1 Sound and Fourier series 11

Exercise 1.20: Fourier series of antisymmetric functions are

sine series

Prove the second part of Theorem 10 in the book, i.e. show that if f is antisymmetric
about 0 (i.e. f (−t) = −f (t) for all t), then an = 0, i.e. the Fourier series is actually a
sine series.

Solution.

If f is symmetric we obtain

Z T /2
2
an = f (t) cos(2πnt/T )dt
T −T /2
Z 0 Z T /2
2 2
= f (t) cos(2πnt/T )dt + f (t) cos(2πnt/T )dt
T −T /2 T 0
0
2 −T /2
Z Z
2
= f (t) cos(2πnt/T )dt − f (−t) cos(−2πnt/T )dt
T −T /2 T 0
2 0 2 0
Z Z
= f (t) cos(2πnt/T )dt − f (t) cos(2πnt/T )dt = 0.
T −T /2 T −T /2

where we again made the substitution u = −t, and used that cos is symmetric.

Exercise 1.21: More connections between

symmetric-/antisymmetric functions and sine-/cosine series

Show that
PN
a) Any cosine series a0 + n=1 an cos(2πnt/T ) is a symmetric function.

Solution.

This follows since each cos(2πnt/T ) is symmetric.

PN
b) Any sine series n=1 bn sin(2πnt/T ) is an antisymmetric function.

Solution.

This follows since each sin(2πnt/T ) is antisymmetric.

c) Any periodic function can be written as a sum of a symmetric - and an antisymmetric
function by writing f (t) = f (t)+f
2
(−t)
+ f (t)−f
2
(−t)
.
12 1 Sound and Fourier series

Solution.
f (t)+f (−t) f (t)−f (−t)
It suffices to show that g(t) = 2 is symmetric, and h(t) = 2 is antisym-
metric. We have that

f (−t) + f (t) f (t) + f (−t)

g(−t) = = = g(t)
2 2
f (−t) − f (t) f (t) − f (−t)
h(−t) = =− = −h(t).
2 2
PN
d) If fN (t) = a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T )), then

N
fN (t) + fN (−t) X
= a0 + an cos(2πnt/T )
2 n=1
N
fN (t) − fN (−t) X
= bn sin(2πnt/T ).
2 n=1

Solution.

Using the symmetry of cos and the antisymmetry of sin we obtain

PN
fN (t) + fN (−t) a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T ))
=
2 2
PN
a0 + n=1 (an cos(2πnt/T ) − bn sin(2πnt/T ))
+
2
XN
=a0 + an cos(2πnt/T )
n=1
PN
fN (t) − fN (−t) a0 + n=1 (an cos(2πnt/T ) + bn sin(2πnt/T ))
=
2 2
PN
a0 + n=1 (an cos(2πnt/T ) − bn sin(2πnt/T ))
−
2
XN
= bn sin(2πnt/T ).
n=1

Exercise 1.22: Fourier series of low-degree polynomials

Find the Fourier series coefficients of the periodic functions with period T defined by
f (t) = t, f (t) = t2 , and f (t) = t3 , on [0, T ].
1 Sound and Fourier series 13

Solution.

1
RT T
For f (t) = t we get that a0 = T 0
tdt = 2 . We also get

Z T
2
an = t cos(2πnt/T )dt
T 0
T !
 Z T
2 T T
= t sin(2πnt/T ) − sin(2πnt/T )dt =0
T 2πn 0 2πn 0
Z T
2
bn = t sin(2πnt/T )dt
T 0
 T Z T !
2 T T T
= − t cos(2πnt/T ) + cos(2πnt/T )dt = − .
T 2πn 0 2πn 0 πn

The Fourier series is thus

T X T
− sin(2πnt/T ).
2 πn
n≥1

Note that this is almost a sine series, since it has a constant term, but no other
cosine terms. If we had subtracted T /2 we would have obtained a function which is
antisymmetric, and thus a pure sine series.
RT 2
For f (t) = t2 we get that a0 = T1 0 t2 dt = T3 . We also get

Z T
2
an = t2 cos(2πnt/T )dt
T 0
 T Z T !
2 T 2 T
= t sin(2πnt/T ) − t sin(2πnt/T )dt
T 2πn 0 πn 0
T2
  
T T
= − − = 2 2
πn πn π n
Z T
2
bn = t2 sin(2πnt/T )dt
T 0
 T Z T !
2 T 2 T
= − t cos(2πnt/T ) + t cos(2πnt/T )dt
T 2πn 0 πn 0
T2
=− .
πn
Here we see that we could use the expressions for the Fourier coefficients of f (t) = t to
save some work. The Fourier series is thus

T2 X
 2
T2

T
+ cos(2πnt/T ) − sin(2πnt/T ) .
3 π 2 n2 πn
n≥1

1
RT T3
For f (t) = t3 we get that a0 = T 0
t3 dt = 4 . We also get
14 1 Sound and Fourier series
Z T
2
an = t3 cos(2πnt/T )dt
T0
 T Z T !
2 T 3 3T 2
= t sin(2πnt/T ) − t sin(2πnt/T )dt
T 2πn 0 2πn 0
T2 3T 3
  
3T
= − − =
2πn πn 2π 2 n2
Z T
2
bn = t3 sin(2πnt/T )dt
T 0
 T Z T !
2 T 3 3T 2
= − t cos(2πnt/T ) + t cos(2πnt/T )dt
T 2πn 0 2πn 0
T3 3T T 2 T3 3T 3
=− + = − + .
πn 2πn π 2 n2 πn 2π 3 n3
Also here we saved some work, by reusing the expressions for the Fourier coefficients of
f (t) = t2 . The Fourier series is thus

T3 X 3T 3 T3 3T 3
   
+ cos(2πnt/T ) + − + sin(2πnt/T ) .
4 2π 2 n2 πn 2π 3 n3
n≥1

We see that all three Fourier series converge slowly. This is connected to the fact that
none of the functions are continuous at the borders of the periods.

Exercise 1.23: Fourier series of polynomials

Write down difference equations for finding the Fourier coefficients of f (t) = tk from
those of f (t) = tk−1 , and write a program which uses this recursion to compute the
Fourier coefficients of f (t) = tk . Use the program to verify what you computed in
Exercise 1.22.

Solution.

Let us define an,k , bn,k as the Fourier coefficients of tk . When k > 0 and n > 0, integration
by parts gives us the following difference equations:
1 Sound and Fourier series 15
Z T
2
an,k = tk cos(2πnt/T )dt
T 0
 T Z T !
2 T k kT k−1
= t sin(2πnt/T ) − t sin(2πnt/T )dt
T 2πn 0 2πn 0
kT
=− bn,k−1
2πn
Z T
2
bn,k = tk sin(2πnt/T )dt
T 0
 T Z T !
2 T k kT k−1
= − t cos(2πnt/T ) + t cos(2πnt/T )dt
T 2πn 0 2πn 0
Tk kT
=− + an,k−1 .
πn 2πn
When n > 0, these can be used to express an,k , bn,k in terms of an,0 , bn,0 , for which we
Tk
clearly have an,0 = bn,0 = 0. For n = 0 we have that a0,k = k+1 for all k.
The following program computes an,k , bn,k recursively when n > 0.

function [ank,bnk] = find_fourier_coeffs(n, k, T)

if n==0
ank = T^k/(k+1); bnk = 0;
else
ank=0; bnk=0;
if k > 0
[ankprev,bnkprev] = find_fourier_coeffs(n, k-1, T);
ank = -k*T*bnkprev/(2*pi*n);
bnk = -T^k/(pi*n) + k*T*ankprev/(2*pi*n);
end
end
end

Exercise 1.24: Fourier series of a given polynomial

Use the previous exercise to find the Fourier series for f (x) = − 13 x3 + 12 x2 − 16
3
x + 1 on
the interval [0, 1]. Plot the 9th order Fourier series for this function. You should obtain
the plots from Figure 1.5 in the book.

Solution.

The following code generates the left plot of Figure 1.5 in the book.

figure()
polycoeffs = [1 -3/16 1/2 -1/3];
antterms=9;
ank = zeros(length(polycoeffs),antterms+1);
bnk = ank;
for k=0:length(polycoeffs)-1
for n=0:antterms
[ank(k+1,n+1),bnk(k+1,n+1)] = find_fourier_coeffs(n, k, 1);
end
end
t=0:0.01:1; n=(0:antterms)’;
16 1 Sound and Fourier series

vals = polycoeffs*(ank*cos(2*pi*n*t) + bnk*sin(2*pi*n*t));

polyvals = zeros(length(polycoeffs),length(t));
for k=1:length(polycoeffs)
polyvals(k,:) = t.^(k - 1);
end
plot(t, polycoeffs*polyvals,’r’);
hold on
axis([0 1 0.975 1])
plot(t,vals,’b’)

The polynomial coefficients are stored in the row vector polycoeffs at the top. The
code is written so that it will work for any polynomial: You just need to replace the
vector of polynomial coefficients. The Fourier coefficients are stored in the matrices
ank, bnk, using the function implemented in the previous exercise. Each row in those
matrices correspond to Fourier coefficients of a fixed polynomial. We also multiplied
with matrices containing cosine and sine values at all instances in time.

Exercise 1.28: Orthonormality of Complex Fourier basis

Show that the complex functions e2πint/T are orthonormal.

Solution.

For n1 6= n2 we have that

Z T Z T
1 1
he2πin1 t/T , e2πin2 t/T i = e2πin1 t/T e−2πin2 t/T dt = e2πi(n1 −n2 )t/T dt
T 0 T 0
 T
1 T
= e2πi(n1 −n2 )t/T
T 2πi(n1 − n2 ) 0
1 1
= − = 0.
2πi(n1 − n2 ) 2πi(n1 − n2 )

When n1 = n2 the integrand computes to 1, so that ke2πint/T k = 1.

Exercise 1.29: Complex Fourier series of f (t) = sin2 (2πt/T )

Compute the complex Fourier series of the function f (t) = sin2 (2πt/T ).

Solution.

We have that
1 Sound and Fourier series 17
 2
2 1 2πit/T −2πit/T
f (t) = sin (2πt/T ) = (e −e
2i
1 1 1 1
= − (e2πi2t/T − 2 + e−2πi2t/T ) = − e2πi2t/T + − e−2πi2t/T .
4 4 2 4
This gives the Fourier series of the function (with y2 = y−2 = −1/4, y0 = 1/2). This
could also have been shown by using the trigonometric identity sin2 x = 12 (1 − cos(2x))
RT
first, or by computing the integral T1 0 f (t)e−2πint/T dt (but this is rather cumbersome).

Exercise 1.30: Complex Fourier series of polynomials

Repeat Exercise 1.22, computing the complex Fourier series instead of the real Fourier
series.

Solution.

We have that

1 T −2πint/T
Z
1 1 h −2πnit/T iT T
te dt = − te =−
T 0 2πin/T T 0 2πin
1 T 2 −2πint/T
Z Z T
1 h 2 −2πnit/T iT 2
t e dt = − t e + te−2πint/T dt
T 0 2πin 0 2πin 0
T2 T2
=− + 2 2
2πin 2π n
1 T 3 −2πint/T
Z Z T
1 h 3 −2πnit/T iT 3
t e dt = − t e + t2 e−2πint/T dt
T 0 2πin 0 2πin 0
T3 T3 T3
 
3
=− + − +
2πin 2πin 2πin 2π 2 n2
T3 3T 3 3T 3
=− + 2 2 + 3 3.
2πin 4π n 4π in

Exercise 1.31: Complex Fourier series and Pascal’s triangle

In this exercise we will find a connection with certain Fourier series and the rows in
Pascal’s triangle.
a) Show that both cosn (t) and sinn (t) are in VN,2π for 1 ≤ n ≤ N .

Solution.

We have that
18 1 Sound and Fourier series
 n
1 it
cosn (t) = (e + e−it )
2
 n
1 it
sinn (t) = (e − e−it )
2i

If we multiply out here, we get a sum of terms of the form eikt , where −n ≤ k ≤ n. As
long as n ≤ N it is clear that this is in VN,2π .
b) Write down the N ’th order complex Fourier series for f1 (t) = cos t, f2 (t) = cos2 t,
and f3 (t) = cos3 t.

Solution.

We have that

1 it
cos(t) = (e + e−it )
2
1 1 1 1 −2it
cos2 (t) = (eit + e−it )2 = e2it + + e
4 4 2 4
1 it −it 3 1 3 it 3 −it 1 −3it
3
cos (t) = (e + e ) = e3it + e + e + e .
8 8 8 8 8
Therefore, for the first function the nonzero Fourier coefficients are y−1 = 1/2, y1 = 1/2,
for the second function y−2 = 1/4, y0 = 1/2, y2 = 1/4, for the third function y−3 = 1/8,
y−1 = 3/8, y1 = 3/8, y3 = 1/8.
c) In b) you should be able to see a connection between the Fourier coefficients and the
three first rows in Pascal’s triangle. Formulate and prove a general relationship between
row n in Pascal’s triangle and the Fourier coefficients of fn (t) = cosn t.

Solution.

In order to find the Fourier coefficients of cosn (t) we have to multiply out the expression
1 it −it n
2n (e + e ) . The coefficients we get after this can also be obtained from Pascal’s
triangle.

Exercise 1.32: Complex Fourier coefficients of the square wave

Compute the complex Fourier coefficients of the square wave using equation (1.15) in
the book, i.e. repeat the calculations from Example 1.13 in the book for the complex
case. Use Theorem 15 in the book to verify your result.

Solution.

We obtain that
1 Sound and Fourier series 19
Z T /2 Z T
1 −2πint/T 1
yn = e dt − e−2πint/T dt
T 0 T T /2
 T /2  T
1 T −2πint/T T −2πint/T 1
=− e e +
T 2πin 0 2πin T T /2
1
−e−πin + 1 + 1 − e−πin +


=
2πin (
1 −πin

0, if n is even;
= 1−e = .
πin 2/(πin), if n is odd.

2(1−cos(nπ)
Instead using Theorem 15 in the book together with the coefficients bn = nπ we
computed in Example 1.13 in the book, we obtain

( (
1 1 0, if n is even; 0, if n is even;
yn = (an − ibn ) = − i =
2 2 4/(nπ), if n is odd. 2/(πin), if n is odd.

when n > 0. The case n < 0 follows similarly.

Exercise 1.33: Complex Fourier coefficients of the triangle

wave

Repeat Exercise 1.32 for the triangle wave.

Solution.

Using Integration by parts we get

Z T /2
1
(4t/T − 1)e−2πint/T dt
T 0
Z T /2 !
1 h iT /2 4
= − (4t/T − 1)e−2πint/T + e−2πint/T dt
2πin 0 T 0
 
1 4 1
= −1 − (−1)n − ((−1)n − 1)
2πin T 2πin/T
Z T
1
(3 − 4t/T )e−2πint/T dt
T T /2
Z T !
1 h iT 4
= − (3 − 4t/T )e−2πint/T − e−2πint/T dt
2πin T /2 T T /2
 
1 4 1
= 1 + (−1)n + (1 − (−1)n ) .
2πin T 2πin/T

Adding the two several terms cancel, and we obtain − π22n2 (1 − (−1)n ). This means that
yn = −4/π 2 n2 for n odd, and 0 otherwise. Using Theorem 15 in the book, this is clearly
20 1 Sound and Fourier series

compatible with our previous computation of the real Fourier coefficients of the triangle
wave.

Exercise 1.34: Complex Fourier coefficients of low-degree

polynomials

Use equation (1.15) in the book to compute the complex Fourier coefficients of the
periodic functions with period T defined by, respectively, f (t) = t, f (t) = t2 , and
f (t) = t3 , on [0, T ]. Use Theorem 15 in the book to verify your calculations from
Exercise 1.22.

Solution.

For f (t) = t we get

T Z T !
Z T 
1 −2πint/T 1 T −2πint/T T −2πint/T
yn = te dt = − te + e dt
T 0 T 2πin 0 0 2πin

T T
=− = i.
2πin 2πn
T T
From Exercise 1.22 we had bn = − πn , for which Theorem 15 in the book gives yn = 2πn i
for n > 0, which coincides with the expression we obtained. The case n < 0 follows
similarly.
For f (t) = t2 we get

T !
Z T  Z T
1 1 T 2 −2πint/T T
yn = t2 e−2πint/T dt = − t e +2 te−2πint/T dt
T 0 T 2πin 0 0 2πin

T2 T2 T2 T2
=− + 2 2 = 2 2
+ i.
2πin 2π n 2π n 2πn
2 2
T T
From Exercise 1.22  we2 had a2n= π2 n2 and bn = − πn , for which Theorem 15 in the
book gives yn = 12 πT2 n2 + i πn
T
for n > 0, which also is seen to coincide with what we
obtained. The case n < 0 follows similarly.
For f (t) = t3 we get

T !
Z T  Z T
1 T 3 −2πint/T
3 −2πint/T 1 T 2 −2πint/T
yn = t e − t edt = +3 t e dt
T 0 2πin T 0 0 2πin

T3 T2 T2 T3
 3
T3

T T
=− +3 ( + i) = 3 2 2 + −3 3 3 i=
2πin 2πin 2π 2 n2 2πn 4π n 2πn 4π n
3T 3 3
T 3T 3
From Exercise 1.22 we had an = 2π 2 n2 and bn = − πn + 2π 3 n3 for which Theorem 15 in
the book gives
1 Sound and Fourier series 21

3T 3 T3 3T 3 3T 3 T3 3T 3
    
1
yn = +i − 3 3 = + − 3 3 i
2 2π 2 n2 πn 2π n 4π 2 n2 2πn 4π n
for n > 0, which also is seen to coincide with what we obtained. The case n < 0 follows
similarly.

Exercise 1.35: Complex Fourier coefficients for symmetric

and antisymmetric functions

In this exercise we will prove a version of Theorem 10 in the book for complex Fourier
coefficients.
a) If f is symmetric about 0, show that yn is real, and that y−n = yn .

Solution.

If f is symmetric about 0 we have that bn = 0. Theorem 15 in the book then gives that
yn = 12 an , which is real. The same theorem gives that y−n = 12 an = yn .
b) If f is antisymmetric about 0, show that the yn are purely imaginary, y0 = 0, and
that y−n = −yn .

Solution.

If f is antisymmetric about 0 we have that an = 0. Theorem 15 in the book then gives that
yn = − 12 bn , which is purely imaginary. The same theorem gives that y−n = 12 bn = −yn .
PN
c) Show that n=−N yn e2πint/T is symmetric when y−n = yn for all n, and rewrite it
as a cosine-series.

Solution.

When yn = y−n we can write

y−n e2πi(−n)t/T + yn e2πint/T = yn (e2πint/T + e−2πint/T ) = 2yn cos(2πnt/T )

PN
This is clearly symmetric, but then also n=−N yn e2πint/T is symmetric since it is a
sum of symmetric functions.
PN
d) Show that n=−N yn e2πint/T is antisymmetric when y0 = 0 and y−n = −yn for all
n, and rewrite it as a sine-series.

Solution.

When yn = −y−n we can write

22 1 Sound and Fourier series

y−n e2πi(−n)t/T + yn e2πint/T = yn (−e2πint/T + e2πint/T ) = 2iyn sin(2πnt/T )

PN
This is clearly antisymmetric, but then also n=−N yn e2πint/T is antisymmetric since
it is a sum of antisymmetric functions, and since y0 = 0.

Exercise 1.37: Fourier series of a delayed square wave

Define the function f with period T on [−T /2, T /2) by

(
1, if −T /4 ≤ t < T /4;
f (t) =
−1, if T /4 ≤ |t| < T /2.
f is just the square wave, delayed with d = −T /4. Compute the Fourier coefficients of f
directly, and use Property 4 in Theorem 17 in the book to verify your result.

Solution.

We obtain that

T /4
1 −T /4 −2πint/T 1 T /2 −2πint/T
Z Z Z
1 −2πint/T
yn = e dt − e dt − e dt
T −T /4 T −T /2 T T /4
 T /4  −T /4  T /2
1 −2πint/T 1 −2πint/T 1 −2πint/T
=− e + e + e
2πin −T /4 2πin −T /2 2πin T /4
1  −πin/2 
= −e + eπin/2 + eπin/2 − eπin + e−πin − e−πin/2
2πin
1 2
= (2 sin(πn/2) − sin(πn)) = sin(πn/2).
πn πn
The square wave defined in this exercise can be obtained by delaying our original square
wave with −T /4. Using Property 3 in Theorem 17 in the book with d = −T /4 on the
complex Fourier coefficients
(
0, if n is even;
yn =
2/(πin), if n is odd,
which we obtained for the square wave in Exercise 1.32, we obtain the Fourier coefficients

( (
2πin(T /4)/T 0, if n is even; 0, if n is even;
e = 2i sin(πn/2)
2/(πin), if n is odd. πin , if n is odd.
(
0, if n is even;
= 2 .
πn sin(πn/2), if n is odd.

This verifies the result.

1 Sound and Fourier series 23

Exercise 1.38: Find a function from its Fourier series

Find a function f which has the complex Fourier series

X 4
e2πint/T .
π(n + 4)
n odd

Hint.

Attempt to use one of the properties in Theorem 17 in the book on the Fourier series of
the square wave.

Solution.

Since the real Fourier series of the square wave is

X 4
sin(2πnt/T ),
πn
n≥1,n odd

Theorem 15 in the book gives us that the complex Fourier coefficients are yn = − 12 i πn
4
=
2i 1 4 2i 2i
− πn , and y−n = 2 i πn = πn for n > 0. This means that yn = − πn for all n, so that the
complex Fourier series of the square wave is
X 2i
− e2πint/T .
πn
n odd

Using Property 4 in Theorem 17 in the book we get that the e−2πi4t/T (i.e. set d = −4)
2i
times the square wave has its n’th Fourier coefficient equal to − π(n+4) . Using linearity,
this means that 2ie−2πi4t/T times the square wave has its n’th Fourier coefficient equal
4
to π(n+4) . We thus have that the function
(
2ie−2πi4t/T , 0 ≤ t < T /2
f (t) = −2πi4t/T
−2ie , T /2 ≤ t < T
has the desired Fourier series.

Exercise 1.39: Relation between the complex Fourier

coefficients of f and the cosine-coefficients of f˘

Show that the complex Fourier coefficients yn of f , and the cosine-coefficients an of f˘

are related by a2n = yn + y−n . This result is not enough to obtain the entire Fourier
series of f˘, but at least it gives us half of it.

Solution.

The 2nth complex Fourier coefficient of f˘ is

24 1 Sound and Fourier series

Z 2T
1
f˘(t)e−2πi2nt/(2T ) dt
2T 0
Z T Z 2T
1 1
= f (t)e−2πint/T dt + f (2T − t)e−2πint/T dt.
2T 0 2T T

Substituting u = 2T − t in the second integral we see that this is

Z T Z 0
1 1
= f (t)e−2πint/T dt − f (u)e2πinu/T du
2T 0 2T T
Z T Z T
1 −2πint/T 1
= f (t)e dt + f (t)e2πint/T dt
2T 0 2T 0
1 1
= yn + y−n .
2 2
Therefore we have a2n = yn − y−n .

Exercise 1.40: Symmetric filters preserve sine- and

cosine-series

An analog filter where λs (ν) = λs (−ν) is also called a symmetric filter.

a) Prove that, if the input to a symmetric filter is a Fourier series which is a cosine- or
sine series, then the output also is a cosine- or sine series.

Solution.

We have that

 
1 2πint/T −2πint/T
s(cos(2πnt/T )) = s (e +e )
2
1 1
= λs (n/T )e2πint/T + λs (−n/T )e−2πint/T
2  2 
1 2πint/T
= λs (n/T ) (e + e−2πint/T ) = λs (n/T ) cos(2πnt/T ),
2

so that s preserves cosine-series. A similar computation holds for sine-series.

Ra
b) Show that s(f ) = −a g(s)f (t − s)ds is a symmetric filter whenever g is symmetric
around 0, and zero outside [−a, a].

Solution.

We have that
1 Sound and Fourier series 25
Z a Z a
λs (ν) = g(s)e−2πiνs ds = g(s)e2πiνs ds = λs (−ν),
−a −a

so that s is symmetric.

Exercise 1.41: Circular convolution and Fourier series

Let f, g ∈ VN,T , and let h = f ~ g. Assume that f , g, and h have Fourier coefficients
xn , yn , and zn respectively. Show that zn = xn yn .

Solution.

1
RT
The circular convolution of f and g was defined as h(t) = T 0
f (u)g(t − u)du. We have
that

Z T Z T Z T
1 1
zn = h(t)e−2πint/T dt = f (u)g(t − u)e−2πint/T dudt
T 0 T2 0 0
Z TZ T
1
= 2 f (u)e−2πinu/T g(t − u)e−2πin(t−u))/T dudt
T 0 0
1 T 1 T
Z Z
= f (u)e−2πinu/T du g(s)e−2πins/T ds
T 0 T 0
= xn yn .

Exercise 1.42: Approximation in norm with continuous

functions

Assume that f is a function defined on [a, b] with only a finite number of discontinuities,
and with all one-sided limits existing. Show that there exists a continuous function g so
that kf − gk < .

Solution.

Denote the discontinuities by {ai }N i=1 . We can easily construct a continuous function
g which agrees with f outside intervals of the form [ai − , ai + ] (simply connect
(ai − , f (ai − )) and (ai + , f (ai + )) with a linear segment). By choosing  small
enough it is clear that we can make kg − f k as small as we like.

Exercise 1.43: The Dirichlet kernel

Recall that the Dirichlet kernel was defined as

26 1 Sound and Fourier series

N
X
DN (t) = e2πint/T
n=−N

sin(π(2N +1)t/T )
a) Show that DN (t) = sin(πt/T ) .

Solution.

Using the formula for the sum of a geometric series we obtain

N
2N +1
X 1 − e2πint/T
e2πint/T = e−2πiN t/T
n=−N
1 − e2πint/T
eπi(2N +1)t/T − e−πi(2N +1)t/T
= e2πiN t/T +πint/T −πi(2N +1)t/T
eπint/T − e−πint/T
sin(π(2N + 1)t/T )
= .
sin(πt/T )

b) Prove that DN (t) satisfies one of the properties of a summability kernel, but fails on
the other ones.

Solution.

We have that

N
!
Z T Z T
1 X
2πint/T 1
e dt = 1dt = 1,
T 0 T 0
n=−N

so the first property of a summability kernel is satisfied. The second property is not
satisfied, however, since the integral over [0, T /2] goes to infinity as N → ∞. To see
why, note that | sin(x)| ≤ |x| for all x. This means that it is enough to prove that
R T /2 | sin(π(2N +1)t/T )|
0 t dt → ∞ as well. Setting u = π(2N + 1)t/T , we can write this as

π(N +1/2) N −1 Z (k+1)π Z π N −1

| sin(u)| | sin(u)|
Z X X 1
du = du = | sin(u)|du .
0 u kπ u 0 (k + 1)π
k=0 k=0

This sum clearly diverges. That third property is not satisfied also follows by using
Rb
| sin(x)| ≤ |x| (simply note that a | sin(π(2N + 1)t/T )|dt does not converge to zero).

Exercise 1.44: The Fejer summability kernel

The Fejer kernel is defined as

1 Sound and Fourier series 27

N  
X |n|
FN (t) = 1− e2πint/N .
N +1
n=−N

Clearly FN ∈ VN,T , and of degree N .

 2
a) Show that FN (t) = N1+1 sin(π(N +1)t/T )
sin(πt/T ) , and conclude from this that 0 ≤ FN (t) ≤
T2
4(N +1)t2 .

Hint.

2
Use that π |u| ≤ | sin u| when u ∈ [−π/2, π/2].

Solution.

We compute

N  
2
X |n|
sin (πt/T ) 1− e2πint/N
N +1
n=−N
N  
1 X |n|
= (1 − cos(2πt/T )) 1− e2πint/N
2 N +1
n=−N
 N
 X  
1 1 1 |n|
= − e−2πint/T + − e2πint/T 1− e2πint/N .
4 2 4 N +1
n=−N

When one here multiplies out, one sees that most terms cancel (since −1/4 + 1/2 − 1/4 =
0), and we are left with

 
1 1 1 1 1
− e−2πi(N +1)t/T + − e2πi(N +1)t/T = sin2 (π(N + 1)t/T ).
N +1 4 2 4 N +1
2
This proves the claim. Finally, using that π |u| ≤ | sin u| when u ∈ [−π/2, π/2], we obtain
2
1 π2 T2

1 sin(π(N + 1)t/T ) 1
≤ = .
N +1 sin(πt/T ) N + 1 4 π 2 t2 /T 2 4(N + 1)t2

b) Show that FN (t) satisfies the three properties of a summability kernel.

Solution.

The first property of summability is easily seen to hold. Since from a) FN (t) ≥ 0, the
the second property follows from the first. To prove the third, due to a) we only need to
RT
compare with the integral δ t−2 dt, which exists for any δ.
RT PN
c) Show that T1 0 FN (u)f (t − u)du = N1+1 n=0 fn (t).
28 1 Sound and Fourier series

Hint.

1
PN
Show that FN (t) = N +1 n=0 Dn (t), and use Exercise 1.43 b).

Solution.

We have that

N N
!
1 X 1 X
−2πint/T 2πint/T
Dn (t) = N +1+ (N − n + 1)(e +e )
N + 1 n=0 N +1 n=0
N
X N + 1 − n −2πint/T
=1− (e + e2πint/T )
n=0
N + 1
N  
X n
=1− 1− (e−2πint/T + e2πint/T )
n=0
N +1
N  
X |n|
= 1− e2πint/T = FN (t).
N +1
n=−N

Using this we obtain that

T N N
1 X 1 T
Z Z
1 1 X
FN (u)f (t − u)du = DN (u)f (t − u)du = fn (t).
T 0 N + 1 n=0 T 0 N + 1 n=0
Chapter 2
Digital sound and Discrete Fourier analysis

Exercise 2.8: Computing the DFT by hand

Compute F4 x when x = (2, 3, 4, 5).

Solution.

As in Example 2.3 in the book we get

    
2 1 1 1 1 2
3 1 1 −i −1 i  3
F4 
4 = 2 1 −1 1 −1 4
   

5 1 i −1 −i 5
   
2+3+4+5 7
1 2 − 3i − 4 + 5i = −1 + i .
 
= 
2  2 − 3 + 4 − 5   −1 
2 + 3i − 4 − 5i −1 − i

Exercise 2.9: Exact form of low-order DFT matrix

As in Example 2.3 in the book, state the exact Cartesian form of the Fourier matrix for
the cases N = 6, N = 8, and N = 12.

Solution.

For N = 6 the entries are on the form √1 e−2πink/6 = √1 e−πink/3 . This means that the
6 6 √
entries in the Fourier matrix are the numbers √16 e−πi/3 = √16 (1/2−i 3/2), √16 e−2πi/3 =
√
√1 (−1/2 − i 3/2), and so on. The matrix is thus
6

29
30 2 Digital sound and Discrete Fourier analysis
 
1 1√ 1 √ 1 1 √ 1√
1 1/2 − i 3/2 −1/2 − i 3/2 −1 −1/2 + i√3/2 1/2 + i √2/2 
 √ √ 
1 1 −1/2 − i 3/2 −1/2 + i 3/2 1 −1/2 − i 3/2 +1/2 − i 3/2
F6 = √  
1
6 −1 √ 1 √ −1 1 √ −1 √ 

1 −1/2 + i 3/2 −1/2 − i 3/2 1 −1/2 + i√3/2 −1/2 − √
i 3/2
√ √
1 1/2 + i 2/2 −1/2 + i 3/2 −1 −1/2 − i 3/2 1/2 − i 3/2

The cases N = 8 and N = 12 follow similarly, but are even more tedious. For N = 8 the
entries are √18 eπink/4 , which can be expressed exactly since we can express exactly any
sines and cosines of a multiple of π/4. For N = 12 we get the base angle π/6, for which
we also have exact values for sines and cosines for all multiples.

Exercise 2.10: DFT of a delayed vector

We have a real vector x with length N , and define the vector z by delaying all elements in
x with 5 cyclically, i.e. z5 = x0 , z6 = x1 ,. . . ,zN −1 = xN −6 , and z0 = xN −5 ,. . . ,z4 = xN −1 .
For a given n, if |(FN x)n | = 2, what is then |(FN z)n |?

Solution.

z is the vector x delayed with d = 5 samples, and then Property 3 of Theorem 7 in the
book gives us that (FN z)n = e−2πi5k/N (FN x)n . In particular |(FN z)n | = |(FN x)n | = 2,
since |e−2πi5k/N | = 1.

Exercise 2.11: Using the symmetry property

Given a real vector x of length 8 where (F8 (x))2 = 2 − i, what is (F8 (x))6 ?

Solution.

By Theorem 7 in the book we know that (FN (x))N −n = (FN (x))n when x is a real
vector. If we set N = 8 and n = 2 we get that (F8 (x))6 = (F8 (x))2 = 2 − i = 2 + i.

Exercise 2.12: DFT of cos2 (2πk/N )

Let x be the vector of length N where xk = cos2 (2πk/N ). What is then FN x?

Solution.

The idea is to express x as a linear combination of the Fourier basis vectors φn , and use
that FN φn = en . We have that
2 Digital sound and Discrete Fourier analysis 31
 
1 2πik/N  2
2 −2πikn/N
cos (2πk/N ) = e +e
2
1 1 1 1 1 1
= e2πi2k/N + + e−2πi2k/N = e2πi2k/N + + e2πi(N −2)k/N
4  2 4 4 2 4
√

1 1 1
= N φ2 + φ0 + φN −2 .
4 2 4

We here used the periodicity of e2πikn/N , i.e. that e−2πi2k/N = e2πi(N −2)k/N . Since FN
is linear and FN (φn ) = en , we have that

√ √
 
1 1 1
FN (x) = N e2 + e0 + eN −2 = N (1/2, 0, 1/4, 0, . . . , 0, 1/4, 0) .
4 2 4

Exercise 2.13: DFT of ck x

Let x be the vector with entries xk = ck . Show that the DFT of x is given by the vector
with components

1 − cN
yn =
1 − ce−2πin/N
for n = 0, . . . , N − 1.

Solution.

We get

N
X −1 N
X −1
yn = ck e−2πink/N = (ce−2πin/N )k
k=0 k=0
−2πin/N N
1 − (ce ) 1 − cN
= = .
1 − ce−2πin/N 1 − ce−2πin/N

Exercise 2.14: DFT implementation

Extend the code for the function dft_impl in Example 2.4 in the book so that

1. The function also takes a second parameter called forward. If this is true the DFT is
applied. If it is false, the IDFT is applied. If this parameter is not present, then the
forward transform should be assumed.
2. If the input x is two-dimensional (i.e. a matrix), the DFT/IDFT should be applied
to each column of x. This ensures that, in the case of sound, the FFT is applied to
each channel in the sound when the entire sound is used as input.
Also, write documentation for the code.
32 2 Digital sound and Discrete Fourier analysis

Solution.

The library version of this function looks as follows.

function y = dft_impl(x, forward)

sign = -1;
if nargin >= 2 & forward == 0
sign = 1;
end
N = size(x, 1);
y = zeros(size(x));
for n = 1:N
D = exp(-sign*2*pi*1i*(n-1)*(0:(N-1))/N);
for s2 = 1:size(x,2)
y(n, s2) = dot(D, x(:, s2));
end
end
if sign == 1
y = y/N;
end
end

Exercise 2.15: Symmetry

Assume that N is even.

a) Prove that, if xk+N/2 = xk for all 0 ≤ k < N/2, then yn = 0 when n is odd.

Solution.

We have that

 
N/2−1 N −1
1  X X
yn = √ xk e−2πikn/N + xk e−2πikn/N 
N k=0 k=N/2
 
N/2−1 N/2−1
1  X X
=√ xk e−2πikn/N + xk e−2πi(k+N/2)n/N 
N k=0 k=0
N/2−1
1 X
=√ xk (e−2πikn/N + (−1)n e−2πikn/N )
N k=0
N/2−1
1 X
= (1 + (−1)n ) √ xk e−2πikn/N
N k=0

If n is odd, we see that yn = 0.

b) Prove that, if xk+N/2 = −xk for all 0 ≤ k < N/2, then yn = 0 when n is even.
2 Digital sound and Discrete Fourier analysis 33

Solution.

The proof is the same as in a), except for a sign change.

c) Prove the converse statements in a) and b).

Solution.

Clearly the set of vectors which satisfies xk+N/2 = ±xk is a vector space V of dimension
N/2. The set of vectors where every second component is zero is also a vector space of
dimension N/2, let us denote this by W . We have shown that FN (V ) ⊂ W , but since
FN is unitary, FN (V ) also has dimension N/2, so that FN (V ) = W . This shows that
when every second yn is 0, we must have that xk+N/2 = ±xk , and the proof is done.
d) Prove the following:

• xn = 0 for all odd n if and only if yk+N/2 = yk for all 0 ≤ k < N/2.
• xn = 0 for all even n if and only if yk+N/2 = −yk for all 0 ≤ k < N/2.

Solution.

In the proofs above, compute the IDFT instead.

Exercise 2.16: DFT on complex and real data

Let x1 , x2 be real vectors, and set x = x1 + ix2 . Use Theorem 7 in the book to show
that

1 
(FN (x1 ))k = (FN (x))k + (FN (x))N −k
2
1  
(FN (x2 ))k = (FN (x))k − (FN (x))N −k
2i
This shows that we can compute two DFT’s on real data from one DFT on complex
data, and 2N extra additions.

Solution.

We have that

(FN (x))k = (FN (x1 + ix2 ))k = (FN (x1 ))k + i(FN (x2 ))k
(FN (x))N −k = (FN (x1 ))N −k + i(FN (x2 ))N −k = (FN (x1 ))k + i(FN (x2 ))k ,

where we have used Property 1 of Theorem 7 in the book. If we take the complex
conjugate in the last equation, we are left with the two equations
34 2 Digital sound and Discrete Fourier analysis

(FN (x))k = (FN (x1 ))k + i(FN (x2 ))k

(FN (x))N −k = (FN (x1 ))k − i(FN (x2 ))k .

If we add these we get

1 
(FN (x1 ))k = (FN (x))k + (FN (x))N −k ,
2
which is the first equation. If we instead subtract the equations we get
1  
(FN (x2 ))k = (FN (x))k − (FN (x))N −k ,
2i
which is the second equation

Exercise 2.17: Vandermonde matrices

−1
In linear algebra, an N × N -matrix where column j is on the form (x0j , x1j , ..., xN
j ) for
some number xj is called a Vandermonde matrix. The number xj is called a generator
for (the j’th column) of the matrix.
a) Show that a Vandermonde matrix where all the generators are distinct and nonzero
is non-singular.

Hint.

Combine elementary row operations with a cofactor expansion along the last column.

Solution.

If we subtract xN −1 times row k from row k + 1 for k ≥ 0, the last column will become
equal to e0 . The other entries are now aij = (xj )i − xN −1 xi−1
j = xi−1
j (xj − xN −1 ) (the
new matrix has the same determinant). A cofactor expansion is now natural, and leads
to the smaller matrix where we drop the first row and the last column. It is enough to
prove that this smaller matrix is non-singular. Its entries are xij (xj − xN −1 ) (since i
was shifted by one). (xj − xN −1 ) is a common factor for each column which is nonzero
and can be moved out of the matrix without affecting if the determinant is nonzero
or not. After this we are left with a Vandermonde matrix with generators x1 , ..., xN −2 .
The result now follows by induction.
b) Let A be the s × s-matrix obtained by taking s distinct columns of DF TN , and s
successive columns of DF TN . Show that A is non-singular.

Solution.

The DFT-matrix has distinct generators for each column. As long as the rows we choose
are successive, the non-singularity clearly follows from that of the Vandermonde matrix
2 Digital sound and Discrete Fourier analysis 35

(if we don’t start with the first row, simply move out a constant from each column to
relate to a Vandermonde matrix).

Exercise 2.18: Fourier basis vectors with zeros inserted

Later we will encounter vectors created by zeroing out certain components in the Fourier
basis vectors, and we will have use for the DFT of these.
a) Show that
1 2πirk/N
(e2πir·0/N , 0, e2πir·2/N , 0, . . . , e2πir(N −2)/N , 0) = (e + e2πi(r+N/2)k/N )
2
1
(0, e2πir·1/N , 0, e2πir·3/N , . . . , 0, e2πir(N −1)/N ) = (e2πirk/N − e2πi(r+N/2)k/N ).
2
Do this by computing the DFT of the left hand sides.

Solution.

We prove only the first claim. Component n of the DFT of the left hand side is
N/2−1 N/2−1
X X
e2πir(2k)/N e−2πin(2k)/N = e2πi(r−n)k/(N/2) .
k=0 k=0

This is clearly zero unless n = r or n = r + N/2. For these two values this computes to
N/2. It follows that the DFT of the left hand side is N2 (er + er+N/2 ). Taking the IDFT
proves the claim.
b) Assume that N (the length of the vector) is divisible by M , and let s be a
fixed number. Show that the vector where only components on the form M k + s
in (e2πir·0/N , e2πir·1/N , . . . , e2πir(N −1)/N ) are kept (the others zeroed out) can be written
as
M −1
1 X −2πist/M 2πi(r+tN/M )k/N
e e .
M t=0
In other words, such vectors can be written as a sum of M Fourier basis vectors.

Solution.

Again computing component n of the DFT of the vector in question gives

N/M −1 N/M −1
X X
2πir(M k+s)/N −2πin(M k+s)/N 2πi(r−n)s/N
e e =e e2πi(r−n)k/(N/M ) .
k=0 k=0
36 2 Digital sound and Discrete Fourier analysis

The sum here is zero unless n = r + tN/M for some 0 ≤ t < M , for which
the sum computes to N/M . It follows that the DFT of the vector in question is
N
PM −1 −2πits/M
M t=0 e er+tN/M . Again taking the IDFT proves the claim.

Exercise 2.19: Code example

Explain what the code below does, line by line:

[x, fs] = audioread(’sounds/castanets.wav’);

N = size(x, 1);
y = fft(x);
y((round(N/4)+1):(round(N/4)+N/2), :) = 0;
newx = abs(ifft(y));
newx = newx/max(max(newx));
playerobj = audioplayer(newx, fs);
playblocking(playerobj)

Comment in particular why we adjust the sound samples by dividing with the maximum
value of the sound samples. What changes in the sound do you expect to hear?

Solution.

First a sound file is read. We then restrict to the first 212 sound samples, perform a DFT,
zero out the frequencies which correspond to DFT-indices between 210 and 212 − 210 − 1,
and perform an IDFT. Finally we scale the sound samples so that these lie between −1
and 1, which is the range we demand for the sound samples, and play the new sound.

Exercise 2.20: Mapping DFT indices to frequencies

In the code from the previous exercise it turns out that fs = 44100Hz, and that the
number of sound samples is N = 292570. Which frequencies in the sound file will be
changed on the line where we zero out some of the DFT coefficients?

Solution.

As we have seen, DFT index n corresponds to frequency ν = nfs /N . Above N = 217 , so

that we get the connection ν = nfs /N = n × 44100/217 . We zeroed the DFT indices
above n = 215 , so that frequencies above ν = 215 × 44100/217 = 11025Hz are affected.

Exercise 2.21: Plotting the interpolant

Implement code where you do the following:

• at the top you define the function f (x) = cos6 (x), and M = 3,
• compute the unique interpolant from VM,T (i.e. by taking N = 2M + 1 samples over
one period), as guaranteed by Proposition 9 in the book,
2 Digital sound and Discrete Fourier analysis 37

• plot the interpolant against f over one period.

Finally run the code also for M = 4, M = 5, and M = 6. Explain why the plots coincide
for M = 6, but not for M < 6. Does increasing M above M = 6 have any effect on the
plots?

Solution.

The code can look as follows.

f = @(t)cos(t).^6;
M = 5;
T = 2*pi;
N = 2*M + 1;
t = (linspace(0, T, 100))’;
x = f((linspace(0, T - T/N, N))’);
y = fft(x)/N;
s = y(1)*ones(length(t),1);
for k=1:((N-1)/2)
s = s + 2*real(y(k+1)*exp(2*pi*1i*k*t/T));
end
plot(t, s, ’r’, t, f(t), ’g’);
legend(’Interpolant from V_{M,T}’, ’f’)

Exercise 2.22: Proving the general version of the sampling

theorem for functions in a Fourier space

a) Show that the function

(
1 1
sin t − t , if t > 0
g(t) =
0 , if t = 0
is continuous and strictly increasing on (−π, π).

Solution.

It is clear that g is continuous when t 6= 0. We also have that

 
1 1 t − sin t 1 − cos t sin t
lim − = lim = lim = lim =0
t→0 sin t t t→0 t sin t t→0 sin t + t cos t t→0 2 cos t − t sin t

where L’hôpitals rule was used twice. This proves that g is continuous at 0 as well.
We will now prove that g 0 (t) > 0 for all t ∈ (0, π). We have that
cos t 1
g 0 (t) = −
2 + t2 .
sin t
This is obviously > 0 for t ≥ π/2, since cos t is negative then. In the remainder of the
proof we will therefore assume that t < π/2, and cos t > 0.
2
g 0 (t) > 0 is the same as sint2 t > cos t, which can be written as 12 (1 − cos(2t)) > t2 cos t.
This can be written in terms of Taylor series as
38 2 Digital sound and Discrete Fourier analysis

∞ ∞
X 22k−1 2k X 1
(−1)k−1 t > (−1)k−1 t2k .
(2k)! (2(k − 1))!
k=1 k=1

separating the two first terms on both sides this is the same as

t2 − t4 /3 + a1 − a2 + a3 − · · · > t2 − t4 /2 + b1 − b2 + b3 . · · ·
Where the ai and bi are positive. It is easily checked that the ai and bi are decreasing
for t < 2 (so in particular for t < π/2), since a2 is obtained from a1 by multiplying
with 4/(7 · 8)t2 < 1, b2 is obtained from b1 by multiplying with 1/(5 · 6)t2 < 1 (and
in the next steps we multiply with even smaller values). It is also easily checked that
b1 = t6 /24. It follows from a result on the sum of an alternating series that

a1 − a2 + a3 − · · · > 0 b1 − b2 + b3 − · · · < t6 /24

Inserting this in the above we see that it is enough to prove that

t2 − t4 /3 > t2 − t4 /2 + t6 /24,
which can be simplified to t4 /6 > t6 /24, which clearly holds for t < 2.
The case for negative t follows also since g is anti-symmetric.
Any function of period T can also be viewed as a function of period sT , for any
integer s. Replacing N with sN , T with sT , and taking the difference between two
corresponding terms in the ideal interpolation formula for periodic functions, and the
one for non-periodic functions, we get

 
1 1
f (kTs ) sin(π(t − kTs )/Ts ) − .
sN sin(π(t − kTs )/(sT )) π(t − kTs )/Ts

b) With
1 1
ak = − = g(π(t − kTs )/(sT )),
sin(π(t − kTs )/(sT )) π(t − kTs )/(sT )
the difference between the ideal interpolation formula for periodic and non-periodic
functions is
sM
1 X
f (kTs ) sin(π(t − kTs )/Ts )ak
sN
k=−sM

Show that, for N odd, this can be made arbitrarily small by choosing s big enough.

Hint.

Split the sum into N alternating series, so that there a common f (kTs ) in each series.
The result from a) will be needed in order to bound each of those series.
2 Digital sound and Discrete Fourier analysis 39

Solution.

Since −sM ≤ k ≤ sM , the argument π(t − kTs )/(sT ) to g satisfies

πt πM π(t − kTs ) πt πM
− ≤ ≤ + .
sT N sT sT N
Using that N ≥ 2M + 1 this can also be written as

πt π π(t − kTs ) πt π
− (1 − 1/N ) ≤ ≤ + (1 − 1/N ).
sT 2 sT sT 2
By choosing s big enough this clearly lies inside the interval (−π/2, π/2).
If we take N odd, i.e. we take an odd number of samples in each period, the partial
sum sl of the above sum for k on the form rN + l, i.e.
1 X
sl = f ((rN + l)Ts ) sin(π(t − (rN + l)Ts )/Ts )arN +l
sN
r|−sM ≤rN +l≤sM

2
will be alternating. Since the ak increase, the sum is bounded by |sl | ≤ sN |g(π/2)| (since
any alternating series where the terms decrease has sum less than the biggest term).
Clearly we can get this as small as we want by increasing s. Since there are N partial
sums, we can get the total difference as small as we want as well.
Note that this approach avoided t-values close to π. These would be problematic,
since g(t) → ∞ when t → π.
Thus, we can prove the more general interpolation formula for a periodic f by
considering f as a function with period sT , and letting s → ∞.

Exercise 2.23: Properties of the DFT when N is composite

When N = N1 N2 is composite, the following explains a duality between the polyphase

components x(k) with components xrN1 +k , and the polyphase components y (k) of
y = DFTN x with components yrN2 +k .
a) Assume that all x(k) are constant vectors (i.e. x is N1 -periodic). Show that y (k) = 0
for all k 6= 0.

Solution.

We have that x(p) is a constant vector of length N2 for 0 ≤ p < N1 . But then the DFT
of all the x(p) has zero outside entry zero. Multiplying with e−2πikn/N does not affect
this. The last N2 − 1 rows are thus zero before the final DFT is applied, so that these
rows are zero also after this final DFT. After assembling the polyphase components
again we have that yrN2 are the only nonzero DFT-coefficients.
b) Assume that N = N1 N2 , and that x(p) = 0 for p 6= 0. Show that the y (p) are
constant vectors for all p.
40 2 Digital sound and Discrete Fourier analysis

Solution.

Write any number < N as kN1 + l for 0 ≤ k < N2 , 0 ≤ l < N1 . We have that

2 −1 N
NX 1 −1
(l)
X
yrN2 +p = xk e−2πi(kN1 +l)(rN2 +p)/N
k=0 l=0
2 −1
NX 2 −1
NX
(0) (0)
= xk e−2πikN1 (rN2 +p)/N = xk e−2πikN1 p/N ,
k=0 k=0

which is independent of r, so that the y (p) are constant vectors.

Exercise 2.24: FFT implementation which works for both

forward and reverse

Recall that, in Exercise 2.14, we extended the direct DFT implementation so that
it accepted a second parameter telling us if the forward or reverse transform should
be applied. Extend fft_impl and fft_kernel_standard in the same way. Again, the
forward transform should be used if the forward parameter is not present. Assume also
that the kernel accepts only one-dimensional data, and that the general function applies
the kernel to each column in the input if the input is two-dimensional (so that the FFT
can be applied to all channels in a sound with only one call).
It should be straightforward to make the modifications for the reverse transform by
consulting the second part of Theorem 16 in the book. For simplicity, let fft_impl take
care of the additional division with N in case of the IDFT. In the following we will
assume these signatures for the FFT implementation and the corresponding kernels.

Solution.

The library versions of these functions are as follows:

function y = fft_impl(x, f, forward)

fwd = 1;
if nargin >= 3
fwd = forward;
end
x = bit_reversal(x);
[N, n] = size(x);
y = zeros(N, n);
for s2 = 1:size(x, 2)
y(:, s2) = f(x(:,s2), fwd);
end
if ~fwd
y = y/N;
end
end

% Compute the DFT of the column vector x using the FFT algorithm
function y = fft_kernel_standard(x, forward)
N = size(x, 1);
2 Digital sound and Discrete Fourier analysis 41

sign = -1;
if ~forward
sign = 1;
end
if N == 1
y = x;
else
xe = fft_kernel_standard(x(1:(N/2)), forward);
xo = fft_kernel_standard(x((N/2+1):N), forward);
D = exp(sign*2*pi*1j*(0:(N/2-1))’/N);
xo = xo.*D;
y = [ xe + xo; xe - xo];
end
end

Exercise 2.25: Execution times for the FFT

Let us compare execution times for the different methods for computing the DFT.
a) Write code which compares the execution times for an N -point DFT for the following
three cases: Direct implementation of the DFT (as in Example 2.4 in the book), the FFT
implementation used in this chapter, and the built-in fft-function. Your code should
use the sample audio file castanets.wav, apply the different DFT implementations to
the first N = 2r samples of the file for r = 3 to r = 15, store the execution times in a
vector, and plot these. You can use the functions tic and toc to measure the execution
time.

Solution.

The code can look as follows.

[x0, fs] = audioread(’sounds/castanets.wav’);

kvals = 3:15;
slowtime=zeros(1,length(kvals));
fasttime = slowtime; fastesttime = slowtime;
N = 2.^kvals;
for k = kvals
x = x0(1:2^k,1);
tic;
y = dft_impl(x);
slowtime(k-2) = toc;
tic;
y = fft_impl(x, @fft_kernel_standard);
fasttime(k-2) = toc;
tic;
y = fft(x);
fastesttime(k-2) = toc;
end
% a.
plot(kvals, slowtime, ’r’, ...
kvals, fasttime, ’g’, ...
kvals, fastesttime, ’b’)
grid on
title(’time usage of the DFT methods’)
legend(’DFT’, ’Standard FFT’, ’Built-in FFT’)
42 2 Digital sound and Discrete Fourier analysis

xlabel(’log_2 N’)
ylabel(’time used [s]’)
% b.
figure(2)
loglog(N, slowtime, ’r’, N, fasttime, ’g’, N, fastesttime, ’b’)
axis equal
legend(’DFT’, ’Standard FFT’, ’Built-in FFT’)

b) A problem for large N is that there is such a big difference in the execution times
between the two implementations. We can address this by using a loglog-plot. Plot N
against execution times using the function loglog. How should the fact that the number
of arithmetic operations are 8N 2 and 5N log2 N be reflected in the plot?

Solution.

The two different curves you see should have a derivative approximately equal to one
and two, respectively.
c) It seems that the built-in FFT is much faster than our own FFT implementation,
even though they may use similar algorithms. Try to explain what can be the cause of
this.

Solution.

There may be several reasons for this. One is that MATLAB code runs slowly when
compared to native code, which is much used in the built-in FFT. Also, the built-in fft
has been subject to much more optimization than we have covered here. Another reason
is that functions in MATLAB copy the parameters each time a function is called. When
the vectors are large, this leads to extensive copying, also since the recursion depth is
big.

Exercise 2.26: Combining two FFT’s

Let x1 = (1, 3, 5, 7) and x2 = (2, 4, 6, 8). Compute DFT4 x1 and DFT4 x2 . Explain how
you can compute DFT8 (1, 2, 3, 4, 5, 6, 7, 8) based on these computations (you don’t need
to perform the actual computation). What are the benefits of this approach?

Solution.

We get
2 Digital sound and Discrete Fourier analysis 43
    
1 1 1 1 1 16
1 −i −1 i  3 −4 + 4i
DFT4 x1 = 
1 −1 1 −1 5 =  −4 
   

1 i −1 −i 7 −4 − 4i
    
1 1 1 1 2 20
1 −i −1 i  4 −4 + 4i
DFT4 x2 = 
1 −1 1 −1 6 =  −4 
   

1 i −1 −i 8 −4 − 4i

In the FFT-algorithm we split the computation of DFT4 (x) into the computation of
DFT2 (x(e) ) and DFT2 (x(o) ), where x(e) and x(o) are vectors of length 4 with even-
indexed and odd-indexed components, respectively. In this case we have x(e) = (1, 3, 5, 7)
and x(o) = (2, 4, 6, 8). In other words, the FFT-algorithm uses the FFT-computations
we first made, so that we can save computation. The benefit of using the FFT-algorithm
is that we save computations, so that we end up with O(5N log2 N ) real arithmetic
operations.

Exercise 2.27: FFT operation count

When we wrote down the difference equation for the number of multiplications in the
FFT algorithm, you could argue that some multiplications were not counted. Which
multiplications in the FFT algorithm were not counted when writing down this difference
equation? Do you have a suggestion to why these multiplications were not counted?

Solution.

When we compute e−2πin/N , we do some multiplications/divisions in the exponent. These

are not counted because they do not depend on x, and may therefore be pre-computed.

Exercise 2.28: FFT algorithm adapted to real data

It is possible to adapt an FFT algorithm to real input. Since yN −n = yn for real input,
there is no additional complexity in computing the second half of the DFT coefficients,
once the first half has been computed. We will now rewrite equation (2.11) in the book
for indices n and N/2 − n as

yn = (DFTN/2 x(e) )n + e−2πin/N (DFTN/2 x(o) )n

yN/2−n = (DFTN/2 x(e) )N/2−n + e−2πi(N/2−n)/N (DFTN/2 x(o) )N/2−n
= (DFTN/2 x(e) )n − e2πin/N (DFTN/2 x(o) )n
= (DFTN/2 x(e) )n − e−2πin/N (DFTN/2 x(o) )n .

We see here that, if we already have computed DFTN/2 x(e) and DFTN/2 x(o) , we need
one additional complex multiplication for each yn with 0 ≤ n < N/4 (since e−2πin/N
44 2 Digital sound and Discrete Fourier analysis

and (DFTN/2 x(o) )n are complex). No further multiplications are needed in order to
compute yN/2−n , since we simply conjugate terms before adding them. Again yN/2 must
be handled explicitly with this approach. For this we can use the formula

yN/2 = (DFTN/2 x(e) )0 − (DN/2 DFTN/2 x(o) )0

instead.
a) Conclude from this that an FFT algorithm adapted to real data at each step requires
N/4 complex additions and N/2 complex additions. Conclude from this as before that
an algorithm based
on real data requires MN = O(N log2 N ) real multiplications and
AN = O 32 N log2 N real additions (i.e. half the operation count for complex input).

Solution.

The formulas at the top show that we need only compute the e−2πin/N (DFTN/2 x(o) )n
for 0 ≤ n < N/4, i.e. N/4 complex additions. Since N/2 formulas are listed, N/2 complex
additions are needed. This translates to N real multiplications, and N/2 + N = 3N/2
real additions. These are exactly half the numbers we obtained for the complex version of
the algorithm, for which we obtained MN = O(2N log2 (N )) and AN = O(3N log2 (N )).

version will require MN = O(N log2 N ) real
It is then not difficult to see that the real
multiplications and AN = O 32 N log2 N real additions.
b) Find an IFFT algorithm adapted to vectors with conjugate symmetry, which has
the same operation count as this FFT algorithm adapted to real data.

Hint.

Consider the vectors z, w with entries zn = yn + yN/2−n ∈ CN/2 and wn = e2πin/N (yn −
yN/2−n ) ∈ CN/2 . From the equations above, how can these be used in an IFFT?

Solution.

Note that

zN/2−n = yN/2−n + yn = yn + yN/2−n = zn

wN/2−n = e2πi(N/2−n)/N (yN/2−n − yn ) = −e−2πin/N (yN/2−n − yn )
= e2πin/N (yn − yN/2−n ) = wn ,

showing that the vectors z and w also have conjugate symmetry. Note also that, if we
add and subtract the equations at the top, we obtain

2(DFTN/2 x(e) )n = yn + yN/2−n = zn

2(DFTN/2 x(o) )n = e2πin/N (yn − yN/2−n ) = wn .

so that
2 Digital sound and Discrete Fourier analysis 45

1 1
x(e) = IDFTN/2 z x(o) = IDFTN/2 w.
2 2
The point is that the number of arithmetic operations in computing the vectors z and
w is the same as the FFT in a), so that the overall complexity is the same.

Exercise 2.29: Non-recursive FFT algorithm

Use the factorization in (2.16) in the book to write a kernel function for a non-recursive
FFT implementation. In your code, perform the matrix multiplications in equation
(2.16) in the book from right to left in an (outer) for-loop. For each matrix loop through
the different blocks on the diagonal in an (inner) for-loop. Make sure you have the right
number of blocks on the diagonal, each block being on the form
 
I DN/2k
.
I −DN/2k
It may be a good idea to start by implementing multiplication with such a simple matrix
first as these are the building blocks in the algorithm. Do this so that everything is
computed in-place. Also compare the execution times with our original FFT algorithm,
as we did in Exercise 2.25, and try to explain what you see in this comparison.

Solution.

The library version of the non-recursive FFT is as follows

function y = fft_kernel_nonrec(x, forward)

N = size(x, 1);
sign = -1;
if ~forward
sign = 1;
end
D = exp(sign*2*pi*1i*(0:(N/2 - 1))’/N);
nextN = 1;
while nextN < N
k = 1;
while k <= N
xe = x(k:(k + nextN - 1));
xo = x((k + nextN):(k + 2*nextN - 1));
xo = xo.*D(1:(N/(2*nextN)):(N/2));
x(k:(k + 2*nextN - 1)) = [xe + xo; xe - xo];
k = k + 2*nextN;
end
nextN = nextN*2;
end
y = x;
end

If you add the non-recursive algorithm to the code from Exercise 2.25, you will see
that the non-recursive algorithm performs much better. There may be several reasons
for this. First of all, there are no recursive function calls. Secondly, the values in the
matrices DN/2 are constructed once and for all with the non-recursive algorithm. Code
which compares execution times for the original FFT algorithm, our non-recursive
implementation, and the split-radix algorithm of the next exercise, can look as follows:
46 2 Digital sound and Discrete Fourier analysis

[x0, fs] = audioread(’sounds/castanets.wav’);

kvals = 3:15;
slowtime = zeros(1,length(kvals));
fasttime = slowtime; fastesttime = slowtime;
N = 2.^(kvals);
for k = kvals
x = x0(1:2^k,1);
tic;
y = fft_impl(x, @fft_kernel_standard);
slowtime(k-2) = toc;
tic;
y = fft_impl(x, @fft_kernel_nonrec);
fasttime(k-2) = toc;
tic;
y = fft_impl(x, @fft_kernel_splitradix);
fastesttime(k-2) = toc;
end
plot(kvals, slowtime, ’ro-’)
hold on
plot(kvals,fasttime, ’bo-’)
plot(kvals,fastesttime, ’go-’)
grid on
title(’time usage of the DFT methods’)
legend(’Standard FFT algorithm’, ...
’Non-recursive FFT’, ...
’Split radix FFT’)
xlabel(’log_2 N’)
ylabel(’time used [s]’)

Exercise 2.30: The Split-radix FFT algorithm

The split-radix FFT algorithm is a variant of the FFT algorithm. Until recently it held
the record for the lowest operation count for any FFT algorithm. It was first derived in
1968 [6], but largely forgotten until a paper in 1984 [2].
We start by splitting the rightmost DFTN/2 in equation (2.15) in the book by using
this equation again, to obtain
  
DFTN/4 DN/4 DFTN/4  (e) 
DFT N/2 DN/2 x
DFTN/4 −DN/4 DFTN/4  x(oe)  .
 
DFTN x =   DFTN/4 DN/4 DFTN/4 
DFTN/2 −DN/2 x(oo)
DFTN/4 −DN/4 DFTN/4
The term radix describes how an FFT is split into FFT’s of smaller sizes, i.e. how the
sum in an FFT is split into smaller sums. The FFT algorithm we started this section
with is called a radix 2 algorithm, since it splits an FFT of length N into FFT’s of
length N/2. If an algorithm instead splits into FFT’s of length N/4, it is called a radix
4 FFT algorithm. The algorithm we go through here is called the split radix algorithm,
since it uses FFT’s of both length N/2 and N/4.
−2πin/N
 (N/4) × (N/4)
a) Let GN/4 be the  diagonal matrix with e on the diagonal.
GN/4 0
Show that DN/2 = .
0 −iGN/4
2 Digital sound and Discrete Fourier analysis 47

Solution.

We have that

 −2πi·0/N
··· ···

e 0 0 0
 .. .. .. .. .. .. 
 . . . . . . 
   ..

GN/4 0
 
=
 0 · · · e−2πi(N/4−1)/N 0 ··· . 
0 −iGN/4


 0 ··· 0 −ie−2πi·0/N ··· 0 

 .. .. .. .. .. .. 
 . . . . . . 
0 ··· 0 0 · · · −ie−2πi(N/4−1)/N
 −2πi·0/N
··· ···

e 0 0 0
 .. .. .. .. .. .. 
 . . . . . . 
..
 
· · · e−2πi(N/4−1)/N
 
 0 0 ··· . 
=
 ..


−2πi(N/4)/N

 0 . 0 e ··· 0 

 .. .. .. .. .. .. 
 . . . . . . 
−2πi(N/2−1)/N
0 ··· 0 0 ··· e
= DN/2 .

b) Let HN/4 be the (N/4) × (N/4) diagonal matrix GD/4 DN/4 . Verify the following
rewriting of the equation above:

  
GN/4 DFTN/4 HN/4 DFTN/4  (e) 
DFTN/2 −iGN/4 DFTN/4 iHN/4 DFTN/4  x
DFTN x =    x(oe) 
 −GN/4 DFTN/4 −HN/4 DFTN/4 
DFTN/2 x(oo)
iGN/4 DFTN/4 −iHN/4 DFTN/4
 
I 0 GN/4 HN/4    (e) 
0 I −iGN/4 iHN/4  DFTN/2 0 0 x
=  0 DFT N/4 0  x(oe) 
I 0 −GN/4 −HN/4 
0 0 DFTN/4 x(oo)
0 I iGN/4 −iHN/4
   
GN/4 HN/4  (e)

I −iG N/4 iHN/4
 DFTN/2 x
=   DFTN/4 x(oe) 
 GN/4 HN/4 
I− DFTN/4 x(oo)
−iGN/4 iHN/4
GN/4 DFTN/4 x(oe) + HN/4 DFTN/4 x(oo)

  
(e)
DFTN/2 x + −i GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo) 
= 
GN/4 DFTN/4 x(oe) + HN/4 DFTN/4 x(oo)


 (e)
DFTN/2 x −
−i GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo)

Solution.

In the first step we write

48 2 Digital sound and Discrete Fourier analysis
  
GN/4 0 DFTN/4 DN/4 DFTN/4
DN/2 DFTN/2 =
0 −iGN/4 DFTN/4 −DN/4 DFTN/4
 
GN/4 DFTN/4 GN/4 DN/4 DFTN/4
=
−iGN/4 DFTN/4 iGN/4 DN/4 DFTN/4
 
GN/4 DFTN/4 HN/4 DFTN/4
=
−iGN/4 DFTN/4 iHN/4 DFTN/4

c) Explain from the above expression why, once the three FFT’s above have been
computed, the rest can be computed with N/2 complex multiplications, and 2×N/4+N =
3N/2 complex additions. This is equivalent to 2N real multiplications and N + 3N = 4N
real additions.

Hint.

It is important that GN/4 DFTN/4 x(oe) and HN/4 DFTN/4 x(oo) are computed first, and
the sum and difference of these two afterwards.

Solution.

Computing GN/4 DFTN/4 x(oe) and HN/4 DFTN/4 x(oo) each requires N/4 complex mul-
tiplications, i.e. a total of N/2 complex multiplications. Computing GN/4 DFTN/4 x(oe) +
HN/4 DFTN/4 x(oo) needs an additional N/4 complex additions, and the same for
GN/4 DFTN/4 x(oe) − HN/4 DFTN/4 x(oo) , so a total of N/2 complex additions. Finally
we need N more complex additions, since we finally compute DFTN/2 x(e) plus/minus
entries from these matrices.
d) Due to what we just showed, our new algorithm leads to real multiplication and
addition counts which satisfy

MN = MN/2 + 2MN/4 + 2N AN = AN/2 + 2AN/4 + 4N

Find the general solutions
to these difference equations and conclude from these
that MN = O 43 N log2 N , and AN = O 83 N log2 N . The operation count is thus

O (4N log2 N ), which is a reduction of N log2 N from the FFT algorithm.

Solution.

With xr = M2r (or A2r ) we obtain the difference equations xr+2 − xr+1 − 2xr = 8 · 2r
(h)
(or 16 · 2r ). The general solution to the homogeneous equation is xn = A2r + B(−1)r ,
(p) (p)
and a particular solution to the general equation is xn = 43 r2r (or xn = 83 r2r ). The
general solutions are thus xr = 43 r2r + A2r + B(−1)r (or xr = 83 r2r + A2r + B(−1)r ).
Since the first terms here
are the ones which are dominating, setting 2r = N we obtain
4 8


that MN = O 3 N log2 N , and AN = O 3 N log2 N .
e) Write a kernel function for the split-radix FFT algorithm (again this should handle
both forward and reverse transforms). Are there more or less recursive function calls in
this function than in the original FFT algorithm? Also compare the execution times
2 Digital sound and Discrete Fourier analysis 49

with our original FFT algorithm, as we did in Exercise 2.25. Try to explain what you
see in this comparison.

Solution.

The library version of the split-radix algorithm is as follows

function y = fft_kernel_splitradix(x, forward)

N = size(x, 1);
sign = -1;
if ~forward
sign = 1;
end
if N == 1
y = x;
elseif N == 2
y = [x(1) + x(2); x(1) - x(2)];
else
xe = fft_kernel_splitradix(x(1:(N/2)), forward);
xo1 = fft_kernel_splitradix(x((N/2 + 1):(3*N/4)), forward);
xo2 = fft_kernel_splitradix(x((3*N/4 + 1):N), forward);
G = exp(sign*2*pi*1j*(0:(N/4-1))’/N);
H = G.*exp(sign*2*pi*1j*(0:(N/4-1))’/(N/2));
xo1 = G.*xo1;
xo2 = H.*xo2;
xo = [xo1 + xo2; -sign*1i*(xo2 - xo1)];
y = [xe + xo; xe - xo];
end
end

If you add the split-radix FFT algorithm also to the code from Exercise 2.25, you will
see that it performs better than the FFT algorithm, but worse than the non-recursive
algorithm. That it performs better than the FFT algorithm is as expected, since it has
a reduced number of arithmetic operations, and also a smaller number of recursive calls.
It is not surprising that the non-recursive function performs better, since only that
function omits recursive calls, and computes the values in the diagonal matrices once
and for all.
By carefully examining the algorithm we have developed, one can reduce the operation
count to 4N log2 N − 6N + 8. This does not reduce the order of the algorithm, but for
small N (which often is the case in applications) this reduces the number of operations
considerably, since 6N is large compared to 4N log2 N for small N . In addition to having
a lower number of operations than the FFT algorithm of Theorem 14 in the book,
a bigger percentage of the operations are additions for our new algorithm: there are
now twice as many additions than multiplications. Since multiplications may be more
time-consuming than additions (depending on how the CPU computes floating-point
arithmetic), this can be a big advantage.

Exercise 2.31: Bit-reversal

In this exercise we will make some considerations which will help us explain the code
for bit-reversal. This is not a mathematically challenging exercise, but nevertheless a
good exercise in how to think when developing an efficient algorithm. We will use the
notation i for an index, and j for its bit-reverse. If we bit-reverse k bits, we will write
N = 2k for the number of possible indices.
50 2 Digital sound and Discrete Fourier analysis

a) Consider the following code

j = 0;
for i = 0:(N-1)
j
m = N/2;
while (m >= 1 && j >= m)
j = j - m;
m = m/2;
end
j = j + m;
end

Explain that the code prints all numbers in [0, N − 1] in bit-reversed order (i.e. j). Verify
this by running the program, and writing down the bits for all numbers for, say N = 16.
In particular explain the decrements and increments made to the variable j. The code
above thus produces pairs of numbers (i, j), where j is the bit-reverse of i. The function
bit_reversal in the library applies similar code, and then swaps the values xi and xj
in x, as it should.

Solution.

Note that, if the bit representation of i ends with 0 1| .{z

. . 1}, then i + 1 has a bit rep-
n
resentation which ends with 1 0| .{z
. . 0}, with the remaining first bits unaltered. Clearly
n
the bit-reverse of i then starts with 1| .{z
. . 1} 0 and the bit-reverse of i + 1 starts with
n
| .{z
10 . . 0}. We see that the bit reverse of i + 1 can be obtained from the bit-reverse of i by
n
replacing the first consecutive set of ones by zeros, and the following zero by one. This
is performed by the line above where j is decreased by m: Decreasing j by N/2 when
j ≥ N/2 changes the first bit from 1 to 0, and similarly for the next n bits. The line
where j is increased with m changes bit number n + 1 from 0 to 1.
Since bit-reverse is its own inverse (i.e. P 2 = I), it can be performed by swapping
elements i and j. One way to secure that bit-reverse is done only once, is to perform it
only when j > i. bit_reversal makes this check.
b) Explain that N − j − 1 is the bit-reverse of N − i − 1. Due to this, when i, j < N/2,
we have that N − i − 1, N − j − l ≥ N/2, and that bit_reversal can swap them.
Moreover, all swaps where i, j ≥ N/2 can be performed immediately when pairs where
i, j < N/2 are encountered. Explain also that j < N/2 if and only if i is even. In the
code you can see that the swaps (i, j) and (N − i − 1, N − j − 1) are performed together
when i is even, due to this.

Solution.

Clearly N − i − 1 has a bit representation obtained by changing every bit in i. That

N − j − 1 is the bit-reverse of N − i − 1 follows immediately from this. If i is even, the
least significant bit is 0. After bit-reversal, this becomes the most significant bit, and
the most significant bit of j is 0 which is the case if and only if j < N/2.
2 Digital sound and Discrete Fourier analysis 51

c) Assume that i < N/2 is odd. Explain that j ≥ N/2, so that j > i. This says that
when i < N/2 is odd, we can always swap i and j (this is the last swap performed in the
code). All swaps where 0 ≤ j < N/2 and N/2 ≤ j < N can be performed in this way.

Solution.

If i < N/2 is odd, then the least significant bit is 1. This means that the most significant
bit of j is 1, so that j ≥ N/2, so that j > i.
In bit_reversal, you can see that the bit-reversal of 2r and 2r + 1 are handled
together (i.e. i is increased with 2 in the for-loop). The effect of this is that the number
of if-tests can be reduced, due to the observations from b) and c).

Exercise 2.33: Writing down lower order DCT’s

As in Example 2.32 in the book, state the exact Cartesian form of the DCT matrix for
the case N = 3.

Solution.
q q
1
We first see that d0,3 = and dk,3 = 23 for k = 1, 2. We also have that
3
     
n 1 n 1
cos 2π k+ = cos π k+ ,
2N 2 3 2
so that the DCT matrix can be written as

 q q q 
1 1 1
q 3 3 3
π1

q2 π3

q2 π5


DCT3 =  23 cos cos cos
 
3 2 q3 3 2 q3 3 2

q

2 2π 1 2 2π 3 2 2π 5



3 cos 3 2 3 cos 3 2 3 cos 3 2
 q q q 
1 1 1
q 3 q q 3  3

=  23 cos(π/6) 23 cos(π/2) 23 cos(5π/6)

 
q q q 
2 2 2
3 cos(π/3) 3 cos(π) 3 cos(5π/3)
 q q q 
1 1 1
q √ 3 3 q
√
3 
=  23 ( 3/2 + i/2) 0 2
(− 3/2 + i/2)
 
3
q
2
√ q
2
q
2
√ 
3 (1/2 + 3i/2) − 3 3 (1/2 − 3i/2)
52 2 Digital sound and Discrete Fourier analysis

Exercise 2.34: Type-IV DCT

N −1
Show that the vectors {cos(2π(n + 1/2)(k + 1/2)/(2N ))}n=0qin RN are orthogonal,
p
with lengths N/2. This means that the matrix with entries N2 cos(2π(n + 1/2)(k +
1/2)/(2N )) is orthogonal. Since this matrix also is symmetric, it is its own inverse. This
(IV)
is the matrix DCTN , which can be used for type IV cosine matrices.

Hint.

Compare with the orthogonal vectors dn , used in the DCT.

Solution.

We can write

n + 21
     
1 2n + 1 1
cos 2π k+ = cos 2π k+ .
2N 2 4N 2
If we consider these as vectors of length 2N , we recognize thesep as the unit
p vectors
d2n+1 in the 2N -dimensional DCT,
√ divided by the factor dn = 2/(2N ) = 1/N , so
that these vectors have length N . To see that these vectors are orthogonal when we
restrict to the first N elements using the property

cos(2π(n + 1/2)(2N − k)/(2N )) = − cos(2π(n + 1/2)k/(2N ))

as follows:

2N −1
n1 + 12 n2 + 21
     
X 1 1
N δn1 ,n2 = cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
= cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
2N −1
n1 + 21 n2 + 21
     
X 1 1
+ cos 2π k+ cos 2π k+
2N 2 2N 2
k=N
N −1
n1 + 12 n2 + 21
     
X 1 1
= cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
+ cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
=2 cos 2π k+ cos 2π k+ .
2N 2 2N 2
k=0

This shows that

2 Digital sound and Discrete Fourier analysis 53

N −1 1
n2 + 21
     
X n1 + 2 1 1 N
cos 2π k+ cos 2π k+ = δn1 ,n2 ,
2N 2 2N 2 2
k=0
p
so that the vectors are orthogonal with lengths N/2.

Exercise 2.35: Modified Discrete Cosine Transform

The Modified Discrete Cosine Transform (MDCT) is defined as the N × (2N )-matrix
M with elements Mn,k = cos(2π(n + 1/2)(k + 1/2 + N/2)/(2N )). This exercise will take
you through the details of multiplication with this matrix.
a) Show that

r  
N (IV) 0 A
M= DCTN
2 B 0

where A and B are the (N/2) × N -matrices

 
· · · · · · 0 −1 −1 0 · · · · · ·
 .. .. .. .. .. .. .. .. 
A =  . . . . . . . .  = −(IN/2 )↔ −IN/2
 

 0 −1 · · · · · · · · · · · · −1 0 
−1 0 · · · · · · · · · · · · 0 −1
 
1 0 · · · · · · · · · · · · 0 −1
 0 1 · · · · · · · · · · · · −1 0 
B =  . . . . . . . .  = IN/2 −(IN/2 )↔ ,
 

 .. .. .. .. .. .. .. .. 
· · · · · · 0 1 −1 0 · · · · · ·

where where A↔ is the matrix A with the columns reversed. Due to this, any Type-IV
DCT algorithm can be used to compute the MDCT.

Solution.
(IV)
Clearly, columns 0, . . . , N/2 − 1 of the MDCT are columns N/2, . . . , N − 1 of DCTN .
For the remaining columns, note first that, for 0 ≤ k < N , the properties

cos(2π(n + 1/2)((2N − 1 − k) + 1/2)/(2N )) = − cos(2π(n + 1/2)(k + 1/2)/(2N ))

cos(2π(n + 1/2)((k + 2N ) + 1/2)/(2N )) = − cos(2π(n + 1/2)(k + 1/2)/(2N ))

are easy to verify. From the first property it follows that columns N/2, . . . , 3N/2−1 of the
(IV)
MDCT are columns N − 1, N − 2, . . . , 0 of the DCTN , with a sign change (they occur
in opposite order). From the second property, it follows that columns 3N/2, . . . , 2N − 1
(IV)
of the MDCT are columns 0, . . . , N/2 − 1 of DCTN , with a sign change. This means
that, if y is a vector of length 2N , the MDCT of y can be written as
54 2 Digital sound and Discrete Fourier analysis

−y3N/2 − y3N/2−1

−y3N/2+1 − y3N/2−2 
..
 
 
r 
 . 

N (IV)  −y 2N −1 − y N

.
DCTN  
2  y 0 − y N −1



 y 1 − y N/2+1


 .. 
 . 
yN/2−1 − yN/2
q q
N 2
The factor 2 was added since was added in front of the cosine-matrix in order to
N
 
(IV) 0 A
make DCTN orthogonal. The result now follows by noting that we can write y
B 0
for the vector on the right hand side, with A and B as defined in the text of the exercise.
b) The MDCT is singular, since it is not a square matrix. We will show here that it
still can be used in connection with non-singular transformations. We first define the
IMDCT as the matrix M T /N . Transposing what we obtained in a) gives

0 BT
 
1 (IV)
√ DCTN
2N AT 0

for the IMDCT, which thus also has an efficient implementation. Show that if

x0 = (x0 , . . . , xN −1 ) x1 = (xN , . . . , x2N −1 ) x2 = (x2N , . . . , x3N −1 )

and

   
x0 x1
y0,1 = M y1,2 = M
x1 x2

(i.e. we compute two MDCT’s where half of the data overlap), then
2N −1 N −1
x1 = {IMDCT(y0,1 )}k=N + {IMDCT(y1,2 )}k=0 .
Thus, even though the MDCT itself is singular, the input can still be recovered from
two overlapping IMDCT’s.

Solution.

Applying the MDCT first, and then the IMDCT, gives us the matrix

0 BT BT B 0
    
1 0 A 1
=
2 AT 0 B 0 2 0 AT A

Note that
2 Digital sound and Discrete Fourier analysis 55

IN/2 (IN/2 )↔ −(IN/2 )↔

   
IN/2
AT A = BT B = .
(IN/2 )↔ IN/2 −(IN/2 )↔ IN/2

Inserting this in the above gives

−(IN/2 )↔
 
IN/2 0 0
↔ 1 IN − (IN )↔
 
1
−(IN/2 ) IN/2 0 0  0
= .
IN/2 (IN/2 )↔  2 IN + (IN )↔

2 0 0 0
0 0 (IN/2 )↔ IN/2
 
x0
Assume now that we have computed the MDCT of = (x0 , . . . , x2N −1 ), and of
  x1
x1
= (xN , . . . , x3N −1 ). Performing the IMDCT on these two thus gives
x2
1 x0 − xrev 1 x1 − xrev
   
0 1
and
2 x1 + xrev
1 2 x2 + xrev
2

Adding the second component of the first and the first component of the second gives
x1 , which proves the result.

Exercise 2.36: Trick for reducing the number of

multiplications with the DCT

In this exercise we will take a look at a small trick which reduces the number of
multiplications we need for DCT algorithm from Theorem 27 in the book. This exercise
does not reduce the order of the DCT algorithms, but we will see in Exercise 2.37 how
the result can be used to achieve this.
a) Assume that x is a real signal. Equation (2.26) in the book said that

 n   n 
yn = cos π <((DFTN x(1) )n ) + sin π =((DFTN x(1) )n )
2N
 n  2N
 n 
yN −n = sin π <((DFTN x(1) )n ) − cos π =((DFTN x(1) )n )
2N 2N
for the n’th and N − n’th coefficient of the DCT. This can also be rewritten as

   n 
yn = <((DFTN x(1) )n ) + =((DFTN x(1) )n ) cos π
 n   n  2N
(1)
− =((DFTN x )n )(cos π − sin π )
 2N 2N  n 
yN −n = − <((DFTN x(1) )n ) + =((DFTN x(1) )n ) cos π
 n   n  2N
(1)
+ <((DFTN x )n )(sin π + cos π ).
2N 2N
Explain that the first two equations require 4 multiplications to compute yn and yN −n ,
and that the last two equations require 3 multiplications to compute yn and yN −n .
56 2 Digital sound and Discrete Fourier analysis

Solution.

In the last two equations, the first summand is the same, so that one multiplication is
needed. Each of the other lines require one more multiplication, so that we end up with
a total of three. That the first two equations require 4 multiplications is obvious.
b) Explain why the trick in a) reduces the number of additional multiplications in a
DCT from 2N to 3N/2.

Solution.

Since there are N/2 pairs on the form (yn , yN −n ), the number of multiplications is
reduced from 4 · N/2 = 2N to 3 · N/2 = 3N/2.
c) Explain why the trick in a) can be used to reduce the number of multiplications in
an IDCT with the same number.

Solution.

In matrix form the two first two equations above can be written
n n



<((DFTN x(1) )n )
   
yn cos π 2N sin π 2N
= n

n

.
yN −n sin π 2N − cos π 2N =((DFTN x(1) )n )
This system is easily inverted to
n n



<((DFTN x(1) )n )
   
cos π 2N sin π 2N yn
= n

n

.
=((DFTN x(1) )n ) sin π 2N − cos π 2N yN −n
Since this system has the same form, the same trick can be used to reduce the number of
multiplications in an IDCT with the same number. After solving this system, an IDFT
is applied to recover x(1) .
d) Show that the penalty of the trick we here have used to reduce the number of
multiplications, is an increase in the number of additions from N to 3N/2. Why can
this trick still be useful?

Solution.

The first two equations used 2 additions, while the next two used 3. The trick in this
exercise therefore does not change the total number of operations. This trick may still
be useful, however, if the CPU spends more time on computing a multiplication than
an addition.
2 Digital sound and Discrete Fourier analysis 57

Exercise 2.37: An efficient joint implementation of the DCT

and the FFT

In this exercise we will explain another joint implementation of the DFT and the DCT,
which has the benefit of a lower multiplication count, at the expense of a higher addition
count. It also has the benefit that it is specialized to real vectors, with a very structured
implementation (this may not be the case for the quickest FFT implementations, as
one often sacrifices clarity of code in pursuit of higher computational speed). a) of this
exercise can be skipped, as it is difficult and quite technical. For further details of the
algorithm the reader is referred to [5].
a) Let y = DFTN x be the N -point DFT of the real vector x. Show that


 <((DFTN/2 x(e) )n ) + (CN/4 z)n 0 ≤ n ≤ N/4 − 1
<(yn ) = <((DFTN/2 x(e) )n ) n = N/4
<((DFTN/2 x(e) )n ) − (CN/4 z)N/2−n N/4 + 1 ≤ n ≤ N/2 − 1


 =((DFTN/2 x(e) )n ) n=0
=(yn ) = =((DFTN/2 x(e) )n ) + (CN/4 w)N/4−n 1 ≤ n ≤ N/4 − 1
=((DFTN/2 x(e) )n ) + (CN/4 w)n−N/4 N/4 ≤ n ≤ N/2 − 1


where x(e) is as defined in Theorem 14 in the book, where z, w ∈ RN/4 defined by

zk = x2k+1 + xN −2k−1 0 ≤ k ≤ N/4 − 1,

k
wk = (−1) (xN −2k−1 − x2k+1 ) 0 ≤ k ≤ N/4 − 1,

Explain from this how you can make an algorithm which reduces an FFT of length N
to an FFT of length N/2 (on x(e) ), and two DCT’s of length N/4 (on z and w). We
will call this algorithm the revised FFT algorithm.

Solution.

Taking real and imaginary parts in equation (2.11) in the book for the FFT algorithm
we obtain

 
<(yn ) = < (DFTN/2 x(e) )n + <((DN/2 DFTN/2 x(o) )n
 
=(yn ) = = (DFTN/2 x(e) )n + =((DN/2 DFTN/2 x(o) )n ,

These equations explain the first terms <((DFTN/2 x(e) )n ) and =((DFTN/2 x(e) )n ) on
the right hand sides in
the equations for <(yn ) and
=(yn ). It remains to rewrite
< (DN/2 DFTN/2 x(o) )n and = (DN/2 DFTN/2 x(o) )n so that the remaining terms on
the right hand sides can be seen. Let us first consider the equation for the real part with
0 ≤ n ≤ N/4 − 1. In this case we can write
58 2 Digital sound and Discrete Fourier analysis

<((DN/2 DFTN/2 x(o) )n )

   
N/2−1 N/2−1
1
X X
= < e−2πin/N (x(o) )k e−2πink/(N/2)  = <  (x(o) )k e−2πin(k+ 2 )/(N/2) 
k=0 k=0
N/2−1
n(k + 12 )
X  
= (x(o) )k cos 2π
N/2
k=0
N/4−1
n(k + 21 )
X  
(o)
= (x )k cos 2π
N/2
k=0
N/4−1
n(N/2 − 1 − k + 21 )
X  
(o)
+ (x )N/2−1−k cos 2π
N/2
k=0
N/4−1
!
X
(o) (o) n k + 12
= ((x )k + (x )N/2−1−k ) cos 2π
N/2
k=0
N/4−1
!
n k + 21


X
= zk cos 2π ,
N/2
k=0

where we have used that cos is periodic with period 2π, that cos is symmetric, and
where z is the vector defined in the text of the theorem. When 0 ≤ n ≤ N/4 − 1 this
can also be written as
N/4−1
X
(CN/4 )n,k zk = (CN/4 z)n ,
k=0

This proves the first formula in the equation for <(yn ).

For N/4 + 1 ≤ n ≤ N/2 − 1, everything above is valid, except for that cos(2πn(k +
1/2)/(N/2)) are not entries in the matrix CN/4 , since n is outside the legal range of the
indices. However, N/2 − n is now a legal index in CN/4 , and using that
N

!
n(k + 21 ) k + 21


2 −n
 
cos 2π = − cos 2π ,
N/2 N/2

we arrive at −(CN/4 z)N/2−n instead, and this proves the third formula in the equation
for <(yn ). For the case n = N4 all the cosine entries in the sum are zero, and this
completes the proof for the equation for <(yn ).
For the imaginary part, using that sin is periodic with period 2π, and that sin is
anti-symmetric, analogous calculations as above give

N/4−1
!
(o)
X
(o) (o) n k + 21
=((DN/2 DFTN/2 x )n ) = ((x )N/2−1−k − (x )k ) sin 2π .
N/2
k=0

Using that
2 Digital sound and Discrete Fourier analysis 59

n(k + 12 ) n(k + 21 ) (N/4 − n)(k + 21 )

     
π
sin 2π = cos − 2π = cos 2π − kπ
N/2 2 N/2 N/2
1 
(N/4 − n)(k + 2 )

= (−1)k cos 2π ,
N/2

the above equation can be rewritten as

N/4−1
(N/4 − n)(k + 21 )
X  
((x(o) )N/2−1−k − (x(o) )k )(−1)k cos 2π
N/2
k=0
N/4−1
(N/4 − n)(k + 21 )
X  
= wk cos 2π ,
N/2
k=0

where w is the vector defined as in the text of the theorem. When n = 0 this is 0
since all the cosines entries are zero. When 1 ≤ n ≤ N/4 this is (CN/4 w)N/4−n , since
cos(2π(N/4 − n)(k + 1/2)/(N/2)) are entries in the matrix CN/4 . This proves the second
formula in the equation for =(yn ).
For N/4 ≤ n ≤ N/2 − 1 we can use that cos(2π(N/4 − n)(k + 1/2)/(N/2)) =
cos(2π(n − N/4)(k + 1/2)/(N/2)), which is an entry in the matrix DCTN/4 as well, so
that we get (CN/4 z)n−N/4 . This also proves the third formula in the equation for =(yn ),
and the proof is done.
a) says nothing about the coefficients yn for n > N2 . These are obtained in the same
way as before through symmetry. a) also says nothing about yN/2 . This can be obtained
with the same formula as in Theorem 14 in the book.
Let us now compute the number of arithmetic operations our revised algorithm
needs. Denote by the number of real multiplications needed by the revised N -point FFT
algorithm
b) Explain from the algorithm in a) that

MN = 2(MN/4 + 3N/8) + MN/2 AN = 2(AN/4 + 3N/8) + AN/2 + 3N/2

Hint.

3N/8 should come from the extra additions/multiplications (see Exercise 2.36) you need
to compute when you run the algorithm from Theorem 27 in the book for CN/4 . Note
also that the equations in a) require no extra multiplications, but that there are six
equations involved, each needing N/4 additions, so that we need 6N/4 = 3N/2 extra
additions.

Solution.

Two of the six mentioned equations which add to the additions are the ones for computing
zk and wk from the x. The other four come from the formulas for <(yn ) and =(yn ),
except for the two formulas for n = 0 and n = N/4. MN/4 + 3N/8 and AN/4 + 3N/8
come from the number of operations needed to compute DCTN/4 by reducing it to a
DF TN/4 as in the previous exercise. Two such reductions are needed.
60 2 Digital sound and Discrete Fourier analysis

c) Explain why xr = M2r is the solution to the difference equation

xr+2 − xr+1 − 2xr = 3 × 2r ,

and that xr = A2r is the solution to

xr+2 − xr+1 − 2xr = 9 × 2r .

and show that the general solution to these are xr = 12 r2r + C2r + D(−1)r for multipli-
cations, and xr = 32 r2r + C2r + D(−1)r for additions.

Solution.

Inserting N = 2r and xr = MN in MN = 2(MN/4 + 3N/8) + MN/2 gives

xr = 2(xr−2 + 3 · 2r /8) + xr−1 .

Inserting r + 2 for r this is easily reduced to xr+2 − xr+1 − 2xr = 3 × 2r . The case
for additions is deduced similarly. We see that the characteristic equations here are
the same as those for the split radix-algorithm, so that for a particular solution we
(p)
should try xr = Ar2r here as well (since also the right hand side is a power with the
same base as for split-radix). It is easily shown that, for multiplications the particular
(p) (p)
solution obtained is xr = 12 r2r , and for multiplications xr = 32 r2r . Inserting the
1
solution to the homogeneous equation gives xr = 2 r2 + C2 + D(−1)r for the number
r r

of multiplications, and xr = 23 r2r + C2r + D(−1)r for additions.

d) Explain why, regardless of initial conditions to the difference equations, MN =
O 12 N log2 N and AN = O 32 N log2 N both for the revised FFT and the revised

DCT. The total number of operations is thus O(2N log2 N ), i.e. half the operation count
of the split-radix algorithm. The orders of these algorithms are thus the same, since we
here have adapted to read data.

Solution.

Regardless of the initial conditions in c), i.e. of C and D, r2r will be the dominating
term in the solution. Since r2r = N log2 (N ), the result follows. The revised DCT has
the same order as the revised FFT, since they differ only in 3N arithmetic operations.
e) Explain that, if you had not employed the trick from Exercise 2.36, we would instead
have obtained MN = O 32 log2 N , and AN = O 43 log2 N , which equal the orders for

the number of multiplications/additions for the split-radix algorithm. In particular, the

order of the operation count remains the same, but the trick from Exercise 2.36 turned
a bigger percentage of the arithmetic operations into additions.

Solution.

Without the trick we would instead have

MN = 2(MN/4 + N/2) + MN/2 AN = 2(AN/4 + N/4) + AN/2 + 3N/2

2 Digital sound and Discrete Fourier analysis 61

rather than the

MN = 2(MN/4 + 3N/8) + MN/2 AN = 2(AN/4 + 3N/8) + AN/2 + 3N/2

from before, and solving these as above gives the stated solution. But note that for both
the total operation count RN = MN + AN satisfies

RN = 2(RN/4 + 3N/4) + RN/2 + 3N/2

so the total number of operations is of the same order.
The algorithm we here have developed is constructed from the beginning to apply
for real data only. Another advantage of the new algorithm is that it can be used to
compute both the DCT and the DFT.

Exercise 2.38: Implementation of the IFFT/IDCT

In Exercise 2.37 we did not write down corresponding algorithms for the revised IFFT
and IDCT algorithms. Use the equations for <(yn ) and =(yn ) in that exercise to show
that
a)

<(yn ) − <(yN/2−n ) = 2(CN/4 z)n

=(yn ) + =(yN/2−n ) = 2(CN/4 w)N/4−n

for 1 ≤ n ≤ N/4 − 1. Explain how one can compute z and w from this using two IDCT’s
of length N/4.

Solution.

For 1 ≤ n ≤ N/4 − 1 the formulas in Exercise 2.37 said that

<(yn ) = <((DFTN/2 x(e) )n ) + (CN/4 z)n

<(yN/2−n ) = <((DFTN/2 x(e) )N/2−n ) − (CN/4 z)n
=(yn ) = =((DFTN/2 x(e) )n ) + (CN/4 w)N/4−n
=(yN/2−n ) = =((DFTN/2 x(e) )N/2−n ) + (CN/4 w)N/4−n .

Subtracting the second equation from the first and using conjugate symmetry in
DFTN/2 x(e) gives the first equation. Adding the two last equations and using the
same conjugate symmetry gives the second equation.
In an IFFT the yn are the known quantities, so that the values of the right hand
sides are known. z and w can then be computed with the help of IDCT’s of length N/4.
b)
62 2 Digital sound and Discrete Fourier analysis

<(yn ) + <(yN/2−n ) = 2<((DFTN/2 x(e) )n )

=(yn ) − =(yN/2−n ) = 2=((DFTN/2 x(e) )n ),

and explain how one can compute x(e) from this using an IFFT of length N/2.

Solution.

Follows in the same way by adding the two first equations, and subtracting the fourth
equation from the third. As above DFTN/2 x(e) is known since the yn are known, so that
x(e) can be computed with the help of an N/2-point IFFT. Combining a) and b), z, w,
and x(e) can be computed with the help of two length N/4 IDCT’s and one N/2-point
IFFT. It is straightforward to reconstruct x from z, w, and x(e) .
Chapter 3
Discrete time filters

Exercise 3.2: Some linear operations which are not filters

a) In Example 1.2 in the book we looked at time reversal as an operation on digital

sound. In RN this can be defined as the linear mapping which sends the vector ek to
eN −1−k for all 0 ≤ k ≤ N − 1. Write down the matrix for time reversal, and explain
from this why time reversal is not a filter.

Solution.

The matrix for time reversal is the matrix

 
0 0 ··· 0 1
0 0 · · · 1 0
 
 .. .. .. .... 
. . . . .
 
0 1 · · · 0 0
1 0 ··· 00
This is not a circulant Toeplitz matrix, since all diagonals assume the values 0 and 1, so
that they are not constant on each diagonal. Time reversal is thus not a filter.
b) Consider the linear mapping S which keeps every second component in RN ,
i.e. S(e2k ) = e2k , and S(e2k−1 ) = 0. Write down the matrix for this, and explain
why this operation is not a filter.

Solution.

The matrix for the operation which keeps every second component is
 
1 0 ··· 0 0
0 0 · · · 0 0
 
 .. .. .. .. .. 
. . . . . ,
 
0 0 · · · 1 0
0 0 ··· 0 0

63
64 3 Discrete time filters

where 1 and 0 are repeated in alternating order along the main diagonal. Since the
matrix is not constant on the main diagonal, it is not a circulant Toeplitz matrix, and
hence not a filter.

Exercise 3.3: Eigenvectors and eigenvalues

Consider the matrix

 
4 1 3 1
1 4 1 3
S=
3
.
1 4 1
1 3 1 4
a) Compute the eigenvalues and eigenvectors of S using the results of this section. You
should only need to perform one DFT in order to achieve this.

Solution.

The eigenvalues of S are 1, 5, 9, and are found by computing a DFT of the first column.
The eigenvectors are the Fourier basis vectors. 1 has multiplicity 2.
b) Verify the result from a) by computing the eigenvectors and eigenvalues the way you
taught in your first course in linear algebra. This should be a much more tedious task.

Solution.

The characteristic equation will be a fourth degree polynomial. One usually would guess
two of the roots, and then use polynomial division to find a second degree polynomial
which can be solved. To find the eigenvectors corresponding to λ = 1, compute
     
3131 1 3 1 3 1010
1 3 1 3 0 −8 0 −8 0 1 0 1
S − λI = 3 1 3 1 ∼ 0 0 0 0  ∼ 0 0 0 0 .
    

1313 0 0 0 0 0000
This means that the general eigenvector for λ = 1 takes the form
   
−1 0
0 −1
x3 
 1  + x4  0  .
  

0 1
The two vectors here clearly give an orthonormal basis for the eigenspace for λ = 1.
Note that φ1 = 12 (i, −1, −i, 1) and φ3 = 12 (−i, −1, i, 1), and that the vectors above can
be obtained by adding and subtracting these, so that φ1 and φ3 span the eigenspace of
λ = 1. The computations for the other eigenvalues are similar.
c) Use a computer to find the eigenvectors and eigenvalues of S also. For some reason
some of the eigenvectors seem to be different from the Fourier basis vectors, which you
would expect from the theory in this section. Try to find an explanation for this.
3 Discrete time filters 65

Solution.

The computer uses some numerical algorithm to find the eigenvectors. However, eigen-
vectors may not be unique, so you have no control on which eigenvectors it actually
selects. In particular, here the eigenspace for λ = 1 has dimension 2, so that any linear
combination of the two eigenvectors from this eigenspace also is an eigenvector.

Exercise 3.4: Connection between continuous time- and

discrete time filters

Let s be a continuous time filter with frequency response λs (ν), and assume that
f ∈ VM,T (so that also g = s(f ) ∈ VM,T ). Let

x = (f (0 · T /N ), f (1 · T /N ), . . . , f ((N − 1)T /N ))
z = (g(0 · T /N ), g(1 · T /N ), . . . , g((N − 1)T /N ))

be vectors of N = 2M + 1 uniform samples from f and g. Show that the operation

S : x → z (i.e. the operation which sends the samples of the input to the samples of
the output) is well-defined on RN , and corresponds to a filter with frequency response
λS,n = λs (n/T ).
This explains how we can implement the restriction of a continuous time filter to the
Fourier spaces, by means of sampling, a discrete time filter, and interpolation. This also
states that any discrete time filter can be expressed in terms of a continuous time filter.

Solution.

With N = 2M + 1 we know that f ∈ VM,T is uniquely determined from x. This means

that g = s(f ) also is uniquely determined from x, so that z also is uniquely determined
from x. The operation S : x → z is therefore well-defined on RN .
Clearly also s(e2πint/T ) = λs (n/T )e2πint/T . Since the samples of e2πint/T is the vector
2πikn/N
e , and the samples of λs (n/T )e2πint/T is λs (n/T )e2πikn/N , the vector e2πikn/N
is an eigenvector of S with eigenvalue λs (n/T ). Clearly then S corresponds to a filter
with frequency response satisfying λS (2πn/N ) = λs (n/T ).

Exercise 3.5: Replacing a circular convolution with a longer

one

Let s, x ∈ RN . For M ≥ 2N − 1 define the vectors

s̃ = (s0 , s1 , ..., sN −1 , 0, ..., 0, s1 , ..., sN −1 )T

x̃ = (x0 , x1 , ..., xN −1 , 0, ..., 0)T
66 3 Discrete time filters

in RM (in s̃, M − (2N − 1) zeros have been inserted in the middle). Show that the first
N elements of s̃ ~ x̃ equal s ~ x, so that a circular convolution can be computed as part
of a longer circular convolution. A good strategy may now be to choose M as a power
of two, since circular convolution can be computed with the help of the DFT, and since
there are efficient DFT algorithms when the number of points is a power of two.

Solution.

Circular convolution with s and s̃ correspond to N × N - and M × M circulant Toeplitz

matrices. Since x̃ has zero entries after the first N entries, the result would follow if
those two circulant Toeplitz matrices have equal upper left N × N corners. That this is
the case is straightforward to see.

Exercise 3.6: Replacing a sparse circular convolution with

shorter ones

Let s, x ∈ RN , and assume that s has at most k nonzero entries, gathered in one
segment.
a) Show that there exists a s̃ ∈ RM +k−1 and a number a so that, for any r,

(s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 ,

where

x̃ = (xa+r , xa+r+1 , ..., xa+r+M +k−2 ).

In other words, any set of M consecutive entries in s ~ x can be obtained from a circular
convolution of size M + k − 1. Thus, a circular convolution of size N can be computed
N
as M circular convolutions of size M + k − 1.

Solution.

Denote by S the N × N circulant Toeplitz matrix with first column equal to s. Assume
that the nonzero entries in the first row of S occur in the interval [a, a + k − 1]. Then
the nonzero entries in rows r, ..., r + M − 1 of S are in the columns from a + r to
a + r + M − 1 + k − 1 = a + r + M + k − 2, i.e. the interval a + [r, r + M + k − 2] (there
are M + k − 1 such columns). With

x̃ = (xa+r , xr+1 , ..., xa+r+M +k−2 ),

and viewing the submatrix of S from rows r, ..., r + M − 1 and columns from a + [r, r +
M + k − 2] as the first M rows of a (M + k − 1) × (M + k − 1)-circulant Toeplitz matrix
S̃ with first column s̃, we obtain immediately that

(s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 .

b) It is natural to implement the shorter circular convolutions in terms of the FFT. Use
this strategy with M + k − 1 = 2r to find the number of operations needed to compute
3 Discrete time filters 67

N
the M circular convolutions from a), and write a program which finds the r which gives
the lowest number of operations. Create a plot where k is plotted against the number of
operations corresponding to the optimal r, and also against the number of operations
required by a direct implementation of circular convolution.

Solution.

Recall that an N -point split-radix FFT requires 4N log2 N −6N +8 arithmetic operations.
With M + k − 1 = 2r for N this gives (4r − 6)2r + 8 operations for each of the shorter
convolutions. A 2r -point convolution implemented in terms of the FFT (or rather FFT,
multiplication in frequency, and IFFT) thus requires

2((4r − 6)2r + 8) + 6 · 2r = (8r − 6)2r + 16

operations. The factor 2 comes from computing both an FFT and an IFFT, and the
summand 6 · 2r comes from the multiplications in frequency (each complex multiplication
requires 4 real multiplications and 2 real additions, a total of 6 operations for each of
the 2r entries). In total forl all the mN/M (or rather dN/M e) shorter convolutions we
N
thus need ((8r − 6)2r + 16) 2r −k+1 operations. In a direct implementation, for each of
the N entries we need to make k complex multiplications and k − 1 complex additions.
Since each complex addition requires 2 real additions, this totals 6k + 2(k − 1) = 8k − 2
operations for each entry, so that the overall total is (8k − 2)N operations.
The following code finds the r which gives the minimum of ((8r −6)2r +16)/(2r −k +1)
for a fixed k, and then stores that minimum value. This is then repeated for all the k,
and compared with 8k − 2, as we deduced for a direct implementation.

kvals = 1:50;
opt_vals = kvals;
for k=kvals
rvals = ceil(log2(k)):50;
num_ops = ((8*rvals-6).*2.^rvals + 16)./(2.^rvals-k+1);
opt_vals(k) = min(num_ops);
end
plot(kvals,8*kvals-2,kvals,opt_vals)

Although this code does not plot the optimal value for r, this value is very much needed
as a guideline for implementations on how to split convolutions into shorter convolutions.
The reason that we iterate r from log2 k in the code is that k ≤ 2r secures that there is
room for all the filter coefficients inside the vector s̃ used in the smaller convolution.
From the resulting plot we see that when the number of filter coefficients gets above 5,
the FFT-based approach works better. When the number of filter coefficients is below
this, a direct implementation of the filter should be chosen.
When the number of points in an FFT gets high, one usually not cares about the
terms −6N + 8 from the exact operation count of the Split radix FFT, since they
are not dominant terms. In this exercise these terms really mattered, however, since
short FFT’s are in focus. If you had dropped the terms −6N + 8, you can verify that
((8r − 6)2r + 16)/(2r − k + 1) should be replaced by (8r + 6)2r /(2r − k + 1), and that a
new plot shows that at least 8 filter coefficients are needed in order for the FFT-based
approach to outperform the direct implementation.
68 3 Discrete time filters

Exercise 3.7: Dualities between circular convolution and

multiplication

Show that

IDF TN (λ1 ◦ λ2 ) = (IDF TN λ1 ) ~ (IDF TN λ2 )

1
DF TN (s1 ◦ s2 ) = (DF TN s1 ) ~ (DF TN s2 )
N
IDF TN (λ1 ~ λ2 ) = N (IDF TN λ1 ) ◦ (IDF TN λ2 )

Note that there is a factor in front of the last two equations, so that one must be a bit
careful here.

Solution.

Setting λ1 = DF TN s1 , λ1 = DF TN s2 , and taking the IDFT on both sides in

DF TN (s1 ~ s2 ) = (DF TN s1 ) ◦ (DF TN s2 ), it follows that

(IDF TN λ1 ) ~ (IDF TN λ2 ) = IDF TN (λ1 ◦ λ2 ),

which is the first equation. Conjugating both sides in this equation, and using that
IDF TN = N1 DF TN , we obtain that
1 1
(DF TN λ1 ) ~ (DF TN λ2 ) = DF TN (λ1 ◦ λ2 ).
N2 N
Conjugating again gives the second equation. The last equation follows by taking the
DFT on both sides of this.

Exercise 3.8: Finding sparse solutions to under-determined

linear systems

Let x ∈ CN be a vector where we know that at most s components are nonzero. When s
is small such vectors are called sparse. Denote the indices of these nonzero components
by S, and let y = DF TN x. In this exercise we will find a procedure for finding x from
the values y0 , ..., y2s−1 , i.e. we can recover all components of a sparse vector from an
under-determined set of measurements. This procedure is also called a Reed-Solomon
code. In some sense this parallels the sampling theorem (which in a similar way fills in
the values between the samples), with an assumption on the highest frequency replaced
by an assumption on sparsity. Many other results also exist on how to recover sparse
vectors, and these results give rise to the field of compressed sensing. See [3] for a review
on this field.
a) Let z be the vector with components zk = N1 n∈S 1 − e−2πin/N e2πik/N . Show
Q

that zk = 0 for k ∈ S, and also that x ◦ z = 0.

3 Discrete time filters 69

Solution.

This follows from that 1−e−2πik/N e2πik/N = 1−1 = 0 is a factor in zk when k ∈ S. Since
xk = 0 when k 6∈ S, and zk = 0 when k ∈ S, we have for all k that (x ◦ z)k = xk zk = 0.
b) Let w = DF TN z. Show that w0 = 1 and that wn = 0 for n > s. Conclude from
this, that x ◦ z = 0, and due the previous exercise that w1 , w2 , ..., ws must fulfill the
equations

ys + w1 ys−1 + · · · + ws y0 =0
ys+1 + w1 ys + · · · + ws y1 =0
.. .. .. ..
. . . . =0
y2s−1 + w1 y2s−2 + · · · + ws ys−1 =0

Solution.
Ps
If we multiply out the factors in zk we obtain 1 + r=1 cr e2πirk for some values cr .
PN
Since also z = n=1 wn e2πikn/N by the definition of the DFT, we see that w0 = 1 and
that wn = 0 for n > s. From the previous exercise, multiplication in time corresponds to
convolution in frequency, so that y ~ w = 0. The stated equation system corresponds to

(y ~ w)s = · · · = (y ~ w)2s−1 = 0.
where we inserted w0 = 1, wn = 0 for n > s.
We now have s equations with s unknowns. By construction this has a solution, but
there may not be a unique solution. In the last part of the exercise we will show how to
find x, regardless of which solution we choose.
c) Assume that v1 , v2 , ..., vs is any solution to the system in b), and extend this to a
vector v ∈ RN by setting v0 = 1, vs+1 = · · · = vN −1 = 0. Show that (v ~ y)n = 0 for s ≤
n ≤ 2s − 1, and conclude from the previous exercise that (DF TN ((IDF TN v) ◦ x))n = 0
for s ≤ n ≤ 2s − 1.

Solution.

For s ≤ n ≤ 2s − 1 we have that

N
X −1 s
X
(v ~ y)n = vk y(n−k) mod N = yn + vk y(n−k) mod N = 0.
k=0 k=1

Since DF TN ((IDF TN v) ◦ x) = N1 v ~ y from the previous exercise, the result follows.

The vector (IDF TN v) ◦ x has at most s nonzero components (since x has). If we
take the columns from DF TN with indices in S, and the rows with indices between s
and 2s − 1, the resulting s × s-matrix was shown in Exercise 2.17 to be non-singular.
d) Explain that ((IDF TN v) ◦ x)n = 0 for n ∈ S, and conclude that (IDF TN v) ◦ x = 0.
70 3 Discrete time filters

Solution.

The stated non-singular matrix applied to the components of (IDF TN v) ◦ x from S

produces zero. These components must thus also be zero, so that (IDF TN v) ◦ x have all
components in S equal to 0. Since x is zero outside S, it follows that (IDF TN v) ◦ x = 0.
e) In particular we must have that (IDF TN v)k = 0 for k ∈ S, so that
N −1 s
1 X 1 X
vn e2πikn/N = vn e2πikn/N = 0
N n=0 N n=0

for k ∈ S. This is a polynomial in e2πik/N of degree at most s. Explain how one can find
the set S from this polynomial.

Solution.

Simply apply an IDFT, and choose S as the set from the result with smallest coefficients.
Due to roundoff errors the coefficients will not be exactly zero, however.
f) Explain how you can recover x from y0 , ..., y2s−1 when the set S is known.

Solution.

The submatrix A from DFTN with columns from S, and rows from 0, ..., s−1 is again non-
singular. The values of x on S, denoted xs can be found by solving AxS = (y0 , ..., ys−1 ).
g) Write a function

recover_x(y, N)

which uses the procedure obtained by combining b), e), and f) to compute and return
x ∈ RN under the assumption that x has at most s nonzero components, and that y
contains the first 2s components in y = DF TN x. Test that the code works correctly on
a vector x of your own choosing.

Solution.

The function can be implemented as follows.

function x=recover_x(y, N)
s= size(y,1)/2;
A=zeros(s);
for k=1:s
A(:,s-k+1) = y(k:(k+s-1));
end
b=-y((s+1):(2*s));
w=A\b;
v = [1; w; zeros(N-(s+1),1)];
inds = find(abs(ifft(v)) <= 0.00001);
x_S = exp(-2*pi*i*((0:(s-1))’)*(inds-1)’/N)\y(1:s);
x = zeros(N,1);
x(inds)=x_S;
end
3 Discrete time filters 71

The following code test that the function manages to recover a sparse vector with only
four nonzero integer entries, from the 8 first Fourier coefficients.

N=32;
x = zeros(N,1);
x([3,5,9,12])=[1,4,3,2];
y=fft(x);
x=recover_x(y(1:8),N)

Exercise 3.9: Windows

We mentioned in Section 2.2 in the book that we obtain some undesirable effects in the
frequency representation when we restrict to a block of the signal. Assume that x ∈ RM ,
and that

w = {w0 , . . . , wN −1 } with N < M.

We call (w0 x0 , . . . , wN −1 xN −1 , 0, ..., 0) ∈ RM a windowed signal and w ∈ RN a window
of length N .
a) Use Exercise 3.7 to show that the DFT of the windowed signal is

1
(DF TM (w0 , ..., wN −1 , 0, ..., 0)) ~ (DF TM x)
M

Solution.

With s1 = (w0 , ..., wN −1 , 0, ..., 0), s2 = x we have that s1 ◦ s2 is the windowed signal.
The result now follows from the second property in Exercise 3.7.
If DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) is close to (M, 0, ..., 0), this will be close to y =
DF TM x. In other words, a good window should satisfy

DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0).

There is a loss when we pass from the signal to the windowed signal, since we can only
construct a DFT close to, not equal to, (M, 0, ..., 0). We will not go into techniques for
how to find values wi which are close to satisfying this, only evaluate three such in the
rest of the exercise.
b) The rectangular window is defined by w = {1, 1, . . . , 1}. Show that
N −1
X 1 − e−2πinN/M
(DF TM (1, ..., 1, 0, ..., 0))n = e−2πikn/M = ,
k=0
1 − e−2πin/M

and use this to check whether DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0)
72 3 Discrete time filters

Solution.
sin(πnN/M ) sin(N ω/2)
This has absolute value sin(πn/M ) . The yn thus lie on the curve sin(ω/2) , from which
one can see that DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0) ≈ (M, 0, ..., 0) is partially fulfilled.
The frequency response can be plotted as follows

N=32;
omega = linspace(0, 2*pi, 1000);
wd = [ones(1, N) zeros(1, 1000-N)];
plot(omega, abs(fft(wd)), ’k-’);

c) The Hamming window is defined by

wn = 2(0.54 − 0.46 cos(2πn/(N − 1))).

Make a plot of DF TM (w0 , w1 , ..., wN −1 , 0, ..., 0), and compare with (M, 0, ..., 0) to see if
the Hamming window is a good window.

Solution.

1
Figure 3.1 compares M (DF TN (w0 , w1 , ..., wN −1 , 0, ..., 0)) for the rectangular- and the
Hamming window for N = 32. The Hamming window is seen to differ from the rectangular
window in two ways:
• It has much smaller values away from 0,
• the width of the “main lobe” (i.e. the main structure at the center) is bigger.
As a consequence, windowing with the Hamming window reduces the contribution from
higher frequencies, but increases the contribution from the smallest frequencies. The
window coefficients for both windows are shown in Figure 3.2.

1.0 1.0
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0.0 3 2 1 0 1 2 3 0.0 3 2 1 0 1 2 3
1
Fig. 3.1 The values M(DF TN (w0 , w1 , ..., wN −1 , 0, ..., 0)) for the rectangular and Hamming win-
dows, which we considered for restricting to a block of the signal.

The frequency response can be plotted as follows

n = 0:(N-1);
wd = [ 2*(0.54-0.46*cos(2*pi*n/(N-1))) zeros(1, 1000-N)];
plot(omega, abs(fft(wd)), ’k-’);
3 Discrete time filters 73

2.0 2.0
1.5 1.5
1.0 1.0
0.5 0.5
0.0 0 5 10 15 20 25 30 0.0 0 5 10 15 20 25 30
Fig. 3.2 The coefficients of the rectangular and Hamming windows, which we considered for
restricting to a block of the signal.

d) The Hanning window is defined by

wn = 1 − cos(2πn/(N − 1)).
Repeat the analysis you did above for the rectangular and Hamming windows, for a
Hanning window for N = 32.
The Hanning window is used in the MP3 standard, where it is applied to blocks
which overlap, contrary to the non-overlapping block pattern we have used. After the
Hanning window has been applied, the MP3 standard applies an FFT to the windowed
signal in order to make a frequency analysis of that part of the sound.

Solution.

The frequency response can be plotted as follows

wd = [1-cos(2*pi*n/(N-1)) zeros(1,1000-N)];
plot(omega, abs(fft(wd)), ’k-’);

A good survey on the windows mentioned here, as well as other ones, can be found
in [4]. One lesson to be learned from this exercise is that the windowed signal may not
give a good frequency representation. One must therefore be careful when splitting a
sound into blocks, as this alters the frequency representation. You can use the function
forw_comp_rev_DFT from Example 2.5 in the book to experiment with this. This function
accepts a named parameter N , which can be used to split the DFT of a sound into
blocks of length N , and eliminate low frequencies before taking an IDFT to reconstruct
and listen to the new sound (see also Example 3.31 in the book).

Exercise 3.10: The multiplicative group of integers modulo N

In number theory it is known that, when N is a prime number, the numbers 1, ..., N − 1
is a group under multiplication modulo N . In particular this means that for each
1 ≤ a ≤ N − 1 there exists a unique 1 ≤ b ≤ N − 1 so that ab = 1 mod N (the
number 1 acts as a unit). b us then called the inverse of a, and we write b = a−1 (this is
not the same as 1/a!). Also, it is known that there exists an integer g so that the set
74 3 Discrete time filters

−2
{g q mod N }N
q=0 constitutes all numbers 1, ..., N − 1 (the order these numbers appear
in may have changed, though). g is called a generator for the group.
a) Find generators for the multiplicative group of integers modulo 11, 23 and 41. Are
the generators for each group unique?

Solution.

Even for small primes, one saves tedious computation here with an implementation. The
following code returns the smallest generator.

function g=find_smallest_generator(N)
g = 0;
for k=2:(N-1)
res = zeros(N-1,1);
val = 1;
for s=1:(N-1)
if res(val) ~= 0
break;
end
res(val) = s;
val = mod(val*k,N);
if s == N-1
g = k;
return;
end
end
end
end

The outer loop starts at 2, because the unit 1 is never a generator. If your run this
function for the stated primes,

find_smallest_generator(11)
find_smallest_generator(23)
find_smallest_generator(41)

the generators 2, 5, and 6, respectively, are found.

A generator may not be unique: In the next exercise we state that g −1 is a generator
whenever g is.
b) Write a function

reorder(x, g)

which, for a vector x of length N , and a generator g for the numbers 1, ..., N − 1, returns
the vector (xg0 , xg1 , ..., xgN −2 ).
In the next exercise you will see how the multiplicative group of integers modulo N
relates to circular convolution.

Solution.

The following code can be used.

function xnew=reorder(x, g)
N = length(x);
xnew = zeros(N-1,1);
val = 1;
3 Discrete time filters 75

for s=1:(N-1)
xnew(s) = x(val+1);
val = mod(val*g, N);
end
end

Exercise 3.11: The FFT algorithm when N is prime. Rader’s

algorithm

It is possible to show that, if g is a generator, then g −1 is also a generator. This means

−2
that {g −p mod N }N p=0 also constitute the numbers 1, ..., N − 1 (where g
−p
is defined as
−1 p
(g ) ).
Replacing n = g −p and k = g q in the FFT algorithm with N prime, we can rephrase
it as (we pull k = 0 out of the sum, since the number 0 is not an element in our group)

N
X −1
y0 = xk
k=0
N −2
X −(p−q)
yg−p = x0 + xgq e−2πig /N
, 0 ≤ p ≤ N − 2.
q=0

Define

a = (xg0 , xg1 , ..., xgN −2 ) ∈ RN −1

0 −1 −(N −2)
b = (e−2πig /N
, e−2πig /N
, ..., e−2πig /N
) ∈ RN −1
c = (yg0 , yg−1 , ..., yg−(N −2) ) ∈ RN −1

Explain that c = x0 + a ~ b, where x0 is added to every component in the vector.

This explains how to compute an N -point DFT (with N prime) using (N − 1)-point
circular convolution. Since a circular convolution can be computed using a DFT/IDFT
of the same size, so this method effectively reduces an N -point DFT to an (N − 1)-point
DFT. Since N − 1 is not prime, we can use the algorithm for the FFT for composite
N , to reduce the problem to DFT’s of lengths being smaller prime numbers. Another
possibility is to use Exercise 3.5 to replace the circular convolution with a longer one, of
length a power of two, effectively reducing the problem to a DFT of length a power of
two.

Solution.

Inserting the definitions of a, b, and c in the DFT formula above gives

N
X −2 N
X −2
cp = x 0 + aq bp−q = x0 + aq b(p−q) mod (N −1) = x0 + (a ~ b)p .
q=0 q=0
76 3 Discrete time filters

The formula follows. We used that bp−q = b(p−q) mod (N −1) , which follows from that
g N −1 equals the unit e: If this was not the case, we would have that g N −1 = g i for
some i > 0. This gives that g N −1−i = e, which would imply that g does not generate all
numbers from 1 to N − 1. The following code tests that Rader’s algorithm produces the
same result as an N -point FFT when N is prime.

N = 41;
x=rand(N,1);
g=find_smallest_generator(N);
a = reorder(x, g);
b = reorder(exp(-2*pi*i*(0:(N-1))/N), g); b(2:(N-1)) = flip(b(2:(N-1)));
c = cconv(a, b, N-1) + x(1);
c(2:(N-1)) = flip(c(2:(N-1)));
y = x;
y(1) = sum(x);
ind = 1;
for k = 1:(N-1)
y(ind+1) = c(k);
ind = mod(ind*g, N);
end
max(abs(fft(x)-y))

Here we used the built-in function cconv for circular convolution in MATLAB. Later
we will consider our own implementation of circular convolution, but this will have a
different assumption on the input. In the code above, also note that we reversed the
vectors b and c. This is because these used g −1 rather than g to generate their values.

Exercise 3.18: Passing between compact filter notation and

matrix form

a) Assume that a filter is defined by the formula

1 1 1 1
zn = xn+1 + xn + xn−1 + xn−2 .
4 4 4 4
Write down the corresponding compact filter notation, and the corresponding circulant
Toeplitz matrix when N = 8.

Solution.

Here we have that t−1 = 1/4, t0 = 1/4, t1 = 1/4, and t2 = 1/4. We now get that
s0 = t0 = 1/4, s1 = t1 = 1/4, and s2 = t2 = 1/4 (first formula), and sN −1 = s7 =
t−1 = 1/4 (second formula). This means that S = {= 1/4, 1/4, 1/4, 1/4}, and that the
corresponding circulant Toeplitz matrix is
 
11000011
1 1 1 0 0 0 0 1
 
1 1 1 1 0 0 0 0
 
1 0 1 1 1 1 0 0 0
S=   .
4 0 0 1 1 1 1 0 0
0 0 0 1 1 1 1 0
 
0 0 0 0 1 1 1 1
10000111
3 Discrete time filters 77

b) Given the circulant Toeplitz matrix

 
1 2 0 0
0 1 2 0
S=
0
.
0 1 2
2 0 0 1
Write down the corresponding compact filter notation.

Solution.

We obtain that S = {2, 1}.

Exercise 3.19: Composing two filters

Assume that the filters S1 and S2 have the frequency responses λS1 (ω) = 2 + 4 cos(ω),
λS2 (ω) = 3 sin(2ω).
a) Compute and plot the frequency response of the filter S1 S2 .

Solution.

We have that

λS1 S2 (ω) = λS1 (ω)λS2 (ω) = (2 + 4 cos(ω))(3 sin(2ω)).

The frequency response can be plotted as follows.

omega=linspace(-pi,pi,100);
plot(omega, (2+4*cos(omega)).*(3*sin(2*omega)));

b) Write down the filter coefficients tk for the filter S1 S2 .

Solution.

We have that

3 2iω
λS1 S2 (ω) = (2 + 4 cos(ω))(3 sin(2ω)) = (2 + 2eiω + 2e−iω ) (e − e−2iω )
2i
= −3i(1 + eiω + e−iω )(e2iω − e−2iω )
= −3i(e2iω − e−2iω + e3iω + eiω − e−iω − e−3iω ).

From this we see that S1 S2 = 3i{−1, −1, −1, 0, 1, 1, 1}.

78 3 Discrete time filters

Exercise 3.20: Factoring a filter

Factor the frequency response of the filter S = {1, 5, 10, 6} as the product of two
polynomials in e−iω , one of order 1 and one of order 2. Use this to factor S as a product
of two filters, one with two filter coefficients, and one with three filter coefficients.

Solution.

The polynomial 6x3 + 10x2 + 5x + 1 clearly has x = −1 as a root. With polynomial

division we obtain

6x3 + 10x2 + 5x + 1 = (x + 1)(6x2 + 4x + 1),

where the second factor doesn’t have any real roots. For the frequency response we thus
obtain

λS (ω) = 1 + 5e−iω + 10e−2iω + 6e−3iω = (e−iω + 1)(6e−2iω + 4e−iω + 1).

The first factor is the frequency response of S1 = {1, 1}, while the second factor is the
frequency response of S2 = {1, 4, 6}. It follows that S = S1 S2 .

Exercise 3.21: Plotting a simple frequency response

Let S be the filter defined by the equation

1 1 1 1
zn = xn+1 + xn + xn−1 + xn−2 ,
4 4 4 4
as in Exercise 3.18. Compute and plot (the magnitude of) λS (ω).

Solution.

The frequency response is

1 iω eiω (1 − e−4iω ) 1 sin(2ω)

λS (ω) = (e + 1 + e−iω + e−2iω ) = = e−iω/2 .
4 4(1 − e−iω ) 4 sin(ω/2)

Exercise 3.22: Adding two echo filters

Consider the two filters S1 = {1, 0, . . . , 0, c} and S2 = {1, 0, . . . , 0, −c}. Both of these
can be interpreted as filters which add an echo. Show that 12 (S1 + S2 ) = I. What is the
interpretation of this relation in terms of echos?
3 Discrete time filters 79

Solution.

The sum of two filters is again a filter, and the first column in the sum can be obtained
by summing the first columns in the two matrices. This means that the filter coefficients
in 12 (S1 + S2 ) can be obtained by summing the filter coefficients of S1 and S2 , and we
obtain
1
({1, 0, . . . , 0, c} + {1, 0, . . . , 0, −c}) = {1}.
2
This means that 12 (S1 + S2 ) = I, since I is the unique filter with e0 as first column. The
interpretation in terms of echos is that the echo from S2 cancels that from S1 .

Exercise 3.23: Filters with coefficients being powers

Assume that S = {1, c, c2 , . . . , ck }. Compute and plot λS (ω) when k = 4 and k = 8, for
c = 0.5. How do the choices of k and c influence the frequency response?

Solution.

The frequency response is

k
X 1 − ck+1 e−i(k+1)ω
cs e−isω = .
s=0
1 − ce−iω

It is straightforward to compute the limit as ω → 0 as ck (k + 1). This means that as we

increase k or c, this limit also increases. The value of k also dictates oscillations in the
frequency response, since the numerator oscillates fastest. When c = 1, k dictates how
often the frequency response hits 0.

Exercise 3.24: Convolution and polynomials

Compute the convolution of (1, 2, 1) with itself, and interpret the result in terms of a
product of polynomials. Compute also the circular convolution of (1, 2, 1) with itself.

Solution.

Set x = (1, 2, 1), and z = x ∗ x. We obtain that (only the terms contributing in the
convolution are listed).
80 3 Discrete time filters

z0 = x0 x0 = 1
z1 = x0 x1 + x1 x0 = 2 + 2 = 4
x2 = x0 x2 + x1 x1 + x2 x0 = 1 + 4 + 1 = 6
x3 = x1 x2 + x2 x1 = 2 + 2 = 4
x4 = x2 x2 = 1.

It follows that (1, 2, 1) ∗ (1, 2, 1) = (1, 4, 6, 4, 1). For the circular convolution we have

z0 = x0 x0 + x1 x2 + x2 x1 = 1 + 2 + 2 = 5
z1 = x0 x1 + x1 x0 + x2 x2 = 2 + 2 + 1 = 5
x2 = x0 x2 + x1 x1 + x2 x0 = 1 + 4 + 1 = 6.

It follows that (1, 2, 1) ~ (1, 2, 1) = (5, 5, 6).

Exercise 3.25: Execution times for convolution

Implement code which computes t ∗ x in the vectorized- and non-vectorized ways

described in Section 3.2.1 in the book (i.e. as a single loop over k or n with the other
variable vectorized, or a double loop). As your t, take k randomly generated numbers.
Compare execution times with the conv function, for different values of k. Present
the result as a plot where k runs along the x-axis, and execution times run along the
y-axis. Your result will depend on how vectorization is performed by the computing
environment.

Solution.

The code can look as follows.

[x, f_s] = audioread(’sounds/castanets.wav’);

x = x(:,1);
N= length(x);
kmax=100;
vals1 = zeros(1, kmax/10);
vals2 = zeros(1, kmax/10);
vals3 = zeros(1, kmax/10);
ind = 1;
for k=10:10:kmax
t = rand(k, 1);
tic;
conv(t, x);
vals1(ind) = toc;
z = zeros(N, 1);
tic;
for s = 1:k
z(k:N) = z(k:N) + t(s)*x((k-s+1):(N-s+1));
end
vals2(ind)=toc;
z = zeros(N, 1);
tic;
for n=k:N
3 Discrete time filters 81

for s = 1:k
z(n) = z(n) + t(s)*x(n-s+1);
end
end
vals3(ind)=toc;
ind = ind+1;
end
plot(10:10:kmax,log(vals1),10:10:kmax,log(vals2), 10:10:kmax, log(vals3));
legend(’conv’,’simple for’,’double for’)

Exercise 3.26: Implementing convolution in-place

Filters are often implemented in hardware, and hardware implementations have strong
limitations on the number of local variables. Assume that a filter t has n nonzero
coefficients, all known to the hardware, and that the vector x is input. Show that t ∗ x
can be implemented in-place (i.e. that t ∗ x can be computed and stored directly in x)
by using n − 1 local variables. You can assume that two local variables can be swapped
without using an additional local variable. It follows that, since any real convolution
can be split into real convolutions with three filter coefficients, only 2 local variables are
needed in order to compute any convolution in-place.

Solution.

Assume for simplicity that the nonzero filter coefficients are t0 , ..., tn−1 . We have that
x0 is needed in order to compute z0 , ..., zn−1 . Therefore, z0 can’t be stored into the
location of x0 until zn−1 has been computed. When we start the computation of zn−1 ,
z0 , ..., zn−2 must be stored in n − 1 local variables. The final contribution of x0 is with
x0 tn−1 in zn−1 . Now, multiply x0 in-place with tn−1 , and compute zn−1 in the same
memory location. Finally, swap the result with the local variable holding z0 . As a result,
z0 has replaced x0 in the input memory, and the local variables hold z1 , ..., zn−1 instead.
This is continued incrementally.
When the first nonzero filter coefficient is not t0 , it is clear that this procedure will
still work, but that the output will be a delayed version.

Exercise 3.27: Circular convolution when there is a different

number of coefficients with positive and negative indices

Assume that S = {t−E , . . . , t0 , . . . , tF }. Formulate a generalization of Proposition 9 in

the book for such filters, i.e. to filters where there may be a different number of filter
coefficients with positive and negative indices. You should only need to make some small
changes to the proof.

Solution.

We reproduce Figure 3.5 by placing the t−E , t−E+1 and so on in the first col-
umn, starting at the top. We then disregard the first and last E + F rows (not
82 3 Discrete time filters

the first and last 2L entries as in step 3). We can form the shorter vector x̃ =
(xN −F , · · · , xN −1 , x0 , · · · , xN −1 , x0 , · · · , xE−1 ) in step 1, before we proceed with the
convolution.

Exercise 3.28: Implementing circular convolution through

convolution

Implement a function which takes two parameters t and x, and uses Proposition 9 in the
book and convolution to compute the circular convolution of t = (t−L , . . . , t0 , . . . , tL )
and x. Write the function so that, if x has many columns, the circular convolution with
each column is computed.

Hint.

The function filter_impl in the library is slightly more general than the one you are
asked to implement here, and can serve as a template for your implementation.

Solution.

The code can look like this:

function y=filter_impl(t, x)
L = (length(t) - 1)/2;
N = length(x);
y = [x((N - L + 1):N); x; x(1:L)];
y = conv(t, y);
y = y((2*L+1):(length(y)-2*L));

Exercise 3.29: Expressing convolution in terms of circular

convolution

Find a similar statement to that in Proposition 9 in the book, which explains how to
express convolution in terms of circular convolution.

Hint.

Add some zeros at the start and end of the vector. How many zeros do you need to add?

Solution.

Let x̃ be the vector where 2L zeros are placed at the end of x. It is then straightforward
to check that
3 Discrete time filters 83

(t−L , ..., tL , 0, ..., 0) ~ x̃ = t ∗ x.

The circular convolution here corresponds to the N + 2L cirulant Toeplitz matrix with
compact filter notation {t−L , ..., tL }.

Exercise 3.36: Applying low-pass- and high-pass filters

deduced from Pascal’s triangle to the audio sample file

a) Write code where you apply the low-pass and high-pass filters in examples 3.34
and 3.35 in the book to the audio sample file, and verify that the sounds you get are
the same as in these examples. How high must k be in order for you to hear difference
from the actual sound? How high can you choose k and still recognize the sound at all?
If you solved Exercise 3.28 you can also use the function filter_impl to perform the
filtering, rather than using convolution (which, as mentioned, discards circularity).

Solution.

The code can look like this, when we apply low-pass filters.

z = x(:, 1);
for kval=1:k
z = conv(z,[1/2 1/2]);
end

playerobj=audioplayer(z, f_s);
playblocking(playerobj);

If we apply high-pass filters instead, the code can look like this.

z = x(:, 1);
for kval=1:k
z = conv(z,[1/2 -1/2]);
end

playerobj=audioplayer(z, f_s);
playblocking(playerobj);

b) In your code, it is not necessary to scale the values after applying the low-pass or
high-pass filters so that values fit inside [−1, 1]. Explain why this is the case.

Solution.

The low-pass filters compute weighted averages of the input samples. Therefore, as long
as the input values are inside the legal range[−1, 1], the output values will as well. Since
the filter coefficients sum to one it is easy to see that also the high-pass filters produce
values in the legal range.
84 3 Discrete time filters

Exercise 3.37: Constructing a high-pass filter

Consider again Example 3.31 in the book. Find an expression for a filter so that only
frequencies so that |ω − π| < ωc are kept, i.e. the filter should only keep angular
frequencies close to π (i.e. here we construct a high-pass filter).

Solution.

A filter which keeps frequencies so that |ω − π| < ωc is the high-pass filter corresponding
to the filter constructed in Example 3.31 in the book. We therefore simply need to add
an alternating sign to the filter coefficients from that example.

Exercise 3.38: Combining low-pass and high-pass filters

In this exercise we will investigate how we can combine low-pass and high-pass filters to
produce other filters
a) Assume that S1 and S2 are low-pass filters. Is S1 S2 also a low-pass filter? What if
both S1 and S2 are high-pass filters?

Solution.

We have that λS1 S2 (0) = λS1 (0)λS2 (0) which is “large” when the two factors are. Also,
λS1 S2 (π) = λS1 (π)λS2 (π) is approximately 0 when the two factors are. Therefore S1 S2
is also low-pass. That S1 S2 is high-pass when the two factors are follows similarly.
b) Assume that one of S1 , S2 is a high-pass filter, and that the other is a low-pass filter.
Is S1 S2 low-pass or high-pass?

Solution.

In this case one of λS1 (0) and λS2 (0) is large, and the other approximately zero. There
is then no telling if their product is large or approximately zero, so we can’t conclude
anything about the filter S1 S2 .

Exercise 3.39: Filters in the MP3 standard

Assume that tk are the filter coefficients of a filter S1 , and that S2 is the filter with filter
coefficients cos(kωc )tk , where ωc ∈ [0, π). Show that
1
λS2 (ω) =(λS1 (ω − ωc ) + λS1 (ω + ωc )).
2
In other words, when we multiply (modulate) the filter coefficients with a cosine, the
new frequency response can be obtained by shifting the old frequency response with ωc
3 Discrete time filters 85

in both directions, and taking the average of the two. We saw that the filters in the MP3
standard were constructed in this way. The collection of these filters are also called a
cosine modulated filter bank. It is also possible to construct DFT-modulated filter banks
where the shift in frequency is obtained by multiplying with a complex exponential
instead, but this produces filters with complex coefficients.

Solution.

We have that

X 1 X ikωc
λS2 (ω) = cos(kωc )tk e−ikω = (e + e−ikωc )tk e−ikω
2
k k
!
1 X
−ik(ω−ωc )
X
−ik(ω+ωc )
= tk e + tk e
2
k k
1
= (λS1 (ω − ωc ) + λS1 (ω + ωc )).
2

Exercise 3.40: Code example

a) Explain what the code below does, line by line.

[x, f_s] = audioread(’sounds/castanets.wav’);

[N, nchannels] = size(x);
z = zeros(N, nchannels);
for n=2:(N-1)
z(n,:) = 2*x(n+1,:) + 4*x(n,:) + 2*x(n-1,:);
end
z(1,:) = 2*x(2,:) + 4*x(1,:) + 2*x(N,:);
z(N,:) = 2*x(1,:) + 4*x(N,:) + 2*x(N-1,:);
z = z/max(abs(z));
playerobj=audioplayer(z, f_s);
playblocking(playerobj)

Comment in particular on what happens in the three lines directly after the for-loop,
and why we do this. What kind of changes in the sound do you expect to hear?

Solution.

In the code a filter is run on the sound samples from the file castanets.wav. Finally
the new sound is played. In the first two lines after the for-loop, the first and the last
sound samples in the filtered sound are computed, under the assumption that the sound
has been extended to a periodic sound with period N. After this, the sound is normalized
so that the sound samples lie in the range between −1 and 1. In this case the filter is
a low-pass-filter (as we show in b)), so that we can expect that that the treble in the
sound is reduced.
b) Write down the compact filter notation for the filter which is used in the code, and
write down a 5 × 5 circulant Toeplitz matrix which corresponds to this filter. Plot the
(continuous) frequency response. Is the filter a low-pass- or a high-pass filter?
86 3 Discrete time filters

Solution.

Compact filter notation for the filter which is run is {2, 4, 2}. A 5 × 5 circulant Toeplitz
matrix becomes
 
42002
 2 4 2 0 0
 
 0 2 4 2 0 .
 
 0 0 2 4 2
20024
The frequency response is λS (ω) = 2eiω + 4 + 2e−iω = 4 + 4 cos ω. It is clear that this
gives a low-pass filter.
c) Another filter is given by the circulant Toeplitz matrix
 
4 −2 0 0 −2
−2 4 −2 0 0 
 
 0 −2 4 −2 0  .
 
 0 0 −2 4 −2
−2 0 0 −2 4
Express a connection between the frequency responses of this filter and the filter from
b). Is the new filter a low-pass- or a high-pass filter?

Solution.

The frequency response for the new filter is

−2eiω + 4 − 2e−iω = 4 − 4 cos ω = 4 + 4 cos(ω + π) = λS (ω + π),

where S is the filter from the first part of the exercise. The new filter therefore becomes
a high-pass filter, since to add π to ω corresponds to swapping the frequencies 0 and π.
We could also here referred to Proposition 11 in the book, where we stated that adding
an alternating sign in the filter coefficients turns a low-pass filter into a high-pass filter
and vice versa.

Exercise 3.44: All-pass filters

All-pass filters are filters where |λS (ω)| = 1 for all ω. Such filters are uniquely determined
by the phase of the frequency response. Any all-pass FIR filter must clearly be on the
form αEd with |α| = 1. It is, however, possible to find other all-pass filters by considering
realizations in terms of difference equations.
a) If α is any real number, consider the filter S given by the realization

zn − αzn−1 = αxn − xn−1 .

Write down the frequency response of this realization, and show that it is an all-pass
filter.
3 Discrete time filters 87

Solution.
−iω
α−e
The frequency response is λS (ω) = 1−αe −iω , and it is straightforward to check that
−iω −iω
|α − e | = |1 − αe |, so that |λS (ω)| = 1.
b) A filter is said to be linear phase if the phase (argument) of the frequency response is
linear in ω, i.e. λS (ω) = Cei(rω+s) for some real r, s, and C. Any delay filter has linear
phase since λEd (ω) = e−idω . Show that S and the filter {−α2 , 2α, −1} have frequency
responses with the same phase, and explain why it follows from this that S has non-linear
phase as long as |α| = 6 1. Since delay filters have linear phase, these filters can not be
delays, and thus represent a new kind of all-pass filters.

Hint.

Consult Exercise 3.49 on linear phase filters.

Solution.

We have that

(α − e−iω )(1 − αeiω ) −α2 eiω + 2α − e−iω

λS (ω) = = ,
|1 − αe−iω |2 |1 − αe−iω |2
so that S and {−α2 , 2α, −1} have frequency responses with the same phase. Since
{−α2 , 2α, −1} has nonlinear phase, S has so as well.
c) Explain that S has an inverse, and that a realization of the inverse system is

−αzn + zn−1 = −xn + αxn−1 .

Solution.

Since |λS (ω)| = 1, we simply need to find a realization of the filter with frequency
response

α − eiω
λS (ω) = ,
1 − αeiω
which gives the realization

−αzn+1 + zn = −xn+1 + αxn .

Since only coefficients up to n for zn should be listed, a translation with 1 is needed,
and this yields the desired expression.
d) Explain that combining two all-pass filters as above with different values for α will
also give an all-pass filter, different from the ones we combined. This gives some flexibility
in how one can construct all-pass filters as well.
88 3 Discrete time filters

Solution.

Note first that, also in the context of IIR filters, composing two filters gives a new IIR
filter with frequency response equal to the multiplication of the two frequency responses.
To see why, let S1 and S2 be the two IIR filters

X X X X
ak zn−k = bk yn−k ck yn−k = dk xn−k .
0≤k≤M k 0≤k≤M k

They can be represented in matrix form as Az = By and Cy = Dx, where A, B, C, and

D are (infinite) Toeplitz matrices. These matrices also commute (even though they are
infinite), and there are only finitely many non-zeros in each row/column of the matrices.
we thus obtain

CAz = CBy = BCy = BDx.

which gives an expression for the combined filter. Since CA and BD again are Toeplitz
matrices, S1 S2 is also an IIR filter. It is also clear that it can be written as
X X
ek zn−k = fk yn−k ,
0≤k≤M k

with e = a ∗ c and f = b ∗ d, and we obtain that

∗ d)k e−iωk −iωk

d e−iωk
P P P
k (b k bk e Pk k −iωk = λS1 (ω)λS2 (ω),
λS1 S2 (ω) = P −iωk
= P −iωk
0≤k≤M (a ∗ c)k e k ak e k ck e

where we used that convolution in time corresponds to multiplication in frequency. When

S1 and S2 are all-pass filters, it follows that |λS1 S2 (ω)| = |λS1 (ω)λS2 (ω)| = |1||1| = 1,
so that S1 S2 also is an all-pass filter.
Finally, suppose now that we compose all-pass filters with parameters α1 and α2 .
The numerator in the frequency response of the combined filter has zeros in α1 and α2 ,
while for the denominator the zeros are 1/α1 and 1/α2 . Unless we choose α1 = 1/α2
(for which all terms in the frequency response will cancel), we thus will get a different
filter when they are combined.

Exercise 3.45: The IIR filter for the Karplus-Strong algorithm

Let us rewrite the difference equation in Exercise 1.12 as

1
zn − (zn−p + zn−(p+1) ) = 0
2
a) Explain that the frequency response of this realization is 0. Therefore, this realization
is only interesting if the initial conditions are chosen so that we don’t obtain a filter.

Solution.

The numerator in λS (ω) is zero, since all bk = 0. We therefore have that λS (ω) = 0 also.
3 Discrete time filters 89

b) Plot the zeros of the characteristic equation for p = 20. Is the filter BIBO stable?

Solution.

We have that a0 = 1, ap = ap+1 = −1/2, so that the characteristing equation is

PM −k
k=0 ak r = 1 − 12 r−p − 12 r−(p+1) . Its zeros can be plotted as follows.

p=20;
rts = roots([1 zeros(1, p-1) -1/2 -1/2]);
plot(real(rts), imag(rts), ’kx’)

We see from the resulting plot that the zeros are inside the unit circle, so that the filter
is BIBO stable.
c) What happens with the zeros of the characteristic equation when you increase p?
Attempt to explain why the resulting sound changes more slowly to a static sound when
you increase p.

Solution.

It seems that the absolute values of the zeros of the characteristic equation gets closer
and closer to 1 as p increases. This also means that their powers decreases slower to
zero, so that the resulting sound goes to zero somewhat more slowly.
d) A related IIR filter is
1
zn − (zn−p + zn−(p+1) ) = xn .
2
Plot the frequency response of this filter for p = 20. Include only values in the response
between 0 and 10.

Solution.

The following code can be used

p=20;
omega = linspace(0,2*pi,500);
plot(omega, abs(1./(1-0.5*exp(-omega*p*i)-0.5*exp(-omega*(p+1)*i))) );
axis([0,2*pi,0,10]);

Exercise 3.46: Direct proof of the orthogonality of the DCT

basis

The way we proved that the DCT basis was orthonormal was constructive in that it
built on the orthogonality of the DFT. In the literature many instead give a direct proof.
Give a direct proof of the orthonormalilty of the DCT basis using only trigonometric
identities.
90 3 Discrete time filters

Solution.
 
Let f be the vector with components fk = cos 2πn1 (k+1/2) 2N , and g the vector
 
2πn2 (k+1/2)
with components gk = cos 2N . If these vectors have length 2N , they are
clearly orthogonal when n
 1 6
= n ,
2 since both can be written as a sum of two ge-
2πn(k+1/2)
= 2 e2πn(k+1/2)/(2N ) + e−2πn(k+1/2)/(2N ) ), and
1


ometric series (i.e. cos 2N
since (e2πn/(2N ) )2N = 1. It is also straightforward to check that c2N −1−k = ck for
k = 0, ..., N − 1, and similarly for d. This implies that
2N
X −1 N
X −1 2N
X −1 N
X −1 N
X −1 N
X −1
0= ck dk = ck dk + ck dk = ck dk + ck d k = 2 ck dk .
k=0 k=0 k=N k=0 k=0 k=0

It follows that c and d are orthogonal also when viewed as vectors of length N . Since
the DCT basis vectors are scaled versions of these, they are orthogonal as well. √
If n = 0 the cosine vectors have all components
√ equal to one, so that it has length N .
This is the reason why a scaling with 1/ N is needed in d0 to achieve orthonormality.
We also have

N −1   N −1    
X
2 2πn(k + 1/2) 1 X 2πn(k + 1/2)
cos = cos +1
2N 2 N
k=0 k=0
N −1
N 1 X  2πin(k+1/2)/N  N
= + e + e−2πin(k+1/2)/N = ,
2 2 2
k=0

where it was easily seen that

p the two geometric series summed to zero. This explains
the scaling with the factor 2/N in the other DCT basis vectors.

Exercise 3.47: Writing down lower order Sr

Consider the averaging filter S = { 14 , 12 , 41 }. Write down the matrix Sr for the case when
N = 4.

Solution.

First we obtain the matrix S as

1 1 1

2 4 0 0 0 0 0 4
1 1 1

4 2 4 0 0 0 0 0
0 14 21 1
 
4 0 0 0 0

0 0 14 1 1
 
2 4 0 0 0

 
0 0 0 1 1 1 0 0
 4 2 4 
0 0 0 0 1 1 1
0
 4 2 4 
0 0 0 0 0 1 1 1
4 2 4
1 1 1
4 0 0 0 0 0 4 2
3 Discrete time filters 91

where we have drawn the boundaries between the blocks S1 , S2 , S3 , S4 . From this we
see that

1 1
0 0 0 14
   1 
2 4 0 0 4 00 0
1 1 1 0 0 0 0 0 0 00 0
S1 =  4 21 41 1
 S2 = 
0 0 0 0
 (S2 )↔ = 
0
.
0
4 2 4 00 0
1 1
0 0 14 1
2 4 0 0 0 0 00 4

From this we get

3 1

4 4 0 0
1 1 1 0
Sr = S1 + (S2 )↔ =  4 21 14
0 1 .

4 2 4
0 0 14 3
4

Exercise 3.48: Computing eigenvalues

Consider the matrix

 
2 1 0 0 0 0
1 1 1 0 0 0
 
1 0 1 1 1 0 0
S=  
30 0 1 1 1 0
0 0 0 1 1 1
0 0 0 0 1 2
a) Compute the eigenvalues and eigenvectors of S using the results of this section. You
should only need to perform one DFT or one DCT in order to achieve this.

Solution.

In c) we show that this indeed corresponds to a symmetric filter. We then know that
the DCT basis vectors are eigenvectors. The eigenvalues can be found on the diagonal
in DCTN S(DCTN )T . Note that it is not true that the eigenvalues can be computed by
taking a DCT on the first column, as is the case for filters and the DFT.
b) Use a computer to find the eigenvectors and eigenvalues of S also. What are the
differences from what you found in a)?

Solution.

The following code compares what you found in a) with a direct computation.

S = [2 1 0 0 0 0; ...
1 1 1 0 0 0; ...
0 1 1 1 0 0; ...
0 0 1 1 1 0; ...
0 0 0 1 1 1; ...
0 0 0 0 1 2];
92 3 Discrete time filters

idctmatr = idct(eye(6));
idctmatr % Eigenvectors
dct(S*idctmatr) % eigenvalues
[V, D]=eig(S);
V % Eigenvectors
D % eigenvalues

The two typically will list eigenvectors and eigenvalues in different order.
c) Find a filter T so that S = Tr . What kind of filter is T ?

Solution.

Writing down the circulant 12 × 12-matrix for the symmetric filter T = 13 {1, 1, 1} we
obtain the two upper blocks

   
1 1 0 0 0 0 0 0 0 0 0 1
1 1 1 0 0 0 0 0 0 0 0 0
   
1 0 1 1 1 0 0 1 0 0 0 0 0 0
T1 =   T2 =  .
30 0 1 1 1 0 30 0 0 0 0 0
0 0 0 1 1 1 0 0 0 0 0 0
0 0 0 0 1 1 1 0 0 0 0 0

From these expressions it is clear that Tr = T1 + T2↔ = S.

Exercise 3.49: Linear phase filters

For a symmetric (2N ) × (2N )-filter the filter coefficients have symmetry around N − 1/2.
Show that, if a filter instead has symmetry around d (where d may or may not be an
integer), then its frequency response takes the form λS (ω) = Ce−iωd , with C a real
number.
This means that the phase (argument) of the frequency response is −dω or π − dω,
depending on the sign of C. In other words, the phase is linear in ω. Such filters are
therefore called linear phase filters (see also Exercise 3.44). One can show that linear
phase filters also preserve symmetric extensions, but one has to re-define such extensions
to have another point of symmetry. An example of linear phase filters which are not
symmetric are low-pass filters where the coefficients are taken from odd rows in Pascal’s
triangle.

Solution.

If we gather the terms with indices d + k and d − k in the frequency response we obtain

td+k (e−iω(d+k) + e−iω(d−k) ) = 2e−iωd td+k cos(kω),

which has phase −dω. The sum of all such terms also have phase −dω. The claim follows
with C = 2td+k cos(kω).
3 Discrete time filters 93

Exercise 3.50: Component expressions for a symmetric filter

Assume that S = t−L , . . . , t0 , . . . , tL is a symmetric filter. Use equation (3.21) in the

book to show that zn = (Sx)n in this case can be split into the following different
formulas, depending on n:
a) 0 ≤ n < L:
n
X L
X
zn = t0 xn + tk (xn+k + xn−k ) + tk (xn+k + xn−k+N ).
k=1 k=n+1

Solution.

We have that
L
X L
X
tk x(n−k) mod N = t0 xn + tk (x(n−k) mod N + x(n+k) mod N ).
k=−L k=1

When 0 ≤ n < L we have that (n + k) mod N = n + k, and

(
n−k when k ≤ n
(n − k) mod N = .
n − k + N when n < k ≤ L
The result follows.
b) L ≤ n < N − L:
L
X
zn = t0 xn + tk (xn+k + xn−k ).
k=1

Solution.

When L ≤ n < N − L we have that (n − k) mod N = n − k and (n + k) mod N = n + k,

and the result follows.
c) N − L ≤ n < N :
−1−n
NX L
X
zn = t0 xn + tk (xn+k + xn−k ) + tk (xn+k−N + xn−k ).
k=1 k=N −1−n+1

Solution.

When N − L ≤ n < N we have that (n − k) mod N = n − k, and

(
n+k when k ≤ N − 1 − n
(n + k) mod N = .
n + k − N when k > N − 1 − n
The result follows.
Chapter 4
Motivation for wavelets and some simple
examples

Exercise 4.1: The vector of samples is the coordinate vector

Show that the coordinate vector for f ∈ V0 in the basis {φ0,0 , φ0,1 , . . . , φ0,N −1 } is
(f (0), f (1), . . . .f (N − 1)). This shows that, for f ∈ Vm , there is no loss in working with
the samples of f rather than f itself.

Solution.
PN −1
We have that f (t) = n=0 cn φ0,n , where cn are the coordinates of f in the basis
{φ0,0 , φ0,1 , . . . , φ0,N −1 }. We now get that
N
X −1
f (k) = cn φ0,n (k) = ck ,
n=0

6 k. This shows that (f (0), f (1), . . . .f (N − 1)) are the

since φ0,n (k) = 0 when n =
coordinates of f .

Exercise 4.2: Realizing the DWT in terms of averages and

differences

Let f (t) ∈ V1 , and let fn,1 be the value f attains on [n, n + 1/2), and fn,2 the value f
attains on [n + 1/2, n + 1). Show that
• projV0 (f ) is the function in V0 which equals (fn,1 + fn,2 )/2 on the interval [n, n + 1).
• projW0 (f ) is the function in W0 which is (fn,1 − fn,2 )/2 on [n, n + 1/2), and −(fn,1 −
fn,2 )/2 on [n + 1/2, n + 1).

Hint.

Apply Lemma 10 in the book.

95
96 4 Motivation for wavelets and some simple examples

Solution.

Since f is constant and equal to f (n) on [n, n+1/2), and constant and equal to f (n+1/2)
on [n + 1/2, n + 1), we get that

Z N Z n+1
hf, φ0,n i = f (t)φ0,n (t)dt = f (t)dt
0 n
Z n+1/2 Z n+1
= f (t)dt + f (t)dt
n n+1/2
Z n+1/2 Z n+1
= f (n)dt + f (n + 1/2)dt
n n+1/2

= f (n)/2 + f (n + 1/2)/2 = (f (n) + f (n + 1/2))/2.

The orthogonal decomposition theorem gives that

N −1 N −1
X X f (n) + f (n + 1/2)
projV0 f = hf, φ0,n iφ0,n = φ0,n .
n=0 n=0
2

Since φ0,n is 1 on [n, n + 1) and 0 elsewhere, projV0 f is the piecewise constant function
which is equal to (f (n) + f (n + 1/2))/2 on [n, n + 1).

Exercise 4.3: Computing projections

a) Use Lemma 10 in the book to write down the matrix of projV0 : V1 → V0 relative to
φ1 and φ0 , and the matrix of projW0 : V1 → W0 relative to φ1 and ψ0 .

Solution.

From Lemma 10 in the book it follows that

√
projV0 (φ1,2n ) = φ0,n / 2
√
projV0 (φ1,2n+1 ) = φ0,n / 2

This means that

√
[projV0 (φ1,2n )]φ0 = en / 2
√
[projV0 (φ1,2n+1 )]φ0 = en / 2.

These are the columns in the matrix for projV0 relative to the bases φ1 and φ0 . This
matrix is thus
4 Motivation for wavelets and some simple examples 97
 
1 1 0 0 0 ··· 0 0 0
1 0 0 1 1 0 ··· 0 0 0
√ . ..  .

.. .. .. .. .. .. ..
2  .. . . . . . . . .
0 0 0 0 0 ··· 0 1 1
Similarly

√
projW0 (φ1,2n ) = ψ0,n / 2
√
projW0 (φ1,2n+1 ) = −ψ0,n / 2

so that

√
[projW0 (φ1,2n )]ψ0 = en / 2
√
[projW0 (φ1,2n+1 )]ψ0 = −en / 2.

These are the columns in the matrix for projW0 relative to the bases φ1 and ψ0 . This
matrix is thus
 
1 −1 0 0 0 · · · 0 0 0
1 0 0 1 −1 0 · · · 0 0 0 

√ . . . . . . . . .  .
2  .. .. .. .. .. .. .. .. .. 
0 0 0 0 0 · · · 0 1 −1
b) Write down the matrix of projV0 : V1 → V0 relative to (φ0 , ψ0 ) and φ0 , and the
matrix of projW0 : V1 → W0 relative to (φ0 , ψ0 ) and ψ0 .

Solution.

Note first that, since φ ∈ V0 we must have that projV0 (φ) = φ. Also, since ψ is in the
orthogonal complement of V0 in V1 we must have that projV0 (ψ) = 0. It follows that the
first columns in the matrix of projV0 relative to (φ0 , ψ0 ) and φ0 are

[projV0 (φ0,0 )](φ0 ,ψ0 ) = [φ0,0 ](φ0 ,ψ0 ) = e0

[projV0 (φ0,1 )](φ0 ,ψ0 ) = [φ0,1 ](φ0 ,ψ0 ) = e1
....
..

The last columns in the matrix of projV0 relative to (φ0 , ψ0 ) are

[projV0 (ψ0,0 )](φ0 ,ψ0 ) = [0](φ0 ,ψ0 ) = 0

[projV0 (ψ0,1 )](φ0 ,ψ0 ) = [0](φ0 ,ψ0 ) = 0
....
..

It follows that the matrix of projV0 relative to (φ0 , ψ0 ) is given by the diagonal matrix
where the first half of the entries on the diagonal are 1, the second half 0. In the same
98 4 Motivation for wavelets and some simple examples

way it follows that projW0 : V1 → W0 has a matrix relative to (φ0 , ψ0 ) and ψ0 given by
the diagonal matrix where the first half of the entries on the diagonal are 0, the second
half 1.
c) Generalize the points above to the projections from Vm+1 onto Vm and Wm .

Solution.

Since φm,n ∈ Vm we must have that T (φm,n ) = φm,n . Since ψm,n is in the orthogonal
complement of Vm in Vm+1 we must have that T (ψm,n ) = 0. The first half of the columns
in the matrix of projVm relative to (φm , ψm ) are

[projVm (φm,0 )](φm ,ψm ) = [φm,0 ](φm ,ψm ) = e0

[projVm (φm,1 )](φm ,ψm ) = [φm,1 ](φm ,ψm ) = e1
....
...

The second half of the columns are

[T (ψm,0 )](φm ,ψm ) = [0](φm ,ψm ) = 0

[T (ψm,1 )](φm ,ψm ) = [0](φm ,ψm ) = 0
....
...

Thus, as before, the matrix of projVm relative to (φm , ψm ) is given by the diagonal
matrix where the first half of the diagonal consists of 1’s, and the second half consists of
0’s. c) follows in the same way.

Exercise 4.4: Finding the least squares error

Show that

X Z n+1 2
kprojV0 f − f k2 = hf, f i − f (t)dt .
n n

Solution.

We have that

kf − projV0 (f )k2 = hf − projV0 (f ), f − projV0 (f )i

= hf, f i − 2hf, projV0 (f )i + hprojV0 (f ), projV0 (f )i

Now, note that

4 Motivation for wavelets and some simple examples 99

N
X −1 Z n+1 2
hprojV0 (f ), projV0 (f )i = f (t)dt
n=0 n

From the orthogonal decomposition formula it follows that

N
X −1 Z n+1 
projV0 (f ) = f (t)dt φ0,n (t).
n=0 n

This means that the above can be written

N
X −1 Z N Z n+1  N
X −1 Z n+1 2
= hf, f i − 2 f (s)ds φ0,n (t)f (t)dt + f (t)dt
n=0 0 n n=0 n
N
X −1 Z n+1 Z n+1  N
X −1 Z n+1 2
= hf, f i − 2 f (s)ds f (t)dt + f (t)dt
n=0 n n n=0 n
N
X −1 Z n+1 2 NX
−1 Z n+1 2
= hf, f i − 2 f (t)dt + f (t)dt
n=0 n n=0 n
N
X −1 Z n+1 2
= hf, f i − f (t)dt .
n=0 n

Exercise 4.5: Direct sums

Let C1 , C2 . . . , Cn be independent vector spaces, and let Ti : Ci → Ci be linear trans-

formations. The direct sum of T1 , T2 ,. . . ,Tn , written as T1 ⊕ T2 ⊕ . . . ⊕ Tn , denotes the
linear transformation from C1 ⊕ C2 ⊕ · · · ⊕ Cn to itself defined by

T1 ⊕ T2 ⊕ . . . ⊕ Tn (c1 + c2 + · · · + cn ) = T1 (c1 ) + T2 (c2 ) + · · · + Tn (cn )

when c1 ∈ C1 , c2 ∈ C2 , . . . , cn ∈ Cn . Also, when A1 , A2 , . . . , An are square matrices,
diag(A1 , A2 , · · · , An ) is defined as the block matrix where the blocks along the diagonal
are A1 , A2 , . . . , An , and all other blocks are 0 (see Appendix A in the book). Show that,
if Bi is a basis for Ci then

[T1 ⊕ T2 ⊕ . . . ⊕ Tn ](B1 ,B2 ,...,Bn ) = diag([T1 ]B1 , [T2 ]B2 , . . . , [Tn ]Bn ).

Solution.

If bj is the j’th basis vectors in Bi , column j in [Ti ]Bi is [Ti (bj )]Bi . Since (T1 ⊕ T2 ⊕
. . . ⊕ Tn )(bj ) = Ti (bj ), the column in [T1 ⊕ T2 ⊕ . . . ⊕ Tn ](B1 ,B2 ,...,Bn ) corresponding to
the basis vector bj in Bi is

(0, ..., 0, [Ti (bj )]Bi , 0, ..., 0)T ,

which equals the corresponding column in diag([T1 ]B1 , [T2 ]B2 , . . . , [Tn ]Bn ).
100 4 Motivation for wavelets and some simple examples

Exercise 4.6: Properties of block diagonal matrices

Assume that A, B, C, and D are square matrices of the same dimensions.

a) Assume that the eigenvalues of A are equal to those of B. What are the eigenvalues
of diag(A, B)? Can you express the eigenvectors of diag(A, B) in terms of those of A
and B?

Solution.

Assume that λ is an eigenvalue common to both A and B. Then there exists a vector v1
so that Av1 = λv1 , and a vector v2 so that Bv2 = λv2 . We now have that

          
v A 0 v1 Av1 λv1 v
diag(A, B) 1 = = = =λ 1 .
v2 0 B v2 Bv2 λv2 v2
 
v1
This shows that λ is an eigenvalue for diag(A, B) also, with an eigenvector.
v2
b) Assume that A and B also are non-singular. Show that diag(A, B) is non-singular,
and that (diag(A, B))−1 = diag(A−1 , B −1 ).

Solution.

We have that

   −1 
−1 −1 A 0 A 0
diag(A, B)diag(A ,B )=
0 B 0 B −1
AA−1 0
   
I0
= = =I
0 BB −1 0I

where we have multiplied as block matrices. This proves that diag(A, B) is non-singular,
with inverse diag(A−1 , B −1 ).
c) Show that diag(A, B)diag(C, D) = diag(AC, BD).

Solution.

We have that

    
A 0 C 0 AC 0
diag(A, B)diag(C, D) = = = diag(AC, BD)
0 B 0 D 0 BD

where we again have multiplied as block matrices.

4 Motivation for wavelets and some simple examples 101

Exercise 4.7: The DWT when N is odd

When N is odd, the (first stage in a) DWT is defined as the change of coordinates from
(φ1,0 , φ1,1 , . . . , φ1,N −1 ) to

(φ0,0 , ψ0,0 , φ0,1 , ψ0,1 , . . . , φ0,(N −3)/2 , ψ0,(N −3)/2 , φ0,(N −1)/2 ).
Since all functions are assumed to have period N , we have that

1 1
φ0,(N −1)/2 = √ (φ1,N −1 + φ1,N ) = √ (φ1,0 + φ1,N −1 ).
2 2
This says that the last column in the change of coordinate matrix from C1 to φ1 (i.e. the
IDWT matrix) equals √12 (1, 0, ..., 0, 1). In particular, for N = 3, the IDWT matrix equals
 
1 1 1
1 
√ 1 −1 0 ,
2 0 0 1

The DWT matrix, i.e. the inverse of this, is

 
1 1 −1
1
√ 1 −1 −1 .
2 0 0 2

a) Explain from this that, when N is odd, the DWT matrix can be constructed from
the DWT matrix with N − 1 columns by adding √12 (−1, −1, 0, . . . , 0, 2)T as the last
columns, and adding zeros in the last row.

Solution.

Note that the (N − 1) × (N − 1)-IDWT matrix equals the upper left block of the
N × N -IDWT matrix. To be more precise we have that
 √ 
IDWTN −1 e0 /√ 2
IDWTN = .
0 1/ 2
Subtracting the last row from the first we obtain
  √   
I −e0 IDWTN −1 e0 /√ 2 IDWTN −1 0√
= .
0 1 0 1/ 2 0 1/ 2
Thus
  
I e0 IDWTN −1 0√
IDWTN =
0 1 0 1/ 2
Both matrices here are easily inverted and we obtain
102 4 Motivation for wavelets and some simple examples
 √ √ √ 
1/√2 1 √2 0 0 0 ··· 0 −1/√2
   1/ 2 −1 2 0
√ √ 0 ···
0 0 −1/ 2
DWTN −1 √0 I −e0
 
DWTN = = 0
 0 1/ 2 1 2 0 ···0 0 
0 2 0 1

 . .. .. .. .. . .
.. .. 
 .. . . . . . . √. 
0 0 0 0 0 ··· 0 2

where multiplication with the matrix to the right corresponded to subtracting the first
column from the last. This gives the stated expression for the last column.
b) Explain that the DWT matrix is orthogonal if and only if N is even. Also explain
that it is only the last column which spoils the orthogonality.

Solution.
 √ √ 
1/√2 1 √2
When N is even orthogonality follows from the orthogonality of . If
1/ 2 −1 2
DWTN was orthogonal for N odd, the last row of the IDWT-matrix would be equal
to the last column of the DWT matrix. But the last row of the IDWT matrix has one
nonzero entry, while the last column of the DWT matrix has three nonzero entries. This
proves that DWTN is not orthogonal for N odd.

Exercise 4.8: General kernel transformations

In the next sections we will define other functions φ and ψ. For these we will also define
φm,n (t) = 2m/2 φ(2m t − n) and ψm,n (t) = 2m/2 ψ(2m t − n), and we will define resolution-
and detail spaces using these instead, and find similar properties to those we found
in this section. These new functions give rise to new kernel transformations G and H.
From this exercise it will be clear that, also in this more general setting, it causes no
confusion to suppress the resolution m in the notation for the kernel transformations
(i.e. H and G), as we have done.
a) Show that if

X X
φ(t) = an φ1,n (t) ψ(t) = bn φ1,n (t),
n n

then also

X X
φm,k (t) = an φm+1,2k+n (t) ψm,k (t) = bn φm+1,2k+n (t).
n n

Solution.

We have that
4 Motivation for wavelets and some simple examples 103
X
φm,k (t) = 2m/2 φ(2m t − k) = 2m/2 an φ1,n (2m t − k)
n
X
(m+1)/2 m+1
= an 2 φ(2 t − 2k − n)
n
X
= an φm+1,2k+n (t).
n

A similar computation can be done for ψ.

b) Show that if

X
φ1,0 (t) = (cn φ0,n (t) + dn ψ0,n (t))
n
X
φ1,1 (t) = (en φ0,n (t) + fn ψ0,n (t)) ,
n

then also

X
φm+1,2k (t) = (cn φm,n+k (t) + dn ψm,n+k (t))
n
X
φm+1,2k+1 (t) = (en φm,n+k (t) + fn ψm,n+k (t)) .
n

Solution.

We have that

φm+1,2k (t) = 2(m+1)/2 φ(2m+1 t − 2k) = 2m/2 21/2 φ(2(2m t − k))

= 2m/2 φ1,0 (2m t − k)
X 
= bn 2m/2 φ0,n (2m t − k) + cn 2m/2 ψ0,n (2m t − k)
n
X
= (bn φm,n+k (t) + cn ψm,n+k (t))
n

A similar computation is done for φm+1,2k+1 (t).

Exercise 4.13: Finding N

Assume that you run an m-level DWT on a vector of length r. What value of N does
this correspond to?
104 4 Motivation for wavelets and some simple examples

Solution.

The functions φm,n are 2−m apart, and since there are r of them we must have that
N = r2−m . In particular, if r is not divisible by 2m , N will not be an integer.

Exercise 4.14: Different DWT’s for similar vectors

In Figure 4.1 we have plotted the DWT’s of two vectors x1 and x2 . In both vectors
we have 16 ones followed by 16 zeros, and this pattern repeats cyclically so that the
length of both vectors is 256. The only difference is that the second vector is obtained
by delaying the first vector with one element.

1.2 1.2
1.0 1.0
0.8 0.8
0.6 0.6
0.4 0.4
0.2 0.2
0.0 50 100 150 200 250 0.0 50 100 150 200 250
Fig. 4.1 2 vectors x1 and x2 which seem equal, but where the DWT’s are very different.

You see that the two DWT’s are very different: For the first vector we see that there
is much detail present (the second part of the plot), while for the second vector there is
no detail present. Attempt to explain why this is the case. Based on your answer, also
attempt to explain what can happen if you change the point of discontinuity for the
piecewise constant function to something else.

Solution.

The basis vectors in φ1 have support intervals of the form [n/2, (n + 1)/2), while the
basis vectors in φ0 have support intervals of the form [n, n + 1). The functions in V1
with zero detail are thus exactly the functions which are constant on intervals of the
form [n, n + 1). It is straightforward to check that the function corresponding to the
first vector is non-zero and constant on [0, 8), [16, 24), and so on, while for the second
vector we obtain [0.5, 8.5), [16.5, 24.5), and so on. The second vector therefore has detail
associated with it, while the first one has not.
If the function φ was changed so that its support is [0.5, 1.5), the appearance of the
functions in φ1 without detail would change accordingly. In this case you are invited to
check that the first vector above has detail, while the second does not.
4 Motivation for wavelets and some simple examples 105

Exercise 4.15: Construct a sound

Construct a (nonzero) sound where the low resolution approximations equal the sound
itself for m = 1, m = 2.

Solution.

It is enough to consider a sound where the samples satisfy x4r = x4r+1 = x4r+2 = x4r+3
for any r. This makes the function in V2 with coordinates x in φ2 constant on intervals
of the form [n, n + 1), so that it is in V0 , and therefore has no detail associated with it.
It follows that both low resolution approximations equal the sound itself.

Exercise 4.16: Exact computation of wavelet coefficients

For the following functions compute the detail coefficients analytically, i.e. compute the
RN
quantities wm,n = 0 f (t)ψm,n (t)dt similarly to how this was done in Example 4.12 in
the book:
a) The functions in Example 4.11 in the book.

Solution.

Note first that

(n+1/2)2−m (n+1)2−m
!
Z N Z Z
f (t)ψm,n (t)dt = 2m/2 f (t)dt − f (t)dt .
0 n2−m (n+1/2)2−m

With f (t) = 1 − 2|1/2 − t/N | we have two possibilities: when n < N 2m−1 we have
that [n2−m , (n + 1)2−m ) ⊂ [0, N/2], so that f (t) = 2t/N , and we get

(n+1/2)2−m (n+1)2−m
Z Z !
wm,n = 2m/2 2t/N dt − 2t/N dt
n2−m (n+1/2)2−m
(n+1/2)2−m (n+1)2−m
= 2m/2 [t2 /N ]n2−m − 2m/2 [t2 /N ](n+1/2)2−m
2−3m/2 2−3m/2−1
2(n + 1/2)2 − n2 − (n + 1)2 = −


= .
N N
RN
When n ≥ N 2m−1 we have that f (t) = 2 − 2t/N , and using that 0 ψm,n (t)dt = 0 we
−3m/2−1
must get that wm,n = 2 N .
For f (t) = 1/2 + cos(2πt/N )/2, note first that this has the same coefficients as
RN
cos(2πt/N )/2, since 0 ψm,n (t)dt = 0. We now get
106 4 Motivation for wavelets and some simple examples

(n+1/2)2−m (n+1)2−m
Z Z !
m/2
wm,n = 2 cos(2πt/N )/2dt − cos(2πt/N )/2dt
n2−m (n+1/2)2−m
(n+1/2)2−m (n+1)2−m
= 2m/2 [N sin(2πt/N )/(4π)]n2−m − 2m/2 [N sin(2πt/N )/(4π)](n+1/2)2−m
2m/2−2 N
2 sin(2π(n + 1/2)2−m /N ) − sin(2πn2−m /N ) − sin(2π(n + 1)2−m /N ) .


=
π
There seems to be no more possibilities for simplification here.

k
b) The functions f (t) = Nt .

Solution.

We get

(n+1/2)2−m (n+1)2−m
Z Z !
m/2 k k
wm,n = 2 (t/N ) dt − (t/N ) dt
n2−m (n+1/2)2−m
(n+1/2)2−m (n+1)2−m
= 2m/2 [tk+1 /((k + 1)N k )]n2−m − 2m/2 [tk+1 /((k + 1)N k )](n+1/2)2−m
2−m(k+1/2)
2(n + 1/2)k+1 − nk+1 − (n + 1)k+1 .


= k
(k + 1)N

The leading term nk+1 will here cancel, but the others will not, so there is no room for
further simplification here.

Exercise 4.17: Computing the DWT of a simple vector

Suppose that we have the vector x with length 210 = 1024, defined by xn = 1 for n
even, xn = −1 for n odd. What will be the result if you run a 10-level DWT on x? Use
the function dwt_impl to verify what you have found.

Hint.
√
We defined ψ by ψ(t) = (φ √1,0 (t) − φ1,1 (t))/ 2. From this connection
√ it follows that
ψ9,n = (φ10,2n − φ10,2n+1 )/ 2, and thus φ10,2n − φ10,2n+1 = 2ψ9,n . Try to couple this
identity with the alternating sign you see in x.

Solution.
P1023
The vector x is the coordinate vector of the function f (t) = n=0 (−1)n φ10,n in the
√ P511 √
basis φ10 for V10 . Since φ10,2n − φ10,2n+1 = 2ψ9,n , we can write f (t) = n=0 2ψ9,n .
Since a 10-level-DWT gives as a result the coordinate vector of f in

(φ0 , ψ0 , ψ1 , ψ2 , ψ3 , ψ4 , ψ5 , ψ6 , ψ7 , ψ8 , ψ9 ),
4 Motivation for wavelets and some simple examples 107

(the DWT is √ nothing but the change of coordinates from φ10 to this basis), and√since
P511
f (t) = n=0 2ψ9,n , it is clear that the coordinate vector of f in this basis has 2 in
the second part (the ψ9 -coordinates), and 0 elsewhere. The 10-level DWT of x√therefore
gives the vector of length 1024 which is 0 on the first half, and equal to 2 on the
second half. m = 10 is here arbitrarily chosen: The result would have been the same for
m = 1, m = 2, and so on. The following code verifies the result:

dwt_impl(repmat([1; -1],512,1), ’Haar’, 10)

Exercise 4.18: Implementing the DWT when N is odd

Use the results from Exercise 4.7 to rewrite the implementations dwt_kernel_haar and
idwt_kernel_haar so that they also work in the case when N is odd.

Solution.

The library versions of these functions are as follows

function x=dwt_kernel_haar(x, bd_mode)

x = x/sqrt(2);
N = size(x, 1);
if mod(N,2) == 1
x(1:2, :) = [x(1, :) + x(2, :) - x(N, :);x(1, :) - x(2, :) - x(N, :)];
x(N, :) = 2*x(N, :);
else
x(1:2, :) = [x(1, :) + x(2, :); x(1, :) - x(2, :)];
end
for k = 3:2:(N-1)
x(k:(k+1), :) = [x(k, :) + x(k+1, :); x(k, :) - x(k+1, :)];
end
end

function x=idwt_kernel_haar(x, bd_mode)

x = x/sqrt(2);
N = size(x, 1);
if mod(N,2) == 1
x(1:2, :) = [x(1, :) + x(2, :) + x(N, :); x(1, :) - x(2, :)];
else
x(1:2, :) = [x(1, :) + x(2, :); x(1, :) - x(2, :)];
end
for k = 3:2:(N-1)
x(k:(k+1), :) = [x(k, :) + x(k+1, :); x(k, :) - x(k+1, :)];
end
end

Since find_kernel finds and returns the kernels supported by the library,
dwt_kernel_haar is defined within find_kernel.m in the library.

Exercise 4.19: In-place DWT and partial bit-reversal

The kernel transformations in this book can be computed in-place. A DWT is also
required to reorder coordinates as in the (φ0 , ψ0 , ..., ψm−1 )-basis, however, and this
108 4 Motivation for wavelets and some simple examples

exercise addresses how this reordering also can be computed in-place. As a result the
entire DWT can be computed in-place.
a) Show that the coordinates in φ0 after m iterations of a kernel transformation end
up at indices k2m , k = 0, 1, 2, . . ., and that the coordinates in ψ0 end up at indices
2m−1 + k2m , k = 0, 1, 2, . . ..

Solution.

This is easily shown by induction. For m = 1 the statement says that the φ0 -coordinates
are at the even indices, and the ψ0 -coordinates are at the odd indices. Both these
statements are clearly true. For the induction step, if the coordinates in φ1 after m − 1
iterations of the kernel are at the indices k2m−1 , after the next iteration every second
element of these (i.e. the ones at indices 2m−1 · 2k = k2m ) will be coordinates in φ0 .
while the ones at indices 2m−1 (2k + 1) = 2m−1 + k2m will be coordinates in ψ0 .
b) After m iterations of a kernel transformation, show that if the indices are rearranged
so that the last m bits are placed in front in reverse order, then coordinates from φ0
are placed first, while coordinates from ψi are placed before those from {ψj }j>i . This
is also called a partial bit-reversal. When an m-level DWT on a vector of length 2m is
computed, the partial bit-reversal is actually a full bit-reversal.
After a partial bit-reversal, what can you say about the internal ordering in {ψj }j>i ?

Solution.

As proved in a), the ψm−n -coordinates end up at indices 2n−1 + k2n . After a partial
bit-reversal, their bit representations have 0 · · · 01 as the first n bits. Lower n thus implies
higher numbers, so that coordinates from ψi are placed before those from {ψj }j>i . Since
φ0 -coordinates end up at indices k2m , their first m bits are 0 · · · 0, so that these are
placed first.
The internal ordering of ψj will be violated by a partial bit-reversal, however. If one
requires an in-place ordering that produces the same order as (φ0 , ψ0 , ..., ψm−1 ), one
should secure that, for the ψm−j -coordinates, where the last j bits are 0 · · · 01, only
those j bits should be placed in front in reverse order. Thus, partial bit-reversal reverses
too many bits, violating the internal order in the ψm−j . Reversing the last bits and
placing them in front should thus be replaced with a more refined version which adapts
to the different resolutions.
c) Write a function partial_bit_reversal(x, m) which computes a partial bit-
reversal of the vector x in-place. Assume that x has length 2n with n ≥ m.

Hint.

Partial bit-reversal can be split in two: First a full bit-reversal, and then reversing the
last n − m bits again. It turns out that reversal of the last bits can be computed easily
using bit-reversal of n − m bits. Due to this we can easily implement partial bit-reversal
by adapting our previous function for bit-reversal.
4 Motivation for wavelets and some simple examples 109

Solution.

The general code can be found in the solution to d).

d) Contrary to full bit-reversal, partial bit-reversal is not its own inverse. It is straight-
forward, however, to compute the inverse following the same lines as in c). Extend
the function partial_bit_reversal so that it takes takes a third parameter forward,
which indicates whether a forward- or reverse partial bit-reversal should be computed.

Solution.

The code can look as follows. The code calls a function which is a slight modification of
bit_reversal, adapated to the current setting.

function x=partial_bit_reversal(x, m, forward)

N = size(x,1);
if forward
x=bit_reversal_last_bits(x, log2(N));
x=bit_reversal_last_bits(x, m);
else
x=bit_reversal_last_bits(x, m);
x=bit_reversal_last_bits(x, log2(N));
end
end
function x=bit_reversal_last_bits(x, num_bits)
N = 2^num_bits;
j=0;
for i = 0:2:(N/2 - 2)
if (j > i)
temp = x((j + 1):2^num_bits:end, :);
x((j + 1):2^num_bits:end, :) = x((i + 1):2^num_bits:end, :);
x((i + 1):2^num_bits:end, :) = temp;
temp = x((N-i):2^num_bits:end, :);
x((N-i):2^num_bits:end, :) = x((N-j):2^num_bits:end, :);
x((N-j):2^num_bits:end, :) = temp;
end
temp = x((i+2):2^num_bits:end, :);
x((i+2):2^num_bits:end, :) = x((j + N/2 + 1):2^num_bits:end, :);
x((j + N/2 + 1):2^num_bits:end, :) = temp;
m = N/4;
while (m >= 1 && j >= m)
j = j - m;
m = m/2;
end
j = j + m;
end
end

Exercise 4.22: The vector of samples is the coordinate vector

Show that, for f ∈ V0 we have that [f ]φ0 = (f (0), f (1), . . . , f (N − 1)). This shows that,
also for the piecewise linear wavelet, there is no loss in working with the samples of f
rather than f itself.
110 4 Motivation for wavelets and some simple examples

Solution.
PN −1
Let us write f (t) = n=0 cn φ0,n (t). If k is an integer we have that
N
X −1 N
X −1
f (k) = cn φ0,n (k) = cn φ(k − n).
n=0 n=0

Clearly the only integer for which φ(s) = 6 0 is s = 0 (since φ(0) = 1), so that the
only n which contributes in the sum is n = k. This means that f (k) = ck , so that
[f ]φ0 = (f (0), f (1), . . . , f (N − 1)).

Exercise 4.23: The piecewise linear wavelet is not orthogonal

Show that

2 1
hφ0,n , φ0,n i = hφ0,n , φ0,n±1 i = hφ0,n , φ0,n±k i = 0 for k > 1.
3 6
As a consequence, the {φ0,n }n are neither orthogonal, nor have norm 1.

Solution.

We have that

Z n+1
hφ0,n , φ0,n i = (1 − |t − n|)2 dt
n−1
Z n+1
1 − 2|t − n| + (t − n)2 dt


=
n−1
 n+1
1 2
=2−2+ (t − n)3 = .
3 n−1 3

We also have

Z n+1 Z 1
hφ0,n , φ0,n+1 = (1 − (t − n))(1 + (t − n − 1))dt = (1 − u)(1 + u − 1)du
n 0
Z 1
1 1 1
= (t − t2 )dt = − = .
0 2 3 6

Finally, the supports of φ0,n and φ0,n±k are disjoint for k > 1, so that we must have
hφ0,n , φ0,n±k i = 0 in that case.
4 Motivation for wavelets and some simple examples 111

Exercise 4.24: Implementation of elementary lifting of odd

type

Write a function

lifting_odd_symm(lambda, x, bd_mode)

which applies an elementary lifting matrix of odd type (equation (4.28) in the book)
to x. Assume that N is even. The parameter bd_mode should do nothing, as we will
return to this parameter later. The function should not allocate the full matrix, and
apply as few multiplications as possible. How many local variables are needed in your
code? Compare with the corresponding number you found for filters in Exercise 3.26.

Solution.

The library version of this function is as follows.

function x=lifting_odd_symm(lambda, x, bd_mode)

N = size(x, 1);
x(2:2:(N-1), :) = x(2:2:(N-1),:) + lambda*(x(1:2:(N-2),:) + x(3:2:N,:));
if mod(N,2)==0 % last must also be included
if strcmpi(bd_mode, ’symm’)
x(N, :) = x(N, :) + 2*lambda*x(N-1, :); % Symmetric extension
elseif strcmpi(bd_mode, ’per’)
x(N, :) = lambda*(x(1, :) + x(N-1, :)) + x(N, :);
elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
x(N, :) = lambda*x(N-1, :) + x(N, :);
end
end
end

Since λ appears twice in each row, note that we have added entries before multiplication,
to avoid extra multiplication. We also see that there is no need for local variables in the
code.

Exercise 4.25: Computing projections

In this exercise we will show how the projection of φ1,1 onto V0 can be computed.
We will see from this that it is nonzero, and that its support is the entire [0, N ]. Let
f = projV0 φ1,1 , and let xn = f (n) for 0 ≤ n < N . This means that, on (n, n + 1),
f (t) = xn + (xn+1 − xn )(t − n).
R n+1
a) Show that n f (t)2 dt = (x2n + xn xn+1 + x2n+1 )/3.

Solution.

We have that
112 4 Motivation for wavelets and some simple examples
Z n+1 Z n+1 Z 1
f (t)2 dt = (xn + (xn+1 − xn )(t − n))2 dt = (xn + (xn+1 − xn )t)2 dt
n n 0
Z 1
= (x2n + 2xn (xn+1 − xn )t + (xn+1 − xn )2 t2 )dt
0
1
= x2n t + xn (xn+1 − xn )t2 + (xn+1 − xn )2 t3 /3 0


1
= x2n + xn (xn+1 − xn ) + (xn+1 − xn )2 /3 = (x2n + xn xn+1 + x2n+1 ).
3
b) Show that

1/2 √
Z  
1 1
(x0 + (x1 − x0 )t)φ1,1 (t)dt = 2 2 x0 + x1
0 12 24
1 √
Z  
1 1
(x0 + (x1 − x0 )t)φ1,1 (t)dt = 2 2 x0 + x1 .
1/2 24 12

Solution.

We have that

Z 1/2
(x0 + (x1 − x0 )t)φ1,1 (t)dt
0
Z 1/2 √ √ Z 1/2
= (x0 + (x1 − x0 )t)2 2tdt = 2 2 (x0 t + (x1 − x0 )t2 )dt
0 0
1/2
√ 1 √
  
1 1 1
= 2 2 x0 t2 + (x1 − x0 )t3 =2 2 x0 + (x1 − x0 ))
2 3 0 8 24
√
 
1 1
=2 2 x0 + x1 .
12 24

In the same way

Z 1
(x0 + (x1 − x0 )t)φ1,1 (t)dt
1/2
Z 1 √ Z
√ 1
= (x0 + (x1 − x0 )t)2 2(1 − t)tdt = 2 2 (x0 + (x1 − 2x0 )t − (x1 − x0 )t2 )tdt
1/2 1/2
1
√ √
  
1 1 1 3 7
= 2 2 x0 t + (x1 − 2x0 )t2 − (x1 − x0 )t3 =2 2 x0 + (x1 − 2x0 ) − (x1 − x0 )
2 3 1/2 2 8 24
√
 
1 1
=2 2 x0 + x1 .
24 12

c) Use the fact that

4 Motivation for wavelets and some simple examples 113
Z N N
X −1
(φ1,1 (t) − xn φ0,n (t))2 dt
0 n=0
Z 1 Z 1/2 Z 1
= φ1,1 (t)2 dt − 2 (x0 + (x1 − x0 )t)φ1,1 (t)dt − 2 (x0 + (x1 − x0 )t)φ1,1 (t)dt
0 0 1/2
N
X −1 Z n+1
+ (xn + (xn−1 − xn )t)2 dt
n=0 n

PN −1
and a) and b) to find an expression for kφ1,1 (t) − n=0 xn φ0,n (t)k2 .

Solution.

Using a) and b) we see that the above can be written as

N −1
√ √
    
2 X 1 2 1 1 1 1
+ (xn + xn xn+1 + x2n+1 ) − 2 2 2 x0 + x1 − 2 2 x0 + x1
3 n=0 3 12 24 24 12
N −1 N −1 √
2 2 X 2 1 X 2
= + xn + xn xn+1 − (x0 + x1 ).
3 3 n=0 3 n=0 2

d) To find the minimum least squares error, we can set the gradient of the expression
in c) to zero, and thus find the expression for the projection of φ1,1 onto V0 . Show that
−1
the values {xn }N 1
n=0 can be found by solving the equation Sx = b, where S =√3 {1, 4, 1}
is an N × N symmetric filter, and b is the vector with components b0 = b1 = 2/2, and
bk = 0 for k ≥ 2.

Solution.

We see that the partial derivatives of the function in c) are

√
∂f 1 4 1 2
= xN −1 + x0 + x1 −
∂x0 3 3 3 2
√
∂f 1 4 1 2
= x0 + x1 + x1 −
∂x1 3 3 3 2
∂f 1 4 1
= xi−1 + xi + xi+1 2 ≤ i < N − 1
∂xi 3 3 3
∂f 1 4 1
= xN −2 + xN −1 + x0 .
∂xN −1 3 3 3
√
Moving the two terms 22 over to the right hand side, setting the gradient equal to zero
is the same as solving the system Sx = b which we stated.
e) Solve the system in d. for some values of N to verify that the projection of φ1,1 onto
V0 is nonzero, and that its support covers the entire [0, N ].
114 4 Motivation for wavelets and some simple examples

Solution.

The following code can be used

N = 16;
S=zeros(N);
S(1,N)=1/3; S(1,1)=4/3; S(1,2)=1/3; % First row
for k=2:(N-1)
S(k,(k-1):(k+1)) = [1/3 4/3 1/3];
end
S(N,N-1)=1/3; S(N,N)=4/3; S(N,1)=1/3; % Last row
b=zeros(N,1); b(1)=sqrt(2)/2; b(2)=sqrt(2)/2;
plot(0:(N-1),S\b) %Plots the projection

Exercise 4.26: Convolution relation between the piecewise

constant- and piecewise linear scaling functions

The convolution of two functions defined on (−∞, ∞) is defined by

Z ∞
(f ∗ g)(x) = f (t)g(x − t)dt.
−∞

Show that we can obtain the piecewise linear φ we have defined as φ = χ[−1/2,1/2) ∗
χ[−1/2,1/2) (recall that χ[−1/2,1/2) is the function which is 1 on [−1/2, 1/2) and 0, see
equation (1.16) in the book). This gives us a nice connection between the piecewise
constant scaling function (which is similar to χ[−1/2,1/2) ) and the piecewise linear scaling
function in terms of convolution.

Solution.

We have that

Z ∞
χ[−1/2,1/2) ∗ χ[−1/2,1/2) (x) = χ[−1/2,1/2) (t)χ[−1/2,1/2) (x − t)dt.
−∞

The integrand here is 1 when −1/2 < t < 1/2 and −1/2 < x − t < 1/2, or in
other words when max(−1/2, −1/2 + x) < t < min(1/2, 1/2 + x) (else it is 0). When
x > 0 this happens when −1/2 + x < t < 1/2, and when x < 0 this happens when
−1/2 < t < 1/2 + x. This means that
(R 1/2
dt = 1 − x , x > 0
χ[−1/2,1/2) ∗ χ[−1/2,1/2) (x) = R−1/2+x
1/2+x
−1/2
dt = 1 + x , x < 0.
But this is by definition φ.
4 Motivation for wavelets and some simple examples 115

Exercise 4.29: Implementation of elementary lifting of even

type

Write a function

lifting_even_symm(lambda, x, bd_mode)

which applies an elementary lifting matrix of even type (equation (4.36) in the book) to
x. As before, assume that N is even, and that the parameter bd_mode does nothing.

Solution.

The library version of this function is as follows.

function x=lifting_even_symm(lambda, x, bd_mode)

N = size(x, 1);
if strcmpi(bd_mode, ’per’)
assert(mod(N,2) == 0)
end
if strcmpi(bd_mode, ’symm’)
x(1, :) = x(1, :) + 2*lambda*x(2, :); % Symmetric extension
elseif strcmpi(bd_mode, ’per’)
x(1, :) = lambda*(x(2, :) + x(N, :)) + x(1, :);
elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
x(1, :) = lambda*x(2, :) + x(1, :);
end
x(3:2:(N-1), :) = x(3:2:(N-1),:) + lambda*(x(2:2:(N-2),:) + x(4:2:N,:));
if mod(N,2) == 1 % last must also be included
if strcmpi(bd_mode, ’symm’)
x(N, :) = x(N, :) + 2*lambda*x(N-1, :); % Symmetric extension
elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
x(N, :) = x(N, :) + lambda*x(N-1, :);
end
end
end

Since also here λ appears twice in each row, we have added entries before multiplication,
to avoid extra multiplication.

Exercise 4.30: Two vanishing moments

In this exercise we will show that there is a unique function on the form given by
equation (4.30) in the book which has two vanishing moments.
a) Show that, when ψ̂ is defined by equation (4.30) in the book, we have that



 −αt − α for − 1 ≤ t < 0
α − β)t − α 0 ≤ t < 1/2



 (2 + for
ψ̂(t) = (α − β − 2)t − α + 2 for 1/2 ≤ t < 1

βt − 2β for 1 ≤ t < 2





0 for all other t
116 4 Motivation for wavelets and some simple examples

Solution.

The function ψ̂ is a sum of the functions ψ = √12 φ1,1 , φ, and φ0,1 (i.e. we have set n = 0
in equation (4.30) in the book). All these are continuous and piecewise linear, and we
can write


2t
 0 ≤ t < 1/2
ψ(t) = 2 − 2t 1/2 ≤ t < 1

0 elsewhere


1 + t
 −1 ≤ t < 0
φ0,0 (t) = 1 − t 0≤t<1

0 elsewhere


t
 0≤t<1
φ0,1 (t) = 2 − t 1≤t<2 .

0 elsewhere


It follows that ψ̂(t) = ψ(t) − αφ(t) − βφ1,1 is piecewise linear, and linear on the segments
[−1, 0], [0, 1/2], [1/2, 1], [1, 2].
On the segment [−1, 0] only the function φ is seen to be nonzero, and since φ(t) = 1+t
here, we have that ψ̂(t) = −α(1 + t) = −α − αt here.
On the segment [0, 1/2] all three functions are nonzero, and

φ1,1 (t) = 2t
φ0,0 (t) = 1 − t
φ0,1 (t) = t

on this interval. This means that ψ̂(t) = 2t − α(1 − t) − βt = (2 + α − β)t − α on [0, 1/2].
On the segment [0, 1/2] all three functions are nonzero, and

φ1,1 (t) = 2 − 2t
φ0,0 (t) = 1 − t
φ0,1 (t) = t

on this interval. This means that ψ̂(t) = 2 − 2t − α(1 − t) − βt = (α − β − 2)t − α + 2

on [1/2, 1].
On the segment [1, 2] only the function φ0,1 is seen to be nonzero, and since φ0,1 (t) =
2 − t here, we have that ψ̂(t) = −β(2 − t) = βt − 2β here. For all other values of t, ψ̂ is
zero. This proves the formulas for ψ̂ on the different intervals.
b) Show that

Z N Z N
1 1
ψ̂(t)dt = − α − β, tψ̂(t)dt = − β.
0 2 0 4
4 Motivation for wavelets and some simple examples 117

Solution.

We can write

Z N Z 2 Z 0 Z 1/2 Z 1 Z 2
ψ̂(t)dt = ψ̂(t)dt = ψ̂(t)dt + ψ̂(t)dt + ψ̂(t)dt + ψ̂(t)dt
0 −1 −1 0 1/2 1
Z 0 Z 1/2
= (−α − αt)dt + (2 + α − β)t − α)dt
−1 0
Z 1 Z 2
+ ((α − β − 2)t − α + 2)dt + (βt − 2β)dt
1/2 1
 0  1/2
1 2 1 2
= −αt − αt + (2 + α − β)t − αt
2 −1 2 0
 1  2
1 2 1 2
+ (α − β − 2)t + (2 − α)t + βt − 2βt
2 1/2 2 1
1 1 1 3 1 3
= −α + α + (2 + α − β) − α + (α − β − 2) + (2 − α) + β − 2β
2 8 2 8 2 2
1
= − α − β,
2
RN 1
0
tψ̂(t)dt is computed similarly, so that we in the end arrive at 4 − β.
c) Explain why there is a unique function on the form given by equation (4.30) in the
book which has two vanishing moments, and that this function is given by equation
(4.32) in the book.

Solution.

The equation system

1
−α−β =0
2
1
−β =0
4
has the unique solution α = β = 14 , which we already have found.

Exercise 4.31: More than two vanishing moments

In the previous exercise we ended up with a lot of calculations to find α, β in equation

(4.30) in the book. Let us try to make a program which does this for us, and which also
makes us able to generalize the result.
a) Define
118 4 Motivation for wavelets and some simple examples
Z 1 Z 2 Z 1
ak = tk (1 − |t|)dt, bk = tk (1 − |t − 1|)dt, ek = tk (1 − 2|t − 1/2|)dt,
−1 0 0

for k ≥ 0. Explain why finding α, β so that we have two vanishing moments in equation
(4.30) in the book is equivalent to solving the following equation:
    
a0 b0 α e
= 0
a1 b1 β e1
Write a program which sets up and solves this system of equations, and use this program
to verify the values for α, β we previously have found.

Hint.

you can integrate functions in MATLAB with the function quad. As an example, the
function φ(t), which is nonzero only on [−1, 1], can be integrated as follows:

quad(@(t)t.^k.*(1-abs(t)),-1,1)

Solution.
R R
In order for ψ to have vanishing moments we must have that ψ̂(t)dt = tψ̂(t)dt = 0
Substituting ψ̂ = ψ − αφ0,0 − βφ0,1 we see that, for k = 0, 1,
Z Z
t (αφ0,0 + βφ0,1 ) dt = tk ψ(t)dt.
k

The left hand side can here be written

Z Z Z
tk (αφ0,0 + βφ0,1 ) dt = α tk φ0,0 dt + β tk φ0,1 (t)dt
Z 1 Z 2
=α tk (1 − |t|)dt + β tk (1 − |t − 1|)dt = αak + βbk .
−1 0

The right hand side is

Z Z Z 1
k k
t ψ(t)dt = t φ1,1 (t)dt = (1 − 2|t − 1/2|)dt = ek .
0
The following program sets up the corresponding equation systems, and solves it by
finding α, β.

A = zeros(2);
b = zeros(2, 1);
for k = 0:1
A(k + 1, :) = [quad(@(t)t.^k.*(1 - abs(t)), -1, 1)...
quad(@(t)t.^k.*(1 - abs(t - 1)), 0, 2)];
b(k + 1) = quad(@(t)t.^k.*(1 - 2*abs(t - 1/2)), 0, 1);
end
A\b;
4 Motivation for wavelets and some simple examples 119

b) The procedure where we set up a matrix equation in a) allows for generalization to

more vanishing moments. Define

ψ̂ = ψ0,0 − αφ0,0 − βφ0,1 − γφ0,−1 − δφ0,2 .

We would like to choose α, β, γ, δ so that we have 4 vanishing moments. Define also

Z 0 Z 3
gk = tk (1 − |t + 1|)dt, dk = tk (1 − |t − 2|)dt
−2 1

for k ≥ 0. Show that α, β, γ, δ must solve the equation

    
a0 b0 g0 d0 α e0
a1 b1 g1 d1  β  e1 
a2 b2 g2 d2   γ  = e2  ,
    

a3 b3 g3 d3 δ e3
and solve this with your computer.

Solution.

Similarly to a), the equation in b) gives that

Z Z
tk (αφ0,0 + βφ0,1 + γφ0,−1 + δφ0,2 ) dt = tk ψ(t)dt.

The corresponding equation system is deduced exactly as in a). The following program
sets up the corresponding equation systems, and solves it by finding α, β, γ, δ.

A = zeros(4);
b = zeros(4, 1);
for k = 0:3
A(k + 1, :) = [quad(@(t)t.^k.*(1 - abs(t)), - 1, 1)...
quad(@(t)t.^k.*(1 - abs(t - 1)), 0, 2)...
quad(@(t)t.^k.*(1 - abs(t + 1)), -2, 0)...
quad(@(t)t.^k.*(1 - abs(t - 2)), 1, 3)];
b(k + 1) = quad(@(t)t.^k.*(1 - 2*abs(t - 1/2)), 0, 1);
end
coeffs=A\b;

c) Plot the function defined by the equation from b).

Hint.

If t is the vector of t-values, and you write

(t >= 0).*(t <= 1).*(1-2*abs(t-0.5))

you get the points φ1,1 (t).

Solution.

The function ψ̂ now is supported on [−2, 3], and can be plotted as follows:
120 4 Motivation for wavelets and some simple examples

t=linspace(-2,3,100);
plot(t, ( t>= 0).*(t <= 1).*(1-2*abs(t - 0.5)) ...
-coeffs(1)*(t >= -1).*(t <= 1).*(1 - abs(t))...
-coeffs(2)*(t >= 0).*(t <= 2).*(1 - abs(t - 1))...
-coeffs(3)*(t >= -2).*(t <= 0).*(1 - abs(t + 1))...
-coeffs(4)*(t >= 1).*(t <= 3).*(1 - abs(t - 2)))

d) Explain why the coordinate vector of ψ̂ in the basis (φ0 , ψ0 ) is

[ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, . . . , 0 − γ) ⊕ (1, 0, . . . , 0).

Hint.

The placement of −γ may seem a bit strange here, and has to do with that φ0,−1
−1
is not one of the basis functions {φ0,n }N
n=0 . However, we have that φ0,−1 = φ0,N −1 ,
i.e. φ(t + 1) = φ(t − N + 1), since we always assume that the functions we work with
have period N .

Solution.

The coordinates of φ0,0 , φ0,1 , and φ0,2 in ψ̂ are −α, −β, and −δ, respectively. Since
these three basis functions occur first in (φ0 , ψ0 ), they appear first in [ψ̂](φ0 ,ψ0 ) . Since
φ0,−1 = φ0,N −1 , and since this is the last basis function in the first half of (φ0 , ψ0 ),
−γ (the corresponding coordinate) is placed accordingly. Finally, ψ0,0 is th first basis
function in the second half of (φ0 , ψ0 ), so that a 1 (the coordinate) should be placed at
that location.
e) Compute the coordinates of ψ̂ in the basis φ1 (i.e. [ψ̂]φ1 ) with N = 8, i.e. compute
the IDWT of

[ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, 0, 0, 0, −γ) ⊕ (1, 0, 0, 0, 0, 0, 0, 0),

which is the coordinate vector you computed in d). For this, you should use the function
idwt_impl, with the kernel of the piecewise linear wavelet without symmetric extension
as input.

Solution.

The code which can be used looks like this:

g1=idwt_impl([-coeffs(1);-coeffs(2);-coeffs(4);0;0;0;0;-coeffs(3);...
1; 0; 0; 0; 0; 0; 0; 0], ’pwl0’, 1, ’per’);
g1 = [g1(14:16); g1(1:6)]; % Compact filter notation

Note that we have used a kernel which does not make symmetric extensions.
f) Sketch a more general procedure than the one you found in b), which can be used to
find wavelet bases where we have even more vanishing moments.
4 Motivation for wavelets and some simple examples 121

Solution.

If we define
K
X
ψ̂ = ψ0,0 − (αk φ0,−k − βk φ0,k+1 ) ,
k=0

we have 2k unknowns. These can be determined if we require 2k vanishing moments.

Exercise 4.32: Two vanishing moments for the Haar wavelet

Let φ be the scaling function of the Haar wavelet.

a) Compute projV0 f , where f (t) = t2 , and where f is defined on [0, N ).

Solution.

We have that

N
X −1 N
X −1 Z n+1
projV0 f = ht2 , φ0,n iφ0,n = t2 dtφ0,n
n=0 n=0 n
N −1 N −1
X 1 X
= ((n + 1)3 − n3 )φ0,n = (n2 + n + 1/3)φ0,n .
n=0
3 n=0

b) Find constants α, β so that ψ̂ = ψ − αφ0,0 − βφ0,1 has two vanishing moments, i.e. so
that hψ̂, 1i = hψ̂, ti = 0. Plot also the function ψ̂.

Hint.
R R R R
Start
R with computing
R the integrals ψ(t)dt, tψ(t)dt, φ0,0 (t)dt, φ0,1 (t)dt, and
tφ0,0 (t)dt, tφ0,1 (t)dt.

Solution.

We get that
122 4 Motivation for wavelets and some simple examples
Z
ψ(t)dt = 0
Z Z 1/2 Z 1
1/2
tψ(t)dt = tdt − tdt = [t2 /2]0 − [t2 /2]11/2 = 1/8 − 1/2 + 1/8 = −1/4
0 1/2
Z Z
φ0,0 (t)dt = φ0,1 (t)dt = 1
Z Z 1
tφ0,0 (t)dt = tdt = 1/2
0
Z Z 2
tφ0,1 (t)dt = tdt = 3/2.
1

hψ̂, 1i = hψ̂, ti = 0 can be written as the system

αhφ0,0 , 1i + βhφ0,1 , 1i = hψ, 1i

αhφ0,0 , ti + βhφ0,1 , ti = hψ, ti

so that

α+β =0 α/2 + 3β/2 = −1/4

which has the solution α = 1/4, β = −1/4. ψ̂ = ψ − φ0,0 /4 + βφ0,1 /4 can be plotted as
follows.

t=linspace(0,4,100);
phi = @(t) (0<=t).*(t<1);
psi = @(t) (0<=t).*(t<1/2) - (1/2<=t).*(t<1);
plot(t,psi(t)-phi(t)/4+phi(t-1)/4)

c) Express φ and ψ̂ with the help of functions from φ1 , and use this to write down the
change of coordinate matrix from (φ0 , ψ̂0 ) to φ1 .

Solution.

We have that

ψ̂ = ψ − φ0,0 /4 + βφ0,1 /4
1 1 1
= √ (φ1,0 − φ1,1 ) − √ (φ1,0 + φ1,1 ) + √ (φ1,2 + φ1,3 )
2 4 2 4 2
3 5 1 1
= √ φ1,0 − √ φ1,1 + √ φ1,2 + √ φ1,3
4 2 4 2 4 2 4 2
This means that the last half of the columns in the change of coordinates matrix are the
1
translates of 4√ 2
(3, −5, 1, 1). The remaining columns coincide with those of the matrix
for the Haar wavelet.
4 Motivation for wavelets and some simple examples 123

Exercise 4.33: More than two vanishing moments for the

Haar wavelet

It is also possible to add more vanishing moments to the Haar wavelet. Define

ψ̂ = ψ0,0 − a0 φ0,0 − · · · − ak−1 φ0,k−1 .

R l+1 R1
Define also cr,l = l
tr dt, and er = 0 tr ψ(t)dt.

a) Show that ψ̂ has k vanishing moments if and only if a0 , . . . , ak−1 solves the system
    
c0,0 c0,1 · · · c0,k−1 a0 e0
 c1,0 c1,1 · · · c1,k−1   a1   e1 
  ..  =  .. 
    
 .. .. .. ..
 . . . .   .   . 
ck−1,0 ck−1,1 · · · ck−1,k−1 ak−1 ek−1

Solution.

Multiplying with tr and computing the integral we obtain

hψ̂, tr i = hψ0,0 , tr i − a0 hφ0,0 , tr i − · · · − ak−1 hφ0,k−1 , tr i.

Setting those to zero we obtain

0 = er − a0 cr,0 − · · · − ak−1 cr,k−1 ,

which is row r in the stated equation system.
b) Write a function vanishing_moms_haar which takes k as input, solves the system
in a), and returns the vector a = (a0 , a1 , . . . , ak−1 ).

Solution.

1
We first find that cr,l = r+1 ((l + 1)r+1 − lr+1 ), and

1/2 1
er = [tr+1 /(r + 1)]0 (2−r − 1).
− [tr+1 /(r + 1)]11/2 =
r+1
The following code can be used. If you set k = 2 the values from the previous exercise
are obtained.

function sol=vanishing_moms_haar(k)
l=0:(k-1);
r=l’;
C = ((l+1).^(r+1)-l.^(r+1))./(r+1);
e = (2.^(-r)-1)./(r+1);
sol = C\e;
end
124 4 Motivation for wavelets and some simple examples

Exercise 4.34: Listening experiments

Run the function forw_comp_rev_dwt1 for different m for the Haar wavelet, the piecewise
linear wavelet, and the alternative piecewise linear wavelet, but listen to the detail
components instead. Describe the sounds you hear for different m, and try to explain
why the sound seems to get louder when you increase m.

Solution.

The following code can be used:

[x, fs] = forw_comp_rev_dwt1(m, ’Haar’);

playerobj = audioplayer(x, fs);
playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’pwl0’);

playerobj = audioplayer(x, fs);
playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’pwl2’);

playerobj = audioplayer(x, fs);
playblocking(playerobj)
Chapter 5
The filter representation of wavelets

Exercise 5.4: Symmetry of MRA matrices vs. symmetry of

filters

a) Find two symmetric filters, so that the corresponding MRA-matrix is not symmetric.

Solution.

You can set for instance H0 = {1/4, 1/2, 1/4}, and H1 = {1} (when you write down the
corresponding matrix you will see that A0,1 = 1/2, A1,0 = 0, so that the matrix is not
symmetric)
b) Assume that an MRA-matrix is symmetric. Are the corresponding filters H0 , H1 ,
G0 , G1 also symmetric? If not, find a counterexample.

Solution.

The Haar wavelet is a counterexample.

Exercise 5.5: Passing between the MRA-matrices and the

filters

a) Assume that we have the forward filter bank transform

 
1/5 1/5 1/5 0 0 0 · · · 0 1/5 1/5
−1/3 1/3 −1/3 0 0 0 ··· 0 0 0 
 
 1/5 1/5 1/5 1/5 1/5 0 · · · 0 0 0 
H= 
 0 0 −1/3 1/3 −1/3 0 · · · 0 0 0 
.. .. .. .. .. .. .. .. .. ..
 
. . . . . . . . . .
Write down the corresponding filters H0 , H1 , and compute and plot the frequency
responses. Are the filters symmetric?

125
126 5 The filter representation of wavelets

Solution.

We have that H0 = 15 {1, 1, 1, 1, 1}, and H1 = 13 {−1, 1, −1}. The frequency responses are

1 2iω 1 iω 1 1 −iω 1 −2iω

λH0 (ω) = e + e + + e + e
5 5 5 5 5
2 2 1
= cos(2ω) + cos ω +
5 5 5
1 iω 1 1 −iω 2 1
λH1 (ω) = − e + − e = − cos ω + .
3 3 3 3 3
Both filters are symmetric.
b) Assume that we have the reverse filter bank transform
 
1/2 −1/4 0 0 ···
1/4 3/8 1/4 1/16 · · ·
 
 0 −1/4 1/2 −1/4 · · ·
 
 0 1/16 1/4 3/8 · · ·
 
 0 0 0 −1/4 · · ·
G= 0
 
 0 0 1/16 · · ·
 0
 0 0 0 · · ·
 .. .. .. .. .. 
 . . . . . 
 
 0 0 0 0 · · ·
1/4 1/16 0 0 ···
Write down the filters G0 , G1 , and compute and plot the frequency responses. Are the
filters symmetric?

Solution.

We have that G0 = {1/4, 1/2, 1/4}, and G1 = {1/16, −1/4, 3/8, −1/4, 1/16}. The
frequency responses are

1 iω 1 1 −iω
λG0 (ω) = e + + e
4 2 4
1 1
= cos(ω) +
2 2
1 2iω 1 iω 3 1 −iω 1 −2iω
λG1 (ω) = e − e + − e e
16 4 8 4 16
1 1 3
= cos(2ω) − cos ω + .
8 2 8
Both filters are symmetric.
c) Assume that H0 = {1/16, 1/4, 3/8, 1/4, 1/16}, and H1 = {−1/4, 1/2, −1/4}. Plot
the frequency responses of H0 and H1 , and verify that H0 is a low-pass filter, and that
H1 is a high-pass filter. Also write down the corresponding forward filter bank transform
H.
5 The filter representation of wavelets 127

Solution.

The frequency responses are

1 2iω 1 iω 3 1 −iω 1 −2iω

λH0 (ω) = e + e + + e e
16 4 8 4 16
1 1 3
= cos(2ω) + cos ω +
8 2 8
1 iω 1 1 −iω
λH1 (ω) = − e + − e
4 2 4
1 1
= − cos(ω) + .
2 2
The two first rows in H are
 
3/8 1/4 1/16 0 · · · 1/16 1/4
−1/4 1/2 −1/4 0 · · · 0 0
The remaining rows are obtained by translating these in alternating order.
d) Assume that G0 = 13 {1, 1, 1}, and G1 = 15 {1, −1, 1, −1, 1}. Plot the frequency
responses of G0 and G1 , and verify that G0 is a low-pass filter, and that G1 is a
high-pass filter. Also write down the corresponding reverse filter bank transform G.

Solution.

The frequency responses are

1 iω 1 1 −iω 2 1
λG0 (ω) = e + + e = cos ω +
3 3 3 3 3
1 2iω 1 iω 1 1 −iω 1 −2iω
λG1 (ω) = e − e + − e e
5 5 5 5 5
2 2 1
= cos(2ω) − cos ω +
5 5 5
The two first columns in G are
 
1/3 −1/5
1/3 1/5 
 
 0
 −1/5
 0 1/5 
 
 0 0 
 
 .. .. 
 . . 
 
 0 0 
1/3 1/5
The remaining columns are obtained by translating these in alternating order.
128 5 The filter representation of wavelets

Exercise 5.6: Computing by hand

In Exercise 4.14 we computed the DWT of two very simple vectors x1 and x2 , using
the Haar wavelet.
a) Compute H0 x1 , H1 x1 , H0 x2 , and H1 x2 , where H0 and H1 are the filters used by
the Haar wavelet.

Solution.

Recall that x1 consisted of 16 consecutive ones followed by 16 consecutive zeros, repeated

so that the final vector has length 256. Also, x2 was x1 delayed by 1. Using the expressions
for the filters H0 and H1 of the Haar wavelet we obtain

√ √ √ √
H0 x1 = (2/ 2, ..., 2/ 2, 1/ 2, 0, ..., 0, 1/ 2, ...)
| {z } | {z }
16 16
√ √
H1 x1 = (−1/ 2, 0, ..., 0, 1/ 2, 0, ..., 0, ...)
| {z } | {z }
16 16
√ √ √ √
H0 x2 = (1/ 2, 2/ 2, ..., 2/ 2, 1/ 2, 0, ..., 0, ...)
| {z } | {z }
16 16
√ √
H1 x2 = (0, −1/ 2, 0, ..., 0, 0, 1/ 2, 0, ..., 0, ...),
| {z } | {z }
16 16

where the underbraced vectors are repeated 8 times.

b) Compare the odd-indexed elements in H1 x1 with the odd-indexed elements in H1 x2 .
From this comparison, attempt to find an explanation to why the two vectors have very
different detail components.

Solution.

In H1 x1 the only nonzero entries are at indices 0, 16, 32, and so on, while for H1 x2 they
are at indices 1, 17, 33, and so on. Since the odd indices give the detail components, it
follows that the detail of x1 is zero, while the detail of x2 is non-zero.

Exercise 5.7: Code for applying wavelets to sound

Suppose that we run the following algorithm on the sound represented by the vector x:

N=size(x,1);
c = (x(1:2:N, :) + x(2:2:N, :))/sqrt(2);
w = (x(1:2:N, :) - x(2:2:N, :))/sqrt(2);
newx = [c; w];
newx = newx/max(abs(newx));
playerobj=audioplayer(newx,44100);
playblocking(playerobj)
5 The filter representation of wavelets 129

a) Comment the code and explain what happens. Which wavelet is used? What do the
vectors c and w represent? Describe the sound you believe you will hear.

Solution.

c and w represent the coordinates in the wavelet bases φ0 and ψ0 . The code runs a
Haar wavelet transform. The sound is normalized so that the sound samples lie in the
range between −1 and 1, and the resulting sound is played. The sound is split into two
parts, and c represents a low-resolution version of the sound (with half the number of
samples), so that we first will hear the sound played at double pace. After this we will
hear the detail w in the sound, also played at double pace. We should also be able to
recognize the sound from this detail.
b) Assume that we add lines in the code above which sets the elements in the vector w
to 0 before we compute the inverse operation. What will you hear if you play this new
sound?

Solution.

This corresponds to reconstructing a low-resolution approximation of the sound.

Exercise 5.8: Computing filters and frequency responses

a) Write down the filter coefficients for the corresponding filter G1 obtained in Exer-
cise 4.31, and plot its frequency response.

Solution.

The code can look as follows:

omega = linspace(0,2*pi,100);
plot(omega, g1(5) + g1(6)*2*cos(omega) + g1(7)*2*cos(2*omega)...
+ g1(8)*2*cos(3*omega) + g1(9)*2*cos(4*omega))

b) Write down the corresponding filters G0 and G1 for Exercise 4.32. Plot their frequency
responses, and characterize the filters as low-pass- or high-pass.

Solution.

The filter G0 is the same as for the Haar wavelet, and this frequency response has already
1
been plotted. In Exercise 4.32 we obtained G1 = 4√ 2
(3, −5, 1, 1), and the magnitude of
this frequency response can be plotted as follows.

omega=linspace(0,2*pi,100);
plot(omega, abs(fft([[3; -5; 1; 1]/(4*sqrt(2)); zeros(96, 1)])));
130 5 The filter representation of wavelets

c) Repeat b) for the Haar wavelet as in Exercise 4.33, and plot the corresponding
frequency responses for k = 2, 4, 6.

Solution.

Using the function we implemented in Exercise 4.33, the following code can be used.

for k = [2 4 6]
x = vanishing_moms_haar(k);
x = idwt_impl([-x; zeros(50-length(x),1); 1; zeros(49,1)],’haar’,1,’per’);
figure()
plot(omega, abs(fft(x)));
end

Exercise 5.9: Perfect reconstruction for forward and reverse

filter bank transforms

Combine equations (5.4) and (5.5) in Theorem 3 in the book to show that

1
GHφr = (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr
2
1
+ (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .
2
Conclude that, in terms of continuous frequency responses, we have alias cancellation if
and only if

λH0 (ω)λG0 (ω + π) = λH1 (ω)λG1 (ω + π),

and perfect reconstruction if and only if in addition

λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = 2.

These are also called the alias cancellation- and perfect reconstruction conditions, re-
spectively.

Solution.

We have that
5 The filter representation of wavelets 131

GHφr
1 1
= (λH0 ,r + λH1 ,r )Gφr + (λH0 ,r − λH1 ,r )Gφr+N/2
2  2 
1 1 1
= (λH0 ,r + λH1 ,r ) (λG0 ,r + λG1 ,r )φr + (λG0 ,r+N/2 − λG1 ,r+N/2 )φr+N/2
2 2 2
 
1 1 1
+ (λH0 ,r − λH1 ,r ) (λG0 ,r+N/2 + λG1 ,r+N/2 )φr+N/2 + (λG0 ,r − λG1 ,r )φr )
2 2 2
1
= ((λH0 ,r + λH1 ,r )(λG0 ,r + λG1 ,r ) + (λH0 ,r − λH1 ,r )(λG0 ,r − λG1 ,r )) φr
4
1

+ (λH0 ,r + λH1 ,r )(λG0 ,r+N/2 − λG1 ,r+N/2 ) + (λH0 ,r − λH1 ,r )(λG0 ,r+N/2 + λG1 ,r+N/2 ) φr+N/2
4
1 1
= (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr + (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .
2 2

Exercise 5.10: Conditions for perfect reconstruction

In this exercise we will prove that, when G0 and H0 are given filters which satisfy a
certain property, we can define unique (up to a constant) filters H1 and G1 so that all
four filters together give perfect reconstruction. In Section 7.1 in the book we will also
prove that, as long as all filters are FIR, these conditions are also necessary.
a) Show that, if there exist α ∈ R and d ∈ Z so that

(H1 )n = (−1)n α−1 (G0 )n−2d

(G1 )n = (−1)n α(H0 )n+2d
2 = λH0 ,n λG0 ,n + λH0 ,n+N/2 λG0 ,n+N/2

then we also have that

λH1 (ω) = α−1 e−2idω λG0 (ω + π)

λG1 (ω) = αe2idω λH0 (ω + π)
2 = λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π)

Solution.

We have that

X X
λH1 (ω) = (H1 )k e−ikω = (−1)k α−1 (G0 )k−2d e−ikω
k k
X X
−1
=α (−1) (G0 )k e−i(k+2d)ω = α−1 e−2idω
k
(G0 )k e−ik(ω+π)
k k
−1 −2idω
=α e λG0 (ω + π),
132 5 The filter representation of wavelets

which proves the first equation. A similar computation proves the second equation. The
last equation is simply a restatement in terms of continuous frequency responses.
b) Show that the first two conditions in a) implies the alias cancellation condition from
Exercise 5.9, and that the last condition implies the perfect reconstruction condition
from Exercise 5.9.

Solution.

The first two conditions imply the alias cancellation condition is met since

λH1 (ω)λG1 (ω + π) =α−1 e−2idω λG0 (ω + π)(α)e2id(ω+π λH0 (ω)

=λH0 (ω)λG0 (ω + π).

Inserting this in the third condition we get

2 = λH0 (ω)λG0 (ω) + λG0 (ω + π)λH0 (ω + π)

= λH0 (ω)λG0 (ω) + α−1 e−2idω λG0 (ω + π)αe2idω λH0 (ω + π)
= λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω),

so that the perfect reconstruction condition also is met.

c) Show that if the conditions in a) are met, and the H0 , H1 , G0 , G1 are non-trivial
symmetric FIR filters, then necessarily d = 0. It follows that the conditions take the
form

λH1 (ω) = α−1 λG0 (ω + π)

λG1 (ω) = αλH0 (ω + π)
2 = λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π),

Solution.

Since H0 is symmetric, (H0 )n = (H0 )−n , and from the equations in a) it follows that

(G1 )n−2d = (−1)n−2d α(H0 )n = (−1)n α−1 (H0 )−n

= (−1)(−n−2d) α−1 (H0 )(−n−2d)+2d = (G1 )−n−2d

This shows that G1 is symmetric about both −2d, in addition to being symmetric about
0 (by assumption). This means that the filter coefficients take the form of a symmetric
extension if d 6= 0, as there two different symmetry points then. Mirroring about the
symmetry points it is clear that there must be infinitely many filter coefficients, which
is a contradiction. We must thus have that d = 0.
5 The filter representation of wavelets 133

Exercise 5.11: Finding the DWT filter components from the

IDWT filter components

Assume that H and G give perfect reconstruction. It may be that only one of H and G
is known, and that we would like to find an expression for the other. To this end we will
assume that the conditions in Exercise 5.10a) are met for some d and α.
a) Assume that G0 , G1 are known and symmetric (so that d = 0). Deduce that
X
α= (−1)n (G0 )n (G1 )n .
n

Deduce a similar formula for the case when H0 and H1 are known.

Solution.

From the equations in Exercise 5.10a) it follows that

X X X
1= (G1 )n (H1 )n = (G1 )n α−1 (−1)n (G0 )n = α−1 (−1)n (G0 )n (G1 )n ,
n n n

so that α = n (−1)n (G0 )n (G1 )n . On the other hand, if H0 , H1 are known instead, we
P
have similarly that

X X X
1= (G1 )n (H1 )n = α(−1)n (H0 )n (H1 )n = α (−1)n (H0 )n (H1 )n ,
n n n

so that α = 1/( n (−1)n (H0 )n (H1 )n ).

P

b) Use a) to verify the expressions for the filters H0 and H1 of the alternative piecewise
linear wavelet from Example 5.3 in the book.

Solution.

All filters for the alternative piecewise linear wavelet are symmetric. We get that
    
X
n 1 1 1 3 1 1 1
α= (−1) (G0 )n (G1 )n = − − +1· − − = .
n
2 2 4 4 2 4 2
We now get

(H0 )n = α−1 (−1)n (G1 )n = 2(−1)n (G1 )n

(H1 )n = α−1 (−1)n (G0 )n = 2(−1)n (G0 )n ,

so that

√
H0 = 2{−1/8, 1/4, 3/4, 1/4, −1/8}
√
H1 = 2{−1/2, 1, −1/2},
134 5 The filter representation of wavelets

which is what we found in Example 5.3 in the book.

Exercise 5.12: Classical QMF filter bank transforms

A classical QMF filter bank is one where G0 = H0 and G1 = H1 (i.e. the filters in the
forward and reverse transforms are equal), and

λH1 (ω) = λH0 (ω + π).

QMF is short for Quadrature Mirror Filter.
a) Show that, for a classical QMF filter bank transform, the two alias cancellation
conditions in Exercise 5.10a) are satisfied with α = 1, d = 0.

Solution.

We have that

λH1 (ω) = λH0 (ω + π) = λG0 (ω + π)

λG1 (ω) = λH1 (ω) = λH0 (ω + π)

Comparing with the alias cancellation conditions in Exercise 5.10a), we see that we can
set α = 1 and d = 1.
b) Show that, for classical QMF filter bank transforms, the perfect reconstruction
condition in Exercise 5.10a) can be written as

λH0 (ω)2 + λH0 (ω + π)2 = 2.

Solution.

This follows from

λH0 (ω)λG0 (ω) + λH0 (ω + π)λG0 (ω + π) = λH0 (ω)2 + λH1 (ω)λH0 (ω + π)

= λH0 (ω)2 + λH0 (ω + π)2 .

c) Show that it is impossible to find classical QMF filter banks with non-trivial FIR-
filters, and which give perfect reconstruction.

Solution.

Let tk be the nonzero filter coefficient with smallest index k. We have that
5 The filter representation of wavelets 135
X
λH0 (ω)2 = t2k e−2ikω + pr e−irω
r<−2k
X
λH0 (ω + π) = 2
t2k e−2ikω + qr e−irω ,
r<−2k

where pr and qr are numbers. Adding these equations we obtain

X
λH0 (ω)2 + λH0 (ω + π)2 = 2t2k e−2ikω + (pr + qr )e−irω .
r<−2k
−2ikω
Since the term e contributes, in order for this to sum to 2 we must have that k = 0,
so that t0 is the nonzero filter coefficient with smallest index k. By instead considering
the nonzero filter coefficient with largest index k, we prove in the same way that this
also is t0 , so that t0 can be the only nonzero filter coefficient, i.e. the filter is trivial,
which is a contradiction.
Despite this negative result, classical QMF filter banks are still useful, as one can
construct such filter banks which give near-perfect reconstruction. We shall see in
Section 7.3 in the book that the MP3 standard take use of such filters, and this explains
our previous observation that the MP3 standard does not give perfect reconstruction.

Exercise 5.13: Alternative QMF filter bank transforms

An alternative QMF filter bank is one where G0 = (H0 )T and G1 = (H1 )T (i.e. the filter
coefficients in the forward and reverse transforms are reverse of one-another), and

λH1 (ω) = λH0 (ω + π).

There is some confusion in the literature between classical and alternative QMF filter
banks.
a) Show that, for an alternative QMF filter bank, the two alias cancellation conditions
in Exercise 5.10a) are satisfied with α = 1, d = 0.

Solution.

If we in the first alias cancellation condition substitute G0 = (H0 )T we get

λH1 (ω) = α−1 e−2idω λG0 (ω + π) = α−1 e−2idω λH0 (ω + π).

If we set α = 1, d = 0, we get equality here. A similar computation follows for the second
alias cancellation condition.
b) Show that the perfect reconstruction condition for alternative QMF filter banks can
be written as

|λH0 (ω)|2 + |λH0 (ω + π)|2 = 2.

136 5 The filter representation of wavelets

Solution.

We have that

2 = λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = λH0 (ω)λH0 (ω) + λH0 (ω + π)λH0 (ω + π)
= |λH0 (ω)|2 + |λH0 (ω + π)|2 .

We see that the perfect reconstruction conditions of the two definitions of QMF filter
banks only differ in that the latter takes absolute values. It turns out that the latter
also has many interesting solutions, as we will see in Chapter 6 in the book.

Exercise 5.14: Orthogonal filter bank transforms

Show that H is orthogonal (i.e. G = H T ) if and only if the filters satisfy G0 =

(H0 )T , G1 = (H1 )T ,
In the literature, a wavelet is called orthonormal if the filters satisfy G0 = (H0 )T , G1 =
(H1 )T . Thus, alternative QMF filter banks with perfect reconstruction are examples of
orthonormal wavelets.

Solution.

This can be proved simply by observing that, if we transpose the DWT-matrix we get
an IDWT matrix with filters (H0 )T , (H1 )T , and this is equal to the IDWT if and only if
G0 = (H0 )T , G1 = (H1 )T .

Exercise 5.15: The Haar wavelet as an alternative QMF filter

bank transform

Show that the Haar wavelet satisfies λH1 (ω) = −λH0 (ω + π), and G0 = (H0 )T , G1 =
(H1 )T . The Haar wavelet thus differs from an alternative QMF filter bank only in a sign.
Despite this sign difference, the Haar wavelet is often called an alternative QMF filter
bank transform in the literature. The additional sign leads to an orthonormal wavelet
with α = −1, d = 0 instead.

Solution.

We have computed that

1 1
λH0 (ω) = √ (1 + eiω ) λH1 (ω) = √ (e−iω − 1).
2 2
It follows that
5 The filter representation of wavelets 137

1 1
λH0 (ω + π) = √ (1 − eiω ) = √ (1 − e−iω ) = −λH1 (ω).
2 2
This proves the result.

Exercise 5.16: The support of the scaling function and the

mother wavelet

The scaling functions and mother wavelets we encounter will always turn out to be
functions with compact support (for a general proof of this, see [1]). An interesting
consequence of equations (5.2) and (5.3) in the book is that the corresponding filters
then are FIR, and that one can find a connection between the supports of the scaling
function and mother wavelet, and the number of filter coefficients. In the following
we will say that the support of a filter is [E, F ] if tE , ..., tF are the only nonzero filter
coefficients.
a) Assume that G0 has support [M0 , M1 ], G1 has support [N0 , N1 ]. Show that Supp(φ) =
[M0 , M1 ], and Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2].

Solution.

Let [m0 , m1 ] be the support of φ. Then φ1,n clearly has support [(m0 + n)/2, (m1 + n)/2].
In the equation
M1
X
φ(t) = (G0 )n φ1,n ,
n=M0

the function with the leftmost support is φ1,M0 , while the one with the rightmost one is
φ1,M1 . This gives the support [(m0 + M0 )/2, (m1 + M1 )/2] for the sum. In order for the
supports of the two sides to match we clearly must have

m0 = (m0 + M0 )/2 m1 = (m1 + M1 )/2.

It follows that m0 = M0 and m1 = M1 , so that the support of φ is [M0 , M1 ]. Similarly,

let [n0 , n1 ] be the support of ψ. We have that
NX
1 +1

ψ(t) = (G1 )n−1 φ1,n ,

n=N0 +1

and we get in the same way

n0 = (m0 + N0 + 1)/2 n1 = (m1 + N1 + 1)/2.

It follows that n0 = (M0 + N0 + 1)/2. n1 = (M1 + N1 + 1)/2, so that the support of ψ

is ((M0 + N0 + 1)/2, (M1 + N1 + 1)/2).
b) Assume that all the filters are symmetric. Show that Supp(φ) is on the form
[−M1 , M1 ] (i.e. symmetric around 0), and that Supp(ψ) is on the form 1/2 + [−(M1 +
138 5 The filter representation of wavelets

N1 )/2, (M1 + N1 )/2], i.e. symmetric around 1/2. Verify that the alternative piecewise
linear wavelet has Supp(φ) = [−1, 1], and Supp(ψ) = [−1, 2] (as we already know from
Figure 4.18 in the book), and that the piecewise linear wavelet has Supp(ψ) = [0, 1].

Solution.

Supp(φ) is on the stated form due to a), and since the support of a symmetric G0 has
the form [−M1 , M1 ]. Since N0 = −N1 as well when G1 is symmetric, we have that

Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2]

= [(−M1 − N1 + 1)/2, (M1 + N1 + 1)/2]
= 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2].

For both piecewise linear wavelets M1 = 1 so that Supp(φ) = [−1, 1]. For the first
piecewise linear wavelet we had N0 = N1 = 0 so that

Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−1/2, 1/2] = [0, 1].

For the alternative piecewise linear wavelet we had N0 = −2, N1 = 2, so that

Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−3/2, 3/2] = [−1, 2].

c) The wavelet with symmetric filters with most filter coefficients we will consider has
7 and 9 filter coefficients, respectively. Explain why plotting over [−4, 4] captures the
entire support of both the scaling function and the mother wavelet when there are at
most this number of filter coefficients. This explains why [−4, 4] has been used as the
plotting interval in many of the plots.

Solution.

With 7 and 9 filter coefficients it is easily seen that the support of φ is [−3, 3], and the
support of ψ is [−3, 4]. Both these are contained in [−4, 4].
d) Verify that, for the Haar wavelet, G0 has filter coefficients evenly distributed around
1/2, G1 has equally many, and evenly distributed around −1/2, and the supports of
both φ and ψ are symmetric around 1/2. The Haar wavelet is the simplest orthonormal
wavelet, and these properties will be used as a template for the other orthonormal
wavelets.

Solution.

We know that both φ and ψ for the Haar wavelet are supported on [0, 1), so that 1/2
is the midpoint of the support. The symmetries around 1/2 and −1/2 follows from
G0 = √12 {1, 1} and G1 = √12 {1, −1}.
5 The filter representation of wavelets 139

e) We will only consider orthonormal wavelets with at most 8 filter coefficients. Explain
again why [−4, 4] as plotting interval is desirable.

Solution.

This number of filter coefficients is easily seen to give the support [−3, 4], which again is
contained within [−4, 4].

Exercise 5.19: Using the cascade algorithm

In Exercise 4.31 we constructed a new mother wavelet ψ̂ for piecewise linear functions
by finding constants α, β, γ, δ so that

ψ̂ = ψ − αφ0,0 − βφ0,1 − δφ0,2 − γφ0,N −1 .

Use the cascade algorithm to plot ψ̂. Do this by using the wavelet kernel for the piecewise
linear wavelet (do not use the code above, since we have not implemented kernels for
this wavelet yet).

Solution.

Assuming that the vector coeffs has been set as in Exercise 4.31, the code can look as
follows

m = 10;
t = linspace(-2, 6, 8*2^m);
coordsvm=2^(m/2)*idwt_impl([-coeffs(1); -coeffs(2); -coeffs(4); 0;...
0; 0; 0; -coeffs(3); 1; 0; 0; 0; 0; 0;...
0; 0; zeros(8*2^m-16, 1)], ...
’pwl0’, m, ’per’);
plot(t, coordsvm([(6*2^m+1):(8*2^m) 1:(6*2^m)]))

Exercise 5.20: Implementing the dual filter bank transforms

a) Show that ATλ = Bλ and BλT = Aλ , i.e. that the transpose of an elementary lifting
matrix of even/odd type is an elementary lifting matrix of odd/even type.

Solution.

This follows from the fact that the nonzero off-diagonal elements in Aλ come at the
indices (2i, (2i ± 1) mod N ), while the nonzero off-diagonal elements in Bλ come at the
indices ((2i ± 1) mod N, 2i).
b) Assume that we have a factorization of H in terms of elementary lifting matrices.
Use a) to show that the dual forward transform is obtained by replacing each Aλ with
B−λ , Bλ with A−λ , and k with 1/k in the multiplication step. These replacements are
140 5 The filter representation of wavelets

done by the function set_wav_props in the file find_wav_props.m in the library, on

an object which is accessed by the general kernel.

Solution.

The kernel of the dual forward transform is GT = (H −1 )T . Transposing and inverting

means that the terms are reversed twice, so that the order is the same. Transposing
transforms odd lifting steps to even lifting steps and vice versa (according to a)), and
inversion adds the minus sign. The result follows.
c) Previously we expressed the forward and reverse transforms of the piecewise linear
wavelets in terms of elementary lifting operations. Use b) to write down the dual forward
and reverse transforms of these two wavelets in terms of lifting matrices.

Solution.

For the piecewise linear wavelet we had

√ 1
H= 2B−1/2 G = √ B1/2 .
2
This implies that the forward and reverse transforms for the dual wavelet are

1 1
H̃ = GT = √ (B1/2 )T = √ A1/2
2 2
T
√ T
√
G̃ = H = 2(B−1/2 ) = 2A−1/2 .

For the alternative piecewise linear wavelet we had

√ 1
H= 2A1/4 B−1/2 G = √ B1/2 A−1/4 ,
2
and we get similarly

1 1
H̃ = GT = √ (B1/2 A−1/4 )T = √ B−1/4 A1/2
2 2
T
√ T
√
G̃ = H = 2(A1/4 B−1/2 ) = 2A−1/2 B1/4 .

Exercise 5.21: Transpose of the DWT and IDWT

Explain why
• The transpose of the DWT can be computed with an IDWT with the kernel of the
dual IDWT
• The transpose of the dual DWT can be computed with an IDWT with the kernel of
the IDWT
5 The filter representation of wavelets 141

• The transpose of the IDWT can be computed with a DWT with the kernel of the
dual DWT
• The transpose of the dual IDWT can be computed with a DWT with the kernel of
the DWT

Solution.

Assume that the kernel transformations of the DWT and the IDWT are H and G,
respectively. For the transpose of the DWT the point is that, while its kernel transfor-
mation is H T , we compose the kernel with a permutation matrix when we compute the
DWT. When we transpose, the order of the kernel and the permutation changes, so the
transpose must use an IDWT implementation instead. The remaining statements follow
similarly, since the kernel of the transpose of the dual DWT is, G that of the transpose
of the IDWT is GT , and that of the transpose of the dual IDWT is H.

Exercise 5.22: Symmetric filters preserve symmetric

extensions

Prove that symmetric filters preserve symmetric vectors in the sense of Definition 10
in the book also. Show also more generally that, if x ∈ R2N is symmetric around any
"half-number" (i.e. 1/2, 1, 3/2, 2,..,), then Sx is symmetric around the same half-number.
Use Theorem 14 in the book as a guide.

Solution.

If x has symmetry around t, i.e. (xt−n = xt+n ), we obtain

X X
zt+n = tk xt+n−k = t0 xt+n + tk (xt+n−k + xt+n+k )
k k≥1
X X
zt−n = tk xt−n−k = t0 xt−n + tk (xt−n−k + xt−n+k )
k k≥1
X
= t0 xt+n + tk (xt+n+k + xt+n−k ),
k≥1

and we see that z has the same symmetry. This argument also works when t is a
half-number. Then everything makes sense when requiring that n is a half-number.

Exercise 5.23: Implementing symmetric extensions

a) The function you implemented in Exercise 3.28 computed circular convolution, i.e. it
assumed a periodic extension of the input. Reimplement that function so that it also
takes a third parameter bd_mode. If this equals "per" a periodic extension should be
assumed. If it equals "symm" a symmetric extension should be assumed, as described
142 5 The filter representation of wavelets

above. Again, the function filter_impl in the library can serve as a template for your
code.

Solution.

The implementation of filter_impl in the library looks as follows, and handles other
boundary modes which we do not cover here as well:

function x=filter_impl(t, x, bd_mode)

N0 = (length(t) - 1)/2;
N = size(x, 1);
szx = size(x);
n = prod(szx(2:end));
if strcmpi(bd_mode, ’symm’)
y = [x((N0+1):(-1):2, :) ; x(:,:); x((N-1):(-1):(N - N0), :)];
elseif strcmpi(bd_mode, ’per’)
y = [x((N - N0 + 1):N, :); x(:,:); x(1:N0, :)];
elseif strcmpi(bd_mode, ’none’) || strcmpi(bd_mode, ’bd’)
y = [ zeros(N0, n); x(:, :); zeros(N0, n)];
end
for k=1:size(y,2)
z = conv(t, y(:,k));
x(:,k) = z((2*N0+1):(length(z)-2*N0));
end
end

We see that the function conv is called

b) Implement functions

dwt_kernel_filters(x, bd_mode, wav_props)

idwt_kernel_filters(x, bd_mode, wav_props)

which return DWT and IDWT kernels that apply the function filter_impl from a)
together with Theorem 2 in the book. The parameter wav_props should be an object
which gives access to the filters by means of writing wav_props.h0, wav_props.h1,
wav_props.g0, and wav_props.g1.

Solution.

The implementation of these functions in the library is as follows:

function x=dwt_kernel_filters(x, bd_mode, dual_wav_props)

if strcmpi(bd_mode, ’bd’)
y1 = dual_wav_props.A_L’*x(1:size(dual_wav_props.A_L,1), :);
y2 = dual_wav_props.A_R’*x((end-size(dual_wav_props.A_R,1)+1):end, :);
end
x0 = filter_impl(dual_wav_props.g0, x, bd_mode);
x1 = filter_impl(dual_wav_props.g1, x, bd_mode);
x(1:2:end,:) = x0(1:2:end,:);
x(2:2:end,:) = x1(2:2:end,:);
if strcmpi(bd_mode, ’bd’)
x(1:size(dual_wav_props.A_L,2), :) ...
= x(1:size(dual_wav_props.A_L,2), :) + y1;
x((end-size(dual_wav_props.A_R,2)+1):end, :) ...
= x((end-size(dual_wav_props.A_R,2)+1):end, :) + y2;
end
end
5 The filter representation of wavelets 143

function x=idwt_kernel_filters(x, bd_mode, wav_props)

if strcmpi(bd_mode, ’bd’)
y1 = wav_props.A_L*x(1:size(wav_props.A_L,2), :);
y2 = wav_props.A_R*x((end-size(wav_props.A_R,2)+1):end, :);
end
x0 = x; x0(2:2:end,:) = 0;
x1 = x; x1(1:2:end,:) = 0;
x0 = filter_impl(wav_props.g0, x0, bd_mode);
x1 = filter_impl(wav_props.g1, x1, bd_mode);
x = x0 + x1;
if strcmpi(bd_mode, ’bd’)
x(1:size(wav_props.A_L,1), :) ...
= x(1:size(wav_props.A_L,1), :) + y1;
x((end-size(wav_props.A_R,1)+1):end, :) ....
= x((end-size(wav_props.A_R,1)+1):end, :) + y2;
end
end

Using these functions one can define standard DWT and IDWT kernels in the following
way, assuming that the wav_props has been computed:

dwt_kernel = @(x, bd_mode) dwt_kernel_filters(x, bd_mode, wav_props);

idwt_kernel = @(x, bd_mode) idwt_kernel_filters(x, bd_mode, wav_props);

Exercise 5.24: The boundary mode in the cascade algorithm

In Exercise 5.19 we did not use symmetric extensions (the bd_mode-argument was set
to per). Attempt to use symmetric extensions instead (set the bd_mode-argument to
"symm"), and observe the new plots you obtain. Can you explain why these new plots
do not show the correct functions, while the previous plots are correct?

Solution.

When using symmetric extensions, we really consider a function which is made symmetric
at the boundaries, and this will alter the scaling function or the mother wavelet we are
interested in, so that the resulting plot may differ if the function hits the boundaries. A
periodic extension on the other hand should be unproblematic, as long as the support
fits into the interval we plot over. The support is simply split into a left and a right part
if the function hits a boundary in this case.

Exercise 5.25: Restricted matrices for elementary lifting

Show that
144 5 The filter representation of wavelets
 
1 2λ 0 0 · · · 0 0 0
0 1 0 0 · · · 0 0 0 
 
0 λ 1 λ · · · 0 0 0 
(Aλ )r =  . . . . . . . . 
 
 .. .. .. .. .. .. .. .. 
 
0 0 0 0 · · · λ 1 λ
0 0 0 0 ··· 0 0 1
 
1 0 0 0 ··· 0 0 0
λ 1 λ 0 · · · 0 0 0
 
 0 0 1 0 · · · 0 0 0
(Bλ )r =  . . . . . . . .  .
 
 .. .. .. .. .. .. .. .. 
 
 0 0 0 0 · · · 0 1 0
0 0 0 0 · · · 0 2λ 1

Also, change the implementations of lifting_even_symm and lifting_odd_symm so

that these expressions are used (rather than Aλ , Bλ ) when the bd_mode parameter
equals "symm".

Solution.

Elementary lifting matrices can be viewed as wavelet transforms where the filters are
{λ, 1, λ} and {1}. Since these are symmetric, Theorem 11 in the book guarantees that
they preserve symmetric extensions.

Exercise 5.26: Expression for Sr

Show that, with

 
S1 S2
S= ∈ R2N −2 × R2N −2
S3 S4
a symmetric filter, with S1 ∈ RN × RN , S2 ∈ RN × RN −2 , we have that

Sr = S1 + 0 (S2 )↔ 0 .

Use the proof of Theorem 15 in the book as a guide.

Solution.

We compute
5 The filter representation of wavelets 145
 
x0
 .. 
 .         
x0 xN −2 x0 x1
 
xN −2 

   .   .   .  ↔ . 
Sr x = S1 S2  xN −1  = S1  ..  + S2  ..  = S1  ..  + (S2 )  .. 

xN −2 
  xN −1 x1 xN −1 xN −2
 . 
 .. 
x1
   
x0 x0
= S1  ...  + 0 (S2 )↔ 0  ...  = (S1 + 0 (S2 )↔ 0 )x,
 
 

xN −1 xN −1

so that

Sr = S1 + 0 (S2 )↔ 0 .

Exercise 5.27: Orthonormal basis for the symmetric

extensions

In this exercise we will establish an orthonormal basis for the symmetric extensions in
the sense of Definition 10 in the book. This parallels Theorem 23 in the book.
a) Show that

 
1 2π · 0 · k
√ cos ,
2N − 2 2N − 2
  N −2
1 2πnk
√ cos ,
N −1 2N − 2 n=1
 
1 2π(N − 1)k
√ cos
2N − 2 2N − 2

is an orthonormal basis for the symmetric extensions in R2N −2 .

Solution.

We compute the IDFT for all vectors in b). Since the IDFT is unitary, this will give us
an orthonormal basis for the symmetric vectors in R2N −2 . Since (FN )H φn = en we get
that
146 5 The filter representation of wavelets
 
1 0
(FN )H e0 = φ0 = √ cos 2π k
2N − 2 2N − 2
 
1 1
(FN )H √ (en + e2N −2−n ) = √ (φn + φ2N −2−n )
2 2
1 1  
=√ √ e2πikn/(2N −2) + e−2πikn/(2N −2)
2 2N − 2
 
1 n
=√ cos 2π k
N −1 2N − 2
 
H 1 N −1
(FN ) eN −1 = φN −1 = √ cos 2π k .
2N − 2 2N − 2

These coincide with the vectors listed in the exercise.

b) Assume that S is symmetric. Show that the vectors listed in a) are eigenvectors for
Sr , when the vectors are viewed as vectors in RN , and that they are linearly independent.
This shows that Sr is diagonalizable.

Solution.

Since S is symmetric, it preserves vectors which are symmetric around N − 1. In the

frequency domain, applying S to a vector listed in c) corresponds to multiplying the
vectors listed in b). with the frequency response. Since this does not introduce any more
components, it is clear that the new vector must be a multiple of the same vector, so
that these vectors indeed are eigenvectors. But then the vectors restricted to RN are also
eigenvectors for Sr , since this is simply S when viewed on the first N elements. Since
the vectors in R2N −2 are linearly independent, it is immediate that the corresponding
vectors in RN also are linearly independent, since the second part of the vectors mirror
the first part.

Exercise 5.28: Diagonalizing Sr

Let us explain how the matrix Sr can be diagonalized, similarly to how we previously
diagonalized using the DCT. In Exercise 5.27 we showed that the vectors

  N −1
2πnk
cos
2N − 2 n=0

in RN is a basis of eigenvectors for Sr when S is symmetric. Sr itself is not symmetric,

however, so that this basis can not possibly be orthogonal (S is symmetric if and only if
it is orthogonally digonalizable). However, when the vectors are viewed in R2N −2 we
showed in Exercise 5.27a) that
5 The filter representation of wavelets 147

2N −3    
X 2πn1 k 2πn2 k
cos cos
2N − 2 2N − 2
k=0

2 if n1 = n2 ∈ {0, N − 1}

= (N − 1) × 1 if n1 = n2 6∈ {0, N − 1} .

0 if n1 = 6 n2


a) Show that


if n1 = n2 ∈ {0, N − 1}
1

1
(N − 1) × if n1 = n2 6∈ {0, N − 1}
2
0 if n1 6= n2

   
1 2πn1 · 0 2πn2 · 0
= cos cos
2 2N − 2 2N − 2
N −2    
X 2πn1 k 2πn2 k
+ cos cos
2N − 2 2N − 2
k=1
   
1 2πn1 (N − 1) 2πn2 (N − 1)
+ cos cos .
2 2N − 2 2N − 2

Hint.

Use that cos x = cos(2π − x) to pair the summands k and 2N − 2 − k.

Solution.

Using that cos x = cos(2π − x) we can here pair the summands k and 2N − 2 − k to
obtain

2N −3    
X n1 n2
cos 2π k cos 2π k
2N − 2 2N − 2
k=0
   
n1 n2
= cos 2π · 0 cos 2π ·0
2N − 2 2N − 2
N −2    
X n1 n2
+2 cos 2π k cos 2π k
2N − 2 2N − 2
k=1
   
n1 n2
+ cos 2π (N − 1) cos 2π (N − 1) .
2N − 2 2N − 2

If we divide by 2 and combine these equations we get the result.

−1
b) Define the vectors {xn }N
n=0 by
148 5 The filter representation of wavelets
(     N −2  )
1 2πn · 0 2πnk 1 2πn(N − 1)
√ cos , cos , √ cos ,
2 2N − 2 2N − 2 k=1 2 2N − 2

(I)
−1
and the vectors {dn }N
n=0 by

q
1
 x0 n=0
q N −1


d(I)
n = 2
N −1 xn 1≤n<N −1

 q
1
N −1 xN −1 n = N − 1.



(I)
Explain that the vectors dn are orthonormal, and that the matrix
 √   √ 
1/ 2 0 · · · 0 0 1/ 2 0 · · · 0 0
r  0 1 ··· 0 0   0 1 ··· 0 0
2   .. .. .. .. .. 
 
2πnk
 
 .. .. .. ..

..
 . . . . .  cos 2N

−2  . . . . .
N −1

  
 0 0 ··· 1 0   0 0 ··· 1 0√ 
√
0 0 · · · 0 1/ 2 0 0 ··· 0 1/ 2
(I)
has the dn as columns, and is therefore orthogonal. Orthogonality in RN could thus be
achieved by a scaling of the first and last components in the vectors. This orthogonal
(I) (II)
matrix is denoted DCTN . It is much used, just as the DCTN of Section 2.4 in the
book.

Solution.
(I) (I)
In a) we computed the inner products hdn1 , dn2 i, and the vectors are orthogonal since
we obtained zero whenever n1 6= n2 . By a) also
(I) (I) (I) (I)
hd0 , d0 i = hdN −1 , dN −1 i = N − 1,
(I) (I) p
which explains why d0 and dN −1 both must be scaled bt 1/(N − 1). for the other n
(I) (I) p
a) said that hdn , dn i = (N − 1)/2, which explains the scaling with 2/(N − 1).
The pre- and post-multiplication with the stated diagonal √ matrix multiplies the first
and last row, as well as the first and last column, with 1/ 2 (the four corners of the
matrix will thus be multiplied by 1/2, since they appear in both the first/last rows,
and the first/last columns). For the columns between 1 and N − 2 this clearly gives the
(I) −2
vectors {dn }N
p
n=1 , when the leading factor 2/(N − 1)pis added. For the p first and last
√
columns, taking out a factor of 1/ 2 and pairing with 2/(N − 1) gives 1/(N − 1),
so that the right normalizing factor is obtained.
  −1
2πnk
c) Prove that cos 2N −2 can be written as
   
1/2 0 · · · 0 0 1/2 0 · · · 0 0
 0 1 ··· 0 0 
  0 1 · · · 0 0 
 
2   .. .. .. .. .. 
 
2πnk . . . . .
 cos 2N  .. .. .. .. .. 
 
N −1 . . . . .  −2

 
 0 0 ··· 1 0   0 0 ··· 1 0 
0 0 · · · 0 1/2 0 0 · · · 0 1/2
5 The filter representation of wavelets 149
     −1
2πnk 2πnk
Since Sr can be diagonalized as cos 2N −2 D cos 2N −2 , we can substitute
  
2πnk
here with a simple expression for the inverse. Thus, even if the matrix cos 2N −2
was not orthogonal, this represents no problem in the diagonalization/transition to the
frequency domain.

Solution.

Since the product we obtained in b) is an orthogonal matrix, its transpose equals its
inverse. The transpose of the product in b) clearly gives the same matrix back, and the
inverse is
√  √ 
2 0 ··· 0 0 2 0 ··· 0 0
 0 1 ··· 0 0 
−1  0 1 · · · 0 0 
r  
N −1 
 .. .. .. .. .. 

2πnk . . . . .
 cos 2N  .. .. .. .. ..  .
 
2  . . . . .  −2

 
 0 0 ··· 1 0   0 0 ··· 1 0 
√ √
0 0 ··· 0 2 0 0 ··· 0 2
2
Multiplying with N −1 , and then with
 √ 
1/ 2 0 ··· 0 0
 0
 1 ··· 0 0

 .. .. .. .. ..

 . . . . .

 
 0 0 · · · 1 0√ 
0 0 · · · 0 1/ 2
on both sides proves the result.

Exercise 5.29: Verifying a connection between the tables in

the MP3 standard

In addition to the function mp3_ctable(), the library has a function mp3_dtable()

which returns the table D used by the reverse transform. Verify the connection we have
stated between the tables C and D, i.e. that Di = 32Ci for all i.

Solution.

This can be verified as follows.

C = mp3_ctable();
D = mp3_dtable();
max(abs(D-32*C))
150 5 The filter representation of wavelets

Exercise 5.30: Implementing forward and reverse filter bank

transforms

It is not too difficult to implement the encoding and decoding steps as explained in the
MP3 standard. Write a function which implements the encoding steps, and one which
implements the decoding steps. In your code, assume for simplicity that the input and
output vectors to your methods all have lengths which are multiples of 32, and use the
functions mp3_ctable, mp3_dtable.
In the library the functions mp3_forward_fbt and mp3_reverse_fbt implement these
encoding- and decoding steps. Use these as a template for your code.

Solution.

The library versions of mp3_forward_fbt and mp3_reverse_fbt are as follows.

function z = mp3_forward_fbt(x)
N = length(x);
z = zeros(N,1);
C = mp3_ctable(); % The analysis window;
x = flipud(x);
x = [x; zeros(512-32,1)];
% The 32x64 matrix M
M = cos((2*((0:31)’)+1)*((0:63)-16)*pi/64);
start = length(x) - 511;
n = 1;
for n = 1:(N/32)
X = x(start:(start + 511));
Z = C.*X;
Y = zeros(64, 1);
for j = 0:7
Y = Y + Z((64*j + 1):(64*(j + 1)));
end
z((1+(n-1)*32):(n*32)) = M*Y;
start = start - 32;
end

function x = mp3_reverse_fbt(z)
Ns = length(z)/32;
x = zeros(32*Ns, 1);
D = mp3_dtable(); % The synthesis window.
V = zeros(1024,1);
% The 64x32 matrix N
N = cos((16+((0:63)’))*(2*(0:31)+1)*pi/64);
U = zeros(512,1);
for n = 1:Ns;
V(65:1024) = V(1:(1024-64));
V(1:64) = N*z((1+(n-1)*32):(n*32));
for i = 0:7
U((i*64 + 1):(i*64 + 32)) = V((i*128 + 1):(i*128 + 32));
U((i*64 + 33):((i + 1)*64)) = V((i*128 + 97):((i+1)*128));
end
W = U.*D;
for i = 0:15
x(((n-1)*32 + 1):(n*32)) = x(((n-1)*32 + 1):(n*32)) ...
+ W((32*i + 1):(32*(i + 1)));
end
end
5 The filter representation of wavelets 151

Exercise 5.31: Perfect reconstruction for forward and reverse

M -channel filter bank transforms

In this exercise H and G are forward and reverse M -channel filter bank transforms,
respectively.
a) Generalize the proof of Theorem 3 in the book to show that

−1 M −1
M
!
1 X X
−2πits/M
Hφr = λHs ,r e φr+tN/M
M t=0 s=0
−1 M −1
M
!
1 X X
−2πist/M
Gφr = λGs ,r+tN/M e φr+tN/M
M t=0 s=0

where the φr are the Fourier basis vectors. This shows that the output from φr is a sum
−1
of the {φr+sN/M }Ms=0 , so that at most M − 1 frequencies can contribute with aliasing.

Solution.

Following the lines in the proof we first get



 λH0 ,r e2πirk/N k mod M = 0

λH1 ,r e2πirk/N

k mod M = 1
(H(e2πirk/N ))k = . ..

 .. .


λHM −1 ,r e2πirk/N k mod M = M − 1


Component n in the DFT of H(e2πirk/N ) is

M
X X−1
−1 N/M
λHs ,r e2πir(M k+s)/N e−2πi(M k+s)n/N
s=0 k=0
M
X X−1
−1 N/M
= λHs ,r e2πi(r−n)(M k+s)/N
s=0 k=0
M −1
! N/M −1
X X
2πi(r−n)s/N
= λHs ,r e e2πi(r−n)k/(N/M ) .
s=0 k=0

PN/M −1 2πi(r−n)k/(N/M )
Clearly, k=0 e = N/M if n = r + tN/M for some t, and 0 else. It
follows that
−1 M −1
M
!
N X X
DFTN (H(e2πirk/N )) = λHs ,r e−2πits/M er+tN/M .
M t=0 s=0
Taking an IDFT gives
M −1 M −1
1 X X
H(e2πirk/N ) = λHs ,r e−2πits/M e2πi(r+tN/M )k/N ,
M t=0 s=0
152 5 The filter representation of wavelets

and substituting with φr gives the first equation. For the reverse filter bank transform,
recall Exercise 2.18b), which said that the vector e2πirk/N with all components with
indices different from kM + s zeroed out (with s fixed), can be written as
M −1
1 X −2πist/M 2πi(r+tN/M )k/N
e e
M t=0
so that

M −1 −1
M
!
1 X X
G(e2πirk/N ) = G e−2πist/M e2πi(r+tN/M )k/N
M s=0 t=0
−1 −1
M M
!
1 X X
−2πist/M 2πi(r+tN/M )k/N
= Gs e e
M s=0 t=0
M −1 M −1
1 X X −2πist/M
= e λGs ,r+tN/M e2πi(r+tN/M )k/N
M s=0 t=0
M −1 M −1
1 X X −2πist/M
= e λGs ,r+tN/M e2πi(r+tN/M )k/N .
M t=0 s=0

This proves the second equation.

b) Find a formula for GHφr , and conclude that, in terms of continuous frequency
responses, we have alias cancellation if and only if
M
X −1
λHs (ω + 2πq/M )λGs (ω)e2πiqs/M = 0
s=0

for all 0 ≤ q < M/2, and perfect reconstruction if and only if in addition

λH0 (ω)λG0 (ω) + · · · + λHM −1 (ω)λGM −1 (ω) = M.

These expressions clearly generalize the conditions obtained for M = 2 in Exercise 5.9.

Solution.

Note that what we found in a) implies that, for any r, the space spanned by B =
−1
{φr+tN/M }M 0
t=0 is invariant under H and G. Replacing r with r + t N/M we obtain

M −1 M −1
1 X X
Hφr+t0 N/M = λHs ,r+t0 N/M e−2πits/M φr+(t+t0 )N/M
M t=0 s=0
M −1 M −1
1 X X 0
= λHs ,r+t0 N/M e−2πi(q−t )s/M φr+qN/M
M q=0 s=0
M −1 M −1
1 X X −2πiqs/M 0
= e λHs ,r+t0 N/M e2πit s/M φr+qN/M
M q=0 s=0
5 The filter representation of wavelets 153

M −1 M −1
1 X X
Gφr+t0 N/M = λGs ,r+(t+t0 )N/M e−2πist/M φr+(t+t0 )N/M
M t=0 s=0
M −1 M −1
1 X X 0
= λGs ,r+qN/M e−2πis(q−t )/M φr+qN/M
M q=0 s=0
M −1 M −1
1 X X 0
= λGs ,r+qN/M e−2πisq/M e2πist /M φr+qN/M ,
M q=0 s=0

where we did the substitution q = t + t0 . From this we can see that the matrices of H
−1
and G relative to {φr+tN/M }M
t=0 are

λH0 ,r+0·N/M e2πi0·0/M λH0 ,r+(M −1)N/M e2πi0·(M −1)/M

 
···
1 .. .. ..
[H]B = DFTM 
 
M . . . 
λHM −1 ,r+0·N/M e2πi(M −1)·0/M · · · λHM −1 ,r+(M −1)N/M e2πi(M −1)(M −1)/M
λG0 ,r+0·N/M e−2πi0·0/M λGM −1 ,r+0·N/M e−2πi0·(M −1)/M
 
···
1  .. .. ..
[G]B =  DFTM .

M
 . . .
λG0 ,r+(M −1)N/M e−2πi(M −1)·0/M · · · λGM −1 ,r+(M −1)N/M e−2πi(M −1)(M −1)/M

Note that DFTM DFTM is simply M times the identity matrix Thus, in order for perfect
reconstruction, we must have that the product of the two modulated matrices above
must also be M times the identity matrix. For the diagonal entries to be nonzero we
must have that (the modulating complex exponentials cancel in this case)

λH0 (ω)λG0 (ω) + · · · + λHM −1 (ω)λGM −1 (ω) = M

(for M different values of ω). For alias cancellation we must have that
M
X −1
λHs ,r+q1 N/M λGs ,r+q2 N/M e2πi(q1 −q2 )s/M = 0
s=0

for all q1 6= q2 . Rewriting this in terms of angular frequency proves the result.

Exercise 5.32: Time domain view of perfect reconstruction

Assume that Hi and Gi are the filters of forward and reverse M -channel filter bank
transforms. Show that GH can be computed with the formula
−1
!
X M X (n−k) (k−l)
zn = Gk ∗ Hk xl ,
l k=0 0
(n−k) (n−k)
where Gk is polyphase component n − k of Gk , i.e. (Gk )r = (Gk )rM +n−k .
This can be though of as a time domain version of the previous exercise, in that alias
cancellation and perfect reconstruction can be spotted in terms of time domain polyphase
components. We also obtain an expression for the matrix GH as
154 5 The filter representation of wavelets

−1
M
!
(n−k) (k−l)
X
(GH)n,l = (Gk ) ∗ (Hk ) .
k=0 0

Solution.

We have that

..
 
 . 
 0 
 
M −1 M −1
(Hk x)0 
 
Gk  ... 
X X
zn = Gk z k =
 
 
k=0 k=0  0 
 
(Hk x)1 
 
..
.
M
X −1 X
= (Gk )n−(rM +k) (Hk x)rM +k
k=0 r
M
X −1 X X
= (Gk )n−(rM +k) (Hk )rM +k−l xl
k=0 r l
M −1 X X
(n−k) (k−l)
X
= (Gk )−r (Hk )r xl
k=0 r l
−1
M
!
(n−k) (k−l)
X X
= Gk ∗ Hk xl .
l k=0 0

Exercise 5.33: Connection between cosine-modulated forward

and reverse transforms

Show that, if H is a cosine-modulated forward transform with prototype {hr }511 r=0 ,
then G = E481 H T is a cosine-modulated reverse transform with prototype {gr }511 r=0 ,
with gr = h512−r (i.e. the prototype filter is reversed). Deduce also from this that
H = GT E481 .
In Exercise 7.13 we construct an example showing that the forward and reverse
transforms of this section do not invert each other exactly. In Section 7.3 in the book,
we also will see how one can construct similar filter banks with perfect reconstruction. It
may seem strange that the MP3 standard does not do this. A partial explanation may
be that, while perfect reconstruction is important, other design criteria may be even
more important.

Solution.

We know that H T is a reverse filter bank transform with filters

5 The filter representation of wavelets 155

(Hi )T = (Ei−31 h(i) )T = E31−i (h(i) )T .

(h(i) )T has filter coefficients cos((2i + 1)(−k − 16)π/64))h−k . If we delay all (Hi )T with
481 = 512−31 elements as in the theorem, we get a total delay of 512−31+31−i = 512−i
elements, so that we get the filter

E512−i {cos((2i + 1)(−k − 16)π/64))h−k }k

= E−i {cos((2i + 1)(−(k − 512) − 16)π/64))h−(k−512) }k
= E−i {cos((2i + 1)(k + 16)π/64))h−(k−512) }k .

Now, we define the prototype filter g with elements gk = h512−k . This has, just as h, its
support on [1, 511], and consists of the elements from h in reverse order. If we define
g (i) as the filter with coefficients cos((2i + 1)(k + 16)π/64))gk , we see that E481 H T is a
reverse filter bank transform with filters E−i g (i) . Since g (k) now has been defined as for
the MP3 standard, and its elements are the reverse of those in h, the result follows.
Chapter 6
Constructing interesting wavelets

Exercise 6.1: Properties of the CTFT

Prove the properties of the Continuous time Fourier transform listed in Theorem 3 in
the book.

Solution.

To prove the first property write

Z ∞ Z ∞
1 1
fˆ(ν) = √ f (t)e −iνt
dt = √ f (t)e−(−ν)it dt = fˆ(−ν).
2π −∞ 2π −∞
R∞
If f is symmetric the substitution u = −t can be used to prove that −∞ f (t)e−iνt dt =
R∞
−∞
f (t)eiνt dt, so that f (ν) = f (−ν) = f (ν) due to the first property. It follows that
f (ν) is real. The third property follows similarly. For the fourth property,
Z ∞ Z ∞ Z ∞
−iνt −iν(t+d) −iνd
f (t − d)e dt = f (t)e dt = e f (t)e−iνt dt,
−∞ −∞ −∞

and ĝ(ν) = e −iνd

fˆ(ν) follows. The fifth property follows from
Z ∞ Z ∞
eidt f (t)e−iνt dt = f (t)e−i(ν−d)t dt.
−∞ −∞

Exercise 6.2: The CTFT of a pulse

sin(νa)
a) Show that the CTFT of χ[−a,a] (t) is √1 for any a.
2π ν/2

Solution.

We have that

157
158 6 Constructing interesting wavelets
Z a  a
1 1 1 −iνt 1 1 sin(νa)
√ e−iνt dt = √ e = √ (e−iνa − eiνa ) = √ .
2π −a 2π −iν −a −i 2πν 2π ν/2
sin(πt/Ts )
b) Use a) to show that the CTFT of is √Ts χ[−π/T ,π/T ] (t).
πt/Ts 2π s s

Solution.

Setting a = π/Ts and taking the ICTFT of the expression we obtained in a) we get
Z ∞
1 1 sin(πν/Ts ) iνt
χ[−π/Ts ,π/Ts ] (t) = √ √ e dν
2π −∞ 2π ν/2
so that
Z ∞
1 sin(πν/Ts ) −iνt Ts
√ e dν = √ χ[−π/Ts ,π/Ts ] (−t).
2π −∞ πν/Ts 2π
The result follows.

Exercise 6.3: Mixed continuous-discrete expressions

Let f (t) = k xk h(t − nTs ). Show that fˆ(ν) = x̂(νTs )ĥ(ν). This can be considered a
P
mixed continuous-discrete expression for that convolution in time to corresponds to
multiplication in frequency.

Solution.

We have that

Z ∞X Z ∞
1 X 1
fˆ(ν) = √ xk h(t − nTs )e−iνt dt = xk √ h(t − nTs )e−iνt dt
2π −∞ k k
2π −∞
Z ∞
X 1
= xk e−iνnTs √ h(t)e−iνt dt = x̂(νTs )ĥ(ν).
k
2π −∞

Exercise 6.4: The frame inequalities and singular values

Show that

Akf k2 ≤ kU T f k2 ≤ Bkf k2 .
for some constants A, B > 0 if and only if U U T is non-singular. Show also in this case
2
that the optimal values for A and B are A = σm and B = σ12 , where σi are the singular
T
values of U .
6 Constructing interesting wavelets 159

Hint.

Consult Appendix A in the book about the singular value decomposition.

Solution.

Let U T = U0 Σ(V0 )H be a singular value decomposition of U T . Since U0 and V0 are

unitary they preserve norm, i.e. kU0 f k = kf k and kV0 f k = kf k for any f Rn or Rm ,
respectively. Using this and replacing U T with U0 Σ(V0 )H , and f with V0 f in the frame
inequalities we obtain that

Akf k2 ≤ kU0 Σf k2 = kΣf k2 ≤ Bkf k2 .

Here we have that
   
σ1 0 ··· 0 σ1 f1
0 σ2 ··· 0    σ2 f2
 .. .. .. .. ..
   
  
. . . .    .
   
0
Σf =  0 ··· σm  f = σm fm  ,
 
0
 0 ··· 0 
 0 
 
. .. .. ..   . 
 .. . . .   .
. 
0 0 ··· 0 0
so that
m
X
2
σm kf k2 ≤ kΣf k = 2
σi2 fi2 ≤ σ12 kf k2 .
i=1
2
It follows that A = σm and B = σ12
can be used as bounds in the frame inequalities. It
is straightforward to see that these bounds are optimal.
In particular σm > 0 is the same as A > 0. According to Appendix A in the book
the eigenvalues of U U T are {σi2 }m
i=1 , so that 0 is not an eigenvalues in U U
T
if and only
T
σm > 0, and this is again equivalent to U U being non-singular.

Exercise 6.5: Finding a dual frame

Assume that U U T is non-singular, so that {un }n is a frame. From the previous exercise
it follows from this that σm > 0. From Appendix A in the book it also follows that
the rank of U is m, i.e. U has full column rank. Show that Ũ = (U T )† satisfies the
requirements of a dual frame, where (U T )† is the generalized inverse of U T , defined in
the appendix. The appendix then says that Ũ = (U T )† = (U U T )−1 U . It follows that
(U U T )−1 maps the frame vectors uk to the dual frame vectors ũk , and that the frame
operator U U T maps the dual frame vectors to the frame vectors.

Solution.

First we have that

160 6 Constructing interesting wavelets

Ũ U T = (U T )† U T = Im ,
where we used Appendix A in the book. Also, if σi are the singular values of U T , 1/σi
are the singular values of (U T )† . Its smallest singular value is thus 1/σ1 , and its largest
singular value is 1/σm . Again with U T = U0 Σ(V0 )H a singular value decomposition of
U T , we have that Ũ T = U0 Σ −1 (V0 )H . Following the deductions in the previous exercises
we get that
1 1
kf k2 ≤ kŨ T f k2 =≤ 2 kf k2 .
σ12 σm
Since 1/σ1 = 1/B and 1/σm = 1/A the result follows.

Exercise 6.6: Tight frames in finite-dimensional spaces

Let {ui }ni=1 be a set of vectors in Rm . Show that {ui }ni=1 is a tight frame with frame
bound A is equivalent to any of the two following statements.
a) U U T = AIm .

Solution.

Using the singular value decomposition again, a tight frame is equivalent to that
σ1 = · · · = σm , i.e. that U T = σ1 U0 (V0 )H in the singular value decomposition of U T .
But this implies that

U U T = σ12 V0 (U0 )H U0 (V0 )H = σ12 V0 (V0 )H = σ12 Im ,

since V0 is unitary. On the other hand, if U U T = AIm , we have that U U T =
V0 Σ12 (V0 )H = AIm , from which it 2 2
√ follows that Σ1 = AIm , so that σi = A for all
i. It follows that σ1 = · · · = σm = A, so that the frame is tight.
Pn
b) x = A1 i=1 hx, ui iui for all x ∈ Rm .

Solution.

1 T
This statement is equivalent to that x = AUU x for all x, i.e. that U U T = AIm , which
is what we considered in a).

Exercise 6.7: Riesz bases for finite-dimensional spaces

Let {ui }m m
i=1 be a Riesz basis for R .
−1
a) Show that the dual frame of {ui }m
i=1 are the rows of U .
6 Constructing interesting wavelets 161

Solution.

† −1
Since {ui }m
i=1 is a Riesz basis U is non-singular, so that U = U . Since Ũ = (U T )† =
(U ) , the dual frame vectors (i.e. the columns of Ũ ) are the rows of U −1 .
−1 T

b) Show that {ui }mi=1 is a tight frame if and only if the vectors are orthogonal, and
that the dual frame is a scaled version of the frame itself.

Solution.

From the previous exercise we have that {ui }m i=1 is a tight frame is the same as
U U T = AIm for some scalar A, which is equivalent to U T U = AIm . This is clearly again
equivalent to the columns of U being orthogonal. From a) we also have that the dual
frame vectors are the rows of U −1 = A1 U T , i.e. the columns of A1 U . The dual frame is
thus obtained from the frame by scaling with 1/A.

Exercise 6.8: Computing filters

Compute the filters H0 , G0 in Theorem 8 in the book when N = N1 = N2 = 4, and

Q1 = Q(4) , Q2 = 1. Compute also filters H1 , G1 so that we have perfect reconstruction
(note that these are not unique).

Solution.

We have that

 N1 /2  
1 1
λH0 (ω) = (1 + cos ω) Q1 (1 − cos ω)
2 2
 2  
1 (4) 1
= (1 + cos ω) Q (1 − cos ω)
2 2
 N2 /2    2
1 1 1
λG0 (ω) = (1 + cos ω) Q2 (1 − cos ω) = (1 + cos ω)
2 2 2
 2
1 1 1 1 2iω
= 1 + eiω + e−iω = (e + 4eiω + 6 + 4e−iω + e−2iω ).
4 2 2 16
1
Therefore G0 = 16 {1, 4, 6, 4, 1}. We do not recommend to compute H0 by hand. With
symbolic math toolbox in MATLAB you can do as follows to compute H0 .
syms x
expand( ((1+x/2+1/(2*x))/2)^2*...
(2+8*((1-x/2-1/(2*x))/2)+20*((1-x/2-1/(2*x))/2)^2+...
40*((1-x/2-1/(2*x))/2)^3) )

2
Here we have substituted x for eiω , 1/x for e−iω . The first part represents 12 (1 + cos ω) ,
the second part represents Q(4) (u) = 2 + 8u + 20u2 + 40u3 with u = 12 (1 − cos ω) =
1 1 iω 1 −iω


2 1 − 2e − 2e . This gives
162 6 Constructing interesting wavelets

1
H0 = {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5}.
128
Using Exercise 5.10 with α = 1, d = 0, we get

1
H1 = {1, −4, 6, −4, 1}
16
1
G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5}
128

Exercise 6.9: Plotting frequency responses of Spline wavelets

The library represents a Spline wavelet with x and y vanishing moments (i.e. N1 and
N2 ) with the name "splinex.y".
a) Listen to the low-resolution approximations and detail components in sound for the
"spline4.4" wavelet.

Solution.

The following code can be used for listening to the low-resolution approximations for a
given value of m.

m = 1;
[x, fs] = forw_comp_rev_dwt1(m, ’spline4.4’);
playerobj = audioplayer(x, fs);
playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’spline4.4’, 0);

playerobj = audioplayer(x, fs);
playblocking(playerobj)

b) Plot all scaling functions and mother wavelets (using the cascade algorithm), and
frequency responses for the "spline4.4" wavelet.

Solution.

The code to plot the functions can look as follows.

m=10;
cascade_alg(m, -4, 4, ’spline4.4’, 1, 0)
cascade_alg(m, -4, 4, ’spline4.4’, 0, 0)

The frequency response can be plotted as follows

freqresp_alg(’spline4.4’, lowpass, dual)

6 Constructing interesting wavelets 163

Exercise 6.10: Wavelets based on higher degree polynomials

Show that Br (t) = ∗rk=1 χ[−1/2,1/2) (t), the B-spline of order r, is r −2 times differentiable,
and equals a polynomial of degree r − 1 on subintervals of the form [n, n + 1]. Explain
why these functions can be used as bases for the resolution spaces from Definition 11 in
the book.

Solution.

The case for r = 2 follows from Exercise 4.26. Assuming that the result holds for a given
r, it is straightforward to prove by induction that the result also holds for r + 1. That it
equals a polynomial of one degree higher is easiest to prove, and follows by splitting up
the integral over the different intervals [n, n + 1], using a polynomial expression on each.
That the differentiability is increased by one follows from the fundamental theorem of
calculus (i.e. the integral of a continuous function is differentiable), from a result which
says that it is allowed to differentiate under the integral sign, and by writing (for an
r − 2 times differentiable function f )
Z ∞ Z 1/2
χ[−1/2,1/2) (t)f (x − t)dt = f (x − t)dt.
−∞ −1/2
Chapter 7
The polyphase representation of filter bank
transforms

Exercise 7.1: The frequency responses of the polyphase

components

Let H and G be 2-channel forward and reverse filter bank transforms, with filters H0 ,
H1 , G0 , G1 , and polyphase components H (i,j) , G(i,j) .
a) Show that

λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω)

λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω)
λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω)
λG1 (ω) = λG(1,1) (2ω) + eiω λG(0,1) (2ω).

Solution.

G(0,0) , G(1,1) are the even-indexed filter coefficients of G0 , G1 , respectively, so that

λG(0,0) (2ω), λG(1,1) (2ω) represents the half of λG0 (ω), λG1 (ω), respectively, from the even
filter coefficients. G(1,0) are the odd-indexed filter coefficients of G0 . Since coefficient 0
in G(1,0) equals coefficient 1 in G0 , it is clear that e−iω λG(1,0) (2ω) represents the half of
λG0 (ω) from the odd filter coefficients. This proves the first formula. The second formula
follows from the same kind of reasoning.
If we transpose H (also in polyphase form), we get an MRA-matrix where the columns
are given by the filters (H0 )T , (H1 )T . Inserting these in the formulas we just proved we
get that

λ(H0 )T (ω) = λ(H (0,0) )T (2ω) + e−iω λ(H (0,1) )T (2ω)

λ(H1 )T (ω) = λ(H (1,1) )T (2ω) + eiω λ(H (1,0) )T (2ω).

If we conjugate these expressions we get

165
166 7 The polyphase representation of filter bank transforms

λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω)

λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω),

and the proof is done.

b) In the proof of the last part of Exercise 5.10, we deferred the last part, namely that

λH1 (ω) = α−1 e−2idω λG0 (ω + π)

λG1 (ω) = αe2idω λH0 (ω + π).

follow from

G(0,0) G(0,1) αE−d H (1,1) −αE−d H (0,1)

   
= .
G(1,0) G(1,1) −αE−d H (1,0) αE−d H (0,0)

Prove this based on the result from a).

Solution.

The first column in the matrix on the left hand side gives the filter G0 . On the right hand
side, a) states that the even-indexed columns are taken from the filter with frequency
response

λαE−d H (1,1) (2ω) + e−iω λ−αE−d H (1,0) (2ω)

= αλE−d (2ω) λH (1,1) (2ω) − e−iω λH (1,0) (2ω)


 
= αe2idω λH (1,1) (2(ω + π)) + e−i(ω+π) λH (1,0) (2(ω + π)) = αe2idω λH1 (ω + π).

This shows that λG0 (ω) = αe2idω λH1 (ω+π). We obtain the first equation easily from this.
Now, the second column in the matrix on the left hand side gives the filter coefficients
of G1 . On the right hand side, a) states that the odd-indexed columns are taken from
the filter with frequency response

λαE−d H (0,0) (2ω) + eiω λ−αE−d H (0,1) (2ω)

= αλE−d (2ω) λH (0,0) (2ω) − eiω λH (0,1) (2ω)


 
= αe2idω λH (0,0) (2(ω + π)) + ei(ω+π) λH (0,1) (2(ω + π)) = αe2idω λH0 (ω + π).

This shows that λG1 (ω) = αe2idω λH0 (ω + π), which is the second equation.

Exercise 7.2: Polyphase view of time delay

a) Write down the polyphase representation for an arbitrary delay Ed , and for any
given M .
7 The polyphase representation of filter bank transforms 167

Solution.

Write d = kM + s for some 0 ≤ s < M . The polyphase representation will be a block

matrix with only one nonzero block-diagonal. The nonzero blocks will be at positions
(s, 0), (s + 1, 1), and so on, and each nonzero block will equal the delay Ek .
b) Assume that d is a multiple of M . Show that Ed commutes with any M -channel
filter bank transform. Will this be the case if d is not a multiple of M ?

Solution.

If d is a multiple of M , the nonzero block-diagonal from a) will actually be the main

block-diagonal. This means that
 
Ek 0 · · · 0 0
 0 Ek · · · 0 0 
 
Ed =  ... ... . . . ... ...  .
 
 
 0 0 · · · Ek 0 
0 0 · · · 0 Ek
Since Ed commutes with any filter, and since the same element repeats on the main
block-diagonal, such Ed will commute with all filter bank transforms. When d is not a
multiple of M , the nonzero block diagonal will not be the main one. Matrices with one
nonzero off-diagonal do not commute with all other matrices, even if it is constant on
that diagonal. In particular this will be the case for filter bank transforms as well (even
if Ed commutes with filters).

Exercise 7.3: Polyphase view of alias cancellation

How we can spot alias cancellation in terms of the polyphase representation will be
useful to us in the next section when we explain why the filter bank transform of
the MP3 standard provides such cancellation. We will there end up with a polyphase
representation on the form given in a) of this exercise.
a) Assume that the polyphase representation of S is a k × k-block diagonal matrix
where the diagonal elements are M × M -filters which are all equal. Show that S is
a (kM ) × (kM )-filter. Thus, if GH has this form for forward and reverse filter bank
transforms H and G, we have alias cancellation.

Solution.

Since S has a k × k-block diagonal polyphase representation with equal diagonal blocks,
the only nonzero elements in the filter representation are at indices (i + M r1 , i + M r2 )
with 0 ≤ i < M , 0 ≤ r1 , r2 < k. Increasing i with 1 just switches to the next block
on the diagonal, and we get the same value since the blocks are assumed equal. Also,
increasing both r1 and r2 with one clearly gives a new element on the same diagonal in
the same block, which has the same value by the assumption that each block entry is
168 7 The polyphase representation of filter bank transforms

a filter. It follows that S is constant on diagonals 0, ±M , ±2M , and so on, and zero
elsewhere. It follows that S is a (kM ) × (kM )-filter.
b) If S is a general (kM ) × (kM )-filter, give a general description of its polyphase
representation.

Solution.

Each block in the polyphase representation is a k × k-filter. Also, the filters S (i,j) and
S (i+k,j+k) must be equal for any i, j, k.

Exercise 7.4: Polyphase components for symmetric filters

Assume that the filters H0 , H1 of a forward filter bank transform are symmetric, and let
H (i,j) be the corresponding polyphase components. Show that
• H (0,0) and H (1,1) are symmetric filters,
• the filter coefficients of H (1,0) have symmetry about −1/2,
• the filter coefficients of H (0,1) have symmetry about 1/2.
Also show a similar statement for reverse filter bank transforms.

Solution.

Let H0 = {..., t−1 , t0 , t1 , ...} and H1 = {..., r−1 , r0 , r1 , ...}. We clearly have that

H (0,0) = {..., t−4 , t−2 , t0 , t2 , t4 , ...}

H (1,1) = {..., r−4 , r−2 , r0 , r2 , r4 , ...}
H (1,0) = {..., r−3 , r−1 , r1 , r3 , r5 , ...}
H (0,1) = {..., t−5 , t−3 , t−1 , t1 , t3 , ...}

The symmetry statements now follow from the symmetries in tk and rk .

Exercise 7.5: Classical QMF filter banks

Recall from Exercise 5.12 that we defined a classical QMF filter bank as one where M = 2,
G0 = H0 , G1 = H1 , and λH1 (ω) = λH0 (ω + π). Show that the forward and reverse filter
bank transforms of a classical QMF filter bank have polyphase representations of the
form

 
A −E1 B
H=G= .
B A
7 The polyphase representation of filter bank transforms 169

Solution.

We have that

λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω)

λG1 (ω) = eiω λG(0,1) (2ω) + λG(1,1) (2ω).

λG1 (ω) = λG0 (ω + π) thus implies that

λG(1,1) (2ω) = λG(0,0) (2(ω + π)) = λG(0,0) (2ω)

λG(0,1) (2ω) = −e−2iω λG(1,0) (2(ω + π)) = −e−2iω λG(1,0) (2ω).

The first equation says that G(1,1) = G(0,0) , the second says that G(0,1) = −E1 G(1,0) .

Exercise 7.6: Alternative QMF filter banks

Recall from Exercise 5.13 that we defined an alternative QMF filter bank as one where
M = 2, G0 = (H0 )T , G1 = (H1 )T , and λH1 (ω) = λH0 (ω + π).
a) Show that the forward and reverse filter bank transforms of an alternative QMF
filter bank have polyphase representations of the form

T
AT B T A −B T AT B T
    
H= G= = .
−B A B AT −B A

Solution.

We have that

λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω)

λG1 (ω) = eiω λG(0,1) (2ω) + λG(1,1) (2ω).

λG1 (ω) = λG0 (ω + π) thus implies that

λG(1,1) (2ω) = λG(0,0) (2ω) λG(0,1) (2ω) = −λG(1,0) (2ω),

so that G(1,1) = (G(0,0) )T and G(0,1) = −(G(1,0) )T . The result follows.

b) Show that A and B give rise to an alternative QMF filter bank with perfect recon-
struction as above if and only if AT A + B T B = I.
170 7 The polyphase representation of filter bank transforms

Solution.

We have that

AT B T A −B T
 T
A A + BT B
   
0
= ,
−B A B AT 0 AT A + B T B
and the result follows.
c) Consider alternative QMF filter banks where we use an additional sign, so that
λH1 (ω) = −λH0 (ω + π) (the Haar wavelet was an example of such a filter bank, see
exercise 5.15). Show that such forward and reverse filter bank transforms have polyphase
representations of the form

 T T T
A BT AT B T
  
A B
H= G= = .
B −A B −AT B −A

This sign change does not substantially change the properties of alternative QMF filter
banks, so that we will also call these new filter banks as alternative QMF filter banks.

Solution.

This follows as in a), just incorporating the additional minus sign.

Exercise 7.7: View of lifting as altering the filters

This exercise will show that one can think of the steps in a lifting factorization as altering
the low-pass/high-pass filters in alternating order. Let S be a general filter.
a) Show that
 
I 0
G
SI

is an MRA matrix with filters Ĝ0 , G1 , where

λGˆ0 (ω) = λG0 (ω) + λS (2ω)e−iω λG1 (ω),

Solution.

We have that

G(0,0) G(0,1)
  (0,0)
+ SG(0,1) G(0,1)
  
I 0 G
= .
G(1,0) G(1,1) SI G(1,0) + SG(1,1) G(1,1)
Using Exercise 7.1a), the even-indexed columns in this matrix are taken from the filter
with frequency response
7 The polyphase representation of filter bank transforms 171

λG(0,0) +SG(0,1) (2ω) + e−iω λG(1,0) +SG(1,1) (2ω)

= λG(0,0) (2ω) + e−iω λG(1,0) (2ω) + λS (2ω) λG(0,1) (2ω) + e−iω λG(1,1) (2ω)

= λG0 (ω) + λS (2ω)e−iω λG(1,1) (2ω) + eiω λG(0,1) (2ω)

= λG0 (ω) + λS (2ω)e−iω λG1 (ω).

b) Show that
 
I S
G
0I

is an MRA matrix with filters G0 , Ĝ1 , where

λGˆ1 (ω) = λG1 (ω) + λS (2ω)eiω λG0 (ω),

Solution.

We have that

G(0,0) G(0,1)
 (0,0)
SG(0,0) + G(0,1)
   
I S G
= ,
G(1,0) G(1,1) 0I G(1,0) SG(1,0) + G(1,1)
so that the odd-indexed columns in this matrix are taken from the filter with frequency
response

λSG(1,0) +G(1,1) (2ω) + eiω λSG(0,0) +G(0,1) (2ω)

= λG(1,1) (2ω) + eiω λG(0,1) (2ω) + λS (2ω) λG(1,0) (2ω) + eiω λG(0,0) (2ω)

= λG1 (ω) + λS (2ω)eiω λG(0,0) (2ω) + e−iω λG(1,0) (2ω)

= λG1 (ω) + λS (2ω)eiω λG0 (ω).

c) Show that
 
I 0
H
SI

is an MRA-matrix with filters H0 , Ĥ1 , where

λĤ1 (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω).

Solution.

We transpose the expression to obtain

I ST
 
HT
0 I
Since H T has filters (H0 )T and (H1 )T in the columns, from b) it follows that

I ST
 
HT
0 I
172 7 The polyphase representation of filter bank transforms

has columns given by (H0 )T and the filter with frequency response

λ(H1 )T (ω) + λS T (2ω)eiω λ(H0 )T (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω),
so that
 
I 0
H
SI

has row filters H0 and a filter Ĥ1 with frequency response

λĤ1 (ω) = λH1 (ω) + λS (2ω)e−iω λH0 (ω).

d)  
I S
H
0I

is an MRA-matrix with filters Ĥ0 , H1 , where

λĤ0 (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω).

Solution.

We transpose the expression to obtain

 
I 0
HT .
ST I
Since H T has filters (H0 )T and (H1 )T in the columns, using a) we see that
 
I 0
HT
ST I
has columns given by the filter with frequency response

λ(H0 )T (ω) + λS T (2ω)e−iω λ(H1 )T (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω),
and (H1 )T , so that
 
I S
H
0I

has a row filter Ĥ0 with frequency response

λĤ0 (ω) = λH0 (ω) + λS (2ω)eiω λH1 (ω),

and H1 .
7 The polyphase representation of filter bank transforms 173

Exercise 7.8: Lifting-based implementations of the Spline 5/3

and CDF 9/7 wavelets

Let us use the different lifting factorizations obtained in this chapter to implement the
corresponding wavelet kernels. Your functions should call the functions from Exercise 4.24
and 4.29. You will need equations (7.8)-(7.11) here, in order to complete the kernels.
a) Write functions

dwt_kernel_53(x, bd_mode)
idwt_kernel_53(x, bd_mode)

which implement the DWT and IDWT kernels for the Spline 5/3 wavelet. Use the lifting
factorization obtained in Example 7.2.3 in the book.

Solution.

The code can look like this:

function x=dwt_kernel_53(x, bd_mode)

lambda1 = -1;
lambda2 = 0.125;
alpha = 2;
beta = 0.5;
x(1:2:end, :) = x(1:2:end, :)*alpha;
x(2:2:end, :) = x(2:2:end, :)*beta;
x = lifting_odd_symm(-lambda2, x, bd_mode);
x = lifting_even_symm(-lambda1, x, bd_mode);
end

function x=idwt_kernel_53(x, bd_mode)

lambda1 = -1;
lambda2 = 0.125;
alpha = 2;
beta = 0.5;
x = lifting_even_symm(lambda1, x, bd_mode);
x = lifting_odd_symm(lambda2, x, bd_mode);
x(1:2:end, :) = x(1:2:end, :)/alpha;
x(2:2:end, :) = x(2:2:end, :)/beta;
end

b) Write functions

dwt_kernel_97(x, bd_mode)
idwt_kernel_97(x, bd_mode)

which implement the DWT and IDWT kernels for the CDF 9/7 wavelet. Use the lifting
factorization obtained in Example 7.2.4 in the book.

Solution.

The code can look like this:

174 7 The polyphase representation of filter bank transforms

function x=dwt_kernel_97(x, bd_mode)

lambda1 = -0.586134342059950;
lambda2 = -0.668067171029734;
lambda3 = 0.070018009414994;
lambda4 = 1.200171016244178;
alpha = -1.149604398860250;
beta = -0.869864451624777;
x(1:2:end, :) = x(1:2:end, :)*alpha;
x(2:2:end, :) = x(2:2:end, :)*beta;
x = lifting_odd_symm(-lambda4, x, bd_mode);
x = lifting_even_symm(-lambda3, x, bd_mode);
x = lifting_odd_symm(-lambda2, x, bd_mode);
x = lifting_even_symm(-lambda1, x, bd_mode);
end

function x=idwt_kernel_97(x, bd_mode)

lambda1 = -0.586134342059950;
lambda2 = -0.668067171029734;
lambda3 = 0.070018009414994;
lambda4 = 1.200171016244178;
alpha = -1.149604398860250;
beta = -0.869864451624777;
x = lifting_even_symm(lambda1, x, bd_mode);
x = lifting_odd_symm(lambda2, x, bd_mode);
x = lifting_even_symm(lambda3, x, bd_mode);
x = lifting_odd_symm(lambda4, x, bd_mode);
x(1:2:end, :) = x(1:2:end, :)/alpha;
x(2:2:end, :) = x(2:2:end, :)/beta;
end

c) In Chapter 4 in the book, we listened to the low-resolution approximations and detail

components in sound for three different wavelets. Repeat these experiments with the
Spline 5/3 and the CDF 9/7 wavelet, using the new kernels you implemented in a) and
b).

Solution.

The following code can be used for listening to the low-resolution approximations for a
given value of m.

m = 1;
[x, fs] = forw_comp_rev_dwt1(m, ’cdf53’);
playerobj = audioplayer(x, fs);
playblocking(playerobj)

m = 1;
[x, fs] = forw_comp_rev_dwt1(m, ’cdf97’);
playerobj = audioplayer(x, fs);
playblocking(playerobj)

d) Plot all scaling functions and mother wavelets for the Spline 5/3 and the CDF 9/7
wavelets, using the cascade algorithm and the kernels you have implemented.
7 The polyphase representation of filter bank transforms 175

Solution.

The code can look as follows.

m=10;
cascade_alg(m, -4, 4, ’cdf53’, 1, 0)
cascade_alg(m, -4, 4, ’cdf53’, 0, 0)
cascade_alg(m, -4, 4, ’cdf97’, 1, 0)
cascade_alg(m, -4, 4, ’cdf97’, 0, 0)

In the plot for the CDF 9/7 wavelet, it is seen that the functions and their dual
counterparts are close to being equal. This reflects the fact that this wavelet is close to
being orthogonal.

Exercise 7.9: Lifting-based implementation of orthonormal

wavelets

a) Write functions

lifting_even(lambda1, lambda2, x, bd_mode)

lifting_odd(lambda1, lambda2, x, bd_mode)

which apply the elementary lifting matrices (7.14) in the book to x. Assume that N is
even.

Solution.

The library versions of these functions are as follows.

function x=lifting_even(lambda1, lambda2, x, bd_mode)

N = size(x, 1);
if strcmpi(bd_mode, ’per’)
x(1, :) = lambda1*x(2, :) + x(1, :) + lambda2*x(N, :);
elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
x(1, :) = lambda1*x(2, :) + x(1, :);
end
x(3:2:(N-1), :) = ...
lambda1*x(4:2:N, :) + x(3:2:(N-1), :) + lambda2*x(2:2:(N-2), :);
if strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
if mod(N,2) == 1 % last must also be included
x(N, :) = x(N, :) + lambda2*x(N-1, :);
end
end
end

function x=lifting_odd(lambda1, lambda2, x, bd_mode)

N = size(x, 1);
x(2:2:(N-1), :) = ...
lambda1*x(3:2:N, :) + x(2:2:(N-1), :) + lambda2*x(1:2:(N-2), :);
if mod(N,2) == 0 % last must also be included
if strcmpi(bd_mode, ’per’)
x(N, :) = lambda1*x(1, :) + x(N, :) + lambda2*x(N-1, :);
elseif strcmpi(bd_mode, ’bd’) || strcmpi(bd_mode, ’none’)
x(N, :) = x(N, :) + lambda2*x(N-1, :);
end
176 7 The polyphase representation of filter bank transforms

end
end

b) Write functions

dwt_kernel_ortho(x, bd_mode, wav_props)

idwt_kernel_ortho(x, bd_mode, wav_props)

which apply the DWT and IDWT kernels for orthonormal wavelets, using the functions
lifting_even and lifting_odd. Assume that you can access lifting steps so that the
lifting factorization (7.7) in the book holds through the object wav_props by means
of writing wav_props.lambdas, wav_props.alpha, and wav_props.beta, and that
wav_props.last_even indicates whether the last lifting step is even. wav_props.lambdas
is an n × 2-matrix so that the filter coefficients {λ1 , λ2 } or {λ1 , λ2 } in the i’th lifting
step are found in row i.
You can now define standard DWT and IDWT kernels in the following way, once the
wav_props object has been computed for a given N :

dwt_kernel = @(x, bd_mode) dwt_kernel_ortho(x, bd_mode, wav_props);

Solution.

The library versions of these functions are as follows, and again support more general
boundary modes:

function x=dwt_kernel_ortho(x, bd_mode, dual_wav_props)

if strcmpi(bd_mode, ’bd’)
y1 = dual_wav_props.A_L’*x(1:size(dual_wav_props.A_L,1), :);
y2 = dual_wav_props.A_R’*x((end-size(dual_wav_props.A_R,1)+1):end, :);
end
x(1:2:end, :) = x(1:2:end, :)/dual_wav_props.alpha;
x(2:2:end, :) = x(2:2:end, :)/dual_wav_props.beta;
iseven = ~dual_wav_props.last_even;
for stepnr = (size(dual_wav_props.lambdas, 1)):(-1):1
if iseven
x = lifting_even(dual_wav_props.lambdas(stepnr,2), ...
dual_wav_props.lambdas(stepnr,1), x, bd_mode);
else
x = lifting_odd(dual_wav_props.lambdas(stepnr,2), ...
dual_wav_props.lambdas(stepnr,1), x, bd_mode);
end
iseven = ~iseven;
end
if strcmpi(bd_mode, ’bd’)
x(1:size(dual_wav_props.A_L,2), :) = ...
x(1:size(dual_wav_props.A_L,2), :) + y1;
x((end-size(dual_wav_props.A_R,2)+1):end, :) = ...
x((end-size(dual_wav_props.A_R,2)+1):end, :) + y2;
end
end

function x=idwt_kernel_ortho(x, bd_mode, wav_props)

if strcmpi(bd_mode, ’bd’)
y1 = wav_props.A_L*x(1:size(wav_props.A_L,2), :);
y2 = wav_props.A_R*x((end-size(wav_props.A_R,2)+1):end, :);
end
iseven = ( mod(size(wav_props.lambdas, 1), 2) == wav_props.last_even );
for stepnr = 1:(size(wav_props.lambdas, 1))
7 The polyphase representation of filter bank transforms 177

if iseven
x = lifting_even(wav_props.lambdas(stepnr, 1), ...
wav_props.lambdas(stepnr, 2), x, bd_mode);
else
x = lifting_odd(wav_props.lambdas(stepnr, 1), ...
wav_props.lambdas(stepnr, 2), x, bd_mode);
end
iseven = ~iseven;
end
x(1:2:end, :)=x(1:2:end, :)/wav_props.alpha;
x(2:2:end, :)=x(2:2:end, :)/wav_props.beta;
if strcmpi(bd_mode, ’bd’)
x(1:size(wav_props.A_L,1), :) = ...
x(1:size(wav_props.A_L,1), :) + y1;
x((end-size(wav_props.A_R,1)+1):end, :) = ...
x((end-size(wav_props.A_R,1)+1):end, :) + y2;
end
end

c) Listen to the low-resolution approximations and detail components in sound for

orthonormal wavelets for N = 1, 2, 3, 4.

Solution.

The following code can be used for listening to the low-resolution approximations for a
given value of m.
[x, fs] = forw_comp_rev_dwt1(m, ’db2’);
playerobj = audioplayer(x, fs);
playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’db3’);

playerobj = audioplayer(x, fs);
playblocking(playerobj)

[x, fs] = forw_comp_rev_dwt1(m, ’db4’);

playerobj = audioplayer(x, fs);
playblocking(playerobj)

d) Plot all scaling functions and mother wavelets for the orthonormal wavelets for
N = 1, 2, 3, 4, using the cascade algorithm. Since the wavelets are orthonormal, we
should have that φ = φ̃, and ψ = ψ̃. In other words, you should see that the bottom
plots equal the upper plots.

Solution.

The code can look as follows.

m=10;
cascade_alg(m, -4, 4, ’db2’, 1)
cascade_alg(m, -4, 4, ’db2’, 0)
cascade_alg(m, -4, 4, ’db3’, 1)
cascade_alg(m, -4, 4, ’db3’, 0)
cascade_alg(m, -4, 4, ’db4’, 1)
cascade_alg(m, -4, 4, ’db4’, 0)
178 7 The polyphase representation of filter bank transforms

Exercise 7.10: Piecewise linear wavelet with 4 vanishing

moments

In Exercise 4.31 we found constants α, β, γ, δ which give the coordinates of ψ̂ in (φ1 , ψ̂1 ),
where ψ̂ was a mother wavelet in the MRA for piecewise linear functions with four
vanishing moments.
a) Show that the polyphase representation of G when ψ̂ is used as mother wavelet can
be factored as
  
1 I 0 I {−γ, −α, −β, −δ}
√ .
2 {1/2, 1/2} I 0 I
Find also a lifting factorization of H.

Solution.

We have found constants α, βγδ so that

[ψ̂](φ0 ,ψ0 ) = (−α, −β, −δ, 0, 0, 0, 0, −γ) ⊕ (1, 0, 0, 0, 0, 0, 0, 0),

From this it is clear that
 
I S2
P(φ1 ,ψ1 )←(φ1 ,ψ̂1 ) =
0 I
where S2 = {−γ, −α, −β, −δ} This gives as before the lifting factorization
  
1 I 0 I {−γ, −α, −β, −δ}
PD1 ←(φ1 ,ψ̂1 ) = √ . (7.1)
2 S1 I 0 I
where S1 = {1/2, 1/2} as before.

Exercise 7.11: Wavelet based on piecewise quadratic scaling

function

In Exercise 6.8 you should have found the filters

1
H0 = {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5}
128
1
H1 = {1, −4, 6, −4, 1}
16
1
G0 = {1, 4, 6, 4, 1}
16
1
G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5}.
128
Show that
7 The polyphase representation of filter bank transforms 179
1
I − 41 {1, 1}
    1 
I − 128 {5, −29, −29, 5} I 0 0
G= 4 ,
0 I −{1, 1} I 0 I 04
and derive a lifting factorization of G from this.

Solution.

The polyphase representation of the IDWT is

1 1

16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5} .
1 1
16 {4, 4} 128 {20, −96, 280, −96, 20}

We can first apply an even lifting step:

 1 1
I − 14 {1, 1}
 
16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5}
1 1
0 I 16 {4, 4} 128 {20, −96, 280, −96, 20}
 1 1

16 {4} 128 {20, −116, −116, 20}
= 1 1 .
16 {4, 4} 128 {20, −96, 280, −96, 20}

We can now apply an odd lifting step

  1 1
 1 1

I 0 16 {4} 128 {20, −116, −116, 20} = 4 128 {20, −116, −116, 20}
1 1
−{1, 1} I 16 {4, 4} 128 {20, −96, 280, −96, 20} 0 4

Since
1
  1 1
 1 
I − 512 {20, −116, −116, 20} 4 128 {20, −116, −116, 20} 0
= 4 ,
0 I 0 4 04
it follows that
1
I − 41 {1, 1}
    1 
I − 128 {5, −29, −29, 5} I 0 0
G= 4 .
0 I −{1, 1} I 0 I 04

Exercise 7.12: Connection between forward and reverse filter

bank transforms based on prototypes

In Exercise 5.33 we used a direct approach to relate G and H T . Insert instead (7.27)
in the book in equations (7.27) in the book and (7.23) in the book to show that
G = E480 H T and H = E480 GT . Also, combine these with GH = 16E480 to show that
H T H = GT G = 16. Thus, up to a scalar, the forward and reverse filter bank transforms
are orthogonal.

Solution.

We have that
180 7 The polyphase representation of filter bank transforms

GT G = E−480 HG = E−480 16E480 = 16

H T H = E−480 GH = E−480 16E480 = 16

Exercise 7.13: Run forward and reverse transform

Run the forward and then the reverse transform from Exercise 5.30 on the vector
(1, 2, 3, . . . , 8192). Verify that there seems to be a delay on 481 elements, as promised in
Section 7.3.4 in the book. Do you get the exact same result back?

Solution.

The following code can be used:

x = (1:8192)’;
x = mp3_reverse_fbt(mp3_forward_fbt(x));
plot(x)
x = (1:8192)’;
x = mp3_reverse_fbt(x);
x(1:size(x)-481) = x(482:size(x));
x = mp3_forward_fbt(x);
plot(x)

There are some small errors from the original vector in the resulting vector, when one
compensates for the delay of 481 elements.

Exercise 7.14: Verify statement of filters

Use your computer to verify the following symmetries for the prototype filters:
(
−C512−i i 6= 64, 128, . . . , 448
Ci =
C512−i i = 64, 128, . . . , 448.
Explain also that this implies that hi = h512−i for i = 1, . . . , 511. Recall that the
modified version had the symmetry hi = h511−i . When the filters V (m) are defined as in
this section, explain why (7.26) in the book should be replaced by

V (64−k) = −E14 (V (k) )T

in the MP3 standard

Solution.

The following code can be used

ctable = mp3_ctable();
ctable = ctable(2:end);
ctable(64:64:255) = -ctable(64:64:255);
res = ctable + ctable(end:-1:1);
7 The polyphase representation of filter bank transforms 181

res(256)=0;
max(abs(res))

Note that the middle element of the table is set to 0 in this verification. The reason is
that, for i = 256, there is nothing to check. So, since the verification works by producing
zero when there is symmetry, the element corresponding to i = 256 is set to zero. Also,
every 64’th element in one half of the table changed sign, in order to establish symmetry
for the corresponding entries, rather than anti-symmetry as for the other entries.

Exercise 7.15: Verify near perfect reconstruction in the MP3

standard

In Section 7.3.3 in the book we saw that the polyphase representation of GH is

 
S0 · · · 0
16  ... . . . ...  E480 ,
 

0 · · · S31
where Si = (V (16+i) )T V (16+i) + (V (48+i) )T V (48+i) . As explained in Section 7.3.4 in the
book, the MP3 standard chooses the prototype filter so that the Si are equal (so that
the block matrix above is a filter), but only near perfect reconstruction is possible.
a) With Si = {..., t−1 , t0 , t1 , ...}, explain why the (magnitude of the) frequency response
of (the filter) GH equals

X
16 tk e−iM kω .
k

Solution.

Disregarding the delay E480 , which does not affect the absolute value of the frequency
response, the filter coefficients of GH come at indices 0, M , 2M , and so on, and are
16t0 , 16t1 , 16t2 , and so on. This gives

X
|λGH (ω)| = 16 tk e−iM kω .
k

b) Use the values in the table C to find the filter coefficients tk for

S1 = (V (17) )T V (17) + (V (49) )T V (49) .

(use (7.20) in the book, even though this filter bank does not give perfect reconstruction),
and plot this frequency response. If you get an almost flat frequency response, you have
verified near perfect reconstruction in the MP3 standard. It is important that you scale
the vertical axis in order to see that the frequency response is close to constant.
182 7 The polyphase representation of filter bank transforms

Solution.

The following code can be used.

C = mp3_ctable();
V1=zeros(15,1);
V1(1:2:end)=C(18:64:end);
V2=zeros(15,1);
V2(1:2:end)=C(50:64:end);
diagentry = conv(flip(V1),V1)+conv(flip(V2),V2);
S = zeros(32*length(diagentry),1);
S(1:32:end) = diagentry; % upsampled
S = [S; zeros(16384,1)];
N = length(S);
n = (0:(N-1))’;
plot(2*pi*n/N, abs(fft(S)), ’k-’)
axis([0 2*pi 0 1.5/512])

c) Verify that the frequency response of S1 in b) should be close to 1/(16 · 32) = 1/512.
The factor 32 has to do with the relative scaling of the C and D tables.

Solution.

Since the values in the table D are 32 times those in the table C, we will instead have

X X
λS (ω) = (16 · 32) tk e−iM kω = 512 tk e−iM kω
k k

(recall that SiPrepresents values from the table C). For this to be close to one we must
−iM kω
clearly have k tk e ≈ 1/512.

Exercise 7.16: Lifting factorization of M -channel filter bank

transforms

The fact that step 2 and 3 in the MP3 encoding procedure can be computed in-place,
corresponds in our modified scheme to multiplying with the rightmost matrix in (7.21)
in the book in-place. This exercise addresses how this can be done. We omit some details
in that we make some simplifications, and don’t write down any code.
a) Use a lifting factorization of each of the 16 subblocks to show that (7.21) in the book
can be factored as a product of a diagonal matrix with entries on the form αi Edi on the
diagonal, a permutation matrix, and matrices on the form
7 The polyphase representation of filter bank transforms 183
   
I ··· 0 0 ···S15 I ··· 0 0 ··· 0
 .. .. .. .. . .
..   .. . . .. .. . . .. 
.
 . . . .
. 

 .
 . . . . . 
0
 ··· I S0 · · ·
0   0 ···
 , and  I 0 · · · 0 .
0
 ··· 0 I ··· 0  
 0 ···
 S0 I · · · 0 
. .. .. .. . .
..   . . .. .. . . .. 
 .. . . . . .  .. . . . . . .
0 ··· 0 0 ··· I S15 · · · 0 0 ··· I
This shows that lifting factorizations can be useful in saving arithmetic operations for
M -channel filter bank transforms as well.

Solution.

The mentioned permutation matrix is of size 16 × 16, and flips the numbers from 1 to 8
internally, and also the numbers from 9 to 16 internally. Using this it is clear that

· · · V (16) V (15) · · ·
 
0 0
 .. .. .. .. .. .. 
 . . . . . . 
 
 V (31) · · · 0 0 ··· V (0) 
  (P ⊗k I)
−V (63) E1 · · · 0 0 · · · V (32)
E 1

 
 .. .. .. .. .. .. 
 . . . . . . 
(48) (47)
0 · · · −V E1 V E1 · · · 0
(16)
··· · · · V (15)
 
V 0 0
 .. .. .. .. .. .. 
 . . . . . . 
 (31) (0)

 0 ··· V V ··· 0  .
=
 0 · · · −V (63) E1 V (32) E1 · · · 0  
 .. .. .. .. .. .. 
 . . . . . . 
−V (48) E1 · · · 0 0 · · · V (47) E1

The reason for using this permutation matrix is that it places the diagonal entries in each
subblock on the diagonal of the larger matrix. After this transformation the subblocks
have changed, and the first subblock consists of the blocks at positions (0, 0), (0, 31),
(31, 0), (31, 31). The other subblocks are found by going towards the middle of the matrix
in the four diagonal directions. Combining an even lifting step for all subblocks gives a
matrix on the form
 
I · · · 0 0 · · · S15
 .. . . .. .. . . .. 
. . . . . . 
 
0 · · · I S0 · · · 0 
0 · · · 0 I · · · 0  ,
 
 
. . . . . . 
 .. . . .. .. . . .. 
0 ··· 0 0 ··· I
while combining an odd lifting step for all subblocks gives a matrix on the form
184 7 The polyphase representation of filter bank transforms
 
I ··· 0 0 ··· 0
 .. . . .. .. . . .. 
 .
 . . . . . 
 0 ··· I 0 · · · 0
 .
 0 ··· S0 I · · · 0
 
 . . .. .. . . .. 
 .. . . . . . .
S15 · · · 0 0 ··· I
The reason that we can combine the lifting steps in this way is that each corresponds
to elementary row operations which do not affect the other subblocks. Finally, after
all the lifting steps each subblock is diagonal with entries on the form αi Edi on the
diagonal. Clearly then the matrix with 32 × 32 subblocks also is diagonal, with αi Edi
on the diagonal.
b) Assume the the 16 subblocks have a lifting factorization in terms of elementary
lifting matrices. Show that the filter representation of the first matrix listed in a) looks
as follows around the diagonal

0 · · · λM/2−1 1 · · · 0 0 · · · λM/2−1

 .. . . .. .. . . .. .. . . .. 
 .
 . . . . . . . . 

 λ0 · · · 0 0 · · · 1 λ0 · · · 0 
 ,
 0 ··· 0 0 ··· 0 1 ··· 0 
 
 . . . . . . .
 .. . . .. .. . . . .. .. . . . ..


0 ··· 0 0 ··· 0 0 ··· 1
where the shown blocks are of size (M/2) × (M/2), and the ones are on the main
diagonal. Convince yourself that M/2 consecutive rows are unaltered, while the other
M/2 consecutive rows are altered by combining with the M/2 rows above and below,
so that a composition of these matrices can be computed in-place. This also gives a
reduction in the number of arithmetic operations with up to 50 percent, since this is the
case for each subblock.

Solution.

For a block row of the first matrix listed in a), since Si = λi {1, 1} if even elementary
lifting steps are assumed, the only nonzero entries in a row in the filter representation
will be
• a 1 on the diagonal,
• λi 31 − i to the left of the diagonal, and
• λi i + 1 to the right of the diagonal.
Writing this down for different i we obtain the matrix from the text.
c) Show also that the matrices in a) can be inverted by adding a minus sign to all the
Si -components.
7 The polyphase representation of filter bank transforms 185

Solution.

For elementary lifting matrices we know that inversion is achieved by adding a minus
sign. Since the different subblocks don’t interact with the others in multiplication, we
simply have to add a minus sign to all the Si to find the inverse.
This exercise can also be used in constructing useful cosine-modulated filter banks
with perfect reconstruction: Simply let the λi in the “generalized lifting steps” in b) be
parameters one needs to optimize in some way.

Exercise 7.17: DFT modulated filter banks

−1
Assume that {Si }M
i=0 are all-pass filters (see Exercise 3.44). Show that
 
S0 · · · 0
(DFTM ⊗k I)  ... . . . ... 
 

0 · · · SM −1
and

(S0 )T · · ·
 
0
 .. . . .. k
 (IDFTM ⊗ I)

 . . .
0 · · · (SM −1 )T
invert each other. By realizing this in terms of polyphase components, explain how
one can assemble a prototype filter from all-pass filters, and forward and reverse DFT-
modulated filter banks (i.e. where the filters are obtained by modulating the prototype
with e2πikn/M ), so that we have perfect reconstruction.

Solution.

This follows from the fact hat (Si )T Si = I from allpass filters, and from that

(DFTM ⊗k I)(IDFTM ⊗k I) = (DFTM IDFTM ) ⊗k I = I ⊗k I = I.

This is now very similar to the previous construction. The difference is that, while
we previously factored subblocks into lifting steps we now instead factor the all-pass
filters on the diagonal into simpler ones, following for instance the procedure from
Exercise 3.44.
Chapter 8
Digital images

Exercise 8.6: Generating black and white images

Black and white images can be generated from grayscale images (with values between 0
and 255) by replacing each pixel value with the one of 0 and 255 which is closest. Use
this strategy to generate the black and white image shown in the right part of Figure 8.2
in the book.

Solution.

The following code can be used

X2 = 255.*(X1 >= 128);

imshow(uint8(X2))

Exercise 8.7: Adjusting contrast

a) Write a function which instead uses the function fn from Example 8.5 in the book
to adjust the contrast (rather than g ). n should be a parameter to your function.

Solution.

The code could look as follows:

function Z=contrast_adjust0(X, n)
Z = X/255; % Maps the pixel values to [0,1]
Z = atan(n*(Z-1/2))/(2*atan(n/2)) + 1/2;
Z = Z*255; % Maps the values back to [0,255]
end

b) Generate the left and right images in Figure 8.7 in the book on your own by using
code which calls the function you have written, as well as contrast_adjust.

187
188 8 Digital images

Solution.

The following code can be used.

X1 = contrast_adjust0(img, 10);
imshow(uint8(X1))

X2 = contrast_adjust(img, 0.01);
imshow(uint8(X2))

Exercise 8.8: Adjusting contrast with another function

a) Show that the function hn : R → R given by

hn (x) = xn ,
maps the interval [0, 1] → [0, 1] for any n, and that h0n (1) → ∞ as n → ∞.

Solution.

We have that h0n (x) = nxn−1 , so that h0n (1) = n → ∞.

b) The image secret.jpg, shown in Figure 8.1, contains some information that is
nearly invisible to the naked eye on most monitors. Use the function hn , to reveal the
secret message.

Fig. 8.1 Secret message.

8 Digital images 189

Hint.

Convert to a grayscale image before you adjust the contrast with hn .

Solution.

The following code can be used

n = 100;
X = double(imread(’images/secret.jpg’, ’jpg’));
X = (X(:,:,1) + X(:,:,2) + X(:,:,3))/3;
X = map_to_01(X);
X = X.^n;
X = 255*X;
imshow(uint8(X))

The resulting image is shown in Figure 8.2.

Fig. 8.2 Secret message revealed!

Exercise 8.13: Generating images

Write code which calls the function tensor2_impl with appropriate filters to produce
the following images:
a) The right image in Figure 8.10 in the book.

Solution.

The following code can be used

190 8 Digital images

longmolecule = @(x, bd_mode) filter_impl([1, 6, 15, 20, 15, 6, 1]/64, ...

x, bd_mode);
X3 = tensor2_impl(X1, longmolecule, longmolecule, ’symm’);
imshow(uint8(X3))

b) The right image in Figure 8.12 in the book.

Solution.

The following code can be used

X3 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’);

X3 = map_to_01(X3);
X3 = X3*255;
X3 = contrast_adjust0(X3,50);
imshow(uint8(X3))

c) The images in Figure 8.14 in the book.

Solution.

The following code can be used

excerpt=create_excerpt();
X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’);
X2 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’);
X1 = X1.^2 + X2.^2;
imshow(uint8(X1))

X2 = X1;
X2 = map_to_01(X2);
X2 = X2*255;
imshow(uint8(X2))

X3 = X2;
X3 = contrast_adjust0(X3, 50);
imshow(uint8(X3))

d) The images in Figure 8.13 in the book.

Solution.

The following code can be used

excerpt=create_excerpt();
X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’);
X1 = map_to_01(X1);
X1 = X1*255;
X1 = contrast_adjust0(X1, 50);
imshow(uint8(X1))
8 Digital images 191

diffymolecule = @(x, bd_mode) filter_impl([-1 0 1]/2, x, bd_mode);

X2 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’);
X2 = map_to_01(X2);
X2 = X2*255;
X2 = contrast_adjust0(X2, 50);
imshow(uint8(X2))

e) The images in Figure 8.15 in the book.

Solution.

The following code can be used

excerpt=create_excerpt();
X1 = tensor2_impl(excerpt, donothing, diffxmolecule, ’per’);
X1 = tensor2_impl(X1, donothing, diffxmolecule, ’per’);
X1 = map_to_01(X1);
X1 = X1*255;
X1 = contrast_adjust0(X1, 100);
imshow(uint8(X1))

X2 = tensor2_impl(excerpt, diffymolecule, diffxmolecule, ’per’);

X2 = map_to_01(X2);
X2 = X2*255;
X2 = contrast_adjust0(X2, 100);
imshow(uint8(X2))

X3 = tensor2_impl(excerpt, diffymolecule, donothing, ’per’);

X3 = tensor2_impl(X3, diffymolecule, donothing, ’per’);
X3 = map_to_01(X3);
X3 = X3*255;
X3 = contrast_adjust0(X3, 100);
imshow(uint8(X3))

Exercise 8.14: Interpret tensor products

Let the filter S be defined by S = {−1, 1}. Write down the 4 × 4-matrix X = (1, 1, 1, 1) ⊗
(0, 0, 1, 1), and compute (S ⊗ I)X and (I ⊗ S)X.

Solution.

We have that
 
0 0 1 1
0 0 1 1
(1, 1, 1, 1) ⊗ (0, 0, 1, 1) = 
0
.
0 1 1
0 0 1 1
For the tensor products we first compute
192 8 Digital images

S(1, 1, 1, 1) = (0, 0, 0, 0) S(0, 0, 1, 1) = (0, 0, −1, 0)

(assuming symmetric extensions). We then obtain

 
0000
0 0 0 0
(S ⊗ I)X = (S(1, 1, 1, 1)) ⊗ (0, 0, 1, 1) = (0, 0, 0, 0) ⊗ (0, 0, 1, 1) =  
0 0 0 0
0000
 
0 0 −1 0
0 0 −1 0
(I ⊗ S)X = (1, 1, 1, 1) ⊗ (S(0, 0, 1, 1)) = (1, 1, 1, 1) ⊗ (0, 0, −1, 0) = 
0 0 −1 0 .


0 0 −1 0

Exercise 8.15: Computational molecule of moving average

filter

1
Let S be the moving average filter of length 2L + 1, i.e. T = L {1,
|
··· , 1, 1, 1, · · · , 1}.
{z }
2L+1 times
What is the computational molecule of S ⊗ S?

Solution.

The molecule is a (2L + 1) × (2L + 1)-matrix with all entries equal to 1/L2 .

Exercise 8.16: Bilinearity of the tensor product

Show that the mapping F (x, y) = x ⊗ y is bi-linear, i.e. that F (αx1 + βx2 , y) =
αF (x1 , y) + βF (x2 , y), and F (x, αy1 + βy2 ) = αF (x, y1 ) + βF (x, y2 ).

Solution.

We have that

F (αx1 + βx2 , y) = (αx1 + βx2 ) ⊗ y = (αx1 + βx2 )y T

= αx1 y T + βx2 y T = α(x1 ⊗ y) + β(x ⊗ y)
= αF (x1 , y) + βF (x1 , y).

The second statement follows similarly.

8 Digital images 193

Exercise 8.17: Attempt to write as tensor product

Attempt to find matrices S1 : RM → RM and S2 : RN → RN so that the following

mappings from LM,N (R) to LM,N (R) can be written on the form X → S1 X(S2 )T =
(S1 ⊗ S2 )X. In all the cases it may be that no such S1 , S2 can be found. If this is the
case, prove it.
a) The mapping which reverses the order of the rows in a matrix.

Solution.

Multiplication with the matrix

 
0 0 0 ··· 01 0
0
 0 0 ··· 00 1
S =  ... .. .. .. ..
.. ..

 . . . .. .
0 1 0 · · · 0 0 0
1 0 0 ··· 0 0 0
reverses the elements in a vector. This means that

((S ⊗ I)(x ⊗ y))i,j = ((Sx) ⊗ y)i,j = (Sx)i yj = xM −1−i yj = (x ⊗ y)M −1−i,j .

This means that also ((S ⊗ I)X)i,j = XM −1−i,j for all X, so that S ⊗ I reverses rows,
and thus is a solution to a).
b) The mapping which reverses the order of the columns in a matrix.

Solution.

Similarly one shows that I ⊗ S reverses columns, and is thus a solution to b).
c) The mapping which transposes a matrix.

Solution.

It turns out that it is impossible to find S1 and S2 so that transposing a matrix X

corresponds to computing (S1 ⊗ S2 )X. To see why, S1 and S2 would need to fulfill

(S1 ⊗ S2 )(ei ⊗ ej ) = (S1 ei ) ⊗ (S2 ej ) = ej ⊗ ei ,

since ej ⊗ ei is the transpose of ei ⊗ ej . This would require that S1 ei = ej for all i, j,
which is impossible.

Exercise 8.18: Computational molecules

Let the filter S be defined by S = {1, 2, 1}.

194 8 Digital images

a) Write down the computational molecule of S ⊗ S.

Solution.

The computational molecule of S ⊗ S is

   
1
121
(1, 2, 1) ⊗ (1, 2, 1) = (1, 2, 1) ⊗ (1, 2, 1) = 2 1 2 1 = 2 4 2 .
1 121

b) Let us define x = (1, 2, 3), y = (3, 2, 1), z = (2, 2, 2), and w = (1, 4, 2). Compute the
matrix A = x ⊗ y + z ⊗ w.

Solution.

We get that

   
1
2

A = 2 3 2 1 + 2 1 4 2
3 2
     
321 284 5 10 5
= 6 4 2 + 2 8 4 =  8 12 6 .
963 284 11 14 7

c) Compute (S ⊗ S)A by applying the filter S to every row and column in the matrix
the way we have learned. If the matrix A was more generally an image, what can you
say about how the new image will look?

Solution.

We need to compute (S ⊗ S)A = SAS T , which corresponds to first applying S to every

column in the image, and then applying S to every row in the resulting image. If we
apply S to every column in the image we first get the matrix
 
29 46 23
SA = 32 48 24 .
35 50 25
If we apply the filter to the rows here we get
 
127 144 121
SAS T = 136 152 128 .
145 160 135
Since the filter which is applied is a low-pass filter, the new image should look a bit
more smooth than the original image.
8 Digital images 195

Exercise 8.19: Computational molecules 2

Let S = 14 {1, 2, 1} be a filter.

a) What is the effect of applying the tensor products S ⊗ I, I ⊗ S, and S ⊗ S on an
image represented by the matrix X?

Solution.

Note first that the filter is a low-pass filter. We know that S ⊗ I corresponds to applying
S to the columns of the matrix, so that we get the result by applying the low-pass filter
to the columns of the matrix. The result of this is that horizontal edges are smoothed.
Similarly, the tensor product I ⊗ S corresponds to applying S to the rows of the matrix,
so that vertical edges are smoothed. Finally, S ⊗ S corresponds to applying S first to the
columns of the matrix, then to the rows. The result is that both horizontal and vertical
edges are smoothed. You could also have computed the computational molecules for
S ⊗ I, I ⊗ S, and S ⊗ S, by taking the tensor product of the filter coefficients 14 {1, 2, 1}
with itself. From these molecules it is also clear that they either work on the columns,
the rows, or on both rows and columns.
b) Compute (S ⊗ S)(x ⊗ y), where x = (4, 8, 8, 4), y = (8, 4, 8, 4) (i.e. both x and y
are column vectors).

Solution.

A 4 × 4 circulant Toeplitz matrix for S is

 
2 1 0 1
11 2 1 0.
4 0 1 2 1
1 0 1 2
From this we can quickly compute that

      
2 1 0 1 4 2+2+1 5
1 1 2 1 0 8 4 + 2 + 1 7
Sx =    =  = 
4 0
 1 2 1 8 4 + 2 + 1 7
1 0 1 2 4 2+2+1 5
      
2 1 0 1 8 4+1+1 6
1 1 2 1 0 4 2 + 2 + 2 6
Sy =    =   =  .
4 0
 1 2 1 8 4 + 1 + 1 6
1 0 1 2 4 2+2+2 6

From this it is clear that

   
5 30 30 30 30
T
7
42 42 42 42
(S ⊗ S)(x ⊗ y) = (Sx)(Sy) = 
7 6 6 6 6 = 42
  .
42 42 42
5 30 30 30 30
196 8 Digital images

Exercise 8.20: Eigenvectors of tensor products

Let vA be an eigenvector of A with eigenvalue λA , and vB an eigenvector of B with

eigenvalue λB . Show that vA ⊗ vB is an eigenvector of A ⊗ B with eigenvalue λA λB .

Solution.

We have that

(A ⊗ B)(vA ⊗ vB ) = (AvA ) ⊗ (BvB ) = (λA vA ) ⊗ (λB vB ) = λA λB (vA ⊗ vB ).

Exercise 8.21: The Kronecker product

The Kronecker tensor product of two matrices A and B, written A ⊗k B, is defined as

 
a1,1 B a1,2 B · · · a1,M B
a2,1 B a2,2 B · · · a2,M B 
A ⊗k B =  . ..  ,
 
.. ..
 .. . . . 
ap,1 B ap,2 B · · · ap,M B
where the entries of A are ai,j . The tensor product of a p×M -matrix, and a q ×N -matrix
is thus a (pq) × (M N )-matrix. Note that this tensor product in particular gives meaning
for vectors: if x ∈ RM , y ∈ RN are column vectors, then x ⊗k y ∈ RM N is also a column
vector. In this exercise we will investigate how the Kronecker tensor product is related
to tensor products as we have defined them in this section.
a) Explain that, if x ∈ RM , y ∈ RN are column vectors, then x ⊗k y is the column
vector where the rows of x ⊗ y have first been stacked into one large row vector, and
this vector transposed. The linear extension of the operation defined by

x ⊗ y ∈ RM,N → x ⊗k y ∈ RM N
thus stacks the rows of the input matrix into one large row vector, and transposes the
result.

Solution.

We have that
 
x1 y
 x2 y 
x ⊗k y = 
 
.. 
 . 
xM y
The rows of x ⊗ y are xi y T . Stacking these into one large row vector we obtain

x1 y T x2 y T · · · xM y T ,


8 Digital images 197

which is the transpose of what we obtained above.

b) Show that (A ⊗k B)(x ⊗k y) = (Ax) ⊗k (By). We can thus use any of the defined
tensor products ⊗, ⊗k to produce the same result, i.e. we have the commutative diagram
shown in Figure 8.3, where the vertical arrows represent stacking the rows in the matrix,
and transposing, and the horizontal arrows represent the two tensor product linear
transformations we have defined. In particular, we can compute the tensor product
in terms of vectors, or in terms of matrices, and it is clear that the Kronecker tensor
product gives the matrix of tensor product operations.

A⊗B
x⊗y / (Ax) ⊗ (By)

 
A⊗k B
x ⊗k y / (Ax) ⊗k (By),

Fig. 8.3 Tensor products

Solution.

We have that

(A ⊗k B)(x ⊗k y)
    
a11 B a12 B · · · a1M B x1 y (a11 x1 + . . . + a1M xm )By
a21 B a22 B · · · a2M B   x2 y  (a21 x1 + . . . + a2M xm )By 
= . ..   ..  = 
    
.. . . ..
 ..

. . .   .   . 
ap1 B ap2 B · · · apM B xM y (ap1 x1 + . . . + apM xm )By
 
(Ax)1 By
(Ax)2 By 
=  = (Ax) ⊗k (By).
 
..
 . 
(Ax)p By

c) Using the Euclidean inner product on L(M, N ) = RM N , i.e.

M
X −1 N
X −1
hX, Y i = Xi,j Yi,j .
i=0 j=0

and the correspondence in a) we can define the inner product of x1 ⊗ y1 and x2 ⊗ y2 by

hx1 ⊗ y1 , x2 ⊗ y2 i = hx1 ⊗k y1 , x2 ⊗k y2 i.
Show that

hx1 ⊗ y1 , x2 ⊗ y2 i = hx1 , x2 ihy1 , y2 i.

Clearly this extends linearly to an inner product on LM,N .
198 8 Digital images

Solution.

We have that

   
* (x1 )0 y1 (x2 )0 y2 + M −1
hx1 ⊗ y1 , x2 ⊗ y2 i =  .
.. .
..
X
(x1 )i (x2 )i hy1 , y2 i
,  =
   

(x1 )M −1 y1 (x2 )M −1 y2 i=0

M
X −1
= hy1 , y2 i (x1 )i (x2 )i = hx1 , x2 ihy1 , y2 i.
i=0

d) Show that the FFT factorization can be written as

   
FN/2 DN/2 FN/2 I DN/2
= N/2 (I2 ⊗k FN/2 ).
FN/2 −DN/2 FN/2 IN/2 −DN/2
Also rewrite the sparse matrix factorization for the FFT from equation (2.16) in the
book in terms of tensor products.

Solution.

We have that

      
FN/2 DN/2 FN/2 IN/2 DN/2 FN/2 0 IN/2 DN/2
= = (I2 ⊗k FN/2 ).
FN/2 −DN/2 FN/2 IN/2 −DN/2 0 FN/2 IN/2 −DN/2

Equation (2.16) in the book can clearly be rewritten as

log2 N
Y  
I DN/2k

DFTN = I2k−1 ⊗ P.
I −DN/2k
k=1

Exercise 8.24: Implementing DFT and DCT on blocks

In this section we have used functions which apply the DCT and the DFT either to
subblocks of size 8 × 8, or to the full image.
a) Implement functions dft_impl8, idft_impl8, dct_impl8, and idct_impl8 which
apply the DFT, IDFT, DCT, and IDCT, to consecutive blocks of length 8.

Solution.

The following code can be used.

function x = dft_impl8(x, bd_mode)

N = size(x, 1);
for n = 1:8:N
x(n:(n+7), :) = fft(x(n:(n+7), :));
8 Digital images 199

end
end

function x = idft_impl8(x, bd_mode)

N = size(x, 1);
for n = 1:8:N
x(n:(n+7), :) = ifft(x(n:(n+7), :));
end
end

function x = dct_impl8(x, bd_mode)

N = size(x, 1);
for n = 1:8:N
x(n:(n+7), :) = dct(x(n:(n+7), :));
end
end

function x = idct_impl8(x, bd_mode)

N = size(x, 1);
for n = 1:8:N
x(n:(n+7), :) = idct(x(n:(n+7), :));
end
end

b) Implement the two-dimensional FFT, IFFT, DCT, and IDCT on images, with the
help of their one-dimensional counterparts, as well as the function tensor2_impl.

Solution.

The following code can be used.

X = tensor2_impl(img, @(x, bd_mode) fft(x(:,:)), ...

@(x, bd_mode) fft(x(:,:)), ’symm’);
X = tensor2_impl(img, @(x, bd_mode) ifft(x(:,:)), ...
@(x, bd_mode) ifft(x(:,:)), ’symm’);

X = tensor2_impl(img, @(x, bd_mode) dct(x(:,:)), ...

@(x, bd_mode) dct(x(:,:)), ’symm’);
X = tensor2_impl(img, @(x, bd_mode) idct(x(:,:)), ...
@(x, bd_mode) idct(x(:,:)), ’symm’);

c) The function forw_comp_rev_2d in the library applies given transforms to the rows
and columns of an input image, sets the coefficients below a certain threshold to zero,
and transforms back. Run forw_comp_rev_2d for different threshold parameters on the
sample image, and with the functions dct_impl8, idct_impl8, as well as DCT and
IDCT applied to the entire input, as parameters. Check that this reproduces the DCT
test images of this section, and that the correct numbers of values which have been
discarded (i.e. which are below the threshold) are printed on screen.
200 8 Digital images

Solution.

Once

X1 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 30);

imshow(uint8(X1))

X2 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 50);

imshow(uint8(X2))

X3 = forw_comp_rev_2d(X, @dct_impl8, @idct_impl8, 100);

imshow(uint8(X3))

X1 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ...

@(x, bd_mode) idct(x(:,:)), 30);
imshow(uint8(X1))

X2 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ...

@(x, bd_mode) idct(x(:,:)), 50);
imshow(uint8(X2))

X3 = forw_comp_rev_2d(X, @(x, bd_mode) dct(x(:,:)), ...

@(x, bd_mode) idct(x(:,:)), 100);
imshow(uint8(X3))

Exercise 8.25: Code example

Suppose that we have given an image by the matrix X. Consider the following code:

threshold = 30;
X = fft_impl(X, @fft_kernel_standard);
X = X’;
X = fft_impl(X, @fft_kernel_standard);
X = X’;
X = X.*(abs(X) >= threshold);
X = fft_impl(X, @fft_kernel_standard, 0);
X = X’;
X = fft_impl(X, @fft_kernel_standard, 0);
X = X’;

Comment what the code does. Comment in particular on the meaning of the parameter
threshold, and its effect on the image.

Solution.

In the first part of the code one makes a change of coordinates with the DFT. More
precisely, this is a change of coordinates on a tensor product, as we have defined it.
In the last part the change of coordinates is performed the opposite way. Both these
change of coordinates is performed is performed the way we have described them, first
8 Digital images 201

on the rows in the matrix, then on the columns. The parameter threshold is used to
discard DFT-coefficients which are below a certain value. We have seen that this can
give various visual artifacts in the image, even though the main contents of the image
still may be visible. If we increase threshold, these artifacts will be more dominating
since we then discard many DFT-coefficients.
Chapter 9
Using tensor products to apply wavelets to
images

Exercise 9.3: Alternative definition of the 2D DWT

In terms of tensor products, the way we defined the two-dimensional DWT may not seem
very natural. Why not define it as the tensor product of two m-level DWT’s, instead of
iterated tensor products of 1-level DWT’s)?
a) Explain that this alternative definition would alter the vectors listed in (9.10) in the
book to

{φ0,n1 ⊗ ψk2 ,n2 , ψk1 ,n1 ⊗ φ0,n2 , ψk1 ,n1 ⊗ ψk2 ,n2 }n1 ,n2 ,0≤k1 ,k2 <m .

Solution.

This simply amounts to combining functions from {φ0,n1 , ψ0,n1 , ..., ψm−1,n1 } with func-
tions from {φ0,n2 , ψ0,n2 , ..., ψm−1,n2 } in all possible ways. The difference from the previ-
ous situation is that we now allow for combinations where the components have different
resolutions k1 and k2 .
b) Explain that, with this alternative definition, dwt2_impl_internal
and idwt2_impl_internal could be implemented simply as

f = @(x, bd_mode) dwt_impl(x, ’cdf97’, 4, bd_mode, ’none’, 1);

img = tensor2_impl(img, f, f, ’symm’);

and

invf = @(x, bd_mode) idwt_impl(x, ’cdf97’, 4, bd_mode, ’none’, 1);

img = tensor2_impl(img, invf, invf, ’symm’);

This is much simpler than the implementation we have presented.

Solution.

This follows from that the new definition is a simple tensor product, for which
tensor2_impl does the job.
There seems to be few comments in the literature on why the two-dimensional DWT
is not defined as a tensor product of two m-level DWT’s.

203
204 9 Using tensor products to apply wavelets to images

Exercise 9.4: Higher order tensor products

Tensor products can easily be generalized to higher dimensions. For 3D, and for x ∈ RM ,
y ∈ RN , z ∈ RK , we define x ⊗ y ⊗ z as the "3D matrix" with entries

(x ⊗ y ⊗ z)i,j,k = xi yj zk ,
and if S1 , S2 and S3 are linear transformations on RM , RN , and RK , respectively, their
tensor product is the unique linear transformation S1 ⊗ S2 ⊗ S3 defined on 3D matrices
by

(S1 ⊗ S2 ⊗ S3 )(x ⊗ y ⊗ z) = (S1 x) ⊗ (S2 y) ⊗ (S3 z),

for all x, y, and z.
a) Explain that S1 ⊗ S2 ⊗ S3 can be implemented by
• replacing x0:(M −1),j,k by S1 (x0:(M −1),j,k ) for all j, k,
• replacing xi,0:(N −1),k by S2 (xi,0:(N −1),k ) for all i, k,
• replacing xi,j,0:(K−1) by S3 (xi,j,0:(K−1) ) for all j, k,
and that the order of these three operations does not matter. In other words, higher
order tensor products can also be split into a series of one-dimensional operations.

Solution.

Similarly to the 2D case we can write

S1 ⊗ S2 ⊗ S3 = (S1 ⊗ I ⊗ I)(I ⊗ S2 ⊗ I)(I ⊗ I ⊗ S3 ),

where the order of the three right hand factors does not matter. Also, by adding terms,
we can write any 3D matrix as
X
(x0:(M −1),j,k ) ⊗ ej ⊗ ek ,
i,j

and applying (S1 ⊗ I ⊗ I) to this we obtain

X
(S1 x0:(M −1),j,k ) ⊗ ej ⊗ ek .
i,j

This proves the claim that we can apply S1 to the different “(:, j, k)-segments”. The
cases for I ⊗ S2 ⊗ I and I ⊗ I ⊗ S3 follow similarly.
b) In particular we can extend the tensor product to wavelets in 3D. How many detail
components will there be at each level in a 3D DWT, following the setup we did for the
2D DWT? How would you label the different components?

Solution.

Duplicating the theory for changes of coordinates in 2D, we see that the target basis for
a 1-level 3D DWT consists of 23 = 8 different types of basis functions:
9 Using tensor products to apply wavelets to images 205

φ0,n1 ⊗ φ0,n2 ⊗ φ0,n3 ψ0,n1 ⊗ φ0,n2 ⊗ φ0,n3

φ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3 φ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3
ψ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3 ψ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3
φ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3 ψ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3 .

Using as before L for φ, H for ψ, these would be labeled LLL, HLL, LHL, LLH, HHL,
HLH, LHH, and HHH, respectively.
dwt3_impl_internal and idwt3_impl_internal are the functions in the library
which compute the 3D DWT and the 3D IDWT. You are in particular encouraged to
review the function tensor3_impl, which is used in the functions, and is a straightforward
generalization of tensor2_impl. Also in the 3D case there is a simpler alternative
definition for the DWT, following the setup from the previous exercise.

Exercise 9.9: Code example

Assume that we have an image represented by the M × N -matrix X, and consider the
following code:

c = (X(1:2:M, :) + X(2:2:M, :))/sqrt(2);

w = (X(1:2:M, :) - X(2:2:M, :))/sqrt(2);
X = [c; w];
c = (X(:, 1:2:N) + X(:, 2:2:N))/sqrt(2);
w = (X(:, 1:2:N) - X(:, 2:2:N))/sqrt(2);
X = [c w];

a) Comment what the code does, and explain what you will see if you display X as an
image after the code has run.

Solution.

The code runs a DWT over one level, and the Haar wavelet is used. Inside the for-
loops the DWT is applied to every row and column in the image. k=1 i the for-loop
corresponds to applying the DWT to the columns, k=2 corresponds to applying the
DWT to the rows. In the upper left corner we will see a low-resolution version of the
image. In the other three corners you will see different types of detail: In the upper right
corner you will see detail which corresponds to quick vertical changes, in the lower left
corner you will see detail which corresponds to quick horizontal changes, and in the lower
right corner you will see points where quick changes both vertically and horizontally
occur simultaneously.
b) The code above has an inverse transform, which reproduces the original image from
the transformed values. Assume that you zero out the values in the lower left and the
upper right corner of the matrix X after the code above has run, and that you reproduce
the image by applying this inverse transform. What changes do you expect in the image?
206 9 Using tensor products to apply wavelets to images

Solution.

By zeroing out the two corners you remove detail which correspond to quick horizontal
and vertical changes. But since we keep the lower right corner, we keep detail which
corresponds to simultaneous changes vertically and horizontally. The result after the
inverse transformation is that most edges have been smoothed, but we see no smoothing
effect in points where quick changes occur both horizontally and vertically. In Example 9.5
in the book, this corresponds to that we emphasize the grid points in the chess pattern,
mut that we smooth out the horizontal and vertical edges in the chess pattern.

Exercise 9.10: Applying the Haar wavelet to another chess

pattern image

The following code produces a chess pattern type image almost identical to that from
Example 9.5 in the book.

N = 128;
img=zeros(N);
for x=0:(N-1)
for y=0:(N-1)
img(x+1,y+1)=255*( (mod(x,64)>=32) == (mod(y,64)>=32) );
end
end
imshow(uint8(img))

Let us now apply a 2D DWT to this image as well with the Haar wavelet:

img2 = img;
img2 = dwt_impl(img2, ’Haar’, 1, ’per’, ’none’, 2);
img2 = map_to_01(img2);
img2 = img2*255;
imshow(uint8(img2))

The resulting images are shown in Figure 9.1

Fig. 9.1 A simple image before and after one level of the 2D DWT. The Haar wavelet was used.
9 Using tensor products to apply wavelets to images 207

There seem to be no detail components here, which is very different from what you
saw in Example 9.5 in the book, even though the images are very similar. Attempt to
explain what causes this to happen.

Hint.

Compare with Exercise 4.14.

Solution.

In Example 9.5 in the book, the borders in the chess pattern was chosen so that they
occur at odd numbers. This means that the image can not be represented exactly in
Vm−1 ⊗ Vm−1 , so that there is detail present in the image at all the borders in the
chess pattern. In the new image, the borders in the chess pattern was chosen so that
they occur at even numbers. This means that the image can be represented exactly in
Vm−1 ⊗ Vm−1 , so that there is no detail components present in the image.

Exercise 9.11: Implementing the fingerprint compression

scheme

Write code which generates the images shown in figures 9.5 in the book, 9.5 in the book,
and 9.23 in the book. Use the functions dwt_impl and idwt_impl with the CDF 9/7
wavelet.

Solution.

The following code generates the different parts of Figure 9.5 in the book.

img=double(imread(’images/fingerprint_500x500.png’));
imgnew=255*ones(512,512,3);
imgnew(1:500,1:500,:)=img;
img=imgnew;
l1 = 512; l2 = 512; l3 = 3;
X0 = dwt_impl(img, ’cdf97’, 1);
X1 = X0;
X0 = map_to_01(X0);
X0 = X0*255;
imshow(uint8(X0))

X1(1:(l1/2), 1:(l2/2), :) = dwt_impl(X1(1:(l1/2), 1:(l2/2), :), ’cdf97’, 1);

X1(1:(l1/2), (l2/2+1):end, :) = ...
dwt_impl(X1(1:(l1/2), (l2/2+1):end, :), ’cdf97’, 1);
X1((l1/2+1):end, 1:(l2/2), :) = ...
dwt_impl(X1((l1/2+1):end, 1:(l2/2), :), ’cdf97’, 1);
X1((l1/2+1):end, (l2/2+1):512, :) = ...
dwt_impl(X1((l1/2+1):end, (l2/2+1):512, :), ’cdf97’, 1);
X2 = X1;
X1 = map_to_01(X1);
X1 = X1*255;
imshow(uint8(X1))
208 9 Using tensor products to apply wavelets to images

X2(1:(l1/4), 1:(l2/4), :) = dwt_impl(X2(1:(l1/4), 1:(l2/4), :), ’cdf97’, 1);

X2(1:(l1/4), (l2/4+1):(l2/2), :) = ...
dwt_impl(X2(1:(l1/4), (l2/4+1):(l2/2), :), ’cdf97’, 1);
X2((l1/4+1):(l1/2), 1:(l2/4), :) = ...
dwt_impl(X2((l1/4+1):(l1/2), 1:(l2/4), :), ’cdf97’, 1);
X3 = X2;
X2 = map_to_01(X2);
X2 = X2*255;
imshow(uint8(X2))

X3(1:64, 1:64, :) = dwt_impl(X3(1:64, 1:64, :), ’cdf97’, 1);

X3(1:64, 65:128, :) = dwt_impl(X3(1:64, 65:128, :), ’cdf97’, 1);
X3(1:64, 129:192, :) = dwt_impl(X3(1:64, 129:192, :), ’cdf97’, 1);
X3(1:64, 193:256, :) = dwt_impl(X3(1:64, 193:256, :), ’cdf97’, 1);
X3(65:128, 1:64, :) = dwt_impl(X3(65:128, 1:64, :), ’cdf97’, 1);
X3(65:128, 65:128, :) = dwt_impl(X3(65:128, 65:128, :), ’cdf97’, 1);
X3(65:128, 129:192, :) = dwt_impl(X3(65:128, 129:192, :), ’cdf97’, 1);
X3(65:128, 193:256, :) = dwt_impl(X3(65:128, 193:256, :), ’cdf97’, 1);
X3(129:192, 1:64, :) = dwt_impl(X3(129:192, 1:64, :), ’cdf97’, 1);
X3(129:192, 65:128, :) = dwt_impl(X3(129:192, 65:128, :), ’cdf97’, 1);
X3(193:256, 1:64, :) = dwt_impl(X3(193:256, 1:64, :), ’cdf97’, 1);
X3(193:256, 65:128, :) = dwt_impl(X3(193:256, 65:128, :), ’cdf97’, 1);
Z = X3;
X3 = map_to_01(X3);
X3 = X3*255;
imshow(uint8(X3))

The following code generates Figure 9.5 in the book.

Z(1:32,1:32,:) = dwt_impl(Z(1:32,1:32,:), ’cdf97’, 1);

Y0 = Z;
Z = map_to_01(Z);
Z = Z*255;
Z = uint8(Z);
imshow(Z)

The following code generates the parts of Figure 9.23 in the book.

Y0(17:end, :, :) = 0;
Y0(1:16, 17:end, :) = 0;
Y0 = idwt_impl(Y0, ’cdf97’, 5);
Y1 = Y0;
Y0 = map_to_01(Y0);
Y0 = Y0*255;
imshow(uint8(Y0))

Y1 = dwt_impl(img, ’cdf97’, 5);

Y1(1:16, 1:16, :) = 0;
Y1 = idwt_impl(Y1, ’cdf97’, 5);
Y1 = map_to_01(Y1);
Y1 = Y1*255;
imshow(uint8(Y1))
References

[1] A. Cohen, I. Daubechies, and J-C. Feauveau. Biorthogonal bases of compactly

supported wavelets. Communications on Pure and Appl. Math., 45(5):485–560, June
1992.
[2] P. Duhamel and H. Hollmann. ’split-radix’ FFT-algorithm. Electronic letters,
20(1):14–16, 1984.
[3] S. Foucart and H. Rauhut. A mathematical introduction to compressive sensing.
Birkhauser, 2013.
[4] F. J. Harris. On the use of windows for harmonic analysis with the discrete Fourier
transform. Proceedings of the IEEE, 66(1), January 1978.
[5] M. Vetterli and H. J. Nussbaumer. Simple FFT and DCT algorithms with reduced
number of operations. Signal Processing, 6:267–278, 1984.
[6] R. Yavne. An economical method for calculating the discrete Fourier transform.
Proc. AFIPS Fall Joint Computer Conf., 33:115–125, 1968.

209