MMAT5390: Mathematical Image Processing

Chapter 3: Image Decomposition

One useful technique in image processing is to decompose an image into linear combination of
elementary images. Each elementary image has some specific properties. By carefully choosing
appropriate elementary images to decompose an image, a lot of imaging tasks can be done. For
example, if elementary images have different frequencies, we can easily smooth out or denoise an
image by removing the high-frequency components. Also, by truncating the less important compo-
nents in the image decomposition, we can compress an image to reduce the image size.
In this chapter, we will discuss some common image decomposition methods using some popular
linear image transformations. The applications of these decomposition methods for image compres-
sion will be demonstrated.

1 Basic idea of image decomposition

In image processing, it is often desirable to decompose an image f ∈ I into a linear combination
of elementary images. It turns out a separable linear image transformation can naturally give rise
to an image decomposition.
Suppose the point-spread function of a separable linear image transformation is given by: h(x, α, y, β) =
hc (x, α)hr (y, β). Let g be the transformed image. Then, g = hTc f hr and f = (hTc )−1 gh−1
r .

Write:
— ~v1T
 
  —
| | | — ~v2T —
(hTc )−1 ≡ ~u1 ~u2 ··· ~uN  and h−1 ≡ .
 
r ..
| | |  . 
T
— ~vN —
We can easily check that:
N X
X N
f= gij ~ui~vjT ,
i=1 j=1

which is a linear combination of images ~ui~vjT .

Definition 1.1. Each ~ui~vjT is called an elementary image for all i, j. Also ~ui~vjT is called the
outer product of ~ui and ~vj .

Intuitively, image decomposition aims to decompose an image into a linear combination of a basis,
which are called the elementary images. Each elementary image captures some important infor-
mation. Of course, the decomposition depends on the elementary images, which depend on the
choices of hc and hr . The following crucial question must be addressed.
Question: How do we choose hc and hr ?
Answer: We choose hc and hr such that:

1. Transformed image requires less storage (gij contains many zeros);

2. By truncating some terms gij ~ui~vjT (such as high-frequency terms), we can obtain better (such
as smooth) image;

1
 3. h−1
c and h−1
r can be easily computed. For example, hc and hr are unitary. Recall that a
matrix U is unitary if U U H = I where U H = conjugate transpose. That is,
(
H (U )T , if U complex
U = .
UT , if U real

Hence, the inverse of an unitary matrix can be easily obtained.

2 Singular Value Decomposition (SVD)

We will first describe a popular method in linear algebra, called the singular value decomposition,
for image decomposition. It can be effectively applied for image compression.
Definition 2.1. For any g ∈ Mm×n , the singular value decomposition (SVD) of g is a matrix
factorization given by
g = U ΣV T
with U ∈ Mm×m and V ∈ Mn×n both unitary, and Σ ∈ Rm×n is a diagonal matrix (Σij = 0 if
i 6= j) with diagonal elements σ1 ≥ σ2 ≥ · · · ≥ σr > 0 with r ≤ min(m, n). The diagonal elements
are called the singular values of g.
[Recall that A is unitary if A∗ A = AA∗ = I. (A∗ = (Ā)T )]
Theorem 2.2. The rank of g is given by the number of nonzero singular values.

Proof. Since both U and V are full rank, rank(g) = rank(Σ) which is the number of nonzero
singular values.
Theorem 2.3. We have the following relationships between the SVD of g and its fundamental
subspaces.

range(g) = span(u1 , u2 , · · · , ur ),
null(g) = span(vr+1 , vr+2 , · · · , vN ).

Proof. Exercise.
Theorem 2.4. Every m × n image g has a singular value decomposition.

Proof. We will consider the case when m ≤ n. The proof for m > n is similar. To prove the
theorem, let us first recall the following theorem in linear algebra.
Theorem. Let B ∈ Mn×n be a real symmetric matrix. Then, there exist n orthonormal eigenvec-
tors ~v1 , ~v2 , · · · , ~vn with corresponding eigenvalues λ1 , λ2 , · · · , λn such that

— ~v1T —
  
  λ1
| |  λ2  — ~v2T —
B = ~v1 · · · ~vn   .
  
..  ..
| |  .   . 
λn — ~vnT —

Now, note that gg T ∈ Mm×m and g T g ∈ Mn×n are symmetric. Thus, there exist n pairwise
orthonormal eigenvectors ~v1 , ~v2 , · · · , ~vn of g T g.

Let ~v1 , ~v2 , · · · , ~vr be eigenvectors of g T g with non-zero eigenvalues λ1 , λ2 , · · · , λr . Observe that
gg T g~vi = gλi~vi = λi g~vi . Therefore, g~vi is an eigenvector of gg T with eigenvalue λi .

√
Let σi = λi (why λ1 > 0?), then kg~vi k = σi because

kg~vi k2 ≡ (g~vi )T (g~vi ) = ~viT g T g~vi = λi~viT ~vi = λi .

2
 g~vi
Now, define ~ui = , then k~ui k = 1. Also, ~ui are orthonormal (for i = 1, 2, . . . , r) because
σi
(
T (g~vi )T (g~vj ) ~viT g T g~vj λj ~viT ~vj 1 if i = j
~ui ~uj = = = = .
σi σj σi σj σi σj 0 if i 6= j
(
T
λ j ~
v i ~
v j σi if i = j
Besides, ~uTi g~vj = = .
σi 0 if i 6= j
In matrix form,
— ~uT1 —
   
  σ1
— ~uT2 — | |  σ2 
 g ~v1 · · · ~vr  =  .
    
 .. ..
 .  | |  . 
T
— ~ur — σr
We extend {~u1 , · · · , ~ur } to an orthonormal basis {~u1 , · · · , ~ur , · · · , ~um }.
Similarly, {~v1 , · · · , ~vr } can be extended to {~v1 , · · · , ~vr , · · · , ~vn }.

Then, we get
 
σ1
— ~uT1
 
—
— ~uT2
 σ2 
—  

..
   .. 
  | | |  . 
 .   

— ~uTr
 g ~v1 ··· ~vr · · · ~vn = 
 σr .
—  
 | | | | 0
   
 .. }  
 .  {z  .. 
V  . 
— ~uTm —
| {z } 0
UT | {z }
Λ1/2

[Here, we use the fact that g~vj = 0 for j > r because kg~vj k2 = λj = 0 for j > r.]

Then, U T U = U U T = I, V T V = V V T = I and g = U Λ1/2 V T , where

 
λ1

 λ2 

 .. 

 . 

Λ=  λr .


 0 

 .. 
 . 
0

What are inside U and V (g = U Λ1/2 V T )?

T
Note that gg T = U Λ1/2 V 1/2 T T
| {zV} Λ U = U ΛU .
I
∴ U consists of eigenvectors of gg T .
Similarly, g T g = V ΛV T .
∴ V consists of eigenvectors of g T g.

Remark.

1. Note that gg T ~u = λ~u (where λ = eigenvalue, and ~u = eigenvector), then:

~uT gg T ~u (g T ~u)T (g T ~u)

λ= T
= ≥ 0;
~u ~u ~uT ~u

3
 r
1/2
2. g = U Λ1/2 V T = λi ~ui~viT .
P
i=1
~ui~viT are called the eigenimages of g under SVD.
(For a visualization of elementary images under SVD, please refer to the Lecture
2 Powerpoint)
For an N × N image, the required storage after SVD is: (2N + 1) × r. Hence, it saves storage
when r is small.
Definition 2.5. For any k with 0 ≤ k ≤ r, we define
k
X
gk = σj ~uj ~vjT
j=1

where gk is called a rank-k approximation of g.

This low rank matrix approximation can be applied to image compression. For any M × N image
g ∈ RM ×N , one is required to allocate MN intensity levels in total. The rank-ν approximation
Aν , however, needs only store ν singular values and singular vectors ~uj and ~vj , which leads to
ν(1 + M + N ) numbers. In the case when M = N for simplicity, there is a reduction of storage if

N2
ν< .
2N + 1
We can remove the i-th term of which λi is small to further reduce the storage.
Error of the approximation by SVD
Definition 2.6. The Frobenius norm (F-norm) given by
v
um X n
uX
kAkF = t a2ij ,
i=1 j=1

where aij is the i-th row, j-th column entry of A.

Let ai be the j-th column of A. We have
v
u n
uX
p p
kAkF = t kaj k22 = tr(A∗ A) = tr(AA∗ ),
j=1

where tr(·) is the trace of the matrix in the argument.

Theorem 2.7. The F-norm of a matrix is invariant under multiplication by unitary matrices, i.e.
for any A ∈ Rm×n , unitary U ∈ Rm×m , we have kU AkF = kAkF .

Proof. The proof is very simple, which can be explained in one line:
q
q q
kU AkF = tr (U A)T (U A) = tr(AT U T U A) = tr(AT A) = kAkF .

r
σj ~uj ~vjT be the SVD of an M × N image f . For any k with k < r and
P
Theorem 2.8. Let f =
j=1
k
σj ~uj ~vjT , we have
P
fk =
j=1
r
X
kf − fk k2F = σi2 .
i=k+1

r
1/2
λi ~ui~viT .
P
Proof. Let f =
i=1
k
1/2
λi ~ui~viT .
P
Approximate f by fk with k < r where fk =
i=1

4
 r
1/2
λi ~ui~viT ∈ MM ×N .
P
Define the error of the approximation by D ≡ f − fk =
i=k+1
The m-th row, n-th column entry of D is
r
1/2
X
Dmn = λi uim vin
i=k+1
   
ui1 vi1
 ..   .. 
where ~ui =  . , ~vi =  .  . Then,
uiM viN

r
!2 r r r
1/2 1/2 1/2
X X X X
2
Dmn = λi uim vin = λi u2im vin
2
+2 λi λj uim vin ujm vjn .
i=k+1 i=k+1 i=k+1 j=k+1
j6=i

Thus,
XX
kDk2F = 2
Dmn
m n
 
r r r
1/2 1/2
XX X X X
λi u2im vin
2

= 
 +2 λi λj uim vin ujm vjn 

m n i=k+1 i=k+1 j=k+1
j6=i
1 1 * 0X *0
r r
1/2 1/2 
X X > X  X X
2 2>  
= λi uim v + 2 λ λ u u v v
 

 in i j im jm in jn
 
i=k+1 m
 n j=k+1 m
 m

j6=i
r
X r
X
= λi = σi2 .
i=k+1 i=k+1

Therefore, Sum of square error of the approximation = Sum of omitted eigenvalues.

Remark.

• To approximate an image using SVD, arrange the eigenvalues λi in decreasing order, and
r
σi ~ui~viT .
P
remove the last few terms in
i=1

• This is the optimal approximation using k terms in the Frobenius norm.

Example 2.9. Let  
1 2
A = 2 2 .
2 1
We have  
T 9 8
A A= .
8 9
√
Now, eig(A∗ A) are 17 and 1, and so σ1 = 17, σ2 = 1 and
√ 
17 0
Σ= .
0 1

Moreover,    
1 1 1 1
~v1 = √ , ~v2 = √ .
2 1 2 −1
This gives !
√1 √1
V = 2 2 .
√1 −1
√
2 2

5
Since
σ1 ~u1 = A~v1 ,
we have    
1 2   3
1 1  1 1
~u1 = √ √ 2 2 = √ 4 .
17 2 2 1 34 3
1
Similarly, we have    
1 2   −1
1  1 1
~u2 = √ 2 2 = √  0 .
2 2 −1 2
1 1
The matrix U is, therefore, given by
√3 −1
 
√ |
 √434 0
2
~u3 
U=  34 
√3 √1 |
34 2

for some vector u3 orthonormal to both u1 and u2 . One possibility is

 
2
1  
u3 = √ −3 .
17 2

Finally, the SVD of A is given by

   √3 −1
√ √2
√
 
1 2 17 0
!
34 2 17 √1 √1
4 −3 2 2
2 2 =   √34 0 √   0 1 −1
17 √1 √

2 1 √3 √1 √2 0 0 2 2
34 2 17

Application of SVD for image compression

SVD can be applied for image compression, by removing terms associated to small singular values.
We will show some examples of image compression using SVD. Please refer to Lecture 2 Powerpoint
for more illustration.

In the above figure, the left shows the rank-k approximation of a baboon image with k = 10, 20
and 50 respectively. The original rank of the image is 441. Note that the rank-10 approximation

6
of the image can give a very good approximation of the original image. The right shows the image
compression of a fingerprint image, using a rank-40 approximation. SVD is one of the popular
methods for image compression.
Next, we test the image compression using SVD on the ‘Einstein’ image:

The original rank of the ‘Einstein’ image is 202. Using SVD, we compute some rank-k approxima-
tions of the image, which are given below:

k=2 k = 12 k = 22 k = 52

The above figure shows the rank-k approximations of the ‘Einstein’ image with k = 2, 12, 22 and
52 respectively.

3 Haar and Walsh transforms

Definition 3.1 (Haar functions). The Haar functions are defined as follows:
(
1 if 0 ≤ t < 1
H0 (t) ≡ ,
0 elsewhere

1
 if 0 ≤ t < 1/2
H1 (t) ≡ −1 if 1/2 ≤ t < 1 ,

0 elsewhere

√ p
 √
 2 if 2np ≤ t < n+0.5
2p
p
H2p +n (t) ≡ − 2 if n+0.5 2 p ≤ t < n+1
2p

0 elsewhere


where p = 1, 2, · · · ; n = 0, 1, 2, · · · , 2p − 1.
(H2p +n (t) is compactly supported by a smaller region if p is bigger)

7
 H0 H1

Definition 3.2 (Walsh functions). The Walsh functions are defined recursively as follows:
j
W2j+q (t) ≡ (−1)b 2 c+q {Wj (2t) + (−1)j+q Wj (2t − 1)},

where b 2j c = largest integer smaller or equal to 2j ; q = 0 or 1; j = 0, 1, 2, · · · and

(
1 if 0 ≤ t < 1
W0 (t) ≡ .
0 elsewhere

Example 3.3. Compute W1 (t).

Put j = 0, q = 1. Then:

W1 (t) ≡ (−1)b0c+1 {W0 (2t) + (−1)0+1 W0 (2t − 1)}

= −W0 (2t) + W0 (2t − 1).

Check that for 0 ≤ t < 12 , W0 (2t) = 1, W0 (2t − 1) = 0 =⇒ W1 (t) = −1.

For 21 ≤ t < 1, W0 (2t) = 0, W0 (2t − 1) = 1 =⇒ W1 (t) = 1.

Definition 3.4 (Discrete Haar Transform).

The Haar Transform of an N × N image is performed as follows. Divide [0,1] into N partitions.
That is,

0 1 2 i N −1 1
N N N N

8
 
i
Let H(k, i) ≡ Hk where k, i = 0, 1, 2, · · · , N − 1.
N
We obtain the Haar Transform matrix:
1
H̃ ≡ √ H where H ≡ (H(k, i))0≤k,i≤N −1 .
N
The Haar Transform of f ∈ Mn×n is defined as:
g = H̃f H̃ T .
Definition 3.5 (Discrete Walsh Transform). The Walsh Transform of an N × N image is defined
similarly as Haar Transform.
 
i
Define W (k, i) ≡ Wk where k, i = 0, 1, 2, · · · , N − 1.
N
Then, the Walsh Transform matrix is:
1
W̃ ≡ √ H where H ≡ (W (k, i))0≤k,i≤N −1 .
N
The Walsh Transform of f ∈ Mn×n is defined as:
g = W̃ f W̃ T .
Example 3.6. Compute the Haar Transform matrix for a 4 × 4 image.
Solution. Divide [0,1] into 4 portions:

i=0 i=1 i=2 i=3

0 0.25 0.5 0.75 1

Check that:

H0 H1

H2 H3

9
We get that  
1 1 1 1
 1
√ 1
√ −1 −1 
 and H̃ = √1 H = 1 H.
H=
 2 − 2 √0 √0  4 2
0 0 2 − 2
Easy to check that H̃ T H̃ = I.
Example 3.7. Compute the Haar Transform of
 
0 1 1 0
1 0 0 1
f =  .
1 0 0 1
0 1 1 0

Solution.  
2 0 0 0
T
0 0 0 0
g = H̃f H̃ = 
 .
0 0 −1 1
0 0 1 −1
Example 3.8. Suppose g in Example 3.7 is changed to:
 
2 0 0 0
0 0 0 0
g= 0 0 −1 1 .


0 0 1 0

Reconstruct the original image.

Solution.  
0 1 1 0
1 0 0 1
f = H̃ T f H̃ = 

.
1 0 0.5 0.5
0 1 0 0
Remark.

• Haar Transform usually produces coefficient matrix with more zeroes (compression);
• Errors in the coefficient matrix cause localized errors in the reconstructed image (Assign
detail of accuracy in compression).
Example 3.9. Compute the Walsh Transform matrix for a 4 × 4 image.
Solution. Again, divide [0,1] into 4 portions:

i=0 i=1 i=2 i=3

0 0.25 0.5 0.75 1

Check that:

10
 W0 W1

W2 W3

So,  
1 1 1 1
 and W̃ = √1 W = 1 W.
−1 −1 1 1
W =
−1 1 1 −1 4 2
1 −1 1 −1
Example 3.10. Compute the Walsh Transform of
 
0 1 1 0
1 0 0 1
f =1 0 0
.
1
0 1 1 0
 
2 0 0 0
 0 0 0 0
Solution. g = W̃ f W̃ T = 
0 0 −2 0.


0 0 0 0
(Many zeroes =⇒ compression)
Remark.

• The idea of Haar / Walsh Transform is to transform an image to a “transformed image”

with many more zeroes.
• The coefficient in the “transformed” image tells us information of frequency of image intensity
changes.

Another way to define Walsh function

Definition 3.11 (Rademacher functions).

11
A Rademacher function of order n (n 6= 0) is defined as:
Rn (t) ≡ sign[sin(2n πt)] for 0 ≤ t ≤ 1
(where sign(x) = 1 if x > 0, sign(x) = −1 if x < 0 and sign(x) = 0 if x = 0). For n = 0, R0 (t) ≡ 1
for 0 ≤ t ≤ 1.
Let N = bm+1 2m + bm 2m−1 + · · · + b1 20 . Then, the R-Walsh function W̃N is given by:
m+1
Y
W̃N = Ri (t)
i=1
bi 6=0

(where the values at the jumps are defined such that the function is continuous from the right).
Example 3.12. Compute R-Walsh functions W̃3 and W̃4 using Rademacher functions.
Consider sin(8πt): Therefore, R3 (t) =

1 ·22 + |{z}
As 4 = |{z} 0 ·21 + |{z}
0 ·20 , we have
b3 b2 b1

3
Y
W̃4 = Ri (t) = R3 (t)
i=1
bi 6=0

(W2j+q (t) ≡ (−1)bj/2c+q {Wj (2t) + (−1)j+q Wj (2t − 1)}).

1 ·21 + |{z}
For W3 (t): As 3 = |{z} 1 ·20 , we have
b2 b1

2
Y
W̃3 (t) = Ri (t) = R1 (t)R2 (t).
i=1
bi 6=0

R1 (t):

12
R2 (t) :

Therefore, W̃3 (t):

Relationship between Walsh functions and R-Walsh functions

W0 (t) = W̃0 (t), W1 (t) = −W̃1 (t), W2 (t) = −W̃3 (t), W3 (t) = W̃2 (t),
W4 (t) = W̃6 (t), W5 (t) = −W̃7 (t), W6 (t) = −W̃5 (t), W7 (t) = W̃4 (t).
How to determine R-Walsh W̃i (t) associated to Walsh Wj (t)
Idea: Write j as
j = bm+1 2m + bm 2m−1 + · · · + b1 20 .
The binary representation of j is:
bm+1 bm · · · b1 .
Then, i is given by:
cm+1 cm · · · c1
where
cm+1 ≡ bm+1 (mod 2), ck ≡ (bk+1 + bk ) (mod 2).
The sign is determined from Wj (0).

Example 3.13. Consider W7 (t).

Check that W7 (t) > 0.
Now, 7 = 22 + 21 + 20 so the binary representation of 7 is 111.
Therefore, j = 100 (binary) = 4.
Thus, W7 (t) = W̃4 (t).

13
What do the elementary images under the Haar and Walsh Transform look like?
(Please refer to the Lecture 3 and 4 Powerpoint for clearer figures)
Using Haar transform, the 8 × 8 image f can be written as:
X
f = H̃ T g H̃ = H
gij Iij
1≤i,j≤N

H
where Iij is the elementary image given by taking the outer product of the i-th row and the j-th
row of H̃.
The elementary images under the Haar transform (of an 8 × 8 image) look like the following:

H
Here, the i-th row j-th column image represents the elementary image Iij .
[White = positive; Black = negative; Grey = 0]
Note that some elementary images are locally supported (i.e. non-zero at some locally small region).
Therefore, they capture both spatial and frequency information.
Similarly, under the Walsh transform, the 8 × 8 image f can be written as:
X
f = W̃ T g W̃ = W
gij Iij
1≤i,j≤N

W
where Iij is the elementary image given by taking the outer product of the i-th row and j-th row
of W .
The elementary images under the Walsh transform (of an 8 × 8 image) look like the following:

14
 W
Here, the i-th row j-th column image represents the elementary image Iij .
[White = positive; Black = negative; Grey = 0]

Properties of each elementary image under the Haar Transform

Each elementary image captures different levels of details with different levels of resolution in the
horizontal and vertical directions. Please refer to the Lecture 3 and 4 Powerpoint for a clearer
figure.

L-L L-H1 L-H2 L-H3

1
L

H1-H2 H1-H3
1-

1-
H

H
H2-H1
H2-L

H2-H2 H2-H3
H3-H1
H3-L

H3-H2 H3-H3

15
In the above figure, the thick lines divide them into sets of elementary images of the same reso-
lution. Letter L and H are used to indicate low and high resolution, respectively. The numbers
next to letter H indicates which level of high resolution. The pairs of letters used indicate which
resolution we have along the vertical and horizontal axes. For example, pair L-H2 indicates that
the corresponding panels have low resolution along the vertical axis, but second order high resolu-
tion along the horizontal axis.

Application of the Haar and Walsh transforms for image compression

The Haar and Walsh transforms can be used for image compression, by truncating high-frequency
terms. Below we will illustrate the idea on a simple image:

The approximations of the image using the Haar transform are shown below. More precisely, the
following figure shows reconstructed images when the basis images used are those created from the
first one, two, three, . . . , eight Haar functions, from top left to bottom right, respectively.

The approximations of the image using the Walsh transform are shown below. Similarly, the
following figure shows reconstructed images when the basis images used are those created from the
first one, two, three, . . . , eight Walsh functions, from top left to bottom right, respectively.

16
4 Discrete Fourier Transform
Definition 4.1. The 1D discrete Fourier transform (DFT) of a function f (k), defined at
discrete points k = 0, 1, · · · , N − 1, is defined as
N −1  √ 
1 X −2πj mk j = −1
F (m) = f (k)e N .
N ejθ = cos θ + j sin θ
k=0

The 2D discrete Fourier transform (DFT) of an M × N image g = (g(k, l))k,l , where k =

0, 1, · · · , M − 1 and l = 0, 1, · · · , N − 1, is defined as:
M −1 N −1
1 X X km ln
ĝ(m, n) = g(k, l)e−2πj( M + N ) . (*)
MN
k=0 l=0

What is the inverse of DFT?

pm qn
Multiply both sides of (*) by e2πj( M + N ) (with p chosen from 0, 1, · · · , M − 1 and q chosen from
0, 1, · · · , N − 1) and sum m over 0 to M − 1, and n over 0 to N − 1:
M −1 N −1
pm
+ qn
X X
ĝ(m, n)e2πj( M N )

m=0 n=0
M −1 N −1 M −1 N −1
1 X X X X m(p−k) n(q−l)
= g(k, l)e2πj( M + N )
MN m=0 n=0
k=0 l=0
M −1 N −1 M −1 N −1
1 X X X m(p−k) X n(q−l)
= g(k, l) e2πj M e2πj N . (**)
MN m=0 n=0
k=0 l=0

We can show that for s ∈ Z \ {0} and t ∈ Z,

s−1
(
X
2πj tm s if t ∈ sZ
e s = s1sZ (t) = .
m=0
0 otherwise

Therefore, R.H.S. of (**) becomes:

M −1 N −1
1 X X
g(k, l) · M 1M Z (p − k) · N 1N Z (q − l)
MN
k=0 l=0
M
X −1 N
X −1
= g(k, l)δ(p − k)δ(q − l) = g(p, q).
k=0 l=0
(since k, p ∈ [0, M − 1] =⇒ p − k ∈ [1 − M, M − 1], whose only intersection with M Z is {0};
and l, q ∈ ∩[0, N − 1] =⇒ q − l ∈ [1 − N, N − 1], whose only intersection with N Z is {0})

17
Inverse discrete Fourier transform:
M −1 N −1
pm
+ qn
X X
g(p, q) = ĝ(m, n)e2πj( M N )

m=0 n=0

How is DFT in matrix form?

Recall:
  — ~v T — 
| | | 1
T
g = ~u1 ~u2 ··· ~un  f — ~v2 — T
 = U f V if and only if
 
| | | ..
.— ~vnT —
M X
X N
g= fij ~ui~vjT
i=1 j=1

We need to express DFT in terms of matrix multiplication.

For an N × N image g, the DFT of g is given by:
N −1 N −1
1 X X km+ln
ĝ(m, n) = 2
g(k, l)e−2πj N .
N
k=0 l=0

1 xα
Let Uxα = e−2πj N where 0 ≤ x, α ≤ N − 1, and U = (Uxα )0≤x,α≤N −1 ∈ MN ×N (C). Then, U
N
is symmetric and
ĝ = U gU.
Question: Can you write the DFT of an M × N image in matrix form?
Example 4.2. Find the DFT of the following 4 × 4 image
 
0 0 1 0
0 0 1 0
g= 0
.
0 1 0
0 0 1 0

Solution. The matrix U is given by:

 
1 1 1 1
1 1 −j −1 j 
U=  .
4 1 −1 1 −1
1 j −1 −j
1
− 41 1
− 14

4 4
0 0 0 0 
∴ DFT of g = ĝ = U gU = 
0
.
0 0 0 
0 0 0 0

Property of U :
Note that given N ∈ N \ {0} and x1 , x2 ∈ Z ∩ [0, N − 1],
N −1 N −1
1 X −2πj x1 α 2πj x2 α 1 X 2πj (x2 −x1 )α
e N e N = e N
N 2 α=0 N 2 α=0
1
= N δ(x2 − x1 ).
N2
Therefore, rows of U are mutually orthogonal but not orthonormal.
1
We can conclude that U U ∗ = I (so U is NOT unitary).
√ N
If we define Ũ ≡ N U , then Ũ is unitary.

18
Remark.

• Sometimes, people like to define DFT as

1 NP −1 mk
(1D) fˆ(m) = √ f (k)e−2πj N
N k=0
1 NP−1 NP−1 mk+nl
(2D) fˆ(m, n) = f (k, l)e−2πj N so that unitary matrix Ũ may be used.
N k=0 l=0

• If Ũ is used, we must be careful about the formula we derive (as they usually differ by a
scaling factor) (e.g. the inverse DFT).

Elementary images under DFT decomposition

Given an image g, we can compute its DFT ĝ. Then:

g = U g̃U.

Hence, XX
g = (N U )∗ ĝ(N U )∗ = ĝkl w ~ lT
~ kw
k l
∗
where w ~ k is the k-th column of (N U ) . Then
w ~ jT
~ iw are the elementary images of DFT decompo-
sition.
For a visualization of DFT elementary images, please refer to Lecture 4 Powerpoint.
The real part of elementary images under the DFT (of an 8 × 8 image) look like the following:

The imaginary part of elementary images under the DFT look like the following:

19
Further properties of DFT

1. DFT of convolution
Let g and w be two N ×M images. Assume that g and w are periodically extended. The convolution
of them is
N
X −1 M
X −1
v(n, m) = g(n − n0 , m − m0 )w(n0 , m0 ).
n0 =0 m0 =0

We compute the DFT of v:

N −1 M −1
1 X X pn qm
v(n, m)e−2πj( N + M )
N M n=0 m=0
N −1 M −1 N −1 M −1
1 X X X X pn qm
= g(n − n0 , m − m0 )w(n0 , m0 )e−2πj( N + M )
NM 0 0 n=0 m=0
| {z } | {z }
n =0 m =0 00 00n m
0 0
N −1 M −1 N −1−n
X M −1−m
1 X X pn0
+ qm
0 X pn00 00
+ qm
= w(n0 , m0 )e−2πj( N M ) g(n00 , m00 )e−2πj( N M ) .
NM
n0 =0 m0 =0 n00 =n0 m00 =−m0
| {z }| {z }
ŵ(p,q) T

Note that g and w are periodically extended. Hence, we have:

g(n − N, m) = g(n, m) and g(n, m − M ) = g(n, m).

Then:
0
M −1−m −1
X qm00 X pn00
T ≡ e−2πj M g(n00 , m00 )e−2πj N

m00 =−m0 n00 =−n0

0 0
M −1−m 00
N −1−n
pn00
−2πj qm
X X
+ e M g(n00 , m00 )e−2πj N .
m00 =−m0 n00 =0

20
Consider
−1
X pn00
g(n00 , m00 )e−2πj N

n00 =−n0
N −1
n000 =N +n00
X pn000
= g(n000 − N, m00 ) e−2πj N e|2πjp
{z } (∵ p = integer).
n000 =N −n 0
| {z }
g(n000 ,m00 ) 1

(Similarly, we can do the same thing for the index m00 .)

Therefore,
M −1 N −1
pn00 00
+ qm
X X
T = g(n00 , m00 )e−2πj( N M ) = M N ĝ(p, q).
m00 =0 n00 =0

v̂(p, q) = M N ĝ(p, q)ŵ(p, q)

Remark. Conversely, if x(n, m) = g(n, m) · w(n, m),

−1 M
NP P−1
then x̂(k, l) = ĝ(p, q)ŵ(k − p, l − q) (convolution of ĝ and ŵ).
p=0 q=0

2. Average value of image v.s. DFT

Average value of image g:
N −1 N −1 N −1 N −1
1 X X 1 X X
ḡ = g(k, l) = g(k, l) |e−2πj(0)
{z } = ĝ(0, 0).
N2 N2
k=0 l=0 k=0 l=0 1

3. DFT of a rotated image

Let g be an N × N image.
Consider:
N −1 N −1
1 X X km+ln
ĝ(m, n) = g(k, l)e−2πj N .
N2
k=0 l=0

Let k ≡ r cos θ, l ≡ r sin θ (Polar coordinates of (k, l)).

Similarly, let m ≡ w cos φ, n ≡ w sin φ (Polar coordinates of (m, n)).
Note that km + ln = rw(cos θ cos φ + sin θ sin φ) = rw cos(θ − φ).
Denote P(g) = {(r, θ) : (r cos θ, r sin θ) is a pixel of the image g}, which is called the polar
coordinate set of g.
Then:
N −1 N −1
1 X X rw cos(θ−φ)
ĝ(w, φ) = 2 g(r, θ)e−2πj N .
N
k=0 l=0

Here, we identify g(r cos θ, r sin θ) with g(r, θ) and ĝ(w cos φ, w sin φ) with ĝ(w, φ).
Consider the rotated image g̃(r, θ) = g(r, θ − θ0 ). Here, θ is assumed to be defined between θ0 to
π/2 + θ0 . Hence, one rotates image g counterclockwise by θ0 to get g̃.
DFT of g̃ can be computed by:
1 X rw cos(θ−φ)
ˆ
g̃(w, φ) = 2 g(r, θ − θ0 )e−2πj N
N | {z }
(r,θ)∈P(g̃)
θ̃
1 X rw cos(θ̃+θ0 −φ)
= 2 g(r, θ̃)e−2πj N .
N
(r,θ̃)∈P(g)

Therefore,
ˆ
g̃(w, φ) = ĝ(w, φ − θ0 )
Again, φ is assumed to be defined between θ0 to π/2 + θ0 .

21
Remark. DFT of an image rotated by θ0 = DFT of the original image rotated by θ0 .
1
− 14 14 − 41
  
0 0 1 0 4
0 0 1 0 , then ĝ =  0 0 0 0 

Example 4.3. Let g = 
0
.
0 1 0 0 0 0 0 
0 0 1 0 0 0 0 0
 
0 0 0 0
0 0 0 0
Rotate g by 90◦ clockwise: g̃ =  1 1 1 1.


0 0 0 0
Note that the indices
 of g̃ are  as follows (after the rotation): −3 ≤ l ≤ 0 and 0 ≤ k ≤ 3. Then,
taken
1
0 0 0 4
0 0 0 − 1 
ˆ
DFT of g̃ = g̃ = 
 4
1  (with indices taken as follows: −3 ≤ l ≤ 0 and 0 ≤ k ≤ 3).
0 0 0 4
0 0 0 − 41
3 0 km+ln
g̃(k, l)e−2πj 4 .)
P P
(Note that the DFT of g̃ is defined as:
k=0 l=−3

4. DFT of a shifted image

Let g(k 0 , l0 ) be an N × N image.
Assume the indices of the image are taken as −k0 ≤ k 0 ≤ N − 1 − k0 and −l0 ≤ l0 ≤ N − 1 − l0 .
Let g̃ be the shifted image whose indices are taken as 0 ≤ k, l ≤ N − 1, then:
g̃(k, l) = g(k − k0 , l − l0 )
N −1 N −1
1 X X km+ln
ˆ
g̃(m, n) = 2 g(k − k0 , l − l0 )e−2πj N
N
k=0 l=0
X 0 N −1−l
N −1−k X0
1 k0 m+l0 n k0 m+l0 n
= g(k 0 , l0 )e−2πj N e−2πj N .
N2
k0 =−k0 l0 =−l0
| {z }
ĝ(m,n)

Therefore,
k0 m+l0 n
ˆ
g̃(m, n) = ĝ(m, n)e−2πj N

k0 m+l0 n
DFT of the shifted image = DFT of the original image × e−2πj N .
 m0 k+n0 l

Remark. We can show that ĝ(m − m0 , n − n0 ) = DFT image × e2πj N (with carefully
chosen indices).

4.1 Fast Fourier Transform (FFT)

Goal: To compute DFT efficiently.

Fast Fourier Transform (FFT)
DFT is separable =⇒ 2D DFT = Two 1D DFT.
We will discuss how to compute 1D DFT fast.
−1
1 NP 2π
The 1D DFT is: fˆ(u) = ux
f (x)ωN , ωN ≡ e− N j .
N x=0
Assume N = 2n = 2M , then:
2M −1
1 X
fˆ(u) = ux
f (x)ω2M .
2M x=0
Separate it into odd and even parts:
 
 M −1 M −1 
ˆ 1  1 X u(2y) 1 X u(2y+1)

f (u) = f (2y) ω2M + f (2y + 1) ω2M .
2 M y=0 | {z } M y=0 | {z } 
uy uy u
ωM ωM ω2M

22
Then:
M −1 M −1
( )
1 1 X uy 1 X uy u
fˆ(u) = f (2y)ωM + f (2y + 1)ωM ω2M .
2 M y=0 M y=0

For u < M ,
1 nˆ o
fˆ(u) = feven (u) + fˆodd (u)ω2m
u
where
2
M −1
1 X uy
fˆeven (u) ≡ f (2y)ωM (DFT of even part of f )
M y=0
M −1
1 X uy
and fˆodd (u) ≡ f (2y + 1)ωM (DFT of odd part of f ).
M y=0

For u ≥ M , we consider:
 
 M −1 M −1 
ˆ 1  1 X uy+M y 1 X uy+M y u+M

f (u + M ) = f (2y)ωM + f (2y + 1) ωM ω2M
2 M y=0 M y=0 | {z } | {z }

uy u
−ω2M
ωM
1 nˆ o
and fˆ(u + M ) = feven (u) − fˆodd (u)ω2M
u
.
2
The FFT algorithm can now be described as follows.
Fast Fourier Transform (FFT) Algorithm
Let f ∈ RN where N = 2n = 2M .
Step 1: Split f into:

feven = [f (0), f (2), . . . , f (2M − 2)]T

and fodd = [f (1), f (3), . . . , f (2M − 1)]T .

Step 2: Compute fˆeven = FM feven and fˆodd = FM fodd , where FM = (ωM ux

)0≤u,x≤M −1 is an M ×M
matrix.
(Recall that the computation of fˆeven is equivalent to the left multiplication of feven by an M × M
matrix. Similarly for fˆodd ).
Step 3: Compute fˆ using the following formula:
For u = 0, 1, 2, . . . , M − 1,
1
fˆ(u) = [fˆeven (u) + fˆodd (u)ω2M
u
],
2
1
and fˆ(u + M ) = [fˆeven (u) − fˆodd (u)ω2M
u
].
2

Remark. For Step 2, we can apply the splitting idea to compute fˆeven and fˆodd .

Computational cost of FFT

Let Cm be the computational cost of Fm x. Then, C1 = 1.
Obviously,
CN = 2CM + 3M
(2 matrix multiplication, M multiplication, addition, and subtraction)
Hence,

C2n = 2C2n−1 + 3M implies:

−n
2 C2n = 2−(n−1) C2n−1 + 3/2.

23
The above recursive equation gives 2−n C2n = C1 + n(3/2). Thus, C2n = 2n + n2n (3/2). We
conclude that the computational cost CN is bounded by KN log2 N . We denote it by O(N log2 N ).
Application of DFT for image compression
DFT is often used for image compression. It is often done by truncating terms associated to small
Fourier coefficients. Below we will show some examples of image compression using DFT.
The following shows the image compression of the lena image, by keeping the largest 20% of the
Fourier coefficients.

The following shows the image compression of an image of a flower, by keeping the largest 5% of
the Fourier coefficients.

DFT is a very popular method for image compression. In fact, DFT is a very important tool
for many other image processing tasks, such as image denoising, image deblurring and image
sharpening. These will be discussed in the next chapter.

5 Discrete Cosine Transform

5.1 Even symmetric discrete cosine transform

Goal: Write an image as a linear combination of cosine functions only. Hence, the elementary
images are real-valued.
Consider an N × N image f . Extend f to a 2M × 2N image f˜, whose indices are taken from
[−M, M − 1] and [−N, N − 1].

24
Define f (k, l) for −M ≤ k ≤ M − 1 and −N ≤ l ≤ N − 1 such that
1 1
f (−k − 1, −l − 1) = f (k, l)} Reflection about (− , − )
) 2 2
f (−k − 1, l) = f (k, l) 1 1
Reflection about the axes k = − and l = −
f (k, l − 1) = f (k, l) 2 2

Next, we make the extension as a reflection about (0,0), the axis k = 0 and the axis l = 0. This
can be done by shifting the image by ( 12 , 12 ).
 
1 2 3
Here is a simple example illustrating the idea. Let f = 4 5 6. The extended image becomes:
7 8 9
1
2 −3→ 9 8 7 7 8 9
1
2 −2→ 6 5 4 4 5 6
1
2 −1→ 3 2 1 1 2 3
1
2 +0→ 3 2 1 1 2 3
1
2 +1→ 6 5 4 4 5 6
1
2 +2→ 9 8 7 7 8 9
↑ ↑ ↑ ↑ ↑ ↑
1 1 1 1 1 1
2 2 2 2 2 2
− − − + + +
3 2 1 0 1 2

Now, we compute the DFT of (shifted) f˜:

M −1 N −1
1 X X m 1 n 1
F (m, n) = f (k, l)e−2πj 2M (k+ 2 ) e−2πj 2N (l+ 2 )
(2M )(2N )
k=−M l=−N
M −1 N −1
1 X X m 1 n 1
= f (k, l)e−πj M (k+ 2 )−πj N (l+ 2 )
4M N
k=−M l=−N
−1 −1 −1 N −1 M −1 X−1 M −1 N −1
1 X X X X X X X
= ( + + + )
4M N
k=−M l=−N k=−M l=0 k=0 l=−N k=0 l=0
| {z } | {z } | {z } | {z }
A1 A2 A3 A4
m
−πj M (k+ 21 )−πj N
n
(l+ 12 )
f (k, l)e .

For A1 , change of variable: k̃ ≡ −k − 1, ˜l ≡ −l − 1, and then apply trigonometric identity, and we

obtain
M −1 N −1       
X X
˜ mπ 1 nπ ˜ 1
A1 = f (k̃, l) cos k̃ + cos l+
M 2 N 2
k̃=0 l̃=0
     
mπ 1 nπ ˜ 1
− sin k̃ + sin l+
M 2 N 2
     
mπ 1 nπ ˜ 1
+ j sin k̃ + cos l+
M 2 N 2
      
mπ 1 nπ ˜ 1
+ j cos k̃ + sin l+ .
M 2 N 2

Similarly, we can perform the same computation to A2 , A3 , A4 . All together, we can obtain
M −1 N −1      
1 X X mπ 1 nπ 1
A1 + A2 + A3 + A4 = f (k, l) cos k+ cos l+ .
MN M 2 N 2
k=0 l=0

Definition 5.1 (Even symmetric discrete cosine transform).

25
Let f be a M × N image, whose indices are taken as 0 ≤ k ≤ M − 1 and 0 ≤ l ≤ N − 1. The even
symmetric discrete cosine transform (EDCT) of f is given by:
M −1 N −1      
1 X X mπ 1 nπ 1
fˆec (m, n) = f (k, l) cos k+ cos l+
MN M 2 N 2
k=0 l=0

with 0 ≤ m ≤ M − 1, 0 ≤ n ≤ N − 1.
Remark.

• The inverse of EDCT can be explicitly computed. More specifically, the inverse EDCT is
defined as:
M −1 N −1
X X πm(2k + 1) πn(2l + 1)
f (k, l) = C(m)C(n)fˆec (m, n) cos cos (***)
m=0 n=0
2M 2N

• Formula (***) can be expressed as matrix multiplication:

M
X −1 N
X −1
f= fˆec (m, n)T~m T~n0T
m=0 n=0

Tn0 (0)
   
Tm (0)
 Tm (1)   Tn0 (1)
 πm(2k + 1)
where: T~m = 
 ~0
, Tn =   with Tm (k) = C(m) cos and
  
.. ..
 .    . 2M
Tm (M − 1) Tn0 (N − 1)
πn(2k + 1)
Tn0 (k) = C(n) cos .
2N
• Tm Tn0T = elementary images of EDCT.

Image decomposition under EDCT

(Please refer to the Lecture 7 Powerpoint for clearer figures)
The elementary images under the EDCT (of an 8 × 8 image) look like the following:

26
Here, the m-th row n-th column image represents the elementary image Tm Tn0T .

27
Application of EDCT for image compression
EDCT can be applied for image compression by truncating terms associated to small coefficients.
In fact, the famous JPEG compression applies EDCT.
Below, we will show some results of image compression using EDCT.
In the above figure, (a) shows the original Saturn image. (b) shows the compressed image by
keeping the largest 75% of EDCT coefficients. (c) shows the compressed image by keeping the
largest 50% of EDCT coefficients. (d) shows the compressed image by keeping the largest 25% of
EDCT coefficients.
In the above figure, (a) shows the original ‘kid’ image. (b) shows the compressed image by keeping
the largest 75% of EDCT coefficients. (c) shows the compressed image by keeping the largest
50% of EDCT coefficients. (d) shows the compressed image by keeping the largest 25% of EDCT
coefficients.
In the above figure, (a) shows the original ‘baboon’ image. (b) shows the compressed image by
keeping the largest 75% of EDCT coefficients. (c) shows the compressed image by keeping the
largest 50% of EDCT coefficients. (d) shows the compressed image by keeping the largest 25% of
EDCT coefficients.

5.2 Odd symmetric discrete cosine transform

Other than even symmetric discrete cosine transform, the odd symmetric discrete cosine
transform can also be defined. We will skip the details but just state the definition.
Definition 5.2 (Odd symmetric discrete cosine transform).

Let f be a M × N image, whose indices are taken as 0 ≤ k ≤ M − 1 and 0 ≤ l ≤ N − 1. The odd

symmetric discrete cosine transform (ODCT) of f is given by:
M −1 N −1
1 X X 2πmk 2πnl
fˆoc (m, n) = C(k)C(l)f (k, l) cos cos
(2M − 1)(2N − 1) 2M − 1 2N − 1
k=0 l=0

where C(0) = 1 and C(k) = C(l) = 2 for k, l 6= 0, 0 ≤ m ≤ M − 1, 0 ≤ n ≤ N − 1.

The inverse ODCT is given by:

M −1 N −1
X X 2πmk 2πnl
f (k, l) = C(m)C(n)fˆoc (m, n) cos cos
m=0 n=0
2M − 1 2N − 1

where C(0) = 1, C(m) = C(n) = 2 if m, n 6= 0.

Image decomposition under ODCT

(Please refer to the Lecture 7 Powerpoint for clearer images)
Using ODCT, the image f can be written as:
X
ODCT
f= gmn Imn .
0≤m,n≤N −1

The elementary images under the ODCT (of an 8 × 8 image) look like the following:

28
 ODCT
Here, the m-th row n-th column image represents the elementary image Imn .

6 Conclusion
In this chapter, we have studied different image transformation and decomposition techniques.
By image decomposition, we can express an image as a linear combination of elementary images.
Each elementary image has different properties. By truncating unnecessary terms, images can
be effectively compressed to save storage. Image compression is one of the most important and
earliest image processing tasks. We have shown the compression results using SVD, Haar, Walsh,
DFT and DCT. In fact, there are other image transformation tools for image compression (such as
even/odd antisymmetric discrete sine transform). Please refer to Lecture 7 Powerpoint for details.
Amongst different techniques, EDCT usually gives the best compression results. This is the reason
why EDCT is applied for the famous JPEG compression.

Exercises
1. Let g ∈ MM ×N (R), and let
A = U ΣV T
be one of its singular value decompositions, such that σii ≥ σjj whenever i < j.
(a) Show that the K-tuple (σ11 , σ22 , · · · , σKK ), where K = min{M, N }, is uniquely deter-
mined.
(b) Suppose {σii : i = 1, 2, · · · , K} are distinct and nonzero. Prove that the first K columns
of U and V are uniquely determined up to a change of sign. In other words, for each
i = 1, 2, · · · , K, there are exactly two choices for (~ui , ~vi ); denoting one choice by (~u, ~v ),
the other is given by (−~u, −~v ).

29
 (c) Suppose M = N , and {σii : i = 1, 2, · · · , M } are distinct. Prove that the claim in (b)
holds.
(d) Does the claim in (b) hold if we drop the assumptions? For dropping each of the
assumptions (distinctness and nonzero), prove it or give a counterexample.
2. Prove Theorem 2.3.
 
1 0 3 0
3. Let f = .
0 4 0 2

(a) Compute an SVD of f .

(b) Express f as a linear combination of its elementary images.
 
a c b
4. Let a, b, c ∈ R, and let f = circ((a, b, c)T ) =  b a c .
c b a

(a) Compute an SVD of f .

(b) Give a condition on the values of a, b and c such that the rank-2 approximation of f is
uniquely determined.
 
5 4 6 6
6 1 6 3
5. Let f = 
1 2 1 5.


6 4 6 1

(a) Compute the Haar transform fHaar of f .

(b) Suppose there is only enough capacity to store 10 pixel values of fHaar . Choose 10
entries to keep such that the reconstucted image differs as little as possible in Frobenius
norm with the original image, and compute the reconstructed image.
6. Let H := {Hm : m ∈ N ∪ {0}} be the sequence of Haar functions.
Consider the inner product space (L2 (R), h·, ·i) where
 Z 
L2 (R) = f : R → R f2 < ∞ ,
R

and for any f, g ∈ L2 (R), Z

hf, gi = f g.
R
Z
(a) (Unit) Prove that [Hm (t)]2 dt = 1 for any m ∈ N ∪ {0}.
R
(Hence Hm ∈ L2 (R) and kHm k = 1.)
(b) (Orthogonality)
i. Prove that hH0 , Hm i = 0 for any m ∈ N \ {0}.
ii. Let m1 , m2 ∈ N such that 0 6= m1 < m2 . Then m1 = 2p1 + n1 and m2 = 2p2 + n2
for some p1 , p2 ∈ N ∪ {0}, n1 ∈ Z ∩ [0, 2p1 − 1] and n2 ∈ Z ∩ [0, 2p2 − 1].
A. Suppose p1 = p2 . Prove that hHm1 , Hm2 i = 0.
Hint. In this case n1 < n2 .
B. Suppose p1 < p2 . Prove that hHm1 , Hm2 i = 0.
Hint. Consider the possible subset relations between the supports of Hm1 and
Hm2 .
The above establishes that H is orthonormal in (L2 (R), h·, ·, i).
 
3 2 4 4
 4 −2 4 1
7. Let f = 
−2 −1 −1 3 .


4 1 4 −2

30
 (a) Compute the Walsh transform fWalsh of f .
(b) Suppose fWalsh is mistakenly stored as an integer-valued matrix, with each entry rounded
away from 0. Compute the reconstructed image from the modified matrix.
8. Let W := {Wm : m ∈ N ∪ {0}} be the sequence of Walsh functions.
Z
(a) (Unit) Prove that [Wm (t)]2 dt = 1 for any m ∈ N ∪ {0}.
R
(Hence Wm ∈ L2 (R) and kWm k = 1.)
(b) (Orthogonality) Let P (m) be the proposition that {W0 , . . . , Wm } is orthogonal in (L2 (R), h·, ·i).
P (0) is vacuously true. We aim to establish:

P (m) is true =⇒ P (2m + 1) is true (?)

whenever m ∈ N∪{0}. Also note that P (m) implies P (m−1) for any m ∈ N\{0} (since
the latter concerns the orthogonality of a subset of the set concerned by the former), so
by induction we establish P (m) to be true for all m ∈ N \ {0} once (?) holds.
Suppose P (k) holds for some k ∈ N ∪ {0}.
Let m1 , m2 ∈ Z ∩ [0, 2k + 1] such that m1 < m2 . Then m1 = 2j1 + q1 and m2 = 2j2 + q2
for some j1 , j2 ∈ N ∪ {0} and q1 , q2 ∈ {0, 1}.
i. Suppose j1 = j2 . Prove that hWm1 , Wm2 i = 0.
Hint. In this case q1 = 0, q2 = 1.
ii. Suppose j1 < j2 . Prove that hWm1 , Wm2 i = 0.
Hint. j2 ≤ n. Make good use of the induction hypothesis.
The above establishes that W is orthonormal in (L2 (R), h·, ·i).
9. (a) Prove that (
Wm (0) if m is even,
lim− Wm (t) =
t→1 −Wm (t) if m is odd.

(b) Hence or otherwise, prove that for any m ∈ N ∪ {0}, Wm has exactly m “zero-crossings”
on (0, 1), in the sense that there are exactly m points t1 , · · · , tm ∈ (0, 1) at which

lim Wm (t) · lim− Wm (t) ≤ 0, i = 1, 2, · · · , m.

t→t+
i t→ti

10. Let R := {Rm : m ∈ N \ {0}} be the sequence of Rademacher functions.

Z
Q
(a) Prove that Rm = 0 for any nonempty finite subset M ⊆ N \ {0}.
m∈M
R
( )
Y
(b) Hence or otherwise, prove that R̃ := Rm : Rm ∈ R, M ⊆ N \ {0} is finite is
m∈M
orthonormal in (L2 (R), h·, ·i).
Remark. W = R̃.
 
3 2 4 4
 4 −3 4 0
11. Let f = 
−2 −1 −2
.
3
4 1 4 −2

(a) Compute the discrete Fourier transform fˆ of f .

(b) Compute the image reconstructed from fˆ using only the 3 frequencies closest to 0.
12. Verify the following properties of the discrete Fourier transform (i) directly; (ii) via the inverse
transform.
∗ g = M N fˆ
(a) Let f, g ∈ MM ×N (R) be periodically extended. Then f[ ĝ, where fˆ
ˆ
ĝ(m, n) = f (m, n)ĝ(m, n).

31
 (b) Let f, g ∈ MM ×N (R) be periodically extended. Then f[g = fˆ ∗ ĝ, where f g(k, l) =
f (k, l)g(k, l).
ˆ ˜
(c) Let f ∈ MN ×N (R) be periodically extended, and let f˜(k, l) = f (l, −k). Then f˜ = fˆ.
(d) Let f ∈ MM ×N (R) be periodically extended, and let f˜(k, l) = f (k − k0 , l − l0 ) for some
ˆ k0 m l0 n
k0 , l0 ∈ Z. Then f˜ = e−2πj( M + N ) fˆ.
˜
(e) Let f ∈ MM ×N (R) be periodically extended, and let fˆ(m, n) = fˆ(m − m0 , n − n0 ) for
˜ km0 ln0
some m0 , n0 ∈ Z. Then fˆ = DF T (e2πj( M + N ) f ).
13. Let f ∈ MM ×N (R). Its cosine and sine transforms are defined as follows:
M −1 NP
−1
• fˆec (m, n) = 1
f (k, l) cos πm(2k+1) cos πn(2l+1)
P
MN 2M 2N ;
k=0 l=0
M −1 NP
−1
• fˆoc (m, n) = 1 2πmk 2πnl
P
(2M −1)(2N −1) [2 − 1M Z (k)][2 − 1N Z (l)]f (k, l) cos 2M −1 cos 2N −1 ;
k=0 l=0
M −1 NP
−1
• fˆes (m, n) = − M1N f (k, l) sin πm(2k+1) sin πn(2l+1)
P
2M 2N ;
k=0 l=0
M −1 NP
−1
• fˆos (m, n) = − (2M +1)(2N
4
sin 2πm(k+1) 2πn(l+1)
P
+1) 2M +1 sin 2N +1 ,
k=0 l=0

Verify the formulae for the inverse transforms:

M −1 NP
−1
[2 − 1M Z (m)][2 − 1N Z (n)]fˆec (m, n) cos πm(2k+1) cos πn(2l+1)
P
• f (k, l) = 2M 2N ;
m=0 n=0
M −1 NP
−1
[2 − 1M Z (m)][2 − 1N Z (n)]fˆoc (m, n) cos 2M
2πmk 2πnl
P
• f (k, l) = −1 cos 2N −1 ;
m=0 n=0
M P
N
[2 − 1M Z (m)][2 − 1N Z (n)]fˆes (m, n) sin πm(2k+1) sin πn(2l+1)
P
• f (k, l) = − 2M 2N ;
m=1 n=1
M P
N
fˆos (m, n) sin 2πm(k+1) 2πn(l+1)
P
• f (k, l) = −16 2M +1 sin 2M +1 .
m=1 n=1

32