Digital Image Processing

Filtering in the Frequency Domain

(Circulant Matrices and Convolution)

Christophoros Nikou
[email protected]

University of Ioannina - Department of Computer Science

2 Toeplitz matrices
• Elements with constant value along the main
diagonal and sub-diagonals.
• For a NxN matrix, its elements are determined by
a (2N-1)-length sequence tn | ( N  1)  n  N  1

T(m, n)  tmn t0 t1 t2 t ( N 1) 

t t0 t1 
1 
T  t 2 t2 
 
 t1 
t N 1
t2 t1 t0  N N
C. Nikou – Digital Image Processing (E12)
3 Toeplitz matrices (cont.)
• Each row (column) is generated by a shift of the
previous row (column).
− The last element disappears.
− A new element appears.

T(m, n)  tmn t0 t1 t2 t ( N 1) 

t t0 t1 
1 
T  t 2 t2 
 
 t1 
t N 1
t2 t1 t0  N N
C. Nikou – Digital Image Processing (E12)
4 Circulant matrices
• Elements with constant value along the main
diagonal and sub-diagonals.
• For a NxN matrix, its elements are determined by
a N-length sequence cn | 0  n  N  1

c0 c1 c2 cN 1 
C(m, n)  c( mn ) mod N c 
 N 1 c0 c1 
C   cN  2 c2 
 
 c1 
c1 c2 cN 1 c0  N N
C. Nikou – Digital Image Processing (E12)
5 Circulant matrices (cont.)
• Special case of a Toeplitz matrix.
• Each row (column) is generated by a circular shift
(modulo N) of the previous row (column).

c0 c1 c2 cN 1 
C(m, n)  c( mn ) mod N c 
 N 1 c0 c1 
C   cN  2 c2 
 
 c1 
c1 c2 cN 1 c0  N N
C. Nikou – Digital Image Processing (E12)
6
Convolution by matrix-vector
operations
• 1-D linear convolution between two discrete
signals may be expressed as the product of a
Toeplitz matrix constructed by the elements of
one of the signals and a vector constructed by the
elements of the other signal.
• 1-D circular convolution between two discrete
signals may be expressed as the product of a
circulant matrix constructed by the elements of
one of the signals and a vector constructed by the
elements of the other signal.
• Extension to 2D signals.
C. Nikou – Digital Image Processing (E12)
7
1D linear convolution using Toeplitz
matrices
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

• The linear convolution g[n]=f [n]*h [n] will be of

length N=N1+N2-1=3+2-1=4.
• We create a Toeplitz matrix H from the
elements of h [n] (zero-padded if needed) with
− N=4 lines (the length of the result).
− N1=3 columns (the length of f [n]).
− The two signals may be interchanged.

C. Nikou – Digital Image Processing (E12)

8
1D linear convolution using Toeplitz
matrices (cont.)
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

 1 0 0
 1 1 0 
H  Length of the result =4
 0 1 1
 
 0 0 1 43 Notice that H is not
circulant (e.g. a -1
appears in the second line
Length of f [n] = 3 which is not present in the
first line.
Zero-padded h[n] in the first column
C. Nikou – Digital Image Processing (E12)
9
1D linear convolution using Toeplitz
matrices (cont.)
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

 1 0 0  1
 1 1 0  1   1
g = Hf     2    
 0 1 1    0
   2   
 0 0 1  2

g[n]  {1, 1, 0,  2}

C. Nikou – Digital Image Processing (E12)

10
1D circular convolution using
circulant matrices
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

• The circular convolution g[n]=f [n]h [n] will

be of length N=max{N1, N2}=3.
• We create a circulant matrix H from the
elements of h [n] (zero-padded if needed)
of size NxN.
− The two signals may be interchanged.

C. Nikou – Digital Image Processing (E12)

11
1D circular convolution using
circulant matrices (cont.)
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

 1 0 1
H   1 1 0
 0 1 1 33

Zero-padded h[n] in the

first column

C. Nikou – Digital Image Processing (E12)

12
1D circular convolution using
circulant matrices (cont.)
f [n]  {1, 2, 2}, h[n]  {1,  1}, N1  3, N 2  2

 1 0 1 1   1
g = Hf   1 1 0  2   1
 0 1 1  2  0

g[n]  { 1, 1, 0}

C. Nikou – Digital Image Processing (E12)

13 Block matrices
 A11 A12 A1N 
A A22 A2 N 
A  21

 
 
 AM 1 AM 2 AMN 
• Aij are matrices.
• If the structure of A, with respect to its sub-matrices, is
Toeplitz (circulant) then matrix A is called block-Toeplitz
(block-circulant).
• If each individual Aij is also a Toeplitz (circulant) matrix then
A is called doubly block-Toeplitz (doubly block-circulant).

C. Nikou – Digital Image Processing (E12)

14
2D linear convolution using doubly
block Toeplitz matrices
m m

1 4 1 1 1
2 5 3 1 -1
n n
f [m,n] h [m,n]

M1=2, N1=3 M2=2, N2=2

The result will be of size (M1+M2-1) x (N1+N2-1) = 3 x 4

C. Nikou – Digital Image Processing (E12)

15
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

1 4 1 1 1
2 5 3 1 -1
n n
f [m,n] h [m,n]

• At first, h[m,n] is zero-padded

to 3 x 4 (the size of the m
result). 0 0 0 0
• Then, for each row of h[m,n], 1 1 0 0
a Toeplitz matrix with 3 1 -1 0 0
columns (the number of n
h[m,n]
columns of f [m,n]) is
constructed. C. Nikou – Digital Image Processing (E12)
16
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m • For each row of h[m,n], a
0 0 0 0 Toeplitz matrix with 3 columns
1 1 0 0 (the number of columns of f
1 -1 0 0 [m,n]) is constructed.
n
h[m,n]

 1 0 0 1 0 0 0 0 0
 1 1 0  1 0 0 
1 0  0
H1    H2   H3  
 0 1 1 0 1 1 0 0 0
     
 0 0 1 0 0 1 0 0 0

C. Nikou – Digital Image Processing (E12)

17
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

1 4 1 1 1
2 5 3 1 -1
n n
f [m,n] h [m,n]

• Using matrices H1, H2

and H3 as elements, a  H1 H3 
doubly block Toeplitz
H   H 2 H1 
matrix H is then
constructed with 2  H3 H 2 126
columns (the number of
rows of f [m,n]).
C. Nikou – Digital Image Processing (E12)
18
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

1 4 1 1 1
2 5 3 1 -1
n n
f [m,n] h [m,n]
2
 5
 
• We now construct  3 (2 5 3)T 
a vector from the f   T 
elements of f [m,n].  1 (1 4 1) 
4
 
 1
C. Nikou – Digital Image Processing (E12)
19
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

1 4 1 1 1
2 5 3 1 -1
n n
f [m,n] h [m,n]
2
 5
 H1 H3   
 3
g = Hf   H 2 
H1   
 1
 H 3 H 2 
4
 
 1
C. Nikou – Digital Image Processing (E12)
20
2D linear convolution using doubly
block Toeplitz matrices (cont.)
 1 0 0 0 0 0  2
 1 1 0 0 0 0   3
   
 0 1 1 0 0 0   2 
   
 0 0 1 0 0 0     3
2
 1 0 0 1 0 0   5   3
      (2 3 2 3)T 
 1 1 0 1 1 0   3  10   T 
g = Hf          (3 10 5 2) 
0 1 1 0 1 1  1 5 
    (1 5 5 1)T 
 0 0 1 0 0 1  4   2   
 0 0 0 1 0 0   1  1
    
 0 0 0 1 1 0  5
   
 0 0 0 0 1 1   5
 0 0 0 0 0 1  1
C. Nikou – Digital Image Processing (E12)
21
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

1 4 1 1 1
2 5 3
n
* 1 -1
n
f [m,n] h [m,n]

m
1 5 5 1 (2 3 2 3)T 
 T 
= 3 10 5 2
2 3 -2 3
g = (3 10 5 2) 
(1 5 5 1)T 
n  
g [m,n]
C. Nikou – Digital Image Processing (E12)
22
2D linear convolution using doubly
block Toeplitz matrices (cont.)
Another example

m m

3 4
1 2 1 -1
n n
f [m,n] h [m,n]

M1=2, N1=2 M2=1, N2=2

The result will be of size (M1+M2-1) x (N1+N2-1) = 2 x 3

C. Nikou – Digital Image Processing (E12)
23
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

3 4
1 2 1 -1
n n
f [m,n] h [m,n]
• At first, h[m,n] is zero-padded
to 2 x 3 (the size of the m
result).
• Then, for each line of h[m,n], 0 0 0
a Toeplitz matrix with 2 1 -1 0
columns (the number of n
h[m,n]
columns of f [m,n]) is
constructed. C. Nikou – Digital Image Processing (E12)
24
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m • For each row of h[m,n], a Toeplitz
matrix with 2 columns (the
0 0 0 number of columns of f [m,n]) is
1 -1 0 constructed.
n
h[m,n]

 1 0 0 0
H1   1 1 H 2  0 0
 0 1 0 0

C. Nikou – Digital Image Processing (E12)

25
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

3 4
1 2 1 -1
n n
f [m,n] h [m,n]
• Using matrices H1 and
H2 as elements, a
doubly block Toeplitz  H1 H2 
H
matrix H is then H2 H1  64
constructed with 2
columns (the number of
rows of f [m,n]).
C. Nikou – Digital Image Processing (E12)
26
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

3 4
1 2 1 -1
n n
f [m,n] h [m,n]

1 
 2  (1 2)T 
• We now construct a f   T
vector from the elements 3  (3 4) 
of f [m,n].  
4

C. Nikou – Digital Image Processing (E12)

27
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

3 4
1 2 1 -1
n n
f [m,n] h [m,n]

1 
 H1 H 2   2
g = Hf    
H2 H1  3 
 
 4
C. Nikou – Digital Image Processing (E12)
28
2D linear convolution using doubly
block Toeplitz matrices (cont.)

 1 0 0 0  1
 1 1 0 0  1  1
    
 0 1 0 0   2   2  (1 1 2)T 
g = Hf          T
 0 0 1 0   3  3
    (3 1 4) 
 0 0 1 1  4   1
   
 0 0 0 1  4 

C. Nikou – Digital Image Processing (E12)

29
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m m

3 4
1 2
* 1 -1
n n
f [m,n] h [m,n]

m
(1 1 2)T 
= 3 1 4
1 1 -2
g= T
(3 1 4) 
n
g [m,n]
C. Nikou – Digital Image Processing (E12)
30
2D circular convolution using doubly
block circulant matrices
The circular convolution g[m,n]=f [m,n]h [m,n]
with 0  m  M  1, 0  n  N  1,

may be expressed in matrix-vector form as:

g = Hf

where H is a doubly block circulant matrix

generated by h [m,n] and f is a vectorized form
of f [m,n].
C. Nikou – Digital Image Processing (E12)
31
2D circular convolution using doubly
block circulant matrices (cont.)
H0 H M 1 H M 2 H1 
H H0 H M 1 H 2 
 1
H  H2 H1 H0 H3 
 
 
 H M 1 H M  2 H M 3 H 0 

Each Hj, for j=1,..M, is a circulant matrix with N

columns (the number of columns of f [m,n])
generated from the elements of the j-th row of
h [m,n].
C. Nikou – Digital Image Processing (E12)
32
2D circular convolution using doubly
block circulant matrices (cont.)

 h [ j , 0] h [ j, N  1] h [ j,1] 
 h [ j ,1] h [ j , 0] h [ j, 2]
Hj  
 
 
 h [ j , N  1] h [ j , N  2] h [ j, 0] N  N

Each Hj, for j=1,..M, is a NxN circulant matrix

generated from the elements of the j-th row of
h [m,n].
C. Nikou – Digital Image Processing (E12)
33
2D circular convolution using doubly
block circulant matrices (cont.)
m m
0 1 0 0 0 0
1 3 -1  1 0 0
1 2 1 1 -1 0
n n
f [m,n] h [m,n]

 1 0 1  1 0 0 0 0 0
 
H 0   1 1 0  H1  0 1 0  H 2  0 0 0
 0 1 1 0 0 1 0 0 0

C. Nikou – Digital Image Processing (E12)

34
2D circular convolution using doubly
block circulant matrices (cont.)
m m
0 1 0 0 0 0
1 3 -1  1 0 0
1 2 1 1 -1 0
n n
f [m,n] h [m,n]

H1   
  
T
H0 H2 1 2 1

g = Hf   H1 
H 2  1 3 1 
T
H0
 
 H 2 H1 H 0    0 1 0 T 
 

C. Nikou – Digital Image Processing (E12)

35
2D circular convolution using doubly
block circulant matrices (cont.)
 1 0 1 0 0 0 1 0 0   1  0 
 1 1 1 0 0 0 0 1 0   2   2 
    
 0 1 0 0 0 0 0 0 1  1  1
    
 1 0 0 1 0 1 0 0 0   1  3
g = Hf   0 1 0 1 1 1 0 0 0   3   4 
    
 0 0 1 0 1 0 0 0 0   1  3
 0 0 0 0 0 0 1 0 1  0   1
    
 0 0 0 0 0 0 1 1 1  1  4 
 0 0 0 0 0 0 0 1 0   0   2 
    

C. Nikou – Digital Image Processing (E12)

36
2D circular convolution using doubly
block circulant matrices (cont.)
m
m 0 0 0
0 1 0
1 3 -1  1 0 0
1 -1 0
1 2 1 n
n h [m,n]
f [m,n]

m  0 2 1T 
1 4 -2  
= 3 4 -3 g =  3 4 3 

T

T
0 2 -1
n  1 4 2  
g [m,n]
C. Nikou – Digital Image Processing (E12)
37 Diagonalization of circulant matrices
Theorem: The columns of the inverse DFT matrix
are eigenvectors of any circulant matrix. The
corresponding eigenvalues are the DFT values of
the signal generating the circulant matrix.
Proof: Let
2 2 n
j j k
wN  e 
 w e nk
N


be the DFT basis elements of length N with:

0  k  N  1, 0  n  N  1,
C. Nikou – Digital Image Processing (E12)
38
Diagonalization of circulant matrices
(cont.)
We know that the DFT F [k] of a 1D signal f [n]
may be expressed in matrix-vector form:
F = Af
where
f =  f [0], f [1],..., f [ N  1] , F =  F [0], F [1],..., F [ N  1]
T T

  w0  0 w 0 1
w 
0 2
w 0 N 1

 N N N N

 1 0 1 N 1 
w  w 1 1
w  w 
1 2

A N N N N 
 
 
 w N 1 0 w  w 
N 1 1 N 1 2
 wNN 1  
N 1

 N N N

C. Nikou – Digital Image Processing (E12)

39
Diagonalization of circulant matrices
(cont.)
The inverse DFT is then expressed by:
f = A -1F
where  0 * T

   wN   wN0   wN0   wN0 
0 1 2 N 1

 
 1 0 1 N 1 

1  w  w 1 1
w   wN  
2

1
A  A 
11* T
  N N N

N N  
  
  N 1 0 
  wN  w  w   wNN 1  
N 1 1 N 1 2 N 1

 N N


The theorem implies that any circulant matrix has

eigenvectors the columns of A-1.
C. Nikou – Digital Image Processing (E12)
40
Diagonalization of circulant matrices
(cont.)
Let H be a NxN circulant matrix generated by the 1D
N-length signal h[n], that is:
H(m, n)  h (m  n)mod N  h[m  n]N
Let also αk be the k-th column of the inverse DFT
matrix A-1.
We will prove that αk, for any k, is an eigenvector of H.
The m-th element of the vector Hαk, denoted by  Hα k m
is the result of the circular convolution of the signal
h[n] with αk.
C. Nikou – Digital Image Processing (E12)
41
Diagonalization of circulant matrices
(cont.)
N 1 N 1
 Hα k m   h [ m  n ]N  k [ n ] 
1
 h [ m  n] N wk n
N
n 0 N n 0

l mn m  ( N 1) m  ( N 1)

1

1 k m

N l m
h [l ]N w  k ( m l )
N  wN
N

l m
h [l ]N wNk l

1  k m  1 m
kl 
 wN   h [l ]N wN   h [l ]N wN 
kl

N l m( N 1) l 0 

We will break it into two parts

C. Nikou – Digital Image Processing (E12)
42
Diagonalization of circulant matrices
(cont.)
1  k m  1 N 1 N 1
kl 
 wN   h [l ]N wN   h [l ]N wN   h [l ]N wN 
kl kl

N l m( N 1) l 0 l  m1 

Periodic with respect to N.

1  k m  N 1 N 1 N 1
kl 
 wN   h [l ]N wN   h [l ]N wN   h [l ]N wN 
kl kl

N l  N m( N 1) l 0 l  m1 

1  k m  N 1 N 1 N 1
kl 
 wN   h [l ]N wN   h [l ]N wN   h [l ]N wN 
kl kl

N l m1 l 0 l  m 1 

C. Nikou – Digital Image Processing (E12)

43
Diagonalization of circulant matrices
(cont.)
1  k m  N 1 kl 
 Hα k m  wN  h [l ]N wN   H [k ]αk m
N  l 0 
DFT of h[n] at k.

This holds for any value of m. Therefore:

Hα k  H [k ]α k
which means that αk, for any k, is an eigenvector of
H with corresponding eigenvalue the k-th element
of H[k], the DFT of the signal generating H .
C. Nikou – Digital Image Processing (E12)
44
Diagonalization of circulant matrices
(cont.)
The above expression may be written in terms of
the inverse DFT matrix:
1 1
HA  A Λ
Λ=diag H[0], H[1],..., H [ N 1]

or equivalently: Λ  ΑHA 1

Based on this diagonalization, we can prove the

property between circular convolution and DFT.

C. Nikou – Digital Image Processing (E12)

45
Diagonalization of circulant matrices
(cont.)
g = Hf  g = HA -1 Af  Ag = AHA-1 Af  G = ΛF

 G[0]   H [0] 0 0  F [0] 

 G[0]   0 H [1] 0  F [0] 
 =  
    
    
G[ N  1]  0 0 H [ N  1]  F [ N  1]

DFT of g[n] DFT of h [n] DFT of f [n]

C. Nikou – Digital Image Processing (E12)

46
Diagonalization of doubly block
circulant matrices
• These properties may be generalized in 2D.
• We need to define the Kronecker product:
M N K L
A , B

 a11B a12B a1N B 

a B a B 
a2 N B 
AB   21 22

 
 
 aM 1B aM 2B aMN B  MK  NL
C. Nikou – Digital Image Processing (E12)
47
Diagonalization of doubly block
circulant matrices (cont.)
• The 2D signal f [m,n], 0  m  M  1, 0  n  N  1,
may be vectorized in lexicographic ordering
(stacking one column after the other) to a
vector:
f  MN1
• The DFT of f [m,n], may be obtained in
matrix-vector form:
F   A  A f

C. Nikou – Digital Image Processing (E12)

48
Diagonalization of doubly block
circulant matrices (cont.)
Theorem: The columns of the inverse 2D DFT
matrix
A  A
1

are eigenvectors of any doubly block circulant

matrix. The corresponding eigenvalues are the 2D
DFT values of the 2D signal generating the doubly
block circulant matrix:

Λ  A  A H A  A
1

Diagonal, containing the 2D DFT Doubly block circulant

of h[m,n] generating H
C. Nikou – Digital Image Processing (E12)