Convolutions of Discrete Functions

Convolutions of Continuous Functions

Computational Foundations of Cognitive Science

Lecture 15: Convolutions and Kernels

Frank Keller

School of Informatics
University of Edinburgh
[email protected]

February 23, 2010

Frank Keller Computational Foundations of Cognitive Science 1

 Convolutions of Discrete Functions
Convolutions of Continuous Functions

1 Convolutions of Discrete Functions

Definition
Convolution of Vectors
Mid-lecture Problem
Convolution of Matrices

2 Convolutions of Continuous Functions

Definition
Example: Signal Processing

Frank Keller Computational Foundations of Cognitive Science 2

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Definition

Definition: Convolution
If f and g are discrete functions, then f ∗ g is the convolution of f
and g and is defined as:
+∞
X
(f ∗ g )(x) = f (u)g (x − u)
u=−∞

Intuitively, the convolution of two functions represents the amount

of overlap between the two functions. The function g is the input,
f the kernel of the convolution.
Convolutions are often used for filtering, both in the temporal or
frequency domain (one dimensional) and in the spatial domain
(two dimensional).
Frank Keller Computational Foundations of Cognitive Science 3
 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Definition

Theorem: Properties of Convolution

If f , g , and h are functions and a is a constant, then:
f ∗ g = g ∗ f (commutativity)
f ∗ (g ∗ h) = (f ∗ g ) ∗ h (associativity)
f ∗ (g + h) = (f ∗ g ) + (f ∗ h) (distributivity)
a(f ∗ g ) = (af ) ∗ g = f ∗ (ag ) (associativity with scalar
multiplication)

Note that it doesn’t matter if g or f is the kernel, due to

commutativity.

Frank Keller Computational Foundations of Cognitive Science 4

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Vectors

If a function f ranges over a finite set

 of values a = a1 , a2 , . . . , an ,
then it can be represented as vector a1 a2 . . . an .

Definition: Convolution of Vectors

If the functions f and g are represented
 as vectors
a = a1 a2 . . . am and b = b1 b2 . . . bn , then f ∗ g is a
vector c = c1 c2 . . . cm+n−1 as follows:
X
cx = au bx−u+1
u

where u ranges over all legal subscripts for au and bx−u+1 ,

specifically u = max(1, x − n + 1) . . . min(x, m).

Frank Keller Computational Foundations of Cognitive Science 5

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Vectors

If we assume that the two vectors a and b have the same

dimensionality, then the convolution c is:
c1 = a1 b1
c2 = a1 b2 + a2 b1
c3 = a1 b3 + a2 b2 + a3 b1
...
cn = a1 bn + a2 bn−1 + · · · + an b1
...
c2n−1 = an bn
Note that the sum for each component only includes those
products for which the subscripts are valid.

Frank Keller Computational Foundations of Cognitive Science 6

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Vectors

Example
  
1 0
Assume a =  0  , b = 1.
−1 2
Then      
a1 b1 1·0 0

 a1 b2 + a2 b1 
 
 1·1+0·0  1
  
a∗b= a1 b3 + a2 b2 + a3 b1  = 1 · 2 + 0 · 1 + (−1)0 =  2 .
    
 a2 b3 + a3 b2   0 · 2 + (−1)1  −1
a3 b3 (−1)2 −2

Frank Keller Computational Foundations of Cognitive Science 7

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Mid-lecture Problem

What happens if the kernel is smaller than the input vector (this is
actually the typical case)?
 
1
1
 1  
−2 1
−1. Compute a ∗ b.
Assume a = 1 , b =  
2  
−1
−1

What is the purpose of the kernel a?

Frank Keller Computational Foundations of Cognitive Science 8

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Matrices

We can extend convolution to functions of two variables f (x, y )

and g (x, y ).

Definition: Convolution for Functions of two Variables

If f and g are discrete functions of two variables, then f ∗ g is the
convolution of f and g and is defined as:
+∞
X +∞
X
(f ∗ g )(x, y ) = f (u, v )g (x − u, y − v )
u=−∞ v =−∞

Frank Keller Computational Foundations of Cognitive Science 9

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Matrices
We can regard functions of two variables as matrices with
Axy = f (x, y ), and obtain a matrix definition of convolution.

Definition: Convolution of Matrices

If the functions f and g are represented as the n × m matrix A and
the k × l matrix B, then f ∗ g is an (n + k − 1) × (m + l − 1)
matrix C : XX
cxy = auv bx−u+1,y −v +1
u v

where u and v range over all legal subscripts for auv and
bx−u+1,y −v +1 .

Note: the treatment of subscripts can vary from implementation to

implementation, and affects the size of C (this is parameterizable in
Matlab, see documentation of conv2 function).
Frank Keller Computational Foundations of Cognitive Science 10
 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Convolution of Matrices

Example
 
  b11 b12 b13 b14 b15
a11 a12 a13 b21 b22 b23 b24 b25 
Let A = a21 a22 a23  and B = 
b31
.
b32 b33 b34 b35 
a31 a32 a33
b41 b42 b43 b44 b45
Then for C = A ∗ B, the entry c33 = a11 b33 + a12 b32 + a13 b31 +
a21 b23 + a22 b22 + a23 b21 + a31 b13 + a32 b12 + a33 b11 .
Here, B could represent an image, and A could represent a kernel
performing an image operation, for instance.

Frank Keller Computational Foundations of Cognitive Science 11

 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Example: Image Processing

Convolving and image with a kernel (typically a 3 × 3 matrix) is a
powerful tool for image processing.

B= K ∗B =

 
1/9 1/9 1/9
K = 1/9 1/9 1/9 implements a mean filter which smooths an
1/9 1/9 1/9
image by replacing each pixel value with the mean of its neighbors.
Frank Keller Computational Foundations of Cognitive Science 12
 Definition
Convolutions of Discrete Functions Convolution of Vectors
Convolutions of Continuous Functions Mid-lecture Problem
Convolution of Matrices

Example: Image Processing

B= K ∗B =

 
1 0 −1
The kernel K = 2 0 −2 implements the Sobel edge detector.
1 0 −1
It detects gradients in the pixel values (sudden changes in brightness),
which correspond to edges. The example is for vertical edges.

Frank Keller Computational Foundations of Cognitive Science 13

 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Definition

We can also define convolution for continuous functions. In this

case, we replace the sums by integrals in the definition.
Definition: Convolution
If f and g are continuous functions, then f ∗ g is the convolution
of f and g and is defined as:
Z +∞
(f ∗ g )(x) = f (u)g (x − u)du
−∞

Convolutions of continuous functions are widely used in signal

processing for filtering continuous signals, e.g., speech.

Frank Keller Computational Foundations of Cognitive Science 14

 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Definition

Example
Assume  the following step functions: 
1
3 if 0 ≤ x ≤ 4 2 if − 1 ≤ x ≤ 1
g (x) = f (x) =
0 otherwise  0 otherwise
 0 if x ≤ 0
If we integrate g (x), we get: G (x) = 3x if 0 ≤ x ≤ 4 .
12 if x > 4

Then the convolution f ∗ g is:
R +∞ R1
(f ∗ g )(x) = −∞ f (u)g (x − u)du = 21 −1 g (x − u)du =
R x−1
− 21 x+1 g (u)du = − 12 (G (x − 1) − G (x + 1)) =
 3
(x + 1) if − 1 ≤ x < 1
 2


3 if 1 ≤ x ≤ 3
3
 2
 − (x − 1) + 6 if 3<x ≤5
0 otherwise


Frank Keller Computational Foundations of Cognitive Science 15

 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Definition
Function g (x) (blue) and convolution f ∗ g (x) (green):
4

3.5

2.5

1.5

0.5

−0.5

−1
−2 −1 0 1 2 3 4 5 6

Frank Keller Computational Foundations of Cognitive Science 16

 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Example: Signal Processing

Assume we have a function I (x) that represents the intensity of a

signal over time. This can be a very spiky function.
We can make this function less spiky by convolving it with a
Gaussian kernel. This is a kernel given by the Gaussian function:
1 1 2
G (x) = √ e − 2 x
2π
The convolution G ∗ I is a smoothed version of the original
intensity function.
We will learn more about the Gaussian function (aka normal
distribution) in the second half of this course.

Frank Keller Computational Foundations of Cognitive Science 17

 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Example: Signal Processing

Gaussian function G (x):
0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0
−3 −2 −1 0 1 2 3
Frank Keller Computational Foundations of Cognitive Science 18
 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Example: Signal Processing

Original intensity function I (x):
220

200

180

160

140

120

100

80

60

40

20

0
0 20 40 60 80 100 120 140 160 180 200
Frank Keller Computational Foundations of Cognitive Science 19
 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Example: Signal Processing

Smoothed function obtained by convolution with a Gaussian kernel:
220

200

180

160

140

120

100

80

60

40

20

0
0 20 40 60 80 100 120 140 160 180 200
Frank Keller Computational Foundations of Cognitive Science 20
 Convolutions of Discrete Functions Definition
Convolutions of Continuous Functions Example: Signal Processing

Summary

P
The convolution (f ∗ g )(x) = f (u)g (x − u) represents the
overlap between a discrete function g and a kernel f ;
convolutions in one dimension can be represented as vectors,
convolutions in two dimensions as matrices;
in image processing, two dimensional convolution can be used
to filter an image or for edge detection;
for continuousR functions, convolution is defined as
(f ∗ g )(x) = f (u)g (x − u)du;
this can be used in signal processing, e.g., to smooth a signal.

Frank Keller Computational Foundations of Cognitive Science 21