Image Processing

Lecture 2.1 - Image filtering

Idar Dyrdal
Image function y f (x, y)
f : R2 ! R
x

2D signal where f (x, y) gives the intensity at position (x, y)

2
 Digital Image
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 255 255 0 0 0 0 0

0 0 0 255 255 170 0 0 0 0

0 0 255 170 170 255 0 0 0 0

0 0 255 170 170 85 85 0 0 0

0 170 170 255 85 85 170 170 0 0

0 0 85 85 170 170 170 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Discrete (sampled and quantized) version of the (continuous) image function f (x, y)
3
Image Processing

• Point operators

• Image filtering in spatial domain

- Linear filters
- Non-linear filters
• Image filtering in frequency domain
- Fourier transform

f [i, j] ! g[i, j]

4
Point Operators

• Pixel transforms f [i, j] g[i, j]

- Brightness adjustment
- Contrast adjustment 10 5 3 h 20 10 6
- … 4 5 1 8 10 2
• Colour transforms
1 1 7 2 2 14

• Histogram equalization

• … g[i, j] = 2f [i, j]
(Each pixel multiplied by 2)
g[i, j] = h(f [i, j])
(Pixel-by-pixel transformation)

5
Pixel transforms - example
p
f [i, j] g[i, j] = f [i, j]

Original image Processed image

6
Histogram equalization

7
Image filtering

• Image filters in the spatial domain:

– Filtering is a mathematical operation on a grid of numbers
– Smoothing, sharpening (enhancing the image)
– Feature extraction (measuring texture, finding edges, distinctive points and
patterns).

• Image filters in the frequency domain:

– Filtering is a way to modify the (spatial) frequencies of images
– Noise removal, (re)sampling, image compression.

8
Image filtering in spatial domain

Modify the pixels in an image based on some function of a local neighborhood of each pixel:

f [i, j] g[i, j]
10 5 3
h
4 5 1 10

1 1 7

Local image data Modified image data

9
Linear filtering

Convolution or cross-correlation where each pixel in the filtered image is a linear combination of the
pixels in a local neighborhood in the original image:

10 5 3 0 0 0
4 6 1 0 1 0 16
1 1 8 0 2 1
Local image data Kernel Modified image data

The coefficients of the linear combination is contained in the “kernel” (filter mask).

10
Cross-correlation

Let f be the image, h be the kernel (of size 2k+1 x 2k+1), and g
be the output image:
k
X k
X
g[i, j] = h[u, v]f [i + u, j + v]
u= k v= k

This is called a cross-correlation operation:

g =h⌦f
11
Linear filtering

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 10 20 20 10 0 0 0 0

0
0

0
0

0
90

90
90

90
0

90
0

0
0

0
0

0
0

0
h 0 0 20

1 1 1
f 0

0
0

0
90

90
90

90
90

90
90

90
0

90
0

0
0

0
0

0
⌦ 1 1 1 = g
1 1 1
0 90 90 90 90 90 90 90 0 0

0 0 90 90 90 90 90 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

X
g[i, j] = h[u, v]f [i + u, j + v]
u,v
12
Linear filtering

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 10 20 20 10 0 0 0 0

0 0 0 90 90 0 0 0 0 0 0 0 20 40 50 30 10 0 0 0

0 0 0 90 90 90 0 0 0 0 0 10 40 70 80 50 20 0 0 0

1 1 1

⌦
0 0 90 90 90 90 0 0 0 0 0 20 50 80 90 70 40 10

0 0 90 90 90 90 90 0 0 0
1
1
1
1
1
1
=
0 90 90 90 90 90 90 90 0 0

0 0 90 90 90 90 90 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

13
Linear filtering

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 10 20 20 10 0 0 0 0

0 0 0 90 90 0 0 0 0 0 0 0 20 40 50 30 10 0 0 0

0 0 0 90 90 90 0 0 0 0 0 10 40 70 80 50 20 0 0 0

1 1 1
0 0 90 90 90 90 0 0 0 0 0 20 50 80 90 70 40 10 0 0

0 0 90 90 90 90 90 0 0 0
⌦ 1
1
1
1
1
1
= 0 40 70 90 90 80 60 30 10 0

0 90 90 90 90 90 90 90 0 0 10 40 70 90 90 90 70 40 10 0

0 0 90 90 90 90 90 0 0 0 0 30 50 60 60 60 50 30 10 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

14
Moving average filter (box filter)

3 x 3 kernel

1 1 1
1 1 1
1 1 1

Replaces each pixel with an average of

9 x 9 kernel
its neigborhood (smoothing effect)
X
g[i, j] = h[u, v]f [i + u, j + v]
u,v
15
Sharpening filter

Enhances differences with 0 0 0 1 1 1

local average 0
0
2
0
0
0
- 1
1
1
1
1
1

16
Convolution
• Same as cross-correlation, except that the kernel is “flipped” (horizontally and vertically):

k
X k
X
g[i, j] = h[u, v]f [i u, j v]
u= k v= k

• This is called a convolution operation:

g =h⇤f
• Convolution is commutative and associative (no difference between filter and image):

a⇤b=b⇤a a ⇤ (b ⇤ c) = (a ⇤ b) ⇤ c
• Apply several filters, one after the other:

(((a ⇤ b1 ) ⇤ b2 ) ⇤ b3 ) = a ⇤ (b1 ⇤ b2 ⇤ b3 )
17
Gaussian filter (smoothing)

5 ⇥ 5, =1

1 x2 +y 2
( 2 2 )
G = 2
e
100 ⇥ 100, = 10 2⇡
18
Gaussian filtering

Original image = 2 pixels = 4 pixels = 8 pixels

19
Separable filters - example

The 2D Gaussian kernel can be expressed as a product of two 1D kernels:

✓ 2 2
◆ ✓ 2
◆ ✓ 2
◆
1 x +y 1 x 1 y
G (x, y) = 2
exp =p exp ⇥p exp
2⇡ 2 2 2⇡ 2 2 2⇡ 2 2

Discrete 3 x 3 approximation:

1 2 1 1
2 4 2 = 2 ⇥ 1 2 1
1 2 1 1

More efficient to perform two 1D convolutions compared to a single 2D convolution!

20
2D convolution

1 2 1 1 2 2
2 4 2 ⇤ 1 4 4 = 47
1 2 1 3 3 5
Filter kernel 3 x 3 image window Result (center pixel only)

21
1D convolution along rows and columns

Convolution along rows:

1 2 2 7
1 2 1 ⇤ 1 4 4 = 13
3 3 5 14

Convolution along remaining column:

1 7
2 ⇤ 13 = 47
1 14

22
Edge detection

23
Edges and image derivatives
Image gradient:
• An edge is a place of rapid change of the 
image intensity function @f @f
rf = ,
• Corresponds to extrema of the first @x @y
derivative of the image intensity function
Gradient magnitude:
• Discrete approximation to the image s✓
derivatives: ◆2 ✓ ◆2
@f @f
||rf || = +
@x @y
@f
[i, j] ⇡ f [i + 1, j] f [i, j]
@x Prewitt operator:

-1 0 1 -1 -1 -1
@f Gx = -1 0 1 Gy = 0 0 0
[i, j] ⇡ f [i, j + 1] f [i, j] -1 0 1 1 1 1
@y

24
Image gradient

@f
f @x

@f
||rf ||
@y

25
Derivative of Gaussians @
h (u, v)
@u

@
h (u, v)
@v

1 u2 +v 2
( 2 2 )
h (u, v) = 2
e
2⇡
26
Sobel operator

Common approximation of the derivative of a

Gaussian:

-1 0 1 -1 -2 -1

-2 0 2 0 0 0

-1 0 1 1 2 1

x-direction y-direction

27
 Gradient magnitude
Sobel operator - example
x-direction y-direction

28
Non-linear filtering - Median filter

A median filter operates over a neighborhood in the input image by selecting the median intensity:

10 5 3 Compute median
4 6 1 from 4
1 1 8 neighborhood
Local image data Modified image data

Other non-linear filters:

• Bilateral filters (outlier rejection)
• Anisotropic diffusion
• Morphological operations (on binary images)
• …

29
Median filtering - example

Image with Salt & Pepper noise Image after median filtering
30
Morphological operations
• Non-linear filtering
• Typically used to clean up binary images
• Erosion: replace pixel value with minimum
in local neighborhood
• Dilation: replace pixel value with maximum
in local neighborhood
• Structuring element used to define the local
neighborhood:

A shape (in blue) and its morphological dilation (in

green) and erosion (in yellow) by a diamond-
shape structuring element.

31
Morphological operations - Erosion

Structuring element (disk shaped)

32
Morphological operations - Dilation

Structuring element (disk shaped)

33
Opening = Erosion + Dilation

34
Closing = Dilation + Erosion

35
Filtering in frequency domain

Fourier (1807):
Any univariate function can be rewritten as
a weighted sum of sines and cosines of
different frequencies (true with some subtle
restrictions).

This leads to:

• Fourier Series
• Fourier Transform (continuous and discrete)
• Fast Fourier Transform (FFT)
Jean Baptiste Joseph Fourier (1768-1830)

36
Sum of sines

+ + =

A sin(!x + ) Amplitude

The Fourier transform stores the magnitude and phase at

each frequency

Amplitude: Phase:
p 1 I(!)
A = ± R(!)2 + I(!)2 = tan
R(!)
f0 2f0 3f0 Frequency

37
Two-dimensional Fourier transform

Continous transform:
Z 1 Z 1
j(!x x+!y y)
H(!x , !y ) = h(x, y)e dxdy
1 1

Discrete transform:

M
X1 NX1
1 2⇡j km m+k nn
H(km , kn ) = h(m, n)e MN
MN m=0 n=0

38
Fourier analysis in images

Intensity images

Fourier images

39
The Convolution Theorem

The Fourier transform of the convolution of two functions is the product of their
Fourier transforms:

F [g ⇤ h] = F [g]F [h]

Convolution in spatial domain is equivalent to multiplication in frequency domain:

1
g⇤h=F [F [g]F [h]]

40
Example – Gaussian (low pass) filtering

Original colour image Gray level image

41
Example – Gaussian filtering

FFT

F [g]
g
42
Example – Gaussian filtering

FFT

F [h]
h
43
Example – Gaussian filtering

F [g] F [h] F [g]F [h]

44
Example – Gaussian filtering

Inverse
FFT

F [g]F [h] 1
g⇤h=F [F [g]F [h]]

45
Summary

Image Processing

• Point operators

• Image filtering in spatial domain

– Linear filters
– Non-linear filters

• Image filtering in frequency domain

– Fourier transforms
– Gaussian (low pass) filtering

More information: Szeliski 3.1 – 3.4

46