Digital Image Processing

Image Enhancement
(Spatial Filtering 1)
 2
of
19
Contents
In this lecture we will look at spatial filtering
techniques:
– Neighbourhood operations
– What is spatial filtering?
– Smoothing operations
– What happens at the edges?
– Correlation and convolution
 3
of
19
Neighbourhood Operations
Neighbourhood operations simply operate
on a larger neighbourhood of pixels than
point operations Origin x

Neighbourhoods are
mostly a rectangle
around a central pixel
(x, y)
Any size rectangle Neighbourhood

and any shape filter

are possible
y Image f (x, y)
 4
of
19
Simple Neighbourhood Operations
Some simple neighbourhood operations
include:
– Min: Set the pixel value to the minimum in
the neighbourhood
– Max: Set the pixel value to the maximum in
the neighbourhood
– Median: The median value of a set of
numbers is the midpoint value in that set (e.g.
from the set [1, 7, 15, 18, 24] 15 is the
median). Sometimes the median works better
than the average
 5
of
Simple Neighbourhood Operations
19 Example

Original Image x Enhanced Image x

123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y y
 6
of
19
Spatial Filtering

A spatial domain filter consists of:

• A neighborhood

•A pre-defined operation that is performed on the image

pixels encompassed by the neighborhood pixels
 7
of
19
The Spatial Filtering Process
Origin x
a b c r s t
d
g
e
h
f
i
* u
x
v
y
w
z
Original Image Filter
Simple 3*3 Pixels
e 3*3 Filter
Neighbourhood
eprocessed = v*e +
r*a + s*b + t*c +
u*d + w*f +
y Image f (x, y) x*g + y*h + z*i

The above is repeated for every pixel in the

original image to generate the filtered image
 8
of
19
Spatial Filtering: Equation Form
a b
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

g ( x, y )    w( s, t ) f ( x  s, y  t )
s  at  b

Filtering can be given

in equation form as
shown above
Notations are based
on the image shown
to the left
 9
of
19
Smoothing Spatial Filters
One of the simplest spatial filtering
operations we can perform is a smoothing
operation
– Simply average all of the pixels in a
neighbourhood around a central value
– Especially useful
in removing noise
1
/ 9
1
/ 9
1
/9
from images Simple
1
/9 1
/9 1
/9 averaging
– Also useful for
highlighting gross filter
1
/9 1
/9 1
/9
detail
 10
of
19
Smoothing Spatial Filtering
Origin x
104 100 108 1
/9 1
/9 1
/9

* /9 /9 /9
1 1 1
99 106 98

95 90 85 1
/9 1
/9 1
/9
1
/9 100
104 1
/9 108
1
/9
Original Image Filter
Simple 3*3 /9 106
1
99 1
/9 198
/9
3*3 Smoothing Pixels
Neighbourhood /9 190
1
95 /9 185
/9 Filter
e = 1/9*106 +
1
/9*104 + 1/9*100 + 1/9*108 +
1
/9*99 + 1/9*98 +
y Image f (x, y) 1
/9*95 + 1/9*90 + 1/9*85
= 98.3333
The above is repeated for every pixel in the
original image to generate the smoothed image
 11
of
19
Image Smoothing Example
The image at the top left
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

is an original image of
size 500*500 pixels
The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
– 3, 5, 9, 15 and 35
Notice how detail begins
to disappear
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
12
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
13
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
14
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
15
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
16
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
17
Image Smoothing Example
18
of
19
Weighted Smoothing Filters
More effective smoothing filters can be
generated by allowing different pixels in the
neighbourhood different weights in the
averaging function
– Pixels closer to the
1
/16 2
/16 /16
1

central pixel are more

important
2
/16 4
/16 /16
2

– Often referred to as a 1
/16 2
/16 /16
1
weighted averaging
Weighted
averaging filter
 19
of
19
Another Smoothing Example
By smoothing the original image we get rid
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of lots of the finer detail which leaves only

the gross features for thresholding

Original Image Smoothed Image Thresholded Image

 20
of
Averaging Filter Vs. Median Filter
19 Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Original Image Image After Image After

With Noise Averaging Filter Median Filter

Filtering is often used to remove noise from

images
Sometimes a median filter works better than
an averaging filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
21
Averaging Filter Vs. Median Filter
Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
22
Averaging Filter Vs. Median Filter
Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
23
Averaging Filter Vs. Median Filter
Example
24
of
Simple Neighbourhood Operations
19 Example

x
123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y
25
of
19
Strange Things Happen At The Edges!

At the edges of an image we are missing

pixels to form a neighbourhood
Origin x
e e

e e e
y Image f (x, y)
26
of
Strange Things Happen At The Edges!
19 (cont…)
There are a few approaches to dealing with
missing edge pixels:
– Omit missing pixels
• Only works with some filters
• Can add extra code and slow down processing
– Pad the image
• Typically with either all white or all black pixels
– Replicate border pixels
– Truncate the image
– Allow pixels wrap around the image
• Can cause some strange image artefacts
27
of
Simple Neighbourhood Operations
19 Example

x
123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y
 28
of
Strange Things Happen At The Edges!
19 (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Filtered Image:
Zero Padding

Original Filtered Image:

Image Replicate Edge Pixels

Filtered Image:
Wrap Around Edge Pixels
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
29
Strange Things Happen At The Edges!
(cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
30
Strange Things Happen At The Edges!
(cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
19
31
Strange Things Happen At The Edges!
(cont…)
32
of
19
Correlation & Convolution

• Correlation : Moving filter mask over the

image and computing the sum of products at
each location.

• Convolution is same except the filter is

rotated by 180⁰.
33
of
19
Correlation & Convolution
The filtering we have been talking about so far is referred to as
correlation with the filter itself referred to as the correlation
kernel.

Convolution is a similar operation, with just one subtle difference

a b c r s t eprocessed = v*e +
d
f
e
g h
e
* u
x
v
y
w
z
z*a + y*b + x*c +
w*d + u*e +
t*f + s*g + r*h
Original Image Filter
Pixels

For symmetric filters it makes no difference

34
of
19
35
of
19
36
of
19
37
of
19

A Convolutional Neural Network (CNN) is a type of deep learning model particularly

effective for image recognition and processing tasks. Here’s a brief overview of its
architecture:
Input Layer: This layer takes in the raw pixel data of the image. The number of neurons
in this layer equals the number of pixels in the image.
Convolutional Layer: This layer applies a set of filters to the input image to extract
features. Each filter slides over the image to produce a feature map, highlighting specific
patterns like edges or textures.
Pooling Layer: Also known as the subsampling or downsampling layer, this layer
reduces the spatial dimensions of the feature maps, which helps in reducing the
computational load and controlling overfitting. Common types of pooling include max
pooling and average pooling.
Fully Connected Layer: After several convolutional and pooling layers, the high-level
reasoning in the neural network is done via fully connected layers. These layers connect
every neuron in one layer to every neuron in the next layer, similar to a traditional neural
network.
Output Layer: This layer provides the final classification or prediction. For classification
tasks, a softmax activation function is often used to output probabilities for each class.
The network learns the optimal filters and weights through a process called
backpropagation, which minimizes the error by adjusting the weights based on the
gradient descent algorithm
38
of
19
Summary
In this lecture we have looked at the idea of
spatial filtering and in particular:
– Neighbourhood operations
– The filtering process
– Smoothing filters
– Dealing with problems at image edges when
using filtering
– Correlation and convolution
– CNN
Next time we will looking at sharpening
filters and more on filtering and image
enhancement