Image Processing

(Lecture 4)
 Introduction
• The name filter is borrowed from frequency
domain processing where “filtering” refers to
passing, modifying, or rejecting specified
frequency components of an image.
• For example, a filter that passes low frequencies
is called a lowpass filter.
• The net effect produced by a lowpass filter is to
smooth an image by blurring it.
• We can accomplish similar smoothing directly on
the image itself by using spatial filters.
• Spatial filtering modifies an image by replacing
the value of each pixel by a function of the
values of the pixel and its neighbors. (refer
book section 2.6 for linearity)
• If the operation performed on the image
pixels is linear, then the filter is called a linear
spatial filter. Otherwise, the filter is a
nonlinear spatial filter.
 The mechanics of linear spatial
filtering
• A linear spatial filter performs a sum-of-
products operation between an image f and a
filter kernel, w.
• The kernel is an array whose size defines the
neighborhood of operation, and whose
coefficients determine the nature of the filter.
• Other terms used to refer to a spatial filter
kernel are mask, template, and window. We
use the term filter kernel or simply kernel.
 The mechanics of linear spatial
filtering
• At any point (x, y) in the image, the response,
g(x, y), of the filter is the sum of products of
the kernel coefficients and the image pixels
encompassed by the kernel:
The mechanics of linear spatial
filtering using 3*3 kernel
 The mechanics of linear spatial
filtering
• As coordinates x and y are varied, the center
of the kernel moves from pixel to pixel,
generating the filtered image, g, in the
process.
• Observe that the center coefficient of the
kernel, w(0, 0), aligns with the pixel at location
(x, y). For a kernel of size m × n, we assume
that m = 2a + 1 and n = 2b + 1, where a and b
are nonnegative integers.
 Spatial Correlation and Convolution
• Spatial correlation consists of moving the center
of a kernel over an image, and computing the
sum of products at each location.
• The mechanics of spatial convolution are the
same, except that the correlation kernel is
rotated by 180°. Thus, when the values of a
kernel are symmetric about its center, correlation
and convolution yield the same result.
 Spatial Correlation and Convolution
• The reason for rotating the kernel will become
clear in the following discussion. The best way
to explain the differences between the two
concepts is by example. (refer to book) 1-D
For 1-D image
• The first thing we notice is that part of w lies
outside f, so the summation is undefined in
that area. A solution to this problem is to pad
function f with enough 0’s on either side.
• In general, if the kernel is of size 1 × m, we
need (m − 1)/2 zeros on either side of f in
order to handle the beginning and ending
configurations of w with respect to f.
For 2-D Image
For 2-D Image
16
of
43
Spatial Filtering
• Use of spatial masks for filtering is called spatial filtering
– May be linear or nonlinear
• Linear filters
– Lowpass: attenuate (or eliminate) high frequency components
such as characterized by edges and sharp details in an image
• Net effect is image blurring
– Highpass: attenuate (or eliminate) low frequency components
such as slowly varying characteristics
• Net effect is a sharpening of edges and other details
– Bandpass: attenuate (or eliminate) a given frequency range
• Used primarily for image restoration(are of little interest for
image enhancement)
17
of
Order-Statistic nonlinear spatial
43
filters
• Nonlinear spatial filters also operate on neighborhoods
• Their operation is based directly on pixel values in the neighborhood
under consideration
– They do not explicitly use coefficient values as in the linear spatial
filters
• Example nonlinear spatial filters
– Median filter: Computes the median gray-level value of the
neighborhood. Used for noise reduction.
– Max filter: Used to find the brightest points in an image
R=max{zk |k=1,2,...,9}
– Min filter: Used to find the dimmest points in an image
R=min{zk |k=1,2,...,9}
18
of
43
Neighbourhood Operations
Neighbourhood operations simply operate
on a larger neighbourhood of pixels than
point operations Origin x

Neighbourhoods are
mostly a rectangle
around a central pixel
(x, y)
Any size rectangle Neighbourhood

and any shape filter

are possible
y Image f (x, y)
19
of
43
Simple Neighbourhood Operations
Some simple neighbourhood operations
include:
Min: Set the pixel value to the minimum in the
neighbourhood
Max: Set the pixel value to the maximum in the
neighbourhood
Median: The median value of a set of numbers is
the midpoint value in that set (e.g. from the set [1,
7, 15, 18, 24] 15 is the median). Sometimes the
median works better than the average
 20
of
Simple Neighbourhood Operations
43 Example

Original Image x Enhanced Image x

123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y y
 21
of
43
The Spatial Filtering Process
Origin x
a b c r s t
d
g
e
h
f
i
* u
x
v
y
w
z
Original Image Filter
Simple 3*3 Pixels
e 3*3 Filter
Neighbourhood
eprocessed = v*e +
r*a + s*b + t*c +
u*d + w*f +
y Image f (x, y) x*g + y*h + z*i

The above is repeated for every pixel in the

original image to generate the filtered image
 22
of
43
Spatial Filtering: Equation Form
a b

  w(s, t ) f ( x  s, y  t )
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

g ( x, y ) 
s   at   b

Filtering can be given

in equation form as
shown above
Notations are based
on the image shown
to the left
23
of
43
Smoothing Spatial Filters
• The shape of the impulse response needed to implement a lowpass
(smoothing) filter indicates the filter should have all positive coefficients
• For a 3x3 mask, the simplest arrangement is to have all the coefficient
values equal to one (neighborhood averaging)
– The response would be the sum of all gray levels for the nine
pixels in the mask
– This could cause the value of R to be out of the valid gray-level
range
• The solution is to scale the result by dividing by 9
24
of
43
Smoothing Spatial Filters
One of the simplest spatial filtering
operations we can perform is a smoothing
operation
Simply average all of the pixels in a
neighbourhood around a central value
Especially useful
1/ 1/ 1/
in removing noise 9 9 9
from images Simple
1/ 1/ 1/
Also useful for 9 9 9 averaging
highlighting gross filter
1/ 1/ 1/
detail 9 9 9
 25
of
43
Smoothing Spatial Filtering
Origin x
104 100 108 1/ 1/ 1/
9 9 9

99 106 98

95 90 85
* 1/

1/
9
1/

1/
9
1/

1/
9

9 9 9

1/ 100
104
9
1/ 108
9
1/
9
Original Image Filter
Simple 3*3 199
/9 1106
/9 198
/9
3*3 Smoothing Pixels
Neighbourhood 195
/9 190
/9 185
/9
Filter
e = 1/9*106 +
1/ *104 + 1/ *100 + 1/ *108 +
9 9 9
1/ *99 + 1/ *98 +
9 9
y Image f (x, y) 1/ *95 + 1/ *90 + 1/ *85
9 9 9
= 98.3333
The above is repeated for every pixel in the
original image to generate the smoothed image
 26
of
43
Image Smoothing Example
The image at the top left
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

is an original image of
size 500*500 pixels
The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
3, 5, 9, 15 and 35
Notice how detail begins
to disappear
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
27
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
28
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
29
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
30
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
31
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
32
Image Smoothing Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
33
Image Smoothing Example
34
of
43
Weighted Smoothing Filters
More effective smoothing filters can be
generated by allowing different pixels in the
neighbourhood different weights in the
averaging function
1/ 2/ 1/
Pixels closer to the 16 16 16
central pixel are more
2/ 4/ 2/
important 16 16 16

Often referred to as a
1/ 2/ 1/
weighted averaging 16 16 16

Weighted
averaging filter
 35
of
43
Another Smoothing Example
By smoothing the original image we get rid
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of lots of the finer detail which leaves only

the gross features for thresholding

Original Image Smoothed Image Thresholded Image

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
36
Another Smoothing Example
37
of
43
Smoothing Filters
• One problem with the lowpass filter is it blurs edges and other sharp
details
• If the intent is to achieve noise reduction, one approach can be to use
median filtering
– The value of each pixel is replaced by the median pixel value in
the neighborhood (as opposed to the average)
– Particularly effective when the noise consists of strong, spike like
components and edge sharpness is to be preserved
• The median m of a set of values is such that half of the values are
greater than m and half are less than m
• To implement, sort the pixel values in the neighborhood, choose the
median and assign this value to the pixel of interest
• Forces pixels with distinct intensities to be more like their neighbors
 38
of
Averaging Filter Vs. Median Filter
43 Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Original Image Image After Image After

With Noise Averaging Filter Median Filter

Filtering is often used to remove noise from

images
Sometimes a median filter works better than
an averaging filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
39

Example
Averaging Filter Vs. Median Filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
40

Example
Averaging Filter Vs. Median Filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
41

Example
Averaging Filter Vs. Median Filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
42

Example
Averaging Filter Vs. Median Filter
43
of
43
Sharpening Filters (High Pass)
• The shape of the impulse response needed to implement a
high-pass (sharpening) filter indicates the filter should have
positive coefficients near its center and negative coefficients in
the outer periphery
• For a 3x3 mask, the simplest arrangement is to have the
center coefficient positive and all others negative
44
of
43
Sharpening Filters (High Pass)
• Note the sum of the coefficients is zero
– When the mask is over a constant or slowly varying region the
output is zero or very small
• This filter eliminates the zero-frequency term
• Eliminating this term reduces the average gray-level value in
the image to zero (will reduce the global contrast of the image)
• Result will be a some what edge-enhanced image over a dark
background
• Reducing the average gray-level value to zero implies some negative
gray levels
– The output should be scaled back into an appropriate range
[0, L-1]
45
of
43
Sharpening Filters (High Pass)
46
of
43
Sharpening Filters (High-boost)
• A high-pass filter may be computed as:
High-pass = Original - Lowpass
• Multiplying the original by an amplification factor yields a high-boost or
high-frequency-emphasis filter
High-boost = A(Original) − Lowpass
= ( A − 1)(Original ) + Original − Lowpass
= ( A − 1)(Original ) + High-pass
– If A>1, part of the original image is added to the high-pass result
(partially restoring low frequency components)
– Result looks more like the original image with a relative degree of
edge enhancement that depends on the value of A
– May be implemented with the center coefficient value w=9A-1
(A≥1)
47
of
43
Sharpening Filters (High-boost)
48
of
Simple Neighbourhood Operations
43 Example

x
123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y
49
of
43
Strange Things Happen At The Edges!

At the edges of an image we are missing

pixels to form a neighbourhood
Origin x
e e

e e e
y Image f (x, y)
50
of
Strange Things Happen At The Edges!
43 (cont…)
There are a few approaches to dealing with
missing edge pixels:
Omit missing pixels
• Only works with some filters
• Can add extra code and slow down processing
Pad the image
• Typically with either all white or all black pixels
Replicate border pixels
Truncate the image
Allow pixels wrap around the image
• Can cause some strange image artefacts
51
of
Simple Neighbourhood Operations
43 Example

x
123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y
 52
of
Strange Things Happen At The Edges!
43 (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Filtered Image:
Zero Padding

Original Filtered Image:

Image Replicate Edge Pixels

Filtered Image:
Wrap Around Edge Pixels
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
53
Strange Things Happen At The Edges!
(cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
54
Strange Things Happen At The Edges!
(cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
43
55
Strange Things Happen At The Edges!
(cont…)
56
of
43
Correlation & Convolution
The filtering we have been talking about so
far is referred to as correlation with the filter
itself referred to as the correlation kernel
Convolution is a similar operation, with just
one subtle difference
a b c r s t eprocessed = v*e +
z*a + y*b + x*c +
d
f
e
g h
e
* u
x
v
y
w
z
w*d + u*e +
t*f + s*g + r*h
Original Image Filter
Pixels

For symmetric filters it makes no difference