Spatial Filtering

Dr. Navjot Singh

Image and Video Processing
Acknowledgements
⚫ Gonzalez, Rafael C. Digital image processing. Pearson, 4th
edition, 2018.
⚫ Jain, Anil K. Fundamentals of digital image processing. Prentice-
Hall, Inc., 1989.
⚫ Digital Image Processing course by Brian Mac Namee, Dublin
Institute of Technology
⚫ Digital Image Processing course by Christophoros Nikou,
University of Ioannina

2
Contents
In this lecture we will look at spatial filtering techniques:
⚫ Neighbourhood operations
⚫ What is spatial filtering?
⚫ Smoothing operations
⚫ What happens at the edges?
⚫ Correlation and convolution
⚫ Sharpening filters
⚫ Combining filtering techniques

3
Neighbourhood Operations
⚫ Neighbourhood operations simply Origin x

operate on a larger
neighbourhood of pixels than
point operations
⚫ Neighbourhoods are mostly a Neighbourhood
(x, y)

rectangle around a central pixel

⚫ Any size rectangle and any shape
filter are possible
y Image f (x, y)

4
Simple Neighbourhood Operations
Some simple neighbourhood operations include:
⚫ Min: Set the pixel value to the minimum in the neighbourhood
⚫ Max: Set the pixel value to the maximum in the neighbourhood
⚫ Median: The median value of a set of numbers is the midpoint value
in that set (e.g. from the set [1, 7, 15, 18, 24] 15 is the median).
Sometimes the median works better than the average

5
Simple Neighbourhood Operations Example

Original Image x Enhanced Image x

123 127 128 119 115 130

140 145 148 153 167 172

133 154 183 192 194 191

194 199 207 210 198 195

164 170 175 162 173 151

y y

6
The Spatial Filtering Process
Origin x
a b c r s t
d
g
e
h
f
i
* u
x
v
y
w
z
Original Image Filter
Simple 3*3 Pixels
e 3*3 Filter
Neighbourhood
eprocessed = v*e +
r*a + s*b + t*c +
u*d + w*f +
y Image f (x, y) x*g + y*h + z*i
The above is repeated for every pixel in the
original image to generate the filtered image 7
 a b
g ( x, y ) =   w(s, t ) f ( x + s, y + t )
s = − at = − b

Filtering can be given in

equation form as shown
above
Notations are based on the
image shown to the left
8

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 Filtering process

Source: Google Images

Spatial Correlation and Convolution

Spatial Correlation

Spatial Convolution

For symmetric filters it makes no difference.

10
Spatial Convolution and Correlation

11
Spatial Convolution and Correlation

f 0 0 0 1 0 0 0 0

w 1 2 4 2 8

Find result of convolution and correlation.

12
13
14
Spatial Convolution and Correlation:
Properties

15
Spatial filters

Smoothing (lowpass) Sharpening (high

Spatial Filters pass) Spatial Filters

Bandreject Filters Bandpass Filters

16
Smoothing Spatial Filters
⚫ Smoothing filters are used
⚫ Blurring
⚫ Noise reduction

⚫ Blurring is used in removal of small details and bridging of

small gaps in lines or curves

⚫ Smoothing spatial filters include linear filters and nonlinear

filters.
17
Smoothing Spatial Filters
One of the simplest spatial filtering operations
1/ 1/ 1/
we can perform is a smoothing operation 9 9 9

⚫ Simply average all of the pixels in a neighbourhood 1/ 1/ 1/

around a central value 9 9 9

⚫ Especially useful in removing noise from images 1/ 1/ 1/

9 9 9
⚫ Also useful for highlighting gross detail
Simple Averaging Filter
or Box Filter

18
Smoothing Spatial Filtering
Origin x
104 100 108 1/ 1/ 1/
9 9 9

99 106 98

95 90 85
* 1/

1/
9
1/

1/
9
1/

1/
9

9 9 9

1/ 100
104
9
1/ 108
9
1/
9
Original Image Filter
Simple 3*3 1/ 1/ 1/
3*3 Smoothing Pixels
999 106
9 989
Neighbourhood 195
/9 190
/9 185
/9
Filter
e = 1/9*106 +
1/ *104 + 1/ *100 + 1/ *108 +
9 9 9
1/ *99 + 1/ *98 +
9 9
y Image f (x, y) 1/ *95 + 1/ *90 + 1/ *85
9 9 9
= 98.3333
The above is repeated for every pixel in the original image to
generate the smoothed image. 19
 20

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
Weighted Smoothing Filters
More effective smoothing filters can be generated
by allowing different pixels in the neighbourhood 1/
16
2/
16
1/
16
different weights in the averaging function
2/ 4/ 2/
⚫ Pixels closer to the central pixel are more important 16 16 16
⚫ Often referred to as a weighted averaging
1/ 2/ 1/
16 16 16

Weighted
averaging filter

21
 Low-pass Gaussian filter kernels

22

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 23

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 24

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 25

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 26

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 27

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 Importance of understanding relationship between kernel size and the size of objects
28

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 By smoothing the original image we get rid of lots of the finer
detail which leaves only the gross features for thresholding 29

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
Order-statistic (Nonlinear) Filters
⚫ Nonlinear
⚫ Based on ordering (ranking) the pixels contained in the
filter mask
⚫ Replacing the value of the center pixel with the value
determined by the ranking result
⚫ E.g., median filter, max filter, min filter

30
Median Filter: How it works
fˆ ( x, y) = median{g ( s, t )}
( s ,t )S xy

A median filter is good for removing impulse, isolated noise

Salt noise
Pepper noise
Median

Moving
Degraded Window
image
Sorted
Salt noise Array
Pepper noise
Filter output

Normally, impulse noise has high magnitude and is

solated. When we sort pixels in the moving window,
noise pixels are usually at the ends of the array.

31
Averaging Filter Vs. Median Filter Example

Original Image Image After Image After

With Noise Averaging Filter Median Filter

Filtering is often used to remove noise from images

Sometimes a median filter works better than an averaging filter
32
Max and Min Filter
Max Filter:
fˆ ( x, y) = max {g ( s, t )}
( s ,t )S xy

Min Filter:
fˆ ( x, y) = min {g (s, t )}
( s ,t )S xy

Max filter is good for pepper noise and Min filter is good for salt
noise.
33
bipolar 3x3
Noise Median
Pa = 0.1 Filter
Pb = 0.1 Pass 1

3x3 3x3
Median Median
Filter Filter 34

Pass 2 Pass 3
Pepper Salt
noise noise

Max Min
filter filter

35
Spatial smoothing and image approximation
Spatial smoothing may be viewed as a process for estimating
the value of a pixel from its neighbours.

What is the value that “best” approximates the intensity of a

given pixel given the intensities of its neighbours?

We have to define “best” by establishing a criterion.

36
Spatial smoothing and image approximation
(cont...)

A standard criterion is the the sum of squares differences.

N
N 2
E =   x(i ) − m  m = arg min   x(i ) − m 
2

i =1 m  i =1 

E N N N
= 0  −2 ( x(i ) − m ) = 0   x(i ) = m
m i =1 i =1 i =1

N
  x(i ) =Nm  m = 1
N

i =1 N
 x(i)
i =1
The average value
37
Spatial smoothing and image approximation
(cont...)

Another criterion is the the sum of absolute differences.

N
N 
E =  x(i ) − m  m = arg min  x(i ) − m 
i =1 m  i =1 
 1 x0
E N

= 0  − sgn ( x(i ) − m ) = 0, sign( x) =  0 x = 0
m i =1 −1 x  0

There must be equal in quantity positive and negative values.

m = median{x(i )}
38
Spatial smoothing and image approximation
(cont...)

– The median filter is non linear:

median{x + y}  median{x} + median{ y}
– It works well for impulse noise (e.g. salt and pepper).
– It requires sorting of the image values.
– It preserves the edges better than an average filter in the case of
impulse noise.
– It is robust to impulse noise at 50%.

39
Spatial smoothing and image approximation
(cont...)
Example x[n] 1 1 1 1 1 2 2 2 2 2

edge
Impulse
x[n] 1 3 1 1 1 2 3 2 2 3
noise

Median
(N=3) x[n] - 1 1 1 1 2 2 2 2 -

Average x[n] - 1.7 1.7 1 1.3 2 2.3 2.3 2.2 -

(N=3)
40
The edge is smoothed
Strange Things Happen At The Edges!
At the edges of an image we are missing pixels to form a
neighbourhood
Origin x
e e

e e e
41
y Image f (x, y)
Strange Things Happen At The Edges!
(cont…)
There are a few approaches to dealing with missing edge pixels:
⚫ Omit missing pixels
⚫ Only works with some filters
⚫ Can add extra code and slow down processing
⚫ Pad the image
⚫ Typically with either all white or all black pixels
⚫ Replicate border pixels
⚫ Truncate the image
⚫ Allow pixels wrap around the image
⚫ Can cause some strange image artefacts
42
Strange Things Happen At The Edges! (cont…)

Filtered Image:
Zero Padding

Original Filtered Image:

Image Replicate Edge Pixels

Filtered Image:
Wrap Around Edge Pixels
Effect of Low Pass Filtering on White Noise
Let f be an observed instance of the image f0 corrupted by
noise w:
f = f0 + w

with noise samples having mean value E[w(n)]=0 and being

uncorrelated with respect to location:

 2 , m = n
E[ w(m) w(n)] = 
 0, m  n 44
Effect of Low Pass Filtering on White
Noise (cont...)
Applying a low pass filter h (e.g. an average filter) by convolution
to the degraded image:

g = h * f = h *( f 0 + w) = h * f 0 + h * w
The expected value of the output is:
E[ g ] = E[h * f 0 ] + E[h * w] = h * f0 + h * E[w]
= h * f0 + h *0 = h * f0
The noise is removed in average. 45
Effect of Low Pass Filtering on White
Noise (cont...)
What happens to the standard deviation of g?
Let g = h * f0 + h * w = f0 + w
where the bar represents filtered versions of the signals, then

 = E[ g ] − ( E[ g ]) = [( f0 + w)2 ] − ( f0 )2
2 2 2
g

= [( f0 ) + (w) + 2 f0 w] − ( f0 )
2 2 2

= E[(w)2 ] + 2E[ f0 ]E[w] = E[( w) 2 ]

46
Effect of Low Pass Filtering on White
Noise (cont...)
Considering that h is an average filter, we have at pixel n:
1
w(n) = (h * w)(n) =
N

k ( n )
w(k )
Therefore,

 1 
2

E[( w(n)) ] = E 
2

 N
 w(k )  

k ( n ) 

47
Effect of Low Pass Filtering on White
Noise (cont...)
 1 
2

E 
 N
 w(k )  

k ( n ) 

1
= 2    
2
E w( k )
N k ( n )
  Sum of squares

2
+ 2   E  w(n − l )w(n − m)
N l ( n ) m ( n )
m l
Cross products 48
Effect of Low Pass Filtering on White
Noise (cont...)
Sum of squares

1 1
 E w(k )  = 2 
2 2

N 2
k ( n )
  N k ( n )

Cross products (uncorrelated as m l)

2
+ 2   E  w(n − l )w(n − m) = 0
N l ( n ) m ( n )
m l

49
Effect of Low Pass Filtering on White
Noise (cont...)

Finally, substituting the partial results:

 1 
2
 1
 g = E 
2

 N
 w(k )  = 2
 N
 2

k ( n )  k ( n )

1  2
= 2 N =
2

N N
The effect of the noise is reduced.
This processing is not optimal as it also smoothes image edges. 50
Sharpening Spatial Filters
Previously we have looked at smoothing filters which remove
fine detail
Sharpening spatial filters seek to highlight fine detail
⚫ Remove blurring from images
⚫ Highlight edges
Sharpening filters are based on spatial differentiation

51
1st Derivative
The formula for the 1st derivative of a function is as follows:
f
= f ( x + 1) − f ( x)
x
It’s just the difference between subsequent values and
measures the rate of change of the function

52
1st Derivative (cont.)
• The gradient of an image:

The gradient points in the direction of most rapid increase in intensity.

Gradient direction

The edge strength is given by the gradient magnitude

53
1st Derivative (cont.)

f

f f
x y
54
2nd Derivative
The formula for the 2nd derivative of a function is as follows:
 f
2
= f ( x + 1) + f ( x − 1) − 2 f ( x)
 x
2

Simply takes into account the values both before and after the
current value

55
Step and Ramp
⚫ Step edge
⚫ The image intensity abruptly changes from one value to one side of
the discontinuity to a different value on the opposite side.
⚫ Ramp edge
⚫ A step edge where the intensity change is not instantaneous but
occur over a finite distance.

56
Sharpening Spatial Filters: Foundation

Definition for derivatives

First Derivative Second Derivative
Constant Intensity Areas ZERO ZERO
Onset of intensity step or Non-zero Non-zero
ramp + at the end
Along intensity ramp Non-zero ZERO

57
 58

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
Using Second Derivatives For Image
Enhancement

Edges in images are often ramp-like transitions

⚫ 1st derivative is constant and produces thick edges
⚫ 2nd derivative zero crosses the edge (double response at the onset
and end with opposite signs)
A common sharpening filter is the Laplacian
⚫ Isotropic
⚫ One of the simplest sharpening filters
⚫ We will look at a digital implementation

59
The Laplacian
The Laplacian is defined as follows:
 2
f  2
f
 f = 2 + 2
2

 x  y
where the partial 1st order derivative in the x direction is defined
as follows:
2 f
= f ( x + 1, y ) + f ( x − 1, y ) − 2 f ( x, y )
 x
2

and in the y direction as follows:

 f
2
= f ( x, y + 1) + f ( x, y − 1) − 2 f ( x, y)
 y
60
2
 The Laplacian (cont…)

 f = [ f ( x + 1, y) + f ( x − 1, y) + f ( x, y + 1) + f ( x, y − 1)] − 4 f ( x, y)
2

0 1 0

1 -4 1

0 1 0

61
62
The Laplacian (cont…)
Applying the Laplacian to an image we get a new image that
highlights edges and other discontinuities

Original Laplacian Laplacian

Image Filtered Image Filtered Image
Scaled for Display
63
But That Is Not Very Enhanced!

The result of a Laplacian filtering is not an

enhanced image
We have to do more work in order to get our final
image
Subtract the Laplacian result from the original
image to generate our final sharpened enhanced
Laplacian
image Filtered Image
Scaled for Display
g ( x, y) = f ( x, y) −  f
2

64
Laplacian Image Enhancement

- =
Original Laplacian Sharpened
Image Filtered Image Image

In the final sharpened image edges and fine detail are much
more obvious
65
Simplified Image Enhancement
The entire enhancement can be combined into a single filtering
operation
g ( x, y) = f ( x, y) −  f 2

= f ( x, y ) − [ f ( x + 1, y ) + f ( x − 1, y )
+ f ( x, y + 1) + f ( x, y − 1)
− 4 f ( x, y )]
= 5 f ( x, y ) − f ( x + 1, y ) − f ( x − 1, y )
− f ( x, y + 1) − f ( x, y − 1) 66
Simplified Image Enhancement (cont…)
This gives us a new filter which does the whole job for us in one
step

0 -1 0

-1 5 -1

0 -1 0

67
 Variants On The Simple Laplacian
There are lots of slightly different versions of the Laplacian that
can be used:
0 1 0 1 1 1
Simple Variant of
1 -4 1 1 -8 1
Laplacian Laplacian
0 1 0 1 1 1

-1 -1 -1

-1 9 -1

-1 -1 -1 68
Unsharp masking and Highboost Filtering
Used by the printing industry
Subtracts an unsharped (smooth) image from the original image
f(x,y).
⚫ Blur the image b(x,y)=Blur{f(x,y)}
⚫ Subtract the blurred image from the original (the result is called the
mask) gmask(x,y)=f(x,y)-b(x,y)
⚫ Add the mask to the original with k non negative masks
g(x,y)=f(x,y)+k gmask(x,y)

69
 Unsharp masking and Highboost Filtering
(cont...)

70

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 Unsharp masking and Highboost Filtering
(cont...)

71

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
1st Derivative Filtering

Implementing 1st derivative filters is difficult in practice

For a function f(x, y) the gradient of f at coordinates (x, y) is given
as the column vector:

 f 
Gx   x 
f =   =  f 
G y   
 y 
72
1st Derivative Filtering (cont…)
The magnitude of this vector is given by:
f = mag (f )

= G +G
2
x
2
y 
1
2

1
 f   f   2
2 2

=   +   
 x   y  
For practical reasons this can be simplified as:

f  Gx + G y 73
1st Derivative Filtering (cont…)

z1 z2 z3
z4 z5 z6
z7 z8 z9

74
 75

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
 Sobel Example

76

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
1st & 2nd Derivatives
Comparing the 1st and 2nd derivatives we can conclude the
following:
⚫ 1st order derivatives generally produce thicker edges (if thresholded
at ramp edges)
⚫ 2nd order derivatives have a stronger response to fine detail e.g. thin
lines
⚫ 1st order derivatives have stronger response to grey level step
⚫ 2nd order derivatives produce a double response at step changes in
grey level (which helps in detecting zero crossings)

77
Combining Spatial Enhancement Methods

Successful image enhancement is

typically not achieved using a single
operation
Rather we combine a range of techniques
in order to achieve a final result
This example will focus on enhancing the
bone scan to the right

78
Combining Spatial Enhancement Methods
(cont…)

(a)
Laplacian filter of
bone scan (a)
(b)
Sharpened version of
bone scan achieved (c)
by subtracting (a)
and (b) Sobel filter of bone
79
scan (a) (d)
Combining Spatial Enhancement Methods
(cont…)
Result of applying a (h)
power-law trans. to
Sharpened image (g)
which is sum of (a)
and (f) (g)
The product of (c)
and (e) which will be (f)
used as a mask
(e)

80
Image (d) smoothed with
a 5*5 averaging filter
Combining Spatial Enhancement Methods
(cont…)
Compare the original and final images

81
 82

Adapted from Gonzalez, Rafael C. Digital image processing. Pearson education India, 2009.
Summary of Spatial Filter Types

83
Summary
In this lecture we have looked at the idea of spatial filtering and
in particular:
⚫ Neighbourhood operations
⚫ The filtering process
⚫ Smoothing filters
⚫ Dealing with problems at image edges when using filtering
⚫ Correlation and convolution
⚫ Sharpening filters
⚫ Combining filtering techniques

84