Contents

COMPUTER VISION
This lecture will cover:
Basic techniques in

Computer Vision and Image Processing, [email protected]. 2024

Chapter 2: 1.1 Correlation and convolution
image processing
1.2 Neighbourhood operations
Basic Spatial Domain Filters
1.3 What is spatial filtering?
ThS: TRẦN VĂN HÙNG
Email: [email protected] 1.4 Smoothing operations
E-Learning: Trần Văn Hùng
1.5 What happens at the edges?
Company ocw.fet.iuh.edu.vn
LOGO
Company Logo

Correlation & Convolution Correlation & Convolution

The above is repeated for every pixel in the original image to The filtering we have been talking about so far is referred
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

generate the filtered image to as correlation with the filter itself referred to as the
Correlation correlation kernel Convolution is a similar operation, with just
a b c r s t


one subtle difference
d e f u v w
 v*e + r*a + s*b +
t*c + u*d + w*f + x*g +
 v*e + z*a + y*b + x*c + w*d + u*f + t*g + s*h + r*i
g h i x y z
y*h + z*i Correlation Convolution
Original image pixels Filter
a b c r s t a b c z y x
d

g h
e f

i
 u

x y
v w

z
d

g h
e f

i
 w

t
v
s
u

Original Original
Filter Filter rotate 180o
image pixels image pixels

For symmetric filters it makes no difference

 Basic Operations of Image
Spatial Filtering: Equation Form Enhancement


Spatial domain enhancement methods can be
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

 ,  ,   + , +  generalized as
  Pixel Operations
o g(x,y)= T [f(x,y)]
f(x,y): input image
Filtering can be given in
g(x,y): processed (output) image
equation form as shown
above Notations are based T[*]: an operator on f (or a set of input
on the image shown to the images), defined over neighborhood of (x,y)
left Neighborhood about (x,y):
a square or rectangular sub-
image area centered at (x,y)

Simple Neighbourhood Operations:

Example
Smoothing Spatial Filters

Spatial Filters
Pixel Operations
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

One of the simplest spatial filtering operations
we can perform is a smoothing operation

– Simply average all of the pixels in a neighbourhood

   

Simple averaging filter

around a central value
     ,

– Especially useful in removing   

noise from images    

     ,

– Also useful for highlighting   

   
gross detail
  

  
 ,   ,   

, ∈
 Smoothing Spatial Filtering Image Smoothing Example

The image at the top left is an

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

original image of size 500*500
pixels
The subsequent images show
the image after filtering with an
averaging filter of increasing
sizes: 3, 5, 9, 15 and 35
Notice how detail begins to
The above is repeated for every pixel in the original
image to generate the smoothed image disappear

Weighted Smoothing Filters Another Smoothing Example

More effective smoothing filters can be generated by By smoothing the original image we get rid of lots of
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024 the finer detail which leaves only the gross features
allowing different pixels in the neighbourhood
for thresholding
different weights in the averaging function
Weighted averaging filter

   
– Pixels closer to the central     

  
pixel are more important
   
– Often referred to as a     

  
weighted averaging
   
     Original Image Thresholded Image
 Smoothed Image
  
 Averaging Filter vs. Median Filter Strange Things Happen At The
Example Edges!

Filtering is often used to remove noise from images At the edges of an image we are missing pixels
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Sometimes a median filter works better than an to form a neighbourhood
averaging filter x
Origin e e

e e e
Original Image Image After Image After
With Noise Averaging Filter Median Filter y Image f (x, y)

Strange Things Happen At The Strange Things Happen At The

Edges! Edges!

There are a few approaches to dealing with missing

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Filtered Image:
edge pixels: Zero padding
o Omit missing pixels
– Only works with some filters
– Can add extra code and slow down processing
Original Filtered Image:
o Pad the image Image Replicate edge pixels
– Typically with either all white or all black pixels
o Replicate border pixels
o Truncate the image Filtered Image:
Wrap around edge pixels
o Allow pixels wrap around the image
– Can cause some strange image artefacts
 Neighbourhood Operations Neighbourhood Operations

 Neighbourhood operations simply operate on a  Example: A 4x4 grayscale image is given by 5 6 7 8

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

larger neighbourhood of pixels than point 0 6 8 8
operations
impulse? 5 6 15 8
 Neighbourhoods are mostly a rectangle around a
central pixel Any size rectangle and any shape filter 5 6 7 8
x a. Filter the image with a 3x3 median filter, after zero-
are possible Origin
padding
0 0 0 0 0 0

5 6 7 8 0 5 6 7 8 0 0 5 6 0
0 6 8 8 0 0 6 8 8 0 Median 5 6 8 8
zero-
filtering
5 6 15 8 padding 0 5 6 15 8 0 5 6 8 8
5 6 7 8 0 5 6 7 8 0 0 5 6 0
0 0 0 0 0 0

Neighbourhood Operations Simple Neighbourhood Operations

 A 4x4 grayscale image is given by 5 6 7 8  Some simple neighbourhood operations

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

0 6 8 8
include:
5 6 15 8
– Min: Set the pixel value to the minimum in
impulse?
the neighbourhood
5 6 7 8
b. Filter the image with a 3x3 median filter, after replicate- – Max: Set the pixel value to the maximum in
padding at the image borders
the neighbourhood
5 5 6 7 8 8
– Median: The median value of a set of
5 6 7 8 5 5 6 7 8 8 5 6 7 8
numbers is the midpoint value in that set (e.g. from the set
5 6 8 8
0 6 8 8 replicate- 0 0 6 8 8 8 Median [1, 7, 15, 18, 24] 15 is the median). Sometimes the median
padding 5 filtering
5 6 15 8 5 6 15 8 8 5 6 8 8 works better than the average
5 6 7 8 5 5 6 7 8 8 5 6 7 8 – Midpoint filter
5 5 6 7 8 8 Impulse cleaned! – Alpha trimmed mean filter
impulse?
 Order Statistics Filters Median Filter

Spatial filters that are based on ordering the pixel Median Filter:

Computer Vision and Image Processing, [email protected]. 2024

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

 ,  
!  , 
values that make up the neighbourhood operated , ∈

on by the filter
• Excellent at noise removal, without the
Useful spatial filters include smoothing effects that can occur with other
 Median filter smoothing filters.
 Max and min filter
 Midpoint filter • Particularly good when salt and pepper noise is
 Alpha trimmed mean filter present

Noise Corruption Example Max and Min Filter

Max Filter:
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Original Image x Filtered Image x
54 52 57 55 56 52 51  ,    , 
, ∈
50 49 51 50 52 53 58

51 204 52 52 0 57 60 Min Filter:

48 50 51 49 53 59 63  ,    , 
, ∈
49 51 52 55 58 64 67

50 54 57 60 63 67 70 Max filter is good for pepper noise and min is good

51 55 59 62 65 69 72 for salt noise
Image f (x, y) Image f (x, y)
y y
 Noise Corruption Example Midpoint Filter

Midpoint Filter:
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Original Image x Filtered Image x
54 52 57 55 56 52 51 
 ,    ,  +   , 
50 49 51 50 52 53 58  , ∈ , ∈
51 204 52 52 0 57 60

48 50 51 49 53 59 63
Good for random Gaussian and uniform noise
49 51 52 55 58 64 67

50 54 57 60 63 67 70

51 55 59 62 65 69 72
Image f (x, y) Image f (x, y)
y y

Noise Corruption Example Alpha-Trimmed Mean Filter

Alpha-Trimmed Mean Filter:

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Original Image x Filtered Image x
54 52 57 55 56 52 51 
 ,  & , 
50 49 51 50 52 53 58  − "
, ∈
51 204 52 52 0 57 60

 We can delete the " /2 lowest and " /2 highest grey

48 50 51 49 53 59 63
49 51 52 55 58 64 67 levels
50 54 57 60 63 67 70  So gr(s, t) represents the remaining mn – " pixels
51 55 59 62 65 69 72  $ can range from 0 to mn-1
Image f (x, y) Image f (x, y)  If " =0, the filter is reduced to arithmetic mean
y y
 If " =mn-1, the filter become median filter
 Alpha-Trimmed Mean Filter Alpha-Trimmed Mean Filter

Alpha-Trimmed Mean Filter:

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Alpha-Trimmed Mean Filter: 0
Original Image x $2
0
 For other values, the filter is useful in situation involving 10 20 10 15 5 12 15 Trim the data by 2 (1
5 from the top 1 from
multiple types of noise 5 10 13 5 0 15 10 the bottom.)
8
20 11 20 15 5 10 0
o Combination of salt and pepper and Gasussian noise 10
10 8 0 0 10 20 15 Sort intensity
11
10 20 15 0 10 12 15 Output = average
13 intensity of the
10 20 15 12 10 15 20 remaining data = 8.85
15
10 15 20 10 15 12 10
Image f (x, y) 20
y

Noise Corruption Example Kuwahara Filter

 Midpoint Filter: Kuwahara Filter:

 Alpha-Trimmed Mean Filter $=4
Computer Vision and Image Processing, [email protected]. 2024

Original Image x Filtered Image x Computer Vision and Image Processing, [email protected]. 2024  The contour plays a very important role in the process of
54 52 57 55 56 52 51 image analysis
50 49 51 50 52 53 58
 it is very important to ensure that the image smoothing
51 204 52 52 0 57 60

48 50 51 49 53 59 63
does not affect the sharpness of contours
49 51 52 55 58 64 67  The Kuwahara filter is an example of a filter which meets
50 54 57 60 63 67 70
these requirements
51 55 59 62 65 69 72
Image f (x, y) Image f (x, y)
 This filter can be constructed for any window size
y y
 Kuwahara Filter Kuwahara Filter

Kuwahara Filter: Kuwahara Filter:

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

 The window of size j=k=2n+1
 Then, for each of these areas, the values of average (
 The filter window should be divided into four areas
 Let us denote them as θk where k ∈ {0, 1, 2, 3} and variance )( are calculated.

The resulting value of

area 1  The average and variance are calculated according to the
The center formula:
the center pixel is the pixel
area 2
average value of the 
o The average: (  ∑ , ∈.( ,( , )
*(
area for which the
area 3

variance was the o The variance: )(  ∑ , ∈.( ,( , − ( 
*(
smallest. area 4

Kuwahara Filter Kuwahara Filter

Kuwahara Filter: Kuwahara Filter:

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Where: R2 = {P2, P3, P5, P6}
R4 = {P5, P6, P8, P9}
R1 = {P1, P2, P4, P5}
 k ∈ {0, 1, 2, 3}, R3 = {P 4, P 5, P 7, P 8}

 f : is the source image function,

 f (x, y) : is the value of the pixel at coordinates (x, y),
 0 : is a function calculating the value of a particular pixel,
1
 : is the number of pixels in the current area,
231 4 231 
<  = + = + = + =
 n : is the value obtained directly from the filter window 

Finally, we compare the variance of all four areas and look )<  = + = + = + = − ( 
for an index of the area for which the variance is the 
smallest. )  min )(
)  min )( 8∈ 1,9,:,;
8∈ 1,9,:,;
 Kuwahara Filter Noise Removal Examples

Kuwahara Filter: Example

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
0 8 8 5 1 30 3 3 3 0 Image Result of 1
1 4 4 5 1 30 3 3 3 1 ? ? Corrupted Pass With A
2 4 4 5 1 30 30 30 30 2
3 1 1 1 1 30 30 30 30 3 ? By Salt And 3*3 Median
4 31 31 7 7 30 30 30 30 4 Pepper Noise Filter
5 31 31 7 7 30 5 5 5 5
6 2 2 3 3 30 5 5 5 6
7 1 2 3 3 30 5 5 5 7
 Result of 2 Result of 3
<  = + = + = + =
 Passes With Passes With

)<  = + = + = + = − (  A 3*3 Median A 3*3 Median
 Filter Filter
)  min )(
8∈ 1,9,:,;

Noise Removal Examples Noise Removal Examples

Image
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Image Further
Image Corrupted Corrupted
Image
Corrupted By Uniform By Salt and
Corrupted
By Salt Noise Pepper Noise
By Pepper
Noise Noise
Filtered By Filtered By
5*5 Arithmetic 5*5 Geometric
Result Of Result Of Mean Filter Mean Filter
Filtering Filtering
Above Above
With A 3*3 With A 3*3 Filtered By Filtered By
Max Filter Min Filter 5*5 Median 5*5 Alpha-Trimmed
Filter Mean Filter
 Periodic Noise Band Reject Filters

 Typically arises due to Removing periodic noise form an image involves

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

electrical or electromagnetic removing a particular range of frequencies from that
interference image
 Gives rise to regular noise Band reject filters can be used for this purpose
patterns in an image An ideal band reject filter is given as follows:
 Frequency domain techniques  W
in the Fourier domain are  1 if D (u , v )  D0 
2
most effective at removing  W W
periodic noise H (u , v)  0 if D0   D(u , v)  D0 
 2 2
1 if D(u , v)  D0  W
 2

Band Reject Filters Band Reject Filter Example

The ideal band reject filter is shown below, along

Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

Image corrupted by Fourier spectrum of
sinusoidal noise corrupted image
with Butterworth and Gaussian versions of the filter

Ideal Band Butterworth Gaussian

Reject Filter Band Reject Band Reject
Filter (of order 1) Filter
Butterworth band Filtered image
reject filter
 Adaptive Filters Adaptive Median Filtering
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

 The filters discussed so far are applied to an entire  The median filter performs relatively well on impulse noise
as long as the spatial density of the impulse noise is not
image without any regard for how image
large
characteristics vary from one point to another
 The adaptive median filter can handle much more spatially
 The behaviour of adaptive filters changes dense impulse noise, and also performs some smoothing for
depending on the characteristics of the image non-impulse noise

inside the filter region  The key insight in the adaptive median filter is that the filter
size changes depending on the characteristics of the image
 We will take a look at the adaptive median filter

Adaptive Median Filtering Adaptive Median Filtering

Remember that filtering looks at each original pixel Level A: A1 = zmed – zmin
Computer Vision and Image Processing, [email protected]. 2024

image in turn and generates a new filtered pixel Computer Vision and Image Processing, [email protected]. 2024 A2 = zmed – zmax
First examine the following notation: If A1 > 0 and A2 < 0, Go to level B
Else increase the window size
 zmin = minimum grey level in Sxy If window size ≤ repeat Smax level A
 zmax = maximum grey level in Sxy Else output zmed
Level B: B1 = zxy – zmin
 zmed = median of grey levels in Sxy B2 = zxy – zmax
 zxy = grey level at coordinates (x, y) If B1 > 0 and B2 < 0, output zxy
Else output zmed
 Smax =maximum allowed size of Sxy
 Adaptive Median Filtering Adaptive Filtering Example
Computer Vision and Image Processing, [email protected]. 2024

Computer Vision and Image Processing, [email protected]. 2024

The key to understanding the algorithm is to remember
that the adaptive median filter has three purposes:

 Remove impulse noise

 Provide smoothing of other noise

 Reduce distortion
Image corrupted by salt Result of filtering with a 7 Result of adaptive
and pepper noise with * 7 median filter median filtering with i = 7
probabilities Pa = Pb=0.25

Summary

 In this lecture we have looked at the idea of spatial filtering

Computer Vision and Image Processing, [email protected]. 2024

and in particular:
– Neighbourhood operations
– The filtering process
– Smoothing filters
– Dealing with problems at image edges when using
filtering
– Correlation and convolution [email protected]

 Next time we will looking at sharpening filters and more on

filtering and image enhancement
Company
LOGO