EENG 860 Special Topics: Digital Image Processing

Lecture 4: Spatial Filtering

Dr. Ahmadreza Baghaie

Department of Electrical and Computer Engineering
New York Institute of Technology

Spring 2020

Readings: Chapter 3 (sections 3.5-3.8), Chapter 4 (section 4.1)

1 / 38
How to read: Phones OFF! Pens ON!
Table of Content

●
Smoothing (Lowpass) Spatial Filters
●
Sharpening (Highpass) Spatial Filters
●
Highpass, Bandreject and Bandpass Filters From Lowpass Filters
●
Combining Spatial Enhancement Methods

2 / 38
Fundamentals of Spatial Filtering

●
Two components in any spatial
filter definition:
– A neighborhood

– An operation defined in the

neighborhood

●
Spatial Filtering:
– Linear spatial filters (e.g.
mean)
– Nonlinear spatial filters (e.g.
median)

3 / 38
Linear Spatial Filtering

●
Sum-of-products operation between an image f and a filter kernel w.
●
The kernel is an array which defines the neighborhood of operation, and its
coefficients determine the nature of the filter. w (−1,−1) w (−1 , 0) w (−1 ,1)
●
A kernel is also called mask, template, and window. w (0 ,−1) w (0, 0) w (0 , 1)
●
Assume a 3-by-3 kernel like the figure.
w(1 ,−1) w(1, 0) w(1 , 1)

●
At any point (x,y), the response, g(x,y), of the filter is the sum-of-products
of the kernel coefficients and the image pixels encompassed by the kernel:
g ( x , y)=w (−1 ,−1) f ( x−1 , y−1)+w (−1,0) f ( x−1 , y )+...
+w (0,0) f ( x , y)+...+w (1,1) f ( x+1 , y+1)
●
As the coordinates (x,y) change, the center of the kernel moves from pixel
to pixel, generating the filtered image g.

4 / 38
Linear Spatial Filtering

●
The center coefficient of the kernel, w(0,0), aligns with the pixel at location
(x,y).
●
A generalized kernel of size (mxn), with m=2a+1 and n=2b+1, where a
and b are non-negative integers, can be applied to the image to create the
filtered image g(x,y) such as:
a b
g ( x , y)= ∑ ∑ w(s ,t ) f ( x+ s , y+t )
s=−a t =−b

where x and y are changed so the center of the kernel goes through every
pixel in image f once.

5 / 38
Correlation vs. Convolution: 1D Case

●
Correlation: (also shown with )
a
(w⊕f )(x)= ∑ w (s) f (x+s)
s=−a

●
Convolution: (also shown with )
a
(w⊗f )(x)= ∑ w (s) f (x−s)
s=−a

●
Function f is a discrete unit impulse.

{
δ ( x− x 0 )= 1
0
if x=x 0
otherwise
●
Pre-rotating the kernel results in an
exact copy of the kernel.
●
Linear spatial filtering and spatial
convolution are synonymous.
6 / 38
Correlation vs. Convolution: 2D Case

●
The same can be said about 2D case.
●
Correlation:
a b
(w⊕f )(x , y )= ∑ ∑ w( s ,t) f ( x+s , y +t)
s=−a t=−b

●
Convolution: a b
(w⊗f )(x , y )= ∑ ∑ w( s ,t) f ( x−s , y−t )
s=−a t=−b

●
Function f is a discrete unit impulse.

{
δ ( x− x 0 , y− y 0 )= 1 if x=x 0 and y= y 0
0 otherwise
●
Pre-rotating the kernel results in an
exact copy of the kernel.
●
Linear spatial filtering and spatial
convolution are synonymous. 7 / 38
Correlation vs. Convolution: Properties

Properties Convolution Correlation

Commutative f ⊗g=g⊗f -
Associative f ⊗(g⊗h)=(f ⊗g)⊗h -
Distributive f ⊗(g+h)=(f ⊗g)+(f ⊗h) f ⊕(g+h)=(f ⊕g)+(f ⊕h)

●
Because of the commutative property of convolution, it is not important
whether the kernel or the image is pre-rotated.
●
Also, performing multistage filtering is possible, if the kernels are
convolved first, and the resulting kernel is applied to the image.

w=w 1⊗w 2⊗...⊗w N

8 / 38
Smoothing Spatial Filters

●
Smoothing (averaging) spatial filters are used for reducing sharp
transitions in intensity.
●
Random noise consists sharp transitions; therefore they reduce noise too.
●
Smoothing prior to re-sampling reduces aliasing (Chapter 4).
●
Smoothing can reduce false contouring.
●
Two types:
– Linear filters:
●
Box Filter Kernels
●
Lowpass Gaussian Filter Kernels
– Nonlinear filters:
●
Order-Statistic Filters

9 / 38
Box Filter Kernels

●
Simple, separable lowpass filter kernel, with
coefficients of identical values in a small
neighborhood.
●
An mxn box filter, a matrix of 1’s, with a
normalizing constant equal to the number of
elements in the matrix.
●
Why normalize the kernel?
– An area of constant intensity will remain the
same after filtering;
– Preventing bias during filtering, sum of the
pixels in the original and filtered images are
the same.

10 / 38
Box Filter Kernels: Example

●
Original image: 1024x1024 pixels.
●
Kernels of size 3x3, 11x11 and
21x21.
●
Gray border as a result of zero-
padding the image to avoid
undefined operations when part of
the kernel is outside the image.

11 / 38
Lowpass Gaussian Filter Kernels

●
Box filters are simple and suitable for quick implementations. Useful when
we want to reduce the effect of smoothing on edges.
●
Box filters are poor approximations of blurring characteristics of lenses.
●
Box filters favor blurring along the perpendicular lines.
●
To avoid the shortcomings, circularly symmetric (isotropic) kernels are
preferred.
●
Gaussian kernels are circularly symmetric and separable:

s 2 +t 2
−
2σ 2
w (s , t)=G (s , t )= Ke

12 / 38
Lowpass Gaussian Filter Kernels

●
To signify the circularly symmetric form of the kernel:
s 2 +t 2
−
2σ 2
w (s , t)=G (s , t )= Ke
r2
−
r=√ (s +t )⇒ w (s ,t )=G (s , t )= Ke
2 2 2σ 2

●
Figure shows values of r for several
kernel sizes:

13 / 38
Lowpass Gaussian Filter Kernels

●
The Gaussian function is a continuous function, and cannot be applied
directly to the images.
●
We need to sample it at integer spatial locations, and make a discrete
kernel.
●
Normalizing the kernel by dividing its coefficients by the sum of the
coefficients completes the process of specifying the kernel.

14 / 38
Gaussian Kernels Properties

●
Since Gaussian kernels are circularly symmetric and separable, sampling
can be done only along a cross section through the center, then convert it
to a 2D kernel by the procedure explained in previous lecture.
●
Values of a Gaussian function at distance larger than 3σ from the mean
are very small, so we can select the size of Gaussian kernel to be:
Gaussian kernel size =⌈6 σ ⌉×⌈6 σ ⌉
with ⌜c⌝ to denote the ceiling of c.
●
The product and convolution of two Gaussian kernels are also Gaussian.

15 / 38
Gaussian Kernels: Example

16 / 38
Image Padding?

●
We saw image zero-padding when computing correlations and
convolutions.
●
Padding methods:
– Zero-padding
– Mirror (or symmetric) padding: values outside the boundary are
obtained by mirror-reflecting the image across the border.
– Replicate padding: values outside the boundary are set equal to the
nearest image border value.

17 / 38
Example: Shading Correction Using Lowpass
Filtering
●
Shading caused by nonuniform illumination, results in erroneous
measurements and degraded performance of automated DIP.
●
Shading correction (flat-field correction) can be done by estimating the
shading of an image using lowpass filtering.
●
Left is a 2048x2048 checkerboard image, with squares of size 128x128.
Middle is the shading estimation resulted from applying Gaussian kernel of
size 512x512, with K=1 and σ=128. Right is the shading corrected image.

18 / 38
Order-Statistic (Nonlinear) Filters

●
Order-statistic filters are nonlinear filters that work based on ordering
(ranking) of the pixels in the region encompassed by the neighborhood.
●
Median Filter replaces the value of the center pixel by the median of the
intensity values defined in the neighborhood of that pixel. Very useful for
reducing salt-and-pepper noise.
●
Median of a set of values is such that half of the values in the set are less
than or equal to the median. When several values are the same, they will
be grouped.

19 / 38
Sharpening (Highpass) Spatial Filters

●
Sharpening highlights transitions in intensity.
●
Averaging (smoothing) is analogous to integration, then sharpening is
analogous to spatial differentiation.
●
Image differentiation enhances edges and other discontinuities (like noise)
and de-emphasizes areas of slowly varying intensities.
●
First-order derivative:
– Must be zero in areas of constant intensity;
– Must be non-zero at the onset of an intensity step or ramp;
– Must be non-zero along intensity ramps.
●
Second-order derivative:
– Must be zero in areas of constant intensity;
– Must be non-zero at the onset and end of an intensity step or ramp;
– Must be zero along intensity ramps. 20 / 38
Sharpening (Highpass) Spatial Filters

●
For digital functions, the derivatives are
defined in terms of differences.
●
First-order derivative of a 1D function in
difference form:

∂f
=f ( x+1)−f (x)
∂x
●
Second-order derivative of a 1D function
in difference form:

∂2 f
2
=f ( x+1)+f (x−1)−2 f ( x)
∂x
●
Partial derivative is used, so the notation
is consistent between 1D and 2D
functions.
21 / 38
Second Order Derivatives for Image Sharpening –
The Laplacian
●
We are interested in isotropic kernels with response independent of the
direction of the intensity discontinuity.
●
The Laplacian of a function with two variables:

2 ∂ 2 f ∂2 f
∇ f= 2+ 2
∂x ∂ y
●
By replacing the two second-derivative terms we have:

∇ 2 f ( x , y )=f ( x+1 , y )+ f ( x−1 , y )+ f ( x , y+1)+ f ( x , y−1)−4 f ( x , y)

22 / 38
Second Order Derivatives for Image Sharpening –
The Laplacian
●
The kernel is isotropic for rotations of increments of 90 degrees.
●
What if we want to incorporate the diagonal directions?
●
What if the negative sign is used for second derivatives? This needs to be
considered when combining a Laplacian-filtered image with another image.

●
Note that the coefficients sum to
zero, so we don’t introduce any
bias in the constant regions.

23 / 38
Second Order Derivatives for Image Sharpening –
The Laplacian
●
Now that we derived the Laplacian of
an image, we can combine it with the
original image to obtain a sharpened
version:
g ( x , y)=f ( x , y)+c [ ∇ 2 f ( x , y)]
where f(x,y) and g(x,y) are the input
and sharpened images respectively. If
the first two kernels is used, then c=-1;
Otherwise, c=1.

24 / 38
First Order Derivatives for Image Sharpening –
The Gradient
●
First derivatives are implemented using the magnitude of the gradient.
●
The gradient of an image f at coordinates (x,y) is defined as a two-

[]
dimensional column vector:
∂f

[]
g
∇ f =grad (f )= x = ∂ x
gy ∂f
∂y
●
The vector points at the direction of the greatest rate of change of function
f at location (x,y).
●
The magnitude of the gradient vector, denoted by M(x,y) is defined as:

M ( x , y)=‖∇ f‖=mag(∇ f )= √(g x + g y )

2 2

which is the value of the rate of change in the direction of the gradient
vector at location (x,y).

25 / 38
First Order Derivatives for Image Sharpening –
The Gradient
●
A more computationally suitable
approximation is done by using:

M ( x , y)≈|g x|+|g y|

●
Similar to Laplacian, we can use
derivatives in their difference form to
create appropriate kernels for
computing the gradients:
– Roberts kernels (cross gradient)
– Sobel kernels

26 / 38
First Order Derivatives for Image Sharpening –
The Gradient: Example
●
Example: detection of defects
– Converting the original image into the gradient image makes the
defects more clear and easier to detect automatically

27 / 38
Unsharp Masking

●
We can subtract an unsharp (smoothed) version of an image from the
original image to create a mask to sharpen the image.
●
The procedure:
– Blur the original image: f ( x , y)⇒ f̄ (x , y )
– Subtract the blurred image from the
g mask ( x , y )=f (x , y)− f̄ ( x , y)
original to create the unsharp mask:
– Add the mask to the original image: g (x , y)=f ( x , y)+k gmask (x , y)

28 / 38
Unsharp Masking

●
When k=1, we have unsharp masking. For k>1, the process is called
highboost filtering.
●
Since we may have negative values, clipping might happen, causing dark
halos around the edges.

29 / 38
Highpass, Bandreject, And Bandpass Filters From
Lowpass Filters
●
Spatial and frequency-domain linear filters are classified into four broad
categories: lowpass, highpass, bandpass and bandreject filters.
●
The last three types can be constructed from lowpass filters.
●
Fourier Transform (FT) is used to go from the spatial to frequency domain.
●
FT decomposes the signal (image) into its consisting frequency
components.
●
Two fundamental properties:
– Convolution in the spatial domain, is equivalent to multiplication in the
frequency domain, and vice versa.
– An impulse function of amplitude A in the spatial domain, is a
constant function of value A in the frequency domain, and vice versa.

30 / 38
Highpass, Bandreject, And Bandpass Filters From
Lowpass Filters
●
Representation of these filter in the spatial vs frequency domain.
Filter Type Spatial Kernel Frequency Kernel
Lowpass lp( x , y) LP (u , v)
Highpass hp(x , y)=δ (x , y)−lp( x , y) HP (u , v)=1− LP(u , v )
Bandreject br (x , y)=lp1 ( x , y)+hp2 ( x , y) BR(u , v)= LP1 (u , v)+ HP2 (u , v)
Bandpass bp( x , y)=δ ( x , y)−br ( x , y) BP(u , v)=1−BR(u , v)

31 / 38
Highpass, Bandreject, And Bandpass Filters From
Lowpass Filters: Example

●
More on these in Chapter 4.

32 / 38
Combining Spatial Enhancement Methods

●
DIP usually involves application of several
complementary techniques to achieve the
desired result.
●
A nuclear whole body bone scan, which
we want to enhance the skeletal details.
●
Steps:
1- Original image
2- Result of Laplacian operator
3- Adding the Laplacian image to the
original image to get a sharpened (but
noisy) image
4- Result of Sobel operator

33 / 38
Combining Spatial Enhancement Methods

●
Steps (cont’d):
5- Sobel image smoothed by a 5x5
averaging box filter
6- Mask image by multiplying (5) with
(2)
7- Sharpened image by adding (6) to
(1)
8- Finally, applying a power-law based
intensity transformation to enhance
the image contrast.

34 / 38
Introduction to Filtering in the Frequency Domain

●
Fourier Series: any periodic function can
be represented as the sum of sines and/or
cosines of different frequencies, each
multiplied by a different coefficient.
●
Fourier Transform: functions that are not
periodic, but have a finite area under the
curve, can be represented as the integral of
sines and/or cosines of different
frequencies, multiplied by a weighting
function.

35 / 38
Introduction to Filtering in the Frequency Domain

●
Progress of digital computers and the discovery of Fast Fourier Transform
(FFT) algorithms, revolutionized the field greatly.
●
For an MxN image and an mxn kernel, MNmn operations is needed.
●
If the kernel is separable, then MN(m+n) operations are needed.
●
If the same filtering is done in a the frequency domain, using FFT,
2MNlog2MN operations are needed!

36 / 38
What is Next?

●
Preliminary Concepts
●
Fourier Series of Periodic Functions of One Continuous Dimension
●
Fourier Transform of Functions of One Continuous Dimension
●
Convolution in the Fourier Domain
●
Sampling and Fourier Transform of Sampled Functions

37 / 38
Questions?
[email protected]

38 / 38