DATA SCIENCE PROGRAM

DS617

Image Processing

Lecture 3
 Spatial Image Enhancement Methods
 Enhancement methods in spatial domain can be reduced to the form:
g(x,y) = T[ f(x,y) ]
f(x,y) is the input image, g(x,y) is the output enhanced image and T is an
enhancement operator or function defined over a point, some local
neighbourhood mask centered around a point, or the image globally.

 Spatial enhancement methods are divided Into:

(a) Point (Single-Pixel) Methods: The neighbourhood is simply
of size 1x1 (the pixel itself).

(b) Mask (Local) Methods: The neighbourhood is any rectangular group

of pixels of any size centred around the pixel of interest.

(c) Geometric (Global) Methods: The neighbourhood is simply the

whole image.
Spatial Image Enhancement Methods (2)

g(x,y)

g(x,y)
 Spatial Mask (Local) Enhancement Methods
g(x,y) = T[ f(x,y) ]
• T is an enhancement operator or a function on just the neighbours of
the target pixel, f(x,y).

• They are also called neighborhood (area) operations because they

simply operate on a larger neighborhood of pixel than point
operations that only operate on a single pixel.

f(x,y) g(x,y)
 Spatial Mask (Local) Enhancement Methods (2)
 Neighborhood is mostly a rectangle of any size (odd-sized) around a
current central pixel of interest.
 Spatial Mask (Local) Enhancement Methods (3)
• For each pixel in the origin input image, the outcome is written on
the same location at the target enhanced image.

• The mechanism of these methods consists of moving an odd-sized

filter (mask or kernel), that maps to the rectangular neighborhood
area, form pixel to pixel in the original input image.
 Spatial Mask (Local) Enhancement Methods (4)
• At the edges of an image, we are missing pixels to form neighborhood

• There are a few approaches to dealing with missing edge pixels:

 Omit missing pixels: only works with some filters and it can add
extra code and slow down processing.
 Pad the image: typically with either all white or all black pixels
 Replicate border pixels.
 Truncate the image: by cropping the image such that pixels which
have not been adequately filtered are removed entirely.
 Spatial Mask (Local) Enhancement Methods (5)
• They are divided into:
 Local Histogram-Based Contrast Enhancement
 Spatial Filtering Methods (Linear & Nonlinear)

• Histogram-Based Contrast Enhancement Methods

1. Define a neighborhood area and move its center from pixel to pixel.

2. At each target pixel, compute the histogram of the pixels in just the
neighborhood area around the target pixel.

3. Apply histogram equalization or histogram specification method.

4. Map the intensity of the target pixel centered in the neighborhood.

5. Move to the next pixel and repeat the procedure.

Local Histogram-Based Contrast Enhancement
 Spatial Filtering Methods
• At each pixel (x,y) in the original input image, the response of the
mask at that pixel in the output target image is calculated using a
predefined relationship (linear or nonlinear).

• In linear spatial filtering, at each pixel (x,y), the kernel response is

given by the sum of products of the filter coefficients and the
corresponding image pixels in the area spanned by the filter.

• In nonlinear spatial filtering, at each pixel (x,y), the kernel response

is dependant on the values of neighboring pixels of the pixel (x,y)
without explicitly using the filter coefficients in sum of product
manner, e.g., calculating the median gray level value in the
neighborhood in which the filter is located for noise removal.
 Spatial Linear Filtering Methods
• The output filtered image is the correlation between the original
input image and the filter coefficients
Origin x
a b c r s t
d e f • u v w
g h i x y z

Simple 3*3
Original Filter
Neighbourhood
e 3*3 Filter
Image Pixels
eprocessed = v.e +
r.a + s.b + t.c +
u.d + w.f +
y Image f (x, y) x.g + y.h + z.i
• The above is repeated for every pixel in the original input image to
generate the output filtered image
Spatial Linear Filtering Methods (2)
 Spatial Linear Filtering Methods (3)
• For 3x3 filter shown in the previous slide, the response R of a linear
filter w with an image f is calculated as follows:

R = w(−1, −1) f(x−1,y− 1) + w(−1,0) f(x−1, y) + ⋯ + w(0,0) f(x, y)

+ ⋯ + w(1,0)f(x+1,y) + w(1,1) f(x+1,y+1)

• In general, linear filtering of an image f(x,y) of size MxN with a filter

w of size m×n is given by the correlation expression:
a b
g ( x, y )    w( s, t ) f ( x  s, y  t )
s   at   b

Where a = (m - 1)/2, b = (n - l)/2

• To generate a complete filtered image, this equation must be

applied for x = 0,1, 2,..., M-1 and y = 0,1, 2,..., N-1.
Spatial Linear Filtering Methods Example
 Spatial Linear Filters Types
• We will mainly focus on two types of spatial mask linear filters:
1. Smoothing:
 Useful for reducing noise, suppressing (blurring) edges and
eliminating small details.
 The weights of the mask must be positive.
 Sum of mask weights after normalization is 1.
Gaussian Function
Gaussian
 Spatial Linear Filters Types (2)
• We will mainly focus on two types of spatial mask linear filters:
2. Sharpening:
 Useful for highlighting noise, edges and fine details.
 The weights of the mask are both positive and negative.
 Sum of mask weights after normalization is 0.

1st derivative nd derivative

2derivative
2nd
of
st
Gaussian
1 derivative
of Gaussian
of Gaussian
of Gaussian
 Spatial Linear Smoothing Filters: Average (Mean)
• A popular example of spatial linear smoothing filters is the average
filter which is used for blurring and noise reduction.
Origin x
104 100 108 1/ 1/ 1/
9 9 9
1/ 1/ 1/
99 106 98

95 90 85
* 9 9
1/ 1/ 1/
9

9 9 9

Simple 3*3
1/ 100
104
9
1/ 108
9
1/
9 Original
3*3 Smoothing
Average
/ / /
Image
Pixels Filter
1
999 1106 1
9 989
Neighbourhood / / /
195 190 185
9 9 9
Filter
e = 1/9*106 +
1/ *104 + 1/ *100 + 1/ *108 +
9 9 9
1/ *99 + 1/ *98 +
9 9
1/ *95 + 1/ *90 + 1/ *85
y Image f (x, y) 9 9 9
= 98.3333
The above is repeated for every pixel in the original image to
generate the smoothed image
 Spatial Linear Smoothing Filters: Average (Mean) (2)
• A weight of wk = 1/(m*n) is used for any mxn average mask.

• This weighting scheme guarantees that the weights in the mask sum
to one over any given neighborhood size.

• Because all weights are equal in the mean filter then it is also called
a box filter.

• Average mask replaces every pixel in the output image with the mean
value from its mxn neighborhood.

• The mean filtering causes blurring to edges in the image, so to

preserve edge details, a threshold is introduced and only replacing
the current pixel value with the mean of its neighborhood if the
magnitude of the change in pixel value lies below this threshold.
 Average Smoothing Filter Example
• The image at the top left is an
original image of size 500*500
pixels

• The subsequent images show the

image after filtering with an
averaging filter of increasing sizes
– 3, 5, 9, 15 and 35

• Notice how detail begins to

disappear and black border begin
to appear because of padding the
borders of the original image.
 Spatial Linear Smoothing Filters: Weighted Average
• More effective spatial linear smoothing filters can be generated by
allowing different pixels in the neighbourhood with different
weights in the averaging function to reduce blurring effect in the
smoothing process; often referred to as a weighted average filter.

• Pixels closer to the central pixel are more important.

• In practice, it’s difficult to see differences 1/ 2/ 1/

16 16 16
between images smoothed by the average or
weighted average filters because the area 2/
16
4/
16
2/
16
these filters span at any one location in the
image is too small. 1/ 2/ 1/
16 16 16

Weighted Average
Filter
 Weighted Average Smoothing Filter Example
• By smoothing the original image with 15x15 weighted average filter,
we mix lots of the small objects with the background and leave only
the large objects to be highlighted by thresholding

• The size of the filter establishes the relative size of the mixed objects

Original Image Smoothed Image Thresholded Image

 Spatial Linear Smoothing Filters: Gaussian
• The weights are discrete approximations to the 2D continuous
Gaussian function:

σ = 1.4
• Weight values decrease with distance from mask center!
• Mask size is a function of σ:
 Spatial Linear Smoothing Filters: Gaussian (2)
• σ controls the amount of smoothing (↑σ ↑smoothing).
• As σ increases, more weights must be obtained to approximate the
Gaussian function accurately.

σ=3
Gaussian Smoothing Filter Example

= 1 pixel = 5 pixels = 10 pixels = 30 pixels

Gaussian Smoothing Filter Example (2)
Averaging Versus Gaussian Smoothing

Averaging

Gaussian
 Spatial Linear Sharpening Filters
• Previously we have looked at linear smoothing filters which remove
fine details, reduce noise and blur edges.

• Sharpening spatial filters seek to highlight fine detail, remove blurring

from images and highlight edges

• Image smoothing can be achieved using averaging filters, and hence

sharpening can be achieved by operators that invert averaging
operators.

• In mathematics, averaging is equivalent to the concept of integration,

and differentiation inverts integration. Thus, sharpening spatial filters
can be represented by partial derivatives.
 Spatial Linear Sharpening Filters (2)
• Differentiation measures the rate of change of a function.

• The discrete differentiation of a digital image is approximated in terms

of finite differences.

• We can form the finite differences in one of three ways:

– Forward Difference:

– Backward Difference:

– Central Difference:
 Spatial Linear Sharpening Filters (3)
• The 1st partial derivatives of the 2D digital image f(x,y) can be:
f
 f ( x  1, y )  f ( x, y )
x

f
 f ( x, y  1)  f ( x, y )
y

• The 2nd partial derivatives of the 2D digital image f(x,y) are:

 Spatial Linear Sharpening Filters (4)
• Let’s consider a simple example where the image contains a solid
object, a line and a single noise point

• Take a horizontal scan line on the image along the center including the
noise point

• When depicting the gray level profile of this horizontal line, we obtain
1D function.
Spatial Linear Sharpening Filters (5)
 Spatial Linear Sharpening Filters (6)
• 1st derivative is just the difference between subsequent values.
Image Strip

8
7
6
5
4
3
2
1
0

5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
-1 -1 -1 -1 -1 0 0 6 -6 01st 0
Derivative
0 1 2 -2 -1 0 0 0 7 0 0 0
8
6
4
2
0
-2
-4
-6
-8
 Spatial Linear Sharpening Filters (7)
• 2nd derivative simply takes into account the values both before and
after the current value Image Strip

8
7
6
5
4
3
2
1
0

5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7
-1 0 0 0 0 1 0 6 -12 62nd0Derivative
0 1 1 -4 1 1 0 0 7 -7 0 0

10

-5

-10

-15
 Spatial Linear Sharpening Filters (8)

• We conclude that 2nd derivative has much stronger response at gray-

level step than 1st derivative, thus we can expect 2nd derivative to
enhance fine details (thin lines, edges, including noise) much more
than 1st derivative.
 Spatial Linear Sharpening Filters: Laplacian
• Laplacian filter computes the 2nd derivative of a function f(x,y).

• It has the following characteristic:

– Isotropic or rotation-invariant (rotating the image and applying
the filter gives the same result as applying the filter and then
rotating the image).

• The Laplacian is defined as follows:

2 f 2 f
 f  2  2
2

 x  y
where the partial 2nd order derivative in the x direction is defined as:
2 f
 f ( x  1, y )  f ( x  1, y )  2 f ( x, y )
 x
2

and in the y direction as follows:

2 f
 f ( x, y  1)  f ( x, y  1)  2 f ( x, y )
 y
2
 Spatial Linear Sharpening Filters: Laplacian (2)
• So, the Laplacian can be given as follows:
 2 f  [ f ( x  1, y )  f ( x  1, y)  f ( x, y  1)  f ( x, y  1)]  4 f ( x, y)
• Based on the previous equation, the following is the
0 1 0
Laplacian filter that gives isotropic result for
rotations in increments of 90ᵒ 1 -4 1
0 1 0
• If the diagonal directions are incorporated, obtain
1 1 1
the following Laplacian filter that gives isotropic
result for rotations in increments of 45ᵒ 1 -8 1
1 1 1
• Also, the following Laplacian filters are commonly used in practice and
the give the same results despite of the difference in their sign
0 -1 0 -1 -1 -1
-1 4 -1 -1 8 -1
0 -1 0 -1 -1 -1
 Spatial Linear Sharpening Filters: Laplacian (3)
• Applying Laplacian to an image, getting a new image that highlights
grayish edges and other discontinuities on dark featureless background

• Background features can be "recovered" while still preserving the

sharpening effect of Laplacian filter simply by adding or subtracting
(based on Laplacian filter sign) the original and Laplacian images

• The final enhanced sharpened image, g(x,y), is obtain by:

g ( x, y)  f ( x, y)  c[ f ( x, y]
2

where f(x,y) is the original input image and

c is +1 when the center coefficient of the Laplacian filter is positive,
c is -1 when the center coefficient of the Laplacian filter is negative.
 Laplacian Sharpening Filter Example
• Apply Laplacian filter(center = -8) to the moon image to sharpen it

Original Image Laplacian Filtered Laplacian Filtered Image

Image Scaled for Display

- =
Original Image Laplacian Filtered Sharpened Image
Image
 Simplified Spatial Linear Sharpening Filter
• The entire sharpening in the previous slide can be combined into a
single filtering operation

g ( x, y )  f ( x, y )   f ( x, y )
2

 f ( x, y )  [ f ( x  1, y )  f ( x  1, y )
 f ( x, y  1)  f ( x, y  1)
 4 f ( x, y )]
 5 f ( x, y )  f ( x  1, y )  f ( x  1, y )
 f ( x, y  1)  f ( x, y  1)
 Simplified Spatial Linear Sharpening Filter (2)
• This gives us a new filter which does the whole job for us in one step

0 -1 0

-1 5 -1

0 -1 0

-1 -1 -1

-1 9 -1

-1 -1 -1
 Spatial Linear Unsharp Filter
• An alternative spatial linear sharpening filter (high-boost filter) to the
Laplacian filter.

• It operates as follows:
1- Subtracting a smoothed (unsharp) version of an image- typically
obtained by filtering the original with a mean or a Gaussian kernel-
from the original image to emphasize edges in the image.

2- The resulting difference image is then added onto the original image
to effect some degree of sharpening using a given constant scaling
factor k that ensures the resulting sharped image is within the
proper range and the edges are not “oversharp” in the resulting
sharped image.

• Generally, k = 0.2 – 0.7 is acceptable, depending on the level of

sharpening required.
Spatial Linear Unsharp Filter (2)
Unsharp Filter Example
 The End

Questions?