Image & Video Processing (IVP)

Dr Faruk Kazi
VJTI Mumbai
Module VI-
Image Enhancement
Image Enhancement-
Spatial Domain Methods
• Spatial Domain- (Space coordinates)
• Transformed Domain- (DFT, DCT, DWT, Hadamard Transform)
Image Enhancement-
Spatial Domain Methods
• Spatial Domain- (Space coordinates)
• Transformed Domain- (DFT, DCT, DWT, Hadamard Transform)
Image Enhancement-
Spatial Domain Methods
Image Enhancement-
Spatial Domain Methods
Point Operations Neighbourhood Operations
(Zero memory operations)
Image Enhancement-
Spatial Domain Methods
Point Operations • Identity Transform
(Zero memory operations) • Image Negative
• Contrast Stretching
• Thresholding
• Intensity-Level Slicing
• Bit-Plane Slicing
• Log Transform
• Power-Law (Gamma) Transform
Spatial Domain Methods-
Point Operations
1. Identity Transform 2. Image Negative 3. Contrast Stretching
Spatial Domain Methods-
Point Operations
3. Contrast Stretching
Spatial Domain Methods-
Point Operations
3. Contrast Stretching
Spatial Domain Methods-
Point Operations
3. Contrast Stretching
Spatial Domain Methods-
Point Operations
3. Contrast Stretching 4. Thresholding

The picture can't be display ed.

Spatial Domain Methods-
Point Operations
5. Intensity Level Slicing
Without Background With Background
Spatial Domain Methods-
Point Operations
5. Intensity Level Slicing Without Background- With Background-
Aortic Angiogram Sliced on higher intensities Sliced on lower intensities
 Spatial Domain Methods-
Point Operations
6. Bit-plane Slicing (LSB/MSB)
Recall- Introduction lecture
 Spatial Domain Methods-
Point Operations
6. Bit-plane Slicing (LSB/MSB)
0 6 7

1 6 7

2 6 7
 Spatial Domain Methods-
Point Operations
6. Bit-plane Slicing (LSB/MSB)
0 6 7 000 110 111

1 4 5 001 100 101

2 6 3 010 110 011

0 1 1 0 1 1 0 0 1
0 1 1 0 0 0 1 0 1
0 1 0 1 1 1 0 0 1
 Spatial Domain Methods-
Point Operations
6. Bit-plane Slicing (LSB/MSB)
0 6 7 000 110 111

1 4 5 001 100 101

2 6 3 010 110 011

0 1 1 0 1 1 0 0 1
0 1 1 0 0 0 1 0 1
0 1 0 1 1 1 0 0 1

LSB Slicing
 Spatial Domain Methods-
Point Operations
6. Bit-plane Slicing (LSB/MSB)
0 6 7 000 110 111 000 110 110 0 6 6

1 4 5 001 100 101 000 100 100 0 4 4

2 6 3 010 110 011 010 110 010 2 6 2

0 1 1 0 1 1 0 0 1
0 1 1 0 0 0 1 0 1
0 1 0 1 1 1 0 0 1

LSB Slicing
 Spatial Domain Methods-
Point Operations Application-
Image
6. Bit-plane Slicing (LSB/MSB)
Compression
0 6 7 000 110 111 000 110 110 0 6 6

1 4 5 001 100 101 000 100 100 0 4 4

2 6 3 010 110 011 010 110 010 2 6 2

0 1 1 0 1 1 0 0 1
0 1 1 0 0 0 1 0 1
0 1 0 1 1 1 0 0 1

LSB Slicing
 Spatial Domain Methods-
Point Operations Application-
Steganography
6. Bit-plane Slicing (LSB/MSB)
0 6 7 000 110 111 001 110 110 1 6 6

1 4 5 001 100 101 001 101 101 1 5 5

2 6 3 010 110 011 010 111 010 2 7 2

1 0 0
1 1 1
0 1 0
0 1 1 0 1 1 0 0 1
0 1 1 0 0 0 1 0 1
0 1 0 1 1 1 0 0 1

LSB Slicing
 Spatial Domain Methods-
Point Operations
7. Log Transform
• It is a nonlinear transform
• It maps a narrow range of low
intensity values in the inputs
into a wider range of output
levels
• Conversely, higher values of
input levels are mapped to a
narrower range in the output
 Spatial Domain Methods-
Point Operations
7. Log Transform
• It is a nonlinear transform
• It maps a narrow range of low
intensity values in the inputs
into a wider range of output
levels
• Conversely, higher values of
input levels are mapped to a
narrower range in the output
 Spatial Domain Methods-
Point Operations
7. Log Transform
• It is a nonlinear transform
• It maps a narrow range of low
intensity values in the inputs
into a wider range of output
levels
• Conversely, higher values of
input levels are mapped to a
narrower range in the output
• Inverse Log (exponential)
transformation gives the
opposite effect
 Spatial Domain Methods-
Point Operations
7. Log Transform
• It does spreading/compressing
of intensity levels in an image
• The log function has the
important characteristics that
it compresses the dynamic
range of pixel values
 Spatial Domain Methods-
Point Operations
8. Power-Law (Gamma) Transform
• The response of many devices
used for image capture,
printing and display obey a
power law
• CRT have an intensity-to-
voltage response expressed as
a power function (Gamma= 1.8
to 2.5)
• Such display systems tend to
produce images darker than
intended
 Spatial Domain Methods-
Point Operations
8. Power-Law (Gamma) Transform
• Gamma correction- Say, CRT
has gamma of 2.5
• Use transformation s=r1/2.5=r0.4
• Gamma-corrected image
produces an output that is
close in appearance to the
original image
 Spatial Domain Methods-
Point Operations
8. Power-Law (Gamma) Transform
 Spatial Domain Methods-
Point Operations
8. Power-Law (Gamma) Transform

Original Gamma=
MRI Image 0.6
Gamma= Gamma=
0.4 0.3
Image Enhancement-
Spatial Domain Methods
Summary- Point Operations • Identity Transform
(Zero memory operations) • Image Negative
• Contrast Stretching
• Thresholding
• Intensity-Level Slicing
• Bit-Plane Slicing
• Log Transform
• Power-Law (Gamma) Transform
Example
 Exercise- Point Operations
1. Obtain digital negative of the following 8-bit image

121 105 117 156 151

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

 Exercise- Point Operations
2. For the 3-bit 4x4 size image perform the following operations-
a. Negative
b. Thresholding with T=4
c. Intensity level slicing with background r1=2 and r2=5
d. Clipping with r1=2 and r2=5
e. Bit plane slicing for MSB and LSB plane
1 2 3 0
2 4 6 7
5 2 4 3
3 2 6 1
Neighborhood
Operations
Image Enhancement-
Spatial Domain Methods
Point Operations Neighbourhood Operations
(Zero memory operations)
Spatial Domain Methods-
Neighborhood Operations
Spatial Filtering
1. Low-pass filter

2. High-pass filter
Spatial Domain Methods-
Neighborhood Operations
Spatial Filtering
1. Low-pass filter- Averaging filter
Spatial Domain Methods-
Neighborhood Operations

121 105 117 156 151

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

Spatial Domain Methods-
Neighborhood Operations
Point Operation
121 105 117 156 151

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

Spatial Domain Methods-
Neighborhood Operations

Neighbourhood
121 105 117 156 151Operation

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

Spatial Domain Methods-
Neighborhood Operations

Neighbourhood
121 105 117 156 151Operation

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

Spatial Domain Methods-
Neighborhood Operations

Neighbourhood
121 105 117 156 151Operation

139 127 157 117 125

152 117 252 138 142

127 182 178 197 142

101 106 119 151 140

1 1 1
1
Averaging Filter 9
1 1 1
1 1 1
Neighborhood Operations
1 2 3 0
2 4 6 7
5 2 4 3
3 2 6 1

1 1 1
1
9
1 1 1 1
(1+2+3+2+4+6+5+2+4) = 3.22 ≈ 3
1 1 1 9
Neighborhood Operations
1 2 3 0
2 4 6 7
5 2 4 3
3 2 6 1

1 1 1
1
9
1 1 1 1
(1+2+3+2+4+6+5+2+4) = 3.22 ≈ 3
1 1 1 9

What about border pixels?

 Zero Padding
 Replication
Spatial Domain Methods-
Neighborhood Operations
Neighborhood Operations

1 1 1
1
9
1 1 1
1 1 1
Linear Spatial Filtering

1 1 1
1
9
1 1 1
1 1 1
Linear Spatial Filtering
• A linear spatial filter performs a sum-of-product operation between
an image f and a filter kernel w
• The kernel is an array whose size defines the neighbourhood of
operation, and whose coefficients determine the nature of the filter
• Kernel- mask, template, window
• Kernel is usually of odd size in both coordinate direction (3x3, 5x5
etc), in general mxn, where m=2a+1 and n=2b+1
Linear Spatial Filtering

• Linear spatial filtering of an image of size MxN with a kernel mxn is

given by the expression-

• It’s 1D representation becomes-

1D Correlation & Convolution

y(n) = ∑ x(k) h(n – k)

y(n) = x (n)* h(n)
1D Correlation & Convolution
1D Correlation & Convolution
1D Correlation & Convolution
Homework
x(n)={2, 1, 4, 2, 3} & h(n)={3,0,1,2}. Find convolution & correlation

• Convolution-
y(n)={6, 3, 14, 11, 15, 10, 7, 6}

• Correlation-
rxy(n)={4, 4, 9, 14, 11, 15, 6, 9}
ryx(n)={9, 6, 15, 11, 14, 9, 4, 4}
2D Correlation & Convolution
2D Correlation & Convolution
2D Correlation & Convolution
Correlation & Convolution
Kernels
• Lets understand kernels better-

Averaging Gaussian
Filter/Box Kernel Filter/Kernel
Separable Filter Kernels
• A 2D function G(x,y) is said to be separable if it can be written as
the product of two 1D functions G1 (x) and G2(x); that is,
G(x,y)= G1(x)G2(x)

• It allows 2D function to be treated as twice operated 1D functions

• Remember that- a spatial filter kernel is a matrix. A separable kernel
is a matrix that can be expressed as the outer product of two
vectors
 Separable Filter Kernels
• A separable kernel is a matrix that can be expressed as the outer
product of two vectors
• Consider following 2x3 kernel-

Outer product
 Separable Filter Kernels
• A separable kernel is a matrix that can be expressed as the outer
product of two vectors
• Consider following 2x3 kernel-
Important Property-
Rank of matrix =1

Outer product
Spatial Filtering
• Low Pass Filtering (Smoothing)
• Averaging/Box Filtering
• Gaussian Filter
• High Pass Filtering (Sharpening)
• Order-Statistic (Nonlinear) Filtering
• Median
• Min-Max
Spatial Filtering
• Low Pass Filtering (Smoothing)
• Averaging/Box Filtering
• Gaussian Filter
• High Pass Filtering (Sharpening)
• Order-Statistic (Nonlinear) Filtering
• Median
• Min-Max
High Pass Filtering (Sharpening)
• Based on first & second order derivatives
• For digital/discrete functions- Differences

First Derivative Second Derivative

• Must be zero in areas of • Must be zero in areas of constant
constant intensity intensity
• Must be nonzero at the onset • Must be nonzero at the onset and
of an intensity step or ramp end of an intensity step or ramp
• Must be nonzero along • Must be zero along intensity
intensity ramps ramps
Example
Example
Derivative of 1D Function
• Consider one dimensional function
• First-order derivative-

• Second-order derivative-
Derivative of 2D Function
• First-order derivative-
• Second-order derivative- Laplacian
Derivative of 2D Function
Sharpening Procedure-

Because Laplacian is a derivative operator, it highlights sharp

intensity transitions in an image and de-emphasizes regions of
slowly varying intensities.

Background features can be “recovered’ while still preserving

the sharpening effect of the Laplacian by adding the Laplacian
image to the original.

If centre coefficient is negative then we subtract the Laplacian

image from the original to obtain sharpening (c=-1)
 Example- North Pole of the Moon

This kernel provides additional

differentiation (sharpening ) in
the diagonal directions. Hence it
gives significant improvements
in sharpening
Example- North Pole of the Moon
Example- North Pole of the Moon
Unsharp Masking & Highboost
Filtering
Used mainly by printing &
publishing industry to sharpen
images
1. Blur the original image

2. Subtract the blurred image

from the original (the resulting
difference is called the mask)
3. Add the mask to the original

Unsharp Masking k=1

Highboost Filtering k>1
The Gradient-
First order derivatives
Leibniz

Newton
The Gradient-
First order derivatives
The Gradient-
First order derivatives

Roberts (cross ) Operator

The Gradient-
First order derivatives

Prewitt Operator (3X3)

= (Z7 + Z8 + Z9) - (Z1 + Z2 + Z3)

= (Z3 + Z6 + Z9) - (Z1 + Z4 + Z7)

The Gradient-
First order derivatives

Sobel Operator (3X3)

= (Z7 + 2Z8 + Z9) - (Z1 + 2Z2 + Z3)

= (Z3 + 2Z6 + Z9) - (Z1 + 2Z4 + Z7)

Spatial Filtering
• Low Pass Filtering (Smoothing)
• Averaging/Box Filtering
• Gaussian Filter
• High Pass Filtering (Sharpening)
• Order-Statistic (Nonlinear) Filtering
• Median
• Min-Max
• Trimmed Average Filter
Order Statistics Filters
• Median

• Min-max

• Trimmed Average Filter

Histogram Processing
Histogram
• Histogram is a plot/graph which has grey-scale levels on its x-axis
and number of pixels corresponding to that grey-scale on y-axis.
(Normalized histogram will have probability of occurrence on y-axis)
• Histograms are simple to compute & are suitable for fast hardware
implementation. Hence popular tool for real-time image processing
• Histogram is a one-way transform
• Shape of histogram gives a lot of insights about visual quality of the
image
Histogram
Histogram
Histogram
• Plot histogram of the given image

1 2 3 0
2 3 6 7
5 2 3 3
3 2 6 1
Histogram
Histogram
Histogram- Simple Application
• Deciding threshold from simple bimodal histogram

1 2 3 0
2 3 6 7
5 2 3 3
3 2 6 1
Histogram Processing
• Histogram Stretching
• Histogram Equalization- Histogram Matching
Histogram Processing
• Histogram Stretching
• Histogram Equalization- Histogram Matching
Histogram Processing
• Histogram Stretching
Histogram Processing
Example 1- Histogram Stretching
• Perform histogram stretching for the given image of [0-3] to [0-7]
Grey Levels 0 1 2 3
Number of Pixels 70 20 7 3
Histogram Processing
Example 1- Histogram Stretching
• Perform histogram stretching for the given image of [0-3] to [0-7]
Grey Levels 0 1 2 3
Number of Pixels 70 20 7 3

Grey Levels 0 1 2 3 4 5 6 7
Number of Pixels
Histogram Processing
• Histogram Stretching
• Histogram Equalization/ Histogram Matching
Histogram Processing
Histogram Equalization Histogram Matching
• In this method shape of the histogram is altered
Histogram Processing
Example 1- Histogram Equalization
• Perform histogram equalization for the given image
Grey Levels 0 1 2 3
Number of Pixels 70 20 7 3

Grey Levels nk Pr(rk)=nk/n CDF Sk (L-1) x Sk Rounding New nk

Off
0 70 0.7 0.7 2.1 2 70
1 20 0.2 0.9 2.7 3
2 7 0.07 0.97 2.91 3 30
3 3 0.03 1 3 3
L=4 1.00
Grey Levels 0 1 2 3
Number of Pixels 0 0 70 30
Mathematical Preliminaries-
Histogram Equalization

-Functions & invertible functions

-Probability- PDF/CDF

-Calculus- Leibniz’s rule

 Mathematical Preliminaries-
Histogram Equalization

Functions

Additional Reference-
https://ocw.mit.edu/resources/res-18-006-calculus-revisited-single-variable-calculus-fall-2010/part-i-sets-functions-and-limits/lecture-2-functions/
 Mathematical Preliminaries-
Histogram Equalization

Functions

Additional Reference-
https://ocw.mit.edu/resources/res-18-006-calculus-revisited-single-variable-calculus-fall-2010/part-i-sets-functions-and-limits/lecture-2-functions/
Mathematical Preliminaries-
Histogram Equalization

Consider intensity
transformation functions
of the following form
Mathematical Preliminaries-
Histogram Equalization
What will happen if it is not monotonic? Otherwise also, we said histogram operations are not invertible !
Artefacts in the image

Consider intensity
transformation functions
of the following form

It’s inverse is expressed

as below-

Here, we assume that-

Mathematical Preliminaries-
Histogram Equalization
Recall- normalized histogram & probability theory
Grey Levels 0 1 2 3 4 5 6 7
Number of Pixels 790 1023 850 656 329 245 122 81
Probability 0.19 0.25 0.21 0.16 0.08 0.06 0.03 0.02
Mathematical Preliminaries-
Histogram Equalization
Conclusions-

 The intensity of an image may be viewed as a random

variable in the interval [0, L-1]

 Let Pr(r) and Ps(s) denote the PDFs of intensity values

r and s in two different images

 The subscripts on P indicate that Pr and Ps are

different functions
We reviewed-
 Functions
 Probability
Mathematical Preliminaries-
Histogram Equalization
Let us formulate transformation function using ideas of PDF
and CDF (Cumulative Distribution Function) which satisfies our
conditions (a) and (b) defined earlier
Mathematical Preliminaries-
Histogram Equalization
Let us formulate transformation function using ideas of PDF
and CDF (Cumulative Distribution Function) which satisfies our
conditions (a) and (b) defined earlier

 PDFs always are positive, Probability [0,1]

 If we take integral of this function- the area under the
function- then it satisfies the monotonic increasing
function condition. That is, the area under the curve can
not decrease as r increases
Mathematical Preliminaries-
Histogram Equalization
Let us formulate transformation function using ideas of PDF
and CDF (Cumulative Distribution Function) which satisfies our
conditions (a) and (b) defined earlier

 PDFs always are positive, Probability [0,1]

 If we take integral of this function- the area under the
function- then it satisfies the monotonic increasing
function condition. That is, the area under the curve can
not decrease as r increases
 Area under the curve is 1 when r=(L-1)
We need some help!
We now take help of a fundamental result from probability theory-

If Pr(r) and T(r) are known, and T(r) is continuous and differentiable over
the range of values of interest then the PDF of the transformed (mapped)
variable s=T(r) can be obtained as-
 Here it goes!

We differentiate s=T(r) by r

How to solve derivative of a definite integral?

We take help from Leibniz’s rule in
calculus that the derivative of a definite
integral with respect to its upper limit is
the integrand evaluated at the limit.
 Here it goes!

We differentiate s=T(r) by r

We take help from Leibniz’s rule in

calculus that the derivative of a definite
integral with respect to its upper limit is
the integrand evaluated at the limit.
 We are almost done!

We differentiate s=T(r) by r
What it means?

This equation represents

uniform probability
It means- density function
A quick recap…

This equation represents

uniform probability
We said- density function
Histogram Equalization
Process Summary
Lets Discuss!
Image of Mars moon, Phobos, taken by
NASA’s Mars Global Surveyor.

The image is dominated by large, dark

areas

Histogram is characterised by a large

concentration of pixels in the dark end
of grey scale

It suggests, that histogram equalization

would be a good approach to enhance
this image
Histogram equalization transform-
It rises very fast from intensity level 0 to 190
This is because of large number of dark pixels in
original image

In transformed image- it maps a very narrow

interval of dark pixels into the upper end of the
grey scale

This gives a light, washed-out appearance

Can we do something better?
Can we do something better?
Can we do something better?
Histogram Equalization-
Lets see a few more examples!
Histogram Equalization
Histogram Equalization
Example 3
Histogram Equalization
Consider a 3-bit, 64x64 pixel image given below. Perform
histogram equalization and obtain output histogram
Grey Levels 0 1 2 3 4 5 6 7
Number of Pixels 790 1023 850 656 329 245 122 81
Probability 0.19 0.25 0.21 0.16 0.08 0.06 0.03 0.02
Example 3
Histogram Equalization
Consider a 3-bit, 64x64 pixel image given below. Perform
histogram equalization and obtain output histogram
Grey Levels 0-1.33 1-3.08 2-4.55 3-5.67 4-6.23 5-6.65 6-6.86 7-7
Number of Pixels 790 1023 850 656 329 245 122 81
Probability 0.19 0.25 0.21 0.16 0.08 0.06 0.03 0.02
Example 4
Histogram Equalization
Consider PDF as given below for original histogram, Perform
histogram equalization and obtain PDF of output image
Can we do something better?
Can we match histogram of our desire
?
Original (r)

Desired Equalized
(z) Histogram (s)
Can we match histogram of our desire
?
Original (r)

Desired Equalized
(z) Histogram (s)
Example- Histogram
Matching/Specification
Original (r)

Step-1: Obtain s=T(r)

Equalized
Desired (z)
Histogram (s)

Strictly
monotonic?
 Example- Histogram
Matching/Specification
Original

Step-1: Obtain s=T(r)

Desired Actual
Equalized
Histogram

Strictly
monotonic?
Example- Histogram
Matching/Specification

Consider a 3-bit, 64x64 pixel image given below. Perform

histogram matching as per specifications and obtain actual
target histogram
Grey Levels (r) 0 1 2 3 4 5 6 7
Number of Pixels 790 1023 850 656 329 245 122 81
Probability 0.19 0.25 0.21 0.16 0.08 0.06 0.03 0.02

Grey Levels (z) 0 1 2 3 4 5 6 7

Specified 0 0 0 0.15 0.20 0.30 0.20 0.15
Probability
Original

Step-1: Obtain s=T(r)

Grey Levels (r) 0-1.33 1-3.08 2-4.55 3-5.67 4-6.23 5-6.65 6-6.86 7-7
Number of Pixels 790 1023 850 656 329 245 122 81
Probability 0.19 0.25 0.21 0.16 0.08 0.06 0.03 0.02
Desired/Specified

Step-2: Obtain s=G(z)

Grey Levels (z) 0 1 2 3 4 5 6 7

Specified 0 0 0 0.15 0.20 0.30 0.20 0.15
Probability
Desired/Specified

Step-2: Obtain s=G(z)

Grey Levels (z) 0-0 1-0 2-0 3-1.05 4-2.45 5-4.55 6-5.95 7-7
Specified 0 0 0 0.15 0.20 0.30 0.20 0.15
Probability
Example- Histogram
Matching/Specification
r s z
0 0 0
1 1 1
2 2 2 “work-around”
procedure-
3 3 3
We find the smallest
4 4 4 value of zq so that the
value G(zq) is the
5 5 5 closest to sk

6 6 6
7 7 7
Example- Histogram
Matching/Specification
r s z
0 0 0
1 1 1
2 2 2 “work-around”
procedure-
3 3 3
We find the smallest
4 4 4 value of zq so that the
value G(zq) is the
5 5 5 closest to sk
Basically, the strategy
6 6 6 should ensure the
minimum amount of
deformation in the
7 7 7 image.
Example- Histogram
Matching/Specification
r s z
790 0 0 0
1023 1 1 1
850 2 2 2
656 3 3 3
329 4 4 4
245 5 5 5
122 6 6 6
81 7 7 7
Example- Histogram
Matching/Specification
r s z
790 0 0 0
1023 1 1 1
850 2 2 2
656 3 3 3
329 4 4 4
245 5 5 5
122 6 6 6
81 7 7 7
Example- Histogram
Matching/Specification
r s z
790 0 0 0
1023 1 1 1
850 2 2 2
656 3 3 3 790 790/4096=0.19

329 4 4 4
245 5 5 5
122 6 6 6
81 7 7 7
Example- Histogram
Matching/Specification
r s z
790 0 0 0
1023 1 1 1
850 2 2 2
656 3 3 3 790 790/4096=0.19

329 4 4 4 1023 1023/4096=0.25

245 5 5 5 850 850/4096=0.21

122 6 6 6 656+329 985/4096=0.24

81 7 7 7 245+122+81 448/4096=0.11
Example- Histogram
Matching/Specification

Original Specified Actual

Local Histogram Processing