Dr.

Justin Varghese Slide 1

19ECS352: IMAGE PROCESSING

Digital Image Processing

Justin Varghese, Ph.D. (Eng.), SM IEEE

Professor,
Department of Computer Science & Engineering,
Gitam (Deemed to be University)
GST, Bangalore Campus
Mobile: +91-9940955156
Email: [email protected], [email protected]

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 2
19ECS352: IMAGE PROCESSING

Fourier Series and Fourier Transform

• Any function that periodically repeats itself can be
expressed as the sum of sines and/or cosines of
different frequencies, each multiplied by a
different coefficient (Fourier series).
• Even functions that are not periodic (but whose
area under the curve is finite) can be expressed as
the integral of sines and/or cosines multiplied by a
weighting function (Fourier transform).

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 3
19ECS352: IMAGE PROCESSING

Background

• The frequency domain

refers to the plane of the two
dimensional discrete Fourier
transform of an image.
• The purpose of the Fourier
transform is to represent a
signal as a linear combination
of sinusoidal signals of
various frequencies.

Department of Computer Science & Engineering

 Slide 44
Slide
19ECS352: IMAGE PROCESSING Dr. Justin Varghese
Dr. Justin Varghese

Introduction to the Fourier Transform

and the Frequency Domain

• The one-dimensional Fourier transform and its inverse

– Fourier transform (continuous case)

F (u) =  f ( x)e− j 2ux dx where j = −1
−
– Inverse Fourier transform: e j = cos + j sin

f ( x) =  F (u)e j 2ux du
−
• The two-dimensional Fourier transform and its inverse
– Fourier transform (continuous case)
 
F (u, v) =   f ( x, y)e− j 2 (ux+vy ) dxdy
− −
– Inverse Fourier transform:
 
f ( x, y) =   F (u, v)e j 2 (ux+vy ) dudv
− −

Department of Computer Science & Engineering

 Slide 55
Slide
19ECS352: IMAGE PROCESSING Dr. Justin Varghese
Dr. Justin Varghese

Introduction to the Fourier Transform

and the Frequency Domain
• The one-dimensional Fourier transform and its inverse
– Fourier transform (discrete case) DTC
M −1
1
F (u) =
M
 f ( x)e
x =0
− j 2ux / M
for u = 0,1,2,...,M −1

– Inverse Fourier transform:

M −1
f ( x) =  F (u)e j 2ux / M for x = 0,1,2,...,M −1
u =0

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 6
19ECS352: IMAGE PROCESSING

The One-Dimensional Fourier Transform

Example

Department of Computer Science & Engineering

 Slide 77
Slide
19ECS352: IMAGE PROCESSING Dr. Justin Varghese
Dr. Justin Varghese

Introduction to the Fourier Transform

and the Frequency Domain
• The two-dimensional Fourier transform and its inverse
– Fourier transform (discrete case) DTC

1 M −1 N −1
F (u, v) = 
MN x=0 y =0
f ( x, y )e − j 2 ( ux / M + vy / N )

for u = 0,1,2,...,M − 1, v = 0,1,2,...,N − 1

– Inverse Fourier transform:

M −1 N −1
f ( x, y) =  F (u, v)e j 2 (ux / M +vy / N )
u =0 v =0

for x = 0,1,2,...,M − 1, y = 0,1,2,...,N − 1

• u, v : the transform or frequency variables
• x, y : the spatial or image variables
Department of Computer Science & Engineering
 Slide 88
Slide
19ECS352: IMAGE PROCESSING Dr. Justin Varghese
Dr. Justin Varghese

Introduction to the Fourier Transform

and the Frequency Domain
• We define the Fourier spectrum, phase anble, and power
spectrum as follows:

 
1
F (u, v) = R (u, v) + I (u, v)
2 2 2 ( spectrum)
 I (u, v) 
 (u, v) = tan−1   (phase angle)
 R(u, v) 
P(u,v) = F (u, v) = R 2 (u, v) + I 2 (u, v) (powerspectrum)
2

– R(u,v): the real part of F(u,v)

– I(u,v): the imaginary part of F(u,v)

Department of Computer Science & Engineering

 Slide 99
Slide
19ECS352: IMAGE PROCESSING Dr. Justin Varghese
Dr. Justin Varghese

Introduction to the Fourier Transform

and the Frequency Domain
• Some properties of Fourier transform:

 
 f ( x, y)(−1) x + y = F (u −
M
2
N
, v − ) (shift)
2
1 M −1 N −1
F (0,0) = 
MN x =0 y =0
f ( x, y) (average)

F (u, v) = F * (−u,−v) (conujgatesymmetric)

F (u, v) = F (−u,−v) (symmetric)

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 10
19ECS352: IMAGE PROCESSING

The Two-Dimensional DFT and Its Inverse

The 2D DFT F(u,v) can be obtained by

1. taking the 1D DFT of every row of image f(x,y), F(u,y),
2. taking the 1D DFT of every column of F(u,y)

(a)f(x,y) (b)F(u,y) (c)F(u,v)

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 11
19ECS352: IMAGE PROCESSING

The Two-Dimensional DFT and Its Inverse

shift
Department of Computer Science & Engineering
 Dr. Justin Varghese Slide 12
19ECS352: IMAGE PROCESSING

The Two-Dimensional DFT and Its Inverse

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 13
19ECS352: IMAGE PROCESSING

The Property of Two-Dimensional DFT

Rotation

DFT

DFT

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 14
19ECS352: IMAGE PROCESSING

The Property of Two-Dimensional DFT

Linear Combination

A
DFT

B
DFT

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 15
19ECS352: IMAGE PROCESSING

The Property of Two-Dimensional DFT

Expansion

A
DFT

B DFT

Expanding the original image by a factor of n (n=2), filling

the empty new values with zeros, results in the same DFT.
Department of Computer Science & Engineering
 Dr. Justin Varghese Slide 16
19ECS352: IMAGE PROCESSING

Two-Dimensional DFT with Different Functions

Sine wave Its DFT

Rectangle
Its DFT

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 17
19ECS352: IMAGE PROCESSING

Filtering in the Frequency Domain

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 18
19ECS352: IMAGE PROCESSING

• A fast Fourier transform (FFT) algorithm computes the

discrete Fourier transform (DFT) of a sequence, or its
inverse (IDFT). Fourier analysis converts a signal from its
original domain (often time or space) to a representation in
the frequency domain and vice versa.
• An FFT rapidly computes such transformations than DFT.
• It reduces the complexity of computing the DFT from
O(n^2), which arises if one simply applies the definition of
DFT, to O(n log n), where n is the data size.
• The basic ideas of FFT were popularized in 1965, but some
algorithms had been derived as early as 1805.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 19
19ECS352: IMAGE PROCESSING

Image Enhancement in the Frequency Domain

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 20
19ECS352: IMAGE PROCESSING

Basics of filtering in the frequency domain

1. multiply the input image by (-1)x+y to center the transform to

u = M/2 and v = N/2 (if M and N are even numbers, then the
shifted coordinates will be integers)
2. compute F(u,v), the DFT of the image from (1)
3. multiply F(u,v) by a filter function H(u,v)
4. compute the inverse DFT of the result in (3)
5. obtain the real part of the result in (4)
6. multiply the result in (5) by (-1)x+y to cancel the
multiplication of the input image.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 21
19ECS352: IMAGE PROCESSING

Notch filter

• this filter is to force the F(0,0) which is

the average value of an image (dc component of
the spectrum)
• the output has prominent edges
• in reality the average of the displayed image
can’t be zero as it needs to have negative gray
levels. the output image needs to scale the
gray level

0 if (u, v) = (M/2, N/2 )

H (u, v) = 
1 otherwise

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 22
19ECS352: IMAGE PROCESSING

Low pass filter

high pass filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 23
19ECS352: IMAGE PROCESSING

Add the ½ of filter height to F(0,0) of

the high pass filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 24
19ECS352: IMAGE PROCESSING

Correspondence between filter in spatial and frequency

domains
Ideal Lowpass filter
1 if D(u,v)  D0
H (u, v) = 
0 if D(u,v)  D0

Butterworth Lowpass filter

1
H (u, v) =
1 +  D(u, v) D0 
2n

Gaussian Lowpass filter

− D2 ( u ,v ) / 2 D02
H (u, v) = e

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 25
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 26
19ECS352: IMAGE PROCESSING

Smoothing Frequency-domain filters:

Ideal Lowpass filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 27
19ECS352: IMAGE PROCESSING

image power circles

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 28
19ECS352: IMAGE PROCESSING

Result of ILPF

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 29
19ECS352: IMAGE PROCESSING

Example

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 30
19ECS352: IMAGE PROCESSING

Butterworth Lowpass Filter: BLPF

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 31
19ECS352: IMAGE PROCESSING

Example

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 32
19ECS352: IMAGE PROCESSING

Spatial representation of BLPFs

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 33
19ECS352: IMAGE PROCESSING

Sharpening Frequency Domain Filter:

Ideal highpass filter
0 if D(u, v)  D0
H (u, v) = 
1 if D(u, v)  D0

Butterworth highpass filter

1
H (u, v) =
1 + D0 D(u, v)
2n

Gaussian highpass filter

− D 2 ( u ,v ) / 2 D02
H (u, v) = 1 − e

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 34
19ECS352: IMAGE PROCESSING

Spatial representation of Ideal, Butterworth

and Gaussian highpass filters

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 35
19ECS352: IMAGE PROCESSING

Example: result of IHPF

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 36
19ECS352: IMAGE PROCESSING

Example: result of BHPF

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 37
19ECS352: IMAGE PROCESSING

Example: result of GHPF

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 38
19ECS352: IMAGE PROCESSING

Laplacian in the Frequency domain

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 39
19ECS352: IMAGE PROCESSING

Example: Laplacian filtered image

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 40
19ECS352: IMAGE PROCESSING

Example: high-boost filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 41
19ECS352: IMAGE PROCESSING

Examples

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 42
19ECS352: IMAGE PROCESSING

Homomorphic Filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 43
19ECS352: IMAGE PROCESSING

Result of Homomorphic filter

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 44
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 45
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 46
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 47
19ECS352: IMAGE PROCESSING

Color image processing

Image “Types”
(categorized by “color”)
◼ Binary Image
◼ has exactly two colors

◼ Grayscale
◼ has no chromatic content

◼ Color
◼ contains some pixels with color
◼ more than two colors exist

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 48
19ECS352: IMAGE PROCESSING

Color image processing

◼ Why use color in image processing?
◼ Color is a powerful descriptor

◼ Object identification and extraction

◼ eg. Face detection using skin colors

◼ Humans can discern thousands of color shades

and intensities
◼ c.f. Human discern only two dozen shades of

grays

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 49
19ECS352: IMAGE PROCESSING

Primary Colors

• Primary colors of light are additive

– Primary colors are red, green, and blue
– Combining red + green + blue yields white

• Primary colors of pigment are subtractive

– Primary colors are cyan, magenta, and yellow
– Combining cyan + magenta + yellow yields black

Department of Computer Science & Engineering

 Slide 50

Primary colors of light v.s.

19ECS352: IMAGE PROCESSING Dr. Justin Varghese

primary colors of pigments

◼ Primary color of pigments
◼ Color that subtracts or absorbs a primary color
of light and reflects or transmits the other two

Color of light: R G B

Color of pigments: absorb R absorb G absorb B

Cyan Magenta Yellow

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 51
19ECS352: IMAGE PROCESSING

Color Models
• Color Model
– A mathematical system for representing color

• The human eye combines 3 primary colors (using the 3 different types of
cones) to discern all possible colors.

• Colors are just different light frequencies

– red – 700nm wavelength
– green – 546.1 nm wavelength
– blue – 435.8 nm wavelength

• Higher frequencies are color colors

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 52
19ECS352: IMAGE PROCESSING

Color Models

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 53
19ECS352: IMAGE PROCESSING

Color models
◼ Color model, color space, color system
◼ Specify colors in a standard way

◼ A coordinate system that each color is represented by a

single point

◼ RGB model Suitable for hardware or

◼ CYM model applications
◼ CYMK model
◼ HSI model - match the human description

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 54
19ECS352: IMAGE PROCESSING

RGB Color model

• The RGB color system is one of the most well-
known color systems in the world, and perhaps the
most ubiquitous.
• As an additive color system, it combines red, green,
and blue light to create the colors we see on our TV
screens, computer monitors, and smartphones.
• This model is used for modeling color sources that
produces light

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 55
19ECS352: IMAGE PROCESSING

RGB Color model

Source: www.mitsubishi.com

Active displays, such as computer monitors and television sets, emit

combinations of red, green and blue light. This is an additive color model

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 56
19ECS352: IMAGE PROCESSING

Pixel depth
◼ Pixel depth: the number of bits used to represent each
pixel in RGB space
◼ Full-color image: 24-bit RGB color image
◼ (R, G, B) = (8 bits, 8 bits, 8 bits)

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 57
19ECS352: IMAGE PROCESSING

RGB Color model

RGB color is best suited for on-screen applications, such as graphic design.
Each color channel is expressed from 0 (least saturated) to 255 (most
saturated). This means that 16,777,216 different colors can be represented in
the RGB color space.

Advantages of RGB Color

• Almost every well-known application is compatible with RGB, such as
Microsoft Office, Adobe Creative Suite (InDesign, Photoshop, etc.),
and other digital editors.

Drawbacks of RGB Color

• One of the major limitations of the RGB color system is that it doesn’t
translate well to print, which uses the CMYK system.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 58
19ECS352: IMAGE PROCESSING

CMY model (+Black = CMYK)

◼ CMY: secondary colors of light, or primary
colors of pigments
◼ Used to generate hardcopy output

 C  1  R 
M  = 1 − G
    
 Y  1  B 

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 59
19ECS352: IMAGE PROCESSING

CMY model (+Black = CMYK)

cyan, magenta, yellow, black, is known as a subtractive colors
and are used for modeling color pigments that subtract color

Advantages of CMYK Color Model

• CMYK is mainly used for printing

Drawbacks of RGB Color

• Cannot provide good color production for color sources or
designing.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 60
19ECS352: IMAGE PROCESSING

HSI Color Model

◼ Hue is defined as an angle
◼ 0 degrees is RED

◼ 120 degrees is GREEN

◼ 240 degrees is BLUE

◼ Saturation is defined as the percentage of distance from the

center of the HSI triangle to the pyramid surface.
◼ Values range from 0 to 1.

◼ Intensity is denoted as the distance “up” the axis from black.

◼ Values range from 0 to 1

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 61
19ECS352: IMAGE PROCESSING

Light Intensity
◼ Note that intensity is a weighted function of the r, g, b values.
◼ The human eye doesn’t weight each component identically!

intensity = 0.299*Red + 0.587*Green + 0.144*Blue

◼ Assume three light sources have the same actual intensity but
are colored red, green, and blue
The green light will appear brightest followed by red and blue

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 62
19ECS352: IMAGE PROCESSING

HSI and RGB

RGB and HSI are commonly used to

specify colors in software
applications.

HSI has variants such as HSL and

HSB both all of which model color in
the same fundamental way.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 63
19ECS352: IMAGE PROCESSING

Conversion Between RGB and HSI

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 64
19ECS352: IMAGE PROCESSING

Pseudo-color image processing

◼ Assign colors to gray values based on a

specified criterion
◼ For human visualization and interpretation
of gray-scale events
◼ Intensity slicing
◼ Gray level to color transformations

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 65
19ECS352: IMAGE PROCESSING

Types of Pseudo-color image processing

Intensity slicing
Frequency slicing
Color Quantization

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 66
19ECS352: IMAGE PROCESSING

Pseudo-color image processing

◼ Intensity slicing: Display different gray
levels as different colors
◼ Can be useful to visualize medical / scientific
/ vegetation imagery
◼ E.g. if one is interested in features with a certain
intensity range or several intensity ranges
◼ Frequency slicing: Decomposing an image
into different frequency components and
represent them using different colors.

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 67
19ECS352: IMAGE PROCESSING

Intensity Slicing

◼ Pixels with gray-scale (intensity) value in the range of (f i-1 , fi) are
rendered with color Ci

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 68
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 69
19ECS352: IMAGE PROCESSING

Pseudo Color Display of Multiple Images

◼ Display multi-sensor images as a single color
image
◼ Multi-sensor images: e.g. multi-spectral images by
satellite

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 70
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 71
19ECS352: IMAGE PROCESSING

Color Quantization
◼ In low end monitors, the monitor cannot display
all possible colors.
◼ Select a set of colors, save them in a look-up
table (also known as color map or color palette)
◼ Any color is quantized to one of the indexed
colors
◼ Only needs to save the index as the image pixel
value and in the display buffer

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 72
19ECS352: IMAGE PROCESSING

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 73
19ECS352: IMAGE PROCESSING

Basics of full-color image processing

Representing Color pixel

◼ A pixel at (x,y) is a vector in the color space
◼ RGB color space
 R( x, y) 
c( x, y) = G( x, y)
 B( x, y) 
c.f. gray-scale image
f(x,y) = I(x,y)

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 74
19ECS352: IMAGE PROCESSING

Example: spatial mask

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 75
19ECS352: IMAGE PROCESSING

How to deal with color vector?

◼ Per-color-component processing
◼ Process each color component

◼ Vector-based processing
◼ Process the color vector of each pixel

◼ When can the above methods be equivalent?

◼ Process can be applied to both scalars and

vectors
◼ Operation on each component of a vector

must be independent of the other component

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 76
19ECS352: IMAGE PROCESSING

Two spatial processing categories

◼ Similar to gray scale processing studied
before, we have to major categories
◼ Pixel-wise processing
◼ Neighborhood processing

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 77
19ECS352: IMAGE PROCESSING

Color image smoothing

• Neighborhood processing

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 78
19ECS352: IMAGE PROCESSING

Color image smoothing:

averaging mask
1
c( x, y) =  c( x, y)
K ( x, y )S xy
vector processing

Neighborhood
Centered at (x,y)
1 
  R( x, y) 
 K ( x, y )S xy 
1 
c( x, y) =   G( x, y)
per-component processing
 K ( x, y )S xy 
1 
 K  B( x, y) 
 ( x, y )S xy 
Department of Computer Science & Engineering
 Dr. Justin Varghese Slide 79
19ECS352: IMAGE PROCESSING

original R

G B

H S I
Department of Computer Science & Engineering
 Dr. Justin Varghese Slide 80
19ECS352: IMAGE PROCESSING

Example: 5x5 smoothing

mask
Smooth I
RGB model in HSI model difference

Department of Computer Science & Engineering

 Dr. Justin Varghese Slide 81
19ECS352: IMAGE PROCESSING

Questions?

Department of Computer Science & Engineering