Chapter 2: Digital Image Fundamentals

Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
RADARSAT IMAGE
Chapter 1: Introduction
Chapter 1: Introduction
Chapter 1: Introduction
 Image Processing Basics

• Low Level Processing:

– pixel in = pixel out
– Image enhancement: contrast, edges, smoothing, sharpening etc.
– ‘It looks better’ subjective evaluation

• Mid Level Processing:

– Segmentation
– Feature Extraction
– Model-based image restoration
– morphology

• High Level Processing:

– Object recognition
– Computer vision
– Knowledge base
 Image Processing Basics

• Other Concepts:
– Wavelet Theory
– Compression
– Color processing

• Image types:
– Sound
– Radar (microwave)
– Infrared
– Visible: intensity, multispectral, hyperspectral
– UV
– X-Ray
– Gamma Ray
Chapter 1: Introduction
 Image Acquisition

• 3 Basic Configurations

– Single sensor
– Line sensor
– Array sensor

• Basic Operation: incoming energy transformed to voltage. Analog to digital

conversion forms the pixel value.

• 2D array sensor: charged-couple device (CCD) or complementary metal-

oxide-semiconductor (CMOS)
Chapter 2: Digital Image Fundamentals
Chapter 2: Digital Image Fundamentals
 Image Formation

f (x¢, y¢) = i(x¢, y¢)r (x¢, y¢)

Continuous Image illumination reflectance

0 < i( x ", y ") < ∞

0 < r ( x ", y ") < 1

Total absorption Total reflectance
€
Chapter 2: Digital Image Fundamentals
 Image Formation

• Digital image

– Sampling at discrete spatial locations (x,y).

Pixel intensity
é a0,0 a0,1 a0, N -1 ù
ê a ú
ê 1, 0 ú M x N Digital image
f ( x, y ) = ê ú
ê ú
ê ú
êaM -1, 0 aM -1, N -1 úû
ë
Chapter 2: Digital Image Fundamentals
 Image Formation

• Digital image: x, y, and f(x,y) are discrete and finite.

• Digitizing intensity called Quantization.

k
L=2
Number of bits

Number of gray levels

€ ai , j ∈ {0, L −1}
 Image Formation

• Number of bits required to store M x N image:

b = kMN

€
 Image Formation

• Spatial Resolution: smallest discernable detail in an image.

∝ Δx,Δy

• Gray Level Resolution: smallest discernable change in gray level.

€
∝ L(= 2 k )
Chapter 2: Digital Image Fundamentals
Chapter 2: Digital Image Fundamentals
Chapter 2: Digital Image Fundamentals
Chapter 2: Digital Image Fundamentals
 Pixel Operations

• Zooming

– Pixel Replication

– Bilinear Interpolation

– Nearest Neighbour Interpolation

• Shrinking

– Row/column deletion

– Bilinear interpolation

– Nearest Neighbour Interpolation

Chapter 2: Digital Image Fundamentals
 Chapter 3
Image Enhancement in the
Spatial Domain
 2-D Sampling Theorem

• Shannon sampling states if a function is sampled at equal or greater

than twice its highest frequency, it is possible to recover the original
signal from its sampled version.

• Consider sampled version of continuous image

∞ ∞
f s ( x, y) = f ( x, y) ∑ ∑ δ ( x − mΔx, y − nΔy)
m=−∞ n=−∞

Sampling interval in x and y direction

€
 2-D Sampling Theorem

• Taking the Fourier Transform

¥ ¥
Fs (u, v ) = u s vs å å F (u, v ) Ä d (u - ku , v - lv )
s s
k = -¥ l = -¥

Convolution operation

1 1
us = vs =
Δx Δy
 2-D Sampling Theorem

Simplifying:
¥ ¥
Fs (u, v ) = u s vs å å F (u - ku , v - lv ) s s
k = -¥ l = -¥

• This is an infinite number of replicas of the original image spectrum.

• The original spectrum around the origin can be recovered using a low pass filter
if and only if

u s ≥ 2u max v s ≥ 2v max

Highest spatial frequency on the u and v axis of the image spectrum

 2-D Sampling Theorem

Aliasing (i.e. spectral overlap) occurs if

u s < 2u max OR v s < 2v max

• The original spectrum, and therefore the original continuous image,

cannot be recovered from the sampled image.
€ •
€
This can be important when considering the formulation of the
digital image.
 Image Enhancement in the Spatial Domain

• Image Enhancement is highly subjective and Ad Hoc.

“It looks better visually”

• The histogram is the basis for numerous spatial domain processing

techniques

– Image Enhancement
– Image Compression
– Image Segmentation

• Histogram Equalization improves the contrast of the image.

 Image Enhancement in the Spatial Domain

• Image enhancement can be done locally over a moving over the

image.

• Histogram equalization is a global image enhancement technique.

• Arithmetic Operations:

– Image subtraction
– Image Averaging

• Logical Operations:

– AND
– NOT
– OR
 Image Operations on Pixel Basis

• Pixel by pixel operations: add, subtract, multiply and divide

E.g. Division

é a0 , 0 a0,1 ù
éa0, 0 a0,1 ù éb0, 0 b0,1 ù ê b0, 0 b0,1 úú
êa ú ÷ê ú =ê
ë 1, 0 a1,1 û ë b1, 0 b1,1 û ê a1, 0 a1,1 ú
êb b1,1 úû
ë 1,0

• Matlab: element by element operation

[c] = [a]./[b]
 Image Enhancement in the Spatial Domain

• Image Enhancement in the spatial domain:

g (x, y ) = T [ f (x, y )]

where T is a localized operator which operates on a local group of pixels

OR a single pixel operator

s = T [r ]

Output pixel value Input pixel value

 Image Enhancement in the Spatial Domain

• T operator:

– Contrast Stretch, Piecewise Linear Transformation, Gray Level Slicing,

Thresholding

L = 2k
– Image negative

s = L −1− r
– Log transformation

s = c log(1+ r )
€
– Power Law Transformation

€ s = cr γ

€
 Chapter 3
Image Enhancement in the
Spatial Domain
 Image Enhancement in the Spatial Domain

• Histogram Equalization improves the contrast of the image.

r
s = T ( r ) = ∑ pr ( rj )
j=0

r
In discrete nj
implementation s = T (r) = ( L −1) ∑
j=0 n

r = 0,1, 2,..., L −1

r = input pixel value

s = output pixel value
n = total no# of pixels in the image
nj = no# of pixels in bin value j
L = total no# of gray levels in the image
 Image Enhancement in the Spatial Domain

• Image enhancement can be “global” or “local”.

– Global: every pixel modified by complete image statistics.

– Local: small areas or window processing. E.g. local histogram

equalization.
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain

Logical Operations
 Image Enhancement in the Spatial Domain

• Arithmetic Operations:

– Image subtraction

g( x, y) = f1 ( x, y) − f 2 ( x, y)

– Image Averaging

€ g( x, y) = f ( x, y) + n( x, y)

uncorrelated zero mean noise

€
1 K
g ( x, y) = ∑ gi ( x, y)
K i=1
1 2
E{g ( x, y)} = f ( x, y) σ g2 ( x ,y ) = σ n ( x, y)
K
€
 Image Enhancement in the Spatial Domain

• As K increases, the average image noise decreases.

• Image alignment is required prior to averaging. This is referred to as

image registration.

• Incorrect image alignment introduces blurring and other unwanted

artifacts in the averaged image.
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain
 Basics of Spatial Filtering

• New pixel value:

a b
R( x, y) = ∑ ∑ w(i, j) f ( x + i, y + j)
i=− a j=−b

Filter Mask, w(i, j), dimension is generally (2a+1) x (2b+1)

• € R(x,y) can also be a function of a non-linear operation, i.e. median

gray level filter mask.

• Careful with boundary effects i.e. fractional masks at image edges.

• Smoothing or averaging filter reduces spatial detail and can be used

for noise reduction.

• Image smoothing may precede image segmentation.

 Basics of Spatial Filtering

• Averaging filter:

é1 1 1ù
é1 2 1 ù
[w] = 1 êê1 1 1úú [w] = 1 êê2 4 2úú
9 16
êë1 1 1úû êë1 2 1úû

Weighted
Unweighted

a b

So å å w(i, j ) = 1
i =- a j =-b
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain
 Basics of Spatial Filtering

Order Statistics Filters:

• A type of non-linear spatial filter is based ordering (ranking) of the

pixels under the filter mask e.g. Median filter.

• Median filter: arrange pixels intensities under the filter mask in

ascending order. The output pixel, R(x,y), from the filter is the center
value.

– Preserves spatial resolution i.e. less blurring

– Particularly good for “salt and pepper noise”

• Other order statistics filters

R( x, y) = max[ MaskValues] R( x, y) = min[ MaskValues]

 Chapter 3
Image Enhancement in the
Spatial Domain
 Sharpening Spatial Filters

• Highlight fine details, enhanced blurred detail, edge enhancement.

• Blurring can be performed using spatial masks with averaging or

integration of intensity under the mask coverage.

• So blurring -> integration, then sharpening -> differentiation.

• Differentiation response is proportional to the degree of discontinuity

of the image at the point at which the diff. operator is applied.

• Image differentiation enhances edges, points and other discontinuities

(such as noise) and de-emphasizes areas of slowly varying gray
levels.
 Sharpening Spatial Filters

• Highlight fine details, enhanced blurred detail, edge enhancement.

• Based on first and second order derivatives.

∂f ∂2 f
= f ( x −1) − f ( x ) = f ( x +1) + f ( x −1) − 2 f ( x )
∂x ∂x 2

1-D: 1st order 1-D: 2nd order

€ €
1. First order derivatives generally produce thicker edge responses.
2. Second order derivatives have stronger response to fine detail.
3. First order derivatives have stronger response to step changes.
4. Second order derivatives produce double edge response to step
changes.

In general, the second order derivative yields better image enhancement

(fine detail) than the first order derivative.
 Chapter 3
Image Enhancement in the
Spatial Domain
 Sharpening Spatial Filters

• 2-D Laplacian Filter

2 2
2 ∂ f ∂ f
∇ f= 2+ 2
∂x ∂y
• In discrete form

∇ 2 f ( x, y) =€[ f ( x +1, y) + f ( x −1, y) + f ( x, y +1) + f ( x, y −1)] − 4 f ( x, y)

OR
∇ 2 f ( x, y) = [− f ( x +1, y) − f ( x −1, y) − f ( x, y +1) − f ( x, y −1)] + 4 f ( x, y)
€
Laplacian filters tend to highlight gray-level discontinuities and de-
emphasize regions with slowly varying gray levels.
€
 Chapter 3
Image Enhancement in the
Spatial Domain

Laplacian is rotationally invariant or isotropic

 Sharpening Spatial Filters

• Add original image to Laplacian result to retain original background

detail.
ì f ( x, y) - Ñ 2 f (x, y ) If center coefficient is negative
g (x, y ) = í
î f (x, y ) + Ñ f (x, y ) If center coefficient is positive
2

\ g (x, y ) = f (x, y ) + [- f (x + 1, y ) - f (x -1, y ) - f (x, y + 1) - f (x, y -1)] + 4 f (x, y )

• Others sharpening filters: Roberts, Sobel, etc. Each with distinctive

filter masks.

• Spatial enhancements can be combined in sequence. The number of

combinations and permutations is very large. This leads to ad-hoc
image processing methods with subjective measures of
improvement.

• Careful in drawing conclusions of objective improvements.

 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 3
Image Enhancement in the
Spatial Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
Image Enhancement in the Frequency Domain

• Fourier Series: frequency content in periodic signals

• Fourier Transform: generalized version with no requirement of

periodicity.

+¥
F (u ) = ò f (x )e - j 2pux dx 1-D Fourier Transform
-¥
+¥
f (x ) = ò F (u )e + j 2pux
du Inverse 1-D Fourier Transform
-¥

f (x) Û F (u ) Fourier Transform Pair

Image Enhancement in the Frequency Domain

• 2-D Fourier Transform (Continuous)

+¥ +¥
F (u, v ) = òò f (x, y )e - j 2p (ux + vy )dxdy
- ¥- ¥

+¥ +¥
f ( x, y ) = ò ò F (u , v )e + j 2p (ux + vy )
dudv
- ¥- ¥

F (u, v) Û f (x, y )
 Image Enhancement in the Frequency Domain

• 2-D Discrete Fourier Transform (DFT) of M x N Image

$
ux vy '
1 M −1 N −1 − j 2π & + )
%M N (
F (u, v) = ∑ ∑ f ( x, y)e
MN x=0 y=0

Spatial coordinates

€ Spatial Frequencies 0 ≤ x ≤ M −1
0 ≤ y ≤ N −1
0 ≤ u ≤ M −1
0 ≤ v ≤ N −1
€
€ €
Image Enhancement in the Frequency Domain

• 2-D Discrete Fourier Transform (DFT) of M x N Image is

generally complex i.e.

F (u,v) = R(u,v) + jI (u,v)

Real component
Imaginary component

•
€f(x,y) is generally real i.e. intensity

• Other comments:
$ I (u,v) '
−1
∠F (u,v) = tan & )
j∠F ( u,v )
% R(u,v) (
F (u,v) = F (u,v) e
F (u,v) = R 2 (u,v) + I 2 (u,v)
Image Enhancement in the Frequency Domain

• Power spectrum

2
F (u,v) = R 2 (u,v) + I 2 (u,v)
= F (u,v) F * (u,v)

Complex conjugate
€
€
F * (u,v) = R(u,v) − jI (u,v)
• Other notables:

1 1
Δu = Δv =
€ MΔx NΔy
 FFT Shifted centered around M/2

M/2
0 M-1

0 M/2 M-1
RECT function and power spectrum
 Chapter 4
Image Enhancement in the
Frequency Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
 Discrete Fourier Domain
Processing

Gaussian Image FFTSHIFTED version

around M/2 and for prior to DFT
N/2 processing
 Chapter 4
Image Enhancement in the
Frequency Domain

FFTSHIFT
Image Enhancement in the Frequency Domain

• Filtering in the Frequency Domain

G(u,v) = F (u,v) H (u,v)

Filtered image transform
Image transform

€ Filter transfer function

Filter image in the spatial domain

g( x, y) = FT −1 {G(u,v)}
 Image Enhancement in the Frequency Domain

• Linear System:

f(x,y) h(x,y) g(x,y)

Convolution g( x, y) = f ( x, y) ⊗ h( x, y)
Operation

FT {g( x, y)} = F (u, v) H (u, v) = G(u, v)

€
Filter impulse response h( x, y) ⇔ H (u,v) Filter transfer function
€
 Image Enhancement in the Frequency Domain

Some other basic filter properties:

δ ( x, y) ⊗ h( x, y) ⇔ H (u,v)

f ( x, y)h( x, y) = FT −1 {F (u, v) ⊗ H (u, v)}

€

€
Convolution and Padding
Zero Padding for convolution
 Image Enhancement in the Frequency Domain

• Spatial filtering can be performed in the spatial domain using

convolution or in the frequency domain using complex
multiplication.

• In terms of computational requirements, there are advantages to

each technique of which depends on the image size and filter
extent.

• It may be computationally advantageous to take the 2D-FFT of the

image and filter response and perform a multiplication.

• It may be computationally advantageous to perform a convolution

operation directly in the spatial domain.

1 M −1 N −1
f ( x, y) ⊗ h( x, y) = ∑ ∑ f ( m,n)h( x − m, y − n)
MN m=0 n=0
Image Enhancement in the Frequency Domain

Step by step process of filtering in the frequency domain:

1. Determine if padding of dimensions is needed.

2. Compute F(u,v) using 2DFFT function.
3. FFTShift designed filter transfer function H(u,v) (assuming center at
(N/2, N/2)).
4. Multiply element by element F(u,v) and H(u,v).
5. Compute the inverse 2DFFT of step 4 result.
6. Take real part of result from step 5 (assuming input image is real).
 Chapter 4
Image Enhancement in the
Frequency Domain
 Image Enhancement in the Frequency Domain

Some basic filter properties:

• Low Pass Filter, HL(u,v)

– Attenuates high frequency content in the image.
– Passes low frequency content.
– Reduces detail, sharpness, acts like the averaging spatial filter.

• High Pass Filter, HH(u,v)

– Attenuates low frequency content in the image.
– Passes high frequency content.
– Emphasizes detail, edges, boundaries, acts like the sharpening
spatial filter.
 Chapter 4
Image Enhancement in the
Frequency Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
 Image Enhancement in the Frequency Domain

• Ideal Low Pass Filter (ILPF) for image smoothing:

#1 D(u,v) ≤ D0
H ILPF (u,v) = $
%0 D(u,v) > D0

Cut-off frequency

Distance in frequency from center of spectrum

€
2 2
# M & # N&
D(u,v) = % u − ( + % v − (
$ 2' $ 2'

• This is “brickwall” filter only realizable in computer modeling. This

type of filter will have severe ringing artifacts.
€
 Chapter 4
Image Enhancement in the
Frequency Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
 Image Enhancement in the Frequency Domain

• nth order Butterworth Low Pass Filter (BLPF):

1
H BLPF (u,v) = 2n
" D(u,v)%
1+ $ '
# D0 &

• Unlike the ILPF, the BLPF does not have distinct “brickwall”
transition from low frequencies to high frequencies.
€
• Less ringing artifacts.

• Steepness of the transition region increases with the filter order,

n.

• n = 2 is typical for balancing a good LPF transition and ringing

artifacts.
 Chapter 4
Image Enhancement in the
Frequency Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
 Chapter 4
Image Enhancement in the
Frequency Domain
 Image Enhancement in the Frequency Domain

• Gaussian Low Pass Filter (GLPF)

D 2 ( u ,v )
−
H GLPF (u,v) = e 2σ 2

σ is the standard deviation of the Gaussian function

D 2 ( u ,v )
€ −
2D0
2

€ H GLPF (u,v) = e
if σ = D0
• Similar smoothing performance as Butterworth LPF (n=2) but no
ringing artifacts due to its Gaussian shape.
€