.

1
 Muhammad Tahir Mumtaz

M- Phil in computer science

(computer vision)
University of Central Punjab Lahore, Punjab,
Pakistan.

M- Phil in computer science

(Software Engineering)
Pir Mehr Ali Shah Arid
Agriculture University Rawalpindi Islamabad,
Punjab, Pakistan
. 3
 Image
► An image is a visual representation of
something. In information technology, the
term has several usages: 1) An image is
a picture that has been created or copied
and stored in electronic form. An
image can be described in terms of vector
graphics or raster graphics.
► Image – A two-dimensional signal that can
be observed by human visual system
. 4
How are images represented
in the computer?
Image processing is used for two somewhat different
purposes:
• improving the visual appearance of images (pictorial information)
to a human viewer, and
• Preparing (processing) images for measurement of the features
and structures present

. 6
. 7
 Introduction
► What is Digital Image Processing?
Digital Image
— a two-dimensional function f ( x, y )
x and y are spatial coordinates
The f is called intensity or gray level at the point (x, y)

Digital Image Processing

— process digital images by means of computer, it covers low-, mid-, and high-level
processes
low-level: inputs and outputs are images
mid-level: outputs are attributes extracted from input images
high-level: an ensemble of recognition of individual objects

Pixel
— the elements of a digital image

. 8
 Color Image
►A color image is just three functions pasted
together. We can write this as a “vector-
valued” function:

 r ( x, y ) 

f ( x, y )  g ( x, y ) 
 

 b ( x, y ) 

. 9
 Origins of Digital Image Processing

Sent by submarine cable

between London and New
York, the transportation
time was reduced to less
than three hours from
.
more than a week 10
 Origins of Digital Image Processing

. 11
. 12
. 13
. 14
. 15
. 16
. 17
. 18
. 19
. 20
. 21
 Sources for Images
► Electromagnetic (EM) energy spectrum
► Acoustic
► Ultrasonic
► Electronic
► Artificial images produced by computer

. 22
 Electromagnetic (EM) energy spectrum

Major uses
Gamma-ray imaging: nuclear medicine and astronomical observations
X-rays: medical diagnostics, industry, and astronomy, etc.
Ultraviolet: industrial inspection, microscopy, lasers, biological imaging,
and astronomical observations
Visible and infrared bands: light microscopy, astronomy, remote sensing, industry,
and law enforcement
Microwave band: radar
Radio band: medicine (such as MRI) and astronomy

23
. 24
 Examples: Gama-Ray Imaging

. 25
 Examples: X-Ray Imaging

. 26
 Examples: Ultraviolet Imaging

. 27
 Examples: Light Microscopy Imaging

. 28
 Examples: Visual and Infrared Imaging

. 29
 Examples: Infrared Satellite Imaging

.
2003
USA 1993 30
 Examples: Infrared Satellite Imaging

. 31
 Examples: Automated Visual Inspection

. 32
 Examples: Automated Visual Inspection

Results of
automated
reading of the
plate content by
the system

The area in which

the imaging
system detected
the plate

. 33
 Example of Radar Image

. 34
 Examples: MRI (Radio Band)

. 35
 Examples: Ultrasound Imaging

. 36
 Light and EM Spectrum

. 37
 Light and EM Spectrum

► The colors that humans perceive in an object are

determined by the nature of the light reflected
from the object.

. 38
. 39
. 40
. 41
. 42
. 43
. 44
. 45
. 46
. 47
. 48
. 49
. 50
. 51
. 52
. 53
. 54
. 55
. 56
. 57
. 58
. 59
. 60
. 61
 Image Acquisition

Transform
illumination
energy into
digital images

. 62
 Image Acquisition Using a Single Sensor

. 63
 Image Acquisition Using Sensor Strips

. 64
 Image Acquisition Process

. 65
 A Simple Image Formation Model

f ( x, y )  i ( x, y ) r ( x, y )

f ( x, y ) : intensity at the point (x, y )

i ( x, y ) : illumination at the point (x, y )
(the amount of source illumination incident on the scene)
r ( x, y ) : reflectance/transmissivity at the point (x, y )
(the amount of illumination reflected/transmitted by the object)
where 0 < i ( x, y ) <  and 0 < r ( x, y ) < 1

. 66
 Some Typical Ranges of illumination

► Illumination
Lumen — A unit of light flow or luminous flux
Lumen per square meter (lm/m2) — The metric unit of measure
for illuminance of a surface

 On a clear day, the sun may produce in excess of 90,000 lm/m2 of

illumination on the surface of the Earth

 On a cloudy day, the sun may produce less than 10,000 lm/m2 of
illumination on the surface of the Earth

 On a clear evening, the moon yields about 0.1 lm/m2 of illumination

 The typical illumination level in a commercial office is about 1000 lm/m2

. 67
 Some Typical Ranges of Reflectance

► Reflectance

 0.01 for black velvet

 0.65 for stainless steel

 0.80 for flat-white wall paint

 0.90 for silver-plated metal

 0.93 for snow

. 68
 Representing Digital Images

► The representation of an M×N numerical

array as

 f (0, 0) f (0,1) ... f (0, N  1) 

 f (1, 0) f (1,1) ... f (1, N  1) 
f ( x, y )  
 ... ... ... ... 
 
 f ( M  1, 0) f ( M  1,1) ... f ( M  1, N  1) 

. 69
 Representing Digital Images

► The representation of an M×N numerical

array as

 a0,0 a0,1 ... a0, N 1 

 a a1,1 ... a1, N 1 
A  1,0 
 ... ... ... ... 
 
 aM 1,0 aM 1,1 ... aM 1, N 1 

. 70
 Representing Digital Images

► The representation of an M×N numerical

array in MATLAB

 f (1,1) f (1, 2) ... f (1, N ) 

 f (2,1) f (2, 2) ... f (2, N ) 
f ( x, y )  
 ... ... ... ... 
 
 f ( M ,1) f ( M , 2) ... f (M , N )

. 71
 Representing Digital Images

► Discrete intensity interval [0, L-1], L=2k

► The number b of bits required to store a M × N

digitized image

b=M×N×k

. 72
 Representing Digital Images

. 73
 Spatial and Intensity Resolution

► Spatial resolution
— A measure of the smallest discernible detail in an image
— stated with line pairs per unit distance, dots (pixels) per
unit distance, dots per inch (dpi)

► Intensity resolution
— The smallest discernible change in intensity level
— stated with 8 bits, 12 bits, 16 bits, etc.

. 74
 Spatial and Intensity Resolution

. 75
 Spatial and Intensity Resolution

. 76
 Spatial and Intensity Resolution

. 77
 Image Interpolation

► Interpolation — Process of using known data to

estimate unknown values
e.g., zooming, shrinking, rotating, and geometric correction

► Interpolation (sometimes called resampling) —

an imaging method to increase (or decrease) the number
of pixels in a digital image.
Some digital cameras use interpolation to produce a larger image than
the sensor captured or to create digital zoom

http://www.dpreview.com/learn/?/key=interpolation

. 78
 Image Interpolation:
Nearest Neighbor Interpolation

f1(x2,y2) = f(x1,y1)
f(round(x2), round(y2))
=f(x1,y1)

f1(x3,y3) =
f(round(x3), round(y3))
=f(x1,y1)

. 79
 Image Interpolation:
Bilinear Interpolation

(x,y)

f 2 ( x, y )
 (1  a) (1  b) f (l , k )  a (1  b) f (l  1, k )
(1  a) b f (l , k  1)  a b f (l  1, k  1)
l  floor ( x), k  floor ( y ), a  x  l , b  y  k .
. 80
 Image Interpolation:
Bicubic Interpolation
► The intensity value assigned to point (x,y) is obtained by
the following equation
3 3
f3 ( x, y)   aij x y i j

i 0 j 0

► The sixteen coefficients are determined by using the

sixteen nearest neighbors.
http://en.wikipedia.org/wiki/Bicubic_interpolation

. 81
 Examples: Interpolation

. 82
 Examples: Interpolation

. 83
 Examples: Interpolation

. 84
 Examples: Interpolation

. 85
 Examples: Interpolation

. 86
 Basic Relationships Between Pixels

► Neighborhood

► Adjacency

► Connectivity

► Paths

► Regions and boundaries

. 87
 Basic Relationships Between Pixels

► Neighbors of a pixel p at coordinates (x,y)

 4-neighbors of p, denoted by N4(p):

(x-1, y), (x+1, y), (x,y-1), and (x, y+1).

 4 diagonal neighbors of p, denoted by ND(p):

(x-1, y-1), (x+1, y+1), (x+1,y-1), and (x-1, y+1).

 8 neighbors of p, denoted N8(p)

N8(p) = N4(p) U ND(p)

. 88
 Basic Relationships Between Pixels

► Adjacency
Let V be the set of intensity values

 4-adjacency: Two pixels p and q with values from V are

4-adjacent if q is in the set N4(p).

 8-adjacency: Two pixels p and q with values from V are

8-adjacent if q is in the set N8(p).

. 89
 Basic Relationships Between Pixels

► Adjacency
Let V be the set of intensity values

 m-adjacency: Two pixels p and q with values from V are

m-adjacent if

(i) q is in the set N4(p), or

(ii) q is in the set ND(p) and the set N4(p) ∩ N4(p) has no pixels whose
values are from V.

. 90
 Basic Relationships Between Pixels

► Path
 A (digital) path (or curve) from pixel p with coordinates (x0, y0) to pixel
q with coordinates (xn, yn) is a sequence of distinct pixels with
coordinates

(x0, y0), (x1, y1), …, (xn, yn)

Where (xi, yi) and (xi-1, yi-1) are adjacent for 1 ≤ i ≤ n.

 Here n is the length of the path.

 If (x0, y0) = (xn, yn), the path is closed path.

 We can define 4-, 8-, and m-paths based on the type of adjacency
used.

. 91
 Examples: Adjacency and Path
V = {1, 2}

0 1 1 0 1 1 0 1 1
0 2 0 0 2 0 0 2 0
0 0 1 0 0 1 0 0 1

. 92
 Examples: Adjacency and Path
V = {1, 2}

0 1 1 0 1 1 0 1 1
0 2 0 0 2 0 0 2 0
0 0 1 0 0 1 0 0 1
8-adjacent

. 93
 Examples: Adjacency and Path
V = {1, 2}

0 1 1 0 1 1 0 1 1
0 2 0 0 2 0 0 2 0
0 0 1 0 0 1 0 0 1
8-adjacent m-adjacent

. 94
 Examples: Adjacency and Path
V = {1, 2}

0 1 1
1,1 1,2 1,3 0 1 1 0 1 1
0 2 0
2,1 2,2 2,3 0 2 0 0 2 0
0 0 1
3,1 3,2 3,3 0 0 1 0 0 1
8-adjacent m-adjacent

The 8-path from (1,3) to (3,3): The m-path from (1,3) to (3,3):
(i) (1,3), (1,2), (2,2), (3,3) (1,3), (1,2), (2,2), (3,3)
(ii) (1,3), (2,2), (3,3)

. 95
 Basic Relationships Between Pixels

► Connected in S
Let S represent a subset of pixels in an image. Two pixels
p with coordinates (x0, y0) and q with coordinates (xn, yn)
are said to be connected in S if there exists a path

(x0, y0), (x1, y1), …, (xn, yn)

Where i,0  i  n,( xi , yi )  S

. 96
 Basic Relationships Between Pixels

Let S represent a subset of pixels in an image

► For every pixel p in S, the set of pixels in S that are connected to p is

called a connected component of S.

► If S has only one connected component, then S is called Connected

Set.

► We call R a region of the image if R is a connected set

► Two regions, Ri and Rj are said to be adjacent if their union forms a

connected set.
► Regions that are not to be adjacent are said to be disjoint.

. 97
 Basic Relationships Between Pixels

► Boundary (or border)

 The boundary of the region R is the set of pixels in the region that
have one or more neighbors that are not in R.
 If R happens to be an entire image, then its boundary is defined as the
set of pixels in the first and last rows and columns of the image.

► Foreground and background

 An image contains K disjoint regions, Rk, k = 1, 2, …, K. Let Ru denote

the union of all the K regions, and let (Ru)c denote its complement.
All the points in Ru is called foreground;
All the points in (Ru)c is called background.

. 98
 Question 1

► In the following arrangement of pixels, are the two

regions (of 1s) adjacent? (if 8-adjacency is used)

1 1 1
Region 1
1 0 1
0 1 0
0 0 1 Region 2

1 1 1
1 1 1

. 99
 Question 2

► In the following arrangement of pixels, are the two

parts (of 1s) adjacent? (if 4-adjacency is used)

1 1 1
Part 1
1 0 1
0 1 0
0 0 1 Part 2

1 1 1
1 1 1

. 100
 ► In the following arrangement of pixels, the two
regions (of 1s) are disjoint (if 4-adjacency is used)

1 1 1
Region 1
1 0 1
0 1 0
0 0 1 Region 2

1 1 1
1 1 1

. 101
 ► In the following arrangement of pixels, the two
regions (of 1s) are disjoint (if 4-adjacency is used)

1 1 1
foreground
1 0 1
0 1 0
0 0 1 background

1 1 1
1 1 1

. 102
 Question 3

► In the following arrangement of pixels, the circled

point is part of the boundary of the 1-valued pixels
if 8-adjacency is used, true or false?

0 0 0 0 0
0 1 1 0 0
0 1 1 0 0
0 1 1 1 0
0 1 1 1 0
0 0 0 0 0

. 103
 Question 4

► In the following arrangement of pixels, the circled

point is part of the boundary of the 1-valued pixels
if 4-adjacency is used, true or false?

0 0 0 0 0
0 1 1 0 0
0 1 1 0 0
0 1 1 1 0
0 1 1 1 0
0 0 0 0 0

. 104
 Distance Measures

► Given pixels p, q and z with coordinates (x, y), (s, t),

(u, v) respectively, the distance function D has
following properties:

a. D(p, q) ≥ 0 [D(p, q) = 0, iff p = q]

b. D(p, q) = D(q, p)

c. D(p, z) ≤ D(p, q) + D(q, z)

. 105
 Distance Measures

The following are the different Distance measures:

a. Euclidean Distance :
De(p, q) = [(x-s)2 + (y-t)2]1/2

b. City Block Distance:

D4(p, q) = |x-s| + |y-t|

c. Chess Board Distance:

D8(p, q) = max(|x-s|, |y-t|)

. 106
 Question 5

► In the following arrangement of pixels, what’s the

value of the chessboard distance between the
circled two points?

0 0 0 0 0
0 0 1 1 0
0 1 1 0 0
0 1 0 0 0
0 0 0 0 0
0 0 0 0 0

. 107
 Question 6

► In the following arrangement of pixels, what’s the

value of the city-block distance between the circled
two points?

0 0 0 0 0
0 0 1 1 0
0 1 1 0 0
0 1 0 0 0
0 0 0 0 0
0 0 0 0 0

. 108
 Question 7

► In the following arrangement of pixels, what’s the

value of the length of the m-path between the
circled two points?

0 0 0 0 0
0 0 1 1 0
0 1 1 0 0
0 1 0 0 0
0 0 0 0 0
0 0 0 0 0

. 109
 Question 8

► In the following arrangement of pixels, what’s the

value of the length of the m-path between the
circled two points?

0 0 0 0 0
0 0 1 1 0
0 0 1 0 0
0 1 0 0 0
0 0 0 0 0
0 0 0 0 0

. 110
 Introduction to Mathematical Operations in
DIP
► Array vs. Matrix Operation

 a11 a12   b11 b12 

A  B
 21 22 
a a 
 21 22 
Array
product
b b
operator
 a11b11 a12b12  Array product
A .* B   
Matrix
 21 21 22 22 
a b a b
product
operator
 a11b11  a12b21 a11b12  a12b22  Matrix product
A*B 
 a b a b 
 21 11 22 21 21 12 22 22 
a b a b

. 111
 Introduction to Mathematical Operations in
DIP
► Linear vs. Nonlinear Operation

H  f ( x, y)  g ( x, y)
H  ai f i ( x, y )  a j f j ( x, y ) 
Additivity
 H  ai fi ( x, y )   H  a j f j ( x, y ) 
 ai H  f i ( x, y )   a j H  f j ( x, y )  Homogeneity

 ai gi ( x, y )  a j g j ( x, y )
H is said to be a linear operator;
H is said to be a nonlinear operator if it does not meet the
above qualification.
. 112
 Arithmetic Operations

► Arithmetic operations between images are array

operations. The four arithmetic operations are denoted
as

s(x,y) = f(x,y) + g(x,y)

d(x,y) = f(x,y) – g(x,y)
p(x,y) = f(x,y) × g(x,y)
v(x,y) = f(x,y) ÷ g(x,y)

. 113
 Example: Addition of Noisy Images for Noise Reduction

Noiseless image: f(x,y)

Noise: n(x,y) (at every pair of coordinates (x,y), the noise is uncorrelated
and has zero average value)
Corrupted image: g(x,y)
g(x,y) = f(x,y) + n(x,y)

Reducing the noise by adding a set of noisy images, {gi(x,y)}

K
1
g ( x, y ) 
K
 g ( x, y )
i 1
i

. 114
 Example: Addition of Noisy Images for Noise Reduction

K
1
g ( x, y ) 
K
 g ( x, y )
i 1
i

1 K 
E  g ( x, y )  E   gi ( x, y )  
2
2 K
 K i 1  g ( x,y ) 1
 gi ( x , y )
K i 1
1 K 
 E    f ( x, y )  ni ( x, y ) 
 K i 1  1 2
 2
  n( x, y )
1 K
 1 K
K

 ni ( x , y )
 f ( x, y )  E  ni ( x, y )  K i 1

K i 1 
 f ( x, y )
. 115
 Example: Addition of Noisy Images for Noise Reduction

► In astronomy, imaging under very low light levels

frequently causes sensor noise to render single images
virtually useless for analysis.

► In astronomical observations, similar sensors for noise

reduction by observing the same scene over long
periods of time. Image averaging is then used to
reduce the noise.

. 116
. 117
 An Example of Image Subtraction: Mask Mode Radiography

Mask h(x,y): an X-ray image of a region of a patient’s body

Live images f(x,y): X-ray images captured at TV rates after injection of

the contrast medium

Enhanced detail g(x,y)

g(x,y) = f(x,y) - h(x,y)

The procedure gives a movie showing how the contrast medium

propagates through the various arteries in the area being observed.

. 118
. 119
 An Example of Image Multiplication

. 120
 Set and Logical Operations

. 121
 Set and Logical Operations
► Let A be the elements of a gray-scale image
The elements of A are triplets of the form (x, y, z), where
x and y are spatial coordinates and z denotes the intensity
at the point (x, y).

A  {( x, y, z ) | z  f ( x, y)}
► The complement of A is denoted Ac
Ac  {( x, y, K  z ) | ( x, y, z )  A}
K  2k  1; k is the number of intensity bits used to represent z

. 122
 Set and Logical Operations
► The union of two gray-scale images (sets) A and B is
defined as the set

A  B  {max(a, b) | a  A, b  B}
z

. 123
 Set and Logical Operations

. 124
 Set and Logical Operations

. 125
 Spatial Operations

► Single-pixel operations
Alter the values of an image’s pixels based on the intensity.

s  T ( z)

e.g.,

. 126
 Spatial Operations
► Neighborhood operations

The value of this pixel is

determined by a specified
operation involving the pixels in
the input image with coordinates
in Sxy

. 127
 Spatial Operations
► Neighborhood operations

. 128
 Geometric Spatial Transformations

► Geometric transformation (rubber-sheet transformation)

— A spatial transformation of coordinates

( x, y )  T {(v, w)}
— intensity interpolation that assigns intensity values to the spatially
transformed pixels.

► Affine transform
 t11 t12 0
 
x y 1   v w 1 t21 t22 0
t31 t32 1 
. 129
. 130
 Intensity Assignment

► Forward Mapping

( x, y )  T {(v, w)}
It’s possible that two or more pixels can be transformed to the same
location in the output image.

► Inverse Mapping
(v, w)  T 1{( x, y)}
The nearest input pixels to determine the intensity of the output pixel
value.
Inverse mappings are more efficient to implement than forward
mappings.

. 131
 Example: Image Rotation and Intensity
Interpolation

. 132
 Image Registration

► Input and output images are available but the

transformation function is unknown.
Goal: estimate the transformation function and use it to
register the two images.

► One of the principal approaches for image registration is

to use tie points (also called control points)
 The corresponding points are known precisely in the
input and output (reference) images.

. 133
 Image Registration

► A simple model based on bilinear approximation:

 x  c1v  c2 w  c3vw  c4

 y  c5v  c6 w  c7 vw  c8

Where (v, w) and ( x, y ) are the coordinates of

tie points in the input and reference images.

. 134
 Image Registration

. 135
 Image Transform

► A particularly important class of 2-D linear transforms,

denoted T(u, v)
M 1 N 1
T (u, v)    f ( x, y )r ( x, y, u , v)
x 0 y 0

where f ( x, y ) is the input image,

r ( x, y, u, v) is the forward transformation ker nel ,
variables u and v are the transform variables,
u = 0, 1, 2, ..., M-1 and v = 0, 1, ..., N-1.
. 136
 Image Transform

► Given T(u, v), the original image f(x, y) can be recoverd

using the inverse tranformation of T(u, v).

M 1 N 1
f ( x, y )    T (u , v) s ( x, y, u , v)
u 0 v 0

where s ( x, y, u , v) is the inverse transformation ker nel ,

x = 0, 1, 2, ..., M-1 and y = 0, 1, ..., N-1.

. 137
 Image Transform

. 138
 Example: Image Denoising by Using DCT Transform

. 139
 Forward Transform Kernel
M 1 N 1
T (u, v)    f ( x, y )r ( x, y, u, v)
x 0 y 0

The kernel r ( x, y, u, v) is said to be SEPERABLE if

r ( x, y, u, v)  r1 ( x, u )r2 ( y, v)

In addition, the kernel is said to be SYMMETRIC if

r1 ( x, u ) is functionally equal to r2 ( y, v), so that
r ( x, y, u, v)  r1 ( x, u )r1 ( y, u )
. 140
 The Kernels for 2-D Fourier Transform

The forward kernel

 j 2 ( ux / M  vy / N )
r ( x, y , u , v )  e
Where j = 1

The inverse kernel

1 j 2 ( ux / M  vy / N )
s ( x, y , u , v )  e
MN
. 141
 2-D Fourier Transform

M 1 N 1
T (u, v)    f ( x, y )e  j 2 ( ux / M  vy / N )

x 0 y 0

M 1 N 1
1
f ( x, y ) 
MN
  T (u, v)e
u 0 v 0
j 2 ( ux / M  vy / N )

. 142
 Probabilistic Methods
Let zi , i  0, 1, 2, ..., L -1, denote the values of all possible intensities
in an M  N digital image. The probability, p( zk ), of intensity level
zk occurring in a given image is estimated as
nk
p ( zk ) ,
MN
where nk is the number of times that intensity zk occurs in the image.
L 1

 p( z ) 1
k 0
k

The mean (average) intensity is given by

L 1
m =  zk p ( zk )
k 0
. 143
 Probabilistic Methods

The variance of the intensities is given by

L 1
 2 =  ( z k  m) 2 p ( z k )
k 0

The n th moment of the intensity variable z is

L 1
un ( z ) = (z
k 0
k  m) p( zk )
n

. 144
 Example: Comparison of Standard Deviation
Values

  14.3   31.6   49.2

. 145
 Homework
http://cramer.cs.nmt.edu/~ip/assignments.html

. 146