Image Processing and Pattern

Recognition (IPPR):
(Lecture 2)

Sanjeeb Prasad Panday

[email protected]
 2
of
90
Digital image representation
Monochrome image (or simply image) refers to a
2- dimensional light intensity function f(x,y)
– x and y denote spatial coordinates
– the value of f(x,y) at (x,y) is proportional to the brightness
(or gray level) of the image at that point
 3
of
90
Digital image representation
A digital image is an image f(x,y) that has been
discretized both in spatial coordinates and in
brightness
• Considered as a matrix whose row and column
indices represent a point in the image
• The corresponding matrix element value
represents the gray level at that point
• The elements of such an array are referred to as:
– image elements
– picture elements (pixels or pels)
 4
of
90
Steps in image processing
The problem domain in this example consists of pieces of mail
and the objective is to read the address on each piece
Step 1: image acquisition
– Acquire a digital image using an image sensor
• a monochrome or color TV camera: produces an entire image of the
problem domain every 1/30 second
• a line-scan camera: produces a single image line at a time, motion past the
camera produces a 2-dimensional image
– If not digital, an analog-to-digital conversion process is required
– The nature of the image sensor (and the produced image) are
determined by the application
• Mail reading applications rely greatly on line-scan cameras
• CCD and CMOS imaging sensors are very common in many applications
 5
of
90
Steps in image processing
• Step 2: preprocessing
– Key function: improve the image in ways that increase the
chance for success of the other processes
– In the mail example, may deal with contrast enhancement,
removing noise, and isolating regions whose texture indicates a
likelihood of alphanumeric information
 6
of
90
Steps in image processing
• Step 3: segmentation
– Broadly defined: breaking an image into its constituent parts
– In general, one of the most difficult tasks in image processing
• Good segmentation simplifies the rest of the problem
• Poor segmentation make the task impossible
– Output is usually raw pixel data: may represent region boundaries,
points in the region itself, etc.
• Boundary representation can be useful when the focus is on
external shape characteristics (e.g. corners, rounded edges, etc.)
• Region representation is appropriate when the focus is on
internal properties (e.g. texture or skeletal shape)
– For the mail problem (character recognition) both representations
can be necessary
 7
of
90
Steps in image processing
• Step 4: representation & description
– Representation: transforming raw data into a form
suitable for computer processing
– Description (also called feature extraction) deals with
extracting features that result in some quantitative
information of interest or features which are basic for
differentiating one class of objects from another
– In terms of character recognition, descriptors such as
lakes (holes) and bays help differentiate one part of the
alphabet from another
 8
of
90
Steps in image processing
• Step 5: recognition & interpretation
– Recognition: The process which assigns a label to an object
based on the information provided by its descriptors
A may be the alphanumeric character A
– Interpretation: Assigning meaning to an ensemble of
recognized objects
35487-0286 may be a ZIP code
 9
of
90
Steps in image processing
10
of
90
Steps in image processing
11
of
90
A Knowledge Base
• Knowledge about a problem domain is coded into an
image processing system in the form of a knowledge
database
– May be simple:
• detailing areas of an image expected to be of interest
– May be complex
• A list of all possible defects of a material in a vision inspection
system
– Guides operation of each processing module
– Controls interaction between modules
– Provides feedback through the system
12
of
Steps in an image processing
90
system
• Not all image processing systems would require all
steps/processing modules
– Image enhancement for human visual perception may not go
beyond the preprocessing stage
• A knowledge database may not be required
• Processing systems which include recognition and
interpretation are associated with image analysis systems
in which the objective is autonomous (or at least partially
automatic)
13
of
90
A simple imaging model
• An image is a 2-D light intensity function f(x,y)
• As light is a form of energy
0 < f(x,y) < ∞
• f(x,y) may be expressed as the product of 2 components
f(x,y)=i(x,y)r(x,y)
• i(x,y) is the illumination: 0 < i(x,y) < ∞
– Typical values: 9000 foot-candles sunny day, 100 office room, 0.01
moonlight
• r(x,y) is the reflectance: 0 < r(x,y) < 1
– r(x,y)=0 implies total absorption
– r(x,y)=1 implies total reflectance
– Typical values: 0.01 black velvet, 0.80 flat white paint, 0.93 snow
14
of
90
A simple imaging model
• The intensity of a monochrome image f at (x,y) is the gray
level (l) of the image at that point

• In practice Lmin=imin rmin and Lmax=imax rmax

• As a guideline Lmin ≈ 0.005 and Lmax ≈ 100 for indoor
image processing applications
• The interval [Lmin, Lmax] is called the gray scale
• Common practice is to shift the interval to [0,L] where l=0
is considered black and l=L is considered white. All
intermediate values are shades of gray
15
of
90
Sampling and Quantization
• To be suitable for computer processing an image, f(x,y) must
be digitized both spatially and in amplitude
• Digitizing the spatial coordinates is called image sampling
• Amplitude digitization is called gray-level quantization
• f(x,y) is approximated by equally spaced samples in the form
of an NxM array where each element is a discrete quantity
16
of
90
Sampling and Quantization
• Common practice is to let N and M be powers of two; N=2^n
and M=2^k
• And G=2^m where G denotes the number of gray levels
• The assumption here is that gray levels are equally space in
the interval [0,L]
• The number of bits, b, necessary to store the image is then

• For example, a 128x128 image with 64 gray levels would

require 98,304 bits of storage
17
of
90
Sampling and Quantization

• How many samples and gray levels are required for a “good”
approximation?
• The resolution (the degree of discernible detail) depends strongly on
these two parameters
18
of
Effects of reducing spatial
90
resolution
19
of
90
Effects of reducing gray levels
20
of
Basic relationships between
90
pixels
 An image is denoted by: f(x,y)
 Lowercase letters (e.g. p, q) will denote individual pixels
 A subset of f(x,y) is denoted by S
 Neighbors of a pixel:
– A pixel p at (x,y) has 4 horizontal/vertical neighbors at
• (x+1,y),(x-1,y),(x,y+1)and(x,y-1)
• called the 4-neighbors of p: N4(p)
– A pixel p at (x,y) has 4 diagonal neighbors at
• (x+1,y+1),(x+1,y-1),(x-1,y+1)and(x-1,y-1)
• called the diagonal–neighbors of p:ND(p)
– The 4-neighbors and the diagonal-neighbors of p are called
the 8-neighbors of p: N8(p)
21
of
90
Connectivity between pixels
• Connectivity is an important concept in establishing boundaries of
object and components of regions in an image
• When are two pixels connected?
– If they are adjacent in some sense (say they are 4-neighbors)
– and, if their gray levels satisfy a specified criterion of similarity (say
they are equal)
• Example: given a binary image (e.g. gray scale = [0,1]), two pixels
may be 4-neighbors but are not considered connected unless they
have the same value
22
of
90
Connectivity between pixels
• Let V be the set of values used to determine connectivity
– For example, in a binary image, V={1} for the connectivity of pixels
with a value of 1
– In a gray scale image, for the connectivity of pixels with a range of
intensity values of, say, 32 to 64, it follows that V={32,33,...,63,64}
– Consider three types of connectivity
• 4-connectivity: Pixels p and q with values from V are 4-connected if q is
in the set N4(p)
• 8-connectivity: Pixels p and q with values from V are 8-connected if q is
in the set N8(p)
• m-connectivity (mixed): Pixels p and q with values from V are
m-connected if
– q is in the set N4(p),or
– q is in the set ND(p) and the set N4(p) ∩ N4(q) is empty (This is the set of
pixels that are 4-neighbors of p and q and whose values are from V)
23
of
90
Connectivity between pixels
24
of
90
Pixel adjacencies and paths
• Pixel p is adjacent to q if they are connected
– We can define 4-, 8-, or m-adjacency depending on the
specified type of connectivity
• Two image subsets S1 and S2 are adjacent if some pixel in S1
is adjacent to S2
• A path from p at (x,y) to q at (s,t) is a sequence of distinct
pixels with coordinates (x0,y0), (x1,y1),....., (xn,yn)
– Where (x0,y0)=(x,y) and (xn,yn)=(s,t) and
– (xi,yi) is adjacent to (xi-1,yi-1) for 1<= i <= n
– n is the length of the path
• If p and q are in S, then p is connected to q in S if there is a
path from p to q consisting entirely of pixels in S
25
of
90
Example paths
26
of
90
Connected components
• For any pixel p in S, the set of pixels connected to p form a
connected component of S
• Distinct connected components in S are said to be disjoint
27
of
Labeling 4-connected
90
components
• Consider scanning an image pixel by pixel from left to right and
top to bottom

– Assume, for the moment, we are interested in 4-connected

components
– Let p denote the pixel of interest, and r and t denote the
upper and left neighbors of p, respectively
– The nature of the scanning process assures that r and t have
been encountered (and labeled if 1) by the time p is
encountered
28
of
Labeling 4-connected
90
components
• Consider the following procedure
if p=0 continue to the next position
if r=t=0 assign a new label to p (Ln)
if r=t=1 and they have the same label, assign that label to p
if only one of r and t are 1, assign its label to p
if r=t=1 and they have different labels, assign one label to p and note
that the two labels are equivalent (that is r and t are connected through
p)
At the end of the scan, sort pairs of equivalent labels into
equivalence classes and assign a different label to each class
29
of
Labeling 4-connected
90
components (example)
30
of
Labeling 8-connected
90
components
• Proceed as in the 4-connected component labeling case, but
also examine two upper diagonal neighbors (q and s) of p
31
of
Labeling connected components
90
in non-binary images
• The 4-connected or 8-connected labeling schemes can be
extended to gray level images
• The set V may be used to connect into a component only
those pixels within a specified range of pixel values
32
of
90
Distance measures
• Given pixels p, q, and z at (x,y), (s,t) and (u,v) respectively,
• D is a distance function (or metric) if:
– D(p,q) ≥ 0 (D(p,q)=0 iff p=q),
– D(p,q) = D(q,p), and
– D(p,z) ≤ D(p,q) + D(q,z).
• The Euclidean distance between p and q is given by:

• The pixels having distance less than or equal to some value r

from (x,y) are the points contained in a disk of radius r centered
at (x,y)
33
of
90
Distance measures
• The D4 distance (also called the city block distance) between p
and q is given by:

• The pixels having a D4 distance less than some r from (x,y)

form a diamond centered at (x,y)
• Example: pixels where D4 ≤ 2
34
of
90
Distance measures
• The D8 distance (also called the chessboard distance)
between p and q is given by:

• The pixels having a D8 distance less than some r from (x,y)

form a square centered at (x,y)
• Example: pixels where D8 ≤ 2
35
of
Distance measures and
90
connectivity
• The D4 distance between two points p and q is the shortest 4-
path between the two points
• The D8 distance between two points p and q is the shortest 8-
path between the two points
• D4 and D8 may be considered, regardless of whether a
connected path exists between them, because the definition of
these distances involves only the pixel coordinates
• For m-connectivity, the value of the distance (the length of the
path) between two points depends on the values of the pixels
along the path
36
of
Distance measures and m-
90
connectivity

• Consider the given arrangement of pixels and assume

– p, p2 and p4 =1
– p1 and p3 can be 0 or 1
• If V={1} and p1 and p3 are 0, the m-distance (p, p4) is 2 If either
p1 or p3 are 1, the m-distance (p, p4) is 3 If p1 and p3 are 1, the
m-distance (p, p4) is 4
37
of
90
M-connectivity example
38
of
90
Arithmetic & logic operations
• Arithmetic & logic operations on images used extensively in
most image processing applications
– May cover the entire image or a subset Arithmetic operation
between pixels p and q are defined as:
– Addition: (p+q)
• Used often for image averaging to reduce noise
– Subtraction: (p-q)
• Used often for static background removal
– Multiplication: (p*q) (or pq, p×q)
• Used to correct gray-level shading
– Division: (p÷q) (or p/q)
• As in multiplication
39
of
90
Logic operations
• Arithmetic operation between pixels p and q are defined as:
– AND: p AND q (also p⋅q)
– OR: p OR q (also p+q)
– COMPLEMENT: NOTq (also q’)
• Form a functionally complete set
• Applicable to binary images
• Basic tools in binary image processing, used for:
– Masking
– Feature detection
– Shape analysis
40
of
90
Examples of logic operations
41
of
90
Examples of logic operations
42
of
Neighborhood-oriented
90
operations
• Arithmetic and logical operations may take place on a
subset of the image
– Typically neighborhood oriented
• Formulated in the context of mask operations (also
called template, window or filter operations)
• Basic concept:let the value of a pixel be a function of its
(current) gray level and the gray level of its neighbors (in
some sense)
43
of
Neighborhood-oriented
90
operations
• Consider the following subset of pixels in an image
• Suppose we want to filter the image by replacing the value at
Z5 with the average value of the pixels in a 3x3 region centered
around Z5
• Perform an operation of the form:

• and assign to z5 the value of z

44
of
Neighborhood-oriented
90
operations
• In the more general form, the operation may look like:

• This equation is widely used in image processing

• Proper selection of coefficients (weights) allows for operations
such as
– noise reduction
– region thinning
– edge detection
45
of
90
What Is Image Enhancement?
Image enhancement is the process of
making images more useful
The reasons for doing this include:
– Highlighting interesting detail in images
– Removing noise from images
– Making images more visually appealing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
46
Image Enhancement Examples
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
47
Image Enhancement Examples (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
48
Image Enhancement Examples (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
49
Image Enhancement Examples (cont…)
50
of
90
Spatial & Frequency Domains
There are two broad categories of image
enhancement techniques
– Spatial domain techniques
• Direct manipulation of image pixels
– Frequency domain techniques
• Manipulation of Fourier transform or wavelet
transform of an image
For the moment we will concentrate on
techniques that operate in the spatial
domain
51
of
Basic Spatial Domain Image
90 Enhancement
Most spatial domain enhancement operations
can be reduced to the form
Origin x
g (x, y) = T[ f (x, y)]
where f (x, y) is the
input image, g (x, y) is
the processed image (x, y)
and T is some
operator defined over
some neighbourhood
of (x, y) y Image f (x, y)
52
of
90
Point Processing
The simplest spatial domain operations
occur when the neighbourhood is simply the
pixel itself
In this case T is referred to as a grey level
transformation function or a point processing
operation
Point processing operations take the form
s=T(r)
where s refers to the processed image pixel
value and r refers to the original image pixel
value
 53
of
Point Processing Example:
90 Negative Images
Negative images are useful for enhancing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

white or grey detail embedded in dark

regions of an image
– Note how much clearer the tissue is in the
negative image of the mammogram below

Original Negative
s = 1.0 - r
Image Image
54
of
Point Processing Example:
90 Negative Images (cont…)
Original Image Enhanced Image x
x

y Image f (x, y) y Image f (x, y)

s = intensitymax - r
 55
of
Point Processing Example:
90 Thresholding
Thresholding transformations are particularly
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

useful for segmentation in which we want to

isolate an object of interest from a
background

1.0 r > threshold

s=
0.0 r <= threshold
56
of
Point Processing Example:
90 Thresholding (cont…)
Original Image Enhanced Image x
x

y Image f (x, y) y Image f (x, y)

1.0 r > threshold

s=
0.0 r <= threshold
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
57
Intensity Transformations
 58
of
90
Basic Grey Level Transformations
There are many different kinds of grey level
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

transformations
Three of the most
common are shown
here
– Linear
• Negative/Identity
– Logarithmic
• Log/Inverse log
– Power law
• nth power/nth root
59
of
90
Logarithmic Transformations
The general form of the log transformation is
s = c * log(1 + r)
The log transformation maps a narrow range
of low input grey level values into a wider
range of output values
The inverse log transformation performs the
opposite transformation
Compresses the dynamic range of images
with large variations in pixel values
 60
of
90
Logarithmic Transformations (cont…)

Log functions are particularly useful when

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

the input grey level values may have an

extremely large range of values
In the following example the Fourier
transform of an image is put through a log
transform to reveal more detail

s = log(1 + r)
61
of
90
Logarithmic Transformations (cont…)

Original Image Enhanced Image x

x

y Image f (x, y) y Image f (x, y)

s = log(1 + r)

We usually set c to 1
Grey levels must be in the range [0.0, 1.0]
 62
of
90
Power Law Transformations
Power law transformations have the following
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

form
s=c*rγ
Map a narrow range
of dark input values
into a wider range of
output values or vice
versa
Varying  gives a whole
family of curves
63
of
90
Power Law Transformations (cont…)
Original Image Enhanced Image x
x

y Image f (x, y) y Image f (x, y)

s=rγ

We usually set c to 1
Grey levels must be in the range [0.0, 1.0]
64
of
90
Power Law Example
65
of
90
Power Law Example (cont…)

γ = 0.6
1
0.9
T ran sfo rm ed In ten sities

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Old Intensities
66
of
90
Power Law Example (cont…)

γ = 0.4
1
0.9
Transformed Intensities

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
67
of
90
Power Law Example (cont…)

γ = 0.3
1
0.9
Transformed Intensities

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
 68
of
90
Power Law Example (cont…)
The images to the
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

right show a
s = r 0.6
magnetic resonance
(MR) image of a
fractured human

s = r 0.4
spine
Different curves
highlight different
detail
69
of
90
Power Law Example
 70
of
90
Power Law Example (cont…)

γ = 5.0
1
0.9
Transformed Intensities

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
 71
of
90
Power Law Transformations (cont…)
An aerial photo
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of a runway is
shown s = r 3.0

This time
power law
transforms are
s = r 4.0
used to darken
the image
Different curves
highlight
different detail
72
of
90
Gamma Correction
Many devices used for image capture, display and printing
respond according to a power law
• The exponent in the power-law equation is referred to as
gamma
• The process of correcting for the power-law response is
referred to as gamma correction
• Example: – CRT devices have an intensity-to-voltage
response that is
a power function (exponents typically range from 1.8-2.5)
– Gamma correction in this case could be achieved by
applying the transformation s=r1/2.5=r^0.4
 73
of
90
Gamma Correction
Many of you might be familiar with gamma
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

correction of computer monitors

Problem is that
display devices do
not respond linearly
to different
intensities
Can be corrected
using a log
transform
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
74
More Contrast Issues
 75
of
Piecewise Linear Transformation
90 Functions
Rather than using a well defined mathematical
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

function we can use arbitrary user-defined

transforms
The images below show a contrast stretching
linear transform to add contrast to a poor
quality image
 76
of
Piecewise Linear Transformation
90 Functions
• Rather than using a well defined mathematical function we can use
arbitrary user-defined transforms
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• Contrast stretching expands the range of intensity levels in an image so it

spans a given (full) intensity range
• Control points (r1,s1) and (r2,s2) control the shape of the transform T(r)
• r1=r2, s1=0 and s2=L-1 yields a thresholding function
The contrast stretched image shown in the previous slide is obtained using
the transformation obtained from the equation of the line having following
points
• (r1,s1)=(rmin,0) and (r2, s2)=(rmax,L-1)
 77
of
90
Gray Level Slicing
• Used to highlight a specific range of intensities in an
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

image that might be of interest

•Two common approaches
– Set all pixel values within a range of interest to one
value (white) and all others to another value (black)
Produces a binary image
– Brighten (or darken) pixel values in a range of interest
and leave all others unchanged
 78
of
90
Gray Level Slicing
Highlights a specific range of grey levels
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

– Similar to thresholding
– Other levels can be
suppressed or maintained
– Useful for highlighting features
in an image
 79
of
90
Bit Plane Slicing
Often by isolating particular bits of the pixel
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

values in an image we can highlight

interesting aspects of that image
– Higher-order bits usually contain most of the
significant visual information
– Lower-order bits contain
subtle details
 80
of
90
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

[10000000] [01000000]

[00100000] [00001000]

[00000100] [00000001]
 81
of
90
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

[10000000] [01000000]

[00100000] [00001000]

[00000100] [00000001]
82
of
90
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
83
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
84
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
85
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
86
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
87
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
88
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
89
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
90
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
90
91
Bit Plane Slicing (cont…)
 92
of
90
Bit Plane Slicing (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Reconstructed image
using only bit planes 8 and
7

Reconstructed image
using only bit planes 8, 7
and 6

Reconstructed image
using only bit planes 7, 6
and 5