Image Processing and Pattern

Recognition (IPPR):
Introduction (Lecture 1)

Sanjeeb Prasad Panday

[email protected]
 2
of
92
Introduction

“One picture is worth more than ten

thousand words”
Anonymous
 3
of
92
References
“Digital Image Processing”, Rafael C.
Gonzalez & Richard E. Woods,
Addison-Wesley, 2002
– Much of the material that follows is taken from
this book
 4
of
92
Contents
This lecture will cover:
– What is a digital image?
– What is digital image processing?
– History of digital image processing
– State of the art examples of digital image
processing
– Key stages in digital image processing
 5
of
92
What is a Digital Image?
A digital image is a representation of a two-
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

dimensional image as a finite set of digital

values, called picture elements or pixels
 6
of
92
What is a Digital Image? (cont…)
Pixel values typically represent gray levels,
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

colours, heights, opacities etc

Remember digitization implies that a digital
image is an approximation of a real scene

1 pixel
 7
of
92
What is a Digital Image? (cont…)
Common image formats include:
– 1 sample per point (B&W or Grayscale)
– 3 samples per point (Red, Green, and Blue)
– 4 samples per point (Red, Green, Blue, and “Alpha”,
a.k.a. Opacity)

For most of this course we will focus on grey-scale

images
 8
of
92
What is Digital Image Processing?
Digital image processing focuses on two
major tasks
– Improvement of pictorial information for
human interpretation
– Processing of image data for storage,
transmission and representation for
autonomous machine perception
Some argument about where image
processing ends and fields such as image
analysis and computer vision start
 9
of
92
What is DIP? (cont…)
The continuum from image processing to
computer vision can be broken up into low-,
mid- and high-level processes
Low Level Process Mid Level Process High Level Process
Input: Image Input: Image Input: Attributes
Output: Image Output: Attributes Output: Understanding
Examples: Noise Examples: Object Examples: Scene
removal, image recognition, understanding,
sharpening segmentation autonomous navigation

In this course we will

stop here
 10
of
92
History of Digital Image Processing
Early 1920s: One of the first applications of
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

digital imaging was in the news-

paper industry
– The Bartlane cable picture
transmission service Early digital image

– Images were transferred by submarine cable

between London and New York
– Pictures were coded for cable transfer and
reconstructed at the receiving end on a
telegraph printer
 11
of
92
History of DIP (cont…)
Mid to late 1920s: Improvements to the
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Bartlane system resulted in higher quality

images
– New reproduction
processes based
on photographic
techniques
– Increased number
of tones in Improved
digital image Early 15 tone digital
reproduced images image
 12
of
92
History of DIP (cont…)
1960s: Improvements in computing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

technology and the onset of the space race

led to a surge of work in digital image
processing
– 1964: Computers used to
improve the quality of
images of the moon taken
by the Ranger 7 probe
– Such techniques were used
A picture of the moon taken
in other space missions by the Ranger 7 probe
including the Apollo landings minutes before landing
 13
of
92
History of DIP (cont…)
1970s: Digital image processing begins to
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

be used in medical applications

– 1979: Sir Godfrey N.
Hounsfield & Prof. Allan M.
Cormack share the Nobel
Prize in medicine for the
invention of tomography,
the technology behind
Computerised Axial Typical head slice CAT
Tomography (CAT) scans image
14
of
92
History of DIP (cont…)
1980s - Today: The use of digital image
processing techniques has exploded and
they are now used for all kinds of tasks in all
kinds of areas
– Image enhancement/restoration
– Artistic effects
– Medical visualisation
– Industrial inspection
– Law enforcement
– Human computer interfaces
 15
of
92
Examples: Image Enhancement
One of the most common uses of DIP
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

techniques: improve quality, remove noise

etc
16
of
92
Examples: The Hubble Telescope
Launched in 1990 the Hubble
telescope can take images of
very distant objects
However, an incorrect mirror
made many of Hubble’s
images useless
Image processing
techniques were
used to fix this
17
of
92
Examples: Artistic Effects
Artistic effects are
used to make
images more
visually appealing,
to add special
effects and to make
composite images
 18
of
92
Examples: Medicine
Take slice from MRI scan of canine heart,
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

and find boundaries between types of tissue

– Image with gray levels representing tissue
density
– Use a suitable filter to highlight edges

Original MRI Image of a Dog Heart Edge Detection Image

 19
of
92
Examples: GIS
Geographic Information Systems
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

– Digital image processing techniques are used

extensively to manipulate satellite imagery
– Terrain classification
– Meteorology
 20
of
92
Examples: GIS (cont…)
Night-Time Lights of
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

the World data set

– Global inventory of
human settlement
– Not hard to imagine
the kind of analysis
that might be done
using this data
 21
of
92
Examples: Industrial Inspection
Human operators are
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

expensive, slow and

unreliable
Make machines do the
job instead
Industrial vision systems

are used in all kinds of

industries
Can we trust them?
22
of
92
Examples: PCB Inspection
Printed Circuit Board (PCB) inspection
– Machine inspection is used to determine that
all components are present and that all solder
joints are acceptable
– Both conventional imaging and x-ray imaging
are used
 23
of
92
Examples: Law Enforcement
Image processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

techniques are used

extensively by law
enforcers
– Number plate
recognition for speed
cameras/automated
toll systems
– Fingerprint recognition
– Enhancement of
CCTV images
24
of
92
Examples: HCI
Try to make human computer
interfaces more natural
– Face recognition
– Gesture recognition
Does anyone remember the
user interface from “Minority
Report”?
These tasks can be
extremely difficult
25
of
92
Key Stages in Digital Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 26
of
Key Stages in Digital Image Processing:
92 Image Aquisition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 27
of
Key Stages in Digital Image Processing:
92 Image Enhancement
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 28
of
Key Stages in Digital Image Processing:
92 Image Restoration
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 29
of
Key Stages in Digital Image Processing:
92 Morphological Processing
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 30
of
Key Stages in Digital Image Processing:
92 Segmentation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 31
of
Key Stages in Digital Image Processing:
92 Object Recognition
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
 32
of
Key Stages in Digital Image Processing:
92 Representation & Description
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
33
of
Key Stages in Digital Image Processing:
92 Image Compression

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
34
of
Key Stages in Digital Image Processing:
92 Colour Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
35
of
92
Summary
We have looked at:
– What is a digital image?
– What is digital image processing?
– History of digital image processing
– State of the art examples of digital image
processing
– Key stages in digital image processing
36
of
92
Contents
This lecture will cover:
The human visual system
Light and the electromagnetic spectrum
Image representation
Image sensing and acquisition
Sampling, quantisation and resolution
37
of
92
Human Visual System
The best vision model we have!
Knowledge of how images form in the eye
can help us with processing digital images
We will take just a whirlwind tour of the
human visual system
 38
of
92
Structure Of The Human Eye
The lens focuses light from objects onto the
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

retina
The retina is covered with
light receptors called
cones (6-7 million) and
rods (75-150 million)
Cones are concentrated
around the fovea and are
very sensitive to colour
Rods are more spread out
and are sensitive to low levels of illumination
39
of
92
Blind-Spot Experiment
Draw an image similar to that below on a
piece of paper (the dot and cross are about
6 inches apart)

Close your right eye and focus on the cross

with your left eye
Hold the image about 20 inches away from
your face and move it slowly towards you
The dot should disappear!
40
of
92
Image Formation In The Eye
Muscles within the eye can be used to
change the shape of the lens allowing us
focus on objects that are near or far away
An image is focused onto the retina causing
rods and cones to become excited which
ultimately send signals to the brain
41
of
92
Brightness Adaptation & Discrimination

The human visual system can perceive

approximately 1010 different light intensity
levels
However, at any one time we can only
discriminate between a much smaller
number – brightness adaptation
Similarly, the perceived intensity of a region
is related to the light intensities of the
regions surrounding it
 42
of
Brightness Adaptation & Discrimination
92 (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

An example of Mach bands

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
43
Brightness Adaptation & Discrimination
(cont…)
 44
of
Brightness Adaptation & Discrimination
92 (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

An example of simultaneous contrast

45
of
Brightness Adaptation & Discrimination
92 (cont…)

For more great illusion examples take a look at: http://web.mit.edu/persci/gaz/

46
of
92

Available here: http://www.lottolab.org/Visual%20Demos/Demo%2015.html

47
of
92
Optical Illusions

Our visual
systems play lots
of interesting
tricks on us
48
of
92
Light And The Electromagnetic Spectrum

Light is just a particular part of the

electromagnetic spectrum that can be
sensed by the human eye
The electromagnetic spectrum is split up
according to the wavelengths of different
forms of energy
49
of
92
Reflected Light
The colours that we perceive are determined
by the nature of the light reflected from an
object
For example, if white
light is shone onto a Wh
ite Ligh
green object most t
Colours

wavelengths are Absorbed

Light
absorbed, while green
n
Gree

light is reflected from

the object
50
of
92
Sampling, Quantisation And Resolution

In the following slides we will consider what

is involved in capturing a digital image of a
real-world scene
Image sensing and representation
Sampling and quantisation
Resolution
 51
of
92
Image Representation
Before we discuss image acquisition recall
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

that a digital image is composed of M rows

and N columns of pixels
each storing a value col

Pixel values are most

often grey levels in the
range 0-255(black-white)
We will see later on
that images can easily
be represented as f (row, col)

matrices row
 52
of
92
Image Acquisition
Images are typically generated by
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

illuminating a scene and absorbing the

energy reflected by the objects in that scene
– Typical notions of
illumination and
scene can be way off:
• X-rays of a skeleton
• Ultrasound of an
unborn baby
• Electro-microscopic
images of molecules
 53
of
92
Image Sensing
Incoming energy lands on a sensor material
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

responsive to that type of energy and this

generates a voltage
Collections of sensors are arranged to
capture images

Imaging Sensor

Line of Image Sensors

Array of Image Sensors
 Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
54

Using Sensor Strips and Rings

Image Sensing
 55
of
92
Image Sampling And Quantisation
A digital sensor can only measure a limited
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

number of samples at a discrete set of

energy levels
Quantisation is the process of converting a
continuous analogue signal into a digital
representation of this signal
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
56
Image Sampling And Quantisation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
57
Image Sampling And Quantisation
 58
of
Image Sampling And Quantisation
92 (cont…)
Remember that a digital image is always
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

only an approximation of a real world

scene
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
59
Image Representation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
60
Image Representation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
61
Image Representation
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
62
Image Representation
63
of
92
Spatial Resolution
The spatial resolution of an image is
determined by how sampling was carried out
Spatial resolution simply refers to the
smallest discernable detail in an image
Vision specialists will
often talk about pixel
size
Graphic designers will
talk about dots per 5.1
g a pixel
Me
inch (DPI) s
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
64
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
65
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
66
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
67
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
68
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
69
Spatial Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
70
Spatial Resolution (cont…)
71
of
92
Intensity Level Resolution
Intensity level resolution refers to the
number of intensity levels used to represent
the image
The more intensity levels used, the finer the level of detail
discernable in an image
Intensity level resolution is usually given in terms of the
number of bits used to store each intensity level
Number of Intensity
Number of Bits Examples
Levels
1 2 0, 1
2 4 00, 01, 10, 11
4 16 0000, 0101, 1111
8 256 00110011, 01010101
16 65,536 1010101010101010
 72
of
92
Intensity Level Resolution (cont…)
256 grey levels (8 bits per pixel) 128 grey levels (7 bpp) 64 grey levels (6 bpp) 32 grey levels (5 bpp)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

16 grey levels (4 bpp) 8 grey levels (3 bpp) 4 grey levels (2 bpp) 2 grey levels (1 bpp)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
73
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
74
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
75
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
76
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
77
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
78
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
79
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
80
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
81
Saturation & Noise
82
of
92
Resolution: How Much Is Enough?
The big question with resolution is always
how much is enough?
This all depends on what is in the image and what
you would like to do with it
Key questions include
• Does the image look aesthetically pleasing?
• Can you see what you need to see within the
image?
83
of
Resolution: How Much Is Enough?
92 (cont…)

The picture on the right is fine for counting

the number of cars, but not for reading the
number plate
 84
of
92
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Low Detail Medium Detail High Detail

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
85
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
86
Intensity Level Resolution (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

of
92
87
Intensity Level Resolution (cont…)
88
of
92
Summary
We have looked at:
Human visual system
Light and the electromagnetic spectrum
Image representation
Image sensing and acquisition
Sampling, quantisation and resolution
Next time we start to look at techniques for
image enhancement