‫بسم هللا الرحمن‬

‫الرحيم‬

Computer Vision By Dr Mohamed Berbar 1

Pattern recognition and

Image Processing
Dr. Mohamed Berbar

Computer Vision By Dr Mohamed Berbar 2

1
 Recommended Texts

Rafael C Gonzalez, Richard E Woods 2nd Edition,

Digital Image Processing using Matlab
- Pearson Education

Milan Sonka, Vaclav Hlavac, Roger Boyle

Image Processing, Analysis, and Machine Vision

Computer Vision By Dr Mohamed Berbar 3

Unit 1
Introduction
What is “image processing
and computer vision”?

2
 What is “Image Processing and
Computer Vision”?

Image Processing Image Pattern Recognition

manipulate image data process image data
generate another image generate symbolic data

Computer Vision By Dr Mohamed Berbar 5

1. Introduction
* Image Processing
• image in → image out

* Image Analysis
• image in → measurements out

* Image Understanding(Image Pattern

Recognition)
• image in → high-level description out
Computer Vision By Dr Mohamed Berbar 6

3
 Sources of images:
1.Digital cameras
2.Scanners
3. Instruments which produce digital images from X-rays, gamma rays,
and ultrasound waves
4. Satellite sensors from measurements of reflected or emitted visible,
infrared, or microwave radiation.
Computer Vision By Dr Mohamed Berbar 7

Output

Image Description

Input
Image Image Pattern
Image Processing Recognition,
Computer Vision

Description Computer Other Data

Computer Vision By Dr Mohamed Berbar
Graphics Processing 8

4
 Digital Image Definitions
• A digital image a [m, n] described in a 2D discrete space is derived
from an analog image a (x, y) in a 2D continuous space through a
sampling process that is frequently referred to as digitization.
• A digital image is a two dimensional matrix where its elements are
called pixels (abbreviation of picture elements (

• The 2D continuous image a (x, y) is divided into N rows and M

columns.
• The intersection of a row and a column is termed a pixel.
• The value assigned to the integer coordinates [m, n] with {m =
0,1,2,...,M -1} and {n = 0,1,2,...,N - 1} is a [m, n].

Computer Vision By Dr Mohamed Berbar 9

The image shown in Figure 1 has been divided into N = 16

rows and M = 16 columns. The value assigned to every pixel is
the average brightness in the pixel rounded to the nearest
integer value. The process of representing the amplitude of the
2D signal at a given coordinate as an integer value with L
different gray levels is usually referred to as amplitude
quantization or simply quantization.
Computer Vision By Dr Mohamed Berbar 10

10

5
 The process of displaying an image creates a graphical
representation of this matrix where the pixel values are assigned a
particular grey-level (monochromatic image) or a particular color
Pixels with low values are assigned dark grey-levels, and high pixel
values are assigned bright grey-levels .
Below is a graphical representation of the image of a gull

Computer Vision By Dr Mohamed Berbar 11

11

• The image shown is a small image of

size 10x10 extracted from around the
eye of the seagull.
The user can zoom or "expand" the region for
better visualization.
The user can also display or "print" the values of
the 10x10 ROI image
142 174 164 218 250 255 250 252 255 255
107 107 102 80 127 174 237 218 252 255
90 34 24 34 34 24 51 88 127 164
80 26 19 53 34 19 24 85 117 137
78 76 34 44 26 26 34 24 71 90
85 85 90 26 26 26 26 34 76 83
88 102 90 53 26 26 34 73 85 78
102 110 105 90 98 105 105 110 107 93
107 115 110 110 110 117 115 110 107 102
105 110 110 117 110 132 115 110 107 105
Computer Vision By Dr Mohamed Berbar 12

12

6
 Pixels: A "pixel" (picture element) is a relative unit and has
no absolute size. If you divide a picture into any number of
square samples of any size, you could call these samples as
pixels. It doesn't matter what size, only that they are all the
same size.

Image resolution: When a specification says "300

pixels per inch," it means that the device whether it's a
scanner or a printer divides images into 300 separate samples
along each inch of the paper. The number of samples per
inch, the scanner or video device uses to capture a picture, is
called the input resolution.

Computer Vision By Dr Mohamed Berbar 13

13

Bands: A scene may have several

images associated with it
Examples:
1. Color images Red Green
Blue

2. Multispectral scanner instruments on

aircrafts and satellites typically gather
from 3 to 11 images

Computer Vision By Dr Mohamed Berbar 14

14

7
 2. Applications (Examples)

1. Character recognition (OCR)

Examples: Printed text, Hand-printed text,
Cursive text

2. Biometry(Using personal characteristics

to identify a person

Computer Vision By Dr Mohamed Berbar 15

15

2.1 Character recognition (OCR)

Example: Printed text
• Characteristics
– regular shape
– regular orientation
– good contrast

• Can compare characters with a

prototype

Computer Vision By Dr Mohamed Berbar 16

16

8
 Input Output

Template

Non-Normalised Correlation
120

100

80

60

40

20

0
1
26
51
76
101
126
151
176
201
226
251
276
301
326
351
376
401
426
Computer Vision By Dr Mohamed Berbar Position 17

17

2.1 Character recognition (OCR)

Examples: Hand Printed Text

• Characteristics
– less regularity
• Must examine components of character

Computer Vision By Dr Mohamed Berbar 18

18

9
 Character recognition (OCR)
Examples: Cursive Text

• Totally irregular
• Careful analysis of strokes

Computer Vision By Dr Mohamed Berbar 19

19

2.2 Biometry (Using personal

characteristics to identify a person
• Using personal characteristics to
identify a person
– fingerprints
– face
– iris
– DNA

– etc

Computer Vision By Dr Mohamed Berbar 20

20

10
 Iris Scan

• Striations on iris are individually

unique
• Obvious applications
– security
– PIN

Computer Vision By Dr Mohamed Berbar 21

21

Locate the eye in the head image

Radial resampling of iris

} fixed number of samples

Analysis
Numerical description

Computer Vision By Dr Mohamed Berbar 22

22

11
 3. System Overview

Captured data

Enhancement

Feature Extraction

Feature Recognition

Image Recognition

Labels

Computer Vision By Dr Mohamed Berbar 23

23

Image Representation

• Image capture
• Image resolution
• Colour representation
• Camera calibration
• Parallels with human visual system

Computer Vision By Dr Mohamed Berbar 24

24

12
 Image Capture

• Many sources
– consider requirements of system
• resolution
– type of data
• transducer

Computer Vision By Dr Mohamed Berbar 25

25

Representation

• Sampled data
– spatial
– amplitude

Computer Vision By Dr Mohamed Berbar 26

26

13
 Image Resolution

• How many pixels

– spatial resolution
• How many shades of grey/colours
– amplitude resolution
• How many frames per second
– temporal resolution

Computer Vision By Dr Mohamed Berbar 27

27

Spatial Resolution

n, n/2, n/4, n/8, n/16 and n/32 pixels on a side.

Computer Vision By Dr Mohamed Berbar 28

28

14
 Amplitude Resolution

• Humans can see:

– about 40 shades of brightness
– about 7.5 million shades of colour
• Cameras can see:
– depends on signal to noise ratio
– 40 dB equates to about 20 shades
• Images captured:
– 256 shades

Computer Vision By Dr Mohamed Berbar 29

29

Shades of Grey

8, 4, 2 and 1 bit images.

Computer Vision By Dr Mohamed Berbar 30

30

15
 4. Images in Matlab

• Matlab is optimised for operating on matrices

• Images are matrices!
• Many useful built-in functions in the Matlab Image
Processing Toolbox
• Very easy to write your own image processing
functions

31

31

32

• Data types in MATLAB

– Double (64-bit double-precision floating point)
– Single (32-bit single-precision floating point)
– Int32 (32-bit signed integer)
– Int16 (16-bit signed integer)
– Int8 (8-bit signed integer)
– Uint32 (32-bit unsigned integer)
– Uint16 (16-bit unsigned integer)
– Uint8 (8-bit unsigned integer)
– char (2 bytes per element).
– Logical Values are 0 or 1 (1 byte per element).

32

16
 33

• The toolbox supports four types of images:

o Intensity images : [0,1] or uint8, double etc.

o Binary images : {0,1}

o Indexed images : m-by-3 color map

o RGB images : m-by-n-by-3

33

34

• MATLAB stores an intensity image as a single matrix,

with each element of the matrix corresponding to one
image pixel.

• The matrix can be of class double, in which case it

contains values in the range [0,1], or of class uint8, in
which case the data range is [0,255] (0 – black, 255 –
white).

34

17
 35

• In a binary image, each pixel assumes one of only

two discrete values.
• Essentially, these two values correspond to on
and off.
• A binary image is stored as a two-dimensional
matrix of 0’s (off pixels) and 1’s (on pixels).
• Example

35

36

• The colormap is an m-by-3 matrix of class double.

Each row of the colormap matrix specifies the
red, green, and blue (RGB) values for a single
color:
– color = [R G B]
Example

36

18
 37

Like an indexed image, an RGB image represents

each pixel color as a set of three values, representing
the red, green, and blue intensities that make up the
color.
• Unlike an indexed image,
however, these intensity values
are stored directly in the image
array, not indirectly in a
colormap
37

Intensity Images

An intensity image is a data matrix whose value have

been scaled to represent intensities. When the element
of an intensity image are of class unit8, or class
unit16, they have integer value in the range of [0, 255]
and [0, 65535] respectively.

If the image is of class double, the values are of

floating point numbers. Values of scaled, class double
intensity images are in range [0,1] by conventions.

© Dept. of Computer Science, Swami Vivekanand Mahavidyalay, Udgir (Mah )

38

19
 Binary Images

Binary images having very specific meaning in

MATLAB. A binary image is a logical array of 0s and
1s. Thus an arrays of 0s and 1s whose values are of
data class, say, unit8, is not considered a binary image
in MATLAB. Thus, if A is a numeric array consisting
of 0s and 1s, we create a logical array B using the
statement.

B= logical(A);

39

Indexed Images

An indexed image consists of a data matrix, X, and a

colormap matrix, map. map is an m-by-3 array of
class double containing floating-point values in the
range [0, 1]. Each row of map specifies the red, green,
and blue components of a single color. An indexed
image uses "direct mapping" of pixel values to
colormap values. The color of each image pixel is
determined by using the corresponding value of X as
an index into map. The value 1 points to the first row
in map, the value 2 points to the second row, and so
on. You can display an indexed image with the
statements
© Dept. of Computer Science, Swami Vivekanand Mahavidyalay, Udgir (Mah )

40

20
 RGB Images
An RGB image, sometimes referred to as a
"truecolor" image, is stored in MATLAB as an m-by-
n-by-3 data array that defines red, green, and blue
color components for each individual pixel. RGB
images do not use a palette. The color of each pixel is
determined by the combination of the red, green, and
blue intensities stored in each color plane at the
pixel's location. Graphics file formats store RGB
images as 24-bit images, where the red, green, and
blue components are 8 bits each. This yields a
potential of 16 million colors. The precision with
which a real-life image can be replicated has led to
the nickname "truecolor image."
© Dept. of Computer Science, Swami Vivekanand Mahavidyalay, Udgir (Mah )

41

Display a Grayscale Image

corn_gray = imread('corn.tif',3);
// The grayscale version of the image is
the third image in the file
imshow(corn_gray);

Display an Indexed Image

[corn_indexed,map] = imread('corn.tif’,1);
//the image is the first image in the file
imshow(corn_indexed,map);

Display an RGB Image

[corn_rgb] = imread('corn.tif',2);
imshow(corn_rgb)

Display a Binary Image

[corn_gray] = imread('corn.tif’,3);
meanIntensity = mean(corn_gray(:));
corn_binary = corn_gray > meanIntensity;
42

21
 https://www.mathworks.com/help/images/ref/imshow.html

Dr Mohamed Berbar 43

43

44

• gray2ind - intensity image to index image

• im2bw - image to binary
• im2double - image to double precision
• im2uint8 - image to 8-bit unsigned integers
• im2uint16 - image to 16-bit unsigned
integers
• ind2gray - indexed image to intensity image
• mat2gray - matrix to intensity image
• rgb2gray - RGB image to grayscale
• rgb2ind - RGB image to indexed image

44

22
 Image Arithmetic Functions
• The following table lists the image arithmetic functions.
For more complete descriptions, see their reference
pages.
Function Description
•
imabsdiff Absolute difference of two images
•
imadd Add two images
•
imcomplement Complement an image
•
imdivide Divide two images
•
imlincomb Compute linear combination of two
images
•
immultiply Multiply two images
•
45

Loading and displaying images

>> I=imread('mandrill.bmp','bmp'); % load image

image format
Matrix with image filename
image data as a string
>> image(I) % display image
>> whos I

Name Size Bytes Class

I 512x512x3 786432 uint8 array

Grand total is 786432 elements using 786432 bytes

Matlab can only
Dimensions of I (red,
perform arithmetic
green and blue intensity
operations on data
information)
with class double! 46

46

23
 Representation of Images
• Images are just an array of numbers
>> I % ctrl+c to halt output!

• Intensity of each pixel is represented by the pixel element’s

value in the red, green and blue matrices
>> I(1,1,:) % RGB values of element (1,1)
ans(:,:,1) =
135 Red
Images where the pixel value in
ans(:,:,2) = the image represents the intensity
97 Green of the pixel are called intensity
ans(:,:,3) = images.
33 Blue

47

47

Indexed images
• An indexed image is where the pixel values are indices to elements in
a colour map or colour lookup table.
• The colour map will contain entries corresponding to red, green and
blue intensities for each index in the image.
>> jet(20) % Generate a jet colourmap for 20 indices
ans =
0 0 0.6000
0 0 0.8000
0 0 1.0000
RGB Entry for index value 3 3 4 7 3 6 19 8 9 1 2
0 0.2000 1.0000 5 6 14 4 2 5 6 1 4 5
0 0.4000 1.0000 2 8 9 4 2 13 7 8 4 5
0 0.6000 1.0000 5 1 11 5 6 4 1 7 4 4
0 0.8000 1.0000 1 9 5 6 5 5 14 4 6 5
0 1.0000 1.0000 5 9 2 1 11 1 3 6 1 9
0.2000 1.0000 0.8000 7 6 8 18 1 8 1 9 13 3
0.4000 1.0000 0.6000 Values can range 9 2 3 7 2 9 8 16 6 4
0.6000 1.0000 0.4000
from 0.0 to 1.0 7 8 6 7 4 15 8 2 1 3
0.8000 1.0000 0.2000
1.0000 1.0000 0
7 5 10 8 4 10 4 3 6 4
1.0000 0.8000 0
1.0000 0.6000 0
1.0000 0.4000 0 Red, green and blue intensities
1.0000 0.2000 0
1.0000 0 0 of the nearest index in the
0.8000 0 0
0.6000 0 0
colourmap are used to display
the image.

Image Processing using Matlab Sumitha Balasuriya 48

48

24
 Displaying indexed images
>> I2=I(:,:,2); % green values of I
>> image(I2)
>> colorbar % display colourmap
Matlab considers I2 as an indexed image as it doesn’t
contain entries for red, green and blue entries

Index

Associated
color

Colour
Lookup
Table

Image Processing using Matlab Sumitha Balasuriya 49

49

What is Image Histogram?

Measure frequency of occurrence of each
grey/colour value

Grey Value Histogram

12000

10000

8000
Frequency

6000

4000

2000

0
106
121
136
151
166
181
196
211
226
241
256
1
16
31
46
61
76
91

Grey Value

Computer Vision By Dr Mohamed Berbar 50

50

25
 (a) (b) (c)
The one dimensional histogram: The histogram of image in (a)
may be represented as a table (b) or as a plot ( c)
Computer Vision By Dr Mohamed Berbar 51

51

Histograms
• Frequency of the intensity values of the image
• Quantise frequency into intervals (called bins)
• (Un-normalised) probability density function of
image intensities

Image Processing using Matlab Sumitha Balasuriya 52

52

26
 Visualising the intensity surface
>>surf(double(imresize(Lena(:,:,2),[50 50])))

Change type to Remember to reduce

double precision size of image!

Use Matlab’s built-in mesh and

shading surface visualisation
functions

53

53

Useful functions for manipulating

images

• Convert image to grayscale

>>Igray=rgb2gray(I);
• Resize image
>>Ismall=imresize(I,[100 100], 'bilinear');
• Rotate image
>>I90=imrotate(I,90);

54

54

27
 Other useful functions
Convert polar coordinates to Check if a variable is null Trigonometric functions
cartesian coordinates >>isempty(I) sin, cos, tan
>>pol2cart(rho,theta)
Convert polar coordinates to Find indices and elements in a Fast Fourier Transform
cartesian coordinates matrix
>>cart2pol(x,y) >>[X,Y]=find(I>100) fft2(I
)
Get size of matrix Change the dimensions of a Discrete Cosine Transform
>>size(I) matrix
>>reshape(rand(10,10),[100 1])
dct(I)
Add elements of a Matrix Exponentials and Logarithms
(columnwise addition in matrices) exp
>>sum(I) log
log10

Image Processing using Matlab Sumitha Balasuriya 55

55

Colour Representation

• Newton
– white light composed of seven colours
• red, orange, yellow, green, blue, indigo,
violet
– three primaries could approximate
many colours
• red, green, blue

Computer Vision By Dr Mohamed Berbar 56

56

28
 Other Colour Models

• YMCK
• IHS
• YCrCb
• etc

Computer Vision By Dr Mohamed Berbar 57

57

Color models
• They provide a standard way of specifying a
particular color using a 3D coordinate system.
• Hardware oriented
– RGB: additive system (add colors to black)
used for displays.
– CMY: subtractive system used for printing.
– YIQ: used for TV and is good for compression.
• Image processing oriented
– HSV: good for perceptual space for art,
psychology and recognition.

Adapted from Octavia Camps, Penn State

©2007, Selim Aksoy 58

58

29
 Additive and subtractive
colors

CS 484, Spring 2007 59

59

Subtractive Color

• When we mix colors using paint, or

through the printing process, we are
using the subtractive color method.
• Subtractive color mixing means that
one begins with white and ends with
black; as one adds color, the result gets
darker and tends to black.

CSC 558 By Dr Mohamed Berbar 60

60

30
 Additive Color

• if we are working on a computer, the

colors we see on the screen are created
with light using the additive color
method. Additive color mixing begins
with black and ends with white; as more
color is added, the result is lighter and
tends to white.

CSC 558 By Dr Mohamed Berbar 61

61

RGB model
• Additive model.
• An image consists of 3 bands, one for each primary color.
• Appropriate for image displays.

Adapted from Gonzales and Woods

CS 484, Spring 2007 ©2007, Selim Aksoy 62

62

31
 CMY model

• Cyan-Magenta-Yellow is a subtractive model which is

good to model absorption of colors.
• Appropriate for paper printing.

 C  1  R 
 M  = 1 − G 
    
 Y  1  B 

Adapted from Octavia Camps, Penn State

CS 484, Spring 2007 ©2007, Selim Aksoy 63

63

CIE chromaticity model

• The Commission Internationale de l’Eclairage defined 3 standard

primaries: X, Y, Z that can be added to form all visible colors.
• Y was chosen so that its color matching function matches the
sum of the 3 human cone responses.

 X  0.6067 0.1736 0.2001  R 

Y  = 0.2988 0.5868 0.1143  G 
    
 Z  0.0000 0.0661 1.1149   B 
 R   1.9107 − 0.5326 − 0.2883  X 
G  = − 0.9843 1.9984 − 0.0283  Y 
    
 B   0.0583 − 0.1185 0.8986   Z 
CS 484, Spring 2007 ©2007, Selim Aksoy 64
Adapted from Octavia Camps, Penn State

64

32
 YIQ model

• Have better compression properties.

• Luminance Y is encoded using more bits than
chrominance values I and Q (humans are more sensitive
to Y than I and Q).
• Luminance used by black/white TVs.
• All 3 values used by color TVs.

Y  0.299 0.587 0.114   R 

 I  = 0.596 − 0.275 − 0.321 G 
    
Q  0.212 − 0.532 0.311   B 
Adapted from Octavia Camps, Penn State
CS 484, Spring 2007 ©2007, Selim Aksoy 65

65

HSV model
• HSV: Hue, saturation, value are non-linear functions of
RGB.
• Hue relations are naturally expressed in a circle.
( R+G+B )
I=
3
min( R, G, B )
S = 1−
I
 1 / 2[( R − G )+( R − B )] 
H = cos −1   if BG

 ( R − G ) 2 +( R − B )(G − B )  
 1 / 2[( R − G )+( R − B )] 
H = 360 − cos −1   if BG

 ( R − G ) 2 +( R − B )(G − B )  
Adapted from Octavia Camps, Penn State
CS 484, Spring 2007 ©2007, Selim Aksoy 66

66

33
 HSV model

(Left) Image of food originating from a digital camera.

(Center) Saturation value of each pixel decreased 20%.
(Right) Saturation value of each pixel increased 40%.

Adapted from Linda Shapiro, U of Washington

CS 484, Spring 2007 ©2007, Selim Aksoy 67

67

Color models

CMYK

RGB

HSV

Adapted from Gonzales and Woods

CS 484, Spring 2007 ©2007, Selim Aksoy 68

68

34
 Examples: segmentation
• Can cluster on color values and pixel locations.
• Can use connected components and an approximate color
criteria to find regions.
• Can train an algorithm to look for certain colored regions – for
example, skin color.

Original RGB image Color clusters by k-means

CS 484, Spring 2007 ©2007, 69
Adapted from Linda Shapiro, U ofSelim Aksoy
Washington

69

Examples: segmentation
Skin color in RGB
space:
Purple region shows
skin color samples
from several people.
Blue and yellow
regions show skin in
shadow or behind a
beard.

Adapted from Linda Shapiro, U of Washington

CS 484, Spring 2007 ©2007, Selim Aksoy 70

70

35
 Examples: segmentation

(Left) Input video frame.

(Center) Pixels classified according to RGB space.
(Right) Largest connected component with aspect similar to a
face.

Adapted from Linda Shapiro, U of Washington

CS 484, Spring 2007 ©2007, Selim Aksoy 71

71

• A Pattern Recognition system is composed of

1.Pre-processing
2.Feature Extraction
3.Classification
• Feature Extraction is a crucial step in
Pattern Recognition. It is responsible for
measuring features of objects in an image.
• In this experiment we have a binary image
with different objects. The feature used in
this illustrative example is the first invariant
moment. It measures the spread of pixels
from the centroid of the object.
Computer Vision By Dr Mohamed Berbar 72

72

36
 Computer Vision By Dr Mohamed Berbar 73

73

37