Wednesday 24th March, 2010: 11:00 -13:00

Pattern Recognition
and Computer Vision
J M Blackledge
Stokes Professor
Dublin Institute of Technology
http://eleceng.dit.ie/blackledge

Distinguished Professor
Warsaw University of Technology

Lectures co-financed by the European Union in scope of the European Social Fund
 What is the Problem?
A fundamentally difficult one, i.e. ultimately, how
to simulate the human vision system including our
reasoning based on image perception
 The Problem with
Machine Vision
• To date, there is no complete theoretical model for
simulating the processes that take place when a
human interprets an image generated by the eye

• Machine vision is an elusive subject area in which

automatic inspection systems are advanced without
having a fully operational theoretical framework as
a guide

• The subject is therefore ‘littered’ with different

approaches, methods and algorithms that are not
necessarily part of any common theme
 Some Basic Questions
in Computer Vision
• What are the goals and constraints?

• What type of algorithm or set of algorithms

is required to effect vision?

• What are the implications for the process given

the types of hardware that might be available?

• What are the levels of representation required to

achieve vision?
 Related Subject Areas

Segmentation Related issues

& Feature include:
Detection
• Feature Correlation
• Edge Detection
• Geometry
Artificial • Topology
Digital Image Neural • Genetic Algorithms
Processing Networks • Image Compression
 Principal Publications

http://eleceng.dit.ie/papers/103.pdf
 Contents of Presentation I
Part I: Basic Pattern Recognition Methods

• Introduction and Overview

• Correlation Based Pattern Recognition

• Example Pre-processing Methods:

- Homomorphic Filter
- Histogram Equalisation
- Image Statistics
- Statistical Moments
- Binarization

• Edge Detection

• The Marr-Hildreth Algorithm

• Radon Transform Based Computer Vision and the Hough Transform

• Summary

• Q & A + Interval (10 Minutes)

 Contents of Presentation II
Part II: Fractal Computer Vision
• A Short Introduction to Fractal Geometry

• Computer Vision using Fractal Geometry

• Example Applications in Image Analysis:

- Growth of Micro-organisms
- Quality Control of Rolled Steel
- Cytopathology
- A Skin Cancer Screening System

• Summary

•Q&A
 Introduction:
Making Sense of Images
Optical Illusions
Seeing Objects in the Clouds
 The Sun and Vision:
Why do we see in the Visible Spectrum ?
Planck radiation law expressed in Energy per unit
range of wavelength is
 Vision and the
Rayleigh Scattering Effect

For a spherically uniform scatterer

 Why is the Sun Yellow ?

• The sun radiates most energy in Green

• Green is in the middle of the visual spectrum
• The atmosphere filters out Blue light
• The sun therefore appears to be Yellow
 Scattering in the
Visible Spectrum
• Scattering of EM waves in the visible spectrum provides images of
objects where the edges are well defined.

Information ~ Wavelength

• In the infrared region, edges are not so well defined because:

- infrared radiation scatters

from larger scale structures

- the emission of infrared

radiation from a body
tends to dominate, the
process of thermal diffusion
being more significant than
infrared scattering
 A Philosophical Question
• What would a species just as intelligent
ourselves see if it evolved on a suitable
planet orbiting a hotter (or cooler) sun ?

• Would it have developed Pythagoras’ Theorem if

its visual perception was based in the Infrared ?
 Pattern Recognition using
the Correlation Function
• Construct a template based on a replica of the
feature in an image that requires identification
and correlate the template data with the image

• The correlation image or surface will contain a

maximum value (a ‘peak’ or ‘point’) at the
positions in the image which matches the
template
Example of Pattern Recognition
by Digital Correlation
 The Auto-Covariance Function

• The correlation function and the covariance function are

two important metrics used in pattern recognition

• The problem is to decide what feature(s) of an image to

extract in order to generate a template that is robust and
relatively insensitive to noise

• The template is typically constructed by processing the

image first in order to isolate features that may be based
on pixel similarity, discontinuity, and statistical measures
 Limiting Conditions
• The orientation of the pattern must be the same
as that of the template: Fourier-Mellin Transform

• The scale of the pattern must be the same:

Wavelet (multi-resolution) image analysis

• The template should be a good representation of

the pattern

• In practice, this is not always possible and several

image processing methods are required to
implement this method of pattern recognition in
practice
 Example Pre-Processing Methods:
The Homomorphic Filter
• Basic Model:

Image = Illumination x Reflectance

• Assume that reflectance component consists of high

frequency (scattering) information that needs to be
recovered for pattern recognition algorithm

• HPF – High Pass Filter

 Histogram Equalization 1

Problem: Find a transform such that

 Histogram Equalization 2

Solution:
 Noise Reduction Algorithms
• Typically undertaken in Real or Fourier space,
e.g. moving average filter or low pass filter
respectively.

• For linear convolution/correlation type filters

the convolution/correlation theorem holds and
each real space filter has a Fourier based
equivalent.

• Many other moving window filters for which the

convolution/correlation theorem does not hold,
e.g. the Median filter
 Median Filter
• Moving window is applied to the image and
the median computed at each window position
• Of particular value for salt-and-pepper noise – noise spikes
 Image Statistics
Incoherent images have an unlimited range of
statistical distributions whereas coherent
images are of a negative exponential type, e.g.
Rayleigh distributed
Statistical Image Segmentation
Statistical Moments & m-order Entropy
 Binarization

Problem: How to
choose the threshold

Solution:
For bi-modal images
find min between the two modes
 Edge Detection

• One of the most important expects of the human visual

system is the way in which it appears to make use of the
outlines or edges of objects for recognition and the
perception of distance and orientation.

• This has led to a theory for the human visual system

which is based on the idea that the visual cortex contains a
complex of feature detectors that are tuned to edges and
segments of various widths and orientations.

• For this reason, the detection of the edges in an image can

play an important role in pattern recognition.
 What is Edge Detection ?

• Edge detection is basically a method of segmenting an

image into regions of discontinuity; it allows the
observer to identify those features of an image where
there is a more or less abrupt change in grey level
indicating the end of one region in the image and the
beginning of another

• Like other methods of image analysis, edge detection is

sensitive to noise and for this reason, detected edges
can occur in places where the transition between
regions is not abrupt enough or else edges can be
detected in regions of an image that are uniform
 Approaches to Edge Detection
• First order edge detection

• Second order edge detection

 Digital Gradients 1
• Forward Differencing:

• Equivalent to digital convolution with a mask

 Digital Gradients 2
• Centre Differencing

• Magnitude of gradient

• Angle of gradient
 Edge Detectors
• There are a range of edge detectors
(i.e. different masks) which attempt to:
- provide continuous edges
- have robustness to noise

• Examples include:
Prewitt, Sobel, Compass and Canny

• All are FIR-type filters

Example of an Edge Detector:
The Sobel Detector
 The Marr-Hildreth Algorithm
On the theory of edge detection, Proceedings of
The Royal Society, London, B 207: 127-217, 1980

• One of the first approaches in pattern recognition to be based

on a model for the human visual system.

• The basic idea is that our ability to recognize and interpret

different objects in an image scene is based on matching the
edges of the scene over different frequency scales
 The Importance of Edges:
A Story from the Gulf War 1990/91
 Radon Transform Based
Computer Vision
Based on application of the Radon Transform
and Inverse Radon Transform
 The Hough Transform

• Although conceived independently, the Hough transform is a

special case of the Radon transform: the Radon Transform
of a point

• Describes a curve in Radon space with the characteristic

equation
 Example of the
Radon Transform
 Summary

• Pattern recognition is based on a range of image

processing methods designed to extract different
features in the image scene, e.g. edges

• There is no complete theoretical model for a vision

system and the subject of pattern recognition and
computer vision are dominated by a range of
paradigms, algorithms, methods and models
that are not connected other than in terms of a
common goal which is usually applications
dependent
 In the Following Lecture…
We shall consider the role of Fractal Geometry
in image analysis and pattern recognition, i.e.

Fractal Computer Vision

with applications in

Medical Imaging
 Questions
+
Interval (10 Minutes)
http://konwersatorium.pw.edu.pl/wy
klady/2010_VLZ7_06_wyklad.pdf
 Part II: Contents
Part II: Fractal Computer Vision
• A Short Introduction to Fractal Geometry
• Computer Vision using Fractal Geometry
• Example Applications:
- Growth of Micro-organisms
- Quality Control of Rolled Steel
- Cytopathology
- A Skin Cancer Screening System
• Summary
• Q&A
 A Short Introduction to
Fractal Geometry
Euclidean objects Fractal objects
copyright
 Euclidean Geometry
• Based on the theorems and results
associated with simple objects:
triangles, squares, circles, lines etc.

• Some abstract concepts,

e.g. two parallel lines meet at infinity

• Underlying philosophy: combine primitive

objects to construct complex ones - basis of
most man-made objects, computational
geometry, pattern recognition systems etc.
 Fractal Geometry
• Based on the theorems and results
associated with complex objects with
repeating patterns that are scale invariant

• Some abstract concepts,

e.g. repeating patterns continue to infinity

• Underlying philosophy: construct object by

finding simple underlying structure and then
repeat this structure again and again - basis
of natural objects and systems.
Points, Lines, Planes, Volumes and
Common (Integer) Dimensions
 Dimension
• We are all used to the concept of
dimensions 1, 2 and 3.

• The 4th dimension or time is also now

accepted thanks to Albert Einstein

• Higher dimensions, i.e. 5,6,7,8,… are

abstractions but nevertheless of
fundamental significance in modern
theoretical physics
 Dimension and Western Art
• Pre-renaissance art: 2D - flat paintings with
distortions in natural perspective

• Renaissance art: 3D - coming to terms with

perspective in paintings and taking on three
dimensional form – a re-birth of Greek/Roman
concepts and philosophy

• Cubist art: trying to express 4D in paintings.

• Computer graphics: attempts being made to

represent hyper-space.
Medievil Art - 2D Flatness
High Renaissance Art - 3D
 Cubism:
Trying to Representing 4D
 Fractional Dimensions:
Why Should Dimension Always be Integer?
Fundamental Definition
of the Fractal Dimension
Fractal Types
Self-Affine
Structures
 Islamic Art:
Stylised Versions of
Self-Repeating Patterns
Self-Similarity by M C Escher
Self-Similarity and J S Bach
 Fractals and Texture

copyright

“Much of Fractal Geometry can be considered

to be an intrinsic study of texture” B Mandelbrot
Texture by Claude Monet
Texture by Paul Signac
 Fractal Art:
CAD of Natural Objects

copyright
 Universal law of
Critical States

Critical states are governed by the

universal power law:

System(size) =constant . (size)-q

where q is a non-integer value.

 Scaling Law & Poisson’s Equation

• Coulomb’s law (and Newton’s law of gravity)

are based on the inverse square law:

• Result can be expressed as

 Scaling Law and the
Fractional Poisson Equation

• Random fractal self-affine image are characterised

by the spectral density law:

• Result can be expressed as

 Mandelbrot Surfaces
• Can be considered in terms of a solution to the
Fractional Poisson Equation for a white noise
source

• Use the Riesz definition of a fractional Laplacian

Fractal Clouds: D=2.1
Fractal Clouds: D=2.2
Fractal Clouds: D=2.3
Fractal Clouds: D=2.4
Fractal Clouds: D=2.5
Fractal Clouds: D=2.6
Fractal Clouds: D=2.7
Fractal Clouds: D=2.8
Fractal Clouds: D=2.9
Tailoring the Mandelbrot Surface
Sun in the Sky D=2.65
Fractal Flow, Divergent and
Rotational Fields
 Examples of Flow, Divergent
and Rotational Fractal Fields

Flow Divergent Rotational

 Self-Similarity and the Imagination
• Copernicus: Planets orbit the sun

• Kepler: Moons orbit the planets

• Bohr: Electrons orbit the nucleus (except for a

Quantum Mechanic who know better !!!)

• Rees: Galaxies orbit super-massive black holes

Same idea (in terms of images of the physical system) but at

different scales.
 Texture and Medicine

Normal Skin Chronic Dermatitis

 Computer Vision using Fractal
Geometry: Texture Analysis

Include Elements of the Feature Vector that

are based on Fractal Geometric Parameters
of an ‘object’ or ‘target’, e.g.

• Fractal Dimension
• Correlation Dimension
• Lacunarity

associated with boundary and/or surface

properties that are applications dependent
 Example of a Feature:
Fractal Dimension of a Boundary

D=1.61 D=1.68

86
 Machine Learning
Leaf 1 Membership Leaf 2
function
Precision

Leaf 2
Leaf 1

1.61 1.68

Fractal Dimension based Fuzzy Logic engine

 Illustration of Decision Making:
Non-Fuzzy Sets, Two Features

p2

Class A
Class B
Class C

88
p1
88
 Illustration of Decision Making:
Fuzzy Sets, 20 Features

p2

Class A
Class B
Class C

p20

p1
p3
89
Expert System Development
 Example Application of NDE 1:
Growth of Microorganisms
Relating Fractal Dimension to Branching Behaviour in Filamentous
Microorganisms, D Barry et al, ISAST Transactions on Electronics and Signal
Processing, Vol. 4, No. 1, 71 - 76, 2009; http://eleceng.dit.ie/papers/138.pdf
 Example Application of NDE 2:
Quality Control of Rolled Steel
A Surface Inspection Machine Vision System that Includes
Fractal Analysis J Blackledge and D Dubovitski, International Society
for Advanced Science and Technology, Journal of Electronic and Signal
Processing, Vol 3, No 2, 76 - 89, 2008
http://eleceng.dit.ie/papers/112.pdf
 Example Application of NDE 3:
Cytopathology
An Optical Machine Vision System for Applications in Cytopathology
J Blackledge and D Dubovitski, International Society for Advanced Science
and Technology, Journal of Electronic and Signal Processing
To be Published, 2010
 Example Application of NDE 4:
A Skin Cancer Screening System
Object Detection and Classification with Applications to Skin Cancer
Screening J Blackledge and D Dubovitski, International Society for
Advanced Science and Technology, Journal of Intelligent Systems, Vol 1,
No 1 (ISSN 1797-2329), 34 - 45, 2008; http://eleceng.dit.ie/papers/101.pdf

http://www.oxreco.com/setup.zip
 Why Bother?
• Over 5,700 new cases each year in the UK

• Manual screening achieves only

35% identification

• GP’s do not have the

expertise to diagnose skin cancer

• Cancer specialists improve identification rate

to over 65% but are severely overloaded
Commercialization
 Summary

• Inclusion of ‘Fractal Geometry’ significantly enhances the

design of optical computer vision systems for NDE when
images are of objects that are textured

• Getting the right ‘mix of parameters’ (i.e. the right mix of

Euclidean and Fractal parameters) is ‘as much an art as it is a
science’ – applications dependent

• Options in optical computer vision:

OPTION 1: Raw Data – Artificial Neural Network
OPTION 2: Processed Data – Fuzzy Logic Engine
OPTION 2 is preferable using Fractal Geometry for
texture analysis
 Q&A
http://konwersatorium.pw.edu.pl/wy
klady/2010_VLZ7_06_wyklad.pdf