Chapter 1
Overview of Pattern Recognition
Introduction to Pattern Recognition
What is Pattern?
something exhibiting certain characteristics (regularity)
something able to serve as a model (Greek Beauty, Kyoto Beauty)
something representing a concept of what was observed or experienced (Sticky dish)
In engineering, usually represented as a set of measurements or observations in vector or mat
rix forms
Examples:
Introduction to Pattern Recognition
Pattern Recognition
to recognize given information (usually measures or observation) via information reduction, infor
mation mapping, or information labeling process.
Usually involves the concepts of classification, recognition, or description.
Definition
Classification
assigns input data into one or more of C pre-specified classes based on the extracted features.
Description
an alternative to classification where a conceptual or structural description of the input pattern is desire
d.
Recognition
ability to classify , even identify in many cases.
Preprocessing
filtering and transformation of raw input to minimize noise and to extract rather reliable features.
Postprocessing
Refining the classification or recognition results by utilizing properties of the pattern classes
Introduction to Pattern Recognition
Definition
Desirable Feature
distinct characteristics of data (measurements or observations).
allows representation of high dimensional raw measurements in a lower dimensional space
while preserving essential information, i.e., reducing the redundancy.
provides a good separable space for pattern classification.
examples: mean, variance, distribution type, etc.
desirable feature selection is the most essential one for success.
feature selection is a problem dependent task (the concept of tuning).
Noise
undesirable components in data
distortions or errors in raw input data (preprocessing can increase the distortion)
errors in feature extraction, errors in training data
More Classification Errors
Searching for Noise Robust Features
Introduction to Pattern Recognition
Major Four types of pattern recognition approaches
Tree-structure (or decision) based pattern recognition
Just imagine how to classify apples from oranges; usually intuitional tasks
But very powerful in many applications
Still rather hard to generalize the algorithm, i.e., training data dependency
Recommended to design ones own decision based pattern recognizer before imple
menting the main PR algorithm as a practice.
Statistical pattern recognition
statistical or decision theoretic approach
parametric and non-parametric methods; Most engineering tasks
Easy to generalize and usually produces rather good results for new unknown inputs:
Why?
Not easy to understand the inner operation of the PR algorithm
Introduction to Pattern Recognition
Major Four types of pattern recognition approaches
Syntactic pattern recognition
structural analysis on interrelationships or interconnections of features
requires quantification and extraction of structural information and then assesses structural similarit
y of patterns;
Very powerful for bio-pattern, character recognition, and NLP
Neural Network based Pattern recognition
Somehow a block box approach
involves neural networks for pattern recognition
(Given input Trained multilayer NN Classified output)
Almost impossible to analyze the internal decision behaviors; Hard to improve
Fuzzy Pattern recognition
Based on the Fuzzy theory
Not easy to implement but motion compensation for cameras can be an example.
Introduction to Pattern Recognition
Mappings in an abstract representation of pattern generation/classificati
on/interpretation systems
p4
class i
p1
class j
class
k
Class membership or
description space: C
m1
p2
m2
p3
m3
pattern space: P
Observation or
measurement space: F
Introduction to Pattern Recognition
Pattern recognition procedures
Preprocessing
Usually the first step after acquiring raw data from which pattern recognitio
n is performed.
Usually involves:
Noise reduction; Filtering
Adjusting or deduction of the average value of data (In many cases, including normal
ization)
Segmentation of region (object) of interest from the acquired raw data
Processing of the raw data and mapping the results into different representation dom
ains
Various transforms can be used such as Wavelet, Fast Fourier transforms
Introduction to Pattern Recognition
Pattern recognition procedures
Preprocessing noise reduction in speech signals
Primary signal
Noise reference signal
After spectral subtraction
After noise reduction
Introduction to Pattern Recognition
Pattern recognition procedures
Preprocessing noise reduction in image
Before noise reduction
After noise reduction
Introduction to Pattern Recognition
Pattern recognition procedures
Preprocessing segmentation
Region Tree
Object Tree
Root
R1
Region
Segmentation
R3
R2
R4
R5
R8
R9
R12
Body
TV
Jacket
Hair
Face
Torso
R7
R6
R10
R 11
R13
Introduction to Pattern Recognition
Pattern recognition procedures
Feature extraction
needs to be computationally feasible by manual or by automated procedure
s
Feature vector and feature space
Features are formed as a d-dimensional vector, called the feature vector.
Feature vectors are typically used in statistical and neural network based pattern reco
gnition.
Observation
environment
Feature extraction
process
X (feature vector)
X (string)
P (structure)
Input data preprocessing/feature extraction
Introduction to Pattern Recognition
Pattern recognition procedures
Pattern classification (Assuming we already have pre-trained patterns)
action of classifying the raw data into one of the pre-determined classes or patterns
What can we do if a raw datum from a totally unknown class is met?
Signal (information)
Data acquisition
Measure mi
Preprocessing and
enhancement
Feature extraction
Classification
Description
diagram of typical pattern recognition systems
Selection problem of features
An example of fish classification [textbook, Figure 1.1]
Selection problem of features
Which feature is better for good classification between salmo
n and sea bass classes?
Histogram of length feature for salmon and sea
bass classes
Histogram of lightness feature for salmon and sea
bass classes
Changing the position of the threshold affects the classification error!
Representation of Feature Vectors
Scatter Plot
plot of feature vectors in feature space
difficult to visualize feature vectors with greater than 3-D.
A two-class scatter plot
Some Terminologies
Classifiers
partitions feature space into class-labeled decision regions.
Separable or inseparable
If inseparable, any compromise is possible and allowed?
A decision region
represents the corresponding class in feature space.
Discriminant function
computes the class-likelihood value of a feature (vector) with respect to a certain class.
Adaptive or Self-learning .vs. Fixed
Classification Rule
Classification on features can be done based on compari
ng a set of discriminant functions
Decision rules: Assign
gm x gi x
to class
(region
i 1,2, , c and i m
Likelihood, Nearest, Measure, Similarity, .
Rm
) if
Classification Rule
Decision boundary divides decision regions in feature space where
g k x gl x
Linear
discriminant
functions
R1
R2
Decision
boundary
R3
Quadratic
discriminant
function
R1
R2
arbitrary
discriminant
function
R1
R2
R2
Decision
region
R3
R2
R1
R4
Classification Rule
Which decision boundary is the best linear, quadra
tic or arbitrary decision boundary?
Classification Rule
Generalization problem
For a robust classification performance with test data, the decision boundary needs to
be conformed in a robust way.
So, the training samples should represent the nature of the underlying data successful
ly
Sufficient number of training data?
The more, the better. But not practical if too much.
Experimentally, or intuitively, we have to decide the number with some
restraints such as small number of available data.
The classifiers should be sufficiently trained! (Not always possible!)
There are some tricky methods to recover the shortage of the training data but sufficie
nt training data are better!
More insight for the tasks, better pattern recognizers!
Development Procedure of PR systems
Step 1: Study the classes of patterns under consideration to develop possible characteri
zations. (ideation, logic process)
Step 2: Determine the availability of feature/measurement data
Step 3: Consider constraints on desired system performance,
computational resources, and the implementation cost (Engineering)
Step 4: Consider the availability of training data
Step 5: Consider the availability of suitable and known PR technologies
Step 6: Develop a PR system simulation
Step 7: Train the system
Step 8: Simulate system performance
Step 9: Iterate among the above steps until desired performance is
achieved.
If still not successful, then ?
Development Procedure of PR systems
Procedure of designing a pattern classifier
Typical pattern classifier
Summary
Relationship of pattern recognition to other areas
(Adaptive) signal processing and systems
training and learning of adaptive systems
Artificial Intelligence, Expert System
Optimization/estimation theory
parameter estimation towards optimization of systems
Fuzzy sets
Structural Modeling
Formal language, ..
Summary
Pattern recognition applications
Image preprocessing, segmentation and analysis, coding
Seismic /weather/hurricane analysis and forecast
Radar, Sonar signal classification/analysis
Face, Gesture recognition
Speech/Audio recognition/understanding, Speaker and language recognitio
n
Fingerprint, DNA identification
Character (letter or number) recognition
Handwriting analysis; Electronic Signature for e-commerce
Human bio-signals such as ECG and EEG signal analysis/understanding
Medical diagnosis (tumor, mammogram, germ, virus, cancer,.)
Others to be mentioned
Terms
Novel Patterns (New unknown input)
Supervised, Unsupervised, Reinforced(-ment) Learning
Good Features
Translation, Rotation, Scale Invariant
Noise, Environment Robust
Statistically Independent (No Redundancy)
Authors are saying in p.12 that: Although the pattern classification techniques presented in this
book cannot substitute for domain knowledge, they can be helpful in making the feature values
less sensitive to noise.
