PatternRecognitionProcesses

ARealExampleCaseStudy:
Fish Classification:
Sea Bass / Salmon.

Salmon

In our case, assume we have two

types of fish (sea-bass and
salmon) and we need to design a
pattern recognition system that
automatically separates between
them Sea-bass
TheAutomationSystemConsistsof:
Ɣ Conveyorbelt
forincoming
products
Ɣ Twoconveyor
beltsforsorted
products
Ɣ ApickͲandͲplace
roboticarm
Ɣ AvisionsystemwithanoverheadCCDcamera
Ɣ Acomputer toanalyzeimages andcontroltherobotarm
ImplementationofPatternRecognitionSystem
™AnExample
IMAGES FROM CCD
Thetask:Automate CAMERA

theprocessofsorting
incomingFish ona
conveyorbelt
accordingtoitstypes

Three basic
Separate seabass steps
from salmon
Example:“Seabass”vs.“Salmon”(Cont.)
™ StepI:Preprocessing
Anoisemaybecausedbythesensorsisrequiredtoberemoved(the
effectofnoisecanreducethereliabilityofthemeasuredfeature
values).

Goal:Preprocesstheimagecapturedbythecamera,suchthat
subsequentoperationscouldbesimplifiedforreducingthe
noisewithoutlosingrelevantinformation
Ɣ preprocessingsteps:
image • Adjustthelevelofillumination
processing • Removenoising
• Enhancethelevelofcontrast

• Isolatedifferentfishesfromone
segmentation another
• Isolatefishesfromthebackground
Example:“Seabass”vs.“Salmon”(Cont.)
™StepII:FeatureExtraction
isoneofthemostcriticalstepsinthepatternrecognitionsystemdesign
Thisstepisprocessedbymeasuringandselectingcertain"features"
or"properties"of thefishtobeclassified.

Goal: Extractfeatures(withgooddistinguishingability)from
thepreprocessedimagetobeusedforsubsequent
classification
Ɣ Examples of how to extract features:
Seabassisusually
“length” couldbeagood
longer than asalmon
candidateforfeatures
Seabassisusually “lightness”offishscales
brighter thanasalmon couldbeanothergood
candidateforfeatures
Example:“Seabass”vs.“Salmon”(Cont.)
™StepIII:Classification
Featuresarepassedtoaclassifier(model)thatevaluatesthe
measurements ofthe feature&makesadecision.

Goal:Todistinguishdifferenttypesofobjects(inthiscase,sea
bass vs.salmon)basedontheextractedfeatures
ProblemAnalysis
™ Some steps must be taken if we want to design the model
of the system that automate the process of sorting
incoming fish on a conveyor belt according to types
(Salmon or Sea bass).
¾ Set up a camera
¾ Take some sample images to extract features
¾ Note the physical differences between the two types
of fish
Length
Lightness
Width
No. & shape of fins
Position of the mouth
¾ This is the set of all suggested features to be used in our
classifier!
ProblemAnalysis

¾ A modeling step can be taken after feature

extraction and preprocessing steps as described
before according to the three main steps of a
"typical" PR system
Ͳ Preprocessing(SegmentationͲ Noisecleaning)
Ͳ Featureextraction
Ͳ Modeling(Classification)

Modeling: you need correctly to assign one mathematical

description for salmon & one for sea bass which is
based on the following steps.
Ɣ Assuming a class of models
Ɣ Process the sensed data
Ɣ Choose the best model
ProblemAnalysis
• Fish example
¾ Model: Sea bass is generally longer than salmon
(expert)
Sea bass have some typical length Lb
Salmon have some typical length Ls
Lb > Ls
¾ Classifier:
length ! Lcrit Ÿ Bass
length < Lcrit Ÿ Salmon
Lcrit ?

¾ According to the observation of length for the two type

of fish we found Lcrit may be equal to 11
¾ Expert: salmon < 11 and bass >= 11
ProblemAnalysis

¾ Learning:
Obtain training samples
Make measurements
Inspect results (Histogram)
hͲaxis:lengthoffish
vͲaxis:numberoffishes
withacertainlength

ƒ Onaverage,seabassis
somewhat longerthan
salmon
ƒToomuchoverlaps
histogramforlength ƒ poorseparationwiththe
lengthfeature
ProblemAnalysis
• Lengthbyitselfisnotreliable hͲaxis:lightnessoffish
• Tryanotherfeature:Lightness scales
vͲaxis:numberoffishes
withacertain
lightness
ƒ Onaverage,seabassis
much brighterthan
salmon
ƒ Lessoverlaps better
separationwiththe
lightnessfeature,butstill
histogramforlightness
abitunsatisfactory
• Cost of misclassification: depends on application
Is it better to misclassify salmon as bass or vice versa?
¾ Put salmon in a can of bass Ÿ loose profit
¾ Put bass in a can of salmon Ÿ loose customer
ProblemAnalysis
™ There is a cost associated with our decision.
Our task is to make a decision rule to minimize a given cost.
• We can use more than one feature.
• By adopting the lightness and add the width of the fish
The image of each fish has reduced to a point or a feature
vector in a 2-D feature space.
Fish xT = [x1, x2]

Lightness Width
Problem Analysis

• Objective: Partition the feature space into 2 regions:

Region1 Ÿ sea bass
Region2 Ÿ salmon
ƒ Linear model: simple; many false classifications

.
Problem Analysis

• Objective: Partition the feature space into 2 regions:

Region1 Ÿ sea bass
Region2 Ÿ salmon

ƒ Complex models:

In training set you can separate training data perfectly

with good generalization (what about the new fish
that has not yet seen and is considered as test sample)
Problem Analysis
™Thecentralaimofdesigningaclassifieristomakecorrect
decisionsforthenew(unseen/test) patterns (thisnew
patternnottakenfromthegroupoftrainingpatternswhose
labelsarealreadyknown).

e.g. it’suselesstoget100%accuracywhenanswering
homeworkquestionswhilegetlowaccuracywhen
answeringexamquestions.

Performanceon
thetrainingset
scatter plot for the feature vectors

Tradeoff
Simplicityof
theclassifier
scatterplotforthefeaturevectors
Problem Analysis
™ Try and try again to reach to "Optimal" model

Ideally, the best decision boundary should be the one which

provides an optimal performance (highly separation in training
and test set) such as in the following figure:

The decision boundary represent the optimal tradeoff between

performance on the training set and simplicity of classifier, thereby
giving the highest accuracy on new patterns
 Problem Analysis
Python
import cv2
import numpy as np

def extract_fish_length(image):
# Use OpenCV to detect fish head and tail points
# Calculate the Euclidean distance between these points
# Return the fish length

def extract_lightness(image):
# Convert the image to grayscale
# Compute the average pixel intensity (lightness)
# Normalize for illumination variations
# Return the lightness value
 Problem Analysis
def classify_fish(image):
fish_length = extract_fish_length(image)
lightness = extract_lightness(image)
# Set critical values (l* and x*)
l_critical = 10 # Example threshold for fish length
x_critical = 150 # Example threshold for lightness
if fish_length > l_critical and lightness > x_critical:
return "Sea Bass"
else:
return "Salmon"
# Load a test fish image
test_image = cv2.imread("path/to/your/test/image.jpg")

# Classify the fish

result = classify_fish(test_image)
print(f"Predicted species: {result}")
Face Recognition Technology
™ Facialrecognition(orfacerecognition)isatype
of biometric softwareapplicationthatcanidentifyaspecific
individualinadigitalimage byanalyzingandcomparingpatterns.

™Thefaceisanimportantpartofwhoyouareandhowpeople
identifyyou.

™ For facerecognition therearetwotypesofcomparisons.

¾Thefirstisverification andthesecondisidentification.
Ɣ verification iswherethesystemcomparesthegiven
individual(imageofobject)withtherequiredindividualto
givesayesornodecision
Ɣidentification iswherethesystemcomparesthegiven
individualtoalltheotherindividualsinthedatabaseand
givesarankedlistofmatches.
Face Recognition Technology
™Allidentificationorverificationtechnologiesoperateusingthe
followingfourstages:
stages meaning
Ɣ Capture:Aphysicalsamplesarecapturedbythesystemduring
Capture: Aphysicalsamplesarecaptured bythe
recordingandalsoforidentificationorVerification
systemusingcameraforidentificationor
process
Verificationprocess
Ɣ Extraction:Uniquedata(features)isextractedfromeach
Extraction : Uniquedata(features)isextractedfrom
sampleandatemplateiscreated.
eachsampleandatemplate iscreated.
Ɣ Comparison: Thetemplateisthencomparedwiththeprevious
Thetemplateisthencompared withthe
Comparison: recordedsample.
Ɣ Match/Nonmatch:Thesystemdecidesifthefeaturesextracted
previousstoredsamples.
fromthenew sampleismatchedwiththe
Match/Non Thesystemdecides ifthefeatures
recordedsamples
match: extractedfromthenew sampleis
matched withthestoredsamples
Face Recognition Technology
 Face Recognition Technology
™ HOWFACERECOGNITIONSYSTEMSWORK

ƒ Facialrecognitionsoftwarehastheabilitytofirstrecognizefaces
basedonfacialfeatures.

• Ifyoulookatthemirror,youcanseethatyourfacehascertain
clearablelandmarks.

• Thesearethepeaks thatisdetectedtoextractthedifferent
facialfeatures.
 Face Recognition Technology
™ HOWFACERECOGNITIONSYSTEMSWORK
ƒ Facialrecognitionsoftwarehastheabilitytofirstrecognizefaces
basedonfacialfeatures,whichisatechnologicalfeatinitself.

ƒ Ifyoulookatthemirror,youcanseethatyourfacehascertain
clearablelandmarks.

ƒThesearethepeaks thatisdetectedtoextractthedifferentfacial
features.
Face Recognition Technology
 Face Recognition Technology
™ HOWFACERECOGNITIONSYSTEMSWORK(Cont.)
ƒ Thereareabout80nodalpointsonahumanface.
ƒ Herearenodalpointsthataremeasuredbythesoftware.

• Distancebetweentheeyes
• Widthofthenose
• Depthoftheeyesocket
• Cheekbones
• Jawline
• Chin
ƒ Thesenodalpointsaremeasuredtocreateanumericalcode,a
stringofnumbersthatrepresentsafaceinthedatabase,thiscode
iscalledfaceprint.
ƒ Only14to22nodalpointsareneededforfacialsoftwareto
completetherecognitionprocess
 Face Recognition Technology
™ A face recognition system generally consists of four
modules as depicted in Figure

Ɣ Detection,
Ɣ Alignment,
Ɣ Feature extraction,,
Ɣ Matching,

Note: localization and normalization (face detection and alignment) are

processing steps before face recognition (facial feature extraction and
matching) is performed.
Face Recognition Technology

• Face detection
segments the face areas from the background.

• Face alignment
normalizing faces can be achieved with more accurate
localization.

Facial components, such as eyes, nose, and mouth and

facial outline, are located; based on the location of peak
points, the input face image is normalized with respect to
geometrical properties, such as size and pose.
Face Recognition Technology

• Feature extraction
provide effective information (template / feature vector )
that is useful for distinguishing between faces of different
persons.

• Face matching,
the extracted feature vector of the input face is matched
against those of enrolled faces in the database; and then
produces the identity of the face when a match is found or
indicates an unknown face (no match)
Face Recognition Technology

™Face recognition results depend highly on features that

are extracted to represent the face pattern and
classification methods used to distinguish between
faces whereas face localization and normalization are
the basis for extracting effective features.
 Numerical Example:
Face Recognition Technology

Let’s consider a simplified scenario:

• We have grayscale facial images of three individuals: Alice, Bob, and
Carol.
• Each image is represented as a matrix of pixel values.
• Our goal is to extract features that distinguish these individuals.
•Example Features:
• For each face, we extract the following features:
1.Eye Region Intensity: Average intensity of pixels in the eye region.
2.Nose Width: Width of the nose bridge.
3.Mouth Curvature: Curvature of the mouth contour.
•Feature Vectors:
• We create feature vectors for each face:
• Alice: 㼇㻝㻞㻜㻘㻌㻝㻜㻘㻌㻜㻚㻤㼉
• Bob: 㼇㻝㻝㻜㻘㻌㻝㻞㻘㻌㻜㻚㻢㼉
• Carol: 㼇㻝㻝㻡㻘㻌㻝㻝㻘㻌㻜㻚㻣㼉
•Comparison:
• To recognize an unknown face, we extract its features and compare them
with the stored feature vectors.
• The face with the closest feature vector is the likely match.
 Face Recognition Technology
We’ll start with a basic example that detects faces in images. Here are the steps:
Installations: First, make sure you have the necessary libraries installed.
pip install opencv-python For example, using
pip install face_recognition OpenCV in python, let’s
Python assume we have two
import cv2 different images of
import face_recognition Messi, We’ll compare
# Load the image (e.g., Messi1.webp) these images to detect
img = cv2.imread("Messi1.webp") matching or not
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) matching (verification).
# Encode the face in the image
img_encoding = face_recognition.face_encodings(rgb_img)[0]
# Load another image (e.g., images/Messi.webp)
img2 = cv2.imread("images/Messi.webp")
rgb_img2 = cv2.cvtColor(img2, cv2.COLOR_BGR2RGB)
# Encode the face in the second image
img_encoding2 = face_recognition.face_encodings(rgb_img2)[1]
# Compare the faces
result = face_recognition.compare_faces([img_encoding], img_encoding2)
print("Result:", result) # True if the faces match, False otherwise
 OpticalCharacterRecognitionTechnology
™ OpticalCharacterRecognition is
generallyabbreviatedastheOCR.

Ɣ Itistheelectronicconversionof
imagesofatexttothecharacters.

Ɣ Ithastheabilitytoscantextfrom
imagesofhandwrittenand
printedtexts.

Ɣ Ithelpstoeditandsearchthe
wordsinthescanneddocuments
likePDFfiles.
OpticalCharacterRecognitionTechnology
Let’s explore the hardware and software components that make
up (OCR) systems:
• Hardware Components:
–Optical Scanner: It reads physical, printed documents and converts
them into digital data.
• Software Components:
–Image Analysis: The OCR software begins by analyzing the scanned
image. It classifies light areas as the background and dark areas as the
text. This step is crucial for identifying characters.
– Character Recognition Process:
Once the image is pre-processed, the OCR software identifies individual
characters. It matches these characters against known patterns.
ImplementationofanOCRsystem

™AtypicalOCRsystem
consistsofseveral
Processingsteps.

Ɣ Thefirststepinthe
processistodigitizethe
analogdocumentinto
regionsusinganoptical
scanner.
Ɣ Whentheregionscontainingtextarelocated,each
symbolisextractedthroughasegmentationprocess.
ImplementationofanOCRsystem(Con.)
ƔTheextractedsymbolsmay
thenbepreprocessed
(eliminatingnoise),to
facilitatetheextractionof
featuresinthenextstep.

ƔTheidentityofeachsymbolisfoundbycomparingthe
extractedfeatureswithdescriptionsofthesymbolclasses
obtainedthroughapreviouslearningphase.

Ɣ Finallyappropriateinformationisusedtoreconstructthe
wordsandnumbersoftheoriginaltext.
ImplementationofanOCRsystem(Con.)
For example:
Weseparatethe
charactersinto
segments anduse
theirgeometric
propertiestoidentify
them(feature
extraction)andthen
usearecognition
modeltrainedona
priordatabase
(recognition)
ImplementationofanOCRsystem(Con.)
Here’s a simple example using Keras and TensorFlow:
Python

import cv2
import numpy as np
from keras.models import load_model

# Load a pre-trained model (you can train your own if needed)

model = load_model("path/to/your/model.h5")

# Read the image containing handwritten characters

img = cv2.imread("handwritten_sample.png", cv2.IMREAD_GRAYSCALE)

# Preprocess the image (resize, normalize, etc.)

img = cv2.resize(img, (28, 28))
img = img / 255.0
img = np.expand_dims(img, axis=0)
ImplementationofanOCRsystem(Con.)
Here’s a simple example using Keras and TensorFlow:

# Predict the character

predictions = model.predict(img)
predicted_class = np.argmax(predictions)

# Map the predicted class to the corresponding character

char_mapping = {0: 'A', 1: 'B', 2: 'C', ...}

# Define your own mapping

recognized_char = char_mapping.get(predicted_class, "Unknown")
print(f"Recognized character: {recognized_char}")